AI & Machine Learning

Internals & Engineering

• First-Principles Math

  ---

  Full derivations — gradient descent, Lagrangian duality, ELBO, Bellman equations — not just formulas, but why they work.

• Internals, Not APIs

  ---

  What happens inside model.fit(), PyTorch autograd graphs, Ring All-Reduce, LSTM gating dynamics, attention scaling.

• Historical Arcs

  ---

  Every concept placed in its intellectual lineage — from the Perceptron (1958) to the Agentic Era (2024+).

• Production Labs

  ---

  Runnable Python code: Transformers from scratch, drift detection, LoRA fine-tuning, adversarial attacks, RL agents.

---

Site Map

The material spans five parts across 18 chapters — designed for sequential study or targeted reference.

Part I — Foundations Part II — Deep Learning Part III — Systems & Production Part IV — Responsibility Part V — The Agentic Era

The mathematical and algorithmic bedrock of machine learning.

Chapter	Topics	Key Concepts
1.1 The ML Problem	Learning theory, optimization, probability	VC dimension, bias-variance, MLE/MAP, EM algorithm
1.2 Classical Algorithms	SVMs, trees, ensembles, boosting	Kernel trick, CART internals, gradient boosting from scratch
1.3 AI Foundations	Feature engineering, model selection	Cross-validation pitfalls, No Free Lunch theorem
1.4 Classical ML Engineering	Hyperparameter tuning, imbalanced data	Bayesian optimization, SMOTE, SHAP

From single neurons to Transformers to generative AI.

Chapter	Topics	Key Concepts
2.1 Deep Learning Theory	MLPs, CNNs, RNNs, Transformers	Backprop graphs, ResNet gradient proof, attention scaling
2.2 DL Architecture Internals	Initialization, normalization, transfer learning	He init, BatchNorm vs LayerNorm, discriminative LRs
2.3 Generative Models & RL	VAEs, GANs, diffusion, policy gradients	ELBO tightness, WGAN, score matching, RLHF
2.4 Modern Generative AI	LLM engineering, fine-tuning, serving	LoRA/QLoRA, quantization, KV-cache, vLLM
2.5 RL & Decision Systems	RL engineering, reward shaping	PPO, SAC, specification gaming, algorithm selection

Engineering ML systems that work at scale.

Chapter	Topics	Key Concepts
3.1 ML Systems	Data pipelines, distributed training	Ring All-Reduce, batch size scaling, drift detection
3.2 Infrastructure at Scale	GPU internals, networking, serving	Roofline model, NVLink, FlashAttention
3.3 MLOps	Experiment tracking, CI/CD, monitoring	MLflow, quality gates, testing pyramid
3.4 Case Studies	Recommendation, autonomous driving, tools	Two-tower architecture, long-tail safety

Building AI that is fair, private, and accountable.

Chapter	Topics	Key Concepts
4.1 Ethics & Governance	Bias, privacy, robustness	Impossibility theorem, DP-SGD, FGSM
4.2 Safety & Regulation	Red-teaming, alignment, EU AI Act	Constitutional AI, DPO, compliance engineering

Autonomous AI systems that plan, reason, and act.

Chapter	Topics	Key Concepts
5.1 Agentic AI	Reasoning, tool use, memory, multi-agent	CoT, ReAct, RAG internals, reliability engineering

---

The Meta-Narrative

Machine learning is not a bag of disconnected algorithms. It is a story of ideas building on ideas — each breakthrough unlocking the next:

graph LR
    A["🧠 Perceptron<br/>(1958)"] --> B["⛓ Backpropagation<br/>(1986)"]
    B --> C["👁 LeNet / CNNs<br/>(1998)"]
    C --> D["🔥 AlexNet<br/>(2012)"]
    D --> E["🏗 ResNet<br/>(2015)"]

    B --> F["📝 LSTM<br/>(1997)"]
    F --> G["🔍 Attention<br/>(2014)"]
    G --> H["⚡ Transformer<br/>(2017)"]
    H --> I["💬 BERT / GPT<br/>(2018-19)"]
    I --> J["🌐 GPT-4 / LLMs<br/>(2020+)"]
    J --> K["🤖 Agentic AI<br/>(2024+)"]

    B --> L["🎲 VAE<br/>(2014)"]
    B --> M["⚔ GAN<br/>(2014)"]
    L --> N["🌀 Diffusion<br/>(2020)"]
    M --> N
    N --> O["🎨 Stable Diffusion<br/>(2022)"]

The Perceptron begat backpropagation, which enabled CNNs for vision and LSTMs for sequences. The attention mechanism freed us from sequential processing, and the Transformer unified everything — spawning BERT, GPT, and the whole foundation model era. Meanwhile, variational inference and adversarial training opened the generative frontier, culminating in diffusion models that power today's image generation. Now, agentic systems are combining LLMs with tools, memory, and planning to create the next paradigm shift.

Understanding this lineage isn't optional — it's how you develop the intuition to know which tool to reach for and why.

---

Hall of Fame & Tech Giants

• Top 20 AI/ML Papers →

  ---

  The most influential papers in AI history — from Backpropagation (1986) to AlphaFold (2021) — with impact analysis and links to originals.

• Tech Giants & AI Frontier →

  ---

  How Google, OpenAI, Meta, Anthropic, NVIDIA, and Microsoft are shaping the AI landscape — strategies, architectures, and competitive dynamics.

---

Hands-On Labs

The src/labs/ directory contains runnable Python implementations of key concepts:

Lab	File	What You'll Build
Foundations	src/labs/foundations/pca_gmm_knn.py	PCA, GMM/EM, and KNN from scratch
Gradient Boosting	src/labs/foundations/gradient_boosting.py	Gradient boosting from scratch + XGBoost comparison
:material-transformer: Transformer	src/labs/deep_learning/transformer_from_scratch.py	Complete multi-head attention and encoder
Generative	src/labs/generative/vae_and_diffusion.py	VAE with reparameterization + DDPM training
RL	src/labs/rl/q_learning_dqn.py	Q-Learning + Deep Q-Network agent
Systems	src/labs/systems/drift_and_fairness.py	Drift detection (PSI/KS) + fairness auditing

---

Quick Start

# Clone and set up
git clone https://github.com/atulRanaa/machine-learning.git
cd machine-learning

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Serve locally
mkdocs serve
# → http://localhost:8000

---

Built with MkDocs Material · Licensed under MIT · Contributions welcome