MACHINE LEARNING HANDBOOK
A systematic body of knowledge from data to decision — modeling paradigms · feature engineering · optimization & evaluation · deep learning · generative models · MLOps · career paths
50+ articles aren't a library to read cover-to-cover — they're a road map you can assemble on demand
Interviewing in 1–3 months? Jump straight into JD benchmarking, resume rewrite, and the interview question bank — every page maps to what top companies ask.
View route →Plenty of time and want a solid foundation? Progress week by week from the guide into hands-on practice, with clear checkpoints each week.
View route →Working practitioner who doesn't read cover-to-cover? Navigate by "problem → page", flip to what you need, and use this like a reference manual.
View route →If you only read ten articles, read these
The complete skeleton of a production-grade machine learning system: nine stages from problem definition, data pipelines, feature engineering, model training, evaluation and iteration to deployment and monitoring. This page is the site's content map, with each stage linking to in-depth articles.
Getting StartedThe 50+ pages on this site aren't a library you're expected to read cover to cover, but a roadmap you can assemble on demand: a career sprint track for those with interviews within three months, an eight-week systematic track for those who want to build a solid foundation, and a "problem → page" desk reference for working practitioners.
Getting StartedMachine learning is the discipline of automatically discovering patterns from data and using them for prediction or decision-making. This article provides precise definitions, formal frameworks, boundary analysis with AI/statistics/traditional programming/optimization, and the core principles behind why machines can learn.
Getting StartedDeep learning = deep neural networks + representation learning. This article covers everything from neurons and activation functions to forward/backward propagation, CNN/RNN/Transformer architectures, loss and training, providing a complete path to understanding deep learning from scratch, and its selection boundaries with classic ML.
Core ConceptsFeature engineering is the process of transforming raw data into numerical features a model can digest — missing value handling, categorical encoding, numerical scaling, feature transforms, combination and selection. This article provides a complete checklist and the iterative principle of "start with a simple baseline, then add features based on evidence."
Core ConceptsThe model finishing training is only the beginning — deployment, monitoring, and retraining are where production ML is truly won or lost. This article covers the full MLOps landscape: experiment management, model registry, deployment options, inference services, drift monitoring, and the retraining loop, plus "why so many models die after going live."
Core ConceptsModel evaluation is the foundation of machine learning — bias-variance tradeoff, train/validation/test splits, cross-validation, confusion matrix, precision/recall/F1/AUC, handling class imbalance. This article provides a complete framework from metric selection to validation methodology.
Core ConceptsTraining machine learning models means solving an optimization problem. This article systematically breaks down the gradient descent family — batch/stochastic/minibatch, momentum, Adam/AdamW, learning rate scheduling — plus troubleshooting vanishing/exploding gradients, local optima, and non-converging loss.
Core ConceptsOverfitting is machine learning's number one pitfall: the model mistakes noise in the training set for real patterns. This article explains the causes and signals of overfitting, plus six regularization weapons — L1/L2, Dropout, early stopping, data augmentation, pruning, ensembles — their principles, use cases, and tradeoffs.
Core ConceptsSupervised learning is the modeling paradigm driven by "labeled data." This article systematically covers task types (classification/regression/ranking), loss functions and training objectives, algorithm selection logic across classic algorithm families (linear, tree, kernel, neural networks), and a decision framework for when to use which model.
Core ConceptsFrom AlexNet to ViT — how convolutional neural networks dominated computer vision for a decade. This article dissects CNN architecture evolution (AlexNet/ResNet/EfficientNet), image task families (classification/detection/segmentation), training tricks (transfer learning/data augmentation), and the current state of visual foundation models.
Case StudiesGraph neural networks (GNNs) let models "read graphs and speak" through message passing — performing node classification, link prediction, and graph classification on social networks, molecules, and knowledge graphs. This article dissects message passing mechanisms, the three representative models GCN/GraphSAGE/GAT, and deeper issues like over-smoothing and expressive power limits.
Case StudiesLarge language models are the culmination of deep learning and scaling laws. This article dissects the full LLM landscape: pre-training and scaling laws, instruction fine-tuning and RLHF, in-context learning and chain-of-thought, RAG and fine-tuning, inference and deployment, and frontier trends for 2025–2026.
Case StudiesThe 2017 paper "Attention Is All You Need" dominated natural language processing with an attention mechanism. This article dissects the Transformer architecture (self-attention/multi-head/position encoding), the encoder-decoder paradigm, the divergence between BERT and GPT, and modern solutions for the NLP task family.
Case StudiesBrowse by section, or search directly
The complete skeleton of a production-grade machine learning system: nine stages from problem definition, data pipelines, feature engineering, model training, evaluation and iteration to deployment and monitoring. This page is the site's content map, with each stage linking to in-depth articles.
From the 1956 Dartmouth Conference to the large-model era of the 2020s, machine learning has experienced three winters and two paradigm shifts. This article uses a timeline to clarify the rise-and-fall logic of AI/ML/DL over seventy years, and the technical reasons behind each "cooling" and "heating."
The 50+ pages on this site aren't a library you're expected to read cover to cover, but a roadmap you can assemble on demand: a career sprint track for those with interviews within three months, an eight-week systematic track for those who want to build a solid foundation, and a "problem → page" desk reference for working practitioners.
Machine learning, artificial intelligence, deep learning, data science, data analytics, statistical modeling—six terms that get mixed up daily. Where exactly are the boundaries? This article provides a containment diagram, decision criteria, and guidance.
Machine learning is the discipline of automatically discovering patterns from data and using them for prediction or decision-making. This article provides precise definitions, formal frameworks, boundary analysis with AI/statistics/traditional programming/optimization, and the core principles behind why machines can learn.
80% of an ML project's time is spent on data. This article systematically covers data collection, cleaning, labeling, versioning, data leakage, plus feature stores and data pipelines — the full engineering scope from "getting data" to "ready to train."
Deep learning = deep neural networks + representation learning. This article covers everything from neurons and activation functions to forward/backward propagation, CNN/RNN/Transformer architectures, loss and training, providing a complete path to understanding deep learning from scratch, and its selection boundaries with classic ML.
Feature engineering is the process of transforming raw data into numerical features a model can digest — missing value handling, categorical encoding, numerical scaling, feature transforms, combination and selection. This article provides a complete checklist and the iterative principle of "start with a simple baseline, then add features based on evidence."
From GANs to diffusion models to large language models — generative models are a class of methods that "learn to create." This article clarifies the essence of generative modeling (learning data distributions), the principles and comparisons of three major technical routes (VAE/GAN/diffusion), and the capabilities and risks of generative AI.
A model must not only be right, it must be trustworthy. This article clarifies three levels of interpretability (global/local/post-hoc), mainstream methods (feature importance, SHAP, LIME, attention), fairness assessment and bias sources, and trustworthy ML engineering practices.
The model finishing training is only the beginning — deployment, monitoring, and retraining are where production ML is truly won or lost. This article covers the full MLOps landscape: experiment management, model registry, deployment options, inference services, drift monitoring, and the retraining loop, plus "why so many models die after going live."
Model evaluation is the foundation of machine learning — bias-variance tradeoff, train/validation/test splits, cross-validation, confusion matrix, precision/recall/F1/AUC, handling class imbalance. This article provides a complete framework from metric selection to validation methodology.
Training machine learning models means solving an optimization problem. This article systematically breaks down the gradient descent family — batch/stochastic/minibatch, momentum, Adam/AdamW, learning rate scheduling — plus troubleshooting vanishing/exploding gradients, local optima, and non-converging loss.
Overfitting is machine learning's number one pitfall: the model mistakes noise in the training set for real patterns. This article explains the causes and signals of overfitting, plus six regularization weapons — L1/L2, Dropout, early stopping, data augmentation, pruning, ensembles — their principles, use cases, and tradeoffs.
Reinforcement learning is the third modeling paradigm: no labels, no answer key, only reward signals from the environment. This article clarifies the MDP framework, exploration vs. exploitation, value learning vs. policy learning, deep reinforcement learning (DQN/PPO), and what problems RL is suited for.
Supervised learning is the modeling paradigm driven by "labeled data." This article systematically covers task types (classification/regression/ranking), loss functions and training objectives, algorithm selection logic across classic algorithm families (linear, tree, kernel, neural networks), and a decision framework for when to use which model.
What to do without labels? Unsupervised learning discovers structure from data itself — clustering, dimensionality reduction, anomaly detection, and association rules. This article clarifies the principles of four major tasks, representative algorithms, evaluation challenges, and how "unsupervised" approaches land in real business.
With no labels, what can machine learning do? This article deeply dissects K-Means (EM perspective, choosing K), DBSCAN and hierarchical clustering, the internal/external evaluation dilemma, the principles and pitfalls of PCA and t-SNE, and provides sklearn practice cases and real business deployment methods.
From AlexNet to ViT — how convolutional neural networks dominated computer vision for a decade. This article dissects CNN architecture evolution (AlexNet/ResNet/EfficientNet), image task families (classification/detection/segmentation), training tricks (transfer learning/data augmentation), and the current state of visual foundation models.
From DDPM to Stable Diffusion: this article dissects the mathematical principles of diffusion model denoising/counter-denoising, latent space architecture, and sampling acceleration methods, clarifies conditional control techniques like ControlNet and LoRA, and provides local practice code and risk assessment.
Graph neural networks (GNNs) let models "read graphs and speak" through message passing — performing node classification, link prediction, and graph classification on social networks, molecules, and knowledge graphs. This article dissects message passing mechanisms, the three representative models GCN/GraphSAGE/GAT, and deeper issues like over-smoothing and expressive power limits.
Large language models are the culmination of deep learning and scaling laws. This article dissects the full LLM landscape: pre-training and scaling laws, instruction fine-tuning and RLHF, in-context learning and chain-of-thought, RAG and fine-tuning, inference and deployment, and frontier trends for 2025–2026.
Linear regression and logistic regression are the most basic and most underestimated models in machine learning: the most interpretable "white box," and the first baseline every project must establish. This article thoroughly explains the principles from two perspectives — least squares and maximum likelihood — covering closed-form solutions, gradient descent, R², regularization, the GLM family, and sklearn practice.
Starting from user-item interactions, this article systematically covers collaborative filtering and matrix factorization (SVD/SVD++) to the industry's "recall → ranking → re-ranking" three-stage architecture, and provides evaluation metrics, cold start, exploration-exploitation, and complete surprise code.
How did AlphaGo learn from scratch to crush human players? How do robots learn real-world actions in simulation? This article dissects RL deployment cases across five domains — games, robotics, recommendations, scheduling, and LLM alignment — and analyzes the core mechanisms and engineering challenges of MCTS, PPO, SAC, and RLHF.
The 2017 paper "Attention Is All You Need" dominated natural language processing with an attention mechanism. This article dissects the Transformer architecture (self-attention/multi-head/position encoding), the encoder-decoder paradigm, the divergence between BERT and GPT, and modern solutions for the NLP task family.
How do decision trees split? Why do random forests reduce variance? Why does GBDT reduce bias? This article systematically covers the complete progression from a single decision tree to XGBoost and LightGBM, and provides practical code with feature importance and SHAP interpretation.
From the Perceptron to GPT-3, deep reads of nine classic papers that changed machine learning: background, mechanisms, experiments, implications, plus the three-pass paper-reading method and the common patterns behind nine papers, helping you build a paper-reading coordinate system.
ML frontiers change weekly: this article gives methods for tracking research via arXiv, top conferences, and scholar updates, breaks down six key research lines (model efficiency, reasoning models, multimodal, Agents, alignment/safety, data engineering), and honestly assesses open problems to help you choose a specialization direction.
Organizing key papers from 60 years of machine learning development into a searchable map grouped by theme, covering seven major themes: ML foundations, deep learning foundations, Transformer, generative models, reinforcement learning, tree models, and representation learning. Includes a paradigm shift timeline, paper index by task, and deep/skim reading routes.
A methodology manual for reading papers: how to read numbers and figures in papers, how to use red-flag signals to judge paper quality, plus 12 FAQ answers—can't understand, can't remember, can't find related papers, can't finish them all: all have concrete countermeasures.
Papers are vast and challenging—blindly opening them will only discourage you. This article provides three paths—quick start, systematic study, and frontier tracking—along with a three-pass reading method to help you choose a route based on your goals and read papers at your own pace.
Why should ML learners read papers? This article clarifies three values of paper reading, gives three entry routes based on goals, introduces the site's papers section map, and ends with one crucial piece of advice.
Model evaluation isn't about computing a few metrics; it's a systematic method designed around business goals. Starting from evaluation system design, this article gives complete code and engineering practices for classification, regression, cross-validation, error analysis, class imbalance, and LLM evaluation.
Using the classic Titanic survival prediction as a small project, run through the complete workflow from data loading, EDA, feature engineering, baseline model to iterative improvement and error analysis: each step gives runnable code, explanation, repo directory structure, common pitfalls, and advanced directions.
Data leakage, unknowingly overfitting, train-serve skew, hallucinations treated as facts... This checklist covers 25 high-frequency pitfalls across six levels: data, features, models, evaluation, engineering, and the LLM era. Each one provides symptoms, causes, consequences, and correct practices, plus a checkbox self-review checklist at the end.
scikit-learn, PyTorch, vLLM, or cloud API? Starting from data shape, team capability, and deployment environment, this article gives framework selection comparison tables for three scenarios — tabular data, deep learning, and large models — plus a checkbox decision checklist.
ML project failures rarely stem from models not being advanced enough; more often they die from design errors. This article distills ten design principles: run baselines first, simplicity first, evaluation first, data > model, reproducibility, prevent leakage, monitor at deployment, error-analysis-driven iteration, with exceptions and team norms discussed.
Portfolio projects are the most powerful talking points for job interviews. This article gives four criteria for good projects, concretely tiered topic selections by difficulty, a complete deliverables checklist, a README template, and methods for discussing projects using the STAR framework while avoiding anti-patterns.
Using the same Iris dataset, progressively advance through three versions: first a 50-line sklearn logistic regression that runs the full loop, then add feature engineering, cross-validation, and learning curves, finally switch to a PyTorch neural network. Run all three yourself, and you'll master the complete path from ML beginner to engineering practice.
Hyperparameter tuning is an unavoidable step in deploying models, and also where compute and time are most easily wasted. This article explains the correct order for tuning, key hyperparameters for each model family, a comparison of grid/random/Bayesian search methods with code walkthroughs, and a complete methodology for deep learning tuning and experiment discipline.
Translate "proficient in machine learning" and "familiar with deep learning" from job descriptions into itemized knowledge checklists you can review one by one. Five knowledge blocks + a five-level self-assessment table, score yourself from "never heard of it" to "can explain in depth," and auto-generate your own study roadmap.
A full-spectrum walkthrough and high-frequency question analysis for algorithm interviews: the logic behind nine interview rounds, 15 machine learning fundamentals questions, 10 deep learning questions, 8 LLM questions, and frameworks for coding & project deep-dives, plus "I don't know" scripts and 1-week / 1-month sprint checklists.
Curated common skill requirements for ML roles at top companies worldwide, breaking down hard requirements and bonus items for positions like Algorithm Engineer and LLM Engineer, extracting high-frequency skill word counts, and mapping them to learning pages on this site—helping you reverse-engineer your study plan from JDs.
Full panorama of the ML job search landscape: responsibilities, core skills, and salary comparisons for nine role types including Algorithm Engineer, ML Engineer, Data Scientist, and LLM Engineer, paired with skill radar charts and a three-dimensional decision checklist (background / interest / trend), plus a usage map for the four career module pages.
A resume is not a chronological list of experiences—it's a chain of evidence proving you possess the capabilities the role requires. This article presents a JD skill-benchmarking method, an ML-adapted STAR framework for writing project descriptions, a layered tech stack strategy, ten fatal pitfalls, and a resume template skeleton.
From Andrew Ng's courses to CS231n, from ISLR to the "Deep Learning" book, from Kaggle to MLflow — a learning-phase-organized map of machine learning entry and advancement resources, with every official link individually verified.
Machine learning datasets and tools reference: a categorized quick-reference table of datasets by task (Iris, MNIST, ImageNet, SQuAD, MovieLens, etc.), main acquisition channels, annotation and synthetic data tools, common libraries and data engineering tools, and practical advice on selecting datasets for exercises.
Core machine learning glossary: accurate definitions and distinctions for 60+ terms across foundational concepts, data, models & algorithms, deep learning, generative models & LLMs, evaluation & metrics, and production & trusted ML.
The math behind machine learning isn't a barrier but a tool: five pillars — linear algebra, calculus, probability and statistics, information theory, and optimization — each concept mapped to a concrete ML use case, with actionable learning strategies and real self-study resources from 3Blue1Brown, Strang, Khan Academy, and more.
