Paper Deep-Dives: Start Here

In one sentence: this module takes you "back to the source" — from second-hand concept pages to the first-hand original papers. It does not aim to have you "read every paper ever written"; instead, it settles three questions first: why read papers at all, which ones to read, and what tools to read them with.

1. Module Guide: Why AI's Hottest Concepts Demand Reading the Papers

Start with a quick self-check: where does most of what you currently "know" about large language models, RAG, and diffusion models come from? Most likely tutorials, WeChat public accounts, videos, and second-hand survey posts. Those are all fine, but they share one identity — second-hand information.

• Second-hand information: a version that has been curated, paraphrased, and sometimes simplified for the sake of shareability. The upside is that it's easy to understand; the downside is that it loses detail, distorts facts, and goes stale.
• First-hand information: the original paper. The motivations, failures, ablation experiments, and limitations the authors wrote into the paper are things no retelling fully preserves.

For example: nearly every resource will tell you the Transformer has "multi-head attention", but few will tell you why the original authors split attention into multiple heads — because a single attention head may only learn one attention pattern, while multiple heads let the model learn several patterns in parallel (positional relationships, syntactic relationships, coreference, and so on). Design motivations like this can only be fully recovered by reading the original text. This site's concept pages (such as What Are AI's Hot Concepts and Large Language Models) draw the map of the knowledge; the papers module explains why each place on that map is named the way it is.

Three Irreplaceable Values of Reading Papers

1. Build bottom-up intuition for the "why". Concept pages tell you what softmax(QKᵀ/√dₖ)V is; papers tell you why it captures long-range dependencies and why you divide by √dₖ (to keep overly large dot products from pushing softmax into its saturation region). The gap between "can use it" and "can design it" is exactly this layer of intuition.
2. Resist second-hand distortion. Exaggerated retellings are the norm in AI: "this method's results are explosive" usually omits the datasets, compute, and ablation caveats behind it. Reading papers gives you the ability to verify claims yourself instead of passively accepting conclusions.
3. Hard currency for job hunting and R&D. In algorithm-engineer interviews, "walk me through one paper in depth" is nearly a guaranteed question (question bank in the interview question bank), and in day-to-day development, judging "can this new method work in my product?" also depends on reading papers.

The one-sentence verdict

Concept pages decide what you "know"; papers decide whether you "understand why". The former is a map, the latter is fieldwork — someone who only holds the map and never surveys the terrain will never be able to describe how the ground rises and falls.

2. The Module's Four-Step Method: Map → Deep-Dive → Frontier → FAQ

The module has five pages (including this one), and we recommend following the four-step order. Each step answers one question and hands off to the next:

Step	Page	Question it answers	What you'll get
Step 1: Map	Paper Map	"What AI papers are out there, who came first, which are the sources and which the milestones"	A macro coordinate system: see the paper landscape clearly by topic and timeline
Step 2: Deep-dive	Classic Paper Deep-Dives	"How to digest the most important papers paragraph by paragraph"	Paper-by-paper breakdowns: background, method, ablations, limitations, plus interview talking points
Step 3: Frontier	Frontier Progress	"Where has the field been heading in recent years"	A systematic survey of the major breakthroughs of the 2020s, so you can keep pace
Step 4: FAQ	Reading Discipline & FAQ	"What if I can't understand, can't remember, or can't stick with it"	Note-taking methods, the three-pass method, and practical discipline for judging papers

The logic of the four steps is see the whole picture first, then drill down: the Map answers "what to read", the Deep-Dive answers "how deeply", the Frontier answers "where things are heading", and the FAQ answers "how to actually stick with it". If you'd rather jump straight in, pick a route from Reading Paths, which arranges the four steps into a concrete action plan.

When unsure about terminology

Whenever you hit an unfamiliar term while reading, look it up in the glossary — it's the site-wide quick-reference index of terminology, covering high-frequency concepts such as attention, alignment, fine-tuning, and quantization, with English and Chinese terms side by side.

3. The Paper List at a Glance: Core Papers Covered in This Module

Below are the first-hand papers this module repeatedly deep-dives into and cites, grouped by theme. The Deep-dive location column tells you which page has a more detailed breakdown of each paper:

Theme	Paper (authors, year)	One-sentence contribution	Deep-dive location
Large models	Attention Is All You Need (Vaswani et al., 2017)	Introduced the Transformer and self-attention, displacing RNNs as the de facto standard for language models	Classic Paper Deep-Dives
Large models	BERT (Devlin et al., 2018)	Bidirectional Transformer + masked language modeling, establishing the "pretraining + fine-tuning" paradigm	Classic Paper Deep-Dives
Large models	GPT-3 (Brown et al., 2020)	175 billion parameters validating scaling laws, showcasing in-context learning	Classic Paper Deep-Dives
Large models	InstructGPT (Ouyang et al., 2022)	Used RLHF to turn GPT-3 into a model that follows instructions — the direct predecessor of ChatGPT	Frontier Progress
Large models	Llama (Touvron et al., 2023)	An open, reproducible large-model foundation that ignited the open-source ecosystem and local deployment	Frontier Progress
Generative AI	GAN (Goodfellow et al., 2014)	Adversarial training between a generator and a discriminator; the first time machines could generate realistic images	Classic Paper Deep-Dives
Generative AI	DDPM (Ho et al., 2020)	A noise-then-denoise diffusion model with stable training; became the SOTA foundation of image generation	Classic Paper Deep-Dives
Generative AI	LDM / Stable Diffusion (Rombach et al., 2022)	Moved diffusion into latent space, making high-quality image generation feasible on consumer GPUs	Frontier Progress
Generative AI	DiT / Sora report (Peebles et al., 2023; OpenAI, 2024)	Transformers meet diffusion; video generation heads toward a world simulator	Frontier Progress
Applied engineering	RAG (Lewis et al., 2020)	Retrieval + generation, letting LLMs cite external knowledge and easing hallucination	Classic Paper Deep-Dives
Applied engineering	ReAct (Yao et al., 2022)	An alternating loop of reasoning and acting — the classic paradigm for LLM agents	Frontier Progress
Applied engineering	LoRA (Hu et al., 2021)	Efficient fine-tuning via low-rank decomposition, making "everyone can fine-tune an LLM" possible	Classic Paper Deep-Dives
Alignment & safety	Constitutional AI (Bai et al., 2022)	A set of principles lets the model critique and correct itself, reducing manual labeling	Frontier Progress
Alignment & safety	DPO (Rafailov et al., 2023)	Reduced RLHF to a single classification loss; alignment no longer needs reinforcement learning	Frontier Progress

This table isn't meant to be finished in one sitting; it's a master ledger — whenever you read any one of these papers, you know which storyline it belongs to and which papers branch off it. For how the papers evolved and in what order to read them, see Reading Paths.

4. A Word to Different Readers

The same module should be walked differently depending on your goal:

Job Seekers: Use Papers to Back Up Your Interview Talking Points

• Read first: Transformer, GPT-3, RAG, LoRA, InstructGPT — these five cover the five high-frequency interview topics of "mechanisms, scale, engineering, fine-tuning, and alignment".
• Goal: for each paper, be able to lay out "what problem it solves → the core method in one sentence → what the ablations proved → where the limitations are", and prepare an answer for "what would happen if I applied it to my own scenario".
• For how interviews ask these questions and how to answer them, pair this with the interview question bank and the JD knowledge breakdown — used together, the effect doubles.

Engineers: Read to Implement

• Read first: RAG, ReAct, LoRA, InstructGPT, plus the frontier page's coverage of inference optimization and evaluation.
• Goal: judge "can this method move into my system, and at what cost". Focus on each paper's ablation experiments and limitations (Discussion), and don't let SOTA numbers lead you astray.
• Start building the moment you finish reading: begin with Building a RAG Application and Fine-Tuning Your Own LLM — a mechanism from a paper only counts as truly understood once you've verified it in code.

Researchers: Build a Global Coordinate System

• Read first: the Paper Map to build a 60-year coordinate system → Frontier Progress to locate today's hot topics → then back to Classic Paper Deep-Dives for the deep reads.
• Goal: find the "blank spots" in your own research. The test is simple — can you state "what this paper's next step is, and the problems it left unsolved".
• For tool and dataset archives, see the curated resource list and the datasets & tools archive.

The pitfalls each type of reader falls into

Job seekers most often "memorize papers" — they can recite the contributions but can't explain the mechanisms, and one ablation question from the interviewer gives them away. Engineers most often "read only the conclusions" — they skip the long argument and jump straight to the results, then get burned when they transplant the method into their own setting. Researchers most often "chase only the newest" — newest ≠ important; hot papers are plentiful, but very few are worth a deep read.

5. Recommended Reading Tools

Half of your paper-reading efficiency depends on your tools. These four are the most mainstream right now, and all of them are free:

Tool	What it is	Best for	One-line usage
arXiv	The first publication venue for paper preprints	Finding originals, tracking updates	Subscribe to the cs.CL (NLP) / cs.CV (vision) / cs.AI categories; read the abstract before deciding
Semantic Scholar	An AI-powered academic search engine	Searching, citation graphs, TL;DRs	Search keywords, then follow "papers citing this one" along the timeline
Hugging Face Papers	Daily paper picks + a trending leaderboard	Spotting hot topics, following community discussion	Scan the trending list for 5 minutes a day and bookmark what interests you
Papers with Code	Aggregated papers + code + SOTA leaderboards	Verifying results, finding reproduction code	Look up a task's "current best" and find official or high-star implementations

Combined workflow (recommended for everyone): Hugging Face Papers handles "discovery", arXiv handles "reading the original", Semantic Scholar handles "tracing citation chains", and Papers with Code handles "verification and reproduction".

A tool tip for beginners

Start with just one tool: arXiv. Subscribe to the cs.CL category and spend 5 minutes a day reading only the titles. Within two weeks you'll develop an intuition for "which titles are worth clicking" — and that intuition is worth more than any tool.

6. The Most Important Advice: Take the Mechanism, Not the Numbers

The most valuable thing in a paper isn't SOTA

What sticks with you most easily when reading a paper is the number: "GPT-3, 175 billion parameters", "Stable Diffusion generating images on a consumer GPU". But remember: every absolute number depends on the data, compute, and evaluation protocols of its moment in time. GPT-3's 2020 numbers no longer apply in a 2025 evaluation regime.

What's always worth taking away comes down to three things:

1. Mechanism — why does the method work? What structural problem does it solve? (e.g., LoRA uses low-rank decomposition to solve "full fine-tuning is too expensive")
2. Ablation evidence — what did the authors prove by removing which component? This is the only way to tell "genuine innovation" apart from "engineering accretion".
3. Applicability boundary — under what conditions does the method hold, and under what conditions does it fail? (e.g., RAG depends on retrieval corpus quality; with a bad corpus, augmentation becomes a drag)

Leaderboard numbers go stale; mechanisms and boundaries don't.

Further Reading

• Reading Paths — your next step from this page: pick your first paper route based on your goal
• Paper Map — Step 1 of the four-step method: lay the AI paper landscape out as one macro map
• Classic Paper Deep-Dives — Step 2: paper-by-paper breakdowns of the classics that changed the field
• Frontier Progress — Step 3: the latest on large models, diffusion models, and agents
• Reading Discipline & FAQ — Step 4: note-taking, the three-pass method, and common confusions about reading papers
• What Are AI's Hot Concepts — the site-wide overview; look at this big picture before diving into papers
• Glossary — a bilingual quick reference to consult whenever you hit an unfamiliar term

References

All of the following are real, publicly available resources for deeper self-study:

• arXiv preprint library — the first publication venue for the vast majority of AI papers; free full text
• Semantic Scholar — AI-powered academic search with citation graphs and TL;DR summaries
• Hugging Face Papers — daily paper picks and the community trending list
• Papers with Code — aggregated papers, code, and SOTA leaderboards
• Vaswani et al. Attention Is All You Need (NeurIPS 2017) — the original Transformer paper
• Ouyang et al. Training language models to follow instructions with human feedback (2022) — InstructGPT, the landmark work on RLHF alignment
• Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) — the original RAG paper
• Hu et al. LoRA: Low-Rank Adaptation of Large Language Models (ICLR 2022) — the original LoRA paper
• Ho et al. Denoising Diffusion Probabilistic Models (NeurIPS 2020) — DDPM, the landmark diffusion model paper
• Jay Alammar. The Illustrated Transformer (2018) — the classic example of a visually explained Transformer deep-dive
• The Annotated Transformer (Harvard NLP) — a line-by-line annotated, runnable-code close reading of the Transformer