Reading Discipline and FAQ

In one sentence: what matters most in reading papers is not "how many you've read" but "whether you have a repeatable reading discipline" — knowing which questions to bring, how to read in three passes, what notes to take, and which papers to skip outright. This page lays down the discipline first, then answers all the high-frequency questions from "I can't understand it" to "I can't keep up".

How to Use This Page

Read the discipline in section 1 closely once; from then on, check yourself against the "red-flag table" every time you read a paper. The FAQ section doesn't need to be read top to bottom — come back and look things up when the matching problem hits. Treat this page as a method reference manual, not an article you must finish in one sitting.

1. The Discipline of Reading Papers

1. Read with questions: reading starts with asking, not turning pages

Before reading a paper, write down three questions (no more than three lines):

Question	What it does
What problem is this paper trying to solve?	If the problem is irrelevant to you, everything after is wasted reading — confirm the motivation first
What is its core mechanism?	Say "the method in one sentence" in your own words; if you can't, you haven't really read it
Why does it work, and where does it fall short?	Forces you to look for evidence (ablations, baselines, the limitations discussion) instead of accepting the conclusions

This mirrors the fixed structure of Classic Paper Deep-Dives on this site — each one unfolds as "background → core mechanism → experimental results → implications", so you can treat them directly as worked examples of reading with questions. If you're reading only because the hype pushed you there, with no questions of your own, first look at Learning Path Overview and place "what to read" into a bigger coordinate system before you start.

2. The three-pass method: skim → close read → reproduce

Pass	Goal	Actions	Output
First pass: skim	Judge whether it's worth reading	Title, abstract, figures, conclusions — about 30 minutes	A one-sentence summary + a first credibility read
Second pass: close read	Master the mechanism	Read the method and experiments section by section, against the formulas and figures — about 90 minutes	A one-page note + one hand-drawn mechanism diagram
Third pass: reproduce	Verify and internalize	Run the official code / a scaled-down experiment, or explain it fully to someone else	Reproducible experience + your own improvement hypotheses

The goal of the first pass is to filter out, not to finish — most papers stop here; the second pass is the real close reading; the third pass needn't happen for every paper, but at least one in ten should go all the way through. The three-pass method rewrites "read a lot" into "read right" — that is the reading philosophy this papers section keeps hammering on.

Read Concept Pages Alongside

The same topic is covered twice: on the concept pages and in the papers. The concept page gives you the coordinate system; the paper gives you the evidence. The recommended order is concept page first, paper second — for example, read Transformer and the Attention Mechanism before Attention Is All You Need, and read AI Agents before ReAct and the MCP docs. The reverse holds too: when a paper confuses you, going back to the concept page to look up terms is the most efficient way to fill in the prerequisites.

3. Taking notes: the goal is "still legible three months later"

Notes are not excerpts — they are external memory. The value of a fixed template is that it can be searched in bulk and recalled quickly. Recommended template (full Markdown version in the appendix at the end):

Element	Requirement
One sentence	What problem does this paper solve? (in your own words)
Core mechanism	The method in one sentence + one hand-drawn mechanism diagram
Key numbers	The 2–3 most important results + dataset/protocol
Ablation takeaways	Which component's removal hurts the score most? (that's the real innovation)
Limitations	Admitted by the authors + discovered by me
Relevance to me	What use is this for my project/learning? Can I reproduce it?
Credibility rating	High / Medium / Low + reason

Hand-drawing the mechanism diagram is strongly recommended — draw the method with boxes and arrows; if you can draw it, you truly understand it. Tools don't matter (Obsidian, Notion, plain Markdown all work); the key is to use the same template for every paper, so your notes can be searched in bulk.

4. The red-flag table: triaging papers fast

Red flag	What it looks like	What to do
Reports numbers without protocol	Claims "SOTA" but never states the dataset split, the evaluation script, or whether runs were repeated	Check the code/appendix directly; if you can't find it, mark it "not reproducible"
Weak-baseline comparisons	Compares only against "old methods without tuning", dodging the strongest contemporary methods	Look up the task's real SOTA on Papers with Code
No ablation studies	Five or six modules bolted into a method, with no account of "what happens if you remove one"	The core contribution can't be located; the novelty is in doubt
Single dataset	Results reported on one benchmark only	Very likely overfit to a single benchmark
No variance reported	Every experiment reports only the best single run	Gains from cherry-picked random seeds are not credible
No limitations discussed	No Limitations/Discussion section anywhere in the paper	Either the authors haven't thought it through, or they're hiding something
Relative numbers in the abstract	"30% improvement", "greatly surpasses" — but no absolute numbers in the body	Most likely a tiny base or an unfair comparison

Red Flags ≠ a Death Sentence

A red flag is an indicator that "you should raise your guard", not a verdict of "condemned outright". A paper with weaknesses can still have contributed an important idea — maybe the method doesn't hold in other settings, but the problem definition and the line of thinking are right. The right posture: record the red flags in your notes, and give the paper a "credibility rating" once you finish — don't think in black and white.

2. FAQ: Ten Questions on the Road to Reading Papers

Q1. What if I can't understand the math?

Consume the math in layers. Reading a paper does not require mastering all of its math — what you need is the basic skill of "seeing what the symbols are doing", then working level by level:

Level 1  Conclusions : read only the conclusions, skip the math (first pass, skim)
Level 2  Intuition   : what is the formula computing? what are the inputs and outputs? (second pass, close read)
Level 3  Derivation  : step-by-step derivation, checked against the code (third pass, reproduction)

Most hot papers (architecture, prompting, agents, evaluation) are not math-dense — they are concept-dense; when you can't understand something, it's usually not a math problem but missing prerequisite concepts. In that case, going back to this site's concept pages or the Glossary to fill in concepts is far more effective than grinding through derivations. Save the truly line-by-line derivations (reproduction, method improvements) for the third pass.

Q2. Too many papers, can't finish them all?

First, drop one fixation: "finishing the papers" was never the goal, and not finishing is not failure. Papers are an on-demand resource, not a checklist that must be emptied. Three practical moves:

• The 80/20 rule: 20% of papers deserve a close read; for the other 80%, "abstract + figures + one sentence" is enough.
• Read by task, not by hype: lock onto one topic (say, "KV cache compression for long contexts"), read only papers within that topic, and switch only after writing a short summary.
• Tolerate "read it and forget it": your brain is not a database; the trick is making your notes your external memory.

Keep at most two or three papers in close reading at the same time. For concrete routes, see Reading Paths.

Q3. How do I judge whether a paper matters, and who cites it?

Start with the "internal evidence" (is the problem real? does the method have continuity?), then look at the "external signals":

• Citation count: check citations and citation trends on Google Scholar or Semantic Scholar — being heavily cited by follow-up work, with citations still growing, is a strong signal of importance.
• Citation quality: being cited by SOTA methods is worth far more than being cited by marketing posts; look at what kind of work does the citing.
• Code and reproduction: someone actually reproducing it (reproduction status on Papers with Code, GitHub stars) is far more credible than the paper's own boasts.
• Author continuity: when the authors/institution keep producing in the same direction, the line has life in it.

In one line: the citation network + reproducible code are far more reliable than a paper's self-promotion.

Q4. Papers vs. blogs/WeChat accounts? How do I vet secondhand material?

Papers are first-hand facts; blogs and WeChat official accounts are second-hand interpretation. The two relate like a map and the terrain: read good secondhand material first to build the big picture, then go back to the original to verify details. Screening criteria for secondhand sources:

• Who wrote it: prefer interpretations from the original team, front-line researchers, and well-known engineers; skip marketing accounts and content farms outright.
• Does it cite sources: good interpretations link to the original text and its sections; bad ones hand you conclusions with no provenance.
• Does it separate "fact" from "opinion": facts (model parameters, benchmark numbers) must trace back to the original; opinions ("this is a paradigm revolution") should be taken at a discount.
• Timeliness: secondhand AI material goes stale in three months; watch the publication date.

We only include first-hand or high-quality second-hand sources — starting from the curated resource list is the least effortful way in.

Q5. Is reproducing the code mandatory?

It depends on the goal — three levels of intensity:

Goal	Reproduction level	Notes
Getting started	Not needed	Understanding + notes + explaining it to someone else is enough; spend your time on breadth
Engineering adoption	Small-scale reproduction	Until you run it once, you'll never know how many "hidden details" (learning-rate schedules, data augmentation, initialization) the paper left unwritten
Doing research	Deep reproduction + ablations	This is the ticket into research: if you can't reproduce it, every later improvement is a castle in the air

Key insight: the cost of reproducing something once is often the truest measure of the paper's value. If a method's "official code can't produce the paper's numbers", that fact matters more than the paper itself. Prefer work with official open-source implementations — Papers with Code aggregates papers and code by task, and the Hugging Face Hub hosts a huge supply of weights and model cards. Implementing a Transformer from scratch to understand the architecture is fine, but to reproduce SOTA, always find the official code first.

Q6. How do I track the latest developments without anxiety?

The root of the anxiety is "mistaking noise for signal". Three mental habits:

• Throttle: spend a fixed 20 minutes a day on aggregator sources (arXiv, Hugging Face Papers, your watchlists) and ignore them the rest of the time.
• Focus: track only the main lines related to your current topic; "major breakthroughs" in other directions stay out of view for now.
• Leave slack: give the extra time to close reading, reproduction, and note-taking — judgment comes from depth, not breadth.

For concrete tracking methods and tools, see Frontier Developments. A recommended rhythm: of 10 hours a week, spend 5 on close reading and reproduction, 3 on tracking, and 2 on writing notes — after three months, your judgment will visibly outpace your information intake.

Q7. What if reading English is hard?

Read the originals, and start as early as you can. Three reasons: ① translation loses information — the subtext behind "surprisingly" or "to the best of our knowledge" only survives in the original; ② frontier papers always appear in English first, and waiting for a Chinese write-up often means falling months behind — if one exists at all; ③ academic English is highly formulaic, and your reading speed leaps qualitatively after about 20 papers. For beginners, the advice is to read in both languages side by side: read a Chinese-language survey or blog first to build the big picture, then return to the English original to verify details, treating the original as the single source of truth. Skip unfamiliar words that don't block comprehension, and close-read the original sentences only in the "mechanism and results" parts.

Q8. How should I read the benchmarks in papers?

Remember one iron rule: a score is never an absolute measure of "how good the method is"; it is the joint product of method + data + compute + evaluation protocol. For any number you read, ask first:

• Data: which dataset, which split? Did the test set leak into the pretraining corpus (data contamination)? — Contamination inflates scores; "having memorized the answers" is not the same as being able to reason.
• Compute: how many GPUs, how long a training run, how large a batch? The same method with twice the compute often shows clear gains.
• Evaluation: how is the metric computed? Is the best single run reported, or the mean and standard deviation?
• Comparison: are the baselines implemented fairly? Were the baselines tuned? "Weak-baseline comparisons" are the most common trick in the trade.

Always convert "relative improvement" into absolute numbers before passing judgment: "30% better" could be 0.2% → 0.26%, or it could be 90% → 90.26%. Benchmarks are for orienting yourself across the landscape, not for worshipping leaderboards — for the full breakdown see LLM Evaluation and Benchmarks.

Q9. Survey first, or original papers first?

Survey first, then originals: the survey is the map, the originals are the terrain. The value of a survey: ① it gives you the complete coordinate system of a direction in a few pages — who did what first, who improved what, where things are stuck now; ② it builds your terminology; ③ its reference list is itself a high-quality reading list. The drawbacks are shallow depth and possible lag — many directions move so fast that a survey is outdated before it's even finished. The right posture: in a new field, read surveys from the last two years to build the map, then immediately move on to the two or three originals you care about. Just search "survey" plus your topic keywords on arXiv; the representative works under each main line in Frontier Developments can also serve as "condensed surveys".

Q10. How many papers until I've "gotten into" the field?

There is no fixed number, but there are testable milestones: you have entered a field when, without consulting any reference, you can sketch its structure — problem → key methods → mainstream lines → current bottlenecks — and name three or more landmark works and how they relate. Going by this site's arrangement, 15–30 high-quality papers in one direction are enough to support "following academic discussion". The bar for entry is "being able to converse", not "having read it all".

3. Appendix: Paper Reading Note Template (Markdown)

Copy and use:

---
title: "Paper Title"
Authors/Affiliation:
Year/Venue/arXiv:
Related concept page:   # e.g. /concepts/transformer
Reading status: [ ] Skimmed [ ] Close-read [ ] Reproduced
---

## One sentence
What problem does this paper solve? (in your own words)

## Background
Where were earlier approaches stuck? Why was this paper needed?

## Core mechanism
The method in one sentence + a hand-drawn diagram (arrows/boxes/whiteboard photo)

## Key numbers
| Dataset | Metric | This paper | Baseline | Protocol notes |
|---|---|---|---|---|
|        |      |      |      |          |

## Ablation takeaways
Which component's removal hurts the score most? → that's the real innovation

## Limitations
Admitted by the authors:
Discovered by me:

## Relevance to me
What use is this for my project/learning? Can I reproduce it?

## Credibility rating
High / Medium / Low + reason

## Related links
- Code repo:
- Deep-dive page:
- Concept page:

Why the Template Matters

The template isn't for looks — it forces you to answer "what does this paper have to do with me?" Reading without notes equals not reading; understanding you can't explain equals no understanding. Three months later, the notes that can restore your memory in three seconds are the good ones.

4. Summary: Reading Papers Is a Long Game

Viewed on the scale of time, the conclusion is clear:

Short term (weeks):    read a few, can't understand them, doubt yourself → a rite of passage for everyone
Mid term (months):     close-read 20 papers + take notes + explain them → can follow the discussion
Long term (1-2 years): 50-100 papers in one direction → can judge directions and pose questions

The compounding returns of paper reading show up in three places: speed — the first Transformer paper takes a week, the tenth attention variant takes an hour, and patterned reading keeps getting faster; judgment — the more you read, the faster you spot weak baselines and flashy rhetoric, and the red-flag table becomes intuition; expression — the ability to explain a paper clearly is the scarce communication skill you'll rely on in interviews, reviews, and collaboration.

Three Rules of Discipline — the Three Sentences Worth Taking Away

1. Take the mechanism, leave the numbers — every score is the joint product of model + data + compute + protocol.
2. Not understanding is the default state — fill in concepts, switch sources, mark for re-reading; don't grind forever against a single paper.
3. Notes are external memory; explaining it is the only test — reading without notes equals not reading, and understanding you can't explain equals no understanding.

Further Reading

• Start Here — the main entrance to the papers section, with three beginner routes
• Reading Paths — concrete article lists and reading order arranged by goal
• Paper Map — the coordinate system of papers, from the classics to the frontier
• Classic Paper Deep-Dives — paper-by-paper close-reading exemplars that double as note templates
• Frontier Developments — seven frontier lines and judgments for 2024–2025
• Learning Path Overview — placing paper reading into the overall long-term plan
• LLM Evaluation and Benchmarks — how to read benchmarks and guard against contamination
• Curated Resource List — trusted secondhand sources and tool entry points
• Glossary — come back here when a term blocks you while reading

References

• The arXiv preprint library — first-release home of the vast majority of ML papers; full texts free
• Google Scholar — citation counts, who cites whom, related articles
• Semantic Scholar — semantic search and recommendations for papers, with an API
• Papers with Code — aggregation of papers, code, and evaluation benchmarks
• Hugging Face — open weights, model cards, and the open-source ecosystem
• Hugging Face Papers — daily paper-trend aggregation and discussion
• LMArena — blind human-vote battle leaderboard
• arXiv Sanity Preserver — personalized paper filtering
• Jay Alammar. The Illustrated Transformer — the model example of illustrated paper reading; generalize from it
• distill.pub — classic papers explained through interactive visualizations
• Karpathy. A Recipe for Training Neural Networks — how paper experience lands in engineering practice; a famous essay on "hidden details"