Theme
Curated Resource List
This page is a curated resource map for learning about and engineering with large language models, organized by category: introductory courses, official documentation and blogs, essential papers and visual guides, open-source models and weights, open-source engineering frameworks, books, communities and leaderboards, and Chinese-language resources. Each resource includes a one-line description and a recommendation rating (the more stars ★, the higher the priority). A resource-combination decision table by learning objective appears at the end.
How to use this list
Start with "Introductory Courses" to build practical skills, then read "Essential Papers and Visual Guides" to deepen your understanding. For engineering practice, come back to "Open-source Engineering Frameworks" to pick your tools. All links point to real, official, or authoritative sources — just click to navigate.
I. Introductory Courses and Videos
Courses are the best value for your time: spending a few dozen hours following along beats reading hundreds of scattered blog posts. They're listed below in the order from "principles to engineering."
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| Andrej Karpathy's Neural Networks: Zero to Hero | From backpropagation to GPT: build and train a small Transformer by hand. Widely regarded as the best LLM intro course available | Course page | ★★★★★ |
| Stanford CS224n NLP with Deep Learning | A classic NLP course covering the theory behind Transformers and language models, with free lecture notes | Course page | ★★★★ |
| Stanford CS25 Transformers United | A seminar on Transformer architectures from theory to industry applications | Course page | ★★★ |
| Hugging Face Course | Official course paired with the Transformers library, includes a certificate, covering NLP/RL/diffusion/Agent tracks | Learning center | ★★★★★ |
| Hsu-Yang Hsiung's Generative AI series | A Chinese-language intro to LLMs covering both theory and inference plus multimodal topics, with many intuitive demonstrations | Course page | ★★★★ |
| deeplearning.ai short courses | Andrew Ng's 1–2 hour courses on prompting, RAG, fine-tuning, agents, and more — fast track to practical skills | Course list | ★★★★ |
Recommended path: the Karpathy trio
The nn-zero-to-hero video course + nanoGPT (~300 lines of trainable mini-GPT) + the "Let's build GPT" video series form a complete "build a large model from scratch" learning path. We recommend following along with our Build a Large Model from Scratch guide: after watching the videos and tweaking nanoGPT to run on the Shakespeare dataset, you'll already be ahead of most practitioners who only ever call APIs.
II. Official Documentation and Blogs
Official documentation is the most authoritative, most up-to-date first-party source: model specs, API parameters, and licensing terms are always defined by the official docs. If a third-party blog is wrong, nobody cares. If the official docs have an error, the whole internet corrects them.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| OpenAI Platform Docs | Official docs for APIs, model specs, prompting, function calling, and the source of truth for sampling parameters | Official docs | ★★★★★ |
| OpenAI Blog | Launch announcements and technical writeups for GPT-4o, o1, GPT-5, and more | Blog | ★★★★ |
| Anthropic Documentation | Guides for Claude models, prompt engineering, and agents, including Claude Code and other tooling | Official docs | ★★★★★ |
| Anthropic Research Blog | Deep-dive essays on RLHF, hallucination, interpretability (circuits), and more — a benchmark for industry analysis | Research blog | ★★★★ |
| Google DeepMind Blog | Frontier research on Gemini, Gemma, and multimodal AI | Blog | ★★★★ |
| Hugging Face Blog | High-quality technical deep dives on LLM training, quantization, evaluation, and long context — very reader-friendly for non-native English speakers | Blog | ★★★★★ |
| Hugging Face Docs | Authoritative manuals for Transformers, PEFT, TRL, Datasets, and other HF libraries | Official docs | ★★★★★ |
| LMSYS Team Blog | First-party open-source research on MT-Bench, Vicuna, long-context evaluation, and more | Team blog | ★★★ |
How to read docs effectively
Don't memorize docs cover to cover. Look things up when you need them: check the OpenAI docs for "how to set sampling parameters," the PEFT docs for "how to configure LoRA," or a model's official page for licensing terms. Docs are dictionaries, not textbooks.
III. Essential Papers and Visual Guides
Papers are the source of knowledge, but reading them directly can be challenging. Start with visual guides to build intuition, then return to the original papers for the highest efficiency. For a complete paper-reading roadmap, see our paper reading paths and paper map.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| Jay Alammar's The Illustrated Transformer | The most famous visual guide to the attention mechanism — a must-read for beginners with animated-style explanations | Article | ★★★★★ |
| Harvard NLP's The Annotated Transformer | A paragraph-by-paragraph reading of the original paper paired with runnable PyTorch code — the best entry point for understanding "how it's implemented" | Article | ★★★★★ |
| arXiv | The original source for all the papers: Attention Is All You Need, Scaling Laws, Chinchilla, LoRA, FlashAttention, and more | arXiv | ★★★★★ |
| Papers with Code | Search papers, code, and leaderboards in one place — great for checking "who has a working implementation" | Website | ★★★★ |
| Hugging Face Papers | Daily paper recommendations and community discussions — an effortless way to stay current | Papers hub | ★★★ |
Don't fall into the "save = learn" trap
The biggest risk of visual guides and paper lists is bookmarking without reading. Set a rule for yourself: for every paper you bookmark, you must read it within 24 hours and write three lines of notes (what problem does it solve / what's the core method in one sentence / where can I use it). See reading discipline and FAQ for more on reading discipline.
IV. Open-source Models and Weights
If you want to truly "own" a model — fine-tune it and deploy it privately — you need the open-weight ecosystem. At the center is Hugging Face Hub, the global hub for models, datasets, and evaluations.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| Hugging Face Hub | The world's largest model/dataset/applications repository, the main hub for weight downloads and community collaboration | Hub | ★★★★★ |
| Llama (Meta) | Official home and community license for Llama 3.1/4 | Website | ★★★★★ |
| Qwen (Alibaba) | Full series of Tongyi Qianwen weights and documentation — the most complete open-source family for the Chinese ecosystem | GitHub | ★★★★★ |
| DeepSeek | Weights, training details, and technical reports for DeepSeek-V3/R1 | GitHub | ★★★★★ |
| Mistral AI | European open-source models like Mistral 7B and Mixtral, under the permissive Apache 2.0 license | Website | ★★★★ |
| Gemma (Google) | Lightweight open model series, friendly for edge and consumer hardware | Page | ★★★★ |
| Ollama | A desktop tool for running open-source models locally with a single command — the easiest entry point for beginners | GitHub | ★★★★★ |
How to run an open-source model
Recommended learning path: Ollama (simplest) → llama.cpp (understand quantization) → vLLM (production deployment). Start with ollama run qwen3:8b to experience local inference, then dive into Deployment and Serving to learn memory estimation and quantization.
Open weights ≠ permissive license
"Open weights" doesn't mean free for any use: Llama has the Llama Community License, Gemma has the Gemma Terms, and Command R is CC-BY-NC (non-commercial). Always check the license before using a model commercially. For a quick reference on model selection and licensing, see Model Compendium.
V. Open-source Engineering Frameworks
Engineering tools fall into four layers by responsibility: training, inference, evaluation, and application orchestration. Figure out your task before picking your tools — don't just install everything because it's popular.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| vLLM | High-throughput LLM inference engine with PagedAttention + continuous batching — the production deployment standard | GitHub | ★★★★★ |
| llama.cpp | Run GGUF-quantized models on CPU and edge devices — the go-to choice for local deployment | GitHub | ★★★★★ |
| DeepSpeed | Microsoft's distributed training framework with ZeRO memory optimization — a staple for large model training | GitHub | ★★★★ |
| Megatron-LM | NVIDIA's large-scale training framework, reference implementation for tensor/pipe parallelism | GitHub | ★★★★ |
| Hugging Face Transformers | The universal interface for loading and using models — the starting point for every experiment | GitHub | ★★★★★ |
| PEFT | Official library for parameter-efficient fine-tuning (LoRA/QLoRA/Adapters) | GitHub | ★★★★★ |
| TRL | Official library for alignment training (SFT/DPO/PPO) | GitHub | ★★★★★ |
| LLaMA-Factory | One-click fine-tuning tool supporting LoRA/QLoRA/DPO/multimodal, with a zero-code WebUI | GitHub | ★★★★★ |
| Axolotl | YAML-config-based fine-tuning framework with strong reproducibility and a rich community of configs | GitHub | ★★★★ |
| lm-evaluation-harness | EleutherAI's general-purpose evaluation framework for running benchmarks like MMLU, GSM8K, and HumanEval | GitHub | ★★★★★ |
| OpenCompass | Evaluation framework from Shanghai AI Lab with rich Chinese benchmarks and a mature leaderboard service | GitHub | ★★★★ |
| LangChain | The most popular LLM application orchestration framework with a vast agent and tool ecosystem | GitHub | ★★★★ |
| LlamaIndex | A RAG framework centered on data indexing, ideal for document QA scenarios | GitHub | ★★★★ |
| Dify | Visual LLM application platform with built-in RAG/Agent/workflow — friendly to non-programmers | GitHub | ★★★★ |
| Task | Recommended Tool |
|---|---|
| Training / Fine-tuning / Alignment | Transformers + PEFT + TRL, or LLaMA-Factory / Axolotl (for a smoother experience) |
| Inference / Deployment | vLLM (GPU serving), llama.cpp (CPU/edge), Ollama (local experience) |
| Evaluation | lm-evaluation-harness (general), OpenCompass (rich Chinese coverage) |
| Application Orchestration / RAG / Agents | LangChain, LlamaIndex, Dify |
Don't let frameworks boss you around
Pick tools based on your problem: training with PEFT/TRL/LLaMA-Factory, inference with vLLM/llama.cpp, evaluation with lm-eval-harness, and application orchestration with LangChain/LlamaIndex. See Framework and Tool Comparison for a full selection guide and scenario-based decision tree.
VI. Books
Books are ideal for systematic, deep reading — the best antidote to fragmented blog information.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| Sebastian Raschka, Build a Large Language Model (From Scratch) | The best book for implementing and training a small GPT from scratch, with full code, published in 2024 | Publisher page | ★★★★★ |
| Raschka, Machine Learning with PyTorch and Scikit-Learn | A prerequisite read before diving into PyTorch | Publisher page | ★★★ |
| Karpathy's llm.c | A readable implementation that reproduces GPT-2 training in pure C/CUDA — not technically a book, but reads like one | GitHub | ★★★★ |
Free alternatives
Raschka's book code and video lectures are available for free on his GitHub repository. Karpathy's nanoGPT and llm.c are "book-level" hands-on resources. If you're on a budget, Karpathy's videos + nanoGPT already cover about 80% of what Raschka's book teaches.
VII. Communities and Leaderboards
Communities solve the "information gap": new model releases, new tricks, and lessons learned from pitfalls all surface here first. Leaderboards solve the "which model should I pick?" question, but they should only be used as a reference.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| LMArena (formerly LMSYS Chatbot Arena) | A crowdsourced blind-evaluation model ranking — the most authentic publicly available evaluation | Website | ★★★★★ |
| HF Open LLM Leaderboard | A standardized benchmark leaderboard for open models (now archived, but still a valuable historical reference) | Leaderboard | ★★★ |
| r/LocalLLaMA | An active community for locally deploying open-source models, with very fast updates on new models and quantization techniques | Subreddit | ★★★★ |
| Weights & Biases | An experiment tracking platform that is also the birthplace of many outstanding LLM experiment reports | Website | ★★★ |
| Simon Willison's Weblog | A high-quality independent blog on LLM engineering and productization with sharp, opinionated takes | Blog | ★★★★ |
| OpenRouter | A unified API gateway aggregating multiple models — great for comparing real-world model performance | Website | ★★★★ |
Leaderboards are just a reference
Chatbot Arena rankings shift frequently with every new model release, and the Open LLM Leaderboard is no longer updated. Leaderboard scores are affected by the choice of evaluation set, prompt style, and data contamination — the same model can rank very differently depending on the evaluation methodology. When consulting leaderboards, look at the sub-leaderboard that matches "your task type" rather than the overall score. See Evaluation and Benchmarks for more.
VIII. Chinese-Language Resources
The Chinese community has accumulated a wealth of high-quality tutorials and curated resources over the past two years, particularly friendly for Chinese readers.
| Resource | One-line description | Link | Rating |
|---|---|---|---|
| Datawhale | One of China's largest open-source learning communities, continuously updating LLM tutorials (covering principles and practice) | GitHub | ★★★★ |
| Awesome-LLM | A comprehensive LLM learning resource list (papers, courses, tools, leaderboards), actively maintained | GitHub | ★★★★ |
| llm-course (mlabonne) | A structured LLM learning path: from mathematical foundations to fine-tuning and deployment, with code | GitHub | ★★★★ |
| Hsu-Yang Hsiung's Generative AI | See the "Introductory Courses" section above — top-tier quality for Chinese-language explanations | Course page | ★★★★★ |
IX. Practical and Experiment Platforms
Running it yourself teaches more than reading about it ten times. These platforms let you experiment without needing your own hardware:
| Platform | One-line description | Link | Rating |
|---|---|---|---|
| Google Colab | Free GPU notebooks — the go-to choice for running nanoGPT and small model experiments | Website | ★★★★★ |
| Kaggle | Data science competitions + free GPU allowance (~30 hours/week) | Website | ★★★★ |
| Hugging Face Spaces | Deploy a model demo with one line of code — convenient for sharing your work | Website | ★★★★ |
X. Resource Combinations by Goal
| Goal | Must-do (in order) | Supplementary |
|---|---|---|
| Job interviews (3–6 months) | Karpathy videos → The Illustrated Transformer → HF Course → our Learning Paths | Work through our interview Q&A to practice output |
| Quick application build (1–2 weeks) | deeplearning.ai short courses → HF Docs → Ollama → LangChain/LlamaIndex docs | Use Dify to build your first no-code demo |
| Open-source fine-tuning + private deployment | Transformers → PEFT → LLaMA-Factory → vLLM → llama.cpp | See our fine-tuning guide and deployment |
| Research / paper reading | The Annotated Transformer → arXiv → HF Papers | See paper FAQ on reading discipline |
Further Reading
- Paper Reading Paths — Turn the paper resources above into an actionable reading plan
- Reading Discipline and FAQ — Is it OK to only read blogs? How to track arXiv
- Learning Paths — Resource ratio recommendations for three different learning routes
- Datasets and Benchmarks — Detailed profiles of pretraining corpora and evaluation benchmarks
- Model Compendium — Quick-reference specs for the open-source models mentioned above
References
- Karpathy course page: https://karpathy.ai/zero-to-hero.html
- Hugging Face learning center: https://huggingface.co/learn
- Stanford CS224n: https://web.stanford.edu/class/cs224n/
- Book publisher page (Build a Large Language Model): https://www.manning.com/books/build-a-large-language-model-from-scratch
- LMArena: https://lmarena.ai/