Build a Reasoning Model (From Scratch) by Sebastian Raschka
📚 Build a Reasoning Model (From Scratch) by Sebastian Raschka
| Author | Sebastian Raschka |
| Year | 2026 |
| Pages | 440 |
| Read it if | you know Python, want the math behind reasoning models, and prefer building over API calls |
This book is outstanding. Raschka walks you through reasoning models the way a good lab course should: baby steps, clear diagrams, working code, and the inner math explained when you actually need it. You can read the whole thing free on Manning’s site. The exercises live in the official GitHub repo.
The arc is deliberate. Start with a conventional pretrained LLM. Learn text generation, KV caching, and how to evaluate answers with math verifiers. Only then add reasoning tricks at inference time. Later chapters change the weights through reinforcement learning and distillation. Each chapter exists because the build needs it.
Inference-time methods come before training, which is the right order for understanding. Chain-of-thought prompting, sampling strategies, self-consistency, response scoring, and self-refinement are implemented from scratch, not hidden inside a library call. You see latency and cost trade-offs because you wrote the loops.
The math is taught in context, not dumped in a appendix and forgotten. Softmax, temperature, top-p sampling, log-probabilities, KL regularization: each one ties to a decision you are about to make in code. Diagrams and workflow summaries keep you oriented when the material gets dense.
Evaluation is treated as engineering, not an afterthought. Verifier-based scoring for math tasks, Best-of-N selection, failure modes where refinement makes answers worse. That honesty matters. Most hype pieces skip the part where your clever pipeline backfires.
Training chapters cover RL with verifiable rewards, GRPO improvements, format rewards, and distilling from a stronger reasoning model into a smaller one. You are not just reading papers. You are following a path from “prompt harder” to “actually update weights.”
The examples are top notch. Code listings match the narrative, exercises reinforce each section, and seven appendixes hold solutions, larger models, batching notes, and chat-interface extras so the main eight chapters stay focused. It feels like one continuous project.
Raschka’s earlier book built an LLM from scratch. This one assumes that baseline and asks the question everyone is asking now: how do you make it reason? If you read one technical book to understand o1-style systems without mysticism, this is it.
Prerequisites are real: Python and basic machine learning. You do not need a PhD, but you do need to run code and sit with equations when they appear. The payoff is literacy. You stop treating “reasoning model” as marketing and start seeing mechanisms.
Verdict: An amazing piece of literature in the nerdiest sense: clear, detailed, and generous. Read it online for free, clone the repo, work the exercises. Buy the print edition if you want colored diagrams on paper. Skip it if you want career pep talks about AI; read this if you want to know what is actually happening inside the box.
Related TMFNK Content
- The Proletariat of Judgment: Cognitive Stratification in the AI Era Why judgment still matters once models can reason, the human layer this book helps you see through technical clarity.
- Is Grep All You Need? How Agent Harnesses Reshape Agentic Search A different slice of the LLM stack: tool use and search harnesses versus training reasoning into the model itself.
- Open to Work: How to Get Ahead in the Age of AI by Ryan Roslansky and Aneesh Raman Career advice for the AI era; this Raschka book is the technical literacy that soft-skills guides usually skip.
- The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI by Dr. Fei-Fei Li The historical and personal story of modern AI; Raschka picks up where the frontier moved to reasoning models.
Crepi il lupo! 🐺