Large language models like DeepSeek can seem like magic. But how do they really work? Understanding how they are built from scratch gives you the power to create your own model. When you know how a model is made, you can control it, improve it and use it in new ways. This book takes you on a journey to build your own DeepSeek model from the very beginning.
- Use key DeepSeek design ideas like multi-head attention and expert layers.
- Build a training setup that improves speed and efficiency.
- Use parallel processing to make better use of hardware.
- Apply training methods like fine-tuning and reinforcement learning to improve results.
- Reduce large models into smaller versions for real-world use.
Build a DeepSeek Model (From Scratch) is a practical guide to creating a powerful AI. The book breaks down the entire process into clear, manageable steps. It uses the Python language and explains everything you need to build, train and fine-tune your own model.
After reading this book, you will have the skills to build a complete language model. You will understand how to prepare data, train the model and make it ready for use. This book is for AI developers, researchers and students who have some experience with deep learning and Python.
Table of Contents:
1 INTRODUCTION TO DEEPSEEK
2 SOLVING THE INFERENCE BOTTLENECK WITH THE KEY-VALUE CACHE
3 THE DEEPSEEK BREAKTHROUGH: MULTI-HEAD LATENT ATTENTION (MLA)
4 MIXTURE-OF-EXPERTS (MOE) IN DEEPSEEK: SCALING INTELLIGENCE EFFICIENTLY
5 MULTI-TOKEN PREDICTION AND FP8 QUANTIZATION
6 THE DEEPSEEK TRAINING PIPELINE: BUILDING A FOUNDATION MODEL
7 POST-TRAINING: SUPERVISED FINE-TUNING AND REINFORCEMENT LEARNING
8 KNOWLEDGE DISTILLATION: MAKING POWERFUL MODELS PRACTICAL
APPENDIX
APPENDIX A: DEEPSEEK IN CONTEXT: A COMPARISON WITH OTHER LLMS
About the Author :
Raj Dandekar is a computer scientist and co-founder of Vizuara AI Labs. He is an AI researcher who specialises in creating large language models. With his experience in building AI systems, he makes complex ideas easy to follow. He helps you understand how to build powerful models from scratch.
Rajat Dandekar is PhD in Mechanical Engineering from Purdue University. He co-founded Vizuara AI Labs. He is an expert in training and improving AI models. He brings a practical approach to his writing, focusing on real-world results. He shows you how to make your models smarter and more efficient.
Sreedath Panat holds a PhD from MIT and is a co-founder of Vizuara AI Labs. He is a software engineer who focuses on making AI models run faster. He explains how to handle the technical challenges of building large systems. He helps you turn your ideas into a working AI model.
Naman Dwivedi is an AI researcher at Vizuara AI Labs. He is a machine learning specialist known for his work on AI reasoning. He brings a clear and simple style to explaining how models think. He helps you train your models to solve difficult problems.
