Build robust LLM-powered apps, chatbots, and agents while mastering AI engineering principles that will help you outlast the tools and the hype.
Want to build an LLM-powered app but don't know where to begin?With this step-by-step guide, you can master the underlying principles of AI engineering by building an LLM-powered app from the ground up. Tame unpredictable models with prompt and context engineering. Use evals to keep them on track. Give chatbots the knowledge to answer anything a user wants to know. Equip agents with the tools and smarts to actually get the job done. By the end, you'll have the intuition and the confidence to build on top of LLMs in the real world.
Fragmented documentation, obsolete tutorials, and frameworks that deliver a prototype but flop in production can make AI engineering feel overwhelming. But it doesn't have to be that way. With real-world code and step-by-step instructions as your guide, you can learn to build robust LLM-powered apps from the ground up while mastering both the howand why of the most crucial underlying concepts.
Harness context engineering and retrieval systems to create AI assistants that understand your proprietary data. Create chatbots that answer organization-specific questions and help solve users' issues. Design agents that conduct research, make decisions, and take action in the real world. Level up your prompt engineering and get an LLM to do your bidding---not its own. Use automated evals to keep constant tabs on your app's quality while setting up guardrails to protect your users and organization. And implement observability systems that make it easy to debug your app when things do go wrong.
With a systematic approach grounded in the core principles of building AI apps for real users, you'll easily evolve and adapt even as the hype and tools come and go.
Table of Contents:
- Foundations
- HeLLMo,World!
- Signing Up for an LLM-as-a-Service
- Creating Our First App
- Tweaking the Model and Temperature
- Checking API Usage
- Wrapping Up
- Understanding How LLMs Work
- What Is a Large Language Model (LLM)
- Realizing LLMs Are Nondeterministic Creatures
- Gauging the Temperature
- Understanding the Challenges of Nondeterminism
- Wrapping Up
- Diving Deeper into LLMs
- Diving into Tokens
- Diving into Embeddings
- Diving into Fine-Tuning
- Wrapping Up
- Selecting an LLM
- Getting Your Hands on an LLM
- Comparing Different LLMs
- Deciding on an LLM
- Wrapping Up
- Chatbots
- Building a Chatbot
- Getting User Input
- Augmenting the Prompt
- Adding Multi-Turn Dialogue
- Managing State with Memory Systems
- Adding a System Prompt
- Building the Messages Array
- Wrapping Up
- Augmenting a Prompt with Knowledge
- Building a Chatbot
- Augmenting with Knowledge
- Avoiding Context Window Limitations
- Preparing the Data
- Implementing the Knowledge Chatbot
- Running into PACKing Problems
- Wrapping Up
- Efficiently Adding Knowledge with RAG
- Augmenting with Documentation Chunks
- Getting into Search Engines, Retrieval, and RAG
- Searching with Meaning: Keywords Versus Semantics
- Using Embedding-Similarity Search
- Building a Starter Search Engine
- Implementing a RAG Chatbot
- Wrapping Up
- Measuring Quality with Evals
- Introducing Evals
- Setting Up Our App
- Conducting Error Analysis
- Open Coding
- Axial Coding
- Creating an Eval Test Framework
- Running Human Evals
- Wrapping Up
- Prompt Engineering
- Eliminating Ambiguity
- Utilizing the System Prompt
- Rewriting History
- Using Delimiters and Bullet Points
- Reordering Prompt Components
- Wrapping Up
- Reducing Hallucinations
- Understanding Why Our App Hallucinates
- Instructing the LLM to Be Faithful
- Pleading and Threatening
- Upgrading the Model
- Citing Sources and Few-Shot Prompting
- Iterate, Iterate, Iterate
- Chain-of-Thought Prompting
- Final Prompt Engineering Thoughts
- Checking On Our Evals
- Wrapping Up
- Evaluating and Optimizing RAG
- Discovering a RAG Failure
- Evaluating RAG
- Expanding the Query
- Metadata-Based Filtering
- Evaluating RAG Subcomponents
- Dreaming Up an Agentic RAG Wish List
- Wrapping Up
- Agents
- Equipping an LLM with Tools
- Understanding an LLM’s Limitations
- Triggering a Function
- Defining “Agents”
- Feeding Tool Results Back to the LLM
- Building a Website Reader Tool
- Deciding to Use a Tool
- Using the Tools API
- Wrapping Up
- Running the Agent Loop
- Solving a Complex Problem
- Constructing an Agent Loop
- Building a News Podcast Agent
- Exploring Agent Failure Modes and Evals
- Giving the Agent a Plan
- Asking the Agent to Create a Plan
- Wrapping Up
- Architecting Agentic Workflows
- Designing an LLM Assembly Line
- Implementing an LLM Assembly Line
- Weighing Agentic Workflows Against Classic Agent Loops
- Workflow Routing
- Performing Tasks in Parallel
- Wrapping Up
- Enhancing Retrieval with Agentic RAG
- Architecting an Agentic RAG Plan
- Implementing a RAG Agent
- Avoiding Unnecessary RAG
- Generating Structured Outputs
- Researching as an Agent
- Conducting Multi-Hop Research
- Wrapping Up
- Building System-Integrated Agents
- Integrating with Databases
- Reading and Writing
- Writing Safely
- Including a Human in the Loop
- Integrating with Web APIs
- Integrating MCP and Other Third-Party Tools
- Hosting Your Own Tools
- Wrapping Up
- Production
- Setting Guardrails
- Introducing Guardrail Types
- Guarding LLMs with Other Models
- Balancing Guardrail Trade-Offs
- Mitigating Cybersecurity Risks
- Protecting Personally Identifiable Information
- Using Guardrail Frameworks
- Red-Teaming, Evals, and Monitoring
- Wrapping Up
- Observing AI Systems
- Logging All the Things
- Using Observability Tools
- Running Evals in Production
- Monitoring and Alerts
- Gathering User Feedback
- Wrapping Up
- Handling Exceptions
- Understanding Your Errors
- Retrying Requests
- Switching Models
- Falling Back to Semantic Search and Caching
- Fitting in the Context Window
- Aborting Hanging Requests
- Recovering from Tool Failures
- Creating a Fallback Plan
- Wrapping Up
- Automating Evals
- Unit Testing
- Running Reference-Based Evals
- Checking Outputs Deterministically
- Using an LLM-as-Judge
- Running Evals
- Aligning the LLM Judge
- Working with Imperfect Judges
- Wrapping Up
- Final Thoughts
About the Author :
Jay Wengrow is an experienced educator and software engineer. He is the founder of Actualize, a software and AI engineering education company, and specializes in making advanced technical topics approachable for professionals across industries. He is also the author of the popular Common-Sense Guide to Data Structures and Algorithms book series.
Review :
Jay Wengrow demystifies AI engineering, transforming complex topics such as RAG, evals, and agents into practical, actionable steps. This book is an essential guide for any developer looking to build robust, real-world LLM applications.
- Iyanuoluwa Ajao, Senior Applied AI Engineer, Dataligence Labs
This is the guide I wish I had when I started building with LLMs. It masterfully bridges the gap between theory and the practical realities of shipping AI products, teaching the crucial, hard-won lessons about iteration and trade-offs that define professional AI engineering.
- Nithin Singh Mohan, AI and Supercomputing Leader, Hewlett Packard Enterprise
Finally, a book that provides a clear explanation of how prompt engineering works. Jay Wengrow offers a no-nonsense guide to understanding important concepts of AI and ultimately a road map to change the world.
- Jon Glass, Software Engineer, NinjaTrader, LLC
Most LLM failures are not model problems---they are engineering problems. This book stands out by treating agents, tools, guardrails, observability, and evaluation as first-class engineering concerns, making it easier to reason about where systems fail and how to design them responsibly.
- Sree Ram Kishore Kumbham, Senior Software Engineer
I can’t recommend this book enough. It works equally well for people who are completely new to AI and want to get up to speed and those with fragmented knowledge who are looking for a more complete, structured understanding.
- Uberto Barbinin, Author of Process over Magic: Beyond Vibe Coding
Jay Wengrow has a knack for explaining complex ideas in a simple and intuitive way. His latest book will help anyone understand the ins and outs of AI engineering. I highly recommend this book to anyone looking to get into this field.
- Monsur Khan, Senior Associate Data Engineer
Given the limitations of basic LLM usage, I wouldn’t blame readers who quickly become disheartened. However, with advanced techniques, your optimism will grow. Surprisingly, very little code is involved, and much is achieved by fine-tuning the words supplied to the LLM, so a non-technical reader could pick this up.
- Nigel Lowry, Company Director and Principal Consultant, Lemmata
No programming knowledge is needed for the concepts in this book. The writing is clear and easy to follow. Chapters 2 and 3 alone will quickly inform technical and nontechnical readers about what LLMs really do and disabuse them of mistaken ideas about "intelligence" and LLM hallucinations. For those who can program, the book builds, chapter by chapter, upon a simple LLM chatbot toward one using RAG, evaluations, and other, more advanced techniques.
- Robert Ladyman, Director, File-Away, Limited
