Second Edition updated for 2026. This book is a complete rewrite with entirely new code samples while covering the same core topics as the original edition. All code and infographics from the book are available at the GitHub repository BurstBooksPublishing.
GPU Parallel Computing: From Basics to Breakthroughs - A Technical Guide to GPU Programming
If you want to understand how modern GPUs work and how to use them effectively for high-performance workloads, this book provides the technical foundation required.
This book assumes no prior exposure to GPU internals; however, a working knowledge of electronics and general computer architecture is recommended.
It is written for students, engineers, researchers, and data scientists who are new to GPU architecture and parallel programming and want a rigorous introduction before progressing into optimization and large-scale GPU systems.
If you are already an experienced CUDA performance engineer or low-level GPU architect seeking a specialized microarchitectural reference manual, this book is not positioned for that purpose.
What You Will Learn
GPU Architecture FundamentalsStreaming multiprocessors and SIMT execution
Warp scheduling and instruction flow
GPU memory hierarchy and bandwidth considerations
GPU Programming ModelsPerformance OptimizationMemory access patterns and coalescing
Warp divergence and latency hiding
Occupancy principles and kernel configuration
Real-World ApplicationsAdvanced TopicsMulti-GPU communication
Tensor cores and mixed precision
Profiling, debugging, and performance analysis
The early chapters establish architectural clarity and programming fundamentals.
Later chapters address optimization strategies, scalability, and applied GPU workloads.
Who This Book Is ForStudents entering GPU computing
Engineers transitioning into parallel architecture
Researchers and data scientists adopting GPU acceleration
This is a technical book. It builds understanding from architectural principles upward and focuses on performance-oriented reasoning rather than superficial overview.
Why This BookMany GPU resources either assume too much prior knowledge or remain overly abstract.
This book emphasizes structured technical understanding:
How GPUs execute threads
Why performance bottlenecks occur
How architectural constraints shape results
How programming decisions map to hardware behavior
Clear explanations.
Practical code examples.
Architectural context.
