CUDA for Engineers: An Introduction to High-Performance Parallel Computing

CUDA for Engineers gives you direct, hands-on engagement with personal, high-performance parallel computing, enabling you to do computations on a gaming-level PC that would have required a supercomputer just a few years ago.   The authors introduce the essentials of CUDA C programming clearly and concisely, quickly guiding you from running sample programs to building your own code. Throughout, you’ll learn from complete examples you can build, run, and modify, complemented by additional projects that deepen your understanding. All projects are fully developed, with detailed building instructions for all major platforms.   Ideal for any scientist, engineer, or student with at least introductory programming experience, this guide assumes no specialized background in GPU-based or parallel computing. In an appendix, the authors also present a refresher on C programming for those who need it.   Coverage includes Preparing your computer to run CUDA programs Understanding CUDA’s parallelism model and C extensions Transferring data between CPU and GPU Managing timing, profiling, error handling, and debugging Creating 2D grids Interoperating with OpenGL to provide real-time user interactivity Performing basic simulations with differential equations Using stencils to manage related computations across threads Exploiting CUDA’s shared memory capability to enhance performance Interacting with 3D data: slicing, volume rendering, and ray casting Using CUDA libraries Finding more CUDA resources and code   Realistic example applications include Visualizing functions in 2D and 3D Solving differential equations while changing initial or boundary conditions Viewing/processing images or image stacks Computing inner products and centroids Solving systems of linear algebraic equations Monte-Carlo computations  

Table of Contents:
Acknowledgments            xvii About the Authors             xix   Introduction          1 What Is CUDA?     1 What Does “Need-to-Know” Mean for Learning CUDA?     2 What Is Meant by “for Engineers”?     3 What Do You Need to Get Started with CUDA?      4 How Is This Book Structured?      4 Conventions Used in This Book      8 Code Used in This Book      8 User’s Guide      9 Historical Context      10 References      12   Chapter 1: First Steps            13 Running CUDA Samples      13 Running Our Own Serial Apps      19 Summary      22 Suggested Projects      23   Chapter 2: CUDA Essentials           25 CUDA’s Model for Parallelism     25 Need-to-Know CUDA API and C Language Extensions     28 Summary      31 Suggested Projects      31 References      31   Chapter 3: From Loops to Grids           33 Parallelizing dist_v1    33 Parallelizing dist_v2      38 Standard Workflow      42 Simplified Workflow      43 Summary      47 Suggested Projects      48 References      48   Chapter 4: 2D Grids and Interactive Graphics           49 Launching 2D Computational Grids      50 Live Display via Graphics Interop     56 Application: Stability      66 Summary      76 Suggested Projects      76 References      77   Chapter 5: Stencils and Shared Memory           79 Thread Interdependence      80 Computing Derivatives on a 1D Grid      81 Summary     117 Suggested Projects      118 References      119   Chapter 6: Reduction and Atomic Functions          121 Threads Interacting Globally      121 Implementing parallel_dot      123 Computing Integral Properties: centroid_2d      130 Summary      138 Suggested Projects      138 References       138   Chapter 7: Interacting with 3D Data           141 Launching 3D Computational Grids: dist_3d     144 Viewing and Interacting with 3D Data: vis_3d      146 Summary      171 Suggested Projects     171 References     171   Chapter 8: Using CUDA Libraries           173 Custom versus Off-the-Shelf      173 Thrust      175 cuRAND      190 NPP      193 Linear Algebra Using cuSOLVER and cuBLAS      . 201 cuDNN      207 ArrayFire      207 Summary      207 Suggested       208 References     209   Chapter 9: Exploring the CUDA Ecosystem            211 The Go-To List of Primary Sources      211 Further Sources      217 Summary      218 Suggested Projects     219   Appendix A: Hardware Setup           221 Checking for an NVIDIA GPU: Windows      221 Checking for an NVIDIA GPU: OS X     222 Checking for an NVIDIA GPU: Linux     223 Determining Compute Capability      223 Upgrading Compute Capability      225   Appendix B: Software Setup            229 Windows Setup     229 OS X Setup      238 Linux Setup      240   Appendix C: Need-to-Know C Programming          245 Characterization of C     245 C Language Basics      246 Data Types, Declarations, and Assignments      248 Defining Functions      250 Building Apps: Create, Compile, Run, Debug      251 Arrays, Memory Allocation, and Pointers      262 Control Statements: for, if      263 Sample C Programs     267 References     277   Appendix D: CUDA Practicalities: Timing, Profiling, Error Handling, and Debugging            279 Execution Timing and Profiling      279 Error Handling     292 Debugging in Windows      298 Debugging in Linux     305 CUDA-MEMCHECK     308 Using Visual Studio Property Pages      309 References     312   Index            313