High Performance Computing

The computing power that's available on the average desktop has exploded in the past few years. A typical PC has performance exceeding that of a multi-million dollar supercomputer a mere decade ago. To some people, that might mean that it's time to sit back and watch computers get faster: performance is no longer an issue, we'll let hardware do the work. But if you're looking at this book, you're not one of them. Performance is always an issue. Even with the fastest computers, there's a need to harness the processing power and get more work done in a limited amount of time. If you're a software developer, you probably know that getting the most out of a modern workstation or PC can be tricky. Paying closer attention to memory reference patterns and loop structure can have a huge payoff. High Performance Computing discusses how modern workstations get their performance and how you can write code that makes optimal use of your hardware. You'll learn what the newest buzzwords really mean, how caching and other memory design features affect the way your software behaves, and where the newest "post-RISC" architectures are headed. If you're involved with purchasing or evaluating workstations, this book will help you make intelligent comparisons. You'll learn how to interpret the commonly quoted industry benchmarks, what vendors do to show their machines in the best possible light, and how to run your own benchmarks. Whether you're using the latest Pentium PC or a highly specialized multiprocessor, you'll find High Performance Computing an indispensable guide. Topics covered include: CPU and memory architecture for RISC microprocessors and their successors Optimizing compilers Timing and profiling programs Understanding parallelism Loop and memory reference optimizations Benchmarking Parallel Computing and Multiprocessing

Table of Contents:
Preface. PART 1: Modern Computer Architectures. hapter 1. What Is High Performance Computing? Why Worry About Performance? Scope of High Performance Computing Studying High Performance Computing Measuring Performance The Next Step. Chapter 2. High Performance Microprocessors Why CISC? Fundamentals of RISC Second-Generation RISC Processors RISC Means Fast Out-of-Order Execution: The Post-RISC Architecture Future Trends: Intel IA-64 and EPIC Closing Notes. Chapter 3. Memory Memory Technology Registers Caches Cache Organization Virtual Memory Improving Memory Performance Closing Notes. Chapter 4. Floating-Point Numbers Reality Representation Effects of Floating-Point Representation Improving Accuracy Using Guard Digits History of IEEE Floating-Point Format IEEE Floating-Point Standard IEEE Storage Format IEEE Operations Special Values Exceptions and Traps Compiler Issues Closing Notes. PART 2: Programming and Tuning Software. Chapter 5. What a Compiler Does History of Compilers Which Language to Optimize? Optimizing Compiler Tour Optimization Levels Classical Optimizations Closing Notes. Chapter 6. Timing and Profiling Timing Subroutine Profiling Basic Block Profilers Virtual Memory Closing Notes. Chapter 7. Eliminating Clutter Subroutine Calls Branches Branches Within Loops Other Clutter Closing Notes. Chapter 8. Loop Optimizations Operation Counting Basic Loop Unrolling Qualifying Candidates for Loop Unrolling Nested Loops Loop Interchange Memory Access Patterns When Interchange Won't Work Blocking to Ease Memory Access Patterns Programs That Require More Memory Than You Have Closing Notes. PART 3: Shared-Memory Parallel Processors. Chapter 9. Understanding Parallelism Dependencies Loops Loop-Carried Dependencies Ambiguous References Closing Notes. Chapter 10. Shared-Memory Multiprocessors Symmetric Multiprocessing Hardware Multiprocessor Software Concepts Techniques for Multithreaded Programs A Real Example Closing Notes. Chapter 11. Programming Shared-Memory Multiprocessors Automatic Parallelization Assisting the Compiler Closing Notes PART 4: Scalable Parallel Processing. Chapter 12. Large-Scale Parallel Computing Amdahl's Law Interconnect Technology A Taxonomy of Parallel Architectures A Survey of Parallel Architectures The Top 500 Report Shared Uniform Memory MIMD Shared Non-Uniform Memory MIMD Systems Distributed-Memory MIMD Architecture Single Instruction, Multiple Data Closing Notes. Chapter 13. Language Support for Performance Data-Parallel Problem: Heat Flow Explicitly Parallel Languages FORTRAN 90 Problem Decomposition High Performance FORTRAN (HPF) Closing Notes. Chapter 14. Message-Passing Environments Parallel Virtual Machine (PVM) Message-Passing Interface (MPI) Closing Notes PART 5: Benchmarking. Chapter 15. Using Published Benchmarks User Benchmarks Industry Benchmarks Closing Notes. Chapter 16. Running Your Own Benchmarks Choosing What to Benchmark Types of Benchmarks Preparing the Code Closing Notes PART 6: Appendixes Appendix A. Processor Architectures Appendix B. Looking at Assembly Language Appendix C. Future Trends: Intel IA-64 Appendix D. How FORTRAN Manages Threads at Runtime Appendix E. Memory Performance Index

About the Author :
Kevin Dowd is the president of Atlantic Computing Technology Corporation, located in Newington, Connecticut. He is the author of Getting Connected: The Internet at 56K and Up, also published by O'Reilly and Associates. Charles Severance is the director for computing services for the College of Engineering at Michigan State University, where he is also a professor in the Computer Science Department, teaching classes on high performance computing, computer architecture, and the Internet. His current research is in the area of Internet delivery of educational material. He has been active in IEEE standards for many years, and he edits a monthly column in the magazine IEEE Computer on computer standards. Charles is the co-host of a Television show called "Nothin but Net," and was previously the co-host of Television show called "Internet:TCI."