Virtualizing and Tuning Large Scale Java Platforms

Virtualizing and Tuning Large-Scale Java Platforms   Technical best practices and real-world tips for optimizing enterprise Java applications on VMware vSphere®   Enterprises no longer ask, “Can Java be virtualized”? Today, they ask, “Just how large can we scale virtualized Java application platforms, and just how efficiently can we tune them?” Now, the leading expert on Java virtualization answers these questions, offering detailed technical information you can apply in any production or QA/test environment.   Emad Benjamin has spent nine years virtualizing VMware’s own enterprise Java applications and working with nearly 300 leading VMware customers on projects of all types and sizes—from 100 JVMs to 10,000+, with heaps from 1GB to 360GB, and including massive big-data applications built on clustered JVMs. Reflecting all this experience, he shows you how to successfully size and tune any Java workload.   This reference and performance “cookbook” identifies high-value optimization opportunities that apply to physical environments, virtual environments, or both. You learn how to rationalize and scale existing Java infrastructure, modernize architecture for new applications, and systematically benchmark and improve every aspect of virtualized Java performance. Throughout, Benjamin offers real performance studies, specific advice, and “from-the-trenches” insights into monitoring and troubleshooting.   Coverage includes --Performance issues associated with large-scale Java platforms, including consolidation, elasticity, and flexibility --Technical considerations arising from theoretical and practical limits of Java platforms --Building horizontal in-memory databases with VMware vFabric SQLFire to improve scalability and response times --Tuning large-scale Java using throughput/parallel GC and Concurrent Mark and Sweep (CMS) techniques --Designing and sizing a new virtualized Java environment --Designing and sizing new large-scale Java platforms when migrating from physical to virtualized deployments --Designing and sizing large-scale Java platforms for latency-sensitive in-memory databases --Real-world performance studies: SQLFire vs. RDBMS, Spring-based Java web apps, vFabric SpringTrader, application tiers, data tiers, and more --Performance differences between ESXi3, 4.1, and 5 --Best-practice considerations for each type of workload: architecture, performance, design, sizing, and high availability --Identifying bottlenecks in the load balancer, web server, Java application server, or DB Server tiers --Advanced vSphere Java performance troubleshooting with esxtop --Performance FAQs: answers to specific questions enterprise customers have asked    

Table of Contents:
Preface xv Chapter 1 Introduction to Large-Scale Java Platforms 1 Large-Scale Java Platform Categories 1 Large-Scale Java Platform Trends and Requirements 2     Compute-Resource Consolidation 2     JVM Instance Consolidation 3     Elasticity and Flexibility 3     Performance 4 Large-Scale Java Platform Technical Considerations 4     Theoretical and Practical Limits of Java Platforms 4     NUMA 9     Most Common JVM Size Found in Production Environments 14     Horizontal Scaling Versus Vertical Scaling of JVMs and VMs 15 Summary 20 Chapter 2 Modern Scalable Data Platforms 21 SQLFire Topologies 24     Client/Server Topology 24     Peer-to-Peer Topology 27     Redundancy Zones 28     Global Multisite Topology 28 SQLFire Features 30     Server Groups 32     Partitioning 34     Redundancy 37     Colocation 38     Disk Persistence 39     Transactions 41     Cache Plug-In 46     Listeners 47     Writers 50     Asynchronous Listeners 52     DBSynchronizer 54     SQLF Commands and DDLUtils 57 Active-Active Architectures and Modern Data Platforms 57 Chapter Summary 61 Chapter 3 Tuning Large-Scale Java Platforms 63 GC Tuning Approach 70     Step A: Young Generation Tuning 71     Step B: Old Generation Tuning 76     Step C: Survivor Spaces Tuning 78 Chapter Summary 78 Chapter 4 Designing and Sizing Large-Scale Java Platforms 79 Designing and Sizing a New Environment for a Virtualized Large-Scale Java Platform 79     Step1: Establishing Your Current Production Load Profi le 80     Step 2: Establish a Benchmark 82     Step 3: Size the Production Environment 95 Sizing vFabric SQLFire Java Platforms: Category 2 Workloads 96     Step A: Determine Entity Groups 97     Step B: Determine the Memory Size of the Data Fabric 100     Step C: Establish Building Block VM and JVM Size and How Many vFabric SQLFire Members Are Needed 105     Understanding the Internal Memory Sections of HotSpot JVM 106     Understanding NUMA Implications on Sizing Large VMs and JVMs 108     vFabric SQLFire Sizing Example 112 Chapter Summary 119 Chapter 5 Performance Studies 121 SQLFire Versus RDBMS Performance Study 121     Performance Results 123     Summary of Findings 126 The Olio Workload on tc Server and vSphere Performance Study 127     Looking at the Results 127 SpringTrader Performance Study 131     Application and Data Tier vSphere Confi gurations 133     The SpringTrader Performance Study Results 137 Performance Differences Between ESXi 3, 4.1, and 5 139     CPU Scheduling Enhancements 140     Memory Enhancements 140 vSphere 5 Performance Enhancements 142 Chapter Summary 143 Chapter 6 Best Practices 145 Enterprise Java Applications on vSphere Best Practices (Category 1) 148     VM Sizing and Confi guration Best Practices 148     vCPU for VM Best Practices 149     VM Memory Size Best Practices 150     VM Timekeeping Best Practices 156     Vertical Scalability Best Practices 156 Horizontal Scalability, Clusters, and Pools Best Practices 158     Inter-Tier Confi guration Best Practices 160     High-Level vSphere Best Practices 165 SQLFire Best Practices and SQLFire on vSphere Best Practices (Category 2 JVM Workload Best Practices) 166     SQLFire Best Practices 168     vFabric SQLFire Best Practices on vSphere 173 Category 3 Workloads Best Practices 181     IBM JVM and Oracle jRockit JVMs 181 GC Policy Selection 184     IBM GC Choices 186     Oracle jRockit GC Policies 187 Chapter Summary 187 Chapter 7 Monitoring and Troubleshooting Primer 189 Open a Support-Request Ticket 191 Collecting Metrics from vCenter 191 Troubleshooting Techniques for vSphere with esxtop 195 Java Troubleshooting Primer 198     Troubleshooting Java Memory Problems 202     Troubleshooting Java Thread Contentions 203 Chapter Summary 204 Appendix FAQs 205 Glossary 229 Best Practices Best Practice 1: Common Distributed Data Platform 24 Best Practice 2: Client/Server Topology 26 Best Practice 3: Peer-to-Peer Multihomed Machines 27 Best Practice 4: Multisite 29 Best Practice 5: Use Server Groups 33 Best Practice 6: Horizontal Partitioning 37 Best Practice 7: Redundancy 38 Best Practice 8: Colocation 38 Best Practice 9: Disk Persistence 40 Best Practice 10: Transactions 45 Best Practice 11: RowLoader 47 Best Practice 12: Listeners 49 Best Practice 13: Writers 51 Best Practice 14: Asynchronous Listeners 53 Best Practice 15: DBSynchronizer 55 Best Practice 16: VM Sizing and VM-to-JVM Ratio Through a Performance Load Test 149 Best Practice 17: VM vCPU CPU Overcommit 149 Best Practice 18: VM vCPU, Do Not Oversubscribe to CPU Cycles That You Don’t Really Need 150 Best Practice 19: VM Memory Sizing 152 Best Practice 20: Set Memory Reservation for VM Memory Needs 154 Best Practice 21: Use of Large Pages 154 Best Practice 22: Use an NTP Source 156 Best Practice 23: Hot Add or Remove CPU/Memory 157 Best Practice 24: Use vSphere Host Clusters 158 Best Practice 25: Use Resource Pools 159 Best Practice 26: Use Affi nity Rules 159 Best Practice 27: Use vSphere-Aware Load Balancers 160 Best Practice 28: Establish Appropriate Thread Ratios That Prevents Bottlenecks (HTTP threads:Java threads:DB Connections Ratio) 160 Best Practice 29: Apache Web Server Sizing 161 Best Practice 30: Load-Balancer Algorithm Choice and VM Symmetry 164 Best Practice 31: vSphere 5.1 165 Best Practice 32: vSphere Networking 165 Best Practice 33: vSphere Storage 166 Best Practice 34: vSphere Host 166 Best Practice 35: JVM Version 168 Best Practice 36: Use Parallel and CMS GC Policy Combination 168 Best Practice 37: Set Initial Heap Equal to Maximum Heap 170 Best Practice 38: Disable Calls to System.gc() 171 Best Practice 39: New Generation Size 171 Best Practice 40: Using 32-Bit Addressing in a 64-Bit JVM 171 Best Practice 41: Stack Size 172 Best Practice 42: Perm Size 172 Best Practice 43: Table Placements in a JVM 172 Best Practice 44: Enable Hyperthreading and Do Not Overcommit CPU 173 Best Practice 45: CPU Cache Sharing 175 Best Practice 46: vFabric SQLFire Member Server, JVM and VM Ratio 175 Best Practice 47: VM Placement 175 Best Practice 48: Set VM Memory Reservation 175 Best Practice 49: vMotion, DRS Cluster, and vFabric SQLFire Server 176 Best Practice 50: VMware HA and vFabric SQLFire 177 Best Practice 51: Guest OS 177 Best Practice 52: Physical NIC 177 Best Practice 53: Virtual NIC 178 Best Practice 54: Troubleshooting SYN Cookies 179 Best Practice 55: Storage 181     9780133491203    TOC    12/3/2013