The Data Access Handbook
Achieving Optimal Database Application Performance and Scalability
John Goodson • Robert A. Steward
Drive breakthrough database application performance by optimizing middleware and connectivity
Performance and scalability are more critical than ever in today’s enterprise database applications, and traditional database tuning isn’t nearly enough to solve the performance problems you are likely to see in those applications. Nowadays, 75-95% of the time it takes to process a data request is typically spent in the database middleware. Today’s worst performance and scalability problems are generally caused by issues with networking, database drivers, the broader software/hardware environment, and inefficient coding of data requests. In The Data Access Handbook, two of the world’s leading experts on database access systematically address these issues, showing how to achieve remarkable improvements in performance of real-world database applications.
Drawing on their unsurpassed experience with every leading database system and database connectivity API, John Goodson and Rob Steward reveal the powerful ways middleware affects application performance and guide developers with designing and writing API code that will deliver superior performance in each leading environment. In addition to covering essential concepts and techniques that apply across database systems and APIs, they present many API examples for ODBC, JDBC, and ADO.NET as well as database system examples for DB2, Microsoft SQL Server, MySQL, Oracle, and Sybase.
Coverage includes
- Clearly understanding how each component of database middleware can impact performance and scalability
- Writing database applications to reduce network traffic, limit disk I/O, optimize application-to-driver interaction, and simplify queries—including examples for ODBC, JDBC, and ADO.NET
- Managing connections, transactions, and SQL statement execution more efficiently
- Making the most of connection and statement pooling
- Writing good benchmarks to predict your application’s performance
- Systematically resolving performance problems—including eight start-to-finish case-study examples
If you’re a software architect, system designer, or database application developer, The Data Access Handbook will be your most indispensable database application performance resource. It’s the one book that focuses on the areas where you can achieve the greatest improvements—whether you’re designing new database applications or troubleshooting existing ones.
John Goodson is vice president and general manager of the DataDirect division of Progress Software, a leader in
data connectivity and mainframe integration. For 20 years, he has worked with Sun, Microsoft, and others to develop database connectivity standards such as J2EE, JDBC, ODBC, and ADO. He served on the ANSI H2 committee that built the SQL standard and now participates in the JDBC Expert Group and Java Rowsets standards committees.
Rob Steward, vice president of R&D at the DataDirect
division of Progress Software,
is responsible for the development, strategy, and oversight of the company’s data connectivity products. Rob has spent the past 15 years developing high-performing database driver and data providers, including ODBC, JDBC, and ADO.NET.
Both authors have spoken on database application performance at many industry events.
Visit www.dataaccesshandbook.com to get the code examples presented in this book and other supplemental information for DB2, MicrosoftSQL Server, MySQL, Oracle, and Sybase.
Table of Contents:
Preface
CHAPTER 1 Performance Isn’t What It Used To Be
Where Are We Today?
The Network
The Database Driver
The Environment
Your Database Application
Our Goal for This Book
CHAPTER 2 Designing for Performance: What’s Your Strategy?
Your Applications
Database Connections
Transaction Management
SQL Statements
Data Retrieval
Extended Security
Static SQL Versus Dynamic SQL
The Network
The Database Driver
Know Your Database System
Using Object-Relational Mapping Tools
Summary
CHAPTER 3 Database Middleware: Why It’s Important
What Is Database Middleware?
How Database Middleware Affects Application Performance
Database Drivers
What Does a Database Driver Do?
Database Driver Architecture
Runtime Performance Tuning Options
Configuring Database Drivers/Data Providers
Summary
CHAPTER 4 The Environment: Tuning for Performance
Runtime Environment (Java and .NET)
JVM
.NET CLR
Operating System
Network
Database Protocol Packets
Network Packets
Configuring Packet Size
Analyzing the Network Path
Reducing Network Hops and Contention
Avoiding Network Packet Fragmentation
Increasing Network Bandwidth
Hardware
Memory
Disk
CPU (Processor)
Network Adapter
Virtualization
Summary
CHAPTER 5 ODBC Applications: Writing Good Code
Managing Connections
Connecting Efficiently
Using Connection Pooling
Establishing Connections One at a Time
Using One Connection for Multiple Statements
Obtaining Database and Driver Information Efficiently
Managing Transactions
Managing Commits in Transactions
Choosing the Right Transaction Model
Executing SQL Statements
Using Stored Procedures
Using Statements Versus Prepared Statements
Using Arrays of Parameters
Using the Cursor Library
Retrieving Data
Retrieving Long Data
Limiting the Amount of Data Retrieved
Using Bound Columns
Using SQLExtendedFetch Instead of SQLFetch
Determining the Number of Rows in a Result Set
Choosing the Right Data Type
Updating Data
Using SQLSpecialColumns to Optimize Updates and Deletes
Using Catalog Functions
Minimizing the Use of Catalog Functions
Avoiding Search Patterns
Using a Dummy Query to Determine Table Characteristics
Summary
CHAPTER 6 JDBC Applications: Writing Good Code
Managing Connections
Connecting Efficiently
Using Connection Pooling
Establishing Connections One at a Time
Using One Connection for Multiple Statements
Disconnecting Efficiently
Obtaining Database and Driver Information Efficiently
Managing Transactions
Managing Commits in Transactions
Choosing the Right Transaction Model
Executing SQL Statements
Using Stored Procedures
Using Statements Versus Prepared Statements
Using Batches Versus Prepared Statements
Using getXXX Methods to Fetch Data from a Result Set
Retrieving Auto-Generated Keys
Retrieving Data
Retrieving Long Data
Limiting the Amount of Data Retrieved
Determining the Number of Rows in a Result Set
Choosing the Right Data Type
Choosing the Right Cursor
Updating Data
Using Positioned Updates, Inserts, and Deletes (updateXXX Methods)
Using getBestRowIdentifier() to Optimize Updates and Deletes
Using Database Metadata Methods
Minimizing the Use of Database Metadata Methods
Avoiding Search Patterns
Using a Dummy Query to Determine Table Characteristics
Summary
CHAPTER 7 .NET Applications: Writing Good Code
Managing Connections
Connecting Efficiently
Using Connection Pooling
Establishing Connections One at a Time
Disconnecting Efficiently
Obtaining Database and Data Provider Information Efficiently
Managing Transactions
Managing Commits in Transactions
Choosing the Right Transaction Model
Executing SQL Statements
Executing SQL Statements that Retrieve Little or No Data
Using the Command.Prepare Method
Using Arrays of Parameters/Batches Versus Prepared Statements
Using Bulk Load
Using Pure Managed Providers
Selecting .NET Objects and Methods
Avoiding the CommandBuilder Object
Choosing Between a DataReader and DataSet Object
Using GetXXX Methods to Fetch Data from a DataReader
Retrieving Data
Retrieving Long Data
Limiting the Amount of Data Retrieved
Choosing the Right Data Type
Updating Data
Summary
CHAPTER 8 Connection Pooling and Statement Pooling
Connection Pool Model for JDBC
Configuring Connection Pools
Guidelines
Connection Pool Model for ODBC
Connection Pooling as Defined in the ODBC Specification
Configuring Connection Pools
Guidelines
Connection Pool Model for ADO.NET
Configuring Connection Pools
Guidelines
Using Reauthentication with Connection Pooling
Configuring Connection Pooling with Reauthentication in a JDBC Environment
Using Statement Pooling
Using Statement Pooling with Connection Pooling
Guidelines
Summary: The Big Picture
CHAPTER 9 Developing Good Benchmarks
Developing the Benchmark
Define Benchmark Goals
Reproduce the Production Environment
Isolate the Test Environment
Reproduce the Workload
Measure the Right Tasks
Measure over a Sufficient Duration of Time
Prepare the Database
Make Changes One at a Time
Assess Other Factors
Benchmark Example
Summary
CHAPTER 10 Troubleshooting Performance Issues
Where to Start
Changes in Your Database Application Deployment
The Database Application
The Database Driver
Runtime Performance Tuning Options
Architecture
The Environment
Runtime Environment (Java and .NET)
Operating System
Network
Hardware
Case Studies
Case Study 1
Case Study 2
Case Study 3
Case Study 4
Case Study 5
Case Study 6
Case Study 7
Case Study 8
Summary
CHAPTER 11 Data Access in Service-Oriented Architecture (SOA) Environments
What Is Service-Oriented Architecture (SOA)?
Data Access Guidelines for SOA Environments
Involve Data Experts in Addition to SOA Experts
Decouple Data Access from Business Logic
Design and Tune for Performance
Consider Data Integration
Summary
Glossary
0137143931 TOC 2/19/2009
About the Author :
John Goodson: As the executive leader of DataDirect Technologies, John is responsible for daily operations, business development, product direction, and long-term corporate strategy.
John was a principal engineer at Data General for seven years, working on their relational database product, DG/SQL. Since joining DataDirect Technologies in 1992, he has held positions of increasing responsibility in research and development, technical support, and marketing. John is a well-known and respected industry luminary and data connectivity expert. For more than 15 years, he has worked closely with Sun Microsystems and Microsoft on the development and evolution of database connectivity standards including J2EE, JDBC, .NET, ODBC, and ADO. John has been involved with the ANSI NCITS H2 Committee, which is responsible for building the SQL standard, and the X/Open (Open Group) SQL Access Group, which is responsible for building call-level interfaces into relational databases. He is actively involved in Java standards committees, including the JDBC Expert Group. In addition, John has published numerous articles and spoken publicly on topics related to data management. John is also a patent holder in the area of distributed transactions for Microsoft SQL Server Java middleware.
John holds a Bachelor of Science in computer science from Virginia Polytechnic Institute and State University in Blacksburg,Virginia.
Rob Steward: As vice president of research and development for DataDirect Technologies, Rob is responsible for the development, strategy, and oversight of the company’s data connectivity products, including the Shadow mainframe integration suite client software and the industry-leading DataDirect Connect family of database drivers and data providers: Connect for ODBC, Connect for JDBC, and Connect for ADO.NET. Additional product development responsibilities include DataDirect Sequelink and DataDirect XQuery, as well as the management of DataDirect’s Custom Engineering Development group.
Rob has spent more than 15 years developing database access middleware, including .NET data providers, ODBC drivers, JDBC drivers, and OLE DB data providers. He has held a number of management and technical positions at DataDirect Technologies and serves on various standards committees. Earlier in his career, he worked as lead software engineer at Marconi Commerce Systems Inc.
Rob holds a Bachelor of Science in computer science from North Carolina State University, Raleigh.
