Running Xen: A Hands-On Guide to the Art of Virtualization

“This accessible and immediately useful book expertly provides the Xen community with everything it needs to know to download, build, deploy and manage Xen implementations.” —Ian Pratt, Xen Project Leader VP Advanced Technology, Citrix Systems   The Real–World, 100% Practical Guide to Xen Virtualization in Production Environments   Using free, open source Xen virtualization software, you can save money, gain new flexibility, improve utilization, and simplify everything from disaster recovery to software testing. Running Xen brings together all the knowledge you need to create and manage high–performance Xen virtual machines in any environment. Drawing on the unparalleled experience of a world–class Xen team, it covers everything from installation to administration—sharing field-tested insights, best practices, and case studies you can find nowhere else. The authors begin with a primer on virtualization: its concepts, uses, and advantages. Next, they tour Xen’s capabilities, explore the Xen LiveCD, introduce the Xen hypervisor, and walk you through configuring your own hard–disk–based Xen installation. After you’re running, they guide you through each leading method for creating “guests” and migrating existing systems to run as Xen guests. Then they offer comprehensive coverage of managing and securing Xen guests, devices, networks, and distributed resources. Whether you’re an administrator, data center manager, developer, system integrator, or ISP, Running Xen will help you achieve your goals with Xen–reliably, efficiently, with outstanding performance, and at a surprisingly low cost.   •Understanding the Xen hypervisor: what it does, and how it works •Using pre-built system images, including compressed file systems •Managing domains with the xm console •Populating and storing guest images •Planning, designing, and configuring networks in Xen •Utilizing Xen security: special purpose VMs, virtual network segments, remote access, firewalls, network monitors, sHype access control, Xen Security Modules (XSM), and more •Managing guest resources: memory, CPU, and I/O •Employing Xen in the enterprise: tools, products, and techniques

Table of Contents:
    Foreword xxi     Preface xxiii Chapter 1: Xen–Background and Virtualization Basics 1     Common Uses and Benefits of Virtualization  2     Types of Virtualization 5         Emulation 6         Full Virtualization 7         Paravirtualization  8         Operating System Level Virtualization  9         Other Types of Virtualization 11         Overview of Virtualization Types 12     Virtualization Heritage 13         The IBM Mainframe 14         Virtualization on Commodity Hardware 15         Virtualization Extensions for x86 15         Xen Origins and Time Line 15     Other Virtualization Systems for Commodity Hardware 18         Emulation 18         Full Virtualization 19         Paravirtualization 21         Operating System Virtualization 23         Popular Virtualization Products 24     Summary 25     References and Further Reading 26 Chapter 2: A Quick Tour with the Xen LiveCD 27     Running the LiveCD 28     Step 1: Downloading the LiveCD Image and Creating the CD 29     Step 2: Choosing a Domain0 Image from the GRUB Menu 30     Step 3: Logging In and the Desktop 31     Step 4: Creating Guests  33     Step 5: Deleting a Guest  38     Step 6: Interacting with Your Guests 38     Step 7: Testing Your Networking 41     Too Many Guests 44     Summary 44     References and Further Reading 45 Chapter 3: The Xen Hypervisor 47     Xen Hypervisor 48     A Privileged Position 50         Protection Rings 50     Domain0 51     Xen Boot Options 54     Choosing an OS for Domain0 59     xend 60         Controlling xend 60         xend Logs 62         xend Configuration 63     XenStore 67     Summary 73     References and Further Reading 73 Chapter 4: Hardware Requirements and Installation of Xen Domain0 75     Xen Domain0 Processor Requirements 76         Intel VT 77         AMD-V 77         HVM 78     Hardware Device Support and Recommendations 78         Disks and Controllers 78         Networking Devices 80         Graphics Devices 80         Power Management 81         Help for Unsupported Hardware 81     Memory Requirements 81     Choosing and Obtaining a Version of Xen 83         Open Source Distributions 83         Commercially Supported Options 84     Methods of Installing Domain0 Hosts 86         Common Prerequisite: The Grand Unified Boot Loader (GRUB) 87     Linux Distributions 87         OpenSUSE 88         CentOS 91         Ubuntu 98         Xen from Binary Packages 101         Gentoo 105     XenExpress 112     Non-Linux Domain0 Installations 114     Building from Source 116     Summary 118     References and Further Reading 118 Chapter 5: Using Prebuilt Guest Images 121     Introduction to DomU Guests 122         Guest Images 122         Operating System Kernels 123         Configuration Files 123     Working with Prebuilt Guest Images 128         Types of Guest Images 128         Downloading Prebuilt Guest Images 130         Mounting and Booting Prebuilt Images 131         Downloading Compressed File Guest Images 146     Converting Images from Other Virtualization Platforms 161     Summary 162     References and Further Reading 163 Chapter 6: Managing Unprivileged Domains 165     Introduction to the xm Utility 166         Prerequisites for Running the xm Utility 166         Generic Format of an xm Command 167     The xm list Subcommand 169         Basic List Information 169         Listing Information about a Specific Guest 171         long Option 172         Label Option 173     The xm create Subcommand 174         Prerequisites for xm create 174         Simple Examples of xm create 175     Guest Configuration Files 178         Python Format 178         Common Configuration Options 179         S-Expression (SXP) Format 180         Path to Configuration Files 181     Diagnosing Problems with Guest Creation 182         Dry Run 182         Console Output 183         Sample Problems 184     Automatically Starting DomUs 191     Shutting Down Guest Domains 193         xm shutdown 193         xm reboot 196         xm destroy 198     Pausing Domains 199         xm pause 200         xm unpause 200     Interacting with a Guest Nongraphically 201         xm console 202         SSH 204     Interacting with a Guest Graphically 204         X Forwarding with SSH 205         Configuration of SSH Server and Client 205         VNC 207         Virtual Frame Buffer and Integrated VNC/SDL Libraries 210         Freenx 212         Remote Desktop 213     Summary 215     References and Further Reading 216 Chapter 7: Populating Guest Images 217     Hardware Virtual Machine (HVM) Guest Population 218         Populating a Guest Image from a Disc or Disc Image (Windows XP Example) 218         Automated Population with virt-install 225     Paravirtualized (PV) Guest Population 228         OpenSUSE: YaST Virtual Machine Management 229         CentOS/Fedora: virt-manager 233         Debian/Ubuntu: debootstrap 242         Gentoo: quickpkg and domi Scripts 246         Xen Express 256     Guest Image Customization 266         Customizing Hostnames  266         Customizing Users 267         Customizing Packages and Services 268         Customizing the File System Table (/etc/fstab) 268     Converting Existing Installations 270     Summary 274     References and Further Reading 274 Chapter 8: Storing Guest Images 277     Logical Volumes 278         Basic LVM Usage 279         Resizing Images 282         Image Snapshots Using Copy on Write 286     Network Image Storage Options 287         iSCSI 288         ATA over Ethernet (AoE) 293         NFS 297         Comparing Network Storage Options 300     Guest Image Files 301         Preparing Compressed tar Image Files 301         Preparing Disk Image Files 302         Preparing Guest Partition Image Files 312         Mounting Disks and Partition Images 314     Summary 316     References and Further Reading 316 Chapter 9: Device Virtualization and Management 319     Device Virtualization 320         Paravirtualization of Devices 320         Full Virtualization of Devices 321         No Virtualization 321     Backends and Frontends 322         Backend Information in XenStore 323         Frontend Information in XenStore 325     Granting Control of a PCI Device 326         Identifying a PCI Device 326         Hiding a PCI Device from Domain0 at Boot 327         Manually Unbinding/Binding a PCI Device at Runtime 328         Granting a PCI Device to Another Domain 329     Exclusive Device Access Versus Trusted Driver Domains 331         Exclusive Device Access 331         Trusted Driver Domains 332         Problems Using Trusted Driver Domains 333     Device Emulation with QEMU-DM 334     Future Directions 335         More Devices 336         Smart Devices 336     Summary 336     References and Further Reading 337 Chapter 10: Network Configuration 339     Network Virtualization Overview 340     Designing a Virtual Network Topology 341     Bridging, Routing, and Network Address Translation 343     Frontend and Backend Network Drivers and Naming 347     Overview of Network Configuration in Xen 349         High-Level Steps 349         Xend Configuration File 350         Guest Domain’s Configuration File 352     Details of Bridging Mode  354         Bridging Configuration Example 355         Testing Results 361     Details of Routing Mode 364         Routing Configuration Example 365         Testing Results 371     Details of NAT Mode 373         NAT Configuration Example 373         Testing Results 379     Configuring Purely Virtual Network Segments 382         Configuring dummy0 383         Testing dummy0 385         Configuring Dummy Bridge 385         Testing Dummy Bridge 388     Assigning MAC Addresses to Virtual Network Interfaces 389         MAC Addresses 389         Specifying or Generating a MAC Address for a Guest Domain 390     Assigning IP Addresses 391         Using an External DHCP Server to Obtain an IP for a Guest Domain 392         Manually Assigning an IP to a Guest Domain 392         Using an Internal DHCP Server to Obtain an IP for a Guest Domain 393     Handling Multiple Network Interfaces in a Domain 394         Handling Multiple Network Interfaces in a driver domain 394         Handling Multiple Network Interfaces in a Guest Domain 396     vnet—Domain Virtual Network 399         Installing vnet 400         Running vnet 401     Summary 403     References and Further Reading 403 Chapter 11: Securing a Xen System 405     Structuring Your System for Security 406         Special Purpose Virtual Machines 406         Creating Virtual Network Segments 407     Securing the Privileged Domain 407         Removing Software and Services 407         Limiting Remote Access 408         Limiting the Local Users 412         Move Device Drivers into DriverDomains 412     Firewall and Network Monitors 413         Running a Firewall with iptables 413         Snort 419         Obtaining Snort 419         Snort and Network Intrusion Detection Mode 420     Mandatory Access Control with sHype and Xen Security Modules 422         sHype 423         Xen Security Modules (XSM) 432     DomU Security 433         Running VMs Only When Needed 434         Backing Up Virtual Machine Images 434     Summary 435     References and Further Reading 436 Chapter 12: Managing Guest Resources 437     Accessing Information about Guests and the Hypervisor 438         xm info 438         xm dmesg 443         xm log 444         xm top 446         xm uptime 449     Allocating Guest Memory 449         Shadow Page Tables 451         Balloon Driver 451         Improving Stability with Swap Space 454         Managing the Allocation of Guest Memory 454     Managing Guest Virtual CPUs 458         Comparing Virtual, Logical, and Physical Processors 458         HVM VCPU Management 459         VCPU Subcommands 460         When to Manually Administer VCPUs 462     Tuning the Hypervisor Scheduler 463         Weight and Cap 463         Protection from Misbehaving Guests 464         Using the Credit Scheduler Command 465     Choosing a Guest IO Scheduler 466         Noop Scheduler 466         Deadline Scheduler 466         Anticipatory Scheduler (as) 467         Complete Fair Queuing Scheduler (cfq) 467         Using IO Schedulers 467     Summary 469     References and Further Reading 469 Chapter 13: Guest Save, Restore, and Live Migration 471     Representing the State of a Virtual Machine 472     Basic Guest Domain Save and Restore 473         xm save 474         xm restore 476         Possible Save and Restore Errors 478     Types of Guest Relocation 479         Cold Static Relocation 480         Warm Static (Regular) Migration 481         Live Migration 482     Preparing for xm migrate 484         Configuring xend 485         Proximity of Sources and Destinations on the Network 488         Network-Accessible Storage 489         Guest Domain Configuration 489         Version and Physical Resource Requirements 491     Experience with xm migrate 491         xm migrate 491         Using xm migrate for Warm Static Migration 492         Using xm migrate for Live Migration 494         Possible Migration Errors 497     Summary 498     References and Further Reading 498 Chapter 14: An Overview of Xen Enterprise Management Tools 499     Programmatic Interfaces to the Xen Hypervisor 500         Libvirt 500         Xen–CIM 501         Xen API 501         Legacy Interfaces to Xend 502     Citrix XenServer Enterprise, Standard and XenExpress Editions 502     Virtual Iron 504     IBM Virtualization Manager 506     Enomalism 507     virt-manager 509     XenMan  513     Managing Multiple Systems 518     Summary 518     References and Further Reading 519 Appendix A: Resources 521     Xen Community 522     XenWiki 523     Xen Mailing Lists and Bug Reporting 524     Xen Summits 525     Xen Source Code 526     Academic Papers and Conferences 528     Distribution-Specific Resources 530 Appendix B: The xm Command  531 Appendix C: Xend Configuration Parameter  537 Appendix D: Guest Configuration Parameter 541 Appendix E: Xen Performance Evaluation 545     Xen Performance Measurements 546         Repeatability of the Xen Team’s Results 546         Xen and Virtual Web Hosting 548         Comparing XenoLinux to Native Linux on Older PC Hardware 550         Xen on x86 Versus IBM zServer 551     Performance Isolation in Xen 553     Performance of Xen Virtual Network and Real Network 556     Summary 558 Index 559  

About the Author :
Jeanna Matthews is an associate professor of Computer Science at Clarkson University (Potsdam, New York) where she leads several hands-on computing laboratories including the Clarkson Open Source Institute and Clarkson Internet Teaching Laboratory. Students in these labs and in her classes have been winners in a number of prestigious computing contests including the 2001, 2002, and 2004 IBM Linux Challenge, the 2005 IBM North American Grid Scholar’s Challenge, the 2005 Unisys Tuxmaster competition, and the 2006 VMware Ultimate Virtual Appliance Challenge. Her research interests include virtualization, operating systems, computer networks, and computer security. She is actively involved in the Association for Computing Machinery as treasurer of the Special Interest Group on Operating Systems, editor of Operating Systems Review, and is a member of the Executive Committee ACM’s U.S. Public Policy Committee, US-ACM. She is also the author of a computer networking textbook, Computer Networking: Internet Protocols in Action, that has been translated into several languages. Jeanna received her Ph.D. in Computer Science from the University of California at Berkeley in 1999. Eli M. Dow is a software engineer in IBM’s Test and Integration Center for Linux in Poughkeepsie, NY. He holds a B.S. degree in Computer Science and Psychology as well as an M.S. in Computer Science from Clarkson University. He is passionate about open source software and is an alumnus and founding member of the Clarkson Open Source Institute. His interests include virtualization, Linux systems programming, the GNOME desktop, and human-computer interaction. He is the author of numerous IBM developerWorks articles focused on Linux and open source software. Additionally, he has coauthored two books on the mainframe hypervisor z/VM, entitled Introduction to the New Mainframe: z/VM Basics and Linux for IBM System z9 and IBM zSeries. His first published experience with Xen was coauthoring an early academic paper entitled “Xen and the Art of Repeated Research.” Recently he has focused on developing highly available, enterprise customer solutions deployed on virtualized Linux using the z/VM hypervisor. Todd Deshane expects to obtain a Ph.D. in Engineering Science from Clarkson University in 2008. He also has a Master of Science in Computer Science and a Bachelor of Science in Software Engineering from Clarkson. While at Clarkson University, he has had a variety of research publications–many involving Xen. In 2005, a project that was based on Todd’s Master’s thesis–an open source collaborative, large database explorer–won first place in the Unisys TuxMaster competition. Todd’s primary academic and research interests are in the area of operating system technologies, such as virtual machine monitors, high availability, and file systems. His doctoral dissertation focuses on using these technologies to provide desktop users with an attack-resistant experience, with automatic and autonomic recovery from viruses, worms, and adverse system modifications. During his Ph.D. years, Todd has been a teaching assistant and an IBM Ph.D. Fellowship recipient. At IBM, Todd has worked on internship projects involving Xen and IBM technologies. Todd enjoys teaching, tutoring, and helping people. Wenjin Hu graduated from Clarkson University in 2007 with a Master’s degree of Computer Science and is currently working on his Ph.D. His Masters thesis was “A Study of the Performance Isolation Properties of Virtualization Systems.” His research field is applying virtualization techniques to operating systems and security. Jeremy Bongio is currently a Master’s student at Clarkson University. He won second place in the Unisys Tuxmaster competition in 2005 with a project called Xenophilia, an early effort to make Xen more user friendly. He is a current member and former student director of the Clarkson Open Source Institute, where he actively learns and experiments with different kinds of virtualization. Patrick F. Wilbur is currently pursuing graduate studies in Computer Science at Clarkson University. His interests include operating systems, systems and application security, natural language processing, and home automation. In his spare time, Patrick enjoys composing music, experimenting with amateur radio, storm chasing, and working on various electronics, software, and carpentry projects around the house. He is currently a member of the Clarkson Open Source Institute, a volunteer at the Applied Computer Science Laboratories at Clarkson University, an emergency communications volunteer, and a member of the Association for Computing Machinery. Brendan Johnson graduated from Clarkson University in 2002 with a Bachelor’s degree in Computer Science and a minor in Mathematics. Brendan continued his education at Clarkson University and obtained a Master’s of Science in Computer Science with a thesis in quantum computing. Brendan is currently a senior software architect at Mobile Armor, a world leading “Data At Rest” encryption software company.