Apache Hadoop YARN: Moving beyond MapReduce and Batch Processing with Apache Hadoop 2

The full text downloaded to your computer

With eBooks you can:

• search for key concepts, words and phrases
• make highlights and notes as you study
• share your notes with friends

eBooks are downloaded to your computer and accessible either offline through the Bookshelf (available as a free download), available online and also via the iPad and Android apps.

Upon purchase, you'll gain instant access to this eBook.

Time limit

The eBooks products do not have an expiry date. You will continue to access your digital ebook products whilst you have your Bookshelf installed.

In Apache Hadoop YARN, key YARN developer Arun Murthy shows how to get existing code to run on Apache Hadoop 2, and develop new applications that take absolutely full advantage of Hadoop clusters. Drawing on insights from the entire Apache Hadoop 2 team, Murthy and Dr. Douglas Eadline review Apache Hadoop YARN's goals, design, architecture, and components, guide the reader thrugh migrating existing MapReduce applications, identify the functional requirements for each element of an Apache Hadoop 2 application, walk the reader through a sample appliation project, and offer multiple examples and case studies drawn from their cutting-edge experience.

• Chapter 1: Apache Hadoop YARN: A Brief History and Rationale
• Chapter 2: Apache Hadoop YARN Install Quick Start
• Chapter 3: Apache Hadoop YARN Core Concepts
• Chapter 4: Functional Overview of YARN Components
• Chapter 5: Installing Apache Hadoop YARN
• Chapter 6: Apache Hadoop YARN Administration
• Chapter 7: Apache Hadoop YARN Architecture Guide
• Chapter 8: Capacity Scheduler in YARN
• Chapter 9: MapReduce with Apache Hadoop YARN
• Chapter 10: Apache Hadoop YARN Application Example
• Chapter 11: Using Apache Hadoop YARN Distributed-Shell
• Chapter 12: Apache Hadoop YARN Frameworks
• Appendix A: Supplemental Content and Code Downloads
• Appendix B: YARN Installation Scripts
• Appendix C: YARN Administration Scripts
• Appendix D: Nagios Modules
• Appendix E: Resources and Additional Information
• Appendix F: HDFS Quick Reference
•  Index

Arun Murthy has contributed to Apache Hadoop full-time since the inception of the project in early 2006. He is a long-term Hadoop committer and a member of the Apache Hadoop Project Management Committee. Previously, he was the architect and lead of the Yahoo Hadoop MapReduce development team and was ultimately responsible, technically, for providing Hadoop MapReduce as a service for all of Yahoo--currently running on nearly 50,000 machines. Arun is the founder and architect of the Hortonworks Inc., a software company that is helping to accelerate the development and adoption of Apache Hadoop. Hortonworks was formed by the key architects and core Hadoop committers from the Yahoo! Hadoop software engineering team in June 2011. Funded by Yahoo! and Benchmark Capital, one of the preeminent technology investors, their goal is to ensure that Apache Hadoop becomes the standard platform for storing, processing, managing, and analysing big data.

Vinod Kumar Vavilapalli has been contributing to Apache Hadoop project full-time since mid-2007. At Apache Software Foundation, he is a long-term Hadoop contributor, Hadoop committer, member of the Apache Hadoop Project Management Committee, and a foundation member. Vinod is a MapReduce and YARN go-to guy at Hortonworks Inc. For more than five years, he has been working on Hadoop. He was involved in HadoopOnDemand, Hadoop-0.20, CapacityScheduler, Hadoop security, and MapReduce, and is now a lead developer and the project lead for Apache Hadoop YARN. Before Hortonworks, he was at Yahoo!, working in the Grid team that made Hadoop what it is today, running at large scale--up to tens of thousands of nodes. Vinod loves reading books of all kinds and is passionate about using computers to change the world for better, bit by bit. He has a bachelor’s degree in computer science and engineering from the Indian Institute of Technology Roorkee.

Douglas Eadline, Ph.D., began his career as a practitioner and a chronicler of the Linux Cluster HPC revolution and now documents big data analytics. Starting with the first Beowulf How To document, Doug has written hundreds of articles, white papers, and instructional documents covering virtually all aspects of HPC computing. Prior to starting and editing the popular ClusterMonkey.net website in 2005, he served as editor­-in-­chief for ClusterWorld magazine, and was senior HPC editor for Linux Magazine. Currently, he is a consultant to the HPC industry and writes a monthly column in HPC Admin magazine. Both clients and readers have recognised Doug’s ability to present a “technological value proposition” in a clear and accurate style. He has practical, hands-on experience in many aspects of HPC, including hardware and software design, benchmarking, storage, GPU, cloud, and parallel computing.

Joseph Niemiec is a big data solutions engineer whose focus is on designing Hadoop solutions for many Fortune 1000 companies. In this position, Joseph has worked with customers to build multiple YARN applications providing a unique perspective on moving customers beyond batch processing, and has worked on YARN development directly. An avid technologist, Joseph has been focused on technology innovations since 2001. His interest in data analytics originally started in game score optimisation as a teenager, and has shifted to helping customers uptake new technology innovations such as Hadoop and, most recently, building new data applications using YARN.

Jeff Markham is a solution engineer at Hortonworks Inc., the company promoting open source Hadoop. Previously, he was with VMware, Red Hat, and IBM, helping companies