HDP Developer Custom Apache YARN Applications


This course is designed for developers who want to create custom YARN applications for Apache Hadoop. It will include: the YARN architecture, YARN development steps, writing a YARN client and ApplicationMaster, and launching Containers. The course uses Eclipse and Gradle connected remotely to a 7-node HDP cluster running in a virtual machine.

Target Audience

Java software engineers who need to develop YARN applications on Hadoop by writing YARN clients and ApplicationMasters


Students should be experienced Java developers who have attended HDP Developer: Java OR HDP Developer: Pig and Hive OR are experienced with Hadoop and MapReduce development.

Course Objectives

At the completion of the course students will be able to:
  • Describe the YARN architecture
  • Describe the YARN application lifecycle
  • Write a YARN client application
  • Run a YARN application on a Hadoop cluster
  • Monitor the status of a running YARN application
  • View the aggregated logs of a YARN application
  • Configure a ContainerLaunchContext
  • Use a LocalResource to share application files across a cluster
  • Write a YARN ApplicationMaster
  • Describe the differences between synchronous and asynchronous ApplicationMasters
  • Allocate Containers in a cluster
  • Launch Containers on NodeManagers
  • Write a custom Container to perform specific business logic
  • Explain the job schedulers of the ResourceManager
  • Define queues for the Capacity Scheduler

50% Lecture/Discussion
50% Hands-on Labs

Course Outline

Hands-On Labs
  • Run a YARN Application
  • Setup a YARN Development Environment
  • Write a YARN Client
  • Submit an ApplicationMaster
  • Write an ApplicationMaster
  • Requesting Containers
  • Running Containers
  • Writing Custom Containers

SLI Main Menu