Apache Hadoop YARN

This video lesson explains Hadoop YARN with a PI estimator example.

Hadoop YARNMapreducetutorial
1.0x

Apache Hadoop YARN

Created 2 years ago

Duration 0:33:08
lesson view count 65
This video lesson explains Hadoop YARN with a PI estimator example.
Select the file type you wish to download
Slide Content
  1. Apache Hadoop YARN

    Slide 1 - Apache Hadoop YARN

    • Review of MapReduce
    • Simple input data sets
    • map() and reduce() functions
    • HDFS for scalability, availability and data replication
    • Embarrassingly parallel computation
    • Hadoop
    • Open source project by ASF
    • Dean, Jeffrey, and Sanjay Ghemawat. "MapReduce: simplified data processing on large clusters." Communications of the ACM 51.1 (2008): 107-113.
  2. Apache Hadoop YARN

    Slide 2 - Apache Hadoop YARN

    • Yet Another Resource negotiator (YARN)
    • MapReduce Version 2 (MRv2)
  3. Hadoop V1

    Slide 3 - Hadoop V1

    • JobTracker (head node)
    • TaskTrackers (slaves)
    • Master
    • Slave 1
    • Slave 2
    • Slave n
    • Client(User)
    • JobTracker
    • TaskTracker
    • TaskTracker
    • TaskTracker
    • Job #1
    • Task 1
    • of
    • Job #1
    • Task 2
    • of
    • Job #1
    • Task 3
    • of
    • Job #1
    • Task n - 1
    • of
    • Job #1
    • Task n
    • of
    • Job #1
  4. Hadoop V2 YARN

    Slide 4 - Hadoop V2 YARN

    • ResourceManager(RM)
    • ApplicationMaster(AM)
    • NodeManager(NM)
    • Master
    • Slave 1
    • Slave 2
    • Slave n
    • Client(User)
    • Resource
    • Manager (RM)
    • Job #1
    • Task 1
    • of
    • Job #1
    • Task 2
    • of
    • Job #1
    • Task 3
    • of
    • Job #1
    • Task n - 1
    • of
    • Job #1
    • Task n
    • of
    • Job #1
    • Application
    • Master (AM)
    • Application
    • Master (AM)
    • Container
    • Container
    • Container
    • NodeManager(NM)
    • NodeManager(NM)
    • NodeManager(NM)
  5. Hadoop V2 YARN

    Slide 5 - Hadoop V2 YARN

    • Scalability
    • Master
    • Slave 1
    • Slave 2
    • Slave n
    • Client(User)
    • Resource
    • Manager (RM)
    • Job #1
    • Task 1
    • of
    • Job #1
    • Task 2
    • of
    • Job #1
    • Task 3
    • of
    • Job #1
    • Task n - 1
    • of
    • Job #1
    • Task n
    • of
    • Job #1
    • Application
    • Master (AM)
    • Application
    • Master (AM)
    • Container
    • Container
    • Container
    • NodeManager(NM)
    • NodeManager(NM)
    • NodeManager(NM)
  6. Slide 6

    • Hadoop V2 YARN
    • Walkthrough
    • Master
    • Slave 1
    • Slave 2
    • Slave n
    • Client(User)
    • Resource
    • Manager (RM)
    • Job
    • Task 1
    • of
    • Job #1
    • Task 2
    • of
    • Job #1
    • Task 3
    • of
    • Job #1
    • Task n - 1
    • of
    • Job #1
    • Task n
    • of
    • Job #1
    • Application
    • Master (AM)
    • Container
    • Container
    • Container
    • NodeManager(NM)
    • NodeManager(NM)
    • NodeManager(NM)
    • Hadoop YARN Cluster
  7. MapReduce Examples (PI Estimator)

    Slide 7 - MapReduce Examples (PI Estimator)