Task!2:!Elastic!MapReduce!(EMR)!Basics
Let’s now try and use Amazon to run a simple Hadoop work-flow.
Select ‘Elastic MapReduce’ from the ‘Analytics’ options on the welcome page.
Your are presented with a screen which allows you to create new clusters of EC2 nodes automatically. These clusters are ephemeral – that is they will exist only for the duration of the Hadoop programs execution.
In order to have Amazon EMR run your program you must first configure it to do so.
Select ‘Create Cluster’.
To the right of the label 'Cluster Configuration' there is a button labelled 'Configure sample application'
o Click on this button
o Select 'Word count' from the 'Select sample application' drop down menu. o As the output enter
s3n://comp5349/user/your-login/word_count o As the logging enter
s3n://comp5349/user/your-login/word_count_logging o Click 'OK'


Figure 2.
The next step is to select the data input location for the word count job.
o Scroll to the bottom of the page.
o Click on the pencil icon symbolizing the edit control for the program step. o You'll see an image as shown in Figure 3.
o You can choose to either edit the input location to another file (such as place.txt) or leave it as this
sample file.
o Once you've made your choice click 'Create cluster'
If everything went OK you will be redirected to the cluster status page. Here you can see a range of metrics about the status of the cluster. ‘Refresh’ the top right corner of the wizard’s window.
This should open you a task list with your new ‘EMR Test 1’ being shown as ‘STARTING’. Select this entry, and you will see further details in a new panel at the bottom of the window.
COMP5349_2014 Semester1_week3_lab_week3handout_电子/电路_工程科技_专业资料。University of Sydney, COMP5349_Cloud Computing_2014 Semester1...
COMP5349_2014 Semester1_week13_lecture_13_Review_电脑基础知识_IT/计算机_专业资料。University of Sydney, COMP5349_Cloud Computing_2014 Semester1...
COMP5349_2014 Semester1_week7_lecture_07_DataflowSystems_电脑基础知识_IT/计算机_专业资料。University of Sydney, COMP5349_Cloud Computing_2014 Semester1...
COMP5349_2014 Semester1_week3_lecture_03_gfs_hdfs_电脑基础知识_IT/计算机_专业资料。University of Sydney, COMP5349_Cloud Computing_2014 Semester1...
COMP5349_2014 Semester1_week8_lecture_08_NoSQL_Cloud_Storage__KeyValue_Stores_电脑基础知识_IT/计算机_专业资料。University of Sydney, COMP5349_Cloud ...
COMP5349_2014 Semester1_week6_lecture_06_DataAnalytics_电脑基础知识_IT/计算机_专业资料。University of Sydney, COMP5349_Cloud Computing_2014 Semester1...
University of Sydney, COMP5349_Cloud Computing_2014 Semester1COMP5349 – Cloud Computing Week 12: Cloud Computing Security and Data Privacy A/Prof Uwe R?...
University of Sydney, COMP5349_Cloud Computing_2014 Semester1COMP5349 – Cloud Computing Week 5: More Hadoop Features A/Prof Uwe Roehm School of ...
COMP5349_2014 Semester1_week4_lecture_04_MapReduce_Hadoop_电脑基础知识_IT/计算机_专业资料。University of Sydney, COMP5349_Cloud Computing_2014 Semester1...
COMP5138_Database Management System_2014 Semester1_week1_00_Organisation_电脑基础知识_IT/计算机_专业资料。University of Sydney_COMP5138_Database Management ...
热门文档