目录
初识Hadoop ...................................................................................................................................... 2
Hadoop简介 .............................................................................................................................. 2 Hadoop相关名词 ...................................................................................................................... 2 Hadoop的架构 .......................................................................................................................... 2
HDFS .................................................................................................................................. 3 MapReduce ....................................................................................................................... 3 分布式并行运算 ............................................................................................................... 5
MapReduce基本编程 ....................................................................................................................... 6
概述........................................................................................................................................... 6 实践........................................................................................................................................... 6
创建一个maven工程 ........................................................................................................ 6 加入hadoop依赖 .............................................................................................................. 6 编写Map类 ....................................................................................................................... 7 编写Reduce类 ................................................................................................................... 8 定义job ............................................................................................................................. 8
MapReduce测试 ............................................................................................................................... 9
用MRUnit做单元测试 .............................................................................................................. 9
加入mrunit依赖 ................................................................................................................ 9 单独测试Map ................................................................................................................. 10 单独测试Reduce ............................................................................................................. 10 测试MapReduce ............................................................................................................. 11 运行MapReduce Job进行集成测试 ....................................................................................... 13
准备工作 ......................................................................................................................... 13 单机运行测试 ................................................................................................................. 13 伪分布式运行测试 ......................................................................................................... 14 完全分布式运行测试 ..................................................................................................... 15 集成测试总结 ................................................................................................................. 15
使用Eclipse插件 ............................................................................................................................. 16
插件安装 ................................................................................................................................. 16 插件配置与使用 ..................................................................................................................... 16
指定Hadoop安装目录 .................................................................................................... 16 打开Map/Reduce视图。 ................................................................................................ 17 新建 Hadoop location .................................................................................................... 17 DFS视图浏览文件 ........................................................................................................... 18 运行MapReduce作业 ..................................................................................................... 18 查看运行结果 ................................................................................................................. 18
Hadoop学习笔记一 简要介... 5页 免费 Hadoop学习笔记_yedu 19页 免费 ... Paas - 平台及服务,如 Hadoop 4: 什么是云计算: 更大限度的使用网络资源。...
Hadoop学习笔记一 简要介... 5页 免费 Hadoop学习笔记_yedu 19页 免费 ...hadoop 学习笔记 倒排索引简介倒排索引(英语:Inverted index),也常被称为反向...
Hadoop学习笔记一 简要介... 5页 免费 Hadoop学习笔记_yedu 19页 免费 ...Hadoop 学习笔记(2) --- mapreduce 剖析 1 数据类型 1.1 基本数据类型 ? ...
Hadoop学习笔记 7页 免费 Hadoop学习笔记 39页 1下载券 hadoop-1.0.0学习笔记 6页 2下载券 Hadoop学习笔记一 简要介... 5页 免费 Hadoop学习笔记_yedu 19...
hadoop学习笔记_计算机软件及应用_IT/计算机_专业资料。该文档介绍了三方面的... Hadoop学习笔记_yedu 19页 免费 hadoop介绍 24页 免费 Hadoop学习笔记 7页 ...
Hadoop 学习第一章 初识 Hadoop 分布式计算平台 —核心—>HDFS(分布式文件系统)... Hadoop学习笔记_yedu 19页 免费 Hadoop的最新进展 32页 免费 hadoop_介绍 ...
Hadoop学习笔记一 简要介绍... 5页 免费 Hadoop学习笔记_yedu 19页 免费 Hadoop_Ubuntu学习笔记 34页 20财富值 Hadoop Hive入门学习笔记 3页 免费...
Hadoop:The Definitive Guid 学习笔记 mingyuan Email:cn.mingyuan@foxmail.com ... Hadoop学习笔记_yedu 19页 免费 Hadoop的最新进展 32页 免费 hadoop_介绍 ...
Hadoop学习笔记整理_日语学习_外语学习_教育专区。Hadoop 学习笔记整理 hadoop ... Hadoop学习笔记一 简要介... 5页 免费 Hadoop学习笔记_yedu 19页 免费 ...
Hadoop学习笔记_计算机软件及应用_IT/计算机_专业资料 暂无评价0人阅读0次下载举报文档 Hadoop学习笔记_计算机软件及应用_IT/计算机_专业资料。hadoop,大数据,...
热门文档