當前位置:編程學習大全網 - 編程語言 - 在Windows上運行Hadoop,配置好java,cygwin,hadoop之後,要怎麽做

在Windows上運行Hadoop,配置好java,cygwin,hadoop之後,要怎麽做

環境及軟件準備:

win7(64位)

cygwin 1.7.9-1

jdk-6u25-windows-x64.zip

hadoop-0.20.2.tar.gz

1.安裝jdk,並置java環境變量包括:JAVA_HOME,PATH,CLASSPATH

2.安裝Hadoop,版本為0.20.2,我是直接放到/home目錄下,並解壓

tar –zxvf

hadoop-0.20.2.tar.gz

3.配置Hadoop,需要修改hadoop的配置文件,它們位於conf子目錄下,分別是hadoop-env.sh、core-site.xml、hdfs-site.xml

和mapred-site.xml

(1) 修改hadoop-env.sh:

只需要將JAVA_HOME 修改成JDK 的安裝目錄即可

export

JAVA_HOME=/cygdrive/d/java/jdk1.6.0_25

(註意:路徑不能是windows 風格的目錄d:\java\jdk1.6.0_25,而是LINUX

風格/cygdrive/d/java/jdk1.6.0_25)

(2) 修改core-site.xml:(指定namenode)

<configuration>

<property>

<name>fs.default.name</name>

<value>hdfs://localhost:9000</value>

</property>

</configuration>

(3)修改hdfs-site.xml(指定副本為1)

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

</configuration>

(4) 修改mapred-site.xml (指定jobtracker)

<configuration>

<property>

<name>mapred.job.tracker</name>

<value>localhost:9001</value>

</property>

</configuration>

4.驗證安裝是否成功,並運行Hadoop

(1) 驗證安裝

$ bin/hadoop

Usage: hadoop [--config confdir] COMMAND

where COMMAND is one of:

namenode -format format the DFS filesystem

secondarynamenode run the DFS secondary namenode

namenode run the DFS namenode

datanode run a DFS datanode

dfsadmin run a DFS admin client

mradmin run a Map-Reduce admin client

fsck run a DFS filesystem checking utility

fs run a generic filesystem user client

balancer run a cluster balancing utility

jobtracker run the MapReduce job Tracker node

pipes run a Pipes job

tasktracker run a MapReduce task Tracker node

job manipulate MapReduce jobs

queue get information regarding JobQueues

version print the version

jar <jar> run a jar file

distcp <srcurl> <desturl> copy file or directories recursively

archive -archiveName NAME <src>* <dest> create a hadoop archive

daemonlog get/set the log level for each daemon

or

CLASSNAME run the class named CLASSNAME

Most commands print help when invoked w/o parameters.

(2) 格式化並啟動Hadoop

bi

  • 上一篇:大山寫作文
  • 下一篇:修飾編程
  • copyright 2024編程學習大全網