配置hadoop 環境變量 eclipse
a.在master節點上(ubuntu-V01)修改hdfs-site.xml加上以下內容?
<property>?
<name>dfs.permissions</name>?
<value>false</value>?
</property>?
旨在取消權限檢查，原因是為了解決我在windows機器上配置eclipse連接hadoop服務器時，配置map/reduce連接後報以下錯誤，org.apache.hadoop.security.AccessControlException: Permission denied:?
b.同樣在master節點上(ubuntu-V01)修改hdfs-site.xml加上以下內容?
<property>?
<name>dfs.web.ugi</name>?
<value>jack,supergroup</value>?
</property>?
原因是運行時，報如下錯誤 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user jack?
應該是我的windows的用戶名為jack，無訪問權限?
更多權限配置可參看官方說明文檔：?
HDFS權限管理用戶指南/hdfs_permissions_guide.html?
配置修改完後重啟hadoop集群：?
hadoop@ubuntu-V01:~/data$./sbin/stop-dfs.sh?
hadoop@ubuntu-V01:~/data$./sbin/stop-yarn.sh?
hadoop@ubuntu-V01:~/data$./sbin/start-dfs.sh?
hadoop@ubuntu-V01:~/data$./sbin/start-yarn.sh?
二.windows基礎環境準備?
windows7(x64),jdk,ant,eclipse,hadoop?
1.jdk環境配置?
jdk-6u26-windows-i586.exe安裝後好後配置相關JAVA_HOME環境變量,並將bin目錄配置到path?
2.eclipse環境配置?
eclipse-standard-luna-SR1-win32.zip解壓到D:\eclipse\目錄下並命名eclipse-hadoop?
下載地址：/apache//ant/binaries/apache-ant-1.9.4-bin.zip?
4.下載hadoop-2.5.2.tar.gz?
/apache/hadoop/common/hadoop-2.5.2/hadoop-2.5.2.tar.gz?
5.下載hadoop-2.5.2-src.tar.gz?
/apache/hadoop/common/hadoop-2.5.2/hadoop-2.5.2-src.tar.gz?
6.下載hadoop2x-eclipse-plugin?
/winghc/hadoop2x-eclipse-plugin?
7.下載hadoop-common-2.2.0-bin?
/srccodes/hadoop-common-2.2.0-bin?
分別將hadoop-2.5.2.tar.gz、hadoop-2.5.2-src.tar.gz、hadoop2x-eclipse-plugin、hadoop-common-2.2.0-bin下載解壓到F:\hadoop\目錄下?
8.修改本地hosts文件，加入如下內容：
192.168.1.112 ubuntu-V01
三、編譯hadoop-eclipse-plugin-2.5.2.jar配置?
1.添加環境變量HADOOP_HOME=F:\hadoop\hadoop-2.5.2\?
追加環境變量path內容：%HADOOP_HOME%/bin?
2.修改編譯包及依賴包版本信息?
修改F:\hadoop\hadoop2x-eclipse-plugin-master\ivy\libraries.properties?
hadoop.version=2.5.2?
jackson.version=1.9.13?
3.ant編譯?
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>?
ant jar -Dversion=2.5.2 -Declipse.home=D:\eclipse\eclipse-hadoop\eclipse -Dhadoop.home=F:\hadoop\hadoop-2.5.2?
編譯好後hadoop-eclipse-plugin-2.5.2.jar會在F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\eclipse-plugin目錄下?
四、eclipse環境配置?
1.將編譯好的hadoop-eclipse-plugin-2.5.2.jar拷貝至eclipse的plugins目錄下，然後重啟eclipse?
2.打開菜單Window--Preference--Hadoop Map/Reduce進行配置，如下圖所示：?
3.顯示Hadoop連接配置窗口：Window--Show View--Other-MapReduce Tools,如下圖所示：?
4.配置連接Hadoop，如下圖所示：?
查看是否連接成功，能看到如下信息，則表示連接成功：?
五、hadoop集群環境添加測試文件
(如果已有則無需配置)?
a.dfs上創建input目錄?
hadoop@ubuntu-V01:~/data/hadoop-2.5.2$bin/hadoop fs -mkdir -p input?
b.把hadoop目錄下的README.txt拷貝到dfs新建的input裏?
hadoop@ubuntu-V01:~/data/hadoop-2.5.2$bin/hadoop fs -copyFromLocal README.txt input?
六、創建壹個Map/Reduce Project?
1.新建項目 File--New--Other--Map/Reduce Project 命名為MR1,?
然後創建類org.apache.hadoop.examples.WordCount，從hadoop-2.5.2-src中拷貝覆蓋?
（F:\hadoop\hadoop-2.5.2-src\hadoop-mapreduce-project\hadoop-mapreduce-examples\src\main\java\org\apache\hadoop\examples\WordCount.java）?
2.創建log4j.properties文件?
在src目錄下創建log4j.properties文件，內容如下：?
log4j.rootLogger=debug,stdout,R?
log4j.appender.stdout=org.apache.log4j.ConsoleAppender?
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout?
log4j.appender.stdout.layout.ConversionPattern=%5p - %m%n?
log4j.appender.R=org.apache.log4j.RollingFileAppender?
log4j.appender.R.File=mapreduce_test.log?
log4j.appender.R.MaxFileSize=1MB?
log4j.appender.R.MaxBackupIndex=1?
log4j.appender.R.layout=org.apache.log4j.PatternLayout?
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n?
log4j.logger.com.codefutures=DEBUG?
3.解決java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)異常問題?
(由於妳的環境和我的可能不壹致，可以在後面出現相關問題後再進行修改)?
拷貝源碼文件org.apache.hadoop.io.nativeio.NativeIO到項目中?
然後定位到570行，直接修改為return true;?
如下圖所示：?
七、windows下運行環境配置
(如果不生效，則需要重啟機器)?
需要hadoop.dll,winutils.exe?
我是直接拷貝F:\hadoop\hadoop-common-2.2.0-bin-master\bin目錄下內容覆蓋F:\hadoop\hadoop-2.5.2\bin?
八、運行project
在eclipse中點擊WordCount.java，右鍵，點擊Run As—>Run Configurations，配置運行參數，即輸入和輸出文件夾?
hdfs://ubuntu-V01:9000/user/hadoop/input hdfs://ubuntu-V01:9000/user/hadoop/output?
如下圖所示：?