hadoop-examples-0.20.2-cdh3u6.jar wordcount 例子运行出现的问题记录

1.hadoop集群已经能够运行，报告正常，在之前的文章中记录了搭建和启动过程。
2.准备测试数据

   [hadoop@master bin]$ hadoop fs -ls /
   [hadoop@master bin]$ hadoop fs -mkdir /input
   [hadoop@master bin]$ hadoop fs -put /hdata/testlog/SystemOut_13.06.30_13.00.00.log /input
   [hadoop@master bin]$ hadoop fs -ls input

3.运行wordcount

[hadoop@master bin]$ hadoop jar ../hadoop-examples-0.20.2-cdh3u6.jar wordcount input output

4.运行完查看结果 output 存在hdfs文件系统中

[hadoop@master bin]$ hadoop fs -ls output/part-r-00000
[hadoop@master bin]$ hadoop fs –cat /tmp/output/part –r -00000

[hadoop@master bin]$ hadoop jar ../hadoop-examples-0.20.2-cdh3u6.jar wordcount input output
13/07/03 11:38:13 INFO mapred.JobClient: Cleaning up the staging area hdfs://10.132.30.48:9000/hdata/hdfs/tmp/mapred/staging/hadoop/.staging/job_201307030829_0001
13/07/03 11:38:13 ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://10.132.30.48:9000/user/hadoop/input
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://10.132.30.48:9000/user/hadoop/input
        at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:231)
        at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:248)
        at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:998)
        at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1015)
        at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:928)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:881)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:881)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:526)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:556)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:197)

错误原因，input为相对路径，所以目录应该带上根路径。

[hadoop@master bin]$ hadoop jar ../hadoop-examples-0.20.2-cdh3u6.jar wordcount input output
	
13/07/03 11:39:21 INFO input.FileInputFormat: Total input paths to process : 2
13/07/03 11:39:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/03 11:39:21 WARN snappy.LoadSnappy: Snappy native library not loaded
13/07/03 11:39:23 INFO mapred.JobClient: Running job: job_201307030829_0002
13/07/03 11:39:24 INFO mapred.JobClient:  map 0% reduce 0%
13/07/03 11:39:33 INFO mapred.JobClient:  map 50% reduce 0%
13/07/03 11:39:34 INFO mapred.JobClient:  map 100% reduce 0%
13/07/03 11:39:49 INFO mapred.JobClient:  map 100% reduce 100%
13/07/03 11:39:50 INFO mapred.JobClient: Job complete: job_201307030829_0002
13/07/03 11:39:50 INFO mapred.JobClient: Counters: 27
13/07/03 11:39:50 INFO mapred.JobClient:   Job Counters 
13/07/03 11:39:50 INFO mapred.JobClient:     Launched reduce tasks=1
13/07/03 11:39:50 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=11586
13/07/03 11:39:50 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/07/03 11:39:50 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/07/03 11:39:50 INFO mapred.JobClient:     Rack-local map tasks=1
13/07/03 11:39:50 INFO mapred.JobClient:     Launched map tasks=2
13/07/03 11:39:50 INFO mapred.JobClient:     Data-local map tasks=1
13/07/03 11:39:50 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=16140
13/07/03 11:39:50 INFO mapred.JobClient:   FileSystemCounters
13/07/03 11:39:50 INFO mapred.JobClient:     FILE_BYTES_READ=113384502
13/07/03 11:39:50 INFO mapred.JobClient:     HDFS_BYTES_READ=33019383
13/07/03 11:39:50 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=170356594
13/07/03 11:39:50 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=55105594
13/07/03 11:39:50 INFO mapred.JobClient:   Map-Reduce Framework
13/07/03 11:39:50 INFO mapred.JobClient:     Map input records=281310
13/07/03 11:39:50 INFO mapred.JobClient:     Reduce shuffle bytes=56808072
13/07/03 11:39:50 INFO mapred.JobClient:     Spilled Records=879820
13/07/03 11:39:50 INFO mapred.JobClient:     Map output bytes=56089625
13/07/03 11:39:50 INFO mapred.JobClient:     Total committed heap usage (bytes)=381755392
13/07/03 11:39:50 INFO mapred.JobClient:     CPU time spent (ms)=6600
13/07/03 11:39:50 INFO mapred.JobClient:     Combine input records=299960
13/07/03 11:39:50 INFO mapred.JobClient:     SPLIT_RAW_BYTES=224
13/07/03 11:39:50 INFO mapred.JobClient:     Reduce input records=294734
13/07/03 11:39:50 INFO mapred.JobClient:     Reduce input groups=294607
13/07/03 11:39:50 INFO mapred.JobClient:     Combine output records=294734
13/07/03 11:39:50 INFO mapred.JobClient:     Physical memory (bytes) snapshot=379052032
13/07/03 11:39:50 INFO mapred.JobClient:     Reduce output records=294607
13/07/03 11:39:50 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1029636096
13/07/03 11:39:50 INFO mapred.JobClient:     Map output records=299960