我的MapReduce程序出错了，我想在那里收集多年来的最高温度

Question

当我编译hadoop命令时，它最终出现以下错误：“java.io.ioException：Type mismatch in key from map：expected org.apache.hadoop.io.text,received org.apache.hadoop.io.longwritable”

我将数据类型从Text更改为LongWritable，在这种情况下，我会得到其他数据类型不匹配。

主类：

public class CalculateMaximum {
    public static void main(String [] args) throws IllegalArgumentException, IOException, ClassNotFoundException, InterruptedException{

    Configuration config  = new Configuration();
    Job job = new Job(config);
    job.setJarByClass(CalculateMaximum.class);
    job.setMapperClass(CalculateMapper.class);
    job.setNumReduceTasks(1);
    job.setReducerClass(CalReducer.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));
    FileSystem fs = FileSystem.get(config);
    fs.delete(new Path(args[1]));
    job.waitForCompletion(true);
}
}

映射器类：

public class CalculateMapper extends Mapper<LongWritable,Text,Text,IntWritable> {

    public void cal(LongWritable key,Text values,Context context) throws IOException, InterruptedException{
        String row = values.toString();
        String []r1 = row.split(" ");

        //Integer year = Integer.parseInt(row[0]);
        Text yr  = new Text(r1[0]);
        Integer temp = Integer.parseInt(r1[1]);
        IntWritable tp = new IntWritable(temp);
        context.write(yr, tp);
        //context.write(yr, tp);


    }
}

减速器类：

public class CalReducer extends Reducer<Text,Iterable<IntWritable>,Text,IntWritable> {

    public void cal(Text key,Iterable<IntWritable> values,Context context) throws IOException, InterruptedException{

        //Iterable<IntWritable> tmps = values;

        //int temp  = tmps.get();

        int max = 0;
        for(IntWritable temp : values){
            if(temp.get() > max){
                max= temp.get();
            }
        context.write(key, new IntWritable(max));   

        }
            }
}

1900 39
1900 14
1900 5
1900 11
1901 32
1901 40
1901 29
1901 48

1900 39
1901 48

Answer 1

我相信键和值都是int类型。您能尝试使用IntWritable for键吗？