MultipleOutputFormat-Hadoop

Question

我是一个有点新的地图缩小，所以如果任何人可以指导我与下面的问题，这将是伟大的

>

我在map Reduce中使用了多输出格式来写入分离输出文件。让我们假设我的输入文件有水果和蔬菜，因此把它分成两个文件。水果和蔬菜如下。

水果-R-00000，蔬菜-R-00000，部分-R-00000

我搞不清有多少减速器会运转？我知道，默认情况下，减速器的数量被设置为1，由于文件名的数字部分是相同的，我相信只有一个减速器运行。我的理解正确吗？另外，为什么要创建part-r-00000文件？我把所有的输出都写在水果文件或蔬菜文件中。

如果我有1 GB的数据要处理，我如何决定要使用的最优减速器数？

Answer 1

one reducer will run ,it has nothing to do with part of file name , no of reducer would be either specified by the user by default it calculated the size of the input file and amount of work which need to be done in reducers .

part-r-00000 : This is related with partitioning, Since we have one reducer so all partitions will point to this file 

Number of reduces in most cases specified by users. It mostly depends on amount of work, which need to be done in reducers. But their number should not be very big, because of algorithm, used by Mapper to distribute data among reducers. Some frameworks, like Hive can calculate number of reducers using empirical 1GB output per reducer.