under replicated blocks 解决办法

贺季
2023-12-01

ambari 监控页面显示under replicated blocks 显示红色。查了一下发现可以用hadoop fs 的 setrep命令解决这个问题。

下面是运行 hadoop fs -help 时,对这个命令的解释

-setrep [-R] [-w] <rep> <path> ... :
  Set the replication level of a file. If <path> is a directory then the command
  recursively changes the replication factor of all files under the directory tree
  rooted at <path>.

  -w  It requests that the command waits for the replication to complete. This   
      can potentially take a very long time.                                     
  -R  It is accepted for backwards compatibility. It has no effect.

因为我们的测试集群只有两个结点datanode ,所以需要先把dfs.replication 修改成2, 默认是3。
因为我们集群从搭建好都没改过这个配置,所以所有的文件都是需要备份3份,但实际只能复制2份,所有对所有文件都要进行这个操作。
然后执行下面命令,监控页面的监控就没有警告了

hadoop  fs -setrep -R -w 2 /

如果只是某些文件出这个问题,可以指定路径:

hadoop dfs -setrep -w 3 /path/to/file

https://community.hortonworks.com/questions/18819/how-to-fix-missing-and-under-replicated-blocks.html

 类似资料: