Apache Hadoop 2.7.3，套接字超时错误

Question

我有相同的问题，在下面的链接。

Hadoop，套接字超时错误

你能帮我解决这个问题吗，我在Apache Hadoop 2.7.3 EC2安装上也有同样的问题。链接中提到的属性是否需要添加到名称和数据节点配置文件中？如果是，什么都是. xmls？提前谢谢。

此外，应用程序正在尝试访问EC2上的内部ip，错误如下，是否需要打开任何端口？在Web UI上显示8042。

显示所有节点以及节点管理器和资源管理器（RM）都在jps上运行。

尝试运行map reduce示例时，Namenode出现以下错误：

作业job_1506038808044_0002失败，状态FAILED，原因是：应用程序application_1506038808044_0002失败2次，原因是启动错误appattempt_1506038808044_0002_000002。获得异常：org.apache.hadoop.net.ConnectTimeoutException：从ip-172-31-1-10/172.31.1.10到ip-172-31-5-59.ec2.internal:43555的调用在套接字超时异常时失败：org.apache.hadoop.net.ConnectTimeoutException：等待通道准备好连接时20000毫秒超时。ch：java.nio.channels.SocketChannel[连接挂起的远程=ip-172-31-5-59.ec2.internal/172.31.5.59： 43555]

最后，RM web UI在作业运行期间始终显示以下消息：

状态：等待AM容器被分配、启动并注册到RM。

谢谢，阿莎

Answer 1

在尝试了Hadoop上的解决方案Socket Timeout Error（我问题中的链接）并将以下内容添加到hdfs-site.xml文件后，通过允许所有ICMP和UDP规则到ec2实例，以便它们可以在彼此之间ping，问题得到了解决。

<property>
  <name>dfs.namenode.name.dir</name>
  <value>/usr/local/hadoop/hadoop_work/hdfs/namenode</value>
</property>
<property>
  <name>dfs.datanode.data.dir</name>
  <value>/usr/local/hadoop/hadoop_work/hdfs/datanode</value>
</property>
<property>
  <name>dfs.namenode.checkpoint.dir</name>
  <value>/usr/local/hadoop/hadoop_work/hdfs/namesecondary</value>
</property>
<property>
  <name>dfs.block.size</name>
  <value>134217728</value>
</property>
<property>
  <name>dfs.client.use.datanode.hostname</name>
  <value>true</value>
</property>
<property>
  <name>dfs.datanode.socket.write.timeout</name>
  <value>2000000</value>
</property>
<property>
  <name>dfs.socket.timeout</name>
  <value>2000000</value>
</property>

<property>
  <name>dfs.datanode.use.datanode.hostname</name>
  <value>true</value>
  <description>Whether datanodes should use datanode hostnames when
    connecting to other datanodes for data transfer.
  </description>
</property>

<property>
  <name>dfs.namenode.rpc-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual address the RPC server will bind to. If this optional address is
    set, it overrides only the hostname portion of dfs.namenode.rpc-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node listen on all interfaces by
    setting it to 0.0.0.0.
  </description>
</property>

<property>
  <name>dfs.namenode.servicerpc-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual address the service RPC server will bind to. If this optional address is
    set, it overrides only the hostname portion of dfs.namenode.servicerpc-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node listen on all interfaces by
    setting it to 0.0.0.0.
  </description>
</property>

<property>
  <name>dfs.namenode.http-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual address the HTTP server will bind to. If this optional address
    is set, it overrides only the hostname portion of dfs.namenode.http-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node HTTP server listen on all
    interfaces by setting it to 0.0.0.0.
  </description>
</property>

<property>
  <name>dfs.namenode.https-bind-host</name>
  <value>0.0.0.0</value>
  <description>
    The actual address the HTTPS server will bind to. If this optional address
    is set, it overrides only the hostname portion of dfs.namenode.https-address.
    It can also be specified per name node or name service for HA/Federation.
    This is useful for making the name node HTTPS server listen on all
    interfaces by setting it to 0.0.0.0.
  </description>
</property>