上传文件到hdfs,结果50070web控制台的Availability为localhost

缘起

大数据最近入门. 3台centos虚拟机搭建了hadoop环境. 集群部署规划如下

hadoop01 hadoop02 hadoop03
HDFS namenode/datanode datanode secondarynamenode/datanode

但是上传本地文件到hdfs上去之后, 打开50070端口的hdfsのweb控制台一看

我明明有三台hadoop节点, 为什么该block的副本全部在localhost?

分析

hadoop02的日志如下

1
2
3
4
[root@hadoop02 hadoop-2.7.2]# vim logs/hadoop-root-datanode-hadoop02.log 
...
2019-02-11 16:35:52,721 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop01/192.168.119.130:9000. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
...

日志表明, datanode节点联系不上namenode的9000端口. 百度了一波, 无非说什么防火墙没关之类的. 但是我是禁用掉了防火墙的（讲真，稍有点开发经验的人应该都不会犯这种错误或者因此水一篇博客）. 后来百度到了一篇文章, 参见参考文献【1】 . 在hadoop01上

1
2
[root@hadoop01 ~]# netstat -nltp | grep 9000
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      9349/java

原来我可爱的9000端口正在127.0.0.1这个回环地址上listen呢! 那么我的datanode怎么可能访问到这个服务呢? 于是不禁要问，为什么ip是127.0.0.1，而不是hadoop01的机器的ip呢? 查看hosts记录

1
2
3
4
5
6
7
[root@hadoop01 hadoop-2.7.2]# cat /etc/hosts
#127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 hadoop01 localhost
192.168.119.130 hadoop01
192.168.119.132 hadoop02
192.168.119.133 hadoop03

发现第四行手贱了.

解决

修改hadoop01的/etc/hosts记录为

1
2
3
4
5
6
[root@hadoop01 hadoop-2.7.2]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.119.130 hadoop01
192.168.119.132 hadoop02
192.168.119.133 hadoop03

则正常了

1
2
[root@hadoop01 ~]# netstat -nltp | grep 9000
tcp        0      0 192.168.119.130:9000    0.0.0.0:*               LISTEN      4490/java

参考

【1】https://blog.csdn.net/yjc_1111/article/details/53817750