Hadoop: binding multiple IP addresses to a cluster

问题:

I've a four-node Hadoop cluster on Softlayer. The master (NameNode) has a public IP address for external access and a private IP address for cluster access. The slave nodes (datanodes) have private IP address which I'm trying to connect to the master without the need of assigning public IP addresses to each slave node.

I've realised that setting fs.defaultFS to the NameNode's public address allows for external access, except that the NameNode only listens to that address for incoming connections, not the private address. So I get ConnectionRefused exceptions in the datanode logs as they're trying to connect with the NameNode's private IP address.

I figured the solution might be to set both the public and private IP address to the NameNode so that external access is preserved and allows my slaves nodes to connect as well.

So is there a way I can bind both these addresses to the NameNode so that it will listen on both?

Edit: Hadoop version 2.4.1.

回答1:

The asker edited this into his question as an answer:

In hdfs-site.xml, set the value of dfs.namenode.rpc-bind-host to 0.0.0.0 and Hadoop will listen on both the private and public network interfaces allowing remote access and datanode access.