Spark master picking up wrong IP on windows

问题:

When I start a spark master locally on my windows 7 machine for some reason to WebUI displays the URL and REST URL with the wrong ip address.

Below is my batch script I use to start master where %IP% is my local IP address.

set SPARK_MASTER_IP=%IP%
set SPARK_PUBLIC_DNS=%IP%
set SPARK_MASTER_PORT=7077
set SPARK_MASTER_WEBUI_PORT=8080
set SPARK_MASTER_REST_PORT=6066

START "spark-master" /MIN "C:/spark-1.6.0-bin-hadoop2.6/bin/spark-class.cmd" ^
    org.apache.spark.deploy.master.Master

The master Web UI displays the wrong IP. For some reason it is picking up the IP address of the IPv4 Address belonging to the VirtualBox installed on the same machine. (Note I'm am not using Virutalbox in any way to run Spark)

URL: spark://169.254.237.14:7077
REST URL: spark://169.254.237.14:6066 (cluster mode)

回答1:

I think the issue is caused by hostname to ip resolution order after VirtualBox is installed. I solved the problem by added my hostname and IP the my hosts file.

c:\Windows\System32\Drivers\etc\hosts

MY_HOST_NAME MY_IP