Not able to connect to local elasticsearch process

Question

EDIT

I think what is going on is that localhost inside the docker process refers to the container's own localhost, not my system's localhost. So how do I ensure that when the application running the container tries to connect to the container's localhost:9200, it actually connects to my system's localhost:9200?

When I visit localhost:9200, my ES application seems to be running. It looks like this in chrome:

{
  "name" : "H1YDvcg",
  "cluster_name" : "elasticsearch_jwan",
  "cluster_uuid" : "aAorzRYTQPOI0j_OgMGKpA",
  "version" : {
    "number" : "6.8.1",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "1fad4e1",
    "build_date" : "2019-06-18T13:16:52.517138Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

I am running ES in a terminal window and it works after I run the command elasticsearch.

I am running a docker container with this command:

docker run -e DATALOADER_QUEUE='<some aws SQS queue name'\
             -e ES_HOST='localhost'\
             -e ES_PORT='9200'\
             -e AWS_ACCESS_KEY_ID='<somekey>'\
             -e AWS_SECRET_ACCESS_KEY='<somekey>'\
             -e AWS_DEFAULT_REGION='us-west-2'\
             <application name>

and I get this error:

requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: /person/_search (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e4189c90>

Anyone know what is going on? I don't understand why it cannot connect to ES even though ti seems to be running on localhost:9200.

Answer 1

Elasticsearch is running on your host at port 9200 and the application which want to access elasticsearch is running inside container.

The docker container by default runs in bridge networking mode, in which host and container network are different. Hence localhost inside container is not the same as on host.

Here you can do two things:

• In your application code try to access elasticsearch using private/public-ip:9200

OR

• Run docker container in host networking mode, so that localhost inside container is same as that on host. Because in this mode container uses network of host.

docker run -e DATALOADER_QUEUE='<some aws SQS queue name'\
             -e ES_HOST='localhost'\
             -e ES_PORT='9200'\
             -e AWS_ACCESS_KEY_ID='<somekey>'\
             -e AWS_SECRET_ACCESS_KEY='<somekey>'\
             -e AWS_DEFAULT_REGION='us-west-2'\
             --net=host \
             <application name>

NOTE: --net=host option will tell docker container to use host networking mode.

Answer 2

Solution was to use host.docker.internal in ES host setup.

I just used es_client = Elasticsearch(host=_es_host, where es_host = host.docker.internal and made sure to use http while local instead of https.

Answer 3

You can bind a host port to docker port using the -p option docker command. So in your below docker command, you can add one more line for binding the host 9200 port to docker 9200 port. notice I have added -p 9200:9200 and same is explained nicely in point 3 this doc

docker run -p 9200:9200 -e DATALOADER_QUEUE='<some aws SQS queue name'\
             -e ES_HOST='localhost'\
             -e ES_PORT='9200'\
             -e AWS_ACCESS_KEY_ID='<somekey>'\
             -e AWS_SECRET_ACCESS_KEY='<somekey>'\
             -e AWS_DEFAULT_REGION='us-west-2'\
             <application name>