How to bundle elasticsearch templates with docker

Question

Im currently building a custom docker image to be used for integration test. My requirement is to set it up with custom configuration with default ingester pipeline and template mappings.

Dockerfile:

FROM docker.elastic.co/elasticsearch/elasticsearch:5.6.2
ADD config /usr/share/elasticsearch/config/
USER root
RUN chown -R elasticsearch:elasticsearch config
RUN chmod +x config/setup.sh
USER elasticsearch
RUN elasticsearch-plugin remove x-pack
EXPOSE 9200
EXPOSE 9300

where config is a directory which contains:

> elasticsearch.yml  for the configuration
> templates in the form of json files
> setup.sh - script which executes curl to es in order to register pipelines to _ingester and template mappings

The setup script looks like this:

#!/bin/bash
# This script sets up the es5 docker instance with the correct pipelines and templates

baseUrl='127.0.0.1:9200'
contentType='Content-Type:application/json'


# filebeat
filebeatUrl=$baseUrl'/_ingest/pipeline/filebeat-pipeline?pretty'
filebeatPayload='@pipeline/filebeat-pipeline.json'

echo 'setting filebeat pipeline...'
filebeatResult=$(curl -XPUT $filebeatUrl -H$contentType -d$filebeatPayload)
echo -e "filebeat pipeline setup result: \n$filebeatResult"

# template
echo -e "\n\nsetting up templates..."
sleep 1

cd template
for f in *.json
do
    templateName="${f%.*}"
    templateUrl=$baseUrl'/_template/'$templateName

    echo -e "\ncreating index template for $templateName..."
    templateResult=$(curl -XPUT $templateUrl -H$contentType -d@$f)
    echo -e "$templateName result: $templateResult"
    sleep 1
done

echo -e "\n\n\nCompleted ES5 Setup, refer to logs for details"

How do i build and run the image in such a way that the script gets executed AFTER elastic is up and running?

Answer 1

If template mapping is not evolving frequently then you can try below solution:

You can embed template in your custom image by saving container state(creating new image) using following steps:

1. Run your image as per your dockerfile(elasticsearch would have been stared in it)
2. Use docker exec command to run your template(curl command or script)
3. Use docker commit to save container state and create new image which will already have template

Use newly created image which already has template mapping.You don't need to run template mapping as part of script since your image itself will have it.

Answer 2

What I usually do is to include a warmer script like yours and at the beginning I add the following lines. There's no other way that I know of in Docker to wait for the underlying service to launch

# wait until ES is up
until $(curl -o /dev/null -s --head --fail $baseUrl); do
    echo "Waiting for ES to start..."
    sleep 5
done