How do I set up a simple dockerized RabbitMQ clust

Question

I've been doing a bit of reading up about setting up a dockerized RabbitMQ cluster and google turns up all sorts of results for doing so on the same machine.

I am trying to set up a RabbitMQ cluster across multiple machines.

I have three machines with the names dockerswarmmodemaster1, dockerswarmmodemaster2 and dockerswarmmodemaster3

On the first machine (dockerswarmmodemaster1), I issue the following command:

docker run -d -p 4369:4369 -p 5671:5671 -p 5672:5672 -p 15671:15671 -p 15672:15672 \
    -p 25672:25672 --hostname dockerswarmmodemaster1 --name roger_rabbit \
    -e RABBITMQ_ERLANG_COOKIE='secret cookie here' rabbitmq:3-management

Now this starts up a rabbitMQ just fine, and I can go to the admin page on 15672 and see that it is working as expected.

I then SSH to my second machine (dockerswarmmodemaster2) and this is the bit I am stuck on. I have been trying variations on the following command:

docker run -d -p 4369:4369 -p 5671:5671 -p 5672:5672 -p 15671:15671  \
    -p 15672:15672 -p 25672:25672 --name jessica_rabbit -e CLUSTERED=true \
    -e CLUSTER_WITH=rabbit@dockerswarmmodemaster1 \
    -e RABBITMQ_ERLANG_COOKIE='secret cookie here' \
    rabbitmq:3-management

No matter what I try, the web page on both RabbitMQ machines says that there is no cluster under the 'cluster links' section. I haven't tried involving the third machine yet.

So - some more info:
1. The machine names are resolvable by DNS.
2. I have tried using the --net=host switch in the docuker run command on both machines; no change.
3. I am not using docker swarm or swarm mode.
4. I do not have docker compose installed. I'd prefer not to use it if possible.

Is there any way of doing this from the docker run command or will I have to download the rabbit admin cli and manually join to the cluster?

Answer 1

The official container seems to not support environment variables CLUSTERED and CLUSTER_WITH. It supports only a list variables that are specified in RabbitMQ Configuration. According to official Clustering Guide, one of possible solutions is via configuration file. Thus, you can just provide your own configuration to the container. Modified default configuration in your case will look like:

[ 
  { rabbit, [ 
    { loopback_users, [ ] }, 
    { cluster_nodes, {['rabbit@dockerswarmmodemaster1'], disc }} 
  ]} 
].

Save this snippet to, for example, /home/user/rmq/rabbitmq.config. Hint: If you want to see node in management console, you need to add another file /home/user/rmq/enabled_plugins with only string

[rabbitmq_management].

after that, your command will look like

docker run -d -p 4369:4369 -p 5671:5671 -p 5672:5672 -p 15671:15671  \
-p 15672:15672 -p 25672:25672 --name jessica_rabbit \
-v /home/user/rmq:/etc/rabbmitmq \
-e RABBITMQ_ERLANG_COOKIE='secret cookie here' \
rabbitmq:3-management

PS You may also need to consider setting environment variable RABBITMQ_USE_LONGNAME.

Answer 2

You can use this plugin https://github.com/aweber/rabbitmq-autocluster to create a RabbitMQ docker cluster.

The plugin uses etcd2 or consul as service discovery, in this way you don't need to use the rabbitmqctl command line.

I used it with docker swarm, but it is not necessary.

Here is the result

Answer 3

In order to create a cluster all rabbitmq nodes that are to form up a cluster must be accessible (each one by others) by node name (hostname). You need to specify a hostname for each docker container with --hostname option and to add /etc/host entries for all the other containers, this you can do with --add-host option or by manually edditing /etc/hosts file. So, here is the example for a 3 rabbitmq nodes cluster with docker containers (rabbitmq:3-management image):

First create a network so that you can assign IPs: docker network create --subnet=172.18.0.0/16 mynet1. We are going to have the following:

• 3 docker containers named rab1con, rab2con and rab3con
• IPs respectively will be 172.18.0.11 , -12 and -13
• each of them will have the host name respectively rab1, rab2 and rab3
• all of the must share the same erlang cookie

Spin up the first one

docker run -d --net mynet1 --ip 172.18.0.11 --hostname rab1 --add-host rab2:172.18.0.12 --add-host rab3:172.18.0.13 --name rab1con -e RABBITMQ_ERLANG_COOKIE='secret cookie here' rabbitmq:3-management

second one

docker run -d --net mynet1 --ip 172.18.0.12 --hostname rab2 --add-host rab1:172.18.0.11 --add-host rab3:172.18.0.13 --name rab2con -e RABBITMQ_ERLANG_COOKIE='secret cookie here' rabbitmq:3-management

last one

docker run -d --net mynet1 --ip 172.18.0.13 --hostname rab3 --add-host rab2:172.18.0.12 --add-host rab1:172.18.0.11 --name rab3con -e RABBITMQ_ERLANG_COOKIE='secret cookie here' rabbitmq:3-management

Than in contaner rab2con do

rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@rab1
rabbitmqctl start_app

and the same in rab3con and that's it.