How to set up Hadoop in Docker Swarm?

Question

I would like to be able to start a Hadoop cluster in Docker, distributing the Hadoop nodes to the different physical nodes, using swarm.

I have found the sequenceiq image that lets me run hadoop in a docker container, but this doesn't allow me to use multiple nodes. I have also looked at the cloudbreak project, but it seems to need an openstack installation, which seems a bit overkill, because it seems to me like swarm alone should be enough to do what we need.

Also I found this Stackoverflow question+answer which relies on weave, which needs sudo-rights, which our admin won't give to everyone.

Is there a solution so that starting the hadoop cluster comes down to starting a few containers via swarm?

Answer 1

I cannot give a definitive answer, but if you are looking to set this up without administratrator privileges and all answers to this question fail I fear you might be out of luck.

Consider asking the admin why he does not want to give out sudo access, chances are that either you can take away his doubts, or else that it turns out that what you want to do is undesirable.