What's the right way to setup a development en

Question

Intro

I can't figure out a good way to setup a development environment on OS X using Docker and Boot2Docker. The problem I'm hitting is how to manage the source code so that:

1. I can modify the code on OS X using the tools (text editor, IDE, git, etc) I already have installed.
2. Those modifications are reflected in the Docker container so if I re-run tests or refresh a webpage, I can see my changes immediately.

In theory, this should be easy to do by mounting my source code as a volume:

docker run -it -v /path/to/my/source/code:/src some-docker-image

Unfortunately, this has two major issues that make it completely unusable on OS X:

Issue #1: Mounted volumes on Virtual Box (which use vboxsf) are extremely slow

For example, here is how long it takes Jekyll to compile my homepage if the source code is part of the Docker image:

> docker run -it brikis98/yevgeniy-brikman-homepage:v1 bash

root@7aaea30d98a1:/src# time bundle exec jekyll build

[...]

real    0m7.879s
user    0m7.360s
sys     0m0.600s

Here is the exact same Docker image, except this time, I mount the source code from OS X:

> docker run -it -v $(pwd):/src brikis98/yevgeniy-brikman-homepage:v1 bash

root@1521b0b4ce6a:/src# time bundle exec jekyll build

[...]

real    1m14.701s
user    0m9.450s
sys     0m3.410s

Issue #2: File watching is broken

The default watch mechanisms in SBT, Jekyll, and grunt use technologies such as inotify, which do not work if they are running in a Docker container and the changes are made in OS X to a mounted folder.

Workarounds I tried

I searched for solutions (including all the ones on SO) and tried out a few of them, but have not found a successful one:

1. I switched Boot2Docker to use NFS, but it was just as slow.
2. I tried Vagrant + NFS, and that was also just as slow.
3. I tried a Samba mount, but the folder always showed up empty in the Docker container.
4. I tried to use the unison file system, which worked briefly to sync files, but then kept showing connection errors.
5. I enabled polling in Jekyll, but that significantly increased the delay until my changes were picked up.
6. I tried dinghy, a "faster, friendlier Docker on OS X with Vagrant" and got some improvement. Instead of jekyll compilation being 10-15x slower, it was 2-3x slower. That's better, but still not quite usable.

Has anyone found a solution that actually works and allows you to productively develop code with Docker and OS X?

Update: a solution at last!

I have finally found a solution that seems productive using Boot2Docker + rsync. I've captured the details on how to set this up in my own answer as well as an open source project called docker-osx-dev.

Answer 1

Docker Unison works like a charm! https://github.com/leighmcculloch/docker-unison

Bidirectional Sync with a very good performance!

Answer 2

I've decided to add my own answer with the best solution I've found so far. I'll update this if I find better options.

Best solution so far

The best solution I've found for setting up a productive development environment with Docker on OS X is: Boot2Docker + Rsync. With rsync, build times in a Docker container are on par with running the build directly on OSX! Moreover, the file watcher code does not need polling (inotify works since rsync uses normal folders), so hot reload is almost as fast.

There are two ways to set it up: an automated install and a manual install.

Automated install

I've packaged all the steps for setting up Boot2Docker with Rsync into an open source project called docker-osx-dev. The code is a bit rough, but I've been successfully using it for several weeks to easily switch between 3 projects with 3 different tech stacks. Try it out, report bugs, and submit some PRs! Also, see my blog post, A productive development environment with Docker on OS X for more info.

Manual setup

1. Install Boot2Docker: brew install boot2docker.
2. Run Boot2Docker, but with VirtualBox shared folders disabled: boot2docker init && boot2docker start --vbox-share=disable.
3. Run boot2docker shellinit and copy the environment variables it prints out into your ~/.bash_profile file.
4. Install rsync on the Boot2Docker VM: boot2docker ssh "tce-load -wi rsync".
5. Create the base folders you need on the Boot2Docker VM and set permissions correctly for them. For example, if you'll be syncing the /foo/bar folder from OS X, you need to create /foo/bar on the Boot2Docker VM: boot2docker ssh "mkdir -p /foo/bar && chown -R docker /foo/bar".
6. Run rsync to sync the files to the Boot2Docker VM: rsync --archive --rsh="ssh -i $HOME/.ssh/id_boot2docker -o StrictHostKeyChecking=no" /foo/bar docker@dockerhost:/foo. Check the rsync docs for various settings you may want to enable, such as using --exclude .git to exclude the .git folder when syncing.
7. Use a file watcher to keep files in sync. For example, you could use fswatch (brew install fswatch) piped into rsync.
8. At this point, you should be able to use docker run to fire up your Docker container and use the -v flag to mount the folder you're syncing: docker run -v /foo/bar:/src some-docker-image.
9. Update the code on OS X as usual. Changes should propagate very quickly using rsync, the normal file watcher code should pick up the changes as usual (ie, using inotify), and the build should run fast because all the files are "local" to the container.
10. If you need to test a running website, run the boot2docker ip command to find out what IP it's on.

Answer 3

I'm also using Vagrant with parallels and boot2docker (https://github.com/Parallels/boot2docker-vagrant-box). Development was never easier for me. Works really well with docker-compose and large setups. I don't really feel a delay or massive resource consumption.

This is what my Vagrantfile looks like:

Vagrant.configure(2) do |config|

  config.vm.network "private_network", ip: "192.168.33.10"
  config.vm.box = "parallels/boot2docker"

  config.vm.synced_folder "/Users", "/Users", type: "nfs", mount_options: ["nolock", "vers=3", "udp"], id: "nfs-sync"

end

Answer 4

Update: Now that docker for mac is in beta with non-hack functionality, going that route may be a lot more reasonable for local development without a essay's worth of hacks and workarounds.

Don't. I know that's not the answer you are probably hoping for, but take an honest evaluation of the cost/benefit of trying to get local source code + dockerized execution vs just doing local development on OSX.

At some point all the issues, setup effort, and operational pain points MAY be resolved well enough, but as of right now my take on this is it's a net loss.

Issue #1: Mounted volumes on Virtual Box (which use vboxfs) are extremely slow

Wait a while and this will almost certainly improve.

Issue #2: File watching is broken

I'm not sure a fix for this is in the near future. If this type of functionality is key to your development workflow, I would consider this a dealbreaker. It's not worth a major R&D effort when compared to just using rbenv/bundler to manage your jekyll/ruby installs and running them locally on OSX like folks have been doing successfully for the past decade+.

Just like "the cloud" has zero involvement in my local development setup, at the moment, docker is a win for testing/staging/deployment and for running databases and other third party components, but the applications I'm actually coding get run straight on OSX.

Answer 5

Disclaimer: I might be biased, since I am the author of docker-sync.

I probably tried all the solutions named here, including some more (see the compersion https://github.com/EugenMayer/docker-sync/wiki/Alternatives-to-docker-sync), but they basically either failed on the side of performance (most of them) or on the docker-machine (or none) used / enforced.

http://docker-sync.io has been built to merge all the solutions and provide the best strategies (implementing several, you can choose).

It can be used with rsync (1 way sync) including permission fixes for users, and with unison (2 way sync). It does neither force you into docker-machine or a specific hypervisor, nor requires you to have docker for Mac. It works with all of them.

The performance EugenMayer/docker-sync/wiki/4.-Performance is not influenced, it's like you have no shares at all.

docker-sync and its change-watchers are optimized and do work with projects with 12k files without issues.

Give it a try, if you like, I would love to hear feedback!

Answer 6

This method is the latest (Sep 2015) and easiest way to get Docker setup on Mac: link here:

You install Docker using Docker Toolbox link to instructions here:

It is a complete Docker setup package, that includes the following Docker tools:

Docker Machine for running the docker-machine binary

Docker Engine for running the docker binary

Docker Compose for running the docker-compose binary

Kitematic, the Docker GUI a shell preconfigured for a Docker command-line environment

Oracle VM VirtualBox

What's in the toolbox:

• Docker Client
• Docker Machine
• Docker Compose (Mac only)
• Docker Kitematic
• VirtualBox