How to setup learning environment for Udacity Deep

Question

I believe many of those interested in studying DL heard of this course:

https://www.udacity.com/course/deep-learning--ud730

I am taking the course now and would like to share step-by-step instruction on how to setup learning environment on Windows from scratch.

• The first answer named SETTING UP THE ENVIRONMENT is about setting up the learning environment. You run it only once.

• The second answer named AFTER LOCAL MACHINE REBOOT is about how to start the environment over after you reboot your computer.

• See the third answer named HOW IT ALL WORKS to learn how all that stuff works (or you can follow the first answer blindly and check it out later).

Answer 1

AFTER LOCAL MACHINE REBOOT

To start learning environment after computer reboot, create .bat file (I call it udacity-tf-start.bat) with the following content:

call docker-machine start vdocker

FOR /f "tokens=*" %%i IN ('docker-machine env --shell cmd vdocker') DO %%i

call docker start -ai tensorflow-udacity

Important! %% is a kind of escaping and you only need it inside a BAT file. In case you are running the same set of commands via command-line, you should use:

FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i

Answer 2

SETTING UP THE ENVIRONMENT (run it only once!)

N.B. To start ready environment after computer reboot, use AFTER LOCAL MACHINE REBOOT instruction in the second answer.

---

Steps:

1. Download and setup Docker Toolbox:

https://www.docker.com/products/docker-toolbox

Docker is a tool to deploy preconfigured virtual learning environment on your machine. It will be running inside a virtual machine and will not mess with your computer anyhow.

2. (optional step) Docker will put it's files on system disk (C:) and you might want to change that if using SSD. You can do it that way:

mklink /J "C:\Users\USER\.docker" "D:\Docker"

• substitute USER with your username
• substitute "D:\Docker" with a path on other drive where you would like to store Docker files

More at: Change .docker directory on Windows

3. Open Windows CMD. Go to folder where Docker is installed. Create a new docker machine:

docker-machine create vdocker -d virtualbox

4. (magic step) Just run it!

FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i

More at: How do I start tensorflow docker jupyter notebook

5. Download and install preconfigured assignment docker image:

docker run -it -p 8888:8888 -p 6006:6006 --name tensorflow-udacity -it b.gcr.io/tensorflow-udacity/assignments:0.5.0

6. (important step!) Configure port forwarding:
   • Run Oracle VM VirtualBox link (should be created when installing Docker):

• Go to Settings... of vdocker machine:

• Add port forwarding (it will forward 8888 port in virtual environment to 8810 port on your local machine):

P.S. Using :8810 port in case you already have IPython notebook installed on your local machine.

7. In the Settings... menu (from the previous step) allow virtual machine more memory:

N.B. The VirtualBox has to be shut down before you can make any changes to system settings. (by jlarsch)

Use the following command to stop the VM:

docker-machine stop vdocker

(optional) You can also allow it to use more cores in order to run faster:

8. Profit!

Answer 3

HOW IT ALL WORKS

Disclaimer: It may somewhat resemble the plot of Christopher Nolan's Inception movie.

The overall picture

Details

Due to some limitations of Windows operating system Docker couldn't be run there natively (yet). That's why we first create a virtual box:

docker-machine create vdocker -d virtualbox

The next step (denoted as magic step) sets some environment variables for docker command to be able to connect to docker daemon running inside the virtual box:

FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i

>SET DOCKER_TLS_VERIFY=1

>SET DOCKER_HOST=tcp://192.168.99.100:2376

>SET DOCKER_CERT_PATH=C:\Users\USER\.docker\machine\machines\vdocker

>SET DOCKER_MACHINE_NAME=vdocker

Then we run:

docker run -it -p 8888:8888 -p 6006:6006 --name tensorflow-udacity -it b.gcr.io/tensorflow-udacity/assignments:0.5.0

which creates a docker container named tensorflow-udacity from an image it downloads from the specified URL. Important! That container runs inside a virtual box.

Pay attention to -p flags:

-p 8888:8888 -p 6006:6006

it tells docker daemon to forward (publish) a container's port 8888 to host (virtual box) port 8888. It's not available on the Windows machine yet!

Now we add one more port forwarding rule to the virtual box settings:

Answer 4

Complementing to the other answers here is my starting script for creation/running/starting of a docker machine. Setup proceedure now boils down to installing the latest version of docker toolbox (this should autom. install vbox) from https://docs.docker.com/toolbox/toolbox_install_windows/ and running the script:

@echo off
set DOCKERMACHINENAME=tensorflow-udacity
set REPOSITORY=gcr.io/tensorflow/udacity-assignments:0.6.0
set "LOCALDIR0=/%SystemDrive:~0,1%/"
call :LoCase LOCALDIR0
SET "LOCALDIR=%LOCALDIR0%Users/%USERNAME%"
docker-machine.exe env %DOCKERMACHINENAME% > nul 2> nul
if "%errorlevel%"=="0" goto m_exists
::Machine has to be created
docker-machine create -d virtualbox --virtualbox-memory 8196 %DOCKERMACHINENAME%

:m_exists

::Check if machine needs to be restarted
docker-machine ip %DOCKERMACHINENAME% > nul 2>nul
if not "%errorlevel%"=="0" (docker-machine.exe restart %DOCKERMACHINENAME%)

FOR /F "tokens=*" %%i IN ('docker-machine env --shell cmd %DOCKERMACHINENAME%') DO %%i
FOR /F "tokens=*" %%F IN ('docker-machine ip %DOCKERMACHINENAME%') DO (SET DOCKERMACHINEIP=%%F)
echo Access to iPython: %DOCKERMACHINEIP%:8888

docker inspect %DOCKERMACHINENAME% > nul 2> nul
if "%errorlevel%"=="0" goto m_started
:: Machine has to be started
docker run -p 8888:8888 --name %DOCKERMACHINENAME% -v %LOCALDIR%:/mnt/hosttmp:rw -it %REPOSITORY%
goto finished

:m_started
docker start -ai %DOCKERMACHINENAME%
goto finished

:LoCase
FOR %%i IN ("A=a" "B=b" "C=c" "D=d" "E=e" "F=f" "G=g" "H=h" "I=i" "J=j" "K=k" "L=l" "M=m" "N=n" "O=o" "P=p" "Q=q" "R=r" "S=s" "T=t" "U=u" "V=v" "W=w" "X=x" "Y=y" "Z=z") DO CALL SET "%1=%%%1:%%~i%%"

:finished
::hint: to remove container use: docker rm %DOCKERMACHINENAME%