running a python file from the ipython notebook us

Question

I have built a model that trains using training.py. I want to tune the hyperparameters and run the following script from the notebook in loop by varying the arguments passed.

python training.py --cuda --emsize 1500 --nhid 1500 --dropout 0.65 --epochs 10

For eg: If the hyperparameter is dropout, I want to be able to run the script in loop by varying dropout values and plot the graph.

Answer 1

You can run a shell command using ! in an ipython environment as

!ls -l

If you want to use it with variables,then you can use {}.

# Supposing you have epochs in e and dropout size in d
!python training.py --cuda --emsize 1500 --nhid 1500 --dropout {d} --epochs {e}

You can also capture the output of shell runs in ipython using !! instead of !. The !! magic command returns the output of the command as a list of strings. So, you can do something like

files = !! ls - l