IPython autoreload changes in subdirectory

Question

I launch IPython from the main folder /project. Now if I make changes in the file /project/tests/some_module.py, the changes fail to be autoreloaded in IPython. Also, I get the following message after I save the changes and want to run some other script in the prompt:

[autoreload of some_module failed: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/IPython/extensions/autoreload.py", line 229, in check
    superreload(m, reload, self.old_objects)
ImportError: No module named some_module]

It seems it detects changes were made inside folder /tests but it cannot import it. Can someone help me with this?

Edit: For better clarification: I launch IPython from the terminal in the main folder. In this folder, I have another folder tests. Inside tests I have two python files:

some_module.py:

def hello_world():
  print "Hello World!!!"

use_some_module.py:

from some_module import hello_world

hello_world()

Once I launched IPython, further changes in some_module.py won't be loaded in IPython. For example, if I add a second print "Hello Earth!!!" in the definition of hello_world(), and run run tests/use_some_module.py, I get the error message shown above, and will only get the "Hello World!!!" print.

Edit2: I would like a solution where I don't need to either change the working directory, or adding any search paths manually. I want it to be loaded automatically with autoreload.

Answer 1

If all you need is to reload a changed file in Python just do the following:

from main import some_module
....
reload(some_module)

But if your reload purposes are really eager, you can do the following (taken from this question):

%load_ext autoreload
%autoreload 2

The previous code will reload all changed modules every time before executing a new line.

NOTE: You can also check dreload which does a recursive reload of a module, and %run which allows you to run any python script and load all of its data directly into the interactive namespace.

Hope it helps,

Answer 2

Try changing your file use_some_module.py to:

import some_module

some_module.hello_world()

Reload with from import usually does not work.

This works for me:

%autoreload 1
%aimport use_some_module
%aimport some_module

Answer 3

Module /project/tests/some_module.py must be importable from /project. It means /project/tests must be in sys.path.

Either change to /project/tests and %run use_some_module.py or do sys.path.insert(0,tests) to insert tests into module search path.

If some_module is not in search path, IPython's autoreload never gonna find it.

Another way to make it work is to make the tests a package by creating __init__.py in it. And use relative import in use_some_module as from .some_module import hello_world. Now do these from IPython prompt,

In [1]: load_ext autoreload

In [2]: %autoreload 2

In [3]: from tests import some_module

In [4]: run -m tests.use_some_module
Hello World!!!
In [5]: ed tests/some_module.py
"tests/some_module.py" 3L, 69C written
 done. Executing edited code...
In [6]: run -m tests.use_some_module
Hello World!!!
Hello World!!!  

That's you have to run use_some_module module in the tests package as a script.

The first time you do %run tests/use_some_module.py, which is run as if you were running python tests/use_some_module.py. So /project/tests which the script is in, is automatically included in sys.path. That's why from some_module import hello_world in use_some_module succeeds. Additionally, after the run, objects in use_some_module's global namespace are available in IPython session.

But when you change tests/some_module, it has to be loaded again to see the changes. To reload it manually, it has to be imported first. Now import should succeed because use_some_module imported it first when it was run, some_module is in sys.modules. But reload to succeed, some_moudle has to be in search path. So, even manual reload would fail let alone autoreload.

In [2]: #before %run  
In [3]: 'some_module' in globals()
Out[3]: False

In [6]: 'some_module' in sys.modules
Out[6]: False

In [7]: 'hello_world' in globals()
Out[7]: False

In [8]: run tests/use_some_module.py
Hello World!!!

In [9]: 'some_module' in globals()
Out[9]: False

In [11]: 'some_module' in sys.modules
Out[11]: True

In [12]: 'hello_world' in globals()
Out[12]: True
In [13]: import some_module

In [14]: some_module = reload(some_module)
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-14-47e3c0de23e1> in <module>()
----> 1 some_module = reload(some_module)

ImportError: No module named some_module

Another solution is to start IPython as PYTHONPATH=tests ipython to include tests directory in sys.path.
Or set this c.InteractiveShellApp.exec_lines = ['import sys','sys.path.insert(0,"tests")'] in ipython_config.py.
.