Suppose we have two modules with cyclic dependencies:
# a.py
import b
def f(): return b.y
x = 42
# b.py
import a
def g(): return a.x
y = 43
The two modules are in the directory pkg
with an empty __init__.py
. Importing pkg.a
or pkg.b
works fine, as explained in this answer. If I change the imports to relative imports
from . import b
I get an ImportError
when trying to import one of the modules:
>>> import pkg.a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pkg/a.py", line 1, in <module>
from . import b
File "pkg/b.py", line 1, in <module>
from . import a
ImportError: cannot import name a
Why do I get this error? Isn't the situation pretty much the same as above? (Is this related to this question?)
Edit: This question is not about software design. I'm aware of ways to avoid the circular dependency, but I'm interested in the reason for the error anyway.
As an additional Note:
I had the following module Structure:
I wanted to be able to run my script by
import base.basescript
, however this failed with an error since thegui
file had animport base.databaseStuff.db
which caused an import ofbase
. Sincebase
was only registered as__main__
it caused a second execution of the whole imports and the error above unless I used an external script abovebase
thus importingbase
/basescript
only once. To prevent this I put the following in my base script:(Incedentally, the relative import doesn't matter. Using
from pkg import
... reveals the same exception.)I think what's going on here is that the difference between
from foo import bar
andimport foo.bar
is that in the first one, the valuebar
could be module in pkgfoo
or it could be a variable in a modulefoo
. In the second case, it's invalid forbar
to be anything but a module/package.This would matter because if bar is known to be a module, then the contents of
sys.modules
is sufficient to populate it. If it might be a variable in thefoo
module, then the interpreter must actually look into the contents offoo
, but while importingfoo
, that would be invalid; the actual module has not been populated yet.In the case of a relative import, we understand
from . import bar
to mean import the bar module from the package that contains the current module, but this is really just syntactic sugar, the.
name is translated to a fully qualified name and passed to__import__()
, and thus it looks for all the world like the ambigiousfrom foo import bar
First let's start with how
from import
work in python:Well first let's look at the byte code:
hmm interesting :), so
from foo import bar
is translated to firstIMPORT_NAME foo
which equivalent toimport foo
and thenIMPORT_FROM bar
.Now what
IMPORT_FROM
do ?let's see what python do when he found
IMPORT_FROM
:Well basically he get the the names to import from, which is in our
foo()
function will bebar
, then he pop from the frame stack the valuev
which is the return of the the last opcode executed which isIMPORT_NAME
, then call the functionimport_from()
with this two arguments :As you can see the
import_from()
function is quiet easy, it try first to get the attributename
from the modulev
, if it don't exist it raiseImportError
else return this attribute.Now what this have to do with relative import ?
Well relative import like
from . import b
are equivalent for example in the case that is in the OP question tofrom pkg import b
.But how this happen ? To understand this we should take a look to the
import.c
module of python specially to the function get_parent(). As you see the function is quiet long to list here but in general what it does when it see a relative import is to try to replace the dot.
with the parent package depending on the__main__
module, which is again from the OP question is the packagepkg
.Now let's put all this together and try to figure out why we end up with the behavior in the OP question.
For this it will help us if we can see what python do when doing imports, well it's our lucky day python come already with this feature which can be enabled by running it in extra verbose mode
-vv
.So using the command line:
python -vv -c 'import pkg.b'
:hmm what just happen before the
ImportError
?First)
from . import a
inpkg/b.py
is called, which is translated as explained above tofrom pkg import a
, which is again in bytecode is equivalent toimport pkg; getattr(pkg, 'a')
. But wait a minutea
is a module too ?! Well here came the fun part if we have something likefrom module|package import module
in this case a second import will happen which is the import of the module in the import clause. So again in the OP example we need now to importpkg/a.py
, and as you know first of all we set in oursys.modules
a key for our new module which will bepkg.a
and then we continue our interpretation of the modulepkg/a.py
, but before the modulepkg/a.py
finish importing it callfrom . import b
.Now come the Second) part,
pkg/b.py
will be imported and in it turn it will first attempt toimport pkg
which becausepkg
is already imported so there is a keypkg
in oursys.modules
it will just return the value of that key. Then it willimport b
set thepkg.b
key insys.modules
and start the interpretation. And we arrive to this linefrom . import a
!But remember
pkg/a.py
is already imported which mean('pkg.a' in sys.modules) == True
so the import will be skipped, and only thegetattr(pkg, 'a')
will be called , but what will happen ? python didn't finish importingpkg/a.py
!? So onlygetattr(pkg, 'a')
will be called , and this will raise anAttributeError
in theimport_from()
function, which will be translated toImportError(cannot import name a)
.DISCLAIM : This is my own effort to understand what is happening inside the interpreter, i'm far away of being an expert.
EDIt: This answer was rephrased because when i tried to read it again i remarked how my answer was bad formulated, hope now it will be more useful :)