python 3.x C extension module and submodule

Question

How do I make a C extension for python 3.x when a module has sub-modules? For example, I have a file called pet.c:

#include <Python.h>

PyObject* CatMeow(PyObject* self) {
    return PyUnicode_FromString( ">*<" );
}

static PyMethodDef CatFunctions[] = {
    {(char*) "meow", (PyCFunction) CatMeow, METH_NOARGS, NULL},
    {NULL, NULL, 0, NULL}
};

static PyModuleDef CatDef = {
    PyModuleDef_HEAD_INIT, "cat", "cat ext", -1, CatFunctions,
    NULL, NULL, NULL, NULL
};

PyMODINIT_FUNC PyInit_cat(void) {
    return PyModule_Create(&CatDef);
}

static PyModuleDef PetDef = {
    PyModuleDef_HEAD_INIT, "pet", "pet ext", -1, NULL,
    NULL, NULL, NULL, NULL
};

PyMODINIT_FUNC PyInit_pet(void) {
    PyObject* p = PyModule_Create(&PetDef);
    PyObject* c = PyInit_cat();
    Py_INCREF(c);
    PyModule_AddObject( p, "cat", c );
    return p;
}

When I build it with the following setup.py:

from distutils.core import setup, Extension

setup( 
    name='pet', 
    version='0.0', 
    ext_modules=[Extension('pet', ['pet.c'])]
)

I can see

>>> import pet
>>> pet.cat.meow()
'>*<'

or

>>> from pet import cat
>>> cat.meow()
'>*<'

which is as intended, but when I try

>>> from pet.cat import meow

I have a ModuleNotFoundError saying ... No module named 'pet.cat'; 'pet' is not a package, and if I try

>>> from pet import cat
>>> from cat import meow

I have a ModuleNotFoundError saying ... No module named 'cat'. But if I check the type of cat

>>> type(cat)
<class 'module'>

which says it is a module.

How do I make this work? Adding a module object to another module used to work well in python 2.7. Is it not supposed to work in python3 due to absolute import style? Or do I have to work with multi-phase initialisation as described in PEP 489?

Answer 1

Regarding the first error which complains about pet not being a package. If pet is there only to provide a parent for cat, there is an easy way to turn it into a package: remove all the pet related code from pet.c and use ext_package in setup.py

from distutils.core import setup, Extension

setup( 
    name = 'pet',
    version = '0.0',
    ext_package = 'pet',
    ext_modules = [Extension('cat', ['pet.c'])]
)

Running the above will create a directory called 'pet' and a shared library of which name starts with 'cat'. This effectively creates a namespace package -- there are two types of packages, regular and namespace, and the latter is the one without requiring __init__.py (see PEP 420 for details). From outside of pet, you can do

>>> from pet import cat
>>> from pet.cat import meow 
>>> meow()
'>*<'
>>> cat.meow()
'>*<'

The reason you can't do from cat import meow is because the fully qualified name of the module is 'pet.cat' not 'cat', which you can confirm it from cat.__name__. If you are running the interpreter inside of the directory pet, then you can do from cat import meow.