_sha import in python hashlib

Question

Well, today I was checking the hashlib module in python, but then I found something that I still can't figure out.

Inside this python module, there is an import that I can't follow. I goes like this:

def __get_builtin_constructor(name):
    if name in ('SHA1', 'sha1'):
        import _sha
        return _sha.new

I tried to import the _sha module from a python shell, but is seems that it cannot be reached that way.My first guess is that it's a C module, but I'm not sure.

So tell me guys, do you know where is that module? How do they import it?

Answer 1

Seems that you're python installation has sha compiled inside _haslib instead of _sha (both C modules). From hashlib.py in python 2.6:

import _haslib:
    .....
except ImportError:
    # We don't have the _hashlib OpenSSL module?
    # use the built in legacy interfaces via a wrapper function
    new = __py_new

    # lookup the C function to use directly for the named constructors
    md5 = __get_builtin_constructor('md5')
    sha1 = __get_builtin_constructor('sha1')
    sha224 = __get_builtin_constructor('sha224')
    sha256 = __get_builtin_constructor('sha256')
    sha384 = __get_builtin_constructor('sha384')
    sha512 = __get_builtin_constructor('sha512')

Answer 2

Actually, the _sha module is provided by shamodule.c and _md5 is provided by md5module.c and md5.c and both will be built only when your Python is not compiled with OpenSSL by default.

You can find the details in setup.py in your Python Source tarball.

    if COMPILED_WITH_PYDEBUG or not have_usable_openssl:
        # The _sha module implements the SHA1 hash algorithm.
        exts.append( Extension('_sha', ['shamodule.c']) )
        # The _md5 module implements the RSA Data Security, Inc. MD5
        # Message-Digest Algorithm, described in RFC 1321.  The
        # necessary files md5.c and md5.h are included here.
        exts.append( Extension('_md5',
                        sources = ['md5module.c', 'md5.c'],
                        depends = ['md5.h']) )

Most often, your Python is built with Openssl library and in that case, those functions are provided by the OpenSSL libraries itself.

Now, if you want them separately, then you can build your Python without OpenSSL or better yet, you can build with pydebug option and have them.

From your Python Source tarball:

./configure --with-pydebug
make

And there you go:

>>> import _sha
[38571 refs]
>>> _sha.__file__
'/home/senthil/python/release27-maint/build/lib.linux-i686-2.7-pydebug/_sha.so'
[38573 refs]