Textual reference of a method

Question

Say I have the following:

def func():
    print 'this is a function and not a method!!!'

class Test:
    def TestFunc(self):
        print 'this is Test::TestFunc method'

I have the following functions (which are taken from https://bitbucket.org/agronholm/apscheduler/src/d2f00d9ac019/apscheduler/util.py):

def get_callable_name(func):
    """
    Returns the best available display name for the given function/callable.
    """
    f_self = getattr(func, '__self__', None) or getattr(func, 'im_self', None)
    if f_self and hasattr(func, '__name__'):
        if isinstance(f_self, type):
            # class method
            clsname = getattr(f_self, '__qualname__', None) or f_self.__name__
            return '%s.%s' % (clsname, func.__name__)
        # bound method
        return '%s.%s' % (f_self.__class__.__name__, func.__name__)
    if hasattr(func, '__call__'):
        if hasattr(func, '__name__'):
            # function, unbound method or a class with a __call__ method
            return func.__name__
        # instance of a class with a __call__ method
        return func.__class__.__name__
    raise TypeError('Unable to determine a name for %s -- '
                    'maybe it is not a callable?' % repr(func))


def obj_to_ref(obj):
    """
    Returns the path to the given object.
    """
    ref = '%s:%s' % (obj.__module__, get_callable_name(obj))
    try:
        obj2 = ref_to_obj(ref)
        if obj != obj2:
            raise ValueError
    except Exception:
        raise ValueError('Cannot determine the reference to %s' % repr(obj))
    return ref


def ref_to_obj(ref):
    """
    Returns the object pointed to by ``ref``.
    """
    if not isinstance(ref, basestring):
        raise TypeError('References must be strings')
    if not ':' in ref:
        raise ValueError('Invalid reference')
    modulename, rest = ref.split(':', 1)
    try:
        obj = __import__(modulename)
    except ImportError:
        raise LookupError('Error resolving reference %s: '
                          'could not import module' % ref)
    try:
        for name in modulename.split('.')[1:] + rest.split('.'):
            obj = getattr(obj, name)
        return obj
    except Exception:
        raise LookupError('Error resolving reference %s: '
                          'error looking up object' % ref)

The above functions - obj_to_ref returns a textual reference to a given function object and ref_to_obj returns an object for the given textual reference. For example, lets try the func function.

>>> 
>>> func
<function func at 0xb7704924>
>>> 
>>> obj_to_ref(func)
'__main__:func'
>>> 
>>> ref_to_obj('__main__:func')
<function func at 0xb7704924>
>>> 

The func function is working fine. But when tried to use these function on an instance of the class Test, it couldn't get a textual reference.

>>> 
>>> t = Test()
>>> 
>>> t
<__main__.Test instance at 0xb771b28c>
>>> 
>>> t.TestFunc
<bound method Test.TestFunc of <__main__.Test instance at 0xb771b28c>>
>>> 
>>> 
>>> obj_to_ref(t.TestFunc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 11, in obj_to_ref
ValueError: Cannot determine the reference to <bound method Test.TestFunc of <__main__.Test instance at 0xb771b28c>>
>>> 
>>> 

The obj_to_ref function for the given input t.TestFunc comes up with __main__:Test.TestFunc as a textual representation but the same text cannot be used to generate the object.

Question:

Is there a way in Python where we can represent an object like

>>> t.TestFunc
<bound method Test.TestFunc of <__main__.Test instance at 0xb771b28c>>
>>> 

in a string and reconstruct the object from the string?

Would it be possible if we save the address 0xb771b28c as a part of the string and regenerate the object by dereferencing this address?!

Answer 1

Your question is interesting but tangled.

1) You shouldn't call func the parameter of get_callable_name(func)
In my answer I replaced it by X.

2) You put a part of the code at a wrong place.

try:
    obj2 = ref_to_obj(ref)
    print 'obj != obj2  : ',obj != obj2
    if obj != obj2:
        raise ValueError
except Exception:
    raise ValueError('Cannot determine the reference to %s' % repr(obj))
return ref

has nothing to do inside obj_to_ref()

In my answer, I moved it outside of this function.

3) The visible reason of the problem of your code is that the reference obtained for object t.TestFunc (passed to parameter X in my code) is '__main__:Test.TestFunc' , not '__main__:t.TestFunc' as it "should" be.

The secretive step where this is decided is in the function get_callable_name() as said by entropy.
Since f.self is tand X has a name (TestFunc) but is not a class of type type (since t is an instance) ,
the instruction return '%s.%s' % (f_self.__class__.__name__, X.__name__) is executed.

But you're wrong to put the expression f_self.__class__.__name : it is the name of the class of t , not the name of t itself.

.

The problem is that , unlikely to a class (that has an attribute __name__), nothing is intended in the Python language to furnish the name of an instance on demand: an instance has not the same kind of attribute __name__ as a class, that would give the instance's name.

So , being uneasy to get it, a sort of bypass must be employed.
Every time an unreferenced name is needed, the bypass is to search among all the names of a namespace and to test the corresponding object against the concerned object.
That's what does the function get__callable_name() of entropy.

.

With this function, it works.

But I want to underline that it's only a tricky bypass that has not a real fundament.
I mean that the name t.TestFunc for the method is an illusion. There's a subtlety: there is no method belonging to an instance. That seems a weird pretention, but I'm sure it's the truth.
The fact that we call a method thanks to an expression like t.TestFunc leads to the believing that TestFunc belongs to the instance. In reality it belongs to the class and Python goes from an instance to it's class to find the method.

I do not invent anything, I've read it:

A class instance has a namespace implemented as a dictionary which is the first place in which attribute references are searched. When an attribute is not found there, and the instance’s class has an attribute by that name, the search continues with the class attributes. If a class attribute is found that is a user-defined function object or an unbound user-defined method object whose associated class is the class (call it C) of the instance for which the attribute reference was initiated or one of its bases, it is transformed into a bound user-defined method object whose im_class attribute is C and whose im_self attribute is the instance.

http://docs.python.org/2/reference/datamodel.html#new-style-and-classic-classes

But well, it's another story on which I will be disputed, I think, and I 've no time to engage in that.

Just verify the following point:
despite the fact that getattr(t,"TestFunc") gives :
<bound method Test.TestFunc of <__main__.Test instance at 0x011D8DC8>>
the method TestFunc is not in the namespace of t : the result of t.__dict__ is { } !

I just wanted to point it out because the function get_callable_name() only reproduces and mimics the apparent behavior and implementation of Python.
However the real behavior and implementation under the hood is different.

.

In the following code, I get the good result by using the isntruction
return '%s.%s' % ('t', X.__name__) instead of
return '%s.%s' % (f_self.__class__.__name__, func.__name__) or
return '%s.%s' % (variable_name_in_module(__import__(f_self.__module__), f_self), func.__name__)
because it's essentially what does the function get_callanle_name() (it doesn't uses a normal process, it uses a craftiness)

def get_callable_name(X):
    """
    Returns the best available display name for the given function/callable.
    """
    print '- inside get_callable_name()'
    print '  object X arriving in get_callable_name() :\n    ',X
    f_self = getattr(X, '__self__', None) or getattr(X, 'im_self', None) 
    print '  X.__call__ ==',X.__call__  
    print '  X.__name__ ==',X.__name__
    print '\n  X.__self__== X.im_self ==',f_self
    print '  isinstance(%r, type)  is  %r' % (f_self,isinstance(f_self, type))
    if f_self and hasattr(X, '__name__'): # it is a method
        if isinstance(f_self, type):
            # class method
            clsname = getattr(f_self, '__qualname__', None) or f_self.__name__
            return '%s.%s' % (clsname, X.__name__)
        # bound method
        print '\n  f_self.__class__          ==',f_self.__class__
        print '  f_self.__class__.__name__ ==',f_self.__class__.__name__
        return '%s.%s' % ('t', X.__name__)
    if hasattr(X, '__call__'):
        if hasattr(X, '__name__'):
            # function, unbound method or a class with a __call__ method
            return X.__name__
        # instance of a class with a __call__ method
        return X.__class__.__name__
    raise TypeError('Unable to determine a name for %s -- '
                    'maybe it is not a callable?' % repr(X))


def obj_to_ref(obj):
    """
    Returns the path to the given object.
    """
    print '- obj arriving in obj_to_ref :\n  %r' % obj

    ref = '%s:%s' % (obj.__module__, get_callable_name(obj))

    return ref


def ref_to_obj(ref):
    """
    Returns the object pointed to by ``ref``.
    """
    print '- ref arriving in ref_to_obj == %r' % ref

    if not isinstance(ref, basestring):
        raise TypeError('References must be strings')
    if not ':' in ref:
        raise ValueError('Invalid reference')
    modulename, rest = ref.split(':', 1)

    try:
        obj = __import__(modulename)
    except ImportError:
        raise LookupError('Error resolving reference %s: '
                          'could not import module' % ref)

    print '  we start with dictionary obj == ',obj
    try:
        for name in modulename.split('.')[1:] + rest.split('.'):
            print '  object of name ',name,' searched in',obj
            obj = getattr(obj, name)
            print '  got obj ==',obj
        return obj
    except Exception:
        raise LookupError('Error resolving reference %s: '
                          'error looking up object' % ref)

class Test:
    def TestFunc(self):
        print 'this is Test::TestFunc method'


t = Test()

print 't ==',t

print '\nt.TestFunc ==',t.TestFunc

print "getattr(t,'TestFunc') ==",getattr(t,'TestFunc')

print ('\nTrying to obtain reference of t.TestFunc\n'
       '----------------------------------------')

print '- REF = obj_to_ref(t.TestFunc)  done'
REF = obj_to_ref(t.TestFunc)
print '\n- REF obtained: %r' % REF

print ("\n\nVerifying what is ref_to_obj(REF)\n"
       "---------------------------------")
try:
    print '- obj2 = ref_to_obj(REF)  done'
    obj2 = ref_to_obj(REF)
    if obj2 != t.TestFunc:
        raise ValueError
except Exception:
        raise ValueError('Cannot determine the object of reference %s' % REF)
print '\n- object obtained : ',obj2

result

t == <__main__.Test instance at 0x011DF5A8>

t.TestFunc == <bound method Test.TestFunc of <__main__.Test instance at 0x011DF5A8>>
getattr(t,'TestFunc') == <bound method Test.TestFunc of <__main__.Test instance at 0x011DF5A8>>

Trying to obtain reference of t.TestFunc
----------------------------------------
- REF = obj_to_ref(t.TestFunc)  done
- obj arriving in obj_to_ref :
  <bound method Test.TestFunc of <__main__.Test instance at 0x011DF5A8>>
- inside get_callable_name()
  object X arriving in get_callable_name() :
     <bound method Test.TestFunc of <__main__.Test instance at 0x011DF5A8>>
  X.__call__ == <method-wrapper '__call__' of instancemethod object at 0x011DB990>
  X.__name__ == TestFunc

  X.__self__== X.im_self == <__main__.Test instance at 0x011DF5A8>
  isinstance(<__main__.Test instance at 0x011DF5A8>, type)  is  False

  f_self.__class__          == __main__.Test
  f_self.__class__.__name__ == Test

- REF obtained: '__main__:t.TestFunc'


Verifying what is ref_to_obj(REF)
---------------------------------
- obj2 = ref_to_obj(REF)  done
- ref arriving in ref_to_obj == '__main__:t.TestFunc'
  we start with dictionary obj ==  <module '__main__' (built-in)>
  object of name  t  searched in <module '__main__' (built-in)>
  got obj == <__main__.Test instance at 0x011DF5A8>
  object of name  TestFunc  searched in <__main__.Test instance at 0x011DF5A8>
  got obj == <bound method Test.TestFunc of <__main__.Test instance at 0x011DF5A8>>

- object obtained :  <bound method Test.TestFunc of <__main__.Test instance at 0x011DF5A8>>
>>>

Answer 2

As I said in my comment above, the problem is in get_callable_name. get_callable_name(t.TestFunc) yields 'Test.TestFunc' which is obviously wrong. It should be 't.TestFunc'. I added variable_name_in_module and used that in get_callable_name and now the code works. The check at the bottom returns True. However, variable_name_in_module is very hackish and I couldn't find a way to do this in a cleanly.

If you only need this for smallish stuff then this should be fine, but be aware that variable_name_in_module incurs a N dictionary lookup for every call to get_callable_name where N is the number of variables in the module.

Code follows:

def variable_name_in_module(module, var):
    for name in dir(module):
        if getattr(module, name) == var:
            return name

def get_callable_name(func):
    """
    Returns the best available display name for the given function/callable.
    """
    f_self = getattr(func, '__self__', None) or getattr(func, 'im_self', None)
    if f_self and hasattr(func, '__name__'):
        if isinstance(f_self, type):
            # class method
            clsname = getattr(f_self, '__qualname__', None) or f_self.__name__
            return '%s.%s' % (clsname, func.__name__)
        # bound method
        return '%s.%s' % (variable_name_in_module(__import__(f_self.__module__), f_self), func.__name__)
    if hasattr(func, '__call__'):
        if hasattr(func, '__name__'):
            # function, unbound method or a class with a __call__ method
            return func.__name__
        # instance of a class with a __call__ method
        return func.__class__.__name__
    raise TypeError('Unable to determine a name for %s -- '
                    'maybe it is not a callable?' % repr(func))


def obj_to_ref(obj):
    """
    Returns the path to the given object.
    """
    ref = '%s:%s' % (obj.__module__, get_callable_name(obj))
    try:
        obj2 = ref_to_obj(ref)
        if obj != obj2:
            raise ValueError
    except Exception:
        raise ValueError('Cannot determine the reference to %s' % repr(obj))
    return ref


def ref_to_obj(ref):
    """
    Returns the object pointed to by ``ref``.
    """
    if not isinstance(ref, basestring):
        raise TypeError('References must be strings')
    if not ':' in ref:
        raise ValueError('Invalid reference')
    modulename, rest = ref.split(':', 1)
    try:
        obj = __import__(modulename)
    except ImportError:
        raise LookupError('Error resolving reference %s: '
                          'could not import module' % ref)
    try:
        for name in modulename.split('.')[1:] + rest.split('.'):
            obj = getattr(obj, name)
        return obj
    except Exception:
        raise LookupError('Error resolving reference %s: '
                          'error looking up object' % ref)

class Test:
    def TestFunc(self):
        print "test"

t = Test()
print t.TestFunc == ref_to_obj(obj_to_ref(t.TestFunc))

Edit: PS: variable_name_in_module should probably throw an exception if it can't find anything, though I don't see how that can happen.