What could prevent a traceback from being garbage

2019-07-27 22:10发布

问题:

I can't show much code, but see this sample:

class Product(metaclass=abc.ABCMeta):
    __attributes__ = []
    __special_attributes__ = []
    __json_additional_attributes__ = []

    def __init__(self, original_object):
        for attribute in [c_attr for c_attr in self.__attributes__
                            if c_attr not in self.__special_attributes__]:
            try:
                if hasattr(original_object, attribute):
                    # ...
                elif original_object.IsAttributeUsed(attribute):
                    # RAISES HERE - this is a clr binding
                    # ...
            except Exception as err:
                print('cant retrieve attr ' + attribute)

    # ...

I've removed lines for concision. Here, original_object is an object obtained from a clr binding. Sometimes original_object.IsAttributeUsed(attribute) raises an exception on the DLL side. The exception is caught as expected in the except block.

The problem is: for some reason, it leaks memory because the traceback object from this exception never gets collected. The traceback holds reference to the whole frames in the stack which in turn reference all locals. This is a batch, so this stack holds up 8Gb of memory, so I run pretty quickly out of memory.

I've got the graph of the references thanks to the objgraph package, and the references path from leaked objects go up to that traceback object, then nothing. If no exception is thrown, then the memory is freed.

Here is the graph:

Sorry for big image, I've cropped. Stacks on the left are me playing with the debugger, don't mind them. I've highlighted the leak.