Python Wart with Exception and Stack Frame

Last Friday at work we were trying to figure out a strange behaviour with the destructor of some of our wrapper objects in Python. We used a lot of wrapper object in Python to handle object life cycle of our CORBA objects, as they don't have the built-in reference counting and garbage collection. So whenever we receive a CORBA object, we put a wrapper around it, and when the program goes outside the scope, destructor of the wrapper object will be called to clean up the corresponding CORBA object.

If you are from Java background, you know that it would not work because of the way Java does its garbage collection. It would work from a C++ programmer's view, if the wrapper object is declared on the stack. You'll think that it should work with Python as well, as (1) its main garbage collector is reference counting based (2) wrapper object will be so simple that a circular reference should never occur.

However it did not work as expected. I'll use the following code as an illustration:

class Foo:
    def __del__(self):
        print 'called!'

def bar():
    baz = Foo()
    raise 'Doh'

print 1
try:
    bar()
except:
    print 2
print 3

What should be the expected output? I was expecting something like:

1
called
2
3

Because the destructor of Foo should be called when you exit baz(), even if you are exiting via an exception. However the result is:

1
2
3
called

The destructor is only called at the end of the program, and not at when we expect it to be called! Python's reference counting garbage collection not working?

It turns out that there's another reference to baz when the program exited bar() via exception -- it's referenced in sys.exc_info()'s traceback frame's local variables. In fact, the entire frame is stored so none of the destructors of the local variables along the exception frames is called. Therefore, the reference of baz won't reach 0 and be collected until exception stack frame is cleared (when we call sys.exc_clear() or when another exception occurs.

For example, if we have this instead:

print 1
try:
    bar()
except:
    print 2
__import__('sys').exc_clear()
print 3

It will print out

1
2
called
3

It's really annoying as we've got a lot of code depending on local variables being cleaned up when program exited the function and all local variables dereferenced. Now the life cycle of those wrapper objects become indeterministic. So are the CORBA objects that they are trying to wrap. Because exceptions are actually quite frequent in our multi-threaded application servers, sys.exc_info() is often cleared, so wrappers are still destructed pretty soon afterwards. However this non-deterministic behaviour does make deugging much more difficult.

Any convenient solution?