Re: question about garbage collection

Hey,

On Sun, Jul 13, 2008 at 12:28 PM, killian koepsell <koepsell <at> gmail.com> wrote:

> thanks for the explanation. I have a better idea now, where to find
> possible references to my objects and how to get rid of them. I wasn't
> aware of the %reset function and I didn't know that I have to call
> gc.collect() manually (i thought that python takes care of the garbage
> collection automatically).

It does, but gc.collect() may happen at some arbitrary later time that
you have no control over.  The reason you can call it is that
sometimes you have a good reason to need a full collection cycle to be
run *now*, not when the VM gets around to it.

> There is still something I find puzzling in ipython's behavior when I
> run a script using %run as opposed to running it interactively. If I
> run the following script in ipython using the %run command, afterwards
> I have a reference to the object c in the interactive session.
> However, I can't figure out how to get rid of all references (without
> calling %reset):

There is in principle no automatic way of getting all object
references in python.  However, since most important variables you
care about live in a dict called _ip.user_ns, you could write a bit of
code to build the reverse mapping to find all the references bound to
a given object in that namespace:

refs = {}
for k in _ip.user_ns.keys():
    refs.setdefault(id(_ip.user_ns[k]),[k]).append(k)

The output is something like:

In [45]: refs
Out[45]:
{137440468: ['Demo'],
 138146912: ['oid'],
 1075654748: ['__builtins__'],
 1075654772: ['sys'],
 1075654964: ['os'],
 1075655756: ['ipapi'],
 1075656668: ['genutils'],
 1075782688: ['__name__'],
 1075796352: ['k'],
 1076058468: ['In', '_ih'],
 1076064716: ['__IP'],
etc...

It's easy to tweak this code to produce a simple dict which for every
key in the original ns shows you the other names that map to the same
id.

Here's a complete snippet you can run:

refs = {}
xrefs = {}

for k in _ip.user_ns.keys():
    refs.setdefault(id(_ip.user_ns[k]),[]).append(k)

for k,v in _ip.user_ns.items():
    try:
        xr = list(set(refs[id(v)]) - set([k]))
        if xr:
            xrefs[k] = xr
    except KeyError:
        # the key index from the previous loop may cause errors
        pass

On my system right now I get:

In [33]: xrefs
Out[33]:
{'In': ['_ih'],
 'Out': ['_oh', 'v'],
 '_': ['_31'],
 '_28': ['__', '___', '_29'],
 '_29': ['__', '___', '_28'],
 '_31': ['_'],
 '__': ['___', '_28', '_29'],
 '___': ['__', '_28', '_29'],
 '_i': ['_i31'],
 '_i29': ['_iii'],
 '_i30': ['_ii'],
 '_i31': ['_i'],
 '_ih': ['In'],
 '_ii': ['_i30'],
 '_iii': ['_i29'],
 '_ip': ['ip'],
 '_oh': ['v', 'Out'],
 'ip': ['_ip'],
 'v': ['_oh', 'Out']}

If you remove the 'if xr' staement and make it unconditional, you'll
get a table for *all* variables showing the list of other names they
have.  In this format the table only contains entries for variables
that have multiple names.

This should probably be a cookbook entry... (brownie points to whoever
adds it, I need to go to bed...)

Cheers,

f

ps - thanks for the ticket!