Python Cookbook 2Nd Edition Jun 1002005 [Electronic resources] نسخه متنی

Recipe 7.5. Holding Bound Methods in a Picklable Way

Credit: Peter Cogolo

Problem

You need to pickle an
object, but that object holds (as an attribute or item) a bound
method of another object, and bound methods are not
picklable.

Solution

Say you have the following objects:

import cPickle
class Greeter(object):
def _ _init_ _(self, name):
self.name = name
def greet(self):
print 'hello', self.name
class Repeater(object):
def _ _init_ _(self, greeter):
self.greeter = greeter
def greet(self):
self.greeter( )
self.greeter( )
r = Repeater(Greeter('world').greet)

Were it not for the fact that r holds a bound method
as its greeter attribute, you could pickle
r very simply:

s = cPickle.dumps(r)

However, upon encountering the bound method, this call to
cPickle.dumps raises a
TypeError. One simple solution is to have each
instance of class Repeater hold, not a bound method
directly, but rather a picklable wrapper to it. For example:

class picklable_boundmethod(object):
def _ _init_ _(self, mt):
self.mt = mt
def _ _getstate_ _(self):
return self.mt.im_self, self.mt.im_func._ _name_ _
def _ _setstate_ _(self, (s,fn)):
self.mt = getattr(s, fn)
def _ _call_ _(self, *a, **kw):
return self.mt(*a, **kw)

Now, changing Repeater._ _init_ _'s
body to self.greeter =
picklable_boundmethod(greeter) makes the previous snippet
work.

Discussion

The Python Standard Library pickle module (just
like its faster equivalent cousin cPickle) pickles
functions and classes by namethis implies, in particular, that
only functions defined at the top level of a module can be pickled
(the pickling of such a function, in practice, contains just the
names of the module and function).

If you have a graph of objects that hold each other, not directly,
but via one another's bound methods (which is often
a good idea in Python), this limitation can make the whole graph
unpicklable. One solution might be to teach pickle
how to serialize bound methods, along the same lines as described in
Recipe 7.6. Another
possible solution is to define appropriate _ _getstate_
_ and _ _setstate_ _ methods to turn
bound methods into something picklable at dump
time and rebuild them at load time, along the
lines described in Recipe 7.4. However, this latter
possibility is not a good factorization when you have several classes
whose instances hold bound methods.

This recipe pursues a simpler idea, based on holding bound methods,
not directly, but via the picklable_boundmethod
wrapper class. picklable_boundmethod is written
under the assumption that the only thing you usually do with a bound
method is to call it, so it only delegates _ _call_
_ functionality specifically. (You could, in addition, also
use _ _getattr_ _, in order to delegate other
attribute accesses.)

In normal operation, the fact that you're holding an
instance of picklable_boundmethod rather than
holding the bound method object directly is essentially transparent.
When pickling time comes, special method _ _getstate_
_ of picklable_boundmethod comes into
play, as previously covered in Recipe 7.4. In the case of
picklable_boundmethod, _ _getstate_
_ returns the object to which the bound method belongs and
the function name of the bound method. Later, at unpickling time,
_ _setstate_ _ recovers an equivalent bound method
from the reconstructed object by using the getattr
built-in for that name. This approach isn't
infallible because an object might hold its methods under assumed
names (different from the real function names of the methods).
However, assuming you're not specifically doing
something weird for the specific purpose of breaking
picklable_boundmethod's
functionality, you shouldn't ever run into this kind
of obscure problem!

See Also

Library Reference and Python in a
Nutshell docs for modules pickle and
cPickle, bound-method objects, and the
getattr built-in.