Python Cookbook 2Nd Edition Jun 1002005 [Electronic resources] نسخه متنی

Recipe 20.15. Upgrading Class Instances Automatically on reload

Credit: Michael Hudson, Peter Cogolo

Problem

You are developing a Python
module that defines a class, and you're trying
things out in the interactive interpreter. Each time you
reload the module, you have to ensure that
existing instances are updated to instances of the new, rather than
the old class.

Solution

First, we define a custom metaclass, which ensures its classes keep
track of all their existing instances:

import weakref
class MetaInstanceTracker(type):
''' a metaclass which ensures its classes
 keep track of their instances '''
def _ _init_ _(cls, name, bases, ns):
super(MetaInstanceTracker, cls)._ _init_ _(name, bases, ns)
# new class cls starts with no instances
cls._ _instance_refs_ _ = [  ]
def _ _instances_ _(cls):
''' return all instances of cls which are still alive '''
# get ref and obj for refs that are still alive
instances =
 [(r, r( )) for r in cls._ _instance_refs_ _ if r( ) is not None]
# record the still-alive references back into the class
cls._ _instance_refs_ _ = [r for (r, o) in instances]
# return the instances which are still alive
return [o for (r, o) in instances]
def _ _call_ _(cls, *args, **kw):
''' generate an instance, and record it (with a weak reference) '''
instance = super(MetaInstanceTracker, cls)._ _call_ _(*args, **kw)
# record a ref to the instance before returning the instance
cls._ _instance_refs_ _.append(weakref.ref(instance))
return instance
class InstanceTracker:
''' any class may subclass this one, to keep track of its instances '''
_ _metaclass_ _ = MetaInstanceTracker

Now, we can subclass MetaInstanceTracker to obtain
another custom metaclass, which, on top of the instance-tracking
functionality, implements the auto-upgrading functionality required
by this recipe's Problem:

import inspect
class MetaAutoReloader(MetaInstanceTracker):
''' a metaclass which, when one of its classes is re-built, updates all
instances and subclasses of the previous version to the new one '''
def _ _init_ _(cls, name, bases, ns):
# the new class may optionally define an _ _update_ _ method
updater = ns.pop('_ _update_ _', None)
super(MetaInstanceTracker, cls)._ _init_ _(name, bases, ns)
# inspect locals & globals in the stackframe of our caller
f = inspect.currentframe( ).f_back
for d in (f.f_locals, f.f_globals):
if name in d:
# found the name as a variable is it the old class
old_class = d[name]
if not isinstance(old_class, mcl):
# no, keep trying
continue
# found the old class: update its existing instances
for instance in old_class._ _instances_ _( ):
instance._ _class_ _ = cls
if updater: updater(instance)
cls._ _instance_refs_ _.append(weakref.ref(instance))
# also update the old class's subclasses
for subclass in old_class._ _subclasses_ _( ):
bases = list(subclass._ _bases_ _)
bases[bases.index(old_class)] = cls
subclass._ _bases_ _ = tuple(bases)
break
return cls
class AutoReloader:
''' any class may subclass this one, to get automatic updates '''
_ _metaclass_ _ = MetaAutoReloader

Here is a usage example:

# an 'old class'
class Bar(AutoReloader):
def _ _init_ _(self, what=23):
self.old_attribute = what
# a subclass of the old class
class Baz(Bar):
pass
# instances of the old class & of its subclass
b = Bar( )
b2 = Baz( )
# we rebuild the class (normally via 'reload', but, here, in-line!):
class Bar(AutoReloader):
def _ _init_ _(self, what=42):
self.new_attribute = what+100
def _ _update_ _(self):
# compute new attribute from old ones, then delete old ones
self.new_attribute = self.old_attribute+100
del self.old_attribute
def meth(self, arg):
# add a new method which wasn't in the old class
print arg, self.new_attribute
if _ _name_ _ == '_ _main_ _':
# now b is "upgraded" to the new Bar class, so we can call 'meth':
b.meth(1)
# emits: 1 123
# subclass Baz is also upgraded, both for existing instances...:
b2.meth(2)
# emits: 2 123
# ...and for new ones:
Baz( ).meth(3)
# emits: 3 142

Discussion

You're probably familiar with the problem this
recipe is meant to address. The scenario is that
you're editing a Python module with your favorite
text editor. Let's say at some point, your module
mod.py looks like this:

class Foo(object):
def meth1(self, arg):
print arg

In another window, you have an interactive interpreter running to
test your code:

>>> import mod
>>> f = mod.Foo( )
>>> f.meth1(1)
1

and it seems to be working. Now you edit mod.py
to add another method:

class Foo(object):
def meth1(self, arg):
print arg
def meth2(self, arg):
print -arg

Head back to the test session:

>>> reload(mod)
module 'mod' from 'mod.pyc'
>>> f.meth2(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'Foo' object has no attribute 'meth2'

Argh! You forgot that f was an instance of the
old mod.Foo!

You can do two things about this situation. After reloading, either
regenerate the instance:

>>> f = mod.Foo( )
>>> f.meth2(2)
-2

or manually assign to f._ _class_ _:

>>> f._ _class_ _ = mod.Foo
>>> f.meth2(2)
-2

Regenerating works well in simple situations but can become very
tedious. Assigning to the class can be automated, which is what this
recipe is all about.

Class MetaInstanceTracker is a metaclass that tracks
instances of its instances. As metaclasses go, it
isn't too complicated. New classes of this metatype
get an extra _ _instance_refs_ _ class variable
(which is used to store weak references to instances) and an
_ _instances_ _ class method (which strips out dead
references from the _ _instance_refs_ _ list and
returns real references to the still live instances). Each time a
class whose metatype is MetaInstanceTracker gets
instantiated, a weak reference to the instance is appended to the
class' _ _instance_refs_ _ list.

When the definition of a class of metatype
MetaAutoReloader executes, the namespace of the
definition is examined to determine whether a class of the same name
already exists. If it does, then it is assumed that this is a class
redefinition, instead of a class definition, and
all instances of the old class are updated to
the new class.
(MetaAutoReloader inherits from
MetaInstanceTracker, so such instances can easily be
found). All direct subclasses, found through the old
class' intrinsic _ _subclasses_ _
class method, then get their _ _bases_ _ tuples
rebuilt with the same change.

The new class definition can optionally include a method _
_update_ _, whose job is to update the state (meaning the
set of attributes) of each instance, as the
instance's class transitions from the old version of
the class to the new one. The usage example in this
recipe's Solution presents a case in which one
attribute has changed name and is computed by different rules, as you
can tell by observing the way the _ _init_ _
methods of the old and new versions are coded; in this case, the job
of _ _update_ _ is to compute the new attribute
based on the value of the old one, then del the
old attribute for tidiness.

This recipe's code should probably do more thorough
error checking; Net of error-checking issues, this recipe can also
supply some fundamental tools to start solving a problem that is
substantially harder than the one explained in this
recipe's Problem statement: automatically upgrade
classes in a long-running application, without needing to stop and
restart that application.

Doing automatic upgrading in production code is more difficult than
doing it during development because many more issues must be
monitored. For example, you may need a form of locking to ensure the
application is in a quiescent state while a number of classes get
upgraded, since you probably don't want to have the
application answering requests in the middle of the upgrading
procedure, with some classes or instances already upgraded and others
still in their old versions. You also often encounter issues of
persistent storage because the application probably needs to update
whatever persistent storage it keeps from old to new versions when it
upgrades classes. And those are just two examples. Nevertheless, the
key component of such on-the-fly upgrading, which has to do with
updating instances and subclasses of old classes to new ones, can be
tackled with the tools shown in this recipe.

See Also

Docs for the built-in function reload in the
Library Reference and Python in a
Nutshell.