Python Cookbook 2Nd Edition Jun 1002005 [Electronic resources] نسخه متنی

Recipe 9.1. Synchronizing All Methods in an Object

Credit: André Bjärb, Alex Martelli,
Radovan Chytracek

Problem

You
want to share an object among multiple threads, but, to avoid
conflicts, you need to ensure that only one thread at a time is
inside the objectpossibly excepting some methods for which you
want to hand-tune locking behavior.

Solution

Java offers such synchronization as a built-in feature, while in
Python you have to program it explicitly by
wrapping the object and its methods. Wrapping is
so general and useful that it deserves to be factored out into
general tools:

def wrap_callable(any_callable, before, after):
''' wrap any callable with before/after calls '''
def _wrapped(*a, **kw):
before( )
try:
return any_callable(*a, **kw)
finally:
after( )
# In 2.4, only: _wrapped._ _name_ _ = any_callable._ _name_ _
return _wrapped
import inspect
class GenericWrapper(object):
''' wrap all of an object's methods with before/after calls '''
def _ _init_ _(self, obj, before, after, ignore=( )):
# we must set into _ _dict_ _ directly to bypass _ _setattr_ _; so,
# we need to reproduce the name-mangling for double-underscores
clasname = 'GenericWrapper'
self._ _dict_ _['_%s_ _methods' % clasname] = {  }
self._ _dict_ _['_%s_ _obj' % clasname] = obj
for name, method in inspect.getmembers(obj, inspect.ismethod):
if name not in ignore and method not in ignore:
self._ _methods[name] = wrap_callable(method, before, after)
def _ _getattr_ _(self, name):
try:
return self._ _methods[name]
except KeyError:
return getattr(self._ _obj, name)
def _ _setattr_ _(self, name, value):
setattr(self._ _obj, name, value)

Using these simple but general tools, synchronization becomes easy:

class SynchronizedObject(GenericWrapper):
''' wrap an object and all of its methods with synchronization '''
def _ _init_ _(self, obj, ignore=( ), lock=None):
if lock is None:
import threading
lock = threading.RLock( )
GenericWrapper._ _init_ _(self, obj, lock.acquire, lock.release, ignore)

Discussion

As per usual Python practice, we can complete this module with a
small self-test, executed only when the module is run as main script.
This snippet also serves to show how the module's
functionality can be used:

if _ _name_ _ == '_ _main_ _':
import threading
import time
class Dummy(object):
def foo(self):
print 'hello from foo'
time.sleep(1)
def bar(self):
print 'hello from bar'
def baaz(self):
print 'hello from baaz'
tw = SynchronizedObject(Dummy( ), ignore=['baaz'])
threading.Thread(target=tw.foo).start( )
time.sleep(0.1)
threading.Thread(target=tw.bar).start( )
time.sleep(0.1)
threading.Thread(target=tw.baaz).start( )

Thanks to the synchronization, the call to
bar runs only when the call to
foo has completed. However, because of the
ignore= keyword argument, the call to
baaz bypasses synchronization and thus
completes earlier. So the output is:

hello from foo
hello from baaz
hello from bar

When you find yourself using the same single-lock locking code in
almost every method of an object, use this recipe to refactor the
locking away from the object's application-specific
logic. The key effect you get by applying this recipe is to
effectively replace each method with:

self.lock.acquire( )
try:
  # The "real" application code for the method
finally:
self.lock.release( )

This code idiom is, of course, the right way to express locking: the
try/finally statement ensures
that the lock gets released in any circumstance, whether the
application code terminates correctly or raises an exception.
You'll note that factory
wrap_callable returns a closure, which is carefully
coded in exactly this way!

To some extent, this recipe can also be handy when you want to
postpone worrying about a class' locking behavior.
However, if you intend to use this code for production purposes, you
should understand all of it. In particular, this recipe does
not wrap direct accesses (for getting or
setting) to the object's attributes. If you want
such direct accesses to respect the object's lock,
you need to add the try/finally
locking idiom to the wrapper's _ _getattr_
_ and _ _setattr_ _ special methods,
around the calls these methods make to the getattr
and setattr built-in functions, respectively. I
normally don't find that depth of wrapping to be
necessary in my applications. (The way I code, wrapping just the
methods proves sufficient.)

If you're
into custom metaclasses, you may be surprised that I do not offer a
metaclass for these synchronization purposes. However, wrapping is a
more dynamic and flexible approachfor example, an object can
exist in both wrapped (synchronized) and unwrapped (raw)
incarnations, and you can use the most appropriate one case by case.
You pay for wrapping's flexibility with a little bit
more runtime overhead at each method call, but compared to the large
costs of acquiring and releasing locks I don't think
this tiny extra overhead matters. Meanwhile, this recipe shows off,
and effectively reuses, a wrapper-closure factory and a wrapper class
that demonstrate how easy Python makes it to implement that favorite
design pattern of Aspect-Oriented Programming's
fans, the insertion of
"before-and-after" calls around
every call to an object's methods.

See Also

Documentation of the standard library modules
threading and inspect in the
Library Reference and Python in a
Nutshell.