Python Cookbook 2Nd Edition Jun 1002005 [Electronic resources] نسخه متنی

Recipe 16.10. Referring to a List Comprehension While Building It

Credit: Chris Perkins

Problem

You want to refer, from inside
a list comprehension, to the same list object you're
building. However, the object being built by the list comprehension
doesn't have a name while you're
building it.

Solution

Internally, the Python interpreter does create a
"secret" name that exists only
while a list comprehension is being built. In Python 2.3, that name
is usually '_[1]' and refers to the bound method
append of the list object we're
building. We can use this secret name as a back door to refer to the
list object as it gets built. For example, say we want to build a
copy of a list but without duplicates:

>>> L = [1, 2, 2, 3, 3, 3]
>>> [x for x in L if x not in locals( )['_[1]']._ _self_ _]
[1, 2, 3]

Python 2.4 uses the same name to indicate the list object being
built, rather than the bound-method access. In the
case of nested list comprehensions, inner ones are named
'_[2]', '_[3]', and so on, as
nesting goes deeper. Clearly, all of these considerations are best
wrapped up into a function:

import inspect
import sys
version_23 = sys.version_info < (2, 4)
def this_list( ):
import sys
d = inspect.currentframe(1).f_locals
nestlevel = 1
while '_[%d]' % nestlevel in d: nestlevel += 1
result = d['_[%d]' % (nestlevel - 1)]
if version_23: return result._ _self_ _
else: return result

Using this function, we can make the preceding snippet more readable,
as well as making it work properly in Python 2.4 as well as in
version 2.3:

>>> [x for x in L if x not in this_list( )]
[1, 2, 3]

Discussion

List comprehensions may look a little like magic, but the bytecode
that Python generates for them is in fact quite mundane: create an
empty list, give the empty list's bound-method
append, a temporary name in the locals dictionary,
append items one at a time, and then delete the name. All of this
happens, conceptually, between the open square bracket (
[ ) and the close square bracket ( ]),
which enclose the list comprehension.

The temporary name that Python 2.3 assigns to the bound
append method is '_[1]' (or
'_[2]', etc., for nested list comprehensions).
This name is deliberately chosen (to avoid accidental clashes) to
not be a syntactically valid Python identifier,
so we cannot refer to the bound method directly, by name. However, we
can access it as locals(
)['_[1]']. Once we have a
reference to the bound method object, we just use the bound
method's _ _self_ _ attribute to
get at the list object itself. In Python 2.4, the same name refers
directly to the list object, rather than to its bound method, so we
skip the last step.

Having a reference to the list object enables us to do all sorts of
neat party tricks, such as performing if tests
that involve looking at the items that have already been added to the
list, or even modifying or deleting them. These capabilities are just
what the doctor ordered for finding primes in a
"one-liner", for example: for each
odd number, we need to test whether it is divisible by any prime
number less than or equal to the square root of the number being
tested. Since we already have all the smaller primes stored and, with
our new parlor trick, have access to them, this test is a breeze and
requires no auxiliary storage:

import itertools
def primes_less_than(N):
return [ p for p in itertools.chain([2], xrange(3,N,2))
if 0 not in itertools.imap(
lambda x: p % x, itertools.takewhile(
lambda v: v*v <= p, this_list( ) ))]

The list comprehension that's the whole body of this
function primes_less_than, while long enough not to
fit into a single physical line, is all in a
single logical line (indeed, it must be, since any list comprehension
is a single expression), and therefore qualifies as a
"one-liner" if you squint in just
the right way.

This simple prime-finding algorithm is nowhere near as fast as the
Sieve of Eratosthenes shown in Recipe 18.10, but the ability to fit the
entire algorithm inside a single expression is nevertheless kind of
neat. Part of its neatness comes from the just-in-time evaluation
that the functions from standard library module
itertools perform so nicely.

Alas, this neat trick definitely cannot be
recommended for production code. While it works in Python 2.3 and
2.4, it could easily break in future releases, since it depends on
undocumented internals; for the same reason, it's
unlikely to work properly on other implementations of the Python
language, such as Jython or IronPython. So, I suggest you use it to
impress friends, but for any real work, stick to clearer, faster, and
solid good old for loops!

See Also

Documentation for bound methods,
lists' append
method, and the itertools module in the
Library Reference and Python in a
Nutshell.