Python Cookbook 2Nd Edition Jun 1002005 [Electronic resources] نسخه متنی

Recipe 1.6. Combining Strings

Credit: Luther Blissett

Problem

You have several small strings that you need to combine into one
larger string.

Solution

To join a sequence of small strings into one large string, use the
string operator join. Say that
pieces is a list whose items are strings, and you
want one big string with all the items concatenated in order; then,
you should code:

largeString = ''.join(pieces)

To put together pieces stored in a few variables, the
string-formatting operator % can often be even
handier:

largeString = '%s%s something %s yet more' % (small1, small2, small3)

Discussion

In Python, the
+ operator concatenates strings and therefore
offers seemingly obvious solutions for putting small strings together
into a larger one. For example, when you have pieces stored in a few
variables, it seems quite natural to code something like:

largeString = small1 + small2 + ' something ' + small3 + ' yet more'

And similarly, when you have a sequence of small strings named
pieces, it seems quite natural to code
something like:

largeString = ''
for piece in pieces:
largeString += piece

Or, equivalently, but more fancifully and compactly:

import operator
largeString = reduce(operator.add, pieces, '')

However, it's very important to realize that none of
these seemingly obvious solution is goodthe approaches shown
in the "Solution" are
vastly superior.

In Python, string objects are immutable. Therefore, any operation on
a string, including string concatenation, produces a new string
object, rather than modifying an existing one. Concatenating
N strings thus involves building and then
immediately throwing away each of N-1
intermediate results. Performance is therefore vastly better for
operations that build no intermediate results, but rather produce the
desired end result at once.

Python's string-formatting operator
% is one such operation, particularly suitable
when you have a few pieces (e.g., each bound to a different variable)
that you want to put together, perhaps with some constant text in
addition. Performance is not a major issue for this specific kind of
task. However, the % operator also has other
potential advantages, when compared to an expression that uses
multiple + operations on strings. % is more readable, once you get
used to it. Also, you don't have to call
str on pieces that aren't already
strings (e.g., numbers), because the format specifier
%s does so implicitly. Another advantage is that
you can use format specifiers other than %s, so
that, for example, you can control how many significant digits the
string form of a floating-point number should display.

When you have many small string pieces in
a sequence, performance can become a truly important issue. The time
needed to execute a loop using + or
+= (or a fancier but equivalent approach using the
built-in function reduce) grows with the square of
the number of characters you are accumulating, since the time to
allocate and fill a large string is roughly proportional to the
length of that string. Fortunately, Python offers an excellent
alternative. The join method of a string object
s takes as its only argument a sequence of
strings and produces a string result obtained by concatenating all
items in the sequence, with a copy of s
joining each item to its neighbors. For example,
''.join(pieces) concatenates all the items of
pieces in a single gulp, without
interposing anything between them, and ',
'.join(pieces) concatenates the items putting a comma and a
space between each pair of them. It's the fastest,
neatest, and most elegant and readable way to put a large string
together.

When the
pieces are not all available at the same time, but rather come in
sequentially from input or computation, use a list as an intermediate
data structure to hold the pieces (to add items at the end of a list,
you can call the append or
extend methods of the list). At the end, when the
list of pieces is complete, call ''.join(thelist)
to obtain the big string that's the concatenation of
all pieces. Of all the many handy tips and tricks I could give you
about Python strings, I consider this one by far
the most significant: the most frequent reason some Python programs
are too slow is that they build up big strings with
+ or +=. So, train yourself
never to do that. Use, instead, the ''.join
approach recommented in this recipe.

Python 2.4 makes a heroic attempt to ameliorate the issue, reducing a
little the performance penalty due to such erroneous use of
+=. While ''.join is still way
faster and in all ways preferable, at least some newbie or careless
programmer gets to waste somewhat fewer machine cycles. Similarly,
psyco (a specializing just-in-time [JIT] Python compiler found at
http://psyco.sourceforge.net/),
can reduce the += penalty even further.
Nevertheless, ''.join remains the best approach in
all cases.

See Also

The Library Reference and Python in
a Nutshell sections on string methods, string-formatting
operations, and the operator module.