Perl Cd Bookshelf [Electronic resources] نسخه متنی

6.3.1. Problem

Think hard about what you want a word to be and what separates one
word from the next, and then write a regular expression that encodes
your decisions. For example:

/\S+/ # as many non-whitespace characters as possible
/[A-Za-z'-]+/ # as many letters, apostrophes, and hyphens

Because words vary between applications, languages, and input
streams, Perl does not have built-in definitions of words. You must
make them from character classes and quantifiers yourself, as we did
previously. The second pattern is an attempt to recognize
"shepherd's" and
"sheep-shearing" each as single words.

Most approaches have limitations because of the vagaries of written
language. For instance, although the second pattern successfully
identifies "spank'd" and
"counter-clockwise" as words, it also pulls the
"rd" out of "23rd
Psalm". To be more precise when pulling words out
from a string, specify the characters surrounding the word. Normally,
this should be a word boundary, not whitespace:

/\b([A-Za-z]+)\b/ # usually best
/\s([A-Za-z]+)\s/ # fails at ends or w/ punctuation

Although Perl provides \w, which matches a
character that is part of a valid Perl identifier, Perl identifiers
are rarely what you think of as words, since we mean a string of
alphanumerics and underscores, but not colons or quotes. Because it's
defined in terms of \w, \b may
surprise you if you expect to match an English word boundary (or,
even worse, a Mongolian word boundary).

\b and \B can still be useful.
For example, /\Bis\B/ matches the string
"is" within a word only, not at the edges. And
while "thistle" would be found,
"vis-à-vis" wouldn't.