Mastering Regular Expressions (2nd Edition) [Electronic resources] نسخه متنی

Let's say that now we want to match sets of URL and link text from a document,
such as pulling the marked items from:

     ···<a href=" class="docEmphUl">
http://www.oreilly.com"> class="docEmphUl">O'Reilly
 And Associates</a>···

Because the contents of an <A>> tag can be fairly complex, I would approach this
task in two parts. The first is to pluck out the "guts" of the <A>> tag, along with the
link text, and then pluck the URL itself from those <A>> guts.

A simplistic approach to the first part is a case-insensitive, dot-matches-all application
of
<a\b([^>]+)>(.*?)</a>
>, which features the lazy star quantifier. This puts
the <A>> guts into $1> and the link text into $2>. Of course, as earlier, instead of

[^>]+
> I should use what we developed in the previous section. Having said that,
I'll continue with this simpler version, for the sake of keeping that part of the
regex shorter and cleaner for the discussion.

Once we have the <A>> guts in a string, we can inspect them with a separate regex.
In them, the URL is the value for the href=>
class="docEmphasis">value attribute. HTML allows spaces on
either side of the equal sign, and the value can be quoted or not, as described in the previous section. A solution is shown as part of this Perl snippet to report on
links in the variable $Html:>

     #  class="docEmphasis">Note: the regex in the while(...)
 is overly simplisticsee text for discussion
while ($Html =~ m{<a\b([^>]+)>(.*?)</a>}ig)
{
my $Guts = $1; #  class="docEmphasis">Save
 results from the match above, to their own . . .
my $Link = $2; #  class="docEmphasis">
. . . named variables, for clarity below.
if ($Guts =~ m{
\b HREF           #  "href" attribute
\s* = \s*         #  "=" may have whitespace on either side
(?:               #  Value is···
"([^"]*)"       #    double-quoted string,
|               #    or···
'([^']*)'       #    single-quoted string,
|               #    or···
([^'">\s]+)     #    "other stuff"
)                 #
}xi)
{
my $Url = $+; #  class="docEmphasis">
Gives the highest-numbered actually
-filled $1, $2, etc.
print "$Url with link text: $Link\n";
}
}

     if ($url =~ m{^http://([^/:]+(:(\d+))?)(/.*)?$}i)
{
my $host = $1;
my $port = $3 || 80;  #  class="docEmphasis">
Use $3 if it exists; otherwise default to 80.
my $path = $4 || "/"; #  class="docEmphasis">
Use $4 if it exists; otherwise default to "/".
print "host: $host\n";
print "port: $port\n";
print "path: $path\n";
} else {
print "not an http url\n";
}

     Imports System.Text.RegularExpressions
·
·
·
'  class="docEmphasis">Set up the regular 
expressions we'll use in the loop
Dim A_Regex as Regex = New Regex(                   _
"<a\b(?<guts>[^>]+)>(?<Link>.*?)</a>",  _
RegexOptions.IgnoreCase)
Dim GutsRegex as Regex = New Regex( _
"\b HREF                (?#  'href' attribute             )" &  _
"\s* = \s*              (?#  '=' with optional whitespace )" &  _
"(?:                    (?#  Value is ...                 )" &  _
"  "(?<url>[^"]*)"   (?#    double-quoted string,      )" &  _
"  |                    (?#    or ...                     )" &  _
"  '(?<url>[^']*)'      (?#    single-quoted string,      )" &  _
"  |                    (?#    or ...                     )" &  _
"  (?<url>[^'">\s]+)   (?#    'other stuff'              )" &  _
")                      (?#                               )",   _
RegexOptions.IgnoreCase OR RegexOptions.IgnorePatternWhitespace)
'   class="docEmphasis">Now check the 'Html' Variable . . .
Dim CheckA as Match = A_Regex.Match(Html)
'   class="docEmphasis">For each match within . . .
While CheckA.Success
'  class="docEmphasis">We matched an <a> tag, 
so now check for the URL.
Dim UrlCheck as Match = _
GutsRegex.Match(CheckA.Groups("guts").Value)
If UrlCheck.Success
'  class="docEmphasis">We've got a match, so have a URL/link pair
Console.WriteLine("Url " & UrlCheck.Groups("url").Value & _
" WITH LINK " & CheckA.Groups("Link").Value)
End If
CheckA = CheckA.NextMatch
End While
_____________________________________________________________________

     ^
(?:i)  # apply this regex in a case-insensitive manner.
# One or more dot-separated parts···
(?: [a-z0-9]\. | [a-z0-9][-a-z0-9]*[-a-z0-9]\. )+
# Followed by the final suffix part···
(?: com|edu|gov|int|mil|net|org|biz|
info|name|museum|coop|aero|[a-z][a-z] )
$

     ^
(?:i)  # apply this regex in a case-insensitive manner.
# One or more dot-separated parts···
(?: [a-z0-9]\. | [a-z0-9][-a-z0-9]{0,61}[-a-z0-9]\. )*
# Followed by the final suffix part···
(?: com|edu|gov|int|
mil|net|org|biz|info|name|museum|coop|aero|[a-z][a-z] )
$

     ···visit us at www.oreilly.com or mail to orders@oreilly.com.

     (?i: [a-z0-9] (?:[-a-z0-9]*[a-z0-9])? \. )+ 
#  class="docEmphasis">sub domains
#  class="docEmphasis">Now ending .com, etc. 
For these, we require lowercase
(?-i: com\b
| edu\b
| biz\b
| org\b
| gov\b
| in(?:t|fo)\b  #  class="docEmphasis">.int or .info
| mil\b
| net\b
| name\b
| museum\b
| coop\b
| aero\b
| [a-z][a-z]\b  #  class="docEmphasis">two-letter country codes
)

     \b
#  class="docEmphasis">Match the
 leading part (proto://hostname, or just hostname)
(
#  class="docEmphasis">ftp://, http://, or https:// leading part
(ftp|https?)://[-\w]+(\.\w[-\w]*)+
|
#  class="docEmphasis">or, try to find a
 hostname with our more specific sub-expression
full-hostname-regex
)
#  class="docEmphasis">Allow an optional port number
( : \d+ )?
#  class="docEmphasis">
The rest of the URL is optional, and begins with / . . .
(
/  class="docEmphasis">path-part
)?

     Read his comments 
at http://www.oreilly.com/ask_tim/indexl. He ...

     \b
#  class="docEmphasis">Match the leading
 part (proto://hostname, or just hostname)
(
#  class="docEmphasis">ftp://, http://, or https:// leading part
(ftp|https?)://[-\w]+(\.\w[-\w]*)+
|
#  class="docEmphasis">or, try to find a 
hostname with our more specific sub-expression
(?i: [a-z0-9] (?:[-a-z0-9]*[a-z0-9])? \. )+ #  
class="docEmphasis">sub domains
#  class="docEmphasis">Now ending 
.com, etc. For these, require lowercase
(?-i: com\b
| edu\b
| biz\b
| gov\b
| in(?:t|fo)\b #  class="docEmphasis">.int or .info
| mil\b
| net\b
| org\b
| [a-z][a-z]\b #  class="docEmphasis">two-letter country codes
)
)
#  class="docEmphasis">Allow an optional port number
( : \d+ )?
#  class="docEmphasis">The rest of 
the URL is optional, and begins with / . . .
(
/
#  class="docEmphasis">The rest are 
heuristics for what seems to work well
[^;"'<>()\[\]{}\s\x7F-\xFF]*
(?:
[.,?]+ [^;"'<>()\[\]{}\s\x7F-\xFF]+
)*
)?