Google Hacks 2Nd Edition [Electronic resources] نسخه متنی

Hack 95. Loop Around the 10-Result Limit

If you want more than 10 results,
you'll have to loop .

The Google API returns only 10 results per query, plenty for some
queries, but for most applications, 10 results barely scratches the
surface. If you want more than 10 results, you're
going to have to loop, querying for the next set of 10 each time. The
first query returns the top 10. The next, 11 through 20. And so
forth. [Hack #92] that we
showed you in the previous hack. To get at more than the top 10
results, no matter the programming language you're
using, you'll have to create a loop.

Bear in mind that each and every query counts against your daily allotment. Loop three times and you've used up three queries. Ten, and you're down 10. While this doesn't seem like much given your quota of 1,000 queries a day, you'd be surprised how quickly you can reach the bottom of the cookie jar without knowing where they all went.

9.12.1. The Code

Save the following code to a text file named
looply.pl. Again, remember to replace
insert key here with your Google API key,
as explained in "Using Your Google API
Key" earlier in this chapter.

In addition to the Google API Developer's Kit, you'll need the SOAP::Lite Perl module [Hack #93] installed before running this hack.

The alterations to the previous hack needed to support looping
through more than the first 10 results are called out in bold.

#!/usr/local/bin/perl
# looply.pl
# A typical Google Web API Perl script.

# Usage: perl looply.pl <query>
# Your Google API developer's key.
my $google_key='insert key here ';
# Location of the GoogleSearch WSDL file.
my $google_wdsl = "./GoogleSearch.wsdl";

# Number of times to loop, retrieving 10 results at a time .

my $loops = 3; # 3 loops x 10 results per loop = top 30 results
use strict;
# Use the SOAP::Lite Perl module.
use SOAP::Lite;
# Take the query from the command line.
my $query = shift @ARGV or die "Usage: perl looply.pl <query>\n";
# Create a new SOAP::Lite instance, feeding it GoogleSearch.wsdl.
my $google_search = SOAP::Lite->service("file:$google_wdsl");

# Keep track of result number .

my $number = 0;

for (my $offset = 0; $offset <= ($loops-1)*10; $offset += 10) {
# Query Google.
my $results = $google_search ->
doGoogleSearch(
$google_key, $query, $offset , 10, "false", ", "false",
", "latin1", "latin1"
);

# No sense continuing unless there are more results .

last unless @{$results->{resultElements}};
# Loop through the results.
foreach my $result (@{$results->{'resultElements'}}) {
# Print out the main bits of each result.
print
join "\n",
++$number,
$result->{title} || "no title",
$result->{URL},
$result->{snippet} || 'no snippet',
"\n";
}
} Notice that the script tells Google which set of 10 results
it's after by passing an offset
($offset). The offset is increased by 10 each time
($offset += 10).

9.12.2. Running the Script

Run this script from the command line ["How to Run
the Hacks" in the Preface], passing it your Google
search:

$ perl looply.pl " query keywords "

9.12.3. The Results

Here's a sample run. The first attempt
doesn't specify a query and so triggers a usage
message and doesn't go any further. The second
searches for learning perl and prints out the
results. Output is just the same as for the
googly.pl script in the prior hack, but now the
number of results you net is limited only by your specified loop
count (in this case 3, netting 3 10 or 30 results).

$ perl googly.pl
Usage: perl looply.pl <query>

% perl looply.pl "learning perl "
1
oreilly.com -- Online Catalog: Learning Perl , 3rd Edition
http://www.oreilly.com/catalog/lperl3/
... Learning Perl , 3rd Edition Making Easy Things
Easy and Hard Things Possible By Randal<br> L. Schwartz, Tom Phoenix
3rd Edition July 2001 0-596-00132-0, Order Number ...
...
29
Intro to Perl for CGI
http://hotwired.lycos.com/webmonkey/98/47/index2al
... Some people feel that the benefits of learning
Perl scripting are few.<br> But ... part. That's right.
Learning Perl is just like being a cop. ...
30
WebDeveloper.com ®: Where Web Developers and Designers Learn How ...
http://www.webdeveloper.com/reviews/book6l
... Registration CreditCard Processing Compare Prices.
Learning Perl . Learning <br> Perl , 2nd Edition.
Publisher: O'Reilly Author: Randal Schwartz .. .

9.12.4. Hacking the Hack

Alter the value assigned to the $loops variable to
change the number of results. For instance, to loop 9 times and grab
the top 90 results, change things like so:

# Number of times to loop, retrieving 10 results at a time.
my $loops = 9 ; # 9 loops x 10 results per loop = top 90 results