Hacks 1917 Industrial.. Strength Tips and Tools [Electronic resources] نسخه متنی

Hack 83 API Searches

Perform reliable searches with
GetSearchResults.

[Hack #17], a Perl script is used to perform
an automated eBay search and then email new listings as
they're discovered. Although the script serves a
valuable function, it has the notable handicap of relying entirely on
"scraping" (via the
WWW::Search::eBay module) to retrieve its search
results.

Scraping involves parsing standard web pages in order to retrieve the
desired data. As you might expect, any changes to
eBay's search pages, even minor ones, will break the
script until the WWW::Search::eBay module on which
it relies is updated to work with the new version.

The API, on the other hand, provides an officially supported
interface to eBay's search engine, which means that
scripts based on the API will be much more robust and nearly
invulnerable to changes in eBay's search pages.

8.3.1 A Simple Search

Here's a simple Perl script,
search.pl, that performs a search and displays the
results.

#!/usr/bin/perl
require 'ebay.pl';
use Getopt::Std;
getopts('d');
$keywords = shift @ARGV or die "Usage: $0 [-d] keywords";
PAGE:     [1]
while (1) {
my $rsp =  call_api({ Verb => 'GetSearchResults',     [2]
DetailLevel => 0,
Query => $keywords,
SearchInDescription => $opt_d ? 1 : 0,
Skip => $page_number * 100,
});
if ($rsp->{Errors}) {
print_error($rsp);
last PAGE;
}
foreach (@{$rsp->{Search}{Items}{Item}}) {     [3]
my %i = %$_;
($price, $time, $title, $id) = @i{qw/CurrentPrice EndTime Title Id/};     [4]
print "($id) $title [\$$price, ends $time]\n";
}
last PAGE unless $rsp->{Search}{HasMoreItems};     [5]
$page_number++;
}

Given that searches can return hundreds or even thousands of results,
the GetSearchResults API call (line [2]) divides the results into pages, not unlike
the search pages at eBay.com. The loop, which begins on line [1], repeatedly resubmits the call, downloading a
maximum of 100 results each time, until
$rsp->{Search}{HasMoreItems} is
false, on line [3]. That
means that if there are 768 matching listings,
you'll need to retrieve 8 pages, or make 8 API
calls.

Your script might not need to retrieve all matching search results, as this one does. Instead, you may be content to search until a single auction is found, or perhaps to search only the auctions that have started in the last 24 hours. See the API documentation for more ways to limit the result set.

For each page that is found, a secondary loop, line [5], iterates through the result set for the
current page, extracts relevant data (line [4]), and prints it out.

8.3.2 Performing a Search

Run the script to perform a title search, like this:

search.pl keyword

where keyword is the word
you're looking for. To search for multiple keywords,
enclose them in single quotes, like this:

search.pl 'wool mittens'

Or to search titles and descriptions, type:

search.pl -d 'wool mittens'

But the real beauty of API searches is how they can be used in an
automated fashion.

8.3.3 Revising the Robot

Now, if we tie the API search into the script from [Hack #17], we get the following new, more
robust search robot
script:

#!/usr/bin/perl
require 'ebay.pl';
$searchstring = "railex";
$searchdesc = 0;
$localfile = "search.txt";
$a = 0;
# *** perform search ***
PAGE:
while (1) {
my $rsp =  call_api({ Verb => 'GetSearchResults',
DetailLevel => 0,
Query => $keywords,
SearchInDescription => $opt_d ? 1 : 0,
Skip                => $page_number * 100,
});
if ($rsp->{Errors}) {
print_error($rsp);
last PAGE;
}
$current_time = $rsp->{eBayTime};
foreach (@{$rsp->{Search}{Items}{Item}}) {
my %i = %$_;
($title[$a], $itemnumber[$a]) = @i{qw/Title Id/};
write;
}
last PAGE unless $rsp->{Search}{HasMoreItems};
$page_number++;
}
# *** eliminate entries already in file ***
open (INFILE,"$localdir/$localfile");
while ( $line = <INFILE> ) {
for ($b = $a; $b >= 1; $b--) {
if ($line =~ $itemnumber[$b]) {
splice @itemnumber, $b, 1;
splice @title, $b, 1;
}
}
}
close (INFILE);
$a = @itemnumber - 1;
if ($a == 0) { exit; }
# *** save any remaining new entries to file ***
open (OUTFILE,">>$localdir/$localfile");
for ($b = 1; $b <= $a; $b++) {
print OUTFILE "$itemnumber[$b]\n";
}
close (OUTFILE);
# *** send email with new entries found ***
open(MAIL,"|/usr/sbin/sendmail -t");
print MAIL "To: $selleremail\n";
print MAIL "From: $selleremail\n";
print MAIL "Subject: New $searchstring items found\n\n";
print MAIL "The following new items have been listed on eBay:\n";
for ($b = 1; $b <= $a; $b++) {
print MAIL "$title[$b]\n";
print MAIL "http://cgi.ebay.com/ws/eBayISAPI.dll?
ViewItem&item=$itemnumber[$b]\n\n";
}
close(MAIL);

Note that the only difference in the search portion of this script is
that the title and item number are stored in
$title[$a] and $itemnumber[$a]
arrays instead of being printed out.

If you end up scheduling this script as described in [Hack #17], you may not need to retrieve all
matching search results each time. Probably the best way is to set
the Order input value to
MetaStartSort, which will retrieve newly listed
items first. Then, assuming you've scheduled your
search robot to run every 24 hours, you could then stop retrieving
results as soon as an auction older than 24 hours is encountered. Use
the $yesterday variable, introduced in [Hack #85], to do your date
calculations.