Tuesday March 19, 2024

Basic eBay Search Parsing

Date: 08/05/2005
Author: Wayne Eggert

Introduction

This tutorial is meant to explain a basic way to parse eBay's search results to retrieve a page of auctions. This script may soon be depreciated as eBay is constantly changing how its search results are displayed to prevent such parsing of their pages, but techniques used in this tutorial could be carried over to other applications. This script does not use eBay's API, but rather it "scrapes" the content off of eBay's website (ie. window scraping).

Play Nice..
One thing you have to remember whenever you're window scraping is to be considerate. It's best if you can store the content you're pulling on your own site, whether it's in a database (if actual text) or in your file system (if images). Also be considerate when it comes to how often your program actually goes and and scrapes the website -- you don't want every visitor on your site to generate a full page download of CNN.com just to scrape out the latest article. So keep these things in mind as you develop these types of applications and everyone will be happy =)

Grabbing eBay's page
The first step is grabbing the entire search results page created from searching for an item on eBay. I created a variable called $searchstring that can be called from within the URL (ie. "ebayparser.php?searchstring=batman"). You could easily place a search textbox on your site and send it the search words that way.

<?
$URL 
"http://search.ebay.com/search/search.dll?query=".$searchstring."&sosortproperty=1&ht=1&from=R10&BasicSearch=";
$file fopen("$URL""r");
$r "";
do{
    
$data fread($file8192);
    
$r .= $data;
}
while(
strlen($data) != 0);
?>

Explanation:

  • $URL contains the full address to the result page of the item being searched for ($searchstring)
  • The entire data of the search results page is stored in $r

Let the parsing begin..
The next step might look harder than it is. All it does is look for every instance of a new HTML <table> and stores each instance (with whatever data followed it) into an array. Again, for a dynamic site that doesn't change it's layout too often, this works great. eBay, however, is usually changing things pretty often to prevent such window scrapers from using their servers -- they want people to purchase the eBay API to access the auction data. This makes sense for anyone trying to make a buck on commercial software that's integrated with eBay, but not for anyone who just wants some auction listings on their site.

<?
$URL 
"http://search.ebay.com/search/search.dll?query=".$searchstring."&sosortproperty=1&ht=1&from=R10&BasicSearch=";
$file fopen("$URL""r");
$r "";
do{
    
$data fread($file8192);
    
$r .= $data;
}
while(
strlen($data) != 0);

$ebayTABLEArray preg_split ("/<table.*?>/"$r);
?>

Explanation:

  • Seperate all new tables into array elements (using preg_split)
page1 page2 page3


Comments:
Anonymous Techdoser
Posted 01/18/13 2:37PM by Anonymous Techdoser
Possibly out of date for the new ebay, we will see. However, this is the BEST explanation of preg_split I have ever seen and I can see it's potential for many things. Many thanks
Re: Completed listings
Posted 03/23/08 11:23AM by AceBHound
I don't have specifics on how to pass the password credentials to a site, but I know it's possible because there are programs and applications out there that are capable of doing this.
Completed listings?
Posted 12/24/06 3:26AM by Anonymous Techdoser
Is it possible to search through completed listings? So, is it possible to log-in (without the visitor of my site knowing the log-in/password) and extract these auctions as well?
Thanks!
Posted 04/23/06 12:27AM by Anonymous Techdoser
Thanks for writing this article. I appreciate all the work you've put into it and wanted to say IT WORKS! :O)

Best article with a working example I have ever found online!