screen-scraper 3.0

screen-scraper project is a tool used to extract data from web sites. You might use screen-scraper for the following purposes:
- Data Mining and Extraction
- Data Migration
- Application Integration
- Business Intelligence
- Web Task Automation
- Portal Components
- Meta-Searching
- Archiving
The screen-scraper application consists of two primary pieces:
- Workbench: A graphical user interface provides an intuitive approach that allows you to designate pages and specific pieces of information to be extracted.
- Server: After using the workbench to designate the data to be scraped, screen-scraper can be run in a server mode, much like a database. External applications can then connect to screen-scraper, which will pull data off of the designated web sites, then return them to the calling application. For example, you might build a web-based application using Active Server Pages (ASP) or PHP that invokes screen-scraper to search for products found on an external web site in real-time.
Additionally, screen-scraper can be started in a non-graphical mode from the command line such that it can be scheduled or invoked on-demand.
screen-scraper can automate many of the tasks typically required when scraping data from web pages, such as tracking cookies, logging in to web sites, and traversing search results pages.
Depending on the programming languages and platforms you most prefer, screen-scraper is likely to be familiar to you. screen-scraper contains an internal scripting engine that supports:
- VBScript
- JScript
- Perl
- Interpreted Java
- JavaScript
- Python
When invoking screen-scraper externally take your pick from the following languages:
- Java
- PHP
- Anything COM-based (such as Active Server Pages, Visual Basic, and Visual C++)
- .NET (both Microsoft-based and Mono)
- Cold Fusion
Enhancements:
- Several bugfixes and minor features have been added, including automatic backup of the database, enhanced HTML rendering and HTML stripping, fixing an error that caused duplicate scripts to appear at times on import, and fixing multiple errors relating to international character sets and non-ASCII characters.