Free web spider software for linux

Silver Spider is a KDE theme like Fluxbox style.

Wallpaper von Deviant with copyright by dayvid.net

It is for 1024x768 screen resolution. Wallpaper is from Deviant with copyright by dayvid.net. The symbols style is "Amaranth by Michael Doches(Doches@linuxcult.com)"

Windowsdeco, Bar & Button style: Keramik

You have to take same changes in KDE Design Control to look the same as in the preview. Take a look at all the screenshots to know want you have to change (KDE Control, Look&Feel, Symbols, Extra..).Thats why it depends on KDE 3.3, also because the transparency effect for the control bar, and for the menues when do you like that.

Web Site Robot is an easy to use Web site builder with a blog tool, shopping cart, and online photo album. It is free and does not require additional software, HTML, programming, or a hosting company.
It enables secure online payments with credit cards. There is virtually unlimited storage space for HTML, documents, blogs, photos, music, videos, or any multimedia content.
Enhancements:
- The Robot is now able to discover and inform UPnP routers about its presence and the need to have a NAT port available for its Web server.
- In most SOHO environments with UPnP enabled routers, manual configuration is not needed.
- The operator just installs, registers the Robot and the Web site is online effortlessly.

TkSpider project is a game of solitaire is played with the equivalent of two packs of cards.

The last release includes several small new features, including a redo feature.

Some code, and the card images, taken from Jeff Godreys "Once in a Lifetime".

Spider, based on the card-game shipped with Windows XP

Web of Life is a life simulation game.

An isometric game done with C++ and SDL.

In this game you control some beings in an ecosystem, the goal is to make only that the beings you control survive.

Your beings should survive fighting with other beings, reproducting to make a massive attack and eating.
But sometimes you will have to eat some of your own live beings so that others could stay alive, well its life.

WWW::Spyder is a Perl module that acts like a web spider.

A web spider that returns plain text, HTML, and other information per page crawled and can determine what pages to get and parse based on supplied terms compared to the text in links as well as page content.

METHODS

$spyder->new()

Construct a new spyder object. Without at least the seed() set, or go_to_seed() turned on, the spyder isnt ready to crawl.

$spyder = WWW::Spyder->new(shift||die"Gimme a URL!n");
# ...or...
$spyder = WWW::Spyder->new( %options );

Options include: sleep_base (in seconds), exit_on (hash of methods and settings). Examples below.

$spyder->seed($url)

Adds a URL (or URLs) to the top of the queues for crawling. If the spyder is constructed with a single scalar argument, that is considered the seed_url.

$spyder->bell([bool])

This will print a bell ("a") to STDERR on every successfully crawled page. It might seem annoying but it is an excellent way to know your spyder is behaving and working. True value turns it on. Right now it cant be turned off.

$spyder->spyder_time([bool])

Returns raw seconds since Spyder was created if given a boolean value, otherwise returns "D day(s) HH::MM:SS."

$spyder->terms([list of terms to match])

The more terms, the more the spyder is going to grasp at. If you give a straight list of strings, they will be turned into very open regexes. E.g.: "king" would match "sulking" and "kinglet" but not "King." It is case sensitive right now. If you want more specific matching or different behavior, pass your own regexes instead of strings.

$spyder->terms( qr/bkings?b/i, qr/bqueens?b/i );

terms() is only settable once right now, then its a done deal.

$spyder->spyder_data()

A comma formatted number of kilobytes retrieved so far. Dont give it an argument. Its a set/get routine.

$spyder->slept()

Returns the total number of seconds the spyder has slept while running. Useful for getting accurate page/time counts (spyder performance) discounting the added courtesy naps.

$spyder->UA->...

The LWP::UserAgent. You can reset them, I do believe, by calling methods on the UA. Here are the initialized values you might want to tweak (see LWP::UserAgent for more information):

$spyder->UA->timeout(30);
$spyder->UA->max_size(250_000);
$spyder->UA->agent(Mozilla/5.0);

Changing the agent name can hurt your spyder b/c some servers wont return content unless its requested by a "browser" they recognize.

You should probably add your email with from() as well.

$spyder->UA->from(bluefintuna@fish.net);
$spyder->cookie_file([local_file])

They live in $ENV{HOME}/spyderCookie by default but you can set your own file if you prefer or want to save different cookie files for different spyders.

Web Perl module contains a set of useful routines for many webworking purposes.

This module was primarily made for UNIX/Linux-Systems. Parts of it cannot be used on other systems. E.g. the procedures for file locking demand systems that can use symlinks. If you use the modul on systems where symlinks cannot be used, fatal errors may happen.

ABSTRACT

This perl module serves users with several useful routines for many purposes, like generating webpages, processing CGI scripts, working with XML datafiles and net-connections. It also uses own variants of routines, that was invented first in the famous libraries CGI.pm and cgi-lib.pl.

INSTALLATION

If you dont have sufficient privileges to install web.pm in the Perl library directory, you can put web.pm into some convenient spot, such as your home directory, or in cgi-bin itself and prefix all Perl scripts that call it with something along the lines of the following preamble:

use lib /home/myname/perl/lib;
use web;

NLock

This routine allows to set a filelock across NFS-boundaries. The common used perl-routine flock() fails at this point, so this routine is a useable alternative for bigger file-systems. It uses the modular functions link() and unlink() to mark a file locked. In addition to this, it also gives the locked file a counter: A file that is locked for more than $web::MAX_LOCKTIME seconds will be freed by the next process that calls NLock() on this file. A calling process gets either 0 or 1 as a return value, where 1 is returned if the file-locking was successful. 0 is returned only if the process waits for more than $web::MAX_WAITLOCK seconds or if symlink() fails.

Example 1:

$filename = "data.txt";
NLock($filename);
open(f1,"$filename");
# do something
close f1;
NUnlock($filename);

Example 2:

#!/local/bin/perl5
use web;

$stat= &NLock("jump.pl");
print "Lock: stat= $statn";
$stat= &NLock("jump.pl");
print "Lock this file again: stat= $statn";
sleep 8;
$stat= &NLock("jump.pl");
print "Lock this file again: stat= $statn";
$stat= &NUnlock("jump.pl");
print "Unlock: stat= $statn";
exit;

X personal web server is a personal Web server that sits in your desktop notification area (as a tray icon).

X personal web server is a (much) simpler cousin of KPF, but doesnt require kicker and uses the GTK 2 library for its minimal user interface.