Snowlist 1.0.2

Snowlist generates an RDF (RSS 1.0) document from individual email messages or from a mailing list. Rather than producing one news item for each mail message, it will group topics into a single RSS item and present the list in a bulletin board like view.

This program generates RSS feeds from mailing lists, because it does not generate the usual one item per mail feeds that are next to useless. Snowlist is written in Perl and uses an SQL database backend.

Id recommend adding a dedicated user on your system that receives all the mailing list mail to make processing easier. Ill assume youre using the user "snowlist" through the rest of this manual.

Create the database table now. You can use the included script createdb.DBNAME.sql to create the table Snowlist uses, or you can use the script as a reference and create the table manually. Either way, in the end you must have a table that looks like in the sql example file and a user that can access it.

Before running the script edit it and change $YOUR_DB_USER to the desired value!

If you make it work on a database other than the included creation scripts it would be nice if you could send me a createdb.DBNAME.sql script. Right now there are scripts for: PostgreSQL.

Put the three scripts snowlistDB, snowlistOutput and SnowlistConfig.pm into ~snowlist/bin (or wherever you want them). Create a config directory snowlist/.snowlist/ and create the main config file, named config there. See the included file config.example how this should look like. The section names do not have any meaning to snowlist.

compressoutput generated feeds will be compressed with gzip. Useful to
lower server load and bandwidth consumption. Value is
"yes" or "no".
compressionlevel zlib compression level: 0 (fastest) - 8 (highest)

dbusername username for database login

dbpassword password for database login

maxthread maximum number of threads the RSS feed will contain

maxitemsperthread maximum number of mails per thread

msg_expire maximum age of messages kept. See section Database
cleanup below.


If you have created this file you can start to add your first mailing list. Id recommend to subscribe to the list, before you actually set up everything so youll get the sent mails delivered to the user snowlists standard mailbox and can verify that everything works well.

I assume that the mailinglist now forwards mails to the user snowlist and everything works fine.

Create a config for the list in ~snowlist/.snowlist/. You will need to choose a name for it that you need to use in various places from now on. Iwill call this ListID. We choose the name "testlist" for now. Create a file named testlist.config. See the included file examplefeed.config to get an idea how it should look like.

title the title of the resulting feed

link the homepage of this feed. You can set this to the mailing
lists home page for example.

description description of this feed the RSS reader will show

outputfile where to write the resulting feed. This location must be
writable to the user "snowlist" (or whatever user you chose).

maxthreads (optional) You can override the default settings
maxitemsperthreas in the main configuration for this specific feed.


You now need to redirect the list mail to the script snowlistDB. The most convenient way to do this is procmail I would say. Add a .forward file for the user "snowlist" and put "|/path/to/procmail" in there. Add a .procmailrc that contains a filter for the mailing list you subscribed. This filter must pipe its mail to "~snowlist/bin/snowlistDB testfeed". The first (and only) argument given to "snowlistDB" must be the ListID of this
feed. Its the same as the name of the config file for this feed. If you mix them up it will not work!

If you did everything right, mail that will arrive at snowlist@your.domain will now be written into the database. You can now generate the RSS feed.

Run "~snowlist/bin/snowlistOutput testfeed" (once again the first and only argument is the ListID) and the script will write the feed to the location you gave in the config file "testfeed.config".

You should add the last command to the users crontab to regenerate the feed hourly or in a desired interval. If you specify "-" as the outputfile in the config "snowlistOutput" will write the feed to standard output for further processing with other programs.

That should be all that is needed to get everything running. Please read the next section! It contains various things to consider before running a publicly accessable feed.