Free setup log files software for linux

IPTables log analizer displays Linux 2.4 iptables logs (rejected, acepted, masqueraded packets...) in a nice HTML page (it support rough netfilter logs but also Shorewall and Suse Firewall logs).

This page shall be easy to read and understand to reduce the manual analysis time.

This page containts statistics on packets and links to more detailled information on a given host, port, domain and so on.

To convice you, here is a typical syslog entry for iptables :

[IPTABLES DROP] : IN=ppp0 OUT= MAC= SRC=172.186.2.157 DST=193.253.186.217 LEN=36 TOS=0x00 PREC=0x00 TTL=115 ID=4775 PROTO=ICMP TYPE=8 CODE=0 ID=512 SEQ=3663

How does it work ?

A small deamon is launched by a user which can read iptables logs files. Each time a new packet is logged, the daemon insert a new row in the database.

The statistics and so on are elaborated by the PHP page itself.

Nmap Log Stripp3r program is intended to be a way to condense all, or some, of the IPs of a "random" nmap scan into a file for later usage.

Common uses are to be able to feed the file back into nmap with the -iL switch, or feeding it into another port or vulnerability scanner of your choice.

Stripp3r supports stripping the nmap log of all but the IPs of hosts running a certain service, a version of a service, or even an arbitrary banner, and writing them to a file.

This is intended to be a way to condense all the IPs of a "random" Nmap scan into a file for later useage. Common uses are to be able to feed the file back into Nmap its self with the -iL switch, or feeding it into another port or vulnerability scanner.

Useage: ./stripp3r < logfile > < output > "< version string >" -v

Pretty simple. First, you must run an Nmap scan, on random hosts.

Ex. nmap -p 80 -sV -v -iR 500000 -oN nmaplogfile.nmap

This will tell nmap to do a scan service scan of 500,000 random IP addresses for the port 80, vobosely, and save the log to a file named nmaplogfile.nmap. You can change this around, eg, scanning a different service port (if say, you were looking for computers running FTP, you would scan for port 21 instead of 80 for HTTP), scanning a different number of hosts (500,000 or so is good, takes a few hours ususally though), or saving the log file to a different filename.

Nmap will then save a list of hosts that were "up" to a log file, with some informaiton about them, specifically weather the port that you specified was open, closed, or filtered. We are interested in "open" ports, so by default, Stripp3r will take all the log
enteries that have the port your specified listed as "open" and condense them into a file, listing only the IPs, one on each line.

Ex. ./stripp3r nmaplogfile.nmap output.ips

You can be more specific, and have Stripp3r put only the IPs that are running a certain service in the output file. The service string will only register the strings matching EXACTLY, so be careful to get the case and such correct.

apache httpd 1.3.27 (wont work)
Apache 1.3.27 (wont work)
Apache httpd 1.3.27 (works!)

Ex. ./stripp3r nmaplogfile.nmap output.ips "Apache httpd 1.3.27"

If you want to try it with verbosity, say

Ex. ./stripp3r nmaplogfile.nmap output.ips "Apache httpd 1.3.27" -v

And stripp3r will print out what it finds, along with writing it to the file.

You may change, copy, and reproduce this file, as long as the author is given credit for the initial writing of the code.

pflogx is a simple tool that exports OpenBSD packet filter logs to XML files.
pflogx reads a binary log file generated by the pf logging daemon (pflogd) and generates a human-readable and exploitable XML file.
Using an XSLT processor you can convert this XML file to any other format, such as HTML, CSV, or SQL.
Installation:
Installation of pflogx is quite simple. Just type the following commands to compile it and to install it in /usr/local/bin directory:
# make
# cp src/pflogx /usr/local/bin/pflogx
Usage:
Executed without options, pflogx reads packet filter logfile from standard input and generates an XML file containing all entries of logfile to standard output.
Available options are detailed in the next section.
Output XML file contains the following fields:
- Date,
- Interface name,
- Action,
- Rule number,
- Direction,
- Protocol,
- Source address,
- Source port,
- Destination address,
- Destination port.
The last four fields are only defined when protocol is TCP or UDP.
Enhancements:
- The code was fixed to allow building with GCC 2.
- An XSLT file was added for generating XHTML 1.0 Strict files from the output XML file.
- The existing XSLT file for generating HTML was updated to generate HTML 4.01 Strict files.

tichulog-ng is an application to view a previously played tichu game from a log file. It aims to be better than the tichulog tool available at http://tichu-gilde.bsw-portal.de/logs/.

tichulog-ng allows the user to view a logged game with open cards as if one would play the game by itself.

Usage:

unzip tichulog-ng.zip
java -jar tichulog-ng.jar
select the right logfile (*.tch)
Take cup of coffee (depends on the round size in the logfile)

pgFouine is a PostgreSQL log analyzer. It allows you to have a clear overview of your database activity, and can generate reports with slowest queries, most frequent queries, errors, graphs, and so on.

pgFouine is written in PHP and is designed to parse huge log files with a low memory footprint.

Example:
pgfouine.php -file path/to/your/log/file.log > report.html

For comprehensive usage information, just run:

pgfouine.php -help

loggerfs is the virtual file system that allows you to store logs in a database. I just released the first version of it (checkout the News) and will now be focusing on supporting more logging formats. If you dont yet know what loggerfs is, then heres a simple explanation for when you need it:

- Are you running cron scripts periodically to parse log files and put them in a database?
- Do you need to store log files remotely because you have limited space?
- Would you like a more efficient solution than simply polling the log files and waiting for new data?

If you answer yes to any of the above questions, please take a look at loggerfs. It will allow you to create virtual files to which you can then direct syslog/ apache/ squid/ etc. to log to. For example, instead of having syslog store authentication information in /var/log/auth.log, you could create a virtual file in /var/loggerfs/auth.log and then tell syslog to log to that file. Now instead of storing the information in the auth.log file, the data is actually sent to a database server that you defined in the logs.xml file. Every time new information is sent to the file, it is immediately sent to the database server, which means that:

- It doesnt poll for information, it listens, which makes it a lot more efficient.
- It automatically creates the tables in the database and makes sure that all the information can be stored.
- Youll be able to easily sort/ analyze the log files once theyre in the database.

I encourage you to checkout the CVS on the sourceforge project page, and let me know if you have any questions/ problems/ suggestions.

Querylog project is a console tool for performing SQL queries on a (log) file.

Lines from one or more text files or stdin are matched, using regular expressions to an in memory database on which SQL queries can be performed.

You also specify queries in the config file (SELECTs, INSERTs, CREATE VIEWs, etc). Queries that generate output are printed to stdout in plain text at the moment. In the future it will be possible to specify output formatters. The tool is written in C++ using the boost program options and regex library and the sqlite libraries for the in memory database.

I first wrote this tool to extract accounting information from cups page log files (in which accounting infomation was on different lines than job information), but due to the generic nature of the tool it can be used in many situations in which specific information needs to be retrieved from (in the future multiple) text files and presented in a more usable format.

Building:

Youll need the boost headers and libraries for (program_options and regex) which you can get at http://www.boost.org/, and the sqlite3 headers and libraries which you can get at http://www.sqlite.org/.

Im using boost build, so if you have that all you have to do is run:

> bjam

Alternativly, as the program now has only a single source file, you can just use g++ to compile and link it.

Ill create a nicer build enviroment one of these days.

Running:

Options must be specified on the commandline or in a config file (key = ["]value["]). Run querylog --help for details. The input file may be ommited, in which case data will be read from stdin until the eof.