Free log files software for linux

NewSyslog is a highly configurable program for managing and archiving log files.
Main features:
- It is more portable (using GNU Autoconf) and it can be compiled and installed on most any modern Unix or Unix-like system.
- It has support for fixed time-of-day daily archiving with a command-line option to identify the daily roll-over invocation (which may be at midnight, or at any other regular daily time).
- It supports the FreeBSD feature that allows specification of the log roll-over time as a daily, weekly, or monthly interval (with optional time-of-day specification for the last two). [The other overly flexible, ISO 8601 interpretation of the interval "@" option is not supported -- it is too generic and not meaningful enough in the context of log file management.]
- It supports optional PID files so that non-standard daemons can be told to re-open their logfiles after archiving has taken place. (Including /dev/null which disables signalling of any daemon when the specified log file is rolled over.)
- It can send a signal other than SIGHUP to the daemon associated with a given log file.
- It can leave the most recently archived log file uncompressed, which is necessary for daemons like httpd and smail because they continue to write to the current log file until their current jobs have completed. (This also makes it much easier to review recent log data with normal Unix tools.) [NetBSD now has this feature.]
- It supports the FreeBSD feature of being able to restrict processing to just those log files specified on the command line.
- Unlike the NetBSD version it first parses the config file before taking any action, meaning that if any errors are encountered it will report them and quit without doing anything.
- Unlike the FreeBSD version, it will roll a log file if either the interval or size limits have been reached (FreeBSDs version makes it too easy to have a rapidly growing log file overflow the filesystem).
- Unlike the NetBSD version it always creates any missing log file (though this can be disabled on a per-file basis).
- It uses an advisory lock on the current configuration file to prevent multiple invocations from tripping over each other.
- The documentation is far better!

LMon is a package for near real-time monitoring of logs, sending email alerts upon known (rule hits) or unknown data (rule misses).
It features buffering of multiple rule hits within a given interval, cap at a given maximum number of lines, wait for a given interval before sending next alert, and auto- discovery of log rotation.
It can be run from the command line without configuration, or be controlled from a central configuration file with multiple instances monitoring different log files/sending alerts to different people.
Main features:
- Buffer multiple rule hits within a given interval, cap at a given maximum number of lines, wait for a given interval before sending next alert.
- Auto-discovery of log rotation.
- Simplicity. LMon can run from the command line without configuration, or be controlled from a central configuration file with multiple instances monitoring different log files/sending alerts to different people. It is very much intended to be simple (Keep It Simple, Stupid).

OpenWebStats is a PHP stats application that reads Apache log files and imports the data to a MySQL database.
It then provides an awstats-style reporting tool so that you can review stats on demand from the database. Stats updates are available through cron jobs or the Web interface.
Enhancements:
- Changed Year from hardcoded 2005 to a dynamic current and last year.
- Added full page views to hosts, referers, keywords, and pages
- Drastically improved the class_apache_logs_import.php to not readin the full log to an array. Thus no more memory errors.
- Updated the update now message with flush to show that it is running.

Kwatch is a graphical KDE 2.0 user interface for watching log files.
This utility is a port of kwatch 0.3 by Ralph C. Weichert to KDE2/3.
Basically it is a tail -f with a graphical KDE user interface for watching log files.
Enhancements:
- kwatch/kwatchconf.h (DEFAULT_IDLETIMEOUT): default set to 300 s.
- kwatch/prefs.cpp (kwatchPrefs): add input field "Reopen idle timeout".
- (updateConf): set m_param3 from config.
- (changeParam3): new slot for m_param3.
- kwatch/prefs.h (class kwatchPrefs): added m_param3 member with corresponding changeParam3() slot.
- kwatch/klogwidget.cpp (KLogWidget): use config idleTimeout. (timeout): dito.
- kwatch/kwatchconf.cpp (save): save new idleTimeout. (read): read new idleTimeout.
- kwatch/kwatchconf.h (class kwatchConf): added idleTimeout member.
- (DEFAULT_IDLETIMEOUT): default is 30 seconds.
- kwatch/klogwidget.cpp (KLogWidget): start m_idle with 30s.
- (timeout_idle): new slot for timer m_idle timeout signal.
- (timeout): restart m_idle, if text was read from a log file.
- kwatch/klogwidget.h (class KLogWidget): added new QTimer *m_idle member.
- kwatch/klogwidget.cpp (createPopupMenu): removed call to
- QTextEdit::createPopupMenu(), use QCursor::pos(). The position passed in arg pos is the relative position in the KLogWidget, not the absolute position on screen.

Squidview is a program that displays the squid proxy server log file in a nice fashion, providing the log file is in squids native reporting fashion. It has features such as search, report generation, monitor mode and supports three log files.

Thus, the program can be used to monitor Internet usage on a networked site. But please note squid has to be running first and this program is not a proactive resource controller. What it can do is tell you who and which sites are consuming the most bandwidth.

Installation:

If "configure" and "make" are successful:

make install-strip

"make install-strip" will provide a smaller executable than "make install" because the latter has quite a bit of debugging information with it.

Or if you arent the superuser the program will still run if you have read access to the squid log file.

This program assumes the main squid log file is here:
/usr/local/squid/var/logs/access.log

If it isnt just symlink it appropriately from the following default. This directory and links are made in the users home space when squidview is first run by the user.

graeme@localhost:~/.squidview$ ls -l
HOWTO -> /usr/local/share/squidview/HOWTO
log1 -> /usr/local/squid/var/logs/access.log
log2 -> log1
log3 -> log1
users
words

In this case squidview expects the primary log to be "log1", and the symlink redirects it to where the file actually is. Three log locations are supported which gives you a bit of room for accomodating rotating logs.

Visitors is a very fast web log analyzer for Linux, Windows, and other Unix-like operating systems. It takes as input a web server log file, and outputs statistics in form of different reports. The design principles are very different compared to other software of the same type:
No installation required, can process up to 150,000 lines of log entries per second in fast computers (20MB/s with my log files average length).
Designed to be executed by the command line, output html and text reports. The text report can be used in pipe to less to check web stats from ssh.
Support for real time statistics with the Visitors Stream Mode introduced with version 0.3.
To specify the log format is not needed at all. Works out of box with apache and most other web servers with a standard log format (see the documentation for more information on the format).
Its a portable C program, can be compiled on many different systems. Binaries for Windows systems are in the Download section of this page.
The produced html report doesnt contain images or external CSS, is self-contained, you can send it by email to users.
Visitors is free software (and of course, freeware), under the terms of the GPL license. You dont need to pay to use it. Visitors is supported, if you want a custom version made directly by the original author for a modest price, contact me at antirez (at) invece.org. ISPs may take advantage of the high processing speed.
Main features:
- Requested pages.
- Requested images.
- Referers by hits and age.
- Unique visitors in each day.
- Page views per visit.
- Pages accessed by the Google crawler (and the date of googles last access on every page).
- Percentage of visits originated from Google searches for every day.
- Users navigation patterns (web trails).
- Keyphrases used in Google searches.
- User agents.
- Weekdays and Hours distributions of accesses.
- Weekdays/Hours combined bidimentional map.
- Month/Year combined bidimentional map.
- Visual path analysis with Graphviz.
- Operating systems, browsers and domains popularity.
- 404 errors.
Enhancements:
- This release adds an important bugfix in the unique visitors algorithm.
- The output is now nearer to reality (though unique visitors stats are always a guess without the use of a cookie).

LogServ is a module for NeoStats IRC Services that logs channel actions and messages to a log file for processing by channel log file parsers such as pisg, Onis, or MircStats.
LogServ can enable an IRC network to host the channel information Web pages commonly found for popular channels detailing information such as the number of people that have joined an IRC channel, or who spoke the most words.
Enhancements:
- Many internal code cleanups and bugfixes were done.
- This release is compatible with the latest alpha release of NeoStats 3.0.

gpss is a gnuplot front end for analyzing snort perfmonitor log files.

It offers both 2D and 3D interactive or batch graphing of things like bits per second, packets per second, drop percentage, fragments per second, and more.

gpss project is useful as both a performance evaluation tool or a SNOC tool in the short term, and a trending tool for longer terms.

Usage:

For a list of arguments use the --help flag.

./gpss --tutorial | less
./gpss --faq | less

Will get you to either the tutorial or the faq. A short version however is this, and it works for me.

./gpss tlgraph

Assuming your on a machine with a /var/snort/snort.stats, youll get a nice 3d graph to play with on your X display. Itll look something like the following, but you can rotate it around. By default itll check back with the file every 5 seconds and replot if there is new data.