Free network statistics software for linux

RRDStats is a Coyote Linux and BrazilFW add-on package for network traffic monitoring, link quality control, and QOS classes monitoring.
RRD Statistics project is based on RRDtool for storing data to round robin databases, and a slightly modified RRDcgi for visualizing data through a Web interface.
Main features:
- Realtime graphical statistics for bandwidth usage and link quality
- Graphical statistics of QOS priority classes usage
- Historical data stored for one week
Configuration:
All default configuration is stored in /etc/rrd.config. This version supports web based configuration and there is no need to manual configuration for basic package functionality. Just install the packages and browse to your web administration interface (by default its http://192.168.0.1:8180). There should be new link at left menu labeled "RRDStats configuration"
There are some basic options you should set up to fit your configuration. First get sure, the RRDstats package is enabled (its the first option at configuration screen). After that should you set up your line speed (just some basic approximation is good enough). The last this you should set up is your internet gateway IP address. This IP address is used to measure your internet link latency and packet loss.
Ignore other configuration options for now, save your configuration and reboot router. After your system boots up, you can browse RRD statistics.
After system startup, package is initialiazed with /etc/rc.d/pkgs/rc.rrdstats. This file start another copy of tiny webserver which listens by default on port 8080. It reads its homepage files from /var/rrd/www/ directory. After webserver startup there are also started some data gathering threads.
They read transfered data from network interfaces, QOS classes and measure link latency. These values are then stored in RRD databases. RRD databases are by default stored in /var/rrd/data/ directory
For further information how RRD databases work, please visit their homepage. Simply said RRD database has constant size, it does not grow over time and stores average data over period of time.
Last component of RRDStats package are .cgi and template files which display data from RRD databases using web interface. As said before, these files and templates are stored in /var/rrd/www/ and its subdirectories.

Java Network Stack provides a Java library for research oriented network programming.

Java Network Stack is a library used by the DIMES project to create new internet measurements. It provides a clean API for packet manipulation, send, receive, filter, and analysis. It is a unification of raw socket capabilities, MAC level networkingm, and IPv6 capabilities.

Web Statistics Montage is a great web statistics tool to add to your arsenal. Firstly, you need to be using a web host that uses the cPanel control panel and that also has AWStats enabled. Since cPanel is the most popular control panel on the market for Linux servers, this script should be useful to the majority of web masters out there.
I built Web Statistics Montage so that I could get a comprehensive idea of how my web sites were faring with search engines without having to log into each account. With this script, you get e-mailed your stats whenever you want them, no log-ins required!
Main features:
I use this script a couple of times a day because its so handy. Here is a list of the features / capabilities of Web Statistics Montage:
- Compiles key web statistics from multiple sites and compiles it into an easy-to-read e-mail that gets sent to your inbox
- Compiles the following statistics:
- Google PageRank
- Number of unique visitors every day for the last few months
- What keywords were used by search engines to find your site since the last time a report was generated
- The most popular keywords used to get to your site through search engines
- The last time your site was spidered by Google, Yahoo and MSN
- The number of visits Google, Yahoo and MSN are sending your way
- Compiles statistics in easy-to-read, easy-to-compare grids, color coded for the different search engines
- Click on a keyword to show your sites rank for the keyword in Google, Yahoo and MSN instantly
- Click a link on the report to view your AWStats for any site without having to log in!
- Online version of the reports for easy comparison and later viewing

Statistics::SPC is a Perl module with calculations for Stastical Process Control (SPC).

Creates thresholds based on the variability of all data, # of samples not meeting spec, and variablity within sample sets, all from training data.
Note: this is only accurate for data which is normally distributed when the process is under control

Recommended usage: at least 15 sample sets, w/ sample size >=2 (5 is good) This module is fudged to work for sample size 1, but its a better idea to use >= 2

Important: the closer the process your are monitoring to how you would like it to be running (steady state), the better the calculated control limits will be.
Example: we take 5 recordings of the CPU utilization at random intervals over the course of a minute. We do this for 15 minutes, keeping all fifteen samples. Using this will be able to tell whether or not CPU use is in steady state.

SYNOPSIS

my $spc = new Statistics::SPC;
$spc->n(5) # set the number of samples per set
$spc->Uspec(.50); # CPU should not be above 50% utilization
$spc->Lspec(.05); # CPU should not be below 5%
# (0 is boring in an example)

# Now feed training data into our object
$return = $spc->history($history); # "train the system";
# $history is ref to 2d array;
# $return > 1 means process not likely to
# meet the constraints of your specified
# upper and lower bounds

# now check to see if the the latest sample of CPU util indicates
# CPU utilization was under control during the time of the sample

$return = $spc->test($data); # check one sample of size n
# $return < 0 there is something wrong with your data
# $return == 0 the sample is "in control"
# $return > 0 there are $return problems with the sample set

Network Traffic Analyzer analyzes the network traffic on multiple network devices and creates HTML statistics with some network usage graphs. Sometimes it is good to know, how the network is used, how many bytes were received and how many bytes were sent.Therefore, here is Network Traffic Analyzer, which creates simple network usage statistics.


Such statistics can tell you, how good your network connection really is (who cares about what Internet provider say, when was the network down, which time is the best time for downloading large packages of data etc. etc. Or with this software you can just better imagine, how many traffic can your home computer generate.

Statistics::Gap Perl module is an adaptation of the Gap Statistic.

SYNOPSIS

use Statistics::Gap;
$predictedk = &gap("prefix", "vec", INPUTMATRIX, "rbr", "h2", 30, 10, rep, 90, 4);

OR

use Statistics::Gap;
$predictedk = &gap("prefix", "vec", INPUTMATRIX, "rbr", "h2", 30, 10, rep, 90, 4, 7);

INPUTS

1. Prefix: The string that should be used to as a prefix while naming the intermediate files and the .dat files (plot files).
2. Space: Specifies the space in which the clustering should be performed. Valid parameter values: vec - vector space sim - similarity space
3. InputMatrix: Path to input matrix file. (More details about the input file-format below.)
4. ClusteringMethod: Specifies the clustering method to be used. (Learn more about this at: http://glaros.dtc.umn.edu/gkhome/cluto/cluto/overview)
Valid parameter values:
rb - Repeated Bisections
rbr - Repeated Bisections for by k-way refinement
direct - Direct k-way clustering
agglo - Agglomerative clustering
bagglo - Partitional biased Agglomerative clustering
NOTE: bagglo can be used only if space=vec
5. Crfun: Specifies the criterion function to be used for finding clustering solutions. (Learn more about this at: http://glaros.dtc.umn.edu/gkhome/cluto/cluto/overview)
Valid parameter values:
i1 - I1 Criterion function
i2 - I2 Criterion function
e1 - E1 Criterion function
h1 - H1 Criterion function
h2 - H2 Criterion function
6. K: This is an approximate upper bound for the number of clusters that may be present in the dataset.
7. B: The number of replicates/references to be generated.
8. TypeRef: Specifies whether to generate B replicates from a reference or to generate B references.
Valid parameter values:
rep - replicates
ref - references
9. Percentage: Specifies the percentage confidence to be reported in the log file. Since Statistics::Gap uses parametric bootstrap method for reference distribution generation, it is critical to understand the interval around the sample mean that could contain the population ("true") mean and with what certainty.
10. Precision: Specifies the precision to be used while generating the reference distribution.
11. Seed: The seed to be used with the random number generator. (This is an optional parameter. By default no seed is set.)

Statistics::ROC is a Perl module with receiver-operator-characteristic (ROC) curves with nonparametric confidence bounds.

SYNOPSIS

use Statistics::ROC;

my ($y) = loggamma($x);
my ($y) = betain($x, $p, $q, $beta);
my ($y) = Betain($x, $p, $q);
my ($y) = xinbta($p, $q, $beta, $alpha);
my ($y) = Xinbta($p, $q, $alpha);
my (@rk) = rank($type, @r);
my (@ROC) = roc($model_type,$conf,@val_grp);

This program determines the ROC curve and its nonparametric confidence bounds for data categorized into two groups. A ROC curve shows the relationship of probability of false alarm (x-axis) to probability of detection (y-axis) for a certain test. Expressed in medical terms: the probability of a positive test, given no disease to the probability of a positive test, given disease. The ROC curve may be used to determine an optimal cutoff point for the test.
The main function is roc(). The other exported functions are used by roc(), but might be useful for other nonparametric statistical procedures.

loggamma

This procedure evaluates the natural logarithm of gamma(x) for all x>0, accurate to 10 decimal places. Stirlings formula is used for the central polynomial part of the procedure. For x=0 a value of 743.746924740801 will be returned: this is loggamma(9.9999999999E-324).

betain

Computes incomplete beta function ratio

Remarks:
Complete beta function: B(p,q)=gamma(p)*gamma(q)/gamma(p+q)
log(B(p,q))=ln(gamma(p))+ln(gamma(q))-ln(gamma(p+q))

Incomplete beta function ratio:
I_x(p,q)=1/B(p,q) * int_0^x t^{p-1}*(1-t)^{q-1} dt

--> log(B(p,q)) has to be supplied to calculate I_x(p,q)
log denotes the natural logarithm
$beta = log(B(p,q))
$x = x
$p = p
$q = q
The subroutine returns I_x(p,q). If an error occurs a negative value
{-1,-2} is returned.

Betain

Computes the incomplete beta function by calling loggamma() and betain().

xinbta

Computes inverse of incomplete beta function ratio

Remarks:

Complete beta function: B(p,q)=gamma(p)*gamma(q)/gamma(p+q)
log(B(p,q))=ln(gamma(p))+ln(gamma(q))-ln(gamma(p+q))

Incomplete beta function ratio:
alpha = I_x(p,q) = 1/B(p,q) * int_0^x t^{p-1}*(1-t)^{q-1} dt

--> log(B(p,q)) has to be supplied to calculate I_x(p,q)
log denotes the natural logarithm
$beta = log(B(p,q))
$alpha= I_x(p,q)
$p = p
$q = q
The subroutine returns x. If an error occurs a negative value {-1,-2,-3}
is returned.

Xinbta

Computes the inverse of the incomplete beta function by calling loggamma() and xinbta().

rank

Computes the ranks of the values specified as the second argument (an array). Returns a vector of ranks corresponding to the input vector. Different types of ranking are possible (high, low, mean), and are specified as first argument. These differ in the way ties of the input vector, i.e. identical values, are treated:

high:

replace ranks of identical values with their highest rank

low:

replace ranks of identical values with their lowest rank

mean:

replace ranks of identical values with the mean of their ranks

roc

Determines the ROC curve and its nonparametric confidence bounds. The ROC curve shows the relationship of "probability of false alarm" (x-axis) to "probability of detection" (y-axis) for a certain test. Or in medical terms: the "probability of a positive test, given no disease" to the "probability of a positive test, given disease". The ROC curve may be used to determine an "optimal" cutoff point for the test.

The routine takes three arguments:

(1) type of model: decrease or increase, this states the assumption that a higher (increase) value of the data tends to be an indicator of a positive test result or for the model decrease a lower value.
(2) two-sided confidence interval (usually 0.95 is chosen).
(3) the data stored as a list-of-lists: each entry in this list consits of an "value / true group" pair, i.e. value / disease present. Group values are from {0,1}. 0 stands for disease (or signal) not present (prior knowledge) and 1 for disease (or signal) present (prior knowledge). Example: @s=([2, 0], [12.5, 1], [3, 0], [10, 1], [9.5, 0], [9, 1]); Notice the small overlap of the groups. The optimal cutoff point to separate the two groups would be between 9 and 9.5 if the criterion of optimality is to maximize the probability of detection and simultaneously minimize the probability of false alarm.

Returns a list-of-lists with the three curves: @ROC=([@lower_b], [@roc], [@upper_b]) each of the curves is again a list-of-lists with each entry consisting of one (x,y) pair.

Examples:

$,=" ";
print loggamma(10), "n";
print Xinbta(3,4,Betain(.6,3,4)),"n";

@e=(0.7, 0.7, 0.9, 0.6, 1.0, 1.1, 1,.7,.6);
print rank(low,@e),"n";
print rank(high,@e),"n";
print rank(mean,@e),"n";

@var_grp=([1.5,0],[1.4,0],[1.4,0],[1.3,0],[1.2,0],[1,0],[0.8,0],
[1.1,1],[1,1],[1,1],[0.9,1],[0.7,1],[0.7,1],[0.6,1]);
@curves=roc(decrease,0.95,@var_grp);
print "$curves[0][2][0] $curves[0][2][1] n";

Perl IRC Statistics Generator (pisg) is an IRC channel statics generator written in Perl, it creates statistics from different logfile formats.
It was originally written because IRCStats wasnt open source. So heres an open source/GPLed version to anyone interested. Its a funny thing for your IRC channel, and its highly customizeable.
Supported logfile formats:
- X-Chat
- mIRC
- mIRC6
- Eggdrops
- bxlog for BitchX
- irssi
- virc98
- dancer
- Trillian
- Grufti
- mbot
- winbot
- zcbot
- muh
- energymech
- ircII
- psybnc
- ircle
- infobot
- axur
- bobot++
- oer
- perlbot
- Vision
- pircbot
- KVIrc
- HydraIRC
- sirc
- moobot
- supybot
- blootbot
- dircproxy
Enhancements:
Bug fixes:
- Added missing > (SF #1550312).
- Fix typo in docs (SF #1550765).
Christoph:
- The manpage file released with 0.69 was empty, sorry. Thanks to Julien Danjou for spotting.
- Fix some typos in manpage and --help text.