Free data analysis software for linux

Net::Analysis are modules for analysing network traffic.

SYNOPSIS

Using an existing analyser:

$ perl -MNet::Analysis -e main help
$ perl -MNet::Analysis -e main TCP,v=1 dump.tcp - basic TCP info
$ perl -MNet::Analysis -e main HTTP,v=1 dump.tcp - HTTP stuff
$ perl -MNet::Analysis -e main Example2,regex=img dump.tcp - run an example

Writing your own analyser:

package MyExample;
use base qw(Net::Analysis::Listener::Base);
# Listen to events from other modules
sub tcp_monologue {
my ($self, $args) = @_;
my ($mono) = $args->{monologue};
my $t = $mono->t_elapsed()->as_number();
my $l = $mono->length();
# Emit your own event
$self->emit(name => example_event,
args => { kb_sec => ($t) ? $l/($t*1024) : N/A }
);
}
# Process your own event
sub example_event {
my ($self, $args) = @_;
printf "Bandwidth: %10.2f KB/secn", $args->{kb_sec};
}
1;
__top

ABSTRACT

Net::Analysis is a suite of modules that parse tcpdump files, reconstruct TCP sessions from the packets, and provide a very lightweight framework for writing protocol anaylsers.
__top

I wanted a batch version of Ethereal in Perl, so I could:

- sift through parsed protocols with structured filters
- write custom reports that mixed events from multiple protocols

So here it is. Net::Analysis is a stack of protocol handlers that emit, and listen for, events.

Market Analysis System (MAS) is an open-source software application that provides tools for analysis of financial markets using technical analysis.
Market Analysis System provides facilities for stock charting and futures charting, including price, volume, and a wide range of technical analysis indicators. Market Analysis System also allows automated processing of market data - applying technical analysis indicators with user-selected criteria to market data to automatically generate trading signals - and can be used as the main component of a sophisticated trading system.
Main features:
- Includes basic technical analysis indicators, such as Simple Moving Average, Exponential Moving Average, Stochastic, MACD, RSI, On Balance Volume, and Momentum.
- Includes more advanced indicators, such as Standard Deviation, Slope of EMA of Volume, Slope of MACD Signal Line, Bollinger Bands, and Parabolic SAR.
- User can create new technical analysis indicators, including complex indicators based on existing indicators.
- User can configure criteria for automated trading-signal generation.
- Creation of weekly, monthly, quarterly, and yearly data from daily data.
- Handles intraday data.
- Handles stock and futures data.
- Accepts input data from files, from a database, or from the web. (Includes a configuration for obtaining end-of-day data from yahoo.com.)
- Can be configured and run as a server that provides services for several clients at a time running on remote machines.

Statistical Traffic Analysis Kit is a set of command-line traffic analysis tools, designed to help a network administrator to see what is happening at a router at the moment.

Unlike tcpdump (1), the stak set uses statistical and stream-oriented methods, and will rarely produce an output stream at a speed beyond human perception. The output is less accurate.

The kit consists of five different utilities, designed to perform the following tasks:
estimating overall traffic rates (stakrate),
determining network nodes generating the highest traffic (stakhosts)
monitoring the amount of traffic exchanged with particular autonomous
systems (stakasta),
extracting strings from packets (stakextract),
determining connections and flows generating the highest traffic
(stakstreams, experimental),

TA-Lib provides common functions for the technical analysis of stock/future/commodity market data.
TA-Lib can be reused by trading software developers using Excel, .NET, Java, Perl or C/C++.
Main features:
- More than 120 technical analysis indicators such as ADX, MACD, RSI, Stochastic, Bollinger Bands etc...
- bullet Includes candlestick pattern recognition.
- bullet Optional abstract interface allowing your code to support new technical analysis functions without any code change!
Enhancements:
New Features
- New Functions: BETA, MINMAX, MINMAXINDEX, MININDEX, MAXINDEX
- Debian and RPM packaging available.
- Java JAR packaging available.
- New TA_FunctionDescription() returns XML description of API.
- New ta_func_api.xml file generated in root directory of the package.
- Support for unmanaged static libraries with Visual Studio 2005.
Fixes
- #1526632 : Fix bug in LINEARREG_ANGLE
- #1544555 : Now do proper divide by zero detection in TA_ADX
Other Changes
- Better Java/.NET naming convention.
- ta_func_list.txt moved in root directory of the package.
- Removed dependencies on trio and Mersenne Twister functions.
- Volume and Open Interest are now double instead of integers.
- Add license specific to Excel users.

BASE is the Basic Analysis and Security Engine. It is based on the code from the Analysis Console for Intrusion Databases (ACID) project.
This application provides a web front-end to query and analyze the alerts coming from a SNORT IDS system.
BASE is a web interface to perform analysis of intrusions that snort has detected on your network. It uses a user authentication and role-base system, so that you as the security admin can decide what and how much information each user can see. It also has a simple to use, web-based setup program for people not comfortable with editing files directly.
BASE is supported by a group of volunteers. They are available to answer any questions you may have or help you out in setting up your system. They are also skilled in intrusion detection systems and make use of that knowledge in the development of BASE.
Enhancements:
- This release fixes a number of bugs with PHP 5.
- It also adds a number of new features.

Audio::Data is a module for representing audio data to perl.

SYNOPSIS

use Audio::Data;
my $audio = Audio::Data->new(rate => , ...);

$audio->method(...)

$audio OP ...

Audio::Data represents audio data to perl in a fairly compact and efficient manner using C via XS to hold data as a C array of float values. The use of float avoids many issues with dynamic range, and typical float has 24-bit mantissa so quantization noise should be acceptable. Many machines have floating point hardware these days, and in such cases operations on float should be as fast or faster than some kind of "scaled integer".

Nominally data is expected to be between +1.0 and -1.0 - although only code which interacts with outside world (reading/writing files or devices) really cares.
It can also represent elements (samples) which are "complex numbers" which simplifies many Digital Signal Processing methods.

Methods

The interface is object-oriented, and provides the methods below.
$audio = Audio::Data->new([method => value ...])
The "constructor" - takes a list of method/value pairs and calls $audio->method(value) on the object in order. Typically first "method" will be rate to set sampling rate of the object.

$rate = $audio->rate
Get sampling rate of object.
$audio->rate($newrate)
Set sampling rate of the object. If object contains existing data it is re-sampled to the new rate. (Code to do this was derived from a now dated version of sox.)
$audio->comment($string)
Sets simple string comment associated with data.
$string = $audio->comment
Get the comment
$time = $audio->duration
Return duration of object (in seconds).
$time = $audio->samples
Return number of samples in the object.
@data = $audio->data
Return data as list of values - not recommended for large data.
$audio->data(@data)
Sets elements from @data.
$audio->length($N)
Set number of samples to $N - tuncating or padding with zeros (silence).
($max,$min) = $audio->bounds([$start_time[,$end_time]])
Returns a list of two values representing the limits of the values between the two times if $end_time isnt specified it defaults to the duration of the object, and if start time isnt specified it defaults to zero.
$copy = $audio->clone
Creates copy of data carrying over sample rate and complex-ness of data.
$slice = $audio->timerange($start_time,$end_time);
Returns a time-slice between specified times.
$audio->Load($fh)
Reads Sun/NeXT .au data from the perl file handle (which should have binmode() applied to it.)
This will eventually change - to allow it to load other formats and perhaps to return list of Audio::Data objects to represnt multiple channels (e.g. stereo).
$audio->Save($fh[,$comment])
Write a Sun/NeXT .au file to perl file handle. $comment if specified is used as the comment.
$audio->tone($freq,$dur,$amp);
Append a sinusoidal tone of specified freqency (in Hz) and duration (in seconds), and peak amplitude $amp.
$audio->silence($dur);
Append a period of 0 value of specified duration.
$audio->noise($dur,$amp);
Append burst of (white) noise of specified duration and peak amplitude.
$window = $audio->hamming($SIZE,$start_sample[,$k])
Returns a "raised cosine window" sample of $SIZE samples starting at specified sample. If $k is specified it overrides the default value of 0.46 (e.g. a value of 0.5 would give a Hanning window as opposed to a Hamming window.)
windowed = ((1.0-k)+k*cos(x*PI))
$freq = $audio->fft($SIZE)
$time = $freq->ifft($SIZE);
Perform a Fast Fourier Transform (or its inverse). (Note that in general result of these methods have complex numbers as the elements. $SIZE should be a power-of-two (if it isnt next larger power of two is used). Data is padded with zeros as necessary to get to $SIZE samples.
@values = $audio->amplitude([$N[,$count]])
Return values of amplitude for sample $N..$N+$count inclusive. if $N is not specified it defaults to zero. If $count is not specified it defaults to 1 for scalar context and rest-of-data in array context.
@values = $audio->dB([$N[,$count]])
Return amplitude - in deci-Bells. (0dB is 1/2**15 i.e. least detectable value to 16-bit device.) Defaults as for amplitude.
@values = $audio->phase([$N [,$count]])
Return Phase - (if data are real returns 0). Defaults as for amplitude.
$diff = $audio->difference
Returns the first difference between successive elements of the data - so result is one sample shorter. This is a simple high-pass filter and is much used to remove DC offsets.
$Avalues = $audio->lpc($NUM_POLES,[$auto [,$refl]])
Perform Linear Predictive Coding analysis of $audio and return coefficents of resulting All-Pole filter. 0th Element is not a filter coefficent (there is no A[0] in such a filter) - but is a measure of the "error" in the matching process. $auto is an output argument and returns computed autocorrelation. $refl is also output and are so-called reflection coefficents used in "lattice" realization of the filter. (Code for this lifted from "Festival" speech systems speech_tools.)
$auto = $audio->autocorrelation($LENGTH)
Returns an (unscaled) autocorrelation function - can be used to cause peaks when data is periodic - and is used as a precursor to LPC analysis.

Plucene::Analysis::PorterStemFilter - Porter stemming on the token stream.

SYNOPSIS

# isa Plucene::Analysis:::TokenFilter

my $token = $porter_stem_filter->next;

This class transforms the token stream as per the Porter stemming algorithm.
Note: the input to the stemming filter must already be in lower case, so you will need to use LowerCaseFilter or LowerCaseTokenizer farther down the Tokenizer chain in order for this to work properly!

The Porter Stemmer implements Porter Algorithm for normalization of English words by stripping their extensions and is used to generalize the searches. For example, the Porter algorithm maps both search and searching (as well as searchnessing) to search such that a query for search will also match documents that contains the word searching.

Note that the Porter algorithm is specific to the English language and may give unpredictable results for other languages. Also, make sure to use the same analyzer during the indexing and the searching.

You can find more information on the Porter algorithm at www.tartarus.org/~martin/PorterStemmer.

A nice online demonstration of the Porter algorithm is available at www.scs.carleton.ca/~dquesnel/java/stuff/PorterApplet.html.

METHODS

next

my $token = $porter_stem_filter->next;

Returns the next input token, after being stemmed.

SciDAVis is a free application for Scientific Data Analysis and Visualization. SciDAVis is a free interactive application aimed at data analysis and publication-quality plotting. It combines a shallow learning curve and an intuitive, easy-to-use graphical user interface with powerful features such as scriptability and extensibility.
SciDAVis runs on GNU/Linux, Windows and MacOS X; possibly also on other platforms like *BSD, although this is untested.
SciDAVis is similar in its field of application to proprietary Windows applications like Origin and SigmaPlot as well as free applications like QtiPlot, Labplot and Gnuplot.
Main features:
- Tables, Matrices, Graphs and Notes are gathered in a Project and can be organized using folders.
- Tables for entering data directly or importing from ASCII files.
- Column-based computations using standard and special functions (and much more if you have Python installed).
- Many built-in analysis operations like column/row statistics, (de)convolution, FFT and FFT-based filters.
- Extensive support for fitting linear and nonlinear functions to the data, including multi-peak fitting.
- Publication-quality 2D plots in a variety of formats, including symbols/lines, bars and pie charts.
- Interactive 3D plots with export to a variety of formats, including EPS and PDF.
- Note windows with support for in-place evaluation of mathematical expressions. With Python installed, you can even access other objects in the project, e.g. for rapidly writing an import filter for a custom data format.
Enhancements:
- Handling of custom decimal separators was improved.
- Many improvements and additions were made to Python scripting.
- The user interface was improved with restructured menus and tool bars and several restructured dialogs.
- The script window was removed, since notes offer the same capabilities.
- Project loading/saving was made more stable.
- ASCII import and export was made easier.
- Appending of projects and Origin graphs was improved.
- Lots of small improvements and bugfixes were made.
- Exporting to EPS and PDF can be done without large white borders.