Free data format software for linux

Data::Secs2 is a Perl module to pack, unpack, format, transform from Perl data SEMI E5-94 nested data.

SYNOPSIS

#####
# Subroutine interface
#
use Data::Secs2 qw(arrayify config listify neuterify numberify perlify
perl_typify secsify secs_elementify stringify textify transify);

@array = arrayify($ref, @options);

$old_value = config( $option );
$old_value = config( $option => $new_value);

$body = secs_elementify($format, $cells, @options);

@secs_obj = listify(@vars);

@secs_obj = neuterify($binary_secs, @options);

@vars = perlify(@secs_obj, @options);

$ref = perl_typify(@array, @options);

$ascii_secs = secsify( @secs_obj, @options);
$binary_secs = secsify( @secs_obj, type => binary);

$string = stringify(@arg, [@options]);

@secs_obj = transify($acsii_secs, @options);

#####
# Class, Object interface
#
# For class interface, use Data::Secs2 instead of $self
# use Data::Secs2;
#
$secs2 = Data::Secs2 # uses built-in config object

$secs2 = new Data::Secs2( @options );

@array = secs2->arrayify( $ref, @options );

$old_value = secs2->secs_config( $option);
$old_value = secs2->secs_config( $option => $new_value);

$body = secs2->secs_elementify($format, $cells, @options);

@secs_obj = secs2->listify(@vars, @options);

@secs_obj = secs2->neuterify($binary_secs, @options);

@vars = secs2->perlify(@secs_obj, @options);

$ref = secs2->perl_typify(@array, @options);

$ascii_secs = secs2->secsify( @secs_obj, @options);
$binary_secs = secs2->secsify( @secs_obj, type => binary);

$body = secs2->stringify( @arg );

@secs_obj = secs2->transify($acsii_secs, @options);

Generally, if a subroutine will process a list of options, @options, that subroutine will also process an array reference, @options, [@options], or hash reference, %options, {@options}. If a subroutine will process an array reference, @options, [@options], that subroutine will also process a hash reference, %options, {@options}. See the description for a subroutine for details and exceptions.

Data::CTable is a Perl module that helps you read, write, manipulate tabular data.

SYNOPSIS

## Read some data files in various tabular formats
use Data::CTable;
my $People = Data::CTable->new("people.merge.mac.txt");
my $Stats = Data::CTable->new("stats.tabs.unix.txt");

## Clean stray whitespace in fields
$People->clean_ws();
$Stats ->clean_ws();

## Retrieve columns
my $First = $People->col(FirstName);
my $Last = $People->col(LastName );

## Calculate a new column based on two others
my $Full = [map {"$First->[$_] $Last->[$_]"} @{$People->all()}];

## Add new column to the table
$People->col(FullName => $Full);

## Another way to calculate a new column
$People->col(Key);
$People->calc(sub {no strict vars; $Key = "$Last,$First";});

## "Left join" records matching Stats:PersonID to People:Key
$Stats->join($People, PersonID => Key);

## Find certain records
$Stats->select_all();
$Stats->select(Department => sub {/Sale/i }); ## Sales depts
$Stats->omit (Department => sub {/Resale/i}); ## not Resales
$Stats->select(UsageIndex => sub {$_ > 20.0}); ## high usage

## Sort the found records
$Stats->sortspec(DeptNum , {SortType => Integer});
$Stats->sortspec(UsageIndex, {SortType => Number });
$Stats->sort([qw(DeptNum UsageIndex Last First)]);

## Make copy of table with only found/sorted data, in order
my $Report = $Stats->snapshot();

## Write an output file
$Report->write(_FileName => "Rept.txt", _LineEnding => "mac");

## Print a final progress message.
$Stats->progress("Done!");

## Dozens more methods and parameters available...

OVERVIEW

Data::CTable is a comprehensive utility for reading, writing, manipulating, cleaning and otherwise transforming tabular data. The distribution includes several illustrative subclasses and utility scripts.

A Columnar Table represents a table as a hash of data columns, making it easy to do data cleanup, formatting, searching, calculations, joins, or other complex operations.

The objects hash keys are the field names and the hash values hold the data columns (as array references).

Tables also store a "selection" -- a list of selected / sorted record numbers, and a "field list" -- an ordered list of all or some fields to be operated on. Select() and sort() methods manipulate the selection list. Later, you can optionally rewrite the table in memory or on disk to reflect changes in the selection list or field list.
Data::CTable reads and writes any tabular text file format including Merge, CSV, Tab-delimited, and variants. It transparently detects, reads, and preserves Unix, Mac, and/or DOS line endings and tab or comma field delimiters -- regardless of the runtime platform.

In addition to reading data files, CTable is a good way to gather, store, and operate on tabular data in memory, and to export data to delimited text files to be read by other programs or interactive productivity applications.

To achieve extremely fast data loading, CTable caches data file contents using the Storable module. This can be helpful in CGI environments or when operating on very large data files. CTable can read an entire cached table of about 120 megabytes into memory in about 10 seconds on an average mid-range computer.

For simple data-driven applications needing to store and quickly retrieve simple tabular data sets, CTable provides a credible alternative to DBM files or SQL.
For data hygiene applications, CTable forms the foundation for writing utility scripts or compilers to transfer data from external sources, such as FileMaker, Excel, Access, personal organizers, etc. into compiled or validated formats -- or even as a gateway to loading data into SQL databases or other destinations. You can easily write short, repeatable scripts in Perl to do reporting, error checking, analysis, or validation that would be hard to duplicate in less-flexible application environments.

The data representation is simple and open so you can directly access the data in the object if you feel like it -- or you can use accessors to request "clean" structures containing only the data or copies of it. Or you can build your own columns in memory and then when youre ready, turn them into a table object using the very flexible new() method.

The highly factored interface and implementation allow fine-grained subclassing so you can easily create useful lightweight subclasses. Several subclasses are included with the distribution.

Most defaults and parameters can be customized by subclassing, overridden at the instance level (avoiding the need to subclass too often), and further overridden via optional named-parameter arguments to most major method calls.