Free encoded data structures software for linux

Erwin Data Structures is a library that is meant to be the ultimate data structure library for mixed usage of C and C++.

Arbitrary key and value types are implemented by template files that dont use C++ templates, but are instantiated by a Perl script.

This way, mixed usage in C and C++ is possible. However, a C++ interface is generated to support the advantages of the C++ language. No templates, no void*.
Erwin contains a number of tools, too, all of them written in Perl. The following list shows the data structures and tools, together with some typical examples.

Xephyrus Data Structures Tag Library provides an easy way to create and manipulate the contents of common Java data-structures such as maps and lists.
Enhancements:
- The library was polished up.
- Several improvements were made and a few bugs were fixed.
- This version is aimed at Java 5 and JSP 2.0.

GraphViz::Data::Structure can visualise data structures.

SYNOPSIS

use GraphViz::Data::Structure;

my $gvds = GraphViz:Data::Structure->new($data_structure);
print $gvds->graph()->as_png;

This module makes it easy to visualise data structures, even recursive or circular ones.
It is provided as an alternative to GraphViz::Data::Grapher. Differences:

GraphViz::Data::Structure handles structures of arbitrary depth and complexity, automatically following links using a standard graph traversal algorithm.
GraphViz::Data::Grapher creates graphics of indiividual substructures (arrays, scalars, hashes) which keep the substructure type and data together; GraphViz::Data::Structure does this by shape alone.
GraphViz::Data::Structure encapsulates object info (if any) directly into the node being used to represent the class.
GraphViz::Data::Grapher colors its graphs; GraphViz::Data::Structure doesnt by default.
GraphViz::Data:Structure can parse out globs and CODE references (almost as well as the debugger does).

XML::Structured is a simple conversion API from XML to perl structures and back.

SYNOPSIS

use XML::Structured;

$dtd = [
element =>
attribute1,
attribute2,
[],
element1,
[ element2 ],
[ element3 =>
...
],
[[ element4 =>
...
]],
];

$hashref = XMLin($dtd, $xmlstring);
$hashref = XMLinfile($dtd, $filename_or_glob);
$xmlstring = XMLout($dtd, $hashref);

The XML::Structured module provides a way to convert xml data into a predefined perl data structure and back to xml. Unlike with modules like XML::Simple it is an error if the xml data does not match the provided skeleton (the "dtd"). Another advantage is that the order of the attributes and elements is taken from the dtd when converting back to xml.

Data::SecsPack is a Perl module pack and unpack numbers in accordance with SEMI E5-94.

SYNOPSIS

#####
# Subroutine interface
#
use Data::SecsPack qw(bytes2int config float2binary
ifloat2binary int2bytes
pack_float pack_int pack_num
str2float str2int
unpack_float unpack_int unpack_num);

$big_integer = bytes2int( @bytes );

$old_value = config( $option );
$old_value = config( $option => $new_value);

($binary_magnitude, $binary_exponent) = float2binary($magnitude, $exponent, @options);

($binary_magnitude, $binary_exponent) = ifloat2binary($imagnitude, $iexponent, @options);

@bytes = int2bytes( $big_integer );

($format, $floats) = pack_float($format, @string_floats, [@options]);

($format, $integers) = pack_int($format, @string_integers, [@options]);

($format, $numbers, @string) = pack_num($format, @strings, [@options]);

$float = str2float($string, [@options]);
(@strings, @floats) = str2float(@strings, [@options]);

$integer = str2int($string, [@options]);
(@strings, @integers) = str2int(@strings, [@options]);

@ingegers = unpack_int($format, $integer_string, @options);

@floats = unpack_float($format, $float_string, @options);

@numbers = unpack_num($format, $number_string), @options;

#####
# Class, Object interface
#
# For class interface, use Data::SecsPack instead of $self
#
use Data::SecsPack;

$secspack = Data::SecsPack; # uses built-in config object

$secspack = new Data::SecsPack(@options);

$big_integer = bytes2int( @bytes );

($binary_magnitude, $binary_exponent) = $secspack->float2binary($magnitude, $exponent, @options);

($binary_magnitude, $binary_exponent) = $secspack->ifloat2binary($imagnitude, $iexponent, @options);

@bytes = $secspack->int2bytes( $big_integer );

($format, $floats) = $secspack->pack_float($format, @string_integers, [@options]);

($format, $integers) = $secspack->pack_int($format, @string_integers, [@options]);

($format, $numbers, @strings) = $secspack->pack_num($format, @strings, [@options]);

$integer = $secspack->str2int($string, [@options])
(@strings, @integers) = $secspack->str2int(@strings, [@options]);

$float = $secspack->str2float($string, [@options]);
(@strings, @floats) = $secspack->str2float(@strings, [@options]);

@ingegers = $secspack->unpack_int($format, $integer_string, @options);

@floats = $secspack->unpack_float($format, $float_string, @options);

@numbers = $secspack->unpack_num($format, $number_string, @options);

Generally, if a subroutine will process a list of options, @options, that subroutine will also process an array reference, @options, [@options], or hash reference, %options, {@options}. If a subroutine will process an array reference, @options, [@options], that subroutine will also process a hash reference, %options, {@options}. See the description for a subroutine for details and exceptions.

The subroutines in the Data::SecsPack module packs and unpacks numbers in accordance with SEMI E5-94. The E5-94 establishes the standard for communication between the equipment used to fabricate semiconductors and the host computer that controls the fabrication. The equipment in a semiconductor factory (fab) or any other fab contains every conceivable known microprocessor and operating system known to man. And there are a lot of specialize real-time embedded processors and speciallize real-time embedded operating systems in addition to the those in the PC world.

The communcication between host and equipment used packed nested list data structures that include arrays of characters, integers and floats. The standard has been in place and widely used in China, Germany, Korea, Japan, France, Italy and the most remote corners on this planent for decades. The basic data structure and packed data formats have not changed for decades.

This stands in direct contradiction to the common conceptions of many in the Perl community and most other communities. The following quote is taken from page 761, Programming Perl third edition, discussing the pack subroutine:

"Floating-point numbers are in the native machine format only. Because of the variety of floating format and lack of a standard "network" represenation, no facility for interchange has been made. This means that packed floating-point data written on one machine may not be readable on another. That is a problem even when both machines use IEEE floating-point arithmetic, because the endian-ness of memory representation is not part of the IEEE spec."

There are a lot of things that go over the net that have industry or military standards but no RFCs. So unless you dig them out, you will never know they exist. While RFC and military standards may be freely copyied, industry standards are usually copyrighted. This means if you want to read the standard, you have to pay whatever the market bears. ISO standards, SEMI stardards, American National Standards, IEEE standards beside being boring are expensive. In other words, you do not see them flying out the door at the local Barnes and Nobles. In fact, you will not even find them inside the door.

It very easy to run these non RFC standard protocols over the net. Out of 64,000 ports, pick a port of opportunity (hopefully not one of those low RFC preassigned ports) and configure the equipment and host to the same IP and port. Many times the software will allow a remote console that is watch only. The watch console may even be a web server on port 80. If there is a remote soft console, you can call up or e-mail the equipment manufacturers engineer in say Glouster, MA, USA and tell him the IP and port so he can watch his manchine mangle a cassette of wafers with a potential retail value of half million dollars.

SEMI E5-94 and their precessors do standardize the endian-ness of floating point, the packing of nested data, used in many programming languages, and much, much more. The endian-ness of SEMI E5-94 is the first MSB byte, floats sign bit first. Maybe this is because it makes it easy to spot numbers in a packed data structure.
The nested data has many performance advantages over the common SQL culture of viewing and representing data as tables. The automated fabs of the world make use of SEMI E5-94 nested data not only for real-time communication (TCP/IP RS-2332 etc) between machines but also for snail-time processing as such things as logs and performance data.

Does this standard communications protocol ensure that everything goes smoothly without any glitches with this wild mixture of hardware and software talking to each other in real time? Of course not. Bytes get reverse. Data gets jumbled from point A to point B. Machine time to test software is non-existance. Big ticket, multi-million dollar fab equipment has to work to earn its keep. And, then there is the everyday business of suiting up, with humblizing hair nets, going through air and other showers with your favorite or not so favorite co-worker just to get into the clean room. And make sure not to do anything that will scatch a wafer with a lot of Intel Pentiums on them. It is totally amazing that the product does get out the door.

XML::Generator::PerlData is a Perl extension for generating SAX2 events from nested Perl data structures.

SYNOPSIS

use XML::Generator::PerlData;
use SomeSAX2HandlerOrFilter;

## Simple style ##

# get a deeply nested Perl data structure...
my $hash_ref = $obj->getScaryNestedDataStructure();

# create an instance of a handler class to forward events to...
my $handler = SomeSAX2HandlerOrFilter->new();

# create an instance of the PerlData driver...
my $driver = XML::Generator::PerlData->new( Handler => $handler );

# generate XML from the data structure...
$driver->parse( $hash_ref );

## Or, Stream style ##

use XML::Generator::PerlData;
use SomeSAX2HandlerOrFilter;

# create an instance of a handler class to forward events to...
my $handler = SomeSAX2HandlerOrFilter->new();

# create an instance of the PerlData driver...
my $driver = XML::Generator::PerlData->new( Handler => $handler );

# start the event stream...
$driver->parse_start();

# pass the data through in chunks
# (from a database handle here)
while ( my $array_ref = $dbd_sth->fetchrow_arrayref ) {
$driver->parse_chunk( $array_ref );
}

# end the event stream...
$driver->parse_end();
and youre done...

XML::Generator::PerlData provides a simple way to generate SAX2 events from nested Perl data structures, while providing finer-grained control over the resulting document streams.
Processing comes in two flavors: Simple Style and Stream Style:

In a nutshell, simple style is best used for those cases where you have a a single Perl data structure that you want to convert to XML as quickly and painlessly as possible. Stream style is more useful for cases where you are receiving chunks of data (like from a DBI handle) and you want to process those chunks as they appear. See PROCESSING METHODS for more info about how each style works.

cpp xstream is a C++ flexible stream library.
cpp xstream is a collection of several streambuf and iostream classes that allow a programmer to compress and decompress data transparently with gzip or bzip2, to serialize composite types to xdr, to encode or decode with base64, and to tee output from one channel to several others. These streams can be stacked on each other.
Main features:
- Compressing and decompressing data using zlib and bzlib
- Encode and decode base64 data
- Serialize and deserialize structures to XDR
- Calculate digests of data
- Fork data written to a stream to several others
- Use file descriptors using a iostream interface (where available)