Free multidimensional software for linux

Common Data Format is a self-describing data abstraction for the storage and manipulation of multidimensional data in a platform- and discipline-independent fashion.
It consists of a scientific data management package (known as the "CDF Library") that allows programmers and application developers to manage and manipulate scalar, vector, and multi-dimensional data arrays.
Enhancements:
- Adds new sets of APIs to allow Standard Interface to interact with zVariables and other CDF-related information.
- Adds MingW and FreeBSD ports.
- Adds support for Intel C++ and Fortran for Linux.
- Adds the ability to create legacy CDF 2.7 files.
- Fixes a bug that prevented directories from having .cdf or .skt extensions.

Template Numerical Toolkit (TNT) is a collection of interfaces and reference implementations of numerical objects useful for scientific computing in C++.
The toolkit defines interfaces for basic data structures, such as multidimensional arrays and sparse matrices, commonly used in numerical applications. Template Numerical Toolkits goal is to provide reusable software components that address many of the portability and maintennace problems with C++ codes.
TNT provides a distinction between interfaces and implementations of TNT components. For example, there is a TNT interface for two-dimensional arrays which describes how individual elements are accessed and how certain information, such as the array dimensions, can be used in algorithms; however, there can be several implementations of such an interface: one that uses expression templates, or one that uses BLAS kernels, or another that is instrumented to provide debugging information.
By specifying only the interface, applications codes may utilize such algorithms, while giving library developers the greatest flexibility in employing optimization or portability strategies.
TNT Data Structures
- C-style arrays
- Fortran-style arrays
- Sparse Matrices
- Vector/Matrix
TNT utilities
- array I/O
- math routines (hypot(), sign(), etc.)
- Stopwatch class for timing measurements
Libraries that utilize TNT
- JAMA: a linear algebra library with QR, SVD, Cholesky and Eigenvector solvers.
- old (pre 1.0) TNT routines for LU, QR, and Eigenvalue problems

Hierarchical Data Format is a general purpose library and file format for storing scientific data.
HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic objects, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs.
Efficient storage and I/O.
HDF5 was created to address the data management needs of scientists and engineers working in high performance, data intensive computing environments. As a result, the HDF5 library and format emphasize storage and I/O efficiency. For instance, the HDF5 format can accommodate data in a variety of ways, such as compressed or chunked. And the library is tuned and adapted to read and write data efficiently on parallel computing systems.
Software.
NCSA maintains a suite of free, open source software, including the HDF5 I/O library and several utilities. The HDF5 user community also develops and contributes software, much of it freely available. Unlike HDF4, there is little commercial support for HDF5 at this time, but we are successfully working with vendors to change this.
Emphasis on standards.
Data can be stored in HDF5 in an endless variety of ways, so it is important for communities of users to standardize on how their data is to be organized in HDF5. This makes it possible to share data easily, and also to build and share tools for accessing and analyzing data stored in HDF5. The NCSA HDF team works with users to encourage them to organize HDF5 files in standard ways.
Large and varied user community.
HDF5 users range across a variety of engineering and scientific fields, and even some non-technical fields. Data stored in HDF5 is used for a wide range of applications, from computational fluid dynamics to film making.
Main features:
- Parallel HDF5 - Information on installing and using Parallel HDF5
- SZIP Compression - Information about SZIP Compression in HDF5
- Thread Safe HDF5 - Information on thread-safe capabilities of HDF5 and how to install
- The High Level HDF5 APIs, previously distributed separately, are now distributed as part of the main HDF5 Library:
- High Level HDF5 APIs - Information on installing and using the High Level HDF5 APIs
Applications:
- HDF Java Products - HDF4/HDF5 Java interfaces and viewer, HDFView.
- HDF Web-browser Plug-in - The HDF Web-browser plug-in is a windowed browser plug-in that is launched from a web browser to display HDF4 and HDF5 files.
- netCDF-4 - The NCSA and NetCDF groups are collaborating on a version of NetCDF built on top of HDF5.
- HDF5 XML Information Page - DTD and tools for using HDF5 with XML
- HDF5 WRF I/O Module - I/O module that reads HDF5 datasets for the Weather Research and Forecasting Model
- HDF5 Mesh API (prototype) - API for storing and retrieving structured and unstructured mesh data
Enhancements:
- The default Fortran was switched to G95 when using GCC.
- The autoconf build tools were updated. Fortran interfaces were added for the Image, Table, and Lite APIs.
- A Dimension Scale API (H5DS) was added.
- FreeBSD is now supported on AMD64 with GNU C and Fortran compilers.
- Support for sequential and parallel libraries was added for Intel 64 Linux clusters.
- Several bugs with writing fill values for datasets that have a variable-length datatype or component datatype were fixed.

EZMorph is simple Java library for transforming an Object to another Object.
It supports transformations for primitives, Objects, and multidimensional arrays, compatibility with JDK 1.3.1, and small memory footprint (~60K).
EZMorph began life as the converter package in Json-lib but became a project on its own.
- Supports transformations for primitives and Objects
- Supports transformations for multidimensional arrays
- JDK 1.3.1 compatible
- Small memory footprint (~60K)
EZMorph comes with another feature: ArrayAssertions . JUnit 3.x does not have an assertEquals() method for asserting array equality, and JUnit 4.x has a limited one (it only supports Object[] not primitive arrays).
With ArrayAssertions is possible to compare a boolean[] with a boolean[] or even a Boolean[], an those arrays can be multidimensional too. EZMorph began life as the converter package on Json-lib but seeing that the features provided were more generic than JSON parsing, it became a project on its own.
Enhancements:
- This release can override existing morphers for a type when registering a new one, and clear all morphers for a type.
- The Javadocs have been updated.

PDL::Indexing Perl module contains a tutorial on how to index piddles.

This manpage should serve as a first tutorial on the indexing and threading features of PDL.

This manpage is still in alpha development and not yet complete. "Meta" comments that point out deficiencies/omissions of this document will be surrounded by square brackets ([]), e.g. [ Hopefully I will be able to remove this paragraph at some time in the future ]. Furthermore, it is possible that there are errors in the code examples. Please report any errors to Christian Soeller (c.soeller@auckland.ac.nz).

Still to be done are (please bear with us and/or ask on the mailing list, see PDL::FAQ):

document perl level threading
threadids
update and correct description of slice
new functions in slice.pd (affine, lag, splitdim)
reworking of paragraph on explicit threading

Indexing and threading with PDL

A lot of the flexibility and power of PDL relies on the indexing and looping features of the perl extension. Indexing allows access to the data of a pdl object in a very flexible way. Threading provides efficient implicit looping functionality (since the loops are implemented as optimized C code).

Pdl objects (later often called "pdls") are perl objects that represent multidimensional arrays and operations on those. In contrast to simple perl @x style lists the array data is compactly stored in a single block of memory thus taking up a lot less memory and enabling use of fast C code to implement operations (e.g. addition, etc) on pdls.

pdls can have children

Central to many of the indexing capabilities of PDL are the relation of "parent" and "child" between pdls. Many of the indexing commands create a new pdl from an existing pdl. The new pdl is the "child" and the old one is the "parent". The data of the new pdl is defined by a transformation that specifies how to generate (compute) its data from the parents data. The relation between the child pdl and its parent are often bidirectional, meaning that changes in the childs data are propagated back to the parent. (Note: You see, we are aiming in our terminology already towards the new dataflow features. The kind of dataflow that is used by the indexing commands (about which you will learn in a minute) is always in operation, not only when you have explicitly switched on dataflow in your pdl by saying $a->doflow. For further information about data flow check the dataflow manpage.)

Another way to interpret the pdls created by our indexing commands is to view them as a kind of intelligent pointer that points back to some portion or all of its parents data. Therefore, it is not surprising that the parents data (or a portion of it) changes when manipulated through this "pointer". After these introductory remarks that hopefully prepared you for what is coming (rather than confuse you too much) we are going to dive right in and start with a description of the indexing commands and some typical examples how they might be used in PDL programs. We will further illustrate the pointer/dataflow analogies in the context of some of the examples later on.

There are two different implementations of this ``smart pointer relationship: the first one, which is a little slower but works for any transformation is simply to do the transformation forwards and backwards as necessary. The other is to consider the child piddle a ``virtual piddle, which only stores a pointer to the parent and access information so that routines which use the child piddle actually directly access the data in the parent. If the virtual piddle is given to a routine which cannot use it, PDL transparently physicalizes the virtual piddle before letting the routine use it.

Currently (1.94_01) all transformations which are ``affine, i.e. the indices of the data item in the parent piddle are determined by a linear transformation (+ constant) from the indices of the child piddle result in virtual piddles. All other indexing routines (e.g. ->index(...)) result in physical piddles. All routines compiled by PP can accept affine piddles (except those routines that pass pointers to external library functions).

Note that whether something is affine or not does not affect the semantics of what you do in any way: both

$a->index(...) .= 5;
$a->slice(...) .= 5;

change the data in $a. The affinity does, however, have a significant impact on memory usage and performance.

Zoem is a macro/programming language. Zoem project can be used as an all-round macro language, but has more specialized uses as well.
One such specific use is its support for creating small mark-up languages that map to different devices. It has character filtering capabilities tailored to this application.
Two such languages come packaged with zoem; one for creating manual pages that can be output either in troff or in HTML, the other for creating FAQs again in either troff or HTML output.
It supports arithmetic evaluation, regular expressions, multidimensional data storage, iteration, comprehensive IO, control operators, dictionary stacks, system commands, and more.
Enhancements:
- Zoem is now licensed under the GNU General Public License version 3 (or later).
- The bottom user dictionary functions as a global namespace, and syntax exists to access this global namespace directly.
- A macro set#3 was added that provides the modalities of existing set variants (optional warning and expansion) as well as additional modalities.
- The primitive done was reimplemented as throw{done} and throw{done} can be be used to cleanly cut processing in a variety of ways.
- Two example solutions to the N-queen problem illustrate various zoem capabilities.