Free sun solaris cluster software for linux

OSCAR version 4.0 is a snapshot of the best known methods for building, programming, and using clusters. OSCAR Cluster project consists of a fully integrated and easy to install software bundle designed for high performance cluster computing.

Everything needed to install, build, maintain, and use a modest sized Linux cluster is included in the suite, making it unnecessary to download or even install any individual software packages on your cluster.

Rocks Cluster is a complete "cluster on a CD" solution for x86 and IA64 Red Hat Linux COTS clusters.
Building a Rocks cluster does not require any experience in clustering, yet a cluster architect will find a flexible and programmatic way to redesign the entire software stack just below the surface (appropriately hidden from the majority of users).
Although Rocks includes the tools expected from any clustering software stack (PBS, Maui, GM support, Ganglia, etc), it is unique in its simplicity of installation.
From a hardware component and raw processing power perspective, commodity clusters are phenomenal price/performance compute engines. However, if a scalable ``cluster management strategy is not adopted, the favorable economics of clusters are offset by the additional on-going personnel costs involved to ``care and feed for the machine. The complexity of cluster management (e.g., determining if all nodes have a consistent set of software) often overwhelms part-time cluster administrators, who are usually domain application scientists. When this occurs, machine state is forced to either of two extremes: the cluster is not stable due to configuration problems, or software becomes stale, security holes abound, and known software bugs remain unpatched.
While earlier clustering toolkits expend a great deal of effort (i.e., software) to compare configurations of nodes, Rocks makes complete Operating System (OS) installation on a node the basic management tool. With attention to complete automation of this process, it becomes faster to reinstall all nodes to a known configuration than it is to determine if nodes were out of synchronization in the first place. Unlike a users desktop, the OS on a cluster node is considered to be soft state that can be changed and/or updated rapidly.
This is clearly more heavywieght than the philosophy of configuration management tools [Cfengine] that perform exhaustive examination and parity checking of an installed OS. At first glance, it seems wrong to reinstall the OS when a configuration parameter needs to be changed. Indeed, for a single node this might seem too severe. However, this approach scales exceptionally well, making it a preferred mode for even a modest-sized cluster. Because the OS can be installed from scratch in a short period of time, different (and perhaps incompatible) application-specific configurations can easily be installed on nodes. In addition, this structure insures any upgrade will not interfere with actively running jobs.
One of the key ingredients of Rocks is a robust mechanism to produce customized distributions (with security patches pre-applied) that define the complete set of software for a particular node. A cluster may require several node types including compute nodes, frontend nodes file servers, and monitoring nodes. Each of these roles requires a specialized software set. Within a distribution, different node types are defined with a machine specific Red Hat Kickstart file, made from a Rocks Kickstart Graph.
A Kickstart file is a text-based description of all the software packages and software configuration to be deployed on a node. The Rocks Kickstart Graph is an XML-based tree structure used to define RedHat Kickstart files. By using a graph, Rocks can efficiently define node types without duplicating shared components. Similiar to mammalian species sharing 80% of their genes, Rocks node types share much of their software set. The Rocks Kickstart Graph easily defines the differences between node types without duplicating the description of their similarities. See the Bibliography section for papers that describe the design of this structure in more depth.
By leveraging this installation technology, we can abstract out many of the hardware differences and allow the Kickstart process to autodetect the correct hardware modules to load (e.g., disk subsystem type: SCSI, IDE, integrated RAID adapter; Ethernet interfaces; and high-speed network interfaces). Further, we benefit from the robust and rich support that commercial Linux distributions must have to be viable in todays rapidly advancing marketplace.
Wherever possible, Rocks uses automatic methods to determine configuration differences. Yet, because clusters are unified machines, there are a few services that require ``global knowledge of the machine -- e.g., a listing of all compute nodes for the hosts database and queuing system. Rocks uses an SQL database to store the definitions of these global configurations and then generates database reports to create service-specific configuration files (e.g., DHCP configuration file, /etc/hosts, and PBS nodes file).
Enhancements:
- Rocks v4.3 is released for i386 and x86_64 CPU architectures. New features: Rocks command line - initial release of the Rocks command line which facilitates non-SQL administrative access to the database; PXE First - hosts can now be configured in BIOS with a boot order of CD, PXE, hard disk. Enhancements: based on CentOS 4.5 and all updates as of July 4, 2007; Anaconda installer updated to 10.1.1.63; performance improvement when building torrent files for the Avalanche Installer; database indirects, more flexibility with Rocks variables; Globus updated to gt4.0.4 with web services....

Image Cluster copies and renames images based on Exif data and file number names. Image Cluster also clusters those images into directories based on a variable sliding window (with a default of 36 hours), which makes it easy to group images based on events without manual intervention.

The inspiration for this program came from recently getting a new Canon SD500 camera to replace my Canon S30 that Id had for years. The upside, the Canon SD500 rocks! The downside, I now have 2 cameras that are burning through the same sequence numbers, so my previous solution of just putting all the files in to directories by the first 2 digits of the sequence numbers was no longer going to work.

Imagecluster solves this problem, plus another grouping problem that Id been thinking about, by extracting the CreateDate and FileNumber exif tags from the images, and using that as the basis of a new filename (typically YYYY:mm:dd_HH:MM:SS_FileNumber.jpg). This ensures that 2 images taken at the same second have an even smaller chance of colliding, as their camera sequence numbers would have to also be the same at that second.

But that is just the first step. I have noticed that I am an occational photographer, so take pictures in bursts, often for a weekend of hanging out with folks, though sometimes for a vacation as well. This got me thinking. What I really needed is a tool that also creates directories that allows for some minimum tollerance between CreateDate, that is used to cluster images. For me, the optimum value seems to be 36 hours, though this is configurable via the command line.

This took me an afternoon to pull together, Im sure it could be smarter, but it is useful enough to post for others to use.

Options:

-d directory
Set the target directory for images. Defaults to /tmp/photos, which is probably not what you want.

-D
Dryrun. Tells you what the program would have done.

-h
Print out help message

-s
Seperator character. It defaults to : (i.e. 2005:10:09...), but is user configurable because my friend Clemens wants to use - (i.e. 2005-10-09) instead.

-t
Set the tollerance for image clustering. This is the maximum time between any 2 pictures in a cluster, which will cause a new cluster to be created. The name of the cluster will be YYYY:MM:DD of the first image in the cluster, even if it spans multiple days. Because this tollerance is the maximum time between any two images in the cluster, it is possible that all images you have ever taken could be in 1 cluster, if you took a picture every day of your life. Hence, this feature isnt useful to everyone. If you are that kind of person, set tollerance to 16 hours or something, and youll tend to get 1 day sized buckets.

-v
Prints verbose output

Algorithm::Cluster is a Perl interface to the C Clustering Library.

This module is an interface to the C Clustering Library, a general purpose library implementing functions for hierarchical clustering (pairwise simple, complete, average, and centroid linkage), along with k-means and k-medians clustering, and 2D self-organizing maps.

This library was developed at the Human Genome Center of the University of Tokyo. The C Clustering Library is distributed along with Cluster 3.0, an enhanced version of the famous Cluster program originally written by Michael Eisen while at Stanford University.

Solaris::Kstat is a Perl module to access Solaris Kstats from Perl.

SYNOPSIS

use Solaris::Kstat;
my $kstat = Solaris::Kstat->new();
my ($usr1, $sys1, $wio1, $idle1) =
@{$kstat->{cpu_stat}{0}{cpu_stat0}}{qw(user kernel wait idle)};
print("usr sys wio idlen");
while (1)
{
sleep 5;
if ($kstat->update()) { print("Configuration changedn"); }
my ($usr2, $sys2, $wio2, $idle2) =
@{$kstat->{cpu_stat}{0}{cpu_stat0}}{qw(user kernel wait idle)};
printf(" %.2d %.2d %.2d %.2dn",
($usr2 - $usr1) / 5, ($sys2 - $sys1) / 5,
($wio2 - $wio1) / 5, ($idle2 - $idle1) / 5);
$usr1 = $usr2; $sys1 = $sys2; $wio1 = $wio2; $idle1 = $idle2;
}

This module provides a tied hash interface to the Solaris kstats library. The kstats library allows you to get access to all the stats used by sar, iostat, vmstat etc, plus a lot of others that arent accessible through the usual utilities.
Solaris categorises statistics using a 3-part key - module, instance and name. For example, the root disk stats can be found under sd.0.sd0, and the cpu statistics can be found under cpu_stat.0.cpu_stat0, as in the above example. The method Solaris::Kstats-new()> creates a new 3-layer tree of perl hashes with exactly the same structure - i.e. the stats for disk 0 can be accessed as $ks-{sd}{0}{sd0}>. The bottom (4th) layer is a tied hash used to hold the individual statistics values for a particular system resource.

Creating a Solaris::Kstat object doesnt actually read all the possible statistics in, as this would be horribly slow and inefficient. Instead it creates a 3-layer structure as described above, and only reads in the individual statistics as you reference them. For example, accessing $ks-{sd}{0}{sd0}{reads} will read in all the statistics for sd0, including writes, bytes read/written, service times etc. Once you have accessed a bottom level statitics value, calling $ks->update() will automatically update all the individual values of any statistics that you have accessed.

Note that there are two values per bottom-level hash that can be read without causing the full set of statistics to be read from the kernel. These are "class" which is the kstat class of the statistics and "crtime" which is the time that the kstat was created. See kstat(3K) for full details of these fields.

Document clustering project is a data mining suite to cluster a document set. This set of tools was implemented from a series of papers: "Clustering Web Pages Semantically using Combinatorial Topology", "Data mining using granular computing", and "A fast association rule algorithm based on bitmap and granular computing".
Enhancements:
- A bug with hash table has been fixed.

BCCD - Bootable Cluster CD was created to facilitate instruction of parallel computing aspects and paradigms. Part of the difficulty instructors face is lack of dedicated resources to explore distributed computing aspects lack of time to preconfigure and test the supporting environment.

The BCCD image addresses this problem by providing a non-destructive overlay way to run a full-fledged parallel computing environment on just about any workstation-class system...Were happy to say that this now includes the MAC too!

The BCCD does share similarities with a few diskless solutions for clustering, such as the Warewulf project, the thin-OSCAR approach, Cluster Knoppix (only an openMosix system, no MPI/LAM/PVM build tools, ...), and so on. This is definitely the trend in HPC. But the main differences are that the BCCD will always fit in your pocket, be highly customizable for specific institutions needs, and will always be geared toward education and not dedicated clusters.

The "gar" build system also sets the BCCD apart from other projects. "gar" is a mix between BSDs "ports" system, Linux from scratch, and gentoo Linux. With gar, you can build an entire BCCD image from net-fetched sources in about two hours (assuming you have a primed ccache!).

The BCCD is also distinctly different from NPACI-Rocks, OSCAR, Cluster in a box or other type of mass-imaging clustering project for two reasons:

1. Its a non-destructive overlay on top of the current hardware. Once a system is rebooted, it reverts back to its original state. It is intended to be booted "over top" of a currently-configured Windows/Linux/BSD/etc. system.
2. Its focus in on educational aspects of High-Performance Computing (HPC) instead of the HPC core. Students will have a much better appreciation and understanding of how to tweak an MTU setting or wire the topology across a cluster if they understand how a distributed computation is laid out! Emphasis is placed upon building, configuring, and running distributed applications.

MySQL High Availability clustering is a set of scripts and programs that provide a high availability database cluster using MySQL replication.
MySQL High Availability clustering is transparent to client applications, as the cluster uses a shared logical IP to provide the service.
Enhancements:
- References have been changed from MASTER_NODE to CLUSTER_IP in takeover, failover, and slave_routine, when it was appropiate.
- This release introduces changes in compat.sh, several main cluster files, and the installation documentation, according to bug reports 1707251 and 1707212.