Free clustering software for linux

MySQL High Availability clustering is a set of scripts and programs that provide a high availability database cluster using MySQL replication.
MySQL High Availability clustering is transparent to client applications, as the cluster uses a shared logical IP to provide the service.
Enhancements:
- References have been changed from MASTER_NODE to CLUSTER_IP in takeover, failover, and slave_routine, when it was appropiate.
- This release introduces changes in compat.sh, several main cluster files, and the installation documentation, according to bug reports 1707251 and 1707212.

GlusterFS package contains clustered file storage that can scale to peta bytes. GlusterFS is a programmable system. With little thinking, you can even redesign the GlusterFS file system by re-arranging the GlusterFS components using translator interface. It is all achieved through volume specification file. This allows GlusterFS to be flexible for all kinds of storage needs. Even with all these advanced features, GlusterFS is very easy to setup and manage.

Gluster is a GNU cluster distribution aimed at commoditizing Supercomputing and Superstorage. Core of the Gluster provides a platform for developing clustering applications tailored for a specific tasks such as HPC Clustering, Storage Clustering, Enterprise Provisioning, Database Clustering etc.

Rocks Cluster is a complete "cluster on a CD" solution for x86 and IA64 Red Hat Linux COTS clusters.
Building a Rocks cluster does not require any experience in clustering, yet a cluster architect will find a flexible and programmatic way to redesign the entire software stack just below the surface (appropriately hidden from the majority of users).
Although Rocks includes the tools expected from any clustering software stack (PBS, Maui, GM support, Ganglia, etc), it is unique in its simplicity of installation.
From a hardware component and raw processing power perspective, commodity clusters are phenomenal price/performance compute engines. However, if a scalable ``cluster management strategy is not adopted, the favorable economics of clusters are offset by the additional on-going personnel costs involved to ``care and feed for the machine. The complexity of cluster management (e.g., determining if all nodes have a consistent set of software) often overwhelms part-time cluster administrators, who are usually domain application scientists. When this occurs, machine state is forced to either of two extremes: the cluster is not stable due to configuration problems, or software becomes stale, security holes abound, and known software bugs remain unpatched.
While earlier clustering toolkits expend a great deal of effort (i.e., software) to compare configurations of nodes, Rocks makes complete Operating System (OS) installation on a node the basic management tool. With attention to complete automation of this process, it becomes faster to reinstall all nodes to a known configuration than it is to determine if nodes were out of synchronization in the first place. Unlike a users desktop, the OS on a cluster node is considered to be soft state that can be changed and/or updated rapidly.
This is clearly more heavywieght than the philosophy of configuration management tools [Cfengine] that perform exhaustive examination and parity checking of an installed OS. At first glance, it seems wrong to reinstall the OS when a configuration parameter needs to be changed. Indeed, for a single node this might seem too severe. However, this approach scales exceptionally well, making it a preferred mode for even a modest-sized cluster. Because the OS can be installed from scratch in a short period of time, different (and perhaps incompatible) application-specific configurations can easily be installed on nodes. In addition, this structure insures any upgrade will not interfere with actively running jobs.
One of the key ingredients of Rocks is a robust mechanism to produce customized distributions (with security patches pre-applied) that define the complete set of software for a particular node. A cluster may require several node types including compute nodes, frontend nodes file servers, and monitoring nodes. Each of these roles requires a specialized software set. Within a distribution, different node types are defined with a machine specific Red Hat Kickstart file, made from a Rocks Kickstart Graph.
A Kickstart file is a text-based description of all the software packages and software configuration to be deployed on a node. The Rocks Kickstart Graph is an XML-based tree structure used to define RedHat Kickstart files. By using a graph, Rocks can efficiently define node types without duplicating shared components. Similiar to mammalian species sharing 80% of their genes, Rocks node types share much of their software set. The Rocks Kickstart Graph easily defines the differences between node types without duplicating the description of their similarities. See the Bibliography section for papers that describe the design of this structure in more depth.
By leveraging this installation technology, we can abstract out many of the hardware differences and allow the Kickstart process to autodetect the correct hardware modules to load (e.g., disk subsystem type: SCSI, IDE, integrated RAID adapter; Ethernet interfaces; and high-speed network interfaces). Further, we benefit from the robust and rich support that commercial Linux distributions must have to be viable in todays rapidly advancing marketplace.
Wherever possible, Rocks uses automatic methods to determine configuration differences. Yet, because clusters are unified machines, there are a few services that require ``global knowledge of the machine -- e.g., a listing of all compute nodes for the hosts database and queuing system. Rocks uses an SQL database to store the definitions of these global configurations and then generates database reports to create service-specific configuration files (e.g., DHCP configuration file, /etc/hosts, and PBS nodes file).
Enhancements:
- Rocks v4.3 is released for i386 and x86_64 CPU architectures. New features: Rocks command line - initial release of the Rocks command line which facilitates non-SQL administrative access to the database; PXE First - hosts can now be configured in BIOS with a boot order of CD, PXE, hard disk. Enhancements: based on CentOS 4.5 and all updates as of July 4, 2007; Anaconda installer updated to 10.1.1.63; performance improvement when building torrent files for the Avalanche Installer; database indirects, more flexibility with Rocks variables; Globus updated to gt4.0.4 with web services....

Image Cluster copies and renames images based on Exif data and file number names. Image Cluster also clusters those images into directories based on a variable sliding window (with a default of 36 hours), which makes it easy to group images based on events without manual intervention.

The inspiration for this program came from recently getting a new Canon SD500 camera to replace my Canon S30 that Id had for years. The upside, the Canon SD500 rocks! The downside, I now have 2 cameras that are burning through the same sequence numbers, so my previous solution of just putting all the files in to directories by the first 2 digits of the sequence numbers was no longer going to work.

Imagecluster solves this problem, plus another grouping problem that Id been thinking about, by extracting the CreateDate and FileNumber exif tags from the images, and using that as the basis of a new filename (typically YYYY:mm:dd_HH:MM:SS_FileNumber.jpg). This ensures that 2 images taken at the same second have an even smaller chance of colliding, as their camera sequence numbers would have to also be the same at that second.

But that is just the first step. I have noticed that I am an occational photographer, so take pictures in bursts, often for a weekend of hanging out with folks, though sometimes for a vacation as well. This got me thinking. What I really needed is a tool that also creates directories that allows for some minimum tollerance between CreateDate, that is used to cluster images. For me, the optimum value seems to be 36 hours, though this is configurable via the command line.

This took me an afternoon to pull together, Im sure it could be smarter, but it is useful enough to post for others to use.

Options:

-d directory
Set the target directory for images. Defaults to /tmp/photos, which is probably not what you want.

-D
Dryrun. Tells you what the program would have done.

-h
Print out help message

-s
Seperator character. It defaults to : (i.e. 2005:10:09...), but is user configurable because my friend Clemens wants to use - (i.e. 2005-10-09) instead.

-t
Set the tollerance for image clustering. This is the maximum time between any 2 pictures in a cluster, which will cause a new cluster to be created. The name of the cluster will be YYYY:MM:DD of the first image in the cluster, even if it spans multiple days. Because this tollerance is the maximum time between any two images in the cluster, it is possible that all images you have ever taken could be in 1 cluster, if you took a picture every day of your life. Hence, this feature isnt useful to everyone. If you are that kind of person, set tollerance to 16 hours or something, and youll tend to get 1 day sized buckets.

-v
Prints verbose output

SenseClusters is a natural language processing package that allows you to cluster similar contexts or to identify clusters of related words.
SenseClusters supports its own native methods based on first and second order representations of context, and also supports Latent Semantic Analysis. It is fully unsupervised, and can automatically discover the optimal number of clusters in your text.
SenseClusters is a complete system that takes users from preprocessing of raw text to providing clustered output.
Enhancements:
- Full support for Latent Semantic Analysis was introduced.
- Both contexts and words may be clustered using either native SenseClusters methods (first or second order) or Latent Semantic Analysis.

BioCluster is a peer-to-peer clustering platform for Asterisk, the open source PBX, which allows Asterisk to be used as a full carrier-grade telephony solution. This project is meant to be installed on several machines together with Asterisk, turning them into a VoIP cluster.

While the BioCluster peer-to-peer protocol was initially designed to cater to Asterisk-based clustering solutions, the BioCluster framework is capable of being extended to support various forms of normally unclustered devices or software packages.

The Finishing Scripts for Cluster Installations handle specific post-installation configuration that might not be convienent nor possible using existing cluster installation methods.

The usual installation process is used to build a reasonably configured node, and the system then reboots into normal mode, achieves network visibility, and executes the finishing script.

The finishing script handles all of the finer details of installing packaged or non-packaged software, tweaking installation, setting host/net specific parameters/files, etc. It is controlled via a single, easily modified script.