Free distributed system software for linux

Coda is an advanced networked filesystem. The project has been developed at CMU since 1987 by the systems group of M. Satyanarayanan. in the SCS department.
Coda is a distributed filesystem with its origin in AFS2. It has many features that are very desirable for network filesystems. Currently, Coda has several features not found elsewhere.
1. disconnected operation for mobile computing
2. is freely available under a liberal license
3. high performance through client side persistent caching
4. server replication
5. security model for authentication, encryption and access control
6. continued operation during partial network failures in server network
7. network bandwith adaptation
8. good scalability
9. well defined semantics of sharing, even in the presence of network failures
Enhancements:
- Coda now works on 64-bit systems.
- The client and server were successfully built and used on a machine with em64t extensions with a 64-bit Linux kernel.
- The new RVM-1.14 and RPC2-2.5 releases also incorporate some essential 64-bit fixes.
- Another welcome change is that venus now tries to keep cached access rights available for users even across system reboots or client restarts, which should improve life for people who frequently shut down or dual boot their machines.

Distributed Internet Backup System works by doing all its communication through email. The benefit of using email for transport is that clients behind firewalls or with intermittent connections to the Internet can use DIBS reliably.
Since disk drives are cheap, backup should be cheap too. Of course it does not help to mirror your data by adding more disks to your own computer because a virus, fire, flood, robbery, power surge, etc. could still wipe out your local data center.
Instead, you should give your files to peers (and in return store their files) so that if a catastrophe strikes your area, you can recover data from surviving peers. The Distributed Internet Backup System (DIBS) is designed to implement this vision.
Note that DIBS is a backup system not a file sharing system like Napster, Gnutella, Kazaa, etc. In fact, DIBS encrypts all data transmissions so that the peers you trade files with can not access your data.
Enhancements:
- Added Peer Finder service to allow advertising and automated exchange of peering information. See manual for details.
- Updated add_peer and edit_peer commands to allow sizes to be specified using k, m, g, t (e.g., 10k, 10m, 10g, 10t).
- Updated the protocol DIBS uses to exchange store, unstore, probe, and other messages between peers. The new protocol is XML based to allow easier debugging, parsing, and extensions.
- Fixed a deadlock bug reported by Jason Martin in a message to the dibs-discussion mailing list.
- Made the daemon run its periodic checks like spawnning auto_check, process_message, etc., as soon as it starts up.
- Other minor bug fixes and improvements

Distributed BEAGLE provides a distributed evolutionary computation system.
Distributed BEAGLE is a master-slave distribution extension of the evolutionary computation framework, Open BEAGLE. Its key features are robustness, fault tolerance, adaptability for heterogeneous networks, and transparency for the user.
Main features:
- Based on the generic framework Open BEAGLE
- Minimal changes required to modify an Open BEAGLE application for distribution
- Ten GA and GP examples converted from Open BEAGLE
- Communication based on TCP sockets using XML encoding
- Data persistency insured on the server by a SQL Database (SQLite)
- Load balancing of computational tasks for uses on heterogeneous LAN
- Open source (LGPL license)
- OS-calls wrapped into generic C++ classes
- Compiles on UNIX (Linux/OS X) with gcc 3.x and Windows with Visual Studio .NET
Enhancements:
- Bug fixe: variable deme size could crash the server (Thanks CG).
- Suggestion: Compression of connections can now be activated by clients if the option is set to "-1".

Distributed Concurrent Versioning System (DCVS) project extends the well-known version control system CVS and the file distribution and synchronization program CVSup with functionality to distribute CVS repositories with local lines of development and handle synchronization of the distributed repositories automatically in the background.
Development lines (branches) are owned by a repository server, repository servers efficiently update each other via CVSup, and CVS ensures correct server use on checkin and branch creation.
Enhancements:
- This release adds miscellaneous bugfixes and the security patches of CVS 1.12.13.
- There are now installation packages for Windows to be installed on Cygwin, for SUSE Linux 10, for FreeBSD 4, 5, and 6, and for Fedora Core 4.

Distributed RSA Key Generation is a software to generate shared RSA keys. Currently, only keys for two parties using Gilboas protocol are supported. But the protocol stack is extensible, so in the future there may be other protocols supported.

Work has begun on Straubs protocol but is not finished yet, multi-party protocols like Boneh-Franklins may also be added in the far future.

Distributed Aide Runtime Controller is a multi-threaded Python application designed for managing AIDE installations in large heterogeneous networks.
AIDE is an open-source improvement upon the academic release of Tripwire. It can be used to detect filesystem changes in unix environments, which is useful for forensics on compromised systems and tracing illicit system configuration changes.
Darc provides a mechanism to run AIDE integrity checks across many unix systems from a single management station. It has the following features not available in a traditional AIDE installation:
- Maintaining read-only media databases on each system - not a trivial task! - is not required for day to day operations.
- Unified reporting - the admin doesnt have to read individual reports for each system.
- Integrated syslog support to notify admins when a system may have been compromised.
- Databases and configs are never written to the filesystems on the monitored hosts.
Enhancements:
- Better error handling and reporting
- HTML reports for easier navigation within the report
- Configurable timeout values for all relevant metrics - TCP connection timeouts, SFTP transfer time, AIDE run time
- Built-in support for aide database maintenance tasks (init, update) so the manual file manipulation required in 0.2 is completely eliminated.

Twisted Storage is an application that will take your data and manage it for you, just like a file system. But Twisted Storage goes way beyond what a normal file system does.
What types of "objects" are stored in Twisted Storage : just regular files. The data stored in the system is expected to be fixed content, meaning it doesnt change at all. If you read the data back, change it, and store it, it is considered a different file. There are plenty of examples of fixed content data : emails you received, movies or mp3 files you made or own, documents you wrote, etc. In fact it is estimated that 80% of the data you have on your computer is fixed, reference data.
Your Twisted Storage system can start out small - a single computer with a few disk drives. When you need more storage you can add more disk drives and more computers. But it has been designed to store petabytes worth of data without requiring any special hardware. For example storing a petabyte worth of data, using the biggest disk drives available today (500 GB), would require 2000 disk drives. To overcome the need of special hardware, Twisted Storage is a scalable, distributed system using ordinary commodity computers. Even so that petabyte of data would require 500 computers networked together and that requires special handling.
Twisted Storage has a few features you wont find in any file system. For example once a file is added to the Twisted Storage repository it can be written in multiple locations simultaneously. The system has been designed to be compliant with Sarbanes-Oxley Act: when an object is added to the repository you can no longer delete it and the integrity of the contents are guaranteed. In addition any access to the object is recorded.
Main features:
Scalable to thousands of servers and multiple petabytes of data.
- Regardless of how much data you want to store,Twisted Storage is designed to handle it. The system can grow from one computer and a few disks to hundreds of systems and thousands of disk drivers.
Automatic data recovery without human intervention.
- Twisted Storage doesnt require backup/restore software or procedures. All data written to the system is recorded in multiple locations.
No special disaster recover procedures.
- So long as you have your systems in physically different locations and you have instructed the software to write the multiple copies to those systems, you have no-hassle disaster recover. If one of the sites goes down, the system will retrieve any data from the other systems. Additionally, once the disaster site is up and running, the system will automatically restore the data to the it.
Simplified configuration.
- To ease management of the system, you can change any one systems configuration and have it ripple through out the system.

SPyDI Is a powerful engine to create distributed full text indexing systems and distributed search engines.
Simple Python Distributed Indexing library supports harvesting, crawling (pull mehtods), and push methods (via a Web interface or SPyRO Web services).
It supports boolean and vector Information retrieval models. It has few dependencies, and comes with its own HTTP server and HTML embedded pages language (called pyew and wey pages), and session manager.
It can use the SMTP of the Python library. It supports replacing the default modules with some better modules (Apache, exim, etc).
Enhancements:
- Monarca updates to support SPyROs new HTTP protocol management.
- Some bugfixes in pyew pages.
- General code cleanup.