GAMMA 05-08-22

GAMMA is a low latency, high throughput communication system for clusters of PCs.
Network Of Workstations (NOWs) and clusters of PCs interconnected by modern, industry-standard LAN fabrics (Gigabit Ethernet, Myrinet, SCI) and running the Linux operating system, have became an attractive and cost-effective architecture for parallel and distributed applications.
The usual drawback of a standard PC cluster is the poor performance of the support to inter-process communication over the interconnect.
Current implementations of industry-standard communication primitives, APIs, and protocols, usually show high communication latencies and low communication throughput.
We have developed a system for inter-process communication, called the Genoa Active Message MAchine (GAMMA). GAMMA runs on Linux clusters of PCs with Intel IA-32 processors (Intel Pentium, AMD K6, and superior models), networked by a Gigabit Ethernet.
The core of GAMMA is a custom Linux network device driver, which operates the Network Interface Card (NIC).
The GAMMA driver delivers low latency, high throughput communication services based on Active Ports, a mechanism derived from Active Messages.
Both point-to-point and broadcast communications are provided. Broadcast communication exploits the Ethernet broadcast directly.
The GAMMA driver is able to manage standard IP traffic as long as no parallel job is running. Therefore, all IP services are up and running whenever the cluster is not in use by any parallel job.
The communication mechanisms implemented in the GAMMA driver are made available to application writers through the GAMMA user library.
The GAMMA library provides support to application launch, process grouping, point-to-point/broadcast communications based on the Active Ports mechanisms, and some collective routines (barrier synchronization, and broadcast).
GAMMA provides two levels of QoS. The lower one, corresponding to the fastest communications, is a best-effort service. With this service, network congestion and ``hot spots may cause the receiver NIC or even the LAN switch to loose packets by overrun.
The other QoS level provides flow-controlled communication, ensuring reliability up to hardware faults, at a negligible performance penalty.
Installing the GAMMA driver requires only two small and marginal patches to the original Linux kernel. The Linux kernel extended by the GAMMA driver must be installed on each PC in the cluster.
Main features:
- Supports both single and dual CPU processing nodes (Intel IA-32)
- Runs on Gigabit Ethernet
- SPMD parallel processing with message passing
- Can run IP traffic when not in use
- Good programmability thanks to fairly high abstraction level
- Reliable thanks to mechanisms for retransmission of missing packets
- Implemented as a network device driver for Linux 2.6, and released under GNU GPL
Version restrictions:
GAMMA allows more process instances of the same parallel job to run on the same CPU. Thread safety is granted with point-to-point communications when running on distinct GAMMA ports.
However, collective routines (barrier synchronization, and broadcast) are not thread compatible, as they use predetermined GAMMA ports.
The packet retransmission mechanism is not perfect. It will not work if the missing packet originates from a non-blocking send.