Free hpc software for linux

HPCToolkit is an open-source suite of multi-platform tools for profile-based performance analysis of applications. The figure provides an overview of the toolkit components and their relationships.
Main features:
- hpcrun: a tool for profiling executions of unmodified application binaries using statistical sampling of hardware performance counters.
- hpcprof & xprof: tools for interpeting sample-based execution profiles and relating them back to program source lines.
- bloop: a tool for analyzing application binaries to recover program structure; namely, to identify where loops are present and what program source lines they contain.
- hpcview: a tool for correlating program structure information, multiple sample-based performance profiles, and program source code to produce a performance database.
- hpcviewer: a java-based GUI for exploring databases consisting of performance information correlated with program source.
A program called hpcview is at the toolkits center. It takes performance profiles, program structure information, and, under the direction of a configuration file, correlates it with application source code to produce a browsable performance database.
hpcview also enables the user to define expressions to compute derived metrics as functions other metrics already defined (e.g. measured metrics read from data files or previously-computed derived metrics).
Performance databases are explored using our Java-based hpcviewer user interface that enables one to explore an applications performance data in a top-down fashion and enables one to easily navigate back and forth between performance data and source code.
The user interface presents performance data in a hierarchical display. At any time, you are looking at some program context (program, file, procedure, loop, or line). Also displayed is the data for both the parent and the children of the current context. Up and down arrows on the lines of the display are used to walk the hierarchy.
In order to speed up top-down analysis, the interface also provides `flatten and `un-flatten buttons. Their icons hint at their function. `Flatten modifies the hierarchy by eliding non-leaf children of the current node and replacing them with the grandchildren.
Unflatten reverses this. Since the tables are sorted, the flatten operation makes short work of diving into the program from the top to identify the most important files, procedures, loops and statements.
Performance data manipulated by hpcview can come from any source, as long as the profile data can be translated or saved directly to a standard, profile-like input format. To date, the principal sources of input data for hpcview have been hardware performance counter profiles.
Such profiles are generated by setting up a hardware counter to monitor events of interest (e.g., primary cache misses), to generate a trap when the counter overflows, and then to histogram the program counter values at which these traps occur. For Linux, we developed the hpcrun tool to collect profiles by sampling hardware performance counters.
This tool uses UTKs PAPI library for access to hardware performance counters. A second tool, hpcprof is used to map profiles collected using hpcrun back to program source lines. hpcprof is based on code from Curt Janssens cprof/vprof profiler. On operating systems other than Linux, we use vendor-supplied tools to collect profile data. On MIPS+Irix platforms, we use SGIs ssrun tool to collect profiles. On Alpha+Tru64, we use either with Compaqs uprofile or DCPI utilities for this purpose.
hpcview and hpcviewer can be used to view profile-like data of any type, not just data sampled from hardware performance counters. To analyze one program that contained many register spills, we built a perl script to examine assembly code generated by the SGI compilers for MIPS+Irix and create profiles that map register spills back to source code lines.
To facilitate automation, the programs in HPCToolkit are intended to be run using scripts and configuration files. Once these are set up, rerunning the program to collect new data, and all of the steps that go into generating a browsable dataset can be completely automated. The scripts automate the collection of data and conversion of profile data into a common, XML-based format.
Other performance tools (e.g. SGIs ssrun) report performance data at the line, procedure, and program level. However, since much of the time in scientific programs is spent in loops; having data at the loop level as well is critical to facilitate performance tuning.
For this reason, HPCToolkit includes a binary analyzer bloop that extracts loop nesting structure from application binaries and uses symbol table line map information to map this structure back to the source programs level. Because bloop works on binaries, this process is independent of the language used (though in practice it can be somewhat compiler dependent).
The loop nesting structure information produced by bloop enables hpcview to associate performance data with each loop in a program without incurring any additional overhead for data collection during program execution.
Supported platforms: Pentium+Linux, Opteron+Linux, Athlon+Linux, Itanium+Linux, Alpha+Tru64 and MIPS+Irix.
HPCToolkit is open-source software released with a BSD-like license.

ComputeMode project is an Icatis product that extends an organizations Computational Grid through aggregation of unused computing resources.
For instance, ComputeMode lets a virtual cluster to be built using employees PCs at their idle times. Practically most PCs in large companies are not used at night, on weekends, and during employee vacations, training periods or business trips.
So ComputeMode allows you to take the benefits of the powerful computation infrastructure you already have, but you are not aware of!
Another interesting use of ComputeMode is within computer classrooms, where PCs can be automatically harnessed into a virtual cluster when they are not being used (night, week-ends, holidays).
ComputeMode relies on a ComputeMode Server that keeps track of the availability of some dedicated PCs on the local network. Each PC Owner has a weekly availability Schedule, identifying periods when he or she is not using his/her PC. For instance, a PC owner can declare working periods from 8:00 to 18:00, Monday to Friday.
In the remainder of this document, we will refer to PCs handled by ComputeMode as Processing Nodes.
The ComputeMode Administrator can easily manage the computing PCs through a Web-based interface, accessed through the ComputeMode Server.
A Grid User can submit some computational Jobs to the system through the use of a classical Batch Manager (also called "Job Management System", "Distributed Resource Managers", or "Queuing System").
The Grid User can log onto the Batch Manager from any machine of the local network (usually through the "ssh" secure shell). The Batch Manager will then reserve appropriate resources for the computations, Schedule the execution of the Jobs depending on the overall load on the Computing Grid and allocate Jobs to available computing resources.
The OAR Batch Manager is installed by default with ComputeMode, even though other products such as Platform LSF, OpenPBS or Sun Grid Engine are supported by ComputeMode.
ComputeMode can monitor the load on the Batch Manager, and detect overloads. In such cases it can allocate available Processing Nodes to computational tasks.
Each Processing Node has two different operating modes:
- The User Mode, in which the machine is working in its standard way under Microsoft Windows for instance. The Owner of the machine will not even notice that his or her PC is managed as a ComputeMode Processing Node. In particular, the computational resources of the PC will not be used while in User Mode.
- The Computation Mode (this is where the name of our product comes from) is activated when the machine is in a time period where the PC is declared as "idle". In case the ComputeMode Server detects some computational peak, the PC can be remotely switched to Computation Mode. The switch from user mode is done through an automatic reboot of the machine and proceeds to a remote boot. The remote boot is handled by the ComputeMode Server with the "Preboot eXecution Environment" (PXE) protocol, which is natively available from the BIOS of PCs since 1999. While in Computation Mode, the machine is running under the Linux Operating System, and does not have any access to any local hard disk.
When the ComputeMode Server detects that some given Processing Node is no longer available for computation, it restores the machine back to its User Mode, so that the Owner will not even notice that his or her PC has been used by ComputeMode.
The submitted Jobs can take advantage of the NFS distributed file system, which is made available from the ComputeMode Server. Each Grid User has his or her own private directory and can use it for data required by the computational Jobs, as well as to retrieve the produced output files.
A specific case may happen if a PC Owner comes back and needs his or her PC while a computational Job is being processed. This can happen, for instance, if the Owner has an urgent need and comes back at night to work on his/her PC. In such a situation, the owner always has priority over the machine. Just by using the keyboard, the Owner can abort any ongoing computational activity on his/her PC, and ComputeMode will restore the machine to its User Mode in about one minute.
Enhancements:
- UI usability improvements, code cleanup, and fixes were made.
- The client and server system were upgraded.
- New, simpler node import/export was added.
- It is available as a virtual appliance.

MUNGE Uid N Gid Emporium is an authentication service for creating and validating credentials. It is designed to be highly scalable for use in an HPC cluster environment.
It allows a process to authenticate the UID and GID of another local or remote process within a group of hosts having common users and groups. These hosts form a security realm that is defined by a shared cryptographic key.
Clients within this security realm can create and validate credentials without the use of root privileges, reserved ports, or platform-specific methods.
Rationale
The need for MUNGE arose out of the HPC cluster environment. Consider the scenario in which a local daemon running on a login node receives a client request and forwards it on to remote daemons running on compute nodes within the cluster. Since the user has already logged on to the login node, the local daemon just needs a reliable means of ascertaining the UID and GID of the client process. Furthermore, the remote daemons need a mechanism to ensure the forwarded authentication data has not been subsequently altered.
A common solution to this problem is to use Unix domain sockets to determine the identity of the local client, and then forward this information on to remote hosts via trusted rsh connections. But this presents several new problems. First, there is no portable API for determining the identity of a client over a Unix domain socket. Second, rsh connections must originate from a reserved port; the limited number of reserved ports available on a given host directly limits scalability. Third, root privileges are required in order to bind to a reserved port. Finally, the remote daemons have no means of determining whether the client identity is authentic.
Overview
A process creates a credential by requesting one from the local MUNGE service. The encoded credential contains the UID and GID of the originating process. This process sends the credential to another process within the security realm as a means of proving its identity. The receiving process validates the credential with the use of its local MUNGE service. The decoded credential provides the receiving process with a reliable means of ascertaining the UID and GID of the originating process. This information can be used for accounting or access control decisions.
The contents of the credential (including any optional payload data) are encrypted with a key shared by all munged daemons within the security realm. The integrity of the credential is ensured by a message authentication code (MAC). The credential is valid for a limited time defined by its time-to-live (TTL). The daemon ensures unexpired credentials are not replayed on a particular host. Decoding of a credential can be restricted to a particular user and/or group ID. The payload data can be used for purposes such as embedding the destinations address to ensure the credential is only valid on a specific host. The internal format of the credential is encoded in a platform-independent manner. And the credential itself is base64 encoded to allow it to be transmitted over virtually any transport.
Enhancements:
- A bug was fixed that caused stack corruption on AMD-64 when using Libgcrypt.

GlusterFS package contains clustered file storage that can scale to peta bytes. GlusterFS is a programmable system. With little thinking, you can even redesign the GlusterFS file system by re-arranging the GlusterFS components using translator interface. It is all achieved through volume specification file. This allows GlusterFS to be flexible for all kinds of storage needs. Even with all these advanced features, GlusterFS is very easy to setup and manage.

Gluster is a GNU cluster distribution aimed at commoditizing Supercomputing and Superstorage. Core of the Gluster provides a platform for developing clustering applications tailored for a specific tasks such as HPC Clustering, Storage Clustering, Enterprise Provisioning, Database Clustering etc.