Free designed performance software for linux

HPL is a software package that solves a (random) dense linear system in double precision (64 bits) arithmetic on distributed-memory computers. It can thus be regarded as a portable as well as freely available implementation of the High Performance Computing Linpack Benchmark.

The algorithm used by HPL can be summarized by the following keywords: Two-dimensional block-cyclic data distribution - Right-looking variant of the LU factorization with row partial pivoting featuring multiple look-ahead depths - Recursive panel factorization with pivot search and column broadcast combined - Various virtual panel broadcast topologies - bandwidth reducing swap-broadcast algorithm - backward substitution with look-ahead of depth 1.

The HPL package provides a testing and timing program to quantify the accuracy of the obtained solution as well as the time it took to compute it. The best performance achievable by this software on your system depends on a large variety of factors.

Nonetheless, with some restrictive assumptions on the interconnection network, the algorithm described here and its attached implementation are scalable in the sense that their parallel efficiency is maintained constant with respect to the per processor memory usage.

The HPL software package requires the availibility on your system of an implementation of the Message Passing Interface MPI (1.1 compliant). An implementation of either the Basic Linear Algebra Subprograms BLAS or the Vector Signal Image Processing Library VSIPL is also needed. Machine-specific as well as generic implementations of MPI, the BLAS and VSIPL are available for a large variety of systems.

pcpViewer is a 3D viewer of data gathered through the excellent "Performance Co-Pilot" library.

You can see usage of CPU time, net devices, memory, hard drives, and virtually any data exported by the pcp library and daemon.

I first started this "pet project" as a 3D xosview replacement (thanks for inspiration), so one of the goal is to get the same level of responsiveness as xosview.

The Bioinformatics Benchmark System is an attempt to build a reasonable testing framework, tests, and data, to enable end users and vendors to probe the performance of their systems.

What we are trying to do is to create a framework for testing, and a core set of tests that all may download and use to probe specific elements of systems performance.

Moreover, the source to these tests are available under GPL, and are hosted on Bioinformatics.org and Scalable Informatics LLC The idea is to enable end users, consumers, systems developers, and others to easily build and use meaningful tests for measurement and tuning reasons.

Joe Landman from Scalable Informatics LLC conceived the idea and wrote the original codes. We are looking for additional benchmark code suggestions, tests, data sets, etc.

Current baseline tests are several NCBI BLAST runs, several HMMer runs, and a variety of others. We plan to include ClustalW, X!Tandem, various chemistry, dynamics, and related tests, as well as several others.

Tests such as LINPACK or HPL simply do not provide meaningful performance indicators or predictive models for high performance informatics. Unfortunately, nor do a number of more recent and focused tests.

This is a problem as LINPACK and HPL specifically test the performance on various matrix operations, where you have effectively regular memory access patterns, and specific mathematical operations.

These codes are most useful for comparison to codes with heavy floating point operations, and interleaved memory traffic. These codes were not designed for comprehensive systems benchmarking, where disk I/O, memory latency, and other factors all contribute to the performance issues.

The best tests are the ones that are most similar to the codes you will run on the machine. The tests themselves should be reasonable approximations to a real execution of your code, using real data. You may need to pare it back in order to get realistic run times.

You should have a reasonable subset of data sizes. A single test does not tell you how your system scales, and one of the reasons for the existance of this test is specifically to allow you to test the performance while you increase various aspects of the workload.

You rarely get a quiescent system in a cluster, so we would recommend that you try to run in as realistic an operating environment as possible. A baseline in a quiescent system is fine, but it may set your expectations unreasonably.
top

hYPerSonic is a program for building and manipulating signal processing pipelines from Python scripts. It is designed for real-time control. It includes objects for oscillators, filters, file IO, and soundcard and memory operations. It is low-level: every byte counts.

Just recently worked out how to call back into the python interpreter from the portaudio (v18) callback. This is primarily for OSX (and windows?) where the mutex performance was unacceptable. Unfortunately this callback style (so far) breaks all existing code (which is based on a main loop in python).

It is working currently only on Linux and OSX.

PAPI aims to provide the tool designer and application engineer with a consistent interface and methodology for use of the performance counter hardware found in most major microprocessors.
PAPI enables software engineers to see, in near real time, the relation between software performance and processor events.
The Performance API (PAPI) project specifies a standard application programming interface (API) for accessing hardware performance counters available on most modern microprocessors.
These counters exist as a small set of registers that count Events, occurrences of specific signals related to the processors function. Monitoring these events facilitates correlation between the structure of source/object code and the efficiency of the mapping of that code to the underlying architecture.
This correlation has a variety of uses in performance analysis including hand tuning, compiler optimization, debugging, benchmarking, monitoring and performance modeling. In addition, it is hoped that this information will prove useful in the development of new compilation technology as well as in steering architectural development towards alleviating commonly occurring bottlenecks in high performance computing.
PAPI provides two interfaces to the underlying counter hardware; a simple, high level interface for the acquisition of simple measurements and a fully programmable, low level interface directed towards users with more sophisticated needs.
The low level PAPI interface deals with hardware events in groups called EventSets. EventSets reflect how the counters are most frequently used, such as taking simultaneous measurements of different hardware events and relating them to one another.
For example, relating cycles to memory references or flops to level 1 cache misses can indicate poor locality and memory management. In addition, EventSets allow a highly efficient implementation which translates to more detailed and accurate measurements.
EventSets are fully programmable and have features such as guaranteed thread safety, writing of counter values, multiplexing and notification on threshold crossing, as well as processor specific features. The high level interface simply provides the ability to start, stop and read specific events, one at a time.
PAPI provides portability across different platforms. It uses the same routines with similar argument lists to control and access the counters for every architecture. As part of PAPI, we have predefined a set of events that we feel represents the lowest common denominator of every good counter implementation.
Our intent is that the same source code will count similar and possibly comparable events when run on different platforms. If the programmer chooses to use this set of standardized events, then the source code need not be changed and only a fresh compilation and link is necessary. However, should the developer wish to access machine specific events, the low level API provides access to all available events and counting modes.
If an event or feature does not exist on the current platform, PAPI returns an appropriate error code. This significantly reduces the porting effort of code using PAPI because the semantics of each call to PAPI remains the same, just the argument lists need updating. In addition to the standard set, each PAPI implementation supports all native events through the ability to directly accept platform specific counter numbers. Definitions for most, if not all of these, are included as conditional macros in the header file. In this way, PAPI avoids having inefficient code to translate all events for all platforms into a uniform representation and back again.
This translation is only done for the relatively few events defined in the standardized set. Some processors like those in the POWER series have counter groups. They enable access to specific groups of counters, instead of individual events. This presents a serious portability problem, thus PAPI abstracts hardware counters from their groups with a packed naming scheme. Each counter control value or event is made up of the counter group number and the number of the specific counter in that group.
PAPI can be divided into two layers of software. The upper layer consists of the API and machine independent support functions. The lower layer defines and exports a machine independent interface to machine dependent functions and data structures. These functions access the substrate, which may consist of the operating system, a kernel extension or assembly functions to directly access the processors registers.
PAPI tries to use the most efficient and flexible of the three, depending on what is available. Naturally, the functionality of the upper layers heavily depends on that provided by the substrate. In cases where the substrates do not provide highly desirable features, PAPI attempts to emulate them as described below.
PAPI makes sure the underlying operating system or library guards against overflow of counter values.
Each counter can potentially be incremented multiple times in a single clock cycle. This combined with increasing clock speeds and the small precision of some of the physical counters means that overflow is likely to occur.
One of the more advanced features of PAPI is to provide a portable implementation of asynchronous notification when counters exceed a user specified value.
This functionality provides the basis for PAPIs SVR4 compatible profiling calls, that generate an accurate histogram of performance interrupts based on hardware metrics, not on time. Such functionality provides the basis for all line level performance analysis software, from the antiquated days of AT&Ts prof to SGIs SpeedShop. Thus for any architecture with even the most rudimentary access to hardware performance counters, PAPI provides the foundation for a truly portable, source level, performance analysis tool based on real processor statistics.
Enhancements:
- The API was extended to decouple abstraction layers from hardware support and to provide initial support for different types of performance counters.

Unified Parallel C, in short UPC, is an extension of the C programming language designed for high performance computing on large-scale parallel machines.

The language provides a uniform programming model for both shared and distributed memory hardware.

The programmer is presented with a single shared, partitioned address space, where variables may be directly read and written by any processor, but each variable is physically associated with a single processor.

UPC uses a Single Program Multiple Data (SPMD) model of computation in which the amount of parallelism is fixed at program startup time, typically with a single thread of execution per processor.

Whats New in This Release:

- Add initial native support for the Cray XT3 via new portals network
- Implement the GASP 1.5 performance instrumentation interface, supporting the
Parallel Performance Wizard (PPW) and other third-party profiling tools.
- Add bupc_ticks_to_ns() - finer granularity timer query
- Add the Berkeley implementations of the UPC collectives and UPC-IO to GCCUPC+UPCR
- Add most of the Berkeley UPC library extensions to GCCUPC+UPCR
- Add upcdecl command-line tool (also online at: http://upc.lbl.gov/upcdecl)
- Add support for alloca() and stdarg.h
- Performance improvements to the BUPC semaphore library for signalling store
- Add bupc_thread_distance() - runtime thread layout query for hierarchical systems
- Add a remote fetch-and-add UPC library extension (initially just for 64-bit ints)
- Allow configure-time tuning of bit distribution in packed pointer-to-shared rep
- Fix the following notable bugs in 2.2.2 (see http://upc-bugs.lbl.gov for details):
- bug525: optimizer crashes on Tru64/CompaqC for libgasnet
- bug1229: More robust preprocessing on Compaq C
- bug1389: ansi-aliasing violations on small local put/get copies
- bug1531: improved lock fairness to remote lock requests
- bug1594: timer inaccuracies on Cray X1E
- bug1645: preprocess-time failure Backslash found where operator expected
- bug1657: PACKAGE_* symbols exposed to UPC code on GCCUPC+UPCR
- bug1683: improve upcrun handling of -shared-heap-max
- bug 1743: More robust behavior when backend C compiler changes
- Improved SRV-based DNS failover for upcc HTTP translation
- Add gzip compression to HTTP netcompile, for faster compiles over slow links
- Improved robustness for SSH netcompile to handle stray output from dotfiles
- Numerous misc minor bug fixes