Free benchmark software for linux

HPL is a software package that solves a (random) dense linear system in double precision (64 bits) arithmetic on distributed-memory computers. It can thus be regarded as a portable as well as freely available implementation of the High Performance Computing Linpack Benchmark.

The algorithm used by HPL can be summarized by the following keywords: Two-dimensional block-cyclic data distribution - Right-looking variant of the LU factorization with row partial pivoting featuring multiple look-ahead depths - Recursive panel factorization with pivot search and column broadcast combined - Various virtual panel broadcast topologies - bandwidth reducing swap-broadcast algorithm - backward substitution with look-ahead of depth 1.

The HPL package provides a testing and timing program to quantify the accuracy of the obtained solution as well as the time it took to compute it. The best performance achievable by this software on your system depends on a large variety of factors.

Nonetheless, with some restrictive assumptions on the interconnection network, the algorithm described here and its attached implementation are scalable in the sense that their parallel efficiency is maintained constant with respect to the per processor memory usage.

The HPL software package requires the availibility on your system of an implementation of the Message Passing Interface MPI (1.1 compliant). An implementation of either the Basic Linear Algebra Subprograms BLAS or the Vector Signal Image Processing Library VSIPL is also needed. Machine-specific as well as generic implementations of MPI, the BLAS and VSIPL are available for a large variety of systems.

Bonnie++ is a benchmark suite that is aimed at performing a number of simple tests of hard drive and file system performance.

Then you can decide which test is important and decide how to compare different systems after running it. I have no plans to ever have it produce a single number, because I dont think that a single number can be useful when comparing such things.

The main program tests database type access to a single file (or a set of files if you wish to test more than 1G of storage), and it tests creation, reading, and deleting of small files which can simulate the usage of programs such as Squid, INN, or Maildir format email.

A benchmarking tool for GNUstep, originated from NXBench.

GSBench is released under the GNU GPL. It is copyrighted by Philippe C.D. Robert.

Pipebench shows the current throughput and amount of data going through a pipe. It can be used to show the progress of a large md5sum process: cat bigfile | pipebench | md5sum.

Pipebench measures the speed of a pipe, by sitting in the middle passing the data along to the next process. Works on at least Linux, OpenBSD, NetBSD, Solaris and x86, Alpha, HPPA, Sparc and Sparc64.

Compiling

Just type make to compile.

Type make install to have pipebench be installed in /usr/local/bin

Blogbench is a portable filesystem benchmark that tries to reproduce the load of a real-world busy file server.

It stresses the filesystem with multiple threads performing random reads, writes and rewrites in order to get a realistic idea of the scalability and the concurrency a system can handle.

Blogbench was initially designed to mimic the behavior of the Skyblog.com
blog service.

4 different types of threads are started:

- The writers. They create new blogs (directories) with a random amount of
fake articles and fake pictures.

- The rewriters. They add or they modify articles and pictures of existing
blogs.

- The "commenters". They add fake comments to existing blogs in random order.

- The readers. They read articles, pictures and comments of random blogs. They
sometimes even try to access non-existent files.

New files are written atomically. The content is pushed with 8 Kb chunks in a
temporary file that gets renamed if everything completes. 8 Kb is the default
PHP buffer size for writes.

Reads are performed with a 64 Kb buffer.

Concurrent writers and rewriters can quickly create fragmentation if the
preallocation is not optimal. But it is very interesting to check how
different filesystems reacts to fragmentation.

Every blog is a new directory withing the same parent directory. Since some
filesystems are unable to manage more than 32k or 64k links to the same
directory (an example is UFS), you should not force the test to run a silly
amount of time on these filesystems.

The Bioinformatics Benchmark System is an attempt to build a reasonable testing framework, tests, and data, to enable end users and vendors to probe the performance of their systems.

What we are trying to do is to create a framework for testing, and a core set of tests that all may download and use to probe specific elements of systems performance.

Moreover, the source to these tests are available under GPL, and are hosted on Bioinformatics.org and Scalable Informatics LLC The idea is to enable end users, consumers, systems developers, and others to easily build and use meaningful tests for measurement and tuning reasons.

Joe Landman from Scalable Informatics LLC conceived the idea and wrote the original codes. We are looking for additional benchmark code suggestions, tests, data sets, etc.

Current baseline tests are several NCBI BLAST runs, several HMMer runs, and a variety of others. We plan to include ClustalW, X!Tandem, various chemistry, dynamics, and related tests, as well as several others.

Tests such as LINPACK or HPL simply do not provide meaningful performance indicators or predictive models for high performance informatics. Unfortunately, nor do a number of more recent and focused tests.

This is a problem as LINPACK and HPL specifically test the performance on various matrix operations, where you have effectively regular memory access patterns, and specific mathematical operations.

These codes are most useful for comparison to codes with heavy floating point operations, and interleaved memory traffic. These codes were not designed for comprehensive systems benchmarking, where disk I/O, memory latency, and other factors all contribute to the performance issues.

The best tests are the ones that are most similar to the codes you will run on the machine. The tests themselves should be reasonable approximations to a real execution of your code, using real data. You may need to pare it back in order to get realistic run times.

You should have a reasonable subset of data sizes. A single test does not tell you how your system scales, and one of the reasons for the existance of this test is specifically to allow you to test the performance while you increase various aspects of the workload.

You rarely get a quiescent system in a cluster, so we would recommend that you try to run in as realistic an operating environment as possible. A baseline in a quiescent system is fine, but it may set your expectations unreasonably.
top

Acovea implements a genetic algorithm for finding the "best" options for compiling programs with the GCC C and C++ compilers.
ACOVEA (Analysis of Compiler Options via Evolutionary Algorithm) implements a genetic algorithm to find the "best" options for compiling programs with the GNU Compiler Collection (GCC) C and C++ compilers.
"Best", in this context, is defined as those options that produce the fastest executable program from a given source code. Acovea is a C++ framework that can be extended to test other programming languages and non-GCC compilers.
I envision Acovea as an optimization tool, similar in purpose to profiling. Traditional function-level profiling identifies the algorithms most influential in a programs performance; Acovea is then applied to those algorithms to find the compiler flags and options that generate the fastest code.
Acovea is also useful for testing combinations of flags for pessimistic interactions, and for testing the reliability of the compiler.
Modern software is difficult to understand and verify by traditional means. Millions of lines of code produce applications containing intricate interactions, defying simple description or brute-force investigation.
A guided, deterministic approach to testing relies on human testers to envision every possible combination of actions -- an unrealistic proposition given software complexity. Yet, despite that complexity, we need answers to important questions about modern, large-scale software.
What sort of important questions? Consider the GNU Compiler Collection. I write articles that benchmark code generation, a task fraught with difficulties due to the myriad options provided by different compilers. For my benchmarks to have any meaning, I need to know which combination of options produces the fastest code for a given application.
Finding the "best" set of options sounds like a simple task, given the extent of GCC documentation and the conventional wisdom of the GCC developer community. Ah, if it were only so easy! The GCC documentation, while extensive, is also honestly imprecise.
I appreciate this style of documentation; unlike many commercial vendors, who make absolute statements about the "quality" of their products, GCCs documenters admit uncertainties in how various options alter code generation. Indeed, code generation is entirely dependent on the type of application being compiled and the target platform. An option that produces fast executable code for one source code may be detrimental to the performance of another program.
"Conventional wisdom" arrives in my inbox whenever I publish a new article. Ranging from the polite to the insistent to the rude, these e-mails contain contradictory suggestions for producing fast code.
In the vast majority of cases, such anecdotal assertions lack any formal proof of their validity, and, more often than not, the suggested "improvement" is ineffective or detrimental. It has become increasingly obvious that no one --myself included -- knows precisely how all these GCC options work together in generating program code.
I seek the Holy Grail of Optimization -- but exactly what is optimization? Understanding the problem is the first step in finding a solution.
Optimization attempts to produce the "best" machine code from source code. "Best" means different things to different applications; a database shovels chunks of information, while a scientific application is concerned with fast and accurate results; the first concern for an embedded system may be code size.
And it is quite possible that small code is fast, or fast code accurate. Optimization is far from being an exact science, given the diversity of hardware and software configurations.
An optimization algorithm may be as simple as removing a loop invariant, or as complex as examining an entire program to eliminate global common sub-expressions. Many optimizations change what the programmer wrote into a more efficient form, producing the same result while altering underlying details for efficiency; other "optimizations" produce code that uses specific characteristics of the underlying hardware, such as special instruction sets.
Memory architectures, pipelines, on- and off-chip caches -- all affect code performance in ways that are not obvious to programmers using a high-level language. An optimization that may seem to produce faster code may, in fact, create large code that causes more cache misses, thus degrading performance.
Even the best hand-tuned C code contains areas of interpretation; there is no absolute, one-to-one correspondence between C statements and machine instructions. Almost any sequence of source code can be compiled into different -- but functionally equivalent -- machine instruction streams with different sizes and performance characteristics.
Inlining functions is a classic example of this phenomena: replacing a call to a function with the function code itself may produce a faster program, but may also increase program size. Increased program size, may, in turn, prevent an algorithm from fitting inside high-speed cache memory, thus slowing a program due to cache misses.
Notice my use of the weasel word "may" -- inlining small functions sometimes allows other optimization algorithms a chance to further improve code for local conditions, producing faster and smaller code.
Optimization is not simple or obvious, and combinations of algorithms can lead to unexpected results. Which brings me back to the question: For any given application, what are the most effective optimization options?
Enhancements:
- Minor changes in the non-free license.
- Support has been added for the latest versions of libcoyotl and libevocosm.