Free benchmark lending software for linux

Benchmark::Forking is a Perl module to run benchmarks in separate processes.

SYNOPSIS

use Benchmark::Forking qw( timethis timethese cmpthese );

timethis ($count, "code");

timethese($count, {
Name1 => sub { ...code1... },
Name2 => sub { ...code2... },
});

cmpthese($count, {
Name1 => sub { ...code1... },
Name2 => sub { ...code2... },
});

Benchmark::Forking->enabled(0); # Stop using forking feature
...
Benchmark::Forking->enabled(1); # Begin using forking again

The Benchmark::Forking module changes the behavior of the standard Benchmark module, running each piece of code to be timed in a separate forked process. Because each child exits after running its timing loop, the computations it performs cant propogate back to affect subsequent test cases.

This can make benchmark comparisons more accurate, because the separate test cases are mostly isolated from side-effects caused by the others. Benchmark scripts typically dont depend on those side-effects, so in most cases you can simply use or require this module at the top of your existing code without having to change anything else. (A few key exceptions are noted in "BUGS".)

CompBenchmarks project is a benchmarking environment for compilers:
- It provides a package for downloading and easing usage of some well-known C/C++ benchmarks,
- The package allows you to specify compilation options and compilers to use, giving results in a common format,
- This web-site provides a convenient browsing formular to analyse imported benchmarks.
For now, Ive concentrated my efforts on benchmarking of GCC and espacially on its embedded C and C++ compilers on the Linux/x86 platform, yet support for others languages, compilers or platforms can be added (Cygwin is supported).
Enhancements:
- 25 more benchmarks are supported (C++ language).
- There is an improved build mechanism, and a useless (nonexistent) script call has been removed.
- The Web site has reached 10,000 published results.

Cache::Benchmark is a Perl module that tests the quality and speed of a cache module to compare cachemodules and algorithms.

SYNOPSIS

use Cache::Benchmark();
use Cache::MemoryCache();
use Cache::SizeAwareMemoryCache();

my $cache_1 = new Cache::MemoryCache({
namespace => my,
default_expires_in => 1,
});
my $cache_2 = new Cache::SizeAwareMemoryCache({
namespace => my,
default_expires_in => 1,
max_size => 400,
});

my $test = new Cache::Benchmark();
$test->init( access_counter => 10_000 );

$test->run($cache_1);
print $test->get_printable_result();

$test->run($cache_2);
print $test->get_printable_result();

Data::BenchmarkResults is a Perl extension for averaging and comparing multiple benchmark runs.

SYNOPSIS

use Data::BenchmarkResults;
$conditionA_results = new Data::BenchmarkResults;
$conditionB_results = new Data::BenchmarkResults;

#Load test result runs for the first condition
$conditionA_results->add_result_set("test1","run1",@data1);
$conditionA_results->add_result_set("test2","run1",@data2);
$conditionA_results->add_result_set("test1","run2",@data3);
$conditionA_results->add_result_set("test2","run2",@data4);

#Load test result runs for the second condition
$conditionB_results->add_result_set("test1","run1",@data5);
$conditionB_results->add_result_set("test2","run2",@data6);
$conditionB_results->add_result_set("test1","run1",@data7);
$conditionB_results->add_result_set("test2","run2",@data8);

#Average (mean average) the results of all the the runs of test1
# w/o tossing the highest and lowest values (replace the 0 with 1to
# toss the highest and lowest values

my $computed = $conditionA_results->process_result_set("test1","mean",0);
my $computed2 = $conditionB_results->process_result_set("test1","mean",0);

#OR process all of the tests at once (tossing the highest and lowest value) :

$conditionA_results->process_all_result_sets("mean",1);
$conditionB_results->process_all_result_sets("mean",1);


#Print out all of the processed test results
print "Condition A results.... nn"
$conditionA_results->print_calculated_sets;
print "Condition B results.... nn"
$conditionB_results->print_calculated_sets;


#Compare results of test1 of condition B against those with condition A
# as a percentage change from A to B

my $compared = $conditionB_results->compare_result_set($conditionA_results,"test1");

#OR compare all the processed test results from one condition to those of another
my $total_comparison = $conditionB_results->compare_all_result_sets($conditionA_results);

The Bioinformatics Benchmark System is an attempt to build a reasonable testing framework, tests, and data, to enable end users and vendors to probe the performance of their systems.

What we are trying to do is to create a framework for testing, and a core set of tests that all may download and use to probe specific elements of systems performance.

Moreover, the source to these tests are available under GPL, and are hosted on Bioinformatics.org and Scalable Informatics LLC The idea is to enable end users, consumers, systems developers, and others to easily build and use meaningful tests for measurement and tuning reasons.

Joe Landman from Scalable Informatics LLC conceived the idea and wrote the original codes. We are looking for additional benchmark code suggestions, tests, data sets, etc.

Current baseline tests are several NCBI BLAST runs, several HMMer runs, and a variety of others. We plan to include ClustalW, X!Tandem, various chemistry, dynamics, and related tests, as well as several others.

Tests such as LINPACK or HPL simply do not provide meaningful performance indicators or predictive models for high performance informatics. Unfortunately, nor do a number of more recent and focused tests.

This is a problem as LINPACK and HPL specifically test the performance on various matrix operations, where you have effectively regular memory access patterns, and specific mathematical operations.

These codes are most useful for comparison to codes with heavy floating point operations, and interleaved memory traffic. These codes were not designed for comprehensive systems benchmarking, where disk I/O, memory latency, and other factors all contribute to the performance issues.

The best tests are the ones that are most similar to the codes you will run on the machine. The tests themselves should be reasonable approximations to a real execution of your code, using real data. You may need to pare it back in order to get realistic run times.

You should have a reasonable subset of data sizes. A single test does not tell you how your system scales, and one of the reasons for the existance of this test is specifically to allow you to test the performance while you increase various aspects of the workload.

You rarely get a quiescent system in a cluster, so we would recommend that you try to run in as realistic an operating environment as possible. A baseline in a quiescent system is fine, but it may set your expectations unreasonably.
top

The benchmark program takes less than 10 minutes to run (on most machines) and compares the system it is run on to two benchmark systems (a Dell Pentium 90 with 256 KB cache running MSDOS and an AMD K6/233 with 512 KB cache running Linux).

The archive contains the complete source, documentation, and a binary (Linux elf). The source has been successfully compiled on various operating systems, including SunOS, DEC Unix 4.0, DEC OSF1, HP-UX, DEC Ultrix, MS-DOS, and of course Linux.

This release is based on the Unix port of beta release 2 of BYTE Magazines BYTEmark benchmark program (previously known as BYTEs Native Mode Benchmarks). The port to Linux/Unix was done by Uwe F. Mayer.

Additional changes to the code were made to make the code work with egcs compiler and to make the software packagable. This is a CPU benchmark providing indexes for integer, floating, and memory performance. It is single-threaded and is not designed to measure the performance gain on multi-processor machines.

Running a "make" will create the binary if all goes well. It is called "nbench" and performs a suite of 10 tests and compares the results to a Dell Pentium 90 with 16 MB RAM and 256 KB L2 cache running MSDOS and compiling with the Watcom 10.0 C/C++ compiler.

If you define -DLINUX during compilation (the default) then you also get a comparison to an AMD K6/233 with 32 MB RAM and 512 KB L2-cache running Linux 2.0.32 and using a binary which was compiled with GNU gcc version 2.7.2.3 and GNU libc-5.4.38.

The algorithms were not changed from the source which was obtained from the BYTE web site at http://www.byte.com/bmark/bmark.htm on December 14, 1996. However, the source was modified to better work with 64-bit machines (in particular the random number generator was modified to always work with 32 bit, no matter what kind of hardware you run it on).

Furthermore, for some of the algorithms additional resettings of the data was added to increase the consistency across different hardware. Some extra debugging code was added, which has no impact on normal runs.

In case there is uneven system load due to other processes while this benchmark suite executes, it might take longer to run than on an unloaded system.

This is because the benchmark does some statistical analysis to make sure that the reported results are statistically significant, and an increased variation in individual runs requires more runs to achieve the required statistical confidence.

This is a single-threaded benchmark and is not designed to measure the performance gain on multi-processor machines.

virtbench is a set of benchmarks for virtualization environments / hypervisors. The project is designed to be easy to run and easy to extend. The main audience will be people developing hypervisors.