Free round trip part 1 software for linux

The Wonder Shaper is a very special network shaper script with a lot of features. Works on Linux 2.4 & higher.

Goals

I attempted to create the holy grail:

* Maintain low latency for interfactive traffic at all times.

This means that downloading or uploading files should not disturb SSH or even telnet. These are the most important things, even 200ms latency is sluggish to work over.

* Allow surfing at reasonable speeds while up or downloading

Even though http is bulk traffic, other traffic should not drown it out too much.

* Make sure uploads dont harm downloads, and the other way around

This is a much observed phenomenon where upstream traffic simply destroys download speed. It turns out that all this is possible, at the cost of a tiny bit of bandwidth. The reason that uploads, downloads and ssh hurt eachother is the presence of large queues in many domestic access devices like cable or DSL modems.

Why it doesnt work well by default

ISPs know that they are benchmarked solely on how fast people can download. Besides available bandwidth, download speed is influenced heavily by packet loss, which seriously hampers TCP/IP performance. Large queues can help prevent packetloss, and speed up downloads. So ISPs configure large queues.

These large queues however damage interactivity. A keystroke must first travel the upstream queue, which may be seconds (!) long and go to your remote host. It is then displayed, which leads to a packet coming back, which must then traverse the downstream queue, located at your ISP, before it appears on your screen.

This HOWTO teaches you how to mangle and process the queue in many ways, but sadly, not all queues are accessible to us. The queue over at the ISP is completely off-limits, whereas the upstream queue probably lives inside your cable modem or DSL device. You may or may not be able to configure it. Most probably not.

So, what next? As we cant control either of those queues, they must be eliminated, and moved to your Linux router. Luckily this is possible.

Limit upload speed somewhat

By limiting our upload speed to slightly less than the truly available rate, no queues are built up in our modem. The queue is now moved to Linux.

Limit download speed

This is slightly trickier as we cant really influence how fast the internet ships us data. We can however drop packets that are coming in too fast, which causes TCP/IP to slow down to just the rate we want. Because we dont want to drop traffic unnecessarily, we configure a burst size we allow at higher speed.

Now, once we have done this, we have eliminated the downstream queue totally (except for short bursts), and gain the ability to manage the upstream queue with all the power Linux offers.

Let interactive traffic skip the queue

What remains to be done is to make sure interactive traffic jumps to the front of the upstream queue. To make sure that uploads dont hurt downloads, we also move ACK packets to the front of the queue. This is what normally causes the huge slowdown observed when generating bulk traffic both ways. The ACKnowledgements for downstream traffic must compete with upstream traffic, and get delayed in the process.

We also move other small packets to the front of the queue - this helps operating systems which do not set TOS bits, like everything from Microsoft.

Allow the user to specify low priority traffic (new in 1.1!)

Sometimes you may notice low priority OUTGOING traffic slowing down important traffic. In that case, the following options may help you:

NOPRIOHOSTSRC
Set this to hosts or netmasks in your network that should have low priority

NOPRIOHOSTDST
Set this to hosts or netmasks on the internet that should have low priority

NOPRIOPORTSRC
Set this to source ports that should have low priority. If you have an unimportant webserver on your traffic, set this to 80

NOPRIOPORTDST
Set this to destination ports that should have low priority.

See the start of wshaper and wshaper.htb

Results

If we do all this we get the following measurements using an excellent ADSL connection from xs4all in the Netherlands:

Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms

Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms

Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms

With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms

With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms

When uploading, downloads proceed at ~80% of the available speed. Uploads at around 90%. Latency then jumps to 850 ms, still figuring out why.

What you can expect from this script depends a lot on your actual uplink speed. When uploading at full speed, there will always be a single packet ahead of your keystroke. That is the lower limit to the latency you can achieve - divide your MTU by your upstream speed to calculate. Typical values will be somewhat higher than that. Lower your MTU for better effects!

A small table:

Uplink speed | Expected latency due to upload
--------------------------------------------------
32 | 234ms
64 | 117ms
128 | 58ms
256 | 29ms

So to calculate your effective latency, take a baseline measurement (ping on an unloaded link), and look up the number in the table, and add it. That is about the best you can expect. This number comes from a calculation that assumes that your upstream keystroke will have at most half a full sized packet ahead of it.

This boils down to:

mtu * 0.5 * 10
-------------- + baseline_latency
kbit

The factor 10 is not quite correct but works well in practice.

Your kernel

If you run a recent distribution, everything should be ok. You need 2.4 with QoS options turned on.

If you compile your own kernel, it must have some options enabled. Most notably, in the Networking Options menu, QoS and/or Fair Queueing, turn at least CBQ, PRIO, SFQ, Ingress, Traffic Policing, QoS support, Rate Estimator, QoS classifier, U32 classifier, fwmark classifier.

In practice, I (and most distributions) just turn on everything.

The scripts

The script comes in two versions, one which works on standard kernels and is implemented using CBQ. The other one uses the excellent HTB qdisc which is not in the default kernel. The CBQ version is more tested than the HTB one!

See wshaper and wshaper.htb.

Tuning

These scripts need to know the real rate of your ISP connection. This is hard to determine upfront as different ISPs use different kinds of bits it appears. People report success using the following technique:

Estimate both your upstream and downstream at half the rate your ISP specifies. Now verify if the script is functioning - check interactivity while uploading and while downloading. This should deliver the latency as calculated above. If not, check if the script executed without errors.

Now slowly increase the upstream & downstream numbers in the script until the latency comes back. This way you can find optimum values for your connection. If you are happy, please report to me so I can make a list of numbers that work well. Please let me know which ISP you use and the name of your subscription, and its reputed specifications, so I can list you here and save others the trouble.

Installation

If you dial in, you can copy the script to /etc/ppp/ip-up.d and it will be run at each connect.

If you want to remove the shaper from an interface, run wshaper stop. To see status information, run wshaper status.

KNOWN PROBLEMS

If you get errors, add an -x to the first line, as follows:

#!/bin/bash -x

And retry. This will show you which line gives an error. Before contacting me, make sure that you are running a recent version of iproute!

Recent versions can be found at your Linux distributor, or if you prefer compiling, here:
ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz

Command Line calculator "CLC" is the calculator that can calculate direct at the Linux console.

Extract the CLCC file to /usr/bin you might need to login as root or equvalent account, to be able to access it whenever you need it.

Then just write CLCC 67+89 and the answer will be presented to you directly.

CLCC calculates from left to right with no exceptions. CLC also have some extra parameters that is sent to CLC using backslash parameters.

CLCC r2 10/3 this will round the result to 2 decimals you can use 1 to 9 if you want to round the sum of.

the r parameter can also be combined with the other parameters but it must be added first like this

CLCC r3c 10/3 - to get 10 diveded with 3 rounded down to 3 decimals.

CLCC is tested with Ubuntu 6.10 and Puppy OS 2.13 but should work with any Linux version in console mode.

The program is developed using Hotbasic for Linux and is Freeware, hope you find it useful, send and email if you have any questions.

Hotbasic is a basic language that compiles into assembler direct, no interpeter that slows down the code, just pure assembler.

So if you want assembler in Linux but doesnt want to code in nasm, HotBasic is the compiler for you.

Trip on the Funny Boat is a side scrolling shooter game starring a steamboat on the sea.

Trip on the Funny Boat is side scrolling arcade shooter game on a steamboat equipped with a cannon and the ability to jump. The player will need to take advantage of waves to defeat the enemies and dodge hazards.

This game was made for the second PyWeek competition during the week from 25.3.2006 to 2.4.2006.

Label Templates are free Opendocument Format label templates for Openoffice.org, KOffice or any other Office suite ODF ready for making labels.

Collection includes CD, DVD, address, mailing, round, media label templates, Avery sizes and more. US Letter size and 14 size paper formats available.

Setup time is quick with these templates and printing labels from these templates is easy.

List::Part is a Perl module that allows you to partition one array into several.

SYNOPSIS

use List::Part;
($good, $bad)=part { !/substring/ } @array; #store arrayrefs into $good and $bad
(*good, *bad)=part { !/substring/ } @array; #store into @good and @bad

ABSTRACT

List::Part implements the part function, allowing one array to be "partitioned" into several based on the results of a code reference.

There are many applications in which the items of a list need to be categorized. For example, lets say you want to categorize lines in a log file:

my($success, $failure)=part { /^ERR/ } ;

Or, suppose you have a list of employees, and you need to determine their fate:

my($lay_off, $give_raise, $keep)=part {
$_->is_talented ? 0
: $_->is_executive ? 1
: 2
} @employees;

Actually, the second one is better suited to parts alternate form, parta:

my($lay_off, $give_raise, $keep)=parta
[ sub { $_->talented }, sub { $_->is_executive }, qr// ] =>
@employees;

Or maybe you just want yet another way to write the traditional Perl signoff:

perl -MList::Part -e"print map{@$_}part{$i++%5}split,JAercunrlkso ettPHr hea,"

List::Part can help you do those sorts of things.

GridMPI is a new open-source free-software implementation of the standard MPI (Message Passing Interface) library designed for the Grid. GridMPI project enables unmodified applications to run on cluster computers distributed across the Grid environment.
GridMPI team found that it is feasible to connect cluster computers and to run ordinary scientific applications in distance upto 500 miles. Simple experiment has shown that most MPI benchmarks scale fine upto 20 millisecond round-trip latency which corresponds to about 500 miles in distance, when the clusters are connected by fast 1 to 10 Gbps networks. 500 miles covers the major cities between Tokyo--Osaka in Japan.
Thus, applications which are too large to run on a local cluster should run on multiple clusters in the Grid environment with acceptable performance. However, it is only feasible when using an efficient MPI implementation [1]. Existing implementations are not efficient enough mainly because of the two reasons: their focus on security features and TCP performance problems.
GridMPI skips security layers assuming dedicated secure links. The institutes housing large clusters tend to have their own networks to connect to other institutes in most cases. GridMPI so focuses on the performance on TCP. Since existing implementations are in most cases designed for MPP machines and recently clusters with special hardware, their performance on TCP with Ethernet is not optimal.
Also TCP performance itself is not optimal for the work load of the MPI traffic. In addition, support for heterogeneous combinations of computers of the existing MPI implementations is not satisfactory. Thus, GridMPI is designed and implemented from the scratch. GridMPI is carefully coded and tested with heterogeneity in mind.
Main features:
- Full conformance to the standard: GridMPI passes 100% of the functional tests of the large test suites from ANL and Intel (MPI-1.2 level).
- Full heterogeneity support: GridMPI is fully tested with combinations of processors of 32bit/64bit and big/little-endian.
- Primary support of TCP/IP and sockets: GridMPI is written from scratch and it is new and clean. It is efficient with sockets, and thus suitable for the Grid as well as ordinary Ethernet-based clusters.
- Cooperation with Grid job submission: GridMPI can be used with Globus, Unicore, tool from NAREGI project, etc.
- Checkpointing support: GridMPI supports checkpointing on Linux/IA32 platforms to restart long-running applications from failure.
- Vendor MPI support: GridMPI supports IBM-MPI, Fujitsu-Solaris-MPI, Intel-MPI, and any MPICH-based MPI for clusters with special communication hardware.
Enhancements:
- Minor bugfixes were made.

Livestat project is a statistics management system for academic competition tournaments.
Livestat is a Perl-based statistics management system designed for academic competition tournaments, a.k.a "College Bowl" or "quiz bowl" events.
Livestat is used to automate the processing of statistics from individual matches, and to quickly publish statistics on teams and players to the Web.
Enhancements:
- Now sorts teams by name when printing list in game.pl.
- New files common.head and common.foot store the top and bottom of all HTML pages to be generated.
- No longer prints "N/A: " in round views when no player has a positive score.
- Restructured head2head hash in standings.pl; now has fields for wins, losses, ties, points-for, and points-against.
- Added style number in tables -- useful for right-aligning numeric fields

DataGridField is a product which consists of a table input component for Plone.
It uses Javascript to make entering tabular data more user friendly process - there are no round trip HTTP requests to the server when inserting or deleting rows.
Main features:
- Any number of columns set by a developer
- Any number of rows filled by a user
- Insert and deleting rows without submitting a form
- Many different column types