Free qos software for linux

QoS Connection Tuning HOWTO is a document which explains how to tune network connection performance.

This enables you to get the maximum benefit out of your connection without lag and loss.

The Wonder Shaper is a very special network shaper script with a lot of features. Works on Linux 2.4 & higher.

Goals

I attempted to create the holy grail:

* Maintain low latency for interfactive traffic at all times.

This means that downloading or uploading files should not disturb SSH or even telnet. These are the most important things, even 200ms latency is sluggish to work over.

* Allow surfing at reasonable speeds while up or downloading

Even though http is bulk traffic, other traffic should not drown it out too much.

* Make sure uploads dont harm downloads, and the other way around

This is a much observed phenomenon where upstream traffic simply destroys download speed. It turns out that all this is possible, at the cost of a tiny bit of bandwidth. The reason that uploads, downloads and ssh hurt eachother is the presence of large queues in many domestic access devices like cable or DSL modems.

Why it doesnt work well by default

ISPs know that they are benchmarked solely on how fast people can download. Besides available bandwidth, download speed is influenced heavily by packet loss, which seriously hampers TCP/IP performance. Large queues can help prevent packetloss, and speed up downloads. So ISPs configure large queues.

These large queues however damage interactivity. A keystroke must first travel the upstream queue, which may be seconds (!) long and go to your remote host. It is then displayed, which leads to a packet coming back, which must then traverse the downstream queue, located at your ISP, before it appears on your screen.

This HOWTO teaches you how to mangle and process the queue in many ways, but sadly, not all queues are accessible to us. The queue over at the ISP is completely off-limits, whereas the upstream queue probably lives inside your cable modem or DSL device. You may or may not be able to configure it. Most probably not.

So, what next? As we cant control either of those queues, they must be eliminated, and moved to your Linux router. Luckily this is possible.

Limit upload speed somewhat

By limiting our upload speed to slightly less than the truly available rate, no queues are built up in our modem. The queue is now moved to Linux.

Limit download speed

This is slightly trickier as we cant really influence how fast the internet ships us data. We can however drop packets that are coming in too fast, which causes TCP/IP to slow down to just the rate we want. Because we dont want to drop traffic unnecessarily, we configure a burst size we allow at higher speed.

Now, once we have done this, we have eliminated the downstream queue totally (except for short bursts), and gain the ability to manage the upstream queue with all the power Linux offers.

Let interactive traffic skip the queue

What remains to be done is to make sure interactive traffic jumps to the front of the upstream queue. To make sure that uploads dont hurt downloads, we also move ACK packets to the front of the queue. This is what normally causes the huge slowdown observed when generating bulk traffic both ways. The ACKnowledgements for downstream traffic must compete with upstream traffic, and get delayed in the process.

We also move other small packets to the front of the queue - this helps operating systems which do not set TOS bits, like everything from Microsoft.

Allow the user to specify low priority traffic (new in 1.1!)

Sometimes you may notice low priority OUTGOING traffic slowing down important traffic. In that case, the following options may help you:

NOPRIOHOSTSRC
Set this to hosts or netmasks in your network that should have low priority

NOPRIOHOSTDST
Set this to hosts or netmasks on the internet that should have low priority

NOPRIOPORTSRC
Set this to source ports that should have low priority. If you have an unimportant webserver on your traffic, set this to 80

NOPRIOPORTDST
Set this to destination ports that should have low priority.

See the start of wshaper and wshaper.htb

Results

If we do all this we get the following measurements using an excellent ADSL connection from xs4all in the Netherlands:

Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms

Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms

Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms

With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms

With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms

When uploading, downloads proceed at ~80% of the available speed. Uploads at around 90%. Latency then jumps to 850 ms, still figuring out why.

What you can expect from this script depends a lot on your actual uplink speed. When uploading at full speed, there will always be a single packet ahead of your keystroke. That is the lower limit to the latency you can achieve - divide your MTU by your upstream speed to calculate. Typical values will be somewhat higher than that. Lower your MTU for better effects!

A small table:

Uplink speed | Expected latency due to upload
--------------------------------------------------
32 | 234ms
64 | 117ms
128 | 58ms
256 | 29ms

So to calculate your effective latency, take a baseline measurement (ping on an unloaded link), and look up the number in the table, and add it. That is about the best you can expect. This number comes from a calculation that assumes that your upstream keystroke will have at most half a full sized packet ahead of it.

This boils down to:

mtu * 0.5 * 10
-------------- + baseline_latency
kbit

The factor 10 is not quite correct but works well in practice.

Your kernel

If you run a recent distribution, everything should be ok. You need 2.4 with QoS options turned on.

If you compile your own kernel, it must have some options enabled. Most notably, in the Networking Options menu, QoS and/or Fair Queueing, turn at least CBQ, PRIO, SFQ, Ingress, Traffic Policing, QoS support, Rate Estimator, QoS classifier, U32 classifier, fwmark classifier.

In practice, I (and most distributions) just turn on everything.

The scripts

The script comes in two versions, one which works on standard kernels and is implemented using CBQ. The other one uses the excellent HTB qdisc which is not in the default kernel. The CBQ version is more tested than the HTB one!

See wshaper and wshaper.htb.

Tuning

These scripts need to know the real rate of your ISP connection. This is hard to determine upfront as different ISPs use different kinds of bits it appears. People report success using the following technique:

Estimate both your upstream and downstream at half the rate your ISP specifies. Now verify if the script is functioning - check interactivity while uploading and while downloading. This should deliver the latency as calculated above. If not, check if the script executed without errors.

Now slowly increase the upstream & downstream numbers in the script until the latency comes back. This way you can find optimum values for your connection. If you are happy, please report to me so I can make a list of numbers that work well. Please let me know which ISP you use and the name of your subscription, and its reputed specifications, so I can list you here and save others the trouble.

Installation

If you dial in, you can copy the script to /etc/ppp/ip-up.d and it will be run at each connect.

If you want to remove the shaper from an interface, run wshaper stop. To see status information, run wshaper status.

KNOWN PROBLEMS

If you get errors, add an -x to the first line, as follows:

#!/bin/bash -x

And retry. This will show you which line gives an error. Before contacting me, make sure that you are running a recent version of iproute!

Recent versions can be found at your Linux distributor, or if you prefer compiling, here:
ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz

FireHOL is a stateful iptables packet filtering firewall configurator. It is abstracted, extensible, easy and powerful. It can handle any kind of firewall, but most importantly, it gives you the ways to configure it, the same way you think of it.
Main features:
- FireHOL handles firewalls protecting one host on all its interfaces and any combination of stateful firewalls routing traffic from one interface to another. There are no limitations on the number of interfaces or on the number of routing routes (except the ones iptables has, if any).
- FireHOL, still lacks a few features: QoS for example is not supported directly. You are welcome to extend FireHOL and send me your patches to integrate within FireHOL. In any case however, you can embed normal iptables commands in a FireHOL configuration to do whatever iptables supports.
- Since FireHOL produces stateful commands, for every supported service it needs to know the flow of requests and replies. Today FireHOL supports the following services:
- Many single socket protocols, such as HTTP, NNTP, SMTP, POP3, IMAP4, RADIUS, SSH, LDAP, MySQL, Telnet, NTP, DNS, etc. There are a few dozens of such services defined in FireHOL. Check this list. Even if something is missing, you can define it.
- Many complex protocols, such as FTP, NFS, SAMBA, PPTP, etc. If you need some complex protocol that is not present, you will have to program it (in simple BASH scripting - there are many commented examples on how this is done). Again, you will just create one BASH function with the rules of the protocol, and FireHOL will turn it to a client, a server or a router.
Enhancements:
- Minor updates were made for the latest IANA reservations.
- A check-iana.sh cron job script was provided to notify the administrator when IANA reservations change.

CBQ.init is a shell script that allows for easy setup of simple CBQ-based traffic control on Linux. CBQ (Class Based Queueing) is part of the Linux QoS implementation, which is accesible via the iproute2 package.
Enhancements:
- v0.7.3- Deepak Singhal
- fix timecheck to not ignore regular TIME rules after encountering a TIME rule that spans over midnight
- Nathan Shafer
- allow symlinks to class files
- Seth J. Blank
- replace hardcoded ip/tc location with variables
- Mark Davis
- allow setting of PRIO_{MARK,RULE,REALM} in class file
- Fernando Sanch
- allow underscores in interface names

Linphone is a web phone that it let you phone to your friends anywhere in the whole world, freely, simply by using the internet. The cost of the phone call is the cost that you spend connected to the internet.
To call somebody, you must provide to linphone a SIP URL : It is something like toto@machine.com, where toto is a linux user that runs linphone, and machine.com is the name of a host on a network. If you dont know the machines name you can specify simply an IP address in dot notation (as 192.0.0.1)
Linphone is mostly sip compliant. It works successfully with these implementations:
- eStara softphone (commercial software for windows)
- Pingtel phones (with DNS enabled and VLAN QOS support disabled).
- Hotsip, a free of charge phone for Windows.
- Vocal, an open source SIP stack from Vovida that includes a SIP proxy that works with linphone since version 0.7.1.
- Siproxd is a free sip proxy being developped by Thomas Ries because he would like to have linphone working behind his firewall. Siproxd is simple to setup and works perfectly with linphone.
- Partysip aims at being a generic and fully functionnal SIP proxy. Visit the web page for more details on its functionalities.
Linphone may work also with other sip phones, but this has not been tested yet.
Linphone uses the SIP protocol to establish calls, for that reason it cannot work with H323 phones, because SIP and H323 are different and opposite protocols. H323 phones are Netmeeting (for windows), Gnome-meeting (Unix), OpenPhone...
Main features:
- Works with the Gnome Desktop under linux, (maybe some others Unixes, but this has never been tested). Nevertheless you can use linphone under KDE, of course !
- Since version 0.9.0, linphone can be compiled and used without gnome, in console mode, by using the program called "linphonec"
- Works as simply as a cellular phone. Two buttons, no more.
- Linphones includes a large variety of codecs (G711-ulaw, G711-alaw, LPC10-15, GSM, and SPEEX). Thanks to the Speex codec it is able to provide high quality talks even with slow internet connections, like 28k modems.
- Understands the SIP protocol. SIP is a standardised protocol from the IETF (http://www.ietf.org), that is the organisation that made most of the protocols used in the internet. This guaranties compatibility with most SIP - compatible web phones.
- You just require a soundcard to use linphone.
- Other technical functionnalities include DTMF (dial tones) support though RFC2833 and ENUM support (to use SIP numbers instead of SIP addresses).
- Linphone is free software, released under the General Public Licence.
- Linphone is documented: there is a complete user manual readable from the application that explains you all you need to know.
- Linphone includes a sip test server called "sipomatic" that automatically answers to calls by playing a pre-recorded message.
Enhancements:
- This version fixes a compilation error, an incorrect icon path and updates the cz translation.

p2pshaper is a Linux QoS script to ensure good latency and fairness on a slow network connection overloaded by p2p applications. It also includes generic optimizations to improve fairness. This release includes a large number of improvements since the 1.3 test release.
In short p2pshaper is a set of scripts for the Linux OS, that is used to administer bandwidth on a network overloaded by massive p2p traffic.
Normally, the amount of peer-to-peer traffic on such networks, will make it impossible to surf, not to mention doing interactive traffic (ssh). The normal action is to shut down p2p traffic totally, and set up the net, so that Internet connection is only possible through a proxy. This breaks many useful programs. And not all p2p traffic is evil. Normally, one would mark packets coming to a specific port (emule ports) to be of a lower priority than packets to other ports. This used to be a good solution. But now, there exist a huge number of p2p clients, eact with different port ranges, and also, some programs are able to change the ports. Normally the Internet newsgroup port should have a high priority. But binary newsgroups are sometimes used for massive downloads, which will use up all bandwidth. Better packet markers are needed.
p2pshaper currently exists in two versions p2pshaper v1 and p2pshaper v2. They both works in different ways, so v2 is not "better" than v1. Which one you use depends on what kind of functonality you want.
Enhancements:
- Added unified patches against recent kernels With the amount of patches p2pshaper needs, people have had problems patching kernels. To make this easier, p2pshaper now includes patches for most recent kernels. This will probably ease the installation of
- p2pshaper. Old patches have been moved to the old/ directory.
- Normal connections will never get into the lowest priority bandIntroduced already in the 1.4.1 release. This makes sure that normal traffic isnt nuked all the way into oblivion, where only evil traffic (marked by l7_filter, ipp2p or band8ports) should be.

PhishBouncer project is an advanced Java HTTP(S) proxy with anti-phishing capabilities.
PhishBouncer is an anti-phishing platform based on an HTTP/HTTPS proxy integrating anti-phishing checks that do not depend on block lists or Phish signatures. The checking algorithms make use of the attributes of the web-site being visited, the structure and properties of the referring URL, and the web-sites association with other legitimate web-sites that the user interacts with. The checks are implemented as plug-in interceptors, and it is easy to modify them and add or remove new checks. Apart from defense against Phishing, PhishBouncer is also a platform for developing and testing new anti-Phishing checks.
For ease of rapid prototyping and testing of anti-Phishing checks with real and reliable test data, a crawl-and-drive framework is also provided-- all you need is an APWG membership to be able to download Phish Reports from APWG and follow the instructions provided. This framework will periodically download new Phish URLs from APWG, and visit the Phish sites using the PhishBouncer proxy first without and then with the anti-Phishing checks. All results are logged so that dead or broken sites (i.e., sites that produced errors in either visit) can be culled, and the remaining data can be used to obtain an accurate count of how many Phish sites were flagged by the currently active checks.
The HTTP/HTTPS proxy framework can also be used to insert other types of adaptive behavior in the HTTP/HTTPS based interaction by replacing the plug-in interceptors executing anti-phishing checks by other interceptors that performs logging, filtering (as in parental control), load-balancing, QoS-based redirection etc.
PhishBouncer was developed by BBN under an R&D project supported by the Homeland Security Advanced Research Project Agency (HSARPA), under its Cyber Security R&D program.
Main features:
- Implemented in Java, therefore less vulnerable to traditional exploits (e.g., buffer overflow attacks)
- Architectural solution with stronger guarantees than browser plug-ins (can catch phishing attacks even if the browser is closed or not part of the communication)
- Browser independent - supports all web browsers
- Operating system independent - supports all operating systems that can run Java
- Highly customizable deployment options - runs on user hosts, wireless routers, or network server
- Open framework and plug-in architecture - allows easy addition of new checks
- Attribute-based detection - provides protection against unknown phishing attacks
- Supports reactive and proactive anti-phishing checks
- Supports HTTP and HTTPS