Free tos software for linux

TOS is an experimental operating system kernel which is written in our strictly and statically typed assembly language, TALK.
Today, computers (PCs, cell-phones, etc.) are widely used in the world and their network become one of the indispensable social infrastructures.
Therefore, the importance of ensuring safety of software is commonly-recognized and many programs come to be written in strongly-typed languages (Java, C#, Objective Caml). This is because the program that is written in a strongly-typed language and passes its type-check is ensured not to raise errors at runtime.
However, there is one kind of programs that are never written in typed-languages: Operating systems. For example, existing OSes (e.g., Linux, FreeBSD, Windows XP, Solaris, etc.) are written in C and assembly languages.
One of the reason that OSes were not written in typed-languages is that it is believed that OS facilities, such as memory management, multi-thread management and device drivers, cannot be written in the typed-languages.
To break the mistaken belief, we have developed a statically and strictly typed assembly language, called TALK, and implemented a prototype OS kernel, called TOS, in TALK.
So far, the functionality of TOS is very poor (no user programs, no file systems ...), but we plan to extend TOS to have the same facilities as existing OS kernels.
Enhancements:
- Memory management code was rewritten.

NRL has implemented a link-state routing protocol oriented for mobile ad hoc networks (MANETs). NRL OLSR is largely based on the Optimized Link State Routing (OLSR) protocol specification (RFC 3626).
Main features:
- Support for IPv6
- Operational in Windows, MacOS, Linux, and various embedded PDA systems such as Zaurus and PocketPC.
- Full link state topology can be distributed including non-MPR cross links
- A "willingness" attribute for localized MPR activation
- Support for several MPR selection protocols (Classical flooding, NS-MPR, S-MPR, MPR-CDS, and E-CDS)
- Neighbor link quality assessed by a smoothed hysteresis function.
- Many run-time parameters available including: HELLO interval, link state update interval, timeout factors, link quality assessment parameters, MPR willingness, and message TOS
- Configureable debugging verboseness
- Experimental features such as fuzzy-sighted routing and support for Simplified Multicast Forwarding
Enhancements:
- NS-2 support and various bugfixes.

The Wonder Shaper is a very special network shaper script with a lot of features. Works on Linux 2.4 & higher.

Goals

I attempted to create the holy grail:

* Maintain low latency for interfactive traffic at all times.

This means that downloading or uploading files should not disturb SSH or even telnet. These are the most important things, even 200ms latency is sluggish to work over.

* Allow surfing at reasonable speeds while up or downloading

Even though http is bulk traffic, other traffic should not drown it out too much.

* Make sure uploads dont harm downloads, and the other way around

This is a much observed phenomenon where upstream traffic simply destroys download speed. It turns out that all this is possible, at the cost of a tiny bit of bandwidth. The reason that uploads, downloads and ssh hurt eachother is the presence of large queues in many domestic access devices like cable or DSL modems.

Why it doesnt work well by default

ISPs know that they are benchmarked solely on how fast people can download. Besides available bandwidth, download speed is influenced heavily by packet loss, which seriously hampers TCP/IP performance. Large queues can help prevent packetloss, and speed up downloads. So ISPs configure large queues.

These large queues however damage interactivity. A keystroke must first travel the upstream queue, which may be seconds (!) long and go to your remote host. It is then displayed, which leads to a packet coming back, which must then traverse the downstream queue, located at your ISP, before it appears on your screen.

This HOWTO teaches you how to mangle and process the queue in many ways, but sadly, not all queues are accessible to us. The queue over at the ISP is completely off-limits, whereas the upstream queue probably lives inside your cable modem or DSL device. You may or may not be able to configure it. Most probably not.

So, what next? As we cant control either of those queues, they must be eliminated, and moved to your Linux router. Luckily this is possible.

Limit upload speed somewhat

By limiting our upload speed to slightly less than the truly available rate, no queues are built up in our modem. The queue is now moved to Linux.

Limit download speed

This is slightly trickier as we cant really influence how fast the internet ships us data. We can however drop packets that are coming in too fast, which causes TCP/IP to slow down to just the rate we want. Because we dont want to drop traffic unnecessarily, we configure a burst size we allow at higher speed.

Now, once we have done this, we have eliminated the downstream queue totally (except for short bursts), and gain the ability to manage the upstream queue with all the power Linux offers.

Let interactive traffic skip the queue

What remains to be done is to make sure interactive traffic jumps to the front of the upstream queue. To make sure that uploads dont hurt downloads, we also move ACK packets to the front of the queue. This is what normally causes the huge slowdown observed when generating bulk traffic both ways. The ACKnowledgements for downstream traffic must compete with upstream traffic, and get delayed in the process.

We also move other small packets to the front of the queue - this helps operating systems which do not set TOS bits, like everything from Microsoft.

Allow the user to specify low priority traffic (new in 1.1!)

Sometimes you may notice low priority OUTGOING traffic slowing down important traffic. In that case, the following options may help you:

NOPRIOHOSTSRC
Set this to hosts or netmasks in your network that should have low priority

NOPRIOHOSTDST
Set this to hosts or netmasks on the internet that should have low priority

NOPRIOPORTSRC
Set this to source ports that should have low priority. If you have an unimportant webserver on your traffic, set this to 80

NOPRIOPORTDST
Set this to destination ports that should have low priority.

See the start of wshaper and wshaper.htb

Results

If we do all this we get the following measurements using an excellent ADSL connection from xs4all in the Netherlands:

Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms

Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms

Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms

With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms

With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms

When uploading, downloads proceed at ~80% of the available speed. Uploads at around 90%. Latency then jumps to 850 ms, still figuring out why.

What you can expect from this script depends a lot on your actual uplink speed. When uploading at full speed, there will always be a single packet ahead of your keystroke. That is the lower limit to the latency you can achieve - divide your MTU by your upstream speed to calculate. Typical values will be somewhat higher than that. Lower your MTU for better effects!

A small table:

Uplink speed | Expected latency due to upload
--------------------------------------------------
32 | 234ms
64 | 117ms
128 | 58ms
256 | 29ms

So to calculate your effective latency, take a baseline measurement (ping on an unloaded link), and look up the number in the table, and add it. That is about the best you can expect. This number comes from a calculation that assumes that your upstream keystroke will have at most half a full sized packet ahead of it.

This boils down to:

mtu * 0.5 * 10
-------------- + baseline_latency
kbit

The factor 10 is not quite correct but works well in practice.

Your kernel

If you run a recent distribution, everything should be ok. You need 2.4 with QoS options turned on.

If you compile your own kernel, it must have some options enabled. Most notably, in the Networking Options menu, QoS and/or Fair Queueing, turn at least CBQ, PRIO, SFQ, Ingress, Traffic Policing, QoS support, Rate Estimator, QoS classifier, U32 classifier, fwmark classifier.

In practice, I (and most distributions) just turn on everything.

The scripts

The script comes in two versions, one which works on standard kernels and is implemented using CBQ. The other one uses the excellent HTB qdisc which is not in the default kernel. The CBQ version is more tested than the HTB one!

See wshaper and wshaper.htb.

Tuning

These scripts need to know the real rate of your ISP connection. This is hard to determine upfront as different ISPs use different kinds of bits it appears. People report success using the following technique:

Estimate both your upstream and downstream at half the rate your ISP specifies. Now verify if the script is functioning - check interactivity while uploading and while downloading. This should deliver the latency as calculated above. If not, check if the script executed without errors.

Now slowly increase the upstream & downstream numbers in the script until the latency comes back. This way you can find optimum values for your connection. If you are happy, please report to me so I can make a list of numbers that work well. Please let me know which ISP you use and the name of your subscription, and its reputed specifications, so I can list you here and save others the trouble.

Installation

If you dial in, you can copy the script to /etc/ppp/ip-up.d and it will be run at each connect.

If you want to remove the shaper from an interface, run wshaper stop. To see status information, run wshaper status.

KNOWN PROBLEMS

If you get errors, add an -x to the first line, as follows:

#!/bin/bash -x

And retry. This will show you which line gives an error. Before contacting me, make sure that you are running a recent version of iproute!

Recent versions can be found at your Linux distributor, or if you prefer compiling, here:
ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz

pmacct is a small set of passive network monitoring tools to measure, account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, AS numbers, ports, IP protocol and ToS/DSCP field are supported) which may be arbitrarily combined to build custom aggregation methods; support for historical data breakdown, triggers and packet tagging, filtering and sampling.

Aggregates can be stored into memory tables, SQL databases (MySQL or PostgreSQL) or simply printed to stdout. Data is collected from the network either using libpcap (and optionally promiscuous mode) or reading NetFlow v1/v5/v7/v8/v9 and sFlow v2/v4/v5 datagrams, both unicast and multicast.

IP accounting is the key in ISP/IXP network operations like billing, graphing network resources usage, live or historical traffic analysis, handling of network thresholds, provisioning and SLA monitoring. SNMP counters sometime are not this useful because of their coarse granularity.

Finer granularities become valuable if data are historical and match logical entities of interest such as Autonomous System Numbers, departmental or customer network chunks, specific traffic flows, etc. Further, actual large-scale networks are able to produce, in very short times, high amounts of data that become quickly difficult to be processed in a meaningful way

In this context, traffic aggregation and advanced filtering and sampling capabilities are requirements that cannot be missed anymore.

Either using memory or SQL tables as backend storage, pmacct can easily feed data to tools like MRTG, RRDtool, Cacti and Gnuplot among the others. A little scripting abilities are required to glue pmacct with external tools and a number of sample scripts and some tutorials are already available.

ActivePython project is the complete, quality-assured and ready-to-install Python distribution for AIX, HP-UX, Linux, Mac OS X, Solaris, and Windows.
Whats included:
- Python core
- Core extensions zlib and bzip2 for data compression
- bsddb database library
- Tix GUI widgets for Tkinter
- Suite of Windows tools developed by Mark Hammond, including the PythonWin IDE, bindings to the Win32 API, and Windows COM
Complete documentation for new and experienced Python users
- Core Python docs
- Whats New in Python
- Dive into Python
- Python FAQs
- How-tos
- Python Enhancement Proposals (PEPs)

IPTables log analizer displays Linux 2.4 iptables logs (rejected, acepted, masqueraded packets...) in a nice HTML page (it support rough netfilter logs but also Shorewall and Suse Firewall logs).

This page shall be easy to read and understand to reduce the manual analysis time.

This page containts statistics on packets and links to more detailled information on a given host, port, domain and so on.

To convice you, here is a typical syslog entry for iptables :

[IPTABLES DROP] : IN=ppp0 OUT= MAC= SRC=172.186.2.157 DST=193.253.186.217 LEN=36 TOS=0x00 PREC=0x00 TTL=115 ID=4775 PROTO=ICMP TYPE=8 CODE=0 ID=512 SEQ=3663

How does it work ?

A small deamon is launched by a user which can read iptables logs files. Each time a new packet is logged, the daemon insert a new row in the database.

The statistics and so on are elaborated by the PHP page itself.

Traffic Control Super Script implements traffic shaping for IP traffic passing through a NAT/bridge box with a single configuration file with one line per host.
Traffic Control Super Script can manage bandwidth to user-specified speeds based on the u32 classifier, and can identify traffic by source, destination, source and destination port, protocol, and ToS field. It then limits the rate of connection in either a single or a bidirectional fashion.
Enhancements:
- This release adds multiple interface support.
- It adds an option to choose between flat file and MySQL rules databases.
- It fixes bug #1469742 (duplicate group breaking child / parent relationships when direction=bi on group definitions).
- Various minor bugs have been fixed.
- There are major code cleanups, and major documentation updates on the Web site.

Aps is a small tool for analyzing network traffic. It prints out a great deal of information about the relevant protocols including TCP, UDP, ARP, and ICMP.
It allows you to filter IP addresses, hardware addresses, ports, and specific protocols. It comes with a little GTK-GUI displaying packet counters for each protocol.
APS tries to print detailed info about network frames that are received from the SOCK_RAW (ETH_P_ALL) socket. I am not sure if this is the clean way, but it works fine. APS prints info about the hardware layer and the IP and TCP/UDP/ICMP header.
The tail of the packet (mostly the data) wich could not be interpreted is written on the screen as ascii/hex-dump or both (your choice).
Example
HW-ADDR: 00:60:8c:f6:40:96 -----> 00:80:ad:30:8f:3b
IP-ADDR: 192.168.17.52 -----> 192.168.17.50
IP-Ver4 || Head:0x0a (bytes) || Service(TOS):16 || Length over all:0061
Fragmentation: ID:0x4079 - Flags: 0 1 0 - Offset:00000
TTL:064 || Protokoll:006 (TCP) || HeaderCRC:0x567b
TCP-HEADER:
Ports: 0023-->1034 (telnet) Seq./Ack. Nr.:0x70843468 / 0xeae29434
Data-Offset:0x05 Reserved-6Bit:00 Flags:-urg-ACK-PSH-rst-syn-fin-
Window:0x7fe0 CRC:0x9420 Urgent-Pointer:0x0000
73 61 74 75 72 6e 32 3a 2f 73 72 76 2f 70 72 69 6e 74 71 23 20
HW-ADDR: 52:54:40:25:8d:88 -----> ff:ff:ff:ff:ff:ff
SAMBA/NetBios
e0 e0 03 ff ff 00 22 00 11 00 00 00 00 ff ff ff ff ff ff 04 52 00 00 00 00 52
40 25 8d 88 40 08 00 03 00 04 20 20 20 20 20 20 20 20 20
HW-ADDR: 00:80:ad:30:8f:3b -----> 00:60:8c:f6:40:96
IP-ADDR: 192.168.17.50 -----> 194.112.123.200
IP-Ver4 || Head:0x0a (bytes) || Service(TOS):0 || Length over all:0029
Fragmentation: ID:0x29ae - Flags: 0 0 0 - Offset:00000
TTL:064 || Protokoll:001 (ICMP) || HeaderCRC:0x411f
echo request CODE:0x0 CRC:0xf9f5 SIG:0x602 NUM:0x0
00 ea
Enhancements:
- added break for Packet-counter and fixed some minor bugs