Free isp bonanza software for linux

isp-connect project is a set of scripts which should make it easy to connect from a single GNU/Linux workstation to an ISP using a dial-up connection.
It supports a single PPP connection over ISDN or modem, and support is planned for raw-IP (over ISDN) and hooking up to several providers at one time.
The package comes with the helper script schedule.pl, which is a kind of an offline cron.
It gets a crontab-like file as input and executes the commands given in the file not exactly at the times given (like cron), but only when it is executed itself (of course, because it is not a daemon) and the time in the config-file has passed.
Enhancements:
- Modem support
- Signal, when line goes up or down
- Disconnect can close all active connections

ispCP project was founded to build a Multi Server Control and Administration Panel without limits. Until there is an usable version a lot of time will pass by. This Control Panel should be usable by any Internet Service Provider (isp). To develop such a tool, we need a lot of time.

To gain your interest, we decided to support the dying VHCS project. VHCS means Virtual Hosting Control Panel. It is a powerful tool to administrate your webhosting services. Our fork of VHCS is called the ispCP Omega.

The ispCP Omega release is based on the original VHCS, an open source project. As core members of the former VHCS developing team, we - Malte aka ephigenie and Benedikt aka RatS - had the wish to create a real open development of VHCS. On this platform it is possible.

Since we cannot change the copyright of the parts were using out of the VHCS project which is licensed under the Mozilla Public License, were looking forward to make all new parts available under the Gnu Public License (V2).

The Wonder Shaper is a very special network shaper script with a lot of features. Works on Linux 2.4 & higher.

Goals

I attempted to create the holy grail:

* Maintain low latency for interfactive traffic at all times.

This means that downloading or uploading files should not disturb SSH or even telnet. These are the most important things, even 200ms latency is sluggish to work over.

* Allow surfing at reasonable speeds while up or downloading

Even though http is bulk traffic, other traffic should not drown it out too much.

* Make sure uploads dont harm downloads, and the other way around

This is a much observed phenomenon where upstream traffic simply destroys download speed. It turns out that all this is possible, at the cost of a tiny bit of bandwidth. The reason that uploads, downloads and ssh hurt eachother is the presence of large queues in many domestic access devices like cable or DSL modems.

Why it doesnt work well by default

ISPs know that they are benchmarked solely on how fast people can download. Besides available bandwidth, download speed is influenced heavily by packet loss, which seriously hampers TCP/IP performance. Large queues can help prevent packetloss, and speed up downloads. So ISPs configure large queues.

These large queues however damage interactivity. A keystroke must first travel the upstream queue, which may be seconds (!) long and go to your remote host. It is then displayed, which leads to a packet coming back, which must then traverse the downstream queue, located at your ISP, before it appears on your screen.

This HOWTO teaches you how to mangle and process the queue in many ways, but sadly, not all queues are accessible to us. The queue over at the ISP is completely off-limits, whereas the upstream queue probably lives inside your cable modem or DSL device. You may or may not be able to configure it. Most probably not.

So, what next? As we cant control either of those queues, they must be eliminated, and moved to your Linux router. Luckily this is possible.

Limit upload speed somewhat

By limiting our upload speed to slightly less than the truly available rate, no queues are built up in our modem. The queue is now moved to Linux.

Limit download speed

This is slightly trickier as we cant really influence how fast the internet ships us data. We can however drop packets that are coming in too fast, which causes TCP/IP to slow down to just the rate we want. Because we dont want to drop traffic unnecessarily, we configure a burst size we allow at higher speed.

Now, once we have done this, we have eliminated the downstream queue totally (except for short bursts), and gain the ability to manage the upstream queue with all the power Linux offers.

Let interactive traffic skip the queue

What remains to be done is to make sure interactive traffic jumps to the front of the upstream queue. To make sure that uploads dont hurt downloads, we also move ACK packets to the front of the queue. This is what normally causes the huge slowdown observed when generating bulk traffic both ways. The ACKnowledgements for downstream traffic must compete with upstream traffic, and get delayed in the process.

We also move other small packets to the front of the queue - this helps operating systems which do not set TOS bits, like everything from Microsoft.

Allow the user to specify low priority traffic (new in 1.1!)

Sometimes you may notice low priority OUTGOING traffic slowing down important traffic. In that case, the following options may help you:

NOPRIOHOSTSRC
Set this to hosts or netmasks in your network that should have low priority

NOPRIOHOSTDST
Set this to hosts or netmasks on the internet that should have low priority

NOPRIOPORTSRC
Set this to source ports that should have low priority. If you have an unimportant webserver on your traffic, set this to 80

NOPRIOPORTDST
Set this to destination ports that should have low priority.

See the start of wshaper and wshaper.htb

Results

If we do all this we get the following measurements using an excellent ADSL connection from xs4all in the Netherlands:

Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms

Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms

Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms

With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms

With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms

When uploading, downloads proceed at ~80% of the available speed. Uploads at around 90%. Latency then jumps to 850 ms, still figuring out why.

What you can expect from this script depends a lot on your actual uplink speed. When uploading at full speed, there will always be a single packet ahead of your keystroke. That is the lower limit to the latency you can achieve - divide your MTU by your upstream speed to calculate. Typical values will be somewhat higher than that. Lower your MTU for better effects!

A small table:

Uplink speed | Expected latency due to upload
--------------------------------------------------
32 | 234ms
64 | 117ms
128 | 58ms
256 | 29ms

So to calculate your effective latency, take a baseline measurement (ping on an unloaded link), and look up the number in the table, and add it. That is about the best you can expect. This number comes from a calculation that assumes that your upstream keystroke will have at most half a full sized packet ahead of it.

This boils down to:

mtu * 0.5 * 10
-------------- + baseline_latency
kbit

The factor 10 is not quite correct but works well in practice.

Your kernel

If you run a recent distribution, everything should be ok. You need 2.4 with QoS options turned on.

If you compile your own kernel, it must have some options enabled. Most notably, in the Networking Options menu, QoS and/or Fair Queueing, turn at least CBQ, PRIO, SFQ, Ingress, Traffic Policing, QoS support, Rate Estimator, QoS classifier, U32 classifier, fwmark classifier.

In practice, I (and most distributions) just turn on everything.

The scripts

The script comes in two versions, one which works on standard kernels and is implemented using CBQ. The other one uses the excellent HTB qdisc which is not in the default kernel. The CBQ version is more tested than the HTB one!

See wshaper and wshaper.htb.

Tuning

These scripts need to know the real rate of your ISP connection. This is hard to determine upfront as different ISPs use different kinds of bits it appears. People report success using the following technique:

Estimate both your upstream and downstream at half the rate your ISP specifies. Now verify if the script is functioning - check interactivity while uploading and while downloading. This should deliver the latency as calculated above. If not, check if the script executed without errors.

Now slowly increase the upstream & downstream numbers in the script until the latency comes back. This way you can find optimum values for your connection. If you are happy, please report to me so I can make a list of numbers that work well. Please let me know which ISP you use and the name of your subscription, and its reputed specifications, so I can list you here and save others the trouble.

Installation

If you dial in, you can copy the script to /etc/ppp/ip-up.d and it will be run at each connect.

If you want to remove the shaper from an interface, run wshaper stop. To see status information, run wshaper status.

KNOWN PROBLEMS

If you get errors, add an -x to the first line, as follows:

#!/bin/bash -x

And retry. This will show you which line gives an error. Before contacting me, make sure that you are running a recent version of iproute!

Recent versions can be found at your Linux distributor, or if you prefer compiling, here:
ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz

dslKonnector is a Kommander script to start, stop, restart DSL connection (ppp over ethernet), using tools "pon dsl-provider" and “poff dsl-provider".

To use it you will need to have superuser privileges. dslKonnector does not configure dsl connection, so you will have to setup it using tool "pppoeconf".

I created this script because there was no graphical way to control dsl connection in Kubuntu Dapper, and I needed it sometimes, since some dynamic IPs my wife would receive from our ISP would be blocked from my IRC network.

Would that kind of app be needed? If so, how do I improve it? Suggestions welcome!

IP::Country is a tool for fast lookup of country codes from IP addresses. Finding out the country of a client using only the IP address can be difficult.

Looking up the domain name associated with that address can provide some help, but many IP address are not reverse mapped to any useful domain, and the most common domain (.com) offers no help when looking for country.

IP::Country module comes bundled with a database of countries where various IP addresses have been assigned. Although the country of assignment will probably be the country associated with a large ISP rather than the client herself, this is probably good enough for most log analysis applications, and under test has proved to be as accurate as reverse-DNS and WHOIS lookup.

nionet is a complete client/server VPN solution for raw ethernet tunnels. Clients are available for Linux and Windows; the server is Linux only.
Due to its object oriented architecture, it is possible to easily plug in new transports. So far it features UDP and DNS tunneling. Being an ethernet layer 2 bridge, several network segments can be connected transparently.
When used in an ISP environment, nionet allows per-user traffic-shaping and precise reporting of used bandwidth. On the fly load balancing is possible. nionet uses sqLite or MySQL for account storage; changes to accounts are immediately active.
Main features:
- Compatible with all protocols that work over 802.3 Ethernet
- Uses mySQL or sqlite as backend for user accounts and session log
- Includes traffic accounting and billing for usage in an ISP environment
- Built-in traffic-shaper allows individual bandwidth-limits
- Values in the database can be changed in flight - e.g. new shaper-values will be activated during a running session. This allows dynamic load balancing.
- Built-In fragmentation - full Ethernet MTU of 1514 is tunnelled
- Transport via UDP or DNS
- Uses a virtual Ethernet adapter to be compatible with all existing software (Linux tap-driver, Windows NDIS driver)

debris is a free text mode HTML browser package. It provides support for tables and forms. It is a secure environment suited for public terminals, and is small (only 25% of the size of lynx) and fast
DBOX has all the features of a BBS package for Linux.Unix DBOX was made for heavy-use, professional systems. However, you can also use it for light-use on Windows 95/98/2000/NT.
Up to now, only a German version is available, but an English
version is coming soon.
Main features:
- DBOX has its own user database, designed for *really* many users - Janus/ZConnect/UUCP mail transfers
- user interface: menu system or command line (geonet)
- If you are an ISP, you can charge online time, net traffic and e-mail
- SLIP for internet access with automatic assignment of an IP-address
- support for CD-ROM downloads
- chat
- file boards
- mailing list daemon and mail file server included
- Internet interface (POP3, HTTP/HTML)
- homepage manager for user designed WWW-homepages
- online shopping (HTML/HTTP) with user accounts, logos/images are customizeable
- RADIUS(R) server
Enhancements:
- now compiles on new compilers and libraries
- ostat: SLIP -> IP
- webonline: account.set now also obeyed from group profile
- webonline: webstats now in ~/boxname/webstats/user
- listgen: account statements now from/to abs_host

Iptoip is a perl script to help maintaining an ipvsadm table on a non fixed ip adress or an intermitent connection.
It is specially useful when using an intermittent internet connection or when your ISP breaks conections to reaffect IP addresses (like in France) It use the ipvsadm tool to build and update a forwarding routing table.
The program is written in perl. It uses a configuration file to know which interface to use, which port to forward, and to which box in the internal network. For example your external ip is xxx.xxx.xxx.xxx and you forward the http default port (80) to the computer zzz.zzz.zzz.zzz in your internal network. Everytime youve got a new public ip you should launch iptoip to update your ipvsadm table with your new ip yyy.yyy.yyy.yyy:80 to zzz.zzz.zzz.zzz:80
The name iptoip have no particuliar history, it was found when I was asked by sourceforge to find a name for my project, before it was : toto, test, titi, test1... But anyway, I think iptoip expresses well the idea to transfer something from one ip to another ip...but you know...hey..."whats in my mind" (c) Pixies.
Main features:
- external XML configuration file
- only one file for several configurations
- check if IP is really public
- able to use syslog to log in /var/log/messages
- no-check mode
- verbose mode
Enhancements:
- add udp/tcp support ( Thx Florin Grad )
- change verbose mode
- now an update line is dropped for every forward we add
- should me a lot clearer than before with some cosmetic changes
- remove redundant displaying
- fix usage of alternate config file
- now we can use "iptoip file.xml" we dont need to do "iptoip ./file.xml" as before
- update man/doc pages