Free shaper software for linux

MyShaper is a traffic shaper for use over an Internet interface. It is easily configurable from a configuration file.

This is an init script for RedHat or SYSV Linux to enable a shaper on a ppp0 interface.
This shaper limit your upstream and allow you to keep a few bandwidth for mail, messagers, irc, news (...) and brownsing your usuals Websites easily. Useful if you have a server or if you are using a sharing client like Edonkey or Napster. Just put the ip of your messager, news and mail server (..) in /etc/myshaper.cfg ./myshaper start and its done !

DShaper is a variable bandwidth traffic shaper that can be configured to adjust the size of a DUMMYNET pipe, depending on traffic volume flowing through the pipe.
TODO:
- Add feature: autonegote
Feature will give ability to adjust dshaper ruleset based on actual pipe usage. This is a discipline function.
- Add feature: snmpstat
Feature will enable dshaper statistics to be exported via snmpd.
Useful for monitoring pipe traffic via an snmp application such as cacti or mrtg.
NOTE: Unless requested, these features and additional features will NOT be actively worked on.
Currently Im reviewing my license options, and should have source code available shortly. I am leaning toward the BSD license. Please read the license file for Current binary release.
Enhancements:
- ReStructed Code
- Cleaned Code
- minbw, maxbw, burst, pointpercent, minempty supported
- Tested on FreeBSD4.11

Album Shaper is a graphical application used to create, maintain, and share photo albums using open formats like HTML, XSLT, and JPG.
Two-layer albums can be created in a drag-n-drop interface which allows quick and easy arrangement and catagorization of photos. Batch rotations make getting your photos ready a quick and easy task.
You can also crop, enhance, and manipulate your photos using a powerful but intuitive editing interface. Photos, collections, and albums themselves can be labeled as needed and modified at a later time by saving and loading from a simple XML format.
Albums are exported as HTML which can then be posted directly on the web or viewed straight from your hard drive.
While mainly a bugfix release, Album Shaper 2.1 adds complete French and German translations, a new Radiant theme, and a new Mosaic image manipulation.
The compilation and installation process for Linux/FreeBSD users has also seen significant
improvements.
Enhancements:
New Features / Feature Enhancements:
- New mosaic manipulation!
- Aspect ratio selection usability improvements (smarter placement with regard to rotating selections, positioning, and support for tall photos)
- Improved B/W and Sepia effects (weights now based on modern display phosphor characteristics)
- Added French translation
- German translation brought up to date
Minor Improvements:
- Rewrote AlbumShaper.pro project file to support changing install location on Unix systems
- Changed "Disable checking for..." checkbox to "Check for photo modifications..." under settings
- Cut down window and dialog title text by removing "Album Shaper: " prefix
- Fixed alert dialogs to handle long message translations
- Various spelling mistakes and other minor visible text improvements
- Fixed fonts in status area and various dialogs
- Cosmetic fixes regarding buttons and layout in Save as dialog
- Added missing red color to some button references in help system
Bug Fixes:
- Fixed a bug where dropping objects from the desktop / file browsers on album / collection image caused Album Shaper to crash
- Fixed a bug where Album Shaper would crash when trying to edit 8 bit grayscale and color images
- Fixed a bug where after removing photos in a collection the edit tab and various buttons below were still enabled
- Fixed a threading bug in the file preview feature that could cause the program to hang
- Fixed the rare 0kb bug! (reverting a photo sometimes caused the photo to be corrupted)
- Disabled user input while loading albums
- Fixed a bug where loading recent albums using the keyboard shortcuts could cause Album Shaper to crash
- Fixed a bugs where using save-as to copy an album from one location to another did not copy over the original form of an image
- Fixed minor bug in Slick theme where carriage returns in photo descriptions caused problems in slide show mode
- Fixed a bug in the Metallic theme where collections that had no cover images could not be reached using the navigation bar
- Fixed various compiler warnings under Linux / FreeBSD

pyshaper is a simple yet very versatile dynamic bandwidth manager application for Linux platforms.
Development of pyshaper was started by a number of factors:
Broadband internet access within New Zealand is abysmal, due to the local telco mafia controlling the local loop. You have to either put up with 128kb/s up/down (with 5-15GB monthly traffic), or suffer stupidly restrictive traffic caps (as little as 400MB/month) if you want faster connections
Existing traffic shaping software is either extremely limited or extremely complicated
I like to participate in a few different Peer2peer networks (eg Freenet, I2P etc - not your typical warez/MP3-type networks, but more privacy/anonymity networks), and I needed a simple way to stop these programs from blowing out my traffic
tc has a steep learning curve, and doesnt allow easy filtering on any criteria other than source/destination host/port.
I scoured the net, and came across the wondershaper script, as well as a prototype easy-shaper program called snitch. These programs helped me to start fathoming the occult mysteries of the arcane tc utility (part of the iproute2 suite). tc in its present state is very lacking in doco and examples, so these programs helped heaps.
So, as is an Open Source motto - If you cant find it, write it! - I realised I had to pull my finger out and write something myself.
Main features:
- pyshaper lets you set bandwidth minimum and maximum limits on several criteria:
-
- remote host/port, and local host/port (most shaper apps have this)
- pid of locally connected program
- username under which local program is running
- command line and arguments under which local program was launched
- country in which remmote peer resides
-
- With all these filtering criteria available, you can set up some pretty sophisticated filters.
-
- For instance, you can use the by program filtering to put bandwidth caps on peer2peer programs that often talk via several different protocols, to different ports (which evades most other traffic-shaping programs).
-
- Or, you can set individual inbound and/or outbound limits based on specific countries (or all countries other than your own).
-
- Configuration file syntax is pretty simple and straightforward. No arcane nutsnbolts TCP/IP grease-monkey bit-bashing knowledge needed. After a quick pass through the doco and examples, youll be building your shaping configuration within a few short minutes.

The Wonder Shaper is a very special network shaper script with a lot of features. Works on Linux 2.4 & higher.

Goals

I attempted to create the holy grail:

* Maintain low latency for interfactive traffic at all times.

This means that downloading or uploading files should not disturb SSH or even telnet. These are the most important things, even 200ms latency is sluggish to work over.

* Allow surfing at reasonable speeds while up or downloading

Even though http is bulk traffic, other traffic should not drown it out too much.

* Make sure uploads dont harm downloads, and the other way around

This is a much observed phenomenon where upstream traffic simply destroys download speed. It turns out that all this is possible, at the cost of a tiny bit of bandwidth. The reason that uploads, downloads and ssh hurt eachother is the presence of large queues in many domestic access devices like cable or DSL modems.

Why it doesnt work well by default

ISPs know that they are benchmarked solely on how fast people can download. Besides available bandwidth, download speed is influenced heavily by packet loss, which seriously hampers TCP/IP performance. Large queues can help prevent packetloss, and speed up downloads. So ISPs configure large queues.

These large queues however damage interactivity. A keystroke must first travel the upstream queue, which may be seconds (!) long and go to your remote host. It is then displayed, which leads to a packet coming back, which must then traverse the downstream queue, located at your ISP, before it appears on your screen.

This HOWTO teaches you how to mangle and process the queue in many ways, but sadly, not all queues are accessible to us. The queue over at the ISP is completely off-limits, whereas the upstream queue probably lives inside your cable modem or DSL device. You may or may not be able to configure it. Most probably not.

So, what next? As we cant control either of those queues, they must be eliminated, and moved to your Linux router. Luckily this is possible.

Limit upload speed somewhat

By limiting our upload speed to slightly less than the truly available rate, no queues are built up in our modem. The queue is now moved to Linux.

Limit download speed

This is slightly trickier as we cant really influence how fast the internet ships us data. We can however drop packets that are coming in too fast, which causes TCP/IP to slow down to just the rate we want. Because we dont want to drop traffic unnecessarily, we configure a burst size we allow at higher speed.

Now, once we have done this, we have eliminated the downstream queue totally (except for short bursts), and gain the ability to manage the upstream queue with all the power Linux offers.

Let interactive traffic skip the queue

What remains to be done is to make sure interactive traffic jumps to the front of the upstream queue. To make sure that uploads dont hurt downloads, we also move ACK packets to the front of the queue. This is what normally causes the huge slowdown observed when generating bulk traffic both ways. The ACKnowledgements for downstream traffic must compete with upstream traffic, and get delayed in the process.

We also move other small packets to the front of the queue - this helps operating systems which do not set TOS bits, like everything from Microsoft.

Allow the user to specify low priority traffic (new in 1.1!)

Sometimes you may notice low priority OUTGOING traffic slowing down important traffic. In that case, the following options may help you:

NOPRIOHOSTSRC
Set this to hosts or netmasks in your network that should have low priority

NOPRIOHOSTDST
Set this to hosts or netmasks on the internet that should have low priority

NOPRIOPORTSRC
Set this to source ports that should have low priority. If you have an unimportant webserver on your traffic, set this to 80

NOPRIOPORTDST
Set this to destination ports that should have low priority.

See the start of wshaper and wshaper.htb

Results

If we do all this we get the following measurements using an excellent ADSL connection from xs4all in the Netherlands:

Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms

Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms

Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms

With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms

With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms

When uploading, downloads proceed at ~80% of the available speed. Uploads at around 90%. Latency then jumps to 850 ms, still figuring out why.

What you can expect from this script depends a lot on your actual uplink speed. When uploading at full speed, there will always be a single packet ahead of your keystroke. That is the lower limit to the latency you can achieve - divide your MTU by your upstream speed to calculate. Typical values will be somewhat higher than that. Lower your MTU for better effects!

A small table:

Uplink speed | Expected latency due to upload
--------------------------------------------------
32 | 234ms
64 | 117ms
128 | 58ms
256 | 29ms

So to calculate your effective latency, take a baseline measurement (ping on an unloaded link), and look up the number in the table, and add it. That is about the best you can expect. This number comes from a calculation that assumes that your upstream keystroke will have at most half a full sized packet ahead of it.

This boils down to:

mtu * 0.5 * 10
-------------- + baseline_latency
kbit

The factor 10 is not quite correct but works well in practice.

Your kernel

If you run a recent distribution, everything should be ok. You need 2.4 with QoS options turned on.

If you compile your own kernel, it must have some options enabled. Most notably, in the Networking Options menu, QoS and/or Fair Queueing, turn at least CBQ, PRIO, SFQ, Ingress, Traffic Policing, QoS support, Rate Estimator, QoS classifier, U32 classifier, fwmark classifier.

In practice, I (and most distributions) just turn on everything.

The scripts

The script comes in two versions, one which works on standard kernels and is implemented using CBQ. The other one uses the excellent HTB qdisc which is not in the default kernel. The CBQ version is more tested than the HTB one!

See wshaper and wshaper.htb.

Tuning

These scripts need to know the real rate of your ISP connection. This is hard to determine upfront as different ISPs use different kinds of bits it appears. People report success using the following technique:

Estimate both your upstream and downstream at half the rate your ISP specifies. Now verify if the script is functioning - check interactivity while uploading and while downloading. This should deliver the latency as calculated above. If not, check if the script executed without errors.

Now slowly increase the upstream & downstream numbers in the script until the latency comes back. This way you can find optimum values for your connection. If you are happy, please report to me so I can make a list of numbers that work well. Please let me know which ISP you use and the name of your subscription, and its reputed specifications, so I can list you here and save others the trouble.

Installation

If you dial in, you can copy the script to /etc/ppp/ip-up.d and it will be run at each connect.

If you want to remove the shaper from an interface, run wshaper stop. To see status information, run wshaper status.

KNOWN PROBLEMS

If you get errors, add an -x to the first line, as follows:

#!/bin/bash -x

And retry. This will show you which line gives an error. Before contacting me, make sure that you are running a recent version of iproute!

Recent versions can be found at your Linux distributor, or if you prefer compiling, here:
ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz

CMS RSS Grabber is a system that grabs RSS feeds and fills the content of a CMS system. In other words, CMS content syndication through RSS
It is built in such a flexible way that it doesnt matter which CMS system it connects to. Settings inside the script let you define how you access the database.
Enhancements:
- An option to strip the http:// from the URL that is being exported to CMS.
- Usage is [$url=nohttp$].
- A CMS authors query is added to CMS Settings.
- Selection of author that adds content can be inserted with the [$authorid$] variable.
- Misc. variables for dates and time are added.
- An installation script is added for easy install.
- Settings for WordPress CMS are added.
- The ImportCMS() function didnt include the prefix to the feeddata table, imports to Joomla gave double texts in articles and the URLs had double http:// in them, and CMS update error messages were shown before logging into the system.
- These issues have been fixed.

chkrootkit is a tool to locally check for signs of a rootkit.
It contains:
- chkrootkit: shell script that checks system binaries for rootkit modification.
- ifpromisc.c: checks if the interface is in promiscuous mode.
- chklastlog.c: checks for lastlog deletions.
- chkwtmp.c: checks for wtmp deletions.
- check_wtmpx.c: checks for wtmpx deletions. (Solaris only)
- chkproc.c: checks for signs of LKM trojans.
- chkdirs.c: checks for signs of LKM trojans.
- strings.c: quick and dirty strings replacement.
- chkutmp.c: checks for utmp deletions.
The following tests are made:
aliens asp bindshell lkm rexedcs sniffer w55808 wted scalper slapper z2 chkutmp amd basename biff chfn chsh cron date du dirname echo egrep env find fingerd gpm grep hdparm su ifconfig inetd inetdconf identd init killall ldsopreload login ls lsof mail mingetty netstat named passwd pidof pop2 pop3 ps pstree rpcinfo rlogind rshd slogin sendmail sshd syslogd tar tcpd tcpdump top telnetd timed traceroute vdir w write
The following rootkits, worms and LKMs are currently detected:
01. lrk3, lrk4, lrk5, lrk6 (and variants);
02. Solaris rootkit;
03. FreeBSD rootkit;
04. t0rn (and variants);
05. Ambients Rootkit (ARK);
06. Ramen Worm;
07. rh[67]-shaper;
08. RSHA;
09. Romanian rootkit;
10. RK17;
11. Lion Worm;
12. Adore Worm;
13. LPD Worm;
14. kenny-rk;
15. Adore LKM;
16. ShitC Worm;
17. Omega Worm;
18. Wormkit Worm;
19. Maniac-RK;
20. dsc-rootkit;
21. Ducoci rootkit;
22. x.c Worm;
23. RST.b trojan;
24. duarawkz;
25. knark LKM;
26. Monkit;
27. Hidrootkit;
28. Bobkit;
29. Pizdakit;
30. t0rn v8.0;
31. Showtee;
32. Optickit;
33. T.R.K;
34. MithRas Rootkit;
35. George;
36. SucKIT;
37. Scalper;
38. Slapper A, B, C and D;
39. OpenBSD rk v1;
40. Illogic rootkit;
41. SK rootkit.
42. sebek LKM;
43. Romanian rootkit;
44. LOC rootkit;
45. shv4 rootkit;
46. Aquatica rootkit;
47. ZK rootkit;
48. 55808.A Worm;
49. TC2 Worm;
50. Volc rootkit;
51. Gold2 rootkit;
52. Anonoying rootkit;
53. Shkit rootkit;
54. AjaKit rootkit;
55. zaRwT rootkit;
56. Madalin rootkit;
57. Fu rootkit;
58. Kenga3 rootkit;
59. ESRK rootkit;
chkrootkit has been tested on: Linux 2.0.x, 2.2.x, 2.4.x and 2.6.x, FreeBSD 2.2.x, 3.x, 4.x and 5.x, OpenBSD 2.x and 3.x., NetBSD 1.6.x, Solaris 2.5.1, 2.6, 8.0 and 9.0, HP-UX 11, Tru64 and BSDI.
Enhancements:
chkutmp.c (Thanks to Jeremy Miller)
- the idea of this program is to display users that may have wiped themselves from the utmp log
chkproc.c
- fix: better support for Linux threads
chkrootkit
- new test: chkutmp
- new rootkits detected: Fu, Kenga3, ESRK
- some bug fixes
homepage redesign (Thanks to Cristine Hoepers)
- navigability improvement
- the page now validates as strict XHTML
- still lynx friendly