Free netflow software for linux

DNA (Deep Network Analyser) is an open, flexible, and extensible deep network analyzer server and software architecture for passively gathering and analyzing network packets, network sessions, and applications protocols.
Deep Network Analyser project is designed to be used for Internet security, network management, intrustion detection, protocol and network analysis, information gathering, and network monitoring applications.
Main features:
- Extensible Java based network sensor (processing layers 2-7)
Configurable processing and output:
- Packet flows like Ethereal
- IP Flows like CISCO netflow
- Stateful Sessions (client/server flow pairs)
- Application protocol element output
- Configurable and extensible application protocol element parsing.
- Application protocol parsing toolkit APIs allows for new protocol parser to be easily developed and extended
- Targeting based full session capture facility, like a realtime targeted TCPDump.
- Flexible targeting from IPAddr, Port tuple to Application sensitive targeting.
- Configurable and extensible output forwarding (file, DB, Streams, JMS, RMI, etc.)
- Extensible realtime collection portable to many OS/Packet processing environments
Easily adaptable to packet processing environments:
- Specialized linux drivers mechanismon
- Network Appliances
- Network Switches / Routers
- Highly mutithreaded for increased performance over multi processor environments
Enhancements:
- Adoption of OpenAdaptor(tm) as the Output Adapter mechanism.
- Support for local-only administration.
- A new targeted packet capture parser, new run scripts, and a new install mechanism.
- Many bugfixes.

flow-tools is a set of programs for processing and managing NetFlow exports from Cisco and Juniper routers. The tools included are: flow-capture, flow-cat, flow-dscan, flow-expire, flow-export, flow-fanout, flow-filter, flow-gen, flow-header, flow-import, flow-mask, flow-merge, flow-nfilter, flow-print, flow-receive, flow-report, flow-send, flow-split, flow-stat, flow-tag, and flow-xlate.

Flow data is collected and stored by default in host byte ordera nd the files are portable across every endian architectures.

Commands that utilize the network use a localip/remoteip/port designation for communication. "localip" is the IP address the host will use as a source for sending or bind to when receiving NetFlow PDUs (ie the destination address of the exporter. Configuring the "localip" to 0 will force the kernel to decide what IP address to use for sending and listen on all IP addresses for receiving. "remoteip" is the destination IP address used for sending or the expected address of the source when receiving. If the "remoteip" is 0 then the application will accept flows from any source address. The "port" is the UDP port number used for sending or receiving. When using multicast addresses the localip/remoteip/port is used to represent the source, group, and port respectively.

Flows are exported from a router in a number of different configurable versions. A flow is a collection of key fields and additional data. The flow key is {srcaddr, dstaddr, input, output, srcport, dstport, prot, ToS}. Flow-tools supports one export version per file.

Export versions 1, 5, 6, and 7 all maintain {nexthop, dPkts, dOctets, First, Last, flags}, ie the next-hop IP address, number of packets, number of octets (bytes), start time, end time, and flags such as the TCP header bits. Version 5 adds the additional fields {src_as, dst_as, src_mask, dst_mask}, ie source AS, destination AS, source network mask, and destination network mask. Version 7 which is specific to the Catalyst switches adds in addition to the version 5 fields {router_sc}, which is the Router IP address which populates the flow cache shortcut in the Supervisor. Version 6 which is not officially supported by Cisco adds in addition to the version 5 fields {in_encaps, out_encaps, peer_nexthop}, ie the input and output interface encapsulation size, and the IP address of the next hop within the peer. Version 1 exports do not contain a sequence number and therefore should be avoided, although it is safe to store the data as version 1 if the additional fields are not used.

Version 8 IOS NetFlow is a second level flow cache that reduces the data exported from the router. There are currently 11 formats, all of which provide {dFlows, dOctets, dPkts, First, Last} for the key fields.

8.1 - Source and Destination AS, Input and Output interface
8.2 - Protocol and Port
8.3 - Source Prefix and Input interface
8.4 - Destination Prefix and Output interface
8.5 - Source/Destination Prefix and Input/Output interface
8.9 - 8.1 + ToS
8.10 - 8.2 + ToS
8.11 - 8.3 + ToS
8.12 - 8.5 + ToS
8.13 - 8.2 + ToS
8.14 - 8.3 + ports + ToS

Version 8 CatIOS NetFlow appears to be a less fine grained first level flow cache.

8.6 - Destination IP, ToS, Marked ToS,
8.7 - Source/Destination IP, Input/Output interface, ToS, Marked ToS,
8.8 - Source/Destination IP, Source/Destination Port,
Input/Output interface, ToS, Marked ToS,

The following programs are included in the flow-tools distribution.

flow-capture - Collect, compress, store, and manage disk space for exported flows from a router.
flow-cat - Concatenate flow files. Typically flow files will contain a small window of 5 or 15 minutes of exports. Flow-cat can be used to append files for generating reports that span longer time periods.
flow-fanout - Replicate NetFlow datagrams to unicast or multicast destinations. Flow-fanout is used to facilitate multiple collectors attached to a single router.
flow-report - Generate reports for NetFlow data sets. Reports include source/destination IP pairs, source/destination AS, and top talkers. Over 50 reports are currently supported.
flow-tag - Tag flows based on IP address or AS #. Flow-tag is used to group flows by customer network. The tags can later be used with flow-fanout or flow-report to generate customer based traffic reports.
flow-filter - Filter flows based on any of the export fields. Flow-filter is used in-line with other programs to generate reports based on flows matching filter expressions.
flow-import - Import data from ASCII or cflowd format.
flow-export - Export data to ASCII or cflowd format.
flow-send - Send data over the network using the NetFlow protocol.
flow-receive - Receive exports using the NetFlow protocol without storing to disk like flow-capture.
flow-gen - Generate test data.
flow-dscan - Simple tool for detecting some types of network scanning and Denial of Service attacks.
flow-merge - Merge flow files in chronoligical order.
flow-xlate - Perform translations on some flow fields.
flow-expire - Expire flows using the same policy of flow-capture.
flow-header - Display meta information in flow file.
flow-split - Split flow files into smaller files based on size, time, or tags.

The Argus Open Project is focused on developing network activity audit strategies that can do real work for the network architect, administrator and network user.
LATEST NEWS
Mon Jun 19 10:44:52 EDT 2006 *argus-3.0.0 testing has started!
Welcome to the Argus Open Project, home of Argus, the network Audit Record Generation and Utilization System. The Argus Open Project main goal is developing network activity audit strategies that can do real work for the network architect, administrator and network user.
Argus is a fixed-model Real Time Flow Monitor designed to track and report on the status and performance of all network transactions seen in a data network traffic stream. Argus provides a common data format for reporting flow metrics such as connectivity, capacity, demand, loss, delay, and jitter on a per transaction basis. The record format that Argus uses is flexible and extensible, supporting generic flow identifiers and metrics, as well as application/protocol specific information.
Argus can be used to analyze and report on the contents of packet capture files or it can run as a continuous monitor, examining data from a live interface; generating an audit log of all the network activity seen in the packet stream. Argus can be deployed to monitor individual end-systems, or an entire enterprises network activity. As a continuous monitor, Argus provides both push and pull data handling models, to
allow flexible strategies for collecting network audit data. Argus data clients support a range of operations, such as sorting, aggregation, archival and reporting. There is XML support for Argus data, which makes handling Argus data a bit easier, see ArgusRecord.xsd.
The network transaction audit data that Argus generates has been used for a wide range of tasks including Security Management, Network Billing and Accounting, Network Operations Management and Performance Analysis.
Argus currently runs on Linux, Solaris, FreeBSD, OpenBSD, NetBSD, and MAC OS X and its client programs have also been ported to Cygwin. The software should be portable to many versions of Unix with little or no modification. Performance is such that auditing an entire enterprises Internet activity can be accomplished using
modest computing resources. The Argus Open Project is an ongoing and active project. If you areinterested in participating, check out the mailing lists and sign up today!
Enhancements:
- Multithreaded
- Daemon Support
- Configuration Files
- Syslog Support
- Secure Access
- Audit Record Changes
- Variable Length Records
- Argus Source Identifier
- Sequence Number
- Transaction Reference Number
- Security Layer (ESP) Support
- Application Layer Byte Counts
- Application Layer Data Capture
- Multiprotocol Support
- Enhanced Performance Reporting
- Enhanced TCP Status Reporting
- Enhanced Aggregation Support
- Server Changes
- Improved Accuracy
- Improved Reliability
- Improved Fragment Support
- Multiprotocol Support
- Authenticated Access
- Confidential Access
- Enhanced Physical Interface Support
- Multiple Physical Interface Support
- Multiple Output File Support
- Independant Output Filters
- Server Side Filtering
- Improved Signal Handling
- Daemon Support
- Syslog Event Reporting
- System Configuration
- Environment Variable Support
- Enhanced Performance Reporting
- Response Time Determination Support
- User Data Capture Support
- Client Changes
- Multiple Server Support
- Configurable Output Formats
- Cisco Netflow Record Support
- Environment Variable Support
- Configuration
- XML Data Support
- Excel Data Importation Support
- User Data Printing
- ragrep()
- Support Scripts and Programs
- System startup routines
- Sample configurations
- Sample Argus Archiving scripts
- argusbug Bug reporting tool
- Magic file support
- Documentation
- Better documenation?
- HTML man pages.
- FAQ
- HOW-TO

bbnfc is useful for debugging netflow exports as produced by Cisco, Juniper, etc. routers. This simple program sits on a user-specified UDP port and displays to stdout all netflow exports that are sent to the machine. Complete program is 300 lines long (C source).

The source should compile under most flavours of UNIX, with little or no modification.
Note: Some web browsers will tend to display .tgz archives on the screen rather than saving them to a file. You may need to hold down the shift key when selecting the link below (or possibly some other brain-dead combination).


In this archive you will see the following files:
readme.bbnfc - This readme file
bbnfc.c - Source code for bbnfc
netflow.h - Header file
bbnfc - Pre-compiled binary for Linux (Intel)

To compile from source the following (or some variation thereof) should work:
gcc -o bbnfc bbnfc.c

Basic usage instructions can be obtained with
./bbnfc -h

EHNT is a tool which turns streams of Netflow (version 5) data into something useful and human-readable. (Netflow is a UDP-based traffic reporting protocol created by Cisco, generated by Cisco, Juniper, Foundry, and other routers.)
Netflow operates in many ways. It will dump flow records in human-readable form. It will also provide reports on top ASes, IP protocols, and tcp/udp ports. The reports can be generated over various intervals, from 1 minute to 1 day.
Component programs are:
1. ehntserv listens to netflow version 5 UDP packets, and also listens for client TCP connections. When a TCP client connects, the server starts forwarding all the netflow packets it receives (plus the IP address of the originating device) to that client.
ehntserv does not currently do any IP access control. I suggest that you use ipchains or iptables on your linux box, or IP Filter (ipf) (http://coombs.anu.edu.au/ipfilter/) on your Solaris or BSD box. I dont know what the current state of packet filtering is on other Unixes; IP Filter seems to support several.
2. ehnt connects to ehntserv and displays the flows it receives in various ways. It currently has four modes (-m ):
- top mode displays average utilization by top ASes, IP protocols, or tcp/udp ports over a given interval (from 1 minute to 1 day).
Top mode is different when it focuses on a single interface on a single router, because then you get to see summaries of source and destionation for both inbound and outbound traffic. Otherwise, you just get summaries of source and destination.
- dump mode displays individual flows
- shortdump mode display individual flows in a more compact but hard
to read fashion
- colondump mode display individual flows in a machine-readable format.
And yes, I recognize that the name of this mode is unpleasant.
In all three modes, simple (REALLY simple) filtering can be done for AS
number, TCP/UDP port, IP protocol number, device sending the flow record,
and SNMP interface index.
You may think of ehnt in the three dump modes as a brain-dead and incredibly
simple tcpdump for netflow.
ehnt also has the silly and uninspiredly-named big filter, in which it
only displays flows with are bigger (in packets or bytes) than any flow
received before it. This only makes sense in the three dump modes.
Enhancements:
- Added Unix domain support for client connections, enabled by default

FlowScan is a network analysis and reporting tool.[ COPYRIGHT=1]
Enhancements:
- The CampusIO and SubNetIO reports were enhanced with a new optional configuration directive: TopN. When defined, this directive causes ``Top Talker reports to be produced. These HTML reports contain the most active (i.e. ``top) source and destination addresses.
- The CampusIO and SubNetIO reports were enhanced to record the number of local IP addresses that where active for each network and subnet into the RRD files. This enables users to estimate the number of active hosts hosts over time, detect ``scans which systematically sweep across network address space, and to calculate the average bytes, packets, and flows per host.
- The template Makefile used to produce the graphs was enhanced to allow the inclusion of ``events in the graphs, similarly to what can be done with Cricket. This allows you to label events such as configuration changes and outages to discover correlations with traffic measurement.
- Two new utilities suitable for stand-alone use, are included. ip2hostname converts IP addresses to their respective hostnames. event2vrule adds ``events to rrdtool graphs.
- Added support for LFAP (Lightweight Flow Accouting Protocol) used by Riverstone and Enterasys (formerly Cabletron) routers. This currently requires slate (from http://www.nmops.org) and lfapd by Steven Premeau . lfapd produces time-stamped raw flow files in the same cflowd-defined format that is processed by FlowScan.
- Added the ability for the CampusIO report to identify outbound flows based solely on the flows destination IP address. While this is less trustworthy than using NextHops or OutputIfIndexes, it is now the default and will be useful for environments where the flow nexthop or output ifIndex values are not meaningful.
- The CampusIO report contains a new experimental feature which reads a BGP routing table, and therefore can determine which Autonomous systems source, transit, or sink most of your institutions traffic. The CampusIO report was enhanced with new optional configuration directives: BGPDumpFile, TopN, ReportPrefixFormat. When properly defined, these directives cause CampusIO to create tabular HTML reports named {origin|path}_{in|out}.html under OutputDir after analyzing each raw flow file. These reports show the ``top Autonomous Systems with which your site exchanges traffic.
- A WebProxyIfIndex directive was added to the CampusIO report. This allows one to specify the index of the interface to which HTTP traffic is being transparently redirected. This enables FlowScan to properly count HTTP flows even though NetFlow v5 does not accurately report the nexthop value for flows which are transparently redirected via a Cisco route-map.
- CampusIO now contains a fix for a bug introduced in FlowScan-1.005 which would sometimes cause perl to abort with this message: patricia.c:645: patricia_lookup: Assertion `prefix failed.

JNFA project is a netflow analyzer. It uses a MySQL database to store accounting information.

Filters are used in JNFA to allow very flexible classification any kind of traffic and to store it in the different fields in a database.