per-connection byte counts

per-connection byte counts

[Adam Rice]

I'd like to be able to get per-connection byte-counts from the kernel, so I
can create a sort of top program showing what processes and users are
currently using the network and how much. Since I want this to be something I
can keep running all the time, I don't want to do it by snooping the
interface. Is there some way to do this with netfilter? Sadly
/proc/net/ip_conntrack doesn't appear to provide this information.

Thanks,
  Adam

-- 
Adam Rice -- adamrice@ntlworld.com -- Blackburn, Lancashire, England

Re: [despammed] per-connection byte counts

[Andreas Kretschmer]

am  Sun, dem 18.01.2004, um 14:09:03 +0000 mailte Adam Rice folgendes:
> I'd like to be able to get per-connection byte-counts from the kernel, so I
> can create a sort of top program showing what processes and users are
> currently using the network and how much. Since I want this to be something I

,----[  apt-cache show ipac-ng  ]
| Description: IP Accounting for iptables( kernel >=2.4)
|  Inserts iptables rules to classify network traffic and monitors these
|  rules, writing the data to a file at a certain interval. It will then
|  allow one to calculate IP accounting data and statistics.
`----

Andreas
-- 
Diese Message wurde erstellt mit freundlicher Unterstützung eines freilau-
fenden Pinguins aus artgerechter Freilandhaltung.   Er ist garantiert frei
von Micro$oft'schen Viren. (#97922 http://counter.li.org)     GPG 7F4584DA
Was, Sie wissen nicht, wo Kaufbach ist? Hier: N 51.05082°, E 13.56889° ;-)

Re: per-connection byte counts

[Antony Stone]

On Sunday 18 January 2004 2:44 pm, Andreas Kretschmer wrote:

> am  Sun, dem 18.01.2004, um 14:09:03 +0000 mailte Adam Rice folgendes:
> > I'd like to be able to get per-connection byte-counts from the kernel, so
> > I can create a sort of top program showing what processes and users are
> > currently using the network and how much. Since I want this to be
> > something I
>
> ,----[  apt-cache show ipac-ng  ]
>
> | Description: IP Accounting for iptables( kernel >=2.4)
> |  Inserts iptables rules to classify network traffic and monitors these
> |  rules, writing the data to a file at a certain interval. It will then
> |  allow one to calculate IP accounting data and statistics.

But this will show statistics per rule, not per connection.   Anything which 
works in the rule tables, rather than the connection tracking table, will 
have this problem, because the rules don't distinguish between separate 
connections.

ie: if you have one rule allowing ftp transfers, you will see how much ftp 
traffic goes through the machine, but with all connections combined into a 
single value.

Even if you specify separate source and/or destination addresses in the rules, 
you still only see a combined value for all the connections which match.

I suspect Adam is looking for something which can provide per-connection 
accounting statistics similar to what CheckPoint FW-1 can show.

Regards,

Antony.

-- 
Ramdisk is not an installation procedure.

                                                     Please reply to the list;
                                                           please don't CC me.

Re: per-connection byte counts

[Carlos Carvalho]

Adam Rice (adamrice@ntlworld.com) wrote on 18 January 2004 14:09:
 >I'd like to be able to get per-connection byte-counts from the kernel, so I
 >can create a sort of top program showing what processes and users are
 >currently using the network and how much. Since I want this to be something I
 >can keep running all the time, I don't want to do it by snooping the
 >interface. Is there some way to do this with netfilter? Sadly
 >/proc/net/ip_conntrack doesn't appear to provide this information.

iftop and iptraf give statistics per interface, per machine and per
traffic type plus other interesting info. That's all that can be
obtained from a firewall since you cannot retrieve user info from
another machine. The performance impact is usually negligible.

If you want to run the monitor in a multi-user system where users are
logged in you can associate the network connections with processes and
users but I don't know a program that does it. This is a "me too"...

Re: per-connection byte counts

[bino-psn]

For me, I will pu another netfilter-box at the same eth backbone as the
user.
Put this interface in promiscuous mode.
and ...... i thing you can check http://ipaudit.sourceforge.net

Sincerely
-bino-
----- Original Message -----
From: "Carlos Carvalho" <carlos@fisica.ufpr.br>
To: "Adam Rice" <adamrice@ntlworld.com>
Cc: <netfilter@lists.netfilter.org>
Sent: Monday, January 19, 2004 3:16 AM
Subject: Re: per-connection byte counts


> Adam Rice (adamrice@ntlworld.com) wrote on 18 January 2004 14:09:
>  >I'd like to be able to get per-connection byte-counts from the kernel,
so I
>  >can create a sort of top program showing what processes and users are
>  >currently using the network and how much. Since I want this to be
something I
>  >can keep running all the time, I don't want to do it by snooping the
>  >interface. Is there some way to do this with netfilter? Sadly
>  >/proc/net/ip_conntrack doesn't appear to provide this information.
>
> iftop and iptraf give statistics per interface, per machine and per
> traffic type plus other interesting info. That's all that can be
> obtained from a firewall since you cannot retrieve user info from
> another machine. The performance impact is usually negligible.
>
> If you want to run the monitor in a multi-user system where users are
> logged in you can associate the network connections with processes and
> users but I don't know a program that does it. This is a "me too"...
>
>

Re: per-connection byte counts

[Antony Stone]

On Monday 19 January 2004 12:14 am, bino-psn wrote:

> For me, I will pu another netfilter-box at the same eth backbone as the
> user.
> Put this interface in promiscuous mode.
> and ...... i thing you can check http://ipaudit.sourceforge.net

If this package provides the information you need, why not just run it on the 
netfilter machine?   What's the need for a separate box?

Antony.

> ----- Original Message -----
> From: "Carlos Carvalho" <carlos@fisica.ufpr.br>
> To: "Adam Rice" <adamrice@ntlworld.com>
> Cc: <netfilter@lists.netfilter.org>
> Sent: Monday, January 19, 2004 3:16 AM
> Subject: Re: per-connection byte counts
>
> > Adam Rice (adamrice@ntlworld.com) wrote on 18 January 2004 14:09:
> >  >I'd like to be able to get per-connection byte-counts from the kernel,
>
> so I
>
> >  >can create a sort of top program showing what processes and users are
> >  >currently using the network and how much. Since I want this to be
>
> something I
>
> >  >can keep running all the time, I don't want to do it by snooping the
> >  >interface. Is there some way to do this with netfilter? Sadly
> >  >/proc/net/ip_conntrack doesn't appear to provide this information.
> >
> > iftop and iptraf give statistics per interface, per machine and per
> > traffic type plus other interesting info. That's all that can be
> > obtained from a firewall since you cannot retrieve user info from
> > another machine. The performance impact is usually negligible.
> >
> > If you want to run the monitor in a multi-user system where users are
> > logged in you can associate the network connections with processes and
> > users but I don't know a program that does it. This is a "me too"...

-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

                                                     Please reply to the list;
                                                           please don't CC me.

Re: per-connection byte counts

[bino-psn]

----- Original Message -----
From: "Antony Stone" <Antony@Soft-Solutions.co.uk>
To: <netfilter@lists.netfilter.org>
Sent: Monday, January 19, 2004 7:31 AM
Subject: Re: per-connection byte counts


> If this package provides the information you need, why not just run it on
the
> netfilter machine?   What's the need for a separate box?

No problem Sir.
I just want to split the Monitoring-function from the Packet-Forwarding
function.
I just don't want a fail of one function savere the other.
I never mix complex function in a single box, unless they need tobe.
my small corporate network is build from a bunch of used classic pentium
class PC, with small RAM .. and WISP-Dist in it.
Let's say that the network is in a peak state ... super high traffic. It
will eat more CPU of the monitoring function. I don't want this to kick-down
the packet-forward function. That why I put it in another box.

Sincerely
-bino-

Re: per-connection byte counts

[Nuno Miguel Pais Fernandes]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

Try http://luxik.cdi.cz/~devik/connbytes/

It works very well....

Ahh, connection counter restarts at zero after 4GB of data (32bits).


Nuno Fernandes


On Sun, 2004-01-18 at 14:09, Adam Rice wrote:
> I'd like to be able to get per-connection byte-counts from the kernel, so I
> can create a sort of top program showing what processes and users are
> currently using the network and how much. Since I want this to be something I
> can keep running all the time, I don't want to do it by snooping the
> interface. Is there some way to do this with netfilter? Sadly
> /proc/net/ip_conntrack doesn't appear to provide this information.
> 
> Thanks,
>   Adam
-- 
Nuno Miguel Pais Fernandes <npf@eurotux.com>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: per-connection byte counts

[Ted Kaczmarek]

Ntop does a wonderful job of this, and according to what I read, the
latest version will allow you to store to a file . I mostly use it to
collect sflow data, but have run it on our nms as a local agent as well.

Ted
On Sun, 2004-01-18 at 19:50, bino-psn wrote:
> ----- Original Message -----
> From: "Antony Stone" <Antony@Soft-Solutions.co.uk>
> To: <netfilter@lists.netfilter.org>
> Sent: Monday, January 19, 2004 7:31 AM
> Subject: Re: per-connection byte counts
> 
> 
> > If this package provides the information you need, why not just run it on
> the
> > netfilter machine?   What's the need for a separate box?
> 
> No problem Sir.
> I just want to split the Monitoring-function from the Packet-Forwarding
> function.
> I just don't want a fail of one function savere the other.
> I never mix complex function in a single box, unless they need tobe.
> my small corporate network is build from a bunch of used classic pentium
> class PC, with small RAM .. and WISP-Dist in it.
> Let's say that the network is in a peak state ... super high traffic. It
> will eat more CPU of the monitoring function. I don't want this to kick-down
> the packet-forward function. That why I put it in another box.
> 
> Sincerely
> -bino-
> 
> 

Re: per-connection byte counts

[Adam Rice]

Quoting Nuno Miguel Pais Fernandes (npf@eurotux.com):
> Try http://luxik.cdi.cz/~devik/connbytes/
> 
> It works very well....

Thanks, that's exactly what I wanted! Pity it seems to be unmaintained. Still,
it's simple enough that that shouldn't matter for the time being.

Adam

-- 
Adam Rice -- adamrice@ntlworld.com -- Blackburn, Lancashire, England

Re: per-connection byte counts

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]

On Tue, Jan 20, 2004 at 10:12:59PM +0000, Adam Rice wrote:
> Quoting Nuno Miguel Pais Fernandes (npf@eurotux.com):
> > Try http://luxik.cdi.cz/~devik/connbytes/
> > 
> > It works very well....
> 
> Thanks, that's exactly what I wanted! Pity it seems to be unmaintained. Still,
> it's simple enough that that shouldn't matter for the time being.

erm, connbytes is even in patch-o-matic...

> Adam

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]