RE: IP Acounting Idea for 2.5

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

> > Jesse Pollard replies:
> > to Leif Sawyer who wrote: 
> > >> Besides, what would be gained in making the counters RO, if 
> > >> they were cleared every time the module was loaded/unloaded?
> > > 
> > > 1. Knowlege that the module was reloaded.
> > > 2. Knowlege that the data being measured is correct
> > > 3. Having reliable measures
> > > 4. being able to derive valid statistics
> > > ....
> > 
> > Good.  Now that we have valid objectives to reach, which of these
> > are NOT met by making the fixes entirely in userspace, say by
> > incorporating a wrapper script to ensure that no external 
> applications
> > can flush the table counters?
> > 
> > They're still all met, right?
> 
> Nope - some of the applications that may be purchased do not have
> to go through the scripts to reset the counters. They just need access
> to the counters, and reset is built into the applications. 
> This violates
> all 4 objectives.
> 
> > And we haven't had to fill the kernel with more cruft.
> 
> Removing/no-oping the reset code would make the module
> SMALLER, and simpler.
> 

NO.  Don't remove the functionality that is required.  Fix your
userspace applications to behave correctly.  If _you_ require your
userspace applications to not clear counters, then fix the application.

The counters are not gospel.  The modules are not omnipresent.

Time and again, people beg and plead the kernel developers to change
the kernel to suit their personal biases and applications.  Historically
they have been vehemently against it.  Of what purpose does breaking EVERY
application out there by performing this kernel mod have?

As Matti Aarnio points out, this is nearly beaten to death.  Since none
of the 'Big Boys' have commented, my guess is that this is a moot argument,
and no changes to the kernel will take place.

/me steps off of soapbox
/me retires this thread

 

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Leif Sawyer <lsawyer@gci.com> writes:

> > > Jesse Pollard replies:

> > Removing/no-oping the reset code would make the module
> > SMALLER, and simpler.

> NO.  Don't remove the functionality that is required.  

Please explain where counter reset capability provides any 
functionality that is not already available without it.

You might want to read RFC2724 ``Traffic Flow Measurement''.
Search for ``reset''.

Counter resets have always caused problems, that's why mission 
critical counters never have a reset.  Have you ever seen an
electricity, gas or water meter with a reset?  The same reasoning
applies when you have accounting rules that are used to charge
customers for traffic volume.

> Fix your userspace applications to behave correctly.  If _you_
> require your userspace applications to not clear counters, then fix
> the application.

You are confused.  What would you say if a close() by another,
unrelated application closed all open descriptors for that file,
including the one you just opened?  Just fix your applications?

-- 
Manfred Bartz

Re: IP Acounting Idea for 2.5

[Alan Cox]

> > Fix your userspace applications to behave correctly.  If _you_
> > require your userspace applications to not clear counters, then fix
> > the application.
> 
> You are confused.  What would you say if a close() by another,

No he isnt confused, you are trying to dictate policy.

> unrelated application closed all open descriptors for that file,
> including the one you just opened?  Just fix your applications?

That would be a bug. The standards and common sense say so, but your argument
is more akin to 'if I delete a file in one app it hurts when I try and use
it in another'.

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > > Fix your userspace applications to behave correctly.  If _you_
> > > require your userspace applications to not clear counters, then fix
> > > the application.
> > 
> > You are confused.  What would you say if a close() by another,
> 
> No he isnt confused, you are trying to dictate policy.

What then *is* the policy?

AFAIK, no counters in the Linux kernel have resets, except in netfilter.

-- 
Manfred Bartz

Re: IP Acounting Idea for 2.5

[Alan Cox]

> > No he isnt confused, you are trying to dictate policy.
> 
> What then *is* the policy?

The policy is not to have policy. It works as well in kernel design as politics.

Alan

Re: IP Acounting Idea for 2.5

[Ton Hospel]

In article <E14pfQ3-0003bG-00@the-village.bc.nu>,
	Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
>> > No he isnt confused, you are trying to dictate policy.
>> 
>> What then *is* the policy?
> 
> The policy is not to have policy. It works as well in kernel design as politics.
> 
> Alan
> 
Since my job is in fact mainly this kind of apps, i really feel strongly
about this. 

Resettable counters are evil.

Having resettable counters may not sound like it, but it is in fact policy.
It forces all apps to add code to detect these resets (and then give 
warnings/errors, since there is just no way to do anything sensible with
them), since ignoring them will seemingly cause up to 2**32 counts suddenly.

It is also doing something in kernelspace which can be done in userspace,
which is normally considered a big nono.

Proposal: have a snapshot command, that remembers the current value of a
counter. Then have two interfaces: one that shows the continuous counter
and one that shows the subtraction of the current value from the snapshot.
The first can be used by used by serious applications (don't have to
add code to give warnings about dataloss), and the second can be used by 
users who want to watch the counters a bit to get a feel for what a rule
is doing.

I really think cisco got this right: from the commandline interface
you can reset counters, and watch them, the SNMP counters however just
keep going and going and going independently from this.

(I think this snapshotting belongs in the apps reading the counters
really, but if you really insist on a kernel based reset, this might be
reasonable).

Re: IP Acounting Idea for 2.5

[Mark van Walraven]

On Fri, Apr 20, 2001 at 02:51:55AM +0000, Ton Hospel wrote:
> Resettable counters are evil.

Perhaps "evil" should be reserved to describe counters which automatically
reset as a side effect of being read.

> I really think cisco got this right: from the commandline interface
> you can reset counters, and watch them, the SNMP counters however just
> keep going and going and going independently from this.

Except for the IP accounting table?  The 'checkpoint' operation copies and
*clears* the table ...

I don't really need snapshots, just hazard-free reads.  It's quite
easy to work out deltas in userspace, but I'd hate for things to be
unnecessarily painful for multiple readers.

Mark.

Re: IP Acounting Idea for 2.5

[Kenneth Johansson]

Alan Cox wrote:

> > > Fix your userspace applications to behave correctly.  If _you_
> > > require your userspace applications to not clear counters, then fix
> > > the application.
> >
> > You are confused.  What would you say if a close() by another,
>
> No he isnt confused, you are trying to dictate policy.

Well it's not actually possible to do a fix in userspace for a odometer type of counter that can be reset. I don't know what you mean about policy but this reset "feature" is a shure way to get bad values. I have not seen one good reason to have a reset other than it easier to read and that is something that can be fixed in userspace.

Re: IP Acounting Idea for 2.5

[Jonathan Lundell]

At 10:16 AM +0200 2001-04-18, Kenneth Johansson wrote:
>Alan Cox wrote:
>
>> > > Fix your userspace applications to behave correctly.  If _you_
>> > > require your userspace applications to not clear counters, then fix
>> > > the application.
>> >
>> > You are confused.  What would you say if a close() by another,
>>
>> No he isnt confused, you are trying to dictate policy.
>
>Well it's not actually possible to do a fix in userspace for a odometer type of counter that can be reset. I don't know what you mean about policy but this reset "feature" is a shure way to get bad values. I have not seen one good reason to have a reset other than it easier to read and that is something that can be fixed in userspace.

Moreover, in a non-resettable-counter environment, any client of the counter can trivially achieve the effect of a reset by (locally) storing a snapshot and doing a subtract. Conversely, if the counter is truly reset, information is lost permanently.
-- 
/Jonathan Lundell.

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

Manfred Bartz writes:
> You are confused.  What would you say if a close() by another,
> unrelated application closed all open descriptors for that file,
> including the one you just opened?  Just fix your applications?

Of course I would fix the application.
And I certainly wouldn't rip the close() out of libc, which is
the tactic that you are suggesting.

Sheesh.



 

RE: IP Acounting Idea for 2.5

[Jesse Pollard]

---------  Received message begins Here  ---------

> 
> Jesse Pollard replies:
> to Leif Sawyer who wrote: 
> >> Besides, what would be gained in making the counters RO, if 
> >> they were cleared every time the module was loaded/unloaded?
> > 
> > 1. Knowlege that the module was reloaded.
> > 2. Knowlege that the data being measured is correct
> > 3. Having reliable measures
> > 4. being able to derive valid statistics
> > ....
> 
> Good.  Now that we have valid objectives to reach, which of these
> are NOT met by making the fixes entirely in userspace, say by
> incorporating a wrapper script to ensure that no external applications
> can flush the table counters?
> 
> They're still all met, right?

Nope - some of the applications that may be purchased do not have
to go through the scripts to reset the counters. They just need access
to the counters, and reset is built into the applications. This violates
all 4 objectives.

> And we haven't had to fill the kernel with more cruft.

Removing/no-oping the reset code would make the module
SMALLER, and simpler.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

Jesse Pollard replies:
to Leif Sawyer who wrote: 
>> Besides, what would be gained in making the counters RO, if 
>> they were cleared every time the module was loaded/unloaded?
> 
> 1. Knowlege that the module was reloaded.
> 2. Knowlege that the data being measured is correct
> 3. Having reliable measures
> 4. being able to derive valid statistics
> ....

Good.  Now that we have valid objectives to reach, which of these
are NOT met by making the fixes entirely in userspace, say by
incorporating a wrapper script to ensure that no external applications
can flush the table counters?

They're still all met, right?

And we haven't had to fill the kernel with more cruft.

Re: IP Acounting Idea for 2.5

[Matti Aarnio]

I repeat myself, fighting is apparently so pleasant that you are stuck on
fighting over dead-end technology:

  I seriously suggest that for the primary (subject given) topic
  you are SERIOUSLY OFF TARGET.  Look around, counting hits on
  some fw rules is waste of time!  (And mightly inaccurate!)

  You absolutely don't want to do any sort of counting aggeration policy
  control within kernel ( = FW rules ).   You want to collect accounting
  per flow, and send those data records to offline analysis.

  No more fighting of when to clear counters, and when not.

  Having used (with own custom analyzers) cisco netflow, I can say
  that any sort of "count hits on access-list elements" things are
  from stone-age:

http://www.cisco.com/warp/public/cc/pd/iosw/ioft/neflct/tech/napps_wp.htm


  Yet another nice thing to cook up -- if I had time ...

/Matti Aarnio

RE: IP Acounting Idea for 2.5

[Michael Clark]

> I repeat myself, fighting is apparently so pleasant that
> you are stuck on
> fighting over dead-end technology:
>
>   I seriously suggest that for the primary (subject given) topic
>   you are SERIOUSLY OFF TARGET.  Look around, counting hits on
>   some fw rules is waste of time!  (And mightly inaccurate!)

I agree. We could all stop re-inventing the wheel and use a
RFC2724/RFC2722/RFC2720 compliant traffic meter such as NeTraMet -
which has already solved most of the mentioned problems - has a
flexible rule language for matching flows and managing counters -
support for multiple protocols; not just IP - a distributed
architecture - SNMP accessable meter and remote Manager and Controller
(NeMaC) which can concurrently read from multiiple meters (including
NetFlow meters).

>   You absolutely don't want to do any sort of counting
> aggeration policy
>   control within kernel ( = FW rules ).   You want to
> collect accounting
>   per flow, and send those data records to offline analysis.

Yes, the IP accounting effort could do well by creating a fastpath for
feeding packet headers (can't say I know how optimal libpcap is
currently on Linux) to a userspace meter (like NeTraMet) letting it
deal with all of the policy.

I remember the DOS version of NeTraMet performed much better than the
Linux version (some years ago) due to custom ethernet drivers (for
some cards) that only generate interupts when a ring buffer is full of
packet headers - maybe the same sort of infrastructure (some of the
Linux GigE drivers also avoid the interupt per packet performance hit)
could be added to Linux and integrated with libpcap and leave the rest
up to a userspace meter application.

I'm sure Neville (traffic metering god - Hi Neville) would be pleased
to have optimized support for NeTraMet in the Linux kernel.

>   No more fighting of when to clear counters, and when not.
>
>   Having used (with own custom analyzers) cisco netflow, I can say
>   that any sort of "count hits on access-list elements" things are
>   from stone-age:

NetFlow really sucks alot doesn't - I remeber having bad aliasing
problems (trying to generate 5min averages) due to its minumum flow
export interval of 1 minute. Is this still the case?

Michael Clark.

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Leif Sawyer <lsawyer@gci.com> writes:

> Jesse Pollard replies:
> to Leif Sawyer who wrote: 
> > > Besides, what would be gained in making the counters RO, if 
> > > they were cleared every time the module was loaded/unloaded?
> > 
> > 1. Knowlege that the module was reloaded.
> > 2. Knowlege that the data being measured is correct
> > 3. Having reliable measures
> > 4. being able to derive valid statistics

> Good.  Now that we have valid objectives to reach, which of these
> are NOT met by making the fixes entirely in userspace, ...

By this logic you could ask why we need separate user spaces,
just fix all applications so they cooperate nicely.

> They're still all met, right?

No.  Because they require well behaved, cooperative user programs.

> And we haven't had to fill the kernel with more cruft.

Nonsense.  Removing counter reset capabilities reduces kernel cruft.

-- 
Manfred Bartz

RE: IP Acounting Idea for 2.5

[Jesse Pollard]

Leif Sawyer <lsawyer@gci.com>:
> > And that introduces errors in measurement. It also depends on 
> > how frequently an uncontroled process is clearing the counters.
> > You may never be able to get a valid measurement.
> 
> This is true.  Which is why application programmers need to write
> code as if they are not the only [ab]users of data.
> 
> Which brings me back to my point.
> 
> Don't force the kernel to uphold your local application requirements
> of stable counters.
> 
> Enforce it in the userspace portion of the code.
> 
> <subtopic>
> Yes, you could extend the proc filesystem (ugh) with a flag that could
> be read by the ip[chains|tables] user app to determine if clearing flags
> were allowed.  Then a simple
> 
> echo 1 > /proc/sys/net/ipv4/counters_locked
> 
> or some such cruft.  But I don't see this extension making into the
> standard kernel at this time.  It just seems to be wasteful.
> </subtopic>
> 
> If you (at your site) really need this type of functionality, it's
> pretty darn simple to write a wrapper to ip[tables|chains] which
> silently (or not so) drops the option to clear the counters before
> calling the real version.
> 
> Besides, what would be gained in making the counters RO, if they were
> cleared every time the module was loaded/unloaded?

1. Knowlege that the module was reloaded.
2. Knowlege that the data being measured is correct
3. Having reliable measures
4. being able to derive valid statistics
....

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

Jesse Pollard continues with:
> Leif Sawyer <lsawyer@gci.com>:
>>> Ian Stirling [mailto:root@mauve.demon.co.uk]
>>>> Manfred Bartz responded to
>>>>> Russell King <rmk@arm.linux.org.uk> who writes:
>>>>>
>>>>> You just illustrated my point.  While there is a 
>>>>> reset capability people will use it and accounting/
>>>>> logging programs will get wrong data.  Resetable
>>>>> counters might be a minor convenience when debugging
>>>>> but the price is unreliable programs and the loss of the 
>>>>> ability of several programs to use the same counters.
>>>> 
>>>> You of course, are commenting from the fact that your 
>>>> applications are stupid, written poorly, and cannot handle
>>>> 'wrapped' data.  Take MRTG
>>>> <snip>
>>>> Similarly, if my InPackets are at 102345 at one read, and 
>>>> 2345 the next read, and I know that my counter is 32 bits,
>>>> then I know i've wrapped and can do
>>> 
>>> I think the point being made is that if InPackets are at 
>>> 102345 at one read, and 2345 the next, and you know it's
>>> a 32 bit counter, it's completely unreliable to assume that
>>> you have in fact recieved 4294867295 packets, if the counter
>>> can be zeroed. You can say nothing other than at least 2345
>>> packets, at most 2345+n*2^32 have been got since you last
>>> checked.
>> 
>> Ah, yes.. I seem to have misplaced a bit of text in my reply.
>> 
>> The continuation of thought:
>> 
>> How the application derives the status of a wrapped counter or
>> a zero'ed counter is dependant on the device being monitored.
>> 
>> Yes, you have to know what your interface is capable of 
>> (maxbytes/sec) so that you can do a simple calculation where:
>> 
>> maximum_throughput = maxbytes_sec * (time_now - time_last_read)
>> 
>> and if your previous good counter + the maximum throughput wraps
>> the counter, you have a good chance that you've simply wrapped.
>> 
>> If not, then you can assume that your counters were cleared 
>> at some point, log the data you've got, and keep moving forward.
> 
> And that introduces errors in measurement. It also depends on 
> how frequently an uncontroled process is clearing the counters.
> You may never be able to get a valid measurement.

This is true.  Which is why application programmers need to write
code as if they are not the only [ab]users of data.

Which brings me back to my point.

Don't force the kernel to uphold your local application requirements
of stable counters.

Enforce it in the userspace portion of the code.

<subtopic>
Yes, you could extend the proc filesystem (ugh) with a flag that could
be read by the ip[chains|tables] user app to determine if clearing flags
were allowed.  Then a simple

echo 1 > /proc/sys/net/ipv4/counters_locked

or some such cruft.  But I don't see this extension making into the
standard kernel at this time.  It just seems to be wasteful.
</subtopic>

If you (at your site) really need this type of functionality, it's
pretty darn simple to write a wrapper to ip[tables|chains] which
silently (or not so) drops the option to clear the counters before
calling the real version.

Besides, what would be gained in making the counters RO, if they were
cleared every time the module was loaded/unloaded?

RE: IP Acounting Idea for 2.5

[Jesse Pollard]

Leif Sawyer <lsawyer@gci.com>:
> > From: Ian Stirling [mailto:root@mauve.demon.co.uk]
> > > Manfred Bartz responded to
> > > > Russell King <rmk@arm.linux.org.uk> who writes:
> > <snip>
> > > > You just illustrated my point.  While there is a reset capability
> > > > people will use it and accounting/logging programs will get wrong
> > > > data.  Resetable counters might be a minor convenience 
> > when debugging
> > > > but the price is unreliable programs and the loss of the 
> > ability of
> > > > several programs to use the same counters.
> > > 
> > > You of course, are commenting from the fact that your 
> > applications are
> > > stupid, written poorly, and cannot handle 'wrapped' data.  Take MRTG
> > <snip>
> > > Similarly, if my InPackets are at 102345 at one read, and 
> > 2345 the next
> > > read,
> > > and I know that my counter is 32 bits, then I know i've 
> > wrapped and can do
> > 
> > I think the point being made is that if InPackets are at 
> > 102345 at one read,
> > and 2345 the next, and you know it's a 32 bit counter, it's completely
> > unreliable to assume that you have in fact recieved 4294867295
> > packets, if the counter can be zeroed.
> > You can say nothing other than at least 2345 packets, at most 
> > 2345+n*2^32 have been got since you last checked.
> 
> Ah, yes.. I seem to have misplaced a bit of text in my reply.
> 
> The continuation of thought:
> 
> How the application derives the status of a wrapped counter or
> a zero'ed counter is dependant on the device being monitored.
> 
> Yes, you have to know what your interface is capable of (maxbytes/sec)
> so that you can do a simple calculation where:
> 
> maximum_throughput = maxbytes_sec * (time_now - time_last_read)
> 
> and if your previous good counter + the maximum throughput wraps the
> counter, you have a good chance that you've simply wrapped.
> 
> If not, then you can assume that your counters were cleared at some point,
> log the data you've got, and keep moving forward.

And that introduces errors in measurement. It also depends on how frequently
an uncontroled process is clearing the counters. You may never be able to
get a valid measurement.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

> From: Ian Stirling [mailto:root@mauve.demon.co.uk]
> > Manfred Bartz responded to
> > > Russell King <rmk@arm.linux.org.uk> who writes:
> <snip>
> > > You just illustrated my point.  While there is a reset capability
> > > people will use it and accounting/logging programs will get wrong
> > > data.  Resetable counters might be a minor convenience 
> when debugging
> > > but the price is unreliable programs and the loss of the 
> ability of
> > > several programs to use the same counters.
> > 
> > You of course, are commenting from the fact that your 
> applications are
> > stupid, written poorly, and cannot handle 'wrapped' data.  Take MRTG
> <snip>
> > Similarly, if my InPackets are at 102345 at one read, and 
> 2345 the next
> > read,
> > and I know that my counter is 32 bits, then I know i've 
> wrapped and can do
> 
> I think the point being made is that if InPackets are at 
> 102345 at one read,
> and 2345 the next, and you know it's a 32 bit counter, it's completely
> unreliable to assume that you have in fact recieved 4294867295
> packets, if the counter can be zeroed.
> You can say nothing other than at least 2345 packets, at most 
> 2345+n*2^32 have been got since you last checked.

Ah, yes.. I seem to have misplaced a bit of text in my reply.

The continuation of thought:

How the application derives the status of a wrapped counter or
a zero'ed counter is dependant on the device being monitored.

Yes, you have to know what your interface is capable of (maxbytes/sec)
so that you can do a simple calculation where:

maximum_throughput = maxbytes_sec * (time_now - time_last_read)

and if your previous good counter + the maximum throughput wraps the
counter, you have a good chance that you've simply wrapped.

If not, then you can assume that your counters were cleared at some point,
log the data you've got, and keep moving forward.

RE: IP Acounting Idea for 2.5

[Leif Sawyer]

Manfred Bartz responded to
> Russell King <rmk@arm.linux.org.uk> who writes:
> 
> > On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> > > There is another issue with logging in general:
> > > 
> > >                 *COUNTERS MUST NOT BE RESETABLE!!!*
> > 
> > Umm, no.  Counters can be resetable - you just specify that 
> > accounting programs should not reset them, ever.
> > 
> > The ability to reset counters is extremely useful if you're a human
> > looking at the output of iptables -L -v.  (I thus far know of no one
> > who can memorise the counter values for around 40 rules).
> 
> You just illustrated my point.  While there is a reset capability
> people will use it and accounting/logging programs will get wrong
> data.  Resetable counters might be a minor convenience when debugging
> but the price is unreliable programs and the loss of the ability of
> several programs to use the same counters.

You of course, are commenting from the fact that your applications are
stupid, written poorly, and cannot handle 'wrapped' data.  Take MRTG
as one example of a good application, for it can handle counters that
have been cleared between reads.
 
> Also, do you always reset the date and time everytime you make time 
> measurements?

No, but i realize that if my time has incremented, and my date has not
changed, then it's later.  If my time has _decremented_ and my date has not,
then I've time-warped.  If my time has decremented and my date has
incremented,
then it's simply the next-date-increment.

Similarly, if my InPackets are at 102345 at one read, and 2345 the next
read,
and I know that my counter is 32 bits, then I know i've wrapped and can do
my
own math.

Applications need to get smarter.  Don't place arbitrary rules where they
don't
belong.

Re: IP Acounting Idea for 2.5

[Ian Stirling]

> 
> Manfred Bartz responded to
> > Russell King <rmk@arm.linux.org.uk> who writes:
<snip>
> > You just illustrated my point.  While there is a reset capability
> > people will use it and accounting/logging programs will get wrong
> > data.  Resetable counters might be a minor convenience when debugging
> > but the price is unreliable programs and the loss of the ability of
> > several programs to use the same counters.
> 
> You of course, are commenting from the fact that your applications are
> stupid, written poorly, and cannot handle 'wrapped' data.  Take MRTG
<snip>
> Similarly, if my InPackets are at 102345 at one read, and 2345 the next
> read,
> and I know that my counter is 32 bits, then I know i've wrapped and can do

I think the point being made is that if InPackets are at 102345 at one read,
and 2345 the next, and you know it's a 32 bit counter, it's completely
unreliable to assume that you have in fact recieved 4294867295
packets, if the counter can be zeroed.
You can say nothing other than at least 2345 packets, at most 
2345+n*2^32 have been got since you last checked.

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Leif Sawyer <lsawyer@gci.com> writes:

> Manfred Bartz responded to
> > Russell King <rmk@arm.linux.org.uk> who writes:
> > 
> > > On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> > > > There is another issue with logging in general:
> > > > 
> > > >                 *COUNTERS MUST NOT BE RESETABLE!!!*
> > > 
> > > Umm, no.  Counters can be resetable - you just specify that 
> > > accounting programs should not reset them, ever.
> > > 
> > > The ability to reset counters is extremely useful if you're a human
> > > looking at the output of iptables -L -v.  (I thus far know of no one
> > > who can memorise the counter values for around 40 rules).
> > 
> > You just illustrated my point.  While there is a reset capability
> > people will use it and accounting/logging programs will get wrong
> > data.  Resetable counters might be a minor convenience when debugging
> > but the price is unreliable programs and the loss of the ability of
> > several programs to use the same counters.
> 
> You of course, are commenting from the fact that your applications are
> stupid, written poorly, and cannot handle 'wrapped' data.  

Well, apps written by me *can handle* wrapped data; it is actually far
easier that resetting at each read.

> Take MRTG as one example of a good application, for it can handle
> counters that have been cleared between reads.

Regardless of how well a logging application is written, there is no
way that it does not loose data when someone or something resets its
counter.  F.e. if an app reads a counter every 10 minutes and a
counter reset occurs from elsewhere, then anywhere between 0 and 10
minutes worth of data is lost.

I had a brief look at MRTG.  It seems to be a well written app and
while it can handle counter reset (with potential loss of an unknown 
amount of data), it does not actively reset counters.  It also doesn't
use iptables.

> Applications need to get smarter.  

Agreed.

> Don't place arbitrary rules where they don't belong.

Agreed too.  Counters should not arbitrarily be equipped with a reset
capability, there is hardly any benefit in that and it causes nothing
but problems.

As far as I can see, the counters in /proc/net/snmp don't have a
reset, same with /proc/net/dev and possibly other counters elsewhere.

Ideally iptables would fall in line with that.  Rules can still be
unloaded and reloaded, also causing counter reset and loss of data,
but since that is a lot more involved, application authors would have
an incentive to handle counters properly.

-- 
Manfred Bartz

Re: IP Acounting Idea for 2.5

[Olaf Titz]

> Similarly, if my InPackets are at 102345 at one read, and 2345 the
> next read, and I know that my counter is 32 bits, then I know i've
> wrapped and can do my own math.

No. When you have resettable counters, you don't know if the counter
has wrapped or been reset. Either you have received 2345 packets in
between, or 2^32-102345. The difference is not negligible. ;-)

Unless you have a second counter which is incremented with every wrap
(_or_, perhaps better, with every reset) and can _not_ be reset
manually. This would be the date in your date/time example.

Olaf

IP Acounting Idea for 2.5

[David Findlay]

I am using the kernel IP Accounting in Linux to record the amount of data 
transfered via my Linux internet gateway from individual IP addresses. This 
currently requires me to set up an accounting rule for each IP address that I 
want to record accounting info for. If I had 200 machines to individually log 
this would require me to set 200 rules.

In the 2.5 series of kernels, working towards 2.6, could you please make the 
IP Accounting so that I can set a single rule that will make it watch all IP 
traffic going from the local network, through the masquerading service to the 
internet, and log local IP Addresses using it? This would allow me to set 1 
rule, but have the information I want on a per IP address system.

One other person I have talked to would like to see this too, and he 
basically says we need a software version of the Cisco IP Accounting 
server/router.

Could you please add this to the next kernel? Please CC me your responses as 
I am not a member of the kernel mailing list. Thanks,

David

RE: IP Acounting Idea for 2.5

[Michael Clark]

> In the 2.5 series of kernels, working towards 2.6, could you please make the
> IP Accounting so that I can set a single rule that will make it watch all IP
> traffic going from the local network, through the masquerading service to the
> internet, and log local IP Addresses using it? This would allow me to set 1
> rule, but have the information I want on a per IP address system.

You could try using a mature userspace traffic meter like 'NeTraMet' (uses
libpcap).

ftp://ftp.auckland.ac.nz/pub/iawg/NeTraMet/

> One other person I have talked to would like to see this too, and he
> basically says we need a software version of the Cisco IP Accounting
> server/router.

NeTraMet can also account using Cisco Netflow accounting records.

> Could you please add this to the next kernel? Please CC me your responses as
> I am not a member of the kernel mailing list. Thanks,
>
> David
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>

Re: IP Acounting Idea for 2.5

[Mike A. Harris]

On Tue, 17 Apr 2001, David Findlay wrote:

>I am using the kernel IP Accounting in Linux to record the amount of data
>transfered via my Linux internet gateway from individual IP addresses. This
>currently requires me to set up an accounting rule for each IP address that I
>want to record accounting info for. If I had 200 machines to individually log
>this would require me to set 200 rules.
>
>In the 2.5 series of kernels, working towards 2.6, could you please make the
>IP Accounting so that I can set a single rule that will make it watch all IP
>traffic going from the local network, through the masquerading service to the
>internet, and log local IP Addresses using it? This would allow me to set 1
>rule, but have the information I want on a per IP address system.
>
>One other person I have talked to would like to see this too, and he
>basically says we need a software version of the Cisco IP Accounting
>server/router.
>
>Could you please add this to the next kernel? Please CC me your responses as
>I am not a member of the kernel mailing list. Thanks,

Perhaps I misunderstand what it is exactly you are trying to do,
but I would think that this could be done entirely in userland by
software that just adds rules for you instead of you having to do
it manually.

Just a thought.

----------------------------------------------------------------------
    Mike A. Harris  -  Linux advocate  -  Free Software advocate
          This message is copyright 2001, all rights reserved.
  Views expressed are my own, not necessarily shared by my employer.
----------------------------------------------------------------------
There are two major products that come out of Berkeley: LSD and BSD.
We don't believe this to be a coincidence.     -- Jeremy S. Anderson

Re: IP Acounting Idea for 2.5

[David Findlay]

On Monday 16 April 2001 10:40, you wrote:
> Perhaps I misunderstand what it is exactly you are trying to do,
> but I would think that this could be done entirely in userland by
> software that just adds rules for you instead of you having to do
> it manually.

I suppose, but it would be so much easier if the kernel did it automatically. 
Having a rule to go through for each IP address to be logged would be slower 
than implementing one rule that would log all of them. Doing this in the 
kernel would improve preformance.

David

Re: IP Acounting Idea for 2.5

[Mike A. Harris]

On Tue, 17 Apr 2001, David Findlay wrote:

>> Perhaps I misunderstand what it is exactly you are trying to do,
>> but I would think that this could be done entirely in userland by
>> software that just adds rules for you instead of you having to do
>> it manually.
>
>I suppose, but it would be so much easier if the kernel did it automatically.
>Having a rule to go through for each IP address to be logged would be slower
>than implementing one rule that would log all of them. Doing this in the
>kernel would improve preformance.

I don't think it would, but then only benchmarking it both ways
would know for sure.  Even with incredibly large rulesets,
ipchains &&/|| netfilter works admirably well.  Rusty?


----------------------------------------------------------------------
    Mike A. Harris  -  Linux advocate  -  Free Software advocate
          This message is copyright 2001, all rights reserved.
  Views expressed are my own, not necessarily shared by my employer.
----------------------------------------------------------------------

Re: IP Acounting Idea for 2.5

[swds.mlowe]

No, one rule would be MUCH faster. What's do you think would be faster of the two:

if ((ipaddr>=3232235521)&&(ipaddr<=3232235774))
    return 1;
else
    return 0;

or

for (a=0;a<(3232235774-3232235521);a++)
    if (ipaddr==a)
        return 1;
return 0;

Obviously it compares the 192.168.0.1 - 192.168.0.255 range, but I think we both
know which one will be faster. Not to mention countless other redundant checks
will be added in on both, but on the second one the time of the checks is
multiplied by the number of times you have too loop.

But, I'm just a newbie so I don't really know :P Just taking up bandwith :(

L8ers,
Matt

"Mike A. Harris" wrote:

> On Tue, 17 Apr 2001, David Findlay wrote:
>
> >> Perhaps I misunderstand what it is exactly you are trying to do,
> >> but I would think that this could be done entirely in userland by
> >> software that just adds rules for you instead of you having to do
> >> it manually.
> >
> >I suppose, but it would be so much easier if the kernel did it automatically.
> >Having a rule to go through for each IP address to be logged would be slower
> >than implementing one rule that would log all of them. Doing this in the
> >kernel would improve preformance.
>
> I don't think it would, but then only benchmarking it both ways
> would know for sure.  Even with incredibly large rulesets,
> ipchains &&/|| netfilter works admirably well.  Rusty?
>
> ----------------------------------------------------------------------
>     Mike A. Harris  -  Linux advocate  -  Free Software advocate
>           This message is copyright 2001, all rights reserved.
>   Views expressed are my own, not necessarily shared by my employer.
> ----------------------------------------------------------------------
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: IP Acounting Idea for 2.5

[Matti Aarnio]

On Tue, Apr 17, 2001 at 08:46:12AM +1000, David Findlay wrote:
> On Monday 16 April 2001 10:40, you wrote:
> > Perhaps I misunderstand what it is exactly you are trying to do,
> > but I would think that this could be done entirely in userland by
> > software that just adds rules for you instead of you having to do
> > it manually.
> 
> I suppose, but it would be so much easier if the kernel did it automatically. 
> Having a rule to go through for each IP address to be logged would be slower 
> than implementing one rule that would log all of them. Doing this in the 
> kernel would improve preformance.

	Perhaps "netflow" would be an answer ?

	It is cisco idea for collecting accounting data for
	network flows.   System has a cache of flows for
	which it collects data, and once the cache overflows
	(or flow times out), data is sent to designated flow
	collection server(s).

http://www.cisco.com/warp/public/cc/pd/iosw/ioft/neflct/tech/napps_wp.htm

> David

/Matti Aarnio

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

David Findlay <david_j_findlay@yahoo.com.au> writes:

> On Monday 16 April 2001 10:40, you wrote:

> > Perhaps I misunderstand what it is exactly you are trying to do,
> > but I would think that this could be done entirely in userland by
> > software that just adds rules for you instead of you having to do
> > it manually.
> 
> I suppose, but it would be so much easier if the kernel did it
> automatically. 

Yes, but by what criteria?  You want to log on a per host basis.
Someone else wants to log by service (www, ftp, mail, news), or
any number of other criteria.

> Having a rule to go through for each IP address to be logged would
> be slower than implementing one rule that would log all of
> them. Doing this in the kernel would improve preformance.

If there really is a performance issue with a few hundred rules, then
it can be overcome by grouping rules in separate custom chains.  F.e.
if you have 1024 rules create 32 custom chains with 32 rules each.
Then have 32 rules in your main table which jump to the appropriate
custom chain --> maximum rules traversed by each packet = 64.

There is another issue with logging in general:

                *COUNTERS MUST NOT BE RESETABLE!!!*

Resetable counters guarantee that no two programs can co-exists if
they happen to reset the same counters.

All logging counters should be implemented with 32bit or 64bit
unsigned integers.  Any software using correct unsigned integer
arithmetic can then simply subtract a previous value from the current
value to get the difference.  This works reliably across counter
wrap-arounds.  There is absolutely *no need for reset* !

-- 
Manfred Bartz
---------------------------------------------------------------
ipchainsLogAnalyzer, NetCalc, whois at: <http://logi.cc/linux/>
     NEW: <http://logi.cc/linux/NetfilterLogAnalyzer.php3>

Re: IP Acounting Idea for 2.5

[Russell King]

On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> There is another issue with logging in general:
> 
>                 *COUNTERS MUST NOT BE RESETABLE!!!*

Umm, no.  Counters can be resetable - you just specify that accounting
programs should not reset them, ever.

The ability to reset counters is extremely useful if you're a human
looking at the output of iptables -L -v.  (I thus far know of no one
who can memorise the counter values for around 40 rules).

--
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Russell King <rmk@arm.linux.org.uk> writes:

> On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> > There is another issue with logging in general:
> > 
> >                 *COUNTERS MUST NOT BE RESETABLE!!!*
> 
> Umm, no.  Counters can be resetable - you just specify that accounting
> programs should not reset them, ever.
> 
> The ability to reset counters is extremely useful if you're a human
> looking at the output of iptables -L -v.  (I thus far know of no one
> who can memorise the counter values for around 40 rules).

You just illustrated my point.  While there is a reset capability
people will use it and accounting/logging programs will get wrong
data.  Resetable counters might be a minor convenience when debugging
but the price is unreliable programs and the loss of the ability of
several programs to use the same counters.

Also, do you always reset the date and time everytime you make time 
measurements?

-- 
Manfred Bartz

Re: IP Acounting Idea for 2.5

[Olaf Titz]

> Umm, no.  Counters can be resetable - you just specify that accounting
> programs should not reset them, ever.
>
> The ability to reset counters is extremely useful if you're a human
> looking at the output of iptables -L -v.  (I thus far know of no one
> who can memorise the counter values for around 40 rules).

You'll get bogus accounting results unless you stop/restart the
accounting programs every time you manually deal with the counters.
This sounds dangerously easy to make mistakes to me.

Olaf

Re: IP Acounting Idea for 2.5

[Harald Welte]

On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> 
> If there really is a performance issue with a few hundred rules, then
> it can be overcome by grouping rules in separate custom chains.  F.e.
> if you have 1024 rules create 32 custom chains with 32 rules each.
> Then have 32 rules in your main table which jump to the appropriate
> custom chain --> maximum rules traversed by each packet = 64.

still.. if you want to log multiple class B's it's a PITA, especially
if your ruleset changes and you have to atomically replace all chains
in one table to update the ruleset from userspace->kernel.

> There is another issue with logging in general:
> 
>                 *COUNTERS MUST NOT BE RESETABLE!!!*
> 
> Resetable counters guarantee that no two programs can co-exists if
> they happen to reset the same counters.

That sounds like crap (sorry). Counters are resettable, and will be.
If you run two applications resetting counters individually, you have
a problem with your applications.

> All logging counters should be implemented with 32bit or 64bit
> unsigned integers.  Any software using correct unsigned integer

they are uint32 right now.

> arithmetic can then simply subtract a previous value from the current
> value to get the difference.  This works reliably across counter
> wrap-arounds.  There is absolutely *no need for reset* !

so why are your applications resetting counters then? Nobody forces you
to reset them right now.

> Manfred Bartz
> ---------------------------------------------------------------

-- 
Live long and prosper
- Harald Welte / laforge@gnumonks.org                http://www.gnumonks.org
============================================================================
GCS/E/IT d- s-: a-- C+++ UL++++$ P+++ L++++$ E--- W- N++ o? K- w--- O- M- 
V-- PS+ PE-- Y+ PGP++ t++ 5-- !X !R tv-- b+++ DI? !D G+ e* h+ r% y+(*)

Re: IP Acounting Idea for 2.5

[Manfred Bartz]

Harald Welte <laforge@gnumonks.org> writes:

> On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:
> > Resetable counters guarantee that no two programs can co-exists if
> > they happen to reset the same counters.
> 
> That sounds like crap (sorry). 

Care to explain how two independent apps can co-exists if they happen
to reset the same counters?

> so why are your applications resetting counters then? 

Ask the authors, not me.  I wiped those apps from my system.
Its just a pity about the time wasted.

> Nobody forces you to reset them right now.

Why do authors then write apps (f.e. ipac) which reset counters?  
Perhaps only because they can, not because it makes sense.

-- 
Manfred Bartz

Re: IP Acounting Idea for 2.5

[Henning P. Schmiedehausen]

Harald Welte <laforge@gnumonks.org> writes:

>On Mon, Apr 16, 2001 at 12:07:31PM +1000, Manfred Bartz wrote:

>>                 *COUNTERS MUST NOT BE RESETABLE!!!*
>> 
>> Resetable counters guarantee that no two programs can co-exists if
>> they happen to reset the same counters.

>That sounds like crap (sorry). Counters are resettable, and will be.
>If you run two applications resetting counters individually, you have
>a problem with your applications.

Resettable counters in a security sensitive environment are just a
call for trouble. That's why you can't reset the SNMP counters on any
Cisco device I've encountered today. They learned their lesson. Maybe
you will, too.

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: IP Acounting Idea for 2.5

[Harald Welte]

On Tue, Apr 17, 2001 at 06:56:42AM +0000, Henning P. Schmiedehausen wrote:
> 
> Resettable counters in a security sensitive environment are just a
> call for trouble. That's why you can't reset the SNMP counters on any
> Cisco device I've encountered today. They learned their lesson. Maybe
> you will, too.

Well, I'm not sure about which SNTP counters you are talking, but I suppose
it is not about per-filtering-rule counters, but something like per-interface
counters, etc.

There's always a way for somebody with root access to reset the counters of
a rule: 

just delete and re-insert the rule.

If somebody wants to reset the counter, he can. If we remove the functionality
from iptables, people still can - but it's more difficult.

> 	Regards
> 		Henning

-- 
Live long and prosper
- Harald Welte / laforge@gnumonks.org                http://www.gnumonks.org
============================================================================
GCS/E/IT d- s-: a-- C+++ UL++++$ P+++ L++++$ E--- W- N++ o? K- w--- O- M- 
V-- PS+ PE-- Y+ PGP++ t++ 5-- !X !R tv-- b+++ DI? !D G+ e* h+ r% y+(*)

Re: IP Acounting Idea for 2.5

[Henning P. Schmiedehausen]

Harald Welte <laforge@gnumonks.org> writes:

>On Tue, Apr 17, 2001 at 06:56:42AM +0000, Henning P. Schmiedehausen wrote:
>> 
>> Resettable counters in a security sensitive environment are just a
>> call for trouble. That's why you can't reset the SNMP counters on any
>> Cisco device I've encountered today. They learned their lesson. Maybe
>> you will, too.

>Well, I'm not sure about which SNTP counters you are talking, but I suppose
>it is not about per-filtering-rule counters, but something like per-interface
>counters, etc.

You don't want your counters going backward. Full stop. If a program
can reset your counter, your application will never know, if it was a
legal, correct, valid reason or just a hacker trying to hide his
traces. At least provide some sort of lock-down.

>There's always a way for somebody with root access to reset the counters of
>a rule: 

>just delete and re-insert the rule.

Bad thing. If you want to use the rules in a security sensitive
environment, don't allow removal. If you need to, reset the whole
module and notify the user. Better, shut down the filter and yell for
help.

ipfilter is about security, isn't it?

>If somebody wants to reset the counter, he can. If we remove the functionality
>from iptables, people still can - but it's more difficult.

There is no "more difficult", just "different ways". If you bother to
use a filtering environment where the filter counters tell e.g. "we
rejected xxx attacks", you don't want anyone to mess with these
counters. If this is a counter for a filtering rule, don't allow the
rule (and its counters) to be removed. Inactivate the rule but keep
the evidence (counter settings) around till module removal or reboot.

Sorry, I may be anal about security but as more and more people start
thinking "why should I bother with FW-1 or PIX when I can get a $99
Linux box and hack some filters", at least I want the $99 software
trying not to be sloppy about security.


 	Regards
 		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: IP Acounting Idea for 2.5

[Andreas Ferber]

Hi,

On Tue, Apr 17, 2001 at 08:46:12AM +1000, David Findlay wrote:
> 
> I suppose, but it would be so much easier if the kernel did it automatically. 
> Having a rule to go through for each IP address to be logged would be slower 
> than implementing one rule that would log all of them. Doing this in the 
> kernel would improve preformance.

You can use the iptables ULOG facility (see netfilter CVS for the
patches). With this you can copy packets to userspace programs which
can do any further processing, like the accounting stuff. You can even
choose to only copy part of a packet (say only the IP header) to
userspace, to reduce the amount of data your application has to
handle.

Simply copy the headers of all IP packets going through your router to
userspace, and write a small application (possibly using libipulog
which comes with the netfilter userspace code) which does the actual
accounting. This has the additional benefit that further processing of
the packet in kernel (i.e. outputting it on the destination interface)
can continue while your application is processing the accounting data.

Andreas
-- 
You are in a maze of little twisting passages, all alike.

Re: IP Acounting Idea for 2.5

[Dax Kelson]

David Findlay said once upon a time (Tue, 17 Apr 2001):

> I am using the kernel IP Accounting in Linux to record the amount of data
> transfered via my Linux internet gateway from individual IP addresses. This
> currently requires me to set up an accounting rule for each IP address that I
> want to record accounting info for. If I had 200 machines to individually log
> this would require me to set 200 rules.

In 1999 I setup a Linux kernel 2.2 box to monitor traffic for about 2500
hosts.  I created two rules for each host (to monitor inbound and
outbound).  I had over 5000 total rules on a Pentium II 300 Mhz box.  I
wrote some perl glue so I could accounts on a per customer basis (each
customer had one or more IPs, not necessarily contiguous) and graph the
results in MRTG.

There was ZERO performance problems on the box.  The box is still
runningly happily today.  I don't there is a "problem" to fix.

Dax

Re: IP Acounting Idea for 2.5

[Harald Welte]

On Tue, Apr 17, 2001 at 07:53:28AM +1000, David Findlay wrote:

> In the 2.5 series of kernels, working towards 2.6, could you please make the 
> IP Accounting so that I can set a single rule that will make it watch all IP 
> traffic going from the local network, through the masquerading service to the 
> internet, and log local IP Addresses using it? This would allow me to set 1 
> rule, but have the information I want on a per IP address system.

:) Well... there was a discussion about this on the netfilter-devel
list some weeks ago. 

This is definitely not a 2.5 / 2.6 issue, as it could be easily implemented
by using the already existing flexibility of netfilter/iptables.

It's just a matter of somebody who needs it getting around actually 
implementing it. Could be either implemented as a 'raw' netfilter 
hook-attaching module or (more convenient) as a new iptables target, 
which allocates some internal tables for storing the accounting data.

Doesn't make sense to have a discussion about it on linux-kernel, and
it certainly doesn't belong into a 2.5 featurelist.

Please refer to the netfilter-devel mailinglist (subscription information
available at http://netfilter.samba.org/)

> David

-- 
Live long and prosper
- Harald Welte / laforge@gnumonks.org                http://www.gnumonks.org
============================================================================
GCS/E/IT d- s-: a-- C+++ UL++++$ P+++ L++++$ E--- W- N++ o? K- w--- O- M- 
V-- PS+ PE-- Y+ PGP++ t++ 5-- !X !R tv-- b+++ DI? !D G+ e* h+ r% y+(*)