Why are network counters only updated once a second?

Why are network counters only updated once a second?

[Mark Seger]

I had mentioned this in my previous post but perhaps it might get more 
attention all by itself.  I can't say for sure when this changed, but 
for the longest time network counters were only updated once every 
0.9765 seconds and unless you used a tools like collectl that could 
monitor at fractional intervals, your traffic was under-reported AND 
you'd get periodic spikes of double the actual rate.  See 
http://collectl.sourceforge.net/NetworkStats.html for a more complete 
explanation.

Eventually the frequency became better aligned at a 1 second interval 
because now the number look better, but the problem I see is that when 
the sampling interval is very close to the monitoring interval you still 
get periodic incorrect data.  Furthermore, you now need to know which 
way the counters are updated before you pick a sampling interval!  But 
the real point is if anyone ever wants to do finer grained monitoring, 
say every 1/2 or even tenth of a second, they can't because the counters 
won't change between samples.  Has this ever been discussed before?

-mark

Re: Why are network counters only updated once a second?

[Eric Dumazet]

Mark Seger a écrit :
> I had mentioned this in my previous post but perhaps it might get more 
> attention all by itself.  I can't say for sure when this changed, but 
> for the longest time network counters were only updated once every 
> 0.9765 seconds and unless you used a tools like collectl that could 
> monitor at fractional intervals, your traffic was under-reported AND 
> you'd get periodic spikes of double the actual rate.  See 
> http://collectl.sourceforge.net/NetworkStats.html for a more complete 
> explanation.
> 
> Eventually the frequency became better aligned at a 1 second interval 
> because now the number look better, but the problem I see is that when 
> the sampling interval is very close to the monitoring interval you still 
> get periodic incorrect data.  Furthermore, you now need to know which 
> way the counters are updated before you pick a sampling interval!  But 
> the real point is if anyone ever wants to do finer grained monitoring, 
> say every 1/2 or even tenth of a second, they can't because the counters 
> won't change between samples.  Has this ever been discussed before?
> 

Yes it was discussed before. Some devices perform counters updates directly at 
the NIC level, and one in a while a transfert of counters is done to the host.

This is supposed to be better, especially on SMP.

Maybe you need to setup accounting rules with iptables, so that you can 
perform counter sampling at whatever rate you want ?

Re: Why are network counters only updated once a second?

[Mark Seger]

Eric Dumazet wrote:
> Mark Seger a écrit :
>> I had mentioned this in my previous post but perhaps it might get 
>> more attention all by itself.  I can't say for sure when this 
>> changed, but for the longest time network counters were only updated 
>> once every 0.9765 seconds and unless you used a tools like collectl 
>> that could monitor at fractional intervals, your traffic was 
>> under-reported AND you'd get periodic spikes of double the actual 
>> rate.  See http://collectl.sourceforge.net/NetworkStats.html for a 
>> more complete explanation.
>>
>> Eventually the frequency became better aligned at a 1 second interval 
>> because now the number look better, but the problem I see is that 
>> when the sampling interval is very close to the monitoring interval 
>> you still get periodic incorrect data.  Furthermore, you now need to 
>> know which way the counters are updated before you pick a sampling 
>> interval!  But the real point is if anyone ever wants to do finer 
>> grained monitoring, say every 1/2 or even tenth of a second, they 
>> can't because the counters won't change between samples.  Has this 
>> ever been discussed before?
>>
>
> Yes it was discussed before. Some devices perform counters updates 
> directly at the NIC level, and one in a while a transfert of counters 
> is done to the host.
>
> This is supposed to be better, especially on SMP.
>
> Maybe you need to setup accounting rules with iptables, so that you 
> can perform counter sampling at whatever rate you want ?
Maybe I wasn't clear enough.  I'm grabbing the counters from 
/proc/net/dev and whatever mechanism is being used only ports them with 
a granularity of about once a second.  This means any of the standard 
tools that use /proc to get their data will all have the same problem.
-mark

Re: Why are network counters only updated once a second?

[Michael Chan]

On Mon, 2008-01-14 at 15:30 -0500, Mark Seger wrote:

> Eventually the frequency became better aligned at a 1 second interval 
> because now the number look better, but the problem I see is that when 
> the sampling interval is very close to the monitoring interval you still 
> get periodic incorrect data.  Furthermore, you now need to know which 
> way the counters are updated before you pick a sampling interval!  But 
> the real point is if anyone ever wants to do finer grained monitoring, 
> say every 1/2 or even tenth of a second, they can't because the counters 
> won't change between samples.  Has this ever been discussed before?

On most Broadcom NICs, the statistics counters are periodically DMA'ed
from the chip and the default interval is roughly 1 second.  On most of
these chips, you can override this interval using ethtool -C eth0.

As I mentioned earlier, the 5706/5708 has a bug in the statistics DMA
engine that can corrupt counters from time to time.  The workaround is
effective, but you now lose the ability to control the DMA interval.