Re: NAPI, e100, and system performance problem

[jamal <hadi@cyberus.ca>]

On Fri, 2005-22-04 at 13:36 +0200, Andi Kleen wrote:
> Greg Banks <gnb@sgi.com> writes:
> >
> > An inordinate amount of CPU is being spent running around polling the
> > device instead of dealing with the packets in IP, TCP and NFS land.
> > By inordinate, we mean twice as much or more cpu% than a MIPS/Irix
> > box with slower CPUs.
> 
> We have seen similar behaviour. With NAPI some benchmarks run
> a lot slower than on a driver on the same hardware/NIC without NAPI.

They should not run slower - but they may consume more CPU.

> This can be even observed with simple tests like netperf single stream
> between two boxes.
> 

Yes, slow traffic coming into the system would chew more CPU if you have
a fast CPU ;-> You should know this Andi, but let me explain the reason
for about the 100th time:
If your CPU is fast enough such that the interupt only causes 1-2
packets to be processed before reenabling, then you will have anywhere
between 1-2 extra IO calls (recall dis/reenabling interupts) - this will
be noticed in the CPU % chewed. It shouldnt be really a lot of CPU.
If you use non-NAPI then you dont have the extra IO happening and
therefore you will see less CPU used at those rates.
OTOH, Try more than 1 netperfs and it becomes beneficial to have NAPI
around. 
Yes, for low rates - this will make some of those benchmarks look bad.
But for such cases you can use mitigation in addition to NAPi - except
your latency in the benchmark will now look bad ;->

> There seems to be also some problems with bidirectional traffic, although
> I have not fully tracked them down to NAPI yet.
> 

doubt it has to do with NAPI.

> There is definitely some problem in NAPI land ...
> 

this is a design choice - a solution could be created to "fix" this but
hasnt happened because there has not been a good reason to complicate
things. The people who are bitching about this are benchmarkers who want
to win at both high and low rates and they are not happy because while
they can win at high rates, they cant at low rates.

cheers,
jamal