Re: [PATCH 3/16] Spidernet RX Locking

[linas@austin.ibm.com (Linas Vepstas)]

On Sat, Dec 09, 2006 at 09:47:05AM +1100, Benjamin Herrenschmidt wrote:
> A spinlock is expensive in the fast path, which is why Jeff says it's
> invasive.
> 
> > spider_net_decode_one_descr() is called from
> > spider_net_poll() (which is the netdev->poll callback)
> > and also from spider_net_handle_rxram_full(). 
> > 
> > The rxramfull routine is called from a tasklet that
> > is fired off after a "RX ram full" interrupt is receved.
> > This interrupt is generated when the hardware runs out
> > of space to store incoming packets. We are seeing this
> > interrupt fire when the CPU is heavily loaded, and a
> > lot of traffic is being fired at the device.
> 
> How often does that interrupt happen in that case ?

It is hard to reproduce; it is highly dependent on kernel version
and network config. It seems to occur when the system is somehow
loaded, and the tcp stack is unable to empty out the rx ring in a
timely manner. Jim is able o trigger this trivially for some kernels, 
but not others.

> A better approach is to keep the fast path (ie. poll()) lockless, and in
> handle_rxram_full(), the slow path, protect against poll using
> netif_disable_poll(). Though that means using a work queue, not a
> tasklet, since it needs to schedule.

Yes. Actually, I am thinking of treating this interrupt as if it were
just another RX interrupt. What the original drivers seemed to want to
do was to treat this as some sort of "high priority" rx interrupt, but
there doesn't seem to be any real way of doing this, so it seems simpler
just to rip out the tasklet and leave it at that.

> or you can schedule rx work from the rxramfull interrupt after setting a
> "something bad happened" flag. Then, poll can check this flag and do the
> right thing.

Yes, exactly.

--linas
>