proposed optimization for network drivers

proposed optimization for network drivers

[Don Cohen]

I notice that netif_rx (net/core/dev.c) discards packets when it's
in throttle mode.  However, before it does this the driver has already
wasted a lot of time allocating the skb, calling eth_type_trans, etc.
That effort could be saved if the driver checked throttle first.


Sample diff:

--- /usr/src/linux-2.4.18-clean/drivers/net/8139too.c   Mon Feb 25 11:37:59 2002
+++ /usr/src/linux-2.4.18/drivers/net/8139too.c Wed Apr 16 23:30:30 2003
@@ -1908,6 +1908,11 @@
                        return;
                }

+               if (softnet_data[smp_processor_id()].throttle) {
+                 dev->last_rx = jiffies;
+                 netdev_rx_stat[smp_processor_id()].dropped++;
+               } else {
+
                /* Malloc up new buffer, compatible with net-2e. */
                /* Omit the four octet CRC from the length. */

@@ -1936,6 +1941,7 @@
                                dev->name);
                        tp->stats.rx_dropped++;
                }
+               }

                cur_rx = (cur_rx + rx_size + 4 + 3) & ~3;
                RTL_W16 (RxBufPtr, cur_rx - 16);

Hmm, now that I notice, there is one small change in semantics here.
Before every packet that was counted by netdev_rx_stat[].dropped had
also been counted in stats.rx_bytes and rx_packets and this is no
longer the case.  If that bothers you then perhaps there should just
be a separate counter for packets dropped like this.

In my limited experimentation this sort of change has significantly
increased the number of packets I could process when they were
arriving too fast. 

Please cc me on replies.

Re: proposed optimization for network drivers

[David S. Miller]

What is we change the congestion implementation?  Then we'll
have to edit every single driver.  I don't think that's very
maintainable.

The whole idea is to abstract things out as far as possible so that
the device drivers are totally agnostic about the details of the
generic network queueing implementation.

I mean, it's an interesting idea, but it exposes details that
should not be exposed.

Re: proposed optimization for network drivers

[Don Cohen]

In part I agree.  I would have preferred to make my change in one
place instead of one driver at a time.  On the other hand, it seems to
me that some of these details are already spread around all the
drivers.  For instance, why does every driver have to call
eth_type_trans?  Could that be delayed for netif_rx ?

I do think it's reasonable for a driver to test whether the upper
layers are ready to process another packet.  I suggest that this 
test be encapsulated into a new function that can be changed at the
cost of only recompiling all the drivers.

David S. Miller writes:
 > 
 > What is we change the congestion implementation?  Then we'll
 > have to edit every single driver.  I don't think that's very
 > maintainable.
 > 
 > The whole idea is to abstract things out as far as possible so that
 > the device drivers are totally agnostic about the details of the
 > generic network queueing implementation.
 > 
 > I mean, it's an interesting idea, but it exposes details that
 > should not be exposed.

Re: proposed optimization for network drivers

[David S. Miller]

   From: don-linux@isis.cs3-inc.com (Don Cohen)
   Date: Fri, 18 Apr 2003 08:37:01 -0700

   For instance, why does every driver have to call
   eth_type_trans?  Could that be delayed for netif_rx ?

Silly example.  Ethernet specific routines do not belong in
device generic netif_rx().

Re: proposed optimization for network drivers

[Don Cohen]

David S. Miller writes:
 >    From: don-linux@isis.cs3-inc.com (Don Cohen)
 >    Date: Fri, 18 Apr 2003 08:37:01 -0700
 > 
 >    For instance, why does every driver have to call
 >    eth_type_trans?  Could that be delayed for netif_rx ?
 > 
 > Silly example.  Ethernet specific routines do not belong in
 > device generic netif_rx().

Ok, fair enough.
How about the larger point that it's reasonable to check early whether
the labor you're about to expend will be wasted and the suggestion
that this be encapsulated in a function that all drivers can share?

Re: proposed optimization for network drivers

[Jeff Garzik]

Don Cohen wrote:
> In part I agree.  I would have preferred to make my change in one
> place instead of one driver at a time.  On the other hand, it seems to
> me that some of these details are already spread around all the
> drivers.  For instance, why does every driver have to call
> eth_type_trans?  Could that be delayed for netif_rx ?
> 
> I do think it's reasonable for a driver to test whether the upper
> layers are ready to process another packet.  I suggest that this 
> test be encapsulated into a new function that can be changed at the
> cost of only recompiling all the drivers.


Why not NAPI?  That is the existing mechanism provided to let the upper 
layer feedback to the low-level driver system congestion information.

	Jeff