[PATCH] SCHED: Fix unnecesary driver entries when queue is stopped

[PATCH] SCHED: Fix unnecesary driver entries when queue is stopped

[PJ Waskiewicz]

Dave,

This patch addresses an issue with multiqueue devices and non-multiqueue
qdiscs which is causing performance issues.  This patch should be 
considered for both 2.6.23-stable and 2.6.24 upstream.  Basically, if
a driver is using the netif_*_subqueue() calls, then qdisc_restart() will
happily call hard_start_xmit() even if subqueue 0 is stopped, which is bad.
This re-adds the check for the subqueue state.

Note that this check was removed when qdisc_restart() was rewritten.  At that
time though, we didn't understand the full effect of multiqueue with respect
to the qdiscs and queue management from a driver to kernel perspective.  Since
the driver doesn't know what qdisc capabilities live above it, it needs to
decide to use the queue or subqueue functions ahead of time.  This patch is
just cleaning up a miss from that rewrite.

Patch 1 is for 2.6.24, patch 2 is for 2.6.23 stable.

Thanks,
-- 
PJ Waskiewicz <peter.p.waskiewicz.jr@intel.com>

[PATCH] PATCH 1/2 [SCHED 2.6.24]: Check subqueue status before calling hard_start_xmit

[PJ Waskiewicz]

The only qdiscs that check subqueue state before dequeue'ing are PRIO
and RR.  The other qdiscs, including the default pfifo_fast qdisc, will
allow traffic bound for subqueue 0 through to hard_start_xmit.  The check
for netif_queue_stopped() is done above in pkt_sched.h, so it is
unnecessary for qdisc_restart().  However, if the underlying driver is
multiqueue capable, and only sets queue states on subqueues, this will
allow packets to enter the driver when it's currently unable to process
packets, resulting in expensive requeues and driver entries.  This patch
re-adds the check for the subqueue status before calling hard_start_xmit,
so we can try and avoid the driver entry when the queues are stopped.

Signed-off-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
---

 net/sched/sch_generic.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index fa1a6f4..e595e65 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -134,7 +134,7 @@ static inline int qdisc_restart(struct net_device *dev)
 {
 	struct Qdisc *q = dev->qdisc;
 	struct sk_buff *skb;
-	int ret;
+	int ret = NETDEV_TX_BUSY;
 
 	/* Dequeue packet */
 	if (unlikely((skb = dev_dequeue_skb(dev, q)) == NULL))
@@ -145,7 +145,8 @@ static inline int qdisc_restart(struct net_device *dev)
 	spin_unlock(&dev->queue_lock);
 
 	HARD_TX_LOCK(dev, smp_processor_id());
-	ret = dev_hard_start_xmit(skb, dev);
+	if (!netif_subqueue_stopped(dev, skb))
+		ret = dev_hard_start_xmit(skb, dev);
 	HARD_TX_UNLOCK(dev);
 
 	spin_lock(&dev->queue_lock);

Re: [PATCH] PATCH 1/2 [SCHED 2.6.24]: Check subqueue status before calling hard_start_xmit

[David Miller]

From: PJ Waskiewicz <peter.p.waskiewicz.jr@intel.com>
Date: Tue, 13 Nov 2007 09:44:50 -0800

> The only qdiscs that check subqueue state before dequeue'ing are PRIO
> and RR.  The other qdiscs, including the default pfifo_fast qdisc, will
> allow traffic bound for subqueue 0 through to hard_start_xmit.  The check
> for netif_queue_stopped() is done above in pkt_sched.h, so it is
> unnecessary for qdisc_restart().  However, if the underlying driver is
> multiqueue capable, and only sets queue states on subqueues, this will
> allow packets to enter the driver when it's currently unable to process
> packets, resulting in expensive requeues and driver entries.  This patch
> re-adds the check for the subqueue status before calling hard_start_xmit,
> so we can try and avoid the driver entry when the queues are stopped.
> 
> Signed-off-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>

Applied, and I'll queue up the other one for -stable, thanks.