[PATCH net-next] xen-netback: Schedule NAPI from dealloc thread instead of callback

[PATCH net-next] xen-netback: Schedule NAPI from dealloc thread instead of callback

[Zoltan Kiss]

If there are unconsumed requests in the ring, but there isn't enough free
pending slots, the NAPI instance deschedule itself. As the frontend won't send
any more interrupts in this case, it is the task of whoever release the pending
slots to schedule the NAPI instance in this case. Originally it was done in the
callback, but it's better at the end of the dealloc thread, otherwise there is
a risk that the NAPI instance just deschedule itself as the dealloc thread
couldn't release any used slot yet. However, as there are a lot of pending
packets, NAPI will be scheduled again, and it is very unlikely that the dealloc
thread can't release enough slots in the meantime.

Signed-off-by: Zoltan Kiss <zoltan.kiss@citrix.com>
---

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index eae9724..07c9677 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -1516,13 +1516,6 @@ void xenvif_zerocopy_callback(struct ubuf_info *ubuf, bool zerocopy_success)
 	wake_up(&vif->dealloc_wq);
 	spin_unlock_irqrestore(&vif->callback_lock, flags);
 
-	if (RING_HAS_UNCONSUMED_REQUESTS(&vif->tx) &&
-	    xenvif_tx_pending_slots_available(vif)) {
-		local_bh_disable();
-		napi_schedule(&vif->napi);
-		local_bh_enable();
-	}
-
 	if (likely(zerocopy_success))
 		vif->tx_zerocopy_success++;
 	else
@@ -1594,6 +1587,13 @@ static inline void xenvif_tx_dealloc_action(struct xenvif *vif)
 	for (i = 0; i < gop - vif->tx_unmap_ops; ++i)
 		xenvif_idx_release(vif, pending_idx_release[i],
 				   XEN_NETIF_RSP_OKAY);
+
+	if (RING_HAS_UNCONSUMED_REQUESTS(&vif->tx) &&
+	    xenvif_tx_pending_slots_available(vif)) {
+		local_bh_disable();
+		napi_schedule(&vif->napi);
+		local_bh_enable();
+	}
 }
 
 

Re: [PATCH net-next] xen-netback: Schedule NAPI from dealloc thread instead of callback

[Wei Liu]

On Wed, Mar 12, 2014 at 09:04:41PM +0000, Zoltan Kiss wrote:
> If there are unconsumed requests in the ring, but there isn't enough free
> pending slots, the NAPI instance deschedule itself. As the frontend won't send
> any more interrupts in this case, it is the task of whoever release the pending
> slots to schedule the NAPI instance in this case. Originally it was done in the
> callback, but it's better at the end of the dealloc thread, otherwise there is
> a risk that the NAPI instance just deschedule itself as the dealloc thread
> couldn't release any used slot yet. However, as there are a lot of pending
> packets, NAPI will be scheduled again, and it is very unlikely that the dealloc
> thread can't release enough slots in the meantime.
> 

So this patch restores the property that "only two parties access the
ring", right?

Wei.

Re: [PATCH net-next] xen-netback: Schedule NAPI from dealloc thread instead of callback

[Ian Campbell]

On Thu, 2014-03-13 at 10:17 +0000, Wei Liu wrote:
> On Wed, Mar 12, 2014 at 09:04:41PM +0000, Zoltan Kiss wrote:
> > If there are unconsumed requests in the ring, but there isn't enough free
> > pending slots, the NAPI instance deschedule itself. As the frontend won't send
> > any more interrupts in this case, it is the task of whoever release the pending
> > slots to schedule the NAPI instance in this case. Originally it was done in the
> > callback, but it's better at the end of the dealloc thread, otherwise there is
> > a risk that the NAPI instance just deschedule itself as the dealloc thread
> > couldn't release any used slot yet. However, as there are a lot of pending
> > packets, NAPI will be scheduled again, and it is very unlikely that the dealloc
> > thread can't release enough slots in the meantime.
> > 
> 
> So this patch restores the property that "only two parties access the
> ring", right?

I think so, and therefore:
Acked-by: Ian Campbell <ian.campbell@citrix.com>

Ian.

Re: [PATCH net-next] xen-netback: Schedule NAPI from dealloc thread instead of callback

[Zoltan Kiss]

On 13/03/14 10:42, Ian Campbell wrote:
> On Thu, 2014-03-13 at 10:17 +0000, Wei Liu wrote:
>> On Wed, Mar 12, 2014 at 09:04:41PM +0000, Zoltan Kiss wrote:
>>> If there are unconsumed requests in the ring, but there isn't enough free
>>> pending slots, the NAPI instance deschedule itself. As the frontend won't send
>>> any more interrupts in this case, it is the task of whoever release the pending
>>> slots to schedule the NAPI instance in this case. Originally it was done in the
>>> callback, but it's better at the end of the dealloc thread, otherwise there is
>>> a risk that the NAPI instance just deschedule itself as the dealloc thread
>>> couldn't release any used slot yet. However, as there are a lot of pending
>>> packets, NAPI will be scheduled again, and it is very unlikely that the dealloc
>>> thread can't release enough slots in the meantime.
>>>
>>
>> So this patch restores the property that "only two parties access the
>> ring", right?
>
> I think so, and therefore:
> Acked-by: Ian Campbell <ian.campbell@citrix.com>

I've just discussed this with Ian, this solution still doesn't solve the 
racing between NAPI and the dealloc thread, however it only causes some 
unnecessary napi_schedule's, not ring stalls.
There is a better solution, I'll post it as soon as it passes all the test!

Zoli

Re: [PATCH net-next] xen-netback: Schedule NAPI from dealloc thread instead of callback

[David Miller]

From: Zoltan Kiss <zoltan.kiss@citrix.com>
Date: Thu, 13 Mar 2014 15:44:37 +0000

> I've just discussed this with Ian, this solution still doesn't solve
> the racing between NAPI and the dealloc thread, however it only causes
> some unnecessary napi_schedule's, not ring stalls.
> There is a better solution, I'll post it as soon as it passes all the
> test!

Ok, I'm skipping this patch therefore.