All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Michael S. Tsirkin" <mst@redhat.com>
To: Jason Wang <jasowang@redhat.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	virtualization@lists.linux-foundation.org, edumazet@google.com,
	kuba@kernel.org, pabeni@redhat.com, davem@davemloft.net
Subject: Re: [PATCH net V2] virtio-net: correctly enable callback during start_xmit
Date: Thu, 15 Dec 2022 04:02:49 -0500	[thread overview]
Message-ID: <20221215034740-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <20221215032719.72294-1-jasowang@redhat.com>

On Thu, Dec 15, 2022 at 11:27:19AM +0800, Jason Wang wrote:
> Commit a7766ef18b33("virtio_net: disable cb aggressively") enables
> virtqueue callback via the following statement:
> 
>         do {
>            ......
> 	} while (use_napi && kick &&
>                unlikely(!virtqueue_enable_cb_delayed(sq->vq)));
> 
> When NAPI is used and kick is false, the callback won't be enabled
> here. And when the virtqueue is about to be full, the tx will be
> disabled, but we still don't enable tx interrupt which will cause a TX
> hang. This could be observed when using pktgen with burst enabled.
> 
> Fixing this by trying to enable tx interrupt after we disable TX when
> we're not using napi or kick is false.
> 
> Fixes: a7766ef18b33 ("virtio_net: disable cb aggressively")
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> The patch is needed for -stable.
> Changes since V1:
> - enable tx interrupt after we disable tx
> ---
>  drivers/net/virtio_net.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 86e52454b5b5..dcf3a536d78a 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -1873,7 +1873,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>  	 */
>  	if (sq->vq->num_free < 2+MAX_SKB_FRAGS) {
>  		netif_stop_subqueue(dev, qnum);
> -		if (!use_napi &&
> +		if ((!use_napi || !kick) &&
>  		    unlikely(!virtqueue_enable_cb_delayed(sq->vq))) {
>  			/* More just got used, free them then recheck. */
>  			free_old_xmit_skbs(sq, false);

This will work but the following lines are:

                       if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) {
                                netif_start_subqueue(dev, qnum);
                                virtqueue_disable_cb(sq->vq);
                        }


and I thought we are supposed to keep callbacks enabled with napi?
One of the ideas of napi is to free on napi callback, not here
immediately.

I think it is easier to just do a separate branch here. Along the
lines of:

		if (use_napi) {
			if (unlikely(!virtqueue_enable_cb_delayed(sq->vq)))
				virtqueue_napi_schedule(napi, vq);
		} else {
			... old code ...
		}

also reduces chances of regressions on !napi (which is not well tested)
and keeps callbacks off while we free skbs.

No?


> -- 
> 2.25.1

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

WARNING: multiple messages have this Message-ID (diff)
From: "Michael S. Tsirkin" <mst@redhat.com>
To: Jason Wang <jasowang@redhat.com>
Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
	pabeni@redhat.com, virtualization@lists.linux-foundation.org,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	xuanzhuo@linux.alibaba.com
Subject: Re: [PATCH net V2] virtio-net: correctly enable callback during start_xmit
Date: Thu, 15 Dec 2022 04:02:49 -0500	[thread overview]
Message-ID: <20221215034740-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <20221215032719.72294-1-jasowang@redhat.com>

On Thu, Dec 15, 2022 at 11:27:19AM +0800, Jason Wang wrote:
> Commit a7766ef18b33("virtio_net: disable cb aggressively") enables
> virtqueue callback via the following statement:
> 
>         do {
>            ......
> 	} while (use_napi && kick &&
>                unlikely(!virtqueue_enable_cb_delayed(sq->vq)));
> 
> When NAPI is used and kick is false, the callback won't be enabled
> here. And when the virtqueue is about to be full, the tx will be
> disabled, but we still don't enable tx interrupt which will cause a TX
> hang. This could be observed when using pktgen with burst enabled.
> 
> Fixing this by trying to enable tx interrupt after we disable TX when
> we're not using napi or kick is false.
> 
> Fixes: a7766ef18b33 ("virtio_net: disable cb aggressively")
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> The patch is needed for -stable.
> Changes since V1:
> - enable tx interrupt after we disable tx
> ---
>  drivers/net/virtio_net.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 86e52454b5b5..dcf3a536d78a 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -1873,7 +1873,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
>  	 */
>  	if (sq->vq->num_free < 2+MAX_SKB_FRAGS) {
>  		netif_stop_subqueue(dev, qnum);
> -		if (!use_napi &&
> +		if ((!use_napi || !kick) &&
>  		    unlikely(!virtqueue_enable_cb_delayed(sq->vq))) {
>  			/* More just got used, free them then recheck. */
>  			free_old_xmit_skbs(sq, false);

This will work but the following lines are:

                       if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) {
                                netif_start_subqueue(dev, qnum);
                                virtqueue_disable_cb(sq->vq);
                        }


and I thought we are supposed to keep callbacks enabled with napi?
One of the ideas of napi is to free on napi callback, not here
immediately.

I think it is easier to just do a separate branch here. Along the
lines of:

		if (use_napi) {
			if (unlikely(!virtqueue_enable_cb_delayed(sq->vq)))
				virtqueue_napi_schedule(napi, vq);
		} else {
			... old code ...
		}

also reduces chances of regressions on !napi (which is not well tested)
and keeps callbacks off while we free skbs.

No?


> -- 
> 2.25.1


  reply	other threads:[~2022-12-15  9:03 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-12-15  3:27 [PATCH net V2] virtio-net: correctly enable callback during start_xmit Jason Wang
2022-12-15  3:27 ` Jason Wang
2022-12-15  9:02 ` Michael S. Tsirkin [this message]
2022-12-15  9:02   ` Michael S. Tsirkin
2022-12-15  9:15   ` Jason Wang
2022-12-15  9:15     ` Jason Wang
2022-12-15  9:34     ` Michael S. Tsirkin
2022-12-15  9:34       ` Michael S. Tsirkin
2022-12-16  3:43       ` Jason Wang
2022-12-16  3:43         ` Jason Wang
2022-12-23  6:29         ` Jason Wang
2022-12-23  6:29           ` Jason Wang
2023-01-04  4:23           ` Jason Wang
2023-01-04  4:23             ` Jason Wang
2023-01-04  6:46             ` Michael S. Tsirkin
2023-01-04  6:46               ` Michael S. Tsirkin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221215034740-mutt-send-email-mst@kernel.org \
    --to=mst@redhat.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=jasowang@redhat.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.