* Re: [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg fails
[not found] <1608881073-19004-1-git-send-email-wangyunjian@huawei.com>
@ 2020-12-27 11:09 ` Michael S. Tsirkin
2020-12-27 11:20 ` Michael S. Tsirkin
1 sibling, 0 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2020-12-27 11:09 UTC (permalink / raw)
To: wangyunjian
Cc: willemdebruijn.kernel, netdev, jerry.lilijun, virtualization,
xudingke, brian.huangbin, chenchanghu
On Fri, Dec 25, 2020 at 03:24:33PM +0800, wangyunjian wrote:
> From: Yunjian Wang <wangyunjian@huawei.com>
>
> Currently the driver doesn't drop a packet which can't be sent by tun
> (e.g bad packet). In this case, the driver will always process the
> same packet lead to the tx queue stuck.
>
> To fix this issue:
> 1. in the case of persistent failure (e.g bad packet), the driver
> can skip this descriptor by ignoring the error.
> 2. in the case of transient failure (e.g -ENOBUFS, -EAGAIN and -ENOMEM),
> the driver schedules the worker to try again.
>
> Fixes: 3a4d5c94e959 ("vhost_net: a kernel-level virtio server")
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> Acked-by: Willem de Bruijn <willemb@google.com>
> ---
> drivers/vhost/net.c | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index c8784dfafdd7..01558fb2c552 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -827,14 +827,13 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock)
> msg.msg_flags &= ~MSG_MORE;
> }
>
> - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> err = sock->ops->sendmsg(sock, &msg, len);
> - if (unlikely(err < 0)) {
> + if (unlikely(err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS)) {
> vhost_discard_vq_desc(vq, 1);
> vhost_net_enable_vq(net, vq);
> break;
> }
Hmm, there's the case of link being temporarily down (e.g. for
reconfigure), which IIRC returns EIO.
It's also probably a good idea to keep unlikely(err < 0) around
and then just regular err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS
since that first test can be done faster.
> - if (err != len)
> + if (err >= 0 && err != len)
> pr_debug("Truncated TX packet: len %d != %zd\n",
> err, len);
> done:
> @@ -922,7 +921,6 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock)
> msg.msg_flags &= ~MSG_MORE;
> }
>
> - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> err = sock->ops->sendmsg(sock, &msg, len);
> if (unlikely(err < 0)) {
> if (zcopy_used) {
> @@ -931,11 +929,13 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock)
> nvq->upend_idx = ((unsigned)nvq->upend_idx - 1)
> % UIO_MAXIOV;
> }
> - vhost_discard_vq_desc(vq, 1);
> - vhost_net_enable_vq(net, vq);
> - break;
> + if (err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS) {
same here
> + vhost_discard_vq_desc(vq, 1);
> + vhost_net_enable_vq(net, vq);
> + break;
> + }
> }
> - if (err != len)
> + if (err >= 0 && err != len)
> pr_debug("Truncated TX packet: "
> " len %d != %zd\n", err, len);
> if (!zcopy_used)
> --
> 2.23.0
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg fails
[not found] <1608881073-19004-1-git-send-email-wangyunjian@huawei.com>
2020-12-27 11:09 ` [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg fails Michael S. Tsirkin
@ 2020-12-27 11:20 ` Michael S. Tsirkin
[not found] ` <34EFBCA9F01B0748BEB6B629CE643AE60DBA7B3C@DGGEMM533-MBX.china.huawei.com>
[not found] ` <34EFBCA9F01B0748BEB6B629CE643AE60DBA8C53@DGGEMM533-MBX.china.huawei.com>
1 sibling, 2 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2020-12-27 11:20 UTC (permalink / raw)
To: wangyunjian
Cc: willemdebruijn.kernel, netdev, jerry.lilijun, virtualization,
xudingke, brian.huangbin, chenchanghu
On Fri, Dec 25, 2020 at 03:24:33PM +0800, wangyunjian wrote:
> From: Yunjian Wang <wangyunjian@huawei.com>
>
> Currently the driver doesn't drop a packet which can't be sent by tun
> (e.g bad packet). In this case, the driver will always process the
> same packet lead to the tx queue stuck.
So not making progress on a bad packet has some advantages,
e.g. this is easier to debug.
When is it important to drop the packet and continue?
> To fix this issue:
> 1. in the case of persistent failure (e.g bad packet), the driver
> can skip this descriptor by ignoring the error.
> 2. in the case of transient failure (e.g -ENOBUFS, -EAGAIN and -ENOMEM),
> the driver schedules the worker to try again.
>
> Fixes: 3a4d5c94e959 ("vhost_net: a kernel-level virtio server")
I'd just drop this tag, looks more like a feature than a bug ...
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> Acked-by: Willem de Bruijn <willemb@google.com>
> ---
> drivers/vhost/net.c | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index c8784dfafdd7..01558fb2c552 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -827,14 +827,13 @@ static void handle_tx_copy(struct vhost_net *net, struct socket *sock)
> msg.msg_flags &= ~MSG_MORE;
> }
>
> - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> err = sock->ops->sendmsg(sock, &msg, len);
> - if (unlikely(err < 0)) {
> + if (unlikely(err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS)) {
> vhost_discard_vq_desc(vq, 1);
> vhost_net_enable_vq(net, vq);
> break;
> }
> - if (err != len)
> + if (err >= 0 && err != len)
> pr_debug("Truncated TX packet: len %d != %zd\n",
> err, len);
> done:
> @@ -922,7 +921,6 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock)
> msg.msg_flags &= ~MSG_MORE;
> }
>
> - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> err = sock->ops->sendmsg(sock, &msg, len);
> if (unlikely(err < 0)) {
> if (zcopy_used) {
> @@ -931,11 +929,13 @@ static void handle_tx_zerocopy(struct vhost_net *net, struct socket *sock)
> nvq->upend_idx = ((unsigned)nvq->upend_idx - 1)
> % UIO_MAXIOV;
> }
> - vhost_discard_vq_desc(vq, 1);
> - vhost_net_enable_vq(net, vq);
> - break;
> + if (err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS) {
> + vhost_discard_vq_desc(vq, 1);
> + vhost_net_enable_vq(net, vq);
> + break;
> + }
> }
> - if (err != len)
> + if (err >= 0 && err != len)
> pr_debug("Truncated TX packet: "
> " len %d != %zd\n", err, len);
> if (!zcopy_used)
> --
> 2.23.0
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg fails
[not found] ` <34EFBCA9F01B0748BEB6B629CE643AE60DBA7B3C@DGGEMM533-MBX.china.huawei.com>
@ 2020-12-28 12:19 ` Michael S. Tsirkin
0 siblings, 0 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2020-12-28 12:19 UTC (permalink / raw)
To: wangyunjian
Cc: willemdebruijn.kernel@gmail.com, netdev@vger.kernel.org,
Lilijun (Jerry), virtualization@lists.linux-foundation.org,
xudingke, huangbin (J), chenchanghu
On Mon, Dec 28, 2020 at 11:55:36AM +0000, wangyunjian wrote:
> > -----Original Message-----
> > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > Sent: Sunday, December 27, 2020 7:21 PM
> > To: wangyunjian <wangyunjian@huawei.com>
> > Cc: netdev@vger.kernel.org; jasowang@redhat.com;
> > willemdebruijn.kernel@gmail.com; virtualization@lists.linux-foundation.org;
> > Lilijun (Jerry) <jerry.lilijun@huawei.com>; chenchanghu
> > <chenchanghu@huawei.com>; xudingke <xudingke@huawei.com>; huangbin (J)
> > <brian.huangbin@huawei.com>
> > Subject: Re: [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg
> > fails
> >
> > On Fri, Dec 25, 2020 at 03:24:33PM +0800, wangyunjian wrote:
> > > From: Yunjian Wang <wangyunjian@huawei.com>
> > >
> > > Currently the driver doesn't drop a packet which can't be sent by tun
> > > (e.g bad packet). In this case, the driver will always process the
> > > same packet lead to the tx queue stuck.
> >
> > So not making progress on a bad packet has some advantages, e.g. this is
> > easier to debug.
> > When is it important to drop the packet and continue?
>
> In the case, the VM will not be able to send packets persistently. Services of VM
> are affected.
>
> Thanks
Well VM can always harm itself right? Just halt the CPU, services will
be affected ;)
> >
> >
> > > To fix this issue:
> > > 1. in the case of persistent failure (e.g bad packet), the driver
> > > can skip this descriptor by ignoring the error.
> > > 2. in the case of transient failure (e.g -ENOBUFS, -EAGAIN and -ENOMEM),
> > > the driver schedules the worker to try again.
> > >
> > > Fixes: 3a4d5c94e959 ("vhost_net: a kernel-level virtio server")
> >
> > I'd just drop this tag, looks more like a feature than a bug ...
>
> OK
>
> >
> >
> > > Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> > > Acked-by: Willem de Bruijn <willemb@google.com>
> > > ---
> > > drivers/vhost/net.c | 16 ++++++++--------
> > > 1 file changed, 8 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index
> > > c8784dfafdd7..01558fb2c552 100644
> > > --- a/drivers/vhost/net.c
> > > +++ b/drivers/vhost/net.c
> > > @@ -827,14 +827,13 @@ static void handle_tx_copy(struct vhost_net *net,
> > struct socket *sock)
> > > msg.msg_flags &= ~MSG_MORE;
> > > }
> > >
> > > - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> > > err = sock->ops->sendmsg(sock, &msg, len);
> > > - if (unlikely(err < 0)) {
> > > + if (unlikely(err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS))
> > > +{
> > > vhost_discard_vq_desc(vq, 1);
> > > vhost_net_enable_vq(net, vq);
> > > break;
> > > }
> > > - if (err != len)
> > > + if (err >= 0 && err != len)
> > > pr_debug("Truncated TX packet: len %d != %zd\n",
> > > err, len);
> > > done:
> > > @@ -922,7 +921,6 @@ static void handle_tx_zerocopy(struct vhost_net
> > *net, struct socket *sock)
> > > msg.msg_flags &= ~MSG_MORE;
> > > }
> > >
> > > - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> > > err = sock->ops->sendmsg(sock, &msg, len);
> > > if (unlikely(err < 0)) {
> > > if (zcopy_used) {
> > > @@ -931,11 +929,13 @@ static void handle_tx_zerocopy(struct vhost_net
> > *net, struct socket *sock)
> > > nvq->upend_idx = ((unsigned)nvq->upend_idx - 1)
> > > % UIO_MAXIOV;
> > > }
> > > - vhost_discard_vq_desc(vq, 1);
> > > - vhost_net_enable_vq(net, vq);
> > > - break;
> > > + if (err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS) {
> > > + vhost_discard_vq_desc(vq, 1);
> > > + vhost_net_enable_vq(net, vq);
> > > + break;
> > > + }
> > > }
> > > - if (err != len)
> > > + if (err >= 0 && err != len)
> > > pr_debug("Truncated TX packet: "
> > > " len %d != %zd\n", err, len);
> > > if (!zcopy_used)
> > > --
> > > 2.23.0
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg fails
[not found] ` <34EFBCA9F01B0748BEB6B629CE643AE60DBA8C53@DGGEMM533-MBX.china.huawei.com>
@ 2020-12-28 16:11 ` Michael S. Tsirkin
0 siblings, 0 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2020-12-28 16:11 UTC (permalink / raw)
To: wangyunjian
Cc: willemdebruijn.kernel@gmail.com, netdev@vger.kernel.org,
Lilijun (Jerry), virtualization@lists.linux-foundation.org,
xudingke, huangbin (J), chenchanghu
On Mon, Dec 28, 2020 at 01:27:42PM +0000, wangyunjian wrote:
> > -----Original Message-----
> > From: Michael S. Tsirkin [mailto:mst@redhat.com]
> > Sent: Sunday, December 27, 2020 7:21 PM
> > To: wangyunjian <wangyunjian@huawei.com>
> > Cc: netdev@vger.kernel.org; jasowang@redhat.com;
> > willemdebruijn.kernel@gmail.com; virtualization@lists.linux-foundation.org;
> > Lilijun (Jerry) <jerry.lilijun@huawei.com>; chenchanghu
> > <chenchanghu@huawei.com>; xudingke <xudingke@huawei.com>; huangbin (J)
> > <brian.huangbin@huawei.com>
> > Subject: Re: [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg
> > fails
> >
> > On Fri, Dec 25, 2020 at 03:24:33PM +0800, wangyunjian wrote:
> > > From: Yunjian Wang <wangyunjian@huawei.com>
> > >
> > > Currently the driver doesn't drop a packet which can't be sent by tun
> > > (e.g bad packet). In this case, the driver will always process the
> > > same packet lead to the tx queue stuck.
> >
> > So not making progress on a bad packet has some advantages, e.g. this is
> > easier to debug.
> > When is it important to drop the packet and continue?
> >
> >
> > > To fix this issue:
> > > 1. in the case of persistent failure (e.g bad packet), the driver
> > > can skip this descriptor by ignoring the error.
> > > 2. in the case of transient failure (e.g -ENOBUFS, -EAGAIN and -ENOMEM),
> > > the driver schedules the worker to try again.
> > >
> > > Fixes: 3a4d5c94e959 ("vhost_net: a kernel-level virtio server")
> >
> > I'd just drop this tag, looks more like a feature than a bug ...
>
> Do these two patches need to be sent separately?
>
> Thanks
Makes sense to me.
> >
> >
> > > Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> > > Acked-by: Willem de Bruijn <willemb@google.com>
> > > ---
> > > drivers/vhost/net.c | 16 ++++++++--------
> > > 1 file changed, 8 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index
> > > c8784dfafdd7..01558fb2c552 100644
> > > --- a/drivers/vhost/net.c
> > > +++ b/drivers/vhost/net.c
> > > @@ -827,14 +827,13 @@ static void handle_tx_copy(struct vhost_net *net,
> > struct socket *sock)
> > > msg.msg_flags &= ~MSG_MORE;
> > > }
> > >
> > > - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> > > err = sock->ops->sendmsg(sock, &msg, len);
> > > - if (unlikely(err < 0)) {
> > > + if (unlikely(err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS))
> > > +{
> > > vhost_discard_vq_desc(vq, 1);
> > > vhost_net_enable_vq(net, vq);
> > > break;
> > > }
> > > - if (err != len)
> > > + if (err >= 0 && err != len)
> > > pr_debug("Truncated TX packet: len %d != %zd\n",
> > > err, len);
> > > done:
> > > @@ -922,7 +921,6 @@ static void handle_tx_zerocopy(struct vhost_net
> > *net, struct socket *sock)
> > > msg.msg_flags &= ~MSG_MORE;
> > > }
> > >
> > > - /* TODO: Check specific error and bomb out unless ENOBUFS? */
> > > err = sock->ops->sendmsg(sock, &msg, len);
> > > if (unlikely(err < 0)) {
> > > if (zcopy_used) {
> > > @@ -931,11 +929,13 @@ static void handle_tx_zerocopy(struct vhost_net
> > *net, struct socket *sock)
> > > nvq->upend_idx = ((unsigned)nvq->upend_idx - 1)
> > > % UIO_MAXIOV;
> > > }
> > > - vhost_discard_vq_desc(vq, 1);
> > > - vhost_net_enable_vq(net, vq);
> > > - break;
> > > + if (err == -EAGAIN || err == -ENOMEM || err == -ENOBUFS) {
> > > + vhost_discard_vq_desc(vq, 1);
> > > + vhost_net_enable_vq(net, vq);
> > > + break;
> > > + }
> > > }
> > > - if (err != len)
> > > + if (err >= 0 && err != len)
> > > pr_debug("Truncated TX packet: "
> > > " len %d != %zd\n", err, len);
> > > if (!zcopy_used)
> > > --
> > > 2.23.0
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-12-28 16:12 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1608881073-19004-1-git-send-email-wangyunjian@huawei.com>
2020-12-27 11:09 ` [PATCH net v5 2/2] vhost_net: fix tx queue stuck when sendmsg fails Michael S. Tsirkin
2020-12-27 11:20 ` Michael S. Tsirkin
[not found] ` <34EFBCA9F01B0748BEB6B629CE643AE60DBA7B3C@DGGEMM533-MBX.china.huawei.com>
2020-12-28 12:19 ` Michael S. Tsirkin
[not found] ` <34EFBCA9F01B0748BEB6B629CE643AE60DBA8C53@DGGEMM533-MBX.china.huawei.com>
2020-12-28 16:11 ` Michael S. Tsirkin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).