From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAA00C43217 for ; Thu, 10 Mar 2022 04:54:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234883AbiCJEzY (ORCPT ); Wed, 9 Mar 2022 23:55:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239525AbiCJEzU (ORCPT ); Wed, 9 Mar 2022 23:55:20 -0500 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 027E5129BA3; Wed, 9 Mar 2022 20:54:18 -0800 (PST) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R761e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=33;SR=0;TI=SMTPD_---0V6n5VCi_1646888052; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V6n5VCi_1646888052) by smtp.aliyun-inc.com(127.0.0.1); Thu, 10 Mar 2022 12:54:13 +0800 Message-ID: <1646887597.810321-1-xuanzhuo@linux.alibaba.com> Subject: Re: [PATCH v7 09/26] virtio_ring: split: implement virtqueue_reset_vring_split() Date: Thu, 10 Mar 2022 12:46:37 +0800 From: Xuan Zhuo To: Jason Wang Cc: Jeff Dike , Richard Weinberger , Anton Ivanov , "Michael S. Tsirkin" , "David S. Miller" , Jakub Kicinski , Hans de Goede , Mark Gross , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Alexander Gordeev , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Johannes Berg , Vincent Whitchurch , linux-um@lists.infradead.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org References: <20220308123518.33800-1-xuanzhuo@linux.alibaba.com> <20220308123518.33800-10-xuanzhuo@linux.alibaba.com> <512de020-b36e-8473-69c8-8b3925fbb6c1@redhat.com> In-Reply-To: <512de020-b36e-8473-69c8-8b3925fbb6c1@redhat.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, 9 Mar 2022 15:55:44 +0800, Jason Wang wrote: > > =E5=9C=A8 2022/3/8 =E4=B8=8B=E5=8D=888:35, Xuan Zhuo =E5=86=99=E9=81=93: > > virtio ring supports reset. > > > > Queue reset is divided into several stages. > > > > 1. notify device queue reset > > 2. vring release > > 3. attach new vring > > 4. notify device queue re-enable > > > > After the first step is completed, the vring reset operation can be > > performed. If the newly set vring num does not change, then just reset > > the vq related value. > > > > Otherwise, the vring will be released and the vring will be reallocated. > > And the vring will be attached to the vq. If this process fails, the > > function will exit, and the state of the vq will be the vring release > > state. You can call this function again to reallocate the vring. > > > > In addition, vring_align, may_reduce_num are necessary for reallocating > > vring, so they are retained when creating vq. > > > > Signed-off-by: Xuan Zhuo > > --- > > drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++++++++++ > > 1 file changed, 69 insertions(+) > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > index e0422c04c903..148fb1fd3d5a 100644 > > --- a/drivers/virtio/virtio_ring.c > > +++ b/drivers/virtio/virtio_ring.c > > @@ -158,6 +158,12 @@ struct vring_virtqueue { > > /* DMA address and size information */ > > dma_addr_t queue_dma_addr; > > size_t queue_size_in_bytes; > > + > > + /* The parameters for creating vrings are reserved for > > + * creating new vrings when enabling reset queue. > > + */ > > + u32 vring_align; > > + bool may_reduce_num; > > } split; > > > > /* Available for packed ring */ > > @@ -217,6 +223,12 @@ struct vring_virtqueue { > > #endif > > }; > > > > +static void vring_free(struct virtqueue *vq); > > +static void __vring_virtqueue_init_split(struct vring_virtqueue *vq, > > + struct virtio_device *vdev); > > +static int __vring_virtqueue_attach_split(struct vring_virtqueue *vq, > > + struct virtio_device *vdev, > > + struct vring vring); > > > > /* > > * Helpers. > > @@ -1012,6 +1024,8 @@ static struct virtqueue *vring_create_virtqueue_s= plit( > > return NULL; > > } > > > > + to_vvq(vq)->split.vring_align =3D vring_align; > > + to_vvq(vq)->split.may_reduce_num =3D may_reduce_num; > > to_vvq(vq)->split.queue_dma_addr =3D vring.dma_addr; > > to_vvq(vq)->split.queue_size_in_bytes =3D vring.queue_size_in_bytes; > > to_vvq(vq)->we_own_ring =3D true; > > @@ -1019,6 +1033,59 @@ static struct virtqueue *vring_create_virtqueue_= split( > > return vq; > > } > > > > +static int virtqueue_reset_vring_split(struct virtqueue *_vq, u32 num) > > +{ > > > So what this function does is to resize the virtqueue actually, I > suggest to rename it as virtqueue_resize_split(). In addition to resize, when num is 0, the function is to reinitialize vq ri= ng related variables. For example avail_idx_shadow. So I think 'reset' is more appropriate. Thanks. > > > > + struct vring_virtqueue *vq =3D to_vvq(_vq); > > + struct virtio_device *vdev =3D _vq->vdev; > > + struct vring_split vring; > > + int err; > > + > > + if (num > _vq->num_max) > > + return -E2BIG; > > + > > + switch (vq->vq.reset) { > > + case VIRTIO_VQ_RESET_STEP_NONE: > > + return -ENOENT; > > + > > + case VIRTIO_VQ_RESET_STEP_VRING_ATTACH: > > + case VIRTIO_VQ_RESET_STEP_DEVICE: > > + if (vq->split.vring.num =3D=3D num || !num) > > + break; > > + > > + vring_free(_vq); > > + > > + fallthrough; > > + > > + case VIRTIO_VQ_RESET_STEP_VRING_RELEASE: > > + if (!num) > > + num =3D vq->split.vring.num; > > + > > + err =3D vring_create_vring_split(&vring, vdev, > > + vq->split.vring_align, > > + vq->weak_barriers, > > + vq->split.may_reduce_num, num); > > + if (err) > > + return -ENOMEM; > > > We'd better need a safe fallback here like: > > If we can't allocate new memory, we can keep using the current one. > Otherwise an ethtool -G fail may make the device not usable. > > This could be done by not freeing the old vring and virtqueue states > until new is allocated. > > > > + > > + err =3D __vring_virtqueue_attach_split(vq, vdev, vring.vring); > > + if (err) { > > + vring_free_queue(vdev, vring.queue_size_in_bytes, > > + vring.queue, > > + vring.dma_addr); > > + return -ENOMEM; > > + } > > + > > + vq->split.queue_dma_addr =3D vring.dma_addr; > > + vq->split.queue_size_in_bytes =3D vring.queue_size_in_bytes; > > + } > > + > > + __vring_virtqueue_init_split(vq, vdev); > > + vq->we_own_ring =3D true; > > > This seems wrong, we have the transport (rproc/mlxtbf) that allocate the > vring by themselves. I think we need to fail the resize for we_own_ring > =3D=3D false. > > Thanks > > > > > + vq->vq.reset =3D VIRTIO_VQ_RESET_STEP_VRING_ATTACH; > > + > > + return 0; > > +} > > + > > > > /* > > * Packed ring specific functions - *_packed(). > > @@ -2317,6 +2384,8 @@ static int __vring_virtqueue_attach_split(struct = vring_virtqueue *vq, > > static void __vring_virtqueue_init_split(struct vring_virtqueue *vq, > > struct virtio_device *vdev) > > { > > + vq->vq.reset =3D VIRTIO_VQ_RESET_STEP_NONE; > > + > > vq->packed_ring =3D false; > > vq->we_own_ring =3D false; > > vq->broken =3D false; >