From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8EFAC54E68 for ; Wed, 20 Mar 2024 00:50:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qFVQPk7qwgYAVT+Fw8GI1j8bIsc+xHeiVWWGKGBpDpw=; b=XpQrLvhjhENPVq 9YENMW3vbtzTy9tiCzf+qTh2BbDJ+MlH6cGXmMhcLSRGRidoLonmzWIA4CP95hSZoExiflM271hAi URTnujlE0z+OKHr3L11Ov74Y7XMK0lzfnlJerFCukfQy1dnI37g45Uusx7L+ip1Cqo+KxtZv0Qs5M hJMo7+CNoAHF36lMoCvfRLqq4UJkFsIScEjR9OwgwVBF9rwioMRIrYacbWjRu01HPdr1EfATHZV78 4GpDu7erMt3EusfhebvRfkeS5DiRWY2oFXHaCzPSFS1MmCzQIADqSU2cX3EqtyH6L41g0ruvafZmD m0O0sa+nf9rqToVYS4dQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmk9R-0000000EoF5-1QGM; Wed, 20 Mar 2024 00:49:53 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmk9O-0000000EoDm-0ouC for linux-arm-kernel@lists.infradead.org; Wed, 20 Mar 2024 00:49:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710895788; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CE0aiUgp6PdfvujdkzIH46kf740RuCaPAFwYm77rL6c=; b=Krt5aa+S5E8CVvqGuUN3l/BcWwUedlLSOiyJs1rlht0IzNZS3zwXHTH8f2Ifl2o/vrgDjy Q3NEAAdvBzLfe3IQdHRJF8WUhKxdCHbKCwgNhLPJqFD4R43P2tCFyWBMwldBzasMRXih7I S2OXMt7Rz4kVuiPQbPplA4QCiWlDxok= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-2-zeyRefYPNd6FVyn-GGB5gg-1; Tue, 19 Mar 2024 20:49:47 -0400 X-MC-Unique: zeyRefYPNd6FVyn-GGB5gg-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-33ecc0f0c95so3437814f8f.3 for ; Tue, 19 Mar 2024 17:49:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710895786; x=1711500586; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=CE0aiUgp6PdfvujdkzIH46kf740RuCaPAFwYm77rL6c=; b=pvs3Z4A9sUUsHIP90HFKkyBlO65KOJ/Y1X3ZDy0AhRJtvKGtqCIkh0CPAlLqMC8sFY enM6iPjH+FrmXR8Ajd4sAVmwrKvDdLzJ0lg2TE86rpU5zFO76Dc69JAzmAuX4rIbole6 SFtK9udtXyv25YFMX+OuS4D4n0h0J6kER8r8iQEqjIrn7miweSELN/rTqGJ/z3ZfIGTZ qVEPYd1PshAms4ip2iEHEx0yf7wSpQOxTzE+JOWjpv7J/omzv1MYtozxkvX9L/C8Bpd+ pCfFF8BwVyEY6i+J+S8aClSaWIYwh39bH8mbzeYlY4fQQ8YfsznS4RkocLKE6zbrgLGW KUZg== X-Forwarded-Encrypted: i=1; AJvYcCURSGnl1bRGSL/L5PPtjCdtFiK4/lzXv29gGk1Y2y1+vyGIVNOKcacyNn7v8vTZfDc/SIdfHZsXgOvF93iqtw91n8BcPvpcQoJFBX0GU/YizlSUhnw= X-Gm-Message-State: AOJu0Ywcp3P/LMqSb0jy6VRFGJStcdZOZbrPxx1SbYBYQuAQktFfjhD7 ll+e4GblmauaWCJ2lzlnrlXsxjlY2TIqx8T0X5ZCzAEwnLckCZsE5uTmv7vUXubcgyVcnIvLevq /+PXm2Ddva+ZwY1ewQXLT1K+6N3SXHFEmIoWWWjam3mk145fv52u8jS7TmEb5Yih5gTUxVHiL X-Received: by 2002:a5d:6605:0:b0:33e:c9ce:23fe with SMTP id n5-20020a5d6605000000b0033ec9ce23femr366578wru.25.1710895785866; Tue, 19 Mar 2024 17:49:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEk6XEl3djgMno5+rKGDJaHnEp73VZSbNTX00Jd72jtRf2nDEaEUxaqAu/J9M21qKE29pVSWQ== X-Received: by 2002:a5d:6605:0:b0:33e:c9ce:23fe with SMTP id n5-20020a5d6605000000b0033ec9ce23femr366565wru.25.1710895785255; Tue, 19 Mar 2024 17:49:45 -0700 (PDT) Received: from redhat.com ([2.52.6.254]) by smtp.gmail.com with ESMTPSA id dd12-20020a0560001e8c00b0033eab3520a9sm13483749wrb.43.2024.03.19.17.49.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Mar 2024 17:49:44 -0700 (PDT) Date: Tue, 19 Mar 2024 20:49:40 -0400 From: "Michael S. Tsirkin" To: Gavin Shan Cc: Will Deacon , virtualization@lists.linux.dev, linux-kernel@vger.kernel.org, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, yihyu@redhat.com, shan.gavin@gmail.com, linux-arm-kernel@lists.infradead.org, Catalin Marinas , mochs@nvidia.com Subject: Re: [PATCH] virtio_ring: Fix the stale index in available ring Message-ID: <20240319203540-mutt-send-email-mst@kernel.org> References: <20240314074923.426688-1-gshan@redhat.com> <20240318165924.GA1824@willie-the-truck> <35a6bcef-27cf-4626-a41d-9ec0a338fe28@redhat.com> <20240319182251.GB3121@willie-the-truck> <9500adaf-0075-4ae9-92db-7e310b6598b0@redhat.com> MIME-Version: 1.0 In-Reply-To: <9500adaf-0075-4ae9-92db-7e310b6598b0@redhat.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240319_174950_569684_BD50188C X-CRM114-Status: GOOD ( 41.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Mar 20, 2024 at 09:56:58AM +1000, Gavin Shan wrote: > On 3/20/24 04:22, Will Deacon wrote: > > On Tue, Mar 19, 2024 at 02:59:23PM +1000, Gavin Shan wrote: > > > On 3/19/24 02:59, Will Deacon wrote: > > > > > drivers/virtio/virtio_ring.c | 12 +++++++++--- > > > > > 1 file changed, 9 insertions(+), 3 deletions(-) > > > > > > > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > > > > > index 49299b1f9ec7..7d852811c912 100644 > > > > > --- a/drivers/virtio/virtio_ring.c > > > > > +++ b/drivers/virtio/virtio_ring.c > > > > > @@ -687,9 +687,15 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, > > > > > avail = vq->split.avail_idx_shadow & (vq->split.vring.num - 1); > > > > > vq->split.vring.avail->ring[avail] = cpu_to_virtio16(_vq->vdev, head); > > > > > - /* Descriptors and available array need to be set before we expose the > > > > > - * new available array entries. */ > > > > > - virtio_wmb(vq->weak_barriers); > > > > > + /* > > > > > + * Descriptors and available array need to be set before we expose > > > > > + * the new available array entries. virtio_wmb() should be enough > > > > > + * to ensuere the order theoretically. However, a stronger barrier > > > > > + * is needed by ARM64. Otherwise, the stale data can be observed > > > > > + * by the host (vhost). A stronger barrier should work for other > > > > > + * architectures, but performance loss is expected. > > > > > + */ > > > > > + virtio_mb(false); > > > > > vq->split.avail_idx_shadow++; > > > > > vq->split.vring.avail->idx = cpu_to_virtio16(_vq->vdev, > > > > > vq->split.avail_idx_shadow); > > > > > > > > Replacing a DMB with a DSB is _very_ unlikely to be the correct solution > > > > here, especially when ordering accesses to coherent memory. > > > > > > > > In practice, either the larger timing different from the DSB or the fact > > > > that you're going from a Store->Store barrier to a full barrier is what > > > > makes things "work" for you. Have you tried, for example, a DMB SY > > > > (e.g. via __smb_mb()). > > > > > > > > We definitely shouldn't take changes like this without a proper > > > > explanation of what is going on. > > > > > > > > > > Thanks for your comments, Will. > > > > > > Yes, DMB should work for us. However, it seems this instruction has issues on > > > NVidia's grace-hopper. It's hard for me to understand how DMB and DSB works > > > from hardware level. I agree it's not the solution to replace DMB with DSB > > > before we fully understand the root cause. > > > > > > I tried the possible replacement like below. __smp_mb() can avoid the issue like > > > __mb() does. __ndelay(10) can avoid the issue, but __ndelay(9) doesn't. > > > > > > static inline int virtqueue_add_split(struct virtqueue *_vq, ...) > > > { > > > : > > > /* Put entry in available array (but don't update avail->idx until they > > > * do sync). */ > > > avail = vq->split.avail_idx_shadow & (vq->split.vring.num - 1); > > > vq->split.vring.avail->ring[avail] = cpu_to_virtio16(_vq->vdev, head); > > > > > > /* Descriptors and available array need to be set before we expose the > > > * new available array entries. */ > > > // Broken: virtio_wmb(vq->weak_barriers); > > > // Broken: __dma_mb(); > > > // Work: __mb(); > > > // Work: __smp_mb(); > > > > It's pretty weird that __dma_mb() is "broken" but __smp_mb() "works". How > > confident are you in that result? > > > > Yes, __dma_mb() is even stronger than __smp_mb(). I retried the test, showing > that both __dma_mb() and __smp_mb() work for us. I had too many tests yesterday > and something may have been messed up. > > Instruction Hitting times in 10 tests > --------------------------------------------- > __smp_wmb() 8 > __smp_mb() 0 > __dma_wmb() 7 > __dma_mb() 0 > __mb() 0 > __wmb() 0 > > It's strange that __smp_mb() works, but __smp_wmb() fails. It seems we need a > read barrier here. I will try WRITE_ONCE() + __smp_wmb() as suggested by Michael > in another reply. Will update the result soon. > > Thanks, > Gavin I think you are wasting the time with these tests. Even if it helps what does this tell us? Try setting a flag as I suggested elsewhere. Then check it in vhost. Or here's another idea - possibly easier. Copy the high bits from index into ring itself. Then vhost can check that head is synchronized with index. Warning: completely untested, not even compiled. But should give you the idea. If this works btw we should consider making this official in the spec. static inline int vhost_get_avail_flags(struct vhost_virtqueue *vq, diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 6f7e5010a673..79456706d0bd 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -685,7 +685,8 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, /* Put entry in available array (but don't update avail->idx until they * do sync). */ avail = vq->split.avail_idx_shadow & (vq->split.vring.num - 1); - vq->split.vring.avail->ring[avail] = cpu_to_virtio16(_vq->vdev, head); + u16 headwithflag = head | (q->split.avail_idx_shadow & ~(vq->split.vring.num - 1)); + vq->split.vring.avail->ring[avail] = cpu_to_virtio16(_vq->vdev, headwithflag); /* Descriptors and available array need to be set before we expose the * new available array entries. */ diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 045f666b4f12..bd8f7c763caa 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1299,8 +1299,15 @@ static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq, static inline int vhost_get_avail_head(struct vhost_virtqueue *vq, __virtio16 *head, int idx) { - return vhost_get_avail(vq, *head, + unsigned i = idx; + unsigned flag = i & ~(vq->num - 1); + unsigned val = vhost_get_avail(vq, *head, &vq->avail->ring[idx & (vq->num - 1)]); + unsigned valflag = val & ~(vq->num - 1); + + WARN_ON(valflag != flag); + + return val & (vq->num - 1); } -- MST _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel