From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2597C54E60 for ; Tue, 19 Mar 2024 07:42:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l+4FOOV5kk90CnM7LuyyEDOWdpgz6AC4UVJsJBB3UAo=; b=dRvEaS3KP4lV0a 5JOQK+333L6OUeVna5l8okHuw06J3LfzhuSajMxCK5w2dvS32F20fD1ByBzeNIAbZDIcMIctDAOCu gNO8jwSO0444gx9n2yvRmwlLn5MhbOZxGxZMap+cetpk+WMfwO2kub4YRzvh4A5eOTtim7BIPt7hX lrXhTnVpgZrj35law2GXakfBqtP5c5ugxIHrKoONl8oISDJRW3EqJgbMhHee27mDxyhM86m0avo+R RZx1eqT3RB2vzHbSiRnN/W1GUHuRpcw+fsB1YAIAwEkGPRGfTUaQBktNfv5/G6ISkALuAP5W//CLv MoF000R4qT6fodlu5ghA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmU6X-0000000Bk2z-0AUb; Tue, 19 Mar 2024 07:41:49 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmU6U-0000000Bk1v-1x7H for linux-arm-kernel@lists.infradead.org; Tue, 19 Mar 2024 07:41:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710834105; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZENGAKckd1InNLrAPOyF9uUy4qFGPBOjBi+IwPE3MtM=; b=Crm/7FBxmsD3naOK1ta+fzyBNRkfSBXL+ojYx7PzJeqYJWSKEQ5aVJZkO12BJHWnkjx5ex bN2fvasDSCTFw1xObTnCz0/CM6XxVnVaxGZO6PdEfhnXMI9KJqsUbMVrwoI6TjuWnZUcMY 52EhbUlquGyP5x4MPuS665qtCcJ/QAo= Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-502-qXe-7CM1Pauab0kNaDvDXw-1; Tue, 19 Mar 2024 03:41:43 -0400 X-MC-Unique: qXe-7CM1Pauab0kNaDvDXw-1 Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-29f9bffaa42so1630940a91.0 for ; Tue, 19 Mar 2024 00:41:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710834102; x=1711438902; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZENGAKckd1InNLrAPOyF9uUy4qFGPBOjBi+IwPE3MtM=; b=I/ORAsJLbgSqndL0c2jjj5kt948GXuWOylmCzat4f3MnyH6dy6kBe7wrcte8nG8x90 /6yGtDU0jN82yVqJzuv7A46MDMv6CeHA/El4w41y3Uihea9vlZnC4xWQXtxGctiIpcR3 iiJSpuUDIDhGQO46r09lnJ7sve+Xcckj2Sq/DA6h3phjlULy1l2APkCem5kBtUKXE5jM hODZ1GPBQrNUXWK6xRF52AZ5dOMOtQImejC1Cmp148Zw4ytZG/CPUK/ORe/3h8tP0z8q UZz/wWP64lPjeFGGSj4YavDpgK/GseUDK4iF0PBWVrggxI2F8rvsZiitKLPS24xQkyMP juYQ== X-Forwarded-Encrypted: i=1; AJvYcCUSB5Z/Z8HxuVB7XbInLa61g8Jw+3d+WDze19Lko3tdO9YWIbXzkOBIY7VPyj2J2qqdW06DiNNCV6E9zv3rJwfwkTNCa8mRZLQjgb83z8d+86q+aGM= X-Gm-Message-State: AOJu0YxXRLIODWnOeVG4HeLqbbqoBwP2yALBsztmdSIp5EY6E3eq4Jhy 0sjT/+hbmXsnK9XJU1EsdkH7riBIOa80t5HZVpXhR4GzZoVzPiQivzAonSBIVcmWK1KZ9GtjpyD BiAS8wYUZI0RryoJs6BiADfI9q4yG+KTdxG7gGUjxeyVB1QWObcB+d0/AY81gKq+jWEN+o4rx X-Received: by 2002:a17:90a:f3c5:b0:29d:e75b:e73a with SMTP id ha5-20020a17090af3c500b0029de75be73amr1621492pjb.49.1710834102096; Tue, 19 Mar 2024 00:41:42 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGjzCbjypzQuA5Jo77zZ7OknwZTx0eJueh9GmXOAvslsuDsH2aDA/AR+72V97FVuzm6mzOBDQ== X-Received: by 2002:a17:90a:f3c5:b0:29d:e75b:e73a with SMTP id ha5-20020a17090af3c500b0029de75be73amr1621479pjb.49.1710834101764; Tue, 19 Mar 2024 00:41:41 -0700 (PDT) Received: from [192.168.68.51] ([43.252.115.31]) by smtp.gmail.com with ESMTPSA id pt7-20020a17090b3d0700b0029fe026ae17sm475104pjb.51.2024.03.19.00.41.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 00:41:41 -0700 (PDT) Message-ID: <6df97716-6669-4d94-8923-41758f71fc07@redhat.com> Date: Tue, 19 Mar 2024 17:41:35 +1000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] virtio_ring: Fix the stale index in available ring To: "Michael S. Tsirkin" Cc: Will Deacon , virtualization@lists.linux.dev, linux-kernel@vger.kernel.org, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, yihyu@redhat.com, shan.gavin@gmail.com, linux-arm-kernel@lists.infradead.org, Catalin Marinas , mochs@nvidia.com References: <20240314074923.426688-1-gshan@redhat.com> <20240318165924.GA1824@willie-the-truck> <35a6bcef-27cf-4626-a41d-9ec0a338fe28@redhat.com> <20240319020905-mutt-send-email-mst@kernel.org> <20240319020949-mutt-send-email-mst@kernel.org> <6b829cfc-9cbe-42eb-9935-62d2cf5fbcc4@redhat.com> <20240319030148-mutt-send-email-mst@kernel.org> From: Gavin Shan In-Reply-To: <20240319030148-mutt-send-email-mst@kernel.org> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240319_004146_616828_F7D4A490 X-CRM114-Status: GOOD ( 20.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 3/19/24 17:04, Michael S. Tsirkin wrote: > On Tue, Mar 19, 2024 at 04:54:15PM +1000, Gavin Shan wrote: >> On 3/19/24 16:10, Michael S. Tsirkin wrote: >>> On Tue, Mar 19, 2024 at 02:09:34AM -0400, Michael S. Tsirkin wrote: >>>> On Tue, Mar 19, 2024 at 02:59:23PM +1000, Gavin Shan wrote: >>>>> On 3/19/24 02:59, Will Deacon wrote: >> [...] >>>>>>> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c >>>>>>> index 49299b1f9ec7..7d852811c912 100644 >>>>>>> --- a/drivers/virtio/virtio_ring.c >>>>>>> +++ b/drivers/virtio/virtio_ring.c >>>>>>> @@ -687,9 +687,15 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, >>>>>>> avail = vq->split.avail_idx_shadow & (vq->split.vring.num - 1); >>>>>>> vq->split.vring.avail->ring[avail] = cpu_to_virtio16(_vq->vdev, head); >>>>>>> - /* Descriptors and available array need to be set before we expose the >>>>>>> - * new available array entries. */ >>>>>>> - virtio_wmb(vq->weak_barriers); >>>>>>> + /* >>>>>>> + * Descriptors and available array need to be set before we expose >>>>>>> + * the new available array entries. virtio_wmb() should be enough >>>>>>> + * to ensuere the order theoretically. However, a stronger barrier >>>>>>> + * is needed by ARM64. Otherwise, the stale data can be observed >>>>>>> + * by the host (vhost). A stronger barrier should work for other >>>>>>> + * architectures, but performance loss is expected. >>>>>>> + */ >>>>>>> + virtio_mb(false); >>>>>>> vq->split.avail_idx_shadow++; >>>>>>> vq->split.vring.avail->idx = cpu_to_virtio16(_vq->vdev, >>>>>>> vq->split.avail_idx_shadow); >>>>>> >>>>>> Replacing a DMB with a DSB is _very_ unlikely to be the correct solution >>>>>> here, especially when ordering accesses to coherent memory. >>>>>> >>>>>> In practice, either the larger timing different from the DSB or the fact >>>>>> that you're going from a Store->Store barrier to a full barrier is what >>>>>> makes things "work" for you. Have you tried, for example, a DMB SY >>>>>> (e.g. via __smb_mb()). >>>>>> >>>>>> We definitely shouldn't take changes like this without a proper >>>>>> explanation of what is going on. >>>>>> >>>>> >>>>> Thanks for your comments, Will. >>>>> >>>>> Yes, DMB should work for us. However, it seems this instruction has issues on >>>>> NVidia's grace-hopper. It's hard for me to understand how DMB and DSB works >>>>> from hardware level. I agree it's not the solution to replace DMB with DSB >>>>> before we fully understand the root cause. >>>>> >>>>> I tried the possible replacement like below. __smp_mb() can avoid the issue like >>>>> __mb() does. __ndelay(10) can avoid the issue, but __ndelay(9) doesn't. >>>>> >>>>> static inline int virtqueue_add_split(struct virtqueue *_vq, ...) >>>>> { >>>>> : >>>>> /* Put entry in available array (but don't update avail->idx until they >>>>> * do sync). */ >>>>> avail = vq->split.avail_idx_shadow & (vq->split.vring.num - 1); >>>>> vq->split.vring.avail->ring[avail] = cpu_to_virtio16(_vq->vdev, head); >>>>> >>>>> /* Descriptors and available array need to be set before we expose the >>>>> * new available array entries. */ >>>>> // Broken: virtio_wmb(vq->weak_barriers); >>>>> // Broken: __dma_mb(); >>>>> // Work: __mb(); >>>>> // Work: __smp_mb(); >>> >>> Did you try __smp_wmb ? And wmb? >>> >> >> virtio_wmb(false) is equivalent to __smb_wmb(), which is broken. >> >> __wmb() works either. No issue found with it. > > Oh interesting. So how do smp_mb() and wmb() disassemble on this > platform? Can you please check? > I don't see they have been translated wrongly on Nvidia's grace-hopper: ===> virtio_wmb(vq->weak_barriers) 0xffff8000807b07c8 <+1168>: ldrb w0, [x20, #66] 0xffff8000807b07cc <+1172>: cbz w0, 0xffff8000807b089c 0xffff8000807b07d0 <+1176>: dmb ishst // same to __smp_wmb() : 0xffff8000807b089c <+1380>: dmb oshst // same to __dma_wmb() 0xffff8000807b08a0 <+1384>: b 0xffff8000807b07d4 ===> wmb() 0xffff8000807b07c8 <+1168>: dsb st > >>>>> // Work: __ndelay(100); >>>>> // Work: __ndelay(10); >>>>> // Broken: __ndelay(9); >>>>> >>>>> vq->split.avail_idx_shadow++; >>>>> vq->split.vring.avail->idx = cpu_to_virtio16(_vq->vdev, >>>>> vq->split.avail_idx_shadow); >>>> >>>> What if you stick __ndelay here? >>> >>> And keep virtio_wmb above? >>> >> >> The result has been shared through a separate reply. >> >>>> >>>>> vq->num_added++; >>>>> >>>>> pr_debug("Added buffer head %i to %p\n", head, vq); >>>>> END_USE(vq); >>>>> : >>>>> } >>>>> >>>>> I also tried to measure the consumed time for various barrier-relative instructions using >>>>> ktime_get_ns() which should have consumed most of the time. __smb_mb() is slower than >>>>> __smp_wmb() but faster than __mb() >>>>> >>>>> Instruction Range of used time in ns >>>>> ---------------------------------------------- >>>>> __smp_wmb() [32 1128032] >>>>> __smp_mb() [32 1160096] >>>>> __mb() [32 1162496] >>>>> Thanks, Gavin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel