From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AA53C77B75 for ; Fri, 21 Apr 2023 10:25:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231130AbjDUKZt (ORCPT ); Fri, 21 Apr 2023 06:25:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230124AbjDUKZs (ORCPT ); Fri, 21 Apr 2023 06:25:48 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 97B28173F for ; Fri, 21 Apr 2023 03:25:44 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3260A1480; Fri, 21 Apr 2023 03:26:28 -0700 (PDT) Received: from monolith.localdoman (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A2F723F6C4; Fri, 21 Apr 2023 03:25:42 -0700 (PDT) Date: Fri, 21 Apr 2023 11:25:39 +0100 From: Alexandru Elisei To: Eric Auger Cc: eric.auger.pro@gmail.com, kvm@vger.kernel.org, kvmarm@lists.linux.dev, andrew.jones@linux.dev, maz@kernel.org, will@kernel.org, oliver.upton@linux.dev, ricarkol@google.com, reijiw@google.com Subject: Re: [kvm-unit-tests PATCH 3/6] arm: pmu: Add extra DSB barriers in the mem_access loop Message-ID: References: <20230315110725.1215523-1-eric.auger@redhat.com> <20230315110725.1215523-4-eric.auger@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230315110725.1215523-4-eric.auger@redhat.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi, On Wed, Mar 15, 2023 at 12:07:22PM +0100, Eric Auger wrote: > The mem access loop currently features ISB barriers only. However > the mem_access loop counts the number of accesses to memory. ISB > do not garantee the PE cannot reorder memory access. Let's > add a DSB ISH before the write to PMCR_EL0 that enables the PMU > and after the last iteration, before disabling the PMU. > > Signed-off-by: Eric Auger > Suggested-by: Alexandru Elisei > > --- > > This was discussed in https://lore.kernel.org/all/YzxmHpV2rpfaUdWi@monolith.localdoman/ > --- > arm/pmu.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/arm/pmu.c b/arm/pmu.c > index b88366a8..dde399e2 100644 > --- a/arm/pmu.c > +++ b/arm/pmu.c > @@ -301,6 +301,7 @@ static void mem_access_loop(void *addr, long loop, uint32_t pmcr) > { > uint64_t pmcr64 = pmcr; > asm volatile( > + " dsb ish\n" I think it might still be possible to reorder memory accesses which are part of the loop after the DSB above and before the PMU is enabled below. But the DSB above is needed to make sure previous memory accesses, which shouldn't be counted as part of the loop, are completed. I would put another DSB after the ISB which enables the PMU, that way all memory accesses are neatly sandwitches between two DSBs. Having 3 DSBs might look like overdoing it, but I reason it to be correct. What do you think? Thanks, Alex > " msr pmcr_el0, %[pmcr]\n" > " isb\n" > " mov x10, %[loop]\n" > @@ -308,6 +309,7 @@ asm volatile( > " ldr x9, [%[addr]]\n" > " cmp x10, #0x0\n" > " b.gt 1b\n" > + " dsb ish\n" > " msr pmcr_el0, xzr\n" > " isb\n" > : > -- > 2.38.1 > >