From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89DF4EB64DA for ; Fri, 14 Jul 2023 08:24:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=z6hFilECo29zyic17nt/6yt2qTA5bkjGmyyQC3oDvgo=; b=TK0HPzl5sjZM96 vJ8FGiyFlP7ydIQTPnSf90XuVteSC/hxtJxwf+emvw8jeeogbxSs3byqeExSjaivcKDxg/F6ZYV3M 7emxC1oeUnIZpcfm3+h8ehpv3c49/O2EarMGTzUNttH8jse9cXQyhzXJp/WzC4JsM+D78HmYrVeMS YPLOGXfjhd2QeI02N62ckzrPNtN6WB8XeXUJeTCHTpR6Pl4oNEMONsgi+tewkYgxInSzDxhLdoJQM SDiHF9fIG+aUW8afcnjYdJQ68tV1ITA735L3iiigtjQs5BibgORv0rFenKnVaigkX99o02iOLBlQd jOvSS0KCSAUl8LWytw8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qKE5m-005WCz-2B; Fri, 14 Jul 2023 08:23:58 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qKE5k-005WC8-1j for linux-arm-kernel@lists.infradead.org; Fri, 14 Jul 2023 08:23:58 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5734A61BCB; Fri, 14 Jul 2023 08:23:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87BBAC433C8; Fri, 14 Jul 2023 08:23:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689323034; bh=vzHIGLT92Won/B+RdarvMWgfIJZa19KQoX8ChdEHMg8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=AskWsqQaO75T/DiVs/BB5UDGtJvAKhKPuM097sDiW6fUZOYHFrFKYQ5y9pu0Dlgj3 go1lNSQtftxWnc8pIgT9bexrb5IrtorOqcnQfX5n5CkbdVZ9lSZmvUrHhJNlttGx3s phXYBswGDYE+qruo00bdQ83nCIhwMJYcCX4PTvi4HtR21J6j/kCA3KSUsRjtZuT/cr 0kQa1AHRxdpKKDIMOmgUjO4VcMl2PxG3vtMEDR41p/UTZ05l9/O8cNuPxzzopWK+or F0osdwWgs8io0eK2jKPRFJisr565qCNZ9cPI7OSiuKMPlUTADqqtImqDCJUDk/rEO2 +zuUDEzTXaQWg== Date: Fri, 14 Jul 2023 09:23:48 +0100 From: Will Deacon To: "Aiqun(Maria) Yu" Cc: Mark Rutland , corbet@lwn.net, catalin.marinas@arm.com, maz@kernel.org, quic_pkondeti@quicinc.com, quic_kaushalk@quicinc.com, quic_satyap@quicinc.com, quic_shashim@quicinc.com, quic_songxue@quicinc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH] arm64: Add the arm64.nolse_atomics command line option Message-ID: <20230714082348.GA5240@willie-the-truck> References: <20230711082226.GA1554@willie-the-truck> <84f0994a-26de-c20a-a32f-ec8fe41df3a3@quicinc.com> <20230711102510.GA1809@willie-the-truck> <67c2621f-4cad-2495-9785-7737246d3e90@quicinc.com> <604ac52d-4336-744f-2ab8-44d1c93fbaa8@quicinc.com> <6d1a6691-f858-71bf-97fe-97e13fcb93b6@quicinc.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <6d1a6691-f858-71bf-97fe-97e13fcb93b6@quicinc.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230714_012356_681505_8B248A0D X-CRM114-Status: GOOD ( 44.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Jul 14, 2023 at 09:56:27AM +0800, Aiqun(Maria) Yu wrote: > On 7/14/2023 3:08 AM, Mark Rutland wrote: > > On Thu, Jul 13, 2023 at 10:08:34PM +0800, Aiqun(Maria) Yu wrote: > > > On 7/13/2023 7:20 PM, Mark Rutland wrote: > > > > Are you saying that LSE atomics to *cacheable* mappings do not work on your > > > > system? > > > > > > > > Specifically, when using a Normal Inner-Shareable Inner-Writeback > > > > Outer-Writeback mapping, do the LSE atomics work or not work? > > > *cacheable* mapping have the LSE atomic is not working if far atomic is > > > performed. > > > > Thanks for confirming; the fact that this doesn't work on *cacheable* memory is > > definitely a major issue. I think everyone is confused here because of the > > earlier mention of non-cachable accesses (which don't matter). > > > Maybe I can have the information collected in a summary to see if that > helps. > > I know that some CPU implementations have EL3 control bits to force LSE atomics > > to be performed near (e.g. in Cortex-A55, the CPUECTLR.ATOM control bits), > > which would avoid the issue while still allowing the LSE atomics to be used. > > > > If those can be configured in EL3 firmware, that would be a preferable > > workaround. > > > > Can you say which CPUs are integrated in this system? and/or can you check if > > such control bits exist? > > We have CPUECTLR_EL1.ATOM bit can force LSE atomics to be perform near. > CPUECTLR_EL1 is also an option to EL1 kernel drivers to be configuarable. > > Try to a detailed summarise of the whole discussions, anyone can ignore some > part if you are already know. > > * Part 1: Solution for this issue. > While we still want to have options to let third party and end users can > have options: > 1.Disable lse atomic cap. > 2.*Disallow* far atomic by "CPUECTLR_EL1.atom force near atomic" and > non-cachable mappling for lse atomic only. Sorry, but this still isn't making sense to me. Which CPUs do you have on this SoC? My understanding of the CPUs from ARM is that LSE atomics are not supposed to be sent to blocks that don't support them. That doesn't mean you have to do everything near, however -- you can still execute them at e.g. L2. For example, the Cortex-X1 TRM states: | Atomic instructions to cacheable memory can be performed as either | near atomics or far atomics, depending on where the cache line | containing the data resides. | | When an instruction hits in the L1 data cache in a unique state, then | it is performed as a near atomic in the L1 memory system. If the atomic | operation misses in the L1 cache, or the line is shared with another | core, then the atomic is sent as a far atomic on the core CHI interface. | | If the operation misses everywhere within the cluster, and the | interconnect supports far atomics, then the atomic is passed on to the | interconnect to perform the operation. | | When the operation hits anywhere inside the cluster, or when an | interconnect does not support atomics, the L3 memory system performs | the atomic operation. If the line is not already there, it allocates | the line into the L3 cache. This depends on whether the DSU is configured | with an L3 cache. So something doesn't add up. > * Part 2: Why we need the solution > 1. There is also some case far atomic is better performance than near > atomic. end user may still can still try to do allow far atomic. > while this driver is also use kerenl LSE ATOMIC macro, so it can be running > on cpu don't support lse atomic and cpu support lse atomic already. > while current system, cpu have feature register said lse atomic is > supported, but memory controller is not supported is currently not yet > supported. I think you're forgetting the fact that these instructions can be executed by userspace, so the kernel option is completely bogus. If you're saying that cacheable atomics can cause external aborts, then I can write an app which will crash your device even if you've set this command line option. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel