From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1BF1C433DB for ; Wed, 3 Mar 2021 22:44:22 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 094D964EED for ; Wed, 3 Mar 2021 22:44:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 094D964EED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=marcan.st Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Date:Message-ID:Subject:From:Cc:To: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=UqRREqtqfHtx81nswj/mZM4Wo1uDVmE0cSO6ZxldlAM=; b=cwAt+fsp42bIARGkZSFciU7N4z ybYOB1tpk23vG562oY41SGyRdyR1pLgeFZsL2Rqcaaz47y4GgIqviyIt7wXL+KgML0K0Y8azJAEu0 3T082AAKWYKocvxf8L3fr3jfzL5lL/RvIfXDOR1poyipOT7DJX/CMLXZriQNiA1Nuaq3KUaSqB4ei MsOgQM7RydfSnFHogKJajNCdJxE2W+9FA7N1nlTWniLpDH1gZ5ID+pYDeMONgkBYsTbMga8ch7jbk 5tc/uK61ZUIjaBUwJzX9MkUHeIM+Y1daT+krIVKfuHG87Vudby48fJocCvYm9ZSb9kYM3sIwL2r4p JF7nqtig==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHa8C-006mCK-Bc; Wed, 03 Mar 2021 22:38:12 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHTB6-005Kpt-HU for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:12:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Date:Message-ID:Subject:From:Cc:To:Sender:Reply-To:Content-ID: Content-Description:In-Reply-To:References; bh=LHO0MZb4IQQ7qu5cp294y81LBoTQ7GKGmtlEqL7QfC8=; b=cpa+EmXnIRIfHsm/vyZ3j0d7ZF snl4F1UEQF8mJ/0JNYHu1ZOFau3j/ibh1393+r+l59fJGQlTFrBsz2GbgOlqOa3wZzr5P9hiW4kk6 rsRefVOkntsCikhKWkf55Cl7LnN/6uaU9mIFEVRT65blNFyQ8AyzO9q62FMhqZQ2lMZdypN64Oo4l E9xIBwpuc1/qiggfxyzVoWMeXygCo3q8oMcL4TCAoFv7xl8PMlGte0AdhIQYYQWWDsVoO7BIVbfIF aMM0OUWERLmla3Qn5ixuS/UgNDy7isIq8PYGAgQ0vfJAOU0kJyspjmB/ysS/E+hmLvPQYqtahSyeQ 9dCmr6nA==; Received: from marcansoft.com ([212.63.210.85] helo=mail.marcansoft.com) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHRII-002gRY-NF for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 13:12:03 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: marcan@marcan.st) by mail.marcansoft.com (Postfix) with ESMTPSA id B71603FA1B; Wed, 3 Mar 2021 13:05:22 +0000 (UTC) To: Will Deacon , Linux ARM Cc: Marc Zyngier , Mark Rutland , Peter Zijlstra , Boqun Feng , Catalin Marinas From: Hector Martin Subject: LSE atomic op ordering is weaker than intended? Message-ID: <90ea3e27-b2ef-41ac-75c7-5f0686603b2a@marcan.st> Date: Wed, 3 Mar 2021 22:05:19 +0900 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 Content-Language: es-ES X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_131203_447787_821D6014 X-CRM114-Status: GOOD ( 11.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Will and everyone else, While yak shaving the AIC driver ordering minutiae, I came across this. atomic_t.txt describes "fully ordered" atomic ops as follows: > Fully ordered primitives are ordered against everything prior and > everything subsequent. Therefore a fully ordered primitive is like > having an smp_mb() before and an smp_mb() after the primitive. And among those ops are the atomic_fetch_* ops. These are implemented as e.g. LDSETAL, with acquire-release semantics. However, the *AL LSE ops have acquire semantics *for the read* and release semantics *for the write*. As independent components of the same atomic op, I cannot find anything in the ARM ARM that would imply ordering between the Load-Acquire and *prior* memory operations, nor ordering between the Store-Release and *subsequent* memory operations. So it would seem these ops are not in fact fully ordered, but rather, only order the read component against prior ops, and the write component against subsequent ops. Put another way: the current implementation means that unqualified ops are equal to _acquire + _release semantics as they are described in atomic_t.txt, but that is weaker than "fully ordered". Throwing this litmus test at herd7 seems to confirm this theory: AArch64 lse-atomic-al-ops-are-not-fully-ordered "" { 0:X1=x; 0:X3=y; 1:X1=x; 1:X3=y; } P0 | P1 ; MOV X0, #1 | MOV X0, #1 ; LDSETAL X0, X2, [X1] | LDSETAL X0, X2, [X3]; LDR X4, [X3] | LDR X4, [X1] ; exists (0:X4=0 /\ 1:X4=0) The positive result goes away adding a DMB ISH (i.e. smp_mb()) after the atomic ops, which contradicts the atomic_t.txt claim. Did I miss something, or is this in fact an issue? (And while I'm talking to the right people: this issue aside, do atomic ops on Normal memory create ordering with Device memory ops, or are there no guarantees there due to the fact that Normal memory is mapped inner-shareable and the ordering guarantees thus do not extend to outer-shareable Device accesses? My currenty understanding is the latter, but I find the ARM ARM wording hard to conclusively grok here.) -- Hector Martin (marcan@marcan.st) Public Key: https://mrcn.st/pub _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel