From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7683C00140 for ; Mon, 15 Aug 2022 13:57:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232410AbiHON5B (ORCPT ); Mon, 15 Aug 2022 09:57:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229527AbiHON46 (ORCPT ); Mon, 15 Aug 2022 09:56:58 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C869912D3F for ; Mon, 15 Aug 2022 06:56:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=8H7M++LHjLPagFvc25ZTit0lX+9Q+HaL3iQSNzDSx5I=; b=oPueY7bmQ32LIM+kvz9NC4LcWF ZD8OH1HwyTDACmg3Hu/I8PIzoB78jKBUoOqmXDNaUFoR1wBSaJcM4PR7aqexT9t14F7sUCvPSk6y/ eLCd1hYPIRpm5qbrVJAXCeEAtY2lkGYfKM9lhIFgbH0Oa7rg9altls4wuBad9FmfNICyklfd/s80Y oozevP/mlOKuoPREQj2RpimUfCeNhFUhXwpahsbOeHTDHYvErQ6nH7ob7Kyq1Ww+CsB9hIpS7VZqo pcOq0+38PWUQC7f5AQCyPgE4DftYv1nkk3gfmGIOWEYIF4K+0rsuoyOtSqB88iJWj5ohpH4Mi1eM5 g01TU4tg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=worktop.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1oNaaC-005le7-Dw; Mon, 15 Aug 2022 13:56:44 +0000 Received: by worktop.programming.kicks-ass.net (Postfix, from userid 1000) id 9CB96980153; Mon, 15 Aug 2022 15:56:43 +0200 (CEST) Date: Mon, 15 Aug 2022 15:56:43 +0200 From: Peter Zijlstra To: Will Deacon Cc: Hector Martin , Linux ARM , Greg KH , jirislaby@kernel.org, Marc Zyngier , Mark Rutland , Boqun Feng , Catalin Marinas , Asahi Linux , Oliver Neukum , LKML Subject: Re: Debugging a TTY race condition on M1 (memory ordering dragons) Message-ID: References: <6c089268-4f2c-9fdf-7bcb-107b611fbc21@marcan.st> <20220815134711.GA10374@willie-the-truck> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220815134711.GA10374@willie-the-truck> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 15, 2022 at 02:47:11PM +0100, Will Deacon wrote: > > Behind the scenes, the work pending flag is atomically set with > > test_and_set_bit() by queue_work_on(). That compiles down to my old > > friend LDSETAL, which I already showed [2] does not provide the > > guarantees test_and_set_bit() claims to have (== full memory barrier). > > However, I can't get that litmus test to fail on real hardware, so that > > may be a red herring as far as this bug goes. > > As I mentioned in the thread you linked to, the architecture was undergoing > review in this area. I should've followed back up, but in the end it was > tightened retrospectively to provide the behaviour you wanted. This was > achieved by augmenting the barrier-ordered-before relation with: > > * RW1 is a memory write effect W1 and is generated by an atomic instruction > with both Acquire and Release semantics. > > You can see this in the latest Arm ARM. > > However, test_and_set_bit() is unordered on failure (i.e. when the bit is > unchanged) and uses READ_ONCE() as a quick check before the RmW. See the > "ORDERING" section of Documentation/atomic_bitops.txt. Damn, I forgot that too... :/ > I think you're missing the "shortcut" in test_and_set_bit(): > > if (READ_ONCE(*p) & mask) > return 1; > > old = arch_atomic_long_fetch_or(mask, (atomic_long_t *)p); > > so if the bit is already set (which I think is the 'ret == false' case) > then you've only got a control dependency here and we elide writing to > B. Given all that, I think workqueue wants to be fixed, it really does seem to rely on full ordering for it's test_and_set_bit() usage.