From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.7 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4532C433DB for ; Tue, 29 Dec 2020 03:32:43 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9BF0F20725 for ; Tue, 29 Dec 2020 03:32:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9BF0F20725 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-Id:MIME-Version:In-Reply-To:References:To: Subject:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JRjNUKlqbP47KfbDWvFvUjBrVlb/rB4inxP6OYS2cow=; b=qBWAg6HvZmy3nrweRemxx2FW9 /OP5l0Ilhyxz+yd8K7/V1WZMzF72DRtJ77FndzRwQ3lZB51pl6jmTzSCAQWigo4nc/XZDYQJuteLV u55x/IctcEKrscnryu+b6tmSsT1WZMOK/uPCBWJgRPH+ff1DIffSd0y3oY5+OloQnV3ilyIWAn4Hp saaJBqmng9YxnQeoIzquA4RZcf4GlODHATUiNWEk7HLstm0Q53H7NPpoF+jgfiTTdv0lOlrGBeZ6D +Y90Ta3GjD8m4B0esgCfw12MbdKhxL7XvLzueTMpOUprRRPKun3d6fonSGlbfRCEMOv6rJeWwRIu+ na5apoNiQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ku5jE-0006Eu-D9; Tue, 29 Dec 2020 03:31:20 +0000 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ku5jA-0006Dt-Mh for linux-arm-kernel@lists.infradead.org; Tue, 29 Dec 2020 03:31:18 +0000 Received: by mail-pj1-x1034.google.com with SMTP id v1so779651pjr.2 for ; Mon, 28 Dec 2020 19:31:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=96UXaeOlfR1Q5+tTEnga5fScpJmHKA3DSdXvEcGqAHw=; b=prkubl66lIi/GgR/wP2FgRAKmNw8Bg1Qki2S0fnVxx7x7FobnN7KbgmoYRMuSYuhdL ZNF6RuxjghqQ3Tsc45xC4FmIhdrI7F3+QKx3lo7FQ3e6xo6jugcvwDjAtQD/+wdEsQji xBZFCqd1tz0BDNby5Qzt7JH4Q9f8TenHXok/NJw3NL0xoj1g1rmYBXz9KPqPQh3ceNig b0ceQRC6DdkLgl7ZWs1Iy+uOphCpO8N+Nts8XZOJLASCrZsKbQKgUYCOfMqUxvPbd2mw dumWFpJW5di66qpB25bWboSpSNvUXaxEG2ToWF4C4/yV3DsR0wn0nPKdvkcJ9XG0WgSR MYoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=96UXaeOlfR1Q5+tTEnga5fScpJmHKA3DSdXvEcGqAHw=; b=py1e9CQ+umEWXogyUcBRfvJ69mIn5olJb7vbgAh/668Tb2dBRl0JwTL3a1x0xua9Ea h4PkehvyM8UP8mvu+jhBSKaPPuFlL/xMxpFrrrxVweqSLZ4UBA+n/WF27aO4z9F87vF6 2qxH7Y0zsU6jNfTrG+VQRwL2ePlxNxqsw8sWYUosQHfeOZOrMts9KJ3c64LRE5NvQ0ap WupDXmF/i5E8kBZYZvzzSyW+cM2Bj37F3nrnTi4+bbdh2mgyoRytMWZRMfGWcp3GSgm9 DDhOsWXpTRmtUV9SjOxfAkoJlHxGu5H5x9BIekw4g9+4BhKJTGo4Xhcsil59KAjbQppL dFew== X-Gm-Message-State: AOAM5322uThYKMoT1r433sRcMAHTr8IL9Wo0kzRYmSnU2eSUdIDIf46s Su44eAZQB/idnytvUGjy5R4= X-Google-Smtp-Source: ABdhPJy5Cp6FEUhm9tRoIQwa5tlkz7MkfEmj0P0PUAmAYPzJvqxnVKoLR4KS6GMs4rxkqs0h/maH8A== X-Received: by 2002:a17:902:eb0c:b029:db:c0d6:6289 with SMTP id l12-20020a170902eb0cb02900dbc0d66289mr47514422plb.12.1609212674478; Mon, 28 Dec 2020 19:31:14 -0800 (PST) Received: from localhost (193-116-97-30.tpgi.com.au. [193.116.97.30]) by smtp.gmail.com with ESMTPSA id w27sm33493195pfq.104.2020.12.28.19.31.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Dec 2020 19:31:13 -0800 (PST) Date: Tue, 29 Dec 2020 13:31:08 +1000 From: Nicholas Piggin Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() To: Andy Lutomirski References: <1609199804.yrsu9vagzk.astroid@bobo.none> In-Reply-To: MIME-Version: 1.0 Message-Id: <1609212362.g5jhvfarip.astroid@bobo.none> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201228_223116_894122_AF6758E9 X-CRM114-Status: GOOD ( 27.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , Benjamin Herrenschmidt , X86 ML , LKML , stable , Will Deacon , Mathieu Desnoyers , Michael Ellerman , Catalin Marinas , Paul Mackerras , linuxppc-dev , linux-arm-kernel Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Excerpts from Andy Lutomirski's message of December 29, 2020 10:36 am: > On Mon, Dec 28, 2020 at 4:11 PM Nicholas Piggin wrote: >> >> Excerpts from Andy Lutomirski's message of December 28, 2020 4:28 am: >> > The old sync_core_before_usermode() comments said that a non-icache-syncing >> > return-to-usermode instruction is x86-specific and that all other >> > architectures automatically notice cross-modified code on return to >> > userspace. Based on my general understanding of how CPUs work and based on >> > my atttempt to read the ARM manual, this is not true at all. In fact, x86 >> > seems to be a bit of an anomaly in the other direction: x86's IRET is >> > unusually heavyweight for a return-to-usermode instruction. >> >> "sync_core_before_usermode" as I've said says nothing to arch, or to the >> scheduler, or to membarrier. > > Agreed. My patch tries to fix this. I agree that the name is bad and > could be improved further. We should define what > membarrier(...SYNC_CORE) actually does and have arch hooks to make it > happen. > >> > So let's drop any pretense that we can have a generic way implementation >> > behind membarrier's SYNC_CORE flush and require all architectures that opt >> > in to supply their own. This means x86, arm64, and powerpc for now. Let's >> > also rename the function from sync_core_before_usermode() to >> > membarrier_sync_core_before_usermode() because the precise flushing details >> > may very well be specific to membarrier, and even the concept of >> > "sync_core" in the kernel is mostly an x86-ism. >> >> The concept of "sync_core" (x86: serializing instruction, powerpc: context >> synchronizing instruction, etc) is not an x86-ism at all. x86 just wanted >> to add a serializing instruction to generic code so it grew this nasty API, >> but the concept applies broadly. > > I mean that the mapping from the name "sync_core" to its semantics is > x86 only. The string "sync_core" appears in the kernel only in > arch/x86, membarrier code, membarrier docs, and a single SGI driver > that is x86-only. Sure, the idea of serializing things is fairly > generic, but exactly what operations serialize what, when things need > serialization, etc is quite architecture specific. Okay, well yes it's x86 only in name, I was more talking about the concept. > Heck, on 486 you serialize the instruction stream with JMP. x86-specific aside, I did think the semantics of a "serializing instruction" was reasonably well architected in x86. Sure it could do other things as well, but if you executed a serializing instruction, then you had a decent set of guarantees (e.g., what you might want for code modification). > >> > +static inline void membarrier_sync_core_before_usermode(void) >> > +{ >> > + /* >> > + * XXX: I know basically nothing about powerpc cache management. >> > + * Is this correct? >> > + */ >> > + isync(); >> >> This is not about memory ordering or cache management, it's about >> pipeline management. Powerpc's return to user mode serializes the >> CPU (aka the hardware thread, _not_ the core; another wrongness of >> the name, but AFAIKS the HW thread is what is required for >> membarrier). So this is wrong, powerpc needs nothing here. > > Fair enough. I'm happy to defer to you on the powerpc details. In > any case, this just illustrates that we need feedback from a person > who knows more about ARM64 than I do. > Thanks, Nick _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel