From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0898C54E58 for ; Mon, 25 Mar 2024 23:42:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IwFWtuuhUKbxmkHpj9Pk6wxBy8Ky4wt9nlhvFeeS81g=; b=FmxLAbE6ySXpjM 6D0G26bky5wc+AkcZ9r5GXujRV3vLEaG2jzEYvEg2wicdTiZ0QnjmTv10w32xv9PrUgZBmQU8q3iE eXJZjdQZznXwRBLi9HpbBivQmYaAhY8P+Bv2Lu5snXE5I9Z/dKFQilGmi483HzblooNdU05G9Z42L LMS0e5C0sUbbBeZRg/3JeW/PVuiVme9dGePt1OpEzGiwEbCkCu+++mifFuUhJh/qYH4aFvF2EGjj7 zsVmGLFOEm1pJt/ZI6IPQY9LcG9qrxRQIzE0Mq9y418lUAXj7nTqCAMi2OZ6F4AsvVM5U4dFne5SN aEkusovvk/S2B+6P8YSw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rotx3-00000002NpA-3kXW; Mon, 25 Mar 2024 23:42:01 +0000 Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rotx0-00000002Nnc-0ICm for linux-arm-kernel@lists.infradead.org; Mon, 25 Mar 2024 23:41:59 +0000 Received: by mail-qv1-xf2b.google.com with SMTP id 6a1803df08f44-6918781a913so41496096d6.3 for ; Mon, 25 Mar 2024 16:41:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711410114; x=1712014914; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:from:to:cc:subject:date :message-id:reply-to; bh=yJGcRjWxujI/xbMJS+9v8/3RkR6IQk/HoBXdHo4IvJM=; b=KS1BD3Tb1wJ1UlQESXGA3so31IXdMdhfXsxoKSMsQJbxld/ZYK4sUBjX/sMEO5bea5 Fsqd0kNiUBCd/Kn4DZ3SukeYlEXBUtthf91UT/nFJenXy5Va90RngS4spJPTXTR8aaOe eSD1exlm5SYDQ6lTXb+3P5eF2Ok8NNqNWzhqmoQtFMOx9obNWE8SjAD3V4okqAzYbaP/ X8frAagASunySaeVwsSii8kcy6fOkrzfQND4JRdBZuu++ARHKUoeFxyswe0VBCdXDBbm qw4ueq6oB20EQyqMdZzuE8NQqJkUu04aMSkx4URfW9x+q/SeQ+7xX1YFdIj84oAAPMFG //Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711410114; x=1712014914; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yJGcRjWxujI/xbMJS+9v8/3RkR6IQk/HoBXdHo4IvJM=; b=ND+tq+GZTBcuj6HUa4pwxg3WZCmp2qdNX5dT0Gy7PEH2FoECE5Y+kOx4HQpjSfL3XV yV143Dez0MRRnrj653TPfGAc5pvMnEHLajya5X0pCBGa0JY+q73uVGC63rpQhscFXlzR sNT16vsHxrkR68zfPmGpgPj+N449jxj+sjxMHsruq3zq2BuDQQN+fl33KMX1p4u7cWPU m+1XfQEz8qp1eDzSoPvHSY34pYAGW4otC+kGhiPb0jmvMaA/eHy/MP9goo3wWLf7GtGb QWk5Bmlt6/iH7dP4ToukuKyuknm+pgCaYTUvaKbdO7RIVeM8fRlNatFbc2gy5MPeuYyM gMYQ== X-Forwarded-Encrypted: i=1; AJvYcCXBUV27/Vqp7hTMRv4jh6mlSPYu3zj4bZ2IzZyMS5QaLrRqA/12LQoVy9Wm+J1NJukl0Q2zyRfHmvflj441V7KLKNg4IM5+V3wxKO/o0Doq7NMnTxQ= X-Gm-Message-State: AOJu0YxwK5joE2aB1oz5EweRCEVUSuJDqpvpu0bQNgHcWZlFe9DMPqOU mKgfEsKY8LBZC5XsalKki9/QzyJjWc4S64Hpvw4yHwRfSXlI9smE X-Google-Smtp-Source: AGHT+IEdPU4dTEy2JZqf6+fOq13iZJz6jJM2/V2qB77B0kCBgeR/Ks/8yVDV8iurZcon/ktLSAu9Dw== X-Received: by 2002:a05:6214:20e7:b0:696:89f6:28d5 with SMTP id 7-20020a05621420e700b0069689f628d5mr5443333qvk.50.1711410113750; Mon, 25 Mar 2024 16:41:53 -0700 (PDT) Received: from fauth2-smtp.messagingengine.com (fauth2-smtp.messagingengine.com. [103.168.172.201]) by smtp.gmail.com with ESMTPSA id pn16-20020a056214131000b00690afbf56d5sm4549850qvb.66.2024.03.25.16.41.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 16:41:53 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfauth.nyi.internal (Postfix) with ESMTP id 0DD1A1200032; Mon, 25 Mar 2024 19:41:52 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 25 Mar 2024 19:41:52 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudduvddgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesthdtredttddtvdenucfhrhhomhepuehoqhhu nhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrdgtohhmqeenucggtffrrg htthgvrhhnpeevvdejffefgeffhfelffeikeeigeehjeekgfefudeugfejfefhteekvedu teelhfenucffohhmrghinheprhhushhtqdhlrghnghdrohhrghdpiihulhhiphgthhgrth drtghomhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pegsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqdeiledvgeehtd eigedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmhgrihhlrdgtohhmsehf ihigmhgvrdhnrghmvg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 25 Mar 2024 19:41:50 -0400 (EDT) Date: Mon, 25 Mar 2024 16:41:19 -0700 From: Boqun Feng To: Kent Overstreet Cc: Linus Torvalds , Philipp Stanner , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, llvm@lists.linux.dev, Miguel Ojeda , Alex Gaynor , Wedson Almeida Filho , Gary Guo , =?iso-8859-1?Q?Bj=F6rn?= Roy Baron , Benno Lossin , Andreas Hindborg , Alice Ryhl , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , Joel Fernandes , Nathan Chancellor , Nick Desaulniers , kent.overstreet@gmail.com, Greg Kroah-Hartman , elver@google.com, Mark Rutland , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Catalin Marinas , linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: Re: [WIP 0/3] Memory model and atomic API in Rust Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240325_164158_151754_AF0B23AE X-CRM114-Status: GOOD ( 72.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Mar 25, 2024 at 07:02:12PM -0400, Kent Overstreet wrote: > On Mon, Mar 25, 2024 at 03:38:32PM -0700, Boqun Feng wrote: > > On Mon, Mar 25, 2024 at 06:09:19PM -0400, Kent Overstreet wrote: > > > On Mon, Mar 25, 2024 at 02:37:14PM -0700, Boqun Feng wrote: > > > > On Mon, Mar 25, 2024 at 05:14:41PM -0400, Kent Overstreet wrote: > > > > > On Mon, Mar 25, 2024 at 12:44:34PM -0700, Linus Torvalds wrote: > > > > > > On Mon, 25 Mar 2024 at 11:59, Kent Overstreet wrote: > > > > > > > > > > > > > > To be fair, "volatile" dates from an era when we didn't have the haziest > > > > > > > understanding of what a working memory model for C would look like or > > > > > > > why we'd even want one. > > > > > > > > > > > > I don't disagree, but I find it very depressing that now that we *do* > > > > > > know about memory models etc, the C++ memory model basically doubled > > > > > > down on the same "object" model. > > > > > > > > > > > > > The way the kernel uses volatile in e.g. READ_ONCE() is fully in line > > > > > > > with modern thinking, just done with the tools available at the time. A > > > > > > > more modern version would be just > > > > > > > > > > > > > > __atomic_load_n(ptr, __ATOMIC_RELAXED) > > > > > > > > Note that Rust does have something similiar: > > > > > > > > https://doc.rust-lang.org/std/ptr/fn.read_volatile.html > > > > > > > > pub unsafe fn read_volatile(src: *const T) -> T > > > > > > > > (and also write_volatile()). So they made a good design putting the > > > > volatile on the accesses rather than the type. However, per the current > > > > Rust memory model these two primitives will be UB when data races happen > > > > :-( > > > > > > > > I mean, sure, if I use read_volatile() on an enum (whose valid values > > > > are only 0, 1, 2), and I get a value 3, and the compiler says "you have > > > > a logic bug and I refuse to compile the program correctly", I'm OK. But > > > > if I use read_volatile() to read something like a u32, and I know it's > > > > racy so my program actually handle that, I don't know any sane compiler > > > > would miss-compile, so I don't know why that has to be a UB. > > > > > > Well, if T is too big to read/write atomically then you'll get torn > > > reads, including potentially a bit representation that is not a valid T. > > > > > > Which is why the normal read_volatile<> or Volatile<> should disallow > > > that. > > > > > > > Well, why a racy read_volatile<> is UB on a T who is valid for all bit > > representations is what I was complaining about ;-) > > yeah, that should not be considered UB; that should be an easy fix. Are > you talking to Rust compiler people about this stuff? I've been meaning Here you go: https://rust-lang.zulipchat.com/#narrow/stream/136281-t-opsem/topic/UB.20caused.20by.20races.20on.20.60.7Bread.2Cwrite.7D_volatile.60/near/399343771 but maybe instead of Rust, LLVM is the best one to talk with on this. Because my take from the communication with Rust folks is more like "it's more up to LLVM on this". > to make my own contacts there, but - sadly, busy as hell. > > > > > > where T is any type that fits in a machine word, and the only operations > > > > > it supports are get(), set(), xchg() and cmpxchG(). > > > > > > > > > > You DO NOT want it to be possible to transparantly use Volatile in > > > > > place of a regular T - in exactly the same way as an atomic_t can't be > > > > > used in place of a regular integer. > > > > > > > > Yes, this is useful. But no it's not that useful, how could you use that > > > > to read another CPU's stack during some debug functions in a way you > > > > know it's racy? > > > > > > That's a pretty difficult thing to do, because you don't know the > > > _layout_ of the other CPU's stack, and even if you do it's going to be > > > changing underneath you without locking. > > > > > > > It's a debug function, I don't care whether the data is accurate, I just > > want to get much information as possible. > > yeah, if you just want the typical backtrace functionality where you're > just looking for instruction pointers - that's perfectly > straightforward. > > > This kinda of usage, along > > with cases where the alorigthms are racy themselves are the primary > > reasons of volatile _accesses_ instead of volatile _types_. For example, > > you want to read ahead of a counter protected by a lock: > > > > if (unlikely(READ_ONCE(cnt))) { > > spin_lock(lock); > > int c = cnt; // update of the cnt is protected by a lock. > > ... > > } > > > > because you want to skip the case where cnt == 0 in a hotpath, and you > > know someone is going to check this again in some slowpath, so > > inaccurate data doesn't matter. > > That's an interesting one because in Rust cnt is "inside" the lock, you > can't access it at all without taking the lock - and usually that's > exactly right. > (Now you mention that, once I was trying to construct a deadlock case with some Rust code, but I couldn't since the locks are naturally hierarchical because of the types, therefore it's impossible to reverse the lock ordering. Yes, you still can have deadlocks in Rust, but that hierarchial type trees really help a lot). > So to allow this we'd annotate in the type definition (with an > attribute) which fields we allow read access to without the lock, then > with some proc macro wizardry we'd get accessors that we can call without > the lock held. > Right, that's field projection: https://internals.rust-lang.org/t/pre-rfc-field-projection/17383 > So that probably wouldn't be a Volatile thing, that'd be coupled with > the lock implementation because that's where the accessors would hang > off of and they'd internally probably just use mem::volatile_read(). > So we can play the type game as deep as we can, and I'm sure it'll be helpful, but in the same time, having a reasonable {read,write}_volatile() semantics on UB and races is also what we need. > > > So the races thare are equivalent to a bad mem::transmute(), and that is > > > very much UB. > > > > > > For a more typical usage of volatile, consider a ringbuffer with one > > > thread producing and another thread consuming. Then you've got head and > > > tail pointers, each written by one thread and read by another. > > > > > > You don't need any locking, just memory barriers and > > > READ_ONCE()/WRITE_ONCE() to update the head and tail pointers. If you > > > were writing this in Rust today the easy way would be an atomic integer, > > > but that's not really correct - you're not doing atomic operations > > > (locked arithmetic), just volatile reads and writes. > > > > > > > Confused, I don't see how Volatile is better than just atomic in this > > case, since atomc_load() and atomic_store() are also not locked in any > > memory model if lockless implementation is available. > > It certainly compiles to the same code, yeah. But Volatile really is > the more primitive/generic concept, Atomic is a specialization. > > > > Volatile would be Send and Sync, just like atomic integers. You don't > > > need locking if you're just working with single values that are small > > > enough for the machine to read/write atomically. > > > > So to me Volatile can help in the cases where we know some memory is > > "external", for example a MMIO address, or ringbuffer between guests and > > hypervisor. But it doesn't really fix the missing functionality here: > > allow generating a plain "mov" instruction on x86 for example on _any > > valid memory_, and programmers can take care of the result. > > You're talking about going completely outside the type system, though. > There is a need for that, but it's very rare and something we really > want to discourage. Usually, even with volatile access, you do know the Hey, in memory ordering model areas, we are supposed to work on these rare cases ;-) These are building blocks for high level synchronization constructions, so I'm entitled to complain ;-) But yes, to your point, type system can help a lot, however, there are still cases we need to roll our own. Regard, Boqun > type - and even if you don't, you have to treat it as _something_ so > Volatile is probably as good as anything. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel