From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 652151EB1AA; Mon, 21 Apr 2025 16:42:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745253766; cv=none; b=c9Nyh0wTw1jKCKn0nJ8rPD8C+Gi+GF7dRyyFOpLFRANZZG9PIFebLOgoWJ4Sv8u7WOM7o62BJtCCLnMiCqZkw0LuXOTEKe6nlok/1yGlxIMFaLrXmEox5+/IUKXmNlaApHvCjZD8sQAaLSRlE7CcsuryRw7GA2C/VMBRSZIWUlo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745253766; c=relaxed/simple; bh=UGNCuXxFioUqqIrmBuMgTay3Y8/FsLszX3rIkRfGS/g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MFg7ilpLJjLEdGz0gjJ+vV0a8e0U7+VK4oWr2HTk/cewZDv328C3zR99XDAjGmtbNj61rtFtPlsw9wpxp+4PIy/8pHsMDaTZOriENYtrUx7qXUL2yywIDyyk/XsE88MRjmmt+Zw0T3iKJcqmfotl4Ou/1Ax4mtKMdwnxy6NsdzU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=P/KcN0co; arc=none smtp.client-ip=209.85.160.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="P/KcN0co" Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-476ab588f32so61355281cf.2; Mon, 21 Apr 2025 09:42:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745253762; x=1745858562; darn=lists.linux.dev; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:from:to:cc:subject :date:message-id:reply-to; bh=Nnh9FhtmfBr7vF/QXBT4Oa0ku2sBn0D0ph9GeI/BdhY=; b=P/KcN0coPAPoIOcy0FcYVKFxYOT76rp3yDOBXurklhdEHd/dVL/DEC2HB71KE36Pog kkHt6PmeNrKRYvPe371jJtyYUtbKQFdF7mZ50qxzZppTbsc+U3eFqy374JhwqNd5HDMC kHzL54mZHWtyAXdFHcTmlL8ryEH9qc0Mjrdc6694W1N1CM9zyOfvSnpPQSz6Mix2c6VI pLkVxgDNp7oygraGaQkyu5kp4JQJ3oDLnwXQxu8v+62j0qxRxxRTR+BT8mwFvS83WoM2 7M82HjoeDzH0BMqcyMfY2XHD4jU9rnBDXnT0U7sIHYdvTQAZVbbewWjdY5PXuoMTH4rA M8SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745253762; x=1745858562; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:feedback-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Nnh9FhtmfBr7vF/QXBT4Oa0ku2sBn0D0ph9GeI/BdhY=; b=VLVycoTRxTocutF+88GZIeULNR7+YftrQbgJndpWoSuZEhw+gA5cYCDvM+ZdZ7h9rH 9F4JTjJ4ZelpP0EO0L31ywEGxBuM4955fha8etmO961jerya91AkbdeIRDVUhY+fEtKb 1N38eEUD49ndMWWbjPpH+GSAYdBntoSzdiKjFL2N34SQTE6RFYoubDzbGb8afQATmKaZ IpUURzTS/QG/5HaFoP2J5nWGy04AQ7guF8AgGLTeh1AJ8dmycCnhwwdE34GW1EWhZUPd DcASniOWAxcg0cZQlKBT7VKbpe5niDzWz2HGcsyxbpD1moTvA5hHWl+qtfd7XageRbDH 35yA== X-Forwarded-Encrypted: i=1; AJvYcCUnIHdi9syhikYymnYVqAJ0qCDRsSRXley1XgG5Isr7eZt+YN3ENCsgqin4ds9uszrPZOme@lists.linux.dev, AJvYcCVkYeV5p9PHzbqW6gf0/1c2NM8z286bnEBrhA+fAGaRhBF9n5NSoj9v9tQAT9/JpVc41i+zYQ==@lists.linux.dev X-Gm-Message-State: AOJu0Yy3hsad+se7sdbk9ScTOsIJ8ij5fEwcOwqX+jjxr+KCMQh2hwWQ VnNN+pI0bSo0UNu2hfJyUEBZ54xL/usnGc3ZcJwSlWnXBgH7OjyX X-Gm-Gg: ASbGncsY4RZLZ38JYbzhYT5G2EcrJ1C0DfS5hKWUT/4ZMzst3IsOWJkGofsIUSMwGOi GFflVF9yt71hyBbPbeM7x3KP0kqLjukqbQwzGSGL0OXaZSTnleziCuR6AjfcA1sAysMWFPt58UQ n6Zmp4cU2le4iR+Bcz66XErQWupyy2YCjBV6vj9qGMp2/gopRnzwTmMOySp7bXSZJBe5FJQ1dyf UblGoI+NX+LY/HkfvgxT4ri6F9eSW7/y5V6pqGQRukm3xLn1/vIAcHdk9hiTe3Zi8uF4l4cJSz8 OdcL/uzm8sGBhCfWr2LAGEK15gi3wh5o0jxqbmE3L/jlApXQHIyxFa6dHPClvmanFEwtBsmH7SY cxT4Va/J8gz/ogBiN+dY9CxJeiKWGBL8= X-Google-Smtp-Source: AGHT+IHC/Dq56MCJluGVIJBeEpkDMocnnFXsmiBG2wnX8WM9F2Rlnt1zCMigH9mIoQi7juMLhqL5WA== X-Received: by 2002:ac8:5f51:0:b0:477:6f6d:607a with SMTP id d75a77b69052e-47aec365c6dmr179026611cf.7.1745253761990; Mon, 21 Apr 2025 09:42:41 -0700 (PDT) Received: from fauth-a2-smtp.messagingengine.com (fauth-a2-smtp.messagingengine.com. [103.168.172.201]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-47ae9c4c608sm44836131cf.41.2025.04.21.09.42.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Apr 2025 09:42:41 -0700 (PDT) Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfauth.phl.internal (Postfix) with ESMTP id B3FC11200043; Mon, 21 Apr 2025 12:42:40 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Mon, 21 Apr 2025 12:42:40 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvgedufeeiucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggv pdfurfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpih gvnhhtshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredt tdenucfhrhhomhepuehoqhhunhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrih hlrdgtohhmqeenucggtffrrghtthgvrhhnpeegleejiedthedvheeggfejveefjeejkefg veffieeujefhueeigfegueehgeeggfenucevlhhushhtvghrufhiiigvpeefnecurfgrrh grmhepmhgrihhlfhhrohhmpegsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgr lhhithihqdeiledvgeehtdeigedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppe hgmhgrihhlrdgtohhmsehfihigmhgvrdhnrghmvgdpnhgspghrtghpthhtohepheeipdhm ohguvgepshhmthhpohhuthdprhgtphhtthhopehruhhsthdqfhhorhdqlhhinhhugiesvh hgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehrtghusehvghgvrhdrkhgvrhhn vghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrh hnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqrghrtghhsehvghgvrhdrkhgvrhhn vghlrdhorhhgpdhrtghpthhtoheplhhlvhhmsehlihhsthhsrdhlihhnuhigrdguvghvpd hrtghpthhtoheplhhkmhhmsehlihhsthhsrdhlihhnuhigrdguvghvpdhrtghpthhtohep ohhjvggurgeskhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghlvgigrdhgrgihnhhorh esghhmrghilhdrtghomhdprhgtphhtthhopegsohhquhhnrdhfvghnghesghhmrghilhdr tghomh X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 21 Apr 2025 12:42:39 -0400 (EDT) From: Boqun Feng To: rust-for-linux@vger.kernel.org, rcu@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, llvm@lists.linux.dev, lkmm@lists.linux.dev Cc: Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?UTF-8?q?Bj=C3=B6rn=20Roy=20Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , Joel Fernandes , Nathan Chancellor , Nick Desaulniers , kent.overstreet@gmail.com, Greg Kroah-Hartman , elver@google.com, Mark Rutland , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Catalin Marinas , torvalds@linux-foundation.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org, Trevor Gross , dakr@redhat.com, Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Uladzislau Rezki , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Subject: [RFC v3 06/12] rust: sync: atomic: Add the framework of arithmetic operations Date: Mon, 21 Apr 2025 09:42:15 -0700 Message-ID: <20250421164221.1121805-7-boqun.feng@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250421164221.1121805-1-boqun.feng@gmail.com> References: <20250421164221.1121805-1-boqun.feng@gmail.com> Precedence: bulk X-Mailing-List: llvm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit One important set of atomic operations is the arithmetic operations, i.e. add(), sub(), fetch_add(), add_return(), etc. However it may not make senses for all the types that `AllowAtomic` to have arithmetic operations, for example a `Foo(u32)` may not have a reasonable add() or sub(), plus subword types (`u8` and `u16`) currently don't have atomic arithmetic operations even on C side and might not have them in the future in Rust (because they are usually suboptimal on a few architecures). Therefore add a subtrait of `AllowAtomic` describing which types have and can do atomic arithemtic operations. A few things about this `AllowAtomicArithmetic` trait: * It has an associate type `Delta` instead of using `AllowAllowAtomic::Repr` because, a `Bar(u32)` (whose `Repr` is `i32`) may not wants an `add(&self, i32)`, but an `add(&self, u32)`. * `AtomicImpl` types already implement an `AtomicHasArithmeticOps` trait, so add blanket implementation for them. In the future, `i8` and `i16` may impl `AtomicImpl` but not `AtomicHasArithmeticOps` if arithemtic operations are not available. Only add() and fetch_add() are added. The rest will be added in the future. Signed-off-by: Boqun Feng --- rust/kernel/sync/atomic/generic.rs | 102 +++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/rust/kernel/sync/atomic/generic.rs b/rust/kernel/sync/atomic/generic.rs index 73aacfac381b..2de4cdbce58e 100644 --- a/rust/kernel/sync/atomic/generic.rs +++ b/rust/kernel/sync/atomic/generic.rs @@ -3,6 +3,7 @@ //! Generic atomic primitives. use super::ops::*; +use super::ordering; use super::ordering::*; use crate::types::Opaque; @@ -55,6 +56,23 @@ fn from_repr(repr: Self::Repr) -> Self { } } +/// Atomics that allows arithmetic operations with an integer type. +pub trait AllowAtomicArithmetic: AllowAtomic { + /// The delta types for arithmetic operations. + type Delta; + + /// Converts [`Self::Delta`] into the representation of the atomic type. + fn delta_into_repr(d: Self::Delta) -> Self::Repr; +} + +impl AllowAtomicArithmetic for T { + type Delta = Self; + + fn delta_into_repr(d: Self::Delta) -> Self::Repr { + d + } +} + impl Atomic { /// Creates a new atomic. pub const fn new(v: T) -> Self { @@ -374,3 +392,87 @@ fn try_cmpxchg(&self, old: &mut T, new: T, _: Ordering) -> bool { } } } + +impl Atomic +where + T::Repr: AtomicHasArithmeticOps, +{ + /// Atomic add. + /// + /// The addition is a wrapping addition. + /// + /// # Examples + /// + /// ```rust + /// use kernel::sync::atomic::{Atomic, Relaxed}; + /// + /// let x = Atomic::new(42); + /// + /// assert_eq!(42, x.load(Relaxed)); + /// + /// x.add(12, Relaxed); + /// + /// assert_eq!(54, x.load(Relaxed)); + /// ``` + #[inline(always)] + pub fn add(&self, v: T::Delta, _: Ordering) { + let v = T::delta_into_repr(v); + let a = self.as_ptr().cast::(); + + // SAFETY: + // - For calling the atomic_add() function: + // - `self.as_ptr()` is a valid pointer, and per the safety requirement of `AllocAtomic`, + // a `*mut T` is a valid `*mut T::Repr`. Therefore `a` is a valid pointer, + // - per the type invariants, the following atomic operation won't cause data races. + // - For extra safety requirement of usage on pointers returned by `self.as_ptr(): + // - atomic operations are used here. + unsafe { + T::Repr::atomic_add(a, v); + } + } + + /// Atomic fetch and add. + /// + /// The addition is a wrapping addition. + /// + /// # Examples + /// + /// ```rust + /// use kernel::sync::atomic::{Atomic, Acquire, Full, Relaxed}; + /// + /// let x = Atomic::new(42); + /// + /// assert_eq!(42, x.load(Relaxed)); + /// + /// assert_eq!(54, { x.fetch_add(12, Acquire); x.load(Relaxed) }); + /// + /// let x = Atomic::new(42); + /// + /// assert_eq!(42, x.load(Relaxed)); + /// + /// assert_eq!(54, { x.fetch_add(12, Full); x.load(Relaxed) } ); + /// ``` + #[inline(always)] + pub fn fetch_add(&self, v: T::Delta, _: Ordering) -> T { + let v = T::delta_into_repr(v); + let a = self.as_ptr().cast::(); + + // SAFETY: + // - For calling the atomic_fetch_add*() function: + // - `self.as_ptr()` is a valid pointer, and per the safety requirement of `AllocAtomic`, + // a `*mut T` is a valid `*mut T::Repr`. Therefore `a` is a valid pointer, + // - per the type invariants, the following atomic operation won't cause data races. + // - For extra safety requirement of usage on pointers returned by `self.as_ptr(): + // - atomic operations are used here. + let ret = unsafe { + match Ordering::ORDER { + ordering::OrderingDesc::Full => T::Repr::atomic_fetch_add(a, v), + ordering::OrderingDesc::Acquire => T::Repr::atomic_fetch_add_acquire(a, v), + ordering::OrderingDesc::Release => T::Repr::atomic_fetch_add_release(a, v), + ordering::OrderingDesc::Relaxed => T::Repr::atomic_fetch_add_relaxed(a, v), + } + }; + + T::from_repr(ret) + } +} -- 2.47.1