From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2FE7E77198 for ; Tue, 7 Jan 2025 23:39:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 506CF6B0088; Tue, 7 Jan 2025 18:39:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B6756B0089; Tue, 7 Jan 2025 18:39:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37E0B6B008A; Tue, 7 Jan 2025 18:39:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1A31C6B0088 for ; Tue, 7 Jan 2025 18:39:16 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C2E3E1C7F0E for ; Tue, 7 Jan 2025 23:39:15 +0000 (UTC) X-FDA: 82982274270.28.5B0F8C9 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf04.hostedemail.com (Postfix) with ESMTP id D7BCA40003 for ; Tue, 7 Jan 2025 23:39:13 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="eZC/7Ukm"; spf=pass (imf04.hostedemail.com: domain of levymitchell0@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=levymitchell0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736293153; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nRG2Q26lwxOG4ZhPhifd7QSAEVOV9ie22DwNBCukzpA=; b=d1XroSnrofy9v2jTgqJA2e30ND6UzLBYv8wtSu260sECH7JIVrieWk6x7qAjsm2wbIv88l cXqUES/6ZqGWrubR6TC06TTB9Rjpke1NELI58RN8BNcolXFXBJTj5iS3OrxP15cj0D8UKW hvHlcYczJMsGJHrsfjUv+eksoAwv4Aw= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="eZC/7Ukm"; spf=pass (imf04.hostedemail.com: domain of levymitchell0@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=levymitchell0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736293153; a=rsa-sha256; cv=none; b=s9kOzqNDfWdg8K5C9SFb2DFYvXfNZ9cRjuWKVv+f6elKBZstvVDgb8AxTvKJypNquYs/l8 amGkNcQNTuE3acG8OTMVtm4dXp6LpB5wIxLNFIyY19JiLNx2sihgGLJbYh0rreRAtQtuOq hSgkTO2esmObrxCHo80OKfUPmfeqdu0= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-219f8263ae0so189734115ad.0 for ; Tue, 07 Jan 2025 15:39:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736293152; x=1736897952; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:subject:cc :to:from:date:message-id:from:to:cc:subject:date:message-id:reply-to; bh=nRG2Q26lwxOG4ZhPhifd7QSAEVOV9ie22DwNBCukzpA=; b=eZC/7UkmSlBHamjwvqCmy+RyMiQorT25VOnxG0wI0mJDNMMPRC1TiOHt+PG5IZhtu5 h1FqYc2VrM5O7GgfjWFnPEondazc5ydPGT4WuZiKTd45dzjsh8WdWnlpvOav5LUjAO5R 00Ln8cnrA8oNmDUepAzoUDOCzmJTlquBgnePjQbf5I0E1Q5HBD49HwmGw+tDsSvi7GUN aqV9RY55/uySXumPGKZC6/AzS5u5U0jCTqVwReLQVXpFbcFum2k0gtyVFiArq2OOVLPr GnIkEJ1y/dcgq+eujmEsh1RLc5pgR/NJXYRcDAt9EsWquQODUChy/nUbuUz8kaWXbPLp PrAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736293152; x=1736897952; h=in-reply-to:content-disposition:mime-version:references:subject:cc :to:from:date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nRG2Q26lwxOG4ZhPhifd7QSAEVOV9ie22DwNBCukzpA=; b=wyB92oc50lRamheWSHqMKu8ZVjP7WWATxC8IzgNn5D6VLApnQHD6PVkoDKiuf3s3Qx Xv/7nVS5KfvBRDU9HsF5CN2UldFLjyrY7oKPpOqmYESSW8F31FYeMhMYZNv1GN5wR+dg H2BaNfNO3au6LCi48ZcG5hYX5Hq48XgiUSwyRPGwji4/M/Ft2in/5COcmbfG7gu+MiSq OJsHJxbbyZ+s9zI5EePtbmDOndGt7RpsKK5idTO40RtUeODpAWXDuYSKLYY7QI/XDEXZ PtG5XbsbDnjE51mQSHrYzDCF0EpYu7P00WrIo95Uirk3HGFGEMMLl2j7oIfMflIIkeQC NGiw== X-Forwarded-Encrypted: i=1; AJvYcCV33NW2w4d2A54pUel07J6HOcSUo4yJiY9eGAvW36r9hiXZOrF0QhJYnfqYCZ8Oef6x2114uP3Y/A==@kvack.org X-Gm-Message-State: AOJu0YzZ3EQr1rt9a+QN84A2RQXbubFK/ka/ilAAUPxu1YKcC/y2Yys4 v/TzpKDhD51QL2OgISvH5b41G+XuuWSVNCKELMc4+iFItIMy79OA X-Gm-Gg: ASbGnctUpVU+ZdvMmmU1toNeY/stPWD27DdMWnnc7r5KvmMwHw2FE3Voer7Frm/ZWzp 3gkQfAYWT7f0Caknddi9AchzbPzqtFAFEg5BOaupPOkmYC+aid5NjRW20p/Y/eMSqQ1YXHNNX3m Er5nZbcid29zESybvA7N8jC0cZ/dh8D8eQppS4BrV9Y4z339aT/hTVI0sMZ2HVxcWrGqzPo61nF kVTP2Z1A6nQL4paVdcAMqWwSJFqFi4Nm63SuOVCageh/XqE7bK4IACqnHIl1d7PQIx4erCfxqFt kvkt6nO97bo= X-Google-Smtp-Source: AGHT+IEFKWH9blqwhs5UIzeKV2UjiBvuwYrNIKw3yW57Smk9dySUEDMO2gKtAXjXY7kl90vWv3tYTQ== X-Received: by 2002:a05:6a20:6a0a:b0:1e0:d380:fe71 with SMTP id adf61e73a8af0-1e88cf079f0mr1780991637.0.1736293152195; Tue, 07 Jan 2025 15:39:12 -0800 (PST) Received: from Cyndaquil. (71.sub-174-227-40.myvzw.com. [174.227.40.71]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad90c18fsm35371336b3a.184.2025.01.07.15.39.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 15:39:11 -0800 (PST) Message-ID: <677dbb1f.050a0220.14d4ee.e7df@mx.google.com> X-Google-Original-Message-ID: Date: Tue, 7 Jan 2025 15:39:07 -0800 From: Mitchell Levy To: Dennis Zhou Cc: Hudson Ayres , Miguel Ojeda , Alex Gaynor , Boqun Feng , Gary Guo , =?iso-8859-1?Q?Bj=F6rn?= Roy Baron , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Andrew Morton , Tejun Heo , Christoph Lameter , linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH RFC 0/3] rust: Add Per-CPU Variable API References: <20241219-rust-percpu-v1-0-209117e822b1@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D7BCA40003 X-Rspam-User: X-Stat-Signature: sm3bxm1ynqas5po1aes57g3oq6dxec7f X-HE-Tag: 1736293153-234625 X-HE-Meta: U2FsdGVkX1+EVsw9c2HxDRnYqC9sc1Iq10YmzpqRb3DA4e2K7QC+mNTUV9vRXStLD/Jknia8a7anyrFZz3hzyuXIQf6/tK+5Q6oyfGo1IjbAYw4fvBRBa3Teu3zGvzNONU0ztp+ZMsNrPhJ9DC9KXATgCMb63fCNS4mG6Hspt+Oc0pN7WBLI3Pp7Z2VzETaP7mAbPyG07QFWtf7hDgVRKEhnxycmaauACPeUCagOktTmlDqf+TkU3SYZlyl1iP5AvM34doIz/u/Zwi35wQkndM6EJWpZHbnxKmrU28lykRAwkdt4Vw/c3lViwgZkeMTTp8k80JAfWiLvTb7SDLGGfg8hv4+ITmsTdamQb7KFKicWpRvwbL7CGkz6dw3mtWR68ho/Fv7nbc4+Q2yqBnTQE63LMua6OHFteJU5k37ZvzjSi0/UMChx2j+Xzl1ChXaPu712TgymLwAe7Vvg3FrjPMZlGfSH409pCbrJugWW0khCDcWvMyY5NFQLTDR8oaBcNioa3w7xoQRFKmY3n8irm0YTKqKlVXek7nlAYipYeXgveHR9VUCC6TWXWtk7pAjFGTFmsPMdE4WhhrKQbamh0jQrwGSSHJtVAWHGEHk0jznic3sUyBfcbGh2COj+3YXB7NM8tpv8XHgQBN2FxmPC8C/VX2OrIRXyMP7mV2tKTBa4uMOmxDcgGI8H+0W+J9UkwhCqJlbLe8WJtVKHj7CIUhOFMTtXQo5/vnL1hO7nxiYm4khhhJEEn79a2MM+nnNPkPGY5AfPyM6WBy4n1c3EhlRKMx+34xvTUxq2OEfmW18IrjZ4sPkqAI4ubN3m41DJKQgYN/JIMF7Mal3J94WkRRFt+FLIwAbcRSfmDz7JIqB3CrYOpyxu4zoJEErTAnm72D1Yepf+gVI7uiU1GSm6AL1l8vqVhTjRS+gU4MJTB+1pNc3PazW8zELYf6g/ufslRv5UZz4+E+JwwiN5Aes sXfbsv51 V/5zSRlOcxUCXOpqDnL100WnU5uAG1bdKppfoRpJ4NSVug3jxEutwwKaEwevIiNDqPkdOPNqeXs5uxBSpnEvZvMAeJEkyQLnh9L6VgWQ6B2JlTaYgg5l4jF/KUFt9117VKv6NmHi33xi2/yMyuf28+oUcX2OTER+ndR8V7B1XJGnnbeYXYBEZNAdJDq5DHno4WBfp72PHm2YBTKZIwxv/xIXsVUVk3FMJJakXZEyUmWU0dQ2a4W3PZf4neN9vPsMbc3AUxrHxJk9bnC4Uioq1ocOe7xD8YQP6FAr/XZMBnxb/G8KZztmbXl4A0oIngPYfP+S1QsPQVWHjwDetM38K5ptghnocxZLGXKZGKb0zFa7qB/5F/XEylNSQ38me0oaEUVA66bivcHD6LAl0pmmBAOvEL4h8PAK/Klka+u1cnKG6sEagtVuDpcZRqn9z6Zlm73BbWq4FGcbbSnM= X-Bogosity: Unsure, tests=bogofilter, spamicity=0.463149, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Jan 04, 2025 at 02:45:43PM -0800, Dennis Zhou wrote: > Hello, > > On Thu, Dec 19, 2024 at 01:08:25PM -0800, Mitchell Levy wrote: > > This series adds an API for declaring an using per-CPU variables from > > Rust, and it also adds support for Rust access to C per-CPU variables > > (subject to some soundness requirements). It also adds a small test > > module, lib/percpu_test_rust.rs, in the vein of lib/percpu_test.c. > > > > --- > > I am sending this patch as an RFC to gather feedback on the API as well > > as get some input on correctness. In particular, the unsafe getter macro > > is somewhat cumbersome, though its safety requirements for pure-Rust > > code aren't too difficult to verify (interoperation with C code is > > certainly much more tricky). > > > > I've been thinking a little bit about this over the holidays and fwiw am > not really the right person to evaluate the rust side of this. I did > briefly discuss this with Hudson (cced) who is more familiar with rust. > > In addition to Christoph's comments: > 1. This implementation seems targeted at static allocations and the > safety of this api relies on the end user doing the right thing. > > There are many percpu variables created dynamically via > alloc_percpu() and the lifetime of those variables are tied to a > larger object. So access to those variables is safe via the > understanding the larger object exists. I feel like there is room > here for rust to do this checking somehow? Maybe in rust-only percpu > variables, but something better nonetheless. You're correct; my hope is that adding dynamic allocations on top of this API should be straightforward. Because we don't have to worry about aliasing concerns (since we're in control of the memory that gets set aside), we can just directly spit out a PerCpuRef. The one wrinkle would be adding a lifetime parameter to PerCpuRef and making everything line up. > 2. As Christoph mentioned, the percpu api relies heavily on macros for > architecture specific optimizations. Your implementation seems > asm-generic correct, but we should find a way to export the > architecture specific percpu macros. > > `this_cpu_off` is a percpu variable that helps x86 `arch_raw_cpu_ptr()`. Definitely! I'm hoping to add some operator overloading to make using the optimized operations really smooth. I didn't want to bite off too much for this RFC, though. > 3. Maybe a good use case to start off with would be percpu_refcount and > how rust can use it or how it can make something like that safer? This looks like a great idea, thank you! > Some of Hudson's direct comments about the Rust implementation: > 1. DerefMut for PerCpuRef is ultimately constructing a reference from a > raw integer without preserving the provenance of the original > pointer in any way (since this integer is being constructed from raw > register reads). I believe this violates Rust's pointer provenance > rules for unsafe code. Caveat: I'm not super familiar with Rust's provenance model. My understanding is that there are no "provenance issues" in a loose sense. What I mean by this is that, if everything is treated as the pointer it "should be" the process looks like: per-cpu subsystem owns per-cpu area -> pointer is stored in this_cpu_area -> we create a new pointer (via an in-bounds offset + shrinking to sizeof(T)) to the contents of the per-cpu variable in question It looks like asm! actually supports reading into pointer types (I had misread the table in the Rust Reference, and thought it only supported reading into integer types) --- this might help avoid discarding provenance during a cast to usize? I'm definitely interested in hearing more on this point. > 2. The safety requirements of unsafe_get_per_cpu_ref seem like they > would often not be possible to verify at the callsite of the macro > without global knowledge -- especially for percpu variables defined > in C code. Is this a misunderstanding? Can you show an example of > where it would be straightforward to build a higher-level safe API > on top of this unsafe one? Yeah interoperation with C per-cpu variables is certainly very difficult. Unfortunately, I think this problem crops up any time you're sharing a piece of memory between Rust and C since the existence of &T that points at some piece of memory is a guarantee to the compiler that the pointed-to memory will not change. This facility is intended more as a trap-door for when there's no better option than as an API building block. > > It was suggested that I base my implementation on Rust's thread-local > > storage API. However, this API requires that for a thread-local T, any > > particular usage must be via a &T. Thus, many times it is required that > > thread-local variables be in the form RefCell or Cell. This has > > some pretty severe disadvantages in the context of per-CPU variables. > > Namely, RefCell may panic if .borrow_mut is called multiple times at > > different points in the call stack. This is similar to the problem > > inherent in my current implementation (that is, unsafe_get_per_cpu_ref > > must be used carefully so as to not create aliasing mut references), but > > this implementation flags this potential problem via an unsafe block and > > the resulting PerCpuRef can be easily passed along and used fairly > > easily. Cell on the other hand requires a copy any time the > > underlying value is used, which is entirely avoided here. > > > > Signed-off-by: Mitchell Levy > > > > I think this is a cool topic and has helped inspire me to learn a bit > more rust. Hopefully I can be more helpful in the future. I appreciate it; thanks to you and Hudson for your help! > Thanks, > Dennis