From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7153C4345F for ; Fri, 19 Apr 2024 19:35:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3ECD46B0092; Fri, 19 Apr 2024 15:35:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 39D726B0093; Fri, 19 Apr 2024 15:35:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23C856B0095; Fri, 19 Apr 2024 15:35:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 07AE76B0092 for ; Fri, 19 Apr 2024 15:35:51 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A3AB01403AF for ; Fri, 19 Apr 2024 19:35:50 +0000 (UTC) X-FDA: 82027286460.19.9E77561 Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf30.hostedemail.com (Postfix) with ESMTP id 6E1B88000B for ; Fri, 19 Apr 2024 19:35:48 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZB19LJYk; spf=pass (imf30.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.222.178 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713555348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MhQJU3CrlVoAGP7ZGiJSNVNFVjdkVbIc18tFWpLQjSo=; b=JwHfU8FoWh/TFBujmlNZiKvC6/EqEreSitPTcBWYX7UzLJ1r9tHjT8rR18K+23Xr056wFa CuuO3Wk5A049bDjVXpOL0HldtcY5CYVwayxuS5WhsVNLCFAZOUwk0tOMwdoK2s/Tw/Vj/p JPxM+yx7QwmPhXwJGZ3SN0PzTHaxh8g= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713555348; a=rsa-sha256; cv=none; b=MT/CwDcLFzPl0TTq7dyVi921LUj9oGb6LyR2/DV1lycbyo52CagQlEWMWRHV6IppLKyhhe p3ZTQa0Sa7FgtGsfnezlhVPnHEvqHnoVgKvyLJVsUxqbLhupL4TUVn+jTueOHWJtgH01tK CtxsCB2ObIYqe8qwRRL+NbO4QyQyDYU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZB19LJYk; spf=pass (imf30.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.222.178 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-78edc3e80e6so162116485a.2 for ; Fri, 19 Apr 2024 12:35:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713555347; x=1714160147; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:from:to:cc:subject:date :message-id:reply-to; bh=MhQJU3CrlVoAGP7ZGiJSNVNFVjdkVbIc18tFWpLQjSo=; b=ZB19LJYkswQ0sgeDp4XnOFFpJSSJdhtUG0c9UAANb0TrHPniej7ySkf1DTcHRbGSC1 ZKpf+UUGcW0ubR5Cw4bzYfoCI5MC+XtFLZxoZvdtLrTRdI0gcgYz4YAnmcTU1OoSwHLa tyleVtwQq0P6OfvX107GRZjXiD1KskM21KTYp2XEMae8B76nQLXwSRcsMUI30G5IVclF iK258o/a2hHzi5AxprsW1BUNyZ757ZUlc1aLqdI3y0N7d5GA+NgLuRSfl+IGu/vktWGz xgyU+MC0BcBQ3aZybpW6xTPbAU3VrZguZ8RFtC67llm3RnFCPaz0Yf84kJF9ldTqLdvl 595A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713555347; x=1714160147; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MhQJU3CrlVoAGP7ZGiJSNVNFVjdkVbIc18tFWpLQjSo=; b=F9PdbsuaZlkDaiOQUSaMnqrNHwhrfWfykq0CsMAKT4sDkUm4HARYKkzum0tRRux9Zd UEs7A3SWwh6tG81GnPP3wXIDX/su/zNPobRRH1YSZ0wH3zd35vHORkqC0A+CDwoEXfYx JCaTqn0wwZisXbl16kYLlIwZMuKJLxG7ZnZVWW8/315ocq4hwfOc/3Z+YcNkaBPwpHFl AEFWUKLh5rsvN1zlz++EIwyXjDtQ/PnPqY25EA1nV88KfbreuL/71QQku7JJOuoiCOYl IM+zMYAF4PhRTC01wRA1it2DETZArz6BimMI+8ciqidFQ0VuQfv3oeQK2Eb18ZZXs7tP tFRg== X-Forwarded-Encrypted: i=1; AJvYcCWIx5z8cMP4oE59pN2u1HuZinwOcLPcdxPfjZVm2H1uUKl7iC6H7BTRVPUVwi4nQc+NEChk4jaY41z4C2BHLBi2uz0= X-Gm-Message-State: AOJu0Yx73P2c8iTxO0/uzWN92IjAKNB514ipq8uxcv+HxcpN+KpfFpVj ht+Kn/wCiRo9I1Rvofykv3DSBvuL1bIFuFJCaCbzbpKCqHn5hsda X-Google-Smtp-Source: AGHT+IGMzxbgo4MC6cWlaiaYeVDGJBl+nICEONZyC/N08VTx98tp07xM/2TX/V8ztom/HuGyYxampA== X-Received: by 2002:a05:620a:d53:b0:78a:724b:af30 with SMTP id o19-20020a05620a0d5300b0078a724baf30mr3536555qkl.24.1713555347483; Fri, 19 Apr 2024 12:35:47 -0700 (PDT) Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id h6-20020a05620a13e600b0078f044ff474sm1874956qkl.35.2024.04.19.12.35.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 12:35:46 -0700 (PDT) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfauth.nyi.internal (Postfix) with ESMTP id 1CE041200032; Fri, 19 Apr 2024 15:35:46 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 19 Apr 2024 15:35:46 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudekvddgudegudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehttdertddttddvnecuhfhrohhmpeeuohhq uhhnucfhvghnghcuoegsohhquhhnrdhfvghnghesghhmrghilhdrtghomheqnecuggftrf grthhtvghrnhephedugfduffffteeutddvheeuveelvdfhleelieevtdeguefhgeeuveei udffiedvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epsghoqhhunhdomhgvshhmthhprghuthhhphgvrhhsohhnrghlihhthidqieelvdeghedt ieegqddujeejkeehheehvddqsghoqhhunhdrfhgvnhhgpeepghhmrghilhdrtghomhesfh higihmvgdrnhgrmhgv X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 19 Apr 2024 15:35:45 -0400 (EDT) Date: Fri, 19 Apr 2024 12:35:18 -0700 From: Boqun Feng To: Benno Lossin Cc: Alice Ryhl , Miguel Ojeda , Matthew Wilcox , Al Viro , Andrew Morton , Kees Cook , Alex Gaynor , Wedson Almeida Filho , Gary Guo , =?iso-8859-1?Q?Bj=F6rn?= Roy Baron , Andreas Hindborg , Greg Kroah-Hartman , Arve =?iso-8859-1?B?SGr4bm5lduVn?= , Todd Kjos , Martijn Coenen , Joel Fernandes , Carlos Llamas , Suren Baghdasaryan , Arnd Bergmann , Trevor Gross , linux-mm@kvack.org, linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, Christian Brauner Subject: Re: [PATCH v6 4/4] rust: add abstraction for `struct page` Message-ID: References: <20240418-alice-mm-v6-0-cb8f3e5d688f@google.com> <20240418-alice-mm-v6-4-cb8f3e5d688f@google.com> <87dc4cdf-ccf6-4b08-8915-313aad313f93@proton.me> <079c88af-2e6d-45fe-bf58-afebbf7583b4@proton.me> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: b5hfgnicwfwkdyigbsc87gier1rh15oi X-Rspamd-Queue-Id: 6E1B88000B X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1713555348-609131 X-HE-Meta: U2FsdGVkX1/E37641n2K6mak8TQOLp0Ac+mNriwmOcjZScKzOI76GazAj0PUOH3Gwp7noFK4EKv4/+tC8QRp2Fc8VDF8vA44GF8THMGxo4gGGIxKVjCRaPIjyXUQu2Pwr/PP3h65apQlE1f2wVkAkjOK8agaXNwc3a6QyaQjMdkbPrdZWmS8k1VhDsC1/vp/fTv/6ci7of9n7x/Hf1aBmxS8dajjKOF8BFcRh0hb0pUFGUcsSCAcwBrQ/aeJV+Pxh/RddjW4EJQyVtdBvfDYx5qiCDNvZJG+v0Nh8ZAoY/fP4nMoKjMHBpn0dC80JXpBtVos3t0JvJJbbKc7PR5k/e/Na+a6gtu+ti2ru0m3e2XmXmxqSjGViT8ncwUGGMYk0TDvQTOgJfdCvOW7NAsfjtAw0X0Ee893gZcIjn7RkxVxKBhtw0w6kJuzw15rgJTfYlyKPy5UCYPvyzupH7CACdejNTUBiSFOFcUvUFYHMKFZqlaD1vUIRSYNvi/1qK6crVHYV7pMT3YwQ1EZB5fngkspfwzrw0UyZXDCRftztN87b7NsgAcKm0H2QZc9EVELPTkeBg/mcx/Vc+TjpMQyVlYl2AdcNy12LIko+As2uCiDDCLGYsL6uaITxVSZ57f7bJl7UPfqt+bMqNO5NFbds8gNhN4G0GTMPYfZjCa6C9dnSO9s2+EiQuJG1MI8AzdfgFGxjuuabsAxD+4TN6UyzfBstl9mN/DMfoyjFID+/0ChKxZkX8I21/kuY/grjZqKMabYZd7ksvQNDQLp+KUsKCr46Z+77S74HHcAQ7OGf9atcJTBJXyNafCynQqCVqltzxkPopUFheXjTh7TtpnFf/muBX/Y1fI7ndh5jlO0iJutGuYPsy1byggnSB4ol+sqDtnXpoj6Nfgs1jgJRiTpV4d5Mnr+HcjKDJULGgMbPl1jD7MB8gPLWmJaDLnf3+ptkTlEK6MmCVVPbTBj07W LS66Fi9B BlcdcFzj4RDwsVK6wB6/W05/u4LvLJh3CnYAt4l5WtAvyIbTLJvcgHMsPTrai2G8sAVBjCntm2SO/hOVBsQK5TSCq2xMui8kE7aAQfpGOPeOGfxA6aJ/85c+/Xc4VOiddOeg5RZfkpbEd1urEbiaPGfGuc3VCFoHOlHe2OIdtBGw67EvRGqZ+D/ohpaWxV8+EXdzo/jzFedRWtZZU17beOuuyRiO4bfjJnjwlehBTWdexFPfbeWhKfcY711Da/WOX5yofFwzIOwi8agOOplqQFfSkgAdx6ZGmr6uzHEAOIS1abZAJrfUHW8mfqizl2QUgFGbwtbxA3d27Q27VdG05QG+Uls9lKn3POnGQigKpdPI2Bp8adPLg5kYQv5TyOxOsfTxmC4mabSJX3rsWuBukmYHn8LENiy9m/iEOEnakvY9F9ilmYIW2e96GNLhdpFUX8gjBYoKMVbH4JZp7qV7PSkiD35lQbATAQs1qVA6BuRqCxd4FTndYTKbB57DBOZRxzslGRqjFA1FlsHE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Apr 19, 2024 at 07:24:31PM +0000, Benno Lossin wrote: > On 19.04.24 19:23, Boqun Feng wrote: > > On Fri, Apr 19, 2024 at 08:36:11AM +0000, Benno Lossin wrote: > >> On 19.04.24 01:04, Boqun Feng wrote: > >>> On Thu, Apr 18, 2024 at 03:56:11PM -0700, Boqun Feng wrote: > >>>> On Thu, Apr 18, 2024 at 10:08:40PM +0000, Benno Lossin wrote: > >>>>> On 18.04.24 20:52, Boqun Feng wrote: > >>>>>> On Thu, Apr 18, 2024 at 08:59:20AM +0000, Alice Ryhl wrote: > >>>>>>> + /// Runs a piece of code with a raw pointer to a slice of this page, with bounds checking. > >>>>>>> + /// > >>>>>>> + /// If `f` is called, then it will be called with a pointer that points at `off` bytes into the > >>>>>>> + /// page, and the pointer will be valid for at least `len` bytes. The pointer is only valid on > >>>>>>> + /// this task, as this method uses a local mapping. > >>>>>>> + /// > >>>>>>> + /// If `off` and `len` refers to a region outside of this page, then this method returns > >>>>>>> + /// `EINVAL` and does not call `f`. > >>>>>>> + /// > >>>>>>> + /// # Using the raw pointer > >>>>>>> + /// > >>>>>>> + /// It is up to the caller to use the provided raw pointer correctly. The pointer is valid for > >>>>>>> + /// `len` bytes and for the duration in which the closure is called. The pointer might only be > >>>>>>> + /// mapped on the current thread, and when that is the case, dereferencing it on other threads > >>>>>>> + /// is UB. Other than that, the usual rules for dereferencing a raw pointer apply: don't cause > >>>>>>> + /// data races, the memory may be uninitialized, and so on. > >>>>>>> + /// > >>>>>>> + /// If multiple threads map the same page at the same time, then they may reference with > >>>>>>> + /// different addresses. However, even if the addresses are different, the underlying memory is > >>>>>>> + /// still the same for these purposes (e.g., it's still a data race if they both write to the > >>>>>>> + /// same underlying byte at the same time). > >>>>>>> + fn with_pointer_into_page( > >>>>>>> + &self, > >>>>>>> + off: usize, > >>>>>>> + len: usize, > >>>>>>> + f: impl FnOnce(*mut u8) -> Result, > >>>>>> > >>>>>> I wonder whether the way to go here is making this function signature: > >>>>>> > >>>>>> fn with_slice_in_page ( > >>>>>> &self, > >>>>>> off: usize, > >>>>>> len: usize, > >>>>>> f: iml FnOnce(&UnsafeCell<[u8]>) -> Result > >>>>>> ) -> Result > >>>>>> > >>>>>> , because in this way, it makes a bit more clear that what memory that > >>>>>> `f` can access, in other words, the users are less likely to use the > >>>>>> pointer in a wrong way. > >>>>>> > >>>>>> But that depends on whether `&UnsafeCell<[u8]>` is the correct > >>>>>> abstraction and the ecosystem around it: for example, I feel like these > >>>>>> two functions: > >>>>>> > >>>>>> fn len(slice: &UnsafeCell<[u8]>) -> usize > >>>>>> fn as_ptr(slice: &UnsafeCell<[u8]>) -> *mut u8 > >>>>>> > >>>>>> should be trivially safe, but I might be wrong. Again this is just for > >>>>>> future discussion. > >>>>> > >>>>> I think the "better" type would be `&[UnsafeCell]`. Since there you > >>>>> can always access the length. > >>>>> > >>>> > >>>> Hmm.. here is the thing, having `&UnsafeCell<[u8]>` means having a `*mut > >>>> [u8]>`, and it should always be safe to get a "length" of `*mut [u8]`, > >>>> right? I haven't found any method doing that, but the length should be > >>>> just a part of fat pointer, so I think getting that is a defined > >>>> behavior. But maybe I'm missing something. > >> > >> There is `to_raw_parts` [1], but that is unstable. (Note that > >> `<[T] as Pointee>::Metadata = usize`, see [2]) > >> > > > > Oh, that's good to know, thank you! ;-) > > > >>> Hmm... but I guess one of the problems of this approach, is how to > >>> construct a `&UnsafeCell<[u8]>` from a pointer and length... > >> > >> We could use `from_raw_parts` [3]. But when making the slice the outer > >> type, we can use a stable function to convert a pointer and a length to > >> a slice [4]. > >> > > > > Yes, but there appears no way to get a pointer with larger provenance > > from a `&[UnsafeCell]`, right? > > What do you mean by "larger provenance"? > Say you have a `&[UnsafeCell]` whose length is 64, what's the proper way to get a `*mut u8` (or any other pointer) has the provenance for the whole 64 bytes, so that you can pass it to a memcpy like function? "larger" means the size of the provenance is larger than u8. > >>>>> Another question would be if page allows for uninitialized bits, in that > >>>>> case, we would need `&[Opaque]`. > >>>>> > >>>> > >>>> Yes, or `&Opaque<[u8>]`. > >> > >> I don't think that putting the slice on the inside is what we want. Also > > > > Hmm.. why? So in `&UnsafeCell<[u8]>` vs `&[UnsafeCell]` case, I > > think the former represent "a slice of u8 that can be modified in the > > same time" very well, and this is what a pointer-and-length pair usually > > represents in kernel, I think. But yes, the latter is OK to me as well, > > just hard to play the provenance game I guess? > > Ultimately it again comes down to missing field projections :) > > The type `&UnsafeCell<[u8]>` is less *useful*, since you cannot even get > the length of the slice. Also indexing into this type is not easily > possible. This is because the only way to get/change the inner value of > an `UnsafeCell` is via `get`. > > Compare this with the slice type. It allows getting the length, indexing > into it (ie a form of field projections, if we consider slices as having > a variable amount of fields). > > All those issues would be solved by (good) field projections. > > > Field projections also give a reason for why using `&[UnsafeCell]` > is not really different from `&UnsafeCell<[u8]>`: At any point in time > we ought to be able to project `&UnsafeCell<[u8]> -> &[UnsafeCell]`. > Right, to me there is no significant difference between these two. Maybe because I'm full field projected minded ;-) > So it's fine to just use that from the get-go. > > >> note that `Opaque` requires that `T: Sized` and that is not the case > >> for `[u8]`. > > > > Oh, you're right. In case of MaybeUninit, it requires `T: Sized`, so > > `Opaque<[u8]>` doesn't quite work. > > > > Moving forward, maybe the first step is to see whether `&[Opaque]` > > and `&[UnsafeCell]` can have a good way to generate a pointer with > > proper provenance? Time to ping t-opsem maybe? > > Good idea, do you want to do that, or should I do it? > A way to get a larger provenance (explained above) is currently my only question, so if you think that's something reasonable to ask, i.e. nothing you know of can help. I will post a message there. Regards, Boqun > -- > Cheers, > Benno >