From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30172CF6499 for ; Sun, 29 Sep 2024 15:12:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CABD6B00EB; Sun, 29 Sep 2024 11:12:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 651A86B00EE; Sun, 29 Sep 2024 11:12:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 47D616B00F1; Sun, 29 Sep 2024 11:12:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2589D6B00EB for ; Sun, 29 Sep 2024 11:12:30 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A039AC16E9 for ; Sun, 29 Sep 2024 15:12:29 +0000 (UTC) X-FDA: 82618117218.11.CC6BB24 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf17.hostedemail.com (Postfix) with ESMTP id 1056B40009 for ; Sun, 29 Sep 2024 15:12:27 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=YcUtCgTO; spf=pass (imf17.hostedemail.com: domain of dakr@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=dakr@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727622685; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/E4lJKrhvvRXPPWlGE3qYEnSFUjPLoFfeRWnnyc/d8s=; b=1cwS8KWlZa5cGye7tL59r8OspW/4dD9Oc9hb7oZcjwQNQ3A1q+i68A5H7XkL5BR6AFxDKf aUz1yGCb05PGgjPxcGQdTXQ+lcPQ8Rvc32ffTCn80vHK2yrclpzRoK2I7HU1t3S6V7oLxr tY6lvnccZ6NZW3cSuuX0Lj760I2YKUA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=YcUtCgTO; spf=pass (imf17.hostedemail.com: domain of dakr@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=dakr@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727622685; a=rsa-sha256; cv=none; b=D8syIM3JRy0s3sfvjiYocppxQQcCjvAch6e0DfMGonq+9m+WpvpxcA6VRgo/+n0GHt5HNh HR2adVVSl99/5X7GbY17bsipn+tGnaY/hidvK7Pg7OPhkCp27L6fohxvQvU/Jj2yZaT0gS HCcNheJ5ukWxkHVGogyV80hoNU8T6vQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id B1D92A40B07; Sun, 29 Sep 2024 15:12:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF25BC4CEC5; Sun, 29 Sep 2024 15:12:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727622746; bh=8iK+wLu+DSCR1nDTiqALV97qNsthNxMyDvf/+B2qlKs=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=YcUtCgTO72dzAyuttuqQtWi9zdBJAWcr3eRvd60dIoNA6FCOtAMXPnlkEb1+lhou1 Qq0KSxOJy330zrUcZmKiAwyzEZJMb2kDfwCA35JL14xzgtTiXhFxRsApVHQ9dd7Qr/ y264RiKnpKrGjCUlSpmqsGPWj6dse15GQ11ybg8DZVVV/LqnXfBTa8/TEbdAqYxFQk DxmY02iHZmQ2X0BE0s1Qnsy6dC89umYmEA5Z8G8OVzx5ghctwyLQloAiTfvxcYjRmu D7Rw6DsRtOYaaLwSorgAdEcxlHX94/pwrxXyf8ZmhWP1YM3A1E62xth2wPFbPv9SAP 3Akn0LafpJ2ZA== Date: Sun, 29 Sep 2024 17:12:19 +0200 From: Danilo Krummrich To: Gary Guo Cc: ojeda@kernel.org, alex.gaynor@gmail.com, wedsonaf@gmail.com, boqun.feng@gmail.com, bjorn3_gh@protonmail.com, benno.lossin@proton.me, a.hindborg@samsung.com, aliceryhl@google.com, akpm@linux-foundation.org, daniel.almeida@collabora.com, faith.ekstrand@collabora.com, boris.brezillon@collabora.com, lina@asahilina.net, mcanal@igalia.com, zhiw@nvidia.com, cjia@nvidia.com, jhubbard@nvidia.com, airlied@redhat.com, ajanulgu@redhat.com, lyude@redhat.com, linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v7 15/26] rust: alloc: implement `collect` for `IntoIter` Message-ID: References: <20240911225449.152928-1-dakr@kernel.org> <20240911225449.152928-16-dakr@kernel.org> <20240928202734.4b518854.gary@garyguo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240928202734.4b518854.gary@garyguo.net> X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 1056B40009 X-Stat-Signature: 9o3erak3rsoo7cnbok6d37zjnipudzc4 X-HE-Tag: 1727622747-785376 X-HE-Meta: U2FsdGVkX1+TtbxljueMxOsEIsvPEoSdM2o9YwSBmKsRvtnHD9z8y+4e6w8Cd+xPEL5+nTSGpvKW4fs2Zd5iBhKRwTBJPi0AUDhL4SDnBMQDSpZU+PvaEb5y69QM9CiE+icZaLyTx7v33PDulhK6nL7yPssosJTQ0yzgQ3tzk9NpxTTb/XFAI9qIFc46xRuujLrwKh7fX3D6TlK8d71Ombp/jFH4O4kGf1LGvJ4YJjJ1lM+I04HWXM3D2Oaheao1BIbGDiYw4m5YqnKP3z3mrP0snbgD2Q2ORyK5vPXoX99QZDMSvnOB7gNiT0maJqLkJhj3exdci8aY4SOVbPAUAfzCSWac9Gdf8JnWo+62U+dM2R9UPxxxq0eBkOGM/Thr02QHfR6kDSm6dydAtPiqYOo4eFFfkqBML2VjQZyL5oaamyFwHOnVfyvNSsoeca+IDEpW20KIyM232f21C6XiHn+gJZhwHJktJBfF0/Rr4AtVnzFewHSStCe0lIDyWPRFsnxYhXSqjrjlE2H8dErdFxmNVklSQKktZ0+RiTNs2tq+A3Kjx/JMXS1PSXEoCYueii+nvbzTjMOcsx9U+JcrcFl4beNKx0hDnsjy+MJ9+Xy95ObEaVnSNICN0QotgqeXcUIVqbhC/KubHsNB9cCzUN6eZiJGTYEDyNFyWIUd9TCi3rAF3WQ1jJMaIxOlhO+qksR4PC5TxOGyahoEsuN9+FvG7ef9Sn6ptqk0wa4T1rcDYqkPgojpPxAuH1W8SiMk97nSDtipO9B4YjOCStb2V2ck/DUkxm012kfhCsJ5Cl1yUEga1ZXFTeOAVx4cJm/Xnvs5ux0bYI1ouUcmAcylwuD0aj0zi+JvrfQW2pesn+H1uF/TGroQhaAkDiN5i47OmC9G1fBG7n7n7rRzTCS00xQnPRCBA/iWpniCyiw3fEXUwnH0fRWMuzPvEXIZvK8gUdn3W0bz7OW7Qv0578z B27ELghO REB3ZCV7/RtuL18DErtjPeNl1qqeACB2/xmAvlFNWNCklNgIBC+Stto6Irl6U9/dnUI8QG/+AYlI7yfFFiyLFXyvH75mlfzrjXPRBUst5QP2+tfKFoP4Vm7N30UmKbaoG41cGy8agPCituSyVN6Km+aCGOQi8rYE2D72J0Ddi5jRBZDHQQfZ9ZETt4QhEVFTijxkuLFJhi6IeM6gij2kNHtzPsGMOTnUdpsTNC66uf3Z9SSe+au49RW7lN7eJ8Gtff0w+Buz2VfQ8lY9SUGNTmTYE0EO4m5UP2yVn4OxIvMzUoqu2A02A9MoIItNlywVGbHW91XTwj1UJoWzZHGY7ZHgRLuVGDAT5nQCypHn3DeIJzpy9qQ+LhTjPPg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Sep 28, 2024 at 08:27:34PM +0100, Gary Guo wrote: > On Thu, 12 Sep 2024 00:52:51 +0200 > Danilo Krummrich wrote: > > > Currently, we can't implement `FromIterator`. There are a couple of > > issues with this trait in the kernel, namely: > > > > - Rust's specialization feature is unstable. This prevents us to > > optimze for the special case where `I::IntoIter` equals `Vec`'s > > `IntoIter` type. > > - We also can't use `I::IntoIter`'s type ID either to work around this, > > since `FromIterator` doesn't require this type to be `'static`. > > - `FromIterator::from_iter` does return `Self` instead of > > `Result`, hence we can't properly handle allocation > > failures. > > - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle > > additional allocation flags. > > > > Instead, provide `IntoIter::collect`, such that we can at least convert > > `IntoIter` into a `Vec` again. > > > > Reviewed-by: Alice Ryhl > > Signed-off-by: Danilo Krummrich > > A question is how useful is this? The way this can be used seems > fairly limited: you `into_iter`, consume a few elements, and the > `collect()`? Well, it allows us to convert back to a `Vec`, which we otherwise can't until we implement our own `FromIterator` trait. Also note that we want to have this specialization of `collect` for performance reasons anyways. The Rust stdlib uses specialization traits (which aren't yet stable) for this optimization. > > It feels whatever user this serves, it would make more sense for them > to use a `VecDeque` and just `pop_front()`. We don't have `VecDeque` (yet). > > Also, inline comments below. > > > --- > > rust/kernel/alloc/kvec.rs | 86 +++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 86 insertions(+) > > > > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs > > index e91761c5c52d..686e969463f8 100644 > > --- a/rust/kernel/alloc/kvec.rs > > +++ b/rust/kernel/alloc/kvec.rs > > @@ -690,6 +690,92 @@ impl IntoIter > > fn as_raw_mut_slice(&mut self) -> *mut [T] { > > ptr::slice_from_raw_parts_mut(self.ptr, self.len) > > } > > + > > + fn into_raw_parts(self) -> (*mut T, NonNull, usize, usize) { > > + let me = ManuallyDrop::new(self); > > + let ptr = me.ptr; > > + let buf = me.buf; > > + let len = me.len; > > + let cap = me.cap; > > + (ptr, buf, len, cap) > > + } > > + > > + /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. > > + /// > > + /// # Examples > > + /// > > + /// ``` > > + /// let v = kernel::kvec![1, 2, 3]?; > > + /// let mut it = v.into_iter(); > > + /// > > + /// assert_eq!(it.next(), Some(1)); > > + /// > > + /// let v = it.collect(GFP_KERNEL); > > + /// assert_eq!(v, [2, 3]); > > + /// > > + /// # Ok::<(), Error>(()) > > + /// ``` > > + /// # Implementation Details > > + /// > > + /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait > > + /// in the kernel, namely: > > + /// > > + /// - Rust's specialization feature is unstable. This prevents us to optimze for the special > > + /// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. > > + /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` > > + /// doesn't require this type to be `'static`. > > + /// - `FromIterator::from_iter` does return `Self` instead of `Result`, hence > > + /// we can't properly handle allocation failures. > > + /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation > > + /// flags. > > + /// > > + /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a > > + /// `Vec` again. > > + /// > > + /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing > > + /// buffer. However, this backing buffer may be shrunk to the actual count of elements. > > + pub fn collect(self, flags: Flags) -> Vec { > > + let (mut ptr, buf, len, mut cap) = self.into_raw_parts(); > > + let has_advanced = ptr != buf.as_ptr(); > > + > > + if has_advanced { > > + // Copy the contents we have advanced to at the beginning of the buffer. > > + // > > + // SAFETY: > > + // - `ptr` is valid for reads of `len * size_of::()` bytes, > > + // - `buf.as_ptr()` is valid for writes of `len * size_of::()` bytes, > > + // - `ptr` and `buf.as_ptr()` are not be subject to aliasing restrictions relative to > > + // each other, > > + // - both `ptr` and `buf.ptr()` are properly aligned. > > + unsafe { ptr::copy(ptr, buf.as_ptr(), len) }; > > + ptr = buf.as_ptr(); > > + } > > + > > + // This can never fail, `len` is guaranteed to be smaller than `cap`. > > + let layout = core::alloc::Layout::array::(len).unwrap(); > > nit: could be `unwrap_unchecked()`. Although feel free to leave it to > avoid an additional unsafe. This will be replaced with `ArrayLayout` anyways. > > > + > > + // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be > > + // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves > > + // it as it is. > > + ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } { > > + // If we fail to shrink, which likely can't even happen, continue with the existing > > + // buffer. > > + Err(_) => ptr, > > + Ok(ptr) => { > > + cap = len; > > + ptr.as_ptr().cast() > > + } > > + }; > > This should be moved to `Vec::shrink_to_fit`. And then this function > can just `Vec::from_raw_parts(...)` and then `vec.shrink_to_fit`. I'll put it on my list for a follow up patch, I really think we should focus on landing the series now. > > But my question would be why this function needs to shrink in the first > place. It's meant as an optimization for `Iterator::collect` in the mid / long term. A user expects that the resulting allocation isn't larger than actually needed for `Iterator::collect`. > > > + > > + // SAFETY: If the iterator has been advanced, the advanced elements have been copied to > > + // the beginning of the buffer and `len` has been adjusted accordingly. > > + // > > + // - `ptr` is guaranteed to point to the start of the backing buffer. > > + // - `cap` is either the original capacity or, after shrinking the buffer, equal to `len`. > > + // - `alloc` is guaranteed to be unchanged since `into_iter` has been called on the original > > + // `Vec`. > > + unsafe { Vec::from_raw_parts(ptr, len, cap) } > > + } > > } > > > > impl Iterator for IntoIter >