public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/3] rust: alloc: add Vec shrinking methods
@ 2026-01-31 15:40 Shivam Kalra
  2026-01-31 15:40 ` [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec Shivam Kalra
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Shivam Kalra @ 2026-01-31 15:40 UTC (permalink / raw)
  To: dakr, cmllamas, gregkh
  Cc: aliceryhl, rust-for-linux, linux-kernel, Shivam Kalra

This is a resend of v1 [1] to fix threading issues - patches 2-3 were
sent without In-Reply-To headers due to an email quota issue.

This series adds shrink_to() and shrink_to_fit() methods to Vec<T, A>
(and by extension KVVec, KVec, VVec) to allow explicit capacity
reduction for memory reclamation.

Problem:
When elements are removed from a KVVec, the allocated capacity is not
reduced. The underlying C allocators (krealloc/kvrealloc) don't shrink
memory in-place. This can cause significant memory waste in scenarios
with variable workloads.

Solution:
- Patch 1: Adds shrink_to(min_capacity, flags) and shrink_to_fit(flags)
  methods that reallocate to a smaller buffer when beneficial.
- Patch 2: Adds KUnit tests for the new methods.
- Patch 3: Uses shrink_to() in the Rust binder driver to reclaim memory
  when processes are deregistered. Shrinking is triggered only when the
  vector capacity exceeds 128 elements and less than half is used.

This series depends on:
  https://lore.kernel.org/lkml/20260130182842.217821-1-shivamklr@cock.li/

Testing:
- KUnit tests pass (rust_kvec test suite).
- Kernel boots successfully in QEMU with CONFIG_ANDROID_BINDER_IPC_RUST=y.

Changes since v1:
- Resend with correct threading (no code changes).
- Removed base-commit.
- Added explicit lore link to dependency in cover letter.

[1] https://lore.kernel.org/lkml/20260130205424.261700-1-shivamklr@cock.li/

Shivam Kalra (3):
  rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  rust: alloc: add KUnit tests for Vec shrinking
  binder: context: shrink all_procs vector to reclaim memory

 drivers/android/binder/context.rs |  10 ++
 rust/kernel/alloc/kvec.rs         | 176 ++++++++++++++++++++++++++++++
 2 files changed, 186 insertions(+)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-01-31 15:40 [PATCH v2 0/3] rust: alloc: add Vec shrinking methods Shivam Kalra
@ 2026-01-31 15:40 ` Shivam Kalra
  2026-02-03 15:19   ` Danilo Krummrich
  2026-01-31 15:40 ` [PATCH v2 2/3] rust: alloc: add KUnit tests for Vec shrinking Shivam Kalra
  2026-02-01  0:08 ` [PATCH v2 3/3] binder: context: shrink all_procs vector to reclaim memory Shivam Kalra
  2 siblings, 1 reply; 11+ messages in thread
From: Shivam Kalra @ 2026-01-31 15:40 UTC (permalink / raw)
  To: dakr, cmllamas, gregkh
  Cc: aliceryhl, rust-for-linux, linux-kernel, Shivam Kalra

Add methods to shrink the capacity of a Vec to free excess memory.
This is useful for drivers that experience variable workloads and want
to reclaim memory after spikes.

- `shrink_to(min_capacity, flags)`: Shrinks the capacity of the vector
  with a lower bound. The capacity will remain at least as large as
  both the length and the supplied value.

- `shrink_to_fit(flags)`: Shrinks the capacity of the vector as much
  as possible.

This implementation guarantees shrinking (unless already optimal),
because the kernel allocators don't support in-place shrinking,
a new allocation is always made.

Suggested-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Shivam Kalra <shivamklr@cock.li>
---
 rust/kernel/alloc/kvec.rs | 111 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 111 insertions(+)

diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index ac8d6f763ae81..9c02734ced88f 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -733,6 +733,117 @@ pub fn retain(&mut self, mut f: impl FnMut(&mut T) -> bool) {
         }
         self.truncate(num_kept);
     }
+
+    /// Shrinks the capacity of the vector with a lower bound.
+    ///
+    /// The capacity will remain at least as large as both the length
+    /// and the supplied value.
+    ///
+    /// If the current capacity is less than the lower limit, this is a no-op.
+    ///
+    /// This requires allocating a new buffer and copying the elements, then freeing
+    /// the old buffer.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = KVec::with_capacity(100, GFP_KERNEL)?;
+    /// v.push(1, GFP_KERNEL)?;
+    /// v.push(2, GFP_KERNEL)?;
+    /// assert!(v.capacity() >= 100);
+    ///
+    /// v.shrink_to(50, GFP_KERNEL)?;
+    /// assert!(v.capacity() >= 50);
+    /// assert!(v.capacity() < 100);
+    ///
+    /// v.shrink_to(0, GFP_KERNEL)?;
+    /// assert!(v.capacity() >= 2);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn shrink_to(&mut self, min_capacity: usize, flags: Flags) -> Result<(), AllocError> {
+        // Calculate the target capacity: max(len, min_capacity).
+        let target_cap = core::cmp::max(self.len(), min_capacity);
+
+        // If we're already within limits, nothing to do.
+        if self.capacity() <= target_cap {
+            return Ok(());
+        }
+
+        // ZSTs have no allocation to shrink.
+        if Self::is_zst() {
+            return Ok(());
+        }
+
+        // Handle empty vector or target capacity 0: just free the allocation and reset.
+        if target_cap == 0 {
+            // Only free if we actually have an allocation.
+            if !self.layout.is_empty() {
+                // SAFETY:
+                // - `self.ptr` was previously allocated with `A`.
+                // - `self.layout` matches the `ArrayLayout` of the preceding allocation.
+                unsafe { A::free(self.ptr.cast(), self.layout.into()) };
+            }
+            self.ptr = NonNull::dangling();
+            self.layout = ArrayLayout::empty();
+            return Ok(());
+        }
+
+        // Create a new layout that exactly fits the target capacity.
+        // SAFETY: `target_cap` is guaranteed to be <= `self.capacity()`, and the original
+        // capacity was validated, so `target_cap * size_of::<T>() <= isize::MAX`.
+        let new_layout = unsafe { ArrayLayout::<T>::new_unchecked(target_cap) };
+
+        // Allocate a new, smaller buffer.
+        let new_ptr = A::alloc(new_layout.into(), flags, NumaNode::NO_NODE)?;
+
+        // SAFETY:
+        // - `self.as_ptr()` is valid for reads of `self.len` elements by the type invariant.
+        // - `new_ptr` is valid for writes of `self.len` elements (we just allocated it).
+        // - The regions do not overlap (different allocations).
+        // - Both pointers are properly aligned.
+        unsafe {
+            ptr::copy_nonoverlapping(self.as_ptr(), new_ptr.as_ptr().cast::<T>(), self.len);
+        }
+
+        // Free the old buffer.
+        // SAFETY:
+        // - `self.ptr` was previously allocated with `A`.
+        // - `self.layout` matches the `ArrayLayout` of the preceding allocation.
+        unsafe { A::free(self.ptr.cast(), self.layout.into()) };
+
+        // SAFETY: `new_ptr.as_ptr()` is non-null because `A::alloc` succeeded.
+        self.ptr = unsafe { NonNull::new_unchecked(new_ptr.as_ptr().cast::<T>()) };
+        self.layout = new_layout;
+
+        Ok(())
+    }
+
+    /// Shrinks the capacity of the vector as much as possible.
+    ///
+    /// The capacity will be reduced to match the length, freeing any excess memory.
+    /// This requires allocating a new buffer and copying the elements, then freeing
+    /// the old buffer.
+    ///
+    /// If the allocation of the new buffer fails, the vector is left unchanged and
+    /// an error is returned.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = KVec::with_capacity(100, GFP_KERNEL)?;
+    /// v.push(1, GFP_KERNEL)?;
+    /// v.push(2, GFP_KERNEL)?;
+    /// v.push(3, GFP_KERNEL)?;
+    /// assert!(v.capacity() >= 100);
+    ///
+    /// v.shrink_to_fit(GFP_KERNEL)?;
+    /// assert_eq!(v.capacity(), 3);
+    /// assert_eq!(&v, &[1, 2, 3]);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn shrink_to_fit(&mut self, flags: Flags) -> Result<(), AllocError> {
+        self.shrink_to(0, flags)
+    }
 }
 
 impl<T: Clone, A: Allocator> Vec<T, A> {
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 2/3] rust: alloc: add KUnit tests for Vec shrinking
  2026-01-31 15:40 [PATCH v2 0/3] rust: alloc: add Vec shrinking methods Shivam Kalra
  2026-01-31 15:40 ` [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec Shivam Kalra
@ 2026-01-31 15:40 ` Shivam Kalra
  2026-02-01  0:08 ` [PATCH v2 3/3] binder: context: shrink all_procs vector to reclaim memory Shivam Kalra
  2 siblings, 0 replies; 11+ messages in thread
From: Shivam Kalra @ 2026-01-31 15:40 UTC (permalink / raw)
  To: dakr, cmllamas, gregkh
  Cc: aliceryhl, rust-for-linux, linux-kernel, Shivam Kalra

Add comprehensive KUnit tests for Vec::shrink_to() and
Vec::shrink_to_fit() to verify that capacity is reduced correctly.

The tests cover:
- shrinking from excess capacity
- correct handling of empty vectors
- no-op behavior when capacity is already optimal
- respecting the minimum capacity limit for shrink_to()
- no growth when the requested minimum exceeds current capacity

Testing:
- KUnit tests pass locally (command above).
- Built with CONFIG_RUST=y on x86_64 (user-mode testing via --arch um).

Signed-off-by: Shivam Kalra <shivamklr@cock.li>
---
 rust/kernel/alloc/kvec.rs | 65 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index 9c02734ced88f..5ab78d6c03e40 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -1509,4 +1509,69 @@ fn add(value: &mut [bool]) {
             func.push_within_capacity(false).unwrap();
         }
     }
+
+    #[test]
+    fn test_shrink_to_fit() {
+        // Create a vector with excess capacity.
+        let mut v: KVec<u32> = KVec::with_capacity(100, GFP_KERNEL).unwrap();
+        v.push(1, GFP_KERNEL).unwrap();
+        v.push(2, GFP_KERNEL).unwrap();
+        v.push(3, GFP_KERNEL).unwrap();
+
+        assert!(v.capacity() >= 100);
+        assert_eq!(v.len(), 3);
+
+        // Shrink with 0 (equivalent to shrink_to_fit).
+        v.shrink_to(0, GFP_KERNEL).unwrap();
+
+        // Capacity should now equal length.
+        assert_eq!(v.capacity(), 3);
+        assert_eq!(v.len(), 3);
+        assert_eq!(&v, &[1, 2, 3]);
+
+        // Shrink empty vector.
+        let mut v: KVec<u32> = KVec::with_capacity(50, GFP_KERNEL).unwrap();
+        v.shrink_to(0, GFP_KERNEL).unwrap();
+        assert_eq!(v.capacity(), 0);
+
+        // Shrink already optimal (no-op).
+        let mut v: KVec<u32> = KVec::new();
+        v.push(1, GFP_KERNEL).unwrap();
+        v.shrink_to(0, GFP_KERNEL).unwrap();
+        assert!(v.capacity() >= 1);
+    }
+
+    #[test]
+    fn test_shrink_to_min() {
+        let mut v: KVec<u32> = KVec::with_capacity(100, GFP_KERNEL).unwrap();
+        for i in 0..10 {
+            v.push(i, GFP_KERNEL).unwrap();
+        }
+        assert_eq!(v.len(), 10);
+        assert!(v.capacity() >= 100);
+
+        // Shrink to a specific minimum higher than len.
+        v.shrink_to(50, GFP_KERNEL).unwrap();
+        assert!(v.capacity() >= 50);
+        assert!(v.capacity() < 100); // Should have shrunk
+        assert_eq!(v.len(), 10);
+        assert_eq!(v[0], 0);
+        assert_eq!(v[9], 9);
+
+        // Shrink to a minimum lower than len (should stop at len).
+        v.shrink_to(5, GFP_KERNEL).unwrap();
+        assert_eq!(v.capacity(), 10);
+        assert_eq!(v.len(), 10);
+    }
+
+    #[test]
+    fn test_shrink_to_no_growth() {
+        let mut v: KVec<u32> = KVec::with_capacity(10, GFP_KERNEL).unwrap();
+        v.push(1, GFP_KERNEL).unwrap();
+        let cap = v.capacity();
+
+        // Requesting larger capacity should NOT grow the vector.
+        v.shrink_to(100, GFP_KERNEL).unwrap();
+        assert_eq!(v.capacity(), cap);
+    }
 }
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 3/3] binder: context: shrink all_procs vector to reclaim memory
  2026-01-31 15:40 [PATCH v2 0/3] rust: alloc: add Vec shrinking methods Shivam Kalra
  2026-01-31 15:40 ` [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec Shivam Kalra
  2026-01-31 15:40 ` [PATCH v2 2/3] rust: alloc: add KUnit tests for Vec shrinking Shivam Kalra
@ 2026-02-01  0:08 ` Shivam Kalra
  2 siblings, 0 replies; 11+ messages in thread
From: Shivam Kalra @ 2026-02-01  0:08 UTC (permalink / raw)
  To: dakr, cmllamas, gregkh
  Cc: aliceryhl, rust-for-linux, linux-kernel, Shivam Kalra

After removing a process from the all_procs list, shrink the vector
if it has significant unused capacity. Specifically, if the capacity
exceeds 128 and the length is less than half the capacity, shrink
the vector to its current length.

This prevents unbounded memory retention in long-running systems where
the number of binder processes fluctuates significantly over time.

Link: https://lore.kernel.org/rust-for-linux/aW4DWebqCg1c8Fgb@google.com/
Suggested-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Shivam Kalra <shivamklr@cock.li>
---
 drivers/android/binder/context.rs | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/android/binder/context.rs b/drivers/android/binder/context.rs
index 9cf437c025a20..f2505fbf17403 100644
--- a/drivers/android/binder/context.rs
+++ b/drivers/android/binder/context.rs
@@ -94,6 +94,16 @@ pub(crate) fn deregister_process(self: &Arc<Self>, proc: &Arc<Process>) {
         }
         let mut manager = self.manager.lock();
         manager.all_procs.retain(|p| !Arc::ptr_eq(p, proc));
+
+        // Shrink the vector if it has significant unused capacity.
+        // Only shrink if capacity > 128 to avoid repeated reallocations for small vectors.
+        let len = manager.all_procs.len();
+        let cap = manager.all_procs.capacity();
+        if cap > 128 && len < cap / 2 {
+            // Shrink to current length. Ignore allocation failures since this is just an
+            // optimization; the vector remains valid even if shrinking fails.
+            let _ = manager.all_procs.shrink_to(len, GFP_KERNEL);
+        }
     }
 
     pub(crate) fn set_manager_node(&self, node_ref: NodeRef) -> Result {
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-01-31 15:40 ` [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec Shivam Kalra
@ 2026-02-03 15:19   ` Danilo Krummrich
  2026-02-03 15:38     ` Alice Ryhl
  0 siblings, 1 reply; 11+ messages in thread
From: Danilo Krummrich @ 2026-02-03 15:19 UTC (permalink / raw)
  To: Shivam Kalra
  Cc: cmllamas, gregkh, aliceryhl, rust-for-linux, linux-kernel,
	lorenzo.stoakes, vbabka, Liam.Howlett, urezki

On Sat Jan 31, 2026 at 4:40 PM CET, Shivam Kalra wrote:
> This implementation guarantees shrinking (unless already optimal),
> because the kernel allocators don't support in-place shrinking,
> a new allocation is always made.

I'm not sure we should go in this direction. There is a reason why krealloc()
does not migrate memory between kmalloc buckets, i.e. the cost of migration vs.
memory saving.

For Vmalloc buffers the story is a bit different though. When I wrote vrealloc()
I left some TODO comments [1][2].

  (1) If a smaller buffer is requested we can shrink the vm_area, i.e. unmap and
      free unused pages.

  (2) If a bigger buffer is requested we can grow the vm_area, i.e. allocate and
      map additional pages. (At least as long as we have enough space in the
      virtual address space.)

So, I think we should just use A::realloc(), leave the rest to the underlying
specific realloc() implementations and address the TODOs in vrealloc() if
necessary.

[1] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4162
[2] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4192

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-02-03 15:19   ` Danilo Krummrich
@ 2026-02-03 15:38     ` Alice Ryhl
  2026-02-03 15:43       ` Danilo Krummrich
  0 siblings, 1 reply; 11+ messages in thread
From: Alice Ryhl @ 2026-02-03 15:38 UTC (permalink / raw)
  To: Danilo Krummrich
  Cc: Shivam Kalra, cmllamas, gregkh, rust-for-linux, linux-kernel,
	lorenzo.stoakes, vbabka, Liam.Howlett, urezki

On Tue, Feb 03, 2026 at 04:19:14PM +0100, Danilo Krummrich wrote:
> On Sat Jan 31, 2026 at 4:40 PM CET, Shivam Kalra wrote:
> > This implementation guarantees shrinking (unless already optimal),
> > because the kernel allocators don't support in-place shrinking,
> > a new allocation is always made.
> 
> I'm not sure we should go in this direction. There is a reason why krealloc()
> does not migrate memory between kmalloc buckets, i.e. the cost of migration vs.
> memory saving.
> 
> For Vmalloc buffers the story is a bit different though. When I wrote vrealloc()
> I left some TODO comments [1][2].
> 
>   (1) If a smaller buffer is requested we can shrink the vm_area, i.e. unmap and
>       free unused pages.
> 
>   (2) If a bigger buffer is requested we can grow the vm_area, i.e. allocate and
>       map additional pages. (At least as long as we have enough space in the
>       virtual address space.)
> 
> So, I think we should just use A::realloc(), leave the rest to the underlying
> specific realloc() implementations and address the TODOs in vrealloc() if
> necessary.
> 
> [1] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4162
> [2] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4192

If kvrealloc() does the right thing, then let's use it.

Alice

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-02-03 15:38     ` Alice Ryhl
@ 2026-02-03 15:43       ` Danilo Krummrich
  2026-02-03 17:48         ` Shivam Kalra
  0 siblings, 1 reply; 11+ messages in thread
From: Danilo Krummrich @ 2026-02-03 15:43 UTC (permalink / raw)
  To: Alice Ryhl
  Cc: Shivam Kalra, cmllamas, gregkh, rust-for-linux, linux-kernel,
	lorenzo.stoakes, vbabka, Liam.Howlett, urezki

On Tue Feb 3, 2026 at 4:38 PM CET, Alice Ryhl wrote:
> On Tue, Feb 03, 2026 at 04:19:14PM +0100, Danilo Krummrich wrote:
>> On Sat Jan 31, 2026 at 4:40 PM CET, Shivam Kalra wrote:
>> > This implementation guarantees shrinking (unless already optimal),
>> > because the kernel allocators don't support in-place shrinking,
>> > a new allocation is always made.
>> 
>> I'm not sure we should go in this direction. There is a reason why krealloc()
>> does not migrate memory between kmalloc buckets, i.e. the cost of migration vs.
>> memory saving.
>> 
>> For Vmalloc buffers the story is a bit different though. When I wrote vrealloc()
>> I left some TODO comments [1][2].
>> 
>>   (1) If a smaller buffer is requested we can shrink the vm_area, i.e. unmap and
>>       free unused pages.
>> 
>>   (2) If a bigger buffer is requested we can grow the vm_area, i.e. allocate and
>>       map additional pages. (At least as long as we have enough space in the
>>       virtual address space.)
>> 
>> So, I think we should just use A::realloc(), leave the rest to the underlying
>> specific realloc() implementations and address the TODOs in vrealloc() if
>> necessary.
>> 
>> [1] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4162
>> [2] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4192
>
> If kvrealloc() does the right thing, then let's use it.

It should once the TODOs of vrealloc() are addressed. The reason I left them as
TODOs was that I didn't want to implement all the shrink and grow logic for
struct vm_area without having a user that actually needs it.

If binder needs it, I think we should do it.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-02-03 15:43       ` Danilo Krummrich
@ 2026-02-03 17:48         ` Shivam Kalra
  2026-02-04 10:32           ` Alice Ryhl
  0 siblings, 1 reply; 11+ messages in thread
From: Shivam Kalra @ 2026-02-03 17:48 UTC (permalink / raw)
  To: Danilo Krummrich, Alice Ryhl
  Cc: cmllamas, gregkh, rust-for-linux, linux-kernel, lorenzo.stoakes,
	vbabka, Liam.Howlett, urezki

On 03/02/26 21:13, Danilo Krummrich wrote:
> On Tue Feb 3, 2026 at 4:38 PM CET, Alice Ryhl wrote:
>> On Tue, Feb 03, 2026 at 04:19:14PM +0100, Danilo Krummrich wrote:
>>> On Sat Jan 31, 2026 at 4:40 PM CET, Shivam Kalra wrote:
>>>> This implementation guarantees shrinking (unless already optimal),
>>>> because the kernel allocators don't support in-place shrinking,
>>>> a new allocation is always made.
>>>
>>> I'm not sure we should go in this direction. There is a reason why krealloc()
>>> does not migrate memory between kmalloc buckets, i.e. the cost of migration vs.
>>> memory saving.
>>>
>>> For Vmalloc buffers the story is a bit different though. When I wrote vrealloc()
>>> I left some TODO comments [1][2].
>>>
>>>   (1) If a smaller buffer is requested we can shrink the vm_area, i.e. unmap and
>>>       free unused pages.
>>>
>>>   (2) If a bigger buffer is requested we can grow the vm_area, i.e. allocate and
>>>       map additional pages. (At least as long as we have enough space in the
>>>       virtual address space.)
>>>
>>> So, I think we should just use A::realloc(), leave the rest to the underlying
>>> specific realloc() implementations and address the TODOs in vrealloc() if
>>> necessary.
>>>
>>> [1] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4162
>>> [2] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4192
>>
>> If kvrealloc() does the right thing, then let's use it.
> 
> It should once the TODOs of vrealloc() are addressed. The reason I left them as
> TODOs was that I didn't want to implement all the shrink and grow logic for
> struct vm_area without having a user that actually needs it.
> 
> If binder needs it, I think we should do it.
Hi Danilo, Alice,

Thanks for the detailed feedback - I hadn't considered the kmalloc bucket
migration costs.

Given that:
- krealloc() intentionally avoids migrating data to smaller buckets when
shrinking
- vrealloc() has TODOs for in-place shrinking
- The immediate need is binder, which uses KVec (could use either allocator)

I'm thinking the pragmatic path is:

1. For v3: Simplify shrink_to() to use A::realloc() instead of
   alloc+copy+free. This ensures we get whatever optimization
   the allocator provides (including the bucket preservation for kmalloc).

2. The vrealloc() in-place shrinking could be a separate follow-up
   series, as it's a larger change to the allocator itself.

Does this approach make sense, or would you prefer I tackle the
vrealloc TODOs first?

Thanks,
Shivam

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-02-03 17:48         ` Shivam Kalra
@ 2026-02-04 10:32           ` Alice Ryhl
  2026-02-04 11:50             ` Danilo Krummrich
  0 siblings, 1 reply; 11+ messages in thread
From: Alice Ryhl @ 2026-02-04 10:32 UTC (permalink / raw)
  To: Shivam Kalra
  Cc: Danilo Krummrich, cmllamas, gregkh, rust-for-linux, linux-kernel,
	lorenzo.stoakes, vbabka, Liam.Howlett, urezki

On Tue, Feb 03, 2026 at 11:18:17PM +0530, Shivam Kalra wrote:
> On 03/02/26 21:13, Danilo Krummrich wrote:
> > On Tue Feb 3, 2026 at 4:38 PM CET, Alice Ryhl wrote:
> >> On Tue, Feb 03, 2026 at 04:19:14PM +0100, Danilo Krummrich wrote:
> >>> On Sat Jan 31, 2026 at 4:40 PM CET, Shivam Kalra wrote:
> >>>> This implementation guarantees shrinking (unless already optimal),
> >>>> because the kernel allocators don't support in-place shrinking,
> >>>> a new allocation is always made.
> >>>
> >>> I'm not sure we should go in this direction. There is a reason why krealloc()
> >>> does not migrate memory between kmalloc buckets, i.e. the cost of migration vs.
> >>> memory saving.
> >>>
> >>> For Vmalloc buffers the story is a bit different though. When I wrote vrealloc()
> >>> I left some TODO comments [1][2].
> >>>
> >>>   (1) If a smaller buffer is requested we can shrink the vm_area, i.e. unmap and
> >>>       free unused pages.
> >>>
> >>>   (2) If a bigger buffer is requested we can grow the vm_area, i.e. allocate and
> >>>       map additional pages. (At least as long as we have enough space in the
> >>>       virtual address space.)
> >>>
> >>> So, I think we should just use A::realloc(), leave the rest to the underlying
> >>> specific realloc() implementations and address the TODOs in vrealloc() if
> >>> necessary.
> >>>
> >>> [1] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4162
> >>> [2] https://elixir.bootlin.com/linux/v6.18.6/source/mm/vmalloc.c#L4192
> >>
> >> If kvrealloc() does the right thing, then let's use it.
> > 
> > It should once the TODOs of vrealloc() are addressed. The reason I left them as
> > TODOs was that I didn't want to implement all the shrink and grow logic for
> > struct vm_area without having a user that actually needs it.
> > 
> > If binder needs it, I think we should do it.
> Hi Danilo, Alice,
> 
> Thanks for the detailed feedback - I hadn't considered the kmalloc bucket
> migration costs.
> 
> Given that:
> - krealloc() intentionally avoids migrating data to smaller buckets when
> shrinking
> - vrealloc() has TODOs for in-place shrinking
> - The immediate need is binder, which uses KVec (could use either allocator)

Binder uses KVVec not KVec, which is the one that could use either allocator.

> I'm thinking the pragmatic path is:
> 
> 1. For v3: Simplify shrink_to() to use A::realloc() instead of
>    alloc+copy+free. This ensures we get whatever optimization
>    the allocator provides (including the bucket preservation for kmalloc).
> 
> 2. The vrealloc() in-place shrinking could be a separate follow-up
>    series, as it's a larger change to the allocator itself.
> 
> Does this approach make sense, or would you prefer I tackle the
> vrealloc TODOs first?

I would kind of prefer that we do this in two steps. First have
shrink_to() use the implementation it does right now. Then a follow-up
patch fix the TODOs in vrealloc().

Alice

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-02-04 10:32           ` Alice Ryhl
@ 2026-02-04 11:50             ` Danilo Krummrich
  2026-02-04 21:55               ` Shivam Kalra
  0 siblings, 1 reply; 11+ messages in thread
From: Danilo Krummrich @ 2026-02-04 11:50 UTC (permalink / raw)
  To: Alice Ryhl
  Cc: Shivam Kalra, cmllamas, gregkh, rust-for-linux, linux-kernel,
	lorenzo.stoakes, vbabka, Liam.Howlett, urezki

On Wed Feb 4, 2026 at 11:32 AM CET, Alice Ryhl wrote:
> I would kind of prefer that we do this in two steps. First have
> shrink_to() use the implementation it does right now. Then a follow-up
> patch fix the TODOs in vrealloc().

This is fine with me as a short term workaround, but we should only do the full
copy under certain conditions only:

  (1) is_vmalloc_addr() returns true.

  (2) The new size of the allocation requires at least one page less in total.

I.e. if it is a kmalloc() buffer we don't do anything. And if it's a vmalloc()
buffer, we only shrink if we can get rid of at least one page, since otherwise
there are no savings effectively.

Shivam do you plan to follow up on the vrealloc() TODOs subsequently?

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec
  2026-02-04 11:50             ` Danilo Krummrich
@ 2026-02-04 21:55               ` Shivam Kalra
  0 siblings, 0 replies; 11+ messages in thread
From: Shivam Kalra @ 2026-02-04 21:55 UTC (permalink / raw)
  To: Danilo Krummrich, Alice Ryhl
  Cc: cmllamas, gregkh, rust-for-linux, linux-kernel, lorenzo.stoakes,
	vbabka, Liam.Howlett, urezki

On 04/02/26 17:20, Danilo Krummrich wrote:
> This is fine with me as a short term workaround, but we should only do the full
> copy under certain conditions only:
> 
>   (1) is_vmalloc_addr() returns true.
> 
>   (2) The new size of the allocation requires at least one page less in total.
> 
> I.e. if it is a kmalloc() buffer we don't do anything. And if it's a vmalloc()
> buffer, we only shrink if we can get rid of at least one page, since otherwise
> there are no savings effectively.
> 
> Shivam do you plan to follow up on the vrealloc() TODOs subsequently?
I'll work on v3 that addresses your conditions
(only shrink for vmalloc buffers when freeing >= 1 page).
I'll check if the necessary primitives (like is_vmalloc_addr) are
exposed to Rust - if not, I may need to add helper functions or bindings.

As for the vrealloc() TODOs - yes, I'm happy to follow up on that as a
separate series. Since it would be my first time touching the mm
subsystem C code, I'd appreciate any pointers on the preferred approach
when I get there.


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2026-02-04 21:55 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-31 15:40 [PATCH v2 0/3] rust: alloc: add Vec shrinking methods Shivam Kalra
2026-01-31 15:40 ` [PATCH v2 1/3] rust: alloc: Add shrink_to and shrink_to_fit methods to Vec Shivam Kalra
2026-02-03 15:19   ` Danilo Krummrich
2026-02-03 15:38     ` Alice Ryhl
2026-02-03 15:43       ` Danilo Krummrich
2026-02-03 17:48         ` Shivam Kalra
2026-02-04 10:32           ` Alice Ryhl
2026-02-04 11:50             ` Danilo Krummrich
2026-02-04 21:55               ` Shivam Kalra
2026-01-31 15:40 ` [PATCH v2 2/3] rust: alloc: add KUnit tests for Vec shrinking Shivam Kalra
2026-02-01  0:08 ` [PATCH v2 3/3] binder: context: shrink all_procs vector to reclaim memory Shivam Kalra

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox