Re: [PATCH RFC] rcu/kfree: Do not request RCU when not needed

[Uladzislau Rezki <urezki@gmail.com>]

> 
> 
> Nice, I am happy you worked on it. A comment bellow:
> 
> Below what i have came up with to switch for polling APIs:
> >
> > From 799ce1653d159ef3d35f34a284f738c2c267c75f Mon Sep 17 00:00:00 2001
> > From: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
> > Date: Wed, 2 Nov 2022 19:26:27 +0100
> > Subject: [PATCH 1/1] rcu: kvfree_rcu: Reduce a memory footptint by using
> >  polling APIs
> >
> > Total time taken by all kfree'ers: 6564718459 ns, loops: 10000, batches:
> > 1110, memory footprint: 5057MB
> > Total time taken by all kfree'ers: 8431051895 ns, loops: 10000, batches:
> > 1109, memory footprint: 2749MB
> > Total time taken by all kfree'ers: 9477830789 ns, loops: 10000, batches:
> > 1158, memory footprint: 2934MB
> > Total time taken by all kfree'ers: 9950211144 ns, loops: 10000, batches:
> > 981, memory footprint: 2704MB
> >
> > with a patch:
> 
> 
> >
> > Total time taken by all kfree'ers: 7712110118 ns, loops: 10000, batches:
> > 1660, memory footprint: 91MB
> > Total time taken by all kfree'ers: 7002403664 ns, loops: 10000, batches:
> > 1482, memory footprint: 86MB
> > Total time taken by all kfree'ers: 7842282319 ns, loops: 10000, batches:
> > 1738, memory footprint: 86MB
> > Total time taken by all kfree'ers: 7230161977 ns, loops: 10000, batches:
> > 1542, memory footprint: 72MB
> >
> > Tested with NOCB option, all offloading CPUs:
> 
> 
> > kvm.sh --memory 10G --torture rcuscale --allcpus --duration 1 \
> >   --kconfig CONFIG_NR_CPUS=64 \
> >   --kconfig CONFIG_RCU_NOCB_CPU=y \
> >   --kconfig CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y \
> >   --bootargs "rcuscale.kfree_rcu_test=1 rcuscale.kfree_nthreads=16 \
> >   rcuscale.holdoff=20 rcuscale.kfree_loops=10000
> > torture.disable_onoff_at_boot" --trust-make
> >
> > According to data there is a big gain in memory footprint with a patch.
> > It is because of call_rcu() and call_rcu_flush() take more effort and
> > time to queue a callback and then wait for a gp.
> 
> 
> >
> > With polling API:
> >   a) we do not need to queue any callback;
> >   b) we might not even need wait for a GP completion.
> >
> 
> Wow, I did not expect an extra grace period or 2 to cause so much memory
> unreclaimed.
> 
> Does this kernel have the 5 second timeout change?
> 
Yes. But it does not matter. Why? Because 5 second time out happens only
when a system lightly loaded. It can be 10 or 100 seconds.

Once a flood is detected we start to offload back-logged memory.

>
> If yes, then I am confused why extra grace period matters for memory. My
> goal with the idea was to avoid pointless RCU work when we don’t need it.
> But I can’t complaint.
> 
This patch does the same. It avoids of using any call_rcu() because we
do not need it. I mean extra layer.

Why it matters for the memory. I explained it in previous answer. But it
is because of extra layer. Since it is a flood scenario such delays can
lead to higher memory footprint.

The test easily detects it.

>
> Or is it that this test floods the pointer page so we do not wait full 5
> seconds, and benefit when we reclaim sooner with polled API, than waiting
> for grace periods?
> 
Right. Reclaim sooner with polling API.

--
Uladzislau Rezki