[PATCH 0/1] rcu: drain kfree_rcu sheaves from the userspace barrier hook

[Matthias Goergens <matthias.goergens@gmail.com>]

The rcutree.do_rcu_barrier test hook currently waits for ordinary RCU
callbacks, but it can return while kfree_rcu() still retains an object in a
partial per-CPU sheaf.  This defeats the hook's purpose of preventing deferred
frees from one test spilling into the next.

The patch drains kfree_rcu sheaves and batches before retaining the hook's
explicit ordinary rcu_barrier().  Four counterbalanced fresh-VM pairs with the
full validation fixture reported 60 -> 60 active objects on the unpatched
kernel and 60 -> 59 on the patched kernel.  An ordinary-callback regression
test passed on both kernels.

The primary reproducer below removes that separate regression machinery.  One
fresh control/treatment pair with this exact 41-line source reproduced the
same 60 -> 60 versus 60 -> 59 split; both cells reached TEST SUCCESS with no
problem-class kernel records.

Save the source as rcu_barrier_sheaf_repro.c and create a Makefile containing:

  obj-m := rcu_barrier_sheaf_repro.o

Build it with:

  make -C /lib/modules/$(uname -r)/build M="$PWD" modules

Then, as root on a disposable test kernel:

  insmod rcu_barrier_sheaf_repro.ko
  awk '$1 == "rcu_barrier_sheaf_repro" { print $2 }' /proc/slabinfo
  cat /sys/kernel/slab/rcu_barrier_sheaf_repro/sheaf_capacity
  echo 1 > /sys/module/rcutree/parameters/do_rcu_barrier
  awk '$1 == "rcu_barrier_sheaf_repro" { print $2 }' /proc/slabinfo
  rmmod rcu_barrier_sheaf_repro

The first and second slabinfo readings are 60 and 60 without the patch, and
60 and 59 with it.  kmem_cache_destroy() performs per-cache deferred-free
cleanup when the module is removed, after the measurement.

// SPDX-License-Identifier: GPL-2.0
#include <linux/init.h>
#include <linux/module.h>
#include <linux/rcupdate.h>
#include <linux/slab.h>

struct repro_object {
	struct rcu_head rcu;
	unsigned long payload;
};

static struct kmem_cache *repro_cache;

static int __init rcu_barrier_sheaf_repro_init(void)
{
	struct repro_object *object;

	repro_cache = kmem_cache_create("rcu_barrier_sheaf_repro",
					sizeof(*object), 0, SLAB_NO_MERGE, NULL);
	if (!repro_cache)
		return -ENOMEM;

	object = kmem_cache_alloc(repro_cache, GFP_KERNEL);
	if (!object) {
		kmem_cache_destroy(repro_cache);
		return -ENOMEM;
	}

	kfree_rcu(object, rcu);
	return 0;
}

static void __exit rcu_barrier_sheaf_repro_exit(void)
{
	kmem_cache_destroy(repro_cache);
}

module_init(rcu_barrier_sheaf_repro_init);
module_exit(rcu_barrier_sheaf_repro_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Reproduce incomplete rcutree.do_rcu_barrier drains");

Matthias Goergens (1):
  rcu: drain kfree_rcu sheaves from the userspace barrier hook

 .../admin-guide/kernel-parameters.txt         |  7 ++---
 kernel/rcu/tree.c                             | 27 ++++++++++++-------
 2 files changed, 21 insertions(+), 13 deletions(-)


base-commit: 50d05c7c76c96b90462f24debacca971d2e86713
-- 
2.55.0