Re: [PATCH 3/4] sched/isolation: Infrastructure to support rcu nocb cpumask changes

[Tobias Huschle <huschle@linux.ibm.com>]

On 2022-05-26 00:10, Frederic Weisbecker wrote:
> Provide a minimal infrastructure to change the housekeeping cpumasks.
> For now only RCU NOCB cpumask is handled.
> 
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> Cc: Zefan Li <lizefan.x@bytedance.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Paul E. McKenney <paulmck@kernel.org>
> Cc: Phil Auld <pauld@redhat.com>
> Cc: Nicolas Saenz Julienne <nsaenz@kernel.org>
> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
> Cc: Waiman Long <longman@redhat.com>
> Cc: Daniel Bristot de Oliveira <bristot@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> ---
>  include/linux/sched/isolation.h | 13 +++++++++++
>  kernel/sched/isolation.c        | 38 +++++++++++++++++++++++++++++++++
>  2 files changed, 51 insertions(+)
> 
> diff --git a/include/linux/sched/isolation.h 
> b/include/linux/sched/isolation.h
> index 8c15abd67aed..c6d0e3f83a20 100644
> --- a/include/linux/sched/isolation.h
> +++ b/include/linux/sched/isolation.h
> @@ -25,6 +25,8 @@ extern const struct cpumask
> *housekeeping_cpumask(enum hk_type type);
>  extern bool housekeeping_enabled(enum hk_type type);
>  extern void housekeeping_affine(struct task_struct *t, enum hk_type 
> type);
>  extern bool housekeeping_test_cpu(int cpu, enum hk_type type);
> +extern int housekeeping_cpumask_set(struct cpumask *cpumask, enum
> hk_type type);
> +extern int housekeeping_cpumask_clear(struct cpumask *cpumask, enum
> hk_type type);
>  extern void __init housekeeping_init(void);
> 
>  #else
> @@ -46,6 +48,17 @@ static inline bool housekeeping_enabled(enum hk_type 
> type)
> 
>  static inline void housekeeping_affine(struct task_struct *t,
>  				       enum hk_type type) { }
> +
> +static inline int housekeeping_cpumask_set(struct cpumask *cpumask,
> enum hk_type type)
> +{
> +	return -EINVAL;
> +}
> +
> +static inline int housekeeping_cpumask_clear(struct cpumask *cpumask,
> enum hk_type type)
> +{
> +	return -EINVAL;
> +}
> +
>  static inline void housekeeping_init(void) { }
>  #endif /* CONFIG_CPU_ISOLATION */
> 
> diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
> index 373d42c707bc..ab4aba795c01 100644
> --- a/kernel/sched/isolation.c
> +++ b/kernel/sched/isolation.c
> @@ -79,6 +79,44 @@ bool housekeeping_test_cpu(int cpu, enum hk_type 
> type)
>  }
>  EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
> 
> +static int housekeeping_cpumask_update(struct cpumask *cpumask,
> +				       enum hk_type type, bool on)
> +{
> +	int err;
> +
> +	switch (type) {
> +	case HK_TYPE_RCU:
> +		err = rcu_nocb_cpumask_update(cpumask, on);
> +		break;
> +	default:
> +		err = -EINVAL;
> +	}
> +
> +	if (err >= 0) {
> +		if (on) {
> +			cpumask_or(housekeeping.cpumasks[type],
> +				   housekeeping.cpumasks[type],
> +				   cpumask);
> +		} else {
> +			cpumask_andnot(housekeeping.cpumasks[type],
> +				       housekeeping.cpumasks[type],
> +				       cpumask);
> +		}
> +	}
> +
> +	return err;
> +}
> +
> +int housekeeping_cpumask_set(struct cpumask *cpumask, enum hk_type 
> type)
> +{
> +	return housekeeping_cpumask_update(cpumask, type, true);
> +}
> +
> +int housekeeping_cpumask_clear(struct cpumask *cpumask, enum hk_type 
> type)
> +{
> +	return housekeeping_cpumask_update(cpumask, type, false);
> +}
> +
>  void __init housekeeping_init(void)
>  {
>  	enum hk_type type;

Just stumbled upon this patch.

I would be interested to have a way to fully isolate CPUs during 
runtime.
I tried some things similar to the patch above and the results looked
promising (removing certain CPUs from the housekeeping cpumasks during 
runtime).
Offlining them might be too expensive and also go a bit too far, as I 
might want to
be able to reactivate these CPUs quickly.

What kind of problems would you expect when making the housekeeping 
masks editable?
Not just rcu_nocb, but all of them.

--
Tobias