[PATCH v2 0/3] Small cleanup around sparse report

[PATCH v2 0/3] Small cleanup around sparse report

[Pierre Gondois]

v2:
- Added __rcu annotations to variabbles in sched_reset_numa(),
  and casts for calls to kfree().
- Renamed idle_cpu_without() to idle_rq_without()
- Added sched/fair: Use rq in update_blocked_averages()

While checking spare's tool report, some RCU pointers appeared
to be not annotated. In the same effort, idle_cpu_without() and
update_blocked_averages() seemed to be subject to a small
optimizations.

Pierre Gondois (3):
  sched/topology: Annotate RCU pointers properly
  sched/fair: Use rq in idle_cpu_without()
  sched/fair: Use rq in update_blocked_averages()

 kernel/sched/fair.c     | 25 +++++++++++--------------
 kernel/sched/topology.c | 12 ++++++------
 2 files changed, 17 insertions(+), 20 deletions(-)

-- 
2.25.1

[PATCH v2 1/3] sched/topology: Annotate RCU pointers properly

[Pierre Gondois]

Cleanup RCU-related spare errors by annotating RCU pointers.

sched_domains_numa_distance:
  error: incompatible types in comparison expression
  (different address spaces):
      int [noderef] __rcu *
      int *

sched_domains_numa_masks:
  error: incompatible types in comparison expression
  (different address spaces):
      struct cpumask **[noderef] __rcu *
      struct cpumask ***

The cast to (void *) adds the following sparse warning:
  warning: cast removes address space '__rcu' of expression
but this should be normal.

Fixes: 0fb3978b0aac ("sched/numa: Fix NUMA topology for systems with CPU-less nodes")
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
---
 kernel/sched/topology.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 10d1391e7416..2a2da9b33e31 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1542,8 +1542,8 @@ static int			sched_domains_numa_levels;
 static int			sched_domains_curr_level;
 
 int				sched_max_numa_distance;
-static int			*sched_domains_numa_distance;
-static struct cpumask		***sched_domains_numa_masks;
+static int			__rcu *sched_domains_numa_distance;
+static struct cpumask		** __rcu *sched_domains_numa_masks;
 #endif
 
 /*
@@ -1988,8 +1988,8 @@ void sched_init_numa(int offline_node)
 
 static void sched_reset_numa(void)
 {
-	int nr_levels, *distances;
-	struct cpumask ***masks;
+	int nr_levels, __rcu *distances;
+	struct cpumask ** __rcu *masks;
 
 	nr_levels = sched_domains_numa_levels;
 	sched_domains_numa_levels = 0;
@@ -2003,7 +2003,7 @@ static void sched_reset_numa(void)
 		int i, j;
 
 		synchronize_rcu();
-		kfree(distances);
+		kfree((void *)distances);
 		for (i = 0; i < nr_levels && masks; i++) {
 			if (!masks[i])
 				continue;
@@ -2011,7 +2011,7 @@ static void sched_reset_numa(void)
 				kfree(masks[i][j]);
 			kfree(masks[i]);
 		}
-		kfree(masks);
+		kfree((void *)masks);
 	}
 	if (sched_domain_topology_saved) {
 		kfree(sched_domain_topology);
-- 
2.25.1

[PATCH v2 2/3] sched/fair: Use rq in idle_cpu_without()

[Pierre Gondois]

idle_cpu_without() could receive a 'struct rq' instead of a
CPU number to avoid converting the CPU number to a 'struct rq'
two times. Indeed update_sg_wakeup_stats() already makes the
conversion.
idle_cpu_without() is also renamed to idle_rq_without()
to match the input parameter.

Reviewed-by: Shrikanth Hegde <sshegde@linux.vnet.ibm.com>
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
---
 kernel/sched/fair.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d7a3c63a2171..97d9e0d32337 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10105,23 +10105,21 @@ static unsigned int task_running_on_cpu(int cpu, struct task_struct *p)
 }
 
 /**
- * idle_cpu_without - would a given CPU be idle without p ?
- * @cpu: the processor on which idleness is tested.
+ * idle_rq_without - would a given rq be idle without p ?
+ * @rq: the rq on which idleness is tested.
  * @p: task which should be ignored.
  *
- * Return: 1 if the CPU would be idle. 0 otherwise.
+ * Return: 1 if the rq would be idle. 0 otherwise.
  */
-static int idle_cpu_without(int cpu, struct task_struct *p)
+static int idle_rq_without(struct rq *rq, struct task_struct *p)
 {
-	struct rq *rq = cpu_rq(cpu);
-
 	if (rq->curr != rq->idle && rq->curr != p)
 		return 0;
 
 	/*
 	 * rq->nr_running can't be used but an updated version without the
 	 * impact of p on cpu must be used instead. The updated nr_running
-	 * be computed and tested before calling idle_cpu_without().
+	 * be computed and tested before calling idle_rq_without().
 	 */
 
 #ifdef CONFIG_SMP
@@ -10166,9 +10164,9 @@ static inline void update_sg_wakeup_stats(struct sched_domain *sd,
 		sgs->sum_nr_running += nr_running;
 
 		/*
-		 * No need to call idle_cpu_without() if nr_running is not 0
+		 * No need to call idle_rq_without() if nr_running is not 0
 		 */
-		if (!nr_running && idle_cpu_without(i, p))
+		if (!nr_running && idle_rq_without(rq, p))
 			sgs->idle_cpus++;
 
 		/* Check if task fits in the CPU */
-- 
2.25.1

[PATCH v2 3/3] sched/fair: Use rq in update_blocked_averages()

[Pierre Gondois]

update_blocked_averages() could receive a 'struct rq' instead of a
CPU number to avoid converting the CPU number to a 'struct rq'
multiple times.

Suggested-by: Shrikanth Hegde <sshegde@linux.vnet.ibm.com>
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
---
 kernel/sched/fair.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 97d9e0d32337..c70bb993f473 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9349,10 +9349,9 @@ static unsigned long task_h_load(struct task_struct *p)
 }
 #endif
 
-static void update_blocked_averages(int cpu)
+static void update_blocked_averages(struct rq *rq)
 {
 	bool decayed = false, done = true;
-	struct rq *rq = cpu_rq(cpu);
 	struct rq_flags rf;
 
 	rq_lock_irqsave(rq, &rf);
@@ -12026,7 +12025,7 @@ static bool update_nohz_stats(struct rq *rq)
 	if (!time_after(jiffies, READ_ONCE(rq->last_blocked_load_update_tick)))
 		return true;
 
-	update_blocked_averages(cpu);
+	update_blocked_averages(rq);
 
 	return rq->has_blocked_load;
 }
@@ -12286,7 +12285,7 @@ static int newidle_balance(struct rq *this_rq, struct rq_flags *rf)
 	raw_spin_rq_unlock(this_rq);
 
 	t0 = sched_clock_cpu(this_cpu);
-	update_blocked_averages(this_cpu);
+	update_blocked_averages(this_rq);
 
 	rcu_read_lock();
 	for_each_domain(this_cpu, sd) {
@@ -12376,7 +12375,7 @@ static __latent_entropy void run_rebalance_domains(struct softirq_action *h)
 		return;
 
 	/* normal load balance */
-	update_blocked_averages(this_rq->cpu);
+	update_blocked_averages(this_rq);
 	rebalance_domains(this_rq, idle);
 }
 
-- 
2.25.1

Re: [PATCH v2 1/3] sched/topology: Annotate RCU pointers properly

[Huang, Ying]

Pierre Gondois <pierre.gondois@arm.com> writes:

> Cleanup RCU-related spare errors by annotating RCU pointers.
>
> sched_domains_numa_distance:
>   error: incompatible types in comparison expression
>   (different address spaces):
>       int [noderef] __rcu *
>       int *
>
> sched_domains_numa_masks:
>   error: incompatible types in comparison expression
>   (different address spaces):
>       struct cpumask **[noderef] __rcu *
>       struct cpumask ***
>
> The cast to (void *) adds the following sparse warning:
>   warning: cast removes address space '__rcu' of expression
> but this should be normal.
>
> Fixes: 0fb3978b0aac ("sched/numa: Fix NUMA topology for systems with CPU-less nodes")
> Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
> ---
>  kernel/sched/topology.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 10d1391e7416..2a2da9b33e31 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -1542,8 +1542,8 @@ static int			sched_domains_numa_levels;
>  static int			sched_domains_curr_level;
>  
>  int				sched_max_numa_distance;
> -static int			*sched_domains_numa_distance;
> -static struct cpumask		***sched_domains_numa_masks;
> +static int			__rcu *sched_domains_numa_distance;
> +static struct cpumask		** __rcu *sched_domains_numa_masks;
>  #endif
>  
>  /*
> @@ -1988,8 +1988,8 @@ void sched_init_numa(int offline_node)
>  
>  static void sched_reset_numa(void)
>  {
> -	int nr_levels, *distances;
> -	struct cpumask ***masks;
> +	int nr_levels, __rcu *distances;
> +	struct cpumask ** __rcu *masks;

No.  distances and masks are not accessed via RCU in the function.
Instead, they should be assigned like below,

         distances = rcu_dereference_raw(sched_domains_numa_distance);

Because sched_domains_numa_distance is protected by cpu_hotplug_lock,
but the lock is static.  Some comments are deserved.

Anyway, please read RCU document before making the change.

https://docs.kernel.org/RCU/index.html

>  	nr_levels = sched_domains_numa_levels;
>  	sched_domains_numa_levels = 0;
> @@ -2003,7 +2003,7 @@ static void sched_reset_numa(void)
>  		int i, j;
>  
>  		synchronize_rcu();
> -		kfree(distances);
> +		kfree((void *)distances);
>  		for (i = 0; i < nr_levels && masks; i++) {
>  			if (!masks[i])
>  				continue;
> @@ -2011,7 +2011,7 @@ static void sched_reset_numa(void)
>  				kfree(masks[i][j]);
>  			kfree(masks[i]);
>  		}
> -		kfree(masks);
> +		kfree((void *)masks);
>  	}
>  	if (sched_domain_topology_saved) {
>  		kfree(sched_domain_topology);

--
Best Regards,
Huang, Ying

Re: [PATCH v2 1/3] sched/topology: Annotate RCU pointers properly

[kernel test robot]

Hi Pierre,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tip/sched/core]
[also build test WARNING on linus/master v6.7 next-20240112]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Pierre-Gondois/sched-topology-Annotate-RCU-pointers-properly/20240115-223658
base:   tip/sched/core
patch link:    https://lore.kernel.org/r/20240115143427.1820628-2-pierre.gondois%40arm.com
patch subject: [PATCH v2 1/3] sched/topology: Annotate RCU pointers properly
config: arm64-randconfig-r133-20240116 (https://download.01.org/0day-ci/archive/20240116/202401161144.OJH5PySm-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 13.2.0
reproduce: (https://download.01.org/0day-ci/archive/20240116/202401161144.OJH5PySm-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202401161144.OJH5PySm-lkp@intel.com/

sparse warnings: (new ones prefixed by >>)
   kernel/sched/build_utility.c: note: in included file:
   kernel/sched/debug.c:468:17: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/debug.c:468:17: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/debug.c:468:17: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/debug.c:793:9: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct task_struct *tsk @@     got struct task_struct [noderef] __rcu *curr @@
   kernel/sched/debug.c:793:9: sparse:     expected struct task_struct *tsk
   kernel/sched/debug.c:793:9: sparse:     got struct task_struct [noderef] __rcu *curr
   kernel/sched/debug.c:793:9: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct task_struct *tsk @@     got struct task_struct [noderef] __rcu *curr @@
   kernel/sched/debug.c:793:9: sparse:     expected struct task_struct *tsk
   kernel/sched/debug.c:793:9: sparse:     got struct task_struct [noderef] __rcu *curr
   kernel/sched/build_utility.c: note: in included file:
   kernel/sched/stats.c:148:17: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/stats.c:148:17: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/stats.c:148:17: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/build_utility.c: note: in included file:
   kernel/sched/topology.c:107:56: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:107:56: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:107:56: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:126:60: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:126:60: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:126:60: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:149:20: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:149:20: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:149:20: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:485:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct perf_domain *pd @@     got struct perf_domain [noderef] __rcu *pd @@
   kernel/sched/topology.c:485:19: sparse:     expected struct perf_domain *pd
   kernel/sched/topology.c:485:19: sparse:     got struct perf_domain [noderef] __rcu *pd
   kernel/sched/topology.c:647:49: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain *parent @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:647:49: sparse:     expected struct sched_domain *parent
   kernel/sched/topology.c:647:49: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:732:50: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain *parent @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:732:50: sparse:     expected struct sched_domain *parent
   kernel/sched/topology.c:732:50: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:740:55: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain [noderef] __rcu *[noderef] __rcu child @@     got struct sched_domain *[assigned] tmp @@
   kernel/sched/topology.c:740:55: sparse:     expected struct sched_domain [noderef] __rcu *[noderef] __rcu child
   kernel/sched/topology.c:740:55: sparse:     got struct sched_domain *[assigned] tmp
   kernel/sched/topology.c:753:29: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] tmp @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:753:29: sparse:     expected struct sched_domain *[assigned] tmp
   kernel/sched/topology.c:753:29: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:758:20: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:758:20: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:758:20: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:779:13: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] tmp @@     got struct sched_domain [noderef] __rcu *sd @@
   kernel/sched/topology.c:779:13: sparse:     expected struct sched_domain *[assigned] tmp
   kernel/sched/topology.c:779:13: sparse:     got struct sched_domain [noderef] __rcu *sd
   kernel/sched/topology.c:941:70: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:941:70: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:941:70: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:970:59: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:970:59: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:970:59: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1016:57: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:1016:57: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:1016:57: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1018:25: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *sibling @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:1018:25: sparse:     expected struct sched_domain *sibling
   kernel/sched/topology.c:1018:25: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1026:55: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:1026:55: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:1026:55: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1028:25: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *sibling @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:1028:25: sparse:     expected struct sched_domain *sibling
   kernel/sched/topology.c:1028:25: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1098:62: sparse: sparse: incorrect type in argument 1 (different address spaces) @@     expected struct sched_domain *sd @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:1098:62: sparse:     expected struct sched_domain *sd
   kernel/sched/topology.c:1098:62: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1202:40: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain *child @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:1202:40: sparse:     expected struct sched_domain *child
   kernel/sched/topology.c:1202:40: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1622:43: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain [noderef] __rcu *child @@     got struct sched_domain *child @@
   kernel/sched/topology.c:1622:43: sparse:     expected struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:1622:43: sparse:     got struct sched_domain *child
>> kernel/sched/topology.c:2006:24: sparse: sparse: cast removes address space '__rcu' of expression
   kernel/sched/topology.c:2014:24: sparse: sparse: cast removes address space '__rcu' of expression
   kernel/sched/topology.c:2321:31: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain [noderef] __rcu *parent @@     got struct sched_domain *sd @@
   kernel/sched/topology.c:2321:31: sparse:     expected struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:2321:31: sparse:     got struct sched_domain *sd
   kernel/sched/topology.c:2425:57: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:2425:57: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/topology.c:2425:57: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:2446:56: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain *child @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/topology.c:2446:56: sparse:     expected struct sched_domain *child
   kernel/sched/topology.c:2446:56: sparse:     got struct sched_domain [noderef] __rcu *child
   kernel/sched/topology.c:2445:57: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:2445:57: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/topology.c:2445:57: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/topology.c:2500:57: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/topology.c:2500:57: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/topology.c:2500:57: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/build_utility.c: note: in included file:
   kernel/sched/core_sched.c:276:37: sparse: sparse: incompatible types in conditional expression (different address spaces):
   kernel/sched/core_sched.c:276:37: sparse:    struct task_struct *
   kernel/sched/core_sched.c:276:37: sparse:    struct task_struct [noderef] __rcu *
   kernel/sched/build_utility.c: note: in included file:
   kernel/sched/sched.h:2142:25: sparse: sparse: incompatible types in comparison expression (different address spaces):
   kernel/sched/sched.h:2142:25: sparse:    struct task_struct [noderef] __rcu *
   kernel/sched/sched.h:2142:25: sparse:    struct task_struct *

vim +/__rcu +2006 kernel/sched/topology.c

  1987	
  1988	
  1989	static void sched_reset_numa(void)
  1990	{
  1991		int nr_levels, __rcu *distances;
  1992		struct cpumask ** __rcu *masks;
  1993	
  1994		nr_levels = sched_domains_numa_levels;
  1995		sched_domains_numa_levels = 0;
  1996		sched_max_numa_distance = 0;
  1997		sched_numa_topology_type = NUMA_DIRECT;
  1998		distances = sched_domains_numa_distance;
  1999		rcu_assign_pointer(sched_domains_numa_distance, NULL);
  2000		masks = sched_domains_numa_masks;
  2001		rcu_assign_pointer(sched_domains_numa_masks, NULL);
  2002		if (distances || masks) {
  2003			int i, j;
  2004	
  2005			synchronize_rcu();
> 2006			kfree((void *)distances);
  2007			for (i = 0; i < nr_levels && masks; i++) {
  2008				if (!masks[i])
  2009					continue;
  2010				for_each_node(j)
  2011					kfree(masks[i][j]);
  2012				kfree(masks[i]);
  2013			}
  2014			kfree((void *)masks);
  2015		}
  2016		if (sched_domain_topology_saved) {
  2017			kfree(sched_domain_topology);
  2018			sched_domain_topology = sched_domain_topology_saved;
  2019			sched_domain_topology_saved = NULL;
  2020		}
  2021	}
  2022	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH v2 1/3] sched/topology: Annotate RCU pointers properly

[Valentin Schneider]

On 16/01/24 09:23, Huang, Ying wrote:
> Pierre Gondois <pierre.gondois@arm.com> writes:
>
>> Cleanup RCU-related spare errors by annotating RCU pointers.
>>
>> sched_domains_numa_distance:
>>   error: incompatible types in comparison expression
>>   (different address spaces):
>>       int [noderef] __rcu *
>>       int *
>>
>> sched_domains_numa_masks:
>>   error: incompatible types in comparison expression
>>   (different address spaces):
>>       struct cpumask **[noderef] __rcu *
>>       struct cpumask ***
>>
>> The cast to (void *) adds the following sparse warning:
>>   warning: cast removes address space '__rcu' of expression
>> but this should be normal.
>>
>> Fixes: 0fb3978b0aac ("sched/numa: Fix NUMA topology for systems with CPU-less nodes")
>> Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
>> ---
>>  kernel/sched/topology.c | 12 ++++++------
>>  1 file changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
>> index 10d1391e7416..2a2da9b33e31 100644
>> --- a/kernel/sched/topology.c
>> +++ b/kernel/sched/topology.c
>> @@ -1542,8 +1542,8 @@ static int			sched_domains_numa_levels;
>>  static int			sched_domains_curr_level;
>>
>>  int				sched_max_numa_distance;
>> -static int			*sched_domains_numa_distance;
>> -static struct cpumask		***sched_domains_numa_masks;
>> +static int			__rcu *sched_domains_numa_distance;
>> +static struct cpumask		** __rcu *sched_domains_numa_masks;
>>  #endif
>>
>>  /*
>> @@ -1988,8 +1988,8 @@ void sched_init_numa(int offline_node)
>>
>>  static void sched_reset_numa(void)
>>  {
>> -	int nr_levels, *distances;
>> -	struct cpumask ***masks;
>> +	int nr_levels, __rcu *distances;
>> +	struct cpumask ** __rcu *masks;
>
> No.  distances and masks are not accessed via RCU in the function.
> Instead, they should be assigned like below,
>
>          distances = rcu_dereference_raw(sched_domains_numa_distance);
>
> Because sched_domains_numa_distance is protected by cpu_hotplug_lock,
> but the lock is static.  Some comments are deserved.
>
> Anyway, please read RCU document before making the change.
>

IIUC, something like so could also do?

  distances = rcu_dereference_check(sched_domains_numa_distance, lockdep_is_cpus_held());

Re: [PATCH v2 1/3] sched/topology: Annotate RCU pointers properly

[Huang, Ying]

Valentin Schneider <vschneid@redhat.com> writes:

> On 16/01/24 09:23, Huang, Ying wrote:
>> Pierre Gondois <pierre.gondois@arm.com> writes:
>>
>>> Cleanup RCU-related spare errors by annotating RCU pointers.
>>>
>>> sched_domains_numa_distance:
>>>   error: incompatible types in comparison expression
>>>   (different address spaces):
>>>       int [noderef] __rcu *
>>>       int *
>>>
>>> sched_domains_numa_masks:
>>>   error: incompatible types in comparison expression
>>>   (different address spaces):
>>>       struct cpumask **[noderef] __rcu *
>>>       struct cpumask ***
>>>
>>> The cast to (void *) adds the following sparse warning:
>>>   warning: cast removes address space '__rcu' of expression
>>> but this should be normal.
>>>
>>> Fixes: 0fb3978b0aac ("sched/numa: Fix NUMA topology for systems with CPU-less nodes")
>>> Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
>>> ---
>>>  kernel/sched/topology.c | 12 ++++++------
>>>  1 file changed, 6 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
>>> index 10d1391e7416..2a2da9b33e31 100644
>>> --- a/kernel/sched/topology.c
>>> +++ b/kernel/sched/topology.c
>>> @@ -1542,8 +1542,8 @@ static int			sched_domains_numa_levels;
>>>  static int			sched_domains_curr_level;
>>>
>>>  int				sched_max_numa_distance;
>>> -static int			*sched_domains_numa_distance;
>>> -static struct cpumask		***sched_domains_numa_masks;
>>> +static int			__rcu *sched_domains_numa_distance;
>>> +static struct cpumask		** __rcu *sched_domains_numa_masks;
>>>  #endif
>>>
>>>  /*
>>> @@ -1988,8 +1988,8 @@ void sched_init_numa(int offline_node)
>>>
>>>  static void sched_reset_numa(void)
>>>  {
>>> -	int nr_levels, *distances;
>>> -	struct cpumask ***masks;
>>> +	int nr_levels, __rcu *distances;
>>> +	struct cpumask ** __rcu *masks;
>>
>> No.  distances and masks are not accessed via RCU in the function.
>> Instead, they should be assigned like below,
>>
>>          distances = rcu_dereference_raw(sched_domains_numa_distance);
>>
>> Because sched_domains_numa_distance is protected by cpu_hotplug_lock,
>> but the lock is static.  Some comments are deserved.
>>
>> Anyway, please read RCU document before making the change.
>>
>
> IIUC, something like so could also do?
>
>   distances = rcu_dereference_check(sched_domains_numa_distance, lockdep_is_cpus_held());

Yes.  You are right.  We should do that.

--
Best Regards,
Huang, Ying