[PATCH linux-next v2] sched/core: Add WARN() to check overflow in migrate_disable()

[PATCH linux-next v2] sched/core: Add WARN() to check overflow in migrate_disable()

[xu.xin16]

From: Peilin He <he.peilin@zte.com.cn>

Background
==========
When repeated migrate_disable() calls are made with missing the
corresponding migrate_enable() calls, there is a risk of
'migration_disabled' going upper overflow because
'migration_disabled' is a type of unsigned short whose max value is
65535.

In PREEMPT_RT kernel, if 'migration_disabled' goes upper overflow, it may
make the migrate_disable() ineffective within local_lock_irqsave(). This
is because, during the scheduling procedure, the value of
'migration_disabled' will be checked, which can trigger CPU migration.
Consequently, the count of 'rcu_read_lock_nesting' may leak due to
local_lock_irqsave() and local_unlock_irqrestore() occurring on different
CPUs.

Usecase
========
For example, When I developed a driver, I encountered a warning like
"WARNING: CPU: 4 PID: 260 at kernel/rcu/tree_plugin.h:315
rcu_note_context_switch+0xa8/0x4e8" warning. It took me half a month
to locate this issue. Ultimately, I discovered that the lack of upper
overflow detection mechanism in migrate_disable() was the root cause,
leading to a significant amount of time spent on problem localization.

If the upper overflow detection mechanism was added to migrate_disable(),
the root cause could be very quickly and easily identified.

Effect
======
Using WARN() to check if 'migration_disabled' is upper overflow can help
developers identify the issue quickly.

Signed-off-by: Peilin He<he.peilin@zte.com.cn>
Signed-off-by: xu xin <xu.xin16@zte.com.cn>
Reviewed-by: Yunkai Zhang <zhang.yunkai@zte.com.cn>
Reviewed-by: Qiang Tu <tu.qiang35@zte.com.cn>
Reviewed-by: Kun Jiang <jiang.kun2@zte.com.cn>
Reviewed-by: Fan Yu <fan.yu9@zte.com.cn>
Cc: Yang Yang <yang.yang29@zte.com.cn>
Cc: Liu Chun <liu.chun2@zte.com.cn>
---
v1->v2:
Some fixes according to:
https://lore.kernel.org/all/20240702124334.762dbd5a@rorschach.local.home/
1.Merge if conditions into WARN().
2.Remove the newline character '\n'. Right, we don't need the redundant \n.

 kernel/sched/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8cc4975d6b2b..327010af6ce9 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2259,6 +2259,7 @@ void migrate_disable(void)
 	struct task_struct *p = current;

 	if (p->migration_disabled) {
+		WARN(p->migration_disabled == USHRT_MAX, "migration_disabled has encountered an overflow.");
 		p->migration_disabled++;
 		return;
 	}
-- 
2.17.1

Re: [PATCH linux-next v2] sched/core: Add WARN() to check overflow in migrate_disable()

[Peter Zijlstra]

On Wed, Jul 03, 2024 at 08:53:25PM +0800, xu.xin16@zte.com.cn wrote:

> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 8cc4975d6b2b..327010af6ce9 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2259,6 +2259,7 @@ void migrate_disable(void)
>  	struct task_struct *p = current;
> 
>  	if (p->migration_disabled) {
> +		WARN(p->migration_disabled == USHRT_MAX, "migration_disabled has encountered an overflow.");
>  		p->migration_disabled++;
>  		return;
>  	}

How about we do something like this?

---
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7a5ea8fc8abb..06a559532ed6 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2237,6 +2237,12 @@ void migrate_disable(void)
 
 	if (p->migration_disabled) {
 		p->migration_disabled++;
+#ifdef CONFIG_DEBUG_PREEMPT
+		/*
+		 * Warn about overflow half-way through the range.
+		 */
+		WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
 		return;
 	}
 
@@ -2254,14 +2260,20 @@ void migrate_enable(void)
 		.flags     = SCA_MIGRATE_ENABLE,
 	};
 
+#ifdef CONFIG_DEBUG_PREEMPT
+	/*
+	 * Check both overflow from migrate_disable() and superfluous
+	 * migrate_enable().
+	 */
+	if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+		return;
+#endif
+
 	if (p->migration_disabled > 1) {
 		p->migration_disabled--;
 		return;
 	}
 
-	if (WARN_ON_ONCE(!p->migration_disabled))
-		return;
-
 	/*
 	 * Ensure stop_task runs either before or after this, and that
 	 * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().

Re: Re: [PATCH linux-next v2] sched/core: Add WARN() to check overflow in migrate_disable()

[Peilin He]

> How about we do something like this?
>
> ---
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 7a5ea8fc8abb..06a559532ed6 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2237,6 +2237,12 @@ void migrate_disable(void)
>  
>      if (p->migration_disabled) {
>          p->migration_disabled++;
> +#ifdef CONFIG_DEBUG_PREEMPT
> +        /*
> +         * Warn about overflow half-way through the range.
> +         */
> +        WARN_ON_ONCE((s16)p->migration_disabled < 0);
> +#endif
>          return;
>      }

Agreed, converting p->migration_disabled to a signed number 
will detect overflow occurrences earlier than our current method.

> @@ -2254,14 +2260,20 @@ void migrate_enable(void)
>          .flags     = SCA_MIGRATE_ENABLE,
>      };
>  
> +#ifdef CONFIG_DEBUG_PREEMPT
> +    /*
> +     * Check both overflow from migrate_disable() and superfluous
> +     * migrate_enable().
> +     */
> +    if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
> +        return;
> +#endif
> +
>      if (p->migration_disabled > 1) {
>          p->migration_disabled--;
>          return;
>      }
>  
> -    if (WARN_ON_ONCE(!p->migration_disabled))
> -        return;
> -
>      /*
>       * Ensure stop_task runs either before or after this, and that
>       * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().

Agreed, similarly checking for overflow in p->migration_disabled 
within migrate_enable is fine, but placing WARN_ON_ONCE inside 
the CONFIG_DEBUG_PREEMPT macro may cause return logic deviations. 
It is recommended to support WARN_ON_ONCE as a standard feature.
The fix will be reflected in version v3.

Re: Re: [PATCH linux-next v2] sched/core: Add WARN() to check overflow in migrate_disable()

[Peter Zijlstra]

On Fri, Jul 05, 2024 at 05:41:32AM +0000, Peilin He wrote:
> > How about we do something like this?
> >
> > ---
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 7a5ea8fc8abb..06a559532ed6 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -2237,6 +2237,12 @@ void migrate_disable(void)
> >  
> >      if (p->migration_disabled) {
> >          p->migration_disabled++;
> > +#ifdef CONFIG_DEBUG_PREEMPT
> > +        /*
> > +         * Warn about overflow half-way through the range.
> > +         */
> > +        WARN_ON_ONCE((s16)p->migration_disabled < 0);
> > +#endif
> >          return;
> >      }
> 
> Agreed, converting p->migration_disabled to a signed number 
> will detect overflow occurrences earlier than our current method.
> 
> > @@ -2254,14 +2260,20 @@ void migrate_enable(void)
> >          .flags     = SCA_MIGRATE_ENABLE,
> >      };
> >  
> > +#ifdef CONFIG_DEBUG_PREEMPT
> > +    /*
> > +     * Check both overflow from migrate_disable() and superfluous
> > +     * migrate_enable().
> > +     */
> > +    if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
> > +        return;
> > +#endif
> > +
> >      if (p->migration_disabled > 1) {
> >          p->migration_disabled--;
> >          return;
> >      }
> >  
> > -    if (WARN_ON_ONCE(!p->migration_disabled))
> > -        return;
> > -
> >      /*
> >       * Ensure stop_task runs either before or after this, and that
> >       * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
> 
> Agreed, similarly checking for overflow in p->migration_disabled 
> within migrate_enable is fine, but placing WARN_ON_ONCE inside 
> the CONFIG_DEBUG_PREEMPT macro may cause return logic deviations. 
> It is recommended to support WARN_ON_ONCE as a standard feature.

It is debug stuff, it should only be needed for debug builds. Same as
the preemption things, they also only check for overflow on debug
builds.