[PATCH] sched/mmcid: Fix affinity-triggered switch back to per-task mode

[PATCH] sched/mmcid: Fix affinity-triggered switch back to per-task mode

[Hui Su]

MM CID switches to per-CPU ownership when the number of users of an
mm exceeds the maximum number of CIDs. When the affinity union grows,
the per-CPU threshold is recalculated and the MM can switch back to
per-task ownership through deferred work.

mm_cid_calc_pcpu_thrs() incorrectly uses
min(nr_cpus_allowed, users) to calculate that switch-back threshold.
This makes pcpu_thrs no larger than users, so the
users < pcpu_thrs check in mm_update_cpus_allowed() can never succeed.

For example, with three users restricted to two CPUs, the MM enters
per-CPU mode. On an eight-CPU system, widening the affinity union to
all eight CPUs should produce a threshold of four, but the old
calculation produces three and returns early. The MM therefore remains
in per-CPU mode even though the widened CPU set makes per-task
ownership preferable.

Calculate pcpu_thrs from nr_cpus_allowed directly. The users bound in
__mm_update_max_cids() still controls when per-CPU mode is entered, so
this does not change the threshold calculated when switching from
per-task to per-CPU ownership.

Tested with an x86_64 QEMU reproducer using three users, an initial
two-CPU affinity, and an eight-CPU affinity expansion. The baseline
did not invoke mm_cid_work_fn() after the affinity expansion. The
fixed kernel invoked the deferred work once and completed the switch
back to per-task mode.

Fixes: fbd0e71dc370 ("sched/mmcid: Provide CID ownership mode fixup functions")
Signed-off-by: Hui Su <sh_def@163.com>
---
 kernel/sched/core.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7885ff76e69f..0280ed132d0a 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -10845,11 +10845,10 @@ static void __mm_update_max_cids(struct mm_mm_cid *mc)
 
 static inline unsigned int mm_cid_calc_pcpu_thrs(struct mm_mm_cid *mc)
 {
-	unsigned int opt_cids;
+	unsigned int nr_cpus = mc->nr_cpus_allowed;
 
-	opt_cids = min(mc->nr_cpus_allowed, mc->users);
 	/* Has to be at least 1 because 0 indicates PCPU mode off */
-	return max(min(opt_cids - opt_cids / 4, num_possible_cpus() / 2), 1);
+	return max(min(nr_cpus - nr_cpus / 4, num_possible_cpus() / 2), 1);
 }
 
 static bool mm_update_max_cids(struct mm_struct *mm)

base-commit: 587858367581b9c55c3690f4e63382ad622719d4
-- 
2.55.0

Re: [PATCH] sched/mmcid: Fix affinity-triggered switch back to per-task mode

[Mathieu Desnoyers]

On 2026-09-15 11:59, Hui Su wrote:
> MM CID switches to per-CPU ownership when the number of users of an
> mm exceeds the maximum number of CIDs. When the affinity union grows,
> the per-CPU threshold is recalculated and the MM can switch back to
> per-task ownership through deferred work.
> 
> mm_cid_calc_pcpu_thrs() incorrectly uses
> min(nr_cpus_allowed, users) to calculate that switch-back threshold.
> This makes pcpu_thrs no larger than users, so the
> users < pcpu_thrs check in mm_update_cpus_allowed() can never succeed.
> 
> For example, with three users restricted to two CPUs, the MM enters
> per-CPU mode. On an eight-CPU system, widening the affinity union to
> all eight CPUs should produce a threshold of four, but the old
> calculation produces three and returns early. The MM therefore remains
> in per-CPU mode even though the widened CPU set makes per-task
> ownership preferable.
> 
> Calculate pcpu_thrs from nr_cpus_allowed directly. The users bound in
> __mm_update_max_cids() still controls when per-CPU mode is entered, so
> this does not change the threshold calculated when switching from
> per-task to per-CPU ownership.
> 
> Tested with an x86_64 QEMU reproducer using three users, an initial
> two-CPU affinity, and an eight-CPU affinity expansion. The baseline
> did not invoke mm_cid_work_fn() after the affinity expansion. The
> fixed kernel invoked the deferred work once and completed the switch
> back to per-task mode.

I think you're onto something.

We should cover those scenarios with selftests in
tools/testing/selftests/rseq/.

Thomas, I recall you had tests for those state transitions when
rewriting rseq mm_cid. Were those local or did they make it upstream ?

Thanks,

Mathieu

> 
> Fixes: fbd0e71dc370 ("sched/mmcid: Provide CID ownership mode fixup functions")
> Signed-off-by: Hui Su <sh_def@163.com>
> ---
>   kernel/sched/core.c | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 7885ff76e69f..0280ed132d0a 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -10845,11 +10845,10 @@ static void __mm_update_max_cids(struct mm_mm_cid *mc)
>   
>   static inline unsigned int mm_cid_calc_pcpu_thrs(struct mm_mm_cid *mc)
>   {
> -	unsigned int opt_cids;
> +	unsigned int nr_cpus = mc->nr_cpus_allowed;
>   
> -	opt_cids = min(mc->nr_cpus_allowed, mc->users);
>   	/* Has to be at least 1 because 0 indicates PCPU mode off */
> -	return max(min(opt_cids - opt_cids / 4, num_possible_cpus() / 2), 1);
> +	return max(min(nr_cpus - nr_cpus / 4, num_possible_cpus() / 2), 1);
>   }
>   
>   static bool mm_update_max_cids(struct mm_struct *mm)
> 
> base-commit: 587858367581b9c55c3690f4e63382ad622719d4


-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [PATCH] sched/mmcid: Fix affinity-triggered switch back to per-task mode

[Hui Su]

> Thomas, I recall you had tests for those state transitions when
> rewriting rseq mm_cid. Were those local or did they make it upstream ?

Thanks for the feedback.

I checked the current rseq selftests and the related MM CID history.
Thomas' January [0/4] series mentions a thread-pool emulator which was used
to stress-test the ownership transitions. The series did not include a
selftest patch, and I couldn't find equivalent transition coverage in the
current upstream tools/testing/selftests/rseq/.

In particular, I couldn't find a test covering the affinity-expansion case
which switches ownership from per-CPU back to per-task. Unless I missed an
existing test, I'll add a focused selftest for this case and send a v2.

Thomas, if that emulator is still available, or if there is a preferred way
to exercise these transitions, I'd be happy to use it as a reference for
the selftest.

Thanks,
Hui

Re: [PATCH] sched/mmcid: Fix affinity-triggered switch back to per-task mode

[Thomas Gleixner]

On Tue, Sep 15 2026 at 12:10, Mathieu Desnoyers wrote:
> On 2026-09-15 11:59, Hui Su wrote:
>> MM CID switches to per-CPU ownership when the number of users of an
>> mm exceeds the maximum number of CIDs. When the affinity union grows,
>> the per-CPU threshold is recalculated and the MM can switch back to
>> per-task ownership through deferred work.
>> 
>> mm_cid_calc_pcpu_thrs() incorrectly uses
>> min(nr_cpus_allowed, users) to calculate that switch-back threshold.
>> This makes pcpu_thrs no larger than users, so the
>> users < pcpu_thrs check in mm_update_cpus_allowed() can never succeed.
>> 
>> For example, with three users restricted to two CPUs, the MM enters
>> per-CPU mode. On an eight-CPU system, widening the affinity union to
>> all eight CPUs should produce a threshold of four, but the old
>> calculation produces three and returns early. The MM therefore remains
>> in per-CPU mode even though the widened CPU set makes per-task
>> ownership preferable.
>> 
>> Calculate pcpu_thrs from nr_cpus_allowed directly. The users bound in
>> __mm_update_max_cids() still controls when per-CPU mode is entered, so
>> this does not change the threshold calculated when switching from
>> per-task to per-CPU ownership.
>> 
>> Tested with an x86_64 QEMU reproducer using three users, an initial
>> two-CPU affinity, and an eight-CPU affinity expansion. The baseline
>> did not invoke mm_cid_work_fn() after the affinity expansion. The
>> fixed kernel invoked the deferred work once and completed the switch
>> back to per-task mode.
>
> I think you're onto something.
>
> We should cover those scenarios with selftests in
> tools/testing/selftests/rseq/.
>
> Thomas, I recall you had tests for those state transitions when
> rewriting rseq mm_cid. Were those local or did they make it upstream ?

They were local. I thought I had a test for that affinity widening part,
but it looks like I only tested the affinity shrinking.

But yes, the patch makes sense. I'll pick it up.

[tip: sched/urgent] sched/mmcid: Fix affinity-triggered switch back to per-task mode

[tip-bot2 for Hui Su]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     7dbc307b6c72a5412e3f4e9fa2e0e926925fc5d6
Gitweb:        https://git.kernel.org/tip/7dbc307b6c72a5412e3f4e9fa2e0e926925fc5d6
Author:        Hui Su <sh_def@163.com>
AuthorDate:    Wed, 16 Sep 2026 00:59:53 +09:00
Committer:     Thomas Gleixner <tglx@kernel.org>
CommitterDate: Tue, 20 Oct 2026 15:31:33 +02:00

sched/mmcid: Fix affinity-triggered switch back to per-task mode

MM CID switches to per-CPU ownership when the number of users of an mm
exceeds the maximum number of CIDs. When the affinity union grows, the
per-CPU threshold is recalculated and the MM can switch back to per-task
ownership through deferred work.

mm_cid_calc_pcpu_thrs() incorrectly uses min(nr_cpus_allowed, users) to
calculate that switch-back threshold.  This makes pcpu_thrs no larger than
users, so the users < pcpu_thrs check in mm_update_cpus_allowed() can never
succeed.

For example, with three users restricted to two CPUs, the MM enters per-CPU
mode. On an eight-CPU system, widening the affinity union to all eight CPUs
should produce a threshold of four, but the old calculation produces three
and returns early. The MM therefore remains in per-CPU mode even though the
widened CPU set makes per-task ownership preferable.

Calculate pcpu_thrs from nr_cpus_allowed directly. The users bound in
__mm_update_max_cids() still controls when per-CPU mode is entered, so this
does not change the threshold calculated when switching from per-task to
per-CPU ownership.

Tested with an x86_64 QEMU reproducer using three users, an initial two-CPU
affinity, and an eight-CPU affinity expansion. The baseline does not invoke
mm_cid_work_fn() after the affinity expansion. The fixed kernel invokes the
deferred work once and completes the switch back to per-task mode.

Fixes: fbd0e71dc370 ("sched/mmcid: Provide CID ownership mode fixup functions")
Signed-off-by: Hui Su <sh_def@163.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260915155953.2856524-1-sh_def@163.com
---
 kernel/sched/core.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1754d9f..76c6499 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -10852,11 +10852,10 @@ static void __mm_update_max_cids(struct mm_mm_cid *mc)
 
 static inline unsigned int mm_cid_calc_pcpu_thrs(struct mm_mm_cid *mc)
 {
-	unsigned int opt_cids;
+	unsigned int nr_cpus = mc->nr_cpus_allowed;
 
-	opt_cids = min(mc->nr_cpus_allowed, mc->users);
 	/* Has to be at least 1 because 0 indicates PCPU mode off */
-	return max(min(opt_cids - opt_cids / 4, num_possible_cpus() / 2), 1);
+	return max(min(nr_cpus - nr_cpus / 4, num_possible_cpus() / 2), 1);
 }
 
 static bool mm_update_max_cids(struct mm_struct *mm)