From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48B623126AD for ; Sun, 16 Nov 2025 20:49:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763326144; cv=none; b=QKtfvPWfzAqOW24lmnQf7H00++jQohh50TE3oPsX82WSf4dTq6ziTCra3lzPRE/dKCSfblCM1xbdRHmMs5gGn7VnWYv2q676S7jyga7rmgHc2evkgiIh7cmK62QmDveK/jqciEKxQXbQIza6eWBe32CVnXElPPZKsZTiJTe4Zp8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763326144; c=relaxed/simple; bh=WnUVkRAPvVW3mfAkOqh1HXAGS6jo2BnDDgA42ZvOLC8=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=eRTx0hjc8s/3NgMcrMJ2F2a6zJZsNXdsJ2XqNVymM56Sx7HGpA7IuPG+UPdKJ9Kwo7AoOI+gEdW2X6zh+OVdQXNi9wGFWhpOsM5jXGbKzormju4VW3DOQVrrrS+U1Yw//VAUdKj6dQjwYL/Hw1NI8/EUzovkTOrPQlIDr9HS39M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=IjOxmsLI; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=pKCRcgIj; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="IjOxmsLI"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="pKCRcgIj" Message-ID: <20251104075427.567240652@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1763326141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QubYrHjbKQY5DulA+/e6rLVwgn2JK4krZLQhvPTGzyI=; b=IjOxmsLIEZFpK4XqizCfDLdAmV8iReuvURqNhTESc+fEBZnj7/JnwC6RVB2jqTEQK0qaA8 aLvstkR9cPBI5gs3iOuBDpMCnUVBAAKmxPiLJHpT8aRnHsBV4cb8kqsSNs8C1cAyaQUznQ Uglf1n4PcVcvWzIGfiYVKzhZj19dsOXY45qqA2hBH6o5CyRmaNeZWRs32qQLDLYSXdOwHW RkNyJxquV7A7dC628JMTz7OB3h9GhBTHs06ReEpuEJe1MdYyyUgzuAtS0PIhwHcS0JDI9g yZK8lq3QXdIdgSLbSbUrhJzQKxdTIQdEq+v2/6qEqCGIHC2rxtxomkXpi/KSFw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1763326141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QubYrHjbKQY5DulA+/e6rLVwgn2JK4krZLQhvPTGzyI=; b=pKCRcgIjuOPC2wn6aUkAX07FKyB4DO+0ZvixgY4YTAgPVqfr9vjPG23Lejlk9y/5LkmoCk WtAQRd7KBe5zx/AA== From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , Gabriele Monaco , Mathieu Desnoyers , Michael Jeanson , Jens Axboe , "Paul E. McKenney" , "Gautham R. Shenoy" , Florian Weimer , Tim Chen , Yury Norov , Shrikanth Hegde Subject: [patch V4 14/20] sched/mmcid: Serialize sched_mm_cid_fork()/exit() with a mutex References: <20251104075053.700034556@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Date: Sun, 16 Nov 2025 21:49:00 +0100 (CET) Prepare for the new CID management scheme which puts the CID ownership transition into the fork() and exit() slow path by serializing sched_mm_cid_fork()/exit() with it, so task list and cpu mask walks can be done in interruptible and preemptible code. The contention on it is not worse than on other concurrency controls in the fork()/exit() machinery. Signed-off-by: Thomas Gleixner Reviewed-by: Mathieu Desnoyers --- include/linux/rseq_types.h | 2 ++ kernel/sched/core.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+) --- a/include/linux/rseq_types.h +++ b/include/linux/rseq_types.h @@ -125,6 +125,7 @@ struct mm_cid_pcpu { * do not actually share the MM. * @lock: Spinlock to protect all fields except @pcpu. It also protects * the MM cid cpumask and the MM cidmask bitmap. + * @mutex: Mutex to serialize forks and exits related to this mm */ struct mm_mm_cid { struct mm_cid_pcpu __percpu *pcpu; @@ -132,6 +133,7 @@ struct mm_mm_cid { unsigned int nr_cpus_allowed; unsigned int users; raw_spinlock_t lock; + struct mutex mutex; }____cacheline_aligned_in_smp; #else /* CONFIG_SCHED_MM_CID */ struct mm_mm_cid { }; --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10370,6 +10370,25 @@ void call_trace_sched_update_nr_running( #ifdef CONFIG_SCHED_MM_CID /* + * Concurrency IDentifier management + * + * Serialization rules: + * + * mm::mm_cid::mutex: Serializes fork() and exit() and therefore + * protects mm::mm_cid::users. + * + * mm::mm_cid::lock: Serializes mm_update_max_cids() and + * mm_update_cpus_allowed(). Nests in mm_cid::mutex + * and runqueue lock. + * + * The mm_cidmask bitmap is not protected by any of the mm::mm_cid locks + * and can only be modified with atomic operations. + * + * The mm::mm_cid:pcpu per CPU storage is protected by the CPUs runqueue + * lock. + */ + +/* * Update the CID range properties when the constraints change. Invoked via * fork(), exit() and affinity changes */ @@ -10412,6 +10431,7 @@ void sched_mm_cid_fork(struct task_struc WARN_ON_ONCE(!mm || t->mm_cid.cid != MM_CID_UNSET); + guard(mutex)(&mm->mm_cid.mutex); guard(raw_spinlock)(&mm->mm_cid.lock); t->mm_cid.active = 1; mm->mm_cid.users++; @@ -10431,6 +10451,7 @@ void sched_mm_cid_exit(struct task_struc if (!mm || !t->mm_cid.active) return; + guard(mutex)(&mm->mm_cid.mutex); guard(raw_spinlock)(&mm->mm_cid.lock); t->mm_cid.active = 0; mm->mm_cid.users--; @@ -10467,6 +10488,7 @@ void mm_init_cid(struct mm_struct *mm, s mm->mm_cid.nr_cpus_allowed = p->nr_cpus_allowed; mm->mm_cid.users = 0; raw_spin_lock_init(&mm->mm_cid.lock); + mutex_init(&mm->mm_cid.mutex); cpumask_copy(mm_cpus_allowed(mm), &p->cpus_mask); bitmap_zero(mm_cidmask(mm), num_possible_cpus()); }