From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from LO0P265CU003.outbound.protection.outlook.com (mail-uksouthazon11022101.outbound.protection.outlook.com [52.101.96.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F252C3812E9 for ; Sat, 8 Aug 2026 23:55:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.96.101 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786233344; cv=fail; b=o7TjwdEeyApBMGv9AuXQdogG7ahs91Am0MVUt+4Ks/Hf/4KLUjJhoDur7qsrLc2VrGhxvcVfsYTL6QfZC3LMy+Azgn60RU4mForGsBCzB+i8mGSKLj57QCt902GiUzVxKULS2u7xxDgWsPpDfHDVKjZ6//z39OjW00EiFXSGD9Y= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786233344; c=relaxed/simple; bh=R8XpHP0BEHfCd6cr/D0/q2OQnUTbexeEL863itpAuM8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=dKp+icpFr2hG6tCYk7fBbHLMvvm1OtwCU6Bg3QU33xfglZZhMFTv57XWup5UVkRiMUzwBwi3tigAngZpGjXsL2uz45YYZAM0/mtYOaMenLxEvZsV73K/sNIbwp+FzeZp+wHmL28cKJyFal9H1/pMAgBM+eYiHT3xcAuYLu9vHXI= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com; spf=pass smtp.mailfrom=atomlin.com; arc=fail smtp.client-ip=52.101.96.101 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=atomlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=atomlin.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OlWFgHztx6DiaJ89Pxzt4iTFs1wj3CuZnEYCdwayprdgaSdrCtVZuV1JG8LDWYVonYefQ/qPA6YIIZ5KD/JFvzmKvfU7ABb8VHbZ7TREo5gQemigp5k3MHqE3znn8JcrkLr0oL6hc45onf65X1uNok1ZSvUje6mwQD5Lwk7S7U0xgaMxlTemp1bfrKBJLX9y7/7MEWGjHTge03TkB5TwWosGwOIIz+KKBOzwTnMrV+Qfl1iP6cDzZZI6rq3opBd80Pp9cDlS4t9SbN5jJpa5lPGcRNgtR7lGc+xEGUBYTTzxYIW0kpunsozkgOpQa8kW4jAt1Sz3n7tSUHxPCpHftw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:MIME-Version; bh=HQYhVylvDYjLsFP2kpLVR0AAuJBxMVAAsW//INzUk3o=; b=vHqaSvZR6ohe+q8BySrLUIjlh/z7ws2NjpVYadXGfiM9TTS0fTTr5o6Led1eraAbS4SXJ2/TNZBgaViqGYSQstDukQcT0graApQgru5aNPHZjHlIjEKqL3843YaYo3Zwzm75DVXZmh7W/fqRf22gbs9OEJUlp/qxqiLFcieZzDrTP3S0sCplUdiPf1/6Bb8nydytD0q8/xqbkk/v5dtKnyCNJBLL+2bemB0B0m0bBXlvqVsyhL2LhEfa5A565UO5v8httWF5QiZDj2JUKM2mX0e/8hLcg4rjWyDyXQcJyfhB+bbDZFQCauTK0gjP/A4cr1zeiOvk2n6kyRuNzIzMkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=atomlin.com; dmarc=pass action=none header.from=atomlin.com; dkim=pass header.d=atomlin.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=atomlin.com; Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) by CW1P123MB8837.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:274::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.25; Sat, 8 Aug 2026 23:55:38 +0000 Received: from CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230]) by CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM ([fe80::cec4:77ab:262e:d230%4]) with mapi id 15.21.0292.024; Sat, 8 Aug 2026 23:55:38 +0000 From: Aaron Tomlin To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com, zhanxusheng1024@gmail.com, neelx@suse.com, chjohnst@mail.com, mproche@mail.com, sean@ashe.io, steve@abita.co, linux-kernel@vger.kernel.org Subject: [PATCH v3 3/4] sched/fair: Use list_for_each_entry_rcu() in print_cfs_stats() Date: Sat, 8 Aug 2026 19:55:21 -0400 Message-ID: <20260808235522.380038-4-atomlin@atomlin.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260808235522.380038-1-atomlin@atomlin.com> References: <20260808235522.380038-1-atomlin@atomlin.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: BN9PR03CA0242.namprd03.prod.outlook.com (2603:10b6:408:ff::7) To CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM (2603:10a6:400:183::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CWLP123MB6607:EE_|CW1P123MB8837:EE_ X-MS-Office365-Filtering-Correlation-Id: c6428c04-01b8-4958-e22f-08def5a88ba3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016|23010399003|10067099003|56012099006|22082099003|18002099003|3023799007; X-Microsoft-Antispam-Message-Info: 4WRlj3EDmieJ0XgMfadQN97dMbMw+3ax0IH6y6mo/FwWfCBRRu0OIbcqifM1uTI0G9IcqMrSWDG7/v6M6b6hEiGsxWYSPkIOH1hPalYtCdt3BQMUWMcdj1im/MnfGim+oUqDuu8B+BXihrmmSFIt2tyL/bRwt7g2Jdc/+G6UjDvyO7B7fp7EA1DsYu/0/sbpivzRkTmq4eFSdO/d0NM5osYi0+iWBJQWJoGzX4Lv/Q44j0TtYRZsrfjvRfaAdElUUACC3Y5KrGdQOoyb7J8AGQAxuo5q0JZCaDLR92AojZKm0Dhx23m1tPUxLqOIZu7CbYpTH7VA97eFw/8OjYCwJTqg9pUEh/EbtkUC1GneS7rn9pAW8a7GyCCe46IcdviZw7g7jVnwtMaxLIqfkJDfsd4z7cXNH+rrAQQv6AJ+hS4aSclCTv+hBPbcRxN4I3r5K2mI8Y6Y6tCcP+3gCu57k2cOG6BIkW0aqgSMhNagGPNQfeAjAxfxqfLD7gRf4tYp6hR7Ms5C6w9f9jfP0xjLR1fCotpoJ8jzLuIPTWpPaTFdZYiNomuIPdJLSOQBk/S9wxxfpT4rThlmAnZCV0oktwpmUNgC+UNhy8EIHQXG2z1dcyvy8azIjMpSLOlT+2aAjuZz+icV66XngAWS1d2P5Wkco7FN0RXwWNVki3+HYw0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(366016)(23010399003)(10067099003)(56012099006)(22082099003)(18002099003)(3023799007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Bbb6FgWWvBpo9TThEJ8WbCawMj7fCtiMZMYyBTv65kZN+R89wkZyaV6Ol4IQ?= =?us-ascii?Q?y6SckL/4YVKxtHhbVUzjd3ZgA3yshWJmrbyZxeLQZwa/jPhdEWodP0wpKIhB?= =?us-ascii?Q?8th+p0+4l2iAOPiDIR3AA62Ekfj0ms8HPXRvedfVt2EsKiDGmJIzO1E08CIo?= =?us-ascii?Q?mPwPNhAnIDADH+F7RTXgHAm/GRFWRWucsY5sI/nsg+KXwxgHe3zKBnRGJdWU?= =?us-ascii?Q?3jl2I4dXUJ0a6T4bCAHTCBBzn6O8tNTzoMPj1tZRTLQngZuRNV0ECV/PlmXw?= =?us-ascii?Q?pST/+jXq3D+7Nr8EcIcFehnkvCoQXTLb8rTQBCEZXmJnMJlsPXbLBgs1h60e?= =?us-ascii?Q?Z7S9cJdD6hB62cZ4q9q42jXGMfNIZU2mAgfnesiLaknZgN1CO2pBham81Nr2?= =?us-ascii?Q?uXA+HV0NMdUlGY4lcPUes2BFOVAD/CvzbAEnZukv1Y079B3X1jTvA405tyAW?= =?us-ascii?Q?QHFcCx8PBHd6VWHukusAXrJlfKqjZUpFWvsVOABtp0ArB56BMq0d/ITD2mxG?= =?us-ascii?Q?md5QFc4yovBN6nxh68zH0Kv9//aKG9wFKDbQwb2ezG4IiS6zfGgLBt20TpjL?= =?us-ascii?Q?uvO407t6dHlH1YUg6FTY2NDJbaUYMXIkg0CZsDhxbOI6y/QjFuNCjh2mF4UU?= =?us-ascii?Q?HSw+8E4II4S6D6VJ3hSibOrBeRy9+jMis+ISf47QP6HZREjmjSUQFM2dF9ML?= =?us-ascii?Q?dMA8FDQuKcjifV0+8YrJ/iSQ3DupIAJVckmgOqDnF40TIZtW85q35Ps+WtZD?= =?us-ascii?Q?0a0nVj2G9hITjUAZ1GiLn0Br0Uy20E5qwEp/+e3N8jCmh26RpqR6a24PTzDE?= =?us-ascii?Q?QCbswjn8qUQ8G3nCqhkE9Odt7Upc+HrpvZDcWhTLBTr8Wmkrdl519XmRBunq?= =?us-ascii?Q?YifjHPwDcat0RnNOxw9HUBnhlCCRgF32pGPuYoKErixR9jZkJNYBPnVHpTJA?= =?us-ascii?Q?GrRXYjoZT/Pi7uhkm+APTP8U385bWvXTCrixJq4DBoQCKGLsrJ1O7gqe7EaN?= =?us-ascii?Q?D/YrHUUy/v8AqFg+ckLeqcqAqI6mXvunnsFyGzGurywwfxo8o3t0VMFlvjW2?= =?us-ascii?Q?aYTmVWyWscung3TAe8JtX5oICleQiJUXVL2BN1HKaE5wZ7gOQVqhPvssmcQc?= =?us-ascii?Q?UcJqsWaSun2ExTdBw4+CiilRIC5nLwxWuKQpPM7t6F6mHW2Jdz/wgmS01MlA?= =?us-ascii?Q?dF2ozI0bLzyZSM+p6GJGZekDik3+upgI0sZgmmiFNK/BhNcMa/6fDjTU8Rhx?= =?us-ascii?Q?06WfVM5iXzbSkuGEDEc5UKWx9Wy2qWS76lpcPK8jPLg8/4PAScqARPZp8eac?= =?us-ascii?Q?iBqb2q7STJZJJwaxNO8nv8EUsiqYaMF0UJGFSKD2yB+30Zow45XbZMnQjV48?= =?us-ascii?Q?9nkvOiafm3KKSIuhoPHbFJ9ulcs7uwF1o3lA6tLsneKTyfF/vqmChm0S0Ezt?= =?us-ascii?Q?c3ikaBcAU3A5n5K+8Il4Z+4/xqHZDeSqFY/We/i7/kAqWdQ8BDYjKnY+kk6X?= =?us-ascii?Q?9dC8jUH8ZjV4KhHv08ciPXMCTlqkaLz1WBDpKkQFPDYJm9FGlwPWH/8xYLXA?= =?us-ascii?Q?Ma9rWjH9GR7JCY28j9O7IlTq7KctNGKSj3iBqAUoDgv/ldNwgYnYlQ9w17ea?= =?us-ascii?Q?nJq1J+Lm2GXv9VR2C8jAMMArnKZxu7+DkWhJexCKiBFyOC+1Eybx43EGw9Np?= =?us-ascii?Q?sUz7b0Bk6IBdK0SIMEBoGJGKii06pqrXEfwlS65yXrcnqfvgiuzgjkqij45i?= =?us-ascii?Q?xtJna2Wg+A=3D=3D?= X-OriginatorOrg: atomlin.com X-MS-Exchange-CrossTenant-Network-Message-Id: c6428c04-01b8-4958-e22f-08def5a88ba3 X-MS-Exchange-CrossTenant-AuthSource: CWLP123MB6607.GBRP123.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2026 23:55:38.0860 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6a32402-7d7b-4830-9a2b-76945bbbcb57 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: R1VMwJcfvmD5xX7LdEphndnYd4rlEKacAG8psv6Xvq24SPRoLLfCzzP2LIUl1Alzz/xZnpJY2TuqsEjIH8k/oQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CW1P123MB8837 In print_cfs_stats(), rq->leaf_cfs_rq_list is traversed using for_each_leaf_cfs_rq_safe(), which expands to list_for_each_entry_safe(). Although rq->leaf_cfs_rq_list is RCU-protected, list_for_each_entry_safe() is a non-RCU iteration macro. It dereferences pointer links without READ_ONCE() and pre-fetches the next pointer. When a writer concurrently adds a new cfs_rq to the list using list_add_rcu(), a reader traversing with list_for_each_entry_safe() lacks READ_ONCE() protection. Without READ_ONCE(), the compiler is free to re-fetch pointers or reorder instructions. As a result, the reader can observe a newly inserted cfs_rq's pointer before its internal fields are fully visible, leading to reading uninitialised data or dereferencing invalid pointers. This is illustrated below: CPU 0 CPU 1 Writer (under rq->lock) Reader (print_cfs_stats) --------------------------- -------------------------- 1. Initialise cfs_rq rcu_read_lock() cfs_rq->tg = tg cfs_rq->load = 1024 2. list_add_rcu(&cfs_rq->list, ...): 3. for_each_leaf_cfs_rq_safe(): smp_store_release() Reads prev->next (cfs_rq) prev->next = cfs_rq -----> Missing READ_ONCE() 4. Reads cfs_rq fields; sees garbage data cfs_rq->tg cfs_rq->load Fix this by introducing for_each_leaf_cfs_rq_rcu(), which expands to list_for_each_entry_rcu(). This uses READ_ONCE() during list traversal. Fixes: 0601267ca673 ("sched/fair: Rewrite list_add_leaf_cfs_rq()") Reported-by: sashiko-bot Signed-off-by: Aaron Tomlin --- kernel/sched/fair.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d78467ec6ee1..25f941ad637a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -412,6 +412,10 @@ static inline void assert_list_leaf_cfs_rq(struct rq *rq) list_for_each_entry_safe(cfs_rq, pos, &rq->leaf_cfs_rq_list, \ leaf_cfs_rq_list) +#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq) \ + list_for_each_entry_rcu(cfs_rq, &(rq)->leaf_cfs_rq_list, \ + leaf_cfs_rq_list) + /* Do the two (enqueued) entities belong to the same group ? */ static inline struct cfs_rq * is_same_group(struct sched_entity *se, struct sched_entity *pse) @@ -497,6 +501,9 @@ static inline void assert_list_leaf_cfs_rq(struct rq *rq) #define for_each_leaf_cfs_rq_safe(rq, cfs_rq, pos) \ for (cfs_rq = &rq->cfs, pos = NULL; cfs_rq; cfs_rq = pos) +#define for_each_leaf_cfs_rq_rcu(rq, cfs_rq) \ + for (cfs_rq = &rq->cfs; cfs_rq; cfs_rq = NULL) + static inline struct sched_entity *parent_entity(struct sched_entity *se) { return NULL; @@ -15401,10 +15408,10 @@ DEFINE_SCHED_CLASS(fair) = { void print_cfs_stats(struct seq_file *m, int cpu) { - struct cfs_rq *cfs_rq, *pos; + struct cfs_rq *cfs_rq; rcu_read_lock(); - for_each_leaf_cfs_rq_safe(cpu_rq(cpu), cfs_rq, pos) + for_each_leaf_cfs_rq_rcu(cpu_rq(cpu), cfs_rq) print_cfs_rq(m, cpu, cfs_rq); rcu_read_unlock(); } -- 2.55.0