From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7AE6C61CE8 for ; Mon, 9 Jun 2025 22:56:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 27D226B008A; Mon, 9 Jun 2025 18:56:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 254596B008C; Mon, 9 Jun 2025 18:56:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16A0C6B0092; Mon, 9 Jun 2025 18:56:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EF4E46B008A for ; Mon, 9 Jun 2025 18:56:25 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 711C959242 for ; Mon, 9 Jun 2025 22:56:25 +0000 (UTC) X-FDA: 83537372730.01.3907DC8 Received: from out-171.mta0.migadu.com (out-171.mta0.migadu.com [91.218.175.171]) by imf13.hostedemail.com (Postfix) with ESMTP id A44762000B for ; Mon, 9 Jun 2025 22:56:23 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=eswxoG3G; spf=pass (imf13.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.171 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749509784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=j3bC3QS+YLVQDjDxB/dMj/zdRv0EphT3+T18L+fEShg=; b=f9wyVXWqsvrMl+zHsAVyCBQYeJI8MaDFLjtaQof6wwGf3aTt0/MaRcaCxgH6EIRCZC9MOa lM7Yg0H2OgwoWLTF2bnG6FzeZffLoNKJiUqxFL3+Bvtms8O89g8tdslZ8Zgu5T0btjS2zp qqTvDmZnNM1eDkMxwob/9uTb3uU2kVo= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=eswxoG3G; spf=pass (imf13.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.171 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749509784; a=rsa-sha256; cv=none; b=yI5c3ZErCIU6gLVTsQztaUXWrAskPvVzoPZh8NGC2+HmpjaH9QH7GvCXsHeWJzdltWRZw+ LxK8Gl9VN40Pjr39YG79DJNqAY41ABfBnTCtE1en5/xwduVRGuivlc/w+Lo0h7CPI4RUld 6IP+5SJ93OLNd69ATNx6OtMWv/t57I8= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1749509781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=j3bC3QS+YLVQDjDxB/dMj/zdRv0EphT3+T18L+fEShg=; b=eswxoG3GLoy2gFPGrmXzRUrJQnzdp7StbXSBdaIEFckRQ+Xr1i44A8obRqGLQyOcuIJA2g NATLzItcQAxPHnD5rHiC/nBSU/rC8PGx9wR01no1RefepK2ElUk/5mQJukswVhj1sryRJE 5SiXj4Br0I35cLEZ1F4doWgqWizaF5Y= From: Shakeel Butt To: Tejun Heo , Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Alexei Starovoitov , Sebastian Andrzej Siewior , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Harry Yoo , Yosry Ahmed , bpf@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH 0/3] cgroup: nmi safe css_rstat_updated Date: Mon, 9 Jun 2025 15:56:08 -0700 Message-ID: <20250609225611.3967338-1-shakeel.butt@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: A44762000B X-Stat-Signature: mx5u6hm7hyhn8jfcojtbqrcsb68ac9gt X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1749509783-143960 X-HE-Meta: U2FsdGVkX1+LjoqC6s2I6wde64MlKZ7ozlK/Zw9M7KJWg/CzwuQKpi7WKRLLJhUzxspX/seBPdRzeeEstPYGpXlQQtF5Psz++lFfvHyFPg+2ODYLd8hO0r8yMYvdIRyUqnpAn2EyqRKOnoXL0TMGe0OK053lAsO9t5Tp3Z3V8dWXy+GRd5EdnhXwrkMxjOrx7/BBrnQnfC6CKB0tUXxr4J/vKwFT/TN12K8YJJHrO3VXuL3cVJ0bHmIeJ/GXhDCPyrg3MprmngfdgftJQOkvhuvoaklFSDYgo2HSMzlWO1yEFRUHi43POm9Rcp/uVShZm3iUQ6CfwPDGMKVxNBnvYjo98+vD3z0az5SqV6WeS5pNRi/cWxpcje6/hlJDNB1Wpbb+Yq/v1Bx/lOXKSE8DyBq/YKBO3yKHEEbKXOXv0K31hJ+Zj6Qd+CvlPwKMNkbyT/f/UCQpYCEm64m57A4YgHapzk+zpWp34RQZXSB+1sl3v83zZh0SHmucfag16lEDlOYK0XFU1T/AzNDawJp59ACJfZc/YDay6EaBFARVQcCpsrf9EDi/wyxQ+Ie2zHNVEADJKPD1hG8aQsV1bad1V+FdMb9LogUIvX/k2VCSpiPp45MkAZyEHk2b4rBkOZM1BTw2Io+G7+VF/TNFztNuN3sFEtip2AJoPrQ0gQ3RfuMgTyjkxK+qFr2gcWgQZp62VaZCwtSobniWhiMIJVa22br1l70SpPXDxl/uEXJMSas9ozoskN6p3pfrOumg4LouLfst9ZIwg6qm7aFkoOUsgx2ew/dRnJEaNHSjTyt3QQvctl39bVoYQDMH4S8jOir5WypZDxXWKkdIa1r6/lO3Y70pgMYzt8ckODF2XrwJKmN+t6HS7NeBNwGnocVmdClSlWATUinHEgQWXstQOUR2Ui4i4uL0/hQSDZ1pndv+7gDc8/ZFxTU6HOq3ZNUoPm2QEEar8nO0cJNPQOpVhai LV3Y7i8k hO+MQFrke90Us/8+MHms0NP2ZMCmmEz8T3qaZdI5Wo5UdSthELC81f9p9SWFfTHZa8hZq3qksYdEQhJNSrytac4amvAQB6S0Jntw1PHANZICNN7HxxeYdo/42BqVUI06haFcEU8IIx64iG15pb9rJCAtrfNQ+vUAmtnX1qPzrPMzCoJOKc7rx6A17jHUxDXniQnkwG9aYUChwGqTcpNZc+3+YLcHkV/pHPnhDrYxzGyGlz7ZqKuXf0FzQaqCrmevb5O8j0ZH7tCxcCWCD3MUB683LDw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: BPF programs can run in nmi context and may trigger memcg charged memory allocation in such context. Recently linux added support to nmi safe page allocation along with memcg charging of such allocations. However the kmalloc/slab support and corresponding memcg charging is still lacking, To provide nmi safe support for memcg charging for kmalloc/slab allocations, we need nmi safe memcg stats and for that we need nmi safe css_rstat_updated() which adds the given cgroup state whose stats are updated into the per-cpu per-ss update tree. This series took the aim to make css_rstat_updated() nmi safe. This series made css_rstat_updated by using per-cpu lockless lists whose node in embedded in individual struct cgroup_subsys_state and the per-cpu head is placed in struct cgroup_subsys. For rstat users without cgroup_subsys, a global per-cpu lockless list head is created. The main challenge to use lockless in this scenario was the potential multiple inserters using the same lockless node of a cgroup_subsys_state which is different from traditional users of lockless lists. The multiple inserters using potentially same lockless node was resolved by making one of them succeed on reset the lockless node and the winner gets to insert the lockless node in the corresponding lockless list. Changelog since v1: - Based on Yosry's suggestion always use llist on the update side and create the update tree on flush side [v1] https://lore.kernel.org/cgroups/20250429061211.1295443-1-shakeel.butt@linux.dev/ Shakeel Butt (3): cgroup: support to enable nmi-safe css_rstat_updated cgroup: make css_rstat_updated nmi safe memcg: cgroup: call memcg_rstat_updated irrespective of in_nmi() include/linux/cgroup-defs.h | 4 ++ kernel/cgroup/rstat.c | 80 ++++++++++++++++++++++++++++++------- mm/memcontrol.c | 10 ++--- 3 files changed, 75 insertions(+), 19 deletions(-) -- 2.47.1