From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25C36C53209 for ; Mon, 27 Jul 2026 12:54:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 224A26B00DB; Mon, 27 Jul 2026 08:54:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D5BA6B00DC; Mon, 27 Jul 2026 08:54:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C45F6B00DD; Mon, 27 Jul 2026 08:54:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6D34D6B00DB for ; Mon, 27 Jul 2026 08:54:11 -0400 (EDT) Received: from smtpin07.hostedemail.com (lb01a-stub [10.200.18.249]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C1D651606BC for ; Mon, 27 Jul 2026 12:54:10 +0000 (UTC) X-FDA: 85034549460.07.E2BCEF6 Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf22.hostedemail.com (Postfix) with ESMTP id 2D677C0004 for ; Mon, 27 Jul 2026 12:54:09 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20260515 header.b=ls8MpmHA; spf=pass (imf22.hostedemail.com: domain of vbabka@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=vbabka@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1785156849; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=y9olFz4RC6C1U7zYxBQoZxiDRdf5OldBtN1enHJw5Kc=; b=lvOMdwZAya8frMSTVBqUspuvP8ZlcX1y+YOfR4t9gOxtlm6vptVz6AV8EsW2zbJ6eAR3IV SPwqaW96Un75ms2vTDbYvscFbfQyd6th/8blZghY+3SJxu9hHj66VwGhwnevLKQT0/OQLt WPXWHZZLdWrJrd+Bv/WQNRXNDm5fdbU= ARC-Seal: i=1; a=rsa-sha256; d=hostedemail.com; s=arc-20220608; cv=none; t=1785156849; b=S0QgFyIoGkJzY1WswI2Ln1bomr71+wj4zzDMVdMwuYkXipRkjvfNnkOQKp8WrZ3KfGpaf4 pj7qaUqaJLri8CRgsg5HA75p76OoC4Y7qvHQ2ePowJPFkyhoKF40d6nGhjf2gxEF1W4EOs bf6RsM4GF6mhq7f1YMezr5WLHR0jlY0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20260515 header.b=ls8MpmHA; spf=pass (imf22.hostedemail.com: domain of vbabka@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=vbabka@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by tor.source.kernel.org (Postfix) with ESMTP id 6A262600AA; Mon, 27 Jul 2026 12:54:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 548CD1F00A3A; Mon, 27 Jul 2026 12:54:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785156848; bh=y9olFz4RC6C1U7zYxBQoZxiDRdf5OldBtN1enHJw5Kc=; h=From:Subject:Date:To:Cc; b=ls8MpmHAhsKvwNvNa/Q/2fVB4LTcEBCBfbTZKm2UUX2BO7+/BUTL7sNffmT1Ve+nY aeBznRVglidHOLdmOzrmE+NWF3ZmFlV3K14BdaSDfDCGk4lKpIYtOYnK+TlI6+QcU2 mB8hFL5MLoflRqpTyupnU5a/1A18lQ/pgZZKbbZQffF04SSTqxmC7KTf7wA4piH2dp 9Ml+7RJN7lFQex4koS3qSgxEW/NPz8yzLUOcSFy1QqW2XszJCM8nnanHdeL3o/ROdx PKYNROUsWzXLiUVGTiZ8rFzf9RYVl6zvQwhXYwB6xUk4sOXeEDFiRYgti7leuPhbNT XaxZHjsBTBsqg== From: "Vlastimil Babka (SUSE)" Subject: [PATCH v3 00/13] mm/slab, alloc_tag: reduce obj_ext memory waste Date: Mon, 27 Jul 2026 14:53:54 +0200 Message-Id: <20260727-b4-objext_split-v3-0-c29ef0f1f257@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-B4-Tracking: v=1; b=H4sIAOJUZ2oC/2XN0QqCMBgF4FeJXbeYv1Ozq94jIub2qytxsq1hi O/eNILCywPnfGciDq1GR067iVgM2mnTx5Dud0S2om+QahUzAQY5KxJOK05NdcfR39zQaU+VOAK HHLJCZSSuBou1Hlfxcv1k94wD6RdmabTaeWNf62VIlt5XzzZ6SCijpeCl5FLWXKbnB9oeu4OxD Vn4AD8AsC0AEYBaFDKvc6Yq/APmeX4D9bEfMgIBAAA= X-Change-ID: 20260714-b4-objext_split-da82426257d5 To: Harry Yoo , Suren Baghdasaryan Cc: Hao Li , Shakeel Butt , Alexander Potapenko , Marco Elver , Andrew Morton , Christoph Lameter , David Rientjes , Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, "Vlastimil Babka (SUSE)" X-Mailer: b4 0.15.2 X-Rspamd-Queue-Id: 2D677C0004 X-Stat-Signature: mmfqjy3juh475jg5maqg8f67uht6cpg1 X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1785156849-744759 X-HE-Meta: U2FsdGVkX18oNFlmTNcquOSXkrULAFKQo56WulL7Vd5LNVz+06NjYedsOQHnDU7fwraCmU3aGl+WVYEsIMM28rC8d2d8vl3pxZt9ZB0yMHxAtX1QltphndG0/3Fl4JK7/h0JdI/Ahegoc91syQq2L3yAr5xjfRtX9qzJ3AFyKWmGLz16BRFJhgdUzbxYoh09nI9iwcd4dHFJ3IOmVaLyUJtpsCbTM9SW7rTWkLd5nVBtXHYiYV/RgEOcFgDq70BIwqWt5IRw/Cprwd1FQKj1g3fkZYT5S4OeNa4bNl3Ov5P0yZM7S+X4MNlgVjczBozzoqi8P6igzUCdTsdFS7NasBPYYnRfs5hzfoXExyTPsknZD8x15W6c4GKj19QvIH4ucPv/dnIwdCz/Lvt/yWu7WCfcySWlxZTdQA/38VItg2yNGiGj5njvaLaoUcIiumMp9Be4o2PNbuaTcqGgHRgmQfE5MGXD0fwLOR7LOzanCsr+gRQ42U1ntiN76QffpQYnj7CTOcA3HvIRXogZC1WkchYD7JttM16O1SOVFElDWlGzVQTMK4XtQKLCDjp7cf/AzPWDBEDr2ghhLVv22Bf9KjOYQmRSVbmQr+R2gSxu9dipEG0I525bnpvoHMfO4GpJM1kKXRjfrWUAjPzC2Kao9x+SGbQ+kEuqg82evb68C1DG8aC/i++a2itv3v5e45LabmxEokcTeQzWnWgIrdsLhtynXZCtqwOHVMJZBpQ9/bd145cS1Ju9tBrVxyLDPTmsEvLBaTlDOVcyQtlk8nJJsr3StgDpspO0fQq8tGvTm7iL398MkRWDnmp+CTkabLFYByarwwTvDzsXY334v5Y+WS4uoiy9XpazcAbIKLHi0sOHWmIeTq4qozvH1rfja6290HSnFkPDrYot1S7mCr6tusgq+sI9HLgaZK3pPuTgALwjZLyMduZcW5E8W8LYZShs3h+Ljyp1JUUkFOAsveC 41tP/tF7 uFyAWPlJQ7L01/fJrShink9IQPWhX+nM0fkDUleyqcbqmqEjx2VDX1WcAsiXwGeD8b4Gb1LdTKSCWgid7UJkzkZu3yyM+IwpSjLdkh7QOlKPqyz7Y9Dop4ty7n/r7PuqT1ugGXoKKHYdKb8qjXpPqQ45bAxuCARuF+Df+YqLwaJ4fvNjpxJUsGykaWwamiXrs40RZlLtnuUTN/97hFYd3mo6uALg6IQbrw8Y/bYr33ziQiITqTs7m65OziPrFGvvznFd2GBS2I1ODalmKhefGT0IhBQAdp4s0yna2RRbdFSnc1yQ5uSve3PxZNfCNeQ6Ci0/zas2f2CGc16Gtu30ErpoDNUSYP1+O6NZa+wnerUJ7Bo7DON2TJB5OvA== Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The recent fixes for objext array handling inspired me to look into this finally. It's been bothering me that the memory usage of struct slabobj_ext depend only on config options and not whether the fields are actually used. So with both CONFIG_MEMCG=y and CONFIG_MEM_ALLOC_PROFILING=y there is always objcg field and codetag_ref field. And thus: 1) Having memory allocation profiling config-enabled but not boot-enabled means wasted memory on unused codetag_refs. This makes it less suitable for a general distro config and the page allocator side doesn't suffer from this, only slab and percpu. 2) Complementary, with memory allocation profiling enabled, there are caches/slabs that don't need the objcg field, so memory is wasted on those. This series should solve the point 1) fully for slab, pcpuobj_ext handling can be perhaps improved similarly, haven't looked into that. For 2) it avoids allocating objcg fields for KMALLOC_NORMAL and KMALLOC_NO_OBJ_EXT caches where we know they are not necessary because kmalloc() with __GFP_ACCOUNT will pick a KMALLOC_CGROUP type (except with SLUB_TINY). The named kmem_caches are tricky. They can be created with SLAB_ACCOUNT and then we know objcg fields are always needed. But also they can be created without SLAB_ACCOUNT and then some allocations have __GFP_ACCOUNT and some not and we don't know that in advance. This series introduces a SLAB_MAY_ACCOUNT flag that's currently internal only and is applied to all caches (unless kmem accounting is disabled) except KMALLOC_NORMAL (unless that aliases KMALLOC_RECLAIM) and KMALLOC_NO_OBJ_EXT. As a followup we can make SLAB_MAY_ACCOUNT explicit and add it to to caches where we know __GFP_ACCOUNT is used. Then we could only honour __GFP_ACCOUNT for those, while warning for an unexpected usage elsewhere. Based on slab/for-next-fixes Git branch: https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/objext_split Moderately tested, nothing seems to crash. To check for regressions, I forward-ported a microbenchmark hacked into slub_kunit that was used to evaluate sheaves. You can find it here along with the series: https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=objext_split-v3-bench Tried 3 scenarios, MEMCG and KFENCE were always enabled: - CONFIG_MEM_ALLOC_PROFILING=n - CONFIG_MEM_ALLOC_PROFILING=y but _ENABLED_BY_DEFAULT=n - same but booted with sysctl.vm.mem_profiling=1 The results are quite noisy, but no regression was apparent, except perhaps few percents for the last case. I don't expect it will be visible in any real workloads. Signed-off-by: Vlastimil Babka (SUSE) --- Changes in v3: - Review tags from Harry, Hao, Suren - thanks! - Patch "skip handle_failed_objexts_alloc() with profiling disabled" removed, the check is done as part of "reduce slabobj_ext memory with allocation profiling disabled" using slab_obj_ext_has_codetag() (Harry). - Move kfence checks earlier in mark_obj_codetag_empty() in Patch 1 (Harry). - Remove SLAB_MAY_ACCOUNT also for KMALLOC_NO_OBJ_EXT caches and more precise description in Patch 11 (Hao) - Add a new cleanup patch 13 that removes also memcg accounting from kfence allocations, as suggested by Harry. - Link to v2: https://patch.msgid.link/20260720-b4-objext_split-v2-0-2fa7c6f60dbe@kernel.org Changes in v2: - Apply Suren's R-b:, thanks! - Expanded explanation about no longer accounting KFENCE objects (Suren) - Also skip them in in mark_obj_codetag_empty() (sashiko) - Update slab_obj_ext() comments (sashiko) - Separate slab_obj_ext_objcg() and slab_obj_ext_set_objcg() (Suren) - Add SLAB_MAY_ACCOUNT internal flag instead of relying on is_kmalloc_normal(); also handle mem_cgroup_kmem_disabled() - Also fix the SLUB_TINY kmalloc aliasing handling, per Harry. - Link to v1: https://patch.msgid.link/20260715-b4-objext_split-v1-0-9a49c4ccf4c3@kernel.org --- Vlastimil Babka (SUSE) (13): mm/slab: skip kfence objects in allocation profiling mm/slab: remove objs_per_slab() mm: move struct slabobj_ext to mm/slab.h mm/slab: make slab_obj_ext() determine object index mm/slab: abstract slabobj_ext.objcg access mm/slab: abstract slabobj_ext.ref access mm/slab: replace slab.stride with obj_exts_in_object mm/slab: change struct slabobj_ext to a union mm/slab: introduce slab_obj_ext_has_codetag() mm/slab: reduce slabobj_ext memory with allocation profiling disabled mm/slab: add cache_ and slab_needs_objcg() helpers mm/slab: stop allocating objcg pointers when unnecessary mm/slab, kfence, memcg: completely remove obj_ext for kfence objects Documentation/mm/allocation-profiling.rst | 7 ++ include/linux/memcontrol.h | 13 -- include/linux/slab.h | 3 + mm/kfence/core.c | 12 -- mm/kfence/kfence.h | 3 - mm/kfence/kfence_test.c | 2 +- mm/memcontrol.c | 40 ++++--- mm/slab.h | 189 ++++++++++++++++++++++++------ mm/slab_common.c | 26 +++- mm/slub.c | 184 ++++++++++++++++++----------- 10 files changed, 325 insertions(+), 154 deletions(-) --- base-commit: 5ff172f6c94d282d83cb88bdfec5f647ad9c6105 change-id: 20260714-b4-objext_split-da82426257d5