From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACD58C83F2D for ; Thu, 29 Aug 2024 16:00:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20F7D6B0085; Thu, 29 Aug 2024 12:00:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E62F6B0088; Thu, 29 Aug 2024 12:00:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 087786B0089; Thu, 29 Aug 2024 12:00:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D18186B0085 for ; Thu, 29 Aug 2024 12:00:55 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7324EA87BF for ; Thu, 29 Aug 2024 16:00:55 +0000 (UTC) X-FDA: 82505746470.24.ED89624 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf16.hostedemail.com (Postfix) with ESMTP id ED42E180028 for ; Thu, 29 Aug 2024 16:00:51 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sDtH60jf; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of surenb@google.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724947179; a=rsa-sha256; cv=none; b=EOjwHkp/1NZ9baL28/m0/9L+jIINTiHuAhaZpNj+FoH5QaLuu9if7DphI0Tv3Y+BI1SxGR S3q4hYihGF3QIsnnDn8aUfonVkqDlcwN7XD/4dM5mEkIo2TELNjNBAuLuvbKSNbuwl43AG 9NpaxtV5Fb350CWvJoWAtLV3CFb9NPM= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sDtH60jf; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of surenb@google.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724947179; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=faZRfuIh6tyx/STII3KFiHR3ZroVBJPLCECLO13IVSA=; b=v6nZZ+3mV2iz/nQpSdbqJzVkgiAZ5+O/RymmnImTXUROd0RZMEaj+G6Vq7+89rxrQfONFo Cq08LYXU4WRiOoENiF4Whnolwv8S/etIhuZUxrLDYRs4/Ely4G2HNsgOjdk1EK/JUl4G2m 8E+4ENwU3trpx87SL4PrjLJ2oBHmS70= Received: by mail-qt1-f170.google.com with SMTP id d75a77b69052e-4567fe32141so375481cf.0 for ; Thu, 29 Aug 2024 09:00:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724947251; x=1725552051; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=faZRfuIh6tyx/STII3KFiHR3ZroVBJPLCECLO13IVSA=; b=sDtH60jfrrsVbOM/nG+OLhYX7ygHTDCfMxirbJTzlicxuTJMrj8B3fm2KrHoWweiWV rxoC6WUxSd8hUqZqI+8qNnLl8tUlyzHNHCjJw03chgyXfbDCyx+J5SKIMGl+g9V2rh7E IgRoSOeyviYYR6RiU+55Topa0TMOkbSAGPVbXns1W+e7YFjov0rp/ahrtsVLkjoG+Jf5 sDQhNHBrmDL1aZDbT8I5N0ObFDaM0EBgPVAZZSBtMssMLnlFrD53nPKsunq3tecFK2o4 4Qyba5SvGseSJzd30sW51iIqQXWplClR53dooI+qDsKWsrIMo1i9KSYx1W1ibDIB927I W2kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724947251; x=1725552051; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=faZRfuIh6tyx/STII3KFiHR3ZroVBJPLCECLO13IVSA=; b=LbPJBvFDj3ig4SM5JyqaFh2MJPyBZYkr8zaUmR5n1J+4Xoz6D0ud/NCiEh/CQc6ww+ wQPQKxm/NHMwGTjjvt2dNiAH32L/+mHd4SGJryPIGUeSSY6/WeSI1fA/kJfzjtfPfGSM UD/u6TYK/14PMrPqaNjSZ+Dbs60ZoN540tu/Q2nk2VpbN/ZJSY0M4tMM2LiByGLbj/6O r4xTHeeTHi1I5fc2t/YL1hfznVfd+uSs4iz1BDZb9/in53taF+qG+Bl3dGIq/A2Bf3ta nc+5R9pIpDyhzP07gWSedp544hNR5/MtElh3Zy4wn+kpRoXvfYt8F12yPe95rQzr9M3K KyJg== X-Forwarded-Encrypted: i=1; AJvYcCU1j7T0Gw7ZRaDlr11podi4zw4jyF84BS+v0uSxYn9zj1r8h2oLoSpXYW8/yTUJdgctK7eyWK1ffg==@kvack.org X-Gm-Message-State: AOJu0YxsIewy06wcQOTYeNRXGfMnTUkOEEAl6lZu30/Izy64FcSWd+5s HYFryZgnag7+x81Xa5k9uGMyrQiHuk2XzbLAKSMj4bNTIK3hjQxijOxRIgbBzNthb3fzi7UDCzh astfPsWCaqCwZB+F6Rb1l1whpopfUL2CsOWBm X-Google-Smtp-Source: AGHT+IGClq65i5Muaoi1TfggLqS1pEc+H5xQXkuciLf9CNJ+ANij1+dnSrrj3ldpiRigHV4bJA1zM0r7w8EHkaU6qQY= X-Received: by 2002:a05:622a:83cc:b0:447:e847:486 with SMTP id d75a77b69052e-45681509806mr3139301cf.3.1724947250230; Thu, 29 Aug 2024 09:00:50 -0700 (PDT) MIME-Version: 1.0 References: <20240809072532.work.266-kees@kernel.org> <20240809073309.2134488-4-kees@kernel.org> In-Reply-To: <20240809073309.2134488-4-kees@kernel.org> From: Suren Baghdasaryan Date: Thu, 29 Aug 2024 09:00:37 -0700 Message-ID: Subject: Re: [PATCH 4/5] alloc_tag: Track fixed vs dynamic sized kmalloc calls To: Kees Cook Cc: Vlastimil Babka , Kent Overstreet , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, "GONG, Ruiqi" , Jann Horn , Matteo Rizzo , jvoisin , Xiu Jianfeng , linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: ED42E180028 X-Stat-Signature: j4a8gkp9c71zkfuynppuxcmtajq9ep3w X-Rspam-User: X-HE-Tag: 1724947251-879624 X-HE-Meta: U2FsdGVkX1/ZZRIThX1bmhZx6OgjKbvgvrzxfzGNPw7FJzIvqUXb8oG+B2ww76ZQNQwCKNZpH1DG6ELtM068c9xoyyv6nKjorEF3sQSQ3gmGOXBvSyiXzF2GVIFfTaXGmLKCK64iarP7BMYkM69KXNIl6oAdnM1X/oK9bePH7I3ATdf6YuZteUIDFkbSI6ZFIP8BdnvYBC2EEBetNj9MRj6xZRXzK/JMAOroRfsfijJ3PGhPiCXc+2P28eIQlSydKtapEVyTgdk9raA2365AEJanv/9W93N7zlCjrsxTHZd29xRB0yVeD3se+7GF5/upWB9DyKJucwIkKNvQL296xuoZumCHJfxbiY6Z96eQoZOQz/86eRU9dLoDNbTYgDcLTJhXV1qzUVJaKFzUlyYypVl9bYgjKLEdz6R/EkmmQWP6+tt9p7eUSte18TWMIP4EgIcHSLwjk3ilzvseimnzjUiaysmclDt25eDDAMerpKehGgTzBv/ghTcNshBQVoC01VRSbmLOgvx9jICulcOrw9R3qCvpDhcPY1uzeklvXKyelht6nCikL7wqBUmVdz3iVkRH013u6sEpRY6dXMvylxROcPn+ODGXN6huL7hgFwDrkSs8w5UNec52WUA3RzUsfZkeLsCwH+XgPjb9Jx6ijszkj3Sd8Y0kKTZoB9KxH/xXnelxCsNHRUsuMeSj8RSVKVjy/N7aVKazgzAy/RwiWdr6TLPs5trFIXnXyX0CgQN3rSBk7+zyFXPvaqFq/GzqAq4TLIzPPWEVJ9mGAbXE1b/1L6el8mgfr8Ei2+T4ytds2fTIxF9KgfmteSx6nRh8FDIjC8tOyWq6ipuiohCzMr8ZdOcs5hbwngcR3Qz+wmZuV+rt7Ctt+Ls2OgliQ5RLQ/jfTMZJBJo8GWZh25xIujrQK6NFCjgMneDDlLTG10hprJvaMrtchUXyyyKiYcbRNjKunHDbLuOjUrtSwEz ZiVmvhmC 4qULoHJ0wMK3Yw4DRgaYpNOTus8KVv4Jcy2BvMTfl2bmc6XyKwprbSJ80eBHwQSEk1EgVusmqOKR0SvkIbuXtXg4h/8ucVYw7nBGm+FrL7iy7J5VNag3zc4fp864PKGHFADbynl/d8eTpm3xgNHSGAYWGcOkn9simEAkGgy9eDrD81Yw1KC1PnuIbbLNsqIIzRbJOg8HREMLJYjMcXyVJ05ZbhehcAxIWVPV5EigNqr+lA4abykCgxMa7GmVFQiL7CFIqEXgcm93qBVo6LwsBay2LFStqjLl6gZaTX7fwTLWRaAoNXxCevZG/KSsLykBsgAFPyPE4CzlLEIOKMMjRxbn4thBd0HyqNiQimUaQ7TxFLas5F83jeHEvank1KzKW8laKVlJGPZe2tyq4u20moGm+xQFdx/LO5y/URXZQJ4jrx0M/6W908AgLXvcRGVjrDLKZr5qUre65uodBVj9xtWr10sSmtgO78qWp7uTTh1tq0uCnOuDjrAt5IzusBqO6TbHDbFT5arrx8ya7o3e25rcBm681jbdDyUJ3zyXh4Nwi7CJJRBWrs01DucdiBCgM82p2XUrGjeFHmDyA6YZZa8sx36g08Z0SG5AorOT5fTs+uU1/ebrbRbBiIW6LlZiApZqv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Aug 9, 2024 at 12:33=E2=80=AFAM Kees Cook wrote: > > For slab allocations, record whether the call site is using a fixed > size (i.e. compile time constant) or a dynamic size. Report the results > in /proc/allocinfo. > > Improvements needed: > - examine realloc routines for needed coverage > > Signed-off-by: Kees Cook > --- > Cc: Suren Baghdasaryan > Cc: Kent Overstreet > Cc: Vlastimil Babka > Cc: Christoph Lameter > Cc: Pekka Enberg > Cc: David Rientjes > Cc: Joonsoo Kim > Cc: Andrew Morton > Cc: Roman Gushchin > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Cc: linux-mm@kvack.org > --- > include/linux/alloc_tag.h | 30 ++++++++++++++++++++++++++---- > include/linux/slab.h | 16 ++++++++-------- > lib/alloc_tag.c | 8 ++++++++ > mm/Kconfig | 8 ++++++++ > 4 files changed, 50 insertions(+), 12 deletions(-) > > diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h > index 8c61ccd161ba..f5d8c5849b82 100644 > --- a/include/linux/alloc_tag.h > +++ b/include/linux/alloc_tag.h > @@ -20,6 +20,19 @@ struct alloc_tag_counters { > u64 calls; > }; > > +#ifdef CONFIG_SLAB_PER_SITE > +struct alloc_meta { > + /* 0 means non-slab, SIZE_MAX means dynamic, and everything else = is fixed-size. */ > + size_t sized; > +}; > +#define ALLOC_META_INIT(_size) { \ > + .sized =3D (__builtin_constant_p(_size) ? (_size) : SIZE_= MAX), \ > + } > +#else > +struct alloc_meta { }; > +#define ALLOC_META_INIT(_size) { } > +#endif > + > /* > * An instance of this structure is created in a special ELF section at = every > * allocation callsite. At runtime, the special section is treated as > @@ -27,6 +40,7 @@ struct alloc_tag_counters { > */ > struct alloc_tag { > struct codetag ct; > + struct alloc_meta meta; > struct alloc_tag_counters __percpu *counters; > } __aligned(8); > > @@ -74,19 +88,21 @@ static inline struct alloc_tag *ct_to_alloc_tag(struc= t codetag *ct) > */ > DECLARE_PER_CPU(struct alloc_tag_counters, _shared_alloc_tag); > > -#define DEFINE_ALLOC_TAG(_alloc_tag) = \ > +#define DEFINE_ALLOC_TAG(_alloc_tag, _meta_init) = \ > static struct alloc_tag _alloc_tag __used __aligned(8) = \ > __section("alloc_tags") =3D { = \ > .ct =3D CODE_TAG_INIT, = \ > + .meta =3D _meta_init, = \ > .counters =3D &_shared_alloc_tag }; > > #else /* ARCH_NEEDS_WEAK_PER_CPU */ > > -#define DEFINE_ALLOC_TAG(_alloc_tag) = \ > +#define DEFINE_ALLOC_TAG(_alloc_tag, _meta_init) = \ > static DEFINE_PER_CPU(struct alloc_tag_counters, _alloc_tag_cntr)= ; \ > static struct alloc_tag _alloc_tag __used __aligned(8) = \ > __section("alloc_tags") =3D { = \ > .ct =3D CODE_TAG_INIT, = \ > + .meta =3D _meta_init, = \ > .counters =3D &_alloc_tag_cntr }; > > #endif /* ARCH_NEEDS_WEAK_PER_CPU */ > @@ -191,7 +207,7 @@ static inline void alloc_tag_sub(union codetag_ref *r= ef, size_t bytes) > > #else /* CONFIG_MEM_ALLOC_PROFILING */ > > -#define DEFINE_ALLOC_TAG(_alloc_tag) > +#define DEFINE_ALLOC_TAG(_alloc_tag, _meta_init) > static inline bool mem_alloc_profiling_enabled(void) { return false; } > static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_ta= g *tag, > size_t bytes) {} > @@ -210,8 +226,14 @@ static inline void alloc_tag_sub(union codetag_ref *= ref, size_t bytes) {} > > #define alloc_hooks(_do_alloc) \ > ({ \ > - DEFINE_ALLOC_TAG(_alloc_tag); \ > + DEFINE_ALLOC_TAG(_alloc_tag, { }); \ > alloc_hooks_tag(&_alloc_tag, _do_alloc); \ > }) > > +#define alloc_sized_hooks(_do_alloc, _size, ...) \ > +({ \ > + DEFINE_ALLOC_TAG(_alloc_tag, ALLOC_META_INIT(_size)); \ > + alloc_hooks_tag(&_alloc_tag, _do_alloc(_size, __VA_ARGS__)); \ > +}) > + > #endif /* _LINUX_ALLOC_TAG_H */ > diff --git a/include/linux/slab.h b/include/linux/slab.h > index 86cb61a0102c..314d24c79e05 100644 > --- a/include/linux/slab.h > +++ b/include/linux/slab.h > @@ -541,7 +541,7 @@ static_assert(PAGE_SHIFT <=3D 20); > */ > void *kmem_cache_alloc_noprof(struct kmem_cache *cachep, > gfp_t flags) __assume_slab_alignment __mall= oc; > -#define kmem_cache_alloc(...) alloc_hooks(kmem_cache_al= loc_noprof(__VA_ARGS__)) > +#define kmem_cache_alloc(...) alloc_hooks(kmem_cache_alloc_nopr= of(__VA_ARGS__)) nit: seems like an unnecessary churn. > > void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru = *lru, > gfp_t gfpflags) __assume_slab_alignment __mal= loc; > @@ -685,7 +685,7 @@ static __always_inline __alloc_size(1) void *kmalloc_= noprof(size_t size, gfp_t f > } > return __kmalloc_noprof(size, flags); > } > -#define kmalloc(...) alloc_hooks(kmalloc_nopro= f(__VA_ARGS__)) > +#define kmalloc(size, ...) alloc_sized_hooks(kmalloc_noprof, size, _= _VA_ARGS__) > > #define kmem_buckets_alloc(_b, _size, _flags) \ > alloc_hooks(__kmalloc_node_noprof(PASS_BUCKET_PARAMS(_size, _b), = _flags, NUMA_NO_NODE)) > @@ -708,7 +708,7 @@ static __always_inline __alloc_size(1) void *kmalloc_= node_noprof(size_t size, gf > } > return __kmalloc_node_noprof(PASS_BUCKET_PARAMS(size, NULL), flag= s, node); > } > -#define kmalloc_node(...) alloc_hooks(kmalloc_node_= noprof(__VA_ARGS__)) > +#define kmalloc_node(size, ...) alloc_sized_hooks(kmalloc= _node_noprof, size, __VA_ARGS__) > > /** > * kmalloc_array - allocate memory for an array. > @@ -726,7 +726,7 @@ static inline __alloc_size(1, 2) void *kmalloc_array_= noprof(size_t n, size_t siz > return kmalloc_noprof(bytes, flags); > return kmalloc_noprof(bytes, flags); > } > -#define kmalloc_array(...) alloc_hooks(kmalloc_array= _noprof(__VA_ARGS__)) > +#define kmalloc_array(...) alloc_hooks(kmalloc_array_noprof(= __VA_ARGS__)) ditto. > > /** > * krealloc_array - reallocate memory for an array. > @@ -761,8 +761,8 @@ void *__kmalloc_node_track_caller_noprof(DECL_BUCKET_= PARAMS(size, b), gfp_t flag > unsigned long caller) __alloc_si= ze(1); > #define kmalloc_node_track_caller_noprof(size, flags, node, caller) \ > __kmalloc_node_track_caller_noprof(PASS_BUCKET_PARAMS(size, NULL)= , flags, node, caller) > -#define kmalloc_node_track_caller(...) \ > - alloc_hooks(kmalloc_node_track_caller_noprof(__VA_ARGS__, _RET_IP= _)) > +#define kmalloc_node_track_caller(size, ...) \ > + alloc_sized_hooks(kmalloc_node_track_caller_noprof, size, __VA_AR= GS__, _RET_IP_) > > /* > * kmalloc_track_caller is a special version of kmalloc that records the > @@ -807,13 +807,13 @@ static inline __alloc_size(1) void *kzalloc_noprof(= size_t size, gfp_t flags) > { > return kmalloc_noprof(size, flags | __GFP_ZERO); > } > -#define kzalloc(...) alloc_hooks(kzalloc_nopro= f(__VA_ARGS__)) > +#define kzalloc(size, ...) alloc_sized_hooks(kzalloc= _noprof, size, __VA_ARGS__) > #define kzalloc_node(_size, _flags, _node) kmalloc_node(_size, (_fla= gs)|__GFP_ZERO, _node) > > void *__kvmalloc_node_noprof(DECL_BUCKET_PARAMS(size, b), gfp_t flags, i= nt node) __alloc_size(1); > #define kvmalloc_node_noprof(size, flags, node) \ > __kvmalloc_node_noprof(PASS_BUCKET_PARAMS(size, NULL), flags, nod= e) > -#define kvmalloc_node(...) alloc_hooks(kvmalloc_node= _noprof(__VA_ARGS__)) > +#define kvmalloc_node(size, ...) alloc_sized_hooks(kvmallo= c_node_noprof, size, __VA_ARGS__) > > #define kvmalloc(_size, _flags) kvmalloc_node(_si= ze, _flags, NUMA_NO_NODE) > #define kvmalloc_noprof(_size, _flags) kvmalloc_node_noprof(_siz= e, _flags, NUMA_NO_NODE) > diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c > index 81e5f9a70f22..6d2cb72bf269 100644 > --- a/lib/alloc_tag.c > +++ b/lib/alloc_tag.c > @@ -78,6 +78,14 @@ static void alloc_tag_to_text(struct seq_buf *out, str= uct codetag *ct) > > seq_buf_printf(out, "%12lli %8llu ", bytes, counter.calls); > codetag_to_text(out, ct); > +#ifdef CONFIG_SLAB_PER_SITE > + seq_buf_putc(out, ' '); > + seq_buf_printf(out, "size:%s(%zu) slab:%s", > + tag->meta.sized =3D=3D 0 ? "non-slab" : "non-slab" term sounds overly specific and we might extend this to some other allocations as well in the future. I would suggest "unknown" instead. > + tag->meta.sized =3D=3D SIZE_MAX ?= "dynamic" : "fixed", > + tag->meta.sized =3D=3D SIZE_MAX ? 0 : tag= ->meta.sized, > + tag->meta.cache ? "ready" : "unused"); I don't see "struct alloc_meta" having a "cache" member... Since you are changing the format of this file, you want to also bump up the file version inside print_allocinfo_header(). > +#endif > seq_buf_putc(out, ' '); > seq_buf_putc(out, '\n'); > } > diff --git a/mm/Kconfig b/mm/Kconfig > index b72e7d040f78..855c63c3270d 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -296,6 +296,14 @@ config SLAB_BUCKETS > > If unsure, say Y. > > +config SLAB_PER_SITE > + bool "Separate slab allocations by call size" > + depends on !SLUB_TINY > + default SLAB_FREELIST_HARDENED > + select SLAB_BUCKETS > + help > + Track sizes of kmalloc() call sites. > + > config SLUB_STATS > default n > bool "Enable performance statistics" > -- > 2.34.1 >