Re: Memory allocation profiling warnings in memory bound systems

[Johannes Weiner <hannes@cmpxchg.org>]

On Mon, May 19, 2025 at 08:50:28AM -0700, Suren Baghdasaryan wrote:
> On Mon, May 19, 2025 at 6:33 AM Usama Arif <usamaarif642@gmail.com> wrote:
> >
> >
> > +cc Vlad
> >
> > On 19/05/2025 14:31, Usama Arif wrote:
> > > Hi,
> > >
> > > We have started enabling memory allocation profiling (with kernel 6.13) in our fleet
> > > and are seeing a large number of warnings (reported by Vlad Poenaru) due to failure
> > > in allocation of slab object extensions on services that are memory bound. I have attached
> > > one of the logs at the end.
> > >
> > > Does it make sense to change the slabobj_ext to be allocated via kvcalloc and also change
> > > the WARN to WARN_ONCE (or maybe even pr_debug?) like the diff below? A large number of
> > > prints for this in a short time may mask any real issues in the system during memory
> > > pressure being reported in dmesg. I tried to see if there were any changes after 6.13
> > > to this code but didn't find any, but thought will check before sending below as a patch.
> > >
> > > diff --git a/mm/slub.c b/mm/slub.c
> > > index c2151c9fee22..4595ca190cd9 100644
> > > --- a/mm/slub.c
> > > +++ b/mm/slub.c
> > > @@ -1961,7 +1961,7 @@ int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s,
> > >         gfp &= ~OBJCGS_CLEAR_MASK;
> > >         /* Prevent recursive extension vector allocation */
> > >         gfp |= __GFP_NO_OBJ_EXT;
> > > -       vec = kcalloc_node(objects, sizeof(struct slabobj_ext), gfp,
> > > +       vec = kvcalloc_node(objects, sizeof(struct slabobj_ext), gfp,
> 
> Hi Usama,
> Is the allocation larger than page size? IIUC, unless allocation size
> is over PAGE_SIZE, kvcalloc_node() will not fall back to vmalloc (see:
> https://elixir.bootlin.com/linux/v6.14.7/source/mm/util.c#L668). How
> big is the allocation when it fails in your case?

Digging through the reports, it appears we're encountering both. We've
seen a zswap slab where the slab is order-0 and slabext is
higher-order (8 byte objects, 512 objsperslab, 1 pageperslab), but
also biovec-max where it's the other way round (4k byte objects, 8
objsperslab, 8 pagesperslab).

In the first case, vmalloc would help. In the second it wouldn't.

The second case is interesting. The higher-order slab succeeds because
bios use a mempool; but the system is so depleted that the order-0 for
the slabext fails.

I'm not sure there is much we can do about this tbh. It would seem
overkill to add a mempool or grant the tracking access to system-wide
emergency reserves.

A warn-once would probably make sense nonetheless.

It might also make sense to flag the line item for that callsite in
the reporting file, to make it obvious that the counter is compromised
and is missing allocations?