Re: [PATCH] slub: Don't throw away partial remote slabs if there is no local memory

David Rientjes
2014-01-25T00:26:08Z

On Fri, 24 Jan 2014, Nishanth Aravamudan wrote:

> Thank you for clarifying and providing a test patch. I ran with this on
> the system showing the original problem, configured to have 15GB of
> memory.
>
> With your patch after boot:
>
> MemTotal: 15604736 kB
> MemFree: 8768192 kB
> Slab: 3882560 kB
> SReclaimable: 105408 kB
> SUnreclaim: 3777152 kB
>
> With Anton's patch after boot:
>
> MemTotal: 15604736 kB
> MemFree: 11195008 kB
> Slab: 1427968 kB
> SReclaimable: 109184 kB
> SUnreclaim: 1318784 kB
>
>
> I know that's fairly unscientific, but the numbers are reproducible.
>

I don't think the goal of the discussion is to reduce the amount of slab
allocated, but rather get the most local slab memory possible by use of
kmalloc_node(). When a memoryless node is being passed to kmalloc_node(),
which is probably cpu_to_node() for a cpu bound to a node without memory,
my patch is allocating it on the most local node; Anton's patch is
allocating it on whatever happened to be the cpu slab.

> > diff --git a/mm/slub.c b/mm/slub.c
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -2278,10 +2278,14 @@ redo:
> >
> > if (unlikely(!node_match(page, node))) {
> > stat(s, ALLOC_NODE_MISMATCH);
> > - deactivate_slab(s, page, c->freelist);
> > - c->page = NULL;
> > - c->freelist = NULL;
> > - goto new_slab;
> > + if (unlikely(!node_present_pages(node)))
> > + node = numa_mem_id();
> > + if (!node_match(page, node)) {
> > + deactivate_slab(s, page, c->freelist);
> > + c->page = NULL;
> > + c->freelist = NULL;
> > + goto new_slab;
> > + }
>
> Semantically, and please correct me if I'm wrong, this patch is saying
> if we have a memoryless node, we expect the page's locality to be that
> of numa_mem_id(), and we still deactivate the slab if that isn't true.
> Just wanting to make sure I understand the intent.
>

Yeah, the default policy should be to fallback to local memory if the node
passed is memoryless.

> What I find odd is that there are only 2 nodes on this system, node 0
> (empty) and node 1. So won't numa_mem_id() always be 1? And every page
> should be coming from node 1 (thus node_match() should always be true?)
>

The nice thing about slub is its debugging ability, what is
/sys/kernel/slab/cache/objects showing in comparison between the two
patches?