Re: [PATCH 3/5] mm: slub allocation fairness

Christoph Lameter
2007-05-14T16:35:34Z

On Mon, 14 May 2007, Peter Zijlstra wrote:

> On Mon, 2007-05-14 at 08:49 -0700, Christoph Lameter wrote:
> > On Mon, 14 May 2007, Peter Zijlstra wrote:
> >
> > > Index: linux-2.6-git/include/linux/slub_def.h
> > > ===================================================================
> > > --- linux-2.6-git.orig/include/linux/slub_def.h
> > > +++ linux-2.6-git/include/linux/slub_def.h
> > > @@ -52,6 +52,7 @@ struct kmem_cache {
> > > struct kmem_cache_node *node[MAX_NUMNODES];
> > > #endif
> > > struct page *cpu_slab[NR_CPUS];
> > > + int rank;
> > > };
> >
> > Ranks as part of the kmem_cache structure? I thought this is a temporary
> > thing?
>
> No it needs to store the current state to verity subsequent allocations
> their gfp flags against.

What state? This is a global state? The kmem_cache struct is rarely
written to after setting up the slab. Any writes could create a serious
performance problem on large scale systems.

> > > * Lock order:
> > > @@ -961,6 +962,8 @@ static struct page *allocate_slab(struct
> > > if (!page)
> > > return NULL;
> > >
> > > + s->rank = page->index;
> > > +
> >
> > Argh.... Setting a cache structure field from a page struct field? What
> > about concurrency?
>
> Oh, right; allocate_slab is not serialized itself.

Nor should you ever write to the kmem_cache structure concurrently at all.

> > >
> > > - else {
> > > + } else {
> > > object = page->lockless_freelist;
> > > page->lockless_freelist = object[page->offset];
> > > }
> >
> > This is the hot path. No modifications please.
>
> Yes it is, but sorry, I have to. I really need to validate each slab
> alloc its GFP flags. Thats what the whole thing is about, I thought you
> understood that.

You are accessing a kmem_cache structure field in the hot path. That
cacheline is never used in the hot path. Sorry this is way to intrusive
for the problem you are trying to solve.