Re: [patch] SLQB slab allocator

Hugh Dickins
2009-02-06T12:33:53Z

On Fri, 6 Feb 2009, Pekka Enberg wrote:
> On Thu, 2009-02-05 at 19:04 +0000, Hugh Dickins wrote:
> > I then tried a patch I thought obviously better than yours: just mask
> > off __GFP_WAIT in that __GFP_NOWARN|__GFP_NORETRY preliminary call to
> > alloc_slab_page(): so we're not trying to infer anything about high-
> > order availability from the number of free order-0 pages, but actually
> > going to look for it and taking it if it's free, forgetting it if not.
> >
> > That didn't work well at all: almost as bad as the unmodified slub.c.
> > I decided that was due to __alloc_pages_internal()'s
> > wakeup_kswapd(zone, order): just expressing an interest in a high-
> > order page was enough to send it off trying to reclaim them, though
> > not directly. Hacked in a condition to suppress that in this case:
> > worked a lot better, but not nearly as well as yours. I supposed
> > that was somehow(?) due to the subsequent get_page_from_freelist()
> > calls with different watermarking: hacked in another __GFP flag to
> > break out to nopage just like the NUMA_BUILD GFP_THISNODE case does.
> > Much better, getting close, but still not as good as yours.
>
> Did you look at it with oprofile?

No, I didn't. I didn't say so, but again it was elapsed time that
I was focussing on, so I don't think oprofile would be relevant.
There are some differences in system time, of course, consistent
with your point; but they're generally an order of magnitude less,
so didn't excite my interest.

> One thing to keep in mind is that if
> there are 4K allocations going on, your approach will get double the
> overhead of page allocations (which can be substantial performance hit
> for slab).

Sure, and even the current allocate_slab() is inefficient in that
respect: I've followed it because I do for now have an interest in
the stats, but if stats are configured off then there's no point in
dividing it into two stages; and if they are really intended to be
ORDER_FALLBACK stats, then it shouldn't divide into two stages when
oo_order(s->oo) == oo_order(s->min). On the other hand, I find it
interesting to see how often the __GFP_NORETRY fails, even when
the order is the same each time (and usually 0).

Hugh