Re: Performance regression in scsi sequential throughput (iozone) due to "e084b - page-allocator: preserve PFN ordering when __GFP_COLD is set"

[Nick Piggin <npiggin@suse.de>]

On Tue, Mar 02, 2010 at 10:04:02AM +0000, Mel Gorman wrote:
> On Tue, Mar 02, 2010 at 05:52:25PM +1100, Nick Piggin wrote:
> > The zone pressure waitqueue patch makes sense.
> 
> I've just started the rebase and considering what sort of test is best
> for it.
> 
> > We may even want to make
> > it more strictly FIFO (eg. check upfront if there are waiters on the
> > queue before allocating a page, and if yes then add ourself to the back
> > of the waitqueue).
> 
> To be really strict about this, we'd have to check in the hot-path of the
> per-cpu allocator which would be undesirable.

Yes, but it would also be desirable for other reasons (eliminating
all unnecessary latency here). Which is why I say it is a tradeoff
but it might be worthwhile.

We obviously wouldn't load the waitqueue itself in the fastpath, but
probably some field in a cacheline we already touch.


> We could check further in the
> slow-path but I bet it'd be very rare that the logic would be triggered. For
> a process to enter the FIFO due to waiters that were not yet woken up, the
> system would have to be a) under heavy memory pressure b) reclaim taking such
> a long time that check_zone_pressure() is not being called in time and c)
> a process exiting or otherwise freeing memory such that the watermarks are
> cleared without reclaim being involved.

I don't think it would be too rare. Things can get freed up and
other allocations come in while reclaim is happening. But anyway
the nasty thing about the "rare" events is that they do add a
rare source of unexpected latency or starvation.

I'm not saying necessarily that it will make a noticable improvement
to throughput for this test case, but that now that we have the
waitqueue there, we should think about exactly how far we want to
go with it.