Re: alloc_heap_pages is low efficient with more CPUs

[Keir Fraser <keir@xen.org>]

On 16/10/2012 09:28, "Jan Beulich" <JBeulich@suse.com> wrote:

>> It's just the small factors multiplying up. A 40G domain is 10M page
>> allocations, each of which does 64x per-cpu cpumask bitops and timestamp
>> compares. That's going on for a billion (10^9) times round
>> tlbflush_filter()s loop. Each iteration need only take a few CPU cycles for
>> the effect to actually be noticeable. If the stuff being touched were not in
>> the cache (which of course it is, since this path is so hot and not touching
>> much memory) it would probably take an hour to create the domain!
> 
> Which means that your TODO remark in the change description
> is more than a nice-to-have, since
> - in the fallback case using 4k allocations your change doesn't
>   win anything
> - even larger domains would still suffer from this very problem
>   (albeit in 4.2 we try 1G allocations first, so your change should
>   have an even more visible effect there, as long as falling back
>   to smaller chunks isn't needed)

Agreed. This is a simple starting point which is also easy to backport
though, and solves the immediate observed problem.

> One question of course is whether, for sufficiently large allocation
> requests on sufficiently large systems, trying to avoid the TLB
> flush is worth it at all, considering that determining where to skip
> the flushes is costing so much more than the flushes themselves.

It might be a simpler option. I'll see how doing it the filtering way looks.

 -- Keir