Re: [PATCH 1/4] x86_64: Switch to SPARSE_VIRTUAL

Martin J. Bligh
2007-04-02T15:58:38Z

cc: apw ... seeing as he wrote sparsemem in the first place, please copy
him on this stuff ?

Andi Kleen wrote:
> On Monday 02 April 2007 17:37, Christoph Lameter wrote:
>> On Sun, 1 Apr 2007, Andi Kleen wrote:
>>
>>> Hmm, this means there is at least 2MB worth of struct page on every node?
>>> Or do you have overlaps with other memory (I think you have)
>>> In that case you have to handle the overlap in change_page_attr()
>> Correct. 2MB worth of struct page is 128 mb of memory. Are there nodes
>> with smaller amounts of memory?
>
> Yes the discontigmem minimum is 64MB and there are some setups
> (mostly with numa emulation) where you end up with nodes that small.

We're actually using numa emulation to do real (container) things with.
However, 128MB is still pretty small for that ... and worst case, we
just waste 1MB for a 64MB node, right? Which isn't beautiful, but
doesn't seem like the end of the world for an obscure corner case.

>>> Do you have any benchmarks numbers to prove it? There seem to be a few
>>> benchmarks where the discontig virt_to_page is a problem
>>> (although I know ways to make it more efficient), and sparsemem
>>> is normally slower. Still some numbers would be good.
>> You want a benchmark to prove that the removal of memory references and
>> code improves performance?
>
> You're just moving them into MMU, not really removing it. And need more TLB entries.
> It might be faster or it might not. There are some unexpected issues, like most x86-64
> CPUs have a quite small number of large TLBs so you can get thrashing etc.
>
> So numbers with TLB intensive workloads would be good.

There's also the possibility it just doesn't make enough difference
to affect a real benchmark ...

M.