Re: PAE issue (32-on-64 work)

[Joe Bonasera <joe.bonasera@sun.com>]

Keir Fraser wrote:

> On 19/10/06 17:19, "Joe Bonasera" <joe.bonasera@sun.com> wrote:
> 
> 
>>The code path in Solaris currently supports 2 possibilities for PAE top level
>>tables. The normal code we use on bare metal allocates only 1 page
>>that all cpu's share for the the top level page table. For
>>example, cpu0 uses the 1st four quads for its current process'
>>L3, cpu1 uses the next four, etc. On context switch or cr3 reload
>>we (re)copy in the 4 entries of the process for that CPU's section
>>of the page.
>>
>>That code path is, as so much of the 32 bit PAE support, a special
>>case. So it was easily turned off and made to just use
>>an entire page for each unique top level L3 on Xen.  I did that just for
>>my initial bring up on PAE Xen, but was hoping to go back to some
>>form of the optimized version next.
> 
> 
> You should just allocate a page-sized L3 per process and be done with it. A
> page overhead per process is nothing to be concerned about (clearly the
> overhead can be even bigger if, for example, you run 4-level tables on
> x86_64). Recopying the L3 entries every TLB flush will *not* perform well on
> current Xen.
> 

Well we actually don't do complete TLB flushes very often at all, essentially
only the first time a new L3 entry is created by a running process which
for most processes means never - as >1Gig processes are rare.
So it shouldn't matter if they hit one or two slowish flushes.

Are there any other performance implications to watch out for?

Joe