Re: 2MB page PV guest support clarification

[Mick Jordan <Mick.Jordan@sun.com>]

On 02/27/09 15:28, Jeremy Fitzhardinge wrote:
> Mick Jordan wrote:
>> I was inspired by the talk from Ben Serebrin at this weeks' summit to 
>> investigate using 2MB pages in my Java VM on Xen for x86-64.
>>
>> I think I have done everything correctly, but Xen (3.1.4) rejects my 
>> attempt to set the PSE bit in the L2 frame for the 2MB page. Looking 
>> at the Xen code the L2_DISALLOW_MASK (0xFF800180U) simply rejects the 
>> update if the PSE bit is set.
>
> Yes.  Xen doesn't support large mappings for PV guests.  However, 
> there's a lot less to worry about for PV guests compared to hvm 
> guests. A PV guest directly uses the CPU's pagetable+tlb hardware, and 
> so a tlb miss results in a single simple walk of the pagetable, and 
> the overall tlb pressure is a lot less.  The desire to use large pages 
> for hvm guests is driven by the cost of a tlb miss when you have 4k 
> guest pages layered on 4k host pages, resulting in 24 memory accesses 
> in the worst case; a PV tlb miss is no more expensive than a native 
> tlb miss by comparison.
>
> Large pages could potentially reduce the cost of a PV tlb miss as 
> well, but also pose quite a few tradeoffs.  You can't generally use 
> large mappings for the kernel, as you can native, because of all the 
> pages which need RO mappings (pagetables, gdt, etc).  Also, IO and the 
> balloon driver operate at 4k page resolution, so breaking a contiguous 
> 2M page would require the mapping to be shattered.
Well that's disappointing! The Java heap is a perfect candidate for 
large pages and, since the heap tends to be large, would result in a  
TLB size reduction of a factor of 512, thereby reducing the misses. I 
have the luxury of a lot more semantics on memory usage than a typical 
OS, so would only use large pages where it makes sense (heap and runtime 
compiled code). I have my own equivalent of the balloon driver.

Mick