Re: swapping with MMU Notifiers V2

Andrea Arcangeli
2008-01-30T11:42:06Z

On Wed, Jan 30, 2008 at 12:26:39PM +0100, Carsten Otte wrote:
> Andrea Arcangeli wrote:
>> By your conclusion I suppose you thought NPT maps guest physical to
>> host virtual. If it was the case the cpu would to walk three layer of
>> pagetables (each layer is an arrow): guest virtual -> guest physical
>> -> host virtual -> host physical. Instead it's just guest virtual ->
>> guest physical -> host physical. Or even less for shadow: guest
>> virtual -> host physical, which is why shadow should be faster for
>> number crunching still (and definitely slower for dbms).
> No, I was assuming guest virtual -> guest physical -> host physical. We do
> the same. We just re-use the very same page table that maps gp to hp to map
> host userland to hp. The only tricky part for AMD64 is to hide host

Oh I see! So when linux pte is cleared, the NPT equivalent is
implicitly and atomically cleared too. That really requires _identical_
semantics and formats for both pagetables.

> userland parts from the guest that are not supposed to be visible to the
> guest-os as far as I see.

That problem is quite easily fixable by only sharing a 3rd level
pagetable hand having the NPT-pgd filled with only that shared 3rd
level linux pagetable. The guest ram can be allocated with MAP_FIXED
at a proper 512G aligned address. That has the cons to reduce the max
VM (_virtual_ adding more swap won't help) size to 512G. For the 64bit
kernel that may not be really a problem, for 32bit it's likely a
blocker instead. What worries me more are the details of the pte and
tlbs, I don't know the NPT format well enough yet to comment, but I
think when Avi just said we can't flush both tlbs in a single op
through tlb tagging like s390, will still require mmu notifiers. Still
if we can share the ptes it can save quite some memory for large VM,
so even if the mmu notifiers only have to deal with tlb flush + unpin
(without having to clear the NPT themself, anymore, because it was
already cleared by ptep_clear_flush) that may be still interesting to
consider. Which pdf were you reading?

Thanks!