Re: Two shadow page tables for HVM

[Emre Can Sezer <ecsezer@ncsu.edu>]

I finally got around to implementing two paging modes.  Everything works 
fine until I swap modes :)

I get a shadow page fault with error_code 0.  This happens right after I 
swap the paging mode.  Any clues as to what might be the cause?

I walked through the code that updates paging modes.  It appears that we 
simply make an *empty* top level shadow and install it as top level 
shadow page table.  If this is the case, shouldn't the first fault have 
a non-zero error code?

Thanks,

John

Tim Deegan wrote:
> Hi,
>
> At 11:16 -0500 on 29 Dec (1230549392), Emre Can Sezer wrote:
>   
>> I've been trying to understand the shadow code for a while now and I have 
>> one last question about this approach.  In my case, the guest OS will have 
>> only a single set of page tables and in return I will have two sets of 
>> shadows for them.  I understand that once you change your shadow mode, the 
>> shadow pages are still kept and the mapping is stored in the shadow cache.  
>> However, if a page table is updated in one mode, how does the other mode 
>> know of this change? As far as I understand, the same gfn will be inserted 
>> to the hash twice with two types.  However, does Xen determine that the 
>> guest page's contents have changed? That change must somehow propagate to 
>> the second shadow mode's page tables with the appropriate permission 
>> changes. How is this being done? How does Xen determine that the page 
>> contents have changed?
>>     
>
> When making PTEs in the shadow tables, we never put in a writeable
> mapping of a page that has any shadows.  Then, in the pagefault handler,
> we spot that the guest is writing to a shadowed page and call the
> emulator so we can figure out what it's trying to write.  The emulator
> calls us back when it wants to write to memory, and in the callback we
> call the propagation code for _every_ kind of shadow the page has.
>
> We need to do that even within a single paging mode, because if the
> guest uses linear page tables a single page can be shadowed as l4, l3,
> l2 and l1 at the same time.
>
> The "out of sync" optimization relaxes the rule of never allowing a
> writeable mapping of a shadowed page, but it doesn't apply to pages that
> are shadowed more than once anyway.  Might be best to turn it off
> while you're working in this, anyway. :)
>
> Cheers,
>
> Tim.
>
>   
>>>> The reason I was thinking of synchronized page tables is because I will 
>>>> have to switch between them quite often - several times during a system 
>>>> call.  So I want to minimize the tlb flushes and make the switch as fast 
>>>> as possible.  With synced PT's, my plan was to set the guest CR3 to point 
>>>> to the new top level page table and only flush the kernel pages.
>>>>     
>>>>         
>>> That might be just as expensive -- ISTR Keir measured the cost of invlpg
>>> vs TLB flush a while ago and found that invlpg'ing more than one or two
>>> PTEs was slower than just flushing the whole TLB.
>>>
>>>   
>>>       
>>>> When considering the performance penalties of flushing the kernel page 
>>>> tables from the TLB, how significant is traversing all the shadow page 
>>>> tables for the guest kernel and updating their permissions?  If there 
>>>> isn't an order of magnitude of difference, it might be reasonable to take 
>>>> the short cut in implementation.
>>>>     
>>>>         
>>> I don't have any measurements for doing walks of the whole set of
>>> shadows, but in general we've found it's worth doing almost any trick
>>> that will avoid that.
>>>
>>> Cheers,
>>>
>>> Tim.
>>>
>>>   
>>>       
>>     
>
>