Tracking changes to writable page tables

Tracking changes to writable page tables

[Jacob Gorm Hansen]

hi,

It looks like xenlinux 2.6 now uses writable page tables by default (I 
can't find the switch to enable/disable them).

I need to track writes to page tables, and I was wondering if it is 
possible to get a page fault upon writes to read-only mapped page 
tables, as when writing to other read-only mappings?

Thanks,
Jacob

RE: Tracking changes to writable page tables

[Ian Pratt]

> It looks like xenlinux 2.6 now uses writable page tables by 
> default (I can't find the switch to enable/disable them).
> 
> I need to track writes to page tables, and I was wondering if 
> it is possible to get a page fault upon writes to read-only 
> mapped page tables, as when writing to other read-only mappings?

Hmm, I can see that the current model makes like difficult for you
(though self-migration is a pretty twisted thing to be trying to do
anyway :-)

I wander if we should steal one of the OS bits in the PTE to
differentiate writable pagetable mappings? You could then take a normal
RO fault, set the bit, and then retry to have Xen do pagetable the work.

This would arguably be cleaner, though we've been reluctant to steal OS
bits just incase there's some OS out there that uses them all.

Ian

Re: Tracking changes to writable page tables

[Jacob Gorm Hansen]

Ian Pratt wrote:
>>It looks like xenlinux 2.6 now uses writable page tables by 
>>default (I can't find the switch to enable/disable them).
>>
>>I need to track writes to page tables, and I was wondering if 
>>it is possible to get a page fault upon writes to read-only 
>>mapped page tables, as when writing to other read-only mappings?
> 
> 
> Hmm, I can see that the current model makes like difficult for you
> (though self-migration is a pretty twisted thing to be trying to do
> anyway :-)
> 

Hmm while we are discussing twisted things; would it be possible to 
implement writable page tables in the guest rather than in Xen?

The guest would then take the fault, unpin and decouple the page table 
from the page directory, change the protection, and do the reverse when 
the page table is next used.

I suppose that would be a (somewhat radical) solution to my problem.

Jacob

Re: Tracking changes to writable page tables

[Jacob Gorm Hansen]

Jacob Gorm Hansen wrote:
> Ian Pratt wrote:

>>
>>
>> Hmm, I can see that the current model makes like difficult for you
>> (though self-migration is a pretty twisted thing to be trying to do
>> anyway :-)
>>
> 
> Hmm while we are discussing twisted things; would it be possible to 
> implement writable page tables in the guest rather than in Xen?
> 

On a related note, is there any way of querying the pinning status of 
pages from Xen? I used to have two global bitmaps in the guest, one for 
l1 and one for l2, but I suppose I could just as well add a 
get-pinned-status(mfn) hypercall to Xen, if there is not one already?

Or I could add this info to struct page in Linux.

thanks,
Jacob

Re: Tracking changes to writable page tables

[Jacob Gorm Hansen]

Ian Pratt wrote:

> I wander if we should steal one of the OS bits in the PTE to
> differentiate writable pagetable mappings? You could then take a normal
> RO fault, set the bit, and then retry to have Xen do pagetable the work.
> 
> This would arguably be cleaner, though we've been reluctant to steal OS
> bits just incase there's some OS out there that uses them all.

Yes, that would also be a solution, though it is a little nasty to steal
the AVL bits.

Alternatively, I can just add the tracking to set_pte(), assuming that
all updates come through there, but that will probably slow down page
table updates a bit, even while not checkpointing the kernel.

Jacob

Re: Tracking changes to writable page tables

[Jacob Gorm Hansen]

Ian Pratt wrote:
>>It looks like xenlinux 2.6 now uses writable page tables by 
>>default (I can't find the switch to enable/disable them).
>>
>>I need to track writes to page tables, and I was wondering if 
>>it is possible to get a page fault upon writes to read-only 
>>mapped page tables, as when writing to other read-only mappings?
> 
> 
> Hmm, I can see that the current model makes like difficult for you
> (though self-migration is a pretty twisted thing to be trying to do
> anyway :-)
> 
> I wander if we should steal one of the OS bits in the PTE to
> differentiate writable pagetable mappings? You could then take a normal
> RO fault, set the bit, and then retry to have Xen do pagetable the work.
> 
> This would arguably be cleaner, though we've been reluctant to steal OS
> bits just incase there's some OS out there that uses them all.

Actually, it seems this could be done without using an AVL bit, as this
is really per-frame rather than per-VA information. If the pte-pin call
could add a 'propagate page fault' flag to the PTE frame, then I could
still get the first fault to the guest, and then remove the pinning bit
to prevent the rest of the faults.

Jacob

RE: Tracking changes to writable page tables

[Ian Pratt]

> Hmm while we are discussing twisted things; would it be 
> possible to implement writable page tables in the guest 
> rather than in Xen?

We thought about doing this, but the data that's needed for doing the
tracking is very different from what an OS normally keeps. It would make
the hypervisor API very messy, and significantly increase the difficulty
of porting new OSes.

Doing it in Xen hides most of the ugliness and reduces the number of
protection domain crossings as well.

Ian

RE: Tracking changes to writable page tables

[Ian Pratt]

 
> On a related note, is there any way of querying the pinning 
> status of pages from Xen? I used to have two global bitmaps 
> in the guest, one for
> l1 and one for l2, but I suppose I could just as well add a
> get-pinned-status(mfn) hypercall to Xen, if there is not one already?

See the the dom0_op GETPAGEFRAMEINFO2

Ian