Dirty page tracking in EPT

Dirty page tracking in EPT

[Prateek Sharma]

Hello ,
    I came across the dirty-page tracking patch here:
[https://lkml.org/lkml/2011/6/21/169] .
There is some mention of dirty-bit tracking not working with EPT. Can
someone please clarify this? Does this patch only work when using
software shadow page tables ?

   On a related note (i've asked a related question on this list a
couple of days back), why can we not use the
kvm_vm_ioctl_get_dirty_log function
[http://lxr.linux.no/#linux+v3.0/arch/x86/kvm/x86.c#L3297] ?  Is it
because it is too expensive? If yes, can we use the dirty-page
tracking ideas in this patch and use it for the migration code?


Thanks for reading,
Prateek

Re: Dirty page tracking in EPT

[Nai Xia]

Hi, 

On Sunday 02 October 2011 17:43:23 Prateek Sharma wrote:
> Hello ,
>     I came across the dirty-page tracking patch here:
> [https://lkml.org/lkml/2011/6/21/169] .
> There is some mention of dirty-bit tracking not working with EPT. Can
> someone please clarify this? Does this patch only work when using
> software shadow page tables ?

When ksm is scanning the memory, it is working on the host virutal
addresses, so this patch tries to figure out if the page pointed to by
the pte is actually dirty in the guests through the scanning of sptes.
But, Intel EPT does not update the dirty bit (and other similar bits)
in sptes, so for Intel EPT we fall back to checksum based approach
in the patch.

This patch should work on AMD RVI/NPT although I hadn't the hardware
to test it yet. And also, my v2 patch considers huge page as a whole, and
try to skip it if any of the subpage is dirty, and I am still trying to 
quantify the cost balance between breaking up active huge pages and 
potential swapping. So maybe sometime I will give out a v3 patch totally
disabling the huge page consideration. 

> 
>    On a related note (i've asked a related question on this list a
> couple of days back), why can we not use the
> kvm_vm_ioctl_get_dirty_log function

While I am not an expert in kvm, I don't think kvm_vm_ioctl_get_dirty_log()
or kvm_memory_slot->dirty_bitmap can catch the normal writes issued from inside
guest OS. Maybe some experts in this list can help you understand the code ;-)

Thanks,

Nai


> [http://lxr.linux.no/#linux+v3.0/arch/x86/kvm/x86.c#L3297] ?  Is it
> because it is too expensive? If yes, can we use the dirty-page
> tracking ideas in this patch and use it for the migration code?
> 
> 
> Thanks for reading,
> Prateek
> 

Re: Dirty page tracking in EPT

[Prateek Sharma]

On Mon, Oct 3, 2011 at 9:04 AM, Nai Xia <nai.xia@gmail.com> wrote:
> Hi,
>
> On Sunday 02 October 2011 17:43:23 Prateek Sharma wrote:
>> Hello ,
>>     I came across the dirty-page tracking patch here:
>> [https://lkml.org/lkml/2011/6/21/169] .
>> There is some mention of dirty-bit tracking not working with EPT. Can
>> someone please clarify this? Does this patch only work when using
>> software shadow page tables ?
>
> When ksm is scanning the memory, it is working on the host virutal
> addresses, so this patch tries to figure out if the page pointed to by
> the pte is actually dirty in the guests through the scanning of sptes.
> But, Intel EPT does not update the dirty bit (and other similar bits)
> in sptes, so for Intel EPT we fall back to checksum based approach
> in the patch.
>
> This patch should work on AMD RVI/NPT although I hadn't the hardware
> to test it yet. And also, my v2 patch considers huge page as a whole, and
> try to skip it if any of the subpage is dirty, and I am still trying to
> quantify the cost balance between breaking up active huge pages and
> potential swapping. So maybe sometime I will give out a v3 patch totally
> disabling the huge page consideration.
>
>>
>>    On a related note (i've asked a related question on this list a
>> couple of days back), why can we not use the
>> kvm_vm_ioctl_get_dirty_log function
>
> While I am not an expert in kvm, I don't think kvm_vm_ioctl_get_dirty_log()
> or kvm_memory_slot->dirty_bitmap can catch the normal writes issued from inside
> guest OS. Maybe some experts in this list can help you understand the code ;-)
>

Thanks for responding!
I always wondered why KSM doesnt use the dirty page info and had
started making some changes until i came across your patch :) .
Without the dirty-page logging KSM just cannot scale - scanning 100s
of GB memory can take hours!


I am a bit confused about how live migration can proceed if qemu
doesnt have accurate guest dirty info, file-backed page or not. AFAIK
Xen toggles the read-only bits in its shadow tables during the
migration to get a dirty-bitmap. Does KVM do the same?

My question mainly is: how does KVM acquire the dirty-page info for
migration when using EPT and the like (ie no shadow MMU).

Thanks!

> Thanks,
>
> Nai
>
>
>> [http://lxr.linux.no/#linux+v3.0/arch/x86/kvm/x86.c#L3297] ?  Is it
>> because it is too expensive? If yes, can we use the dirty-page
>> tracking ideas in this patch and use it for the migration code?
>>
>>
>> Thanks for reading,
>> Prateek
>>
>