Question about physical page allocation to the guest

Question about physical page allocation to the guest

[Steven]

Hi, Eric,
I am trying to understand how KVM allocates physical pages to the
guest and your slides clarify a lot of questions.
(https://events.linuxfoundation.org/slides/2011/linuxcon-japan/lcj2011_guangrong.pdf)

However, I still have some difficulty in figuring out what happens in
the kvm code.

The host kernel is 3.2.14 and EPT is disabled.

In the guest VM, I run a micro-benchmark program that touches 1024
pages (as an integer array). So in the guest VM, I can trace 1024
mm_page_alloc event. However, in the hypervisor I can only trace about
45 (sometimes < 45) kvm_page_fault events, which means that most page
faults in the guest are not exposed to the hypervisor. My questions
are

(1) why such kind of page faults are not exposed to the hypervisor as
EPT is disabled? (My doubt is that it is related to non-presetn PTE as
you discussed in the slides. But could you give some more details?)

(2) In such case, when the physical page are allocated to the VM to
backup the virtual pages? Could you give some hint about which piece
of KVM code calling the get get_user_pages()?

Thanks in advance.

- Steven

Re: Question about physical page allocation to the guest

[Xiao Guangrong]

On 10/20/14 2:09 AM, Steven wrote:
> Hi, Eric,
> I am trying to understand how KVM allocates physical pages to the
> guest and your slides clarify a lot of questions.
> (https://events.linuxfoundation.org/slides/2011/linuxcon-japan/lcj2011_guangrong.pdf)
>
> However, I still have some difficulty in figuring out what happens in
> the kvm code.
>
> The host kernel is 3.2.14 and EPT is disabled.
>
> In the guest VM, I run a micro-benchmark program that touches 1024
> pages (as an integer array). So in the guest VM, I can trace 1024
> mm_page_alloc event. However, in the hypervisor I can only trace about
> 45 (sometimes < 45) kvm_page_fault events, which means that most page
> faults in the guest are not exposed to the hypervisor. My questions
> are
>
> (1) why such kind of page faults are not exposed to the hypervisor as
> EPT is disabled? (My doubt is that it is related to non-presetn PTE as
> you discussed in the slides. But could you give some more details?)

Two cases, the one is the page you accessed have already mapped into the 
guest (the Present bit in the SPTE (shadow page table entry) is set). 
Another is that we can do page prefetch in KVM that can fill
more nonpresent sptes in one vm-exit.

>
> (2) In such case, when the physical page are allocated to the VM to
> backup the virtual pages? Could you give some hint about which piece
> of KVM code calling the get get_user_pages()?

In the case if the hva of the gpa you are accessing is not mapped in
the hypervisor's address space. Please follow the code of try_async_pf

>
> Thanks in advance.

Enjoy it. :)

Re: Question about physical page allocation to the guest

[Wanpeng Li]

Hi Xiao,
On Thu, Oct 23, 2014 at 12:39:56AM +0800, Xiao Guangrong wrote:

I'm not sure if jump into this thread is good. ;-)

One question about mmio spte.

set_spte
 -> set_mmio_spte
 	 -> if (unlikely(is_noslot_pfn(pfn))) mark_mmio_spte();

If mmio doesn't have memslot?

Regards,
Wanpeng Li 

Re: Question about physical page allocation to the guest

[Steven]

On Wed, Oct 22, 2014 at 12:39 PM, Xiao Guangrong
<xiaoguangrong.eric@gmail.com> wrote:
>
>
> On 10/20/14 2:09 AM, Steven wrote:
>>
>> Hi, Eric,
>> I am trying to understand how KVM allocates physical pages to the
>> guest and your slides clarify a lot of questions.
>>
>> (https://events.linuxfoundation.org/slides/2011/linuxcon-japan/lcj2011_guangrong.pdf)
>>
>> However, I still have some difficulty in figuring out what happens in
>> the kvm code.
>>
>> The host kernel is 3.2.14 and EPT is disabled.
>>
>> In the guest VM, I run a micro-benchmark program that touches 1024
>> pages (as an integer array). So in the guest VM, I can trace 1024
>> mm_page_alloc event. However, in the hypervisor I can only trace about
>> 45 (sometimes < 45) kvm_page_fault events, which means that most page
>> faults in the guest are not exposed to the hypervisor. My questions
>> are
>>
>> (1) why such kind of page faults are not exposed to the hypervisor as
>> EPT is disabled? (My doubt is that it is related to non-presetn PTE as
>> you discussed in the slides. But could you give some more details?)
>
>
> Two cases, the one is the page you accessed have already mapped into the
> guest (the Present bit in the SPTE (shadow page table entry) is set).
> Another is that we can do page prefetch in KVM that can fill
> more nonpresent sptes in one vm-exit.

Hi, Eric,
Thanks for these insightful comments. I have done some traces and
found that there is very few vm_try_async_get_page event (as called in
the try_async_pf). So I think the major reason of few page allocation
I observed is due to the page to be accessed that have already mapped
into the
guest (the Present bit in the SPTE (shadow page table entry) is set).
My questions are

(1) Is this related to unsync shadow pages as in page 15 of your slides?
https://events.linuxfoundation.org/slides/2011/linuxcon-japan/lcj2011_guangrong.pdf

(2) If this is true, then there will be no vmexit to the hypervisor on
the page fault in guest user space. When will the shadow page table be
synched with the guest page table? Thanks.


>
>>
>> (2) In such case, when the physical page are allocated to the VM to
>> backup the virtual pages? Could you give some hint about which piece
>> of KVM code calling the get get_user_pages()?
>
>
> In the case if the hva of the gpa you are accessing is not mapped in
> the hypervisor's address space. Please follow the code of try_async_pf
>
>>
>> Thanks in advance.
>
>
> Enjoy it. :)
>