Pre-populate TDP table to avoid page faults at VM boot

Pre-populate TDP table to avoid page faults at VM boot

[Federico Parola]

Hi everyone,
is it possible to pre-populate the TDP table (EPT in my case) when 
configuring the VM environment, so that there won't be a page fault / VM 
exit every time the guest tries to access a RAM page for the first time?
At the moment I see a lot of page faults when the VM boots, is it 
possible to prevent them to reduce boot time?
(I'm using QEMU to run my VMs.)

Best regards,
Federico Parola

Re: Pre-populate TDP table to avoid page faults at VM boot

[Sean Christopherson]

On Wed, Jul 26, 2023, Federico Parola wrote:
> Hi everyone,
> is it possible to pre-populate the TDP table (EPT in my case) when
> configuring the VM environment, so that there won't be a page fault / VM
> exit every time the guest tries to access a RAM page for the first time?

No, not yet.

> At the moment I see a lot of page faults when the VM boots, is it possible
> to prevent them to reduce boot time?

You can't currently prevent the page faults, but you can _significantly_ reduce
them by backing guest memory with hugepages.  E.g. using 2MiB instead of 4KiB
pages reduces the number of faults by 512x, and 1GiB (HugeTLB only) instead of
2MiB by another 512x.

But the word yet...

KVM needs to add internal APIs to allow userspace to tell to KVM map a particular
GPA in order to support upcoming flavors of confidential VMs[1].  I could have
sworn that I requested that that API be exposed to userspace via a common ioctl(),
e.g. so that userspace can prefault all of guest memory if userspace is so inclined.
Ah, I only made that comment in passing[2].

I'll follow-up in the TDX series to "officially" float the idea of exposing the
helper as an ioctl().

[1] https://lkml.kernel.org/r/6a4c029af70d41b63bcee3d6a1f0c2377f6eb4bd.1690322424.git.isaku.yamahata%40intel.com
[2] https://lore.kernel.org/all/ZGuh1J6AOw5v2R1W@google.com

Re: Pre-populate TDP table to avoid page faults at VM boot

[Federico Parola]

On 7/26/23 10:31, Sean Christopherson wrote:
> On Wed, Jul 26, 2023, Federico Parola wrote:
>> Hi everyone,
>> is it possible to pre-populate the TDP table (EPT in my case) when
>> configuring the VM environment, so that there won't be a page fault / VM
>> exit every time the guest tries to access a RAM page for the first time?
> 
> No, not yet.
> 
>> At the moment I see a lot of page faults when the VM boots, is it possible
>> to prevent them to reduce boot time?
> 
> You can't currently prevent the page faults, but you can _significantly_ reduce
> them by backing guest memory with hugepages.  E.g. using 2MiB instead of 4KiB
> pages reduces the number of faults by 512x, and 1GiB (HugeTLB only) instead of
> 2MiB by another 512x.
> 
> But the word yet...
> 
> KVM needs to add internal APIs to allow userspace to tell to KVM map a particular
> GPA in order to support upcoming flavors of confidential VMs[1].  I could have
> sworn that I requested that that API be exposed to userspace via a common ioctl(),
> e.g. so that userspace can prefault all of guest memory if userspace is so inclined.
> Ah, I only made that comment in passing[2].
> 
> I'll follow-up in the TDX series to "officially" float the idea of exposing the
> helper as an ioctl().
> 
> [1] https://lkml.kernel.org/r/6a4c029af70d41b63bcee3d6a1f0c2377f6eb4bd.1690322424.git.isaku.yamahata%40intel.com
> [2] https://lore.kernel.org/all/ZGuh1J6AOw5v2R1W@google.com

Thank you very much for the prompt reply and useful tips. I'll keep an 
eye on the topic.

Best regards,
Federico Parola

Re: Pre-populate TDP table to avoid page faults at VM boot

[Sean Christopherson]

On Wed, Jul 26, 2023, Federico Parola wrote:
> On 7/26/23 10:31, Sean Christopherson wrote:
> > On Wed, Jul 26, 2023, Federico Parola wrote:
> > > Hi everyone,
> > > is it possible to pre-populate the TDP table (EPT in my case) when
> > > configuring the VM environment, so that there won't be a page fault / VM
> > > exit every time the guest tries to access a RAM page for the first time?
> > 
> > No, not yet.
> > 
> > > At the moment I see a lot of page faults when the VM boots, is it possible
> > > to prevent them to reduce boot time?
> > 
> > You can't currently prevent the page faults, but you can _significantly_ reduce
> > them by backing guest memory with hugepages.  E.g. using 2MiB instead of 4KiB
> > pages reduces the number of faults by 512x, and 1GiB (HugeTLB only) instead of
> > 2MiB by another 512x.
> > 
> > But the word yet...
> > 
> > KVM needs to add internal APIs to allow userspace to tell to KVM map a particular
> > GPA in order to support upcoming flavors of confidential VMs[1].  I could have
> > sworn that I requested that that API be exposed to userspace via a common ioctl(),
> > e.g. so that userspace can prefault all of guest memory if userspace is so inclined.
> > Ah, I only made that comment in passing[2].
> > 
> > I'll follow-up in the TDX series to "officially" float the idea of exposing the
> > helper as an ioctl().
> > 
> > [1] https://lkml.kernel.org/r/6a4c029af70d41b63bcee3d6a1f0c2377f6eb4bd.1690322424.git.isaku.yamahata%40intel.com
> > [2] https://lore.kernel.org/all/ZGuh1J6AOw5v2R1W@google.com
> 
> Thank you very much for the prompt reply and useful tips. I'll keep an eye
> on the topic.

FYI, I finally followed through on this.

https://lore.kernel.org/all/Zbrj5WKVgMsUFDtb@google.com