Migration for SPP(Sub-Page Protection) bitmaps

Migration for SPP(Sub-Page Protection) bitmaps

[Yang Weijiang]

Hi, Paolo,

I'm working on migration support for SPP(Sub-Page Protection) feature,
the previous patch set is here:
https://lkml.org/lkml/2018/11/30/574, basically it's used to protect
128-byte subpages within a normal 4KB page.

but I'm not sure whether we need to add migration support for such kind of feature. 

Currently, my solution is to store the SPP bitmaps to destination KVM, when
there's EPT violation, search the bitmap mapping table to check if the
guest page needs to be protected.

Another solution is to setup EPT table directly at the end of migration.

Do you have perferred approach? 
Thanks for your comments!

Re: Migration for SPP(Sub-Page Protection) bitmaps

[Paolo Bonzini]

On 17/04/19 11:59, Yang Weijiang wrote:
> Hi, Paolo,
> 
> I'm working on migration support for SPP(Sub-Page Protection) feature,
> the previous patch set is here:
> https://lkml.org/lkml/2018/11/30/574, basically it's used to protect
> 128-byte subpages within a normal 4KB page.
> 
> but I'm not sure whether we need to add migration support for such kind of feature. 
> 
> Currently, my solution is to store the SPP bitmaps to destination KVM, when
> there's EPT violation, search the bitmap mapping table to check if the
> guest page needs to be protected.
> 
> Another solution is to setup EPT table directly at the end of migration.

The destination userspace should simply call KVM_SUBPAGES_SET_ACCESS,
and KVM will process it just like it would on the source.  In other
words, it shouldn't matter if the ioctl is done as part of migration or
the first time a VM is setup.  And in both cases, lazy setup should be fine.

Thanks,

Paolo

Re: Migration for SPP(Sub-Page Protection) bitmaps

[Yang Weijiang]

On Fri, Apr 26, 2019 at 10:06:17AM +0200, Paolo Bonzini wrote:
> On 17/04/19 11:59, Yang Weijiang wrote:
> > Hi, Paolo,
> > 
> > I'm working on migration support for SPP(Sub-Page Protection) feature,
> > the previous patch set is here:
> > https://lkml.org/lkml/2018/11/30/574, basically it's used to protect
> > 128-byte subpages within a normal 4KB page.
> > 
> > but I'm not sure whether we need to add migration support for such kind of feature. 
> > 
> > Currently, my solution is to store the SPP bitmaps to destination KVM, when
> > there's EPT violation, search the bitmap mapping table to check if the
> > guest page needs to be protected.
> > 
> > Another solution is to setup EPT table directly at the end of migration.
> 
> The destination userspace should simply call KVM_SUBPAGES_SET_ACCESS,
> and KVM will process it just like it would on the source.  In other
> words, it shouldn't matter if the ioctl is done as part of migration or
> the first time a VM is setup.  And in both cases, lazy setup should be fine.
> 
> Thanks,
> 
> Paolo
Thanks Paolo! 

The problem is in current patch (as in link), it assumes
the target pages are there in EPT, but in migration case, these pages
are not setup in ETP, so KVM_SUBPAGES_SET_ACCESS during migration or
destination bootup will fail, should I modify the patch to setup the 
ETP in lazy mode? 

Re: Migration for SPP(Sub-Page Protection) bitmaps

[Paolo Bonzini]

> > The destination userspace should simply call KVM_SUBPAGES_SET_ACCESS,
> > and KVM will process it just like it would on the source.  In other
> > words, it shouldn't matter if the ioctl is done as part of migration or
> > the first time a VM is setup.  And in both cases, lazy setup should be
> > fine.
> 
> The problem is in current patch (as in link), it assumes
> the target pages are there in EPT, but in migration case, these pages
> are not setup in ETP, so KVM_SUBPAGES_SET_ACCESS during migration or
> destination bootup will fail, should I modify the patch to setup the
> ETP in lazy mode?

Yes, definitely.  Remember that EPT page tables come and go (pages could
be swapped out, or moved from one host physical address to another; there
is also a shrinker that will release EPT page tables if memory is low),
so SPP support should be ready for that.  Migration is just a special case.

Paolo

Re: Migration for SPP(Sub-Page Protection) bitmaps

[Yang Weijiang]

On Sat, Apr 27, 2019 at 03:37:39AM -0400, Paolo Bonzini wrote:
> 
> > > The destination userspace should simply call KVM_SUBPAGES_SET_ACCESS,
> > > and KVM will process it just like it would on the source.  In other
> > > words, it shouldn't matter if the ioctl is done as part of migration or
> > > the first time a VM is setup.  And in both cases, lazy setup should be
> > > fine.
> > 
> > The problem is in current patch (as in link), it assumes
> > the target pages are there in EPT, but in migration case, these pages
> > are not setup in ETP, so KVM_SUBPAGES_SET_ACCESS during migration or
> > destination bootup will fail, should I modify the patch to setup the
> > ETP in lazy mode?
> 
> Yes, definitely.  Remember that EPT page tables come and go (pages could
> be swapped out, or moved from one host physical address to another; there
> is also a shrinker that will release EPT page tables if memory is low),
> so SPP support should be ready for that.  Migration is just a special case.
> 
> Paolo
 OK, got it, thanks a lot for the reply!