* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation
[not found] <1587104999-28927-1-git-send-email-decui@microsoft.com>
@ 2020-04-17 10:03 ` Vitaly Kuznetsov
2020-04-17 10:55 ` Wei Liu
0 siblings, 1 reply; 5+ messages in thread
From: Vitaly Kuznetsov @ 2020-04-17 10:03 UTC (permalink / raw)
To: Dexuan Cui
Cc: bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel, mingo,
sthemmin, tglx, x86, mikelley, wei.liu, kvm
Dexuan Cui <decui@microsoft.com> writes:
> Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the
> resume path, the "new" kernel's VP assist page is not suspended (i.e.
> disabled), and later when we jump to the "old" kernel, the page is not
> properly re-enabled for CPU0 with the allocated page from the old kernel.
>
> So far, the VP assist page is only used by hv_apic_eoi_write().
No, not only for that ('git grep hv_get_vp_assist_page')
KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In
particular, Enlightened VMPTR is written there.
This makes me wonder: how does hibernation work with KVM in case we use
Enlightened VMCS and we have VMs running? We need to make sure VP Assist
page content is preserved.
--
Vitaly
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation
2020-04-17 10:03 ` [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation Vitaly Kuznetsov
@ 2020-04-17 10:55 ` Wei Liu
2020-04-17 12:03 ` Vitaly Kuznetsov
0 siblings, 1 reply; 5+ messages in thread
From: Wei Liu @ 2020-04-17 10:55 UTC (permalink / raw)
To: Vitaly Kuznetsov
Cc: Dexuan Cui, bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel,
mingo, sthemmin, tglx, x86, mikelley, wei.liu, kvm
On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote:
> Dexuan Cui <decui@microsoft.com> writes:
>
> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the
> > resume path, the "new" kernel's VP assist page is not suspended (i.e.
> > disabled), and later when we jump to the "old" kernel, the page is not
> > properly re-enabled for CPU0 with the allocated page from the old kernel.
> >
> > So far, the VP assist page is only used by hv_apic_eoi_write().
>
> No, not only for that ('git grep hv_get_vp_assist_page')
>
> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In
> particular, Enlightened VMPTR is written there.
>
> This makes me wonder: how does hibernation work with KVM in case we use
> Enlightened VMCS and we have VMs running? We need to make sure VP Assist
> page content is preserved.
The page itself is preserved, isn't it?
hv_cpu_die never frees the vp_assit page. It merely disables it.
hv_cpu_init only allocates a new page if necessary.
Wei.
>
> --
> Vitaly
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation
2020-04-17 10:55 ` Wei Liu
@ 2020-04-17 12:03 ` Vitaly Kuznetsov
2020-04-17 13:08 ` Wei Liu
2020-04-17 23:07 ` Dexuan Cui
0 siblings, 2 replies; 5+ messages in thread
From: Vitaly Kuznetsov @ 2020-04-17 12:03 UTC (permalink / raw)
To: Wei Liu
Cc: Dexuan Cui, bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel,
mingo, sthemmin, tglx, x86, mikelley, wei.liu, kvm
Wei Liu <wei.liu@kernel.org> writes:
> On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote:
>> Dexuan Cui <decui@microsoft.com> writes:
>>
>> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the
>> > resume path, the "new" kernel's VP assist page is not suspended (i.e.
>> > disabled), and later when we jump to the "old" kernel, the page is not
>> > properly re-enabled for CPU0 with the allocated page from the old kernel.
>> >
>> > So far, the VP assist page is only used by hv_apic_eoi_write().
>>
>> No, not only for that ('git grep hv_get_vp_assist_page')
>>
>> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In
>> particular, Enlightened VMPTR is written there.
>>
>> This makes me wonder: how does hibernation work with KVM in case we use
>> Enlightened VMCS and we have VMs running? We need to make sure VP Assist
>> page content is preserved.
>
> The page itself is preserved, isn't it?
>
Right, unlike hyperv_pcpu_input_arg is is not freed.
> hv_cpu_die never frees the vp_assit page. It merely disables it.
> hv_cpu_init only allocates a new page if necessary.
I'm not really sure that Hyper-V will like us when we disable VP Assist
page and have an active L2 guest using Enlightened VMCS, who knows what
it caches and when. I'll try to at least test if/how it works.
This all is not really related to Dexuan's patch)
--
Vitaly
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation
2020-04-17 12:03 ` Vitaly Kuznetsov
@ 2020-04-17 13:08 ` Wei Liu
2020-04-17 23:07 ` Dexuan Cui
1 sibling, 0 replies; 5+ messages in thread
From: Wei Liu @ 2020-04-17 13:08 UTC (permalink / raw)
To: Vitaly Kuznetsov
Cc: Wei Liu, Dexuan Cui, bp, haiyangz, hpa, kys, linux-hyperv,
linux-kernel, mingo, sthemmin, tglx, x86, mikelley, kvm
On Fri, Apr 17, 2020 at 02:03:38PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
>
> > On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote:
> >> Dexuan Cui <decui@microsoft.com> writes:
> >>
> >> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the
> >> > resume path, the "new" kernel's VP assist page is not suspended (i.e.
> >> > disabled), and later when we jump to the "old" kernel, the page is not
> >> > properly re-enabled for CPU0 with the allocated page from the old kernel.
> >> >
> >> > So far, the VP assist page is only used by hv_apic_eoi_write().
> >>
> >> No, not only for that ('git grep hv_get_vp_assist_page')
> >>
> >> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In
> >> particular, Enlightened VMPTR is written there.
> >>
> >> This makes me wonder: how does hibernation work with KVM in case we use
> >> Enlightened VMCS and we have VMs running? We need to make sure VP Assist
> >> page content is preserved.
> >
> > The page itself is preserved, isn't it?
> >
>
> Right, unlike hyperv_pcpu_input_arg is is not freed.
>
> > hv_cpu_die never frees the vp_assit page. It merely disables it.
> > hv_cpu_init only allocates a new page if necessary.
>
> I'm not really sure that Hyper-V will like us when we disable VP Assist
> page and have an active L2 guest using Enlightened VMCS, who knows what
> it caches and when. I'll try to at least test if/how it works.
>
I'm curious to know that as well. :-)
> This all is not really related to Dexuan's patch)
Right.
Wei.
>
> --
> Vitaly
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation
2020-04-17 12:03 ` Vitaly Kuznetsov
2020-04-17 13:08 ` Wei Liu
@ 2020-04-17 23:07 ` Dexuan Cui
1 sibling, 0 replies; 5+ messages in thread
From: Dexuan Cui @ 2020-04-17 23:07 UTC (permalink / raw)
To: vkuznets, Wei Liu
Cc: bp@alien8.de, Haiyang Zhang, hpa@zytor.com, KY Srinivasan,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
mingo@redhat.com, Stephen Hemminger, tglx@linutronix.de,
x86@kernel.org, Michael Kelley, wei.liu@kernel.org,
kvm@vger.kernel.org
> From: Vitaly Kuznetsov <vkuznets@redhat.com>
> Sent: Friday, April 17, 2020 5:04 AM
> To: Wei Liu <wei.liu@kernel.org>
>
> Wei Liu <wei.liu@kernel.org> writes:
>
> > On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote:
> >> Dexuan Cui <decui@microsoft.com> writes:
> >>
> >> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the
> >> > resume path, the "new" kernel's VP assist page is not suspended (i.e.
> >> > disabled), and later when we jump to the "old" kernel, the page is not
> >> > properly re-enabled for CPU0 with the allocated page from the old kernel.
> >> >
> >> > So far, the VP assist page is only used by hv_apic_eoi_write().
> >>
> >> No, not only for that ('git grep hv_get_vp_assist_page')
Sorry, I unintentionally ignored that, as I have few knowledge about the
optimization for nested virtualization. :-)
> >> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In
> >> particular, Enlightened VMPTR is written there.
> >>
> >> This makes me wonder: how does hibernation work with KVM in case we
> use
> >> Enlightened VMCS and we have VMs running? We need to make sure VP
> Assist
> >> page content is preserved.
> >
> > The page itself is preserved, isn't it?
> >
>
> Right, unlike hyperv_pcpu_input_arg is is not freed.
>
> > hv_cpu_die never frees the vp_assit page. It merely disables it.
> > hv_cpu_init only allocates a new page if necessary.
>
> I'm not really sure that Hyper-V will like us when we disable VP Assist
> page and have an active L2 guest using Enlightened VMCS, who knows what
> it caches and when. I'll try to at least test if/how it works.
>
> This all is not really related to Dexuan's patch)
> --
> Vitaly
It looks you imply that: if there is no active L2 guests, it should be safe to
disable/reenable the assist page upon hibernation?
Can you please write a patch for KVM (when KVM runs on Hyper-V) to abort
the hibernation request if there is any active L2 guest? The pm_notifier can
be used for this.
Thanks,
-- Dexuan
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-04-17 23:07 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1587104999-28927-1-git-send-email-decui@microsoft.com>
2020-04-17 10:03 ` [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation Vitaly Kuznetsov
2020-04-17 10:55 ` Wei Liu
2020-04-17 12:03 ` Vitaly Kuznetsov
2020-04-17 13:08 ` Wei Liu
2020-04-17 23:07 ` Dexuan Cui
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox