* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation [not found] <1587104999-28927-1-git-send-email-decui@microsoft.com> @ 2020-04-17 10:03 ` Vitaly Kuznetsov 2020-04-17 10:55 ` Wei Liu 0 siblings, 1 reply; 5+ messages in thread From: Vitaly Kuznetsov @ 2020-04-17 10:03 UTC (permalink / raw) To: Dexuan Cui Cc: bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel, mingo, sthemmin, tglx, x86, mikelley, wei.liu, kvm Dexuan Cui <decui@microsoft.com> writes: > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the > resume path, the "new" kernel's VP assist page is not suspended (i.e. > disabled), and later when we jump to the "old" kernel, the page is not > properly re-enabled for CPU0 with the allocated page from the old kernel. > > So far, the VP assist page is only used by hv_apic_eoi_write(). No, not only for that ('git grep hv_get_vp_assist_page') KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In particular, Enlightened VMPTR is written there. This makes me wonder: how does hibernation work with KVM in case we use Enlightened VMCS and we have VMs running? We need to make sure VP Assist page content is preserved. -- Vitaly ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation 2020-04-17 10:03 ` [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation Vitaly Kuznetsov @ 2020-04-17 10:55 ` Wei Liu 2020-04-17 12:03 ` Vitaly Kuznetsov 0 siblings, 1 reply; 5+ messages in thread From: Wei Liu @ 2020-04-17 10:55 UTC (permalink / raw) To: Vitaly Kuznetsov Cc: Dexuan Cui, bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel, mingo, sthemmin, tglx, x86, mikelley, wei.liu, kvm On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote: > Dexuan Cui <decui@microsoft.com> writes: > > > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the > > resume path, the "new" kernel's VP assist page is not suspended (i.e. > > disabled), and later when we jump to the "old" kernel, the page is not > > properly re-enabled for CPU0 with the allocated page from the old kernel. > > > > So far, the VP assist page is only used by hv_apic_eoi_write(). > > No, not only for that ('git grep hv_get_vp_assist_page') > > KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In > particular, Enlightened VMPTR is written there. > > This makes me wonder: how does hibernation work with KVM in case we use > Enlightened VMCS and we have VMs running? We need to make sure VP Assist > page content is preserved. The page itself is preserved, isn't it? hv_cpu_die never frees the vp_assit page. It merely disables it. hv_cpu_init only allocates a new page if necessary. Wei. > > -- > Vitaly > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation 2020-04-17 10:55 ` Wei Liu @ 2020-04-17 12:03 ` Vitaly Kuznetsov 2020-04-17 13:08 ` Wei Liu 2020-04-17 23:07 ` Dexuan Cui 0 siblings, 2 replies; 5+ messages in thread From: Vitaly Kuznetsov @ 2020-04-17 12:03 UTC (permalink / raw) To: Wei Liu Cc: Dexuan Cui, bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel, mingo, sthemmin, tglx, x86, mikelley, wei.liu, kvm Wei Liu <wei.liu@kernel.org> writes: > On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote: >> Dexuan Cui <decui@microsoft.com> writes: >> >> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the >> > resume path, the "new" kernel's VP assist page is not suspended (i.e. >> > disabled), and later when we jump to the "old" kernel, the page is not >> > properly re-enabled for CPU0 with the allocated page from the old kernel. >> > >> > So far, the VP assist page is only used by hv_apic_eoi_write(). >> >> No, not only for that ('git grep hv_get_vp_assist_page') >> >> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In >> particular, Enlightened VMPTR is written there. >> >> This makes me wonder: how does hibernation work with KVM in case we use >> Enlightened VMCS and we have VMs running? We need to make sure VP Assist >> page content is preserved. > > The page itself is preserved, isn't it? > Right, unlike hyperv_pcpu_input_arg is is not freed. > hv_cpu_die never frees the vp_assit page. It merely disables it. > hv_cpu_init only allocates a new page if necessary. I'm not really sure that Hyper-V will like us when we disable VP Assist page and have an active L2 guest using Enlightened VMCS, who knows what it caches and when. I'll try to at least test if/how it works. This all is not really related to Dexuan's patch) -- Vitaly ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation 2020-04-17 12:03 ` Vitaly Kuznetsov @ 2020-04-17 13:08 ` Wei Liu 2020-04-17 23:07 ` Dexuan Cui 1 sibling, 0 replies; 5+ messages in thread From: Wei Liu @ 2020-04-17 13:08 UTC (permalink / raw) To: Vitaly Kuznetsov Cc: Wei Liu, Dexuan Cui, bp, haiyangz, hpa, kys, linux-hyperv, linux-kernel, mingo, sthemmin, tglx, x86, mikelley, kvm On Fri, Apr 17, 2020 at 02:03:38PM +0200, Vitaly Kuznetsov wrote: > Wei Liu <wei.liu@kernel.org> writes: > > > On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote: > >> Dexuan Cui <decui@microsoft.com> writes: > >> > >> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the > >> > resume path, the "new" kernel's VP assist page is not suspended (i.e. > >> > disabled), and later when we jump to the "old" kernel, the page is not > >> > properly re-enabled for CPU0 with the allocated page from the old kernel. > >> > > >> > So far, the VP assist page is only used by hv_apic_eoi_write(). > >> > >> No, not only for that ('git grep hv_get_vp_assist_page') > >> > >> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In > >> particular, Enlightened VMPTR is written there. > >> > >> This makes me wonder: how does hibernation work with KVM in case we use > >> Enlightened VMCS and we have VMs running? We need to make sure VP Assist > >> page content is preserved. > > > > The page itself is preserved, isn't it? > > > > Right, unlike hyperv_pcpu_input_arg is is not freed. > > > hv_cpu_die never frees the vp_assit page. It merely disables it. > > hv_cpu_init only allocates a new page if necessary. > > I'm not really sure that Hyper-V will like us when we disable VP Assist > page and have an active L2 guest using Enlightened VMCS, who knows what > it caches and when. I'll try to at least test if/how it works. > I'm curious to know that as well. :-) > This all is not really related to Dexuan's patch) Right. Wei. > > -- > Vitaly > ^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation 2020-04-17 12:03 ` Vitaly Kuznetsov 2020-04-17 13:08 ` Wei Liu @ 2020-04-17 23:07 ` Dexuan Cui 1 sibling, 0 replies; 5+ messages in thread From: Dexuan Cui @ 2020-04-17 23:07 UTC (permalink / raw) To: vkuznets, Wei Liu Cc: bp@alien8.de, Haiyang Zhang, hpa@zytor.com, KY Srinivasan, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, mingo@redhat.com, Stephen Hemminger, tglx@linutronix.de, x86@kernel.org, Michael Kelley, wei.liu@kernel.org, kvm@vger.kernel.org > From: Vitaly Kuznetsov <vkuznets@redhat.com> > Sent: Friday, April 17, 2020 5:04 AM > To: Wei Liu <wei.liu@kernel.org> > > Wei Liu <wei.liu@kernel.org> writes: > > > On Fri, Apr 17, 2020 at 12:03:18PM +0200, Vitaly Kuznetsov wrote: > >> Dexuan Cui <decui@microsoft.com> writes: > >> > >> > Unlike the other CPUs, CPU0 is never offlined during hibernation. So in the > >> > resume path, the "new" kernel's VP assist page is not suspended (i.e. > >> > disabled), and later when we jump to the "old" kernel, the page is not > >> > properly re-enabled for CPU0 with the allocated page from the old kernel. > >> > > >> > So far, the VP assist page is only used by hv_apic_eoi_write(). > >> > >> No, not only for that ('git grep hv_get_vp_assist_page') Sorry, I unintentionally ignored that, as I have few knowledge about the optimization for nested virtualization. :-) > >> KVM on Hyper-V also needs VP assist page to use Enlightened VMCS. In > >> particular, Enlightened VMPTR is written there. > >> > >> This makes me wonder: how does hibernation work with KVM in case we > use > >> Enlightened VMCS and we have VMs running? We need to make sure VP > Assist > >> page content is preserved. > > > > The page itself is preserved, isn't it? > > > > Right, unlike hyperv_pcpu_input_arg is is not freed. > > > hv_cpu_die never frees the vp_assit page. It merely disables it. > > hv_cpu_init only allocates a new page if necessary. > > I'm not really sure that Hyper-V will like us when we disable VP Assist > page and have an active L2 guest using Enlightened VMCS, who knows what > it caches and when. I'll try to at least test if/how it works. > > This all is not really related to Dexuan's patch) > -- > Vitaly It looks you imply that: if there is no active L2 guests, it should be safe to disable/reenable the assist page upon hibernation? Can you please write a patch for KVM (when KVM runs on Hyper-V) to abort the hibernation request if there is any active L2 guest? The pm_notifier can be used for this. Thanks, -- Dexuan ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-04-17 23:07 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1587104999-28927-1-git-send-email-decui@microsoft.com>
2020-04-17 10:03 ` [PATCH] x86/hyperv: Suspend/resume the VP assist page for hibernation Vitaly Kuznetsov
2020-04-17 10:55 ` Wei Liu
2020-04-17 12:03 ` Vitaly Kuznetsov
2020-04-17 13:08 ` Wei Liu
2020-04-17 23:07 ` Dexuan Cui
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox