From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Dexuan Cui <decui@microsoft.com>
Cc: "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
"arnd@arndb.de" <arnd@arndb.de>, "bp@alien8.de" <bp@alien8.de>,
"daniel.lezcano@linaro.org" <daniel.lezcano@linaro.org>,
Haiyang Zhang <haiyangz@microsoft.com>,
"hpa@zytor.com" <hpa@zytor.com>,
KY Srinivasan <kys@microsoft.com>,
"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"mingo@redhat.com" <mingo@redhat.com>,
"sashal@kernel.org" <sashal@kernel.org>,
Stephen Hemminger <sthemmin@microsoft.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"x86@kernel.org" <x86@kernel.org>,
Michael Kelley <mikelley@microsoft.com>,
Sasha Levin <Alexander.Levin@microsoft.com>
Subject: RE: [PATCH v5 1/3] x86/hyper-v: Suspend/resume the hypercall page for hibernation
Date: Fri, 27 Sep 2019 11:05:24 +0200 [thread overview]
Message-ID: <877e5u6re3.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <PU1P153MB01695A159E9843B4E1EC13AEBF810@PU1P153MB0169.APCP153.PROD.OUTLOOK.COM>
Dexuan Cui <decui@microsoft.com> writes:
>> From: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Sent: Thursday, September 26, 2019 3:44 AM
>> > [...]
>> > +static int hv_suspend(void)
>> > +{
>> > + union hv_x64_msr_hypercall_contents hypercall_msr;
>> > +
>> > + /* Reset the hypercall page */
>> > + rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > + hypercall_msr.enable = 0;
>> > + wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > +
>>
>> (trying to think out loud, not sure there's a real issue):
>>
>> When PV IPIs (or PV TLB flush) are enabled we do the following checks:
>>
>> if (!hv_hypercall_pg)
>> return false;
>>
>> or
>> if (!hv_hypercall_pg)
>> goto do_native;
>>
>> which will pass as we're not invalidating the pointer. Can we actually
>> be sure that the kernel will never try to send an IPI/do TLB flush
>> before we resume?
>>
>> Vitaly
>
> When hv_suspend() and hv_resume() are called by syscore_suspend()
> and syscore_resume(), respectively, all the non-boot CPUs are disabled and
> only CPU0 is active and interrupts are disabled, e.g. see
>
> hibernate() ->
> hibernation_snapshot() ->
> create_image() ->
> suspend_disable_secondary_cpus()
> local_irq_disable()
>
> syscore_suspend()
> swsusp_arch_suspend
> syscore_resume
>
> local_irq_enable
> enable_nonboot_cpus
>
>
> So, I'm pretty sure no IPI can happen between hv_suspend() and hv_resume().
> self-IPI is not supposed to happen either, since interrupts are disabled.
>
> IMO TLB flush should not be an issue either, unless the kernel changes page
> tables between hv_suspend() and hv_resume(), which is not the case as I
> checked the related code, but it looks in theory that might happen, say, in
> the future, so if you insist we should save the variable "hv_hypercall_pg"
> to a temporary variable and set the "hv_hypercall_pg" to NULL before we
> disable the hypercall page
Let's do it as a future proof so we can keep relying on !hv_hypercall_pg
everywhere we need. No need to change this patch IMO, a follow-up would
do.
--
Vitaly
WARNING: multiple messages have this Message-ID (diff)
From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Dexuan Cui <decui@microsoft.com>
Cc: "linux-arch\@vger.kernel.org" <linux-arch@vger.kernel.org>,
"arnd\@arndb.de" <arnd@arndb.de>, "bp\@alien8.de" <bp@alien8.de>,
"daniel.lezcano\@linaro.org" <daniel.lezcano@linaro.org>,
Haiyang Zhang <haiyangz@microsoft.com>,
"hpa\@zytor.com" <hpa@zytor.com>,
KY Srinivasan <kys@microsoft.com>,
"linux-hyperv\@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
"linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"mingo\@redhat.com" <mingo@redhat.com>,
"sashal\@kernel.org" <sashal@kernel.org>,
Stephen Hemminger <sthemmin@microsoft.com>,
"tglx\@linutronix.de" <tglx@linutronix.de>,
"x86\@kernel.org" <x86@kernel.org>,
Michael Kelley <mikelley@microsoft.com>,
Sasha Levin <Alexander.Levin@microsoft.com>
Subject: RE: [PATCH v5 1/3] x86/hyper-v: Suspend/resume the hypercall page for hibernation
Date: Fri, 27 Sep 2019 11:05:24 +0200 [thread overview]
Message-ID: <877e5u6re3.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <PU1P153MB01695A159E9843B4E1EC13AEBF810@PU1P153MB0169.APCP153.PROD.OUTLOOK.COM>
Dexuan Cui <decui@microsoft.com> writes:
>> From: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Sent: Thursday, September 26, 2019 3:44 AM
>> > [...]
>> > +static int hv_suspend(void)
>> > +{
>> > + union hv_x64_msr_hypercall_contents hypercall_msr;
>> > +
>> > + /* Reset the hypercall page */
>> > + rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > + hypercall_msr.enable = 0;
>> > + wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > +
>>
>> (trying to think out loud, not sure there's a real issue):
>>
>> When PV IPIs (or PV TLB flush) are enabled we do the following checks:
>>
>> if (!hv_hypercall_pg)
>> return false;
>>
>> or
>> if (!hv_hypercall_pg)
>> goto do_native;
>>
>> which will pass as we're not invalidating the pointer. Can we actually
>> be sure that the kernel will never try to send an IPI/do TLB flush
>> before we resume?
>>
>> Vitaly
>
> When hv_suspend() and hv_resume() are called by syscore_suspend()
> and syscore_resume(), respectively, all the non-boot CPUs are disabled and
> only CPU0 is active and interrupts are disabled, e.g. see
>
> hibernate() ->
> hibernation_snapshot() ->
> create_image() ->
> suspend_disable_secondary_cpus()
> local_irq_disable()
>
> syscore_suspend()
> swsusp_arch_suspend
> syscore_resume
>
> local_irq_enable
> enable_nonboot_cpus
>
>
> So, I'm pretty sure no IPI can happen between hv_suspend() and hv_resume().
> self-IPI is not supposed to happen either, since interrupts are disabled.
>
> IMO TLB flush should not be an issue either, unless the kernel changes page
> tables between hv_suspend() and hv_resume(), which is not the case as I
> checked the related code, but it looks in theory that might happen, say, in
> the future, so if you insist we should save the variable "hv_hypercall_pg"
> to a temporary variable and set the "hv_hypercall_pg" to NULL before we
> disable the hypercall page
Let's do it as a future proof so we can keep relying on !hv_hypercall_pg
everywhere we need. No need to change this patch IMO, a follow-up would
do.
--
Vitaly
next prev parent reply other threads:[~2019-09-27 9:05 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-05 22:47 [PATCH v5 0/3] Enhance Hyper-V for hibernation Dexuan Cui
2019-09-05 22:47 ` [PATCH v5 1/3] x86/hyper-v: Suspend/resume the hypercall page " Dexuan Cui
2019-09-26 10:44 ` Vitaly Kuznetsov
2019-09-26 10:44 ` Vitaly Kuznetsov
2019-09-27 6:48 ` Dexuan Cui
2019-09-27 9:05 ` Vitaly Kuznetsov [this message]
2019-09-27 9:05 ` Vitaly Kuznetsov
2019-09-30 18:49 ` Dexuan Cui
2019-09-05 22:47 ` [PATCH v5 2/3] x86/hyper-v: Implement hv_is_hibernation_supported() Dexuan Cui
2019-09-05 22:47 ` [PATCH v5 3/3] clocksource/drivers: Suspend/resume Hyper-V clocksource for hibernation Dexuan Cui
2019-09-25 23:21 ` Daniel Lezcano
2019-09-25 23:35 ` Dexuan Cui
2019-09-26 13:16 ` Daniel Lezcano
2019-09-27 5:57 ` Dexuan Cui
2019-09-25 22:49 ` [PATCH v5 0/3] Enhance Hyper-V " Dexuan Cui
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=877e5u6re3.fsf@vitty.brq.redhat.com \
--to=vkuznets@redhat.com \
--cc=Alexander.Levin@microsoft.com \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=daniel.lezcano@linaro.org \
--cc=decui@microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=hpa@zytor.com \
--cc=kys@microsoft.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mikelley@microsoft.com \
--cc=mingo@redhat.com \
--cc=sashal@kernel.org \
--cc=sthemmin@microsoft.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.