From: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
To: James Morse <james.morse-5wv7dgnIgG8@public.gmane.org>
Cc: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org,
kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org,
devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>,
Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>,
Rob Herring <robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
Marc Zyngier <marc.zyngier-5wv7dgnIgG8@public.gmane.org>,
Christoffer Dall
<christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>,
Lorenzo Pieralisi
<lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org>,
Loc Ho <lho-qTEPVZfXA3Y@public.gmane.org>
Subject: Re: [PATCH v4 15/13] firmware: arm_sdei: be more robust against cpu-hotplug
Date: Mon, 13 Nov 2017 11:01:10 +0000 [thread overview]
Message-ID: <20171113110109.GB8968@arm.com> (raw)
In-Reply-To: <20171108160624.10355-1-james.morse-5wv7dgnIgG8@public.gmane.org>
On Wed, Nov 08, 2017 at 04:06:24PM +0000, James Morse wrote:
> dpm_suspend() calls the freeze/thaw callbacks for hibernate before
> disable_non_bootcpus() takes down secondaries.
>
> This leads to a fun race where the freeze/thaw callbacks reset the
> SDEI interface (as we may be restoring a kernel with a different
> layout due to KASLR), then the cpu-hotplug callbacks come in to
> save the current state, which has already been reset.
>
> I tried to solve this with a 'frozen' flag that stops the hotplug
> callback from overwriting the saved values. Instead this just
> moves the race around and makes it even harder to think about.
>
> Instead, make it look like the secondaries have gone offline.
> Call cpuhp_remove_state() in the freeze callback, this will call the
> teardown hook on all online CPUs, then remove the state. This saves
> all private events and makes future CPU up/down events invisible.
>
> Change sdei_event_unregister_all()/sdei_reregister_events() to
> only save/restore shared events, which are all that is left. With
> this we can remove the frozen flag. We can remove the device
> suspend/resume calls too as cpuhotplug's teardown call has masked
> the CPUs.
>
> All that is left is the reboot notifier, (which was abusing the
> frozen flag). Call cpuhp_remove_state() to make it look like
> secondary CPUs have gone offline.
>
> Suggested-by: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
> Signed-off-by: James Morse <james.morse-5wv7dgnIgG8@public.gmane.org>
> ---
> drivers/firmware/arm_sdei.c | 60 +++++++++++++++++++++++----------------------
> 1 file changed, 31 insertions(+), 29 deletions(-)
Thanks, this appears to address my concerns. It's too late for 4.15 now,
but please resend for 4.16 and Catalin can pick this series up.
Will
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: will.deacon@arm.com (Will Deacon)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v4 15/13] firmware: arm_sdei: be more robust against cpu-hotplug
Date: Mon, 13 Nov 2017 11:01:10 +0000 [thread overview]
Message-ID: <20171113110109.GB8968@arm.com> (raw)
In-Reply-To: <20171108160624.10355-1-james.morse@arm.com>
On Wed, Nov 08, 2017 at 04:06:24PM +0000, James Morse wrote:
> dpm_suspend() calls the freeze/thaw callbacks for hibernate before
> disable_non_bootcpus() takes down secondaries.
>
> This leads to a fun race where the freeze/thaw callbacks reset the
> SDEI interface (as we may be restoring a kernel with a different
> layout due to KASLR), then the cpu-hotplug callbacks come in to
> save the current state, which has already been reset.
>
> I tried to solve this with a 'frozen' flag that stops the hotplug
> callback from overwriting the saved values. Instead this just
> moves the race around and makes it even harder to think about.
>
> Instead, make it look like the secondaries have gone offline.
> Call cpuhp_remove_state() in the freeze callback, this will call the
> teardown hook on all online CPUs, then remove the state. This saves
> all private events and makes future CPU up/down events invisible.
>
> Change sdei_event_unregister_all()/sdei_reregister_events() to
> only save/restore shared events, which are all that is left. With
> this we can remove the frozen flag. We can remove the device
> suspend/resume calls too as cpuhotplug's teardown call has masked
> the CPUs.
>
> All that is left is the reboot notifier, (which was abusing the
> frozen flag). Call cpuhp_remove_state() to make it look like
> secondary CPUs have gone offline.
>
> Suggested-by: Will Deacon <will.deacon@arm.com>
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> drivers/firmware/arm_sdei.c | 60 +++++++++++++++++++++++----------------------
> 1 file changed, 31 insertions(+), 29 deletions(-)
Thanks, this appears to address my concerns. It's too late for 4.15 now,
but please resend for 4.16 and Catalin can pick this series up.
Will
next prev parent reply other threads:[~2017-11-13 11:01 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-17 17:44 [PATCH v4 00/13] arm64/firmware: Software Delegated Exception Interface James Morse
2017-10-17 17:44 ` James Morse
[not found] ` <20171017174432.1684-1-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-10-17 17:44 ` [PATCH v4 01/13] KVM: arm64: Store vcpu on the stack during __guest_enter() James Morse
2017-10-17 17:44 ` James Morse
2017-10-17 17:44 ` [PATCH v4 02/13] KVM: arm/arm64: Convert kvm_host_cpu_state to a static per-cpu allocation James Morse
2017-10-17 17:44 ` James Morse
2017-10-17 17:44 ` [PATCH v4 03/13] KVM: arm64: Change hyp_panic()s dependency on tpidr_el2 James Morse
2017-10-17 17:44 ` James Morse
2017-10-17 17:44 ` [PATCH v4 07/13] firmware: arm_sdei: Add driver for Software Delegated Exceptions James Morse
2017-10-17 17:44 ` James Morse
2017-10-18 11:09 ` Catalin Marinas
2017-10-18 11:09 ` Catalin Marinas
2017-10-17 17:44 ` [PATCH v4 08/13] arm64: Add vmap_stack header file James Morse
2017-10-17 17:44 ` James Morse
[not found] ` <20171017174432.1684-9-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-10-18 11:10 ` Catalin Marinas
2017-10-18 11:10 ` Catalin Marinas
2017-10-17 17:44 ` [PATCH v4 10/13] firmware: arm_sdei: Add support for CPU and system power states James Morse
2017-10-17 17:44 ` James Morse
[not found] ` <20171017174432.1684-11-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-10-18 11:11 ` Catalin Marinas
2017-10-18 11:11 ` Catalin Marinas
2017-10-18 17:17 ` Will Deacon
2017-10-18 17:17 ` Will Deacon
[not found] ` <20171018171747.GI21820-5wv7dgnIgG8@public.gmane.org>
2017-10-24 17:34 ` James Morse
2017-10-24 17:34 ` James Morse
[not found] ` <59EF799B.4040802-5wv7dgnIgG8@public.gmane.org>
2017-10-25 14:43 ` Will Deacon
2017-10-25 14:43 ` Will Deacon
2017-10-17 17:44 ` [PATCH v4 12/13] arm64: acpi: Remove __init from acpi_psci_use_hvc() for use by SDEI James Morse
2017-10-17 17:44 ` James Morse
2017-10-18 11:11 ` Catalin Marinas
2017-10-18 11:11 ` Catalin Marinas
2017-10-17 17:44 ` [PATCH v4 13/13] firmware: arm_sdei: Discover SDEI support via ACPI James Morse
2017-10-17 17:44 ` James Morse
[not found] ` <20171017174432.1684-14-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-10-18 11:12 ` Catalin Marinas
2017-10-18 11:12 ` Catalin Marinas
2017-10-18 17:16 ` [PATCH v4 00/13] arm64/firmware: Software Delegated Exception Interface Will Deacon
2017-10-18 17:16 ` Will Deacon
2017-10-30 15:58 ` [PATCH v4 14/13] firmware: arm_sdei: Move cpuhotplug registration later James Morse
2017-10-30 15:58 ` James Morse
2017-10-17 17:44 ` [PATCH v4 04/13] arm64: alternatives: use tpidr_el2 on VHE hosts James Morse
2017-10-17 17:44 ` James Morse
2017-10-18 11:00 ` Catalin Marinas
2017-10-18 11:00 ` Catalin Marinas
2017-10-17 17:44 ` [PATCH v4 05/13] KVM: arm64: Stop save/restoring host tpidr_el1 on VHE James Morse
2017-10-17 17:44 ` James Morse
2017-10-17 17:44 ` [PATCH v4 06/13] Docs: dt: add devicetree binding for describing arm64 SDEI firmware James Morse
2017-10-17 17:44 ` James Morse
2017-10-17 17:44 ` [PATCH v4 09/13] arm64: kernel: Add arch-specific SDEI entry code and CPU masking James Morse
2017-10-17 17:44 ` James Morse
[not found] ` <20171017174432.1684-10-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-10-18 11:18 ` Catalin Marinas
2017-10-18 11:18 ` Catalin Marinas
2017-10-17 17:44 ` [PATCH v4 11/13] firmware: arm_sdei: add support for CPU private events James Morse
2017-10-17 17:44 ` James Morse
[not found] ` <20171017174432.1684-12-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-10-18 11:11 ` Catalin Marinas
2017-10-18 11:11 ` Catalin Marinas
2017-10-18 17:19 ` Will Deacon
2017-10-18 17:19 ` Will Deacon
[not found] ` <20171018171959.GJ21820-5wv7dgnIgG8@public.gmane.org>
2017-10-24 17:34 ` James Morse
2017-10-24 17:34 ` James Morse
[not found] ` <59EF798A.9000609-5wv7dgnIgG8@public.gmane.org>
2017-11-01 15:59 ` James Morse
2017-11-01 15:59 ` James Morse
2017-11-01 15:59 ` [PATCH 15/13] firmware: arm_sdei: move the frozen flag under the spinlock James Morse
2017-11-01 15:59 ` James Morse
[not found] ` <20171101155941.19432-1-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-11-08 15:11 ` James Morse
2017-11-08 15:11 ` James Morse
[not found] ` <5A031E9E.2090809-5wv7dgnIgG8@public.gmane.org>
2017-11-08 16:06 ` [PATCH v4 15/13] firmware: arm_sdei: be more robust against cpu-hotplug James Morse
2017-11-08 16:06 ` James Morse
[not found] ` <20171108160624.10355-1-james.morse-5wv7dgnIgG8@public.gmane.org>
2017-11-13 11:01 ` Will Deacon [this message]
2017-11-13 11:01 ` Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171113110109.GB8968@arm.com \
--to=will.deacon-5wv7dgnigg8@public.gmane.org \
--cc=catalin.marinas-5wv7dgnIgG8@public.gmane.org \
--cc=christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org \
--cc=devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=james.morse-5wv7dgnIgG8@public.gmane.org \
--cc=kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org \
--cc=lho-qTEPVZfXA3Y@public.gmane.org \
--cc=linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \
--cc=lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org \
--cc=marc.zyngier-5wv7dgnIgG8@public.gmane.org \
--cc=mark.rutland-5wv7dgnIgG8@public.gmane.org \
--cc=robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.