All of lore.kernel.org
 help / color / mirror / Atom feed
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
To: Mykola Kvach <Mykola_Kvach@epam.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"Andrew Cooper" <andrew.cooper3@citrix.com>,
	"Anthony PERARD" <anthony.perard@vates.tech>,
	"Michal Orzel" <michal.orzel@amd.com>,
	"Jan Beulich" <jbeulich@suse.com>,
	"Julien Grall" <julien@xen.org>,
	"Roger Pau Monné" <roger@xenproject.org>,
	"Stefano Stabellini" <sstabellini@kernel.org>,
	"Juergen Gross" <jgross@suse.com>
Subject: Re: [PATCH v14 04/15] xen/percpu: defer freeing per-CPU areas after resume failures
Date: Wed, 18 Nov 2026 01:51:35 +0000	[thread overview]
Message-ID: <87cxs2oqjc.fsf@epam.com> (raw)
In-Reply-To: <3b0a92602e8cbc2f4df6b376876903e93850be3f.1794873792.git.mykola_kvach@epam.com>

Hi,

CC'ed Juergen, because I have a question about cpupools.

Mykola Kvach <Mykola_Kvach@epam.com> writes:

> When a CPU fails to come back during system resume, cpu_up() sends
> CPU_UP_CANCELED. After trying the other CPUs, enable_nonboot_cpus()
> sends CPU_RESUME_FAILED for the same CPU.
>
> On architectures which do not park offline CPUs, both notifications
> queue free_percpu_area(). The release metadata, including the rcu_head,
> is stored in that CPU's per-CPU area. The first callback may already
> have freed the area when the second notification arrives, or the same
> rcu_head may be queued twice. The CPU_RESUME_FAILED callbacks also need
> the area to clean up timers and CPU-pool state.

A bit convoluted. As I see, problem is caused by the fact
CPU_UP_CANCELED is a terminal state in a "simple" CPU up sequence, but
is a transient state in a system resume sequence. As such, theoretically
other per_cpu callbacks might have a similar issue when they don't
expect one more "CPU state is bad" callback. I can see that
common/timer.c also tries to handle both callbacks as terminal
ones. Probably, you need a similar fix there as well.

Actually, I am not sure why do we need CPU_RESUME_FAILED in the first
place. As I can see, only cpupool handles it in a sensible way. Which
begs a question: why cpupool can't call cpupool_cpu_remove_forced() on
CPU_UP_CANCELED instead?



>
> Skip the per-CPU release for CPU_UP_CANCELED during resume. Let
> CPU_RESUME_FAILED release the area after the other cleanup callbacks.
>
> This prepares the common code for Arm system suspend. x86 parks offline
> CPUs, so the per-CPU notifier does not call free_percpu_area() for either
> CPU_UP_CANCELED or CPU_RESUME_FAILED.
>
> Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
> ---
> Changes in V14:
> - Clarify that x86 skips free_percpu_area() for CPU_UP_CANCELED and
>   CPU_RESUME_FAILED.
> - Remove the blank line after fallthrough. No functional change.
>
> Changes in v13:
> - New patch.
> ---
>  xen/common/percpu.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/xen/common/percpu.c b/xen/common/percpu.c
> index f180f37253..757de111db 100644
> --- a/xen/common/percpu.c
> +++ b/xen/common/percpu.c
> @@ -82,6 +82,14 @@ static int cf_check cpu_percpu_callback(
>          break;
>  
>      case CPU_UP_CANCELED:
> +        /*
> +         * On resume, CPU_RESUME_FAILED is sent after all CPUs have been
> +         * tried. Keep the area alive for its cleanup callbacks and let
> +         * that notification release it.
> +         */
> +        if ( system_state == SYS_STATE_resume )
> +            break;
> +        fallthrough;
>      case CPU_DEAD:
>      case CPU_RESUME_FAILED:
>          if ( !park_offline_cpus && system_state != SYS_STATE_suspend )

-- 
WBR, Volodymyr

  reply	other threads:[~2026-11-18  1:52 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17  1:04 [PATCH v14 00/15] Add initial Xen Suspend-to-RAM support on ARM64 Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 01/15] xen/arm: Add suspend and resume timer helpers Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 02/15] xen/arm: gic-v2: Implement GIC suspend/resume functions Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 03/15] xen/arm: gic-v3: tolerate retained redistributor LPI state across CPU_OFF Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 04/15] xen/percpu: defer freeing per-CPU areas after resume failures Mykola Kvach
2026-11-18  1:51   ` Volodymyr Babchuk [this message]
2026-11-17  1:04 ` [PATCH v14 05/15] xen/arm: gic-v3: Implement GICv3 suspend/resume functions Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 06/15] xen/arm: gic-v3: add ITS suspend/resume support Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 07/15] xen/arm: tee: keep init_tee_secondary() for hotplug and resume Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 08/15] xen/arm: ffa: fix notification SRI across CPU hotplug/suspend Mykola Kvach
2026-11-17  1:04 ` [PATCH v14 09/15] iommu/ipmmu-vmsa: Implement suspend/resume callbacks Mykola Kvach
2026-11-17  1:05 ` [PATCH v14 10/15] xen/arm: smmu-v3: add suspend/resume handlers Mykola Kvach
2026-11-17 17:04   ` Bertrand Marquis
2026-11-17  1:05 ` [PATCH v14 11/15] xen/arm64: Keep the early UART message in the identity mapping Mykola Kvach
2026-11-17 16:59   ` Bertrand Marquis
2026-11-17  1:05 ` [PATCH v14 12/15] xen/arm64: Save/restore CPU context across SYSTEM_SUSPEND Mykola Kvach
2026-11-17  1:05 ` [PATCH v14 13/15] xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface) Mykola Kvach
2026-11-17  1:05 ` [PATCH v14 14/15] xen/arm: Add vPSCI SYSTEM_SUSPEND policy Mykola Kvach
2026-11-17  1:05 ` [PATCH v14 15/15] xen/arm: Add host system suspend backend Mykola Kvach
2026-11-17 17:05   ` Bertrand Marquis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87cxs2oqjc.fsf@epam.com \
    --to=volodymyr_babchuk@epam.com \
    --cc=Mykola_Kvach@epam.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=anthony.perard@vates.tech \
    --cc=jbeulich@suse.com \
    --cc=jgross@suse.com \
    --cc=julien@xen.org \
    --cc=michal.orzel@amd.com \
    --cc=roger@xenproject.org \
    --cc=sstabellini@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.