[PATCH v5] x86/sgx: Report RCU-Tasks quiescent state in EPC sanitization loop

[PATCH v5] x86/sgx: Report RCU-Tasks quiescent state in EPC sanitization loop

[Jun Miao]

When the kernel boots from kexec, the EPC pages may have a stale state.
The kernel sanitizes all EPC pages to reset them to a clean state before
their first use in any enclave.  The EPC size could be several GBs and
resetting them could take a significant amount of time.  Because of that,
the kernel performs the reset in a loop through a kernel thread ksgxd() at
early boot, and there's a cond_resched() after resetting each EPC page.

This is fine in most cases, but becomes a problem when there's other kernel
code waiting for an RCU-Tasks grace period but the cond_resched() in
ksgxd() never triggers rescheduling.  Because cond_resched() doesn't report
a quiescent state when it doesn't trigger rescheduling, the thread that is
waiting for an RCU-Tasks grace period will wait until all EPC pages are
reset.

For instance, BPF LSM subsystem can invoke synchronize_rcu_tasks() at
kernel boot time.  A VM with a large EPC assigned and BPF LSM enabled can
take a long time to boot, with a call trace triggered:

    rcu_tasks_wait_gp: rcu_tasks grace period number 1 (since boot) is
	130631 jiffies old.
    INFO: task systemd:1 blocked for more than 122 seconds.
    ...
    task:systemd  state:D stack:0  pid:1  tpid:1  ppid:0  flags:0x00000002
    Call Trace:
    ...
    schedule_timeout+0x157/0x170
    wait_for_completion+0x88/0x150
    __wait_rcu_gp+0x17e/0x190
    synchronize_rcu_tasks_generic+0x64/0x60
    ...
    synchronize_rcu_tasks+0x15/0x20
    register_ftrace_direct+0x31f/0x350
    ...
    bpf_trampoline_link_prog+0x33/0x60
    bpf_tracing_prog_attach+0x3c5/0x5f0

Replace cond_resched() with cond_resched_tasks_rcu_qs() which explicitly
reports quiescent state regardless of whether actual rescheduling is
triggered.  Resetting all EPC pages in ksgxd() isn't performance critical
so the extra cost of cond_resched_tasks_rcu_qs() isn't a problem.

Tests showed this reduced the VM kernel boot time from ~50s to ~700ms.

Fixes: e7e0545299d8 ("x86/sgx: Initialize metadata for Enclave Page Cache (EPC) sections")
Suggested-by: Kai Huang <kai.huang@intel.com>
Co-developed-by: Fan Du <fan.du@intel.com>
Signed-off-by: Fan Du <fan.du@intel.com>
Signed-off-by: Jun Miao <jun.miao@intel.com>
Tested-by: Challvy Tee <challvy.tee@gmail.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://github.com/systemd/systemd/issues/40423

---
v1 -> v2:
 - Clarify the RCU Tasks stall root cause.
 - Use cond_resched_rcu_qs() following the Kai`s suggestion.

v2 -> v3:
 - cee439398933 ("rcu: Rename cond_resched_rcu_qs() to cond_resched_tasks_rcu_qs()")

v3 -> v4:
 - Trim down/rewrite changelog following Kai`s suggestion.

v4 -> v5:
 - Change the title, not state the problem directly
 - Corrected spelling and grammatical errors by Kai
 - Add "Reviewed-by: Kai Huang"

---
 arch/x86/kernel/cpu/sgx/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 4505f808af5e..7d2f57663177 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -106,7 +106,7 @@ static unsigned long __sgx_sanitize_pages(struct list_head *dirty_page_list)
 			left_dirty++;
 		}

-		cond_resched();
+		cond_resched_tasks_rcu_qs();
 	}

 	list_splice(&dirty, dirty_page_list);
--
2.32.0

Re: [PATCH v5] x86/sgx: Report RCU-Tasks quiescent state in EPC sanitization loop

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] The cond_resched() call in the main loop of ksgxd() was not updated to explicitly report RCU-tasks quiescent states, leaving it vulnerable to stalls.
- [High] SGX initialization error path frees EPC page arrays without removing their physical addresses from the global sgx_epc_address_space XArray, leading to Use-After-Free on hardware memory errors.
--

commit 71875f4f4ed3656160e4c510add3ac4a68c4cd86
Author: Jun Miao <jun.miao@intel.com>

x86/sgx: Report RCU-Tasks quiescent state in EPC sanitization loop

When the kernel boots from kexec, the EPC pages may have a stale state.
The kernel sanitizes all EPC pages to reset them to a clean state before
their first use in any enclave.  The EPC size could be several GBs and
resetting them could take a significant amount of time.

Link: https://github.com/systemd/systemd/issues/40423

> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index dc73194416ac8..545415ee98177 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -103,7 +103,7 @@ static unsigned long __sgx_sanitize_pages(struct list_head *dirty_page_list)
>  			left_dirty++;
>  		}
>  
> -		cond_resched();
> +		cond_resched_tasks_rcu_qs();
>  	}
>  
>  	list_splice(&dirty, dirty_page_list);

[Severity: High]
This isn't a bug introduced by this patch, but looking at the main loop
in ksgxd() in the same file:

arch/x86/kernel/cpu/sgx/main.c:ksgxd() {
...
		if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
			sgx_reclaim_pages();

		cond_resched();
...
}

Could this cond_resched() also starve RCU-tasks grace periods if
sgx_should_reclaim() remains continuously true under heavy SGX memory pressure?

Since wait_event_freezable() won't sleep in this case, would it make sense
to update this to cond_resched_tasks_rcu_qs() as well?


[Severity: High]
This is a pre-existing issue, but does the error path in sgx_init() leave
dangling pointers in the global sgx_epc_address_space XArray?

arch/x86/kernel/cpu/sgx/main.c:sgx_init() {
...
err_page_cache:
	for (i = 0; i < sgx_nr_epc_sections; i++) {
		vfree(sgx_epc_sections[i].pages);
		memunmap(sgx_epc_sections[i].virt_addr);
	}

	return ret;
}

If sgx_page_cache_init() succeeds (which populates the XArray) but a later
initialization step fails, the section->pages arrays are freed here without
being removed from the XArray.

If a hardware memory error occurs later, arch_memory_failure() looks up the
PFN via sgx_paddr_to_page():

arch/x86/kernel/cpu/sgx/main.c:sgx_paddr_to_page() {
...
	section = xa_load(&sgx_epc_address_space, paddr);
	if (!section)
		return NULL;

	return &section->pages[PFN_DOWN(paddr - section->phys_addr)];
}

Could this lead to a use-after-free in arch_memory_failure() when it
dereferences the returned page pointer?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260703084810.145567-1-jun.miao@intel.com?part=1