[PATCH v2] x86/sgx: Fix RCU Tasks stalls in EPC sanitization loop

[PATCH v2] x86/sgx: Fix RCU Tasks stalls in EPC sanitization loop

[Jun Miao]

Large EPC configurations can spend a significant amount of time
sanitizing EPC pages during SGX initialization.  The sanitization
loop invokes cond_resched() while processing pages, but if the
scheduler does not request rescheduling, the CPU may remain in the
kernel for an extended period without reporting a quiescent state to
RCU Tasks.

cond_resched() only schedules when rescheduling is needed and does
not guarantee a quiescent state for RCU Tasks.  Replace it with
cond_resched_rcu_qs(), which explicitly reports an RCU Tasks
quiescent state even when no context switch occurs.

Kai suggested that, this is a common problem at the scheduler and RCU layer,
but not specific to SGX. More detail please see:
bde6c3aa9930 ("rcu: Provide cond_resched_rcu_qs() to force quiescent states in long loops").

Without this patch, instead, virtual machines (VMs) experience a long OS boot times:

[    4.110549] systemd[1]: Detected architecture x86-64.
[    4.115279] systemd[1]: Hostname set to <i2bp1g0g0m0i8406er0g1zX2>.
[    4.115554] systemd[1]: Installed transient /etc/machine-id file.
[   14.262158] rcu_tasks_wait_gp: rcu_tasks grace period number 1 (since boot) is 10087 jiffies old.
[   14.374158] rcu_tasks_wait_gp: rcu_tasks grace period number 1 (since boot) is 40199 jiffies old.
[  134.806157] rcu_tasks_wait_gp: rcu_tasks grace period number 1 (since boot) is 130631 jiffies old.
[  248.086158] INFO: task systemd:1 blocked for more than 122 seconds.
[  248.086491] Not tainted 6.8.0-90-generic #91-Ubuntu
[  248.086739] 'echo 0 > /proc/sys/kernel/hung_task_timeout_secs' disables this message.
[  248.086993] task:systemd    state:D stack:0    pid:1    tpid:1    ppid:0    flags:0x00000002
[  248.087274] Call Trace:
[  248.087434] <TASK>
[  248.087557] __schedule+0x27c/0x6b0
[  248.087770] schedule+0x33/0x110
[  248.087939] schedule_timeout+0x157/0x170
[  248.088120] wait_for_completion+0x88/0x150
[  248.088304] __wait_rcu_gp+0x17e/0x190
[  248.088481] synchronize_rcu_tasks_generic+0x64/0x60
[  248.088672] ? __pfx_call_rcu_tasks+0x10/0x10
[  248.088858] ? __pfx_wakeme_after_rcu+0x10/0x10
[  248.089047] synchronize_rcu_tasks+0x15/0x20
[  248.089260] register_ftrace_direct+0x31f/0x350
[  248.089445] ? __pfx_bpf_lsm_file_open+0x10/0x10
[  248.089629] bpf_trampoline_update+0x469/0x650
[  248.089814] ? 0xffffffffffffffff
[  248.089988] ? 0xffffffffffffffff
[  248.090153] __bpf_trampoline_link_prog+0x10d/0x330
[  248.090339] bpf_trampoline_link_prog+0x33/0x60
[  248.090518] bpf_tracing_prog_attach+0x3c5/0x5f0
[  248.090699] link_create+0x1a5/0x280
[  248.090886] ? security_bpf+0x3c/0x70
[  248.091101] __sys_bpf+0x4ae/0x10
[  248.091312] __x64_sys_bpf+0x1a/0x30
[  248.091477] x64_sys_call+0x199/0x250
[  248.091647] do_syscall_64+0x7f/0x180
[  248.091818] ? arch_exit_to_user_mode_prepare.isa.0+0x1a/0x60
[  248.092022] ? irqentry_exit_to_user_mode+0x38/0x1e0
[  248.092246] ? irqentry_exit+0x43/0x50
[  248.092401] entry_SYSCALL_64_after_hwframe+0x78/0x80
[  248.092590] RIP: 0033:0x7b53e592728d
[  248.092756] RSP: 002b:00007ffdaa9d696 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
[  248.092856] RAX: ffffffffffffffda RBX: 00007ffdaa9d696 RCX: 00007b53e592728d
[  248.092956] RDX: 0000000000000000 RSI: 00007ffdaa9d696 RDI: 0000000000000001
[  248.093056] RBP: 00007ffdaa9d696 R08: 00007b53e5a03a8 R09: 00007ffdaa9d696
[  248.093156] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[  248.093256] R13: 0000000000000000 R14: 00005d81ed2cfd0 R15: 00005d81ed2b7ec0
[  248.093406] </TASK>

After this patch test Results:
Before fixed: boot time ~50s (with rcu_tasks grace period stall)
After fixed: boot time ~10.7s (systemd-analyze: 724ms kernel + 1.575s initrd + 8.481s userspace = 10.782s)

Reported-by: Challvy Tee <challvy.tee@gmail.com>
Link: https://github.com/systemd/systemd/issues/40423
Fixes: e7e0545299d8 ("x86/sgx: Initialize metadata for Enclave Page Cache (EPC) sections")
Tested-by: Challvy Tee <challvy.tee@gmail.com>
Suggested-by: Kai Huang <kai.huang@intel.com>
Co-developed-by: Fan Du <fan.du@intel.com>
Signed-off-by: Fan Du <fan.du@intel.com>
Signed-off-by: Jun Miao <jun.miao@intel.com>

---
v1 -> v2:
 - Clarify the RCU Tasks stall root cause.
 - Use cond_resched_rcu_qs() following the Kai`s suggestion.
 - Change the subject of this patch to fitness function.
---
 arch/x86/kernel/cpu/sgx/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 4505f808af5e..7ba3d0a5a05d 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -106,7 +106,7 @@ static unsigned long __sgx_sanitize_pages(struct list_head *dirty_page_list)
 			left_dirty++;
 		}
 
-		cond_resched();
+		cond_resched_rcu_qs();
 	}
 
 	list_splice(&dirty, dirty_page_list);
-- 
2.32.0

Re: [PATCH v2] x86/sgx: Fix RCU Tasks stalls in EPC sanitization loop

[kernel test robot]

Hi Jun,

kernel test robot noticed the following build errors:

[auto build test ERROR on linus/master]
[also build test ERROR on tip/x86/sgx v7.2 next-20260817]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Jun-Miao/x86-sgx-Fix-RCU-Tasks-stalls-in-EPC-sanitization-loop/20260815-121253
base:   linus/master
patch link:    https://lore.kernel.org/r/20260623030329.2786571-1-jun.miao%40intel.com
patch subject: [PATCH v2] x86/sgx: Fix RCU Tasks stalls in EPC sanitization loop
config: x86_64-rhel-9.4-bpf (https://download.01.org/0day-ci/archive/20260818/202608180846.b3eJmlOU-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260818/202608180846.b3eJmlOU-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202608180846.b3eJmlOU-lkp@intel.com/

All errors (new ones prefixed by >>):

   arch/x86/kernel/cpu/sgx/main.c: In function '__sgx_sanitize_pages':
>> arch/x86/kernel/cpu/sgx/main.c:109:17: error: implicit declaration of function 'cond_resched_rcu_qs'; did you mean 'cond_resched_lock'? [-Wimplicit-function-declaration]
     109 |                 cond_resched_rcu_qs();
         |                 ^~~~~~~~~~~~~~~~~~~
         |                 cond_resched_lock


vim +109 arch/x86/kernel/cpu/sgx/main.c

    54	
    55	/*
    56	 * Reset post-kexec EPC pages to the uninitialized state. The pages are removed
    57	 * from the input list, and made available for the page allocator. SECS pages
    58	 * prepending their children in the input list are left intact.
    59	 *
    60	 * Return 0 when sanitization was successful or kthread was stopped, and the
    61	 * number of unsanitized pages otherwise.
    62	 */
    63	static unsigned long __sgx_sanitize_pages(struct list_head *dirty_page_list)
    64	{
    65		unsigned long left_dirty = 0;
    66		struct sgx_epc_page *page;
    67		LIST_HEAD(dirty);
    68		int ret;
    69	
    70		/* dirty_page_list is thread-local, no need for a lock: */
    71		while (!list_empty(dirty_page_list)) {
    72			if (kthread_should_stop())
    73				return 0;
    74	
    75			page = list_first_entry(dirty_page_list, struct sgx_epc_page, list);
    76	
    77			/*
    78			 * Checking page->poison without holding the node->lock
    79			 * is racy, but losing the race (i.e. poison is set just
    80			 * after the check) just means __eremove() will be uselessly
    81			 * called for a page that sgx_free_epc_page() will put onto
    82			 * the node->sgx_poison_page_list later.
    83			 */
    84			if (page->poison) {
    85				struct sgx_epc_section *section = &sgx_epc_sections[page->section];
    86				struct sgx_numa_node *node = section->node;
    87	
    88				spin_lock(&node->lock);
    89				list_move(&page->list, &node->sgx_poison_page_list);
    90				spin_unlock(&node->lock);
    91	
    92				continue;
    93			}
    94	
    95			ret = __eremove(sgx_get_epc_virt_addr(page));
    96			if (!ret) {
    97				/*
    98				 * page is now sanitized.  Make it available via the SGX
    99				 * page allocator:
   100				 */
   101				list_del(&page->list);
   102				sgx_free_epc_page(page);
   103			} else {
   104				/* The page is not yet clean - move to the dirty list. */
   105				list_move_tail(&page->list, &dirty);
   106				left_dirty++;
   107			}
   108	
 > 109			cond_resched_rcu_qs();
   110		}
   111	
   112		list_splice(&dirty, dirty_page_list);
   113		return left_dirty;
   114	}
   115	

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki