From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 37D9D3491D0 for ; Mon, 27 Apr 2026 15:30:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777303805; cv=none; b=pBuN2eh6okox2KzeNg7AqHn2QC33tNtgdgdHI58i2PgE4Iy3egFhnISGMOyTevUf+LPgTMD29adtEc5rTlbmNB3TOrxydWtnBZiWzX+swIz4sc6mkOdH82mVKW/+y3jz+0REOMqy70TTvK4GffeJOdEV/vGjHTwg5GwLLyZoQrU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777303805; c=relaxed/simple; bh=wEwibR+gTdBFd4dJUSNRJHiWPMrn1nUW4D5LVd9Dmt0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RHIMiCD78N5WISmC4XXM0nfLiqa3MVgQv2vQmD6hJUbdAueKUCbxg7uuPqPpZGtREYTJZ2A552a3iiZ3GhV3IIpDiyvi/7aozp+aXbrmKK621kKpp1/2p8HC64HabN1w6BnQi8EX1qRlVPBeZt+BzNs+kPao5JTyic0wsuCTSsI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LxXzpat2; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LxXzpat2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1777303803; x=1808839803; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wEwibR+gTdBFd4dJUSNRJHiWPMrn1nUW4D5LVd9Dmt0=; b=LxXzpat2dAUfDszj7OvIWREcWAY/sc1A8Eq2BZiPpzkBrphYysPzW8GP xNzIffdR5ZiB7dQXZAsuTseuS/Iufjg4WOGFpvIqZ9nXhW6/8sOmDHZAx vUhXjSzHQGH+vQJaAeUHmE5iWNixwO80/k+lsSqtL/EyGXgeo+KaDG4C6 0wqb11xNf+5lczteto71AgLxkLQtWzyXuu/klWmP+xq/L0JqkLOg04Lz4 xA3VZCRVD3eBFYKRwmVBZkel3+iCXhGVyz5AvpVDa7OvtlPNAxYIxdr9Y h3cm0TkIsHkaT7smA/gncSNAvhXVivd4MqrbvSGsg2elu6ZTBq5C5iurv Q==; X-CSE-ConnectionGUID: tt3Dc5MsR6Ggzql1mVqSIw== X-CSE-MsgGUID: Pwuxft74Ro2Nz15HOwFvqQ== X-IronPort-AV: E=McAfee;i="6800,10657,11769"; a="77900716" X-IronPort-AV: E=Sophos;i="6.23,202,1770624000"; d="scan'208";a="77900716" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2026 08:30:00 -0700 X-CSE-ConnectionGUID: LRBEnch9QjyCYa4WFGi3sQ== X-CSE-MsgGUID: Tb7XvmEkRuGY2vYoJ0Jn5w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,202,1770624000"; d="scan'208";a="232673248" Received: from 984fee019967.jf.intel.com ([10.23.153.244]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2026 08:30:00 -0700 From: Chao Gao To: kvm@vger.kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, x86@kernel.org Cc: binbin.wu@linux.intel.com, dave.hansen@linux.intel.com, djbw@kernel.org, ira.weiny@intel.com, kai.huang@intel.com, kas@kernel.org, nik.borisov@suse.com, paulmck@kernel.org, pbonzini@redhat.com, reinette.chatre@intel.com, rick.p.edgecombe@intel.com, sagis@google.com, seanjc@google.com, tony.lindgren@linux.intel.com, vannapurve@google.com, vishal.l.verma@intel.com, yilun.xu@linux.intel.com, xiaoyao.li@intel.com, yan.y.zhao@intel.com, Chao Gao , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" Subject: [PATCH v8 09/21] x86/virt/seamldr: Introduce skeleton for TDX module updates Date: Mon, 27 Apr 2026 08:28:03 -0700 Message-ID: <20260427152854.101171-10-chao.gao@intel.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260427152854.101171-1-chao.gao@intel.com> References: <20260427152854.101171-1-chao.gao@intel.com> Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TDX module updates require careful synchronization with other TDX operations. The requirements are (#1/#2 reflect current behavior that must be preserved): 1. SEAMCALLs need to be callable from both process and IRQ contexts. 2. SEAMCALLs need to be able to run concurrently across CPUs 3. During updates, only update-related SEAMCALLs are permitted; all other SEAMCALLs shouldn't be called. 4. During updates, all online CPUs must participate in the update work. No single lock primitive satisfies all requirements. For instance, rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled cannot be directed to perform update work. Use stop_machine() as it is the only well-understood mechanism that can meet all requirements. And TDX module updates consist of several steps (See Intel® Trust Domain Extensions (Intel® TDX) Module Base Architecture Specification, Chapter "TD-Preserving TDX module Update"). Ordering requirements between steps mandate lockstep synchronization across all CPUs. multi_cpu_stop() is a good example of performing a multi-step task in lockstep. But it doesn't synchronize steps within the callback function it takes. So, implement one based on its pattern to establish the skeleton for TDX module updates. Specifically, add a global state machine where each state represents a step in the update flow. The state advances only after all CPUs acknowledge completing their work in the current state. This acknowledgment mechanism is what ensures lockstep execution. Potential alternative to stop_machine() ======================================= An alternative approach is to lock all KVM entry points and kick all vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points, implemented in KVM common code (virt/kvm). Adding a locking mechanism there would affect all architectures KVM supports. And to lock only TDX vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM common code currently lacks. This would add significant complexity and maintenance overhead to KVM for this TDX-specific use case, so don't take this approach. Signed-off-by: Chao Gao Reviewed-by: Xu Yilun Reviewed-by: Tony Lindgren Reviewed-by: Kai Huang Reviewed-by: Kiryl Shutsemau (Meta) Reviewed-by: Rick Edgecombe --- v8: - Add a "so don't take this approach" after alternative solution discussion in the changelog [Rick] - Use imperative mood for a comment [Dave] --- arch/x86/virt/vmx/tdx/seamldr.c | 79 ++++++++++++++++++++++++++++++++- 1 file changed, 77 insertions(+), 2 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c index f70be8e2a07b..aa839aaeb79d 100644 --- a/arch/x86/virt/vmx/tdx/seamldr.c +++ b/arch/x86/virt/vmx/tdx/seamldr.c @@ -7,8 +7,10 @@ #define pr_fmt(fmt) "seamldr: " fmt #include +#include #include #include +#include #include @@ -190,6 +192,77 @@ static struct seamldr_params *init_seamldr_params(const u8 *data, u32 size) return alloc_seamldr_params(blob, size); } +/* + * During a TDX module update, all CPUs start from MODULE_UPDATE_START and + * progress to MODULE_UPDATE_DONE. Each state is associated with certain + * work. For some states, just one CPU needs to perform the work, while + * other CPUs just wait during those states. + */ +enum module_update_state { + MODULE_UPDATE_START, + MODULE_UPDATE_DONE, +}; + +static struct { + enum module_update_state state; + int thread_ack; + /* + * Protect update_data. Raw spinlock as it will be acquired from + * interrupt-disabled contexts. + */ + raw_spinlock_t lock; +} update_data = { + .lock = __RAW_SPIN_LOCK_UNLOCKED(update_data.lock) +}; + +static void set_target_state(enum module_update_state state) +{ + /* Reset ack counter. */ + update_data.thread_ack = num_online_cpus(); + update_data.state = state; +} + +/* Last one to ack a state moves to the next state. */ +static void ack_state(void) +{ + guard(raw_spinlock)(&update_data.lock); + update_data.thread_ack--; + if (!update_data.thread_ack) + set_target_state(update_data.state + 1); +} + +/* + * See multi_cpu_stop() from where this multi-cpu state-machine was + * adopted, and the rationale for touch_nmi_watchdog(). + */ +static int do_seamldr_install_module(void *seamldr_params) +{ + enum module_update_state newstate, curstate = MODULE_UPDATE_START; + int ret = 0; + + do { + /* Chill out and re-read update_data. */ + cpu_relax(); + newstate = READ_ONCE(update_data.state); + + if (newstate != curstate) { + curstate = newstate; + switch (curstate) { + /* TODO: add the update steps. */ + default: + break; + } + + ack_state(); + } else { + touch_nmi_watchdog(); + rcu_momentary_eqs(); + } + } while (curstate != MODULE_UPDATE_DONE); + + return ret; +} + DEFINE_FREE(free_seamldr_params, struct seamldr_params *, if (!IS_ERR_OR_NULL(_T)) free_page((unsigned long)_T)) @@ -207,7 +280,9 @@ int seamldr_install_module(const u8 *data, u32 size) if (IS_ERR(params)) return PTR_ERR(params); - /* TODO: Update TDX module here */ - return 0; + /* Ensure a stable set of online CPUs for the update process. */ + guard(cpus_read_lock)(); + set_target_state(MODULE_UPDATE_START + 1); + return stop_machine_cpuslocked(do_seamldr_install_module, params, cpu_online_mask); } EXPORT_SYMBOL_FOR_MODULES(seamldr_install_module, "tdx-host"); -- 2.47.1