From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0174F3290AD for ; Thu, 16 Jul 2026 14:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784213742; cv=none; b=qG7R1DPPL1V2Ygnf7GIjKn+aReGjeiXuJy3Yx/t7xpiy1G1VnhUee5aKKSpVcKVGU4gs1Zx3plpxtmhGKvM5KUu+fAqiPYVLuX6202M/qdyVpucJRnNvv3hp1GtXn3VILp+SHsuiOjDnxo9WDZDNUa4qpQ8XSOO8FUcadMy0bd0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784213742; c=relaxed/simple; bh=2w8HiNG2AtKnE5EBD3uoew04RxAmSAbTK8nBUgNjM3s=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=cdjQJOAVY+fXj+XcMbPo0hwudULI3SP9G+tVqwQE0FsiAywJQW/XNNkoquxssd1Vd8iRYDjNkRaDIZAkHrPI/AxK1MBexMd73NQYY0kz5H2cPbiDI1rZY216iCunDHe0E3DoustwQHNN4lXT9TfxunnRZNKTraIlHZOFYa22eU0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com; spf=pass smtp.mailfrom=soleen.com; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b=f83jKS2x; arc=none smtp.client-ip=209.85.128.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=soleen.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="f83jKS2x" Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-8114a4542b2so7509327b3.1 for ; Thu, 16 Jul 2026 07:55:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1784213740; x=1784818540; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=ldlKJTt83OroR2A75ml0k59gQsXf+w2G0f8AwORm5GQ=; b=f83jKS2xYqZuvnr+hObFmCc+AwxN5s4MgH0vdnPTI+NSCccMaDUBE94txBqkMg47A2 Oxy0jx/YRlj0+LsJzHLwQfU9xUvmLdbEtRajVtRpeAifZ1vdrtlOK1T7VAWk7M3KYKsT JSEhG80qdUMkp6x/AUfBptxJRI9OaDv5gI6Ywsr+FgyYefsuds9iCbWkWe0Out6vhyqw qAXRBcTSVS9sQAZ4Mdul9OxA8+MwRIEst9sG7Y+L7M0FunMzq7iDDL9te6mEdNBjwvDd 0L1yPR5EIKDOshBobs0A93DVz+7Bm2NGUUHTC3eWhmajPRkwbqDVEnqOn/cmQRwd3DdN 0gag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784213740; x=1784818540; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ldlKJTt83OroR2A75ml0k59gQsXf+w2G0f8AwORm5GQ=; b=bL4AWI4uxsPvPo6Z+Rp5YFzt+8AzFGutOAfbpO8StHO4C7GENS17Vj6KWwE78bkpmF KJiRmo6RROhqK2pf2ENHDXLxmnRg9cg69ejBmdJO0oCtJQReJNcvJU7O/0EzOX621XRq YLTgSSbFUsJgZPo/yVaMKLlipOrW59N5jug5xtYncsmX3HuioLmOuN/iXc2sKrWZGCsn UfOd93SWNhmxtfaGD8TFsyrpDBn/OM1f76OdF5mveNGpNq5K3EPWH3vKiq0J/bL1Vr02 QNwNmPIo0GfIc/xzHRcgimVDWz1+BiJqO3Fvvc4a0/xkbJYRuSvCcRfcQ1eZt5H2Kaq2 HB2w== X-Forwarded-Encrypted: i=1; AHgh+Rp/XTpLcaJtWOVEuKK74wcnvSocs73aWLqMv//kSm2iheorbNN1/IN1Zy6BS8JSUS02jF3gEOqM0Yoeb9A=@vger.kernel.org X-Gm-Message-State: AOJu0YxZ9bqNWSfzt6GOEQBWPsGAwtIaDhmtYc+cu0HvpMHme0M8E9yp 7JL+BjBrOJW5Ij5QyCdZPE+qNouhMJRlwWom6ySK9Sc5XktPlPkuvQC19aYp73eFxLw= X-Gm-Gg: AfdE7cnPvX+aw0Ja0SKoAYnkPuSmr/griEGsPPc4kJ1lNeBW/3Q1lWiraLdIq7RnyGk Tm+ICQ5SFhTHXeLt43jN0tvBjIoEQCqF+CZGWii+1dwF76ePR30TpElTarJ7poDdvnK4IGqxJdX M1yE2wfqHbdrEjoO+B+wBHm05SRUnLuohs3cbNfXsVYz8Ghwx92ypYALsodq8MdzwMAFacHJ25r FzkZNkZLKg8xF4WqmORhEL8t+Vlq3UPIGOteqY6YZhwbSNNU5Iar5UcDKoyAe7KjnOiinFufiD+ ABvM15lq2o+vlZ/IQrPV41UVCqd2yD80NYcbIbW/HbryknRoa9rK07g8CBB8UYcVO9pIohLfATO zHbPi+FwXZTiziueWYCY1xv6UsKr04wEW9UulXsYdO6tTzgGkt9LZPf+VYEqeMkx99n4wenuHru D8MnTnRCGoC9eV0fOR0ssmQvzhmjKWp+5YYsnWQ5Z5ks4VQqRf5LUWcr598/likS5qUqbPxPaXa VXv7zOf9vK4 X-Received: by 2002:a05:690c:688e:b0:81d:ff0a:2cfe with SMTP id 00721157ae682-81ecf776a35mr56530767b3.8.1784213739802; Thu, 16 Jul 2026 07:55:39 -0700 (PDT) Received: from google.com (138.200.150.34.bc.googleusercontent.com. [34.150.200.138]) by smtp.gmail.com with ESMTPSA id 00721157ae682-81ed77e8ca7sm35824487b3.1.2026.07.16.07.55.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jul 2026 07:55:39 -0700 (PDT) Date: Thu, 16 Jul 2026 10:55:38 -0400 From: Pasha Tatashin To: Sagi Shahar Cc: Pasha Tatashin , Alexander Graf , "Andersen, Tycho" , Anthony Yznaga , Baolu Lu , David Hildenbrand , David Matlack , "Heyne, Maximillian" , James Gowans , Jason Gunthorpe , Mike Rapoport , Pankaj Gupta , Pratyush Yadav , Praveen Kumar , Vipin Sharma , Vishal Annapurve , "Woodhouse, David" , Luca Boccassi , Samiullah Khawaja , Jork Loeser , Chenghao Duan , Zhu Yanjun , Sean Christopherson , "Verma, Vishal L" , linux-mm@kvack.org, kexec@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: Preserving TDX module during kexec with live update Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On 07-15 18:28, Sagi Shahar wrote: > Hi, > > I started looking into preserving the TDX module across live updates > and I'm not sure what the best approach is. I tried looking into using > flb as David M suggested but I don't think this approach works for the > TDX module. > > Right now the way that TDX supports kexec is by disabling the TDX > module on kexec and then reinitilizing it on the next boot [1]. This > approach clears the TDX module state so any TDX VM from the source > kernel won't be able to resume running on the destination kernel. > > In order to support live update while keeping the VMs alive we need to > skip the TDH_SYS_DISABLE call and instead preserve pages used by the > TDX module such as the TDMR pages from the source kernel. Agreed. > > From what I understand, using flb will tie the TDX module save/restore > to the lifecycle of the VMs depending on it. This means the module > will be restored too late during kernel boot, after the VMs are > restored using the live update IOCTLs. That is not correct. FLBs specifically can be restored very early in boot. We do that for HugeTLB preservation, for example. liveupdate_flb_get_incoming() does not require userspace to resume or restore VMs. You can call it super early in boot to see if there is FLB data present and, if so, change the way the module is initialized during boot. > > I'm thinking we should preserve the TDX module during the shutdown > stage of the source kernel. Perhaps we can preserve the pages used by > the TDX module as we allocate them on the source kernel and pass > pointers and additional metadata during KHO. Then on the destination > we need to pipe this information to where the TDX module gets > initialized during the KVM module initialization. Please don't invent another state machine; instead, use the existing one. Pasha > > I haven't found a device that acts in a similar way so I'm not sure > how complicated this will be or if there's a better approach. > > I would love to hear your thoughts and if you have any pointers on > where this code can be implemented. > > Thanks, > Sagi > > [1] https://lore.kernel.org/all/20260323-fuller_tdx_kexec_support-v2-0-87a36409e051@intel.com/