From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F2D8ECAAA1 for ; Thu, 27 Oct 2022 12:35:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDEBC8E0003; Thu, 27 Oct 2022 08:35:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E90548E0001; Thu, 27 Oct 2022 08:35:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D57138E0003; Thu, 27 Oct 2022 08:35:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C79918E0001 for ; Thu, 27 Oct 2022 08:35:56 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A4E074087A for ; Thu, 27 Oct 2022 12:35:56 +0000 (UTC) X-FDA: 80066676312.27.A5DB1C4 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by imf16.hostedemail.com (Postfix) with ESMTP id 9D6A918000C for ; Thu, 27 Oct 2022 12:35:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666874155; x=1698410155; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=kMIurRzsa691osTcsV/hECDNaAIwmHkC2vJzZbTA8CU=; b=EpTgZsQTcTM1IlcnT+osv/mpgCz9qnzg/XM5pAxwlWrH3mZPIYV92xrD S3GLwM91VEFKoEeUVMGbBrJVIIQrRCW9bCJ7pybSx9iW4Txyxv9TrJJan 7BgmsmbN9wHBXb9zPifGNYOmFlrEuB931KroRTPASt/U3iDUqh84dmZvz CGqwRBWPy2xCZd5FCUMiGmvy89sVjwt18orJ6QOUKqJH3uIiFLAy5sf98 pCemHxGHvinDWuZJcwzWh+4lLL90U1uZj++vTK7W5H3qH8ZNXQowHZcie aXcXO/cF6wpkixW5xJ7deQAkB+5l5GFB77VZu//DJZgF+uVUNiLfGPVzv w==; X-IronPort-AV: E=McAfee;i="6500,9779,10512"; a="309300417" X-IronPort-AV: E=Sophos;i="5.95,217,1661842800"; d="scan'208";a="309300417" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2022 05:35:54 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10512"; a="583539685" X-IronPort-AV: E=Sophos;i="5.95,217,1661842800"; d="scan'208";a="583539685" Received: from akleen-mobl1.amr.corp.intel.com (HELO [10.251.5.115]) ([10.251.5.115]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2022 05:35:53 -0700 Message-ID: <49a6cf62-547f-0f70-3c40-4d7cc077a5c7@linux.intel.com> Date: Thu, 27 Oct 2022 05:35:53 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.4.0 Subject: Re: [PATCH v6 18/21] x86/virt/tdx: Configure global KeyID on all packages Content-Language: en-US To: Kai Huang , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: linux-mm@kvack.org, seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, kirill.shutemov@linux.intel.com, reinette.chatre@intel.com, len.brown@intel.com, tony.luck@intel.com, peterz@infradead.org, isaku.yamahata@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com References: From: Andi Kleen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666874155; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ydNeowuxkR/QHosznOOmYTFeG11h+A08k7EQEDKgVMQ=; b=agZt2IV3oBAJtEUIM9xAHQfPNW0UopvjlU0Xu1T2TGWKYaglyGZnvwByvE7VWAyUvo2Zic wLYPOZqZH97775d6Hdrd16EZxoRynx8vBwUgFhLeLquWZWLI6hgCZak7etc3VI2O0bTGiW WSzzVLHttOjxqPl/dUWfqc/4g4wt6Io= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=EpTgZsQT; spf=none (imf16.hostedemail.com: domain of ak@linux.intel.com has no SPF policy when checking 134.134.136.24) smtp.mailfrom=ak@linux.intel.com; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666874155; a=rsa-sha256; cv=none; b=C3UYUzYCkh2I78J6em98IFo51vt2Xlj7Z10HTp2f5nQ4QLKTMQTlEvfkT7kq0ele2YqFkq Lzga/nRlqxZgLVzXx0+CWUMNpHSFKZCx00/NMaBrkiRTVl+xnEqb0KolH0yTftQlAfACU1 uVzomHcpL/G+anIUNSpc6v0jm8oUTmY= X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9D6A918000C X-Rspam-User: Authentication-Results: imf16.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=EpTgZsQT; spf=none (imf16.hostedemail.com: domain of ak@linux.intel.com has no SPF policy when checking 134.134.136.24) smtp.mailfrom=ak@linux.intel.com; dmarc=fail reason="No valid SPF" header.from=intel.com (policy=none) X-Stat-Signature: 5f17bjo31uueyfe1jo6qtzq63ccg333d X-HE-Tag: 1666874155-880231 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10/26/2022 4:16 PM, Kai Huang wrote: > After the array of TDMRs and the global KeyID are configured to the TDX > module, use TDH.SYS.KEY.CONFIG to configure the key of the global KeyID > on all packages. > > TDH.SYS.KEY.CONFIG must be done on one (any) cpu for each package. And > it cannot run concurrently on different CPUs. Implement a helper to > run SEAMCALL on one cpu for each package one by one, and use it to > configure the global KeyID on all packages. > > Intel hardware doesn't guarantee cache coherency across different > KeyIDs. The kernel needs to flush PAMT's dirty cachelines (associated > with KeyID 0) before the TDX module uses the global KeyID to access the > PAMT. Following the TDX module specification, flush cache before > configuring the global KeyID on all packages. > > Given the PAMT size can be large (~1/256th of system RAM), just use > WBINVD on all CPUs to flush. > > Reviewed-by: Isaku Yamahata > Signed-off-by: Kai Huang > --- > arch/x86/virt/vmx/tdx/tdx.c | 83 ++++++++++++++++++++++++++++++++++++- > arch/x86/virt/vmx/tdx/tdx.h | 1 + > 2 files changed, 82 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c > index fdfce715dda6..9cfb01e7666a 100644 > --- a/arch/x86/virt/vmx/tdx/tdx.c > +++ b/arch/x86/virt/vmx/tdx/tdx.c > @@ -354,6 +354,46 @@ static void seamcall_on_each_cpu(struct seamcall_ctx *sc) > on_each_cpu(seamcall_smp_call_function, sc, true); > } > > +/* > + * Call one SEAMCALL on one (any) cpu for each physical package in > + * serialized way. Return immediately in case of any error if > + * SEAMCALL fails on any cpu. It's not clear what are you serializing against (against itself or other calls of this functions) I assume its because the TDX module errors out for parallel calls instead of waiting. The code seems to only do itself, so where is the check against others? I assume in the callers but that would need to be explained. Also could it need serialization against other kinds of seam calls? Perhaps it might be more efficient to just broad cast and handle a retry with some synchronization in the low level code. That likely would cause less review thrash than just reimplementing a common function like this here. -Andi