* [PATCH v2 0/2] KVM: x86: gmem populate fix and cleanups
From: Sean Christopherson @ 2026-06-30 21:37 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: Dave Hansen, Rick Edgecombe, kvm, x86, linux-coco, linux-kernel,
Sashiko Bot, Joerg Roedel, Yan Zhao, Ackerley Tng
Fix a user-triggerable WARN due to KVM not pre-checking that userspace
provided a source page for non-ZERO pages for SNP_LAUNCH_UPDATE, and then
clean up the equivalent TDX code to also explicitly check the incoming
source page *before* calling into guest_memfd, and to return -EINVAL, not
-EOPNOTSUPP.
v2:
- Rewrite the SNP patch changelog.
- Tweak the code to avoid checking KVM_SEV_SNP_PAGE_TYPE_ZERO twice.
- Drop what is now effectively a sanity check in sev_gmem_post_populate(),
so that we don't have to duplicate the logic when in-place conversion comes
along.
- Tack on the TDX change.
v1: https://lore.kernel.org/all/20260623091556.1500930-2-joro@8bytes.org
Joerg Roedel (1):
KVM: SEV: Explicitly disallow NULL user address for SNP_LAUNCH_UPDATE
Sean Christopherson (1):
KVM: TDX: Return EINVAL, not EOPNOTSUPP, for NULL INIT_MEM_REGION
source
arch/x86/kvm/svm/sev.c | 11 +++++------
arch/x86/kvm/vmx/tdx.c | 7 ++-----
2 files changed, 7 insertions(+), 11 deletions(-)
base-commit: a204badd8432f93b7e862e7dac6db0fe3d65f370
--
2.55.0.rc0.799.gd6f94ed593-goog
^ permalink raw reply
* [PATCH v2 1/2] KVM: SEV: Explicitly disallow NULL user address for SNP_LAUNCH_UPDATE
From: Sean Christopherson @ 2026-06-30 21:37 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: Dave Hansen, Rick Edgecombe, kvm, x86, linux-coco, linux-kernel,
Sashiko Bot, Joerg Roedel, Yan Zhao, Ackerley Tng
In-Reply-To: <20260630213711.479692-1-seanjc@google.com>
From: Joerg Roedel <joerg.roedel@amd.com>
Explicitly reject a NULL userspace virtual address for the source page of
SNP_LAUNCH_UPDATE instead of relying on the post-populate callback to do
the check, and don't WARN on failure, as the scenario is blatantly user-
triggerable, as reported by Sashiko. Waiting until post-populate to check
the address "works", but makes it unnecessarily difficult to see that KVM's
ABI is to disallow a NULL source page for non-ZERO pages.
Note, several existing VMMs pass a valid userspace address for the ZERO
case, i.e. KVM can't *require* the userspace address to be NULL for ZERO
pages, at least not without breaking userspace.
Fixes: dee5a47cc7a4 ("KVM: SEV: Add KVM_SEV_SNP_LAUNCH_UPDATE command")
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260611125849.9ED631F00893@smtp.kernel.org
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/svm/sev.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 74fb15551e83..621a2eaa58f2 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -2330,9 +2330,6 @@ static int sev_gmem_post_populate(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
int level;
int ret;
- if (WARN_ON_ONCE(sev_populate_args->type != KVM_SEV_SNP_PAGE_TYPE_ZERO && !src_page))
- return -EINVAL;
-
ret = snp_lookup_rmpentry((u64)pfn, &assigned, &level);
if (ret || assigned) {
pr_debug("%s: Failed to ensure GFN 0x%llx RMP entry is initial shared state, ret: %d assigned: %d\n",
@@ -2421,10 +2418,12 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
params.type != KVM_SEV_SNP_PAGE_TYPE_CPUID))
return -EINVAL;
- src = params.type == KVM_SEV_SNP_PAGE_TYPE_ZERO ? NULL : u64_to_user_ptr(params.uaddr);
-
- if (!PAGE_ALIGNED(src))
+ if (params.type == KVM_SEV_SNP_PAGE_TYPE_ZERO)
+ src = NULL;
+ else if (!params.uaddr || !PAGE_ALIGNED(params.uaddr))
return -EINVAL;
+ else
+ src = u64_to_user_ptr(params.uaddr);
npages = params.len / PAGE_SIZE;
--
2.55.0.rc0.799.gd6f94ed593-goog
^ permalink raw reply related
* [PATCH v2 2/2] KVM: TDX: Return EINVAL, not EOPNOTSUPP, for NULL INIT_MEM_REGION source
From: Sean Christopherson @ 2026-06-30 21:37 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: Dave Hansen, Rick Edgecombe, kvm, x86, linux-coco, linux-kernel,
Sashiko Bot, Joerg Roedel, Yan Zhao, Ackerley Tng
In-Reply-To: <20260630213711.479692-1-seanjc@google.com>
Return EINVAL instead of EOPNOTSUPP if userspace attempts to pass a NULL
pointer for the source page of INIT_MEM_REGION, so that KVM's ABI is
consistent between TDX and SNP (for LAUNCH_UPDATE). EOPNOTSUPP was chosen
to be a forward-looking error code for when guest_memfd supports in-place
conversion, but even when in-place conversion comes along, it's an awkward
error code as KVM is deliberately choosing to disallow virtual address '0',
which is technically a legal userspace address. I.e. it's not so much a
lack of support as it is that KVM reserves address '0' to simplify KVM's
internal implementation.
Opportunistically move the check so that it's co-located with the other
checks on the userspace address, and so that it's more obvious that a NULL
source address is explicitly disallowed.
Fixes: 2a62345b3052 ("KVM: guest_memfd: GUP source pages prior to populating guest memory")
Cc: Yan Zhao <yan.y.zhao@intel.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/tdx.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index ffe9d0db58c5..b0ec054732b9 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -3198,9 +3198,6 @@ static int tdx_gmem_post_populate(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
if (KVM_BUG_ON(kvm_tdx->page_add_src, kvm))
return -EIO;
- if (!src_page)
- return -EOPNOTSUPP;
-
kvm_tdx->page_add_src = src_page;
ret = kvm_tdp_mmu_map_private_pfn(arg->vcpu, gfn, pfn);
kvm_tdx->page_add_src = NULL;
@@ -3247,8 +3244,8 @@ static int tdx_vcpu_init_mem_region(struct kvm_vcpu *vcpu, struct kvm_tdx_cmd *c
if (copy_from_user(®ion, u64_to_user_ptr(cmd->data), sizeof(region)))
return -EFAULT;
- if (!PAGE_ALIGNED(region.source_addr) || !PAGE_ALIGNED(region.gpa) ||
- !region.nr_pages ||
+ if (!PAGE_ALIGNED(region.source_addr) || !region.source_addr ||
+ !PAGE_ALIGNED(region.gpa) || !region.nr_pages ||
region.gpa + (region.nr_pages << PAGE_SHIFT) <= region.gpa ||
!vt_is_tdx_private_gpa(kvm, region.gpa) ||
!vt_is_tdx_private_gpa(kvm, region.gpa + (region.nr_pages << PAGE_SHIFT) - 1))
--
2.55.0.rc0.799.gd6f94ed593-goog
^ permalink raw reply related
* Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
From: Edgecombe, Rick P @ 2026-06-30 23:33 UTC (permalink / raw)
To: seanjc@google.com, Fang, Peter
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Li, Xiaoyao,
Hansen, Dave, dave.hansen@linux.intel.com,
baolu.lu@linux.intel.com, Hunter, Adrian, kas@kernel.org,
linux-kernel@vger.kernel.org, Xu, Yilun,
tony.lindgren@linux.intel.com, Mehta, Sohil, Duan, Zhenzhong,
djbw@kernel.org, Maloor, Kishen, yilun.xu@linux.intel.com,
x86@kernel.org
In-Reply-To: <akMQ3J6fK3DUBsPH@google.com>
On Mon, 2026-06-29 at 17:42 -0700, Sean Christopherson wrote:
> Answering my own question (though probably poorly), IIUC the answer is that
> DICE-based quoting is done through the TDX Module, whereas existing quoting is
> done through an SGX enclave and so was routed through userspace.
>
> If that's all there is too this, then why is KVM involved? I.e. why doesn't the
> TDX Module provide the quote directly to the guest?
That is a good question. The answer is partly historical reasons, but I think
the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side
TDVMCALL quoting interface can stay the same. There is also a wrinkle in that
there is a limited HW resource involved in the quoting, so we want to do these
operations one at a time. Having a mutex on the host is the KISS way of
accomplishing some level of fairness for DOS prevention.
We should've explained this more, but TBH this solution is *way* simpler than
the initial one that never saw the light of day. So this extra host work seemed
quite small compared to what we have been staring at and we kinda overlooked it.
The other relevant tidbit is that the TDX module folks have some problems to
solve before they can support TDG calls to TDX module extensions. I think we can
get them to though. The question is probably really: do we want the guest
trying/selecting multiple interfaces, or the host.
^ permalink raw reply
* Re: [PATCH v4 2/3] x86/insn-eval: Add insn_assign_reg() helper
From: Edgecombe, Rick P @ 2026-07-01 0:06 UTC (permalink / raw)
To: mingo@redhat.com, kas@kernel.org, tglx@kernel.org, bp@alien8.de,
dave.hansen@linux.intel.com
Cc: linux-kernel@vger.kernel.org, ak@linux.intel.com,
seanjc@google.com, binbin.wu@linux.intel.com, Li, Xiaoyao,
sathyanarayanan.kuppuswamy@linux.intel.com,
david.laight.linux@gmail.com, Huang, Kai, pbonzini@redhat.com,
kvm@vger.kernel.org, tsyrulnikov.borys@gmail.com, djbw@kernel.org,
linux-coco@lists.linux.dev, x86@kernel.org
In-Reply-To: <daac026e677c10b68740b16ed8ad2556bd9583f8.1780584300.git.kas@kernel.org>
On Thu, 2026-06-04 at 15:47 +0100, Kiryl Shutsemau (Meta) wrote:
> KVM's instruction emulator has a small helper, assign_register(), that
> writes a value into a sub-register with x86 partial-register-write
> semantics: 1- and 2-byte writes leave the upper bits of the destination
> untouched, 4-byte writes zero-extend to 64 bits, 8-byte writes overwrite
> the full register.
>
> The TDX guest #VE handler needs the same logic for port I/O emulation
> to get 32-bit zero-extension right. Rather than copy-pasting the helper,
> lift it to <asm/insn-eval.h> as insn_assign_reg() so both can use it.
>
> Rewrite the body using arithmetic instead of pointer punning so the
> helper does not depend on -fno-strict-aliasing or little-endian byte
> order, and add <asm/insn.h> to the header's includes so it builds
> standalone in callers that have not pulled it in transitively.
>
> No functional change.
>
> Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
We should probably ping Sean/Paolo to ack this if we want Dave to take it. Is it
the plan?
^ permalink raw reply
* Re: [PATCH v4 1/3] x86/tdx: Fix off-by-one in port I/O handling
From: Edgecombe, Rick P @ 2026-07-01 0:06 UTC (permalink / raw)
To: mingo@redhat.com, kas@kernel.org, tglx@kernel.org, bp@alien8.de,
dave.hansen@linux.intel.com
Cc: linux-kernel@vger.kernel.org, ak@linux.intel.com,
seanjc@google.com, binbin.wu@linux.intel.com,
stable@vger.kernel.org, Li, Xiaoyao,
sathyanarayanan.kuppuswamy@linux.intel.com,
david.laight.linux@gmail.com, Huang, Kai, pbonzini@redhat.com,
kvm@vger.kernel.org, tsyrulnikov.borys@gmail.com, djbw@kernel.org,
linux-coco@lists.linux.dev, x86@kernel.org
In-Reply-To: <e5a75bb68a6a778c95cac2ef77acd55cfd24d389.1780584300.git.kas@kernel.org>
On Thu, 2026-06-04 at 15:46 +0100, Kiryl Shutsemau (Meta) wrote:
> handle_in() and handle_out() in arch/x86/coco/tdx/tdx.c use:
>
> u64 mask = GENMASK(BITS_PER_BYTE * size, 0);
>
> GENMASK(h, l) includes bit h. For size=1 (INB), this produces
> GENMASK(8, 0) = 0x1FF (9 bits) instead of GENMASK(7, 0) = 0xFF (8
> bits). The mask is one bit too wide for all I/O sizes.
>
> Fix the mask calculation.
>
> Fixes: 03149948832a ("x86/tdx: Port I/O: Add runtime hypercalls")
> Reported-by: Borys Tsyrulnikov <tsyrulnikov.borys@gmail.com>
> Link: https://lore.kernel.org/all/CAKw_Dz96rfSQc6Rn+9QBcUFHhmkK+9zu+P=bxowfZwxrATCBRg@mail.gmail.com/
> Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
> Cc: stable@vger.kernel.org
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
^ permalink raw reply
* Re: [PATCH v6 01/11] x86/virt/tdx: Simplify tdmr_get_pamt_sz()
From: Edgecombe, Rick P @ 2026-07-01 0:08 UTC (permalink / raw)
To: kas@kernel.org
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Hansen, Dave, Zhao, Yan Y, linux-kernel@vger.kernel.org,
seanjc@google.com, mingo@redhat.com, pbonzini@redhat.com,
binbin.wu@linux.intel.com, nik.borisov@suse.com,
linux-doc@vger.kernel.org, hpa@zytor.com, tglx@kernel.org,
Annapurve, Vishal, bp@alien8.de, Gao, Chao, x86@kernel.org
In-Reply-To: <aiGheH8YceumotUU@thinkstation>
On Thu, 2026-06-04 at 17:05 +0100, Kiryl Shutsemau wrote:
> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Thanks.
>
> Couple of nits below.
Yep, thanks.
^ permalink raw reply
* Re: [PATCH v6 02/11] x86/virt/tdx: Allocate page bitmap for Dynamic PAMT
From: Edgecombe, Rick P @ 2026-07-01 0:14 UTC (permalink / raw)
To: kas@kernel.org
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Hansen, Dave, Zhao, Yan Y, kirill.shutemov@linux.intel.com,
linux-kernel@vger.kernel.org, seanjc@google.com, mingo@redhat.com,
pbonzini@redhat.com, binbin.wu@linux.intel.com,
nik.borisov@suse.com, linux-doc@vger.kernel.org, hpa@zytor.com,
tglx@kernel.org, Annapurve, Vishal, bp@alien8.de, Gao, Chao,
x86@kernel.org
In-Reply-To: <aiGi36_YpJBRechp@thinkstation>
On Thu, 2026-06-04 at 17:14 +0100, Kiryl Shutsemau wrote:
> On Mon, May 25, 2026 at 07:35:06PM -0700, Rick Edgecombe wrote:
> > @@ -579,7 +591,12 @@ static __init int tdmr_set_up_pamt(struct tdmr_info *tdmr,
> > * Calculate the PAMT size for each TDX supported page size
> > * and the total PAMT size.
> > */
> > - tdmr->pamt_4k_size = tdmr_get_pamt_sz(tdmr, TDX_PS_4K);
> > + if (tdx_supports_dynamic_pamt(&tdx_sysinfo)) {
> > + /* With Dynamic PAMT, PAMT_4K is replaced with a bitmap */
> > + tdmr->pamt_4k_size = tdmr_get_pamt_bitmap_sz(tdmr);
> > + } else {
> > + tdmr->pamt_4k_size = tdmr_get_pamt_sz(tdmr, TDX_PS_4K);
> > + }
> > tdmr->pamt_2m_size = tdmr_get_pamt_sz(tdmr, TDX_PS_2M);
> > tdmr->pamt_1g_size = tdmr_get_pamt_sz(tdmr, TDX_PS_1G);
> > tdmr_pamt_size = tdmr->pamt_4k_size + tdmr->pamt_2m_size + tdmr->pamt_1g_size;
>
> Maybe it would more readable if we reverse the size order:
>
> /*
> * Calculate the PAMT size for each TDX supported page size
> * and the total PAMT size.
> */
> tdmr->pamt_1g_size = tdmr_get_pamt_sz(tdmr, TDX_PS_1G);
> tdmr->pamt_2m_size = tdmr_get_pamt_sz(tdmr, TDX_PS_2M);
>
> if (tdx_supports_dynamic_pamt(&tdx_sysinfo)) {
> /* With Dynamic PAMT, PAMT_4K is replaced with a bitmap */
> tdmr->pamt_4k_size = tdmr_get_pamt_bitmap_sz(tdmr);
> } else {
> tdmr->pamt_4k_size = tdmr_get_pamt_sz(tdmr, TDX_PS_4K);
> }
>
> tdmr_pamt_size = tdmr->pamt_1g_size + tdmr->pamt_2m_size + tdmr->pamt_4k_size;
>
> It allows split it into logical blocks while keeping the comment attached.
Uhh, yea. I can see it a little. I'll swap it.
^ permalink raw reply
* Re: [PATCH v6 03/11] x86/virt/tdx: Add tdx_alloc/free_control_page() helpers
From: Edgecombe, Rick P @ 2026-07-01 0:15 UTC (permalink / raw)
To: Zhao, Yan Y, binbin.wu@linux.intel.com
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Hansen, Dave, kirill.shutemov@linux.intel.com, kas@kernel.org,
seanjc@google.com, mingo@redhat.com, linux-kernel@vger.kernel.org,
pbonzini@redhat.com, nik.borisov@suse.com,
linux-doc@vger.kernel.org, hpa@zytor.com, tglx@kernel.org,
Annapurve, Vishal, bp@alien8.de, Gao, Chao, x86@kernel.org
In-Reply-To: <aiYmkfHBDHlqQI/d@yzhao56-desk.sh.intel.com>
On Mon, 2026-06-08 at 10:18 +0800, Yan Zhao wrote:
> On Mon, Jun 08, 2026 at 10:11:58AM +0800, Binbin Wu wrote:
> > > diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> > > index 82dc27aecf297..74e75db5728c7 100644
> > > --- a/arch/x86/include/asm/tdx.h
> > > +++ b/arch/x86/include/asm/tdx.h
> > > @@ -37,6 +37,7 @@
> > >
> > > #include <uapi/asm/mce.h>
> > > #include <asm/tdx_global_metadata.h>
> > > +#include <linux/mm.h>
> >
> > I think the header is not needed here.
> Right. This version does not invoke page_address() in tdx.h for
> tdx_alloc_control_page() any more.
>
> Also no need to include mm.h for tdx.c (which has invoked page_address() before
> this patch), since tdx.c includes memblock.h which further includes mm.h.
Thanks for the analysis. Will remove both.
^ permalink raw reply
* Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
From: Dan Williams (nvidia) @ 2026-07-01 0:24 UTC (permalink / raw)
To: Edgecombe, Rick P, seanjc@google.com, Fang, Peter
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Li, Xiaoyao,
Hansen, Dave, dave.hansen@linux.intel.com,
baolu.lu@linux.intel.com, Hunter, Adrian, kas@kernel.org,
linux-kernel@vger.kernel.org, Xu, Yilun,
tony.lindgren@linux.intel.com, Mehta, Sohil, Duan, Zhenzhong,
djbw@kernel.org, Maloor, Kishen, yilun.xu@linux.intel.com,
x86@kernel.org
In-Reply-To: <23a9173f6e278ca7dfedce3374626c6ea3e1b47a.camel@intel.com>
Edgecombe, Rick P wrote:
> On Mon, 2026-06-29 at 17:42 -0700, Sean Christopherson wrote:
> > Answering my own question (though probably poorly), IIUC the answer is that
> > DICE-based quoting is done through the TDX Module, whereas existing quoting is
> > done through an SGX enclave and so was routed through userspace.
> >
> > If that's all there is too this, then why is KVM involved? I.e. why doesn't the
> > TDX Module provide the quote directly to the guest?
>
> That is a good question. The answer is partly historical reasons, but I think
> the pros/cons don’t really move the needle too much.
>
> The main benefit of doing it with the host in the loop is that the guest side
> TDVMCALL quoting interface can stay the same. There is also a wrinkle in that
> there is a limited HW resource involved in the quoting, so we want to do these
> operations one at a time. Having a mutex on the host is the KISS way of
> accomplishing some level of fairness for DOS prevention.
>
> We should've explained this more, but TBH this solution is *way* simpler than
> the initial one that never saw the light of day. So this extra host work seemed
> quite small compared to what we have been staring at and we kinda overlooked it.
>
> The other relevant tidbit is that the TDX module folks have some problems to
> solve before they can support TDG calls to TDX module extensions. I think we can
> get them to though. The question is probably really: do we want the guest
> trying/selecting multiple interfaces, or the host.
tl;dr: FWIW, host, if only because this is one of many blob transports
across multiple archs that need host coordination.
While TDG calls to do the same helps a TDX problem it does not generally
reduce the Linux problem of multiple archs having multiple blob protocols
to shuffle data from host to guest in shared memory.
The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI
attestation collateral) is shift the burden from arch/*/kvm/ to
drivers/. There might be a later opportunity to consolidate some of
those drivers' internals on a cross-arch generic transport (AF_VSOCK),
but there remains a requirement to talk to the host.
^ permalink raw reply
* Re: [PATCH v4 2/3] x86/insn-eval: Add insn_assign_reg() helper
From: Borislav Petkov @ 2026-07-01 0:28 UTC (permalink / raw)
To: Edgecombe, Rick P
Cc: mingo@redhat.com, kas@kernel.org, tglx@kernel.org,
dave.hansen@linux.intel.com, linux-kernel@vger.kernel.org,
ak@linux.intel.com, seanjc@google.com, binbin.wu@linux.intel.com,
Li, Xiaoyao, sathyanarayanan.kuppuswamy@linux.intel.com,
david.laight.linux@gmail.com, Huang, Kai, pbonzini@redhat.com,
kvm@vger.kernel.org, tsyrulnikov.borys@gmail.com, djbw@kernel.org,
linux-coco@lists.linux.dev, x86@kernel.org
In-Reply-To: <a038a3dabd917b0da28059fc7b0116996b099868.camel@intel.com>
On Wed, Jul 01, 2026 at 12:06:33AM +0000, Edgecombe, Rick P wrote:
> We should probably ping Sean/Paolo to ack this if we want Dave to take it. Is it
> the plan?
Not before Sashiko comments have been addressed:
https://sashiko.dev/#/patchset/cover.1780584300.git.kas%40kernel.org
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply
* Re: [PATCH v6 06/11] x86/virt/tdx: Optimize tdx_pamt_get/put()
From: Edgecombe, Rick P @ 2026-07-01 1:05 UTC (permalink / raw)
To: Hansen, Dave, kas@kernel.org, Gao, Chao
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Zhao, Yan Y, linux-kernel@vger.kernel.org, seanjc@google.com,
mingo@redhat.com, pbonzini@redhat.com, nik.borisov@suse.com,
linux-doc@vger.kernel.org, hpa@zytor.com, tglx@kernel.org,
Annapurve, Vishal, bp@alien8.de, kirill.shutemov@linux.intel.com,
x86@kernel.org
In-Reply-To: <572868d7-4794-4fec-b80f-97d8434d5fb6@intel.com>
On Fri, 2026-06-05 at 09:23 -0700, Dave Hansen wrote:
> On 6/5/26 04:42, Kiryl Shutsemau wrote:
> > > > I don't see a reason why we can't keep the scoped_guard() on get side.
> > > One additional reason to drop scoped_guard() is that it mixes cleanup helpers
> > > with goto, which is discouraged. See [*]
> > >
> > > :Lastly, given that the benefit of cleanup helpers is removal of “goto”, and
> > > :that the “goto” statement can jump between scopes, the expectation is that
> > > :usage of “goto” and cleanup helpers is never mixed in the same function.
> > Fair enough.
> >
> > But it can also be address if we free the PAMT page array with the guard
> > too :P
>
> How important is this patch? I see "Optimize" but I read "Optional".
>
> If we're arguing about it, maybe we should just kick it out and focus on
> the more important bits.
I had done some testing previously to see if the refcount solution avoided
contention enough:
V2 of the series includes a global lock to be used around actual
installation/removal of the DPAMT backing, combined with opportunistic
checking outside the lock to avoid taking it most of the time. In testing,
booting 10 16GB TDs, the lock only hit contention 1136 times, with 4ms
waiting. This is very small for an operation that took 60s of wall time.
So despite being an (ugly) global lock, the actual impact was small. It
will probably further be reduced in the case of huge pages, where most of
the time 4KB DPAMT installation will not be necessary.
https://lore.kernel.org/all/20250918232224.2202592-1-rick.p.edgecombe@intel.com/
But I did not test without this global lock avoiding optimization. I can look
into it.
The other way to drop this patch I was considering was to limit the situations
when dynamic PAMT gets enabled somehow, like early LASS support. A boot time
param would be the simplest. Or define a policy based on the number of keyids
with the reasoning that if you are only able to run one TD, you will at least
not contend with other TDs. Then we could revisit the optimization after huge
pages is settled.
For the subject of the arguing though, my vote would be to drop the scope guard
stuff and stick with the old pattern from the beginning.
^ permalink raw reply
* Re: [PATCH v6 10/11] x86/virt/tdx: Enable Dynamic PAMT
From: Edgecombe, Rick P @ 2026-07-01 1:20 UTC (permalink / raw)
To: kas@kernel.org, Gao, Chao
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Hansen, Dave, Zhao, Yan Y, linux-kernel@vger.kernel.org,
seanjc@google.com, mingo@redhat.com, pbonzini@redhat.com,
nik.borisov@suse.com, linux-doc@vger.kernel.org, hpa@zytor.com,
tglx@kernel.org, Annapurve, Vishal, bp@alien8.de,
kirill.shutemov@linux.intel.com, x86@kernel.org
In-Reply-To: <aiJd4XTToKX20j/R@intel.com>
On Fri, 2026-06-05 at 13:25 +0800, Chao Gao wrote:
> On Thu, Jun 04, 2026 at 06:14:17PM +0100, Kiryl Shutsemau wrote:
> > On Mon, May 25, 2026 at 07:35:14PM -0700, Rick Edgecombe wrote:
> > > @@ -152,7 +156,12 @@ const struct tdx_sys_info *tdx_get_sysinfo(void);
> > >
> > > static inline bool tdx_supports_dynamic_pamt(const struct tdx_sys_info *sysinfo)
> > > {
> > > - return false; /* To be enabled when kernel is ready */
> > > + /*
> > > + * The TDX Module's internal Dynamic PAMT tree structure can't
> > > + * handle physical addresses with more than 48 bits.
> > > + */
> > > + return sysinfo->features.tdx_features0 & TDX_FEATURES0_DYNAMIC_PAMT &&
> > > + boot_cpu_data.x86_phys_bits <= 48;
> >
> > Should we warn for >48?
>
> Maybe we should drop this check. If the TDX module cannot handle that case,
> advertising TDX_FEATURES0_DYNAMIC_PAMT is a bug and should be fixed by the
> module.
I totally agree this is an awkward thing to make the VMM check. But I think
tdx_features0 is normally about what the *TDX Module* supports? When this is
kind of a valid configuration check. The user can configure enough keyids to
make Dynamic PAMT workable.
Looking... Actually there are some dynamic supported ones. Ok, I'll ask.
^ permalink raw reply
* Re: [PATCH v6 06/11] x86/virt/tdx: Optimize tdx_pamt_get/put()
From: Edgecombe, Rick P @ 2026-07-01 1:45 UTC (permalink / raw)
To: Hansen, Dave, Zhao, Yan Y
Cc: kirill.shutemov@linux.intel.com, linux-coco@lists.linux.dev,
Huang, Kai, kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
kas@kernel.org, seanjc@google.com, mingo@redhat.com,
pbonzini@redhat.com, nik.borisov@suse.com,
linux-doc@vger.kernel.org, hpa@zytor.com, Annapurve, Vishal,
tglx@kernel.org, Gao, Chao, bp@alien8.de, x86@kernel.org
In-Reply-To: <aiaQaCzL8o2yLu/2@yzhao56-desk.sh.intel.com>
Oh! You already did some testing.
On Mon, 2026-06-08 at 17:50 +0800, Yan Zhao wrote:
> > How important is this patch? I see "Optimize" but I read "Optional".
> This patch reduces the number of global pamt_lock acquisitions.
>
> Reference testing data with/without the optimization:
> (collected on my SPR test machine)
>
> Booting/teardown of 1 TD (8 vcpus/8G memory) per iteration:
> |--------------|-------------|------------|
> | avg (us) | max (us) | min (us) |
> | w/o | w/ | w/o | w/ | w/o | w/ |
> ----------------|-------|------|-------|-----|------|-----|
> __tdx_pamt_get()| 2 | 0 | 578 | 505 | 2 | 0 |
^ it looks like the old version. Is this on a huge page stack? Or just plain
dynamic PAMT?
> __tdx_pamt_put()| 0 | 0 | 563 | 496 | 0 | 0 |
> ----------------|--------------|-------------|------------|
>
> Boot/teardown of 5 TDs (each TD: 8 vcpus/8G memory) concurrently:
> |--------------|-------------|------------|
> | avg (us) | max (us) | min (us) |
> | w/o | w/ | w/o | w/ | w/o | w/ |
> ----------------|-------|------|-------|-----|------|-----|
> __tdx_pamt_get()| 15 | 0 | 1723 | 1386| 2 | 0 |
> __tdx_pamt_put()| 0 | 0 | 562 | 733| 0 | 0 |
> ----------------|--------------|-------------|------------|
Did you record total time, or number of invocations that we could use to turn
this into some more real world impact?
^ permalink raw reply
* Re: [PATCH v14 00/44] arm64: Support for Arm CCA in KVM
From: Kohei Enju @ 2026-07-01 2:15 UTC (permalink / raw)
To: Steven Price
Cc: kvm, kvmarm, Catalin Marinas, Marc Zyngier, Will Deacon,
James Morse, Oliver Upton, Suzuki K Poulose, Zenghui Yu,
linux-arm-kernel, linux-kernel, Joey Gouly, Alexandru Elisei,
Christoffer Dall, Fuad Tabba, linux-coco, Ganapatrao Kulkarni,
Gavin Shan, Shanker Donthineni, Alper Gun, Aneesh Kumar K . V,
Emi Kisanuki, Vishal Annapurve, WeiLin.Chang, Lorenzo.Pieralisi2
In-Reply-To: <20260513131757.116630-1-steven.price@arm.com>
On 05/13 14:17, Steven Price wrote:
> This series adds support for running protected VMs using KVM under the
> Arm Confidential Compute Architecture (CCA).
>
> This is rebased on v7.1-rc1, but still targets RMM v2.0-bet1[1].
>
> The major updates from v13 remain but have been more fully implemented:
> the RMM uses the host's page size, range based RMI APIs mean we don't
> have to break everything down to base page sizes, the GIC state is
> passed via system registers, and the uAPI has been simplified.
>
> The main changes since v13 are:
>
> * The RMI definitions and wrappers have been fully updated for RMM
> v2.0-bet1. In particular the temporary RMM v1.0 SMC compatibility
> patch has been dropped.
>
> * The PSCI completion ioctl has been removed. RMM v2.0-bet1 still
> requires the host to provide the target REC for PSCI calls which
> name another vCPU, but KVM now performs the RMI PSCI completion
> automatically before entering the REC again. Userspace no longer
> needs to issue KVM_ARM_VCPU_RMI_PSCI_COMPLETE. A future spec should
> remove the need for the host to provide the MPIDR mapping.
>
> * The generic RMI init, RMM configuration, GPT setup,
> delegate/undelegate helpers and SRO infrastructure have moved out of
> KVM into arch/arm64/kernel/rmi.c. RMI is expected to be used by
> features outside KVM, so this code should be available even when KVM
> is not built.
>
> * RMI_GRANULE_TRACKING_GET has been updated to work on a range, this
> allows it to work when the region is not aligned to the tracking
> size. Solves the problem reported by Mathieu[2].
>
> * SRO support has been moved earlier in the series and improved. It
> provides a cleaner way for the host to provide the RMM with the extra
> memory it requires. However support is still incomplete where the
> TF-RMM code does not yet implement it. This is noted by FIXMEs in the
> code.
>
> * The ARM VM type encoding has been reworked to coexist with the
> upstream pKVM KVM_VM_TYPE_ARM_PROTECTED bit.
>
> * The private-memory documentation now notes that arm64 uses
> KVM_CAP_MEMORY_ATTRIBUTES.
>
> * PMU support is dropped for now. It will be added later in a separate
> series. Similarly for selecting the hash algorithm and RPV.
Hi Steven,
Is there any plan to add support for selecting the MEC policy (shared or
private)? We have been working on adding support for this on top of your
series. If this is not already in the works, we may upstream our
implementation later.
Thanks,
Kohei
>
> There are also the usual rebase updates and smaller fixes, including
> changes to the RMM v2.0-bet1 range APIs, removal of REC auxiliary
> granule handling, fixes to the address range descriptor encoding, and
> cleanups around realm stage-2 teardown.
>
> Stateful RMI Operations
> -----------------------
>
> The RMM v2.0 spec introduces Stateful RMI Operations (SROs), which allow
> the RMM to complete an operation over several SMC calls while requesting
> or returning memory to the host. This allows interrupts to be handled in
> the middle of an operation and lets the RMM dynamically allocate memory
> for internal tracking purposes. For example, RMI_REC_CREATE no longer
> needs auxiliary granules to be provided up front, and can instead
> request memory during the operation.
>
> This series includes the generic SRO infrastructure in
> arch/arm64/kernel/rmi.c and uses it for REC create/destroy. The other
> cases are not yet used by TF-RMM and a future revision will be needed to
> finish those paths in Linux.
>
> This series is based on v7.1-rc1. It is also available as a git
> repository:
>
> https://gitlab.arm.com/linux-arm/linux-cca cca-host/v14
>
> Work in progress changes for kvmtool are available from the git
> repository below:
>
> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/v12
>
> The TF-RMM has not yet merged the RMM v2.0 support, so you will need to
> use a branch with RMM v2.0-bet1 support. At the time of writing the
> following branch is being used:
>
> https://git.trustedfirmware.org/TF-RMM/tf-rmm.git topics/rmm-v2.0-poc_2
> (tested on commit 3340667a291a)
>
> There is a kvm-unit-test branch which has been updated to support the
> attestation used in RMMv2.0 available here:
>
> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/v4
>
> [1] https://developer.arm.com/documentation/den0137/2-0bet1/
> [2] https://lore.kernel.org/all/acrj-cKphy4hJsEG@p14s/
^ permalink raw reply
* Re: [PATCH v6 00/20] dma-mapping: Use DMA_ATTR_CC_SHARED through direct, pool and swiotlb paths
From: Aneesh Kumar K.V @ 2026-07-01 3:09 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alexey Kardashevskiy, Catalin Marinas, iommu, linux-arm-kernel,
linux-kernel, linux-coco, Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Jiri Pirko, Mostafa Saleh, Petr Tesarik, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86
In-Reply-To: <20260630174216.GK7525@ziepe.ca>
Jason Gunthorpe <jgg@ziepe.ca> writes:
> On Mon, Jun 29, 2026 at 12:16:30PM +0530, Aneesh Kumar K.V wrote:
>> >> Thinking about this more, I guess we should mark the swiotlb as
>> >> cc_shared only with CC_ATTR_GUEST_MEM_ENCRYPT instead of
>> >> CC_ATTR_MEM_ENCRYPT as we have below.
>> >
>> > The name cc_shared should be used for GUEST scenarios only.
>> >
>> > I guess there is some merit in keeping swiotlb using "decrypted" to
>> > mean it usinig pgprot_decrypted and set_memory_decyped() which AMD
>> > gives meaning to on both host and guest.
>>
>> Are you suggesting to change the struct io_tlb_mem::cc_shared back to
>> struct io_tlb_mem::unencrypted?.
>
> Yes
>
>> > IDK what AMD should do on the host by default. I guess it should setup
>> > a swiotlb pool of low dma addrs "unencrypted", but not "cc_shared"?
>> >
>>
>> If by low DMA address you mean using an address with the C-bit
>> cleared.
>
> Yes
>
>> The current code already does this and uses the swiotlb pool correctly
>> on SME.
>
> Well, through the force_dma_unencrypted() hack...
>
>> The challenge arises when we want to force SWIOTLB
>> bouncing even for devices that can handle encrypted DMA addresses (more
>> on that below). For such a config force_dma_uencrypted(dev) will return
>> false and swiotlb will be marked cc_shared/decrypted = true; This trip
>> the new check we added.
>
> Yes, because cc_shared (guest) and unencrypted (host) are very
> different things and we've mixed them:
>
>> if (unlikely(mem->cc_shared != force_dma_unencrypted(dev)))
>
> I'm aruging force_dma_unencrypted should mean cc_shared and be
> guest_only, but the SME hack breaks this.
>
>> We can also do
>>
>> if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
>> /* swiotlb pool is incorrect for this device */
>> if (unlikely(mem->cc_shared != force_dma_unencrypted(dev)))
>> return (phys_addr_t)DMA_MAPPING_ERROR;
>>
>> /* Force attrs to match the kind of memory in the pool */
>> if (mem->cc_shared)
>> *attrs |= DMA_ATTR_CC_SHARED;
>> else
>> *attrs &= ~DMA_ATTR_CC_SHARED;
>> } else {
>> /*
>> * Host memory encryption where device requires an
>> * unencrypted dma_addr_t due to dma mask limit
>> */
>> if (force_dma_unencrypted(dev))
>> *attrs |= DMA_ATTR_CC_SHARED;
>> else
>> *attrs &= ~DMA_ATTR_CC_SHARED;
>> }
>
> If we do this I would like to split the force_dma_.. functions into
> guest and host, ie force_dma_cc_shared() and force_host_decrypted()
>
> To make it clear there are two very different things here.
>
I have now folded the below change into
modified kernel/dma/swiotlb.c
@@ -1514,9 +1514,23 @@ phys_addr_t swiotlb_tbl_map_single(struct device *dev, phys_addr_t orig_addr,
if (cc_platform_has(CC_ATTR_MEM_ENCRYPT))
pr_warn_once("Memory encryption is active and system is using DMA bounce buffers\n");
- /* swiotlb pool is incorrect for this device */
- if (unlikely(mem->cc_shared != force_dma_unencrypted(dev)))
- return (phys_addr_t)DMA_MAPPING_ERROR;
+ if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
+
+ /* swiotlb pool is incorrect for this device */
+ if (unlikely(mem->cc_shared != force_dma_unencrypted(dev)))
+ return (phys_addr_t)DMA_MAPPING_ERROR;
+
+ } else if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
+ /*
+ * On hosts with memory encryption, SWIOTLB-backed memory is
+ * unencrypted. DMA addresses returned for bounce buffers must
+ * therefore be marked unencrypted, even for devices that can
+ * address encrypted memory. This also preserves swiotlb=force
+ * behavior for those devices.
+ */
+ if (unlikely(!mem->cc_shared))
+ return (phys_addr_t)DMA_MAPPING_ERROR;
+ }
[PATCH] dma: swiotlb: track pool encryption state and honor DMA_ATTR_CC_SHARED
This is the only code path where we need to special-case host memory
encryption. For this reason, I have avoided renaming
io_tlb_mem::cc_shared to io_tlb_mem::unencrypted. I can send a v7 with
the above and we can review the changes based on that?
-aneesh
^ permalink raw reply
* Re: [PATCH v6 06/11] x86/virt/tdx: Optimize tdx_pamt_get/put()
From: Yan Zhao @ 2026-07-01 5:37 UTC (permalink / raw)
To: Edgecombe, Rick P
Cc: Hansen, Dave, kirill.shutemov@linux.intel.com,
linux-coco@lists.linux.dev, Huang, Kai, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org, kas@kernel.org, seanjc@google.com,
mingo@redhat.com, pbonzini@redhat.com, nik.borisov@suse.com,
linux-doc@vger.kernel.org, hpa@zytor.com, Annapurve, Vishal,
tglx@kernel.org, Gao, Chao, bp@alien8.de, x86@kernel.org
In-Reply-To: <173092378dcc803463b8d2af9f6f3ab6a908f77b.camel@intel.com>
On Wed, Jul 01, 2026 at 09:45:24AM +0800, Edgecombe, Rick P wrote:
> Oh! You already did some testing.
>
> On Mon, 2026-06-08 at 17:50 +0800, Yan Zhao wrote:
> > > How important is this patch? I see "Optimize" but I read "Optional".
> > This patch reduces the number of global pamt_lock acquisitions.
> >
> > Reference testing data with/without the optimization:
> > (collected on my SPR test machine)
> >
> > Booting/teardown of 1 TD (8 vcpus/8G memory) per iteration:
> > |--------------|-------------|------------|
> > | avg (us) | max (us) | min (us) |
> > | w/o | w/ | w/o | w/ | w/o | w/ |
> > ----------------|-------|------|-------|-----|------|-----|
> > __tdx_pamt_get()| 2 | 0 | 578 | 505 | 2 | 0 |
>
> ^ it looks like the old version. Is this on a huge page stack? Or just plain
> dynamic PAMT?
It's DPAMT v6 with the huge page stack. But the data was collected with huge
pages disabled.
>
> > __tdx_pamt_put()| 0 | 0 | 563 | 496 | 0 | 0 |
> > ----------------|--------------|-------------|------------|
> >
> > Boot/teardown of 5 TDs (each TD: 8 vcpus/8G memory) concurrently:
> > |--------------|-------------|------------|
> > | avg (us) | max (us) | min (us) |
> > | w/o | w/ | w/o | w/ | w/o | w/ |
> > ----------------|-------|------|-------|-----|------|-----|
> > __tdx_pamt_get()| 15 | 0 | 1723 | 1386| 2 | 0 |
> > __tdx_pamt_put()| 0 | 0 | 562 | 733| 0 | 0 |
> > ----------------|--------------|-------------|------------|
>
>
> Did you record total time, or number of invocations that we could use to turn
> this into some more real world impact?
Below data was collected on SPR, averaged over 3 runs (huge pages disabled):
Bootup+teardown of 1 TD (8 vcpus/8G memory):
|--------------|-------------|------------|
| avg (us) | #cnt | total (us) |
| w/o | w/ | w/o | w/ | w/o | w/ |
----------------|-------|------|-------|-----|------|-----|
__tdx_pamt_get()| 2 | 0 | 706k |704k |2025k | 51k |
__tdx_pamt_put()| 0 | 0 | 706k |704k | 130k | 34k |
----------------|--------------|-------------|------------|
Bootup+teardown of 5 TDs (each TD: 8 vcpus/8G memory) concurrently:
|--------------|-------------|------------|
| avg (us) | #cnt | total (us) |
| w/o | w/ | w/o | w/ | w/o | w/ |
----------------|-------|------|-------|-----|------|-----|
__tdx_pamt_get()| 15 | 0 | 3431k |3420k|55517k|334k |
__tdx_pamt_put()| 0 | 0 | 3431k |3420k| 2066k|137k |
----------------|--------------|-------------|------------|
^ permalink raw reply
* [PATCH v7 00/22] dma-mapping: Track shared DMA state through direct, pool and swiotlb paths
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86
This series tracks confidential-computing shared DMA state through the
dma-direct, dma-pool, and swiotlb paths so that encrypted and decrypted
DMA buffers are handled consistently.
Today, the direct DMA path mostly relies on force_dma_unencrypted() for
shared/decrypted buffer handling. This series consolidates the
force_dma_unencrypted() checks in the top-level functions and ensures
that the remaining DMA interfaces use DMA attributes to make the correct
decisions.
The series separates mapping and allocation state:
- DMA_ATTR_CC_SHARED describes the DMA address attribute requested for a
mapping. It tells the DMA mapping path that the DMA address must target
shared/decrypted memory.
- __DMA_ATTR_ALLOC_CC_SHARED is an internal DMA-mapping attribute used only
by allocation paths after the DMA core decides that the backing pages
must be allocated as shared/decrypted memory.
The series:
- moves swiotlb-backed allocations out of __dma_direct_alloc_pages(),
- uses __DMA_ATTR_ALLOC_CC_SHARED through the dma-direct alloc/free paths
- teaches the atomic DMA pools to track encrypted versus decrypted
state
- tracks swiotlb pool encryption state and enforces strict pool
selection
- centralizes encrypted/decrypted pgprot handling in dma_pgprot() using
DMA attributes
- passes DMA attributes down to dma_capable() so capability checks can
validate whether the selected DMA address encoding matches
DMA_ATTR_CC_SHARED
- makes dma_direct_map_phys() choose the DMA address encoding from
DMA_ATTR_CC_SHARED and fall back to swiotlb when a shared DMA request
cannot use the direct mapping, which lets arm64 and x86 CCA guests stop
relying on SWIOTLB_FORCE for DMA mappings
- use the selected swiotlb pool state to derive the returned DMA
address
- reports CC_ATTR_GUEST_MEM_ENCRYPT for arm64 Realms, powerpc secure
guests, and s390 protected virtualization guests.
Dependency:
This series depends on the pKVM changes posted at:
https://lore.kernel.org/all/20260603110522.3331819-1-smostafa@google.com
Please merge this series only after the pKVM changes above are merged.
Otherwise pKVM will be broken.
Changes since v6:
* Rebase onto the latest kernel.
* Add __DMA_ATTR_ALLOC_CC_SHARED for allocation paths. DMA_ATTR_CC_SHARED
is now used to describe the requested DMA mapping address attribute,
while __DMA_ATTR_ALLOC_CC_SHARED is used internally when allocating
shared/decrypted backing pages.
* Report CC_ATTR_GUEST_MEM_ENCRYPT for arm64 Realms, powerpc secure
guests, and s390 protected virtualization guests.
* Add CC_ATTR_HOST_MEM_ENCRYPT and swiotlb=force fixes.
Changes since v5:
https://lore.kernel.org/all/20260522042815.370873-1-aneesh.kumar@kernel.org
* Add Tested-by
* Drop the pKVM patch, which has now been posted separately:
https://lore.kernel.org/all/20260603110522.3331819-1-smostafa@google.com
* Remove the DO_NOT_MERGE tag from the s390 change.
* Add a patch to drop the SWIOTLB_FORCE flag.
* Rebase onto the latest kernel.
Changes since v4:
https://lore.kernel.org/all/20260512090408.794195-1-aneesh.kumar@kernel.org
* Add new patches based on Sashiko review:
swiotlb: Preserve allocation virtual address for dynamic pools
dma: free atomic pool pages by physical address
dma: swiotlb: handle set_memory_decrypted() failures
dma: swiotlb: free dynamic pools from process context
iommu/dma: Check atomic pool allocation result directly
* Include pKVM and s390 changes as dependent patches. These are not yet
ready to merge and are waiting for subsystem testing feedback.
* Drop the AMD GART patch because it requires wider testing.
* Update swiotlb_tbl_map_single() to take attrs by reference.
* Switch swiotlb_free() to use rcu_work.
* Avoid calling swiotlb_find_pool() multiple times in the free path.
* Make DMA_ATTR_MMIO imply DMA_ATTR_CC_SHARED for devices requiring unencrypted DMA.
Changes from v3:
https://lore.kernel.org/all/20260427055509.898190-1-aneesh.kumar@kernel.org
* Handle DMA_ATTR_MMIO correctly in dma_direct_map_phys()
* Address most of sashiko review
* Rebase to latest kernel
* drop SWIOTLB_FORCE for s390 and powerpc secure guest.
Changes from v2:
https://lore.kernel.org/all/20260420061415.3650870-1-aneesh.kumar@kernel.org
* pass attrs to dma_capable() and update direct, swiotlb, Xen swiotlb, and
x86 GART paths so the capability checks see the DMA address attr value
DMA_ATTR_CC_SHARED.
* rework dma_direct_map_phys() so DMA_ATTR_CC_SHARED selects
phys_to_dma_unencrypted() while the default path uses
phys_to_dma_encrypted(), with swiotlb fallback when the requested
shared/private state cannot be satisfied by a direct DMA address.
* stop relying on SWIOTLB_FORCE for arm64 and x86 CC guest DMA mappings;
swiotlb is still enabled there, but shared mappings is now selected
through the generic dma_direct_map_phys()/dma_capable() decision instead
of a global force-bounce flag.
Changes from v1:
https://lore.kernel.org/all/20260417085900.3062416-1-aneesh.kumar@kernel.org
* rebased to latest kernel (change from DMA_ATTR_CC_DECRYPTED -> DMA_ATTR_CC_SHARED)
* update the alloc path so DMA_ATTR_CC_SHARED is not a caller-visible attribute.
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Will Deacon <will@kernel.org>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Steven Price <steven.price@arm.com>
Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Mostafa Saleh <smostafa@google.com>
Cc: Petr Tesarik <ptesarik@suse.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Xu Yilun <yilun.xu@linux.intel.com>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: "Christophe Leroy (CS GROUP)" <chleroy@kernel.org>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: x86@kernel.org
Aneesh Kumar K.V (Arm) (22):
dma-direct: return struct page from dma_direct_alloc_from_pool()
dma-pool: fix page leak in atomic_pool_expand() cleanup
iommu/dma: Check atomic pool allocation result directly
dma: free atomic pool pages by physical address
swiotlb: Preserve allocation virtual address for dynamic pools
s390: Expose protected virtualization through cc_platform_has()
dma-direct: swiotlb: handle swiotlb alloc/free outside
__dma_direct_alloc_pages
coco: arm64: s390: powerpc: Mark secure guests with
CC_ATTR_GUEST_MEM_ENCRYPT
dma-mapping: Add internal shared allocation attribute
dma-direct: use __DMA_ATTR_ALLOC_CC_SHARED in alloc/free paths
dma-pool: track decrypted atomic pools and select them via attrs
dma: swiotlb: pass mapping attributes by reference
dma: swiotlb: track pool encryption state and honor DMA_ATTR_CC_SHARED
dma-mapping: make dma_pgprot() honor __DMA_ATTR_ALLOC_CC_SHARED
dma-direct: pass attrs to dma_capable() for DMA_ATTR_CC_SHARED checks
dma-direct: make dma_direct_map_phys() honor DMA_ATTR_CC_SHARED
dma-direct: set decrypted flag for remapped DMA allocations
dma-direct: select DMA address encoding from
__DMA_ATTR_ALLOC_CC_SHARED
dma-direct: rename ret to cpu_addr in alloc helpers
dma: swiotlb: free dynamic pools from process context
dma: swiotlb: handle set_memory_decrypted() failures
swiotlb: remove unused SWIOTLB_FORCE flag
Documentation/core-api/dma-attributes.rst | 29 ++
arch/arm64/kernel/rsi.c | 1 +
arch/arm64/mm/init.c | 4 +-
arch/powerpc/platforms/pseries/cc_platform.c | 1 +
arch/powerpc/platforms/pseries/svm.c | 2 +-
arch/s390/Kconfig | 1 +
arch/s390/mm/init.c | 17 +-
arch/x86/kernel/amd_gart_64.c | 30 +-
arch/x86/kernel/pci-dma.c | 4 +-
drivers/iommu/dma-iommu.c | 20 +-
drivers/xen/swiotlb-xen.c | 8 +-
include/linux/dma-direct.h | 20 +-
include/linux/dma-map-ops.h | 3 +-
include/linux/dma-mapping.h | 8 +
include/linux/swiotlb.h | 25 +-
include/trace/events/dma.h | 3 +-
kernel/dma/direct.c | 264 ++++++++++++-----
kernel/dma/direct.h | 47 +--
kernel/dma/mapping.c | 25 +-
kernel/dma/pool.c | 221 ++++++++++----
kernel/dma/swiotlb.c | 292 ++++++++++++++-----
21 files changed, 756 insertions(+), 269 deletions(-)
base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
--
2.43.0
^ permalink raw reply
* [PATCH v7 01/22] dma-direct: return struct page from dma_direct_alloc_from_pool()
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, stable, Michael Kelley,
Jason Gunthorpe
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
Commit 5b138c534fda ("dma-direct: factor out a dma_direct_alloc_from_pool
helper") changed dma_direct_alloc_from_pool() to return the CPU address
from dma_alloc_from_pool(). That fits dma_direct_alloc(), but
dma_direct_alloc_pages() also uses the helper and expects a struct page *.
Fix this by making dma_direct_alloc_from_pool() return the struct page *
again, and pass the CPU address back through an out-parameter for the
dma_direct_alloc() caller.
Fixes: 5b138c534fda ("dma-direct: factor out a dma_direct_alloc_from_pool helper")
Cc: stable@vger.kernel.org
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/direct.c | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 4391b797d4db..b4cb2c03e5d7 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -164,22 +164,21 @@ static bool dma_direct_use_pool(struct device *dev, gfp_t gfp)
return !gfpflags_allow_blocking(gfp) && !is_swiotlb_for_alloc(dev);
}
-static void *dma_direct_alloc_from_pool(struct device *dev, size_t size,
- dma_addr_t *dma_handle, gfp_t gfp)
+static struct page *dma_direct_alloc_from_pool(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, void **cpu_addr, gfp_t gfp)
{
struct page *page;
u64 phys_limit;
- void *ret;
if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_DMA_COHERENT_POOL)))
return NULL;
gfp |= dma_direct_optimal_gfp_mask(dev, &phys_limit);
- page = dma_alloc_from_pool(dev, size, &ret, gfp, dma_coherent_ok);
+ page = dma_alloc_from_pool(dev, size, cpu_addr, gfp, dma_coherent_ok);
if (!page)
return NULL;
*dma_handle = phys_to_dma_direct(dev, page_to_phys(page));
- return ret;
+ return page;
}
static void *dma_direct_alloc_no_mapping(struct device *dev, size_t size,
@@ -247,8 +246,11 @@ void *dma_direct_alloc(struct device *dev, size_t size,
* the atomic pools instead if we aren't allowed block.
*/
if ((remap || force_dma_unencrypted(dev)) &&
- dma_direct_use_pool(dev, gfp))
- return dma_direct_alloc_from_pool(dev, size, dma_handle, gfp);
+ dma_direct_use_pool(dev, gfp)) {
+ page = dma_direct_alloc_from_pool(dev, size, dma_handle,
+ &ret, gfp);
+ return page ? ret : NULL;
+ }
/* we always manually zero the memory once we are done */
page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true);
@@ -357,7 +359,7 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
void *ret;
if (force_dma_unencrypted(dev) && dma_direct_use_pool(dev, gfp))
- return dma_direct_alloc_from_pool(dev, size, dma_handle, gfp);
+ return dma_direct_alloc_from_pool(dev, size, dma_handle, &ret, gfp);
page = __dma_direct_alloc_pages(dev, size, gfp, false);
if (!page)
--
2.43.0
^ permalink raw reply related
* [PATCH v7 02/22] dma-pool: fix page leak in atomic_pool_expand() cleanup
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, Michael Kelley
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
atomic_pool_expand() frees the allocated pages from the remove_mapping
error path only when CONFIG_DMA_DIRECT_REMAP is enabled.
When CONFIG_DMA_DIRECT_REMAP is disabled, failures after page allocation,
such as gen_pool_add_virt(), jump to remove_mapping and return without
freeing the pages.
Move __free_pages(page, order) out of the CONFIG_DMA_DIRECT_REMAP block so
that cleanup paths always release the allocation.
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/pool.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index 2b2fbb709242..b0303efbc153 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -81,6 +81,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
{
unsigned int order;
struct page *page = NULL;
+ bool leak_pages = false;
void *addr;
int ret = -ENOMEM;
@@ -115,8 +116,10 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
*/
ret = set_memory_decrypted((unsigned long)page_to_virt(page),
1 << order);
- if (ret)
+ if (ret) {
+ leak_pages = true;
goto remove_mapping;
+ }
ret = gen_pool_add_virt(pool, (unsigned long)addr, page_to_phys(page),
pool_size, NUMA_NO_NODE);
if (ret)
@@ -130,14 +133,15 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
1 << order);
if (WARN_ON_ONCE(ret)) {
/* Decrypt succeeded but encrypt failed, purposely leak */
- goto out;
+ leak_pages = true;
}
remove_mapping:
#ifdef CONFIG_DMA_DIRECT_REMAP
dma_common_free_remap(addr, pool_size);
free_page:
- __free_pages(page, order);
#endif
+ if (!leak_pages)
+ __free_pages(page, order);
out:
return ret;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v7 03/22] iommu/dma: Check atomic pool allocation result directly
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, Michael Kelley
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
The non-blocking, non-coherent allocation path uses dma_alloc_from_pool(),
which returns the allocated page and fills cpu_addr only on success.
Do not rely on cpu_addr to detect allocation failure in this path. Check
the returned page directly before using it for the IOMMU mapping.
Fixes: 9420139f516d ("dma-pool: fix coherent pool allocations for IOMMU mappings")
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
drivers/iommu/dma-iommu.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 9abaec0703ef..68c686c1e81a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -1671,13 +1671,16 @@ void *iommu_dma_alloc(struct device *dev, size_t size, dma_addr_t *handle,
}
if (IS_ENABLED(CONFIG_DMA_DIRECT_REMAP) &&
- !gfpflags_allow_blocking(gfp) && !coherent)
+ !gfpflags_allow_blocking(gfp) && !coherent) {
page = dma_alloc_from_pool(dev, PAGE_ALIGN(size), &cpu_addr,
- gfp, NULL);
- else
+ gfp, NULL);
+ if (!page)
+ return NULL;
+ } else {
cpu_addr = iommu_dma_alloc_pages(dev, size, &page, gfp, attrs);
- if (!cpu_addr)
- return NULL;
+ if (!cpu_addr)
+ return NULL;
+ }
*handle = __iommu_dma_map(dev, page_to_phys(page), size, ioprot,
dev->coherent_dma_mask);
--
2.43.0
^ permalink raw reply related
* [PATCH v7 04/22] dma: free atomic pool pages by physical address
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, Michael Kelley
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
dma_direct_alloc_pages() may satisfy atomic allocations from the coherent
atomic pools. The pool allocation is keyed by the virtual address stored in
the gen_pool, but the pages API returns only the backing struct page.
On architectures with CONFIG_DMA_DIRECT_REMAP, atomic pool chunks are added
to the gen_pool using their remapped virtual address.
dma_direct_free_pages() reconstructs a linear-map address with
page_address(page) and passes that to dma_free_from_pool(). That address
does not match the gen_pool virtual range, so the pool lookup can fail and
the code can fall through to freeing a pool-owned page through the normal
page allocator path.
Add a page-based pool free helper that looks up the owning pool chunk by
physical address, translates it back to the gen_pool virtual address, and
frees that address to the pool. Use it from dma_direct_free_pages() while
keeping the existing virtual-address helper for coherent allocation frees.
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/dma-map-ops.h | 1 +
kernel/dma/direct.c | 4 +--
kernel/dma/pool.c | 54 +++++++++++++++++++++++++++++++++++++
3 files changed, 57 insertions(+), 2 deletions(-)
diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index bcb5b5428aea..137e015c1750 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -215,6 +215,7 @@ struct page *dma_alloc_from_pool(struct device *dev, size_t size,
void **cpu_addr, gfp_t flags,
bool (*phys_addr_ok)(struct device *, phys_addr_t, size_t));
bool dma_free_from_pool(struct device *dev, void *start, size_t size);
+bool dma_free_from_pool_page(struct device *dev, struct page *page, size_t size);
int dma_direct_set_offset(struct device *dev, phys_addr_t cpu_start,
dma_addr_t dma_start, u64 size);
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index b4cb2c03e5d7..17f1e097499e 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -381,9 +381,9 @@ void dma_direct_free_pages(struct device *dev, size_t size,
{
void *vaddr = page_address(page);
- /* If cpu_addr is not from an atomic pool, dma_free_from_pool() fails */
+ /* If page is not from an atomic pool, dma_free_from_pool_page() fails */
if (IS_ENABLED(CONFIG_DMA_COHERENT_POOL) &&
- dma_free_from_pool(dev, vaddr, size))
+ dma_free_from_pool_page(dev, page, size))
return;
if (dma_set_encrypted(dev, vaddr, size))
diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
index b0303efbc153..76bcafe03e44 100644
--- a/kernel/dma/pool.c
+++ b/kernel/dma/pool.c
@@ -311,3 +311,57 @@ bool dma_free_from_pool(struct device *dev, void *start, size_t size)
return false;
}
+
+struct dma_pool_phys_match {
+ phys_addr_t phys;
+ size_t size;
+ unsigned long addr;
+ bool found;
+};
+
+static void dma_pool_find_phys(struct gen_pool *pool, struct gen_pool_chunk *chunk,
+ void *data)
+{
+ struct dma_pool_phys_match *match = data;
+ phys_addr_t end = match->phys + match->size - 1;
+ phys_addr_t chunk_end;
+
+ if (match->found)
+ return;
+
+ chunk_end = chunk->phys_addr + (chunk->end_addr - chunk->start_addr);
+ if (match->phys < chunk->phys_addr || end > chunk_end)
+ return;
+
+ match->addr = chunk->start_addr + (match->phys - chunk->phys_addr);
+ match->found = true;
+}
+
+static bool dma_free_from_pool_phys(struct gen_pool *pool, phys_addr_t phys,
+ size_t size)
+{
+ struct dma_pool_phys_match match = {
+ .phys = phys,
+ .size = size,
+ };
+
+ gen_pool_for_each_chunk(pool, dma_pool_find_phys, &match);
+ if (!match.found)
+ return false;
+
+ gen_pool_free(pool, match.addr, size);
+ return true;
+}
+
+bool dma_free_from_pool_page(struct device *dev, struct page *page, size_t size)
+{
+ struct gen_pool *pool = NULL;
+ phys_addr_t phys = page_to_phys(page);
+
+ while ((pool = dma_guess_pool(pool, 0))) {
+ if (dma_free_from_pool_phys(pool, phys, size))
+ return true;
+ }
+
+ return false;
+}
--
2.43.0
^ permalink raw reply related
* [PATCH v7 05/22] swiotlb: Preserve allocation virtual address for dynamic pools
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, Michael Kelley
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
swiotlb_alloc_tlb() can allocate from the DMA atomic pool when a decrypted
pool is needed from atomic context. With CONFIG_DMA_DIRECT_REMAP, the
atomic pool is backed by remapped virtual addresses, which are not the same
as the direct-map addresses returned by phys_to_virt().
swiotlb_init_io_tlb_pool() currently reconstructs the pool virtual address
from the physical start address. For atomic-pool backed allocations this
stores the wrong address in pool->vaddr. Later, swiotlb_free_tlb() passes
that address to dma_free_from_pool(), which will fail to recognize the
chunk
Pass the virtual address returned by the allocation path into
swiotlb_init_io_tlb_pool(), and store that address in pool->vaddr. This
keeps the pool free path using the same virtual address as the allocator.
Fixes: 79636caad361 ("swiotlb: if swiotlb is full, fall back to a transient memory pool")
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Petr Tesarik <ptesarik@suse.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
kernel/dma/swiotlb.c | 31 +++++++++++++++++++------------
1 file changed, 19 insertions(+), 12 deletions(-)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 1abd3e6146f4..6e8db52866bf 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -266,9 +266,9 @@ void __init swiotlb_update_mem_attributes(void)
}
static void swiotlb_init_io_tlb_pool(struct io_tlb_pool *mem, phys_addr_t start,
- unsigned long nslabs, bool late_alloc, unsigned int nareas)
+ void *vaddr, unsigned long nslabs, bool late_alloc,
+ unsigned int nareas)
{
- void *vaddr = phys_to_virt(start);
unsigned long bytes = nslabs << IO_TLB_SHIFT, i;
mem->nslabs = nslabs;
@@ -409,7 +409,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
return;
}
- swiotlb_init_io_tlb_pool(mem, __pa(tlb), nslabs, false, nareas);
+ swiotlb_init_io_tlb_pool(mem, __pa(tlb), tlb, nslabs, false, nareas);
add_mem_pool(&io_tlb_default_mem, mem);
if (flags & SWIOTLB_VERBOSE)
@@ -507,7 +507,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
set_memory_decrypted((unsigned long)vstart,
(nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT);
- swiotlb_init_io_tlb_pool(mem, virt_to_phys(vstart), nslabs, true,
+ swiotlb_init_io_tlb_pool(mem, virt_to_phys(vstart), vstart, nslabs, true,
nareas);
add_mem_pool(&io_tlb_default_mem, mem);
@@ -605,25 +605,26 @@ static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes, u64 phys_limit)
* @bytes: Size of the buffer.
* @phys_limit: Maximum allowed physical address of the buffer.
* @gfp: GFP flags for the allocation.
+ * @vaddr: Receives the virtual address for the allocated buffer.
*
* Return: Allocated pages, or %NULL on allocation failure.
*/
static struct page *swiotlb_alloc_tlb(struct device *dev, size_t bytes,
- u64 phys_limit, gfp_t gfp)
+ u64 phys_limit, gfp_t gfp, void **vaddr)
{
struct page *page;
+ *vaddr = NULL;
+
/*
* Allocate from the atomic pools if memory is encrypted and
* the allocation is atomic, because decrypting may block.
*/
if (!gfpflags_allow_blocking(gfp) && dev && force_dma_unencrypted(dev)) {
- void *vaddr;
-
if (!IS_ENABLED(CONFIG_DMA_COHERENT_POOL))
return NULL;
- return dma_alloc_from_pool(dev, bytes, &vaddr, gfp,
+ return dma_alloc_from_pool(dev, bytes, vaddr, gfp,
dma_coherent_ok);
}
@@ -645,6 +646,8 @@ static struct page *swiotlb_alloc_tlb(struct device *dev, size_t bytes,
return NULL;
}
+ if (page)
+ *vaddr = phys_to_virt(page_to_phys(page));
return page;
}
@@ -685,6 +688,7 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
{
struct io_tlb_pool *pool;
unsigned int slot_order;
+ void *tlb_vaddr;
struct page *tlb;
size_t pool_size;
size_t tlb_size;
@@ -701,7 +705,8 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
pool->areas = (void *)pool + sizeof(*pool);
tlb_size = nslabs << IO_TLB_SHIFT;
- while (!(tlb = swiotlb_alloc_tlb(dev, tlb_size, phys_limit, gfp))) {
+ while (!(tlb = swiotlb_alloc_tlb(dev, tlb_size, phys_limit, gfp,
+ &tlb_vaddr))) {
if (nslabs <= minslabs)
goto error_tlb;
nslabs = ALIGN(nslabs >> 1, IO_TLB_SEGSIZE);
@@ -715,11 +720,12 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
if (!pool->slots)
goto error_slots;
- swiotlb_init_io_tlb_pool(pool, page_to_phys(tlb), nslabs, true, nareas);
+ swiotlb_init_io_tlb_pool(pool, page_to_phys(tlb), tlb_vaddr, nslabs,
+ true, nareas);
return pool;
error_slots:
- swiotlb_free_tlb(page_address(tlb), tlb_size);
+ swiotlb_free_tlb(tlb_vaddr, tlb_size);
error_tlb:
kfree(pool);
error:
@@ -1851,7 +1857,8 @@ static int rmem_swiotlb_device_init(struct reserved_mem *rmem,
set_memory_decrypted((unsigned long)phys_to_virt(rmem->base),
rmem->size >> PAGE_SHIFT);
- swiotlb_init_io_tlb_pool(pool, rmem->base, nslabs,
+ swiotlb_init_io_tlb_pool(pool, rmem->base, phys_to_virt(rmem->base),
+ nslabs,
false, nareas);
mem->force_bounce = true;
mem->for_alloc = true;
--
2.43.0
^ permalink raw reply related
* [PATCH v7 06/22] s390: Expose protected virtualization through cc_platform_has()
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, Halil Pasic,
Matthew Rosato, Jaehoon Kim
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
Protected virtualization guests use memory encryption, so advertise that to
the rest of the kernel through cc_platform_has(CC_ATTR_MEM_ENCRYPT).
s390 already forces DMA mappings to be unencrypted for protected
virtualization guests through force_dma_unencrypted(). Add
ARCH_HAS_CC_PLATFORM and provide the matching cc_platform_has()
implementation
Cc: Halil Pasic <pasic@linux.ibm.com>
Cc: Matthew Rosato <mjrosato@linux.ibm.com>
Cc: Jaehoon Kim <jhkim@linux.ibm.com>
Tested-by: Jaehoon Kim <jhkim@linux.ibm.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
arch/s390/Kconfig | 1 +
arch/s390/mm/init.c | 14 ++++++++++++++
2 files changed, 15 insertions(+)
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 84404e6778d5..ab1bf55d197a 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -84,6 +84,7 @@ config S390
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2
select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
select ARCH_HAS_CC_CAN_LINK
+ select ARCH_HAS_CC_PLATFORM
select ARCH_HAS_CPU_FINALIZE_INIT
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index f07168a0d3dd..c29326c44af0 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -50,6 +50,7 @@
#include <linux/virtio_anchor.h>
#include <linux/virtio_config.h>
#include <linux/execmem.h>
+#include <linux/cc_platform.h>
pgd_t swapper_pg_dir[PTRS_PER_PGD] __section(".bss..swapper_pg_dir");
pgd_t invalid_pg_dir[PTRS_PER_PGD] __section(".bss..invalid_pg_dir");
@@ -142,6 +143,19 @@ bool force_dma_unencrypted(struct device *dev)
return is_prot_virt_guest();
}
+
+bool cc_platform_has(enum cc_attr attr)
+{
+ switch (attr) {
+ case CC_ATTR_MEM_ENCRYPT:
+ return is_prot_virt_guest();
+
+ default:
+ return false;
+ }
+}
+EXPORT_SYMBOL_GPL(cc_platform_has);
+
/* protected virtualization */
static void __init pv_init(void)
{
--
2.43.0
^ permalink raw reply related
* [PATCH v7 07/22] dma-direct: swiotlb: handle swiotlb alloc/free outside __dma_direct_alloc_pages
From: Aneesh Kumar K.V (Arm) @ 2026-07-01 5:49 UTC (permalink / raw)
To: iommu, linux-arm-kernel, linux-kernel, linux-coco
Cc: Aneesh Kumar K.V (Arm), Robin Murphy, Marek Szyprowski,
Will Deacon, Marc Zyngier, Steven Price, Suzuki K Poulose,
Catalin Marinas, Jiri Pirko, Jason Gunthorpe, Mostafa Saleh,
Petr Tesarik, Alexey Kardashevskiy, Dan Williams, Xu Yilun,
linuxppc-dev, linux-s390, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Alexander Gordeev,
Gerald Schaefer, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, x86, Jiri Pirko,
Michael Kelley
In-Reply-To: <20260701054926.825925-1-aneesh.kumar@kernel.org>
Move swiotlb allocation out of __dma_direct_alloc_pages() and handle it in
dma_direct_alloc() / dma_direct_alloc_pages().
This is needed for follow-up changes that simplify the handling of
memory encryption/decryption based on the DMA attribute flags.
swiotlb backing pages are already mapped decrypted by
swiotlb_update_mem_attributes() and rmem_swiotlb_device_init(), so
dma-direct should not call dma_set_decrypted() on allocation nor
dma_set_encrypted() on free for swiotlb-backed memory.
Update alloc/free paths to detect swiotlb-backed pages and skip
encrypt/decrypt transitions for those paths. Keep the existing highmem
rejection in dma_direct_alloc_pages() for swiotlb allocations.
Only for "restricted-dma-pool", we currently set `for_alloc = true`, while
rmem_swiotlb_device_init() decrypts the whole pool up front. This pool is
typically used together with "shared-dma-pool", where the shared region is
accessed after remap/ioremap and the returned address is suitable for
decrypted memory access. So existing code paths remain valid.
Tested-by: Jiri Pirko <jiri@nvidia.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
---
include/linux/swiotlb.h | 6 ++++
kernel/dma/direct.c | 71 ++++++++++++++++++++++++++++++-----------
kernel/dma/swiotlb.c | 6 ++++
3 files changed, 65 insertions(+), 18 deletions(-)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 3dae0f592063..c92ff6791595 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -284,6 +284,8 @@ extern void swiotlb_print_info(void);
#ifdef CONFIG_DMA_RESTRICTED_POOL
struct page *swiotlb_alloc(struct device *dev, size_t size);
bool swiotlb_free(struct device *dev, struct page *page, size_t size);
+void swiotlb_free_from_pool(struct device *dev,
+ phys_addr_t tlb_addr, struct io_tlb_pool *pool);
static inline bool is_swiotlb_for_alloc(struct device *dev)
{
@@ -299,6 +301,10 @@ static inline bool swiotlb_free(struct device *dev, struct page *page,
{
return false;
}
+static inline void swiotlb_free_from_pool(struct device *dev,
+ phys_addr_t tlb_addr, struct io_tlb_pool *pool)
+{
+}
static inline bool is_swiotlb_for_alloc(struct device *dev)
{
return false;
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 17f1e097499e..0cbf2b0835c4 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -96,14 +96,6 @@ static int dma_set_encrypted(struct device *dev, void *vaddr, size_t size)
return ret;
}
-static void __dma_direct_free_pages(struct device *dev, struct page *page,
- size_t size)
-{
- if (swiotlb_free(dev, page, size))
- return;
- dma_free_contiguous(dev, page, size);
-}
-
static struct page *dma_direct_alloc_swiotlb(struct device *dev, size_t size)
{
struct page *page = swiotlb_alloc(dev, size);
@@ -125,9 +117,6 @@ static struct page *__dma_direct_alloc_pages(struct device *dev, size_t size,
WARN_ON_ONCE(!PAGE_ALIGNED(size));
- if (is_swiotlb_for_alloc(dev))
- return dma_direct_alloc_swiotlb(dev, size);
-
gfp |= dma_direct_optimal_gfp_mask(dev, &phys_limit);
page = dma_alloc_contiguous(dev, size, gfp);
if (page) {
@@ -203,6 +192,7 @@ void *dma_direct_alloc(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs)
{
bool remap = false, set_uncached = false;
+ bool mark_mem_decrypt = true;
struct page *page;
void *ret;
@@ -252,11 +242,21 @@ void *dma_direct_alloc(struct device *dev, size_t size,
return page ? ret : NULL;
}
+ if (is_swiotlb_for_alloc(dev)) {
+ page = dma_direct_alloc_swiotlb(dev, size);
+ if (page) {
+ mark_mem_decrypt = false;
+ goto setup_page;
+ }
+ return NULL;
+ }
+
/* we always manually zero the memory once we are done */
page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true);
if (!page)
return NULL;
+setup_page:
/*
* dma_alloc_contiguous can return highmem pages depending on a
* combination the cma= arguments and per-arch setup. These need to be
@@ -283,7 +283,7 @@ void *dma_direct_alloc(struct device *dev, size_t size,
goto out_free_pages;
} else {
ret = page_address(page);
- if (dma_set_decrypted(dev, ret, size))
+ if (mark_mem_decrypt && dma_set_decrypted(dev, ret, size))
goto out_leak_pages;
}
@@ -300,10 +300,11 @@ void *dma_direct_alloc(struct device *dev, size_t size,
return ret;
out_encrypt_pages:
- if (dma_set_encrypted(dev, page_address(page), size))
+ if (mark_mem_decrypt && dma_set_encrypted(dev, page_address(page), size))
return NULL;
out_free_pages:
- __dma_direct_free_pages(dev, page, size);
+ if (!swiotlb_free(dev, page, size))
+ dma_free_contiguous(dev, page, size);
return NULL;
out_leak_pages:
return NULL;
@@ -312,6 +313,9 @@ void *dma_direct_alloc(struct device *dev, size_t size,
void dma_direct_free(struct device *dev, size_t size,
void *cpu_addr, dma_addr_t dma_addr, unsigned long attrs)
{
+ phys_addr_t phys;
+ bool mark_mem_encrypted = true;
+ struct io_tlb_pool *swiotlb_pool;
unsigned int page_order = get_order(size);
if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) &&
@@ -340,16 +344,25 @@ void dma_direct_free(struct device *dev, size_t size,
dma_free_from_pool(dev, cpu_addr, PAGE_ALIGN(size)))
return;
+ phys = dma_to_phys(dev, dma_addr);
+ swiotlb_pool = swiotlb_find_pool(dev, phys);
+ if (swiotlb_pool)
+ /* Swiotlb doesn't need a page attribute update on free */
+ mark_mem_encrypted = false;
+
if (is_vmalloc_addr(cpu_addr)) {
vunmap(cpu_addr);
} else {
if (IS_ENABLED(CONFIG_ARCH_HAS_DMA_CLEAR_UNCACHED))
arch_dma_clear_uncached(cpu_addr, size);
- if (dma_set_encrypted(dev, cpu_addr, size))
+ if (mark_mem_encrypted && dma_set_encrypted(dev, cpu_addr, size))
return;
}
- __dma_direct_free_pages(dev, dma_direct_to_page(dev, dma_addr), size);
+ if (swiotlb_pool)
+ swiotlb_free_from_pool(dev, phys, swiotlb_pool);
+ else
+ dma_free_contiguous(dev, dma_direct_to_page(dev, dma_addr), size);
}
struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
@@ -361,6 +374,15 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
if (force_dma_unencrypted(dev) && dma_direct_use_pool(dev, gfp))
return dma_direct_alloc_from_pool(dev, size, dma_handle, &ret, gfp);
+ if (is_swiotlb_for_alloc(dev)) {
+ page = dma_direct_alloc_swiotlb(dev, size);
+ if (!page)
+ return NULL;
+
+ ret = page_address(page);
+ goto setup_page;
+ }
+
page = __dma_direct_alloc_pages(dev, size, gfp, false);
if (!page)
return NULL;
@@ -368,6 +390,7 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
ret = page_address(page);
if (dma_set_decrypted(dev, ret, size))
goto out_leak_pages;
+setup_page:
memset(ret, 0, size);
*dma_handle = phys_to_dma_direct(dev, page_to_phys(page));
return page;
@@ -379,16 +402,28 @@ void dma_direct_free_pages(struct device *dev, size_t size,
struct page *page, dma_addr_t dma_addr,
enum dma_data_direction dir)
{
+ phys_addr_t phys;
void *vaddr = page_address(page);
+ struct io_tlb_pool *swiotlb_pool;
+ bool mark_mem_encrypted = true;
/* If page is not from an atomic pool, dma_free_from_pool_page() fails */
if (IS_ENABLED(CONFIG_DMA_COHERENT_POOL) &&
dma_free_from_pool_page(dev, page, size))
return;
- if (dma_set_encrypted(dev, vaddr, size))
+ phys = page_to_phys(page);
+ swiotlb_pool = swiotlb_find_pool(dev, phys);
+ if (swiotlb_pool)
+ mark_mem_encrypted = false;
+
+ if (mark_mem_encrypted && dma_set_encrypted(dev, vaddr, size))
return;
- __dma_direct_free_pages(dev, page, size);
+
+ if (swiotlb_pool)
+ swiotlb_free_from_pool(dev, phys, swiotlb_pool);
+ else
+ dma_free_contiguous(dev, page, size);
}
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index 6e8db52866bf..d54154c165e5 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -1815,6 +1815,12 @@ bool swiotlb_free(struct device *dev, struct page *page, size_t size)
return true;
}
+void swiotlb_free_from_pool(struct device *dev,
+ phys_addr_t tlb_addr, struct io_tlb_pool *pool)
+{
+ swiotlb_release_slots(dev, tlb_addr, pool);
+}
+
static int rmem_swiotlb_device_init(struct reserved_mem *rmem,
struct device *dev)
{
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox