* Re: [PATCH v2 5/7] KVM: guest_memfd: Add cleanup interface for guest teardown
From: Ackerley Tng @ 2026-03-11 6:00 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <98313534-af6a-4c00-a016-9d9010f145da@amd.com>
"Kalra, Ashish" <ashish.kalra@amd.com> writes:
> Hello Ackerley,
>
> On 3/9/2026 4:01 AM, Ackerley Tng wrote:
>> Ashish Kalra <Ashish.Kalra@amd.com> writes:
>>
>>> From: Ashish Kalra <ashish.kalra@amd.com>
>>>
>>> Introduce kvm_arch_gmem_cleanup() to perform architecture-specific
>>> cleanups when the last file descriptor for the guest_memfd inode is
>>> closed. This typically occurs during guest shutdown and termination
>>> and allows for final resource release.
>>>
>>> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
>>> ---
>>>
>>> [...snip...]
>>>
>>> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
>>> index 017d84a7adf3..2724dd1099f2 100644
>>> --- a/virt/kvm/guest_memfd.c
>>> +++ b/virt/kvm/guest_memfd.c
>>> @@ -955,6 +955,14 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
>>>
>>> static void kvm_gmem_free_inode(struct inode *inode)
>>> {
>>> +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
>>> + /*
>>> + * Finalize cleanup for the inode once the last guest_memfd
>>> + * reference is released. This usually occurs after guest
>>> + * termination.
>>> + */
>>> + kvm_arch_gmem_cleanup();
>>> +#endif
>>
>> Folks have already talked about the performance implications of doing
>> the scan and rmpopt, I just want to call out that one VM could have more
>> than one associated guest_memfd too.
>
> Yes, i have observed that kvm_gmem_free_inode() gets invoked multiple times
> at SNP guest shutdown.
>
> And the same is true for kvm_gmem_destroy_inode() too.
>
>>
>> I think the cleanup function should be thought of as cleanup for the
>> inode (even if it doesn't take an inode pointer since it's not (yet)
>> required).
>>
>> So, the gmem cleanup function should not handle deduplicating cleanup
>> requests, but the arch function should, if the cleanup needs
>> deduplicating.
>
> I agree, the arch function will have to handle deduplicating, and for that
> the arch function will probably need to be passed the inode pointer,
> to have a parameter to assist with deduplicating.
>
By the time .free_folio() is called, folio->mapping may no longer exist,
so if we definitely want to deduplicate using something in the inode,
.free_folio() won't be the right callback to use.
I was thinking that deduplicating using something in the folio would be
better. Can rmpopt take a PFN range? Then there's really no
deduplication, the cleanup would be nicely narrowed to whatever was just
freed. Perhaps the PFNs could be aligned up to the nearest PMD or PUD
size for rmpopt to do the right thing.
Or perhaps some more tracking is required to check that the entire
aligned range is freed before doing the rmpopt.
I need to implement some of this tracking for guest_memfd HugeTLB
support, so if the tracking is useful for you, we should discuss!
>>
>> Also, .free_inode() is called through RCU, so it could be called after
>> some delay. Could it be possible that .free_inode() ends up being called
>> way after the associated VM gets torn down, or after KVM the module gets
>> unloaded? Does rmpopt still work fine if KVM the module got unloaded?
>
> Yes, .free_inode() can probably get called after the associated VM has
> been torn down and which should be fine for issuing RMPOPT to do
> RMP re-optimizations.
>
> As far as about KVM module getting unloaded, then as part of the forthcoming patch-series,
> during KVM module unload, X86_SNP_SHUTDOWN would be issued which means SNP would get
> disabled and therefore, RMP checks are also disabled.
>
> And as CC_ATTR_HOST_SEV_SNP would then be cleared, therefore, snp_perform_rmp_optimization()
> will simply return.
>
I think relying on CC_ATTR_HOST_SEV_SNP to skip optimization should be
best as long as there are no races (like the .free_inode() will
definitely not try to optimize when SNP is half shut down or something
like that.
> Another option is to add a new guest_memfd superblock operation, and then do the
> final guest_memfd cleanup using the .evict_inode() callback. This will then ensure
> that the cleanup is not called through RCU and avoids any kind of delays, as following:
>
> +static void kvm_gmem_evict_inode(struct inode *inode)
> +{
> +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
> + kvm_arch_gmem_cleanup();
> +#endif
> + truncate_inode_pages_final(&inode->i_data);
> + clear_inode(inode);
> +}
> +
>
At the point of .evict_inode(), CoCo-shared guest_memfd pages could
still be pinned (for DMA or whatever, accidentally or maliciously), can
rmpopt work on shared pages that might still be used for DMA?
.invalidate_folio() and .free_folio() both actually happen on removal
from guest_memfd ownership, though both are not exactly when the folio
is completely not in use.
Is the best time to optimize when the pages are truly freed?
> @@ -971,6 +979,7 @@ static const struct super_operations kvm_gmem_super_operations = {
> .alloc_inode = kvm_gmem_alloc_inode,
> .destroy_inode = kvm_gmem_destroy_inode,
> .free_inode = kvm_gmem_free_inode,
> + .evict_inode = kvm_gmem_evict_inode,
> };
>
>
> Thanks,
> Ashish
>
>>
>> IIUC the current kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
>> is fine because in kvm_gmem_exit(), there is a rcu_barrier() before
>> kmem_cache_destroy(kvm_gmem_inode_cachep);.
>>
>>> kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
>>> }
>>>
>>> --
>>> 2.43.0
^ permalink raw reply
* Re: [PATCH 4/7] KVM: x86: Add wrapper APIs to reset dirty/available register masks
From: Yosry Ahmed @ 2026-03-11 2:03 UTC (permalink / raw)
To: Sean Christopherson
Cc: Paolo Bonzini, Kiryl Shutsemau, kvm, x86, linux-coco,
linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-5-seanjc@google.com>
On Tue, Mar 10, 2026 at 5:34 PM Sean Christopherson <seanjc@google.com> wrote:
>
> Add wrappers for setting regs_{avail,dirty} in anticipation of turning the
> fields into proper bitmaps, at which point direct writes won't work so
> well.
>
> Deliberately leave the initialization in kvm_arch_vcpu_create() as-is,
> because the regs_avail logic in particular is special in that it's the one
> and only place where KVM marks eagerly synchronized registers as available.
>
> No functional change intended.
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> arch/x86/kvm/kvm_cache_regs.h | 19 +++++++++++++++++++
> arch/x86/kvm/svm/svm.c | 4 ++--
> arch/x86/kvm/vmx/nested.c | 4 ++--
> arch/x86/kvm/vmx/tdx.c | 2 +-
> arch/x86/kvm/vmx/vmx.c | 4 ++--
> 5 files changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h
> index ac1f9867a234..94e31cf38cb8 100644
> --- a/arch/x86/kvm/kvm_cache_regs.h
> +++ b/arch/x86/kvm/kvm_cache_regs.h
> @@ -105,6 +105,25 @@ static __always_inline bool kvm_register_test_and_mark_available(struct kvm_vcpu
> return arch___test_and_set_bit(reg, (unsigned long *)&vcpu->arch.regs_avail);
> }
>
> +static __always_inline void kvm_reset_available_registers(struct kvm_vcpu *vcpu,
> + u32 available_mask)
Not closely following this series and don't know this code well, but
this API is very confusing for me tbh. Especially in comparison with
kvm_reset_dirty_registers().
Maybe rename this to kvm_clear_available_registers(), and pass in a
"clear_mask", then reverse the polarity:
vcpu->arch.regs_avail &= ~clear_mask;
Most callers are already passing in an inverse of a mask, so might as
well pass the mask as-is and invert it here, and it helps make the
name clear, we're passing in a bitmask to clear from regs_avail.
> +{
> + /*
> + * Note the bitwise-AND! In practice, a straight write would also work
> + * as KVM initializes the mask to all ones and never clears registers
> + * that are eagerly synchronized. Using a bitwise-AND adds a bit of
> + * sanity checking as incorrectly marking an eagerly sync'd register
> + * unavailable will generate a WARN due to an unexpected cache request.
> + */
> + vcpu->arch.regs_avail &= available_mask;
> +}
> +
> +static __always_inline void kvm_reset_dirty_registers(struct kvm_vcpu *vcpu,
> + u32 dirty_mask)
> +{
> + vcpu->arch.regs_dirty = dirty_mask;
> +}
> +
> /*
> * The "raw" register helpers are only for cases where the full 64 bits of a
> * register are read/written irrespective of current vCPU mode. In other words,
^ permalink raw reply
* [PATCH 7/7] *** DO NOT MERGE *** KVM: x86: Pretend that APX is supported on 64-bit kernels
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index cff9023f12c7..3d9c8cc9d515 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -190,6 +190,27 @@ enum kvm_reg {
VCPU_REGS_R13 = __VCPU_REGS_R13,
VCPU_REGS_R14 = __VCPU_REGS_R14,
VCPU_REGS_R15 = __VCPU_REGS_R15,
+#define CONFIG_X86_APX
+
+#endif
+
+#ifdef CONFIG_X86_APX
+ VCPU_REG_R16 = VCPU_REGS_R15 + 1,
+ VCPU_REG_R17,
+ VCPU_REG_R18,
+ VCPU_REG_R19,
+ VCPU_REG_R20,
+ VCPU_REG_R21,
+ VCPU_REG_R22,
+ VCPU_REG_R23,
+ VCPU_REG_R24,
+ VCPU_REG_R25,
+ VCPU_REG_R26,
+ VCPU_REG_R27,
+ VCPU_REG_R28,
+ VCPU_REG_R29,
+ VCPU_REG_R30,
+ VCPU_REG_R31,
#endif
NR_VCPU_GENERAL_PURPOSE_REGS,
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 6/7] KVM: x86: Use a proper bitmap for tracking available/dirty registers
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
Define regs_{avail,dirty} as bitmaps instead of U32s to harden against
overflow, and to allow for dynamically sizing the bitmaps when APX comes
along, which will add 16 more GPRs (R16-R31) and thus increase the total
number of registers beyond 32.
Open code writes in the "reset" APIs, as the writes are hot paths and
bitmap_write() is complete overkill for what KVM needs. Even better,
hardcoding writes to entry '0' in the array is a perfect excuse to assert
that the array contains exactly one entry, e.g. to effectively add guard
against defining R16-R31 in 32-bit kernels.
For all intents and purposes, no functional change intended even though
using bitmap_fill() will mean "undefined" registers are no longer marked
available and dirty (KVM should never be querying those bits).
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 6 ++++--
arch/x86/kvm/kvm_cache_regs.h | 21 +++++++++++++--------
arch/x86/kvm/x86.c | 4 ++--
3 files changed, 19 insertions(+), 12 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 734c2eee58e0..cff9023f12c7 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -211,6 +211,8 @@ enum kvm_reg {
VCPU_REG_SEGMENTS,
VCPU_REG_EXIT_INFO_1,
VCPU_REG_EXIT_INFO_2,
+
+ NR_VCPU_TOTAL_REGS,
};
enum {
@@ -802,8 +804,8 @@ struct kvm_vcpu_arch {
*/
unsigned long regs[NR_VCPU_GENERAL_PURPOSE_REGS];
unsigned long rip;
- unsigned long regs_avail;
- unsigned long regs_dirty;
+ DECLARE_BITMAP(regs_avail, NR_VCPU_TOTAL_REGS);
+ DECLARE_BITMAP(regs_dirty, NR_VCPU_TOTAL_REGS);
unsigned long cr0;
unsigned long cr0_guest_owned_bits;
diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h
index 5de6c7dfd63b..782710829608 100644
--- a/arch/x86/kvm/kvm_cache_regs.h
+++ b/arch/x86/kvm/kvm_cache_regs.h
@@ -67,29 +67,29 @@ static inline bool kvm_register_is_available(struct kvm_vcpu *vcpu,
enum kvm_reg reg)
{
kvm_assert_register_caching_allowed(vcpu);
- return test_bit(reg, (unsigned long *)&vcpu->arch.regs_avail);
+ return test_bit(reg, vcpu->arch.regs_avail);
}
static inline bool kvm_register_is_dirty(struct kvm_vcpu *vcpu,
enum kvm_reg reg)
{
kvm_assert_register_caching_allowed(vcpu);
- return test_bit(reg, (unsigned long *)&vcpu->arch.regs_dirty);
+ return test_bit(reg, vcpu->arch.regs_dirty);
}
static inline void kvm_register_mark_available(struct kvm_vcpu *vcpu,
enum kvm_reg reg)
{
kvm_assert_register_caching_allowed(vcpu);
- __set_bit(reg, (unsigned long *)&vcpu->arch.regs_avail);
+ __set_bit(reg, vcpu->arch.regs_avail);
}
static inline void kvm_register_mark_dirty(struct kvm_vcpu *vcpu,
enum kvm_reg reg)
{
kvm_assert_register_caching_allowed(vcpu);
- __set_bit(reg, (unsigned long *)&vcpu->arch.regs_avail);
- __set_bit(reg, (unsigned long *)&vcpu->arch.regs_dirty);
+ __set_bit(reg, vcpu->arch.regs_avail);
+ __set_bit(reg, vcpu->arch.regs_dirty);
}
/*
@@ -102,12 +102,15 @@ static __always_inline bool kvm_register_test_and_mark_available(struct kvm_vcpu
enum kvm_reg reg)
{
kvm_assert_register_caching_allowed(vcpu);
- return arch___test_and_set_bit(reg, (unsigned long *)&vcpu->arch.regs_avail);
+ return arch___test_and_set_bit(reg, vcpu->arch.regs_avail);
}
static __always_inline void kvm_reset_available_registers(struct kvm_vcpu *vcpu,
unsigned long available_mask)
{
+ BUILD_BUG_ON(sizeof(available_mask) != sizeof(vcpu->arch.regs_avail[0]));
+ BUILD_BUG_ON(ARRAY_SIZE(vcpu->arch.regs_avail) != 1);
+
/*
* Note the bitwise-AND! In practice, a straight write would also work
* as KVM initializes the mask to all ones and never clears registers
@@ -115,13 +118,15 @@ static __always_inline void kvm_reset_available_registers(struct kvm_vcpu *vcpu,
* sanity checking as incorrectly marking an eagerly sync'd register
* unavailable will generate a WARN due to an unexpected cache request.
*/
- vcpu->arch.regs_avail &= available_mask;
+ vcpu->arch.regs_avail[0] &= available_mask;
}
static __always_inline void kvm_reset_dirty_registers(struct kvm_vcpu *vcpu,
unsigned long dirty_mask)
{
- vcpu->arch.regs_dirty = dirty_mask;
+ BUILD_BUG_ON(sizeof(dirty_mask) != sizeof(vcpu->arch.regs_dirty[0]));
+ BUILD_BUG_ON(ARRAY_SIZE(vcpu->arch.regs_dirty) != 1);
+ vcpu->arch.regs_dirty[0] = dirty_mask;
}
/*
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index dd39ccbff0d6..c1e1b3030786 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -12809,8 +12809,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
int r;
vcpu->arch.last_vmentry_cpu = -1;
- vcpu->arch.regs_avail = ~0;
- vcpu->arch.regs_dirty = ~0;
+ bitmap_fill(vcpu->arch.regs_avail, NR_VCPU_TOTAL_REGS);
+ bitmap_fill(vcpu->arch.regs_dirty, NR_VCPU_TOTAL_REGS);
kvm_gpc_init(&vcpu->arch.pv_time, vcpu->kvm);
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 5/7] KVM: x86: Track available/dirty register masks as "unsigned long" values
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
Convert regs_{avail,dirty} and all related masks to "unsigned long" values
as an intermediate step towards declaring the fields as actual bitmaps, and
as a step toward support APX, which will push the total number of registers
beyond 32 on 64-bit kernels.
Opportunistically convert TDX's ULL bitmask to a UL to match everything
else (TDX is 64-bit only, so it's a nop in the end).
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 4 ++--
arch/x86/kvm/kvm_cache_regs.h | 4 ++--
arch/x86/kvm/svm/svm.h | 2 +-
arch/x86/kvm/vmx/tdx.c | 34 ++++++++++++++++-----------------
arch/x86/kvm/vmx/vmx.h | 20 +++++++++----------
5 files changed, 32 insertions(+), 32 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3af5e2661ade..734c2eee58e0 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -802,8 +802,8 @@ struct kvm_vcpu_arch {
*/
unsigned long regs[NR_VCPU_GENERAL_PURPOSE_REGS];
unsigned long rip;
- u32 regs_avail;
- u32 regs_dirty;
+ unsigned long regs_avail;
+ unsigned long regs_dirty;
unsigned long cr0;
unsigned long cr0_guest_owned_bits;
diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h
index 94e31cf38cb8..5de6c7dfd63b 100644
--- a/arch/x86/kvm/kvm_cache_regs.h
+++ b/arch/x86/kvm/kvm_cache_regs.h
@@ -106,7 +106,7 @@ static __always_inline bool kvm_register_test_and_mark_available(struct kvm_vcpu
}
static __always_inline void kvm_reset_available_registers(struct kvm_vcpu *vcpu,
- u32 available_mask)
+ unsigned long available_mask)
{
/*
* Note the bitwise-AND! In practice, a straight write would also work
@@ -119,7 +119,7 @@ static __always_inline void kvm_reset_available_registers(struct kvm_vcpu *vcpu,
}
static __always_inline void kvm_reset_dirty_registers(struct kvm_vcpu *vcpu,
- u32 dirty_mask)
+ unsigned long dirty_mask)
{
vcpu->arch.regs_dirty = dirty_mask;
}
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index dea46130aa24..7010db21e8cc 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -460,7 +460,7 @@ static inline bool svm_is_vmrun_failure(u64 exit_code)
* KVM_REQ_LOAD_MMU_PGD is always requested when the cached vcpu->arch.cr3
* is changed. svm_load_mmu_pgd() then syncs the new CR3 value into the VMCB.
*/
-#define SVM_REGS_LAZY_LOAD_SET (1 << VCPU_REG_PDPTR)
+#define SVM_REGS_LAZY_LOAD_SET (BIT(VCPU_REG_PDPTR))
static inline void __vmcb_set_intercept(unsigned long *intercepts, u32 bit)
{
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index d4cb6dc8098f..1e4f59cfdc0a 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1013,23 +1013,23 @@ static fastpath_t tdx_exit_handlers_fastpath(struct kvm_vcpu *vcpu)
return EXIT_FASTPATH_NONE;
}
-#define TDX_REGS_AVAIL_SET (BIT_ULL(VCPU_REG_EXIT_INFO_1) | \
- BIT_ULL(VCPU_REG_EXIT_INFO_2) | \
- BIT_ULL(VCPU_REGS_RAX) | \
- BIT_ULL(VCPU_REGS_RBX) | \
- BIT_ULL(VCPU_REGS_RCX) | \
- BIT_ULL(VCPU_REGS_RDX) | \
- BIT_ULL(VCPU_REGS_RBP) | \
- BIT_ULL(VCPU_REGS_RSI) | \
- BIT_ULL(VCPU_REGS_RDI) | \
- BIT_ULL(VCPU_REGS_R8) | \
- BIT_ULL(VCPU_REGS_R9) | \
- BIT_ULL(VCPU_REGS_R10) | \
- BIT_ULL(VCPU_REGS_R11) | \
- BIT_ULL(VCPU_REGS_R12) | \
- BIT_ULL(VCPU_REGS_R13) | \
- BIT_ULL(VCPU_REGS_R14) | \
- BIT_ULL(VCPU_REGS_R15))
+#define TDX_REGS_AVAIL_SET (BIT(VCPU_REG_EXIT_INFO_1) | \
+ BIT(VCPU_REG_EXIT_INFO_2) | \
+ BIT(VCPU_REGS_RAX) | \
+ BIT(VCPU_REGS_RBX) | \
+ BIT(VCPU_REGS_RCX) | \
+ BIT(VCPU_REGS_RDX) | \
+ BIT(VCPU_REGS_RBP) | \
+ BIT(VCPU_REGS_RSI) | \
+ BIT(VCPU_REGS_RDI) | \
+ BIT(VCPU_REGS_R8) | \
+ BIT(VCPU_REGS_R9) | \
+ BIT(VCPU_REGS_R10) | \
+ BIT(VCPU_REGS_R11) | \
+ BIT(VCPU_REGS_R12) | \
+ BIT(VCPU_REGS_R13) | \
+ BIT(VCPU_REGS_R14) | \
+ BIT(VCPU_REGS_R15))
static void tdx_load_host_xsave_state(struct kvm_vcpu *vcpu)
{
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index d3255a054185..0962374c4cd3 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -623,16 +623,16 @@ BUILD_CONTROLS_SHADOW(tertiary_exec, TERTIARY_VM_EXEC_CONTROL, 64)
* cache on demand. Other registers not listed here are synced to
* the cache immediately after VM-Exit.
*/
-#define VMX_REGS_LAZY_LOAD_SET ((1 << VCPU_REG_RIP) | \
- (1 << VCPU_REGS_RSP) | \
- (1 << VCPU_REG_RFLAGS) | \
- (1 << VCPU_REG_PDPTR) | \
- (1 << VCPU_REG_SEGMENTS) | \
- (1 << VCPU_REG_CR0) | \
- (1 << VCPU_REG_CR3) | \
- (1 << VCPU_REG_CR4) | \
- (1 << VCPU_REG_EXIT_INFO_1) | \
- (1 << VCPU_REG_EXIT_INFO_2))
+#define VMX_REGS_LAZY_LOAD_SET (BIT(VCPU_REGS_RSP) | \
+ BIT(VCPU_REG_RIP) | \
+ BIT(VCPU_REG_RFLAGS) | \
+ BIT(VCPU_REG_PDPTR) | \
+ BIT(VCPU_REG_SEGMENTS) | \
+ BIT(VCPU_REG_CR0) | \
+ BIT(VCPU_REG_CR3) | \
+ BIT(VCPU_REG_CR4) | \
+ BIT(VCPU_REG_EXIT_INFO_1) | \
+ BIT(VCPU_REG_EXIT_INFO_2))
static inline unsigned long vmx_l1_guest_owned_cr0_bits(void)
{
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 4/7] KVM: x86: Add wrapper APIs to reset dirty/available register masks
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
Add wrappers for setting regs_{avail,dirty} in anticipation of turning the
fields into proper bitmaps, at which point direct writes won't work so
well.
Deliberately leave the initialization in kvm_arch_vcpu_create() as-is,
because the regs_avail logic in particular is special in that it's the one
and only place where KVM marks eagerly synchronized registers as available.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/kvm_cache_regs.h | 19 +++++++++++++++++++
arch/x86/kvm/svm/svm.c | 4 ++--
arch/x86/kvm/vmx/nested.c | 4 ++--
arch/x86/kvm/vmx/tdx.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 4 ++--
5 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h
index ac1f9867a234..94e31cf38cb8 100644
--- a/arch/x86/kvm/kvm_cache_regs.h
+++ b/arch/x86/kvm/kvm_cache_regs.h
@@ -105,6 +105,25 @@ static __always_inline bool kvm_register_test_and_mark_available(struct kvm_vcpu
return arch___test_and_set_bit(reg, (unsigned long *)&vcpu->arch.regs_avail);
}
+static __always_inline void kvm_reset_available_registers(struct kvm_vcpu *vcpu,
+ u32 available_mask)
+{
+ /*
+ * Note the bitwise-AND! In practice, a straight write would also work
+ * as KVM initializes the mask to all ones and never clears registers
+ * that are eagerly synchronized. Using a bitwise-AND adds a bit of
+ * sanity checking as incorrectly marking an eagerly sync'd register
+ * unavailable will generate a WARN due to an unexpected cache request.
+ */
+ vcpu->arch.regs_avail &= available_mask;
+}
+
+static __always_inline void kvm_reset_dirty_registers(struct kvm_vcpu *vcpu,
+ u32 dirty_mask)
+{
+ vcpu->arch.regs_dirty = dirty_mask;
+}
+
/*
* The "raw" register helpers are only for cases where the full 64 bits of a
* register are read/written irrespective of current vCPU mode. In other words,
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 1712c21f4128..1a6626c32188 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4524,7 +4524,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
vcpu->arch.regs[VCPU_REGS_RSP] = svm->vmcb->save.rsp;
vcpu->arch.rip = svm->vmcb->save.rip;
}
- vcpu->arch.regs_dirty = 0;
+ kvm_reset_dirty_registers(vcpu, 0);
if (unlikely(svm->vmcb->control.exit_code == SVM_EXIT_NMI))
kvm_before_interrupt(vcpu, KVM_HANDLING_NMI);
@@ -4570,7 +4570,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
vcpu->arch.apf.host_apf_flags =
kvm_read_and_reset_apf_flags();
- vcpu->arch.regs_avail &= ~SVM_REGS_LAZY_LOAD_SET;
+ kvm_reset_available_registers(vcpu, ~SVM_REGS_LAZY_LOAD_SET);
if (!msr_write_intercepted(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, vcpu_to_pmu(vcpu)->global_ctrl);
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index af2aaef38502..d4ba64bde709 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -310,13 +310,13 @@ static void vmx_switch_vmcs(struct kvm_vcpu *vcpu, struct loaded_vmcs *vmcs)
vmx_sync_vmcs_host_state(vmx, prev);
put_cpu();
- vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET;
+ kvm_reset_available_registers(vcpu, ~VMX_REGS_LAZY_LOAD_SET);
/*
* All lazily updated registers will be reloaded from VMCS12 on both
* vmentry and vmexit.
*/
- vcpu->arch.regs_dirty = 0;
+ kvm_reset_dirty_registers(vcpu, 0);
}
static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index c23ec4ac8bc8..d4cb6dc8098f 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1098,7 +1098,7 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
tdx_load_host_xsave_state(vcpu);
- vcpu->arch.regs_avail &= TDX_REGS_AVAIL_SET;
+ kvm_reset_available_registers(vcpu, TDX_REGS_AVAIL_SET);
if (unlikely(tdx->vp_enter_ret == EXIT_REASON_EPT_MISCONFIG))
return EXIT_FASTPATH_NONE;
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index ed44eb5b4349..217ea6e72c2f 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7472,7 +7472,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
flags);
vcpu->arch.cr2 = native_read_cr2();
- vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET;
+ kvm_reset_available_registers(vcpu, ~VMX_REGS_LAZY_LOAD_SET);
vmx->idt_vectoring_info = 0;
@@ -7538,7 +7538,7 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
vmcs_writel(GUEST_RSP, vcpu->arch.regs[VCPU_REGS_RSP]);
if (kvm_register_is_dirty(vcpu, VCPU_REG_RIP))
vmcs_writel(GUEST_RIP, vcpu->arch.rip);
- vcpu->arch.regs_dirty = 0;
+ kvm_reset_dirty_registers(vcpu, 0);
if (run_flags & KVM_RUN_LOAD_GUEST_DR6)
set_debugreg(vcpu->arch.dr6, 6);
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 3/7] KVM: nVMX: Do a bitwise-AND of regs_avail when switching active VMCS
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
When switching between vmcs01 and vmcs02, do a bitwise-AND of regs_avail
to effectively reset the mask for the new VMCS, purely to be consistent
with all other "full" writes of regs_avail. In practice, a straight write
versus a bitwise-AND will yield the same result, as kvm_arch_vcpu_create()
marks *all* registers available (and dirty), and KVM never marks registers
unavailable unless they're lazily loaded.
This will allow adding wrapper APIs to set regs_{avail,dirty} without
having to add special handling for a nVMX use case that doesn't exist in
practice.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/nested.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 942acc46f91d..af2aaef38502 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -310,7 +310,7 @@ static void vmx_switch_vmcs(struct kvm_vcpu *vcpu, struct loaded_vmcs *vmcs)
vmx_sync_vmcs_host_state(vmx, prev);
put_cpu();
- vcpu->arch.regs_avail = ~VMX_REGS_LAZY_LOAD_SET;
+ vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET;
/*
* All lazily updated registers will be reloaded from VMCS12 on both
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 2/7] KVM: x86: Drop the "EX" part of "EXREG" to avoid collision with APX
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
Now that NR_VCPU_REGS is no longer a thing, drop the "EX" is for
extended (or maybe extra?") prefix from non-GRP registers to avoid a
collision with APX (Advanced Performance Extensions), which adds:
16 additional general-purpose registers (GPRs) R16–R31, also referred
to as Extended GPRs (EGPRs) in this document;
I.e. KVM's version of "extended" won't match with APX's definition.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 18 +++++++--------
arch/x86/kvm/kvm_cache_regs.h | 16 ++++++-------
arch/x86/kvm/svm/svm.c | 6 ++---
arch/x86/kvm/svm/svm.h | 2 +-
arch/x86/kvm/vmx/nested.c | 6 ++---
arch/x86/kvm/vmx/tdx.c | 4 ++--
arch/x86/kvm/vmx/vmx.c | 40 ++++++++++++++++-----------------
arch/x86/kvm/vmx/vmx.h | 20 ++++++++---------
arch/x86/kvm/x86.c | 16 ++++++-------
9 files changed, 64 insertions(+), 64 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 0461ba97a3be..3af5e2661ade 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -195,8 +195,8 @@ enum kvm_reg {
VCPU_REG_RIP = NR_VCPU_GENERAL_PURPOSE_REGS,
- VCPU_EXREG_PDPTR,
- VCPU_EXREG_CR0,
+ VCPU_REG_PDPTR,
+ VCPU_REG_CR0,
/*
* Alias AMD's ERAPS (not a real register) to CR3 so that common code
* can trigger emulation of the RAP (Return Address Predictor) with
@@ -204,13 +204,13 @@ enum kvm_reg {
* is cleared on writes to CR3, i.e. marking CR3 dirty will naturally
* mark ERAPS dirty as well.
*/
- VCPU_EXREG_CR3,
- VCPU_EXREG_ERAPS = VCPU_EXREG_CR3,
- VCPU_EXREG_CR4,
- VCPU_EXREG_RFLAGS,
- VCPU_EXREG_SEGMENTS,
- VCPU_EXREG_EXIT_INFO_1,
- VCPU_EXREG_EXIT_INFO_2,
+ VCPU_REG_CR3,
+ VCPU_REG_ERAPS = VCPU_REG_CR3,
+ VCPU_REG_CR4,
+ VCPU_REG_RFLAGS,
+ VCPU_REG_SEGMENTS,
+ VCPU_REG_EXIT_INFO_1,
+ VCPU_REG_EXIT_INFO_2,
};
enum {
diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h
index 9b7df9de0e87..ac1f9867a234 100644
--- a/arch/x86/kvm/kvm_cache_regs.h
+++ b/arch/x86/kvm/kvm_cache_regs.h
@@ -159,8 +159,8 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index)
{
might_sleep(); /* on svm */
- if (!kvm_register_is_available(vcpu, VCPU_EXREG_PDPTR))
- kvm_x86_call(cache_reg)(vcpu, VCPU_EXREG_PDPTR);
+ if (!kvm_register_is_available(vcpu, VCPU_REG_PDPTR))
+ kvm_x86_call(cache_reg)(vcpu, VCPU_REG_PDPTR);
return vcpu->arch.walk_mmu->pdptrs[index];
}
@@ -174,8 +174,8 @@ static inline ulong kvm_read_cr0_bits(struct kvm_vcpu *vcpu, ulong mask)
{
ulong tmask = mask & KVM_POSSIBLE_CR0_GUEST_BITS;
if ((tmask & vcpu->arch.cr0_guest_owned_bits) &&
- !kvm_register_is_available(vcpu, VCPU_EXREG_CR0))
- kvm_x86_call(cache_reg)(vcpu, VCPU_EXREG_CR0);
+ !kvm_register_is_available(vcpu, VCPU_REG_CR0))
+ kvm_x86_call(cache_reg)(vcpu, VCPU_REG_CR0);
return vcpu->arch.cr0 & mask;
}
@@ -196,8 +196,8 @@ static inline ulong kvm_read_cr4_bits(struct kvm_vcpu *vcpu, ulong mask)
{
ulong tmask = mask & KVM_POSSIBLE_CR4_GUEST_BITS;
if ((tmask & vcpu->arch.cr4_guest_owned_bits) &&
- !kvm_register_is_available(vcpu, VCPU_EXREG_CR4))
- kvm_x86_call(cache_reg)(vcpu, VCPU_EXREG_CR4);
+ !kvm_register_is_available(vcpu, VCPU_REG_CR4))
+ kvm_x86_call(cache_reg)(vcpu, VCPU_REG_CR4);
return vcpu->arch.cr4 & mask;
}
@@ -211,8 +211,8 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu *vcpu,
static inline ulong kvm_read_cr3(struct kvm_vcpu *vcpu)
{
- if (!kvm_register_is_available(vcpu, VCPU_EXREG_CR3))
- kvm_x86_call(cache_reg)(vcpu, VCPU_EXREG_CR3);
+ if (!kvm_register_is_available(vcpu, VCPU_REG_CR3))
+ kvm_x86_call(cache_reg)(vcpu, VCPU_REG_CR3);
return vcpu->arch.cr3;
}
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 4b9d79412da7..1712c21f4128 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -1512,7 +1512,7 @@ static void svm_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
kvm_register_mark_available(vcpu, reg);
switch (reg) {
- case VCPU_EXREG_PDPTR:
+ case VCPU_REG_PDPTR:
/*
* When !npt_enabled, mmu->pdptrs[] is already available since
* it is always updated per SDM when moving to CRs.
@@ -4197,7 +4197,7 @@ static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva)
static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu)
{
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_ERAPS);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS);
svm_flush_tlb_asid(vcpu);
}
@@ -4473,7 +4473,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
svm->vmcb->save.cr2 = vcpu->arch.cr2;
if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS) &&
- kvm_register_is_dirty(vcpu, VCPU_EXREG_ERAPS))
+ kvm_register_is_dirty(vcpu, VCPU_REG_ERAPS))
svm->vmcb->control.erap_ctl |= ERAP_CONTROL_CLEAR_RAP;
svm_fixup_nested_rips(vcpu);
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 9909bb7d2d31..dea46130aa24 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -460,7 +460,7 @@ static inline bool svm_is_vmrun_failure(u64 exit_code)
* KVM_REQ_LOAD_MMU_PGD is always requested when the cached vcpu->arch.cr3
* is changed. svm_load_mmu_pgd() then syncs the new CR3 value into the VMCB.
*/
-#define SVM_REGS_LAZY_LOAD_SET (1 << VCPU_EXREG_PDPTR)
+#define SVM_REGS_LAZY_LOAD_SET (1 << VCPU_REG_PDPTR)
static inline void __vmcb_set_intercept(unsigned long *intercepts, u32 bit)
{
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 101588914cbb..942acc46f91d 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -1189,7 +1189,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,
}
vcpu->arch.cr3 = cr3;
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_CR3);
/* Re-initialize the MMU, e.g. to pick up CR4 MMU role changes. */
kvm_init_mmu(vcpu);
@@ -4972,7 +4972,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)
nested_ept_uninit_mmu_context(vcpu);
vcpu->arch.cr3 = vmcs_readl(GUEST_CR3);
- kvm_register_mark_available(vcpu, VCPU_EXREG_CR3);
+ kvm_register_mark_available(vcpu, VCPU_REG_CR3);
/*
* Use ept_save_pdptrs(vcpu) to load the MMU's cached PDPTRs
@@ -5074,7 +5074,7 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
kvm_service_local_tlb_flush_requests(vcpu);
/*
- * VCPU_EXREG_PDPTR will be clobbered in arch/x86/kvm/vmx/vmx.h between
+ * VCPU_REG_PDPTR will be clobbered in arch/x86/kvm/vmx/vmx.h between
* now and the new vmentry. Ensure that the VMCS02 PDPTR fields are
* up-to-date before switching to L1.
*/
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 1e47c194af53..c23ec4ac8bc8 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1013,8 +1013,8 @@ static fastpath_t tdx_exit_handlers_fastpath(struct kvm_vcpu *vcpu)
return EXIT_FASTPATH_NONE;
}
-#define TDX_REGS_AVAIL_SET (BIT_ULL(VCPU_EXREG_EXIT_INFO_1) | \
- BIT_ULL(VCPU_EXREG_EXIT_INFO_2) | \
+#define TDX_REGS_AVAIL_SET (BIT_ULL(VCPU_REG_EXIT_INFO_1) | \
+ BIT_ULL(VCPU_REG_EXIT_INFO_2) | \
BIT_ULL(VCPU_REGS_RAX) | \
BIT_ULL(VCPU_REGS_RBX) | \
BIT_ULL(VCPU_REGS_RCX) | \
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 802cc5d8bf43..ed44eb5b4349 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -843,8 +843,8 @@ static bool vmx_segment_cache_test_set(struct vcpu_vmx *vmx, unsigned seg,
bool ret;
u32 mask = 1 << (seg * SEG_FIELD_NR + field);
- if (!kvm_register_is_available(&vmx->vcpu, VCPU_EXREG_SEGMENTS)) {
- kvm_register_mark_available(&vmx->vcpu, VCPU_EXREG_SEGMENTS);
+ if (!kvm_register_is_available(&vmx->vcpu, VCPU_REG_SEGMENTS)) {
+ kvm_register_mark_available(&vmx->vcpu, VCPU_REG_SEGMENTS);
vmx->segment_cache.bitmask = 0;
}
ret = vmx->segment_cache.bitmask & mask;
@@ -1609,8 +1609,8 @@ unsigned long vmx_get_rflags(struct kvm_vcpu *vcpu)
struct vcpu_vmx *vmx = to_vmx(vcpu);
unsigned long rflags, save_rflags;
- if (!kvm_register_is_available(vcpu, VCPU_EXREG_RFLAGS)) {
- kvm_register_mark_available(vcpu, VCPU_EXREG_RFLAGS);
+ if (!kvm_register_is_available(vcpu, VCPU_REG_RFLAGS)) {
+ kvm_register_mark_available(vcpu, VCPU_REG_RFLAGS);
rflags = vmcs_readl(GUEST_RFLAGS);
if (vmx->rmode.vm86_active) {
rflags &= RMODE_GUEST_OWNED_EFLAGS_BITS;
@@ -1633,7 +1633,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
* if L1 runs L2 as a restricted guest.
*/
if (is_unrestricted_guest(vcpu)) {
- kvm_register_mark_available(vcpu, VCPU_EXREG_RFLAGS);
+ kvm_register_mark_available(vcpu, VCPU_REG_RFLAGS);
vmx->rflags = rflags;
vmcs_writel(GUEST_RFLAGS, rflags);
return;
@@ -2607,17 +2607,17 @@ void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
case VCPU_REG_RIP:
vcpu->arch.rip = vmcs_readl(GUEST_RIP);
break;
- case VCPU_EXREG_PDPTR:
+ case VCPU_REG_PDPTR:
if (enable_ept)
ept_save_pdptrs(vcpu);
break;
- case VCPU_EXREG_CR0:
+ case VCPU_REG_CR0:
guest_owned_bits = vcpu->arch.cr0_guest_owned_bits;
vcpu->arch.cr0 &= ~guest_owned_bits;
vcpu->arch.cr0 |= vmcs_readl(GUEST_CR0) & guest_owned_bits;
break;
- case VCPU_EXREG_CR3:
+ case VCPU_REG_CR3:
/*
* When intercepting CR3 loads, e.g. for shadowing paging, KVM's
* CR3 is loaded into hardware, not the guest's CR3.
@@ -2625,7 +2625,7 @@ void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
if (!(exec_controls_get(to_vmx(vcpu)) & CPU_BASED_CR3_LOAD_EXITING))
vcpu->arch.cr3 = vmcs_readl(GUEST_CR3);
break;
- case VCPU_EXREG_CR4:
+ case VCPU_REG_CR4:
guest_owned_bits = vcpu->arch.cr4_guest_owned_bits;
vcpu->arch.cr4 &= ~guest_owned_bits;
@@ -3350,7 +3350,7 @@ void vmx_ept_load_pdptrs(struct kvm_vcpu *vcpu)
{
struct kvm_mmu *mmu = vcpu->arch.walk_mmu;
- if (!kvm_register_is_dirty(vcpu, VCPU_EXREG_PDPTR))
+ if (!kvm_register_is_dirty(vcpu, VCPU_REG_PDPTR))
return;
if (is_pae_paging(vcpu)) {
@@ -3373,7 +3373,7 @@ void ept_save_pdptrs(struct kvm_vcpu *vcpu)
mmu->pdptrs[2] = vmcs_read64(GUEST_PDPTR2);
mmu->pdptrs[3] = vmcs_read64(GUEST_PDPTR3);
- kvm_register_mark_available(vcpu, VCPU_EXREG_PDPTR);
+ kvm_register_mark_available(vcpu, VCPU_REG_PDPTR);
}
#define CR3_EXITING_BITS (CPU_BASED_CR3_LOAD_EXITING | \
@@ -3416,7 +3416,7 @@ void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
vmcs_writel(CR0_READ_SHADOW, cr0);
vmcs_writel(GUEST_CR0, hw_cr0);
vcpu->arch.cr0 = cr0;
- kvm_register_mark_available(vcpu, VCPU_EXREG_CR0);
+ kvm_register_mark_available(vcpu, VCPU_REG_CR0);
#ifdef CONFIG_X86_64
if (vcpu->arch.efer & EFER_LME) {
@@ -3434,8 +3434,8 @@ void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
* (correctly) stop reading vmcs.GUEST_CR3 because it thinks
* KVM's CR3 is installed.
*/
- if (!kvm_register_is_available(vcpu, VCPU_EXREG_CR3))
- vmx_cache_reg(vcpu, VCPU_EXREG_CR3);
+ if (!kvm_register_is_available(vcpu, VCPU_REG_CR3))
+ vmx_cache_reg(vcpu, VCPU_REG_CR3);
/*
* When running with EPT but not unrestricted guest, KVM must
@@ -3472,7 +3472,7 @@ void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0)
* GUEST_CR3 is still vmx->ept_identity_map_addr if EPT + !URG.
*/
if (!(old_cr0_pg & X86_CR0_PG) && (cr0 & X86_CR0_PG))
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_CR3);
}
/* depends on vcpu->arch.cr0 to be set to a new value */
@@ -3501,7 +3501,7 @@ void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level)
if (!enable_unrestricted_guest && !is_paging(vcpu))
guest_cr3 = to_kvm_vmx(kvm)->ept_identity_map_addr;
- else if (kvm_register_is_dirty(vcpu, VCPU_EXREG_CR3))
+ else if (kvm_register_is_dirty(vcpu, VCPU_REG_CR3))
guest_cr3 = vcpu->arch.cr3;
else /* vmcs.GUEST_CR3 is already up-to-date. */
update_guest_cr3 = false;
@@ -3561,7 +3561,7 @@ void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
}
vcpu->arch.cr4 = cr4;
- kvm_register_mark_available(vcpu, VCPU_EXREG_CR4);
+ kvm_register_mark_available(vcpu, VCPU_REG_CR4);
if (!enable_unrestricted_guest) {
if (enable_ept) {
@@ -5021,7 +5021,7 @@ void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
vmcs_write32(GUEST_IDTR_LIMIT, 0xffff);
vmx_segment_cache_clear(vmx);
- kvm_register_mark_available(vcpu, VCPU_EXREG_SEGMENTS);
+ kvm_register_mark_available(vcpu, VCPU_REG_SEGMENTS);
vmcs_write32(GUEST_ACTIVITY_STATE, GUEST_ACTIVITY_ACTIVE);
vmcs_write32(GUEST_INTERRUPTIBILITY_INFO, 0);
@@ -7514,9 +7514,9 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
vmx->vt.exit_reason.full = EXIT_REASON_INVALID_STATE;
vmx->vt.exit_reason.failed_vmentry = 1;
- kvm_register_mark_available(vcpu, VCPU_EXREG_EXIT_INFO_1);
+ kvm_register_mark_available(vcpu, VCPU_REG_EXIT_INFO_1);
vmx->vt.exit_qualification = ENTRY_FAIL_DEFAULT;
- kvm_register_mark_available(vcpu, VCPU_EXREG_EXIT_INFO_2);
+ kvm_register_mark_available(vcpu, VCPU_REG_EXIT_INFO_2);
vmx->vt.exit_intr_info = 0;
return EXIT_FASTPATH_NONE;
}
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index 31bee8b0e4a1..d3255a054185 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -320,7 +320,7 @@ static __always_inline unsigned long vmx_get_exit_qual(struct kvm_vcpu *vcpu)
{
struct vcpu_vt *vt = to_vt(vcpu);
- if (!kvm_register_test_and_mark_available(vcpu, VCPU_EXREG_EXIT_INFO_1) &&
+ if (!kvm_register_test_and_mark_available(vcpu, VCPU_REG_EXIT_INFO_1) &&
!WARN_ON_ONCE(is_td_vcpu(vcpu)))
vt->exit_qualification = vmcs_readl(EXIT_QUALIFICATION);
@@ -331,7 +331,7 @@ static __always_inline u32 vmx_get_intr_info(struct kvm_vcpu *vcpu)
{
struct vcpu_vt *vt = to_vt(vcpu);
- if (!kvm_register_test_and_mark_available(vcpu, VCPU_EXREG_EXIT_INFO_2) &&
+ if (!kvm_register_test_and_mark_available(vcpu, VCPU_REG_EXIT_INFO_2) &&
!WARN_ON_ONCE(is_td_vcpu(vcpu)))
vt->exit_intr_info = vmcs_read32(VM_EXIT_INTR_INFO);
@@ -625,14 +625,14 @@ BUILD_CONTROLS_SHADOW(tertiary_exec, TERTIARY_VM_EXEC_CONTROL, 64)
*/
#define VMX_REGS_LAZY_LOAD_SET ((1 << VCPU_REG_RIP) | \
(1 << VCPU_REGS_RSP) | \
- (1 << VCPU_EXREG_RFLAGS) | \
- (1 << VCPU_EXREG_PDPTR) | \
- (1 << VCPU_EXREG_SEGMENTS) | \
- (1 << VCPU_EXREG_CR0) | \
- (1 << VCPU_EXREG_CR3) | \
- (1 << VCPU_EXREG_CR4) | \
- (1 << VCPU_EXREG_EXIT_INFO_1) | \
- (1 << VCPU_EXREG_EXIT_INFO_2))
+ (1 << VCPU_REG_RFLAGS) | \
+ (1 << VCPU_REG_PDPTR) | \
+ (1 << VCPU_REG_SEGMENTS) | \
+ (1 << VCPU_REG_CR0) | \
+ (1 << VCPU_REG_CR3) | \
+ (1 << VCPU_REG_CR4) | \
+ (1 << VCPU_REG_EXIT_INFO_1) | \
+ (1 << VCPU_REG_EXIT_INFO_2))
static inline unsigned long vmx_l1_guest_owned_cr0_bits(void)
{
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 879cdeb6adde..dd39ccbff0d6 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1090,14 +1090,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3)
}
/*
- * Marking VCPU_EXREG_PDPTR dirty doesn't work for !tdp_enabled.
+ * Marking VCPU_REG_PDPTR dirty doesn't work for !tdp_enabled.
* Shadow page roots need to be reconstructed instead.
*/
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs)))
kvm_mmu_free_roots(vcpu->kvm, mmu, KVM_MMU_ROOT_CURRENT);
memcpy(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs));
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_PDPTR);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_PDPTR);
kvm_make_request(KVM_REQ_LOAD_MMU_PGD, vcpu);
vcpu->arch.pdptrs_from_userspace = false;
@@ -1478,7 +1478,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
kvm_mmu_new_pgd(vcpu, cr3);
vcpu->arch.cr3 = cr3;
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_CR3);
/* Do not call post_set_cr3, we do not get here for confidential guests. */
handle_tlb_flush:
@@ -12446,7 +12446,7 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs,
vcpu->arch.cr2 = sregs->cr2;
*mmu_reset_needed |= kvm_read_cr3(vcpu) != sregs->cr3;
vcpu->arch.cr3 = sregs->cr3;
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_CR3);
kvm_x86_call(post_set_cr3)(vcpu, sregs->cr3);
kvm_set_cr8(vcpu, sregs->cr8);
@@ -12539,7 +12539,7 @@ static int __set_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2)
for (i = 0; i < 4 ; i++)
kvm_pdptr_write(vcpu, i, sregs2->pdptrs[i]);
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_PDPTR);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_PDPTR);
mmu_reset_needed = 1;
vcpu->arch.pdptrs_from_userspace = true;
}
@@ -13084,7 +13084,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
kvm_rip_write(vcpu, 0xfff0);
vcpu->arch.cr3 = 0;
- kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3);
+ kvm_register_mark_dirty(vcpu, VCPU_REG_CR3);
/*
* CR0.CD/NW are set on RESET, preserved on INIT. Note, some versions
@@ -14296,7 +14296,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva)
* the RAP (Return Address Predicator).
*/
if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
- kvm_register_is_dirty(vcpu, VCPU_EXREG_ERAPS);
+ kvm_register_is_dirty(vcpu, VCPU_REG_ERAPS);
kvm_invalidate_pcid(vcpu, operand.pcid);
return kvm_skip_emulated_instruction(vcpu);
@@ -14312,7 +14312,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva)
fallthrough;
case INVPCID_TYPE_ALL_INCL_GLOBAL:
/*
- * Don't bother marking VCPU_EXREG_ERAPS dirty, SVM will take
+ * Don't bother marking VCPU_REG_ERAPS dirty, SVM will take
* care of doing so when emulating the full guest TLB flush
* (the RAP is cleared on all implicit TLB flushes).
*/
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 1/7] KVM: x86: Add dedicated storage for guest RIP
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
In-Reply-To: <20260311003346.2626238-1-seanjc@google.com>
Add kvm_vcpu_arch.rip to track guest RIP instead of including it in the
generic regs[] array. Decoupling RIP from regs[] will allow using a
*completely* arbitrary index for RIP, as opposed to the mostly-arbitrary
index that is currently used. That in turn will allow using indices
16-31 to track R16-R31 that are coming with APX.
Note, although RIP can used for addressing, it does NOT have an
architecturally defined index, and so can't be reached via flows like
get_vmx_mem_address() where KVM "blindly" reads a general purpose register
given the SIB information reported by hardware. For RIP-relative
addressing, hardware reports the full "offset" in vmcs.EXIT_QUALIFICATION.
Note #2, keep the available/dirty tracking as RSP is context switched
through the VMCS, i.e. needs to be cached for VMX.
Opportunistically rename NR_VCPU_REGS to NR_VCPU_GENERAL_PURPOSE_REGS to
better capture what it tracks, and so that KVM can slot in R16-R13 without
running into weirdness where KVM's definition of "EXREG" doesn't line up
with APX's definition of "extended reg".
No functional change intended.
Cc: Chang S. Bae <chang.seok.bae@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/include/asm/kvm_host.h | 10 ++++++----
arch/x86/kvm/kvm_cache_regs.h | 12 ++++++++----
arch/x86/kvm/svm/sev.c | 2 +-
arch/x86/kvm/svm/svm.c | 6 +++---
arch/x86/kvm/vmx/vmx.c | 8 ++++----
arch/x86/kvm/vmx/vmx.h | 2 +-
6 files changed, 23 insertions(+), 17 deletions(-)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index c94556fefb75..0461ba97a3be 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -191,10 +191,11 @@ enum kvm_reg {
VCPU_REGS_R14 = __VCPU_REGS_R14,
VCPU_REGS_R15 = __VCPU_REGS_R15,
#endif
- VCPU_REGS_RIP,
- NR_VCPU_REGS,
+ NR_VCPU_GENERAL_PURPOSE_REGS,
- VCPU_EXREG_PDPTR = NR_VCPU_REGS,
+ VCPU_REG_RIP = NR_VCPU_GENERAL_PURPOSE_REGS,
+
+ VCPU_EXREG_PDPTR,
VCPU_EXREG_CR0,
/*
* Alias AMD's ERAPS (not a real register) to CR3 so that common code
@@ -799,7 +800,8 @@ struct kvm_vcpu_arch {
* rip and regs accesses must go through
* kvm_{register,rip}_{read,write} functions.
*/
- unsigned long regs[NR_VCPU_REGS];
+ unsigned long regs[NR_VCPU_GENERAL_PURPOSE_REGS];
+ unsigned long rip;
u32 regs_avail;
u32 regs_dirty;
diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h
index 8ddb01191d6f..9b7df9de0e87 100644
--- a/arch/x86/kvm/kvm_cache_regs.h
+++ b/arch/x86/kvm/kvm_cache_regs.h
@@ -112,7 +112,7 @@ static __always_inline bool kvm_register_test_and_mark_available(struct kvm_vcpu
*/
static inline unsigned long kvm_register_read_raw(struct kvm_vcpu *vcpu, int reg)
{
- if (WARN_ON_ONCE((unsigned int)reg >= NR_VCPU_REGS))
+ if (WARN_ON_ONCE((unsigned int)reg >= NR_VCPU_GENERAL_PURPOSE_REGS))
return 0;
if (!kvm_register_is_available(vcpu, reg))
@@ -124,7 +124,7 @@ static inline unsigned long kvm_register_read_raw(struct kvm_vcpu *vcpu, int reg
static inline void kvm_register_write_raw(struct kvm_vcpu *vcpu, int reg,
unsigned long val)
{
- if (WARN_ON_ONCE((unsigned int)reg >= NR_VCPU_REGS))
+ if (WARN_ON_ONCE((unsigned int)reg >= NR_VCPU_GENERAL_PURPOSE_REGS))
return;
vcpu->arch.regs[reg] = val;
@@ -133,12 +133,16 @@ static inline void kvm_register_write_raw(struct kvm_vcpu *vcpu, int reg,
static inline unsigned long kvm_rip_read(struct kvm_vcpu *vcpu)
{
- return kvm_register_read_raw(vcpu, VCPU_REGS_RIP);
+ if (!kvm_register_is_available(vcpu, VCPU_REG_RIP))
+ kvm_x86_call(cache_reg)(vcpu, VCPU_REG_RIP);
+
+ return vcpu->arch.rip;
}
static inline void kvm_rip_write(struct kvm_vcpu *vcpu, unsigned long val)
{
- kvm_register_write_raw(vcpu, VCPU_REGS_RIP, val);
+ vcpu->arch.rip = val;
+ kvm_register_mark_dirty(vcpu, VCPU_REG_RIP);
}
static inline unsigned long kvm_rsp_read(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index b1aa85a6ca5a..0dec619490c3 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -913,7 +913,7 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
save->r14 = svm->vcpu.arch.regs[VCPU_REGS_R14];
save->r15 = svm->vcpu.arch.regs[VCPU_REGS_R15];
#endif
- save->rip = svm->vcpu.arch.regs[VCPU_REGS_RIP];
+ save->rip = svm->vcpu.arch.rip;
/* Sync some non-GPR registers before encrypting */
save->xcr0 = svm->vcpu.arch.xcr0;
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 3407deac90bd..4b9d79412da7 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4436,7 +4436,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
svm->vmcb->save.rax = vcpu->arch.regs[VCPU_REGS_RAX];
svm->vmcb->save.rsp = vcpu->arch.regs[VCPU_REGS_RSP];
- svm->vmcb->save.rip = vcpu->arch.regs[VCPU_REGS_RIP];
+ svm->vmcb->save.rip = vcpu->arch.rip;
/*
* Disable singlestep if we're injecting an interrupt/exception.
@@ -4522,7 +4522,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
vcpu->arch.cr2 = svm->vmcb->save.cr2;
vcpu->arch.regs[VCPU_REGS_RAX] = svm->vmcb->save.rax;
vcpu->arch.regs[VCPU_REGS_RSP] = svm->vmcb->save.rsp;
- vcpu->arch.regs[VCPU_REGS_RIP] = svm->vmcb->save.rip;
+ vcpu->arch.rip = svm->vmcb->save.rip;
}
vcpu->arch.regs_dirty = 0;
@@ -4954,7 +4954,7 @@ static int svm_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram)
svm->vmcb->save.rax = vcpu->arch.regs[VCPU_REGS_RAX];
svm->vmcb->save.rsp = vcpu->arch.regs[VCPU_REGS_RSP];
- svm->vmcb->save.rip = vcpu->arch.regs[VCPU_REGS_RIP];
+ svm->vmcb->save.rip = vcpu->arch.rip;
nested_svm_simple_vmexit(svm, SVM_EXIT_SW);
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 9302c16571cd..802cc5d8bf43 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -2604,8 +2604,8 @@ void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
case VCPU_REGS_RSP:
vcpu->arch.regs[VCPU_REGS_RSP] = vmcs_readl(GUEST_RSP);
break;
- case VCPU_REGS_RIP:
- vcpu->arch.regs[VCPU_REGS_RIP] = vmcs_readl(GUEST_RIP);
+ case VCPU_REG_RIP:
+ vcpu->arch.rip = vmcs_readl(GUEST_RIP);
break;
case VCPU_EXREG_PDPTR:
if (enable_ept)
@@ -7536,8 +7536,8 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
if (kvm_register_is_dirty(vcpu, VCPU_REGS_RSP))
vmcs_writel(GUEST_RSP, vcpu->arch.regs[VCPU_REGS_RSP]);
- if (kvm_register_is_dirty(vcpu, VCPU_REGS_RIP))
- vmcs_writel(GUEST_RIP, vcpu->arch.regs[VCPU_REGS_RIP]);
+ if (kvm_register_is_dirty(vcpu, VCPU_REG_RIP))
+ vmcs_writel(GUEST_RIP, vcpu->arch.rip);
vcpu->arch.regs_dirty = 0;
if (run_flags & KVM_RUN_LOAD_GUEST_DR6)
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index 70bfe81dea54..31bee8b0e4a1 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -623,7 +623,7 @@ BUILD_CONTROLS_SHADOW(tertiary_exec, TERTIARY_VM_EXEC_CONTROL, 64)
* cache on demand. Other registers not listed here are synced to
* the cache immediately after VM-Exit.
*/
-#define VMX_REGS_LAZY_LOAD_SET ((1 << VCPU_REGS_RIP) | \
+#define VMX_REGS_LAZY_LOAD_SET ((1 << VCPU_REG_RIP) | \
(1 << VCPU_REGS_RSP) | \
(1 << VCPU_EXREG_RFLAGS) | \
(1 << VCPU_EXREG_PDPTR) | \
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply related
* [PATCH 0/7] KVM: x86: APX reg prep work
From: Sean Christopherson @ 2026-03-11 0:33 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini, Kiryl Shutsemau
Cc: kvm, x86, linux-coco, linux-kernel, Chang S . Bae
Clean up KVM's register tracking and storage in preparation for landing APX,
which expands the maximum number of GPRs from 16 to 32.
This is kinda sorta an RFC, as there are some very opinionated changes. I.e.
if you dislike something, please speak up.
My thought is to treat R16-R31 as much like other GPRs as possible (though
maybe we don't need to expand regs[] as sketched out in the last patch?).
Sean Christopherson (7):
KVM: x86: Add dedicated storage for guest RIP
KVM: x86: Drop the "EX" part of "EXREG" to avoid collision with APX
KVM: nVMX: Do a bitwise-AND of regs_avail when switching active VMCS
KVM: x86: Add wrapper APIs to reset dirty/available register masks
KVM: x86: Track available/dirty register masks as "unsigned long"
values
KVM: x86: Use a proper bitmap for tracking available/dirty registers
*** DO NOT MERGE *** KVM: x86: Pretend that APX is supported on 64-bit
kernels
arch/x86/include/asm/kvm_host.h | 53 +++++++++++++++++++--------
arch/x86/kvm/kvm_cache_regs.h | 64 +++++++++++++++++++++++----------
arch/x86/kvm/svm/sev.c | 2 +-
arch/x86/kvm/svm/svm.c | 16 ++++-----
arch/x86/kvm/svm/svm.h | 2 +-
arch/x86/kvm/vmx/nested.c | 10 +++---
arch/x86/kvm/vmx/tdx.c | 36 +++++++++----------
arch/x86/kvm/vmx/vmx.c | 52 +++++++++++++--------------
arch/x86/kvm/vmx/vmx.h | 24 ++++++-------
arch/x86/kvm/x86.c | 20 +++++------
10 files changed, 166 insertions(+), 113 deletions(-)
base-commit: 5128b972fb2801ad9aca54d990a75611ab5283a9
--
2.53.0.473.g4a7958ca14-goog
^ permalink raw reply
* Re: [PATCH v2 5/7] KVM: guest_memfd: Add cleanup interface for guest teardown
From: Kalra, Ashish @ 2026-03-10 22:18 UTC (permalink / raw)
To: Ackerley Tng, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <CAEvNRgFCTNr=LUR_RM7+A4z+qHCWBZOYKe_Cbokwx0UsCtzaVw@mail.gmail.com>
Hello Ackerley,
On 3/9/2026 4:01 AM, Ackerley Tng wrote:
> Ashish Kalra <Ashish.Kalra@amd.com> writes:
>
>> From: Ashish Kalra <ashish.kalra@amd.com>
>>
>> Introduce kvm_arch_gmem_cleanup() to perform architecture-specific
>> cleanups when the last file descriptor for the guest_memfd inode is
>> closed. This typically occurs during guest shutdown and termination
>> and allows for final resource release.
>>
>> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
>> ---
>>
>> [...snip...]
>>
>> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
>> index 017d84a7adf3..2724dd1099f2 100644
>> --- a/virt/kvm/guest_memfd.c
>> +++ b/virt/kvm/guest_memfd.c
>> @@ -955,6 +955,14 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
>>
>> static void kvm_gmem_free_inode(struct inode *inode)
>> {
>> +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
>> + /*
>> + * Finalize cleanup for the inode once the last guest_memfd
>> + * reference is released. This usually occurs after guest
>> + * termination.
>> + */
>> + kvm_arch_gmem_cleanup();
>> +#endif
>
> Folks have already talked about the performance implications of doing
> the scan and rmpopt, I just want to call out that one VM could have more
> than one associated guest_memfd too.
Yes, i have observed that kvm_gmem_free_inode() gets invoked multiple times
at SNP guest shutdown.
And the same is true for kvm_gmem_destroy_inode() too.
>
> I think the cleanup function should be thought of as cleanup for the
> inode (even if it doesn't take an inode pointer since it's not (yet)
> required).
>
> So, the gmem cleanup function should not handle deduplicating cleanup
> requests, but the arch function should, if the cleanup needs
> deduplicating.
I agree, the arch function will have to handle deduplicating, and for that
the arch function will probably need to be passed the inode pointer,
to have a parameter to assist with deduplicating.
>
> Also, .free_inode() is called through RCU, so it could be called after
> some delay. Could it be possible that .free_inode() ends up being called
> way after the associated VM gets torn down, or after KVM the module gets
> unloaded? Does rmpopt still work fine if KVM the module got unloaded?
Yes, .free_inode() can probably get called after the associated VM has
been torn down and which should be fine for issuing RMPOPT to do
RMP re-optimizations.
As far as about KVM module getting unloaded, then as part of the forthcoming patch-series,
during KVM module unload, X86_SNP_SHUTDOWN would be issued which means SNP would get
disabled and therefore, RMP checks are also disabled.
And as CC_ATTR_HOST_SEV_SNP would then be cleared, therefore, snp_perform_rmp_optimization()
will simply return.
Another option is to add a new guest_memfd superblock operation, and then do the
final guest_memfd cleanup using the .evict_inode() callback. This will then ensure
that the cleanup is not called through RCU and avoids any kind of delays, as following:
+static void kvm_gmem_evict_inode(struct inode *inode)
+{
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
+ kvm_arch_gmem_cleanup();
+#endif
+ truncate_inode_pages_final(&inode->i_data);
+ clear_inode(inode);
+}
+
@@ -971,6 +979,7 @@ static const struct super_operations kvm_gmem_super_operations = {
.alloc_inode = kvm_gmem_alloc_inode,
.destroy_inode = kvm_gmem_destroy_inode,
.free_inode = kvm_gmem_free_inode,
+ .evict_inode = kvm_gmem_evict_inode,
};
Thanks,
Ashish
>
> IIUC the current kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
> is fine because in kvm_gmem_exit(), there is a rcu_barrier() before
> kmem_cache_destroy(kvm_gmem_inode_cachep);.
>
>> kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
>> }
>>
>> --
>> 2.43.0
^ permalink raw reply
* Re: [PATCH 1/1] virt: tdx-guest: Optimize the get-quote polling interval time
From: Kuppuswamy Sathyanarayanan @ 2026-03-10 18:58 UTC (permalink / raw)
To: Miao, Jun, kas@kernel.org, dave.hansen@linux.intel.com,
Edgecombe, Rick P
Cc: linux-coco@lists.linux.dev, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <PH7PR11MB8455A677E5BB797F9074E1EC9A76A@PH7PR11MB8455.namprd11.prod.outlook.com>
Hi Jun,
On 2/21/2026 6:17 PM, Miao, Jun wrote:
>> On 2/11/2026 12:58 AM, Jun Miao wrote:
>>> The TD guest sends TDREPORT to the TD Quoting Enclave via a vsock or a
>>> tdvmcall. In general, vsock is indeed much faster than tdvmcall, and
>>> Quote requests usually take a few millisecond to complete rather than
>>> seconds based on actual measurements.
>>>
>>> The following get quote time via tdvmcall were obtained on the GNR:
>>>
>>> | msleep_interruptible(time) | 1s | 5ms | 1ms |
>>> | ------------------------------ | -------- | -------- | ---------- |
>>> | Duration | 1.004 s | 1.005 s | 1.036 s |
>>> | Total(Get Quote) | 167 | 142 | 167 |
>>> | Success: | 167 | 142 | 167 |
>>> | Failure: | 0 | 0 | 0 |
>>> | Avg total / 1s | 0.97 | 141.31 | 166.35 |
>>> | Avg success / 1s | 0.97 | 141.31 | 166.35 |
>>> | Avg total / 1s / thread | 0.97 | 141.31 | 166.35 |
>>> | Avg success / 1s / thread | 0.97 | 141.31 | 166.35 |
>>> | Min elapsed_time | 1025.95ms| 6.85 ms | 2.99 ms |
>>> | Max elapsed_time | 1025.95ms| 10.93 ms | 10.76 ms |
>>>
>>
>> Thanks for sharing the data!
>>
>>> According to trace analysis, the typical execution tdvmcall get the
>>> quote time is 4 ms. Therefore, 5 ms is a reasonable balance between
>>> performance efficiency and CPU overhead.
>>
>> Since the average is 4 ms, why choose 5ms?
>>
>>>
>>> And compared to the previous throughput of one request per second, the
>>> current 5ms can get 142 requests per second delivers a 142×
>>> performance improvement, which is critical for high-frequency use
>>> cases without vsock.
>>
>> Is this addressing a real customer issue or a theoretical improvement?
>
> Hi Kuppuswamy,
>
> From the customer issue, the more detail "Test Report"
> [PATCH 0/1] [Test Report] get qutoe time via tdvmcall
> [Background]
> Currently, many mobile device vendors (such as OPPO and Xiaomi) use TDVM for security management.
> Each mobile terminal must perform remote attestation before it can access the TDVM confidential container.
> As a result, there are a large number of remote attestation get-quote requests, especially in cases where vsock
> is not configured or misconfigured and cannot be used.
>
Thanks for the details.
Since it's a real issue, I'm fine with updating the polling interval to 5ms.
Given that deployed QEs respond fast, we should also reduce the maximum wait
time to 2 seconds (from 30 seconds) to fail faster on errors.
You can use read_poll_timeout() from <linux/iopoll.h> to simplify the
timeout handling instead of manual loop counters.
That said, polling with fixed intervals doesn't scale well since QE response
times vary by implementation. The proper long-term solution is still an
interrupt-based approach to eliminate the polling overhead entirely.
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
^ permalink raw reply
* SVSM Development Call March 11, 2026
From: Jörg Rödel @ 2026-03-10 17:10 UTC (permalink / raw)
To: coconut-svsm, linux-coco
Hi,
Here is the call for agenda items for this weeks SVSM development call. Please
send any agenda items you have in mind as a reply to this email or raise them
in the meeting.
There is one item on the agenda so far:
- IGVM-Measure discussion
We will use the LF Zoom instance. Details of the meeting can be found in our
governance repository at:
https://github.com/coconut-svsm/governance
The link to the COCONUT-SVSM calendar is:
https://zoom-lfx.platform.linuxfoundation.org/meetings/coconut-svsm?view=week
The meeting will be recorded and the recording eventually published.
Regards,
Jörg
^ permalink raw reply
* Re: [PATCH v4 07/24] coco/tdx-host: Implement firmware upload sysfs ABI for TDX Module updates
From: Yan Zhao @ 2026-03-10 2:31 UTC (permalink / raw)
To: Chao Gao
Cc: linux-coco, linux-kernel, kvm, x86, reinette.chatre, ira.weiny,
kai.huang, dan.j.williams, yilun.xu, sagis, vannapurve, paulmck,
nik.borisov, zhenzhong.duan, seanjc, rick.p.edgecombe, kas,
dave.hansen, vishal.l.verma, binbin.wu, tony.lindgren,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin
In-Reply-To: <20260212143606.534586-8-chao.gao@intel.com>
On Thu, Feb 12, 2026 at 06:35:10AM -0800, Chao Gao wrote:
> diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
> index cb2219302dfc..ffadbf64d0c1 100644
> --- a/arch/x86/include/asm/tdx.h
> +++ b/arch/x86/include/asm/tdx.h
> @@ -103,6 +103,11 @@ int tdx_enable(void);
> const char *tdx_dump_mce_info(struct mce *m);
> const struct tdx_sys_info *tdx_get_sysinfo(void);
>
> +static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinfo)
> +{
> + return false; /* To be enabled when kernel is ready */
> +}
Nit: Tail comments are not preferred.
^ permalink raw reply
* [PATCH v2 4/4] x86/vmware: Support steal time clock for encrypted guests
From: Alexey Makhalov @ 2026-03-09 23:52 UTC (permalink / raw)
To: x86, virtualization, bp, hpa, dave.hansen, mingo, tglx
Cc: ajay.kaher, brennan.lamoreaux, bo.gan, bcm-kernel-feedback-list,
linux-kernel, kas, rick.p.edgecombe, linux-coco, Alexey Makhalov
In-Reply-To: <20260309235250.2611115-1-alexey.makhalov@broadcom.com>
Shared memory containing steal time counter should be set to
decrypted when guest memory is encrypted.
Co-developed-by: Bo Gan <bo.gan@broadcom.com>
Signed-off-by: Bo Gan <bo.gan@broadcom.com>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
---
arch/x86/kernel/cpu/vmware.c | 41 ++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index c631e577348a..523a9b99847d 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -32,6 +32,7 @@
#include <linux/sched/cputime.h>
#include <linux/kmsg_dump.h>
#include <linux/panic_notifier.h>
+#include <linux/set_memory.h>
#include <asm/div64.h>
#include <asm/x86_init.h>
#include <asm/hypervisor.h>
@@ -39,6 +40,7 @@
#include <asm/apic.h>
#include <asm/vmware.h>
#include <asm/svm.h>
+#include <asm/coco.h>
#undef pr_fmt
#define pr_fmt(fmt) "vmware: " fmt
@@ -379,9 +381,47 @@ static struct notifier_block vmware_pv_reboot_nb = {
.notifier_call = vmware_pv_reboot_notify,
};
+/*
+ * Map per-CPU variables for all possible CPUs as decrypted.
+ * Do this early in boot, before sharing the corresponding
+ * guest physical addresses with the hypervisor.
+ */
+static void __init set_shared_memory_decrypted(void)
+{
+ int cpu;
+
+ if (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
+ return;
+
+ for_each_possible_cpu(cpu) {
+ unsigned long size = sizeof(vmw_steal_time);
+ unsigned long addr = (unsigned long)&per_cpu(vmw_steal_time,
+ cpu);
+
+ /*
+ * There is no generic high-level API to mark memory as
+ * decrypted. Intel's set_memory_decrypted() depends on the
+ * buddy allocator and can fail early in boot if a page split
+ * is required and allocation is not possible. Use AMD's
+ * early_set_memory_decrypted() instead, which can perform
+ * the split during early boot.
+ */
+ early_set_memory_decrypted(addr, size);
+
+ /* That's it for AMD */
+ if (cc_vendor == CC_VENDOR_AMD)
+ continue;
+
+ set_memory_decrypted(addr & PAGE_MASK, 1UL <<
+ get_order((addr & ~PAGE_MASK) + size));
+
+ }
+}
+
#ifdef CONFIG_SMP
static void __init vmware_smp_prepare_boot_cpu(void)
{
+ set_shared_memory_decrypted();
vmware_guest_cpu_init();
native_smp_prepare_boot_cpu();
}
@@ -444,6 +484,7 @@ static void __init vmware_paravirt_ops_setup(void)
vmware_cpu_down_prepare) < 0)
pr_err("vmware_guest: Failed to install cpu hotplug callbacks\n");
#else
+ set_shared_memory_decrypted();
vmware_guest_cpu_init();
#endif
}
--
2.43.7
^ permalink raw reply related
* [PATCH v2 3/4] x86/vmware: Report guest crash to the hypervisor
From: Alexey Makhalov @ 2026-03-09 23:52 UTC (permalink / raw)
To: x86, virtualization, bp, hpa, dave.hansen, mingo, tglx
Cc: ajay.kaher, brennan.lamoreaux, bo.gan, bcm-kernel-feedback-list,
linux-kernel, kas, rick.p.edgecombe, linux-coco, Alexey Makhalov
In-Reply-To: <20260309235250.2611115-1-alexey.makhalov@broadcom.com>
Register the guest crash reporter to panic_notifier_list,
which will be called at panic time. Guest crash reporter
will report the crash to the hypervisor through
a hypercall.
Co-developed-by: Brennan Lamoreaux <brennan.lamoreaux@broadcom.com>
Signed-off-by: Brennan Lamoreaux <brennan.lamoreaux@broadcom.com>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
---
arch/x86/include/asm/vmware.h | 1 +
arch/x86/kernel/cpu/vmware.c | 21 +++++++++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index c23164503e54..bf6141353774 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -97,6 +97,7 @@
#define VMWARE_CMD_GETHZ 45
#define VMWARE_CMD_GETVCPU_INFO 68
#define VMWARE_CMD_STEALCLOCK 91
+#define VMWARE_CMD_REPORTGUESTCRASH 102
/*
* Hypercall command mask:
* bits [6:0] command, range [0, 127]
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index 9955f2ea0c84..c631e577348a 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -31,6 +31,7 @@
#include <linux/static_call.h>
#include <linux/sched/cputime.h>
#include <linux/kmsg_dump.h>
+#include <linux/panic_notifier.h>
#include <asm/div64.h>
#include <asm/x86_init.h>
#include <asm/hypervisor.h>
@@ -451,6 +452,24 @@ static void __init vmware_paravirt_ops_setup(void)
#define vmware_paravirt_ops_setup() do {} while (0)
#endif
+static int vmware_report_guest_crash(struct notifier_block *self,
+ unsigned long action, void *data)
+{
+ vmware_hypercall1(VMWARE_CMD_REPORTGUESTCRASH, 0);
+ return 0;
+}
+
+static struct notifier_block guest_crash_reporter = {
+ .notifier_call = vmware_report_guest_crash
+};
+
+static int __init register_guest_crash_reporter(void)
+{
+ atomic_notifier_chain_register(&panic_notifier_list,
+ &guest_crash_reporter);
+
+ return 0;
+}
/*
* VMware hypervisor takes care of exporting a reliable TSC to the guest.
* Still, due to timing difference when running on virtual cpus, the TSC can
@@ -545,6 +564,8 @@ static void __init vmware_platform_setup(void)
vmware_set_capabilities();
kmsg_dump_register(&kmsg_dumper);
+
+ register_guest_crash_reporter();
}
static u8 __init get_hypercall_mode(void)
--
2.43.7
^ permalink raw reply related
* [PATCH v2 2/4] x86/vmware: Log kmsg dump on panic
From: Alexey Makhalov @ 2026-03-09 23:52 UTC (permalink / raw)
To: x86, virtualization, bp, hpa, dave.hansen, mingo, tglx
Cc: ajay.kaher, brennan.lamoreaux, bo.gan, bcm-kernel-feedback-list,
linux-kernel, kas, rick.p.edgecombe, linux-coco, Alexey Makhalov
In-Reply-To: <20260309235250.2611115-1-alexey.makhalov@broadcom.com>
Improve debugability of VMware Linux guests by dumping
kernel messages during a panic to VM log file (vmware.log).
Co-developed-by: Bo Gan <bo.gan@broadcom.com>
Signed-off-by: Bo Gan <bo.gan@broadcom.com>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
---
arch/x86/include/asm/vmware.h | 1 +
arch/x86/kernel/cpu/vmware.c | 132 ++++++++++++++++++++++++++++++++++
2 files changed, 133 insertions(+)
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 6a084e088b30..c23164503e54 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -93,6 +93,7 @@
#define VMWARE_HYPERVISOR_MAGIC 0x564d5868U
#define VMWARE_CMD_GETVERSION 10
+#define VMWARE_CMD_MESSAGE 30
#define VMWARE_CMD_GETHZ 45
#define VMWARE_CMD_GETVCPU_INFO 68
#define VMWARE_CMD_STEALCLOCK 91
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index 93acd3414e37..9955f2ea0c84 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -30,6 +30,7 @@
#include <linux/reboot.h>
#include <linux/static_call.h>
#include <linux/sched/cputime.h>
+#include <linux/kmsg_dump.h>
#include <asm/div64.h>
#include <asm/x86_init.h>
#include <asm/hypervisor.h>
@@ -211,6 +212,13 @@ static unsigned long vmware_get_tsc_khz(void)
return vmware_tsc_khz;
}
+static void kmsg_dumper_vmware_log(struct kmsg_dumper *dumper,
+ struct kmsg_dump_detail *detail);
+
+static struct kmsg_dumper kmsg_dumper = {
+ .dump = kmsg_dumper_vmware_log
+};
+
#ifdef CONFIG_PARAVIRT
static struct cyc2ns_data vmware_cyc2ns __ro_after_init;
static bool vmw_sched_clock __initdata = true;
@@ -535,6 +543,8 @@ static void __init vmware_platform_setup(void)
#endif
vmware_set_capabilities();
+
+ kmsg_dump_register(&kmsg_dumper);
}
static u8 __init get_hypercall_mode(void)
@@ -630,3 +640,125 @@ const __initconst struct hypervisor_x86 x86_hyper_vmware = {
.runtime.sev_es_hcall_finish = vmware_sev_es_hcall_finish,
#endif
};
+
+#define VMWARE_HB_CMD_MESSAGE 0
+#define MESSAGE_STATUS_SUCCESS (0x01 << 16)
+#define MESSAGE_STATUS_CPT (0x10 << 16)
+#define MESSAGE_STATUS_HB (0x80 << 16)
+
+#define RPCI_PROTOCOL_NUM 0x49435052 /* 'RPCI' */
+#define GUESTMSG_FLAG_COOKIE 0x80000000
+
+#define MESSAGE_TYPE_OPEN (0 << 16)
+#define MESSAGE_TYPE_SENDSIZE (1 << 16)
+#define MESSAGE_TYPE_SEND (2 << 16)
+#define MESSAGE_TYPE_CLOSE (6 << 16)
+
+struct vmw_msg {
+ u32 id;
+ u32 cookie_high;
+ u32 cookie_low;
+};
+
+static int
+vmware_log_open(struct vmw_msg *msg)
+{
+ u32 info;
+
+ vmware_hypercall6(VMWARE_CMD_MESSAGE | MESSAGE_TYPE_OPEN,
+ RPCI_PROTOCOL_NUM | GUESTMSG_FLAG_COOKIE,
+ 0, &info, &msg->id, &msg->cookie_high,
+ &msg->cookie_low);
+
+ if ((info & MESSAGE_STATUS_SUCCESS) == 0)
+ return 1;
+
+ msg->id &= 0xffff0000UL;
+ return 0;
+}
+
+static int
+vmware_log_close(struct vmw_msg *msg)
+{
+ u32 info;
+
+ vmware_hypercall5(VMWARE_CMD_MESSAGE | MESSAGE_TYPE_CLOSE, 0, msg->id,
+ msg->cookie_high, msg->cookie_low, &info);
+
+ if ((info & MESSAGE_STATUS_SUCCESS) == 0)
+ return 1;
+ return 0;
+}
+
+static int
+vmware_log_send(struct vmw_msg *msg, const char *string)
+{
+ u32 info;
+ u32 len = strlen(string);
+
+retry:
+ vmware_hypercall5(VMWARE_CMD_MESSAGE | MESSAGE_TYPE_SENDSIZE, len,
+ msg->id, msg->cookie_high, msg->cookie_low, &info);
+
+ if (!(info & MESSAGE_STATUS_SUCCESS))
+ return 1;
+
+ /* HB port can't access encrypted memory. */
+ if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT) && (info & MESSAGE_STATUS_HB)) {
+ vmware_hypercall_hb_out(
+ VMWARE_HB_CMD_MESSAGE | MESSAGE_STATUS_SUCCESS,
+ len, msg->id, (uintptr_t) string, msg->cookie_low,
+ msg->cookie_high, &info);
+ } else {
+ do {
+ u32 word;
+ size_t s = min_t(u32, len, sizeof(word));
+
+ memcpy(&word, string, s);
+ len -= s;
+ string += s;
+
+ vmware_hypercall5(VMWARE_CMD_MESSAGE | MESSAGE_TYPE_SEND,
+ word, msg->id, msg->cookie_high,
+ msg->cookie_low, &info);
+ } while (len && (info & MESSAGE_STATUS_SUCCESS));
+ }
+
+ if ((info & MESSAGE_STATUS_SUCCESS) == 0) {
+ if (info & MESSAGE_STATUS_CPT)
+ /* A checkpoint occurred. Retry. */
+ goto retry;
+ return 1;
+ }
+ return 0;
+}
+STACK_FRAME_NON_STANDARD(vmware_log_send);
+
+/*
+ * kmsg_dumper_vmware_log - dumps kmsg to vmware.log file on the host
+ */
+static void kmsg_dumper_vmware_log(struct kmsg_dumper *dumper,
+ struct kmsg_dump_detail *detail)
+{
+ struct vmw_msg msg;
+ struct kmsg_dump_iter iter;
+ static char line[1024];
+ size_t len = 0;
+
+ /* Line prefix to send to VM log file. */
+ line[0] = 'l';
+ line[1] = 'o';
+ line[2] = 'g';
+ line[3] = ' ';
+
+ kmsg_dump_rewind(&iter);
+ while (kmsg_dump_get_line(&iter, true, line + 4, sizeof(line) - 4,
+ &len)) {
+ line[len + 4] = '\0';
+ if (vmware_log_open(&msg))
+ return;
+ if (vmware_log_send(&msg, line))
+ return;
+ vmware_log_close(&msg);
+ }
+}
--
2.43.7
^ permalink raw reply related
* [PATCH v2 1/4] x86/vmware: Introduce common vmware_hypercall()
From: Alexey Makhalov @ 2026-03-09 23:52 UTC (permalink / raw)
To: x86, virtualization, bp, hpa, dave.hansen, mingo, tglx
Cc: ajay.kaher, brennan.lamoreaux, bo.gan, bcm-kernel-feedback-list,
linux-kernel, kas, rick.p.edgecombe, linux-coco, Alexey Makhalov,
Linus Torvalds
In-Reply-To: <20260309235250.2611115-1-alexey.makhalov@broadcom.com>
Introduce vmware_hypercall(), a unified low-bandwidth VMware hypercall
API, and convert the static inlines vmware_hypercallX() family into thin
wrappers on top of it.
vmware_hypercall() is implemented as a static call with four backend
implementations: backdoor, vmcall, vmmcall, and tdxcall. All share the
same logical API but differ in their underlying register mappings.
By updating the static call target early during boot, before the first
hypercall is issued, the !alternatives_patched case no longer needs to
be handled. This allows removal of vmware_hypercall_slow().
The new API implements the widest practical hypercall use case: up to
six input and six output arguments. While this may be slightly less
efficient due to clobbering all six registers and moving unused
arguments - it avoids subtle ABI issues, including cases where other
hypervisors implementing VMware hypercalls corrupt registers.
See QEMU issue #3293 ("vmmouse driver corrupts upper 32 bits of
registers on x86-64") for an example of such behavior.
Additionally, enhance the VMware hypercall ABI documentation in
<asm/vmware.h>.
Link: https://gitlab.com/qemu-project/qemu/-/issues/3293
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
---
arch/x86/include/asm/vmware.h | 274 ++++++++++++++-------------------
arch/x86/kernel/cpu/vmware.c | 276 +++++++++++++++++++---------------
2 files changed, 267 insertions(+), 283 deletions(-)
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 4220dae14a2d..6a084e088b30 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -3,48 +3,84 @@
#define _ASM_X86_VMWARE_H
#include <asm/cpufeatures.h>
-#include <asm/alternative.h>
#include <linux/stringify.h>
+#include <linux/static_call.h>
/*
* VMware hypercall ABI.
*
- * - Low bandwidth (LB) hypercalls (I/O port based, vmcall and vmmcall)
- * have up to 6 input and 6 output arguments passed and returned using
- * registers: %eax (arg0), %ebx (arg1), %ecx (arg2), %edx (arg3),
- * %esi (arg4), %edi (arg5).
- * The following input arguments must be initialized by the caller:
- * arg0 - VMWARE_HYPERVISOR_MAGIC
- * arg2 - Hypercall command
- * arg3 bits [15:0] - Port number, LB and direction flags
+ * - Low bandwidth (LB) hypercalls: I/O port based (aka backdoor), vmcall and
+ * vmmcall have up to 6 input and 6 output on registers arguments, with the
+ * register mapping:
+ * +------+----------------------------------------+-----------------+
+ * | Reg | Input argument | Output argument |
+ * +======+========================================+=================+
+ * | %eax | VMWARE_HYPERVISOR_MAGIC | out0 |
+ * +------+----------------------------------------+-----------------+
+ * | %ebx | (in1) | out1 |
+ * +------+----------------------------------------+-----------------+
+ * | %ecx | (cmd) - Hypercall command | out2 |
+ * +------+----------------------------------------+-----------------+
+ * | %edx | Bits [15:0] - Port number for backdoor | out3 |
+ * | | Zero for vmcall/vmmcall | |
+ * | | Bits [31:16] - (in3) | |
+ * +------+----------------------------------------+-----------------+
+ * | %esi | (in4) | out4 |
+ * +------+----------------------------------------+-----------------+
+ * | %edi | (in5) | out5 |
+ * +------+----------------------------------------+-----------------+
*
- * - Low bandwidth TDX hypercalls (x86_64 only) are similar to LB
- * hypercalls. They also have up to 6 input and 6 output on registers
- * arguments, with different argument to register mapping:
- * %r12 (arg0), %rbx (arg1), %r13 (arg2), %rdx (arg3),
- * %rsi (arg4), %rdi (arg5).
+ * - Low bandwidth TDX hypercalls (x86_64 only) are similar to LB hypercalls.
+ * They also have up to 6 input and 6 output on registers arguments, with
+ * different argument to register mapping:
+ * +------+----------------------------------------+-----------------+
+ * | Reg | Input argument | Output argument |
+ * +======+========================================+=================+
+ * | %r12 | VMWARE_HYPERVISOR_MAGIC | out0 |
+ * +------+----------------------------------------+-----------------+
+ * | %ebx | (in1) | out1 |
+ * +------+----------------------------------------+-----------------+
+ * | %r13 | (cmd) - Hypercall command | out2 |
+ * +------+----------------------------------------+-----------------+
+ * | %edx | Bits [15:0] - Must be zero | out3 |
+ * | | Bits [31:16] - (in3) | |
+ * +------+----------------------------------------+-----------------+
+ * | %esi | (in4) | out4 |
+ * +------+----------------------------------------+-----------------+
+ * | %edi | (in5) | out5 |
+ * +------+----------------------------------------+-----------------+
*
- * - High bandwidth (HB) hypercalls are I/O port based only. They have
- * up to 7 input and 7 output arguments passed and returned using
- * registers: %eax (arg0), %ebx (arg1), %ecx (arg2), %edx (arg3),
- * %esi (arg4), %edi (arg5), %ebp (arg6).
- * The following input arguments must be initialized by the caller:
- * arg0 - VMWARE_HYPERVISOR_MAGIC
- * arg1 - Hypercall command
- * arg3 bits [15:0] - Port number, HB and direction flags
+ * - High bandwidth (HB) hypercalls are I/O port based only. They have up to 7
+ * input and 7 output on reegister arguments with the following mapping:
+ * +------+----------------------------------------+-----------------+
+ * | Reg | Input argument | Output argument |
+ * +======+========================================+=================+
+ * | %eax | VMWARE_HYPERVISOR_MAGIC | out0 |
+ * +------+----------------------------------------+-----------------+
+ * | %ebx | (cmd) - Hypercall command | out1 |
+ * +------+----------------------------------------+-----------------+
+ * | %ebx | (in2) | out2 |
+ * +------+----------------------------------------+-----------------+
+ * | %edx | Bits [15:0] - Port number and HB flag | out3 |
+ * | | Bits [31:16] - (in3) | |
+ * +------+----------------------------------------+-----------------+
+ * | %esi | (in4) | out4 |
+ * +------+----------------------------------------+-----------------+
+ * | %edi | (in5) | out5 |
+ * +------+----------------------------------------+-----------------+
+ * | %ebp | (in6) | out6 |
+ * +------+----------------------------------------+-----------------+
*
- * For compatibility purposes, x86_64 systems use only lower 32 bits
- * for input and output arguments.
+ * For compatibility purposes, x86_64 systems use only lower 32 bits for input
+ * and output arguments.
*
- * The hypercall definitions differ in the low word of the %edx (arg3)
- * in the following way: the old I/O port based interface uses the port
- * number to distinguish between high- and low bandwidth versions, and
- * uses IN/OUT instructions to define transfer direction.
+ * The hypercall definitions differ in the low word of the %edx (arg3) in the
+ * following way: the old I/O port based interface uses the port number, the
+ * bandwidth mode flag, and uses IN/OUT instructions to define transfer
+ * direction.
*
- * The new vmcall interface instead uses a set of flags to select
- * bandwidth mode and transfer direction. The flags should be loaded
- * into arg3 by any user and are automatically replaced by the port
- * number if the I/O port method is used.
+ * The new vmcall interface instead uses a set of flags to select bandwidth
+ * mode and transfer direction.
*/
#define VMWARE_HYPERVISOR_HB BIT(0)
@@ -70,103 +106,64 @@
#define CPUID_VMWARE_FEATURES_ECX_VMMCALL BIT(0)
#define CPUID_VMWARE_FEATURES_ECX_VMCALL BIT(1)
-extern unsigned long vmware_hypercall_slow(unsigned long cmd,
- unsigned long in1, unsigned long in3,
- unsigned long in4, unsigned long in5,
- u32 *out1, u32 *out2, u32 *out3,
- u32 *out4, u32 *out5);
-
#define VMWARE_TDX_VENDOR_LEAF 0x1af7e4909ULL
#define VMWARE_TDX_HCALL_FUNC 1
-extern unsigned long vmware_tdx_hypercall(unsigned long cmd,
- unsigned long in1, unsigned long in3,
- unsigned long in4, unsigned long in5,
- u32 *out1, u32 *out2, u32 *out3,
- u32 *out4, u32 *out5);
+unsigned long dummy_vmware_hypercall(unsigned long cmd,
+ unsigned long in1, unsigned long in3,
+ unsigned long in4, unsigned long in5,
+ u32 *out1, u32 *out2, u32 *out3,
+ u32 *out4, u32 *out5);
/*
- * The low bandwidth call. The low word of %edx is presumed to have OUT bit
- * set. The high word of %edx may contain input data from the caller.
+ * Low bandwidth (LB) VMware hypercall.
+ *
+ * It is backed by the backdoor, vmcall, vmmcall or tdx call implementation.
+ *
+ * Use inX/outX arguments naming as the register mappings vary between
+ * different implementations. See VMware hypercall ABI above.
+ * These 10 arguments could be nicely wrapped in in/out structures, but it
+ * will introduce unnecessary structs copy in vmware_tdx_hypercall().
+ *
+ * NOTE:
+ * Do not merge vmware_{backdoor,vmcall,vmmcall}_hypercall implementations
+ * using alternative instructions. Such patching mechanism can not be used
+ * in vmware_hypercall path, as the first hypercall will be called much
+ * before the apply_alternatives(). See vmware_platform_setup().
*/
-#define VMWARE_HYPERCALL \
- ALTERNATIVE_2("movw %[port], %%dx\n\t" \
- "inl (%%dx), %%eax", \
- "vmcall", X86_FEATURE_VMCALL, \
- "vmmcall", X86_FEATURE_VMW_VMMCALL)
+DECLARE_STATIC_CALL(vmware_hypercall, dummy_vmware_hypercall);
+/*
+ * Set of commonly used vmware_hypercallX functions - wrappers on top of the
+ * vmware_hypercall.
+ */
static inline
unsigned long vmware_hypercall1(unsigned long cmd, unsigned long in1)
{
- unsigned long out0;
-
- if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
- return vmware_tdx_hypercall(cmd, in1, 0, 0, 0,
- NULL, NULL, NULL, NULL, NULL);
-
- if (unlikely(!alternatives_patched) && !__is_defined(MODULE))
- return vmware_hypercall_slow(cmd, in1, 0, 0, 0,
- NULL, NULL, NULL, NULL, NULL);
+ u32 out1, out2, out3, out4, out5;
- asm_inline volatile (VMWARE_HYPERCALL
- : "=a" (out0)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (0)
- : "cc", "memory");
- return out0;
+ return static_call_mod(vmware_hypercall)(cmd, in1, 0, 0, 0,
+ &out1, &out2, &out3, &out4, &out5);
}
static inline
unsigned long vmware_hypercall3(unsigned long cmd, unsigned long in1,
u32 *out1, u32 *out2)
{
- unsigned long out0;
-
- if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
- return vmware_tdx_hypercall(cmd, in1, 0, 0, 0,
- out1, out2, NULL, NULL, NULL);
-
- if (unlikely(!alternatives_patched) && !__is_defined(MODULE))
- return vmware_hypercall_slow(cmd, in1, 0, 0, 0,
- out1, out2, NULL, NULL, NULL);
+ u32 out3, out4, out5;
- asm_inline volatile (VMWARE_HYPERCALL
- : "=a" (out0), "=b" (*out1), "=c" (*out2)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (0)
- : "di", "si", "cc", "memory");
- return out0;
+ return static_call_mod(vmware_hypercall)(cmd, in1, 0, 0, 0,
+ out1, out2, &out3, &out4, &out5);
}
static inline
unsigned long vmware_hypercall4(unsigned long cmd, unsigned long in1,
u32 *out1, u32 *out2, u32 *out3)
{
- unsigned long out0;
-
- if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
- return vmware_tdx_hypercall(cmd, in1, 0, 0, 0,
- out1, out2, out3, NULL, NULL);
-
- if (unlikely(!alternatives_patched) && !__is_defined(MODULE))
- return vmware_hypercall_slow(cmd, in1, 0, 0, 0,
- out1, out2, out3, NULL, NULL);
+ u32 out4, out5;
- asm_inline volatile (VMWARE_HYPERCALL
- : "=a" (out0), "=b" (*out1), "=c" (*out2), "=d" (*out3)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (0)
- : "di", "si", "cc", "memory");
- return out0;
+ return static_call_mod(vmware_hypercall)(cmd, in1, 0, 0, 0,
+ out1, out2, out3, &out4, &out5);
}
static inline
@@ -174,27 +171,10 @@ unsigned long vmware_hypercall5(unsigned long cmd, unsigned long in1,
unsigned long in3, unsigned long in4,
unsigned long in5, u32 *out2)
{
- unsigned long out0;
-
- if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
- return vmware_tdx_hypercall(cmd, in1, in3, in4, in5,
- NULL, out2, NULL, NULL, NULL);
+ u32 out1, out3, out4, out5;
- if (unlikely(!alternatives_patched) && !__is_defined(MODULE))
- return vmware_hypercall_slow(cmd, in1, in3, in4, in5,
- NULL, out2, NULL, NULL, NULL);
-
- asm_inline volatile (VMWARE_HYPERCALL
- : "=a" (out0), "=c" (*out2)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (in3),
- "S" (in4),
- "D" (in5)
- : "cc", "memory");
- return out0;
+ return static_call_mod(vmware_hypercall)(cmd, in1, in3, in4, in5,
+ &out1, out2, &out3, &out4, &out5);
}
static inline
@@ -202,26 +182,10 @@ unsigned long vmware_hypercall6(unsigned long cmd, unsigned long in1,
unsigned long in3, u32 *out2,
u32 *out3, u32 *out4, u32 *out5)
{
- unsigned long out0;
-
- if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
- return vmware_tdx_hypercall(cmd, in1, in3, 0, 0,
- NULL, out2, out3, out4, out5);
+ u32 out1;
- if (unlikely(!alternatives_patched) && !__is_defined(MODULE))
- return vmware_hypercall_slow(cmd, in1, in3, 0, 0,
- NULL, out2, out3, out4, out5);
-
- asm_inline volatile (VMWARE_HYPERCALL
- : "=a" (out0), "=c" (*out2), "=d" (*out3), "=S" (*out4),
- "=D" (*out5)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (in3)
- : "cc", "memory");
- return out0;
+ return static_call_mod(vmware_hypercall)(cmd, in1, in3, 0, 0,
+ &out1, out2, out3, out4, out5);
}
static inline
@@ -230,27 +194,10 @@ unsigned long vmware_hypercall7(unsigned long cmd, unsigned long in1,
unsigned long in5, u32 *out1,
u32 *out2, u32 *out3)
{
- unsigned long out0;
-
- if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
- return vmware_tdx_hypercall(cmd, in1, in3, in4, in5,
- out1, out2, out3, NULL, NULL);
+ u32 out4, out5;
- if (unlikely(!alternatives_patched) && !__is_defined(MODULE))
- return vmware_hypercall_slow(cmd, in1, in3, in4, in5,
- out1, out2, out3, NULL, NULL);
-
- asm_inline volatile (VMWARE_HYPERCALL
- : "=a" (out0), "=b" (*out1), "=c" (*out2), "=d" (*out3)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (in3),
- "S" (in4),
- "D" (in5)
- : "cc", "memory");
- return out0;
+ return static_call_mod(vmware_hypercall)(cmd, in1, in3, in4, in5,
+ out1, out2, out3, &out4, &out5);
}
#ifdef CONFIG_X86_64
@@ -322,6 +269,5 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2,
return out0;
}
#undef VMW_BP_CONSTRAINT
-#undef VMWARE_HYPERCALL
#endif
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index a3e6936839b1..93acd3414e37 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -64,70 +64,140 @@ struct vmware_steal_time {
};
static unsigned long vmware_tsc_khz __ro_after_init;
-static u8 vmware_hypercall_mode __ro_after_init;
-
-unsigned long vmware_hypercall_slow(unsigned long cmd,
- unsigned long in1, unsigned long in3,
- unsigned long in4, unsigned long in5,
- u32 *out1, u32 *out2, u32 *out3,
- u32 *out4, u32 *out5)
-{
- unsigned long out0, rbx, rcx, rdx, rsi, rdi;
-
- switch (vmware_hypercall_mode) {
- case CPUID_VMWARE_FEATURES_ECX_VMCALL:
- asm_inline volatile ("vmcall"
- : "=a" (out0), "=b" (rbx), "=c" (rcx),
- "=d" (rdx), "=S" (rsi), "=D" (rdi)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (in3),
- "S" (in4),
- "D" (in5)
- : "cc", "memory");
- break;
- case CPUID_VMWARE_FEATURES_ECX_VMMCALL:
- asm_inline volatile ("vmmcall"
- : "=a" (out0), "=b" (rbx), "=c" (rcx),
- "=d" (rdx), "=S" (rsi), "=D" (rdi)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (in3),
- "S" (in4),
- "D" (in5)
- : "cc", "memory");
- break;
- default:
- asm_inline volatile ("movw %[port], %%dx; inl (%%dx), %%eax"
- : "=a" (out0), "=b" (rbx), "=c" (rcx),
- "=d" (rdx), "=S" (rsi), "=D" (rdi)
- : [port] "i" (VMWARE_HYPERVISOR_PORT),
- "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (in1),
- "c" (cmd),
- "d" (in3),
- "S" (in4),
- "D" (in5)
- : "cc", "memory");
- break;
- }
+static u8 vmware_hypercall_mode __initdata;
+
+static unsigned long vmware_backdoor_hypercall(unsigned long cmd,
+ unsigned long in1, unsigned long in3,
+ unsigned long in4, unsigned long in5,
+ u32 *out1, u32 *out2, u32 *out3,
+ u32 *out4, u32 *out5)
+{
+ unsigned long out0;
+
+ /* The low word of in3(%edx) must have the backdoor port number */
+ in3 = (in3 & ~0xffff) | VMWARE_HYPERVISOR_PORT;
+
+ asm_inline volatile ("inl (%%dx), %%eax"
+ : "=a" (out0), "=b" (*out1), "=c" (*out2),
+ "=d" (*out3), "=S" (*out4), "=D" (*out5)
+ : "a" (VMWARE_HYPERVISOR_MAGIC),
+ "b" (in1),
+ "c" (cmd),
+ "d" (in3),
+ "S" (in4),
+ "D" (in5)
+ : "cc", "memory");
- if (out1)
- *out1 = rbx;
- if (out2)
- *out2 = rcx;
- if (out3)
- *out3 = rdx;
- if (out4)
- *out4 = rsi;
- if (out5)
- *out5 = rdi;
+ return out0;
+}
+
+static unsigned long vmware_vmcall_hypercall(unsigned long cmd,
+ unsigned long in1, unsigned long in3,
+ unsigned long in4, unsigned long in5,
+ u32 *out1, u32 *out2, u32 *out3,
+ u32 *out4, u32 *out5)
+{
+ unsigned long out0;
+
+ /* The low word of in3(%edx) must be zero: LB, IN */
+ in3 &= ~0xffff;
+
+ asm_inline volatile ("vmcall"
+ : "=a" (out0), "=b" (*out1), "=c" (*out2),
+ "=d" (*out3), "=S" (*out4), "=D" (*out5)
+ : "a" (VMWARE_HYPERVISOR_MAGIC),
+ "b" (in1),
+ "c" (cmd),
+ "d" (in3),
+ "S" (in4),
+ "D" (in5)
+ : "cc", "memory");
return out0;
}
+static unsigned long vmware_vmmcall_hypercall(unsigned long cmd,
+ unsigned long in1, unsigned long in3,
+ unsigned long in4, unsigned long in5,
+ u32 *out1, u32 *out2, u32 *out3,
+ u32 *out4, u32 *out5)
+{
+ unsigned long out0;
+
+ /* The low word of in3(%edx) must be zero: LB, IN */
+ in3 &= ~0xffff;
+
+ asm_inline volatile ("vmmcall"
+ : "=a" (out0), "=b" (*out1), "=c" (*out2),
+ "=d" (*out3), "=S" (*out4), "=D" (*out5)
+ : "a" (VMWARE_HYPERVISOR_MAGIC),
+ "b" (in1),
+ "c" (cmd),
+ "d" (in3),
+ "S" (in4),
+ "D" (in5)
+ : "cc", "memory");
+
+ return out0;
+}
+
+/*
+ * TDCALL[TDG.VP.VMCALL] uses %rax (arg0) and %rcx (arg2). Therefore,
+ * we remap those registers to %r12 and %r13, respectively.
+ */
+static unsigned long vmware_tdx_hypercall(unsigned long cmd,
+ unsigned long in1, unsigned long in3,
+ unsigned long in4, unsigned long in5,
+ u32 *out1, u32 *out2, u32 *out3,
+ u32 *out4, u32 *out5)
+{
+#ifdef CONFIG_INTEL_TDX_GUEST
+ struct tdx_module_args args = {};
+
+ if (!hypervisor_is_type(X86_HYPER_VMWARE)) {
+ pr_warn_once("Incorrect usage\n");
+ return ULONG_MAX;
+ }
+
+ if (cmd & ~VMWARE_CMD_MASK) {
+ pr_warn_once("Out of range command %lx\n", cmd);
+ return ULONG_MAX;
+ }
+
+ args.rbx = in1;
+ /* The low word of in3(%rdx) must be zero: LB, IN */
+ args.rdx = in3 & ~0xffff;
+ args.rsi = in4;
+ args.rdi = in5;
+ args.r10 = VMWARE_TDX_VENDOR_LEAF;
+ args.r11 = VMWARE_TDX_HCALL_FUNC;
+ args.r12 = VMWARE_HYPERVISOR_MAGIC;
+ args.r13 = cmd;
+ /* CPL */
+ args.r15 = 0;
+
+ __tdx_hypercall(&args);
+
+ *out1 = args.rbx;
+ *out2 = args.r13;
+ *out3 = args.rdx;
+ *out4 = args.rsi;
+ *out5 = args.rdi;
+
+ return args.r12;
+#else
+ return ULONG_MAX;
+#endif
+}
+
+
+DEFINE_STATIC_CALL(vmware_hypercall, vmware_backdoor_hypercall);
+EXPORT_STATIC_CALL_GPL(vmware_hypercall);
+
+/*
+ * Perform backdoor probbing of the hypervisor when
+ * X86_FEATURE_HYPERVISOR bit is not set.
+ */
static inline int __vmware_platform(void)
{
u32 eax, ebx, ecx;
@@ -397,11 +467,35 @@ static void __init vmware_set_capabilities(void)
setup_force_cpu_cap(X86_FEATURE_VMW_VMMCALL);
}
+static void __init vmware_select_hypercall(void)
+{
+ char *mode;
+
+ if (IS_ENABLED(CONFIG_INTEL_TDX_GUEST) &&
+ cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) {
+ static_call_update(vmware_hypercall, vmware_tdx_hypercall);
+ mode = "tdcall";
+ } else if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMCALL) {
+ static_call_update(vmware_hypercall, vmware_vmcall_hypercall);
+ mode = "vmcall";
+ } else if (vmware_hypercall_mode == CPUID_VMWARE_FEATURES_ECX_VMMCALL) {
+ static_call_update(vmware_hypercall, vmware_vmmcall_hypercall);
+ mode = "vmmcall";
+ } else {
+ mode = "backdoor";
+ }
+
+ pr_info("hypercall mode: %s\n", mode);
+}
+
static void __init vmware_platform_setup(void)
{
u32 eax, ebx, ecx;
u64 lpj, tsc_khz;
+ /* Update vmware_hypercall() before the first use. */
+ vmware_select_hypercall();
+
eax = vmware_hypercall3(VMWARE_CMD_GETHZ, UINT_MAX, &ebx, &ecx);
if (ebx != UINT_MAX) {
@@ -443,7 +537,7 @@ static void __init vmware_platform_setup(void)
vmware_set_capabilities();
}
-static u8 __init vmware_select_hypercall(void)
+static u8 __init get_hypercall_mode(void)
{
int eax, ebx, ecx, edx;
@@ -456,8 +550,8 @@ static u8 __init vmware_select_hypercall(void)
* While checking the dmi string information, just checking the product
* serial key should be enough, as this will always have a VMware
* specific string when running under VMware hypervisor.
- * If !boot_cpu_has(X86_FEATURE_HYPERVISOR), vmware_hypercall_mode
- * intentionally defaults to 0.
+ * If !boot_cpu_has(X86_FEATURE_HYPERVISOR), __vmware_platform()
+ * intentionally defaults to backdoor hypercall.
*/
static u32 __init vmware_platform(void)
{
@@ -470,11 +564,7 @@ static u32 __init vmware_platform(void)
if (!memcmp(hyper_vendor_id, "VMwareVMware", 12)) {
if (eax >= CPUID_VMWARE_FEATURES_LEAF)
vmware_hypercall_mode =
- vmware_select_hypercall();
-
- pr_info("hypercall mode: 0x%02x\n",
- (unsigned int) vmware_hypercall_mode);
-
+ get_hypercall_mode();
return CPUID_VMWARE_INFO_LEAF;
}
} else if (dmi_available && dmi_name_in_serial("VMware") &&
@@ -494,58 +584,6 @@ static bool __init vmware_legacy_x2apic_available(void)
(eax & GETVCPU_INFO_LEGACY_X2APIC);
}
-#ifdef CONFIG_INTEL_TDX_GUEST
-/*
- * TDCALL[TDG.VP.VMCALL] uses %rax (arg0) and %rcx (arg2). Therefore,
- * we remap those registers to %r12 and %r13, respectively.
- */
-unsigned long vmware_tdx_hypercall(unsigned long cmd,
- unsigned long in1, unsigned long in3,
- unsigned long in4, unsigned long in5,
- u32 *out1, u32 *out2, u32 *out3,
- u32 *out4, u32 *out5)
-{
- struct tdx_module_args args = {};
-
- if (!hypervisor_is_type(X86_HYPER_VMWARE)) {
- pr_warn_once("Incorrect usage\n");
- return ULONG_MAX;
- }
-
- if (cmd & ~VMWARE_CMD_MASK) {
- pr_warn_once("Out of range command %lx\n", cmd);
- return ULONG_MAX;
- }
-
- args.rbx = in1;
- args.rdx = in3;
- args.rsi = in4;
- args.rdi = in5;
- args.r10 = VMWARE_TDX_VENDOR_LEAF;
- args.r11 = VMWARE_TDX_HCALL_FUNC;
- args.r12 = VMWARE_HYPERVISOR_MAGIC;
- args.r13 = cmd;
- /* CPL */
- args.r15 = 0;
-
- __tdx_hypercall(&args);
-
- if (out1)
- *out1 = args.rbx;
- if (out2)
- *out2 = args.r13;
- if (out3)
- *out3 = args.rdx;
- if (out4)
- *out4 = args.rsi;
- if (out5)
- *out5 = args.rdi;
-
- return args.r12;
-}
-EXPORT_SYMBOL_GPL(vmware_tdx_hypercall);
-#endif
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
static void vmware_sev_es_hcall_prepare(struct ghcb *ghcb,
struct pt_regs *regs)
--
2.43.7
^ permalink raw reply related
* [PATCH v2 0/4] x86/vmware: Hypercall refactoring and improved guest support
From: Alexey Makhalov @ 2026-03-09 23:52 UTC (permalink / raw)
To: x86, virtualization, bp, hpa, dave.hansen, mingo, tglx
Cc: ajay.kaher, brennan.lamoreaux, bo.gan, bcm-kernel-feedback-list,
linux-kernel, kas, rick.p.edgecombe, linux-coco, Alexey Makhalov
In-Reply-To: <20260307004238.1181299-1-alexey.makhalov@broadcom.com>
This series improves VMware guest support on x86 by refactoring the
hypercall infrastructure and adding better crash diagnostics, along
with encrypted guest support for the steal time clock.
The first patch introduces a common vmware_hypercall() backend selected
via static calls. It consolidates the existing hypercall mechanisms
(backdoor, VMCALL/VMMCALL, and TDX) behind a single interface and
selects the optimal implementation at boot. This reduces duplication
and simplifies future extensions.
Building on top of the new hypercall infrastructure, the next two
patches improve post-mortem debugging of VMware guests. They export
panic information to the hypervisor by dumping kernel messages to the
VM vmware.log on the host and explicitly reporting guest crash event
to the hypervisor.
The final patch adds support for encrypted guests by ensuring that the
shared memory used for the steal time clock is mapped as decrypted
before being shared with the hypervisor. This enables steal time
accounting to function correctly when guest memory encryption is
enabled.
Patch overview:
1. x86/vmware: Introduce common vmware_hypercall
* Consolidate hypercall implementations behind a common API
* Select backend via static_call at boot
2. x86/vmware: Log kmsg dump on panic
* Register a kmsg dumper
* Export panic logs to the host
3. x86/vmware: Report guest crash to the hypervisor
* Register a panic notifier
* Notify the hypervisor about guest crashes
4. x86/vmware: Support steal time clock for encrypted guests
* Mark shared steal time memory as decrypted early in boot
Changelog:
V1 -> V2
* Fix compilation warnings in patch 2 "x86/vmware: Log kmsg dump on panic"
reported by kernel test robot <lkp@intel.com>
Alexey Makhalov (4):
x86/vmware: Introduce common vmware_hypercall()
x86/vmware: Log kmsg dump on panic
x86/vmware: Report guest crash to the hypervisor
x86/vmware: Support steal time clock for encrypted guests
arch/x86/include/asm/vmware.h | 276 ++++++++------------
arch/x86/kernel/cpu/vmware.c | 470 +++++++++++++++++++++++++---------
2 files changed, 463 insertions(+), 283 deletions(-)
base-commit: 7d08a6ad25f85c9bb7d0382142838cb54713f1a3
--
2.43.7
^ permalink raw reply
* Re: [PATCH net-next v3 1/2] dma-mapping: introduce DMA_ATTR_CC_DECRYPTED for pre-decrypted memory
From: Jiri Pirko @ 2026-03-09 17:51 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Leon Romanovsky, dri-devel, linaro-mm-sig, iommu, linux-media,
sumit.semwal, benjamin.gaignard, Brian.Starkey, jstultz,
tjmercier, christian.koenig, m.szyprowski, robin.murphy,
sean.anderson, ptesarik, catalin.marinas, aneesh.kumar,
suzuki.poulose, steven.price, thomas.lendacky, john.allen,
ashish.kalra, suravee.suthikulpanit, linux-coco
In-Reply-To: <20260309151857.GO1687929@ziepe.ca>
Mon, Mar 09, 2026 at 04:18:57PM +0100, jgg@ziepe.ca wrote:
>On Mon, Mar 09, 2026 at 04:02:33PM +0200, Leon Romanovsky wrote:
>> On Mon, Mar 09, 2026 at 10:15:30AM -0300, Jason Gunthorpe wrote:
>> > On Sun, Mar 08, 2026 at 12:19:48PM +0200, Leon Romanovsky wrote:
>> >
>> > > > +/*
>> > > > + * DMA_ATTR_CC_DECRYPTED: Indicates memory that has been explicitly decrypted
>> > > > + * (shared) for confidential computing guests. The caller must have
>> > > > + * called set_memory_decrypted(). A struct page is required.
>> > > > + */
>> > > > +#define DMA_ATTR_CC_DECRYPTED (1UL << 12)
>> > >
>> > > While adding the new attribute is fine, I would expect additional checks in
>> > > dma_map_phys() to ensure the attribute cannot be misused. For example,
>> > > WARN_ON(attrs & (DMA_ATTR_CC_DECRYPTED | DMA_ATTR_MMIO)), along with a check
>> > > that we are taking the direct path only.
>> >
>> > DECRYPYED and MMIO is something that needs to work, VFIO (inside a
>> > TVM) should be using that combination.
>>
>> So this sentence "A struct page is required" from the comment above is
>> not accurate.
>
>It would be clearer to say "Unless DMA_ATTR_MMIO is provided a struct
>page is required"
>
>We need to audit if that works properly, IIRC it does, but I don't
>remember.. Jiri?
How can you do set_memory_decrypted if you don't have page/folio ?
^ permalink raw reply
* Re: [PATCH v2 05/19] device core: Autoprobe considered harmful?
From: Jonathan Cameron @ 2026-03-09 16:58 UTC (permalink / raw)
To: Dan Williams
Cc: linux-coco, linux-pci, gregkh, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, jgg, Christoph Hellwig,
Marek Szyprowski, Robin Murphy, Roman Kisel, Samuel Ortiz,
Rafael J. Wysocki, Danilo Krummrich
In-Reply-To: <20260303000207.1836586-6-dan.j.williams@intel.com>
On Mon, 2 Mar 2026 16:01:53 -0800
Dan Williams <dan.j.williams@intel.com> wrote:
> The threat model of PCI Trusted Execution Environment Device Interface
> Security Protocol (TDISP), is that an adversary may be impersonating the
> device's identity, redirecting the device's MMIO interface, and/or
> snooping/manipulating the physical link. Outside of PCI TDISP, PCI ATS
> (that allows IOMMU bypass) comes to mind as another threat vector that
> warrants additional device verification beyond whether ACPI enumerates the
> device as "internal" [1].
>
> The process of verifying a device ranges from the traditional default
> "accept everything" to gathering signed evidence from a locked device,
> shipping it to a relying party and acting on that disposition. That policy
> belongs in userspace. A natural way for userspace to get a control point
> for verifying a device before use is when the driver for the device comes
> from a module.
>
> For deployments that are concerned about adversarial devices, introduce a
> mechanism to disable autoprobe. When a driver originates from a module,
> consult that driver's autoprobe policy at initial device or driver attach.
>
> Note that with TDISP, unaccepted devices do not have access to private
> memory (so called "T=0" mode). However, a deployment may still not want to
> operate more than a handful of boot devices until confirming the system
> device topology with a verifier.
>
> Yes, this is a security vs convenience tradeoff. Yes, devices with
> non-modular drivers are out of scope. Yes, there are known regression cases
> for subsystems where device objects are expected to auto-attach outside of
> fatal probe failure. For navigating regressions, a per-module "autoprobe"
> option is included to allow fine grained policy.
>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Jason Gunthorpe <jgg@nvidia.com>
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Roman Kisel <romank@linux.microsoft.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Samuel Ortiz <sameo@rivosinc.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Xu Yilun <yilun.xu@linux.intel.com>
> Cc: "Aneesh Kumar K.V" <aneesh.kumar@kernel.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: "Rafael J. Wysocki" <rafael@kernel.org>
> Cc: Danilo Krummrich <dakr@kernel.org>
> Link: http://lore.kernel.org/6971b9406d069_1d33100df@dwillia2-mobl4.notmuch [1]
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Approach seems reasonable to me.
A few trivial things inline.
Jonathan
> ---
> drivers/base/Kconfig | 24 ++++++++++++++++++++++++
> Documentation/ABI/stable/sysfs-module | 10 ++++++++++
> drivers/base/base.h | 1 +
> include/linux/module.h | 14 ++++++++++++++
> drivers/base/bus.c | 7 ++++++-
> drivers/base/dd.c | 26 +++++++++++++++++++++++---
> kernel/module/main.c | 13 +++++++++++++
> 7 files changed, 91 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> index d4743bf978ec..7c1da5df9745 100644
> --- a/drivers/base/Kconfig
> +++ b/drivers/base/Kconfig
> @@ -253,4 +253,28 @@ config CONFIDENTIAL_DEVICES
> depends on ARCH_HAS_CC_PLATFORM
> bool
>
> +config MODULES_AUTOPROBE
> + bool "Automatic probe of drivers from modules"
> + default y
> + help
> + Say Y for the typical and traditional Linux behavior of automatically
> + attaching devices to drivers when a module is loaded.
> +
> + Say N to opt into a threat model where userspace verification of a
> + device is required before driver attach. This includes Confidential
> + Computing use cases where the device needs to have its configuration
> + locked and verified by a relying party. It also includes use cases
> + like leaving devices with Address Translation (IOMMU protection
> + bypass) capability disabled until userspace attests the device and
> + binds a driver.
> +
> + This default value can be overridden by the "autoprobe" module option.
> + Note that some subsystems may not be prepared for autoprobe to be
> + disabled, take care to test your selected drivers. Built-in drivers are
Stray extra space.
> + unaffected by this policy and will autoprobe unless the bus itself has
> + disabled autoprobe.
> +
> + If in doubt, say Y. The N case is only for expert configurations, and
> + selective "autoprobe=0" in modprobe policy is the common expectation.
> +
> endmenu
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index 349f31bedfa1..926e120b3cc4 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -917,6 +917,12 @@ struct device_attach_data {
> * driver, we'll encounter one that requests asynchronous probing.
> */
> bool have_async;
> +
> + /*
> + * On initial device arrival driver attach is subject to
> + * driver_autoprobe() policy.
> + */
> + bool initial_probe;
I'm not sure 'initial' naming works. How does that work with drivers that
have not autobound anyway? E.g. VFIO.
Seems they'll be fine even if it is their initial probe. Also the
deferred cases remain 'initial' for repeated probing.
Why not stick to the auto probing terminology here?
There is clearly history of this naming though in device_initial_probe()
So maybe that name is fine...
> };
>
>
^ permalink raw reply
* Re: [PATCH v2 03/19] device core: Introduce confidential device acceptance
From: Jonathan Cameron @ 2026-03-09 16:42 UTC (permalink / raw)
To: Dan Williams
Cc: linux-coco, linux-pci, gregkh, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, jgg, Christoph Hellwig,
Jason Gunthorpe, Marek Szyprowski, Robin Murphy, Roman Kisel,
Samuel Ortiz, Rafael J. Wysocki, Danilo Krummrich
In-Reply-To: <20260303000207.1836586-4-dan.j.williams@intel.com>
On Mon, 2 Mar 2026 16:01:51 -0800
Dan Williams <dan.j.williams@intel.com> wrote:
> An "accepted" device is one that is allowed to access private memory within
> a Trusted Computing Boundary (TCB). The concept of "acceptance" is distinct
> from other device properties like usb_device::authorized, or
> tb_switch::authorized. The entry to the accepted state is a violent
> operation in which the device will reject MMIO requests that are not
> encrypted, and the device enters a new IOMMU protection domain to allow it
> to access addresses that were previously off-limits.
>
> Subsystems like the DMA mapping layer, that need to modify their behavior
> based on the accept state, may only have access to the base 'struct
> device'. It is also likely that the concept of TCB acceptance grows beyond
> PCI devices over time. For these reasons, introduce the concept of
> acceptance in 'struct device_private' which is device common, but only
> suitable for buses and built-in infrastructure to consume.
>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Roman Kisel <romank@linux.microsoft.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Samuel Ortiz <sameo@rivosinc.com>
> Cc: Alexey Kardashevskiy <aik@amd.com>
> Cc: Xu Yilun <yilun.xu@linux.intel.com>
> Cc: "Aneesh Kumar K.V" <aneesh.kumar@kernel.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: "Rafael J. Wysocki" <rafael@kernel.org>
> Cc: Danilo Krummrich <dakr@kernel.org>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Seems reasonable to me.
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
^ permalink raw reply
* Re: [PATCH v2 02/19] device core: Fix kernel-doc warnings in base.h
From: Jonathan Cameron @ 2026-03-09 16:39 UTC (permalink / raw)
To: Dan Williams
Cc: linux-coco, linux-pci, gregkh, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, jgg
In-Reply-To: <20260303000207.1836586-3-dan.j.williams@intel.com>
On Mon, 2 Mar 2026 16:01:50 -0800
Dan Williams <dan.j.williams@intel.com> wrote:
> In preparation for adding new fields to 'struct device_private' fix up
> existing kernel-doc warnings in this header file of the form:
>
> Warning: drivers/base/base.h:59 struct member 'subsys' not described in
> 'subsys_private'
> Warning: drivers/base/base.h:59 struct member 'devices_kset' not described
> in 'subsys_private'
> Warning: drivers/base/base.h:59 struct member 'interfaces' not described in
> 'subsys_private'
> Warning: drivers/base/base.h:59 struct member 'mutex' not described in
> 'subsys_private'
>
> ...which are simple replacements of " - " with ": ".
>
> Add new descriptions for these previously undescribed fields:
>
> Warning: drivers/base/base.h:58 struct member 'drivers_autoprobe' not
> described in 'subsys_private'
> Warning: drivers/base/base.h:117 struct member 'deferred_probe_reason' not
> described in 'device_private'
>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Maybe if the rest looks 'slow' can send this one ahead?
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Jonathan
^ permalink raw reply
* Re: [PATCH v2 01/19] PCI/TSM: Report active IDE streams per host bridge
From: Jonathan Cameron @ 2026-03-09 16:36 UTC (permalink / raw)
To: Dan Williams
Cc: linux-coco, linux-pci, gregkh, aik, aneesh.kumar, yilun.xu,
bhelgaas, alistair23, lukas, jgg
In-Reply-To: <20260303000207.1836586-2-dan.j.williams@intel.com>
On Mon, 2 Mar 2026 16:01:49 -0800
Dan Williams <dan.j.williams@intel.com> wrote:
> The first attempt at an ABI for this failed to account for naming
> collisions across host bridges:
>
> Commit a4438f06b1db ("PCI/TSM: Report active IDE streams")
>
> Revive this ABI with a per host bridge link that appears at first stream
> creation for a given host bridge and disappears after the last stream is
> removed.
>
> For systems with many host bridge objects it allows:
>
> ls /sys/class/tsm/tsmN/pci*/stream*
>
> ...to find all the host bridges with active streams without first iterating
> over all host bridges. Yilun notes that is handy to have this short cut [1]
> and from an administrator perspective it helps with inventory for
> constrained stream resources.
>
> Link: http://lore.kernel.org/aXLtILY85oMU5qlb@yilunxu-OptiPlex-7050 [1]
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
> index 8712df8596a1..3c99c38cfaa5 100644
> --- a/drivers/virt/coco/tsm-core.c
> +++ b/drivers/virt/coco/tsm-core.c
> @@ -4,10 +4,12 @@
> +
> +static struct tsm_ide_stream *create_streams(struct tsm_dev *tsm_dev,
> + struct pci_host_bridge *bridge)
> +{
> + int rc;
> +
> + struct tsm_ide_stream *streams __free(kfree) =
> + kzalloc(sizeof(*streams), GFP_KERNEL);
Crossed with kzalloc_obj() etc being introduced which seems appropriate here.
> + if (!streams)
> + return NULL;
> +
> + streams->tsm_dev = tsm_dev;
> + streams->bridge = bridge;
> + kref_init(&streams->kref);
> + rc = xa_insert(&tsm_ide_streams, (unsigned long)bridge, streams,
> + GFP_KERNEL);
> + if (rc)
> + return NULL;
> +
> + rc = sysfs_create_link(&tsm_dev->dev.kobj, &bridge->dev.kobj,
> + dev_name(&bridge->dev));
> + if (rc) {
> + xa_erase(&tsm_ide_streams, (unsigned long)bridge);
> + return NULL;
> + }
> +
> + return no_free_ptr(streams);
> +}
^ permalink raw reply
* Re: [PATCH 4/4] KVM: x86: Disable the TDX module during kexec and kdump
From: Edgecombe, Rick P @ 2026-03-09 16:24 UTC (permalink / raw)
To: Gao, Chao
Cc: tglx@kernel.org, Hansen, Dave, seanjc@google.com, bp@alien8.de,
kas@kernel.org, ackerleytng@google.com, hpa@zytor.com,
linux-kernel@vger.kernel.org, mingo@redhat.com, x86@kernel.org,
kvm@vger.kernel.org, linux-coco@lists.linux.dev,
pbonzini@redhat.com, Verma, Vishal L, Huang, Kai
In-Reply-To: <aa6BmJzypU1o53rB@intel.com>
On Mon, 2026-03-09 at 16:15 +0800, Chao Gao wrote:
> > - /*
> > - * Some early TDX-capable platforms have an erratum. A
> > kernel
> > - * partial write (a write transaction of less than
> > cacheline
> > - * lands at memory controller) to TDX private memory
> > poisons that
> > - * memory, and a subsequent read triggers a machine check.
> > - *
> > - * On those platforms the old kernel must reset TDX
> > private
> > - * memory before jumping to the new kernel otherwise the
> > new
> > - * kernel may see unexpected machine check. For
> > simplicity
> > - * just fail kexec/kdump on those platforms.
> > - */
> > - if (boot_cpu_has_bug(X86_BUG_TDX_PW_MCE)) {
> > - pr_info_once("Not allowed on platform with
> > tdx_pw_mce bug\n");
> > - return -EOPNOTSUPP;
> > - }
>
> With this series, we need to update the "Kexec" section in tdx.rst.
Nice catch, and I agree on the others. Will update it. Thanks.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox