* Re: [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)
2025-06-25 13:53 [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX) Manuel Andreas
@ 2025-06-25 16:00 ` Vitaly Kuznetsov
2025-06-25 22:25 ` Sean Christopherson
1 sibling, 0 replies; 4+ messages in thread
From: Vitaly Kuznetsov @ 2025-06-25 16:00 UTC (permalink / raw)
To: Manuel Andreas, kvm, linux-kernel; +Cc: Sean Christopherson, pbonzini
Manuel Andreas <manuel.andreas@tum.de> writes:
> In KVM guests with Hyper-V hypercalls enabled, the hypercalls
> HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST and HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX
> allow a guest to request invalidation of portions of a virtual TLB.
> For this, the hypercall parameter includes a list of GVAs that are supposed
> to be invalidated.
>
> However, when non-canonical GVAs are passed, there is currently no
> filtering in place and they are eventually passed to checked invocations of
> INVVPID on Intel / INVLPGA on AMD.
> While the AMD variant (INVLPGA) will silently ignore the non-canonical
> address and perform a no-op, the Intel variant (INVVPID) will fail and end
> up in invvpid_error, where a WARN_ONCE is triggered:
>
> invvpid failed: ext=0x0 vpid=1 gva=0xaaaaaaaaaaaaa000
> WARNING: CPU: 6 PID: 326 at arch/x86/kvm/vmx/vmx.c:482
> invvpid_error+0x91/0xa0 [kvm_intel]
> Modules linked in: kvm_intel kvm 9pnet_virtio irqbypass fuse
> CPU: 6 UID: 0 PID: 326 Comm: kvm-vm Not tainted 6.15.0 #14 PREEMPT(voluntary)
> RIP: 0010:invvpid_error+0x91/0xa0 [kvm_intel]
> Call Trace:
> <TASK>
> vmx_flush_tlb_gva+0x320/0x490 [kvm_intel]
> ? __pfx_vmx_flush_tlb_gva+0x10/0x10 [kvm_intel]
> ? kfifo_copy_out+0xcf/0x120
> kvm_hv_vcpu_flush_tlb+0x24f/0x4f0 [kvm]
> ? __pfx_kvm_hv_vcpu_flush_tlb+0x10/0x10 [kvm]
> ? kvm_pmu_is_valid_msr+0x6e/0x80 [kvm]
> ? kvm_get_msr_common+0x219/0x20f0 [kvm]
> kvm_arch_vcpu_ioctl_run+0x3013/0x5810 [kvm]
> /* ... */
>
> Hyper-V documents that invalid GVAs (those that are beyond a partition's
> GVA space) are to be ignored. While not completely clear whether this
> ruling also applies to non-canonical GVAs, it is likely fine to make that
> assumption.
I wrote a simple test and ran it on Azure. Unless I screwed up, the
result confirms this assumption. The test was:
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
struct hv_tlb_flush { /* HV_INPUT_FLUSH_VIRTUAL_ADDRESS_LIST */
u64 address_space;
u64 flags;
u64 processor_mask;
u64 gva_list[];
} __packed;
static inline u64 __hyperv_hypercall(u64 control, u64 input_address,
u64 output_address)
{
u64 res;
asm volatile("mov %[output_address], %%r8\n\t"
"vmcall"
: "=a" (res),
"+c" (control), "+d" (input_address)
: [output_address] "r"(output_address)
: "cc", "memory", "r8", "r9", "r10", "r11");
return res;
}
int init_module(void)
{
u64 rcx = 0x0003UL | 1UL << 32; /* 1 rep */
u64 status;
struct hv_tlb_flush *flush = (struct hv_tlb_flush *)kzalloc(4096, GFP_KERNEL);
void *out = kzalloc(4096, GFP_KERNEL);
flush->flags = 0x3; /* all CPUS all address spaces */
flush->processor_mask = 0x1;
flush->gva_list[0] = 0xaaaaaaaaaaaaa000;
status = __hyperv_hypercall(rcx, __pa(flush), __pa(out));
printk("Flush result: %llx\n", status);
kfree(flush);
kfree(out);
return -1;
}
void cleanup_module(void)
{
}
MODULE_LICENSE("GPL");
And the result of loading this module is:
[ 4228.888451] Flush result: 100000000
(1 in bit 32 means 1 rep completed, lower 16 bits == 0 indicate
HV_STATUS_SUCCESS).
>
> The following patch addresses the issue by skipping non-canonical GVAs
> before calling the architecture-specific invalidation primitive.
> I've validated it against a PoC and the issue seems to be fixed.
>
> Signed-off-by: Manuel Andreas <manuel.andreas@tum.de>
> Suggested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> arch/x86/kvm/hyperv.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
> index 24f0318c50d7..644a7aae6dab 100644
> --- a/arch/x86/kvm/hyperv.c
> +++ b/arch/x86/kvm/hyperv.c
> @@ -1979,6 +1979,9 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
> if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY)
> goto out_flush_all;
>
> + if (is_noncanonical_invlpg_address(entries[i], vcpu))
> + continue;
> +
> /*
> * Lower 12 bits of 'address' encode the number of additional
> * pages to flush.
--
Vitaly
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)
2025-06-25 13:53 [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX) Manuel Andreas
2025-06-25 16:00 ` Vitaly Kuznetsov
@ 2025-06-25 22:25 ` Sean Christopherson
2025-06-26 13:49 ` Manuel Andreas
1 sibling, 1 reply; 4+ messages in thread
From: Sean Christopherson @ 2025-06-25 22:25 UTC (permalink / raw)
To: Sean Christopherson, kvm, linux-kernel, Manuel Andreas
Cc: Vitaly Kuznetsov, pbonzini
On Wed, 25 Jun 2025 15:53:19 +0200, Manuel Andreas wrote:
> In KVM guests with Hyper-V hypercalls enabled, the hypercalls
> HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST and HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX
> allow a guest to request invalidation of portions of a virtual TLB.
> For this, the hypercall parameter includes a list of GVAs that are supposed
> to be invalidated.
>
> However, when non-canonical GVAs are passed, there is currently no
> filtering in place and they are eventually passed to checked invocations of
> INVVPID on Intel / INVLPGA on AMD.
> While the AMD variant (INVLPGA) will silently ignore the non-canonical
> address and perform a no-op, the Intel variant (INVVPID) will fail and end
> up in invvpid_error, where a WARN_ONCE is triggered:
>
> [...]
Applied to kvm-x86 fixes, with a massaged changelog, e.g. to call out that
"real" Hyper-V behaves this way. Thanks!
[1/1] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)
https://github.com/kvm-x86/linux/commit/fa787ac07b3c
--
https://github.com/kvm-x86/kvm-unit-tests/tree/next
^ permalink raw reply [flat|nested] 4+ messages in thread