* [PATCH V1 0/2] Hyper-V: Fix bugs and issues around repeat hypercall @ 2026-11-14 2:53 Mukesh R 2026-11-14 2:53 ` [PATCH V1 1/2] Drivers: hv: Add new function hv_isolation_type_hw() Mukesh R 2026-11-14 2:53 ` [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall Mukesh R 0 siblings, 2 replies; 7+ messages in thread From: Mukesh R @ 2026-11-14 2:53 UTC (permalink / raw) To: linux-hyperv, linux-kernel Cc: haiyangz, wei.liu, decui, mhkelley58, kameroncarr This short series addresses few issues around repeat hypercall, most importantly the bug around not checking completed entries in case of failure. V1: o Rename hv_isolation_hw_cvm() to hv_isolation_type_hw() o Add HV_REP_COUNT_MAX in case this goes in before the pre-commit patch that adds it. Mukesh R (2): Drivers: hv: Add new function hv_isolation_type_hw() Drivers: hv: Fix bugs and issues around repeat hypercall arch/x86/hyperv/ivm.c | 7 +++- drivers/hv/hv_common.c | 11 ++++-- drivers/hv/mshv_common.c | 19 +++++++--- drivers/hv/mshv_root_hv_call.c | 21 +++++++---- drivers/hv/mshv_root_main.c | 6 +-- include/asm-generic/mshyperv.h | 67 +++++++++++++++++++++++++--------- include/hyperv/hvgdk_mini.h | 2 +- 7 files changed, 94 insertions(+), 39 deletions(-) -- 2.51.2.vfs.0.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH V1 1/2] Drivers: hv: Add new function hv_isolation_type_hw() 2026-11-14 2:53 [PATCH V1 0/2] Hyper-V: Fix bugs and issues around repeat hypercall Mukesh R @ 2026-11-14 2:53 ` Mukesh R 2026-11-14 2:53 ` [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall Mukesh R 1 sibling, 0 replies; 7+ messages in thread From: Mukesh R @ 2026-11-14 2:53 UTC (permalink / raw) To: linux-hyperv, linux-kernel Cc: haiyangz, wei.liu, decui, mhkelley58, kameroncarr Add a new function hv_isolation_type_hw() for convenient check of whether a VM is running in HW confidential mode, ie, TDX or SNP on x86 or RME on ARM. Signed-off-by: Mukesh R <mrathor@linux.microsoft.com> --- arch/x86/hyperv/ivm.c | 5 +++++ drivers/hv/hv_common.c | 9 +++++++-- include/asm-generic/mshyperv.h | 7 +++++++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 2ce4dfe53472..26e570cf77be 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -944,3 +944,8 @@ bool hv_isolation_type_tdx(void) { return static_branch_unlikely(&isolation_type_tdx); } + +bool hv_isolation_type_hw(void) +{ + return hv_isolation_type_snp() || hv_isolation_type_tdx(); +} diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 27b9b79d4f4c..b537cafdc2a2 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -528,8 +528,7 @@ int hv_common_cpu_init(unsigned int cpu) *outputarg = (char *)mem + HV_HYP_PAGE_SIZE; } - if (!ms_hyperv.paravisor_present && - (hv_isolation_type_snp() || hv_isolation_type_tdx())) { + if (!ms_hyperv.paravisor_present && hv_isolation_type_hw()) { ret = set_memory_decrypted((unsigned long)mem, pgcount); if (ret) { /* It may be unsafe to free 'mem' */ @@ -696,6 +695,12 @@ bool __weak hv_isolation_type_tdx(void) } EXPORT_SYMBOL_GPL(hv_isolation_type_tdx); +bool __weak hv_isolation_type_hw(void) +{ + return false; +} +EXPORT_SYMBOL_GPL(hv_isolation_type_hw); + void __weak hv_setup_vmbus_handler(void (*handler)(void)) { } diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index 4c9f3353074a..396bcb3db0bc 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -84,6 +84,13 @@ u64 hv_do_fast_hypercall16(u16 control, u64 input1, u64 input2); bool hv_isolation_type_snp(void); bool hv_isolation_type_tdx(void); +/* This needs to be before hv_do_rep_hypercall() below */ +#if IS_ENABLED(CONFIG_HYPERV) +bool hv_isolation_type_hw(void); +#else +static inline bool hv_isolation_type_hw(void) { return false; } +#endif + /* * On architectures where Hyper-V doesn't support AEOI (e.g., ARM64), * it doesn't provide a recommendation flag and AEOI must be disabled. -- 2.51.2.vfs.0.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall 2026-11-14 2:53 [PATCH V1 0/2] Hyper-V: Fix bugs and issues around repeat hypercall Mukesh R 2026-11-14 2:53 ` [PATCH V1 1/2] Drivers: hv: Add new function hv_isolation_type_hw() Mukesh R @ 2026-11-14 2:53 ` Mukesh R 2026-11-14 3:18 ` sashiko-bot 2026-11-15 23:48 ` Michael Kelley 1 sibling, 2 replies; 7+ messages in thread From: Mukesh R @ 2026-11-14 2:53 UTC (permalink / raw) To: linux-hyperv, linux-kernel Cc: haiyangz, wei.liu, decui, mhkelley58, kameroncarr There are few issues/bugs around repeat hypercall: o If the hypercall fails, it could still have processed some entries. Fix places where it assumes none of the entries were processed and will try to undo entries already processed or print incorrect completions. o Hypercall returns back with timeouts only for HW confidential VMs. In that case it should be retried instead of returning back. Returning back results in error conditions and take down of VMs in most cases. o For non confidential VMs, the hypercall could be streamlined as it will never return back with timeouts. o Rename HV_STATUS_TIME_OUT to HV_STATUS_TIMEOUT to match hypervisor public header file. There is no reason for it to be different. o Minor formatting fix for checkpatch. Error printing missing in couple places when rep hypercall fails. Signed-off-by: Mukesh R <mrathor@linux.microsoft.com> --- arch/x86/hyperv/ivm.c | 2 +- drivers/hv/hv_common.c | 2 +- drivers/hv/mshv_common.c | 19 +++++++---- drivers/hv/mshv_root_hv_call.c | 21 +++++++----- drivers/hv/mshv_root_main.c | 6 ++-- include/asm-generic/mshyperv.h | 60 ++++++++++++++++++++++++---------- include/hyperv/hvgdk_mini.h | 2 +- 7 files changed, 75 insertions(+), 37 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 26e570cf77be..742f6399ecf6 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -365,7 +365,7 @@ int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip, unsigned int cpu) do { ret = hv_do_hypercall(HVCALL_START_VP, start_vp_input, NULL); - } while (hv_result(ret) == HV_STATUS_TIME_OUT && retry--); + } while (hv_result(ret) == HV_STATUS_TIMEOUT && retry--); local_irq_restore(flags); diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index b537cafdc2a2..db3c0c571e9e 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -835,7 +835,7 @@ static const struct hv_status_info hv_status_infos[] = { _STATUS_INFO(HV_STATUS_INVALID_REGISTER_VALUE, -EINVAL), _STATUS_INFO(HV_STATUS_DEVICE_ALREADY_IN_DOMAIN, -EEXIST), _STATUS_INFO(HV_STATUS_OPERATION_FAILED, -EIO), - _STATUS_INFO(HV_STATUS_TIME_OUT, -EIO), + _STATUS_INFO(HV_STATUS_TIMEOUT, -EIO), _STATUS_INFO(HV_STATUS_CALL_PENDING, -EIO), _STATUS_INFO(HV_STATUS_VTL_ALREADY_ENABLED, -EIO), #undef _STATUS_INFO diff --git a/drivers/hv/mshv_common.c b/drivers/hv/mshv_common.c index 63f09bb5136e..606f86673d51 100644 --- a/drivers/hv/mshv_common.c +++ b/drivers/hv/mshv_common.c @@ -56,16 +56,20 @@ int hv_call_get_vp_registers(u32 vp_index, u64 partition_id, u16 count, status = hv_do_rep_hypercall(HVCALL_GET_VP_REGISTERS, rep_count, 0, input_page, output_page); - if (!hv_result_success(status)) - break; - completed = hv_repcomp(status); for (i = 0; i < completed; ++i) registers[i].value = output_page[i]; registers += completed; remaining -= completed; + + if (!hv_result_success(status)) { + pr_err("%s: count:%d comp:%d st:0x%llx\n", __func__, + count, completed, status); + break; + } } + local_irq_restore(flags); return hv_result_to_errno(status); @@ -99,12 +103,15 @@ int hv_call_set_vp_registers(u32 vp_index, u64 partition_id, u16 count, status = hv_do_rep_hypercall(HVCALL_SET_VP_REGISTERS, rep_count, 0, input_page, NULL); - if (!hv_result_success(status)) - break; - completed = hv_repcomp(status); registers += completed; remaining -= completed; + + if (!hv_result_success(status)) { + pr_err("%s: count:%d comp:%d st:0x%llx\n", __func__, + count, completed, status); + break; + } } local_irq_restore(flags); diff --git a/drivers/hv/mshv_root_hv_call.c b/drivers/hv/mshv_root_hv_call.c index e761addda50e..010b5060b272 100644 --- a/drivers/hv/mshv_root_hv_call.c +++ b/drivers/hv/mshv_root_hv_call.c @@ -239,6 +239,7 @@ static int hv_do_map_gpa_hcall(u64 partition_id, u64 gfn, u64 page_struct_count, local_irq_restore(irq_flags); completed = hv_repcomp(status); + done += completed; if (hv_result_needs_memory(status)) { ret = hv_deposit_memory(partition_id, status); @@ -246,11 +247,13 @@ static int hv_do_map_gpa_hcall(u64 partition_id, u64 gfn, u64 page_struct_count, break; } else if (!hv_result_success(status)) { + pr_err("%s: failed to map pages at gfn %#llx: done %u/%llu, flags=%#x, status=%#llx (%s)\n", + __func__, gfn, done, page_count, flags, status, + hv_result_to_string(hv_result(status))); + ret = hv_result_to_errno(status); break; } - - done += completed; } if (ret && done) { @@ -356,20 +359,22 @@ int hv_call_get_gpa_access_states(u64 partition_id, u32 count, u64 gpa_base_pfn, input_page->flags = state_flags; rep_count = min(remaining, HV_GET_GPA_ACCESS_STATES_BATCH_SIZE); - status = hv_do_rep_hypercall(HVCALL_GET_GPA_PAGES_ACCESS_STATES, rep_count, - 0, input_page, output_page); - if (!hv_result_success(status)) { - local_irq_restore(flags); - break; - } + status = hv_do_rep_hypercall(HVCALL_GET_GPA_PAGES_ACCESS_STATES, + rep_count, 0, input_page, + output_page); + completed = hv_repcomp(status); for (i = 0; i < completed; ++i) states[i].as_uint8 = output_page[i].as_uint8; local_irq_restore(flags); + states += completed; *written_total += completed; remaining -= completed; + + if (!hv_result_success(status)) + break; } return hv_result_to_errno(status); diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c index cc2cfce2aefd..dfb24599c257 100644 --- a/drivers/hv/mshv_root_main.c +++ b/drivers/hv/mshv_root_main.c @@ -231,9 +231,9 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition, reps_completed = 0; do { if (args.reps) { - status = hv_do_rep_hypercall_ex(args.code, args.reps, - 0, reps_completed, - input_pg, output_pg); + status = hv_do_rep_hypercall_reg(args.code, args.reps, + 0, reps_completed, + input_pg, output_pg); reps_completed = hv_repcomp(status); } else { status = hv_do_hypercall(args.code, input_pg, output_pg); diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index 396bcb3db0bc..87c784b9daa6 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -28,6 +28,9 @@ #define VTPM_BASE_ADDRESS 0xfed40000 +#define HV_REP_COUNT_MAX \ + (HV_HYPERCALL_REP_COMP_MASK >> HV_HYPERCALL_REP_COMP_OFFSET) + enum hv_partition_type { HV_PARTITION_TYPE_GUEST, HV_PARTITION_TYPE_ROOT, @@ -128,6 +131,11 @@ static inline bool hv_result_success(u64 status) return hv_result(status) == HV_STATUS_SUCCESS; } +static inline bool hv_result_timeout(u64 status) +{ + return hv_result(status) == HV_STATUS_TIMEOUT; +} + static inline unsigned int hv_repcomp(u64 status) { /* Bits [43:32] of status have 'Reps completed' data. */ @@ -135,45 +143,63 @@ static inline unsigned int hv_repcomp(u64 status) HV_HYPERCALL_REP_COMP_OFFSET; } -/* - * Rep hypercalls. Callers of this functions are supposed to ensure that - * rep_count, varhead_size, and rep_start comply with Hyper-V hypercall - * definition. - */ -static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count, - u16 varhead_size, u16 rep_start, - void *input, void *output) +static inline u64 hv_do_rep_hypercall_reg(u16 code, u16 rep_count, + u16 varhead_size, u16 rep_start, + void *input, void *output) { u64 control = code; - u64 status; - u16 rep_comp; control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; - do { + return hv_do_hypercall(control, input, output); +} + +static inline u64 hv_do_rep_hypercall_cvm(u16 code, u16 rep_count, + u16 varhead_size, u16 rep_start, + void *input, void *output) +{ + u64 status, control = code; + u16 rep_comp = 0; + + control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; + control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; + control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; + + for (;;) { status = hv_do_hypercall(control, input, output); - if (!hv_result_success(status)) + + if (!hv_result_timeout(status)) return status; rep_comp = hv_repcomp(status); - control &= ~HV_HYPERCALL_REP_START_MASK; control |= (u64)rep_comp << HV_HYPERCALL_REP_START_OFFSET; touch_nmi_watchdog(); - } while (rep_comp < rep_count); + } return status; } -/* For the typical case where rep_start is 0 */ +/* + * Rep hypercalls for HW confidential VMs return back when timeouts occur, + * whereas for regular VMs they return only on success or failure. In case + * of successful completion, entire rep_count is done. But a hypercall could + * fail with partial completions, hence callers must always check that + * in case of failure. Lastly, a rep_count value of 0 is allowed for some + * hypercalls. + */ static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size, void *input, void *output) { - return hv_do_rep_hypercall_ex(code, rep_count, varhead_size, 0, - input, output); + if (hv_isolation_type_hw()) + return hv_do_rep_hypercall_cvm(code, rep_count, varhead_size, 0, + input, output); + else + return hv_do_rep_hypercall_reg(code, rep_count, varhead_size, 0, + input, output); } /* Generate the guest OS identifier as described in the Hyper-V TLFS */ diff --git a/include/hyperv/hvgdk_mini.h b/include/hyperv/hvgdk_mini.h index 966945f0504f..bec4e633e381 100644 --- a/include/hyperv/hvgdk_mini.h +++ b/include/hyperv/hvgdk_mini.h @@ -41,7 +41,7 @@ struct hv_u128 { #define HV_STATUS_OPERATION_FAILED 0x71 #define HV_STATUS_INSUFFICIENT_ROOT_MEMORY 0x73 #define HV_STATUS_INSUFFICIENT_CONTIGUOUS_MEMORY 0x75 -#define HV_STATUS_TIME_OUT 0x78 +#define HV_STATUS_TIMEOUT 0x78 #define HV_STATUS_CALL_PENDING 0x79 #define HV_STATUS_INSUFFICIENT_CONTIGUOUS_ROOT_MEMORY 0x83 #define HV_STATUS_VTL_ALREADY_ENABLED 0x86 -- 2.51.2.vfs.0.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall 2026-11-14 2:53 ` [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall Mukesh R @ 2026-11-14 3:18 ` sashiko-bot 2026-11-15 23:48 ` Michael Kelley 1 sibling, 0 replies; 7+ messages in thread From: sashiko-bot @ 2026-11-14 3:18 UTC (permalink / raw) To: Mukesh R, linux-hyperv Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] Drivers: hv: unused HV_REP_COUNT_MAX macro and dead return in mshyperv.h -- commit 9765fd900ad842b6359a24f592cd624ffdc92be5 Author: Mukesh R <mrathor@linux.microsoft.com> Drivers: hv: Fix bugs and issues around repeat hypercall This commit updates repeat hypercall handling to account for partial completions on failure, splits repeat hypercall helpers between hardware confidential VMs and regular VMs, and renames HV_STATUS_TIME_OUT to HV_STATUS_TIMEOUT. > diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h > index 396bcb3db0bcc..87c784b9daa69 100644 > --- a/include/asm-generic/mshyperv.h > +++ b/include/asm-generic/mshyperv.h > @@ -28,6 +28,9 @@ > > #define VTPM_BASE_ADDRESS 0xfed40000 > > +#define HV_REP_COUNT_MAX \ > + (HV_HYPERCALL_REP_COMP_MASK >> HV_HYPERCALL_REP_COMP_OFFSET) > + > enum hv_partition_type { [ ... ] > @@ -135,45 +143,63 @@ static inline unsigned int hv_repcomp(u64 status) > /* Bits [43:32] of status have 'Reps completed' data. */ > return (status & HV_HYPERCALL_REP_COMP_MASK) >> > HV_HYPERCALL_REP_COMP_OFFSET; > } > > -/* > - * Rep hypercalls. Callers of this functions are supposed to ensure that > - * rep_count, varhead_size, and rep_start comply with Hyper-V hypercall > - * definition. > - */ > -static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count, > - u16 varhead_size, u16 rep_start, > - void *input, void *output) > +static inline u64 hv_do_rep_hypercall_reg(u16 code, u16 rep_count, > + u16 varhead_size, u16 rep_start, > + void *input, void *output) > { > u64 control = code; > - u64 status; > - u16 rep_comp; > > control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; > control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; > control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; > > - do { > + return hv_do_hypercall(control, input, output); > +} > + > +static inline u64 hv_do_rep_hypercall_cvm(u16 code, u16 rep_count, > + u16 varhead_size, u16 rep_start, > + void *input, void *output) > +{ > + u64 status, control = code; > + u16 rep_comp = 0; > + > + control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; > + control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; > + control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; > + > + for (;;) { > status = hv_do_hypercall(control, input, output); > - if (!hv_result_success(status)) > + > + if (!hv_result_timeout(status)) > return status; > > rep_comp = hv_repcomp(status); > - > control &= ~HV_HYPERCALL_REP_START_MASK; > control |= (u64)rep_comp << HV_HYPERCALL_REP_START_OFFSET; > > touch_nmi_watchdog(); > - } while (rep_comp < rep_count); > + } > > return status; > } [Severity: Low] Is the trailing return status; in hv_do_rep_hypercall_cvm() needed now that the loop is for (;;) with no break statement? Also, HV_REP_COUNT_MAX added earlier in this file has no users in the tree. -- Sashiko AI review · https://sashiko.dev/#/patchset/20261114025313.1257042-1-mrathor@linux.microsoft.com?part=2 ^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall 2026-11-14 2:53 ` [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall Mukesh R 2026-11-14 3:18 ` sashiko-bot @ 2026-11-15 23:48 ` Michael Kelley 2026-11-16 16:53 ` Vitaly Kuznetsov 2026-11-17 2:29 ` Mukesh R 1 sibling, 2 replies; 7+ messages in thread From: Michael Kelley @ 2026-11-15 23:48 UTC (permalink / raw) To: Mukesh R, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, Vitaly Kuznetsov Cc: haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, kameroncarr@linux.microsoft.com From: Mukesh R <mrathor@linux.microsoft.com> Sent: Friday, November 13, 2026 6:53 PM > > There are few issues/bugs around repeat hypercall: > o If the hypercall fails, it could still have processed some entries. > Fix places where it assumes none of the entries were processed and will > try to undo entries already processed or print incorrect completions. > > o Hypercall returns back with timeouts only for HW confidential VMs. In > that case it should be retried instead of returning back. Returning back > results in error conditions and take down of VMs in most cases. > > o For non confidential VMs, the hypercall could be streamlined as it will > never return back with timeouts. > > o Rename HV_STATUS_TIME_OUT to HV_STATUS_TIMEOUT to match hypervisor > public header file. There is no reason for it to be different. > > o Minor formatting fix for checkpatch. Error printing missing in couple > places when rep hypercall fails. > > Signed-off-by: Mukesh R <mrathor@linux.microsoft.com> [snip] > diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h > index 396bcb3db0bc..87c784b9daa6 100644 > --- a/include/asm-generic/mshyperv.h > +++ b/include/asm-generic/mshyperv.h > @@ -28,6 +28,9 @@ > > #define VTPM_BASE_ADDRESS 0xfed40000 > > +#define HV_REP_COUNT_MAX \ > + (HV_HYPERCALL_REP_COMP_MASK >> HV_HYPERCALL_REP_COMP_OFFSET) Sashiko noted that this new #define is never used anywhere. > + > enum hv_partition_type { > HV_PARTITION_TYPE_GUEST, > HV_PARTITION_TYPE_ROOT, > @@ -128,6 +131,11 @@ static inline bool hv_result_success(u64 status) > return hv_result(status) == HV_STATUS_SUCCESS; > } > > +static inline bool hv_result_timeout(u64 status) > +{ > + return hv_result(status) == HV_STATUS_TIMEOUT; > +} > + > static inline unsigned int hv_repcomp(u64 status) > { > /* Bits [43:32] of status have 'Reps completed' data. */ > @@ -135,45 +143,63 @@ static inline unsigned int hv_repcomp(u64 status) > HV_HYPERCALL_REP_COMP_OFFSET; > } > > -/* > - * Rep hypercalls. Callers of this functions are supposed to ensure that > - * rep_count, varhead_size, and rep_start comply with Hyper-V hypercall > - * definition. > - */ > -static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count, > - u16 varhead_size, u16 rep_start, > - void *input, void *output) > +static inline u64 hv_do_rep_hypercall_reg(u16 code, u16 rep_count, > + u16 varhead_size, u16 rep_start, > + void *input, void *output) > { > u64 control = code; > - u64 status; > - u16 rep_comp; > > control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; > control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; > control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; > > - do { Dropped the looping behavior here may be much more significant than just "streamlining" as described in the commit message. The looping behavior was included when rep hypercalls were originally added in commit 806c89273bab in kernel 4.14, long before any SNP/TDX/CCA functionality was added. So it was present for "normal" VMs doing hypercalls on x86/x64 the normal way. The looping ensured that rep hypercalls could not return with no errors but with the completed rep count less than the requested rep count. However, in my mind it *is* debatable whether that scenario could ever occur. The Hyper-V TLFS describes "Hypercall Continuation" [1] as essentially transparent to the caller. If a rep hypercall stops processing reps because of hitting the 50 microsecond time limit, it should restart and continue with the next set of reps without the caller having to loop. Commit 806c89273bab mentions hypercall continuation, but does not describe why the explicit looping was added or how that aligns with the TLFS statements. Vitaly -- it was a long time ago, but do you have any recollection? Have you gotten an explicit assurance from the Hyper-V team that the transparent hypercall continuation really does work, and the explicit looping has all along been unnecessary? This assurance would be needed back to and including Windows Server 2016, which is now the oldest Hyper-V version supported by Linux. Also note that the rep hypercall code is architecture independent, so the transparent hypercall continuation would also need to work on arm64 when the hypercall is invoked via the standard path of using the HVC instruction (the TLFS doesn't cover arm64, so this is unclear). If you have such assurance, then I'm good with the removal of the looping. But please make the removal a separate patch, with sufficient discussion in the commit message to justify why it's OK to do the removal (including that evidently the original commit misinterpreted how continuation worked). And instead of looping, perhaps do a sanity check that the completed rep count matches the requested rep count. Without such assurance, I think the looping needs to stay. Callers are not prepared for a rep hypercall to return with no errors, but with the completed rep count less than the requested rep count. [1] https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercall-interface#hypercall-continuation > + return hv_do_hypercall(control, input, output); > +} > + > +static inline u64 hv_do_rep_hypercall_cvm(u16 code, u16 rep_count, > + u16 varhead_size, u16 rep_start, > + void *input, void *output) > +{ > + u64 status, control = code; > + u16 rep_comp = 0; > + > + control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; > + control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; > + control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; > + > + for (;;) { > status = hv_do_hypercall(control, input, output); > - if (!hv_result_success(status)) > + > + if (!hv_result_timeout(status)) > return status; > > rep_comp = hv_repcomp(status); > - > control &= ~HV_HYPERCALL_REP_START_MASK; > control |= (u64)rep_comp << HV_HYPERCALL_REP_START_OFFSET; > > touch_nmi_watchdog(); > - } while (rep_comp < rep_count); > + } > > return status; And Sashiko noted that this "return status" is dead code. The only way to exit this function is via the "return status" up in the loop. > } > > -/* For the typical case where rep_start is 0 */ > +/* > + * Rep hypercalls for HW confidential VMs return back when timeouts occur, > + * whereas for regular VMs they return only on success or failure. In case > + * of successful completion, entire rep_count is done. But a hypercall could > + * fail with partial completions, hence callers must always check that > + * in case of failure. Lastly, a rep_count value of 0 is allowed for some > + * hypercalls. > + */ > static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size, > void *input, void *output) > { > - return hv_do_rep_hypercall_ex(code, rep_count, varhead_size, 0, > - input, output); > + if (hv_isolation_type_hw()) I don't think this is the right test, so let me explain. You don't describe the context for when HV_STATUS_TIMEOUT may be returned, and there's no TLFS documentation of this status, but I'm assuming the context is hypercall continuation. This isn't the typical "something isn't responding" timeout. I suspect it means the hypervisor hit the 50 us limit, and returned to the guest VM with some reps still to be done. In a normal VM when a hypercall is initiated by the hypercall page on x86/x64 or the HVC instruction on arm64, the transparent hypercall continuation mechanism makes a TIMEOUT status unnecessary. But with SNP, TDX, and CCA, hypercall invocation may be piggybacked on other transports, and transparent continuation isn't available. So Hyper-V returns HV_STATUS_TIMEOUT with the expectation that the invoking code will loop to do the remaining reps. The key factor is whether the hypercall is piggybacked on other transports. The tests in hv_isolation_type_hw() don't capture that. For example, if a paravisor is present, hv_isolation_type_snp() will return "true" even though hypercalls are invoked via the standard x86/x64 hypercall page. The hypercalls trap to the paravisor. If the paravisor is functioning properly, it should do transparent hypercall continuation just like the hypervisor would. But this is something to be confirmed with the hypervisor and paravisor teams. Piggybacking is done only when no paravisor is present, so the hv_do_rep_hypercall_cvm() path should be taken only in that circumstance. Your current code works, but it goes down the more expensive hv_do_rep_hypercall_cvm() path when it doesn't need to. (I'm not completely clear on how arm64 works with and without a paravisor, so my statement applies only to SNP and TDX. Need to sort out the CCA case with Kameron.) Given the non-trivial complexity in sorting all this out, I wonder if distinguishing the _cvm and _reg cases is really worth it. The current code for the _reg case is admittedly inefficient because it does work for the looping case before likely deciding that no looping is needed. But that could easily be fixed. When no looping is necessary, which is the vast majority of cases, the looping code could end up being only a couple of instructions slower than a separately coded non-looping version (I did some builds and looked at the generated code). Since there's overhead in deciding which to use, there may not be any performance win from having both versions. Michael > + return hv_do_rep_hypercall_cvm(code, rep_count, varhead_size, 0, > + input, output); > + else > + return hv_do_rep_hypercall_reg(code, rep_count, varhead_size, 0, > + input, output); > } > ^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall 2026-11-15 23:48 ` Michael Kelley @ 2026-11-16 16:53 ` Vitaly Kuznetsov 2026-11-17 2:29 ` Mukesh R 1 sibling, 0 replies; 7+ messages in thread From: Vitaly Kuznetsov @ 2026-11-16 16:53 UTC (permalink / raw) To: Michael Kelley, Mukesh R, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Cc: haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, kameroncarr@linux.microsoft.com Michael Kelley <mhklinux@outlook.com> writes: ... >> >> - do { > > Dropped the looping behavior here may be much more significant > than just "streamlining" as described in the commit message. The looping > behavior was included when rep hypercalls were originally added in > commit 806c89273bab in kernel 4.14, long before any SNP/TDX/CCA > functionality was added. So it was present for "normal" VMs doing > hypercalls on x86/x64 the normal way. The looping ensured that > rep hypercalls could not return with no errors but with the completed > rep count less than the requested rep count. > > However, in my mind it *is* debatable whether that scenario could > ever occur. The Hyper-V TLFS describes "Hypercall Continuation" [1] > as essentially transparent to the caller. If a rep hypercall stops > processing reps because of hitting the 50 microsecond time limit, > it should restart and continue with the next set of reps without > the caller having to loop. Commit 806c89273bab mentions > hypercall continuation, but does not describe why the explicit > looping was added or how that aligns with the TLFS statements. > Vitaly -- it was a long time ago, but do you have any recollection? > > Have you gotten an explicit assurance from the Hyper-V team > that the transparent hypercall continuation really does work, > and the explicit looping has all along been unnecessary? This > assurance would be needed back to and including Windows > Server 2016, which is now the oldest Hyper-V version supported > by Linux. Also note that the rep hypercall code is architecture > independent, so the transparent hypercall continuation would > also need to work on arm64 when the hypercall is invoked via > the standard path of using the HVC instruction (the TLFS doesn't > cover arm64, so this is unclear). > > If you have such assurance, then I'm good with the removal of > the looping. But please make the removal a separate patch, with > sufficient discussion in the commit message to justify why it's OK > to do the removal (including that evidently the original commit > misinterpreted how continuation worked). And instead of looping, > perhaps do a sanity check that the completed rep count matches > the requested rep count. > > Without such assurance, I think the looping needs to stay. Callers > are not prepared for a rep hypercall to return with no errors, but > with the completed rep count less than the requested rep count. > > [1] https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercall-interface#hypercall-continuation > It's been almost a decade since the looping behavior was added, it is very possible that I was using WS2012 for development back then. The commit says that the feature was required for HvFlushVirtualAddressList; I don't have any evidence left but I'm pretty sure 'incomplete' returns were observed. It is, of course, possible that I only observed them in synthetic testing when stuffing hypercall input to the maximum by some random data. Overall, I agree with the proposal: keep the looping unless we get an explicit assurance from the host team that it can be dropped. -- Vitaly ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall 2026-11-15 23:48 ` Michael Kelley 2026-11-16 16:53 ` Vitaly Kuznetsov @ 2026-11-17 2:29 ` Mukesh R 1 sibling, 0 replies; 7+ messages in thread From: Mukesh R @ 2026-11-17 2:29 UTC (permalink / raw) To: Michael Kelley, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, Vitaly Kuznetsov Cc: haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, kameroncarr@linux.microsoft.com On 11/15/26 15:48, Michael Kelley wrote: > From: Mukesh R <mrathor@linux.microsoft.com> Sent: Friday, November 13, 2026 6:53 PM >> >> There are few issues/bugs around repeat hypercall: >> o If the hypercall fails, it could still have processed some entries. >> Fix places where it assumes none of the entries were processed and will >> try to undo entries already processed or print incorrect completions. >> >> o Hypercall returns back with timeouts only for HW confidential VMs. In >> that case it should be retried instead of returning back. Returning back >> results in error conditions and take down of VMs in most cases. >> >> o For non confidential VMs, the hypercall could be streamlined as it will >> never return back with timeouts. >> >> o Rename HV_STATUS_TIME_OUT to HV_STATUS_TIMEOUT to match hypervisor >> public header file. There is no reason for it to be different. >> >> o Minor formatting fix for checkpatch. Error printing missing in couple >> places when rep hypercall fails. >> >> Signed-off-by: Mukesh R <mrathor@linux.microsoft.com> > > [snip] > >> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h >> index 396bcb3db0bc..87c784b9daa6 100644 >> --- a/include/asm-generic/mshyperv.h >> +++ b/include/asm-generic/mshyperv.h >> @@ -28,6 +28,9 @@ >> >> #define VTPM_BASE_ADDRESS 0xfed40000 >> >> +#define HV_REP_COUNT_MAX \ >> + (HV_HYPERCALL_REP_COMP_MASK >> HV_HYPERCALL_REP_COMP_OFFSET) > > Sashiko noted that this new #define is never used anywhere. > >> + >> enum hv_partition_type { >> HV_PARTITION_TYPE_GUEST, >> HV_PARTITION_TYPE_ROOT, >> @@ -128,6 +131,11 @@ static inline bool hv_result_success(u64 status) >> return hv_result(status) == HV_STATUS_SUCCESS; >> } >> >> +static inline bool hv_result_timeout(u64 status) >> +{ >> + return hv_result(status) == HV_STATUS_TIMEOUT; >> +} >> + >> static inline unsigned int hv_repcomp(u64 status) >> { >> /* Bits [43:32] of status have 'Reps completed' data. */ >> @@ -135,45 +143,63 @@ static inline unsigned int hv_repcomp(u64 status) >> HV_HYPERCALL_REP_COMP_OFFSET; >> } >> >> -/* >> - * Rep hypercalls. Callers of this functions are supposed to ensure that >> - * rep_count, varhead_size, and rep_start comply with Hyper-V hypercall >> - * definition. >> - */ >> -static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count, >> - u16 varhead_size, u16 rep_start, >> - void *input, void *output) >> +static inline u64 hv_do_rep_hypercall_reg(u16 code, u16 rep_count, >> + u16 varhead_size, u16 rep_start, >> + void *input, void *output) >> { >> u64 control = code; >> - u64 status; >> - u16 rep_comp; >> >> control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; >> control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; >> control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; >> >> - do { > > Dropped the looping behavior here may be much more significant > than just "streamlining" as described in the commit message. The looping > behavior was included when rep hypercalls were originally added in > commit 806c89273bab in kernel 4.14, long before any SNP/TDX/CCA > functionality was added. So it was present for "normal" VMs doing > hypercalls on x86/x64 the normal way. The looping ensured that > rep hypercalls could not return with no errors but with the completed > rep count less than the requested rep count. > > However, in my mind it *is* debatable whether that scenario could > ever occur. The Hyper-V TLFS describes "Hypercall Continuation" [1] > as essentially transparent to the caller. If a rep hypercall stops > processing reps because of hitting the 50 microsecond time limit, > it should restart and continue with the next set of reps without > the caller having to loop. Commit 806c89273bab mentions > hypercall continuation, but does not describe why the explicit > looping was added or how that aligns with the TLFS statements. > Vitaly -- it was a long time ago, but do you have any recollection? > > Have you gotten an explicit assurance from the Hyper-V team > that the transparent hypercall continuation really does work, > and the explicit looping has all along been unnecessary? This Correct, hence the patch. Waiting on confirmation that it is same behavior all the way back to server 2016. Basically, the hypervisor will just adjust the RIP back to the vmcall upon internal timeouts, and reschedule the vcpu. But for hw confidential vms, ie, tdx/snp/cca, it cannot adjust the RIP, hence return back to the guest with TIMEOUT, so cvm has to do in a loop. Now, for paravisor support, I was just told by a paravisor team dev, that it will never return with TIMEOUTs. My view on that set aside, I guess for paravisor we'll just call the _reg function then. Thanks for bringing attention to that... > assurance would be needed back to and including Windows > Server 2016, which is now the oldest Hyper-V version supported > by Linux. Also note that the rep hypercall code is architecture > independent, so the transparent hypercall continuation would > also need to work on arm64 when the hypercall is invoked via > the standard path of using the HVC instruction (the TLFS doesn't > cover arm64, so this is unclear). Both x86 and arm are same for this issue. > If you have such assurance, then I'm good with the removal of > the looping. But please make the removal a separate patch, with > sufficient discussion in the commit message to justify why it's OK > to do the removal (including that evidently the original commit > misinterpreted how continuation worked). And instead of looping, > perhaps do a sanity check that the completed rep count matches > the requested rep count. Not sure if it's worth the overhead of sanity check, looks like callers are checking completed and loop anyways... But I would be OK with BUG(), since it would be a bug for that to happen. FWIW, it's been running internally with couple months, and haven't seen any issues. > Without such assurance, I think the looping needs to stay. Callers > are not prepared for a rep hypercall to return with no errors, but > with the completed rep count less than the requested rep count. > > [1] https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercall-interface#hypercall-continuation > >> + return hv_do_hypercall(control, input, output); >> +} >> + >> +static inline u64 hv_do_rep_hypercall_cvm(u16 code, u16 rep_count, >> + u16 varhead_size, u16 rep_start, >> + void *input, void *output) >> +{ >> + u64 status, control = code; >> + u16 rep_comp = 0; >> + >> + control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET; >> + control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET; >> + control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET; >> + >> + for (;;) { >> status = hv_do_hypercall(control, input, output); >> - if (!hv_result_success(status)) >> + >> + if (!hv_result_timeout(status)) >> return status; >> >> rep_comp = hv_repcomp(status); >> - >> control &= ~HV_HYPERCALL_REP_START_MASK; >> control |= (u64)rep_comp << HV_HYPERCALL_REP_START_OFFSET; >> >> touch_nmi_watchdog(); >> - } while (rep_comp < rep_count); >> + } >> >> return status; > > And Sashiko noted that this "return status" is dead code. The only > way to exit this function is via the "return status" up in the loop. > >> } >> >> -/* For the typical case where rep_start is 0 */ >> +/* >> + * Rep hypercalls for HW confidential VMs return back when timeouts occur, >> + * whereas for regular VMs they return only on success or failure. In case >> + * of successful completion, entire rep_count is done. But a hypercall could >> + * fail with partial completions, hence callers must always check that >> + * in case of failure. Lastly, a rep_count value of 0 is allowed for some >> + * hypercalls. >> + */ >> static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size, >> void *input, void *output) >> { >> - return hv_do_rep_hypercall_ex(code, rep_count, varhead_size, 0, >> - input, output); >> + if (hv_isolation_type_hw()) > > I don't think this is the right test, so let me explain. You don't describe the > context for when HV_STATUS_TIMEOUT may be returned, and there's no > TLFS documentation of this status, but I'm assuming the context is hypercall > continuation. This isn't the typical "something isn't responding" timeout. I > suspect it means the hypervisor hit the 50 us limit, and returned to the guest > VM with some reps still to be done. In a normal VM when a hypercall is > initiated by the hypercall page on x86/x64 or the HVC instruction on arm64, > the transparent hypercall continuation mechanism makes a TIMEOUT status > unnecessary. But with SNP, TDX, and CCA, hypercall invocation may be > piggybacked on other transports, and transparent continuation isn't available. > So Hyper-V returns HV_STATUS_TIMEOUT with the expectation that the > invoking code will loop to do the remaining reps. > > The key factor is whether the hypercall is piggybacked on other transports. > The tests in hv_isolation_type_hw() don't capture that. For example, if > a paravisor is present, hv_isolation_type_snp() will return "true" even > though hypercalls are invoked via the standard x86/x64 hypercall page. > The hypercalls trap to the paravisor. If the paravisor is functioning properly, > it should do transparent hypercall continuation just like the hypervisor would. > But this is something to be confirmed with the hypervisor and paravisor teams. > > Piggybacking is done only when no paravisor is present, so the > hv_do_rep_hypercall_cvm() path should be taken only in that circumstance. > Your current code works, but it goes down the more expensive > hv_do_rep_hypercall_cvm() path when it doesn't need to. (I'm not completely > clear on how arm64 works with and without a paravisor, so my statement > applies only to SNP and TDX. Need to sort out the CCA case with Kameron.) > > Given the non-trivial complexity in sorting all this out, I wonder if > distinguishing the _cvm and _reg cases is really worth it. The current > code for the _reg case is admittedly inefficient because it does work for > the looping case before likely deciding that no looping is needed. But > that could easily be fixed. When no looping is necessary, which is the > vast majority of cases, the looping code could end up being only a > couple of instructions slower than a separately coded non-looping > version (I did some builds and looked at the generated code). Since > there's overhead in deciding which to use, there may not be any > performance win from having both versions. Well, to me it's correctness issue also. If the ABI says that it will not return with TIMEOUTs for non-cvms, then adding loop just masks it instead of opening a bug report against the hypervisor. Hope that makes sense. On a side note, to solicit your thoughts, i am not sure i like the touch_nmi_watchdog()... it appears the the WD is triggered in seconds, and if a hypercall is that long, schedule() really should be done in between. IOW, remove the touch, instead any very long running hypercalls should call schedule(). Of course, that would be a different patch. Thanks, -Mukesh > Michael > >> + return hv_do_rep_hypercall_cvm(code, rep_count, varhead_size, 0, >> + input, output); >> + else >> + return hv_do_rep_hypercall_reg(code, rep_count, varhead_size, 0, >> + input, output); >> } >> ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-11-17 2:29 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-14 2:53 [PATCH V1 0/2] Hyper-V: Fix bugs and issues around repeat hypercall Mukesh R 2026-11-14 2:53 ` [PATCH V1 1/2] Drivers: hv: Add new function hv_isolation_type_hw() Mukesh R 2026-11-14 2:53 ` [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall Mukesh R 2026-11-14 3:18 ` sashiko-bot 2026-11-15 23:48 ` Michael Kelley 2026-11-16 16:53 ` Vitaly Kuznetsov 2026-11-17 2:29 ` Mukesh R
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox