* [PATCH v6 01/10] x86/acpi: Add helper functions to setup and access the wakeup mailbox
From: Ricardo Neri @ 2025-10-17 2:57 UTC (permalink / raw)
To: x86, Krzysztof Kozlowski, Conor Dooley, Rob Herring,
K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
Michael Kelley, Rafael J. Wysocki
Cc: Saurabh Sengar, Chris Oo, Kirill A. Shutemov, linux-hyperv,
devicetree, linux-acpi, linux-kernel, Ricardo Neri,
Rafael J. Wysocki, Ricardo Neri
In-Reply-To: <20251016-rneri-wakeup-mailbox-v6-0-40435fb9305e@linux.intel.com>
In preparation to move the functionality to wake secondary CPUs up from the
ACPI code, add two helper functions.
The function acpi_setup_mp_wakeup_mailbox() stores the physical address of
the mailbox and updates the wakeup_secondary_cpu_64() APIC callback.
There is a slight change in behavior: now the APIC callback is updated
before configuring CPU hotplug offline behavior. This is fine as the APIC
callback continues to be updated unconditionally, regardless of the
restriction on CPU offlining.
The function acpi_madt_multiproc_wakeup_mailbox() returns a pointer to the
mailbox. Use this helper function only in the portions of the code for
which the variable acpi_mp_wake_mailbox will be out of scope once it is
relocated out of the ACPI directory.
The wakeup mailbox is only supported for CONFIG_X86_64 and needed only with
CONFIG_SMP=y.
Reviewed-by: Dexuan Cui <decui@microsoft.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
---
Changes since v5:
- Fixed grammar error in the subject of the patch. (Rafael)
- Added Acked-by tag from Rafael. Thanks!
- Added Reviewed-by tag from Dexuan. Thanks!
Changes since v4:
- None
Changes since v3:
- Squashed the two first patches of the series into one, both introduce
helper functions. (Rafael)
- Renamed setup_mp_wakeup_mailbox() as acpi_setup_mp_wakeup_mailbox().
(Rafael)
- Dropped the function prototype for !CONFIG_X86_64. (Rafael)
Changes since v2:
- Introduced this patch.
Changes since v1:
- N/A
---
arch/x86/include/asm/smp.h | 3 +++
arch/x86/kernel/acpi/madt_wakeup.c | 20 +++++++++++++++-----
2 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h
index 22bfebe6776d..47ac4381a805 100644
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -149,6 +149,9 @@ static inline struct cpumask *cpu_l2c_shared_mask(int cpu)
return per_cpu(cpu_l2c_shared_map, cpu);
}
+void acpi_setup_mp_wakeup_mailbox(u64 addr);
+struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox(void);
+
#else /* !CONFIG_SMP */
#define wbinvd_on_cpu(cpu) wbinvd()
static inline void wbinvd_on_all_cpus(void)
diff --git a/arch/x86/kernel/acpi/madt_wakeup.c b/arch/x86/kernel/acpi/madt_wakeup.c
index 6d7603511f52..c3ac5ecf3e7d 100644
--- a/arch/x86/kernel/acpi/madt_wakeup.c
+++ b/arch/x86/kernel/acpi/madt_wakeup.c
@@ -37,6 +37,7 @@ static void acpi_mp_play_dead(void)
static void acpi_mp_cpu_die(unsigned int cpu)
{
+ struct acpi_madt_multiproc_wakeup_mailbox *mailbox = acpi_get_mp_wakeup_mailbox();
u32 apicid = per_cpu(x86_cpu_to_apicid, cpu);
unsigned long timeout;
@@ -46,13 +47,13 @@ static void acpi_mp_cpu_die(unsigned int cpu)
*
* BIOS has to clear 'command' field of the mailbox.
*/
- acpi_mp_wake_mailbox->apic_id = apicid;
- smp_store_release(&acpi_mp_wake_mailbox->command,
+ mailbox->apic_id = apicid;
+ smp_store_release(&mailbox->command,
ACPI_MP_WAKE_COMMAND_TEST);
/* Don't wait longer than a second. */
timeout = USEC_PER_SEC;
- while (READ_ONCE(acpi_mp_wake_mailbox->command) && --timeout)
+ while (READ_ONCE(mailbox->command) && --timeout)
udelay(1);
if (!timeout)
@@ -227,7 +228,7 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers *header,
acpi_table_print_madt_entry(&header->common);
- acpi_mp_wake_mailbox_paddr = mp_wake->mailbox_address;
+ acpi_setup_mp_wakeup_mailbox(mp_wake->mailbox_address);
if (mp_wake->version >= ACPI_MADT_MP_WAKEUP_VERSION_V1 &&
mp_wake->header.length >= ACPI_MADT_MP_WAKEUP_SIZE_V1) {
@@ -243,7 +244,16 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers *header,
acpi_mp_disable_offlining(mp_wake);
}
+ return 0;
+}
+
+void __init acpi_setup_mp_wakeup_mailbox(u64 mailbox_paddr)
+{
+ acpi_mp_wake_mailbox_paddr = mailbox_paddr;
apic_update_callback(wakeup_secondary_cpu_64, acpi_wakeup_cpu);
+}
- return 0;
+struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox(void)
+{
+ return acpi_mp_wake_mailbox;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v6 00/10] x86/hyperv/hv_vtl: Use a wakeup mailbox to boot secondary CPUs
From: Ricardo Neri @ 2025-10-17 2:57 UTC (permalink / raw)
To: x86, Krzysztof Kozlowski, Conor Dooley, Rob Herring,
K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
Michael Kelley, Rafael J. Wysocki
Cc: Saurabh Sengar, Chris Oo, Kirill A. Shutemov, linux-hyperv,
devicetree, linux-acpi, linux-kernel, Ricardo Neri,
Rafael J. Wysocki, Ricardo Neri, Yunhong Jiang, Thomas Gleixner
Hi,
Many thanks to Rafael, Rob, and Dexuan for their review! This new version
only collects tags from reviewers, fixes several typos and one bug in
patch 9.
Now that DeviceTree, ACPI and Hyper-V maintainers have reviewed the
patches, I think they are ready for review (merging?) from the x86
maintainers. In particular, Wei Liu had a question for tglx in patch 6
[1]. That question is still open.
I did not change the cover letter but I include it here for completeness.
Thanks in advance for your feedback!
...
This patchset adds functionality to use a wakeup mailbox to boot secondary
CPUs in Hyper-V VTL level 2 TDX guests with virtual firmware that describes
hardware using a DeviceTree graph. Although this is the target use case,
the use of the mailbox depends solely on it being enumerated in the
DeviceTree.
On x86 platforms, secondary CPUs are typically booted using INIT assert,
de-assert followed by Start-Up IPI messages. Virtual machines can also use
hypercalls to bring up secondary CPUs to a desired execution state. These
two mechanisms require support from the hypervisor. Confidential computing
VMs in a TDX environment cannot use this mechanism because the hypervisor
is considered an untrusted entity.
Linux already supports the ACPI Multiprocessor Wakeup Structure in which
the guest platform firmware boots the secondary CPUs and transfers control
to the kernel using a mailbox. This mechanism does not need involvement
of the VMM. It can be used in a Hyper-V VTL level 2 TDX guest.
Currently, this mechanism can only be used on x86 platforms with firmware
that supports ACPI. There are platforms that use DeviceTree (e.g., OpenHCL
[2]) instead of ACPI to describe the hardware.
Provided that the wakeup mailbox enumerated in a DeviceTree-based platform
firmware is implemented as described in the ACPI specification, the kernel
can use common code for both DeviceTree and ACPI systems. The DeviceTree
firmware does not need to use any ACPI table to publish the mailbox.
This patchset is structured as follows:
* Relocate portions of the code handling the ACPI Multiprocessor Wakeup
Structure code to a common location. (patches 1, 2)
* Define DeviceTree bindings to enumerate a mailbox as described in
the ACPI specification. (patch 3)
* Find and set up the wakeup mailbox if found in the DeviceTree graph.
(patch 4)
* Prepare Hyper-V VTL2 TDX guests to use the Wakeup Mailbox to boot
secondary CPUs when available. (patches 5-10)
I have tested this patchset on a Hyper-V host with VTL2 OpenHCL, QEMU, and
physical hardware.
Changes since v6:
- Fixed a build error with !CONFIG_X86_MAILBOX_WAKEUP and
CONFIG_HYPER_VTL_MODE.
- Added Acked-by tags from Rafael. Thanks!
- Added Reviewed-by tags from Dexuan and Rob. Thanks!
- Corrected typos and function names in the changelog.
- Link to v5: https://lore.kernel.org/r/20250627-rneri-wakeup-mailbox-v5-0-df547b1d196e@linux.intel.com
Changes in v5:
- Referred in the DeviceTree binding documentation the section and
section of the ACPI specification that defines the wakeup mailbox.
- Moved the dependency on CONFIG_OF to patch 4, where the flattened
DeviceTree is parsed for the mailbox.
- Fixed a warning from yamllint regarding line lengths.
- Link to v4: https://lore.kernel.org/r/20250603-rneri-wakeup-mailbox-v4-0-d533272b7232@linux.intel.com
Changes in v4:
- Added Reviewed-by: tags from Michael Kelley. Thanks!
- Relocated the common wakeup code from acpi/madt_wakeup.c to a new
smpwakeup.c to be used in DeviceTree- and ACPI-based systems.
- Dropped the x86 CPU bindings as they are not a good fit to document
firmware features.
- Dropped the code that parsed and validated of the `enable-method`
property for cpu@N nodes in x86. Instead, unconditionally parse and use
the wakeup mailbox when found.
- Updated the wakeup mailbox schema to avoid redefing the structure and
operation of the mailbox. Instead, refer to the ACPI specification.
Also clarified that the enumeration of the mailbox is done separately.
- Prefixed helper functions of wakeup code with acpi_.
- Link to v3: https://lore.kernel.org/r/20250503191515.24041-1-ricardo.neri-calderon@linux.intel.com
Changes in v3:
- Only move out of the acpi directory acpi_wakeup_cpu() and its
accessory variables. Use helper functions to access the mailbox as
needed. This also fixed the warnings about unused code with CONFIG_
ACPI=n that Michael reported.
- Major rework of the DeviceTree bindings and schema. Now there is a
reserved-memory binding for the mailbox as well as a new x86 CPU
bindings. Both have `compatible` properties.
- Rework of the code parsing the DeviceTree bindings for the mailbox.
Now configuring the mailbox depends solely on its enumeration in the
DeviceTree and not on Hyper-V VTL2 TDX guest.
- Do not make reserving the first 1MB of memory optional. It is not
needed and may introduce bugs.
- Prepare Hyper-V VTL2 guests to unconditionally use the mailbox in TDX
environments. If the mailbox is not available, booting secondary CPUs
will fail gracefully.
- Link to v2: https://lore.kernel.org/r/20240823232327.2408869-1-yunhong.jiang@linux.intel.com
Changes in v2:
- Fix the cover letter's summary phrase.
- Fix the DT binding document to pass validation.
- Change the DT binding document to be ACPI independent.
- Move ACPI-only functions into the #ifdef CONFIG_ACPI.
- Change dtb_parse_mp_wake() to return mailbox physical address.
- Rework the hv_is_private_mmio_tdx().
- Remove unrelated real mode change from the patch that marks mailbox
page private.
- Check hv_isolation_type_tdx() instead of wakeup_mailbox_addr in
hv_vtl_init_platform() because wakeup_mailbox_addr is not parsed yet.
- Add memory range support to reserve_real_mode.
- Remove realmode_reserve callback and use the memory range.
- Move setting the real_mode_header to hv_vtl_init_platform.
- Update comments and commit messages.
- Minor style changes.
- Link to v1: https://lore.kernel.org/r/20240806221237.1634126-1-yunhong.jiang@linux.intel.com
[1]. https://lore.kernel.org/all/aNxGzWMoM_oQ6n1N@liuwe-devbox-ubuntu-v2.lamzopl0uupeniq2etz1fddiyg.xx.internal.cloudapp.net/
[2]. https://openvmm.dev/guide/user_guide/openhcl.html
--
2.43.0
---
Ricardo Neri (6):
x86/acpi: Add helper functions to setup and access the wakeup mailbox
x86/acpi: Move acpi_wakeup_cpu() and helpers to smpwakeup.c
dt-bindings: reserved-memory: Wakeup Mailbox for Intel processors
x86/dt: Parse the Wakeup Mailbox for Intel processors
x86/smpwakeup: Add a helper get the address of the wakeup mailbox
x86/hyperv/vtl: Use the wakeup mailbox to boot secondary CPUs
Yunhong Jiang (4):
x86/hyperv/vtl: Set real_mode_header in hv_vtl_init_platform()
x86/realmode: Make the location of the trampoline configurable
x86/hyperv/vtl: Setup the 64-bit trampoline for TDX guests
x86/hyperv/vtl: Mark the wakeup mailbox page as private
.../reserved-memory/intel,wakeup-mailbox.yaml | 50 ++++++++++++
arch/x86/Kconfig | 7 ++
arch/x86/hyperv/hv_vtl.c | 39 +++++++++-
arch/x86/include/asm/smp.h | 4 +
arch/x86/include/asm/x86_init.h | 3 +
arch/x86/kernel/Makefile | 1 +
arch/x86/kernel/acpi/madt_wakeup.c | 76 ++-----------------
arch/x86/kernel/devicetree.c | 47 ++++++++++++
arch/x86/kernel/smpwakeup.c | 88 ++++++++++++++++++++++
arch/x86/kernel/x86_init.c | 3 +
arch/x86/realmode/init.c | 7 +-
11 files changed, 246 insertions(+), 79 deletions(-)
---
base-commit: 0292ef418ce08aad597fc0bba65b6dbb841808ba
change-id: 20250602-rneri-wakeup-mailbox-328efe72803f
Best regards,
--
Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
^ permalink raw reply
* RE: [PATCH] mshv: Fix deposit memory in MSHV_ROOT_HVCALL
From: Michael Kelley @ 2025-10-17 1:12 UTC (permalink / raw)
To: Nuno Das Neves, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org
Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, arnd@arndb.de, mrathor@linux.microsoft.com,
skinsburskii@linux.microsoft.com
In-Reply-To: <1760644436-19937-1-git-send-email-nunodasneves@linux.microsoft.com>
From: Nuno Das Neves <nunodasneves@linux.microsoft.com> Sent: Thursday, October 16, 2025 12:54 PM
>
> When the MSHV_ROOT_HVCALL ioctl is executing a hypercall, and gets
> HV_STATUS_INSUFFICIENT_MEMORY, it deposits memory and then returns
> -EAGAIN to userspace.
>
> However, it's much easier and efficient if the driver simply deposits
> memory on demand and immediately retries the hypercall as is done with
> all the other hypercall helper functions.
>
> But unlike those, in MSHV_ROOT_HVCALL the input is opaque to the
> kernel. This is problematic for rep hypercalls, because the next part
> of the input list can't be copied on each loop after depositing pages
> (this was the original reason for returning -EAGAIN in this case).
>
> Introduce hv_do_rep_hypercall_ex(), which adds a 'rep_start'
> parameter. This solves the issue, allowing the deposit loop in
> MSHV_ROOT_HVCALL to restart a rep hypercall after depositing pages
> partway through.
From reading the above, I'm pretty sure this code change is an
optimization that lets user space avoid having to deal with the
-EAGAIN result by resubmitting the ioctl with a different
starting point for a rep hypercall. As such, I'd suggest the patch
title should be "Improve deposit memory ...." (or something similar).
The word "Fix" makes it sound like a bug fix.
Or is user space code currently faulty in its handling of -EAGAIN, and
this really is an indirect bug fix to make things work? If so, do you
want a Fixes: tag so the change is backported?
>
> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
> ---
> drivers/hv/mshv_root_main.c | 52 ++++++++++++++++++++--------------
> include/asm-generic/mshyperv.h | 14 +++++++--
> 2 files changed, 42 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
> index 9ae67c6e9f60..731ec8cbbd63 100644
> --- a/drivers/hv/mshv_root_main.c
> +++ b/drivers/hv/mshv_root_main.c
> @@ -159,6 +159,7 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition,
> unsigned int pages_order;
> void *input_pg = NULL;
> void *output_pg = NULL;
> + u16 reps_completed;
>
> if (copy_from_user(&args, user_args, sizeof(args)))
> return -EFAULT;
> @@ -210,28 +211,35 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition,
> */
> *(u64 *)input_pg = partition->pt_id;
>
> - if (args.reps)
> - status = hv_do_rep_hypercall(args.code, args.reps, 0,
> - input_pg, output_pg);
> - else
> - status = hv_do_hypercall(args.code, input_pg, output_pg);
> -
> - if (hv_result(status) == HV_STATUS_CALL_PENDING) {
> - if (is_async) {
> - mshv_async_hvcall_handler(partition, &status);
> - } else { /* Paranoia check. This shouldn't happen! */
> - ret = -EBADFD;
> - goto free_pages_out;
> + reps_completed = 0;
> + do {
> + if (args.reps) {
> + status = hv_do_rep_hypercall_ex(args.code, args.reps,
> + 0, reps_completed,
> + input_pg, output_pg);
> + reps_completed = hv_repcomp(status);
> + } else {
> + status = hv_do_hypercall(args.code, input_pg, output_pg);
> }
> - }
>
> - if (hv_result(status) == HV_STATUS_INSUFFICIENT_MEMORY) {
> - ret = hv_call_deposit_pages(NUMA_NO_NODE, partition->pt_id, 1);
> - if (!ret)
> - ret = -EAGAIN;
> - } else if (!hv_result_success(status)) {
> - ret = hv_result_to_errno(status);
> - }
> + if (hv_result(status) == HV_STATUS_CALL_PENDING) {
> + if (is_async) {
> + mshv_async_hvcall_handler(partition, &status);
> + } else { /* Paranoia check. This shouldn't happen! */
> + ret = -EBADFD;
> + goto free_pages_out;
> + }
> + }
> +
> + if (hv_result_success(status))
> + break;
> +
> + if (hv_result(status) != HV_STATUS_INSUFFICIENT_MEMORY)
> + ret = hv_result_to_errno(status);
> + else
> + ret = hv_call_deposit_pages(NUMA_NO_NODE,
> + partition->pt_id, 1);
> + } while (!ret);
>
> /*
> * Always return the status and output data regardless of result.
This comment about always returning the output data is now incorrect.
> @@ -240,11 +248,11 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition,
> * succeeded.
> */
> args.status = hv_result(status);
> - args.reps = args.reps ? hv_repcomp(status) : 0;
> + args.reps = reps_completed;
> if (copy_to_user(user_args, &args, sizeof(args)))
> ret = -EFAULT;
>
> - if (output_pg &&
> + if (!ret && output_pg &&
> copy_to_user((void __user *)args.out_ptr, output_pg, args.out_sz))
> ret = -EFAULT;
>
> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
> index ebf458dbcf84..31a209f0e18f 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -128,8 +128,9 @@ static inline unsigned int hv_repcomp(u64 status)
> * Rep hypercalls. Callers of this functions are supposed to ensure that
> * rep_count and varhead_size comply with Hyper-V hypercall definition.
Nit: This comment could be updated to include the new "rep_start"
parameter.
> */
> -static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
> - void *input, void *output)
> +static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count,
> + u16 varhead_size, u16 rep_start,
> + void *input, void *output)
> {
> u64 control = code;
> u64 status;
> @@ -137,6 +138,7 @@ static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
>
> control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET;
> control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET;
> + control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET;
>
> do {
> status = hv_do_hypercall(control, input, output);
> @@ -154,6 +156,14 @@ static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
> return status;
> }
>
> +/* For the typical case where rep_start is 0 */
> +static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
> + void *input, void *output)
> +{
> + return hv_do_rep_hypercall_ex(code, rep_count, varhead_size, 0,
> + input, output);
> +}
> +
> /* Generate the guest OS identifier as described in the Hyper-V TLFS */
> static inline u64 hv_generate_guest_id(u64 kernel_version)
> {
Overall, this looks good to me. I don't see any issues with the code.
Michael
^ permalink raw reply
* [PATCH] mshv: Fix deposit memory in MSHV_ROOT_HVCALL
From: Nuno Das Neves @ 2025-10-16 19:53 UTC (permalink / raw)
To: linux-hyperv, linux-kernel
Cc: kys, haiyangz, wei.liu, decui, arnd, mhklinux, mrathor,
skinsburskii, Nuno Das Neves
When the MSHV_ROOT_HVCALL ioctl is executing a hypercall, and gets
HV_STATUS_INSUFFICIENT_MEMORY, it deposits memory and then returns
-EAGAIN to userspace.
However, it's much easier and efficient if the driver simply deposits
memory on demand and immediately retries the hypercall as is done with
all the other hypercall helper functions.
But unlike those, in MSHV_ROOT_HVCALL the input is opaque to the
kernel. This is problematic for rep hypercalls, because the next part
of the input list can't be copied on each loop after depositing pages
(this was the original reason for returning -EAGAIN in this case).
Introduce hv_do_rep_hypercall_ex(), which adds a 'rep_start'
parameter. This solves the issue, allowing the deposit loop in
MSHV_ROOT_HVCALL to restart a rep hypercall after depositing pages
partway through.
Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
---
drivers/hv/mshv_root_main.c | 52 ++++++++++++++++++++--------------
include/asm-generic/mshyperv.h | 14 +++++++--
2 files changed, 42 insertions(+), 24 deletions(-)
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index 9ae67c6e9f60..731ec8cbbd63 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -159,6 +159,7 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition,
unsigned int pages_order;
void *input_pg = NULL;
void *output_pg = NULL;
+ u16 reps_completed;
if (copy_from_user(&args, user_args, sizeof(args)))
return -EFAULT;
@@ -210,28 +211,35 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition,
*/
*(u64 *)input_pg = partition->pt_id;
- if (args.reps)
- status = hv_do_rep_hypercall(args.code, args.reps, 0,
- input_pg, output_pg);
- else
- status = hv_do_hypercall(args.code, input_pg, output_pg);
-
- if (hv_result(status) == HV_STATUS_CALL_PENDING) {
- if (is_async) {
- mshv_async_hvcall_handler(partition, &status);
- } else { /* Paranoia check. This shouldn't happen! */
- ret = -EBADFD;
- goto free_pages_out;
+ reps_completed = 0;
+ do {
+ if (args.reps) {
+ status = hv_do_rep_hypercall_ex(args.code, args.reps,
+ 0, reps_completed,
+ input_pg, output_pg);
+ reps_completed = hv_repcomp(status);
+ } else {
+ status = hv_do_hypercall(args.code, input_pg, output_pg);
}
- }
- if (hv_result(status) == HV_STATUS_INSUFFICIENT_MEMORY) {
- ret = hv_call_deposit_pages(NUMA_NO_NODE, partition->pt_id, 1);
- if (!ret)
- ret = -EAGAIN;
- } else if (!hv_result_success(status)) {
- ret = hv_result_to_errno(status);
- }
+ if (hv_result(status) == HV_STATUS_CALL_PENDING) {
+ if (is_async) {
+ mshv_async_hvcall_handler(partition, &status);
+ } else { /* Paranoia check. This shouldn't happen! */
+ ret = -EBADFD;
+ goto free_pages_out;
+ }
+ }
+
+ if (hv_result_success(status))
+ break;
+
+ if (hv_result(status) != HV_STATUS_INSUFFICIENT_MEMORY)
+ ret = hv_result_to_errno(status);
+ else
+ ret = hv_call_deposit_pages(NUMA_NO_NODE,
+ partition->pt_id, 1);
+ } while (!ret);
/*
* Always return the status and output data regardless of result.
@@ -240,11 +248,11 @@ static int mshv_ioctl_passthru_hvcall(struct mshv_partition *partition,
* succeeded.
*/
args.status = hv_result(status);
- args.reps = args.reps ? hv_repcomp(status) : 0;
+ args.reps = reps_completed;
if (copy_to_user(user_args, &args, sizeof(args)))
ret = -EFAULT;
- if (output_pg &&
+ if (!ret && output_pg &&
copy_to_user((void __user *)args.out_ptr, output_pg, args.out_sz))
ret = -EFAULT;
diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
index ebf458dbcf84..31a209f0e18f 100644
--- a/include/asm-generic/mshyperv.h
+++ b/include/asm-generic/mshyperv.h
@@ -128,8 +128,9 @@ static inline unsigned int hv_repcomp(u64 status)
* Rep hypercalls. Callers of this functions are supposed to ensure that
* rep_count and varhead_size comply with Hyper-V hypercall definition.
*/
-static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
- void *input, void *output)
+static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count,
+ u16 varhead_size, u16 rep_start,
+ void *input, void *output)
{
u64 control = code;
u64 status;
@@ -137,6 +138,7 @@ static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET;
control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET;
+ control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET;
do {
status = hv_do_hypercall(control, input, output);
@@ -154,6 +156,14 @@ static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
return status;
}
+/* For the typical case where rep_start is 0 */
+static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
+ void *input, void *output)
+{
+ return hv_do_rep_hypercall_ex(code, rep_count, varhead_size, 0,
+ input, output);
+}
+
/* Generate the guest OS identifier as described in the Hyper-V TLFS */
static inline u64 hv_generate_guest_id(u64 kernel_version)
{
--
2.34.1
^ permalink raw reply related
* RE: [PATCH v2 2/2] hyperv: Enable clean shutdown for root partition with MSHV
From: Michael Kelley @ 2025-10-16 19:29 UTC (permalink / raw)
To: Praveen K Paladugu, kys@microsoft.com, haiyangz@microsoft.com,
wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de,
mingo@redhat.com, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, bp@alien8.de,
dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
arnd@arndb.de
Cc: anbelski@linux.microsoft.com,
easwar.hariharan@linux.microsoft.com,
nunodasneves@linux.microsoft.com,
skinsburskii@linux.microsoft.com
In-Reply-To: <20251014164150.6935-3-prapal@linux.microsoft.com>
From: Praveen K Paladugu <prapal@linux.microsoft.com> Sent: Tuesday, October 14, 2025 9:41 AM
>
> When a shutdown is initiated in the root partition without configuring
> sleep states, the call to `hv_call_enter_sleep_state` fails. In such cases
> the root falls back to using legacy ACPI mechanisms to poweroff. This call
> is intercepted by MSHV and will result in a Machine Check Exception (MCE).
>
> Root panics with a trace similar to:
>
> [ 81.306348] reboot: Power down
> [ 81.314709] mce: [Hardware Error]: CPU 0: Machine Check Exception: 4 Bank 0: b2000000c0060001
> [ 81.314711] mce: [Hardware Error]: TSC 3b8cb60a66 PPIN 11d98332458e4ea9
> [ 81.314713] mce: [Hardware Error]: PROCESSOR 0:606a6 TIME 1759339405 SOCKET 0 APIC 0 microcode ffffffff
> [ 81.314715] mce: [Hardware Error]: Run the above through 'mcelog --ascii'
> [ 81.314716] mce: [Hardware Error]: Machine check: Processor context corrupt
> [ 81.314717] Kernel panic - not syncing: Fatal machine check
>
> To prevent this, properly configure sleep states within MSHV, allowing
> the root partition to shut down cleanly without triggering a panic.
>
> Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
> Co-developed-by: Anatol Belski <anbelski@linux.microsoft.com>
> Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
> ---
> arch/x86/hyperv/hv_init.c | 7 ++
> arch/x86/include/asm/mshyperv.h | 1 +
> drivers/hv/hv_common.c | 119 ++++++++++++++++++++++++++++++++
> 3 files changed, 127 insertions(+)
>
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index afdbda2dd7b7..57bd96671ead 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -510,6 +510,13 @@ void __init hyperv_init(void)
> memunmap(src);
>
> hv_remap_tsc_clocksource();
> + /*
> + * The notifier registration might fail at various hops.
> + * Corresponding error messages will land in dmesg. There is
> + * otherwise nothing that can be specifically done to handle
> + * failures here.
> + */
> + (void)hv_sleep_notifiers_register();
> } else {
> hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> wrmsrq(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
> index abc4659f5809..fb8d691193df 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -236,6 +236,7 @@ int hyperv_fill_flush_guest_mapping_list(
> void hv_apic_init(void);
> void __init hv_init_spinlocks(void);
> bool hv_vcpu_is_preempted(int vcpu);
> +int hv_sleep_notifiers_register(void);
> #else
> static inline void hv_apic_init(void) {}
> #endif
> diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> index e109a620c83f..cfba9ded7bcb 100644
> --- a/drivers/hv/hv_common.c
> +++ b/drivers/hv/hv_common.c
> @@ -837,3 +837,122 @@ const char *hv_result_to_string(u64 status)
> return "Unknown";
> }
> EXPORT_SYMBOL_GPL(hv_result_to_string);
> +
> +#if IS_ENABLED(CONFIG_ACPI)
> +/*
> + * Corresponding sleep states have to be initialized in order for a subsequent
> + * HVCALL_ENTER_SLEEP_STATE call to succeed. Currently only S5 state as per
> + * ACPI 6.4 chapter 7.4.2 is relevant, while S1, S2 and S3 can be supported.
> + *
> + * ACPI should be initialized and should support S5 sleep state when this method
> + * is called, so that it can extract correct PM values and pass them to hv.
> + */
> +static int hv_initialize_sleep_states(void)
> +{
> + u64 status;
> + unsigned long flags;
> + struct hv_input_set_system_property *in;
> + acpi_status acpi_status;
> + u8 sleep_type_a, sleep_type_b;
> +
> + if (!acpi_sleep_state_supported(ACPI_STATE_S5)) {
> + pr_err("%s: S5 sleep state not supported.\n", __func__);
> + return -ENODEV;
> + }
> +
> + acpi_status = acpi_get_sleep_type_data(ACPI_STATE_S5,
> + &sleep_type_a, &sleep_type_b);
> + if (ACPI_FAILURE(acpi_status))
> + return -ENODEV;
> +
> + local_irq_save(flags);
> + in = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + memset(in, 0, sizeof(*in));
> +
> + in->property_id = HV_SYSTEM_PROPERTY_SLEEP_STATE;
> + in->set_sleep_state_info.sleep_state = HV_SLEEP_STATE_S5;
> + in->set_sleep_state_info.pm1a_slp_typ = sleep_type_a;
> + in->set_sleep_state_info.pm1b_slp_typ = sleep_type_b;
> +
> + status = hv_do_hypercall(HVCALL_SET_SYSTEM_PROPERTY, in, NULL);
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status)) {
> + hv_status_err(status, "\n");
> + return hv_result_to_errno(status);
> + }
> +
> + return 0;
> +}
> +
> +static int hv_call_enter_sleep_state(u32 sleep_state)
> +{
> + u64 status;
> + int ret;
> + unsigned long flags;
> + struct hv_input_enter_sleep_state *in;
> +
> + ret = hv_initialize_sleep_states();
> + if (ret)
> + return ret;
> +
> + local_irq_save(flags);
> + in = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + in->sleep_state = sleep_state;
> +
> + status = hv_do_hypercall(HVCALL_ENTER_SLEEP_STATE, in, NULL);
If this hypercall succeeds, does the root partition (which is the caller) go
to sleep in S5, such that the hypercall never returns? If that's not the case,
what is the behavior of this hypercall?
> + local_irq_restore(flags);
> +
> + if (!hv_result_success(status)) {
> + hv_status_err(status, "\n");
> + return hv_result_to_errno(status);
> + }
> +
> + return 0;
> +}
> +
> +static int hv_reboot_notifier_handler(struct notifier_block *this,
> + unsigned long code, void *another)
> +{
> + int ret = 0;
> +
> + if (code == SYS_HALT || code == SYS_POWER_OFF)
> + ret = hv_call_enter_sleep_state(HV_SLEEP_STATE_S5);
If hv_call_enter_sleep_state() never returns, here's an issue. There may be
multiple entries on the reboot notifier chain. For example,
mshv_root_partition_init() puts an entry on the reboot notifier chain. At
reboot time, the entries are executed in some order, with the expectation
that all entries will be executed prior to the reboot actually happening. But
if this hypercall never returns, some entries may never be executed.
Notifier chains support a notion of priority to control the order in
which they are executed, but that priority isn't set in hv_reboot_notifier
below, or in mshv_reboot_nb. And most other reboot notifiers throughout
Linux appear to not set it. So the ordering is unspecified, and having
this notifier never return may be problematic.
> +
> + return ret ? NOTIFY_DONE : NOTIFY_OK;
> +}
> +
> +static struct notifier_block hv_reboot_notifier = {
> + .notifier_call = hv_reboot_notifier_handler,
> +};
> +
> +static int hv_acpi_sleep_handler(u8 sleep_state, u32 pm1a_cnt, u32 pm1b_cnt)
> +{
> + int ret = 0;
> +
> + if (sleep_state == ACPI_STATE_S5)
> + ret = hv_call_enter_sleep_state(HV_SLEEP_STATE_S5);
> +
> + return ret == 0 ? 1 : -1;
> +}
> +
> +static int hv_acpi_extended_sleep_handler(u8 sleep_state, u32 val_a, u32 val_b)
> +{
> + return hv_acpi_sleep_handler(sleep_state, val_a, val_b);
> +}
Is this function needed? The function signature is identical to hv_acpi_sleep_handler().
So it seems like acpi_os_set_prepare_extended_sleep() could just use
hv_acpi_sleep_handler() directly.
> +
> +int hv_sleep_notifiers_register(void)
> +{
> + int ret;
> +
> + acpi_os_set_prepare_sleep(&hv_acpi_sleep_handler);
> + acpi_os_set_prepare_extended_sleep(&hv_acpi_extended_sleep_handler);
I'm not clear on why these handlers are set. If the hv_reboot_notifier is
called, are these ACPI handlers ever called? Or are these to catch any cases
where the hv_reboot_notifier is somehow bypassed? Or maybe I'm just
not understanding something .... :-)
> +
> + ret = register_reboot_notifier(&hv_reboot_notifier);
> + if (ret)
> + pr_err("%s: cannot register reboot notifier %d\n",
> + __func__, ret);
> +
> + return ret;
> +}
> +#endif
I'm wondering if all this code belongs in hv_common.c, since it is only needed
for Linux in the root partition. Couldn't it go in mshv_common.c? It would still
be built-in code (i.e., not in a loadable module), but only if CONFIG_MSHV_ROOT
is set.
Michael
> --
> 2.51.0
>
^ permalink raw reply
* Re: [PATCH v5 5/5] Drivers: hv: Add support for movable memory regions
From: kernel test robot @ 2025-10-16 14:47 UTC (permalink / raw)
To: Stanislav Kinsburskii, kys, haiyangz, wei.liu, decui
Cc: oe-kbuild-all, linux-hyperv, linux-kernel
In-Reply-To: <176057443695.74314.10584965103467299030.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
Hi Stanislav,
kernel test robot noticed the following build errors:
[auto build test ERROR on linus/master]
[also build test ERROR on v6.18-rc1 next-20251015]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Stanislav-Kinsburskii/Drivers-hv-Refactor-and-rename-memory-region-handling-functions/20251016-082944
base: linus/master
patch link: https://lore.kernel.org/r/176057443695.74314.10584965103467299030.stgit%40skinsburskii-cloud-desktop.internal.cloudapp.net
patch subject: [PATCH v5 5/5] Drivers: hv: Add support for movable memory regions
config: x86_64-buildonly-randconfig-002-20251016 (https://download.01.org/0day-ci/archive/20251016/202510162231.7UOw1jQq-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251016/202510162231.7UOw1jQq-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202510162231.7UOw1jQq-lkp@intel.com/
All errors (new ones prefixed by >>):
mm/hmm.c: In function 'hmm_range_fault':
>> mm/hmm.c:667:21: error: implicit declaration of function 'mmu_interval_check_retry' [-Wimplicit-function-declaration]
667 | if (mmu_interval_check_retry(range->notifier,
| ^~~~~~~~~~~~~~~~~~~~~~~~
vim +/mmu_interval_check_retry +667 mm/hmm.c
7b86ac3371b70c Christoph Hellwig 2019-08-28 634
9a4903e49e495b Christoph Hellwig 2019-07-25 635 /**
9a4903e49e495b Christoph Hellwig 2019-07-25 636 * hmm_range_fault - try to fault some address in a virtual address range
f970b977e068aa Jason Gunthorpe 2020-03-27 637 * @range: argument structure
9a4903e49e495b Christoph Hellwig 2019-07-25 638 *
be957c886d92aa Jason Gunthorpe 2020-05-01 639 * Returns 0 on success or one of the following error codes:
73231612dc7c90 Jérôme Glisse 2019-05-13 640 *
9a4903e49e495b Christoph Hellwig 2019-07-25 641 * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma
9a4903e49e495b Christoph Hellwig 2019-07-25 642 * (e.g., device file vma).
73231612dc7c90 Jérôme Glisse 2019-05-13 643 * -ENOMEM: Out of memory.
9a4903e49e495b Christoph Hellwig 2019-07-25 644 * -EPERM: Invalid permission (e.g., asking for write and range is read
9a4903e49e495b Christoph Hellwig 2019-07-25 645 * only).
9a4903e49e495b Christoph Hellwig 2019-07-25 646 * -EBUSY: The range has been invalidated and the caller needs to wait for
9a4903e49e495b Christoph Hellwig 2019-07-25 647 * the invalidation to finish.
f970b977e068aa Jason Gunthorpe 2020-03-27 648 * -EFAULT: A page was requested to be valid and could not be made valid
f970b977e068aa Jason Gunthorpe 2020-03-27 649 * ie it has no backing VMA or it is illegal to access
74eee180b935fc Jérôme Glisse 2017-09-08 650 *
f970b977e068aa Jason Gunthorpe 2020-03-27 651 * This is similar to get_user_pages(), except that it can read the page tables
f970b977e068aa Jason Gunthorpe 2020-03-27 652 * without mutating them (ie causing faults).
74eee180b935fc Jérôme Glisse 2017-09-08 653 */
be957c886d92aa Jason Gunthorpe 2020-05-01 654 int hmm_range_fault(struct hmm_range *range)
74eee180b935fc Jérôme Glisse 2017-09-08 655 {
d28c2c9a487708 Ralph Campbell 2019-11-04 656 struct hmm_vma_walk hmm_vma_walk = {
d28c2c9a487708 Ralph Campbell 2019-11-04 657 .range = range,
d28c2c9a487708 Ralph Campbell 2019-11-04 658 .last = range->start,
d28c2c9a487708 Ralph Campbell 2019-11-04 659 };
a22dd506400d0f Jason Gunthorpe 2019-11-12 660 struct mm_struct *mm = range->notifier->mm;
74eee180b935fc Jérôme Glisse 2017-09-08 661 int ret;
74eee180b935fc Jérôme Glisse 2017-09-08 662
42fc541404f249 Michel Lespinasse 2020-06-08 663 mmap_assert_locked(mm);
74eee180b935fc Jérôme Glisse 2017-09-08 664
a3e0d41c2b1f86 Jérôme Glisse 2019-05-13 665 do {
a3e0d41c2b1f86 Jérôme Glisse 2019-05-13 666 /* If range is no longer valid force retry. */
a22dd506400d0f Jason Gunthorpe 2019-11-12 @667 if (mmu_interval_check_retry(range->notifier,
a22dd506400d0f Jason Gunthorpe 2019-11-12 668 range->notifier_seq))
2bcbeaefde2f03 Christoph Hellwig 2019-07-24 669 return -EBUSY;
d28c2c9a487708 Ralph Campbell 2019-11-04 670 ret = walk_page_range(mm, hmm_vma_walk.last, range->end,
7b86ac3371b70c Christoph Hellwig 2019-08-28 671 &hmm_walk_ops, &hmm_vma_walk);
be957c886d92aa Jason Gunthorpe 2020-05-01 672 /*
be957c886d92aa Jason Gunthorpe 2020-05-01 673 * When -EBUSY is returned the loop restarts with
be957c886d92aa Jason Gunthorpe 2020-05-01 674 * hmm_vma_walk.last set to an address that has not been stored
be957c886d92aa Jason Gunthorpe 2020-05-01 675 * in pfns. All entries < last in the pfn array are set to their
be957c886d92aa Jason Gunthorpe 2020-05-01 676 * output, and all >= are still at their input values.
be957c886d92aa Jason Gunthorpe 2020-05-01 677 */
d28c2c9a487708 Ralph Campbell 2019-11-04 678 } while (ret == -EBUSY);
73231612dc7c90 Jérôme Glisse 2019-05-13 679 return ret;
74eee180b935fc Jérôme Glisse 2017-09-08 680 }
73231612dc7c90 Jérôme Glisse 2019-05-13 681 EXPORT_SYMBOL(hmm_range_fault);
8cad4713056612 Leon Romanovsky 2025-04-28 682
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* [PATCH v5 5/5] Drivers: hv: Add support for movable memory regions
From: Stanislav Kinsburskii @ 2025-10-16 0:27 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui; +Cc: linux-hyperv, linux-kernel
In-Reply-To: <176057396465.74314.10055784909009416453.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
Introduce support for movable memory regions in the Hyper-V root partition
driver, thus improving memory management flexibility and preparing the
driver for advanced use cases such as dynamic memory remapping.
Integrate mmu_interval_notifier for movable regions, implement functions to
handle HMM faults and memory invalidation, and update memory region mapping
logic to support movable regions.
While MMU notifiers are commonly used in virtualization drivers, this
implementation leverages HMM (Heterogeneous Memory Management) for its
tailored functionality. HMM provides a ready-made framework for mirroring,
invalidation, and fault handling, avoiding the need to reimplement these
mechanisms for a single callback. Although MMU notifiers are more generic,
using HMM reduces boilerplate and ensures maintainability by utilizing a
mechanism specifically designed for such use cases.
Signed-off-by: Anirudh Rayabharam <anrayabh@linux.microsoft.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
---
drivers/hv/Kconfig | 1
drivers/hv/mshv_root.h | 8 +
drivers/hv/mshv_root_main.c | 328 ++++++++++++++++++++++++++++++++++++++++++-
3 files changed, 327 insertions(+), 10 deletions(-)
diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig
index 0b8c391a0342..5f1637cbb6e3 100644
--- a/drivers/hv/Kconfig
+++ b/drivers/hv/Kconfig
@@ -75,6 +75,7 @@ config MSHV_ROOT
depends on PAGE_SIZE_4KB
select EVENTFD
select VIRT_XFER_TO_GUEST_WORK
+ select HMM_MIRROR
default n
help
Select this option to enable support for booting and running as root
diff --git a/drivers/hv/mshv_root.h b/drivers/hv/mshv_root.h
index 97e64d5341b6..13367c84497c 100644
--- a/drivers/hv/mshv_root.h
+++ b/drivers/hv/mshv_root.h
@@ -15,6 +15,7 @@
#include <linux/hashtable.h>
#include <linux/dev_printk.h>
#include <linux/build_bug.h>
+#include <linux/mmu_notifier.h>
#include <uapi/linux/mshv.h>
/*
@@ -81,9 +82,14 @@ struct mshv_mem_region {
struct {
u64 large_pages: 1; /* 2MiB */
u64 range_pinned: 1;
- u64 reserved: 62;
+ u64 is_ram : 1; /* mem region can be ram or mmio */
+ u64 reserved: 61;
} flags;
struct mshv_partition *partition;
+#if defined(CONFIG_MMU_NOTIFIER)
+ struct mmu_interval_notifier mni;
+ struct mutex mutex; /* protects region pages remapping */
+#endif
struct page *pages[];
};
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index c4f114376435..b2738443ac5d 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -29,6 +29,7 @@
#include <linux/crash_dump.h>
#include <linux/panic_notifier.h>
#include <linux/vmalloc.h>
+#include <linux/hmm.h>
#include "mshv_eventfd.h"
#include "mshv.h"
@@ -36,6 +37,8 @@
#define VALUE_PMD_ALIGNED(c) (!((c) & (PTRS_PER_PMD - 1)))
+#define MSHV_MAP_FAULT_IN_PAGES HPAGE_PMD_NR
+
MODULE_AUTHOR("Microsoft");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Microsoft Hyper-V root partition VMM interface /dev/mshv");
@@ -76,6 +79,11 @@ static int mshv_vp_mmap(struct file *file, struct vm_area_struct *vma);
static vm_fault_t mshv_vp_fault(struct vm_fault *vmf);
static int mshv_init_async_handler(struct mshv_partition *partition);
static void mshv_async_hvcall_handler(void *data, u64 *status);
+static struct mshv_mem_region
+ *mshv_partition_region_by_gfn(struct mshv_partition *pt, u64 gfn);
+static int mshv_region_remap_pages(struct mshv_mem_region *region,
+ u32 map_flags, u64 page_offset,
+ u64 page_count);
static const union hv_input_vtl input_vtl_zero;
static const union hv_input_vtl input_vtl_normal = {
@@ -602,14 +610,197 @@ static long mshv_run_vp_with_root_scheduler(struct mshv_vp *vp)
static_assert(sizeof(struct hv_message) <= MSHV_RUN_VP_BUF_SZ,
"sizeof(struct hv_message) must not exceed MSHV_RUN_VP_BUF_SZ");
+#ifdef CONFIG_X86_64
+
+#if defined(CONFIG_MMU_NOTIFIER)
+/**
+ * mshv_region_hmm_fault_and_lock - Handle HMM faults and lock the memory region
+ * @region: Pointer to the memory region structure
+ * @range: Pointer to the HMM range structure
+ *
+ * This function performs the following steps:
+ * 1. Reads the notifier sequence for the HMM range.
+ * 2. Acquires a read lock on the memory map.
+ * 3. Handles HMM faults for the specified range.
+ * 4. Releases the read lock on the memory map.
+ * 5. If successful, locks the memory region mutex.
+ * 6. Verifies if the notifier sequence has changed during the operation.
+ * If it has, releases the mutex and returns -EBUSY to match with
+ * hmm_range_fault() return code for repeating.
+ *
+ * Return: 0 on success, a negative error code otherwise.
+ */
+static int mshv_region_hmm_fault_and_lock(struct mshv_mem_region *region,
+ struct hmm_range *range)
+{
+ int ret;
+
+ range->notifier_seq = mmu_interval_read_begin(range->notifier);
+ mmap_read_lock(region->mni.mm);
+ ret = hmm_range_fault(range);
+ mmap_read_unlock(region->mni.mm);
+ if (ret)
+ return ret;
+
+ mutex_lock(®ion->mutex);
+
+ if (mmu_interval_read_retry(range->notifier, range->notifier_seq)) {
+ mutex_unlock(®ion->mutex);
+ cond_resched();
+ return -EBUSY;
+ }
+
+ return 0;
+}
+
+/**
+ * mshv_region_range_fault - Handle memory range faults for a given region.
+ * @region: Pointer to the memory region structure.
+ * @page_offset: Offset of the page within the region.
+ * @page_count: Number of pages to handle.
+ *
+ * This function resolves memory faults for a specified range of pages
+ * within a memory region. It uses HMM (Heterogeneous Memory Management)
+ * to fault in the required pages and updates the region's page array.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+static int mshv_region_range_fault(struct mshv_mem_region *region,
+ u64 page_offset, u64 page_count)
+{
+ struct hmm_range range = {
+ .notifier = ®ion->mni,
+ .default_flags = HMM_PFN_REQ_FAULT | HMM_PFN_REQ_WRITE,
+ };
+ unsigned long *pfns;
+ int ret;
+ u64 i;
+
+ pfns = kmalloc_array(page_count, sizeof(unsigned long), GFP_KERNEL);
+ if (!pfns)
+ return -ENOMEM;
+
+ range.hmm_pfns = pfns;
+ range.start = region->start_uaddr + page_offset * HV_HYP_PAGE_SIZE;
+ range.end = range.start + page_count * HV_HYP_PAGE_SIZE;
+
+ do {
+ ret = mshv_region_hmm_fault_and_lock(region, &range);
+ } while (ret == -EBUSY);
+
+ if (ret)
+ goto out;
+
+ for (i = 0; i < page_count; i++)
+ region->pages[page_offset + i] = hmm_pfn_to_page(pfns[i]);
+
+ if (PageHuge(region->pages[page_offset]))
+ region->flags.large_pages = true;
+
+ ret = mshv_region_remap_pages(region, region->hv_map_flags,
+ page_offset, page_count);
+
+ mutex_unlock(®ion->mutex);
+out:
+ kfree(pfns);
+ return ret;
+}
+#else /* CONFIG_MMU_NOTIFIER */
+static int mshv_region_range_fault(struct mshv_mem_region *region,
+ u64 page_offset, u64 page_count)
+{
+ return -ENODEV;
+}
+#endif /* CONFIG_MMU_NOTIFIER */
+
+static bool mshv_region_handle_gfn_fault(struct mshv_mem_region *region, u64 gfn)
+{
+ u64 page_offset, page_count;
+ int ret;
+
+ if (WARN_ON_ONCE(region->flags.range_pinned))
+ return false;
+
+ /* Align the page offset to the nearest MSHV_MAP_FAULT_IN_PAGES. */
+ page_offset = ALIGN_DOWN(gfn - region->start_gfn,
+ MSHV_MAP_FAULT_IN_PAGES);
+
+ /* Map more pages than requested to reduce the number of faults. */
+ page_count = min(region->nr_pages - page_offset,
+ MSHV_MAP_FAULT_IN_PAGES);
+
+ ret = mshv_region_range_fault(region, page_offset, page_count);
+
+ WARN_ONCE(ret,
+ "p%llu: GPA intercept failed: region %#llx-%#llx, gfn %#llx, page_offset %llu, page_count %llu\n",
+ region->partition->pt_id, region->start_uaddr,
+ region->start_uaddr + (region->nr_pages << HV_HYP_PAGE_SHIFT),
+ gfn, page_offset, page_count);
+
+ return !ret;
+}
+
+/**
+ * mshv_handle_gpa_intercept - Handle GPA (Guest Physical Address) intercepts.
+ * @vp: Pointer to the virtual processor structure.
+ *
+ * This function processes GPA intercepts by identifying the memory region
+ * corresponding to the intercepted GPA, aligning the page offset, and
+ * mapping the required pages. It ensures that the region is valid and
+ * handles faults efficiently by mapping multiple pages at once.
+ *
+ * Return: true if the intercept was handled successfully, false otherwise.
+ */
+static bool mshv_handle_gpa_intercept(struct mshv_vp *vp)
+{
+ struct mshv_partition *p = vp->vp_partition;
+ struct mshv_mem_region *region;
+ struct hv_x64_memory_intercept_message *msg;
+ u64 gfn;
+
+ msg = (struct hv_x64_memory_intercept_message *)
+ vp->vp_intercept_msg_page->u.payload;
+
+ gfn = HVPFN_DOWN(msg->guest_physical_address);
+
+ region = mshv_partition_region_by_gfn(p, gfn);
+ if (!region)
+ return false;
+
+ if (WARN_ON_ONCE(!region->flags.is_ram))
+ return false;
+
+ if (WARN_ON_ONCE(region->flags.range_pinned))
+ return false;
+
+ return mshv_region_handle_gfn_fault(region, gfn);
+}
+
+#else /* CONFIG_X86_64 */
+
+static bool mshv_handle_gpa_intercept(struct mshv_vp *vp) { return false; }
+
+#endif /* CONFIG_X86_64 */
+
+static bool mshv_vp_handle_intercept(struct mshv_vp *vp)
+{
+ switch (vp->vp_intercept_msg_page->header.message_type) {
+ case HVMSG_GPA_INTERCEPT:
+ return mshv_handle_gpa_intercept(vp);
+ }
+ return false;
+}
+
static long mshv_vp_ioctl_run_vp(struct mshv_vp *vp, void __user *ret_msg)
{
long rc;
- if (hv_scheduler_type == HV_SCHEDULER_TYPE_ROOT)
- rc = mshv_run_vp_with_root_scheduler(vp);
- else
- rc = mshv_run_vp_with_hyp_scheduler(vp);
+ do {
+ if (hv_scheduler_type == HV_SCHEDULER_TYPE_ROOT)
+ rc = mshv_run_vp_with_root_scheduler(vp);
+ else
+ rc = mshv_run_vp_with_hyp_scheduler(vp);
+ } while (rc == 0 && mshv_vp_handle_intercept(vp));
if (rc)
return rc;
@@ -1209,6 +1400,110 @@ mshv_partition_region_by_uaddr(struct mshv_partition *partition, u64 uaddr)
return NULL;
}
+#if defined(CONFIG_MMU_NOTIFIER)
+static void mshv_region_movable_fini(struct mshv_mem_region *region)
+{
+ if (region->flags.range_pinned)
+ return;
+
+ mmu_interval_notifier_remove(®ion->mni);
+}
+
+/**
+ * mshv_region_interval_invalidate - Invalidate a range of memory region
+ * @mni: Pointer to the mmu_interval_notifier structure
+ * @range: Pointer to the mmu_notifier_range structure
+ * @cur_seq: Current sequence number for the interval notifier
+ *
+ * This function invalidates a memory region by remapping its pages with
+ * no access permissions. It locks the region's mutex to ensure thread safety
+ * and updates the sequence number for the interval notifier. If the range
+ * is blockable, it uses a blocking lock; otherwise, it attempts a non-blocking
+ * lock and returns false if unsuccessful.
+ *
+ * NOTE: Failure to invalidate a region is a serious error, as the pages will
+ * be considered freed while they are still mapped by the hypervisor.
+ * Any attempt to access such pages will likely crash the system.
+ *
+ * Return: true if the region was successfully invalidated, false otherwise.
+ */
+static bool
+mshv_region_interval_invalidate(struct mmu_interval_notifier *mni,
+ const struct mmu_notifier_range *range,
+ unsigned long cur_seq)
+{
+ struct mshv_mem_region *region = container_of(mni,
+ struct mshv_mem_region,
+ mni);
+ u64 page_offset, page_count;
+ unsigned long mstart, mend;
+ int ret = -EPERM;
+
+ if (mmu_notifier_range_blockable(range))
+ mutex_lock(®ion->mutex);
+ else if (!mutex_trylock(®ion->mutex))
+ goto out_fail;
+
+ mmu_interval_set_seq(mni, cur_seq);
+
+ mstart = max(range->start, region->start_uaddr);
+ mend = min(range->end, region->start_uaddr +
+ (region->nr_pages << HV_HYP_PAGE_SHIFT));
+
+ page_offset = HVPFN_DOWN(mstart - region->start_uaddr);
+ page_count = HVPFN_DOWN(mend - mstart);
+
+ ret = mshv_region_remap_pages(region, HV_MAP_GPA_NO_ACCESS,
+ page_offset, page_count);
+ if (ret)
+ goto out_fail;
+
+ mshv_region_invalidate_pages(region, page_offset, page_count);
+
+ mutex_unlock(®ion->mutex);
+
+ return true;
+
+out_fail:
+ WARN_ONCE(ret,
+ "Failed to invalidate region %#llx-%#llx (range %#lx-%#lx, event: %u, pages %#llx-%#llx, mm: %#llx): %d\n",
+ region->start_uaddr,
+ region->start_uaddr + (region->nr_pages << HV_HYP_PAGE_SHIFT),
+ range->start, range->end, range->event,
+ page_offset, page_offset + page_count - 1, (u64)range->mm, ret);
+ return false;
+}
+
+static const struct mmu_interval_notifier_ops mshv_region_mni_ops = {
+ .invalidate = mshv_region_interval_invalidate,
+};
+
+static bool mshv_region_movable_init(struct mshv_mem_region *region)
+{
+ int ret;
+
+ ret = mmu_interval_notifier_insert(®ion->mni, current->mm,
+ region->start_uaddr,
+ region->nr_pages << HV_HYP_PAGE_SHIFT,
+ &mshv_region_mni_ops);
+ if (ret)
+ return false;
+
+ mutex_init(®ion->mutex);
+
+ return true;
+}
+#else
+static inline void mshv_region_movable_fini(struct mshv_mem_region *region)
+{
+}
+
+static inline bool mshv_region_movable_init(struct mshv_mem_region *region)
+{
+ return false;
+}
+#endif
+
/*
* NB: caller checks and makes sure mem->size is page aligned
* Returns: 0 with regionpp updated on success, or -errno
@@ -1241,9 +1536,14 @@ static int mshv_partition_create_region(struct mshv_partition *partition,
if (mem->flags & BIT(MSHV_SET_MEM_BIT_EXECUTABLE))
region->hv_map_flags |= HV_MAP_GPA_EXECUTABLE;
- /* Note: large_pages flag populated when we pin the pages */
- if (!is_mmio)
- region->flags.range_pinned = true;
+ /* Note: large_pages flag populated when pages are allocated. */
+ if (!is_mmio) {
+ region->flags.is_ram = true;
+
+ if (mshv_partition_encrypted(partition) ||
+ !mshv_region_movable_init(region))
+ region->flags.range_pinned = true;
+ }
region->partition = partition;
@@ -1363,9 +1663,16 @@ mshv_map_user_memory(struct mshv_partition *partition,
if (is_mmio)
ret = hv_call_map_mmio_pages(partition->pt_id, mem.guest_pfn,
mmio_pfn, HVPFN_DOWN(mem.size));
- else
+ else if (region->flags.range_pinned)
ret = mshv_prepare_pinned_region(region);
-
+ else
+ /*
+ * For non-pinned regions, remap with no access to let the
+ * hypervisor track dirty pages, enabling pre-copy live
+ * migration.
+ */
+ ret = mshv_region_remap_pages(region, HV_MAP_GPA_NO_ACCESS,
+ 0, region->nr_pages);
if (ret)
goto errout;
@@ -1388,6 +1695,9 @@ static void mshv_partition_destroy_region(struct mshv_mem_region *region)
hlist_del(®ion->hnode);
+ if (region->flags.is_ram)
+ mshv_region_movable_fini(region);
+
if (mshv_partition_encrypted(partition)) {
ret = mshv_partition_region_share(region);
if (ret) {
^ permalink raw reply related
* [PATCH v5 4/5] Drivers: hv: Ensure large page GPA mapping is PMD-aligned
From: Stanislav Kinsburskii @ 2025-10-16 0:27 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui; +Cc: linux-hyperv, linux-kernel
In-Reply-To: <176057396465.74314.10055784909009416453.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
With the upcoming introduction of movable pages, a region doesn't guarantee
always having large pages mapped. Both mapping on fault and unmapping
during PTE invalidation may not be 2M-aligned, while the hypervisor
requires both the GFN and page count to be 2M-aligned to use the large page
flag.
Update the logic for large page mapping in mshv_region_remap_pages() to
require both page_offset and page_count to be PMD-aligned.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
---
drivers/hv/mshv_root_main.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index a3e5b41f3a7f..c4f114376435 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -34,6 +34,8 @@
#include "mshv.h"
#include "mshv_root.h"
+#define VALUE_PMD_ALIGNED(c) (!((c) & (PTRS_PER_PMD - 1)))
+
MODULE_AUTHOR("Microsoft");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Microsoft Hyper-V root partition VMM interface /dev/mshv");
@@ -1100,7 +1102,9 @@ mshv_region_remap_pages(struct mshv_mem_region *region, u32 map_flags,
if (page_offset + page_count > region->nr_pages)
return -EINVAL;
- if (region->flags.large_pages)
+ if (region->flags.large_pages &&
+ VALUE_PMD_ALIGNED(page_offset) &&
+ VALUE_PMD_ALIGNED(page_count))
map_flags |= HV_MAP_GPA_LARGE_PAGE;
/* ask the hypervisor to map guest ram */
^ permalink raw reply related
* [PATCH v5 3/5] Drivers: hv: Batch GPA unmap operations to improve large region performance
From: Stanislav Kinsburskii @ 2025-10-16 0:27 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui; +Cc: linux-hyperv, linux-kernel
In-Reply-To: <176057396465.74314.10055784909009416453.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
Reduce overhead when unmapping large memory regions by batching GPA unmap
operations in 2MB-aligned chunks.
Use a dedicated constant for batch size to improve code clarity and
maintainability.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
---
drivers/hv/mshv_root.h | 2 ++
drivers/hv/mshv_root_hv_call.c | 2 +-
drivers/hv/mshv_root_main.c | 28 +++++++++++++++++++++++++---
3 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/drivers/hv/mshv_root.h b/drivers/hv/mshv_root.h
index e3931b0f1269..97e64d5341b6 100644
--- a/drivers/hv/mshv_root.h
+++ b/drivers/hv/mshv_root.h
@@ -32,6 +32,8 @@ static_assert(HV_HYP_PAGE_SIZE == MSHV_HV_PAGE_SIZE);
#define MSHV_PIN_PAGES_BATCH_SIZE (0x10000000ULL / HV_HYP_PAGE_SIZE)
+#define MSHV_MAX_UNMAP_GPA_PAGES 512
+
struct mshv_vp {
u32 vp_index;
struct mshv_partition *vp_partition;
diff --git a/drivers/hv/mshv_root_hv_call.c b/drivers/hv/mshv_root_hv_call.c
index c9c274f29c3c..0696024ccfe3 100644
--- a/drivers/hv/mshv_root_hv_call.c
+++ b/drivers/hv/mshv_root_hv_call.c
@@ -17,7 +17,7 @@
/* Determined empirically */
#define HV_INIT_PARTITION_DEPOSIT_PAGES 208
#define HV_MAP_GPA_DEPOSIT_PAGES 256
-#define HV_UMAP_GPA_PAGES 512
+#define HV_UMAP_GPA_PAGES MSHV_MAX_UNMAP_GPA_PAGES
#define HV_PAGE_COUNT_2M_ALIGNED(pg_count) (!((pg_count) & (0x200 - 1)))
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index 97e322f3c6b5..a3e5b41f3a7f 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -1378,6 +1378,7 @@ mshv_map_user_memory(struct mshv_partition *partition,
static void mshv_partition_destroy_region(struct mshv_mem_region *region)
{
struct mshv_partition *partition = region->partition;
+ u64 gfn, gfn_count, start_gfn, end_gfn;
u32 unmap_flags = 0;
int ret;
@@ -1396,9 +1397,30 @@ static void mshv_partition_destroy_region(struct mshv_mem_region *region)
if (region->flags.large_pages)
unmap_flags |= HV_UNMAP_GPA_LARGE_PAGE;
- /* ignore unmap failures and continue as process may be exiting */
- hv_call_unmap_gpa_pages(partition->pt_id, region->start_gfn,
- region->nr_pages, unmap_flags);
+ start_gfn = region->start_gfn;
+ end_gfn = region->start_gfn + region->nr_pages;
+
+ for (gfn = start_gfn; gfn < end_gfn; gfn += gfn_count) {
+ if (gfn % MSHV_MAX_UNMAP_GPA_PAGES)
+ gfn_count = ALIGN(gfn, MSHV_MAX_UNMAP_GPA_PAGES) - gfn;
+ else
+ gfn_count = MSHV_MAX_UNMAP_GPA_PAGES;
+
+ if (gfn + gfn_count > end_gfn)
+ gfn_count = end_gfn - gfn;
+
+ /* Skip all pages in this range if none are mapped */
+ if (!memchr_inv(region->pages + (gfn - start_gfn), 0,
+ gfn_count * sizeof(struct page *)))
+ continue;
+
+ ret = hv_call_unmap_gpa_pages(partition->pt_id, gfn,
+ gfn_count, unmap_flags);
+ if (ret)
+ pt_err(partition,
+ "Failed to unmap GPA pages %#llx-%#llx: %d\n",
+ gfn, gfn + gfn_count - 1, ret);
+ }
mshv_region_invalidate(region);
^ permalink raw reply related
* [PATCH v5 2/5] Drivers: hv: Centralize guest memory region destruction
From: Stanislav Kinsburskii @ 2025-10-16 0:27 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui; +Cc: linux-hyperv, linux-kernel
In-Reply-To: <176057396465.74314.10055784909009416453.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
Centralize guest memory region destruction to prevent resource leaks and
inconsistent cleanup across unmap and partition destruction paths.
Unify region removal, encrypted partition access recovery, and region
invalidation to improve maintainability and reliability. Reduce code
duplication and make future updates less error-prone by encapsulating
cleanup logic in a single helper.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
---
drivers/hv/mshv_root_main.c | 65 ++++++++++++++++++++++---------------------
1 file changed, 34 insertions(+), 31 deletions(-)
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index e923947d3c54..97e322f3c6b5 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -1375,13 +1375,42 @@ mshv_map_user_memory(struct mshv_partition *partition,
return ret;
}
+static void mshv_partition_destroy_region(struct mshv_mem_region *region)
+{
+ struct mshv_partition *partition = region->partition;
+ u32 unmap_flags = 0;
+ int ret;
+
+ hlist_del(®ion->hnode);
+
+ if (mshv_partition_encrypted(partition)) {
+ ret = mshv_partition_region_share(region);
+ if (ret) {
+ pt_err(partition,
+ "Failed to regain access to memory, unpinning user pages will fail and crash the host error: %d\n",
+ ret);
+ return;
+ }
+ }
+
+ if (region->flags.large_pages)
+ unmap_flags |= HV_UNMAP_GPA_LARGE_PAGE;
+
+ /* ignore unmap failures and continue as process may be exiting */
+ hv_call_unmap_gpa_pages(partition->pt_id, region->start_gfn,
+ region->nr_pages, unmap_flags);
+
+ mshv_region_invalidate(region);
+
+ vfree(region);
+}
+
/* Called for unmapping both the guest ram and the mmio space */
static long
mshv_unmap_user_memory(struct mshv_partition *partition,
struct mshv_user_mem_region mem)
{
struct mshv_mem_region *region;
- u32 unmap_flags = 0;
if (!(mem.flags & BIT(MSHV_SET_MEM_BIT_UNMAP)))
return -EINVAL;
@@ -1396,18 +1425,8 @@ mshv_unmap_user_memory(struct mshv_partition *partition,
region->nr_pages != HVPFN_DOWN(mem.size))
return -EINVAL;
- hlist_del(®ion->hnode);
+ mshv_partition_destroy_region(region);
- if (region->flags.large_pages)
- unmap_flags |= HV_UNMAP_GPA_LARGE_PAGE;
-
- /* ignore unmap failures and continue as process may be exiting */
- hv_call_unmap_gpa_pages(partition->pt_id, region->start_gfn,
- region->nr_pages, unmap_flags);
-
- mshv_region_invalidate(region);
-
- vfree(region);
return 0;
}
@@ -1743,8 +1762,8 @@ static void destroy_partition(struct mshv_partition *partition)
{
struct mshv_vp *vp;
struct mshv_mem_region *region;
- int i, ret;
struct hlist_node *n;
+ int i;
if (refcount_read(&partition->pt_ref_count)) {
pt_err(partition,
@@ -1804,25 +1823,9 @@ static void destroy_partition(struct mshv_partition *partition)
remove_partition(partition);
- /* Remove regions, regain access to the memory and unpin the pages */
hlist_for_each_entry_safe(region, n, &partition->pt_mem_regions,
- hnode) {
- hlist_del(®ion->hnode);
-
- if (mshv_partition_encrypted(partition)) {
- ret = mshv_partition_region_share(region);
- if (ret) {
- pt_err(partition,
- "Failed to regain access to memory, unpinning user pages will fail and crash the host error: %d\n",
- ret);
- return;
- }
- }
-
- mshv_region_invalidate(region);
-
- vfree(region);
- }
+ hnode)
+ mshv_partition_destroy_region(region);
/* Withdraw and free all pages we deposited */
hv_call_withdraw_memory(U64_MAX, NUMA_NO_NODE, partition->pt_id);
^ permalink raw reply related
* [PATCH v5 1/5] Drivers: hv: Refactor and rename memory region handling functions
From: Stanislav Kinsburskii @ 2025-10-16 0:26 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui; +Cc: linux-hyperv, linux-kernel
In-Reply-To: <176057396465.74314.10055784909009416453.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
Simplify and unify memory region management to improve code clarity and
reliability. Consolidate pinning and invalidation logic, adopt consistent
naming, and remove redundant checks to reduce complexity.
Enhance documentation and update call sites for maintainability.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
---
drivers/hv/mshv_root_main.c | 80 +++++++++++++++++++------------------------
1 file changed, 36 insertions(+), 44 deletions(-)
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index fa42c40e1e02..e923947d3c54 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -1120,8 +1120,8 @@ mshv_region_map(struct mshv_mem_region *region)
}
static void
-mshv_region_evict_pages(struct mshv_mem_region *region,
- u64 page_offset, u64 page_count)
+mshv_region_invalidate_pages(struct mshv_mem_region *region,
+ u64 page_offset, u64 page_count)
{
if (region->flags.range_pinned)
unpin_user_pages(region->pages + page_offset, page_count);
@@ -1131,29 +1131,24 @@ mshv_region_evict_pages(struct mshv_mem_region *region,
}
static void
-mshv_region_evict(struct mshv_mem_region *region)
+mshv_region_invalidate(struct mshv_mem_region *region)
{
- mshv_region_evict_pages(region, 0, region->nr_pages);
+ mshv_region_invalidate_pages(region, 0, region->nr_pages);
}
static int
-mshv_region_populate_pages(struct mshv_mem_region *region,
- u64 page_offset, u64 page_count)
+mshv_region_pin(struct mshv_mem_region *region)
{
u64 done_count, nr_pages;
struct page **pages;
__u64 userspace_addr;
int ret;
- if (page_offset + page_count > region->nr_pages)
- return -EINVAL;
-
- for (done_count = 0; done_count < page_count; done_count += ret) {
- pages = region->pages + page_offset + done_count;
+ for (done_count = 0; done_count < region->nr_pages; done_count += ret) {
+ pages = region->pages + done_count;
userspace_addr = region->start_uaddr +
- (page_offset + done_count) *
- HV_HYP_PAGE_SIZE;
- nr_pages = min(page_count - done_count,
+ done_count * HV_HYP_PAGE_SIZE;
+ nr_pages = min(region->nr_pages - done_count,
MSHV_PIN_PAGES_BATCH_SIZE);
/*
@@ -1164,34 +1159,23 @@ mshv_region_populate_pages(struct mshv_mem_region *region,
* with the FOLL_LONGTERM flag does a large temporary
* allocation of contiguous memory.
*/
- if (region->flags.range_pinned)
- ret = pin_user_pages_fast(userspace_addr,
- nr_pages,
- FOLL_WRITE | FOLL_LONGTERM,
- pages);
- else
- ret = -EOPNOTSUPP;
-
+ ret = pin_user_pages_fast(userspace_addr, nr_pages,
+ FOLL_WRITE | FOLL_LONGTERM,
+ pages);
if (ret < 0)
goto release_pages;
}
- if (PageHuge(region->pages[page_offset]))
+ if (PageHuge(region->pages[0]))
region->flags.large_pages = true;
return 0;
release_pages:
- mshv_region_evict_pages(region, page_offset, done_count);
+ mshv_region_invalidate_pages(region, 0, done_count);
return ret;
}
-static int
-mshv_region_populate(struct mshv_mem_region *region)
-{
- return mshv_region_populate_pages(region, 0, region->nr_pages);
-}
-
static struct mshv_mem_region *
mshv_partition_region_by_gfn(struct mshv_partition *partition, u64 gfn)
{
@@ -1264,19 +1248,27 @@ static int mshv_partition_create_region(struct mshv_partition *partition,
return 0;
}
-/*
- * Map guest ram. if snp, make sure to release that from the host first
- * Side Effects: In case of failure, pages are unpinned when feasible.
+/**
+ * mshv_prepare_pinned_region - Pin and map memory regions
+ * @region: Pointer to the memory region structure
+ *
+ * This function processes memory regions that are explicitly marked as pinned.
+ * Pinned regions are preallocated, mapped upfront, and do not rely on fault-based
+ * population. The function ensures the region is properly populated, handles
+ * encryption requirements for SNP partitions if applicable, maps the region,
+ * and performs necessary sharing or eviction operations based on the mapping
+ * result.
+ *
+ * Return: 0 on success, negative error code on failure.
*/
-static int
-mshv_partition_mem_region_map(struct mshv_mem_region *region)
+static int mshv_prepare_pinned_region(struct mshv_mem_region *region)
{
struct mshv_partition *partition = region->partition;
int ret;
- ret = mshv_region_populate(region);
+ ret = mshv_region_pin(region);
if (ret) {
- pt_err(partition, "Failed to populate memory region: %d\n",
+ pt_err(partition, "Failed to pin memory region: %d\n",
ret);
goto err_out;
}
@@ -1294,7 +1286,7 @@ mshv_partition_mem_region_map(struct mshv_mem_region *region)
pt_err(partition,
"Failed to unshare memory region (guest_pfn: %llu): %d\n",
region->start_gfn, ret);
- goto evict_region;
+ goto invalidate_region;
}
}
@@ -1304,7 +1296,7 @@ mshv_partition_mem_region_map(struct mshv_mem_region *region)
shrc = mshv_partition_region_share(region);
if (!shrc)
- goto evict_region;
+ goto invalidate_region;
pt_err(partition,
"Failed to share memory region (guest_pfn: %llu): %d\n",
@@ -1318,8 +1310,8 @@ mshv_partition_mem_region_map(struct mshv_mem_region *region)
return 0;
-evict_region:
- mshv_region_evict(region);
+invalidate_region:
+ mshv_region_invalidate(region);
err_out:
return ret;
}
@@ -1368,7 +1360,7 @@ mshv_map_user_memory(struct mshv_partition *partition,
ret = hv_call_map_mmio_pages(partition->pt_id, mem.guest_pfn,
mmio_pfn, HVPFN_DOWN(mem.size));
else
- ret = mshv_partition_mem_region_map(region);
+ ret = mshv_prepare_pinned_region(region);
if (ret)
goto errout;
@@ -1413,7 +1405,7 @@ mshv_unmap_user_memory(struct mshv_partition *partition,
hv_call_unmap_gpa_pages(partition->pt_id, region->start_gfn,
region->nr_pages, unmap_flags);
- mshv_region_evict(region);
+ mshv_region_invalidate(region);
vfree(region);
return 0;
@@ -1827,7 +1819,7 @@ static void destroy_partition(struct mshv_partition *partition)
}
}
- mshv_region_evict(region);
+ mshv_region_invalidate(region);
vfree(region);
}
^ permalink raw reply related
* [PATCH v5 0/5] Introduce movable pages for Hyper-V guests
From: Stanislav Kinsburskii @ 2025-10-16 0:26 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui; +Cc: linux-hyperv, linux-kernel
From the start, the root-partition driver allocates, pins, and maps all
guest memory into the hypervisor at guest creation. This is simple: Linux
cannot move the pages, so the guest’s view in Linux and in Microsoft
Hypervisor never diverges.
However, this approach has major drawbacks:
- NUMA: affinity can’t be changed at runtime, so you can’t migrate guest memory closer to the CPUs running it → performance hit.
- Memory management: unused guest memory can’t be swapped out, compacted, or merged.
- Provisioning time: upfront allocation/pinning slows guest create/destroy.
- Overcommit: no memory overcommit on hosts with pinned-guest memory.
This series adds movable memory pages for Hyper-V child partitions. Guest
pages are no longer allocated upfront; they’re allocated and mapped into
the hypervisor on demand (i.e., when the guest touches a GFN that isn’t yet
backed by a host PFN).
When a page is moved, Linux no longer holds it and it is unmapped from the hypervisor.
As a result, Hyper-V guests behave like regular Linux processes, enabling standard Linux memory features to apply to guests.
Exceptions (still pinned):
1. Encrypted guests (explicit).
2. Guests with passthrough devices (implicitly pinned by the VFIO framework).
v5:
- Fix a bug in MMU notifier handling where an uninitialized 'ret' variable
could cause the warning about failed page invalidation to be skipped.
- Improve comment grammar regarding skipping the unmapping of non-mapped pages.
v4:
- Fix a bug in batch unmapping can skip mapped pages when selecting a new
batch due to wrong offset calculation.
- Fix an error message in case of failed memory region pinning.
v3:
- Region is invalidated even if the mm has no users.
- Page remapping logic is updated to support 2M-unaligned remappings for
regions that are PMD-aligned, which can occur during both faults and
invalidations.
v2:
- Split unmap batching into a separate patch.
- Fixed commit messages from v1 review.
- Renamed a few functions for clarity.
---
Stanislav Kinsburskii (5):
Drivers: hv: Refactor and rename memory region handling functions
Drivers: hv: Centralize guest memory region destruction
Drivers: hv: Batch GPA unmap operations to improve large region performance
Drivers: hv: Ensure large page GPA mapping is PMD-aligned
Drivers: hv: Add support for movable memory regions
drivers/hv/Kconfig | 1
drivers/hv/mshv_root.h | 10 +
drivers/hv/mshv_root_hv_call.c | 2
drivers/hv/mshv_root_main.c | 495 +++++++++++++++++++++++++++++++++-------
4 files changed, 424 insertions(+), 84 deletions(-)
^ permalink raw reply
* Re: [PATCH v3 21/21] x86/pvlocks: Move paravirt spinlock functions into own header
From: kernel test robot @ 2025-10-15 8:53 UTC (permalink / raw)
To: Juergen Gross, linux-kernel, x86, linux-hyperv, virtualization,
kvm
Cc: oe-kbuild-all, Juergen Gross, K. Y. Srinivasan, Haiyang Zhang,
Wei Liu, Dexuan Cui, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Ajay Kaher,
Alexey Makhalov, Broadcom internal kernel review list,
Paolo Bonzini, Vitaly Kuznetsov, Boris Ostrovsky, Josh Poimboeuf,
Peter Zijlstra, xen-devel
In-Reply-To: <20251006074606.1266-22-jgross@suse.com>
Hi Juergen,
kernel test robot noticed the following build errors:
[auto build test ERROR on tip/sched/core]
[also build test ERROR on kvm/queue kvm/next linus/master v6.18-rc1 next-20251014]
[cannot apply to tip/x86/core kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Juergen-Gross/x86-paravirt-Remove-not-needed-includes-of-paravirt-h/20251010-094850
base: tip/sched/core
patch link: https://lore.kernel.org/r/20251006074606.1266-22-jgross%40suse.com
patch subject: [PATCH v3 21/21] x86/pvlocks: Move paravirt spinlock functions into own header
config: x86_64-randconfig-001-20251015 (https://download.01.org/0day-ci/archive/20251015/202510151611.uYXVunzo-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251015/202510151611.uYXVunzo-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202510151611.uYXVunzo-lkp@intel.com/
All errors (new ones prefixed by >>):
ld: vmlinux.o: in function `kvm_guest_init':
arch/x86/kernel/kvm.c:828:(.init.text+0x440f4): undefined reference to `pv_ops_lock'
>> ld: arch/x86/kernel/kvm.c:828:(.init.text+0x4410e): undefined reference to `pv_ops_lock'
ld: arch/x86/kernel/kvm.c:828:(.init.text+0x4411a): undefined reference to `pv_ops_lock'
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH v2 2/2] hyperv: Enable clean shutdown for root partition with MSHV
From: kernel test robot @ 2025-10-15 8:01 UTC (permalink / raw)
To: Praveen K Paladugu, kys, haiyangz, wei.liu, decui, tglx, mingo,
linux-hyperv, linux-kernel, bp, dave.hansen, x86, hpa, arnd
Cc: oe-kbuild-all, anbelski, prapal, easwar.hariharan, nunodasneves,
skinsburskii
In-Reply-To: <20251014164150.6935-3-prapal@linux.microsoft.com>
Hi Praveen,
kernel test robot noticed the following build errors:
[auto build test ERROR on tip/x86/core]
[also build test ERROR on linus/master v6.18-rc1 next-20251014]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Praveen-K-Paladugu/hyperv-Add-definitions-for-MSHV-sleep-state-configuration/20251015-004650
base: tip/x86/core
patch link: https://lore.kernel.org/r/20251014164150.6935-3-prapal%40linux.microsoft.com
patch subject: [PATCH v2 2/2] hyperv: Enable clean shutdown for root partition with MSHV
config: i386-buildonly-randconfig-002-20251015 (https://download.01.org/0day-ci/archive/20251015/202510151539.w9IHg8lU-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251015/202510151539.w9IHg8lU-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202510151539.w9IHg8lU-lkp@intel.com/
All error/warnings (new ones prefixed by >>):
arch/x86/hyperv/hv_init.c: In function 'hyperv_init':
>> arch/x86/hyperv/hv_init.c:556:23: error: implicit declaration of function 'hv_sleep_notifiers_register'; did you mean 'preempt_notifier_register'? [-Wimplicit-function-declaration]
556 | (void)hv_sleep_notifiers_register();
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| preempt_notifier_register
--
>> drivers/hv/hv_common.c:944:5: warning: no previous prototype for 'hv_sleep_notifiers_register' [-Wmissing-prototypes]
944 | int hv_sleep_notifiers_register(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
vim +556 arch/x86/hyperv/hv_init.c
431
432 /*
433 * This function is to be invoked early in the boot sequence after the
434 * hypervisor has been detected.
435 *
436 * 1. Setup the hypercall page.
437 * 2. Register Hyper-V specific clocksource.
438 * 3. Setup Hyper-V specific APIC entry points.
439 */
440 void __init hyperv_init(void)
441 {
442 u64 guest_id;
443 union hv_x64_msr_hypercall_contents hypercall_msr;
444 int cpuhp;
445
446 if (x86_hyper_type != X86_HYPER_MS_HYPERV)
447 return;
448
449 if (hv_common_init())
450 return;
451
452 /*
453 * The VP assist page is useless to a TDX guest: the only use we
454 * would have for it is lazy EOI, which can not be used with TDX.
455 */
456 if (hv_isolation_type_tdx())
457 hv_vp_assist_page = NULL;
458 else
459 hv_vp_assist_page = kcalloc(nr_cpu_ids,
460 sizeof(*hv_vp_assist_page),
461 GFP_KERNEL);
462 if (!hv_vp_assist_page) {
463 ms_hyperv.hints &= ~HV_X64_ENLIGHTENED_VMCS_RECOMMENDED;
464
465 if (!hv_isolation_type_tdx())
466 goto common_free;
467 }
468
469 if (ms_hyperv.paravisor_present && hv_isolation_type_snp()) {
470 /* Negotiate GHCB Version. */
471 if (!hv_ghcb_negotiate_protocol())
472 hv_ghcb_terminate(SEV_TERM_SET_GEN,
473 GHCB_SEV_ES_PROT_UNSUPPORTED);
474
475 hv_ghcb_pg = alloc_percpu(union hv_ghcb *);
476 if (!hv_ghcb_pg)
477 goto free_vp_assist_page;
478 }
479
480 cpuhp = cpuhp_setup_state(CPUHP_AP_HYPERV_ONLINE, "x86/hyperv_init:online",
481 hv_cpu_init, hv_cpu_die);
482 if (cpuhp < 0)
483 goto free_ghcb_page;
484
485 /*
486 * Setup the hypercall page and enable hypercalls.
487 * 1. Register the guest ID
488 * 2. Enable the hypercall and register the hypercall page
489 *
490 * A TDX VM with no paravisor only uses TDX GHCI rather than hv_hypercall_pg:
491 * when the hypercall input is a page, such a VM must pass a decrypted
492 * page to Hyper-V, e.g. hv_post_message() uses the per-CPU page
493 * hyperv_pcpu_input_arg, which is decrypted if no paravisor is present.
494 *
495 * A TDX VM with the paravisor uses hv_hypercall_pg for most hypercalls,
496 * which are handled by the paravisor and the VM must use an encrypted
497 * input page: in such a VM, the hyperv_pcpu_input_arg is encrypted and
498 * used in the hypercalls, e.g. see hv_mark_gpa_visibility() and
499 * hv_arch_irq_unmask(). Such a VM uses TDX GHCI for two hypercalls:
500 * 1. HVCALL_SIGNAL_EVENT: see vmbus_set_event() and _hv_do_fast_hypercall8().
501 * 2. HVCALL_POST_MESSAGE: the input page must be a decrypted page, i.e.
502 * hv_post_message() in such a VM can't use the encrypted hyperv_pcpu_input_arg;
503 * instead, hv_post_message() uses the post_msg_page, which is decrypted
504 * in such a VM and is only used in such a VM.
505 */
506 guest_id = hv_generate_guest_id(LINUX_VERSION_CODE);
507 wrmsrq(HV_X64_MSR_GUEST_OS_ID, guest_id);
508
509 /* With the paravisor, the VM must also write the ID via GHCB/GHCI */
510 hv_ivm_msr_write(HV_X64_MSR_GUEST_OS_ID, guest_id);
511
512 /* A TDX VM with no paravisor only uses TDX GHCI rather than hv_hypercall_pg */
513 if (hv_isolation_type_tdx() && !ms_hyperv.paravisor_present)
514 goto skip_hypercall_pg_init;
515
516 hv_hypercall_pg = __vmalloc_node_range(PAGE_SIZE, 1, MODULES_VADDR,
517 MODULES_END, GFP_KERNEL, PAGE_KERNEL_ROX,
518 VM_FLUSH_RESET_PERMS, NUMA_NO_NODE,
519 __builtin_return_address(0));
520 if (hv_hypercall_pg == NULL)
521 goto clean_guest_os_id;
522
523 rdmsrq(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
524 hypercall_msr.enable = 1;
525
526 if (hv_root_partition()) {
527 struct page *pg;
528 void *src;
529
530 /*
531 * For the root partition, the hypervisor will set up its
532 * hypercall page. The hypervisor guarantees it will not show
533 * up in the root's address space. The root can't change the
534 * location of the hypercall page.
535 *
536 * Order is important here. We must enable the hypercall page
537 * so it is populated with code, then copy the code to an
538 * executable page.
539 */
540 wrmsrq(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
541
542 pg = vmalloc_to_page(hv_hypercall_pg);
543 src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
544 MEMREMAP_WB);
545 BUG_ON(!src);
546 memcpy_to_page(pg, 0, src, HV_HYP_PAGE_SIZE);
547 memunmap(src);
548
549 hv_remap_tsc_clocksource();
550 /*
551 * The notifier registration might fail at various hops.
552 * Corresponding error messages will land in dmesg. There is
553 * otherwise nothing that can be specifically done to handle
554 * failures here.
555 */
> 556 (void)hv_sleep_notifiers_register();
557 } else {
558 hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
559 wrmsrq(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
560 }
561
562 hv_set_hypercall_pg(hv_hypercall_pg);
563
564 skip_hypercall_pg_init:
565 /*
566 * hyperv_init() is called before LAPIC is initialized: see
567 * apic_intr_mode_init() -> x86_platform.apic_post_init() and
568 * apic_bsp_setup() -> setup_local_APIC(). The direct-mode STIMER
569 * depends on LAPIC, so hv_stimer_alloc() should be called from
570 * x86_init.timers.setup_percpu_clockev.
571 */
572 old_setup_percpu_clockev = x86_init.timers.setup_percpu_clockev;
573 x86_init.timers.setup_percpu_clockev = hv_stimer_setup_percpu_clockev;
574
575 hv_apic_init();
576
577 x86_init.pci.arch_init = hv_pci_init;
578
579 register_syscore_ops(&hv_syscore_ops);
580
581 if (ms_hyperv.priv_high & HV_ACCESS_PARTITION_ID)
582 hv_get_partition_id();
583
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH v2 2/2] hyperv: Enable clean shutdown for root partition with MSHV
From: kernel test robot @ 2025-10-15 5:23 UTC (permalink / raw)
To: Praveen K Paladugu, kys, haiyangz, wei.liu, decui, tglx, mingo,
linux-hyperv, linux-kernel, bp, dave.hansen, x86, hpa, arnd
Cc: llvm, oe-kbuild-all, anbelski, prapal, easwar.hariharan,
nunodasneves, skinsburskii
In-Reply-To: <20251014164150.6935-3-prapal@linux.microsoft.com>
Hi Praveen,
kernel test robot noticed the following build warnings:
[auto build test WARNING on tip/x86/core]
[also build test WARNING on linus/master v6.18-rc1 next-20251014]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Praveen-K-Paladugu/hyperv-Add-definitions-for-MSHV-sleep-state-configuration/20251015-004650
base: tip/x86/core
patch link: https://lore.kernel.org/r/20251014164150.6935-3-prapal%40linux.microsoft.com
patch subject: [PATCH v2 2/2] hyperv: Enable clean shutdown for root partition with MSHV
config: i386-randconfig-006-20251015 (https://download.01.org/0day-ci/archive/20251015/202510151359.vRXcys2P-lkp@intel.com/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251015/202510151359.vRXcys2P-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202510151359.vRXcys2P-lkp@intel.com/
All warnings (new ones prefixed by >>):
>> drivers/hv/hv_common.c:944:5: warning: no previous prototype for function 'hv_sleep_notifiers_register' [-Wmissing-prototypes]
944 | int hv_sleep_notifiers_register(void)
| ^
drivers/hv/hv_common.c:944:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
944 | int hv_sleep_notifiers_register(void)
| ^
| static
1 warning generated.
vim +/hv_sleep_notifiers_register +944 drivers/hv/hv_common.c
943
> 944 int hv_sleep_notifiers_register(void)
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* [PATCH -next] arch/x86: mshyperv: Remove duplicate asm/msr.h header
From: Jiapeng Chong @ 2025-10-15 1:50 UTC (permalink / raw)
To: kys
Cc: haiyangz, wei.liu, decui, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-hyperv, linux-kernel, Jiapeng Chong, Abaci Robot
./arch/x86/kernel/cpu/mshyperv.c: asm/msr.h is included more than once.
Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=26164
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
---
arch/x86/kernel/cpu/mshyperv.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 80a641a6ac48..6802d89ca790 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -31,7 +31,6 @@
#include <asm/msr.h>
#include <asm/nmi.h>
#include <clocksource/hyperv_timer.h>
-#include <asm/msr.h>
#include <asm/numa.h>
#include <asm/svm.h>
--
2.43.5
^ permalink raw reply related
* Re: [PATCH] mshv: Fix VpRootDispatchThreadBlocked value
From: Wei Liu @ 2025-10-14 22:58 UTC (permalink / raw)
To: Nuno Das Neves
Cc: linux-hyperv, linux-kernel, kys, haiyangz, wei.liu, decui,
skinsburskii, mhklinux
In-Reply-To: <1760466017-29523-1-git-send-email-nunodasneves@linux.microsoft.com>
On Tue, Oct 14, 2025 at 11:20:17AM -0700, Nuno Das Neves wrote:
> This value in the VP stats page is used to track if the VP can be
> dispatched for execution when there are no fast interrupts injected.
>
> The original value of 201 was used in a version of the hypervisor
> which did not ship. It was subsequently changed to 202 so that is the
> correct value.
>
> Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
Applied to hyperv-fixes. Thanks.
> ---
> drivers/hv/mshv_root_main.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
> index e3b2bd417c46..8a42d9961466 100644
> --- a/drivers/hv/mshv_root_main.c
> +++ b/drivers/hv/mshv_root_main.c
> @@ -41,7 +41,7 @@ MODULE_DESCRIPTION("Microsoft Hyper-V root partition VMM interface /dev/mshv");
> /* TODO move this to another file when debugfs code is added */
> enum hv_stats_vp_counters { /* HV_THREAD_COUNTER */
> #if defined(CONFIG_X86)
> - VpRootDispatchThreadBlocked = 201,
> + VpRootDispatchThreadBlocked = 202,
> #elif defined(CONFIG_ARM64)
> VpRootDispatchThreadBlocked = 94,
> #endif
> --
> 2.34.1
>
^ permalink raw reply
* [PATCH net-next,v2] net: mana: Support HW link state events
From: Haiyang Zhang @ 2025-10-14 21:26 UTC (permalink / raw)
To: linux-hyperv, netdev
Cc: haiyangz, paulros, decui, kys, wei.liu, edumazet, davem, kuba,
pabeni, longli, ssengar, ernis, dipayanroy, kotaranov, horms,
shradhagupta, leon, mlevitsk, yury.norov, shirazsaleem,
andrew+netdev, linux-rdma, linux-kernel
From: Haiyang Zhang <haiyangz@microsoft.com>
Handle the HW link state events received from HW channel, and
set the proper link state, also stop/wake queues accordingly.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
v2:
Updated link up/down to be symmetric, and other minor changes based
on comments from Andrew Lunn.
---
.../net/ethernet/microsoft/mana/gdma_main.c | 1 +
.../net/ethernet/microsoft/mana/hw_channel.c | 19 +++++
drivers/net/ethernet/microsoft/mana/mana_en.c | 76 +++++++++++++++++--
include/net/mana/gdma.h | 4 +-
include/net/mana/hw_channel.h | 2 +
include/net/mana/mana.h | 6 ++
6 files changed, 99 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index 43f034e180c4..effe0a2f207a 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -528,6 +528,7 @@ static void mana_gd_process_eqe(struct gdma_queue *eq)
case GDMA_EQE_HWC_INIT_DONE:
case GDMA_EQE_HWC_SOC_SERVICE:
case GDMA_EQE_RNIC_QP_FATAL:
+ case GDMA_EQE_HWC_SOC_RECONFIG_DATA:
if (!eq->eq.callback)
break;
diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
index ada6c78a2bef..eae68995492a 100644
--- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
+++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
@@ -118,6 +118,7 @@ static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_self,
struct gdma_dev *gd = hwc->gdma_dev;
union hwc_init_type_data type_data;
union hwc_init_eq_id_db eq_db;
+ struct mana_context *ac;
u32 type, val;
int ret;
@@ -196,6 +197,24 @@ static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_self,
hwc->hwc_timeout = val;
break;
+ case HWC_DATA_HW_LINK_CONNECT:
+ case HWC_DATA_HW_LINK_DISCONNECT:
+ ac = gd->gdma_context->mana.driver_data;
+ if (!ac)
+ break;
+
+ if (ac->mana_removing) {
+ dev_info(hwc->dev,
+ "Removing: skip link event %u\n",
+ type);
+ break;
+ }
+
+ ac->link_event = type;
+ schedule_work(&ac->link_change_work);
+
+ break;
+
default:
dev_warn(hwc->dev, "Received unknown reconfig type %u\n", type);
break;
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 0142fd98392c..51959d37b0a7 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -20,6 +20,7 @@
#include <net/mana/mana.h>
#include <net/mana/mana_auxiliary.h>
+#include <net/mana/hw_channel.h>
static DEFINE_IDA(mana_adev_ida);
@@ -84,8 +85,9 @@ static int mana_open(struct net_device *ndev)
/* Ensure port state updated before txq state */
smp_wmb();
- netif_carrier_on(ndev);
- netif_tx_wake_all_queues(ndev);
+ if (netif_carrier_ok(ndev))
+ netif_tx_wake_all_queues(ndev);
+
netdev_dbg(ndev, "%s successful\n", __func__);
return 0;
}
@@ -100,6 +102,59 @@ static int mana_close(struct net_device *ndev)
return mana_detach(ndev, true);
}
+static void mana_link_state_handle(struct work_struct *w)
+{
+ struct mana_port_context *apc;
+ struct mana_context *ac;
+ struct net_device *ndev;
+ bool link_up;
+ int i;
+
+ ac = container_of(w, struct mana_context, link_change_work);
+
+ if (ac->mana_removing)
+ return;
+
+ rtnl_lock();
+
+ if (ac->link_event == HWC_DATA_HW_LINK_CONNECT)
+ link_up = true;
+ else if (ac->link_event == HWC_DATA_HW_LINK_DISCONNECT)
+ link_up = false;
+ else
+ goto out;
+
+ /* Process all ports */
+ for (i = 0; i < ac->num_ports; i++) {
+ ndev = ac->ports[i];
+ if (!ndev)
+ continue;
+
+ apc = netdev_priv(ndev);
+
+ if (link_up) {
+ if (!netif_carrier_ok(ndev)) {
+ netif_carrier_on(ndev);
+
+ if (apc->port_is_up)
+ netif_tx_wake_all_queues(ndev);
+ }
+
+ __netdev_notify_peers(ndev);
+ } else {
+ if (netif_carrier_ok(ndev)) {
+ if (apc->port_is_up)
+ netif_tx_disable(ndev);
+
+ netif_carrier_off(ndev);
+ }
+ }
+ }
+
+out:
+ rtnl_unlock();
+}
+
static bool mana_can_tx(struct gdma_queue *wq)
{
return mana_gd_wq_avail_space(wq) >= MAX_TX_WQE_SIZE;
@@ -3059,9 +3114,6 @@ int mana_attach(struct net_device *ndev)
/* Ensure port state updated before txq state */
smp_wmb();
- if (apc->port_is_up)
- netif_carrier_on(ndev);
-
netif_device_attach(ndev);
return 0;
@@ -3153,8 +3205,8 @@ int mana_detach(struct net_device *ndev, bool from_close)
/* Ensure port state updated before txq state */
smp_wmb();
- netif_tx_disable(ndev);
- netif_carrier_off(ndev);
+ if (netif_carrier_ok(ndev))
+ netif_tx_disable(ndev);
if (apc->port_st_save) {
err = mana_dealloc_queues(ndev);
@@ -3212,7 +3264,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
netif_set_tso_max_size(ndev, GSO_MAX_SIZE);
- netif_carrier_off(ndev);
+ netif_carrier_on(ndev);
netdev_rss_key_fill(apc->hashkey, MANA_HASH_KEY_SIZE);
@@ -3431,6 +3483,8 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
if (!resuming) {
ac->num_ports = num_ports;
+
+ INIT_WORK(&ac->link_change_work, mana_link_state_handle);
} else {
if (ac->num_ports != num_ports) {
dev_err(dev, "The number of vPorts changed: %d->%d\n",
@@ -3481,6 +3535,8 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
if (err) {
mana_remove(gd, false);
} else {
+ ac->mana_removing = false;
+
dev_dbg(dev, "gd=%p, id=%u, num_ports=%d, type=%u, instance=%u\n",
gd, gd->dev_id.as_uint32, ac->num_ports,
gd->dev_id.type, gd->dev_id.instance);
@@ -3500,6 +3556,10 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
int err;
int i;
+ ac->mana_removing = true;
+
+ cancel_work_sync(&ac->link_change_work);
+
/* adev currently doesn't support suspending, always remove it */
if (gd->adev)
remove_adev(gd);
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 57df78cfbf82..637f42485dba 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -590,6 +590,7 @@ enum {
/* Driver can self reset on FPGA Reconfig EQE notification */
#define GDMA_DRV_CAP_FLAG_1_HANDLE_RECONFIG_EQE BIT(17)
+#define GDMA_DRV_CAP_FLAG_1_HW_VPORT_LINK_AWARE BIT(6)
#define GDMA_DRV_CAP_FLAGS1 \
(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \
@@ -599,7 +600,8 @@ enum {
GDMA_DRV_CAP_FLAG_1_DEV_LIST_HOLES_SUP | \
GDMA_DRV_CAP_FLAG_1_DYNAMIC_IRQ_ALLOC_SUPPORT | \
GDMA_DRV_CAP_FLAG_1_SELF_RESET_ON_EQE | \
- GDMA_DRV_CAP_FLAG_1_HANDLE_RECONFIG_EQE)
+ GDMA_DRV_CAP_FLAG_1_HANDLE_RECONFIG_EQE | \
+ GDMA_DRV_CAP_FLAG_1_HW_VPORT_LINK_AWARE)
#define GDMA_DRV_CAP_FLAGS2 0
diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h
index 83cf93338eb3..16feb39616c1 100644
--- a/include/net/mana/hw_channel.h
+++ b/include/net/mana/hw_channel.h
@@ -24,6 +24,8 @@
#define HWC_INIT_DATA_PF_DEST_CQ_ID 11
#define HWC_DATA_CFG_HWC_TIMEOUT 1
+#define HWC_DATA_HW_LINK_CONNECT 2
+#define HWC_DATA_HW_LINK_DISCONNECT 3
#define HW_CHANNEL_WAIT_RESOURCE_TIMEOUT_MS 30000
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 0921485565c0..d59e1f4656ce 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -477,6 +477,12 @@ struct mana_context {
struct dentry *mana_eqs_debugfs;
struct net_device *ports[MAX_PORTS_IN_MANA_DEV];
+
+ /* Link state change work */
+ struct work_struct link_change_work;
+ u32 link_event;
+
+ bool mana_removing;
};
struct mana_port_context {
--
2.34.1
^ permalink raw reply related
* [PATCH] mshv: Fix VpRootDispatchThreadBlocked value
From: Nuno Das Neves @ 2025-10-14 18:20 UTC (permalink / raw)
To: linux-hyperv, linux-kernel
Cc: kys, haiyangz, wei.liu, decui, skinsburskii, mhklinux,
Nuno Das Neves
This value in the VP stats page is used to track if the VP can be
dispatched for execution when there are no fast interrupts injected.
The original value of 201 was used in a version of the hypervisor
which did not ship. It was subsequently changed to 202 so that is the
correct value.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
---
drivers/hv/mshv_root_main.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
index e3b2bd417c46..8a42d9961466 100644
--- a/drivers/hv/mshv_root_main.c
+++ b/drivers/hv/mshv_root_main.c
@@ -41,7 +41,7 @@ MODULE_DESCRIPTION("Microsoft Hyper-V root partition VMM interface /dev/mshv");
/* TODO move this to another file when debugfs code is added */
enum hv_stats_vp_counters { /* HV_THREAD_COUNTER */
#if defined(CONFIG_X86)
- VpRootDispatchThreadBlocked = 201,
+ VpRootDispatchThreadBlocked = 202,
#elif defined(CONFIG_ARM64)
VpRootDispatchThreadBlocked = 94,
#endif
--
2.34.1
^ permalink raw reply related
* [PATCH v2 2/2] hyperv: Enable clean shutdown for root partition with MSHV
From: Praveen K Paladugu @ 2025-10-14 16:41 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, tglx, mingo, linux-hyperv,
linux-kernel, bp, dave.hansen, x86, hpa, arnd
Cc: anbelski, prapal, easwar.hariharan, nunodasneves, skinsburskii
In-Reply-To: <20251014164150.6935-1-prapal@linux.microsoft.com>
When a shutdown is initiated in the root partition without configuring
sleep states, the call to `hv_call_enter_sleep_state` fails. In such cases
the root falls back to using legacy ACPI mechanisms to poweroff. This call
is intercepted by MSHV and will result in a Machine Check Exception (MCE).
Root panics with a trace similar to:
[ 81.306348] reboot: Power down
[ 81.314709] mce: [Hardware Error]: CPU 0: Machine Check Exception: 4 Bank 0: b2000000c0060001
[ 81.314711] mce: [Hardware Error]: TSC 3b8cb60a66 PPIN 11d98332458e4ea9
[ 81.314713] mce: [Hardware Error]: PROCESSOR 0:606a6 TIME 1759339405 SOCKET 0 APIC 0 microcode ffffffff
[ 81.314715] mce: [Hardware Error]: Run the above through 'mcelog --ascii'
[ 81.314716] mce: [Hardware Error]: Machine check: Processor context corrupt
[ 81.314717] Kernel panic - not syncing: Fatal machine check
To prevent this, properly configure sleep states within MSHV, allowing
the root partition to shut down cleanly without triggering a panic.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Co-developed-by: Anatol Belski <anbelski@linux.microsoft.com>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
---
arch/x86/hyperv/hv_init.c | 7 ++
arch/x86/include/asm/mshyperv.h | 1 +
drivers/hv/hv_common.c | 119 ++++++++++++++++++++++++++++++++
3 files changed, 127 insertions(+)
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index afdbda2dd7b7..57bd96671ead 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -510,6 +510,13 @@ void __init hyperv_init(void)
memunmap(src);
hv_remap_tsc_clocksource();
+ /*
+ * The notifier registration might fail at various hops.
+ * Corresponding error messages will land in dmesg. There is
+ * otherwise nothing that can be specifically done to handle
+ * failures here.
+ */
+ (void)hv_sleep_notifiers_register();
} else {
hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
wrmsrq(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index abc4659f5809..fb8d691193df 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -236,6 +236,7 @@ int hyperv_fill_flush_guest_mapping_list(
void hv_apic_init(void);
void __init hv_init_spinlocks(void);
bool hv_vcpu_is_preempted(int vcpu);
+int hv_sleep_notifiers_register(void);
#else
static inline void hv_apic_init(void) {}
#endif
diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
index e109a620c83f..cfba9ded7bcb 100644
--- a/drivers/hv/hv_common.c
+++ b/drivers/hv/hv_common.c
@@ -837,3 +837,122 @@ const char *hv_result_to_string(u64 status)
return "Unknown";
}
EXPORT_SYMBOL_GPL(hv_result_to_string);
+
+#if IS_ENABLED(CONFIG_ACPI)
+/*
+ * Corresponding sleep states have to be initialized in order for a subsequent
+ * HVCALL_ENTER_SLEEP_STATE call to succeed. Currently only S5 state as per
+ * ACPI 6.4 chapter 7.4.2 is relevant, while S1, S2 and S3 can be supported.
+ *
+ * ACPI should be initialized and should support S5 sleep state when this method
+ * is called, so that it can extract correct PM values and pass them to hv.
+ */
+static int hv_initialize_sleep_states(void)
+{
+ u64 status;
+ unsigned long flags;
+ struct hv_input_set_system_property *in;
+ acpi_status acpi_status;
+ u8 sleep_type_a, sleep_type_b;
+
+ if (!acpi_sleep_state_supported(ACPI_STATE_S5)) {
+ pr_err("%s: S5 sleep state not supported.\n", __func__);
+ return -ENODEV;
+ }
+
+ acpi_status = acpi_get_sleep_type_data(ACPI_STATE_S5,
+ &sleep_type_a, &sleep_type_b);
+ if (ACPI_FAILURE(acpi_status))
+ return -ENODEV;
+
+ local_irq_save(flags);
+ in = *this_cpu_ptr(hyperv_pcpu_input_arg);
+ memset(in, 0, sizeof(*in));
+
+ in->property_id = HV_SYSTEM_PROPERTY_SLEEP_STATE;
+ in->set_sleep_state_info.sleep_state = HV_SLEEP_STATE_S5;
+ in->set_sleep_state_info.pm1a_slp_typ = sleep_type_a;
+ in->set_sleep_state_info.pm1b_slp_typ = sleep_type_b;
+
+ status = hv_do_hypercall(HVCALL_SET_SYSTEM_PROPERTY, in, NULL);
+ local_irq_restore(flags);
+
+ if (!hv_result_success(status)) {
+ hv_status_err(status, "\n");
+ return hv_result_to_errno(status);
+ }
+
+ return 0;
+}
+
+static int hv_call_enter_sleep_state(u32 sleep_state)
+{
+ u64 status;
+ int ret;
+ unsigned long flags;
+ struct hv_input_enter_sleep_state *in;
+
+ ret = hv_initialize_sleep_states();
+ if (ret)
+ return ret;
+
+ local_irq_save(flags);
+ in = *this_cpu_ptr(hyperv_pcpu_input_arg);
+ in->sleep_state = sleep_state;
+
+ status = hv_do_hypercall(HVCALL_ENTER_SLEEP_STATE, in, NULL);
+ local_irq_restore(flags);
+
+ if (!hv_result_success(status)) {
+ hv_status_err(status, "\n");
+ return hv_result_to_errno(status);
+ }
+
+ return 0;
+}
+
+static int hv_reboot_notifier_handler(struct notifier_block *this,
+ unsigned long code, void *another)
+{
+ int ret = 0;
+
+ if (code == SYS_HALT || code == SYS_POWER_OFF)
+ ret = hv_call_enter_sleep_state(HV_SLEEP_STATE_S5);
+
+ return ret ? NOTIFY_DONE : NOTIFY_OK;
+}
+
+static struct notifier_block hv_reboot_notifier = {
+ .notifier_call = hv_reboot_notifier_handler,
+};
+
+static int hv_acpi_sleep_handler(u8 sleep_state, u32 pm1a_cnt, u32 pm1b_cnt)
+{
+ int ret = 0;
+
+ if (sleep_state == ACPI_STATE_S5)
+ ret = hv_call_enter_sleep_state(HV_SLEEP_STATE_S5);
+
+ return ret == 0 ? 1 : -1;
+}
+
+static int hv_acpi_extended_sleep_handler(u8 sleep_state, u32 val_a, u32 val_b)
+{
+ return hv_acpi_sleep_handler(sleep_state, val_a, val_b);
+}
+
+int hv_sleep_notifiers_register(void)
+{
+ int ret;
+
+ acpi_os_set_prepare_sleep(&hv_acpi_sleep_handler);
+ acpi_os_set_prepare_extended_sleep(&hv_acpi_extended_sleep_handler);
+
+ ret = register_reboot_notifier(&hv_reboot_notifier);
+ if (ret)
+ pr_err("%s: cannot register reboot notifier %d\n",
+ __func__, ret);
+
+ return ret;
+}
+#endif
--
2.51.0
^ permalink raw reply related
* [PATCH v2 1/2] hyperv: Add definitions for MSHV sleep state configuration
From: Praveen K Paladugu @ 2025-10-14 16:41 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, tglx, mingo, linux-hyperv,
linux-kernel, bp, dave.hansen, x86, hpa, arnd
Cc: anbelski, prapal, easwar.hariharan, nunodasneves, skinsburskii
In-Reply-To: <20251014164150.6935-1-prapal@linux.microsoft.com>
Add the definitions required to configure sleep states in mshv hypervsior.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Co-developed-by: Anatol Belski <anbelski@linux.microsoft.com>
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
---
include/hyperv/hvgdk_mini.h | 4 +++-
include/hyperv/hvhdk_mini.h | 33 +++++++++++++++++++++++++++++++++
2 files changed, 36 insertions(+), 1 deletion(-)
diff --git a/include/hyperv/hvgdk_mini.h b/include/hyperv/hvgdk_mini.h
index 1be7f6a02304..25041417b54b 100644
--- a/include/hyperv/hvgdk_mini.h
+++ b/include/hyperv/hvgdk_mini.h
@@ -464,18 +464,20 @@ union hv_vp_assist_msr_contents { /* HV_REGISTER_VP_ASSIST_PAGE */
#define HVCALL_RESET_DEBUG_SESSION 0x006b
#define HVCALL_MAP_STATS_PAGE 0x006c
#define HVCALL_UNMAP_STATS_PAGE 0x006d
+#define HVCALL_SET_SYSTEM_PROPERTY 0x006f
#define HVCALL_ADD_LOGICAL_PROCESSOR 0x0076
#define HVCALL_GET_SYSTEM_PROPERTY 0x007b
#define HVCALL_MAP_DEVICE_INTERRUPT 0x007c
#define HVCALL_UNMAP_DEVICE_INTERRUPT 0x007d
#define HVCALL_RETARGET_INTERRUPT 0x007e
+#define HVCALL_ENTER_SLEEP_STATE 0x0084
#define HVCALL_NOTIFY_PORT_RING_EMPTY 0x008b
#define HVCALL_REGISTER_INTERCEPT_RESULT 0x0091
#define HVCALL_ASSERT_VIRTUAL_INTERRUPT 0x0094
#define HVCALL_CREATE_PORT 0x0095
#define HVCALL_CONNECT_PORT 0x0096
#define HVCALL_START_VP 0x0099
-#define HVCALL_GET_VP_INDEX_FROM_APIC_ID 0x009a
+#define HVCALL_GET_VP_INDEX_FROM_APIC_ID 0x009a
#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
#define HVCALL_SIGNAL_EVENT_DIRECT 0x00c0
diff --git a/include/hyperv/hvhdk_mini.h b/include/hyperv/hvhdk_mini.h
index 858f6a3925b3..8fa86c014c25 100644
--- a/include/hyperv/hvhdk_mini.h
+++ b/include/hyperv/hvhdk_mini.h
@@ -114,10 +114,24 @@ enum hv_snp_status {
enum hv_system_property {
/* Add more values when needed */
+ HV_SYSTEM_PROPERTY_SLEEP_STATE = 3,
HV_SYSTEM_PROPERTY_SCHEDULER_TYPE = 15,
HV_DYNAMIC_PROCESSOR_FEATURE_PROPERTY = 21,
};
+enum hv_sleep_state {
+ HV_SLEEP_STATE_S1 = 1,
+ HV_SLEEP_STATE_S2 = 2,
+ HV_SLEEP_STATE_S3 = 3,
+ HV_SLEEP_STATE_S4 = 4,
+ HV_SLEEP_STATE_S5 = 5,
+ /*
+ * After hypervisor has received this, any follow up sleep
+ * state registration requests will be rejected.
+ */
+ HV_SLEEP_STATE_LOCK = 6
+};
+
enum hv_dynamic_processor_feature_property {
/* Add more values when needed */
HV_X64_DYNAMIC_PROCESSOR_FEATURE_MAX_ENCRYPTED_PARTITIONS = 13,
@@ -145,6 +159,25 @@ struct hv_output_get_system_property {
};
} __packed;
+struct hv_sleep_state_info {
+ u32 sleep_state; /* enum hv_sleep_state */
+ u8 pm1a_slp_typ;
+ u8 pm1b_slp_typ;
+} __packed;
+
+struct hv_input_set_system_property {
+ u32 property_id; /* enum hv_system_property */
+ u32 reserved;
+ union {
+ /* More fields to be filled in when needed */
+ struct hv_sleep_state_info set_sleep_state_info;
+ };
+} __packed;
+
+struct hv_input_enter_sleep_state { /* HV_INPUT_ENTER_SLEEP_STATE */
+ u32 sleep_state; /* enum hv_sleep_state */
+} __packed;
+
struct hv_input_map_stats_page {
u32 type; /* enum hv_stats_object_type */
u32 padding;
--
2.51.0
^ permalink raw reply related
* [PATCH v2 0/2] Add support for clean shutdown with MSHV
From: Praveen K Paladugu @ 2025-10-14 16:41 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, tglx, mingo, linux-hyperv,
linux-kernel, bp, dave.hansen, x86, hpa, arnd
Cc: anbelski, prapal, easwar.hariharan, nunodasneves, skinsburskii
Add support for clean shutdown of the root partition when running on MSHV
hypervisor.
v2:
- Addressed review comments from v1.
- Moved all sleep state handling methods under CONFIG_ACPI stub
- - This fixes build issues on non-x86 architectures.
Moving the declartion of `hv_sleep_notifiers_register` to
arch/x86/include/asm/mshyperv.h, required the use of CONFIG_X86 stub
within hv_common.c. As this sleep configuration is dependent on ACPI,
I moved all the methods under CONFIG_ACPI to keep the code cleaner,
without introducing CONFIG_X86 stub.
Praveen K Paladugu (2):
hyperv: Add definitions for MSHV sleep state configuration
hyperv: Enable clean shutdown for root partition with MSHV
arch/x86/hyperv/hv_init.c | 7 ++
arch/x86/include/asm/mshyperv.h | 1 +
drivers/hv/hv_common.c | 119 ++++++++++++++++++++++++++++++++
include/hyperv/hvgdk_mini.h | 4 +-
include/hyperv/hvhdk_mini.h | 33 +++++++++
5 files changed, 163 insertions(+), 1 deletion(-)
--
2.51.0
^ permalink raw reply
* RE: [EXTERNAL] Re: [PATCH net-next] net: mana: Support HW link state events
From: Haiyang Zhang @ 2025-10-14 14:11 UTC (permalink / raw)
To: Andrew Lunn
Cc: Haiyang Zhang, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, Paul Rosswurm, Dexuan Cui, KY Srinivasan,
wei.liu@kernel.org, edumazet@google.com, davem@davemloft.net,
kuba@kernel.org, pabeni@redhat.com, Long Li,
ssengar@linux.microsoft.com, ernis@linux.microsoft.com,
dipayanroy@linux.microsoft.com, Konstantin Taranov,
horms@kernel.org, shradhagupta@linux.microsoft.com,
leon@kernel.org, mlevitsk@redhat.com, yury.norov@gmail.com,
Shiraz Saleem, andrew+netdev@lunn.ch, linux-rdma@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <4ea9acfd-be02-4299-b8c4-95bb69ad04cd@lunn.ch>
> -----Original Message-----
> From: Andrew Lunn <andrew@lunn.ch>
> Sent: Monday, October 13, 2025 7:22 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: Haiyang Zhang <haiyangz@linux.microsoft.com>; linux-
> hyperv@vger.kernel.org; netdev@vger.kernel.org; Paul Rosswurm
> <paulros@microsoft.com>; Dexuan Cui <decui@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>; wei.liu@kernel.org; edumazet@google.com;
> davem@davemloft.net; kuba@kernel.org; pabeni@redhat.com; Long Li
> <longli@microsoft.com>; ssengar@linux.microsoft.com;
> ernis@linux.microsoft.com; dipayanroy@linux.microsoft.com; Konstantin
> Taranov <kotaranov@microsoft.com>; horms@kernel.org;
> shradhagupta@linux.microsoft.com; leon@kernel.org; mlevitsk@redhat.com;
> yury.norov@gmail.com; Shiraz Saleem <shirazsaleem@microsoft.com>;
> andrew+netdev@lunn.ch; linux-rdma@vger.kernel.org; linux-
> kernel@vger.kernel.org
> Subject: Re: [EXTERNAL] Re: [PATCH net-next] net: mana: Support HW link
> state events
>
> > > > + if (link_up) {
> > > > + netif_carrier_on(ndev);
> > > > +
> > > > + if (apc->port_is_up)
> > > > + netif_tx_wake_all_queues(ndev);
> > > > +
> > > > + __netdev_notify_peers(ndev);
> > > > + } else {
> > > > + if (netif_carrier_ok(ndev)) {
> > > > + netif_tx_disable(ndev);
> > > > + netif_carrier_off(ndev);
> > > > + }
> > > > + }
> > >
> > > It is odd this is asymmetric. Up and down should really be opposites.
> > For the up event, we need to delay the wake up queues if the
> > mana_close() is called, or mana_open() isn't called yet.
> >
> > Also, we notify peers only when link up.
>
> But why is this not symmetric?
>
> On down, if port_is_up is not true, there is no need to disable tx and
> set the carrier off. There are also counters associated with
> netif_carrier_off() and netif_carrier_on(), and if you don't call them
> in symmetric pairs, the counters are going to look odd.
I see. Will update the patch.
Thanks,
- Haiyang
^ permalink raw reply
* Re: [PATCH 2/2] net: mana: Add standard counter rx_missed_errors
From: Erni Sri Satya Vennela @ 2025-10-14 9:36 UTC (permalink / raw)
To: Paolo Abeni
Cc: kys, haiyangz, wei.liu, decui, andrew+netdev, davem, edumazet,
kuba, longli, kotaranov, horms, shradhagupta, dipayanroy,
shirazsaleem, rosenp, linux-hyperv, netdev, linux-kernel,
linux-rdma
In-Reply-To: <20250925042405.GA5594@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
On Wed, Sep 24, 2025 at 09:24:05PM -0700, Erni Sri Satya Vennela wrote:
> On Tue, Sep 16, 2025 at 03:22:54PM +0200, Paolo Abeni wrote:
> > On 9/15/25 5:58 AM, Erni Sri Satya Vennela wrote:
> > > Report standard counter stats->rx_missed_errors
> > > using hc_rx_discards_no_wqe from the hardware.
> > >
> > > Add a dedicated workqueue to periodically run
> > > mana_query_gf_stats every 2 seconds to get the latest
> > > info in eth_stats and define a driver capability flag
> > > to notify hardware of the periodic queries.
> > >
> > > To avoid repeated failures and log flooding, the workqueue
> > > is not rescheduled if mana_query_gf_stats fails.
> >
> > Can the failure root cause be a "transient" one? If so, this looks like
> > a dangerous strategy; is such scenario, AFAICS, stats will be broken
> > until the device is removed and re-probed.
> >
> > /P
Hi Paolo,
I wanted to follow up on the clarification I shared regarding my
patch. Please let me know if there's anything further needed from my
side.
- Vennela
^ permalink raw reply
* Re: [PATCH v8 2/2] Drivers: hv: Introduce mshv_vtl driver
From: Naman Jain @ 2025-10-14 5:34 UTC (permalink / raw)
To: Sean Christopherson
Cc: K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H . Peter Anvin, linux-hyperv, linux-kernel, x86, Mukesh Rathor,
Stanislav Kinsburskii, Nuno Das Neves, Peter Zijlstra,
Paolo Bonzini, Michael Kelley, Christoph Hellwig, Saurabh Sengar,
ALOK TIWARI
In-Reply-To: <aO0I2va2HQ6mA-u0@google.com>
On 10/13/2025 7:42 PM, Sean Christopherson wrote:
> On Mon, Oct 13, 2025, Naman Jain wrote:
>> +static int mshv_vtl_ioctl_return_to_lower_vtl(void)
>> +{
>> + preempt_disable();
>> + for (;;) {
>> + u32 cancel;
>> + unsigned long irq_flags;
>> + struct hv_vp_assist_page *hvp;
>> + int ret;
>> +
>> + local_irq_save(irq_flags);
>> + cancel = READ_ONCE(mshv_vtl_this_run()->cancel);
>> + if (cancel)
>> + current_thread_info()->flags |= _TIF_SIGPENDING;
>
> There's no need to force SIGPENDING, this code can return directly if cancel is
> set[1]. And then you can wait to disable IRQs until after handling pending work,
> and thus avoid having to immediately re-enable IRQs[2].
>
> [1] https://lore.kernel.org/all/20250828000156.23389-3-seanjc@google.com
> [2] https://lore.kernel.org/all/20250828000156.23389-4-seanjc@google.com
Thank you for reviewing and sharing your inputs. I will incorporate
these changes in the next version.
Regards,
Naman
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox