[PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[Dexuan Cui]

If vmbus_reserve_fb() in the kdump/kexec kernel fails to properly reserve
the framebuffer MMIO range (which is below 4GB) due to a Gen2 VM's
screen.lfb_base being zero [1], there is an MMIO conflict between the
drivers hyperv-drm and pci-hyperv: when the driver pci-hyperv's
hv_pci_allocate_bridge_windows() calls vmbus_allocate_mmio() to get a
32-bit MMIO range, it may get an MMIO range that overlaps with the
framebuffer MMIO range, and later hv_pci_enter_d0() fails with an
error message "PCI Pass-through VSP failed D0 Entry with status" since
the host thinks that PCI devices must not use MMIO space that the
host has assigned to the framebuffer.

This is especially an issue if pci-hyperv is built-in and hyperv-drm is
built as a module. Consequently, the kdump/kexec kernel fails to detect
PCI devices via pci-hyperv, and may fail to mount the root file system,
which may reside in a NVMe disk. The issue described here has existed
for SR-IOV VF NICs since day one of the pci-hyperv driver, and has been
worked around on x64 when possible. With the recent introduction of
ARM64 VMs that boot from NVMe, there is no workaround, so we need a
formal fix.

On Gen2 VMs, if the screen.lfb_base is 0 in the kdump/kexec kernel [1],
fall back to the low MMIO base, which should be equal to the framebuffer
MMIO base [2] (the statement is true according to my testing on x64
Windows Server 2016, and on x64 and ARM64 Windows Server 2025 and on
Azure. I checked with the Hyper-V team and they said the statement should
continue to be true for Gen2 VMs). In the first kernel, screen.lfb_base
is not 0; if the user specifies a very high resolution, it's not enough
to only reserve 8MB: in this case, reserve half of the space below 4GB,
but cap the reservation to 128MB, which is the required framebuffer size
of the highest resolution 7680*4320 supported by Hyper-V.

While at it, fix the comparison "end > VTPM_BASE_ADDRESS" by changing
the > to >=. Here the 'end' is an inclusive end (typically, it's
0xFFFF_FFFF for the low MMIO range).

Note: vmbus_reserve_fb() now also reserves an MMIO range at the beginning
of the low MMIO range on CVMs, which have no framebuffers (the
'screen.lfb_base' in vmbus_reserve_fb() is 0 for CVMs), just in case the
host might treat the beginning of the low MMIO range specially [4]. BTW,
the OpenHCL kernel is not affected by the change, because that kernel
boots with DeviceTree rather than ACPI (so vmbus_reserve_fb() won't run
there), and there is no framebuffer device for that kernel.

Note: normally Gen1 VMs don't have the MMIO conflict issue because the
framebuffer MMIO range (which is hardcoded to base=4GB-128MB and
size=64MB for Gen1 VMs by the host) is always reported via the legacy PCI
graphics device's BAR, so the kdump/kexec kernel can reserve the 64MB
MMIO range; however, if the VM is configured to use a very high resolution
and the required framebuffer size exceeds 64MB (AFAIK, in practice, this
isn't a typical configuration by users), the hyperv-drm driver may need to
allocate an MMIO range above 4GB and change the framebuffer MMIO location
to the allocated MMIO range -- in this case, there can still be issues [3]
which can't be easily fixed: any possible affected Gen1 users would have
to use a resolution whose framebuffer size is <= 64MB, or switch to Gen2
VMs.

[1] https://lore.kernel.org/all/SA1PR21MB692176C1BC53BFC9EAE5CF8EBF51A@SA1PR21MB6921.namprd21.prod.outlook.com/
[2] https://lore.kernel.org/all/SA1PR21MB69218F955B62DFF62E3E88D2BF222@SA1PR21MB6921.namprd21.prod.outlook.com/
[3] https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR21MB6921.namprd21.prod.outlook.com/
[4] https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6PR02MB4157.namprd02.prod.outlook.com/

Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
CC: stable@vger.kernel.org
Signed-off-by: Dexuan Cui <decui@microsoft.com>
---

Changes since v1 (https://lore.kernel.org/all/20260416183529.838321-1-decui@microsoft.com/):
  Fixed a typo in the subject: s/logc/logic/.

  In the commit message, better explained fb_mmio_base is equal to
  low_mmio_base for Gen2 VMs.

  Addressed Michael Kelley's comments:

    In the commit message:
         Changed the "kdump" to "kdump/kexec" since the described
         issue is applicable to both kdump and kexec.

         Provided more detail about the MMIO conflict.

         Described an scenario where Gen1 VMs can also be affected.

    Added a pr_warn() in vmbus_reserve_fb() in case the 'start' is 0.

    Dropped the CVM check in vmbus_reserve(), meaning vmbus_reserve_fb()
    also reserves MMIO for CVMs.

  Changed "low_mmio_base >= SZ_4G" to "upper_32_bits(low_mmio_base)"
  to avoid a compilation warning for the i386 build.

  Changed "0x%pa" to "%pa", because %pa already adds a "0x" prefix.
  

Hi Krister, Matthew, sorry -- I'm not adding your Tested-by's since
the code changed, though the change is small. If the v2 looks good
to Michael, please test the patch again. 

Hi Hardik, I'm not adding your Reviewed-by since the patch changed.
Please review the v2. 

 drivers/hv/vmbus_drv.c | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index f0d0803d1e16..d73ac5c8dd04 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -2327,8 +2327,8 @@ static acpi_status vmbus_walk_resources(struct acpi_resource *res, void *ctx)
 		return AE_NO_MEMORY;
 
 	/* If this range overlaps the virtual TPM, truncate it. */
-	if (end > VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS)
-		end = VTPM_BASE_ADDRESS;
+	if (end >= VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS)
+		end = VTPM_BASE_ADDRESS - 1;
 
 	new_res->name = "hyperv mmio";
 	new_res->flags = IORESOURCE_MEM;
@@ -2395,6 +2395,7 @@ static void vmbus_mmio_remove(void)
 static void __maybe_unused vmbus_reserve_fb(void)
 {
 	resource_size_t start = 0, size;
+	resource_size_t low_mmio_base;
 	struct pci_dev *pdev;
 
 	if (efi_enabled(EFI_BOOT)) {
@@ -2402,6 +2403,24 @@ static void __maybe_unused vmbus_reserve_fb(void)
 		if (IS_ENABLED(CONFIG_SYSFB)) {
 			start = sysfb_primary_display.screen.lfb_base;
 			size = max_t(__u32, sysfb_primary_display.screen.lfb_size, 0x800000);
+
+			low_mmio_base = hyperv_mmio->start;
+			if (!low_mmio_base || upper_32_bits(low_mmio_base) ||
+			    (start && start < low_mmio_base)) {
+				pr_warn("Unexpected low mmio base %pa\n", &low_mmio_base);
+			} else {
+				/*
+				 * If the kdump kernel's lfb_base is 0,
+				 * fall back to the low mmio base.
+				 */
+				if (!start)
+					start = low_mmio_base;
+				/*
+				 * Reserve half of the space below 4GB for high
+				 * resolutions, but cap the reservation to 128MB.
+				 */
+				size = min((SZ_4G - start) / 2, SZ_128M);
+			}
 		}
 	} else {
 		/* Gen1 VM: get FB base from PCI */
@@ -2422,8 +2441,10 @@ static void __maybe_unused vmbus_reserve_fb(void)
 		pci_dev_put(pdev);
 	}
 
-	if (!start)
+	if (!start) {
+		pr_warn("Unexpected framebuffer mmio base of zero\n");
 		return;
+	}
 
 	/*
 	 * Make a claim for the frame buffer in the resource tree under the
@@ -2433,6 +2454,8 @@ static void __maybe_unused vmbus_reserve_fb(void)
 	 */
 	for (; !fb_mmio && (size >= 0x100000); size >>= 1)
 		fb_mmio = __request_region(hyperv_mmio, start, size, fb_mmio_name, 0);
+
+	pr_info("hv_mmio=%pR,%pR fb=%pR\n", hyperv_mmio, hyperv_mmio->sibling, fb_mmio);
 }
 
 /**
-- 
2.34.1

RE: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[Dexuan Cui]

> From: Dexuan Cui <decui@microsoft.com>
> Sent: Monday, May 4, 2026 5:49 PM
>  ...
> If vmbus_reserve_fb() in the kdump/kexec kernel fails to properly reserve
> the framebuffer MMIO range (which is below 4GB) due to a Gen2 VM's
> screen.lfb_base being zero [1], there is an MMIO conflict between the
> drivers hyperv-drm and pci-hyperv: when the driver pci-hyperv's
> hv_pci_allocate_bridge_windows() calls vmbus_allocate_mmio() to get a

The " hv_pci_allocate_bridge_windows()" should be changed to
       "hv_allocate_config_window()"

> 32-bit MMIO range, it may get an MMIO range that overlaps with the
> framebuffer MMIO range, and later hv_pci_enter_d0() fails with an
> error message "PCI Pass-through VSP failed D0 Entry with status" since
> the host thinks that PCI devices must not use MMIO space that the
> host has assigned to the framebuffer.

RE: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[Michael Kelley]

From: Dexuan Cui <decui@microsoft.com> Sent: Monday, May 4, 2026 5:49 PM
> 
> If vmbus_reserve_fb() in the kdump/kexec kernel fails to properly reserve
> the framebuffer MMIO range (which is below 4GB) due to a Gen2 VM's
> screen.lfb_base being zero [1], there is an MMIO conflict between the
> drivers hyperv-drm and pci-hyperv: when the driver pci-hyperv's
> hv_pci_allocate_bridge_windows() calls vmbus_allocate_mmio() to get a
> 32-bit MMIO range, it may get an MMIO range that overlaps with the
> framebuffer MMIO range, and later hv_pci_enter_d0() fails with an
> error message "PCI Pass-through VSP failed D0 Entry with status" since
> the host thinks that PCI devices must not use MMIO space that the
> host has assigned to the framebuffer.
> 
> This is especially an issue if pci-hyperv is built-in and hyperv-drm is
> built as a module. Consequently, the kdump/kexec kernel fails to detect
> PCI devices via pci-hyperv, and may fail to mount the root file system,
> which may reside in a NVMe disk. The issue described here has existed
> for SR-IOV VF NICs since day one of the pci-hyperv driver, and has been
> worked around on x64 when possible. With the recent introduction of
> ARM64 VMs that boot from NVMe, there is no workaround, so we need a
> formal fix.
> 
> On Gen2 VMs, if the screen.lfb_base is 0 in the kdump/kexec kernel [1],
> fall back to the low MMIO base, which should be equal to the framebuffer
> MMIO base [2] (the statement is true according to my testing on x64
> Windows Server 2016, and on x64 and ARM64 Windows Server 2025 and on
> Azure. I checked with the Hyper-V team and they said the statement should
> continue to be true for Gen2 VMs). In the first kernel, screen.lfb_base
> is not 0; if the user specifies a very high resolution, it's not enough
> to only reserve 8MB: in this case, reserve half of the space below 4GB,
> but cap the reservation to 128MB, which is the required framebuffer size
> of the highest resolution 7680*4320 supported by Hyper-V.
> 
> While at it, fix the comparison "end > VTPM_BASE_ADDRESS" by changing
> the > to >=. Here the 'end' is an inclusive end (typically, it's
> 0xFFFF_FFFF for the low MMIO range).
> 
> Note: vmbus_reserve_fb() now also reserves an MMIO range at the beginning
> of the low MMIO range on CVMs, which have no framebuffers (the
> 'screen.lfb_base' in vmbus_reserve_fb() is 0 for CVMs), just in case the
> host might treat the beginning of the low MMIO range specially [4]. BTW,
> the OpenHCL kernel is not affected by the change, because that kernel
> boots with DeviceTree rather than ACPI (so vmbus_reserve_fb() won't run
> there), and there is no framebuffer device for that kernel.
> 
> Note: normally Gen1 VMs don't have the MMIO conflict issue because the
> framebuffer MMIO range (which is hardcoded to base=4GB-128MB and
> size=64MB for Gen1 VMs by the host) is always reported via the legacy PCI
> graphics device's BAR, so the kdump/kexec kernel can reserve the 64MB
> MMIO range; however, if the VM is configured to use a very high resolution
> and the required framebuffer size exceeds 64MB (AFAIK, in practice, this
> isn't a typical configuration by users), the hyperv-drm driver may need to
> allocate an MMIO range above 4GB and change the framebuffer MMIO location
> to the allocated MMIO range -- in this case, there can still be issues [3]
> which can't be easily fixed: any possible affected Gen1 users would have
> to use a resolution whose framebuffer size is <= 64MB, or switch to Gen2
> VMs.
> 
> [1]
> https://lore.kernel.org/all/SA1PR21MB692176C1BC53BFC9EAE5CF8EBF51A@SA1PR2
> 1MB6921.namprd21.prod.outlook.com/
> [2]
> https://lore.kernel.org/all/SA1PR21MB69218F955B62DFF62E3E88D2BF222@SA1PR2
> 1MB6921.namprd21.prod.outlook.com/
> [3]
> https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR
> 21MB6921.namprd21.prod.outlook.com/
> [4]
> https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6P
> R02MB4157.namprd02.prod.outlook.com/
> 
> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
> CC: stable@vger.kernel.org
> Signed-off-by: Dexuan Cui <decui@microsoft.com>
> ---
> 
> Changes since v1 (https://lore.kernel.org/all/20260416183529.838321-1-decui@microsoft.com/):
>   Fixed a typo in the subject: s/logc/logic/.
> 
>   In the commit message, better explained fb_mmio_base is equal to
>   low_mmio_base for Gen2 VMs.
> 
>   Addressed Michael Kelley's comments:
> 
>     In the commit message:
>          Changed the "kdump" to "kdump/kexec" since the described
>          issue is applicable to both kdump and kexec.
> 
>          Provided more detail about the MMIO conflict.
> 
>          Described an scenario where Gen1 VMs can also be affected.
> 
>     Added a pr_warn() in vmbus_reserve_fb() in case the 'start' is 0.
> 
>     Dropped the CVM check in vmbus_reserve(), meaning vmbus_reserve_fb()
>     also reserves MMIO for CVMs.
> 
>   Changed "low_mmio_base >= SZ_4G" to "upper_32_bits(low_mmio_base)"
>   to avoid a compilation warning for the i386 build.
> 
>   Changed "0x%pa" to "%pa", because %pa already adds a "0x" prefix.
> 
> 
> Hi Krister, Matthew, sorry -- I'm not adding your Tested-by's since
> the code changed, though the change is small. If the v2 looks good
> to Michael, please test the patch again.
> 
> Hi Hardik, I'm not adding your Reviewed-by since the patch changed.
> Please review the v2.
> 
>  drivers/hv/vmbus_drv.c | 29 ++++++++++++++++++++++++++---
>  1 file changed, 26 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index f0d0803d1e16..d73ac5c8dd04 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -2327,8 +2327,8 @@ static acpi_status vmbus_walk_resources(struct acpi_resource *res, void *ctx)
>  		return AE_NO_MEMORY;
> 
>  	/* If this range overlaps the virtual TPM, truncate it. */
> -	if (end > VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS)
> -		end = VTPM_BASE_ADDRESS;
> +	if (end >= VTPM_BASE_ADDRESS && start < VTPM_BASE_ADDRESS)
> +		end = VTPM_BASE_ADDRESS - 1;
> 
>  	new_res->name = "hyperv mmio";
>  	new_res->flags = IORESOURCE_MEM;
> @@ -2395,6 +2395,7 @@ static void vmbus_mmio_remove(void)
>  static void __maybe_unused vmbus_reserve_fb(void)
>  {
>  	resource_size_t start = 0, size;
> +	resource_size_t low_mmio_base;
>  	struct pci_dev *pdev;
> 
>  	if (efi_enabled(EFI_BOOT)) {
> @@ -2402,6 +2403,24 @@ static void __maybe_unused vmbus_reserve_fb(void)
>  		if (IS_ENABLED(CONFIG_SYSFB)) {
>  			start = sysfb_primary_display.screen.lfb_base;
>  			size = max_t(__u32, sysfb_primary_display.screen.lfb_size, 0x800000);
> +
> +			low_mmio_base = hyperv_mmio->start;
> +			if (!low_mmio_base || upper_32_bits(low_mmio_base) ||
> +			    (start && start < low_mmio_base)) {
> +				pr_warn("Unexpected low mmio base %pa\n", &low_mmio_base);
> +			} else {
> +				/*
> +				 * If the kdump kernel's lfb_base is 0,

Nit:  The case of lfb_base is 0 applies to kexec and kdump kernels, and also to
CVMs.

> +				 * fall back to the low mmio base.
> +				 */
> +				if (!start)
> +					start = low_mmio_base;
> +				/*
> +				 * Reserve half of the space below 4GB for high
> +				 * resolutions, but cap the reservation to 128MB.
> +				 */
> +				size = min((SZ_4G - start) / 2, SZ_128M);
> +			}
>  		}
>  	} else {
>  		/* Gen1 VM: get FB base from PCI */
> @@ -2422,8 +2441,10 @@ static void __maybe_unused vmbus_reserve_fb(void)
>  		pci_dev_put(pdev);
>  	}
> 
> -	if (!start)
> +	if (!start) {
> +		pr_warn("Unexpected framebuffer mmio base of zero\n");
>  		return;
> +	}
> 
>  	/*
>  	 * Make a claim for the frame buffer in the resource tree under the
> @@ -2433,6 +2454,8 @@ static void __maybe_unused vmbus_reserve_fb(void)
>  	 */
>  	for (; !fb_mmio && (size >= 0x100000); size >>= 1)
>  		fb_mmio = __request_region(hyperv_mmio, start, size, fb_mmio_name, 0);
> +
> +	pr_info("hv_mmio=%pR,%pR fb=%pR\n", hyperv_mmio, hyperv_mmio->sibling, fb_mmio);
>  }

Modulo my nit about the comment,

Reviewed-by: Michael Kelley <mhklinux@outlook.com>

RE: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[Dexuan Cui]

> From: Michael Kelley <mhklinux@outlook.com>
> Sent: Wednesday, May 6, 2026 8:14 AM
> > ...
> > +				/*
> > +				 * If the kdump kernel's lfb_base is 0,
> 
> Nit:  The case of lfb_base is 0 applies to kexec and kdump kernels, and also to
> CVMs.

Thanks for catching this! I'm going to post this v3 later today.

--- v2-0001-Drivers-hv-vmbus-Improve-the-logic-of-reserving-fb_m.patch  2026-05-04 17:48:23.486911073 -0700
+++ v3-0001-Drivers-hv-vmbus-Improve-the-logic-of-reserving-fb_m.patch  2026-05-06 18:03:42.922469286 -0700
@@ -1,15 +1,15 @@
 From 5d817788d65febdc0451e8a88277778794fe87b2 Mon Sep 17 00:00:00 2001
 From: Dexuan Cui <decui@microsoft.com>
 Date: Thu, 16 Apr 2026 04:30:21 +0000
-Subject: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on
+Subject: [PATCH v3] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on
  Gen2 VMs

 If vmbus_reserve_fb() in the kdump/kexec kernel fails to properly reserve
 the framebuffer MMIO range (which is below 4GB) due to a Gen2 VM's
 screen.lfb_base being zero [1], there is an MMIO conflict between the
 drivers hyperv-drm and pci-hyperv: when the driver pci-hyperv's
-hv_pci_allocate_bridge_windows() calls vmbus_allocate_mmio() to get a
-32-bit MMIO range, it may get an MMIO range that overlaps with the
+hv_allocate_config_window() calls vmbus_allocate_mmio() to get an
+MMIO range, typically it gets a 32-bit MMIO range that overlaps with the
 framebuffer MMIO range, and later hv_pci_enter_d0() fails with an
 error message "PCI Pass-through VSP failed D0 Entry with status" since
 the host thinks that PCI devices must not use MMIO space that the
@@ -31,7 +31,7 @@
 Azure. I checked with the Hyper-V team and they said the statement should
 continue to be true for Gen2 VMs). In the first kernel, screen.lfb_base
 is not 0; if the user specifies a very high resolution, it's not enough
-to only reserve 8MB: in this case, reserve half of the space below 4GB,
+to only reserve 8MB: let's always reserve half of the space below 4GB,
 but cap the reservation to 128MB, which is the required framebuffer size
 of the highest resolution 7680*4320 supported by Hyper-V.

@@ -42,7 +42,7 @@
 Note: vmbus_reserve_fb() now also reserves an MMIO range at the beginning
 of the low MMIO range on CVMs, which have no framebuffers (the
 'screen.lfb_base' in vmbus_reserve_fb() is 0 for CVMs), just in case the
-host might treat the beginning of the low MMIO range specially [4]. BTW,
+host might treat the beginning of the low MMIO range specially [3]. BTW,
 the OpenHCL kernel is not affected by the change, because that kernel
 boots with DeviceTree rather than ACPI (so vmbus_reserve_fb() won't run
 there), and there is no framebuffer device for that kernel.
@@ -55,18 +55,20 @@
 and the required framebuffer size exceeds 64MB (AFAIK, in practice, this
 isn't a typical configuration by users), the hyperv-drm driver may need to
 allocate an MMIO range above 4GB and change the framebuffer MMIO location
-to the allocated MMIO range -- in this case, there can still be issues [3]
+to the allocated MMIO range -- in this case, there can still be issues [4]
 which can't be easily fixed: any possible affected Gen1 users would have
 to use a resolution whose framebuffer size is <= 64MB, or switch to Gen2
 VMs.

 [1] https://lore.kernel.org/all/SA1PR21MB692176C1BC53BFC9EAE5CF8EBF51A@SA1PR21MB6921.namprd21.prod.outlook.com/
 [2] https://lore.kernel.org/all/SA1PR21MB69218F955B62DFF62E3E88D2BF222@SA1PR21MB6921.namprd21.prod.outlook.com/
-[3] https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR21MB6921.namprd21.prod.outlook.com/
-[4] https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6PR02MB4157.namprd02.prod.outlook.com/
+[3] https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6PR02MB4157.namprd02.prod.outlook.com/
+[4] https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR21MB6921.namprd21.prod.outlook.com/

 Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
 CC: stable@vger.kernel.org
+Reviewed-by: Michael Kelley <mhklinux@outlook.com>
+Tested-by: Krister Johansen <kjlx@templeofstupid.com>
 Signed-off-by: Dexuan Cui <decui@microsoft.com>
 ---

@@ -104,6 +106,18 @@
 Hi Hardik, I'm not adding your Reviewed-by since the patch changed.
 Please review the v2.

+
+Changes since v2:
+    Fixed the commit message:
+        hv_pci_allocate_bridge_windows() -> hv_allocate_config_window()
+
+    Changed the "kdump" in the comment to "kdump/kexec or CVM" [Michael Kelley]
+
+    Fixed the order of the "[3]" and "[4]" in the commit message.
+
+    Added Krister's Tested-by.
+    Added Michael's Reviewed-by.
+
  drivers/hv/vmbus_drv.c | 29 ++++++++++++++++++++++++++---
  1 file changed, 26 insertions(+), 3 deletions(-)

@@ -141,8 +155,8 @@
 +                              pr_warn("Unexpected low mmio base %pa\n", &low_mmio_base);
 +                      } else {
 +                              /*
-+                               * If the kdump kernel's lfb_base is 0,
-+                               * fall back to the low mmio base.
++                               * If the kdump/kexec or CVM kernel's lfb_base
++                               * is 0, fall back to the low mmio base.
 +                               */
 +                              if (!start)
 +                                      start = low_mmio_base;


> Modulo my nit about the comment,
> 
> Reviewed-by: Michael Kelley <mhklinux@outlook.com>

Thanks a lot!

Re: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[Matthew Ruffell]

Hi Dexuan,

Thanks for making the amendments, and thank you Michael for all your reviews.

Since you posted the diff to the V3, I went and tested the V3 patch.

I have tested this patch on Azure with:
- Standard_D4ads_v5
- Standard_D4ads_v6

with the following images:
"Ubuntu Server 22.04 LTS - x64 Gen2"
"Ubuntu Server 24.04 LTS - x64 Gen2"

with the following kernels:
- 7.1-rc2 at 5862221fddede6bb15566ab3c1f23a3c353da5e1
- 7.1-rc2 at 5862221fddede6bb15566ab3c1f23a3c353da5e1 + the V3 patch

Without this patch, I could reproduce the issue on 22.04 + v6 based instance
types.

I can confirm that with this patch, v6 instance types can correctly kdump and
create a vmcore correctly and restart correctly without running into
MMIO issues.

I can confirm that with this patch, v5 instance types continue to operate the
same as they did previously.

Tested-by: Matthew Ruffell <matthew.ruffell@canonical.com>

Thanks,
Matthew

On Thu, 7 May 2026 at 13:11, Dexuan Cui <DECUI@microsoft.com> wrote:
>
> > From: Michael Kelley <mhklinux@outlook.com>
> > Sent: Wednesday, May 6, 2026 8:14 AM
> > > ...
> > > +                           /*
> > > +                            * If the kdump kernel's lfb_base is 0,
> >
> > Nit:  The case of lfb_base is 0 applies to kexec and kdump kernels, and also to
> > CVMs.
>
> Thanks for catching this! I'm going to post this v3 later today.
>
> --- v2-0001-Drivers-hv-vmbus-Improve-the-logic-of-reserving-fb_m.patch  2026-05-04 17:48:23.486911073 -0700
> +++ v3-0001-Drivers-hv-vmbus-Improve-the-logic-of-reserving-fb_m.patch  2026-05-06 18:03:42.922469286 -0700
> @@ -1,15 +1,15 @@
>  From 5d817788d65febdc0451e8a88277778794fe87b2 Mon Sep 17 00:00:00 2001
>  From: Dexuan Cui <decui@microsoft.com>
>  Date: Thu, 16 Apr 2026 04:30:21 +0000
> -Subject: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on
> +Subject: [PATCH v3] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on
>   Gen2 VMs
>
>  If vmbus_reserve_fb() in the kdump/kexec kernel fails to properly reserve
>  the framebuffer MMIO range (which is below 4GB) due to a Gen2 VM's
>  screen.lfb_base being zero [1], there is an MMIO conflict between the
>  drivers hyperv-drm and pci-hyperv: when the driver pci-hyperv's
> -hv_pci_allocate_bridge_windows() calls vmbus_allocate_mmio() to get a
> -32-bit MMIO range, it may get an MMIO range that overlaps with the
> +hv_allocate_config_window() calls vmbus_allocate_mmio() to get an
> +MMIO range, typically it gets a 32-bit MMIO range that overlaps with the
>  framebuffer MMIO range, and later hv_pci_enter_d0() fails with an
>  error message "PCI Pass-through VSP failed D0 Entry with status" since
>  the host thinks that PCI devices must not use MMIO space that the
> @@ -31,7 +31,7 @@
>  Azure. I checked with the Hyper-V team and they said the statement should
>  continue to be true for Gen2 VMs). In the first kernel, screen.lfb_base
>  is not 0; if the user specifies a very high resolution, it's not enough
> -to only reserve 8MB: in this case, reserve half of the space below 4GB,
> +to only reserve 8MB: let's always reserve half of the space below 4GB,
>  but cap the reservation to 128MB, which is the required framebuffer size
>  of the highest resolution 7680*4320 supported by Hyper-V.
>
> @@ -42,7 +42,7 @@
>  Note: vmbus_reserve_fb() now also reserves an MMIO range at the beginning
>  of the low MMIO range on CVMs, which have no framebuffers (the
>  'screen.lfb_base' in vmbus_reserve_fb() is 0 for CVMs), just in case the
> -host might treat the beginning of the low MMIO range specially [4]. BTW,
> +host might treat the beginning of the low MMIO range specially [3]. BTW,
>  the OpenHCL kernel is not affected by the change, because that kernel
>  boots with DeviceTree rather than ACPI (so vmbus_reserve_fb() won't run
>  there), and there is no framebuffer device for that kernel.
> @@ -55,18 +55,20 @@
>  and the required framebuffer size exceeds 64MB (AFAIK, in practice, this
>  isn't a typical configuration by users), the hyperv-drm driver may need to
>  allocate an MMIO range above 4GB and change the framebuffer MMIO location
> -to the allocated MMIO range -- in this case, there can still be issues [3]
> +to the allocated MMIO range -- in this case, there can still be issues [4]
>  which can't be easily fixed: any possible affected Gen1 users would have
>  to use a resolution whose framebuffer size is <= 64MB, or switch to Gen2
>  VMs.
>
>  [1] https://lore.kernel.org/all/SA1PR21MB692176C1BC53BFC9EAE5CF8EBF51A@SA1PR21MB6921.namprd21.prod.outlook.com/
>  [2] https://lore.kernel.org/all/SA1PR21MB69218F955B62DFF62E3E88D2BF222@SA1PR21MB6921.namprd21.prod.outlook.com/
> -[3] https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR21MB6921.namprd21.prod.outlook.com/
> -[4] https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6PR02MB4157.namprd02.prod.outlook.com/
> +[3] https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6PR02MB4157.namprd02.prod.outlook.com/
> +[4] https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR21MB6921.namprd21.prod.outlook.com/
>
>  Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
>  CC: stable@vger.kernel.org
> +Reviewed-by: Michael Kelley <mhklinux@outlook.com>
> +Tested-by: Krister Johansen <kjlx@templeofstupid.com>
>  Signed-off-by: Dexuan Cui <decui@microsoft.com>
>  ---
>
> @@ -104,6 +106,18 @@
>  Hi Hardik, I'm not adding your Reviewed-by since the patch changed.
>  Please review the v2.
>
> +
> +Changes since v2:
> +    Fixed the commit message:
> +        hv_pci_allocate_bridge_windows() -> hv_allocate_config_window()
> +
> +    Changed the "kdump" in the comment to "kdump/kexec or CVM" [Michael Kelley]
> +
> +    Fixed the order of the "[3]" and "[4]" in the commit message.
> +
> +    Added Krister's Tested-by.
> +    Added Michael's Reviewed-by.
> +
>   drivers/hv/vmbus_drv.c | 29 ++++++++++++++++++++++++++---
>   1 file changed, 26 insertions(+), 3 deletions(-)
>
> @@ -141,8 +155,8 @@
>  +                              pr_warn("Unexpected low mmio base %pa\n", &low_mmio_base);
>  +                      } else {
>  +                              /*
> -+                               * If the kdump kernel's lfb_base is 0,
> -+                               * fall back to the low mmio base.
> ++                               * If the kdump/kexec or CVM kernel's lfb_base
> ++                               * is 0, fall back to the low mmio base.
>  +                               */
>  +                              if (!start)
>  +                                      start = low_mmio_base;
>
>
> > Modulo my nit about the comment,
> >
> > Reviewed-by: Michael Kelley <mhklinux@outlook.com>
>
> Thanks a lot!

Re: [PATCH v2] Drivers: hv: vmbus: Improve the logic of reserving fb_mmio on Gen2 VMs

[Krister Johansen]

On Mon, May 04, 2026 at 05:48:46PM -0700, Dexuan Cui wrote:
> If vmbus_reserve_fb() in the kdump/kexec kernel fails to properly reserve
> the framebuffer MMIO range (which is below 4GB) due to a Gen2 VM's
> screen.lfb_base being zero [1], there is an MMIO conflict between the
> drivers hyperv-drm and pci-hyperv: when the driver pci-hyperv's
> hv_pci_allocate_bridge_windows() calls vmbus_allocate_mmio() to get a
> 32-bit MMIO range, it may get an MMIO range that overlaps with the
> framebuffer MMIO range, and later hv_pci_enter_d0() fails with an
> error message "PCI Pass-through VSP failed D0 Entry with status" since
> the host thinks that PCI devices must not use MMIO space that the
> host has assigned to the framebuffer.
> 
> This is especially an issue if pci-hyperv is built-in and hyperv-drm is
> built as a module. Consequently, the kdump/kexec kernel fails to detect
> PCI devices via pci-hyperv, and may fail to mount the root file system,
> which may reside in a NVMe disk. The issue described here has existed
> for SR-IOV VF NICs since day one of the pci-hyperv driver, and has been
> worked around on x64 when possible. With the recent introduction of
> ARM64 VMs that boot from NVMe, there is no workaround, so we need a
> formal fix.
> 
> On Gen2 VMs, if the screen.lfb_base is 0 in the kdump/kexec kernel [1],
> fall back to the low MMIO base, which should be equal to the framebuffer
> MMIO base [2] (the statement is true according to my testing on x64
> Windows Server 2016, and on x64 and ARM64 Windows Server 2025 and on
> Azure. I checked with the Hyper-V team and they said the statement should
> continue to be true for Gen2 VMs). In the first kernel, screen.lfb_base
> is not 0; if the user specifies a very high resolution, it's not enough
> to only reserve 8MB: in this case, reserve half of the space below 4GB,
> but cap the reservation to 128MB, which is the required framebuffer size
> of the highest resolution 7680*4320 supported by Hyper-V.
> 
> While at it, fix the comparison "end > VTPM_BASE_ADDRESS" by changing
> the > to >=. Here the 'end' is an inclusive end (typically, it's
> 0xFFFF_FFFF for the low MMIO range).
> 
> Note: vmbus_reserve_fb() now also reserves an MMIO range at the beginning
> of the low MMIO range on CVMs, which have no framebuffers (the
> 'screen.lfb_base' in vmbus_reserve_fb() is 0 for CVMs), just in case the
> host might treat the beginning of the low MMIO range specially [4]. BTW,
> the OpenHCL kernel is not affected by the change, because that kernel
> boots with DeviceTree rather than ACPI (so vmbus_reserve_fb() won't run
> there), and there is no framebuffer device for that kernel.
> 
> Note: normally Gen1 VMs don't have the MMIO conflict issue because the
> framebuffer MMIO range (which is hardcoded to base=4GB-128MB and
> size=64MB for Gen1 VMs by the host) is always reported via the legacy PCI
> graphics device's BAR, so the kdump/kexec kernel can reserve the 64MB
> MMIO range; however, if the VM is configured to use a very high resolution
> and the required framebuffer size exceeds 64MB (AFAIK, in practice, this
> isn't a typical configuration by users), the hyperv-drm driver may need to
> allocate an MMIO range above 4GB and change the framebuffer MMIO location
> to the allocated MMIO range -- in this case, there can still be issues [3]
> which can't be easily fixed: any possible affected Gen1 users would have
> to use a resolution whose framebuffer size is <= 64MB, or switch to Gen2
> VMs.
> 
> [1] https://lore.kernel.org/all/SA1PR21MB692176C1BC53BFC9EAE5CF8EBF51A@SA1PR21MB6921.namprd21.prod.outlook.com/
> [2] https://lore.kernel.org/all/SA1PR21MB69218F955B62DFF62E3E88D2BF222@SA1PR21MB6921.namprd21.prod.outlook.com/
> [3] https://lore.kernel.org/all/SA1PR21MB69213486F821CA5A2C793C81BF342@SA1PR21MB6921.namprd21.prod.outlook.com/
> [4] https://lore.kernel.org/all/SN6PR02MB415726B17D5A6027CD1717E8D4342@SN6PR02MB4157.namprd02.prod.outlook.com/
> 
> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs")
> CC: stable@vger.kernel.org
> Signed-off-by: Dexuan Cui <decui@microsoft.com>
> ---
 
Thanks for the updated patch.  I re-tested this on a D2pdsv6 and was
able to confirm that without the patch the NIC drivers in the dump
environment didn't attach because of a PCI conflict. With the patch the
drivers attached and it was possible to successfully collect a kdump.

Tested-by: Krister Johansen <kjlx@templeofstupid.com>

-K