Default NVMe MDTS value causes Widnows Server 2025 hangs and BSODs (during install at least)

Default NVMe MDTS value causes Widnows Server 2025 hangs and BSODs (during install at least)

[Roger Pau Monné]

Hello,

As part of XenServer Windows Server 2025 testing using Xen plus QEMU
we discovered an issue during install that would lead to either the
Windows installer getting stuck without making progress (albeit the
screen was still showing the spinning circle correctly) or a BSOD that
doesn't seem to have a unique number, the most common one was 0x50
(PAGE_FAULT_IN_NONPAGED_AREA).

After a fair amount of debugging and following incorrect leads we have
narrowed down what triggers the issue to QEMU emulated NVMe reporting
a MDTS value of 7 by default (so max request size of 512KiB).
Switching to higher MDTS values seemed to solve the issue.

The commit that made that change:

e137d20e7dff hw/block/nvme: add check for mdts

Didn't contain much justification for the change from unlimited to
512KiB max request size.

Windows is like a black box to me, but I believe there's some error in
the Windows logic that splits requests, and hence when MDTS is set to
a sufficiently low value, and Windows has to split NVMe requests, it
causes Windows to hit an internal bug.  This will be raised with
Microsoft to get the issue debugged and possibly fixed on their side.

From limited experimentation setting mdts to 10 (so 4MiB max request
size) or 9 (2MiB) workarounds the issue.

Would it be acceptable for QEMU NVMe to consider increasing the
default MDTS value to something higher than 7 to workaround the issue?
I've tested both 9 and 10 and they prevent the issue, I would avoid 8
as it's too close to the current one that causes issues.  I don't have
many references of other emulated NVMe implementations, I just know
about Bhyve emulated NVMe, which sets MDTS to 9.

If bumping MDTS to a higher value is acceptable please let me know and
I will prepare a patch.

Regards, Roger.

Re: Default NVMe MDTS value causes Widnows Server 2025 hangs and BSODs (during install at least)

[Keith Busch]

On Fri, Nov 28, 2025 at 11:28:47AM +0100, Roger Pau Monné wrote:
> Hello,
> 
> As part of XenServer Windows Server 2025 testing using Xen plus QEMU
> we discovered an issue during install that would lead to either the
> Windows installer getting stuck without making progress (albeit the
> screen was still showing the spinning circle correctly) or a BSOD that
> doesn't seem to have a unique number, the most common one was 0x50
> (PAGE_FAULT_IN_NONPAGED_AREA).
> 
> After a fair amount of debugging and following incorrect leads we have
> narrowed down what triggers the issue to QEMU emulated NVMe reporting
> a MDTS value of 7 by default (so max request size of 512KiB).
> Switching to higher MDTS values seemed to solve the issue.
> 
> The commit that made that change:
> 
> e137d20e7dff hw/block/nvme: add check for mdts
> 
> Didn't contain much justification for the change from unlimited to
> 512KiB max request size.
> 
> Windows is like a black box to me, but I believe there's some error in
> the Windows logic that splits requests, and hence when MDTS is set to
> a sufficiently low value, and Windows has to split NVMe requests, it
> causes Windows to hit an internal bug.  This will be raised with
> Microsoft to get the issue debugged and possibly fixed on their side.
> 
> From limited experimentation setting mdts to 10 (so 4MiB max request
> size) or 9 (2MiB) workarounds the issue.
> 
> Would it be acceptable for QEMU NVMe to consider increasing the
> default MDTS value to something higher than 7 to workaround the issue?
> I've tested both 9 and 10 and they prevent the issue, I would avoid 8
> as it's too close to the current one that causes issues.  I don't have
> many references of other emulated NVMe implementations, I just know
> about Bhyve emulated NVMe, which sets MDTS to 9.
> 
> If bumping MDTS to a higher value is acceptable please let me know and
> I will prepare a patch.

The mdts value is a prameter you can set at the command line. The 512k
default is really pretty reasonable, but you can set it up to 2MB
(mdts=9) if you want before hitting a different software limit that
won't let you go any higher. So rather than patch the device, I'd
recommend you just modify your startup command line parameters instead.

Not sure if it's still the case, but there was a time when Windows
Storport's maximum IO size had been limited to 128k, and that's exactly
what many nvme devices reported for MDTS, so Windows is certainly
supposed to work correctly with such values.