Re: [External] Re: [PATCH] Revert "nvme-pci: clamp max_hw_sectors based on DMA optimized limitation"

[John Garry <john.garry@linux.dev>]

On 10/16/26 15:07, Adrian Huang12 wrote:
>> BTW, @Adrian, it would be good to run your test with the IOMMU in strict
>> mode as well, just to give an idea if there is a problem there as well.
> Sorry for the delayed update. The original test system used three years ago
> is no longer available, so I needed additional time to assemble two new
> test systems.
> 
> == Test Environment ==
>    1. 448-core AMD server (2-socket) with an NVMe device (mdts=10)
>    2. 960-core Intel server (8-socket) with an NVMe device (mdts=10)
>    3. The issue can be reproduced with a single NVMe device (mdts=10)
>    4. IOMMU lazy mode is enabled. We did not test strict mode three years
>      ago (and now). @John, let me know if you need me to run the
>      strict mode test.

Not necessarily, unless you want to. strict mode generally shows a more 
severe performance degradation in terms of throughput, but strict mode 
would not be relevant to IOVA FQ timeouts which you were seeing.

>    5. stress-ng and fio used for reproduction
>    6. A 100-run test was performed for each tested kernel
> 
> == Summary ==
>    1. Since the original patch was merged in v6.4-rc3, I used v6.3 as the
>      baseline to verify whether the issue could be reproduced on these
>      two systems. The issue was successfully reproduced on both platforms.
>    2. On v6.7 and later kernels, after reverting commit 3710e2b056cb, the
>      issue no longer occurs.
>    3. Detailed test information and logs can be found in [1].
> 
> == Detailed Results (Both Systems) ==
>    * v7.3.0-rc7 + revert 3710e2b056cb: Passed
>    * v6.12 + revert 3710e2b056cb: Passed
>    * v6.8 + revert 3710e2b056cb: Passed
>    * v6.7 + revert 3710e2b056cb: Passed
>    * v6.6 + revert 3710e2b056cb: Failed
>    * v6.5 + revert 3710e2b056cb: Failed
>    * Vanilla 6.3: Failed
> 
> Based on these results, I believe it is safe to apply the revert. Please feel
> free to add:

thanks for testing

> 
> Tested-by: Adrian Huang<ahuang12@lenovo.com>
> 
> Also, my apologies for introducing the performance regression. The results
> show that changes in the IOMMU subsystem introduced around v6.7
> significantly improved scalability, 

Nothing sticks out to me which could affect this. The iova rcache code 
changed in terms of removing the fixed depot size upper limit in v6.7, 
but this should not affect you (as your data block size exceeds that of 
the rcache limit). Having said that, you mention that stress-ng is being 
used also, and I don't know the profile of the storage data size used there.

> although I have not yet investigated
> which specific commit resolved the underlying issue.
> 
> [1] Detailed test info:https://hackmd.io/@lQBEPrYxRoOVV2MatUYusg/B14XRbZ1nGx