Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mario Limonciello <mario.limonciello@amd.com>]

 > I CANNOT realistically: dedicate the machine to a multi-day
 > v6.19..v7.0
 > bisection. Classifying a kernel as "good" currently costs several
 > hours
 > and several TiB of reads, which makes ~13 bisection steps impractical
 > for me. If someone can suggest a faster trigger, that changes.

Even if it's going to take two weeks to do (perhaps run a test kernel 
for 4 hours a day) getting a specific commit will be really helpful if 
you're 100% sure it's a failure caused by a kernel change.

I will note that there are some other bug reports that are showing 
generic IOMMU changes earlier this summer that /might/ be similar.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1141183
https://github.com/Alvinwylim/asm1166-iommu-dma-corruption

> 
> Mario -- if the issue you were debugging internally has a comparable signature,
> I would be glad to compare. I have the raw btrfs csum lines showing eleven
> consecutive 4 KiB blocks returned in exact reverse order around a pivot, plus
> full dmesg, kernel config, lspci -nnvv and IOMMU domain dumps for both the
> failing and the working configuration. I can also collect whatever specific
> diagnostics you want, including with a debug patch if you provide one.
The corruption issue that my team is looking at is specifically with 
NVME and doesn't follow a kernel version.

So it's not a slam dunk to say it's the same.  BUT the issue internally 
does show the issue is specificially once the 32-bit IOVA space is 
exhausted.

Unfortunately; the solution is currently a BIOS change in how the type 
bytes of the IOVA is handled.

But the workaround that Vasant suggested (amd_iommu=pgtbl_v2 
iommu.forcedac=1) will help confirm if it's the exact same failure path.

Without BIOS change issue can't be reproduced with those applied.