[REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mikael Etienne]

[Resend as plain text. The first attempt was rejected by all four lists for
containing an HTML part -- apologies to those in Cc who receive this twice.]

Hi,

Since Linux 7.0 I observe silent read corruption on SATA disks behind an AMD
600-series chipset AHCI controller. AHCI/libata, the block layer, the IOMMU and
PCIe AER report no error at all; only consumers that validate the returned bytes
(btrfs data checksums, fio --verify) notice. SMART stays PASSED throughout.

Hardware, up front because it may matter: AMD Ryzen 7 8700G, with the SATA
controller integrated in the chipset:

  0e:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD]
          600 Series Chipset SATA Controller [1022:43f6] (rev 01)

This is the same SoC as bug 219609, where dma-iommu was already identified as the
trigger for a different device class (NVMe). Details in section 6.

Two observations sharply constrain the failure:

1. Once corruption starts, rebooting with the same kernel and the same command
   line, WITHOUT rewriting the test file, makes the very same blocks verify
   correctly again. Continued reads can later trigger the failure again. I have
   therefore observed no persistent corruption of the test file contents; this
   behaves like transient read-path state.

2. Adding only "iommu=pt" to the command line (nothing else changed): no
   corruption observed over 40 consecutive verification passes, 10 TiB re-read
   in 17 h. In the default translated mode the same test failed after 768 GiB
   / 3 h 25 m.

Good/bad boundary: 6.18.16 and 6.19.x are clean, every 7.0.x and 7.1.x is
affected. Kernel is untainted (/proc/sys/kernel/tainted = 0), no out-of-tree or
DKMS modules loaded.

Important caveat, stated up front: all kernels tested so far are Fedora-packaged.
I have not confirmed this on vanilla upstream and I have not bisected. See "What
I can and cannot do" at the end.

#regzbot introduced: v6.19..v7.0


## 1. Test matrix

Test file: 256 GiB fio canary with crc32c verification, on btrfs, on the 12 TB
drive (WDC WD120EFGX-68CPHN0).

  7.1.10-200.fc44, translated (DMA-FQ):
      first verify failure after 768 GiB re-read / 3 h 25 m
      once in the failed state, a plain 4 GiB O_DIRECT read of the same file
      produced tens of thousands of further csum failures
      btrfs corruption_errs reached 20,858,094 in that session
      reboot -> counters back to 0, same file verifies clean again

  7.1.10-200.fc44, iommu=pt (identity):
      40 consecutive clean passes, 10 TiB re-read, 17 h
      btrfs corruption_errs = 0, zero kernel error lines

  Earlier, translated mode, other workloads on the 8 TB drive:
      read-only workload      9 h 30 m clean
      deduplication workload  failed at 3 h 17 m
      sequential write        failed at 6 h 35 m

I have not yet run enough boots to give a proper min/median/max distribution.
The 3 h 25 m figure above is a single, carefully instrumented data point.


## 2. Reproducer

fio 3.40, io_uring engine (libaio not tested yet).

Write the canary once:

  fio --name=canary --filename=/srv/12to/.sata-canary --size=256G --bs=128k \
      --ioengine=io_uring --direct=1 --iodepth=32 \
      --verify=crc32c --verify_interval=4096 --rw=write \
      --do_verify=0 --fsync_on_close=1

Then loop the verification until it fails:

  while :; do
      fio --name=canary --filename=/srv/12to/.sata-canary --size=256G --bs=128k \
          --ioengine=io_uring --direct=1 --iodepth=32 \
          --verify=crc32c --verify_interval=4096 --rw=write \
          --verify_only=1 --verify_fatal=1 || break
  done

The reboot-without-rewrite protocol: when it fails, reboot and re-run only the
verification loop above. The canary file is never rewritten. It verifies clean.


## 3. Corruption signature

The frequent mode is a 4096-byte page read back as all zeros. btrfs reports
"csum 0x8941f998", which is the CRC32C of a zero-filled 4 KiB block.

The interesting mode is a structured permutation. From one btrfs report, inode
6646:

  offset A   offset B   delta
  1314816    1355776    40960
  1318912    1351680    32768
  1323008    1347584    24576
  1327104    1343488    16384
  1331200    1339392     8192

Verified programmatically from the raw btrfs csum lines: the data read at offset
A is exactly what was expected at offset B, and vice versa. Five reciprocal
pairs, symmetric around the pivot at 1335296 -- a run of 11 consecutive 4 KiB
blocks in reverse order, the centre block mapping onto itself.

This pattern suggests incorrect DMA/scatterlist mapping or descriptor handling,
because the affected chunks form a structured permutation rather than random bit
corruption. It does not by itself identify the faulty layer.


## 4. Hardware and storage stack

  Gigabyte X870I AORUS PRO ICE, BIOS FB1c (2026-07-21)
  AMD Ryzen 7 8700G w/ Radeon 780M Graphics
  32 GB DDR5 non-ECC, single stick, JEDEC 4800 (EXPO/XMP off)

  0e:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD]
          600 Series Chipset SATA Controller [1022:43f6] (rev 01)

  ahci flags: 64bit ncq sntf stag pm led clo only pmp pio slum part sxs deso
              sadm sds apst
  32 command slots, queue_depth 32, max_segment_size 65536, max_segments 168,
  max_sectors_kb 4096, scheduler bfq

  IOMMU: AMD-Vi, "Default domain type: Translated",
         "DMA domain TLB invalidation policy: lazy mode"
  Controller iommu_group type: DMA-FQ, becomes "identity" with iommu=pt

  Test drive: /dev/sdb1 -> /srv/12to
    btrfs, data single, metadata DUP, mounted rw,noatime,compress=zstd:3,
    space_cache=v2. Plain partition: no LVM, no dm-crypt, no mdraid.

  Drives that showed the symptom (three different drives, one taken new out of
  its box and affected within hours of first use):
    WDC WD120EFGX-68CPHN0, fw 85.00B85   (btrfs, quantified above)
    Seagate ST8000VN004-2M2101, fw SC60  (btrfs)
    WDC WD101EFBX-68B0AN0                (ext4)

  NVMe devices in the same machine have never shown the symptom.


## 5. What I ruled out

  - NCQ: queue_depth 32 -> 1, no change (304 vs 769 csum failures per 4 GiB read)
  - transfer size: max_sectors_kb 4096 -> 64, no change (388 vs 342)
  - both combined: no change
  - SATA link power management: already max_performance on the affected ports
  - PCIe ASPM: disabled on that link
  - PCIe AER: all correctable and non-fatal counters at 0
  - SATA link CRC (SMART attribute 199): 0 on every drive
  - temperature: 41-56 C
  - swiotlb=force does NOT help, which is consistent with the dma-iommu path
    still being used underneath
  - not btrfs-specific: at the same moment, ext4 on a second drive reported
    "bad header/extent: invalid magic - magic 0", and parted reported a corrupt
    GPT on it. Both drives read correctly again after a reboot.

btrfs is what first exposed the problem, through its data checksums. Filesystems
without user-data checksums may hand affected data to userspace without noticing,
although metadata validation or application-level checksums can still catch some
of it -- which is what happened with ext4 and parted above.


## 6. Possibly related

Bug 219609, "File corruptions on SSD in 1st M.2 socket of AsRock X600M-STX +
Ryzen 8700G". Christoph Hellwig writes there that "the problem only happens when
using the dma-iommu code (with or without swiotlb buffering for unaligned /
untrusted data)", and that iommu=pt or amd_iommu=off fix it.

Same SoC (Ryzen 8700G) as this machine, but a different device class (SATA/AHCI
here, NVMe there) and a different kernel window, so I am reporting separately and
cross-referencing rather than piling onto that bug.


## 7. What I can and cannot do

This is a production home server, not a test bench, so I want to be straight
about it:

  I CAN: run any specific test, boot parameter or debug patch you ask for, and
         report back with full instrumentation. I have a working reproducer and
         a drive I can dedicate to it.

  I CANNOT realistically: dedicate the machine to a multi-day v6.19..v7.0
         bisection. Classifying a kernel as "good" currently costs several hours
         and several TiB of reads, which makes ~13 bisection steps impractical
         for me. If someone can suggest a faster trigger, that changes.

I have not yet tested: vanilla upstream kernels, libaio instead of io_uring,
iommu.strict=1, or amd_iommu=off. I am happy to test any of these.

Available on request, immediately: full dmesg from a bad boot and from an
iommu=pt boot, kernel .config for good and bad, /proc/cmdline, uname -a,
/proc/sys/kernel/tainted, lspci -nnvv, lspci -t, IOMMU group and domain types,
queue/DMA sysfs attributes, SMART reports, AER counters, raw fio logs, the raw
btrfs csum lines, and the script that proved the A/B swaps.

Thanks,
Mikael Etienne

Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Vasant Hegde]

Mikael,

Thanks for the detailed report!

Can you please apply Commit 1e75a8255f11c81fb and retest. Also boot with
`amd_iommu=pgtbl_v2`?


@Mario, Can you look into this issue? Is this similar to the one you were
debugging internally?


-Vasant


On 8/28/2026 10:20 AM, Mikael Etienne wrote:
> You don't often get email from mikael1022bzh@gmail.com. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
> Hi,
> 
> Since Linux 7.0 I observe silent read corruption on SATA disks behind an AMD
> 600-series chipset AHCI controller. AHCI/libata, the block layer, the IOMMU and
> PCIe AER report no error at all; only consumers that validate the returned bytes
> (btrfs data checksums, fio --verify) notice. SMART stays PASSED throughout.
> 
> Hardware, up front because it may matter: AMD Ryzen 7 8700G, with the SATA
> controller integrated in the chipset:
> 
> 0e:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD]
> 600 Series Chipset SATA Controller [1022:43f6] (rev 01)
> 
> This is the same SoC as bug 219609, where dma-iommu was already identified as the
> trigger for a different device class (NVMe). Details in section 6.
> 
> Two observations sharply constrain the failure:
> 
> 1. Once corruption starts, rebooting with the same kernel and the same command
> line, WITHOUT rewriting the test file, makes the very same blocks verify
> correctly again. Continued reads can later trigger the failure again. I have
> therefore observed no persistent corruption of the test file contents; this
> behaves like transient read-path state.
> 
> 2. Adding only "iommu=pt" to the command line (nothing else changed): no
> corruption observed over 40 consecutive verification passes, 10 TiB re-read
> in 17 h. In the default translated mode the same test failed after 768 GiB
> / 3 h 25 m.
> 
> Good/bad boundary: 6.18.16 and 6.19.x are clean, every 7.0.x and 7.1.x is
> affected. Kernel is untainted (/proc/sys/kernel/tainted = 0), no out-of-tree or
> DKMS modules loaded.
> 
> Important caveat, stated up front: all kernels tested so far are Fedora-packaged.
> I have not confirmed this on vanilla upstream and I have not bisected. See "What
> I can and cannot do" at the end.
> 
> #regzbot introduced: v6.19..v7.0
> 
> 
> ## 1. Test matrix
> 
> Test file: 256 GiB fio canary with crc32c verification, on btrfs, on the 12 TB
> drive (WDC WD120EFGX-68CPHN0).
> 
> 7.1.10-200.fc44, translated (DMA-FQ):
> first verify failure after 768 GiB re-read / 3 h 25 m
> once in the failed state, a plain 4 GiB O_DIRECT read of the same file
> produced tens of thousands of further csum failures
> btrfs corruption_errs reached 20,858,094 in that session
> reboot -> counters back to 0, same file verifies clean again
> 
> 7.1.10-200.fc44, iommu=pt (identity):
> 40 consecutive clean passes, 10 TiB re-read, 17 h
> btrfs corruption_errs = 0, zero kernel error lines
> 
> Earlier, translated mode, other workloads on the 8 TB drive:
> read-only workload 9 h 30 m clean
> deduplication workload failed at 3 h 17 m
> sequential write failed at 6 h 35 m
> 
> I have not yet run enough boots to give a proper min/median/max distribution.
> The 3 h 25 m figure above is a single, carefully instrumented data point.
> 
> 
> ## 2. Reproducer
> 
> fio 3.40, io_uring engine (libaio not tested yet).
> 
> Write the canary once:
> 
> fio --name=canary --filename=/srv/12to/.sata-canary --size=256G --bs=128k \
> --ioengine=io_uring --direct=1 --iodepth=32 \
> --verify=crc32c --verify_interval=4096 --rw=write \
> --do_verify=0 --fsync_on_close=1
> 
> Then loop the verification until it fails:
> 
> while :; do
> fio --name=canary --filename=/srv/12to/.sata-canary --size=256G --bs=128k \
> --ioengine=io_uring --direct=1 --iodepth=32 \
> --verify=crc32c --verify_interval=4096 --rw=write \
> --verify_only=1 --verify_fatal=1 || break
> done
> 
> The reboot-without-rewrite protocol: when it fails, reboot and re-run only the
> verification loop above. The canary file is never rewritten. It verifies clean.
> 
> 
> ## 3. Corruption signature
> 
> The frequent mode is a 4096-byte page read back as all zeros. btrfs reports
> "csum 0x8941f998", which is the CRC32C of a zero-filled 4 KiB block.
> 
> The interesting mode is a structured permutation. From one btrfs report, inode
> 6646:
> 
> offset A offset B delta
> 1314816 1355776 40960
> 1318912 1351680 32768
> 1323008 1347584 24576
> 1327104 1343488 16384
> 1331200 1339392 8192
> 
> Verified programmatically from the raw btrfs csum lines: the data read at offset
> A is exactly what was expected at offset B, and vice versa. Five reciprocal
> pairs, symmetric around the pivot at 1335296 -- a run of 11 consecutive 4 KiB
> blocks in reverse order, the centre block mapping onto itself.
> 
> This pattern suggests incorrect DMA/scatterlist mapping or descriptor handling,
> because the affected chunks form a structured permutation rather than random bit
> corruption. It does not by itself identify the faulty layer.
> 
> 
> ## 4. Hardware and storage stack
> 
> Gigabyte X870I AORUS PRO ICE, BIOS FB1c (2026-07-21)
> AMD Ryzen 7 8700G w/ Radeon 780M Graphics
> 32 GB DDR5 non-ECC, single stick, JEDEC 4800 (EXPO/XMP off)
> 
> 0e:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD]
> 600 Series Chipset SATA Controller [1022:43f6] (rev 01)
> 
> ahci flags: 64bit ncq sntf stag pm led clo only pmp pio slum part sxs deso
> sadm sds apst
> 32 command slots, queue_depth 32, max_segment_size 65536, max_segments 168,
> max_sectors_kb 4096, scheduler bfq
> 
> IOMMU: AMD-Vi, "Default domain type: Translated",
> "DMA domain TLB invalidation policy: lazy mode"
> Controller iommu_group type: DMA-FQ, becomes "identity" with iommu=pt
> 
> Test drive: /dev/sdb1 -> /srv/12to
> btrfs, data single, metadata DUP, mounted rw,noatime,compress=zstd:3,
> space_cache=v2. Plain partition: no LVM, no dm-crypt, no mdraid.
> 
> Drives that showed the symptom (three different drives, one taken new out of
> its box and affected within hours of first use):
> WDC WD120EFGX-68CPHN0, fw 85.00B85 (btrfs, quantified above)
> Seagate ST8000VN004-2M2101, fw SC60 (btrfs)
> WDC WD101EFBX-68B0AN0 (ext4)
> 
> NVMe devices in the same machine have never shown the symptom.
> 
> 
> ## 5. What I ruled out
> 
> - NCQ: queue_depth 32 -> 1, no change (304 vs 769 csum failures per 4 GiB read)
> - transfer size: max_sectors_kb 4096 -> 64, no change (388 vs 342)
> - both combined: no change
> - SATA link power management: already max_performance on the affected ports
> - PCIe ASPM: disabled on that link
> - PCIe AER: all correctable and non-fatal counters at 0
> - SATA link CRC (SMART attribute 199): 0 on every drive
> - temperature: 41-56 C
> - swiotlb=force does NOT help, which is consistent with the dma-iommu path
> still being used underneath
> - not btrfs-specific: at the same moment, ext4 on a second drive reported
> "bad header/extent: invalid magic - magic 0", and parted reported a corrupt
> GPT on it. Both drives read correctly again after a reboot.
> 
> btrfs is what first exposed the problem, through its data checksums. Filesystems
> without user-data checksums may hand affected data to userspace without noticing,
> although metadata validation or application-level checksums can still catch some
> of it -- which is what happened with ext4 and parted above.
> 
> 
> ## 6. Possibly related
> 
> Bug 219609, "File corruptions on SSD in 1st M.2 socket of AsRock X600M-STX +
> Ryzen 8700G". Christoph Hellwig writes there that "the problem only happens when
> using the dma-iommu code (with or without swiotlb buffering for unaligned /
> untrusted data)", and that iommu=pt or amd_iommu=off fix it.
> 
> Same SoC (Ryzen 8700G) as this machine, but a different device class (SATA/AHCI
> here, NVMe there) and a different kernel window, so I am reporting separately and
> cross-referencing rather than piling onto that bug.
> 
> 
> ## 7. What I can and cannot do
> 
> This is a production home server, not a test bench, so I want to be straight
> about it:
> 
> I CAN: run any specific test, boot parameter or debug patch you ask for, and
> report back with full instrumentation. I have a working reproducer and
> a drive I can dedicate to it.
> 
> I CANNOT realistically: dedicate the machine to a multi-day v6.19..v7.0
> bisection. Classifying a kernel as "good" currently costs several hours
> and several TiB of reads, which makes ~13 bisection steps impractical
> for me. If someone can suggest a faster trigger, that changes.
> 
> I have not yet tested: vanilla upstream kernels, libaio instead of io_uring,
> iommu.strict=1, or amd_iommu=off. I am happy to test any of these.
> 
> Available on request, immediately: full dmesg from a bad boot and from an
> iommu=pt boot, kernel .config for good and bad, /proc/cmdline, uname -a,
> /proc/sys/kernel/tainted, lspci -nnvv, lspci -t, IOMMU group and domain types,
> queue/DMA sysfs attributes, SMART reports, AER counters, raw fio logs, the raw
> btrfs csum lines, and the script that proved the A/B swaps.
> 
> Thanks,
> 

Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mikael Etienne]

On 8/28/2026 11:13 AM, Vasant Hegde wrote:
> Can you please apply Commit 1e75a8255f11c81fb and retest. Also boot with
> `amd_iommu=pgtbl_v2`?
>
> @Mario, Can you look into this issue? Is this similar to the one you were
> debugging internally?

Hi Vasant,

Thanks for the quick response.

I checked before testing, and I believe that commit is already in my kernel.
CVE-2026-68329 lists it as fixed in 7.1.6 with 02f8cefa2ad95ea3754f0cfd6fbae7f866202ccb
(and in 7.2-rc5 with 1e75a8255f11c81fb07e81e5029cfd75804350a0, the one you
pointed at). I am running Fedora's 7.1.10-200.fc44, whose changelog says
"Linux v7.1.10", so the 7.1.y backport should be present.

And I have reproduced the corruption on kernels that already contain it. On
7.1.9 my journal has 119 corruption lines for a single boot on 2026-08-23, and
the 20-million-error run described in my first mail was on 7.1.10.

There is a second, independent reason to think it may not be my bug: that CVE
says the race was introduced in 3.0. My persistent journal goes back to
2026-02-27 and shows zero storage errors under 6.18.16 and 6.19.x, including two
full multi-TB backup reads of the same drive on 2026-04-01 and 2026-05-01. If I
were hitting the 2011 race, I would expect it to show there too. What I see
looks like something that appeared between 6.19 and 7.0.

I may of course be wrong about the Fedora build. If you want me to verify the
exact source, tell me what to check and I will.

On amd_iommu=pgtbl_v2: I can run it, and I will report numbers rather than
impressions -- bytes re-read, wall time and the exact workload. Two constraints,
so you know what to expect:

  - It is a production machine, and testing means removing the iommu=pt
    workaround. I also assume a misdirected DMA can land in RAM belonging to
    unrelated processes, not just in the read buffer, so I will stop all
    services for the test rather than run it live.

  - My only quantified data point in translated mode is a single failure after
    768 GiB / 3h25m of verification reads. To honestly call a setting clean I
    would want on the order of 15 TB re-read, which is roughly a day. I will
    not report "this fixes it" on a short clean run.

A 7 TB integrity verification is currently running on this machine and I would
rather let it finish before rebooting. Given the above, would you still like the
pgtbl_v2 run, or is there a cheaper test that would tell you more? I am equally
happy to try libaio instead of io_uring, iommu.strict=1, or amd_iommu=off, in
whatever order is most useful to you.

Mario -- if the issue you were debugging internally has a comparable signature,
I would be glad to compare. I have the raw btrfs csum lines showing eleven
consecutive 4 KiB blocks returned in exact reverse order around a pivot, plus
full dmesg, kernel config, lspci -nnvv and IOMMU domain dumps for both the
failing and the working configuration. I can also collect whatever specific
diagnostics you want, including with a debug patch if you provide one.

Thanks,
Mikael Etienne

Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mario Limonciello]

 > I CANNOT realistically: dedicate the machine to a multi-day
 > v6.19..v7.0
 > bisection. Classifying a kernel as "good" currently costs several
 > hours
 > and several TiB of reads, which makes ~13 bisection steps impractical
 > for me. If someone can suggest a faster trigger, that changes.

Even if it's going to take two weeks to do (perhaps run a test kernel 
for 4 hours a day) getting a specific commit will be really helpful if 
you're 100% sure it's a failure caused by a kernel change.

I will note that there are some other bug reports that are showing 
generic IOMMU changes earlier this summer that /might/ be similar.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1141183
https://github.com/Alvinwylim/asm1166-iommu-dma-corruption

> 
> Mario -- if the issue you were debugging internally has a comparable signature,
> I would be glad to compare. I have the raw btrfs csum lines showing eleven
> consecutive 4 KiB blocks returned in exact reverse order around a pivot, plus
> full dmesg, kernel config, lspci -nnvv and IOMMU domain dumps for both the
> failing and the working configuration. I can also collect whatever specific
> diagnostics you want, including with a debug patch if you provide one.
The corruption issue that my team is looking at is specifically with 
NVME and doesn't follow a kernel version.

So it's not a slam dunk to say it's the same.  BUT the issue internally 
does show the issue is specificially once the 32-bit IOVA space is 
exhausted.

Unfortunately; the solution is currently a BIOS change in how the type 
bytes of the IOVA is handled.

But the workaround that Vasant suggested (amd_iommu=pgtbl_v2 
iommu.forcedac=1) will help confirm if it's the exact same failure path.

Without BIOS change issue can't be reproduced with those applied.

Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mikael Etienne]

On 8/28/2026 6:22 PM, Limonciello, Mario wrote:
> BUT the issue internally does show the issue is specificially once the
> 32-bit IOVA space is exhausted.
>
> Unfortunately; the solution is currently a BIOS change in how the type
> bytes of the IOVA is handled.

Hi Mario,

Thanks -- the 32-bit IOVA exhaustion detail is the first thing that explains
the timing I see. My failure takes 3h25m of sustained reads to appear, and a
reboot clears it completely with the same kernel and the same on-disk data.
"Low IOVA space works, high IOVA space does not, and a reboot starts from an
empty low space again" fits that exactly.

The ASM1166 write-up you linked is also a very close match: a controller that
advertises CAP.S64A but cannot actually reach above 4 GB. My SATA controller is
the AMD 600-series chipset one (1022:43f6), which is Promontory/ASMedia silicon,
and it likewise advertises 64bit in its AHCI flags.

Firmware, since you mention a BIOS fix: I am already on the latest available for
this board.

  Gigabyte X870I AORUS PRO ICE
  BIOS FB1c, 2026-07-21, AMI, platform firmware revision 5.41
  AGESA!V9 ComboAm5PI 1.3.0.1c
  CPU microcode 0x0a70520a

So whatever BIOS-side change you have internally is either not in AGESA
1.3.0.1c, or not sufficient on this board.

On the bisection: before committing to two weeks, I would like to try making the
reproducer fast. If the trigger really is 32-bit IOVA exhaustion, then booting
with iommu.forcedac=1 alone should hand out high IOVAs immediately and fail in
minutes rather than after 3h25m. If that works, a v6.19..v7.0 bisection becomes
an evening rather than a fortnight, and you would also have a reproducer that is
practical to run in a lab.

I will try that tonight, with all services stopped, then the
amd_iommu=pgtbl_v2 iommu.forcedac=1 combination Vasant asked for. I will report
bytes re-read, wall time and workload rather than a verdict.

If you already know that forcedac alone will not behave that way, please tell me
and I will not waste the evening on it.

And to be straight about the rest: this is a production home server, so a
two-week bisection is a real cost for me. The reproducer is three lines of fio.
If you can run it on comparable hardware in your lab, that would very likely be
faster than me doing it here -- and I am happy to run any specific kernel or
debug patch you want in the meantime.

Thanks,
Mikael Etienne

Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mario Limonciello]

On 8/28/26 10:24, Mikael Etienne wrote:
> On 8/28/2026 6:22 PM, Limonciello, Mario wrote:
>> BUT the issue internally does show the issue is specificially once the
>> 32-bit IOVA space is exhausted.
>>
>> Unfortunately; the solution is currently a BIOS change in how the type
>> bytes of the IOVA is handled.
> 
> Hi Mario,
> 
> Thanks -- the 32-bit IOVA exhaustion detail is the first thing that explains
> the timing I see. My failure takes 3h25m of sustained reads to appear, and a
> reboot clears it completely with the same kernel and the same on-disk data.
> "Low IOVA space works, high IOVA space does not, and a reboot starts from an
> empty low space again" fits that exactly.
> 
> The ASM1166 write-up you linked is also a very close match: a controller that
> advertises CAP.S64A but cannot actually reach above 4 GB. My SATA controller is
> the AMD 600-series chipset one (1022:43f6), which is Promontory/ASMedia silicon,
> and it likewise advertises 64bit in its AHCI flags.
> 
> Firmware, since you mention a BIOS fix: I am already on the latest available for
> this board.
> 
>    Gigabyte X870I AORUS PRO ICE
>    BIOS FB1c, 2026-07-21, AMI, platform firmware revision 5.41
>    AGESA!V9 ComboAm5PI 1.3.0.1c
>    CPU microcode 0x0a70520a
> 
> So whatever BIOS-side change you have internally is either not in AGESA
> 1.3.0.1c, or not sufficient on this board.

It's not in any AGESA release yet.  This is very fresh information I am 
sharing that we have root caused the issue and have a proposed 
modification.  It will take a while to make it through the process 
machinery.
> 
> On the bisection: before committing to two weeks, I would like to try making the
> reproducer fast. If the trigger really is 32-bit IOVA exhaustion, then booting
> with iommu.forcedac=1 alone should hand out high IOVAs immediately and fail in
> minutes rather than after 3h25m. If that works, a v6.19..v7.0 bisection becomes
> an evening rather than a fortnight, and you would also have a reproducer that is
> practical to run in a lab.

We do have a reproducer in our lab environment that will rapidly 
allocate and trip this issue which is how we could analyze it and root 
cause it.

> 
> I will try that tonight, with all services stopped, then the
> amd_iommu=pgtbl_v2 iommu.forcedac=1 combination Vasant asked for. I will report
> bytes re-read, wall time and workload rather than a verdict.
> 
> If you already know that forcedac alone will not behave that way, please tell me
> and I will not waste the evening on it.
> 
> And to be straight about the rest: this is a production home server, so a
> two-week bisection is a real cost for me. The reproducer is three lines of fio.
> If you can run it on comparable hardware in your lab, that would very likely be
> faster than me doing it here -- and I am happy to run any specific kernel or
> debug patch you want in the meantime.

I don't yet have any confirmation we can patch this at runtime.  If I do 
come up with a way to do that which works will let you know.

By chance did this issue coincide with you switching from something 
different to the Ryzen 7 8700G?  For example switching from Raphael or 
Granite Ridge parts to that Phoenix part.

Re: [REGRESSION] Silent SATA read corruption with dma-iommu on AMD 600-series AHCI (6.19 good, 7.0+ bad)

[Mikael Etienne]

On 8/28/26 10:37, Limonciello, Mario wrote:
> By chance did this issue coincide with you switching from something
> different to the Ryzen 7 8700G?  For example switching from Raphael or
> Granite Ridge parts to that Phoenix part.

No -- same CPU throughout, no hardware change at all.

My persistent journal goes back to 2026-02-27 and every boot in it reports the
same part:

  smpboot: CPU0: AMD Ryzen 7 8700G w/ Radeon 780M Graphics
  family 25, model 117, stepping 2

That covers 2026-03-17, 03-27, 05-04, 06-03, 08-23 and today. Same board, same
CPU, same SATA controller. The 8 TB drive that first showed the problem has been
in the machine since 2026-03-11, on the same port, and it absorbed two full
multi-TB backup reads on 2026-04-01 and 2026-05-01 with zero storage errors
under 6.18.16 / 6.19.x.

So on this machine the only thing that changed between "three months clean" and
"corruption" is the kernel: 6.19.14 until 2026-05-10, then 7.0.4. First
corruption 2026-06-02.

I realise that sits awkwardly next to your internal issue not following a kernel
version. Two readings I can think of, without picking one:

  - the hardware/firmware behaviour is constant, and something in 7.0 changed
    how quickly or how often the 32-bit IOVA space gets exhausted here, so the
    latent problem simply became reachable; or
  - they are genuinely two different problems that happen to share a
    workaround.

I have no way to tell those apart from here, and I am not going to guess.

> We do have a reproducer in our lab environment that will rapidly
> allocate and trip this issue which is how we could analyze it and root
> cause it.

Good -- then I will not spend the evening trying to build a fast one, and I will
drop the bisection idea unless you tell me it would still add something. If at
some point you want my slow reproducer run against a specific kernel or debug
patch, I am happy to do that; it is only my time that is expensive, not the
machine's.

> I don't yet have any confirmation we can patch this at runtime.  If I do
> come up with a way to do that which works will let you know.

Thank you, that is appreciated. For reference, iommu=pt has been completely
clean here: 40 consecutive 256 GiB verification passes, 10 TiB re-read over 17 h,
btrfs corruption counters at zero. I am running with it permanently for now and I
am not in any hurry.

If it helps your case, I am happy to be a data point on the SATA/AHCI side of
this, since the ASM1166 report and mine are both ASMedia silicon advertising
CAP.S64A.

Thanks,
Mikael Etienne