Re: AMD IOMMU issue with hibernate on Asrock X600

[Johannes Stezenbach <js@sig21.net>]

On Wed, Feb 04, 2026 at 04:16:51PM +0100, Johannes Stezenbach wrote:
> I'm also attaching the ACPI IVRS table.
> 
> Side note: I wanted to check AMD documents at https://docs.amd.com/
> but the "Create Account" button doesn't work. :-(
> (And why do they require an account anyway, Intel and ARM don't...)

Eventually I could download the "AMD I/O Virtualization Technology
(IOMMU) Specification" document (link on https://uefi.org/acpi
is stale, BTW).

> [074h 0116 001h]               Subtable Type : 11 [Hardware Definition Block (IVHD)]
> [075h 0117 001h]       Flags (decoded below) : 30
>                                      HtTunEn : 0
>                                       PassPW : 0
>                                    ResPassPW : 0
>                                 Isoc Control : 0
>                                Iotlb Support : 1
>                                     Coherent : 1
>                             Prefetch Support : 0
>                                  PPR Support : 0
> [076h 0118 002h]                      Length : 0054
> [078h 0120 002h]                    DeviceId : 0002
> [07Ah 0122 002h]           Capability Offset : 0040
> [07Ch 0124 008h]                Base Address : 00000000F7500000
> [084h 0132 002h]           PCI Segment Group : 0000
> [086h 0134 002h]         Virtualization Info : 0000
> [088h 0136 004h]                  Attributes : 00048000
> [08Ch 0140 008h]                   EFR Image : 246577EFA2254AFA
> [094h 0148 008h]                    Reserved : 0000000000000000

So the platform doesn't support x2APIC, but it's no problem.
GALog should be supported.

I added a printk for the MMIO_STATUS_OFFSET register after
the wait loop, and a WARN_ON(1) at the end of
iommu_ga_log_enable() to find all call chains.
Below are the important parts of the log, it shows on resume
iommu_ga_log_enable() is called twice:

1. from lapic_resume() syscore_ops via
   irq_remapping_reenable() -> amd_iommu_reenable()

2. from amd_iommu_resume() syscore_ops

The first time works, the second time fails.
amd_iommu_resume() calls early_enable_iommu() -> iommu_disable(),
maybe this double enable+disable in quick succession triggers
the problem?

BTW, iommu_init_ga_log() is called everytime in
enable_iommus_vapic() and allocates new ga_log
and ga_log_tail buffers, but the previous allocation
is never freed.


Best Regards,
Johannes



Log excerpts:
At bootup:

[    2.609023] [    T1] AMD-Vi: Extended features (0x246577efa2254afa, 0x0): PPR NX GT [5] IA GA PC GA_vAPIC
[    2.609036] [    T1] AMD-Vi: Interrupt remapping enabled
[    2.643865] [    T1] AMD-Vi: iommu_ga_log_enable status 00000118

[    2.643887] [    T1] WARNING: drivers/iommu/amd/init.c:956 at amd_iommu_enable_interrupts.cold+0xf/0x9d, CPU#4: swapper/0/1
[    2.644009] [    T1] Call Trace:
[    2.644015] [    T1]  <TASK>
[    2.644021] [    T1]  iommu_go_to_state+0xb20/0x1830
[    2.644031] [    T1]  ? __pfx_pci_iommu_init+0x10/0x10
[    2.644039] [    T1]  pci_iommu_init+0x15/0x40
[    2.644047] [    T1]  do_one_initcall+0x5b/0x310

[    2.644137] [    T1] AMD-Vi: Virtual APIC enabled


S3 suspend + resume:

[   55.915000] [ T3615] ACPI: PM: Low-level resume complete
[   55.915029] [ T3615] ACPI: PM: Restoring platform NVS memory
[   58.883988] [ T3615] AMD-Vi: iommu_ga_log_enable status 00000118

[   58.883997] [ T3615] WARNING: drivers/iommu/amd/init.c:956 at amd_iommu_enable_interrupts.cold+0xf/0x9d, CPU#0: pm-suspend/3615

[   58.884026] [ T3615] Call Trace:
[   58.884028] [ T3615]  <TASK>
[   58.884030] [ T3615]  amd_iommu_reenable+0x57/0x70
[   58.884034] [ T3615]  lapic_resume+0x176/0x260
[   58.884038] [ T3615]  syscore_resume+0x5c/0x190
[   58.884040] [ T3615]  suspend_devices_and_enter+0x752/0x870
[   58.884044] [ T3615]  pm_suspend.cold+0x2e1/0x35f

[   58.884167] [ T3615] AMD-Vi: Virtual APIC enabled
[   63.874343] [ T3615] AMD-Vi: iommu_ga_log_enable status 00000118

[   63.874351] [ T3615] WARNING: drivers/iommu/amd/init.c:956 at amd_iommu_enable_interrupts.cold+0xf/0x9d, CPU#0: pm-suspend/3615

[   63.874382] [ T3615] Call Trace:
[   63.874384] [ T3615]  <TASK>
[   63.874386] [ T3615]  syscore_resume+0x5c/0x190
[   63.874388] [ T3615]  suspend_devices_and_enter+0x752/0x870
[   63.874391] [ T3615]  pm_suspend.cold+0x2e1/0x35f
[   63.874394] [ T3615]  state_store+0x6b/0xd0
[   63.874398] [ T3615]  kernfs_fop_write_iter+0x159/0x210
[   63.874401] [ T3615]  vfs_write+0x270/0x4c0
[   63.874407] [ T3615]  ksys_write+0x73/0xf0

[   63.874563] [ T3615] AMD-Vi: Virtual APIC enabled


S4 hibernate + resume:

[  113.664694] [ T4023] PM: hibernation: Normal pages needed: 594162 + 1024, available pages: 15549652
[  114.020974] [ T4023] PM: hibernation: free pages cleared after restore
[  114.020993] [ T4023] ACPI: PM: Restoring platform NVS memory
[  115.909880] [ T4023] AMD-Vi: iommu_ga_log_enable status 00000118

[  115.909892] [ T4023] WARNING: drivers/iommu/amd/init.c:956 at amd_iommu_enable_interrupts.cold+0xf/0x9d, CPU#0: pm-hibernate/4023

[  115.909926] [ T4023] Call Trace:
[  115.909928] [ T4023]  <TASK>
[  115.909931] [ T4023]  amd_iommu_reenable+0x57/0x70
[  115.909936] [ T4023]  lapic_resume+0x176/0x260
[  115.909940] [ T4023]  syscore_resume+0x5c/0x190
[  115.909943] [ T4023]  hibernation_snapshot+0x292/0x590
[  115.909947] [ T4023]  hibernate.cold+0x110/0x491
[  115.909950] [ T4023]  state_store+0xce/0xd0
[  115.909954] [ T4023]  kernfs_fop_write_iter+0x159/0x210
[  115.909958] [ T4023]  vfs_write+0x270/0x4c0
[  115.909964] [ T4023]  ksys_write+0x73/0xf0

[  115.910063] [ T4023] AMD-Vi: Virtual APIC enabled
[  142.633907] [ T4023] AMD-Vi: iommu_ga_log_enable status 00000018
[  142.633912] [ T4023] ------------[ cut here ]------------
[  142.633912] [ T4023] i >= MMIO_STATUS_TIMEOUT
[  142.633913] [ T4023] WARNING: drivers/iommu/amd/init.c:953 at amd_iommu_enable_interrupts+0x4d2/0x500, CPU#0: pm-hibernate/4023

[  142.633937] [ T4023] Call Trace:
[  142.633938] [ T4023]  <TASK>
[  142.633939] [ T4023]  syscore_resume+0x5c/0x190
[  142.633941] [ T4023]  hibernation_snapshot+0x292/0x590
[  142.633943] [ T4023]  hibernate.cold+0x110/0x491
[  142.633945] [ T4023]  state_store+0xce/0xd0
[  142.633948] [ T4023]  kernfs_fop_write_iter+0x159/0x210
[  142.633950] [ T4023]  vfs_write+0x270/0x4c0
[  142.633955] [ T4023]  ksys_write+0x73/0xf0

[  142.634072] [ T4023] LVT offset 0 assigned for vector 0x400
[  142.634583] [ T4023] Enabling non-boot CPUs ...