Re: [BUG] s2idle: unrecoverable sleep on ThinkPad P16s Gen 4 AMD, (Strix Point) when more than 16 logical CPUs are online

[Mario Limonciello <superm1@gmail.com>]

> Captured acpidumps for all three configurations. Your hunch was right 
> that the IVRS table is still created with AmdVt disabled, but it is not 
> identical - and the difference looks like the actual mechanism.

OK.  That makes a lot of sense for this issue now.

> 
> Summary, all with the IOMMU enabled and 24 CPUs online:
> 
>    config          IVinfo      IVRS len   IVMDs   MSFT0201 in IVRS 
> MSFT0201 ACPI dev   SSDTs
>    both enabled    0x00203043  0x216      3       yes                yes 
>                  29
>    AmdVt off       0x00203041  0x1F6      2       yes                yes 
>                  29
>    Pluton off      0x00203043  0x216      3       yes                NO 
>                  28
> 
> AmdVt off changes two things. The IVinfo word loses bit 0x2, the 
> DmaRemap bit you already parse at offset 36. And the table is 32 bytes 
> shorter because one IVMD disappears:
> 
>    both enabled:
>      IVMD DeviceId=0060 Flags=07 Start=0x7D900000 Len=0x100000
>      IVMD DeviceId=C507 Flags=08 Start=0x6B400000 Len=0x20000
>      IVMD DeviceId=C100 Flags=08 Start=0x6B1F5000 Len=0x28000
> 
>    AmdVt off:
>      IVMD DeviceId=C507 Flags=08 Start=0x6B400000 Len=0x20000
>      IVMD DeviceId=C100 Flags=08 Start=0x6B1F5000 Len=0x28000
> 
> DeviceId 0x0060 is MSFT0201:
> 
>    AMD-Vi: ivrs, add hid:MSFT0201, uid:1, rdevid:0x60
> 
> and Flags 0x07 is unity mapped with read and write. So disabling AMD 
> virtualization removes Pluton's unity mapped DMA region from IVRS, while 
> the device is still declared in IVRS, still present as an ACPI device, 
> and still attached to iommu group 0:
> 
>    platform MSFT0201:00: Adding to iommu group 0
> 
> That would leave Pluton's DMA translated with nothing mapped for it, 
> which fits every observation: it only breaks with the IOMMU enabled, 
> amd_iommu=off is the only thing that fixes it, and neither intremap=off 
> nor iommu=pt helps, since iommu=pt gives other devices identity domains 
> but the IVMD is how firmware asks for one for this device specifically.
> 
> Pluton off is a different failure. The IVRS is byte identical to the 
> working case, IVMD 0060 included, but one SSDT is gone and with it the 
> MSFT0201 ACPI device. Your existing check already catches that case 
> correctly:
> 
>    found_iommu=True  found_acpi=False
>    -> IOMMU is misconfigured: missing MSFT0201 ACPI device
> 
> The AmdVt case is the one that slips through. In check_iommu():
> 
>    if not found_ivrs_dmar and not found_ivrs_msft0201:
> 
> with AmdVt off, found_ivrs_dmar is False because bit 0x2 is cleared, but 
> found_ivrs_msft0201 is True, so the and makes it pass and the tool 
> reports "IOMMU properly configured" on a machine that cannot be woken.
> 
> So to answer your question about keying off something else: yes, and I 
> think the signal is "IVRS declares an ACPI HID device but provides no 
> IVMD covering its device id". That needs no vendor attributes and no DMI 
> matching. The device id sits 3 bytes before the HID string in the type 
> 0xF0 entry, and IVMDs are subtable types 0x20/0x21/0x22 with the device 
> id at entry offset 4:
> 
>    devid = struct.unpack_from("<H", data, data.find(b"MSFT0201") - 3)[0]
>    off = 48
>    while off + 4 <= len(data):
>        length = struct.unpack_from("<H", data, off + 2)[0]
>        if length == 0:
>            break
>        if data[off] in (0x20, 0x21, 0x22):
>            if struct.unpack_from("<H", data, off + 4)[0] == devid:
>                mapped = True
>                break
>        off += length
> 
> I checked this against all three dumps: it reports mapped for both- 
> enabled and Pluton-off, and unmapped for AmdVt-off. Note that it would 
> contradict test_check_iommu_no_dma_protection_BUT_msft0201, which 
> currently asserts that MSFT0201 in IVRS is an acceptable substitute for 
> pre-boot DMA protection, so whether to change that semantic is your call.
> 
> I am happy to send you the three acpidumps, with the extracted and 
> disassembled tables and a record of the BIOS settings each was taken under.
> 
> Beyond that I have to stop here. This has taken the better part of a 
> week, a lot of forced power cycles, and around $700 in LLM tokens 
> working through it, and I am out of time and energy to take it further. 
> The machine works now with AMD virtualization and Pluton enabled, which 
> is a perfectly acceptable outcome for me.
> 
> Everything I found is in this thread, and I hope the IVRS observation is 
> useful to you or to whoever picks it up. Thanks for the help getting 
> here - the amd-s2idle tool, the pointer to e9f850ba66cd, and the 
> suggestion to reset the BIOS were all what moved it forward.
> 
> 
I've opened up a PR that should hopefully adjust the tool against your 
failure cases.  Would you be able to confirm this against your system?

https://github.com/superm1/amd-debug-tools/pull/58

If it doesn't work, can you please send me the acpidumps and I'll adjust.