* AMD-Vi: Unable to read/write to IOMMU perf counter @ 2021-01-25 9:24 Tj (Elloe Linux) 2021-02-02 5:54 ` Suravee Suthikulpanit 0 siblings, 1 reply; 6+ messages in thread From: Tj (Elloe Linux) @ 2021-01-25 9:24 UTC (permalink / raw) To: iommu Lenovo E495 reports: pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter. pci 0000:00:00.2: can't derive routing for PCI INT A pci 0000:00:00.2: PCI INT A: not connected I found an existing identical bug report that doesn't seem to have gained any attention: https://bugzilla.kernel.org/show_bug.cgi?id=201753 Linux version 5.11.0-rc4+ (tj@elloe000) (gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #12 SMP PREEMPT Sun Jan 24 11:28:01 GMT 2021 Command line: BOOT_IMAGE=/vmlinuz-5.11.0-rc4+ root=/dev/mapper/ELLOE000-rootfs ro acpi_osi=! "acpi_osi=Windows 2016" systemd.unified_cgroup_hierarchy=1 nosplash ... DMI: LENOVO 20NECTO1WW/20NECTO1WW, BIOS R11ET32W (1.12 ) 12/23/2019 ... AMD-Vi: ivrs, add hid:PNPD0040, uid:, rdevid:152 ... smpboot: CPU0: AMD Ryzen 7 3700U with Radeon Vega Mobile Gfx (family: 0x17, model: 0x18, stepping: 0x1) ... pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter. pci 0000:00:00.2: can't derive routing for PCI INT A pci 0000:00:00.2: PCI INT A: not connected pci 0000:00:01.0: Adding to iommu group 0 pci 0000:00:01.1: Adding to iommu group 1 pci 0000:00:01.2: Adding to iommu group 2 pci 0000:00:01.3: Adding to iommu group 3 pci 0000:00:01.6: Adding to iommu group 4 pci 0000:00:08.0: Adding to iommu group 5 pci 0000:00:08.1: Adding to iommu group 6 pci 0000:00:14.0: Adding to iommu group 7 pci 0000:00:14.3: Adding to iommu group 7 pci 0000:00:18.0: Adding to iommu group 8 pci 0000:00:18.1: Adding to iommu group 8 pci 0000:00:18.2: Adding to iommu group 8 pci 0000:00:18.3: Adding to iommu group 8 pci 0000:00:18.4: Adding to iommu group 8 pci 0000:00:18.5: Adding to iommu group 8 pci 0000:00:18.6: Adding to iommu group 8 pci 0000:00:18.7: Adding to iommu group 8 pci 0000:01:00.0: Adding to iommu group 9 pci 0000:02:00.0: Adding to iommu group 10 pci 0000:03:00.0: Adding to iommu group 11 pci 0000:04:00.0: Adding to iommu group 12 pci 0000:05:00.0: Adding to iommu group 13 pci 0000:05:00.1: Adding to iommu group 14 pci 0000:05:00.2: Adding to iommu group 14 pci 0000:05:00.3: Adding to iommu group 14 pci 0000:05:00.4: Adding to iommu group 14 pci 0000:05:00.5: Adding to iommu group 14 pci 0000:05:00.6: Adding to iommu group 14 pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada): PPR NX GT IA GA PC GA_vAPIC AMD-Vi: Interrupt remapping enabled AMD-Vi: Virtual APIC enabled AMD-Vi: Lazy IO/TLB flushing enabled amd_uncore: 4 amd_df counters detected _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: AMD-Vi: Unable to read/write to IOMMU perf counter 2021-01-25 9:24 AMD-Vi: Unable to read/write to IOMMU perf counter Tj (Elloe Linux) @ 2021-02-02 5:54 ` Suravee Suthikulpanit 2021-02-04 6:25 ` Tj (Elloe Linux) 0 siblings, 1 reply; 6+ messages in thread From: Suravee Suthikulpanit @ 2021-02-02 5:54 UTC (permalink / raw) To: Tj (Elloe Linux), iommu [-- Attachment #1: Type: text/plain, Size: 3736 bytes --] Could you please try the attached patch to see if the problem still persist. Thanks, Suravee On 1/25/21 4:24 PM, Tj (Elloe Linux) wrote: > Lenovo E495 reports: > > pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter. > pci 0000:00:00.2: can't derive routing for PCI INT A > pci 0000:00:00.2: PCI INT A: not connected > > I found an existing identical bug report that doesn't seem to have > gained any attention: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.kernel.org%2Fshow_bug.cgi%3Fid%3D201753&data=04%7C01%7Csuravee.suthikulpanit%40amd.com%7C7c56640fcf24465050f008d8c145eba4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637471853347946970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uykr%2FZMpr%2BuLrw3k1bKVcwywfJB4CU0p2qJSZXgLNK8%3D&reserved=0 > > Linux version 5.11.0-rc4+ (tj@elloe000) (gcc (Ubuntu > 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #12 > SMP PREEMPT Sun Jan 24 11:28:01 GMT 2021 > Command line: BOOT_IMAGE=/vmlinuz-5.11.0-rc4+ > root=/dev/mapper/ELLOE000-rootfs ro acpi_osi=! "acpi_osi=Windows 2016" > systemd.unified_cgroup_hierarchy=1 nosplash > ... > DMI: LENOVO 20NECTO1WW/20NECTO1WW, BIOS R11ET32W (1.12 ) 12/23/2019 > ... > AMD-Vi: ivrs, add hid:PNPD0040, uid:, rdevid:152 > ... > smpboot: CPU0: AMD Ryzen 7 3700U with Radeon Vega Mobile Gfx (family: > 0x17, model: 0x18, stepping: 0x1) > ... > pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter. > pci 0000:00:00.2: can't derive routing for PCI INT A > pci 0000:00:00.2: PCI INT A: not connected > pci 0000:00:01.0: Adding to iommu group 0 > pci 0000:00:01.1: Adding to iommu group 1 > pci 0000:00:01.2: Adding to iommu group 2 > pci 0000:00:01.3: Adding to iommu group 3 > pci 0000:00:01.6: Adding to iommu group 4 > pci 0000:00:08.0: Adding to iommu group 5 > pci 0000:00:08.1: Adding to iommu group 6 > pci 0000:00:14.0: Adding to iommu group 7 > pci 0000:00:14.3: Adding to iommu group 7 > pci 0000:00:18.0: Adding to iommu group 8 > pci 0000:00:18.1: Adding to iommu group 8 > pci 0000:00:18.2: Adding to iommu group 8 > pci 0000:00:18.3: Adding to iommu group 8 > pci 0000:00:18.4: Adding to iommu group 8 > pci 0000:00:18.5: Adding to iommu group 8 > pci 0000:00:18.6: Adding to iommu group 8 > pci 0000:00:18.7: Adding to iommu group 8 > pci 0000:01:00.0: Adding to iommu group 9 > pci 0000:02:00.0: Adding to iommu group 10 > pci 0000:03:00.0: Adding to iommu group 11 > pci 0000:04:00.0: Adding to iommu group 12 > pci 0000:05:00.0: Adding to iommu group 13 > pci 0000:05:00.1: Adding to iommu group 14 > pci 0000:05:00.2: Adding to iommu group 14 > pci 0000:05:00.3: Adding to iommu group 14 > pci 0000:05:00.4: Adding to iommu group 14 > pci 0000:05:00.5: Adding to iommu group 14 > pci 0000:05:00.6: Adding to iommu group 14 > pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 > pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada): > PPR NX GT IA GA PC GA_vAPIC > AMD-Vi: Interrupt remapping enabled > AMD-Vi: Virtual APIC enabled > AMD-Vi: Lazy IO/TLB flushing enabled > amd_uncore: 4 amd_df counters detected > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Fiommu&data=04%7C01%7Csuravee.suthikulpanit%40amd.com%7C7c56640fcf24465050f008d8c145eba4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637471853347946970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5w2IiD7Cjsvk9qyiYC9eLmFaBIJLXdLQx4kg27LWycg%3D&reserved=0 > [-- Attachment #2: 0001-iommu-amd-Fix-performance-counter-initialization.patch --] [-- Type: text/plain, Size: 3646 bytes --] From c103d631285cf376420e7f7869837302f2ac38c0 Mon Sep 17 00:00:00 2001 From: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Date: Mon, 1 Feb 2021 18:38:26 -0600 Subject: [RFC PATCH] iommu/amd: Fix performance counter initialization Certain AMD platforms enable power gating feature for IOMMU PMC, which prevents the IOMMU driver from updating the counter while trying to validate the PMC functionality in the init_iommu_perf_ctr(). This results in disabling PMC support and the following error message: "AMD-Vi: Unable to write to IOMMU perf counter" To workaround this issue, disable power gating temporarily by programming the counter source to non-zero value while validating the counter, and restore the prior state afterward. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=201753 Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> --- NOTE: I have tested this patch only on certain platforms. It might need more testing coverage on other mobile and desktop platforms. Thank you, Suravee drivers/iommu/amd/init.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 83d8ab2aed9f..edb885625e47 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -254,6 +254,8 @@ static enum iommu_init_state init_state = IOMMU_START_STATE; static int amd_iommu_enable_interrupts(void); static int __init iommu_go_to_state(enum iommu_init_state state); static void init_device_table_dma(void); +static int iommu_pc_get_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, + u8 fxn, u64 *value, bool is_write); static bool amd_iommu_pre_enabled = true; @@ -1712,13 +1714,10 @@ static int __init init_iommu_all(struct acpi_table_header *table) return 0; } -static int iommu_pc_get_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, - u8 fxn, u64 *value, bool is_write); - -static void init_iommu_perf_ctr(struct amd_iommu *iommu) +static void __init init_iommu_perf_ctr(struct amd_iommu *iommu) { struct pci_dev *pdev = iommu->dev; - u64 val = 0xabcd, val2 = 0, save_reg = 0; + u64 val, val2 = 0, save_reg = 0, save_src = 0; if (!iommu_feature(iommu, FEATURE_PC)) return; @@ -1726,17 +1725,33 @@ static void init_iommu_perf_ctr(struct amd_iommu *iommu) amd_iommu_pc_present = true; /* save the value to restore, if writable */ - if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, false)) + if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, false) || + iommu_pc_get_set_reg(iommu, 0, 0, 8, &save_src, false)) + goto pc_false; + + /* + * Disable power gating by programing the performance counter + * source to 20 (i.e. counts the reads and writes from/to IOMMU + * Reserved Register [MMIO Offset 1FF8h] that are ignored.), + * which never get incremented during this init phase. + * (Note: The event is also deprecated.) + */ + val = 20; + if (iommu_pc_get_set_reg(iommu, 0, 0, 8, &val, true)) goto pc_false; /* Check if the performance counters can be written to */ + val = 0xabcd; if ((iommu_pc_get_set_reg(iommu, 0, 0, 0, &val, true)) || - (iommu_pc_get_set_reg(iommu, 0, 0, 0, &val2, false)) || - (val != val2)) + (iommu_pc_get_set_reg(iommu, 0, 0, 0, &val2, false))) goto pc_false; /* restore */ - if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, true)) + if (iommu_pc_get_set_reg(iommu, 0, 0, 0, &save_reg, true) || + iommu_pc_get_set_reg(iommu, 0, 0, 8, &save_src, true)) + goto pc_false; + + if (val != val2) goto pc_false; pci_info(pdev, "IOMMU performance counters supported\n"); -- 2.17.1 [-- Attachment #3: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: AMD-Vi: Unable to read/write to IOMMU perf counter 2021-02-02 5:54 ` Suravee Suthikulpanit @ 2021-02-04 6:25 ` Tj (Elloe Linux) 2021-02-06 4:02 ` Suravee Suthikulpanit 0 siblings, 1 reply; 6+ messages in thread From: Tj (Elloe Linux) @ 2021-02-04 6:25 UTC (permalink / raw) To: iommu On 02/02/2021 05:54, Suravee Suthikulpanit wrote: > Could you please try the attached patch to see if the problem still > persist. Tested on top of commit 61556703b610 doesn't appear to have solved the issue. Linux version 5.11.0-rc6+ (tj@elloe000) (gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubunt> Command line: BOOT_IMAGE=/vmlinuz-5.11.0-rc6+ root=/dev/mapper/ELLOE000-rootfs ro acpi_osi=! "acpi_osi=Windows 20> ... DMI: LENOVO 20NECTO1WW/20NECTO1WW, BIOS R11ET32W (1.12 ) 12/23/2019 ... AMD-Vi: ivrs, add hid:PNPD0040, uid:, rdevid:152 ... smpboot: CPU0: AMD Ryzen 7 3700U with Radeon Vega Mobile Gfx (family: 0x17, model: 0x18, stepping: 0x1) ... pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter. pci 0000:00:00.2: can't derive routing for PCI INT A pci 0000:00:00.2: PCI INT A: not connected pci 0000:00:01.0: Adding to iommu group 0 pci 0000:00:01.1: Adding to iommu group 1 ... pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada): PPR NX GT IA GA PC GA_vAPIC AMD-Vi: Interrupt remapping enabled AMD-Vi: Virtual APIC enabled AMD-Vi: Lazy IO/TLB flushing enabled amd_uncore: 4 amd_df counters detected _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: AMD-Vi: Unable to read/write to IOMMU perf counter 2021-02-04 6:25 ` Tj (Elloe Linux) @ 2021-02-06 4:02 ` Suravee Suthikulpanit 2021-02-07 17:18 ` Tj (Elloe Linux) 0 siblings, 1 reply; 6+ messages in thread From: Suravee Suthikulpanit @ 2021-02-06 4:02 UTC (permalink / raw) To: Tj (Elloe Linux), iommu Tj, I have posted RFCv3 in the BZ https://bugzilla.kernel.org/show_bug.cgi?id=201753. RFCv3 patch adds the logic to retry checking after 20msec wait for each retry loop since I have founded that certain platform takes about 10msec for the power gating to disable. Please give this a try to see if this works better on your platform. Thanks, Suravee On 2/4/21 1:25 PM, Tj (Elloe Linux) wrote: > On 02/02/2021 05:54, Suravee Suthikulpanit wrote: >> Could you please try the attached patch to see if the problem still >> persist. > > Tested on top of commit 61556703b610 doesn't appear to have solved the > issue. > > > > Linux version 5.11.0-rc6+ (tj@elloe000) (gcc (Ubuntu > 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubunt> > Command line: BOOT_IMAGE=/vmlinuz-5.11.0-rc6+ > root=/dev/mapper/ELLOE000-rootfs ro acpi_osi=! "acpi_osi=Windows 20> > ... > DMI: LENOVO 20NECTO1WW/20NECTO1WW, BIOS R11ET32W (1.12 ) 12/23/2019 > ... > AMD-Vi: ivrs, add hid:PNPD0040, uid:, rdevid:152 > ... > smpboot: CPU0: AMD Ryzen 7 3700U with Radeon Vega Mobile Gfx (family: > 0x17, model: 0x18, stepping: 0x1) > ... > pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter. > pci 0000:00:00.2: can't derive routing for PCI INT A > pci 0000:00:00.2: PCI INT A: not connected > pci 0000:00:01.0: Adding to iommu group 0 > pci 0000:00:01.1: Adding to iommu group 1 > ... > pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 > pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada): > PPR NX GT IA GA PC GA_vAPIC > AMD-Vi: Interrupt remapping enabled > AMD-Vi: Virtual APIC enabled > AMD-Vi: Lazy IO/TLB flushing enabled > amd_uncore: 4 amd_df counters detected > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: AMD-Vi: Unable to read/write to IOMMU perf counter 2021-02-06 4:02 ` Suravee Suthikulpanit @ 2021-02-07 17:18 ` Tj (Elloe Linux) 2021-02-08 11:57 ` Suravee Suthikulpanit 0 siblings, 1 reply; 6+ messages in thread From: Tj (Elloe Linux) @ 2021-02-07 17:18 UTC (permalink / raw) To: Suravee Suthikulpanit, iommu On 06/02/2021 04:02, Suravee Suthikulpanit wrote: > Tj, > > I have posted RFCv3 in the BZ > https://bugzilla.kernel.org/show_bug.cgi?id=201753. > > RFCv3 patch adds the logic to retry checking after 20msec wait for each > retry loop since I have founded that certain platform takes about 10msec > for the power gating to disable. Added on top of commit 825b5991a46e. kernel: Linux version 5.11.0-rc6+ (tj@elloe000) (gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #18 SMP PREEMPT Sun Feb 7 15:58:14 GMT 2021 kernel: Command line: BOOT_IMAGE=/vmlinuz-5.11.0-rc6+ root=/dev/mapper/ELLOE000-rootfs ro acpi_osi=! "acpi_osi=Windows 2016" systemd.unified_cgroup_hierarchy=1 nosplash ... kernel: DMI: LENOVO 20NECTO1WW/20NECTO1WW, BIOS R11ET32W (1.12 ) 12/23/2019 ... kernel: AMD-Vi: ivrs, add hid:PNPD0040, uid:, rdevid:152 ... kernel: Freeing initrd memory: 37592K kernel: pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported kernel: pci 0000:00:00.2: can't derive routing for PCI INT A kernel: pci 0000:00:00.2: PCI INT A: not connected ... kernel: pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 kernel: pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada): kernel: PPR NX GT IA GA PC GA_vAPIC kernel: AMD-Vi: Interrupt remapping enabled kernel: AMD-Vi: Virtual APIC enabled kernel: AMD-Vi: Lazy IO/TLB flushing enabled ... Looks to have solved the: kernel: AMD-Vi: Unable to read/write to IOMMU perf counter. Would this be in any way related to the following from the same device: kernel: pci 0000:00:00.2: can't derive routing for PCI INT A kernel: pci 0000:00:00.2: PCI INT A: not connected _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: AMD-Vi: Unable to read/write to IOMMU perf counter 2021-02-07 17:18 ` Tj (Elloe Linux) @ 2021-02-08 11:57 ` Suravee Suthikulpanit 0 siblings, 0 replies; 6+ messages in thread From: Suravee Suthikulpanit @ 2021-02-08 11:57 UTC (permalink / raw) To: Tj (Elloe Linux), iommu TJ, Thanks for testing. I will submit this change upstream w/ you as Tested-by. On 2/8/21 12:18 AM, Tj (Elloe Linux) wrote: > On 06/02/2021 04:02, Suravee Suthikulpanit wrote: > Would this be in any way related to the following from the same device: > > kernel: pci 0000:00:00.2: can't derive routing for PCI INT A > kernel: pci 0000:00:00.2: PCI INT A: not connected This is not related, but should not cause issues. Thanks, Suravee _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-02-08 11:57 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-01-25 9:24 AMD-Vi: Unable to read/write to IOMMU perf counter Tj (Elloe Linux) 2021-02-02 5:54 ` Suravee Suthikulpanit 2021-02-04 6:25 ` Tj (Elloe Linux) 2021-02-06 4:02 ` Suravee Suthikulpanit 2021-02-07 17:18 ` Tj (Elloe Linux) 2021-02-08 11:57 ` Suravee Suthikulpanit
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.