* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems [not found] <81b4f4bf-b721-4b5b-8ddf-d040865f02b2@amd.com> @ 2026-10-13 22:40 ` Bjorn Helgaas 2026-10-14 0:43 ` Borislav Petkov ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Bjorn Helgaas @ 2026-10-13 22:40 UTC (permalink / raw) To: Mario Limonciello Cc: Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Hans de Goede, Kai-Heng Feng, Andy Shevchenko, linux-acpi [+cc Hans, Kai-Heng, Andy, linux-acpi] On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: > On 10/13/26 12:26, Ilpo Järvinen wrote: > > On Tue, 13 Oct 2026, Mario Limonciello wrote: > > > On 10/13/26 12:00, Mario Limonciello wrote: > > > > On 10/13/26 08:09, Ilpo Järvinen wrote: > > > > > While testing the resource placement changes, tests hit a case where > > > > > igb fails to probe when BAR 0 is placed at 0x9c000000: > > > > > > > > > > 90000000-9cffffff : PCI Bus 0000:a0 > > > > > - 90000000-902fffff : PCI Bus 0000:a1 > > > > > - 90000000-900fffff : 0000:a1:00.0 > > > > > - 90000000-900fffff : igb > > > > > - 90100000-901fffff : 0000:a1:00.0 > > > > > - 90200000-90203fff : 0000:a1:00.0 > > > > > - 90200000-90203fff : igb > > > > > + 9be00000-9c0fffff : PCI Bus 0000:a1 > > > > > + 9be00000-9befffff : 0000:a1:00.0 > > > > > + 9bf00000-9bf03fff : 0000:a1:00.0 > > > > > + 9c000000-9c0fffff : 0000:a1:00.0 > > > > > 9c100000-9c17ffff : amd_iommu > > > > > 9c180000-9c1803ff : IOAPIC 8 > > > > > > > > > > - Region 0: Memory at 90000000 (32-bit, non-prefetchable) > > > > > [size=1M] > > > > > - Region 3: Memory at 90200000 (32-bit, non-prefetchable) > > > > > [size=16K] > > > > > - Expansion ROM at 90100000 [disabled] [size=1M] > > > > > + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) > > > > > [size=1M] > > > > > + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) > > > > > [size=16K] > > > > > + Expansion ROM at 9be00000 [disabled] [size=1M] > > > > > > > > > > igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost > > > > > ------------[ cut here ]------------ > > > > > igb: Failed to read reg 0x18! > > > > > WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at > > > > > igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 > > > > > ... > > > > > igb_get_invariants_82575+0xff/0xf00 [igb] > > > > > igb_probe+0x3c8/0x1190 [igb] > > > > > local_pci_probe+0x3b/0x80 > > > > > > > > > > It turns out there is a 64kB iomem black hole at 9c000000 that returns > > > > > ~0 and this is where igb's BAR 0 resides. If another BAR of the same > > > > > card is placed into that address, it is similarly black holed. > > > > > > > > > > Mark the problematic 64kB range reserved using a quirk bound to the > > > > > bridge found in the problematic system. > > > > > > > > > > Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 > > > > > Suggested-by: Mario Limonciello <mario.limonciello@amd.com> > > > > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> > > > > > --- > > > > > Cc: Borislav Petkov <bp@alien8.de> > > > > > > > > > > As an alternative to this, I've also tested reverting the commit > > > > > 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and > > > > > that also resolves the issue by retaining "9c000000-9cffffff : Reserved" > > > > > entry in the resource tree. > > > > > > > > > > --- > > > > > arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ > > > > > 1 file changed, 36 insertions(+) > > > > > > > > > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > > > > > index b301c6c8df75..cc463bf492b6 100644 > > > > > --- a/arch/x86/pci/fixup.c > > > > > +++ b/arch/x86/pci/fixup.c > > > > > @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, > > > > > pci_amd_enable_64bit_bar); > > > > > DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, > > > > > pci_amd_enable_64bit_bar); > > > > > DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, > > > > > pci_amd_enable_64bit_bar); > > > > > +/* > > > > > + * TODO: Remove when/if root cause is found. > > > > > + * > > > > > + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 > > > > > + * address for which all reads return ~0 blocking an overlapping BAR from > > > > > + * working. > > > > > + * > > > > > + * Work around the problem by reserving the space prior to making any > > > > > iomem > > > > > + * allocations that could overlap with the black hole. > > > > > + */ > > > > > +static struct resource genoa_black_hole = > > > > > DEFINE_RES_MEM_NAMED(0x9c000000, > > > > > + SZ_64K, > > > > > + "reserved"); > > > > > + > > > > > +static void genoa_iomem_black_hole(struct pci_dev *dev) > > > > > +{ > > > > > + struct resource *r; > > > > > + > > > > > + pci_bus_for_each_resource(dev->bus, r) { > > > > > + if (!r || !r->flags || !resource_assigned(r)) > > > > > + continue; > > > > > + > > > > > + if (!__resource_contains_unbound(r, &genoa_black_hole)) > > > > > + continue; > > > > > + > > > > > + if (resource_assigned(&genoa_black_hole)) > > > > > + pci_dbg(dev, "iomem black hole workaround already > > > > > applied\n"); > > > > > + else if (!insert_resource(r, &genoa_black_hole)) > > > > > + pci_info(dev, "iomem black hole workaround enabled\n"); > > > > > + else > > > > > + pci_dbg(dev, "iomem black hole workaround add failed\n"); > > > > > + return; > > > > > + } > > > > > +} > > > > > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, > > > > > genoa_iomem_black_hole); > > > > > + > > > > > #define RS690_LOWER_TOP_OF_DRAM2 0x30 > > > > > #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 > > > > > #define RS690_UPPER_TOP_OF_DRAM2 0x31 > > > > > > > > I guess it's good we know the root cause is 07eab0901ede. If we're going > > > > down the path of making holes in that we may well put the hole in > > > > efi_remove_e820_mmio() like this (attached). > > > > > > I don't really like that though. If Genoa is affected I would expect plenty > > > of other machines are too they just haven't been found yet. > > > > That's what I'm worried as well. My own testing can only cover very little > > ground here. > > > > > Can we roll back 07eab0901ede and add quirks for the busted machines instead? > > I had an agent dig through all the reports linked in 07eab0901ede and > drafted up what should be the remaining quirks for a revert. > > https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess > > Bjorn, thoughts on this? I'll send it out if that's the way we want > to go (or Ilpo feel free to subsume into your series as it's only > really exposed by your series at the moment) I would LOVE it if we can use those patches because it's way less magic than 07eab0901ede is. But I'm frankly terrified to touch it because I'm skeptical that two more quirks are enough. I just expect that whatever strategy BIOS uses to build these maps is likely to be used on many models. I do hate 07eab0901ede though, so I would love to see it gone. I do think the description [1] is overly specific because AFAIK there's nothing that connects this to AMD EPYC 9354 or igb other than the fact that it was seen there. It could happen to any PCI device on any system with similar memory maps. [1] https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/commit/?h=superm1/genoa-mess&id=0a49c704477153d6597d34b3c6e3141007ff058c ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-13 22:40 ` [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems Bjorn Helgaas @ 2026-10-14 0:43 ` Borislav Petkov 2026-10-14 2:41 ` Mario Limonciello 2026-10-14 9:34 ` Hans de Goede 2026-10-14 10:46 ` Andy Shevchenko 2 siblings, 1 reply; 11+ messages in thread From: Borislav Petkov @ 2026-10-14 0:43 UTC (permalink / raw) To: Bjorn Helgaas, Mario Limonciello Cc: Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Hans de Goede, Kai-Heng Feng, Andy Shevchenko, linux-acpi On Tue, Oct 13, 2026 at 05:40:10PM -0500, Bjorn Helgaas wrote: > But I'm frankly terrified to touch it because I'm skeptical that two > more quirks are enough. I just expect that whatever strategy BIOS > uses to build these maps is likely to be used on many models. I do Do we have an explanation why BIOS is doing that shit? Mario? Btw, Mario, you should replace "Genoa" with "Zen4" everywhere so that it is more clear to people. Thx. -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 0:43 ` Borislav Petkov @ 2026-10-14 2:41 ` Mario Limonciello 2026-10-14 10:49 ` Andy Shevchenko 0 siblings, 1 reply; 11+ messages in thread From: Mario Limonciello @ 2026-10-14 2:41 UTC (permalink / raw) To: Borislav Petkov, Bjorn Helgaas Cc: Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Hans de Goede, Kai-Heng Feng, Andy Shevchenko, linux-acpi On 10/13/26 19:43, Borislav Petkov wrote: > On Tue, Oct 13, 2026 at 05:40:10PM -0500, Bjorn Helgaas wrote: >> But I'm frankly terrified to touch it because I'm skeptical that two >> more quirks are enough. I just expect that whatever strategy BIOS >> uses to build these maps is likely to be used on many models. I do > > Do we have an explanation why BIOS is doing that shit? > > Mario? If I was to guess; because Windows allocates MMIO space differently it hasn't been a problem. Which begs the question; does Ilpo's series improve things enough that we wouldn't have these problems either? Maybe we should just tail my patches on Ilpos' series and then land them linux-next the very beginning of next cycle and beg everyone who was affected to test linux-next. > > Btw, Mario, you should replace "Genoa" with "Zen4" everywhere so that it is > more clear to people. > > Thx. > Ack; if I send mine out on their own I'll adjust. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 2:41 ` Mario Limonciello @ 2026-10-14 10:49 ` Andy Shevchenko 0 siblings, 0 replies; 11+ messages in thread From: Andy Shevchenko @ 2026-10-14 10:49 UTC (permalink / raw) To: Mario Limonciello Cc: Borislav Petkov, Bjorn Helgaas, Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Hans de Goede, Kai-Heng Feng, linux-acpi On Tue, Oct 13, 2026 at 09:41:48PM -0500, Mario Limonciello wrote: > On 10/13/26 19:43, Borislav Petkov wrote: > > On Tue, Oct 13, 2026 at 05:40:10PM -0500, Bjorn Helgaas wrote: > > > But I'm frankly terrified to touch it because I'm skeptical that two > > > more quirks are enough. I just expect that whatever strategy BIOS > > > uses to build these maps is likely to be used on many models. I do > > > > Do we have an explanation why BIOS is doing that shit? > > > > Mario? > > If I was to guess; because Windows allocates MMIO space differently it > hasn't been a problem. Oh, I can confirm this part as I noticed the issue on 64-bit addresses for Intel LPSS at some point exactly due to different ways of allocating space for the devices for PCI. Id est one is goes upwards (from the start to the end of the region), while the other downwards (from the end to the start of the region). > Which begs the question; does Ilpo's series improve things enough that we > wouldn't have these problems either? > > Maybe we should just tail my patches on Ilpos' series and then land them > linux-next the very beginning of next cycle and beg everyone who was > affected to test linux-next. > > > Btw, Mario, you should replace "Genoa" with "Zen4" everywhere so that it is > > more clear to people. > > Ack; if I send mine out on their own I'll adjust. -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-13 22:40 ` [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems Bjorn Helgaas 2026-10-14 0:43 ` Borislav Petkov @ 2026-10-14 9:34 ` Hans de Goede 2026-10-14 10:51 ` Andy Shevchenko 2026-10-14 20:37 ` Mario Limonciello 2026-10-14 10:46 ` Andy Shevchenko 2 siblings, 2 replies; 11+ messages in thread From: Hans de Goede @ 2026-10-14 9:34 UTC (permalink / raw) To: Bjorn Helgaas, Mario Limonciello Cc: Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Kai-Heng Feng, Andy Shevchenko, linux-acpi Hi, On 14-Oct-26 00:40, Bjorn Helgaas wrote: > [+cc Hans, Kai-Heng, Andy, linux-acpi] > > On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: >> On 10/13/26 12:26, Ilpo Järvinen wrote: >>> On Tue, 13 Oct 2026, Mario Limonciello wrote: >>>> On 10/13/26 12:00, Mario Limonciello wrote: >>>>> On 10/13/26 08:09, Ilpo Järvinen wrote: >>>>>> While testing the resource placement changes, tests hit a case where >>>>>> igb fails to probe when BAR 0 is placed at 0x9c000000: >>>>>> >>>>>> 90000000-9cffffff : PCI Bus 0000:a0 >>>>>> - 90000000-902fffff : PCI Bus 0000:a1 >>>>>> - 90000000-900fffff : 0000:a1:00.0 >>>>>> - 90000000-900fffff : igb >>>>>> - 90100000-901fffff : 0000:a1:00.0 >>>>>> - 90200000-90203fff : 0000:a1:00.0 >>>>>> - 90200000-90203fff : igb >>>>>> + 9be00000-9c0fffff : PCI Bus 0000:a1 >>>>>> + 9be00000-9befffff : 0000:a1:00.0 >>>>>> + 9bf00000-9bf03fff : 0000:a1:00.0 >>>>>> + 9c000000-9c0fffff : 0000:a1:00.0 >>>>>> 9c100000-9c17ffff : amd_iommu >>>>>> 9c180000-9c1803ff : IOAPIC 8 >>>>>> >>>>>> - Region 0: Memory at 90000000 (32-bit, non-prefetchable) >>>>>> [size=1M] >>>>>> - Region 3: Memory at 90200000 (32-bit, non-prefetchable) >>>>>> [size=16K] >>>>>> - Expansion ROM at 90100000 [disabled] [size=1M] >>>>>> + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) >>>>>> [size=1M] >>>>>> + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) >>>>>> [size=16K] >>>>>> + Expansion ROM at 9be00000 [disabled] [size=1M] >>>>>> >>>>>> igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost >>>>>> ------------[ cut here ]------------ >>>>>> igb: Failed to read reg 0x18! >>>>>> WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at >>>>>> igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 >>>>>> ... >>>>>> igb_get_invariants_82575+0xff/0xf00 [igb] >>>>>> igb_probe+0x3c8/0x1190 [igb] >>>>>> local_pci_probe+0x3b/0x80 >>>>>> >>>>>> It turns out there is a 64kB iomem black hole at 9c000000 that returns >>>>>> ~0 and this is where igb's BAR 0 resides. If another BAR of the same >>>>>> card is placed into that address, it is similarly black holed. >>>>>> >>>>>> Mark the problematic 64kB range reserved using a quirk bound to the >>>>>> bridge found in the problematic system. >>>>>> >>>>>> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 >>>>>> Suggested-by: Mario Limonciello <mario.limonciello@amd.com> >>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> >>>>>> --- >>>>>> Cc: Borislav Petkov <bp@alien8.de> >>>>>> >>>>>> As an alternative to this, I've also tested reverting the commit >>>>>> 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and >>>>>> that also resolves the issue by retaining "9c000000-9cffffff : Reserved" >>>>>> entry in the resource tree. >>>>>> >>>>>> --- >>>>>> arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ >>>>>> 1 file changed, 36 insertions(+) >>>>>> >>>>>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c >>>>>> index b301c6c8df75..cc463bf492b6 100644 >>>>>> --- a/arch/x86/pci/fixup.c >>>>>> +++ b/arch/x86/pci/fixup.c >>>>>> @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, >>>>>> pci_amd_enable_64bit_bar); >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, >>>>>> pci_amd_enable_64bit_bar); >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, >>>>>> pci_amd_enable_64bit_bar); >>>>>> +/* >>>>>> + * TODO: Remove when/if root cause is found. >>>>>> + * >>>>>> + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 >>>>>> + * address for which all reads return ~0 blocking an overlapping BAR from >>>>>> + * working. >>>>>> + * >>>>>> + * Work around the problem by reserving the space prior to making any >>>>>> iomem >>>>>> + * allocations that could overlap with the black hole. >>>>>> + */ >>>>>> +static struct resource genoa_black_hole = >>>>>> DEFINE_RES_MEM_NAMED(0x9c000000, >>>>>> + SZ_64K, >>>>>> + "reserved"); >>>>>> + >>>>>> +static void genoa_iomem_black_hole(struct pci_dev *dev) >>>>>> +{ >>>>>> + struct resource *r; >>>>>> + >>>>>> + pci_bus_for_each_resource(dev->bus, r) { >>>>>> + if (!r || !r->flags || !resource_assigned(r)) >>>>>> + continue; >>>>>> + >>>>>> + if (!__resource_contains_unbound(r, &genoa_black_hole)) >>>>>> + continue; >>>>>> + >>>>>> + if (resource_assigned(&genoa_black_hole)) >>>>>> + pci_dbg(dev, "iomem black hole workaround already >>>>>> applied\n"); >>>>>> + else if (!insert_resource(r, &genoa_black_hole)) >>>>>> + pci_info(dev, "iomem black hole workaround enabled\n"); >>>>>> + else >>>>>> + pci_dbg(dev, "iomem black hole workaround add failed\n"); >>>>>> + return; >>>>>> + } >>>>>> +} >>>>>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, >>>>>> genoa_iomem_black_hole); >>>>>> + >>>>>> #define RS690_LOWER_TOP_OF_DRAM2 0x30 >>>>>> #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 >>>>>> #define RS690_UPPER_TOP_OF_DRAM2 0x31 >>>>> >>>>> I guess it's good we know the root cause is 07eab0901ede. If we're going >>>>> down the path of making holes in that we may well put the hole in >>>>> efi_remove_e820_mmio() like this (attached). >>>> >>>> I don't really like that though. If Genoa is affected I would expect plenty >>>> of other machines are too they just haven't been found yet. >>> >>> That's what I'm worried as well. My own testing can only cover very little >>> ground here. >>> >>>> Can we roll back 07eab0901ede and add quirks for the busted machines instead? >> >> I had an agent dig through all the reports linked in 07eab0901ede and >> drafted up what should be the remaining quirks for a revert. >> >> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess >> >> Bjorn, thoughts on this? I'll send it out if that's the way we want >> to go (or Ilpo feel free to subsume into your series as it's only >> really exposed by your series at the moment) > > I would LOVE it if we can use those patches because it's way less > magic than 07eab0901ede is. > > But I'm frankly terrified to touch it because I'm skeptical that two > more quirks are enough. I just expect that whatever strategy BIOS > uses to build these maps is likely to be used on many models. I do > hate 07eab0901ede though, so I would love to see it gone. It has been about 4 years since we came up with 07eab0901ede IIRC we had a huge email thread with a whole bunch of attempts at fixing the issue in a generic manner. IIRC some of those attempts tried to identify the exact problematic issue the quirks try to fix where there is an EFI MMIO reservation covering the entire bridge window. Maybe we can dig through the old thread and see if with the knowledge we have now one of the previous attempts of fixing it from the thread is better? I agree that needing to go back to quirks is likely going to be painful and break a whole bunch of other systems. IIRC 07eab0901ede fixed a whole bunch of laptops having issues around the time not just the Clevo NL4XLU and Lenovo Yoga C940. But this info might be spread over various bugzilla entries, etc. + some models where likely never reported but just fixed in time for users to not report things. I also wonder if given we only have this magic black-hole on geneo/zen4 if it would not be better to go with Mario's original approach here ? Esp. since Mario's quirk is chipset based where as the other problems seem to be model/product based so that is 1 quirk vs many quirks. Regards, Hans ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 9:34 ` Hans de Goede @ 2026-10-14 10:51 ` Andy Shevchenko 2026-10-14 11:17 ` Ilpo Järvinen 2026-10-14 20:37 ` Mario Limonciello 1 sibling, 1 reply; 11+ messages in thread From: Andy Shevchenko @ 2026-10-14 10:51 UTC (permalink / raw) To: Hans de Goede Cc: Bjorn Helgaas, Mario Limonciello, Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Kai-Heng Feng, linux-acpi On Wed, Oct 14, 2026 at 11:34:44AM +0200, Hans de Goede wrote: > On 14-Oct-26 00:40, Bjorn Helgaas wrote: > > [+cc Hans, Kai-Heng, Andy, linux-acpi] > > > > On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: > >> On 10/13/26 12:26, Ilpo Järvinen wrote: > >>> On Tue, 13 Oct 2026, Mario Limonciello wrote: > >>>> On 10/13/26 12:00, Mario Limonciello wrote: > >>>>> On 10/13/26 08:09, Ilpo Järvinen wrote: > >>>>>> While testing the resource placement changes, tests hit a case where > >>>>>> igb fails to probe when BAR 0 is placed at 0x9c000000: > >>>>>> > >>>>>> 90000000-9cffffff : PCI Bus 0000:a0 > >>>>>> - 90000000-902fffff : PCI Bus 0000:a1 > >>>>>> - 90000000-900fffff : 0000:a1:00.0 > >>>>>> - 90000000-900fffff : igb > >>>>>> - 90100000-901fffff : 0000:a1:00.0 > >>>>>> - 90200000-90203fff : 0000:a1:00.0 > >>>>>> - 90200000-90203fff : igb > >>>>>> + 9be00000-9c0fffff : PCI Bus 0000:a1 > >>>>>> + 9be00000-9befffff : 0000:a1:00.0 > >>>>>> + 9bf00000-9bf03fff : 0000:a1:00.0 > >>>>>> + 9c000000-9c0fffff : 0000:a1:00.0 > >>>>>> 9c100000-9c17ffff : amd_iommu > >>>>>> 9c180000-9c1803ff : IOAPIC 8 > >>>>>> > >>>>>> - Region 0: Memory at 90000000 (32-bit, non-prefetchable) > >>>>>> [size=1M] > >>>>>> - Region 3: Memory at 90200000 (32-bit, non-prefetchable) > >>>>>> [size=16K] > >>>>>> - Expansion ROM at 90100000 [disabled] [size=1M] > >>>>>> + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) > >>>>>> [size=1M] > >>>>>> + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) > >>>>>> [size=16K] > >>>>>> + Expansion ROM at 9be00000 [disabled] [size=1M] > >>>>>> > >>>>>> igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost > >>>>>> ------------[ cut here ]------------ > >>>>>> igb: Failed to read reg 0x18! > >>>>>> WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at > >>>>>> igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 > >>>>>> ... > >>>>>> igb_get_invariants_82575+0xff/0xf00 [igb] > >>>>>> igb_probe+0x3c8/0x1190 [igb] > >>>>>> local_pci_probe+0x3b/0x80 > >>>>>> > >>>>>> It turns out there is a 64kB iomem black hole at 9c000000 that returns > >>>>>> ~0 and this is where igb's BAR 0 resides. If another BAR of the same > >>>>>> card is placed into that address, it is similarly black holed. > >>>>>> > >>>>>> Mark the problematic 64kB range reserved using a quirk bound to the > >>>>>> bridge found in the problematic system. > >>>>>> > >>>>>> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 > >>>>>> Suggested-by: Mario Limonciello <mario.limonciello@amd.com> > >>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> > >>>>>> --- > >>>>>> Cc: Borislav Petkov <bp@alien8.de> > >>>>>> > >>>>>> As an alternative to this, I've also tested reverting the commit > >>>>>> 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and > >>>>>> that also resolves the issue by retaining "9c000000-9cffffff : Reserved" > >>>>>> entry in the resource tree. > >>>>>> > >>>>>> --- > >>>>>> arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ > >>>>>> 1 file changed, 36 insertions(+) > >>>>>> > >>>>>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > >>>>>> index b301c6c8df75..cc463bf492b6 100644 > >>>>>> --- a/arch/x86/pci/fixup.c > >>>>>> +++ b/arch/x86/pci/fixup.c > >>>>>> @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, > >>>>>> pci_amd_enable_64bit_bar); > >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, > >>>>>> pci_amd_enable_64bit_bar); > >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, > >>>>>> pci_amd_enable_64bit_bar); > >>>>>> +/* > >>>>>> + * TODO: Remove when/if root cause is found. > >>>>>> + * > >>>>>> + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 > >>>>>> + * address for which all reads return ~0 blocking an overlapping BAR from > >>>>>> + * working. > >>>>>> + * > >>>>>> + * Work around the problem by reserving the space prior to making any > >>>>>> iomem > >>>>>> + * allocations that could overlap with the black hole. > >>>>>> + */ > >>>>>> +static struct resource genoa_black_hole = > >>>>>> DEFINE_RES_MEM_NAMED(0x9c000000, > >>>>>> + SZ_64K, > >>>>>> + "reserved"); > >>>>>> + > >>>>>> +static void genoa_iomem_black_hole(struct pci_dev *dev) > >>>>>> +{ > >>>>>> + struct resource *r; > >>>>>> + > >>>>>> + pci_bus_for_each_resource(dev->bus, r) { > >>>>>> + if (!r || !r->flags || !resource_assigned(r)) > >>>>>> + continue; > >>>>>> + > >>>>>> + if (!__resource_contains_unbound(r, &genoa_black_hole)) > >>>>>> + continue; > >>>>>> + > >>>>>> + if (resource_assigned(&genoa_black_hole)) > >>>>>> + pci_dbg(dev, "iomem black hole workaround already > >>>>>> applied\n"); > >>>>>> + else if (!insert_resource(r, &genoa_black_hole)) > >>>>>> + pci_info(dev, "iomem black hole workaround enabled\n"); > >>>>>> + else > >>>>>> + pci_dbg(dev, "iomem black hole workaround add failed\n"); > >>>>>> + return; > >>>>>> + } > >>>>>> +} > >>>>>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, > >>>>>> genoa_iomem_black_hole); > >>>>>> + > >>>>>> #define RS690_LOWER_TOP_OF_DRAM2 0x30 > >>>>>> #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 > >>>>>> #define RS690_UPPER_TOP_OF_DRAM2 0x31 > >>>>> > >>>>> I guess it's good we know the root cause is 07eab0901ede. If we're going > >>>>> down the path of making holes in that we may well put the hole in > >>>>> efi_remove_e820_mmio() like this (attached). > >>>> > >>>> I don't really like that though. If Genoa is affected I would expect plenty > >>>> of other machines are too they just haven't been found yet. > >>> > >>> That's what I'm worried as well. My own testing can only cover very little > >>> ground here. > >>> > >>>> Can we roll back 07eab0901ede and add quirks for the busted machines instead? > >> > >> I had an agent dig through all the reports linked in 07eab0901ede and > >> drafted up what should be the remaining quirks for a revert. > >> > >> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess > >> > >> Bjorn, thoughts on this? I'll send it out if that's the way we want > >> to go (or Ilpo feel free to subsume into your series as it's only > >> really exposed by your series at the moment) > > > > I would LOVE it if we can use those patches because it's way less > > magic than 07eab0901ede is. > > > > But I'm frankly terrified to touch it because I'm skeptical that two > > more quirks are enough. I just expect that whatever strategy BIOS > > uses to build these maps is likely to be used on many models. I do > > hate 07eab0901ede though, so I would love to see it gone. > > It has been about 4 years since we came up with 07eab0901ede IIRC we > had a huge email thread with a whole bunch of attempts at fixing > the issue in a generic manner. > > IIRC some of those attempts tried to identify the exact problematic > issue the quirks try to fix where there is an EFI MMIO reservation > covering the entire bridge window. > > Maybe we can dig through the old thread and see if with the knowledge > we have now one of the previous attempts of fixing it from the thread > is better? > > I agree that needing to go back to quirks is likely going to be painful > and break a whole bunch of other systems. IIRC 07eab0901ede fixed > a whole bunch of laptops having issues around the time not just > the Clevo NL4XLU and Lenovo Yoga C940. But this info might be spread > over various bugzilla entries, etc. + some models where likely never > reported but just fixed in time for users to not report things. > > I also wonder if given we only have this magic black-hole on geneo/zen4 > if it would not be better to go with Mario's original approach here ? > Esp. since Mario's quirk is chipset based where as the other problems > seem to be model/product based so that is 1 quirk vs many quirks. Can we simply change the algo of allocating resources to follow what Windows does? -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 10:51 ` Andy Shevchenko @ 2026-10-14 11:17 ` Ilpo Järvinen 2026-10-14 13:32 ` Andy Shevchenko 0 siblings, 1 reply; 11+ messages in thread From: Ilpo Järvinen @ 2026-10-14 11:17 UTC (permalink / raw) To: Andy Shevchenko Cc: Hans de Goede, Bjorn Helgaas, Mario Limonciello, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Kai-Heng Feng, linux-acpi [-- Attachment #1: Type: text/plain, Size: 10369 bytes --] On Wed, 14 Oct 2026, Andy Shevchenko wrote: > On Wed, Oct 14, 2026 at 11:34:44AM +0200, Hans de Goede wrote: > > On 14-Oct-26 00:40, Bjorn Helgaas wrote: > > > [+cc Hans, Kai-Heng, Andy, linux-acpi] > > > > > > On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: > > >> On 10/13/26 12:26, Ilpo Järvinen wrote: > > >>> On Tue, 13 Oct 2026, Mario Limonciello wrote: > > >>>> On 10/13/26 12:00, Mario Limonciello wrote: > > >>>>> On 10/13/26 08:09, Ilpo Järvinen wrote: > > >>>>>> While testing the resource placement changes, tests hit a case where > > >>>>>> igb fails to probe when BAR 0 is placed at 0x9c000000: > > >>>>>> > > >>>>>> 90000000-9cffffff : PCI Bus 0000:a0 > > >>>>>> - 90000000-902fffff : PCI Bus 0000:a1 > > >>>>>> - 90000000-900fffff : 0000:a1:00.0 > > >>>>>> - 90000000-900fffff : igb > > >>>>>> - 90100000-901fffff : 0000:a1:00.0 > > >>>>>> - 90200000-90203fff : 0000:a1:00.0 > > >>>>>> - 90200000-90203fff : igb > > >>>>>> + 9be00000-9c0fffff : PCI Bus 0000:a1 > > >>>>>> + 9be00000-9befffff : 0000:a1:00.0 > > >>>>>> + 9bf00000-9bf03fff : 0000:a1:00.0 > > >>>>>> + 9c000000-9c0fffff : 0000:a1:00.0 > > >>>>>> 9c100000-9c17ffff : amd_iommu > > >>>>>> 9c180000-9c1803ff : IOAPIC 8 > > >>>>>> > > >>>>>> - Region 0: Memory at 90000000 (32-bit, non-prefetchable) > > >>>>>> [size=1M] > > >>>>>> - Region 3: Memory at 90200000 (32-bit, non-prefetchable) > > >>>>>> [size=16K] > > >>>>>> - Expansion ROM at 90100000 [disabled] [size=1M] > > >>>>>> + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) > > >>>>>> [size=1M] > > >>>>>> + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) > > >>>>>> [size=16K] > > >>>>>> + Expansion ROM at 9be00000 [disabled] [size=1M] > > >>>>>> > > >>>>>> igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost > > >>>>>> ------------[ cut here ]------------ > > >>>>>> igb: Failed to read reg 0x18! > > >>>>>> WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at > > >>>>>> igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 > > >>>>>> ... > > >>>>>> igb_get_invariants_82575+0xff/0xf00 [igb] > > >>>>>> igb_probe+0x3c8/0x1190 [igb] > > >>>>>> local_pci_probe+0x3b/0x80 > > >>>>>> > > >>>>>> It turns out there is a 64kB iomem black hole at 9c000000 that returns > > >>>>>> ~0 and this is where igb's BAR 0 resides. If another BAR of the same > > >>>>>> card is placed into that address, it is similarly black holed. > > >>>>>> > > >>>>>> Mark the problematic 64kB range reserved using a quirk bound to the > > >>>>>> bridge found in the problematic system. > > >>>>>> > > >>>>>> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 > > >>>>>> Suggested-by: Mario Limonciello <mario.limonciello@amd.com> > > >>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> > > >>>>>> --- > > >>>>>> Cc: Borislav Petkov <bp@alien8.de> > > >>>>>> > > >>>>>> As an alternative to this, I've also tested reverting the commit > > >>>>>> 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and > > >>>>>> that also resolves the issue by retaining "9c000000-9cffffff : Reserved" > > >>>>>> entry in the resource tree. > > >>>>>> > > >>>>>> --- > > >>>>>> arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ > > >>>>>> 1 file changed, 36 insertions(+) > > >>>>>> > > >>>>>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > > >>>>>> index b301c6c8df75..cc463bf492b6 100644 > > >>>>>> --- a/arch/x86/pci/fixup.c > > >>>>>> +++ b/arch/x86/pci/fixup.c > > >>>>>> @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, > > >>>>>> pci_amd_enable_64bit_bar); > > >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, > > >>>>>> pci_amd_enable_64bit_bar); > > >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, > > >>>>>> pci_amd_enable_64bit_bar); > > >>>>>> +/* > > >>>>>> + * TODO: Remove when/if root cause is found. > > >>>>>> + * > > >>>>>> + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 > > >>>>>> + * address for which all reads return ~0 blocking an overlapping BAR from > > >>>>>> + * working. > > >>>>>> + * > > >>>>>> + * Work around the problem by reserving the space prior to making any > > >>>>>> iomem > > >>>>>> + * allocations that could overlap with the black hole. > > >>>>>> + */ > > >>>>>> +static struct resource genoa_black_hole = > > >>>>>> DEFINE_RES_MEM_NAMED(0x9c000000, > > >>>>>> + SZ_64K, > > >>>>>> + "reserved"); > > >>>>>> + > > >>>>>> +static void genoa_iomem_black_hole(struct pci_dev *dev) > > >>>>>> +{ > > >>>>>> + struct resource *r; > > >>>>>> + > > >>>>>> + pci_bus_for_each_resource(dev->bus, r) { > > >>>>>> + if (!r || !r->flags || !resource_assigned(r)) > > >>>>>> + continue; > > >>>>>> + > > >>>>>> + if (!__resource_contains_unbound(r, &genoa_black_hole)) > > >>>>>> + continue; > > >>>>>> + > > >>>>>> + if (resource_assigned(&genoa_black_hole)) > > >>>>>> + pci_dbg(dev, "iomem black hole workaround already > > >>>>>> applied\n"); > > >>>>>> + else if (!insert_resource(r, &genoa_black_hole)) > > >>>>>> + pci_info(dev, "iomem black hole workaround enabled\n"); > > >>>>>> + else > > >>>>>> + pci_dbg(dev, "iomem black hole workaround add failed\n"); > > >>>>>> + return; > > >>>>>> + } > > >>>>>> +} > > >>>>>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, > > >>>>>> genoa_iomem_black_hole); > > >>>>>> + > > >>>>>> #define RS690_LOWER_TOP_OF_DRAM2 0x30 > > >>>>>> #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 > > >>>>>> #define RS690_UPPER_TOP_OF_DRAM2 0x31 > > >>>>> > > >>>>> I guess it's good we know the root cause is 07eab0901ede. If we're going > > >>>>> down the path of making holes in that we may well put the hole in > > >>>>> efi_remove_e820_mmio() like this (attached). > > >>>> > > >>>> I don't really like that though. If Genoa is affected I would expect plenty > > >>>> of other machines are too they just haven't been found yet. > > >>> > > >>> That's what I'm worried as well. My own testing can only cover very little > > >>> ground here. > > >>> > > >>>> Can we roll back 07eab0901ede and add quirks for the busted machines instead? > > >> > > >> I had an agent dig through all the reports linked in 07eab0901ede and > > >> drafted up what should be the remaining quirks for a revert. > > >> > > >> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess > > >> > > >> Bjorn, thoughts on this? I'll send it out if that's the way we want > > >> to go (or Ilpo feel free to subsume into your series as it's only > > >> really exposed by your series at the moment) > > > > > > I would LOVE it if we can use those patches because it's way less > > > magic than 07eab0901ede is. > > > > > > But I'm frankly terrified to touch it because I'm skeptical that two > > > more quirks are enough. I just expect that whatever strategy BIOS > > > uses to build these maps is likely to be used on many models. I do > > > hate 07eab0901ede though, so I would love to see it gone. > > > > It has been about 4 years since we came up with 07eab0901ede IIRC we > > had a huge email thread with a whole bunch of attempts at fixing > > the issue in a generic manner. > > > > IIRC some of those attempts tried to identify the exact problematic > > issue the quirks try to fix where there is an EFI MMIO reservation > > covering the entire bridge window. > > > > Maybe we can dig through the old thread and see if with the knowledge > > we have now one of the previous attempts of fixing it from the thread > > is better? > > > > I agree that needing to go back to quirks is likely going to be painful > > and break a whole bunch of other systems. IIRC 07eab0901ede fixed > > a whole bunch of laptops having issues around the time not just > > the Clevo NL4XLU and Lenovo Yoga C940. But this info might be spread > > over various bugzilla entries, etc. + some models where likely never > > reported but just fixed in time for users to not report things. > > > > I also wonder if given we only have this magic black-hole on geneo/zen4 > > if it would not be better to go with Mario's original approach here ? > > Esp. since Mario's quirk is chipset based where as the other problems > > seem to be model/product based so that is 1 quirk vs many quirks. > > Can we simply change the algo of allocating resources to follow what Windows > does? I've no idea what it does. You seem to simplify things to just being about direction of allocation but there are other constraints, most importantly alignment ones. There are many cases where there simply isn't any extra room to play with so the algorithm must produce as tight packing as possible. The assignment step is greedy that could easily lead us into dead ends where not all resources can be assigned because something was placed such that it is on the way. This series of mine tries to make placement such that it maximizes success possibility of subsequent assignments (leaving the best free space candidate for future assignments). Side-effect of these changes is that more allocations do happen at the tail (to maximize the alignment order of the free space, that is usually larger at the left edge/start of the window). So maybe you actually end up getting what you wish in many cases. Retrying anything in controlled manner is also challenging. This might be fixable on medium term by holding some setup structure across algorithm "tries" but currently we hold nothing else than what's in the struct resource and in/not-in-resource-tree information. Changing anything in this area hasn't been easy at all because almost everything seems to break something (in part it's my own bugs but there are also many cases I just could not foresee nor encounter in my tests). So I'm sorry don't think any large scale rewrite would be viable. -- i. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 11:17 ` Ilpo Järvinen @ 2026-10-14 13:32 ` Andy Shevchenko 2026-10-14 20:23 ` Geramy Loveless 0 siblings, 1 reply; 11+ messages in thread From: Andy Shevchenko @ 2026-10-14 13:32 UTC (permalink / raw) To: Ilpo Järvinen Cc: Hans de Goede, Bjorn Helgaas, Mario Limonciello, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Kai-Heng Feng, linux-acpi On Wed, Oct 14, 2026 at 02:17:36PM +0300, Ilpo Järvinen wrote: > On Wed, 14 Oct 2026, Andy Shevchenko wrote: > > On Wed, Oct 14, 2026 at 11:34:44AM +0200, Hans de Goede wrote: > > > On 14-Oct-26 00:40, Bjorn Helgaas wrote: > > > > On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: > > > >> On 10/13/26 12:26, Ilpo Järvinen wrote: > > > >>> On Tue, 13 Oct 2026, Mario Limonciello wrote: > > > >>>> On 10/13/26 12:00, Mario Limonciello wrote: > > > >>>>> On 10/13/26 08:09, Ilpo Järvinen wrote: > > > >>>>>> While testing the resource placement changes, tests hit a case where > > > >>>>>> igb fails to probe when BAR 0 is placed at 0x9c000000: > > > >>>>>> > > > >>>>>> 90000000-9cffffff : PCI Bus 0000:a0 > > > >>>>>> - 90000000-902fffff : PCI Bus 0000:a1 > > > >>>>>> - 90000000-900fffff : 0000:a1:00.0 > > > >>>>>> - 90000000-900fffff : igb > > > >>>>>> - 90100000-901fffff : 0000:a1:00.0 > > > >>>>>> - 90200000-90203fff : 0000:a1:00.0 > > > >>>>>> - 90200000-90203fff : igb > > > >>>>>> + 9be00000-9c0fffff : PCI Bus 0000:a1 > > > >>>>>> + 9be00000-9befffff : 0000:a1:00.0 > > > >>>>>> + 9bf00000-9bf03fff : 0000:a1:00.0 > > > >>>>>> + 9c000000-9c0fffff : 0000:a1:00.0 > > > >>>>>> 9c100000-9c17ffff : amd_iommu > > > >>>>>> 9c180000-9c1803ff : IOAPIC 8 > > > >>>>>> > > > >>>>>> - Region 0: Memory at 90000000 (32-bit, non-prefetchable) > > > >>>>>> [size=1M] > > > >>>>>> - Region 3: Memory at 90200000 (32-bit, non-prefetchable) > > > >>>>>> [size=16K] > > > >>>>>> - Expansion ROM at 90100000 [disabled] [size=1M] > > > >>>>>> + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) > > > >>>>>> [size=1M] > > > >>>>>> + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) > > > >>>>>> [size=16K] > > > >>>>>> + Expansion ROM at 9be00000 [disabled] [size=1M] > > > >>>>>> > > > >>>>>> igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost > > > >>>>>> ------------[ cut here ]------------ > > > >>>>>> igb: Failed to read reg 0x18! > > > >>>>>> WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at > > > >>>>>> igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 > > > >>>>>> ... > > > >>>>>> igb_get_invariants_82575+0xff/0xf00 [igb] > > > >>>>>> igb_probe+0x3c8/0x1190 [igb] > > > >>>>>> local_pci_probe+0x3b/0x80 > > > >>>>>> > > > >>>>>> It turns out there is a 64kB iomem black hole at 9c000000 that returns > > > >>>>>> ~0 and this is where igb's BAR 0 resides. If another BAR of the same > > > >>>>>> card is placed into that address, it is similarly black holed. > > > >>>>>> > > > >>>>>> Mark the problematic 64kB range reserved using a quirk bound to the > > > >>>>>> bridge found in the problematic system. > > > >>>>>> > > > >>>>>> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 > > > >>>>>> Suggested-by: Mario Limonciello <mario.limonciello@amd.com> > > > >>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> > > > >>>>>> --- > > > >>>>>> Cc: Borislav Petkov <bp@alien8.de> > > > >>>>>> > > > >>>>>> As an alternative to this, I've also tested reverting the commit > > > >>>>>> 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and > > > >>>>>> that also resolves the issue by retaining "9c000000-9cffffff : Reserved" > > > >>>>>> entry in the resource tree. > > > >>>>>> > > > >>>>>> --- > > > >>>>>> arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ > > > >>>>>> 1 file changed, 36 insertions(+) > > > >>>>>> > > > >>>>>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > > > >>>>>> index b301c6c8df75..cc463bf492b6 100644 > > > >>>>>> --- a/arch/x86/pci/fixup.c > > > >>>>>> +++ b/arch/x86/pci/fixup.c > > > >>>>>> @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, > > > >>>>>> pci_amd_enable_64bit_bar); > > > >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, > > > >>>>>> pci_amd_enable_64bit_bar); > > > >>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, > > > >>>>>> pci_amd_enable_64bit_bar); > > > >>>>>> +/* > > > >>>>>> + * TODO: Remove when/if root cause is found. > > > >>>>>> + * > > > >>>>>> + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 > > > >>>>>> + * address for which all reads return ~0 blocking an overlapping BAR from > > > >>>>>> + * working. > > > >>>>>> + * > > > >>>>>> + * Work around the problem by reserving the space prior to making any > > > >>>>>> iomem > > > >>>>>> + * allocations that could overlap with the black hole. > > > >>>>>> + */ > > > >>>>>> +static struct resource genoa_black_hole = > > > >>>>>> DEFINE_RES_MEM_NAMED(0x9c000000, > > > >>>>>> + SZ_64K, > > > >>>>>> + "reserved"); > > > >>>>>> + > > > >>>>>> +static void genoa_iomem_black_hole(struct pci_dev *dev) > > > >>>>>> +{ > > > >>>>>> + struct resource *r; > > > >>>>>> + > > > >>>>>> + pci_bus_for_each_resource(dev->bus, r) { > > > >>>>>> + if (!r || !r->flags || !resource_assigned(r)) > > > >>>>>> + continue; > > > >>>>>> + > > > >>>>>> + if (!__resource_contains_unbound(r, &genoa_black_hole)) > > > >>>>>> + continue; > > > >>>>>> + > > > >>>>>> + if (resource_assigned(&genoa_black_hole)) > > > >>>>>> + pci_dbg(dev, "iomem black hole workaround already > > > >>>>>> applied\n"); > > > >>>>>> + else if (!insert_resource(r, &genoa_black_hole)) > > > >>>>>> + pci_info(dev, "iomem black hole workaround enabled\n"); > > > >>>>>> + else > > > >>>>>> + pci_dbg(dev, "iomem black hole workaround add failed\n"); > > > >>>>>> + return; > > > >>>>>> + } > > > >>>>>> +} > > > >>>>>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, > > > >>>>>> genoa_iomem_black_hole); > > > >>>>>> + > > > >>>>>> #define RS690_LOWER_TOP_OF_DRAM2 0x30 > > > >>>>>> #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 > > > >>>>>> #define RS690_UPPER_TOP_OF_DRAM2 0x31 > > > >>>>> > > > >>>>> I guess it's good we know the root cause is 07eab0901ede. If we're going > > > >>>>> down the path of making holes in that we may well put the hole in > > > >>>>> efi_remove_e820_mmio() like this (attached). > > > >>>> > > > >>>> I don't really like that though. If Genoa is affected I would expect plenty > > > >>>> of other machines are too they just haven't been found yet. > > > >>> > > > >>> That's what I'm worried as well. My own testing can only cover very little > > > >>> ground here. > > > >>> > > > >>>> Can we roll back 07eab0901ede and add quirks for the busted machines instead? > > > >> > > > >> I had an agent dig through all the reports linked in 07eab0901ede and > > > >> drafted up what should be the remaining quirks for a revert. > > > >> > > > >> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess > > > >> > > > >> Bjorn, thoughts on this? I'll send it out if that's the way we want > > > >> to go (or Ilpo feel free to subsume into your series as it's only > > > >> really exposed by your series at the moment) > > > > > > > > I would LOVE it if we can use those patches because it's way less > > > > magic than 07eab0901ede is. > > > > > > > > But I'm frankly terrified to touch it because I'm skeptical that two > > > > more quirks are enough. I just expect that whatever strategy BIOS > > > > uses to build these maps is likely to be used on many models. I do > > > > hate 07eab0901ede though, so I would love to see it gone. > > > > > > It has been about 4 years since we came up with 07eab0901ede IIRC we > > > had a huge email thread with a whole bunch of attempts at fixing > > > the issue in a generic manner. > > > > > > IIRC some of those attempts tried to identify the exact problematic > > > issue the quirks try to fix where there is an EFI MMIO reservation > > > covering the entire bridge window. > > > > > > Maybe we can dig through the old thread and see if with the knowledge > > > we have now one of the previous attempts of fixing it from the thread > > > is better? > > > > > > I agree that needing to go back to quirks is likely going to be painful > > > and break a whole bunch of other systems. IIRC 07eab0901ede fixed > > > a whole bunch of laptops having issues around the time not just > > > the Clevo NL4XLU and Lenovo Yoga C940. But this info might be spread > > > over various bugzilla entries, etc. + some models where likely never > > > reported but just fixed in time for users to not report things. > > > > > > I also wonder if given we only have this magic black-hole on geneo/zen4 > > > if it would not be better to go with Mario's original approach here ? > > > Esp. since Mario's quirk is chipset based where as the other problems > > > seem to be model/product based so that is 1 quirk vs many quirks. > > > > Can we simply change the algo of allocating resources to follow what Windows > > does? > > I've no idea what it does. > > You seem to simplify things to just being about direction of allocation > but there are other constraints, most importantly alignment ones. > > There are many cases where there simply isn't any extra room to play with > so the algorithm must produce as tight packing as possible. > > The assignment step is greedy that could easily lead us into dead ends > where not all resources can be assigned because something was placed such > that it is on the way. This series of mine tries to make placement such > that it maximizes success possibility of subsequent assignments (leaving > the best free space candidate for future assignments). Side-effect of > these changes is that more allocations do happen at the tail (to maximize > the alignment order of the free space, that is usually larger at the left > edge/start of the window). So maybe you actually end up getting what you > wish in many cases. > > Retrying anything in controlled manner is also challenging. This might be > fixable on medium term by holding some setup structure across algorithm > "tries" but currently we hold nothing else than what's in the struct > resource and in/not-in-resource-tree information. > > Changing anything in this area hasn't been easy at all because almost > everything seems to break something (in part it's my own bugs but there > are also many cases I just could not foresee nor encounter in my tests). > > So I'm sorry don't think any large scale rewrite would be viable. I understand. Thanks for providing the explanation for the reason behind your series. -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 13:32 ` Andy Shevchenko @ 2026-10-14 20:23 ` Geramy Loveless 0 siblings, 0 replies; 11+ messages in thread From: Geramy Loveless @ 2026-10-14 20:23 UTC (permalink / raw) To: Andy Shevchenko, Ilpo Järvinen Cc: Hans de Goede, Bjorn Helgaas, Mario Limonciello, Maciej Grochowski, Nikolas Joshua Britton, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Kai-Heng Feng, linux-acpi As I recall, on Windows, when I tried to pass through a GPU, it read the PCIe Extended Configuration Space during boot, unlike on Linux. I can provide QEMU pcie analyzer logs if anyone is interested. You can essentially debug the pcie protocol this way and find the difference in how the two boot from a reverse engineering perspective. (sorry for the resubmit html kept getting included) On 10/14/26 6:32 AM, Andy Shevchenko wrote: > On Wed, Oct 14, 2026 at 02:17:36PM +0300, Ilpo Järvinen wrote: >> On Wed, 14 Oct 2026, Andy Shevchenko wrote: >>> On Wed, Oct 14, 2026 at 11:34:44AM +0200, Hans de Goede wrote: >>>> On 14-Oct-26 00:40, Bjorn Helgaas wrote: >>>>> On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: >>>>>> On 10/13/26 12:26, Ilpo Järvinen wrote: >>>>>>> On Tue, 13 Oct 2026, Mario Limonciello wrote: >>>>>>>> On 10/13/26 12:00, Mario Limonciello wrote: >>>>>>>>> On 10/13/26 08:09, Ilpo Järvinen wrote: >>>>>>>>>> While testing the resource placement changes, tests hit a case where >>>>>>>>>> igb fails to probe when BAR 0 is placed at 0x9c000000: >>>>>>>>>> >>>>>>>>>> 90000000-9cffffff : PCI Bus 0000:a0 >>>>>>>>>> - 90000000-902fffff : PCI Bus 0000:a1 >>>>>>>>>> - 90000000-900fffff : 0000:a1:00.0 >>>>>>>>>> - 90000000-900fffff : igb >>>>>>>>>> - 90100000-901fffff : 0000:a1:00.0 >>>>>>>>>> - 90200000-90203fff : 0000:a1:00.0 >>>>>>>>>> - 90200000-90203fff : igb >>>>>>>>>> + 9be00000-9c0fffff : PCI Bus 0000:a1 >>>>>>>>>> + 9be00000-9befffff : 0000:a1:00.0 >>>>>>>>>> + 9bf00000-9bf03fff : 0000:a1:00.0 >>>>>>>>>> + 9c000000-9c0fffff : 0000:a1:00.0 >>>>>>>>>> 9c100000-9c17ffff : amd_iommu >>>>>>>>>> 9c180000-9c1803ff : IOAPIC 8 >>>>>>>>>> >>>>>>>>>> - Region 0: Memory at 90000000 (32-bit, non-prefetchable) >>>>>>>>>> [size=1M] >>>>>>>>>> - Region 3: Memory at 90200000 (32-bit, non-prefetchable) >>>>>>>>>> [size=16K] >>>>>>>>>> - Expansion ROM at 90100000 [disabled] [size=1M] >>>>>>>>>> + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) >>>>>>>>>> [size=1M] >>>>>>>>>> + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) >>>>>>>>>> [size=16K] >>>>>>>>>> + Expansion ROM at 9be00000 [disabled] [size=1M] >>>>>>>>>> >>>>>>>>>> igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost >>>>>>>>>> ------------[ cut here ]------------ >>>>>>>>>> igb: Failed to read reg 0x18! >>>>>>>>>> WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at >>>>>>>>>> igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 >>>>>>>>>> ... >>>>>>>>>> igb_get_invariants_82575+0xff/0xf00 [igb] >>>>>>>>>> igb_probe+0x3c8/0x1190 [igb] >>>>>>>>>> local_pci_probe+0x3b/0x80 >>>>>>>>>> >>>>>>>>>> It turns out there is a 64kB iomem black hole at 9c000000 that returns >>>>>>>>>> ~0 and this is where igb's BAR 0 resides. If another BAR of the same >>>>>>>>>> card is placed into that address, it is similarly black holed. >>>>>>>>>> >>>>>>>>>> Mark the problematic 64kB range reserved using a quirk bound to the >>>>>>>>>> bridge found in the problematic system. >>>>>>>>>> >>>>>>>>>> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 >>>>>>>>>> Suggested-by: Mario Limonciello <mario.limonciello@amd.com> >>>>>>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> >>>>>>>>>> --- >>>>>>>>>> Cc: Borislav Petkov <bp@alien8.de> >>>>>>>>>> >>>>>>>>>> As an alternative to this, I've also tested reverting the commit >>>>>>>>>> 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and >>>>>>>>>> that also resolves the issue by retaining "9c000000-9cffffff : Reserved" >>>>>>>>>> entry in the resource tree. >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ >>>>>>>>>> 1 file changed, 36 insertions(+) >>>>>>>>>> >>>>>>>>>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c >>>>>>>>>> index b301c6c8df75..cc463bf492b6 100644 >>>>>>>>>> --- a/arch/x86/pci/fixup.c >>>>>>>>>> +++ b/arch/x86/pci/fixup.c >>>>>>>>>> @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, >>>>>>>>>> pci_amd_enable_64bit_bar); >>>>>>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, >>>>>>>>>> pci_amd_enable_64bit_bar); >>>>>>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, >>>>>>>>>> pci_amd_enable_64bit_bar); >>>>>>>>>> +/* >>>>>>>>>> + * TODO: Remove when/if root cause is found. >>>>>>>>>> + * >>>>>>>>>> + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 >>>>>>>>>> + * address for which all reads return ~0 blocking an overlapping BAR from >>>>>>>>>> + * working. >>>>>>>>>> + * >>>>>>>>>> + * Work around the problem by reserving the space prior to making any >>>>>>>>>> iomem >>>>>>>>>> + * allocations that could overlap with the black hole. >>>>>>>>>> + */ >>>>>>>>>> +static struct resource genoa_black_hole = >>>>>>>>>> DEFINE_RES_MEM_NAMED(0x9c000000, >>>>>>>>>> + SZ_64K, >>>>>>>>>> + "reserved"); >>>>>>>>>> + >>>>>>>>>> +static void genoa_iomem_black_hole(struct pci_dev *dev) >>>>>>>>>> +{ >>>>>>>>>> + struct resource *r; >>>>>>>>>> + >>>>>>>>>> + pci_bus_for_each_resource(dev->bus, r) { >>>>>>>>>> + if (!r || !r->flags || !resource_assigned(r)) >>>>>>>>>> + continue; >>>>>>>>>> + >>>>>>>>>> + if (!__resource_contains_unbound(r, &genoa_black_hole)) >>>>>>>>>> + continue; >>>>>>>>>> + >>>>>>>>>> + if (resource_assigned(&genoa_black_hole)) >>>>>>>>>> + pci_dbg(dev, "iomem black hole workaround already >>>>>>>>>> applied\n"); >>>>>>>>>> + else if (!insert_resource(r, &genoa_black_hole)) >>>>>>>>>> + pci_info(dev, "iomem black hole workaround enabled\n"); >>>>>>>>>> + else >>>>>>>>>> + pci_dbg(dev, "iomem black hole workaround add failed\n"); >>>>>>>>>> + return; >>>>>>>>>> + } >>>>>>>>>> +} >>>>>>>>>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, >>>>>>>>>> genoa_iomem_black_hole); >>>>>>>>>> + >>>>>>>>>> #define RS690_LOWER_TOP_OF_DRAM2 0x30 >>>>>>>>>> #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 >>>>>>>>>> #define RS690_UPPER_TOP_OF_DRAM2 0x31 >>>>>>>>> >>>>>>>>> I guess it's good we know the root cause is 07eab0901ede. If we're going >>>>>>>>> down the path of making holes in that we may well put the hole in >>>>>>>>> efi_remove_e820_mmio() like this (attached). >>>>>>>> >>>>>>>> I don't really like that though. If Genoa is affected I would expect plenty >>>>>>>> of other machines are too they just haven't been found yet. >>>>>>> >>>>>>> That's what I'm worried as well. My own testing can only cover very little >>>>>>> ground here. >>>>>>> >>>>>>>> Can we roll back 07eab0901ede and add quirks for the busted machines instead? >>>>>> >>>>>> I had an agent dig through all the reports linked in 07eab0901ede and >>>>>> drafted up what should be the remaining quirks for a revert. >>>>>> >>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess >>>>>> >>>>>> Bjorn, thoughts on this? I'll send it out if that's the way we want >>>>>> to go (or Ilpo feel free to subsume into your series as it's only >>>>>> really exposed by your series at the moment) >>>>> >>>>> I would LOVE it if we can use those patches because it's way less >>>>> magic than 07eab0901ede is. >>>>> >>>>> But I'm frankly terrified to touch it because I'm skeptical that two >>>>> more quirks are enough. I just expect that whatever strategy BIOS >>>>> uses to build these maps is likely to be used on many models. I do >>>>> hate 07eab0901ede though, so I would love to see it gone. >>>> >>>> It has been about 4 years since we came up with 07eab0901ede IIRC we >>>> had a huge email thread with a whole bunch of attempts at fixing >>>> the issue in a generic manner. >>>> >>>> IIRC some of those attempts tried to identify the exact problematic >>>> issue the quirks try to fix where there is an EFI MMIO reservation >>>> covering the entire bridge window. >>>> >>>> Maybe we can dig through the old thread and see if with the knowledge >>>> we have now one of the previous attempts of fixing it from the thread >>>> is better? >>>> >>>> I agree that needing to go back to quirks is likely going to be painful >>>> and break a whole bunch of other systems. IIRC 07eab0901ede fixed >>>> a whole bunch of laptops having issues around the time not just >>>> the Clevo NL4XLU and Lenovo Yoga C940. But this info might be spread >>>> over various bugzilla entries, etc. + some models where likely never >>>> reported but just fixed in time for users to not report things. >>>> >>>> I also wonder if given we only have this magic black-hole on geneo/zen4 >>>> if it would not be better to go with Mario's original approach here ? >>>> Esp. since Mario's quirk is chipset based where as the other problems >>>> seem to be model/product based so that is 1 quirk vs many quirks. >>> >>> Can we simply change the algo of allocating resources to follow what Windows >>> does? >> >> I've no idea what it does. >> >> You seem to simplify things to just being about direction of allocation >> but there are other constraints, most importantly alignment ones. >> >> There are many cases where there simply isn't any extra room to play with >> so the algorithm must produce as tight packing as possible. >> >> The assignment step is greedy that could easily lead us into dead ends >> where not all resources can be assigned because something was placed such >> that it is on the way. This series of mine tries to make placement such >> that it maximizes success possibility of subsequent assignments (leaving >> the best free space candidate for future assignments). Side-effect of >> these changes is that more allocations do happen at the tail (to maximize >> the alignment order of the free space, that is usually larger at the left >> edge/start of the window). So maybe you actually end up getting what you >> wish in many cases. >> >> Retrying anything in controlled manner is also challenging. This might be >> fixable on medium term by holding some setup structure across algorithm >> "tries" but currently we hold nothing else than what's in the struct >> resource and in/not-in-resource-tree information. >> >> Changing anything in this area hasn't been easy at all because almost >> everything seems to break something (in part it's my own bugs but there >> are also many cases I just could not foresee nor encounter in my tests). >> >> So I'm sorry don't think any large scale rewrite would be viable. > > I understand. Thanks for providing the explanation for the reason behind > your series. > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-14 9:34 ` Hans de Goede 2026-10-14 10:51 ` Andy Shevchenko @ 2026-10-14 20:37 ` Mario Limonciello 1 sibling, 0 replies; 11+ messages in thread From: Mario Limonciello @ 2026-10-14 20:37 UTC (permalink / raw) To: Hans de Goede, Bjorn Helgaas Cc: Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Kai-Heng Feng, Andy Shevchenko, linux-acpi On 10/14/26 04:34, Hans de Goede wrote: > Hi, > > On 14-Oct-26 00:40, Bjorn Helgaas wrote: >> [+cc Hans, Kai-Heng, Andy, linux-acpi] >> >> On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: >>> On 10/13/26 12:26, Ilpo Järvinen wrote: >>>> On Tue, 13 Oct 2026, Mario Limonciello wrote: >>>>> On 10/13/26 12:00, Mario Limonciello wrote: >>>>>> On 10/13/26 08:09, Ilpo Järvinen wrote: >>>>>>> While testing the resource placement changes, tests hit a case where >>>>>>> igb fails to probe when BAR 0 is placed at 0x9c000000: >>>>>>> >>>>>>> 90000000-9cffffff : PCI Bus 0000:a0 >>>>>>> - 90000000-902fffff : PCI Bus 0000:a1 >>>>>>> - 90000000-900fffff : 0000:a1:00.0 >>>>>>> - 90000000-900fffff : igb >>>>>>> - 90100000-901fffff : 0000:a1:00.0 >>>>>>> - 90200000-90203fff : 0000:a1:00.0 >>>>>>> - 90200000-90203fff : igb >>>>>>> + 9be00000-9c0fffff : PCI Bus 0000:a1 >>>>>>> + 9be00000-9befffff : 0000:a1:00.0 >>>>>>> + 9bf00000-9bf03fff : 0000:a1:00.0 >>>>>>> + 9c000000-9c0fffff : 0000:a1:00.0 >>>>>>> 9c100000-9c17ffff : amd_iommu >>>>>>> 9c180000-9c1803ff : IOAPIC 8 >>>>>>> >>>>>>> - Region 0: Memory at 90000000 (32-bit, non-prefetchable) >>>>>>> [size=1M] >>>>>>> - Region 3: Memory at 90200000 (32-bit, non-prefetchable) >>>>>>> [size=16K] >>>>>>> - Expansion ROM at 90100000 [disabled] [size=1M] >>>>>>> + Region 0: Memory at 9c000000 (32-bit, non-prefetchable) >>>>>>> [size=1M] >>>>>>> + Region 3: Memory at 9bf00000 (32-bit, non-prefetchable) >>>>>>> [size=16K] >>>>>>> + Expansion ROM at 9be00000 [disabled] [size=1M] >>>>>>> >>>>>>> igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost >>>>>>> ------------[ cut here ]------------ >>>>>>> igb: Failed to read reg 0x18! >>>>>>> WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at >>>>>>> igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706 >>>>>>> ... >>>>>>> igb_get_invariants_82575+0xff/0xf00 [igb] >>>>>>> igb_probe+0x3c8/0x1190 [igb] >>>>>>> local_pci_probe+0x3b/0x80 >>>>>>> >>>>>>> It turns out there is a 64kB iomem black hole at 9c000000 that returns >>>>>>> ~0 and this is where igb's BAR 0 resides. If another BAR of the same >>>>>>> card is placed into that address, it is similarly black holed. >>>>>>> >>>>>>> Mark the problematic 64kB range reserved using a quirk bound to the >>>>>>> bridge found in the problematic system. >>>>>>> >>>>>>> Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074 >>>>>>> Suggested-by: Mario Limonciello <mario.limonciello@amd.com> >>>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> >>>>>>> --- >>>>>>> Cc: Borislav Petkov <bp@alien8.de> >>>>>>> >>>>>>> As an alternative to this, I've also tested reverting the commit >>>>>>> 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and >>>>>>> that also resolves the issue by retaining "9c000000-9cffffff : Reserved" >>>>>>> entry in the resource tree. >>>>>>> >>>>>>> --- >>>>>>> arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++ >>>>>>> 1 file changed, 36 insertions(+) >>>>>>> >>>>>>> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c >>>>>>> index b301c6c8df75..cc463bf492b6 100644 >>>>>>> --- a/arch/x86/pci/fixup.c >>>>>>> +++ b/arch/x86/pci/fixup.c >>>>>>> @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, >>>>>>> pci_amd_enable_64bit_bar); >>>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, >>>>>>> pci_amd_enable_64bit_bar); >>>>>>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, >>>>>>> pci_amd_enable_64bit_bar); >>>>>>> +/* >>>>>>> + * TODO: Remove when/if root cause is found. >>>>>>> + * >>>>>>> + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000 >>>>>>> + * address for which all reads return ~0 blocking an overlapping BAR from >>>>>>> + * working. >>>>>>> + * >>>>>>> + * Work around the problem by reserving the space prior to making any >>>>>>> iomem >>>>>>> + * allocations that could overlap with the black hole. >>>>>>> + */ >>>>>>> +static struct resource genoa_black_hole = >>>>>>> DEFINE_RES_MEM_NAMED(0x9c000000, >>>>>>> + SZ_64K, >>>>>>> + "reserved"); >>>>>>> + >>>>>>> +static void genoa_iomem_black_hole(struct pci_dev *dev) >>>>>>> +{ >>>>>>> + struct resource *r; >>>>>>> + >>>>>>> + pci_bus_for_each_resource(dev->bus, r) { >>>>>>> + if (!r || !r->flags || !resource_assigned(r)) >>>>>>> + continue; >>>>>>> + >>>>>>> + if (!__resource_contains_unbound(r, &genoa_black_hole)) >>>>>>> + continue; >>>>>>> + >>>>>>> + if (resource_assigned(&genoa_black_hole)) >>>>>>> + pci_dbg(dev, "iomem black hole workaround already >>>>>>> applied\n"); >>>>>>> + else if (!insert_resource(r, &genoa_black_hole)) >>>>>>> + pci_info(dev, "iomem black hole workaround enabled\n"); >>>>>>> + else >>>>>>> + pci_dbg(dev, "iomem black hole workaround add failed\n"); >>>>>>> + return; >>>>>>> + } >>>>>>> +} >>>>>>> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab, >>>>>>> genoa_iomem_black_hole); >>>>>>> + >>>>>>> #define RS690_LOWER_TOP_OF_DRAM2 0x30 >>>>>>> #define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 >>>>>>> #define RS690_UPPER_TOP_OF_DRAM2 0x31 >>>>>> >>>>>> I guess it's good we know the root cause is 07eab0901ede. If we're going >>>>>> down the path of making holes in that we may well put the hole in >>>>>> efi_remove_e820_mmio() like this (attached). >>>>> >>>>> I don't really like that though. If Genoa is affected I would expect plenty >>>>> of other machines are too they just haven't been found yet. >>>> >>>> That's what I'm worried as well. My own testing can only cover very little >>>> ground here. >>>> >>>>> Can we roll back 07eab0901ede and add quirks for the busted machines instead? >>> >>> I had an agent dig through all the reports linked in 07eab0901ede and >>> drafted up what should be the remaining quirks for a revert. >>> >>> https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess >>> >>> Bjorn, thoughts on this? I'll send it out if that's the way we want >>> to go (or Ilpo feel free to subsume into your series as it's only >>> really exposed by your series at the moment) >> >> I would LOVE it if we can use those patches because it's way less >> magic than 07eab0901ede is. >> >> But I'm frankly terrified to touch it because I'm skeptical that two >> more quirks are enough. I just expect that whatever strategy BIOS >> uses to build these maps is likely to be used on many models. I do >> hate 07eab0901ede though, so I would love to see it gone. > > It has been about 4 years since we came up with 07eab0901ede IIRC we > had a huge email thread with a whole bunch of attempts at fixing > the issue in a generic manner. > > IIRC some of those attempts tried to identify the exact problematic > issue the quirks try to fix where there is an EFI MMIO reservation > covering the entire bridge window. > > Maybe we can dig through the old thread and see if with the knowledge > we have now one of the previous attempts of fixing it from the thread > is better? > > I agree that needing to go back to quirks is likely going to be painful > and break a whole bunch of other systems. IIRC 07eab0901ede fixed > a whole bunch of laptops having issues around the time not just > the Clevo NL4XLU and Lenovo Yoga C940. But this info might be spread > over various bugzilla entries, etc. + some models where likely never > reported but just fixed in time for users to not report things. > So I did have an agent at least dig into all the linked bugzillas. That's how the quirk list came up. Whether or not this series is going to help any of those systems I think is a totally separately question. Could we perhaps loop in a all of the reporters from those bugzillas and ask them to test Ilpo's v3 series + the revert? BTW - I think we would want to roll back all the e820 quirks for the purpose of this test. Even if it doesn't help we might be able to get a "live" failure from someone to iterate on trying to find a better solution. > I also wonder if given we only have this magic black-hole on geneo/zen4 > if it would not be better to go with Mario's original approach here ? > Esp. since Mario's quirk is chipset based where as the other problems > seem to be model/product based so that is 1 quirk vs many quirks. > As Boris said - it was found on Genoa, but there is no reason it *should* be Genoa only. IMV the firmware on that Genoa system advertised not to use the region but 07eab0901ede says ignore the firmware and use it anyway. So we're adding hacks on top of hacks if we go that direction. Our ("collective") goal should be to get rid of unscable tech debt not add more that's even harder to peel back. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems 2026-10-13 22:40 ` [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems Bjorn Helgaas 2026-10-14 0:43 ` Borislav Petkov 2026-10-14 9:34 ` Hans de Goede @ 2026-10-14 10:46 ` Andy Shevchenko 2 siblings, 0 replies; 11+ messages in thread From: Andy Shevchenko @ 2026-10-14 10:46 UTC (permalink / raw) To: Bjorn Helgaas Cc: Mario Limonciello, Ilpo Järvinen, Maciej Grochowski, Nikolas Joshua Britton, Geramy Loveless, Eric Auger, Alexey Fomenko, Bjorn Helgaas, linux-pci, Lorenzo Pieralisi, Rob Herring, Krzysztof Wilczyński, Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, LKML, Bradley Morgan, Hans de Goede, Kai-Heng Feng, linux-acpi On Tue, Oct 13, 2026 at 05:40:10PM -0500, Bjorn Helgaas wrote: > [+cc Hans, Kai-Heng, Andy, linux-acpi] If you want my opinion, I think that Mario's quirk series (with the revert of the 07eab0901ede) [2] is the way to go. I have not much time to dive into this, though. I also have no HW to reproduce either of the problems. > On Tue, Oct 13, 2026 at 01:59:23PM -0500, Mario Limonciello wrote: > > On 10/13/26 12:26, Ilpo Järvinen wrote: > > > On Tue, 13 Oct 2026, Mario Limonciello wrote: > > > > On 10/13/26 12:00, Mario Limonciello wrote: > > > > > On 10/13/26 08:09, Ilpo Järvinen wrote: [2] https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/log/?h=superm1/genoa-mess > > Bjorn, thoughts on this? I'll send it out if that's the way we want > > to go (or Ilpo feel free to subsume into your series as it's only > > really exposed by your series at the moment) > > I would LOVE it if we can use those patches because it's way less > magic than 07eab0901ede is. > > But I'm frankly terrified to touch it because I'm skeptical that two > more quirks are enough. I just expect that whatever strategy BIOS > uses to build these maps is likely to be used on many models. I do > hate 07eab0901ede though, so I would love to see it gone. > > I do think the description [1] is overly specific because AFAIK > there's nothing that connects this to AMD EPYC 9354 or igb other than > the fact that it was seen there. It could happen to any PCI device on > any system with similar memory maps. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/superm1/linux.git/commit/?h=superm1/genoa-mess&id=0a49c704477153d6597d34b3c6e3141007ff058c -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-10-14 20:37 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <81b4f4bf-b721-4b5b-8ddf-d040865f02b2@amd.com>
2026-10-13 22:40 ` [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems Bjorn Helgaas
2026-10-14 0:43 ` Borislav Petkov
2026-10-14 2:41 ` Mario Limonciello
2026-10-14 10:49 ` Andy Shevchenko
2026-10-14 9:34 ` Hans de Goede
2026-10-14 10:51 ` Andy Shevchenko
2026-10-14 11:17 ` Ilpo Järvinen
2026-10-14 13:32 ` Andy Shevchenko
2026-10-14 20:23 ` Geramy Loveless
2026-10-14 20:37 ` Mario Limonciello
2026-10-14 10:46 ` Andy Shevchenko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox