Re: [REGRESSION] 527139d738d7 ("PCI/sysfs: Convert "rom" to static attribute")

["Ville Syrjälä" <ville.syrjala@linux.intel.com>]

On Fri, Jan 21, 2022 at 06:24:45PM -0600, Bjorn Helgaas wrote:
> On Fri, Dec 17, 2021 at 07:45:44PM +0200, Ville Syrjälä wrote:
> > On Fri, Dec 17, 2021 at 11:29:28AM -0600, Bjorn Helgaas wrote:
> > > On Fri, Dec 17, 2021 at 12:44:51PM +0200, Ville Syrjälä wrote:
> > > > The pci sysfs "rom" file has disappeared for VGA devices.
> > > > Looks to be a regression from commit 527139d738d7 ("PCI/sysfs:
> > > > Convert "rom" to static attribute").
> > > > 
> > > > Some kind of ordering issue between the sysfs file creation 
> > > > vs. pci_fixup_video() perhaps?
> > > 
> > > Can you attach your complete "lspci -vv" output?  Also, which is the
> > > default device?  I think there's a "boot_vga" sysfs file that shows
> > > this.  "find /sys -name boot_vga | xargs grep ."
> > 
> > All I have is Intel iGPUs so it's always 00:02.0. 
> > 
> > $ cat /sys/bus/pci/devices/0000\:00\:02.0/boot_vga 
> > 1
> > $ cat /sys/bus/pci/devices/0000\:00\:02.0/rom
> > cat: '/sys/bus/pci/devices/0000:00:02.0/rom': No such file or directory
> > 
> > I've attached the full lspci from my IVB laptop, but the problem
> > happens on every machine (with an iGPU at least).
> > 
> > I presume with a discrete GPU it might not happen since they
> > actually have a real ROM.
> > 
> > > I think the relevant path is something like this:
> > > 
> > >   acpi_pci_root_add
> > >     pci_acpi_scan_root
> > >       ...
> > >         pci_scan_single_device
> > >           pci_device_add
> > >             device_add
> > >               ...
> > >                 sysfs_create_groups
> > >                   ...
> > >                     if (grp->is_visible())
> > >                       pci_dev_rom_attr_is_visible  # after 527139d738d7
> > >                         if (pci_resource_len(pdev, PCI_ROM_RESOURCE))
> > >                           ...
> > >     pci_bus_add_devices
> > >       pci_bus_add_device
> > >         pci_fixup_device(pci_fixup_final)
> > >           pci_fixup_video
> > >             if (vga_default_device() ...)
> > >               # update PCI_ROM_RESOURCE
> > >         pci_create_sysfs_dev_files
> > >           if (pci_resource_len(pdev, PCI_ROM_RESOURCE))
> > >             sysfs_create_bin_file("rom")           # before 527139d738d7
> > > 
> > > Prior to 527139d738d7, we ran pci_fixup_video() in
> > > pci_bus_add_devices().  The vga_default_device() there might depend on
> > > the fact that we've discovered all the PCI devices.  
> > > 
> > > After 527139d738d7, we create the "rom" file in pci_device_add(),
> > > which happens as we discover each device, so maybe we don't yet know
> > > which device is the default VGA device.
> 
> Thanks, Ville!
> 
> I think this is happening because the iGPU has no ROM BAR,
> so pci_resource_len(PCI_ROM_RESOURCE) is 0 when we run
> pci_dev_rom_attr_is_visible().  Prior to 527139d738d7, we also
> used pci_resource_len(PCI_ROM_RESOURCE), but pci_fixup_video() had
> been run, and it filled in the ROM resource with the shadow ROM
> start and size.
> 
> Can you collect the dmesg output with "pci=earlydump"?  I think
> that will show zeros at PCI_ROM_ADDRESS for your 00:02.0 device.

Yeah no real ROM on these:
 pci 0000:00:02.0: config space:
 00000000: 86 80 26 01 07 00 90 00 09 00 00 03 00 00 00 00
 00000010: 04 00 00 f0 00 00 00 00 0c 00 00 e0 00 00 00 00
 00000020: 01 50 00 00 00 00 00 00 00 00 00 00 aa 17 da 21
 00000030: 00 00 00 00 90 00 00 00 00 00 00 00 0b 01 00 00
 00000040: 09 00 0c 01 9e 61 00 e2 90 00 08 14 00 00 00 00
 00000050: 11 02 00 00 11 00 00 00 00 00 00 00 01 00 a0 db
 00000060: 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
 00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00000080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00000090: 05 d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 000000a0: 00 00 00 00 13 00 06 03 00 00 00 00 00 00 00 00
 000000b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 000000c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 000000d0: 01 a4 22 00 00 00 00 00 00 00 00 00 00 00 00 00
 000000e0: 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
 000000f0: 00 00 00 00 00 00 00 00 00 00 06 00 18 60 ef da
 pci 0000:00:02.0: [8086:0126] type 00 class 0x030000
 pci 0000:00:02.0: reg 0x10: [mem 0xf0000000-0xf03fffff 64bit]
 pci 0000:00:02.0: reg 0x18: [mem 0xe0000000-0xefffffff 64bit pref]
 pci 0000:00:02.0: reg 0x20: [io  0x5000-0x503f]

> 
> And can you try the test patch below?  I reproduced the problem in
> qemu by instrumenting to clear the VGA ROM BAR, and moving the
> pci_fixup_video() quirk earlier makes it run before
> pci_dev_rom_attr_is_visible(), which fixes the problem for me.
> 
> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
> index 2edd86649468..615a76d70019 100644
> --- a/arch/x86/pci/fixup.c
> +++ b/arch/x86/pci/fixup.c
> @@ -353,8 +353,8 @@ static void pci_fixup_video(struct pci_dev *pdev)
>  		}
>  	}
>  }
> -DECLARE_PCI_FIXUP_CLASS_FINAL(PCI_ANY_ID, PCI_ANY_ID,
> -				PCI_CLASS_DISPLAY_VGA, 8, pci_fixup_video);
> +DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_ANY_ID, PCI_ANY_ID,
> +			       PCI_CLASS_DISPLAY_VGA, 8, pci_fixup_video);
>  
>  
>  static const struct dmi_system_id msi_k8t_dmi_table[] = {

pci=earlydump now reports:
 pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
and poking it via sysfs works fine.

Tested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>

-- 
Ville Syrjälä
Intel