Query Regarding PCI Configuration Space Mapping and BAR Programming

Query Regarding PCI Configuration Space Mapping and BAR Programming

[Muni Sekhar]

Dear Linux-PCI Maintainers,

I have a few questions regarding PCI configuration space and Base
Address Register (BAR) programming:

PCI Configuration Space Mapping:
PCI devices have standard registers (Device ID, Vendor ID, Status,
Command, etc.) in their configuration space.
These registers are mapped to memory locations. How can we determine
the exact memory location where these configuration space registers
are mapped?

Base Address Registers (BARs) Programming:
How are BAR registers programmed, and what ensures they do not
conflict with other devices mapped memory in the system?
If a BAR mapping clashes with another device’s memory range, how does
the system handle this?

Does the BIOS/firmware allocate BAR addresses, or does the OS have a
role in reconfiguring them during device initialization?
If you could provide any source code references from the Linux kernel
that handle these aspects, it would be greatly appreciated.

Looking forward to your insights.

--
Thanks,
Sekhar

Re: Query Regarding PCI Configuration Space Mapping and BAR Programming

[Bjorn Helgaas]

On Wed, Mar 19, 2025 at 11:27:07PM +0530, Muni Sekhar wrote:
> Dear Linux-PCI Maintainers,
> 
> I have a few questions regarding PCI configuration space and Base
> Address Register (BAR) programming:
> 
> PCI Configuration Space Mapping:
> PCI devices have standard registers (Device ID, Vendor ID, Status,
> Command, etc.) in their configuration space.
> These registers are mapped to memory locations. 

In general, config registers are not mapped to memory locations.

Some systems use the ECAM mechanism for config access, which is
basically memory-mapped I/O that converts CPU memory accesses to PCI
config accesses.  This mechanism is hidden inside the Linux config
accessors (pci_read_config_word(), etc), and drivers should not use
ECAM directly.

> How can we determine
> the exact memory location where these configuration space registers
> are mapped?

Drivers use pci_read_config_word() and similar interfaces to access
config registers.  Userspace can use the setpci utility.

> Base Address Registers (BARs) Programming:
> How are BAR registers programmed, and what ensures they do not
> conflict with other devices mapped memory in the system?
> If a BAR mapping clashes with another device’s memory range, how does
> the system handle this?
> 
> Does the BIOS/firmware allocate BAR addresses, or does the OS have a
> role in reconfiguring them during device initialization?

On x86 systems, the BIOS generally assigns BAR addresses.  Linux
doesn't change these unless they are invalid.  Some firmware does not
assign BARs, and generally firmware does not assign BARs for hot-added
devices.  In those cases, Linux assigns them.

If Linux notices a conflict, it tries to reassign BARs to avoid the
conflict.

> If you could provide any source code references from the Linux kernel
> that handle these aspects, it would be greatly appreciated.

Here's the function that reads BARs when Linux enumerates a device:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/probe.c?id=v6.13#n176

Re: Query Regarding PCI Configuration Space Mapping and BAR Programming

[Hans Zhang]

On 2025/3/20 03:15, Bjorn Helgaas wrote:
> EXTERNAL EMAIL
> 
> On Wed, Mar 19, 2025 at 11:27:07PM +0530, Muni Sekhar wrote:
>> Dear Linux-PCI Maintainers,
>>
>> I have a few questions regarding PCI configuration space and Base
>> Address Register (BAR) programming:
>>
>> PCI Configuration Space Mapping:
>> PCI devices have standard registers (Device ID, Vendor ID, Status,
>> Command, etc.) in their configuration space.
>> These registers are mapped to memory locations.
> 
> In general, config registers are not mapped to memory locations.
> 

Yes, this is just a segment of the address visible to the CPU, not the 
actual memory.

Best regards,
Hans

> Some systems use the ECAM mechanism for config access, which is
> basically memory-mapped I/O that converts CPU memory accesses to PCI
> config accesses.  This mechanism is hidden inside the Linux config
> accessors (pci_read_config_word(), etc), and drivers should not use
> ECAM directly.
> 
>> How can we determine
>> the exact memory location where these configuration space registers
>> are mapped?
> 
> Drivers use pci_read_config_word() and similar interfaces to access
> config registers.  Userspace can use the setpci utility.
> 
>> Base Address Registers (BARs) Programming:
>> How are BAR registers programmed, and what ensures they do not
>> conflict with other devices mapped memory in the system?
>> If a BAR mapping clashes with another device’s memory range, how does
>> the system handle this?
>>
>> Does the BIOS/firmware allocate BAR addresses, or does the OS have a
>> role in reconfiguring them during device initialization?
> 
> On x86 systems, the BIOS generally assigns BAR addresses.  Linux
> doesn't change these unless they are invalid.  Some firmware does not
> assign BARs, and generally firmware does not assign BARs for hot-added
> devices.  In those cases, Linux assigns them.
> 
> If Linux notices a conflict, it tries to reassign BARs to avoid the
> conflict.
> 
>> If you could provide any source code references from the Linux kernel
>> that handle these aspects, it would be greatly appreciated.
> 
> Here's the function that reads BARs when Linux enumerates a device:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/probe.c?id=v6.13#n176
> 

Re: Query Regarding PCI Configuration Space Mapping and BAR Programming

[Muni Sekhar]

On Thu, Mar 20, 2025 at 12:45 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Wed, Mar 19, 2025 at 11:27:07PM +0530, Muni Sekhar wrote:
> > Dear Linux-PCI Maintainers,
> >
> > I have a few questions regarding PCI configuration space and Base
> > Address Register (BAR) programming:
> >
> > PCI Configuration Space Mapping:
> > PCI devices have standard registers (Device ID, Vendor ID, Status,
> > Command, etc.) in their configuration space.
> > These registers are mapped to memory locations.
>
> In general, config registers are not mapped to memory locations.
>
> Some systems use the ECAM mechanism for config access, which is
> basically memory-mapped I/O that converts CPU memory accesses to PCI
> config accesses.  This mechanism is hidden inside the Linux config
> accessors (pci_read_config_word(), etc), and drivers should not use
> ECAM directly.
Thanks for the clarification.

Both pci_bus_read_config_word() and pci_user_read_config_word() eventually call:
res = bus->ops->read(bus, devfn, pos, len, &data);
ret = dev->bus->ops->read(dev->bus, dev->devfn, pos, sizeof(type), &data);

This suggests that both kernel-space (pci_read_config_*()) and
user-space (pci_user_read_config_*()) accessors use the same
underlying function for configuration space access.
Could you confirm if my understanding is correct?

I tried locating the implementation of bus->ops->read() but could not
find its definition in the kernel source.Even using the function_graph
tracer, I was unable to capture the exact function being executed.
Could you point me to where this function is defined in the kernel?

>
> > How can we determine
> > the exact memory location where these configuration space registers
> > are mapped?
>
> Drivers use pci_read_config_word() and similar interfaces to access
> config registers.  Userspace can use the setpci utility.
>
> > Base Address Registers (BARs) Programming:
> > How are BAR registers programmed, and what ensures they do not
> > conflict with other devices mapped memory in the system?
> > If a BAR mapping clashes with another device’s memory range, how does
> > the system handle this?
> >
> > Does the BIOS/firmware allocate BAR addresses, or does the OS have a
> > role in reconfiguring them during device initialization?
>
> On x86 systems, the BIOS generally assigns BAR addresses.  Linux
> doesn't change these unless they are invalid.  Some firmware does not
> assign BARs, and generally firmware does not assign BARs for hot-added
> devices.  In those cases, Linux assigns them.
Once a hotplug event is detected (either via a PCIe hotplug interrupt
or an ACPI event), which kernel function is responsible for
dynamically assigning the Bus, Device, and Function (BDF) number to
the new device?
Additionally, which function in the kernel assigns Base Address
Registers (BARs) for the hotplugged device?

I appreciate your time and assistance in clarifying these points.

>
> If Linux notices a conflict, it tries to reassign BARs to avoid the
> conflict.
>
> > If you could provide any source code references from the Linux kernel
> > that handle these aspects, it would be greatly appreciated.
>
> Here's the function that reads BARs when Linux enumerates a device:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/probe.c?id=v6.13#n176



-- 
Thanks,
Sekhar

Re: Query Regarding PCI Configuration Space Mapping and BAR Programming

[Bjorn Helgaas]

On Fri, Mar 21, 2025 at 04:38:09PM +0530, Muni Sekhar wrote:
> On Thu, Mar 20, 2025 at 12:45 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > On Wed, Mar 19, 2025 at 11:27:07PM +0530, Muni Sekhar wrote:
> > > Dear Linux-PCI Maintainers,
> > >
> > > I have a few questions regarding PCI configuration space and Base
> > > Address Register (BAR) programming:
> > >
> > > PCI Configuration Space Mapping:
> > > PCI devices have standard registers (Device ID, Vendor ID, Status,
> > > Command, etc.) in their configuration space.
> > > These registers are mapped to memory locations.
> >
> > In general, config registers are not mapped to memory locations.
> >
> > Some systems use the ECAM mechanism for config access, which is
> > basically memory-mapped I/O that converts CPU memory accesses to PCI
> > config accesses.  This mechanism is hidden inside the Linux config
> > accessors (pci_read_config_word(), etc), and drivers should not use
> > ECAM directly.
> 
> Both pci_bus_read_config_word() and pci_user_read_config_word() eventually call:
> res = bus->ops->read(bus, devfn, pos, len, &data);
> ret = dev->bus->ops->read(dev->bus, dev->devfn, pos, sizeof(type), &data);
> 
> This suggests that both kernel-space (pci_read_config_*()) and
> user-space (pci_user_read_config_*()) accessors use the same
> underlying function for configuration space access.
> Could you confirm if my understanding is correct?

That's correct.

> I tried locating the implementation of bus->ops->read() but could not
> find its definition in the kernel source.Even using the function_graph
> tracer, I was unable to capture the exact function being executed.
> Could you point me to where this function is defined in the kernel?

Each struct pci_bus has a "struct pci_ops *ops" pointer.  The struct
pci_ops supplies config accessors.  These are platform-dependent or
host bridge-dependent.

For x86, it's typically pci_root_ops:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/pci/common.c?id=v6.13#n72

> > > How can we determine
> > > the exact memory location where these configuration space registers
> > > are mapped?
> >
> > Drivers use pci_read_config_word() and similar interfaces to access
> > config registers.  Userspace can use the setpci utility.
> >
> > > Base Address Registers (BARs) Programming:
> > > How are BAR registers programmed, and what ensures they do not
> > > conflict with other devices mapped memory in the system?
> > > If a BAR mapping clashes with another device’s memory range, how does
> > > the system handle this?
> > >
> > > Does the BIOS/firmware allocate BAR addresses, or does the OS have a
> > > role in reconfiguring them during device initialization?
> >
> > On x86 systems, the BIOS generally assigns BAR addresses.  Linux
> > doesn't change these unless they are invalid.  Some firmware does not
> > assign BARs, and generally firmware does not assign BARs for hot-added
> > devices.  In those cases, Linux assigns them.
>
> Once a hotplug event is detected (either via a PCIe hotplug interrupt
> or an ACPI event), which kernel function is responsible for
> dynamically assigning the Bus, Device, and Function (BDF) number to
> the new device?

The kernel doesn't assign the BDF.  The bus number is the bus on which
the new device lives.  This is determined by the PCI host bridge and
any bridges in the path to the device.  The PCI bus number below a
host bridge is determined by firmware or a native host bridge driver.

The bus numbers below other bridges are configured either by firmware
or by the Linux core setting the bridge secondary bus number.  This is
mostly in pci_scan_child_bus_extend().

The device and function numbers are basically a function of the
hardware device.

> Additionally, which function in the kernel assigns Base Address
> Registers (BARs) for the hotplugged device?

Start in pciehp_configure_device(), which calls
pci_assign_unassigned_bridge_resources():

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/hotplug/pciehp_pci.c?id=v6.13#n32