pciehp

pciehp

[Ludwig Petrosyan]

Hi All

we are working at DESY (Deutsches Elektronen-Synchrotron, Hamburg Germany)
and involved in evaluation of MTCA.4 standard (MTCA for Physics)
I have question concerning PCIe hotplag:
The Hot-Swap is working, I mean, if there was a PCIe endpoint card in
the PCIe slot this module could be later removed and inserted back in
running system. But if the Slot was empty and one insert the new PCIe
endpoint card in slot, the card will be connected in PCIe bus (I could
see it in lspci) but memory not enabled. Only after rebooting of the OS
the memory will be enabled.
For MTCA.4 standard it is very important to have possibility to add new
PCIe endpoint card in running system.
So, my question is:
 is there some OS boot parameter I could reserve memory for all PCIe
ports which hase slot implemented (if slot implemented on the PCIe
switch or RootComplex port this means later could be some PCIe endpoint
card inserted)

with best regards

Ludwig Petrosyan

Re: pciehp

[Bjorn Helgaas]

On Fri, Dec 13, 2013 at 7:35 AM, Ludwig Petrosyan
<ludwig.petrosyan@desy.de> wrote:
> Hi All
>
> we are working at DESY (Deutsches Elektronen-Synchrotron, Hamburg Germany)
> and involved in evaluation of MTCA.4 standard (MTCA for Physics)
> I have question concerning PCIe hotplag:
> The Hot-Swap is working, I mean, if there was a PCIe endpoint card in
> the PCIe slot this module could be later removed and inserted back in
> running system. But if the Slot was empty and one insert the new PCIe
> endpoint card in slot, the card will be connected in PCIe bus (I could
> see it in lspci) but memory not enabled. Only after rebooting of the OS
> the memory will be enabled.
> For MTCA.4 standard it is very important to have possibility to add new
> PCIe endpoint card in running system.
> So, my question is:
>  is there some OS boot parameter I could reserve memory for all PCIe
> ports which hase slot implemented (if slot implemented on the PCIe
> switch or RootComplex port this means later could be some PCIe endpoint
> card inserted)

If a Root Port or switch Downstream Port has the PCI_EXP_SLTCAP_HPC
(Slot Capabilities Hot-Plug Capable) bit is set, Linux should reserve
2MB for the port's MMIO aperture.  That isn't very much, but you can
change that amount with the "pci=hpmemsize=" boot option.  And I think
if you have firmware that configures a larger aperture, Linux will
keep that.

A complete dmesg log and "lspci -vv" output would be the place to
start investigating if Linux isn't doing this correctly.

Bjorn