Assign BAR address for pci device after hotplug

Assign BAR address for pci device after hotplug

[Chitturi, Divakar]

Hi

Below are two scenarios for which i would like to understand the right way to assign the BAR address for my PCI device

Device details:
pci switch upstrem port connected to root complex
pci switch downstream port 1 connected to my_pci_device
my_pci_device is FPGA
FPGA has bunch of controllers and other logic

Driver details:
Have a dedicated pci kernel driver for my_device
Enabled pcihp kernel driver
 
At power on, BIOS assigns/reserves specific region of memory for my pci device.
during the linux boot process, pci core will assign the bar and other initialization and call my
probe function in the driver. probe function will perform some initialization which includes reading
the BAR and setting some registers in IO space of the device. 

Scn1:
At some point the device is removed and reinserted

Scn2:
At some point similar device is inserted in a new pci slot

In both the scenarios explained about, BIOS is not involved as system was never turned off completely. So
1. how do we obtain the BAR address for the device. 
2. Should i assign it explicitly? 
3. if yes, should it be done in my driver ( in both the case i am assuming driver's probe function will be called )
4. can i use pci_assign_resource() in drivers probe function
5. is something like below the right way to do it ?

static int my_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
         if( pci_resource_start (pdev,BAR0) <= 0) {
                pr_debug(" Assign BAR0 \n");
	ret = pci_assign_resource(pdev,BAR0);
                if( 0 > ret) {
                    dev_err(&pdev->dev, " Failed to assign resource \n");
                    return ret;
                }
        }
		
       ret = pci_enable_device(pdev);
       ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
       base_addr = pci_iomap(pdev, BAR0, 0);
}

Thanks
Divakar

Re: Assign BAR address for pci device after hotplug

[Greg KH]

On Tue, Aug 02, 2016 at 09:11:09PM +0000, Chitturi, Divakar wrote:
> Hi
> 
> Below are two scenarios for which i would like to understand the right
> way to assign the BAR address for my PCI device
> 
> Device details:
> pci switch upstrem port connected to root complex
> pci switch downstream port 1 connected to my_pci_device
> my_pci_device is FPGA
> FPGA has bunch of controllers and other logic
> 
> Driver details:
> Have a dedicated pci kernel driver for my_device
> Enabled pcihp kernel driver

Which pci hotplug driver?  Just the PCI Hotplug core isn't going to do
anything, you need a controller driver for your specific PCI hotplug
hardware.

> At power on, BIOS assigns/reserves specific region of memory for my pci device.
> during the linux boot process, pci core will assign the bar and other initialization and call my
> probe function in the driver. probe function will perform some initialization which includes reading
> the BAR and setting some registers in IO space of the device. 
> 
> Scn1:
> At some point the device is removed and reinserted
> 
> Scn2:
> At some point similar device is inserted in a new pci slot

How are they added/removed without a hotplug controller?

> In both the scenarios explained about, BIOS is not involved as system was never turned off completely. So
> 1. how do we obtain the BAR address for the device. 

What architecture is this?  x86?  If so, it just uses whatever the BIOS
said was there for the last time.

> 2. Should i assign it explicitly? 

Yes.

> 3. if yes, should it be done in my driver ( in both the case i am assuming driver's probe function will be called )

No, in your pci hotplug driver, you need to handle it there.

> 4. can i use pci_assign_resource() in drivers probe function

Nope.

> 5. is something like below the right way to do it ?
> 
> static int my_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
>          if( pci_resource_start (pdev,BAR0) <= 0) {
>                 pr_debug(" Assign BAR0 \n");
> 	ret = pci_assign_resource(pdev,BAR0);
>                 if( 0 > ret) {
>                     dev_err(&pdev->dev, " Failed to assign resource \n");
>                     return ret;
>                 }
>         }
> 		
>        ret = pci_enable_device(pdev);
>        ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
>        base_addr = pci_iomap(pdev, BAR0, 0);
> }

You need to write a pci hotplug controller driver to manage the
resources and assign them properly.  That's its responsibility, see the
PCI hotplug specification for all of the details...

good luck!

greg k-h

RE: Assign BAR address for pci device after hotplug

[Chitturi, Divakar]

Thanks for your comments.
I am not sure if I can attach a picture to explain the HW config. Please find my comments for your questions inline.

Thanks
Divakar



-----Original Message-----
From: linux-hotplug-owner@vger.kernel.org [mailto:linux-hotplug-owner@vger.kernel.org] On Behalf Of Greg KH
Sent: Tuesday, August 02, 2016 10:12 PM
Cc: linux-hotplug@vger.kernel.org
Subject: Re: Assign BAR address for pci device after hotplug

On Tue, Aug 02, 2016 at 09:11:09PM +0000, Chitturi, Divakar wrote:
> Hi
> 
> Below are two scenarios for which i would like to understand the right 
> way to assign the BAR address for my PCI device
> 
> Device details:
> pci switch upstrem port connected to root complex pci switch 
> downstream port 1 connected to my_pci_device my_pci_device is FPGA 
> FPGA has bunch of controllers and other logic
> 
> Driver details:
> Have a dedicated pci kernel driver for my_device Enabled pcihp kernel 
> driver

Which pci hotplug driver?  Just the PCI Hotplug core isn't going to do anything, you need a controller driver for your specific PCI hotplug hardware.

There is a hotplug/hotswap controller in the plx switch but this is not being used. The hp signals are no connect. However the is a kernel driver loaded for this switch/bridge.
The power control for the downstream devices is handled by a master FPGA (apart from the fpga in downstream port of pcieswitch ) in the system. So the master fpga acts like a hotplug controller ?
If that's the case, then there is no driver for the master fpga. I plan to develop one. Even in this case what is/are the functions to reassign the bar for the downstream devices.

> At power on, BIOS assigns/reserves specific region of memory for my pci device.
> during the linux boot process, pci core will assign the bar and other 
> initialization and call my probe function in the driver. probe 
> function will perform some initialization which includes reading the BAR and setting some registers in IO space of the device.
> 
> Scn1:
> At some point the device is removed and reinserted
> 
> Scn2:
> At some point similar device is inserted in a new pci slot

How are they added/removed without a hotplug controller?

> In both the scenarios explained about, BIOS is not involved as system 
> was never turned off completely. So 1. how do we obtain the BAR address for the device.

What architecture is this?  x86?  If so, it just uses whatever the BIOS said was there for the last time.
Yes this is x86. When a new device (downstream fpga of pcie switch)  is inserted after linux boot, and the BIOS has reserved the memory region for the BUS on which new device sits, who's responsibility is to assign the BAR to the new device?
If it's the Hotplug controller drivers responsibility, then what are the functions that needs to be invoked. 

> 2. Should i assign it explicitly? 

Yes.

> 3. if yes, should it be done in my driver ( in both the case i am 
> assuming driver's probe function will be called )

No, in your pci hotplug driver, you need to handle it there.
I would like to understand how to handle in the pci hotplug driver

> 4. can i use pci_assign_resource() in drivers probe function

Nope.

Right now by using this function in the probe function of the kernel driver for the new device (downstream fpga of pcie switch ) inserted ( scn2 ) it is able to obtain the BAR and I can access the IO space and read /write.
However I am not convinced if this is the right way to do it.

> 5. is something like below the right way to do it ?
> 
> static int my_probe(struct pci_dev *pdev, const struct pci_device_id 
> *id) {
>          if( pci_resource_start (pdev,BAR0) <= 0) {
>                 pr_debug(" Assign BAR0 \n");
> 	ret = pci_assign_resource(pdev,BAR0);
>                 if( 0 > ret) {
>                     dev_err(&pdev->dev, " Failed to assign resource \n");
>                     return ret;
>                 }
>         }
> 		
>        ret = pci_enable_device(pdev);
>        ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
>        base_addr = pci_iomap(pdev, BAR0, 0); }

You need to write a pci hotplug controller driver to manage the resources and assign them properly.  That's its responsibility, see the PCI hotplug specification for all of the details...

Can you point me to a sample which handles the BAR assignment in hotplug scenario for me to understand better?

good luck!

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-hotplug" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Assign BAR address for pci device after hotplug

[Greg KH]

On Thu, Aug 04, 2016 at 05:09:21PM +0000, Chitturi, Divakar wrote:
> Thanks for your comments.
> I am not sure if I can attach a picture to explain the HW config.
> Please find my comments for your questions inline.

I can't distinguish your responses from my responses, please use an
email client that does this correctly and I'll be glad to respond.

thanks,

greg k-h

Re: Assign BAR address for pci device after hotplug

[divakar]

Hi Greg, 

Thanks for your comments. I managed to install a client which will do 
the auto format. Hopefully this is better than before. Please find my 
responses inline. Also added extra questions and information about the 
setup.

Questions:
1. Since the modified_my_driver_probe is working, is that the recommended 
way of doing it? I couldn't find any sample drivers using it. So wanted 
your guidance

2. There is no hotplug controller as far is know off in the setup ( 
assuming native x86 soc has root port which has inbuilt hp controller 
and pcieport driver takes care of it). Am i missing something?

3. When card 2 is inserted there is no hotplug event. Not sure why. 
Should anything be done from driver side ? I just started looking into 
the pcie switch eeprom config space settings to check if there is any 
mask for the event notification , assuming if masks cleared, pcieport 
driver can be notified by pciehp similar to card 1 insertion.


HW details:
x86 ( Soc) -> pcie root port 0 -> Device 0 (FPGA_0 )
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 1 -> Device 1 
(FPGA_1 ) ]
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 2 -> Device 2 ]
x86 ( Soc) -> pcie root port 2 ->   pcie-switch_2_port 1 -> [ pcie-
switch_3_port 1 -> Device 3 ( FPGA_2) ]
x86 ( Soc) -> pcie root port 2 ->   pcie-switch_2_port 1 -> [ pcie-
switch_3_port 2 -> Device 4 ]

card 1- pcie-switch_1, Device 1 and 2 are on the same physical pcb card 
which can be hotplugged ( inserted or removed )
card 2- pcie-switch_3, Device 3 and 4 are on the same physical pcb card 
which can be hotplugged ( inserted or removed )

Drivers details:
- FPGA_0-2 are devices have "my_pci_driver" 
- Device 2 and Device 4 are third party devices which have their drivers
- All pcie ports has handled by "pcieport" driver (output of lspci -vvv) 
- pciehp core driver is enabled

Sequence of operations:
- Boot up
- Insmod my_pci_driver, third party device driver
- Insert card 1 ( output sequence explained below happens )
- Insert card 2 ( nothing happens )
- issue /sys/pci/bus/rescan command ( output sequence explained below 
happens again )

Output sequence:
- pcieport driver see an event and will start enabling downstream 
devices
- my_pci_driver and third party device drivers probe functions get 
called
- third party driver probe function completes and the BAR is assigned ( 
output of lspci -vvv -s B:D:F )
- my_pci_driver probe function throws out an error and the probe fails. 
error as below

        pcieport 0000:54:05.0: enabling device (0000 -> 0002)
        my-driver 0000:56:00.3: can't enable device: BAR 0 [mem 
0x00000000-0x000fffff 64bit pref] not claimed
        my-driver 0000:56:00.3: Failed to enable My device rc=-22
        my-driver: probe of 0000:56:00.3 failed with error -22
        
Modify driver:
- when i add the pci_reassign_resource function before the 
pci_enable_device i nolonger see the error
- BAR is assigned in the lspci -vvv output

my_driver_probe:
static int my_probe(struct pci_dev *pdev, const struct pci_device_id 
*id)
{
        ret = pci_enable_device(pdev);
        if (ret) {
                dev_err(&pdev->dev,"Failed to enable my device 
rc=%d\n",ret);
                goto err_pci_enable;
        }

        ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
        if (ret) {
                dev_err(&pdev->dev, "Failed to request PCI regions 
rc=%d\n", ret);
                goto err_pci_req;
        }
        base_addr = pci_iomap(pdev, 0, 0);
        if (IS_ERR(base_addr)) {
                dev_err(&pdev->dev, "Failed to iomap PCI BAR %d\n", 0);
                goto err_pci_iomap;
        }
...
...
...
}


modified_my_driver_probe:
static int my_probe(struct pci_dev *pdev, const struct pci_device_id 
*id)
{
 
        /* Enable hotplug support
           TODO: Check if this is right way to do it */
        if( pci_resource_start (pdev,0) <= 0) {
                ret = pci_assign_resource(pdev,0);
                if( 0 > ret) {
                        dev_err(&pdev->dev, " Failed to assign resource 
\n");
                        return ret;
                }
        }

        ret = pci_enable_device(pdev);
        if (ret) {
                dev_err(&pdev->dev,"Failed to enable my device 
rc=%d\n",ret);
                goto err_pci_enable;
        }

        ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
        if (ret) {
                dev_err(&pdev->dev, "Failed to request PCI regions 
rc=%d\n", ret);
                goto err_pci_req;
        }
        base_addr = pci_iomap(pdev, 0, 0);
        if (IS_ERR(base_addr)) {
                dev_err(&pdev->dev, "Failed to iomap PCI BAR %d\n", 0);
                goto err_pci_iomap;
        }
}


-Divakar

On Wednesday, August 03, 2016 07:12:16 AM Greg KH wrote:
> On Tue, Aug 02, 2016 at 09:11:09PM +0000, Chitturi, Divakar wrote:
> > Hi
> > 
> > Below are two scenarios for which i would like to understand the
> > right way to assign the BAR address for my PCI device
> > 
> > Device details:
> > pci switch upstrem port connected to root complex
> > pci switch downstream port 1 connected to my_pci_device
> > my_pci_device is FPGA
> > FPGA has bunch of controllers and other logic
> > 
> > Driver details:
> > Have a dedicated pci kernel driver for my_device
> > Enabled pcihp kernel driver
> 
> Which pci hotplug driver?  Just the PCI Hotplug core isn't going to do
> anything, you need a controller driver for your specific PCI hotplug
> hardware.
> 
> > At power on, BIOS assigns/reserves specific region of memory for my
> > pci device. during the linux boot process, pci core will assign the
> > bar and other initialization and call my probe function in the
> > driver. probe function will perform some initialization which
> > includes reading the BAR and setting some registers in IO space of
> > the device.
> > 
> > Scn1:
> > At some point the device is removed and reinserted
> > 
> > Scn2:
> > At some point similar device is inserted in a new pci slot
> 
> How are they added/removed without a hotplug controller?
> 
> > In both the scenarios explained about, BIOS is not involved as
> > system was never turned off completely. So 1. how do we obtain the
> > BAR address for the device.
> 
> What architecture is this?  x86?  If so, it just uses whatever the
> BIOS said was there for the last time.

Yes this is x86 CPU. BIOS has reserved the memory region for the bus 
slot on which new device sits during boot. When a new device (downstream 
fpga of pcie switch)  is inserted and i issue the "echo 1 > 
/sys/bus/pci/rescan/ the device shows up in lscpi -t but no BAR is 
assigned. 

> 
> > 2. Should i assign it explicitly?
> 
> Yes.
> 
> > 3. if yes, should it be done in my driver ( in both the case i am
> > assuming driver's probe function will be called )
> No, in your pci hotplug driver, you need to handle it there.
> 
> > 4. can i use pci_assign_resource() in drivers probe function
> 
> Nope.
> 
> > 5. is something like below the right way to do it ?
> > 
> > static int my_probe(struct pci_dev *pdev, const struct pci_device_id
> > *id) {
> > 
> >          if( pci_resource_start (pdev,BAR0) <= 0) {
> >          
> >                 pr_debug(" Assign BAR0 \n");
> > 	
> > 	ret = pci_assign_resource(pdev,BAR0);
> > 	
> >                 if( 0 > ret) {
> >                 
> >                     dev_err(&pdev->dev, " Failed to assign resource
> >                     \n");
> >                     return ret;
> >                 
> >                 }
> >         
> >         }
> >        
> >        ret = pci_enable_device(pdev);
> >        ret = pci_request_regions(pdev,
> >        dev_driver_string(&pdev->dev));
> >        base_addr = pci_iomap(pdev, BAR0, 0);
> > 
> > }
> 
> You need to write a pci hotplug controller driver to manage the
> resources and assign them properly.  That's its responsibility, see
> the PCI hotplug specification for all of the details...

Right now by using this function in the probe function of the kernel 
driver for the new device (downstream fpga of pcie switch ) inserted ( 
scn2 ) it is able to obtain the BAR and I can access the IO space and 
read /write.
However I am not convinced if this is the right way to do it.


> 
> good luck!
> 
> greg k-h
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-hotplug" in the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Assign BAR address for pci device after hotplug

[Greg KH]

On Mon, Aug 08, 2016 at 05:32:42PM -0700, divakar wrote:
> Hi Greg, 
> 
> Thanks for your comments. I managed to install a client which will do 
> the auto format. Hopefully this is better than before. Please find my 
> responses inline. Also added extra questions and information about the 
> setup.
> 
> Questions:
> 1. Since the modified_my_driver_probe is working, is that the recommended 
> way of doing it? I couldn't find any sample drivers using it. So wanted 
> your guidance

I dont understand, what "modified" probe function?

> 2. There is no hotplug controller as far is know off in the setup ( 
> assuming native x86 soc has root port which has inbuilt hp controller 
> and pcieport driver takes care of it). Am i missing something?

Then your hardware does not support PCI hotplug, so I don't recommend
trying to use it.

Seriously, this is a hardware and BIOS requirement to support this, if
you don't have that hardware, you can't really work around it in the
kernel.  Go buy some real hardware for this please.

> 3. When card 2 is inserted there is no hotplug event. Not sure why. 

Why would there be?  You don't have hardware that notices this.

good luck,

greg k-h

Re: Assign BAR address for pci device after hotplug

[divakar]

Hi ,

The modified probe function was the one with the pci_reassign_resource 
function. 

modified_my_driver_probe:
static int my_probe(struct pci_dev *pdev, const struct pci_device_id 
*id)
{
 
        /* Enable hotplug support
           TODO: Check if this is right way to do it */
        if( pci_resource_start (pdev,0) <= 0) {
                ret = pci_assign_resource(pdev,0);
                if( 0 > ret) {
                        dev_err(&pdev->dev, " Failed to assign resource 
\n");
                        return ret;
                }
        }

        ret = pci_enable_device(pdev);

}

Thanks for your comments on the hardware. I hope my previous message 
with the output log snippets came through. My observation of the events 
happening seem to suggest there is a hotplug controller as i can see the 
difference between with and without enabling "pciehp". Also 
pci_resource_start in the probe function of the endpoint device driver 
seem to work. Hence my persistence in understanding the behavior. 

Thanks 


On Tuesday, August 09, 2016 02:49:39 AM Greg KH wrote:
> On Mon, Aug 08, 2016 at 05:32:42PM -0700, divakar wrote:
> > Hi Greg,
> > 
> > Thanks for your comments. I managed to install a client which will
> > do
> > the auto format. Hopefully this is better than before. Please find
> > my
> > responses inline. Also added extra questions and information about
> > the setup.
> > 
> > Questions:
> > 1. Since the modified_my_driver_probe is working, is that the
> > recommended way of doing it? I couldn't find any sample drivers
> > using it. So wanted your guidance
> 
> I dont understand, what "modified" probe function?
> 
> > 2. There is no hotplug controller as far is know off in the setup (
> > assuming native x86 soc has root port which has inbuilt hp
> > controller
> > and pcieport driver takes care of it). Am i missing something?
> 
> Then your hardware does not support PCI hotplug, so I don't recommend
> trying to use it.
> 
> Seriously, this is a hardware and BIOS requirement to support this, if
> you don't have that hardware, you can't really work around it in the
> kernel.  Go buy some real hardware for this please.
> 
> > 3. When card 2 is inserted there is no hotplug event. Not sure why.
> 
> Why would there be?  You don't have hardware that notices this.
> 
> good luck,
> 
> greg k-h

Re: Assign BAR address for pci device after hotplug

[Greg KH]

A: Top-posting.
Q: What is the most annoying thing in e-mail?

A: No.
Q: Should I include quotations after my reply?

http://daringfireball.net/2007/07/on_top

On Tue, Aug 09, 2016 at 10:00:22AM -0700, divakar wrote:
> Hi ,
> 
> The modified probe function was the one with the pci_reassign_resource 
> function. 

Don't do that.  That's for the pci core to use only.


> 
> modified_my_driver_probe:
> static int my_probe(struct pci_dev *pdev, const struct pci_device_id 
> *id)
> {
>  
>         /* Enable hotplug support
>            TODO: Check if this is right way to do it */
>         if( pci_resource_start (pdev,0) <= 0) {
>                 ret = pci_assign_resource(pdev,0);
>                 if( 0 > ret) {
>                         dev_err(&pdev->dev, " Failed to assign resource 
> \n");
>                         return ret;
>                 }
>         }
> 
>         ret = pci_enable_device(pdev);
> 
> }
> 
> Thanks for your comments on the hardware. I hope my previous message 
> with the output log snippets came through. My observation of the events 
> happening seem to suggest there is a hotplug controller as i can see the 
> difference between with and without enabling "pciehp". Also 
> pci_resource_start in the probe function of the endpoint device driver 
> seem to work. Hence my persistence in understanding the behavior. 

If the pciehp driver does not bind to your hardware, then it will not
work properly.

Again, PCI hotplug requires a PCI hotplug controller, or special
firmware/bios support for it.  You can't do it properly without it, read
the specification for all of the nasty details.

good luck!

greg k-h

Re: Assign BAR address for pci device after hotplug

[divakar]

May be i missed one piece of information. I thought i had that in my 
inline response but just realized i missed it. 

Device 0 ( FPGA 0), has custom logic which will detect the presence of 
the card 1 and card 2 and controls the power switches to the same. So If 
this makes fpga0 a "hotplug controller " then i can add the resource 
management logic/code for devices on card 1 and card 2 to this driver. 
Currently fpga_0 driver does some init for some of the devices on 
itself. can i follow any hp controller driver example to do the above or 
is there any specific driver/eg i can follow? noted your point on reading 
the spec for the details and i am on it. 

HW details:
x86 ( Soc) -> pcie root port 0 -> Device 0 (FPGA_0 )
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 1 -> Device 1 
(FPGA_1 ) ]

x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 2 -> Device 2 ]
x86 ( Soc) -> pcie root port 2 ->   pcie-switch_2_port 1 -> [ pcie-
switch_3_port 1 -> Device 3 ( FPGA_2) ]

x86 ( Soc) -> pcie root port 2 ->   pcie-switch_2_port 1 -> [ pcie-
switch_3_port 2 -> Device 4 ]

Thanks for your patience
Divakar

Re: Assign BAR address for pci device after hotplug

[Greg KH]

On Tue, Aug 09, 2016 at 11:48:59AM -0700, divakar wrote:
> May be i missed one piece of information. I thought i had that in my 
> inline response but just realized i missed it. 
> 
> Device 0 ( FPGA 0), has custom logic which will detect the presence of 
> the card 1 and card 2 and controls the power switches to the same. So If 
> this makes fpga0 a "hotplug controller " then i can add the resource 
> management logic/code for devices on card 1 and card 2 to this driver. 
> Currently fpga_0 driver does some init for some of the devices on 
> itself. can i follow any hp controller driver example to do the above or 
> is there any specific driver/eg i can follow? noted your point on reading 
> the spec for the details and i am on it. 

You might get away with it for your limited hardware platform, but all
bets are off if you plug it into something else.

Again, PCI resource management is handled by the firmware/bios and the
pci hotplug controller.  You can try to fake it out, with limited
success, but really, why do you want to do so?  Just use a real machine,
that's what the pci hotplug hardware was designed for.  It's much
cheaper than trying to work around it in the end, trust me...

greg k-h

Re: Assign BAR address for pci device after hotplug

[divakar]

I don't doubt the truth in your comments about the HW. 
Unfortunately i have to work with the HW  i have and i am trying to find 
the best possible way to do this. I can certainly provide feedback for 
the next product/design based on the learnings from the current one.
At this point these cards are not expected to be plugged into different 
machine. So based on your statements if i can fake it i will take that 
path as a starter. Is there a writeup/documentation/eg which i can 
follow to understand the required steps? for now i have started looking 
at /drivers/pci/hotplug/* . 

Again thanks for your patience and time. 
Thanks


On Tuesday, August 09, 2016 08:59:43 PM Greg KH wrote:
> On Tue, Aug 09, 2016 at 11:48:59AM -0700, divakar wrote:
> > May be i missed one piece of information. I thought i had that in my
> > inline response but just realized i missed it.
> > 
> > Device 0 ( FPGA 0), has custom logic which will detect the presence
> > of the card 1 and card 2 and controls the power switches to the
> > same. So If this makes fpga0 a "hotplug controller " then i can add
> > the resource management logic/code for devices on card 1 and card 2
> > to this driver. Currently fpga_0 driver does some init for some of
> > the devices on itself. can i follow any hp controller driver
> > example to do the above or is there any specific driver/eg i can
> > follow? noted your point on reading the spec for the details and i
> > am on it.
> 
> You might get away with it for your limited hardware platform, but all
> bets are off if you plug it into something else.
> 
> Again, PCI resource management is handled by the firmware/bios and the
> pci hotplug controller.  You can try to fake it out, with limited
> success, but really, why do you want to do so?  Just use a real
> machine, that's what the pci hotplug hardware was designed for.  It's
> much cheaper than trying to work around it in the end, trust me...
> 
> greg k-h

Re: Assign BAR address for pci device after hotplug

[Greg KH]

On Tue, Aug 09, 2016 at 05:26:12PM -0700, divakar wrote:
> I don't doubt the truth in your comments about the HW. 
> Unfortunately i have to work with the HW  i have and i am trying to find 
> the best possible way to do this. I can certainly provide feedback for 
> the next product/design based on the learnings from the current one.
> At this point these cards are not expected to be plugged into different 
> machine. So based on your statements if i can fake it i will take that 
> path as a starter. Is there a writeup/documentation/eg which i can 
> follow to understand the required steps? for now i have started looking 
> at /drivers/pci/hotplug/* . 

Nope, you are on your own, and get to keep the pieces of your kernel
when things blow up...

Please realize that you have already spent more money on this than it
would have cost to buy "real" hardware that will solve these issues for
you automatically :(

greg k-h