PCIe Hotplugging not working

PCIe Hotplugging not working

[Anand Kanaginhal]

I am trying to get PCIe endpoint hot - plugging working

The Set up is such that there is one endpoint (our custom device) connected to PCI  switch PLX 8609 8 port 8 lane switch. 
1. I have my driver module compiled, copied into /lib/modules/`uname -r`/kernel/drivers/char/
2. I run depmod and "modules.pcimap" also got updated with my module. 

However
 if I unplug and plug in the endpoint, my driver .probe and .remove functions
 are not being invoked. Is there something more to be done at the bus 
level to get the driver to hot plug ?

I have tried various kernel version, Currently I am trying on 2.6.38.12 (UBUNTU flavor).
First is there anything missing ? Also Please give me some pointers and debug direction.

Thanks
Anand

 		 	   		  

Re: PCIe Hotplugging not working

[Greg KH]

On Tue, Nov 29, 2011 at 03:56:12AM -0800, Anand Kanaginhal wrote:
> 
> 
> I am trying to get PCIe endpoint hot - plugging working
> 
> The Set up is such that there is one endpoint (our custom device) connected to PCI  switch PLX 8609 8 port 8 lane switch. 
> 1. I have my driver module compiled, copied into /lib/modules/`uname -r`/kernel/drivers/char/

Why is a pci driver a "char" driver?  Do you have a pointer to the
source of this?

> 2. I run depmod and "modules.pcimap" also got updated with my module. 
> 
> However
>  if I unplug and plug in the endpoint, my driver .probe and .remove functions
>  are not being invoked. Is there something more to be done at the bus 
> level to get the driver to hot plug ?

You need a pci hotplug controller driver, is that loaded?  That is what
is responsible for eventually calling your remove function.

> I have tried various kernel version, Currently I am trying on 2.6.38.12 (UBUNTU flavor).

We can't help you with distro-specific kernels, please use a kernel.org
kernel and we will be glad to help you out.

greg k-h

RE: PCIe Hotplugging not working

[Anand Kanaginhal]

Resending as earlier sends failed
> Date: Tue, 29 Nov 2011 23:22:14 +0900
> From: greg@kroah.com
> To: anand_dk@hotmail.com
> CC: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> Subject: Re: PCIe Hotplugging not working
> 
> On Tue, Nov 29, 2011 at 03:56:12AM -0800, Anand Kanaginhal wrote:
> > 
> > 
> > I am trying to get PCIe endpoint hot - plugging working
> > 
>
 > The Set up is such that there is one endpoint (our custom device) 
connected to PCI  switch PLX 8609 8 port 8 lane switch. 
> > 1. I have my driver module compiled, copied into /lib/modules/`uname -r`/kernel/drivers/char/
> 
> Why is a pci driver a "char" driver?  Do you have a pointer to the
> source of this?

I
 am doing an test before putting it in its right place. Another reason 
is that I am treating it as character device and using IOCTLs for 
various operations.
I have attached the framework code here.

> > 2. I run depmod and "modules.pcimap" also got updated with my module. 
> > 
> > However
> >  if I unplug and plug in the endpoint, my driver .probe and .remove functions
> >  are not being invoked. Is there something more to be done at the bus 
> > level to get the driver to hot plug ?
> 
> You need a pci hotplug controller driver, is that loaded?  That is what
> is responsible for eventually calling your remove function.

How do I figure out if a pci hotplug controller driver is installed? 


> > I have tried various kernel version, Currently I am trying on 2.6.38.12 (UBUNTU flavor).
> 
> We can't help you with distro-specific kernels, please use a kernel.org
> kernel and we will be glad to help you out.

I have tried with vanilla kernel as well with the same results.

> greg k-h

Thanks
Anand 		 	   		  

Re: PCIe Hotplugging not working

[Greg KH]

On Tue, Nov 29, 2011 at 09:50:41AM -0800, Anand Kanaginhal wrote:
> 
> Resending as earlier sends failed

You can not send html messages to vger.kernel.org mailing lists.

> > Why is a pci driver a "char" driver?  Do you have a pointer to the
> > source of this?
> 
> I am doing an test before putting it in its right place. Another
> reason is that I am treating it as character device and using IOCTLs
> for various operations.  I have attached the framework code here.

Just a note, but your framework is backwards, you should allocate your
character device in your probe function, not in the module init
function.

Also, what type of driver is this, why do you need ioctls for it?

> > > 2. I run depmod and "modules.pcimap" also got updated with my module. 
> > > 
> > > However
> > >  if I unplug and plug in the endpoint, my driver .probe and .remove functions
> > >  are not being invoked. Is there something more to be done at the bus 
> > > level to get the driver to hot plug ?
> > 
> > You need a pci hotplug controller driver, is that loaded?  That is what
> > is responsible for eventually calling your remove function.
> 
> How do I figure out if a pci hotplug controller driver is installed? 

How do you know that your hardware supports PCI hotplugging at all?

You need a PCI hotplug controller in the system, generally if you have
one, the driver will be loaded for it, but if you are using a custom
kernel, you might have to build and install it yourself.

> > > I have tried various kernel version, Currently I am trying on 2.6.38.12 (UBUNTU flavor).
> > 
> > We can't help you with distro-specific kernels, please use a kernel.org
> > kernel and we will be glad to help you out.
> 
> I have tried with vanilla kernel as well with the same results.

What release did you try?

greg k-h

Re: PCIe Hotplugging not working

[Ludvig Petrossian]

On 29/11/11 12:56, Anand Kanaginhal wrote:
>
> I am trying to get PCIe endpoint hot - plugging working
>
> The Set up is such that there is one endpoint (our custom device) connected to PCI  switch PLX 8609 8 port 8 lane switch. 
> 1. I have my driver module compiled, copied into /lib/modules/`uname -r`/kernel/drivers/char/
> 2. I run depmod and "modules.pcimap" also got updated with my module. 
>
> However
>  if I unplug and plug in the endpoint, my driver .probe and .remove functions
>  are not being invoked. Is there something more to be done at the bus 
> level to get the driver to hot plug ?
>
> I have tried various kernel version, Currently I am trying on 2.6.38.12 (UBUNTU flavor).
> First is there anything missing ? Also Please give me some pointers and debug direction.
>
> Thanks
> Anand
>
>  		 	   		  --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Hi Anand

Did You enable the pcie hotplug driver (pciehp)
One can do it adding the next options to the /boot/grub/menu.lst
pciehp.pciehp_force=1 pciehp.pciehp_debug=1
(example of my menu.lst: kernel        /boot/vmlinuz-2.6.32-28-generic
root=UUID=330631e1-99a3-46d3-bf52-f4ca23c6b6ef ro pciehp.pciehp_force=1
pciehp.pciehp_debug=1 acpi=off noapic console=tty0 console=ttyS0 quiet )

so You can see the pciehp outputs in /var/log/kern.log

regards

Ludwig

Re: PCIe Hotplugging not working

[Matthew Garrett]

Make sure that you have CONFIG_HOTPLUG_PCI_ACPI=y and 
CONFIG_HOTPLUG_PCI_PCIE=y.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

RE: PCIe Hotplugging not working

[Anand Kanaginhal]

Hi Mathew,
I have following options set 

CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_HOTPLUG_PCI_ACPI=m

Hi -Greg,
lspci -k output results indicate the standard hotplug controller module controls this device ? 

03:09.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
    Kernel driver in use: pcieport
    Kernel modules: shpchp
04:00.0 Memory controller: Device 174a:0701
    Kernel driver in use: pcidrv



Thanks
Anand

----------------------------------------
> Date: Wed, 30 Nov 2011 15:40:22 +0000
> From: mjg59@srcf.ucam.org
> To: anand_dk@hotmail.com
> CC: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> Subject: Re: PCIe Hotplugging not working
>
> Make sure that you have CONFIG_HOTPLUG_PCI_ACPI=y and
> CONFIG_HOTPLUG_PCI_PCIE=y.
>
> --
> Matthew Garrett | mjg59@srcf.ucam.org
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
 		 	   		  

Re: PCIe Hotplugging not working

[Matthew Garrett]

On Wed, Nov 30, 2011 at 09:18:39AM -0800, Anand Kanaginhal wrote:
> 
> Hi Mathew,
> I have following options set 
> 
> CONFIG_HOTPLUG_PCI_PCIE=y
> CONFIG_HOTPLUG_PCI_ACPI=m

Please try with this built in, not as a module. What kind of board are 
you using?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: PCIe Hotplugging not working

[Bjorn Helgaas]

On Wed, Nov 30, 2011 at 11:24 AM, Matthew Garrett <mjg59@srcf.ucam.org> wrote:
> On Wed, Nov 30, 2011 at 09:18:39AM -0800, Anand Kanaginhal wrote:
>>
>> Hi Mathew,
>> I have following options set
>>
>> CONFIG_HOTPLUG_PCI_PCIE=y
>> CONFIG_HOTPLUG_PCI_ACPI=m
>
> Please try with this built in, not as a module. What kind of board are
> you using?

Is this resolved?  If not, a complete dmesg log would be interesting.
You might also try current upstream in case there were relevant
changes since 2.6.38.12.

RE: PCIe Hotplugging not working

[Anand Kanaginhal]

I am afraid this issue is not resolved, however to give 
complete picture and my understanding so far, I have the following description:
In the diagram:
---------------
p1, p8 p9 are downstream hotplug slots(ports) on switch 8533.  
c1, c2 are the custom designed pcie cards for which I have my driver.


    +----------+
    |          |
    |          |
    |  CPUS    |
    |          |
    |          |
    +----+-----+
         |
         |
         |
         |
    +----+-----+                    +-------------+               
    |          |                    |             | 
    |          |                    |  PEX 8533   |
    |  ROOT    |--------------------|  switch     |
    |  CMPLX   |                    | (10b5:8533) |
    |          |                    |             |
    +----------+                    +-------------+
                                      |    |    |
                                    p1|  p8|  p9|
                                      |    |    |
                                      c1   c2   |
                                                |
                                                |
                                                |
                                                |
                                       +-------------+               
                                       |             | 
                                       |   Another   |
                                       |   switch    |
                                       |   (chain)   |
                                       |             |
                                       +-------------+



I have built 2.6.38.13 linux kernel with following options set:
CONFIG_HOTPLUG=y
CONFIG_HOTPLUG_CPU=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_FAKE=y
CONFIG_HOTPLUG_PCI_COMPAQ=m
CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM=y
CONFIG_HOTPLUG_PCI_IBM=m
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=y
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=y

--------------------------------------------------------------------------------
Please correct me if any of my understanding below is incorrect:
1. With the above setup, If I hotplug card c1 and/or c2, will my driver probe 
get invoked. 
2. I dont have to add any rules for in /etc/udev/rules.d/*.rules for my driver
 to work. 
3. The standard hotplug controller driver would get the hotplug event from the
 switch and handle it. 
4. The standard hotplug controller driver will be responsible for invoking 
.probe and .remove functions of my driver.
---
My simplified version of driver is pasted below, I understand that I create a char dev before any of the device is found, however my main concern is detection of card insertion and removal.
-----------



static struct pci_device_id my_pci_tbl[] = {
       { PCI_DEVICE (0x174a, 0x0701) },
       { PCI_DEVICE (0x174a, 0x0801) },
       { 0, }
       };


MODULE_DEVICE_TABLE(pci, my_pci_tbl);

static struct pci_driver my_pci_driver =
{
    .name       = DRV_NAME,
    .probe      = my_pci_device_init,
    .remove     = __devexit_p(my_pci_device_remove),
    .id_table   = my_pci_tbl,
};


/*******************************************************************************
    module init 
******************************************************************************/
static int __init my_pci_module_init(void)
{

#ifdef MODULE
    printk(version);
#endif
    major = register_chrdev(0, DRV_NAME, &pcie_fops);
    if (major <= 0) {
        return -1;
    }

    /* Now register the PCI drivers */
    return pci_register_driver(&my_pci_driver);

}
/*******************************************************************************
    module_cleanup
 ******************************************************************************/
static void __exit my_pci_module_cleanup(void)
{

    if (major) {
        unregister_chrdev(major, DRV_NAME);
    }

    /* delete IO memory allocated */
    /* unregister pci driver */
    pci_unregister_driver (&my_pci_driver);
}

/*******************************************************************************
probe 
******************************************************************************/
static int my_pci_device_init (struct          pci_dev       *pdev, 
                               const struct    pci_device_id *ent )
{
    /* probe and device inits */
} 

/*******************************************************************************
remove
 ******************************************************************************/
static void __devexit my_pci_device_remove (struct pci_dev *pdev)
{
    /* remove operation */
}

/*******************************************************************************
 ******************************************************************************/
module_init(my_pci_module_init);
module_exit(my_pci_module_cleanup);

Thanks 
Anand

----------------------------------------
> From: bhelgaas@google.com
> Date: Wed, 7 Dec 2011 10:23:26 -0700
> Subject: Re: PCIe Hotplugging not working
> To: mjg59@srcf.ucam.org
> CC: anand_dk@hotmail.com; greg@kroah.com; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
>
> On Wed, Nov 30, 2011 at 11:24 AM, Matthew Garrett <mjg59@srcf.ucam.org> wrote:
> > On Wed, Nov 30, 2011 at 09:18:39AM -0800, Anand Kanaginhal wrote:
> >>
> >> Hi Mathew,
> >> I have following options set
> >>
> >> CONFIG_HOTPLUG_PCI_PCIE=y
> >> CONFIG_HOTPLUG_PCI_ACPI=m
> >
> > Please try with this built in, not as a module. What kind of board are
> > you using?
>
> Is this resolved? If not, a complete dmesg log would be interesting.
> You might also try current upstream in case there were relevant
> changes since 2.6.38.12.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
 		 	   		  

Re: PCIe Hotplugging not working

[Matthew Garrett]

Do you have anything in /sys/class/pci/slots ? Does your card show up in 
lspci when you hotplug it?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

RE: PCIe Hotplugging not working

[Anand Kanaginhal]

$ ls /sys/class/pci_bus
0000:00  0000:02  0000:04  0000:06  0000:08
0000:01  0000:03  0000:05  0000:07  0000:09

The card doesn't show up in lspci after hotplugging ( or doesnt disappear when pulled out), 
However the card shows up if the card is plugged in while booting up the PC. 

Below is the lspci output when the cards are connected at booting.

$ lspci 
...
02:00.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
03:01.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
03:08.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
03:09.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)

anand card c1>>>>>>>>04:00.0 Memory controller: Device 174a:0701  
anand card c2>>>>>>>>05:00.0 Ethernet controller: Agere Systems ET-131x PCI-E Ethernet Controller (rev 02) 
....
$ 
Also wanted to know if my assumptions below are correct ?
--------------------------------------------------------------------------------
Please correct me if any of my understanding below is incorrect:
1. With the above setup, If I hotplug card c1 and/or c2, will my driver probe 
get invoked. 
2. I dont have to add any rules for in /etc/udev/rules.d/*.rules for my driver
 to work. 
3. The standard hotplug controller driver would get the hotplug event from the
 switch and handle it. 
4. The standard hotplug controller driver will be responsible for invoking 
.probe and .remove functions of my driver.
---


Thanks
Anand



> Date: Fri, 9 Dec 2011 12:45:25 +0000
> From: mjg59@srcf.ucam.org
> To: anand_dk@hotmail.com
> CC: bhelgaas@google.com; greg@kroah.com; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> Subject: Re: PCIe Hotplugging not working
> 
> Do you have anything in /sys/class/pci/slots ? Does your card show up in 
> lspci when you hotplug it?
> 
> -- 
> Matthew Garrett | mjg59@srcf.ucam.org
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
 		 	   		  

Re: PCIe Hotplugging not working

[Greg KH]

A: No.
Q: Should I include quotations after my reply?

http://daringfireball.net/2007/07/on_top

On Fri, Dec 09, 2011 at 07:03:48AM -0800, Anand Kanaginhal wrote:
> 
> $ ls /sys/class/pci_bus
> 0000:00  0000:02  0000:04  0000:06  0000:08
> 0000:01  0000:03  0000:05  0000:07  0000:09

That's not what was asked, please answer the original question.

greg k-h

RE: PCIe Hotplugging not working

[Anand Kanaginhal]

Following Daring fireball :).. my comments inline----------------------------------------
> Date: Fri, 9 Dec 2011 12:45:25 +0000
> From: mjg59@srcf.ucam.org
> To: anand_dk@hotmail.com
> CC: bhelgaas@google.com; greg@kroah.com; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> Subject: Re: PCIe Hotplugging not working
>
> Do you have anything in /sys/class/pci/slots ? Does your card show up in
> lspci when you hotplug it?
I dont see the directory structure /sys/class/pci/slots 

$ ls /sys/class   
ata_device  dmi		 leds	   power_supply  scsi_disk	tty
ata_link    firmware	 mdio_bus  ppdev	 scsi_generic	usbmon
ata_port    gpio	 mem	   ppp		 scsi_host	vc
backlight   graphics	 misc	   printer	 sound		vtconsole
bdi	    hidraw	 mmc_host  regulator	 spi_host	wmi
block	    hwmon	 net	   rfkill	 spi_master
bsg	    i2c-adapter  pci_bus   rtc		 spi_transport
dma	    input	 pktcdvd   scsi_device	 thermaldirectories. 


However, slots was found in /sys/bus/pci/slots

$ ls /sys/bus/pci/slots
0000:00:00.0  0000:00:10.0  0000:00:1c.0  0000:00:1f.2	0000:03:01.0
0000:00:03.0  0000:00:10.1  0000:00:1c.4  0000:00:1f.3	0000:03:08.0
0000:00:08.0  0000:00:16.0  0000:00:1c.6  0000:00:1f.5	0000:03:09.0
0000:00:08.1  0000:00:19.0  0000:00:1d.0  0000:01:00.0	0000:04:00.0
0000:00:08.2  0000:00:1a.0  0000:00:1e.0  0000:01:00.1	0000:07:00.0
0000:00:08.3  0000:00:1b.0  0000:00:1f.0  0000:02:00.0	0000:08:00.0

lspci neither adds nor removes entries on hotplugging either card c1,c2.
Output of lspci is as follows 

$ lspci 
...
02:00.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
03:01.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
03:08.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
03:09.0 PCI bridge: PLX Technology, Inc. PEX 8533 32-lane, 6-port PCI Express Switch (rev aa)
 
anand card c1>>>>>>>>04:00.0 Memory controller: Device 174a:0701  
anand card c2>>>>>>>>05:00.0 Ethernet controller: Agere Systems ET-131x PCI-E Ethernet Controller (rev 02) 
....
$ 
Also along the other stuffs, for the topology I am having, please let me know if my assumptions are technically correct
---
1. With the above setup, If I hotplug card c1 and/or c2, will my driver probe 
get invoked. 
2. I dont have to add any rules for in /etc/udev/rules.d/*.rules for my driver
 to work. 
3. The standard hotplug controller driver would get the hotplug event from the
 switch and handle it. 
4. The standard hotplug controller driver will be responsible for invoking 
.probe and .remove functions of my driver.
---
Thanks
Anand


>
> --
> Matthew Garrett | mjg59@srcf.ucam.org
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
 		 	   		  

Re: PCIe Hotplugging not working

[Greg KH]

On Fri, Dec 09, 2011 at 09:19:39AM -0800, Anand Kanaginhal wrote:
> 1. With the above setup, If I hotplug card c1 and/or c2, will my driver probe 
> get invoked. 

Stop right here.  Please get your pci hotplug controller driver working
first, for your platform.  That seems to be the problem.  After that,
the rest of your questions should just resolve themselves.

greg k-h

Re: PCIe Hotplugging not working

[Greg KH]

On Fri, Dec 09, 2011 at 01:17:53PM -0800, Anand Kanaginhal wrote:
> 
> 
> > Date: Fri, 9 Dec 2011 10:07:08 -0800
> > From: greg@kroah.com
> > To: anand_dk@hotmail.com
> > CC: mjg59@srcf.ucam.org; bhelgaas@google.com; linux-pci@vger.kernel.org;
> linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> > Subject: Re: PCIe Hotplugging not working
> >
> > On Fri, Dec 09, 2011 at 09:19:39AM -0800, Anand Kanaginhal wrote:
> > > 1. With the above setup, If I hotplug card c1 and/or c2, will my driver
> probe
> > > get invoked.
> >
> > Stop right here. Please get your pci hotplug controller driver working
> > first, for your platform. That seems to be the problem. After that,
> > the rest of your questions should just resolve themselves.
> Thank you Greg for your assistance, perhaps it would be little more helpful, if
> you could tell me the basis on which you drew the conclusion. I am no kernel
> expert :( 

As your driver's callbacks aren't being called, that's probably why this
isn't working.

Please make sure your pci hotplug driver is working first.  Try turning
on and off the power to the card slot (in the sysfs directory for the
slot) and see if that's working.

You also have failed to say exactly what pci hotplug controller you have
in your system, what is it?  You have to have one, and the driver loaded
for it, for any of this to work at all.

greg k-h

RE: PCIe Hotplugging not working

[Anand Kanaginhal]

Thanks for the feedback again Greg, MY Comments inline

> Date: Fri, 9 Dec 2011 14:25:20 -0800
> From: greg@kroah.com
> To: anand_dk@hotmail.com
> CC: mjg59@srcf.ucam.org; bhelgaas@google.com; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> Subject: Re: PCIe Hotplugging not working
> 
> On Fri, Dec 09, 2011 at 01:17:53PM -0800, Anand Kanaginhal wrote:
> > 
> > 
> > > Date: Fri, 9 Dec 2011 10:07:08 -0800
> > > From: greg@kroah.com
> > > To: anand_dk@hotmail.com
> > > CC: mjg59@srcf.ucam.org; bhelgaas@google.com; linux-pci@vger.kernel.org;
> > linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
> > > Subject: Re: PCIe Hotplugging not working
> > >
> > > On Fri, Dec 09, 2011 at 09:19:39AM -0800, Anand Kanaginhal wrote:
> > > > 1. With the above setup, If I hotplug card c1 and/or c2, will my driver
> > probe
> > > > get invoked.
> > >
> > > Stop right here. Please get your pci hotplug controller driver working
> > > first, for your platform. That seems to be the problem. After that,
> > > the rest of your questions should just resolve themselves.
> > Thank you Greg for your assistance, perhaps it would be little more helpful, if
> > you could tell me the basis on which you drew the conclusion. I am no kernel
> > expert :( 
> 
> As your driver's callbacks aren't being called, that's probably why this
> isn't working.
> 
Right that makes sense

> Please make sure your pci hotplug driver is working first.  Try turning
> on and off the power to the card slot (in the sysfs directory for the
> slot) and see if that's working.
When I do this, my driver probe and remove gets invoked. 

> You also have failed to say exactly what pci hotplug controller you have
> in your system, what is it?  You have to have one, and the driver loaded
> for it, for any of this to work at all.

We are using plx 8533 in our system, it hasnt provided controller driver for linux, however the support team insists that any hotplug controller driver which is compliant to PCI SIG, should work fine.My assumption was shpc in linux would work just fine. 

ThanksAnand
> greg k-h
 		 	   		   		 	   		  

Re: PCIe Hotplugging not working

[Greg KH]

On Fri, Dec 09, 2011 at 03:24:00PM -0800, Anand Kanaginhal wrote:
> > Please make sure your pci hotplug driver is working first.  Try turning
> > on and off the power to the card slot (in the sysfs directory for the
> > slot) and see if that's working.
> When I do this, my driver probe and remove gets invoked. 

Then part of the pci hotplug controller driver is working, good.

> > You also have failed to say exactly what pci hotplug controller you have
> > in your system, what is it?  You have to have one, and the driver loaded
> > for it, for any of this to work at all.
> 
> We are using plx 8533 in our system,

What exactly is that?

> it hasnt provided controller driver for linux, however the support
> team insists that any hotplug controller driver which is compliant to
> PCI SIG, should work fine.My assumption was shpc in linux would work
> just fine. 

The PCI Hotplug SIG said that ACPI would be the way forward for "PCI"
hotplug capabilities.

For PCI Express, the pciehp driver should work for you, as that follows
the spec for PCI Express devices.

So it depends on your hardware, what pci hotplug driver do you have
loaded, and what does the kernel log messages say when you load it, and
then what does it say when you remove your device?

Remember, PCI devices are not supposed to be hot-removed, you have to
tell the OS ahead of time it is going to go away.  PCI Express is a bit
different, depending on the type of device (ExpressCard is allowed to be
removed without notifying the OS ahead of time, some other PCI Express
devices are not, see the spec for details.)

greg k-h

Re: PCIe Hotplugging not working

[John Stoffel]

>>>>> "Greg" == Greg KH <greg@kroah.com> writes:

Greg> On Fri, Dec 09, 2011 at 01:17:53PM -0800, Anand Kanaginhal wrote:
>> 
>> 
>> > Date: Fri, 9 Dec 2011 10:07:08 -0800
>> > From: greg@kroah.com
>> > To: anand_dk@hotmail.com
>> > CC: mjg59@srcf.ucam.org; bhelgaas@google.com; linux-pci@vger.kernel.org;
>> linux-kernel@vger.kernel.org; linux-hotplug@vger.kernel.org
>> > Subject: Re: PCIe Hotplugging not working
>> >
>> > On Fri, Dec 09, 2011 at 09:19:39AM -0800, Anand Kanaginhal wrote:
>> > > 1. With the above setup, If I hotplug card c1 and/or c2, will my driver
>> probe
>> > > get invoked.
>> >
>> > Stop right here. Please get your pci hotplug controller driver working
>> > first, for your platform. That seems to be the problem. After that,
>> > the rest of your questions should just resolve themselves.
>> Thank you Greg for your assistance, perhaps it would be little more helpful, if
>> you could tell me the basis on which you drew the conclusion. I am no kernel
>> expert :( 

Greg> As your driver's callbacks aren't being called, that's probably why this
Greg> isn't working.

Greg> Please make sure your pci hotplug driver is working first.  Try turning
Greg> on and off the power to the card slot (in the sysfs directory for the
Greg> slot) and see if that's working.

Greg> You also have failed to say exactly what pci hotplug controller
Greg> you have in your system, what is it?  You have to have one, and
Greg> the driver loaded for it, for any of this to work at all.


And it would make sense to test your hotplug setup with a known
working card that *does* support hotplugging properly already.  Then
you know your setup works and you can then concentrate on getting your
new card working.  

John