From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from manexowa.empirix.com (unknown [66.151.190.243]) by ozlabs.org (Postfix) with ESMTP id 621CFB70B3 for ; Wed, 23 Sep 2009 05:42:07 +1000 (EST) From: "Morrison, Tom" To: "linuxppc-dev@ozlabs.org" , "linux-kernel@vger.kernel.org" Date: Tue, 22 Sep 2009 15:36:22 -0400 Subject: PCI HotPlug and Adding Resources after Linux Boots Message-ID: <80955630EE34634AB5030FB8CE9BDCEFD7459DF2@EXBEDCLST.empirix.com> Content-Type: multipart/alternative; boundary="_000_80955630EE34634AB5030FB8CE9BDCEFD7459DF2EXBEDCLSTempiri_" MIME-Version: 1.0 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , --_000_80955630EE34634AB5030FB8CE9BDCEFD7459DF2EXBEDCLSTempiri_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I am not exactly sure who to direct this question to (general Linux kernel = or LinuxPPC), so I am directing to both - in hopes that someone will recognize this probl= em - and perhaps give me some suggestions on how to proceed... I am running Linux (2.6.23x (and 2.6.27.x)) on a MPC8572 based system. I have an 8616 switch that has a Port (6) connected to a FPGA that is NOT loaded at before Linux boots (note: this port is configured for HOTPLUG events - which we do get after FPGA is loaded). We are NOT using a static device tree map (because the devices in the system are very dynamic)= . We use instead the pci auto scan mechanism(s) to scan/assign resources (including into the BAR registers) at bootup to all of the devices that are attached to this MPC8572... Here is the port that is attached to the device (note: there are NO resources assigned at this point this port): ---------------------------------------------------------------------------= ---------------------- 02:06.0 PCI bridge: PLX Technology, Inc.: Unknown device 8616 (rev bb) (pro= g-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=3D02, secondary=3D05, subordinate=3D05, sec-latency=3D= 0 Capabilities: [40] Power Management version 3 Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=3D0/2= Enable+ Capabilities: [68] #10 [0162] Capabilities: [a4] #0d [0000] root@slave7 ~ # lspci -t -+-[01]---00.0-[02-05]--+-01.0 | +-04.0-[03]-- | +-05.0-[04]-- | \-06.0-[05]- ---------------------------------------------------------------------------= ---------------------- Later, after I detect there is an FPGA to load - I load it. At completion o= f the loading of the FPGA - the 8616 detects the FPGA - and creates a HotPlug event that the PCI Express HotPlug Driver handles: ---------------------------------------------------------------------------= ---------------------- root@slave7 ~ # pciehp: pcie_isr: intr_loc 8 pciehp: pciehp: Presence/Notify input change. pciehp: Card present on Slot(0005_0070) pciehp: Surprise Removal pciehp: hpc_get_power_status: SLOTCTRL 80 value read 8 pciehp: hpc_get_attention_status: SLOTCTRL 80, value read 8 pciehp: board_added: slot device, slot offset, hp slot =3D 0, 0 ,0 pciehp: hpc_check_lnk_status: lnk_status =3D 2021 PCI: Found 0000:05:00.0 [1172/0004] 00ff00 00 PCI: Calling quirk c0012d3c for 0000:05:00.0 program_fw_provided_values: Could not get hotplug parameters entering assign resources (size: 2000000) PCI: Failed to allocate mem resource #0:2000000@0 for 0000:05:00.0 bus pci: add device 0000:05:00.0 entering uevent pci: Trying to Match Device 0000:05:00.0 with Driver pcieport-driver pci: Trying to Match Device 0000:05:00.0 with Driver serial pci: Trying to Match Device 0000:05:00.0 with Driver pexntb pciehp: hpc_get_power_status: SLOTCTRL 80 value read 8 pciehp: hpc_get_attention_status: SLOTCTRL 80, value read 8 02:06.0 PCI bridge: PLX Technology, Inc.: Unknown device 8616 (rev bb) (pro= g-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=3D02, secondary=3D05, subordinate=3D05, sec-latency=3D= 0 Capabilities: [40] Power Management version 3 Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=3D0/2= Enable+ Capabilities: [68] #10 [0162] Capabilities: [a4] #0d [0000] 05:00.0 Class ff00: Altera Corporation: Unknown device 0004 (rev 01) Subsystem: Altera Corporation: Unknown device 0004 Flags: fast devsel Capabilities: [50] Message Signalled Interrupts: 64bit+ Queue=3D0/5= Enable- Capabilities: [78] Power Management version 3 Capabilities: [80] #10 [0001] root@slave7 ~ # lspci -t -+-[01]---00.0-[02-05]--+-01.0 | +-04.0-[03]-- | +-05.0-[04]-- | \-06.0-[05]----00.0 \-[00]---00.0 ---------------------------------------------------------------------------= ---------------------- So, as you can see - the device has been read - and it requires 32M of reso= urces, but because its parent doesn't have any resources allocated - it seemingly can'= t allocate and use any additional resources. How do I 'customize' and/or add resources at this point for this device (us= ing semi-standard mechanisms)? Thanks in advance for any/all ideas... I Tom Morrison Principal Software Engineer EMPIRIX 20 Crosby Drive - Bedford, MA 01730 p: 781.266.3567 f: 781.266.3670 email: tmorrison@empirix.com www.empirix.com --_000_80955630EE34634AB5030FB8CE9BDCEFD7459DF2EXBEDCLSTempiri_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I am not exactly sure who to direct this question to (ge= neral Linux kernel or LinuxPPC),

so I am directing to both – in hopes that someone = will recognize this problem – and perhaps

give me some suggestions on how to proceed…

 

I am running Linux (2.6.23x (and 2.6.27.x)) on a MPC8572= based system.

 

I have an 8616 switch that has a Port (6) connected to a FPGA that is

NOT loaded at before Linux boots (note: this port is configured for HOTPLUG

events - which we do get after FPGA  is loaded). We= are NOT using a

static device tree map (because the devices in the syste= m are very dynamic).


We use instead the pci auto scan mechanism(s) to scan/assign resources

(including into the BAR registers) at bootup to all of t= he devices that are

attached to this MPC8572…=

 

Here is the port that is attached to the device (note: t= here are NO

resources assigned at this point this port):<= /span>

 

--------------------------------------------------------= -----------------------------------------

02:06.0 PCI bridge: PLX Techno= logy, Inc.: Unknown device 8616 (rev bb) (prog-if 00 [Normal decode])

     =    Flags: bus master, fast devsel, latency 0

     =    Bus: primary=3D02, secondary=3D05, subordinate=3D05, sec-latency=3D0

     =    Capabilities: [40] Power Management version 3

     =    Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=3D0/2 Enable+=

     =    Capabilities: [68] #10 [0162]

     =    Capabilities: [a4] #0d [0000]

 

root@slave7 ~ # lspci -t<= /o:p>

-+-[01]---00.0-[02-05]--+-01.0=

 |    = ;               &nb= sp;  +-04.0-[03]--

 |    = ;            &n= bsp;     +-05.0-[04]--

 |    = ;            &n= bsp;     \-06.0-[05]—

 

--------------------------------------------------------= -----------------------------------------

 

Later, after I detect there is an FPGA to load - I load = it. At completion of the

loading of the FPGA - the 8616  detects the FPGA &#= 8211; and creates a HotPlug

event that the PCI Express HotPlug Driver handles:<= /o:p>

--------------------------------------------------------= -----------------------------------------

 

root@slave7 ~ # pciehp: pcie_i= sr: intr_loc 8

pciehp: pciehp:  Presence/Notify input change.

pciehp: Card present on Slot(0005_0070)

pciehp: Surprise Removal<= /o:p>

pciehp: hpc_get_power_status: SLOTCTRL 80 value read 8

pciehp: hpc_get_attention_stat= us: SLOTCTRL 80, value read 8

pciehp: board_added: slot devi= ce, slot offset, hp slot =3D 0, 0 ,0

pciehp: hpc_check_lnk_status: lnk_status =3D 2021

PCI: Found 0000:05:00.0 [1172/= 0004] 00ff00 00

PCI: Calling quirk c0012d3c fo= r 0000:05:00.0

program_fw_provided_values: Co= uld not get hotplug parameters

entering assign resources (siz= e: 2000000)

PCI: Failed to allocate mem re= source #0:2000000@0 for 0000:05:00.0

bus pci: add device 0000:05:00= .0

entering uevent

pci: Trying to Match Device 0000:05:00.0 with Driver pcieport-driver

pci: Trying to Match Device 0000:05:00.0 with Driver serial

pci: Trying to Match Device 0000:05:00.0 with Driver pexntb

pciehp: hpc_get_power_status: SLOTCTRL 80 value read 8

pciehp: hpc_get_attention_stat= us: SLOTCTRL 80, value read 8

 

02:06.0 PCI bridge: PLX Techno= logy, Inc.: Unknown device 8616 (rev bb) (prog-if 00 [Normal decode])

     =    Flags: bus master, fast devsel, latency 0

     =    Bus: primary=3D02, secondary=3D05, subordinate=3D05, sec-latency=3D0

     =    Capabilities: [40] Power Management version 3

     =    Capabilities: [48] Message Signalled Interrupts: 64bit+ Queue=3D0/2 Enable+=

     =    Capabilities: [68] #10 [0162]

     =    Capabilities: [a4] #0d [0000]

 

05:00.0 Class ff00: Altera Corporation: Unknown device 0004 (rev 01)

     =    Subsystem: Altera Corporation: Unknown device 0004=

     =    Flags: fast devsel

     =    Capabilities: [50] Message Signalled Interrupts: 64bit+ Queue=3D0/5 Enable-=

     =    Capabilities: [78] Power Management version 3

     =    Capabilities: [80] #10 [0001]

 

root@slave7 ~ # lspci -t<= /o:p>

-+-[01]---00.0-[02-05]--+-01.0=

 |    = ;            &n= bsp;     +-04.0-[03]--

 |    = ;            &n= bsp;     +-05.0-[04]--

 |    = ;            &n= bsp;     \-06.0-[05]----00.0

 \-[00]---00.0=

 

--------------------------------------------------------= -----------------------------------------

 

So, as you can see – the device has been read R= 11; and it requires 32M of resources, but

because its parent doesn’t have any resources allo= cated – it seemingly can’t allocate and

use any additional resources.

 

How do I ‘customize’ and/or add resources at this point for this device (using semi-standard mechanisms)?

 

Thanks in advance for any/all ideas…

 

 

I

 

Tom Morrison
Principal Softw= are Engineer

EMPIRIX
20 Crosby Drive - Bedford, MA <= /font> 01730<= /span>
p= : 781.266.3567 f: 781.266.3670
email: tmorrison@empirix.com =
www.empirix.com


 

--_000_80955630EE34634AB5030FB8CE9BDCEFD7459DF2EXBEDCLSTempiri_--