* Re: pci hotplug problem
@ 2011-06-14 5:32 shivprashant
0 siblings, 0 replies; 3+ messages in thread
From: shivprashant @ 2011-06-14 5:32 UTC (permalink / raw)
To: linux-hotplug
Hi Kenji,
I too presumed that. The board (based on MPC8315erdb from Freescale) has
two PCIe controllers to which two slots are connected. The controllers
are compatible with PCI Express Base Specification Revision 1.0a and the
PCI Interface is compatible with PCI Local Bus Specification Rev. 2.3.
The driver makes use of PCI Hotplug controller core to
register/deregister hotplug slots.
Did I answer your question?
Regards,
Shivaprashanth
On Tuesday 14 June 2011 07:42 AM, Kenji Kaneshige wrote:
> Shivaprashanth,
>
> Sorry for my delayed responding.
> (I'm on vacation this week)
>
> I could not find any hotplug capable slot defined by PCIe spec in your
> lspci output. It should be the reason why pciehp detects no slot on
> your system. What kind of hotplug controller does your hotplug driver
> handle?
>
> Regards,
> Kenji Kaneshige
>
>
> (2011/06/09 18:30), shivprashant wrote:
>> below is the dmesg log after loading pciehp.ko with debug enabled
>> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>> -sh-2.05b# insmod /pciehp.ko pciehp_debug=1
>> pciehp: PCI Express Hot Plug Controller Driver version: 0.4
>> -sh-2.05b# dmesg
>> Using MPC831x RDB machine description
>> Linux version 2.6.36.1 (root@shiva-desktop) (gcc version 4.1.2) #5 Thu
>> Jun 9 14:40:17 IST 2011
>> Found legacy serial port 0 for /immr@e0000000/serial@4500
>> memà004500, taddrà004500, irq=0, clk\x133333333, speed=0
>> Found legacy serial port 1 for /immr@e0000000/serial@4600
>> memà004600, taddrà004600, irq=0, clk\x133333333, speed=0
>> Found FSL PCI host bridge at 0x00000000e0008500. Firmware bus number: 0->0
>> PCI host bridge /pci@e0008500 (primary) ranges:
>> MEM 0x0000000090000000..0x000000009fffffff -> 0x0000000090000000
>> MEM 0x0000000080000000..0x000000008fffffff -> 0x0000000080000000 Prefetch
>> IO 0x00000000e0300000..0x00000000e03fffff -> 0x0000000000000000
>> Found FSL PCI host bridge at 0x00000000e0009000. Firmware bus number: 0->255
>> PCI host bridge /pcie@e0009000 ranges:
>> MEM 0x0000000040000000..0x000000004fffffff -> 0x0000000040000000
>> IO 0x0000000051000000..0x00000000517fffff -> 0x0000000000000000
>> Found FSL PCI host bridge at 0x00000000e000a000. Firmware bus number: 0->255
>> PCI host bridge /pcie@e000a000 ranges:
>> MEM 0x0000000060000000..0x000000006fffffff -> 0x0000000060000000
>> IO 0x0000000071000000..0x00000000717fffff -> 0x0000000000000000
>> Top of RAM: 0x10000000, Total RAM: 0x10000000
>> Memory hole size: 0MB
>> Zone PFN ranges:
>> DMA 0x00000000 -> 0x00010000
>> Normal empty
>> Movable zone start PFN for each node
>> early_node_map[1] active PFN ranges
>> 0: 0x00000000 -> 0x00010000
>> On node 0 totalpages: 65536
>> free_area_init_node: node 0, pgdat c03fde3c, node_mem_map c0441000
>> DMA zone: 512 pages used for memmap
>> DMA zone: 0 pages reserved
>> DMA zone: 65024 pages, LIFO batch:15
>> Built 1 zonelists in Zone order, mobility grouping on. Total pages: 65024
>> Kernel command line: console=ttyS0,115200 root=/dev/nfs rw
>> nfsroot\x172.16.9.85:/home/shiva/Projects/AJA/pciehp-v2/aja-ltib/ro
>> otfs ip\x172.16.9.100:172.16.9.85:172.16.9.1:255.255.255.0::eth0:off
>> PID hash table entries: 1024 (order: 0, 4096 bytes)
>> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
>> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
>> Memory: 255496k/262144k available (3972k kernel code, 6648k reserved,
>> 144k data, 204k bss, 176k init)
>> Kernel virtual memory layout:
>> * 0xfffdf000..0xfffff000 : fixmap
>> * 0xfcef8000..0xfe000000 : early ioremap
>> * 0xd1000000..0xfcef8000 : vmalloc& ioremap
>> Hierarchical RCU implementation.
>> RCU-based detection of stalled CPUs is disabled.
>> Verbose stalled-CPUs detection is disabled.
>> NR_IRQS:512 nr_irqs:512
>> IPIC (128 IRQ sources) at d1000700
>> time_init: decrementer frequency = 33.333334 MHz
>> time_init: processor frequency = 400.000008 MHz
>> clocksource: timebase mult[77ffffd] shift[22] registered
>> clockevent: decrementer mult[888888b] shift[32] cpu[0]
>> Console: colour dummy device 80x25
>> pid_max: default: 32768 minimum: 301
>> Mount-cache hash table entries: 512
>> NET: Registered protocol family 16
>> alloc irq_desc for 38 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 38 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 38
>> PCI: Probing PCI hardware
>> pci 0000:00:00.0: reg 10: [mem 0x00000000-0x000fffff]
>> pci 0000:00:00.0: reg 18: [mem 0x00000000-0x0fffffff 64bit pref]
>> pci 0000:00:00.0: supports D1 D2
>> pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot
>> pci 0000:00:00.0: PME# disabled
>> pci 0000:00:10.0: reg 10: [mem 0x90000000-0x900007ff]
>> pci 0000:00:10.0: reg 14: [mem 0x90004000-0x90007fff]
>> pci 0000:00:10.0: supports D1 D2
>> pci 0000:00:10.0: PME# supported from D0 D1 D2 D3hot
>> pci 0000:00:10.0: PME# disabled
>> alloc irq_desc for 48 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 48 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 48
>> pci_bus 0000:00: resource 0 [io 0x0000-0xfffff]
>> pci_bus 0000:00: resource 1 [mem 0x90000000-0x9fffffff]
>> pci_bus 0000:00: resource 2 [mem 0x80000000-0x8fffffff pref]
>> pci_bus 0001:01: resource 0 [io 0xff7fe000-0xffffdfff]
>> pci_bus 0001:01: resource 1 [mem 0x40000000-0x4fffffff]
>> pci_bus 0002:02: resource 0 [io 0xfeffc000-0xff7fbfff]
>> pci_bus 0002:02: resource 1 [mem 0x60000000-0x6fffffff]
>> Registering ipic with sysfs...
>> bio: create slab<bio-0> at 0
>> vgaarb: loaded
>> SCSI subsystem initialized
>> libata version 3.00 loaded.
>> Freescale Elo / Elo Plus DMA driver
>> Switching to clocksource timebase
>> NET: Registered protocol family 2
>> IP route cache hash table entries: 2048 (order: 1, 8192 bytes)
>> TCP established hash table entries: 8192 (order: 4, 65536 bytes)
>> TCP bind hash table entries: 8192 (order: 3, 32768 bytes)
>> TCP: Hash tables configured (established 8192 bind 8192)
>> TCP reno registered
>> UDP hash table entries: 256 (order: 0, 4096 bytes)
>> UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
>> NET: Registered protocol family 1
>> RPC: Registered udp transport module.
>> RPC: Registered tcp transport module.
>> RPC: Registered tcp NFSv4.1 backchannel transport module.
>> PCI: CLS 32 bytes, default 32
>> alloc irq_desc for 16 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 9 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 16
>> alloc irq_desc for 17 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 10 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 17
>> alloc irq_desc for 77 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 77 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 77
>> alloc irq_desc for 18 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 16 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 18
>> alloc irq_desc for 71 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 71 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 71
>> of:fsl-elo-dma e00082a8.dma: #0 (fsl,elo-dma-channel), irq 71
>> of:fsl-elo-dma e00082a8.dma: #1 (fsl,elo-dma-channel), irq 71
>> of:fsl-elo-dma e00082a8.dma: #2 (fsl,elo-dma-channel), irq 71
>> of:fsl-elo-dma e00082a8.dma: #3 (fsl,elo-dma-channel), irq 71
>> alloc irq_desc for 32 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 32 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 32
>> alloc irq_desc for 33 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 33 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 33
>> alloc irq_desc for 34 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 34 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 34
>> alloc irq_desc for 44 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 44 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 44
>> alloc irq_desc for 45 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 45 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 45
>> alloc irq_desc for 67 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 67 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 67
>> alloc irq_desc for 19 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 4 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 19
>> alloc irq_desc for 81 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 81 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 81
>> alloc irq_desc for 82 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 82 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 82
>> alloc irq_desc for 86 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 86 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 86
>> alloc irq_desc for 87 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 87 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 87
>> alloc irq_desc for 88 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 88 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 88
>> alloc irq_desc for 89 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 89 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 89
>> Setting up Freescale MSI support
>> alloc irq_desc for 80 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 80 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 80
>> alloc irq_desc for 90 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 90 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 90
>> alloc irq_desc for 78 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 78 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 78
>> alloc irq_desc for 84 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 84 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 84
>> alloc irq_desc for 72 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 72 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 72
>> alloc irq_desc for 91 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 91 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 91
>> alloc irq_desc for 79 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 79 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 79
>> alloc irq_desc for 85 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 85 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 85
>> alloc irq_desc for 73 on node 0
>> alloc kstat_irqs on node 0
>> irq: irq 73 on host /immr@e0000000/interrupt-controller@700 mapped to
>> virtual irq 73
>> msgmni has been set to 499
>> io scheduler noop registered
>> io scheduler deadline registered
>> io scheduler cfq registered (default)
>> pci_hotplug: PCI Hot Plug PCI Core version: 0.5
>> Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
>> serial8250.0: ttyS0 at MMIO 0xe0004500 (irq = 16) is a 16550A
>> console [ttyS0] enabled
>> serial8250.0: ttyS1 at MMIO 0xe0004600 (irq = 17) is a 16550A
>> e0004500.serial: ttyS0 at MMIO 0xe0004500 (irq = 16) is a 16550
>> e0004600.serial: ttyS1 at MMIO 0xe0004600 (irq = 17) is a 16550
>> brd: module loaded
>> loop: module loaded
>> f8000000.flash: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer
>> ID 0x000001 Chip ID 0x002301
>> f8000000.flash: Found 1 x16 devices at 0x4000000 in 16-bit bank
>> Amd/Fujitsu Extended Query Table at 0x0040
>> Amd/Fujitsu Extended Query version 1.3.
>> number of CFI chips: 2
>> RedBoot partition parsing not available
>> Creating 8 MTD partitions on "f8000000.flash":
>> 0x000000000000-0x000002000000 : "AppA"
>> 0x000002000000-0x000004000000 : "AppB"
>> 0x000004000000-0x000006000000 : "SafeBoot"
>> 0x000007ea0000-0x000007ec0000 : "ConfigB"
>> 0x000007f60000-0x000007f80000 : "UserRegsA"
>> 0x000007f80000-0x000007fa0000 : "UserRegsB"
>> 0x000007fa0000-0x000007fc0000 : "ConfigA"
>> 0x000006400000-0x000006420000 : "UBootShared"
>> of:mpc8xxx_spi e0007000.spi: at 0xd103c000 (irq = 18), CPU mode
>> Fixed MDIO Bus: probed
>> eth0: Gianfar Ethernet Controller Version 1.2, 00:0c:17:88:02:35
>> eth0: Running with NAPI enabled
>> eth0: RX BD ring size for Q[0]: 256
>> eth0: TX BD ring size for Q[0]: 256
>> Freescale PowerQUICC MII Bus: probed
>> i2c /dev entries driver
>> of:mpc-i2c e0003000.i2c: timeout 1000000 us
>> rtc-ds1307 0-0068: rtc core: registered ds1339 as rtc0
>> TCP cubic registered
>> NET: Registered protocol family 17
>> rtc-ds1307 0-0068: setting system clock to 2011-06-09 02:04:50 UTC
>> (1307585090)
>> IP-Config: Complete:
>> device=eth0, addr\x172.16.9.100, mask%5.255.255.0, gw\x172.16.9.1,
>> host\x172.16.9.100, domain=, nis-domain=(none),
>> bootserver\x172.16.9.85, rootserver\x172.16.9.85, rootpath>> Looking up port of RPC 100003/2 on 172.16.9.85
>> PHY: mdio@e0024520:01 - Link is Up - 100/Full
>> Looking up port of RPC 100005/1 on 172.16.9.85
>> VFS: Mounted root (nfs filesystem) on device 0:12.
>> Freeing unused kernel memory: 176k init
>> pciehp: pcie_port_service_register = 0
>> pciehp: PCI Express Hot Plug Controller Driver version: 0.4
>> -sh-2.05b#
>> -sh-2.05b#
>> -sh-2.05b# lspci -vv
>> 00:00.0 Power PC: Freescale Semiconductor Inc Unknown device 00b5 (rev 12)
>> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
>> Stepping- SERR+ FastB2B-
>> Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSELúst>TAbort-<TAbort-
>> <MAbort+>SERR-<PERR-
>> Latency: 128, Cache Line Size: 32 bytes
>> Interrupt: pin A routed to IRQ 0
>> Region 2: Memory at<unassigned> (64-bit, prefetchable)
>> Region 4: Memory at<unassigned> (64-bit, non-prefetchable)
>> Capabilities: [48] #06 [0000]
>> Capabilities: [80] Power Management version 3
>> Flags: PMEClk+ DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
>> Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>>
>> 00:10.0 FireWire (IEEE 1394): Texas Instruments TSB82AA2 IEEE-1394b Link
>> Layer Controller (rev 02) (prog-if 10 [OHCI])
>> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
>> Stepping- SERR- FastB2B-
>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-
>> <TAbort-<MAbort->SERR-<PERR-
>> Latency: 128 (500ns min, 1000ns max), Cache Line Size: 32 bytes
>> Interrupt: pin A routed to IRQ 48
>> Region 0: Memory at 90000000 (32-bit, non-prefetchable) [size=2K]
>> Region 1: Memory at 90004000 (32-bit, non-prefetchable) [size\x16K]
>> Capabilities: [44] Power Management version 2
>> Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
>> Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>>
>> On Thursday 09 June 2011 02:01 PM, Kenji Kaneshige wrote:
>>> (2011/06/09 16:54), shivprashant wrote:
>>>> i built pciehp statically. the driver gets loaded but probe is not
>>>> called. do i need to modify anything in pciehp driver to make it work
>>>> with my platform?
>>>>
>>> I have no idea so far.
>>>
>>> Can you send me the following information?
>>>
>>> - dmesg output after loading pciehp with pciehp_debug module option
>>> (if you built pciehp as builtin driver, please add pciehp.pciehp_debug
>>> to kernel parameter and boot)
>>>
>>> - "lspci -vv" output (as root user).
>>>
>>> Regards,
>>> Kenji Kaneshige
>>>
>>>
>>>
>>>> On Thursday 09 June 2011 01:09 PM, Kenji Kaneshige wrote:
>>>>> Why not use pciehp?
>>>>>
>>>>> Thanks,
>>>>> Kenji Kaneshige
>>>>>
>>>>>
>>>>> (2011/06/09 14:31), shivprashant wrote:
>>>>>> hi,
>>>>>> i am writing pci express hotplug driver for mpc8315erdb(powerpc). the driver uses PCI hotplug subsystem not pcie port bus driver. i am able to do insertion and removal of pci express cards and confirm using 'lspci' command. driver is working with SATA and PATA memory cards with some minor modifications in the kernel. the problem i am facing now is when i try to repeatedly insert and remove the kernel is crashing, although non-fatal. normal insertion/removal works fine but repeated insertion/removal fails. pls guide me where am i wrong or what changes i should make to recover from pci errors.
>>>>>>
>>>>>> Following is the crash log when repeatedly inserting and removing the PATA cards (i am using delkin 32GB cards),
>>>>>> scsi20 : pata_jmicron
>>>>>> scsi21 : pata_jmicron
>>>>>> ata21: PATA max UDMA/100 cmd 0xff7fa010 ctl 0xff7fa020 bmdma 0xff7fa000 irq 21
>>>>>> ata22: PATA max UDMA/100 cmd 0xff7fa018 ctl 0xff7fa024 bmdma 0xff7fa008 irq 21
>>>>>> PCIE Card Removed from Slot 0
>>>>>> Oops: Machine check, sig: 7 [#1]
>>>>>> MPC831x RDB
>>>>>> last sysfs file: /sys/devices/pci0002:02/0002:02:00.0/0002:03:00.0/class
>>>>>> Modules linked in:
>>>>>> NIP: c00124c4 LR: c00124b0 CTR: c0017998
>>>>>> REGS: cfad3cb0 TRAP: 0200 Not tainted (2.6.36.1)
>>>>>> MSR: 00049030<EE,ME,IR,DR> CR: 22204084 XER: 20000000
>>>>>> TASK = cfa9c030[1282] 'scsi_eh_20' THREAD: cfad2000
>>>>>> GPR00: 00000000 cfad3d60 cfa9c030 00000000 cfad3d08 00e00000 a91e4fb9 00000001
>>>>>> GPR08: c03fd688 fdef7000 fd6f7000 c042a0e0 00005a07 00020d78 c02fbd0c ffffffff
>>>>>> GPR16: 00000000 00000000 00000000 00000004 c01ff498 00000001 cfac94c8 cfac8000
>>>>>> GPR24: c01fa87c 00000003 ffff2a2d cfac801c 00000001 00000000 cfac9430 fd6f8012
>>>>>> NIP [c00124c4] ioread8+0x2c/0x328
>>>>>> LR [c00124b0] ioread8+0x18/0x328
>>>>>> Call Trace:
>>>>>> [cfad3d60] [c002bccc] msleep+0x1c/0x34 (unreliable)
>>>>>> [cfad3d70] [c01fa77c] ata_sff_wait_after_reset+0x6c/0x16c
>>>>>> [cfad3da0] [c01fa934] ata_sff_softreset+0xb8/0x204
>>>>>> [cfad3de0] [c01f2604] ata_do_reset+0xa0/0xb4
>>>>>> [cfad3e00] [c01f33ac] ata_eh_reset+0x3e4/0xcf4
>>>>>> [cfad3e80] [c01f59e8] ata_eh_recover+0x284/0x10e0
>>>>>> [cfad3f00] [c01f6a0c] ata_do_eh+0x4c/0xb0
>>>>>> [cfad3f20] [c01f926c] ata_sff_error_handler+0x110/0x140
>>>>>> [cfad3f40] [c01f7680] ata_scsi_error+0x2ec/0x4e0
>>>>>> [cfad3f70] [c01d2664] scsi_error_handler+0xc4/0x39c
>>>>>> [cfad3fb0] [c0039d4c] kthread+0x7c/0x80
>>>>>> [cfad3ff0] [c000f200] kernel_thread+0x4c/0x68
>>>>>> Instruction dump:
>>>>>> 4e800020 9421fff0 7c0802a6 93e1000c 7c7f1b78 90010014 481835f5 2f830000
>>>>>> 38600000 419e0018 7c0004ac 881f0000<0c000000> 4c00012c 5403063e 80010014
>>>>>> ---[ end trace fb9766f858e66ef2 ]---
>>>>>>
>>>>>>
>>>>>>
>>
>
--
With Regards,
Shivaprashanth H
^ permalink raw reply [flat|nested] 3+ messages in thread
* PV driver domains and S3 sleep
@ 2010-09-16 11:44 Rafal Wojtczuk
2010-09-16 11:52 ` Keir Fraser
0 siblings, 1 reply; 3+ messages in thread
From: Rafal Wojtczuk @ 2010-09-16 11:44 UTC (permalink / raw)
To: xen-devel
Hello,
The topic is self-explanatory: how to ensure that a PV driver domain correctly
prepares its PCI devices for S3 sleep?
If I do "pm-suspend" in dom0, and the driver domain has active network interfaces,
suspend hangs the system. Yes, in case of this particular machine, suspend works
fine when there is no driver domain.
It is possible to manually invoke scripts from /usr/lib64/pm-utils/sleep.d/ in driver
domain. In the test case, "ifconfig down wlan0" in the driver domain allows
the suspend to go smoothly. But generally, is it enough ? The kernel device driver should
prepare the PCI device properly for S3, shouldn't it ?
Would it be more proper to [somehow] notify a driver domain _kernel_ that we are
going to S3 (just like dom0 kernel is notified), and let it execute all necessary actions
(including, but not only, launching of usermode pm-utils scripts), just like dom0 kernel
does ? Would it work at all, considering that driver domain kernel has no access to
ACPI tables ?
Currently, how are these issues taken care of in the mainstream Xen?
Thanks in advance,
Rafal Wojtczuk
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: PV driver domains and S3 sleep
2010-09-16 11:44 PV driver domains and S3 sleep Rafal Wojtczuk
@ 2010-09-16 11:52 ` Keir Fraser
2010-09-24 14:24 ` PCI hotplug problem [was: PV driver domains and S3 sleep] Rafal Wojtczuk
0 siblings, 1 reply; 3+ messages in thread
From: Keir Fraser @ 2010-09-16 11:52 UTC (permalink / raw)
To: Rafal Wojtczuk, xen-devel@lists.xensource.com
On 16/09/2010 12:44, "Rafal Wojtczuk" <rafal@invisiblethingslab.com> wrote:
> The topic is self-explanatory: how to ensure that a PV driver domain correctly
> prepares its PCI devices for S3 sleep?
> If I do "pm-suspend" in dom0, and the driver domain has active network
> interfaces,
> suspend hangs the system. Yes, in case of this particular machine, suspend
> works
> fine when there is no driver domain.
> It is possible to manually invoke scripts from /usr/lib64/pm-utils/sleep.d/ in
> driver
> domain. In the test case, "ifconfig down wlan0" in the driver domain allows
> the suspend to go smoothly. But generally, is it enough ? The kernel device
> driver should
> prepare the PCI device properly for S3, shouldn't it ?
> Would it be more proper to [somehow] notify a driver domain _kernel_ that we
> are
> going to S3 (just like dom0 kernel is notified), and let it execute all
> necessary actions
> (including, but not only, launching of usermode pm-utils scripts), just like
> dom0 kernel
> does ? Would it work at all, considering that driver domain kernel has no
> access to
> ACPI tables ?
> Currently, how are these issues taken care of in the mainstream Xen?
I don't think it currently is handled. HVM driver domains (using VT-d or
equivalent) can be put into virtual S3. We would need an equivalent concept
for PV driver domains. Or for devices to be hot-unplugged from the driver
domain, and re-plugged on resume?
-- Keir
^ permalink raw reply [flat|nested] 3+ messages in thread
* PCI hotplug problem [was: PV driver domains and S3 sleep]
2010-09-16 11:52 ` Keir Fraser
@ 2010-09-24 14:24 ` Rafal Wojtczuk
2010-09-27 17:07 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 3+ messages in thread
From: Rafal Wojtczuk @ 2010-09-24 14:24 UTC (permalink / raw)
To: Keir Fraser; +Cc: xen-devel@lists.xensource.com
On Thu, Sep 16, 2010 at 12:52:02PM +0100, Keir Fraser wrote:
> > The topic is self-explanatory: how to ensure that a PV driver domain correctly
> > prepares its PCI devices for S3 sleep?
[cut]
> > Currently, how are these issues taken care of in the mainstream Xen?
> I don't think it currently is handled. HVM driver domains (using VT-d or
> equivalent) can be put into virtual S3. We would need an equivalent concept
> for PV driver domains. Or for devices to be hot-unplugged from the driver
> domain, and re-plugged on resume?
The idea of using PCI hotplug is nice, however, PCI hotplug does not seem to
work with the used setup (xen-3.4.3, all 64bit). Hot-unplug works, however the
following hotplug makes the driver domain kernel spit out the following:
Sep 24 09:46:01 localhost kernel: [ 113.045927] pcifront pci-0: Rescanning
PCI Frontend Bus 0000:00
Sep 24 09:46:15 localhost kernel: [ 126.843990] pcifront pci-0: Rescanning
PCI Frontend Bus 0000:00
Sep 24 09:46:15 localhost kernel: [ 126.846217] pcifront pci-0: New device
on 0000:00:01.00 found.
Sep 24 09:46:15 localhost kernel: [ 126.846523] iwlagn 0000:00:01.0: device
not available (can't reserve [mem 0xf8000000-0xf8001fff 64bit])
^C
[root@localhost ~]# cat /proc/iomem
f6000000-f600ffff : 0000:00:00.0
f6000000-f600ffff : tg3
[root@localhost ~]# lspci
00:00.0 Ethernet controller: Broadcom Corporation NetLink BCM5787M Gigabit
Ethernet PCI Express (rev 02)
00:01.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN
[Kedron] Network Connection (rev 61)
Nothing suspicious in xend, Xen and dom0 logs.
The domU and dom0 kernels are the same, 2.6.34.1-10.xenlinux (SUSE patches
for 2.6.34.1).
With old pvops (2.6.31.9-1.pvops0) in domU, the message on the hot-plug is similar:
Sep 24 09:50:40 localhost kernel: pcifront pci-0: Rescanning PCI Frontend
Bus 0000:00
Sep 24 09:50:51 localhost kernel: pcifront pci-0: Rescanning PCI Frontend
Bus 0000:00
Sep 24 09:50:51 localhost kernel: pcifront pci-0: New device on
0000:00:01.00 found.
Sep 24 09:50:51 localhost kernel: iwlagn 0000:00:01.0: device not available
because of BAR 0 [0xf8000000-0xf8001fff] collisions
Others seem to experience similar problems (e.g.
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/80766). Does
anyone know the solution ?
Regards,
Rafal Wojtczuk
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: PCI hotplug problem [was: PV driver domains and S3 sleep]
2010-09-24 14:24 ` PCI hotplug problem [was: PV driver domains and S3 sleep] Rafal Wojtczuk
@ 2010-09-27 17:07 ` Konrad Rzeszutek Wilk
2010-10-01 14:24 ` PCI hotplug problem Rafal Wojtczuk
0 siblings, 1 reply; 3+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-09-27 17:07 UTC (permalink / raw)
To: Rafal Wojtczuk; +Cc: xen-devel@lists.xensource.com, Keir Fraser
On Fri, Sep 24, 2010 at 04:24:58PM +0200, Rafal Wojtczuk wrote:
> On Thu, Sep 16, 2010 at 12:52:02PM +0100, Keir Fraser wrote:
> > > The topic is self-explanatory: how to ensure that a PV driver domain correctly
> > > prepares its PCI devices for S3 sleep?
> [cut]
> > > Currently, how are these issues taken care of in the mainstream Xen?
>
> > I don't think it currently is handled. HVM driver domains (using VT-d or
> > equivalent) can be put into virtual S3. We would need an equivalent concept
> > for PV driver domains. Or for devices to be hot-unplugged from the driver
> > domain, and re-plugged on resume?
>
> The idea of using PCI hotplug is nice, however, PCI hotplug does not seem to
> work with the used setup (xen-3.4.3, all 64bit). Hot-unplug works, however the
> following hotplug makes the driver domain kernel spit out the following:
>
> Sep 24 09:46:01 localhost kernel: [ 113.045927] pcifront pci-0: Rescanning
> PCI Frontend Bus 0000:00
> Sep 24 09:46:15 localhost kernel: [ 126.843990] pcifront pci-0: Rescanning
> PCI Frontend Bus 0000:00
> Sep 24 09:46:15 localhost kernel: [ 126.846217] pcifront pci-0: New device
> on 0000:00:01.00 found.
> Sep 24 09:46:15 localhost kernel: [ 126.846523] iwlagn 0000:00:01.0: device
> not available (can't reserve [mem 0xf8000000-0xf8001fff 64bit])
>
> ^C
> [root@localhost ~]# cat /proc/iomem
> f6000000-f600ffff : 0000:00:00.0
> f6000000-f600ffff : tg3
> [root@localhost ~]# lspci
> 00:00.0 Ethernet controller: Broadcom Corporation NetLink BCM5787M Gigabit
> Ethernet PCI Express (rev 02)
> 00:01.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN
> [Kedron] Network Connection (rev 61)
>
> Nothing suspicious in xend, Xen and dom0 logs.
>
> The domU and dom0 kernels are the same, 2.6.34.1-10.xenlinux (SUSE patches
> for 2.6.34.1).
>
> With old pvops (2.6.31.9-1.pvops0) in domU, the message on the hot-plug is similar:
> Sep 24 09:50:40 localhost kernel: pcifront pci-0: Rescanning PCI Frontend
> Bus 0000:00
> Sep 24 09:50:51 localhost kernel: pcifront pci-0: Rescanning PCI Frontend
> Bus 0000:00
> Sep 24 09:50:51 localhost kernel: pcifront pci-0: New device on
> 0000:00:01.00 found.
> Sep 24 09:50:51 localhost kernel: iwlagn 0000:00:01.0: device not available
> because of BAR 0 [0xf8000000-0xf8001fff] collisions
>
> Others seem to experience similar problems (e.g.
> http://permalink.gmane.org/gmane.comp.emulators.xen.devel/80766). Does
> anyone know the solution ?
I had an off-mailing list conversation with that fellow and I spun out
a bunch of patches to fix his issue.
You need these patches:
Konrad Rzeszutek Wilk (3):
xen-pcifront: Enforce scanning of device functions on initial execution.
xen-pcifront: Claim PCI resources before going live.
xen-pcifront: Don't race with udev when discovering new devices.
I think they are in Jeremy's upstream tree.. ah, right you guys aren't using
Jeremy's tree.
Get them from: git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
pv/pcifront-2.6.34
you also might want to update your pciback driver too (pv/pciback-2.6.32)
>
> Regards,
> Rafal Wojtczuk
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: PCI hotplug problem
2010-09-27 17:07 ` Konrad Rzeszutek Wilk
@ 2010-10-01 14:24 ` Rafal Wojtczuk
2010-10-01 15:23 ` Jan Beulich
0 siblings, 1 reply; 3+ messages in thread
From: Rafal Wojtczuk @ 2010-10-01 14:24 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel@lists.xensource.com, Keir Fraser
On Mon, Sep 27, 2010 at 01:07:05PM -0400, Konrad Rzeszutek Wilk wrote:
> > The idea of using PCI hotplug is nice, however, PCI hotplug does not seem to
> > work with the used setup (xen-3.4.3, all 64bit). Hot-unplug works, however the
> > following hotplug makes the driver domain kernel spit out the following:
[cut]
> > Sep 24 09:46:15 localhost kernel: [ 126.846523] iwlagn 0000:00:01.0: device
> > not available (can't reserve [mem 0xf8000000-0xf8001fff 64bit])
[cut]
> > Others seem to experience similar problems (e.g.
> > http://permalink.gmane.org/gmane.comp.emulators.xen.devel/80766). Does
> > anyone know the solution ?
>
> I had an off-mailing list conversation with that fellow and I spun out
> a bunch of patches to fix his issue.
>
> You need these patches:
> Konrad Rzeszutek Wilk (3):
> xen-pcifront: Enforce scanning of device functions on initial execution.
> xen-pcifront: Claim PCI resources before going live.
> xen-pcifront: Don't race with udev when discovering new devices.
>
> I think they are in Jeremy's upstream tree.. ah, right you guys aren't using
> Jeremy's tree.
>
> Get them from: git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
>
> pv/pcifront-2.6.34
Indeed these patches help, thank you.
There is one more problem with the linux-2.6.18-xen.hg pcifront (that affect
derived code, e.g. OpenSUSE kernel, too). unbind_from_irqhandler() is
mistakenly passed evtchn, instead of irq. Compare line 68 of
http://xenbits.xensource.com/linux-2.6.18-xen.hg?file/a66a7c64b1d0/drivers/xen/pcifront/xenbus.c
with pvops equivalent
http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=blob;f=drivers/pci/xen-pcifront.c;h=10868aeae818d69980b8519f8a77b38d6ab58a4c;hb=HEAD#l758
The following patch helps.
Regards,
Rafal Wojtczuk
unbind_from_irqhandler takes irq, not evtchn, as its first argument.
Signed-off-by: Rafal Wojtczuk <rafal@invisiblethingslab.com>
--- linux-2.6.34.1/drivers/xen/pcifront/xenbus.c.orig 2010-09-29 16:47:39.961674359 +0200
+++ linux-2.6.34.1/drivers/xen/pcifront/xenbus.c 2010-09-29 16:47:49.458675391 +0200
@@ -61,7 +61,7 @@ static void free_pdev(struct pcifront_de
/*For PCIE_AER error handling job*/
flush_scheduled_work();
- unbind_from_irqhandler(pdev->evtchn, pdev);
+ unbind_from_irqhandler(irq_from_evtchn(pdev->evtchn), pdev);
if (pdev->evtchn != INVALID_EVTCHN)
xenbus_free_evtchn(pdev->xdev, pdev->evtchn);
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: PCI hotplug problem
2010-10-01 14:24 ` PCI hotplug problem Rafal Wojtczuk
@ 2010-10-01 15:23 ` Jan Beulich
0 siblings, 0 replies; 3+ messages in thread
From: Jan Beulich @ 2010-10-01 15:23 UTC (permalink / raw)
To: Rafal Wojtczuk
Cc: xen-devel@lists.xensource.com, Keir Fraser, Konrad Rzeszutek Wilk
>>> On 01.10.10 at 16:24, Rafal Wojtczuk <rafal@invisiblethingslab.com> wrote:
> There is one more problem with the linux-2.6.18-xen.hg pcifront (that affect
> derived code, e.g. OpenSUSE kernel, too). unbind_from_irqhandler() is
> mistakenly passed evtchn, instead of irq. Compare line 68 of
> http://xenbits.xensource.com/linux-2.6.18-xen.hg?file/a66a7c64b1d0/drivers/xen
> /pcifront/xenbus.c
> with pvops equivalent
> http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=blob;f=drivers/pc
> i/xen-pcifront.c;h=10868aeae818d69980b8519f8a77b38d6ab58a4c;hb=HEAD#l758
>
> The following patch helps.
Except there is no irq_from_evtchn() in the original tree. I'll post a
better, more complete patch later.
Jan
> Regards,
> Rafal Wojtczuk
>
>
> unbind_from_irqhandler takes irq, not evtchn, as its first argument.
>
> Signed-off-by: Rafal Wojtczuk <rafal@invisiblethingslab.com>
> --- linux-2.6.34.1/drivers/xen/pcifront/xenbus.c.orig 2010-09-29
> 16:47:39.961674359 +0200
> +++ linux-2.6.34.1/drivers/xen/pcifront/xenbus.c 2010-09-29
> 16:47:49.458675391 +0200
> @@ -61,7 +61,7 @@ static void free_pdev(struct pcifront_de
>
> /*For PCIE_AER error handling job*/
> flush_scheduled_work();
> - unbind_from_irqhandler(pdev->evtchn, pdev);
> + unbind_from_irqhandler(irq_from_evtchn(pdev->evtchn), pdev);
>
> if (pdev->evtchn != INVALID_EVTCHN)
> xenbus_free_evtchn(pdev->xdev, pdev->evtchn);
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2011-06-14 5:32 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-14 5:32 pci hotplug problem shivprashant
-- strict thread matches above, loose matches on Subject: below --
2010-09-16 11:44 PV driver domains and S3 sleep Rafal Wojtczuk
2010-09-16 11:52 ` Keir Fraser
2010-09-24 14:24 ` PCI hotplug problem [was: PV driver domains and S3 sleep] Rafal Wojtczuk
2010-09-27 17:07 ` Konrad Rzeszutek Wilk
2010-10-01 14:24 ` PCI hotplug problem Rafal Wojtczuk
2010-10-01 15:23 ` Jan Beulich
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.