* [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
@ 2012-11-11 23:19 Matthew Thode
2012-11-12 9:26 ` Doug Goldstein
0 siblings, 1 reply; 11+ messages in thread
From: Matthew Thode @ 2012-11-11 23:19 UTC (permalink / raw)
To: linux-kernel; +Cc: bhelgaas, linux-pci, mthode, iommu
[-- Attachment #1.1: Type: text/plain, Size: 7139 bytes --]
System boots with vt-d disabled in bios. Otherwise I get the errors in
the attached log. I can do whatever testing you need as this system is
not in production yet. gonna paste the important part here. Let me
know if you want anything else.
Please CC me directly as I am not subscribed to the LKML.
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 5124k freed
IOMMU 0 0xfbffe000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
BUG: unable to handle kernel NULL pointer dereference at 000000000000003c
IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
PGD 0
Oops: 0000 [#1] SMP
Modules linked in:
CPU 7
Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
Relion 1751/X8DTU
RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
pci_get_dma_source+0xf/0x41
RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
ffff8806264cf910)
Stack:
ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
Call Trace:
[<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
[<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
[<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
[<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
[<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
[<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
[<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
[<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
[<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
[<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
[<ffffffff81000231>] do_one_initcall+0x7a/0x129
[<ffffffff816dac14>] kernel_init+0x139/0x2a2
[<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
[<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a <66> 3b
4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
RSP <ffff8806264d1d88>
CR2: 000000000000003c
---[ end trace 5c5a2ceca067e0ec ]---
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
------------[ cut here ]------------
WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x25/0x51()
Hardware name: Relion 1751
Modules linked in:
Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
Call Trace:
<IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
[<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
[<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
[<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
[<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
[<ffffffff810a132f>] update_process_times+0x62/0x73
[<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
[<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
[<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
[<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
[<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
<EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
[<ffffffff816f32fc>] ? panic+0x189/0x1c5
[<ffffffff816f3261>] ? panic+0xee/0x1c5
[<ffffffff8109ab6b>] do_exit+0x357/0x7b2
[<ffffffff810371b8>] oops_end+0xb2/0xba
[<ffffffff8105841d>] no_context+0x266/0x275
[<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
[<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
[<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
[<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
[<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
[<ffffffff8112a077>] ? slab_node+0x59/0xa2
[<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
[<ffffffff816da653>] ? klist_put+0x4c/0x70
[<ffffffff816da581>] ? klist_next+0x30/0xb6
[<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
[<ffffffff81058c42>] do_page_fault+0x9/0xb
[<ffffffff816f6232>] page_fault+0x22/0x30
[<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
[<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
[<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
[<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
[<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
[<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
[<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
[<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
[<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
[<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
[<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
[<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
[<ffffffff81000231>] do_one_initcall+0x7a/0x129
[<ffffffff816dac14>] kernel_init+0x139/0x2a2
[<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
[<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
---[ end trace 5c5a2ceca067e0ed ]---
--
-- Matthew Thode (prometheanfire)
[-- Attachment #1.2: 3.7.0-rc5.log --]
[-- Type: text/plain, Size: 24018 bytes --]
Initializing cgroup subsys cpuset
Linux version 3.7.0-rc5 (root@vm-master-01) (gcc version 4.5.4 (Gentoo Hardened 4.5.4 p1.0, pie-0.4.7) ) #1 SMP Sun Nov 11 13:51:19 CST 2012
Command line: rootfstype=ext4 real_root=/dev/mapper/vg-rootfs real_rootflags=noatime,nodev,acl,user_xattr crypt_root=UUID=4b11f604-66b2-4673-a50d-d88e5fc9a8b3 domdadm dolvm
netconsole=4444@10.10.2.2/eth0,4444@10.10.2.1/4e:9c:bc:82:6d:d6;netconsole=4444@10.10.2.2/eth1,4444@10.10.2.1/4e:9c:bc:82:6d:d6;netconsole=4444@10.10.2.2/eth2,4444@10.10.2.1/4e:9c:bc:82:6d:d6 console=ttyS0,115200n8 ro
e820: BIOS-provided physical RAM map:
BIOS-e820: [mem 0x0000000000000000-0x0000000000097bff] usable
BIOS-e820: [mem 0x0000000000097c00-0x000000000009ffff] reserved
BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
BIOS-e820: [mem 0x0000000000100000-0x00000000bf77ffff] usable
BIOS-e820: [mem 0x00000000bf78e000-0x00000000bf78ffff] type 9
BIOS-e820: [mem 0x00000000bf790000-0x00000000bf79dfff] ACPI data
BIOS-e820: [mem 0x00000000bf79e000-0x00000000bf7cffff] ACPI NVS
BIOS-e820: [mem 0x00000000bf7d0000-0x00000000bf7dffff] reserved
BIOS-e820: [mem 0x00000000bf7ec000-0x00000000bfffffff] reserved
BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
BIOS-e820: [mem 0x00000000ffc00000-0x00000000ffffffff] reserved
BIOS-e820: [mem 0x0000000100000000-0x000000063fffffff] usable
NX (Execute Disable) protection: active
DMI present.
No AGP bridge found
e820: last_pfn = 0x640000 max_arch_pfn = 0x400000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
e820: last_pfn = 0xbf780 max_arch_pfn = 0x400000000
found SMP MP-table at [mem 0x000ff780-0x000ff78f] mapped at [ffff8800000ff780]
init_memory_mapping: [mem 0x00000000-0xbf77ffff]
init_memory_mapping: [mem 0x100000000-0x63fffffff]
RAMDISK: [mem 0x37aef000-0x37feffff]
ACPI: RSDP 00000000000face0 00024 (v02 ACPIAM)
ACPI: XSDT 00000000bf790100 00084 (v01 SMCI 20120803 MSFT 00000097)
ACPI: FACP 00000000bf790290 000F4 (v04 080312 FACP1521 20120803 MSFT 00000097)
ACPI: DSDT 00000000bf7906a0 06580 (v02 10600 10600000 00000000 INTL 20051117)
ACPI: FACS 00000000bf79e000 00040
ACPI: APIC 00000000bf790390 0011E (v02 080312 APIC1521 20120803 MSFT 00000097)
ACPI: MCFG 00000000bf7904b0 0003C (v01 080312 OEMMCFG 20120803 MSFT 00000097)
ACPI: SLIT 00000000bf7904f0 00030 (v01 080312 OEMSLIT 20120803 MSFT 00000097)
ACPI: OEMB 00000000bf79e040 00086 (v01 080312 OEMB1521 20120803 MSFT 00000097)
ACPI: HPET 00000000bf79a6a0 00038 (v01 080312 OEMHPET 20120803 MSFT 00000097)
ACPI: DMAR 00000000bf79e0d0 00130 (v01 AMI OEMDMAR 00000001 MSFT 00000097)
ACPI: SSDT 00000000bf79fe40 00363 (v01 DpgPmm CpuPm 00000012 INTL 20051117)
ACPI: EINJ 00000000bf79a6e0 00130 (v01 AMIER AMI_EINJ 20120803 MSFT 00000097)
ACPI: BERT 00000000bf79a870 00030 (v01 AMIER AMI_BERT 20120803 MSFT 00000097)
ACPI: ERST 00000000bf79a8a0 001B0 (v01 AMIER AMI_ERST 20120803 MSFT 00000097)
ACPI: HEST 00000000bf79aa50 000A8 (v01 AMIER ABC_HEST 20120803 MSFT 00000097)
NUMA: Warning: node ids are out of bound, from=-1 to=-1 distance=10
No NUMA configuration found
Faking a node at [mem 0x0000000000000000-0x000000063fffffff]
Initmem setup node 0 [mem 0x00000000-0x63fffffff]
NODE_DATA [mem 0x63fffc000-0x63fffffff]
Zone ranges:
DMA [mem 0x00010000-0x00ffffff]
DMA32 [mem 0x01000000-0xffffffff]
Normal [mem 0x100000000-0x63fffffff]
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x00010000-0x00096fff]
node 0: [mem 0x00100000-0xbf77ffff]
node 0: [mem 0x100000000-0x63fffffff]
ACPI: PM-Timer IO Port: 0x808
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x88] disabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x89] disabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x8a] disabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x8b] disabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x8c] disabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x8d] disabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x8e] disabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x8f] disabled)
ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled)
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled)
ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled)
ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled)
ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled)
ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled)
ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled)
ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled)
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24])
IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 20 low level)
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a301 base: 0xfed00000
smpboot: Allowing 24 CPUs, 16 hotplug CPUs
PM: Registered nosave memory: 0000000000097000 - 0000000000098000
PM: Registered nosave memory: 0000000000098000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
PM: Registered nosave memory: 00000000bf780000 - 00000000bf78e000
PM: Registered nosave memory: 00000000bf78e000 - 00000000bf790000
PM: Registered nosave memory: 00000000bf790000 - 00000000bf79e000
PM: Registered nosave memory: 00000000bf79e000 - 00000000bf7d0000
PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ec000
PM: Registered nosave memory: 00000000bf7ec000 - 00000000c0000000
PM: Registered nosave memory: 00000000c0000000 - 00000000e0000000
PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
PM: Registered nosave memory: 00000000f0000000 - 00000000fee00000
PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
PM: Registered nosave memory: 00000000fee01000 - 00000000ffc00000
PM: Registered nosave memory: 00000000ffc00000 - 0000000100000000
e820: [mem 0xc0000000-0xdfffffff] available for PCI devices
setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:24 nr_node_ids:1
PERCPU: Embedded 25 pages/cpu @ffff88063f200000 s73728 r8192 d20480 u131072
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 6199553
Policy zone: Normal
Kernel command line: rootfstype=ext4 real_root=/dev/mapper/vg-rootfs real_rootflags=noatime,nodev,acl,user_xattr crypt_root=UUID=4b11f604-66b2-4673-a50d-d88e5fc9a8b3 domdadm dolvm
netconsole=4444@10.10.2.2/eth0,4444@10.10.2.1/4e:9c:bc:82:6d:d6;netconsole=4444@10.10.2.2/eth1,4444@10.10.2.1/4e:9c:bc:82:6d:d6;netconsole=4444@10.10.2.2/eth2,4444@10.10.2.1/4e:9c:bc:82:6d:d6 console=ttyS0,115200n8 ro
PID hash table entries: 4096 (order: 3, 32768 bytes)
__ex_table already sorted, skipping sort
Checking aperture...
No AGP bridge found
Memory: 24724712k/26214400k available (7151k kernel code, 1057764k absent, 431924k reserved, 5688k data, 620k init)
Hierarchical RCU implementation.
RCU restricting CPUs from NR_CPUS=32 to nr_cpu_ids=24.
NR_IRQS:4352 nr_irqs:1280 16
Extended CMOS year: 2000
Console: colour VGA+ 80x25
console [ttyS0] enabled
tsc: Fast TSC calibration failed
tsc: PIT calibration matches HPET. 1 loops
tsc: Detected 2266.735 MHz processor
Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.47 BogoMIPS (lpj=22667350)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys cpuacct
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 9 MCE banks
CPU0: Thermal monitoring enabled (TM1)
process: using mwait in idle threads
Last level iTLB entries: 4KB 512, 2MB 7, 4MB 7
Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
tlb_flushall_shift: 6
Freeing SMP alternatives: 32k freed
ACPI: Core revision 20120913
dmar: Host address width 40
dmar: DRHD base: 0x000000fbffe000 flags: 0x1
dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap c90780106f0462 ecap f020fe
dmar: RMRR base: 0x000000000ec000 end: 0x000000000effff
dmar: RMRR base: 0x000000bf7ec000 end: 0x000000bf7fffff
dmar: ATSR flags: 0x0
IOAPIC id 8 under DRHD base 0xfbffe000 IOMMU 0
IOAPIC id 9 under DRHD base 0xfbffe000 IOMMU 0
Enabled IRQ remapping in xapic mode
Switched APIC routing to physical flat.
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
smpboot: CPU0: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (fam: 06, model: 1a, stepping: 05)
Performance Events: PEBS fmt1+, 16-deep LBR, Nehalem events, Intel PMU driver.
perf_event_intel: CPU erratum AAJ80 worked around
perf_event_intel: CPUID marked event: 'bus cycles' unavailable
... version: 3
... bit width: 48
... generic registers: 4
... value mask: 0000ffffffffffff
... max period: 000000007fffffff
... fixed-purpose events: 3
... event mask: 000000070000000f
smpboot: Booting Node 0, Processors #1 #2 #3 #4 #5 #6 #7
Brought up 8 CPUs
smpboot: Total of 8 processors activated (36267.76 BogoMIPS)
devtmpfs: initialized
PM: Registering ACPI NVS region [mem 0xbf79e000-0xbf7cffff] (204800 bytes)
xor: automatically using best checksumming function:
generic_sse: 8139.200 MB/sec
NET: Registered protocol family 16
ACPI: bus type pci registered
dca service started, version 1.12.1
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
raid6: sse2x1 5511 MB/s
raid6: sse2x2 6460 MB/s
raid6: sse2x4 7317 MB/s
raid6: using algorithm sse2x4 (7317 MB/s)
raid6: using ssse3x2 recovery algorithm
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: Executed 1 blocks of module-level executable AML code
ACPI: SSDT 00000000bf79e200 015DC (v01 DpgPmm P001Ist 00000011 INTL 20051117)
ACPI: Dynamic OEM Table Load:
ACPI: SSDT (null) 015DC (v01 DpgPmm P001Ist 00000011 INTL 20051117)
ACPI: SSDT 00000000bf79f7e0 00659 (v01 PmRef P001Cst 00003001 INTL 20051117)
ACPI: Dynamic OEM Table Load:
ACPI: SSDT (null) 00659 (v01 PmRef P001Cst 00003001 INTL 20051117)
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S4 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A08:00: host bridge window expanded to [mem 0xf0000000-0xfed8ffff]; [mem 0xfed40000-0xfed44fff] ignored
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [io 0x0000-0x03af]
pci_bus 0000:00: root bus resource [io 0x03e0-0x0cf7]
pci_bus 0000:00: root bus resource [io 0x03b0-0x03bb]
pci_bus 0000:00: root bus resource [io 0x03c0-0x03df]
pci_bus 0000:00: root bus resource [io 0x0d00-0xefff]
pci_bus 0000:00: root bus resource [io 0xf000-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000dffff]
pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff]
pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfed8ffff]
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0a00 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 4700 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 3 PIO at 1640 (mask 000f)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 4 PIO at 0ca0 (mask 000f)
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:03.0: PCI bridge to [bus 02]
pci 0000:00:05.0: PCI bridge to [bus 03]
pci 0000:04:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
pci 0000:00:07.0: PCI bridge to [bus 04-05]
pci 0000:04:00.0: PCI bridge to [bus 05]
pci 0000:06:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
pci 0000:00:09.0: PCI bridge to [bus 06]
pci 0000:00:1e.0: PCI bridge to [bus 07] (subtractive decode)
pci0000:00: Requesting ACPI _OSC control (0x1d)
pci0000:00: ACPI _OSC control (0x1c) granted
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 6 7 10 11 12 *14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 *7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 *6 7 10 11 12 14 15)
vgaarb: device added: PCI:0000:07:01.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:07:01.0
SCSI subsystem initialized
ACPI: bus type scsi registered
ACPI: bus type usb registered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Initialized.
PCI: Using ACPI for IRQ routing
PCI: Discovered peer bus ff
PCI host bridge to bus 0000:ff
pci_bus 0000:ff: root bus resource [io 0x0000-0xffff]
pci_bus 0000:ff: root bus resource [mem 0x00000000-0xffffffffff]
pci_bus 0000:ff: No busn resource found for root bus, will use [bus ff-ff]
HPET: 4 timers in total, 0 timers will be used for per-cpu timer
Switching to clocksource hpet
pnp: PnP ACPI init
ACPI: bus type pnp registered
system 00:01: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:08: [io 0x0a10-0x0a1f] has been reserved
system 00:0b: [io 0x0ca2-0x0ca3] has been reserved
system 00:0b: [io 0x0cf8-0x0cff] could not be reserved
system 00:0b: [io 0x04d0-0x04d1] has been reserved
system 00:0b: [io 0x0800-0x087f] has been reserved
system 00:0b: [io 0x0500-0x057f] has been reserved
system 00:0b: [mem 0x00000400-0x000004ff] could not be reserved
system 00:0b: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:0b: [mem 0xfed20000-0xfed3ffff] has been reserved
system 00:0b: [mem 0xfed40000-0xfed8ffff] has been reserved
system 00:0d: [mem 0xfec00000-0xfec00fff] could not be reserved
system 00:0d: [mem 0xfee00000-0xfee00fff] has been reserved
system 00:0e: [mem 0xe0000000-0xefffffff] has been reserved
system 00:0f: [mem 0x000c0000-0x000cffff] could not be reserved
system 00:0f: [mem 0x000e0000-0x000fffff] could not be reserved
system 00:0f: [mem 0xfed90000-0xffffffff] could not be reserved
pnp: PnP ACPI: found 16 devices
ACPI: ACPI bus type pnp unregistered
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:01.0: bridge window [io 0xc000-0xcfff]
pci 0000:00:01.0: bridge window [mem 0xfbb00000-0xfbbfffff]
pci 0000:00:03.0: PCI bridge to [bus 02]
pci 0000:00:05.0: PCI bridge to [bus 03]
pci 0000:04:00.0: PCI bridge to [bus 05]
pci 0000:04:00.0: bridge window [io 0xd000-0xdfff]
pci 0000:04:00.0: bridge window [mem 0xfbc00000-0xfbcfffff]
pci 0000:00:07.0: PCI bridge to [bus 04-05]
pci 0000:00:07.0: bridge window [io 0xd000-0xdfff]
pci 0000:00:07.0: bridge window [mem 0xfbc00000-0xfbcfffff]
pci 0000:00:09.0: PCI bridge to [bus 06]
pci 0000:00:09.0: bridge window [io 0xe000-0xefff]
pci 0000:00:09.0: bridge window [mem 0xfbd00000-0xfbdfffff]
pci 0000:00:1e.0: PCI bridge to [bus 07]
pci 0000:00:1e.0: bridge window [mem 0xfaf00000-0xfb7fffff]
pci 0000:00:1e.0: bridge window [mem 0xf9000000-0xf9ffffff 64bit pref]
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP: reno registered
UDP hash table entries: 16384 (order: 7, 524288 bytes)
UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
pci 0000:04:00.0: TI XIO2000a quirk detected; secondary bus fast back-to-back transfers disabled
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 5124k freed
IOMMU 0 0xfbffe000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 - 0xbf7fffff]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
BUG: unable to handle kernel NULL pointer dereference at 000000000000003c
IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
PGD 0
Oops: 0000 [#1] SMP
Modules linked in:
CPU 7
Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing Relion 1751/X8DTU
RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task ffff8806264cf910)
Stack:
ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
Call Trace:
[<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
[<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
[<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
[<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
[<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
[<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
[<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
[<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
[<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
[<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
[<ffffffff81000231>] do_one_initcall+0x7a/0x129
[<ffffffff816dac14>] kernel_init+0x139/0x2a2
[<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
[<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8 e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a <66> 3b 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
RSP <ffff8806264d1d88>
CR2: 000000000000003c
---[ end trace 5c5a2ceca067e0ec ]---
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
------------[ cut here ]------------
WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x25/0x51()
Hardware name: Relion 1751
Modules linked in:
Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
Call Trace:
<IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
[<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
[<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
[<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
[<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
[<ffffffff810a132f>] update_process_times+0x62/0x73
[<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
[<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
[<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
[<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
[<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
<EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
[<ffffffff816f32fc>] ? panic+0x189/0x1c5
[<ffffffff816f3261>] ? panic+0xee/0x1c5
[<ffffffff8109ab6b>] do_exit+0x357/0x7b2
[<ffffffff810371b8>] oops_end+0xb2/0xba
[<ffffffff8105841d>] no_context+0x266/0x275
[<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
[<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
[<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
[<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
[<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
[<ffffffff8112a077>] ? slab_node+0x59/0xa2
[<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
[<ffffffff816da653>] ? klist_put+0x4c/0x70
[<ffffffff816da581>] ? klist_next+0x30/0xb6
[<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
[<ffffffff81058c42>] do_page_fault+0x9/0xb
[<ffffffff816f6232>] page_fault+0x22/0x30
[<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
[<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
[<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
[<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
[<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
[<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
[<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
[<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
[<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
[<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
[<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
[<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
[<ffffffff81000231>] do_one_initcall+0x7a/0x129
[<ffffffff816dac14>] kernel_init+0x139/0x2a2
[<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
[<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
[<ffffffff816daadb>] ? rest_init+0x6f/0x6f
---[ end trace 5c5a2ceca067e0ed ]---
[-- Attachment #1.3: pipacs-pci-null-deref.log --]
[-- Type: text/plain, Size: 1149 bytes --]
14:25 < prometheanfire > pipacs: intrested in a null pointer deref in the kernel (not pax)?
14:25 < pipacs > define 'interested' ;)
14:25 < prometheanfire > http://bpaste.net/show/v5y2sI6l3WWx3Rcpu5CS/
14:26 < pipacs > what do you expect me to do with it?
14:26 < pipacs > help debug it?
14:26 < prometheanfire > sure lol
14:26 < prometheanfire > it only happens when I have this one pci card in, so I'm confused, if you have something better to do, do that though :P
14:27 < pipacs > do you have CONFIG_PCI_QUIRKS?
14:28 < prometheanfire > dunno
14:28 < prometheanfire > gonna have to reboot to get to it
14:28 < prometheanfire > I saw that too
14:28 < pipacs > like, check it? ;P
14:28 < pipacs > my guess is that 'dev' is null
14:28 < pipacs > the one passed to pci_get_dma_source
14:30 < pipacs > better the kernel devs
14:30 < prometheanfire > probably
14:30 < pipacs > i don't know if the code is supposed to handle a null dev there or not
14:30 < prometheanfire > it's a pcie-pci bridge
14:32 < prometheanfire > CONFIG_PCI_QUIRKS=y
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-11 23:19 [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu Matthew Thode
@ 2012-11-12 9:26 ` Doug Goldstein
2012-11-12 19:57 ` Don Dutile
0 siblings, 1 reply; 11+ messages in thread
From: Doug Goldstein @ 2012-11-12 9:26 UTC (permalink / raw)
To: prometheanfire; +Cc: linux-kernel, bhelgaas, linux-pci, mthode, iommu
On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
<prometheanfire@gentoo.org> wrote:
> System boots with vt-d disabled in bios. Otherwise I get the errors in
> the attached log. I can do whatever testing you need as this system is
> not in production yet. gonna paste the important part here. Let me
> know if you want anything else.
>
> Please CC me directly as I am not subscribed to the LKML.
>
>
> Trying to unpack rootfs image as initramfs...
> Freeing initrd memory: 5124k freed
> IOMMU 0 0xfbffe000: using Queued invalidation
> IOMMU: Setting RMRR:
> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
> 0xbf7fffff]
> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
> IOMMU: Prepare 0-16MiB unity mapping for LPC
> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
> BUG: unable to handle kernel NULL pointer dereference at 000000000000003c
> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> PGD 0
> Oops: 0000 [#1] SMP
> Modules linked in:
> CPU 7
> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
> Relion 1751/X8DTU
> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
> pci_get_dma_source+0xf/0x41
> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
> ffff8806264cf910)
> Stack:
> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
> Call Trace:
> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a <66> 3b
> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> RSP <ffff8806264d1d88>
> CR2: 000000000000003c
> ---[ end trace 5c5a2ceca067e0ec ]---
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>
> ------------[ cut here ]------------
> WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x25/0x51()
> Hardware name: Relion 1751
> Modules linked in:
> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
> Call Trace:
> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
> [<ffffffff810a132f>] update_process_times+0x62/0x73
> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
> [<ffffffff816f3261>] ? panic+0xee/0x1c5
> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
> [<ffffffff810371b8>] oops_end+0xb2/0xba
> [<ffffffff8105841d>] no_context+0x266/0x275
> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
> [<ffffffff816da653>] ? klist_put+0x4c/0x70
> [<ffffffff816da581>] ? klist_next+0x30/0xb6
> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
> [<ffffffff81058c42>] do_page_fault+0x9/0xb
> [<ffffffff816f6232>] page_fault+0x22/0x30
> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> ---[ end trace 5c5a2ceca067e0ed ]---
>
> --
> -- Matthew Thode (prometheanfire)
The root cause of Matt's issue is that intel_iommu_add_device() calls
pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
expected value. The reason NULL is being returned is that Matt has a
card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
8231) on it. This device already has a quirk setup for disabling fast
back to back transfers on its secondary bus. If we cause it to use the
primary bus, that appears to resolve the issue. I'm not sure exactly
how to proceed from here due to relative lack of knowledge of PCI. Do
all PCIe-PCI bridges with secondary buses need their DMA parent to be
the primary bus or is that just something that should be done for the
TI XIO2000A due to the existing quirk?
The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
that didn't panic and a device that worked.
$ lspci -tvn
-+-[0000:ff]-+-00.0 8086:2c40
| +-00.1 8086:2c01
| +-02.0 8086:2c10
| +-02.1 8086:2c11
| +-02.4 8086:2c14
| +-02.5 8086:2c15
| +-03.0 8086:2c18
| +-03.1 8086:2c19
| +-03.2 8086:2c1a
| +-03.4 8086:2c1c
| +-04.0 8086:2c20
| +-04.1 8086:2c21
| +-04.2 8086:2c22
| +-04.3 8086:2c23
| +-05.0 8086:2c28
| +-05.1 8086:2c29
| +-05.2 8086:2c2a
| +-05.3 8086:2c2b
| +-06.0 8086:2c30
| +-06.1 8086:2c31
| +-06.2 8086:2c32
| \-06.3 8086:2c33
\-[0000:00]-+-00.0 8086:3406
+-01.0-[01]--+-00.0 8086:10c9
| \-00.1 8086:10c9
+-03.0-[02]--
+-05.0-[03]--
+-07.0-[04-05]----00.0-[05]----08.0 d161:8006
+-09.0-[06]----00.0 8086:10b9
+-13.0 8086:342d
+-14.0 8086:342e
+-14.1 8086:3422
+-14.2 8086:3423
+-14.3 8086:3438
+-16.0 8086:3430
+-16.1 8086:3431
+-16.2 8086:3432
+-16.3 8086:3433
+-16.4 8086:3429
+-16.5 8086:342a
+-16.6 8086:342b
+-16.7 8086:342c
+-1a.0 8086:3a37
+-1a.1 8086:3a38
+-1a.2 8086:3a39
+-1a.7 8086:3a3c
+-1d.0 8086:3a34
+-1d.1 8086:3a35
+-1d.2 8086:3a36
+-1d.7 8086:3a3a
+-1e.0-[07]----01.0 102b:0532
+-1f.0 8086:3a16
+-1f.2 8086:3a22
\-1f.3 8086:3a30
If someone can craft the correct patch that'd be great or answer the
above question and I'll gladly craft it.
Thanks.
--
Doug Goldstein
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-12 9:26 ` Doug Goldstein
@ 2012-11-12 19:57 ` Don Dutile
2012-11-12 21:05 ` Matthew Thode
2012-11-12 22:19 ` Alex Williamson
0 siblings, 2 replies; 11+ messages in thread
From: Don Dutile @ 2012-11-12 19:57 UTC (permalink / raw)
To: Doug Goldstein
Cc: prometheanfire, linux-kernel, bhelgaas, linux-pci, mthode, iommu,
Alex Williamson
On 11/12/2012 04:26 AM, Doug Goldstein wrote:
> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
> <prometheanfire@gentoo.org> wrote:
>> System boots with vt-d disabled in bios. Otherwise I get the errors in
>> the attached log. I can do whatever testing you need as this system is
>> not in production yet. gonna paste the important part here. Let me
>> know if you want anything else.
>>
>> Please CC me directly as I am not subscribed to the LKML.
>>
>>
>> Trying to unpack rootfs image as initramfs...
>> Freeing initrd memory: 5124k freed
>> IOMMU 0 0xfbffe000: using Queued invalidation
>> IOMMU: Setting RMRR:
>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
>> 0xbf7fffff]
>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
>> IOMMU: Prepare 0-16MiB unity mapping for LPC
>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>> BUG: unable to handle kernel NULL pointer dereference at 000000000000003c
>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>> PGD 0
>> Oops: 0000 [#1] SMP
>> Modules linked in:
>> CPU 7
>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
>> Relion 1751/X8DTU
>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
>> pci_get_dma_source+0xf/0x41
>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
>> ffff8806264cf910)
>> Stack:
>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
>> Call Trace:
>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>> RSP<ffff8806264d1d88>
>> CR2: 000000000000003c
>> ---[ end trace 5c5a2ceca067e0ec ]---
>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>>
>> ------------[ cut here ]------------
>> WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x25/0x51()
>> Hardware name: Relion 1751
>> Modules linked in:
>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
>> Call Trace:
>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
>> [<ffffffff810a132f>] update_process_times+0x62/0x73
>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
>> [<ffffffff810371b8>] oops_end+0xb2/0xba
>> [<ffffffff8105841d>] no_context+0x266/0x275
>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
>> [<ffffffff816f6232>] page_fault+0x22/0x30
>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>> ---[ end trace 5c5a2ceca067e0ed ]---
>>
>> --
>> -- Matthew Thode (prometheanfire)
>
> The root cause of Matt's issue is that intel_iommu_add_device() calls
> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
> expected value. The reason NULL is being returned is that Matt has a
> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
> 8231) on it. This device already has a quirk setup for disabling fast
> back to back transfers on its secondary bus. If we cause it to use the
> primary bus, that appears to resolve the issue. I'm not sure exactly
> how to proceed from here due to relative lack of knowledge of PCI. Do
> all PCIe-PCI bridges with secondary buses need their DMA parent to be
> the primary bus or is that just something that should be done for the
> TI XIO2000A due to the existing quirk?
>
DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
so the source of the device generating the DMA is unknown. When bridging
to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the source
of a transaction -- in this case, DMA read/write transaction.
This (sw) mapping should have happened by default, unless a recent change from VFIO
broke this mapping.... or the TI bridge didn't report itself correctly
as a PCIe-PCI bridge.
Alex ?
> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
> that didn't panic and a device that worked.
>
> $ lspci -tvn
> -+-[0000:ff]-+-00.0 8086:2c40
> | +-00.1 8086:2c01
> | +-02.0 8086:2c10
> | +-02.1 8086:2c11
> | +-02.4 8086:2c14
> | +-02.5 8086:2c15
> | +-03.0 8086:2c18
> | +-03.1 8086:2c19
> | +-03.2 8086:2c1a
> | +-03.4 8086:2c1c
> | +-04.0 8086:2c20
> | +-04.1 8086:2c21
> | +-04.2 8086:2c22
> | +-04.3 8086:2c23
> | +-05.0 8086:2c28
> | +-05.1 8086:2c29
> | +-05.2 8086:2c2a
> | +-05.3 8086:2c2b
> | +-06.0 8086:2c30
> | +-06.1 8086:2c31
> | +-06.2 8086:2c32
> | \-06.3 8086:2c33
> \-[0000:00]-+-00.0 8086:3406
> +-01.0-[01]--+-00.0 8086:10c9
> | \-00.1 8086:10c9
> +-03.0-[02]--
> +-05.0-[03]--
> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
> +-09.0-[06]----00.0 8086:10b9
> +-13.0 8086:342d
> +-14.0 8086:342e
> +-14.1 8086:3422
> +-14.2 8086:3423
> +-14.3 8086:3438
> +-16.0 8086:3430
> +-16.1 8086:3431
> +-16.2 8086:3432
> +-16.3 8086:3433
> +-16.4 8086:3429
> +-16.5 8086:342a
> +-16.6 8086:342b
> +-16.7 8086:342c
> +-1a.0 8086:3a37
> +-1a.1 8086:3a38
> +-1a.2 8086:3a39
> +-1a.7 8086:3a3c
> +-1d.0 8086:3a34
> +-1d.1 8086:3a35
> +-1d.2 8086:3a36
> +-1d.7 8086:3a3a
> +-1e.0-[07]----01.0 102b:0532
> +-1f.0 8086:3a16
> +-1f.2 8086:3a22
> \-1f.3 8086:3a30
>
> If someone can craft the correct patch that'd be great or answer the
> above question and I'll gladly craft it.
>
> Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-12 19:57 ` Don Dutile
@ 2012-11-12 21:05 ` Matthew Thode
2012-11-13 15:38 ` Alex Williamson
2012-11-12 22:19 ` Alex Williamson
1 sibling, 1 reply; 11+ messages in thread
From: Matthew Thode @ 2012-11-12 21:05 UTC (permalink / raw)
To: Don Dutile
Cc: Doug Goldstein, linux-kernel, bhelgaas, linux-pci, mthode, iommu,
Alex Williamson
[-- Attachment #1: Type: text/plain, Size: 12893 bytes --]
On 11/12/2012 01:57 PM, Don Dutile wrote:
> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
>> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
>> <prometheanfire@gentoo.org> wrote:
>>> System boots with vt-d disabled in bios. Otherwise I get the errors in
>>> the attached log. I can do whatever testing you need as this system is
>>> not in production yet. gonna paste the important part here. Let me
>>> know if you want anything else.
>>>
>>> Please CC me directly as I am not subscribed to the LKML.
>>>
>>>
>>> Trying to unpack rootfs image as initramfs...
>>> Freeing initrd memory: 5124k freed
>>> IOMMU 0 0xfbffe000: using Queued invalidation
>>> IOMMU: Setting RMRR:
>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
>>> 0xbf7fffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
>>> IOMMU: Prepare 0-16MiB unity mapping for LPC
>>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
>>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>>> BUG: unable to handle kernel NULL pointer dereference at
>>> 000000000000003c
>>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>> PGD 0
>>> Oops: 0000 [#1] SMP
>>> Modules linked in:
>>> CPU 7
>>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
>>> Relion 1751/X8DTU
>>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
>>> pci_get_dma_source+0xf/0x41
>>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
>>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
>>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
>>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
>>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
>>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
>>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000)
>>> knlGS:0000000000000000
>>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
>>> ffff8806264cf910)
>>> Stack:
>>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
>>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
>>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
>>> Call Trace:
>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
>>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
>>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
>>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>> RSP<ffff8806264d1d88>
>>> CR2: 000000000000003c
>>> ---[ end trace 5c5a2ceca067e0ec ]---
>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>>>
>>> ------------[ cut here ]------------
>>> WARNING: at arch/x86/kernel/smp.c:123
>>> native_smp_send_reschedule+0x25/0x51()
>>> Hardware name: Relion 1751
>>> Modules linked in:
>>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
>>> Call Trace:
>>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
>>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
>>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
>>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
>>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
>>> [<ffffffff810a132f>] update_process_times+0x62/0x73
>>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
>>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
>>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
>>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
>>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
>>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
>>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
>>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
>>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
>>> [<ffffffff810371b8>] oops_end+0xb2/0xba
>>> [<ffffffff8105841d>] no_context+0x266/0x275
>>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
>>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
>>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
>>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
>>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
>>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
>>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
>>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
>>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
>>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
>>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
>>> [<ffffffff816f6232>] page_fault+0x22/0x30
>>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
>>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>> ---[ end trace 5c5a2ceca067e0ed ]---
>>>
>>> --
>>> -- Matthew Thode (prometheanfire)
>>
>> The root cause of Matt's issue is that intel_iommu_add_device() calls
>> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
>> expected value. The reason NULL is being returned is that Matt has a
>> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
>> 8231) on it. This device already has a quirk setup for disabling fast
>> back to back transfers on its secondary bus. If we cause it to use the
>> primary bus, that appears to resolve the issue. I'm not sure exactly
>> how to proceed from here due to relative lack of knowledge of PCI. Do
>> all PCIe-PCI bridges with secondary buses need their DMA parent to be
>> the primary bus or is that just something that should be done for the
>> TI XIO2000A due to the existing quirk?
>>
> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
> so the source of the device generating the DMA is unknown. When bridging
> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the
> source
> of a transaction -- in this case, DMA read/write transaction.
> This (sw) mapping should have happened by default, unless a recent
> change from VFIO
> broke this mapping.... or the TI bridge didn't report itself correctly
> as a PCIe-PCI bridge.
> Alex ?
>
>
>> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
>> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
>> that didn't panic and a device that worked.
>>
>> $ lspci -tvn
>> -+-[0000:ff]-+-00.0 8086:2c40
>> | +-00.1 8086:2c01
>> | +-02.0 8086:2c10
>> | +-02.1 8086:2c11
>> | +-02.4 8086:2c14
>> | +-02.5 8086:2c15
>> | +-03.0 8086:2c18
>> | +-03.1 8086:2c19
>> | +-03.2 8086:2c1a
>> | +-03.4 8086:2c1c
>> | +-04.0 8086:2c20
>> | +-04.1 8086:2c21
>> | +-04.2 8086:2c22
>> | +-04.3 8086:2c23
>> | +-05.0 8086:2c28
>> | +-05.1 8086:2c29
>> | +-05.2 8086:2c2a
>> | +-05.3 8086:2c2b
>> | +-06.0 8086:2c30
>> | +-06.1 8086:2c31
>> | +-06.2 8086:2c32
>> | \-06.3 8086:2c33
>> \-[0000:00]-+-00.0 8086:3406
>> +-01.0-[01]--+-00.0 8086:10c9
>> | \-00.1 8086:10c9
>> +-03.0-[02]--
>> +-05.0-[03]--
>> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
>> +-09.0-[06]----00.0 8086:10b9
>> +-13.0 8086:342d
>> +-14.0 8086:342e
>> +-14.1 8086:3422
>> +-14.2 8086:3423
>> +-14.3 8086:3438
>> +-16.0 8086:3430
>> +-16.1 8086:3431
>> +-16.2 8086:3432
>> +-16.3 8086:3433
>> +-16.4 8086:3429
>> +-16.5 8086:342a
>> +-16.6 8086:342b
>> +-16.7 8086:342c
>> +-1a.0 8086:3a37
>> +-1a.1 8086:3a38
>> +-1a.2 8086:3a39
>> +-1a.7 8086:3a3c
>> +-1d.0 8086:3a34
>> +-1d.1 8086:3a35
>> +-1d.2 8086:3a36
>> +-1d.7 8086:3a3a
>> +-1e.0-[07]----01.0 102b:0532
>> +-1f.0 8086:3a16
>> +-1f.2 8086:3a22
>> \-1f.3 8086:3a30
>>
>> If someone can craft the correct patch that'd be great or answer the
>> above question and I'll gladly craft it.
>>
>> Thanks.
>
because I didn't see it. Here was the patch that got it working for me
(ignore the printks), applies against 3.6.6 and 3.7-rc5.
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d4a4cd4..df0d7ef 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4118,11 +4118,20 @@ static int intel_iommu_add_device(struct device
*dev)
bridge = pci_find_upstream_pcie_bridge(pdev);
if (bridge) {
- if (pci_is_pcie(bridge))
+ if (pci_is_pcie(bridge)) {
dma_pdev = pci_get_domain_bus_and_slot(
pci_domain_nr(pdev->bus),
bridge->subordinate->number, 0);
- else
+ if (!dma_pdev) {
+ printk("MTHODE: debugging\n");
+ printk("domain: %d, bus: %d, bridge->sub %d\n",
+ pci_domain_nr(pdev->bus), pdev->bus,
+ bridge->subordinate->number);
+ dma_pdev = pci_get_domain_bus_and_slot(
+ pci_domain_nr(pdev->bus),
+ bridge->subordinate->number - 1, 0);
+ }
+ } else
dma_pdev = pci_dev_get(bridge);
} else
dma_pdev = pci_dev_get(pdev);
--
-- Matthew Thode (prometheanfire)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-12 19:57 ` Don Dutile
2012-11-12 21:05 ` Matthew Thode
@ 2012-11-12 22:19 ` Alex Williamson
2012-11-12 22:25 ` Matthew Thode
1 sibling, 1 reply; 11+ messages in thread
From: Alex Williamson @ 2012-11-12 22:19 UTC (permalink / raw)
To: Don Dutile
Cc: Doug Goldstein, prometheanfire, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
On Mon, 2012-11-12 at 14:57 -0500, Don Dutile wrote:
> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
> > On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
> > <prometheanfire@gentoo.org> wrote:
> >> System boots with vt-d disabled in bios. Otherwise I get the errors in
> >> the attached log. I can do whatever testing you need as this system is
> >> not in production yet. gonna paste the important part here. Let me
> >> know if you want anything else.
> >>
> >> Please CC me directly as I am not subscribed to the LKML.
> >>
> >>
> >> Trying to unpack rootfs image as initramfs...
> >> Freeing initrd memory: 5124k freed
> >> IOMMU 0 0xfbffe000: using Queued invalidation
> >> IOMMU: Setting RMRR:
> >> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
> >> 0xbf7fffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
> >> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
> >> IOMMU: Prepare 0-16MiB unity mapping for LPC
> >> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
> >> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
> >> BUG: unable to handle kernel NULL pointer dereference at 000000000000003c
> >> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> >> PGD 0
> >> Oops: 0000 [#1] SMP
> >> Modules linked in:
> >> CPU 7
> >> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
> >> Relion 1751/X8DTU
> >> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
> >> pci_get_dma_source+0xf/0x41
> >> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
> >> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
> >> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
> >> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
> >> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
> >> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
> >> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000) knlGS:0000000000000000
> >> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> >> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
> >> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
> >> ffff8806264cf910)
> >> Stack:
> >> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
> >> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
> >> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
> >> Call Trace:
> >> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> >> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> >> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> >> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> >> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> >> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> >> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> >> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> >> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> >> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> >> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> >> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> >> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> >> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> >> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
> >> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
> >> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
> >> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> >> RSP<ffff8806264d1d88>
> >> CR2: 000000000000003c
> >> ---[ end trace 5c5a2ceca067e0ec ]---
> >> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
> >>
> >> ------------[ cut here ]------------
> >> WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x25/0x51()
> >> Hardware name: Relion 1751
> >> Modules linked in:
> >> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
> >> Call Trace:
> >> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
> >> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
> >> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
> >> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
> >> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
> >> [<ffffffff810a132f>] update_process_times+0x62/0x73
> >> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
> >> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
> >> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
> >> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
> >> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
> >> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
> >> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
> >> [<ffffffff816f3261>] ? panic+0xee/0x1c5
> >> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
> >> [<ffffffff810371b8>] oops_end+0xb2/0xba
> >> [<ffffffff8105841d>] no_context+0x266/0x275
> >> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
> >> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
> >> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
> >> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
> >> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
> >> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
> >> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
> >> [<ffffffff816da653>] ? klist_put+0x4c/0x70
> >> [<ffffffff816da581>] ? klist_next+0x30/0xb6
> >> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
> >> [<ffffffff81058c42>] do_page_fault+0x9/0xb
> >> [<ffffffff816f6232>] page_fault+0x22/0x30
> >> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
> >> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
> >> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> >> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> >> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> >> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> >> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> >> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> >> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> >> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> >> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> >> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> >> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> >> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> >> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> >> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> >> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >> ---[ end trace 5c5a2ceca067e0ed ]---
> >>
> >> --
> >> -- Matthew Thode (prometheanfire)
> >
> > The root cause of Matt's issue is that intel_iommu_add_device() calls
> > pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
> > expected value. The reason NULL is being returned is that Matt has a
> > card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
> > 8231) on it. This device already has a quirk setup for disabling fast
> > back to back transfers on its secondary bus. If we cause it to use the
> > primary bus, that appears to resolve the issue. I'm not sure exactly
> > how to proceed from here due to relative lack of knowledge of PCI. Do
> > all PCIe-PCI bridges with secondary buses need their DMA parent to be
> > the primary bus or is that just something that should be done for the
> > TI XIO2000A due to the existing quirk?
> >
> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
> so the source of the device generating the DMA is unknown. When bridging
> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the source
> of a transaction -- in this case, DMA read/write transaction.
> This (sw) mapping should have happened by default, unless a recent change from VFIO
> broke this mapping.... or the TI bridge didn't report itself correctly
> as a PCIe-PCI bridge.
> Alex ?
Please post the full lspci -vvv log, I'm not quite sure I understand the
topology yet. Thanks,
Alex
> > The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
> > 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
> > that didn't panic and a device that worked.
> >
> > $ lspci -tvn
> > -+-[0000:ff]-+-00.0 8086:2c40
> > | +-00.1 8086:2c01
> > | +-02.0 8086:2c10
> > | +-02.1 8086:2c11
> > | +-02.4 8086:2c14
> > | +-02.5 8086:2c15
> > | +-03.0 8086:2c18
> > | +-03.1 8086:2c19
> > | +-03.2 8086:2c1a
> > | +-03.4 8086:2c1c
> > | +-04.0 8086:2c20
> > | +-04.1 8086:2c21
> > | +-04.2 8086:2c22
> > | +-04.3 8086:2c23
> > | +-05.0 8086:2c28
> > | +-05.1 8086:2c29
> > | +-05.2 8086:2c2a
> > | +-05.3 8086:2c2b
> > | +-06.0 8086:2c30
> > | +-06.1 8086:2c31
> > | +-06.2 8086:2c32
> > | \-06.3 8086:2c33
> > \-[0000:00]-+-00.0 8086:3406
> > +-01.0-[01]--+-00.0 8086:10c9
> > | \-00.1 8086:10c9
> > +-03.0-[02]--
> > +-05.0-[03]--
> > +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
> > +-09.0-[06]----00.0 8086:10b9
> > +-13.0 8086:342d
> > +-14.0 8086:342e
> > +-14.1 8086:3422
> > +-14.2 8086:3423
> > +-14.3 8086:3438
> > +-16.0 8086:3430
> > +-16.1 8086:3431
> > +-16.2 8086:3432
> > +-16.3 8086:3433
> > +-16.4 8086:3429
> > +-16.5 8086:342a
> > +-16.6 8086:342b
> > +-16.7 8086:342c
> > +-1a.0 8086:3a37
> > +-1a.1 8086:3a38
> > +-1a.2 8086:3a39
> > +-1a.7 8086:3a3c
> > +-1d.0 8086:3a34
> > +-1d.1 8086:3a35
> > +-1d.2 8086:3a36
> > +-1d.7 8086:3a3a
> > +-1e.0-[07]----01.0 102b:0532
> > +-1f.0 8086:3a16
> > +-1f.2 8086:3a22
> > \-1f.3 8086:3a30
> >
> > If someone can craft the correct patch that'd be great or answer the
> > above question and I'll gladly craft it.
> >
> > Thanks.
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-12 22:19 ` Alex Williamson
@ 2012-11-12 22:25 ` Matthew Thode
0 siblings, 0 replies; 11+ messages in thread
From: Matthew Thode @ 2012-11-12 22:25 UTC (permalink / raw)
To: Alex Williamson
Cc: Don Dutile, Doug Goldstein, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
[-- Attachment #1: Type: text/plain, Size: 85922 bytes --]
On 11/12/2012 04:19 PM, Alex Williamson wrote:
> On Mon, 2012-11-12 at 14:57 -0500, Don Dutile wrote:
>> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
>>> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
>>> <prometheanfire@gentoo.org> wrote:
>>>> System boots with vt-d disabled in bios. Otherwise I get the errors in
>>>> the attached log. I can do whatever testing you need as this system is
>>>> not in production yet. gonna paste the important part here. Let me
>>>> know if you want anything else.
>>>>
>>>> Please CC me directly as I am not subscribed to the LKML.
>>>>
>>>>
>>>> Trying to unpack rootfs image as initramfs...
>>>> Freeing initrd memory: 5124k freed
>>>> IOMMU 0 0xfbffe000: using Queued invalidation
>>>> IOMMU: Setting RMRR:
>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
>>>> 0xbf7fffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
>>>> IOMMU: Prepare 0-16MiB unity mapping for LPC
>>>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
>>>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>>>> BUG: unable to handle kernel NULL pointer dereference at 000000000000003c
>>>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>> PGD 0
>>>> Oops: 0000 [#1] SMP
>>>> Modules linked in:
>>>> CPU 7
>>>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
>>>> Relion 1751/X8DTU
>>>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
>>>> pci_get_dma_source+0xf/0x41
>>>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
>>>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
>>>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
>>>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
>>>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
>>>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
>>>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000) knlGS:0000000000000000
>>>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
>>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
>>>> ffff8806264cf910)
>>>> Stack:
>>>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
>>>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
>>>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
>>>> Call Trace:
>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
>>>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
>>>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
>>>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>> RSP<ffff8806264d1d88>
>>>> CR2: 000000000000003c
>>>> ---[ end trace 5c5a2ceca067e0ec ]---
>>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>>>>
>>>> ------------[ cut here ]------------
>>>> WARNING: at arch/x86/kernel/smp.c:123 native_smp_send_reschedule+0x25/0x51()
>>>> Hardware name: Relion 1751
>>>> Modules linked in:
>>>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
>>>> Call Trace:
>>>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
>>>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
>>>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
>>>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
>>>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
>>>> [<ffffffff810a132f>] update_process_times+0x62/0x73
>>>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
>>>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
>>>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
>>>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
>>>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
>>>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
>>>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
>>>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
>>>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
>>>> [<ffffffff810371b8>] oops_end+0xb2/0xba
>>>> [<ffffffff8105841d>] no_context+0x266/0x275
>>>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
>>>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
>>>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
>>>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
>>>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
>>>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
>>>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
>>>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
>>>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
>>>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
>>>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
>>>> [<ffffffff816f6232>] page_fault+0x22/0x30
>>>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
>>>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>> ---[ end trace 5c5a2ceca067e0ed ]---
>>>>
>>>> --
>>>> -- Matthew Thode (prometheanfire)
>>>
>>> The root cause of Matt's issue is that intel_iommu_add_device() calls
>>> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
>>> expected value. The reason NULL is being returned is that Matt has a
>>> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
>>> 8231) on it. This device already has a quirk setup for disabling fast
>>> back to back transfers on its secondary bus. If we cause it to use the
>>> primary bus, that appears to resolve the issue. I'm not sure exactly
>>> how to proceed from here due to relative lack of knowledge of PCI. Do
>>> all PCIe-PCI bridges with secondary buses need their DMA parent to be
>>> the primary bus or is that just something that should be done for the
>>> TI XIO2000A due to the existing quirk?
>>>
>> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
>> so the source of the device generating the DMA is unknown. When bridging
>> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the source
>> of a transaction -- in this case, DMA read/write transaction.
>> This (sw) mapping should have happened by default, unless a recent change from VFIO
>> broke this mapping.... or the TI bridge didn't report itself correctly
>> as a PCIe-PCI bridge.
>> Alex ?
>
> Please post the full lspci -vvv log, I'm not quite sure I understand the
> topology yet. Thanks,
>
> Alex
>
ok, this is with the device passed though to a VM now, so the driver for
it is now pci-stub, but that should be it I think.
00:00.0 Host bridge: Intel Corporation 5520 I/O Hub to ESI Port (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Capabilities: [60] MSI: Enable- Count=1/2 Maskable+ 64bit-
Address: 00000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [90] Express (v2) Root Port (Slot-), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0
<512ns, L1 <64us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range BCD, TimeoutDis+, LTR-, OBFF Not
Supported ARIFwd+
DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF
Disabled ARIFwd-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [150 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0
Len=00c <?>
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express
Root Port 1 (rev 22) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fbb00000-fbbfffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: Super Micro Computer Inc Device 0600
Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
Address: fee00358 Data: 0000
Masking: 00000002 Pending: 00000000
Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 256 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0
<512ns, L1 <64us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+
BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range BCD, TimeoutDis+, LTR-, OBFF Not
Supported ARIFwd+
DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF
Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [150 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0
Len=00c <?>
Kernel driver in use: pcieport
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express
Root Port 3 (rev 22) (prog-if 00 [Normal decode])
Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000f000-00000fff
Memory behind bridge: fff00000-000fffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: Super Micro Computer Inc Device 0600
Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
Address: fee00378 Data: 0000
Masking: 00000002 Pending: 00000000
Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0
<512ns, L1 <64us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #3, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range BCD, TimeoutDis+, LTR-, OBFF Not
Supported ARIFwd+
DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF
Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [150 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0
Len=00c <?>
Kernel driver in use: pcieport
00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root
Port 5 (rev 22) (prog-if 00 [Normal decode])
Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
I/O behind bridge: 0000f000-00000fff
Memory behind bridge: fff00000-000fffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: Super Micro Computer Inc Device 0600
Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
Address: fee00398 Data: 0000
Masking: 00000002 Pending: 00000000
Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0
<512ns, L1 <64us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #2, PowerLimit 25.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState-
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range BCD, TimeoutDis+, LTR-, OBFF Not
Supported ARIFwd+
DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF
Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [150 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
Kernel driver in use: pcieport
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express
Root Port 7 (rev 22) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Bus: primary=00, secondary=04, subordinate=05, sec-latency=0
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: fbc00000-fbcfffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: Super Micro Computer Inc Device 0600
Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
Address: fee003b8 Data: 0000
Masking: 00000002 Pending: 00000000
Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 256 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0
<512ns, L1 <64us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #1, PowerLimit 10.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range BCD, TimeoutDis+, LTR-, OBFF Not
Supported ARIFwd+
DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF
Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [150 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0
Len=00c <?>
Kernel driver in use: pcieport
00:09.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI
Express Root Port 9 (rev 22) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
I/O behind bridge: 0000e000-0000efff
Memory behind bridge: fbd00000-fbdfffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: Super Micro Computer Inc Device 0600
Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
Address: fee003d8 Data: 0000
Masking: 00000002 Pending: 00000000
Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ RBE+ FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 256 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0
<512ns, L1 <64us
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #1, PowerLimit 10.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd Off, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range BCD, TimeoutDis+, LTR-, OBFF Not
Supported ARIFwd+
DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF
Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [150 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+
EgressCtrl- DirectTrans-
Kernel driver in use: pcieport
00:13.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub I/OxAPIC
Interrupt Controller (rev 22) (prog-if 20 [IO(X)-APIC])
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
Region 0: Memory at fec8a000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [6c] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System
Management Registers (rev 22) (prog-if 00 [8259])
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0
unlimited, L1 unlimited
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and
Scratch Pad Registers (rev 22) (prog-if 00 [8259])
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0
unlimited, L1 unlimited
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status
and RAS Registers (rev 22) (prog-if 00 [8259])
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0
unlimited, L1 unlimited
ClockPM- Surprise+ LLActRep+ BwNot+
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle
Registers (rev 22) (prog-if 00 [8259])
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin A routed to IRQ 43
Region 0: Memory at fbef8000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin B routed to IRQ 44
Region 0: Memory at fbef4000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.2 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin C routed to IRQ 45
Region 0: Memory at fbef0000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.3 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin D routed to IRQ 46
Region 0: Memory at fbeec000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.4 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin A routed to IRQ 43
Region 0: Memory at fbee8000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.5 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin B routed to IRQ 44
Region 0: Memory at fbee4000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.6 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin C routed to IRQ 45
Region 0: Memory at fbee0000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:16.7 System peripheral: Intel Corporation 5520/5500/X58 Chipset
QuickData Technology Device (rev 22)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin D routed to IRQ 46
Region 0: Memory at fbedc000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [80] MSI-X: Enable+ Count=1 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0
<64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF
Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: ioatdma
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #4 (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 16
Region 4: I/O ports at bc00 [size=32]
Capabilities: [50] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: uhci_hcd
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #5 (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin B routed to IRQ 21
Region 4: I/O ports at b880 [size=32]
Capabilities: [50] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: uhci_hcd
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #6 (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin D routed to IRQ 19
Region 4: I/O ports at b800 [size=32]
Capabilities: [50] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: uhci_hcd
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2
EHCI Controller #2 (prog-if 20 [EHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin C routed to IRQ 18
Region 0: Memory at fbeda000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: ehci_hcd
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #1 (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 23
Region 4: I/O ports at b480 [size=32]
Capabilities: [50] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: uhci_hcd
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #2 (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin B routed to IRQ 19
Region 4: I/O ports at b400 [size=32]
Capabilities: [50] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: uhci_hcd
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB
UHCI Controller #3 (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin C routed to IRQ 18
Region 4: I/O ports at b080 [size=32]
Capabilities: [50] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: uhci_hcd
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2
EHCI Controller #1 (prog-if 20 [EHCI])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 23
Region 0: Memory at fbed8000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: ehci_hcd
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) (prog-if
01 [Subtractive decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
Bus: primary=00, secondary=07, subordinate=07, sec-latency=32
I/O behind bridge: 0000f000-00000fff
Memory behind bridge: faf00000-fb7fffff
Prefetchable memory behind bridge: 00000000f9000000-00000000f9ffffff
Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Subsystem: Super Micro Computer Inc Device 0600
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface
Controller
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Capabilities: [e0] Vendor Specific Information: Len=0c <?>
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA
AHCI Controller (prog-if 01 [AHCI 1.0])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin B routed to IRQ 78
Region 0: I/O ports at a480 [size=8]
Region 1: I/O ports at b000 [size=4]
Region 2: I/O ports at ac00 [size=8]
Region 3: I/O ports at a880 [size=4]
Region 4: I/O ports at a800 [size=32]
Region 5: Memory at fbed6000 (32-bit, non-prefetchable) [size=2K]
Capabilities: [80] MSI: Enable+ Count=1/16 Maskable- 64bit-
Address: fee00578 Data: 0000
Capabilities: [70] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [a8] SATA HBA v1.0 BAR4 Offset=00000004
Capabilities: [b0] PCI Advanced Features
AFCap: TP+ FLR+
AFCtrl: FLR-
AFStatus: TP-
Kernel driver in use: ahci
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin C routed to IRQ 18
Region 0: Memory at fbed4000 (64-bit, non-prefetchable) [size=256]
Region 4: I/O ports at 0400 [size=32]
Kernel driver in use: i801_smbus
Kernel modules: i2c-i801
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin A routed to IRQ 28
Region 0: Memory at fbbe0000 (32-bit, non-prefetchable) [size=128K]
Region 1: Memory at fbbc0000 (32-bit, non-prefetchable) [size=128K]
Region 2: I/O ports at cc00 [size=32]
Region 3: Memory at fbb9c000 (32-bit, non-prefetchable) [size=16K]
Expansion ROM at fbba0000 [disabled] [size=128K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00002000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0
<4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not
Supported
DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance-
ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [140 v1] Device Serial Number 00-25-90-ff-ff-0c-d8-ea
Kernel driver in use: igb
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin B routed to IRQ 40
Region 0: Memory at fbb60000 (32-bit, non-prefetchable) [size=128K]
Region 1: Memory at fbb40000 (32-bit, non-prefetchable) [size=128K]
Region 2: I/O ports at c880 [size=32]
Region 3: Memory at fbb1c000 (32-bit, non-prefetchable) [size=16K]
Expansion ROM at fbb20000 [disabled] [size=128K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00002000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0
<4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not
Supported
DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR-, OBFF
Disabled
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [140 v1] Device Serial Number 00-25-90-ff-ff-0c-d8-ea
Kernel driver in use: igb
04:00.0 PCI bridge: Texas Instruments XIO2000(A)/XIO2200A PCI
Express-to-PCI Bridge (rev 03) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Bus: primary=04, secondary=05, subordinate=05, sec-latency=64
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: fbc00000-fbcfffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+
Capabilities: [60] MSI: Enable- Count=1/16 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [80] Subsystem: Gammagraphx, Inc. (or missing ID) Device 0000
Capabilities: [90] Express (v1) PCI/PCI-X Bridge, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <4us, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ BrConfRtry-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0
<512ns, L1 <16us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM L0s Enabled; Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn-
05:08.0 Ethernet controller: Digium, Inc. Wildcard AEX410 4-port analog
card (PCI-Express) (rev 11)
Subsystem: Digium, Inc. Wildcard AEX410 4-port analog card (PCI-Express)
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV+ VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 30
Region 0: I/O ports at d800 [size=256]
Region 1: Memory at fbcde000 (32-bit, non-prefetchable) [size=1K]
Expansion ROM at fbce0000 [size=128K]
Capabilities: [c0] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: pci-stub
06:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
Controller (Copper) (rev 06)
Subsystem: Intel Corporation PRO/1000 PT Desktop Adapter
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 256 bytes
Interrupt: pin A routed to IRQ 79
Region 0: Memory at fbde0000 (32-bit, non-prefetchable) [size=128K]
Region 1: Memory at fbdc0000 (32-bit, non-prefetchable) [size=128K]
Region 2: I/O ports at ec00 [size=32]
Expansion ROM at fbda0000 [disabled] [size=128K]
Capabilities: [c8] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee005b8 Data: 0000
Capabilities: [e0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us,
L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF-
MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+
MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Capabilities: [140 v1] Device Serial Number 00-15-17-ff-ff-1e-24-ca
Kernel driver in use: e1000e
07:01.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA
G200eW WPCM450 (rev 0a) (prog-if 00 [VGA controller])
Subsystem: Super Micro Computer Inc Device 0600
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (4000ns min, 8000ns max), Cache Line Size: 256 bytes
Interrupt: pin A routed to IRQ 10
Region 0: Memory at f9000000 (32-bit, prefetchable) [size=16M]
Region 1: Memory at faffc000 (32-bit, non-prefetchable) [size=16K]
Region 2: Memory at fb000000 (32-bit, non-prefetchable) [size=8M]
Expansion ROM at <unassigned> [disabled]
Capabilities: [dc] Power Management version 1
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath
Architecture Generic Non-Core Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath
Architecture System Address Decoder (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0
(rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:02.4 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 1 (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:02.5 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 1
(rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Target Address Decoder (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:03.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller RAS Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Test Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 0 Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 0 Address Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 0 Rank Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 0 Thermal Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 1 Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 1 Address Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 1 Rank Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 1 Thermal Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 2 Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 2 Address Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 2 Rank Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated
Memory Controller Channel 2 Thermal Control Registers (rev 05)
Subsystem: Intel Corporation Device 8086
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0
>>> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
>>> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
>>> that didn't panic and a device that worked.
>>>
>>> $ lspci -tvn
>>> -+-[0000:ff]-+-00.0 8086:2c40
>>> | +-00.1 8086:2c01
>>> | +-02.0 8086:2c10
>>> | +-02.1 8086:2c11
>>> | +-02.4 8086:2c14
>>> | +-02.5 8086:2c15
>>> | +-03.0 8086:2c18
>>> | +-03.1 8086:2c19
>>> | +-03.2 8086:2c1a
>>> | +-03.4 8086:2c1c
>>> | +-04.0 8086:2c20
>>> | +-04.1 8086:2c21
>>> | +-04.2 8086:2c22
>>> | +-04.3 8086:2c23
>>> | +-05.0 8086:2c28
>>> | +-05.1 8086:2c29
>>> | +-05.2 8086:2c2a
>>> | +-05.3 8086:2c2b
>>> | +-06.0 8086:2c30
>>> | +-06.1 8086:2c31
>>> | +-06.2 8086:2c32
>>> | \-06.3 8086:2c33
>>> \-[0000:00]-+-00.0 8086:3406
>>> +-01.0-[01]--+-00.0 8086:10c9
>>> | \-00.1 8086:10c9
>>> +-03.0-[02]--
>>> +-05.0-[03]--
>>> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
>>> +-09.0-[06]----00.0 8086:10b9
>>> +-13.0 8086:342d
>>> +-14.0 8086:342e
>>> +-14.1 8086:3422
>>> +-14.2 8086:3423
>>> +-14.3 8086:3438
>>> +-16.0 8086:3430
>>> +-16.1 8086:3431
>>> +-16.2 8086:3432
>>> +-16.3 8086:3433
>>> +-16.4 8086:3429
>>> +-16.5 8086:342a
>>> +-16.6 8086:342b
>>> +-16.7 8086:342c
>>> +-1a.0 8086:3a37
>>> +-1a.1 8086:3a38
>>> +-1a.2 8086:3a39
>>> +-1a.7 8086:3a3c
>>> +-1d.0 8086:3a34
>>> +-1d.1 8086:3a35
>>> +-1d.2 8086:3a36
>>> +-1d.7 8086:3a3a
>>> +-1e.0-[07]----01.0 102b:0532
>>> +-1f.0 8086:3a16
>>> +-1f.2 8086:3a22
>>> \-1f.3 8086:3a30
>>>
>>> If someone can craft the correct patch that'd be great or answer the
>>> above question and I'll gladly craft it.
>>>
>>> Thanks.
>>
>
>
>
--
-- Matthew Thode (prometheanfire)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-12 21:05 ` Matthew Thode
@ 2012-11-13 15:38 ` Alex Williamson
2012-11-13 15:50 ` Matthew Thode
2012-11-13 19:05 ` Don Dutile
0 siblings, 2 replies; 11+ messages in thread
From: Alex Williamson @ 2012-11-13 15:38 UTC (permalink / raw)
To: prometheanfire
Cc: Don Dutile, Doug Goldstein, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
On Mon, 2012-11-12 at 15:05 -0600, Matthew Thode wrote:
> On 11/12/2012 01:57 PM, Don Dutile wrote:
> > On 11/12/2012 04:26 AM, Doug Goldstein wrote:
> >> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
> >> <prometheanfire@gentoo.org> wrote:
> >>> System boots with vt-d disabled in bios. Otherwise I get the errors in
> >>> the attached log. I can do whatever testing you need as this system is
> >>> not in production yet. gonna paste the important part here. Let me
> >>> know if you want anything else.
> >>>
> >>> Please CC me directly as I am not subscribed to the LKML.
> >>>
> >>>
> >>> Trying to unpack rootfs image as initramfs...
> >>> Freeing initrd memory: 5124k freed
> >>> IOMMU 0 0xfbffe000: using Queued invalidation
> >>> IOMMU: Setting RMRR:
> >>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
> >>> 0xbf7fffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
> >>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
> >>> IOMMU: Prepare 0-16MiB unity mapping for LPC
> >>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
> >>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
> >>> BUG: unable to handle kernel NULL pointer dereference at
> >>> 000000000000003c
> >>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> >>> PGD 0
> >>> Oops: 0000 [#1] SMP
> >>> Modules linked in:
> >>> CPU 7
> >>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
> >>> Relion 1751/X8DTU
> >>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
> >>> pci_get_dma_source+0xf/0x41
> >>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
> >>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
> >>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
> >>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
> >>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
> >>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
> >>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000)
> >>> knlGS:0000000000000000
> >>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> >>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
> >>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
> >>> ffff8806264cf910)
> >>> Stack:
> >>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
> >>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
> >>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
> >>> Call Trace:
> >>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> >>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> >>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> >>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> >>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> >>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> >>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> >>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> >>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> >>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> >>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> >>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> >>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> >>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> >>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
> >>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
> >>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
> >>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> >>> RSP<ffff8806264d1d88>
> >>> CR2: 000000000000003c
> >>> ---[ end trace 5c5a2ceca067e0ec ]---
> >>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
> >>>
> >>> ------------[ cut here ]------------
> >>> WARNING: at arch/x86/kernel/smp.c:123
> >>> native_smp_send_reschedule+0x25/0x51()
> >>> Hardware name: Relion 1751
> >>> Modules linked in:
> >>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
> >>> Call Trace:
> >>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
> >>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
> >>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
> >>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
> >>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
> >>> [<ffffffff810a132f>] update_process_times+0x62/0x73
> >>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
> >>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
> >>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
> >>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
> >>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
> >>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
> >>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
> >>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
> >>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
> >>> [<ffffffff810371b8>] oops_end+0xb2/0xba
> >>> [<ffffffff8105841d>] no_context+0x266/0x275
> >>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
> >>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
> >>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
> >>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
> >>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
> >>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
> >>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
> >>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
> >>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
> >>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
> >>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
> >>> [<ffffffff816f6232>] page_fault+0x22/0x30
> >>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
> >>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
> >>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> >>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> >>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> >>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> >>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> >>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> >>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> >>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> >>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> >>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> >>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> >>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> >>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> >>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> >>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>> ---[ end trace 5c5a2ceca067e0ed ]---
> >>>
> >>> --
> >>> -- Matthew Thode (prometheanfire)
> >>
> >> The root cause of Matt's issue is that intel_iommu_add_device() calls
> >> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
> >> expected value. The reason NULL is being returned is that Matt has a
> >> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
> >> 8231) on it. This device already has a quirk setup for disabling fast
> >> back to back transfers on its secondary bus. If we cause it to use the
> >> primary bus, that appears to resolve the issue. I'm not sure exactly
> >> how to proceed from here due to relative lack of knowledge of PCI. Do
> >> all PCIe-PCI bridges with secondary buses need their DMA parent to be
> >> the primary bus or is that just something that should be done for the
> >> TI XIO2000A due to the existing quirk?
> >>
> > DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
> > so the source of the device generating the DMA is unknown. When bridging
> > to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the
> > source
> > of a transaction -- in this case, DMA read/write transaction.
> > This (sw) mapping should have happened by default, unless a recent
> > change from VFIO
> > broke this mapping.... or the TI bridge didn't report itself correctly
> > as a PCIe-PCI bridge.
> > Alex ?
> >
> >
> >> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
> >> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
> >> that didn't panic and a device that worked.
> >>
> >> $ lspci -tvn
> >> -+-[0000:ff]-+-00.0 8086:2c40
> >> | +-00.1 8086:2c01
> >> | +-02.0 8086:2c10
> >> | +-02.1 8086:2c11
> >> | +-02.4 8086:2c14
> >> | +-02.5 8086:2c15
> >> | +-03.0 8086:2c18
> >> | +-03.1 8086:2c19
> >> | +-03.2 8086:2c1a
> >> | +-03.4 8086:2c1c
> >> | +-04.0 8086:2c20
> >> | +-04.1 8086:2c21
> >> | +-04.2 8086:2c22
> >> | +-04.3 8086:2c23
> >> | +-05.0 8086:2c28
> >> | +-05.1 8086:2c29
> >> | +-05.2 8086:2c2a
> >> | +-05.3 8086:2c2b
> >> | +-06.0 8086:2c30
> >> | +-06.1 8086:2c31
> >> | +-06.2 8086:2c32
> >> | \-06.3 8086:2c33
> >> \-[0000:00]-+-00.0 8086:3406
> >> +-01.0-[01]--+-00.0 8086:10c9
> >> | \-00.1 8086:10c9
> >> +-03.0-[02]--
> >> +-05.0-[03]--
> >> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
> >> +-09.0-[06]----00.0 8086:10b9
> >> +-13.0 8086:342d
> >> +-14.0 8086:342e
> >> +-14.1 8086:3422
> >> +-14.2 8086:3423
> >> +-14.3 8086:3438
> >> +-16.0 8086:3430
> >> +-16.1 8086:3431
> >> +-16.2 8086:3432
> >> +-16.3 8086:3433
> >> +-16.4 8086:3429
> >> +-16.5 8086:342a
> >> +-16.6 8086:342b
> >> +-16.7 8086:342c
> >> +-1a.0 8086:3a37
> >> +-1a.1 8086:3a38
> >> +-1a.2 8086:3a39
> >> +-1a.7 8086:3a3c
> >> +-1d.0 8086:3a34
> >> +-1d.1 8086:3a35
> >> +-1d.2 8086:3a36
> >> +-1d.7 8086:3a3a
> >> +-1e.0-[07]----01.0 102b:0532
> >> +-1f.0 8086:3a16
> >> +-1f.2 8086:3a22
> >> \-1f.3 8086:3a30
> >>
> >> If someone can craft the correct patch that'd be great or answer the
> >> above question and I'll gladly craft it.
> >>
> >> Thanks.
> >
> because I didn't see it. Here was the patch that got it working for me
> (ignore the printks), applies against 3.6.6 and 3.7-rc5.
I think you're on the right track, but the solution is too specific.
Here's a version that will fall back to the bridge device for the base
of the group. There may be opportunities to get rid of the pci_get_
call altogether, but this seems pretty safe. Can you please test it?
Thanks,
Alex
commit ca15170f05b140ab8c611db5cb7cb9c218ddc930
Author: Alex Williamson <alex.williamson@redhat.com>
Date: Tue Nov 13 08:34:08 2012 -0700
intel-iommu: Fix lookup in add device
We can't assume this device exists, fall back to the bridge itself.
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d4a4cd4..0badfa4 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4108,7 +4108,7 @@ static void swap_pci_ref(struct pci_dev **from, struct pci_dev *to)
static int intel_iommu_add_device(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
- struct pci_dev *bridge, *dma_pdev;
+ struct pci_dev *bridge, *dma_pdev = NULL;
struct iommu_group *group;
int ret;
@@ -4122,7 +4122,7 @@ static int intel_iommu_add_device(struct device *dev)
dma_pdev = pci_get_domain_bus_and_slot(
pci_domain_nr(pdev->bus),
bridge->subordinate->number, 0);
- else
+ if (!dma_pdev)
dma_pdev = pci_dev_get(bridge);
} else
dma_pdev = pci_dev_get(pdev);
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-13 15:38 ` Alex Williamson
@ 2012-11-13 15:50 ` Matthew Thode
2012-11-13 19:05 ` Don Dutile
1 sibling, 0 replies; 11+ messages in thread
From: Matthew Thode @ 2012-11-13 15:50 UTC (permalink / raw)
To: Alex Williamson
Cc: Don Dutile, Doug Goldstein, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
[-- Attachment #1: Type: text/plain, Size: 13998 bytes --]
On 11/13/2012 09:38 AM, Alex Williamson wrote:
> On Mon, 2012-11-12 at 15:05 -0600, Matthew Thode wrote:
>> On 11/12/2012 01:57 PM, Don Dutile wrote:
>>> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
>>>> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
>>>> <prometheanfire@gentoo.org> wrote:
>>>>> System boots with vt-d disabled in bios. Otherwise I get the errors in
>>>>> the attached log. I can do whatever testing you need as this system is
>>>>> not in production yet. gonna paste the important part here. Let me
>>>>> know if you want anything else.
>>>>>
>>>>> Please CC me directly as I am not subscribed to the LKML.
>>>>>
>>>>>
>>>>> Trying to unpack rootfs image as initramfs...
>>>>> Freeing initrd memory: 5124k freed
>>>>> IOMMU 0 0xfbffe000: using Queued invalidation
>>>>> IOMMU: Setting RMRR:
>>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
>>>>> IOMMU: Prepare 0-16MiB unity mapping for LPC
>>>>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
>>>>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>>>>> BUG: unable to handle kernel NULL pointer dereference at
>>>>> 000000000000003c
>>>>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>>> PGD 0
>>>>> Oops: 0000 [#1] SMP
>>>>> Modules linked in:
>>>>> CPU 7
>>>>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
>>>>> Relion 1751/X8DTU
>>>>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
>>>>> pci_get_dma_source+0xf/0x41
>>>>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
>>>>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
>>>>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
>>>>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
>>>>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
>>>>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
>>>>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000)
>>>>> knlGS:0000000000000000
>>>>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>>>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
>>>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>>>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
>>>>> ffff8806264cf910)
>>>>> Stack:
>>>>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
>>>>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
>>>>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
>>>>> Call Trace:
>>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
>>>>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
>>>>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
>>>>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>>> RSP<ffff8806264d1d88>
>>>>> CR2: 000000000000003c
>>>>> ---[ end trace 5c5a2ceca067e0ec ]---
>>>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>>>>>
>>>>> ------------[ cut here ]------------
>>>>> WARNING: at arch/x86/kernel/smp.c:123
>>>>> native_smp_send_reschedule+0x25/0x51()
>>>>> Hardware name: Relion 1751
>>>>> Modules linked in:
>>>>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
>>>>> Call Trace:
>>>>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
>>>>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
>>>>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
>>>>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
>>>>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
>>>>> [<ffffffff810a132f>] update_process_times+0x62/0x73
>>>>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
>>>>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
>>>>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
>>>>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
>>>>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
>>>>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
>>>>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
>>>>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
>>>>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
>>>>> [<ffffffff810371b8>] oops_end+0xb2/0xba
>>>>> [<ffffffff8105841d>] no_context+0x266/0x275
>>>>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
>>>>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
>>>>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
>>>>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
>>>>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
>>>>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
>>>>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
>>>>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
>>>>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
>>>>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
>>>>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
>>>>> [<ffffffff816f6232>] page_fault+0x22/0x30
>>>>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
>>>>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
>>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> ---[ end trace 5c5a2ceca067e0ed ]---
>>>>>
>>>>> --
>>>>> -- Matthew Thode (prometheanfire)
>>>>
>>>> The root cause of Matt's issue is that intel_iommu_add_device() calls
>>>> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
>>>> expected value. The reason NULL is being returned is that Matt has a
>>>> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
>>>> 8231) on it. This device already has a quirk setup for disabling fast
>>>> back to back transfers on its secondary bus. If we cause it to use the
>>>> primary bus, that appears to resolve the issue. I'm not sure exactly
>>>> how to proceed from here due to relative lack of knowledge of PCI. Do
>>>> all PCIe-PCI bridges with secondary buses need their DMA parent to be
>>>> the primary bus or is that just something that should be done for the
>>>> TI XIO2000A due to the existing quirk?
>>>>
>>> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
>>> so the source of the device generating the DMA is unknown. When bridging
>>> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the
>>> source
>>> of a transaction -- in this case, DMA read/write transaction.
>>> This (sw) mapping should have happened by default, unless a recent
>>> change from VFIO
>>> broke this mapping.... or the TI bridge didn't report itself correctly
>>> as a PCIe-PCI bridge.
>>> Alex ?
>>>
>>>
>>>> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
>>>> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
>>>> that didn't panic and a device that worked.
>>>>
>>>> $ lspci -tvn
>>>> -+-[0000:ff]-+-00.0 8086:2c40
>>>> | +-00.1 8086:2c01
>>>> | +-02.0 8086:2c10
>>>> | +-02.1 8086:2c11
>>>> | +-02.4 8086:2c14
>>>> | +-02.5 8086:2c15
>>>> | +-03.0 8086:2c18
>>>> | +-03.1 8086:2c19
>>>> | +-03.2 8086:2c1a
>>>> | +-03.4 8086:2c1c
>>>> | +-04.0 8086:2c20
>>>> | +-04.1 8086:2c21
>>>> | +-04.2 8086:2c22
>>>> | +-04.3 8086:2c23
>>>> | +-05.0 8086:2c28
>>>> | +-05.1 8086:2c29
>>>> | +-05.2 8086:2c2a
>>>> | +-05.3 8086:2c2b
>>>> | +-06.0 8086:2c30
>>>> | +-06.1 8086:2c31
>>>> | +-06.2 8086:2c32
>>>> | \-06.3 8086:2c33
>>>> \-[0000:00]-+-00.0 8086:3406
>>>> +-01.0-[01]--+-00.0 8086:10c9
>>>> | \-00.1 8086:10c9
>>>> +-03.0-[02]--
>>>> +-05.0-[03]--
>>>> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
>>>> +-09.0-[06]----00.0 8086:10b9
>>>> +-13.0 8086:342d
>>>> +-14.0 8086:342e
>>>> +-14.1 8086:3422
>>>> +-14.2 8086:3423
>>>> +-14.3 8086:3438
>>>> +-16.0 8086:3430
>>>> +-16.1 8086:3431
>>>> +-16.2 8086:3432
>>>> +-16.3 8086:3433
>>>> +-16.4 8086:3429
>>>> +-16.5 8086:342a
>>>> +-16.6 8086:342b
>>>> +-16.7 8086:342c
>>>> +-1a.0 8086:3a37
>>>> +-1a.1 8086:3a38
>>>> +-1a.2 8086:3a39
>>>> +-1a.7 8086:3a3c
>>>> +-1d.0 8086:3a34
>>>> +-1d.1 8086:3a35
>>>> +-1d.2 8086:3a36
>>>> +-1d.7 8086:3a3a
>>>> +-1e.0-[07]----01.0 102b:0532
>>>> +-1f.0 8086:3a16
>>>> +-1f.2 8086:3a22
>>>> \-1f.3 8086:3a30
>>>>
>>>> If someone can craft the correct patch that'd be great or answer the
>>>> above question and I'll gladly craft it.
>>>>
>>>> Thanks.
>>>
>> because I didn't see it. Here was the patch that got it working for me
>> (ignore the printks), applies against 3.6.6 and 3.7-rc5.
>
> I think you're on the right track, but the solution is too specific.
> Here's a version that will fall back to the bridge device for the base
> of the group. There may be opportunities to get rid of the pci_get_
> call altogether, but this seems pretty safe. Can you please test it?
> Thanks,
>
> Alex
>
>
> commit ca15170f05b140ab8c611db5cb7cb9c218ddc930
> Author: Alex Williamson <alex.williamson@redhat.com>
> Date: Tue Nov 13 08:34:08 2012 -0700
>
> intel-iommu: Fix lookup in add device
>
> We can't assume this device exists, fall back to the bridge itself.
>
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
>
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index d4a4cd4..0badfa4 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -4108,7 +4108,7 @@ static void swap_pci_ref(struct pci_dev **from, struct pci_dev *to)
> static int intel_iommu_add_device(struct device *dev)
> {
> struct pci_dev *pdev = to_pci_dev(dev);
> - struct pci_dev *bridge, *dma_pdev;
> + struct pci_dev *bridge, *dma_pdev = NULL;
> struct iommu_group *group;
> int ret;
>
> @@ -4122,7 +4122,7 @@ static int intel_iommu_add_device(struct device *dev)
> dma_pdev = pci_get_domain_bus_and_slot(
> pci_domain_nr(pdev->bus),
> bridge->subordinate->number, 0);
> - else
> + if (!dma_pdev)
> dma_pdev = pci_dev_get(bridge);
> } else
> dma_pdev = pci_dev_get(pdev);
>
>
It works :D
--
-- Matthew Thode (prometheanfire)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-13 15:38 ` Alex Williamson
2012-11-13 15:50 ` Matthew Thode
@ 2012-11-13 19:05 ` Don Dutile
2012-11-13 19:10 ` Alex Williamson
1 sibling, 1 reply; 11+ messages in thread
From: Don Dutile @ 2012-11-13 19:05 UTC (permalink / raw)
To: Alex Williamson
Cc: prometheanfire, Doug Goldstein, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
On 11/13/2012 10:38 AM, Alex Williamson wrote:
> On Mon, 2012-11-12 at 15:05 -0600, Matthew Thode wrote:
>> On 11/12/2012 01:57 PM, Don Dutile wrote:
>>> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
>>>> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
>>>> <prometheanfire@gentoo.org> wrote:
>>>>> System boots with vt-d disabled in bios. Otherwise I get the errors in
>>>>> the attached log. I can do whatever testing you need as this system is
>>>>> not in production yet. gonna paste the important part here. Let me
>>>>> know if you want anything else.
>>>>>
>>>>> Please CC me directly as I am not subscribed to the LKML.
>>>>>
>>>>>
>>>>> Trying to unpack rootfs image as initramfs...
>>>>> Freeing initrd memory: 5124k freed
>>>>> IOMMU 0 0xfbffe000: using Queued invalidation
>>>>> IOMMU: Setting RMRR:
>>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
>>>>> 0xbf7fffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
>>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
>>>>> IOMMU: Prepare 0-16MiB unity mapping for LPC
>>>>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
>>>>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>>>>> BUG: unable to handle kernel NULL pointer dereference at
>>>>> 000000000000003c
>>>>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>>> PGD 0
>>>>> Oops: 0000 [#1] SMP
>>>>> Modules linked in:
>>>>> CPU 7
>>>>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
>>>>> Relion 1751/X8DTU
>>>>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
>>>>> pci_get_dma_source+0xf/0x41
>>>>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
>>>>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
>>>>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
>>>>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
>>>>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
>>>>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
>>>>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000)
>>>>> knlGS:0000000000000000
>>>>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>>>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
>>>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>>>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
>>>>> ffff8806264cf910)
>>>>> Stack:
>>>>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
>>>>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
>>>>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
>>>>> Call Trace:
>>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
>>>>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
>>>>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
>>>>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>>> RSP<ffff8806264d1d88>
>>>>> CR2: 000000000000003c
>>>>> ---[ end trace 5c5a2ceca067e0ec ]---
>>>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>>>>>
>>>>> ------------[ cut here ]------------
>>>>> WARNING: at arch/x86/kernel/smp.c:123
>>>>> native_smp_send_reschedule+0x25/0x51()
>>>>> Hardware name: Relion 1751
>>>>> Modules linked in:
>>>>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
>>>>> Call Trace:
>>>>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
>>>>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
>>>>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
>>>>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
>>>>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
>>>>> [<ffffffff810a132f>] update_process_times+0x62/0x73
>>>>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
>>>>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
>>>>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
>>>>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
>>>>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
>>>>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
>>>>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
>>>>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
>>>>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
>>>>> [<ffffffff810371b8>] oops_end+0xb2/0xba
>>>>> [<ffffffff8105841d>] no_context+0x266/0x275
>>>>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
>>>>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
>>>>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
>>>>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
>>>>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
>>>>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
>>>>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
>>>>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
>>>>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
>>>>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
>>>>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
>>>>> [<ffffffff816f6232>] page_fault+0x22/0x30
>>>>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
>>>>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
>>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>> ---[ end trace 5c5a2ceca067e0ed ]---
>>>>>
>>>>> --
>>>>> -- Matthew Thode (prometheanfire)
>>>>
>>>> The root cause of Matt's issue is that intel_iommu_add_device() calls
>>>> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
>>>> expected value. The reason NULL is being returned is that Matt has a
>>>> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
>>>> 8231) on it. This device already has a quirk setup for disabling fast
>>>> back to back transfers on its secondary bus. If we cause it to use the
>>>> primary bus, that appears to resolve the issue. I'm not sure exactly
>>>> how to proceed from here due to relative lack of knowledge of PCI. Do
>>>> all PCIe-PCI bridges with secondary buses need their DMA parent to be
>>>> the primary bus or is that just something that should be done for the
>>>> TI XIO2000A due to the existing quirk?
>>>>
>>> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
>>> so the source of the device generating the DMA is unknown. When bridging
>>> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the
>>> source
>>> of a transaction -- in this case, DMA read/write transaction.
>>> This (sw) mapping should have happened by default, unless a recent
>>> change from VFIO
>>> broke this mapping.... or the TI bridge didn't report itself correctly
>>> as a PCIe-PCI bridge.
>>> Alex ?
>>>
>>>
>>>> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
>>>> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
>>>> that didn't panic and a device that worked.
>>>>
>>>> $ lspci -tvn
>>>> -+-[0000:ff]-+-00.0 8086:2c40
>>>> | +-00.1 8086:2c01
>>>> | +-02.0 8086:2c10
>>>> | +-02.1 8086:2c11
>>>> | +-02.4 8086:2c14
>>>> | +-02.5 8086:2c15
>>>> | +-03.0 8086:2c18
>>>> | +-03.1 8086:2c19
>>>> | +-03.2 8086:2c1a
>>>> | +-03.4 8086:2c1c
>>>> | +-04.0 8086:2c20
>>>> | +-04.1 8086:2c21
>>>> | +-04.2 8086:2c22
>>>> | +-04.3 8086:2c23
>>>> | +-05.0 8086:2c28
>>>> | +-05.1 8086:2c29
>>>> | +-05.2 8086:2c2a
>>>> | +-05.3 8086:2c2b
>>>> | +-06.0 8086:2c30
>>>> | +-06.1 8086:2c31
>>>> | +-06.2 8086:2c32
>>>> | \-06.3 8086:2c33
>>>> \-[0000:00]-+-00.0 8086:3406
>>>> +-01.0-[01]--+-00.0 8086:10c9
>>>> | \-00.1 8086:10c9
>>>> +-03.0-[02]--
>>>> +-05.0-[03]--
>>>> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
>>>> +-09.0-[06]----00.0 8086:10b9
>>>> +-13.0 8086:342d
>>>> +-14.0 8086:342e
>>>> +-14.1 8086:3422
>>>> +-14.2 8086:3423
>>>> +-14.3 8086:3438
>>>> +-16.0 8086:3430
>>>> +-16.1 8086:3431
>>>> +-16.2 8086:3432
>>>> +-16.3 8086:3433
>>>> +-16.4 8086:3429
>>>> +-16.5 8086:342a
>>>> +-16.6 8086:342b
>>>> +-16.7 8086:342c
>>>> +-1a.0 8086:3a37
>>>> +-1a.1 8086:3a38
>>>> +-1a.2 8086:3a39
>>>> +-1a.7 8086:3a3c
>>>> +-1d.0 8086:3a34
>>>> +-1d.1 8086:3a35
>>>> +-1d.2 8086:3a36
>>>> +-1d.7 8086:3a3a
>>>> +-1e.0-[07]----01.0 102b:0532
>>>> +-1f.0 8086:3a16
>>>> +-1f.2 8086:3a22
>>>> \-1f.3 8086:3a30
>>>>
>>>> If someone can craft the correct patch that'd be great or answer the
>>>> above question and I'll gladly craft it.
>>>>
>>>> Thanks.
>>>
>> because I didn't see it. Here was the patch that got it working for me
>> (ignore the printks), applies against 3.6.6 and 3.7-rc5.
>
> I think you're on the right track, but the solution is too specific.
> Here's a version that will fall back to the bridge device for the base
> of the group. There may be opportunities to get rid of the pci_get_
> call altogether, but this seems pretty safe. Can you please test it?
> Thanks,
>
> Alex
>
going through the logic, I don't see why the pci_get_domain_bus_and_slot()
is even called. once there is a !NULL return for bridge, then
it should just do the pci_dev_get(bridge).
- Don
>
> commit ca15170f05b140ab8c611db5cb7cb9c218ddc930
> Author: Alex Williamson<alex.williamson@redhat.com>
> Date: Tue Nov 13 08:34:08 2012 -0700
>
> intel-iommu: Fix lookup in add device
>
> We can't assume this device exists, fall back to the bridge itself.
>
> Signed-off-by: Alex Williamson<alex.williamson@redhat.com>
>
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index d4a4cd4..0badfa4 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -4108,7 +4108,7 @@ static void swap_pci_ref(struct pci_dev **from, struct pci_dev *to)
> static int intel_iommu_add_device(struct device *dev)
> {
> struct pci_dev *pdev = to_pci_dev(dev);
> - struct pci_dev *bridge, *dma_pdev;
> + struct pci_dev *bridge, *dma_pdev = NULL;
> struct iommu_group *group;
> int ret;
>
> @@ -4122,7 +4122,7 @@ static int intel_iommu_add_device(struct device *dev)
> dma_pdev = pci_get_domain_bus_and_slot(
> pci_domain_nr(pdev->bus),
> bridge->subordinate->number, 0);
> - else
> + if (!dma_pdev)
> dma_pdev = pci_dev_get(bridge);
> } else
> dma_pdev = pci_dev_get(pdev);
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-13 19:05 ` Don Dutile
@ 2012-11-13 19:10 ` Alex Williamson
2012-11-13 19:32 ` Matthew Thode
0 siblings, 1 reply; 11+ messages in thread
From: Alex Williamson @ 2012-11-13 19:10 UTC (permalink / raw)
To: Don Dutile
Cc: prometheanfire, Doug Goldstein, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
On Tue, 2012-11-13 at 14:05 -0500, Don Dutile wrote:
> On 11/13/2012 10:38 AM, Alex Williamson wrote:
> > On Mon, 2012-11-12 at 15:05 -0600, Matthew Thode wrote:
> >> On 11/12/2012 01:57 PM, Don Dutile wrote:
> >>> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
> >>>> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
> >>>> <prometheanfire@gentoo.org> wrote:
> >>>>> System boots with vt-d disabled in bios. Otherwise I get the errors in
> >>>>> the attached log. I can do whatever testing you need as this system is
> >>>>> not in production yet. gonna paste the important part here. Let me
> >>>>> know if you want anything else.
> >>>>>
> >>>>> Please CC me directly as I am not subscribed to the LKML.
> >>>>>
> >>>>>
> >>>>> Trying to unpack rootfs image as initramfs...
> >>>>> Freeing initrd memory: 5124k freed
> >>>>> IOMMU 0 0xfbffe000: using Queued invalidation
> >>>>> IOMMU: Setting RMRR:
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
> >>>>> 0xbf7fffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
> >>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
> >>>>> IOMMU: Prepare 0-16MiB unity mapping for LPC
> >>>>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
> >>>>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
> >>>>> BUG: unable to handle kernel NULL pointer dereference at
> >>>>> 000000000000003c
> >>>>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> >>>>> PGD 0
> >>>>> Oops: 0000 [#1] SMP
> >>>>> Modules linked in:
> >>>>> CPU 7
> >>>>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
> >>>>> Relion 1751/X8DTU
> >>>>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
> >>>>> pci_get_dma_source+0xf/0x41
> >>>>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
> >>>>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
> >>>>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
> >>>>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
> >>>>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
> >>>>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
> >>>>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000)
> >>>>> knlGS:0000000000000000
> >>>>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> >>>>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
> >>>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >>>>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> >>>>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
> >>>>> ffff8806264cf910)
> >>>>> Stack:
> >>>>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
> >>>>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
> >>>>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
> >>>>> Call Trace:
> >>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> >>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> >>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> >>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> >>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> >>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> >>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> >>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> >>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> >>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> >>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> >>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> >>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> >>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> >>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>>>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
> >>>>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
> >>>>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
> >>>>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
> >>>>> RSP<ffff8806264d1d88>
> >>>>> CR2: 000000000000003c
> >>>>> ---[ end trace 5c5a2ceca067e0ec ]---
> >>>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
> >>>>>
> >>>>> ------------[ cut here ]------------
> >>>>> WARNING: at arch/x86/kernel/smp.c:123
> >>>>> native_smp_send_reschedule+0x25/0x51()
> >>>>> Hardware name: Relion 1751
> >>>>> Modules linked in:
> >>>>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
> >>>>> Call Trace:
> >>>>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
> >>>>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
> >>>>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
> >>>>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
> >>>>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
> >>>>> [<ffffffff810a132f>] update_process_times+0x62/0x73
> >>>>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
> >>>>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
> >>>>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
> >>>>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
> >>>>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
> >>>>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
> >>>>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
> >>>>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
> >>>>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
> >>>>> [<ffffffff810371b8>] oops_end+0xb2/0xba
> >>>>> [<ffffffff8105841d>] no_context+0x266/0x275
> >>>>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
> >>>>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
> >>>>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
> >>>>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
> >>>>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
> >>>>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
> >>>>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
> >>>>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
> >>>>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
> >>>>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
> >>>>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
> >>>>> [<ffffffff816f6232>] page_fault+0x22/0x30
> >>>>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
> >>>>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
> >>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
> >>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
> >>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
> >>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
> >>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
> >>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
> >>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
> >>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
> >>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
> >>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
> >>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
> >>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
> >>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
> >>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
> >>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
> >>>>> ---[ end trace 5c5a2ceca067e0ed ]---
> >>>>>
> >>>>> --
> >>>>> -- Matthew Thode (prometheanfire)
> >>>>
> >>>> The root cause of Matt's issue is that intel_iommu_add_device() calls
> >>>> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
> >>>> expected value. The reason NULL is being returned is that Matt has a
> >>>> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
> >>>> 8231) on it. This device already has a quirk setup for disabling fast
> >>>> back to back transfers on its secondary bus. If we cause it to use the
> >>>> primary bus, that appears to resolve the issue. I'm not sure exactly
> >>>> how to proceed from here due to relative lack of knowledge of PCI. Do
> >>>> all PCIe-PCI bridges with secondary buses need their DMA parent to be
> >>>> the primary bus or is that just something that should be done for the
> >>>> TI XIO2000A due to the existing quirk?
> >>>>
> >>> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
> >>> so the source of the device generating the DMA is unknown. When bridging
> >>> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the
> >>> source
> >>> of a transaction -- in this case, DMA read/write transaction.
> >>> This (sw) mapping should have happened by default, unless a recent
> >>> change from VFIO
> >>> broke this mapping.... or the TI bridge didn't report itself correctly
> >>> as a PCIe-PCI bridge.
> >>> Alex ?
> >>>
> >>>
> >>>> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
> >>>> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
> >>>> that didn't panic and a device that worked.
> >>>>
> >>>> $ lspci -tvn
> >>>> -+-[0000:ff]-+-00.0 8086:2c40
> >>>> | +-00.1 8086:2c01
> >>>> | +-02.0 8086:2c10
> >>>> | +-02.1 8086:2c11
> >>>> | +-02.4 8086:2c14
> >>>> | +-02.5 8086:2c15
> >>>> | +-03.0 8086:2c18
> >>>> | +-03.1 8086:2c19
> >>>> | +-03.2 8086:2c1a
> >>>> | +-03.4 8086:2c1c
> >>>> | +-04.0 8086:2c20
> >>>> | +-04.1 8086:2c21
> >>>> | +-04.2 8086:2c22
> >>>> | +-04.3 8086:2c23
> >>>> | +-05.0 8086:2c28
> >>>> | +-05.1 8086:2c29
> >>>> | +-05.2 8086:2c2a
> >>>> | +-05.3 8086:2c2b
> >>>> | +-06.0 8086:2c30
> >>>> | +-06.1 8086:2c31
> >>>> | +-06.2 8086:2c32
> >>>> | \-06.3 8086:2c33
> >>>> \-[0000:00]-+-00.0 8086:3406
> >>>> +-01.0-[01]--+-00.0 8086:10c9
> >>>> | \-00.1 8086:10c9
> >>>> +-03.0-[02]--
> >>>> +-05.0-[03]--
> >>>> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
> >>>> +-09.0-[06]----00.0 8086:10b9
> >>>> +-13.0 8086:342d
> >>>> +-14.0 8086:342e
> >>>> +-14.1 8086:3422
> >>>> +-14.2 8086:3423
> >>>> +-14.3 8086:3438
> >>>> +-16.0 8086:3430
> >>>> +-16.1 8086:3431
> >>>> +-16.2 8086:3432
> >>>> +-16.3 8086:3433
> >>>> +-16.4 8086:3429
> >>>> +-16.5 8086:342a
> >>>> +-16.6 8086:342b
> >>>> +-16.7 8086:342c
> >>>> +-1a.0 8086:3a37
> >>>> +-1a.1 8086:3a38
> >>>> +-1a.2 8086:3a39
> >>>> +-1a.7 8086:3a3c
> >>>> +-1d.0 8086:3a34
> >>>> +-1d.1 8086:3a35
> >>>> +-1d.2 8086:3a36
> >>>> +-1d.7 8086:3a3a
> >>>> +-1e.0-[07]----01.0 102b:0532
> >>>> +-1f.0 8086:3a16
> >>>> +-1f.2 8086:3a22
> >>>> \-1f.3 8086:3a30
> >>>>
> >>>> If someone can craft the correct patch that'd be great or answer the
> >>>> above question and I'll gladly craft it.
> >>>>
> >>>> Thanks.
> >>>
> >> because I didn't see it. Here was the patch that got it working for me
> >> (ignore the printks), applies against 3.6.6 and 3.7-rc5.
> >
> > I think you're on the right track, but the solution is too specific.
> > Here's a version that will fall back to the bridge device for the base
> > of the group. There may be opportunities to get rid of the pci_get_
> > call altogether, but this seems pretty safe. Can you please test it?
> > Thanks,
> >
> > Alex
> >
> going through the logic, I don't see why the pci_get_domain_bus_and_slot()
> is even called. once there is a !NULL return for bridge, then
> it should just do the pci_dev_get(bridge).
I agree, if we were earlier in the 3.7 cycle I think I'd drop it
altogether, but I'm nervous that we're forgetting something and opted to
only fix the clearly broken path. I can queue a patch for 3.8 that does
the remaining cleanup. Thanks,
Alex
> > commit ca15170f05b140ab8c611db5cb7cb9c218ddc930
> > Author: Alex Williamson<alex.williamson@redhat.com>
> > Date: Tue Nov 13 08:34:08 2012 -0700
> >
> > intel-iommu: Fix lookup in add device
> >
> > We can't assume this device exists, fall back to the bridge itself.
> >
> > Signed-off-by: Alex Williamson<alex.williamson@redhat.com>
> >
> > diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> > index d4a4cd4..0badfa4 100644
> > --- a/drivers/iommu/intel-iommu.c
> > +++ b/drivers/iommu/intel-iommu.c
> > @@ -4108,7 +4108,7 @@ static void swap_pci_ref(struct pci_dev **from, struct pci_dev *to)
> > static int intel_iommu_add_device(struct device *dev)
> > {
> > struct pci_dev *pdev = to_pci_dev(dev);
> > - struct pci_dev *bridge, *dma_pdev;
> > + struct pci_dev *bridge, *dma_pdev = NULL;
> > struct iommu_group *group;
> > int ret;
> >
> > @@ -4122,7 +4122,7 @@ static int intel_iommu_add_device(struct device *dev)
> > dma_pdev = pci_get_domain_bus_and_slot(
> > pci_domain_nr(pdev->bus),
> > bridge->subordinate->number, 0);
> > - else
> > + if (!dma_pdev)
> > dma_pdev = pci_dev_get(bridge);
> > } else
> > dma_pdev = pci_dev_get(pdev);
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu
2012-11-13 19:10 ` Alex Williamson
@ 2012-11-13 19:32 ` Matthew Thode
0 siblings, 0 replies; 11+ messages in thread
From: Matthew Thode @ 2012-11-13 19:32 UTC (permalink / raw)
To: Alex Williamson
Cc: Don Dutile, Doug Goldstein, linux-kernel, bhelgaas, linux-pci,
mthode, iommu
[-- Attachment #1: Type: text/plain, Size: 15570 bytes --]
On 11/13/2012 01:10 PM, Alex Williamson wrote:
> On Tue, 2012-11-13 at 14:05 -0500, Don Dutile wrote:
>> On 11/13/2012 10:38 AM, Alex Williamson wrote:
>>> On Mon, 2012-11-12 at 15:05 -0600, Matthew Thode wrote:
>>>> On 11/12/2012 01:57 PM, Don Dutile wrote:
>>>>> On 11/12/2012 04:26 AM, Doug Goldstein wrote:
>>>>>> On Sun, Nov 11, 2012 at 5:19 PM, Matthew Thode
>>>>>> <prometheanfire@gentoo.org> wrote:
>>>>>>> System boots with vt-d disabled in bios. Otherwise I get the errors in
>>>>>>> the attached log. I can do whatever testing you need as this system is
>>>>>>> not in production yet. gonna paste the important part here. Let me
>>>>>>> know if you want anything else.
>>>>>>>
>>>>>>> Please CC me directly as I am not subscribed to the LKML.
>>>>>>>
>>>>>>>
>>>>>>> Trying to unpack rootfs image as initramfs...
>>>>>>> Freeing initrd memory: 5124k freed
>>>>>>> IOMMU 0 0xfbffe000: using Queued invalidation
>>>>>>> IOMMU: Setting RMRR:
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 -
>>>>>>> 0xbf7fffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xeffff]
>>>>>>> IOMMU: Prepare 0-16MiB unity mapping for LPC
>>>>>>> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
>>>>>>> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>>>>>>> BUG: unable to handle kernel NULL pointer dereference at
>>>>>>> 000000000000003c
>>>>>>> IP: [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>>>>> PGD 0
>>>>>>> Oops: 0000 [#1] SMP
>>>>>>> Modules linked in:
>>>>>>> CPU 7
>>>>>>> Pid: 1, comm: swapper/0 Not tainted 3.7.0-rc5 #1 Penguin Computing
>>>>>>> Relion 1751/X8DTU
>>>>>>> RIP: 0010:[<ffffffff813bd796>] [<ffffffff813bd796>]
>>>>>>> pci_get_dma_source+0xf/0x41
>>>>>>> RSP: 0000:ffff8806264d1d88 EFLAGS: 00010282
>>>>>>> RAX: ffffffff813bd3a8 RBX: ffff8806261d1000 RCX: 00000000e8221180
>>>>>>> RDX: ffffffff818624f0 RSI: ffff88062635b0c0 RDI: 0000000000000000
>>>>>>> RBP: ffff8806264d1d88 R08: ffff8806263d6000 R09: 00000000ffffffff
>>>>>>> R10: ffff8806264d1ca8 R11: 0000000000000005 R12: 0000000000000000
>>>>>>> R13: ffff8806261d1098 R14: 0000000000000000 R15: 0000000000000000
>>>>>>> FS: 0000000000000000(0000) GS:ffff88063f2e0000(0000)
>>>>>>> knlGS:0000000000000000
>>>>>>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>>>>>> CR2: 000000000000003c CR3: 0000000001c0b000 CR4: 00000000000007e0
>>>>>>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>>>>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>>>>>> Process swapper/0 (pid: 1, threadinfo ffff8806264d0000, task
>>>>>>> ffff8806264cf910)
>>>>>>> Stack:
>>>>>>> ffff8806264d1dc8 ffffffff815d02c9 0000000000000000 ffff880600000000
>>>>>>> ffff8806264d1dd8 ffffffff81c64b00 ffff8806261d1098 ffff8806264d1df8
>>>>>>> ffff8806264d1de8 ffffffff815cd5a4 ffffffff81c64b00 ffffffff815cd56a
>>>>>>> Call Trace:
>>>>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>>>> Code: ff c1 75 04 ff d0 eb 12 48 83 c2 10 48 8b 42 08 48 85 c0 75 d3 b8
>>>>>>> e7 ff ff ff c9 c3 55 48 c7 c2 f0 24 86 81 48 89 e5 eb 24 8b 0a<66> 3b
>>>>>>> 4f 3c 74 05 66 ff c1 75 13 66 8b 4a 02 66 3b 4f 3e 74 05
>>>>>>> RIP [<ffffffff813bd796>] pci_get_dma_source+0xf/0x41
>>>>>>> RSP<ffff8806264d1d88>
>>>>>>> CR2: 000000000000003c
>>>>>>> ---[ end trace 5c5a2ceca067e0ec ]---
>>>>>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
>>>>>>>
>>>>>>> ------------[ cut here ]------------
>>>>>>> WARNING: at arch/x86/kernel/smp.c:123
>>>>>>> native_smp_send_reschedule+0x25/0x51()
>>>>>>> Hardware name: Relion 1751
>>>>>>> Modules linked in:
>>>>>>> Pid: 1, comm: swapper/0 Tainted: G D 3.7.0-rc5 #1
>>>>>>> Call Trace:
>>>>>>> <IRQ> [<ffffffff810968ee>] warn_slowpath_common+0x80/0x98
>>>>>>> [<ffffffff8109691b>] warn_slowpath_null+0x15/0x17
>>>>>>> [<ffffffff8104e1a3>] native_smp_send_reschedule+0x25/0x51
>>>>>>> [<ffffffff810bc81b>] trigger_load_balance+0x1e8/0x214
>>>>>>> [<ffffffff810b731f>] scheduler_tick+0xd8/0xe1
>>>>>>> [<ffffffff810a132f>] update_process_times+0x62/0x73
>>>>>>> [<ffffffff810cb78b>] tick_sched_timer+0x7c/0x9b
>>>>>>> [<ffffffff810b0f83>] __run_hrtimer.clone.24+0x4e/0xc1
>>>>>>> [<ffffffff810b15b0>] hrtimer_interrupt+0xc7/0x1ac
>>>>>>> [<ffffffff8104ef01>] smp_apic_timer_interrupt+0x81/0x94
>>>>>>> [<ffffffff816f71ca>] apic_timer_interrupt+0x6a/0x70
>>>>>>> <EOI> [<ffffffff81097ffc>] ? console_unlock+0x2c2/0x2ed
>>>>>>> [<ffffffff816f32fc>] ? panic+0x189/0x1c5
>>>>>>> [<ffffffff816f3261>] ? panic+0xee/0x1c5
>>>>>>> [<ffffffff8109ab6b>] do_exit+0x357/0x7b2
>>>>>>> [<ffffffff810371b8>] oops_end+0xb2/0xba
>>>>>>> [<ffffffff8105841d>] no_context+0x266/0x275
>>>>>>> [<ffffffff810585e7>] __bad_area_nosemaphore+0x1bb/0x1db
>>>>>>> [<ffffffff8118de46>] ? sysfs_addrm_finish+0x2f/0xa6
>>>>>>> [<ffffffff81058615>] bad_area_nosemaphore+0xe/0x10
>>>>>>> [<ffffffff81058bdb>] __do_page_fault+0x360/0x39f
>>>>>>> [<ffffffff81394afa>] ? ida_get_new_above+0xf9/0x19e
>>>>>>> [<ffffffff8112a077>] ? slab_node+0x59/0xa2
>>>>>>> [<ffffffff816f3ffd>] ? mutex_unlock+0x9/0xb
>>>>>>> [<ffffffff816da653>] ? klist_put+0x4c/0x70
>>>>>>> [<ffffffff816da581>] ? klist_next+0x30/0xb6
>>>>>>> [<ffffffff813b8cf9>] ? pci_do_find_bus+0x49/0x49
>>>>>>> [<ffffffff81058c42>] do_page_fault+0x9/0xb
>>>>>>> [<ffffffff816f6232>] page_fault+0x22/0x30
>>>>>>> [<ffffffff813bd3a8>] ? nv_msi_ht_cap_quirk_all+0x10/0x10
>>>>>>> [<ffffffff813bd796>] ? pci_get_dma_source+0xf/0x41
>>>>>>> [<ffffffff815d02c9>] intel_iommu_add_device+0x95/0x167
>>>>>>> [<ffffffff815cd5a4>] add_iommu_group+0x3a/0x41
>>>>>>> [<ffffffff815cd56a>] ? bus_set_iommu+0x44/0x44
>>>>>>> [<ffffffff8145eca1>] bus_for_each_dev+0x54/0x81
>>>>>>> [<ffffffff815cd563>] bus_set_iommu+0x3d/0x44
>>>>>>> [<ffffffff81cd3fa3>] intel_iommu_init+0xae5/0xb5e
>>>>>>> [<ffffffff81ca0277>] ? free_initrd+0x9e/0x9e
>>>>>>> [<ffffffff81ca4248>] ? memblock_find_dma_reserve+0x13f/0x13f
>>>>>>> [<ffffffff81ca425e>] pci_iommu_init+0x16/0x41
>>>>>>> [<ffffffff81cc4140>] ? pci_proc_init+0x6b/0x6b
>>>>>>> [<ffffffff81000231>] do_one_initcall+0x7a/0x129
>>>>>>> [<ffffffff816dac14>] kernel_init+0x139/0x2a2
>>>>>>> [<ffffffff81c9d4c7>] ? loglevel+0x31/0x31
>>>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>>>> [<ffffffff816f66ac>] ret_from_fork+0x7c/0xb0
>>>>>>> [<ffffffff816daadb>] ? rest_init+0x6f/0x6f
>>>>>>> ---[ end trace 5c5a2ceca067e0ed ]---
>>>>>>>
>>>>>>> --
>>>>>>> -- Matthew Thode (prometheanfire)
>>>>>>
>>>>>> The root cause of Matt's issue is that intel_iommu_add_device() calls
>>>>>> pci_get_domain_bus_and_slot() which is returning NULL. Which is not an
>>>>>> expected value. The reason NULL is being returned is that Matt has a
>>>>>> card with a TI XIO2000A/XIO2200A PCIe-PCI bridge (VID: 104C, DID:
>>>>>> 8231) on it. This device already has a quirk setup for disabling fast
>>>>>> back to back transfers on its secondary bus. If we cause it to use the
>>>>>> primary bus, that appears to resolve the issue. I'm not sure exactly
>>>>>> how to proceed from here due to relative lack of knowledge of PCI. Do
>>>>>> all PCIe-PCI bridges with secondary buses need their DMA parent to be
>>>>>> the primary bus or is that just something that should be done for the
>>>>>> TI XIO2000A due to the existing quirk?
>>>>>>
>>>>> DMA from a (legacy) PCI device does not have a SRC-ID in the transaction,
>>>>> so the source of the device generating the DMA is unknown. When bridging
>>>>> to a PCIe device, the Parent PPB's dev-id is inserted on the PCIe as the
>>>>> source
>>>>> of a transaction -- in this case, DMA read/write transaction.
>>>>> This (sw) mapping should have happened by default, unless a recent
>>>>> change from VFIO
>>>>> broke this mapping.... or the TI bridge didn't report itself correctly
>>>>> as a PCIe-PCI bridge.
>>>>> Alex ?
>>>>>
>>>>>
>>>>>> The failing call with arguments was pci_get_domain_bus_and_slot(0, 5,
>>>>>> 0), while pci_get_domain_bus_and_slot(0, 4, 0) resulted in a system
>>>>>> that didn't panic and a device that worked.
>>>>>>
>>>>>> $ lspci -tvn
>>>>>> -+-[0000:ff]-+-00.0 8086:2c40
>>>>>> | +-00.1 8086:2c01
>>>>>> | +-02.0 8086:2c10
>>>>>> | +-02.1 8086:2c11
>>>>>> | +-02.4 8086:2c14
>>>>>> | +-02.5 8086:2c15
>>>>>> | +-03.0 8086:2c18
>>>>>> | +-03.1 8086:2c19
>>>>>> | +-03.2 8086:2c1a
>>>>>> | +-03.4 8086:2c1c
>>>>>> | +-04.0 8086:2c20
>>>>>> | +-04.1 8086:2c21
>>>>>> | +-04.2 8086:2c22
>>>>>> | +-04.3 8086:2c23
>>>>>> | +-05.0 8086:2c28
>>>>>> | +-05.1 8086:2c29
>>>>>> | +-05.2 8086:2c2a
>>>>>> | +-05.3 8086:2c2b
>>>>>> | +-06.0 8086:2c30
>>>>>> | +-06.1 8086:2c31
>>>>>> | +-06.2 8086:2c32
>>>>>> | \-06.3 8086:2c33
>>>>>> \-[0000:00]-+-00.0 8086:3406
>>>>>> +-01.0-[01]--+-00.0 8086:10c9
>>>>>> | \-00.1 8086:10c9
>>>>>> +-03.0-[02]--
>>>>>> +-05.0-[03]--
>>>>>> +-07.0-[04-05]----00.0-[05]----08.0 d161:8006
>>>>>> +-09.0-[06]----00.0 8086:10b9
>>>>>> +-13.0 8086:342d
>>>>>> +-14.0 8086:342e
>>>>>> +-14.1 8086:3422
>>>>>> +-14.2 8086:3423
>>>>>> +-14.3 8086:3438
>>>>>> +-16.0 8086:3430
>>>>>> +-16.1 8086:3431
>>>>>> +-16.2 8086:3432
>>>>>> +-16.3 8086:3433
>>>>>> +-16.4 8086:3429
>>>>>> +-16.5 8086:342a
>>>>>> +-16.6 8086:342b
>>>>>> +-16.7 8086:342c
>>>>>> +-1a.0 8086:3a37
>>>>>> +-1a.1 8086:3a38
>>>>>> +-1a.2 8086:3a39
>>>>>> +-1a.7 8086:3a3c
>>>>>> +-1d.0 8086:3a34
>>>>>> +-1d.1 8086:3a35
>>>>>> +-1d.2 8086:3a36
>>>>>> +-1d.7 8086:3a3a
>>>>>> +-1e.0-[07]----01.0 102b:0532
>>>>>> +-1f.0 8086:3a16
>>>>>> +-1f.2 8086:3a22
>>>>>> \-1f.3 8086:3a30
>>>>>>
>>>>>> If someone can craft the correct patch that'd be great or answer the
>>>>>> above question and I'll gladly craft it.
>>>>>>
>>>>>> Thanks.
>>>>>
>>>> because I didn't see it. Here was the patch that got it working for me
>>>> (ignore the printks), applies against 3.6.6 and 3.7-rc5.
>>>
>>> I think you're on the right track, but the solution is too specific.
>>> Here's a version that will fall back to the bridge device for the base
>>> of the group. There may be opportunities to get rid of the pci_get_
>>> call altogether, but this seems pretty safe. Can you please test it?
>>> Thanks,
>>>
>>> Alex
>>>
>> going through the logic, I don't see why the pci_get_domain_bus_and_slot()
>> is even called. once there is a !NULL return for bridge, then
>> it should just do the pci_dev_get(bridge).
>
> I agree, if we were earlier in the 3.7 cycle I think I'd drop it
> altogether, but I'm nervous that we're forgetting something and opted to
> only fix the clearly broken path. I can queue a patch for 3.8 that does
> the remaining cleanup. Thanks,
>
> Alex
>
Sounds good, I'll keep patching til 3.8 then, thanks guys :D
Matthew Thode
>>> commit ca15170f05b140ab8c611db5cb7cb9c218ddc930
>>> Author: Alex Williamson<alex.williamson@redhat.com>
>>> Date: Tue Nov 13 08:34:08 2012 -0700
>>>
>>> intel-iommu: Fix lookup in add device
>>>
>>> We can't assume this device exists, fall back to the bridge itself.
>>>
>>> Signed-off-by: Alex Williamson<alex.williamson@redhat.com>
>>>
>>> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
>>> index d4a4cd4..0badfa4 100644
>>> --- a/drivers/iommu/intel-iommu.c
>>> +++ b/drivers/iommu/intel-iommu.c
>>> @@ -4108,7 +4108,7 @@ static void swap_pci_ref(struct pci_dev **from, struct pci_dev *to)
>>> static int intel_iommu_add_device(struct device *dev)
>>> {
>>> struct pci_dev *pdev = to_pci_dev(dev);
>>> - struct pci_dev *bridge, *dma_pdev;
>>> + struct pci_dev *bridge, *dma_pdev = NULL;
>>> struct iommu_group *group;
>>> int ret;
>>>
>>> @@ -4122,7 +4122,7 @@ static int intel_iommu_add_device(struct device *dev)
>>> dma_pdev = pci_get_domain_bus_and_slot(
>>> pci_domain_nr(pdev->bus),
>>> bridge->subordinate->number, 0);
>>> - else
>>> + if (!dma_pdev)
>>> dma_pdev = pci_dev_get(bridge);
>>> } else
>>> dma_pdev = pci_dev_get(pdev);
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
>
>
--
-- Matthew Thode (prometheanfire)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2012-11-13 19:32 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-11-11 23:19 [BUG 3.7-rc5] NULL pointer deref when using a pcie-pci bridged pci device and intel-iommu Matthew Thode
2012-11-12 9:26 ` Doug Goldstein
2012-11-12 19:57 ` Don Dutile
2012-11-12 21:05 ` Matthew Thode
2012-11-13 15:38 ` Alex Williamson
2012-11-13 15:50 ` Matthew Thode
2012-11-13 19:05 ` Don Dutile
2012-11-13 19:10 ` Alex Williamson
2012-11-13 19:32 ` Matthew Thode
2012-11-12 22:19 ` Alex Williamson
2012-11-12 22:25 ` Matthew Thode
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).