netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Bruno Prémont" <bonbons@linux-vserver.org>
To: "Benjamin Li" <benli@broadcom.com>
Cc: NetDEV <netdev@vger.kernel.org>,
	"Michael Chan" <mchan@broadcom.com>,
	Linux-Kernel <linux-kernel@vger.kernel.org>
Subject: Re: BNX2: Kernel crashes with 2.6.31 and 2.6.31.9
Date: Fri, 19 Feb 2010 09:10:34 +0100	[thread overview]
Message-ID: <20100219091034.5fbb0165@pluto.restena.lu> (raw)
In-Reply-To: <1262149691.2788.63.camel@localhost>

[-- Attachment #1: Type: text/plain, Size: 6812 bytes --]

Hi Benjamin,

On Tue, 29 Dec 2009 21:08:11 "Benjamin Li" wrote:
> Could you try running with the attached patch?  This debug patch is
> built against the linux-2.6.31.9 kernel.  I think the panic is
> occuring right before a reset has occured due to a TX timeout.  To
> see if this is happening, this patch will print hardware state
> information when a TX timeout occurs.  If you could run with this
> patch and send the logs when the panic occurs, I would really
> appreciate it.
> 
> Thanks again.
> 
> -Ben

Sorry for replying only this late but I've been too busy with other
things.

Anyhow, I've been doing some more testing yesterday and today and now
am able to reproduce the/a crash pretty easily.

Either running netconsole and doing 'echo t > /proc/sysrq-trigger' via
SSH on otherwise idle server (from local console nothing bad happens),
but then I have no means to communicate with the kernel (I guess it's
deadlocked somewhere in printk code)

The slightly less easy way to trigger it is with a dummy module that
kind of simulates netconsole behavior but with dummy data (see
attached). I have to have some more traffic (TCP?) going on for the bug
to trigger and tell my module multiple times to push data. This way
server is still accessible via VGA or serial console.

Attached are my 'netbomb.c' (which is a modified netconsole.c) and
full kernel log. This time running a 2.6.33-rc8-git3 kernel, having
forward-ported your patch above (e.g. half of it was already present)


I this time I got the following trace:
[  134.643292] BUG: unable to handle kernel NULL pointer dereference at (null)
[  134.643304] IP: [<ffffffffa003edc2>] bnx2_poll_work+0x32/0x13d0 [bnx2]
[  134.643314] PGD 2a972a067 PUD 2aa245067 PMD 0
[  134.643319] Oops: 0000 [#1] SMP
[  134.643323] last sysfs file: /sys/devices/pci0000:00/0000:00:1e.0/0000:01:04.6/class
[  134.643328] CPU 4
[  134.643334] Pid: 3226, comm: cat Not tainted 2.6.33-rc8-git3-x86_64 #3 /ProLiant DL360 G5
[  134.643339] RIP: 0010:[<ffffffffa003edc2>]  [<ffffffffa003edc2>] bnx2_poll_work+0x32/0x13d0 [bnx2]
[  134.643347] RSP: 0018:ffff8802a9643b38  EFLAGS: 00010092
[  134.643351] RAX: 0000000000000000 RBX: ffff8802afab57c0 RCX: 0000000000000010
[  134.643355] RDX: 0000000000000000 RSI: ffff8802afab57c0 RDI: ffff8802afab4580
[  134.643359] RBP: ffff8802a9643cd8 R08: ffff8802af051000 R09: 0000000000000007
[  134.643363] R10: 000000000000000e R11: 0000000000000000 R12: 0000000000000000
[  134.643367] R13: 0000000000000010 R14: 0000000000000000 R15: ffff8802afab4580
[  134.643371] FS:  0000000000000000(0000) GS:ffff880028300000(0063) knlGS:00000000f765f6c0
[  134.643376] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[  134.643380] CR2: 0000000000000000 CR3: 00000002a9606000 CR4: 00000000000006e0
[  134.643384] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  134.643388] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  134.643392] Process cat (pid: 3226, threadinfo ffff8802a9642000, task ffff8802aa14bff0)
[  134.643396] Stack:
[  134.643398]  0000000000000070 0000000000000002 0000000000000010 ffff8802afab57c0
[  134.643404] <0> ffff8802afab4580 0000000300000002 0000000000000000 0000000100000002
[  134.643410] <0> 0000000000000000 0000000200025220 ffffffff81862e80 0000000000000001
[  134.643418] Call Trace:
[  134.643427]  [<ffffffff8107e3fe>] ? __alloc_pages_nodemask+0xfe/0x660
[  134.643433]  [<ffffffff811b6de6>] ? msi_set_mask_bit+0x26/0xc0
[  134.643438]  [<ffffffff811b6e8b>] ? unmask_msi_irq+0xb/0x10
[  134.643443]  [<ffffffff8106db54>] ? default_enable+0x24/0x40
[  134.643448]  [<ffffffff8106d9b6>] ? check_irq_resend+0x26/0x70
[  134.643453]  [<ffffffff8106cc23>] ? __enable_irq+0x73/0x80
[  134.643459]  [<ffffffffa004019e>] bnx2_poll_msix+0x3e/0xd0 [bnx2]
[  134.643465]  [<ffffffff8135bcd1>] netpoll_poll+0xe1/0x3c0
[  134.643470]  [<ffffffff8135c168>] netpoll_send_skb+0x118/0x210
[  134.643475]  [<ffffffff8135c45b>] netpoll_send_udp+0x1fb/0x210
[  134.643480]  [<ffffffffa00981c5>] write_msg+0x95/0xd0 [netbomb]
[  134.643485]  [<ffffffffa0098255>] netbomb_write+0x55/0xa4 [netbomb]
[  134.643492]  [<ffffffff810f6581>] proc_reg_write+0x71/0xb0
[  134.643498]  [<ffffffff810ab6cb>] vfs_write+0xcb/0x180
[  134.643503]  [<ffffffff810ab870>] sys_write+0x50/0x90
[  134.643509]  [<ffffffff8102a1a4>] sysenter_dispatch+0x7/0x2b
[  134.643513] Code: 56 41 55 41 54 53 48 81 ec 78 01 00 00 48 89 bd 80 fe ff ff 48 89 b5 78 fe ff ff 89 95 74 fe ff ff 89 8d 70 fe ff ff 48 8b 46 70 <0f> b7 10 31 c0 80 fa ff 0f 94 c0 01 c2 66 39 96 12 02 00 00 0f
[  134.643551] RIP  [<ffffffffa003edc2>] bnx2_poll_work+0x32/0x13d0 [bnx2]
[  134.643557]  RSP <ffff8802a9643b38>
[  134.643559] CR2: 0000000000000000
[  134.643563] ---[ end trace 48bdec67d6d7aadb ]---

Running objdump on kernel compile with debugging symbols this matches:
000000000000ad90 <bnx2_poll_work>:
        }
}

static int bnx2_poll_work(struct bnx2 *bp, struct bnx2_napi *bnapi,
                          int work_done, int budget)
{
    ad90:       55                      push   %rbp
    ad91:       48 89 e5                mov    %rsp,%rbp
    ad94:       41 57                   push   %r15
    ad96:       41 56                   push   %r14
    ad98:       41 55                   push   %r13
    ad9a:       41 54                   push   %r12
    ad9c:       53                      push   %rbx
    ad9d:       48 81 ec 78 01 00 00    sub    $0x178,%rsp
    ada4:       48 89 bd 80 fe ff ff    mov    %rdi,-0x180(%rbp)
    adab:       48 89 b5 78 fe ff ff    mov    %rsi,-0x188(%rbp)
    adb2:       89 95 74 fe ff ff       mov    %edx,-0x18c(%rbp)
    adb8:       89 8d 70 fe ff ff       mov    %ecx,-0x190(%rbp)
{
        u16 cons;

        /* Tell compiler that status block fields can change. */
        barrier();
        cons = *bnapi->hw_tx_cons_ptr;
    adbe:       48 8b 46 70             mov    0x70(%rsi),%rax
    adc2:       0f b7 10                movzwl (%rax),%edx
        barrier();
        if (unlikely((cons & MAX_TX_DESC_CNT) == MAX_TX_DESC_CNT))
                cons++;
    adc5:       31 c0                   xor    %eax,%eax
    adc7:       80 fa ff                cmp    $0xff,%dl
    adca:       0f 94 c0                sete   %al
    adcd:       01 c2                   add    %eax,%edx
                          int work_done, int budget)
{
        struct bnx2_tx_ring_info *txr = &bnapi->tx_ring;
        struct bnx2_rx_ring_info *rxr = &bnapi->rx_ring;

        if (bnx2_get_hw_tx_cons(bnapi) != txr->hw_tx_cons)
    adcf:       66 39 96 12 02 00 00    cmp    %dx,0x212(%rsi)
    add6:       0f 84 4f 03 00 00       je     b12b <bnx2_poll_work+0x39b>


So as already determined bnapi->hw_tx_cons_ptr is NULL... but nothing is
happening after that on network side.

Regards,
Bruno

[-- Attachment #2: bnx2.dmesg --]
[-- Type: application/octet-stream, Size: 56462 bytes --]

[    0.000000] Linux version 2.6.33-rc8-git3-x86_64 (bruno@pluto) (gcc version 4.3.4 (Gentoo 4.3.4 p1.0, pie-10.1.5) ) #3 SMP Fri Feb 19 08:31:57 CET 2010
[    0.000000] Command line: vga=normal root=/dev/cciss/c0d0p1 radeon.modeset=0 modeset=0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000100 - 000000000009f400 (usable)
[    0.000000]  BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000cfe58000 (usable)
[    0.000000]  BIOS-e820: 00000000cfe58000 - 00000000cfe60000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000cfe60000 - 00000000d0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fec00000 - 00000000fed00000 (reserved)
[    0.000000]  BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
[    0.000000]  BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 00000002affff000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.3 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x2affff max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: write-back
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 0D0000000 mask FF0000000 uncachable
[    0.000000]   1 base 0E0000000 mask FE0000000 uncachable
[    0.000000]   2 base 0D8000000 mask FF8000000 write-combining
[    0.000000]   3 disabled
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7010600070106, new 0x7010600070106
[    0.000000] last_pfn = 0xcfe58 max_arch_pfn = 0x400000000
[    0.000000] initial memory mapped : 0 - 20000000
[    0.000000] found SMP MP-table at [ffff8800000f4f80] f4f80
[    0.000000] init_memory_mapping: 0000000000000000-00000000cfe58000
[    0.000000]  0000000000 - 00cfe00000 page 2M
[    0.000000]  00cfe00000 - 00cfe58000 page 4k
[    0.000000] kernel direct mapping tables up to cfe58000 @ 8000-e000
[    0.000000] init_memory_mapping: 0000000100000000-00000002affff000
[    0.000000]  0100000000 - 02afe00000 page 2M
[    0.000000]  02afe00000 - 02affff000 page 4k
[    0.000000] kernel direct mapping tables up to 2affff000 @ c000-19000
[    0.000000] ACPI: RSDP 00000000000f4f00 00024 (v02 HP    )
[    0.000000] ACPI: XSDT 00000000cfe58300 00054 (v01 HP     P58      00000002   �? 0000162E)
[    0.000000] ACPI: FACP 00000000cfe58380 000F4 (v03 HP     P58      00000002   �? 0000162E)
[    0.000000] ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default 16 (20091214/tbfadt-607)
[    0.000000] ACPI: DSDT 00000000cfe58480 01E95 (v01 HP         DSDT 00000001 INTL 20030228)
[    0.000000] ACPI: FACS 00000000cfe580c0 00040
[    0.000000] ACPI: SPCR 00000000cfe58100 00050 (v01 HP     SPCRRBSU 00000001   �? 0000162E)
[    0.000000] ACPI: MCFG 00000000cfe58180 0003C (v01 HP     ProLiant 00000001      00000000)
[    0.000000] ACPI: HPET 00000000cfe581c0 00038 (v01 HP     P58      00000002   �? 0000162E)
[    0.000000] ACPI: SPMI 00000000cfe58200 00040 (v05 HP     ProLiant 00000001   �? 0000162E)
[    0.000000] ACPI: APIC 00000000cfe58240 0009E (v01 HP     00000083 00000002      00000000)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] (11 early reservations) ==> bootmem [0000000000 - 02affff000]
[    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
[    0.000000]   #1 [0001000000 - 0001932350]    TEXT DATA BSS ==> [0001000000 - 0001932350]
[    0.000000]   #2 [0001933000 - 0001933134]              BRK ==> [0001933000 - 0001933134]
[    0.000000]   #3 [00000f4f90 - 0000100000]    BIOS reserved ==> [00000f4f90 - 0000100000]
[    0.000000]   #4 [00000f4f80 - 00000f4f90]     MP-table mpf ==> [00000f4f80 - 00000f4f90]
[    0.000000]   #5 [000009f400 - 00000f1d10]    BIOS reserved ==> [000009f400 - 00000f1d10]
[    0.000000]   #6 [00000f1fcc - 00000f4f80]    BIOS reserved ==> [00000f1fcc - 00000f4f80]
[    0.000000]   #7 [00000f1d10 - 00000f1fcc]     MP-table mpc ==> [00000f1d10 - 00000f1fcc]
[    0.000000]   #8 [0000001000 - 0000003000]       TRAMPOLINE ==> [0000001000 - 0000003000]
[    0.000000]   #9 [0000008000 - 000000c000]          PGTABLE ==> [0000008000 - 000000c000]
[    0.000000]   #10 [000000c000 - 0000014000]          PGTABLE ==> [000000c000 - 0000014000]
[    0.000000]  [ffffea0000000000-ffffea00097fffff] PMD -> [ffff880028600000-ffff8800313fffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000001 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x002affff
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000001 -> 0x0000009f
[    0.000000]     0: 0x00000100 -> 0x000cfe58
[    0.000000]     0: 0x00100000 -> 0x002affff
[    0.000000] On node 0 totalpages: 2620917
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 110 pages reserved
[    0.000000]   DMA zone: 3832 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 833168 pages, LIFO batch:31
[    0.000000]   Normal zone: 24192 pages used for memmap
[    0.000000]   Normal zone: 1745279 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x908
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[24])
[    0.000000] IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 24-47
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 48
[    0.000000] Allocating PCI resources starting at d0000000 (gap: d0000000:2ec00000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff880028200000 s77912 r8192 d24488 u262144
[    0.000000] pcpu-alloc: s77912 r8192 d24488 u262144 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2582279
[    0.000000] Kernel command line: vga=normal vga=normal root=/dev/cciss/c0d0p1 radeon.modeset=0 modeset=0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.000000] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 10236192k/11272188k available (4108k kernel code, 788520k absent, 246548k reserved, 4509k data, 436k init)
[    0.000000] SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:512
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [tty0] enabled
[    0.000000] hpet clockevent registered
[    0.000000] Fast TSC calibration using PIT
[    0.000000] Detected 2333.158 MHz processor.
[    0.020006] Calibrating delay loop (skipped), value calculated using timer frequency.. 4666.31 BogoMIPS (lpj=23331580)
[    0.020245] Mount-cache hash table entries: 256
[    0.020496] CPU: Physical Processor ID: 0
[    0.020600] CPU: Processor Core ID: 0
[    0.020703] mce: CPU supports 6 MCE banks
[    0.020811] CPU0: Thermal LVT vector (0xfa) already installed
[    0.020814] using mwait in idle threads.
[    0.020918] Performance Events: Core2 events, Intel PMU driver.
[    0.021092] ... version:                2
[    0.021195] ... bit width:              40
[    0.021298] ... generic registers:      2
[    0.021402] ... value mask:             000000ffffffffff
[    0.021507] ... max period:             000000007fffffff
[    0.021612] ... fixed-purpose events:   3
[    0.021716] ... event mask:             0000000700000003
[    0.021831] Freeing SMP alternatives: 27k freed
[    0.021957] ACPI: Core revision 20091214
[    0.025695] Setting APIC routing to flat
[    0.026161] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.127027] CPU0: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz stepping 07
[    0.130000] Booting Node   0, Processors  #1
[    0.030000] CPU1: Thermal LVT vector (0xfa) already installed
[    0.290070]  #2
[    0.030000] CPU2: Thermal LVT vector (0xfa) already installed
[    0.460077]  #3
[    0.030000] CPU3: Thermal LVT vector (0xfa) already installed
[    0.630069]  #4
[    0.030000] CPU4: Thermal LVT vector (0xfa) already installed
[    0.800103]  #5
[    0.030000] CPU5: Thermal LVT vector (0xfa) already installed
[    0.970066]  #6
[    0.030000] CPU6: Thermal LVT vector (0xfa) already installed
[    1.140071]  #7 Ok.
[    0.030000] CPU7: Thermal LVT vector (0xfa) already installed
[    1.310032] Brought up 8 CPUs
[    1.310135] Total of 8 processors activated (37334.51 BogoMIPS).
[    1.313946] NET: Registered protocol family 16
[    1.313946] ACPI: bus type pci registered
[    1.313946] dca service started, version 1.12.1
[    1.313946] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    1.313946] PCI: not using MMCONFIG
[    1.313946] PCI: Using configuration type 1 for base access
[    1.313946] PCI: HP ProLiant DL360 detected, enabling pci=bfsort.
[    1.320035] bio: create slab <bio-0> at 0
[    1.320482] ACPI: EC: Look up EC in DSDT
[    1.322597] ACPI: Interpreter enabled
[    1.322701] ACPI: (supports S0 S5)
[    1.322875] ACPI: Using IOAPIC for interrupt routing
[    1.323010] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    1.323697] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
[    1.331712] ACPI: No dock devices found.
[    1.331712] ACPI: PCI Root Bridge [PCI0] (0000:00)
[    1.331712] pci_root PNP0A03:00: ignoring host bridge windows from ACPI; boot with "pci=use_crs" to use them
[    1.331712] pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7] (ignored)
[    1.331712] pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff] (ignored)
[    1.331712] pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff] (ignored)
[    1.331712] pci_root PNP0A03:00: host bridge window [mem 0xd0000000-0xdfffffff] (ignored)
[    1.331712] pci_root PNP0A03:00: host bridge window [mem 0xf0000000-0xfebfffff] (ignored)
[    1.331712] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:00.0: PME# disabled
[    1.331712] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:02.0: PME# disabled
[    1.331712] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:03.0: PME# disabled
[    1.331712] pci 0000:00:04.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:04.0: PME# disabled
[    1.331712] pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:05.0: PME# disabled
[    1.331712] pci 0000:00:06.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:06.0: PME# disabled
[    1.331712] pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:07.0: PME# disabled
[    1.331712] pci 0000:00:1d.0: reg 20: [io  0x1000-0x101f]
[    1.331712] pci 0000:00:1d.1: reg 20: [io  0x1020-0x103f]
[    1.331712] pci 0000:00:1d.2: reg 20: [io  0x1040-0x105f]
[    1.331712] pci 0000:00:1d.3: reg 20: [io  0x1060-0x107f]
[    1.331712] pci 0000:00:1d.7: reg 10: [mem 0xf7df0000-0xf7df03ff]
[    1.331712] pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
[    1.331712] pci 0000:00:1d.7: PME# disabled
[    1.331712] pci 0000:00:1f.1: reg 10: [io  0x01f0-0x01f7]
[    1.331712] pci 0000:00:1f.1: reg 14: [io  0x03f4-0x03f7]
[    1.331712] pci 0000:00:1f.1: reg 18: [io  0x0170-0x0177]
[    1.331712] pci 0000:00:1f.1: reg 1c: [io  0x0374-0x0377]
[    1.331712] pci 0000:00:1f.1: reg 20: [io  0x0500-0x050f]
[    1.333351] pci 0000:09:00.0: PME# supported from D0 D3hot D3cold
[    1.333442] pci 0000:09:00.0: PME# disabled
[    1.341461] pci 0000:09:00.3: PME# supported from D0 D3hot D3cold
[    1.341507] pci 0000:09:00.3: PME# disabled
[    1.341947] pci 0000:00:02.0: PCI bridge to [bus 09-12]
[    1.342054] pci 0000:00:02.0:   bridge window [io  0x5000-0x6fff]
[    1.342057] pci 0000:00:02.0:   bridge window [mem 0xfdd00000-0xfdffffff]
[    1.342062] pci 0000:00:02.0:   bridge window [mem 0xd0000000-0xd01fffff 64bit pref]
[    1.344661] pci 0000:0a:00.0: PME# supported from D0 D3hot D3cold
[    1.344752] pci 0000:0a:00.0: PME# disabled
[    1.347678] pci 0000:0a:01.0: PME# supported from D0 D3hot D3cold
[    1.347769] pci 0000:0a:01.0: PME# disabled
[    1.350685] pci 0000:0a:02.0: PME# supported from D0 D3hot D3cold
[    1.350776] pci 0000:0a:02.0: PME# disabled
[    1.352559] pci 0000:09:00.0: PCI bridge to [bus 0a-0f]
[    1.352742] pci 0000:09:00.0:   bridge window [io  0x5000-0x5fff]
[    1.352833] pci 0000:09:00.0:   bridge window [mem 0xfde00000-0xfdefffff]
[    1.353016] pci 0000:09:00.0:   bridge window [mem 0xd0000000-0xd00fffff 64bit pref]
[    1.353976] pci 0000:0b:00.0: reg 10: [mem 0xfdee0000-0xfdefffff]
[    1.354159] pci 0000:0b:00.0: reg 14: [mem 0xfdec0000-0xfdedffff]
[    1.354342] pci 0000:0b:00.0: reg 18: [io  0x5000-0x501f]
[    1.355073] pci 0000:0b:00.0: reg 30: [mem 0x00000000-0x0001ffff pref]
[    1.356033] pci 0000:0b:00.0: PME# supported from D0 D3hot D3cold
[    1.356124] pci 0000:0b:00.0: PME# disabled
[    1.357405] pci 0000:0b:00.1: reg 10: [mem 0xfdea0000-0xfdebffff]
[    1.357587] pci 0000:0b:00.1: reg 14: [mem 0xfde80000-0xfde9ffff]
[    1.357770] pci 0000:0b:00.1: reg 18: [io  0x5020-0x503f]
[    1.358502] pci 0000:0b:00.1: reg 30: [mem 0x00000000-0x0001ffff pref]
[    1.359461] pci 0000:0b:00.1: PME# supported from D0 D3hot D3cold
[    1.359553] pci 0000:0b:00.1: PME# disabled
[    1.361644] pci 0000:0a:00.0: PCI bridge to [bus 0b-0d]
[    1.361827] pci 0000:0a:00.0:   bridge window [io  0x5000-0x5fff]
[    1.361919] pci 0000:0a:00.0:   bridge window [mem 0xfde00000-0xfdefffff]
[    1.362101] pci 0000:0a:00.0:   bridge window [mem 0xd0000000-0xd00fffff 64bit pref]
[    1.363747] pci 0000:0a:01.0: PCI bridge to [bus 0e-0e]
[    1.365850] pci 0000:0a:02.0: PCI bridge to [bus 0f-0f]
[    1.368227] pci 0000:10:01.0: reg 10: [io  0x6000-0x60ff]
[    1.368593] pci 0000:10:01.0: reg 14: [mem 0xfdff0000-0xfdff0fff 64bit]
[    1.369324] pci 0000:10:01.0: reg 30: [mem 0x00000000-0x0003ffff pref]
[    1.372056] pci 0000:09:00.3: PCI bridge to [bus 10-12]
[    1.372194] pci 0000:09:00.3:   bridge window [io  0x6000-0x6fff]
[    1.372240] pci 0000:09:00.3:   bridge window [mem 0xfdf00000-0xfdffffff]
[    1.372331] pci 0000:09:00.3:   bridge window [mem 0xd0100000-0xd01fffff 64bit pref]
[    1.372677] pci 0000:06:00.0: reg 10: [mem 0xfdc00000-0xfdcfffff 64bit]
[    1.372683] pci 0000:06:00.0: reg 18: [io  0x4000-0x40ff]
[    1.372693] pci 0000:06:00.0: reg 1c: [mem 0xfdbf0000-0xfdbf0fff 64bit]
[    1.372704] pci 0000:06:00.0: reg 30: [mem 0x00000000-0x0003ffff pref]
[    1.372729] pci 0000:06:00.0: supports D1
[    1.372768] pci 0000:00:03.0: PCI bridge to [bus 06-08]
[    1.372875] pci 0000:00:03.0:   bridge window [io  0x4000-0x4fff]
[    1.372878] pci 0000:00:03.0:   bridge window [mem 0xfdb00000-0xfdcfffff]
[    1.372882] pci 0000:00:03.0:   bridge window [mem 0xd0200000-0xd02fffff 64bit pref]
[    1.372917] pci 0000:00:04.0: PCI bridge to [bus 13-15]
[    1.373061] pci 0000:00:05.0: PCI bridge to [bus 16-16]
[    1.373217] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    1.373221] pci 0000:02:00.0: PME# disabled
[    1.373259] pci 0000:00:06.0: PCI bridge to [bus 02-03]
[    1.373367] pci 0000:00:06.0:   bridge window [mem 0xf8000000-0xf9ffffff]
[    1.373372] pci 0000:00:06.0:   bridge window [mem 0xd0300000-0xd03fffff 64bit pref]
[    1.373405] pci 0000:03:00.0: reg 10: [mem 0xf8000000-0xf9ffffff 64bit]
[    1.373427] pci 0000:03:00.0: reg 30: [mem 0x00000000-0x0001ffff pref]
[    1.373450] pci 0000:03:00.0: PME# supported from D3hot D3cold
[    1.373453] pci 0000:03:00.0: PME# disabled
[    1.373494] pci 0000:02:00.0: PCI bridge to [bus 03-03]
[    1.373606] pci 0000:02:00.0:   bridge window [mem 0xf8000000-0xf9ffffff]
[    1.373612] pci 0000:02:00.0:   bridge window [mem 0xd0300000-0xd03fffff 64bit pref]
[    1.373666] pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
[    1.373669] pci 0000:04:00.0: PME# disabled
[    1.373708] pci 0000:00:07.0: PCI bridge to [bus 04-05]
[    1.373816] pci 0000:00:07.0:   bridge window [mem 0xfa000000-0xfbffffff]
[    1.373820] pci 0000:00:07.0:   bridge window [mem 0xd0400000-0xd04fffff 64bit pref]
[    1.373854] pci 0000:05:00.0: reg 10: [mem 0xfa000000-0xfbffffff 64bit]
[    1.373875] pci 0000:05:00.0: reg 30: [mem 0x00000000-0x0001ffff pref]
[    1.373898] pci 0000:05:00.0: PME# supported from D3hot D3cold
[    1.373901] pci 0000:05:00.0: PME# disabled
[    1.373942] pci 0000:04:00.0: PCI bridge to [bus 05-05]
[    1.374054] pci 0000:04:00.0:   bridge window [mem 0xfa000000-0xfbffffff]
[    1.374060] pci 0000:04:00.0:   bridge window [mem 0xd0400000-0xd04fffff 64bit pref]
[    1.374097] pci 0000:01:03.0: reg 10: [mem 0xd8000000-0xdfffffff pref]
[    1.374104] pci 0000:01:03.0: reg 14: [io  0x3000-0x30ff]
[    1.374111] pci 0000:01:03.0: reg 18: [mem 0xf7ff0000-0xf7ffffff]
[    1.374132] pci 0000:01:03.0: reg 30: [mem 0x00000000-0x0001ffff pref]
[    1.374151] pci 0000:01:03.0: supports D1 D2
[    1.374178] pci 0000:01:04.0: reg 10: [io  0x2800-0x28ff]
[    1.374185] pci 0000:01:04.0: reg 14: [mem 0xf7fe0000-0xf7fe01ff]
[    1.374226] pci 0000:01:04.0: PME# supported from D0 D3hot D3cold
[    1.374230] pci 0000:01:04.0: PME# disabled
[    1.374261] pci 0000:01:04.2: reg 10: [io  0x3400-0x34ff]
[    1.374269] pci 0000:01:04.2: reg 14: [mem 0xf7fd0000-0xf7fd07ff]
[    1.374276] pci 0000:01:04.2: reg 18: [mem 0xf7fc0000-0xf7fc3fff]
[    1.374284] pci 0000:01:04.2: reg 1c: [mem 0xf7f00000-0xf7f7ffff]
[    1.374301] pci 0000:01:04.2: reg 30: [mem 0x00000000-0x0000ffff pref]
[    1.374322] pci 0000:01:04.2: PME# supported from D0 D3hot D3cold
[    1.374326] pci 0000:01:04.2: PME# disabled
[    1.374379] pci 0000:01:04.4: reg 20: [io  0x3800-0x381f]
[    1.374410] pci 0000:01:04.4: PME# supported from D0 D3hot D3cold
[    1.374414] pci 0000:01:04.4: PME# disabled
[    1.374443] pci 0000:01:04.6: reg 10: [mem 0xf7ef0000-0xf7ef00ff]
[    1.374489] pci 0000:01:04.6: PME# supported from D0 D3hot D3cold
[    1.374493] pci 0000:01:04.6: PME# disabled
[    1.374534] pci 0000:00:1e.0: PCI bridge to [bus 01-01] (subtractive decode)
[    1.374645] pci 0000:00:1e.0:   bridge window [io  0x2000-0x3fff]
[    1.374648] pci 0000:00:1e.0:   bridge window [mem 0xf7e00000-0xf7ffffff]
[    1.374654] pci 0000:00:1e.0:   bridge window [mem 0xd8000000-0xdfffffff 64bit pref]
[    1.374674] pci_bus 0000:00: on NUMA node 0
[    1.374678] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    1.374752] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.IP2P._PRT]
[    1.374803] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT02._PRT]
[    1.374826] ACPI Warning for \_SB_.PCI0.PT02._PRT: Return Package has no elements (empty) (20091214/nspredef-455)
[    1.375077] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT02.IPE4._PRT]
[    1.375128] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT02.IPE4.IPE1._PRT]
[    1.375183] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT02.P2P2._PRT]
[    1.375235] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT03._PRT]
[    1.375286] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT04._PRT]
[    1.375343] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT06.NB01._PRT]
[    1.375399] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT07.NB02._PRT]
[    1.380057] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 7 10 11), disabled.
[    1.380461] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *7 10 11), disabled.
[    1.380862] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 7 *10 11), disabled.
[    1.381263] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 7 *10 11), disabled.
[    1.381661] ACPI: PCI Interrupt Link [LNKE] (IRQs 5 7 10 11) *0, disabled.
[    1.382093] ACPI: PCI Interrupt Link [LNKF] (IRQs *5 7 10 11), disabled.
[    1.382493] ACPI: PCI Interrupt Link [LNKG] (IRQs 5 7 *10 11), disabled.
[    1.382892] ACPI: PCI Interrupt Link [LNKH] (IRQs 5 *7 10 11), disabled.
[    1.383265] vgaarb: device added: PCI:0000:01:03.0,decodes=io+mem,owns=io+mem,locks=none
[    1.383265] vgaarb: loaded
[    1.383265] SCSI subsystem initialized
[    1.383265] usbcore: registered new interface driver usbfs
[    1.383265] usbcore: registered new interface driver hub
[    1.383265] usbcore: registered new device driver usb
[    1.383265] PCI: Using ACPI for IRQ routing
[    1.383265] PCI: pci_cache_line_size set to 64 bytes
[    1.383265] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    1.383265] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    1.383265] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
[    1.430006] Switching to clocksource tsc
[    1.430288] pnp: PnP ACPI init
[    1.430413] ACPI: bus type pnp registered
[    1.432259] pnp: PnP ACPI: found 10 devices
[    1.432365] ACPI: ACPI bus type pnp unregistered
[    1.432481] system 00:01: [io  0x0408-0x040f] has been reserved
[    1.432590] system 00:01: [io  0x04d0-0x04d1] has been reserved
[    1.432699] system 00:01: [io  0x0700-0x071f] has been reserved
[    1.432807] system 00:01: [io  0x0800-0x083f] has been reserved
[    1.432915] system 00:01: [io  0x0900-0x097f] has been reserved
[    1.433024] system 00:01: [io  0x0c80-0x0c83] has been reserved
[    1.433132] system 00:01: [io  0x0cd4-0x0cd7] has been reserved
[    1.433241] system 00:01: [io  0x0f50-0x0f58] has been reserved
[    1.433350] system 00:01: [io  0x0ca0-0x0ca1] has been reserved
[    1.433458] system 00:01: [io  0x0ca4-0x0ca5] has been reserved
[    1.433566] system 00:01: [io  0x02f8-0x02ff] has been reserved
[    1.433676] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
[    1.433786] system 00:01: [mem 0xfe000000-0xfebfffff] has been reserved
[    1.440340] pci 0000:0b:00.0: BAR 6: assigned [mem 0xd0000000-0xd001ffff pref]
[    1.440516] pci 0000:0b:00.1: BAR 6: assigned [mem 0xd0020000-0xd003ffff pref]
[    1.440691] pci 0000:0a:00.0: PCI bridge to [bus 0b-0d]
[    1.440798] pci 0000:0a:00.0:   bridge window [io  0x5000-0x5fff]
[    1.441027] pci 0000:0a:00.0:   bridge window [mem 0xfde00000-0xfdefffff]
[    1.441210] pci 0000:0a:00.0:   bridge window [mem 0xd0000000-0xd00fffff 64bit pref]
[    1.441529] pci 0000:0a:01.0: PCI bridge to [bus 0e-0e]
[    1.441636] pci 0000:0a:01.0:   bridge window [io  disabled]
[    1.441849] pci 0000:0a:01.0:   bridge window [mem disabled]
[    1.442037] pci 0000:0a:01.0:   bridge window [mem pref disabled]
[    1.442311] pci 0000:0a:02.0: PCI bridge to [bus 0f-0f]
[    1.442418] pci 0000:0a:02.0:   bridge window [io  disabled]
[    1.442631] pci 0000:0a:02.0:   bridge window [mem disabled]
[    1.442814] pci 0000:0a:02.0:   bridge window [mem pref disabled]
[    1.443088] pci 0000:09:00.0: PCI bridge to [bus 0a-0f]
[    1.443226] pci 0000:09:00.0:   bridge window [io  0x5000-0x5fff]
[    1.443454] pci 0000:09:00.0:   bridge window [mem 0xfde00000-0xfdefffff]
[    1.443637] pci 0000:09:00.0:   bridge window [mem 0xd0000000-0xd00fffff 64bit pref]
[    1.443958] pci 0000:10:01.0: BAR 6: assigned [mem 0xd0100000-0xd013ffff pref]
[    1.444132] pci 0000:09:00.3: PCI bridge to [bus 10-12]
[    1.444277] pci 0000:09:00.3:   bridge window [io  0x6000-0x6fff]
[    1.444505] pci 0000:09:00.3:   bridge window [mem 0xfdf00000-0xfdffffff]
[    1.444688] pci 0000:09:00.3:   bridge window [mem 0xd0100000-0xd01fffff 64bit pref]
[    1.445008] pci 0000:00:02.0: PCI bridge to [bus 09-12]
[    1.445115] pci 0000:00:02.0:   bridge window [io  0x5000-0x6fff]
[    1.445225] pci 0000:00:02.0:   bridge window [mem 0xfdd00000-0xfdffffff]
[    1.445336] pci 0000:00:02.0:   bridge window [mem 0xd0000000-0xd01fffff 64bit pref]
[    1.445514] pci 0000:06:00.0: BAR 6: assigned [mem 0xd0200000-0xd023ffff pref]
[    1.445689] pci 0000:00:03.0: PCI bridge to [bus 06-08]
[    1.445796] pci 0000:00:03.0:   bridge window [io  0x4000-0x4fff]
[    1.445905] pci 0000:00:03.0:   bridge window [mem 0xfdb00000-0xfdcfffff]
[    1.446016] pci 0000:00:03.0:   bridge window [mem 0xd0200000-0xd02fffff 64bit pref]
[    1.446194] pci 0000:00:04.0: PCI bridge to [bus 13-15]
[    1.446300] pci 0000:00:04.0:   bridge window [io  disabled]
[    1.446409] pci 0000:00:04.0:   bridge window [mem disabled]
[    1.446517] pci 0000:00:04.0:   bridge window [mem pref disabled]
[    1.446628] pci 0000:00:05.0: PCI bridge to [bus 16-16]
[    1.446734] pci 0000:00:05.0:   bridge window [io  disabled]
[    1.446843] pci 0000:00:05.0:   bridge window [mem disabled]
[    1.446951] pci 0000:00:05.0:   bridge window [mem pref disabled]
[    1.447063] pci 0000:03:00.0: BAR 6: assigned [mem 0xd0300000-0xd031ffff pref]
[    1.447237] pci 0000:02:00.0: PCI bridge to [bus 03-03]
[    1.447343] pci 0000:02:00.0:   bridge window [io  disabled]
[    1.447453] pci 0000:02:00.0:   bridge window [mem 0xf8000000-0xf9ffffff]
[    1.447564] pci 0000:02:00.0:   bridge window [mem 0xd0300000-0xd03fffff 64bit pref]
[    1.448070] pci 0000:00:06.0: PCI bridge to [bus 02-03]
[    1.448176] pci 0000:00:06.0:   bridge window [io  disabled]
[    1.448286] pci 0000:00:06.0:   bridge window [mem 0xf8000000-0xf9ffffff]
[    1.448396] pci 0000:00:06.0:   bridge window [mem 0xd0300000-0xd03fffff 64bit pref]
[    1.448574] pci 0000:05:00.0: BAR 6: assigned [mem 0xd0400000-0xd041ffff pref]
[    1.448748] pci 0000:04:00.0: PCI bridge to [bus 05-05]
[    1.448854] pci 0000:04:00.0:   bridge window [io  disabled]
[    1.448964] pci 0000:04:00.0:   bridge window [mem 0xfa000000-0xfbffffff]
[    1.449076] pci 0000:04:00.0:   bridge window [mem 0xd0400000-0xd04fffff 64bit pref]
[    1.449255] pci 0000:00:07.0: PCI bridge to [bus 04-05]
[    1.449361] pci 0000:00:07.0:   bridge window [io  disabled]
[    1.449470] pci 0000:00:07.0:   bridge window [mem 0xfa000000-0xfbffffff]
[    1.449581] pci 0000:00:07.0:   bridge window [mem 0xd0400000-0xd04fffff 64bit pref]
[    1.449760] pci 0000:01:03.0: BAR 6: assigned [mem 0xf7e00000-0xf7e1ffff pref]
[    1.449934] pci 0000:01:04.2: BAR 6: assigned [mem 0xf7e20000-0xf7e2ffff pref]
[    1.450108] pci 0000:00:1e.0: PCI bridge to [bus 01-01]
[    1.450216] pci 0000:00:1e.0:   bridge window [io  0x2000-0x3fff]
[    1.450327] pci 0000:00:1e.0:   bridge window [mem 0xf7e00000-0xf7ffffff]
[    1.450438] pci 0000:00:1e.0:   bridge window [mem 0xd8000000-0xdfffffff 64bit pref]
[    1.450624] pci 0000:00:02.0: setting latency timer to 64
[    1.450857] pci 0000:09:00.0: setting latency timer to 64
[    1.451090] pci 0000:0a:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.451268] pci 0000:0a:00.0: setting latency timer to 64
[    1.451499] pci 0000:0a:01.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.451679] pci 0000:0a:01.0: setting latency timer to 64
[    1.451931] pci 0000:0a:02.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    1.452111] pci 0000:0a:02.0: setting latency timer to 64
[    1.452431] pci 0000:09:00.3: setting latency timer to 64
[    1.452481] pci 0000:00:03.0: setting latency timer to 64
[    1.452487] pci 0000:00:04.0: setting latency timer to 64
[    1.452494] pci 0000:00:05.0: setting latency timer to 64
[    1.452501] pci 0000:00:06.0: setting latency timer to 64
[    1.452509] pci 0000:02:00.0: setting latency timer to 64
[    1.452516] pci 0000:00:07.0: setting latency timer to 64
[    1.452525] pci 0000:04:00.0: setting latency timer to 64
[    1.452531] pci 0000:00:1e.0: setting latency timer to 64
[    1.452535] pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
[    1.452537] pci_bus 0000:00: resource 1 [mem 0x00000000-0xffffffffffffffff]
[    1.452539] pci_bus 0000:09: resource 0 [io  0x5000-0x6fff]
[    1.452541] pci_bus 0000:09: resource 1 [mem 0xfdd00000-0xfdffffff]
[    1.452544] pci_bus 0000:09: resource 2 [mem 0xd0000000-0xd01fffff 64bit pref]
[    1.452546] pci_bus 0000:0a: resource 0 [io  0x5000-0x5fff]
[    1.452548] pci_bus 0000:0a: resource 1 [mem 0xfde00000-0xfdefffff]
[    1.452550] pci_bus 0000:0a: resource 2 [mem 0xd0000000-0xd00fffff 64bit pref]
[    1.452553] pci_bus 0000:0b: resource 0 [io  0x5000-0x5fff]
[    1.452555] pci_bus 0000:0b: resource 1 [mem 0xfde00000-0xfdefffff]
[    1.452557] pci_bus 0000:0b: resource 2 [mem 0xd0000000-0xd00fffff 64bit pref]
[    1.452560] pci_bus 0000:10: resource 0 [io  0x6000-0x6fff]
[    1.452562] pci_bus 0000:10: resource 1 [mem 0xfdf00000-0xfdffffff]
[    1.452564] pci_bus 0000:10: resource 2 [mem 0xd0100000-0xd01fffff 64bit pref]
[    1.452566] pci_bus 0000:06: resource 0 [io  0x4000-0x4fff]
[    1.452568] pci_bus 0000:06: resource 1 [mem 0xfdb00000-0xfdcfffff]
[    1.452571] pci_bus 0000:06: resource 2 [mem 0xd0200000-0xd02fffff 64bit pref]
[    1.452573] pci_bus 0000:02: resource 1 [mem 0xf8000000-0xf9ffffff]
[    1.452576] pci_bus 0000:02: resource 2 [mem 0xd0300000-0xd03fffff 64bit pref]
[    1.452578] pci_bus 0000:03: resource 1 [mem 0xf8000000-0xf9ffffff]
[    1.452580] pci_bus 0000:03: resource 2 [mem 0xd0300000-0xd03fffff 64bit pref]
[    1.452583] pci_bus 0000:04: resource 1 [mem 0xfa000000-0xfbffffff]
[    1.452585] pci_bus 0000:04: resource 2 [mem 0xd0400000-0xd04fffff 64bit pref]
[    1.452587] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfbffffff]
[    1.452590] pci_bus 0000:05: resource 2 [mem 0xd0400000-0xd04fffff 64bit pref]
[    1.452592] pci_bus 0000:01: resource 0 [io  0x2000-0x3fff]
[    1.452594] pci_bus 0000:01: resource 1 [mem 0xf7e00000-0xf7ffffff]
[    1.452596] pci_bus 0000:01: resource 2 [mem 0xd8000000-0xdfffffff 64bit pref]
[    1.452599] pci_bus 0000:01: resource 3 [io  0x0000-0xffff]
[    1.452601] pci_bus 0000:01: resource 4 [mem 0x00000000-0xffffffffffffffff]
[    1.452622] NET: Registered protocol family 2
[    1.452786] IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    1.454270] TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
[    1.456503] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.457161] TCP: Hash tables configured (established 262144 bind 65536)
[    1.457270] TCP reno registered
[    1.457381] UDP hash table entries: 8192 (order: 6, 262144 bytes)
[    1.457671] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
[    1.458058] NET: Registered protocol family 1
[    1.458305] pci 0000:01:03.0: Boot video device
[    1.459610] PCI: CLS 64 bytes, default 64
[    1.459632] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    1.459743] Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
[    1.459918] software IO TLB at phys 0x20000000 - 0x24000000
[    1.460332] platform rtc_cmos: registered platform RTC device (no PNP device found)
[    1.463614] microcode: CPU0 sig=0x6f7, pf=0x40, revision=0x66
[    1.463730] microcode: CPU1 sig=0x6f7, pf=0x40, revision=0x66
[    1.463867] microcode: CPU2 sig=0x6f7, pf=0x40, revision=0x66
[    1.463984] microcode: CPU3 sig=0x6f7, pf=0x40, revision=0x66
[    1.464099] microcode: CPU4 sig=0x6f7, pf=0x40, revision=0x66
[    1.464215] microcode: CPU5 sig=0x6f7, pf=0x40, revision=0x66
[    1.464331] microcode: CPU6 sig=0x6f7, pf=0x40, revision=0x66
[    1.464446] microcode: CPU7 sig=0x6f7, pf=0x40, revision=0x66
[    1.464611] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    1.469318] SGI XFS with ACLs, security attributes, large block/inode numbers, no debug enabled
[    1.470653] msgmni has been set to 19994
[    1.470861] io scheduler noop registered
[    1.471096] io scheduler cfq registered (default)
[    1.471344] pcieport 0000:00:02.0: setting latency timer to 64
[    1.471373] pcieport 0000:00:02.0: irq 48 for MSI/MSI-X
[    1.471499] pcieport 0000:00:03.0: setting latency timer to 64
[    1.471523] pcieport 0000:00:03.0: irq 49 for MSI/MSI-X
[    1.471650] pcieport 0000:00:04.0: setting latency timer to 64
[    1.471673] pcieport 0000:00:04.0: irq 50 for MSI/MSI-X
[    1.471792] pcieport 0000:00:05.0: setting latency timer to 64
[    1.471816] pcieport 0000:00:05.0: irq 51 for MSI/MSI-X
[    1.471950] pcieport 0000:00:06.0: setting latency timer to 64
[    1.471973] pcieport 0000:00:06.0: irq 52 for MSI/MSI-X
[    1.472095] pcieport 0000:00:07.0: setting latency timer to 64
[    1.472118] pcieport 0000:00:07.0: irq 53 for MSI/MSI-X
[    1.472393] pcieport 0000:09:00.0: setting latency timer to 64
[    1.473352] pcieport 0000:0a:00.0: setting latency timer to 64
[    1.474952] pcieport 0000:0a:00.0: irq 54 for MSI/MSI-X
[    1.476650] pcieport 0000:0a:01.0: setting latency timer to 64
[    1.478249] pcieport 0000:0a:01.0: irq 55 for MSI/MSI-X
[    1.479948] pcieport 0000:0a:02.0: setting latency timer to 64
[    1.481547] pcieport 0000:0a:02.0: irq 56 for MSI/MSI-X
[    1.483123] aer 0000:00:02.0:pcie02: AER service couldn't init device: no _OSC support
[    1.483129] aer 0000:00:03.0:pcie02: AER service couldn't init device: no _OSC support
[    1.483134] aer 0000:00:04.0:pcie02: AER service couldn't init device: no _OSC support
[    1.483139] aer 0000:00:05.0:pcie02: AER service couldn't init device: no _OSC support
[    1.483144] aer 0000:00:06.0:pcie02: AER service couldn't init device: no _OSC support
[    1.483149] aer 0000:00:07.0:pcie02: AER service couldn't init device: no _OSC support
[    1.483538] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    1.483714] ACPI: Power Button [PWRF]
[    1.484892] thermal LNXTHERM:01: registered as thermal_zone0
[    1.485008] ACPI: Thermal Zone [THM0] (8 C)
[    1.488863] Linux agpgart interface v0.103
[    1.489082] [drm] Initialized drm 1.1.0 20060810
[    1.489566] pci 0000:01:03.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    1.489806] [drm] Initialized radeon 1.31.0 20080528 for 0000:01:03.0 on minor 0
[    1.491131] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.491369] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    1.491580] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[    1.492156] 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    1.492463] HP CISS Driver (v 3.6.20)
[    1.492674] cciss 0000:06:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.492819] cciss 0000:06:00.0: irq 57 for MSI/MSI-X
[    1.492823] cciss 0000:06:00.0: irq 58 for MSI/MSI-X
[    1.492826] cciss 0000:06:00.0: irq 59 for MSI/MSI-X
[    1.492829] cciss 0000:06:00.0: irq 60 for MSI/MSI-X
[    1.531940] IRQ 59/cciss0: IRQF_DISABLED is not guaranteed on shared IRQs
[    1.532063] cciss0: <0x3230> at PCI 0000:06:00.0 IRQ 59 using DAC
[    1.533982]  cciss/c0d0: p1 p2
[    1.534794] hpilo 0000:01:04.2: PCI INT B -> GSI 22 (level, low) -> IRQ 22
[    1.535856] PNP: PS/2 Controller [PNP0303:KBD,PNP0f0e:PS2M] at 0x60,0x64 irq 1,12
[    1.537720] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.537837] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.538152] mice: PS/2 mouse device common for all mice
[    1.538470] rtc_cmos rtc_cmos: RTC can wake from S4
[    1.538649] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    1.538779] rtc0: alarms up to one year, y3k, 114 bytes nvram, hpet irqs
[    1.539152] cpuidle: using governor ladder
[    1.539257] cpuidle: using governor menu
[    1.539361] ioatdma: Intel(R) QuickData Technology Driver 4.00
[    1.540805] usbcore: registered new interface driver usbhid
[    1.540913] usbhid: USB HID core driver
[    1.541017] Netfilter messages via NETLINK v0.30.
[    1.541146] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    1.541403] CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
[    1.543007] nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or
[    1.543182] sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
[    1.543313] xt_time: kernel timezone is -0000
[    1.543469] ip_tables: (C) 2000-2006 Netfilter Core Team
[    1.543587] TCP cubic registered
[    1.544002] NET: Registered protocol family 10
[    1.544307] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    1.544433] NET: Registered protocol family 17
[    1.549117] rtc_cmos rtc_cmos: setting system clock to 2010-02-19 07:42:14 UTC (1266565334)
[    1.592569] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    2.474221] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2
[    2.479325] Filesystem "cciss/c0d0p1": Disabling barriers, trial barrier write failed
[    2.486463] XFS mounting filesystem cciss/c0d0p1
[    2.531461] Ending clean XFS mount for filesystem: cciss/c0d0p1
[    2.531509] VFS: Mounted root (xfs filesystem) readonly on device 104:1.
[    2.531636] Freeing unused kernel memory: 436k freed
[    2.531895] Write protecting the kernel read-only data: 8192k
[    2.532228] Freeing unused kernel memory: 2012k freed
[    2.533135] Freeing unused kernel memory: 688k freed
[    2.933541] ioctl32(showconsole:735): Unknown cmd fd(0) cmd(80045432){t:'T';sz:4} arg(ff954358) on /dev/console
[    3.278226] udevd version 128 started
[    3.381623] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.381841] ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    3.381996] ehci_hcd 0000:00:1d.7: setting latency timer to 64
[    3.382000] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[    3.382115] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
[    3.382329] ehci_hcd 0000:00:1d.7: debug port 1
[    3.386309] ehci_hcd 0000:00:1d.7: cache line size of 64 is not supported
[    3.386332] ehci_hcd 0000:00:1d.7: irq 16, io mem 0xf7df0000
[    3.402520] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
[    3.402659] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    3.402769] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.402943] usb usb1: Product: EHCI Host Controller
[    3.403049] usb usb1: Manufacturer: Linux 2.6.33-rc8-git3-x86_64 ehci_hcd
[    3.403158] usb usb1: SerialNumber: 0000:00:1d.7
[    3.403359] hub 1-0:1.0: USB hub found
[    3.403466] hub 1-0:1.0: 8 ports detected
[    3.453419] libata version 3.00 loaded.
[    3.527187] ipmi message handler version 39.2
[    3.610042] QLogic Fibre Channel HBA Driver: 8.03.01-k10
[    3.610338] qla2xxx 0000:10:01.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
[    3.611605] qla2xxx 0000:10:01.0: Found an ISP2422, irq 24, iobase 0xffffc900100ae000
[    3.613267] qla2xxx 0000:10:01.0: irq 61 for MSI/MSI-X
[    3.613828] qla2xxx 0000:10:01.0: Configuring PCI space...
[    3.644398] qla2xxx 0000:10:01.0: Configure NVRAM parameters...
[    3.669230] Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.3 (Dec 03, 2009)
[    3.669430] bnx2 0000:03:00.0: BAR 0: set to [mem 0xf8000000-0xf9ffffff 64bit] (PCI address [0xf8000000-0xf9ffffff]
[    3.669618] bnx2 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    3.679958] qla2xxx 0000:10:01.0: Verifying loaded RISC code...
[    3.694402] qla2xxx 0000:10:01.0: firmware: requesting ql2400_fw.bin
[    3.732396] hpwdt 0000:01:04.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
[    3.732551] hpwdt: New timer passed in is 30 seconds.
[    3.732628] hp Watchdog Timer Driver: 1.1.1, timer margin: 30 seconds (nowayout=0), allow kernel dump: OFF (default = 0/OFF), priority: LAST (default = 0/LAST).
[    3.746182] uhci_hcd: USB Universal Host Controller Interface driver
[    3.746391] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    3.746512] uhci_hcd 0000:00:1d.0: setting latency timer to 64
[    3.746515] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[    3.746630] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
[    3.746832] uhci_hcd 0000:00:1d.0: irq 16, io base 0x00001000
[    3.746992] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
[    3.747102] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.747276] usb usb2: Product: UHCI Host Controller
[    3.747382] usb usb2: Manufacturer: Linux 2.6.33-rc8-git3-x86_64 uhci_hcd
[    3.747492] usb usb2: SerialNumber: 0000:00:1d.0
[    3.747675] hub 2-0:1.0: USB hub found
[    3.747783] hub 2-0:1.0: 2 ports detected
[    3.747932] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17
[    3.748047] uhci_hcd 0000:00:1d.1: setting latency timer to 64
[    3.748050] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[    3.748161] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
[    3.748368] uhci_hcd 0000:00:1d.1: irq 17, io base 0x00001020
[    3.748509] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
[    3.748620] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.748794] usb usb3: Product: UHCI Host Controller
[    3.748900] usb usb3: Manufacturer: Linux 2.6.33-rc8-git3-x86_64 uhci_hcd
[    3.749010] usb usb3: SerialNumber: 0000:00:1d.1
[    3.749178] hub 3-0:1.0: USB hub found
[    3.749287] hub 3-0:1.0: 2 ports detected
[    3.749429] uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    3.749543] uhci_hcd 0000:00:1d.2: setting latency timer to 64
[    3.749546] uhci_hcd 0000:00:1d.2: UHCI Host Controller
[    3.749658] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4
[    3.749859] uhci_hcd 0000:00:1d.2: irq 18, io base 0x00001040
[    3.750002] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
[    3.750112] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.750288] usb usb4: Product: UHCI Host Controller
[    3.750394] usb usb4: Manufacturer: Linux 2.6.33-rc8-git3-x86_64 uhci_hcd
[    3.750504] usb usb4: SerialNumber: 0000:00:1d.2
[    3.750666] hub 4-0:1.0: USB hub found
[    3.750773] hub 4-0:1.0: 2 ports detected
[    3.750917] uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    3.751032] uhci_hcd 0000:00:1d.3: setting latency timer to 64
[    3.751035] uhci_hcd 0000:00:1d.3: UHCI Host Controller
[    3.751145] uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 5
[    3.751346] uhci_hcd 0000:00:1d.3: irq 19, io base 0x00001060
[    3.751482] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
[    3.751592] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.751765] usb usb5: Product: UHCI Host Controller
[    3.751871] usb usb5: Manufacturer: Linux 2.6.33-rc8-git3-x86_64 uhci_hcd
[    3.751980] usb usb5: SerialNumber: 0000:00:1d.3
[    3.752141] hub 5-0:1.0: USB hub found
[    3.752248] hub 5-0:1.0: 2 ports detected
[    3.752396] uhci_hcd 0000:01:04.4: PCI INT B -> GSI 22 (level, low) -> IRQ 22
[    3.752511] uhci_hcd 0000:01:04.4: UHCI Host Controller
[    3.752622] uhci_hcd 0000:01:04.4: new USB bus registered, assigned bus number 6
[    3.752806] uhci_hcd 0000:01:04.4: port count misdetected? forcing to 2 ports
[    3.753897] uhci_hcd 0000:01:04.4: irq 22, io base 0x00003800
[    3.754250] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
[    3.754360] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.754535] usb usb6: Product: UHCI Host Controller
[    3.754641] usb usb6: Manufacturer: Linux 2.6.33-rc8-git3-x86_64 uhci_hcd
[    3.754751] usb usb6: SerialNumber: 0000:01:04.4
[    3.754918] hub 6-0:1.0: USB hub found
[    3.755026] hub 6-0:1.0: 2 ports detected
[    3.758664] ata_piix 0000:00:1f.1: version 2.13
[    3.758680] ata_piix 0000:00:1f.1: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    3.758840] ata_piix 0000:00:1f.1: setting latency timer to 64
[    3.758925] scsi1 : ata_piix
[    3.759144] scsi2 : ata_piix
[    3.759277] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0x500 irq 14
[    3.759387] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0x508 irq 15
[    3.789177] IPMI System Interface driver.
[    3.789288] ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0
[    3.815117] qla2xxx 0000:10:01.0: FW: Loading via request-firmware...
[    3.930304] ata1.00: ATAPI: HL-DT-STCD-RW/DVD DRIVE GCC-4244N, 2.00, max UDMA/33
[    3.970197] ata1.00: configured for UDMA/33
[    3.975325] scsi 1:0:0:0: CD-ROM            HL-DT-ST RW/DVD GCC-4244N 2.00 PQ: 0 ANSI: 5
[    4.008102] sr0: scsi3-mmc drive: 24x/24x writer cd/rw xa/form2 cdda tray
[    4.008215] Uniform CD-ROM driver Revision: 3.20
[    4.008395] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    4.012489] sr 1:0:0:0: Attached scsi generic sg0 type 5
[    4.073141] usb 6-1: new full speed USB device using uhci_hcd and address 2
[    4.096395] ipmi: Found new BMC (man_id: 0x00000b,  prod_id: 0x0000, dev_id: 0x11)
[    4.096579] IPMI kcs interface initialized
[    4.096695] ipmi_si: Trying SPMI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
[    4.096874] ipmi_si: duplicate interface
[    4.151939] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
[    4.152118] ipmi_si: duplicate interface
[    4.214400] qla2xxx 0000:10:01.0: Allocated (64 KB) for EFT...
[    4.214592] qla2xxx 0000:10:01.0: Allocated (1285 KB) for firmware dump...
[    4.231890] ipmi_si: probe of 00:02 failed with error -16
[    4.232040] ipmi_si 0000:01:04.6: PCI INT A -> GSI 21 (level, low) -> IRQ 21
[    4.232152] ipmi_si: Trying PCI-specified kcs state machine at mem address 0xf7ef0000, slave address 0x0, irq 21
[    4.234433] scsi0 : qla2xxx
[    4.235197] qla2xxx 0000:10:01.0:
[    4.235199]  QLogic Fibre Channel HBA Driver: 8.03.01-k10
[    4.235200]   QLogic QLA2460 - PCI-X 2.0 Single Channel 4Gb Fibre Channel HBA
[    4.235201]   ISP2422: PCI-X Mode 1 (133 MHz) @ 0000:10:01.0 hdma+, host#=0, fw=4.00.16 (2)
[    4.239239] usb 6-1: New USB device found, idVendor=03f0, idProduct=1027
[    4.239353] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    4.239464] usb 6-1: Product: Virtual Keyboard
[    4.239570] usb 6-1: Manufacturer: HP
[    4.249426] input: HP Virtual Keyboard as /devices/pci0000:00/0000:00:1e.0/0000:01:04.4/usb6/6-1/6-1:1.0/input/input3
[    4.251070] generic-usb 0003:03F0:1027.0001: input: USB HID v1.01 Keyboard [HP Virtual Keyboard] on usb-0000:01:04.4-1/input0
[    4.259108] input: HP Virtual Keyboard as /devices/pci0000:00/0000:00:1e.0/0000:01:04.4/usb6/6-1/6-1:1.1/input/input4
[    4.259323] generic-usb 0003:03F0:1027.0002: input: USB HID v1.01 Mouse [HP Virtual Keyboard] on usb-0000:01:04.4-1/input1
[    4.294392] bnx2 0000:03:00.0: firmware: requesting bnx2/bnx2-mips-06-5.0.0.j3.fw
[    4.310577] bnx2 0000:03:00.0: firmware: requesting bnx2/bnx2-rv2p-06-5.0.0.j3.fw
[    4.315196] eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem f8000000, IRQ 18, node addr xx:xx:xx:xx:xx:xx
[    4.315407] bnx2 0000:05:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    4.381263] usb 6-2: new full speed USB device using uhci_hcd and address 3
[    4.491889] IRQ 21/ipmi_si: IRQF_DISABLED is not guaranteed on shared IRQs
[    4.492016]   Using irq 21
[    4.496042] qla2xxx 0000:10:01.0: LIP reset occurred (f7f7).
[    4.524915] ipmi: interfacing existing BMC (man_id: 0x00000b, prod_id: 0x0000, dev_id: 0x11)
[    4.525108] IPMI kcs interface initialized
[    4.539515] usb 6-2: New USB device found, idVendor=03f0, idProduct=1327
[    4.539626] usb 6-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    4.539737] usb 6-2: Product: Virtual Hub
[    4.539841] usb 6-2: Manufacturer: HP
[    4.545016] hub 6-2:1.0: USB hub found
[    4.547216] hub 6-2:1.0: 7 ports detected
[    4.611570] qla2xxx 0000:10:01.0: LOOP UP detected (4 Gbps).
[    4.910016] bnx2 0000:05:00.0: firmware: requesting bnx2/bnx2-mips-06-5.0.0.j3.fw
[    4.914741] bnx2 0000:05:00.0: firmware: requesting bnx2/bnx2-rv2p-06-5.0.0.j3.fw
[    4.919614] eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 19, node addr xx:xx:xx:xx:xx:xx
[    5.256683] scsi 0:0:0:0: Direct-Access     IBM      1814      FAStT  1060 PQ: 0 ANSI: 5
[    5.257023] sd 0:0:0:0: Attached scsi generic sg1 type 0
[    5.262622] scsi 0:0:0:1: Direct-Access     IBM      1814      FAStT  1060 PQ: 0 ANSI: 5
[    5.262655] sd 0:0:0:0: [sda] 41943040 512-byte logical blocks: (21.4 GB/20.0 GiB)
[    5.263113] sd 0:0:0:1: Attached scsi generic sg2 type 0
[    5.263688] sd 0:0:0:0: [sda] Write Protect is off
[    5.263710] sd 0:0:0:1: [sdb] 10485760 512-byte logical blocks: (5.36 GB/5.00 GiB)
[    5.263971] sd 0:0:0:0: [sda] Mode Sense: 77 00 10 08
[    5.264192] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    5.264280] sd 0:0:0:1: [sdb] Write Protect is off
[    5.264283] sd 0:0:0:1: [sdb] Mode Sense: 77 00 10 08
[    5.264383] scsi 0:0:1:0: Direct-Access     IBM      1814      FAStT  1060 PQ: 0 ANSI: 5
[    5.264505] sd 0:0:0:1: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    5.264537] sd 0:0:1:0: Attached scsi generic sg3 type 0
[    5.265847]  sdb:
[    5.266046]  sda:
[    5.266875] sd 0:0:1:0: [sdc] 41943040 512-byte logical blocks: (21.4 GB/20.0 GiB)
[    5.266920] scsi 0:0:1:1: Direct-Access     IBM      1814      FAStT  1060 PQ: 0 ANSI: 5
[    5.267061] sd 0:0:1:1: Attached scsi generic sg4 type 0
[    5.267227] sd 0:0:1:1: [sdd] 10485760 512-byte logical blocks: (5.36 GB/5.00 GiB)
[    5.267880] sd 0:0:1:1: [sdd] Write Protect is off
[    5.267992] sd 0:0:1:1: [sdd] Mode Sense: 77 00 10 08
[    5.268159] sd 0:0:1:0: [sdc] Write Protect is off
[    5.268210] sd 0:0:1:1: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    5.268446] sd 0:0:1:0: [sdc] Mode Sense: 77 00 10 08
[    5.268690] sd 0:0:1:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    5.269063]  sdd: unknown partition table
[    5.270047]  sdc: unknown partition table
[    5.270334] sd 0:0:1:1: [sdd] Attached SCSI disk
[    5.271189] sd 0:0:1:0: [sdc] Attached SCSI disk
[    5.556639] device-mapper: ioctl: 4.16.0-ioctl (2009-11-05) initialised: dm-devel@redhat.com
[    5.925637]  unknown partition table
[    5.925852]  unknown partition table
[    5.926947] sd 0:0:0:1: [sdb] Attached SCSI disk
[    5.926995] sd 0:0:0:0: [sda] Attached SCSI disk
[    6.998672] loop: module loaded
[    7.009932] Filesystem "cciss/c0d0p2": Disabling barriers, trial barrier write failed
[    7.017811] XFS mounting filesystem cciss/c0d0p2
[    7.120301] Ending clean XFS mount for filesystem: cciss/c0d0p2
[    7.362755] ioctl32(showconsole:1810): Unknown cmd fd(0) cmd(80045432){t:'T';sz:4} arg(ffe8abf8) on /dev/console
[    8.555133] ioctl32(showconsole:2073): Unknown cmd fd(0) cmd(80045432){t:'T';sz:4} arg(ff93b968) on /dev/console
[    9.716169] bnx2 0000:03:00.0: irq 62 for MSI/MSI-X
[    9.836673] bnx2: eth0: using MSI
[    9.839469] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   10.521198] ipmi device interface
[   13.048157] bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex
[   13.051005] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   23.920006] eth0: no IPv6 routers present
[   88.744269] netbomb: local port 10000
[   88.744273] netbomb: local IP xxx.xxx.xxx.xxx
[   88.744274] netbomb: interface eth0
[   88.744276] netbomb: remote port 515
[   88.744277] netbomb: remote IP xxx.xxx.xxx.xxx
[   88.744279] netbomb: remote ethernet address xx:xx:xx:xx:xx:xx
[   88.823849] netbomb: network logging started
[  134.643292] BUG: unable to handle kernel NULL pointer dereference at (null)
[  134.643304] IP: [<ffffffffa003edc2>] bnx2_poll_work+0x32/0x13d0 [bnx2]
[  134.643314] PGD 2a972a067 PUD 2aa245067 PMD 0
[  134.643319] Oops: 0000 [#1] SMP
[  134.643323] last sysfs file: /sys/devices/pci0000:00/0000:00:1e.0/0000:01:04.6/class
[  134.643328] CPU 4
[  134.643334] Pid: 3226, comm: cat Not tainted 2.6.33-rc8-git3-x86_64 #3 /ProLiant DL360 G5
[  134.643339] RIP: 0010:[<ffffffffa003edc2>]  [<ffffffffa003edc2>] bnx2_poll_work+0x32/0x13d0 [bnx2]
[  134.643347] RSP: 0018:ffff8802a9643b38  EFLAGS: 00010092
[  134.643351] RAX: 0000000000000000 RBX: ffff8802afab57c0 RCX: 0000000000000010
[  134.643355] RDX: 0000000000000000 RSI: ffff8802afab57c0 RDI: ffff8802afab4580
[  134.643359] RBP: ffff8802a9643cd8 R08: ffff8802af051000 R09: 0000000000000007
[  134.643363] R10: 000000000000000e R11: 0000000000000000 R12: 0000000000000000
[  134.643367] R13: 0000000000000010 R14: 0000000000000000 R15: ffff8802afab4580
[  134.643371] FS:  0000000000000000(0000) GS:ffff880028300000(0063) knlGS:00000000f765f6c0
[  134.643376] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[  134.643380] CR2: 0000000000000000 CR3: 00000002a9606000 CR4: 00000000000006e0
[  134.643384] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  134.643388] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  134.643392] Process cat (pid: 3226, threadinfo ffff8802a9642000, task ffff8802aa14bff0)
[  134.643396] Stack:
[  134.643398]  0000000000000070 0000000000000002 0000000000000010 ffff8802afab57c0
[  134.643404] <0> ffff8802afab4580 0000000300000002 0000000000000000 0000000100000002
[  134.643410] <0> 0000000000000000 0000000200025220 ffffffff81862e80 0000000000000001
[  134.643418] Call Trace:
[  134.643427]  [<ffffffff8107e3fe>] ? __alloc_pages_nodemask+0xfe/0x660
[  134.643433]  [<ffffffff811b6de6>] ? msi_set_mask_bit+0x26/0xc0
[  134.643438]  [<ffffffff811b6e8b>] ? unmask_msi_irq+0xb/0x10
[  134.643443]  [<ffffffff8106db54>] ? default_enable+0x24/0x40
[  134.643448]  [<ffffffff8106d9b6>] ? check_irq_resend+0x26/0x70
[  134.643453]  [<ffffffff8106cc23>] ? __enable_irq+0x73/0x80
[  134.643459]  [<ffffffffa004019e>] bnx2_poll_msix+0x3e/0xd0 [bnx2]
[  134.643465]  [<ffffffff8135bcd1>] netpoll_poll+0xe1/0x3c0
[  134.643470]  [<ffffffff8135c168>] netpoll_send_skb+0x118/0x210
[  134.643475]  [<ffffffff8135c45b>] netpoll_send_udp+0x1fb/0x210
[  134.643480]  [<ffffffffa00981c5>] write_msg+0x95/0xd0 [netbomb]
[  134.643485]  [<ffffffffa0098255>] netbomb_write+0x55/0xa4 [netbomb]
[  134.643492]  [<ffffffff810f6581>] proc_reg_write+0x71/0xb0
[  134.643498]  [<ffffffff810ab6cb>] vfs_write+0xcb/0x180
[  134.643503]  [<ffffffff810ab870>] sys_write+0x50/0x90
[  134.643509]  [<ffffffff8102a1a4>] sysenter_dispatch+0x7/0x2b
[  134.643513] Code: 56 41 55 41 54 53 48 81 ec 78 01 00 00 48 89 bd 80 fe ff ff 48 89 b5 78 fe ff ff 89 95 74 fe ff ff 89 8d 70 fe ff ff 48 8b 46 70 <0f> b7 10 31 c0 80 fa ff 0f 94 c0 01 c2 66 39 96 12 02 00 00 0f
[  134.643551] RIP  [<ffffffffa003edc2>] bnx2_poll_work+0x32/0x13d0 [bnx2]
[  134.643557]  RSP <ffff8802a9643b38>
[  134.643559] CR2: 0000000000000000
[  134.643563] ---[ end trace 48bdec67d6d7aadb ]---

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: netbomb.c --]
[-- Type: text/x-c++src, Size: 9845 bytes --]

/*
 *  linux/drivers/net/netbomb.c
 *
 *  Based on linux/drivers/net/netconsole.c,
 *  adapted by Bruno Prémont <bonbons@linux-vserver.org>
 */

/****************************************************************
 *      This program is free software; you can redistribute it and/or modify
 *      it under the terms of the GNU General Public License as published by
 *      the Free Software Foundation; either version 2, or (at your option)
 *      any later version.
 *
 *      This program is distributed in the hope that it will be useful,
 *      but WITHOUT ANY WARRANTY; without even the implied warranty of
 *      MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *      GNU General Public License for more details.
 *
 *      You should have received a copy of the GNU General Public License
 *      along with this program; if not, write to the Free Software
 *      Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 *
 ****************************************************************/

#include <linux/mm.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/console.h>
#include <linux/moduleparam.h>
#include <linux/string.h>
#include <linux/netpoll.h>
#include <linux/inet.h>
#include <linux/seq_file.h>
#include <linux/configfs.h>

MODULE_AUTHOR("Maintainer: Bruno Premont <bruno.premont@restena.lu>");
MODULE_DESCRIPTION("Network traffic generator");
MODULE_LICENSE("GPL");

#define MAX_PARAM_LENGTH	256
#define MAX_PRINT_CHUNK		1000

static char config[MAX_PARAM_LENGTH];
module_param_string(netbomb, config, MAX_PARAM_LENGTH, 0);
MODULE_PARM_DESC(netbomb, " netbomb=[src-port]@[src-ip]/[dev],[tgt-port]@<tgt-ip>/[tgt-macaddr]");

#ifndef	MODULE
static int __init option_setup(char *opt)
{
	strlcpy(config, opt, MAX_PARAM_LENGTH);
	return 1;
}
__setup("netbomb=", option_setup);
#endif	/* MODULE */

/* Linked list of all configured targets */
static LIST_HEAD(target_list);

/* This needs to be a spinlock because write_msg() cannot sleep */
static DEFINE_SPINLOCK(target_list_lock);

/**
 * struct netbomb_target - Represents a configured netbomb target.
 * @list:	Links this target into the target_list.
 * @item:	Links us into the configfs subsystem hierarchy.
 * @enabled:	On / off knob to enable / disable target.
 *		Visible from userspace (read-write).
 *		We maintain a strict 1:1 correspondence between this and
 *		whether the corresponding netpoll is active or inactive.
 *		Also, other parameters of a target may be modified at
 *		runtime only when it is disabled (enabled == 0).
 * @np:		The netpoll structure for this target.
 *		Contains the other userspace visible parameters:
 *		dev_name	(read-write)
 *		local_port	(read-write)
 *		remote_port	(read-write)
 *		local_ip	(read-write)
 *		remote_ip	(read-write)
 *		local_mac	(read-only)
 *		remote_mac	(read-write)
 */
struct netbomb_target {
	struct list_head	list;
	int			enabled;
	struct netpoll		np;
};

/*
 * No danger of targets going away from under us when dynamic
 * reconfigurability is off.
 */
static void netbomb_target_get(struct netbomb_target *nt)
{
}

static void netbomb_target_put(struct netbomb_target *nt)
{
}

/* Allocate new target (from boot/module param) and setup netpoll for it */
static struct netbomb_target *alloc_param_target(char *target_config)
{
	int err = -ENOMEM;
	struct netbomb_target *nt;

	/*
	 * Allocate and initialize with defaults.
	 * Note that these targets get their config_item fields zeroed-out.
	 */
	nt = kzalloc(sizeof(*nt), GFP_KERNEL);
	if (!nt) {
		printk(KERN_ERR "netbomb: failed to allocate memory\n");
		goto fail;
	}

	nt->np.name = "netbomb";
	strlcpy(nt->np.dev_name, "eth0", IFNAMSIZ);
	nt->np.local_port = 6665;
	nt->np.remote_port = 6666;
	memset(nt->np.remote_mac, 0xff, ETH_ALEN);

	/* Parse parameters and setup netpoll */
	err = netpoll_parse_options(&nt->np, target_config);
	if (err)
		goto fail;

	err = netpoll_setup(&nt->np);
	if (err)
		goto fail;

	nt->enabled = 1;

	return nt;

fail:
	kfree(nt);
	return ERR_PTR(err);
}

/* Cleanup netpoll for given target (from boot/module param) and free it */
static void free_param_target(struct netbomb_target *nt)
{
	netpoll_cleanup(&nt->np);
	kfree(nt);
}

/* Handle network interface device notifications */
static int netbomb_netdev_event(struct notifier_block *this,
				   unsigned long event,
				   void *ptr)
{
	unsigned long flags;
	struct netbomb_target *nt;
	struct net_device *dev = ptr;

	if (!(event == NETDEV_CHANGENAME || event == NETDEV_UNREGISTER))
		goto done;

	spin_lock_irqsave(&target_list_lock, flags);
	list_for_each_entry(nt, &target_list, list) {
		netbomb_target_get(nt);
		if (nt->np.dev == dev) {
			switch (event) {
			case NETDEV_CHANGENAME:
				strlcpy(nt->np.dev_name, dev->name, IFNAMSIZ);
				break;
			case NETDEV_UNREGISTER:
				if (!nt->enabled)
					break;
				netpoll_cleanup(&nt->np);
				nt->enabled = 0;
				printk(KERN_INFO "netbomb: network logging stopped"
					", interface %s unregistered\n",
					dev->name);
				break;
			}
		}
		netbomb_target_put(nt);
	}
	spin_unlock_irqrestore(&target_list_lock, flags);

done:
	return NOTIFY_DONE;
}

static struct notifier_block netbomb_netdev_notifier = {
	.notifier_call  = netbomb_netdev_event,
};

static void write_msg(struct console *con, const char *msg, unsigned int len)
{
	int frag, left;
	unsigned long flags;
	struct netbomb_target *nt;
	const char *tmp;

	/* Avoid taking lock and disabling interrupts unnecessarily */
	if (list_empty(&target_list))
		return;

	spin_lock_irqsave(&target_list_lock, flags);
	list_for_each_entry(nt, &target_list, list) {
		netbomb_target_get(nt);
		if (nt->enabled && netif_running(nt->np.dev)) {
			/*
			 * We nest this inside the for-each-target loop above
			 * so that we're able to get as much logging out to
			 * at least one target if we die inside here, instead
			 * of unnecessarily keeping all targets in lock-step.
			 */
			tmp = msg;
			for (left = len; left;) {
				frag = min(left, MAX_PRINT_CHUNK);
				netpoll_send_udp(&nt->np, tmp, frag);
				tmp += frag;
				left -= frag;
			}
		}
		netbomb_target_put(nt);
	}
	spin_unlock_irqrestore(&target_list_lock, flags);
}

static struct console netbomb = {
	.name	= "netcon",
	.flags	= CON_ENABLED,
	.write	= write_msg,
};

static int netbomb_show(struct seq_file *m, void *v)
{
	seq_printf(m, "Array of bytes indicating packet lengths\n");
	return 0;
}

static int netbomb_open(struct inode *inode, struct file *file)
{
	return single_open(file, netbomb_show, NULL);
}

static const char * bufs[256];

static ssize_t netbomb_write(struct file *file,
        const char __user *buffer, size_t count, loff_t *pos)
{
	int i;

	for (i = 0; i < count; i++) {
		unsigned char c;
		const char *buff;

		if (copy_from_user(&c, buffer+i, 1))
			return -EFAULT;

		if ((buff = bufs[c]) != NULL)
			write_msg(NULL, buff, strlen(buff));
	}
	return count;
}

static const struct file_operations netbomb_fops = {
	.owner   = THIS_MODULE,
	.open    = netbomb_open,
	.read    = seq_read,
	.llseek  = seq_lseek,
	.release = single_release,
	.write   = netbomb_write,
};

static int __init init_netbomb(void)
{
	int err, i;
	struct netbomb_target *nt, *tmp;
	unsigned long flags;
	char *target_config;
	char *input = config;

	if (strnlen(input, MAX_PARAM_LENGTH)) {
		while ((target_config = strsep(&input, ";"))) {
			nt = alloc_param_target(target_config);
			if (IS_ERR(nt)) {
				err = PTR_ERR(nt);
				goto fail;
			}
			/* Dump existing printks when we register */
			netbomb.flags |= CON_PRINTBUFFER;

			spin_lock_irqsave(&target_list_lock, flags);
			list_add(&nt->list, &target_list);
			spin_unlock_irqrestore(&target_list_lock, flags);
		}
	}

	for (i = 0; i < 256; i++) {
		char *p = kmalloc(i > 0 ? i+1 : 2, GFP_KERNEL);
		bufs[i] = p;
		if (p == NULL)
			continue;
		if (i == 0) {
			p[0] = '\n';
			p[1] = '\0';
		} else {
			memset(p, '_', i > 0 ? i+1 : 2);
			if (i < 10)
				sprintf(p + i - 1, "%d", i);
			else if (i < 100)
				sprintf(p + i - 2, "%d", i);
			else
				sprintf(p + i - 3, "%d", i);
			p[i] = '\0';
		}
	}

	err = register_netdevice_notifier(&netbomb_netdev_notifier);
	if (err)
		goto fail;
	proc_create("netbomb-trigger", 0, NULL, &netbomb_fops);

	printk(KERN_INFO "netbomb: network logging started\n");

	return err;

undonotifier:
	unregister_netdevice_notifier(&netbomb_netdev_notifier);

fail:
	printk(KERN_ERR "netbomb: cleaning up\n");

	/*
	 * Remove all targets and destroy them (only targets created
	 * from the boot/module option exist here). Skipping the list
	 * lock is safe here, and netpoll_cleanup() will sleep.
	 */
	list_for_each_entry_safe(nt, tmp, &target_list, list) {
		list_del(&nt->list);
		free_param_target(nt);
	}

	return err;
}

static void __exit cleanup_netbomb(void)
{
	struct netbomb_target *nt, *tmp;
	int i;

	remove_proc_entry("netbomb-trigger", NULL);
	unregister_netdevice_notifier(&netbomb_netdev_notifier);

	/*
	 * Targets created via configfs pin references on our module
	 * and would first be rmdir(2)'ed from userspace. We reach
	 * here only when they are already destroyed, and only those
	 * created from the boot/module option are left, so remove and
	 * destroy them. Skipping the list lock is safe here, and
	 * netpoll_cleanup() will sleep.
	 */
	list_for_each_entry_safe(nt, tmp, &target_list, list) {
		list_del(&nt->list);
		free_param_target(nt);
	}

	for (i = 0; i < 256; i++)
		kfree(bufs[i]);
}

module_init(init_netbomb);
module_exit(cleanup_netbomb);

  reply	other threads:[~2010-02-19  8:10 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-12-29  7:49 BNX2: Kernel crashes with 2.6.31 and 2.6.31.9 Bruno Prémont
2009-12-29  9:05 ` Benjamin Li
2009-12-29  9:33   ` Bruno Prémont
2009-12-29 13:54   ` Bruno Prémont
2009-12-30  5:08     ` Benjamin Li
2010-02-19  8:10       ` Bruno Prémont [this message]
2010-02-19 19:57         ` Benjamin Li
2010-02-19 21:03           ` Brian Haley
2010-02-19 21:47             ` Benjamin Li
2010-02-23 12:15           ` Bruno Prémont
2010-03-02  1:26             ` Benjamin Li
2010-03-02  7:10               ` Bruno Prémont
2010-03-02  8:20                 ` Bruno Prémont
2010-03-02 22:12                   ` Michael Chan
2010-03-04 20:31                     ` Brian Haley
2010-03-10 23:09                       ` Brian Haley
2010-03-10 23:32                         ` Michael Chan
2010-03-11  2:09                           ` Brian Haley
2010-03-11 17:49                             ` Michael Chan
2010-03-11 18:05                               ` David Miller
2010-03-11 18:38                                 ` Michael Chan
2010-03-11 19:40                                   ` Brian Haley
2010-03-11 19:47                                     ` Michael Chan
2010-03-11 21:57                                       ` Brian Haley

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100219091034.5fbb0165@pluto.restena.lu \
    --to=bonbons@linux-vserver.org \
    --cc=benli@broadcom.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mchan@broadcom.com \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).