All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: Kernel crash at md_seq_show of drivers/md/md.c
@ 2013-05-15 12:11 Mo, Moore
  2013-05-16  1:19 ` NeilBrown
  0 siblings, 1 reply; 3+ messages in thread
From: Mo, Moore @ 2013-05-15 12:11 UTC (permalink / raw)
  To: linux-raid@vger.kernel.org

[-- Attachment #1: Type: text/plain, Size: 4783 bytes --]

Dear Neil,

	Thank you for your clue. I make deeper trace after that and found another suspicion point. Kernel maybe wakeup a mdk_thread which was kfree in failed case of mddev->pers->run(mddev).

	For make sure my guess, I add some printk in order to get more info. It seems that md_seq_show try to wake up a mdk_thread in mddev_unlock() before md_seq_show return, but the thread was kfree already by "out_free_conf" case of run() function in drivers/md/raid10.c.

	The Oops report as attached.

----------------- Patch of my debug code ---------------------------
Index: drivers/md/md.c
===================================================================
--- drivers/md/md.c     (revision 1911)
+++ drivers/md/md.c     (working copy)
@@ -675,6 +675,8 @@
        } else
                mutex_unlock(&mddev->reconfig_mutex);
 
+       if(mddev->thread)
+               printk("%s:%d Try to md_wakeup_thread(%p) of dev(%p)\n", __func__, __LINE__, mddev->thread, mddev);
        md_wakeup_thread(mddev->thread);
 }
 
@@ -4531,7 +4533,7 @@
 
        err = mddev->pers->run(mddev);
        if (err)
-               printk(KERN_ERR "md: pers->run() failed ...\n");
+               printk(KERN_ERR "md: mddev(%p) pers->run() failed ...\n", mddev);
        else if (mddev->pers->size(mddev, 0, 0) < mddev->array_sectors) {
                WARN_ONCE(!mddev->external_size, "%s: default size too small,"
                          " but 'external_size' not in effect?\n", __func__);
@@ -6451,6 +6453,7 @@
 
                seq_printf(seq, "\n");
        }
+       printk("%s:%d Try to mddev_unlock(%p)\n", __func__, __LINE__, mddev);
        mddev_unlock(mddev);
 
        return 0;
Index: drivers/md/raid10.c
===================================================================
--- drivers/md/raid10.c (revision 1911)
+++ drivers/md/raid10.c (working copy)
@@ -2398,7 +2398,9 @@
        return 0;
 
 out_free_conf:
+       printk("%s:%d Before md_unregister_thread(%p), Queue=%p\n", __func__, __LINE__, mddev->thread, mddev->thread?&mddev->thread->wqueue:NULL);
        md_unregister_thread(mddev->thread);
+       printk("%s:%d md_unregister_thread(%p) done.\n", __func__, __LINE__, mddev->thread);
        if (conf->r10bio_pool)
                mempool_destroy(conf->r10bio_pool);
        safe_put_page(conf->tmppage);
--------------------------------------------------------------------


Best Regards,
Moore(莫谋鑫)




-----Original Message-----
From: NeilBrown [mailto:neilb@suse.de] 
Sent: 2013年4月29日 9:40
To: Mo, Moore
Cc: jbrassow@redhat.com
Subject: Re: Kernel crash at md_seq_show of drivers/md/md.c


Hi,
 thanks for the report.
 For future reference, it is best to post questions like this to 
   linux-raid@vger.kernel.org
 as listed in the MAINTAINERS file.

I couldn't find an "Oops" output attached....
(see below)


On Thu, 25 Apr 2013 11:05:45 +0800 "Mo, Moore" <Moore.Mo@Polycom.com> wrote:

> Dear Neil & Jonathan,
> 
>         Sorry for disturb you. I got you mail info from git HEAD. I think you are the right person for solve a drivers/md crash issue which I met.
>         Currently, I met a NULL pointer kernel Oops during exception coverage test. I intended make an existed RAID volume under “Fail” status via BIOS SATA OROM Utility, then  kernel(2.6.37.6  x86_64) crashed during try to “mdadm -l10 …” in /etc/rc3.d.  The enclosed is log of Oops output.
> 
>         I through code path follow the backtrace of Oops and found sth. question. It seems that “md_seq_show” NOT handle NULL case which maybe return from md_seq_start.
> // piece code of seq_read
> ssize_t seq_read(struct file *file, char __user *buf, size_t size, 
> loff_t *ppos) {
>         struct seq_file *m = file->private_data;
>         size_t copied = 0;
>         loff_t pos;
> 
>         ………….
> 
>         /* we need at least one record in buffer */
>         pos = m->index;
>         p = m->op->start(m, &pos);                  // md_seq_show maybe will return NULL.
>         while (1) {
>                 err = PTR_ERR(p);
>                 if (!p || IS_ERR(p))
>                         break;

If md_seq_show returned NULL, then the above "if" would notice that "!p" is true, and would break out of the loop.  So the following line will never get executed with 'p == NULL'.

So I don't know what the problem might be.  Maybe if you can post the Oops report (or linux-raid - don't send any HTML, just plain text) I might be able to see what is happening.

NeilBrown



>                 err = m->op->show(m, p);
>                 if (err < 0)
>                         break;
>                 if (unlikely(err))
>                         m->count = 0;
> 

[-- Attachment #2: RAID_crash.log --]
[-- Type: application/octet-stream, Size: 47227 bytes --]

PID hash table entries: 4096 (order: 3, 32768 bytes)
xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
Checking aperture...
No AGP bridge found
Queued invalidation will be enabled to support x2apic and Intr-remapping.
Memory: 32777336k/35651584k available (4891k kernel code, 2130856k absent, 743392k reserved, 7316k data, 9760k init)
Hierarchical RCU implementation.
        RCU-based detection of stalled CPUs is disabled.
NR_IRQS:262400 nr_irqs:1688 16
Extended CMOS year: 2000
Console: colour VGA+ 80x25
console [tty1] enabled
console [ttyS0] enabled
allocated 335544320 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Fast TSC calibration using PIT
Detected 1999.871 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 3999.74 BogoMIPS (lpj=1999871)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 18 MCE banks
CPU0: Thermal monitoring enabled (TM1)
using mwait in idle threads.
Performance Events: PEBS fmt1+, generic architected perfmon, Intel PMU driver.
... version:                3
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             000000007fffffff
... fixed-purpose events:   3
... event mask:             000000070000000f
ACPI: Core revision 20101013
ftrace: allocating 18949 entries in 75 pages
DMAR: Host address width 46
DMAR: DRHD base: 0x000000fbffe000 flags: 0x0
IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0462 ecap f020fe
DMAR: DRHD base: 0x000000dfffc000 flags: 0x1
IOMMU 1: reg_base_addr dfffc000 ver 1:0 cap d2078c106f0462 ecap f020fe
DMAR: RMRR base: 0x0000007dfa8000 end: 0x0000007dfb4fff
DMAR: ATSR flags: 0x0
DMAR: RHSA base: 0x000000fbffe000 proximity domain: 0x1
DMAR: RHSA base: 0x000000dfffc000 proximity domain: 0x0
IOAPIC id 3 under DRHD base  0xfbffe000 IOMMU 0
IOAPIC id 0 under DRHD base  0xdfffc000 IOMMU 1
IOAPIC id 2 under DRHD base  0xdfffc000 IOMMU 1
HPET id 0 under DRHD base 0xdfffc000
Enabled Interrupt-remapping
Enabling x2apic
Enabled x2apic
Setting APIC routing to cluster x2apic
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz stepping 07
NMI watchdog enabled, takes one hw-pmu counter.
Booting Node   0, Processors  #1
NMI watchdog enabled, takes one hw-pmu counter.
 #2
NMI watchdog enabled, takes one hw-pmu counter.
 #3
NMI watchdog enabled, takes one hw-pmu counter.
 #4
NMI watchdog enabled, takes one hw-pmu counter.
 #5
NMI watchdog enabled, takes one hw-pmu counter.
 Ok.
Booting Node   1, Processors  #6
NMI watchdog enabled, takes one hw-pmu counter.
 #7
NMI watchdog enabled, takes one hw-pmu counter.
 #8
NMI watchdog enabled, takes one hw-pmu counter.
 #9
NMI watchdog enabled, takes one hw-pmu counter.
 #10
NMI watchdog enabled, takes one hw-pmu counter.
 #11
NMI watchdog enabled, takes one hw-pmu counter.
 Ok.
Booting Node   0, Processors  #12
NMI watchdog enabled, takes one hw-pmu counter.
 #13
NMI watchdog enabled, takes one hw-pmu counter.
 #14
NMI watchdog enabled, takes one hw-pmu counter.
 #15
NMI watchdog enabled, takes one hw-pmu counter.
 #16
NMI watchdog enabled, takes one hw-pmu counter.
 #17
NMI watchdog enabled, takes one hw-pmu counter.
 Ok.
Booting Node   1, Processors  #18
NMI watchdog enabled, takes one hw-pmu counter.
 #19
NMI watchdog enabled, takes one hw-pmu counter.
 #20
NMI watchdog enabled, takes one hw-pmu counter.
 #21
NMI watchdog enabled, takes one hw-pmu counter.
 #22
NMI watchdog enabled, takes one hw-pmu counter.
 #23 Ok.
NMI watchdog enabled, takes one hw-pmu counter.
Brought up 24 CPUs
Total of 24 processors activated (95987.94 BogoMIPS).
devtmpfs: initialized
regulator: core version 0.5
regulator: dummy: 
NET: Registered protocol family 16
ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
ACPI: bus type pci registered
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000)
PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: Executed 1 blocks of module-level executable AML code
ACPI: Dynamic OEM Table Load:
ACPI: PRAD           (null) 000BE (v02 PRADID  PRADTID 00000001 MSFT 04000000)
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
[Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])
pci_root PNP0A08:00: host bridge window [io  0x0000-0x03af]
pci_root PNP0A08:00: host bridge window [io  0x03e0-0x0cf7]
pci_root PNP0A08:00: host bridge window [io  0x03b0-0x03df]
pci_root PNP0A08:00: host bridge window [io  0x0d00-0x9fff]
pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A08:00: host bridge window [mem 0x000c0000-0x000dffff]
pci_root PNP0A08:00: host bridge window [mem 0xfed08000-0xfed08fff]
pci_root PNP0A08:00: host bridge window [mem 0xfed0e000-0xfed0ffff]
pci_root PNP0A08:00: host bridge window [mem 0x80000000-0xdfffffff]
pci_root PNP0A08:00: host bridge window [mem 0x3c0000000000-0x3c0fffffffff]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.1: PCI bridge to [bus 02-03]
pci 0000:00:02.0: PCI bridge to [bus 04-0e]
pci 0000:04:00.0: PCI bridge to [bus 05-0e]
pci 0000:05:01.0: PCI bridge to [bus 06-06]
pci 0000:05:02.0: PCI bridge to [bus 07-07]
pci 0000:05:04.0: PCI bridge to [bus 08-08]
pci 0000:05:05.0: PCI bridge to [bus 09-09]
pci 0000:05:06.0: PCI bridge to [bus 0a-0a]
pci 0000:05:08.0: PCI bridge to [bus 0b-0b]
pci 0000:05:09.0: PCI bridge to [bus 0c-0c]
pci 0000:05:0a.0: PCI bridge to [bus 0d-0d]
pci 0000:05:0b.0: PCI bridge to [bus 0e-0e]
pci 0000:00:02.2: PCI bridge to [bus 0f-0f]
pci 0000:00:03.0: PCI bridge to [bus 10-1a]
pci 0000:10:00.0: PCI bridge to [bus 11-1a]
pci 0000:11:01.0: PCI bridge to [bus 12-12]
pci 0000:11:02.0: PCI bridge to [bus 13-13]
pci 0000:11:04.0: PCI bridge to [bus 14-14]
pci 0000:11:05.0: PCI bridge to [bus 15-15]
pci 0000:11:06.0: PCI bridge to [bus 16-16]
pci 0000:11:08.0: PCI bridge to [bus 17-17]
pci 0000:11:09.0: PCI bridge to [bus 18-18]
pci 0000:11:0a.0: PCI bridge to [bus 19-19]
pci 0000:11:0b.0: PCI bridge to [bus 1a-1a]
pci 0000:00:03.2: PCI bridge to [bus 1b-1b]
pci 0000:00:11.0: PCI bridge to [bus 1c-1c]
pci 0000:00:1c.0: PCI bridge to [bus 1d-1d]
pci 0000:00:1c.7: PCI bridge to [bus 1e-22]
pci 0000:1e:00.0: PCI bridge to [bus 1f-22]
pci 0000:1f:00.0: PCI bridge to [bus 20-21]
pci 0000:20:00.0: PCI bridge to [bus 21-21]
pci 0000:1f:01.0: PCI bridge to [bus 22-22]
pci 0000:00:1e.0: PCI bridge to [bus 23-23] (subtractive decode)
ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus 7f])
ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 80-fe])
pci_root PNP0A08:01: host bridge window [io  0xa000-0xffff]
pci_root PNP0A08:01: host bridge window [mem 0xe0000000-0xfbffffff]
pci_root PNP0A08:01: host bridge window [mem 0x3c1000000000-0x3c1fffffffff]
pci 0000:80:02.0: PCI bridge to [bus 81-81]
ACPI: PCI Root Bridge [UNC1] (domain 0000 [bus ff])
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 *7 10 11 12 14 15)
HEST: HEST table parsing is initialized.
vgaarb: device added: PCI:0000:21:00.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
hpet0: 8 comparators, 64-bit 14.318180 MHz counter
Switching to clocksource tsc
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 23 devices
ACPI: ACPI bus type pnp unregistered
system 00:01: [mem 0xfc000000-0xfcffffff] has been reserved
system 00:01: [mem 0xfd000000-0xfdffffff] has been reserved
system 00:01: [mem 0xfe000000-0xfeafffff] has been reserved
system 00:01: [mem 0xfeb00000-0xfebfffff] has been reserved
system 00:02: [mem 0xdfffc000-0xdfffdfff] has been reserved
system 00:06: [io  0x04d0-0x04d1] has been reserved
system 00:06: [mem 0x00000400-0x000004ff] could not be reserved
system 00:08: [io  0x0a00-0x0a07] has been reserved
system 00:0a: [io  0x0370-0x0371] has been reserved
system 00:0a: [io  0x0a00-0x0a07] has been reserved
system 00:0a: [io  0x0a10-0x0a3f] has been reserved
system 00:0e: [io  0x0400-0x0453] has been reserved
system 00:0e: [io  0x0458-0x047f] has been reserved
system 00:0e: [io  0x1180-0x119f] has been reserved
system 00:0e: [io  0x0500-0x057f] has been reserved
system 00:0e: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:0e: [mem 0xfec00000-0xfecfffff] could not be reserved
system 00:0e: [mem 0xff000000-0xffffffff] has been reserved
system 00:0f: [io  0x0454-0x0457] has been reserved
system 00:14: [mem 0xfbffe000-0xfbffffff] has been reserved
system 00:16: [mem 0x00000000-0x0009ffff] could not be reserved
pci 0000:00:02.0: BAR 13: assigned [io  0x2000-0x4fff]
pci 0000:00:03.0: BAR 13: can't assign io (size 0x3000)
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  disabled]
pci 0000:00:01.0:   bridge window [mem disabled]
pci 0000:00:01.0:   bridge window [mem pref disabled]
pci 0000:00:01.1: PCI bridge to [bus 02-03]
pci 0000:00:01.1:   bridge window [io  0x8000-0x8fff]
pci 0000:00:01.1:   bridge window [mem 0xdf700000-0xdf7fffff]
pci 0000:00:01.1:   bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 13: assigned [io  0x2000-0x4fff]
pci 0000:05:01.0: BAR 14: can't assign mem (size 0x200000)
pci 0000:05:01.0: BAR 15: assigned [mem 0x3c0d50000000-0x3c0d501fffff 64bit pref]
pci 0000:05:01.0: BAR 13: assigned [io  0x2000-0x2fff]
pci 0000:05:05.0: BAR 13: assigned [io  0x3000-0x3fff]
pci 0000:05:09.0: BAR 13: assigned [io  0x4000-0x4fff]
pci 0000:05:01.0: PCI bridge to [bus 06-06]
pci 0000:05:01.0:   bridge window [io  0x2000-0x2fff]
pci 0000:05:01.0:   bridge window [mem disabled]
pci 0000:05:01.0:   bridge window [mem 0x3c0d50000000-0x3c0d501fffff 64bit pref]
pci 0000:05:02.0: PCI bridge to [bus 07-07]
pci 0000:05:02.0:   bridge window [io  disabled]
pci 0000:05:02.0:   bridge window [mem disabled]
pci 0000:05:02.0:   bridge window [mem pref disabled]
pci 0000:05:04.0: PCI bridge to [bus 08-08]
pci 0000:05:04.0:   bridge window [io  disabled]
pci 0000:05:04.0:   bridge window [mem disabled]
pci 0000:05:04.0:   bridge window [mem pref disabled]
pci 0000:05:05.0: PCI bridge to [bus 09-09]
pci 0000:05:05.0:   bridge window [io  0x3000-0x3fff]
pci 0000:05:05.0:   bridge window [mem 0xdf500000-0xdf5fffff]
pci 0000:05:05.0:   bridge window [mem 0x3c0f80000000-0x3c0fcfffffff 64bit pref]
pci 0000:05:06.0: PCI bridge to [bus 0a-0a]
pci 0000:05:06.0:   bridge window [io  disabled]
pci 0000:05:06.0:   bridge window [mem 0xdf400000-0xdf4fffff]
pci 0000:05:06.0:   bridge window [mem 0x3c0f00000000-0x3c0f4fffffff 64bit pref]
pci 0000:05:08.0: PCI bridge to [bus 0b-0b]
pci 0000:05:08.0:   bridge window [io  disabled]
pci 0000:05:08.0:   bridge window [mem 0xdf300000-0xdf3fffff]
pci 0000:05:08.0:   bridge window [mem 0x3c0e80000000-0x3c0ecfffffff 64bit pref]
pci 0000:05:09.0: PCI bridge to [bus 0c-0c]
pci 0000:05:09.0:   bridge window [io  0x4000-0x4fff]
pci 0000:05:09.0:   bridge window [mem 0xdf200000-0xdf2fffff]
pci 0000:05:09.0:   bridge window [mem 0x3c0e00000000-0x3c0e4fffffff 64bit pref]
pci 0000:05:0a.0: PCI bridge to [bus 0d-0d]
pci 0000:05:0a.0:   bridge window [io  disabled]
pci 0000:05:0a.0:   bridge window [mem 0xdf100000-0xdf1fffff]
pci 0000:05:0a.0:   bridge window [mem 0x3c0d80000000-0x3c0dcfffffff 64bit pref]
pci 0000:05:0b.0: PCI bridge to [bus 0e-0e]
pci 0000:05:0b.0:   bridge window [io  disabled]
pci 0000:05:0b.0:   bridge window [mem 0xdf000000-0xdf0fffff]
pci 0000:05:0b.0:   bridge window [mem 0x3c0d00000000-0x3c0d4fffffff 64bit pref]
pci 0000:04:00.0: PCI bridge to [bus 05-0e]
pci 0000:04:00.0:   bridge window [io  0x2000-0x4fff]
pci 0000:04:00.0:   bridge window [mem 0xdf000000-0xdf5fffff]
pci 0000:04:00.0:   bridge window [mem 0x3c0d00000000-0x3c0fcfffffff 64bit pref]
pci 0000:00:02.0: PCI bridge to [bus 04-0e]
pci 0000:00:02.0:   bridge window [io  0x2000-0x4fff]
pci 0000:00:02.0:   bridge window [mem 0xdf000000-0xdf6fffff]
pci 0000:00:02.0:   bridge window [mem 0x3c0d00000000-0x3c0fcfffffff 64bit pref]
pci 0000:00:02.2: PCI bridge to [bus 0f-0f]
pci 0000:00:02.2:   bridge window [io  disabled]
pci 0000:00:02.2:   bridge window [mem disabled]
pci 0000:00:02.2:   bridge window [mem pref disabled]
pci 0000:10:00.0: BAR 13: can't assign io (size 0x3000)
pci 0000:11:01.0: BAR 14: can't assign mem (size 0x200000)
pci 0000:11:01.0: BAR 15: assigned [mem 0x3c0a50000000-0x3c0a501fffff 64bit pref]
pci 0000:11:01.0: BAR 13: can't assign io (size 0x1000)
pci 0000:11:05.0: BAR 13: can't assign io (size 0x1000)
pci 0000:11:09.0: BAR 13: can't assign io (size 0x1000)
pci 0000:11:01.0: PCI bridge to [bus 12-12]
pci 0000:11:01.0:   bridge window [io  disabled]
pci 0000:11:01.0:   bridge window [mem disabled]
pci 0000:11:01.0:   bridge window [mem 0x3c0a50000000-0x3c0a501fffff 64bit pref]
pci 0000:11:02.0: PCI bridge to [bus 13-13]
pci 0000:11:02.0:   bridge window [io  disabled]
pci 0000:11:02.0:   bridge window [mem disabled]
pci 0000:11:02.0:   bridge window [mem pref disabled]
pci 0000:11:04.0: PCI bridge to [bus 14-14]
pci 0000:11:04.0:   bridge window [io  disabled]
pci 0000:11:04.0:   bridge window [mem disabled]
pci 0000:11:04.0:   bridge window [mem pref disabled]
pci 0000:11:05.0: PCI bridge to [bus 15-15]
pci 0000:11:05.0:   bridge window [io  disabled]
pci 0000:11:05.0:   bridge window [mem 0xdee00000-0xdeefffff]
pci 0000:11:05.0:   bridge window [mem 0x3c0c80000000-0x3c0ccfffffff 64bit pref]
pci 0000:11:06.0: PCI bridge to [bus 16-16]
pci 0000:11:06.0:   bridge window [io  disabled]
pci 0000:11:06.0:   bridge window [mem 0xded00000-0xdedfffff]
pci 0000:11:06.0:   bridge window [mem 0x3c0c00000000-0x3c0c4fffffff 64bit pref]
pci 0000:11:08.0: PCI bridge to [bus 17-17]
pci 0000:11:08.0:   bridge window [io  disabled]
pci 0000:11:08.0:   bridge window [mem 0xdec00000-0xdecfffff]
pci 0000:11:08.0:   bridge window [mem 0x3c0b80000000-0x3c0bcfffffff 64bit pref]
pci 0000:11:09.0: PCI bridge to [bus 18-18]
pci 0000:11:09.0:   bridge window [io  disabled]
pci 0000:11:09.0:   bridge window [mem 0xdeb00000-0xdebfffff]
pci 0000:11:09.0:   bridge window [mem 0x3c0b00000000-0x3c0b4fffffff 64bit pref]
pci 0000:11:0a.0: PCI bridge to [bus 19-19]
pci 0000:11:0a.0:   bridge window [io  disabled]
pci 0000:11:0a.0:   bridge window [mem 0xdea00000-0xdeafffff]
pci 0000:11:0a.0:   bridge window [mem 0x3c0a80000000-0x3c0acfffffff 64bit pref]
pci 0000:11:0b.0: PCI bridge to [bus 1a-1a]
pci 0000:11:0b.0:   bridge window [io  disabled]
pci 0000:11:0b.0:   bridge window [mem 0xde900000-0xde9fffff]
pci 0000:11:0b.0:   bridge window [mem 0x3c0a00000000-0x3c0a4fffffff 64bit pref]
pci 0000:10:00.0: PCI bridge to [bus 11-1a]
pci 0000:10:00.0:   bridge window [io  disabled]
pci 0000:10:00.0:   bridge window [mem 0xde900000-0xdeefffff]
pci 0000:10:00.0:   bridge window [mem 0x3c0a00000000-0x3c0ccfffffff 64bit pref]
pci 0000:00:03.0: PCI bridge to [bus 10-1a]
pci 0000:00:03.0:   bridge window [io  disabled]
pci 0000:00:03.0:   bridge window [mem 0xde900000-0xdeffffff]
pci 0000:00:03.0:   bridge window [mem 0x3c0a00000000-0x3c0ccfffffff 64bit pref]
pci 0000:00:03.2: PCI bridge to [bus 1b-1b]
pci 0000:00:03.2:   bridge window [io  disabled]
pci 0000:00:03.2:   bridge window [mem disabled]
pci 0000:00:03.2:   bridge window [mem pref disabled]
pci 0000:00:11.0: PCI bridge to [bus 1c-1c]
pci 0000:00:11.0:   bridge window [io  0x7000-0x7fff]
pci 0000:00:11.0:   bridge window [mem disabled]
pci 0000:00:11.0:   bridge window [mem 0x3c0fd0400000-0x3c0fd08fffff 64bit pref]
pci 0000:00:1c.0: PCI bridge to [bus 1d-1d]
pci 0000:00:1c.0:   bridge window [io  disabled]
pci 0000:00:1c.0:   bridge window [mem disabled]
pci 0000:00:1c.0:   bridge window [mem pref disabled]
pci 0000:20:00.0: PCI bridge to [bus 21-21]
pci 0000:20:00.0:   bridge window [io  disabled]
pci 0000:20:00.0:   bridge window [mem 0xde000000-0xde8fffff]
pci 0000:20:00.0:   bridge window [mem 0xdd000000-0xddffffff 64bit pref]
pci 0000:1f:00.0: PCI bridge to [bus 20-21]
pci 0000:1f:00.0:   bridge window [io  disabled]
pci 0000:1f:00.0:   bridge window [mem 0xde000000-0xde8fffff]
pci 0000:1f:00.0:   bridge window [mem 0xdd000000-0xddffffff 64bit pref]
pci 0000:1f:01.0: PCI bridge to [bus 22-22]
pci 0000:1f:01.0:   bridge window [io  disabled]
pci 0000:1f:01.0:   bridge window [mem disabled]
pci 0000:1f:01.0:   bridge window [mem pref disabled]
pci 0000:1e:00.0: PCI bridge to [bus 1f-22]
pci 0000:1e:00.0:   bridge window [io  disabled]
pci 0000:1e:00.0:   bridge window [mem 0xde000000-0xde8fffff]
pci 0000:1e:00.0:   bridge window [mem 0xdd000000-0xddffffff 64bit pref]
pci 0000:00:1c.7: PCI bridge to [bus 1e-22]
pci 0000:00:1c.7:   bridge window [io  disabled]
pci 0000:00:1c.7:   bridge window [mem 0xde000000-0xde8fffff]
pci 0000:00:1c.7:   bridge window [mem 0xdd000000-0xddffffff 64bit pref]
pci 0000:00:1e.0: PCI bridge to [bus 23-23]
pci 0000:00:1e.0:   bridge window [io  disabled]
pci 0000:00:1e.0:   bridge window [mem disabled]
pci 0000:00:1e.0:   bridge window [mem pref disabled]
pci 0000:00:01.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
pci 0000:00:01.1: PCI INT A -> GSI 26 (level, low) -> IRQ 26
pci 0000:00:02.0: PCI INT A -> GSI 33 (level, low) -> IRQ 33
pci 0000:04:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
pci 0000:05:01.0: PCI INT A -> GSI 36 (level, low) -> IRQ 36
pci 0000:05:02.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
pci 0000:05:04.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
pci 0000:05:05.0: PCI INT A -> GSI 36 (level, low) -> IRQ 36
pci 0000:05:06.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
pci 0000:05:08.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
pci 0000:05:09.0: PCI INT A -> GSI 36 (level, low) -> IRQ 36
pci 0000:05:0a.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
pci 0000:05:0b.0: PCI INT A -> GSI 38 (level, low) -> IRQ 38
pci 0000:00:02.2: PCI INT A -> GSI 33 (level, low) -> IRQ 33
pci 0000:00:03.0: PCI INT A -> GSI 41 (level, low) -> IRQ 41
pci 0000:10:00.0: PCI INT A -> GSI 40 (level, low) -> IRQ 40
pci 0000:11:01.0: PCI INT A -> GSI 44 (level, low) -> IRQ 44
pci 0000:11:02.0: PCI INT A -> GSI 45 (level, low) -> IRQ 45
pci 0000:11:04.0: PCI INT A -> GSI 40 (level, low) -> IRQ 40
pci 0000:11:05.0: PCI INT A -> GSI 44 (level, low) -> IRQ 44
pci 0000:11:06.0: PCI INT A -> GSI 45 (level, low) -> IRQ 45
pci 0000:11:08.0: PCI INT A -> GSI 40 (level, low) -> IRQ 40
pci 0000:11:09.0: PCI INT A -> GSI 44 (level, low) -> IRQ 44
pci 0000:11:0a.0: PCI INT A -> GSI 45 (level, low) -> IRQ 45
pci 0000:11:0b.0: PCI INT A -> GSI 46 (level, low) -> IRQ 46
pci 0000:00:03.2: PCI INT A -> GSI 41 (level, low) -> IRQ 41
pci 0000:00:11.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.7: PCI INT D -> GSI 19 (level, low) -> IRQ 19
pci 0000:80:02.0: PCI bridge to [bus 81-81]
pci 0000:80:02.0:   bridge window [io  disabled]
pci 0000:80:02.0:   bridge window [mem disabled]
pci 0000:80:02.0:   bridge window [mem pref disabled]
pci 0000:80:02.0: PCI INT A -> GSI 57 (level, low) -> IRQ 57
NET: Registered protocol family 2
IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
UDP hash table entries: 16384 (order: 7, 524288 bytes)
UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
NET: Registered protocol family 1
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing 64MB software IO TLB between ffff88006e0d2000 - ffff8800720d2000
software IO TLB at phys 0x6e0d2000 - 0x720d2000
audit: initializing netlink socket (disabled)
type=2000 audit(1368613882.049:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 32768
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pcieport 0000:00:01.0: ACPI _OSC control granted for 0x10
pcieport 0000:00:01.1: ACPI _OSC control granted for 0x10
pcieport 0000:00:02.0: ACPI _OSC control granted for 0x10
pcieport 0000:00:02.2: ACPI _OSC control granted for 0x10
pcieport 0000:00:03.0: ACPI _OSC control granted for 0x10
pcieport 0000:00:03.2: ACPI _OSC control granted for 0x10
pcieport 0000:00:11.0: ACPI _OSC control granted for 0x10
pcieport 0000:00:1c.0: ACPI _OSC control granted for 0x10
pcieport 0000:00:1c.7: ACPI _OSC control granted for 0x10
pcieport 0000:04:00.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:01.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:02.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:04.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:05.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:06.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:08.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:09.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:0a.0: ACPI _OSC control granted for 0x10
pcieport 0000:05:0b.0: ACPI _OSC control granted for 0x10
pcieport 0000:10:00.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:01.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:02.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:04.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:05.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:06.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:08.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:09.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:0a.0: ACPI _OSC control granted for 0x10
pcieport 0000:11:0b.0: ACPI _OSC control granted for 0x10
pcieport 0000:1e:00.0: ACPI _OSC control granted for 0x10
pcieport 0000:1f:00.0: ACPI _OSC control granted for 0x10
pcieport 0000:1f:01.0: ACPI _OSC control granted for 0x10
pcieport 0000:80:02.0: ACPI _OSC control granted for 0x10
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
input: Power Button as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
ERST: Error Record Serialization Table (ERST) support is initialized.
GHES: Generic hardware error source: 1 notified via NMI is not supported!
GHES: Generic hardware error source: 2 notified via POLL is not supported!
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
ipmi message handler version 39.2
IPMI System Interface driver.
ipmi_si: probing via ACPI
ipmi_si 00:0d: [io  0x0ca2] regsize 1 spacing 1 irq 0
ipmi_si: Adding ACPI-specified kcs state machine
ipmi_si: probing via SMBIOS
ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0
ipmi_si: Adding SMBIOS-specified kcs state machine duplicate interface
ipmi_si: probing via SPMI
ipmi_si: SPMI: io 0xca2 regsize 1 spacing 1 irq 0
ipmi_si: Adding SPMI-specified kcs state machine duplicate interface
ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi_si 00:0d: Found new BMC (man_id: 0x002a7c, prod_id: 0xaabb, dev_id: 0x20)
ipmi_si 00:0d: IPMI kcs interface initialized
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
erial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:0b: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:0c: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
ehci_hcd 0000:00:1a.0: EHCI Host Controller
ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.0: debug port 2
ehci_hcd 0000:00:1a.0: irq 16, io mem 0xdf802000
ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.37.6.RSS5K ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.0
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
ehci_hcd 0000:00:1d.0: EHCI Host Controller
ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.0: debug port 2
ehci_hcd 0000:00:1d.0: irq 23, io mem 0xdf801000
ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.37.6.RSS5K ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.0
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: PS/2 Controller [PNP0f03:PS2M] at 0x60,0x64 irq 12
PNP: PS/2 controller doesn't have KBD irq; using default 1
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:04: RTC can wake from S4
rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
md: raid10 personality registered for level 10
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
rtc_cmos 00:04: setting system clock to 2013-05-15 10:31:26 UTC (1368613886)
Initalizing network drop monitor service
Freeing unused kernel memory: 9760k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1232k freed
Freeing unused kernel memory: 1848k freed
ahci 0000:00:1f.2: PCI INT A -> GSI 18 (level, low) -> IRQ 18
ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl RAID mode
ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst 
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
scsi4 : ahci
scsi5 : ahci
ata1: SATA max UDMA/133 abar m2048@0xdf800000 port 0xdf800100 irq 110
ata2: SATA max UDMA/133 abar m2048@0xdf800000 port 0xdf800180 irq 110
usb 1-1: new high speed USB device using ehci_hcd and address 2
ata3: SATA max UDMA/133 abar m2048@0xdf800000 port 0xdf800200 irq 110
ata4: SATA max UDMA/133 abar m2048@0xdf800000 port 0xdf800280 irq 110
ata5: SATA max UDMA/133 abar m2048@0xdf800000 port 0xdf800300 irq 110
ata6: SATA max UDMA/133 abar m2048@0xdf800000 port 0xdf800380 irq 110
usb 1-1: New USB device found, idVendor=8087, idProduct=0024
usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
hub 1-1:1.0: USB hub found
hub 1-1:1.0: 6 ports detected
usb 2-1: new high speed USB device using ehci_hcd and address 2
ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata2.00: ATA-8: WDC WD2000FYYZ-01UL1B0, 01.01K01, max UDMA/133
ata2.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
usb 2-1: New USB device found, idVendor=8087, idProduct=0024
usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
hub 2-1:1.0: USB hub found
hub 2-1:1.0: 8 ports detected
ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ATA-8: INTEL SSDSA2BW120G3, 4PC10362, max UDMA/133
ata1.00: 234441648 sectors, multi 16: LBA48 NCQ (depth 31/32)
ata4.00: ATA-8: WDC WD2000FYYZ-01UL1B0, 01.01K01, max UDMA/133
ata4.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata3.00: ATA-8: WDC WD2000FYYZ-01UL1B0, 01.01K01, max UDMA/133
ata3.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
usb 1-1.3: new full speed USB device using ehci_hcd and address 3
ata1.00: configured for UDMA/133
ata5.00: ATA-8: WDC WD2000FYYZ-01UL1B0, 01.01K01, max UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      INTEL SSDSA2BW12 4PC1 PQ: 0 ANSI: 5
ata5.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata2.00: configured for UDMA/133
ata4.00: configured for UDMA/133
scsi 1:0:0:0: Direct-Access     ATA      WDC WD2000FYYZ-0 01.0 PQ: 0 ANSI: 5
ata3.00: configured for UDMA/133
ata5.00: configured for UDMA/133
scsi 2:0:0:0: Direct-Access     ATA      WDC WD2000FYYZ-0 01.0 PQ: 0 ANSI: 5
scsi 3:0:0:0: Direct-Access     ATA      WDC WD2000FYYZ-0 01.0 PQ: 0 ANSI: 5
scsi 4:0:0:0: Direct-Access     ATA      WDC WD2000FYYZ-0 01.0 PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 0:0:0:0: [sda] 234441648 512-byte logical blocks: (120 GB/111 GiB)
sd 2:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 3:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 2:0:0:0: [sdc] Write Protect is off
sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 3:0:0:0: [sdd] Write Protect is off
sd 4:0:0:0: [sde] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 4:0:0:0: [sde] Write Protect is off
sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 >
 sdc: unknown partition table
 sdd: unknown partition table
 sde: unknown partition table
sd 1:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdc] Attached SCSI disk
sd 4:0:0:0: [sde] Attached SCSI disk
sd 3:0:0:0: [sdd] Attached SCSI disk
sd 0:0:0:0: [sda] Attached SCSI disk
usb 1-1.3: New USB device found, idVendor=046b, idProduct=ff10
usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-1.3: Product: Virtual Keyboard and Mouse
usb 1-1.3: Manufacturer: American Megatrends Inc.
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.3/1-1.3:1.0/input/input2
 sdb: unknown partition table
generic-usb 0003:046B:FF10.0001: input,hidraw0: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.0-1.3/input0
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.3/1-1.3:1.1/input/input3
generic-usb 0003:046B:FF10.0002: input,hidraw1: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.0-1.3/input1
sd 1:0:0:0: [sdb] Attached SCSI disk
usb 2-1.4: new full speed USB device using ehci_hcd and address 3
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda1): mounted filesystem with ordered data mode
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda3): mounted filesystem with ordered data mode
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (loop1): mounted filesystem with ordered data mode
usb 2-1.4: New USB device found, idVendor=045e, idProduct=0745
usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 2-1.4: Product: Microsoft庐 2.4GHz Transceiver v8.0
usb 2-1.4: Manufacturer: Microsoft
input: Microsoft Microsoft庐 2.4GHz Transceiver v8.0 as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.0/input/input4
generic-usb 0003:045E:0745.0003: input,hidraw2: USB HID v1.11 Keyboard [Microsoft Microsoft庐 2.4GHz Transceiver v8.0] on usb-0000:00:1d.0-1.4/input0
input: Microsoft Microsoft庐 2.4GHz Transceiver v8.0 as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.1/input/input5
generic-usb 0003:045E:0745.0004: input,hidraw3: USB HID v1.11 Mouse [Microsoft Microsoft庐 2.4GHz Transceiver v8.0] on usb-0000:00:1d.0-1.4/input1
input: Microsoft Microsoft庐 2.4GHz Transceiver v8.0 as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.2/input/input6
generic-usb 0003:045E:0745.0005: input,hiddev0,hidraw4: USB HID v1.11 Device [Microsoft Microsoft庐 2.4GHz Transceiver v8.0] on usb-0000:00:1d.0-1.4/input2
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda2): using internal journal
EXT3-fs (sda2): recovery complete
EXT3-fs (sda2): mounted filesystem with ordered data mode
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda5): using internal journal
EXT3-fs (sda5): recovery complete
EXT3-fs (sda5): mounted filesystem with ordered data mode
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda6): using internal journal
EXT3-fs (sda6): recovery complete
EXT3-fs (sda6): mounted filesystem with ordered data mode
udevd (279): /proc/279/oom_adj is deprecated, please use /proc/279/oom_score_adj instead.
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
dca service started, version 1.12.1
ioatdma: Intel(R) QuickData Technology Driver 4.00
ioatdma 0000:00:04.0: PCI INT A -> GSI 31 (level, low) -> IRQ 31
ioatdma 0000:00:04.1: PCI INT B -> GSI 39 (level, low) -> IRQ 39
ioatdma 0000:00:04.2: PCI INT C -> GSI 31 (level, low) -> IRQ 31
ioatdma 0000:00:04.3: PCI INT D -> GSI 39 (level, low) -> IRQ 39
ioatdma 0000:00:04.4: PCI INT A -> GSI 31 (level, low) -> IRQ 31
ioatdma 0000:00:04.5: PCI INT B -> GSI 39 (level, low) -> IRQ 39
ioatdma 0000:00:04.6: PCI INT C -> GSI 31 (level, low) -> IRQ 31
ioatdma 0000:00:04.7: PCI INT D -> GSI 39 (level, low) -> IRQ 39
ioatdma 0000:80:04.0: PCI INT A -> GSI 55 (level, low) -> IRQ 55
ioatdma 0000:80:04.1: PCI INT B -> GSI 63 (level, low) -> IRQ 63
ioatdma 0000:80:04.2: PCI INT C -> GSI 55 (level, low) -> IRQ 55
ioatdma 0000:80:04.3: PCI INT D -> GSI 63 (level, low) -> IRQ 63
ioatdma 0000:80:04.4: PCI INT A -> GSI 55 (level, low) -> IRQ 55
ioatdma 0000:80:04.5: PCI INT B -> GSI 63 (level, low) -> IRQ 63
ioatdma 0000:80:04.6: PCI INT C -> GSI 55 (level, low) -> IRQ 55
ioatdma 0000:80:04.7: PCI INT D -> GSI 63 (level, low) -> IRQ 63
iTCO_vendor_support: vendor-support=0
iTCO_wdt: Intel TCO WatchDog Timer Driver v1.06
iTCO_wdt: unable to reset NO_REBOOT flag, device disabled by hardware/BIOS
Intel(R) Gigabit Ethernet Network Driver - version 2.1.0-k2
Copyright (c) 2007-2009 Intel Corporation.
igb 0000:02:00.0: PCI INT A -> GSI 27 (level, low) -> IRQ 27
igb 0000:02:00.0: 0 vfs allocated
igb 0000:02:00.0: DCA enabled
igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:02:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 00:25:90:92:b2:c8
igb 0000:02:00.0: eth0: PBA No: 104900-000
igb 0000:02:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
igb 0000:02:00.1: PCI INT B -> GSI 30 (level, low) -> IRQ 30
igb 0000:02:00.1: 0 vfs allocated
igb 0000:02:00.1: DCA enabled
igb 0000:02:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:02:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 00:25:90:92:b2:c9
igb 0000:02:00.1: eth1: PBA No: 104900-000
igb 0000:02:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 1:0:0:0: Attached scsi generic sg1 type 0
sd 2:0:0:0: Attached scsi generic sg2 type 0
sd 3:0:0:0: Attached scsi generic sg3 type 0
sd 4:0:0:0: Attached scsi generic sg4 type 0
i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
input: PC Speaker as /devices/platform/pcspkr/input/input7
Error: Driver 'pcspkr' is already registered, aborting...
isci: Intel(R) C600 SAS Controller Driver - version 1.0.0-rh
isci 0000:1c:00.0: driver configured for rev: 6 silicon
isci 0000:1c:00.0: Loading user firmware failed, using default values
isci 0000:1c:00.0: Default OEM configuration being used: 4 narrow ports, and default SAS Addresses
isci 0000:1c:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
scsi6 : isci
e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
e1000: Copyright (c) 1999-2006 Intel Corporation.
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md: md127 stopped.
md: bind<sdd>
md: bind<sde>
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md: md126 stopped.
md: bind<sdc>
md: bind<sdb>
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e783c00)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md: md125 stopped.
md: bind<sde>
md: bind<sdd>
md: raid10 personality registered for level 10
bio: create slab <bio-1> at 1
md/raid10:md125: not enough operational mirrors.
run:2401 Before md_unregister_thread(ffff88085def6740), Queue=ffff88085def6750
run:2403 md_unregister_thread(ffff88085def6740) done.
md: mddev(ffff88085e783c00) pers->run() failed ...
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
md_seq_show:6456 Try to mddev_unlock(ffff88085e783c00)
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
md_seq_show:6456 Try to mddev_unlock(ffff88085e3ae000)
md_seq_show:6456 Try to mddev_unlock(ffff880855f8d400)
md_seq_show:6456 Try to mddev_unlock(ffff88085e783c00)
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
BUG: unable to handle kernel NULL pointer dereference at           (null)
IP: [<ffffffff81049d31>] __wake_up_common+0x31/0x90
PGD 45d88a067 PUD 45d887067 PMD 0 
Oops: 0000 [#1] SMP 
last sysfs file: /sys/devices/pci0000:00/0000:00:1f.2/host1/uevent
CPU 16 
Modules linked in: raid10 tg3 e1000 isci libsas scsi_transport_sas snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc pcspkr serio_raw i2c_i801 i2c_core sg igb iTCO_wdt iTCO_vendor_support ioatdma dca acpi_pad shpchp ext3 mbcache jbd sd_mod crc_t10dif ahci libahci

Pid: 1101, comm: cat Not tainted 2.6.37.6.RSS5K #4 Supermicro X9DRW/X9DRW
RIP: 0010:[<ffffffff81049d31>]  [<ffffffff81049d31>] __wake_up_common+0x31/0x90
RSP: 0018:ffff8803ea111cd8  EFLAGS: 00010082
RAX: ffffffffffffffe8 RBX: ffff88085def6750 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff88085def6750
RBP: ffff8803ea111d18 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000286
R13: ffff88085def6758 R14: 0000000000000000 R15: 0000000000000000
FS:  00007fcac0377700(0000) GS:ffff88007dd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000045dcbb000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process cat (pid: 1101, threadinfo ffff8803ea110000, task ffff88045dd5d560)
Stack:
 0000000000000000 0000000300000001 0000000000000000 ffff88085def6750
 0000000000000286 0000000000000003 0000000000000001 0000000000000000
 ffff8803ea111d58 ffffffff8104d748 ffff8803ea111d38 ffff88085e783c00
Call Trace:
 [<ffffffff8104d748>] __wake_up+0x48/0x70
 [<ffffffff813b27e8>] md_wakeup_thread+0x28/0x30
 [<ffffffff813b3a72>] mddev_unlock+0x82/0x100
 [<ffffffff813b67bf>] md_seq_show+0x47f/0x7c0
 [<ffffffff8115dac4>] ? do_filp_open+0x264/0x620
 [<ffffffff8116da47>] seq_read+0x277/0x410
 [<ffffffff811a8a26>] proc_reg_read+0x76/0xb0
 [<ffffffff8114ee55>] vfs_read+0xc5/0x190
 [<ffffffff8114f5d1>] sys_read+0x51/0x90
 [<ffffffff8100c042>] system_call_fastpath+0x16/0x1b
Code: 41 56 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 89 75 cc 89 55 c8 4c 8d 6f 08 48 8b 57 08 41 89 cf 4d 89 c6 48 8d 42 e8 49 39 d5 <48> 8b 58 18 74 3f 48 83 eb 18 eb 0a 0f 1f 00 48 89 d8 48 8d 5a 
RIP  [<ffffffff81049d31>] __wake_up_common+0x31/0x90
 RSP <ffff8803ea111cd8>
CR2: 0000000000000000
---[ end trace 5241977f8a38fa5b ]---
md_seq_show:6456 Try to mddev_unlock(ffff88085e783c00)
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
md_seq_show:6456 Try to mddev_unlock(ffff88085e783c00)
mddev_unlock:679 Try to md_wakeup_thread(ffff88085def6740) of dev(ffff88085e783c00)
IPMI message handler: Event queue full, discarding incoming events
ata2.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen
ata2.00: failed command: READ FPDMA QUEUED
ata2.00: cmd 60/01:00:ae:88:e0/00:00:e8:00:00/40 tag 0 ncq 512 in
         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
ata2.00: status: { DRDY }
ata2: hard resetting link
ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen
ata1.00: failed command: WRITE FPDMA QUEUED
ata1.00: cmd 61/08:00:30:28:1d/00:00:05:00:00/40 tag 0 ncq 4096 out
         res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
ata1.00: status: { DRDY }
ata1: hard resetting link
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata2.00: qc timeout (cmd 0xec)
ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata2.00: revalidation failed (errno=-5)
ata2: hard resetting link
ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata1.00: qc timeout (cmd 0xec)
ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata1.00: revalidation failed (errno=-5)
ata1: hard resetting link
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata2.00: qc timeout (cmd 0xec)
ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata2.00: revalidation failed (errno=-5)
ata2: limiting SATA link speed to 3.0 Gbps
ata2: hard resetting link
ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
ata1.00: qc timeout (cmd 0xec)
ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata1.00: revalidation failed (errno=-5)
ata1: limiting SATA link speed to 1.5 Gbps
ata1: hard resetting link
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata2.00: qc timeout (cmd 0xec)
ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata2.00: revalidation failed (errno=-5)
ata2.00: disabled
ata2.00: device reported invalid CHS sector 0
ata2: hard resetting link
ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
ata2: EH complete
ata1.00: qc timeout (cmd 0xec)
ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
ata1.00: revalidation failed (errno=-5)
ata1.00: disabled
ata1.00: device reported invalid CHS sector 0
ata1: hard resetting link
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata1: EH complete

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Kernel crash at md_seq_show of drivers/md/md.c
  2013-05-15 12:11 Kernel crash at md_seq_show of drivers/md/md.c Mo, Moore
@ 2013-05-16  1:19 ` NeilBrown
  2013-05-16 12:35   ` Mo, Moore
  0 siblings, 1 reply; 3+ messages in thread
From: NeilBrown @ 2013-05-16  1:19 UTC (permalink / raw)
  To: Mo, Moore; +Cc: linux-raid@vger.kernel.org

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

On Wed, 15 May 2013 20:11:52 +0800 "Mo, Moore" <Moore.Mo@Polycom.com> wrote:

> Dear Neil,
> 
> 	Thank you for your clue. I make deeper trace after that and found another suspicion point. Kernel maybe wakeup a mdk_thread which was kfree in failed case of mddev->pers->run(mddev).
> 
> 	For make sure my guess, I add some printk in order to get more info. It seems that md_seq_show try to wake up a mdk_thread in mddev_unlock() before md_seq_show return, but the thread was kfree already by "out_free_conf" case of run() function in drivers/md/raid10.c.
> 
> 	The Oops report as attached.
> 

I don't really have time for bug reports against ancient kernels - sorry.
If you think you've found a bug, please at least look in the current mainline
kernel to see if the code has changed, and preferably reproduce against the
current mainline kernel.

In this case the bug was fixed by commit
      01f96c0a9922cd9919baf9d16febdf7016177a12
18 months ago (linux 3.1).

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* RE: Kernel crash at md_seq_show of drivers/md/md.c
  2013-05-16  1:19 ` NeilBrown
@ 2013-05-16 12:35   ` Mo, Moore
  0 siblings, 0 replies; 3+ messages in thread
From: Mo, Moore @ 2013-05-16 12:35 UTC (permalink / raw)
  To: NeilBrown; +Cc: linux-raid@vger.kernel.org

> On Wed, 15 May 2013 20:11:52 +0800 "Mo, Moore" <Moore.Mo@Polycom.com> wrote:
> 
> > Dear Neil,
> > 
> >     Thank you for your clue. I make deeper trace after that and found another suspicion point. Kernel maybe wakeup a mdk_thread which was kfree in failed case of mddev->pers->run(mddev).
> > 
> >     For make sure my guess, I add some printk in order to get more info. It seems that md_seq_show try to wake up a mdk_thread in mddev_unlock() before md_seq_show return, but the thread was kfree already by "out_free_conf" case of run() function in drivers/md/raid10.c.
> > 
> >     The Oops report as attached.
> > 
> 
> I don't really have time for bug reports against ancient kernels - sorry.
> If you think you've found a bug, please at least look in the current mainline kernel to see if the code has changed, and preferably reproduce against the current mainline kernel.
> 
> In this case the bug was fixed by commit
>       01f96c0a9922cd9919baf9d16febdf7016177a12
> 18 months ago (linux 3.1).
> 

Thanks your information and sorry about that I haven't look through latest mainline kernel this time. I will pay attention to it in future.

I have merged the patch from linux 3.1 (with a little adaptation for 2.6.37), and verified on my issue theater. it works well. The Oops disappeared. Thank you.

Could you mind I ask an extended question: There is a NULL pointer protector in md_wakeup_thread. Why not set the thread pointer to NULL instant after md_unregister_thread in fail case of run? Just like as fail case of raid5 implement in 01f96c0a9922cd9919baf9d16febdf7016177a12~1. 

-------------------------------------------------
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 43709fa..ac5e8b5 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4941,8 +4941,7 @@ static int run(mddev_t *mddev)

        return 0;
 abort:
-       md_unregister_thread(mddev->thread);
-       mddev->thread = NULL;
+       md_unregister_thread(&mddev->thread);
        if (conf) {
                print_raid5_conf(conf);
                free_conf(conf);
-------------------------------------------------

I surmise the reason is: the run() procedure is running in the context which was pointed by mddev->thead itself. Is it right?
But I have tried this way before I wrote this mail (actually before merge the commit of 3.1). It seems also work well. The "wakeup thread oops" disappeared too, and no another oops or KERN_ERR print output.

Question just for my tech interest. Excuse me if it disturb you.



Best Regards,
Moore(莫谋鑫)

^ permalink raw reply related	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2013-05-16 12:35 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-05-15 12:11 Kernel crash at md_seq_show of drivers/md/md.c Mo, Moore
2013-05-16  1:19 ` NeilBrown
2013-05-16 12:35   ` Mo, Moore

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.