linux-raid.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Recovery of RAID1 fails (added disks stays as spare)
@ 2013-08-15  9:09 Matthias.Blaesing
  2013-08-17  0:50 ` NeilBrown
  0 siblings, 1 reply; 5+ messages in thread
From: Matthias.Blaesing @ 2013-08-15  9:09 UTC (permalink / raw)
  To: linux-raid

Hello,

I'm currently fighting a server problem and have the feeling, that I'm running into walls.

Summary: On one of our servers we suffered from a hard disk error, that lead to a degraded array.
The hardware was replaced and the array was rebuild. On one of the RAID-Sets the newly added
disk is not activated but stays as spare.

System: SUSE Linux Enterprise Server 11 (x86_64) 11.2

The current state:

# cat /proc/mdstat

Personalities : [raid1] [raid0] [raid10] [raid6] [raid5] [raid4]
md3 : active raid1 sda3[2](S) sdb3[0]
      970888192 blocks [2/1] [U_]

md1 : active raid1 sda1[0] sdb1[1]
      3911680 blocks [2/2] [UU]

unused devices: <none>

# mdadm --detail /dev/md3

/dev/md3:
        Version : 0.90
  Creation Time : Fri Feb  4 11:47:04 2011
     Raid Level : raid1
     Array Size : 970888192 (925.91 GiB 994.19 GB)
  Used Dev Size : 970888192 (925.91 GiB 994.19 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 3
    Persistence : Superblock is persistent

    Update Time : Thu Aug 15 10:22:07 2013
          State : clean, degraded
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1

           UUID : e9d9c5f5:615c789e:3fb6082e:e5593158
         Events : 0.18857541

    Number   Major   Minor   RaidDevice State
       0       8       19        0      active sync   /dev/sdb3
       1       0        0        1      removed

       2       8        3        -      spare   /dev/sda3

I would expect the raid system to move /dev/sda3 to number 1 and mark it as active.

Versions:

# uname -a
Linux 3.0.58-0.6.6-default #1 SMP Tue Feb 19 11:07:00 UTC 2013 (1576ecd) x86_64 x86_64 x86_64 GNU/Linux
# mdadm -V
mdadm - v3.2.2 - 17th June 2011

I tried:

* removing /dev/sda3 from the array and add it back
* removing /dev/sda3 from the array, zero the root block and add it back (--zero-superblock)
* removing /dev/sda3 from the array, reduce raid devices to one, add /dev/sda3 back
* removing /dev/sda3 from the array, zero the first part of the disk (with dd) and add it back

I would really appreciate ideas how to fix this (preferably while running the system).


Mit freundlichen Grüßen

Matthias Bläsing
Abt IT
KompetenzCenter

Tel.: 02351 950-344
Fax.: 02351 950-222
mailto: matthias.blaesing@persona.de
www.persona.de

persona service Verwaltungs AG & Co. KG 
Freisenbergstraße 31 • 58513 Lüdenscheid  
Tel.: (02351) 950-0 • Fax: (02351) 950-222 
Sitz Lüdenscheid • Registergericht Iserlohn, HRA Nr. 2930

persönlich haftende Gesellschafterin: persona service AG
Gartenstraße 93 • CH-4002 Basel
Handelsregister Basel, Nr. CH-270.3.012.836-8
diese vertreten durch den Verwaltungsrat:
Georg Breucker (Präsident) und Dr. Sebastian Burckhardt
www.persona.de
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Recovery of RAID1 fails (added disks stays as spare)
  2013-08-15  9:09 Recovery of RAID1 fails (added disks stays as spare) Matthias.Blaesing
@ 2013-08-17  0:50 ` NeilBrown
  2013-08-19  7:48   ` Matthias.Blaesing
  0 siblings, 1 reply; 5+ messages in thread
From: NeilBrown @ 2013-08-17  0:50 UTC (permalink / raw)
  To: Matthias.Blaesing; +Cc: linux-raid

[-- Attachment #1: Type: text/plain, Size: 2763 bytes --]

On Thu, 15 Aug 2013 09:09:40 +0000 <Matthias.Blaesing@persona.de> wrote:

> Hello,
> 
> I'm currently fighting a server problem and have the feeling, that I'm running into walls.
> 
> Summary: On one of our servers we suffered from a hard disk error, that lead to a degraded array.
> The hardware was replaced and the array was rebuild. On one of the RAID-Sets the newly added
> disk is not activated but stays as spare.
> 
> System: SUSE Linux Enterprise Server 11 (x86_64) 11.2
> 
> The current state:
> 
> # cat /proc/mdstat
> 
> Personalities : [raid1] [raid0] [raid10] [raid6] [raid5] [raid4]
> md3 : active raid1 sda3[2](S) sdb3[0]
>       970888192 blocks [2/1] [U_]
> 
> md1 : active raid1 sda1[0] sdb1[1]
>       3911680 blocks [2/2] [UU]
> 
> unused devices: <none>
> 
> # mdadm --detail /dev/md3
> 
> /dev/md3:
>         Version : 0.90
>   Creation Time : Fri Feb  4 11:47:04 2011
>      Raid Level : raid1
>      Array Size : 970888192 (925.91 GiB 994.19 GB)
>   Used Dev Size : 970888192 (925.91 GiB 994.19 GB)
>    Raid Devices : 2
>   Total Devices : 2
> Preferred Minor : 3
>     Persistence : Superblock is persistent
> 
>     Update Time : Thu Aug 15 10:22:07 2013
>           State : clean, degraded
>  Active Devices : 1
> Working Devices : 2
>  Failed Devices : 0
>   Spare Devices : 1
> 
>            UUID : e9d9c5f5:615c789e:3fb6082e:e5593158
>          Events : 0.18857541
> 
>     Number   Major   Minor   RaidDevice State
>        0       8       19        0      active sync   /dev/sdb3
>        1       0        0        1      removed
> 
>        2       8        3        -      spare   /dev/sda3
> 
> I would expect the raid system to move /dev/sda3 to number 1 and mark it as active.
> 
> Versions:
> 
> # uname -a
> Linux 3.0.58-0.6.6-default #1 SMP Tue Feb 19 11:07:00 UTC 2013 (1576ecd) x86_64 x86_64 x86_64 GNU/Linux
> # mdadm -V
> mdadm - v3.2.2 - 17th June 2011
> 
> I tried:
> 
> * removing /dev/sda3 from the array and add it back
> * removing /dev/sda3 from the array, zero the root block and add it back (--zero-superblock)
> * removing /dev/sda3 from the array, reduce raid devices to one, add /dev/sda3 back
> * removing /dev/sda3 from the array, zero the first part of the disk (with dd) and add it back
> 
> I would really appreciate ideas how to fix this (preferably while running the system).
> 

Strange.  I would definitely have expected one of those to start the recovery.
Does anything appear in the kernel logs (e.g. output of 'dmesg')?
What does
  grep . /sys/block/md3/md/*
show?
I don't suppose
  echo recover > /sys/block/md3/md/sync_action
helps?
Is there still a kernel thread called
    md3_raid1
running?

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: Recovery of RAID1 fails (added disks stays as spare)
  2013-08-17  0:50 ` NeilBrown
@ 2013-08-19  7:48   ` Matthias.Blaesing
  2013-08-19  8:04     ` NeilBrown
  0 siblings, 1 reply; 5+ messages in thread
From: Matthias.Blaesing @ 2013-08-19  7:48 UTC (permalink / raw)
  To: linux-raid

[-- Attachment #1: Type: text/plain, Size: 4873 bytes --]

Hey,

> -----Original Message-----
> From: NeilBrown [mailto:neilb@suse.de]
> Sent: Saturday, August 17, 2013 2:51 AM
> To: Blaesing, Matthias (KC-IT)
> Cc: linux-raid@vger.kernel.org
> Subject: Re: Recovery of RAID1 fails (added disks stays as spare)
> 
> On Thu, 15 Aug 2013 09:09:40 +0000 <Matthias.Blaesing@persona.de> wrote:
> > Summary: On one of our servers we suffered from a hard disk error,
> that lead to a degraded array.
> > The hardware was replaced and the array was rebuild. On one of the
> RAID-Sets the newly added
> > disk is not activated but stays as spare.
> > I tried:
> >
> > * removing /dev/sda3 from the array and add it back
> > * removing /dev/sda3 from the array, zero the root block and add it
> back (--zero-superblock)
> > * removing /dev/sda3 from the array, reduce raid devices to one, add
> /dev/sda3 back
> > * removing /dev/sda3 from the array, zero the first part of the disk
> (with dd) and add it back
> >
> > I would really appreciate ideas how to fix this (preferably while
> running the system).
> >
> 
> Strange.  I would definitely have expected one of those to start the
> recovery.
> Does anything appear in the kernel logs (e.g. output of 'dmesg')?

Please see the attaches boot.msg - from my perspective it only confirms the results
gained from the /proc/mdstat and mdadm output.

> What does
>   grep . /sys/block/md3/md/*
> show?

grep . /sys/block/md3/md/*
/sys/block/md3/md/array_size:default
/sys/block/md3/md/array_state:clean
grep: /sys/block/md3/md/bitmap_set_bits: Permission denied
/sys/block/md3/md/chunk_size:0
/sys/block/md3/md/component_size:970888192
/sys/block/md3/md/degraded:1
/sys/block/md3/md/layout:0
/sys/block/md3/md/level:raid1
/sys/block/md3/md/max_read_errors:20
/sys/block/md3/md/metadata_version:0.90
/sys/block/md3/md/mismatch_cnt:0
grep: /sys/block/md3/md/new_dev: Permission denied
/sys/block/md3/md/raid_disks:2
/sys/block/md3/md/reshape_direction:forwards
/sys/block/md3/md/reshape_position:none
/sys/block/md3/md/resync_start:none
/sys/block/md3/md/safe_mode_delay:0.204
/sys/block/md3/md/suspend_hi:0
/sys/block/md3/md/suspend_lo:0
/sys/block/md3/md/sync_action:idle
/sys/block/md3/md/sync_completed:none
/sys/block/md3/md/sync_force_parallel:0
/sys/block/md3/md/sync_max:max
/sys/block/md3/md/sync_min:0
/sys/block/md3/md/sync_speed:none
/sys/block/md3/md/sync_speed_max:200000 (system)
/sys/block/md3/md/sync_speed_min:1000 (system)

> I don't suppose
>   echo recover > /sys/block/md3/md/sync_action
> helps?

No - at least it shows now detectable reaction:

No direct output, no output in dmesg, no change in the output in mdadm or /proc/mdstat 

> Is there still a kernel thread called
>     md3_raid1
> running?

Yes:

s15434194:~ # ps ax | grep md3
  786 ?        S     44:14 [md3_raid1]
  836 pts/0    S+     0:00 grep md3
s15434194:~ #


To give some more information - her the mdadm.conf:

s15434194:~ # cat /etc/mdadm.conf
CREATE owner=root group=disk mode=0660 auto=yes

MAILADDR root

DEVICE containers partitions
ARRAY /dev/md1 level=raid1 num-devices=2 devices=/dev/sda1,/dev/sdb1
ARRAY /dev/md3 level=raid1 num-devices=2 devices=/dev/sda3,/dev/sdb3


Could it be that the problem lies in the order in the devices row? When I compare the working md1 and the non-working md3 array I see a difference in the mapping of devices to numbers:

For md1:

   Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1

For md3:

    Number   Major   Minor   RaidDevice State
       0       8       19        0      active sync   /dev/sdb3
       1       0        0        1      removed

       2       8        3        -      spare   /dev/sda3


Would this be worth a try (the disks need approx. 24 hours to reach the rebuild-state, so I'm reluctant to try without some hope) to do:

1. change the Array line in /etc/mdadm.conf for md3 to read (switching order):
ARRAY /dev/md3 level=raid1 num-devices=2 devices=/dev/sdb3,/dev/sda3
2. Remove the spare sda3
3. Zero the superblock of sda3
4. Add sda3 again to the array

?

Mit freundlichen Grüßen

Matthias Bläsing
Abt IT
KompetenzCenter

Tel.: 02351 950-344
Fax.: 02351 950-222
mailto: matthias.blaesing@persona.de
www.persona.de


persona service Verwaltungs AG & Co. KG 
Freisenbergstraße 31 • 58513 Lüdenscheid  
Tel.: (02351) 950-0 • Fax: (02351) 950-222 
Sitz Lüdenscheid • Registergericht Iserlohn, HRA Nr. 2930

persönlich haftende Gesellschafterin: persona service AG
Gartenstraße 93 • CH-4002 Basel
Handelsregister Basel, Nr. CH-270.3.012.836-8
diese vertreten durch den Verwaltungsrat:
Georg Breucker (Präsident) und Dr. Sebastian Burckhardt
www.persona.de

[-- Attachment #2: boot.msg --]
[-- Type: application/octet-stream, Size: 64786 bytes --]

klogd 1.4.1, log source = ksyslog started.
<6>[    0.000000] Initializing cgroup subsys cpuset
<6>[    0.000000] Initializing cgroup subsys cpu
<5>[    0.000000] Linux version 3.0.58-0.6.6-default (geeko@buildhost) (gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux) ) #1 SMP Tue Feb 19 11:07:00 UTC 2013 (1576ecd)
<6>[    0.000000] Command line: root=/dev/md1 resume=/dev/disk/by-id/scsi-20004d927fffff800-part1 crashkernel=128M-:64M showopts console=ttyS0,57600 vga=0x31a
<6>[    0.000000] BIOS-provided physical RAM map:
<6>[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
<6>[    0.000000]  BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
<6>[    0.000000]  BIOS-e820: 00000000000d0000 - 0000000000100000 (reserved)
<6>[    0.000000]  BIOS-e820: 0000000000100000 - 00000000f5ed0000 (usable)
<6>[    0.000000]  BIOS-e820: 00000000f5ed0000 - 00000000f5ed8000 (ACPI data)
<6>[    0.000000]  BIOS-e820: 00000000f5ed8000 - 00000000f5eda000 (ACPI NVS)
<6>[    0.000000]  BIOS-e820: 00000000f5eda000 - 00000000fc000000 (reserved)
<6>[    0.000000]  BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
<6>[    0.000000]  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
<6>[    0.000000]  BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
<6>[    0.000000]  BIOS-e820: 0000000100000000 - 0000000208000000 (usable)
<6>[    0.000000] NX (Execute Disable) protection: active
<6>[    0.000000] DMI present.
<7>[    0.000000] DMI: FUJITSU                          D2981-A1                      /D2981-A1, BIOS 6.00 R1.04.2981.A1               07/13/2010
<7>[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
<7>[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
<6>[    0.000000] No AGP bridge found
<6>[    0.000000] last_pfn = 0x208000 max_arch_pfn = 0x400000000
<7>[    0.000000] MTRR default type: uncachable
<7>[    0.000000] MTRR fixed ranges enabled:
<7>[    0.000000]   00000-9FFFF write-back
<7>[    0.000000]   A0000-BFFFF uncachable
<7>[    0.000000]   C0000-D3FFF write-protect
<7>[    0.000000]   D4000-E3FFF uncachable
<7>[    0.000000]   E4000-FFFFF write-protect
<7>[    0.000000] MTRR variable ranges enabled:
<7>[    0.000000]   0 base 000000000000 mask FFFF80000000 write-back
<7>[    0.000000]   1 base 000080000000 mask FFFFC0000000 write-back
<7>[    0.000000]   2 base 0000C0000000 mask FFFFE0000000 write-back
<7>[    0.000000]   3 base 0000E0000000 mask FFFFF0000000 write-back
<7>[    0.000000]   4 base 0000F0000000 mask FFFFFC000000 write-back
<7>[    0.000000]   5 base 0000F4000000 mask FFFFFE000000 write-back
<7>[    0.000000]   6 disabled
<7>[    0.000000]   7 disabled
<7>[    0.000000] TOM2: 0000000208000000 aka 8320M
<6>[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
<7>[    0.000000] e820 update range: 00000000f6000000 - 0000000100000000 (usable) ==> (reserved)
<6>[    0.000000] last_pfn = 0xf5ed0 max_arch_pfn = 0x400000000
<6>[    0.000000] found SMP MP-table at [ffff8800000f9610] f9610
<7>[    0.000000] initial memory mapped : 0 - 20000000
<7>[    0.000000] Base memory trampoline at [ffff880000098000] 98000 size 20480
<6>[    0.000000] Using GB pages for direct mapping
<6>[    0.000000] init_memory_mapping: 0000000000000000-00000000f5ed0000
<7>[    0.000000]  0000000000 - 00c0000000 page 1G
<7>[    0.000000]  00c0000000 - 00f5e00000 page 2M
<7>[    0.000000]  00f5e00000 - 00f5ed0000 page 4k
<7>[    0.000000] kernel direct mapping tables up to 0xf5ecffff @ [mem 0x1fffd000-0x1fffffff]
<6>[    0.000000] init_memory_mapping: 0000000100000000-0000000208000000
<7>[    0.000000]  0100000000 - 0200000000 page 1G
<7>[    0.000000]  0200000000 - 0208000000 page 2M
<7>[    0.000000] kernel direct mapping tables up to 0x207ffffff @ [mem 0xf5ece000-0xf5ecffff]
<6>[    0.000000] RAMDISK: 377f1000 - 37ff0000
<6>[    0.000000] Reserving 64MB of memory at 816MB for crashkernel (System RAM: 8320MB)
<4>[    0.000000] ACPI: RSDP 00000000000f95e0 00014 (v00 PTLTD )
<4>[    0.000000] ACPI: RSDT 00000000f5ed329e 00040 (v01 PTLTD    RSDT   00060000  LTP 00000000)
<4>[    0.000000] ACPI: FACP 00000000f5ed7563 00074 (v01 FTS    D298x    00060000 AMD  000F4240)
<4>[    0.000000] ACPI: DSDT 00000000f5ed32de 04285 (v01 FSC    D2981/A1 00060000 MSFT 03000001)
<4>[    0.000000] ACPI: FACS 00000000f5ed9fc0 00040
<4>[    0.000000] ACPI: TCPA 00000000f5ed75d7 00032 (v01 Phoeni  x       00060000  TL  00000000)
<4>[    0.000000] ACPI: SSDT 00000000f5ed7609 0088C (v01 AMD    POWERNOW 00060000 AMD  00000001)
<4>[    0.000000] ACPI: ASF! 00000000f5ed7e95 0007D (v16 FTS    FTSTBL   00060000 PTL  00000001)
<4>[    0.000000] ACPI: APIC 00000000f5ed7f12 0007A (v01 PTLTD  ? APIC   00060000  LTP 00000000)
<4>[    0.000000] ACPI: MCFG 00000000f5ed7f8c 0003C (v01 PTLTD    MCFG   00060000  LTP 00000000)
<4>[    0.000000] ACPI: HPET 00000000f5ed7fc8 00038 (v01 PTLTD  HPETTBL  00060000  LTP 00000001)
<7>[    0.000000] ACPI: Local APIC address 0xfee00000
<6>[    0.000000] Scanning NUMA topology in Northbridge 24
<6>[    0.000000] No NUMA configuration found
<6>[    0.000000] Faking a node at 0000000000000000-0000000208000000
<6>[    0.000000] Initmem setup node 0 0000000000000000-0000000208000000
<6>[    0.000000]   NODE_DATA [0000000207fd9000 - 0000000207ffffff]
<7>[    0.000000]  [ffffea0000000000-ffffea00071fffff] PMD -> [ffff8801ff600000-ffff8802067fffff] on node 0
<4>[    0.000000] Zone PFN ranges:
<4>[    0.000000]   DMA      0x00000010 -> 0x00001000
<4>[    0.000000]   DMA32    0x00001000 -> 0x00100000
<4>[    0.000000]   Normal   0x00100000 -> 0x00208000
<4>[    0.000000] Movable zone start PFN for each node
<4>[    0.000000] early_node_map[3] active PFN ranges
<4>[    0.000000]     0: 0x00000010 -> 0x0000009d
<4>[    0.000000]     0: 0x00000100 -> 0x000f5ed0
<4>[    0.000000]     0: 0x00100000 -> 0x00208000
<7>[    0.000000] On node 0 totalpages: 2088541
<7>[    0.000000]   DMA zone: 56 pages used for memmap
<7>[    0.000000]   DMA zone: 5 pages reserved
<7>[    0.000000]   DMA zone: 3920 pages, LIFO batch:0
<7>[    0.000000]   DMA32 zone: 14280 pages used for memmap
<7>[    0.000000]   DMA32 zone: 988936 pages, LIFO batch:31
<7>[    0.000000]   Normal zone: 14784 pages used for memmap
<7>[    0.000000]   Normal zone: 1066560 pages, LIFO batch:31
<6>[    0.000000] ACPI: PM-Timer IO Port: 0x8008
<7>[    0.000000] ACPI: Local APIC address 0xfee00000
<6>[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
<6>[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
<6>[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
<6>[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
<6>[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
<6>[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
<6>[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
<6>[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
<6>[    0.000000] ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
<6>[    0.000000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
<6>[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 low level)
<7>[    0.000000] ACPI: IRQ0 used by override.
<7>[    0.000000] ACPI: IRQ2 used by override.
<7>[    0.000000] ACPI: IRQ9 used by override.
<6>[    0.000000] Using ACPI (MADT) for SMP configuration information
<6>[    0.000000] ACPI: HPET id: 0x43538301 base: 0xfed00000
<6>[    0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs
<7>[    0.000000] nr_irqs_gsi: 40
<6>[    0.000000] PM: Registered nosave memory: 000000000009d000 - 000000000009e000
<6>[    0.000000] PM: Registered nosave memory: 000000000009e000 - 00000000000a0000
<6>[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000d0000
<6>[    0.000000] PM: Registered nosave memory: 00000000000d0000 - 0000000000100000
<6>[    0.000000] PM: Registered nosave memory: 00000000f5ed0000 - 00000000f5ed8000
<6>[    0.000000] PM: Registered nosave memory: 00000000f5ed8000 - 00000000f5eda000
<6>[    0.000000] PM: Registered nosave memory: 00000000f5eda000 - 00000000fc000000
<6>[    0.000000] PM: Registered nosave memory: 00000000fc000000 - 00000000fec00000
<6>[    0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fec10000
<6>[    0.000000] PM: Registered nosave memory: 00000000fec10000 - 00000000fee00000
<6>[    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
<6>[    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000fff00000
<6>[    0.000000] PM: Registered nosave memory: 00000000fff00000 - 0000000100000000
<6>[    0.000000] Allocating PCI resources starting at fc000000 (gap: fc000000:2c00000)
<6>[    0.000000] Booting paravirtualized kernel on bare hardware
<6>[    0.000000] setup_percpu: NR_CPUS:4096 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
<6>[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff880207c00000 s73728 r8192 d20480 u524288
<7>[    0.000000] pcpu-alloc: s73728 r8192 d20480 u524288 alloc=1*2097152
<7>[    0.000000] pcpu-alloc: [0] 0 1 2 3 
<4>[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2059416
<4>[    0.000000] Policy zone: Normal
<5>[    0.000000] Kernel command line: root=/dev/md1 resume=/dev/disk/by-id/scsi-20004d927fffff800-part1 crashkernel=128M-:64M showopts console=ttyS0,57600 vga=0x31a
<6>[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
<6>[    0.000000] Checking aperture...
<6>[    0.000000] No AGP bridge found
<6>[    0.000000] Node 0: aperture @ 23e000000 size 32 MB
<6>[    0.000000] Aperture beyond 4GB. Ignoring.
<6>[    0.000000] Your BIOS doesn't leave a aperture memory hole
<6>[    0.000000] Please enable the IOMMU option in the BIOS setup
<6>[    0.000000] This costs you 64 MB of RAM
<6>[    0.000000] Mapping aperture over 65536 KB of RAM @ ec000000
<6>[    0.000000] PM: Registered nosave memory: 00000000ec000000 - 00000000f0000000
<6>[    0.000000] Memory: 8015052k/8519680k available (4419k kernel code, 165516k absent, 339112k reserved, 7676k data, 1344k init)
<6>[    0.000000] Hierarchical RCU implementation.
<6>[    0.000000] 	RCU dyntick-idle grace-period acceleration is enabled.
<6>[    0.000000] NR_IRQS:262400 nr_irqs:712 16
<6>[    0.000000] Console: colour dummy device 80x25
<6>[    0.000000] console [ttyS0] enabled
<6>[    0.000000] allocated 67108864 bytes of page_cgroup
<6>[    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
<7>[    0.000000] hpet clockevent registered
<4>[    0.000000] Fast TSC calibration using PIT
<4>[    0.000000] Detected 2594.573 MHz processor.
<6>[    0.012004] Calibrating delay loop (skipped), value calculated using timer frequency.. 5189.14 BogoMIPS (lpj=10378292)
<6>[    0.020003] pid_max: default: 32768 minimum: 301
<4>[    0.024358] kdb version 4.4 by Keith Owens, Scott Lurndal. Copyright SGI, All Rights Reserved
<6>[    0.044209] Security Framework initialized
<6>[    0.048021] AppArmor: AppArmor initialized
<6>[    0.052526] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
<6>[    0.061780] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
<6>[    0.065204] Mount-cache hash table entries: 256
<6>[    0.068158] Initializing cgroup subsys cpuacct
<6>[    0.072008] Initializing cgroup subsys memory
<6>[    0.076021] Initializing cgroup subsys devices
<6>[    0.080007] Initializing cgroup subsys freezer
<6>[    0.084006] Initializing cgroup subsys net_cls
<6>[    0.088007] Initializing cgroup subsys blkio
<6>[    0.092011] Initializing cgroup subsys perf_event
<7>[    0.096061] tseg: 00f5f00000
<6>[    0.096063] CPU: Physical Processor ID: 0
<6>[    0.100007] CPU: Processor Core ID: 0
<6>[    0.104008] mce: CPU supports 6 MCE banks
<6>[    0.112332] ACPI: Core revision 20110413
<6>[    0.120559] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
<6>[    0.163714] CPU0: AMD Phenom(tm) II X4 910e Processor stepping 03
<6>[    0.172010] Performance Events: AMD PMU driver.
<6>[    0.176012] ... version:                0
<6>[    0.180011] ... bit width:              48
<6>[    0.184011] ... generic registers:      4
<6>[    0.188012] ... value mask:             0000ffffffffffff
<6>[    0.192012] ... max period:             00007fffffffffff
<6>[    0.196012] ... fixed-purpose events:   0
<6>[    0.200012] ... event mask:             000000000000000f
<6>[    0.204118] NMI watchdog enabled, takes one hw-pmu counter.
<6>[    0.208130] Booting Node   0, Processors  #1
<7>[    0.216014] smpboot cpu 1: start_ip = 98000
<6>[    0.228035] NMI watchdog enabled, takes one hw-pmu counter.
<4>[    0.232104]  #2
<7>[    0.236015] smpboot cpu 2: start_ip = 98000
<6>[    0.252028] NMI watchdog enabled, takes one hw-pmu counter.
<4>[    0.256105]  #3 Ok.
<7>[    0.260017] smpboot cpu 3: start_ip = 98000
<6>[    0.276030] NMI watchdog enabled, takes one hw-pmu counter.
<6>[    0.280041] Brought up 4 CPUs
<6>[    0.284019] Total of 4 processors activated (20756.58 BogoMIPS).
<6>[    0.292037] devtmpfs: initialized
<6>[    0.303012] PM: Registering ACPI NVS region at f5ed8000 (8192 bytes)
<6>[    0.304163] print_constraints: dummy: 
<6>[    0.308044] Time: 11:52:54  Date: 07/25/13
<6>[    0.312112] NET: Registered protocol family 16
<7>[    0.316102] node 0 link 0: io port [1000, ffff]
<6>[    0.316104] TOM: 00000000f8000000 aka 3968M
<7>[    0.320022] Fam 10h mmconf [mem 0xf8000000-0xfbffffff]
<7>[    0.320025] node 0 link 0: mmio [a0000, bffff]
<7>[    0.320027] node 0 link 0: mmio [f8000000, fe1fffff] ==> [fc000000, fe1fffff]
<7>[    0.320030] node 0 link 0: mmio [fe200000, ffdfffff]
<6>[    0.320032] TOM2: 0000000208000000 aka 8320M
<7>[    0.324021] bus: [00, ff] on node 0 link 0
<7>[    0.324023] bus: 00 index 0 [io  0x0000-0xffff]
<7>[    0.324025] bus: 00 index 1 [mem 0x000a0000-0x000bffff]
<7>[    0.324026] bus: 00 index 2 [mem 0xfc000000-0xffffffff]
<7>[    0.324028] bus: 00 index 3 [mem 0x208000000-0xfcffffffff]
<6>[    0.324110] ACPI: bus type pci registered
<6>[    0.328065] PCI: MMCONFIG for domain 0000 [bus 00-03] at [mem 0xf8000000-0xf83fffff] (base 0xf8000000)
<6>[    0.332022] PCI: MMCONFIG at [mem 0xf8000000-0xf83fffff] reserved in E820
<6>[    0.336321] PCI: Using configuration type 1 for base access
<6>[    0.340522] bio: create slab <bio-0> at 0
<7>[    0.344813] ACPI: EC: Look up EC in DSDT
<6>[    0.349955] ACPI: Interpreter enabled
<6>[    0.352025] ACPI: (supports S0 S3 S4 S5)
<6>[    0.361081] ACPI: Using IOAPIC for interrupt routing
<6>[    0.371754] ACPI: No dock devices found.
<6>[    0.372027] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
<6>[    0.376069] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
<6>[    0.380110] pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
<6>[    0.384025] pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff]
<6>[    0.388025] pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
<6>[    0.392025] pci_root PNP0A03:00: host bridge window [mem 0x000c0000-0x000dffff]
<6>[    0.396025] pci_root PNP0A03:00: host bridge window [mem 0xf8000000-0xfebfffff]
<6>[    0.400026] pci_root PNP0A03:00: host bridge window [mem 0xfed00000-0xfedfffff]
<3>[    0.404029] pci_root PNP0A03:00: address space collision: host bridge window [mem 0x000c0000-0x000dffff] conflicts with reserved [mem 0x000d0000-0x000fffff]
<7>[    0.408036] pci 0000:00:00.0: [1022:9601] type 0 class 0x000600
<7>[    0.408079] pci 0000:00:01.0: [1734:9602] type 1 class 0x000604
<7>[    0.408183] pci 0000:00:0a.0: [1022:9609] type 1 class 0x000604
<7>[    0.408212] pci 0000:00:0a.0: PME# supported from D0 D3hot D3cold
<7>[    0.408215] pci 0000:00:0a.0: PME# disabled
<7>[    0.408311] pci 0000:00:11.0: [1002:4391] type 0 class 0x000106
<7>[    0.408329] pci 0000:00:11.0: reg 10: [io  0x8420-0x8427]
<7>[    0.408338] pci 0000:00:11.0: reg 14: [io  0x8414-0x8417]
<7>[    0.408347] pci 0000:00:11.0: reg 18: [io  0x8418-0x841f]
<7>[    0.408356] pci 0000:00:11.0: reg 1c: [io  0x8410-0x8413]
<7>[    0.408365] pci 0000:00:11.0: reg 20: [io  0x8400-0x840f]
<7>[    0.408375] pci 0000:00:11.0: reg 24: [mem 0xfe300000-0xfe3003ff]
<7>[    0.408435] pci 0000:00:14.0: [1002:4385] type 0 class 0x000c05
<7>[    0.408531] pci 0000:00:14.3: [1002:439d] type 0 class 0x000601
<7>[    0.408603] pci 0000:00:14.4: [1002:4384] type 1 class 0x000604
<7>[    0.408648] pci 0000:00:18.0: [1022:1200] type 0 class 0x000600
<7>[    0.408662] pci 0000:00:18.1: [1022:1201] type 0 class 0x000600
<7>[    0.408673] pci 0000:00:18.2: [1022:1202] type 0 class 0x000600
<7>[    0.408685] pci 0000:00:18.3: [1022:1203] type 0 class 0x000600
<7>[    0.408699] pci 0000:00:18.4: [1022:1204] type 0 class 0x000600
<7>[    0.408802] pci 0000:01:05.0: [1002:9710] type 0 class 0x000300
<7>[    0.408809] pci 0000:01:05.0: reg 10: [mem 0xfc000000-0xfdffffff pref]
<7>[    0.408813] pci 0000:01:05.0: reg 14: [io  0x9000-0x90ff]
<7>[    0.408817] pci 0000:01:05.0: reg 18: [mem 0xfe100000-0xfe10ffff]
<7>[    0.408826] pci 0000:01:05.0: reg 24: [mem 0xfe000000-0xfe0fffff]
<7>[    0.408841] pci 0000:01:05.0: supports D1 D2
<7>[    0.408850] pci 0000:01:05.1: [1002:970f] type 0 class 0x000403
<7>[    0.408857] pci 0000:01:05.1: reg 10: [mem 0xfe110000-0xfe113fff]
<7>[    0.408883] pci 0000:01:05.1: supports D1 D2
<6>[    0.408982] pci 0000:00:01.0: PCI bridge to [bus 01-01]
<7>[    0.412028] pci 0000:00:01.0:   bridge window [io  0x9000-0x9fff]
<7>[    0.412031] pci 0000:00:01.0:   bridge window [mem 0xfe000000-0xfe1fffff]
<7>[    0.412034] pci 0000:00:01.0:   bridge window [mem 0xfc000000-0xfdffffff 64bit pref]
<7>[    0.412072] pci 0000:02:00.0: [14e4:1692] type 0 class 0x000200
<7>[    0.412089] pci 0000:02:00.0: reg 10: [mem 0xfe200000-0xfe20ffff 64bit]
<7>[    0.412176] pci 0000:02:00.0: PME# supported from D3hot D3cold
<7>[    0.412180] pci 0000:02:00.0: PME# disabled
<6>[    0.420045] pci 0000:00:0a.0: PCI bridge to [bus 02-02]
<7>[    0.424030] pci 0000:00:0a.0:   bridge window [io  0xf000-0x0000] (disabled)
<7>[    0.424033] pci 0000:00:0a.0:   bridge window [mem 0xfe200000-0xfe2fffff]
<7>[    0.424037] pci 0000:00:0a.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
<6>[    0.424099] pci 0000:00:14.4: PCI bridge to [bus 03-03] (subtractive decode)
<7>[    0.428030] pci 0000:00:14.4:   bridge window [io  0xf000-0x0000] (disabled)
<7>[    0.428034] pci 0000:00:14.4:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
<7>[    0.428038] pci 0000:00:14.4:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
<7>[    0.428040] pci 0000:00:14.4:   bridge window [io  0x0000-0x0cf7] (subtractive decode)
<7>[    0.428043] pci 0000:00:14.4:   bridge window [io  0x0d00-0xffff] (subtractive decode)
<7>[    0.428045] pci 0000:00:14.4:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
<7>[    0.428047] pci 0000:00:14.4:   bridge window [mem 0xf8000000-0xfebfffff] (subtractive decode)
<7>[    0.428049] pci 0000:00:14.4:   bridge window [mem 0xfed00000-0xfedfffff] (subtractive decode)
<7>[    0.428063] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
<7>[    0.428171] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GRFB._PRT]
<7>[    0.428206] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PB10._PRT]
<7>[    0.428248] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PB10.LAN_._PRT]
<7>[    0.428270] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P2P_._PRT]
<6>[    0.428355]  pci0000:00: Requesting ACPI _OSC control (0x1d)
<6>[    0.432029]  pci0000:00: ACPI _OSC request failed (AE_NOT_FOUND), returned control mask: 0x1d
<6>[    0.436027] ACPI _OSC control for PCIe not granted, disabling ASPM
<6>[    0.442473] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.452066] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.464064] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.476064] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.488065] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.500420] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.512421] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.524422] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 7 9 10 11 12 14 15) *0, disabled.
<6>[    0.536450] vgaarb: device added: PCI:0000:01:05.0,decodes=io+mem,owns=io+mem,locks=none
<6>[    0.540035] vgaarb: loaded
<6>[    0.544091] PCI: Using ACPI for IRQ routing
<7>[    0.548036] PCI: pci_cache_line_size set to 64 bytes
<4>[    0.548071] Expanded resource reserved due to conflict with PCI Bus 0000:00
<7>[    0.552035] reserve RAM buffer: 000000000009dc00 - 000000000009ffff 
<7>[    0.552038] reserve RAM buffer: 00000000f5ed0000 - 00000000f7ffffff 
<6>[    0.552110] NetLabel: Initializing
<6>[    0.556035] NetLabel:  domain hash size = 128
<6>[    0.560035] NetLabel:  protocols = UNLABELED CIPSOv4
<6>[    0.564044] NetLabel:  unlabeled traffic allowed by default
<6>[    0.568043] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
<6>[    0.574331] hpet0: 4 comparators, 32-bit 14.318180 MHz counter
<6>[    0.581055] Switching to clocksource hpet
<6>[    0.584741] Switched to NOHz mode on CPU #2
<6>[    0.586757] Switched to NOHz mode on CPU #1
<6>[    0.586910] Switched to NOHz mode on CPU #0
<6>[    0.587247] Switched to NOHz mode on CPU #3
<6>[    0.618067] AppArmor: AppArmor Filesystem Enabled
<6>[    0.627456] pnp: PnP ACPI init
<6>[    0.633545] ACPI: bus type pnp registered
<7>[    0.641584] pnp 00:00: [bus 00-ff]
<7>[    0.641586] pnp 00:00: [io  0x0000-0x0cf7 window]
<7>[    0.641588] pnp 00:00: [io  0x0d00-0xffff window]
<7>[    0.641590] pnp 00:00: [io  0x0cf8-0x0cff]
<7>[    0.641592] pnp 00:00: [mem 0x000a0000-0x000bffff window]
<7>[    0.641594] pnp 00:00: [mem 0x000c0000-0x000dffff window]
<7>[    0.641596] pnp 00:00: [mem 0xf8000000-0xfebfffff window]
<7>[    0.641598] pnp 00:00: [mem 0xfed00000-0xfedfffff window]
<7>[    0.641640] pnp 00:00: Plug and Play ACPI device, IDs PNP0a03 (active)
<7>[    0.641933] pnp 00:01: [io  0x0010-0x001f]
<7>[    0.641935] pnp 00:01: [io  0x0022-0x002d]
<7>[    0.641936] pnp 00:01: [io  0x0030-0x003f]
<7>[    0.641938] pnp 00:01: [io  0x0050-0x0053]
<7>[    0.641939] pnp 00:01: [io  0x0062-0x0063]
<7>[    0.641941] pnp 00:01: [io  0x0065-0x006f]
<7>[    0.641943] pnp 00:01: [io  0x0074-0x007f]
<7>[    0.641944] pnp 00:01: [io  0x0090-0x009f]
<7>[    0.641946] pnp 00:01: [io  0x00a2-0x00b1]
<7>[    0.641947] pnp 00:01: [io  0x00b2-0x00b3]
<7>[    0.641949] pnp 00:01: [io  0x00b4-0x00bf]
<7>[    0.641950] pnp 00:01: [io  0x00ec-0x00ef]
<7>[    0.641952] pnp 00:01: [io  0x0072-0x0073]
<7>[    0.641953] pnp 00:01: [io  0x004e-0x004f]
<7>[    0.641955] pnp 00:01: [io  0x0500-0x057f]
<7>[    0.641956] pnp 00:01: [io  0x0580-0x05ff]
<7>[    0.641958] pnp 00:01: [io  0x040b]
<7>[    0.641959] pnp 00:01: [io  0x04d0-0x04d1]
<7>[    0.641961] pnp 00:01: [io  0x04d6]
<7>[    0.641963] pnp 00:01: [io  0x087f]
<7>[    0.641964] pnp 00:01: [io  0x0c00-0x0c01]
<7>[    0.641966] pnp 00:01: [io  0x0c14]
<7>[    0.641967] pnp 00:01: [io  0x0c50-0x0c52]
<7>[    0.641969] pnp 00:01: [io  0x0c6c]
<7>[    0.641970] pnp 00:01: [io  0x0c6f]
<7>[    0.641972] pnp 00:01: [io  0x0cd0-0x0cd1]
<7>[    0.641973] pnp 00:01: [io  0x0cd2-0x0cd3]
<7>[    0.641975] pnp 00:01: [io  0x0cd4-0x0cd5]
<7>[    0.641976] pnp 00:01: [io  0x0cd6-0x0cd7]
<7>[    0.641978] pnp 00:01: [io  0x0cd8-0x0cdf]
<7>[    0.641979] pnp 00:01: [io  0x0f40-0x0f7f]
<7>[    0.641981] pnp 00:01: [io  0x8000-0x807f]
<7>[    0.641982] pnp 00:01: [io  0x8100-0x81fe]
<7>[    0.641984] pnp 00:01: [io  0x8200-0x82fe]
<7>[    0.641985] pnp 00:01: [io  0xff00]
<7>[    0.641987] pnp 00:01: [mem 0xf8000000-0xfbffffff]
<7>[    0.641989] pnp 00:01: [mem 0xfec00000-0xfecfffff]
<7>[    0.641991] pnp 00:01: [mem 0xfee00000-0xfeefffff]
<6>[    0.642050] system 00:01: [io  0x0500-0x057f] has been reserved
<6>[    0.653905] system 00:01: [io  0x0580-0x05ff] has been reserved
<6>[    0.665694] system 00:01: [io  0x040b] has been reserved
<6>[    0.676270] system 00:01: [io  0x04d0-0x04d1] has been reserved
<6>[    0.688058] system 00:01: [io  0x04d6] has been reserved
<6>[    0.698635] system 00:01: [io  0x087f] has been reserved
<6>[    0.709211] system 00:01: [io  0x0c00-0x0c01] has been reserved
<6>[    0.720998] system 00:01: [io  0x0c14] has been reserved
<6>[    0.731574] system 00:01: [io  0x0c50-0x0c52] has been reserved
<6>[    0.743364] system 00:01: [io  0x0c6c] has been reserved
<6>[    0.753941] system 00:01: [io  0x0c6f] has been reserved
<6>[    0.764516] system 00:01: [io  0x0cd0-0x0cd1] has been reserved
<6>[    0.776304] system 00:01: [io  0x0cd2-0x0cd3] has been reserved
<6>[    0.788091] system 00:01: [io  0x0cd4-0x0cd5] has been reserved
<6>[    0.799879] system 00:01: [io  0x0cd6-0x0cd7] has been reserved
<6>[    0.811668] system 00:01: [io  0x0cd8-0x0cdf] has been reserved
<6>[    0.823456] system 00:01: [io  0x0f40-0x0f7f] has been reserved
<6>[    0.835245] system 00:01: [io  0x8000-0x807f] has been reserved
<6>[    0.847032] system 00:01: [io  0x8100-0x81fe] has been reserved
<6>[    0.858820] system 00:01: [io  0x8200-0x82fe] has been reserved
<6>[    0.870606] system 00:01: [io  0xff00] has been reserved
<6>[    0.881182] system 00:01: [mem 0xf8000000-0xfbffffff] has been reserved
<6>[    0.894357] system 00:01: [mem 0xfec00000-0xfecfffff] could not be reserved
<6>[    0.908222] system 00:01: [mem 0xfee00000-0xfeefffff] could not be reserved
<7>[    0.922089] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
<7>[    0.922099] pnp 00:02: [io  0x0000-0x000f]
<7>[    0.922101] pnp 00:02: [io  0x0080-0x008f]
<7>[    0.922103] pnp 00:02: [io  0x00c0-0x00df]
<7>[    0.922104] pnp 00:02: [dma 4]
<7>[    0.922133] pnp 00:02: Plug and Play ACPI device, IDs PNP0200 (active)
<7>[    0.922145] pnp 00:03: [io  0x0070-0x0071]
<7>[    0.922216] pnp 00:03: [irq 8]
<7>[    0.922246] pnp 00:03: Plug and Play ACPI device, IDs PNP0b00 (active)
<7>[    0.922257] pnp 00:04: [io  0x00f0-0x00fe]
<7>[    0.922333] pnp 00:04: [irq 13]
<7>[    0.922362] pnp 00:04: Plug and Play ACPI device, IDs PNP0c04 (active)
<7>[    0.922371] pnp 00:05: [io  0x0061]
<7>[    0.922400] pnp 00:05: Plug and Play ACPI device, IDs PNP0800 (active)
<7>[    0.922496] pnp 00:06: [mem 0xfed00000-0xfed00fff]
<7>[    0.922527] pnp 00:06: Plug and Play ACPI device, IDs PNP0103 (active)
<7>[    0.922562] pnp 00:07: [io  0x0060]
<7>[    0.922564] pnp 00:07: [io  0x0064]
<7>[    0.922633] pnp 00:07: [irq 1]
<7>[    0.922672] system 00:07: Plug and Play ACPI device, IDs PNP0c02 (active)
<7>[    0.923009] pnp 00:08: [io  0x03f8-0x03ff]
<7>[    0.923013] pnp 00:08: [irq 4]
<7>[    0.923065] pnp 00:08: Plug and Play ACPI device, IDs PNP0501 (active)
<6>[    0.923166] pnp: PnP ACPI: found 9 devices
<6>[    0.931321] ACPI: ACPI bus type pnp unregistered
<7>[    0.946831] PCI: max bus depth: 1 pci_try_num: 2
<6>[    0.946844] pci 0000:00:01.0: PCI bridge to [bus 01-01]
<6>[    0.957255] pci 0000:00:01.0:   bridge window [io  0x9000-0x9fff]
<6>[    0.969396] pci 0000:00:01.0:   bridge window [mem 0xfe000000-0xfe1fffff]
<6>[    0.982919] pci 0000:00:01.0:   bridge window [mem 0xfc000000-0xfdffffff 64bit pref]
<6>[    0.998349] pci 0000:00:0a.0: PCI bridge to [bus 02-02]
<6>[    1.008757] pci 0000:00:0a.0:   bridge window [io  disabled]
<6>[    1.020033] pci 0000:00:0a.0:   bridge window [mem 0xfe200000-0xfe2fffff]
<6>[    1.033556] pci 0000:00:0a.0:   bridge window [mem pref disabled]
<6>[    1.045697] pci 0000:00:14.4: PCI bridge to [bus 03-03]
<6>[    1.056102] pci 0000:00:14.4:   bridge window [io  disabled]
<6>[    1.067380] pci 0000:00:14.4:   bridge window [mem disabled]
<6>[    1.078655] pci 0000:00:14.4:   bridge window [mem pref disabled]
<6>[    1.090814] pci 0000:00:0a.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
<7>[    1.104165] pci 0000:00:0a.0: setting latency timer to 64
<7>[    1.104172] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
<7>[    1.104174] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
<7>[    1.104176] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
<7>[    1.104178] pci_bus 0000:00: resource 7 [mem 0xf8000000-0xfebfffff]
<7>[    1.104180] pci_bus 0000:00: resource 8 [mem 0xfed00000-0xfedfffff]
<7>[    1.104182] pci_bus 0000:01: resource 0 [io  0x9000-0x9fff]
<7>[    1.104184] pci_bus 0000:01: resource 1 [mem 0xfe000000-0xfe1fffff]
<7>[    1.104186] pci_bus 0000:01: resource 2 [mem 0xfc000000-0xfdffffff 64bit pref]
<7>[    1.104188] pci_bus 0000:02: resource 1 [mem 0xfe200000-0xfe2fffff]
<7>[    1.104190] pci_bus 0000:03: resource 4 [io  0x0000-0x0cf7]
<7>[    1.104192] pci_bus 0000:03: resource 5 [io  0x0d00-0xffff]
<7>[    1.104194] pci_bus 0000:03: resource 6 [mem 0x000a0000-0x000bffff]
<7>[    1.104196] pci_bus 0000:03: resource 7 [mem 0xf8000000-0xfebfffff]
<7>[    1.104198] pci_bus 0000:03: resource 8 [mem 0xfed00000-0xfedfffff]
<6>[    1.104418] NET: Registered protocol family 2
<6>[    1.113304] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
<6>[    1.129102] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
<6>[    1.146395] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
<6>[    1.160102] TCP: Hash tables configured (established 524288 bind 65536)
<6>[    1.173345] TCP reno registered
<6>[    1.179611] UDP hash table entries: 4096 (order: 5, 131072 bytes)
<6>[    1.191803] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
<6>[    1.205145] NET: Registered protocol family 1
<4>[    1.213830] pci 0000:00:01.0: MSI quirk detected; subordinate MSI disabled
<7>[    1.227551] pci 0000:01:05.0: Boot video device
<7>[    1.227557] PCI: CLS 32 bytes, default 64
<6>[    1.227599] Unpacking initramfs...
<6>[    1.363488] Freeing initrd memory: 8188k freed
<6>[    1.375079] PCI-DMA: Disabling AGP.
<6>[    1.384786] PCI-DMA: aperture base @ ec000000 size 65536 KB
<6>[    1.395884] PCI-DMA: using GART IOMMU.
<6>[    1.403350] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
<6>[    1.420321] audit: initializing netlink socket (disabled)
<5>[    1.431097] type=2000 audit(1374753173.424:1): initialized
<6>[    1.460000] HugeTLB registered 2 MB page size, pre-allocated 0 pages
<5>[    1.474048] VFS: Disk quotas dquot_6.5.2
<4>[    1.481981] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
<6>[    1.494982] msgmni has been set to 3949
<6>[    1.502843] alg: No test for stdrng (krng)
<6>[    1.511080] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
<6>[    1.525878] io scheduler noop registered
<6>[    1.533684] io scheduler deadline registered
<6>[    1.542244] io scheduler cfq registered (default)
<7>[    1.551827] pcieport 0000:00:0a.0: setting latency timer to 64
<7>[    1.551853] pcieport 0000:00:0a.0: irq 40 for MSI/MSI-X
<6>[    1.552034] vesafb: mode is 1280x1024x16, linelength=2560, pages=5
<6>[    1.564417] vesafb: scrolling: redraw
<6>[    1.571707] vesafb: Truecolor: size=0:5:6:5, shift=0:11:5:0
<6>[    1.583203] vesafb: framebuffer at 0xfc000000, mapped to 0xffffc90001c00000, using 5120k, total 16384k
<6>[    1.685921] Console: switching to colour frame buffer device 160x64
<6>[    1.782477] fb0: VESA VGA frame buffer device
<6>[    1.791227] Serial: 8250/16550 driver, 8 ports, IRQ sharing disabled
<6>[    1.804046] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
<6>[    1.921174] 00:08: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
<6>[    1.936778] Non-volatile memory driver v1.3
<6>[    1.945177] Linux agpgart interface v0.103
<6>[    1.953371] Fixed MDIO Bus: probed
<6>[    1.960243] i8042: PNP: No PS/2 controller found. Probing ports directly.
<6>[    1.977474] serio: i8042 KBD port at 0x60,0x64 irq 1
<6>[    1.987370] serio: i8042 AUX port at 0x60,0x64 irq 12
<6>[    1.997494] mousedev: PS/2 mouse device common for all mice
<6>[    2.008739] cpuidle: using governor ladder
<6>[    2.016897] cpuidle: using governor menu
<6>[    2.024711] EFI Variables Facility v0.08 2004-May-17
<6>[    2.034809] TCP cubic registered
<5>[    2.041313] Registering the dns_resolver key type
<7>[    2.050753] PM: Checking hibernation image partition /dev/disk/by-id/scsi-20004d927fffff800-part1
<7>[    2.061795] PM: Hibernation image not present or could not be loaded.
<6>[    2.061816] registered taskstats version 1
<6>[    2.070518]   Magic number: 5:390:884
<6>[    2.079903] Freeing unused kernel memory: 1344k freed
<6>[    2.090317] Write protecting the kernel read-only data: 10240k
<6>[    2.106631] Freeing unused kernel memory: 1708k freed
<6>[    2.119973] Freeing unused kernel memory: 1008k freed
<5>[    2.173306] SCSI subsystem initialized
<7>[    2.182458] libata version 3.00 loaded.
<7>[    2.185640] ahci 0000:00:11.0: version 3.0
<6>[    2.185662] ahci 0000:00:11.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
<6>[    2.199285] ahci 0000:00:11.0: AHCI 0001.0100 32 slots 4 ports 3 Gbps 0xf impl SATA mode
<6>[    2.215403] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pmp pio slum part ccc 
<6>[    2.232021] scsi0 : ahci
<6>[    2.237201] scsi1 : ahci
<6>[    2.242316] scsi2 : ahci
<6>[    2.247429] scsi3 : ahci
<6>[    2.252636] ata1: SATA max UDMA/133 abar m1024@0xfe300000 port 0xfe300100 irq 22
<6>[    2.267369] ata2: SATA max UDMA/133 abar m1024@0xfe300000 port 0xfe300180 irq 22
<6>[    2.282098] ata3: SATA max UDMA/133 abar m1024@0xfe300000 port 0xfe300200 irq 22
<6>[    2.296828] ata4: SATA max UDMA/133 abar m1024@0xfe300000 port 0xfe300280 irq 22
<6>[    2.456097] Refined TSC clocksource calibration: 2594.453 MHz.
<6>[    2.467718] Switching to clocksource tsc
<6>[    2.628128] ata3: SATA link down (SStatus 0 SControl 300)
<6>[    2.638921] ata4: SATA link down (SStatus 0 SControl 300)
<6>[    2.804078] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<6>[    2.816413] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<6>[    2.828935] ata2.00: ATA-8: ST31000520AS, CC32, max UDMA/133
<6>[    2.840211] ata2.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32)
<6>[    2.854257] ata1.00: ATA-8: ST1000DM003-9YN162, CC4C, max UDMA/133
<6>[    2.866570] ata1.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
<6>[    2.881119] ata2.00: configured for UDMA/133
<6>[    2.889944] ata1.00: configured for UDMA/133
<5>[    2.898568] scsi 0:0:0:0: Direct-Access     ATA      ST1000DM003-9YN1 CC4C PQ: 0 ANSI: 5
<5>[    2.914877] scsi 1:0:0:0: Direct-Access     ATA      ST31000520AS     CC32 PQ: 0 ANSI: 5
<7>[    2.937022] ACPI: acpi_idle registered with cpuidle
<6>[    2.968449] BIOS EDD facility v0.16 2004-Jun-25, 2 devices found
<4>[    2.984330] 3ware Storage Controller device driver for Linux v1.26.02.003.
<4>[    3.002003] 3ware 9000 Storage Controller device driver for Linux v2.26.02.014.
<6>[    3.035930] md: raid1 personality registered for level 1
<5>[    3.050851] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
<5>[    3.050866] sd 1:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
<5>[    3.050897] sd 1:0:0:0: [sdb] Write Protect is off
<7>[    3.050899] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
<5>[    3.050912] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
<6>[    3.070480]  sdb: sdb1 sdb2 sdb3
<5>[    3.070734] sd 1:0:0:0: [sdb] Attached SCSI disk
<5>[    3.124531] sd 0:0:0:0: [sda] 4096-byte physical blocks
<5>[    3.135004] sd 0:0:0:0: [sda] Write Protect is off
<7>[    3.144546] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
<5>[    3.144590] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
<6>[    3.182456]  sda: sda1 sda2 sda3
<5>[    3.189114] sd 0:0:0:0: [sda] Attached SCSI disk
<6>[    3.202861] hp_sw: device handler registered
<6>[    3.215229] alua: device handler registered
<6>[    3.227403] emc: device handler registered
<6>[    3.239425] rdac: device handler registered
<6>[    3.250673] udev: starting version 147
<6>[    3.523321] [drm] Initialized drm 1.1.0 20060810
<6>[    3.538083] [drm] radeon defaulting to kernel modesetting.
<6>[    3.549396] [drm] radeon kernel modesetting enabled.
<7>[    3.559331] checking generic (fc000000 1000000) vs hw (fc000000 2000000)
<6>[    3.559334] fb: conflicting fb hw usage radeondrmfb vs VESA VGA - removing generic driver
<6>[    3.578349] Console: switching to colour dummy device 80x25
<6>[    3.592456] radeon 0000:01:05.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
<7>[    3.608831] radeon 0000:01:05.0: setting latency timer to 64
<6>[    3.608956] [drm] initializing kernel modesetting (RS880 0x1002:0x9710 0x1734:0x1199).
<6>[    3.627408] [drm] register mmio base: 0xFE100000
<6>[    3.639327] [drm] register mmio size: 65536
<6>[    3.647719] ATOM BIOS: Fuj_AM3
<6>[    3.656515] radeon 0000:01:05.0: VRAM: 32M 0x00000000C0000000 - 0x00000000C1FFFFFF (32M used)
<6>[    3.673649] radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF
<6>[    3.691022] [drm] Detected VRAM RAM=32M, BAR=32M
<6>[    3.700245] [drm] RAM width 32bits DDR
<6>[    3.707787] [TTM] Zone  kernel: Available graphics memory: 4046626 kiB.
<6>[    3.720959] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB.
<6>[    3.734127] [TTM] Initializing pool allocator.
<6>[    3.742991] [drm] radeon: 32M of VRAM memory ready
<6>[    3.752525] [drm] radeon: 512M of GTT memory ready.
<6>[    3.762244] [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
<6>[    3.775413] [drm] Driver supports precise vblank timestamp query.
<6>[    3.787566] [drm] radeon: irq initialized.
<6>[    3.795716] [drm] GART: num cpu pages 131072, num gpu pages 131072
<6>[    3.808648] [drm] Loading RS780 Microcode
<6>[    3.831065] radeon 0000:01:05.0: WB enabled
<6>[    3.871222] [drm] ring test succeeded in 1 usecs
<6>[    3.880557] [drm] radeon: ib pool ready.
<6>[    3.888479] [drm] ib test succeeded in 0 usecs
<7>[    3.897390] failed to evaluate ATIF got AE_BAD_PARAMETER
<6>[    3.897526] [drm] Radeon Display Connectors
<6>[    3.905848] [drm] Connector 0:
<6>[    3.911920] [drm]   DVI-I
<6>[    3.917123] [drm]   HPD1
<6>[    3.922157] [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
<6>[    3.936883] [drm]   Encoders:
<6>[    3.942780] [drm]     CRT1: INTERNAL_KLDSCP_DAC1
<6>[    3.951965] [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
<6>[    3.970902] [drm] Radeon display connector DVI-I-1: No monitor connected or invalid EDID
<6>[    3.986756] [drm] radeon: power management initialized
<6>[    4.012329] No connectors reported connected with modes
<6>[    4.022797] [drm] Cannot find any crtc or sizes - going 1024x768
<6>[    4.038013] [drm] fb mappable at 0xFC141000
<6>[    4.046336] [drm] vram apper at 0xFC000000
<6>[    4.054485] [drm] size 1572864
<6>[    4.060554] [drm] fb depth is 16
<6>[    4.066973] [drm]    pitch is 2048
<6>[    4.073792] fbcon: radeondrmfb (fb0) is primary device
<6>[    4.084118] Console: switching to colour frame buffer device 128x48
<6>[    4.099907] fb0: radeondrmfb frame buffer device
<6>[    4.109095] drm: registered panic notifier
<6>[    4.117247] [drm] Initialized radeon 2.10.0 20080528 for 0000:01:05.0 on minor 0
<6>[    4.149199] md: raid0 personality registered for level 0
<6>[    4.167215] md: raid10 personality registered for level 10
<6>[    4.181945] async_tx: api initialized (async)
<6>[    4.190934] xor: automatically using best checksumming function: generic_sse
<6>[    4.221506]    generic_sse: 10404.000 MB/sec
<6>[    4.230007] xor: using function: generic_sse (10404.000 MB/sec)
<4>[    4.309535] raid6: int64x1   2143 MB/s
<4>[    4.381513] raid6: int64x2   2957 MB/s
<4>[    4.453525] raid6: int64x4   2875 MB/s
<4>[    4.525527] raid6: int64x8   2213 MB/s
<4>[    4.597508] raid6: sse2x1    3404 MB/s
<4>[    4.669515] raid6: sse2x2    5775 MB/s
<4>[    4.741506] raid6: sse2x4    6912 MB/s
<4>[    4.748973] raid6: using algorithm sse2x4 (6912 MB/s)
<6>[    4.759695] md: raid6 personality registered for level 6
<6>[    4.770323] md: raid5 personality registered for level 5
<6>[    4.780894] md: raid4 personality registered for level 4
<6>[    4.804741] md: md1 stopped.
<6>[    4.811484] md: bind<sdb1>
<6>[    4.817022] md: bind<sda1>
<6>[    4.823409] bio: create slab <bio-1> at 1
<6>[    4.831466] md/raid1:md1: active with 2 out of 2 mirrors
<6>[    4.842093] md1: detected capacity change from 0 to 4005560320
<6>[    4.854833]  md1: unknown partition table
<6>[    5.066599] kjournald starting.  Commit interval 15 seconds
<6>[    5.081753] EXT3-fs (md1): using internal journal
<6>[    5.092456] EXT3-fs (md1): mounted filesystem with ordered data mode
<6>[    6.404373] udev: starting version 147
<6>[    6.485366] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
<6>[    6.502114] ACPI: Power Button [PWRB]
<6>[    6.509550] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
<6>[    6.524304] ACPI: Power Button [PWRF]
<6>[    6.555234] rtc_cmos 00:03: RTC can wake from S4
<6>[    6.564534] rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
<6>[    6.576711] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
<6>[    6.608319] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0x8040, revision 0
<6>[    6.630766] tg3.c:v3.119 (May 18, 2011)
<6>[    6.638421] tg3 0000:02:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
<7>[    6.651766] tg3 0000:02:00.0: setting latency timer to 64
<5>[    6.721658] sd 0:0:0:0: Attached scsi generic sg0 type 0
<5>[    6.732294] sd 1:0:0:0: Attached scsi generic sg1 type 0
<6>[    6.742963] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
<6>[    6.755634] ppdev: user-space parallel port driver
<6>[    6.776029] tg3 mdio bus: probed
<6>[    6.795266] input: PC Speaker as /devices/platform/pcspkr/input/input2
<6>[    6.795425] tg3 0000:02:00.0: eth0: Tigon3 [partno(BCM57780) rev 57780001] (PCI Express) MAC address 00:19:99:84:97:72
<6>[    6.795428] tg3 0000:02:00.0: eth0: attached PHY driver [Broadcom BCM57780] (mii_bus:phy_addr=200:01)
<6>[    6.795431] tg3 0000:02:00.0: eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
<6>[    6.795433] tg3 0000:02:00.0: eth0: dma_rwctrl[76180000] dma_mask[64-bit]
<6>[    6.877322] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
<6>[    7.169806] HDA Intel 0000:01:05.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
<7>[    7.184195] ALSA hda_intel.c:2550 Using LPIB position fix
<7>[    7.184229] HDA Intel 0000:01:05.1: setting latency timer to 64
<7>[    7.196120] ALSA hda_intel.c:1593 Enable sync_write for stable communication
<6>[    7.264558] Adding 1959924k swap on /dev/sdb2.  Priority:-1 extents:1 across:1959924k 
<6>[    8.191101] device-mapper: uevent: version 1.0.3
<6>[    8.200403] device-mapper: ioctl: 4.20.0-ioctl (2011-02-02) initialised: dm-devel@redhat.com
<6>[    8.313049] md: md3 stopped.
<6>[    8.319567] md: bind<sdb3>
<6>[    8.326330] md/raid1:md3: active with 1 out of 2 mirrors
<6>[    8.336928] md3: detected capacity change from 0 to 994189508608
<6>[    8.350358]  md3: unknown partition table
<6>[    9.125855] loop: module loaded
<6>[    9.185378] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled
<6>[    9.205569] SGI XFS Quota Management subsystem
<5>[    9.217349] XFS (dm-0): Mounting Filesystem
<6>[    9.337314] XFS (dm-0): Ending clean mount
<5>[    9.346350] XFS (dm-1): Mounting Filesystem
<6>[    9.470650] XFS (dm-1): Ending clean mount
<5>[    9.479568] XFS (dm-2): Mounting Filesystem
<6>[    9.723315] XFS (dm-2): Ending clean mount
<5>[    9.732315] XFS (dm-3): Mounting Filesystem
<6>[    9.870198] XFS (dm-3): Ending clean mount
<6>[   11.031105] fuse init (API version 7.16)
Kernel logging (ksyslog) stopped.
Kernel log daemon terminating.

Boot logging started on /dev/ttyS0(/dev/console) at Thu Jul 25 13:52:55 2013

mdadm: /dev/md1 has been started with 2 drives.
Trying manual resume from /dev/disk/by-id/scsi-20004d927fffff800-part1
resume device /dev/disk/by-id/scsi-20004d927fffff800-part1 not found (ignoring)
Trying manual resume from /dev/disk/by-id/scsi-20004d927fffff800-part1
resume device /dev/disk/by-id/scsi-20004d927fffff800-part1 not found (ignoring)
Waiting for device /dev/md1 to appear:  ok
fsck from util-linux 2.19.1
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/md1 
root: clean, 6706/489600 files, 185163/977920 blocks
fsck succeeded. Mounting root device read-write.
Mounting root /dev/md1
mount -o rw,acl,user_xattr -t ext3 /dev/md1 /root

Boot logging started on /dev/ttyS0(/dev/console) at Thu Jul 25 13:52:58 2013

<notice -- Jul 25 13:52:58.270120000> boot.debugfs start
Mounting debugfs at /sys/kernel/debugdone
<notice -- Jul 25 13:52:58.284186000> 'boot.debugfs start' exits with status 0
<notice -- Jul 25 13:52:58.284822000> boot.udev start
Starting udevd: done
Loading drivers, configuring devices: done
<notice -- Jul 25 13:52:59.229527000> 'boot.udev start' exits with status 0
<notice -- Jul 25 13:52:59.230174000> boot.rootfsck start
Activating swap-devices in /etc/fstab...
failed
<notice -- Jul 25 13:52:59.297953000> 'boot.rootfsck start' exits with status 0
<notice -- Jul 25 13:52:59.298598000> boot.clock start
Set System Time to the current Hardware Clockdone
<notice -- Jul 25 13:53:00.136690000> 'boot.clock start' exits with status 0
<notice -- Jul 25 13:53:00.137345000> boot.device-mapper start
Activating device mapper...
done
Loading required kernel modules
<notice -- Jul 25 13:53:00.218381000> 'boot.device-mapper start' exits with status 0
<notice -- Jul 25 13:53:00.219022000> boot.loadmodules start
<notice -- Jul 25 13:53:00.235000000> 'boot.loadmodules start' exits with status 0
<notice -- Jul 25 13:53:00.236043000> boot.md start
doneStarting MD RAID mdadm: /dev/md3 has been started with 1 drive (out of 2).
done
Waiting for udev to settle...
<notice -- Jul 25 13:53:00.350161000> 'boot.md start' exits with status 0
<notice -- Jul 25 13:53:00.350831000> boot.lvm start
Scanning for LVM volume groups...
  Reading all physical volumes.  This may take a while...
  Found volume group "vg00" using metadata type lvm2
Activating LVM volume groups...
  Partial mode. Incomplete logical volumes will be processed.
  4 logical volume(s) in volume group "vg00" now active
done
<notice -- Jul 25 13:53:01.52388000> 'boot.lvm start' exits with status 0
<notice -- Jul 25 13:53:01.53044000> boot.localfs start
Checking file systems...
fsck from util-linux 2.19.1
Checking all file systems.
[/sbin/fsck.xfs (1) -- /usr] fsck.xfs -a /dev/mapper/vg00-usr 
/sbin/fsck.xfs: XFS file system.
[/sbin/fsck.xfs (1) -- /var] fsck.xfs -a /dev/mapper/vg00-var 
/sbin/fsck.xfs: XFS file system.
[/sbin/fsck.xfs (1) -- /home] fsck.xfs -a /dev/mapper/vg00-home 
/sbin/fsck.xfs: XFS file system.
[/sbin/fsck.xfs (1) -- /srv] fsck.xfs -a /dev/mapper/vg00-srv 
/sbin/fsck.xfs: XFS file system.
doneMounting local file systems...
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,mode=1777)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
/dev/mapper/vg00-usr on /usr type xfs (rw)
/dev/mapper/vg00-var on /var type xfs (rw)
/dev/mapper/vg00-home on /home type xfs (rw)
/dev/mapper/vg00-srv on /srv type xfs (rw)
done
<notice -- Jul 25 13:53:01.880260000> 'boot.localfs start' exits with status 0
<notice -- Jul 25 13:53:01.881322000> boot.compliance start
Check if the profiles matches the systemdone
<notice -- Jul 25 13:53:02.946802000> 'boot.compliance start' exits with status 0
<notice -- Jul 25 13:53:02.947558000> boot.cycle start
<notice -- Jul 25 13:53:02.980851000> 'boot.cycle start' exits with status 6
<notice -- Jul 25 13:53:02.981530000> boot.fuse start
Loading fuse module done
Mounting fuse control filesystemdone
<notice -- Jul 25 13:53:03.43140000> 'boot.fuse start' exits with status 0
<notice -- Jul 25 13:53:03.43762000> boot.klog start
Creating /var/log/boot.msg
done
<notice -- Jul 25 13:53:03.286692000> pidofproc: /sbin/blogd 539
<notice -- Jul 25 13:53:03.298365000> killproc: kill(539,29)
<notice -- Jul 25 13:53:03.299258000> 'boot.klog start' exits with status 0
<notice -- Jul 25 13:53:03.299866000> boot.proc start
<notice -- Jul 25 13:53:03.312536000> 'boot.proc start' exits with status 0
<notice -- Jul 25 13:53:03.313232000> boot.scpm start
<notice -- Jul 25 13:53:03.371027000> 'boot.scpm start' exits with status 0
<notice -- Jul 25 13:53:03.371627000> boot.swap start
Activating remaining swap-devices in /etc/fstab...
failed
<notice -- Jul 25 13:53:03.681075000> 'boot.swap start' exits with status 0
<notice -- Jul 25 13:53:03.681725000> boot.udev_retry start
<notice -- Jul 25 13:53:03.703229000> 'boot.udev_retry start' exits with status 0
<notice -- Jul 25 13:53:03.703915000> boot.ldconfig start
<notice -- Jul 25 13:53:03.826875000> 'boot.ldconfig start' exits with status 0
<notice -- Jul 25 13:53:03.829587000> boot.quota start
Turning quota on
Checking quotas. This may take some time.
done
<notice -- Jul 25 13:53:03.903843000> 'boot.quota start' exits with status 0
<notice -- Jul 25 13:53:03.904516000> boot.sysctl start
Setting current sysctl status from /etc/sysctl.confdone
<notice -- Jul 25 13:53:03.935472000> 'boot.sysctl start' exits with status 0
<notice -- Jul 25 13:53:03.936139000> boot.cleanup start
<notice -- Jul 25 13:53:04.130830000> 
pidofproc: /sbin/blogd 
539

ATTENTION: You have modified /etc/resolv.conf.  Leaving it untouched...
You can find my version in /etc/resolv.conf.netconfig ...
<notice -- Jul 25 13:53:04.335187000> 'boot.cleanup start' exits with status 0
<notice -- Jul 25 13:53:04.335850000> boot.ipconfig start
Enabling syn flood protectiondone
Disabling IP forwardingdone
done
<notice -- Jul 25 13:53:04.364709000> 'boot.ipconfig start' exits with status 0
<notice -- Jul 25 13:53:04.377587000> boot.apparmor start
Starting AppArmor Mounting securityfs on /sys/kernel/security done
done
<notice -- Jul 25 13:53:05.667939000> 'boot.apparmor start' exits with status 0
<notice -- Jul 25 13:53:05.668600000> boot.localnet start
Setting up hostname 's15434194'done
Setting up loopback interface     lo        
    lo        IP address: 127.0.0.1/8   
              IP address: 127.0.0.2/8   
done
<notice -- Jul 25 13:53:05.870641000> 'boot.localnet start' exits with status 0
<notice -- Jul 25 13:53:05.871281000> boot.kdump start
Loading kdump

done
System Boot Control: The system has been set up
Skipped features: boot.cycle
System Boot Control: Running /etc/init.d/boot.local
<notice -- Jul 25 13:53:06.749258000> 'boot.kdump start' exits with status 0
done
INIT: Entering runlevel: 3


Master Resource Control: previous runlevel: N, switching to runlevel: 3
<notice -- Jul 25 13:53:06.907972000> SuSEfirewall2_init start
Starting Firewall Initialization (phase 1 of 2) done
<notice -- Jul 25 13:53:07.470088000> 'SuSEfirewall2_init start' exits with status 0
<notice -- Jul 25 13:53:07.470818000> acpid start
Starting acpid 
<notice -- Jul 25 13:53:07.503772000> startproc: execve (/sbin/acpid) [ /sbin/acpid ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/sbin/acpid ]
acpid: starting up


done
<notice -- Jul 25 13:53:07.516106000> 'acpid start' exits with status 0
<notice -- Jul 25 13:53:07.516828000> dbus start
acpid: 2 rules loaded


Starting D-Bus daemondone
<notice -- Jul 25 13:53:07.727493000> 'dbus start' exits with status 0
<notice -- Jul 25 13:53:07.728178000> earlysyslog start
Starting syslog services
<notice -- Jul 25 13:53:07.818943000> startproc: execve (/sbin/syslog-ng) [ /sbin/syslog-ng ], [ SELINUX_INIT=YES CONSOLE=/dev/console ROOTFS_FSTYPE=ext3 TERM=vt102 SHELL=/bin/sh ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin DO_CONFIRM= vga=0x31a RUNLEVEL=3 SPLASHCFG= PWD=/ PREVLEVEL=N LINES=24 SHLVL=2 HOME=/ SPLASH=no ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/sbin/syslog-ng ]
<notice -- Jul 25 13:53:07.853147000> startproc: execve (/sbin/klogd) [ /sbin/klogd -c 1 -x ], [ SELINUX_INIT=YES CONSOLE=/dev/console ROOTFS_FSTYPE=ext3 TERM=vt102
 SHELL=/bin/sh ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin DO_CONFIRM= vga=0x31a
 RUNLEVEL=3 SPLASHCFG= PWD=/
 PREVLEVEL=N LINES=24 SHLVL=2
 HOME=/ SPLASH=no ROOTFS_BLKDEV=/dev/md1
 _=/sbin/startproc DAEMON=/sbin/klogd ]
done
<notice -- Jul 25 13:53:07.855546000> 'earlysyslog start' exits with status 0
<notice -- Jul 25 13:53:07.856236000> jexec start
Starting jexec services
<notice -- Jul 25 13:53:07.942000000> 'jexec start' exits with status 0
<notice -- Jul 25 13:53:07.942635000> microcode.ctl start
Checking CPU.. upload AMD microcodedone
<notice -- Jul 25 13:53:08.175276000> 'microcode.ctl start' exits with status 0
<notice -- Jul 25 13:53:08.175974000> purge-kernels start
<notice -- Jul 25 13:53:08.198042000> 'purge-kernels start' exits with status 0
<notice -- Jul 25 13:53:08.198676000> random start
Initializing random number generatordone
<notice -- Jul 25 13:53:08.246691000> 'random start' exits with status 0
<notice -- Jul 25 13:53:08.247413000> xe-linux-distribution start
<notice -- Jul 25 13:53:08.306239000> 'xe-linux-distribution start' exits with status 0
<notice -- Jul 25 13:53:08.312201000> haldaemon start
Loading CPUFreq modulesdone
Starting HAL daemondone
<notice -- Jul 25 13:53:09.167771000> 'haldaemon start' exits with status 0
<notice -- Jul 25 13:53:09.169892000> network start
<notice -- Jul 25 13:53:09.331138000> checkproc: /sbin/udevd 566<notice -- Jul 25 13:53:09.331189000> checkproc: /sbin/udevd  626<notice -- Jul 25 13:53:09.331195000> checkproc: /sbin/udevd  627<notice -- Jul 25 13:53:09.331199000> checkproc: /sbin/udevd  1592<notice -- Jul 25 13:53:09.331203000> checkproc: /sbin/udevd  1598
<notice -- Jul 25 13:53:09.331206000> checkproc: /sbin/udevd  1599<notice -- Jul 25 13:53:09.331211000> checkproc: /sbin/udevd  1604<notice -- Jul 25 13:53:09.331215000> checkproc: /sbin/udevd  1605<notice -- Jul 25 13:53:09.331218000> checkproc: /sbin/udevd  1610<notice -- Jul 25 13:53:09.331222000> checkproc: /sbin/udevd  1611<notice -- Jul 25 13:53:09.331225000> checkproc: /sbin/udevd  1616
Setting up (localfs) network interfaces:
    lo        
    lo        IP address: 127.0.0.1/8   
              IP address: 127.0.0.2/8   
done    eth0      device: Broadcom Corporation NetLink BCM57780 Gigabit
    eth0      Starting DHCP4 client.  
    eth0      IP address: 87.106.180.172/32 (s15434194)
    eth0:1    IP address: 87.106.137.177/32   
    eth0:2    IP address: 87.106.138.58/32   
    eth0:3    IP address: 87.106.138.164/32   
doneSetting up service (localfs) network  .  .  .  .  .  .  .  .  .  .done
<notice -- Jul 25 13:53:15.208074000> 'network start' exits with status 0
<notice -- Jul 25 13:53:15.210099000> syslog start
<notice -- Jul 25 13:53:15.228756000> 'syslog start' exits with status 0
<notice -- Jul 25 13:53:15.230185000> auditd start
Starting auditd 
<notice -- Jul 25 13:53:15.279314000> 
startproc: execve (/sbin/auditd) [
 /sbin/auditd
 -s
 disable
 ], [
 CONSOLE=/dev/console
 SELINUX_INIT=YES
 ROOTFS_FSTYPE=ext3
 SHELL=/bin/sh
 TERM=vt102
 ROOTFS_FSCK=0
 crashkernel=128M-:64M
 LC_ALL=POSIX
 INIT_VERSION=sysvinit-2.86
 REDIRECT=/dev/ttyS0
 COLUMNS=80
 PATH=/bin:/sbin:/usr/bin:/usr/sbin
 vga=0x31a
 DO_CONFIRM=
 RUNLEVEL=3
 PWD=/
 SPLASHCFG=
 PREVLEVEL=N
 LINES=24
 HOME=/
 SHLVL=2
 SPLASH=no
 ROOTFS_BLKDEV=/dev/md1
 _=/sbin/startproc
 DAEMON=/sbin/auditd
 ]
done
<notice -- Jul 25 13:53:15.410691000> 'auditd start' exits with status 0
<notice -- Jul 25 13:53:15.460749000> elasticsearch start
su: warning: cannot change directory to /srv/elasticsearch: No such file or directory
Starting ElasticSearch...
Removed stale pid file: /srv/elasticsearch-0.90.0/lock/elasticsearch.pid
Waiting for ElasticSearch.......
running: PID:3414
<notice -- Jul 25 13:53:21.439603000> 'elasticsearch start' exits with status 0
<notice -- Jul 25 13:53:21.441181000> haveged start
Starting haveged daemon 
<notice -- Jul 25 13:53:21.510012000> startproc: execve (/sbin/haveged) [ /sbin/haveged
 -w 1024 -v 1 ], [
 CONSOLE=/dev/console
 SELINUX_INIT=YES
 ROOTFS_FSTYPE=ext3
 SHELL=/bin/sh
 TERM=vt102
 ROOTFS_FSCK=0
 crashkernel=128M-:64M
 LC_ALL=POSIX
 INIT_VERSION=sysvinit-2.86
 REDIRECT=/dev/ttyS0
 COLUMNS=80
 PATH=/bin:/sbin:/usr/bin:/usr/sbin
 vga=0x31a
 DO_CONFIRM=
 RUNLEVEL=3
 PWD=/
 SPLASHCFG=
 PREVLEVEL=N
 LINES=24
 HOME=/
 SHLVL=2
 SPLASH=no
 ROOTFS_BLKDEV=/dev/md1
 _=/sbin/startproc
 DAEMON=/sbin/haveged
 ]
done
<notice -- Jul 25 13:53:21.676446000> 'haveged start' exits with status 0
<notice -- Jul 25 13:53:21.677935000> nfs start
Not starting NFS client services - no NFS found in /etc/fstab:unused
<notice -- Jul 25 13:53:21.813500000> 'nfs start' exits with status 6
<notice -- Jul 25 13:53:21.815535000> kbd start
Loading console font lat9w-16.psfu  -m trivial G0:loadable
doneLoading keymap assuming iso-8859-15 euro
Loading /usr/share/kbd/keymaps/i386/qwerty/us.map.gz
doneLoading compose table latin1.adddone
Start Unicode mode
done
<notice -- Jul 25 13:53:22.395859000> 'kbd start' exits with status 0
<notice -- Jul 25 13:53:22.397951000> irq_balancer start
Starting irqbalance 
<notice -- Jul 25 13:53:22.567429000> startproc: execve (/usr/sbin/irqbalance) [ /usr/sbin/irqbalance ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0
 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/
 SPLASHCFG= PREVLEVEL=N LINES=24
 HOME=/ SHLVL=2 SPLASH=no
 ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/usr/sbin/irqbalance
 ]
done
<notice -- Jul 25 13:53:22.581407000> 'irq_balancer start' exits with status 0
<notice -- Jul 25 13:53:22.583476000> mcelog start
Starting mcelog... 
<notice -- Jul 25 13:53:22.671594000> startproc: execve (/usr/sbin/mcelog) [ /usr/sbin/mcelog --daemon --config-file /etc/mcelog/mcelog.conf ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0
 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no ROOTFS_BLKDEV=/dev/md1
 _=/sbin/startproc DAEMON=/usr/sbin/mcelog ]
done
<notice -- Jul 25 13:53:22.727527000> 'mcelog start' exits with status 0
<notice -- Jul 25 13:53:22.729095000> mysql start
..dead
Starting service MySQL done
<notice -- Jul 25 13:53:28.285787000> 'mysql start' exits with status 0
<notice -- Jul 25 13:53:28.288012000> network-remotefs start
<notice -- Jul 25 13:53:28.465923000> checkproc: /sbin/udevd 566<notice -- Jul 25 13:53:28.465990000> checkproc: /sbin/udevd  3514
<notice -- Jul 25 13:53:28.465997000> checkproc: /sbin/udevd  3515
Setting up (remotefs) network interfaces:
Setting up service (remotefs) network  .  .  .  .  .  .  .  .  .  .done
<notice -- Jul 25 13:53:28.657327000> 'network-remotefs start' exits with status 0
<notice -- Jul 25 13:53:28.658410000> sshd start
Starting SSH daemon
<notice -- Jul 25 13:53:28.752786000> startproc: execve (/usr/sbin/sshd) [ /usr/sbin/sshd -o PidFile=/var/run/sshd.init.pid ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/usr/sbin/sshd ]
done
<notice -- Jul 25 13:53:28.905446000> 'sshd start' exits with status 0
<notice -- Jul 25 13:53:28.907586000> ntp start
Starting network time protocol daemon (NTPD)
<notice -- Jul 25 13:53:29.50415000> startproc: execve (/usr/sbin/ntpd) [ /usr/sbin/ntpd -p /var/run/ntp/ntpd.pid -g -u ntp:ntp -i /var/lib/ntp -c /etc/ntp.conf
 ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3
 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no
 ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/usr/sbin/ntpd
 ]
done
<notice -- Jul 25 13:53:29.65968000> 'ntp start' exits with status 0
<notice -- Jul 25 13:53:29.67384000> nscd start
Starting Name Service Cache Daemon
<notice -- Jul 25 13:53:29.138196000> startproc: execve (/usr/sbin/nscd) [ /usr/sbin/nscd ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/usr/sbin/nscd ]
done
<notice -- Jul 25 13:53:29.209602000> 'nscd start' exits with status 0
<notice -- Jul 25 13:53:29.211726000> postfix start
Starting mail service (Postfix)done
<notice -- Jul 25 13:53:30.311301000> 'postfix start' exits with status 0
<notice -- Jul 25 13:53:30.353156000> apache2 start
Starting httpd2 (prefork) 
<notice -- Jul 25 13:53:32.759660000> 
startproc: execve (/usr/sbin/httpd2-prefork) [
 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -DSSL ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 get_module_list_done=true
 crashkernel=128M-:64M
 LC_ALL=POSIX
 INIT_VERSION=sysvinit-2.86
 OPENSSL_NO_DEFAULT_ZLIB=1
 REDIRECT=/dev/ttyS0
 COLUMNS=80
 get_includes_done=true
 PATH=/bin:/sbin:/usr/bin:/usr/sbin
 vga=0x31a
 DO_CONFIRM=
 RUNLEVEL=3
 PWD=/
 SPLASHCFG=
 PREVLEVEL=N
 LINES=24
 HOME=/var/lib/apache2
 SHLVL=2
 SPLASH=no
 ROOTFS_BLKDEV=/dev/md1
 _=/sbin/startproc
 DAEMON=/usr/sbin/httpd2-prefork
 ]
done
<notice -- Jul 25 13:53:35.275998000> 'apache2 start' exits with status 0
<notice -- Jul 25 13:53:35.278078000> cron start
Starting CRON daemon
<notice -- Jul 25 13:53:35.316917000> startproc: execve (/usr/sbin/cron) [ /usr/sbin/cron ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/usr/sbin/cron ]
done
<notice -- Jul 25 13:53:35.479400000> 'cron start' exits with status 0
<notice -- Jul 25 13:53:35.480527000> mdadmd start
Starting mdadmd 
<notice -- Jul 25 13:53:35.512207000> startproc: execve (/sbin/mdadm) [ /sbin/mdadm -F -d 60 -m root@localhost -s -c /etc/mdadm.conf ], [ CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=vt102 ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/ttyS0 COLUMNS=80 PATH=/bin:/sbin:/usr/bin:/usr/sbin vga=0x31a DO_CONFIRM= RUNLEVEL=3 PWD=/ SPLASHCFG= PREVLEVEL=N LINES=24 HOME=/ SHLVL=2 SPLASH=no ROOTFS_BLKDEV=/dev/md1 _=/sbin/startproc DAEMON=/sbin/mdadm ]
done
<notice -- Jul 25 13:53:36.22471000> 'mdadmd start' exits with status 0
<notice -- Jul 25 13:53:36.24646000> smartd start
Starting smartd done
<notice -- Jul 25 13:53:37.692672000> 'smartd start' exits with status 0
<notice -- Jul 25 13:53:37.694860000> SuSEfirewall2_setup start
Starting Firewall Initialization (phase 2 of 2) done
Master Resource Control: runlevel 3 has been reached
Skipped services in runlevel 3: nfs
<notice -- Jul 25 13:53:38.974169000> 'SuSEfirewall2_setup start' exits with status 0
<notice -- Jul 25 13:53:38.985106000> killproc: kill(539,3)

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Recovery of RAID1 fails (added disks stays as spare)
  2013-08-19  7:48   ` Matthias.Blaesing
@ 2013-08-19  8:04     ` NeilBrown
  2013-08-20  6:14       ` Matthias.Blaesing
  0 siblings, 1 reply; 5+ messages in thread
From: NeilBrown @ 2013-08-19  8:04 UTC (permalink / raw)
  To: Matthias.Blaesing; +Cc: linux-raid

[-- Attachment #1: Type: text/plain, Size: 2802 bytes --]

On Mon, 19 Aug 2013 07:48:20 +0000 <Matthias.Blaesing@persona.de> wrote:

> Hey,

> > Strange.  I would definitely have expected one of those to start the
> > recovery.
> > Does anything appear in the kernel logs (e.g. output of 'dmesg')?
> 
> Please see the attaches boot.msg - from my perspective it only confirms the results
> gained from the /proc/mdstat and mdadm output.

I guess I should have been more explicit.
It looks like you have just attached a log of what happened at boot.
I mean does anything appear in the dmesg output when you try to add the spare
device to the array.

> 
> To give some more information - her the mdadm.conf:
> 
> s15434194:~ # cat /etc/mdadm.conf
> CREATE owner=root group=disk mode=0660 auto=yes
> 
> MAILADDR root
> 
> DEVICE containers partitions
> ARRAY /dev/md1 level=raid1 num-devices=2 devices=/dev/sda1,/dev/sdb1
> ARRAY /dev/md3 level=raid1 num-devices=2 devices=/dev/sda3,/dev/sdb3
> 
> 
> Could it be that the problem lies in the order in the devices row? When I compare the working md1 and the non-working md3 array I see a difference in the mapping of devices to numbers:

No, the order is irrelevant.

The experiment below wouldn't change anything.

NeilBrown

> 
> For md1:
> 
>    Number   Major   Minor   RaidDevice State
>        0       8        1        0      active sync   /dev/sda1
>        1       8       17        1      active sync   /dev/sdb1
> 
> For md3:
> 
>     Number   Major   Minor   RaidDevice State
>        0       8       19        0      active sync   /dev/sdb3
>        1       0        0        1      removed
> 
>        2       8        3        -      spare   /dev/sda3
> 
> 
> Would this be worth a try (the disks need approx. 24 hours to reach the rebuild-state, so I'm reluctant to try without some hope) to do:
> 
> 1. change the Array line in /etc/mdadm.conf for md3 to read (switching order):
> ARRAY /dev/md3 level=raid1 num-devices=2 devices=/dev/sdb3,/dev/sda3
> 2. Remove the spare sda3
> 3. Zero the superblock of sda3
> 4. Add sda3 again to the array
> 
> ?
> 
> Mit freundlichen Grüßen
> 
> Matthias Bläsing
> Abt IT
> KompetenzCenter
> 
> Tel.: 02351 950-344
> Fax.: 02351 950-222
> mailto: matthias.blaesing@persona.de
> www.persona.de
> 
> 
> persona service Verwaltungs AG & Co. KG 
> Freisenbergstraße 31 • 58513 Lüdenscheid  
> Tel.: (02351) 950-0 • Fax: (02351) 950-222 
> Sitz Lüdenscheid • Registergericht Iserlohn, HRA Nr. 2930
> 
> persönlich haftende Gesellschafterin: persona service AG
> Gartenstraße 93 • CH-4002 Basel
> Handelsregister Basel, Nr. CH-270.3.012.836-8
> diese vertreten durch den Verwaltungsrat:
> Georg Breucker (Präsident) und Dr. Sebastian Burckhardt
> www.persona.de


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: Recovery of RAID1 fails (added disks stays as spare)
  2013-08-19  8:04     ` NeilBrown
@ 2013-08-20  6:14       ` Matthias.Blaesing
  0 siblings, 0 replies; 5+ messages in thread
From: Matthias.Blaesing @ 2013-08-20  6:14 UTC (permalink / raw)
  To: linux-raid

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3647 bytes --]

Hey,

> -----Original Message-----
> From: NeilBrown [mailto:neilb@suse.de]
> Sent: Monday, August 19, 2013 10:04 AM
> To: Blaesing, Matthias (KC-IT)
> Cc: linux-raid@vger.kernel.org
> Subject: Re: Recovery of RAID1 fails (added disks stays as spare)
> 
> On Mon, 19 Aug 2013 07:48:20 +0000 <Matthias.Blaesing@persona.de> wrote:
> 
> > Hey,
> 
> > > Strange.  I would definitely have expected one of those to start the
> > > recovery.
> > > Does anything appear in the kernel logs (e.g. output of 'dmesg')?
> >
> > Please see the attaches boot.msg - from my perspective it only
> confirms the results
> > gained from the /proc/mdstat and mdadm output.
> 
> I guess I should have been more explicit.
> It looks like you have just attached a log of what happened at boot.
> I mean does anything appear in the dmesg output when you try to add the
> spare
> device to the array.

You mean this (the first lines should be the result of mdadm --fail and mdadm --remove):

Aug 19 10:54:28 s15434194 kernel: [2149292.392602] md/raid1:md3: Disk failure on sda3, disabling device.
Aug 19 10:54:28 s15434194 kernel: [2149292.392603] md/raid1:md3: Operation continuing on 1 devices.
Aug 19 10:54:44 s15434194 kernel: [2149307.970181] md: unbind<sda3>
Aug 19 10:54:44 s15434194 kernel: [2149308.033073] md: export_rdev(sda3)
Aug 19 10:56:46 s15434194 kernel: [2149430.007318] md: bind<sda3>
Aug 19 10:56:46 s15434194 kernel: [2149430.043534] RAID1 conf printout:
Aug 19 10:56:46 s15434194 kernel: [2149430.043542]  --- wd:1 rd:2
Aug 19 10:56:46 s15434194 kernel: [2149430.043549]  disk 0, wo:0, o:1, dev:sdb3
Aug 19 10:56:46 s15434194 kernel: [2149430.043554]  disk 1, wo:1, o:1, dev:sda3
Aug 19 10:56:46 s15434194 kernel: [2149430.043833] md: recovery of RAID array md3
Aug 19 10:56:46 s15434194 kernel: [2149430.043841] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
Aug 19 10:56:46 s15434194 kernel: [2149430.043847] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
Aug 19 10:56:46 s15434194 kernel: [2149430.043857] md: using 128k window, over a total of 970888192k.


Aug 20 02:27:56 s15434194 kernel: [2205289.321866] md: md3: recovery done.
Aug 20 02:27:57 s15434194 kernel: [2205300.981062] RAID1 conf printout:
Aug 20 02:27:57 s15434194 kernel: [2205300.981070]  --- wd:1 rd:2
Aug 20 02:27:57 s15434194 kernel: [2205300.981077]  disk 0, wo:0, o:1, dev:sdb3
Aug 20 02:27:57 s15434194 kernel: [2205300.981083]  disk 1, wo:1, o:1, dev:sda3
Aug 20 02:27:57 s15434194 kernel: [2205301.025544] RAID1 conf printout:
Aug 20 02:27:57 s15434194 kernel: [2205301.025551]  --- wd:1 rd:2
Aug 20 02:27:57 s15434194 kernel: [2205301.025557]  disk 0, wo:0, o:1, dev:sdb3

While rebuilding I also see read-errors from sdb:

Aug 20 02:27:54 s15434194 kernel: [2205267.984530] md/raid1:md3: sdb: unrecoverable I/O read error for block 1941766272

I guess as this affects the source, it should not affect the move of sda to be active.


Mit freundlichen Grüßen

Matthias Bläsing
Abt IT
KompetenzCenter

Tel.: 02351 950-344
Fax.: 02351 950-222
mailto: matthias.blaesing@persona.de
www.persona.de




persona service Verwaltungs AG & Co. KG 
Freisenbergstraße 31 • 58513 Lüdenscheid  
Tel.: (02351) 950-0 • Fax: (02351) 950-222 
Sitz Lüdenscheid • Registergericht Iserlohn, HRA Nr. 2930

persönlich haftende Gesellschafterin: persona service AG
Gartenstraße 93 • CH-4002 Basel
Handelsregister Basel, Nr. CH-270.3.012.836-8
diese vertreten durch den Verwaltungsrat:
Georg Breucker (Präsident) und Dr. Sebastian Burckhardt
www.persona.de

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2013-08-20  6:14 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-08-15  9:09 Recovery of RAID1 fails (added disks stays as spare) Matthias.Blaesing
2013-08-17  0:50 ` NeilBrown
2013-08-19  7:48   ` Matthias.Blaesing
2013-08-19  8:04     ` NeilBrown
2013-08-20  6:14       ` Matthias.Blaesing

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).