* Re: mdsadm -A won't assemble my array
From: Phil Turmel @ 2015-02-10 2:04 UTC (permalink / raw)
To: G. Michael Carter; +Cc: linux-raid
In-Reply-To: <CANjusZdgzDxN4ZxsXFc20OU1Y9_LoScYQcZmffyRGHWYUviY0Q@mail.gmail.com>
Hi Michael,
On 02/09/2015 08:36 PM, G. Michael Carter wrote:
> Was actually dreading the complete info question, as the machine has
> no CD/DVD, it is my netboot station and holds all my ISO mirrors. But
> turns out starting it's network interface from the emergency shell was
> easier than I thought.
Almost all good livecds can be put on a thumb drive to boot from,
instead of using a real CD. I highly recommend sysrescuecd.org, FWIW.
> ---- uname
> Linux andromeda 3.16.6-203.fc20.x86_64 #1 SMP Sat Oct 25 12:44:32 UTC
> 2014 x86_64 x86_64 x86_64 GNU/Linux
Not too old, good. There been a steady stream of small bugfixes since
3.16. I'm not sure what fedora's been backporting.
> ---- mdadm -V
> mdadm - v3.3 - 3rd September 2013
Bugfixes to this, too.
> ---- disk-by-id
> lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_W3009GE3
> -> ../../sdp
> lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_W300E08A
> -> ../../sdk
> lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_Z300PYF2
> -> ../../sdb
> lrwxrwxrwx 1 root root 9 Feb 9 20:16 ata-ST4000DM000-1F2168_Z300Q4YS
> -> ../../sdo
I think you missed the 'excerpt' part, but no harm done. Anyways, if I
recall Seagate model numbering (past misery), these are green drives.
Very bad for raid service. After we revive your array, you'll want to
do some reading on 'timeout mismatch'.[1] (You are dangerously close to
option "D" there, and option "C" is your only choice w/ green drives.)
> ---- mdadm -E
> /dev/sdb:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d
> Name : andromeda:3 (local to host andromeda)
> Creation Time : Tue Jul 22 16:02:30 2014
> Raid Level : raid5
> Raid Devices : 4
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 11720661504 (11177.69 GiB 12001.96 GB)
> Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
> Data Offset : 259072 sectors
> Super Offset : 8 sectors
> Unused Space : before=258984 sectors, after=3760 sectors
> State : clean
> Device UUID : e9e7af60:e1bc1c7f:107157b4:4099c48a
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Mon Feb 9 05:13:20 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 57eaf19 - correct
> Events : 158964
>
> Layout : left-symmetric
> Chunk Size : 512K
>
> Device Role : Active device 0
> Array State : AA.. ('A' == active, '.' == missing, 'R' == replacing)
> /dev/sdk:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d
> Name : andromeda:3 (local to host andromeda)
> Creation Time : Tue Jul 22 16:02:30 2014
> Raid Level : raid5
> Raid Devices : 4
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 11720661504 (11177.69 GiB 12001.96 GB)
> Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
> Data Offset : 259072 sectors
> Super Offset : 8 sectors
> Unused Space : before=258984 sectors, after=3760 sectors
> State : active
> Device UUID : ed09357e:36655c6d:b7430500:63d5e540
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Mon Feb 9 05:10:53 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 131587ad - correct
> Events : 158964
matching events.
> Layout : left-symmetric
> Chunk Size : 512K
>
> Device Role : Active device 1
> Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
> /dev/sdo:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d
> Name : andromeda:3 (local to host andromeda)
> Creation Time : Tue Jul 22 16:02:30 2014
> Raid Level : raid5
> Raid Devices : 4
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 11720661504 (11177.69 GiB 12001.96 GB)
> Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
> Data Offset : 259072 sectors
> Super Offset : 8 sectors
> Unused Space : before=258984 sectors, after=3760 sectors
> State : clean
> Device UUID : de849807:80d7f071:9909f3f2:78022d94
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Mon Feb 9 05:11:55 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : e73f681f - correct
> Events : 158962
Events off by two. Pretty minor.
> Layout : left-symmetric
> Chunk Size : 512K
>
> Device Role : Active device 3
> Array State : A.AA ('A' == active, '.' == missing, 'R' == replacing)
> /dev/sdp:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ce6de916:62aeda9c:b5688f54:f5a9249d
> Name : andromeda:3 (local to host andromeda)
> Creation Time : Tue Jul 22 16:02:30 2014
> Raid Level : raid5
> Raid Devices : 4
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 11720661504 (11177.69 GiB 12001.96 GB)
> Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
> Data Offset : 259072 sectors
> Super Offset : 8 sectors
> Unused Space : before=258984 sectors, after=3760 sectors
> State : clean
> Device UUID : 67fcd9a9:432b0c8b:178cc556:67b003b3
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Mon Feb 9 05:11:55 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 254563d8 - correct
> Events : 158962
Also off by two. Again, minor.
> Layout : left-symmetric
> Chunk Size : 512K
>
> Device Role : Active device 2
> Array State : A.AA ('A' == active, '.' == missing, 'R' == replacing)
> ---- dmesg
> [ 576.890380] md: md3 stopped.
> [ 576.891629] md: unbind<sdb>
> [ 576.900841] md: export_rdev(sdb)
> [ 576.902147] md: unbind<sdo>
> [ 576.906824] md: export_rdev(sdo)
> [ 576.908029] md: unbind<sdk>
> [ 576.911845] md: export_rdev(sdk)
> [ 576.913030] md: unbind<sdp>
> [ 576.916862] md: export_rdev(sdp)
> [ 585.164936] md: md3 stopped.
> [ 585.360571] md: bind<sdk>
> [ 585.360833] md: bind<sdp>
> [ 585.361050] md: bind<sdo>
> [ 585.361261] md: bind<sdb>
> [ 585.361294] md: md3 stopped.
> [ 585.361298] md: unbind<sdb>
> [ 585.386616] md: export_rdev(sdb)
> [ 585.387767] md: unbind<sdo>
> [ 585.398644] md: export_rdev(sdo)
> [ 585.399633] md: unbind<sdp>
> [ 585.404662] md: export_rdev(sdp)
> [ 585.405681] md: unbind<sdk>
> [ 585.410647] md: export_rdev(sdk)
> ---- mdadm -A output
> mdadm: looking for devices for /dev/md3
> mdadm: /dev/sdb is identified as a member of /dev/md3, slot 0.
> mdadm: /dev/sdk is identified as a member of /dev/md3, slot 1.
> mdadm: /dev/sdo is identified as a member of /dev/md3, slot 3.
> mdadm: /dev/sdp is identified as a member of /dev/md3, slot 2.
> mdadm: added /dev/sdk to /dev/md3 as 1
> mdadm: added /dev/sdp to /dev/md3 as 2 (possibly out of date)
> mdadm: added /dev/sdo to /dev/md3 as 3 (possibly out of date)
> mdadm: added /dev/sdb to /dev/md3 as 0
> mdadm: /dev/md3 assembled from 2 drives - not enough to start the array.
Please redo this with an explicit command line so we can see what's
going on:
mdadm --assemble --force --verbose /dev/md3 /dev/sd[bkop]
> ---- /proc/mdstat
> Personalities : [raid6] [raid5] [raid4] [raid1]
> md1 : active raid1 sda[0] sde[1]
> 1953383488 blocks super 1.2 [2/2] [UU]
> bitmap: 0/15 pages [0KB], 65536KB chunk
>
> md0 : active raid5 sdl[0] sdn[3] sdd[5] sdm[1]
> 8790405120 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
> bitmap: 0/22 pages [0KB], 65536KB chunk
>
> md2 : active raid6 sdj[0] sdi[1] sdg[3] sdh[2]
> 5860270080 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/4] [UUUU]
> bitmap: 0/22 pages [0KB], 65536KB chunk
>
> unused devices: <none>
>
> ** md3 is shutdown which is why I'm assuming it's not there.
Yup. If --assemble --force doesn't work with your installed OS,
temporarily boot from a recent system rescue cd and do the above over
again (especially the /dev/disk/by-id excerpt).
Phil
[1] http://marc.info/?l=linux-raid&m=135811522817345&w=1
More history:
http://marc.info/?l=linux-raid&m=133761065622164&w=2
http://marc.info/?l=linux-raid&m=135863964624202&w=2
http://marc.info/?l=linux-raid&m=139050322510249&w=2
You might want to read more from those threads than just the mails I've
pointed out...
^ permalink raw reply
* Re: mdsadm -A won't assemble my array
From: G. Michael Carter @ 2015-02-10 2:48 UTC (permalink / raw)
To: Phil Turmel; +Cc: linux-raid
In-Reply-To: <54D9672A.7040104@turmel.org>
Well I'm pouring myself an ice wine and raising a glass to you.
Downloaded fedora 21, ran the force assemble and it fixed my problem.
Glad I did the sensible thing this time and waited for a response
*wink*
Thanks for your help.
It's running a consistency check on those drives and then upgrading
this server to fedora 21... was next on my list anyways.
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: NeilBrown @ 2015-02-10 2:50 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150209091000.GN5029@twins.programming.kicks-ass.net>
[-- Attachment #1: Type: text/plain, Size: 2464 bytes --]
On Mon, 9 Feb 2015 10:10:00 +0100 Peter Zijlstra <peterz@infradead.org> wrote:
> On Mon, Feb 09, 2015 at 12:13:57PM +1100, NeilBrown wrote:
> > I had to re-read the code (And your analysis) a couple of times to be sure ...
>
> Sorry :-)
My point was that actually reading it (rather than assuming I knew what it
said) actually helped!
>
> > However, when io_schedule() explicitly calls blk_flush_plug(), then
> > @from_schedule=false variant is used, and the unplug functions are allowed to
> > allocate memory and block and maybe even call mempool_alloc() which might
> > call io_schedule().
> >
> > This shouldn't be a problem as blk_flush_plug() spliced out the plug list, so
> > any recursive call will find an empty list and do nothing.
>
> Unless, something along the way stuck something back on, right? So
> should we stick an:
>
> WARN_ON(current->in_iowait);
>
> somewhere near where things are added to this plug list? (and move the
> blk_flush_plug() call inside of where that's actually true of course).
No, I don't think so.
It is certainly possible that some request on plug->cb_list could add
something to plug->list - which is processed after ->cb_list.
I think the best way to think about this is that the *problem* was that a
wait_event loop could spin without making any progress. So any time that
clear forward progress is made it is safe sleep without necessitating the
warning. Hence sched_annotate_sleep() is reasonable.
blk_flush_plug() with definitely have dispatched some requests if it
might_sleep(), so the sleep is OK.
>
> The only thing that really goes wrong then is if people 'forget' to put
> a loop around io_schedule().
You mean like in congestion_wait() ??
Though that is mostly called inside a loop...
NeilBrown
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 92f4b4b288dd..7334be27823d 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1111,6 +1111,14 @@ static inline void blk_flush_plug(struct task_struct *tsk)
{
struct blk_plug *plug = tsk->plug;
+ /*
+ * Any sleeping in blk_flush_plug() should not
+ * trigger the "do not call blocking ops" warning
+ * as it makes clear forward process (requests are
+ * dispatched) and so it will not cause indefinite
+ * looping in a higher level wait loop.
+ */
+ sched_annotate_sleep();
if (plug)
blk_flush_plug_list(plug, false);
}
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
* Re: Upgrading storage server
From: John Stoffel @ 2015-02-10 3:16 UTC (permalink / raw)
To: Adam Goryachev; +Cc: linux-raid
In-Reply-To: <54D8A9AD.3060700@websitemanagers.com.au>
Adam> After making a whole string of mistakes in building a iSCSI
Adam> server about 2 years ago, I'm now looking to replace it without
Adam> all the wrong turns/mistakes. I was hoping you could all offer
Adam> some advice on hardware selection/choices.
I remember those discussions, they were quite informative and it was
interesting seeing Stan help you out. Now that you've got this system
working well, or at least well enough, what is the biggest remaining
problem you have?
I've become a big fan of supermicro FatTwin systems, and they might be
what you want here for your setup. But I'd also think about maybe you
want to go to fewer larger PCIe SSD cards in mirrored pairs instead
for better performance. Or is performance a problem still?
There's also *alot* to be said for simply replicating what you have,
but with larger SSDs, say 1Tb each, and keeping the rest of the system
and config exactly the same. Limit the changes, esp since you went
through so much pain before.
Now I might also think about upgrading all the clients to 10Gb as
well, and just moving to a completely 10G network if possible. I seem
to remember that you didn't have any way to throttle or setup Quality
of Service limits on your iSCSI vs. other network traffic, which is
why you ended up splitting up the traffic like this, so that a single
VM couldn't bring the rest to their knees when a user did something
silly.
So again, if it's working well now, don't chage your architecture at
all, just change some of the components for higher capacity or
performance. This will also let you stress test the new cluster pair
next to your production setup before you migrate the VMs over to the
new setup and then move the old offsite.
One warning is that you will need to make sure that the link between
the two sites has enough bandwidth and low enough RTT so that you can
properly replicate between them, esp if the end users will be
generating a bunch of data that changes alot.
Adam> The target usage as above is an iSCSI server as the backend to a bunch
Adam> of VM's. Currently I have two identical storage servers, using 7 x SSD
Adam> with Linux MD Raid, then using LVM to divide it up for each VM, and then
Adam> DRBD on top to sync the two servers together, on the top is ietd to
Adam> share the multiple DRBD devices out. The two servers have a single
Adam> 10Gbps connection between them for DRBD to sync the data. They also have
Adam> a second 10Gbps ethernet for iscsi to use, with a pair of 1Gbps for
Adam> management (on board). I have 8 x PC's running Xen with 2 x 1Gbps
Adam> ethernet for iSCSI and one 1Gbps ethernet for the "user"/management LAN.
Adam> Current hardware of the storage servers are:
Adam> 7 x Intel 480GB SSD Model SSDSC2CW480A3
Adam> 1 x Intel 180GB SSD Model SSDSC2CT180A4 (for the OS)
Adam> 1 x LSI Logic SAS2308 PCI-Express (8 x SATA connections)
Adam> 1 x Intel Dual port 10Gbps 82599EB SFI/SFP+ Ethernet
Adam> 1 x Intel Xeon CPU E3-1230 V2 @ 3.30GHz
Adam> Motherboard Intel S1200
Adam> http://ark.intel.com/products/67494/Intel-Server-Board-S1200BTLR
Adam> What I'm hoping to achieve is to purchase two new (identical) servers,
Adam> using current recommended (and well supported for the new few years)
Adam> parts, and then move the two existing servers to a remote site,
Adam> combining with DRBD proxy to give a full, "live" off-site backup
Adam> solution. (Note, by backup I mean Disaster Recovery, not backup).
Adam> I would also like to be able to grow the total size of the data further
Adam> if needed, currently I have 7 x 480G in RAID5, which is likely somewhat
Adam> sub-optimal. Options include moving to larger size SSD, or at perhaps
Adam> splitting into 2 x RAID5 arrays. The advantage of larger SSD's would be
Adam> a smaller "system", with lower complexity, while using more smaller
Adam> drives would provide (potentially) better performance, since each drive
Adam> (regardless of size) has the same overall performance (both throughput
Adam> and IOPS).
Adam> I would appreciate any advise or suggestions you can make to help me
Adam> avoid the many mistakes I made last time.
Adam> Regards,
Adam> Adam
Adam> --
Adam> Adam Goryachev
Adam> Website Managers
Adam> www.websitemanagers.com.au
Adam> --
Adam> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
Adam> the body of a message to majordomo@vger.kernel.org
Adam> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Wierd: Degrading while recovering raid5
From: Kyle Logue @ 2015-02-10 4:20 UTC (permalink / raw)
To: linux-raid
Hey all:
I have a 5 disk software raid5 that was working fine until I decided
to swap out an old disk with a new one.
mdadm /dev/md0 --add /dev/sda1
mdadm /dev/md0 --fail /dev/sde1
At this point it started automatically rebuilding the array.
About 60%? of the way in it stops and I see a lot of this repeated in my dmesg:
[Mon Feb 9 18:06:48 2015] ata5.00: exception Emask 0x0 SAct 0x0 SErr
0x0 action 0x6 frozen
[Mon Feb 9 18:06:48 2015] ata5.00: failed command: SMART
[Mon Feb 9 18:06:48 2015] ata5.00: cmd
b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 7
[Mon Feb 9 18:06:48 2015] res
40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
[Mon Feb 9 18:06:48 2015] ata5.00: status: { DRDY }
[Mon Feb 9 18:06:48 2015] ata5: hard resetting link
[Mon Feb 9 18:06:58 2015] ata5: softreset failed (1st FIS failed)
[Mon Feb 9 18:06:58 2015] ata5: hard resetting link
[Mon Feb 9 18:07:08 2015] ata5: softreset failed (1st FIS failed)
[Mon Feb 9 18:07:08 2015] ata5: hard resetting link
[Mon Feb 9 18:07:12 2015] ata5: SATA link up 1.5 Gbps (SStatus 113
SControl 310)
[Mon Feb 9 18:07:12 2015] ata5.00: configured for UDMA/33
[Mon Feb 9 18:07:12 2015] ata5: EH complete
ata5 corresponds to my /dev/sdc drive.
So I was worried but it didn't look so terrible when i did examine:
sudo mdadm --examine /dev/sd[dabfec]1 | egrep 'dev|Update|Role|State|Events'
/dev/sda1:
State : clean
Update Time : Sun Feb 8 20:43:27 2015
Device Role : spare
Array State : .A.AA ('A' == active, '.' == missing)
Events : 27009
/dev/sdb1:
State : clean
Update Time : Sun Feb 8 20:43:27 2015
Device Role : Active device 4
Array State : .A.AA ('A' == active, '.' == missing)
Events : 27009
/dev/sdc1:
State : clean
Update Time : Sun Feb 8 20:21:13 2015
Device Role : Active device 0
Array State : AAAAA ('A' == active, '.' == missing)
Events : 26995
/dev/sdd1:
State : clean
Update Time : Sun Feb 8 20:43:27 2015
Device Role : Active device 1
Array State : .A.AA ('A' == active, '.' == missing)
Events : 27009
/dev/sde1:
State : clean
Update Time : Sun Feb 8 12:17:10 2015
Device Role : Active device 2
Array State : AAAAA ('A' == active, '.' == missing)
Events : 21977
/dev/sdf1:
State : clean
Update Time : Sun Feb 8 20:43:27 2015
Device Role : Active device 3
Array State : .A.AA ('A' == active, '.' == missing)
Events : 27009
So the event counts looked pretty close on the drives I was updating, so I did:
mdadm --stop /dev/md0
mdadm --assemble --force /dev/md0 /dev/sd[dabfec]1
But it stopped again during recovery at some point while at work with
the same ATA errors in the dmesg.
Searching the web for these errors show lots of people having this
issue with various linux distros and laying the blame on everything
from faulty SATA cables to BIOS to NVIDIA drivers - nothing
definitive. I powered off my box and reconnected all my SATA cables as
a sanity check.
I tried --assemble --force again and it got to 70%:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid5 sdc1[7] sda1[8] sdb1[6] sdf1[4] sdd1[5]
7814047744 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UU_UU]
[=============>.......] recovery = 68.9%
(1347855508/1953511936) finish=306.1min speed=32967K/sec
...but died again. I was monitoring dmesg like a hawk this time and
saw those ata5 errors every 3-15 minutes with different cmd and res
values. At the very end I got this:
[Mon Feb 9 23:11:01 2015] ata5.00: configured for UDMA/33
[Mon Feb 9 23:11:01 2015] sd 4:0:0:0: [sdc] Unhandled sense code
[Mon Feb 9 23:11:01 2015] sd 4:0:0:0: [sdc]
[Mon Feb 9 23:11:01 2015] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[Mon Feb 9 23:11:01 2015] sd 4:0:0:0: [sdc]
[Mon Feb 9 23:11:01 2015] Sense Key : Medium Error [current] [descriptor]
[Mon Feb 9 23:11:01 2015] Descriptor sense data with sense
descriptors (in hex):
[Mon Feb 9 23:11:01 2015] 72 03 11 04 00 00 00 0c 00 0a 80 00
00 00 00 00
[Mon Feb 9 23:11:01 2015] a4 1c 1d e8
[Mon Feb 9 23:11:01 2015] sd 4:0:0:0: [sdc]
[Mon Feb 9 23:11:01 2015] Add. Sense: Unrecovered read error - auto
reallocate failed
[Mon Feb 9 23:11:01 2015] sd 4:0:0:0: [sdc] CDB:
[Mon Feb 9 23:11:01 2015] Read(10): 28 00 a4 1c 1d e8 00 00 80 00
[Mon Feb 9 23:11:01 2015] end_request: I/O error, dev sdc, sector 2753306088
[Mon Feb 9 23:11:01 2015] md/raid:md0: Disk failure on sdc1, disabling device.
[Mon Feb 9 23:11:01 2015] md/raid:md0: Operation continuing on 3 devices.
[Mon Feb 9 23:11:01 2015] ata5: EH complete
[Mon Feb 9 23:11:01 2015] md: md0: recovery interrupted.
[Mon Feb 9 23:11:01 2015] RAID conf printout:
[Mon Feb 9 23:11:01 2015] --- level:5 rd:5 wd:3
[Mon Feb 9 23:11:01 2015] disk 0, o:0, dev:sdc1
[Mon Feb 9 23:11:01 2015] disk 1, o:1, dev:sdd1
[Mon Feb 9 23:11:01 2015] disk 2, o:1, dev:sda1
[Mon Feb 9 23:11:01 2015] disk 3, o:1, dev:sdf1
[Mon Feb 9 23:11:01 2015] disk 4, o:1, dev:sdb1
[Mon Feb 9 23:11:01 2015] RAID conf printout:
[Mon Feb 9 23:11:01 2015] --- level:5 rd:5 wd:3
[Mon Feb 9 23:11:01 2015] disk 1, o:1, dev:sdd1
[Mon Feb 9 23:11:01 2015] disk 2, o:1, dev:sda1
[Mon Feb 9 23:11:01 2015] disk 3, o:1, dev:sdf1
[Mon Feb 9 23:11:01 2015] disk 4, o:1, dev:sdb1
[Mon Feb 9 23:11:01 2015] RAID conf printout:
[Mon Feb 9 23:11:01 2015] --- level:5 rd:5 wd:3
[Mon Feb 9 23:11:01 2015] disk 1, o:1, dev:sdd1
[Mon Feb 9 23:11:01 2015] disk 2, o:1, dev:sda1
[Mon Feb 9 23:11:01 2015] disk 3, o:1, dev:sdf1
[Mon Feb 9 23:11:01 2015] disk 4, o:1, dev:sdb1
[Mon Feb 9 23:11:01 2015] RAID conf printout:
[Mon Feb 9 23:11:01 2015] --- level:5 rd:5 wd:3
[Mon Feb 9 23:11:01 2015] disk 1, o:1, dev:sdd1
[Mon Feb 9 23:11:01 2015] disk 3, o:1, dev:sdf1
[Mon Feb 9 23:11:01 2015] disk 4, o:1, dev:sdb1
and mdstat now has:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid5 sdc1[7](F) sda1[8](S) sdb1[6] sdf1[4] sdd1[5]
7814047744 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/3] [_U_UU]
And now I am out of ideas. Any thoughts on correcting those ata5
errors? or skipping those sectors maybe? While sde1 is the disk i
manually failed, it hasn't been touched yet. The event count is way
off now, but maybe I can use that somehow? Should i replace the sata
cable for sdc and retry?
Anybody in DC want a beer on me for helping figure this out? I have
more log files stored, but was trying to keep it short.
Thanks for looking,
Kyle L
PS. mdadm v3.2.5 on Ubuntu 14.04 running linux 3.13.0-45
PPS. Last full backup was six months ago. Hmm.
^ permalink raw reply
* Re: Upgrading storage server
From: Adam Goryachev @ 2015-02-10 7:22 UTC (permalink / raw)
To: John Stoffel; +Cc: linux-raid
In-Reply-To: <21721.30729.509462.974269@quad.stoffel.home>
On 10/02/15 14:16, John Stoffel wrote:
> Adam> After making a whole string of mistakes in building a iSCSI
> Adam> server about 2 years ago, I'm now looking to replace it without
> Adam> all the wrong turns/mistakes. I was hoping you could all offer
> Adam> some advice on hardware selection/choices.
>
> I remember those discussions, they were quite informative and it was
> interesting seeing Stan help you out. Now that you've got this system
> working well, or at least well enough, what is the biggest remaining
> problem you have?
The only current requirement is to get some sort of DR
configuration/setup in place, that doesn't involve restoring from
backups. Performance is satisfactory right now, so I don't want to hit
any new performance issues in the process.
> I've become a big fan of supermicro FatTwin systems, and they might be
> what you want here for your setup. But I'd also think about maybe you
> want to go to fewer larger PCIe SSD cards in mirrored pairs instead
> for better performance. Or is performance a problem still?
The users are satisfied with the current performance level, though I
suspect if performance could be improved without drastically increasing
the cost, then it would make sense as well.
Those FatTwin systems look pretty awesome, but since I only need two
systems (nodes) and ideally I want one in each rack, then it doesn't
quite work out. I tend to prefer white box systems, due to it being
easier to find replacement parts, and I am avoiding too much redundancy
within each system (eg dual power, raid6, etc) as I am relying on the
second node to take over, allowing the primary to be repaired and added
back in later.
> There's also *alot* to be said for simply replicating what you have,
> but with larger SSDs, say 1Tb each, and keeping the rest of the system
> and config exactly the same. Limit the changes, esp since you went
> through so much pain before.
That was my thoughts, although I assume motherboards, CPU's, and perhaps
SATA controller cards have changed a lot over the past 3 years (although
I note that Intel suggests the motherboard isn't EOL until this year).
I'd prefer to get current models of hardware so that they will be well
supported (ie, replacements are easy to get) for the next few years.
Basically, at the same time as adding DR capability, I will be
refreshing the model of hardware. I suppose repeating this process every
3 years means that the DR hardware will be up to 6 years old, which is
probably still satisfactory (unless I see a lot of failures there),
considering that there is still a replicated pair (as long as they don't
both fail at the same time, or lose 2 "disks" each at the same time).
> Now I might also think about upgrading all the clients to 10Gb as
> well, and just moving to a completely 10G network if possible. I seem
> to remember that you didn't have any way to throttle or setup Quality
> of Service limits on your iSCSI vs. other network traffic, which is
> why you ended up splitting up the traffic like this, so that a single
> VM couldn't bring the rest to their knees when a user did something
> silly.
Well, I split the iSCSI SAN and the user LAN partly to satisfy "best
practice", improve security, as well as obviously
performance/reliability. I don't think I'll upgrade all the VM servers
to 10G at this stage (aren't planning to replace them all for another 6
months or more). At that stage, it might be something to consider, but I
would still be concerned about one VM "hogging" all the disk bandwidth.
Perhaps in practice, it wouldn't be an issue, since it is more IOPS that
is the limiting factor, and you can steal all available IOPS without
using very much bandwidth. This will likely depend on the cost/ability
to get a 16 port (or minimum of 10port) 10Gbps switch. Maybe something
like this:
http://www.netgear.com.au/business/products/switches/smart/10g-smart-switch.aspx#tab-overview
at approx AUD$1800
> So again, if it's working well now, don't chage your architecture at
> all, just change some of the components for higher capacity or
> performance. This will also let you stress test the new cluster pair
> next to your production setup before you migrate the VMs over to the
> new setup and then move the old offsite.
I'm haven't properly thought about how to do the migration, but I would
think I can bring up one of the new servers, and replace the current
"secondary" in the DRBD. Then, when that has settled in (for a week or
so), I can flip it to become the primary. Again, allow to test for a
week or so (any issues I can easily flip it back to secondary and so
revert back to the known good status), and then remove the second old
server, and replace with the second new server. Finally, reconfigure
both old servers onsite with the DRBD proxy config. Once that is working
well, (and obviously all the data is synced up to date) I can move them
offsite.
> One warning is that you will need to make sure that the link between
> the two sites has enough bandwidth and low enough RTT so that you can
> properly replicate between them, esp if the end users will be
> generating a bunch of data that changes alot.
Yep, that is something I'm looking into at the moment. Supposedly with
drbdproxy, as long as the changed data per day is less than the
bandwidth per day, then it should work. Also, this will relate to how
much RAM is available on the drbdproxy node to cache the changes.
I'm actually struggling a little with getting the "right" data for this.
Currently, I'm pulling all the data from /proc/drbd into an RRD file
(for each drbd device). Hopefully I'm a little crazy, but if I do this:
rrdtool fetch ${i} AVERAGE -s -25h |grep -v nan|tail -288| cut -d' '
-f3| awk '{s+=$1}END{print s}'
Which should select the past 25 hours worth of 5 minute averages, then
remove the unknowns at the end (because the rrd file is only updated
every 30 minutes, the values are cached), and then pick only the last 24
hours of reports (288), pick out the nr (network read) value, sum all
those to get the total of the 5 minute average data read over the
network (by the secondary). Finally, I multiply this by 300 to get the
actual data transferred. (Assuming that a 5 minute average is 3MB/s,
therefore the original amount of data transferred is 3 x 300 = 900MB in
5minutes).
The problem is I got an answer of over 200GB, which isn't going to fit
on my WAN (max 10Mbps, or 1MB/s), unless I upgrade the WAN, or
compression works really well, or my calculations are entirely wrong.....
Anyway, this section is somewhat off-topic for here. I'll follow up the
DRBD side elsewhere.
Thanks for your comments/suggestions.
Regards,
Adam
> Adam> The target usage as above is an iSCSI server as the backend to a bunch
> Adam> of VM's. Currently I have two identical storage servers, using 7 x SSD
> Adam> with Linux MD Raid, then using LVM to divide it up for each VM, and then
> Adam> DRBD on top to sync the two servers together, on the top is ietd to
> Adam> share the multiple DRBD devices out. The two servers have a single
> Adam> 10Gbps connection between them for DRBD to sync the data. They also have
> Adam> a second 10Gbps ethernet for iscsi to use, with a pair of 1Gbps for
> Adam> management (on board). I have 8 x PC's running Xen with 2 x 1Gbps
> Adam> ethernet for iSCSI and one 1Gbps ethernet for the "user"/management LAN.
>
> Adam> Current hardware of the storage servers are:
> Adam> 7 x Intel 480GB SSD Model SSDSC2CW480A3
> Adam> 1 x Intel 180GB SSD Model SSDSC2CT180A4 (for the OS)
> Adam> 1 x LSI Logic SAS2308 PCI-Express (8 x SATA connections)
> Adam> 1 x Intel Dual port 10Gbps 82599EB SFI/SFP+ Ethernet
> Adam> 1 x Intel Xeon CPU E3-1230 V2 @ 3.30GHz
> Adam> Motherboard Intel S1200
> Adam> http://ark.intel.com/products/67494/Intel-Server-Board-S1200BTLR
>
> Adam> What I'm hoping to achieve is to purchase two new (identical) servers,
> Adam> using current recommended (and well supported for the new few years)
> Adam> parts, and then move the two existing servers to a remote site,
> Adam> combining with DRBD proxy to give a full, "live" off-site backup
> Adam> solution. (Note, by backup I mean Disaster Recovery, not backup).
>
> Adam> I would also like to be able to grow the total size of the data further
> Adam> if needed, currently I have 7 x 480G in RAID5, which is likely somewhat
> Adam> sub-optimal. Options include moving to larger size SSD, or at perhaps
> Adam> splitting into 2 x RAID5 arrays. The advantage of larger SSD's would be
> Adam> a smaller "system", with lower complexity, while using more smaller
> Adam> drives would provide (potentially) better performance, since each drive
> Adam> (regardless of size) has the same overall performance (both throughput
> Adam> and IOPS).
>
> Adam> I would appreciate any advise or suggestions you can make to help me
> Adam> avoid the many mistakes I made last time.
>
--
Adam Goryachev
Website Managers
P: +61 2 8304 0000 adam@websitemanagers.com.au
F: +61 2 8304 0001 www.websitemanagers.com.au
^ permalink raw reply
* Re: Wierd: Degrading while recovering raid5
From: Adam Goryachev @ 2015-02-10 7:35 UTC (permalink / raw)
To: Kyle Logue, linux-raid
In-Reply-To: <CAP7a4UQCB=jdf7=sz8MoYL+WGbMbT_09_xL460DLX-epLAS0Sw@mail.gmail.com>
Hi Kyle,
There are other people who will jump in and help you with your problem,
but I'll add a couple of pointers while you are waiting. See below.
On 10/02/15 15:20, Kyle Logue wrote:
> Hey all:
>
> I have a 5 disk software raid5 that was working fine until I decided
> to swap out an old disk with a new one.
>
> mdadm /dev/md0 --add /dev/sda1
> mdadm /dev/md0 --fail /dev/sde1
>
> At this point it started automatically rebuilding the array.
> About 60%? of the way in it stops and I see a lot of this repeated in my dmesg:
>
> [Mon Feb 9 18:06:48 2015] ata5.00: exception Emask 0x0 SAct 0x0 SErr
> 0x0 action 0x6 frozen
> [Mon Feb 9 18:06:48 2015] ata5.00: failed command: SMART
> [Mon Feb 9 18:06:48 2015] ata5.00: cmd
> b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 7
> [Mon Feb 9 18:06:48 2015] res
> 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
> [Mon Feb 9 18:06:48 2015] ata5.00: status: { DRDY }
> [Mon Feb 9 18:06:48 2015] ata5: hard resetting link
> [Mon Feb 9 18:06:58 2015] ata5: softreset failed (1st FIS failed)
> [Mon Feb 9 18:06:58 2015] ata5: hard resetting link
> [Mon Feb 9 18:07:08 2015] ata5: softreset failed (1st FIS failed)
> [Mon Feb 9 18:07:08 2015] ata5: hard resetting link
> [Mon Feb 9 18:07:12 2015] ata5: SATA link up 1.5 Gbps (SStatus 113
> SControl 310)
> [Mon Feb 9 18:07:12 2015] ata5.00: configured for UDMA/33
> [Mon Feb 9 18:07:12 2015] ata5: EH complete
>
> ata5 corresponds to my /dev/sdc drive.
First, check if the drive is faulty.
dd if=/dev/sdc of=/dev/null bs=10M
If that completes without any errors from dd, then the drive can be read
OK. Now check the logs, was there any errors there? Especially if there
were errors in the logs, (or even if not) read about timing mismatches
between the kernel and the hard drive, and how to solve that. There was
another post earlier today with some links to specific posts that will
be helpful (check the online archive).
Finally, I think your first mistake was to fail the drive. You should
have replaced it which will stop you from losing protection from a
failed drive.
See the second answer to this question:
http://unix.stackexchange.com/questions/74924/how-to-safely-replace-a-not-yet-failed-disk-in-a-linux-raid5-array
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-10 9:29 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150210135017.7659e49c@notabene.brown>
On Tue, Feb 10, 2015 at 01:50:17PM +1100, NeilBrown wrote:
> On Mon, 9 Feb 2015 10:10:00 +0100 Peter Zijlstra <peterz@infradead.org> wrote:
> > > However, when io_schedule() explicitly calls blk_flush_plug(), then
> > > @from_schedule=false variant is used, and the unplug functions are allowed to
> > > allocate memory and block and maybe even call mempool_alloc() which might
> > > call io_schedule().
> > >
> > > This shouldn't be a problem as blk_flush_plug() spliced out the plug list, so
> > > any recursive call will find an empty list and do nothing.
> >
> > Unless, something along the way stuck something back on, right? So
> > should we stick an:
> >
> > WARN_ON(current->in_iowait);
> >
> > somewhere near where things are added to this plug list? (and move the
> > blk_flush_plug() call inside of where that's actually true of course).
>
> No, I don't think so.
>
> It is certainly possible that some request on plug->cb_list could add
> something to plug->list - which is processed after ->cb_list.
>
> I think the best way to think about this is that the *problem* was that a
> wait_event loop could spin without making any progress. So any time that
> clear forward progress is made it is safe sleep without necessitating the
> warning. Hence sched_annotate_sleep() is reasonable.
> blk_flush_plug() with definitely have dispatched some requests if it
> might_sleep(), so the sleep is OK.
Well, yes, but you forget that this gets us back into recursion land.
io_schedule() calling io_schedule() calling io_schedule() and *boom*
stack overflow -> dead machine.
We must either guarantee io_schedule() will never call io_schedule() or
that io_schedule() itself will not add new work to the current plug such
that calling io_schedule() itself will not recurse on the blk stuff.
Pick either option, but pick one.
Without providing such a guarantee I'm not comfortable making this warn
go away.
^ permalink raw reply
* Re: RAID1 might_sleep() warning on 3.19-rc7
From: Peter Zijlstra @ 2015-02-10 11:01 UTC (permalink / raw)
To: NeilBrown; +Cc: Tony Battersby, linux-raid, lkml, axboe, Linus Torvalds
In-Reply-To: <20150210092936.GW21418@twins.programming.kicks-ass.net>
On Tue, Feb 10, 2015 at 10:29:36AM +0100, Peter Zijlstra wrote:
> On Tue, Feb 10, 2015 at 01:50:17PM +1100, NeilBrown wrote:
> > On Mon, 9 Feb 2015 10:10:00 +0100 Peter Zijlstra <peterz@infradead.org> wrote:
> > > > However, when io_schedule() explicitly calls blk_flush_plug(), then
> > > > @from_schedule=false variant is used, and the unplug functions are allowed to
> > > > allocate memory and block and maybe even call mempool_alloc() which might
> > > > call io_schedule().
Note that as it stands recursively calling io_schedule() is already
broken. Things like delayacct_blkio_{start,end}() do not nest properly.
^ permalink raw reply
* Re: Wierd: Degrading while recovering raid5
From: Phil Turmel @ 2015-02-10 13:51 UTC (permalink / raw)
To: Adam Goryachev, Kyle Logue, linux-raid
In-Reply-To: <54D9B4AD.8010204@websitemanagers.com.au>
Hi Kyle,
Your symptoms look like classic timeout mismatch. Details interleaved.
On 02/10/2015 02:35 AM, Adam Goryachev wrote:
> There are other people who will jump in and help you with your problem,
> but I'll add a couple of pointers while you are waiting. See below.
> On 10/02/15 15:20, Kyle Logue wrote:
>> Hey all:
>>
>> I have a 5 disk software raid5 that was working fine until I decided
>> to swap out an old disk with a new one.
>>
>> mdadm /dev/md0 --add /dev/sda1
>> mdadm /dev/md0 --fail /dev/sde1
As Adam pointed out, you should have used --replace, but you probably
wouldn't have made it through the replace function anyways.
>> At this point it started automatically rebuilding the array.
>> About 60%? of the way in it stops and I see a lot of this repeated in
>> my dmesg:
>>
>> [Mon Feb 9 18:06:48 2015] ata5.00: exception Emask 0x0 SAct 0x0 SErr
>> 0x0 action 0x6 frozen
>> [Mon Feb 9 18:06:48 2015] ata5.00: failed command: SMART
>> [Mon Feb 9 18:06:48 2015] ata5.00: cmd
>> b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 7
>> [Mon Feb 9 18:06:48 2015] res
>> 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
^^^^^^^^^
Smoking gun.
>> [Mon Feb 9 18:06:48 2015] ata5.00: status: { DRDY }
>> [Mon Feb 9 18:06:48 2015] ata5: hard resetting link
>> [Mon Feb 9 18:06:58 2015] ata5: softreset failed (1st FIS failed)
>> [Mon Feb 9 18:06:58 2015] ata5: hard resetting link
>> [Mon Feb 9 18:07:08 2015] ata5: softreset failed (1st FIS failed)
>> [Mon Feb 9 18:07:08 2015] ata5: hard resetting link
>> [Mon Feb 9 18:07:12 2015] ata5: SATA link up 1.5 Gbps (SStatus 113
>> SControl 310)
>> [Mon Feb 9 18:07:12 2015] ata5.00: configured for UDMA/33
>> [Mon Feb 9 18:07:12 2015] ata5: EH complete
Notice that after a timeout error, the drive is unresponsive for several
more seconds -- about 24 in your case.
> .... read about timing mismatches
> between the kernel and the hard drive, and how to solve that. There was
> another post earlier today with some links to specific posts that will
> be helpful (check the online archive).
That would have been me. Start with this link for a description of what
you are experiencing:
http://marc.info/?l=linux-raid&m=135811522817345&w=1
First, you need to protect yourself from timeout mismatch due to the use
of desktop-grade drives. (Enterprise and raid-rated drives don't have
this problem.)
{ If you were stuck in the middle of a replace a you had just
worked-around your timeout problem, it would likely continue and
complete. You've lost that opportunity. }
Show us the output of "smartctl -x" for all of your drives if you'd like
advice on your particular drives. (Pasted inline is preferred.)
Second, you need to find and overwrite (with zeros) the bad sectors on
your drives. Or ddrescue to a complete set of replacement drives and
assemble those.
Third, you need to set up a cron job to scrub your array regularly to
clean out UREs before they accumulate beyond MD's ability to handle it
(20 read errors in an hour, 10 per hour sustained).
Phil
^ permalink raw reply
* ignoring %s as it reports %s as failed
From: Valentijn @ 2015-02-10 15:54 UTC (permalink / raw)
To: linux-raid
Hello list,
This is a repost - I sent it to linux-raid back in January, but you were
all still having holidays - or hangovers or anything.
I was wondering why "--assemble --force" didn't work for a particular
(failed) raid-device. I got it to work and the data is safe now, but the
question about the non-working assembly remains.
I was trying to resurrect an array from a Synology disk station. As
follows: raid5 array with 4 disks, one disk utterly broken
(click-clack-click), one disk thrown out of the array.
Please note, that I used dmsetup to create an overlay disk, so the
"/dev/mapper/diskXp5" actually points to a /dev/sdN5 which is the
original array; I'm trying to not harm the original data. Here is the
result of mdadm v3.2.5 running assemble+force:
~# mdadm --verbose --assemble --force /dev/md99 /dev/mapper/disk1p5
/dev/mapper/disk2p5 /dev/mapper/disk3p5
mdadm: looking for devices for /dev/md99
mdadm: /dev/mapper/disk1p5 is identified as a member of /dev/md99, slot
32768.
mdadm: /dev/mapper/disk2p5 is identified as a member of /dev/md99, slot 1.
mdadm: /dev/mapper/disk3p5 is identified as a member of /dev/md99, slot 2.
mdadm: ignoring /dev/mapper/disk2p5 as it reports /dev/mapper/disk1p5 as
failed
mdadm: ignoring /dev/mapper/disk3p5 as it reports /dev/mapper/disk1p5 as
failed
mdadm: No suitable drives found for /dev/md99
OK. http://permalink.gmane.org/gmane.linux.raid/45064 says I should use
mdadm 3.3+, and luckily I had v3.3.2 - 21st August lying around, so
there we go:
~# ./mdadm --verbose --assemble --force /dev/md99 /dev/mapper/disk1p5
/dev/mapper/disk2p5 /dev/mapper/disk3p5
mdadm: looking for devices for /dev/md99
mdadm: /dev/mapper/disk1p5 is identified as a member of /dev/md99, slot
32768.
mdadm: /dev/mapper/disk2p5 is identified as a member of /dev/md99, slot 1.
mdadm: /dev/mapper/disk3p5 is identified as a member of /dev/md99, slot 2.
mdadm: no uptodate device for slot 0 of /dev/md99
mdadm: added /dev/mapper/disk3p5 to /dev/md99 as 2
mdadm: no uptodate device for slot 6 of /dev/md99
mdadm: added /dev/mapper/disk2p5 to /dev/md99 as 1
mdadm: /dev/md99 assembled from 2 drives - not enough to start the array.
Hmm, that doesn't help us either. (Slight difference between the two
versions, 3.2.5 deploys a broken array named md99 - which you can stop
afterwards - and 3.3.2 seems to auto-remove the array after trying to
start it - not sure if I describe the inner workings correctly)
Disk1 reports:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 8ad57063:cf592975:9fabcc2b:13922e19
Name : 2
Creation Time : Sun May 13 08:23:21 2012
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 1944057456 (927.00 GiB 995.36 GB)
Array Size : 2916086016 (2781.00 GiB 2986.07 GB)
Used Dev Size : 1944057344 (927.00 GiB 995.36 GB)
Data Offset : 272 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 4ae571dc:3c458e25:d219b536:c8b9a50e
Update Time : Sun Dec 28 10:41:55 2014
Checksum : 9a35a9ae - correct
Events : 744446
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 32768
Array State : .AA. ('A' == active, '.' == missing)
Disk2:
[...]
State : clean
Device UUID : b381cfd4:d2474cc4:e4c0ad21:8565b2a5
Update Time : Sun Dec 28 10:41:55 2014
Checksum : b3569b7e - correct
Events : 744446
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 1
Array State : .AA. ('A' == active, '.' == missing)
Disk3:
[...]
State : clean
Device UUID : c00d4418:c8313607:fb3a2018:d267a693
Update Time : Sun Dec 28 10:41:55 2014
Checksum : 1e1b8de5 - correct
Events : 744446
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 2
Array State : .AA. ('A' == active, '.' == missing)
So the "Active device 32768" seems to be the culprit here, is that correct?
What would be the correct way to reassemble this array?
I managed to get it *working* again by recreating it with:
./mdadm --create --assume-clean --data-offset=136 -e 1.2 --level=5
--bitmap=none --raid-devices=4 --chunk=64 /dev/md99 /dev/mapper/disk1p5
/dev/mapper/disk2p5 /dev/mapper/disk3p5 missing
... but that feels a bit rude.
So, is there a "proper" way to reassemble this array?
For the record: the original array was most likely created on the
Synology disk station itself, which currently contains mdadm - v2.6.7 (I
don't know if that is the version that created the array). Also, I'm not
sure about the kernel version that's running there, but /lib/modules
contains modules that have a string "vermagic=2.6.15 ARMv5 gcc-3.4"
inside, so that might be the kernel version that was running when things
went wrong.
Best regards,
Valentijn
^ permalink raw reply
* Re: [PATCH 00/24] Clustered MD RAID1
From: Goldwyn Rodrigues @ 2015-02-10 17:00 UTC (permalink / raw)
To: NeilBrown; +Cc: lzhong, linux-raid
In-Reply-To: <20150206133952.173f1975@notabene.brown>
Hi Neil,
>
>
> hi Goldwyn,
> thanks for these - and sorry for the long delay. Lots of leave over
> southern summer, and the lots of email etc to deal with.
>
> This patch set is very close and I am tempted to just apply it and then
> fix things up with subsequent patches. In order to allow that, could you
> please:
> - rebase against current upstream
> - fix the checkpatch.pl errors and warnings.
> The "WARNING: line over 80 characters" are often a judgement call
> so I'm not particularly worried about those. Most, if not all, of
> the others should be followed just to have consistent layout.
Done.
>
> Then I'll queue them up for 3.21, providing I don't find anything that would
> hurt non-cluster usage ....
> On that topic: why initialise rv to -EINVAL in "metadata_update sends
> message...". That looks wrong.
Yes, this is fixed.
>
> I noticed that a number of times a patch will revert something that a
> previous patch added. It would be much nicer to fold these changes back into
> the original patch. Often this is just extra blank lines, but occasionally
> variable names are changed (md -> mddev). It should be given the final name
> when introduced. Every chunk in every patch should be directly relevant to
> that patch.
I have cross-checked this and I did not find anything with respect to
variable names. I did some cleanup with respect to the code though.
There is one instance where I have used a variable: cluster_setup_done
and then removed it. I think this is required to understand the patch
and a smooth transition to subsequent patches. However, if you want me
to aggressively remove that part, I should be able to do that.
>
> Some other issues, that could possibly be fixed up afterwards:
>
> - Is a clustername 64 bytes or 63 bytes? I would have thought 64,
> but the use of strlcpy make is 63 plus a nul. Is that really what is
> wanted?
Yes, it is 64 bytes. I haven't fixed this as yet.
>
> - Based on https://lkml.org/lkml/2012/10/23/580 it might be good to add
> "default n" to Kconfig, and possible add a WARN() if anyone tries to use
> the code.
Done. Added pr_warn while loading the module.
>
> - I'm a bit concerned about the behaviour on node failure.
> When a node fails, two things must happen w.r.t the bits in that node's
> bitmap.
> 1/ The corresponding regions of the array need to be resynced. You do have
> code to do this.
> 2/ Other nodes must avoid read-balancing on those regions until the
> resync has completed.
>
> You do have code for this second bit, but it looks wrong. It avoids
> read-balancing if ->area_resyncing(). That isn't sufficient.
> The "area_resyncing" is always (I assume) a relatively small region of
> the array which will be completely resynced quite quickly. It must be
> because writes are blocked to this area. However the region in which
> we must disable re-balancing can be much larger. It covers *all* bits
> that are set in any unsynced bitmap. So it isn't just the area that is
> currently being synced, but all areas that will be synced.
What are unsynced bitmaps? Are they bitmaps which are associated with an
active node or dirty bitmaps with dead nodes? If it is the former, I
agree this is not enough. If it is latter, all nodes maintain a linked
list of all the nodes which are currently performing resync (probably
because of multiple nodes died simultaneously). One node performs the
recovery (aka bitmap resync) of exactly one "dead" node at a time.
area_resyncing goes through all the nodes which are performing resync.
>
> - I think md_reload_sb() might be too simple. It probably should check that
> nothing serious has changed. The "mddev->raid_disks = 0" look suspicious.
> I'll have to think about this a bit more.
Yes, I get that feeling as well. However, I am not sure how to perform
an exact comparison to understand what has changed. Perhaps it needs a
new flag?
>
> That's all I can see for now. I'll have another look once I have it all in my tree.
>
I have put all the changes in my git:
https://github.com/goldwynr/linux
The branch cluster-md is against the latest upstream. I also performed a
small sanity test to check everything is working properly.
Let me know if you would want me to repost the entire patchset to the
mailing list.
--
Goldwyn
^ permalink raw reply
* Re: Wierd: Degrading while recovering raid5
From: Kyle Logue @ 2015-02-10 21:50 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <54DA0CDA.2010800@turmel.org>
Phil:
Thanks for your detailed response. That link does seem to describe my
problem and I do understand that desktop grade drives are sub-optimal.
It was many years ago when I first set up this array on my home
theater pc. Until now I had no idea about the cron job - I'll make
sure to implement that. I am preparing to move to 6 tb disks sometime
soon and i'll definitely go enterprise this time.
Regarding the drive timeout: I understand that I need to increase it
from 30 seconds to something larger (2+ min) but am unaware how to do
this. Is it a kernel variable? I'll keep googling but this seems like
it's whats going to save me.
tl;dr: How do I change the drive timeout?
Here is the smartctl -x for all my drives:
Reminder: SDA is the new drive. SDC is the troublemaker. SDE is the
one I failed.
> sudo smartctl -x /dev/sda
> smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-45-generic] (local build)
> Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
> === START OF INFORMATION SECTION ===
> Model Family: Seagate Barracuda 7200.14 (AF)
> Device Model: ST2000DM001-1CH164
> Serial Number: Z340F2SP
> LU WWN Device Id: 5 000c50 064d5887d
> Firmware Version: CC27
> User Capacity: 2,000,398,934,016 bytes [2.00 TB]
> Sector Sizes: 512 bytes logical, 4096 bytes physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b
> SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
> Local Time is: Tue Feb 10 16:37:52 2015 EST
> ==> WARNING: A firmware update for this drive may be available,
> see the following Seagate web pages:
> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
> http://knowledge.seagate.com/articles/en_US/FAQ/223651en
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
> AAM feature is: Unavailable
> APM level is: 254 (maximum performance)
> Rd look-ahead is: Enabled
> Write cache is: Enabled
> ATA Security is: Disabled, NOT FROZEN [SEC1]
> Write SCT (Get) XXX Error Recovery Control Command failed: scsi error aborted command
> Wt Cache Reorder: N/A
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
> General SMART Values:
> Offline data collection status: (0x82) Offline data collection activity
> was completed without error.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: ( 584) seconds.
> Offline data collection
> capabilities: (0x7b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 1) minutes.
> Extended self-test routine
> recommended polling time: ( 212) minutes.
> Conveyance self-test routine
> recommended polling time: ( 2) minutes.
> SCT capabilities: (0x3085) SCT Status supported.
> SMART Attributes Data Structure revision number: 10
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
> 1 Raw_Read_Error_Rate POSR-- 105 099 006 - 9806192
> 3 Spin_Up_Time PO---- 097 097 000 - 0
> 4 Start_Stop_Count -O--CK 100 100 020 - 4
> 5 Reallocated_Sector_Ct PO--CK 100 100 010 - 0
> 7 Seek_Error_Rate POSR-- 100 253 030 - 289070
> 9 Power_On_Hours -O--CK 100 100 000 - 35
> 10 Spin_Retry_Count PO--C- 100 100 097 - 0
> 12 Power_Cycle_Count -O--CK 100 100 020 - 5
> 183 Runtime_Bad_Block -O--CK 099 099 000 - 1
> 184 End-to-End_Error -O--CK 100 100 099 - 0
> 187 Reported_Uncorrect -O--CK 100 100 000 - 0
> 188 Command_Timeout -O--CK 100 100 000 - 0 0 0
> 189 High_Fly_Writes -O-RCK 100 100 000 - 0
> 190 Airflow_Temperature_Cel -O---K 073 062 045 - 27 (Min/Max 25/27)
> 191 G-Sense_Error_Rate -O--CK 100 100 000 - 0
> 192 Power-Off_Retract_Count -O--CK 100 100 000 - 4
> 193 Load_Cycle_Count -O--CK 100 100 000 - 8
> 194 Temperature_Celsius -O---K 027 040 000 - 27 (0 22 0 0 0)
> 197 Current_Pending_Sector -O--C- 100 100 000 - 0
> 198 Offline_Uncorrectable ----C- 100 100 000 - 0
> 199 UDMA_CRC_Error_Count -OSRCK 200 200 000 - 0
> 240 Head_Flying_Hours ------ 100 253 000 - 35h+41m+13.042s
> 241 Total_LBAs_Written ------ 100 253 000 - 11031892416
> 242 Total_LBAs_Read ------ 100 253 000 - 2769646
> ||||||_ K auto-keep
> |||||__ C event count
> ||||___ R error rate
> |||____ S speed/performance
> ||_____ O updated online
> |______ P prefailure warning
> General Purpose Log Directory Version 1
> SMART Log Directory Version 1 [multi-sector log support]
> Address Access R/W Size Description
> 0x00 GPL,SL R/O 1 Log Directory
> 0x01 SL R/O 1 Summary SMART error log
> 0x02 SL R/O 5 Comprehensive SMART error log
> 0x03 GPL R/O 5 Ext. Comprehensive SMART error log
> 0x06 SL R/O 1 SMART self-test log
> 0x07 GPL R/O 1 Extended self-test log
> 0x09 SL R/W 1 Selective self-test log
> 0x10 GPL R/O 1 NCQ Command Error log
> 0x11 GPL R/O 1 SATA Phy Event Counters
> 0x21 GPL R/O 1 Write stream error log
> 0x22 GPL R/O 1 Read stream error log
> 0x80-0x9f GPL,SL R/W 16 Host vendor specific log
> 0xa1 GPL,SL VS 20 Device vendor specific log
> 0xa2 GPL VS 4496 Device vendor specific log
> 0xa8 GPL,SL VS 129 Device vendor specific log
> 0xa9 GPL,SL VS 1 Device vendor specific log
> 0xab GPL VS 1 Device vendor specific log
> 0xb0 GPL VS 5176 Device vendor specific log
> 0xbe-0xbf GPL VS 65535 Device vendor specific log
> 0xc0 GPL,SL VS 1 Device vendor specific log
> 0xc1 GPL,SL VS 10 Device vendor specific log
> 0xc4 GPL,SL VS 5 Device vendor specific log
> 0xe0 GPL,SL R/W 1 SCT Command/Status
> 0xe1 GPL,SL R/W 1 SCT Data Transfer
> SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
> No Errors Logged
> SMART Extended Self-test Log Version: 1 (1 sectors)
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute delay.
> SCT Data Table command not supported
> SCT Error Recovery Control command not supported
> Device Statistics (GP Log 0x04) not supported
> SATA Phy Event Counters (GP Log 0x11)
> ID Size Value Description
> 0x000a 2 6 Device-to-host register FISes sent due to a COMRESET
> 0x0001 2 0 Command failed due to ICRC error
> 0x0003 2 0 R_ERR response for device-to-host data FIS
> 0x0004 2 0 R_ERR response for host-to-device data FIS
> 0x0006 2 0 R_ERR response for device-to-host non-data FIS
> 0x0007 2 0 R_ERR response for host-to-device non-data FIS
>
> sudo smartctl -x /dev/sdb
> smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-45-generic] (local build)
> Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
> === START OF INFORMATION SECTION ===
> Model Family: Seagate Barracuda 7200.14 (AF)
> Device Model: ST2000DM001-1CH164
> Serial Number: S1E1CW9Y
> LU WWN Device Id: 5 000c50 05c085bef
> Firmware Version: CC24
> User Capacity: 2,000,398,934,016 bytes [2.00 TB]
> Sector Sizes: 512 bytes logical, 4096 bytes physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA8-ACS T13/1699-D revision 4
> SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
> Local Time is: Tue Feb 10 16:40:24 2015 EST
> ==> WARNING: A firmware update for this drive may be available,
> see the following Seagate web pages:
> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
> http://knowledge.seagate.com/articles/en_US/FAQ/223651en
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
> AAM feature is: Unavailable
> APM level is: 254 (maximum performance)
> Rd look-ahead is: Enabled
> Write cache is: Enabled
> ATA Security is: Disabled, NOT FROZEN [SEC1]
> Write SCT (Get) XXX Error Recovery Control Command failed: scsi error aborted command
> Wt Cache Reorder: N/A
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
> General SMART Values:
> Offline data collection status: (0x82) Offline data collection activity
> was completed without error.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: ( 584) seconds.
> Offline data collection
> capabilities: (0x7b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 1) minutes.
> Extended self-test routine
> recommended polling time: ( 225) minutes.
> Conveyance self-test routine
> recommended polling time: ( 2) minutes.
> SCT capabilities: (0x3085) SCT Status supported.
> SMART Attributes Data Structure revision number: 10
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
> 1 Raw_Read_Error_Rate POSR-- 117 099 006 - 153090384
> 3 Spin_Up_Time PO---- 096 096 000 - 0
> 4 Start_Stop_Count -O--CK 100 100 020 - 58
> 5 Reallocated_Sector_Ct PO--CK 100 100 010 - 0
> 7 Seek_Error_Rate POSR-- 063 058 030 - 8594213138
> 9 Power_On_Hours -O--CK 084 084 000 - 14743
> 10 Spin_Retry_Count PO--C- 100 100 097 - 0
> 12 Power_Cycle_Count -O--CK 100 100 020 - 58
> 183 Runtime_Bad_Block -O--CK 100 100 000 - 0
> 184 End-to-End_Error -O--CK 100 100 099 - 0
> 187 Reported_Uncorrect -O--CK 100 100 000 - 0
> 188 Command_Timeout -O--CK 100 099 000 - 1 1 1
> 189 High_Fly_Writes -O-RCK 100 100 000 - 0
> 190 Airflow_Temperature_Cel -O---K 072 057 045 - 28 (Min/Max 26/28)
> 191 G-Sense_Error_Rate -O--CK 100 100 000 - 0
> 192 Power-Off_Retract_Count -O--CK 100 100 000 - 34
> 193 Load_Cycle_Count -O--CK 100 100 000 - 110
> 194 Temperature_Celsius -O---K 028 043 000 - 28 (0 18 0 0 0)
> 197 Current_Pending_Sector -O--C- 100 100 000 - 0
> 198 Offline_Uncorrectable ----C- 100 100 000 - 0
> 199 UDMA_CRC_Error_Count -OSRCK 200 200 000 - 0
> 240 Head_Flying_Hours ------ 100 253 000 - 14740h+55m+31.297s
> 241 Total_LBAs_Written ------ 100 253 000 - 9249405614
> 242 Total_LBAs_Read ------ 100 253 000 - 100539385901
> ||||||_ K auto-keep
> |||||__ C event count
> ||||___ R error rate
> |||____ S speed/performance
> ||_____ O updated online
> |______ P prefailure warning
> General Purpose Log Directory Version 1
> SMART Log Directory Version 1 [multi-sector log support]
> Address Access R/W Size Description
> 0x00 GPL,SL R/O 1 Log Directory
> 0x01 SL R/O 1 Summary SMART error log
> 0x02 SL R/O 5 Comprehensive SMART error log
> 0x03 GPL R/O 5 Ext. Comprehensive SMART error log
> 0x06 SL R/O 1 SMART self-test log
> 0x07 GPL R/O 1 Extended self-test log
> 0x09 SL R/W 1 Selective self-test log
> 0x10 GPL R/O 1 NCQ Command Error log
> 0x11 GPL R/O 1 SATA Phy Event Counters
> 0x21 GPL R/O 1 Write stream error log
> 0x22 GPL R/O 1 Read stream error log
> 0x80-0x9f GPL,SL R/W 16 Host vendor specific log
> 0xa1 GPL,SL VS 20 Device vendor specific log
> 0xa2 GPL VS 4496 Device vendor specific log
> 0xa8 GPL,SL VS 129 Device vendor specific log
> 0xa9 GPL,SL VS 1 Device vendor specific log
> 0xab GPL VS 1 Device vendor specific log
> 0xb0 GPL VS 5176 Device vendor specific log
> 0xbd GPL VS 512 Device vendor specific log
> 0xbe-0xbf GPL VS 65535 Device vendor specific log
> 0xc0 GPL,SL VS 1 Device vendor specific log
> 0xc1 GPL,SL VS 10 Device vendor specific log
> 0xc4 GPL,SL VS 5 Device vendor specific log
> 0xe0 GPL,SL R/W 1 SCT Command/Status
> 0xe1 GPL,SL R/W 1 SCT Data Transfer
> SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
> No Errors Logged
> SMART Extended Self-test Log Version: 1 (1 sectors)
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute delay.
> SCT Data Table command not supported
> SCT Error Recovery Control command not supported
> Device Statistics (GP Log 0x04) not supported
> SATA Phy Event Counters (GP Log 0x11)
> ID Size Value Description
> 0x000a 2 6 Device-to-host register FISes sent due to a COMRESET
> 0x0001 2 0 Command failed due to ICRC error
> 0x0003 2 0 R_ERR response for device-to-host data FIS
> 0x0004 2 0 R_ERR response for host-to-device data FIS
> 0x0006 2 0 R_ERR response for device-to-host non-data FIS
> 0x0007 2 0 R_ERR response for host-to-device non-data FIS
> THIS IS THE BAD DISK:
> sudo smartctl -x /dev/sdc
> smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-45-generic] (local build)
> Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
> === START OF INFORMATION SECTION ===
> Model Family: Seagate Barracuda 7200.14 (AF)
> Device Model: ST2000DM001-1CH164
> Serial Number: S240V6VR
> LU WWN Device Id: 5 000c50 05c05c2e7
> Firmware Version: CC24
> User Capacity: 2,000,398,934,016 bytes [2.00 TB]
> Sector Sizes: 512 bytes logical, 4096 bytes physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA8-ACS T13/1699-D revision 4
> SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
> Local Time is: Tue Feb 10 16:42:53 2015 EST
> ==> WARNING: A firmware update for this drive may be available,
> see the following Seagate web pages:
> http://knowledge.seagate.com/articles/en_US/FAQ/207931en
> http://knowledge.seagate.com/articles/en_US/FAQ/223651en
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
> AAM feature is: Unavailable
> APM level is: 254 (maximum performance)
> Rd look-ahead is: Enabled
> Write cache is: Enabled
> ATA Security is: Disabled, NOT FROZEN [SEC1]
> Write SCT (Get) XXX Error Recovery Control Command failed: scsi error aborted command
> Wt Cache Reorder: N/A
> Read SMART Data failed: scsi error aborted command
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: UNKNOWN!
> SMART Status, Attributes and Thresholds cannot be read.
> General Purpose Log Directory Version 1
> SMART Log Directory Version 1 [multi-sector log support]
> Address Access R/W Size Description
> 0x00 GPL,SL R/O 1 Log Directory
> 0x01 SL R/O 1 Summary SMART error log
> 0x02 SL R/O 5 Comprehensive SMART error log
> 0x03 GPL R/O 5 Ext. Comprehensive SMART error log
> 0x06 SL R/O 1 SMART self-test log
> 0x07 GPL R/O 1 Extended self-test log
> 0x09 SL R/W 1 Selective self-test log
> 0x10 GPL R/O 1 NCQ Command Error log
> 0x11 GPL R/O 1 SATA Phy Event Counters
> 0x21 GPL R/O 1 Write stream error log
> 0x22 GPL R/O 1 Read stream error log
> 0x80-0x9f GPL,SL R/W 16 Host vendor specific log
> 0xa1 GPL,SL VS 20 Device vendor specific log
> 0xa2 GPL VS 4496 Device vendor specific log
> 0xa8 GPL,SL VS 129 Device vendor specific log
> 0xa9 GPL,SL VS 1 Device vendor specific log
> 0xab GPL VS 1 Device vendor specific log
> 0xb0 GPL VS 5176 Device vendor specific log
> 0xbd GPL VS 512 Device vendor specific log
> 0xbe-0xbf GPL VS 65535 Device vendor specific log
> 0xc0 GPL,SL VS 1 Device vendor specific log
> 0xc1 GPL,SL VS 10 Device vendor specific log
> 0xc4 GPL,SL VS 5 Device vendor specific log
> 0xe0 GPL,SL R/W 1 SCT Command/Status
> 0xe1 GPL,SL R/W 1 SCT Data Transfer
> SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
> Device Error Count: 9
> CR = Command Register
> FEATR = Features Register
> COUNT = Count (was: Sector Count) Register
> LBA_48 = Upper bytes of LBA High/Mid/Low Registers ] ATA-8
> LH = LBA High (was: Cylinder High) Register ] LBA
> LM = LBA Mid (was: Cylinder Low) Register ] Register
> LL = LBA Low (was: Sector Number) Register ]
> DV = Device (was: Device/Head) Register
> DC = Device Control Register
> ER = Error register
> ST = Status register
> Powered_Up_Time is measured from power on, and printed as
> DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
> SS=sec, and sss=millisec. It "wraps" after 49.710 days.
> Error 9 [8] occurred at disk power-on lifetime: 14697 hours (612 days + 9 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 00 80 00 00 a4 1c 1d e8 e0 00 04:55:26.791 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 21 00 e0 00 04:55:26.776 READ DMA EXT
> ef 00 10 00 02 00 00 00 00 00 00 a0 00 04:55:26.775 SET FEATURES [Enable SATA feature]
> 27 00 00 00 00 00 00 00 00 00 00 e0 00 04:55:26.775 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
> ec 00 00 00 00 00 00 00 00 00 00 a0 00 04:55:26.774 IDENTIFY DEVICE
> Error 8 [7] occurred at disk power-on lifetime: 14697 hours (612 days + 9 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 04 00 00 00 a4 1c 1d 00 e0 00 04:55:23.631 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 19 00 e0 00 04:55:23.553 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 15 00 e0 00 04:55:23.108 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 11 00 e0 00 04:55:23.004 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 0d 00 e0 00 04:55:22.893 READ DMA EXT
> Error 7 [6] occurred at disk power-on lifetime: 14686 hours (611 days + 22 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 03 c0 00 00 a4 1c 1d e8 e0 00 1d+00:26:44.862 READ DMA EXT
> 25 00 00 00 08 00 00 a4 1c 21 a8 e0 00 1d+00:26:44.852 READ DMA EXT
> ec 00 00 00 01 00 00 00 00 00 00 00 00 1d+00:26:44.851 IDENTIFY DEVICE
> ec 00 00 00 01 00 00 00 00 00 00 00 00 1d+00:26:44.851 IDENTIFY DEVICE
> e5 00 00 00 00 00 00 00 00 00 00 00 00 1d+00:26:44.851 CHECK POWER MODE
> Error 6 [5] occurred at disk power-on lifetime: 14686 hours (611 days + 22 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 04 00 00 00 a4 1c 1d a8 e0 00 1d+00:26:30.653 READ DMA EXT
> ef 00 90 00 03 00 00 00 00 00 00 a0 00 1d+00:26:30.638 SET FEATURES [Disable SATA feature]
> ef 00 10 00 02 00 00 00 00 00 00 a0 00 1d+00:26:30.638 SET FEATURES [Enable SATA feature]
> 27 00 00 00 00 00 00 00 00 00 00 e0 00 1d+00:26:30.638 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
> ec 00 00 00 00 00 00 00 00 00 00 a0 00 1d+00:26:30.638 IDENTIFY DEVICE
> Error 5 [4] occurred at disk power-on lifetime: 14676 hours (611 days + 12 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 00 a8 00 00 a4 1c 1d e8 e0 00 14:43:09.384 READ DMA EXT
> e5 00 00 00 00 00 00 00 00 00 00 00 00 14:43:09.383 CHECK POWER MODE
> 25 00 00 04 00 00 00 a4 1c 1e 90 e0 00 14:43:09.371 READ DMA EXT
> ef 00 10 00 02 00 00 00 00 00 00 a0 00 14:43:09.370 SET FEATURES [Enable SATA feature]
> 27 00 00 00 00 00 00 00 00 00 00 e0 00 14:43:09.370 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
> Error 4 [3] occurred at disk power-on lifetime: 14676 hours (611 days + 12 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 04 00 00 00 a4 1c 1a 90 e0 00 14:43:06.283 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 16 90 e0 00 14:43:06.205 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 12 90 e0 00 14:43:04.892 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 0e 90 e0 00 14:43:04.855 READ DMA EXT
> 25 00 00 04 00 00 00 a4 1c 0a 90 e0 00 14:43:04.819 READ DMA EXT
> Error 3 [2] occurred at disk power-on lifetime: 14670 hours (611 days + 6 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 1d e8 00 00 Error: UNC at LBA = 0xa41c1de8 = 2753306088
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 04 00 00 00 a4 1c 1a 00 e0 00 08:33:02.502 READ DMA EXT
> ef 00 10 00 02 00 00 00 00 00 00 a0 00 08:33:02.501 SET FEATURES [Enable SATA feature]
> 27 00 00 00 00 00 00 00 00 00 00 e0 00 08:33:02.501 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
> ec 00 00 00 00 00 00 00 00 00 00 a0 00 08:33:02.501 IDENTIFY DEVICE
> ef 00 03 00 42 00 00 00 00 00 00 a0 00 08:33:02.501 SET FEATURES [Set transfer mode]
> Error 2 [1] occurred at disk power-on lifetime: 14670 hours (611 days + 6 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 40 -- 51 00 00 00 00 a4 1c 13 d0 00 00 Error: UNC at LBA = 0xa41c13d0 = 2753303504
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 02 30 00 00 a4 1c 13 d0 e0 00 08:32:59.645 READ DMA EXT
> e5 00 00 00 00 00 00 00 00 00 00 00 00 08:32:59.643 CHECK POWER MODE
> 25 00 00 04 00 00 00 a4 1c 16 00 e0 00 08:32:59.581 READ DMA EXT
> ef 00 10 00 02 00 00 00 00 00 00 a0 00 08:32:59.580 SET FEATURES [Enable SATA feature]
> 27 00 00 00 00 00 00 00 00 00 00 e0 00 08:32:59.580 READ NATIVE MAX ADDRESS EXT [OBS-ACS-3]
> SMART Extended Self-test Log Version: 1 (1 sectors)
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
> Selective Self-tests/Logging not supported
> SCT Data Table command not supported
> SCT Error Recovery Control command not supported
> Device Statistics (GP Log 0x04) not supported
> SATA Phy Event Counters (GP Log 0x11)
> ID Size Value Description
> 0x000a 2 6 Device-to-host register FISes sent due to a COMRESET
> 0x0001 2 0 Command failed due to ICRC error
> 0x0003 2 0 R_ERR response for device-to-host data FIS
> 0x0004 2 0 R_ERR response for host-to-device data FIS
> 0x0006 2 0 R_ERR response for device-to-host non-data FIS
> 0x0007 2 0 R_ERR response for host-to-device non-data FIS
> sudo smartctl -x /dev/sdd
> smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-45-generic] (local build)
> Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
> === START OF INFORMATION SECTION ===
> Model Family: Hitachi Deskstar 7K3000
> Device Model: Hitachi HDS723020BLA642
> Serial Number: MN3220F32GX10E
> LU WWN Device Id: 5 000cca 369e2f56f
> Firmware Version: MN6OA5C0
> User Capacity: 2,000,398,934,016 bytes [2.00 TB]
> Sector Size: 512 bytes logical/physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA8-ACS T13/1699-D revision 4
> SATA Version is: SATA 2.6, 6.0 Gb/s (current: 3.0 Gb/s)
> Local Time is: Tue Feb 10 16:45:04 2015 EST
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
> AAM feature is: Unavailable
> APM feature is: Disabled
> Rd look-ahead is: Enabled
> Write cache is: Enabled
> ATA Security is: Disabled, NOT FROZEN [SEC1]
> Wt Cache Reorder: Enabled
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
> General SMART Values:
> Offline data collection status: (0x84) Offline data collection activity
> was suspended by an interrupting command from host.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: (18096) seconds.
> Offline data collection
> capabilities: (0x5b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> No Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 1) minutes.
> Extended self-test routine
> recommended polling time: ( 302) minutes.
> SCT capabilities: (0x003d) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
> 1 Raw_Read_Error_Rate PO-R-- 100 100 016 - 0
> 2 Throughput_Performance P-S--- 136 136 054 - 82
> 3 Spin_Up_Time POS--- 152 152 024 - 434 (Average 320)
> 4 Start_Stop_Count -O--C- 100 100 000 - 97
> 5 Reallocated_Sector_Ct PO--CK 100 100 005 - 0
> 7 Seek_Error_Rate PO-R-- 100 100 067 - 0
> 8 Seek_Time_Performance P-S--- 135 135 020 - 26
> 9 Power_On_Hours -O--C- 097 097 000 - 27235
> 10 Spin_Retry_Count PO--C- 100 100 060 - 0
> 12 Power_Cycle_Count -O--CK 100 100 000 - 97
> 192 Power-Off_Retract_Count -O--CK 100 100 000 - 755
> 193 Load_Cycle_Count -O--C- 100 100 000 - 755
> 194 Temperature_Celsius -O---- 200 200 000 - 30 (Min/Max 19/45)
> 196 Reallocated_Event_Count -O--CK 100 100 000 - 0
> 197 Current_Pending_Sector -O---K 100 100 000 - 0
> 198 Offline_Uncorrectable ---R-- 100 100 000 - 0
> 199 UDMA_CRC_Error_Count -O-R-- 200 200 000 - 0
> ||||||_ K auto-keep
> |||||__ C event count
> ||||___ R error rate
> |||____ S speed/performance
> ||_____ O updated online
> |______ P prefailure warning
> General Purpose Log Directory Version 1
> SMART Log Directory Version 1 [multi-sector log support]
> Address Access R/W Size Description
> 0x00 GPL,SL R/O 1 Log Directory
> 0x01 SL R/O 1 Summary SMART error log
> 0x03 GPL R/O 1 Ext. Comprehensive SMART error log
> 0x04 GPL R/O 7 Device Statistics log
> 0x06 SL R/O 1 SMART self-test log
> 0x07 GPL R/O 1 Extended self-test log
> 0x08 GPL R/O 1 Power Conditions log
> 0x09 SL R/W 1 Selective self-test log
> 0x10 GPL R/O 1 NCQ Command Error log
> 0x11 GPL R/O 1 SATA Phy Event Counters
> 0x20 GPL R/O 1 Streaming performance log [OBS-8]
> 0x21 GPL R/O 1 Write stream error log
> 0x22 GPL R/O 1 Read stream error log
> 0x80-0x9f GPL,SL R/W 16 Host vendor specific log
> 0xe0 GPL,SL R/W 1 SCT Command/Status
> 0xe1 GPL,SL R/W 1 SCT Data Transfer
> SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
> No Errors Logged
> SMART Extended Self-test Log Version: 1 (1 sectors)
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute delay.
> SCT Status Version: 3
> SCT Version (vendor specific): 256 (0x0100)
> SCT Support Level: 1
> Device State: SMART Off-line Data Collection executing in background (4)
> Current Temperature: 30 Celsius
> Power Cycle Min/Max Temperature: 27/30 Celsius
> Lifetime Min/Max Temperature: 19/45 Celsius
> Under/Over Temperature Limit Count: 0/0
> SCT Temperature History Version: 2
> Temperature Sampling Period: 1 minute
> Temperature Logging Interval: 1 minute
> Min/Max recommended Temperature: 0/60 Celsius
> Min/Max Temperature Limit: -40/70 Celsius
> Temperature History Size (Index): 128 (52)
> Index Estimated Time Temperature Celsius
> 53 2015-02-10 14:38 37 ******************
> ... ..( 24 skipped). .. ******************
> 78 2015-02-10 15:03 37 ******************
> 79 2015-02-10 15:04 36 *****************
> 80 2015-02-10 15:05 36 *****************
> 81 2015-02-10 15:06 37 ******************
> ... ..( 5 skipped). .. ******************
> 87 2015-02-10 15:12 37 ******************
> 88 2015-02-10 15:13 36 *****************
> 89 2015-02-10 15:14 37 ******************
> ... ..( 5 skipped). .. ******************
> 95 2015-02-10 15:20 37 ******************
> 96 2015-02-10 15:21 36 *****************
> 97 2015-02-10 15:22 37 ******************
> 98 2015-02-10 15:23 37 ******************
> 99 2015-02-10 15:24 36 *****************
> 100 2015-02-10 15:25 37 ******************
> ... ..( 4 skipped). .. ******************
> 105 2015-02-10 15:30 37 ******************
> 106 2015-02-10 15:31 36 *****************
> 107 2015-02-10 15:32 36 *****************
> 108 2015-02-10 15:33 37 ******************
> ... ..( 6 skipped). .. ******************
> 115 2015-02-10 15:40 37 ******************
> 116 2015-02-10 15:41 36 *****************
> 117 2015-02-10 15:42 36 *****************
> 118 2015-02-10 15:43 36 *****************
> 119 2015-02-10 15:44 37 ******************
> ... ..( 2 skipped). .. ******************
> 122 2015-02-10 15:47 37 ******************
> 123 2015-02-10 15:48 36 *****************
> 124 2015-02-10 15:49 37 ******************
> 125 2015-02-10 15:50 37 ******************
> 126 2015-02-10 15:51 36 *****************
> 127 2015-02-10 15:52 36 *****************
> 0 2015-02-10 15:53 37 ******************
> 1 2015-02-10 15:54 36 *****************
> 2 2015-02-10 15:55 37 ******************
> 3 2015-02-10 15:56 36 *****************
> 4 2015-02-10 15:57 36 *****************
> 5 2015-02-10 15:58 37 ******************
> ... ..( 2 skipped). .. ******************
> 8 2015-02-10 16:01 37 ******************
> 9 2015-02-10 16:02 36 *****************
> 10 2015-02-10 16:03 37 ******************
> ... ..( 2 skipped). .. ******************
> 13 2015-02-10 16:06 37 ******************
> 14 2015-02-10 16:07 36 *****************
> 15 2015-02-10 16:08 37 ******************
> ... ..( 10 skipped). .. ******************
> 26 2015-02-10 16:19 37 ******************
> 27 2015-02-10 16:20 36 *****************
> ... ..( 5 skipped). .. *****************
> 33 2015-02-10 16:26 36 *****************
> 34 2015-02-10 16:27 37 ******************
> ... ..( 4 skipped). .. ******************
> 39 2015-02-10 16:32 37 ******************
> 40 2015-02-10 16:33 ? -
> 41 2015-02-10 16:34 27 ********
> 42 2015-02-10 16:35 28 *********
> 43 2015-02-10 16:36 28 *********
> 44 2015-02-10 16:37 28 *********
> 45 2015-02-10 16:38 29 **********
> ... ..( 2 skipped). .. **********
> 48 2015-02-10 16:41 29 **********
> 49 2015-02-10 16:42 30 ***********
> ... ..( 2 skipped). .. ***********
> 52 2015-02-10 16:45 30 ***********
> SCT Error Recovery Control:
> Read: Disabled
> Write: Disabled
> Device Statistics (GP Log 0x04)
> Page Offset Size Value Description
> 1 ===== = = == General Statistics (rev 1) ==
> 1 0x008 4 97 Lifetime Power-On Resets
> 1 0x010 4 27235 Power-on Hours
> 1 0x018 6 11734342067 Logical Sectors Written
> 1 0x020 6 27559380 Number of Write Commands
> 1 0x028 6 2738754035727 Logical Sectors Read
> 1 0x030 6 5733165681 Number of Read Commands
> 3 ===== = = == Rotating Media Statistics (rev 1) ==
> 3 0x008 4 27229 Spindle Motor Power-on Hours
> 3 0x010 4 27229 Head Flying Hours
> 3 0x018 4 755 Head Load Events
> 3 0x020 4 0 Number of Reallocated Logical Sectors
> 3 0x028 4 276 Read Recovery Attempts
> 3 0x030 4 7 Number of Mechanical Start Failures
> 4 ===== = = == General Errors Statistics (rev 1) ==
> 4 0x008 4 0 Number of Reported Uncorrectable Errors
> 4 0x010 4 2 Resets Between Cmd Acceptance and Completion
> 5 ===== = = == Temperature Statistics (rev 1) ==
> 5 0x008 1 30 Current Temperature
> 5 0x010 1 35~ Average Short Term Temperature
> 5 0x018 1 33~ Average Long Term Temperature
> 5 0x020 1 45 Highest Temperature
> 5 0x028 1 19 Lowest Temperature
> 5 0x030 1 42~ Highest Average Short Term Temperature
> 5 0x038 1 24~ Lowest Average Short Term Temperature
> 5 0x040 1 39~ Highest Average Long Term Temperature
> 5 0x048 1 25~ Lowest Average Long Term Temperature
> 5 0x050 4 0 Time in Over-Temperature
> 5 0x058 1 60 Specified Maximum Operating Temperature
> 5 0x060 4 0 Time in Under-Temperature
> 5 0x068 1 0 Specified Minimum Operating Temperature
> 6 ===== = = == Transport Statistics (rev 1) ==
> 6 0x008 4 1122 Number of Hardware Resets
> 6 0x010 4 1027 Number of ASR Events
> 6 0x018 4 0 Number of Interface CRC Errors
> |_ ~ normalized value
> SATA Phy Event Counters (GP Log 0x11)
> ID Size Value Description
> 0x0001 2 0 Command failed due to ICRC error
> 0x0002 2 0 R_ERR response for data FIS
> 0x0003 2 0 R_ERR response for device-to-host data FIS
> 0x0004 2 0 R_ERR response for host-to-device data FIS
> 0x0005 2 0 R_ERR response for non-data FIS
> 0x0006 2 0 R_ERR response for device-to-host non-data FIS
> 0x0007 2 0 R_ERR response for host-to-device non-data FIS
> 0x0009 2 6 Transition from drive PhyRdy to drive PhyNRdy
> 0x000a 2 5 Device-to-host register FISes sent due to a COMRESET
> 0x000b 2 0 CRC errors within host-to-device FIS
> 0x000d 2 0 Non-CRC errors within host-to-device FIS
> sudo smartctl -x /dev/sde
> smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-45-generic] (local build)
> Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
> === START OF INFORMATION SECTION ===
> Model Family: Hitachi Deskstar 7K2000
> Device Model: Hitachi HDS722020ALA330
> Serial Number: JK1171YAGAD8LS
> LU WWN Device Id: 5 000cca 221c4b9cc
> Firmware Version: JKAOA20N
> User Capacity: 2,000,398,934,016 bytes [2.00 TB]
> Sector Size: 512 bytes logical/physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA8-ACS T13/1699-D revision 4
> SATA Version is: SATA 2.6, 3.0 Gb/s
> Local Time is: Tue Feb 10 16:45:31 2015 EST
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
> AAM feature is: Disabled
> APM feature is: Disabled
> Rd look-ahead is: Enabled
> Write cache is: Enabled
> ATA Security is: Disabled, NOT FROZEN [SEC1]
> Wt Cache Reorder: Enabled
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
> General SMART Values:
> Offline data collection status: (0x84) Offline data collection activity
> was suspended by an interrupting command from host.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: (21007) seconds.
> Offline data collection
> capabilities: (0x5b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> No Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 1) minutes.
> Extended self-test routine
> recommended polling time: ( 350) minutes.
> SCT capabilities: (0x003d) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
> 1 Raw_Read_Error_Rate PO-R-- 100 100 016 - 0
> 2 Throughput_Performance P-S--- 134 134 054 - 98
> 3 Spin_Up_Time POS--- 137 137 024 - 619 (Average 439)
> 4 Start_Stop_Count -O--C- 100 100 000 - 207
> 5 Reallocated_Sector_Ct PO--CK 100 100 005 - 0
> 7 Seek_Error_Rate PO-R-- 100 100 067 - 0
> 8 Seek_Time_Performance P-S--- 112 112 020 - 39
> 9 Power_On_Hours -O--C- 094 094 000 - 44002
> 10 Spin_Retry_Count PO--C- 100 100 060 - 0
> 12 Power_Cycle_Count -O--CK 100 100 000 - 207
> 192 Power-Off_Retract_Count -O--CK 099 099 000 - 1267
> 193 Load_Cycle_Count -O--C- 099 099 000 - 1267
> 194 Temperature_Celsius -O---- 181 181 000 - 33 (Min/Max 20/53)
> 196 Reallocated_Event_Count -O--CK 100 100 000 - 0
> 197 Current_Pending_Sector -O---K 100 100 000 - 0
> 198 Offline_Uncorrectable ---R-- 100 100 000 - 0
> 199 UDMA_CRC_Error_Count -O-R-- 200 200 000 - 9
> ||||||_ K auto-keep
> |||||__ C event count
> ||||___ R error rate
> |||____ S speed/performance
> ||_____ O updated online
> |______ P prefailure warning
> General Purpose Log Directory Version 1
> SMART Log Directory Version 1 [multi-sector log support]
> Address Access R/W Size Description
> 0x00 GPL,SL R/O 1 Log Directory
> 0x01 SL R/O 1 Summary SMART error log
> 0x03 GPL R/O 1 Ext. Comprehensive SMART error log
> 0x04 GPL R/O 7 Device Statistics log
> 0x06 SL R/O 1 SMART self-test log
> 0x07 GPL R/O 1 Extended self-test log
> 0x09 SL R/W 1 Selective self-test log
> 0x10 GPL R/O 1 NCQ Command Error log
> 0x11 GPL R/O 1 SATA Phy Event Counters
> 0x20 GPL R/O 1 Streaming performance log [OBS-8]
> 0x21 GPL R/O 1 Write stream error log
> 0x22 GPL R/O 1 Read stream error log
> 0x80-0x9f GPL,SL R/W 16 Host vendor specific log
> 0xe0 GPL,SL R/W 1 SCT Command/Status
> 0xe1 GPL,SL R/W 1 SCT Data Transfer
> SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
> Device Error Count: 10 (device log contains only the most recent 4 errors)
> CR = Command Register
> FEATR = Features Register
> COUNT = Count (was: Sector Count) Register
> LBA_48 = Upper bytes of LBA High/Mid/Low Registers ] ATA-8
> LH = LBA High (was: Cylinder High) Register ] LBA
> LM = LBA Mid (was: Cylinder Low) Register ] Register
> LL = LBA Low (was: Sector Number) Register ]
> DV = Device (was: Device/Head) Register
> DC = Device Control Register
> ER = Error register
> ST = Status register
> Powered_Up_Time is measured from power on, and printed as
> DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
> SS=sec, and sss=millisec. It "wraps" after 49.710 days.
> Error 10 [1] occurred at disk power-on lifetime: 1655 hours (68 days + 23 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 84 -- 51 01 28 00 00 50 83 5d e8 00 00 Error: ICRC, ABRT 296 sectors at LBA = 0x50835de8 = 1350786536
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 02 a8 00 00 50 83 5c 68 e0 08 23d+05:05:37.425 READ DMA EXT
> 25 00 00 03 68 00 00 50 83 59 00 e0 08 23d+05:05:37.413 READ DMA EXT
> 25 00 00 01 00 00 00 50 83 58 00 e0 08 23d+05:05:37.409 READ DMA EXT
> 25 00 00 00 f0 00 00 50 83 57 10 e0 08 23d+05:05:37.405 READ DMA EXT
> 25 00 00 02 a0 00 00 50 83 54 70 e0 08 23d+05:05:37.352 READ DMA EXT
> Error 9 [0] occurred at disk power-on lifetime: 1654 hours (68 days + 22 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 84 -- 51 00 90 00 00 4e eb 15 70 00 00 Error: ICRC, ABRT 144 sectors at LBA = 0x4eeb1570 = 1324029296
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 01 00 00 00 4e eb 15 00 ee 08 23d+04:47:42.788 READ DMA EXT
> 25 00 00 02 28 00 00 4e eb 12 d8 ee 08 23d+04:47:42.713 READ DMA EXT
> 25 00 00 03 d8 00 00 4e eb 0f 00 ee 08 23d+04:47:42.698 READ DMA EXT
> 25 00 00 01 00 00 00 4e eb 0e 00 ee 08 23d+04:47:42.694 READ DMA EXT
> 25 00 00 01 00 00 00 4e eb 0d 00 ee 08 23d+04:47:42.691 READ DMA EXT
> Error 8 [3] occurred at disk power-on lifetime: 1654 hours (68 days + 22 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 84 -- 51 00 28 00 00 36 08 f1 d8 00 00 Error: ICRC, ABRT 40 sectors at LBA = 0x3608f1d8 = 906555864
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 00 f8 00 00 36 08 f1 08 e6 08 23d+00:06:40.966 READ DMA EXT
> 25 00 00 02 78 00 00 36 08 ee 90 e6 08 23d+00:06:40.914 READ DMA EXT
> 25 00 00 03 90 00 00 36 08 eb 00 e6 08 23d+00:06:40.900 READ DMA EXT
> 25 00 00 01 00 00 00 36 08 ea 00 e6 08 23d+00:06:40.896 READ DMA EXT
> 25 00 00 00 f8 00 00 36 08 e9 08 e6 08 23d+00:06:40.893 READ DMA EXT
> Error 7 [2] occurred at disk power-on lifetime: 1654 hours (68 days + 22 hours)
> When the command that caused the error occurred, the device was active or idle.
> After command completion occurred, registers were:
> ER -- ST COUNT LBA_48 LH LM LL DV DC
> -- -- -- == -- == == == -- -- -- -- --
> 84 -- 51 01 28 00 00 33 d1 bb 40 00 00 Error: ICRC, ABRT 296 sectors at LBA = 0x33d1bb40 = 869382976
> Commands leading to the command that caused the error were:
> CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
> -- == -- == -- == == == -- -- -- -- -- --------------- --------------------
> 25 00 00 03 68 00 00 33 d1 b9 00 e3 08 22d+23:42:04.107 READ DMA EXT
> 25 00 00 01 00 00 00 33 d1 b8 00 e3 08 22d+23:42:04.103 READ DMA EXT
> 25 00 00 00 f0 00 00 33 d1 b7 10 e3 08 22d+23:42:04.099 READ DMA EXT
> 25 00 00 02 b0 00 00 33 d1 b4 60 e3 08 22d+23:42:04.022 READ DMA EXT
> 25 00 00 03 60 00 00 33 d1 b1 00 e3 08 22d+23:42:04.009 READ DMA EXT
> SMART Extended Self-test Log Version: 1 (1 sectors)
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute delay.
> SCT Status Version: 3
> SCT Version (vendor specific): 256 (0x0100)
> SCT Support Level: 1
> Device State: SMART Off-line Data Collection executing in background (4)
> Current Temperature: 33 Celsius
> Power Cycle Min/Max Temperature: 27/33 Celsius
> Lifetime Min/Max Temperature: 20/53 Celsius
> Under/Over Temperature Limit Count: 0/0
> SCT Temperature History Version: 2
> Temperature Sampling Period: 1 minute
> Temperature Logging Interval: 1 minute
> Min/Max recommended Temperature: 0/60 Celsius
> Min/Max Temperature Limit: -40/70 Celsius
> Temperature History Size (Index): 128 (81)
> Index Estimated Time Temperature Celsius
> 82 2015-02-10 14:38 41 **********************
> ... ..(113 skipped). .. **********************
> 68 2015-02-10 16:32 41 **********************
> 69 2015-02-10 16:33 ? -
> 70 2015-02-10 16:34 28 *********
> 71 2015-02-10 16:35 28 *********
> 72 2015-02-10 16:36 29 **********
> 73 2015-02-10 16:37 29 **********
> 74 2015-02-10 16:38 30 ***********
> 75 2015-02-10 16:39 30 ***********
> 76 2015-02-10 16:40 31 ************
> 77 2015-02-10 16:41 31 ************
> 78 2015-02-10 16:42 32 *************
> 79 2015-02-10 16:43 32 *************
> 80 2015-02-10 16:44 33 **************
> 81 2015-02-10 16:45 33 **************
> SCT Error Recovery Control:
> Read: Disabled
> Write: Disabled
> Device Statistics (GP Log 0x04)
> Page Offset Size Value Description
> 1 ===== = = == General Statistics (rev 1) ==
> 1 0x008 4 207 Lifetime Power-On Resets
> 1 0x010 4 44002 Power-on Hours
> 1 0x018 6 19676641503 Logical Sectors Written
> 1 0x020 6 47285021 Number of Write Commands
> 1 0x028 6 4518358603939 Logical Sectors Read
> 1 0x030 6 5982270826 Number of Read Commands
> 3 ===== = = == Rotating Media Statistics (rev 1) ==
> 3 0x008 4 43993 Spindle Motor Power-on Hours
> 3 0x010 4 43993 Head Flying Hours
> 3 0x018 4 1267 Head Load Events
> 3 0x020 4 0 Number of Reallocated Logical Sectors
> 3 0x028 4 14 Read Recovery Attempts
> 3 0x030 4 1 Number of Mechanical Start Failures
> 4 ===== = = == General Errors Statistics (rev 1) ==
> 4 0x008 4 0 Number of Reported Uncorrectable Errors
> 4 0x010 4 180 Resets Between Cmd Acceptance and Completion
> 5 ===== = = == Temperature Statistics (rev 1) ==
> 5 0x008 1 33 Current Temperature
> 5 0x010 1 41~ Average Short Term Temperature
> 5 0x018 1 41~ Average Long Term Temperature
> 5 0x020 1 53 Highest Temperature
> 5 0x028 1 20 Lowest Temperature
> 5 0x030 1 49~ Highest Average Short Term Temperature
> 5 0x038 1 0~ Lowest Average Short Term Temperature
> 5 0x040 1 47~ Highest Average Long Term Temperature
> 5 0x048 1 0~ Lowest Average Long Term Temperature
> 5 0x050 4 0 Time in Over-Temperature
> 5 0x058 1 60 Specified Maximum Operating Temperature
> 5 0x060 4 0 Time in Under-Temperature
> 5 0x068 1 0 Specified Minimum Operating Temperature
> 6 ===== = = == Transport Statistics (rev 1) ==
> 6 0x008 4 1957 Number of Hardware Resets
> 6 0x010 4 1773 Number of ASR Events
> 6 0x018 4 9 Number of Interface CRC Errors
> |_ ~ normalized value
> SATA Phy Event Counters (GP Log 0x11)
> ID Size Value Description
> 0x0001 2 0 Command failed due to ICRC error
> 0x0002 2 0 R_ERR response for data FIS
> 0x0005 2 0 R_ERR response for non-data FIS
> 0x0009 2 6 Transition from drive PhyRdy to drive PhyNRdy
> 0x000a 2 4 Device-to-host register FISes sent due to a COMRESET
> 0x000b 2 0 CRC errors within host-to-device FIS
> 0x000d 2 0 Non-CRC errors within host-to-device FIS
> sudo smartctl -x /dev/sdf
> smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-45-generic] (local build)
> Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
> === START OF INFORMATION SECTION ===
> Model Family: Hitachi Deskstar 7K2000
> Device Model: Hitachi HDS722020ALA330
> Serial Number: JK1171YAGDAD5S
> LU WWN Device Id: 5 000cca 221c59b77
> Firmware Version: JKAOA20N
> User Capacity: 2,000,397,852,160 bytes [2.00 TB]
> Sector Size: 512 bytes logical/physical
> Rotation Rate: 7200 rpm
> Device is: In smartctl database [for details use: -P show]
> ATA Version is: ATA8-ACS T13/1699-D revision 4
> SATA Version is: SATA 2.6, 3.0 Gb/s
> Local Time is: Tue Feb 10 16:46:04 2015 EST
> SMART support is: Available - device has SMART capability.
> SMART support is: Enabled
> AAM feature is: Disabled
> APM feature is: Disabled
> Rd look-ahead is: Enabled
> Write cache is: Enabled
> ATA Security is: Disabled, NOT FROZEN [SEC1]
> Wt Cache Reorder: Enabled
> === START OF READ SMART DATA SECTION ===
> SMART overall-health self-assessment test result: PASSED
> General SMART Values:
> Offline data collection status: (0x84) Offline data collection activity
> was suspended by an interrupting command from host.
> Auto Offline Data Collection: Enabled.
> Self-test execution status: ( 0) The previous self-test routine completed
> without error or no self-test has ever
> been run.
> Total time to complete Offline
> data collection: (22917) seconds.
> Offline data collection
> capabilities: (0x5b) SMART execute Offline immediate.
> Auto Offline data collection on/off support.
> Suspend Offline collection upon new
> command.
> Offline surface scan supported.
> Self-test supported.
> No Conveyance Self-test supported.
> Selective Self-test supported.
> SMART capabilities: (0x0003) Saves SMART data before entering
> power-saving mode.
> Supports SMART auto save timer.
> Error logging capability: (0x01) Error logging supported.
> General Purpose Logging supported.
> Short self-test routine
> recommended polling time: ( 1) minutes.
> Extended self-test routine
> recommended polling time: ( 382) minutes.
> SCT capabilities: (0x003d) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
> SMART Attributes Data Structure revision number: 16
> Vendor Specific SMART Attributes with Thresholds:
> ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
> 1 Raw_Read_Error_Rate PO-R-- 100 100 016 - 0
> 2 Throughput_Performance P-S--- 133 133 054 - 101
> 3 Spin_Up_Time POS--- 134 134 024 - 627 (Average 452)
> 4 Start_Stop_Count -O--C- 100 100 000 - 203
> 5 Reallocated_Sector_Ct PO--CK 100 100 005 - 0
> 7 Seek_Error_Rate PO-R-- 100 100 067 - 0
> 8 Seek_Time_Performance P-S--- 112 112 020 - 39
> 9 Power_On_Hours -O--C- 094 094 000 - 44006
> 10 Spin_Retry_Count PO--C- 100 100 060 - 0
> 12 Power_Cycle_Count -O--CK 100 100 000 - 203
> 192 Power-Off_Retract_Count -O--CK 099 099 000 - 1248
> 193 Load_Cycle_Count -O--C- 099 099 000 - 1248
> 194 Temperature_Celsius -O---- 193 193 000 - 31 (Min/Max 20/50)
> 196 Reallocated_Event_Count -O--CK 100 100 000 - 0
> 197 Current_Pending_Sector -O---K 100 100 000 - 0
> 198 Offline_Uncorrectable ---R-- 100 100 000 - 0
> 199 UDMA_CRC_Error_Count -O-R-- 200 200 000 - 0
> ||||||_ K auto-keep
> |||||__ C event count
> ||||___ R error rate
> |||____ S speed/performance
> ||_____ O updated online
> |______ P prefailure warning
> General Purpose Log Directory Version 1
> SMART Log Directory Version 1 [multi-sector log support]
> Address Access R/W Size Description
> 0x00 GPL,SL R/O 1 Log Directory
> 0x01 SL R/O 1 Summary SMART error log
> 0x03 GPL R/O 1 Ext. Comprehensive SMART error log
> 0x04 GPL R/O 7 Device Statistics log
> 0x06 SL R/O 1 SMART self-test log
> 0x07 GPL R/O 1 Extended self-test log
> 0x09 SL R/W 1 Selective self-test log
> 0x10 GPL R/O 1 NCQ Command Error log
> 0x11 GPL R/O 1 SATA Phy Event Counters
> 0x20 GPL R/O 1 Streaming performance log [OBS-8]
> 0x21 GPL R/O 1 Write stream error log
> 0x22 GPL R/O 1 Read stream error log
> 0x80-0x9f GPL,SL R/W 16 Host vendor specific log
> 0xe0 GPL,SL R/W 1 SCT Command/Status
> 0xe1 GPL,SL R/W 1 SCT Data Transfer
> SMART Extended Comprehensive Error Log Version: 0 (1 sectors)
> No Errors Logged
> SMART Extended Self-test Log Version: 1 (1 sectors)
> No self-tests have been logged. [To run self-tests, use: smartctl -t]
> SMART Selective self-test log data structure revision number 1
> SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
> 1 0 0 Not_testing
> 2 0 0 Not_testing
> 3 0 0 Not_testing
> 4 0 0 Not_testing
> 5 0 0 Not_testing
> Selective self-test flags (0x0):
> After scanning selected spans, do NOT read-scan remainder of disk.
> If Selective self-test is pending on power-up, resume after 0 minute delay.
> SCT Status Version: 3
> SCT Version (vendor specific): 256 (0x0100)
> SCT Support Level: 1
> Device State: SMART Off-line Data Collection executing in background (4)
> Current Temperature: 31 Celsius
> Power Cycle Min/Max Temperature: 27/31 Celsius
> Lifetime Min/Max Temperature: 20/50 Celsius
> Under/Over Temperature Limit Count: 0/0
> SCT Temperature History Version: 2
> Temperature Sampling Period: 1 minute
> Temperature Logging Interval: 1 minute
> Min/Max recommended Temperature: 0/60 Celsius
> Min/Max Temperature Limit: -40/70 Celsius
> Temperature History Size (Index): 128 (47)
> Index Estimated Time Temperature Celsius
> 48 2015-02-10 14:39 39 ********************
> ... ..( 98 skipped). .. ********************
> 19 2015-02-10 16:18 39 ********************
> 20 2015-02-10 16:19 40 *********************
> 21 2015-02-10 16:20 39 ********************
> ... ..( 3 skipped). .. ********************
> 25 2015-02-10 16:24 39 ********************
> 26 2015-02-10 16:25 38 *******************
> ... ..( 6 skipped). .. *******************
> 33 2015-02-10 16:32 38 *******************
> 34 2015-02-10 16:33 ? -
> 35 2015-02-10 16:34 27 ********
> 36 2015-02-10 16:35 28 *********
> 37 2015-02-10 16:36 28 *********
> 38 2015-02-10 16:37 29 **********
> 39 2015-02-10 16:38 29 **********
> 40 2015-02-10 16:39 30 ***********
> ... ..( 2 skipped). .. ***********
> 43 2015-02-10 16:42 30 ***********
> 44 2015-02-10 16:43 31 ************
> ... ..( 2 skipped). .. ************
> 47 2015-02-10 16:46 31 ************
> SCT Error Recovery Control:
> Read: Disabled
> Write: Disabled
> Device Statistics (GP Log 0x04)
> Page Offset Size Value Description
> 1 ===== = = == General Statistics (rev 1) ==
> 1 0x008 4 203 Lifetime Power-On Resets
> 1 0x010 4 44006 Power-on Hours
> 1 0x018 6 15872353160 Logical Sectors Written
> 1 0x020 6 39140100 Number of Write Commands
> 1 0x028 6 4462388816379 Logical Sectors Read
> 1 0x030 6 5927428317 Number of Read Commands
> 3 ===== = = == Rotating Media Statistics (rev 1) ==
> 3 0x008 4 43997 Spindle Motor Power-on Hours
> 3 0x010 4 43997 Head Flying Hours
> 3 0x018 4 1248 Head Load Events
> 3 0x020 4 0 Number of Reallocated Logical Sectors
> 3 0x028 4 32 Read Recovery Attempts
> 3 0x030 4 0 Number of Mechanical Start Failures
> 4 ===== = = == General Errors Statistics (rev 1) ==
> 4 0x008 4 0 Number of Reported Uncorrectable Errors
> 4 0x010 4 192 Resets Between Cmd Acceptance and Completion
> 5 ===== = = == Temperature Statistics (rev 1) ==
> 5 0x008 1 31 Current Temperature
> 5 0x010 1 37~ Average Short Term Temperature
> 5 0x018 1 35~ Average Long Term Temperature
> 5 0x020 1 50 Highest Temperature
> 5 0x028 1 20 Lowest Temperature
> 5 0x030 1 44~ Highest Average Short Term Temperature
> 5 0x038 1 0~ Lowest Average Short Term Temperature
> 5 0x040 1 42~ Highest Average Long Term Temperature
> 5 0x048 1 0~ Lowest Average Long Term Temperature
> 5 0x050 4 0 Time in Over-Temperature
> 5 0x058 1 60 Specified Maximum Operating Temperature
> 5 0x060 4 0 Time in Under-Temperature
> 5 0x068 1 0 Specified Minimum Operating Temperature
> 6 ===== = = == Transport Statistics (rev 1) ==
> 6 0x008 4 1947 Number of Hardware Resets
> 6 0x010 4 1765 Number of ASR Events
> 6 0x018 4 0 Number of Interface CRC Errors
> |_ ~ normalized value
> SATA Phy Event Counters (GP Log 0x11)
> ID Size Value Description
> 0x0001 2 0 Command failed due to ICRC error
> 0x0002 2 0 R_ERR response for data FIS
> 0x0005 2 0 R_ERR response for non-data FIS
> 0x0009 2 6 Transition from drive PhyRdy to drive PhyNRdy
> 0x000a 2 4 Device-to-host register FISes sent due to a COMRESET
> 0x000b 2 0 CRC errors within host-to-device FIS
> 0x000d 2 0 Non-CRC errors within host-to-device FIS
Adam:
I actually read that exact stackexchange article about using the
--replace command but I neither had kernel 3.2+ nor mdadm 3.3+ that
seemed to be a necessary requirement. I suppose I could have booted to
a more recent kernel livecd, but sadly i did not.
Thank you both for your help,
Kyle L
On Tue, Feb 10, 2015 at 8:51 AM, Phil Turmel <philip@turmel.org> wrote:
> Hi Kyle,
>
> Your symptoms look like classic timeout mismatch. Details interleaved.
>
> On 02/10/2015 02:35 AM, Adam Goryachev wrote:
>
>> There are other people who will jump in and help you with your problem,
>> but I'll add a couple of pointers while you are waiting. See below.
>
>> On 10/02/15 15:20, Kyle Logue wrote:
>>> Hey all:
>>>
>>> I have a 5 disk software raid5 that was working fine until I decided
>>> to swap out an old disk with a new one.
>>>
>>> mdadm /dev/md0 --add /dev/sda1
>>> mdadm /dev/md0 --fail /dev/sde1
>
> As Adam pointed out, you should have used --replace, but you probably
> wouldn't have made it through the replace function anyways.
>
>>> At this point it started automatically rebuilding the array.
>>> About 60%? of the way in it stops and I see a lot of this repeated in
>>> my dmesg:
>>>
>>> [Mon Feb 9 18:06:48 2015] ata5.00: exception Emask 0x0 SAct 0x0 SErr
>>> 0x0 action 0x6 frozen
>>> [Mon Feb 9 18:06:48 2015] ata5.00: failed command: SMART
>>> [Mon Feb 9 18:06:48 2015] ata5.00: cmd
>>> b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 7
>>> [Mon Feb 9 18:06:48 2015] res
>>> 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
> ^^^^^^^^^
> Smoking gun.
>
>>> [Mon Feb 9 18:06:48 2015] ata5.00: status: { DRDY }
>>> [Mon Feb 9 18:06:48 2015] ata5: hard resetting link
>>> [Mon Feb 9 18:06:58 2015] ata5: softreset failed (1st FIS failed)
>>> [Mon Feb 9 18:06:58 2015] ata5: hard resetting link
>>> [Mon Feb 9 18:07:08 2015] ata5: softreset failed (1st FIS failed)
>>> [Mon Feb 9 18:07:08 2015] ata5: hard resetting link
>>> [Mon Feb 9 18:07:12 2015] ata5: SATA link up 1.5 Gbps (SStatus 113
>>> SControl 310)
>>> [Mon Feb 9 18:07:12 2015] ata5.00: configured for UDMA/33
>>> [Mon Feb 9 18:07:12 2015] ata5: EH complete
>
> Notice that after a timeout error, the drive is unresponsive for several
> more seconds -- about 24 in your case.
>
>> .... read about timing mismatches
>> between the kernel and the hard drive, and how to solve that. There was
>> another post earlier today with some links to specific posts that will
>> be helpful (check the online archive).
>
> That would have been me. Start with this link for a description of what
> you are experiencing:
>
> http://marc.info/?l=linux-raid&m=135811522817345&w=1
>
> First, you need to protect yourself from timeout mismatch due to the use
> of desktop-grade drives. (Enterprise and raid-rated drives don't have
> this problem.)
>
> { If you were stuck in the middle of a replace a you had just
> worked-around your timeout problem, it would likely continue and
> complete. You've lost that opportunity. }
>
> Show us the output of "smartctl -x" for all of your drives if you'd like
> advice on your particular drives. (Pasted inline is preferred.)
>
> Second, you need to find and overwrite (with zeros) the bad sectors on
> your drives. Or ddrescue to a complete set of replacement drives and
> assemble those.
>
> Third, you need to set up a cron job to scrub your array regularly to
> clean out UREs before they accumulate beyond MD's ability to handle it
> (20 read errors in an hour, 10 per hour sustained).
>
> Phil
^ permalink raw reply
* (unknown),
From: Kyle Logue @ 2015-02-10 23:48 UTC (permalink / raw)
To: linux-raid
Phil:
I figured out that i could echo the larger timeout value into
/sys/block/sde/device/timeout, but when I ran the assemble again I got
a new error right at the very beginning:
mdadm: no RAID superblock on /dev/sdc1
mdadm: /dev/sdc1 has no superblock - assembly aborted
At this point should i try to ddrescue this device to a new 2TB drive
then retry the assemble? The sdc is still marked as a raid member in
the 'Disks' dialog, but is clearly having problems.
Thanks for your help,
Kyle L
On Tue, Feb 10, 2015 at 8:51 AM, Phil Turmel <philip@turmel.org> wrote:
> Hi Kyle,
>
> Your symptoms look like classic timeout mismatch. Details interleaved.
>
> On 02/10/2015 02:35 AM, Adam Goryachev wrote:
>
>> There are other people who will jump in and help you with your problem,
>> but I'll add a couple of pointers while you are waiting. See below.
>
>> On 10/02/15 15:20, Kyle Logue wrote:
>>> Hey all:
>>>
>>> I have a 5 disk software raid5 that was working fine until I decided
>>> to swap out an old disk with a new one.
>>>
>>> mdadm /dev/md0 --add /dev/sda1
>>> mdadm /dev/md0 --fail /dev/sde1
>
> As Adam pointed out, you should have used --replace, but you probably
> wouldn't have made it through the replace function anyways.
>
>>> At this point it started automatically rebuilding the array.
>>> About 60%? of the way in it stops and I see a lot of this repeated in
>>> my dmesg:
>>>
>>> [Mon Feb 9 18:06:48 2015] ata5.00: exception Emask 0x0 SAct 0x0 SErr
>>> 0x0 action 0x6 frozen
>>> [Mon Feb 9 18:06:48 2015] ata5.00: failed command: SMART
>>> [Mon Feb 9 18:06:48 2015] ata5.00: cmd
>>> b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 7
>>> [Mon Feb 9 18:06:48 2015] res
>>> 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
> ^^^^^^^^^
> Smoking gun.
>
>>> [Mon Feb 9 18:06:48 2015] ata5.00: status: { DRDY }
>>> [Mon Feb 9 18:06:48 2015] ata5: hard resetting link
>>> [Mon Feb 9 18:06:58 2015] ata5: softreset failed (1st FIS failed)
>>> [Mon Feb 9 18:06:58 2015] ata5: hard resetting link
>>> [Mon Feb 9 18:07:08 2015] ata5: softreset failed (1st FIS failed)
>>> [Mon Feb 9 18:07:08 2015] ata5: hard resetting link
>>> [Mon Feb 9 18:07:12 2015] ata5: SATA link up 1.5 Gbps (SStatus 113
>>> SControl 310)
>>> [Mon Feb 9 18:07:12 2015] ata5.00: configured for UDMA/33
>>> [Mon Feb 9 18:07:12 2015] ata5: EH complete
>
> Notice that after a timeout error, the drive is unresponsive for several
> more seconds -- about 24 in your case.
>
>> .... read about timing mismatches
>> between the kernel and the hard drive, and how to solve that. There was
>> another post earlier today with some links to specific posts that will
>> be helpful (check the online archive).
>
> That would have been me. Start with this link for a description of what
> you are experiencing:
>
> http://marc.info/?l=linux-raid&m=135811522817345&w=1
>
> First, you need to protect yourself from timeout mismatch due to the use
> of desktop-grade drives. (Enterprise and raid-rated drives don't have
> this problem.)
>
> { If you were stuck in the middle of a replace a you had just
> worked-around your timeout problem, it would likely continue and
> complete. You've lost that opportunity. }
>
> Show us the output of "smartctl -x" for all of your drives if you'd like
> advice on your particular drives. (Pasted inline is preferred.)
>
> Second, you need to find and overwrite (with zeros) the bad sectors on
> your drives. Or ddrescue to a complete set of replacement drives and
> assemble those.
>
> Third, you need to set up a cron job to scrub your array regularly to
> clean out UREs before they accumulate beyond MD's ability to handle it
> (20 read errors in an hour, 10 per hour sustained).
>
> Phil
^ permalink raw reply
* dead RAID6 array on CentOS6.6 / kernel 3.19
From: Adam Thompson @ 2015-02-11 1:53 UTC (permalink / raw)
To: linux-raid-u79uwXL29TY76Z2rM5mHXA
Cc: Board, MUUG, elrepo-XmKyDsxDEzbecEK0+Tl+uA
So, I was doing some work on the new server tonight, stopping/starting
NFS caused a kernel panic, and I thought rebooting would be a good
idea... Apparently not!
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sdd from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sdd>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sdd)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sdb from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sdb>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sdb)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sdc from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sdc>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sdc)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sdh from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sdh>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sdh)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sda from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sda>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sda)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sdg from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sdg>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sdg)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sdf from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sdf>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sdf)
> Feb 10 18:48:57 muug kernel: md: kicking non-fresh sde from array!
> Feb 10 18:48:57 muug kernel: md: unbind<sde>
> Feb 10 18:48:57 muug kernel: md: export_rdev(sde)
> Feb 10 18:48:57 muug kernel: md/raid:md127: not clean -- starting
> background reconstruction
> Feb 10 18:48:57 muug kernel: md/raid:md127: device sdk operational as
> raid disk 10
> Feb 10 18:48:57 muug kernel: md/raid:md127: device sdj operational as
> raid disk 9
> Feb 10 18:48:57 muug kernel: md/raid:md127: device sdi operational as
> raid disk 8
> Feb 10 18:48:57 muug kernel: md/raid:md127: device sdl operational as
> raid disk 0
> Feb 10 18:48:57 muug kernel: md/raid:md127: allocated 0kB
> Feb 10 18:48:57 muug kernel: md/raid:md127: not enough operational
> devices (8/12 failed)
> Feb 10 18:48:57 muug kernel: md/raid:md127: failed to run raid set.
> Feb 10 18:48:57 muug kernel: md: pers->run() failed ...
Sure enough, no array.
> [root@muug ~]# cat /proc/mdstat
> Personalities : [raid1] [raid6] [raid5] [raid4] [raid10]
> md0 : active raid1 sdm1[0] sdn1[1]
> 1048512 blocks super 1.0 [2/2] [UU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> md127 : inactive sdk[10] sdj[9] sdi[8] sdl[12]
> 15627550048 blocks super 1.2
>
> unused devices: <none>
Much to my very great chagrin, I had moved some (un-backed-up) personal
data onto that server temporarily, and [redacted] has some un-backed-up
data on there as well, so while *most* of the data on there is our FTP
mirror and can be recreated (at only the cost of bandwidth), there's a
bunch of stuff (about 2 or 3 TB, out of the ~10-20TB used) on there I'd
really, *really* rather not lose.
The server isn't backed up because it *is* the backup. Except for some
of the data, where this backup copy is currently the only copy. Plus,
where do you back up 40TB *to*?
For the linux-raid and elrepo mailing list readers, this is a CentOS 6.6
(x86_64) server, running the kernel from
kernel-ml-3.19.0-1.el6.elrepo.x86_64 to get support for the motherboard
chipset (ie31200_edac, in particular) and mdadm from
mdadm-3.3-6.el6_6.1.x86_64.
Array is a ~40TB XFS filesystem on LVM running on an md device composed
12 x 4TB drives in RAID6 (no hotspares). Mixture of HGST, Seagate and
WD drives, all NAS series, all in an Asus RS300-H8-PS12 system, and thus
spread across two SATA controllers. Note complete lack of partition
tables on RAID6 devices - this was deliberate, to work around
limitations of the ASUS BIOS compat. layer when presented with so many
partitioned devices. The system is not booting in EFI mode, if that
makes any difference.
/dev/sd[nm] are a pair of SSDs in RAID1 providing boot/root and
lvmcache. (No swap.) I don't think the lvmcache is part of the problem
here, since although the storage setup is moderately complex, it's
cleanly layered - we're NOT using LVM's built-in RAID that somehow uses
MD, we're using LVM on top of MD RAID.
Per the RAID Recovery wiki, I've already attempted to reassemble the
array with --force, which didn't quite work.
I ran the assemble command a second time, with almost identical results
- no messages about changing event counts or flags, but same 9-drive result.
Other than the kernel panic immediately prior to rebooting - which did
not render the system unbootable, we've only seen very occasional
glitches, with a few of the drives timing out once or twice. I'd
attributed that to vibration or controller starvation until now. The
reboot *appeared* to proceed successfully, there's no smoking gun in
/var/log/messages that I can identify other than some I/O errors on one
drive (/dev/sdc) during - AFAIK - the weekly "scrub".
Also per the RAID recovery wiki, at this point I'm stopping and
soliciting help before continuing with the attempt to force re-creation
of the md device.
I've included quite some data and output here, not sure what else will
be helpful at this juncture. Feel free to ask.
I'm noticing in the output that it looks like mdadm is trying to start
the new array before all the drives have been bound... but I'm not sure
what to do about that.
Thanks in advance for any useful insights that will enable me to recover
any of the data.
-Adam Thompson
System Administrator (for now), Past President,
Manitoba UNIX User Group
athompso-gKoiEJA+T1nk1uMJSBkQmQ@public.gmane.org
Output from mdadm --assemble --force :
> [root@muug ~]# mdadm --assemble --force /dev/md127 /dev/sd[lcbdhefgijka]
> Feb 10 19:10:15 muug kernel: md: md127 stopped.
> mdadm: forcing event count in /dev/sdc(1) from 26263 upto 26307
> mdadm: forcing event count in /dev/sdb(2) from 26263 upto 26307
> mdadm: forcing event count in /dev/sdd(3) from 26263 upto 26307
> mdadm: forcing event count in /dev/sdh(4) from 26263 upto 26307
> mdadm: forcing event count in /dev/sde(5) from 26263 upto 26307
> mdadm: clearing FAULTY flag for device 2 in /dev/md127 for /dev/sdc
> mdadm: clearing FAULTY flag for device 1 in /dev/md127 for /dev/sdb
> mdadm: clearing FAULTY flag for device 3 in /dev/md127 for /dev/sdd
> mdadm: clearing FAULTY flag for device 7 in /dev/md127 for /dev/sdh
> mdadm: clearing FAULTY flag for device 4 in /dev/md127 for /dev/sde
> mdadm: Marking array /dev/md127 as 'clean'
> Feb 10 19:10:18 muug kernel: md: bind<sdc>
> Feb 10 19:10:18 muug kernel: md: bind<sdb>
> Feb 10 19:10:18 muug kernel: md: bind<sdd>
> Feb 10 19:10:18 muug kernel: md: bind<sdh>
> Feb 10 19:10:18 muug kernel: md: bind<sde>
> Feb 10 19:10:18 muug kernel: md: bind<sdf>
> Feb 10 19:10:18 muug kernel: md: bind<sdg>
> Feb 10 19:10:18 muug kernel: md: bind<sdi>
> Feb 10 19:10:18 muug kernel: md: bind<sdj>
> mdadm: /dev/md127 assembled from 9 drives - not enough to start the array.
> Feb 10 19:10:18 muug kernel: md: bind<sdk>
> Feb 10 19:10:18 muug kernel: md: bind<sda>
> Feb 10 19:10:18 muug kernel: md: bind<sdl>
> Feb 10 19:10:18 muug kernel: md: md127 stopped.
> Feb 10 19:10:18 muug kernel: md: unbind<sdl>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdl)
> Feb 10 19:10:18 muug kernel: md: unbind<sda>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sda)
> Feb 10 19:10:18 muug kernel: md: unbind<sdk>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdk)
> Feb 10 19:10:18 muug kernel: md: unbind<sdj>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdj)
> Feb 10 19:10:18 muug kernel: md: unbind<sdi>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdi)
> Feb 10 19:10:18 muug kernel: md: unbind<sdg>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdg)
> Feb 10 19:10:18 muug kernel: md: unbind<sdf>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdf)
> Feb 10 19:10:18 muug kernel: md: unbind<sde>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sde)
> Feb 10 19:10:18 muug kernel: md: unbind<sdh>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdh)
> Feb 10 19:10:18 muug kernel: md: unbind<sdd>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdd)
> Feb 10 19:10:18 muug kernel: md: unbind<sdb>
> Feb 10 19:10:18 muug kernel: md: export_rdev(sdb)
> Feb 10 19:10:18 muug kernel: md: unbind<sdc>
> [root@muug ~]# Feb 10 19:10:18 muug kernel: md: export_rdev(sdc)
Output from mdadm --examine /dev/sd[abcdefghijkl] *before* running
--assemble --force :
> /dev/sda:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 85613308:17b75e83:7209c7e5:2dd2f59b
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 1bcd84d - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 11
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdb:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : ea2b9cb8:c2ad12e4:75d8b93b:0f752d85
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 52040b37 - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 2
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdc:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 56455a20:9fa5587d:1f61b896:43e5b8ef
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 1892155d - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 1
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdd:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : d60dd4e5:fb8565ee:1a348e5d:c5740d6a
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 904320b8 - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 3
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sde:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 55e6552e:f32eda98:44ed3844:1b831101
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : e869b0 - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 5
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdf:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : a8862067:9f25c7d1:55819755:54f0e781
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 4d501fb - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 6
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdg:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 8a747d19:da4f2e41:fc132de1:1e172e4a
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 7a74d389 - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 7
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdh:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 5cfa7095:2b13d03e:bed02e9a:cfb5b516
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 7993781c - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 4
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdi:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : fcf4686b:8bc3ed9b:1a88c898:f138d434
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : c93f5dc0 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 8
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdj:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : f069d226:cc3a580c:2ca5d244:2a53f3e6
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 533c8141 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 9
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdk:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 69da9e3f:f31bffff:022ce721:d73acdc3
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 199e4166 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 10
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdl:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : dc4dfde5:0e5c02e3:ba89e83f:e6d1664c
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 499ae9bd - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 0
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> [root@muug ~]#
Output from same *after* --assemble --force:
> /dev/sda:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 85613308:17b75e83:7209c7e5:2dd2f59b
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 1bcd84d - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 11
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdb:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : ea2b9cb8:c2ad12e4:75d8b93b:0f752d85
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 52040b63 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 2
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdc:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 56455a20:9fa5587d:1f61b896:43e5b8ef
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 18921589 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 1
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdd:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : d60dd4e5:fb8565ee:1a348e5d:c5740d6a
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 904320e4 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 3
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sde:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 55e6552e:f32eda98:44ed3844:1b831101
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : e869dc - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 5
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdf:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : a8862067:9f25c7d1:55819755:54f0e781
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 4d501fb - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 6
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdg:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 8a747d19:da4f2e41:fc132de1:1e172e4a
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 7a74d389 - correct
> Events : 26263
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 7
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdh:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 5cfa7095:2b13d03e:bed02e9a:cfb5b516
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:44 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 79937848 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 4
> Array State : AAAAAAAAAAAA ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdi:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : fcf4686b:8bc3ed9b:1a88c898:f138d434
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : c93f5dc0 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 8
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdj:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : f069d226:cc3a580c:2ca5d244:2a53f3e6
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 533c8141 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 9
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdk:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : active
> Device UUID : 69da9e3f:f31bffff:022ce721:d73acdc3
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 199e4166 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 10
> Array State : A.......AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
> /dev/sdl:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Name : muug.ca:ARRAY (local to host muug.ca)
> Creation Time : Thu Nov 20 09:35:10 2014
> Raid Level : raid6
> Raid Devices : 12
>
> Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
> Array Size : 39068875120 (37258.98 GiB 40006.53 GB)
> Data Offset : 262144 sectors
> Super Offset : 8 sectors
> Unused Space : before=262056 sectors, after=0 sectors
> State : clean
> Device UUID : dc4dfde5:0e5c02e3:ba89e83f:e6d1664c
>
> Internal Bitmap : 8 sectors from superblock
> Update Time : Tue Feb 10 18:47:47 2015
> Bad Block Log : 512 entries available at offset 72 sectors
> Checksum : 49a7e9c4 - correct
> Events : 26307
>
> Layout : left-symmetric
> Chunk Size : 4K
>
> Device Role : Active device 0
> Array State : AAAAAA..AAA. ('A' == active, '.' == missing, 'R' ==
> replacing)
Dmesg:
> Initializing cgroup subsys cpuset
> Initializing cgroup subsys cpu
> Initializing cgroup subsys cpuacct
> Linux version 3.19.0-1.el6.elrepo.x86_64 (mockbuild@Build64R6) (gcc
> version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) #1 SMP Mon Feb 9
> 11:00:58 EST 2015
> Command line: ro root=/dev/mapper/vg00-root rd_LVM_LV=vg00/swap
> LANG=en_CA.UTF-8 rd_LVM_LV=vg00/root SYSFONT=latarcyrheb-sun16
> crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us
> e820: BIOS-provided physical RAM map:
> BIOS-e820: [mem 0x0000000000000000-0x000000000009d7ff] usable
> BIOS-e820: [mem 0x000000000009d800-0x000000000009ffff] reserved
> BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
> BIOS-e820: [mem 0x0000000000100000-0x000000006e1d0fff] usable
> BIOS-e820: [mem 0x000000006e1d1000-0x000000006e1d7fff] ACPI NVS
> BIOS-e820: [mem 0x000000006e1d8000-0x000000007d7effff] usable
> BIOS-e820: [mem 0x000000007d7f0000-0x000000007da0bfff] reserved
> BIOS-e820: [mem 0x000000007da0c000-0x000000007da26fff] ACPI data
> BIOS-e820: [mem 0x000000007da27000-0x000000007daf0fff] ACPI NVS
> BIOS-e820: [mem 0x000000007daf1000-0x000000007effefff] reserved
> BIOS-e820: [mem 0x000000007efff000-0x000000007effffff] usable
> BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
> BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
> BIOS-e820: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
> BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
> BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
> BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
> BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable
> NX (Execute Disable) protection: active
> SMBIOS 2.7 present.
> DMI: ASUSTeK COMPUTER INC. RS300-H8-PS12/P9D-MH Series, BIOS 0701
> 07/03/2014
> e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
> e820: remove [mem 0x000a0000-0x000fffff] usable
> AGP: No AGP bridge found
> e820: last_pfn = 0x880000 max_arch_pfn = 0x400000000
> MTRR default type: uncachable
> MTRR fixed ranges enabled:
> 00000-9FFFF write-back
> A0000-BFFFF uncachable
> C0000-CBFFF write-protect
> CC000-E7FFF uncachable
> E8000-FFFFF write-protect
> MTRR variable ranges enabled:
> 0 base 0000000000 mask 7800000000 write-back
> 1 base 0800000000 mask 7F80000000 write-back
> 2 base 0080000000 mask 7F80000000 uncachable
> 3 disabled
> 4 disabled
> 5 disabled
> 6 disabled
> 7 disabled
> 8 disabled
> 9 disabled
> PAT configuration [0-7]: WB WC UC- UC WB WC UC- UC
> original variable MTRRs
> reg 0, base: 0GB, range: 32GB, type WB
> reg 1, base: 32GB, range: 2GB, type WB
> reg 2, base: 2GB, range: 2GB, type UC
> total RAM covered: 32768M
> Found optimal setting for mtrr clean up
> gran_size: 64K chunk_size: 64K num_reg: 5 lose cover RAM: 0G
> New variable MTRRs
> reg 0, base: 0GB, range: 2GB, type WB
> reg 1, base: 4GB, range: 4GB, type WB
> reg 2, base: 8GB, range: 8GB, type WB
> reg 3, base: 16GB, range: 16GB, type WB
> reg 4, base: 32GB, range: 2GB, type WB
> e820: update [mem 0x80000000-0xffffffff] usable ==> reserved
> e820: last_pfn = 0x7f000 max_arch_pfn = 0x400000000
> found SMP MP-table at [mem 0x000fd960-0x000fd96f] mapped at
> [ffff8800000fd960]
> Base memory trampoline at [ffff880000096000] 96000 size 28672
> Using GB pages for direct mapping
> init_memory_mapping: [mem 0x00000000-0x000fffff]
> [mem 0x00000000-0x000fffff] page 4k
> BRK [0x0211e000, 0x0211efff] PGTABLE
> BRK [0x0211f000, 0x0211ffff] PGTABLE
> BRK [0x02120000, 0x02120fff] PGTABLE
> init_memory_mapping: [mem 0x87fe00000-0x87fffffff]
> [mem 0x87fe00000-0x87fffffff] page 1G
> init_memory_mapping: [mem 0x860000000-0x87fdfffff]
> [mem 0x860000000-0x87fdfffff] page 1G
> init_memory_mapping: [mem 0x00100000-0x6e1d0fff]
> [mem 0x00100000-0x001fffff] page 4k
> [mem 0x00200000-0x6dffffff] page 2M
> [mem 0x6e000000-0x6e1d0fff] page 4k
> init_memory_mapping: [mem 0x6e1d8000-0x7d7effff]
> [mem 0x6e1d8000-0x6e1fffff] page 4k
> [mem 0x6e200000-0x7d5fffff] page 2M
> [mem 0x7d600000-0x7d7effff] page 4k
> BRK [0x02121000, 0x02121fff] PGTABLE
> init_memory_mapping: [mem 0x7efff000-0x7effffff]
> [mem 0x7efff000-0x7effffff] page 4k
> BRK [0x02122000, 0x02122fff] PGTABLE
> init_memory_mapping: [mem 0x100000000-0x85fffffff]
> [mem 0x100000000-0x85fffffff] page 1G
> RAMDISK: [mem 0x36a98000-0x37feffff]
> ACPI: Early table checksum verification disabled
> ACPI: RSDP 0x00000000000F0490 000024 (v02 ALASKA)
> ACPI: XSDT 0x000000007DA110A0 0000B4 (v01 ALASKA A M I 01072009 AMI
> 00010013)
> ACPI: FACP 0x000000007DA20A28 00010C (v05 ALASKA A M I 01072009 AMI
> 00010013)
> ACPI: DSDT 0x000000007DA111E8 00F83B (v02 ALASKA A M I 00000010 INTL
> 20091112)
> ACPI: FACS 0x000000007DAEF080 000040
> ACPI: APIC 0x000000007DA20B38 000092 (v03 ALASKA A M I 01072009 AMI
> 00010013)
> ACPI: FPDT 0x000000007DA20BD0 000044 (v01 ALASKA A M I 01072009 AMI
> 00010013)
> ACPI: SSDT 0x000000007DA20C18 000539 (v01 PmRef Cpu0Ist 00003000 INTL
> 20120711)
> ACPI: SSDT 0x000000007DA21158 000AD8 (v01 PmRef CpuPm 00003000 INTL
> 20120711)
> ACPI: SSDT 0x000000007DA21C30 0002DE (v01 PmRef Cpu0Tst 00003000 INTL
> 20120711)
> ACPI: SSDT 0x000000007DA21F10 000348 (v01 PmRef ApTst 00003000 INTL
> 20120711)
> ACPI: MCFG 0x000000007DA22258 00003C (v01 ALASKA A M I 01072009 MSFT
> 00000097)
> ACPI: PRAD 0x000000007DA22298 0000BE (v02 PRADID PRADTID 00000001 MSFT
> 03000001)
> ACPI: HPET 0x000000007DA22358 000038 (v01 ALASKA A M I 01072009 AMI.
> 00000005)
> ACPI: SSDT 0x000000007DA22390 00036D (v01 SataRe SataTabl 00001000
> INTL 20120711)
> ACPI: SSDT 0x000000007DA22700 003562 (v01 SaSsdt SaSsdt 00003000 INTL
> 20091112)
> ACPI: BERT 0x000000007DA26188 000030 (v01 AMI AMI BERT
> 00000000 00000000)
> ACPI: SPMI 0x000000007DA25CC0 000040 (v05 A M I OEMSPMI 00000000 AMI.
> 00000000)
> ACPI: DMAR 0x000000007DA25D00 000080 (v01 INTEL HSW 00000001 INTL
> 00000001)
> ACPI: EINJ 0x000000007DA25D80 000130 (v01 AMI AMI EINJ
> 00000000 00000000)
> ACPI: ERST 0x000000007DA25EB0 000230 (v01 AMIER AMI ERST
> 00000000 00000000)
> ACPI: HEST 0x000000007DA260E0 0000A8 (v01 AMI AMI HEST
> 00000000 00000000)
> ACPI: Local APIC address 0xfee00000
> No NUMA configuration found
> Faking a node at [mem 0x0000000000000000-0x000000087fffffff]
> NODE_DATA(0) allocated [mem 0x87ffd8000-0x87fffdfff]
> kexec: crashkernel: memory value expected
> [ffffea0000000000-ffffea0021ffffff] PMD ->
> [ffff88085f600000-ffff88087f5fffff] on node 0
> Zone ranges:
> DMA [mem 0x00001000-0x00ffffff]
> DMA32 [mem 0x01000000-0xffffffff]
> Normal [mem 0x100000000-0x87fffffff]
> Movable zone start for each node
> Early memory node ranges
> node 0: [mem 0x00001000-0x0009cfff]
> node 0: [mem 0x00100000-0x6e1d0fff]
> node 0: [mem 0x6e1d8000-0x7d7effff]
> node 0: [mem 0x7efff000-0x7effffff]
> node 0: [mem 0x100000000-0x87fffffff]
> Initmem setup node 0 [mem 0x00001000-0x87fffffff]
> On node 0 totalpages: 8378246
> DMA zone: 64 pages used for memmap
> DMA zone: 22 pages reserved
> DMA zone: 3996 pages, LIFO batch:0
> DMA32 zone: 7968 pages used for memmap
> DMA32 zone: 509930 pages, LIFO batch:31
> Normal zone: 122880 pages used for memmap
> Normal zone: 7864320 pages, LIFO batch:31
> ACPI: PM-Timer IO Port: 0x1808
> ACPI: Local APIC address 0xfee00000
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
> ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
> ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
> ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
> ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
> ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
> ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
> ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> ACPI: IRQ0 used by override.
> ACPI: IRQ9 used by override.
> Using ACPI (MADT) for SMP configuration information
> ACPI: HPET id: 0x8086a701 base: 0xfed00000
> smpboot: Allowing 8 CPUs, 0 hotplug CPUs
> PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
> PM: Registered nosave memory: [mem 0x0009d000-0x0009dfff]
> PM: Registered nosave memory: [mem 0x0009e000-0x0009ffff]
> PM: Registered nosave memory: [mem 0x000a0000-0x000dffff]
> PM: Registered nosave memory: [mem 0x000e0000-0x000fffff]
> PM: Registered nosave memory: [mem 0x6e1d1000-0x6e1d7fff]
> PM: Registered nosave memory: [mem 0x7d7f0000-0x7da0bfff]
> PM: Registered nosave memory: [mem 0x7da0c000-0x7da26fff]
> PM: Registered nosave memory: [mem 0x7da27000-0x7daf0fff]
> PM: Registered nosave memory: [mem 0x7daf1000-0x7effefff]
> PM: Registered nosave memory: [mem 0x7f000000-0xdfffffff]
> PM: Registered nosave memory: [mem 0xe0000000-0xefffffff]
> PM: Registered nosave memory: [mem 0xf0000000-0xfebfffff]
> PM: Registered nosave memory: [mem 0xfec00000-0xfec00fff]
> PM: Registered nosave memory: [mem 0xfec01000-0xfecfffff]
> PM: Registered nosave memory: [mem 0xfed00000-0xfed03fff]
> PM: Registered nosave memory: [mem 0xfed04000-0xfed1bfff]
> PM: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff]
> PM: Registered nosave memory: [mem 0xfed20000-0xfedfffff]
> PM: Registered nosave memory: [mem 0xfee00000-0xfee00fff]
> PM: Registered nosave memory: [mem 0xfee01000-0xfeffffff]
> PM: Registered nosave memory: [mem 0xff000000-0xffffffff]
> e820: [mem 0x7f000000-0xdfffffff] available for PCI devices
> Booting paravirtualized kernel on bare hardware
> setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
> PERCPU: Embedded 31 pages/cpu @ffff88087fc00000 s86336 r8192 d32448
> u262144
> pcpu-alloc: s86336 r8192 d32448 u262144 alloc=1*2097152
> pcpu-alloc: [0] 0 1 2 3 4 5 6 7
> Built 1 zonelists in Node order, mobility grouping on. Total pages:
> 8247312
> Policy zone: Normal
> Kernel command line: ro root=/dev/mapper/vg00-root rd_LVM_LV=vg00/swap
> LANG=en_CA.UTF-8 rd_LVM_LV=vg00/root SYSFONT=latarcyrheb-sun16
> crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us
> PID hash table entries: 4096 (order: 3, 32768 bytes)
> xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form
> AGP: Checking aperture...
> AGP: No AGP bridge found
> Memory: 32881732K/33512984K available (6652K kernel code, 1412K
> rwdata, 3316K rodata, 1960K init, 1832K bss, 631252K reserved, 0K
> cma-reserved)
> Hierarchical RCU implementation.
> RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.
> RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8
> NR_IRQS:524544 nr_irqs:488 16
> Offload RCU callbacks from all CPUs
> Offload RCU callbacks from CPUs: 0-7.
> Console: colour VGA+ 80x25
> console [tty0] enabled
> hpet clockevent registered
> tsc: Fast TSC calibration using PIT
> spurious 8259A interrupt: IRQ7.
> tsc: Detected 3292.465 MHz processor
> Calibrating delay loop (skipped), value calculated using timer
> frequency.. 6584.93 BogoMIPS (lpj=3292465)
> pid_max: default: 32768 minimum: 301
> ACPI: Core revision 20141107
> ACPI: All ACPI Tables successfully acquired
> Security Framework initialized
> SELinux: Initializing.
> SELinux: Starting in permissive mode
> Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
> Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
> Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
> Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
> Initializing cgroup subsys memory
> Initializing cgroup subsys devices
> Initializing cgroup subsys freezer
> Initializing cgroup subsys net_cls
> Initializing cgroup subsys blkio
> Initializing cgroup subsys perf_event
> Initializing cgroup subsys net_prio
> CPU: Physical Processor ID: 0
> CPU: Processor Core ID: 0
> ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
> ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
> mce: CPU supports 9 MCE banks
> CPU0: Thermal monitoring enabled (TM1)
> Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024
> Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4
> Freeing SMP alternatives memory: 24K (ffffffff81f4c000 - ffffffff81f52000)
> ftrace: allocating 27349 entries in 107 pages
> dmar: Host address width 39
> dmar: DRHD base: 0x000000fed90000 flags: 0x1
> dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap d2008c20660462 ecap
> f010da
> dmar: RMRR base: 0x0000007d98a000 end: 0x0000007d996fff
> IOAPIC id 8 under DRHD base 0xfed90000 IOMMU 0
> HPET id 0 under DRHD base 0xfed90000
> Queued invalidation will be enabled to support x2apic and Intr-remapping.
> Enabled IRQ remapping in x2apic mode
> Enabling x2apic
> Enabled x2apic
> Switched APIC routing to cluster x2apic.
> ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
> smpboot: CPU0: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz (fam: 06,
> model: 3c, stepping: 03)
> TSC deadline timer enabled
> Performance Events: PEBS fmt2+, 16-deep LBR, Haswell events,
> full-width counters, Intel PMU driver.
> ... version: 3
> ... bit width: 48
> ... generic registers: 4
> ... value mask: 0000ffffffffffff
> ... max period: 0000ffffffffffff
> ... fixed-purpose events: 3
> ... event mask: 000000070000000f
> x86: Booting SMP configuration:
> .... node #0, CPUs: #1
> NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU
> counter.
> #2 #3 #4 #5 #6 #7
> x86: Booted up 1 node, 8 CPUs
> smpboot: Total of 8 processors activated (52679.44 BogoMIPS)
> devtmpfs: initialized
> PM: Registering ACPI NVS region [mem 0x6e1d1000-0x6e1d7fff] (28672 bytes)
> PM: Registering ACPI NVS region [mem 0x7da27000-0x7daf0fff] (827392 bytes)
> NET: Registered protocol family 16
> cpuidle: using governor ladder
> cpuidle: using governor menu
> ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
> ACPI: bus type PCI registered
> PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
> 0xe0000000-0xefffffff] (base 0xe0000000)
> PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
> PCI: Using configuration type 1 for base access
> ACPI: Added _OSI(Module Device)
> ACPI: Added _OSI(Processor Device)
> ACPI: Added _OSI(3.0 _SCP Extensions)
> ACPI: Added _OSI(Processor Aggregator Device)
> ACPI: Executed 1 blocks of module-level executable AML code
> [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
> ACPI: Dynamic OEM Table Load:
> ACPI: SSDT 0xFFFF88085B926400 0003D3 (v01 PmRef Cpu0Cst 00003001 INTL
> 20120711)
> ACPI: Dynamic OEM Table Load:
> ACPI: SSDT 0xFFFF88085B413800 0005AA (v01 PmRef ApIst 00003000 INTL
> 20120711)
> ACPI: Dynamic OEM Table Load:
> ACPI: SSDT 0xFFFF88085B921800 000119 (v01 PmRef ApCst 00003000 INTL
> 20120711)
> ACPI: Interpreter enabled
> ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_]
> (20141107/hwxface-580)
> ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_]
> (20141107/hwxface-580)
> ACPI: (supports S0 S3 S4 S5)
> ACPI: Using IOAPIC for interrupt routing
> HEST: Table parsing has been initialized.
> PCI: Using host bridge windows from ACPI; if necessary, use
> "pci=nocrs" and report a bug
> ACPI: Power Resource [FN00] (off)
> ACPI: Power Resource [FN01] (off)
> ACPI: Power Resource [FN02] (off)
> ACPI: Power Resource [FN03] (off)
> ACPI: Power Resource [FN04] (off)
> ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])
> acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM
> Segments MSI]
> acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER
> PCIeCapability]
> PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [bus 00-fe]
> pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7]
> pci_bus 0000:00: root bus resource [io 0x0d00-0xffff]
> pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
> pci_bus 0000:00: root bus resource [mem 0x000cc000-0x000cffff]
> pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000d3fff]
> pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff]
> pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff]
> pci_bus 0000:00: root bus resource [mem 0x000dc000-0x000dffff]
> pci_bus 0000:00: root bus resource [mem 0x000e0000-0x000e3fff]
> pci_bus 0000:00: root bus resource [mem 0x000e4000-0x000e7fff]
> pci_bus 0000:00: root bus resource [mem 0x80000000-0xfeafffff]
> pci 0000:00:00.0: [8086:0c08] type 00 class 0x060000
> pci 0000:00:01.0: [8086:0c01] type 01 class 0x060400
> pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
> pci 0000:00:01.0: System wakeup disabled by ACPI
> pci 0000:00:01.2: [8086:0c09] type 01 class 0x060400
> pci 0000:00:01.2: PME# supported from D0 D3hot D3cold
> pci 0000:00:01.2: System wakeup disabled by ACPI
> pci 0000:00:14.0: [8086:8c31] type 00 class 0x0c0330
> pci 0000:00:14.0: reg 0x10: [mem 0xde600000-0xde60ffff 64bit]
> pci 0000:00:14.0: PME# supported from D3hot D3cold
> pci 0000:00:14.0: System wakeup disabled by ACPI
> pci 0000:00:16.0: [8086:8c3a] type 00 class 0x078000
> pci 0000:00:16.0: reg 0x10: [mem 0xde615000-0xde61500f 64bit]
> pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
> pci 0000:00:16.1: [8086:8c3b] type 00 class 0x078000
> pci 0000:00:16.1: reg 0x10: [mem 0xde614000-0xde61400f 64bit]
> pci 0000:00:16.1: PME# supported from D0 D3hot D3cold
> pci 0000:00:1a.0: [8086:8c2d] type 00 class 0x0c0320
> pci 0000:00:1a.0: reg 0x10: [mem 0xde612000-0xde6123ff]
> pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
> pci 0000:00:1a.0: System wakeup disabled by ACPI
> pci 0000:00:1c.0: [8086:8c10] type 01 class 0x060400
> pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
> pci 0000:00:1c.0: Enabling MPC IRBNCE
> pci 0000:00:1c.0: Intel PCH root port ACS workaround enabled
> pci 0000:00:1c.0: System wakeup disabled by ACPI
> pci 0000:00:1c.1: [8086:8c12] type 01 class 0x060400
> pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
> pci 0000:00:1c.1: Enabling MPC IRBNCE
> pci 0000:00:1c.1: Intel PCH root port ACS workaround enabled
> pci 0000:00:1c.1: System wakeup disabled by ACPI
> pci 0000:00:1c.2: [8086:8c14] type 01 class 0x060400
> pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold
> pci 0000:00:1c.2: Enabling MPC IRBNCE
> pci 0000:00:1c.2: Intel PCH root port ACS workaround enabled
> pci 0000:00:1c.2: System wakeup disabled by ACPI
> pci 0000:00:1d.0: [8086:8c26] type 00 class 0x0c0320
> pci 0000:00:1d.0: reg 0x10: [mem 0xde611000-0xde6113ff]
> pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
> pci 0000:00:1d.0: System wakeup disabled by ACPI
> pci 0000:00:1f.0: [8086:8c54] type 00 class 0x060100
> pci 0000:00:1f.2: [8086:8c02] type 00 class 0x010601
> pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077]
> pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063]
> pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057]
> pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043]
> pci 0000:00:1f.2: reg 0x20: [io 0xf020-0xf03f]
> pci 0000:00:1f.2: reg 0x24: [mem 0xde610000-0xde6107ff]
> pci 0000:00:1f.2: PME# supported from D3hot
> pci 0000:00:1f.3: [8086:8c22] type 00 class 0x0c0500
> pci 0000:00:1f.3: reg 0x10: [mem 0x7ffff01000-0x7ffff010ff 64bit]
> pci 0000:00:1f.3: reg 0x20: [io 0xf000-0xf01f]
> pci 0000:01:00.0: [1000:0087] type 00 class 0x010700
> pci 0000:01:00.0: reg 0x10: [io 0xe000-0xe0ff]
> pci 0000:01:00.0: reg 0x14: [mem 0xde240000-0xde24ffff 64bit]
> pci 0000:01:00.0: reg 0x1c: [mem 0xde200000-0xde23ffff 64bit]
> pci 0000:01:00.0: reg 0x30: [mem 0xde100000-0xde1fffff pref]
> pci 0000:01:00.0: supports D1 D2
> pci 0000:01:00.0: System wakeup disabled by ACPI
> pci 0000:00:01.0: PCI bridge to [bus 01]
> pci 0000:00:01.0: bridge window [io 0xe000-0xefff]
> pci 0000:00:01.0: bridge window [mem 0xde100000-0xde2fffff]
> pci 0000:02:00.0: [14e4:16a1] type 00 class 0x020000
> pci 0000:02:00.0: reg 0x10: [mem 0xdb000000-0xdb7fffff 64bit pref]
> pci 0000:02:00.0: reg 0x18: [mem 0xda800000-0xdaffffff 64bit pref]
> pci 0000:02:00.0: reg 0x20: [mem 0xdb810000-0xdb81ffff 64bit pref]
> pci 0000:02:00.0: reg 0x30: [mem 0xde580000-0xde5fffff pref]
> pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
> pci 0000:02:00.0: System wakeup disabled by ACPI
> pci 0000:02:00.1: [14e4:16a1] type 00 class 0x020000
> pci 0000:02:00.1: reg 0x10: [mem 0xda000000-0xda7fffff 64bit pref]
> pci 0000:02:00.1: reg 0x18: [mem 0xd9800000-0xd9ffffff 64bit pref]
> pci 0000:02:00.1: reg 0x20: [mem 0xdb800000-0xdb80ffff 64bit pref]
> pci 0000:02:00.1: reg 0x30: [mem 0xde500000-0xde57ffff pref]
> pci 0000:02:00.1: PME# supported from D0 D3hot D3cold
> pci 0000:00:01.2: PCI bridge to [bus 02]
> pci 0000:00:01.2: bridge window [mem 0xde500000-0xde5fffff]
> pci 0000:00:01.2: bridge window [mem 0xd9800000-0xdb8fffff 64bit pref]
> pci 0000:03:00.0: [1a03:1150] type 01 class 0x060400
> pci 0000:03:00.0: supports D1 D2
> pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold
> pci 0000:03:00.0: System wakeup disabled by ACPI
> pci 0000:00:1c.0: PCI bridge to [bus 03-04]
> pci 0000:00:1c.0: bridge window [io 0xd000-0xdfff]
> pci 0000:00:1c.0: bridge window [mem 0xdc000000-0xde0fffff]
> pci 0000:04:00.0: [1a03:2000] type 00 class 0x030000
> pci 0000:04:00.0: reg 0x10: [mem 0xdc000000-0xddffffff]
> pci 0000:04:00.0: reg 0x14: [mem 0xde000000-0xde01ffff]
> pci 0000:04:00.0: reg 0x18: [io 0xd000-0xd07f]
> pci 0000:04:00.0: supports D1 D2
> pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
> pci 0000:03:00.0: PCI bridge to [bus 04]
> pci 0000:03:00.0: bridge window [io 0xd000-0xdfff]
> pci 0000:03:00.0: bridge window [mem 0xdc000000-0xde0fffff]
> pci 0000:05:00.0: [8086:1533] type 00 class 0x020000
> pci 0000:05:00.0: reg 0x10: [mem 0xde400000-0xde47ffff]
> pci 0000:05:00.0: reg 0x18: [io 0xc000-0xc01f]
> pci 0000:05:00.0: reg 0x1c: [mem 0xde480000-0xde483fff]
> pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
> pci 0000:05:00.0: System wakeup disabled by ACPI
> pci 0000:00:1c.1: PCI bridge to [bus 05]
> pci 0000:00:1c.1: bridge window [io 0xc000-0xcfff]
> pci 0000:00:1c.1: bridge window [mem 0xde400000-0xde4fffff]
> pci 0000:06:00.0: [8086:1533] type 00 class 0x020000
> pci 0000:06:00.0: reg 0x10: [mem 0xde300000-0xde37ffff]
> pci 0000:06:00.0: reg 0x18: [io 0xb000-0xb01f]
> pci 0000:06:00.0: reg 0x1c: [mem 0xde380000-0xde383fff]
> pci 0000:06:00.0: PME# supported from D0 D3hot D3cold
> pci 0000:06:00.0: System wakeup disabled by ACPI
> pci 0000:00:1c.2: PCI bridge to [bus 06]
> pci 0000:00:1c.2: bridge window [io 0xb000-0xbfff]
> pci 0000:00:1c.2: bridge window [mem 0xde300000-0xde3fffff]
> acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported)
> ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 10 *11 12 14 15)
> ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 *10 11 12 14 15)
> ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 *10 11 12 14 15)
> ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 10 11 12 14 15)
> ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 10 *11 12 14 15)
> ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 10 11 12 14 15) *0,
> disabled.
> ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 10 11 12 14 15) *0,
> disabled.
> ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 *5 6 10 11 12 14 15)
> ACPI: Enabled 5 GPEs in block 00 to 3F
> vgaarb: setting as boot device: PCI:0000:04:00.0
> vgaarb: device added:
> PCI:0000:04:00.0,decodes=io+mem,owns=io+mem,locks=none
> vgaarb: loaded
> vgaarb: bridge control possible 0000:04:00.0
> SCSI subsystem initialized
> libata version 3.00 loaded.
> ACPI: bus type USB registered
> usbcore: registered new interface driver usbfs
> usbcore: registered new interface driver hub
> usbcore: registered new device driver usb
> PCI: Using ACPI for IRQ routing
> PCI: pci_cache_line_size set to 64 bytes
> pci 0000:00:1f.3: can't claim BAR 0 [mem 0x7ffff01000-0x7ffff010ff
> 64bit]: no compatible bridge window
> e820: reserve RAM buffer [mem 0x0009d800-0x0009ffff]
> e820: reserve RAM buffer [mem 0x6e1d1000-0x6fffffff]
> e820: reserve RAM buffer [mem 0x7d7f0000-0x7fffffff]
> e820: reserve RAM buffer [mem 0x7f000000-0x7fffffff]
> NetLabel: Initializing
> NetLabel: domain hash size = 128
> NetLabel: protocols = UNLABELED CIPSOv4
> NetLabel: unlabeled traffic allowed by default
> hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
> hpet0: 8 comparators, 64-bit 14.318180 MHz counter
> Switched to clocksource hpet
> pnp: PnP ACPI init
> system 00:00: [mem 0xfed40000-0xfed44fff] has been reserved
> system 00:00: Plug and Play ACPI device, IDs PNP0c01 (active)
> system 00:01: [io 0x0680-0x069f] has been reserved
> system 00:01: [io 0xffff] has been reserved
> system 00:01: [io 0xffff] has been reserved
> system 00:01: [io 0xffff] has been reserved
> system 00:01: [io 0x1c00-0x1cfe] has been reserved
> system 00:01: [io 0x1d00-0x1dfe] has been reserved
> system 00:01: [io 0x1e00-0x1efe] has been reserved
> system 00:01: [io 0x1f00-0x1ffe] has been reserved
> system 00:01: [io 0x1800-0x18fe] could not be reserved
> system 00:01: [io 0x164e-0x164f] has been reserved
> system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
> pnp 00:02: Plug and Play ACPI device, IDs PNP0b00 (active)
> system 00:03: [io 0x1854-0x1857] has been reserved
> system 00:03: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active)
> system 00:04: [io 0x0290-0x029f] has been reserved
> system 00:04: [io 0x02a0-0x02af] has been reserved
> system 00:04: Plug and Play ACPI device, IDs PNP0c02 (active)
> pnp 00:05: [dma 0 disabled]
> pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
> pnp 00:06: [dma 0 disabled]
> pnp 00:06: Plug and Play ACPI device, IDs PNP0501 (active)
> system 00:07: [io 0x04d0-0x04d1] has been reserved
> system 00:07: Plug and Play ACPI device, IDs PNP0c02 (active)
> pnp 00:08: Plug and Play ACPI device, IDs IPI0001 (active)
> system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved
> system 00:09: [mem 0xfed10000-0xfed17fff] has been reserved
> system 00:09: [mem 0xfed18000-0xfed18fff] has been reserved
> system 00:09: [mem 0xfed19000-0xfed19fff] has been reserved
> system 00:09: [mem 0xe0000000-0xefffffff] has been reserved
> system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
> system 00:09: [mem 0xfed90000-0xfed93fff] could not be reserved
> system 00:09: [mem 0xfed45000-0xfed8ffff] has been reserved
> system 00:09: [mem 0xff000000-0xffffffff] has been reserved
> system 00:09: [mem 0xfee00000-0xfeefffff] could not be reserved
> system 00:09: [mem 0xdffdf000-0xdffdffff] has been reserved
> system 00:09: [mem 0xdffe0000-0xdffeffff] has been reserved
> system 00:09: Plug and Play ACPI device, IDs PNP0c02 (active)
> pnp: PnP ACPI: found 10 devices
> pci 0000:00:1f.3: BAR 0: assigned [mem 0x80000000-0x800000ff 64bit]
> pci 0000:00:01.0: PCI bridge to [bus 01]
> pci 0000:00:01.0: bridge window [io 0xe000-0xefff]
> pci 0000:00:01.0: bridge window [mem 0xde100000-0xde2fffff]
> pci 0000:00:01.2: PCI bridge to [bus 02]
> pci 0000:00:01.2: bridge window [mem 0xde500000-0xde5fffff]
> pci 0000:00:01.2: bridge window [mem 0xd9800000-0xdb8fffff 64bit pref]
> pci 0000:03:00.0: PCI bridge to [bus 04]
> pci 0000:03:00.0: bridge window [io 0xd000-0xdfff]
> pci 0000:03:00.0: bridge window [mem 0xdc000000-0xde0fffff]
> pci 0000:00:1c.0: PCI bridge to [bus 03-04]
> pci 0000:00:1c.0: bridge window [io 0xd000-0xdfff]
> pci 0000:00:1c.0: bridge window [mem 0xdc000000-0xde0fffff]
> pci 0000:00:1c.1: PCI bridge to [bus 05]
> pci 0000:00:1c.1: bridge window [io 0xc000-0xcfff]
> pci 0000:00:1c.1: bridge window [mem 0xde400000-0xde4fffff]
> pci 0000:00:1c.2: PCI bridge to [bus 06]
> pci 0000:00:1c.2: bridge window [io 0xb000-0xbfff]
> pci 0000:00:1c.2: bridge window [mem 0xde300000-0xde3fffff]
> pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7]
> pci_bus 0000:00: resource 5 [io 0x0d00-0xffff]
> pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
> pci_bus 0000:00: resource 7 [mem 0x000cc000-0x000cffff]
> pci_bus 0000:00: resource 8 [mem 0x000d0000-0x000d3fff]
> pci_bus 0000:00: resource 9 [mem 0x000d4000-0x000d7fff]
> pci_bus 0000:00: resource 10 [mem 0x000d8000-0x000dbfff]
> pci_bus 0000:00: resource 11 [mem 0x000dc000-0x000dffff]
> pci_bus 0000:00: resource 12 [mem 0x000e0000-0x000e3fff]
> pci_bus 0000:00: resource 13 [mem 0x000e4000-0x000e7fff]
> pci_bus 0000:00: resource 14 [mem 0x80000000-0xfeafffff]
> pci_bus 0000:01: resource 0 [io 0xe000-0xefff]
> pci_bus 0000:01: resource 1 [mem 0xde100000-0xde2fffff]
> pci_bus 0000:02: resource 1 [mem 0xde500000-0xde5fffff]
> pci_bus 0000:02: resource 2 [mem 0xd9800000-0xdb8fffff 64bit pref]
> pci_bus 0000:03: resource 0 [io 0xd000-0xdfff]
> pci_bus 0000:03: resource 1 [mem 0xdc000000-0xde0fffff]
> pci_bus 0000:04: resource 0 [io 0xd000-0xdfff]
> pci_bus 0000:04: resource 1 [mem 0xdc000000-0xde0fffff]
> pci_bus 0000:05: resource 0 [io 0xc000-0xcfff]
> pci_bus 0000:05: resource 1 [mem 0xde400000-0xde4fffff]
> pci_bus 0000:06: resource 0 [io 0xb000-0xbfff]
> pci_bus 0000:06: resource 1 [mem 0xde300000-0xde3fffff]
> NET: Registered protocol family 2
> TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
> TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> TCP: Hash tables configured (established 262144 bind 65536)
> TCP: reno registered
> UDP hash table entries: 16384 (order: 7, 524288 bytes)
> UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
> NET: Registered protocol family 1
> pci 0000:04:00.0: Video device with shadowed ROM
> PCI: CLS 64 bytes, default 64
> Trying to unpack rootfs image as initramfs...
> Freeing initrd memory: 21856K (ffff880036a98000 - ffff880037ff0000)
> DMAR: No ATSR found
> IOMMU: dmar0 using Queued invalidation
> IOMMU: Setting RMRR:
> IOMMU: Setting identity map for device 0000:00:14.0 [0x7d98a000 -
> 0x7d996fff]
> IOMMU: Setting identity map for device 0000:00:1a.0 [0x7d98a000 -
> 0x7d996fff]
> IOMMU: Setting identity map for device 0000:00:1d.0 [0x7d98a000 -
> 0x7d996fff]
> IOMMU: Prepare 0-16MiB unity mapping for LPC
> IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
> PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
> RAPL PMU detected, hw unit 2^-14 Joules, API unit is 2^-32 Joules, 4
> fixed counters 655360 ms ovfl timer
> futex hash table entries: 2048 (order: 5, 131072 bytes)
> audit: initializing netlink subsys (disabled)
> audit: type=2000 audit(1423615719.608:1): initialized
> HugeTLB registered 2 MB page size, pre-allocated 0 pages
> zpool: loaded
> zbud: loaded
> VFS: Disk quotas dquot_6.5.2
> VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> SELinux: Registering netfilter hooks
> alg: No test for stdrng (krng)
> Key type asymmetric registered
> Asymmetric key parser 'x509' registered
> bounce: pool size: 64 pages
> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> io scheduler noop registered
> io scheduler deadline registered
> io scheduler cfq registered (default)
> pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
> pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
> pcie_pme 0000:00:01.0:pcie01: service driver pcie_pme loaded
> pcieport 0000:00:01.2: Signaling PME through PCIe PME interrupt
> pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
> pci 0000:02:00.1: Signaling PME through PCIe PME interrupt
> pcie_pme 0000:00:01.2:pcie01: service driver pcie_pme loaded
> pcieport 0000:00:1c.0: Signaling PME through PCIe PME interrupt
> pci 0000:03:00.0: Signaling PME through PCIe PME interrupt
> pci 0000:04:00.0: Signaling PME through PCIe PME interrupt
> pcie_pme 0000:00:1c.0:pcie01: service driver pcie_pme loaded
> pcieport 0000:00:1c.1: Signaling PME through PCIe PME interrupt
> pci 0000:05:00.0: Signaling PME through PCIe PME interrupt
> pcie_pme 0000:00:1c.1:pcie01: service driver pcie_pme loaded
> pcieport 0000:00:1c.2: Signaling PME through PCIe PME interrupt
> pci 0000:06:00.0: Signaling PME through PCIe PME interrupt
> pcie_pme 0000:00:1c.2:pcie01: service driver pcie_pme loaded
> pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> pciehp: PCI Express Hot Plug Controller Driver version: 0.4
> intel_idle: MWAIT substates: 0x42120
> intel_idle: v0.4 model 0x3C
> intel_idle: lapic_timer_reliable_states 0xffffffff
> input: Power Button as
> /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
> ACPI: Power Button [PWRB]
> input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
> ACPI: Power Button [PWRF]
> thermal LNXTHERM:00: registered as thermal_zone0
> ACPI: Thermal Zone [TZ00] (28 C)
> thermal LNXTHERM:01: registered as thermal_zone1
> ACPI: Thermal Zone [TZ01] (30 C)
> ERST: Error Record Serialization Table (ERST) support is initialized.
> pstore: Registered erst as persistent store backend
> GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC.
> Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
> 00:06: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
> Non-volatile memory driver v1.3
> Linux agpgart interface v0.103
> [drm] Initialized drm 1.1.0 20060810
> brd: module loaded
> loop: module loaded
> libphy: Fixed MDIO Bus: probed
> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> ehci-pci: EHCI PCI platform driver
> ehci-pci 0000:00:1a.0: EHCI Host Controller
> ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1
> ehci-pci 0000:00:1a.0: debug port 2
> ehci-pci 0000:00:1a.0: cache line size of 64 is not supported
> ehci-pci 0000:00:1a.0: irq 20, io mem 0xde612000
> ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00
> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb1: Product: EHCI Host Controller
> usb usb1: Manufacturer: Linux 3.19.0-1.el6.elrepo.x86_64 ehci_hcd
> usb usb1: SerialNumber: 0000:00:1a.0
> hub 1-0:1.0: USB hub found
> hub 1-0:1.0: 2 ports detected
> ehci-pci 0000:00:1d.0: EHCI Host Controller
> ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2
> ehci-pci 0000:00:1d.0: debug port 2
> ehci-pci 0000:00:1d.0: cache line size of 64 is not supported
> ehci-pci 0000:00:1d.0: irq 23, io mem 0xde611000
> ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00
> usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb2: Product: EHCI Host Controller
> usb usb2: Manufacturer: Linux 3.19.0-1.el6.elrepo.x86_64 ehci_hcd
> usb usb2: SerialNumber: 0000:00:1d.0
> hub 2-0:1.0: USB hub found
> hub 2-0:1.0: 2 ports detected
> ehci-platform: EHCI generic platform driver
> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> ohci-pci: OHCI PCI platform driver
> ohci-platform: OHCI generic platform driver
> uhci_hcd: USB Universal Host Controller Interface driver
> i8042: PNP: No PS/2 controller found. Probing ports directly.
> serio: i8042 KBD port at 0x60,0x64 irq 1
> serio: i8042 AUX port at 0x60,0x64 irq 12
> mousedev: PS/2 mouse device common for all mice
> rtc_cmos 00:02: RTC can wake from S4
> rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
> rtc_cmos 00:02: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
> Intel P-state driver initializing.
> ledtrig-cpu: registered to indicate activity on CPUs
> hidraw: raw HID events driver (C) Jiri Kosina
> usbcore: registered new interface driver usbhid
> usbhid: USB HID core driver
> TCP: cubic registered
> Initializing XFRM netlink socket
> NET: Registered protocol family 17
> registered taskstats version 1
> ima: No TPM chip found, activating TPM-bypass!
> rtc_cmos 00:02: setting system clock to 2015-02-11 00:48:40 UTC
> (1423615720)
> Freeing unused kernel memory: 1960K (ffffffff81d62000 - ffffffff81f4c000)
> Write protecting the kernel read-only data: 12288k
> Freeing unused kernel memory: 1528K (ffff880001682000 - ffff880001800000)
> Freeing unused kernel memory: 780K (ffff880001b3d000 - ffff880001c00000)
> dracut: dracut-004-356.el6
> device-mapper: uevent: version 1.0.3
> device-mapper: ioctl: 4.29.0-ioctl (2014-10-28) initialised:
> dm-devel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
> udev: starting version 147
> udevd (135): /proc/135/oom_adj is deprecated, please use
> /proc/135/oom_score_adj instead.
> [drm] AST 2300 detected
> [drm] Analog VGA only
> [drm] dram 1632000000 6 32 02000000
> [TTM] Zone kernel: Available graphics memory: 16486916 kiB
> [TTM] Zone dma32: Available graphics memory: 2097152 kiB
> [TTM] Initializing pool allocator
> [TTM] Initializing DMA pool allocator
> fbcon: astdrmfb (fb0) is primary device
> Console: switching to colour frame buffer device 128x48
> ast 0000:04:00.0: fb0: astdrmfb frame buffer device
> ast 0000:04:00.0: registered panic notifier
> [drm] Initialized ast 0.1.0 20120228 for 0000:04:00.0 on minor 0
> dracut: Starting plymouth daemon
> usb 1-1: new high-speed USB device number 2 using ehci-pci
> usb 2-1: new high-speed USB device number 2 using ehci-pci
> mpt2sas version 18.100.00.00 loaded
> mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (32973832 kB)
> mpt2sas0: MSI-X vectors supported: 8, no of cores: 8, max_msix_vectors: 8
> mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 30
> mpt2sas0-msix1: PCI-MSI-X enabled: IRQ 31
> mpt2sas0-msix2: PCI-MSI-X enabled: IRQ 32
> mpt2sas0-msix3: PCI-MSI-X enabled: IRQ 33
> mpt2sas0-msix4: PCI-MSI-X enabled: IRQ 34
> mpt2sas0-msix5: PCI-MSI-X enabled: IRQ 35
> mpt2sas0-msix6: PCI-MSI-X enabled: IRQ 36
> mpt2sas0-msix7: PCI-MSI-X enabled: IRQ 37
> mpt2sas0: iomem(0x00000000de240000), mapped(0xffffc90013980000),
> size(65536)
> mpt2sas0: ioport(0x000000000000e000), size(256)
> usb 1-1: New USB device found, idVendor=8087, idProduct=8008
> usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
> hub 1-1:1.0: USB hub found
> hub 1-1:1.0: 6 ports detected
> usb 2-1: New USB device found, idVendor=8087, idProduct=8000
> usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
> hub 2-1:1.0: USB hub found
> hub 2-1:1.0: 6 ports detected
> mpt2sas0: sending diag reset !!
> tsc: Refined TSC clocksource calibration: 3292.393 MHz
> mpt2sas0: diag reset: SUCCESS
> mpt2sas0: Allocated physical memory: size(16361 kB)
> mpt2sas0: Current Controller Queue Depth(7931), Max Controller Queue
> Depth(8192)
> mpt2sas0: Scatter Gather Elements per IO(128)
> mpt2sas0: LSISAS2308: FWVersion(15.00.00.00), ChipRevision(0x05),
> BiosVersion(07.29.00.00)
> mpt2sas0: Protocol=(Initiator), Capabilities=(Raid,TLR,EEDP,Snapshot
> Buffer,Diag Trace Buffer,Task Set Full,NCQ)
> scsi host0: Fusion MPT SAS Host
> mpt2sas0: sending port enable !!
> Switched to clocksource tsc
> mpt2sas0: host_add: handle(0x0001), sas_addr(0x5e8611f000009558), phys(8)
> mpt2sas0: port enable: SUCCESS
> scsi 0:0:0:0: Direct-Access ATA WDC WD40EFRX-68W 0A82 PQ: 0
> ANSI: 6
> scsi 0:0:0:0: SATA: handle(0x0009), sas_addr(0x4433221100000000),
> phy(0), device_name(0x50014ee2b56c689d)
> scsi 0:0:0:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(0)
> scsi 0:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:1:0: Direct-Access ATA WDC WD40EFRX-68W 0A80 PQ: 0
> ANSI: 6
> scsi 0:0:1:0: SATA: handle(0x000a), sas_addr(0x4433221107000000),
> phy(7), device_name(0x50014ee25fe9eb7c)
> scsi 0:0:1:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(7)
> scsi 0:0:1:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:2:0: Direct-Access ATA WDC WD40EFRX-68W 0A80 PQ: 0
> ANSI: 6
> scsi 0:0:2:0: SATA: handle(0x000b), sas_addr(0x4433221106000000),
> phy(6), device_name(0x50014ee25feacd73)
> scsi 0:0:2:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(6)
> scsi 0:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:3:0: Direct-Access ATA WDC WD40EFRX-68W 0A80 PQ: 0
> ANSI: 6
> scsi 0:0:3:0: SATA: handle(0x000c), sas_addr(0x4433221103000000),
> phy(3), device_name(0x50014ee25feaa1a3)
> scsi 0:0:3:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(3)
> scsi 0:0:3:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:4:0: Direct-Access ATA ST4000VN000-1H41 SC44 PQ: 0
> ANSI: 6
> scsi 0:0:4:0: SATA: handle(0x000d), sas_addr(0x4433221102000000),
> phy(2), device_name(0x5000c5007918aac1)
> scsi 0:0:4:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(2)
> scsi 0:0:4:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:5:0: Direct-Access ATA ST4000VN000-1H41 SC44 PQ: 0
> ANSI: 6
> scsi 0:0:5:0: SATA: handle(0x000e), sas_addr(0x4433221104000000),
> phy(4), device_name(0x5000c50079182a14)
> scsi 0:0:5:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(4)
> scsi 0:0:5:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:6:0: Direct-Access ATA ST4000VN000-1H41 SC44 PQ: 0
> ANSI: 6
> scsi 0:0:6:0: SATA: handle(0x000f), sas_addr(0x4433221105000000),
> phy(5), device_name(0x5000c5007919413d)
> scsi 0:0:6:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(5)
> scsi 0:0:6:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> scsi 0:0:7:0: Direct-Access ATA HGST HDN724040AL A5E0 PQ: 0
> ANSI: 6
> scsi 0:0:7:0: SATA: handle(0x0010), sas_addr(0x4433221101000000),
> phy(1), device_name(0x5000cca24cd8834e)
> scsi 0:0:7:0: SATA: enclosure_logical_id(0x5e8611f000009558), slot(1)
> scsi 0:0:7:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y),
> sw_preserve(y)
> ahci 0000:00:1f.2: version 3.0
> ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
> ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl
> SATA mode
> ahci 0000:00:1f.2: flags: 64bit ncq stag pm led clo pio slum part ems
> apst
> scsi host1: ahci
> scsi host2: ahci
> scsi host3: ahci
> scsi host4: ahci
> scsi host5: ahci
> scsi host6: ahci
> ata1: SATA max UDMA/133 abar m2048@0xde610000 port 0xde610100 irq 38
> ata2: SATA max UDMA/133 abar m2048@0xde610000 port 0xde610180 irq 38
> ata3: SATA max UDMA/133 abar m2048@0xde610000 port 0xde610200 irq 38
> ata4: SATA max UDMA/133 abar m2048@0xde610000 port 0xde610280 irq 38
> ata5: SATA max UDMA/133 abar m2048@0xde610000 port 0xde610300 irq 38
> ata6: SATA max UDMA/133 abar m2048@0xde610000 port 0xde610380 irq 38
> ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> ata1.00: ATA-9: ST4000VN000-1H4168, SC44, max UDMA/133
> ata1.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
> ata1.00: configured for UDMA/133
> scsi 1:0:0:0: Direct-Access ATA ST4000VN000-1H41 SC44 PQ: 0
> ANSI: 5
> ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
> ata2.00: ATA-9: WDC WD40EFRX-68WT0N0, 80.00A80, max UDMA/133
> ata2.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
> ata2.00: configured for UDMA/133
> scsi 2:0:0:0: Direct-Access ATA WDC WD40EFRX-68W 0A80 PQ: 0
> ANSI: 5
> ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
> ata3.00: ATA-8: HGST HDN724040ALE640, MJAOA5E0, max UDMA/133
> ata3.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
> ata3.00: configured for UDMA/133
> scsi 3:0:0:0: Direct-Access ATA HGST HDN724040AL A5E0 PQ: 0
> ANSI: 5
> ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> ata4.00: ATA-9: ST4000VN000-1H4168, SC44, max UDMA/133
> ata4.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
> ata4.00: configured for UDMA/133
> scsi 4:0:0:0: Direct-Access ATA ST4000VN000-1H41 SC44 PQ: 0
> ANSI: 5
> ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> ata5.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
> ata5.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
> ata5.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY)
> filtered out
> ata5.00: ATA-8: KINGSTON SHFS37A120G, 560ABBF0, max UDMA/133
> ata5.00: 234441648 sectors, multi 1: LBA48 NCQ (depth 31/32), AA
> ata5.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
> ata5.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
> ata5.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY)
> filtered out
> ata5.00: configured for UDMA/133
> scsi 5:0:0:0: Direct-Access ATA KINGSTON SHFS37A BBF0 PQ: 0
> ANSI: 5
> ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
> ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
> ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
> ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY)
> filtered out
> ata6.00: ATA-8: KINGSTON SV300S37A120G, 580ABBF0, max UDMA/133
> ata6.00: 234441648 sectors, multi 1: LBA48 NCQ (depth 31/32), AA
> ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
> ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
> ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY)
> filtered out
> ata6.00: configured for UDMA/133
> scsi 6:0:0:0: Direct-Access ATA KINGSTON SV300S3 BBF0 PQ: 0
> ANSI: 5
> sd 1:0:0:0: [sdi] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 2:0:0:0: [sdj] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 2:0:0:0: [sdj] 4096-byte physical blocks
> sd 2:0:0:0: [sdj] Write Protect is off
> sd 2:0:0:0: [sdj] Mode Sense: 00 3a 00 00
> sd 3:0:0:0: [sdk] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 3:0:0:0: [sdk] 4096-byte physical blocks
> sd 2:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 4:0:0:0: [sdl] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 4:0:0:0: [sdl] 4096-byte physical blocks
> sd 3:0:0:0: [sdk] Write Protect is off
> sd 3:0:0:0: [sdk] Mode Sense: 00 3a 00 00
> sd 4:0:0:0: [sdl] Write Protect is off
> sd 4:0:0:0: [sdl] Mode Sense: 00 3a 00 00
> sd 3:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 4:0:0:0: [sdl] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 5:0:0:0: [sdm] 234441648 512-byte logical blocks: (120 GB/111 GiB)
> sd 5:0:0:0: [sdm] Write Protect is off
> sd 5:0:0:0: [sdm] Mode Sense: 00 3a 00 00
> sd 6:0:0:0: [sdn] 234441648 512-byte logical blocks: (120 GB/111 GiB)
> sd 5:0:0:0: [sdm] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 6:0:0:0: [sdn] Write Protect is off
> sd 6:0:0:0: [sdn] Mode Sense: 00 3a 00 00
> sd 6:0:0:0: [sdn] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sd 0:0:0:0: [sda] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:0:0: [sda] 4096-byte physical blocks
> sd 0:0:1:0: [sdb] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:1:0: [sdb] 4096-byte physical blocks
> sd 0:0:2:0: [sdc] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:2:0: [sdc] 4096-byte physical blocks
> sd 0:0:5:0: [sdf] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:5:0: [sdf] 4096-byte physical blocks
> sd 0:0:3:0: [sdd] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:3:0: [sdd] 4096-byte physical blocks
> sd 0:0:6:0: [sdg] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:6:0: [sdg] 4096-byte physical blocks
> sd 0:0:4:0: [sde] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:4:0: [sde] 4096-byte physical blocks
> sdn: sdn1 sdn2
> sd 6:0:0:0: [sdn] Attached SCSI disk
> sd 0:0:7:0: [sdh] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
> sd 0:0:7:0: [sdh] 4096-byte physical blocks
> sdk: unknown partition table
> sd 3:0:0:0: [sdk] Attached SCSI disk
> sdm: sdm1 sdm2
> sd 5:0:0:0: [sdm] Attached SCSI disk
> sd 0:0:0:0: [sda] Write Protect is off
> sd 0:0:0:0: [sda] Mode Sense: 7f 00 10 08
> sd 0:0:1:0: [sdb] Write Protect is off
> sd 0:0:1:0: [sdb] Mode Sense: 7f 00 10 08
> sd 0:0:2:0: [sdc] Write Protect is off
> sd 0:0:2:0: [sdc] Mode Sense: 7f 00 10 08
> sd 0:0:3:0: [sdd] Write Protect is off
> sd 0:0:3:0: [sdd] Mode Sense: 7f 00 10 08
> sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sd 0:0:1:0: [sdb] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sd 0:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sd 0:0:2:0: [sdc] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sdl: unknown partition table
> sd 4:0:0:0: [sdl] Attached SCSI disk
> sdj: unknown partition table
> sd 2:0:0:0: [sdj] Attached SCSI disk
> sd 1:0:0:0: [sdi] 4096-byte physical blocks
> sd 1:0:0:0: [sdi] Write Protect is off
> sd 1:0:0:0: [sdi] Mode Sense: 00 3a 00 00
> sd 1:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't
> support DPO or FUA
> sdc: unknown partition table
> sdb: unknown partition table
> sdd: unknown partition table
> sda: unknown partition table
> sd 0:0:2:0: [sdc] Attached SCSI disk
> sdi: unknown partition table
> sd 0:0:1:0: [sdb] Attached SCSI disk
> sd 1:0:0:0: [sdi] Attached SCSI disk
> sd 0:0:0:0: [sda] Attached SCSI disk
> sd 0:0:3:0: [sdd] Attached SCSI disk
> sd 0:0:4:0: [sde] Write Protect is off
> sd 0:0:4:0: [sde] Mode Sense: 7f 00 10 08
> sd 0:0:6:0: [sdg] Write Protect is off
> sd 0:0:6:0: [sdg] Mode Sense: 7f 00 10 08
> sd 0:0:4:0: [sde] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sd 0:0:6:0: [sdg] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sd 0:0:5:0: [sdf] Write Protect is off
> sd 0:0:5:0: [sdf] Mode Sense: 7f 00 10 08
> sd 0:0:5:0: [sdf] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sde: unknown partition table
> random: nonblocking pool is initialized
> sdg: unknown partition table
> sdf: unknown partition table
> sd 0:0:7:0: [sdh] Write Protect is off
> sd 0:0:7:0: [sdh] Mode Sense: 7f 00 10 08
> sd 0:0:7:0: [sdh] Write cache: enabled, read cache: enabled, supports
> DPO and FUA
> sd 0:0:4:0: [sde] Attached SCSI disk
> sd 0:0:6:0: [sdg] Attached SCSI disk
> sd 0:0:5:0: [sdf] Attached SCSI disk
> sdh: unknown partition table
> sd 0:0:7:0: [sdh] Attached SCSI disk
> md: bind<sde>
> md: bind<sdf>
> md: bind<sdg>
> md: bind<sdn1>
> md: bind<sdm1>
> md: raid1 personality registered for level 1
> md/raid1:md0: active with 2 out of 2 mirrors
> created bitmap (1 pages) for device md0
> md0: bitmap initialized from disk: read 1 pages, set 0 of 16 bits
> md0: detected capacity change from 0 to 1073676288
> md0: unknown partition table
> md: bind<sdl>
> md: bind<sda>
> md: bind<sdh>
> md: bind<sdc>
> md: bind<sdb>
> md: bind<sdi>
> md: bind<sdj>
> md: bind<sdk>
> md: bind<sdd>
> dracut: Scanning devices sdm2 sdn2 for LVM logical volumes vg00/swap
> vg00/root
> dracut: Couldn't find device with uuid
> Cnh3S6-xbqX-v0o7-G9B1-0jQ7-YLFo-edps3D.
> dracut: inactive '/dev/vg00/root' [64.00 GiB] inherit
> dracut: inactive '/dev/vg00/swap' [32.00 GiB] inherit
> dracut: inactive '/dev/vg00/ARRAY' [36.39 TiB] inherit
> dracut: inactive '/dev/vg00/cache' [30.71 GiB] inherit
> dracut: Couldn't find device with uuid
> Cnh3S6-xbqX-v0o7-G9B1-0jQ7-YLFo-edps3D.
> async_tx: api initialized (async)
> xor: automatically using best checksumming function:
> avx : 33704.000 MB/sec
> raid6: sse2x1 11054 MB/s
> raid6: sse2x2 14019 MB/s
> raid6: sse2x4 16148 MB/s
> raid6: using algorithm sse2x4 (16148 MB/s)
> raid6: using ssse3x2 recovery algorithm
> md: raid6 personality registered for level 6
> md: raid5 personality registered for level 5
> md: raid4 personality registered for level 4
> md: raid10 personality registered for level 10
> device-mapper: raid: Loading target version 1.6.0
> md/raid1:mdX: active with 2 out of 2 mirrors
> created bitmap (64 pages) for device mdX
> mdX: bitmap initialized from disk: read 5 pages, set 31 of 131072 bits
> EXT4-fs (dm-4): mounted filesystem with ordered data mode. Opts: (null)
> dracut: Mounted root filesystem /dev/mapper/vg00-root
> SELinux: Disabled at runtime.
> SELinux: Unregistering netfilter hooks
> audit: type=1404 audit(1423615732.033:2): selinux=0 auid=4294967295
> ses=4294967295
> dracut:
> dracut: Switching root
> readahead: starting
> udev: starting version 147
> WARNING! power/level is deprecated; use power/control instead
> EDAC MC: Ver: 3.0.0
> EDAC MC0: Giving out device to module ie31200_edac controller IE31200:
> DEV 0000:00:00.0 (POLLED)
> sd 0:0:0:0: Attached scsi generic sg0 type 0
> sd 0:0:1:0: Attached scsi generic sg1 type 0
> sd 0:0:2:0: Attached scsi generic sg2 type 0
> sd 0:0:3:0: Attached scsi generic sg3 type 0
> sd 0:0:4:0: Attached scsi generic sg4 type 0
> sd 0:0:5:0: Attached scsi generic sg5 type 0
> sd 0:0:6:0: Attached scsi generic sg6 type 0
> sd 0:0:7:0: Attached scsi generic sg7 type 0
> sd 1:0:0:0: Attached scsi generic sg8 type 0
> sd 2:0:0:0: Attached scsi generic sg9 type 0
> sd 3:0:0:0: Attached scsi generic sg10 type 0
> sd 4:0:0:0: Attached scsi generic sg11 type 0
> sd 5:0:0:0: Attached scsi generic sg12 type 0
> sd 6:0:0:0: Attached scsi generic sg13 type 0
> pps_core: LinuxPPS API ver. 1 registered
> pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti
> <giometti-k2GhghHVRtY@public.gmane.org>
> PTP clock support registered
> bnx2x: Broadcom NetXtreme II 5771x/578xx 10/20-Gigabit Ethernet Driver
> bnx2x 1.710.51-0 (2014/02/10)
> bnx2x 0000:02:00.0: msix capability found
> bnx2x 0000:02:00.0: part number 394D4342-34383735-30345430-473030
> bnx2x 0000:02:00.1: msix capability found
> bnx2x 0000:02:00.1: part number 394D4342-34383735-30345430-473030
> xhci_hcd 0000:00:14.0: xHCI Host Controller
> xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3
> xhci_hcd 0000:00:14.0: cache line size of 64 is not supported
> usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
> usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb3: Product: xHCI Host Controller
> usb usb3: Manufacturer: Linux 3.19.0-1.el6.elrepo.x86_64 xhci-hcd
> usb usb3: SerialNumber: 0000:00:14.0
> hub 3-0:1.0: USB hub found
> hub 3-0:1.0: 12 ports detected
> xhci_hcd 0000:00:14.0: xHCI Host Controller
> xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4
> usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
> usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb4: Product: xHCI Host Controller
> usb usb4: Manufacturer: Linux 3.19.0-1.el6.elrepo.x86_64 xhci-hcd
> usb usb4: SerialNumber: 0000:00:14.0
> hub 4-0:1.0: USB hub found
> hub 4-0:1.0: 6 ports detected
> mei_me 0000:00:16.0: Device doesn't have valid ME Interface
> shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
> dca service started, version 1.12.1
> igb: Intel(R) Gigabit Ethernet Network Driver - version 5.2.15-k
> igb: Copyright (c) 2007-2014 Intel Corporation.
> igb 0000:05:00.0: added PHC on eth2
> igb 0000:05:00.0: Intel(R) Gigabit Ethernet Network Connection
> igb 0000:05:00.0: eth2: (PCIe:2.5Gb/s:Width x1) 40:16:7e:36:09:a6
> igb 0000:05:00.0: eth2: PBA No: 001300-000
> igb 0000:05:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
> igb 0000:06:00.0: added PHC on eth3
> igb 0000:06:00.0: Intel(R) Gigabit Ethernet Network Connection
> igb 0000:06:00.0: eth3: (PCIe:2.5Gb/s:Width x1) 40:16:7e:36:09:a7
> igb 0000:06:00.0: eth3: PBA No: 001300-000
> igb 0000:06:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
> ipmi message handler version 39.2
> IPMI System Interface driver.
> ipmi_si: probing via ACPI
> ipmi_si 00:08: [io 0x0ca2] regsize 1 spacing 1 irq 0
> ipmi_si: Adding ACPI-specified kcs state machine
> ipmi_si: probing via SMBIOS
> ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0
> ipmi_si: Adding SMBIOS-specified kcs state machine duplicate interface
> ipmi_si: probing via SPMI
> ipmi_si: SPMI: io 0xca2 regsize 1 spacing 1 irq 0
> ipmi_si: Adding SPMI-specified kcs state machine duplicate interface
> ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2,
> slave address 0x0, irq 0
> ipmi_si 00:08: Found new BMC (man_id: 0x001043, prod_id: 0x0d63,
> dev_id: 0x20)
> ipmi_si 00:08: IPMI kcs interface initialized
> ipmi device interface
> ACPI Warning: SystemIO range 0x000000000000f000-0x000000000000f01f
> conflicts with OpRegion 0x000000000000f000-0x000000000000f00f
> (\_SB_.PCI0.SBUS.SMBI) (20141107/utaddress-258)
> ACPI: If an ACPI driver is available for this device, you should use
> it instead of the native driver
> input: PC Speaker as /devices/platform/pcspkr/input/input5
> microcode: CPU0 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU1 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU2 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU3 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU4 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU5 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU6 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU7 sig=0x306c3, pf=0x2, revision=0x17
> microcode: Microcode Update Driver: v2.00
> <tigran-ppwZ4lME3+KI6QP4U9MhSdBc4/FLrbF6@public.gmane.org>, Peter Oruba
> iTCO_vendor_support: vendor-support=0
> iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
> iTCO_wdt: Found a Lynx Point TCO device (Version=2, TCOBASE=0x1860)
> iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
> microcode: CPU0 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU0 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU1 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU1 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU2 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU2 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU3 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU3 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU4 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU4 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU5 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU5 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU6 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU6 updated to revision 0x1a, date = 2014-05-23
> microcode: CPU7 sig=0x306c3, pf=0x2, revision=0x17
> microcode: CPU7 updated to revision 0x1a, date = 2014-05-23
> usb 3-3: new high-speed USB device number 2 using xhci_hcd
> usb 3-3: New USB device found, idVendor=046b, idProduct=ff01
> usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> usb 3-3: Product: Virtual Hub
> usb 3-3: Manufacturer: American Megatrends Inc.
> usb 3-3: SerialNumber: serial
> hub 3-3:1.0: USB hub found
> hub 3-3:1.0: 5 ports detected
> usb 3-3.1: new low-speed USB device number 3 using xhci_hcd
> usb 3-3.1: New USB device found, idVendor=046b, idProduct=ff10
> usb 3-3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
> usb 3-3.1: Product: Virtual Keyboard and Mouse
> usb 3-3.1: Manufacturer: American Megatrends Inc.
> input: American Megatrends Inc. Virtual Keyboard and Mouse as
> /devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3.1/3-3.1:1.0/0003:046B:FF10.0001/input/input6
> hid-generic 0003:046B:FF10.0001: input,hidraw0: USB HID v1.10 Keyboard
> [American Megatrends Inc. Virtual Keyboard and Mouse] on
> usb-0000:00:14.0-3.1/input0
> input: American Megatrends Inc. Virtual Keyboard and Mouse as
> /devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3.1/3-3.1:1.1/0003:046B:FF10.0002/input/input7
> hid-generic 0003:046B:FF10.0002: input,hidraw1: USB HID v1.10 Mouse
> [American Megatrends Inc. Virtual Keyboard and Mouse] on
> usb-0000:00:14.0-3.1/input1
> usb 3-3.2: new high-speed USB device number 4 using xhci_hcd
> usb 3-3.2: New USB device found, idVendor=046b, idProduct=ff20
> usb 3-3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> usb 3-3.2: Product: Virtual Cdrom Device
> usb 3-3.2: Manufacturer: American Megatrends Inc.
> usb 3-3.2: SerialNumber: AAAABBBBCCCC1
> usb-storage 3-3.2:1.0: USB Mass Storage device detected
> scsi host7: usb-storage 3-3.2:1.0
> usbcore: registered new interface driver usb-storage
> usbcore: registered new interface driver uas
> usb 3-3.3: new high-speed USB device number 5 using xhci_hcd
> usb 3-3.3: New USB device found, idVendor=046b, idProduct=ff40
> usb 3-3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> usb 3-3.3: Product: Virtual Floppy Device
> usb 3-3.3: Manufacturer: American Megatrends Inc.
> usb 3-3.3: SerialNumber: AAAABBBBCCCC2
> usb-storage 3-3.3:1.0: USB Mass Storage device detected
> usb-storage 3-3.3:1.0: Quirks match for vid 046b pid ff40: 200
> scsi host8: usb-storage 3-3.3:1.0
> usb 3-3.4: new high-speed USB device number 6 using xhci_hcd
> md: kicking non-fresh sdd from array!
> md: unbind<sdd>
> md: export_rdev(sdd)
> md: kicking non-fresh sdb from array!
> md: unbind<sdb>
> md: export_rdev(sdb)
> md: kicking non-fresh sdc from array!
> md: unbind<sdc>
> md: export_rdev(sdc)
> md: kicking non-fresh sdh from array!
> md: unbind<sdh>
> md: export_rdev(sdh)
> md: kicking non-fresh sda from array!
> md: unbind<sda>
> md: export_rdev(sda)
> md: kicking non-fresh sdg from array!
> md: unbind<sdg>
> md: export_rdev(sdg)
> md: kicking non-fresh sdf from array!
> md: unbind<sdf>
> md: export_rdev(sdf)
> md: kicking non-fresh sde from array!
> md: unbind<sde>
> md: export_rdev(sde)
> md/raid:md127: not clean -- starting background reconstruction
> md/raid:md127: device sdk operational as raid disk 10
> md/raid:md127: device sdj operational as raid disk 9
> md/raid:md127: device sdi operational as raid disk 8
> md/raid:md127: device sdl operational as raid disk 0
> md/raid:md127: allocated 0kB
> md/raid:md127: not enough operational devices (8/12 failed)
> RAID conf printout:
> --- level:6 rd:12 wd:4
> disk 0, o:1, dev:sdl
> disk 8, o:1, dev:sdi
> disk 9, o:1, dev:sdj
> disk 10, o:1, dev:sdk
> md/raid:md127: failed to run raid set.
> md: pers->run() failed ...
> usb 3-3.4: New USB device found, idVendor=046b, idProduct=ff31
> usb 3-3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> usb 3-3.4: Product: Virtual HardDisk Device
> usb 3-3.4: Manufacturer: American Megatrends Inc.
> usb 3-3.4: SerialNumber: AAAABBBBCCCC3
> usb-storage 3-3.4:1.0: USB Mass Storage device detected
> scsi host9: usb-storage 3-3.4:1.0
> EXT4-fs (dm-4): re-mounted. Opts: (null)
> SGI XFS with ACLs, security attributes, no debug enabled
> Adding 33554428k swap on /dev/mapper/vg00-swap. Priority:-1 extents:1
> across:33554428k SSFS
> NET: Registered protocol family 10
> scsi 7:0:0:0: CD-ROM AMI Virtual CDROM0 1.00 PQ: 0
> ANSI: 0 CCS
> scsi 7:0:0:0: Attached scsi generic sg14 type 5
> sr 7:0:0:0: [sr0] scsi-1 drive
> cdrom: Uniform CD-ROM driver Revision: 3.20
> sr 7:0:0:0: Attached scsi CD-ROM sr0
> scsi 8:0:0:0: Direct-Access AMI Virtual Floppy0 1.00 PQ: 0
> ANSI: 0 CCS
> sd 8:0:0:0: Attached scsi generic sg15 type 0
> sd 8:0:0:0: [sdo] Attached SCSI removable disk
> IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
> scsi 9:0:0:0: Direct-Access AMI Virtual HDISK0 1.00 PQ: 0
> ANSI: 0 CCS
> sd 9:0:0:0: Attached scsi generic sg16 type 0
> sd 9:0:0:0: [sdp] Attached SCSI removable disk
> IPMI Watchdog: Unable to register misc device
> IPMI Watchdog: set timeout error: -22
> IPMI Watchdog: driver initialized
> Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot.
> IPMI poweroff: Power cycle is enabled.
> IPMI poweroff: ATCA Detect mfg 0x1043 prod 0xD63
> IPMI poweroff: Found a chassis style poweroff function
> 8021q: 802.1Q VLAN Support v1.8
> 8021q: adding VLAN 0 to HW filter on device eth2
> igb 0000:05:00.0 eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex,
> Flow Control: RX/TX
> IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
> Process accounting resumed
> RPC: Registered named UNIX socket transport module.
> RPC: Registered udp transport module.
> RPC: Registered tcp transport module.
> RPC: Registered tcp NFSv4.1 backchannel transport module.
> Installing knfsd (copyright (C) 1996 okir-pn4DOG8n3UYbFoVRYvo4fw@public.gmane.org).
> svc: failed to register nfsdv2 RPC service (errno 111).
> svc: failed to register nfsaclv2 RPC service (errno 111).
> svc: failed to register nfsdv2 RPC service (errno 97).
> svc: failed to register nfsaclv2 RPC service (errno 97).
> md: md127 stopped.
> md: unbind<sdk>
> md: export_rdev(sdk)
> md: unbind<sdj>
> md: export_rdev(sdj)
> md: unbind<sdi>
> md: export_rdev(sdi)
> md: unbind<sdl>
> md: export_rdev(sdl)
> md: md127 stopped.
> md: bind<sdc>
> md: bind<sdb>
> md: bind<sdd>
> md: bind<sdh>
> md: bind<sde>
> md: bind<sdf>
> md: bind<sdg>
> md: bind<sdi>
> md: bind<sdj>
> md: bind<sdk>
> md: bind<sda>
> md: bind<sdl>
> md: md127 stopped.
> md: unbind<sdl>
> md: export_rdev(sdl)
> md: unbind<sda>
> md: export_rdev(sda)
> md: unbind<sdk>
> md: export_rdev(sdk)
> md: unbind<sdj>
> md: export_rdev(sdj)
> md: unbind<sdi>
> md: export_rdev(sdi)
> md: unbind<sdg>
> md: export_rdev(sdg)
> md: unbind<sdf>
> md: export_rdev(sdf)
> md: unbind<sde>
> md: export_rdev(sde)
> md: unbind<sdh>
> md: export_rdev(sdh)
> md: unbind<sdd>
> md: export_rdev(sdd)
> md: unbind<sdb>
> md: export_rdev(sdb)
> md: unbind<sdc>
> md: export_rdev(sdc)
> md: md127 stopped.
> md: bind<sdc>
> md: bind<sdb>
> md: bind<sdd>
> md: bind<sdh>
> md: bind<sde>
> md: bind<sdf>
> md: bind<sdg>
> md: bind<sdi>
> md: bind<sdj>
> md: bind<sdk>
> md: bind<sda>
> md: bind<sdl>
> md: md127 stopped.
> md: unbind<sdl>
> md: export_rdev(sdl)
> md: unbind<sda>
> md: export_rdev(sda)
> md: unbind<sdk>
> md: export_rdev(sdk)
> md: unbind<sdj>
> md: export_rdev(sdj)
> md: unbind<sdi>
> md: export_rdev(sdi)
> md: unbind<sdg>
> md: export_rdev(sdg)
> md: unbind<sdf>
> md: export_rdev(sdf)
> md: unbind<sde>
> md: export_rdev(sde)
> md: unbind<sdh>
> md: export_rdev(sdh)
> md: unbind<sdd>
> md: export_rdev(sdd)
> md: unbind<sdb>
> md: export_rdev(sdb)
> md: unbind<sdc>
> md: export_rdev(sdc)
--
-Adam Thompson
athompso-gKoiEJA+T1nk1uMJSBkQmQ@public.gmane.org
+1 (204) 291-7950 - cell
+1 (204) 489-6515 - fax
^ permalink raw reply
* Re: Wierd: Degrading while recovering raid5
From: Phil Turmel @ 2015-02-11 2:14 UTC (permalink / raw)
To: Kyle Logue, linux-raid
In-Reply-To: <CAP7a4USu+650JJh9yX-WUU6eyxOGowa5r72W8JABCJpLnO=gtg@mail.gmail.com>
Hi Kyle,
{ Convention on kernel.org lists is reply-to-all, trim replies, and
either bottom post or interleave }
On 02/10/2015 04:50 PM, Kyle Logue wrote:
> Phil:
>
> Thanks for your detailed response. That link does seem to describe my
> problem and I do understand that desktop grade drives are sub-optimal.
> It was many years ago when I first set up this array on my home
> theater pc. Until now I had no idea about the cron job - I'll make
> sure to implement that. I am preparing to move to 6 tb disks sometime
> soon and i'll definitely go enterprise this time.
>
> Regarding the drive timeout: I understand that I need to increase it
> from 30 seconds to something larger (2+ min) but am unaware how to do
> this. Is it a kernel variable? I'll keep googling but this seems like
> it's whats going to save me.
>
> tl;dr: How do I change the drive timeout?
Put something like this in /etc/rc.local or wherever your distro suggests:
for x in /sys/block/sd[a-f]/device/timeout ; do
echo 180 > $x
done
Where the [a-f] is adjusted to suit your needs, and only for non-raid
non-scterc drives.
Phil
^ permalink raw reply
* Re: ignoring %s as it reports %s as failed
From: Phil Turmel @ 2015-02-11 2:26 UTC (permalink / raw)
To: Valentijn, linux-raid
In-Reply-To: <54DA29C4.2010000@lentijn.sess.ink>
Hi Valentijn,
On 02/10/2015 10:54 AM, Valentijn wrote:
> Hello list,
>
> This is a repost - I sent it to linux-raid back in January, but you were
> all still having holidays - or hangovers or anything.
Lots of work travel for me for the past year or so. :-( I'm sorry we
couldn't help then.
> So the "Active device 32768" seems to be the culprit here, is that correct?
Yup.
> I managed to get it *working* again by recreating it with:
> ./mdadm --create --assume-clean --data-offset=136 -e 1.2 --level=5
> --bitmap=none --raid-devices=4 --chunk=64 /dev/md99 /dev/mapper/disk1p5
> /dev/mapper/disk2p5 /dev/mapper/disk3p5 missing
>
> ... but that feels a bit rude.
Indeed. Many people use --create to their eventual dismay. But --force
won't fix a misrecorded device role.
> So, is there a "proper" way to reassemble this array?
I would dig around in the superblock to fix the role. Not terribly
user-friendly though.
A carefully constructed --create --assume-clean would have been my next
recommended step, preferably with a bootable thumb drive with the latest
stable kernel and latest mdadm version.
Phil
^ permalink raw reply
* Re: dead RAID6 array on CentOS6.6 / kernel 3.19
From: Phil Turmel @ 2015-02-11 2:39 UTC (permalink / raw)
To: athompso, linux-raid; +Cc: Board, MUUG, elrepo
In-Reply-To: <54DAB614.70302@athompso.net>
Hi Adam,
On 02/10/2015 08:53 PM, Adam Thompson wrote:
> So, I was doing some work on the new server tonight, stopping/starting
> NFS caused a kernel panic, and I thought rebooting would be a good
> idea... Apparently not!
> Sure enough, no array.
>> [root@muug ~]# cat /proc/mdstat
>> Personalities : [raid1] [raid6] [raid5] [raid4] [raid10]
>> md0 : active raid1 sdm1[0] sdn1[1]
>> 1048512 blocks super 1.0 [2/2] [UU]
>> bitmap: 0/1 pages [0KB], 65536KB chunk
>>
>> md127 : inactive sdk[10] sdj[9] sdi[8] sdl[12]
>> 15627550048 blocks super 1.2
>>
>> unused devices: <none>
Not really no array -- this remnant is inactive. On older kernel /
mdadm combinations, you have to --stop the inactive array before you try
again. Please add --verbose to your --assemble --force
If that doesn't work, use a bootable thumb drive with current kernel and
mdadm and --assemble --force again.
Phil
^ permalink raw reply
* Re: dead RAID6 array on CentOS6.6 / kernel 3.19
From: Adam Thompson @ 2015-02-11 2:53 UTC (permalink / raw)
To: Phil Turmel, linux-raid; +Cc: Board, MUUG, elrepo
In-Reply-To: <54DAC0E2.2070303@turmel.org>
On 2015-02-10 08:39 PM, Phil Turmel wrote:
> Hi Adam,
>
> On 02/10/2015 08:53 PM, Adam Thompson wrote:
>> So, I was doing some work on the new server tonight, stopping/starting
>> NFS caused a kernel panic, and I thought rebooting would be a good
>> idea... Apparently not!
>> Sure enough, no array.
>>> [root@muug ~]# cat /proc/mdstat
>>> Personalities : [raid1] [raid6] [raid5] [raid4] [raid10]
>>> md0 : active raid1 sdm1[0] sdn1[1]
>>> 1048512 blocks super 1.0 [2/2] [UU]
>>> bitmap: 0/1 pages [0KB], 65536KB chunk
>>>
>>> md127 : inactive sdk[10] sdj[9] sdi[8] sdl[12]
>>> 15627550048 blocks super 1.2
>>>
>>> unused devices: <none>
> Not really no array -- this remnant is inactive. On older kernel /
> mdadm combinations, you have to --stop the inactive array before you try
> again. Please add --verbose to your --assemble --force
>
> If that doesn't work, use a bootable thumb drive with current kernel and
> mdadm and --assemble --force again.
>
> Phil
I do note that the array was stopped before I tried re-assembling it the
first time (second invocation overall). It stopped itself, as can be
seen from the output.
Only with the addition of --no-degraded does the array assemble and
remain assembled but un-started:
> [root@muug ~]# mdadm --verbose --assemble --force --no-degraded
> /dev/md127 /dev/sd[a-l]
> mdadm: looking for devices for /dev/md127
> mdadm: /dev/sda is identified as a member of /dev/md127, slot 11.
> mdadm: /dev/sdb is identified as a member of /dev/md127, slot 2.
> mdadm: /dev/sdc is identified as a member of /dev/md127, slot 1.
> mdadm: /dev/sdd is identified as a member of /dev/md127, slot 3.
> mdadm: /dev/sde is identified as a member of /dev/md127, slot 5.
> mdadm: /dev/sdf is identified as a member of /dev/md127, slot 6.
> mdadm: /dev/sdg is identified as a member of /dev/md127, slot 7.
> mdadm: /dev/sdh is identified as a member of /dev/md127, slot 4.
> mdadm: /dev/sdi is identified as a member of /dev/md127, slot 8.
> mdadm: /dev/sdj is identified as a member of /dev/md127, slot 9.
> mdadm: /dev/sdk is identified as a member of /dev/md127, slot 10.
> mdadm: /dev/sdl is identified as a member of /dev/md127, slot 0.
> mdadm: added /dev/sdc to /dev/md127 as 1
> mdadm: added /dev/sdb to /dev/md127 as 2
> mdadm: added /dev/sdd to /dev/md127 as 3
> mdadm: added /dev/sdh to /dev/md127 as 4
> mdadm: added /dev/sde to /dev/md127 as 5
> mdadm: added /dev/sdf to /dev/md127 as 6 (possibly out of date)
> mdadm: added /dev/sdg to /dev/md127 as 7 (possibly out of date)
> mdadm: added /dev/sdi to /dev/md127 as 8
> mdadm: added /dev/sdj to /dev/md127 as 9
> mdadm: added /dev/sdk to /dev/md127 as 10
> mdadm: added /dev/sda to /dev/md127 as 11 (possibly out of date)
> mdadm: added /dev/sdl to /dev/md127 as 0
> mdadm: /dev/md127 assembled from 9 drives (out of 12), but not started.
Which then gives:
> [root@muug ~]# mdadm --detail /dev/md127
> /dev/md127:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 12
> Persistence : Superblock is persistent
>
> State : inactive
>
> Name : muug.ca:ARRAY (local to host muug.ca)
> UUID : ca23c162:1ba5d5bd:e20e321a:54fc97e0
> Events : 26263
>
> Number Major Minor RaidDevice
>
> - 8 0 - /dev/sda
> - 8 16 - /dev/sdb
> - 8 32 - /dev/sdc
> - 8 48 - /dev/sdd
> - 8 64 - /dev/sde
> - 8 80 - /dev/sdf
> - 8 96 - /dev/sdg
> - 8 112 - /dev/sdh
> - 8 128 - /dev/sdi
> - 8 144 - /dev/sdj
> - 8 160 - /dev/sdk
> - 8 176 - /dev/sdl
and
> [root@muug ~]# cat /proc/mdstat
> Personalities : [raid1] [raid6] [raid5] [raid4] [raid10]
> md127 : inactive sdl[12](S) sda[13](S) sdk[10](S) sdj[9](S) sdi[8](S)
> sdg[7](S) sdf[6](S) sde[5](S) sdh[4](S) sdd[3](S) sdb[2](S) sdc[1](S)
> 46882650144 blocks super 1.2
>
> md0 : active raid1 sdm1[0] sdn1[1]
> 1048512 blocks super 1.0 [2/2] [UU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> unused devices: <none>
and...
> [root@muug ~]# mdadm --verbose --misc --run --readonly /dev/md127
> mdadm: /dev/md127 does not appear to be active.
Also, kernel 3.19, which I mentioned we're running, pretty much *is* my
definition of an up-to-date kernel... how much newer do you want me to
try, and where would you recommend I find such a thing in a bootable image?
Thanks,
-Adam
--
-Adam Thompson
athompso@athompso.net
+1 (204) 291-7950 - cell
+1 (204) 489-6515 - fax
^ permalink raw reply
* Re: dead RAID6 array on CentOS6.6 / kernel 3.19
From: Phil Turmel @ 2015-02-11 3:08 UTC (permalink / raw)
To: athompso, linux-raid, NeilBrown
In-Reply-To: <54DAC42F.3090600@athompso.net>
Hi Adam,
{ Dropped members-only lists from CC }
On 02/10/2015 09:53 PM, Adam Thompson wrote:
>> [root@muug ~]# mdadm --verbose --misc --run --readonly /dev/md127
>> mdadm: /dev/md127 does not appear to be active.
>
>
> Also, kernel 3.19, which I mentioned we're running, pretty much *is* my
> definition of an up-to-date kernel... how much newer do you want me to
> try, and where would you recommend I find such a thing in a bootable image?
You're right, 3.19 should be fine. I'm stumped. Looks like a bug.
Adding Neil ....
Phil
^ permalink raw reply
* Re: dead RAID6 array on CentOS6.6 / kernel 3.19
From: Adam Thompson @ 2015-02-11 3:26 UTC (permalink / raw)
To: Phil Turmel, linux-raid, NeilBrown
In-Reply-To: <54DAC7A4.40407@turmel.org>
On 2015-02-10 09:08 PM, Phil Turmel wrote:
> Hi Adam,
>
> { Dropped members-only lists from CC }
>
> On 02/10/2015 09:53 PM, Adam Thompson wrote:
>
>>> [root@muug ~]# mdadm --verbose --misc --run --readonly /dev/md127
>>> mdadm: /dev/md127 does not appear to be active.
>>
>> Also, kernel 3.19, which I mentioned we're running, pretty much *is* my
>> definition of an up-to-date kernel... how much newer do you want me to
>> try, and where would you recommend I find such a thing in a bootable image?
> You're right, 3.19 should be fine. I'm stumped. Looks like a bug.
> Adding Neil ....
>
> Phil
One thing I just remembered... this was originally a nine-drive array
that got expanded. The server has been rebooted many times since then,
however, without incident until tonight.
--
-Adam Thompson
athompso@athompso.net
+1 (204) 291-7950 - cell
+1 (204) 489-6515 - fax
^ permalink raw reply
* Re: [Board] dead RAID6 array on CentOS6.6 / kernel 3.19
From: Adam Thompson @ 2015-02-11 3:37 UTC (permalink / raw)
To: Trevor Cordes; +Cc: linux-raid
In-Reply-To: <b8upe28pe441l7y4sd2vv61c.1423623489558@email.android.com>
On 2015-02-10 08:58 PM, Trevor Cordes wrote:
> The big first question is was it lying when it said starting bg recon.
> Probably. How can it rebuild a 3+ drive failure.
>
> If so it should be possible to really force. As in wild wacky stuff.
> I've done that before.
>
> I can look at it more after the meeting our tomorrow morning.
>
> Was the reboot a clean or a sysreq? Did it sync first?
Manually initiated by reboot(8), which is a link to telinit(8), which
merely sends a signal to init(8), which actually did the reboot.
No idea if /dev/md127 correctly stopped or not, syslogd stops before
those messages get logged, and I was working remotely through SSH
instead of to the virtual console connection.
I would assume the sync(2) happened correctly; this is breakage way down
at the MD layer, not the FS layer.
You're right, RAID6 can't rebuild from a 3-drive failure. I've gone
back through the messages I sent, and it looks like there was no
hot-spare, so I suspect this is just the default code path, where it
tries to rebuild anything it can, then figures out it can't.
--
-Adam Thompson
athompso@athompso.net
+1 (204) 291-7950 - cell
+1 (204) 489-6515 - fax
^ permalink raw reply
* Re: [PATCH 00/24] Clustered MD RAID1
From: NeilBrown @ 2015-02-11 4:17 UTC (permalink / raw)
To: Goldwyn Rodrigues; +Cc: lzhong, linux-raid
In-Reply-To: <54DA392F.1050506@suse.de>
[-- Attachment #1: Type: text/plain, Size: 8437 bytes --]
On Tue, 10 Feb 2015 11:00:31 -0600 Goldwyn Rodrigues <rgoldwyn@suse.de> wrote:
> Hi Neil,
>
> >
> >
> > hi Goldwyn,
> > thanks for these - and sorry for the long delay. Lots of leave over
> > southern summer, and the lots of email etc to deal with.
> >
> > This patch set is very close and I am tempted to just apply it and then
> > fix things up with subsequent patches. In order to allow that, could you
> > please:
> > - rebase against current upstream
> > - fix the checkpatch.pl errors and warnings.
> > The "WARNING: line over 80 characters" are often a judgement call
> > so I'm not particularly worried about those. Most, if not all, of
> > the others should be followed just to have consistent layout.
>
> Done.
ERROR: code indent should use tabs where possible
when you use spaces, they show up in red for me!!
Ditto for
WARNING: please, no space before tabs
WARNING: quoted string split across lines
It really is best to fix those, even though it makes the line long.
When grepping to find out where a message comes from, it is very annoying
if the grep fails because the line was split.
WARNING: Missing a blank line after declarations
Worth fixing I think.
WARNING: printk() should include KERN_ facility level
Definitely should be fixed, maybe make it pr_warn()??
>
> >
> > Then I'll queue them up for 3.21, providing I don't find anything that would
> > hurt non-cluster usage ....
> > On that topic: why initialise rv to -EINVAL in "metadata_update sends
> > message...". That looks wrong.
>
> Yes, this is fixed.
>
> >
> > I noticed that a number of times a patch will revert something that a
> > previous patch added. It would be much nicer to fold these changes back into
> > the original patch. Often this is just extra blank lines, but occasionally
> > variable names are changed (md -> mddev). It should be given the final name
> > when introduced. Every chunk in every patch should be directly relevant to
> > that patch.
>
> I have cross-checked this and I did not find anything with respect to
> variable names. I did some cleanup with respect to the code though.
>
> There is one instance where I have used a variable: cluster_setup_done
> and then removed it. I think this is required to understand the patch
> and a smooth transition to subsequent patches. However, if you want me
> to aggressively remove that part, I should be able to do that.
No, "cluster_setup_done" makes sense. It is scaffolding that you later need
to remove.
I'm probably letting me OCD tendencies get carried away, but these some of
the things that I noticed:
"Introduce md_cluster_info" moves 'bast' to a new location in
dlm_lock_resource for no apparent reason.
Also 'leave()' has a parameter which is changed from 'md' to 'mddev',
as does 'join'.
"Add node recovery callbacks" adds a comment to the 'nodes' field of 'struct
mddev'. Why not add the comment when the field is added?
Oh, and it mis-spells "unmber".
In "Gather on-going resync information of other nodes" you have:
static struct md_cluster_operations cluster_ops = {
.join = join,
.leave = leave,
- .slot_number = slot_number
+ .slot_number = slot_number,
+ .resync_info_update = resync_info_update
};
It is really best to put a comma at the end of each entry, even the last.
Then the patch would have been:
static struct md_cluster_operations cluster_ops = {
.join = join,
.leave = leave,
.slot_number = slot_number,
+ .resync_info_update = resync_info_update,
};
which is much nicer to read. You finally get this right in
"Suspend writes in RAID1 if within range" :-)
>
> >
> > Some other issues, that could possibly be fixed up afterwards:
> >
> > - Is a clustername 64 bytes or 63 bytes? I would have thought 64,
> > but the use of strlcpy make is 63 plus a nul. Is that really what is
> > wanted?
>
> Yes, it is 64 bytes. I haven't fixed this as yet.
>
> >
> > - Based on https://lkml.org/lkml/2012/10/23/580 it might be good to add
> > "default n" to Kconfig, and possible add a WARN() if anyone tries to use
> > the code.
>
> Done. Added pr_warn while loading the module.
>
> >
> > - I'm a bit concerned about the behaviour on node failure.
> > When a node fails, two things must happen w.r.t the bits in that node's
> > bitmap.
> > 1/ The corresponding regions of the array need to be resynced. You do have
> > code to do this.
> > 2/ Other nodes must avoid read-balancing on those regions until the
> > resync has completed.
> >
> > You do have code for this second bit, but it looks wrong. It avoids
> > read-balancing if ->area_resyncing(). That isn't sufficient.
> > The "area_resyncing" is always (I assume) a relatively small region of
> > the array which will be completely resynced quite quickly. It must be
> > because writes are blocked to this area. However the region in which
> > we must disable re-balancing can be much larger. It covers *all* bits
> > that are set in any unsynced bitmap. So it isn't just the area that is
> > currently being synced, but all areas that will be synced.
>
> What are unsynced bitmaps? Are they bitmaps which are associated with an
> active node or dirty bitmaps with dead nodes? If it is the former, I
> agree this is not enough. If it is latter, all nodes maintain a linked
> list of all the nodes which are currently performing resync (probably
> because of multiple nodes died simultaneously). One node performs the
> recovery (aka bitmap resync) of exactly one "dead" node at a time.
> area_resyncing goes through all the nodes which are performing resync.
The later - bitmaps associated with a dead node.
Bitmaps associated with an active node contain transient information, and the
filesystem will ensure that it never reads from somewhere that someone else
might be writing (or if it does, it will know that the data cannot be
trusted).
I looked at the code again, and discovered that I had the problem backwards.
But there is still a problem.
when any node is resyncing, your code blocks writes for the entire span of the
array from where-ever the resync is up to, to the end of the device.
So a write to a location near the end of the device will hang until all
resyncs finish. This could be a much longer time than you would like writes
to hang for.
I think that the resyncing host should only report that it is resyncing a
relatively small range of the array, maybe 100Meg. Maybe 1G.
Then that would only block access to that small part of the array, which
should clear in just a few seconds at most.
This information on the range being synced is not enough to limit
read-balancing.
I imagined that *every* node would read the bitmap for a failed node, and
would use that information to limit read-balancing. There are some
complexities in this though.
So the current code isn't "wrong" exactly, but it think it could cause sever
delays in some (unusual) circumstances.
>
> >
> > - I think md_reload_sb() might be too simple. It probably should check that
> > nothing serious has changed. The "mddev->raid_disks = 0" look suspicious.
> > I'll have to think about this a bit more.
>
> Yes, I get that feeling as well. However, I am not sure how to perform
> an exact comparison to understand what has changed. Perhaps it needs a
> new flag?
Probably. I haven't thought much about it.
>
> >
> > That's all I can see for now. I'll have another look once I have it all in my tree.
> >
>
> I have put all the changes in my git:
> https://github.com/goldwynr/linux
> The branch cluster-md is against the latest upstream. I also performed a
> small sanity test to check everything is working properly.
>
> Let me know if you would want me to repost the entire patchset to the
> mailing list.
>
>
I don't think there is any need for that.
I won't pull it in just yet - to give you a chance to resolve the last of the
checkpatch problems.
Then I'll double check that there is no risk to non-cluster users and try to
get it into -next after 3.20-rc2 is out.
Thanks,
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: dead RAID6 array on CentOS6.6 / kernel 3.19
From: NeilBrown @ 2015-02-11 4:26 UTC (permalink / raw)
To: Phil Turmel; +Cc: athompso, linux-raid
In-Reply-To: <54DAC7A4.40407@turmel.org>
[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]
On Tue, 10 Feb 2015 22:08:20 -0500 Phil Turmel <philip@turmel.org> wrote:
> Hi Adam,
>
> { Dropped members-only lists from CC }
>
> On 02/10/2015 09:53 PM, Adam Thompson wrote:
>
> >> [root@muug ~]# mdadm --verbose --misc --run --readonly /dev/md127
> >> mdadm: /dev/md127 does not appear to be active.
Supporting this sort of usage seemed like a good idea long long ago, but
it doesn't turn out to be really useful, so I haven't maintained it at all.
> >
> >
> > Also, kernel 3.19, which I mentioned we're running, pretty much *is* my
> > definition of an up-to-date kernel... how much newer do you want me to
> > try, and where would you recommend I find such a thing in a bootable image?
>
> You're right, 3.19 should be fine. I'm stumped. Looks like a bug.
> Adding Neil ....
I think it is an mdadm bug. I don't see a mention of mdadm version number
(but I didn't look very hard).
If you are using 3.3, update to at least 3.3.1
(just
cd /tmp
git clone git://neil.brown.name/mdadm
cd mdadm
make
./mdadm --assemble --force /dev/md127 .....
)
NeilBrown
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply
* Re: [PATCH RESEND] Monitor: fix for regression with container devices
From: NeilBrown @ 2015-02-11 4:38 UTC (permalink / raw)
To: Artur Paszkiewicz; +Cc: linux-raid, pawel.baldysiak
In-Reply-To: <1423476830-15635-1-git-send-email-artur.paszkiewicz@intel.com>
[-- Attachment #1: Type: text/plain, Size: 2426 bytes --]
On Mon, 9 Feb 2015 11:13:50 +0100 Artur Paszkiewicz
<artur.paszkiewicz@intel.com> wrote:
> This patch fixes 2 problems introduced by commit 9a518d8: not closing a
> file descriptor and ignoring container devices. Array state is always
> "inactive" for containers, so we make sure that the device is not a
> container by reading also the "level" sysfs entry.
>
> Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
> Reviewed-by: Pawel Baldysiak <pawel.baldysiak@intel.com>
> ---
> Monitor.c | 14 ++++++++++----
> 1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/Monitor.c b/Monitor.c
> index 971d2ec..66d67ba 100644
> --- a/Monitor.c
> +++ b/Monitor.c
> @@ -483,11 +483,17 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
> strncmp(buf,"inact",5) == 0) {
> if (fd >= 0)
> close(fd);
> - if (!st->err)
> - alert("DeviceDisappeared", dev, NULL, ainfo);
> - st->err++;
> - return 0;
> + fd = sysfs_open(st->devnm, NULL, "level");
> + if (fd < 0 || read(fd, buf, 10) != 0) {
> + if (fd >= 0)
> + close(fd);
> + if (!st->err)
> + alert("DeviceDisappeared", dev, NULL, ainfo);
> + st->err++;
> + return 0;
> + }
> }
> + close(fd);
> }
> fd = open(dev, O_RDONLY);
> if (fd < 0) {
Thanks for the patch.
I don't think I agree with the logic of using 'level' though.
For the sort of arrays that I need to ignore here, 'level' will be empty.
It would make sense to test 'metadata' though. If that starts 'external:',
then we don't want to ignore the array.
Could you confirm that this works please?
Thanks,
NeilBrown
diff --git a/Monitor.c b/Monitor.c
index 971d2ecbea72..6e085cb24993 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -483,11 +483,18 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
strncmp(buf,"inact",5) == 0) {
if (fd >= 0)
close(fd);
- if (!st->err)
- alert("DeviceDisappeared", dev, NULL, ainfo);
- st->err++;
- return 0;
+ fd = sysfs_open(st->devnm, NULL, "metadata");
+ if (fd < 0 || read(fd, buf, 9) != 9 ||
+ strncmp(buf, "external:", 9) != 0) {
+ if (fd >= 0)
+ close(fd);
+ if (!st->err)
+ alert("DeviceDisappeared", dev, NULL, ainfo);
+ st->err++;
+ return 0;
+ }
}
+ close(fd);
}
fd = open(dev, O_RDONLY);
if (fd < 0) {
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox