linux-raid.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* raid10 recovery assistance requested
@ 2013-09-16  3:30 Dave Gomboc
  2013-09-19  3:20 ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-16  3:30 UTC (permalink / raw)
  To: linux-raid

Last night, I had booted using sysresccd-3.7.1 and been able to see
the contents of the raid10, with one problem (I believe /proc/mdstat
was showing a _ amongst the AAA).  However, I was able to mount the md
device, and lvm commands were finding data just fine.  I even mounted
a logical volume that normally mounts to /srv -- it had not been
checked for a long time so it automatically did an fsck.  (I was not
thrilled about this, but I wasn't positive I should interrupt it
either.)  The fsck appeared to succesfully complete: at least, no
error was reported amongst the 5 steps by the time I got out of bed
this morning.  Without intending to change anything related to the
raid setup, it appears that I somehow did so while using
sysresccd-3.7.1 when booting the machine again this morning, because I
am now unable to mount the raid10.

I have run mdadm --examine --scan 2>&1 | less (though I am typing this
by hand from another computer off of a written copy of data that was
on-screen earlier today).

No md superblock detected on /dev/sdl, /dev/sdk, /dev/sdj, /dev/sdi,
/dev/sda3, /dev/sda2, /dev/sda1, /dev/sda, /dev/sr0, /dev/loop0.  No
surprises there.

Specific fields that were different between the four devices, I list
after this paragraph.  For each of /dev/sdi1, /dev/sdj1, /dev/sdk1,
and /dev/sdl1, the following fields were identical: magic (a92b4efc),
version (1.2), feature map (0x0), array uuid (3c76...), name
(cheap:teramooch), creation time, raid level (raid10), raid devices
(4), avail dev size: 3907021954 (1863.01 GiB 2000.40 GB), array size:
7814041600 (3726.03 GiB 4000.79 GB), used dev size: 3907020800
(1863.01 GiB, 2000.39 GB), data offset (2048 sectors), super offset (8
sectors), state (clean), layout (near=2), chunk size(512K)

Two of the four drives have in addition to the above the following,
identical information:
device uuid: 6680...
update time: Sat Sep 14 06:14:13 2013
checksum: 6b7397f - correct [yes, only 7 hex digits]
events: 520
device role: Active device 0
array state: AAAA

The other two drives have identical alternative information:
device uuid: 45ff...
update time: Sun Sep 15 09:54:22 2013
checksum: 1cbfeaea - correct
events: 599
device role: Active device 3
array state: .AAA

I do also have the four drive serial numbers (from ls -l
/dev/disk/by-id), and have figured out which pair are "active device
0" versus "active device 3" (though I cannot currently distinguish
between the two drives that report identical data).  I had also been
seeing something like "inactive" with two (S) when catting
/proc/mdstat, but unfortunately I did not record the precise text that
was given.

Subsequent to performing the above, I have invoked mdadm --stop, used
smartctl (which appears to report that all of the drives are actually
working, though "old age" is listed in a few categories) and the
machine has since been powered down again.

I hope (and suspect) that some variation of --build (or perhaps even
--assemble) involving --assume-clean might restore access to my data,
but I figured that it would be prudent to ask here before I fscked
anything up further.  I have four larger (GPT) drives ready to migrate
all of the data to, so if I could make the raid10 available again,
then I am ready to migrate the data off without any further reboots.

Your help is appreciated,
Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-16  3:30 raid10 recovery assistance requested Dave Gomboc
@ 2013-09-19  3:20 ` Dave Gomboc
  2013-09-19  4:29   ` Stan Hoeppner
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-19  3:20 UTC (permalink / raw)
  To: linux-raid

[new text is below quoted material]

On 15 September 2013 20:30, Dave Gomboc <dave_gomboc@acm.org> wrote:
> Last night, I had booted using sysresccd-3.7.1 and been able to see
> the contents of the raid10, with one problem (I believe /proc/mdstat
> was showing a _ amongst the AAA).  However, I was able to mount the md
> device, and lvm commands were finding data just fine.  I even mounted
> a logical volume that normally mounts to /srv -- it had not been
> checked for a long time so it automatically did an fsck.  (I was not
> thrilled about this, but I wasn't positive I should interrupt it
> either.)  The fsck appeared to succesfully complete: at least, no
> error was reported amongst the 5 steps by the time I got out of bed
> this morning.  Without intending to change anything related to the
> raid setup, it appears that I somehow did so while using
> sysresccd-3.7.1 when booting the machine again this morning, because I
> am now unable to mount the raid10.
>
> I have run mdadm --examine --scan 2>&1 | less (though I am typing this
> by hand from another computer off of a written copy of data that was
> on-screen earlier today).
>
> No md superblock detected on /dev/sdl, /dev/sdk, /dev/sdj, /dev/sdi,
> /dev/sda3, /dev/sda2, /dev/sda1, /dev/sda, /dev/sr0, /dev/loop0.  No
> surprises there.
>
> Specific fields that were different between the four devices, I list
> after this paragraph.  For each of /dev/sdi1, /dev/sdj1, /dev/sdk1,
> and /dev/sdl1, the following fields were identical: magic (a92b4efc),
> version (1.2), feature map (0x0), array uuid (3c76...), name
> (cheap:teramooch), creation time, raid level (raid10), raid devices
> (4), avail dev size: 3907021954 (1863.01 GiB 2000.40 GB), array size:
> 7814041600 (3726.03 GiB 4000.79 GB), used dev size: 3907020800
> (1863.01 GiB, 2000.39 GB), data offset (2048 sectors), super offset (8
> sectors), state (clean), layout (near=2), chunk size(512K)
>
> Two of the four drives have in addition to the above the following,
> identical information:
> device uuid: 6680...
> update time: Sat Sep 14 06:14:13 2013
> checksum: 6b7397f - correct [yes, only 7 hex digits]
> events: 520
> device role: Active device 0
> array state: AAAA
>
> The other two drives have identical alternative information:
> device uuid: 45ff...
> update time: Sun Sep 15 09:54:22 2013
> checksum: 1cbfeaea - correct
> events: 599
> device role: Active device 3
> array state: .AAA
>
> I do also have the four drive serial numbers (from ls -l
> /dev/disk/by-id), and have figured out which pair are "active device
> 0" versus "active device 3" (though I cannot currently distinguish
> between the two drives that report identical data).  I had also been
> seeing something like "inactive" with two (S) when catting
> /proc/mdstat, but unfortunately I did not record the precise text that
> was given.
>
> Subsequent to performing the above, I have invoked mdadm --stop, used
> smartctl (which appears to report that all of the drives are actually
> working, though "old age" is listed in a few categories) and the
> machine has since been powered down again.
>
> I hope (and suspect) that some variation of --build (or perhaps even
> --assemble) involving --assume-clean might restore access to my data,
> but I figured that it would be prudent to ask here before I fscked
> anything up further.  I have four larger (GPT) drives ready to migrate
> all of the data to, so if I could make the raid10 available again,
> then I am ready to migrate the data off without any further reboots.
>
> Your help is appreciated,
> Dave

Having received no response to date, I am wondering if this perhaps
isn't the right forum for asking for advice regarding my problem.  If
it is not, could someone please suggest where I should be asking
instead?

Thank you,
Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-19  3:20 ` Dave Gomboc
@ 2013-09-19  4:29   ` Stan Hoeppner
  2013-09-20  5:29     ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Stan Hoeppner @ 2013-09-19  4:29 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 9/18/2013 10:20 PM, Dave Gomboc wrote:
...
> Having received no response to date, I am wondering if this perhaps
> isn't the right forum for asking for advice regarding my problem.  If
> it is not, could someone please suggest where I should be asking
> instead?

This is the right place.  I suspect your somewhat convoluted report had
folks scratching their heads, all hoping someone else would pick this
one up.  Could you condense it into the bare required facts?

Also, I didn't see a statement as to why you booted the recovery disk to
begin with.  That fact alone suggests you already had some problem.
What was it?

-- 
Stan


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-19  4:29   ` Stan Hoeppner
@ 2013-09-20  5:29     ` Dave Gomboc
  2013-09-22 17:15       ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-20  5:29 UTC (permalink / raw)
  To: stan; +Cc: linux-raid

Thanks for replying, Stan.

I am not actually sure what are the bare, required facts, so I had
just tried to include the information that I could obtain from mdadm.

At this point, I have been running GNU ddrescue (e.g., ddrescue
--cluster-size=65536 --no-split --force /dev/source /dev/destination
/.../log.txt) to transfer two of the complete drives to two of the
larger four drives that I have.  No errors have shown up whatsoever so
far, which jives with the SMART results.  In a couple of more days, I
should have all four copied over, then I could try risk writing to the
four larger drives.

I know that the set of 4 drives had been set aside for quite a while.
They were completely out of space.  I booted the recovery disk for a
couple of reasons.  One was that when I booted normally, they had
shown up with one marked as failed.  However, there seems to be no
actual error, so I can only guess that prior to having set them aside
some time ago, I had failed one on purpose in order to install a
larger drive and begin a migration, but got interrupted by life's
events.  The other reason was that I had (mistakenly) thought it would
be a good idea to use the recovery disk as an environment from which
to migrate the data to larger drives.  I was also adding hardware to
support port multiplication (don't have 8 sata ports onboard) and
thought it would be helpful to be running a relatively bare-bones
setup.

Dave


On 18 September 2013 21:29, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> On 9/18/2013 10:20 PM, Dave Gomboc wrote:
> ...
>> Having received no response to date, I am wondering if this perhaps
>> isn't the right forum for asking for advice regarding my problem.  If
>> it is not, could someone please suggest where I should be asking
>> instead?
>
> This is the right place.  I suspect your somewhat convoluted report had
> folks scratching their heads, all hoping someone else would pick this
> one up.  Could you condense it into the bare required facts?
>
> Also, I didn't see a statement as to why you booted the recovery disk to
> begin with.  That fact alone suggests you already had some problem.
> What was it?
>
> --
> Stan
>

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-20  5:29     ` Dave Gomboc
@ 2013-09-22 17:15       ` Dave Gomboc
  2013-09-22 21:52         ` Phil Turmel
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-22 17:15 UTC (permalink / raw)
  To: linux-raid; +Cc: stan

Okay,

I am nervous about losing 10+ years of personal data.  Your help would
be greatly appreciated!

I now have all four drives copied (with ddrescue) from the originals.
The new copies are connected to the system; the original drives are
not.  When attempting to boot from my normal boot drive, I see this
very early on in the boot process:

Booting...
errror: found two disks with the index 0 for RAID md/teramooch.
error: found two disks with the index 3 for RAID md/teramooch.

Later, once the dmesg material appears, I get to:

** WARNING: There appears to be one or more degraded RAID devices **

The system may have suffered a hardware fault, such as a disk drive
failure.  The root device may depend on the RAID devices being online.
One or more of the following RAID devices are degraded:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : inactive sdi1[3](S) sdg1[0](S)
      3907021954 blocks super 1.2

unused devices: <none>
You may attempt to start the system anyway, or stop now and attempt
manual recovery operations.  To do this automatically in the future,
add "bootdegraded=true" to the kernel boot options.

If you choose to start the degraded RAID, the system may boot
normally, but performance may be degraded, and a further hardware
fault could result in permanent data loss.

If you abort now, you will be provided with a recovery shell.

Do you wish to start the degraded RAID [y/N]: Timed out
Gave up waiting for root device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay=  (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules: ls /dev)
ALERT!  /dev/mapper/teramooch-root does not exist.  Dropping to a shell!

BusyBox v1.18.5 (Ubuntu 1:1.18.5-1ubuntu4.1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs) _

I then rebooted and allowed the system rescue cd to boot.  cat
/proc/mdstat gives

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md127 : inactive sda1[3](S) sdc1[0](S)
      3907021954 blocks super 1.2

unused devices: <none>

What should I do from here?

Thanks,
Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-22 17:15       ` Dave Gomboc
@ 2013-09-22 21:52         ` Phil Turmel
  2013-09-22 22:45           ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-22 21:52 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid, stan

On 09/22/2013 01:15 PM, Dave Gomboc wrote:
> Okay,
> 
> I am nervous about losing 10+ years of personal data.  Your help would
> be greatly appreciated!

I reviewed this thread and realized you haven't gotten anywhere.
Unfortunately, reassembly of your array is not likely to be automatic.
Raid10 in default form cannot tolerate adjacent failures, so the fact
that you lack indices 1 & 2 suggest your data is lost.

More, very detailed data will be needed.  Please start with the complete
output of "mdadm -E" for all of the affected partitions, "smartctl -x"
for all of the drives they're on, and if you have it, a copy of the LVM
backup file that describes that part of your system.

Phil

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-22 21:52         ` Phil Turmel
@ 2013-09-22 22:45           ` Dave Gomboc
  2013-09-22 23:04             ` Dave Gomboc
  2013-09-22 23:25             ` Phil Turmel
  0 siblings, 2 replies; 25+ messages in thread
From: Dave Gomboc @ 2013-09-22 22:45 UTC (permalink / raw)
  To: Phil Turmel, linux-raid

Yikes.  The data had been accessible on the evening of Sat. Sept 14.
I had (apparently falsely) hoped that there were two complete copies
of all the data, and that to restore access it would be enough to
assign the correct drives to position 1 and 2 based on what matched
against the contents of positions 0 and 3.

I have ran these commands on the four 4TB drives that are the copies.
I think the smartctl -x commands in particular you want to see run
against the original set of drives, so after sending this message, I
will take down the machine, swap out the four copied drives for the
four original drives, then run these commands again on the original
drives.

I'm not knowledgeable about the LVM backup file.  Would one typically
happen to be stored in /boot?  That was the only directory I had that
was not part of the raid10 system.  I had distinct logical volumes for
home, opt, usr, var, srv, root (/), and tmp.


Results from the copied drives:


root@sysresccd /root % mdadm -E /dev/sda1
/dev/sda1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 45ffddf1:080172d2:ff5db863:5f6d84e8

    Update Time : Sun Sep 15 09:54:22 2013
       Checksum : 1cbfeaea - correct
         Events : 599

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : .AAA ('A' == active, '.' == missing)


root@sysresccd /root % mdadm -E /dev/sdb1
/dev/sdb1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 66806913:402f8cfb:a74e11fd:5ca17fee

    Update Time : Sat Sep 14 06:14:13 2013
       Checksum : 6b7397f - correct
         Events : 520

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)


root@sysresccd /root % mdadm -E /dev/sdc1
/dev/sdc1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 66806913:402f8cfb:a74e11fd:5ca17fee

    Update Time : Sat Sep 14 06:14:13 2013
       Checksum : 6b7397f - correct
         Events : 520

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)


root@sysresccd /root % mdadm -E /dev/sdd1
/dev/sdd1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 45ffddf1:080172d2:ff5db863:5f6d84e8

    Update Time : Sun Sep 15 09:54:22 2013
       Checksum : 1cbfeaea - correct
         Events : 599

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : .AAA ('A' == active, '.' == missing)


smartctl -x results from the copy drives:

root@sysresccd /root % smartctl -x /dev/sda
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     Hitachi HDS724040ALE640
Serial Number:    PK1310PAG62T2J
LU WWN Device Id: 5 000cca 22bc2c4a6
Firmware Version: MJAOA3B0
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:28:46 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
was never started.
Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (   24) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
SCT capabilities:       (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0
  2 Throughput_Performance  P-S---   136   136   054    -    81
  3 Spin_Up_Time            POS---   133   133   024    -    541 (Average 624)
  4 Start_Stop_Count        -O--C-   100   100   000    -    25
  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0
  7 Seek_Error_Rate         PO-R--   100   100   067    -    0
  8 Seek_Time_Performance   P-S---   117   117   020    -    36
  9 Power_On_Hours          -O--C-   100   100   000    -    203
 10 Spin_Retry_Count        PO--C-   100   100   060    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    25
192 Power-Off_Retract_Count -O--CK   100   100   000    -    29
193 Load_Cycle_Count        -O--C-   100   100   000    -    29
194 Temperature_Celsius     -O----   181   181   000    -    33 (Min/Max 19/57)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    30
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
SMART Log at address 0x01 has    1 sectors [Summary SMART error log]
GP    Log at address 0x03 has    1 sectors [Ext. Comprehensive SMART error log]
GP    Log at address 0x04 has    7 sectors [Device Statistics log]
SMART Log at address 0x06 has    1 sectors [SMART self-test log]
GP    Log at address 0x07 has    1 sectors [Extended self-test log]
GP    Log at address 0x08 has    2 sectors [Power Conditions log]
SMART Log at address 0x09 has    1 sectors [Selective self-test log]
GP    Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP    Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP    Log at address 0x20 has    1 sectors [Streaming performance log]
GP    Log at address 0x21 has    1 sectors [Write stream error log]
GP    Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
Device Error Count: 30 (device log contains only the most recent 4 errors)
CR     = Command Register
FEATR  = Features Register
COUNT  = Count (was: Sector Count) Register
LBA_48 = Upper bytes of LBA High/Mid/Low Registers ]  ATA-8
LH     = LBA High (was: Cylinder High) Register    ]   LBA
LM     = LBA Mid (was: Cylinder Low) Register      ] Register
LL     = LBA Low (was: Sector Number) Register     ]
DV     = Device (was: Device/Head) Register
DC     = Device Control Register
ER     = Error register
ST     = Status register
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 30 [1] occurred at disk power-on lifetime: 60 hours (2 days + 12 hours)
  When the command that caused the error occurred, the device was
active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  84 -- 51 00 70 00 00 ca 27 03 10 0a 00  Error: ICRC, ABRT 112
sectors at LBA = 0xca270310 = 3391554320

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  35 00 00 00 80 00 00 ca 27 03 00 e0 00  1d+01:00:14.036  WRITE DMA EXT
  35 00 00 00 80 00 00 ca 27 02 80 e0 00  1d+01:00:14.035  WRITE DMA EXT
  35 00 00 00 80 00 00 ca 27 02 00 e0 00  1d+01:00:14.034  WRITE DMA EXT
  35 00 00 00 80 00 00 ca 27 01 80 e0 00  1d+01:00:14.032  WRITE DMA EXT
  35 00 00 00 80 00 00 ca 27 01 00 e0 00  1d+01:00:14.031  WRITE DMA EXT

Error 29 [0] occurred at disk power-on lifetime: 35 hours (1 days + 11 hours)
  When the command that caused the error occurred, the device was
active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  84 -- 51 00 01 00 00 00 00 03 37 00 00  Error: ICRC, ABRT 1 sectors
at LBA = 0x00000337 = 823

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  c8 00 00 00 08 00 03 2f 00 03 30 e0 08     00:15:42.040  READ DMA
  c8 00 00 00 08 00 03 27 00 03 28 e0 08     00:15:42.040  READ DMA
  c8 00 00 00 08 00 03 1f 00 03 20 e0 08     00:15:42.040  READ DMA
  c8 00 00 00 08 00 03 17 00 03 18 e0 08     00:15:42.040  READ DMA
  c8 00 00 00 08 00 03 0f 00 03 10 e0 08     00:15:42.040  READ DMA

Error 28 [3] occurred at disk power-on lifetime: 35 hours (1 days + 11 hours)
  When the command that caused the error occurred, the device was
active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  84 -- 51 00 01 00 00 00 00 00 87 00 00  Error: ICRC, ABRT 1 sectors
at LBA = 0x00000087 = 135

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  c8 00 00 00 08 00 00 17 00 00 80 e0 08     00:15:41.223  READ DMA
  c8 00 00 00 08 00 00 7f 00 00 10 e0 08     00:15:41.223  READ DMA
  c8 00 00 00 08 00 00 3f 00 00 78 e0 08     00:15:41.223  READ DMA
  c8 00 00 00 08 00 00 1f 00 00 38 e0 08     00:15:41.223  READ DMA
  c8 00 00 00 08 00 08 07 00 00 18 e0 08     00:15:41.219  READ DMA

Error 27 [2] occurred at disk power-on lifetime: 35 hours (1 days + 11 hours)
  When the command that caused the error occurred, the device was
active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  84 -- 51 00 01 00 01 d1 c0 be af 01 00  Error: ICRC, ABRT 1 sectors
at LBA = 0x1d1c0beaf = 7814037167

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  25 00 00 00 08 00 01 d1 c0 be a8 e0 08     00:15:40.351  READ DMA EXT
  c8 00 00 00 08 00 00 07 00 00 08 e0 08     00:15:40.351  READ DMA
  c8 00 00 00 08 c0 be a7 00 00 00 e0 08     00:15:40.326  READ DMA
  25 00 00 00 08 00 01 d1 c0 be a0 e0 08     00:15:40.325  READ DMA EXT
  25 00 00 00 08 00 01 d1 c0 be 00 e0 08     00:15:40.321  READ DMA EXT

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    33 Celsius
Power Cycle Min/Max Temperature:     26/34 Celsius
Lifetime    Min/Max Temperature:     19/57 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (52)

Index    Estimated Time   Temperature Celsius
  53    2013-09-22 13:21    43  ************************
 ...    ..( 18 skipped).    ..  ************************
  72    2013-09-22 13:40    43  ************************
  73    2013-09-22 13:41    42  ***********************
 ...    ..(  3 skipped).    ..  ***********************
  77    2013-09-22 13:45    42  ***********************
  78    2013-09-22 13:46    43  ************************
  79    2013-09-22 13:47    43  ************************
  80    2013-09-22 13:48    43  ************************
  81    2013-09-22 13:49    42  ***********************
 ...    ..(  5 skipped).    ..  ***********************
  87    2013-09-22 13:55    42  ***********************
  88    2013-09-22 13:56    43  ************************
  89    2013-09-22 13:57    42  ***********************
  90    2013-09-22 13:58    42  ***********************
  91    2013-09-22 13:59    42  ***********************
  92    2013-09-22 14:00    43  ************************
  93    2013-09-22 14:01    42  ***********************
 ...    ..(  8 skipped).    ..  ***********************
 102    2013-09-22 14:10    42  ***********************
 103    2013-09-22 14:11    43  ************************
 ...    ..(  4 skipped).    ..  ************************
 108    2013-09-22 14:16    43  ************************
 109    2013-09-22 14:17    44  *************************
 110    2013-09-22 14:18    43  ************************
 ...    ..(  3 skipped).    ..  ************************
 114    2013-09-22 14:22    43  ************************
 115    2013-09-22 14:23    44  *************************
 ...    ..(  5 skipped).    ..  *************************
 121    2013-09-22 14:29    44  *************************
 122    2013-09-22 14:30    43  ************************
 ...    ..( 28 skipped).    ..  ************************
  23    2013-09-22 14:59    43  ************************
  24    2013-09-22 15:00    42  ***********************
  25    2013-09-22 15:01    42  ***********************
  26    2013-09-22 15:02    43  ************************
  27    2013-09-22 15:03    42  ***********************
  28    2013-09-22 15:04    43  ************************
  29    2013-09-22 15:05    43  ************************
  30    2013-09-22 15:06    43  ************************
  31    2013-09-22 15:07    42  ***********************
  32    2013-09-22 15:08    42  ***********************
  33    2013-09-22 15:09    43  ************************
 ...    ..(  2 skipped).    ..  ************************
  36    2013-09-22 15:12    43  ************************
  37    2013-09-22 15:13    42  ***********************
  38    2013-09-22 15:14    43  ************************
  39    2013-09-22 15:15     ?  -
  40    2013-09-22 15:16    27  ********
  41    2013-09-22 15:17    27  ********
  42    2013-09-22 15:18    28  *********
  43    2013-09-22 15:19    29  **********
  44    2013-09-22 15:20    29  **********
  45    2013-09-22 15:21    30  ***********
  46    2013-09-22 15:22    31  ************
  47    2013-09-22 15:23    32  *************
  48    2013-09-22 15:24    32  *************
  49    2013-09-22 15:25    32  *************
  50    2013-09-22 15:26    33  **************
  51    2013-09-22 15:27    33  **************
  52    2013-09-22 15:28    33  **************

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0009  2            9  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            5  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS



root@sysresccd /root % smartctl -x /dev/sdb
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     Hitachi HDS724040ALE640
Serial Number:    PK1311PAG4W5TS
LU WWN Device Id: 5 000cca 22bc2378a
Firmware Version: MJAOA3B0
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:37:10 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
was never started.
Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (   24) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
SCT capabilities:       (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0
  2 Throughput_Performance  P-S---   137   137   054    -    77
  3 Spin_Up_Time            POS---   140   140   024    -    545 (Average 558)
  4 Start_Stop_Count        -O--C-   100   100   000    -    33
  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0
  7 Seek_Error_Rate         PO-R--   100   100   067    -    0
  8 Seek_Time_Performance   P-S---   117   117   020    -    36
  9 Power_On_Hours          -O--C-   100   100   000    -    204
 10 Spin_Retry_Count        PO--C-   100   100   060    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    33
192 Power-Off_Retract_Count -O--CK   100   100   000    -    40
193 Load_Cycle_Count        -O--C-   100   100   000    -    40
194 Temperature_Celsius     -O----   166   166   000    -    36 (Min/Max 19/58)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
SMART Log at address 0x01 has    1 sectors [Summary SMART error log]
GP    Log at address 0x03 has    1 sectors [Ext. Comprehensive SMART error log]
GP    Log at address 0x04 has    7 sectors [Device Statistics log]
SMART Log at address 0x06 has    1 sectors [SMART self-test log]
GP    Log at address 0x07 has    1 sectors [Extended self-test log]
GP    Log at address 0x08 has    2 sectors [Power Conditions log]
SMART Log at address 0x09 has    1 sectors [Selective self-test log]
GP    Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP    Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP    Log at address 0x20 has    1 sectors [Streaming performance log]
GP    Log at address 0x21 has    1 sectors [Write stream error log]
GP    Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    36 Celsius
Power Cycle Min/Max Temperature:     26/36 Celsius
Lifetime    Min/Max Temperature:     19/58 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (9)

Index    Estimated Time   Temperature Celsius
  10    2013-09-22 13:30    45  **************************
 ...    ..(105 skipped).    ..  **************************
 116    2013-09-22 15:16    45  **************************
 117    2013-09-22 15:17     ?  -
 118    2013-09-22 15:18    27  ********
 119    2013-09-22 15:19    27  ********
 120    2013-09-22 15:20    28  *********
 121    2013-09-22 15:21    28  *********
 122    2013-09-22 15:22    29  **********
 123    2013-09-22 15:23    30  ***********
 124    2013-09-22 15:24    30  ***********
 125    2013-09-22 15:25    31  ************
 126    2013-09-22 15:26    31  ************
 127    2013-09-22 15:27    32  *************
   0    2013-09-22 15:28    32  *************
   1    2013-09-22 15:29    33  **************
   2    2013-09-22 15:30    33  **************
   3    2013-09-22 15:31    34  ***************
   4    2013-09-22 15:32    34  ***************
   5    2013-09-22 15:33    35  ****************
   6    2013-09-22 15:34    35  ****************
   7    2013-09-22 15:35    35  ****************
   8    2013-09-22 15:36    36  *****************
   9    2013-09-22 15:37    36  *****************

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            1  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS


root@sysresccd /root % smartctl -x /dev/sdc
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     Hitachi HDS724040ALE640
Serial Number:    PK1310PAG62REJ
LU WWN Device Id: 5 000cca 22bc2c473
Firmware Version: MJAOA3B0
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:38:06 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
was never started.
Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (   24) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
SCT capabilities:       (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0
  2 Throughput_Performance  P-S---   133   133   054    -    91
  3 Spin_Up_Time            POS---   156   156   024    -    530 (Average 460)
  4 Start_Stop_Count        -O--C-   100   100   000    -    77
  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0
  7 Seek_Error_Rate         PO-R--   100   100   067    -    0
  8 Seek_Time_Performance   P-S---   117   117   020    -    36
  9 Power_On_Hours          -O--C-   100   100   000    -    151
 10 Spin_Retry_Count        PO--C-   100   100   060    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    44
192 Power-Off_Retract_Count -O--CK   100   100   000    -    80
193 Load_Cycle_Count        -O--C-   100   100   000    -    80
194 Temperature_Celsius     -O----   162   162   000    -    37 (Min/Max 19/56)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
SMART Log at address 0x01 has    1 sectors [Summary SMART error log]
GP    Log at address 0x03 has    1 sectors [Ext. Comprehensive SMART error log]
GP    Log at address 0x04 has    7 sectors [Device Statistics log]
SMART Log at address 0x06 has    1 sectors [SMART self-test log]
GP    Log at address 0x07 has    1 sectors [Extended self-test log]
GP    Log at address 0x08 has    2 sectors [Power Conditions log]
SMART Log at address 0x09 has    1 sectors [Selective self-test log]
GP    Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP    Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP    Log at address 0x20 has    1 sectors [Streaming performance log]
GP    Log at address 0x21 has    1 sectors [Write stream error log]
GP    Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    37 Celsius
Power Cycle Min/Max Temperature:     26/37 Celsius
Lifetime    Min/Max Temperature:     19/56 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (12)

Index    Estimated Time   Temperature Celsius
  13    2013-09-22 13:31    46  ***************************
  14    2013-09-22 13:32    45  **************************
  15    2013-09-22 13:33    45  **************************
  16    2013-09-22 13:34    46  ***************************
  17    2013-09-22 13:35    45  **************************
 ...    ..(  2 skipped).    ..  **************************
  20    2013-09-22 13:38    45  **************************
  21    2013-09-22 13:39    46  ***************************
  22    2013-09-22 13:40    46  ***************************
  23    2013-09-22 13:41    45  **************************
 ...    ..( 45 skipped).    ..  **************************
  69    2013-09-22 14:27    45  **************************
  70    2013-09-22 14:28    46  ***************************
  71    2013-09-22 14:29    45  **************************
 ...    ..( 46 skipped).    ..  **************************
 118    2013-09-22 15:16    45  **************************
 119    2013-09-22 15:17     ?  -
 120    2013-09-22 15:18    27  ********
 121    2013-09-22 15:19    28  *********
 122    2013-09-22 15:20    28  *********
 123    2013-09-22 15:21    29  **********
 124    2013-09-22 15:22    30  ***********
 125    2013-09-22 15:23    30  ***********
 126    2013-09-22 15:24    31  ************
 127    2013-09-22 15:25    31  ************
   0    2013-09-22 15:26    32  *************
   1    2013-09-22 15:27    32  *************
   2    2013-09-22 15:28    33  **************
   3    2013-09-22 15:29    33  **************
   4    2013-09-22 15:30    34  ***************
   5    2013-09-22 15:31    34  ***************
   6    2013-09-22 15:32    34  ***************
   7    2013-09-22 15:33    35  ****************
   8    2013-09-22 15:34    35  ****************
   9    2013-09-22 15:35    36  *****************
  10    2013-09-22 15:36    36  *****************
  11    2013-09-22 15:37    36  *****************
  12    2013-09-22 15:38    37  ******************

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            1  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS


root@sysresccd /root % smartctl -x /dev/sdd
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     Hitachi HDS724040ALE640
Serial Number:    PK1310PAG5ZY0J
LU WWN Device Id: 5 000cca 22bc2b9fc
Firmware Version: MJAOA3B0
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:39:16 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
was never started.
Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (   24) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
SCT capabilities:       (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0
  2 Throughput_Performance  P-S---   137   137   054    -    77
  3 Spin_Up_Time            POS---   129   129   024    -    631 (Average 562)
  4 Start_Stop_Count        -O--C-   100   100   000    -    21
  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0
  7 Seek_Error_Rate         PO-R--   100   100   067    -    0
  8 Seek_Time_Performance   P-S---   117   117   020    -    36
  9 Power_On_Hours          -O--C-   100   100   000    -    166
 10 Spin_Retry_Count        PO--C-   100   100   060    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    14
192 Power-Off_Retract_Count -O--CK   100   100   000    -    28
193 Load_Cycle_Count        -O--C-   100   100   000    -    28
194 Temperature_Celsius     -O----   171   171   000    -    35 (Min/Max 19/59)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
SMART Log at address 0x01 has    1 sectors [Summary SMART error log]
GP    Log at address 0x03 has    1 sectors [Ext. Comprehensive SMART error log]
GP    Log at address 0x04 has    7 sectors [Device Statistics log]
SMART Log at address 0x06 has    1 sectors [SMART self-test log]
GP    Log at address 0x07 has    1 sectors [Extended self-test log]
GP    Log at address 0x08 has    2 sectors [Power Conditions log]
SMART Log at address 0x09 has    1 sectors [Selective self-test log]
GP    Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP    Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP    Log at address 0x20 has    1 sectors [Streaming performance log]
GP    Log at address 0x21 has    1 sectors [Write stream error log]
GP    Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    36 Celsius
Power Cycle Min/Max Temperature:     26/36 Celsius
Lifetime    Min/Max Temperature:     19/59 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (15)

Index    Estimated Time   Temperature Celsius
  16    2013-09-22 13:32    42  ***********************
  17    2013-09-22 13:33    43  ************************
  18    2013-09-22 13:34    42  ***********************
 ...    ..( 19 skipped).    ..  ***********************
  38    2013-09-22 13:54    42  ***********************
  39    2013-09-22 13:55    43  ************************
  40    2013-09-22 13:56    42  ***********************
 ...    ..(  2 skipped).    ..  ***********************
  43    2013-09-22 13:59    42  ***********************
  44    2013-09-22 14:00    41  **********************
  45    2013-09-22 14:01    42  ***********************
  46    2013-09-22 14:02    41  **********************
 ...    ..( 73 skipped).    ..  **********************
 120    2013-09-22 15:16    41  **********************
 121    2013-09-22 15:17     ?  -
 122    2013-09-22 15:18    26  *******
 123    2013-09-22 15:19    27  ********
 124    2013-09-22 15:20    28  *********
 125    2013-09-22 15:21    29  **********
 126    2013-09-22 15:22    29  **********
 127    2013-09-22 15:23    30  ***********
   0    2013-09-22 15:24    30  ***********
   1    2013-09-22 15:25    31  ************
   2    2013-09-22 15:26    31  ************
   3    2013-09-22 15:27    31  ************
   4    2013-09-22 15:28    32  *************
   5    2013-09-22 15:29    32  *************
   6    2013-09-22 15:30    33  **************
   7    2013-09-22 15:31    33  **************
   8    2013-09-22 15:32    33  **************
   9    2013-09-22 15:33    34  ***************
  10    2013-09-22 15:34    34  ***************
  11    2013-09-22 15:35    34  ***************
  12    2013-09-22 15:36    35  ****************
 ...    ..(  2 skipped).    ..  ****************
  15    2013-09-22 15:39    35  ****************

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            1  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS


The next post will contain the data from the original 4 drives.

Dave


On 22 September 2013 21:52, Phil Turmel <philip@turmel.org> wrote:
> On 09/22/2013 01:15 PM, Dave Gomboc wrote:
>> Okay,
>>
>> I am nervous about losing 10+ years of personal data.  Your help would
>> be greatly appreciated!
>
> I reviewed this thread and realized you haven't gotten anywhere.
> Unfortunately, reassembly of your array is not likely to be automatic.
> Raid10 in default form cannot tolerate adjacent failures, so the fact
> that you lack indices 1 & 2 suggest your data is lost.
>
> More, very detailed data will be needed.  Please start with the complete
> output of "mdadm -E" for all of the affected partitions, "smartctl -x"
> for all of the drives they're on, and if you have it, a copy of the LVM
> backup file that describes that part of your system.
>
> Phil

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-22 22:45           ` Dave Gomboc
@ 2013-09-22 23:04             ` Dave Gomboc
  2013-09-22 23:25             ` Phil Turmel
  1 sibling, 0 replies; 25+ messages in thread
From: Dave Gomboc @ 2013-09-22 23:04 UTC (permalink / raw)
  To: Phil Turmel, linux-raid

Here is the information retrieved from the original 4 drives:


root@sysresccd /root % mdadm -E /dev/sda1
/dev/sda1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 45ffddf1:080172d2:ff5db863:5f6d84e8

    Update Time : Sun Sep 15 09:54:22 2013
       Checksum : 1cbfeaea - correct
         Events : 599

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : .AAA ('A' == active, '.' == missing)


root@sysresccd /root % mdadm -E /dev/sdb1
/dev/sdb1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 66806913:402f8cfb:a74e11fd:5ca17fee

    Update Time : Sat Sep 14 06:14:13 2013
       Checksum : 6b7397f - correct
         Events : 520

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)


root@sysresccd /root % mdadm -E /dev/sdc1
/dev/sdc1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 66806913:402f8cfb:a74e11fd:5ca17fee

    Update Time : Sat Sep 14 06:14:13 2013
       Checksum : 6b7397f - correct
         Events : 520

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)


root@sysresccd /root % mdadm -E /dev/sdd1
/dev/sdd1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 3c76af95:fa16ca54:330b7547:b9040b8a
           Name : cheap:teramooch
  Creation Time : Mon Jan  3 05:47:11 2011
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 3907021954 (1863.01 GiB 2000.40 GB)
     Array Size : 7814041600 (3726.03 GiB 4000.79 GB)
  Used Dev Size : 3907020800 (1863.01 GiB 2000.39 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 45ffddf1:080172d2:ff5db863:5f6d84e8

    Update Time : Sun Sep 15 09:54:22 2013
       Checksum : 1cbfeaea - correct
         Events : 599

         Layout : near=2
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : .AAA ('A' == active, '.' == missing)


root@sysresccd /root % smartctl -x /dev/sda
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda LP
Device Model:     ST32000542AS
Serial Number:    5XW1P41K
LU WWN Device Id: 5 000c50 02abd04fd
Firmware Version: CC34
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:57:05 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (  643) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
Conveyance self-test routine
recommended polling time: (   2) minutes.
SCT capabilities:       (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   117   099   006    -    167407251
  3 Spin_Up_Time            PO----   100   100   000    -    0
  4 Start_Stop_Count        -O--CK   100   100   020    -    95
  5 Reallocated_Sector_Ct   PO--CK   100   100   036    -    0
  7 Seek_Error_Rate         POSR--   075   060   030    -    37488616
  9 Power_On_Hours          -O--CK   086   086   000    -    12539
 10 Spin_Retry_Count        PO--C-   100   100   097    -    0
 12 Power_Cycle_Count       -O--CK   100   100   020    -    86
183 Runtime_Bad_Block       -O--CK   095   095   000    -    5
184 End-to-End_Error        -O--CK   100   100   099    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
188 Command_Timeout         -O--CK   100   100   000    -    196611
189 High_Fly_Writes         -O-RCK   100   100   000    -    0
190 Airflow_Temperature_Cel -O---K   072   051   045    -    28 (Min/Max 26/28)
194 Temperature_Celsius     -O---K   028   049   000    -    28 (0 16 0 0 0)
195 Hardware_ECC_Recovered  -O-RC-   048   014   000    -    167407251
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    4
240 Head_Flying_Hours       ------   100   253   000    -    195481141522647
241 Total_LBAs_Written      ------   100   253   000    -    243233184
242 Total_LBAs_Read         ------   100   253   000    -    4126437910
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
GP/S  Log at address 0x01 has    1 sectors [Summary SMART error log]
GP/S  Log at address 0x02 has    5 sectors [Comprehensive SMART error log]
GP/S  Log at address 0x03 has    5 sectors [Ext. Comprehensive SMART error log]
GP/S  Log at address 0x06 has    1 sectors [SMART self-test log]
GP/S  Log at address 0x07 has    1 sectors [Extended self-test log]
GP/S  Log at address 0x09 has    1 sectors [Selective self-test log]
GP/S  Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP/S  Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP/S  Log at address 0x21 has    1 sectors [Write stream error log]
GP/S  Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xa1 has   20 sectors [Device vendor specific log]
GP    Log at address 0xa2 has 2248 sectors [Device vendor specific log]
GP/S  Log at address 0xa8 has  129 sectors [Device vendor specific log]
GP/S  Log at address 0xa9 has    1 sectors [Device vendor specific log]
GP    Log at address 0xb0 has 2928 sectors [Device vendor specific log]
GP/S  Log at address 0xbd has  252 sectors [Device vendor specific log]
GP    Log at address 0xbe has 65535 sectors [Device vendor specific log]
GP    Log at address 0xbf has 65535 sectors [Device vendor specific log]
GP/S  Log at address 0xc0 has    1 sectors [Device vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       522 (0x020a)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    28 Celsius
Power Cycle Min/Max Temperature:     26/28 Celsius
Lifetime    Min/Max Temperature:     16/49 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         10 minutes
Temperature Logging Interval:        59 minutes
Min/Max recommended Temperature:     14/55 Celsius
Min/Max Temperature Limit:           10/60 Celsius
Temperature History Size (Index):    128 (21)

Index    Estimated Time   Temperature Celsius
  22    2013-09-17 10:35    29  **********
  23    2013-09-17 11:34     ?  -
  24    2013-09-17 12:33    29  **********
  25    2013-09-17 13:32     ?  -
  26    2013-09-17 14:31    29  **********
  27    2013-09-17 15:30     ?  -
  28    2013-09-17 16:29    29  **********
  29    2013-09-17 17:28     ?  -
  30    2013-09-17 18:27    29  **********
  31    2013-09-17 19:26     ?  -
  32    2013-09-17 20:25    29  **********
  33    2013-09-17 21:24     ?  -
  34    2013-09-17 22:23    30  ***********
  35    2013-09-17 23:22     ?  -
  36    2013-09-18 00:21    30  ***********
  37    2013-09-18 01:20     ?  -
  38    2013-09-18 02:19    30  ***********
  39    2013-09-18 03:18     ?  -
  40    2013-09-18 04:17    25  ******
  41    2013-09-18 05:16     ?  -
  42    2013-09-18 06:15    29  **********
  43    2013-09-18 07:14     ?  -
  44    2013-09-18 08:13    29  **********
  45    2013-09-18 09:12     ?  -
  46    2013-09-18 10:11    29  **********
  47    2013-09-18 11:10    29  **********
  48    2013-09-18 12:09    29  **********
  49    2013-09-18 13:08    30  ***********
  50    2013-09-18 14:07    29  **********
  51    2013-09-18 15:06    30  ***********
  52    2013-09-18 16:05    29  **********
  53    2013-09-18 17:04    30  ***********
  54    2013-09-18 18:03    30  ***********
  55    2013-09-18 19:02    29  **********
  56    2013-09-18 20:01    30  ***********
  57    2013-09-18 21:00    29  **********
  58    2013-09-18 21:59    30  ***********
  59    2013-09-18 22:58    29  **********
  60    2013-09-18 23:57    30  ***********
  61    2013-09-19 00:56    29  **********
  62    2013-09-19 01:55    30  ***********
  63    2013-09-19 02:54    29  **********
  64    2013-09-19 03:53    29  **********
  65    2013-09-19 04:52    30  ***********
  66    2013-09-19 05:51    30  ***********
  67    2013-09-19 06:50    30  ***********
  68    2013-09-19 07:49     ?  -
  69    2013-09-19 08:48    29  **********
  70    2013-09-19 09:47    29  **********
  71    2013-09-19 10:46    30  ***********
  72    2013-09-19 11:45    31  ************
  73    2013-09-19 12:44    29  **********
  74    2013-09-19 13:43    29  **********
  75    2013-09-19 14:42    28  *********
  76    2013-09-19 15:41     ?  -
  77    2013-09-19 16:40    35  ****************
  78    2013-09-19 17:39     ?  -
  79    2013-09-19 18:38    30  ***********
  80    2013-09-19 19:37     ?  -
  81    2013-09-19 20:36    29  **********
  82    2013-09-19 21:35     ?  -
  83    2013-09-19 22:34    26  *******
  84    2013-09-19 23:33     ?  -
  85    2013-09-20 00:32    25  ******
  86    2013-09-20 01:31     ?  -
  87    2013-09-20 02:30    23  ****
  88    2013-09-20 03:29    23  ****
  89    2013-09-20 04:28     ?  -
  90    2013-09-20 05:27    26  *******
  91    2013-09-20 06:26     ?  -
  92    2013-09-20 07:25    27  ********
  93    2013-09-20 08:24     ?  -
  94    2013-09-20 09:23    27  ********
  95    2013-09-20 10:22     ?  -
  96    2013-09-20 11:21    30  ***********
  97    2013-09-20 12:20    30  ***********
  98    2013-09-20 13:19    29  **********
  99    2013-09-20 14:18    28  *********
 100    2013-09-20 15:17    27  ********
 101    2013-09-20 16:16    27  ********
 102    2013-09-20 17:15    26  *******
 103    2013-09-20 18:14    26  *******
 104    2013-09-20 19:13    25  ******
 105    2013-09-20 20:12    25  ******
 106    2013-09-20 21:11    25  ******
 107    2013-09-20 22:10    24  *****
 108    2013-09-20 23:09    24  *****
 109    2013-09-21 00:08     ?  -
 110    2013-09-21 01:07    22  ***
 111    2013-09-21 02:06     ?  -
 112    2013-09-21 03:05    23  ****
 113    2013-09-21 04:04     ?  -
 114    2013-09-21 05:03    29  **********
 115    2013-09-21 06:02     ?  -
 116    2013-09-21 07:01    29  **********
 117    2013-09-21 08:00     ?  -
 118    2013-09-21 08:59    25  ******
 119    2013-09-21 09:58    25  ******
 120    2013-09-21 10:57    35  ****************
 121    2013-09-21 11:56     ?  -
 122    2013-09-21 12:55    31  ************
 123    2013-09-21 13:54     ?  -
 124    2013-09-21 14:53    26  *******
 125    2013-09-21 15:52    26  *******
 126    2013-09-21 16:51    28  *********
 127    2013-09-21 17:50     ?  -
   0    2013-09-21 18:49    36  *****************
   1    2013-09-21 19:48     ?  -
   2    2013-09-21 20:47    34  ***************
   3    2013-09-21 21:46     ?  -
   4    2013-09-21 22:45    40  *********************
   5    2013-09-21 23:44    40  *********************
   6    2013-09-22 00:43    41  **********************
   7    2013-09-22 01:42     ?  -
   8    2013-09-22 02:41    31  ************
   9    2013-09-22 03:40     ?  -
  10    2013-09-22 04:39    25  ******
  11    2013-09-22 05:38    25  ******
  12    2013-09-22 06:37    40  *********************
  13    2013-09-22 07:36    41  **********************
  14    2013-09-22 08:35    40  *********************
 ...    ..(  3 skipped).    ..  *********************
  18    2013-09-22 12:31    40  *********************
  19    2013-09-22 13:30     ?  -
  20    2013-09-22 14:29    26  *******
  21    2013-09-22 15:28    26  *******

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x000a  2            6  Device-to-host register FISes sent due to a COMRESET
0x0001  2            0  Command failed due to ICRC error
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS


root@sysresccd /root % smartctl -x /dev/sdb
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda LP
Device Model:     ST32000542AS
Serial Number:    5XW1P852
LU WWN Device Id: 5 000c50 02abf5ce5
Firmware Version: CC34
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:57:56 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (  633) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
Conveyance self-test routine
recommended polling time: (   2) minutes.
SCT capabilities:       (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   117   099   006    -    140233800
  3 Spin_Up_Time            PO----   100   100   000    -    0
  4 Start_Stop_Count        -O--CK   099   099   020    -    1385
  5 Reallocated_Sector_Ct   PO--CK   100   100   036    -    0
  7 Seek_Error_Rate         POSR--   066   051   030    -    30097052991
  9 Power_On_Hours          -O--CK   086   086   000    -    12590
 10 Spin_Retry_Count        PO--C-   100   100   097    -    0
 12 Power_Cycle_Count       -O--CK   100   100   020    -    85
183 Runtime_Bad_Block       -O--CK   085   085   000    -    15
184 End-to-End_Error        -O--CK   100   100   099    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
188 Command_Timeout         -O--CK   100   100   000    -    4295032833
189 High_Fly_Writes         -O-RCK   100   100   000    -    0
190 Airflow_Temperature_Cel -O---K   075   051   045    -    25 (Min/Max 24/25)
194 Temperature_Celsius     -O---K   025   049   000    -    25 (0 14 0 0 0)
195 Hardware_ECC_Recovered  -O-RC-   048   016   000    -    140233800
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
240 Head_Flying_Hours       ------   100   253   000    -    85594403254559
241 Total_LBAs_Written      ------   100   253   000    -    686988228
242 Total_LBAs_Read         ------   100   253   000    -    3116625490
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
GP/S  Log at address 0x01 has    1 sectors [Summary SMART error log]
GP/S  Log at address 0x02 has    5 sectors [Comprehensive SMART error log]
GP/S  Log at address 0x03 has    5 sectors [Ext. Comprehensive SMART error log]
GP/S  Log at address 0x06 has    1 sectors [SMART self-test log]
GP/S  Log at address 0x07 has    1 sectors [Extended self-test log]
GP/S  Log at address 0x09 has    1 sectors [Selective self-test log]
GP/S  Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP/S  Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP/S  Log at address 0x21 has    1 sectors [Write stream error log]
GP/S  Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xa1 has   20 sectors [Device vendor specific log]
GP    Log at address 0xa2 has 2248 sectors [Device vendor specific log]
GP/S  Log at address 0xa8 has  129 sectors [Device vendor specific log]
GP/S  Log at address 0xa9 has    1 sectors [Device vendor specific log]
GP    Log at address 0xb0 has 2928 sectors [Device vendor specific log]
GP/S  Log at address 0xbd has  252 sectors [Device vendor specific log]
GP    Log at address 0xbe has 65535 sectors [Device vendor specific log]
GP    Log at address 0xbf has 65535 sectors [Device vendor specific log]
GP/S  Log at address 0xc0 has    1 sectors [Device vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       522 (0x020a)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    25 Celsius
Power Cycle Min/Max Temperature:     24/25 Celsius
Lifetime    Min/Max Temperature:     14/49 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         10 minutes
Temperature Logging Interval:        59 minutes
Min/Max recommended Temperature:     14/55 Celsius
Min/Max Temperature Limit:           10/60 Celsius
Temperature History Size (Index):    128 (68)

Index    Estimated Time   Temperature Celsius
  69    2013-09-17 10:35     ?  -
  70    2013-09-17 11:34    29  **********
  71    2013-09-17 12:33     ?  -
  72    2013-09-17 13:32    28  *********
  73    2013-09-17 14:31     ?  -
  74    2013-09-17 15:30    25  ******
  75    2013-09-17 16:29     ?  -
  76    2013-09-17 17:28    24  *****
  77    2013-09-17 18:27     ?  -
  78    2013-09-17 19:26    22  ***
  79    2013-09-17 20:25     ?  -
  80    2013-09-17 21:24    25  ******
  81    2013-09-17 22:23     ?  -
  82    2013-09-17 23:22    26  *******
  83    2013-09-18 00:21     ?  -
  84    2013-09-18 01:20    26  *******
  85    2013-09-18 02:19     ?  -
  86    2013-09-18 03:18    28  *********
  87    2013-09-18 04:17    28  *********
  88    2013-09-18 05:16    27  ********
  89    2013-09-18 06:15    27  ********
  90    2013-09-18 07:14    26  *******
  91    2013-09-18 08:13    25  ******
  92    2013-09-18 09:12    24  *****
  93    2013-09-18 10:11    24  *****
  94    2013-09-18 11:10    24  *****
  95    2013-09-18 12:09    23  ****
  96    2013-09-18 13:08    23  ****
  97    2013-09-18 14:07    22  ***
  98    2013-09-18 15:06    22  ***
  99    2013-09-18 16:05     ?  -
 100    2013-09-18 17:04    20  *
 101    2013-09-18 18:03     ?  -
 102    2013-09-18 19:02    21  **
 103    2013-09-18 20:01     ?  -
 104    2013-09-18 21:00    28  *********
 105    2013-09-18 21:59     ?  -
 106    2013-09-18 22:58    28  *********
 107    2013-09-18 23:57     ?  -
 108    2013-09-19 00:56    24  *****
 109    2013-09-19 01:55    24  *****
 110    2013-09-19 02:54    39  ********************
 111    2013-09-19 03:53    43  ************************
 112    2013-09-19 04:52    44  *************************
 113    2013-09-19 05:51     ?  -
 114    2013-09-19 06:50    31  ************
 115    2013-09-19 07:49     ?  -
 116    2013-09-19 08:48    35  ****************
 117    2013-09-19 09:47     ?  -
 118    2013-09-19 10:46    32  *************
 119    2013-09-19 11:45     ?  -
 120    2013-09-19 12:44    33  **************
 121    2013-09-19 13:43    33  **************
 122    2013-09-19 14:42    33  **************
 123    2013-09-19 15:41     ?  -
 124    2013-09-19 16:40    27  ********
 125    2013-09-19 17:39    27  ********
 126    2013-09-19 18:38    35  ****************
 127    2013-09-19 19:37    36  *****************
   0    2013-09-19 20:36    35  ****************
   1    2013-09-19 21:35    35  ****************
   2    2013-09-19 22:34    35  ****************
   3    2013-09-19 23:33    33  **************
 ...    ..(  7 skipped).    ..  **************
  11    2013-09-20 07:25    33  **************
  12    2013-09-20 08:24    34  ***************
  13    2013-09-20 09:23    35  ****************
  14    2013-09-20 10:22    35  ****************
  15    2013-09-20 11:21    36  *****************
 ...    ..(  2 skipped).    ..  *****************
  18    2013-09-20 14:18    36  *****************
  19    2013-09-20 15:17    35  ****************
  20    2013-09-20 16:16    36  *****************
  21    2013-09-20 17:15    35  ****************
 ...    ..(  2 skipped).    ..  ****************
  24    2013-09-20 20:12    35  ****************
  25    2013-09-20 21:11    36  *****************
  26    2013-09-20 22:10    35  ****************
 ...    ..(  3 skipped).    ..  ****************
  30    2013-09-21 02:06    35  ****************
  31    2013-09-21 03:05    33  **************
  32    2013-09-21 04:04    32  *************
  33    2013-09-21 05:03    32  *************
  34    2013-09-21 06:02    33  **************
 ...    ..(  3 skipped).    ..  **************
  38    2013-09-21 09:58    33  **************
  39    2013-09-21 10:57    34  ***************
  40    2013-09-21 11:56    35  ****************
  41    2013-09-21 12:55    34  ***************
 ...    ..(  3 skipped).    ..  ***************
  45    2013-09-21 16:51    34  ***************
  46    2013-09-21 17:50    33  **************
  47    2013-09-21 18:49    33  **************
  48    2013-09-21 19:48    34  ***************
  49    2013-09-21 20:47    33  **************
  50    2013-09-21 21:46    33  **************
  51    2013-09-21 22:45    32  *************
  52    2013-09-21 23:44    33  **************
  53    2013-09-22 00:43    33  **************
  54    2013-09-22 01:42    32  *************
  55    2013-09-22 02:41    32  *************
  56    2013-09-22 03:40    34  ***************
  57    2013-09-22 04:39    34  ***************
  58    2013-09-22 05:38    33  **************
  59    2013-09-22 06:37    33  **************
  60    2013-09-22 07:36    32  *************
  61    2013-09-22 08:35    33  **************
  62    2013-09-22 09:34    33  **************
  63    2013-09-22 10:33    32  *************
  64    2013-09-22 11:32    35  ****************
  65    2013-09-22 12:31    35  ****************
  66    2013-09-22 13:30     ?  -
  67    2013-09-22 14:29    24  *****
  68    2013-09-22 15:28    24  *****

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x000a  2            2  Device-to-host register FISes sent due to a COMRESET
0x0001  2            0  Command failed due to ICRC error
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS


root@sysresccd /root % smartctl -x /dev/sdc
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda LP
Device Model:     ST32000542AS
Serial Number:    6XW1T14N
LU WWN Device Id: 5 000c50 02b5bf6b9
Firmware Version: CC34
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 15:59:14 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (  643) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
Conveyance self-test routine
recommended polling time: (   2) minutes.
SCT capabilities:       (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   114   099   006    -    65310841
  3 Spin_Up_Time            PO----   100   100   000    -    0
  4 Start_Stop_Count        -O--CK   100   100   020    -    107
  5 Reallocated_Sector_Ct   PO--CK   098   098   036    -    107
  7 Seek_Error_Rate         POSR--   065   052   030    -    25791702551
  9 Power_On_Hours          -O--CK   086   086   000    -    12575
 10 Spin_Retry_Count        PO--C-   100   100   097    -    0
 12 Power_Cycle_Count       -O--CK   100   100   020    -    89
183 Runtime_Bad_Block       -O--CK   100   100   000    -    0
184 End-to-End_Error        -O--CK   100   100   099    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
188 Command_Timeout         -O--CK   100   099   000    -    4295032833
189 High_Fly_Writes         -O-RCK   100   100   000    -    0
190 Airflow_Temperature_Cel -O---K   075   052   045    -    25 (Min/Max 24/25)
194 Temperature_Celsius     -O---K   025   048   000    -    25 (0 14 0 0 0)
195 Hardware_ECC_Recovered  -O-RC-   048   019   000    -    65310841
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
240 Head_Flying_Hours       ------   100   253   000    -    4707284168973
241 Total_LBAs_Written      ------   100   253   000    -    3046759665
242 Total_LBAs_Read         ------   100   253   000    -    3165076617
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
GP/S  Log at address 0x01 has    1 sectors [Summary SMART error log]
GP/S  Log at address 0x02 has    5 sectors [Comprehensive SMART error log]
GP/S  Log at address 0x03 has    5 sectors [Ext. Comprehensive SMART error log]
GP/S  Log at address 0x06 has    1 sectors [SMART self-test log]
GP/S  Log at address 0x07 has    1 sectors [Extended self-test log]
GP/S  Log at address 0x09 has    1 sectors [Selective self-test log]
GP/S  Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP/S  Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP/S  Log at address 0x21 has    1 sectors [Write stream error log]
GP/S  Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xa1 has   20 sectors [Device vendor specific log]
GP    Log at address 0xa2 has 2248 sectors [Device vendor specific log]
GP/S  Log at address 0xa8 has  129 sectors [Device vendor specific log]
GP/S  Log at address 0xa9 has    1 sectors [Device vendor specific log]
GP    Log at address 0xb0 has 2928 sectors [Device vendor specific log]
GP/S  Log at address 0xbd has  252 sectors [Device vendor specific log]
GP    Log at address 0xbe has 65535 sectors [Device vendor specific log]
GP    Log at address 0xbf has 65535 sectors [Device vendor specific log]
GP/S  Log at address 0xc0 has    1 sectors [Device vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       522 (0x020a)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    25 Celsius
Power Cycle Min/Max Temperature:     24/25 Celsius
Lifetime    Min/Max Temperature:     14/48 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         10 minutes
Temperature Logging Interval:        59 minutes
Min/Max recommended Temperature:     14/55 Celsius
Min/Max Temperature Limit:           10/60 Celsius
Temperature History Size (Index):    128 (55)

Index    Estimated Time   Temperature Celsius
  56    2013-09-17 10:35    28  *********
  57    2013-09-17 11:34    27  ********
  58    2013-09-17 12:33    28  *********
  59    2013-09-17 13:32    28  *********
  60    2013-09-17 14:31    27  ********
  61    2013-09-17 15:30    28  *********
  62    2013-09-17 16:29    27  ********
  63    2013-09-17 17:28    28  *********
  64    2013-09-17 18:27    27  ********
  65    2013-09-17 19:26    29  **********
  66    2013-09-17 20:25    27  ********
  67    2013-09-17 21:24    28  *********
 ...    ..(  3 skipped).    ..  *********
  71    2013-09-18 01:20    28  *********
  72    2013-09-18 02:19     ?  -
  73    2013-09-18 03:18    28  *********
 ...    ..(  2 skipped).    ..  *********
  76    2013-09-18 06:15    28  *********
  77    2013-09-18 07:14    27  ********
  78    2013-09-18 08:13    27  ********
  79    2013-09-18 09:12    26  *******
  80    2013-09-18 10:11     ?  -
  81    2013-09-18 11:10    33  **************
  82    2013-09-18 12:09     ?  -
  83    2013-09-18 13:08    28  *********
  84    2013-09-18 14:07     ?  -
  85    2013-09-18 15:06    27  ********
  86    2013-09-18 16:05     ?  -
  87    2013-09-18 17:04    23  ****
  88    2013-09-18 18:03     ?  -
  89    2013-09-18 19:02    23  ****
  90    2013-09-18 20:01     ?  -
  91    2013-09-18 21:00    20  *
  92    2013-09-18 21:59    20  *
  93    2013-09-18 22:58     ?  -
  94    2013-09-18 23:57    25  ******
  95    2013-09-19 00:56     ?  -
  96    2013-09-19 01:55    27  ********
  97    2013-09-19 02:54     ?  -
  98    2013-09-19 03:53    28  *********
  99    2013-09-19 04:52     ?  -
 100    2013-09-19 05:51    30  ***********
 101    2013-09-19 06:50    30  ***********
 102    2013-09-19 07:49    26  *******
 103    2013-09-19 08:48    26  *******
 104    2013-09-19 09:47    26  *******
 105    2013-09-19 10:46    25  ******
 106    2013-09-19 11:45    25  ******
 107    2013-09-19 12:44    24  *****
 108    2013-09-19 13:43    24  *****
 109    2013-09-19 14:42    23  ****
 110    2013-09-19 15:41    23  ****
 111    2013-09-19 16:40    22  ***
 112    2013-09-19 17:39    22  ***
 113    2013-09-19 18:38     ?  -
 114    2013-09-19 19:37    20  *
 115    2013-09-19 20:36     ?  -
 116    2013-09-19 21:35    21  **
 117    2013-09-19 22:34     ?  -
 118    2013-09-19 23:33    28  *********
 119    2013-09-20 00:32     ?  -
 120    2013-09-20 01:31    28  *********
 121    2013-09-20 02:30     ?  -
 122    2013-09-20 03:29    24  *****
 123    2013-09-20 04:28    24  *****
 124    2013-09-20 05:27    36  *****************
 125    2013-09-20 06:26    42  ***********************
 126    2013-09-20 07:25    26  *******
 127    2013-09-20 08:24     ?  -
   0    2013-09-20 09:23    30  ***********
   1    2013-09-20 10:22     ?  -
   2    2013-09-20 11:21    33  **************
   3    2013-09-20 12:20     ?  -
   4    2013-09-20 13:19    34  ***************
   5    2013-09-20 14:18     ?  -
   6    2013-09-20 15:17    34  ***************
   7    2013-09-20 16:16    34  ***************
   8    2013-09-20 17:15    33  **************
   9    2013-09-20 18:14     ?  -
  10    2013-09-20 19:13    22  ***
  11    2013-09-20 20:12    22  ***
  12    2013-09-20 21:11    34  ***************
  13    2013-09-20 22:10    34  ***************
  14    2013-09-20 23:09    35  ****************
 ...    ..(  3 skipped).    ..  ****************
  18    2013-09-21 03:05    35  ****************
  19    2013-09-21 04:04    36  *****************
 ...    ..(  6 skipped).    ..  *****************
  26    2013-09-21 10:57    36  *****************
  27    2013-09-21 11:56    35  ****************
 ...    ..(  5 skipped).    ..  ****************
  33    2013-09-21 17:50    35  ****************
  34    2013-09-21 18:49    34  ***************
  35    2013-09-21 19:48    34  ***************
  36    2013-09-21 20:47    34  ***************
  37    2013-09-21 21:46    35  ****************
  38    2013-09-21 22:45    36  *****************
  39    2013-09-21 23:44    36  *****************
  40    2013-09-22 00:43    36  *****************
  41    2013-09-22 01:42    35  ****************
  42    2013-09-22 02:41    35  ****************
  43    2013-09-22 03:40    36  *****************
  44    2013-09-22 04:39    36  *****************
  45    2013-09-22 05:38    37  ******************
  46    2013-09-22 06:37    38  *******************
  47    2013-09-22 07:36    37  ******************
  48    2013-09-22 08:35    40  *********************
  49    2013-09-22 09:34    39  ********************
 ...    ..(  2 skipped).    ..  ********************
  52    2013-09-22 12:31    39  ********************
  53    2013-09-22 13:30     ?  -
  54    2013-09-22 14:29    24  *****
  55    2013-09-22 15:28    24  *****

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x000a  2            2  Device-to-host register FISes sent due to a COMRESET
0x0001  2            0  Command failed due to ICRC error
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS


root@sysresccd /root % smartctl -x /dev/sdd
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.4.52-std371-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda LP
Device Model:     ST32000542AS
Serial Number:    6XW1T1CG
LU WWN Device Id: 5 000c50 02b5c5911
Firmware Version: CC34
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Sun Sep 22 16:01:32 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (  633) seconds.
Offline data collection
capabilities: (0x73) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: (   1) minutes.
Extended self-test routine
recommended polling time: ( 255) minutes.
Conveyance self-test routine
recommended polling time: (   2) minutes.
SCT capabilities:       (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   119   099   006    -    212966341
  3 Spin_Up_Time            PO----   100   100   000    -    0
  4 Start_Stop_Count        -O--CK   100   100   020    -    106
  5 Reallocated_Sector_Ct   PO--CK   100   100   036    -    0
  7 Seek_Error_Rate         POSR--   065   053   030    -    30090297916
  9 Power_On_Hours          -O--CK   086   086   000    -    12579
 10 Spin_Retry_Count        PO--C-   100   100   097    -    0
 12 Power_Cycle_Count       -O--CK   100   100   020    -    89
183 Runtime_Bad_Block       -O--CK   099   099   000    -    1
184 End-to-End_Error        -O--CK   100   100   099    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
188 Command_Timeout         -O--CK   100   099   000    -    17180131333
189 High_Fly_Writes         -O-RCK   100   100   000    -    0
190 Airflow_Temperature_Cel -O---K   075   053   045    -    25 (Min/Max 24/25)
194 Temperature_Celsius     -O---K   025   047   000    -    25 (0 14 0 0 0)
195 Hardware_ECC_Recovered  -O-RC-   052   018   000    -    212966341
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
240 Head_Flying_Hours       ------   100   253   000    -    133221295599862
241 Total_LBAs_Written      ------   100   253   000    -    1574498906
242 Total_LBAs_Read         ------   100   253   000    -    2425164431
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
GP/S  Log at address 0x01 has    1 sectors [Summary SMART error log]
GP/S  Log at address 0x02 has    5 sectors [Comprehensive SMART error log]
GP/S  Log at address 0x03 has    5 sectors [Ext. Comprehensive SMART error log]
GP/S  Log at address 0x06 has    1 sectors [SMART self-test log]
GP/S  Log at address 0x07 has    1 sectors [Extended self-test log]
GP/S  Log at address 0x09 has    1 sectors [Selective self-test log]
GP/S  Log at address 0x10 has    1 sectors [NCQ Command Error log]
GP/S  Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP/S  Log at address 0x21 has    1 sectors [Write stream error log]
GP/S  Log at address 0x22 has    1 sectors [Read stream error log]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xa1 has   20 sectors [Device vendor specific log]
GP    Log at address 0xa2 has 2248 sectors [Device vendor specific log]
GP/S  Log at address 0xa8 has  129 sectors [Device vendor specific log]
GP/S  Log at address 0xa9 has    1 sectors [Device vendor specific log]
GP    Log at address 0xb0 has 2928 sectors [Device vendor specific log]
GP/S  Log at address 0xbd has  252 sectors [Device vendor specific log]
GP    Log at address 0xbe has 65535 sectors [Device vendor specific log]
GP    Log at address 0xbf has 65535 sectors [Device vendor specific log]
GP/S  Log at address 0xc0 has    1 sectors [Device vendor specific log]
GP/S  Log at address 0xe0 has    1 sectors [SCT Command/Status]
GP/S  Log at address 0xe1 has    1 sectors [SCT Data Transfer]

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       522 (0x020a)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    25 Celsius
Power Cycle Min/Max Temperature:     24/25 Celsius
Lifetime    Min/Max Temperature:     14/47 Celsius
Under/Over Temperature Limit Count:   0/0
SCT Temperature History Version:     2
Temperature Sampling Period:         10 minutes
Temperature Logging Interval:        59 minutes
Min/Max recommended Temperature:     14/55 Celsius
Min/Max Temperature Limit:           10/60 Celsius
Temperature History Size (Index):    128 (53)

Index    Estimated Time   Temperature Celsius
  54    2013-09-17 10:35    27  ********
  55    2013-09-17 11:34    27  ********
  56    2013-09-17 12:33    28  *********
  57    2013-09-17 13:32    27  ********
  58    2013-09-17 14:31    28  *********
  59    2013-09-17 15:30    27  ********
  60    2013-09-17 16:29    28  *********
  61    2013-09-17 17:28    28  *********
  62    2013-09-17 18:27    27  ********
  63    2013-09-17 19:26    28  *********
  64    2013-09-17 20:25    27  ********
  65    2013-09-17 21:24    28  *********
  66    2013-09-17 22:23    27  ********
  67    2013-09-17 23:22    28  *********
  68    2013-09-18 00:21    27  ********
  69    2013-09-18 01:20    28  *********
 ...    ..(  3 skipped).    ..  *********
  73    2013-09-18 05:16    28  *********
  74    2013-09-18 06:15     ?  -
  75    2013-09-18 07:14    27  ********
  76    2013-09-18 08:13    27  ********
  77    2013-09-18 09:12    28  *********
  78    2013-09-18 10:11    29  **********
  79    2013-09-18 11:10    28  *********
  80    2013-09-18 12:09    26  *******
  81    2013-09-18 13:08    26  *******
  82    2013-09-18 14:07     ?  -
  83    2013-09-18 15:06    33  **************
  84    2013-09-18 16:05     ?  -
  85    2013-09-18 17:04    28  *********
  86    2013-09-18 18:03     ?  -
  87    2013-09-18 19:02    27  ********
  88    2013-09-18 20:01     ?  -
  89    2013-09-18 21:00    22  ***
  90    2013-09-18 21:59     ?  -
  91    2013-09-18 22:58    23  ****
  92    2013-09-18 23:57     ?  -
  93    2013-09-19 00:56    20  *
  94    2013-09-19 01:55    20  *
  95    2013-09-19 02:54    22  ***
  96    2013-09-19 03:53     ?  -
  97    2013-09-19 04:52    25  ******
  98    2013-09-19 05:51     ?  -
  99    2013-09-19 06:50    26  *******
 100    2013-09-19 07:49     ?  -
 101    2013-09-19 08:48    26  *******
 102    2013-09-19 09:47     ?  -
 103    2013-09-19 10:46    27  ********
 104    2013-09-19 11:45     ?  -
 105    2013-09-19 12:44    29  **********
 106    2013-09-19 13:43    29  **********
 107    2013-09-19 14:42    27  ********
 108    2013-09-19 15:41    26  *******
 109    2013-09-19 16:40    25  ******
 110    2013-09-19 17:39    25  ******
 111    2013-09-19 18:38    24  *****
 112    2013-09-19 19:37    24  *****
 113    2013-09-19 20:36    23  ****
 114    2013-09-19 21:35    23  ****
 115    2013-09-19 22:34    23  ****
 116    2013-09-19 23:33    22  ***
 117    2013-09-20 00:32    22  ***
 118    2013-09-20 01:31     ?  -
 119    2013-09-20 02:30    20  *
 120    2013-09-20 03:29     ?  -
 121    2013-09-20 04:28    21  **
 122    2013-09-20 05:27     ?  -
 123    2013-09-20 06:26    28  *********
 124    2013-09-20 07:25     ?  -
 125    2013-09-20 08:24    28  *********
 126    2013-09-20 09:23     ?  -
 127    2013-09-20 10:22    24  *****
   0    2013-09-20 11:21    24  *****
   1    2013-09-20 12:20    37  ******************
   2    2013-09-20 13:19    42  ***********************
   3    2013-09-20 14:18    26  *******
   4    2013-09-20 15:17     ?  -
   5    2013-09-20 16:16    31  ************
   6    2013-09-20 17:15     ?  -
   7    2013-09-20 18:14    34  ***************
   8    2013-09-20 19:13     ?  -
   9    2013-09-20 20:12    36  *****************
  10    2013-09-20 21:11     ?  -
  11    2013-09-20 22:10    39  ********************
  12    2013-09-20 23:09    39  ********************
  13    2013-09-21 00:08    40  *********************
  14    2013-09-21 01:07     ?  -
  15    2013-09-21 02:06    31  ************
  16    2013-09-21 03:05    31  ************
  17    2013-09-21 04:04    40  *********************
  18    2013-09-21 05:03    40  *********************
  19    2013-09-21 06:02    36  *****************
  20    2013-09-21 07:01    36  *****************
  21    2013-09-21 08:00    35  ****************
  22    2013-09-21 08:59    35  ****************
  23    2013-09-21 09:58    35  ****************
  24    2013-09-21 10:57    33  **************
 ...    ..(  7 skipped).    ..  **************
  32    2013-09-21 18:49    33  **************
  33    2013-09-21 19:48    34  ***************
  34    2013-09-21 20:47    35  ****************
  35    2013-09-21 21:46    35  ****************
  36    2013-09-21 22:45    36  *****************
  37    2013-09-21 23:44    36  *****************
  38    2013-09-22 00:43    37  ******************
  39    2013-09-22 01:42    36  *****************
 ...    ..( 10 skipped).    ..  *****************
  50    2013-09-22 12:31    36  *****************
  51    2013-09-22 13:30     ?  -
  52    2013-09-22 14:29    24  *****
  53    2013-09-22 15:28    24  *****

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x000a  2            2  Device-to-host register FISes sent due to a COMRESET
0x0001  2            0  Command failed due to ICRC error
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS


Dave


On 22 September 2013 22:45, Dave Gomboc <dave_gomboc@acm.org> wrote:
> Yikes.  The data had been accessible on the evening of Sat. Sept 14.
[remainder snipped]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-22 22:45           ` Dave Gomboc
  2013-09-22 23:04             ` Dave Gomboc
@ 2013-09-22 23:25             ` Phil Turmel
       [not found]               ` <CA+dwz-0eskPSQ44v0vgwfjwRpTbQaokQ3Q258Em1W2eRi1SO4w@mail.gmail.com>
  1 sibling, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-22 23:25 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

[Please trim replies/avoid top-posting]

On 09/22/2013 06:45 PM, Dave Gomboc wrote:
> Yikes.  The data had been accessible on the evening of Sat. Sept 14.
> I had (apparently falsely) hoped that there were two complete copies
> of all the data, and that to restore access it would be enough to
> assign the correct drives to position 1 and 2 based on what matched
> against the contents of positions 0 and 3.

We will certainly want to inspect your drives to try to determine which
ones belong where.

> I have ran these commands on the four 4TB drives that are the copies.
> I think the smartctl -x commands in particular you want to see run
> against the original set of drives, so after sending this message, I
> will take down the machine, swap out the four copied drives for the
> four original drives, then run these commands again on the original
> drives.

Not really.  Once we start poking at the devices at a low level, the
most important thing is to keep track of drive serial numbers versus
linux device name.  I created a utility to help document a running
system [1], but that wouldn't be much help here.

> I'm not knowledgeable about the LVM backup file.  Would one typically
> happen to be stored in /boot?  That was the only directory I had that
> was not part of the raid10 system.  I had distinct logical volumes for
> home, opt, usr, var, srv, root (/), and tmp.

No, on my systems they are stored under /etc/lvm/backup/.  You don't
have your root volume, so you can only use the backup embedded in the PV
metadata.

> SCT Error Recovery Control:
>            Read: Disabled
>           Write: Disabled

While you are fixing things, you *really* need to fix this.  Consumer
drives don't wake up with this enabled, if supported at all.  You need
to set these in a boot script.  Search the archives for various
combinations of "scterc", "error recovery control", "tler", and "ure"
for a detailed explanation.

Now, please show the hex dump of your superblocks and the beginning of
the data area on each partition:

dd if=/dev/sdX1 bs=4096 skip=1 count=1 2>/dev/null |hexdump -C
dd if=/dev/sdX1 bs=4096 skip=2048 count=16 2>/dev/null |hexdump -C

And reverify which drive name in linux corresponds to which serial number.

Phil

[1] http://github.com/pturmel/lsdrv


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
       [not found]               ` <CA+dwz-0eskPSQ44v0vgwfjwRpTbQaokQ3Q258Em1W2eRi1SO4w@mail.gmail.com>
@ 2013-09-23  2:54                 ` Phil Turmel
  2013-09-23  3:19                   ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-23  2:54 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 09/22/2013 08:29 PM, Dave Gomboc wrote:
> I now have the 4TB Hitachis in again.  I'll leave them in unless you
> ask me to switch back to the 2TB Seagates.

Ok.

>> dd if=/dev/sdX1 bs=4096 skip=1 count=1 2>/dev/null |hexdump -C
>> dd if=/dev/sdX1 bs=4096 skip=2048 count=16 2>/dev/null |hexdump -C
>>
>> And reverify which drive name in linux corresponds to which serial number.
> 
> ls -l /dev/disk/by-id output includes:
> 
> lrwxrwxrwx 1 root root  9 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J -> ../../sdb
> lrwxrwxrwx 1 root root 10 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J-part1 -> ../../sdb1
> lrwxrwxrwx 1 root root  9 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ -> ../../sdc
> lrwxrwxrwx 1 root root 10 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ-part1 -> ../../sdc1
> lrwxrwxrwx 1 root root  9 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J -> ../../sdd
> lrwxrwxrwx 1 root root 10 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J-part1 -> ../../sdd1
> lrwxrwxrwx 1 root root  9 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS -> ../../sda
> lrwxrwxrwx 1 root root 10 Sep 22 17:20
> ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS-part1 -> ../../sda1

Ok.

> root@sysresccd /root % dd if=/dev/sda1 bs=4096 skip=1 count=1
> 2>/dev/null |hexdump -C
> 00000000  fc 4e 2b a9 01 00 00 00  00 00 00 00 00 00 00 00  |.N+.............|
> 00000010  3c 76 af 95 fa 16 ca 54  33 0b 75 47 b9 04 0b 8a  |<v.....T3.uG....|
> 00000020  63 68 65 61 70 3a 74 65  72 61 6d 6f 6f 63 68 00  |cheap:teramooch.|
> 00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> 00000040  df 62 21 4d 00 00 00 00  0a 00 00 00 02 01 00 00  |.b!M............|
> 00000050  00 68 e0 e8 00 00 00 00  00 04 00 00 04 00 00 00  |.h..............|
> 00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> *
> 00000080  00 08 00 00 00 00 00 00  82 6c e0 e8 00 00 00 00  |.........l......|
> 00000090  08 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> 000000a0  00 00 00 00 00 00 00 00  66 80 69 13 40 2f 8c fb  |........f.i.@/..|
> 000000b0  a7 4e 11 fd 5c a1 7f ee  00 00 00 00 00 00 00 00  |.N..\...........|
> 000000c0  b5 fe 33 52 00 00 00 00  08 02 00 00 00 00 00 00  |..3R............|
> 000000d0  ff ff ff ff ff ff ff ff  7f 39 b7 06 80 01 00 00  |.........9......|
> 000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> *
> 00000100  00 00 01 00 02 00 03 00  fe ff fe ff fe ff fe ff  |................|
> 00000110  fe ff fe ff fe ff fe ff  fe ff fe ff fe ff fe ff  |................|
> *
> 00000400  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> *
> 00001000



> root@sysresccd /root % dd if=/dev/sda1 bs=4096 skip=2048 count=16
> 2>/dev/null |hexdump -C
> 00000000  73 65 74 0d 0a 20 20 20  62 20 20 20 20 20 20 20  |set..   b       |

Oops.  Should have dropped the "bs=4096" on that one.  The default
"bs=512" is what I was trying for there.

However, based on the data you've reported, sda1 and sdc1 are identical,
down to the superblock bytes.  And sdb1 and sdd1 are also identical,
down to the superblock bytes.

Are you sure there were no misdirected "dd" operations?

Phil

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  2:54                 ` Phil Turmel
@ 2013-09-23  3:19                   ` Dave Gomboc
  2013-09-23  3:27                     ` Phil Turmel
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-23  3:19 UTC (permalink / raw)
  To: Phil Turmel; +Cc: linux-raid

[snipped]
> However, based on the data you've reported, sda1 and sdc1 are identical,
> down to the superblock bytes.  And sdb1 and sdd1 are also identical,
> down to the superblock bytes.
>
> Are you sure there were no misdirected "dd" operations?
>
> Phil

I did not start using ddrescue until after I was no longer able to
boot the raid array.  When I did the ddrescuing, I specifically used
the /dev/disk/by-id symlinks, not the sdX names, precisely in order to
avoid accidents.  I was extremely alarmed by the inability to boot the
raid10, and took extra care to make sure I was doing it properly.

Here are my ddrescue log files:

# Rescue Logfile. Created by GNU ddrescue version 1.16
# Command line: ddrescue --cluster-size=65536 --no-split --force
/dev/disk/by-id/ata-ST32000542AS_5XW1P852
/dev/disk/by-id/ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ
/mnt/windows/5XW1P852_to_PK1310PAG62REJ.txt
# current_pos  current_status
0x1D1C0000000     +
#      pos        size  status
0x00000000  0x1D1C1116000  +


# Rescue Logfile. Created by GNU ddrescue version 1.16
# Command line: ddrescue --cluster-size=65536 --no-split --force
/dev/disk/by-id/ata-ST32000542AS_6XW1T1CG
/dev/disk/by-id/ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J
/mnt/windows/6XW1T1CG_to_PK1310PAG62T2J.txt
# current_pos  current_status
0x1D1C0000000     +
#      pos        size  status
0x00000000  0x1D1C1116000  +


# Rescue Logfile. Created by GNU ddrescue version 1.16
# Command line: ddrescue --cluster-size=65536 --no-split --force
/dev/disk/by-id/ata-ST32000542AS_6XW1T14N
/dev/disk/by-id/ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS
/mnt/windows/6XW1T14N_to_PK1311PAG4W5TS.txt
# current_pos  current_status
0x1D1C0000000     +
#      pos        size  status
0x00000000  0x1D1C1116000  +


# Rescue Logfile. Created by GNU ddrescue version 1.16
# Command line: ddrescue --cluster-size=65536 --no-split --force
/dev/disk/by-id/ata-ST32000542AS_5XW1P41K
/dev/disk/by-id/ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J
/mnt/windows/5XW1P41K_to_PK1310PAG5ZY0J.txt
# current_pos  current_status
0x1D1C0000000     +
#      pos        size  status
0x00000000  0x1D1C1116000  +

(/mnt/windows is just where I mounted a spare 64Gb fat32 partition
that is an extra, previously empty partition on the boot drive that
also holds /boot and a linux swap partition).

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  3:19                   ` Dave Gomboc
@ 2013-09-23  3:27                     ` Phil Turmel
  2013-09-23  3:34                       ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-23  3:27 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 09/22/2013 11:19 PM, Dave Gomboc wrote:
> [snipped]
>> However, based on the data you've reported, sda1 and sdc1 are identical,
>> down to the superblock bytes.  And sdb1 and sdd1 are also identical,
>> down to the superblock bytes.
>>
>> Are you sure there were no misdirected "dd" operations?
>>
>> Phil
> 
> I did not start using ddrescue until after I was no longer able to
> boot the raid array.  When I did the ddrescuing, I specifically used
> the /dev/disk/by-id symlinks, not the sdX names, precisely in order to
> avoid accidents.  I was extremely alarmed by the inability to boot the
> raid10, and took extra care to make sure I was doing it properly.

Ok.

If you'll generate the following for each member, we'll see what we can do:

dd if=/dev/sdX1 bs=512 skip=2048 count=16 2>/dev/null |hexdump -C
dd if=/dev/sdX1 bs=512 skip=3072 count=16 2>/dev/null |hexdump -C

(These will be smaller than the other report...  which didn't make it to
the list.)

Phil

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  3:27                     ` Phil Turmel
@ 2013-09-23  3:34                       ` Dave Gomboc
  2013-09-23  3:51                         ` Phil Turmel
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-23  3:34 UTC (permalink / raw)
  To: Phil Turmel, linux-raid

> If you'll generate the following for each member, we'll see what we can do:
>
> dd if=/dev/sdX1 bs=512 skip=2048 count=16 2>/dev/null |hexdump -C
> dd if=/dev/sdX1 bs=512 skip=3072 count=16 2>/dev/null |hexdump -C

root@sysresccd /root % ls -l /dev/disk/by-id | grep Hitachi
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J -> ../../sdb
lrwxrwxrwx 1 root root 10 Sep 22 17:20
ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J-part1 -> ../../sdb1
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ -> ../../sdc
lrwxrwxrwx 1 root root 10 Sep 22 17:20
ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ-part1 -> ../../sdc1
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J -> ../../sdd
lrwxrwxrwx 1 root root 10 Sep 22 17:20
ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J-part1 -> ../../sdd1
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS -> ../../sda
lrwxrwxrwx 1 root root 10 Sep 22 17:20
ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS-part1 -> ../../sda1
root@sysresccd /root % dd if=/dev/sda1 bs=512 skip=2048 count=16
2>/dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00  |LABELONE........|
00000210  e9 00 72 aa 20 00 00 00  4c 56 4d 32 20 30 30 31  |..r. ...LVM2 001|
00000220  43 50 76 4e 59 59 32 38  44 6e 4d 39 36 76 6f 72  |CPvNYY28DnM96vor|
00000230  70 73 31 38 38 32 49 4f  52 33 66 6d 37 77 48 4f  |ps1882IOR3fm7wHO|
00000240  00 00 a0 81 a3 03 00 00  00 00 10 00 00 00 00 00  |................|
00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000260  00 00 00 00 00 00 00 00  00 10 00 00 00 00 00 00  |................|
00000270  00 f0 0f 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000280  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  34 1a 4b f1 20 4c 56 4d  32 20 78 5b 35 41 25 72  |4.K. LVM2 x[5A%r|
00001010  30 4e 2a 3e 01 00 00 00  00 10 00 00 00 00 00 00  |0N*>............|
00001020  00 f0 0f 00 00 00 00 00  00 0a 01 00 00 00 00 00  |................|
00001030  e0 0d 00 00 00 00 00 00  a5 31 b3 8a 00 00 00 00  |.........1......|
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001200  74 65 72 61 6d 6f 6f 63  68 20 7b 0a 69 64 20 3d  |teramooch {.id =|
00001210  20 22 6a 34 53 76 6c 79  2d 58 31 75 4f 2d 4f 56  | "j4Svly-X1uO-OV|
00001220  4c 31 2d 63 66 43 64 2d  65 4a 34 73 2d 73 45 51  |L1-cfCd-eJ4s-sEQ|
00001230  73 2d 4f 35 30 64 47 71  22 0a 73 65 71 6e 6f 20  |s-O50dGq".seqno |
00001240  3d 20 31 0a 73 74 61 74  75 73 20 3d 20 5b 22 52  |= 1.status = ["R|
00001250  45 53 49 5a 45 41 42 4c  45 22 2c 20 22 52 45 41  |ESIZEABLE", "REA|
00001260  44 22 2c 20 22 57 52 49  54 45 22 5d 0a 66 6c 61  |D", "WRITE"].fla|
00001270  67 73 20 3d 20 5b 5d 0a  65 78 74 65 6e 74 5f 73  |gs = [].extent_s|
00001280  69 7a 65 20 3d 20 38 31  39 32 0a 6d 61 78 5f 6c  |ize = 8192.max_l|
00001290  76 20 3d 20 30 0a 6d 61  78 5f 70 76 20 3d 20 30  |v = 0.max_pv = 0|
000012a0  0a 6d 65 74 61 64 61 74  61 5f 63 6f 70 69 65 73  |.metadata_copies|
000012b0  20 3d 20 30 0a 0a 70 68  79 73 69 63 61 6c 5f 76  | = 0..physical_v|
000012c0  6f 6c 75 6d 65 73 20 7b  0a 0a 70 76 30 20 7b 0a  |olumes {..pv0 {.|
000012d0  69 64 20 3d 20 22 43 50  76 4e 59 59 2d 32 38 44  |id = "CPvNYY-28D|
000012e0  6e 2d 4d 39 36 76 2d 6f  72 70 73 2d 31 38 38 32  |n-M96v-orps-1882|
000012f0  2d 49 4f 52 33 2d 66 6d  37 77 48 4f 22 0a 64 65  |-IOR3-fm7wHO".de|
00001300  76 69 63 65 20 3d 20 22  2f 64 65 76 2f 6d 64 31  |vice = "/dev/md1|
00001310  32 37 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |27"..status = ["|
00001320  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001330  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001340  65 20 3d 20 37 38 31 34  30 34 31 36 30 30 0a 70  |e = 7814041600.p|
00001350  65 5f 73 74 61 72 74 20  3d 20 32 30 34 38 0a 70  |e_start = 2048.p|
00001360  65 5f 63 6f 75 6e 74 20  3d 20 39 35 33 38 36 32  |e_count = 953862|
00001370  0a 7d 0a 7d 0a 0a 7d 0a  23 20 47 65 6e 65 72 61  |.}.}..}.# Genera|
00001380  74 65 64 20 62 79 20 4c  56 4d 32 20 76 65 72 73  |ted by LVM2 vers|
00001390  69 6f 6e 20 32 2e 30 32  2e 37 33 28 32 29 20 28  |ion 2.02.73(2) (|
000013a0  32 30 31 30 2d 30 39 2d  31 38 29 3a 20 4d 6f 6e  |2010-09-18): Mon|
000013b0  20 4a 61 6e 20 20 33 20  31 37 3a 30 38 3a 35 34  | Jan  3 17:08:54|
000013c0  20 32 30 31 31 0a 0a 63  6f 6e 74 65 6e 74 73 20  | 2011..contents |
000013d0  3d 20 22 54 65 78 74 20  46 6f 72 6d 61 74 20 56  |= "Text Format V|
000013e0  6f 6c 75 6d 65 20 47 72  6f 75 70 22 0a 76 65 72  |olume Group".ver|
000013f0  73 69 6f 6e 20 3d 20 31  0a 0a 64 65 73 63 72 69  |sion = 1..descri|
00001400  70 74 69 6f 6e 20 3d 20  22 22 0a 0a 63 72 65 61  |ption = ""..crea|
00001410  74 69 6f 6e 5f 68 6f 73  74 20 3d 20 22 73 79 73  |tion_host = "sys|
00001420  72 65 73 63 63 64 22 09  23 20 4c 69 6e 75 78 20  |resccd".# Linux |
00001430  73 79 73 72 65 73 63 63  64 20 32 2e 36 2e 33 35  |sysresccd 2.6.35|
00001440  2d 73 74 64 31 36 34 2d  69 33 38 36 20 23 33 20  |-std164-i386 #3 |
00001450  53 4d 50 20 54 75 65 20  4e 6f 76 20 33 30 20 32  |SMP Tue Nov 30 2|
00001460  30 3a 34 32 3a 34 36 20  55 54 43 20 32 30 31 30  |0:42:46 UTC 2010|
00001470  20 69 36 38 36 0a 63 72  65 61 74 69 6f 6e 5f 74  | i686.creation_t|
00001480  69 6d 65 20 3d 20 31 32  39 34 30 37 34 35 33 34  |ime = 1294074534|
00001490  09 23 20 4d 6f 6e 20 4a  61 6e 20 20 33 20 31 37  |.# Mon Jan  3 17|
000014a0  3a 30 38 3a 35 34 20 32  30 31 31 0a 0a 00 00 00  |:08:54 2011.....|
000014b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001600  74 65 72 61 6d 6f 6f 63  68 20 7b 0a 69 64 20 3d  |teramooch {.id =|
00001610  20 22 6a 34 53 76 6c 79  2d 58 31 75 4f 2d 4f 56  | "j4Svly-X1uO-OV|
00001620  4c 31 2d 63 66 43 64 2d  65 4a 34 73 2d 73 45 51  |L1-cfCd-eJ4s-sEQ|
00001630  73 2d 4f 35 30 64 47 71  22 0a 73 65 71 6e 6f 20  |s-O50dGq".seqno |
00001640  3d 20 32 0a 73 74 61 74  75 73 20 3d 20 5b 22 52  |= 2.status = ["R|
00001650  45 53 49 5a 45 41 42 4c  45 22 2c 20 22 52 45 41  |ESIZEABLE", "REA|
00001660  44 22 2c 20 22 57 52 49  54 45 22 5d 0a 66 6c 61  |D", "WRITE"].fla|
00001670  67 73 20 3d 20 5b 5d 0a  65 78 74 65 6e 74 5f 73  |gs = [].extent_s|
00001680  69 7a 65 20 3d 20 38 31  39 32 0a 6d 61 78 5f 6c  |ize = 8192.max_l|
00001690  76 20 3d 20 30 0a 6d 61  78 5f 70 76 20 3d 20 30  |v = 0.max_pv = 0|
000016a0  0a 6d 65 74 61 64 61 74  61 5f 63 6f 70 69 65 73  |.metadata_copies|
000016b0  20 3d 20 30 0a 0a 70 68  79 73 69 63 61 6c 5f 76  | = 0..physical_v|
000016c0  6f 6c 75 6d 65 73 20 7b  0a 0a 70 76 30 20 7b 0a  |olumes {..pv0 {.|
000016d0  69 64 20 3d 20 22 43 50  76 4e 59 59 2d 32 38 44  |id = "CPvNYY-28D|
000016e0  6e 2d 4d 39 36 76 2d 6f  72 70 73 2d 31 38 38 32  |n-M96v-orps-1882|
000016f0  2d 49 4f 52 33 2d 66 6d  37 77 48 4f 22 0a 64 65  |-IOR3-fm7wHO".de|
00001700  76 69 63 65 20 3d 20 22  2f 64 65 76 2f 6d 64 31  |vice = "/dev/md1|
00001710  32 37 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |27"..status = ["|
00001720  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001730  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001740  65 20 3d 20 37 38 31 34  30 34 31 36 30 30 0a 70  |e = 7814041600.p|
00001750  65 5f 73 74 61 72 74 20  3d 20 32 30 34 38 0a 70  |e_start = 2048.p|
00001760  65 5f 63 6f 75 6e 74 20  3d 20 39 35 33 38 36 32  |e_count = 953862|
00001770  0a 7d 0a 7d 0a 0a 6c 6f  67 69 63 61 6c 5f 76 6f  |.}.}..logical_vo|
00001780  6c 75 6d 65 73 20 7b 0a  0a 72 6f 6f 74 20 7b 0a  |lumes {..root {.|
00001790  69 64 20 3d 20 22 39 31  34 6f 75 75 2d 52 49 61  |id = "914ouu-RIa|
000017a0  6e 2d 35 38 6a 39 2d 48  55 38 78 2d 6c 4c 31 72  |n-58j9-HU8x-lL1r|
000017b0  2d 73 68 6a 72 2d 42 46  75 70 50 6e 22 0a 73 74  |-shjr-BFupPn".st|
000017c0  61 74 75 73 20 3d 20 5b  22 52 45 41 44 22 2c 20  |atus = ["READ", |
000017d0  22 57 52 49 54 45 22 2c  20 22 56 49 53 49 42 4c  |"WRITE", "VISIBL|
000017e0  45 22 5d 0a 66 6c 61 67  73 20 3d 20 5b 5d 0a 73  |E"].flags = [].s|
000017f0  65 67 6d 65 6e 74 5f 63  6f 75 6e 74 20 3d 20 31  |egment_count = 1|
00001800  0a 0a 73 65 67 6d 65 6e  74 31 20 7b 0a 73 74 61  |..segment1 {.sta|
00001810  72 74 5f 65 78 74 65 6e  74 20 3d 20 30 0a 65 78  |rt_extent = 0.ex|
00001820  74 65 6e 74 5f 63 6f 75  6e 74 20 3d 20 31 32 38  |tent_count = 128|
00001830  30 0a 0a 74 79 70 65 20  3d 20 22 73 74 72 69 70  |0..type = "strip|
00001840  65 64 22 0a 73 74 72 69  70 65 5f 63 6f 75 6e 74  |ed".stripe_count|
00001850  20 3d 20 31 09 23 20 6c  69 6e 65 61 72 0a 0a 73  | = 1.# linear..s|
00001860  74 72 69 70 65 73 20 3d  20 5b 0a 22 70 76 30 22  |tripes = [."pv0"|
00001870  2c 20 30 0a 5d 0a 7d 0a  7d 0a 7d 0a 7d 0a 23 20  |, 0.].}.}.}.}.# |
00001880  47 65 6e 65 72 61 74 65  64 20 62 79 20 4c 56 4d  |Generated by LVM|
00001890  32 20 76 65 72 73 69 6f  6e 20 32 2e 30 32 2e 37  |2 version 2.02.7|
000018a0  33 28 32 29 20 28 32 30  31 30 2d 30 39 2d 31 38  |3(2) (2010-09-18|
000018b0  29 3a 20 4d 6f 6e 20 4a  61 6e 20 20 33 20 31 37  |): Mon Jan  3 17|
000018c0  3a 31 32 3a 31 31 20 32  30 31 31 0a 0a 63 6f 6e  |:12:11 2011..con|
000018d0  74 65 6e 74 73 20 3d 20  22 54 65 78 74 20 46 6f  |tents = "Text Fo|
000018e0  72 6d 61 74 20 56 6f 6c  75 6d 65 20 47 72 6f 75  |rmat Volume Grou|
000018f0  70 22 0a 76 65 72 73 69  6f 6e 20 3d 20 31 0a 0a  |p".version = 1..|
00001900  64 65 73 63 72 69 70 74  69 6f 6e 20 3d 20 22 22  |description = ""|
00001910  0a 0a 63 72 65 61 74 69  6f 6e 5f 68 6f 73 74 20  |..creation_host |
00001920  3d 20 22 73 79 73 72 65  73 63 63 64 22 09 23 20  |= "sysresccd".# |
00001930  4c 69 6e 75 78 20 73 79  73 72 65 73 63 63 64 20  |Linux sysresccd |
00001940  32 2e 36 2e 33 35 2d 73  74 64 31 36 34 2d 69 33  |2.6.35-std164-i3|
00001950  38 36 20 23 33 20 53 4d  50 20 54 75 65 20 4e 6f  |86 #3 SMP Tue No|
00001960  76 20 33 30 20 32 30 3a  34 32 3a 34 36 20 55 54  |v 30 20:42:46 UT|
00001970  43 20 32 30 31 30 20 69  36 38 36 0a 63 72 65 61  |C 2010 i686.crea|
00001980  74 69 6f 6e 5f 74 69 6d  65 20 3d 20 31 32 39 34  |tion_time = 1294|
00001990  30 37 34 37 33 31 09 23  20 4d 6f 6e 20 4a 61 6e  |074731.# Mon Jan|
000019a0  20 20 33 20 31 37 3a 31  32 3a 31 31 20 32 30 31  |  3 17:12:11 201|
000019b0  31 0a 0a 00 00 00 00 00  00 00 00 00 00 00 00 00  |1...............|
000019c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001a00  74 65 72 61 6d 6f 6f 63  68 20 7b 0a 69 64 20 3d  |teramooch {.id =|
00001a10  20 22 6a 34 53 76 6c 79  2d 58 31 75 4f 2d 4f 56  | "j4Svly-X1uO-OV|
00001a20  4c 31 2d 63 66 43 64 2d  65 4a 34 73 2d 73 45 51  |L1-cfCd-eJ4s-sEQ|
00001a30  73 2d 4f 35 30 64 47 71  22 0a 73 65 71 6e 6f 20  |s-O50dGq".seqno |
00001a40  3d 20 33 0a 73 74 61 74  75 73 20 3d 20 5b 22 52  |= 3.status = ["R|
00001a50  45 53 49 5a 45 41 42 4c  45 22 2c 20 22 52 45 41  |ESIZEABLE", "REA|
00001a60  44 22 2c 20 22 57 52 49  54 45 22 5d 0a 66 6c 61  |D", "WRITE"].fla|
00001a70  67 73 20 3d 20 5b 5d 0a  65 78 74 65 6e 74 5f 73  |gs = [].extent_s|
00001a80  69 7a 65 20 3d 20 38 31  39 32 0a 6d 61 78 5f 6c  |ize = 8192.max_l|
00001a90  76 20 3d 20 30 0a 6d 61  78 5f 70 76 20 3d 20 30  |v = 0.max_pv = 0|
00001aa0  0a 6d 65 74 61 64 61 74  61 5f 63 6f 70 69 65 73  |.metadata_copies|
00001ab0  20 3d 20 30 0a 0a 70 68  79 73 69 63 61 6c 5f 76  | = 0..physical_v|
00001ac0  6f 6c 75 6d 65 73 20 7b  0a 0a 70 76 30 20 7b 0a  |olumes {..pv0 {.|
00001ad0  69 64 20 3d 20 22 43 50  76 4e 59 59 2d 32 38 44  |id = "CPvNYY-28D|
00001ae0  6e 2d 4d 39 36 76 2d 6f  72 70 73 2d 31 38 38 32  |n-M96v-orps-1882|
00001af0  2d 49 4f 52 33 2d 66 6d  37 77 48 4f 22 0a 64 65  |-IOR3-fm7wHO".de|
00001b00  76 69 63 65 20 3d 20 22  2f 64 65 76 2f 6d 64 31  |vice = "/dev/md1|
00001b10  32 37 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |27"..status = ["|
00001b20  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001b30  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001b40  65 20 3d 20 37 38 31 34  30 34 31 36 30 30 0a 70  |e = 7814041600.p|
00001b50  65 5f 73 74 61 72 74 20  3d 20 32 30 34 38 0a 70  |e_start = 2048.p|
00001b60  65 5f 63 6f 75 6e 74 20  3d 20 39 35 33 38 36 32  |e_count = 953862|
00001b70  0a 7d 0a 7d 0a 0a 6c 6f  67 69 63 61 6c 5f 76 6f  |.}.}..logical_vo|
00001b80  6c 75 6d 65 73 20 7b 0a  0a 72 6f 6f 74 20 7b 0a  |lumes {..root {.|
00001b90  69 64 20 3d 20 22 39 31  34 6f 75 75 2d 52 49 61  |id = "914ouu-RIa|
00001ba0  6e 2d 35 38 6a 39 2d 48  55 38 78 2d 6c 4c 31 72  |n-58j9-HU8x-lL1r|
00001bb0  2d 73 68 6a 72 2d 42 46  75 70 50 6e 22 0a 73 74  |-shjr-BFupPn".st|
00001bc0  61 74 75 73 20 3d 20 5b  22 52 45 41 44 22 2c 20  |atus = ["READ", |
00001bd0  22 57 52 49 54 45 22 2c  20 22 56 49 53 49 42 4c  |"WRITE", "VISIBL|
00001be0  45 22 5d 0a 66 6c 61 67  73 20 3d 20 5b 5d 0a 73  |E"].flags = [].s|
00001bf0  65 67 6d 65 6e 74 5f 63  6f 75 6e 74 20 3d 20 31  |egment_count = 1|
00001c00  0a 0a 73 65 67 6d 65 6e  74 31 20 7b 0a 73 74 61  |..segment1 {.sta|
00001c10  72 74 5f 65 78 74 65 6e  74 20 3d 20 30 0a 65 78  |rt_extent = 0.ex|
00001c20  74 65 6e 74 5f 63 6f 75  6e 74 20 3d 20 31 32 38  |tent_count = 128|
00001c30  30 0a 0a 74 79 70 65 20  3d 20 22 73 74 72 69 70  |0..type = "strip|
00001c40  65 64 22 0a 73 74 72 69  70 65 5f 63 6f 75 6e 74  |ed".stripe_count|
00001c50  20 3d 20 31 09 23 20 6c  69 6e 65 61 72 0a 0a 73  | = 1.# linear..s|
00001c60  74 72 69 70 65 73 20 3d  20 5b 0a 22 70 76 30 22  |tripes = [."pv0"|
00001c70  2c 20 30 0a 5d 0a 7d 0a  7d 0a 0a 75 73 72 20 7b  |, 0.].}.}..usr {|
00001c80  0a 69 64 20 3d 20 22 61  68 38 43 46 71 2d 72 39  |.id = "ah8CFq-r9|
00001c90  34 36 2d 47 52 78 57 2d  71 31 79 76 2d 76 63 56  |46-GRxW-q1yv-vcV|
00001ca0  48 2d 53 33 34 31 2d 41  6f 4a 6d 42 48 22 0a 73  |H-S341-AoJmBH".s|
00001cb0  74 61 74 75 73 20 3d 20  5b 22 52 45 41 44 22 2c  |tatus = ["READ",|
00001cc0  20 22 57 52 49 54 45 22  2c 20 22 56 49 53 49 42  | "WRITE", "VISIB|
00001cd0  4c 45 22 5d 0a 66 6c 61  67 73 20 3d 20 5b 5d 0a  |LE"].flags = [].|
00001ce0  73 65 67 6d 65 6e 74 5f  63 6f 75 6e 74 20 3d 20  |segment_count = |
00001cf0  31 0a 0a 73 65 67 6d 65  6e 74 31 20 7b 0a 73 74  |1..segment1 {.st|
00001d00  61 72 74 5f 65 78 74 65  6e 74 20 3d 20 30 0a 65  |art_extent = 0.e|
00001d10  78 74 65 6e 74 5f 63 6f  75 6e 74 20 3d 20 31 32  |xtent_count = 12|
00001d20  38 30 0a 0a 74 79 70 65  20 3d 20 22 73 74 72 69  |80..type = "stri|
00001d30  70 65 64 22 0a 73 74 72  69 70 65 5f 63 6f 75 6e  |ped".stripe_coun|
00001d40  74 20 3d 20 31 09 23 20  6c 69 6e 65 61 72 0a 0a  |t = 1.# linear..|
00001d50  73 74 72 69 70 65 73 20  3d 20 5b 0a 22 70 76 30  |stripes = [."pv0|
00001d60  22 2c 20 31 32 38 30 0a  5d 0a 7d 0a 7d 0a 7d 0a  |", 1280.].}.}.}.|
00001d70  7d 0a 23 20 47 65 6e 65  72 61 74 65 64 20 62 79  |}.# Generated by|
00001d80  20 4c 56 4d 32 20 76 65  72 73 69 6f 6e 20 32 2e  | LVM2 version 2.|
00001d90  30 32 2e 37 33 28 32 29  20 28 32 30 31 30 2d 30  |02.73(2) (2010-0|
00001da0  39 2d 31 38 29 3a 20 4d  6f 6e 20 4a 61 6e 20 20  |9-18): Mon Jan  |
00001db0  33 20 31 37 3a 31 32 3a  34 38 20 32 30 31 31 0a  |3 17:12:48 2011.|
00001dc0  0a 63 6f 6e 74 65 6e 74  73 20 3d 20 22 54 65 78  |.contents = "Tex|
00001dd0  74 20 46 6f 72 6d 61 74  20 56 6f 6c 75 6d 65 20  |t Format Volume |
00001de0  47 72 6f 75 70 22 0a 76  65 72 73 69 6f 6e 20 3d  |Group".version =|
00001df0  20 31 0a 0a 64 65 73 63  72 69 70 74 69 6f 6e 20  | 1..description |
00001e00  3d 20 22 22 0a 0a 63 72  65 61 74 69 6f 6e 5f 68  |= ""..creation_h|
00001e10  6f 73 74 20 3d 20 22 73  79 73 72 65 73 63 63 64  |ost = "sysresccd|
00001e20  22 09 23 20 4c 69 6e 75  78 20 73 79 73 72 65 73  |".# Linux sysres|
00001e30  63 63 64 20 32 2e 36 2e  33 35 2d 73 74 64 31 36  |ccd 2.6.35-std16|
00001e40  34 2d 69 33 38 36 20 23  33 20 53 4d 50 20 54 75  |4-i386 #3 SMP Tu|
00001e50  65 20 4e 6f 76 20 33 30  20 32 30 3a 34 32 3a 34  |e Nov 30 20:42:4|
00001e60  36 20 55 54 43 20 32 30  31 30 20 69 36 38 36 0a  |6 UTC 2010 i686.|
00001e70  63 72 65 61 74 69 6f 6e  5f 74 69 6d 65 20 3d 20  |creation_time = |
00001e80  31 32 39 34 30 37 34 37  36 38 09 23 20 4d 6f 6e  |1294074768.# Mon|
00001e90  20 4a 61 6e 20 20 33 20  31 37 3a 31 32 3a 34 38  | Jan  3 17:12:48|
00001ea0  20 32 30 31 31 0a 0a 00  00 00 00 00 00 00 00 00  | 2011...........|
00001eb0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sda1 bs=512 skip=3072 count=16
2>/dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400  00 00 05 00 00 00 14 00  33 33 00 00 b7 03 0b 00  |........33......|
00000410  ef 14 04 00 00 00 00 00  02 00 00 00 02 00 00 00  |................|
00000420  00 80 00 00 00 80 00 00  00 20 00 00 ae fd 33 52  |......... ....3R|
00000430  5f fd 33 52 01 00 21 00  53 ef 01 00 01 00 00 00  |_.3R..!.S.......|
00000440  5f fd 33 52 00 4e ed 00  00 00 00 00 01 00 00 00  |_.3R.N..........|
00000450  00 00 00 00 0b 00 00 00  00 01 00 00 3c 00 00 00  |............<...|
00000460  06 00 00 00 03 00 00 00  42 50 18 5f 9c 6a 4f af  |........BP._.jO.|
00000470  8c 21 8a 6f d6 1d 6f 38  00 00 00 00 00 00 00 00  |.!.o..o8........|
00000480  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000004c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 3f 01  |..............?.|
000004d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000004e0  08 00 00 00 00 00 00 00  00 00 00 00 f5 23 d7 ab  |.............#..|
000004f0  b4 bf 42 c5 b4 63 7a 88  99 90 78 6e 01 01 00 00  |..B..cz...xn....|
00000500  00 00 00 00 00 00 00 00  29 5b a6 4b 02 82 09 00  |........)[.K....|
00000510  03 82 09 00 04 82 09 00  05 82 09 00 06 82 09 00  |................|
00000520  07 82 09 00 08 82 09 00  09 82 09 00 0a 82 09 00  |................|
00000530  0b 82 09 00 0c 82 09 00  0d 82 09 00 0e 82 09 00  |................|
00000540  0f 86 09 00 00 00 00 00  00 00 00 00 00 00 00 08  |................|
00000550  00 00 00 00 00 00 00 00  00 00 00 00 1c 00 1c 00  |................|
00000560  01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000570  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  41 01 00 00 42 01 00 00  43 01 00 00 fa 22 50 1e  |A...B...C...."P.|
00001010  50 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |P...............|
00001020  41 81 00 00 42 81 00 00  43 81 00 00 b0 53 0c 12  |A...B...C....S..|
00001030  4b 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |K...............|
00001040  00 00 01 00 01 00 01 00  02 00 01 00 e7 22 68 1f  |............."h.|
00001050  07 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001060  41 81 01 00 42 81 01 00  43 81 01 00 e7 3f 1e 12  |A...B...C....?..|
00001070  f9 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001080  00 00 02 00 01 00 02 00  02 00 02 00 25 20 24 1c  |............% $.|
00001090  72 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |r...............|
000010a0  41 81 02 00 42 81 02 00  43 81 02 00 1f 35 1c 12  |A...B...C....5..|
000010b0  40 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |@...............|
000010c0  00 00 03 00 01 00 03 00  02 00 03 00 1c 26 4c 12  |.............&L.|
000010d0  8e 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000010e0  41 81 03 00 42 81 03 00  43 81 03 00 bb 1e 82 14  |A...B...C.......|
000010f0  3f 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |?...............|
00001100  00 00 04 00 01 00 04 00  02 00 04 00 7d 20 56 18  |............} V.|
00001110  22 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |"...............|
00001120  41 81 04 00 42 81 04 00  43 81 04 00 ac 1f f3 15  |A...B...C.......|
00001130  ab 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001140  00 00 05 00 01 00 05 00  02 00 05 00 fe 25 90 12  |.............%..|
00001150  b9 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001160  00 80 05 00 01 80 05 00  02 80 05 00 05 14 d4 19  |................|
00001170  4c 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |L...............|
00001180  00 00 06 00 01 00 06 00  02 00 06 00 bb 1e c2 15  |................|
00001190  67 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |g...............|
000011a0  00 80 06 00 01 80 06 00  02 80 06 00 7d 20 99 16  |............} ..|
000011b0  c5 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000011c0  00 00 07 00 01 00 07 00  02 00 07 00 fd 25 47 15  |.............%G.|
000011d0  05 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000011e0  00 80 07 00 01 80 07 00  02 80 07 00 67 26 e6 18  |............g&..|
000011f0  51 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00001200  00 00 08 00 01 00 08 00  02 00 08 00 c7 44 f3 18  |.............D..|
00001210  08 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001220  00 80 08 00 01 80 08 00  02 80 08 00 fe 7d 00 20  |.............}. |
00001230  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001240  00 00 09 00 01 00 09 00  02 00 09 00 fe 7d 00 20  |.............}. |
00001250  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001260  00 80 09 00 01 80 09 00  02 80 09 00 00 00 00 20  |............... |
00001270  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001280  00 00 0a 00 01 00 0a 00  02 00 0a 00 db 7b 00 20  |.............{. |
00001290  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000012a0  00 80 0a 00 01 80 0a 00  02 80 0a 00 fe 7d 00 20  |.............}. |
000012b0  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000012c0  00 00 0b 00 01 00 0b 00  02 00 0b 00 fd 7d fd 1f  |.............}..|
000012d0  01 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000012e0  00 80 0b 00 01 80 0b 00  02 80 0b 00 fe 7d 00 20  |.............}. |
000012f0  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001300  00 00 0c 00 01 00 0c 00  02 00 0c 00 43 7b 28 1f  |............C{(.|
00001310  4d 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |M...............|
00001320  41 81 0c 00 42 81 0c 00  43 81 0c 00 bd 7c 00 20  |A...B...C....|. |
00001330  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001340  00 00 0d 00 01 00 0d 00  02 00 0d 00 1e 75 6e 1f  |.............un.|
00001350  01 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001360  41 81 0d 00 42 81 0d 00  43 81 0d 00 bd 7c 00 20  |A...B...C....|. |
00001370  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001380  00 00 0e 00 01 00 0e 00  02 00 0e 00 fe 7d 00 20  |.............}. |
00001390  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000013a0  00 80 0e 00 01 80 0e 00  02 80 0e 00 fe 7d 00 20  |.............}. |
000013b0  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000013c0  00 00 0f 00 01 00 0f 00  02 00 0f 00 fb 7d fb 1f  |.............}..|
000013d0  03 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000013e0  00 80 0f 00 01 80 0f 00  02 80 0f 00 fd 7d ff 1f  |.............}..|
000013f0  01 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001400  00 00 10 00 01 00 10 00  02 00 10 00 14 27 d3 15  |.............'..|
00001410  4c 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |L...............|
00001420  00 80 10 00 01 80 10 00  02 80 10 00 de 14 7b 15  |..............{.|
00001430  bb 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001440  00 00 11 00 01 00 11 00  02 00 11 00 75 20 75 16  |............u u.|
00001450  97 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001460  00 80 11 00 01 80 11 00  02 80 11 00 e0 23 73 16  |.............#s.|
00001470  85 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001480  00 00 12 00 01 00 12 00  02 00 12 00 dd 23 62 14  |.............#b.|
00001490  c3 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000014a0  00 80 12 00 01 80 12 00  02 80 12 00 72 53 7c 12  |............rS|.|
000014b0  22 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |"...............|
000014c0  00 00 13 00 01 00 13 00  02 00 13 00 4a 22 bb 1f  |............J"..|
000014d0  11 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000014e0  00 80 13 00 01 80 13 00  02 80 13 00 16 58 7b 12  |.............X{.|
000014f0  17 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001500  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sdb1 bs=512 skip=2048 count=16
2>/dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sdb1 bs=512 skip=3072 count=16
2>/dev/null | hexdump -C
00000000  80 80 00 00 80 80 01 00  80 80 02 00 80 80 03 00  |................|
00000010  80 80 04 00 80 80 0c 00  80 80 0d 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  81 80 00 00 81 80 01 00  81 80 02 00 81 80 03 00  |................|
00001010  81 80 04 00 81 80 0c 00  81 80 0d 00 00 00 00 00  |................|
00001020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sdc1 bs=512 skip=2048 count=16
2>/dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00  |LABELONE........|
00000210  e9 00 72 aa 20 00 00 00  4c 56 4d 32 20 30 30 31  |..r. ...LVM2 001|
00000220  43 50 76 4e 59 59 32 38  44 6e 4d 39 36 76 6f 72  |CPvNYY28DnM96vor|
00000230  70 73 31 38 38 32 49 4f  52 33 66 6d 37 77 48 4f  |ps1882IOR3fm7wHO|
00000240  00 00 a0 81 a3 03 00 00  00 00 10 00 00 00 00 00  |................|
00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000260  00 00 00 00 00 00 00 00  00 10 00 00 00 00 00 00  |................|
00000270  00 f0 0f 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000280  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  34 1a 4b f1 20 4c 56 4d  32 20 78 5b 35 41 25 72  |4.K. LVM2 x[5A%r|
00001010  30 4e 2a 3e 01 00 00 00  00 10 00 00 00 00 00 00  |0N*>............|
00001020  00 f0 0f 00 00 00 00 00  00 0a 01 00 00 00 00 00  |................|
00001030  e0 0d 00 00 00 00 00 00  a5 31 b3 8a 00 00 00 00  |.........1......|
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001200  74 65 72 61 6d 6f 6f 63  68 20 7b 0a 69 64 20 3d  |teramooch {.id =|
00001210  20 22 6a 34 53 76 6c 79  2d 58 31 75 4f 2d 4f 56  | "j4Svly-X1uO-OV|
00001220  4c 31 2d 63 66 43 64 2d  65 4a 34 73 2d 73 45 51  |L1-cfCd-eJ4s-sEQ|
00001230  73 2d 4f 35 30 64 47 71  22 0a 73 65 71 6e 6f 20  |s-O50dGq".seqno |
00001240  3d 20 31 0a 73 74 61 74  75 73 20 3d 20 5b 22 52  |= 1.status = ["R|
00001250  45 53 49 5a 45 41 42 4c  45 22 2c 20 22 52 45 41  |ESIZEABLE", "REA|
00001260  44 22 2c 20 22 57 52 49  54 45 22 5d 0a 66 6c 61  |D", "WRITE"].fla|
00001270  67 73 20 3d 20 5b 5d 0a  65 78 74 65 6e 74 5f 73  |gs = [].extent_s|
00001280  69 7a 65 20 3d 20 38 31  39 32 0a 6d 61 78 5f 6c  |ize = 8192.max_l|
00001290  76 20 3d 20 30 0a 6d 61  78 5f 70 76 20 3d 20 30  |v = 0.max_pv = 0|
000012a0  0a 6d 65 74 61 64 61 74  61 5f 63 6f 70 69 65 73  |.metadata_copies|
000012b0  20 3d 20 30 0a 0a 70 68  79 73 69 63 61 6c 5f 76  | = 0..physical_v|
000012c0  6f 6c 75 6d 65 73 20 7b  0a 0a 70 76 30 20 7b 0a  |olumes {..pv0 {.|
000012d0  69 64 20 3d 20 22 43 50  76 4e 59 59 2d 32 38 44  |id = "CPvNYY-28D|
000012e0  6e 2d 4d 39 36 76 2d 6f  72 70 73 2d 31 38 38 32  |n-M96v-orps-1882|
000012f0  2d 49 4f 52 33 2d 66 6d  37 77 48 4f 22 0a 64 65  |-IOR3-fm7wHO".de|
00001300  76 69 63 65 20 3d 20 22  2f 64 65 76 2f 6d 64 31  |vice = "/dev/md1|
00001310  32 37 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |27"..status = ["|
00001320  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001330  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001340  65 20 3d 20 37 38 31 34  30 34 31 36 30 30 0a 70  |e = 7814041600.p|
00001350  65 5f 73 74 61 72 74 20  3d 20 32 30 34 38 0a 70  |e_start = 2048.p|
00001360  65 5f 63 6f 75 6e 74 20  3d 20 39 35 33 38 36 32  |e_count = 953862|
00001370  0a 7d 0a 7d 0a 0a 7d 0a  23 20 47 65 6e 65 72 61  |.}.}..}.# Genera|
00001380  74 65 64 20 62 79 20 4c  56 4d 32 20 76 65 72 73  |ted by LVM2 vers|
00001390  69 6f 6e 20 32 2e 30 32  2e 37 33 28 32 29 20 28  |ion 2.02.73(2) (|
000013a0  32 30 31 30 2d 30 39 2d  31 38 29 3a 20 4d 6f 6e  |2010-09-18): Mon|
000013b0  20 4a 61 6e 20 20 33 20  31 37 3a 30 38 3a 35 34  | Jan  3 17:08:54|
000013c0  20 32 30 31 31 0a 0a 63  6f 6e 74 65 6e 74 73 20  | 2011..contents |
000013d0  3d 20 22 54 65 78 74 20  46 6f 72 6d 61 74 20 56  |= "Text Format V|
000013e0  6f 6c 75 6d 65 20 47 72  6f 75 70 22 0a 76 65 72  |olume Group".ver|
000013f0  73 69 6f 6e 20 3d 20 31  0a 0a 64 65 73 63 72 69  |sion = 1..descri|
00001400  70 74 69 6f 6e 20 3d 20  22 22 0a 0a 63 72 65 61  |ption = ""..crea|
00001410  74 69 6f 6e 5f 68 6f 73  74 20 3d 20 22 73 79 73  |tion_host = "sys|
00001420  72 65 73 63 63 64 22 09  23 20 4c 69 6e 75 78 20  |resccd".# Linux |
00001430  73 79 73 72 65 73 63 63  64 20 32 2e 36 2e 33 35  |sysresccd 2.6.35|
00001440  2d 73 74 64 31 36 34 2d  69 33 38 36 20 23 33 20  |-std164-i386 #3 |
00001450  53 4d 50 20 54 75 65 20  4e 6f 76 20 33 30 20 32  |SMP Tue Nov 30 2|
00001460  30 3a 34 32 3a 34 36 20  55 54 43 20 32 30 31 30  |0:42:46 UTC 2010|
00001470  20 69 36 38 36 0a 63 72  65 61 74 69 6f 6e 5f 74  | i686.creation_t|
00001480  69 6d 65 20 3d 20 31 32  39 34 30 37 34 35 33 34  |ime = 1294074534|
00001490  09 23 20 4d 6f 6e 20 4a  61 6e 20 20 33 20 31 37  |.# Mon Jan  3 17|
000014a0  3a 30 38 3a 35 34 20 32  30 31 31 0a 0a 00 00 00  |:08:54 2011.....|
000014b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001600  74 65 72 61 6d 6f 6f 63  68 20 7b 0a 69 64 20 3d  |teramooch {.id =|
00001610  20 22 6a 34 53 76 6c 79  2d 58 31 75 4f 2d 4f 56  | "j4Svly-X1uO-OV|
00001620  4c 31 2d 63 66 43 64 2d  65 4a 34 73 2d 73 45 51  |L1-cfCd-eJ4s-sEQ|
00001630  73 2d 4f 35 30 64 47 71  22 0a 73 65 71 6e 6f 20  |s-O50dGq".seqno |
00001640  3d 20 32 0a 73 74 61 74  75 73 20 3d 20 5b 22 52  |= 2.status = ["R|
00001650  45 53 49 5a 45 41 42 4c  45 22 2c 20 22 52 45 41  |ESIZEABLE", "REA|
00001660  44 22 2c 20 22 57 52 49  54 45 22 5d 0a 66 6c 61  |D", "WRITE"].fla|
00001670  67 73 20 3d 20 5b 5d 0a  65 78 74 65 6e 74 5f 73  |gs = [].extent_s|
00001680  69 7a 65 20 3d 20 38 31  39 32 0a 6d 61 78 5f 6c  |ize = 8192.max_l|
00001690  76 20 3d 20 30 0a 6d 61  78 5f 70 76 20 3d 20 30  |v = 0.max_pv = 0|
000016a0  0a 6d 65 74 61 64 61 74  61 5f 63 6f 70 69 65 73  |.metadata_copies|
000016b0  20 3d 20 30 0a 0a 70 68  79 73 69 63 61 6c 5f 76  | = 0..physical_v|
000016c0  6f 6c 75 6d 65 73 20 7b  0a 0a 70 76 30 20 7b 0a  |olumes {..pv0 {.|
000016d0  69 64 20 3d 20 22 43 50  76 4e 59 59 2d 32 38 44  |id = "CPvNYY-28D|
000016e0  6e 2d 4d 39 36 76 2d 6f  72 70 73 2d 31 38 38 32  |n-M96v-orps-1882|
000016f0  2d 49 4f 52 33 2d 66 6d  37 77 48 4f 22 0a 64 65  |-IOR3-fm7wHO".de|
00001700  76 69 63 65 20 3d 20 22  2f 64 65 76 2f 6d 64 31  |vice = "/dev/md1|
00001710  32 37 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |27"..status = ["|
00001720  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001730  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001740  65 20 3d 20 37 38 31 34  30 34 31 36 30 30 0a 70  |e = 7814041600.p|
00001750  65 5f 73 74 61 72 74 20  3d 20 32 30 34 38 0a 70  |e_start = 2048.p|
00001760  65 5f 63 6f 75 6e 74 20  3d 20 39 35 33 38 36 32  |e_count = 953862|
00001770  0a 7d 0a 7d 0a 0a 6c 6f  67 69 63 61 6c 5f 76 6f  |.}.}..logical_vo|
00001780  6c 75 6d 65 73 20 7b 0a  0a 72 6f 6f 74 20 7b 0a  |lumes {..root {.|
00001790  69 64 20 3d 20 22 39 31  34 6f 75 75 2d 52 49 61  |id = "914ouu-RIa|
000017a0  6e 2d 35 38 6a 39 2d 48  55 38 78 2d 6c 4c 31 72  |n-58j9-HU8x-lL1r|
000017b0  2d 73 68 6a 72 2d 42 46  75 70 50 6e 22 0a 73 74  |-shjr-BFupPn".st|
000017c0  61 74 75 73 20 3d 20 5b  22 52 45 41 44 22 2c 20  |atus = ["READ", |
000017d0  22 57 52 49 54 45 22 2c  20 22 56 49 53 49 42 4c  |"WRITE", "VISIBL|
000017e0  45 22 5d 0a 66 6c 61 67  73 20 3d 20 5b 5d 0a 73  |E"].flags = [].s|
000017f0  65 67 6d 65 6e 74 5f 63  6f 75 6e 74 20 3d 20 31  |egment_count = 1|
00001800  0a 0a 73 65 67 6d 65 6e  74 31 20 7b 0a 73 74 61  |..segment1 {.sta|
00001810  72 74 5f 65 78 74 65 6e  74 20 3d 20 30 0a 65 78  |rt_extent = 0.ex|
00001820  74 65 6e 74 5f 63 6f 75  6e 74 20 3d 20 31 32 38  |tent_count = 128|
00001830  30 0a 0a 74 79 70 65 20  3d 20 22 73 74 72 69 70  |0..type = "strip|
00001840  65 64 22 0a 73 74 72 69  70 65 5f 63 6f 75 6e 74  |ed".stripe_count|
00001850  20 3d 20 31 09 23 20 6c  69 6e 65 61 72 0a 0a 73  | = 1.# linear..s|
00001860  74 72 69 70 65 73 20 3d  20 5b 0a 22 70 76 30 22  |tripes = [."pv0"|
00001870  2c 20 30 0a 5d 0a 7d 0a  7d 0a 7d 0a 7d 0a 23 20  |, 0.].}.}.}.}.# |
00001880  47 65 6e 65 72 61 74 65  64 20 62 79 20 4c 56 4d  |Generated by LVM|
00001890  32 20 76 65 72 73 69 6f  6e 20 32 2e 30 32 2e 37  |2 version 2.02.7|
000018a0  33 28 32 29 20 28 32 30  31 30 2d 30 39 2d 31 38  |3(2) (2010-09-18|
000018b0  29 3a 20 4d 6f 6e 20 4a  61 6e 20 20 33 20 31 37  |): Mon Jan  3 17|
000018c0  3a 31 32 3a 31 31 20 32  30 31 31 0a 0a 63 6f 6e  |:12:11 2011..con|
000018d0  74 65 6e 74 73 20 3d 20  22 54 65 78 74 20 46 6f  |tents = "Text Fo|
000018e0  72 6d 61 74 20 56 6f 6c  75 6d 65 20 47 72 6f 75  |rmat Volume Grou|
000018f0  70 22 0a 76 65 72 73 69  6f 6e 20 3d 20 31 0a 0a  |p".version = 1..|
00001900  64 65 73 63 72 69 70 74  69 6f 6e 20 3d 20 22 22  |description = ""|
00001910  0a 0a 63 72 65 61 74 69  6f 6e 5f 68 6f 73 74 20  |..creation_host |
00001920  3d 20 22 73 79 73 72 65  73 63 63 64 22 09 23 20  |= "sysresccd".# |
00001930  4c 69 6e 75 78 20 73 79  73 72 65 73 63 63 64 20  |Linux sysresccd |
00001940  32 2e 36 2e 33 35 2d 73  74 64 31 36 34 2d 69 33  |2.6.35-std164-i3|
00001950  38 36 20 23 33 20 53 4d  50 20 54 75 65 20 4e 6f  |86 #3 SMP Tue No|
00001960  76 20 33 30 20 32 30 3a  34 32 3a 34 36 20 55 54  |v 30 20:42:46 UT|
00001970  43 20 32 30 31 30 20 69  36 38 36 0a 63 72 65 61  |C 2010 i686.crea|
00001980  74 69 6f 6e 5f 74 69 6d  65 20 3d 20 31 32 39 34  |tion_time = 1294|
00001990  30 37 34 37 33 31 09 23  20 4d 6f 6e 20 4a 61 6e  |074731.# Mon Jan|
000019a0  20 20 33 20 31 37 3a 31  32 3a 31 31 20 32 30 31  |  3 17:12:11 201|
000019b0  31 0a 0a 00 00 00 00 00  00 00 00 00 00 00 00 00  |1...............|
000019c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001a00  74 65 72 61 6d 6f 6f 63  68 20 7b 0a 69 64 20 3d  |teramooch {.id =|
00001a10  20 22 6a 34 53 76 6c 79  2d 58 31 75 4f 2d 4f 56  | "j4Svly-X1uO-OV|
00001a20  4c 31 2d 63 66 43 64 2d  65 4a 34 73 2d 73 45 51  |L1-cfCd-eJ4s-sEQ|
00001a30  73 2d 4f 35 30 64 47 71  22 0a 73 65 71 6e 6f 20  |s-O50dGq".seqno |
00001a40  3d 20 33 0a 73 74 61 74  75 73 20 3d 20 5b 22 52  |= 3.status = ["R|
00001a50  45 53 49 5a 45 41 42 4c  45 22 2c 20 22 52 45 41  |ESIZEABLE", "REA|
00001a60  44 22 2c 20 22 57 52 49  54 45 22 5d 0a 66 6c 61  |D", "WRITE"].fla|
00001a70  67 73 20 3d 20 5b 5d 0a  65 78 74 65 6e 74 5f 73  |gs = [].extent_s|
00001a80  69 7a 65 20 3d 20 38 31  39 32 0a 6d 61 78 5f 6c  |ize = 8192.max_l|
00001a90  76 20 3d 20 30 0a 6d 61  78 5f 70 76 20 3d 20 30  |v = 0.max_pv = 0|
00001aa0  0a 6d 65 74 61 64 61 74  61 5f 63 6f 70 69 65 73  |.metadata_copies|
00001ab0  20 3d 20 30 0a 0a 70 68  79 73 69 63 61 6c 5f 76  | = 0..physical_v|
00001ac0  6f 6c 75 6d 65 73 20 7b  0a 0a 70 76 30 20 7b 0a  |olumes {..pv0 {.|
00001ad0  69 64 20 3d 20 22 43 50  76 4e 59 59 2d 32 38 44  |id = "CPvNYY-28D|
00001ae0  6e 2d 4d 39 36 76 2d 6f  72 70 73 2d 31 38 38 32  |n-M96v-orps-1882|
00001af0  2d 49 4f 52 33 2d 66 6d  37 77 48 4f 22 0a 64 65  |-IOR3-fm7wHO".de|
00001b00  76 69 63 65 20 3d 20 22  2f 64 65 76 2f 6d 64 31  |vice = "/dev/md1|
00001b10  32 37 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |27"..status = ["|
00001b20  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001b30  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001b40  65 20 3d 20 37 38 31 34  30 34 31 36 30 30 0a 70  |e = 7814041600.p|
00001b50  65 5f 73 74 61 72 74 20  3d 20 32 30 34 38 0a 70  |e_start = 2048.p|
00001b60  65 5f 63 6f 75 6e 74 20  3d 20 39 35 33 38 36 32  |e_count = 953862|
00001b70  0a 7d 0a 7d 0a 0a 6c 6f  67 69 63 61 6c 5f 76 6f  |.}.}..logical_vo|
00001b80  6c 75 6d 65 73 20 7b 0a  0a 72 6f 6f 74 20 7b 0a  |lumes {..root {.|
00001b90  69 64 20 3d 20 22 39 31  34 6f 75 75 2d 52 49 61  |id = "914ouu-RIa|
00001ba0  6e 2d 35 38 6a 39 2d 48  55 38 78 2d 6c 4c 31 72  |n-58j9-HU8x-lL1r|
00001bb0  2d 73 68 6a 72 2d 42 46  75 70 50 6e 22 0a 73 74  |-shjr-BFupPn".st|
00001bc0  61 74 75 73 20 3d 20 5b  22 52 45 41 44 22 2c 20  |atus = ["READ", |
00001bd0  22 57 52 49 54 45 22 2c  20 22 56 49 53 49 42 4c  |"WRITE", "VISIBL|
00001be0  45 22 5d 0a 66 6c 61 67  73 20 3d 20 5b 5d 0a 73  |E"].flags = [].s|
00001bf0  65 67 6d 65 6e 74 5f 63  6f 75 6e 74 20 3d 20 31  |egment_count = 1|
00001c00  0a 0a 73 65 67 6d 65 6e  74 31 20 7b 0a 73 74 61  |..segment1 {.sta|
00001c10  72 74 5f 65 78 74 65 6e  74 20 3d 20 30 0a 65 78  |rt_extent = 0.ex|
00001c20  74 65 6e 74 5f 63 6f 75  6e 74 20 3d 20 31 32 38  |tent_count = 128|
00001c30  30 0a 0a 74 79 70 65 20  3d 20 22 73 74 72 69 70  |0..type = "strip|
00001c40  65 64 22 0a 73 74 72 69  70 65 5f 63 6f 75 6e 74  |ed".stripe_count|
00001c50  20 3d 20 31 09 23 20 6c  69 6e 65 61 72 0a 0a 73  | = 1.# linear..s|
00001c60  74 72 69 70 65 73 20 3d  20 5b 0a 22 70 76 30 22  |tripes = [."pv0"|
00001c70  2c 20 30 0a 5d 0a 7d 0a  7d 0a 0a 75 73 72 20 7b  |, 0.].}.}..usr {|
00001c80  0a 69 64 20 3d 20 22 61  68 38 43 46 71 2d 72 39  |.id = "ah8CFq-r9|
00001c90  34 36 2d 47 52 78 57 2d  71 31 79 76 2d 76 63 56  |46-GRxW-q1yv-vcV|
00001ca0  48 2d 53 33 34 31 2d 41  6f 4a 6d 42 48 22 0a 73  |H-S341-AoJmBH".s|
00001cb0  74 61 74 75 73 20 3d 20  5b 22 52 45 41 44 22 2c  |tatus = ["READ",|
00001cc0  20 22 57 52 49 54 45 22  2c 20 22 56 49 53 49 42  | "WRITE", "VISIB|
00001cd0  4c 45 22 5d 0a 66 6c 61  67 73 20 3d 20 5b 5d 0a  |LE"].flags = [].|
00001ce0  73 65 67 6d 65 6e 74 5f  63 6f 75 6e 74 20 3d 20  |segment_count = |
00001cf0  31 0a 0a 73 65 67 6d 65  6e 74 31 20 7b 0a 73 74  |1..segment1 {.st|
00001d00  61 72 74 5f 65 78 74 65  6e 74 20 3d 20 30 0a 65  |art_extent = 0.e|
00001d10  78 74 65 6e 74 5f 63 6f  75 6e 74 20 3d 20 31 32  |xtent_count = 12|
00001d20  38 30 0a 0a 74 79 70 65  20 3d 20 22 73 74 72 69  |80..type = "stri|
00001d30  70 65 64 22 0a 73 74 72  69 70 65 5f 63 6f 75 6e  |ped".stripe_coun|
00001d40  74 20 3d 20 31 09 23 20  6c 69 6e 65 61 72 0a 0a  |t = 1.# linear..|
00001d50  73 74 72 69 70 65 73 20  3d 20 5b 0a 22 70 76 30  |stripes = [."pv0|
00001d60  22 2c 20 31 32 38 30 0a  5d 0a 7d 0a 7d 0a 7d 0a  |", 1280.].}.}.}.|
00001d70  7d 0a 23 20 47 65 6e 65  72 61 74 65 64 20 62 79  |}.# Generated by|
00001d80  20 4c 56 4d 32 20 76 65  72 73 69 6f 6e 20 32 2e  | LVM2 version 2.|
00001d90  30 32 2e 37 33 28 32 29  20 28 32 30 31 30 2d 30  |02.73(2) (2010-0|
00001da0  39 2d 31 38 29 3a 20 4d  6f 6e 20 4a 61 6e 20 20  |9-18): Mon Jan  |
00001db0  33 20 31 37 3a 31 32 3a  34 38 20 32 30 31 31 0a  |3 17:12:48 2011.|
00001dc0  0a 63 6f 6e 74 65 6e 74  73 20 3d 20 22 54 65 78  |.contents = "Tex|
00001dd0  74 20 46 6f 72 6d 61 74  20 56 6f 6c 75 6d 65 20  |t Format Volume |
00001de0  47 72 6f 75 70 22 0a 76  65 72 73 69 6f 6e 20 3d  |Group".version =|
00001df0  20 31 0a 0a 64 65 73 63  72 69 70 74 69 6f 6e 20  | 1..description |
00001e00  3d 20 22 22 0a 0a 63 72  65 61 74 69 6f 6e 5f 68  |= ""..creation_h|
00001e10  6f 73 74 20 3d 20 22 73  79 73 72 65 73 63 63 64  |ost = "sysresccd|
00001e20  22 09 23 20 4c 69 6e 75  78 20 73 79 73 72 65 73  |".# Linux sysres|
00001e30  63 63 64 20 32 2e 36 2e  33 35 2d 73 74 64 31 36  |ccd 2.6.35-std16|
00001e40  34 2d 69 33 38 36 20 23  33 20 53 4d 50 20 54 75  |4-i386 #3 SMP Tu|
00001e50  65 20 4e 6f 76 20 33 30  20 32 30 3a 34 32 3a 34  |e Nov 30 20:42:4|
00001e60  36 20 55 54 43 20 32 30  31 30 20 69 36 38 36 0a  |6 UTC 2010 i686.|
00001e70  63 72 65 61 74 69 6f 6e  5f 74 69 6d 65 20 3d 20  |creation_time = |
00001e80  31 32 39 34 30 37 34 37  36 38 09 23 20 4d 6f 6e  |1294074768.# Mon|
00001e90  20 4a 61 6e 20 20 33 20  31 37 3a 31 32 3a 34 38  | Jan  3 17:12:48|
00001ea0  20 32 30 31 31 0a 0a 00  00 00 00 00 00 00 00 00  | 2011...........|
00001eb0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sdc1 bs=512 skip=3072 count=16
2>/dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400  00 00 05 00 00 00 14 00  33 33 00 00 b7 03 0b 00  |........33......|
00000410  ef 14 04 00 00 00 00 00  02 00 00 00 02 00 00 00  |................|
00000420  00 80 00 00 00 80 00 00  00 20 00 00 ae fd 33 52  |......... ....3R|
00000430  5f fd 33 52 01 00 21 00  53 ef 01 00 01 00 00 00  |_.3R..!.S.......|
00000440  5f fd 33 52 00 4e ed 00  00 00 00 00 01 00 00 00  |_.3R.N..........|
00000450  00 00 00 00 0b 00 00 00  00 01 00 00 3c 00 00 00  |............<...|
00000460  06 00 00 00 03 00 00 00  42 50 18 5f 9c 6a 4f af  |........BP._.jO.|
00000470  8c 21 8a 6f d6 1d 6f 38  00 00 00 00 00 00 00 00  |.!.o..o8........|
00000480  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000004c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 3f 01  |..............?.|
000004d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000004e0  08 00 00 00 00 00 00 00  00 00 00 00 f5 23 d7 ab  |.............#..|
000004f0  b4 bf 42 c5 b4 63 7a 88  99 90 78 6e 01 01 00 00  |..B..cz...xn....|
00000500  00 00 00 00 00 00 00 00  29 5b a6 4b 02 82 09 00  |........)[.K....|
00000510  03 82 09 00 04 82 09 00  05 82 09 00 06 82 09 00  |................|
00000520  07 82 09 00 08 82 09 00  09 82 09 00 0a 82 09 00  |................|
00000530  0b 82 09 00 0c 82 09 00  0d 82 09 00 0e 82 09 00  |................|
00000540  0f 86 09 00 00 00 00 00  00 00 00 00 00 00 00 08  |................|
00000550  00 00 00 00 00 00 00 00  00 00 00 00 1c 00 1c 00  |................|
00000560  01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000570  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  41 01 00 00 42 01 00 00  43 01 00 00 fa 22 50 1e  |A...B...C...."P.|
00001010  50 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |P...............|
00001020  41 81 00 00 42 81 00 00  43 81 00 00 b0 53 0c 12  |A...B...C....S..|
00001030  4b 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |K...............|
00001040  00 00 01 00 01 00 01 00  02 00 01 00 e7 22 68 1f  |............."h.|
00001050  07 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001060  41 81 01 00 42 81 01 00  43 81 01 00 e7 3f 1e 12  |A...B...C....?..|
00001070  f9 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001080  00 00 02 00 01 00 02 00  02 00 02 00 25 20 24 1c  |............% $.|
00001090  72 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |r...............|
000010a0  41 81 02 00 42 81 02 00  43 81 02 00 1f 35 1c 12  |A...B...C....5..|
000010b0  40 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |@...............|
000010c0  00 00 03 00 01 00 03 00  02 00 03 00 1c 26 4c 12  |.............&L.|
000010d0  8e 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000010e0  41 81 03 00 42 81 03 00  43 81 03 00 bb 1e 82 14  |A...B...C.......|
000010f0  3f 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |?...............|
00001100  00 00 04 00 01 00 04 00  02 00 04 00 7d 20 56 18  |............} V.|
00001110  22 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |"...............|
00001120  41 81 04 00 42 81 04 00  43 81 04 00 ac 1f f3 15  |A...B...C.......|
00001130  ab 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001140  00 00 05 00 01 00 05 00  02 00 05 00 fe 25 90 12  |.............%..|
00001150  b9 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001160  00 80 05 00 01 80 05 00  02 80 05 00 05 14 d4 19  |................|
00001170  4c 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |L...............|
00001180  00 00 06 00 01 00 06 00  02 00 06 00 bb 1e c2 15  |................|
00001190  67 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |g...............|
000011a0  00 80 06 00 01 80 06 00  02 80 06 00 7d 20 99 16  |............} ..|
000011b0  c5 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000011c0  00 00 07 00 01 00 07 00  02 00 07 00 fd 25 47 15  |.............%G.|
000011d0  05 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000011e0  00 80 07 00 01 80 07 00  02 80 07 00 67 26 e6 18  |............g&..|
000011f0  51 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00001200  00 00 08 00 01 00 08 00  02 00 08 00 c7 44 f3 18  |.............D..|
00001210  08 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001220  00 80 08 00 01 80 08 00  02 80 08 00 fe 7d 00 20  |.............}. |
00001230  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001240  00 00 09 00 01 00 09 00  02 00 09 00 fe 7d 00 20  |.............}. |
00001250  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001260  00 80 09 00 01 80 09 00  02 80 09 00 00 00 00 20  |............... |
00001270  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001280  00 00 0a 00 01 00 0a 00  02 00 0a 00 db 7b 00 20  |.............{. |
00001290  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000012a0  00 80 0a 00 01 80 0a 00  02 80 0a 00 fe 7d 00 20  |.............}. |
000012b0  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000012c0  00 00 0b 00 01 00 0b 00  02 00 0b 00 fd 7d fd 1f  |.............}..|
000012d0  01 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000012e0  00 80 0b 00 01 80 0b 00  02 80 0b 00 fe 7d 00 20  |.............}. |
000012f0  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001300  00 00 0c 00 01 00 0c 00  02 00 0c 00 43 7b 28 1f  |............C{(.|
00001310  4d 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |M...............|
00001320  41 81 0c 00 42 81 0c 00  43 81 0c 00 bd 7c 00 20  |A...B...C....|. |
00001330  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001340  00 00 0d 00 01 00 0d 00  02 00 0d 00 1e 75 6e 1f  |.............un.|
00001350  01 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001360  41 81 0d 00 42 81 0d 00  43 81 0d 00 bd 7c 00 20  |A...B...C....|. |
00001370  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001380  00 00 0e 00 01 00 0e 00  02 00 0e 00 fe 7d 00 20  |.............}. |
00001390  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000013a0  00 80 0e 00 01 80 0e 00  02 80 0e 00 fe 7d 00 20  |.............}. |
000013b0  00 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000013c0  00 00 0f 00 01 00 0f 00  02 00 0f 00 fb 7d fb 1f  |.............}..|
000013d0  03 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000013e0  00 80 0f 00 01 80 0f 00  02 80 0f 00 fd 7d ff 1f  |.............}..|
000013f0  01 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001400  00 00 10 00 01 00 10 00  02 00 10 00 14 27 d3 15  |.............'..|
00001410  4c 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |L...............|
00001420  00 80 10 00 01 80 10 00  02 80 10 00 de 14 7b 15  |..............{.|
00001430  bb 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001440  00 00 11 00 01 00 11 00  02 00 11 00 75 20 75 16  |............u u.|
00001450  97 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001460  00 80 11 00 01 80 11 00  02 80 11 00 e0 23 73 16  |.............#s.|
00001470  85 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001480  00 00 12 00 01 00 12 00  02 00 12 00 dd 23 62 14  |.............#b.|
00001490  c3 01 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000014a0  00 80 12 00 01 80 12 00  02 80 12 00 72 53 7c 12  |............rS|.|
000014b0  22 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |"...............|
000014c0  00 00 13 00 01 00 13 00  02 00 13 00 4a 22 bb 1f  |............J"..|
000014d0  11 00 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000014e0  00 80 13 00 01 80 13 00  02 80 13 00 16 58 7b 12  |.............X{.|
000014f0  17 02 04 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00001500  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sdd1 bs=512 skip=2048 count=16
2>/dev/null | hexdump -C
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root % dd if=/dev/sdd1 bs=512 skip=3072 count=16
2>/dev/null | hexdump -C
00000000  80 80 00 00 80 80 01 00  80 80 02 00 80 80 03 00  |................|
00000010  80 80 04 00 80 80 0c 00  80 80 0d 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  81 80 00 00 81 80 01 00  81 80 02 00 81 80 03 00  |................|
00001010  81 80 04 00 81 80 0c 00  81 80 0d 00 00 00 00 00  |................|
00001020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000
root@sysresccd /root %

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  3:34                       ` Dave Gomboc
@ 2013-09-23  3:51                         ` Phil Turmel
  2013-09-23  4:04                           ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-23  3:51 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 09/22/2013 11:34 PM, Dave Gomboc wrote:
>> If you'll generate the following for each member, we'll see what we can do:
>>
>> dd if=/dev/sdX1 bs=512 skip=2048 count=16 2>/dev/null |hexdump -C
>> dd if=/dev/sdX1 bs=512 skip=3072 count=16 2>/dev/null |hexdump -C

Ok.  I can't determine how the superblocks ended up the way they did,
but the first two chunks appear to follow the proper patterns.

I think you're best bet is to disconnect two of the drives, leaving one
that identifies as "0" and one that identifies as "3".

Then use "mdadm -Af /dev/mdX /dev/sdY1 /dev/sdZ1"

The "-f" will force the assembly without regard to the event counts.
Then you can take a backup.  Finally you can add devices as "new" ones
to rebuild back to full redundancy.  (Fix your timeouts before
attempting the latter.)

Phil

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  3:51                         ` Phil Turmel
@ 2013-09-23  4:04                           ` Dave Gomboc
  2013-09-23  4:12                             ` Phil Turmel
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-23  4:04 UTC (permalink / raw)
  To: Phil Turmel; +Cc: linux-raid

> Ok.  I can't determine how the superblocks ended up the way they did,
> but the first two chunks appear to follow the proper patterns.
>
> I think you're best bet is to disconnect two of the drives, leaving one
> that identifies as "0" and one that identifies as "3".

root@sysresccd /root % ls -l /dev/disk/by-id | grep Hitachi | grep -v part1
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J -> ../../sdb
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ -> ../../sdc
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J -> ../../sdd
lrwxrwxrwx 1 root root  9 Sep 22 20:17
ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS -> ../../sda
root@sysresccd /root % cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md127 : inactive sdd1[3](S) sda1[0](S)
      3907021954 blocks super 1.2

unused devices: <none>

Should I be disconnecting sdb and sdc, disconnecting sda and sdd, or
does it matter?

I should reboot using the rescue disk before attempting the forced
assembly, not my boot drive, right?

Sorry if the answers to these questions seem obvious to you: I want to
make sure that I understand you exactly.  I am moderately terrified at
the moment.

> Then use "mdadm -Af /dev/mdX /dev/sdY1 /dev/sdZ1"
>
> The "-f" will force the assembly without regard to the event counts.
> Then you can take a backup.  Finally you can add devices as "new" ones
> to rebuild back to full redundancy.  (Fix your timeouts before
> attempting the latter.)

When following up on your advice to search for those other terms, I
saw several examples where people specified 7 seconds to the disk
drive using that control program, and also read somewhere that while
Linux's software raid will wait, that Linux's scsi subsystem has a 30
second timeout.  So, 7 seconds sounds good?

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  4:04                           ` Dave Gomboc
@ 2013-09-23  4:12                             ` Phil Turmel
  2013-09-23  4:55                               ` Dave Gomboc
  0 siblings, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-23  4:12 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 09/23/2013 12:04 AM, Dave Gomboc wrote:
>> Ok.  I can't determine how the superblocks ended up the way they did,
>> but the first two chunks appear to follow the proper patterns.
>>
>> I think you're best bet is to disconnect two of the drives, leaving one
>> that identifies as "0" and one that identifies as "3".
> 
> root@sysresccd /root % ls -l /dev/disk/by-id | grep Hitachi | grep -v part1
> lrwxrwxrwx 1 root root  9 Sep 22 20:17
> ata-Hitachi_HDS724040ALE640_PK1310PAG5ZY0J -> ../../sdb
> lrwxrwxrwx 1 root root  9 Sep 22 20:17
> ata-Hitachi_HDS724040ALE640_PK1310PAG62REJ -> ../../sdc
> lrwxrwxrwx 1 root root  9 Sep 22 20:17
> ata-Hitachi_HDS724040ALE640_PK1310PAG62T2J -> ../../sdd
> lrwxrwxrwx 1 root root  9 Sep 22 20:17
> ata-Hitachi_HDS724040ALE640_PK1311PAG4W5TS -> ../../sda
> root@sysresccd /root % cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md127 : inactive sdd1[3](S) sda1[0](S)
>       3907021954 blocks super 1.2
> 
> unused devices: <none>
> 
> Should I be disconnecting sdb and sdc, disconnecting sda and sdd, or
> does it matter?

Actually, from that report, just do "mdadm /dev/md127 --run".

> I should reboot using the rescue disk before attempting the forced
> assembly, not my boot drive, right?

Yes, but only necessary if the above fails.  And if there's a partial
assembly, you might need to use "mdadm --stop".

> Sorry if the answers to these questions seem obvious to you: I want to
> make sure that I understand you exactly.  I am moderately terrified at
> the moment.

You have duplicated the disks.  You have all of the insurance possible.

>> Then use "mdadm -Af /dev/mdX /dev/sdY1 /dev/sdZ1"
>>
>> The "-f" will force the assembly without regard to the event counts.
>> Then you can take a backup.  Finally you can add devices as "new" ones
>> to rebuild back to full redundancy.  (Fix your timeouts before
>> attempting the latter.)
> 
> When following up on your advice to search for those other terms, I
> saw several examples where people specified 7 seconds to the disk
> drive using that control program, and also read somewhere that while
> Linux's software raid will wait, that Linux's scsi subsystem has a 30
> second timeout.  So, 7 seconds sounds good?

Most traditional enterprise drives power up with t=7 seconds.  The SSDs
I've used use t=4 seconds.

Keep in mind that the setting is forgotten when the drive powers down.
You need the commands in rc.local or your distro's equivalent.

Phil


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  4:12                             ` Phil Turmel
@ 2013-09-23  4:55                               ` Dave Gomboc
  2013-09-23  5:07                                 ` Dave Gomboc
                                                   ` (2 more replies)
  0 siblings, 3 replies; 25+ messages in thread
From: Dave Gomboc @ 2013-09-23  4:55 UTC (permalink / raw)
  To: Phil Turmel; +Cc: linux-raid

>> Should I be disconnecting sdb and sdc, disconnecting sda and sdd, or
>> does it matter?
>
> Actually, from that report, just do "mdadm /dev/md127 --run".

This didn't seem to work for me.  A subsequent mdadm --stop did affect
/proc/mdstat, though.

I then went on to verify which drive was providing what using mdadm --examine.

Rebooting with just one of each drive available, mdadm /dev/md127
--run again didn't work.  I then went to your earlier advice:

>>> Then use "mdadm -Af /dev/mdX /dev/sdY1 /dev/sdZ1"
>>>
>>> The "-f" will force the assembly without regard to the event counts.
>>> Then you can take a backup.  Finally you can add devices as "new" ones
>>> to rebuild back to full redundancy.  (Fix your timeouts before
>>> attempting the latter.)

The drives are now sda and sdb (instead of sda and sdd).

root@sysresccd /root % mdadm -Af /dev/md127 /dev/sda1 /dev/sdb1
mdadm: forcing event count in /dev/sda1(0) from 520 upto 599
mdadm: /dev/md127 has been started with 2 drives (out of 4).

The partition information seems to be visible:

root@sysresccd /root % cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md127 : active raid10 sda1[0] sdb1[3]
      3907020800 blocks super 1.2 512K chunks 2 near-copies [4/2] [U__U]

unused devices: <none>
root@sysresccd /root % pvdisplay
  --- Physical volume ---
  PV Name               /dev/md127
  VG Name               teramooch
  PV Size               3.64 TiB / not usable 2.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              953862
  Free PE               0
  Allocated PE          953862
  PV UUID               CPvNYY-28Dn-M96v-orps-1882-IOR3-fm7wHO

root@sysresccd /root % vgdisplay
  --- Volume group ---
  VG Name               teramooch
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  26
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                8
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               3.64 TiB
  PE Size               4.00 MiB
  Total PE              953862
  Alloc PE / Size       953862 / 3.64 TiB
  Free  PE / Size       0 / 0
  VG UUID               j4Svly-X1uO-OVL1-cfCd-eJ4s-sEQs-O50dGq

At first, lvdisplay showed the logical volumes as not available, so I
then invoked

root@sysresccd /root % vgchange -a y
  8 logical volume(s) in volume group "teramooch" now active
root@sysresccd /root % lvdisplay
  --- Logical volume ---
  LV Path                /dev/teramooch/root
  LV Name                root
  VG Name                teramooch
  LV UUID                914ouu-RIan-58j9-HU8x-lL1r-shjr-BFupPn
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                5.00 GiB
  Current LE             1280
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:0

  --- Logical volume ---
  LV Path                /dev/teramooch/usr
  LV Name                usr
  VG Name                teramooch
  LV UUID                ah8CFq-r946-GRxW-q1yv-vcVH-S341-AoJmBH
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                5.00 GiB
  Current LE             1280
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:1

  --- Logical volume ---
  LV Path                /dev/teramooch/var
  LV Name                var
  VG Name                teramooch
  LV UUID                eybfZ0-Vy8N-P5XG-S0C1-dXzw-gaCG-TCry0d
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                5.00 GiB
  Current LE             1280
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:2

  --- Logical volume ---
  LV Path                /dev/teramooch/tmp
  LV Name                tmp
  VG Name                teramooch
  LV UUID                2SSk0A-Mv2b-gI5X-37TE-1zoX-9afj-wUbvkT
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                1.00 GiB
  Current LE             256
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:3

  --- Logical volume ---
  LV Path                /dev/teramooch/opt
  LV Name                opt
  VG Name                teramooch
  LV UUID                0HN60s-NOr9-qQ5m-Kgds-Y7i0-jDHl-RlpEhg
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                1.00 GiB
  Current LE             256
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:4

  --- Logical volume ---
  LV Path                /dev/teramooch/home
  LV Name                home
  VG Name                teramooch
  LV UUID                j1Vtft-WRg8-yBIM-fhjb-6EP4-Df56-4lTHz7
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                5.00 GiB
  Current LE             1280
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:5

  --- Logical volume ---
  LV Path                /dev/teramooch/srv
  LV Name                srv
  VG Name                teramooch
  LV UUID                L3n5AZ-XgTa-7uV8-GzCG-82oX-JqrJ-VugwT6
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                2.91 TiB
  Current LE             762123
  Segments               7
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:6

  --- Logical volume ---
  LV Path                /dev/teramooch/backupduffycapemay
  LV Name                backupduffycapemay
  VG Name                teramooch
  LV UUID                83RVKO-txsd-q1oE-2p9n-z0dk-zIQe-Sx3Eol
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              available
  # open                 0
  LV Size                726.98 GiB
  Current LE             186107
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:7

I made /mnt/root, /mnt/home, etc., then

[Actually, I mounted these rw first, then realized that I shouldn't
have, unmounted them, and re-mounted them read-only.]

root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-root /mnt/root
root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-home /mnt/home
root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-tmp /mnt/tmp
root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-usr /mnt/usr
root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-var /mnt/var
root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-opt /mnt/opt

These mount attempts returned without error, and ls within the
directories is possible.

However, the large, important one doesn't mount:

root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-srv /mnt/srv
mount: wrong fs type, bad option, bad superblock on /dev/mapper/teramooch-srv,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so

root@sysresccd /mnt % mount -o ro /dev/mapper/teramooch-srv /mnt/srv
NTFS signature is missing.
Failed to mount '/dev/mapper/teramooch-srv': Invalid argument
The device '/dev/mapper/teramooch-srv' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

It should be ext3, not NTFS.

Yikes.

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  4:55                               ` Dave Gomboc
@ 2013-09-23  5:07                                 ` Dave Gomboc
  2013-09-23  5:19                                 ` Adam Goryachev
  2013-09-23 12:32                                 ` Phil Turmel
  2 siblings, 0 replies; 25+ messages in thread
From: Dave Gomboc @ 2013-09-23  5:07 UTC (permalink / raw)
  To: Phil Turmel; +Cc: linux-raid

[-- Attachment #1: Type: text/plain, Size: 4161 bytes --]

I can get at that /etc/lvm area now (it's part of root).  I pasted the
contents of /etc/lvm/backup/teramooch here, and I have also attached
it to this email.

Dave

root@sysresccd /root % cat /mnt/root/etc/lvm/backup/teramooch
# Generated by LVM2 version 2.02.66(2) (2010-05-20): Sat Aug  4 21:25:35 2012

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'vgreduce teramooch /dev/sda'"

creation_host = "cheap" # Linux cheap 3.2.0-26-generic #41-Ubuntu SMP
Thu Jun 14 17:49:24 UTC 2012 x86_64
creation_time = 1344140735 # Sat Aug  4 21:25:35 2012

teramooch {
id = "j4Svly-X1uO-OVL1-cfCd-eJ4s-sEQs-O50dGq"
seqno = 26
status = ["RESIZEABLE", "READ", "WRITE"]
flags = []
extent_size = 8192 # 4 Megabytes
max_lv = 0
max_pv = 0

physical_volumes {

pv0 {
id = "CPvNYY-28Dn-M96v-orps-1882-IOR3-fm7wHO"
device = "/dev/md0" # Hint only

status = ["ALLOCATABLE"]
flags = []
dev_size = 7814041600 # 3.6387 Terabytes
pe_start = 2048
pe_count = 953862 # 3.63869 Terabytes
}
}

logical_volumes {

root {
id = "914ouu-RIan-58j9-HU8x-lL1r-shjr-BFupPn"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 1280 # 5 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 0
]
}
}

usr {
id = "ah8CFq-r946-GRxW-q1yv-vcVH-S341-AoJmBH"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 1280 # 5 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 1280
]
}
}

var {
id = "eybfZ0-Vy8N-P5XG-S0C1-dXzw-gaCG-TCry0d"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 1280 # 5 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 2560
]
}
}

tmp {
id = "2SSk0A-Mv2b-gI5X-37TE-1zoX-9afj-wUbvkT"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 256 # 1024 Megabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 3840
]
}
}

opt {
id = "0HN60s-NOr9-qQ5m-Kgds-Y7i0-jDHl-RlpEhg"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 256 # 1024 Megabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 5120
]
}
}

home {
id = "j1Vtft-WRg8-yBIM-fhjb-6EP4-Df56-4lTHz7"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 1

segment1 {
start_extent = 0
extent_count = 1280 # 5 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 6400
]
}
}

srv {
id = "L3n5AZ-XgTa-7uV8-GzCG-82oX-JqrJ-VugwT6"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 7

segment1 {
start_extent = 0
extent_count = 393216 # 1.5 Terabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 268544
]
}
segment2 {
start_extent = 393216
extent_count = 112902 # 441.023 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 840960
]
}
segment3 {
start_extent = 506118
extent_count = 97280 # 380 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 743680
]
}
segment4 {
start_extent = 603398
extent_count = 80640 # 315 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 187904
]
}
segment5 {
start_extent = 684038
extent_count = 1024 # 4 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 4096
]
}
segment6 {
start_extent = 685062
extent_count = 1024 # 4 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 5376
]
}
segment7 {
start_extent = 686086
extent_count = 76037 # 297.02 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 111867
]
}
}

backupduffycapemay {
id = "83RVKO-txsd-q1oE-2p9n-z0dk-zIQe-Sx3Eol"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
segment_count = 2

segment1 {
start_extent = 0
extent_count = 81920 # 320 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 661760
]
}
segment2 {
start_extent = 81920
extent_count = 104187 # 406.98 Gigabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv0", 7680
]
}
}
}
}
root@sysresccd /root %

[-- Attachment #2: teramooch --]
[-- Type: application/octet-stream, Size: 4601 bytes --]

# Generated by LVM2 version 2.02.66(2) (2010-05-20): Sat Aug  4 21:25:35 2012

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'vgreduce teramooch /dev/sda'"

creation_host = "cheap"	# Linux cheap 3.2.0-26-generic #41-Ubuntu SMP Thu Jun 14 17:49:24 UTC 2012 x86_64
creation_time = 1344140735	# Sat Aug  4 21:25:35 2012

teramooch {
	id = "j4Svly-X1uO-OVL1-cfCd-eJ4s-sEQs-O50dGq"
	seqno = 26
	status = ["RESIZEABLE", "READ", "WRITE"]
	flags = []
	extent_size = 8192		# 4 Megabytes
	max_lv = 0
	max_pv = 0

	physical_volumes {

		pv0 {
			id = "CPvNYY-28Dn-M96v-orps-1882-IOR3-fm7wHO"
			device = "/dev/md0"	# Hint only

			status = ["ALLOCATABLE"]
			flags = []
			dev_size = 7814041600	# 3.6387 Terabytes
			pe_start = 2048
			pe_count = 953862	# 3.63869 Terabytes
		}
	}

	logical_volumes {

		root {
			id = "914ouu-RIan-58j9-HU8x-lL1r-shjr-BFupPn"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1280	# 5 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 0
				]
			}
		}

		usr {
			id = "ah8CFq-r946-GRxW-q1yv-vcVH-S341-AoJmBH"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1280	# 5 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 1280
				]
			}
		}

		var {
			id = "eybfZ0-Vy8N-P5XG-S0C1-dXzw-gaCG-TCry0d"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1280	# 5 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 2560
				]
			}
		}

		tmp {
			id = "2SSk0A-Mv2b-gI5X-37TE-1zoX-9afj-wUbvkT"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 256	# 1024 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 3840
				]
			}
		}

		opt {
			id = "0HN60s-NOr9-qQ5m-Kgds-Y7i0-jDHl-RlpEhg"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 256	# 1024 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 5120
				]
			}
		}

		home {
			id = "j1Vtft-WRg8-yBIM-fhjb-6EP4-Df56-4lTHz7"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1280	# 5 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 6400
				]
			}
		}

		srv {
			id = "L3n5AZ-XgTa-7uV8-GzCG-82oX-JqrJ-VugwT6"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 7

			segment1 {
				start_extent = 0
				extent_count = 393216	# 1.5 Terabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 268544
				]
			}
			segment2 {
				start_extent = 393216
				extent_count = 112902	# 441.023 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 840960
				]
			}
			segment3 {
				start_extent = 506118
				extent_count = 97280	# 380 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 743680
				]
			}
			segment4 {
				start_extent = 603398
				extent_count = 80640	# 315 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 187904
				]
			}
			segment5 {
				start_extent = 684038
				extent_count = 1024	# 4 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 4096
				]
			}
			segment6 {
				start_extent = 685062
				extent_count = 1024	# 4 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 5376
				]
			}
			segment7 {
				start_extent = 686086
				extent_count = 76037	# 297.02 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 111867
				]
			}
		}

		backupduffycapemay {
			id = "83RVKO-txsd-q1oE-2p9n-z0dk-zIQe-Sx3Eol"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 2

			segment1 {
				start_extent = 0
				extent_count = 81920	# 320 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 661760
				]
			}
			segment2 {
				start_extent = 81920
				extent_count = 104187	# 406.98 Gigabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 7680
				]
			}
		}
	}
}

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  4:55                               ` Dave Gomboc
  2013-09-23  5:07                                 ` Dave Gomboc
@ 2013-09-23  5:19                                 ` Adam Goryachev
  2013-09-23 12:32                                 ` Phil Turmel
  2 siblings, 0 replies; 25+ messages in thread
From: Adam Goryachev @ 2013-09-23  5:19 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 23/09/13 14:55, Dave Gomboc wrote:
> root@sysresccd /mnt % mount -o ro /dev/mapper/teramooch-srv /mnt/srv
> NTFS signature is missing.
> Failed to mount '/dev/mapper/teramooch-srv': Invalid argument
> The device '/dev/mapper/teramooch-srv' doesn't seem to have a valid NTFS.
> Maybe the wrong device is used? Or the whole disk instead of a
> partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
>
> It should be ext3, not NTFS.

Try:
mount -o ro -t ext3 /dev/mapper/teramooch-srv /mnt/srv
This just specifies it is ext3 format, but will probably still fail... 
check dmesg for details, but hopefully the next step will fix it.


If that fails, take a look at:
fsck.ext3 -n /dev/mapper/teramooch-srv
(Which should do a read-only check, if there are not too many changes, 
re-run with -y). Worst case, if the fsck destroys the data, you can 
re-copy the entire array from the original and try again.

Hope that helps...

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23  4:55                               ` Dave Gomboc
  2013-09-23  5:07                                 ` Dave Gomboc
  2013-09-23  5:19                                 ` Adam Goryachev
@ 2013-09-23 12:32                                 ` Phil Turmel
  2013-09-23 12:57                                   ` Dave Gomboc
  2 siblings, 1 reply; 25+ messages in thread
From: Phil Turmel @ 2013-09-23 12:32 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

Good morning Dave,

On 09/23/2013 12:55 AM, Dave Gomboc wrote:

[trim /]

Good news.

> [Actually, I mounted these rw first, then realized that I shouldn't
> have, unmounted them, and re-mounted them read-only.]
> 
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-root /mnt/root
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-home /mnt/home
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-tmp /mnt/tmp
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-usr /mnt/usr
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-var /mnt/var
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-opt /mnt/opt
> 
> These mount attempts returned without error, and ls within the
> directories is possible.
> 
> However, the large, important one doesn't mount:
> 
> root@sysresccd /mnt % mount -o ro -t ext3 /dev/mapper/teramooch-srv /mnt/srv
> mount: wrong fs type, bad option, bad superblock on /dev/mapper/teramooch-srv,
>        missing codepage or helper program, or other error
>        In some cases useful info is found in syslog - try
>        dmesg | tail or so
> 
> root@sysresccd /mnt % mount -o ro /dev/mapper/teramooch-srv /mnt/srv
> NTFS signature is missing.
> Failed to mount '/dev/mapper/teramooch-srv': Invalid argument
> The device '/dev/mapper/teramooch-srv' doesn't seem to have a valid NTFS.
> Maybe the wrong device is used? Or the whole disk instead of a
> partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
> 
> It should be ext3, not NTFS.

Since you are sure it is ext3, then you should use "fsck.ext3 -y" to fix
it.  I usually try that operation first with "-n" instead of "-y", but
your mount attempts show that the "-n" would be superfluous.  If fsck
scrambles that volume worse, then you'll have to recopy from your
backups and redo the forced assembly.  Or try with the other two disks.

Phil


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23 12:32                                 ` Phil Turmel
@ 2013-09-23 12:57                                   ` Dave Gomboc
  2013-09-24  0:29                                     ` Adam Goryachev
  2013-09-28 15:47                                     ` Dave Gomboc
  0 siblings, 2 replies; 25+ messages in thread
From: Dave Gomboc @ 2013-09-23 12:57 UTC (permalink / raw)
  To: Phil Turmel; +Cc: linux-raid, mailinglists

> Good morning Dave,

Good morning!

>> It should be ext3, not NTFS.
>
> Since you are sure it is ext3, then you should use "fsck.ext3 -y" to fix
> it.  I usually try that operation first with "-n" instead of "-y", but
> your mount attempts show that the "-n" would be superfluous.  If fsck
> scrambles that volume worse, then you'll have to recopy from your
> backups and redo the forced assembly.  Or try with the other two disks.

I'm pretty darn sure it's ext3.  I guess there is a small chance that
all of them were upgraded to ext4 from ext3.

I saw Adam's post before yours, so I did first try various
command-lines using -n:

fsck.ext3 -n /dev/mapper/teramooch-srv
e2fsck 1.42.7 (21-Jan-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext3: Superblock invalid, trying backup blocks...
fsck.ext3: Bad magic number in super-block while trying to open
/dev/mapper/teramooch-srv

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

[man fsck.ext3 mentions a backup superblock possibly being located at
8193, 16384, or 32768].

root@sysresccd /root % fsck.ext3 -n -b 8193 /dev/mapper/teramooch-srv
e2fsck 1.42.7 (21-Jan-2013)
fsck.ext3: Bad magic number in super-block while trying to open
/dev/mapper/teramooch-srv

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

root@sysresccd /root % fsck.ext3 -n -b 16384 /dev/mapper/teramooch-srv
e2fsck 1.42.7 (21-Jan-2013)
fsck.ext3: Bad magic number in super-block while trying to open
/dev/mapper/teramooch-srv

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

root@sysresccd /root % fsck.ext3 -n -b 32768 /dev/mapper/teramooch-srv
e2fsck 1.42.7 (21-Jan-2013)
fsck.ext3: Bad magic number in super-block while trying to open
/dev/mapper/teramooch-srv

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

[The same commands, except using fsck.ext4 instead of fsck.ext3, also
gave the same generic output.  Finally, I tried the -y:]

root@sysresccd /root % fsck.ext3 -y /dev/mapper/teramooch-srv
e2fsck 1.42.7 (21-Jan-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext3: Superblock invalid, trying backup blocks...
fsck.ext3: Bad magic number in super-block while trying to open
/dev/mapper/teramooch-srv

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

The above output was also returned instantaneously, despite the use of -y.

I have to head to work, but I would be happy to make further attempts
when I'm home again.

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23 12:57                                   ` Dave Gomboc
@ 2013-09-24  0:29                                     ` Adam Goryachev
  2013-09-24  5:55                                       ` Dave Gomboc
  2013-09-28 15:47                                     ` Dave Gomboc
  1 sibling, 1 reply; 25+ messages in thread
From: Adam Goryachev @ 2013-09-24  0:29 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid

On 23/09/13 22:57, Dave Gomboc wrote:
> It should be ext3, not NTFS.
>
> I'm pretty darn sure it's ext3.  I guess there is a small chance that
> all of them were upgraded to ext4 from ext3.

I think that wouldn't matter, AFAIK, ext4 is backward compatible...

> I saw Adam's post before yours, so I did first try various
> command-lines using -n:
>
> fsck.ext3 -n /dev/mapper/teramooch-srv
> e2fsck 1.42.7 (21-Jan-2013)
> ext2fs_open2: Bad magic number in super-block
> fsck.ext3: Superblock invalid, trying backup blocks...
> fsck.ext3: Bad magic number in super-block while trying to open
> /dev/mapper/teramooch-srv
>
> The superblock could not be read or does not describe a correct ext2
> filesystem.  If the device is valid and it really contains an ext2
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt, and you might try running e2fsck with an alternate superblock:
>      e2fsck -b 8193 <device>
>
> [man fsck.ext3 mentions a backup superblock possibly being located at
> 8193, 16384, or 32768].
I strongly suspect that this is not an ext2/3/4 filesystem....

> [The same commands, except using fsck.ext4 instead of fsck.ext3, also
> gave the same generic output.  Finally, I tried the -y:]
>
> The above output was also returned instantaneously, despite the use of -y.
If -n can't find it, -y won't find it either.
> I have to head to work, but I would be happy to make further attempts
> when I'm home again.

I'd suggest you take a look at the raw LV to try and determine what 
format it should be. You should be able to check /etc/fstab if it was 
auto-mounted previously...

In any case, one method I've used before is:
strings /dev/vg/lv | less

Just look for anything that might be a description of the FS format.

Can also try:
file /dev/vg/lv
which can give some hints sometimes (or might just say "data")

Or even:
dd if=/dev/vg/lv of=/tmp/begin bs=1024 count=512
less /tmp/begin
Again, looking for some text which might describe what format this thing 
is in.

It seems strange that this LV would be totally corrupted while the other 
LV's are perfect, but that is another possibility (however unlikely). 
Again, looking at the raw drive should show some file contents (if you 
look through enough of it), and this might also provide some help in 
working out what the problem is. I once assembled my raid0 (striping) in 
the wrong order, and looking at the individual member drives raw 
contents helped me work out the correct order.

I hope the above helps. For now, it sounds like your FS format is not 
ext2/3/4 based. Maybe another one of the popular linux formats?

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-24  0:29                                     ` Adam Goryachev
@ 2013-09-24  5:55                                       ` Dave Gomboc
  0 siblings, 0 replies; 25+ messages in thread
From: Dave Gomboc @ 2013-09-24  5:55 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: linux-raid

> I strongly suspect that this is not an ext2/3/4 filesystem....

The recovered /etc/fstab mounts it as an ext3 filesystem:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/mapper/teramooch-root /               ext3
relatime,errors=remount-ro 0       1
UUID=20be44f0-4da3-439e-ae4e-cc4e08cc8e37 /boot           ext2
relatime        0       2
UUID=B627-AFF0  /fat32          vfat    utf8,umask=007,gid=46 0       1
/dev/mapper/teramooch-home /home           ext3    relatime        0       2
/dev/mapper/teramooch-opt /opt            ext3    relatime        0       2
/dev/mapper/teramooch-srv /srv            ext3
relatime,acl,user_xattr        0       2
/dev/mapper/teramooch-tmp /tmp            ext3    relatime        0       2
/dev/mapper/teramooch-usr /usr            ext3    relatime        0       2
/dev/mapper/teramooch-var /var            ext3    relatime        0       2
/dev/mapper/teramooch-backupduffycapemay
/srv/samba/mac_backups/duffycapemay            ext3    relatime
0       2
UUID=7fb288ca-400a-415a-97b7-35ceccddeb0f none            swap    sw
           0       0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0

> It seems strange that this LV would be totally corrupted while the other
> LV's are perfect, but that is another possibility (however unlikely). Again,
> looking at the raw drive should show some file contents (if you look through
> enough of it), and this might also provide some help in working out what the
> problem is. I once assembled my raid0 (striping) in the wrong order, and
> looking at the individual member drives raw contents helped me work out the
> correct order.
>
> I hope the above helps. For now, it sounds like your FS format is not
> ext2/3/4 based. Maybe another one of the popular linux formats?

The /dev/mapper/teramooch-srv logical volume does represent a huge
majority of the disk space.

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-23 12:57                                   ` Dave Gomboc
  2013-09-24  0:29                                     ` Adam Goryachev
@ 2013-09-28 15:47                                     ` Dave Gomboc
  2013-09-28 16:01                                       ` Phil Turmel
  1 sibling, 1 reply; 25+ messages in thread
From: Dave Gomboc @ 2013-09-28 15:47 UTC (permalink / raw)
  To: Phil Turmel, linux-raid; +Cc: Adam Goryachev

> fsck.ext3 -n /dev/mapper/teramooch-srv
> e2fsck 1.42.7 (21-Jan-2013)
> ext2fs_open2: Bad magic number in super-block
> fsck.ext3: Superblock invalid, trying backup blocks...
> fsck.ext3: Bad magic number in super-block while trying to open
> /dev/mapper/teramooch-srv

So, I am now restoring the content of the two drives that I altered
(by running ddrescue again) , so that I have all four drives copied as
before.

If each pair of two drives genuinely have the same information on
them, then I only seem to have that one method of array assembly
available to me, in which case I suppose I will need to try to scan at
a lower level, searching for parts of the ext3 filesystem of the
logical volume that I cannot mount.  I could certainly use some advice
on what might be worth trying in this scenario.  Even if I can't get
everything back, I should try to get anything back that I can.

However, if there is some variation between what two drives that
report place '0' (or the two drives that report place '3') are
actually holding, then I have more array assembly options to try
before dealing with the above issue.

Is there already some software out there (or do you think it would not
be particularly difficult for an experienced software developer who
does not have extensive knowledge of linux internals -- such as myself
:-) that could be used to treat the physical drives as read-only, and
do any "writes" to them to a layer in RAM instead?  That is, it would
read from RAM first, then from the disk only if nothing was present in
RAM, and always leave the contents of the disks unchanged when writing
back to RAM (never writing through to disk)?  If I had something like
that, I could make various attempts, then discard the changes easily
after failed tries, without having to restore the hard drive contents
from the actual original drive afterwards  Of course, I don't have
several terabytes of RAM, but I think there is really only a small
portion of the disk that is being edited when I am doing various
operations such as mdadm --assemble and vgchange.

Dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: raid10 recovery assistance requested
  2013-09-28 15:47                                     ` Dave Gomboc
@ 2013-09-28 16:01                                       ` Phil Turmel
  0 siblings, 0 replies; 25+ messages in thread
From: Phil Turmel @ 2013-09-28 16:01 UTC (permalink / raw)
  To: Dave Gomboc; +Cc: linux-raid, Adam Goryachev

Hi Dave,

On 09/28/2013 11:47 AM, Dave Gomboc wrote:
>> fsck.ext3 -n /dev/mapper/teramooch-srv
>> e2fsck 1.42.7 (21-Jan-2013)
>> ext2fs_open2: Bad magic number in super-block
>> fsck.ext3: Superblock invalid, trying backup blocks...
>> fsck.ext3: Bad magic number in super-block while trying to open
>> /dev/mapper/teramooch-srv
> 
> So, I am now restoring the content of the two drives that I altered
> (by running ddrescue again) , so that I have all four drives copied as
> before.
> 
> If each pair of two drives genuinely have the same information on
> them, then I only seem to have that one method of array assembly
> available to me, in which case I suppose I will need to try to scan at
> a lower level, searching for parts of the ext3 filesystem of the
> logical volume that I cannot mount.  I could certainly use some advice
> on what might be worth trying in this scenario.  Even if I can't get
> everything back, I should try to get anything back that I can.
> 
> However, if there is some variation between what two drives that
> report place '0' (or the two drives that report place '3') are
> actually holding, then I have more array assembly options to try
> before dealing with the above issue.
> 
> Is there already some software out there (or do you think it would not
> be particularly difficult for an experienced software developer who
> does not have extensive knowledge of linux internals -- such as myself
> :-) that could be used to treat the physical drives as read-only, and
> do any "writes" to them to a layer in RAM instead?  That is, it would
> read from RAM first, then from the disk only if nothing was present in
> RAM, and always leave the contents of the disks unchanged when writing
> back to RAM (never writing through to disk)?  If I had something like
> that, I could make various attempts, then discard the changes easily
> after failed tries, without having to restore the hard drive contents
> from the actual original drive afterwards  Of course, I don't have
> several terabytes of RAM, but I think there is really only a small
> portion of the disk that is being edited when I am doing various
> operations such as mdadm --assemble and vgchange.

I've never had to do this, but I would use the device mapper "snapshot"
target device.  You could use it with a ram device, but I'd use another
partition in persistent mode.  That way, once you succeed, you can merge
the result back into the real devices.  Start with the device mapper
documentation in the kernel tree.

Phil


^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2013-09-28 16:01 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-16  3:30 raid10 recovery assistance requested Dave Gomboc
2013-09-19  3:20 ` Dave Gomboc
2013-09-19  4:29   ` Stan Hoeppner
2013-09-20  5:29     ` Dave Gomboc
2013-09-22 17:15       ` Dave Gomboc
2013-09-22 21:52         ` Phil Turmel
2013-09-22 22:45           ` Dave Gomboc
2013-09-22 23:04             ` Dave Gomboc
2013-09-22 23:25             ` Phil Turmel
     [not found]               ` <CA+dwz-0eskPSQ44v0vgwfjwRpTbQaokQ3Q258Em1W2eRi1SO4w@mail.gmail.com>
2013-09-23  2:54                 ` Phil Turmel
2013-09-23  3:19                   ` Dave Gomboc
2013-09-23  3:27                     ` Phil Turmel
2013-09-23  3:34                       ` Dave Gomboc
2013-09-23  3:51                         ` Phil Turmel
2013-09-23  4:04                           ` Dave Gomboc
2013-09-23  4:12                             ` Phil Turmel
2013-09-23  4:55                               ` Dave Gomboc
2013-09-23  5:07                                 ` Dave Gomboc
2013-09-23  5:19                                 ` Adam Goryachev
2013-09-23 12:32                                 ` Phil Turmel
2013-09-23 12:57                                   ` Dave Gomboc
2013-09-24  0:29                                     ` Adam Goryachev
2013-09-24  5:55                                       ` Dave Gomboc
2013-09-28 15:47                                     ` Dave Gomboc
2013-09-28 16:01                                       ` Phil Turmel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).