Manually specifying data_offset in component device superblock?

Manually specifying data_offset in component device superblock?

[Mark F]

Hi

__The Problem__

I'm attempting to recover a RAID5 (6 disks) with zeroed superblocks.

mdadm -C --assume-clean -f -e 1.2 -l 5 -p ls -c 128 -n6

But I'm unable to get the data offsets of some of the disks in the
array to match what I had before:
Diff of mdadm --examine /dev/sdb3 (which became /dev/sde3 after reboot)

--- md3.sdb3.orig       2009-07-07 10:14:25.000000000 +0100
+++ md3.sde3.new        2009-07-07 10:14:27.000000000 +0100
@@ -2,26 +2,26 @@
           Magic : a92b4efc
         Version : 1.2
     Feature Map : 0x0
-     Array UUID : 2b7ca9c9:c9fa9a28:086e0f83:90cbef62
+     Array UUID : 679bc68c:aeb0464c:8f11e607:c8e58161
            Name : hastur:3  (local to host hastur)
-  Creation Time : Thu Oct 18 14:46:47 2007
+  Creation Time : Sun Jul  5 03:32:38 2009
      Raid Level : raid5
    Raid Devices : 6

- Avail Dev Size : 870353369 (415.02 GiB 445.62 GB)
+ Avail Dev Size : 870353233 (415.02 GiB 445.62 GB)
      Array Size : 4351765760 (2075.08 GiB 2228.10 GB)
   Used Dev Size : 870353152 (415.02 GiB 445.62 GB)
-    Data Offset : 136 sectors
+    Data Offset : 272 sectors
    Super Offset : 8 sectors
           State : clean
-    Device UUID : c4983266:9ee820fd:106bbf9d:20a69333
+    Device UUID : 1b87acce:883de3fc:881f279e:e2b84a9b

-    Update Time : Mon Jun 29 21:05:55 2009
-       Checksum : 5a501eb5 - correct
-         Events : 320840
+    Update Time : Sun Jul  5 03:32:38 2009
+       Checksum : 94ba3ae4 - correct
+         Events : 0

          Layout : left-symmetric
      Chunk Size : 128K

-    Array Slot : 0 (failed, 1, 2, failed, failed, 4, 5)
-   Array State : _uu_uu 3 failed
+    Array Slot : 0 (0, 1, 2, 3, 4, 5)
+   Array State : Uuuuuu

Data offset for this disk was originally 136, but on recreation, mdadm
chooses 272...

The array started off as RAID5 (5 disks plus a hot spare).
One disk (array slot 4) failed about a year ago and was replaced.
The array was grown to RAID5 (6 disks) shortly after.

__The Question__

So, is there a way to force a particular data offset? (short of
recompiling mdadm?)

__Some things I've tried__

It seems to be feasible to *manually* tweak data_offset with a hexeditor.
To test things I've created some sparse loopback devices of the
correct size (dd seek) with just the first 1MB or so of data in each
device.
After updating data_offset and the checksum of the first disk I
assembled the array and cryptsetup found its header. (Yay!)

But does anyone know if other parts of the superblock are dependent on
the data_offset value?
Am I going to bork things horribly by fiddling with the data_offset?

What data is present between the end of the superblock and the
beginning of data?

Thanks
  Mark

Re: Manually specifying data_offset in component device superblock?

[Mark F]

Sorry, I should have added that the array was created with mdadm from
Debian Etch (mdadm-2.5.6-9 it appears).

I'm currently using Debian Lenny's mdadm-2.6.7.2-1

thanks
 Mark

Re: Manually specifying data_offset in component device superblock?

[Neil Brown]

On Tuesday July 7, gnomeza@gmail.com wrote:
> Hi
> 
> __The Problem__
> 
> I'm attempting to recover a RAID5 (6 disks) with zeroed superblocks.
> 
> mdadm -C --assume-clean -f -e 1.2 -l 5 -p ls -c 128 -n6
> 
> But I'm unable to get the data offsets of some of the disks in the
> array to match what I had before:
> Diff of mdadm --examine /dev/sdb3 (which became /dev/sde3 after reboot)
> 

> 
> - Avail Dev Size : 870353369 (415.02 GiB 445.62 GB)
> + Avail Dev Size : 870353233 (415.02 GiB 445.62 GB)
>       Array Size : 4351765760 (2075.08 GiB 2228.10 GB)
>    Used Dev Size : 870353152 (415.02 GiB 445.62 GB)
> -    Data Offset : 136 sectors
> +    Data Offset : 272 sectors

That is unfortunate, isn't it...

> 
> Data offset for this disk was originally 136, but on recreation, mdadm
> chooses 272...

It must have been created with mdadm-2.5.6 or earlier

> 
> The array started off as RAID5 (5 disks plus a hot spare).
> One disk (array slot 4) failed about a year ago and was replaced.
> The array was grown to RAID5 (6 disks) shortly after.
> 
> __The Question__
> 
> So, is there a way to force a particular data offset? (short of
> recompiling mdadm?)

No.

> 
> __Some things I've tried__
> 
> It seems to be feasible to *manually* tweak data_offset with a hexeditor.
> To test things I've created some sparse loopback devices of the
> correct size (dd seek) with just the first 1MB or so of data in each
> device.
> After updating data_offset and the checksum of the first disk I
> assembled the array and cryptsetup found its header. (Yay!)

You would need to update the data_offset on each device.

> 
> But does anyone know if other parts of the superblock are dependent on
> the data_offset value?
> Am I going to bork things horribly by fiddling with the data_offset?

No, that should be safe (as long as you do it on every device).


> 
> What data is present between the end of the superblock and the
> beginning of data?

That space is used for an internal bitmap.  If you don't have a bitmap
configured, then that space is not used.

NeilBrown

Re: Manually specifying data_offset in component device superblock?

[Mark F]

Hi Neil

Firstly, thanks!
I recovered all the data this evening.

2009/7/9 Neil Brown <neilb@suse.de>:
> On Tuesday July 7, gnomeza@gmail.com wrote:
>>
>> I'm attempting to recover a RAID5 (6 disks) with zeroed superblocks.
>>
>> But I'm unable to get the data offsets of some of the disks in the
>> array to match what I had before:
>> ...
>> Data offset for this disk was originally 136, but on recreation, mdadm
>> chooses 272...
>
> It must have been created with mdadm-2.5.6 or earlier
>
Ta, I downloaded and compiled mdadm-2.5.6.

>> After updating data_offset and the checksum of the first disk I
>> assembled the array and cryptsetup found its header. (Yay!)
>
> You would need to update the data_offset on each device.
>
Thanks again. This info was crucial.
The dump of mdadm --examine /dev/sd* I grabbed before I shut
everything down showed two out of six disks with data_offset of 272
sectors and the rest with 136.
(I didn't think that was weird at first.)

I'm guessing this happened because I dumped the info after trying to
add the two failed disks back to the array with the new version of
mdadm.
Following your hint, I recreated the array with data_offset of 136 on
all the disks.

A script to permute the order of the disks and run mdadm -C
--assume-clean && cryptsetup && mount did the rest.

So...yay!
  Mark