[dm-crypt] Open raid1 with luks encryption after a raid re-create

[dm-crypt] Open raid1 with luks encryption after a raid re-create

[Luis Alexandre]

Hi.

I have a raid1 (mdadm-based) setup with two disks. I had them encrypted with
luks. Everything was ok for 2 years. My PC had a problem and I had to mount
this on a new PC.
When I tried to start the raid on the new PC it only started 1 of the disks
because the other had been replaced on a different PC and had a different
hostname (my original PC had a script to assemble the raid even with the two
disks having a different hostname).

So I tried to fix this different hostname problem by re-creating the raid in
the new PC using


mdadm -C /dev/md127 -l1 -n2 --assume-clean --metadata=1.2 /dev/sdb /dev/sdc1
--uuid=1d925c8d:8c8bb953:4e9070f7:43344cf9

mdadm: /dev/sdb appears to be part of a raid array:
    level=raid1 devices=2 ctime=Sun Feb 12 19:40:32 2012
mdadm: partition table exists on /dev/sdb but will be lost or
       meaningless after creating array
mdadm: /dev/sdc1 appears to contain an ext2fs file system
    size=976760832K  mtime=Mon Aug 25 19:50:27 2014
mdadm: /dev/sdc1 appears to be part of a raid array:
    level=raid1 devices=2 ctime=Tue Aug 26 09:10:39 2014
Continue creating array? y
mdadm: array /dev/md127 started.

All appeared to be OK:

cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4]
[raid10] 
md127 : active (auto-read-only) raid1 sdc1[1] sdb[0]
      976629568 blocks super 1.2 [2/2] [UU]
      
unused devices: <none>


but now luks does not open the raid:

sudo cryptsetup luksOpen /dev/md127 raid1
Device /dev/md127 is not a valid LUKS device.

Any ideas on how to re-open the raid with luks?

Note: I thought there would be no problem with the create command because of
this in the mdadm man page:
"Create Create a new array with per-device metadata (superblocks).
Appropriate metadata is written to each device, and then the array
comprising those devices is activated. A 'resync' process is started to make
sure that the array is consistent (e.g. both sides of a mirror contain the
same data) but *the content of the device is left otherwise untouched*. "

Thanks for any help you can provide.
Luis

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Arno Wagner]

On Sat, Nov 21, 2015 at 17:29:10 CET, Luis Alexandre wrote:
> Hi.
> 
> I have a raid1 (mdadm-based) setup with two disks. I had them encrypted with
> luks. Everything was ok for 2 years. My PC had a problem and I had to mount
> this on a new PC.
> When I tried to start the raid on the new PC it only started 1 of the disks
> because the other had been replaced on a different PC and had a different
> hostname (my original PC had a script to assemble the raid even with the two
> disks having a different hostname).

That is not a good idea at all. It would have been far better to kick
one drive and add it again, giving both disks the same superblock.
 
> So I tried to fix this different hostname problem by re-creating the raid in
> the new PC using
> 
> 
> mdadm -C /dev/md127 -l1 -n2 --assume-clean --metadata=1.2 /dev/sdb /dev/sdc1
> --uuid=1d925c8d:8c8bb953:4e9070f7:43344cf9

Is /dev/sdb correct here? The warning below indicates it should
have been a partition, not the complete device.

maybe you should have done that the other way round?

Also, did teh original array use superblock 1.2? 

> mdadm: /dev/sdb appears to be part of a raid array:
>     level=raid1 devices=2 ctime=Sun Feb 12 19:40:32 2012
> mdadm: partition table exists on /dev/sdb but will be lost or
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
>        meaningless after creating array
> mdadm: /dev/sdc1 appears to contain an ext2fs file system
>     size=976760832K  mtime=Mon Aug 25 19:50:27 2014
> mdadm: /dev/sdc1 appears to be part of a raid array:
>     level=raid1 devices=2 ctime=Tue Aug 26 09:10:39 2014
> Continue creating array? y
> mdadm: array /dev/md127 started.
> 
> All appeared to be OK:
> 
> cat /proc/mdstat 
> Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4]
> [raid10] 
> md127 : active (auto-read-only) raid1 sdc1[1] sdb[0]
>       976629568 blocks super 1.2 [2/2] [UU]
>       
> unused devices: <none>

That tells you nothing. If you create a new RAID array (and you did) it
will just ride over everyhting in the superblock's place (as you used
--assume-clean).

> 
> 
> but now luks does not open the raid:
> 
> sudo cryptsetup luksOpen /dev/md127 raid1
> Device /dev/md127 is not a valid LUKS device.
> 
> Any ideas on how to re-open the raid with luks?

See below.

> Note: I thought there would be no problem with the create command because of
> this in the mdadm man page:
> "Create Create a new array with per-device metadata (superblocks).
> Appropriate metadata is written to each device, and then the array
> comprising those devices is activated. A 'resync' process is started to make
> sure that the array is consistent (e.g. both sides of a mirror contain the
> same data) but *the content of the device is left otherwise untouched*. "
> 

It can sync in the wrong direction. And second, unfortunately,
superblock format 1.2 is a dirty hack designed by incompetents.
It places the RAID header 4k from the start of the device. For 
LUKS, this kills the first keyslow of misaligned. Unfortunately, 
this is also the default. No, that has not happened here or you 
would get the header.

> Thanks for any help you can provide.

Ok, first stop writing to the disks. Second, make a full, binary backup
of each disk. And third, try whether either disk works individually
as degraded array.

If neither gives you a LUKS header, you can still search on the raw 
disks by looking for the LUKS signature. If that also fails, you are 
out of luck and all your data is gone.

Regards,
Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Arno Wagner]

CC'ing to the list, as it serves as a sort-of archive of 
how to solve problems as well and there are several clueful 
and helpful people in it that may spot more things than I do.

On Sun, Nov 22, 2015 at 13:15:11 CET, Luis Alexandre wrote:
[...]
> >It can sync in the wrong direction. And second, unfortunately,
> >superblock format 1.2 is a dirty hack designed by incompetents.
> >It places the RAID header 4k from the start of the device. For
> >LUKS, this kills the first keyslow of misaligned. Unfortunately,
> >this is also the default. No, that has not happened here or you
> >would get the header.
> 
> Since the original raid was already 1.2 format, the location would
> already be the 4k from the start of the device. So where was LUKS
> info placed in terms of distance from the start of the device?

Right at the start. That is why using superblock 0.90 or 1.0 
with LUKS is a good diea as it becomes exceptionally unlikely 
that the MD-header damages the LUKS header.
 
> >>Thanks for any help you can provide.
> >
> >Ok, first stop writing to the disks. Second, make a full, binary backup
> >of each disk. And third, try whether either disk works individually
> >as degraded array.
> 
> 1-Done.
> 2-Done. Dumped the first 2MB of each disk.

Make that 1GB at least to be sure to capture the LUKS header
in it if it is still anywhere. 

> 3-They appear as raid disks but again I cannot open the encryption.
> 
> >If neither gives you a LUKS header, you can still search on the raw
> >disks by looking for the LUKS signature. If that also fails, you are
> >out of luck and all your data is gone.
> 
> The LUKS signature is simply 'LUKS'?

Not quite. FAQ Item 6.12 
(https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions)
gives you a brief version, the LUKS on-disk spec gives everything.
 
> I grepped like this:
> 
>  grep LUKS header_sdb_backup.dmp
> 
> is it the correct way to do it?
> Did not find any match...

Ok, lets repeat that with the full disks and including the full signature

hd /dev/sdx | grep "0  4c 55 4b 53 ba be 00 01"

with x one of your RAID disks. Do this for both. May take a while.
This gives you the alignment as well. The "hd" start of a good
luks header and container (header starts at offset 0) looks like
this:

00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00 |LUKS....aes.....|
 
Only the first 6 bytes are fixed. Bytes 6 and 7 are the version
of which there currently is onlyy "0001". This will always be
aligned to a 512 byte boundary. Doing it this way has the 
advantage that you get the offset as well.

Regards,
Arno 

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Luís Alexandre]

On 22-11-2015 12:52, Arno Wagner wrote:
> CC'ing to the list, as it serves as a sort-of archive of
> how to solve problems as well and there are several clueful
> and helpful people in it that may spot more things than I do.
>
> On Sun, Nov 22, 2015 at 13:15:11 CET, Luis Alexandre wrote:
> [...]
>>> It can sync in the wrong direction. And second, unfortunately,
>>> superblock format 1.2 is a dirty hack designed by incompetents.
>>> It places the RAID header 4k from the start of the device. For
>>> LUKS, this kills the first keyslow of misaligned. Unfortunately,
>>> this is also the default. No, that has not happened here or you
>>> would get the header.
>> Since the original raid was already 1.2 format, the location would
>> already be the 4k from the start of the device. So where was LUKS
>> info placed in terms of distance from the start of the device?
> Right at the start. That is why using superblock 0.90 or 1.0
> with LUKS is a good diea as it becomes exceptionally unlikely
> that the MD-header damages the LUKS header.
>   
>>>> Thanks for any help you can provide.
>>> Ok, first stop writing to the disks. Second, make a full, binary backup
>>> of each disk. And third, try whether either disk works individually
>>> as degraded array.
>> 1-Done.
>> 2-Done. Dumped the first 2MB of each disk.
> Make that 1GB at least to be sure to capture the LUKS header
> in it if it is still anywhere.
>
>> 3-They appear as raid disks but again I cannot open the encryption.
>>
>>> If neither gives you a LUKS header, you can still search on the raw
>>> disks by looking for the LUKS signature. If that also fails, you are
>>> out of luck and all your data is gone.
>> The LUKS signature is simply 'LUKS'?
> Not quite. FAQ Item 6.12
> (https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions)
> gives you a brief version, the LUKS on-disk spec gives everything.
>   
>> I grepped like this:
>>
>>   grep LUKS header_sdb_backup.dmp
>>
>> is it the correct way to do it?
>> Did not find any match...
> Ok, lets repeat that with the full disks and including the full signature
>
> hd /dev/sdx | grep "0  4c 55 4b 53 ba be 00 01"
>
> with x one of your RAID disks. Do this for both. May take a while.
> This gives you the alignment as well. The "hd" start of a good
> luks header and container (header starts at offset 0) looks like
> this:
>
> 00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00 |LUKS....aes.....|
>   
> Only the first 6 bytes are fixed. Bytes 6 and 7 are the version
> of which there currently is onlyy "0001". This will always be
> aligned to a 512 byte boundary. Doing it this way has the
> advantage that you get the offset as well.

found it in one of the disks:
08100000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00 
|LUKS....aes.....|

Can you tell me how should I proceed now?

(the other is still being searched: the first one took a few seconds, 
this one is now over 1 hour search)

Many thanks,
Luis

> Regards,
> Arno
>

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Arno Wagner]

On Sun, Nov 22, 2015 at 16:05:23 CET, Luís Alexandre wrote: 
> On 22-11-2015 12:52, Arno Wagner wrote:
> >Ok, lets repeat that with the full disks and including the full signature
> >
> >hd /dev/sdx | grep "0  4c 55 4b 53 ba be 00 01"
> >
> >with x one of your RAID disks. Do this for both. May take a while.
> >This gives you the alignment as well. The "hd" start of a good
> >luks header and container (header starts at offset 0) looks like
> >this:
> >
> >00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00 |LUKS....aes.....|
> >Only the first 6 bytes are fixed. Bytes 6 and 7 are the version
> >of which there currently is onlyy "0001". This will always be
> >aligned to a 512 byte boundary. Doing it this way has the
> >advantage that you get the offset as well.
> 
> found it in one of the disks:
> 08100000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00
> |LUKS....aes.....|
> 

Ok, you may be in luck and may just have killed the partition 
table.

> Can you tell me how should I proceed now?

If you still do not have that binary backup of the full disk, 
make it now. Seriously. A tiny mistake can kill everything 
permanently at this stage.

Next, we will try to copy this LUKS container to the start 
of a different disk. You need a second disk that is the same 
size or larger as your LUKS container. (A different option would
be to create a partiton at exactly the right spot, but that is
tricky. Copying can also be done with dd, but I find the 
option syntacs rather convoluted. Opter options exist.)

Do this: 

  tail -c +135266305 /dev/sdx > /dev/sdy

with sdx your source and sdy the target (fresh, empty disk). 
This essentially makes a copy shifted 129MB forwards on /dev/sdy. 
May take a long time. (Unfortunately, tail counts from 1, 
hence the argiment is 129MB + 1)

Then try to open the LUKS container on /dev/sdy

  cryptsetup luksOpen /dev/sdy e1

If that works, you can then mount /dev/mapper/e1 and copy your 
data off. If it does not, I may have screwed up the offset.
The LUKS container should be right at the beginning of
/dev/sdy. To just test the header, you can make the copy as above
and interrupt after a few seconds. luksOpen will still work if 
the header is fine. 

While this may seem a lot of effort, remember that you are very 
close to a complete, unrecoverable loss of your data. I would
advise you not to cut corners and in particular make that 
full disk binary backup before you do anything else. 

Regards,
Arno

> (the other is still being searched: the first one took a few
> seconds, this one is now over 1 hour search)
> 
> Many thanks,
> Luis
> 
> >Regards,
> >Arno
> >
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Sven Eschenberg]

Luis,

Looking at the lsblk output you sent:

sdb         8:16   0 931,5G  0 disk
├─sdb1      8:17   0 931,5G  0 part
└─md127     9:127  0 931,4G  0 raid1
sdc         8:32   0 931,5G  0 disk
└─sdc1      8:33   0 931,5G  0 part
   └─md127   9:127  0 931,4G  0 raid1

mdadm complaint about the partition table on /dev/sdb and you had sdb1 
which covered basically the whole drive. Assuming this was the original 
md raid component - did you find the LUKS header somewhere on sdb?

I am asking this, because mdadm placed it's metadata 4k from the start 
of /sdb after your command meaning it is located behind the 
MBR(partition table) and most certainly before the Partition starts.
At least the output of mdstat suggests that sdb is component zero and 
thus was chosen as source.

Originally your mdadm metadata (assuming it was v1.2) was 4k from the 
start of sdb1 and thus the LUKS header would be located somewhere behind 
that, if you had LUKS on top of mdraid. Depending on the parameters the 
md header is quite big as it might include an intent bitmap etc. . It 
would be interesting though to see if there's an md metadata area 4k 
from the start of sdb1.

Now to your question, once you know the offset of the header:
1.)Setup a loop device from your image (You can use an offset into the 
image where your loop device starts with sector 0) see --offset in 
losetup man page.
Inspect loopdevice if the LUKS Header now is on sector 0
2.)Try a cryptsetup luksopen in readonly mode
3.)If you got this far, inspect the contents of the mapping

Regards

-Sven

P.S.: Once you verified everything works, you can still create an image 
of the physical raid component hdd and skip into the drive to the right 
offset. This image would then hopefully include the contents of your 
original md array...

Am 22.11.2015 um 16:05 schrieb Luís Alexandre:
>
>
> On 22-11-2015 12:52, Arno Wagner wrote:
>> CC'ing to the list, as it serves as a sort-of archive of
>> how to solve problems as well and there are several clueful
>> and helpful people in it that may spot more things than I do.
>>
>> On Sun, Nov 22, 2015 at 13:15:11 CET, Luis Alexandre wrote:
>> [...]
>>>> It can sync in the wrong direction. And second, unfortunately,
>>>> superblock format 1.2 is a dirty hack designed by incompetents.
>>>> It places the RAID header 4k from the start of the device. For
>>>> LUKS, this kills the first keyslow of misaligned. Unfortunately,
>>>> this is also the default. No, that has not happened here or you
>>>> would get the header.
>>> Since the original raid was already 1.2 format, the location would
>>> already be the 4k from the start of the device. So where was LUKS
>>> info placed in terms of distance from the start of the device?
>> Right at the start. That is why using superblock 0.90 or 1.0
>> with LUKS is a good diea as it becomes exceptionally unlikely
>> that the MD-header damages the LUKS header.
>>>>> Thanks for any help you can provide.
>>>> Ok, first stop writing to the disks. Second, make a full, binary backup
>>>> of each disk. And third, try whether either disk works individually
>>>> as degraded array.
>>> 1-Done.
>>> 2-Done. Dumped the first 2MB of each disk.
>> Make that 1GB at least to be sure to capture the LUKS header
>> in it if it is still anywhere.
>>
>>> 3-They appear as raid disks but again I cannot open the encryption.
>>>
>>>> If neither gives you a LUKS header, you can still search on the raw
>>>> disks by looking for the LUKS signature. If that also fails, you are
>>>> out of luck and all your data is gone.
>>> The LUKS signature is simply 'LUKS'?
>> Not quite. FAQ Item 6.12
>> (https://gitlab.com/cryptsetup/cryptsetup/wikis/FrequentlyAskedQuestions)
>> gives you a brief version, the LUKS on-disk spec gives everything.
>>> I grepped like this:
>>>
>>>   grep LUKS header_sdb_backup.dmp
>>>
>>> is it the correct way to do it?
>>> Did not find any match...
>> Ok, lets repeat that with the full disks and including the full signature
>>
>> hd /dev/sdx | grep "0  4c 55 4b 53 ba be 00 01"
>>
>> with x one of your RAID disks. Do this for both. May take a while.
>> This gives you the alignment as well. The "hd" start of a good
>> luks header and container (header starts at offset 0) looks like
>> this:
>>
>> 00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00
>> |LUKS....aes.....|
>> Only the first 6 bytes are fixed. Bytes 6 and 7 are the version
>> of which there currently is onlyy "0001". This will always be
>> aligned to a 512 byte boundary. Doing it this way has the
>> advantage that you get the offset as well.
>
> found it in one of the disks:
> 08100000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00
> |LUKS....aes.....|
>
> Can you tell me how should I proceed now?
>
> (the other is still being searched: the first one took a few seconds,
> this one is now over 1 hour search)
>
> Many thanks,
> Luis
>
>> Regards,
>> Arno
>>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Arno Wagner]

On Sun, Nov 22, 2015 at 23:30:23 CET, Sven Eschenberg wrote:
[...]
> Now to your question, once you know the offset of the header:
> 1.)Setup a loop device from your image (You can use an offset into
> the image where your loop device starts with sector 0) see --offset
> in losetup man page.

Ah, yes. That would save copying it.

> Inspect loopdevice if the LUKS Header now is on sector 0
> 2.)Try a cryptsetup luksopen in readonly mode

Good idea. With that it may be reasonaly safe to work
with the original disk. I still would make a full
backup before.

Regards,
Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Sven Eschenberg]

Am 23.11.2015 um 04:35 schrieb Arno Wagner:
> On Sun, Nov 22, 2015 at 23:30:23 CET, Sven Eschenberg wrote:
> [...]
>> Now to your question, once you know the offset of the header:
>> 1.)Setup a loop device from your image (You can use an offset into
>> the image where your loop device starts with sector 0) see --offset
>> in losetup man page.
>
> Ah, yes. That would save copying it.

That was the plan. In general using dmsetup to create a mapping manually 
should work too, if loop device support is missing - dmsetup is pretty 
cryptic to use though.

>
>> Inspect loopdevice if the LUKS Header now is on sector 0
>> 2.)Try a cryptsetup luksopen in readonly mode
>
> Good idea. With that it may be reasonaly safe to work
> with the original disk. I still would make a full
> backup before.
>
> Regards,
> Arno
>

Well, I thought about using the loop on the file while the physical disk 
stays unchanged. Otherwise it would be possible to work on the physical 
disk, and keep a safety image. No matter which way one chooses, always 
have a safety copy.

If the disk is having mechanical problems or something similiar one 
would of course use 2 images, one 'master binary backup' and the replica 
to work on.

Once mapping and opening works, one can choose to either copy out the 
files and backup (usually a good idea) or to create a copy in the manner 
you described. Possibly such an image could then be remerged onto a new 
clean array, if it is otherwise intact. Not without some remaining risks 
though.

Regards

-Sven

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Arno Wagner]

On Mon, Nov 23, 2015 at 04:56:34 CET, Sven Eschenberg wrote:
> 
> 
> Am 23.11.2015 um 04:35 schrieb Arno Wagner:
> >On Sun, Nov 22, 2015 at 23:30:23 CET, Sven Eschenberg wrote:
> >[...]
> >>Now to your question, once you know the offset of the header:
> >>1.)Setup a loop device from your image (You can use an offset into
> >>the image where your loop device starts with sector 0) see --offset
> >>in losetup man page.
> >
> >Ah, yes. That would save copying it.
> 
> That was the plan. In general using dmsetup to create a mapping
> manually should work too, if loop device support is missing -
> dmsetup is pretty cryptic to use though.

"Cryptic" is not good here...

I was not aware that losetup allows read-only mappings, or I 
would probably have looked at it too. 

Excellent! So I learned something too!
 
> >>Inspect loopdevice if the LUKS Header now is on sector 0
> >>2.)Try a cryptsetup luksopen in readonly mode
> >
> >Good idea. With that it may be reasonaly safe to work
> >with the original disk. I still would make a full
> >backup before.
> >
> >Regards,
> >Arno
> >
> 
> Well, I thought about using the loop on the file while the physical
> disk stays unchanged. Otherwise it would be possible to work on the
> physical disk, and keep a safety image. No matter which way one
> chooses, always have a safety copy.
> 
> If the disk is having mechanical problems or something similiar one
> would of course use 2 images, one 'master binary backup' and the
> replica to work on.
> 
> Once mapping and opening works, one can choose to either copy out
> the files and backup (usually a good idea) or to create a copy in
> the manner you described. Possibly such an image could then be
> remerged onto a new clean array, if it is otherwise intact. Not
> without some remaining risks though.

Indeed. And first things first, lets see whether that header is 
viable before goping any further.

Regards,
Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Luis Alexandre]

Dear Arno and Sven,

Thanks very much for your support.

I was able to open the copy I made with

tail -c +135266305 /dev/sdx > /dev/sdy
and
cryptsetup luksOpen /dev/sdy e1

and have all my data back.

Thanks again for your support.

Best regards,
Luis

On 23-11-2015 06:26, Arno Wagner wrote:
> On Mon, Nov 23, 2015 at 04:56:34 CET, Sven Eschenberg wrote:
>>
>>
>> Am 23.11.2015 um 04:35 schrieb Arno Wagner:
>>> On Sun, Nov 22, 2015 at 23:30:23 CET, Sven Eschenberg wrote:
>>> [...]
>>>> Now to your question, once you know the offset of the header:
>>>> 1.)Setup a loop device from your image (You can use an offset into
>>>> the image where your loop device starts with sector 0) see --offset
>>>> in losetup man page.
>>>
>>> Ah, yes. That would save copying it.
>>
>> That was the plan. In general using dmsetup to create a mapping
>> manually should work too, if loop device support is missing -
>> dmsetup is pretty cryptic to use though.
>
> "Cryptic" is not good here...
>
> I was not aware that losetup allows read-only mappings, or I
> would probably have looked at it too.
>
> Excellent! So I learned something too!
>
>>>> Inspect loopdevice if the LUKS Header now is on sector 0
>>>> 2.)Try a cryptsetup luksopen in readonly mode
>>>
>>> Good idea. With that it may be reasonaly safe to work
>>> with the original disk. I still would make a full
>>> backup before.
>>>
>>> Regards,
>>> Arno
>>>
>>
>> Well, I thought about using the loop on the file while the physical
>> disk stays unchanged. Otherwise it would be possible to work on the
>> physical disk, and keep a safety image. No matter which way one
>> chooses, always have a safety copy.
>>
>> If the disk is having mechanical problems or something similiar one
>> would of course use 2 images, one 'master binary backup' and the
>> replica to work on.
>>
>> Once mapping and opening works, one can choose to either copy out
>> the files and backup (usually a good idea) or to create a copy in
>> the manner you described. Possibly such an image could then be
>> remerged onto a new clean array, if it is otherwise intact. Not
>> without some remaining risks though.
>
> Indeed. And first things first, lets see whether that header is
> viable before goping any further.
>
> Regards,
> Arno
>

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Arno Wagner]

Excellent! You are welcome.

Regards,
Arno

On Tue, Nov 24, 2015 at 12:15:18 CET, Luis Alexandre wrote:
> 
> Dear Arno and Sven,
> 
> Thanks very much for your support.
> 
> I was able to open the copy I made with
> 
> tail -c +135266305 /dev/sdx > /dev/sdy
> and
> cryptsetup luksOpen /dev/sdy e1
> 
> and have all my data back.
> 
> Thanks again for your support.
> 
> Best regards,
> Luis
> 
> On 23-11-2015 06:26, Arno Wagner wrote:
> >On Mon, Nov 23, 2015 at 04:56:34 CET, Sven Eschenberg wrote:
> >>
> >>
> >>Am 23.11.2015 um 04:35 schrieb Arno Wagner:
> >>>On Sun, Nov 22, 2015 at 23:30:23 CET, Sven Eschenberg wrote:
> >>>[...]
> >>>>Now to your question, once you know the offset of the header:
> >>>>1.)Setup a loop device from your image (You can use an offset into
> >>>>the image where your loop device starts with sector 0) see --offset
> >>>>in losetup man page.
> >>>
> >>>Ah, yes. That would save copying it.
> >>
> >>That was the plan. In general using dmsetup to create a mapping
> >>manually should work too, if loop device support is missing -
> >>dmsetup is pretty cryptic to use though.
> >
> >"Cryptic" is not good here...
> >
> >I was not aware that losetup allows read-only mappings, or I
> >would probably have looked at it too.
> >
> >Excellent! So I learned something too!
> >
> >>>>Inspect loopdevice if the LUKS Header now is on sector 0
> >>>>2.)Try a cryptsetup luksopen in readonly mode
> >>>
> >>>Good idea. With that it may be reasonaly safe to work
> >>>with the original disk. I still would make a full
> >>>backup before.
> >>>
> >>>Regards,
> >>>Arno
> >>>
> >>
> >>Well, I thought about using the loop on the file while the physical
> >>disk stays unchanged. Otherwise it would be possible to work on the
> >>physical disk, and keep a safety image. No matter which way one
> >>chooses, always have a safety copy.
> >>
> >>If the disk is having mechanical problems or something similiar one
> >>would of course use 2 images, one 'master binary backup' and the
> >>replica to work on.
> >>
> >>Once mapping and opening works, one can choose to either copy out
> >>the files and backup (usually a good idea) or to create a copy in
> >>the manner you described. Possibly such an image could then be
> >>remerged onto a new clean array, if it is otherwise intact. Not
> >>without some remaining risks though.
> >
> >Indeed. And first things first, lets see whether that header is
> >viable before goping any further.
> >
> >Regards,
> >Arno
> >
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Sven Eschenberg]

Seems the mail got a little delayed on my side...

You are welcome Luis, good it worked out.

Regards

-Sven


Am 24.11.2015 um 12:15 schrieb Luis Alexandre:
>
> Dear Arno and Sven,
>
> Thanks very much for your support.
>
> I was able to open the copy I made with
>
> tail -c +135266305 /dev/sdx > /dev/sdy
> and
> cryptsetup luksOpen /dev/sdy e1
>
> and have all my data back.
>
> Thanks again for your support.
>
> Best regards,
> Luis
>
> On 23-11-2015 06:26, Arno Wagner wrote:
>> On Mon, Nov 23, 2015 at 04:56:34 CET, Sven Eschenberg wrote:
>>>
>>>
>>> Am 23.11.2015 um 04:35 schrieb Arno Wagner:
>>>> On Sun, Nov 22, 2015 at 23:30:23 CET, Sven Eschenberg wrote:
>>>> [...]
>>>>> Now to your question, once you know the offset of the header:
>>>>> 1.)Setup a loop device from your image (You can use an offset into
>>>>> the image where your loop device starts with sector 0) see --offset
>>>>> in losetup man page.
>>>>
>>>> Ah, yes. That would save copying it.
>>>
>>> That was the plan. In general using dmsetup to create a mapping
>>> manually should work too, if loop device support is missing -
>>> dmsetup is pretty cryptic to use though.
>>
>> "Cryptic" is not good here...
>>
>> I was not aware that losetup allows read-only mappings, or I
>> would probably have looked at it too.
>>
>> Excellent! So I learned something too!
>>
>>>>> Inspect loopdevice if the LUKS Header now is on sector 0
>>>>> 2.)Try a cryptsetup luksopen in readonly mode
>>>>
>>>> Good idea. With that it may be reasonaly safe to work
>>>> with the original disk. I still would make a full
>>>> backup before.
>>>>
>>>> Regards,
>>>> Arno
>>>>
>>>
>>> Well, I thought about using the loop on the file while the physical
>>> disk stays unchanged. Otherwise it would be possible to work on the
>>> physical disk, and keep a safety image. No matter which way one
>>> chooses, always have a safety copy.
>>>
>>> If the disk is having mechanical problems or something similiar one
>>> would of course use 2 images, one 'master binary backup' and the
>>> replica to work on.
>>>
>>> Once mapping and opening works, one can choose to either copy out
>>> the files and backup (usually a good idea) or to create a copy in
>>> the manner you described. Possibly such an image could then be
>>> remerged onto a new clean array, if it is otherwise intact. Not
>>> without some remaining risks though.
>>
>> Indeed. And first things first, lets see whether that header is
>> viable before goping any further.
>>
>> Regards,
>> Arno
>>
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

Re: [dm-crypt] Open raid1 with luks encryption after a raid re-create

[Sven Eschenberg]

Hi,

It does say, that metadata is written to the disks. Depending on the 
metadata version, this could still fry your LUKS-Header. v1.2 Header is 
stored 4k from the start of the drive.

You really should have started the array degraded, check the contents 
and then hot-add the new (empty) drive.

--assume-clean is something you use, when you assume or know, that your 
array is ok, even if one of the metadata headers says differently.

And mdadm gave you quite some lines to consider first:
/dev/sdc1 appears to contain an ext2fs file system - This probably was 
not The old raid component, when it had a LUKS header ontop. A check 
would have been wise and then remove the ext2fs signatures, once you 
really made sure.
mdadm: partition table exists on /dev/sdb but will be lost or...
Now THIS drive had a partitions table, maybe your raid component was on 
one of the partitions?

Regards

-Sven

P.S.: For a recovery, create an image of the running md, then of each 
component drive. Inspect the images if you can find the LUKS header at 
all. If you do, dd the header and keyslots out of the image and check 
the header throroughly, if it's broken, try to fix the copy you made, if 
it can be used and you are able to create a ro-LUKS-mapping, you can 
possibly recover your data (You really would want to back it up at this 
point!)

Am 21.11.2015 um 17:29 schrieb Luis Alexandre:
> Hi.
>
> I have a raid1 (mdadm-based) setup with two disks. I had them encrypted with
> luks. Everything was ok for 2 years. My PC had a problem and I had to mount
> this on a new PC.
> When I tried to start the raid on the new PC it only started 1 of the disks
> because the other had been replaced on a different PC and had a different
> hostname (my original PC had a script to assemble the raid even with the two
> disks having a different hostname).
>
> So I tried to fix this different hostname problem by re-creating the raid in
> the new PC using
>
>
> mdadm -C /dev/md127 -l1 -n2 --assume-clean --metadata=1.2 /dev/sdb /dev/sdc1
> --uuid=1d925c8d:8c8bb953:4e9070f7:43344cf9
>
> mdadm: /dev/sdb appears to be part of a raid array:
>      level=raid1 devices=2 ctime=Sun Feb 12 19:40:32 2012
> mdadm: partition table exists on /dev/sdb but will be lost or
>         meaningless after creating array
> mdadm: /dev/sdc1 appears to contain an ext2fs file system
>      size=976760832K  mtime=Mon Aug 25 19:50:27 2014
> mdadm: /dev/sdc1 appears to be part of a raid array:
>      level=raid1 devices=2 ctime=Tue Aug 26 09:10:39 2014
> Continue creating array? y
> mdadm: array /dev/md127 started.
>
> All appeared to be OK:
>
> cat /proc/mdstat
> Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4]
> [raid10]
> md127 : active (auto-read-only) raid1 sdc1[1] sdb[0]
>        976629568 blocks super 1.2 [2/2] [UU]
>
> unused devices: <none>
>
>
> but now luks does not open the raid:
>
> sudo cryptsetup luksOpen /dev/md127 raid1
> Device /dev/md127 is not a valid LUKS device.
>
> Any ideas on how to re-open the raid with luks?
>
> Note: I thought there would be no problem with the create command because of
> this in the mdadm man page:
> "Create Create a new array with per-device metadata (superblocks).
> Appropriate metadata is written to each device, and then the array
> comprising those devices is activated. A 'resync' process is started to make
> sure that the array is consistent (e.g. both sides of a mirror contain the
> same data) but *the content of the device is left otherwise untouched*. "
>
> Thanks for any help you can provide.
> Luis
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>