[dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Willie]

Evening all,

I'm not very hopeful of a positive response, but having just made my
worst mistake in thirty years of computing I thought this would be where
most of the relevant knowledge is.

I have an external 1.5TB Seagate drive, encrypted with dm-crypt/luks and
formatted xfs.

In a state of dog-tiredness, thinking I was pointing at a USB stick, I
have inadvertently wiped a few hundred MB of the beginning of this disk
with:

   dd if=./archlinux-2010.05-netinstall-i686.iso of=/dev/sdc

My question, as you might guess - is there any possibility of recovering
the vast amount of data still on the drive? I could do it with an
unencrypted disk, but I have no idea how to proceed in this case.

Thanks for any suggestions. (I've managed not to cry so far...)

Willie


-- 
http://www.fastmail.fm - Accessible with your email software
                          or over the web

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 499 bytes --]

On Sun, 2010-08-01 at 14:11 -0700, Willie wrote:
> My question, as you might guess - is there any possibility of recovering
> the vast amount of data still on the drive? I could do it with an
> unencrypted disk, but I have no idea how to proceed in this case.
If you have no backups of the dm-crypt key (and in the case of LUKS: the
LUKS header + the passphrase required to decrypt the volume key) you
have no chance (at least unless the cipher you've used is broken ;) ).

Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3387 bytes --]

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Arno Wagner]

On Sun, Aug 01, 2010 at 02:11:36PM -0700, Willie wrote:
> Evening all,
> 
> I'm not very hopeful of a positive response, but having just made my
> worst mistake in thirty years of computing I thought this would be where
> most of the relevant knowledge is.
> 
> I have an external 1.5TB Seagate drive, encrypted with dm-crypt/luks and
> formatted xfs.
> 
> In a state of dog-tiredness, thinking I was pointing at a USB stick, I
> have inadvertently wiped a few hundred MB of the beginning of this disk
> with:
> 
>    dd if=./archlinux-2010.05-netinstall-i686.iso of=/dev/sdc
> 
> My question, as you might guess - is there any possibility of recovering
> the vast amount of data still on the drive? I could do it with an
> unencrypted disk, but I have no idea how to proceed in this case.
> 
> Thanks for any suggestions. (I've managed not to cry so far...)
> 
> Willie

Hi Willie,

sorry, but you will have wiped the salt in the header, which 
makes recovery impossible. You will also have wiped all keys
(they take about the first 8.5MB), which again does make recovery 
impossible. In fact, any recovery from this would mean that
LUKS is badly broken security-wise.

The only protection against this type of error is (besides a 
conventional backup), a header backup, see the FAQ at 
http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions

I did something similar recently, (tired and thinking I was blanking
an USB stick), fortunately I had a backup of the whole disk. But the 
lession to me was: Hands away from dd and family when tired. 

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Willie]

On Mon, 02 Aug 2010 01:01 +0200, "Arno Wagner" <arno@wagner.name> wrote:
> On Sun, Aug 01, 2010 at 02:11:36PM -0700, Willie wrote:
> > Evening all,
> > 
> > I'm not very hopeful of a positive response, but having just made my
> > worst mistake in thirty years of computing I thought this would be where
> > most of the relevant knowledge is.
> > 
> > I have an external 1.5TB Seagate drive, encrypted with dm-crypt/luks and
> > formatted xfs.
> > 
> > In a state of dog-tiredness, thinking I was pointing at a USB stick, I
> > have inadvertently wiped a few hundred MB of the beginning of this disk
> > with:
> > 
> >    dd if=./archlinux-2010.05-netinstall-i686.iso of=/dev/sdc
> > 
> > My question, as you might guess - is there any possibility of recovering
> > the vast amount of data still on the drive? I could do it with an
> > unencrypted disk, but I have no idea how to proceed in this case.
> > 
> > Thanks for any suggestions. (I've managed not to cry so far...)
> > 
> > Willie
> 
> Hi Willie,
> 
> sorry, but you will have wiped the salt in the header, which 
> makes recovery impossible. You will also have wiped all keys
> (they take about the first 8.5MB), which again does make recovery 
> impossible. In fact, any recovery from this would mean that
> LUKS is badly broken security-wise.
> 
> The only protection against this type of error is (besides a 
> conventional backup), a header backup, see the FAQ at 
> http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions
> 
> I did something similar recently, (tired and thinking I was blanking
> an USB stick), fortunately I had a backup of the whole disk. But the 
> lession to me was: Hands away from dd and family when tired. 
> 
> Arno
> 

Oh well. Some you lose. Chin up. Onward and upward, etc etc...

Thanks for the replies lads.



-- 
http://www.fastmail.fm - IMAP accessible web-mail

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Arno Wagner]

On Sun, Aug 01, 2010 at 04:20:38PM -0700, Willie wrote:
>  
> Oh well. Some you lose. Chin up. Onward and upward, etc etc...

That's the spirit. Only those doing nothing of importance
do not screw up sometimes.

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Willie]

On Mon, 02 Aug 2010 02:27 +0200, "Arno Wagner" <arno@wagner.name> wrote:
> On Sun, Aug 01, 2010 at 04:20:38PM -0700, Willie wrote:
> >  
> > Oh well. Some you lose. Chin up. Onward and upward, etc etc...
> 
> That's the spirit. Only those doing nothing of importance
> do not screw up sometimes.
> 
> Arno
> 
> -- 

Well, it's kind of strange. I aborted the dd as soon as I became aware
of my moment of crass stupidity and I can still see part of the
directory structure, with quite a few file names in there showing proper
file sizes, but these seem to be empty of data.

I still have
"/dev/mapper/udisks-luks-uuid-c1a534b4-d1ba-40d0-adb8-6d2490f06ade-uid9330
on /media/Seagate_1.5 type xfs (rw,nosuid,nodev,uhelper=udisks)", and
obviously I have the pass phrase.

I suppose it's the key that's gone, and there's the kicker, right?



-- 
http://www.fastmail.fm - Email service worth paying for. Try it for free

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Arno Wagner]

Is this LUKS? If it is, it should refuse to open becaue of 
missing header. If it is plain dm--crypt, you just lost some 
data at the beginning of the disk and can use the usual 
recovery methods...

Arno

On Mon, Aug 02, 2010 at 01:38:47AM -0700, Willie wrote:
> 
> 
> On Mon, 02 Aug 2010 02:27 +0200, "Arno Wagner" <arno@wagner.name> wrote:
> > On Sun, Aug 01, 2010 at 04:20:38PM -0700, Willie wrote:
> > >  
> > > Oh well. Some you lose. Chin up. Onward and upward, etc etc...
> > 
> > That's the spirit. Only those doing nothing of importance
> > do not screw up sometimes.
> > 
> > Arno
> > 
> > -- 
> 
> Well, it's kind of strange. I aborted the dd as soon as I became aware
> of my moment of crass stupidity and I can still see part of the
> directory structure, with quite a few file names in there showing proper
> file sizes, but these seem to be empty of data.
> 
> I still have
> "/dev/mapper/udisks-luks-uuid-c1a534b4-d1ba-40d0-adb8-6d2490f06ade-uid9330
> on /media/Seagate_1.5 type xfs (rw,nosuid,nodev,uhelper=udisks)", and
> obviously I have the pass phrase.
> 
> I suppose it's the key that's gone, and there's the kicker, right?
> 
> 
> 
> -- 
> http://www.fastmail.fm - Email service worth paying for. Try it for free
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Milan Broz]

On 08/02/2010 10:38 AM, Willie wrote:
> I still have
> "/dev/mapper/udisks-luks-uuid-c1a534b4-d1ba-40d0-adb8-6d2490f06ade-uid9330
> on /media/Seagate_1.5 type xfs (rw,nosuid,nodev,uhelper=udisks)", and
> obviously I have the pass phrase.

If you have live mapping still, you are not lost completely yet.
Do not reboot! First run "dmsetup table --showkeys" and "dmsetup info -c"
and store the full mapping to some file.

If you see dm-crypt mapping there mapped to proper drive, you can still recreate
LUKS header with some some magic.

(If you have saved that dmsetup mapping, I'll describe hot to save you data
- still depends how many of fs was overwritten.)

If you want help with that, paste here "dmsetup table" (*without* using --showkeys,
we do not want see your master key:-). For recovery you will need to know that key,
so be sure you have full table with key stored as written above.

(Table is created according to LUKS header which is lost, so after reboot you are
lost completely. BTW Passphrase will not help here at all.)

Milan

Re: [dm-crypt] Hosed encrypted drive. Is disaster recovery possible?

[Mikko Rauhala]

ma, 2010-08-02 kello 11:58 +0200, Milan Broz kirjoitti:
> If you have live mapping still, you are not lost completely yet.
> Do not reboot! First run "dmsetup table --showkeys" and "dmsetup info -c"
> and store the full mapping to some file.

_After_ doing that to have the mapping safe, it occurs to one that if
the mapping is indeed live, you could just dump the decrypted block
device image elsewhere (either re-encrypted or not). This would reduce
the problem into a generic filesystem recovery one, with no need to
tinker with reconstructing the original mapping.

-- 
Mikko Rauhala <mjrauhal@cc.helsinki.fi>
University of Helsinki

[dm-crypt] How to gather LUKS parameters from active device (if LUKS header lost)

[Milan Broz]

On 08/02/2010 11:58 AM, Milan Broz wrote:
> If you see dm-crypt mapping there mapped to proper drive, you can still recreate
> LUKS header with some some magic.

Well, here is the idea how to reconstruct LUKS header from active mapping
if header is lost but mapping is still active.
(Note: if device is not active, recovery is impossible).

- it will change LUKS UUID!
- no passphrase needed, it asks for new one (root access required of course)
- cryptsetup 1.1.x required.

Do not save master key file (second param) to unencrypted filesystem!

I'll add something similar to cryptsetup distro into DOC install,
for now take this as an idea - see attached script (it will not touch device,
only saves master key to file and print required parameters for cryptsetup).

BEWARE: NO GUARANTEES AT ALL. NOT PROPERLY TESTED.

Example:
  If you have mapped device named "luks_sdb", script will produce this:

  # <script> luks_sdb /mnt/safedisk/sdb_master_key

  Generating master key to file /mnt/safedisk/sdb_master_key.
  You can now try to reformat LUKS device using:
  cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 --align-payload=2056 --master-key-file=/mnt/safedisk/sdb_master_key /dev/sdb

Milan

[---cut here---]
#!/bin/bash

# Try to get LUKS info and master key from active mapping and prepare parameters for cryptsetup"
# (C) 2010 Milan Broz <asi@ucw.cz>


fail() { echo -e $1 ; exit 1 ; }
field() { echo $(dmsetup table --target crypt --showkeys $DEVICE | cut -d' ' -f$1) ; }
field_cryptsetup() { echo $(cryptsetup status $DEVICE | grep $1 | sed "s/.*$1:\s*//;s/\ .*//") ; }

which xxd >/dev/null || fail "You need xxd (part of vim package) installed to convert key."

[ -z "$2" ] && fail "LUKS header from active mapping, use:\n $0 crypt_mapped_device mk_file_name";

DEVICE=$1
MK_FILE=$2

[ -z "$(field 4)" ] && fail "Mapping $1 not active or it is not crypt target."

CIPHER=$(field_cryptsetup cipher)
OFFSET=$(field_cryptsetup offset)
REAL_DEVICE=$(field_cryptsetup device)
KEY_SIZE=$(field_cryptsetup keysize)
KEY=$(field 5)

[ -z "$CIPHER" -o -z "$OFFSET" -o "$OFFSET" -le 383 -o -z "$KEY" ] && fail "Incompatible device, sorry."

echo "Generating master key to file $MK_FILE."
echo -E -n $KEY| xxd -r -p >$MK_FILE

echo "You can now try to reformat LUKS device using:"
echo "  cryptsetup luksFormat -c $CIPHER -s $KEY_SIZE --align-payload=$OFFSET --master-key-file=$MK_FILE $REAL_DEVICE"

Re: [dm-crypt] How to gather LUKS parameters from active device (if LUKS header lost)

[Willie]

On Mon, 02 Aug 2010 15:43 +0200, "Milan Broz" <mbroz@redhat.com> wrote:
> 
> 
> On 08/02/2010 11:58 AM, Milan Broz wrote:
> > If you see dm-crypt mapping there mapped to proper drive, you can still recreate
> > LUKS header with some some magic.
> 
> Well, here is the idea how to reconstruct LUKS header from active mapping
> if header is lost but mapping is still active.
> (Note: if device is not active, recovery is impossible).
> 
> - it will change LUKS UUID!
> - no passphrase needed, it asks for new one (root access required of
> course)
> - cryptsetup 1.1.x required.
> 
> Do not save master key file (second param) to unencrypted filesystem!
> 
> I'll add something similar to cryptsetup distro into DOC install,
> for now take this as an idea - see attached script (it will not touch
> device,
> only saves master key to file and print required parameters for
> cryptsetup).
> 
> BEWARE: NO GUARANTEES AT ALL. NOT PROPERLY TESTED.
> 
> Example:
>   If you have mapped device named "luks_sdb", script will produce this:
> 
>   # <script> luks_sdb /mnt/safedisk/sdb_master_key
> 
>   Generating master key to file /mnt/safedisk/sdb_master_key.
>   You can now try to reformat LUKS device using:
>   cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256
>   --align-payload=2056 --master-key-file=/mnt/safedisk/sdb_master_key
>   /dev/sdb
> 
> Milan
> 
> [---cut here---]
> #!/bin/bash
> 
> # Try to get LUKS info and master key from active mapping and prepare
> parameters for cryptsetup"
> # (C) 2010 Milan Broz <asi@ucw.cz>
> 
> 
> fail() { echo -e $1 ; exit 1 ; }
> field() { echo $(dmsetup table --target crypt --showkeys $DEVICE | cut
> -d' ' -f$1) ; }
> field_cryptsetup() { echo $(cryptsetup status $DEVICE | grep $1 | sed
> "s/.*$1:\s*//;s/\ .*//") ; }
> 
> which xxd >/dev/null || fail "You need xxd (part of vim package)
> installed to convert key."
> 
> [ -z "$2" ] && fail "LUKS header from active mapping, use:\n $0
> crypt_mapped_device mk_file_name";
> 
> DEVICE=$1
> MK_FILE=$2
> 
> [ -z "$(field 4)" ] && fail "Mapping $1 not active or it is not crypt
> target."
> 
> CIPHER=$(field_cryptsetup cipher)
> OFFSET=$(field_cryptsetup offset)
> REAL_DEVICE=$(field_cryptsetup device)
> KEY_SIZE=$(field_cryptsetup keysize)
> KEY=$(field 5)
> 
> [ -z "$CIPHER" -o -z "$OFFSET" -o "$OFFSET" -le 383 -o -z "$KEY" ] &&
> fail "Incompatible device, sorry."
> 
> echo "Generating master key to file $MK_FILE."
> echo -E -n $KEY| xxd -r -p >$MK_FILE
> 
> echo "You can now try to reformat LUKS device using:"
> echo "  cryptsetup luksFormat -c $CIPHER -s $KEY_SIZE
> --align-payload=$OFFSET --master-key-file=$MK_FILE $REAL_DEVICE"
> 


It gets worse and worse: I go to work, come back and my woman has turned
off the computer. Whatever I was seeing earlier today is no longer there
- just the iso image I wrote to the disk.

I think I'm stuffed, but very very grateful for the helpful replies
here.



-- 
http://www.fastmail.fm - Faster than the air-speed velocity of an
                          unladen european swallow

Re: [dm-crypt] How to gather LUKS parameters from active device (if LUKS header lost)

[Arno Wagner]

Hmm, this was still mounted? Ah, I see. Another thing to add to
my list to things not to do when tired: Giving advice in critical 
situations. Sorry.

One thing you can do before trying Milan's instructions is to 
make an image backup (with dd/dd_rescue) of the decrypted device,
i.e. the device in /dev/mapper/<something>.
That will fix the current state in case something goes wrong
and you can do conventional recovery on the image.

Arno


On Mon, Aug 02, 2010 at 03:43:01PM +0200, Milan Broz wrote:
> 
> 
> On 08/02/2010 11:58 AM, Milan Broz wrote:
> > If you see dm-crypt mapping there mapped to proper drive, you can still recreate
> > LUKS header with some some magic.
> 
> Well, here is the idea how to reconstruct LUKS header from active mapping
> if header is lost but mapping is still active.
> (Note: if device is not active, recovery is impossible).
> 
> - it will change LUKS UUID!
> - no passphrase needed, it asks for new one (root access required of course)
> - cryptsetup 1.1.x required.
> 
> Do not save master key file (second param) to unencrypted filesystem!
> 
> I'll add something similar to cryptsetup distro into DOC install,
> for now take this as an idea - see attached script (it will not touch device,
> only saves master key to file and print required parameters for cryptsetup).
> 
> BEWARE: NO GUARANTEES AT ALL. NOT PROPERLY TESTED.
> 
> Example:
>   If you have mapped device named "luks_sdb", script will produce this:
> 
>   # <script> luks_sdb /mnt/safedisk/sdb_master_key
> 
>   Generating master key to file /mnt/safedisk/sdb_master_key.
>   You can now try to reformat LUKS device using:
>   cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 --align-payload=2056 --master-key-file=/mnt/safedisk/sdb_master_key /dev/sdb
> 
> Milan
> 
> [---cut here---]
> #!/bin/bash
> 
> # Try to get LUKS info and master key from active mapping and prepare parameters for cryptsetup"
> # (C) 2010 Milan Broz <asi@ucw.cz>
> 
> 
> fail() { echo -e $1 ; exit 1 ; }
> field() { echo $(dmsetup table --target crypt --showkeys $DEVICE | cut -d' ' -f$1) ; }
> field_cryptsetup() { echo $(cryptsetup status $DEVICE | grep $1 | sed "s/.*$1:\s*//;s/\ .*//") ; }
> 
> which xxd >/dev/null || fail "You need xxd (part of vim package) installed to convert key."
> 
> [ -z "$2" ] && fail "LUKS header from active mapping, use:\n $0 crypt_mapped_device mk_file_name";
> 
> DEVICE=$1
> MK_FILE=$2
> 
> [ -z "$(field 4)" ] && fail "Mapping $1 not active or it is not crypt target."
> 
> CIPHER=$(field_cryptsetup cipher)
> OFFSET=$(field_cryptsetup offset)
> REAL_DEVICE=$(field_cryptsetup device)
> KEY_SIZE=$(field_cryptsetup keysize)
> KEY=$(field 5)
> 
> [ -z "$CIPHER" -o -z "$OFFSET" -o "$OFFSET" -le 383 -o -z "$KEY" ] && fail "Incompatible device, sorry."
> 
> echo "Generating master key to file $MK_FILE."
> echo -E -n $KEY| xxd -r -p >$MK_FILE
> 
> echo "You can now try to reformat LUKS device using:"
> echo "  cryptsetup luksFormat -c $CIPHER -s $KEY_SIZE --align-payload=$OFFSET --master-key-file=$MK_FILE $REAL_DEVICE"
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier