[dm-crypt] yet another "lost my partition" message

[dm-crypt] yet another "lost my partition" message

[Hugo Melo]

[-- Attachment #1: Type: text/plain, Size: 495 bytes --]

Hi everyone,
I've just lost my home partition trying to reinstall ubuntu.
On the installation process, trying to configure my encrypted home
partition, I thought it was going to mount my partition but it was asking
for the new key.
After that I rebooted and didn't make any other changes to my partition.
It seems the partitioner used "cryptsetup luksFormat". Do you think I can
still restore my partition?
I used the same passphrase configured for my home.
Thanks on any light
-- 
-- Hugo Melo

[-- Attachment #2: Type: text/html, Size: 562 bytes --]

Re: [dm-crypt] yet another "lost my partition" message

[Arno Wagner]

Hi Hugo,

first, there is a lot of info inthe FAQ here:
http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions

Best boot a resuce system and try to decrypt the partition
manually with cryptsetup. If that works with any of the
possible passphrases, then all your data is still there.
If not, best post the error message here.

Manual decryption should work with a command like this:

> cryptsetup luksOpen /dev/<your device> e1

Afterwards the decrypted device is  /dev/mapper/e1
You can mount that normally.

You can get header details like this:

> cryptsetup luksDump /dev/<your partition>

Ubuntu should have a resuce mode or life-cd mode that
allows you to do this.

Gruesse,
Arno


On Tue, Apr 12, 2011 at 05:13:17PM -0300, Hugo Melo wrote:
> Hi everyone,
> I've just lost my home partition trying to reinstall ubuntu.
> On the installation process, trying to configure my encrypted home
> partition, I thought it was going to mount my partition but it was asking
> for the new key.
> After that I rebooted and didn't make any other changes to my partition.
> It seems the partitioner used "cryptsetup luksFormat". Do you think I can
> still restore my partition?
> I used the same passphrase configured for my home.
> Thanks on any light
> -- 
> -- Hugo Melo

> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] yet another "lost my partition" message

[Hugo Melo]

[-- Attachment #1: Type: text/plain, Size: 2786 bytes --]

Hi Arno,
Thank you for replying
As it was not the root partition the one I was using with encryption I can
boot without problems on my old system without mounting /home ( the
partition with issue).
So, booted again in my old distro and ran the comand:
 cryptsetup luksOpen /dev/sda10 home
It worked with my passphrase and I could see the /dev/mapper/home device
created.
Trying to mount it I get:
mount /dev/mapper/home /mnt/
mount: you must specify the filesystem type

It seems to have lost the partition table so.
I am running testdisk now and trying to understand if this is really my
partition without some blocks
I will be back with updates
Thanks again!
On Tue, Apr 12, 2011 at 10:10 PM, Arno Wagner <arno@wagner.name> wrote:

> Hi Hugo,
>
> first, there is a lot of info inthe FAQ here:
> http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions
>
> Best boot a resuce system and try to decrypt the partition
> manually with cryptsetup. If that works with any of the
> possible passphrases, then all your data is still there.
> If not, best post the error message here.
>
> Manual decryption should work with a command like this:
>
> > cryptsetup luksOpen /dev/<your device> e1
>
> Afterwards the decrypted device is  /dev/mapper/e1
> You can mount that normally.
>
> You can get header details like this:
>
> > cryptsetup luksDump /dev/<your partition>
>
> Ubuntu should have a resuce mode or life-cd mode that
> allows you to do this.
>
> Gruesse,
> Arno
>
>
> On Tue, Apr 12, 2011 at 05:13:17PM -0300, Hugo Melo wrote:
> > Hi everyone,
> > I've just lost my home partition trying to reinstall ubuntu.
> > On the installation process, trying to configure my encrypted home
> > partition, I thought it was going to mount my partition but it was asking
> > for the new key.
> > After that I rebooted and didn't make any other changes to my partition.
> > It seems the partitioner used "cryptsetup luksFormat". Do you think I can
> > still restore my partition?
> > I used the same passphrase configured for my home.
> > Thanks on any light
> > --
> > -- Hugo Melo
>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
>
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email:
> arno@wagner.name
> GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25
> 338F
> ----
> Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
>
> If it's in the news, don't worry about it.  The very definition of
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>



-- 
-- Hugo Melo

[-- Attachment #2: Type: text/html, Size: 3891 bytes --]

Re: [dm-crypt] yet another "lost my partition" message

[Cristian KLEIN]

Hello list,

> I've just lost my home partition trying to reinstall ubuntu.
> On the installation process, trying to configure my encrypted home
> partition, I thought it was going to mount my partition but it was asking
> for the new key.
> After that I rebooted and didn't make any other changes to my partition.
> It seems the partitioner used "cryptsetup luksFormat". Do you think I can
> still restore my partition?
> I used the same passphrase configured for my home.

This is *exactly* what I did yesterday. I thought I was tired or
something, but seeing that it happened to somebody else too, I start
thinking that the installer's interface sucks ... badly!

I assume there is no way to recover the original file system. Ubuntu has
most likely overwritten the LUKS header where the pretious salt is being
stored. The unencrypted disk most likely looks like random data now.
According to the FAQ [1], you can still resort to the dm-crypt
mailing-list to get over the five stages of grief.

A posteriori, I cannot help wonder why such pretious information isn't
kept redundantly. Surely LUKS could have stored the header in 10 random
sectors with an easy-to-grep "HERE I AM" banner. Wouldn't this allow
users to recover the master-key (and part of the file-system) without
compromising security?

Cristi.

P.S. Could anybody donate 2^256 CPU-seconds?
P.S.2. What stage of grief am I in? :D

[1] http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions

Re: [dm-crypt] yet another "lost my partition" message

[Roscoe]

On Fri, Apr 15, 2011 at 11:52 PM, Cristian KLEIN <cristiklein@gmail.com> wrote:
...
> A posteriori, I cannot help wonder why such pretious information isn't
> kept redundantly. Surely LUKS could have stored the header in 10 random
> sectors with an easy-to-grep "HERE I AM" banner. Wouldn't this allow
> users to recover the master-key (and part of the file-system) without
> compromising security?
...

It's supposed to be fragile and easily destroyed, this is by design.

I don't think we should bother with complicated safeguards for people
doing silly things. That installer interface should be modified long,
long, long before the on-disk format is.

Accidently running cryptsetup luksFormat is unfortunate, as is running
mkfs or dd on the wrong device. Good thing for backups.

-- Roscoe

Re: [dm-crypt] yet another "lost my partition" message

[Cristian KLEIN]

On 15/04/2011 16:15, Roscoe wrote:
> On Fri, Apr 15, 2011 at 11:52 PM, Cristian KLEIN <cristiklein@gmail.com> wrote:
> ...
>> A posteriori, I cannot help wonder why such pretious information isn't
>> kept redundantly. Surely LUKS could have stored the header in 10 random
>> sectors with an easy-to-grep "HERE I AM" banner. Wouldn't this allow
>> users to recover the master-key (and part of the file-system) without
>> compromising security?
> ...
> 
> It's supposed to be fragile and easily destroyed, this is by design.

I think users expect it to be *secure*, i.e., if a laptop gets stolen in
an airport, the user has no worries. I'm not sure users appreciate
"fragile". Personally, this is not what I expect from full-disk encryption.

> Accidently running cryptsetup luksFormat is unfortunate, as is running
> mkfs or dd on the wrong device. Good thing for backups.

Still, mkfs and dd give you a second chance (see testdisk and friends).
Why not luksFormat?

Cristi.

Re: [dm-crypt] yet another "lost my partition" message

[Arno Wagner]

On Fri, Apr 15, 2011 at 04:21:29PM +0200, Cristian KLEIN wrote:
> On 15/04/2011 16:15, Roscoe wrote:
> > On Fri, Apr 15, 2011 at 11:52 PM, Cristian KLEIN <cristiklein@gmail.com> wrote:
> > ...
> >> A posteriori, I cannot help wonder why such pretious information isn't
> >> kept redundantly. Surely LUKS could have stored the header in 10 random
> >> sectors with an easy-to-grep "HERE I AM" banner. Wouldn't this allow
> >> users to recover the master-key (and part of the file-system) without
> >> compromising security?
> > ...
> > 
> > It's supposed to be fragile and easily destroyed, this is by design.
> 
> I think users expect it to be *secure*, i.e., if a laptop gets stolen in
> an airport, the user has no worries. I'm not sure users appreciate
> "fragile". Personally, this is not what I expect from full-disk encryption.

It is not fragile. It is designed to be fast to destroy.

It also is not "full-disk encrypton", it is partition encryption. 
The anti-forensic features require some way to destroy 
data fast and reliable. And a luksFormat will allways wipe the 
headers, no mater how you safeguard them, unless you want to store 
a backlog of old headers and anti-forensic stripes (10-20MB per 
instance) on the filesystem? With all the very serious negative 
security implicatins that has?

Cryptsetup also has adequate safeguards against accidentally
luksFormat-ing your headers and the documentation rather
strongly advises header and data backup.

The mess-up here is by Ubuntu, not cryptsetup. You are
welcome to post here of course and you are welcome to 
argue design mistakes in cryptsetup, we may have 
overlooked something after all. Currently I do not see
that.

> > Accidently running cryptsetup luksFormat is unfortunate, as is running
> > mkfs or dd on the wrong device. Good thing for backups.
> 
> Still, mkfs and dd give you a second chance (see testdisk and friends).
> Why not luksFormat?

Because it cannot without making the design a lot worse. It does
adequately warn you however. The Ubuntu people chose to hide
that warning for whatever reasons. I think I am starting to 
get angry at them....

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] yet another "lost my partition" message

[Arno Wagner]

On Sat, Apr 16, 2011 at 12:15:59AM +1000, Roscoe wrote:
> On Fri, Apr 15, 2011 at 11:52 PM, Cristian KLEIN <cristiklein@gmail.com> wrote:
> ...
> > A posteriori, I cannot help wonder why such pretious information isn't
> > kept redundantly. Surely LUKS could have stored the header in 10 random
> > sectors with an easy-to-grep "HERE I AM" banner. Wouldn't this allow
> > users to recover the master-key (and part of the file-system) without
> > compromising security?
> ...
> 
> It's supposed to be fragile and easily destroyed, this is by design.
> 
> I don't think we should bother with complicated safeguards for people
> doing silly things. That installer interface should be modified long,
> long, long before the on-disk format is.

Indeed. Sorry, but security is always a trade-off against
usability. The trade-off in LUKS is still sound.
 
> Accidently running cryptsetup luksFormat is unfortunate, as is running
> mkfs or dd on the wrong device. Good thing for backups.

It is rather unlikely to accidentally do this with cryptsetup
in interactive mode. It warns you and asks for an uppercase "YES".
This seems to be a serious bug n the Ubuntu installer, which
seems to use --batch-mode on cryptsetup and then do inadequate
(one may say "incompetent") checking and asking. It also
seems not to warn clear enough that this will kill the
existing LUKS partition instead of mounting it. Pretty
bad design. Doing bad things to your partitions and filesystems
should be hard not easy. Don't forget that if you do this
to a FAT filesystem you also lose all data. It is not that 
LUKS is special in that regard.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] yet another "lost my partition" message

[Arno Wagner]

On Fri, Apr 15, 2011 at 03:52:34PM +0200, Cristian KLEIN wrote:
> Hello list,
> 
> > I've just lost my home partition trying to reinstall ubuntu.
> > On the installation process, trying to configure my encrypted home
> > partition, I thought it was going to mount my partition but it was asking
> > for the new key.
> > After that I rebooted and didn't make any other changes to my partition.
> > It seems the partitioner used "cryptsetup luksFormat". Do you think I can
> > still restore my partition?
> > I used the same passphrase configured for my home.
> 
> This is *exactly* what I did yesterday. I thought I was tired or
> something, but seeing that it happened to somebody else too, I start
> thinking that the installer's interface sucks ... badly!

I agree. However, as far as I can tell this is entirely an Ubuntu 
problem, not a cryptsetup problem.  It is also easy to avoid, as 
they could be checking for an existing LUKS header and ask you 
whether you are really, really sure you want to kill it.  Or 
offering to just integrate the existing partition. 

I guess the installer is using --batch-mode which disables any 
checks and questions cryptsetup would do. If you kill your 
LUKS partition with cryptsetup, you have to go through this
process:
-------------------------
R gatewagner:~/f/luks# cryptsetup luksFormat /dev/loop0

WARNING!
========
This will overwrite data on /dev/loop0 irrevocably.

Are you sure? (Type uppercase yes): YES
-------------------------

That should be enough warning, I believe.

> I assume there is no way to recover the original file system. Ubuntu has
> most likely overwritten the LUKS header where the pretious salt is being
> stored. The unencrypted disk most likely looks like random data now.
> According to the FAQ [1], you can still resort to the dm-crypt
> mailing-list to get over the five stages of grief.

This may sound like sarcasm, but it is not. I wrote that and I 
realize the pain is real. This passage however serves a dual 
purpose and the second one is to warn people.
 
> A posteriori, I cannot help wonder why such pretious information isn't
> kept redundantly. 

The FAQ discusses this. It is a design-choice as keeping the
header redundantly lowers security significantly. There is
really no way to keep a backup header without making the
anti-forensic measures ineffective. 

> Surely LUKS could have stored the header in 10 random
> sectors with an easy-to-grep "HERE I AM" banner. Wouldn't this allow
> users to recover the master-key (and part of the file-system) without
> compromising security?

No. Sorry. One case when you wipe the header intentionally is when
your passphrase gets compromised. If the header is in some random 
sectors, you a) would have to remember where it is (making a header
backup is not harder than that) and b) an atacker yould just try
all sectors. Not that much effort.

So, While I am sorry you people got hit by a serious design and 
implementation problem in the Ubuntu installer, the design of
LUKS is not at fault here.


If I find where to complain, I may file an Ubuntu bug report. 
However I do not use Ubuntu. What versions do have this broken
installer?


> Cristi.
> 
> P.S. Could anybody donate 2^256 CPU-seconds?
> P.S.2. What stage of grief am I in? :D

Denial/Anger, but you are trying to compensate with humour. Good!
 
Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] yet another "lost my partition" message

[Jonas Meurer]

[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]

Hey,

On 15/04/2011 Arno Wagner wrote:
> On Fri, Apr 15, 2011 at 03:52:34PM +0200, Cristian KLEIN wrote:
> > I assume there is no way to recover the original file system. Ubuntu has
> > most likely overwritten the LUKS header where the pretious salt is being
> > stored. The unencrypted disk most likely looks like random data now.
> > According to the FAQ [1], you can still resort to the dm-crypt
> > mailing-list to get over the five stages of grief.
> 
> This may sound like sarcasm, but it is not. I wrote that and I 
> realize the pain is real. This passage however serves a dual 
> purpose and the second one is to warn people.
>  
> > A posteriori, I cannot help wonder why such pretious information isn't
> > kept redundantly. 
> 
> The FAQ discusses this. It is a design-choice as keeping the
> header redundantly lowers security significantly. There is
> really no way to keep a backup header without making the
> anti-forensic measures ineffective. 

can you please elaborate on that? why not consider the following:?

luksFormat could choose a random sector at the second half of the
to-be-encrypted device, save the sector number in the primary header,
and store a backup of the primary header at this place.
obviously, all commands that write to the LUKS header would need to
write to primary and backup header in that case.

in case that one accidentially overwrites the primary header (which
is easily done as many device tools write to the first sectors of a
device), you still may grep the device for 'LUKS' in order to find the
backup header. that even would work in most cases if the device is
luksFormated again: the likeliness that luksFormat chooses the same
random sector for its backup header both times, is not very high, and
decreases with the size of device.

as all luks commands write to both primary and backup header, shredding
the device would still be easy enough with luksKillSlot/luksRemoveKey.
nly thing that wouldn't work anymore, is overwriting the first few MB
of the device.

but then, it could be configurable by commandline whether luksFormat
creates a backup header at all, and everybody would be happy, no?

greetings,
 jonas

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [dm-crypt] yet another "lost my partition" message

[Arno Wagner]

On Fri, Apr 15, 2011 at 09:37:52PM +0200, Jonas Meurer wrote:
> Hey,
> 
> On 15/04/2011 Arno Wagner wrote:
> > On Fri, Apr 15, 2011 at 03:52:34PM +0200, Cristian KLEIN wrote:
> > > I assume there is no way to recover the original file system. Ubuntu has
> > > most likely overwritten the LUKS header where the pretious salt is being
> > > stored. The unencrypted disk most likely looks like random data now.
> > > According to the FAQ [1], you can still resort to the dm-crypt
> > > mailing-list to get over the five stages of grief.
> > 
> > This may sound like sarcasm, but it is not. I wrote that and I 
> > realize the pain is real. This passage however serves a dual 
> > purpose and the second one is to warn people.
> >  
> > > A posteriori, I cannot help wonder why such pretious information isn't
> > > kept redundantly. 
> > 
> > The FAQ discusses this. It is a design-choice as keeping the
> > header redundantly lowers security significantly. There is
> > really no way to keep a backup header without making the
> > anti-forensic measures ineffective. 
> 
> can you please elaborate on that? why not consider the following:?
> 
> luksFormat could choose a random sector at the second half of the
> to-be-encrypted device, save the sector number in the primary header,
> and store a backup of the primary header at this place.
> obviously, all commands that write to the LUKS header would need to
> write to primary and backup header in that case.

First, a single sector does not cut it. You need to store the
keyslots as well, giving you 10 or 20MB to store. Second,
LUKS has no idea about the device size and what is to be 
put in it. It can therefore not reserve any sectors. The
sectors ''reserved'' at the beginning are done by ''shifting'' 
the start, but that does not work in any other place.
 
> in case that one accidentially overwrites the primary header (which
> is easily done as many device tools write to the first sectors of a
> device), you still may grep the device for 'LUKS' in order to find the
> backup header. that even would work in most cases if the device is
> luksFormated again: the likeliness that luksFormat chooses the same
> random sector for its backup header both times, is not very high, and
> decreases with the size of device.
> 
> as all luks commands write to both primary and backup header, shredding
> the device would still be easy enough with luksKillSlot/luksRemoveKey.
> nly thing that wouldn't work anymore, is overwriting the first few MB
> of the device.

And that is what kills the anti-forensics. 
 
> but then, it could be configurable by commandline whether luksFormat
> creates a backup header at all, and everybody would be happy, no?

Not possible, see above. 

I really do not see the point. Many filesystems are gone if you
overwrite the start of the partition. It is one of the things 
careful users guard against by backing up. 

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] yet another "lost my partition" message

[Cristian KLEIN]

>> but then, it could be configurable by commandline whether luksFormat
>> creates a backup header at all, and everybody would be happy, no?
>
> I really do not see the point. Many filesystems are gone if you
> overwrite the start of the partition. It is one of the things
> careful users guard against by backing up.

I don't agree with this. I have recovered FAT32, reiserfs and ext4
file-systems which have been formatted. The idea is that files are
(with high probability) stored contiguously. Therefore, a "grep-like"
utility can recover many files, albeit the file-names are lost.

Testdisk is a nice utility which does just that. It scans the disk for
known file headers, then uses knowledge about the file format to find
the end of the file. It can recover pretty much everything that can be
lost (e.g., XML, gzipped XML, JPEG, videos, etc.). Basically, if you
format your drive you don't lose anything. :) I have even recovered
damaged file-systems inside dm-crypt volumes with great success.

In contrast, if the LUKS header is overwritten, you lost everything,
without having any chance of recovery. Testdisk is ready to help you,
but without the master-key the hard-drive looks like random data.

I understand that LUKS' concern is to protect data from unauthorized
access, but couldn't a compromise be found? The randomly-placed backup
header idea proposed by Jonas seems nice. Implementation-wise, if the
position of the backup header is known, then LUKS can just make a
second shift when accessing sectors after that position. Wouldn't this
be an acceptable solution?

Cristi.

Re: [dm-crypt] yet another "lost my partition" message

[Rick Moritz]

> I understand that LUKS' concern is to protect data from unauthorized
> access, but couldn't a compromise be found? The randomly-placed backup
> header idea proposed by Jonas seems nice. Implementation-wise, if the
> position of the backup header is known, then LUKS can just make a
> second shift when accessing sectors after that position. Wouldn't this
> be an acceptable solution?

No.
The answer is, use plan cryptsetup, and don't use LUKS unless you know what you are doing (same goes for plain cryptsetup, but there the worst that can happen is a leaked key and complete reencryption becoming necessary, or a forgotten key, and all data gone).

Sadly LUKS is being peddled as "more comfortable, more awesome, the future", which it is, in some scenarios, but it introduces additional weaknesses (cryptographic and data safety), that have to be managed very thoughtfully.

The real prolbem, that I see, is that cryptsetup -luksblah doesn't quite warn/block enough, and why anyone would use --batch-mode anywhere but in mass deployments of new clients, is beyond comprehension.
Something that might be doable is creating a backup-LUKS header on a separate device, which is itself setup to contain a single key to unlock the main key, which isn't used in the production LUKS.
That way revocation of keys isn't a problem. You will likely just forget the back-up passphrase. Still, that should provide a cryptographically strong means of keeping a LUKS-header around that will unlock your data in case of a major problem. Best to put it onto an USB-key or something and lock it in a safe.
Still, plain dm-crypt is a much simpler solution to avoid single-line-erasing terabytes of data in the blink of an eye.

Also, I'd like to hear from the devs/maintainers, if there's any reason why my proposed approach to a LUKS back-up might be flawed. If not, that might be something to implement in a future version, on luks-create (asking for a second device, and a second passphrase).

Have a nice week-end, all.

Rick

[dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Arno Wagner]

Dear all,

I just sent the email below to ubuntu-bugsquad@lists.ubuntu.com.
I did not find another way to report a bug without installing
Ubuntu (which am an not about to do). If somebody knows a better
way or has Ubuntu installed and can file this directly for the
installer package, please feel free.

I hope this communicates the problem adequately.

Arno

---

Subject: Ubuntu installer kills LUKS partitions without warning


Hi,

first, I don't have Ubuntu and I don't plan on using it.
As I did not find another way that allows me to report
a serious issue without subscribing to something, I use
this venue.


Please forward this to the Ubuntu installer development team.


The dm-crypt mailing list (archive here:
http://dir.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt)
has had several reports that the Ubuntu installer creates 
new LUKS partitions (encrypted partitions) on top of exisiting 
ones without giving adequate warning and at the same time
giving the impression that the existing LUKS partition would be 
activated. The rather clear warning and verification question 
asked by the cryptsetup tool seems to get disabled with the 
"--batch-mode" option.

This causes irreversible loss of all data! Nothing
(in the cryptographically strong sense) can be done
to get the data back!

Checking for an existing LUKS partition is easy,
it has the magic string 'L','U','K','S, 0xBA, 0xBe  
at the beginning. It seems this is not checked for, 
or no additional warning or only inadequate additional 
warning is given if a pre-exisiting LUKS partition 
is about to be permanently and irretrivably erased. 

Please fix the installer ASAP and if you need help doing 
it right, _ask_! We will be happy to help.                             

Regards,
Arno Wagner
Maintainer cryptsetup FAQ

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Claudio Moretti]

[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]

On Fri, Apr 15, 2011 at 18:58, Arno Wagner <arno@wagner.name> wrote:

> Dear all,
>
> I just sent the email below to ubuntu-bugsquad@lists.ubuntu.com.
> I did not find another way to report a bug without installing
> Ubuntu (which am an not about to do). If somebody knows a better
> way or has Ubuntu installed and can file this directly for the
> installer package, please feel free.
>
> I hope this communicates the problem adequately.
>
> Arno
>
> As far as I know (I'm an Ubuntu user, with a little experience) you can
choose between automatic and manual partitioning; partitioning options are
shown here: https://help.ubuntu.com/community/GraphicalInstall and the
partition step in the installation process is fully described here:
https://help.ubuntu.com/10.10/installation-guide/i386/module-details.html#di-partition

If you choose automatic partitioning, you will lose the content of your
disks, but the system warns you; if you choose manual partitioning and
delete the dm-crypt partitions to make new ones, I don't see how the
installer can warn you, aside from telling you you're destroying a partition
(and if you choose manual partitioning, you should know what can happen,
otherwise you shouldn't use it).

In order to use encrypted disks, the installer needs to write the partition
table: before doing it, it warns you that when writing the partition table
changes will be written on the disk and you may lose your data.

In _my_ opinion, the process is quite clear. I see that there may be
problems, but, well (and with this I don't mean to offend anybody, because I
made really stupid mistakes with encrypted partitions) if you don't read
what you're doing, it's not an Ubuntu problem; like if you type that
uppercase YES while luksFormat-ting a partition and then complain that it
should have warned you better...

Claudio

[-- Attachment #2: Type: text/html, Size: 2404 bytes --]

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Arno Wagner]

On Fri, Apr 15, 2011 at 09:38:36PM +0200, Claudio Moretti wrote:
> On Fri, Apr 15, 2011 at 18:58, Arno Wagner <arno@wagner.name> wrote:
> 
> > Dear all,
> >
> > I just sent the email below to ubuntu-bugsquad@lists.ubuntu.com.
> > I did not find another way to report a bug without installing
> > Ubuntu (which am an not about to do). If somebody knows a better
> > way or has Ubuntu installed and can file this directly for the
> > installer package, please feel free.
> >
> > I hope this communicates the problem adequately.
> >
> > Arno
> >
> > As far as I know (I'm an Ubuntu user, with a little experience) you can
> choose between automatic and manual partitioning; partitioning options are
> shown here: https://help.ubuntu.com/community/GraphicalInstall and the
> partition step in the installation process is fully described here:
> https://help.ubuntu.com/10.10/installation-guide/i386/module-details.html#di-partition
> 
> If you choose automatic partitioning, you will lose the content of your
> disks, but the system warns you; if you choose manual partitioning and
> delete the dm-crypt partitions to make new ones, I don't see how the
> installer can warn you, aside from telling you you're destroying a partition
> (and if you choose manual partitioning, you should know what can happen,
> otherwise you shouldn't use it).
> 
> In order to use encrypted disks, the installer needs to write the partition
> table: before doing it, it warns you that when writing the partition table
> changes will be written on the disk and you may lose your data.

Huh? Why does it need to write the partition table? LUKS causes
no such requirement...

> In _my_ opinion, the process is quite clear. I see that there may be
> problems, but, well (and with this I don't mean to offend anybody, because I
> made really stupid mistakes with encrypted partitions) if you don't read
> what you're doing, it's not an Ubuntu problem; like if you type that
> uppercase YES while luksFormat-ting a partition and then complain that it
> should have warned you better...

I think this is something else. But I would not know.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Claudio Moretti]

[-- Attachment #1: Type: text/plain, Size: 3327 bytes --]

>
> > > As far as I know (I'm an Ubuntu user, with a little experience) you can
> > choose between automatic and manual partitioning; partitioning options
> are
> > shown here: https://help.ubuntu.com/community/GraphicalInstall and the
> > partition step in the installation process is fully described here:
> >
> https://help.ubuntu.com/10.10/installation-guide/i386/module-details.html#di-partition
> >
> > If you choose automatic partitioning, you will lose the content of your
> > disks, but the system warns you; if you choose manual partitioning and
> > delete the dm-crypt partitions to make new ones, I don't see how the
> > installer can warn you, aside from telling you you're destroying a
> partition
> > (and if you choose manual partitioning, you should know what can happen,
> > otherwise you shouldn't use it).
> >
> > In order to use encrypted disks, the installer needs to write the
> partition
> > table: before doing it, it warns you that when writing the partition
> table
> > changes will be written on the disk and you may lose your data.
>
> Huh? Why does it need to write the partition table? LUKS causes
> no such requirement...
>

The alternate installer provides you with a "Prepare encrypted disks" option
(something like this, I don't remember the exact name); You create your
partitions and select the "physical volume for encryption" filesystem type ;
then you choose the "Prepare encrypted disk" option and the installer writes
the partition table, asks for which partitions you want to encrypt and makes
you choose passphrases.
Then it shows you new "disks" that are your encrypted partitions; inside of
those "disks" you can create your ext{2,3,4}/FAT/ReiserFS/etc. partitions
(screenshot of what I mean, taken from Google Images:
https://learninginlinux.files.wordpress.com/2008/04/dscf0021mod.jpg)
I think this happens because the right partitions are needed to use the
device mapper and create filesystems in which the system can be installed

>
> > In _my_ opinion, the process is quite clear. I see that there may be
> > problems, but, well (and with this I don't mean to offend anybody,
> because I
> > made really stupid mistakes with encrypted partitions) if you don't read
> > what you're doing, it's not an Ubuntu problem; like if you type that
> > uppercase YES while luksFormat-ting a partition and then complain that it
> > should have warned you better...
>
> I think this is something else. But I would not know.
>
> Then I don't know, either. But I think that when choosing the alternate
installer (which is more advanced than the graphical one) you should know
you're using an "expert" tool..
I'm attaching a few screenshot I found on GI of what the installer does

Main menu:
http://www.linuxlog.org/wp-content/uploads/2011/01/Ubuntu-64-bit-@-2011-01-20-134415.png
Main menu (manual partitioning):
https://help.ubuntu.com/community/Installation/SoftwareRAID?action=AttachFile&do=get&target=ubuntu_raid_06.png
Partition usage list:
http://www.phoronix.net/image.php?id=ubuntu_1010_btrfs&image=ubuntu_1010_btrfs1_med
Partitioning confirmation:
http://members.iinet.net.au/~herman546/p2.2/2_006-confirm-partitions.png
Partition schema with dm-crypt and LVM:
http://static.screencasts.ubuntu.com/videos/20070910_installing_ubuntu_part_2_427x240.png

Hope this may help.

Claudio

[-- Attachment #2: Type: text/html, Size: 4842 bytes --]

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Cristian KLEIN]

> Then I don't know, either. But I think that when choosing the alternate
> installer (which is more advanced than the graphical one) you should know
> you're using an "expert" tool..
> I'm attaching a few screenshot I found on GI of what the installer does

Hello,

Let me elaborate a little bit on what happened. I had a computer with
one ext2 /boot partition and one huge dm-crypt volume, inside which I
had three logical volumes: root, swap and home. My intend was to
install Ubuntu Natty on root, without touching home.

LVM and dm-crypt can only be configured using the alternate Installer,
so I downloaded the daily version of Ubuntu Natty Alternate Installer
and hoped for the best. When I got to the menu asking me to create
partitions / file systems, Ubuntu did not automatically activate the
existing dm-crypt volume. In fact, it didn't even show that the
partition was of type "crypto". IIRC, the partition was just shown to
be unknown. Trying to activate the dm-crypt volume, the only option I
could find was "configure encrypted volumes" [1]. Later I found out
that it only allows to create a new volume, but not activate an
existing one.

I understand that the alternate installer is supposed to be for
experts, but I still think that:
1) Ubuntu failed to detect the existing dm-crypt volume, thus
confusing the user.
2) The installer did not provide an easy-to-find command for
activating the dm-crypt volume.
3) Airbags have been turned off, without the user's knowledge.

Please comment and tell me if I am wrong.

Cheers,
Cristi.

[1] https://help.ubuntu.com/community/Installation/SoftwareRAID?action=AttachFile&do=get&target=ubuntu_raid_06.png

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[PsiStormYamato]

I've never fooled with encrypted LVMs; I always encrypt separate
partitions. I had a feeling that going the LVM route could lead to
problems, and it looks like you've found one.

I brief search turned up this post from 2009, so it's apparently a known
issue. A workaround is presented, but it's probably not something that
most folks would think to try.
http://ubuntuforums.org/archive/index.php/t-1009031.html


On Sat, 2011-04-16 at 18:18 +0200, Cristian KLEIN wrote:

> Hello,
> 
> Let me elaborate a little bit on what happened. I had a computer with
> one ext2 /boot partition and one huge dm-crypt volume, inside which I
> had three logical volumes: root, swap and home. My intend was to
> install Ubuntu Natty on root, without touching home.
> 
> LVM and dm-crypt can only be configured using the alternate Installer,
> so I downloaded the daily version of Ubuntu Natty Alternate Installer
> and hoped for the best. When I got to the menu asking me to create
> partitions / file systems, Ubuntu did not automatically activate the
> existing dm-crypt volume. In fact, it didn't even show that the
> partition was of type "crypto". IIRC, the partition was just shown to
> be unknown. Trying to activate the dm-crypt volume, the only option I
> could find was "configure encrypted volumes" [1]. Later I found out
> that it only allows to create a new volume, but not activate an
> existing one.
> 
> I understand that the alternate installer is supposed to be for
> experts, but I still think that:
> 1) Ubuntu failed to detect the existing dm-crypt volume, thus
> confusing the user.
> 2) The installer did not provide an easy-to-find command for
> activating the dm-crypt volume.
> 3) Airbags have been turned off, without the user's knowledge.
> 
> Please comment and tell me if I am wrong.
> 
> Cheers,
> Cristi.
> 
> [1] https://help.ubuntu.com/community/Installation/SoftwareRAID?action=AttachFile&do=get&target=ubuntu_raid_06.png
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[M Thomas Frederiksen]

[-- Attachment #1: Type: text/plain, Size: 1014 bytes --]

On Sat, Apr 16, 2011 at 12:18, Cristian KLEIN <cristiklein@gmail.com> wrote:

> ...
>
> I understand that the alternate installer is supposed to be for
> experts, but I still think that:
> 1) Ubuntu failed to detect the existing dm-crypt volume, thus
> confusing the user.
> 2) The installer did not provide an easy-to-find command for
> activating the dm-crypt volume.
> 3) Airbags have been turned off, without the user's knowledge.
>
> Please comment and tell me if I am wrong.
>

I've had the same problem with Ubuntu.  It gives you the impression that
you're activating the partition, and then scribbles all over your data.  Any
other type of partition you can activate w/out damage, or reformat.  It's a
horrible installer.  One thing that LUKS might do, is remove this
--batch-mode option (if a header is found).  That way if a LUKS header is
already there, it will not overwrite the header.  If the user really wants
to scribble on the data, make them pop open a console and type YES.

-- 
Cheers,
~Thomas

[-- Attachment #2: Type: text/html, Size: 1359 bytes --]

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Arno Wagner]

On Sat, Apr 16, 2011 at 08:20:02PM -0400, M Thomas Frederiksen wrote:
> On Sat, Apr 16, 2011 at 12:18, Cristian KLEIN <cristiklein@gmail.com> wrote:
> 
> > ...
> >
> > I understand that the alternate installer is supposed to be for
> > experts, but I still think that:
> > 1) Ubuntu failed to detect the existing dm-crypt volume, thus
> > confusing the user.
> > 2) The installer did not provide an easy-to-find command for
> > activating the dm-crypt volume.
> > 3) Airbags have been turned off, without the user's knowledge.
> >
> > Please comment and tell me if I am wrong.


I agree with you. While there are a number of things you could
have done to prevent this problem, the root cause of the 
problem is that the installer made it far too easy to kill your 
LUKS volume.

Still, backup is non-optional with LUKS as well. Remember that
disks have about a 5%/year failure rate (if treated well) and
that about half die without warning.
 
> I've had the same problem with Ubuntu.  It gives you the impression that
> you're activating the partition, and then scribbles all over your data.  Any
> other type of partition you can activate w/out damage, or reformat.  It's a
> horrible installer.  One thing that LUKS might do, is remove this
> --batch-mode option (if a header is found).  That way if a LUKS header is
> already there, it will not overwrite the header.  If the user really wants
> to scribble on the data, make them pop open a console and type YES.

The --batch-mode option is there for a reason. It assumes that
the wrapper process knows what it is doing. It is also necessary
in some circuymstances, so removing it is not the right approach.

What I don't like is that the Ubuntu installer turns off the checks
and then does not supply an adequate replacement for the situation
it is run in. We will see whether anything will come from my complaint.

Arno 
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[PsiStormYamato]

Is this an old issue, or one that only exists in recent versions of
Ubuntu?

I've never had any problems, but I always use Ubuntu's Alternate
install, which I'm guessing would have more textual warnings and whatnot
since it is a text-based installer after all.



On Fri, 2011-04-15 at 18:58 +0200, Arno Wagner wrote

> The dm-crypt mailing list (archive here:
> http://dir.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt)
> has had several reports that the Ubuntu installer creates 
> new LUKS partitions (encrypted partitions) on top of exisiting 
> ones without giving adequate warning and at the same time
> giving the impression that the existing LUKS partition would be 
> activated. The rather clear warning and verification question 
> asked by the cryptsetup tool seems to get disabled with the 
> "--batch-mode" option.

Re: [dm-crypt] Bug Report to Ubuntu regarding dangerous installer

[Arno Wagner]

There is a bug report about this already in the Ubuntu bug
tracking system here, whih was upgraded to "critical" as
result of my bug report. 

http://launchpad.net/bugs/420080

It should get looked at for 11.04 now and hopefully be fixed.
The only thing really needed is a detection of existing 
LUKS headers and a strong warning that you will kill them
by proceeding.

For my side this closes the discussion. Any clunky 
cryptsetup-integrated backup is not a clean option, as
"luksHeaderBackup" works just fine and avoids numerous
possible problems with the other solutions proposed here.
It just has to be used. 

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier