Re: [dm-crypt] (More) Questions about LUKS / LVM

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Robbie Smith]

Ah, that makes sense. It clicked with me after reading the paper on the
wiki. When you set up the system, it generates a random "master key",
which each key slot encrypts separately. So unlocking any key slot
unlocks the master key, which is then used to decrypt the disk. That's
rather clever actually.


At the moment I'm only planning to encrypt the onboard HDD of the
laptop, mainly to protect it against unauthorised access. It's a
brand-new machine, so I guess there won't be any noticeable latency with
an i3 or i5 processor. I had a few concerns as at the moment I'm using a
5+ year old Pentium "D" (P4 with hyperthreading?) and I get noticeable
latency with some applications; I didn't want to potentially add to
that.


What are some potential worst-case scenarios? i.e. the system had a hard
reset, either because the power got cut or (somehow) an application
brought the system to a complete halt? How would this affect the
encryption, and could it result in total data loss? 

The FAQ makes mention that the most frequent cause of data loss is
either losing access to the keys or somehow corrupting the LUKS header.
The former I can understand, and "common" sense would dictate to have a
couple of backup keys in secure locations. I am at a loss though as to
how someone could unintentionally corrupt the header though. 

I'm inclined to set up my system with /boot and a LUKS partition, and
then use LVM inside that, so if I decide to rearrange virtual partitions
I won't run the risk of messing up the LUKS header. This also seems like
the simplest setup.

(I keep daily backups of $HOME and of essential system settings, the
rest can be reinstalled if needed, but I'd prefer not to have to spend a
few days recovering everything if I had a hard reset or something like
that.)


Robbie

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Quentin Lefebvre]

On 20/09/2011 12:36, Robbie Smith wrote :
> At the moment I'm only planning to encrypt the onboard HDD of the
> laptop, mainly to protect it against unauthorised access. It's a
> brand-new machine, so I guess there won't be any noticeable latency
> with an i3 or i5 processor.

You guess right.

> What are some potential worst-case scenarios? i.e. the system had a hard
> reset, either because the power got cut or (somehow) an application
> brought the system to a complete halt? How would this affect the
> encryption, and could it result in total data loss?

I've never had any trouble with hard reset or so on, as this is
typically handled by the file-system checks. Encryption is really
transparent.

> The FAQ makes mention that the most frequent cause of data loss is
> either losing access to the keys or somehow corrupting the LUKS header.
> The former I can understand, and "common" sense would dictate to have a
> couple of backup keys in secure locations. I am at a loss though as to
> how someone could unintentionally corrupt the header though.

The FAQ is right here. I actually lost data, each time it was a stupid
mistake, like :
 -> bad 'cryptsetup' invocation, removing (and deleting) a volume
 -> bad 'dd' invocation, writing on the LUKS header.
Indeed, corrupting the LUKS header seems to be the only potential
problem, and you do have to make a mistake to do this (or be unlucky
enough).
AFAIK you can backup the LUKS header, with the security concerns it can
raise.

> I'm inclined to set up my system with /boot and a LUKS partition, and
> then use LVM inside that, so if I decide to rearrange virtual partitions
> I won't run the risk of messing up the LUKS header. This also seems like
> the simplest setup.

This look like a nice design.

Best,
Quentin

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Arno Wagner]

On Tue, Sep 20, 2011 at 08:36:58PM +1000, Robbie Smith wrote:
[...]
> What are some potential worst-case scenarios? i.e. the system had a hard
> reset, either because the power got cut or (somehow) an application
> brought the system to a complete halt? How would this affect the
> encryption, and could it result in total data loss? 

The encryption can be established as long as the header and
at least on ekeyslot are intact. If you cut the power just in the
microsecond while a keyslot is written you would damage that
keyslot. If it was your only one and you do not have a header backup,
then you would have total data loss. That is the only scenario
I can think of. In normal operation, the header is not written.

With an SSD, things are a bit different. Due to the large
internal sector size, the header can be in a sector that
also has data that gets rewritten in it. As sectors are
always written completely, the header then is at risk whenever
that data gets rewritten.

Also keep in mind that HDDs have about a 5%/year rate of failure
(even if some vendors claim 0.5%, but that is under perfect
conditions). A backup of the data is non-optional. The
header-backup is just an additional precaution.

> The FAQ makes mention that the most frequent cause of data loss is
> either losing access to the keys or somehow corrupting the LUKS header.
> The former I can understand, and "common" sense would dictate to have a
> couple of backup keys in secure locations. I am at a loss though as to
> how someone could unintentionally corrupt the header though. 

This is from observations on the mailing-list. It happens pretty
often. And most people are surprised by it. 
 
> I'm inclined to set up my system with /boot and a LUKS partition, and
> then use LVM inside that, so if I decide to rearrange virtual partitions
> I won't run the risk of messing up the LUKS header. This also seems like
> the simplest setup.

Keep in mind that LVM adds to the complexity when you have to do
data recovery when something went wrong. Other that that it sounds
like a good approach.

> (I keep daily backups of $HOME and of essential system settings, the
> rest can be reinstalled if needed, but I'd prefer not to have to spend a
> few days recovering everything if I had a hard reset or something like
> that.)

You will not damage the encrypted data in normal operation.
All the header-damages reported were done during installation,
repartitioning, moving partitions, etc. 

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Milan Broz]

On 09/20/2011 01:47 PM, Arno Wagner wrote:
> The encryption can be established as long as the header and
> at least on ekeyslot are intact. If you cut the power just in the
> microsecond while a keyslot is written you would damage that
> keyslot. If it was your only one and you do not have a header backup,
> then you would have total data loss. That is the only scenario
> I can think of. In normal operation, the header is not written.

Yes, for open and normal activation/deactivation nothing is written
to LUKS header.

And all keyslot operation are done through sync|direct io path
(avoids cache) so it should hit hw immediately.

> Keep in mind that LVM adds to the complexity when you have to do
> data recovery when something went wrong. Other that that it sounds
> like a good approach.

You can say that the same for MD.

Btw LVM has much better recovery abilities than other systems.
Just people are not so familiar with them.

I tried to show it some time ago in some talk,
you can check how easy is to recover complete disaster
(slides are not perfect, missing most of the comments)
http://mbroz.fedorapeople.org/talks/LinuxAlt2009_2/ 

But use of lvm2 is completely optional.
What is complex is incredible complicated lvm2 user interface (CLI),
here I fully agree. But even for notebook, pvmove or online resize
is useful sometimes.

>> (I keep daily backups of $HOME and of essential system settings, the
>> rest can be reinstalled if needed, but I'd prefer not to have to spend a
>> few days recovering everything if I had a hard reset or something like
>> that.)
> 
> You will not damage the encrypted data in normal operation.
> All the header-damages reported were done during installation,
> repartitioning, moving partitions, etc. 

These days is my favorite to LUKS damage bad MD resync
(usually mistake in partition change or MD metadata format change).
(No idea why such problems come in batches to lists :)

Milan

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Arno Wagner]

On Tue, Sep 20, 2011 at 03:13:24PM +0200, Milan Broz wrote:
> On 09/20/2011 01:47 PM, Arno Wagner wrote:
> > The encryption can be established as long as the header and
> > at least on ekeyslot are intact. If you cut the power just in the
> > microsecond while a keyslot is written you would damage that
> > keyslot. If it was your only one and you do not have a header backup,
> > then you would have total data loss. That is the only scenario
> > I can think of. In normal operation, the header is not written.
> 
> Yes, for open and normal activation/deactivation nothing is written
> to LUKS header.
> 
> And all keyslot operation are done through sync|direct io path
> (avoids cache) so it should hit hw immediately.
>
> > Keep in mind that LVM adds to the complexity when you have to do
> > data recovery when something went wrong. Other that that it sounds
> > like a good approach.
> 
> You can say that the same for MD.

Indeed. Especially with the incredible mess that MD superblock
positioning is. I only use superblock format 0.9 for that
reason. Then I at least know it is at the end and that the 
kernel can auto-detect. They should have let it stay 
there. That would have been massively better than the insanity 
of having 3 possible positions.
 
> Btw LVM has much better recovery abilities than other systems.
> Just people are not so familiar with them.
> 
> I tried to show it some time ago in some talk,
> you can check how easy is to recover complete disaster
> (slides are not perfect, missing most of the comments)
> http://mbroz.fedorapeople.org/talks/LinuxAlt2009_2/ 

Nice nonetheless!
 
> But use of lvm2 is completely optional.
> What is complex is incredible complicated lvm2 user interface (CLI),
> here I fully agree. But even for notebook, pvmove or online resize
> is useful sometimes.

But strictly not necessary. I usually test my backup and restore 
procedure when having to resize something. (Yes, I do 2 current 
backups and a very careful verify before.)

> >> (I keep daily backups of $HOME and of essential system settings, the
> >> rest can be reinstalled if needed, but I'd prefer not to have to spend a
> >> few days recovering everything if I had a hard reset or something like
> >> that.)
> > 
> > You will not damage the encrypted data in normal operation.
> > All the header-damages reported were done during installation,
> > repartitioning, moving partitions, etc. 
> 
> These days is my favorite to LUKS damage bad MD resync
> (usually mistake in partition change or MD metadata format change).
> (No idea why such problems come in batches to lists :)

Probably because default metadata format for mdadm is now 1.2
(places superblock at 4k from start). Personally I have never 
damaged anything with MD resync but I only use metadata 0.90. 

Seems to me the kernel folks are not happy woth the > 0.90 
formats either or autodetection would work for them.

No idea what I am going to to when I hit the 2TB size 
restriction on the 0.90 metadata format, but that will still
be a while. 

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Milan Broz]

On 09/20/2011 04:14 PM, Arno Wagner wrote:
> Seems to me the kernel folks are not happy woth the > 0.90 
> formats either or autodetection would work for them.
> 
> No idea what I am going to to when I hit the 2TB size 
> restriction on the 0.90 metadata format, but that will still
> be a while. 

The real problem (and IMHO this caused several LUKS overwrites recently)
is that sometimes it is impossible to distinguish if it is partition or
the whole device it is referring. See mdadm man page:

| 0.90 format superblock.
| This format limits arrays to 28 component devices and limits
| component devices of levels 1 and  greater  to  2  terabytes.
| It  is  also possible for there to be confusion about whether
| the superblock applies to a whole device or just the last partition,
| if that partition starts on a 64K boundary.

(Unfortunately, seems MD wiki with more precise description is still down
because of the kernel.org/linuxfoundation servers security breach...)

Milan

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Alexander Koch]

Am 20.09.2011 13:47, schrieb Arno Wagner:
> With an SSD, things are a bit different. Due to the large
> internal sector size, the header can be in a sector that
> also has data that gets rewritten in it. As sectors are
> always written completely, the header then is at risk whenever
> that data gets rewritten.

Did I get that right: by using the TRIM-support available with kernel
3.1 I risk loosing my LUKS headers at regular use??

No chance to align the payload (data) in such a way that it starts at a
new sector, which then can be TRIMed without loosing the header?


Thanks,

Alex

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Milan Broz]

On 09/20/2011 05:21 PM, Alexander Koch wrote:
> Am 20.09.2011 13:47, schrieb Arno Wagner:
>> With an SSD, things are a bit different. Due to the large
>> internal sector size, the header can be in a sector that
>> also has data that gets rewritten in it. As sectors are
>> always written completely, the header then is at risk whenever
>> that data gets rewritten.
> 
> Did I get that right: by using the TRIM-support available with kernel
> 3.1 I risk loosing my LUKS headers at regular use??

NO. TRIM can be enabled only for data area.

LUKS header is mapped separately from data area. I will never enable
TRIM when manipulating with LUKS metadata area of disk in cryptsetup.
 
(But you cannot say what is going inside of disk itself during keyslot write.)

> No chance to align the payload (data) in such a way that it starts at a
> new sector, which then can be TRIMed without loosing the header?

Not to sector, but to trimmed block size (256/512k usually).
It should be aligned this way. If someone misalign that by mistake,
maybe drive internally need to reallocate some pages but it should never
see TRIM command for this area from kernel.

Milan

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Arno Wagner]

On Tue, Sep 20, 2011 at 05:21:54PM +0200, Alexander Koch wrote:
> Am 20.09.2011 13:47, schrieb Arno Wagner:
> > With an SSD, things are a bit different. Due to the large
> > internal sector size, the header can be in a sector that
> > also has data that gets rewritten in it. As sectors are
> > always written completely, the header then is at risk whenever
> > that data gets rewritten.
> 
> Did I get that right: by using the TRIM-support available with kernel
> 3.1 I risk loosing my LUKS headers at regular use??
> 
> No chance to align the payload (data) in such a way that it starts at a
> new sector, which then can be TRIMed without loosing the header?

No connection with TRIM support. It is just that if the real
block size is larger than the block size you are using,
the SSD will rewrite/reloacte other data on writes smaller
that the real sector size. 

What would need ot be done is to
1. Align LUKS header (partition start) to a real sector
   boundary. 
2. Align the start of the filesystem area in that
   partition to a real sector boundary.

Both are only possible to do realiably when you know the
sector size. All of my 3 SSDs claim 512 byte sectors,
which is almost certainly a lie.

Incidentally, you get a similar problem with 4k sector
drives claiming to be 512B sector drives.

A not quiote certain approach to deal with this is to
align the partition start on a 1MB boundary and the
data area within the LUKS container as well. 

All that said, catastrophic disk failure (from some reports
even more likely for SSDs than normal disks) is still the
main risk to data in LUKS containers, after user error.
 
Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Karl O. Pinc]

On 09/20/2011 12:41:29 PM, Arno Wagner wrote:
> On Tue, Sep 20, 2011 at 05:21:54PM +0200, Alexander Koch wrote:
> > Am 20.09.2011 13:47, schrieb Arno Wagner:
> > > With an SSD, things are a bit different. Due to the large
> > > internal sector size, the header can be in a sector that
> > > also has data that gets rewritten in it. As sectors are
> > > always written completely, the header then is at risk whenever
> > > that data gets rewritten.


> No connection with TRIM support. It is just that if the real
> block size is larger than the block size you are using,
> the SSD will rewrite/reloacte other data on writes smaller
> that the real sector size. 
> 
> What would need ot be done is to
> 1. Align LUKS header (partition start) to a real sector
>    boundary. 
> 2. Align the start of the filesystem area in that
>    partition to a real sector boundary.
> 
> Both are only possible to do realiably when you know the
> sector size. All of my 3 SSDs claim 512 byte sectors,
> which is almost certainly a lie.

Performance is also going to suck if you don't get
alignment right on drives with 4Kb sectors.

FWIW here are the links I use to deal with 4Kb sector issues.

http://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues
http://lists.debian.org/debian-user/2011/01/msg02021.html


Karl <kop@meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Milan Broz]

On 09/20/2011 08:06 PM, Karl O. Pinc wrote:
> Performance is also going to suck if you don't get
> alignment right on drives with 4Kb sectors.

Data area for LUKS was always aligned to 4k offset, so
you should never see problem here.
(Underlying disk partitioning is another issue of course.)

> 
> FWIW here are the links I use to deal with 4Kb sector issues.
> 
> http://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues
(down, for the same reason like all kernel.org services...)

> http://lists.debian.org/debian-user/2011/01/msg02021.html

Not completely up to date. Almost all tools actively use topology
information (from kernel) or default to 1MiB alignment.
(fdisk, parted, lvm, cryptsetup, mdadm, etc)

I would say - if you are using recent distro, you have aligned
this correctly by default (when creating new storage stack).
Do not try to use any magic options, only do this if you are
sure that default result is wrong.
(And report bug if you see it with new versions of tools,
storage people spent a lot of time fixing to work it correctly.)

Milan

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Arno Wagner]

On Tue, Sep 20, 2011 at 08:19:54PM +0200, Milan Broz wrote:
[...]
> > http://lists.debian.org/debian-user/2011/01/msg02021.html
> 
> Not completely up to date. Almost all tools actively use topology
> information (from kernel) or default to 1MiB alignment.
> (fdisk, parted, lvm, cryptsetup, mdadm, etc)

Careful with that. My fdisk (from util-linux-ng 2.17.2)
does 1MB alignment for the first partition, but then it does
no alignment at all for the 3 primary partitions and
while it puts 1MB between the logical partitions, that
also does not result in alignment to anything as far as
I can tell.

I suspect that you have to align yourself for the
partitions. This was with a new, blank disk, but I 
gave sizes in kB, as the partitions go into RAID sets.
Maybe it aligns if you give sizes in MB.

Arno

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Dragan Milivojevic]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

fdisk -u=cylinders -c=dos -H 256 -S 32 /dev/sda

Start the partition at any cylinder (it will always be a multiply of 4MB)
-c=dos (dos compatibility) will make sure that other partitions start at
offset that is a multiply of 4MB.

I didn't test the creation of logical partition.
These days I only use md + lvm.

This methodology also ensures proper partition alignment for hardware raid.

For lvm I use --metadatasize 16320K in case I do a reshape of the underlying
md device.
It's suitable for raid 5 or 6 with 2,4 or 8 data drives and chunk sizes from
64kb to 2048kb.


Careful with that. My fdisk (from util-linux-ng 2.17.2)
> does 1MB alignment for the first partition, but then it does
> no alignment at all for the 3 primary partitions and
> while it puts 1MB between the logical partitions, that
> also does not result in alignment to anything as far as
> I can tell.
>
> I suspect that you have to align yourself for the
> partitions. This was with a new, blank disk, but I
> gave sizes in kB, as the partitions go into RAID sets.
> Maybe it aligns if you give sizes in MB.
>
> Arno
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email:
> arno@wagner.name
> GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25
> 338F
> ----
> Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
>
> If it's in the news, don't worry about it.  The very definition of
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

[-- Attachment #2: Type: text/html, Size: 2164 bytes --]

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Luca Berra]

On Tue, Sep 20, 2011 at 04:14:31PM +0200, Arno Wagner wrote:
>Indeed. Especially with the incredible mess that MD superblock
>positioning is. I only use superblock format 0.9 for that
>reason. Then I at least know it is at the end and that the 
>kernel can auto-detect. They should have let it stay 
>there. That would have been massively better than the insanity 
>of having 3 possible positions.

Please, before speaking against something do some research.

There is no reason on earth to use 0.90 superblocks nowadays.
Even if it seems easier to do that with in-kernel autodetection and
being able to access two halves of a mirror like they were a single
disk, the drawbacks are unacceptable.
In kernel autodetection is not smart enough and can backfire, just plug
a usb or e-sata device with an md superblock with the same md minor
number as your root mirror.
It has been left as is for historical reasons, the proper fix is using
an initramfs, without bloating the kernel with unneeded code.
accessing a raid member like it was a non raid device is also a bad
idea. it is better to force assembly of a degraded array.

Putting metadata at the end also raises a lot of confusion with
partition tables, which are at the start of the disk.

If you create a partition ending at the end of the disk, then add the
partition to an md array, 0.9 metadata would be at the same location
than if you added the whole device to the array.

If you create an raid 0/5/6 array using whole devices then partition it, the
kernel will see a broken partition table on one or more of the component
devices. This extends to any other kind of data besides partition.
Add udev and event-driven activation of disks (especially in its first
very early stages) and people started having the weirdest problems.

Then there are limitations on number of components and array sizes which
are possible to reach, and have already been reached by a number of
users.

The only reason nowadays to keep metadata at the end of a device is a
limitation of grub 1, which cannot boot otherwise.

The latter case is covered by metadata 1.0, which addresses most of the
limitations of 0.9, still keeping metadata at end in order to please
grub.

Then in order to protect the innocent, a schema with metadata at the
start was implemented, first attempt was 1.0 (which imho was a bad
idea).
It puts metadata at the very beginning of disk, which poses metadata at
risk of being overwritten (since that location is often used by mbr and
partition table).

In order to avoid that metadata 1.2 was devised, it is stored at
beginnning of disk, with an offset from the start, in order for it to be
somewath protected. There is also room on the disk to store some form of
boot code.

Consensus now is use metadata 1.2 for almost everything except for
mirrors containing /boot, which need to use 1.0.

Regards,
L.

-- 
Luca Berra -- bluca@comedia.it

Re: [dm-crypt] (More) Questions about LUKS / LVM

[Arno Wagner]

On Mon, Oct 03, 2011 at 08:17:36AM +0200, Luca Berra wrote:
> On Tue, Sep 20, 2011 at 04:14:31PM +0200, Arno Wagner wrote:
> >Indeed. Especially with the incredible mess that MD superblock
> >positioning is. I only use superblock format 0.9 for that
> >reason. Then I at least know it is at the end and that the kernel
> >can auto-detect. They should have let it stay there. That would
> >have been massively better than the insanity of having 3 possible
> >positions.
> 
> Please, before speaking against something do some research.

Why do you assume I have not done research? 
 
> There is no reason on earth to use 0.90 superblocks nowadays.

I do not agree.

> Even if it seems easier to do that with in-kernel autodetection and
> being able to access two halves of a mirror like they were a single
> disk, the drawbacks are unacceptable.

And I do not agree to that either.

> In kernel autodetection is not smart enough and can backfire, just plug
> a usb or e-sata device with an md superblock with the same md minor
> number as your root mirror.

Whyever would I have a MD element on a removable disk, except
on purpose and being very careful with it? 

And are you telling me the Kernel _lies_ to me when it
tells me right during boot, that it is using the UUID 
for array assembly? That would be incredibly bad and outright 
malicious! The following certainly had me assume that UUIDs 
are used as basis for assembly:

md: autorun ...
md: considering sdb9 ...
md:  adding sdb9 ...
md: sdb8 has different UUID to sdb9
md: sdb7 has different UUID to sdb9
md: sdb6 has different UUID to sdb9
md: sdb5 has different UUID to sdb9
md: sdb2 has different UUID to sdb9
md: sdb1 has different UUID to sdb9
md: sdc10 has different UUID to sdb9
md:  adding sdc9 ...
...

> It has been left as is for historical reasons, the proper fix is using
> an initramfs, without bloating the kernel with unneeded code.

I do not use initramfs. It adds massive complexity and
intransparency, and increases maintenance effort without good 
reasons, at least in my set-ups.

I do see the distribution-kernel benefit, but I have not used
them since the early days of kernel 1.1.x.

I also do not use modular kernels, except where no other option
exists, again to decrease complexity. 

> accessing a raid member like it was a non raid device is also a bad
> idea. it is better to force assembly of a degraded array.
> 
> Putting metadata at the end also raises a lot of confusion with
> partition tables, which are at the start of the disk.
> 
> If you create a partition ending at the end of the disk, then add the
> partition to an md array, 0.9 metadata would be at the same location
> than if you added the whole device to the array.

So what? 

> If you create an raid 0/5/6 array using whole devices then partition it, the
> kernel will see a broken partition table on one or more of the component
> devices. This extends to any other kind of data besides partition.
> Add udev and event-driven activation of disks (especially in its first
> very early stages) and people started having the weirdest problems.

Well, If people are not competent to remove an md superblock, that 
can of course have any kind of bad effects. However, this is not a 
low-competence area to be messing around in.

Incidentally, I think udev also is a mess that creates significantly
more problems than it solves. Automagic gone wrong. Replacing something
simple with something complex needs a _very_ _good_ reason, especially
in infrastructure. It seems to me some people in kernel design have
never heard of the "Second System Effect" and are making beginners
mistakes. (As designers, not as coders.)
 
> Then there are limitations on number of components and array sizes which
> are possible to reach, and have already been reached by a number of
> users.

I agree on that. Design mistake, the same old story all around, 
people never learn or look more than a few years ahead. Or back. 

> The only reason nowadays to keep metadata at the end of a device is a
> limitation of grub 1, which cannot boot otherwise.

Your criticism of kernel autodetection is unconvincing. I have
been using it for about a decade now in numerous installations.
I like it and never had any problems with it.

Give me autodetection with the other formats and maybe I will switch. 

And let me add that conceptually, automatic RAID assembly
is definitiely the task of the RAID controller. Here that 
would be the md driver in the kernel (or the kernel itself),
not some external scripting or tool. 

> The latter case is covered by metadata 1.0, which addresses most of the
> limitations of 0.9, still keeping metadata at end in order to please
> grub.
> 
> Then in order to protect the innocent, a schema with metadata at the
> start was implemented, first attempt was 1.0 (which imho was a bad
> idea).
>
> It puts metadata at the very beginning of disk, which poses metadata at
> risk of being overwritten (since that location is often used by mbr and
> partition table).

And from the number of people that manage to trash their LUKS 
headers, this is a real risk. However, the fix with the offset
is just a nightmare as it breaks layering.

Never break something fundamental unless there is absolutely 
no other option! 

The principles of least surprise and simplicity are fundamental 
to all engineering and ignoring them is a recipe for disaster!

Fixing one bad decision (end placement) with another bad
decision (start placement which breaks grub) and then a third 
bad decision (offset placement that still breaks grub and in 
addition breaks layering) shows fundamental problems in the 
dev team. 

Better to lose some md components now and then, that to lose the 
overall hierachical layering and surprise competent folks. 
That is how you create huge disasters. The history of engineering
failure is full of them.

The incompetent will always be surprised, even by obvious things. 
Trying to fix that is just stupid. 

> In order to avoid that metadata 1.2 was devised, it is stored at
> beginnning of disk, with an offset from the start, in order for it to be
> somewath protected. There is also room on the disk to store some form of
> boot code.
> 
> Consensus now is use metadata 1.2 for almost everything except for
> mirrors containing /boot, which need to use 1.0.

And there is the mess. It would be very hard to do this any
worse and have it still working. To misquote "Those that
prefer a little safety over architectural soundness 
shall neither have safety nor architectural soundness."

It is far better to have one format in one place or at least 
several similar formats in the same place, than having them 
all over the device, even if that one place is not optimal.

We have seen people here desparately trying to figure out 
where their MD superblocks where, all the while juggeling 
partitioning, LUKS headers and LVM headers as well, with some 
of them partially or fully overwritten and, to make matters 
worse, intransparent magic hidden in the initramfs. At least 
the other superblocks are all in a consistent place, only the 
md-folks have gone off the deep end.

Sorry, my original comment stands. This is a mess and shows
bad decision making, possibly due to inexperience.

At this time, format 0.90 is the best option (or rather
"least bad") and the efforts of fixing it have been an
impressive failure.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier