Superblock V 1.2

Superblock V 1.2

[Martin Cracauer]

I created a new array after upgrading kernel and OS
(2.6.32.25-cracauer and Debian/squeeze, respectively, resulting in
mdadm - v3.1.4 - 31st August 2010)

The new array reads
md0 : active raid5 sdc2[4] sda2[2] sdb2[1] sdd2[0]
      292998144 blocks super 1.2 level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

I figure the "super 1.2" means the new version of the superblock.

Can I read that array if I ever connect the machine to an older
kernel/mdadm? I see that I can control which version to use in
mdadm(8), but I don't get what the advantage of the new format is if I
don't run out of # of components or total capacity.  I can see how
storing the superblock at 4 KB makes it more robust against accidents
that wipe out of the first 512 bytes or similar fun.  I think storing
it at the end will be a pain if you ever have to hexdump recover the
thing, no?

I also wonder why this is the only version announced in /proc/mdstat
if it is what is now the default? Shouldn't it announce the v0.9
blocks?

Is anybody here using the DDF format? I don't plan to get hardware
raid but why not prepare for it?

Thanks
	Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org>   http://www.cons.org/cracauer/

RE: Superblock V 1.2

[Leslie Rhorer]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Martin Cracauer
> Sent: Sunday, November 07, 2010 1:15 PM
> To: linux-raid@vger.kernel.org
> Subject: Superblock V 1.2
> 
> I created a new array after upgrading kernel and OS
> (2.6.32.25-cracauer and Debian/squeeze, respectively, resulting in
> mdadm - v3.1.4 - 31st August 2010)
> 
> The new array reads
> md0 : active raid5 sdc2[4] sda2[2] sdb2[1] sdd2[0]
>       292998144 blocks super 1.2 level 5, 256k chunk, algorithm 2 [4/4]
> [UUUU]
> 
> I figure the "super 1.2" means the new version of the superblock.

	It's one of the new versions, yes.  Your statement suggests there is
only 1 new version.

> Can I read that array if I ever connect the machine to an older
> kernel/mdadm?

	That depends on just how old.  The new superblocks have been
supported for quite some time.  Worst case you will need to upgrade the old
kernel to a newer one.

> I see that I can control which version to use in
> mdadm(8), but I don't get what the advantage of the new format is if I
> don't run out of # of components or total capacity.

	The older 0.90 superblock is still quite serviceable for many uses,
but unless the array is to be booted, I would recommend a 1.x superblock.

> I can see how
> storing the superblock at 4 KB makes it more robust against accidents
> that wipe out of the first 512 bytes or similar fun.  I think storing
> it at the end will be a pain if you ever have to hexdump recover the
> thing, no?

	Each one has its strong points and it drawbacks.  You've got to
decide which one will likely serve you best given your approach to managing
your system.

Re: Superblock V 1.2

[Martin Cracauer]

Thanks, Leslie,

Leslie Rhorer wrote on Sun, Nov 07, 2010 at 03:03:39PM -0600: 
> 
> > Can I read that array if I ever connect the machine to an older
> > kernel/mdadm?
> 
> 	That depends on just how old.  The new superblocks have been
> supported for quite some time.  Worst case you will need to upgrade the old
> kernel to a newer one.

Is mdadm actually involved? If I have a new enough kernel but an old
mdadm, will the kernel code alone be enough to start the array?

I guess it should since it can start the arrays at boot time with no
mdadm involved.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org>   http://www.cons.org/cracauer/

Re: Superblock V 1.2

[John Robinson]

On 07/11/2010 21:08, Martin Cracauer wrote:
> Thanks, Leslie,
>
> Leslie Rhorer wrote on Sun, Nov 07, 2010 at 03:03:39PM -0600:
>>
>>> Can I read that array if I ever connect the machine to an older
>>> kernel/mdadm?
>>
>> 	That depends on just how old.  The new superblocks have been
>> supported for quite some time.  Worst case you will need to upgrade the old
>> kernel to a newer one.
>
> Is mdadm actually involved? If I have a new enough kernel but an old
> mdadm, will the kernel code alone be enough to start the array?
>
> I guess it should since it can start the arrays at boot time with no
> mdadm involved.

That's the other difference between 0.90 and 1.x metadata. In-kernel 
auto-assembly is only available for 0.90, and Neil Brown has made it 
clear that there will never be in-kernel auto-assembly for 1.x, and 
explained at length why. You need mdadm to start 1.x arrays, in your 
initrd if your root filesystem is on such an array. All modern distros 
create suitable initrds automatically. We are moving towards having 
array assembly handled by udev invoking mdadm as devices are discovered.

Cheers,

John.

Re: Superblock V 1.2

[Martin Cracauer]

John Robinson wrote on Sun, Nov 07, 2010 at 09:39:03PM +0000: 
> On 07/11/2010 21:08, Martin Cracauer wrote:
> >Thanks, Leslie,
> >
> >Leslie Rhorer wrote on Sun, Nov 07, 2010 at 03:03:39PM -0600:
> >>
> >>>Can I read that array if I ever connect the machine to an older
> >>>kernel/mdadm?
> >>
> >>	That depends on just how old.  The new superblocks have been
> >>supported for quite some time.  Worst case you will need to upgrade the 
> >>old
> >>kernel to a newer one.
> >
> >Is mdadm actually involved? If I have a new enough kernel but an old
> >mdadm, will the kernel code alone be enough to start the array?
> >
> >I guess it should since it can start the arrays at boot time with no
> >mdadm involved.
> 
> That's the other difference between 0.90 and 1.x metadata. In-kernel 
> auto-assembly is only available for 0.90, and Neil Brown has made it 
> clear that there will never be in-kernel auto-assembly for 1.x, and 
> explained at length why. You need mdadm to start 1.x arrays, in your 
> initrd if your root filesystem is on such an array. All modern distros 
> create suitable initrds automatically. We are moving towards having 
> array assembly handled by udev invoking mdadm as devices are discovered.

That's good it know.  I still think that udev is a plot to destroy
Linux as we know it so I guess I'm gonna recreate this particular
 array :-)

What do you do if you plug in a couple harddrives that have arrays on
them that you do not want assembled? Right now you just set them to a
partition type other than autodetect and you are good.

What do you do when udev maintainers in distributions screw up and
replace good udev entires with bad ones? That happened to me
repeatedly and is the major reason for the attitude above.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org>   http://www.cons.org/cracauer/

Re: Superblock V 1.2

[John Robinson]

On 09/11/2010 16:35, Martin Cracauer wrote:
> John Robinson wrote on Sun, Nov 07, 2010 at 09:39:03PM +0000:
[...]
>> That's the other difference between 0.90 and 1.x metadata. In-kernel
>> auto-assembly is only available for 0.90, and Neil Brown has made it
>> clear that there will never be in-kernel auto-assembly for 1.x, and
>> explained at length why. You need mdadm to start 1.x arrays, in your
>> initrd if your root filesystem is on such an array. All modern distros
>> create suitable initrds automatically. We are moving towards having
>> array assembly handled by udev invoking mdadm as devices are discovered.
>
> That's good it know.  I still think that udev is a plot to destroy
> Linux as we know it so I guess I'm gonna recreate this particular
>   array :-)
>
> What do you do if you plug in a couple harddrives that have arrays on
> them that you do not want assembled? Right now you just set them to a
> partition type other than autodetect and you are good.

You can still do that, but the "right" way to do it (I think) is to use 
the AUTO keyword and only to include the arrays you want assembled at 
boot time in the mdadm.conf that goes into the initrd.

> What do you do when udev maintainers in distributions screw up and
> replace good udev entires with bad ones? That happened to me
> repeatedly and is the major reason for the attitude above.

Oh dear, you've had a bad experience there then. I like udev. I disliked 
the old huge /dev with gazillions of entries for devices you don't have, 
and you still ended up using MAKEDEV or mknod whenever you added 
something new. I did say "moving towards" because udev-based assembly 
isn't 100% there yet, and I don't think any distros are trying to use 
it. But the other great thing we will get with udev is hotplugging, 
including the option to configure in advance auto-rebuilding of arrays, 
something that hardware RAID adapters have always had but we still don't 
for software RAID.

Cheers,

John.

Re: Superblock V 1.2

[Neil Brown]

On Tue, 09 Nov 2010 17:08:49 +0000
John Robinson <john.robinson@anonymous.org.uk> wrote:

> On 09/11/2010 16:35, Martin Cracauer wrote:
> > John Robinson wrote on Sun, Nov 07, 2010 at 09:39:03PM +0000:
> [...]
> >> That's the other difference between 0.90 and 1.x metadata. In-kernel
> >> auto-assembly is only available for 0.90, and Neil Brown has made it
> >> clear that there will never be in-kernel auto-assembly for 1.x, and
> >> explained at length why. You need mdadm to start 1.x arrays, in your
> >> initrd if your root filesystem is on such an array. All modern distros
> >> create suitable initrds automatically. We are moving towards having
> >> array assembly handled by udev invoking mdadm as devices are discovered.
> >
> > That's good it know.  I still think that udev is a plot to destroy
> > Linux as we know it so I guess I'm gonna recreate this particular
> >   array :-)
> >
> > What do you do if you plug in a couple harddrives that have arrays on
> > them that you do not want assembled? Right now you just set them to a
> > partition type other than autodetect and you are good.
> 
> You can still do that, but the "right" way to do it (I think) is to use 
> the AUTO keyword and only to include the arrays you want assembled at 
> boot time in the mdadm.conf that goes into the initrd.

Yes, "auto" is the way to make this do what you want.  The problem is that
different people want different behaviours and it is impossible to please
everyone.
md arrays will normally be tagged with the name of the host they were created
on.  So if you have

  AUTO homehost -all

in mdadm.conf, then it should auto-assemble arrays that were created on
'this' host, and ignore the rest, which might be what you want.

When mdadm does auto-assemble an array that doesn't obviously belong to this
host, it tries to be careful and assembles it 'read-auto', which means that
it is completely read-only until you try to write to the array.  So no
recovery or resync happens.
Once you write to the array, md assumes that you are happy to have the array
assembled on this device, and allows any resync etc to start.

I would be nice if read-auto didn't claim the device, so you would still be
able to access the device 'normally' if you wanted to. But that is hard.
So if you plug some devices in and mdadm thinks like look like they might
belong to it, then it will claim them and you want be able to e.g. mkfs on
them.  But you can be sure they won't be been written to at all, and you can
"mdadm -S /dev/md/whatever" to get full access to your devices back.



> 
> > What do you do when udev maintainers in distributions screw up and
> > replace good udev entires with bad ones? That happened to me
> > repeatedly and is the major reason for the attitude above.
> 
> Oh dear, you've had a bad experience there then. I like udev. I disliked 
> the old huge /dev with gazillions of entries for devices you don't have, 
> and you still ended up using MAKEDEV or mknod whenever you added 
> something new. I did say "moving towards" because udev-based assembly 
> isn't 100% there yet, and I don't think any distros are trying to use 
> it. But the other great thing we will get with udev is hotplugging, 
> including the option to configure in advance auto-rebuilding of arrays, 
> something that hardware RAID adapters have always had but we still don't 
> for software RAID.

Yes, udev has its problems (doesn't everything?) but I think that in general
it is heading in the right direction.

And we are getting close to configurable auto-rebuilding on hot-plug.
mdadm-3.2 will have most of the code in place so some things will work, but
you can probably expect bugs in that release.

NeilBrown



> 
> Cheers,
> 
> John.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Superblock V 1.2

[Martin Cracauer]

Neil Brown wrote on Mon, Nov 15, 2010 at 12:11:58PM +1100: 
> 
> When mdadm does auto-assemble an array that doesn't obviously belong to this
> host, it tries to be careful and assembles it 'read-auto', which means that
> it is completely read-only until you try to write to the array.  So no
> recovery or resync happens.
> Once you write to the array, md assumes that you are happy to have the array
> assembled on this device, and allows any resync etc to start.

Speaking of which, are there any plans to do a true readonly mode?

I missed that quite a few times.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org>   http://www.cons.org/cracauer/

Re: Superblock V 1.2

[Neil Brown]

On Mon, 15 Nov 2010 11:16:46 -0500
Martin Cracauer <cracauer@cons.org> wrote:

> Neil Brown wrote on Mon, Nov 15, 2010 at 12:11:58PM +1100: 
> > 
> > When mdadm does auto-assemble an array that doesn't obviously belong to this
> > host, it tries to be careful and assembles it 'read-auto', which means that
> > it is completely read-only until you try to write to the array.  So no
> > recovery or resync happens.
> > Once you write to the array, md assumes that you are happy to have the array
> > assembled on this device, and allows any resync etc to start.
> 
> Speaking of which, are there any plans to do a true readonly mode?
> 
> I missed that quite a few times.
> 
> Martin

In what does
   mdadm --readonly /dev/mdXX

not meet your needs?
(which is to say:  I think there is a true readonly mode, but I've never
seriously used it so don't know for certain that it is actually useful).

NeilBrown

Re: Superblock V 1.2

[Martin Cracauer]

Neil Brown wrote on Tue, Nov 16, 2010 at 06:47:46AM +1100: 
> On Mon, 15 Nov 2010 11:16:46 -0500
> Martin Cracauer <cracauer@cons.org> wrote:
> 
> > Neil Brown wrote on Mon, Nov 15, 2010 at 12:11:58PM +1100: 
> > > 
> > > When mdadm does auto-assemble an array that doesn't obviously belong to this
> > > host, it tries to be careful and assembles it 'read-auto', which means that
> > > it is completely read-only until you try to write to the array.  So no
> > > recovery or resync happens.
> > > Once you write to the array, md assumes that you are happy to have the array
> > > assembled on this device, and allows any resync etc to start.
> > 
> > Speaking of which, are there any plans to do a true readonly mode?
> > 
> > I missed that quite a few times.
> > 
> > Martin
> 
> In what does
>    mdadm --readonly /dev/mdXX
> 
> not meet your needs?
> (which is to say:  I think there is a true readonly mode, but I've never
> seriously used it so don't know for certain that it is actually useful).

Isn't that the mode that upgrades to read-write when you write?

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@cons.org>   http://www.cons.org/cracauer/

Re: Superblock V 1.2

[Neil Brown]

On Mon, 15 Nov 2010 14:52:20 -0500
Martin Cracauer <cracauer@cons.org> wrote:

> Neil Brown wrote on Tue, Nov 16, 2010 at 06:47:46AM +1100: 
> > On Mon, 15 Nov 2010 11:16:46 -0500
> > Martin Cracauer <cracauer@cons.org> wrote:
> > 
> > > Neil Brown wrote on Mon, Nov 15, 2010 at 12:11:58PM +1100: 
> > > > 
> > > > When mdadm does auto-assemble an array that doesn't obviously belong to this
> > > > host, it tries to be careful and assembles it 'read-auto', which means that
> > > > it is completely read-only until you try to write to the array.  So no
> > > > recovery or resync happens.
> > > > Once you write to the array, md assumes that you are happy to have the array
> > > > assembled on this device, and allows any resync etc to start.
> > > 
> > > Speaking of which, are there any plans to do a true readonly mode?
> > > 
> > > I missed that quite a few times.
> > > 
> > > Martin
> > 
> > In what does
> >    mdadm --readonly /dev/mdXX
> > 
> > not meet your needs?
> > (which is to say:  I think there is a true readonly mode, but I've never
> > seriously used it so don't know for certain that it is actually useful).
> 
> Isn't that the mode that upgrades to read-write when you write?
> 

No.

NeilBrown