linux raid wiki - backup files

linux raid wiki - backup files

[Wols Lists]

I'm updating the mdadm overview at the moment, and I'm giving examples
for changing raid levels etc.

Is a backup file still required? I get the impression with current
kernels and mdadm, any required backup is put in the space left by "data
offset", or if you're adding a disk it stores it in the space on that disk.

Okay, I'll have to document that older versions still need it, but if
that's not needed any more, I'd rather not emphasize it.

I'd also like to confirm my understanding of how a resize works ...

If the number of raid devices is increased, am I right in thinking that
a new stripe one is created from the old stripes one and two, then
written, then the new stripe two is created from the old two and three,
etc etc? In effect, the data slumps down from the old array into the new?

And of course the reverse when a device is removed - it starts with the
highest stripe, which gets pulled upwards so the old array gets pulled
up into the new?

(And during the reshape, you have a marker of the current stripe being
updated, so any stripes above that get read from one array, and below
they get read from the other.)

Cheers,
Wol

Re: linux raid wiki - backup files

[Phil Turmel]

On 11/19/2016 11:03 AM, Wols Lists wrote:
> I'm updating the mdadm overview at the moment, and I'm giving examples
> for changing raid levels etc.
> 
> Is a backup file still required? I get the impression with current
> kernels and mdadm, any required backup is put in the space left by "data
> offset", or if you're adding a disk it stores it in the space on that disk.

You can't change the data offset of zero for metadata v0.90 and v1.0,
the latter of which isn't obsolete.  So a backup file would be needed
for many --grow operations on those arrays.

> I'd also like to confirm my understanding of how a resize works ...

Just to clarify:  --grow with a change in number of devices or layout is
called a reshape, not a resize, even though the size may change.  Just
changing the amount of space used per device is a "resize".  A simple
resize never requires a backup file, as no chunks will move.

> If the number of raid devices is increased, am I right in thinking that
> a new stripe one is created from the old stripes one and two, then
> written, then the new stripe two is created from the old two and three,
> etc etc? In effect, the data slumps down from the old array into the new?
> 
> And of course the reverse when a device is removed - it starts with the
> highest stripe, which gets pulled upwards so the old array gets pulled
> up into the new?

Yes.  But the new stripes lay on top of the old stripes, unless you move
the data offset.  Which is why a backup file holds the old stripe just
in case.  If you can move the offset, you use the lower offset for the
lower addresses in the array, and the higher offset for the higher
addresses, on either side of the reshape position.

> (And during the reshape, you have a marker of the current stripe being
> updated, so any stripes above that get read from one array, and below
> they get read from the other.)

Yes.  And the stripe being worked on is "frozen" to prevent an upper
layer from interfering.

Phil

Re: linux raid wiki - backup files

[Wols Lists]

On 20/11/16 00:27, Phil Turmel wrote:
> Yes.  But the new stripes lay on top of the old stripes, unless you move
> the data offset.  Which is why a backup file holds the old stripe just
> in case.  If you can move the offset, you use the lower offset for the
> lower addresses in the array, and the higher offset for the higher
> addresses, on either side of the reshape position.

Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

But if we have a data offset with v1.2, a reshape will use that space if
it can rather than needing a backup file?

(And changing topic slightly, that space is also used for an internal
bitmap?)

Cheers,
Wol

Re: linux raid wiki - backup files

[Phil Turmel]

On 11/20/2016 09:48 AM, Wols Lists wrote:
> On 20/11/16 00:27, Phil Turmel wrote:
>> Yes.  But the new stripes lay on top of the old stripes, unless you move
>> the data offset.  Which is why a backup file holds the old stripe just
>> in case.  If you can move the offset, you use the lower offset for the
>> lower addresses in the array, and the higher offset for the higher
>> addresses, on either side of the reshape position.
> 
> Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.
> 
> But if we have a data offset with v1.2, a reshape will use that space if
> it can rather than needing a backup file?

If there's room to move the entire data area (direction varies with
operation), yes.

> (And changing topic slightly, that space is also used for an internal
> bitmap?)

And bad block log.

Phil

Re: linux raid wiki - backup files

[Anthony Youngman]

On 21/11/16 14:07, Phil Turmel wrote:
> On 11/20/2016 09:48 AM, Wols Lists wrote:
>> On 20/11/16 00:27, Phil Turmel wrote:
>>> Yes.  But the new stripes lay on top of the old stripes, unless you move
>>> the data offset.  Which is why a backup file holds the old stripe just
>>> in case.  If you can move the offset, you use the lower offset for the
>>> lower addresses in the array, and the higher offset for the higher
>>> addresses, on either side of the reshape position.
>>
>> Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

Having looked at the man page, this now seems obvious - the superblock 
is at the end, so the data offset is 0. But for a 1.0 array, could we 
create a data offset?

(So, if we created a data offset, we could then move the superblock and 
convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to 
me like it shouldn't be that hard ... ?)
>>
>> But if we have a data offset with v1.2, a reshape will use that space if
>> it can rather than needing a backup file?
>
I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? 
And if we (can) create a decent data offset, we can then use that in 
exactly the same way as with v1.2?

Cheers,
Wol

Re: linux raid wiki - backup files

[Phil Turmel]

On 11/21/2016 03:22 PM, Anthony Youngman wrote:
> Having looked at the man page, this now seems obvious - the superblock
> is at the end, so the data offset is 0. But for a 1.0 array, could we
> create a data offset?
> 
> (So, if we created a data offset, we could then move the superblock and
> convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to
> me like it shouldn't be that hard ... ?)

I suppose it would be possible, but defeats the purpose of having data
offset == 0: making non-parity array contents directly readable outside
the array.  Commonly used to raid boot partitions but still have grub
able to read them directly.  Less of an issue today with grub2, I
suppose (but I don't use bootloaders anymore, so I'm not a good resource
for that).

>>> But if we have a data offset with v1.2, a reshape will use that space if
>>> it can rather than needing a backup file?
>>
> I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of?
> And if we (can) create a decent data offset, we can then use that in
> exactly the same way as with v1.2?

V1.1 and v1.2 are identical except for the superblock offset (one 4k
block difference).  v1.1 reshapes just like v1.2.

Phil

Re: linux raid wiki - backup files

[Anthony Youngman]

On 21/11/16 21:02, Phil Turmel wrote:
> On 11/21/2016 03:22 PM, Anthony Youngman wrote:
>> Having looked at the man page, this now seems obvious - the superblock
>> is at the end, so the data offset is 0. But for a 1.0 array, could we
>> create a data offset?
>>
>> (So, if we created a data offset, we could then move the superblock and
>> convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to
>> me like it shouldn't be that hard ... ?)
>
> I suppose it would be possible, but defeats the purpose of having data
> offset == 0: making non-parity array contents directly readable outside
> the array.  Commonly used to raid boot partitions but still have grub
> able to read them directly.  Less of an issue today with grub2, I
> suppose (but I don't use bootloaders anymore, so I'm not a good resource
> for that).

Only works for mirrors ...

So basically, there's no point using 1.0 for any form of parity raid. In 
which case, if you convert a mirror to parity, it would be nice to be 
able to create said offset and move the superblock.
>
>>>> But if we have a data offset with v1.2, a reshape will use that space if
>>>> it can rather than needing a backup file?
>>>
>> I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of?
>> And if we (can) create a decent data offset, we can then use that in
>> exactly the same way as with v1.2?
>
> V1.1 and v1.2 are identical except for the superblock offset (one 4k
> block difference).  v1.1 reshapes just like v1.2.
>
Any reason for that difference? How big is the superblock? Did v1.1 fill 
most of the first 4K and just leave not much room with a default 4K data 
offset initially?

> Phil
>
Cheers,
Wol

Re: linux raid wiki - backup files

[Phil Turmel]

On 11/21/2016 04:30 PM, Anthony Youngman wrote:
> So basically, there's no point using 1.0 for any form of parity raid. In
> which case, if you convert a mirror to parity, it would be nice to be
> able to create said offset and move the superblock.

Not for new parity arrays, no.  v1.0 is what you get if you convert an
old v0.90 array to v1.x using --assemble --update=metadata.  I simply
don't know if a v1.0 superblock will run with a non-zero data offset.
That would have to be an intermediate step before the superblock and the
rest of the metadata could be relocated.

>> V1.1 and v1.2 are identical except for the superblock offset (one 4k
>> block difference).  v1.1 reshapes just like v1.2.
>>
> Any reason for that difference? How big is the superblock? Did v1.1 fill
> most of the first 4K and just leave not much room with a default 4K data
> offset initially?

v1.1 places the superblock in the first sector of the device.  v1.2
skips the first 4k, making such arrays resistant to destruction by
accidental creation of MBR partition tables or similar crises.

Phil

Re: linux raid wiki - backup files

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

On Tue, Nov 22 2016, Anthony Youngman wrote:

> On 21/11/16 14:07, Phil Turmel wrote:
>> On 11/20/2016 09:48 AM, Wols Lists wrote:
>>> On 20/11/16 00:27, Phil Turmel wrote:
>>>> Yes.  But the new stripes lay on top of the old stripes, unless you move
>>>> the data offset.  Which is why a backup file holds the old stripe just
>>>> in case.  If you can move the offset, you use the lower offset for the
>>>> lower addresses in the array, and the higher offset for the higher
>>>> addresses, on either side of the reshape position.
>>>
>>> Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.
>
> Having looked at the man page, this now seems obvious - the superblock 
> is at the end, so the data offset is 0. But for a 1.0 array, could we 
> create a data offset?

Yes.  But usually the purpose for using 1.0 is to have data_offset == 0,
so you might not want to.

>
> (So, if we created a data offset, we could then move the superblock and 
> convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to 
> me like it shouldn't be that hard ... ?)

It would be quite easy to extend "--update=metadata" to change the
version once the data_offset had been changed.

>>>
>>> But if we have a data offset with v1.2, a reshape will use that space if
>>> it can rather than needing a backup file?
>>
> I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? 
> And if we (can) create a decent data offset, we can then use that in 
> exactly the same way as with v1.2?

1.0 defaults to no data_offset.
1.1 uses the safe choice function as 1.2.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: linux raid wiki - backup files

[Anthony Youngman]

Okay, after all that I've ended up not mentioning backup files at all. I 
get the feeling that in pretty much all modern scenarios they are not 
required. If we have v1.1 or v1.2 there should be enough spare metadata 
space, and a modern system shouldn't be using v1.0. It's only of 
interest if you are doing a bios boot off a mirror with no initramfs. 
Not a likely option for a system being set up today. And from the man 
page, mdadm leaves 128K at the end of each device - would a v1.0 array 
have that? Would that be used for backup?

I probably ought to work out how I can get it to fit neatly in my mdadm 
page, especially if it's only needed for older setups.

Can you take a look at the page and make sure it's not got anything 
obviously wrong ...

https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm

And then I'll try and sort out a page on finding lost arrays - we've had 
a few on the list recently ...

Cheers,
Wol

On 21/11/16 22:45, NeilBrown wrote:
> On Tue, Nov 22 2016, Anthony Youngman wrote:
>
>> On 21/11/16 14:07, Phil Turmel wrote:
>>> On 11/20/2016 09:48 AM, Wols Lists wrote:
>>>> On 20/11/16 00:27, Phil Turmel wrote:
>>>>> Yes.  But the new stripes lay on top of the old stripes, unless you move
>>>>> the data offset.  Which is why a backup file holds the old stripe just
>>>>> in case.  If you can move the offset, you use the lower offset for the
>>>>> lower addresses in the array, and the higher offset for the higher
>>>>> addresses, on either side of the reshape position.
>>>>
>>>> Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.
>>
>> Having looked at the man page, this now seems obvious - the superblock
>> is at the end, so the data offset is 0. But for a 1.0 array, could we
>> create a data offset?
>
> Yes.  But usually the purpose for using 1.0 is to have data_offset == 0,
> so you might not want to.
>
>>
>> (So, if we created a data offset, we could then move the superblock and
>> convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to
>> me like it shouldn't be that hard ... ?)
>
> It would be quite easy to extend "--update=metadata" to change the
> version once the data_offset had been changed.
>
>>>>
>>>> But if we have a data offset with v1.2, a reshape will use that space if
>>>> it can rather than needing a backup file?
>>>
>> I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of?
>> And if we (can) create a decent data offset, we can then use that in
>> exactly the same way as with v1.2?
>
> 1.0 defaults to no data_offset.
> 1.1 uses the safe choice function as 1.2.
>
> NeilBrown
>