recovery starts from 0 after reboot - normal?

recovery starts from 0 after reboot - normal?

[Tomasz Chmielewski]

I built RAID-5 on a Debian Etch machine running 2.6.22.5 with this command:

mdadm --create /dev/md0 --chunk=64 --level=raid5 --raid-devices=5 
/dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1


After some time, it was synchronized just fine.


I removed one drive, put it back again, readded the drive to the array.

It started rebuilding.
However, if the rebuilding is not 100% done, and I reboot the machine, 
rebuilding starts from 0.

Is it normal?


-- 
Tomasz Chmielewski
http://wpkg.org

Re: recovery starts from 0 after reboot - normal?

[Justin Piszcz]

On Fri, 24 Aug 2007, Tomasz Chmielewski wrote:

> I built RAID-5 on a Debian Etch machine running 2.6.22.5 with this command:
>
> mdadm --create /dev/md0 --chunk=64 --level=raid5 --raid-devices=5 /dev/sda1 
> /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
>
>
> After some time, it was synchronized just fine.
>
>
> I removed one drive, put it back again, readded the drive to the array.
>
> It started rebuilding.
> However, if the rebuilding is not 100% done, and I reboot the machine, 
> rebuilding starts from 0.
>
> Is it normal?
>
>
> -- 
> Tomasz Chmielewski
> http://wpkg.org
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

It should continue (at least that is what I have seen in the past), not 
sure if you need bitmaps or not (I dont use them).

Re: recovery starts from 0 after reboot - normal?

[Tomasz Chmielewski]

Justin Piszcz schrieb:
> 
> 
> On Fri, 24 Aug 2007, Tomasz Chmielewski wrote:
> 
>> I built RAID-5 on a Debian Etch machine running 2.6.22.5 with this 
>> command:
>>
>> mdadm --create /dev/md0 --chunk=64 --level=raid5 --raid-devices=5 
>> /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
>>
>>
>> After some time, it was synchronized just fine.
>>
>>
>> I removed one drive, put it back again, readded the drive to the array.
>>
>> It started rebuilding.
>> However, if the rebuilding is not 100% done, and I reboot the machine, 
>> rebuilding starts from 0.

(...)

> It should continue (at least that is what I have seen in the past), not 
> sure if you need bitmaps or not (I dont use them).

I had the same issue, with 4 drive RAID-10, on another Debian box.

Perhaps, the bitmap is needed then? I guess by default, no internal 
bitmap is added?

# mdadm -X /dev/md0
         Filename : /dev/md0
            Magic : 00000000
mdadm: invalid bitmap magic 0x0, the bitmap file appears to be corrupted
          Version : 0
mdadm: unknown bitmap version 0, either the bitmap file is corrupted or 
you need to upgrade your tools


The fine manual says:

   BITMAP CHANGES
   A write-intent bitmap can be added to, or removed from, an active
   array. Either internal bitmaps, or bitmaps stored in a separate file
   can be added.


However, I'm not sure how to achieve it? How to add an internal bitmap 
to an array without a bitmap?


-- 
Tomasz Chmielewski
http://wpkg.org

Re: recovery starts from 0 after reboot - normal?

[Justin Piszcz]

On Fri, 24 Aug 2007, Tomasz Chmielewski wrote:

> Justin Piszcz schrieb:
>> 
>> 
>> On Fri, 24 Aug 2007, Tomasz Chmielewski wrote:
>> 
>>> I built RAID-5 on a Debian Etch machine running 2.6.22.5 with this 
>>> command:
>>> 
>>> mdadm --create /dev/md0 --chunk=64 --level=raid5 --raid-devices=5 
>>> /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
>>> 
>>> 
>>> After some time, it was synchronized just fine.
>>> 
>>> 
>>> I removed one drive, put it back again, readded the drive to the array.
>>> 
>>> It started rebuilding.
>>> However, if the rebuilding is not 100% done, and I reboot the machine, 
>>> rebuilding starts from 0.
>
> (...)
>
>> It should continue (at least that is what I have seen in the past), not 
>> sure if you need bitmaps or not (I dont use them).
>
> I had the same issue, with 4 drive RAID-10, on another Debian box.
>
> Perhaps, the bitmap is needed then? I guess by default, no internal bitmap is 
> added?
>
> # mdadm -X /dev/md0
>        Filename : /dev/md0
>           Magic : 00000000
> mdadm: invalid bitmap magic 0x0, the bitmap file appears to be corrupted
>         Version : 0
> mdadm: unknown bitmap version 0, either the bitmap file is corrupted or you 
> need to upgrade your tools
>
>
> The fine manual says:
>
>  BITMAP CHANGES
>  A write-intent bitmap can be added to, or removed from, an active
>  array. Either internal bitmaps, or bitmaps stored in a separate file
>  can be added.
>
>
> However, I'm not sure how to achieve it? How to add an internal bitmap to an 
> array without a bitmap?
>
>
> -- 
> Tomasz Chmielewski
> http://wpkg.org
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

That has been discussed on the mailing list before I believe.

        -e ,  --metadata=
               Declare the style of superblock (raid metadata) to be used.  The
               default is 0.90 for --create, and to guess for other operations.
               The default can be overridden by setting the metadata value  for
               the CREATE keyword in mdadm.conf.

Check the archives or ping Neil (especially) if you do not have a backup 
of your environment.

Re: recovery starts from 0 after reboot - normal?

[Tomasz Chmielewski]

Tomasz Chmielewski schrieb:

(...)

> Perhaps, the bitmap is needed then? I guess by default, no internal 
> bitmap is added?
> 
> # mdadm -X /dev/md0
>         Filename : /dev/md0
>            Magic : 00000000
> mdadm: invalid bitmap magic 0x0, the bitmap file appears to be corrupted
>          Version : 0
> mdadm: unknown bitmap version 0, either the bitmap file is corrupted or 
> you need to upgrade your tools
> 
> 
> The fine manual says:
> 
>   BITMAP CHANGES
>   A write-intent bitmap can be added to, or removed from, an active
>   array. Either internal bitmaps, or bitmaps stored in a separate file
>   can be added.
> 
> 
> However, I'm not sure how to achieve it? How to add an internal bitmap 
> to an array without a bitmap?

According to the fine manual, BITMAP CHANGES belong to the grow mode.
So, let's try to do what the manual says - try to add a bitmap to the 
active array:


# mdadm --grow /dev/md0 --bitmap=internal
mdadm: failed to set internal bitmap.

# dmesg
md: couldn't update array info. -16


So, either I don't understand the manual, or there are more caveats in it.


-- 
Tomasz Chmielewski
http://wpkg.org

Re: recovery starts from 0 after reboot - normal?

[Justin Piszcz]

On Fri, 24 Aug 2007, Tomasz Chmielewski wrote:

> Tomasz Chmielewski schrieb:
>
> (...)
>
>> Perhaps, the bitmap is needed then? I guess by default, no internal bitmap 
>> is added?
>> 
>> # mdadm -X /dev/md0
>>         Filename : /dev/md0
>>            Magic : 00000000
>> mdadm: invalid bitmap magic 0x0, the bitmap file appears to be corrupted
>>          Version : 0
>> mdadm: unknown bitmap version 0, either the bitmap file is corrupted or you 
>> need to upgrade your tools
>> 
>> 
>> The fine manual says:
>>
>>   BITMAP CHANGES
>>   A write-intent bitmap can be added to, or removed from, an active
>>   array. Either internal bitmaps, or bitmaps stored in a separate file
>>   can be added.
>> 
>> 
>> However, I'm not sure how to achieve it? How to add an internal bitmap to 
>> an array without a bitmap?
>
> According to the fine manual, BITMAP CHANGES belong to the grow mode.
> So, let's try to do what the manual says - try to add a bitmap to the active 
> array:
>
>
> # mdadm --grow /dev/md0 --bitmap=internal
> mdadm: failed to set internal bitmap.
>
> # dmesg
> md: couldn't update array info. -16
>
>
> So, either I don't understand the manual, or there are more caveats in it.
>
>
> -- 
> Tomasz Chmielewski
> http://wpkg.org
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Check: http://lkml.org/lkml/2007/6/17/235

Re: recovery starts from 0 after reboot - normal?

[Tomasz Chmielewski]

Justin Piszcz schrieb:

>> According to the fine manual, BITMAP CHANGES belong to the grow mode.
>> So, let's try to do what the manual says - try to add a bitmap to the 
>> active array:
>>
>>
>> # mdadm --grow /dev/md0 --bitmap=internal
>> mdadm: failed to set internal bitmap.
>>
>> # dmesg
>> md: couldn't update array info. -16
>>
>>
>> So, either I don't understand the manual, or there are more caveats in 
>> it.

(...)

> Check: http://lkml.org/lkml/2007/6/17/235

It's an external bitmap; I'd prefer an internal one, as my root 
filesystem is on a flash-IDE disk.

However, let's try the external bitmap:

# mdadm --grow /dev/md0 --bitmap=/bitmapfile
mdadm: Cannot set bitmap file for /dev/md0: Device or resource busy


So, it also fails. Perhaps because the array still rebuilds?

A 95644 byte /bitmapfile was created.


I'll try again when the array is done with rebuilding. According to 
/proc/mdstat, it would be 6 more hours.


-- 
Tomasz Chmielewski
http://wpkg.org

Re: recovery starts from 0 after reboot - normal?

[Tomasz Chmielewski]

Tomasz Chmielewski schrieb:
> Justin Piszcz schrieb:
> 
>>> According to the fine manual, BITMAP CHANGES belong to the grow mode.
>>> So, let's try to do what the manual says - try to add a bitmap to the 
>>> active array:
>>>
>>>
>>> # mdadm --grow /dev/md0 --bitmap=internal
>>> mdadm: failed to set internal bitmap.
>>>
>>> # dmesg
>>> md: couldn't update array info. -16
>>>
>>>
>>> So, either I don't understand the manual, or there are more caveats 
>>> in it.
> 
> (...)
> 
>> Check: http://lkml.org/lkml/2007/6/17/235
> 
> It's an external bitmap; I'd prefer an internal one, as my root 
> filesystem is on a flash-IDE disk.
> 
> However, let's try the external bitmap:
> 
> # mdadm --grow /dev/md0 --bitmap=/bitmapfile
> mdadm: Cannot set bitmap file for /dev/md0: Device or resource busy
> 
> 
> So, it also fails. Perhaps because the array still rebuilds?

It looks it only works when the array is in a clean state (or, not 
rebuilding anymore).

Just for reference:

# mdadm --grow --bitmap=/bitmapfile /dev/md0
# dmesg -c
md0: bitmap file is out of date (0 < 43130) -- forcing full recovery
md0: bitmap file is out of date, doing full recovery
md0: bitmap initialized from disk: read 24/24 pages, set 763103 bits, 
status: 0
created bitmap (373 pages) for device md0

# mdadm --grow /dev/md0 --bitmap=internal
mdadm: /dev/md0 already has a bitmap (/bitmapfile)

# mdadm --grow /dev/md0 --bitmap=none

# mdadm --grow /dev/md0 --bitmap=internal
# dmesg -c
md0: bitmap file is out of date (0 < 43130) -- forcing full recovery
md0: bitmap file is out of date, doing full recovery
md0: bitmap initialized from disk: read 12/12 pages, set 381552 bits, 
status: 0
created bitmap (187 pages) for device md0


What are pages/bits in bitmap? It seems to differ, whether internal or 
external is used.

-- 
Tomasz Chmielewski
http://wpkg.org