mdadm 3.1.1: level change won't start

mdadm 3.1.1: level change won't start

[Kristleifur Daðason]

Hi all,

I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html

I have three live drives and three spares added to the array. When I
initialize the command, mdadm does the initial checks and aborts with
a "cannot set device shape" without doing anything to the array.

Following are some md stats and growth command output:

___

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
      2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
      bitmap: 1/350 pages [4KB], 2048KB chunk

$ mdadm --detail --scan
ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
UUID=da547022:042a6f68:d5fe251e:5e89f263

$ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
--backup-file=/root/backup.md1_to_r6
mdadm: metadata format 1.10 unknown, ignored.
mdadm: metadata format 1.10 unknown, ignored.
mdadm level of /dev/md_d1 changed to raid6
mdadm: Need to backup 1024K of critical section..
mdadm: Cannot set device shape for /dev/md_d1
mdadm: aborting level change
___


Three questions -

1. What does the stuff about "metadata format 1.10 unknown" mean?
Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
disrepancy between mdsat, --detail and --grow output.

2. Am I doing something wrong? :)

3. How can I get more info about what is causing the failure to
initialize the growth?

Thanks!

-- Kristleifur


(P.s., I'm cc-ing this directly to Neil Brown as the maintainer of
mdadm. I hope that isn't rude of me.)
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Michael Evans]

On Sun, Dec 20, 2009 at 7:41 PM, Kristleifur Daðason
<kristleifur@gmail.com> wrote:
> Hi all,
>
> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
>
> I have three live drives and three spares added to the array. When I
> initialize the command, mdadm does the initial checks and aborts with
> a "cannot set device shape" without doing anything to the array.
>
> Following are some md stats and growth command output:
>
> ___
>
> $ cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
>       2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
>       bitmap: 1/350 pages [4KB], 2048KB chunk
>
> $ mdadm --detail --scan
> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
> UUID=da547022:042a6f68:d5fe251e:5e89f263
>
> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
> --backup-file=/root/backup.md1_to_r6
> mdadm: metadata format 1.10 unknown, ignored.
> mdadm: metadata format 1.10 unknown, ignored.
> mdadm level of /dev/md_d1 changed to raid6
> mdadm: Need to backup 1024K of critical section..
> mdadm: Cannot set device shape for /dev/md_d1
> mdadm: aborting level change
> ___
>
>
> Three questions -
>
> 1. What does the stuff about "metadata format 1.10 unknown" mean?
> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
> disrepancy between mdsat, --detail and --grow output.
>
> 2. Am I doing something wrong? :)
>
> 3. How can I get more info about what is causing the failure to
> initialize the growth?
>
> Thanks!
>
> -- Kristleifur
>
>
> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of
> mdadm. I hope that isn't rude of me.)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2
 Maybe the extra 1 or 0 at the end refers to a reshape in progress?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Kristleifur Daðason]

On Mon, Dec 21, 2009 at 4:10 AM, Michael Evans <mjevans1983@gmail.com> wrote:
> On Sun, Dec 20, 2009 at 7:41 PM, Kristleifur Daðason
> <kristleifur@gmail.com> wrote:
>> Hi all,
>>
>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
>>
>> I have three live drives and three spares added to the array. When I
>> initialize the command, mdadm does the initial checks and aborts with
>> a "cannot set device shape" without doing anything to the array.
>>
>> Following are some md stats and growth command output:
>>
>> ___
>>
>> $ cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
>>       2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
>>       bitmap: 1/350 pages [4KB], 2048KB chunk
>>
>> $ mdadm --detail --scan
>> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
>> UUID=da547022:042a6f68:d5fe251e:5e89f263
>>
>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
>> --backup-file=/root/backup.md1_to_r6
>> mdadm: metadata format 1.10 unknown, ignored.
>> mdadm: metadata format 1.10 unknown, ignored.
>> mdadm level of /dev/md_d1 changed to raid6
>> mdadm: Need to backup 1024K of critical section..
>> mdadm: Cannot set device shape for /dev/md_d1
>> mdadm: aborting level change
>> ___
>>
>>
>> Three questions -
>>
>> 1. What does the stuff about "metadata format 1.10 unknown" mean?
>> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
>> disrepancy between mdsat, --detail and --grow output.
>>
>> 2. Am I doing something wrong? :)
>>
>> 3. How can I get more info about what is causing the failure to
>> initialize the growth?
>>
>> Thanks!
>>
>> -- Kristleifur
>>
>>
>> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of
>> mdadm. I hope that isn't rude of me.)
>> --
>
>
> I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2
>  Maybe the extra 1 or 0 at the end refers to a reshape in progress?
>

Thanks for the reply. FWIW, I intended the array to have metadata
format 1.1 when I created it. I've noticed these "1.01" messages
before, so I believe it's not due to the reshape. However, I'm
thinking the funny metadata might be interfering with the reshape
somehow.

Hmm ...
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Michael Evans]

On Sun, Dec 20, 2009 at 8:39 PM, Kristleifur Daðason
<kristleifur@gmail.com> wrote:
> On Mon, Dec 21, 2009 at 4:10 AM, Michael Evans <mjevans1983@gmail.com> wrote:
>> On Sun, Dec 20, 2009 at 7:41 PM, Kristleifur Daðason
>> <kristleifur@gmail.com> wrote:
>>> Hi all,
>>>
>>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
>>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
>>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
>>>
>>> I have three live drives and three spares added to the array. When I
>>> initialize the command, mdadm does the initial checks and aborts with
>>> a "cannot set device shape" without doing anything to the array.
>>>
>>> Following are some md stats and growth command output:
>>>
>>> ___
>>>
>>> $ cat /proc/mdstat
>>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>>> [raid4] [raid10]
>>> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
>>>       2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
>>>       bitmap: 1/350 pages [4KB], 2048KB chunk
>>>
>>> $ mdadm --detail --scan
>>> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
>>> UUID=da547022:042a6f68:d5fe251e:5e89f263
>>>
>>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
>>> --backup-file=/root/backup.md1_to_r6
>>> mdadm: metadata format 1.10 unknown, ignored.
>>> mdadm: metadata format 1.10 unknown, ignored.
>>> mdadm level of /dev/md_d1 changed to raid6
>>> mdadm: Need to backup 1024K of critical section..
>>> mdadm: Cannot set device shape for /dev/md_d1
>>> mdadm: aborting level change
>>> ___
>>>
>>>
>>> Three questions -
>>>
>>> 1. What does the stuff about "metadata format 1.10 unknown" mean?
>>> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
>>> disrepancy between mdsat, --detail and --grow output.
>>>
>>> 2. Am I doing something wrong? :)
>>>
>>> 3. How can I get more info about what is causing the failure to
>>> initialize the growth?
>>>
>>> Thanks!
>>>
>>> -- Kristleifur
>>>
>>>
>>> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of
>>> mdadm. I hope that isn't rude of me.)
>>> --
>>
>>
>> I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2
>>  Maybe the extra 1 or 0 at the end refers to a reshape in progress?
>>
>
> Thanks for the reply. FWIW, I intended the array to have metadata
> format 1.1 when I created it. I've noticed these "1.01" messages
> before, so I believe it's not due to the reshape. However, I'm
> thinking the funny metadata might be interfering with the reshape
> somehow.
>
> Hmm ...
>

This is odd; cat /proc/mdstat shows 1.0 and 1.1 for my metadata
formats; however using --detail --scan  I _DO_ see 1.00 and 1.01 on my
system.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1363 bytes --]

On Mon Dec 21, 2009 at 03:41:33AM +0000, Kristleifur Daðason wrote:

> Hi all,
> 
> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
> 
> I have three live drives and three spares added to the array. When I
> initialize the command, mdadm does the initial checks and aborts with
> a "cannot set device shape" without doing anything to the array.
> 
> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
> --backup-file=/root/backup.md1_to_r6
> mdadm: metadata format 1.10 unknown, ignored.
> mdadm: metadata format 1.10 unknown, ignored.
> mdadm level of /dev/md_d1 changed to raid6
> mdadm: Need to backup 1024K of critical section..
> mdadm: Cannot set device shape for /dev/md_d1
> mdadm: aborting level change
> 
I don't think you can grow and reshape simultaneously.  I think you'll
need to do this as two operations - either grow to a 5 disk RAID5 then
reshape to a 6 disk RAID6, or reshape to a 4 disk RAID6 then grow.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: mdadm 3.1.1: level change won't start

[Kristleifur Daðason]

> On Mon Dec 21, 2009 at 03:41:33AM +0000, Kristleifur Daðason wrote:
>
>> Hi all,
>>
>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
>>
>> I have three live drives and three spares added to the array. When I
>> initialize the command, mdadm does the initial checks and aborts with
>> a "cannot set device shape" without doing anything to the array.
>>
>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
>> --backup-file=/root/backup.md1_to_r6
>> mdadm: metadata format 1.10 unknown, ignored.
>> mdadm: metadata format 1.10 unknown, ignored.
>> mdadm level of /dev/md_d1 changed to raid6
>> mdadm: Need to backup 1024K of critical section..
>> mdadm: Cannot set device shape for /dev/md_d1
>> mdadm: aborting level change
>>

On Mon, Dec 21, 2009 at 7:25 AM, Michael Evans <mjevans1983@gmail.com> wrote:
> This is odd; cat /proc/mdstat shows 1.0 and 1.1 for my metadata
> formats; however using --detail --scan  I _DO_ see 1.00 and 1.01 on my
> system.
>

On Mon, Dec 21, 2009 at 9:35 AM, Robin Hill <robin@robinhill.me.uk> wrote:
> I don't think you can grow and reshape simultaneously.  I think you'll
> need to do this as two operations - either grow to a 5 disk RAID5 then
> reshape to a 6 disk RAID6, or reshape to a 4 disk RAID6 then grow.
>

Thanks for the replies, Michael and Robin.

Re. metadata version number discrepancy, I'm going to assume for now
that it doesn't affect my reshape, and take it up later in its own
list thread.

Robin, it makes sense that I can't actually grow and reshape
simultaneously - it fits to the mdadm halt I'm seeing - but I did note
Neil Brown saying here >> http://neil.brown.name/blog/20090817000931
<< that

--- "One way to avoid the extra cost of doing the backup [reshape] is
to increase the number of devices at the same time. e.g. if you had a
4-drive RAID5 you could convert it to a 6-drive RAID6 with a command
like mdadm --grow /dev/md0 --level=6 --raid-disk=6 then it will
combine the increase in the number of devices with the change to the
layout and will copy every block only once. " ---

I'm OK with doing the grow and reshape in two separate steps, but the
indication that it should be possible in one step makes me a bit
anxious about the health of my array as it is now. Can anyone confirm
/ deny that a simultaneous reshape+grow is possible or not? Anyone
done one? :)

Thanks!

-- Kristleifur
-- Kristleifur
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Neil Brown]

On Mon, 21 Dec 2009 03:41:33 +0000
Kristleifur Daðason <kristleifur@gmail.com> wrote:

> Hi all,
> 
> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
> 
> I have three live drives and three spares added to the array. When I
> initialize the command, mdadm does the initial checks and aborts with
> a "cannot set device shape" without doing anything to the array.
> 
> Following are some md stats and growth command output:
> 
> ___
> 
> $ cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
>       2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
>       bitmap: 1/350 pages [4KB], 2048KB chunk
> 
> $ mdadm --detail --scan
> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
> UUID=da547022:042a6f68:d5fe251e:5e89f263
> 
> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
> --backup-file=/root/backup.md1_to_r6
> mdadm: metadata format 1.10 unknown, ignored.
> mdadm: metadata format 1.10 unknown, ignored.
> mdadm level of /dev/md_d1 changed to raid6
> mdadm: Need to backup 1024K of critical section..
> mdadm: Cannot set device shape for /dev/md_d1
> mdadm: aborting level change
> ___
> 
> 
> Three questions -
> 
> 1. What does the stuff about "metadata format 1.10 unknown" mean?
> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
> disrepancy between mdsat, --detail and --grow output.

The metadata format .. unknown means that your /etc/mdadm.conf contains
something like
       metadata=1.10

> 
> 2. Am I doing something wrong? :)

Not obviously.

> 
> 3. How can I get more info about what is causing the failure to
> initialize the growth?

Look in the kernel logs.  e.g. 
   dmesg | tail -20

immediately after the "mdadm --grow" attempt.

I just tried the same thing and it worked for me.

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Neil Brown]

On Mon, 21 Dec 2009 09:35:28 +0000
Robin Hill <robin@robinhill.me.uk> wrote:

> On Mon Dec 21, 2009 at 03:41:33AM +0000, Kristleifur Daðason wrote:
> 
> > Hi all,
> > 
> > I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
> > Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
> > patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
> > 
> > I have three live drives and three spares added to the array. When I
> > initialize the command, mdadm does the initial checks and aborts with
> > a "cannot set device shape" without doing anything to the array.
> > 
> > $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
> > --backup-file=/root/backup.md1_to_r6
> > mdadm: metadata format 1.10 unknown, ignored.
> > mdadm: metadata format 1.10 unknown, ignored.
> > mdadm level of /dev/md_d1 changed to raid6
> > mdadm: Need to backup 1024K of critical section..
> > mdadm: Cannot set device shape for /dev/md_d1
> > mdadm: aborting level change
> > 
> I don't think you can grow and reshape simultaneously.  I think you'll
> need to do this as two operations - either grow to a 5 disk RAID5 then
> reshape to a 6 disk RAID6, or reshape to a 4 disk RAID6 then grow.
>

Why do you think that?  If there is something in the documentation that
suggests that, please point it out and I'll change it.

You should be able to change level, layout, number of devices, chunksize
all at the same time.

NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Kristleifur Daðason]

On Mon, Dec 21, 2009 at 10:57 PM, Neil Brown <neilb@suse.de> wrote:
> On Mon, 21 Dec 2009 03:41:33 +0000
> Kristleifur Daðason <kristleifur@gmail.com> wrote:
>
>> Hi all,
>>
>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
>>
>> I have three live drives and three spares added to the array. When I
>> initialize the command, mdadm does the initial checks and aborts with
>> a "cannot set device shape" without doing anything to the array.
>>
>> Following are some md stats and growth command output:
>>
>> ___
>>
>> $ cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
>>       2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
>>       bitmap: 1/350 pages [4KB], 2048KB chunk
>>
>> $ mdadm --detail --scan
>> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
>> UUID=da547022:042a6f68:d5fe251e:5e89f263
>>
>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
>> --backup-file=/root/backup.md1_to_r6
>> mdadm: metadata format 1.10 unknown, ignored.
>> mdadm: metadata format 1.10 unknown, ignored.
>> mdadm level of /dev/md_d1 changed to raid6
>> mdadm: Need to backup 1024K of critical section..
>> mdadm: Cannot set device shape for /dev/md_d1
>> mdadm: aborting level change
>> ___
>>
>>
>> Three questions -
>>
>> 1. What does the stuff about "metadata format 1.10 unknown" mean?
>> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
>> disrepancy between mdsat, --detail and --grow output.
>
> The metadata format .. unknown means that your /etc/mdadm.conf contains
> something like
>       metadata=1.10
>
>>
>> 2. Am I doing something wrong? :)
>
> Not obviously.
>
>>
>> 3. How can I get more info about what is causing the failure to
>> initialize the growth?
>
> Look in the kernel logs.  e.g.
>   dmesg | tail -20
>
> immediately after the "mdadm --grow" attempt.
>
> I just tried the same thing and it worked for me.
>
> NeilBrown
>

Thank you very much for the reply. You were right, mdadm.conf indeed
contained metadata=1.10. I fixed it, updated the initramfs and
rebooted.

---

mdadm --detail --scan now gives:

 sudo mdadm --detail --scan
ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
UUID=da547022:042a6f68:d5fe251e:5e89f263

---
 I tried the grow command again, and it aborts again. Could it be that
the device sizes are wrong? I thought I meticulously created exactly
identical partitions on each of the drives. The command output is:

 sudo mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
--backup-file=/root/backup.md1_to_r6
mdadm level of /dev/md_d1 changed to raid6
mdadm: Need to backup 1024K of critical section..
mdadm: Cannot set device shape for /dev/md_d1
mdadm: aborting level change

---

dmesg says:

[   96.482937] raid5: device sdl1 operational as raid disk 2
[   96.482940] raid5: device sdf1 operational as raid disk 1
[   96.482942] raid5: device sde1 operational as raid disk 0
[   96.483299] raid5: allocated 4282kB for md_d1
[   96.511577] 2: w=1 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0
[   96.511581] 1: w=2 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0
[   96.511583] 0: w=3 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0
[   96.511585] raid5: raid level 6 set md_d1 active with 3 out of 4
devices, algorithm 18
[   96.511588] RAID5 conf printout:
[   96.511589]  --- rd:4 wd:3
[   96.511591]  disk 0, o:1, dev:sde1
[   96.511593]  disk 1, o:1, dev:sdf1
[   96.511595]  disk 2, o:1, dev:sdl1
[   96.671315] raid5: device sdl1 operational as raid disk 2
[   96.671318] raid5: device sdf1 operational as raid disk 1
[   96.671320] raid5: device sde1 operational as raid disk 0
[   96.671642] raid5: allocated 3230kB for md_d1
[   96.720331] 2: w=1 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[   96.720334] 1: w=2 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[   96.720336] 0: w=3 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[   96.720338] raid5: raid level 5 set md_d1 active with 3 out of 3
devices, algorithm 2
[   96.720340] RAID5 conf printout:
[   96.720341]  --- rd:3 wd:3
[   96.720343]  disk 0, o:1, dev:sde1
[   96.720345]  disk 1, o:1, dev:sdf1
[   96.720346]  disk 2, o:1, dev:sdl1
[  100.202834] raid5: device sdl1 operational as raid disk 2
[  100.202837] raid5: device sdf1 operational as raid disk 1
[  100.202839] raid5: device sde1 operational as raid disk 0
[  100.203194] raid5: allocated 4282kB for md_d1
[  100.241576] 2: w=1 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0
[  100.241579] 1: w=2 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0
[  100.241582] 0: w=3 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0
[  100.241584] raid5: raid level 6 set md_d1 active with 3 out of 4
devices, algorithm 18
[  100.241586] RAID5 conf printout:
[  100.241588]  --- rd:4 wd:3
[  100.241590]  disk 0, o:1, dev:sde1
[  100.241592]  disk 1, o:1, dev:sdf1
[  100.241593]  disk 2, o:1, dev:sdl1
[  100.401030] raid5: device sdl1 operational as raid disk 2
[  100.401033] raid5: device sdf1 operational as raid disk 1
[  100.401035] raid5: device sde1 operational as raid disk 0
[  100.401348] raid5: allocated 3230kB for md_d1
[  100.460458] 2: w=1 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[  100.460461] 1: w=2 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[  100.460463] 0: w=3 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[  100.460466] raid5: raid level 5 set md_d1 active with 3 out of 3
devices, algorithm 2
[  100.460467] RAID5 conf printout:
[  100.460468]  --- rd:3 wd:3
[  100.460470]  disk 0, o:1, dev:sde1
[  100.460472]  disk 1, o:1, dev:sdf1
[  100.460474]  disk 2, o:1, dev:sdl1
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Kristleifur Daðason]

On Mon, Dec 21, 2009 at 11:18 PM, Kristleifur Daðason
<kristleifur@gmail.com> wrote:
> On Mon, Dec 21, 2009 at 10:57 PM, Neil Brown <neilb@suse.de> wrote:
>> On Mon, 21 Dec 2009 03:41:33 +0000
>> Kristleifur Daðason <kristleifur@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on
>>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size
>>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html
>>>
>>> I have three live drives and three spares added to the array. When I
>>> initialize the command, mdadm does the initial checks and aborts with
>>> a "cannot set device shape" without doing anything to the array.
>>>
>>> Following are some md stats and growth command output:
>>>
>>> ___
>>>
>>> $ cat /proc/mdstat
>>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>>> [raid4] [raid10]
>>> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3]
>>>       2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU]
>>>       bitmap: 1/350 pages [4KB], 2048KB chunk
>>>
>>> $ mdadm --detail --scan
>>> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
>>> UUID=da547022:042a6f68:d5fe251e:5e89f263
>>>
>>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
>>> --backup-file=/root/backup.md1_to_r6
>>> mdadm: metadata format 1.10 unknown, ignored.
>>> mdadm: metadata format 1.10 unknown, ignored.
>>> mdadm level of /dev/md_d1 changed to raid6
>>> mdadm: Need to backup 1024K of critical section..
>>> mdadm: Cannot set device shape for /dev/md_d1
>>> mdadm: aborting level change
>>> ___
>>>
>>>
>>> Three questions -
>>>
>>> 1. What does the stuff about "metadata format 1.10 unknown" mean?
>>> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
>>> disrepancy between mdsat, --detail and --grow output.
>>
>> The metadata format .. unknown means that your /etc/mdadm.conf contains
>> something like
>>       metadata=1.10
>>
>>>
>>> 2. Am I doing something wrong? :)
>>
>> Not obviously.
>>
>>>
>>> 3. How can I get more info about what is causing the failure to
>>> initialize the growth?
>>
>> Look in the kernel logs.  e.g.
>>   dmesg | tail -20
>>
>> immediately after the "mdadm --grow" attempt.
>>
>> I just tried the same thing and it worked for me.
>>
>> NeilBrown
>>
>
> Thank you very much for the reply. You were right, mdadm.conf indeed
> contained metadata=1.10. I fixed it, updated the initramfs and
> rebooted.
>
> ---
>
> mdadm --detail --scan now gives:
>
>  sudo mdadm --detail --scan
> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1
> UUID=da547022:042a6f68:d5fe251e:5e89f263
>
> ---
>  I tried the grow command again, and it aborts again. Could it be that
> the device sizes are wrong? I thought I meticulously created exactly
> identical partitions on each of the drives. The command output is:
>
>  sudo mdadm --grow /dev/md_d1 --level=6 --raid-devices=6
> --backup-file=/root/backup.md1_to_r6
> mdadm level of /dev/md_d1 changed to raid6
> mdadm: Need to backup 1024K of critical section..
> mdadm: Cannot set device shape for /dev/md_d1
> mdadm: aborting level change
>
> ---

I figured it out - I just needed to disable the write-intent bitmap by
doing "mdadm --grow --bitmap=none /dev/md_d1". Then the reshape was
startable and is going quite well.

I'm seeing resync speeds of around 30-40 MB/second during the
reshape/relevel. I'm running on 1.5TB SATA drives. Is this speed OK?

Thanks everybody for the help and suggestions!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: mdadm 3.1.1: level change won't start

[Leslie Rhorer]

> >> Three questions -
> >>
> >> 1. What does the stuff about "metadata format 1.10 unknown" mean?
> >> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
> >> disrepancy between mdsat, --detail and --grow output.
> >>
> >> 2. Am I doing something wrong? :)
> >>
> >> 3. How can I get more info about what is causing the failure to
> >> initialize the growth?
> >>
> >> Thanks!
> >>
> >> -- Kristleifur
> >>
> >>
> >> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of
> >> mdadm. I hope that isn't rude of me.)
> >> --
> >
> >
> > I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2
> >  Maybe the extra 1 or 0 at the end refers to a reshape in progress?
> >
> 
> Thanks for the reply. FWIW, I intended the array to have metadata
> format 1.1 when I created it. I've noticed these "1.01" messages
> before, so I believe it's not due to the reshape. However, I'm
> thinking the funny metadata might be interfering with the reshape
> somehow.

	I'm getting the same thing on one of my systems and not on the
other:

RAID-Server:/etc/rcS.d# mdadm -V
mdadm - v2.6.7.2 - 14th November 2008
RAID-Server:/etc/rcS.d# mdadm --detail --scan
ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1
name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca

Backup:/etc/initramfs-tools# mdadm -V
mdadm - v2.6.7.2 - 14th November 2008
Backup:/etc/initramfs-tools# mdadm --detail --scan
mdadm: metadata format 01.02 unknown, ignored.
mdadm: metadata format 01.02 unknown, ignored.
mdadm: metadata format 01.02 unknown, ignored.
mdadm: metadata format 01.00 unknown, ignored.
ARRAY /dev/md0 level=raid5 num-devices=8 metadata=01.02 name='Backup':0
UUID=940ae4e4:04057ffc:5e92d2fb:63e3efb7
ARRAY /dev/md3 level=raid1 num-devices=2 metadata=01.02 spares=1
name='Backup':3 UUID=3615c4a2:33786b6d:b13863d9:458cd054
ARRAY /dev/md2 level=raid1 num-devices=2 metadata=01.02 spares=1
name='Backup':2 UUID=d45ff663:9e53774c:6fcf9968:21692025
ARRAY /dev/md1 level=raid1 num-devices=2 metadata=01.00 name='Backup':1
UUID=d6a2c60b:7345e957:05aefe0b:f8d1527f

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: mdadm 3.1.1: level change won't start

[Leslie Rhorer]

	I never received a response on this one either.  Can anyone guess
why one of my systems would suddenly start claiming it doesn't recognize the
superblock versions?  The two systems are highly similar.

> > >> Three questions -
> > >>
> > >> 1. What does the stuff about "metadata format 1.10 unknown" mean?
> > >> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10"
> > >> disrepancy between mdsat, --detail and --grow output.
> > >>
> > >> 2. Am I doing something wrong? :)
> > >>
> > >> 3. How can I get more info about what is causing the failure to
> > >> initialize the growth?
> > >>
> > >> Thanks!
> > >>
> > >> -- Kristleifur
> > >>
> > >>
> > >> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of
> > >> mdadm. I hope that isn't rude of me.)
> > >> --
> > >
> > >
> > > I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2
> > >  Maybe the extra 1 or 0 at the end refers to a reshape in progress?
> > >
> >
> > Thanks for the reply. FWIW, I intended the array to have metadata
> > format 1.1 when I created it. I've noticed these "1.01" messages
> > before, so I believe it's not due to the reshape. However, I'm
> > thinking the funny metadata might be interfering with the reshape
> > somehow.
> 
> 	I'm getting the same thing on one of my systems and not on the
> other:
> 
> RAID-Server:/etc/rcS.d# mdadm -V
> mdadm - v2.6.7.2 - 14th November 2008
> RAID-Server:/etc/rcS.d# mdadm --detail --scan
> ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1
> name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca
> 
> Backup:/etc/initramfs-tools# mdadm -V
> mdadm - v2.6.7.2 - 14th November 2008
> Backup:/etc/initramfs-tools# mdadm --detail --scan
> mdadm: metadata format 01.02 unknown, ignored.
> mdadm: metadata format 01.02 unknown, ignored.
> mdadm: metadata format 01.02 unknown, ignored.
> mdadm: metadata format 01.00 unknown, ignored.
> ARRAY /dev/md0 level=raid5 num-devices=8 metadata=01.02 name='Backup':0
> UUID=940ae4e4:04057ffc:5e92d2fb:63e3efb7
> ARRAY /dev/md3 level=raid1 num-devices=2 metadata=01.02 spares=1
> name='Backup':3 UUID=3615c4a2:33786b6d:b13863d9:458cd054
> ARRAY /dev/md2 level=raid1 num-devices=2 metadata=01.02 spares=1
> name='Backup':2 UUID=d45ff663:9e53774c:6fcf9968:21692025
> ARRAY /dev/md1 level=raid1 num-devices=2 metadata=01.00 name='Backup':1
> UUID=d6a2c60b:7345e957:05aefe0b:f8d1527f
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm 3.1.1: level change won't start

[Neil Brown]

On Tue, 29 Dec 2009 23:25:02 -0600
"Leslie Rhorer" <lrhorer@satx.rr.com> wrote:

> 
> 	I never received a response on this one either.  Can anyone guess
> why one of my systems would suddenly start claiming it doesn't recognize the
> superblock versions?  The two systems are highly similar.
> 
...
> > RAID-Server:/etc/rcS.d# mdadm -V
> > mdadm - v2.6.7.2 - 14th November 2008
> > RAID-Server:/etc/rcS.d# mdadm --detail --scan
> > ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1
> > name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca
> > 
> > Backup:/etc/initramfs-tools# mdadm -V
> > mdadm - v2.6.7.2 - 14th November 2008
> > Backup:/etc/initramfs-tools# mdadm --detail --scan
> > mdadm: metadata format 01.02 unknown, ignored.
>
Fixed in 2.6.8

NeilBrown

RE: mdadm 3.1.1: level change won't start

[Leslie Rhorer]

> > 	I never received a response on this one either.  Can anyone guess
> > why one of my systems would suddenly start claiming it doesn't recognize
> the
> > superblock versions?  The two systems are highly similar.
> >
> ...
> > > RAID-Server:/etc/rcS.d# mdadm -V
> > > mdadm - v2.6.7.2 - 14th November 2008
> > > RAID-Server:/etc/rcS.d# mdadm --detail --scan
> > > ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1
> > > name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca
> > >
> > > Backup:/etc/initramfs-tools# mdadm -V
> > > mdadm - v2.6.7.2 - 14th November 2008
> > > Backup:/etc/initramfs-tools# mdadm --detail --scan
> > > mdadm: metadata format 01.02 unknown, ignored.
> >
> Fixed in 2.6.8
> 
> NeilBrown

	Oh, OK.  Thanks Neil.  When 2.6.8 is moved into a stable version of
Debian, it will get picked up automatically.  Since it doesn't seem to
impact reliability, I can wait.