* mdadm 3.1.1: level change won't start @ 2009-12-21 3:41 Kristleifur Daðason 2009-12-21 4:10 ` Michael Evans ` (2 more replies) 0 siblings, 3 replies; 14+ messages in thread From: Kristleifur Daðason @ 2009-12-21 3:41 UTC (permalink / raw) To: linux-raid; +Cc: Neil Brown Hi all, I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html I have three live drives and three spares added to the array. When I initialize the command, mdadm does the initial checks and aborts with a "cannot set device shape" without doing anything to the array. Following are some md stats and growth command output: ___ $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] bitmap: 1/350 pages [4KB], 2048KB chunk $ mdadm --detail --scan ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 UUID=da547022:042a6f68:d5fe251e:5e89f263 $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 --backup-file=/root/backup.md1_to_r6 mdadm: metadata format 1.10 unknown, ignored. mdadm: metadata format 1.10 unknown, ignored. mdadm level of /dev/md_d1 changed to raid6 mdadm: Need to backup 1024K of critical section.. mdadm: Cannot set device shape for /dev/md_d1 mdadm: aborting level change ___ Three questions - 1. What does the stuff about "metadata format 1.10 unknown" mean? Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" disrepancy between mdsat, --detail and --grow output. 2. Am I doing something wrong? :) 3. How can I get more info about what is causing the failure to initialize the growth? Thanks! -- Kristleifur (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of mdadm. I hope that isn't rude of me.) -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 3:41 mdadm 3.1.1: level change won't start Kristleifur Daðason @ 2009-12-21 4:10 ` Michael Evans 2009-12-21 4:39 ` Kristleifur Daðason 2009-12-21 9:35 ` Robin Hill 2009-12-21 22:57 ` Neil Brown 2 siblings, 1 reply; 14+ messages in thread From: Michael Evans @ 2009-12-21 4:10 UTC (permalink / raw) To: Kristleifur Daðason; +Cc: linux-raid, Neil Brown On Sun, Dec 20, 2009 at 7:41 PM, Kristleifur Daðason <kristleifur@gmail.com> wrote: > Hi all, > > I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on > Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size > patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html > > I have three live drives and three spares added to the array. When I > initialize the command, mdadm does the initial checks and aborts with > a "cannot set device shape" without doing anything to the array. > > Following are some md stats and growth command output: > > ___ > > $ cat /proc/mdstat > Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] > [raid4] [raid10] > md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] > 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] > bitmap: 1/350 pages [4KB], 2048KB chunk > > $ mdadm --detail --scan > ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 > UUID=da547022:042a6f68:d5fe251e:5e89f263 > > $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 > --backup-file=/root/backup.md1_to_r6 > mdadm: metadata format 1.10 unknown, ignored. > mdadm: metadata format 1.10 unknown, ignored. > mdadm level of /dev/md_d1 changed to raid6 > mdadm: Need to backup 1024K of critical section.. > mdadm: Cannot set device shape for /dev/md_d1 > mdadm: aborting level change > ___ > > > Three questions - > > 1. What does the stuff about "metadata format 1.10 unknown" mean? > Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" > disrepancy between mdsat, --detail and --grow output. > > 2. Am I doing something wrong? :) > > 3. How can I get more info about what is causing the failure to > initialize the growth? > > Thanks! > > -- Kristleifur > > > (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of > mdadm. I hope that isn't rude of me.) > -- > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2 Maybe the extra 1 or 0 at the end refers to a reshape in progress? -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 4:10 ` Michael Evans @ 2009-12-21 4:39 ` Kristleifur Daðason 2009-12-21 7:25 ` Michael Evans 2009-12-23 23:47 ` Leslie Rhorer 0 siblings, 2 replies; 14+ messages in thread From: Kristleifur Daðason @ 2009-12-21 4:39 UTC (permalink / raw) To: Michael Evans; +Cc: linux-raid On Mon, Dec 21, 2009 at 4:10 AM, Michael Evans <mjevans1983@gmail.com> wrote: > On Sun, Dec 20, 2009 at 7:41 PM, Kristleifur Daðason > <kristleifur@gmail.com> wrote: >> Hi all, >> >> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on >> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size >> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html >> >> I have three live drives and three spares added to the array. When I >> initialize the command, mdadm does the initial checks and aborts with >> a "cannot set device shape" without doing anything to the array. >> >> Following are some md stats and growth command output: >> >> ___ >> >> $ cat /proc/mdstat >> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] >> [raid4] [raid10] >> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] >> 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] >> bitmap: 1/350 pages [4KB], 2048KB chunk >> >> $ mdadm --detail --scan >> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 >> UUID=da547022:042a6f68:d5fe251e:5e89f263 >> >> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 >> --backup-file=/root/backup.md1_to_r6 >> mdadm: metadata format 1.10 unknown, ignored. >> mdadm: metadata format 1.10 unknown, ignored. >> mdadm level of /dev/md_d1 changed to raid6 >> mdadm: Need to backup 1024K of critical section.. >> mdadm: Cannot set device shape for /dev/md_d1 >> mdadm: aborting level change >> ___ >> >> >> Three questions - >> >> 1. What does the stuff about "metadata format 1.10 unknown" mean? >> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" >> disrepancy between mdsat, --detail and --grow output. >> >> 2. Am I doing something wrong? :) >> >> 3. How can I get more info about what is causing the failure to >> initialize the growth? >> >> Thanks! >> >> -- Kristleifur >> >> >> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of >> mdadm. I hope that isn't rude of me.) >> -- > > > I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2 > Maybe the extra 1 or 0 at the end refers to a reshape in progress? > Thanks for the reply. FWIW, I intended the array to have metadata format 1.1 when I created it. I've noticed these "1.01" messages before, so I believe it's not due to the reshape. However, I'm thinking the funny metadata might be interfering with the reshape somehow. Hmm ... -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 4:39 ` Kristleifur Daðason @ 2009-12-21 7:25 ` Michael Evans 2009-12-23 23:47 ` Leslie Rhorer 1 sibling, 0 replies; 14+ messages in thread From: Michael Evans @ 2009-12-21 7:25 UTC (permalink / raw) To: Kristleifur Daðason; +Cc: linux-raid On Sun, Dec 20, 2009 at 8:39 PM, Kristleifur Daðason <kristleifur@gmail.com> wrote: > On Mon, Dec 21, 2009 at 4:10 AM, Michael Evans <mjevans1983@gmail.com> wrote: >> On Sun, Dec 20, 2009 at 7:41 PM, Kristleifur Daðason >> <kristleifur@gmail.com> wrote: >>> Hi all, >>> >>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on >>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size >>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html >>> >>> I have three live drives and three spares added to the array. When I >>> initialize the command, mdadm does the initial checks and aborts with >>> a "cannot set device shape" without doing anything to the array. >>> >>> Following are some md stats and growth command output: >>> >>> ___ >>> >>> $ cat /proc/mdstat >>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] >>> [raid4] [raid10] >>> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] >>> 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] >>> bitmap: 1/350 pages [4KB], 2048KB chunk >>> >>> $ mdadm --detail --scan >>> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 >>> UUID=da547022:042a6f68:d5fe251e:5e89f263 >>> >>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 >>> --backup-file=/root/backup.md1_to_r6 >>> mdadm: metadata format 1.10 unknown, ignored. >>> mdadm: metadata format 1.10 unknown, ignored. >>> mdadm level of /dev/md_d1 changed to raid6 >>> mdadm: Need to backup 1024K of critical section.. >>> mdadm: Cannot set device shape for /dev/md_d1 >>> mdadm: aborting level change >>> ___ >>> >>> >>> Three questions - >>> >>> 1. What does the stuff about "metadata format 1.10 unknown" mean? >>> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" >>> disrepancy between mdsat, --detail and --grow output. >>> >>> 2. Am I doing something wrong? :) >>> >>> 3. How can I get more info about what is causing the failure to >>> initialize the growth? >>> >>> Thanks! >>> >>> -- Kristleifur >>> >>> >>> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of >>> mdadm. I hope that isn't rude of me.) >>> -- >> >> >> I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2 >> Maybe the extra 1 or 0 at the end refers to a reshape in progress? >> > > Thanks for the reply. FWIW, I intended the array to have metadata > format 1.1 when I created it. I've noticed these "1.01" messages > before, so I believe it's not due to the reshape. However, I'm > thinking the funny metadata might be interfering with the reshape > somehow. > > Hmm ... > This is odd; cat /proc/mdstat shows 1.0 and 1.1 for my metadata formats; however using --detail --scan I _DO_ see 1.00 and 1.01 on my system. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: mdadm 3.1.1: level change won't start 2009-12-21 4:39 ` Kristleifur Daðason 2009-12-21 7:25 ` Michael Evans @ 2009-12-23 23:47 ` Leslie Rhorer 2009-12-30 5:25 ` Leslie Rhorer 1 sibling, 1 reply; 14+ messages in thread From: Leslie Rhorer @ 2009-12-23 23:47 UTC (permalink / raw) To: 'Michael Evans'; +Cc: 'linux-raid' > >> Three questions - > >> > >> 1. What does the stuff about "metadata format 1.10 unknown" mean? > >> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" > >> disrepancy between mdsat, --detail and --grow output. > >> > >> 2. Am I doing something wrong? :) > >> > >> 3. How can I get more info about what is causing the failure to > >> initialize the growth? > >> > >> Thanks! > >> > >> -- Kristleifur > >> > >> > >> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of > >> mdadm. I hope that isn't rude of me.) > >> -- > > > > > > I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2 > > Maybe the extra 1 or 0 at the end refers to a reshape in progress? > > > > Thanks for the reply. FWIW, I intended the array to have metadata > format 1.1 when I created it. I've noticed these "1.01" messages > before, so I believe it's not due to the reshape. However, I'm > thinking the funny metadata might be interfering with the reshape > somehow. I'm getting the same thing on one of my systems and not on the other: RAID-Server:/etc/rcS.d# mdadm -V mdadm - v2.6.7.2 - 14th November 2008 RAID-Server:/etc/rcS.d# mdadm --detail --scan ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1 name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca Backup:/etc/initramfs-tools# mdadm -V mdadm - v2.6.7.2 - 14th November 2008 Backup:/etc/initramfs-tools# mdadm --detail --scan mdadm: metadata format 01.02 unknown, ignored. mdadm: metadata format 01.02 unknown, ignored. mdadm: metadata format 01.02 unknown, ignored. mdadm: metadata format 01.00 unknown, ignored. ARRAY /dev/md0 level=raid5 num-devices=8 metadata=01.02 name='Backup':0 UUID=940ae4e4:04057ffc:5e92d2fb:63e3efb7 ARRAY /dev/md3 level=raid1 num-devices=2 metadata=01.02 spares=1 name='Backup':3 UUID=3615c4a2:33786b6d:b13863d9:458cd054 ARRAY /dev/md2 level=raid1 num-devices=2 metadata=01.02 spares=1 name='Backup':2 UUID=d45ff663:9e53774c:6fcf9968:21692025 ARRAY /dev/md1 level=raid1 num-devices=2 metadata=01.00 name='Backup':1 UUID=d6a2c60b:7345e957:05aefe0b:f8d1527f -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: mdadm 3.1.1: level change won't start 2009-12-23 23:47 ` Leslie Rhorer @ 2009-12-30 5:25 ` Leslie Rhorer 2009-12-30 5:37 ` Neil Brown 0 siblings, 1 reply; 14+ messages in thread From: Leslie Rhorer @ 2009-12-30 5:25 UTC (permalink / raw) To: 'linux-raid' I never received a response on this one either. Can anyone guess why one of my systems would suddenly start claiming it doesn't recognize the superblock versions? The two systems are highly similar. > > >> Three questions - > > >> > > >> 1. What does the stuff about "metadata format 1.10 unknown" mean? > > >> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" > > >> disrepancy between mdsat, --detail and --grow output. > > >> > > >> 2. Am I doing something wrong? :) > > >> > > >> 3. How can I get more info about what is causing the failure to > > >> initialize the growth? > > >> > > >> Thanks! > > >> > > >> -- Kristleifur > > >> > > >> > > >> (P.s., I'm cc-ing this directly to Neil Brown as the maintainer of > > >> mdadm. I hope that isn't rude of me.) > > >> -- > > > > > > > > > I thought there were three metadata variants for 1.x: 1.0 1.1 and 1.2 > > > Maybe the extra 1 or 0 at the end refers to a reshape in progress? > > > > > > > Thanks for the reply. FWIW, I intended the array to have metadata > > format 1.1 when I created it. I've noticed these "1.01" messages > > before, so I believe it's not due to the reshape. However, I'm > > thinking the funny metadata might be interfering with the reshape > > somehow. > > I'm getting the same thing on one of my systems and not on the > other: > > RAID-Server:/etc/rcS.d# mdadm -V > mdadm - v2.6.7.2 - 14th November 2008 > RAID-Server:/etc/rcS.d# mdadm --detail --scan > ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1 > name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca > > Backup:/etc/initramfs-tools# mdadm -V > mdadm - v2.6.7.2 - 14th November 2008 > Backup:/etc/initramfs-tools# mdadm --detail --scan > mdadm: metadata format 01.02 unknown, ignored. > mdadm: metadata format 01.02 unknown, ignored. > mdadm: metadata format 01.02 unknown, ignored. > mdadm: metadata format 01.00 unknown, ignored. > ARRAY /dev/md0 level=raid5 num-devices=8 metadata=01.02 name='Backup':0 > UUID=940ae4e4:04057ffc:5e92d2fb:63e3efb7 > ARRAY /dev/md3 level=raid1 num-devices=2 metadata=01.02 spares=1 > name='Backup':3 UUID=3615c4a2:33786b6d:b13863d9:458cd054 > ARRAY /dev/md2 level=raid1 num-devices=2 metadata=01.02 spares=1 > name='Backup':2 UUID=d45ff663:9e53774c:6fcf9968:21692025 > ARRAY /dev/md1 level=raid1 num-devices=2 metadata=01.00 name='Backup':1 > UUID=d6a2c60b:7345e957:05aefe0b:f8d1527f > > -- > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-30 5:25 ` Leslie Rhorer @ 2009-12-30 5:37 ` Neil Brown 2009-12-30 5:43 ` Leslie Rhorer 0 siblings, 1 reply; 14+ messages in thread From: Neil Brown @ 2009-12-30 5:37 UTC (permalink / raw) To: Leslie Rhorer; +Cc: 'linux-raid' On Tue, 29 Dec 2009 23:25:02 -0600 "Leslie Rhorer" <lrhorer@satx.rr.com> wrote: > > I never received a response on this one either. Can anyone guess > why one of my systems would suddenly start claiming it doesn't recognize the > superblock versions? The two systems are highly similar. > ... > > RAID-Server:/etc/rcS.d# mdadm -V > > mdadm - v2.6.7.2 - 14th November 2008 > > RAID-Server:/etc/rcS.d# mdadm --detail --scan > > ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1 > > name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca > > > > Backup:/etc/initramfs-tools# mdadm -V > > mdadm - v2.6.7.2 - 14th November 2008 > > Backup:/etc/initramfs-tools# mdadm --detail --scan > > mdadm: metadata format 01.02 unknown, ignored. > Fixed in 2.6.8 NeilBrown ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: mdadm 3.1.1: level change won't start 2009-12-30 5:37 ` Neil Brown @ 2009-12-30 5:43 ` Leslie Rhorer 0 siblings, 0 replies; 14+ messages in thread From: Leslie Rhorer @ 2009-12-30 5:43 UTC (permalink / raw) To: 'Neil Brown'; +Cc: 'linux-raid' > > I never received a response on this one either. Can anyone guess > > why one of my systems would suddenly start claiming it doesn't recognize > the > > superblock versions? The two systems are highly similar. > > > ... > > > RAID-Server:/etc/rcS.d# mdadm -V > > > mdadm - v2.6.7.2 - 14th November 2008 > > > RAID-Server:/etc/rcS.d# mdadm --detail --scan > > > ARRAY /dev/md0 level=raid6 num-devices=10 metadata=01.02 spares=1 > > > name=RAID-Server:0 UUID=5ff10d73:a096195f:7a646bba:a68986ca > > > > > > Backup:/etc/initramfs-tools# mdadm -V > > > mdadm - v2.6.7.2 - 14th November 2008 > > > Backup:/etc/initramfs-tools# mdadm --detail --scan > > > mdadm: metadata format 01.02 unknown, ignored. > > > Fixed in 2.6.8 > > NeilBrown Oh, OK. Thanks Neil. When 2.6.8 is moved into a stable version of Debian, it will get picked up automatically. Since it doesn't seem to impact reliability, I can wait. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 3:41 mdadm 3.1.1: level change won't start Kristleifur Daðason 2009-12-21 4:10 ` Michael Evans @ 2009-12-21 9:35 ` Robin Hill 2009-12-21 13:34 ` Kristleifur Daðason 2009-12-21 23:00 ` Neil Brown 2009-12-21 22:57 ` Neil Brown 2 siblings, 2 replies; 14+ messages in thread From: Robin Hill @ 2009-12-21 9:35 UTC (permalink / raw) To: linux-raid [-- Attachment #1: Type: text/plain, Size: 1363 bytes --] On Mon Dec 21, 2009 at 03:41:33AM +0000, Kristleifur Daðason wrote: > Hi all, > > I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on > Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size > patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html > > I have three live drives and three spares added to the array. When I > initialize the command, mdadm does the initial checks and aborts with > a "cannot set device shape" without doing anything to the array. > > $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 > --backup-file=/root/backup.md1_to_r6 > mdadm: metadata format 1.10 unknown, ignored. > mdadm: metadata format 1.10 unknown, ignored. > mdadm level of /dev/md_d1 changed to raid6 > mdadm: Need to backup 1024K of critical section.. > mdadm: Cannot set device shape for /dev/md_d1 > mdadm: aborting level change > I don't think you can grow and reshape simultaneously. I think you'll need to do this as two operations - either grow to a 5 disk RAID5 then reshape to a 6 disk RAID6, or reshape to a 4 disk RAID6 then grow. Cheers, Robin -- ___ ( ' } | Robin Hill <robin@robinhill.me.uk> | / / ) | Little Jim says .... | // !! | "He fallen in de water !!" | [-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 9:35 ` Robin Hill @ 2009-12-21 13:34 ` Kristleifur Daðason 2009-12-21 23:00 ` Neil Brown 1 sibling, 0 replies; 14+ messages in thread From: Kristleifur Daðason @ 2009-12-21 13:34 UTC (permalink / raw) To: linux-raid > On Mon Dec 21, 2009 at 03:41:33AM +0000, Kristleifur Daðason wrote: > >> Hi all, >> >> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on >> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size >> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html >> >> I have three live drives and three spares added to the array. When I >> initialize the command, mdadm does the initial checks and aborts with >> a "cannot set device shape" without doing anything to the array. >> >> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 >> --backup-file=/root/backup.md1_to_r6 >> mdadm: metadata format 1.10 unknown, ignored. >> mdadm: metadata format 1.10 unknown, ignored. >> mdadm level of /dev/md_d1 changed to raid6 >> mdadm: Need to backup 1024K of critical section.. >> mdadm: Cannot set device shape for /dev/md_d1 >> mdadm: aborting level change >> On Mon, Dec 21, 2009 at 7:25 AM, Michael Evans <mjevans1983@gmail.com> wrote: > This is odd; cat /proc/mdstat shows 1.0 and 1.1 for my metadata > formats; however using --detail --scan I _DO_ see 1.00 and 1.01 on my > system. > On Mon, Dec 21, 2009 at 9:35 AM, Robin Hill <robin@robinhill.me.uk> wrote: > I don't think you can grow and reshape simultaneously. I think you'll > need to do this as two operations - either grow to a 5 disk RAID5 then > reshape to a 6 disk RAID6, or reshape to a 4 disk RAID6 then grow. > Thanks for the replies, Michael and Robin. Re. metadata version number discrepancy, I'm going to assume for now that it doesn't affect my reshape, and take it up later in its own list thread. Robin, it makes sense that I can't actually grow and reshape simultaneously - it fits to the mdadm halt I'm seeing - but I did note Neil Brown saying here >> http://neil.brown.name/blog/20090817000931 << that --- "One way to avoid the extra cost of doing the backup [reshape] is to increase the number of devices at the same time. e.g. if you had a 4-drive RAID5 you could convert it to a 6-drive RAID6 with a command like mdadm --grow /dev/md0 --level=6 --raid-disk=6 then it will combine the increase in the number of devices with the change to the layout and will copy every block only once. " --- I'm OK with doing the grow and reshape in two separate steps, but the indication that it should be possible in one step makes me a bit anxious about the health of my array as it is now. Can anyone confirm / deny that a simultaneous reshape+grow is possible or not? Anyone done one? :) Thanks! -- Kristleifur -- Kristleifur -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 9:35 ` Robin Hill 2009-12-21 13:34 ` Kristleifur Daðason @ 2009-12-21 23:00 ` Neil Brown 1 sibling, 0 replies; 14+ messages in thread From: Neil Brown @ 2009-12-21 23:00 UTC (permalink / raw) To: Robin Hill; +Cc: linux-raid On Mon, 21 Dec 2009 09:35:28 +0000 Robin Hill <robin@robinhill.me.uk> wrote: > On Mon Dec 21, 2009 at 03:41:33AM +0000, Kristleifur Daðason wrote: > > > Hi all, > > > > I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on > > Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size > > patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html > > > > I have three live drives and three spares added to the array. When I > > initialize the command, mdadm does the initial checks and aborts with > > a "cannot set device shape" without doing anything to the array. > > > > $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 > > --backup-file=/root/backup.md1_to_r6 > > mdadm: metadata format 1.10 unknown, ignored. > > mdadm: metadata format 1.10 unknown, ignored. > > mdadm level of /dev/md_d1 changed to raid6 > > mdadm: Need to backup 1024K of critical section.. > > mdadm: Cannot set device shape for /dev/md_d1 > > mdadm: aborting level change > > > I don't think you can grow and reshape simultaneously. I think you'll > need to do this as two operations - either grow to a 5 disk RAID5 then > reshape to a 6 disk RAID6, or reshape to a 4 disk RAID6 then grow. > Why do you think that? If there is something in the documentation that suggests that, please point it out and I'll change it. You should be able to change level, layout, number of devices, chunksize all at the same time. NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 3:41 mdadm 3.1.1: level change won't start Kristleifur Daðason 2009-12-21 4:10 ` Michael Evans 2009-12-21 9:35 ` Robin Hill @ 2009-12-21 22:57 ` Neil Brown 2009-12-21 23:18 ` Kristleifur Daðason 2 siblings, 1 reply; 14+ messages in thread From: Neil Brown @ 2009-12-21 22:57 UTC (permalink / raw) To: Kristleifur Daðason; +Cc: linux-raid, Neil Brown On Mon, 21 Dec 2009 03:41:33 +0000 Kristleifur Daðason <kristleifur@gmail.com> wrote: > Hi all, > > I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on > Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size > patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html > > I have three live drives and three spares added to the array. When I > initialize the command, mdadm does the initial checks and aborts with > a "cannot set device shape" without doing anything to the array. > > Following are some md stats and growth command output: > > ___ > > $ cat /proc/mdstat > Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] > [raid4] [raid10] > md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] > 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] > bitmap: 1/350 pages [4KB], 2048KB chunk > > $ mdadm --detail --scan > ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 > UUID=da547022:042a6f68:d5fe251e:5e89f263 > > $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 > --backup-file=/root/backup.md1_to_r6 > mdadm: metadata format 1.10 unknown, ignored. > mdadm: metadata format 1.10 unknown, ignored. > mdadm level of /dev/md_d1 changed to raid6 > mdadm: Need to backup 1024K of critical section.. > mdadm: Cannot set device shape for /dev/md_d1 > mdadm: aborting level change > ___ > > > Three questions - > > 1. What does the stuff about "metadata format 1.10 unknown" mean? > Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" > disrepancy between mdsat, --detail and --grow output. The metadata format .. unknown means that your /etc/mdadm.conf contains something like metadata=1.10 > > 2. Am I doing something wrong? :) Not obviously. > > 3. How can I get more info about what is causing the failure to > initialize the growth? Look in the kernel logs. e.g. dmesg | tail -20 immediately after the "mdadm --grow" attempt. I just tried the same thing and it worked for me. NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 22:57 ` Neil Brown @ 2009-12-21 23:18 ` Kristleifur Daðason 2009-12-22 18:35 ` Kristleifur Daðason 0 siblings, 1 reply; 14+ messages in thread From: Kristleifur Daðason @ 2009-12-21 23:18 UTC (permalink / raw) To: Neil Brown; +Cc: linux-raid, Neil Brown On Mon, Dec 21, 2009 at 10:57 PM, Neil Brown <neilb@suse.de> wrote: > On Mon, 21 Dec 2009 03:41:33 +0000 > Kristleifur Daðason <kristleifur@gmail.com> wrote: > >> Hi all, >> >> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on >> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size >> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html >> >> I have three live drives and three spares added to the array. When I >> initialize the command, mdadm does the initial checks and aborts with >> a "cannot set device shape" without doing anything to the array. >> >> Following are some md stats and growth command output: >> >> ___ >> >> $ cat /proc/mdstat >> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] >> [raid4] [raid10] >> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] >> 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] >> bitmap: 1/350 pages [4KB], 2048KB chunk >> >> $ mdadm --detail --scan >> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 >> UUID=da547022:042a6f68:d5fe251e:5e89f263 >> >> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 >> --backup-file=/root/backup.md1_to_r6 >> mdadm: metadata format 1.10 unknown, ignored. >> mdadm: metadata format 1.10 unknown, ignored. >> mdadm level of /dev/md_d1 changed to raid6 >> mdadm: Need to backup 1024K of critical section.. >> mdadm: Cannot set device shape for /dev/md_d1 >> mdadm: aborting level change >> ___ >> >> >> Three questions - >> >> 1. What does the stuff about "metadata format 1.10 unknown" mean? >> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" >> disrepancy between mdsat, --detail and --grow output. > > The metadata format .. unknown means that your /etc/mdadm.conf contains > something like > metadata=1.10 > >> >> 2. Am I doing something wrong? :) > > Not obviously. > >> >> 3. How can I get more info about what is causing the failure to >> initialize the growth? > > Look in the kernel logs. e.g. > dmesg | tail -20 > > immediately after the "mdadm --grow" attempt. > > I just tried the same thing and it worked for me. > > NeilBrown > Thank you very much for the reply. You were right, mdadm.conf indeed contained metadata=1.10. I fixed it, updated the initramfs and rebooted. --- mdadm --detail --scan now gives: sudo mdadm --detail --scan ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 UUID=da547022:042a6f68:d5fe251e:5e89f263 --- I tried the grow command again, and it aborts again. Could it be that the device sizes are wrong? I thought I meticulously created exactly identical partitions on each of the drives. The command output is: sudo mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 --backup-file=/root/backup.md1_to_r6 mdadm level of /dev/md_d1 changed to raid6 mdadm: Need to backup 1024K of critical section.. mdadm: Cannot set device shape for /dev/md_d1 mdadm: aborting level change --- dmesg says: [ 96.482937] raid5: device sdl1 operational as raid disk 2 [ 96.482940] raid5: device sdf1 operational as raid disk 1 [ 96.482942] raid5: device sde1 operational as raid disk 0 [ 96.483299] raid5: allocated 4282kB for md_d1 [ 96.511577] 2: w=1 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0 [ 96.511581] 1: w=2 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0 [ 96.511583] 0: w=3 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0 [ 96.511585] raid5: raid level 6 set md_d1 active with 3 out of 4 devices, algorithm 18 [ 96.511588] RAID5 conf printout: [ 96.511589] --- rd:4 wd:3 [ 96.511591] disk 0, o:1, dev:sde1 [ 96.511593] disk 1, o:1, dev:sdf1 [ 96.511595] disk 2, o:1, dev:sdl1 [ 96.671315] raid5: device sdl1 operational as raid disk 2 [ 96.671318] raid5: device sdf1 operational as raid disk 1 [ 96.671320] raid5: device sde1 operational as raid disk 0 [ 96.671642] raid5: allocated 3230kB for md_d1 [ 96.720331] 2: w=1 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0 [ 96.720334] 1: w=2 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0 [ 96.720336] 0: w=3 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0 [ 96.720338] raid5: raid level 5 set md_d1 active with 3 out of 3 devices, algorithm 2 [ 96.720340] RAID5 conf printout: [ 96.720341] --- rd:3 wd:3 [ 96.720343] disk 0, o:1, dev:sde1 [ 96.720345] disk 1, o:1, dev:sdf1 [ 96.720346] disk 2, o:1, dev:sdl1 [ 100.202834] raid5: device sdl1 operational as raid disk 2 [ 100.202837] raid5: device sdf1 operational as raid disk 1 [ 100.202839] raid5: device sde1 operational as raid disk 0 [ 100.203194] raid5: allocated 4282kB for md_d1 [ 100.241576] 2: w=1 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0 [ 100.241579] 1: w=2 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0 [ 100.241582] 0: w=3 pa=0 pr=4 m=2 a=18 r=4 op1=0 op2=0 [ 100.241584] raid5: raid level 6 set md_d1 active with 3 out of 4 devices, algorithm 18 [ 100.241586] RAID5 conf printout: [ 100.241588] --- rd:4 wd:3 [ 100.241590] disk 0, o:1, dev:sde1 [ 100.241592] disk 1, o:1, dev:sdf1 [ 100.241593] disk 2, o:1, dev:sdl1 [ 100.401030] raid5: device sdl1 operational as raid disk 2 [ 100.401033] raid5: device sdf1 operational as raid disk 1 [ 100.401035] raid5: device sde1 operational as raid disk 0 [ 100.401348] raid5: allocated 3230kB for md_d1 [ 100.460458] 2: w=1 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0 [ 100.460461] 1: w=2 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0 [ 100.460463] 0: w=3 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0 [ 100.460466] raid5: raid level 5 set md_d1 active with 3 out of 3 devices, algorithm 2 [ 100.460467] RAID5 conf printout: [ 100.460468] --- rd:3 wd:3 [ 100.460470] disk 0, o:1, dev:sde1 [ 100.460472] disk 1, o:1, dev:sdf1 [ 100.460474] disk 2, o:1, dev:sdl1 -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: mdadm 3.1.1: level change won't start 2009-12-21 23:18 ` Kristleifur Daðason @ 2009-12-22 18:35 ` Kristleifur Daðason 0 siblings, 0 replies; 14+ messages in thread From: Kristleifur Daðason @ 2009-12-22 18:35 UTC (permalink / raw) To: Neil Brown; +Cc: linux-raid, Neil Brown On Mon, Dec 21, 2009 at 11:18 PM, Kristleifur Daðason <kristleifur@gmail.com> wrote: > On Mon, Dec 21, 2009 at 10:57 PM, Neil Brown <neilb@suse.de> wrote: >> On Mon, 21 Dec 2009 03:41:33 +0000 >> Kristleifur Daðason <kristleifur@gmail.com> wrote: >> >>> Hi all, >>> >>> I wish to convert my 3-drive RAID-5 array to a 6-drive RAID-6. I'm on >>> Linux 2.6.32.2 and have mdadm version 3.1.1 with the 32-bit-array-size >>> patch from here: http://osdir.com/ml/linux-raid/2009-11/msg00534.html >>> >>> I have three live drives and three spares added to the array. When I >>> initialize the command, mdadm does the initial checks and aborts with >>> a "cannot set device shape" without doing anything to the array. >>> >>> Following are some md stats and growth command output: >>> >>> ___ >>> >>> $ cat /proc/mdstat >>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] >>> [raid4] [raid10] >>> md_d1 : active raid5 sdd1[6](S) sdc1[5](S) sdb1[4](S) sdf1[1] sde1[0] sdl1[3] >>> 2930078720 blocks super 1.1 level 5, 256k chunk, algorithm 2 [3/3] [UUU] >>> bitmap: 1/350 pages [4KB], 2048KB chunk >>> >>> $ mdadm --detail --scan >>> ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 >>> UUID=da547022:042a6f68:d5fe251e:5e89f263 >>> >>> $ mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 >>> --backup-file=/root/backup.md1_to_r6 >>> mdadm: metadata format 1.10 unknown, ignored. >>> mdadm: metadata format 1.10 unknown, ignored. >>> mdadm level of /dev/md_d1 changed to raid6 >>> mdadm: Need to backup 1024K of critical section.. >>> mdadm: Cannot set device shape for /dev/md_d1 >>> mdadm: aborting level change >>> ___ >>> >>> >>> Three questions - >>> >>> 1. What does the stuff about "metadata format 1.10 unknown" mean? >>> Notice the "super 1.1" vs. "metadata 1.01" vs. "metadata format 1.10" >>> disrepancy between mdsat, --detail and --grow output. >> >> The metadata format .. unknown means that your /etc/mdadm.conf contains >> something like >> metadata=1.10 >> >>> >>> 2. Am I doing something wrong? :) >> >> Not obviously. >> >>> >>> 3. How can I get more info about what is causing the failure to >>> initialize the growth? >> >> Look in the kernel logs. e.g. >> dmesg | tail -20 >> >> immediately after the "mdadm --grow" attempt. >> >> I just tried the same thing and it worked for me. >> >> NeilBrown >> > > Thank you very much for the reply. You were right, mdadm.conf indeed > contained metadata=1.10. I fixed it, updated the initramfs and > rebooted. > > --- > > mdadm --detail --scan now gives: > > sudo mdadm --detail --scan > ARRAY /dev/md/d1 metadata=1.01 spares=3 name=mamma:d1 > UUID=da547022:042a6f68:d5fe251e:5e89f263 > > --- > I tried the grow command again, and it aborts again. Could it be that > the device sizes are wrong? I thought I meticulously created exactly > identical partitions on each of the drives. The command output is: > > sudo mdadm --grow /dev/md_d1 --level=6 --raid-devices=6 > --backup-file=/root/backup.md1_to_r6 > mdadm level of /dev/md_d1 changed to raid6 > mdadm: Need to backup 1024K of critical section.. > mdadm: Cannot set device shape for /dev/md_d1 > mdadm: aborting level change > > --- I figured it out - I just needed to disable the write-intent bitmap by doing "mdadm --grow --bitmap=none /dev/md_d1". Then the reshape was startable and is going quite well. I'm seeing resync speeds of around 30-40 MB/second during the reshape/relevel. I'm running on 1.5TB SATA drives. Is this speed OK? Thanks everybody for the help and suggestions! -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2009-12-30 5:43 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-12-21 3:41 mdadm 3.1.1: level change won't start Kristleifur Daðason 2009-12-21 4:10 ` Michael Evans 2009-12-21 4:39 ` Kristleifur Daðason 2009-12-21 7:25 ` Michael Evans 2009-12-23 23:47 ` Leslie Rhorer 2009-12-30 5:25 ` Leslie Rhorer 2009-12-30 5:37 ` Neil Brown 2009-12-30 5:43 ` Leslie Rhorer 2009-12-21 9:35 ` Robin Hill 2009-12-21 13:34 ` Kristleifur Daðason 2009-12-21 23:00 ` Neil Brown 2009-12-21 22:57 ` Neil Brown 2009-12-21 23:18 ` Kristleifur Daðason 2009-12-22 18:35 ` Kristleifur Daðason
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).