Linux mdadm --grow RAID6: Something wrong - reshape aborted

Linux mdadm --grow RAID6: Something wrong - reshape aborted

[Ole Tange]

I have a RAID60 that I want to expand.

The current setup is: 2 axles each having 9 disks + 2 spares.

The future wanted setup is: 4 axles each having 10 disks + 1 spare.

So I need to do some --grow to reshape the drives.

I thought this would be enough:

  mdadm -v --grow /dev/md1 --raid-devices=10 --backup-file=/root/back-md1
  mdadm -v --grow /dev/md2 --raid-devices=10 --backup-file=/root/back-md2
  mdadm -v --grow /dev/md0 --raid-devices=4 --add /dev/md3 /dev/md4

The last command works, but the 2 first commands fail with:

  mdadm: Need to backup 7168K of critical section..
  mdadm: /dev/md2: Something wrong - reshape aborted

How can I --grow a RAID6 to use more devices?

Info about the system:

  $ mdadm --version
  mdadm - v3.2.5 - 18th May 2012
  $ uname -r
  3.5.0-17-generic

This URL includes a Makefile to reproduce the problem:
http://serverfault.com/questions/503832/linux-mdadm-grow-raid6-something-wrong-reshape-aborted


/Ole

Re: Linux mdadm --grow RAID6: Something wrong - reshape aborted

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1791 bytes --]

On Thu, 2 May 2013 13:56:09 +0200 Ole Tange <tange@binf.ku.dk> wrote:

> I have a RAID60 that I want to expand.
> 
> The current setup is: 2 axles each having 9 disks + 2 spares.
> 
> The future wanted setup is: 4 axles each having 10 disks + 1 spare.
> 
> So I need to do some --grow to reshape the drives.
> 
> I thought this would be enough:
> 
>   mdadm -v --grow /dev/md1 --raid-devices=10 --backup-file=/root/back-md1
>   mdadm -v --grow /dev/md2 --raid-devices=10 --backup-file=/root/back-md2
>   mdadm -v --grow /dev/md0 --raid-devices=4 --add /dev/md3 /dev/md4
> 
> The last command works, but the 2 first commands fail with:
> 
>   mdadm: Need to backup 7168K of critical section..
>   mdadm: /dev/md2: Something wrong - reshape aborted

I assume you are getting this error when running a test with small loop-back
devices - yes?
Very sensible to test first.  However to reshape a RAID6 from 9 to 10, you
need each device to be at least 7*8*chunksize*2 - possibly a bit more than
that.
Your script uses a chunksize of 128K, so you need at least 15Meg
Your script creates 1Meg devices.

Try your test with a  much smaller chunk size and a somewhat larger device
size.

NeilBrown

> 
> How can I --grow a RAID6 to use more devices?
> 
> Info about the system:
> 
>   $ mdadm --version
>   mdadm - v3.2.5 - 18th May 2012
>   $ uname -r
>   3.5.0-17-generic
> 
> This URL includes a Makefile to reproduce the problem:
> http://serverfault.com/questions/503832/linux-mdadm-grow-raid6-something-wrong-reshape-aborted
> 
> 
> /Ole
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Linux mdadm --grow RAID6: Something wrong - reshape aborted

[Ole Tange]

On Mon, May 6, 2013 at 8:44 AM, NeilBrown <neilb@suse.de> wrote:
> On Thu, 2 May 2013 13:56:09 +0200 Ole Tange <tange@binf.ku.dk> wrote:
>
:
>> I thought this would be enough:
>>
>>   mdadm -v --grow /dev/md1 --raid-devices=10 --backup-file=/root/back-md1
:
>> The last command works, but the 2 first commands fail with:
>>
>>   mdadm: Need to backup 7168K of critical section..
>>   mdadm: /dev/md2: Something wrong - reshape aborted
:
> Very sensible to test first.  However to reshape a RAID6 from 9 to 10, you
> need each device to be at least 7*8*chunksize*2 - possibly a bit more than
> that.
> Your script uses a chunksize of 128K, so you need at least 15Meg
> Your script creates 1Meg devices.
>
> Try your test with a  much smaller chunk size and a somewhat larger device
> size.

Spot on. If I adjust the size of the loop-back devices to 30M instead
of 1M, then everything works.


/Ole