Can't reshape RAID1 to RAID5 due to chunk size

Can't reshape RAID1 to RAID5 due to chunk size

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 1666 bytes --]

Hello,

# mdadm --detail /dev/md1 
/dev/md1:
        Version : 1.2
  Creation Time : Fri May 27 09:50:54 2011
     Raid Level : raid1
     Array Size : 488372863 (465.75 GiB 500.09 GB)
  Used Dev Size : 488372863 (465.75 GiB 500.09 GB)
   Raid Devices : 2
  Total Devices : 3
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Wed Sep 12 10:54:33 2012
          State : active 
 Active Devices : 2
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 1

           Name : avdeb:1  (local to host avdeb)
           UUID : e29a222d:e6245302:5ff3f834:ad471a01
         Events : 26

    Number   Major   Minor   RaidDevice State
       0       8       82        0      active sync   /dev/sdf2
       1       8       34        1      active sync   /dev/sdc2

       2       8       50        -      spare   /dev/sdd2

# mdadm --grow /dev/md1 --chunk=64K --level=5 --raid-devices=3
mdadm: New chunk size does not divide component size

-----
Shouldn't mdadm be able to figure out a way to somehow proceed in this case? :)
So what if it does not divide, I am increasing the array size by 33%, it
has plenty of new space, why not leave a bit of it in the end of all devices so
the chunk size does divide...
Also I heard of cases (on #linux-raid IRC, I think) when people reshaped like
this without specifying the chunk size explicitly, and ended up with something
like a 4K chunk, which is certainly less than optimal.

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Can't reshape RAID1 to RAID5 due to chunk size

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2472 bytes --]

On Wed, 12 Sep 2012 10:59:56 +0600 Roman Mamedov <rm@romanrm.ru> wrote:

> Hello,
> 
> # mdadm --detail /dev/md1 
> /dev/md1:
>         Version : 1.2
>   Creation Time : Fri May 27 09:50:54 2011
>      Raid Level : raid1
>      Array Size : 488372863 (465.75 GiB 500.09 GB)
>   Used Dev Size : 488372863 (465.75 GiB 500.09 GB)
>    Raid Devices : 2
>   Total Devices : 3
>     Persistence : Superblock is persistent
> 
>   Intent Bitmap : Internal
> 
>     Update Time : Wed Sep 12 10:54:33 2012
>           State : active 
>  Active Devices : 2
> Working Devices : 3
>  Failed Devices : 0
>   Spare Devices : 1
> 
>            Name : avdeb:1  (local to host avdeb)
>            UUID : e29a222d:e6245302:5ff3f834:ad471a01
>          Events : 26
> 
>     Number   Major   Minor   RaidDevice State
>        0       8       82        0      active sync   /dev/sdf2
>        1       8       34        1      active sync   /dev/sdc2
> 
>        2       8       50        -      spare   /dev/sdd2
> 
> # mdadm --grow /dev/md1 --chunk=64K --level=5 --raid-devices=3
> mdadm: New chunk size does not divide component size
> 
> -----
> Shouldn't mdadm be able to figure out a way to somehow proceed in this case? :)
> So what if it does not divide, I am increasing the array size by 33%, it
> has plenty of new space, why not leave a bit of it in the end of all devices so
> the chunk size does divide...
> Also I heard of cases (on #linux-raid IRC, I think) when people reshaped like
> this without specifying the chunk size explicitly, and ended up with something
> like a 4K chunk, which is certainly less than optimal.
> 

Yes, there is room for improvement here.

The difficulty is that the RAID1 must be converted to a 2-device RAID5 before
devices can be added, and the RAID5 must have a chunk size that is a multiple
of 4K.
Your array cannot even manage 1K.

Newer versions of mdadm will create raid1 arrays to be a multiple of 64K (I
think) so this will be less of a problem.

What you need to do is:
  - make sure the filesystem in /dev/md1 doesn't use the last 3k (it probably 
    uses a 4K block size, so cannot use that last bit
  - resize the array down to 488372860K  (mdadm -G /dev/md0 --size 488372860)
  - convert to a 2-device RAID5 with a 4K chunk size:
      mdadm -G /dev/md1 -c 4K -l5 -n2
  - convert to a 3-device RAID5 with a 64K chunk size:
      mdadm -G /dev/md1 -c 64k -n3

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Can't reshape RAID1 to RAID5 due to chunk size

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

On Wed, 19 Sep 2012 16:20:58 +1000
NeilBrown <neilb@suse.de> wrote:

> What you need to do is:
>   - make sure the filesystem in /dev/md1 doesn't use the last 3k (it probably 
>     uses a 4K block size, so cannot use that last bit
>   - resize the array down to 488372860K  (mdadm -G /dev/md0 --size 488372860)
>   - convert to a 2-device RAID5 with a 4K chunk size:
>       mdadm -G /dev/md1 -c 4K -l5 -n2
>   - convert to a 3-device RAID5 with a 64K chunk size:
>       mdadm -G /dev/md1 -c 64k -n3
> 
> NeilBrown

Thanks, Neil.

I have since finished this migration in another way, luckily I had some spare
drives lying around, so had an option to create a separate RAID5 from scratch
and move over & replace the drives one by one.

By the way the "want_replacement" feature is totally awesome, thanks to it at
no point in my operations I was left without redundancy (after all a N-disk
degraded RAID5 reliability-wise is like a RAID0 of N disks, and that's
something I wanted to avoid having even for a short period).

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]