Growing Linear RAID

Growing Linear RAID

[senthilkumar.muthukalai]

Hi,

I am trying to add a disk to a Linear Raid which was created with 2
disks.
We use linux-2.6.18 and mdadm-2.6.4.
I guess these support the above mentioned disk addition.
But I get the error quoted below.
Could you pls guide me?

Observations
------------

[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow -n 3 /dev/sata3 
mdadm: --size, --raiddisks, and --add are exclusing in --grow mode
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow -n 3            
mdadm: linear array /dev/md0 cannot be reshaped.
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3     
mdadm: Cannot add new disk to this array
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --add /dev/sata3
mdadm: add new device failed for /dev/sata3 as 2: No space left on
device
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow -n 3 --add /dev/sata3
mdadm: --size, --raiddisks, and --add are exclusing in --grow mode
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# 
[root@NAS00226b1755a0 ~]# cat /proc/mdstat 
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4] 
md0 : active linear sdb[1] sda[0]
      444265856 blocks super 1.2 32k rounding
      
unused devices: <none>
[root@NAS00226b1755a0 ~]# mdadm -D /dev/md0 
/dev/md0:
        Version : 01.02.03
  Creation Time : Wed Dec  9 03:02:50 2009
     Raid Level : linear
     Array Size : 444265856 (423.68 GiB 454.93 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed Dec  9 03:02:50 2009
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

       Rounding : 32K

           Name : 0
           UUID : 041dce1e:adafc8da:525455fe:5a9d950f
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sata1
       1       8       16        1      active sync   /dev/sata2

Thanks,
Senthil M

Re: Growing Linear RAID

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

On Wed Dec 09, 2009 at 08:31:54PM +0530, senthilkumar.muthukalai@wipro.com wrote:

> Hi,
> 
> I am trying to add a disk to a Linear Raid which was created with 2
> disks.
> We use linux-2.6.18 and mdadm-2.6.4.
> I guess these support the above mentioned disk addition.
> 
Why do you guess this?  The manual page should tell you exactly what's
supported, and even mdadm 3.1.1 makes no mention of support for growing
linear RAID:

        Currently supported growth options including changing the active
        size of component devices and  changing  the  number  of active
        devices  in RAID levels 1/4/5/6, changing the RAID level between
        1, 5, and 6, changing the chunk size and layout  for RAID5  and
        RAID5, as well as adding or removing a write-intent bitmap

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Growing Linear RAID

[Neil Brown]

On Wed, 9 Dec 2009 15:30:02 +0000
Robin Hill <robin@robinhill.me.uk> wrote:

> On Wed Dec 09, 2009 at 08:31:54PM +0530, senthilkumar.muthukalai@wipro.com wrote:
> 
> > Hi,
> > 
> > I am trying to add a disk to a Linear Raid which was created with 2
> > disks.
> > We use linux-2.6.18 and mdadm-2.6.4.
> > I guess these support the above mentioned disk addition.
> > 
> Why do you guess this?  The manual page should tell you exactly what's
> supported, and even mdadm 3.1.1 makes no mention of support for growing
> linear RAID:
> 
>         Currently supported growth options including changing the active
>         size of component devices and  changing  the  number  of active
>         devices  in RAID levels 1/4/5/6, changing the RAID level between
>         1, 5, and 6, changing the chunk size and layout  for RAID5  and
>         RAID5, as well as adding or removing a write-intent bitmap
> 
>

Hmm... guess I should update that documentation, shouldn't I .....

Adding a device to a linear is supported since 2.6.18.

NeilBrown

Re: Growing Linear RAID

[Neil Brown]

On Wed, 9 Dec 2009 20:31:54 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Hi,
> 
> I am trying to add a disk to a Linear Raid which was created with 2
> disks.
> We use linux-2.6.18 and mdadm-2.6.4.
> I guess these support the above mentioned disk addition.

Yes, should work.  This functionality was introduced in 
2.6.18.


> root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3  

This is the correct usage but,
   
> mdadm: Cannot add new disk to this array

This indicates that something went wrong.

Do you get any messages in the kernel logs at this point?

NeilBrown

RE: Growing Linear RAID

[senthilkumar.muthukalai]

Neil,

I get only the following message under /var/log/

[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3 
mdadm: Cannot add new disk to this array

Dec  9 20:37:58 NAS00226b1755a0 kernel: md: export_rdev(sdc)

[root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata4
mdadm: Cannot add new disk to this array

Dec  9 20:38:30 NAS00226b1755a0 kernel: md: export_rdev(sdd)

Thanks,
Senthil M

-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de] 
Sent: Thursday, December 10, 2009 7:04 AM
To: SenthilKumar Muthukalai (WT01 - Telecom Equipment)
Cc: linux-raid@vger.kernel.org
Subject: Re: Growing Linear RAID

On Wed, 9 Dec 2009 20:31:54 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Hi,
> 
> I am trying to add a disk to a Linear Raid which was created with 2
> disks.
> We use linux-2.6.18 and mdadm-2.6.4.
> I guess these support the above mentioned disk addition.

Yes, should work.  This functionality was introduced in 
2.6.18.


> root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3  

This is the correct usage but,
   
> mdadm: Cannot add new disk to this array

This indicates that something went wrong.

Do you get any messages in the kernel logs at this point?

NeilBrown

Re: Growing Linear RAID

[Neil Brown]

On Thu, 10 Dec 2009 13:54:29 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Neil,
> 
> I get only the following message under /var/log/
> 
> [root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3 
> mdadm: Cannot add new disk to this array
> 
> Dec  9 20:37:58 NAS00226b1755a0 kernel: md: export_rdev(sdc)
> 
> [root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata4
> mdadm: Cannot add new disk to this array
> 
> Dec  9 20:38:30 NAS00226b1755a0 kernel: md: export_rdev(sdd)
>

Thanks - that narrows it down quite a bit.

I think that if you run mdadm under 'strace' you will find that the ioctl
fails with ENOSPC.
It seems that growing a linear array didn't work reliably until 2.6.22.

You need 

  commit a778b73ff793d34c0082f76110f73a7754915067

to make it work properly.

NeilBrown

RE: Growing Linear RAID

[senthilkumar.muthukalai]

Thanks Neil.
Let me try the patch.

- Senthil M

-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de] 
Sent: Thursday, December 10, 2009 3:00 PM
To: SenthilKumar Muthukalai (WT01 - Telecom Equipment)
Cc: linux-raid@vger.kernel.org
Subject: Re: Growing Linear RAID

On Thu, 10 Dec 2009 13:54:29 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Neil,
> 
> I get only the following message under /var/log/
> 
> [root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3 
> mdadm: Cannot add new disk to this array
> 
> Dec  9 20:37:58 NAS00226b1755a0 kernel: md: export_rdev(sdc)
> 
> [root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata4
> mdadm: Cannot add new disk to this array
> 
> Dec  9 20:38:30 NAS00226b1755a0 kernel: md: export_rdev(sdd)
>

Thanks - that narrows it down quite a bit.

I think that if you run mdadm under 'strace' you will find that the
ioctl
fails with ENOSPC.
It seems that growing a linear array didn't work reliably until 2.6.22.

You need 

  commit a778b73ff793d34c0082f76110f73a7754915067

to make it work properly.

NeilBrown

RE: Growing Linear RAID

[senthilkumar.muthukalai]

Neil,

I applied the patch but the disk is getting added as a spare.
Pls guide me here.

Thanks,
Senthil M

-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de] 
Sent: Thursday, December 10, 2009 3:00 PM
To: SenthilKumar Muthukalai (WT01 - Telecom Equipment)
Cc: linux-raid@vger.kernel.org
Subject: Re: Growing Linear RAID

On Thu, 10 Dec 2009 13:54:29 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Neil,
> 
> I get only the following message under /var/log/
> 
> [root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata3 
> mdadm: Cannot add new disk to this array
> 
> Dec  9 20:37:58 NAS00226b1755a0 kernel: md: export_rdev(sdc)
> 
> [root@NAS00226b1755a0 ~]# mdadm /dev/md0 --grow --add /dev/sata4
> mdadm: Cannot add new disk to this array
> 
> Dec  9 20:38:30 NAS00226b1755a0 kernel: md: export_rdev(sdd)
>

Thanks - that narrows it down quite a bit.

I think that if you run mdadm under 'strace' you will find that the
ioctl
fails with ENOSPC.
It seems that growing a linear array didn't work reliably until 2.6.22.

You need 

  commit a778b73ff793d34c0082f76110f73a7754915067

to make it work properly.

NeilBrown

Re: Growing Linear RAID

[Neil Brown]

On Wed, 16 Dec 2009 16:32:30 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Neil,
> 
> I applied the patch but the disk is getting added as a spare.
> Pls guide me here.
>

While I would like to help, it is hard to justify spending much time getting
something working on an old kernel.  Maybe you should try to resolve whatever
issues are keeping you from upgrading to a newer kernel.

At the very least, you would need to provide me with a lot more information
about what if happening.  Exactly what patch you applies, exactly what
command you run, exactly what the output is, exactly what kernel messages you
get, and exactly what the content of "/proc/mdstat" is both before and after.
Even with that I cannot promise anything.

NeilBrown

RE: Growing Linear RAID

[senthilkumar.muthukalai]

Hi Neil,

Upgrading the kernel is quite impossible as there are lots of
open-source packages bundled in our NAS product which would need
enormous testing and effort to check the compliance of the fixes and
features that had gone in all these days, if we upgrade.

Kindly check the observations stated below and see if you can help.
We have linux-2.6.18 and mdadm-2.6.4.

[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md0 : active linear sdb[0]
      156290880 blocks super 1.2 32k rounding

unused devices: <none>
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# mdadm -D /dev/md0
/dev/md0:
        Version : 01.02.03
  Creation Time : Wed Dec 23 07:45:42 2009
     Raid Level : linear
     Array Size : 156290880 (149.05 GiB 160.04 GB)
   Raid Devices : 1
  Total Devices : 1
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed Dec 23 07:45:42 2009
          State : clean
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

       Rounding : 32K

           Name : 0
           UUID : df516a9e:356dc2c0:851c942d:ccab4bda
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sata1
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# mdadm --grow /dev/md0 --add /dev/sata2
md: bind<sda>
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md0 : active linear sda[1](S) sdb[0]
      156290880 blocks super 1.2 32k rounding

unused devices: <none>
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# mdadm -D /dev/md0
/dev/md0:
        Version : 01.02.03
  Creation Time : Wed Dec 23 07:45:42 2009
     Raid Level : linear
     Array Size : 156290880 (149.05 GiB 160.04 GB)
   Raid Devices : 1
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed Dec 23 07:45:42 2009
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1

       Rounding : 32K

           Name : 0
           UUID : df516a9e:356dc2c0:851c942d:ccab4bda
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sata1

       1       8        0        -      spare   /dev/sata2

Thanks,
Senthil M

-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de] 
Sent: Thursday, December 17, 2009 4:45 AM
To: SenthilKumar Muthukalai (WT01 - Telecom Equipment)
Cc: linux-raid@vger.kernel.org
Subject: Re: Growing Linear RAID

On Wed, 16 Dec 2009 16:32:30 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Neil,
> 
> I applied the patch but the disk is getting added as a spare.
> Pls guide me here.
>

While I would like to help, it is hard to justify spending much time
getting
something working on an old kernel.  Maybe you should try to resolve
whatever
issues are keeping you from upgrading to a newer kernel.

At the very least, you would need to provide me with a lot more
information
about what if happening.  Exactly what patch you applies, exactly what
command you run, exactly what the output is, exactly what kernel
messages you
get, and exactly what the content of "/proc/mdstat" is both before and
after.
Even with that I cannot promise anything.

NeilBrown

RE: Growing Linear RAID

[senthilkumar.muthukalai]

Hi Neil,

In addition to the below mentioned details, I would like to add the
details of the patch suggested by you earlier.

commit a778b73ff793d34c0082f76110f73a7754915067.

The below mentioned observations were made after applying this patch.

Kindly help.

New year wishes.

Thanks,
Senthil M

-----Original Message-----
From: SenthilKumar Muthukalai (WT01 - Telecom Equipment) 
Sent: Wednesday, December 23, 2009 10:03 PM
To: Neil Brown
Cc: linux-raid@vger.kernel.org
Subject: RE: Growing Linear RAID

Hi Neil,

Upgrading the kernel is quite impossible as there are lots of
open-source packages bundled in our NAS product which would need
enormous testing and effort to check the compliance of the fixes and
features that had gone in all these days, if we upgrade.

Kindly check the observations stated below and see if you can help.
We have linux-2.6.18 and mdadm-2.6.4.

[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md0 : active linear sdb[0]
      156290880 blocks super 1.2 32k rounding

unused devices: <none>
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# mdadm -D /dev/md0
/dev/md0:
        Version : 01.02.03
  Creation Time : Wed Dec 23 07:45:42 2009
     Raid Level : linear
     Array Size : 156290880 (149.05 GiB 160.04 GB)
   Raid Devices : 1
  Total Devices : 1
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed Dec 23 07:45:42 2009
          State : clean
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

       Rounding : 32K

           Name : 0
           UUID : df516a9e:356dc2c0:851c942d:ccab4bda
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sata1
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# mdadm --grow /dev/md0 --add /dev/sata2
md: bind<sda>
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md0 : active linear sda[1](S) sdb[0]
      156290880 blocks super 1.2 32k rounding

unused devices: <none>
[root@NAS0018f8056151 /]#
[root@NAS0018f8056151 /]# mdadm -D /dev/md0
/dev/md0:
        Version : 01.02.03
  Creation Time : Wed Dec 23 07:45:42 2009
     Raid Level : linear
     Array Size : 156290880 (149.05 GiB 160.04 GB)
   Raid Devices : 1
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed Dec 23 07:45:42 2009
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1

       Rounding : 32K

           Name : 0
           UUID : df516a9e:356dc2c0:851c942d:ccab4bda
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sata1

       1       8        0        -      spare   /dev/sata2

Thanks,
Senthil M

-----Original Message-----
From: Neil Brown [mailto:neilb@suse.de] 
Sent: Thursday, December 17, 2009 4:45 AM
To: SenthilKumar Muthukalai (WT01 - Telecom Equipment)
Cc: linux-raid@vger.kernel.org
Subject: Re: Growing Linear RAID

On Wed, 16 Dec 2009 16:32:30 +0530
<senthilkumar.muthukalai@wipro.com> wrote:

> Neil,
> 
> I applied the patch but the disk is getting added as a spare.
> Pls guide me here.
>

While I would like to help, it is hard to justify spending much time
getting
something working on an old kernel.  Maybe you should try to resolve
whatever
issues are keeping you from upgrading to a newer kernel.

At the very least, you would need to provide me with a lot more
information
about what if happening.  Exactly what patch you applies, exactly what
command you run, exactly what the output is, exactly what kernel
messages you
get, and exactly what the content of "/proc/mdstat" is both before and
after.
Even with that I cannot promise anything.

NeilBrown