Superblock limits / conversion

Superblock limits / conversion

[Ryan Wagoner]

I have a RAID 5 array with 3 TB drives. When reading through the list
I realized that the v0.90 superblock doesn't support arrays over 2TB.
This means I can never grow this array to 4 drives?

Is there a way to convert to the v1.0 superblock? I realize I can't
convert to the v1.1 or v1.2 without recreating the array. Is there any
advantages over the v1.1 or v1.2 formats with their placement of the
metadata not at the start of the device?

I read that the kernel autodetect will not work with v1 superblocks. I
have my mdadm.conf correctly configured. When booting this should be
all I need for the array to be assembled? Out of curiosity can v1
superblocks work on the /boot or / array? I would assume not because
it would need to be auto configured before the mdadm.conf could be
read.

Also "v1 supports restarting driver recovery that was interrupted by a
clean shutdown.". Does this mean that a initial resync or recovery
will continue and not restart when rebooting?

Thanks,
Ryan

Re: Superblock limits / conversion

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 2185 bytes --]

On Thu Nov 13, 2008 at 09:46:23AM -0500, Ryan Wagoner wrote:

> I have a RAID 5 array with 3 TB drives. When reading through the list
> I realized that the v0.90 superblock doesn't support arrays over 2TB.
> This means I can never grow this array to 4 drives?
> 
That's my understanding, yes.

> Is there a way to convert to the v1.0 superblock? I realize I can't
> convert to the v1.1 or v1.2 without recreating the array. Is there any
> advantages over the v1.1 or v1.2 formats with their placement of the
> metadata not at the start of the device?
> 
There's no way to convert superblocks as such.  With both the 0.90 and
1.0 superblocks being at the  end of the disk, you could probably
recreate the array without losing data though - you may need to shrink
the filesystem in case the 1.0 superblock uses more space than the 0.90
(and I'd definitely test this with loopback devices first).

> I read that the kernel autodetect will not work with v1 superblocks. I
> have my mdadm.conf correctly configured. When booting this should be
> all I need for the array to be assembled? Out of curiosity can v1
> superblocks work on the /boot or / array? I would assume not because
> it would need to be auto configured before the mdadm.conf could be
> read.
> 
If this is not the / array then the init scripts should assemble it from
the mdadm.conf file.  You can use 1.x superblocks for /boot anyway (as
that's not accessed during boot outside of the bootloader, and that
needs to access the raw disks anyway), and for / if you're using an
initrd (which will contain the mdadm.conf and assemble the array before
switching root).

> Also "v1 supports restarting driver recovery that was interrupted by a
> clean shutdown.". Does this mean that a initial resync or recovery
> will continue and not restart when rebooting?
> 
Not sure on this one, I though all versions would continue (if using a
bitmap).

HTH,
    Robin

-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Superblock limits / conversion

[Justin Piszcz]

On Thu, 13 Nov 2008, Robin Hill wrote:

> On Thu Nov 13, 2008 at 09:46:23AM -0500, Ryan Wagoner wrote:
>
>> I have a RAID 5 array with 3 TB drives. When reading through the list
>> I realized that the v0.90 superblock doesn't support arrays over 2TB.
>> This means I can never grow this array to 4 drives?
>>
> That's my understanding, yes.

I grew 400GiB drives in a raid5 configuration  from 1.8TiB to 3.3TiB 
one at a time with the 0.90 superblock (I believe), is that limit correct?

It was awhile ago but FYI.

Justin.

Re: Superblock limits / conversion

[Steve Fairbairn]

Apologies to Justin for failing to reply to all the first time.

Justin Piszcz wrote:
> 
> I grew 400GiB drives in a raid5 configuration  from 1.8TiB to 3.3TiB one 
> at a time with the 0.90 superblock (I believe), is that limit correct?
> 
> It was awhile ago but FYI.
> 
> Justin.
> 

I have 8 500GB drives in a RAID 5...

[root@space ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md3 : active raid5 sdh1[3] sdg1[2] sdf1[1] sde1[0] sdd1[7] sdc1[5] 
sdb1[4] sda1[6]
       3418687552 blocks level 5, 64k chunk, algorithm 2 [8/8] [UUUUUUUU]
[root@space ~]# mdadm -D /dev/md3
/dev/md3:
         Version : 00.90.03
   Creation Time : Wed Jan  9 18:57:53 2008
      Raid Level : raid5
      Array Size : 3418687552 (3260.31 GiB 3500.74 GB)
   Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
    Raid Devices : 8
   Total Devices : 8
Preferred Minor : 3
     Persistence : Superblock is persistent

     Update Time : Thu Nov 13 05:43:22 2008
           State : clean
  Active Devices : 8
Working Devices : 8
  Failed Devices : 0
   Spare Devices : 0

          Layout : left-symmetric
      Chunk Size : 64K

            UUID : 382c157a:405e0640:c30f9e9e:888a5e63
          Events : 0.1575598

     Number   Major   Minor   RaidDevice State
        0       8       65        0      active sync   /dev/sde1
        1       8       81        1      active sync   /dev/sdf1
        2       8       97        2      active sync   /dev/sdg1
        3       8      113        3      active sync   /dev/sdh1
        4       8       17        4      active sync   /dev/sdb1
        5       8       33        5      active sync   /dev/sdc1
        6       8        1        6      active sync   /dev/sda1
        7       8       49        7      active sync   /dev/sdd1
[root@space ~]# df -k /dev/md3
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/md3             3365042864 2732709404 598146588  83% /Space
[root@space ~]#

Regards,

Steve.

Re: Superblock limits / conversion

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

On Thu Nov 13, 2008 at 10:20:25AM -0500, Justin Piszcz wrote:

>
>
> On Thu, 13 Nov 2008, Robin Hill wrote:
>
>> On Thu Nov 13, 2008 at 09:46:23AM -0500, Ryan Wagoner wrote:
>>
>>> I have a RAID 5 array with 3 TB drives. When reading through the list
>>> I realized that the v0.90 superblock doesn't support arrays over 2TB.
>>> This means I can never grow this array to 4 drives?
>>>
>> That's my understanding, yes.
>
> I grew 400GiB drives in a raid5 configuration  from 1.8TiB to 3.3TiB one at 
> a time with the 0.90 superblock (I believe), is that limit correct?
>
> It was awhile ago but FYI.
>
You're right - it's definitely not correct.  I actually have a 2.7TiB
RAID array with 0.90 superblock right here as well *oops*.  According to
the manual pages, the limit is on _component devices_, not on the
overall RAID array.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Superblock limits / conversion

[Richard Scobie]

Justin Piszcz wrote:
> 
> 
> On Thu, 13 Nov 2008, Robin Hill wrote:
> 
>> On Thu Nov 13, 2008 at 09:46:23AM -0500, Ryan Wagoner wrote:
>>
>>> I have a RAID 5 array with 3 TB drives. When reading through the list
>>> I realized that the v0.90 superblock doesn't support arrays over 2TB.
>>> This means I can never grow this array to 4 drives?
>>>
>> That's my understanding, yes.
> 
> I grew 400GiB drives in a raid5 configuration  from 1.8TiB to 3.3TiB one 
> at a time with the 0.90 superblock (I believe), is that limit correct?

My understanding is that the 2TB limit on 0.90 superblocks, is for md 
component size, not md device size.

Regards,

Richard

Re: Superblock limits / conversion

[Ryan Wagoner]

Thanks for clarifying the 2TB size. I'm still unsure of whether v1
allows for a recovery to continue after a restart. When I moved
everything over to an LSI controller I had to use the system rescue cd
which somehow marked my array as dirty. During the RAID recovery I had
to reboot and it started the recovery over. It only takes 3 hours to
rebuild the 3 TB drives, but if v1 offered this feature I might switch
in the rare case I need it.

I'm assuming enabling bitmaps could do the same thing for v0.90?
However I've heard there is performance degradation of the array from
this.

Ryan

On Thu, Nov 13, 2008 at 1:17 PM, Richard Scobie <richard@sauce.co.nz> wrote:
> Justin Piszcz wrote:
>>
>>
>> On Thu, 13 Nov 2008, Robin Hill wrote:
>>
>>> On Thu Nov 13, 2008 at 09:46:23AM -0500, Ryan Wagoner wrote:
>>>
>>>> I have a RAID 5 array with 3 TB drives. When reading through the list
>>>> I realized that the v0.90 superblock doesn't support arrays over 2TB.
>>>> This means I can never grow this array to 4 drives?
>>>>
>>> That's my understanding, yes.
>>
>> I grew 400GiB drives in a raid5 configuration  from 1.8TiB to 3.3TiB one
>> at a time with the 0.90 superblock (I believe), is that limit correct?
>
> My understanding is that the 2TB limit on 0.90 superblocks, is for md
> component size, not md device size.
>
> Regards,
>
> Richard
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>