Creating a RAID10 (near) for use in a CentOS 6 system

Creating a RAID10 (near) for use in a CentOS 6 system

[maurice]

I come today, hat in hand, for some suggestions on a new install with 
RAID10.

I wish to install CentOS 6 on a new system.
I am using 4 identical drives, and do not see any need to partition them.
They are 2TB Hitachi Ultrastar (enterprise)drives.

I wish to use RAID10, near

Questions:
Syntax: Creating near vs far. Is there a specific syntax to use?

I *think* this is what I need:
mdadm --create /dev/md0 --run --level=10 --chunk=4 --raid-devices=4 
/dev/sda /dev/sdb /dev/sdc /dev/sdd

Next question is about mdadm versions and kernels.
CentOS 6 currently uses 2.6.32-358.2.1.el6
Is there anything in that I should beware of?

OTOH, one may get newer kernels by enabling elrepo

http://elrepo.org/tiki/tiki-index.php

This will get me up to:
     kernel-lt-3.0.70-1.el6.elrepo
     kernel-ml-3.8.4-1.el6.elrepo

Is that desirable in this case?

Any thoughts and suggestions are most welcome..


-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[maurice]

Further to this, can anyone comment on creating partitions on the RAID10 
device,
versus making separate partitions on the devices, for installation of 
the OS?




-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Chris Murphy]

On Mar 25, 2013, at 11:57 AM, maurice <mhilarius@gmail.com> wrote:

> Further to this, can anyone comment on creating partitions on the RAID10 device,
> versus making separate partitions on the devices, for installation of the OS?

I'd partition the array, rather than make separate raid10 arrays from partitions. It's certainly easier to manage if there are issues. If a drive dies, you replace that drive, whereas if you partition the drives you have to partition it exactly right, and then add the partitions as replacements for each array you've created. I think that's a hassle.

The advantage of partitioning the drives with one partition (I'd use GPT for this), is that fdisk/gdisk/parted and other utils will see each drive as being partitioned (for linux software raid) rather than appearing as free space.


Chris Murphy

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[maurice]

On 3/25/2013 5:05 PM, Chris Murphy wrote:
> On Mar 25, 2013, at 11:57 AM, maurice <mhilarius@gmail.com> wrote:
>
>> Further to this, can anyone comment on creating partitions on the RAID10 device,
>> versus making separate partitions on the devices, for installation of the OS?
> I'd partition the array, rather than make separate raid10 arrays from partitions. It's certainly easier to manage if there are issues. If a drive dies, you replace that drive, whereas if you partition the drives you have to partition it exactly right, and then add the partitions as replacements for each array you've created. I think that's a hassle.
>
> The advantage of partitioning the drives with one partition (I'd use GPT for this), is that fdisk/gdisk/parted and other utils will see each drive as being partitioned (for linux software raid) rather than appearing as free space.
>
>
> Chris Murphy
Thanks Chris.

Is there any problem with installing CentOS directly on an array in 
RAID10, as opposed to on to "regular" partitions?

-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Chris Murphy]

On Mar 25, 2013, at 5:21 PM, maurice <mhilarius@gmail.com> wrote:
> 
> Is there any problem with installing CentOS directly on an array in RAID10, as opposed to on to "regular" partitions?

Oh that's a totally different scenario. I thought you wanted the raid10 array for /home or /data or whatever. That's pretty messy, actually, even on Fedora 18/19 which has the benefit of a new anaconda and GRUB2. GRUB2 definitely can assemble md raid10, and read most any file system, find /boot, and load the kernel and initramfs. That's not something I think GRUB Legacy can do, which is what CentOS 6 is using.

I don't know off hand if partitionable arrays are supported by GRUB2. I haven't tested this. And even if GRUB2 does recognize the partitioned array, the initramfs I think needs kpartx built into it, in order for the kernel to have the ability to find and then mount rootfs. And even if all of that it true, I don't think anaconda new or old will let you install the system the way you're thinking. Fedora 18/19's anaconda does not see arrays, it sees actual physical drives. And it wants to create partitions on those devices, not partitions on the array. It might be possible to do what you want via kickstart.

I have successfully created a bootable raid 10, where /boot, /, /home were all on a four disk raid10. But that's with Fedora 18, which uses GRUB2.

I'm unsure of your use case, but I'm thinking you may be better off with a small SSD for the boot, swap and system disk, and then use a separate raid10 array for /home, /opt, /data, whatever else you want. Otherwise it's a bit of a rabbit hole.


Chris Murphy

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[maurice]

On 3/25/2013 5:44 PM, Chris Murphy wrote:
> ..Oh that's a totally different scenario.
> I thought you wanted the raid10 array for /home or /data or whatever.
> That's pretty messy, actually, even on Fedora 18/19 which has the 
> benefit of a new anaconda and GRUB2.
> GRUB2 definitely can assemble md raid10, and read most any file 
> system, find /boot, and load the kernel and initramfs.
> That's not something I think GRUB Legacy can do, which is what CentOS 
> 6 is using.
>
> ..
>
> I'm unsure of your use case, but I'm thinking you may be better off 
> with a small SSD for the boot, swap and system disk,
> and then use a separate raid10 array for /home, /opt, /data, whatever 
> else you want.
> Otherwise it's a bit of a rabbit hole.
>
Fair enough then.

What about partitioning the drives first, installing a smallish 
partition for boot, then a second much larger partition on each for the 
RAID10?
Do you think that the CentOS installer and GRUB would have any issues 
with that layout?


Thanks Chris.

-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Chris Murphy]

On Mar 25, 2013, at 6:05 PM, maurice <mhilarius@gmail.com> wrote:
> 
> What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?

What's the advantage of this compared to a separate drive? In your proposed scenario, if the /boot drive dies, you have an unbootable system anyway. 

> Do you think that the CentOS installer and GRUB would have any issues with that layout?

No. It'll be fine with that. It does partition drives though, not arrays.


Chris Murphy

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Maurice Hilarius]

On 3/25/2013 6:40 PM, Chris Murphy wrote:
> On Mar 25, 2013, at 6:05 PM, maurice <mhilarius@gmail.com> wrote:
>> What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?
> What's the advantage of this compared to a separate drive? In your proposed scenario, if the /boot drive dies, you have an unbootable system anyway.
Firstly, there would be 4 redundant copies of the boot. Easy enough to 
accomplish. If any one goes down, easy to pick one of the others.
Secondly, there is only room for 4 drives in this chassis
Thirdly, cost

Now, back to my original question:
Making the RAID10, near.
In reading the man pages, it seems near 2 is the default option for create?

Is there any reason to be wary of the stock CentOS 6 2.6.32 kernel?


-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Chris Murphy]

On Mar 25, 2013, at 6:47 PM, Maurice Hilarius <maurice@harddata.com> wrote:

> On 3/25/2013 6:40 PM, Chris Murphy wrote:
>> On Mar 25, 2013, at 6:05 PM, maurice <mhilarius@gmail.com> wrote:
>>> What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?
>> What's the advantage of this compared to a separate drive? In your proposed scenario, if the /boot drive dies, you have an unbootable system anyway.
> Firstly, there would be 4 redundant copies of the boot.

AFAIK you'll have to make those 4 copies yourself, the installer won't do it unless you use kickstart, and it won't partition the drives correctly for it. And you'll have to keep them sync'd. And you'll have to manually install grub to all four disks, as the installer will only install to one disk.

> Secondly, there is only room for 4 drives in this chassis.

USB stick, ext2 to avoid journal wear. And only put boot on it. rootfs can go on the array. And you can setup a swapfile to go in rootfs instead of a swap partition.

> 
> In reading the man pages, it seems near 2 is the default option for create?
> 
> Is there any reason to be wary of the stock CentOS 6 2.6.32 kernel?

No idea. I'd probably use a newer one from EL repo, and keep the 6.4 kernel handy as a fallback which I think is now 2.6.32.358.


Chris Murphy

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Chris Murphy]

On Mar 25, 2013, at 6:51 PM, maurice <mhilarius@gmail.com> wrote:

> On 3/25/2013 6:40 PM, Chris Murphy wrote:
>> ..No. It'll be fine with that. It does partition drives though, not arrays. Chris Murphy
> 
> Are you saying that it could not create the partitions on the RAID?

The only partition layout I've seen any version of anaconda create is partitioning physical devices, and then adding those partitions as member devices in an array. Maybe partitionable arrays are possible with kickstart.

> So, would I have to prepare the RAID first on another machine?

I haven't tried doing this with the old anaconda, so I don't know if it will see a pre-created array that it doesn't create itself; let alone if it will honor the partitions in the array.


Chris Murphy

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Brad Campbell]

On 26/03/13 08:05, maurice wrote:

> What about partitioning the drives first, installing a smallish
> partition for boot, then a second much larger partition on each for the
> RAID10?
> Do you think that the CentOS installer and GRUB would have any issues
> with that layout?

This is how I have a Debian server set up.
The system is on 6 SSD's. Each ssd has a small partition and a big 
partition. All the small partitions are set up in a 6 way RAID1 as ext2 
for /boot, and the big partitions are in a RAID10,n2. This RAID10 is 
then partitioned with GPT to provide / /home /opt.

Can't comment on the Centos part, but the method is sound.

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[maurice]

On 3/26/2013 3:35 AM, Brad Campbell wrote:
> ..
> The system is on 6 SSD's. Each ssd has a small partition and a big 
> partition.
> All the small partitions are set up in a 6 way RAID1 as ext2 for 
> /boot, and the big partitions are in a RAID10,n2.
> This RAID10 is then partitioned with GPT to provide / /home /opt.
>
> Can't comment on the Centos part, but the method is sound.
>
>
Ah, good idea on the small RAID1. Keeps it all synchronized.
Nice.

Thank you!



-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Roy Sigurd Karlsbakk]

If you have 2x4TB and you want to setup the lot for booting and data, I'd recommend doing like the NAS boxes I've seen (well, only QNAP 410 IIRC). On that one, they chose to create partitions for / and swap (and perhaps /boot), and equal-ish partitions for those used for data. On those, you can place a RAID-10, or perhaps two RAID-1s and use LVM to stripe them. The latter is far more flexible, as RAID-10 lacks most of the flexibility other RAID leverls have.

If you don't need the extra IOPS from the RAID-10, I'd recommend using RAID-6 instead for even futher safety and flexibility (the ability to change to RAID-5 in need of more space, easier to grow etc).

roy

----- Opprinnelig melding -----
> I come today, hat in hand, for some suggestions on a new install with
> RAID10.
> 
> I wish to install CentOS 6 on a new system.
> I am using 4 identical drives, and do not see any need to partition
> them.
> They are 2TB Hitachi Ultrastar (enterprise)drives.
> 
> I wish to use RAID10, near
> 
> Questions:
> Syntax: Creating near vs far. Is there a specific syntax to use?
> 
> I *think* this is what I need:
> mdadm --create /dev/md0 --run --level=10 --chunk=4 --raid-devices=4
> /dev/sda /dev/sdb /dev/sdc /dev/sdd
> 
> Next question is about mdadm versions and kernels.
> CentOS 6 currently uses 2.6.32-358.2.1.el6
> Is there anything in that I should beware of?
> 
> OTOH, one may get newer kernels by enabling elrepo
> 
> http://elrepo.org/tiki/tiki-index.php
> 
> This will get me up to:
> kernel-lt-3.0.70-1.el6.elrepo
> kernel-ml-3.8.4-1.el6.elrepo
> 
> Is that desirable in this case?
> 
> Any thoughts and suggestions are most welcome..
> 
> 
> --
> Cheers,
> Maurice Hilarius
> eMail: /mhilarius@gmail.com/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

-- 
Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy@karlsbakk.net
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Maurice Hilarius]

On 3/26/2013 2:21 PM, Roy Sigurd Karlsbakk wrote:
> If you have 2x4TB and you want to setup the lot for booting and data, I'd recommend doing like the NAS boxes I've seen (well, only QNAP 410 IIRC). On that one, they chose to create partitions for / and swap (and perhaps /boot), and equal-ish partitions for those used for data. On those, you can place a RAID-10, or perhaps two RAID-1s and use LVM to stripe them. The latter is far more flexible, as RAID-10 lacks most of the flexibility other RAID leverls have.
I have 4 x 2TB, actually.
I am in the middle of this install now
CentOS Anaconda installer seems to be happy with me creating a 4 disk 
RAID1  (thanks for the suggestion, Brad C!) for /boot.
And I created a 4 disk RAID10 for swap, and another for /

> If you don't need the extra IOPS from the RAID-10, I'd recommend using RAID-6 instead for even futher safety and flexibility (the ability to change to RAID-5 in need of more space, easier to grow etc).
I want the performance.
Yes, RAID6 is an option, too, but R10 should substantially outperform 
it, and makes a lot less CPU load for the trouble.
As far as growth goes, only 4 bays available , so not a factor in this 
case, at least.

Once done I will run some tests and confirm that the performance is good 
( or not).

I still have heard nothing about any problems with the versions of 
kernels and mdadm in this distro version.
So are there none??

Anyone?

-- 
Cheers,
Maurice Hilarius
eMail: /mhilarius@gmail.com/

Re: Creating a RAID10 (near) for use in a CentOS 6 system

[Roy Sigurd Karlsbakk]

> On 3/26/2013 2:21 PM, Roy Sigurd Karlsbakk wrote:
> > If you have 2x4TB and you want to setup the lot for booting and
> > data, I'd recommend doing like the NAS boxes I've seen (well, only
> > QNAP 410 IIRC). On that one, they chose to create partitions for /
> > and swap (and perhaps /boot), and equal-ish partitions for those
> > used for data. On those, you can place a RAID-10, or perhaps two
> > RAID-1s and use LVM to stripe them. The latter is far more flexible,
> > as RAID-10 lacks most of the flexibility other RAID leverls have.

> I have 4 x 2TB, actually.

Erm, that's what I meant. I just wouldn't recommend using RAID-10, since it isn't very flexible. LVM on top of multiple mirrors is better.

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy@karlsbakk.net
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html