Software, Raid 5, Different Size Drives.

Software, Raid 5, Different Size Drives.

[Jon Hardcastle]

Hi quick Q.. I have a 6 drive array made up of 4 500GB's and 2 750GB's.

I know the array will only take on a size based on the smallest drive. To that end as I phase drives out based on usage etc I replace them with the best value drive I can lay my hands on (upto 1TB~ as i have read a raid 5 array of drives over that size approx start to cause problems with read failures on re-builds - but that is for another thread)

So my array is currently ~2.5GB with 2x250GB currently not in use and I am about to phase out a 500GB and replace with a 1TB. Now i have read that you are better of creating a auto detect raid partition on the drive and adding that rather than adding the entire drive (so sda1 not sda). This is where my question comes in. I have read that 2 1TB(or whatever size) can be different block sizes and hence adding a partition that is the entire drive can cause problems as these sizes will differ.

Can anyone tell me what truth there is in this? Should I actually create a partition  that is always going to be a nice multiple size smaller so i can smooth over the bumps?

(also anyone know if raid5 to raid 6 native support is anywhere near?)

-----------------------
N: Jon Hardcastle
E: Jon@eHardcastle.com
'Do not worry about tomorrow, for tomorrow will bring worries of its own.'
-----------------------

Send instant messages to your online friends http://uk.messenger.yahoo.com 

Re: Software, Raid 5, Different Size Drives.

[NeilBrown]

On Sun, August 16, 2009 4:54 am, Jon Hardcastle wrote:
> Hi quick Q.. I have a 6 drive array made up of 4 500GB's and 2 750GB's.
>
> I know the array will only take on a size based on the smallest drive. To
> that end as I phase drives out based on usage etc I replace them with the
> best value drive I can lay my hands on (upto 1TB~ as i have read a raid 5
> array of drives over that size approx start to cause problems with read
> failures on re-builds - but that is for another thread)
>
> So my array is currently ~2.5GB with 2x250GB currently not in use and I am
> about to phase out a 500GB and replace with a 1TB. Now i have read that
> you are better of creating a auto detect raid partition on the drive and
> adding that rather than adding the entire drive (so sda1 not sda). This is
> where my question comes in. I have read that 2 1TB(or whatever size) can
> be different block sizes and hence adding a partition that is the entire
> drive can cause problems as these sizes will differ.

This was a topic of a recent thread on this list.  Apparently there is an
industry standard which sets out exactly how many sectors a 1TB or 2TB
drive should be, and it seems that all drive manufactures adhere to this.

>
> Can anyone tell me what truth there is in this? Should I actually create a
> partition  that is always going to be a nice multiple size smaller so i
> can smooth over the bumps?

Personally I never create a single partition for an md array.  I either
use the whole drive, or create a number of partitions for different
arrays.  I also avoid in-kernel autodetect.

The question of what "best" may well come down to the start up scripts
that your distro uses and any hidden assumptions that might be in them...

I guess that isn't very helpful though... I can say that either approach
can be made to work fine.  The one issue that you particularly need to be
careful off is the boot sector.  Partitions always leave room for a boot
sector.  If you don't use partitions and you want a separate boot sector,
then v1.2 metadata is the thing to choose....



>
> (also anyone know if raid5 to raid 6 native support is anywhere near?)

Yes. It works with 2.6.30 (or preferrable 2.6.31) and the devel-3.1 branch
from my git tree git://neil.brown.name/mdadm

NeilBrown

RE: Software, Raid 5, Different Size Drives.

[Leslie Rhorer]

> > Can anyone tell me what truth there is in this? Should I actually create
> a
> > partition  that is always going to be a nice multiple size smaller so i
> > can smooth over the bumps?
> 
> Personally I never create a single partition for an md array.  I either
> use the whole drive, or create a number of partitions for different
> arrays.  I also avoid in-kernel autodetect.

	Although I am nowhere nearly the expert Neil is, I'm going to add my
support to his post.  I'm much more familiar with flavors of Unix than with
Linux, but my experience strongly suggests what he says is true.  With a
single drive system, partitions are effectively mandatory, but once one
decides to implement a  multiple drive system, in my estimation the paradigm
changes.  If your implementation requires more than one array to be built
from several common disks, then partitioning is required, but if practical I
prefer to build systems with the main data area supplied by an array of
whole disks.  Of course, there are a number of variations of even this
theme, but personally I like to build a rather small OS disk containing the
root, /var, /etc, and so forth.  Sometimes I will create a separate
partition for /boot.  The rest I put on a single large RAID array of whole
disks with no partitions under or above the array.  Although RAID solutions
are certainly possible for the OS areas, I personally prefer to not
implement RAID for the OS, and simply keep a cold spare drive around with
regular backups of the OS configuration kept online.  It's also been my
experience that, all things considered, it's generally best to limit the
number of partitions as much as possible.  Of course, there can be some
performance penalty involved, and valid arguments can be made for creating
specific partitions to meet various needs.

> The question of what "best" may well come down to the start up scripts
> that your distro uses and any hidden assumptions that might be in them...
> 
> I guess that isn't very helpful though... I can say that either approach
> can be made to work fine.  The one issue that you particularly need to be
> careful off is the boot sector.  Partitions always leave room for a boot
> sector.  If you don't use partitions and you want a separate boot sector,
> then v1.2 metadata is the thing to choose....

	Yeah, what he said.  What's more, booting from an array presents
it's own special requirements.  While it is certainly possible to arrange to
meet all the requirements for booting from an array, unless one is working
with an embedded application, or an enclosure with limited power and / or
drive slots, I much prefer to simply circumvent the issue by booting from a
relatively small, very inexpensive drive and making the data area as simple,
straightforward, and primitive as possible.  If the drive dies, I replace
it.  'No big deal.

Re: Software, Raid 5, Different Size Drives.

[Goswin von Brederlow]

"NeilBrown" <neilb@suse.de> writes:

> On Sun, August 16, 2009 4:54 am, Jon Hardcastle wrote:
>> Hi quick Q.. I have a 6 drive array made up of 4 500GB's and 2 750GB's.
>>
>> I know the array will only take on a size based on the smallest drive. To
>> that end as I phase drives out based on usage etc I replace them with the
>> best value drive I can lay my hands on (upto 1TB~ as i have read a raid 5
>> array of drives over that size approx start to cause problems with read
>> failures on re-builds - but that is for another thread)
>>
>> So my array is currently ~2.5GB with 2x250GB currently not in use and I am
>> about to phase out a 500GB and replace with a 1TB. Now i have read that
>> you are better of creating a auto detect raid partition on the drive and
>> adding that rather than adding the entire drive (so sda1 not sda). This is
>> where my question comes in. I have read that 2 1TB(or whatever size) can
>> be different block sizes and hence adding a partition that is the entire
>> drive can cause problems as these sizes will differ.
>
> This was a topic of a recent thread on this list.  Apparently there is an
> industry standard which sets out exactly how many sectors a 1TB or 2TB
> drive should be, and it seems that all drive manufactures adhere to this.

Also note that raid on sda or sda1 both have exactly the same problem.
You can not use a disk smaller than the minimum disk used in the raid.
Really makes no difference if it is partitioned or not.

If you fear size problems then look up the industry standard size and
check your disk follows that. I've never seen a disk smaller than that
size, only bigger.

>> Can anyone tell me what truth there is in this? Should I actually create a
>> partition  that is always going to be a nice multiple size smaller so i
>> can smooth over the bumps?
>
> Personally I never create a single partition for an md array.  I either
> use the whole drive, or create a number of partitions for different
> arrays.  I also avoid in-kernel autodetect.
>
> The question of what "best" may well come down to the start up scripts
> that your distro uses and any hidden assumptions that might be in them...
>
> I guess that isn't very helpful though... I can say that either approach
> can be made to work fine.  The one issue that you particularly need to be
> careful off is the boot sector.  Partitions always leave room for a boot
> sector.  If you don't use partitions and you want a separate boot sector,
> then v1.2 metadata is the thing to choose....

That is actually a verry important thing. Putting your raid on sda
without leaving space for the bootloader is rather dangerous. Over
time it is easy for the bootloader to get installed on the wrong disk
because sda and sdb switched places or you forgot which drive is the
boot drive and so on.

I now actually put the bootloader on all my drives and all are setup
to have space there. That way the system boots even if I rearange
disks or the boot drive fails and so on.

MfG
        Goswin

Re: Software, Raid 5, Different Size Drives.

[Jon Hardcastle]

--- On Sun, 16/8/09, NeilBrown <neilb@suse.de> wrote:

> From: NeilBrown <neilb@suse.de>
> Subject: Re: Software, Raid 5, Different Size Drives.
> To: Jon@eHardcastle.com
> Cc: linux-raid@vger.kernel.org
> Date: Sunday, 16 August, 2009, 11:45 PM
> On Sun, August 16, 2009 4:54 am, Jon
> Hardcastle wrote:
> > Hi quick Q.. I have a 6 drive array made up of 4
> 500GB's and 2 750GB's.
> >
> > I know the array will only take on a size based on the
> smallest drive. To
> > that end as I phase drives out based on usage etc I
> replace them with the
> > best value drive I can lay my hands on (upto 1TB~ as i
> have read a raid 5
> > array of drives over that size approx start to cause
> problems with read
> > failures on re-builds - but that is for another
> thread)
> >
> > So my array is currently ~2.5GB with 2x250GB currently
> not in use and I am
> > about to phase out a 500GB and replace with a 1TB. Now
> i have read that
> > you are better of creating a auto detect raid
> partition on the drive and
> > adding that rather than adding the entire drive (so
> sda1 not sda). This is
> > where my question comes in. I have read that 2 1TB(or
> whatever size) can
> > be different block sizes and hence adding a partition
> that is the entire
> > drive can cause problems as these sizes will differ.
> 
> This was a topic of a recent thread on this list. 
> Apparently there is an
> industry standard which sets out exactly how many sectors a
> 1TB or 2TB
> drive should be, and it seems that all drive manufactures
> adhere to this.
> 
> >
> > Can anyone tell me what truth there is in this? Should
> I actually create a
> > partition  that is always going to be a nice
> multiple size smaller so i
> > can smooth over the bumps?
> 
> Personally I never create a single partition for an md
> array.  I either
> use the whole drive, or create a number of partitions for
> different
> arrays.  I also avoid in-kernel autodetect.
> 
> The question of what "best" may well come down to the start
> up scripts
> that your distro uses and any hidden assumptions that might
> be in them...
> 
> I guess that isn't very helpful though... I can say that
> either approach
> can be made to work fine.  The one issue that you
> particularly need to be
> careful off is the boot sector.  Partitions always
> leave room for a boot
> sector.  If you don't use partitions and you want a
> separate boot sector,
> then v1.2 metadata is the thing to choose....
> 
> 
> 
> >
> > (also anyone know if raid5 to raid 6 native support is
> anywhere near?)
> 
> Yes. It works with 2.6.30 (or preferrable 2.6.31) and the
> devel-3.1 branch
> from my git tree git://neil.brown.name/mdadm
> 
> NeilBrown
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Any idea where i find out this 'industry standard'?

My setup is build on drives that have a partition on them, so the finer points over the pro's and con's of this are moot for me.. until i come to replace the array. Perhaps someone can tell me tho. If I do use a partition on the 1TB drive, and it is is x blocks big. If i add another 1TB drive that is x +/- 10 blocks in size. Will the array just use the size of the smaller and grow accordingly?

-----------------------
N: Jon Hardcastle
E: Jon@eHardcastle.com
'Do not worry about tomorrow, for tomorrow will bring worries of its own.'
-----------------------

Send instant messages to your online friends http://uk.messenger.yahoo.com 
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software, Raid 5, Different Size Drives.

[Michał Przyłuski]

Hi,

2009/8/17 Jon Hardcastle <jd_hardcastle@yahoo.com>:
> Any idea where i find out this 'industry standard'?
>
> My setup is build on drives that have a partition on them, so the finer points over the pro's and con's of this are moot for me.. until i come to replace the array. Perhaps someone can tell me tho. If I do use a partition on the 1TB drive, and it is is x blocks big. If i add another 1TB drive that is x +/- 10 blocks in size. Will the array just use the size of the smaller and grow accordingly?

You might want to check out that linux-raid thread,
http://www.issociate.de/board/post/497059/Adding_a_smaller_drive.html
. It seems to be one of IDEMA standards. I cannot say I've seen it
myself, but the list seems to be pretty confident about it. And, it
appears that standard defines *exact* block count. It would be
pointless if it was just a minimum, as you might've built an array
with "more than minimum" drives, and then couldn't replace them with
drives with exact block count.

On top of that, my personal opinion is that having a partition doesn't
change anything in that matter. You could still, theoretically
assuming there's no IDEMA standard, find a smaller drive, and make a
partition that is smaller than partitions on other drives.

Cheers,
Mike

Re: Software, Raid 5, Different Size Drives.

[John Robinson]

On 17/08/2009 15:07, Jon Hardcastle wrote:
[...]
> Any idea where i find out this 'industry standard'?

Here: 
http://www.idema.org/_smartsite/modules/local/data_file/show_file.php?cmd=download&data_file_id=1066

Cheers,

John.

Re: Software, Raid 5, Different Size Drives.

[Martin K. Petersen]

>>>>> "John" == John Robinson <john.robinson@anonymous.org.uk> writes:

John> On 17/08/2009 15:07, Jon Hardcastle wrote: [...]
>> Any idea where i find out this 'industry standard'?

John> Here:
John> http://www.idema.org/_smartsite/modules/local/data_file/show_file.php?cmd=download&data_file_id=1066

I should add that there's a more up-to-date LBA spec with more detail.
Not sure why version 3 hasn't been published on the IDEMA website--it's
over a year old by now.  I'll see if I can nudge the right people...

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: Software, Raid 5, Different Size Drives.

[NeilBrown]

On Tue, August 18, 2009 12:07 am, Jon Hardcastle wrote:
> My setup is build on drives that have a partition on them, so the finer
> points over the pro's and con's of this are moot for me.. until i come to
> replace the array. Perhaps someone can tell me tho. If I do use a
> partition on the 1TB drive, and it is is x blocks big. If i add another
> 1TB drive that is x +/- 10 blocks in size. Will the array just use the
> size of the smaller and grow accordingly?

The effective size of the drive is rounded to the chunk size
which defaults to 64K.

If you try to add a drive or partition to an array and it does not
have enough space for the data plus metadata, then md will not
accept the drive.
If you add a drive or partition that has more space than is needed,
the extra will simple not be used.
If you eventually replace all drives with drives that have more space,
then every drive will have space on it that is unused.
You can, if you choose, then expand the array to use that extra space
with --grow --size=max.

I hope that answers your question.

NeilBrown

Re: Software, Raid 5, Different Size Drives.

[Jon Hardcastle]

--- On Sun, 16/8/09, NeilBrown <neilb@suse.de> wrote:

...

> >
> > (also anyone know if raid5 to raid 6 native support is
> anywhere near?)
> 
> Yes. It works with 2.6.30 (or preferrable 2.6.31) and the
> devel-3.1 branch
> from my git tree git://neil.brown.name/mdadm
> 
> NeilBrown
> 

I am probably being a bit thick here, but I am running 2.6.8 as per

http://packages.gentoo.org/package/sys-fs/mdadm

Does this imply that I already have these features.. as i am pretty sure i don't (unless it is undocumented)

I want to add 2 more drives to my array and grow it/reshape from raid 5 to 6...


-----------------------
N: Jon Hardcastle
E: Jon@eHardcastle.com
'Do not worry about tomorrow, for tomorrow will bring worries of its own.'
-----------------------



      

Re: Software, Raid 5, Different Size Drives.

[NeilBrown]

On Mon, September 7, 2009 8:35 pm, Jon Hardcastle wrote:
> --- On Sun, 16/8/09, NeilBrown <neilb@suse.de> wrote:
>
> ...
>
>> >
>> > (also anyone know if raid5 to raid 6 native support is
>> anywhere near?)
>>
>> Yes. It works with 2.6.30 (or preferrable 2.6.31) and the
>> devel-3.1 branch
>> from my git tree git://neil.brown.name/mdadm
>>
>> NeilBrown
>>
>
> I am probably being a bit thick here, but I am running 2.6.8 as per
>
> http://packages.gentoo.org/package/sys-fs/mdadm
>
> Does this imply that I already have these features.. as i am pretty sure i
> don't (unless it is undocumented)

No you don't.
2.6.8 is < 3.1, so your mdadm doesn't have the required support
And you need a kernel which is at least 2.6.30.

>
> I want to add 2 more drives to my array and grow it/reshape from raid 5 to
> 6...

You'll need to be a bit more patient.

NeilBrown