Help needed: Partitioned software raid > 2TB

Help needed: Partitioned software raid > 2TB

[Christian Schmidt]

Hi everyone,

I added a drive to a linux software RAID-5 last night. Now that worked
fine... until I changed the partition table.

Disk /dev/md_d5: 2499.9 GB, 2499990978560 bytes
2 heads, 4 sectors/track, 610349360 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d5p1               1      244142      976566   83  Linux
/dev/md_d5p2          244143     5126956    19531256   8e  Linux LVM
/dev/md_d5p3         5126957   488279488  1932610128   8e  Linux LVM

This is how the layout looked before the changes.

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d5p1               1      244142      976566   83  Linux
/dev/md_d5p2          244143     5126956    19531256   8e  Linux LVM
/dev/md_d5p3         5126957    73478448   273405968   8e  Linux LVM

This is how it looks now - the fdisk tools (fdisk/cfdisk) are unable to
crate a single partition > 2TB. Of course now LVM complains that the PV
is too small:

  WARNING: Physical Volume /dev/md_d5p3 is too large for underlying device
  PV Size               260.74 GB / not usable 8192.00 EB

I was able to restore a somewhat close partition table now... Close
enough for the LVM to work correctly.

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d5p1               1      244142      976566   83  Linux
/dev/md_d5p2          244143     5126956    19531256   8e  Linux LVM
/dev/md_d5p3         5126957   488279495  1932610156   8e  Linux LVM
/dev/md_d5p4       488279496    73478448   488279460   8e  Linux LVM

Where is the inherent limit? The partitioning software, or partitioning
all by itself?

Regards,
Christian

Re: Help needed: Partitioned software raid > 2TB

[Andi Kleen]

Christian Schmidt <lkml@digadd.de> writes:
> 
> Where is the inherent limit? The partitioning software, or partitioning
> all by itself?

DOS style partitioning don't support more than 2TB. You either need
to use EFI partitions (e.g. using parted) or LVM. Since parted's
user interface is not good for much more than serving
as a bad example I would recommend LVM.

-Andi

Re: Help needed: Partitioned software raid > 2TB

[Christian Schmidt]

Hi Andi,

Andi Kleen wrote:
> Christian Schmidt <lkml@digadd.de> writes:
>> Where is the inherent limit? The partitioning software, or partitioning
>> all by itself?
> 
> DOS style partitioning don't support more than 2TB. You either need
> to use EFI partitions (e.g. using parted) or LVM. Since parted's
> user interface is not good for much more than serving
> as a bad example I would recommend LVM.

Thanks for the clarification. I didn't use LVM on the device on purpose,
as root on LVM requires initrd (which I strongly dislike as
yet-another-point-of-failure). As LVM is on the large partition anyway
I'll just add the second partition for now, and change the system setup
with the next drive migration. Maybe linux even supports root-on-lvm
natively until then ;)

Regards,
Christian

Re: Help needed: Partitioned software raid > 2TB

[Jan Engelhardt]

On Jun 15 2007 16:03, Christian Schmidt wrote:
>Hi Andi,
>
>Andi Kleen wrote:
>> Christian Schmidt <lkml@digadd.de> writes:
>>> Where is the inherent limit? The partitioning software, or partitioning
>>> all by itself?
>> 
>> DOS style partitioning don't support more than 2TB. You either need
>> to use EFI partitions (e.g. using parted) or LVM. Since parted's
>> user interface is not good for much more than serving
>> as a bad example I would recommend LVM.

parted's interface is no worse than that of [sfc]disk. Anyway...

>Thanks for the clarification. I didn't use LVM on the device on purpose,
>as root on LVM requires initrd (which I strongly dislike as
>yet-another-point-of-failure). As LVM is on the large partition anyway
>I'll just add the second partition for now, and change the system setup
>with the next drive migration. Maybe linux even supports root-on-lvm
>natively until then ;)

Uh, it does. By means of initrd/ramfs image. Blame your distro if it still
can't do root-on-LVM, there is at least one who can.


	Jan
-- 

Re: Help needed: Partitioned software raid > 2TB

[Alexander E. Patrakov]

Jan Engelhardt wrote:
> On Jun 15 2007 16:03, Christian Schmidt wrote:

>> Thanks for the clarification. I didn't use LVM on the device on purpose,
>> as root on LVM requires initrd (which I strongly dislike as
>> yet-another-point-of-failure). As LVM is on the large partition anyway
>> I'll just add the second partition for now, and change the system setup
>> with the next drive migration. Maybe linux even supports root-on-lvm
>> natively until then ;)
> 
> Uh, it does. By means of initrd/ramfs image. Blame your distro if it still
> can't do root-on-LVM, there is at least one who can.

AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself) requires 
LILO, doesn't it? Could you please list a few distributions with an easy 
method to install LILO (and, of course, root and /boot on LVM on whole disk) 
from their default installation media? So far, I only know that Debian can 
do it if you run debootstrap by hand, although they say that such setup is a 
bug (http://bugs.debian.org/401393).

-- 
Alexander E. Patrakov

Re: Help needed: Partitioned software raid > 2TB

[Jan Engelhardt]

On Jun 16 2007 11:38, Alexander E. Patrakov wrote:
> Jan Engelhardt wrote:
>> On Jun 15 2007 16:03, Christian Schmidt wrote:
>
>> > Thanks for the clarification. I didn't use LVM on the device on purpose,
>> > as root on LVM requires initrd (which I strongly dislike as
>> > yet-another-point-of-failure). As LVM is on the large partition anyway
>> > I'll just add the second partition for now, and change the system setup
>> > with the next drive migration. Maybe linux even supports root-on-lvm
>> > natively until then ;)
>> 
>> Uh, it does. By means of initrd/ramfs image. Blame your distro if it still
>> can't do root-on-LVM, there is at least one who can.
>
> AFAIK, root-on-LVM on the whole disk (BTW, I use such setup myself)
> requires LILO, doesn't it? Could you please list a few distributions
> with an easy method to install LILO (and, of course, root and /boot on
> LVM on whole disk) from their default installation media? So far, I only
> know that Debian can do it if you run debootstrap by hand, although they
> say that such setup is a bug (http://bugs.debian.org/401393).

I am not sure (would have to check again), but I believe both opensuse and
fedora (the latter of which uses LVM for all partitions by default) have
that working, while still using GRUB. But, what's much more amazing, is
that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps
it's just luck that the needed files are contiguous?

	Jan
-- 

Re: Help needed: Partitioned software raid > 2TB

[Alexander E. Patrakov]

Jan Engelhardt wrote:

> I am not sure (would have to check again), but I believe both opensuse and
> fedora (the latter of which uses LVM for all partitions by default) have
> that working, while still using GRUB.

Keyword: partitions. I.e., they partition the hard drive (so that the first 
31 sectors are available for GRUB) and use LVM on devices such as /dev/hda2. 
But this is not what was needed. I need to use LVM on /dev/hda, without a 
partition table.

> But, what's much more amazing, is
> that GRUB seems to work with raid0 (both BIOS-based and MD)... perhaps
> it's just luck that the needed files are contiguous?

[wild guess] Maybe they set up a raid1 /boot partition?

-- 
Alexander E. Patrakov