grub2/grub-pc install not possible on mdadm

grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

Hi list,

i've configured a raid1 and i'm trying to install grub on it. But it
doesn't work.

~# grub-mkdevicemap -n
~# cat /boot/grub/device.map
(hd0)   /dev/disk/by-id/ata-INTEL_SSDSA2CW160G3_CVPR14530C6J160DGN
(hd1)   /dev/disk/by-id/ata-INTEL_SSDSA2CW160G3_CVPR145101F9160DGN

~# grub-install /dev/md127
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
/usr/sbin/grub-install: line 661: 20158 Segmentation fault
"$grub_setup" ${allow_floppy} ${setup_verbose} ${setup_force}
--directory="${grubdir}" --device-map="${device_map}" "${install_device}"

When i then add /dev/md127 to my device.map i got this:

~# grub-install /dev/md127
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md/raidone.
error: superfluous RAID member (2 found).
Installation finished. No error reported.

But the device still does not boot. Installing grub to sda / sdb does
not work either as they do not contain any partition table. What's wrong
here?

Greets
Stefan

Raid details:
~# parted
Model: Linux Software RAID Array (md)
Disk /dev/md127: 160GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system     Name       Flags
 1      1049kB  2097kB  1049kB                  GRUB_BOOT  bios_grub
 2      2097kB  99,6MB  97,5MB  ext2            BOOT
 3      99,6MB  157GB   157GB   xfs             ROOT
 4      157GB   160GB   3056MB  linux-swap(v1)  SWAP

~# mdadm -E /dev/md127
/dev/md127:
   MBR Magic : aa55
Partition[0] :    312579615 sectors at            1 (type ee)

~# mdadm -D /dev/md127
/dev/md127:
        Version : 1.2
  Creation Time : Tue Feb 28 09:49:58 2012
     Raid Level : raid1
     Array Size : 156289808 (149.05 GiB 160.04 GB)
  Used Dev Size : 156289808 (149.05 GiB 160.04 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Tue Feb 28 10:01:11 2012
          State : active, resyncing
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

 Rebuild Status : 56% complete

           Name : systembootimage:raidone  (local to host systembootimage)
           UUID : b018eac7:1a44871a:c37f9dca:80e55a6e
         Events : 23

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb

Re: grub2/grub-pc install not possible on mdadm

[Jes Sorensen]

On 02/28/12 10:10, Stefan Priebe - Profihost AG wrote:
> Hi list,
> 
> i've configured a raid1 and i'm trying to install grub on it. But it
> doesn't work.
> 
> ~# grub-mkdevicemap -n
> ~# cat /boot/grub/device.map
> (hd0)   /dev/disk/by-id/ata-INTEL_SSDSA2CW160G3_CVPR14530C6J160DGN
> (hd1)   /dev/disk/by-id/ata-INTEL_SSDSA2CW160G3_CVPR145101F9160DGN
> 
> ~# grub-install /dev/md127
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> /usr/sbin/grub-install: line 661: 20158 Segmentation fault
> "$grub_setup" ${allow_floppy} ${setup_verbose} ${setup_force}
> --directory="${grubdir}" --device-map="${device_map}" "${install_device}"
> 
> When i then add /dev/md127 to my device.map i got this:
> 
> ~# grub-install /dev/md127
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> error: found two disks with the index 0 for RAID md/raidone.
> error: superfluous RAID member (2 found).
> Installation finished. No error reported.
> 
> But the device still does not boot. Installing grub to sda / sdb does
> not work either as they do not contain any partition table. What's wrong
> here?
> 
> Greets
> Stefan
> 
> Raid details:
> ~# parted
> Model: Linux Software RAID Array (md)
> Disk /dev/md127: 160GB
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
> 
> Number  Start   End     Size    File system     Name       Flags
>  1      1049kB  2097kB  1049kB                  GRUB_BOOT  bios_grub
>  2      2097kB  99,6MB  97,5MB  ext2            BOOT
>  3      99,6MB  157GB   157GB   xfs             ROOT
>  4      157GB   160GB   3056MB  linux-swap(v1)  SWAP

I am not sure whether this is the reason, but you have disks with gpt
partition tables, while it looks like you are still using grub1? I
didn't think grub1 was able to handle gpt.

Cheers,
Jes

Re: grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

> I am not sure whether this is the reason, but you have disks with gpt
> partition tables, while it looks like you are still using grub1? I
> didn't think grub1 was able to handle gpt.
~# grub-setup -V
grub-setup (GRUB) 1.99-14

Stefan

Re: grub2/grub-pc install not possible on mdadm

[Jes Sorensen]

On 03/01/12 11:25, Stefan Priebe - Profihost AG wrote:
>> > I am not sure whether this is the reason, but you have disks with gpt
>> > partition tables, while it looks like you are still using grub1? I
>> > didn't think grub1 was able to handle gpt.
> ~# grub-setup -V
> grub-setup (GRUB) 1.99-14

Not grub1 then .....

Looking at your original mail again - you are running a raid with
metadata 1.2, which means your BIOS will not know how to read the disks
as a raid device.

You should be able to install grub into the MBR of /dev/sda or /dev/sdb.
I believe that is possible with 1.2 metadata. 1.1 was totally broken for
this if I remember correctly.

That said, if your system is EFI based, I suspect you will need to have
the bios_grub partition on the physical drives so the BIOS can read it.
It's not going to do much good having it on the raid partition if the
BIOS isn't able to read those.

BIOS and partition tables isn't exactly my area of expertise, so I could
be wrong on some of this.

Cheers,
Jes

Re: grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

> Looking at your original mail again - you are running a raid with
> metadata 1.2, which means your BIOS will not know how to read the disks
> as a raid device.
> You should be able to install grub into the MBR of /dev/sda or /dev/sdb.
> I believe that is possible with 1.2 metadata. 1.1 was totally broken for
> this if I remember correctly.

Correct that's why i've also tried to install it on sda / sdb but then
grub complains about not finding any partition table.

> That said, if your system is EFI based, I suspect you will need to have
> the bios_grub partition on the physical drives so the BIOS can read it.
> It's not going to do much good having it on the raid partition if the
> BIOS isn't able to read those.
The bios is not the problem - grub simply does not want to install it
boot sector to sda / sdb.

It does it fine when using metadata 0.9.

Stefan

Re: grub2/grub-pc install not possible on mdadm

[Lars Täuber]

Hi Stefan,

Am Fri, 02 Mar 2012 09:44:20 +0100
Stefan Priebe - Profihost AG <s.priebe@profihost.ag> schrieb:
> 
> > Looking at your original mail again - you are running a raid with
> > metadata 1.2, which means your BIOS will not know how to read the disks
> > as a raid device.
> > You should be able to install grub into the MBR of /dev/sda or /dev/sdb.
> > I believe that is possible with 1.2 metadata. 1.1 was totally broken for
> > this if I remember correctly.
> 
> Correct that's why i've also tried to install it on sda / sdb but then
> grub complains about not finding any partition table.

do you have a partition table inside the raid?
This will not work I expect. The kernel might find the partition table but
grub?

Or do you have no partition table at all and a filesystem directly on the
raid device?

Lars
-- 
                            Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstrasse 22-23                     10117 Berlin
Tel.: +49 30 20370-352           http://www.bbaw.de
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

Am 02.03.2012 10:06, schrieb Lars Täuber:
>> Correct that's why i've also tried to install it on sda / sdb but then
>> grub complains about not finding any partition table.
> 
> do you have a partition table inside the raid?
> This will not work I expect. The kernel might find the partition table but
> grub?
> 
> Or do you have no partition table at all and a filesystem directly on the
> raid device?

No partition table on sda / sdb. Partition table is on md127. This
scenario works fine with metadata 0.9. And as far as i understand it
should work with 1.2 too.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[John Robinson]

On 02/03/2012 09:11, Stefan Priebe - Profihost AG wrote:
> Am 02.03.2012 10:06, schrieb Lars Täuber:
>>> Correct that's why i've also tried to install it on sda / sdb but then
>>> grub complains about not finding any partition table.
>>
>> do you have a partition table inside the raid?
>> This will not work I expect. The kernel might find the partition table but
>> grub?
>>
>> Or do you have no partition table at all and a filesystem directly on the
>> raid device?
>
> No partition table on sda / sdb. Partition table is on md127. This
> scenario works fine with metadata 0.9. And as far as i understand it
> should work with 1.2 too.

No, I don't think it should - 1.2's metadata is offset 4K into the 
component devices and the data starts later, at 1M or so. That means 
that the BIOS can't find the partition table where it expects to. Using 
1.0, which has its data right at the beginning and metadata at the end, 
might work if 0.90 does. It's still not 100% because a UEFI BIOS isn't 
going to find the backup GPT partition table where it ought to.

Unless you are using IMSM or another BIOS-readable RAID, you ought to be 
creating partition tables on all your drives, installing GRUB into the 
bios-grub partitions, and building arrays from the rest of the 
partitions (typically a RAID-1 /boot and any level you like for the 
rest; I use RAID-10 for swap and on two drives RAID-10,f2 for the root).

Cheers,

John.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

Am 02.03.2012 10:24, schrieb John Robinson:
> On 02/03/2012 09:11, Stefan Priebe - Profihost AG wrote:
>> Am 02.03.2012 10:06, schrieb Lars Täuber:
>>>> Correct that's why i've also tried to install it on sda / sdb but then
>>>> grub complains about not finding any partition table.
>>>
>>> do you have a partition table inside the raid?
>>> This will not work I expect. The kernel might find the partition
>>> table but
>>> grub?
>>>
>>> Or do you have no partition table at all and a filesystem directly on
>>> the
>>> raid device?
>>
>> No partition table on sda / sdb. Partition table is on md127. This
>> scenario works fine with metadata 0.9. And as far as i understand it
>> should work with 1.2 too.
> 
> No, I don't think it should - 1.2's metadata is offset 4K into the
> component devices and the data starts later, at 1M or so. That means
> that the BIOS can't find the partition table where it expects to. Using
> 1.0, which has its data right at the beginning and metadata at the end,
> might work if 0.90 does. It's still not 100% because a UEFI BIOS isn't
> going to find the backup GPT partition table where it ought to.

Maybe i'm wrong here. But shouldn't this get handled by the raid1x
module of grub? So grub installs at the bootsector of sda / sdb and then
reads the correct data? Like it does with 0.9 metadata?

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[John Robinson]

On 02/03/2012 09:33, Stefan Priebe - Profihost AG wrote:
> Am 02.03.2012 10:24, schrieb John Robinson:
>> On 02/03/2012 09:11, Stefan Priebe - Profihost AG wrote:
[...]
>>> No partition table on sda / sdb. Partition table is on md127. This
>>> scenario works fine with metadata 0.9. And as far as i understand it
>>> should work with 1.2 too.
>>
>> No, I don't think it should - 1.2's metadata is offset 4K into the
>> component devices and the data starts later, at 1M or so. That means
>> that the BIOS can't find the partition table where it expects to. Using
>> 1.0, which has its data right at the beginning and metadata at the end,
>> might work if 0.90 does. It's still not 100% because a UEFI BIOS isn't
>> going to find the backup GPT partition table where it ought to.
>
> Maybe i'm wrong here. But shouldn't this get handled by the raid1x
> module of grub? So grub installs at the bootsector of sda / sdb and then
> reads the correct data? Like it does with 0.9 metadata?

No. When booting with MBR, all that goes in the boot sector is enough to 
load the next stage. That next stage will include the raid1x module. It 
lives in the gap between the MBR and the first partition - with old 
fdisk, that was 31K, with more recent fdisk it's 1MB. With no partition 
table on the drives and 1.2 metadata, there's only 3.5K for it, which 
isn't enough.

When booting with UEFI, that next stage lives in the BIOS boot 
partition; the BIOS ignores the MBR, reads the GPT partition table, 
finds the BIOS boot partition, and runs what's there.

What's actually happening with metadata 0.90 or 1.0, because they have 
their data from the beginning, is that the partition table, GRUB etc 
that you have installed into md127 appears to the BIOS as two drives 
with boot sectors, partition tables, etc, and it then proceeds to boot 
off the first drive. Until grub has loaded, you rely on the fact that 
what you installed inside the RAID-1 is laid out in exactly the way the 
BIOS expects to find it on a single drive.

What you've done with metadata 1.2 is put GRUB in a place that needs 
GRUB to find it.

Cheers,

John.

Re: grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

> No. When booting with MBR, all that goes in the boot sector is enough to
> load the next stage. That next stage will include the raid1x module. It
> lives in the gap between the MBR and the first partition - with old
> fdisk, that was 31K, with more recent fdisk it's 1MB. With no partition
> table on the drives and 1.2 metadata, there's only 3.5K for it, which
> isn't enough.
But that's why i've a BIOS_GRUB partition on the raid and so also on the
disk. Shouldn't it be the same size for 0.9 which starts at the
beginning, too?

> What's actually happening with metadata 0.90 or 1.0, because they have
> their data from the beginning, is that the partition table, GRUB etc
> that you have installed into md127 appears to the BIOS as two drives
> with boot sectors, partition tables, etc, and it then proceeds to boot
> off the first drive.
So the bios needs to read the partition table? I thought the MBR would
be enough.

> Until grub has loaded, you rely on the fact that
> what you installed inside the RAID-1 is laid out in exactly the way the
> BIOS expects to find it on a single drive.
>
> What you've done with metadata 1.2 is put GRUB in a place that needs
> GRUB to find it.
With metadata 0.9 i can install grub on sda and sdb even when there's no
part. table.

But with 1.2 grub says it cannot find a partition table so it cannot
install . So it seems to be a grub bug? grub-probe correctly dectecs a
1.2 metadata md raid on the disks but then still doesn't want to install
with the no partition table message.

Stefan

Re: grub2/grub-pc install not possible on mdadm

[John Robinson]

On 02/03/2012 13:15, Stefan Priebe - Profihost AG wrote:
>> No. When booting with MBR, all that goes in the boot sector is enough to
>> load the next stage. That next stage will include the raid1x module. It
>> lives in the gap between the MBR and the first partition - with old
>> fdisk, that was 31K, with more recent fdisk it's 1MB. With no partition
>> table on the drives and 1.2 metadata, there's only 3.5K for it, which
>> isn't enough.
> But that's why i've a BIOS_GRUB partition on the raid and so also on the
> disk.

No, you don't have a BIOS_GRUB partition also on the disk. When you 
partition an array, the partition table goes inside the data area of the 
array.

> Shouldn't it be the same size for 0.9 which starts at the
> beginning, too?

With 0.9 or 1.0, because the data area is at the start of the array, and 
therefore at the start of the disc when you create an array over the 
whole disc, all the boot stuff is visible to the BIOS because it's in 
the same place as it would be on a bare drive.

>> What's actually happening with metadata 0.90 or 1.0, because they have
>> their data from the beginning, is that the partition table, GRUB etc
>> that you have installed into md127 appears to the BIOS as two drives
>> with boot sectors, partition tables, etc, and it then proceeds to boot
>> off the first drive.
> So the bios needs to read the partition table? I thought the MBR would
> be enough.

If you're booting with MBR (traditional BIOS), the MBR plus the gap 
before the first partition is enough.

If you're booting with UEFI (modern BIOS), the MBR is totally irrelevant 
and the BIOS has to read the GPT partition table to find the BIOS_GRUB 
partition.

If we're going to go any further than this, please can you indicate 
which you are using?

>
>> Until grub has loaded, you rely on the fact that
>> what you installed inside the RAID-1 is laid out in exactly the way the
>> BIOS expects to find it on a single drive.
>>
>> What you've done with metadata 1.2 is put GRUB in a place that needs
>> GRUB to find it.
> With metadata 0.9 i can install grub on sda and sdb even when there's no
> part. table.

No, you can't; installing on sda and sdb overwrites the beginning of the 
array, and vice versa; you aren't installing it twice. grub-probe or 
grub-install understands this, which is why it works.

> But with 1.2 grub says it cannot find a partition table so it cannot
> install . So it seems to be a grub bug? grub-probe correctly dectecs a
> 1.2 metadata md raid on the disks but then still doesn't want to install
> with the no partition table message.

It's not a grub bug. It's not a bug at all. When you create a 1.2 array, 
the data area is 1MB into the component devices, and when you partition 
the array, that's where its partition table goes. grub-probe looks for 
what the BIOS will see, and the BIOS will not see the partition table 
because it's not at the start of the component devices, it's 1MB in.

One way to make this work is: partition your discs (either MBR with a 
data partition starting 1M into the disc and spanning the rest of the 
disc, or UEFI with a BIOS_GRUB partition from as early as it'll let you 
up to 1M and a data partition from 1M and spanning the rest of the 
disc). You will need to do this for all your discs separately. Then 
create an array with any metadata and of any type supported by grub on 
the data partitions. The array need not be partitioned, but you want 
separate /boot, swap and root, so partition it. Then install grub and it 
will work it out.

Alternatively, partition all your discs as you previously partitioned 
your array. Create 3 arrays of any metadata and type, the first being 
for /boot, which must be of a metadata and type supported by grub. 
Install grub and it will work it out.

Cheers,

John.

Re: grub2/grub-pc install not possible on mdadm

[Lars Täuber]

Hi Stefan,

the metadata versions 1.0 and 1.1 and 1.2 are the same but for the position of
the metadata on disk. Simply use metadata version 1.0 and everything works as
with 0.9.

Grub does to read partition tables inside raid devices.
Because a raid1 with metadata 1.0 or 0.9 looks like a plain disk to grub it
can read a partition table like there was no raid array.

Simply give it a try.

Lars

Re: grub2/grub-pc install not possible on mdadm

[Stefan Priebe - Profihost AG]

Am 02.03.2012 16:32, schrieb Lars Täuber:
> Hi Stefan,
> 
> the metadata versions 1.0 and 1.1 and 1.2 are the same but for the position of
> the metadata on disk. Simply use metadata version 1.0 and everything works as
> with 0.9.
> 
> Grub does to read partition tables inside raid devices.
> Because a raid1 with metadata 1.0 or 0.9 looks like a plain disk to grub it
> can read a partition table like there was no raid array.
Ah OK thanks. using Metadata 1.0 works fine.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[John Robinson]

On 05/03/2012 07:50, Stefan Priebe - Profihost AG wrote:
> Am 02.03.2012 16:32, schrieb Lars Täuber:
>> Hi Stefan,
>>
>> the metadata versions 1.0 and 1.1 and 1.2 are the same but for the position of
>> the metadata on disk. Simply use metadata version 1.0 and everything works as
>> with 0.9.
>>
>> Grub does to read partition tables inside raid devices.

Sure it does, but that's not the point.

>> Because a raid1 with metadata 1.0 or 0.9 looks like a plain disk to grub it
>> can read a partition table like there was no raid array.

For the fourth time, no.

Because a RAID1 with metadata 0.9 or 1.0 looks like a plain disc to the 
BIOS, it can read a partition table like there was no RAID array, and 
therefore the BIOS can load GRUB.

> Ah OK thanks. using Metadata 1.0 works fine.

It will work OK, but just be very, very careful never to run a GPT 
partition tool directly on /dev/sda or /dev/sdb, because it will write 
the backup of the GPT partition table at the end of the disc, 
potentially destroying the metadata for the array.

You would still be safer having partition tables on the drives, and 
building your array(s) from those partition(s), the opposite way round 
from how you are doing it just now.

Cheers,

John.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[Lars Täuber]

Hi John,

Am Mon, 05 Mar 2012 09:38:34 +0000
John Robinson <john.robinson@anonymous.org.uk> schrieb:
> On 05/03/2012 07:50, Stefan Priebe - Profihost AG wrote:
> > Am 02.03.2012 16:32, schrieb Lars Täuber:
> >> Hi Stefan,
> >>
> >> the metadata versions 1.0 and 1.1 and 1.2 are the same but for the
> >> position of the metadata on disk. Simply use metadata version 1.0 and
> >> everything works as with 0.9.
> >>
> >> Grub does to read partition tables inside raid devices.
> 
> Sure it does, but that's not the point.

that's new to me. This is true for grub2 only?
You mean if Stefan would have used an old fashioned partition table inside a
Linux SW-RAID with metadata 1.2, grub would have booted like normal?

 
> >> Because a raid1 with metadata 1.0 or 0.9 looks like a plain disk to grub
> >> it can read a partition table like there was no raid array.
> 
> For the fourth time, no.
>
> Because a RAID1 with metadata 0.9 or 1.0 looks like a plain disc to the 
> BIOS, it can read a partition table like there was no RAID array, and 
> therefore the BIOS can load GRUB.

This might be true for UEFI firmware but for the plain old BIOS too?
I thought the BIOS isn't interested in partitions at all.

Lars
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[John Robinson]

On 05/03/2012 10:14, Lars Täuber wrote:
> Hi John,
>
> Am Mon, 05 Mar 2012 09:38:34 +0000
> John Robinson<john.robinson@anonymous.org.uk>  schrieb:
>> On 05/03/2012 07:50, Stefan Priebe - Profihost AG wrote:
>>> Am 02.03.2012 16:32, schrieb Lars Täuber:
>>>> Hi Stefan,
>>>>
>>>> the metadata versions 1.0 and 1.1 and 1.2 are the same but for the
>>>> position of the metadata on disk. Simply use metadata version 1.0 and
>>>> everything works as with 0.9.
>>>>
>>>> Grub does to read partition tables inside raid devices.
>>
>> Sure it does, but that's not the point.
>
> that's new to me. This is true for grub2 only?
> You mean if Stefan would have used an old fashioned partition table inside a
> Linux SW-RAID with metadata 1.2, grub would have booted like normal?

Yes, both grub2 and grub legacy (0.97) read partition tables. Only grub2 
can really read inside RAID arrays - any metadata, any RAID level - 
while grub legacy can only read the first drive of RAID-1 with 0.90 or 
1.0 metadata, and that only because they look like bare drives/partitions.

>>>> Because a raid1 with metadata 1.0 or 0.9 looks like a plain disk to grub
>>>> it can read a partition table like there was no raid array.
>>
>> For the fourth time, no.
>>
>> Because a RAID1 with metadata 0.9 or 1.0 looks like a plain disc to the
>> BIOS, it can read a partition table like there was no RAID array, and
>> therefore the BIOS can load GRUB.
>
> This might be true for UEFI firmware but for the plain old BIOS too?
> I thought the BIOS isn't interested in partitions at all.

That's true, but it is interested in the MBR, sector zero of the drive 
it's booting from, which is also where the partition table lives, so for 
plain old BIOS you have to put the stage 1 boot loader there, which can 
only happen with metadata 0.90 and 1.0 if you use whole-drive arrays.

With plain old BIOS, grub legacy and grub2 also depend on the next 31K 
of the disc, up to sector 62, being unused, as traditionally the first 
partition starts at sector 63. (More recent fdisk make it 1023.5K, 
sector 2047 and sector 2048 respectively). This means you have to have a 
partition table, and though the plain old BIOS doesn't use it, grub 
does, to find the /boot or / partition to load a kernel from. With grub 
legacy, it has to be an MBR partition table, not one buried inside an 
array with metadata 1.1 or 1.2.

I suppose you could possibly contrive to have no MBR partition table for 
grub2, but you'd still need that space on the drive for the stage 1.5 
code, which isn't available with a whole-drive metadata 1.2 array 
(because its metadata starts 4K into the drive, leaving 3.5K after the 
stage 1 on the MBR), so you do effectively need an MBR partition table 
to reserve the space.

But you're right, plain old BIOS don't use the MBR partition table, just 
the MBR. I was fashioning my sentence after yours, so sorry for any 
confusion. My point - that it's the BIOS that has to be able to read the 
array like it was a bare drive, rather than grub - remains.

Cheers,

John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[Lars Täuber]

Am Mon, 05 Mar 2012 11:31:54 +0000
John Robinson <john.robinson@anonymous.org.uk> schrieb:
> > You mean if Stefan would have used an old fashioned partition table
> > inside a Linux SW-RAID with metadata 1.2, grub would have booted like
> > normal?
> 
> Yes, both grub2 and grub legacy (0.97) read partition tables. Only grub2 
> can really read inside RAID arrays - any metadata, any RAID level - [...]

Does grub2 also read partitions inside SW-RAID if the RAID consists of
partitions?
How deep works this mechanism?

DISK => partitions => RAID => partition table => RAID => .... 

Lars

Re: grub2/grub-pc install not possible on mdadm

[John Robinson]

On 05/03/2012 11:38, Lars Täuber wrote:
> Am Mon, 05 Mar 2012 11:31:54 +0000
> John Robinson<john.robinson@anonymous.org.uk>  schrieb:
>>> You mean if Stefan would have used an old fashioned partition table
>>> inside a Linux SW-RAID with metadata 1.2, grub would have booted like
>>> normal?
>>
>> Yes, both grub2 and grub legacy (0.97) read partition tables. Only grub2
>> can really read inside RAID arrays - any metadata, any RAID level - [...]
>
> Does grub2 also read partitions inside SW-RAID if the RAID consists of
> partitions?
> How deep works this mechanism?
>
> DISK =>  partitions =>  RAID =>  partition table =>  RAID =>  ....

I don't know. I'm sure DISK => partitions => RAID => partitions will 
work - that's what I was suggesting Stefan should do to use 1.2 
metadata. You can have LVM in the mix too. Oh and don't forget the 
filesystem right at the end. It may be possible to get it to support 
multi-level RAID but I haven't tried - I try to keep it simple with DISK 
=> partition => RAID-1 => ext2; it's enough bother getting all the other 
mess taken care of in the initrd and Linux, without doing it in GRUB too!

Cheers,

John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[Lars Täuber]

Am Fri, 2 Mar 2012 10:06:48 +0100
Lars Täuber <taeuber@bbaw.de> schrieb:
> Hi Stefan,
> 
> Am Fri, 02 Mar 2012 09:44:20 +0100
> Stefan Priebe - Profihost AG <s.priebe@profihost.ag> schrieb:
> > 
> > > Looking at your original mail again - you are running a raid with
> > > metadata 1.2, which means your BIOS will not know how to read the disks
> > > as a raid device.
> > > You should be able to install grub into the MBR of /dev/sda or /dev/sdb.
> > > I believe that is possible with 1.2 metadata. 1.1 was totally broken for
> > > this if I remember correctly.
> > 
> > Correct that's why i've also tried to install it on sda / sdb but then
> > grub complains about not finding any partition table.
> 
> do you have a partition table inside the raid?
> This will not work I expect. The kernel might find the partition table but
> grub?
> 
> Or do you have no partition table at all and a filesystem directly on the
> raid device?
> 
> Lars
> -- 
>                             Informationstechnologie
> Berlin-Brandenburgische Akademie der Wissenschaften
> Jägerstrasse 22-23                     10117 Berlin
> Tel.: +49 30 20370-352           http://www.bbaw.de
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
                            Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstrasse 22-23                     10117 Berlin
Tel.: +49 30 20370-352           http://www.bbaw.de
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub2/grub-pc install not possible on mdadm

[Lars Täuber]

Hi again,

I just read a previous post.
You use a GPT inside the RAID-device. But grub looks for partition tables at
a place where is none, because the partition table is inside a RAID-device.

You should use metadata version 1.0 instead of 1.2.
This metadata is place at the end of the devices like 0.90 was.
Then the partition table is at the same position as if the underlying device
was not part of a raid.

I always use metadata 1.0 for all RAID1 device.
The advantage is you always can use it like a normal disk not beeing a part
of a raid.
Also the grub doesn't need to know that it is part of a RAID.


Good luck
Lars

PS: Sorry for the empty post.

Re: grub2/grub-pc install not possible on mdadm

[Jes Sorensen]

On 03/02/12 09:44, Stefan Priebe - Profihost AG wrote:
> 
>> Looking at your original mail again - you are running a raid with
>> metadata 1.2, which means your BIOS will not know how to read the disks
>> as a raid device.
>> You should be able to install grub into the MBR of /dev/sda or /dev/sdb.
>> I believe that is possible with 1.2 metadata. 1.1 was totally broken for
>> this if I remember correctly.
> 
> Correct that's why i've also tried to install it on sda / sdb but then
> grub complains about not finding any partition table.

Well there is no point in trying to install it onto the actual RAID
since that is guaranteed not to work. Why it fails on sda/sdb like that
I cannot explain.

>> That said, if your system is EFI based, I suspect you will need to have
>> the bios_grub partition on the physical drives so the BIOS can read it.
>> It's not going to do much good having it on the raid partition if the
>> BIOS isn't able to read those.
> The bios is not the problem - grub simply does not want to install it
> boot sector to sda / sdb.
> 
> It does it fine when using metadata 0.9.

I cannot say why it fails like this, maybe someone else knows more but
Neil did reply to you earlier with a summary on metadata and boot loaders:

Quote:

So 1.0 is good for boot loaders
   1.1 is bad for boot loaders
   1.2 is OK for boot loaders and good for other reasons (resizeable
devices).

Alternatively, why not just create a partition table on the drivers,
reserve a couple of MBs for the boot loader information and the problems
should go away?

Jes