grub2 and hybrid MBR booting

grub2 and hybrid MBR booting

[Elliott, Robert (Server Storage)]

I successfully ran ubuntu 10.04 (which uses grub2) on a 4 TB drive 
(presented by a RAID controller) on a system with legacy BIOS.  The 
drive got partitioned as GPT; a BIOS boot partition was installed; 
and grub stage 1 contains the 8-byte LBA of the BIOS boot partition 
hardcoded into its 440 bytes of x86 boot code, so it can load the 
code in that BIOS boot partition.

That works pretty well, but requires any program that moves the BIOS 
boot partition to know that grub is pointing to it and also adjust 
that 8-byte LBA value stored in LBA 0.

In the UEFI Working Group (which defines GPT) and the T13 (ATA) standards 
bodies, we defined a slightly different method:  the GPT partition record 
now includes a Legacy BIOS Bootable bit that can be set for a partition 
like the BIOS boot partition, and the x86 boot code in LBA 0 can search 
through the GPT partition table for a partition with that bit set to 1 
(rather than contain a hardcoded LBA).  The algorithm is documented in 
T13 EDD-4 revision 2 and later (see
http://t13.org/Documents/MinutesDefault.aspx?DocumentType=4&DocumentStage=1).

Would grub2 consider switching to that modified algorithm?

H Peter Anvin has implemented it in syslinux-4.00-pre48 (see
http://syslinux.zytor.com/wiki/index.php/The_Syslinux_Project). The boot code 
doesn't have enough room to verify the GPT CRC and perform all the possible 
GPT verification checks, but it can definitely walk through the structure 
and find the boot partition.


I got some initial offline replies from Colin Watson, who suggested 
forwarding our thread to the list:

Colin:
> I'm just one GRUB 2 developer, but I'm certainly interested in things
> that improve standards-compliance and make things more reliable.  I like
> the look of this new scheme.

> Right now, the GRUB 2 boot code has no bytes free (well, it has eight
> bytes free upstream and no bytes free in Ubuntu, so I'm rather conscious
> of that).  I notice that syslinux uses a separate chunk of boot code for
> this new algorithm, so we could do that in GRUB 2 as well.

> Would you agree that the right way to decide whether to use the new boot
> code is to check whether the BIOS Boot Partition has the Legacy BIOS
> Bootable bit set?  I assume it was previously reserved and always 0.

RE: Correct.

> Do you know if anyone has contacted the GNU Parted developers about
> implementing support for this?  We'd need that in order to use it in
> Debian and Ubuntu, and I guess most other GNU/Linux distributions would
> need this too.  I can do this if nobody else has.

RE: H. Peter recently said he would bring it up with them.  

Colin:
> The specification doesn't seem to say anything about what happens in the
> event of multiple partition entries having the Legacy BIOS Bootable bit
> set.  syslinux simply bails out with the error "Multiple active
> partitions".  Is this as envisioned by the standards bodies?

RE: No, more than partition could have the bit set to one.  I 
envisioned the boot code starting with the first such partition, then 
looping back to the next if there is an error reading from the 
first partition.

---
Rob Elliott    HP ISS Platform Technology - Server Storage

Re: grub2 and hybrid MBR booting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2809 bytes --]

On 06/29/2010 08:37 PM, Elliott, Robert (Server Storage) wrote:
> I successfully ran ubuntu 10.04 (which uses grub2) on a 4 TB drive 
> (presented by a RAID controller) on a system with legacy BIOS.  The 
> drive got partitioned as GPT; a BIOS boot partition was installed; 
> and grub stage 1 contains the 8-byte LBA of the BIOS boot partition 
> hardcoded into its 440 bytes of x86 boot code, so it can load the 
> code in that BIOS boot partition.
>
> That works pretty well, but requires any program that moves the BIOS 
> boot partition to know that grub is pointing to it and also adjust 
> that 8-byte LBA value stored in LBA 0.
>
>   
After moving of partitions containing it's recommended to reinstall
bootloader
> In the UEFI Working Group (which defines GPT) and the T13 (ATA) standards 
> bodies, we defined a slightly different method:  the GPT partition record 
> now includes a Legacy BIOS Bootable bit that can be set for a partition 
> like the BIOS boot partition,   The algorithm is documented in 
> T13 EDD-4 revision 2 and later (see
> http://t13.org/Documents/MinutesDefault.aspx?DocumentType=4&DocumentStage=1).
>
>   
At last. It's refreshing from the usual lie that you need EFI to boot
from GPT-partitioned disk.
But I don't see the need to standartise the interface between MBR code
and the rest. Standartisation is good only for interoperability between
different software. But in this case both parts are from the same
bootloader so it will only reduce flexibility.

> and the x86 boot code in LBA 0 can search 
> through the GPT partition table for a partition with that bit set to 1 
> (rather than contain a hardcoded LBA).
It's closedly modeled after msdos. It annihilates most of advantages of GPT. I suppose you want to put filesystem on bootable partition too.
When we added GPT support virtually unlimited embedding zone was the great plus. Switching to msdos-like scheme would be a huge step backwards (especially that you have no MBR gap). It's a repetion of old mistakes under new sauce. MSDOS scheme already forced anti-patterns and any new scheme must be based on saner pattern. 

> Would grub2 consider switching to that modified algorithm?
>
> H Peter Anvin has implemented it in syslinux-4.00-pre48 (see
> http://syslinux.zytor.com/wiki/index.php/The_Syslinux_Project). 
>   
Syslinux is known for hanging on the old protocol no matter the price.
> The boot code 
> doesn't have enough room to verify the GPT CRC and perform all the possible 
> GPT verification checks, but it can definitely walk through the structure 
> and find the boot partition.
>
>   
Which contradicts your spec. Unimplementable spec only forces devs to
code undefined behaviour.

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

RE: grub2 and hybrid MBR booting

[Elliott, Robert (Server Storage)]

> -----Original Message-----
> From: grub-devel-bounces+elliott=hp.com@gnu.org [mailto:grub-devel-
> bounces+elliott=hp.com@gnu.org] On Behalf Of Vladimir 'f-coder/phcoder'
> Serbinenko
> Sent: Thursday, 01 July, 2010 2:43 PM
> To: grub-devel@gnu.org
> Subject: Re: grub2 and hybrid MBR booting
> 
> On 06/29/2010 08:37 PM, Elliott, Robert (Server Storage) wrote:
> > I successfully ran ubuntu 10.04 (which uses grub2) on a 4 TB drive
> > (presented by a RAID controller) on a system with legacy BIOS.  The
> > drive got partitioned as GPT; a BIOS boot partition was installed;
> > and grub stage 1 contains the 8-byte LBA of the BIOS boot partition
> > hardcoded into its 440 bytes of x86 boot code, so it can load the
> > code in that BIOS boot partition.
> >
> > That works pretty well, but requires any program that moves the BIOS
> > boot partition to know that grub is pointing to it and also adjust
> > that 8-byte LBA value stored in LBA 0.
> >
> >
> After moving of partitions containing it's recommended to reinstall
> bootloader

> > In the UEFI Working Group (which defines GPT) and the T13 (ATA)
> > standards bodies, we defined a slightly different method:  the GPT 
> > partition record now includes a Legacy BIOS Bootable bit that can be 
> > set for a partition like the BIOS boot partition,   The algorithm is 
> > documented in T13 EDD-4 revision 2 and later (see
> > http://t13.org/Documents/MinutesDefault.aspx?DocumentType=4&DocumentSta
> > ge=1).
> >
> >
> At last. It's refreshing from the usual lie that you need EFI to boot
> from GPT-partitioned disk.
> But I don't see the need to standartise the interface between MBR code
> and the rest. Standartisation is good only for interoperability between
> different software. But in this case both parts are from the same
> bootloader so it will only reduce flexibility.

The "handoff" contents are defined to better support MBRs and VBRs that
aren't tightly coupled.  The VBR can ignore the data structure if it 
doesn't need it. Syslinux felt it helpful to support legacy VBRs
(particularly when they are located < 2.2 TB).

> 
> > and the x86 boot code in LBA 0 can search through the GPT partition 
> > table for a partition with that bit set to 1 (rather than contain 
> > a hardcoded LBA).
> It's closedly modeled after msdos. It annihilates most of advantages of
> GPT. I suppose you want to put filesystem on bootable partition too.

The bootable partition can hold whatever it wants; the BIOS
Boot Partition contents would be fine as-is.

UEFI system boots require a FAT filesystem on the UEFI system
partition; this scheme has no such requirement.

> When we added GPT support virtually unlimited embedding zone was the
> great plus. Switching to msdos-like scheme would be a huge step
> backwards (especially that you have no MBR gap). It's a repetion of old
> mistakes under new sauce. MSDOS scheme already forced anti-patterns and
> any new scheme must be based on saner pattern.

What's an "anti-pattern"?

The change here would be that the grub MBR code would locate the
grub VBR code (the BIOS Boot Partition) by looking for the Legacy BIOS 
Bootable bit in the GPT partition record, rather than have hardcode
the LBA at some offset in the MBR.

Although all legacy BIOS boots will fail if LBA 0 goes bad, this allows
inclusion of more than one legacy bootable partition on the disk to 
tolerate a failure in those LBAs.  It would also tolerate failures of 
the GPT region (since there's a backup GPT at the end of the disk
that can be used).

> 
> > Would grub2 consider switching to that modified algorithm?
> >
> > H Peter Anvin has implemented it in syslinux-4.00-pre48 (see
> > http://syslinux.zytor.com/wiki/index.php/The_Syslinux_Project).
> >
> Syslinux is known for hanging on the old protocol no matter the price.

> > The boot code doesn't have enough room to verify the GPT CRC and 
> > perform all the possible GPT verification checks, but it can definitely 
> > walk through the structure and find the boot partition.
> >
> >
> Which contradicts your spec. Unimplementable spec only forces devs to
> code undefined behaviour.

The T13 spec includes a NOTE that the CRC check might not be possible.
A few simpler checks could be done: check the GPT signature ("EFI PART") 
or the My LBA field.  Those are a bit stronger than the 0x55AA magic
number relied on by traditional MBRs.

> 
> --
> Regards
> Vladimir 'φ-coder/phcoder' Serbinenko
> 

Re: grub2 and hybrid MBR booting

[Isaac Dupree]

On 07/01/10 15:42, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> After moving of partitions containing it's recommended to reinstall
> bootloader


> But I don't see the need to standartise the interface between MBR code
> and the rest. Standartisation is good only for interoperability between
> different software. But in this case both parts are from the same
> bootloader so it will only reduce flexibility.

Two pieces of software: bootloader (e.g. GRUB 2) and partition 
mover/rearranger (e.g. Parted).

I say it would be nice if on GPT with recommended GRUB install 
(including using BIOS Boot Partition), if we did not "[recommend] to 
reinstall bootloader" but instead promised that everything* would keep 
working without any bootloader reinstall.

*except if partition numbering is rearranged and grub.cfg uses partition 
numbers not UUIDs, then the grub.cfg may need editing, but this is no 
different than a Linux partition's /etc/fstab maybe needing editing.

Is it possible for us to promise that?--what are the disadvantages of 
doing so?

-Isaac

Re: grub2 and hybrid MBR booting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 3720 bytes --]

On 07/02/2010 02:13 AM, Elliott, Robert (Server Storage) wrote:
>>> In the UEFI Working Group (which defines GPT) and the T13 (ATA)
>>> standards bodies, we defined a slightly different method:  the GPT 
>>> partition record now includes a Legacy BIOS Bootable bit that can be 
>>> set for a partition like the BIOS boot partition,   The algorithm is 
>>> documented in T13 EDD-4 revision 2 and later (see
>>> http://t13.org/Documents/MinutesDefault.aspx?DocumentType=4&DocumentSta
>>> ge=1).
>>>
>>>
>>>       
>> At last. It's refreshing from the usual lie that you need EFI to boot
>> from GPT-partitioned disk.
>> But I don't see the need to standartise the interface between MBR code
>> and the rest. Standartisation is good only for interoperability between
>> different software. But in this case both parts are from the same
>> bootloader so it will only reduce flexibility.
>>     
> The "handoff" contents are defined to better support MBRs and VBRs that
> aren't tightly coupled.  
Why such schemes are of concern? Someone who wrote the whole bootloader
can write 440 bytes more and doesn't need another company for it. If
it's for loading one bootloader from another multiboot is much more
appropriate. GRUB tries to go away from chainloading as far as possible.
It's now still used only for windows but even this will change soon with
"ntldr" command which is able to load ntldr or bootmgr file.
> The VBR can ignore the data structure if it 
> doesn't need it. Syslinux felt it helpful to support legacy VBRs
> (particularly when they are located < 2.2 TB).
>
>   
Unless VBR itself is designed for this you can't load it this way.
>
>
>> When we added GPT support virtually unlimited embedding zone was the
>> great plus. Switching to msdos-like scheme would be a huge step
>> backwards (especially that you have no MBR gap). It's a repetion of old
>> mistakes under new sauce. MSDOS scheme already forced anti-patterns and
>> any new scheme must be based on saner pattern.
>>     
> What's an "anti-pattern"?
>
>   
A design decision to avoid.
> The change here would be that the grub MBR code would locate the
> grub VBR code (the BIOS Boot Partition) by looking for the Legacy BIOS 
> Bootable bit in the GPT partition record, rather than have hardcode
> the LBA at some offset in the MBR.
>
>   
This scheme already showed its flaws in the past. Multibooting by
changing this flag is impratical to say the least and over-reliance on
this flag to locate boot partition made it that bootloader must set this
flag before chainloading, so no bootloader other than microsoft one
implements msdos scheme. You would need a flag per bootloader. It boils
back to GUID.
Using GUID in GPT to locate the embedded zone would be good but it has
nothing to do with this standard. Robert made some work in this
direction but never finished (he wrote MBR but not some required changes
to diskboot.img) and nobody picked up his work.
> Although all legacy BIOS boots will fail if LBA 0 goes bad, this allows
> inclusion of more than one legacy bootable partition on the disk to 
> tolerate a failure in those LBAs.
Bad sectors are pretty rare and overwritten boot partition would need
manual clear of bootale flag which requires some environment which would
be able to just reinstall bootloader.
>   It would also tolerate failures of 
> the GPT region (since there's a backup GPT at the end of the disk
> that can be used).
>
>   
Whether or not use GPT to locate embed partition would be a separate
thread and as I said using GUID would be good but noone ever completed
it for GRUB.

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: grub2 and hybrid MBR booting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1811 bytes --]

On 07/03/2010 01:41 AM, Isaac Dupree wrote:
> On 07/01/10 15:42, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>> After moving of partitions containing it's recommended to reinstall
>> bootloader
>
>
>> But I don't see the need to standartise the interface between MBR code
>> and the rest. Standartisation is good only for interoperability between
>> different software. But in this case both parts are from the same
>> bootloader so it will only reduce flexibility.
>
> Two pieces of software: bootloader (e.g. GRUB 2) and partition
> mover/rearranger (e.g. Parted).
>
> I say it would be nice if on GPT with recommended GRUB install
> (including using BIOS Boot Partition), if we did not "[recommend] to
> reinstall bootloader" but instead promised that everything* would keep
> working without any bootloader reinstall.
>
> *except if partition numbering is rearranged and grub.cfg uses
> partition numbers not UUIDs, then the grub.cfg may need editing, but
> this is no different than a Linux partition's /etc/fstab maybe needing
> editing.
>
> Is it possible for us to promise that?--what are the disadvantages of
> doing so?
>
You answered your question yourself. Every way of locating breaks on
some changes. Which exactly depends on chosen way. While fixed LBA is
perhaps the worse method for locating a partition others offer
advantages only in face of certain kind of changes and not others. You
need to know the mechanism well in order to assess if reinstall is
needed after your changes. It's easier and safer to just always reinstall
> -Isaac
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: grub2 and hybrid MBR booting

[venkat kumar]

[-- Attachment #1: Type: text/plain, Size: 1533 bytes --]

sir,
    i am new to grub and other provisions supported by linux .kindly explain
what is grub,boot partition,mbr code, rtl ,etc.......
thnkyou


On Sat, Jul 3, 2010 at 5:26 AM, Isaac Dupree <ml@isaac.cedarswampstudios.org
> wrote:

> On 07/01/10 15:42, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>
>> After moving of partitions containing it's recommended to reinstall
>> bootloader
>>
>
>
>  But I don't see the need to standartise the interface between MBR code
>> and the rest. Standartisation is good only for interoperability between
>> different software. But in this case both parts are from the same
>> bootloader so it will only reduce flexibility.
>>
>
> Two pieces of software: bootloader (e.g. GRUB 2) and partition
> mover/rearranger (e.g. Parted).
>
> I say it would be nice if on GPT with recommended GRUB install (including
> using BIOS Boot Partition), if we did not "[recommend] to reinstall
> bootloader" but instead promised that everything* would keep working without
> any bootloader reinstall.
>
> *except if partition numbering is rearranged and grub.cfg uses partition
> numbers not UUIDs, then the grub.cfg may need editing, but this is no
> different than a Linux partition's /etc/fstab maybe needing editing.
>
> Is it possible for us to promise that?--what are the disadvantages of doing
> so?
>
> -Isaac
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>

[-- Attachment #2: Type: text/html, Size: 2276 bytes --]

Re: grub2 and hybrid MBR booting

[Robert Millan]

2010/7/3, Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com>:
> Using GUID in GPT to locate the embedded zone would be good but it has
> nothing to do with this standard. Robert made some work in this
> direction but never finished (he wrote MBR but not some required changes
> to diskboot.img) and nobody picked up his work.

If you refer to my "GPT MBR", it was mostly a proof of concept.  I
didn't really have plans for it.  I tend to think setting things up at
install time is more suitable than doing a search every time (and 440
bytes don't give a lot of flexibility...).

Also, HPA persuaded me that since he was going to work on standarising
his handover protocol, and my "GPT MBR" experiment overlapped with it,
I would obselete it to avoid conflict.