[RFC] Multiboot2 drafting

[RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2767 bytes --]

Hello, all. In GRUB2 community we're currently working on a
next-generation multiboot specification. It has goals similar to the
original multiboot specification but with important flaws fixed:
1) Instead of having bunch of pointers to subtables it uses a tagged
structure now. It allows it to be easier expandable, consume less space
if some fields aren't used and is easier to relocate.
2) Now it's portable. It has fields for specification of architecture
and proper field alignment. So now adding a new arcitecture is as easy
as adding a new arcitecture id and a section dealing with
architecture-specific problems like register usage to pass information.
Currently it has only i386 and mips though. Both are implemented in bzr
trunk grub[1]
Short summary:
Multiboot specification is done as an unified protocol between
bootloader and kernel to allow supporting more kernels and bootloaders
with less effort. The easiest way to use it is to create an ELF image
with an additional header which announces multiboot support and specific
kernel requirements. But it's also possible to put loading address in
multiboot header and use a non-ELF format.
When image is loaded one of GPRs contaians a magic number to identify
that kernel was loaded as multiboot and another GPR contains a pointer
to information table. The most needed information is already there, more
is in review progress, and if you feel like we're missing somethin feel
free to contact.
Complete specification is available at
http://bzr.savannah.gnu.org/r/grub/branches/multiboot2/ or at
http://download.savannah.gnu.org/releases-noredirect/grub/phcoder/multiboot.pdf

Some advantages:
1) Removes the need to support a fundamentaly different boot protocol
for every architecture. In future it may even be possible to have just
one image per ISA and rely on information supplied by bootloader to
differentiate between platforms (it doesn't prevent creation of an image
optimised for one platform)
2) Better collaboration between bootloaders and kernels by avoiding to
implement "one protocol per kernel".
3) Possibility to inform bootloader of exact kernel requirements and
supported features.
4) Uniform interface for retrieving e.g. memory map and initial console
info across platforms.
<Anything else you request ;)>

Any comment about multiboot2 spec is welcome.

Would it be possible for Linux in perspective to use multiboot2 instead
or in addition to current protocols?

Thanks for your time.

[1] Available with
bzr co --lightweight http://bzr.savannah.gnu.org/r/grub/trunk/grub/
or as a tarball at
http://download.savannah.gnu.org/releases-noredirect/grub/phcoder/grub-r2283.tgz

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

[RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2767 bytes --]

Hello, all. In GRUB2 community we're currently working on a
next-generation multiboot specification. It has goals similar to the
original multiboot specification but with important flaws fixed:
1) Instead of having bunch of pointers to subtables it uses a tagged
structure now. It allows it to be easier expandable, consume less space
if some fields aren't used and is easier to relocate.
2) Now it's portable. It has fields for specification of architecture
and proper field alignment. So now adding a new arcitecture is as easy
as adding a new arcitecture id and a section dealing with
architecture-specific problems like register usage to pass information.
Currently it has only i386 and mips though. Both are implemented in bzr
trunk grub[1]
Short summary:
Multiboot specification is done as an unified protocol between
bootloader and kernel to allow supporting more kernels and bootloaders
with less effort. The easiest way to use it is to create an ELF image
with an additional header which announces multiboot support and specific
kernel requirements. But it's also possible to put loading address in
multiboot header and use a non-ELF format.
When image is loaded one of GPRs contaians a magic number to identify
that kernel was loaded as multiboot and another GPR contains a pointer
to information table. The most needed information is already there, more
is in review progress, and if you feel like we're missing somethin feel
free to contact.
Complete specification is available at
http://bzr.savannah.gnu.org/r/grub/branches/multiboot2/ or at
http://download.savannah.gnu.org/releases-noredirect/grub/phcoder/multiboot.pdf

Some advantages:
1) Removes the need to support a fundamentaly different boot protocol
for every architecture. In future it may even be possible to have just
one image per ISA and rely on information supplied by bootloader to
differentiate between platforms (it doesn't prevent creation of an image
optimised for one platform)
2) Better collaboration between bootloaders and kernels by avoiding to
implement "one protocol per kernel".
3) Possibility to inform bootloader of exact kernel requirements and
supported features.
4) Uniform interface for retrieving e.g. memory map and initial console
info across platforms.
<Anything else you request ;)>

Any comment about multiboot2 spec is welcome.

Would it be possible for Linux in perspective to use multiboot2 instead
or in addition to current protocols?

Thanks for your time.

[1] Available with
bzr co --lightweight http://bzr.savannah.gnu.org/r/grub/trunk/grub/
or as a tarball at
http://download.savannah.gnu.org/releases-noredirect/grub/phcoder/grub-r2283.tgz

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

[RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2771 bytes --]

Hello, all. In GRUB2 community we're currently working on a
next-generation multiboot specification. It has goals similar to the
original multiboot specification but with important flaws fixed:
1) Instead of having bunch of pointers to subtables it uses a tagged
structure now. It allows it to be easier expandable, consume less space
if some fields aren't used and is easier to relocate.
2) Now it's portable. It has fields for specification of architecture
and proper field alignment. So now adding a new arcitecture is as easy
as adding a new arcitecture id and a section dealing with
architecture-specific problems like register usage to pass information.
Currently it has only i386 and mips though. Both are implemented in bzr
trunk grub[1]
Short summary:
Multiboot specification is done as an unified protocol between
bootloader and kernel to allow supporting more kernels and bootloaders
with less effort. The easiest way to use it is to create an ELF image
with an additional header which announces multiboot support and specific
kernel requirements. But it's also possible to put loading address in
multiboot header and use a non-ELF format.
When image is loaded one of GPRs contaians a magic number to identify
that kernel was loaded as multiboot and another GPR contains a pointer
to information table. The most needed information is already there, more
is in review progress, and if you feel like we're missing somethin feel
free to contact.
Complete specification is available at
http://bzr.savannah.gnu.org/r/grub/branches/multiboot2/ or at
http://download.savannah.gnu.org/releases-noredirect/grub/phcoder/multiboot.pdf

Some advantages:
1) Removes the need to support a fundamentaly different boot protocol
for every architecture. In future it may even be possible to have just
one image per ISA and rely on information supplied by bootloader to
differentiate between platforms (it doesn't prevent creation of an image
optimised for one platform)
2) Better collaboration between bootloaders and kernels by avoiding to
implement "one protocol per kernel".
3) Possibility to inform bootloader of exact kernel requirements and
supported features.
4) Uniform interface for retrieving e.g. memory map and initial console
info across platforms.
<Anything else you request  ;) >

Any comment about multiboot2 spec is welcome.

Would it be possible for FreeBSD in perspective to use multiboot2 instead
or in addition to current protocols?

Thanks for your time.

[1] Available with
bzr co --lightweight http://bzr.savannah.gnu.org/r/grub/trunk/grub/
or as a tarball at
http://download.savannah.gnu.org/releases-noredirect/grub/phcoder/grub-r2283.tgz

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

Re: [RFC] Multiboot2 drafting

[Andrew Reilly]

Hi there,

I know next to nothing about GRUB, and have not yet read the
multiboot spec, but I wonder if you could comment on how or
whether this is related to either the Open Firmware Device Tree
or the Flattened Device Tree used in various embedded OS ports.
It would be cool if there were some convergence going on...

Cheers,

-- 
Andrew

Re: [RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1545 bytes --]

> Hi there,
>
> I know next to nothing about GRUB, and have not yet read the
> multiboot spec, but I wonder if you could comment on how or
> whether this is related to either the Open Firmware Device Tree
> or the Flattened Device Tree used in various embedded OS ports.
> It would be cool if there were some convergence going on...
>
>  =20
Yes and No. multiboot2 describes some aspects of the host system
hardware but I've never heard of device trees outside of IEEE1275 or
xnu, where it's probably a historical leftover. If this specification is
clear and share some of our goals we can think of collaboration. Our
goals in this direction:
1) Allow the same kernel load on all machines implementing the same ISA.
This will require supplying info about machine.
2) Keep the things as advanced as they need to but not more advanced.
E.g. when you supply an info about serial port you tell: it's at I/O
port N rather than: it's in PCI bar X of device Y offset F. Since if OS
doesn't support PCI this info is useless and if it does it will find out
that this address is actually a part of PCI bar. This can be discussed
though.
3) Firmware independency. Ideally OS shouldn't care at all which
firmware it's running on. In some cases we may add pointers to firmware
interfaces if there are good reasons for it but it's not the goal

So if it's something clean and nice we should try integrating it. If
it's however yet another firmware-dependant overkill interface it should
be avoided.

--=20
Regards
Vladimir '=CF=86-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

Re: [RFC] Multiboot2 drafting

[Gary Jennejohn]

On Fri, 14 May 2010 08:31:13 +0200
Vladimir '__-coder/phcoder' Serbinenko <phcoder@gmail.com> wrote:

> > Hi there,
> >
> > I know next to nothing about GRUB, and have not yet read the
> > multiboot spec, but I wonder if you could comment on how or
> > whether this is related to either the Open Firmware Device Tree
> > or the Flattened Device Tree used in various embedded OS ports.
> > It would be cool if there were some convergence going on...
> >
> >
> Yes and No. multiboot2 describes some aspects of the host system
> hardware but I've never heard of device trees outside of IEEE1275 or
> xnu, where it's probably a historical leftover. If this specification is
> clear and share some of our goals we can think of collaboration. Our
> goals in this direction:
> 1) Allow the same kernel load on all machines implementing the same ISA.
> This will require supplying info about machine.
> 2) Keep the things as advanced as they need to but not more advanced.
> E.g. when you supply an info about serial port you tell: it's at I/O
> port N rather than: it's in PCI bar X of device Y offset F. Since if OS
> doesn't support PCI this info is useless and if it does it will find out
> that this address is actually a part of PCI bar. This can be discussed
> though.
> 3) Firmware independency. Ideally OS shouldn't care at all which
> firmware it's running on. In some cases we may add pointers to firmware
> interfaces if there are good reasons for it but it's not the goal
>
> So if it's something clean and nice we should try integrating it. If
> it's however yet another firmware-dependant overkill interface it should
> be avoided.
>

As an example of what I think Andrew was addressing, U-Boot can pass a
Flattened Device Tree to the Linux kernel.  This basically allows a
Linux kernel to handle variants of a board without having to custom
compile Linux for each board.  At the moment I think only powerpc-based
boards can be handled in this way.

--
Gary Jennejohn

Re: [RFC] Multiboot2 drafting

[M. Warner Losh]

In message: <4BECEE31.3060004@gmail.com>
            Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com> writes:
: Yes and No. multiboot2 describes some aspects of the host system
: hardware but I've never heard of device trees outside of IEEE1275 or
: xnu, where it's probably a historical leftover.

It is far from a historical left-over.  Linux critically depends on
the boot loader on PowerPC to provide it with a tree of devices that
it cannot otherwise probe.  On other architectures, it is becoming an
optional way to specify the device tree as well.  There are many
different implementations of this, since primarily it is just data and
boot loaders are good at providing binary blobs to the kernel...

In addition, Rafal Jawarski has ported this technology to FreeBSD.
He's presenting a paper on it today at BSDcan:
	http://www.bsdcan.org/2010/schedule/events/171.en.html
I've reviewed the work, and it goes a long way towards making some of
the more stupid and repetitive parts of doing a port to a new embedded
architecture easy.

Warner

Re: [RFC] Multiboot2 drafting

[M. Warner Losh]

In message: <20100514132142.252b092d@ernst.jennejohn.org>
            Gary Jennejohn <gljennjohn@googlemail.com> writes:
: As an example of what I think Andrew was addressing, U-Boot can pass a
: Flattened Device Tree to the Linux kernel.  This basically allows a
: Linux kernel to handle variants of a board without having to custom
: compile Linux for each board.  At the moment I think only powerpc-based
: boards can be handled in this way.

In the kernel.org kernel, this is correct.  There's a fairly mature
set of patches for ARM that brings many of the newer ARM boards into
the FDT picture.  There's some very immature MIPS patches flowing
around as well.

Warner

Re: [RFC] Multiboot2 drafting

[M. Warner Losh]

In message: <4BECEE31.3060004@gmail.com>
            Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com> writes:
: 2) Keep the things as advanced as they need to but not more advanced.
: E.g. when you supply an info about serial port you tell: it's at I/O
: port N rather than: it's in PCI bar X of device Y offset F. Since if OS
: doesn't support PCI this info is useless and if it does it will find out
: that this address is actually a part of PCI bar. This can be discussed
: though.

FDT, btw, is exactly this.  It tells the OS what hardware is present,
what resources it consumes, what connections between the hardware
there is (important for interrupts, which don't follow, necessarily,
the bus hierarchy).  It tells you about PCI host bridges, and
interesting things about the devices on the PCI bus, but only to the
extent that the hardware itself can't impart this information.  It
also offers firmware independence, so the kernel doesn't care where it
booted, or what booted it, at least to a large extent.

Again, Rafal Jawarski's work in this area in FreeBSD is mature.  While
he has only adapted it to PowerPC and ARM, it would be easy to expand
to include i386, amd64 and MIPS.  It is quite a bit less relevant to
x86, however, since ACPI provides nearly all the information that
would be in an FDT.

Warner

Re: [RFC] Multiboot2 drafting

[Rafal Jaworowski]

On 2010-05-14, at 14:44, M. Warner Losh wrote:

> In message: <4BECEE31.3060004@gmail.com>
>            Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com> writes:
> : Yes and No. multiboot2 describes some aspects of the host system
> : hardware but I've never heard of device trees outside of IEEE1275 or
> : xnu, where it's probably a historical leftover.
> 
> It is far from a historical left-over.  Linux critically depends on
> the boot loader on PowerPC to provide it with a tree of devices that
> it cannot otherwise probe.  On other architectures, it is becoming an
> optional way to specify the device tree as well.  There are many
> different implementations of this, since primarily it is just data and
> boot loaders are good at providing binary blobs to the kernel...
> 
> In addition, Rafal Jawarski has ported this technology to FreeBSD.
> He's presenting a paper on it today at BSDcan:
> 	http://www.bsdcan.org/2010/schedule/events/171.en.html
> I've reviewed the work, and it goes a long way towards making some of
> the more stupid and repetitive parts of doing a port to a new embedded
> architecture easy.

Yes, more on this can be found at the wiki page http://wiki.freebsd.org/FlattenedDeviceTree, there will also be a paper available as a post-conference material about this project.

As of now we have flattened device tree support on 2 PowerPC platforms and 6 ARM-based systems already completed.

Rafal

Re: [RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1345 bytes --]

Andrew Reilly wrote:
> Hi there,
>
> I know next to nothing about GRUB, and have not yet read the
> multiboot spec, but I wonder if you could comment on how or
> whether this is related to either the Open Firmware Device Tree
> or the Flattened Device Tree used in various embedded OS ports.
> It would be cool if there were some convergence going on...
>
>   
I've looked into it and found good and bad things.
Good:
-represents the info needed to OS. It's something definitely good and
goes in direction we need
Bad:
-it basically has all the history ballast of OpenFirmware. OFW is
supposed to be system-independent but in fact there is a multitude of
implementation with various degrees of compatibility. Same goes for
device trees. The same information is present in different places with
lots of special cases. Many of fields are actually useless variant selectors
-it uses string identifiers instead of magic numbers. This makes it more
difficult to parse
-the most useful information is coming at the leaves and interpretation
depends on whole path.

It lacks aspects of complete boot protocol. I think it needs some
cleanup. I would be interested in converging simplified and cleaned up
version of device tree into multiboot2
> Cheers,
>
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

Re: [RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1416 bytes --]

M. Warner Losh wrote:
> In message: <4BECEE31.3060004@gmail.com>
>             Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com> writes:
> : Yes and No. multiboot2 describes some aspects of the host system
> : hardware but I've never heard of device trees outside of IEEE1275 or
> : xnu, where it's probably a historical leftover.
>
> It is far from a historical left-over.
I meant the only xnu on i386. It receives a device tree from bootloader
but on most platforms it's either empty or ignored
>   Linux critically depends on
> the boot loader on PowerPC to provide it with a tree of devices that
> it cannot otherwise probe.  On other architectures, it is becoming an
> optional way to specify the device tree as well.  There are many
> different implementations of this, since primarily it is just data and
> boot loaders are good at providing binary blobs to the kernel...
>   
Could we come up with a standard of passing this information in multiboot2?
> In addition, Rafal Jawarski has ported this technology to FreeBSD.
> He's presenting a paper on it today at BSDcan:
> 	http://www.bsdcan.org/2010/schedule/events/171.en.html
> I've reviewed the work, and it goes a long way towards making some of
> the more stupid and repetitive parts of doing a port to a new embedded
> architecture easy.
>
> Warner
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

Re: [RFC] Multiboot2 drafting

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> Andrew Reilly wrote:
>   
>> Hi there,
>>
>> I know next to nothing about GRUB, and have not yet read the
>> multiboot spec, but I wonder if you could comment on how or
>> whether this is related to either the Open Firmware Device Tree
>> or the Flattened Device Tree used in various embedded OS ports.
>> It would be cool if there were some convergence going on...
>>
>>   
>>     
> I've looked into it and found good and bad things.
> Good:
> -represents the info needed to OS. It's something definitely good and
> goes in direction we need
> Bad:
> -it basically has all the history ballast of OpenFirmware. OFW is
> supposed to be system-independent but in fact there is a multitude of
> implementation with various degrees of compatibility. Same goes for
> device trees. The same information is present in different places with
> lots of special cases. Many of fields are actually useless variant selectors
> -it uses string identifiers instead of magic numbers. This makes it more
> difficult to parse
> -the most useful information is coming at the leaves and interpretation
> depends on whole path.
>   
And traditional OpenFirmware has too much info which is probably
useless. E.g. once you know where PCI controller is and you support PCI
you can probe devices which are on it. If you don't support PCI
controller the info about devices on it is probably useless too.
Only exception is "early" info which is needed to e.g. setup console
before normal drivers are up
> It lacks aspects of complete boot protocol. I think it needs some
> cleanup. I would be interested in converging simplified and cleaned up
> version of device tree into multiboot2
>   
>> Cheers,
>>
>>   
>>     
>
>
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

Re: [RFC] Multiboot2 drafting

[Andrew Reilly]

Hi Vladimir,

On Mon, May 24, 2010 at 08:19:14PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> I've looked into it and found good and bad things.

Thanks for looking!  I didn't expect that it would be all good:
I've only read some papers, and it seemed to be covering much
the same sort of ground.  I hope that some avenues for
convergence have or will be found.

Cheers,

-- 
Andrew

Re: [RFC] Multiboot2 drafting

[Nathan Whitehorn]

Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>   
>> Andrew Reilly wrote:
>>   
>>     
>>> Hi there,
>>>
>>> I know next to nothing about GRUB, and have not yet read the
>>> multiboot spec, but I wonder if you could comment on how or
>>> whether this is related to either the Open Firmware Device Tree
>>> or the Flattened Device Tree used in various embedded OS ports.
>>> It would be cool if there were some convergence going on...
>>>
>>>   
>>>     
>>>       
>> I've looked into it and found good and bad things.
>> Good:
>> -represents the info needed to OS. It's something definitely good and
>> goes in direction we need
>> Bad:
>> -it basically has all the history ballast of OpenFirmware. OFW is
>> supposed to be system-independent but in fact there is a multitude of
>> implementation with various degrees of compatibility. Same goes for
>> device trees. The same information is present in different places with
>> lots of special cases. Many of fields are actually useless variant selectors
>> -it uses string identifiers instead of magic numbers. This makes it more
>> difficult to parse
>> -the most useful information is coming at the leaves and interpretation
>> depends on whole path.
>>   
>>     
> And traditional OpenFirmware has too much info which is probably
> useless. E.g. once you know where PCI controller is and you support PCI
> you can probe devices which are on it. If you don't support PCI
> controller the info about devices on it is probably useless too.
> Only exception is "early" info which is needed to e.g. setup console
> before normal drivers are up
>   
This isn't quite true. The FDT standard does not require you to add 
discoverable devices to the tree, for one, and it actually can be useful 
to have them in many circumstances, as well. For instance, suppose you 
have some kind of complicated object on the PCI bus (a GPIO controller, 
or an I2C bus). The children of the I2C bus need to be enumerated in the 
tree, since there is no other way to discover them, and it is also 
necessary to identify the I2C controller with a PCI device.
-Nathan