multiboot2: remove "a.out kludge" requirement

multiboot2: remove "a.out kludge" requirement

[Hollis Blanchard]

One requirement listed for multiboot2 was that ELF headers should be
ignored, because NetBSD was setting their ELF addresses incorrectly.
This has now been fixed (see
http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=32865), so I've
removed that requirement.

ELF headers tell you where to load an executable far better than the
kludge, so it really doesn't make sense to ignore ELF.

-Hollis

Re: multiboot2: remove "a.out kludge" requirement

[Andrei E. Warkentin]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

How about having a custom e_type for ELF images booted by GRUB?  
Something in the range between ET_LOOS and ET_HIOS (the OS specific  
types). This way one could avoid the Multiboot header in ELF, as the  
file would itself would identify self as GRUB-bootable or not.

Also...
I am not familiar with the module architecture in GRUB2 (whether mods  
are ET_REL or ET_DYN), but having a custom type for those would sure  
simplify those code paths too.

Andrei Evgenievich Warkentin
andrey.warkentin@gmail.com
Cell: (+1) (847) 321-15-55


On 17.11.2006, at 15:10, Hollis Blanchard wrote:

> One requirement listed for multiboot2 was that ELF headers should be
> ignored, because NetBSD was setting their ELF addresses incorrectly.
> This has now been fixed (see
> http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=32865), so  
> I've
> removed that requirement.
>
> ELF headers tell you where to load an executable far better than the
> kludge, so it really doesn't make sense to ignore ELF.
>
> -Hollis
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


[-- Attachment #2: Type: text/html, Size: 3908 bytes --]

Re: multiboot2: remove "a.out kludge" requirement

[Hollis Blanchard]

On Fri, 2006-11-17 at 15:27 -0600, Andrei E. Warkentin wrote:
> How about having a custom e_type for ELF images booted by GRUB?
> Something in the range between ET_LOOS and ET_HIOS (the OS specific
> types). This way one could avoid the Multiboot header in ELF, as the
> file would itself would identify self as GRUB-bootable or not. 

Why would we need a custom e_type? We know how to load ELF; we can
already omit the multiboot header.

Are you worried about a user accidentally running "multiboot /bin/ls"?
I'm not... :)

> Also...
> I am not familiar with the module architecture in GRUB2 (whether mods
> are ET_REL or ET_DYN), but having a custom type for those would sure
> simplify those code paths too. 

How would it simplify the code?

(Modules are ET_REL, for the record.)

-Hollis

Re: multiboot2: remove "a.out kludge" requirement

[Andrei E. Warkentin]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

No, I think Okuji mentioned that he didn't want to omit the Multiboot  
header out of the possibility of not registering the Multiboot  
header's presence if it is past the 8K mark (i.e. corrupt multiboot  
kernel).

With a custom e_type you could be sure it's really a Multiboot kernel  
or not... and not ever be concerned with corrupt files with the  
header past 8K.

Andrei Evgenievich Warkentin
andrey.warkentin@gmail.com
Cell: (+1) (847) 321-15-55


On 17.11.2006, at 16:01, Hollis Blanchard wrote:

> On Fri, 2006-11-17 at 15:27 -0600, Andrei E. Warkentin wrote:
>> How about having a custom e_type for ELF images booted by GRUB?
>> Something in the range between ET_LOOS and ET_HIOS (the OS specific
>> types). This way one could avoid the Multiboot header in ELF, as the
>> file would itself would identify self as GRUB-bootable or not.
>
> Why would we need a custom e_type? We know how to load ELF; we can
> already omit the multiboot header.
>
> Are you worried about a user accidentally running "multiboot /bin/ls"?
> I'm not... :)
>
>> Also...
>> I am not familiar with the module architecture in GRUB2 (whether mods
>> are ET_REL or ET_DYN), but having a custom type for those would sure
>> simplify those code paths too.
>
> How would it simplify the code?
>
> (Modules are ET_REL, for the record.)
>
> -Hollis
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


[-- Attachment #2: Type: text/html, Size: 5463 bytes --]

Re: multiboot2: remove "a.out kludge" requirement

[Hollis Blanchard]

That is true. However I'm not sure how easily that field can be
modified. Would we need a custom tool for that? Could it be done with a
linker script? We are talking specifically about ELF here, so it might
make more sense to require a PT_NOTE segment.

I'm not sure I agree with Okuji though. If the multiboot header is
beyond the 8KB limit, it is an OS programming error, and GRUB cannot be
responsible for trying to prevent all programming errors.

On the other hand, if this is an error we could catch that would make OS
debugging easier, it might be worth it.

Right now in my loader I just have a debug message that appears if the
header is not found. An OS developer who finds her kernel no longer
boots could simply 'set debug=loader' and then boot, and the debug
message would indicate the problem (since she knows she's *supposed* to
have a multiboot header).

I think the debug message solves the debugging issue, and it doesn't
burden OSs that don't need the header.

-Hollis

On Fri, 2006-11-17 at 16:36 -0600, Andrei E. Warkentin wrote:
> No, I think Okuji mentioned that he didn't want to omit the Multiboot
> header out of the possibility of not registering the Multiboot
> header's presence if it is past the 8K mark (i.e. corrupt multiboot
> kernel).
> 
> 
> With a custom e_type you could be sure it's really a Multiboot kernel
> or not... and not ever be concerned with corrupt files with the header
> past 8K. 
> 
> Andrei Evgenievich Warkentin
> andrey.warkentin@gmail.com
> Cell: (+1) (847) 321-15-55
> 
> 
> 
> On 17.11.2006, at 16:01, Hollis Blanchard wrote:
> 
> > On Fri, 2006-11-17 at 15:27 -0600, Andrei E. Warkentin wrote:
> > > How about having a custom e_type for ELF images booted by GRUB?
> > > Something in the range between ET_LOOS and ET_HIOS (the OS
> > > specific
> > > types). This way one could avoid the Multiboot header in ELF, as
> > > the
> > > file would itself would identify self as GRUB-bootable or not. 
> > 
> > 
> > Why would we need a custom e_type? We know how to load ELF; we can
> > already omit the multiboot header.
> > 
> > 
> > Are you worried about a user accidentally running
> > "multiboot /bin/ls"?
> > I'm not... :)
> > 
> > 
> > > Also...
> > > I am not familiar with the module architecture in GRUB2 (whether
> > > mods
> > > are ET_REL or ET_DYN), but having a custom type for those would
> > > sure
> > > simplify those code paths too. 
> > 
> > 
> > How would it simplify the code?
> > 
> > 
> > (Modules are ET_REL, for the record.)
> > 
> > 
> > -Hollis
> > 
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Grub-devel mailing list
> > Grub-devel@gnu.org
> > http://lists.gnu.org/mailman/listinfo/grub-devel
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel

Re: multiboot2: remove "a.out kludge" requirement

[Brano Zarnovican]

On 11/17/06, Andrei E. Warkentin <andrey.warkentin@gmail.com> wrote:
>
> How about having a custom e_type for ELF images booted by GRUB? Something in

Guys at Xen solved it nicely with custom ELF section. Somewhere in
your OS code you create a section with specific name, bootloader will
be looking for. For multiboot it might look like:

.section __multiboot_v2
        .long MAGIC_THIS
        .long FLAGS_THAT

Header position is exactly defined in ELF which Grub already speaks.
It does not need to be in first 8kB. The downside is that it will make
multiboot ELF-only.

Oh. Did I mention that you can have more than one header in a single image ?
(for non-compatible multiboot versions ;)

BranoZ

Re: multiboot2: remove "a.out kludge" requirement

[Tristan Gingold]

On Sun, Nov 19, 2006 at 10:26:54AM +0000, Brano Zarnovican wrote:
> On 11/17/06, Andrei E. Warkentin <andrey.warkentin@gmail.com> wrote:
> >
> >How about having a custom e_type for ELF images booted by GRUB? Something 
> >in
> 
> Guys at Xen solved it nicely with custom ELF section. Somewhere in
> your OS code you create a section with specific name, bootloader will
> be looking for. For multiboot it might look like:
> 
> .section __multiboot_v2
>        .long MAGIC_THIS
>        .long FLAGS_THAT
> 
> Header position is exactly defined in ELF which Grub already speaks.
> It does not need to be in first 8kB. The downside is that it will make
> multiboot ELF-only.
> 
> Oh. Did I mention that you can have more than one header in a single image ?
> (for non-compatible multiboot versions ;)
But using notes make it almost impossible to use a compressed image.
(This is almost a requirement for ia64, but I also think ia64 does not
require these flags).

Tristan.

Re: multiboot2: remove "a.out kludge" requirement

[Andrei E. Warkentin]

[-- Attachment #1: Type: text/plain, Size: 1380 bytes --]

Well, couldn't the compressed .text/.rdata be stored in an ELF  
section itself?

Andrei Evgenievich Warkentin
andrey.warkentin@gmail.com
Cell: (+1) (847) 321-15-55


On 19.11.2006, at 10:55, Tristan Gingold wrote:

> On Sun, Nov 19, 2006 at 10:26:54AM +0000, Brano Zarnovican wrote:
>> On 11/17/06, Andrei E. Warkentin <andrey.warkentin@gmail.com> wrote:
>>>
>>> How about having a custom e_type for ELF images booted by GRUB?  
>>> Something
>>> in
>>
>> Guys at Xen solved it nicely with custom ELF section. Somewhere in
>> your OS code you create a section with specific name, bootloader will
>> be looking for. For multiboot it might look like:
>>
>> .section __multiboot_v2
>>        .long MAGIC_THIS
>>        .long FLAGS_THAT
>>
>> Header position is exactly defined in ELF which Grub already speaks.
>> It does not need to be in first 8kB. The downside is that it will  
>> make
>> multiboot ELF-only.
>>
>> Oh. Did I mention that you can have more than one header in a  
>> single image ?
>> (for non-compatible multiboot versions ;)
> But using notes make it almost impossible to use a compressed image.
> (This is almost a requirement for ia64, but I also think ia64 does not
> require these flags).
>
> Tristan.
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


[-- Attachment #2: Type: text/html, Size: 5693 bytes --]

Re: multiboot2: remove "a.out kludge" requirement

[Tristan Gingold]

On Sun, Nov 19, 2006 at 01:31:17PM -0600, Andrei E. Warkentin wrote:
> Well, couldn't the compressed .text/.rdata be stored in an ELF  
> section itself?
Do you have a more complex idea? :-)

For ia64, we don't have all the x86 issues: grub must just be able to load
an ELF file (as well as modules) and to start it with the correct parameters.

In fact this discussion is more or less x86 specific, isn't it ?

Tristan.

Re: multiboot2: remove "a.out kludge" requirement

[Hollis Blanchard]

On Sun, 2006-11-19 at 17:55 +0100, Tristan Gingold wrote:
> On Sun, Nov 19, 2006 at 10:26:54AM +0000, Brano Zarnovican wrote:
> > On 11/17/06, Andrei E. Warkentin <andrey.warkentin@gmail.com> wrote:
> > >
> > >How about having a custom e_type for ELF images booted by GRUB? Something 
> > >in
> > 
> > Guys at Xen solved it nicely with custom ELF section. Somewhere in
> > your OS code you create a section with specific name, bootloader will
> > be looking for. For multiboot it might look like:
> > 
> > .section __multiboot_v2
> >        .long MAGIC_THIS
> >        .long FLAGS_THAT
> > 
> But using notes make it almost impossible to use a compressed image.
> (This is almost a requirement for ia64, but I also think ia64 does not
> require these flags).

I don't see how. GRUB must already extract .elf.gz files to read the ELF
header to load the executable.

-Hollis

Re: multiboot2: remove "a.out kludge" requirement

[Tristan Gingold]

On Mon, Nov 20, 2006 at 01:13:07PM -0600, Hollis Blanchard wrote:
> On Sun, 2006-11-19 at 17:55 +0100, Tristan Gingold wrote:
> > On Sun, Nov 19, 2006 at 10:26:54AM +0000, Brano Zarnovican wrote:
> > > On 11/17/06, Andrei E. Warkentin <andrey.warkentin@gmail.com> wrote:
> > > >
> > > >How about having a custom e_type for ELF images booted by GRUB? Something 
> > > >in
> > > 
> > > Guys at Xen solved it nicely with custom ELF section. Somewhere in
> > > your OS code you create a section with specific name, bootloader will
> > > be looking for. For multiboot it might look like:
> > > 
> > > .section __multiboot_v2
> > >        .long MAGIC_THIS
> > >        .long FLAGS_THAT
> > > 
> > But using notes make it almost impossible to use a compressed image.
> > (This is almost a requirement for ia64, but I also think ia64 does not
> > require these flags).
> 
> I don't see how. GRUB must already extract .elf.gz files to read the ELF
> header to load the executable.
Generally the first 1kB (or 2 doesn't really matter) are extracted.  They
should contain the header and the segment table.  Then the remaining is read
in order.
But if a special section (such as .note) must be read before the remaining of
the file, things become more complex.

Tristan.

Re: multiboot2: remove "a.out kludge" requirement

[Hollis Blanchard]

On Sun, 2006-11-19 at 10:26 +0000, Brano Zarnovican wrote:
> On 11/17/06, Andrei E. Warkentin <andrey.warkentin@gmail.com> wrote:
> >
> > How about having a custom e_type for ELF images booted by GRUB? Something in
> 
> Guys at Xen solved it nicely with custom ELF section. Somewhere in
> your OS code you create a section with specific name, bootloader will
> be looking for. For multiboot it might look like:
> 
> .section __multiboot_v2
>         .long MAGIC_THIS
>         .long FLAGS_THAT

Sections are for linkers; segments are for loaders. GRUB loads ELF files
by segment, which is why I suggested a PT_LOAD segment.

> Header position is exactly defined in ELF which Grub already speaks.
> It does not need to be in first 8kB. The downside is that it will make
> multiboot ELF-only.

That is a non-starter I'm afraid. Not that I care much about non-ELF
kernels personally, but we must support them.

Of course we could simply say that some other method is needed for
non-ELF files.

-Hollis

Re: multiboot2: remove "a.out kludge" requirement

[Yoshinori K. Okuji]

On Friday 17 November 2006 22:10, Hollis Blanchard wrote:
> One requirement listed for multiboot2 was that ELF headers should be
> ignored, because NetBSD was setting their ELF addresses incorrectly.
> This has now been fixed (see
> http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=32865), so I've
> removed that requirement.
>
> ELF headers tell you where to load an executable far better than the
> kludge, so it really doesn't make sense to ignore ELF.

Still, it is sometimes useful to override information. I do not think it is 
necessary to change the behavior.

Okuji