[multiboot] command-line format

[multiboot] command-line format

[Grégoire Sutre]

Hi,

Vladimir just sent a patch regarding this issue, but I had this long 
(sorry!) email almost finished already, so here it is.  And this might 
give more context for Vladimir's email.

As mentioned in a previous thread, I had some problems getting multiboot 
options recognized by the NetBSD kernel, and this was actually due to 
the fact that GRUB Legacy implicitly passes the booted file as first 
argument whereas GRUB 2 doesn't [1].

I started a thread on NetBSD's port-i386 mailing list [2] on the format 
of the multiboot command-line, and the discussion moved towards the 
change in GRUB regarding the first argument (booted kernel in GRUB 
Legacy, removed in GRUB 2).  I ended up doing tests with Xen, which is 
also multiboot-compliant, and Xen also skips the first argument in its 
command-line parsing code (well, it's a bit more complex, see [3] for 
details if you're interested).

In the end, it may be the case that most multiboot-compliant kernels out 
there still assume that the first argument is the booted file name, and 
skip it without even looking at it.  Do you know of any kernel that does 
not make this assumption?

There are also other multiboot-compliant boot loaders (e.g. NetBSD's 
boot loader).  Apart from GRUB 2, do you know of another boot-loader 
that does not put the booted file name as first argument?

I understand that the specification does not enforce the boot loader to 
follow a specific command-line format.  And, in a perfect world, 
developers should follow the specification, disregarding any particular 
implementation of the specification.  But I guess we don't live in a 
perfect world ;-)

The reference implementation (GRUB legacy) for the specification led 
kernel developers to assume a specific command-line format.  And for the 
sake of backward compatibility, all multiboot command-lines for those 
kernels should contain the booted file name as first argument.  So why 
not change the specification in this way, i.e. require that the 
command-line starts with the booted file name?  I believe that this 
would merely amount to writing down what most users of the specification 
had already in mind...

Grégoire

p.s. I guess that there are not many GRUB developers on the NetBSD
mailing lists, and, for my part, this topic is new...  Feel free to
intervene :-)

[1] http://lists.gnu.org/archive/html/grub-devel/2010-01/msg00146.html
[2] http://mail-index.netbsd.org/port-i386/2010/01/09/msg001747.html
[3] http://mail-index.netbsd.org/port-i386/2010/01/14/msg001768.html

Re: [multiboot] command-line format

[Seth Goldberg]

   Solaris uses the first argument (and it is essential to the kernel  
loading process actually).  I've worked around the removal of the  
original arg0 string by passing the kernel name twice, ie:

   skern=/platform/i86pc/kernel/unix
   multiboot $skern $skern ...

   --S

On Jan 14, 2010, at 8:53 AM, Grégoire Sutre <gregoire.sutre@gmail.com>  
wrote:

> Hi,
>
> Vladimir just sent a patch regarding this issue, but I had this long  
> (sorry!) email almost finished already, so here it is.  And this  
> might give more context for Vladimir's email.
>
> As mentioned in a previous thread, I had some problems getting  
> multiboot options recognized by the NetBSD kernel, and this was  
> actually due to the fact that GRUB Legacy implicitly passes the  
> booted file as first argument whereas GRUB 2 doesn't [1].
>
> I started a thread on NetBSD's port-i386 mailing list [2] on the  
> format of the multiboot command-line, and the discussion moved  
> towards the change in GRUB regarding the first argument (booted  
> kernel in GRUB Legacy, removed in GRUB 2).  I ended up doing tests  
> with Xen, which is also multiboot-compliant, and Xen also skips the  
> first argument in its command-line parsing code (well, it's a bit  
> more complex, see [3] for details if you're interested).
>
> In the end, it may be the case that most multiboot-compliant kernels  
> out there still assume that the first argument is the booted file  
> name, and skip it without even looking at it.  Do you know of any  
> kernel that does not make this assumption?
>
> There are also other multiboot-compliant boot loaders (e.g. NetBSD's  
> boot loader).  Apart from GRUB 2, do you know of another boot-loader  
> that does not put the booted file name as first argument?
>
> I understand that the specification does not enforce the boot loader  
> to follow a specific command-line format.  And, in a perfect world,  
> developers should follow the specification, disregarding any  
> particular implementation of the specification.  But I guess we  
> don't live in a perfect world ;-)
>
> The reference implementation (GRUB legacy) for the specification led  
> kernel developers to assume a specific command-line format.  And for  
> the sake of backward compatibility, all multiboot command-lines for  
> those kernels should contain the booted file name as first  
> argument.  So why not change the specification in this way, i.e.  
> require that the command-line starts with the booted file name?  I  
> believe that this would merely amount to writing down what most  
> users of the specification had already in mind...
>
> Grégoire
>
> p.s. I guess that there are not many GRUB developers on the NetBSD
> mailing lists, and, for my part, this topic is new...  Feel free to
> intervene :-)
>
> [1] http://lists.gnu.org/archive/html/grub-devel/2010-01/msg00146.html
> [2] http://mail-index.netbsd.org/port-i386/2010/01/09/msg001747.html
> [3] http://mail-index.netbsd.org/port-i386/2010/01/14/msg001768.html
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel

Re: [multiboot] command-line format

[Grégoire Sutre]

Seth Goldberg wrote:

>   Solaris uses the first argument (and it is essential to the kernel 
> loading process actually).

That's good to know, thanks.  So the list of multiboot-compliant kernels 
that (are known to) assume a GRUB Legacy command-line format becomes: 
Xen, Solaris, NetBSD.

Does anybody know how Linux works with respect to this?

Grégoire

Re: [multiboot] command-line format

[Seth Goldberg]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 587 bytes --]



Quoting Grégoire Sutre, who wrote the following on Fri, 15 Jan 2010:

> Seth Goldberg wrote:
>
>>   Solaris uses the first argument (and it is essential to the kernel 
>> loading process actually).
>
> That's good to know, thanks.  So the list of multiboot-compliant kernels that 
> (are known to) assume a GRUB Legacy command-line format becomes: Xen, 
> Solaris, NetBSD.
>
> Does anybody know how Linux works with respect to this?

  Linux doesn't use multiboot -- it has its own convention for passing 
information between the boot loader and the kernel.

  --S

Re: [multiboot] command-line format

[Grégoire Sutre]

Seth Goldberg wrote:

>  Linux doesn't use multiboot -- it has its own convention for passing 
> information between the boot loader and the kernel.

Ok.  I was refering to Robert's email mentioning wraplinux:

http://lists.gnu.org/archive/html/grub-devel/2010-01/msg00046.html

I glanced over the code, and it looks like they only add a multiboot 
header.  I did not see any code to access the multiboot information 
structure (which contains the multiboot command line).

Grégoire

Re: [multiboot] command-line format

[Robert Millan]

On Thu, Jan 14, 2010 at 05:53:25PM +0100, Grégoire Sutre wrote:
>
> As mentioned in a previous thread, I had some problems getting multiboot  
> options recognized by the NetBSD kernel, and this was actually due to  
> the fact that GRUB Legacy implicitly passes the booted file as first  
> argument whereas GRUB 2 doesn't [1].
>
> I started a thread on NetBSD's port-i386 mailing list [2] on the format  
> of the multiboot command-line, and the discussion moved towards the  
> change in GRUB regarding the first argument (booted kernel in GRUB  
> Legacy, removed in GRUB 2).  I ended up doing tests with Xen, which is  
> also multiboot-compliant, and Xen also skips the first argument in its  
> command-line parsing code (well, it's a bit more complex, see [3] for  
> details if you're interested).
>
> In the end, it may be the case that most multiboot-compliant kernels out  
> there still assume that the first argument is the booted file name, and  
> skip it without even looking at it.  Do you know of any kernel that does  
> not make this assumption?

Hi Grégoire,

This might sound contradictory to you, but we did this change precisely with
those kernels in mind.  We didn't do this just because we're pedantic when
reading the spec!  Our goal was to improve support for these situations.  I
will explain this:

GRUB Legacy used to pass this information implicitly as you point out.  As a
result, most of these kernels just skip the first parameter, but there are
also a few (like the kernel of OpenSolaris) which make use of the information
in it (to find some other files I think).

The problem we found is that because the file path parameter was implicit,
user was unable to modify it.  As a result, it became impossible to e.g.
load kernel of OpenSolaris from one device/path and its associated files
from somewhere else.  We wanted to provide the best user experience on
those platforms (Multiboot is our primary target, it's the boot protocol of
the GNU system and is much more important for GRUB than Linux loaders),
and this limitation was deemed unacceptable.

With this change, we provided more flexibility.  If users want to retain
the old protocol, they can still do this.  For example if a kernel just
ignores first parameter:

  multiboot /kernel unused foo bar

Or if it really wants to know the pathname:

  multiboot /kernel /kernel foo bar

One might think that this is an unnecessary burden, but the fact is that
users generally don't need to type any of this by hand (or even edit grub.cfg
by hand).  We provide the grub-mkconfig utility which is extensible and can
support this for a variety of different kernels.  So when someone writes a
script snippet for kernel of NetBSD or kernel of OpenSolaris (e.g.
10_knetbsd.in and 10_kopensolaris.in), they can make GRUB add a dummy
parameter for the kernel to skip (NetBSD's) or a file path for the kernel
to rely on (OpenSolaris').

P.S: please forward this message to the relevant NetBSD mailing list(s) if
you think this is appropiate.  TIA

-- 
Robert Millan

  "Be the change you want to see in the world" -- Gandhi

Re: [multiboot] command-line format

[Grégoire Sutre]

Hi Robert,

Thanks for your detailed explanation, it was really helpful to me.  I 
understand that for compatibility with some platforms, GRUB must provide 
a way to specify two potentially different file parameters:

(a) the GRUB path to the booted file; this path does not appear in the 
multiboot command-line.
(b) the path that shall be passed to the booted file; this path (if any) 
is the first argument in the multiboot command-line.

However, my first post in this thread was more about the multiboot 
specification itself.  In light of your explanations, I would re-phrase 
my suggestion as follows: in the multiboot specification, require that 
the first argument of the MB command-line be a path to the booted file 
*in a notation that is specific to the booted kernel*.  Or, if this is 
too constraining or too confusing, simply ask that the first argument is 
an informative string that does not contain kernel options (i.e. it can 
safely be skipped).  This way, the specification would be closer to the 
reference implementation (GRUB Legacy), and, more importantly, to what 
multiboot-compliant kernels already *assume* about the format of the 
command-line (NetBSD, OpenSolaris, Xen, others?).


For GRUB 2, this could be implemented by a multiboot command that, by 
default, behaves as GRUB Legacy, but also offers a way to modify the 
implicit first argument of the multiboot command-line.  Something like:

multiboot $path[;$ospath] ...

(a) $path is the GRUB path to the booted file.
(b) $ospath is the path that shall be passed to the booted file.

By default, if there is no ';' in the first argument, $ospath is set to 
the value that GRUB Legacy would have used.  Maybe a GRUB shell variable 
`multiboot_ospath' would be better than this ';' marker.

This way, we keep the extra flexibility of having different paths, but 
we also:

- keep backward compatibility with GRUB Legacy: kernels can still assume 
that the first argument of the multiboot command-line is not a kernel 
option.
- don't force users to repeat the path for kernels that are happy with 
the way GRUB Legacy worked (all of them except OpenSolaris?).  I agree 
that this point is of no concern to most users, but in some cases 
grub-mkconfig does not work (e.g. grub-mkconfig does not support 
--root-directory at the moment), and some users prefer the flexibility 
of the command line anyway.

Grégoire

Re: [multiboot] command-line format

[Robert Millan]

On Sun, Jan 17, 2010 at 07:02:16PM +0100, Grégoire Sutre wrote:
> However, my first post in this thread was more about the multiboot  
> specification itself.  In light of your explanations, I would re-phrase  
> my suggestion as follows: in the multiboot specification, require that  
> the first argument of the MB command-line be a path to the booted file  
> *in a notation that is specific to the booted kernel*.  Or, if this is  
> too constraining or too confusing, simply ask that the first argument is  
> an informative string that does not contain kernel options (i.e. it can  
> safely be skipped).  This way, the specification would be closer to the  
> reference implementation (GRUB Legacy), and, more importantly, to what  
> multiboot-compliant kernels already *assume* about the format of the  
> command-line (NetBSD, OpenSolaris, Xen, others?).

I really don't think this is something that should be *mandated* by the spec,
specially not for legacy compatibility reasons.

Also, how is a bootloader supposed to know about a notation specific to the
booted kernel?  GRUB doesn't have the slightest clue how kernels name their
devices, and it shouldn't; OTOH the environment who generated grub.cfg knows
all the details.

But we could add a note saying that even if it's not required, the behaviour
of using first parameter this way is widespread.

> For GRUB 2, this could be implemented by a multiboot command that, by  
> default, behaves as GRUB Legacy, but also offers a way to modify the  
> implicit first argument of the multiboot command-line.  Something like:
>
> multiboot $path[;$ospath] ...
>
> (a) $path is the GRUB path to the booted file.
> (b) $ospath is the path that shall be passed to the booted file.
>
> By default, if there is no ';' in the first argument, $ospath is set to  
> the value that GRUB Legacy would have used.  Maybe a GRUB shell variable  
> `multiboot_ospath' would be better than this ';' marker.

I'm sorry, but I don't think this kind of UI complexity benefits users.

Also, I firmly believe that we shouldn't be satisfied with flawed solutions
just because they're part of our legacy baggage.  We can do better than this
because Free Software is more flexible and more powerful.  For example, NetBSD
can distribute its own version of GRUB and patch it, and coordinate an
interface change between GRUB and its kernel, etc.

We're not like BIOS vendors + Microsoft which are stuck with 40-year-old
crappy interfaces because neither of them has the flexibility to change each
others' implementation.  It would almost be funny if it wasn't because they
managed to put their i8086-encumbered legacy in every modern machine around
the world.

-- 
Robert Millan

  "Be the change you want to see in the world" -- Gandhi

Re: [multiboot] command-line format

[richardvoigt]

> Also, I firmly believe that we shouldn't be satisfied with flawed solutions
> just because they're part of our legacy baggage.  We can do better than this
> because Free Software is more flexible and more powerful.  For example, NetBSD
> can distribute its own version of GRUB and patch it, and coordinate an
> interface change between GRUB and its kernel, etc.

I think a bootloader with "universal" in its name should be doing
everything possible to avoid this.  If I want to multiboot between
Linux, NetBSD, OpenSolaris, and OpenBSD, do I load my MBR with the BSD
fork of GRUB, the Linux fork of GRUB, or the Solaris fork of GRUB?

This really is a non-solution.

I think that defaulting the first parameter to the value used by what,
90% of kernels, but providing a way to override it (I like the
variable idea) will be the least surprising for users.  The path is
filesystem-relative anyway, so GRUB isn't required to know how the
kernel names its devices, nor do changes to grub device addressing
change the parameter passed.

But the current behavior requiring the user to explicitly specify the
0th parameter certainly works, and if the format is to be changed then
during the switch from GRUB Legacy to GRUB2 is the ideal time to do
it.

Re: [multiboot] command-line format

[Robert Millan]

On Sun, Jan 17, 2010 at 01:39:48PM -0600, richardvoigt@gmail.com wrote:
> 
> I think a bootloader with "universal" in its name should be doing
> everything possible to avoid this.  If I want to multiboot between
> Linux, NetBSD, OpenSolaris, and OpenBSD, do I load my MBR with the BSD
> fork of GRUB, the Linux fork of GRUB, or the Solaris fork of GRUB?

It doesn't matter, because whichever version of GRUB you use should
generate a grub.cfg that uses multiboot command with appropiate
parameters.

> I think that defaulting the first parameter to the value used by what,
> 90% of kernels, but providing a way to override it (I like the
> variable idea) will be the least surprising for users.  The path is
> filesystem-relative anyway, so GRUB isn't required to know how the
> kernel names its devices, nor do changes to grub device addressing
> change the parameter passed.

Filesystem-relative paths still reflect GRUB's view and aren't really a good
solution for system tools (which deals with absolute paths and mount points
instead).  And even then, there are details like weird characters, slashes,
case sensitivity, etc which might differ.

If you still want GRUB's point of view, you can simply replace:

	GRUB_PATH=$(make_system_path_relative_to_its_root ${SYSTEM_PATH})
	prepare_grub_to_access_device ${SYSTEM_PATH}
	echo "multiboot ${GRUB_PATH}"

with:

	GRUB_PATH=$(make_system_path_relative_to_its_root ${SYSTEM_PATH})
	prepare_grub_to_access_device ${SYSTEM_PATH}
	echo "multiboot ${GRUB_PATH} ${GRUB_PATH}"

in your mkconfig script.  But it is much more straightforward to simply do:

	GRUB_PATH=$(make_system_path_relative_to_its_root ${SYSTEM_PATH})
	prepare_grub_to_access_device ${SYSTEM_PATH}
	echo "multiboot ${GRUB_PATH}	${SYSTEM_PATH}"

Then your system tools obtain a path that makes sense to them.

-- 
Robert Millan

  "Be the change you want to see in the world" -- Gandhi

Re: [multiboot] command-line format

[Grégoire Sutre]

Hi Robert,

> On Sun, Jan 17, 2010 at 01:39:48PM -0600, richardvoigt@gmail.com wrote:
>> I think a bootloader with "universal" in its name should be doing
>> everything possible to avoid this.  If I want to multiboot between
>> Linux, NetBSD, OpenSolaris, and OpenBSD, do I load my MBR with the BSD
>> fork of GRUB, the Linux fork of GRUB, or the Solaris fork of GRUB?
> 
> It doesn't matter, because whichever version of GRUB you use should
> generate a grub.cfg that uses multiboot command with appropiate
> parameters.

According to my little experience, I believe that GRUB should not rely 
too much on an automatic generation of a correct grub.cfg.  This 
requires an os-prober command that (1) is available on the system 
installing GRUB, which is not always the case, and (2) can detect other 
installed Oses correctly, which is not always the case as well.

What if I want to install GRUB to some removable drive or floppy disk 
for the purpose of rescuing systems with a damaged MBR?  You cannot hope 
to generate a grub.cfg that will universally work, so in that case you 
must use the GRUB shell.

As you mentionned in a previous email, it is important to keep the GRUB 
shell user-friendly.  Requiring users to duplicate the first argument of 
multiboot commands is not my idea of user-friendliness.  But I agree 
that it works.

Grégoire