Converting Ubuntu CDs to GRUB

Converting Ubuntu CDs to GRUB

[Colin Watson]

I would like to convert Ubuntu CDs to boot using GRUB for Ubuntu 10.10.

This is a bit ambitious.  We're currently using ISOLINUX with SuSE's
gfxboot extensions, and extensive theming to offer various custom menus.
A number of the facilities we depend on there aren't yet present in
gfxmenu, and while I can probably get away without all of them I don't
think I'll be able to strip it all the way down to the point of a single
boot menu.

Nevertheless, I think this is worth pursuing for various reasons: we're
probably going to need to use GRUB to support EFI-based machines anyway;
maintaining gfxboot-theme-ubuntu is really no fun at all; and I'd like
to see GRUB get to the point where it can handle these requirements
easily.  Judging from the images on the front page of
http://grub.gibibit.com/, I gather that this was at least considered as
a goal for gfxmenu!

I've written up a brief description of our needs here:

  https://wiki.ubuntu.com/FoundationsTeam/Specs/MaverickCDBoot

I think that the extensions we'd need in gfxmenu amount to:

  * Supplementary menu support, with menus bound to keys
   * Either scrolling menus or grid menus, preferably grid
  * The ability to bind functions to menus to process selections
  * Checkbox widgets

Needless to say I'm willing to take on the bulk of the work here, but
I'm very new to gfxmenu and I would greatly appreciate review from those
who've spent more time thinking about it and hacking on it.  In
particular, I'd appreciate thoughts on how this kind of thing might best
be represented in extensions to the theme format.

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Converting Ubuntu CDs to GRUB

[Colin D Bennett]

On Thu, 20 May 2010 18:35:54 +0100
Colin Watson <cjwatson@ubuntu.com> wrote:

> I would like to convert Ubuntu CDs to boot using GRUB for Ubuntu
> 10.10.
> 
> This is a bit ambitious.  We're currently using ISOLINUX with SuSE's
> gfxboot extensions, and extensive theming to offer various custom
> menus. A number of the facilities we depend on there aren't yet
> present in gfxmenu, and while I can probably get away without all of
> them I don't think I'll be able to strip it all the way down to the
> point of a single boot menu.
> 
> Nevertheless, I think this is worth pursuing for various reasons:
> we're probably going to need to use GRUB to support EFI-based
> machines anyway; maintaining gfxboot-theme-ubuntu is really no fun at
> all; and I'd like to see GRUB get to the point where it can handle
> these requirements easily.  Judging from the images on the front page
> of http://grub.gibibit.com/, I gather that this was at least
> considered as a goal for gfxmenu!

It certainly was one of the goals.  In the past, boot loaders have been
second-class citizens from an appearance and usability standpoint, but
I would like to see that change.  GRUB should eventually have mouse
support as well, but at the very least it should have a flexible and
beautiful graphical interface.  I think that the rEFIt menu
<http://refit.sourceforge.net/screen.html> is nice looking and GRUB
should be able to provide such a simple and clean interface.

> I've written up a brief description of our needs here:
> 
>   https://wiki.ubuntu.com/FoundationsTeam/Specs/MaverickCDBoot
> 
> I think that the extensions we'd need in gfxmenu amount to:
> 
>   * Supplementary menu support, with menus bound to keys

How do you envision these supplementary menus?  Are they pop-up menus
spatially separate from the main boot menu?

>    * Either scrolling menus or grid menus, preferably grid

Would this grid be a grid of boot menu entries, like the rEFIt screen
shot I linked to above?

>   * The ability to bind functions to menus to process selections

Currently menu items each run the code defined by the GRUB 'menuitem'
command.  Would you want something different to happen that can't be
implemented this way?

>   * Checkbox widgets

Should checkbox widgets be able to be part of a menu?  How should
checkboxes by selected/deselected?  By navigating to them and pressing
Enter, or via a dedicated hotkey?

Currently the "boot menu" is itself a widget that displays menu items,
but menu items are not widgets themselves. If you want arbitrary
widgets to be selectable with the arrow keys we should figure out how
best to handle it.

> Needless to say I'm willing to take on the bulk of the work here, but
> I'm very new to gfxmenu and I would greatly appreciate review from
> those who've spent more time thinking about it and hacking on it.  In
> particular, I'd appreciate thoughts on how this kind of thing might
> best be represented in extensions to the theme format.

There are different possible approaches to setting up special menus.
One possibility is to create the various menus/menu items in grub.cfg
and then simply have the theme set up the menu widget which will then
use the menu data defined in grub.cfg to create/navigate the menu tree.

Another possibility is to explicitly instantiate widgets in the theme
for specific purposes.  For instance, you could create checkbox widgets
and somehow associate each one with a GRUB environment variable name,
which would be set according to the checkbox state.

The language chooser could be implemented as a special widget, or
perhaps a better option would be to create a generic "choice" widget
which gets its list of possible choices from a file included by
grub.cfg and sets a GRUB environment variable when the selected value
is changed.  (Note that environment variables can then have a "hook"
defined to take some action when the value is changed.)

How could GRUB script code define option names/values for a choice
box?  The issue is that you normally want some internal value for
each choice and a label to display to the user for the choice.  It
seems kind of a kludge but if you had a choice box given the name
"language" then it might get its possible choices from an environment
variable called "language_options", defined in grub.cfg (or more likely
in a file included in grub.cfg) something like

set language_options="en English
de Deutsch 
fr Francais"

and so on.  It would be nice if we could avoid using magic text
characters like space and newline but since GRUB script doesn't have
good data structures, it's simplest to use plain text, unless I'm
missing something.  (Lua makes defining and using such data structures
fantastically easy...)

Another thought:  I think it would be really fantastic to do some of the
more complex widget work in Lua.  I have created some test widgets that
are written 100% in Lua and do animation, etc.  It makes it much easier
to do things like manipulating lists, and creating widgets with
customized behavior.

Regards,
Colin

Re: Converting Ubuntu CDs to GRUB

[Colin Watson]

On Thu, May 20, 2010 at 11:13:02AM -0700, Colin D Bennett wrote:
> On Thu, 20 May 2010 18:35:54 +0100
> Colin Watson <cjwatson@ubuntu.com> wrote:
> > I've written up a brief description of our needs here:
> > 
> >   https://wiki.ubuntu.com/FoundationsTeam/Specs/MaverickCDBoot
> > 
> > I think that the extensions we'd need in gfxmenu amount to:
> > 
> >   * Supplementary menu support, with menus bound to keys
> 
> How do you envision these supplementary menus?  Are they pop-up menus
> spatially separate from the main boot menu?

Yes - e.g. the function-key menus across the bottom in your
http://grub.gibibit.com/Theme_ubuntu1_menu.jpg (which I notice are just
stubs in the theme right now).

> >    * Either scrolling menus or grid menus, preferably grid
> 
> Would this grid be a grid of boot menu entries, like the rEFIt screen
> shot I linked to above?

No, I'm thinking of a menu offering the ability to select the user's
language.  The list of languages will be too long to fit vertically on
the screen; in gfxboot, rendering that list as a pop-up grid menu has
worked well for us.

> >   * The ability to bind functions to menus to process selections
> 
> Currently menu items each run the code defined by the GRUB 'menuitem'
> command.  Would you want something different to happen that can't be
> implemented this way?

That's for main boot menu items.  I wouldn't expect this would be right
for secondary pop-up menus.  For something like language selection, I
don't think you'd want to write a menuentry stanza for each one;
instead, you'd want a single function that's called with the menu
selection as an argument, and it wouldn't implicitly terminate by
booting a kernel.

(I get the sense I may have skipped a few steps in my explanation of
what I'm trying to achieve.  Let me know if you want me to back up.)

> >   * Checkbox widgets
> 
> Should checkbox widgets be able to be part of a menu?  How should
> checkboxes by selected/deselected?  By navigating to them and pressing
> Enter, or via a dedicated hotkey?

I'm thinking of something like:

 /-----------------\
 | [x] Expert mode |
 | [ ] acpi=off    |
 | [x] noapic      |
 \-----------------/
  F6 Other Options

In gfxboot, I used Space or Enter to toggle a checkbox and Escape to
dismiss the menu.  I'm not sure this is ideal.

> > Needless to say I'm willing to take on the bulk of the work here, but
> > I'm very new to gfxmenu and I would greatly appreciate review from
> > those who've spent more time thinking about it and hacking on it.  In
> > particular, I'd appreciate thoughts on how this kind of thing might
> > best be represented in extensions to the theme format.
> 
> There are different possible approaches to setting up special menus.
> One possibility is to create the various menus/menu items in grub.cfg
> and then simply have the theme set up the menu widget which will then
> use the menu data defined in grub.cfg to create/navigate the menu tree.

I was thinking that grub.cfg should define the actions bound to menu
items, but that the theme should declare the menus themselves.

> Another possibility is to explicitly instantiate widgets in the theme
> for specific purposes.  For instance, you could create checkbox widgets
> and somehow associate each one with a GRUB environment variable name,
> which would be set according to the checkbox state.

In the case I have in mind, I'd want to be able to aggregate the whole
lot into a single variable because the end goal is to substitute them
into the kernel command line.  That was one reason I was thinking of a
function: it could be given the list of internal values corresponding to
checked boxes, and concatenate them into a single variable.  (Or
something similar with a single environment variable hook for all the
checkboxes in a menu at once would work too.)

> The language chooser could be implemented as a special widget, or
> perhaps a better option would be to create a generic "choice" widget
> which gets its list of possible choices from a file included by
> grub.cfg and sets a GRUB environment variable when the selected value
> is changed.  (Note that environment variables can then have a "hook"
> defined to take some action when the value is changed.)

I'd forgotten about environment variable hooks.  That does seem like a
nice way to do things, yes - it would save having to have a new binding
mechanism.

> How could GRUB script code define option names/values for a choice
> box?  The issue is that you normally want some internal value for
> each choice and a label to display to the user for the choice.  It
> seems kind of a kludge but if you had a choice box given the name
> "language" then it might get its possible choices from an environment
> variable called "language_options", defined in grub.cfg (or more likely
> in a file included in grub.cfg) something like
> 
> set language_options="en English
> de Deutsch 
> fr Francais"
> 
> and so on.  It would be nice if we could avoid using magic text
> characters like space and newline but since GRUB script doesn't have
> good data structures, it's simplest to use plain text, unless I'm
> missing something.  (Lua makes defining and using such data structures
> fantastically easy...)

Choices often need to be translated, and in this context we'll need to
be able to change the language at run-time: so the easy way to handle
internal vs. human-readable choices (which I agree we need) is to regard
English as just another translation, and then handle the whole thing via
gettext.  Then language_options or whatever can just be space-separated.

(Although the language_options choices always need to be displayed in
the language corresponding to each choice, of course, so I suppose there
would only be one "translation" there.  You have this right above, but
the way I sometimes explain this to Anglophones who haven't thought
about the problem is to ask them whether they'd recognise "英语" if they
saw it in a menu ...)

> Another thought:  I think it would be really fantastic to do some of the
> more complex widget work in Lua.  I have created some test widgets that
> are written 100% in Lua and do animation, etc.  It makes it much easier
> to do things like manipulating lists, and creating widgets with
> customized behavior.

I'm not opposed to Lua, but have no experience with it in GRUB, and I'm
conscious that it's not enabled in the Debian package right now and so
I'd sort of be reversing a decision made by Robert and Felix in their
absence, which niggles at me.  If it really makes things very much
easier, I wouldn't mind using it for this; if possible I would prefer
not to design in a way that actually requires it, though?

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Converting Ubuntu CDs to GRUB

[Michal Suchanek]

On 20 May 2010 20:43, Colin Watson <cjwatson@ubuntu.com> wrote:
> On Thu, May 20, 2010 at 11:13:02AM -0700, Colin D Bennett wrote:
>> On Thu, 20 May 2010 18:35:54 +0100
>> Colin Watson <cjwatson@ubuntu.com> wrote:
>> > I've written up a brief description of our needs here:
>> >
>> >   https://wiki.ubuntu.com/FoundationsTeam/Specs/MaverickCDBoot
>> >
>> > I think that the extensions we'd need in gfxmenu amount to:
>> >
>> >   * Supplementary menu support, with menus bound to keys
>>
>> How do you envision these supplementary menus?  Are they pop-up menus
>> spatially separate from the main boot menu?
>
> Yes - e.g. the function-key menus across the bottom in your
> http://grub.gibibit.com/Theme_ubuntu1_menu.jpg (which I notice are just
> stubs in the theme right now).
>

IMHO there should be a possiblility to bind actions (= scripts) with
descriptions to keys, and a possibility to print this list of actions
somewhere on-screen, automatically. I mean when somebody adds a custom
keybinding it would automatically appear in the list of keybindings
because it would be automatically generated from the actual grub
state, not manually drawn in Gimp.

>> >    * Either scrolling menus or grid menus, preferably grid
>>
>> Would this grid be a grid of boot menu entries, like the rEFIt screen
>> shot I linked to above?
>
> No, I'm thinking of a menu offering the ability to select the user's
> language.  The list of languages will be too long to fit vertically on
> the screen; in gfxboot, rendering that list as a pop-up grid menu has
> worked well for us.

I guess a scrolling menu is probably easier and can be the same as the
kernel menu and just about any other menu the user already knows. Do
all the languages available fit on the screen? Is this scalable to all
world languages?

A scrollable menu with PgUp/PgDn which allows skimming a page of
language names at a time seems easier to me, both the implemenataion
and the usability. Skimming through a table quickly is not that easy.
If the number of languages is really large you could split them by
continent similar to what Debian doses with location/timezone in their
installer.

>
>> >   * The ability to bind functions to menus to process selections
>>
>> Currently menu items each run the code defined by the GRUB 'menuitem'
>> command.  Would you want something different to happen that can't be
>> implemented this way?
>
> That's for main boot menu items.  I wouldn't expect this would be right
> for secondary pop-up menus.  For something like language selection, I
> don't think you'd want to write a menuentry stanza for each one;
> instead, you'd want a single function that's called with the menu
> selection as an argument, and it wouldn't implicitly terminate by
> booting a kernel.

Yes, the implicit termination is a problem. That could be configurable
per menu if we had multiple menus I guess. Otherwise I don't see why
the menu items for languages cannot be autogenerated by a script. A
specialized selection would save some config size, though.

>
> (I get the sense I may have skipped a few steps in my explanation of
> what I'm trying to achieve.  Let me know if you want me to back up.)
>
>> >   * Checkbox widgets
>>
>> Should checkbox widgets be able to be part of a menu?  How should
>> checkboxes by selected/deselected?  By navigating to them and pressing
>> Enter, or via a dedicated hotkey?
>
> I'm thinking of something like:
>
>  /-----------------\
>  | [x] Expert mode |
>  | [ ] acpi=off    |
>  | [x] noapic      |
>  \-----------------/
>  F6 Other Options
>
> In gfxboot, I used Space or Enter to toggle a checkbox and Escape to
> dismiss the menu.  I'm not sure this is ideal.

This is probably the expected way. Ideally it should be customizable
as should be all the keybindings so that people with odd input devices
can still use grub with proper configuraion.

While a specialized radio button would be nice they can be also
scripted as generic items so long as the menu is sufficiently
configurable from insisde grub I guess.

>
>> > Needless to say I'm willing to take on the bulk of the work here, but
>> > I'm very new to gfxmenu and I would greatly appreciate review from
>> > those who've spent more time thinking about it and hacking on it.  In
>> > particular, I'd appreciate thoughts on how this kind of thing might
>> > best be represented in extensions to the theme format.
>>
>> There are different possible approaches to setting up special menus.
>> One possibility is to create the various menus/menu items in grub.cfg
>> and then simply have the theme set up the menu widget which will then
>> use the menu data defined in grub.cfg to create/navigate the menu tree.
>
> I was thinking that grub.cfg should define the actions bound to menu
> items, but that the theme should declare the menus themselves.

You can't easily have manu items without menus, especially if you want
to split them into multiple separate menus.

I would think that a menu should be a list of items ( currently we
have an implicit menu which implicitly gets all items defined with
menuitem).

Also the menus should be included in some layout which specifies the
help texts and such, drawing the help texts in Gimp is not quite
optimal.
It should be possible to switch between multiple layouts because there
is no way how you could fit all installer menus in a single layout.

I find a stack of layouts quite appealing as when the user enters some
submenu (like select language or select extra boot options) and later
leaves it the previous menu will get displayed automatically because
it was "below" the submenu.

>
>> Another possibility is to explicitly instantiate widgets in the theme
>> for specific purposes.  For instance, you could create checkbox widgets
>> and somehow associate each one with a GRUB environment variable name,
>> which would be set according to the checkbox state.
>
> In the case I have in mind, I'd want to be able to aggregate the whole
> lot into a single variable because the end goal is to substitute them
> into the kernel command line.  That was one reason I was thinking of a
> function: it could be given the list of internal values corresponding to
> checked boxes, and concatenate them into a single variable.  (Or
> something similar with a single environment variable hook for all the
> checkboxes in a menu at once would work too.)

I don't think that something of this complexity can be easily solved
with a specialized checkbox widget. You should probably create an item
with associated script which takes care of integrating the state of
the option into the kernel parameters.

The item in question could possibly change appearance on its own and
the associated script would just check its state and act accordingly.

> Choices often need to be translated, and in this context we'll need to
> be able to change the language at run-time: so the easy way to handle
> internal vs. human-readable choices (which I agree we need) is to regard
> English as just another translation, and then handle the whole thing via
> gettext.  Then language_options or whatever can just be space-separated.

Handling translations is somewhat problematic at this point and I have
no idea how it would be done except by basically rebuilding the menu
each time it is translated. Perhaps as a first iteration you could
just define the menus in the different languages and switch to a
different menu when the language changes.

Also gettext is not really the way to translate things. It is based on
taking an arbitrary English string (or a number correstponding to such
string) and looking up a corresponding string in a different language.
The problem is that while English has singular and plural some
languages have more than that. It's similar for English ordianals,
there are more than two forms - 1st, 2nd, 3rd, 4th .. And there are
other issues of stuff not mapping from English to other languages
nicely.

It should work for whole menu items and help texts but may not for
printf format strings and such.

>
> (Although the language_options choices always need to be displayed in
> the language corresponding to each choice, of course, so I suppose there
> would only be one "translation" there.  You have this right above, but
> the way I sometimes explain this to Anglophones who haven't thought
> about the problem is to ask them whether they'd recognise "英语" if they
> saw it in a menu ...)

Yes, the first character looks familiar. I have not encountered the
second one when trying to search for readable translation in Asian
programs, it's usually something like 英語 or 英文 but I am not an expert.

>
>> Another thought:  I think it would be really fantastic to do some of the
>> more complex widget work in Lua.  I have created some test widgets that
>> are written 100% in Lua and do animation, etc.  It makes it much easier
>> to do things like manipulating lists, and creating widgets with
>> customized behavior.
>
> I'm not opposed to Lua, but have no experience with it in GRUB, and I'm
> conscious that it's not enabled in the Debian package right now and so
> I'd sort of be reversing a decision made by Robert and Felix in their
> absence, which niggles at me.  If it really makes things very much
> easier, I wouldn't mind using it for this; if possible I would prefer
> not to design in a way that actually requires it, though?
>

Lua is not going to be the official scripting language for Grub due to
copyright reasons and proabaly also due to maintainers not really
liking the language. I have no idea if or when there would be a
replacement. That does not mean you cannot enable it in your packages
and use it for the more advanced scripting. It would make things quite
a bit easier I imagine as the grub shell scripting is modelled after
Bourne shell scripting and comes with all the shortcoming this has for
handling data structures more complex than scalars or strings.

Thanks

Michal

Re: Converting Ubuntu CDs to GRUB

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2936 bytes --]

On 05/20/2010 07:35 PM, Colin Watson wrote:
> I would like to convert Ubuntu CDs to boot using GRUB for Ubuntu 10.10.
>
>   
I'll gladly support this in any way I can.
> This is a bit ambitious.  We're currently using ISOLINUX with SuSE's
> gfxboot extensions, and extensive theming to offer various custom menus.
> A number of the facilities we depend on there aren't yet present in
> gfxmenu, and while I can probably get away without all of them I don't
> think I'll be able to strip it all the way down to the point of a single
> boot menu.
>
> Nevertheless, I think this is worth pursuing for various reasons: we're
> probably going to need to use GRUB to support EFI-based machines anyway;
> maintaining gfxboot-theme-ubuntu is really no fun at all; and I'd like
> to see GRUB get to the point where it can handle these requirements
> easily.  Judging from the images on the front page of
> http://grub.gibibit.com/, I gather that this was at least considered as
> a goal for gfxmenu!
>
> I've written up a brief description of our needs here:
>
>   https://wiki.ubuntu.com/FoundationsTeam/Specs/MaverickCDBoot
>
> I think that the extensions we'd need in gfxmenu amount to:
>
>   * Supplementary menu support, with menus bound to keys
>   
As we discussed on IRC probably the best way to support it is to have
active and selectable buttonson the bottom of the screen which have a
--hotkey.
Buttons left and right can be reassign to whatever makes sense for
graphical menu.
Since up and down keys are shared between text and graphical menu one
should order the entries in such a way that navigating up and down also
make sense in gfxmenu.
For implementing separate selectable entries (buttons) you need to add
support for vertical menu and to have the ability to hide/show classes
at separate menu. E.g.

vertical-list
{
    class = "buttons,help"
    ...
}

list
{
   class = "remaining"
   ...
}

"remaining" is a magic word which means "everything not matching
anything else".
>    * Either scrolling menus or grid menus, preferably grid
>   
I'm ok with both of them.
>   * The ability to bind functions to menus to process selections
>   
What do you mean? We execute the code in menuentry body on enter. Or do
you mean anything else?
>   * Checkbox widgets
>   
I think it should be sth like in grub.cfg

checkbox "Safe mode" {
   set safe=1
}

Then it will be rendered ncurses-like in text and with a checkbox in
gfxmenu.
> Needless to say I'm willing to take on the bulk of the work here, but
> I'm very new to gfxmenu and I would greatly appreciate review from those
> who've spent more time thinking about it and hacking on it.  In
> particular, I'd appreciate thoughts on how this kind of thing might best
> be represented in extensions to the theme format.
>
> Thanks,
>
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]