grub.cfg and core.img getting out of sync

grub.cfg and core.img getting out of sync

[Colin Watson]

I'm having persistent problems with grub.cfg and core.img getting out of
sync. The usual pattern is:

  * Some shiny new feature appears in core.img
  * We extend grub-mkconfig to use it
  * User runs update-grub but not grub-install (historically a perfectly
    sane thing to do, and indeed basically standard practice)
  * grub either falls over at boot or does something odd

This just bit me again today, and if it bites me it's going to bite the
users of the packages I upload too.

Is there any way we can do better? One thing I was thinking of is that
Ubuntu's carried a patch to GRUB Legacy for some time that stores the
package version at the point when grub-install was last run in
/boot/grub/installed-version. If we did something like that, then
grub-mkconfig could carry a bit of conditional code that says "only use
this feature if the installed version of GRUB is at least <foo>".
grub-mkconfig would end up carrying a bit of bloat, but at least the
bloat is in /usr, and I'm assuming that at this point things are stable
enough that we won't be talking about *too* many changes.

What do people think about this general approach?

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: grub.cfg and core.img getting out of sync

[Pavel Roskin]

On Wed, 2009-08-12 at 16:25 +0100, Colin Watson wrote:
> I'm having persistent problems with grub.cfg and core.img getting out of
> sync. The usual pattern is:
> 
>   * Some shiny new feature appears in core.img
>   * We extend grub-mkconfig to use it
>   * User runs update-grub but not grub-install (historically a perfectly
>     sane thing to do, and indeed basically standard practice)
>   * grub either falls over at boot or does something odd

The assumption is that grub.cfg should be able to deal with it.  That's
why it sets root before searching.

> This just bit me again today, and if it bites me it's going to bite the
> users of the packages I upload too.

By our today's standards, it's a bug, so it should be reported in a
detailed way.

> Is there any way we can do better? One thing I was thinking of is that
> Ubuntu's carried a patch to GRUB Legacy for some time that stores the
> package version at the point when grub-install was last run in
> /boot/grub/installed-version. If we did something like that, then
> grub-mkconfig could carry a bit of conditional code that says "only use
> this feature if the installed version of GRUB is at least <foo>".
> grub-mkconfig would end up carrying a bit of bloat, but at least the
> bloat is in /usr, and I'm assuming that at this point things are stable
> enough that we won't be talking about *too* many changes.
> 
> What do people think about this general approach?

There is already a version hardcoded into the uncompressed part of
core.img.  See kern/i386/pc/startup.S.  Unfortunately, there is no
policy when the version should be updated.

But I'm fine with a separate file in /boot/grub if we decide that the
existing version in core.img cannot be used for that and we cannot keep
grub.cfg backward compatible.

-- 
Regards,
Pavel Roskin

Re: grub.cfg and core.img getting out of sync

[Colin Watson]

On Thu, Aug 13, 2009 at 02:54:29AM -0400, Pavel Roskin wrote:
> On Wed, 2009-08-12 at 16:25 +0100, Colin Watson wrote:
> > I'm having persistent problems with grub.cfg and core.img getting out of
> > sync. The usual pattern is:
> > 
> >   * Some shiny new feature appears in core.img
> >   * We extend grub-mkconfig to use it
> >   * User runs update-grub but not grub-install (historically a perfectly
> >     sane thing to do, and indeed basically standard practice)
> >   * grub either falls over at boot or does something odd
> 
> The assumption is that grub.cfg should be able to deal with it.  That's
> why it sets root before searching.

Hmm, it's sometimes tricky. For example:

> > This just bit me again today, and if it bites me it's going to bite the
> > users of the packages I upload too.
> 
> By our today's standards, it's a bug, so it should be reported in a
> detailed way.

Well, this was in code I was writing. See the hiddenmenu and keystatus
patches I posted recently; with those, it is reasonable to do:

  if sleep --interruptible 0 ; then
    set timeout=0
  fi

(In other words, unless the Shift key is pressed when GRUB starts, boot
the default OS immediately.)

However, the effect of this if the keystatus patch is not applied (or
indeed if you're on a target with no support for keystatus) will be to
boot the default OS with no way to get at the menu. I'm open to ideas on
how to do this better; does it need something more complex than 'sleep
--interruptible 0'?

> > Is there any way we can do better? One thing I was thinking of is that
> > Ubuntu's carried a patch to GRUB Legacy for some time that stores the
> > package version at the point when grub-install was last run in
> > /boot/grub/installed-version. If we did something like that, then
> > grub-mkconfig could carry a bit of conditional code that says "only use
> > this feature if the installed version of GRUB is at least <foo>".
> > grub-mkconfig would end up carrying a bit of bloat, but at least the
> > bloat is in /usr, and I'm assuming that at this point things are stable
> > enough that we won't be talking about *too* many changes.
> > 
> > What do people think about this general approach?
> 
> There is already a version hardcoded into the uncompressed part of
> core.img.  See kern/i386/pc/startup.S.  Unfortunately, there is no
> policy when the version should be updated.
> 
> But I'm fine with a separate file in /boot/grub if we decide that the
> existing version in core.img cannot be used for that and we cannot keep
> grub.cfg backward compatible.

The version in core.img would be fine if there were a way to extract it
from userspace. It seems rather ugly to me to extract this using dd or
od directly - also, presumably grub-mkconfig ought to be at least
somewhat architecture-independent. Does this need a new tool?

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: grub.cfg and core.img getting out of sync

[Robert Millan]

On Thu, Aug 13, 2009 at 09:22:14AM +0100, Colin Watson wrote:
> > > 
> > > What do people think about this general approach?
> > 
> > There is already a version hardcoded into the uncompressed part of
> > core.img.  See kern/i386/pc/startup.S.  Unfortunately, there is no
> > policy when the version should be updated.
> > 
> > But I'm fine with a separate file in /boot/grub if we decide that the
> > existing version in core.img cannot be used for that and we cannot keep
> > grub.cfg backward compatible.
> 
> The version in core.img would be fine if there were a way to extract it
> from userspace. It seems rather ugly to me to extract this using dd or
> od directly - also, presumably grub-mkconfig ought to be at least
> somewhat architecture-independent. Does this need a new tool?

This seems overengineered.  I'd rather integrate grub-install with
grub-mkconfig to ensure they're always run together, than have to deal
with a version number that has to be constantly upgraded.

Maintaining an ABI version number is a significant burden in the long
run.  We should think carefully before committing to it.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub.cfg and core.img getting out of sync

[Robert Millan]

On Wed, Aug 12, 2009 at 04:25:09PM +0100, Colin Watson wrote:
> I'm having persistent problems with grub.cfg and core.img getting out of
> sync. The usual pattern is:
> 
>   * Some shiny new feature appears in core.img
>   * We extend grub-mkconfig to use it
>   * User runs update-grub but not grub-install (historically a perfectly
>     sane thing to do, and indeed basically standard practice)
>   * grub either falls over at boot or does something odd
> 
> This just bit me again today, and if it bites me it's going to bite the
> users of the packages I upload too.

Generally, we make grub-mkconfig generate code that is backward compatible.
Can you give more details on why it wasn't this time?

Also, it's probably a good idea to always run grub-install at the same time
as update-grub.  Latest version of the Debian package includes a debconf
template to run grub-install every time the package is upgraded.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."