Beware using modules from different grub versions (was error: cannot allocate real mode pages)

Beware using modules from different grub versions (was error: cannot allocate real mode pages)

[Glenn Washburn]

[-- Attachment #1: Type: text/plain, Size: 2571 bytes --]

I originally started writing this to request help from the grub-help
list in solving this problem that has been plaguing me for days.  I just
solved the issue and thought others might be able to benefit.  And
perhaps this issue indicates a need for module versioning as the linux
kernel has.  I'm sending to the devel list, because I think much is
related to development.

I'd been having some problems trying to get grub to boot my
x86_64 ubuntu server installation lately.  I've attached some
screenshots of relevant grub output with some debugging on.  I'm using
grub compiled from HEAD and tried both the x86_64 and i386 platforms
for the "pc" target. Actually, I'm curious to know if there's actually
a difference in those platforms.

It turns out this error was a result of loading the linux module (and
many of its dependants, ie mmap) from a different (older) grub
install.  I presume this is not infact because of a bug (otherwise the
original install wouldn't have booted), rather the fact that the
versions are different.

One might suggest chainloading the ubuntu grub installation instead of
using "configfile" to just load its config.  This won't work for me (in
general) because the other grub install doesn't necessarily have the
capability to boot the system (ie doesn't have cryptmount/lvm/raid
support).

Should grub implement some kind of module versioning to eliminate this
issue?  Ideally, this seems the right thing.  However, I'm not sure if
versioning as restrictive as limiting loading to modules of the same
grub version.  In my experience, loading modules from other grub
versions hasn't caused much issue, so this could cause a different pain
for people using this "feature".

Regardless, there's another issue, the difficulty in ensuring where
modules are being loaded from.  Currently, modules are searched for in
the $prefix directory.  But $prefix is used for many other things as
well, like config loading and font loading.  So if you want the fonts
coming from the ubuntu install, but the modules coming from a newer
grub, you have problems.

I see two potential, non-mutually exclusive solutions.  One, have
another $moduledir variable for dividing up the multi-use $prefix
($prefix would be used as a fallback). And two, have $prefix (and
potentially $moduledir) be a list of paths, much like the bash $PATH
variable.  The first suggestion would be almost trivial to implement,
while the second for $prefix will be a pain because potentially all
uses of $prefix would need to be modified.

Comments and suggestions appreciated.
Glenn

[-- Attachment #2: grub-server-alloc-real-mode-pages.png --]
[-- Type: image/x-apple-ios-png, Size: 64255 bytes --]

[-- Attachment #3: grub-server-alloc-real-mode-pages-2.png --]
[-- Type: image/x-apple-ios-png, Size: 55444 bytes --]

Re: Beware using modules from different grub versions (was error: cannot allocate real mode pages)

[Andrey Borzenkov]

В Mon, 9 Sep 2013 21:27:31 -0500
Glenn Washburn <development@efficientek.com> пишет:

> I originally started writing this to request help from the grub-help
> list in solving this problem that has been plaguing me for days.  I just
> solved the issue and thought others might be able to benefit.  And
> perhaps this issue indicates a need for module versioning as the linux
> kernel has.  I'm sending to the devel list, because I think much is
> related to development.
> 
> I'd been having some problems trying to get grub to boot my
> x86_64 ubuntu server installation lately.  I've attached some
> screenshots of relevant grub output with some debugging on.  I'm using
> grub compiled from HEAD and tried both the x86_64 and i386 platforms
> for the "pc" target. Actually, I'm curious to know if there's actually
> a difference in those platforms.
> 
> It turns out this error was a result of loading the linux module (and
> many of its dependants, ie mmap) from a different (older) grub
> install.  I presume this is not infact because of a bug (otherwise the
> original install wouldn't have booted), rather the fact that the
> versions are different.
> 
> One might suggest chainloading the ubuntu grub installation instead of
> using "configfile" to just load its config.  This won't work for me (in
> general) because the other grub install doesn't necessarily have the
> capability to boot the system (ie doesn't have cryptmount/lvm/raid
> support).
> 
> Should grub implement some kind of module versioning to eliminate this
> issue?  Ideally, this seems the right thing.  However, I'm not sure if
> versioning as restrictive as limiting loading to modules of the same
> grub version.  In my experience, loading modules from other grub
> versions hasn't caused much issue, so this could cause a different pain
> for people using this "feature".
> 
> Regardless, there's another issue, the difficulty in ensuring where
> modules are being loaded from.  Currently, modules are searched for in
> the $prefix directory.  But $prefix is used for many other things as
> well, like config loading and font loading.  So if you want the fonts
> coming from the ubuntu install, but the modules coming from a newer
> grub, you have problems.
> 

grub is using whatever font is loaded using loadfont command. So just
load it from your preferred location.

> I see two potential, non-mutually exclusive solutions. 

I'm afraid I miss the problem.

>                                                         One, have
> another $moduledir variable for dividing up the multi-use $prefix
> ($prefix would be used as a fallback). And two, have $prefix (and
> potentially $moduledir) be a list of paths, much like the bash $PATH
> variable.  The first suggestion would be almost trivial to implement,
> while the second for $prefix will be a pain because potentially all
> uses of $prefix would need to be modified.
> 
> Comments and suggestions appreciated.
> Glenn

Re: Beware using modules from different grub versions (was error: cannot allocate real mode pages)

[Glenn Washburn]

On Tue, 10 Sep 2013 06:59:18 +0400
Andrey Borzenkov <arvidjaar@gmail.com> wrote:

> > Regardless, there's another issue, the difficulty in ensuring where
> > modules are being loaded from.  Currently, modules are searched for
> > in the $prefix directory.  But $prefix is used for many other
> > things as well, like config loading and font loading.  So if you
> > want the fonts coming from the ubuntu install, but the modules
> > coming from a newer grub, you have problems.
> >   
> 
> grub is using whatever font is loaded using loadfont command. So just
> load it from your preferred location.

This may not be known until runtime or may not be desirable to
hard-code it into the config.  If you want to load a particular OS's
grub install, you might like to load its interface as if you had booted
from that grub install directly.  So let it load whatever font it
wants.  You shouldn't need to care about it.  This generally works now
by making sure the $prefix is correctly set before a "configfile"
command.

> > I see two potential, non-mutually exclusive solutions.   
> 
> I'm afraid I miss the problem.

The problem, as stated above, is that $prefix is used as a base
location for many types of grub files (fonts, modules, configs,
locales, etc..).  It may not be desirable to load all these from the
same prefix.  As I found out after banging my head against a wall for a
couple of days trying to figure out why grub was failing to load my
ubuntu install with the "error: cannot allocate real mode pages".  I
don't want the grub modules to load from the older ubuntu grub install,
however I do want $prefix to point to the ubuntu grub install location
so that locales, fonts, configs, etc.. are properly located.

I'm using multi-purpose USB stick (boot several different machines and
isos), which is installed with the repo HEAD to boot my ubuntu server
which has a non-bootable (inside an encrypted container) ubuntu
install, which has an older version of grub installed.

Does that help to clarify?