grub-probe fails during grub2 update (Debian)

grub-probe fails during grub2 update (Debian)

[Stefan Weil]

Robert Millan wrote:
> On Fri, Apr 18, 2008 at 09:45:52PM +0200, Stefan Weil wrote:
>> grub-probe fails like this:
>>
>> # grub-probe --target=drive --device /dev/sdb1
>> grub-probe: error: Cannot find a GRUB drive for /dev/sdb1.  Check your
>> device.map.
>>
>> # device.map is unchanged, see previous output
>
> Ah, right.  I'll see what you mean.  Maybe it'd be a good idea to start
> generating device.map dynamically; although this has other 
disadvantages...
>
> Could you bring this up in grub-devel@gnu.org?  I'd like to have it
> discussed in upstream.
>

Hi,

see below a test scenario of a grub2 update failure
which I had sent to Debian's bug tracking system
(http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=467127).

The problem is caused by missing or invalid entries for
removable media (CDROM, DVD, USB flash devices with valid OS)
in device.map.

Missing entries can be caused by insertion of a flash medium.

Invalid entries remain after an OS installation from a boot DVD.

Regards
Stefan Weil





Extract from http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=467127


I should mention a point missing in my last mails which maybe is important:
the removable medium must contain a partition with an operating system.

Here is my test scenario (see comments) with system output and error
messages.

# PC with SATA harddisk, USB card reader, no CF or SD card inserted.

# grub-mkdevicemap
# cat /boot/grub/device.map
(hd0)   /dev/sda


# USB card reader, CF card now inserted.
# The CF card provides an EXT3 partition /dev/sdb1 with DEBIAN Linux.
# Reinstall latest grub-pc (gives same error like upgrade from older
version).

# LANG=C apt-get --reinstall install grub-pc
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 5 not
upgraded.
Need to get 0B/1101kB of archives.
After this operation, 0B of additional disk space will be used.
Do you want to continue [Y/n]?
Preconfiguring packages ...
(Reading database ... 253220 files and directories currently installed.)
Preparing to replace grub-pc 1.96+20080413-1 (using
.../grub-pc_1.96+20080413-1_amd64.deb) ...
Unpacking replacement grub-pc ...
Setting up grub-pc (1.96+20080413-1) ...
Updating /boot/grub/grub.cfg ...
Found Debian background: debian-blueish-wallpaper-640x480.png
Found linux image: /boot/vmlinuz-2.6.24-1-amd64
Found initrd image: /boot/initrd.img-2.6.24-1-amd64
Found linux image: /boot/vmlinuz-2.6.22-3-amd64
Found initrd image: /boot/initrd.img-2.6.22-3-amd64
Found memtest86+ image: /boot/memtest86+.bin
Found openSUSE 10.3 (i586) on /dev/sda5
Found Debian GNU/Linux (lenny/sid) on /dev/sda8
Found Debian GNU/Linux (4.0) on /dev/sdb1
dpkg: error processing grub-pc (--configure):
subprocess post-installation script returned error exit status 1
Errors were encountered while processing:
grub-pc
E: Sub-process /usr/bin/dpkg returned an error code (1)



The failing command hierarchy is given here:

/var/lib/dpkg/info/grub-pc.postinst configure
-- /usr/sbin/update-grub
---- /etc/grub.d/30_os-prober
------ grub-probe --target=drive --device /dev/sdb1

grub-probe fails like this:

# grub-probe --target=drive --device /dev/sdb1
grub-probe: error: Cannot find a GRUB drive for /dev/sdb1.  Check your
device.map.

# device.map is unchanged, see previous output

Re: grub-probe fails during grub2 update (Debian)

[Robert Millan]

On Sat, Apr 19, 2008 at 04:34:27PM +0200, Stefan Weil wrote:
> >
> >Could you bring this up in grub-devel@gnu.org?  I'd like to have it
> >discussed in upstream.
> >
> 
> Hi,
> 
> see below a test scenario of a grub2 update failure
> which I had sent to Debian's bug tracking system
> (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=467127).

Thanks Stefan.

I propose that if grub-probe can't find an entry in device.map to convert the
device it just found to a grub drive, it runs grub-mkdevicemap and tries again.

Other approaches would be to kill device.map completely and just pipe the
information from grub-mkdevicemap directly, but I think that might be too
radical.

What does everyone think?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call… if you are unable to speak?
(as seen on /.)

Re: grub-probe fails during grub2 update (Debian)

[Pavel Roskin]

On Sun, 2008-04-20 at 12:07 +0200, Robert Millan wrote:

> I propose that if grub-probe can't find an entry in device.map to convert the
> device it just found to a grub drive, it runs grub-mkdevicemap and tries again.
> 
> Other approaches would be to kill device.map completely and just pipe the
> information from grub-mkdevicemap directly, but I think that might be too
> radical.
> 
> What does everyone think?

I would explore the possibility of retiring device.map completely or
limiting its use to some rare cases.

In the vast majority of cases, the installer should not rely on knowing
the BIOS numbers of the devices involved in the boot process.  BIOS
provides the boot drive, which should generally be trusted.  The
installer only needs to know whether the media is a hard drive to enable
a workaround for buggy BIOSes providing a wrong boot drive.

I'm not sure about EFI and Open firmware, but I think the situation is
similar.  There should be enough information to find the boot drive at
the boot time without requiring any guesswork in the installer.

We may need to tell GRUB what device to use to look for additional files
when the boot process involves more than one drive.  But even then, I'd
rather prefer that GRUB uses labels, not hardcoded BIOS numbers.

-- 
Regards,
Pavel Roskin

retiring device.map

[Robert Millan]

On Mon, Apr 21, 2008 at 06:31:06AM -0400, Pavel Roskin wrote:
> 
> I would explore the possibility of retiring device.map completely or
> limiting its use to some rare cases.
> 
> In the vast majority of cases, the installer should not rely on knowing
> the BIOS numbers of the devices involved in the boot process.  BIOS
> provides the boot drive, which should generally be trusted.  The
> installer only needs to know whether the media is a hard drive to enable
> a workaround for buggy BIOSes providing a wrong boot drive.
> 
> I'm not sure about EFI and Open firmware, but I think the situation is
> similar.  There should be enough information to find the boot drive at
> the boot time without requiring any guesswork in the installer.
> 
> We may need to tell GRUB what device to use to look for additional files
> when the boot process involves more than one drive.  But even then, I'd
> rather prefer that GRUB uses labels, not hardcoded BIOS numbers.

I completely agree.  How do we go about this?  First of all we need support
for labels, right?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call… if you are unable to speak?
(as seen on /.)

Re: retiring device.map

[Pavel Roskin]

On Tue, 2008-05-06 at 17:18 +0200, Robert Millan wrote:

> I completely agree.  How do we go about this?  First of all we need support
> for labels, right?

We have it already in the "search" command.  Of course, there is always
room for improvement.  GRUB doesn't recognize Linux swap labels.  It
cannot set root to, say, partition 3 on a disk where partition 1 has the
given label.

Anyway, the fix belongs to the installer.  The first step would be not
to use device.map is it's not needed.  In other words, if it's not a
cross-device install, don't read device.map and don't try to create it.

The second step is figuring out what to do with cross-device installs.
device.map serves as a persistent cache that can become out of date.
Perhaps it could be replaced with a table in memory that only contains
entries relevant to the install.  device.map was more justified in the
days when floppy drives were common.  But if there are still realistic
cases that take a lot of time (say, 10 seconds or more), we need to
address that issue somehow.

device.map is also used in grub-emu, but grub-emu could use a different
names based on the OS device names.

Finally, update-grub should be modified to look for partitions by
labels.  That will allow grub-emu use the same menu as the real
bootloader, while using different device labels.

-- 
Regards,
Pavel Roskin

Re: retiring device.map

[Robert Millan]

On Tue, May 06, 2008 at 11:43:48AM -0400, Pavel Roskin wrote:
> On Tue, 2008-05-06 at 17:18 +0200, Robert Millan wrote:
> 
> > I completely agree.  How do we go about this?  First of all we need support
> > for labels, right?
> 
> We have it already in the "search" command.  Of course, there is always
> room for improvement.  GRUB doesn't recognize Linux swap labels.  It
> cannot set root to, say, partition 3 on a disk where partition 1 has the
> given label.

Sorry, I was confusing labels with UUIDs.  The problem with labels is they're
not garanteed to be unique, right?  So maybe we need support for UUIDs..

> Anyway, the fix belongs to the installer.  The first step would be not
> to use device.map is it's not needed.  In other words, if it's not a
> cross-device install, don't read device.map and don't try to create it.

I think this isn't an easy as it looks.  grub-setup works much like grub-emu
in that it uses code from outside util/, which only understands devices when
represented in GRUB syntax.

Maybe we could force-feed it something like "(/dev/something)" as if it
were a GRUB drive, but then how would it know how are partitions represented
by the OS?  (%d, p%d, etc)

> Finally, update-grub should be modified to look for partitions by
> labels.

We can do this already, without changing how device.map works.  Though, we
need to reuse the same code in GRUB to obtain the labels (for robustness),
which would need some new option in grub-probe or so.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call… if you are unable to speak?
(as seen on /.)

Re: retiring device.map

[Pavel Roskin]

On Wed, 2008-05-07 at 15:03 +0200, Robert Millan wrote:
> On Tue, May 06, 2008 at 11:43:48AM -0400, Pavel Roskin wrote:
> > On Tue, 2008-05-06 at 17:18 +0200, Robert Millan wrote:
> > 
> > > I completely agree.  How do we go about this?  First of all we need support
> > > for labels, right?
> > 
> > We have it already in the "search" command.  Of course, there is always
> > room for improvement.  GRUB doesn't recognize Linux swap labels.  It
> > cannot set root to, say, partition 3 on a disk where partition 1 has the
> > given label.
> 
> Sorry, I was confusing labels with UUIDs.  The problem with labels is they're
> not garanteed to be unique, right?  So maybe we need support for UUIDs..

Right.

> > Anyway, the fix belongs to the installer.  The first step would be not
> > to use device.map is it's not needed.  In other words, if it's not a
> > cross-device install, don't read device.map and don't try to create it.
> 
> I think this isn't an easy as it looks.  grub-setup works much like grub-emu
> in that it uses code from outside util/, which only understands devices when
> represented in GRUB syntax.
> 
> Maybe we could force-feed it something like "(/dev/something)" as if it
> were a GRUB drive, but then how would it know how are partitions represented
> by the OS?  (%d, p%d, etc)

Yes, that would be consistent with our use of names having everything
needed for the access, without any mapping (like hd96 or ata2).

We could probably try all possible partition styles for the given OS
(numbers, letters, p + number).  If we only needed mounted partitions,
we could go through the list and check mounted devices for being
partitions of the drive.  Anyway, I'm afraid that part will remain
OS-specific for now.

> > Finally, update-grub should be modified to look for partitions by
> > labels.
> 
> We can do this already, without changing how device.map works.  Though, we
> need to reuse the same code in GRUB to obtain the labels (for robustness),
> which would need some new option in grub-probe or so.

Agreed.  And we could probably use UUIDs too (optionally), although
labels seem more readable to me.

-- 
Regards,
Pavel Roskin

Re: retiring device.map

[Bean]

On Thu, May 8, 2008 at 1:22 AM, Pavel Roskin <proski@gnu.org> wrote:
> On Wed, 2008-05-07 at 15:03 +0200, Robert Millan wrote:
>  > On Tue, May 06, 2008 at 11:43:48AM -0400, Pavel Roskin wrote:
>  > > On Tue, 2008-05-06 at 17:18 +0200, Robert Millan wrote:
>  > >
>  > > > I completely agree.  How do we go about this?  First of all we need support
>  > > > for labels, right?
>  > >
>  > > We have it already in the "search" command.  Of course, there is always
>  > > room for improvement.  GRUB doesn't recognize Linux swap labels.  It
>  > > cannot set root to, say, partition 3 on a disk where partition 1 has the
>  > > given label.
>  >
>  > Sorry, I was confusing labels with UUIDs.  The problem with labels is they're
>  > not garanteed to be unique, right?  So maybe we need support for UUIDs..
>
>  Right.

Yes, UUID can be useful, it's better to add it to grub2.

UUID itself is quite easy to get, but the question is how to integrate
it with the existing disk layer. I'm thinking of the following three
implementation:

1, Add a function pointer to fs structure to retrieve UUID, just like label.
2. Add a data pointer to the disk structure, the fs driver allocated
memory when it found UUID.
3. Use a separate module to handle UUID. fs driver that found UUID can
add disk to uuid mapping using exported service.

Would someone comment on the ideas ?

-- 
Bean

Re: retiring device.map

[Colin D Bennett]

[-- Attachment #1: Type: text/plain, Size: 1629 bytes --]

On Thu, 8 May 2008 01:39:47 +0800
Bean <bean123ch@gmail.com> wrote:

> On Thu, May 8, 2008 at 1:22 AM, Pavel Roskin <proski@gnu.org> wrote:
> > On Wed, 2008-05-07 at 15:03 +0200, Robert Millan wrote:
> >  > On Tue, May 06, 2008 at 11:43:48AM -0400, Pavel Roskin wrote:
> >  > > On Tue, 2008-05-06 at 17:18 +0200, Robert Millan wrote:
> >  > >
> >  > > > I completely agree.  How do we go about this?  First of all
> >  > > > we need support for labels, right?
> >  > >
> >  > > We have it already in the "search" command.  Of course, there
> >  > > is always room for improvement.  GRUB doesn't recognize Linux
> >  > > swap labels.  It cannot set root to, say, partition 3 on a
> >  > > disk where partition 1 has the given label.
> >  >
> >  > Sorry, I was confusing labels with UUIDs.  The problem with
> >  > labels is they're not garanteed to be unique, right?  So maybe
> >  > we need support for UUIDs..
> >
> >  Right.
> 
> Yes, UUID can be useful, it's better to add it to grub2.

I think that while labels (like "linux boot", "swap 2", etc.) are
useful and easy to use for fixed disk drives (and conveniently avoid
problems when drive order is changed!), they are inadequate for
removable drives such as USB flash drives or external hard drives,
where it is easy to run into name clashes--for instance, when swapping
external drives between machines, they might have the same name but have
slightly different purposes/content and need different treatment. In
some cases like this, the UUID seems a better choice than a label (copy
and paste is your friend, however).

Regards,
Colin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: retiring device.map

[Robert Millan]

On Thu, May 08, 2008 at 01:39:47AM +0800, Bean wrote:
> 
> Yes, UUID can be useful, it's better to add it to grub2.
> 
> UUID itself is quite easy to get, but the question is how to integrate
> it with the existing disk layer. I'm thinking of the following three
> implementation:
> 
> 1, Add a function pointer to fs structure to retrieve UUID, just like label.
> 2. Add a data pointer to the disk structure, the fs driver allocated
> memory when it found UUID.
> 3. Use a separate module to handle UUID. fs driver that found UUID can
> add disk to uuid mapping using exported service.
> 
> Would someone comment on the ideas ?

I'm not completely filled on the basics of UUID.  The location of this
information is defined by the filesystem implementation, right?

In that case, I'd suggest a function to fetch it from grub_fs rather than
a location where it's always copied to.  This allows filesystems that don't
implement it to leave a NULL in it, and doesn't force GRUB to process it
every time (saving time when loading filesystems where UUID isn't going to
be used).

Then search.mod could simply do:

  if (fs->get_uuid)
    my_uuid = fs->get_uuid ();

would that be enough?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call… if you are unable to speak?
(as seen on /.)