Next release?

Next release?

[Pavel Roskin]

Hello!

Now that we have lzma compression, we should probably consider making
another release in a week or two.

Before it happens, I'd like to add a configure test for working target
compiler, so that users of pure x86_64 machines are not surprised by a
message that "start" and "_start" are not found.

By the way, that check could be generalized to allow more symbols, such
as "Ltext0" on Cygwin.  It would be great if GRUB 1.97 compiled on
Cygwin out-of-box.  Cygwin lacked some lzo development files, but we
don't need them anymore.

All warnings for i386-pc have been fixed, but perhaps I'll look at other
targets if I have time.  Also, Valgrind finds some memory leaks, that we
may want to fix now.

I hope that once GRUB 1.97 is released, we'll eliminate device.map and
reorganize the build system to make it more flexible.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Christian Franke]

Pavel Roskin wrote:
> Hello!
>
> Now that we have lzma compression, we should probably consider making
> another release in a week or two.
>
> Before it happens, I'd like to add a configure test for working target
> compiler, so that users of pure x86_64 machines are not surprised by a
> message that "start" and "_start" are not found.
>
> By the way, that check could be generalized to allow more symbols, such
> as "Ltext0" on Cygwin.  It would be great if GRUB 1.97 compiled on
> Cygwin out-of-box.  Cygwin lacked some lzo development files, but we
> don't need them anymore.
>   

Thanks for considering Cygwin support for the next release.

The first (and last) grub package released in the Cygwin distribution 
was based on grub codebase from 2008-03-26. My latest reasonably tested 
merge is ~2 month old. If desired, I can merge & test all remaining 
changes to current HEAD and post the patches for review soon.

BTW:
- Supporting "Ltext0" would not help much for Cygwin, because a linker 
script is required due to the lack of "ld -N" support.
- Cygwin liblzo2* packages were released 1 week before the grub package :-)

Christian

Re: Next release?

[Pavel Roskin]

On Mon, 2008-07-14 at 18:55 +0200, Christian Franke wrote:

> Thanks for considering Cygwin support for the next release.
> 
> The first (and last) grub package released in the Cygwin distribution 
> was based on grub codebase from 2008-03-26. My latest reasonably tested 
> merge is ~2 month old. If desired, I can merge & test all remaining 
> changes to current HEAD and post the patches for review soon.

I'm sorry, I forgot that story.  It may be a more radical change than I
expected.  Anyway, let's see what remains to be done.

> BTW:
> - Supporting "Ltext0" would not help much for Cygwin, because a linker 
> script is required due to the lack of "ld -N" support.

We may want to use linker scripts on all platforms for uniformity.

> - Cygwin liblzo2* packages were released 1 week before the grub package :-)

That means the development package too?  That's good.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Christian Franke]

Pavel Roskin wrote:
> On Mon, 2008-07-14 at 18:55 +0200, Christian Franke wrote:
>
>   
>> Thanks for considering Cygwin support for the next release.
>>
>> The first (and last) grub package released in the Cygwin distribution 
>> was based on grub codebase from 2008-03-26. My latest reasonably tested 
>> merge is ~2 month old. If desired, I can merge & test all remaining 
>> changes to current HEAD and post the patches for review soon.
>>     
>
> I'm sorry, I forgot that story.  It may be a more radical change than I
> expected.  Anyway, let's see what remains to be done.
>
>   

Here is a older diffstat:
http://lists.gnu.org/archive/html/grub-devel/2008-04/msg00013.html

The changes for util/biosdisk.c, util/getroot.c, and util/mkdevicemap.c 
are already checked in.


>> BTW:
>> - Supporting "Ltext0" would not help much for Cygwin, because a linker 
>> script is required due to the lack of "ld -N" support.
>>     
>
> We may want to use linker scripts on all platforms for uniformity.
>
>   
Yes, more flexible. And the tests for _start etc. are no longer necessary.


>> - Cygwin liblzo2* packages were released 1 week before the grub package :-)
>>     
>
> That means the development package too?  That's good.
>
>   

Yes, liblzo2-devel too.

Christian

Re: Next release?

[Robert Millan]

On Mon, Jul 14, 2008 at 06:55:24PM +0200, Christian Franke wrote:
> 
> The first (and last) grub package released in the Cygwin distribution 
> was based on grub codebase from 2008-03-26. My latest reasonably tested 
> merge is ~2 month old. If desired, I can merge & test all remaining 
> changes to current HEAD and post the patches for review soon.

Great!

As for the loader issue, did you find a way to generate images in ELF format
from the Cygwin system?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Next release?

[Patrick Georgi]

Robert Millan schrieb:
> On Mon, Jul 14, 2008 at 06:55:24PM +0200, Christian Franke wrote:
>> The first (and last) grub package released in the Cygwin distribution 
>> was based on grub codebase from 2008-03-26. My latest reasonably tested 
>> merge is ~2 month old. If desired, I can merge & test all remaining 
>> changes to current HEAD and post the patches for review soon.
> 
> Great!
> 
> As for the loader issue, did you find a way to generate images in ELF format
> from the Cygwin system?
I do all the time with an i386-elf cross compiler. the easiest way to 
generate one (in my experience) is to create a directory with symlinks to:
- gcc/*
- binutils/* (ignore duplicates)
- gcc/include/* (into include/, after removing the symlink to 
gcc/include and creating a directory in its place)
- binutils/include/* (again, ignore duplicates)

then configure --enable-languages=c --disable-bootstrap from that 
directory of symlinks, and you'll get a full binutils+gcc build (if you 
want gdb, it should be enough to create symlinks to its files, but I 
didn't test that)

at least for coreboot, binutils must be of version 2.18.50.* (eg. as 
found with mingw), gnu's 2.18 isn't enough. no idea if that applies to 
grub2 or not.

as for the start/_start tests, I merely let configure default to _start 
if no other symbol was found, which might break grub-emu, but builds the 
target system grub just fine here. with i386-elf, the target compiler 
has no crt*.o anyway.


Regards,
Patrick Georgi

Re: Next release?

[Christian Franke]

Robert Millan wrote:
> On Mon, Jul 14, 2008 at 06:55:24PM +0200, Christian Franke wrote:
>   
>> The first (and last) grub package released in the Cygwin distribution 
>> was based on grub codebase from 2008-03-26. My latest reasonably tested 
>> merge is ~2 month old. If desired, I can merge & test all remaining 
>> changes to current HEAD and post the patches for review soon.
>>     
>
> Great!
>
> As for the loader issue, did you find a way to generate images in ELF format
> from the Cygwin system?
>
>   

Yes of course - the Cygwin grub package compiles OOTB and works :-)

objcopy is used to convert PE to ELF:
http://lists.gnu.org/archive/html/grub-devel/2007-11/msg00238.html

Due to issues in BFD, objcopy produces ELF with invalid pc-relative 
relocation. This is compensated by a hack in the loader:
http://lists.gnu.org/archive/html/grub-devel/2007-11/msg00232.html

Christian

Re: Next release?

[Christian Franke]

Patrick Georgi wrote:
> Robert Millan schrieb:
>> On Mon, Jul 14, 2008 at 06:55:24PM +0200, Christian Franke wrote:
>>> The first (and last) grub package released in the Cygwin 
>>> distribution was based on grub codebase from 2008-03-26. My latest 
>>> reasonably tested merge is ~2 month old. If desired, I can merge & 
>>> test all remaining changes to current HEAD and post the patches for 
>>> review soon.
>>
>> Great!
>>
>> As for the loader issue, did you find a way to generate images in ELF 
>> format
>> from the Cygwin system?
> I do all the time with an i386-elf cross compiler. the easiest way to 
> generate one (in my experience) is to create a directory with symlinks 
> to:
> ...

The grub Cygwin package is targeted for (and already included in) the 
official Cygwin distribution. This requires that the package can be 
build with tools already included in the distro.

Using a ELF toolchain would be desirable, but is not an option unless 
someone volunteers to maintain it.

Christian

Re: Next release?

[Yoshinori K. Okuji]

On Monday 14 July 2008 15:03:04 Pavel Roskin wrote:
> Now that we have lzma compression, we should probably consider making
> another release in a week or two.
>
> Before it happens, I'd like to add a configure test for working target
> compiler, so that users of pure x86_64 machines are not surprised by a
> message that "start" and "_start" are not found.
>
> By the way, that check could be generalized to allow more symbols, such
> as "Ltext0" on Cygwin.  It would be great if GRUB 1.97 compiled on
> Cygwin out-of-box.  Cygwin lacked some lzo development files, but we
> don't need them anymore.
>
> All warnings for i386-pc have been fixed, but perhaps I'll look at other
> targets if I have time.  Also, Valgrind finds some memory leaks, that we
> may want to fix now.
>
> I hope that once GRUB 1.97 is released, we'll eliminate device.map and
> reorganize the build system to make it more flexible.

I am sorry, but can I ask why you want to remove device.map? Did I miss any 
discussion?

Regards,
Okuji

Re: Next release?

[Pavel Roskin]

On Tue, 2008-07-15 at 23:04 +0200, Yoshinori K. Okuji wrote:

> I am sorry, but can I ask why you want to remove device.map? Did I miss any 
> discussion?

There was a short discussion, but I write it more concisely now.

1) We don't want to cache anything.  Any cached information risks to
become stale.

2) We don't want to rely on knowing how BIOS would see the devices at
the boot time.  If GRUB is installed in a way that more than one drive
is involved, it's our responsibility to do it reliably or not at all.

3) We don't want floppies or any unrelated drives to be touched when
they are not involved.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Yoshinori K. Okuji]

On Wednesday 16 July 2008 00:31:39 Pavel Roskin wrote:
> On Tue, 2008-07-15 at 23:04 +0200, Yoshinori K. Okuji wrote:
> > I am sorry, but can I ask why you want to remove device.map? Did I miss
> > any discussion?
>
> There was a short discussion, but I write it more concisely now.
>
> 1) We don't want to cache anything.  Any cached information risks to
> become stale.
>
> 2) We don't want to rely on knowing how BIOS would see the devices at
> the boot time.  If GRUB is installed in a way that more than one drive
> is involved, it's our responsibility to do it reliably or not at all.
>
> 3) We don't want floppies or any unrelated drives to be touched when
> they are not involved.

OK. Then how do you install GRUB into (hd1) in a development machine, which is 
(hd0) in a booting machine? When GRUB may not correctly determine BIOS 
drives, do you want to just give up?

Regards,
Okuji

Re: Next release?

[Pavel Roskin]

On Wed, 2008-07-16 at 01:15 +0200, Yoshinori K. Okuji wrote:

> OK. Then how do you install GRUB into (hd1) in a development machine, which is 
> (hd0) in a booting machine? When GRUB may not correctly determine BIOS 
> drives, do you want to just give up?

The boot drive can be determined at boot.

Granted, there are buggy BIOSes, but we handle it already.  All we need
is to encode into the bootloader that it was installed on a hard drive
(actually, not on a floppy, real or emulated), and the bootloader would
use 0x80 rather than the value from BIOS.

We don't need specific drive numbers like 0x81.  We need one bit of
information, and we can figure it out at the install time.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Yoshinori K. Okuji]

On Wednesday 16 July 2008 01:21:57 Pavel Roskin wrote:
> On Wed, 2008-07-16 at 01:15 +0200, Yoshinori K. Okuji wrote:
> > OK. Then how do you install GRUB into (hd1) in a development machine,
> > which is (hd0) in a booting machine? When GRUB may not correctly
> > determine BIOS drives, do you want to just give up?
>
> The boot drive can be determined at boot.
>
> Granted, there are buggy BIOSes, but we handle it already.  All we need
> is to encode into the bootloader that it was installed on a hard drive
> (actually, not on a floppy, real or emulated), and the bootloader would
> use 0x80 rather than the value from BIOS.
>
> We don't need specific drive numbers like 0x81.  We need one bit of
> information, and we can figure it out at the install time.

If a boot drive is the same as a root drive, you are right. Otherwise we need 
to do so.

I think we have seen tons of examples with GRUB Legacy which may not be solved 
automatically in all cases. If one digs into the archive of bug-grub, I guess 
several cases would be found easily. With GRUB 2, we can avoid embedding BIOS 
drive numbers in many cases, using UUIDs or labels or files. But this does 
not always work, so I am afraid that we need to support device.map, even if 
it is an evil necessity.

Regards,
Okuji

Re: Next release?

[Pavel Roskin]

On Wed, 2008-07-16 at 01:32 +0200, Yoshinori K. Okuji wrote:

> If a boot drive is the same as a root drive, you are right. Otherwise we need 
> to do so.
>
> I think we have seen tons of examples with GRUB Legacy which may not be solved 
> automatically in all cases. If one digs into the archive of bug-grub, I guess 
> several cases would be found easily. With GRUB 2, we can avoid embedding BIOS 
> drive numbers in many cases, using UUIDs or labels or files. But this does 
> not always work, so I am afraid that we need to support device.map, even if 
> it is an evil necessity.

That's a very advanced setup.  I actually cannot imagine why anyone
would use different boot and root drives.  Well, maybe the boot drive
has no partitions that GRUB or the host OS can access?  It's getting
less likely these days.  Or maybe the boot drive is too small for GRUB?
Anything bigger than a 360K floppy should be able to hold all GRUB
modules.  Or maybe the boot drive is too slow?  It's hard for me to
imagine a system that has hard drives but boots only from a floppy.

And let's not forget that dual drive installs are twice as prone to
failure.  Either drive failure makes the system unbootable.

Now, suppose that we still want to support dual drive installs.  We
should make sure is that it doesn't happen by accident.  Then it's a
fair game to ask the user for an extra option to enable a 
dual drive install, and that option could be the drive number.

Dual drive installs are also highly unportable, which means that UUIDs,
labels and file search should be sufficient in most cases.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Colin D Bennett]

On Tue, 15 Jul 2008 19:52:15 -0400
Pavel Roskin <proski@gnu.org> wrote:

> On Wed, 2008-07-16 at 01:32 +0200, Yoshinori K. Okuji wrote:
> 
> > If a boot drive is the same as a root drive, you are right.
> > Otherwise we need to do so.
> >
> > I think we have seen tons of examples with GRUB Legacy which may
> > not be solved automatically in all cases. If one digs into the
> > archive of bug-grub, I guess several cases would be found easily.
> > With GRUB 2, we can avoid embedding BIOS drive numbers in many
> > cases, using UUIDs or labels or files. But this does not always
> > work, so I am afraid that we need to support device.map, even if it
> > is an evil necessity.
> 
> That's a very advanced setup.  I actually cannot imagine why anyone
> would use different boot and root drives.  Well, maybe the boot drive
> has no partitions that GRUB or the host OS can access? 

I have used machines that have multiple Linux versions spread across
two drives, but one common /boot partition so they can all be booted
from GRUB.  This doesn't seem unusual to me.

Re: Next release?

[Javier Martín]

[-- Attachment #1: Type: text/plain, Size: 1759 bytes --]

El mié, 16-07-2008 a las 07:11 -0700, Colin D Bennett escribió:
> On Tue, 15 Jul 2008 19:52:15 -0400
> Pavel Roskin <proski@gnu.org> wrote:
> 
> > On Wed, 2008-07-16 at 01:32 +0200, Yoshinori K. Okuji wrote:
> > 
> > > If a boot drive is the same as a root drive, you are right.
> > > Otherwise we need to do so.
> > >
> > > I think we have seen tons of examples with GRUB Legacy which may
> > > not be solved automatically in all cases. If one digs into the
> > > archive of bug-grub, I guess several cases would be found easily.
> > > With GRUB 2, we can avoid embedding BIOS drive numbers in many
> > > cases, using UUIDs or labels or files. But this does not always
> > > work, so I am afraid that we need to support device.map, even if it
> > > is an evil necessity.
> > 
> > That's a very advanced setup.  I actually cannot imagine why anyone
> > would use different boot and root drives.  Well, maybe the boot drive
> > has no partitions that GRUB or the host OS can access? 
> 
> I have used machines that have multiple Linux versions spread across
> two drives, but one common /boot partition so they can all be booted
> from GRUB.  This doesn't seem unusual to me.
Same for me: I have the BIOS set up to boot from the second hard drive,
which then becomes (hd0) for GRUB through the BIOS (kinda like what my
proposed drivemap module does), but my /boot partition was on the first
hard drive, which is now (hd1). Took me a bit to realise things, and I
finally had to move around the whole partitioning scheme on the second
hard drive to put /boot in there. 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel

[-- Attachment #2: Esta parte del mensaje está firmada digitalmente --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: Next release?

[Pavel Roskin]

On Wed, 2008-07-16 at 07:11 -0700, Colin D Bennett wrote:

> > That's a very advanced setup.  I actually cannot imagine why anyone
> > would use different boot and root drives.  Well, maybe the boot drive
> > has no partitions that GRUB or the host OS can access? 
> 
> I have used machines that have multiple Linux versions spread across
> two drives, but one common /boot partition so they can all be booted
> from GRUB.  This doesn't seem unusual to me.

As I understand it, there are two cases where we have to hardcode the
drive number.

1) MBR and core.img (embedded or not) are on different drives.

2) core.img and /boot/grub are on different drives.

The second case can be mitigated because core.img can search all
available drives.  We can even tell it whether to search only hard
drives or only floppies.  After switching to lzma, we have some space in
core.img we can use for that logic.

I'm not sure that you are using either of those configurations.  If yes,
I'm not sure you need it, considering that you have Linux partitions on
both drives.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Pavel Roskin]

On Wed, 2008-07-16 at 16:17 +0200, Javier Martín wrote:

> Same for me: I have the BIOS set up to boot from the second hard drive,
> which then becomes (hd0) for GRUB through the BIOS (kinda like what my
> proposed drivemap module does), but my /boot partition was on the first
> hard drive, which is now (hd1). Took me a bit to realise things, and I
> finally had to move around the whole partitioning scheme on the second
> hard drive to put /boot in there. 

I'm not sure I understand why you had to move /boot to the second drive.
But I think if GRUB refused to do cross-drive installs by default, it
would help you consider a single drive install right away.  In any case,
it's a good thing.  Now you can remove the first drive, and GRUB would
load all the way to the menu.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Javier Martín]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

El mié, 16-07-2008 a las 12:17 -0400, Pavel Roskin escribió:
> On Wed, 2008-07-16 at 16:17 +0200, Javier Martín wrote:
> 
> > Same for me: I have the BIOS set up to boot from the second hard drive,
> > which then becomes (hd0) for GRUB through the BIOS (kinda like what my
> > proposed drivemap module does), but my /boot partition was on the first
> > hard drive, which is now (hd1). Took me a bit to realise things, and I
> > finally had to move around the whole partitioning scheme on the second
> > hard drive to put /boot in there. 
> 
> I'm not sure I understand why you had to move /boot to the second drive.
> But I think if GRUB refused to do cross-drive installs by default, it
> would help you consider a single drive install right away.  In any case,
> it's a good thing.  Now you can remove the first drive, and GRUB would
> load all the way to the menu.
> 
Remove it? o_O No I can't, because Windows XP and some of my Linux
partitions (swap and /tmp) are there.

[-- Attachment #2: Esta parte del mensaje está firmada digitalmente --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: Next release?

[Robert Millan]

On Wed, Jul 16, 2008 at 01:32:18AM +0200, Yoshinori K. Okuji wrote:
> On Wednesday 16 July 2008 01:21:57 Pavel Roskin wrote:
> > On Wed, 2008-07-16 at 01:15 +0200, Yoshinori K. Okuji wrote:
> > > OK. Then how do you install GRUB into (hd1) in a development machine,
> > > which is (hd0) in a booting machine? When GRUB may not correctly
> > > determine BIOS drives, do you want to just give up?
> >
> > The boot drive can be determined at boot.
> >
> > Granted, there are buggy BIOSes, but we handle it already.  All we need
> > is to encode into the bootloader that it was installed on a hard drive
> > (actually, not on a floppy, real or emulated), and the bootloader would
> > use 0x80 rather than the value from BIOS.
> >
> > We don't need specific drive numbers like 0x81.  We need one bit of
> > information, and we can figure it out at the install time.
> 
> If a boot drive is the same as a root drive, you are right. Otherwise we need 
> to do so.
> 
> I think we have seen tons of examples with GRUB Legacy which may not be solved 
> automatically in all cases. If one digs into the archive of bug-grub, I guess 
> several cases would be found easily. With GRUB 2, we can avoid embedding BIOS 
> drive numbers in many cases, using UUIDs or labels or files. But this does 
> not always work, so I am afraid that we need to support device.map, even if 
> it is an evil necessity.

Which cases are there that can't be fixed by using UUIDs?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

device.map (Re: Next release?)

[Robert Millan]

On Wed, Jul 16, 2008 at 12:08:57PM -0400, Pavel Roskin wrote:
> On Wed, 2008-07-16 at 07:11 -0700, Colin D Bennett wrote:
> 
> > > That's a very advanced setup.  I actually cannot imagine why anyone
> > > would use different boot and root drives.  Well, maybe the boot drive
> > > has no partitions that GRUB or the host OS can access? 
> > 
> > I have used machines that have multiple Linux versions spread across
> > two drives, but one common /boot partition so they can all be booted
> > from GRUB.  This doesn't seem unusual to me.
> 
> As I understand it, there are two cases where we have to hardcode the
> drive number.
> 
> 1) MBR and core.img (embedded or not) are on different drives.

If embedded, then they're not different drives (core.img is put right after
MBR).

Otherwise it's a no-go, and device.map won't solve your problem since it's
merely guessing which drive it'll be.  I think it's better to detect this at
install time and fail, than make the user rely on our guesswork.

> 2) core.img and /boot/grub are on different drives.
> 
> The second case can be mitigated because core.img can search all
> available drives.  We can even tell it whether to search only hard
> drives or only floppies.  After switching to lzma, we have some space in
> core.img we can use for that logic.

This is mostly implemented already.  I sent a proof of concept in a mail
titled "[PATCH] disk/fs_uuid.c".

It will only search hard drives unless no match is found (in that case your
boot is broken, so you wouldn't care much that floppy is being probed ;-)

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Next release?

[Yoshinori K. Okuji]

On Saturday 19 July 2008 17:06:24 Robert Millan wrote:
> On Wed, Jul 16, 2008 at 01:32:18AM +0200, Yoshinori K. Okuji wrote:
> > On Wednesday 16 July 2008 01:21:57 Pavel Roskin wrote:
> > > On Wed, 2008-07-16 at 01:15 +0200, Yoshinori K. Okuji wrote:
> > > > OK. Then how do you install GRUB into (hd1) in a development machine,
> > > > which is (hd0) in a booting machine? When GRUB may not correctly
> > > > determine BIOS drives, do you want to just give up?
> > >
> > > The boot drive can be determined at boot.
> > >
> > > Granted, there are buggy BIOSes, but we handle it already.  All we need
> > > is to encode into the bootloader that it was installed on a hard drive
> > > (actually, not on a floppy, real or emulated), and the bootloader would
> > > use 0x80 rather than the value from BIOS.
> > >
> > > We don't need specific drive numbers like 0x81.  We need one bit of
> > > information, and we can figure it out at the install time.
> >
> > If a boot drive is the same as a root drive, you are right. Otherwise we
> > need to do so.
> >
> > I think we have seen tons of examples with GRUB Legacy which may not be
> > solved automatically in all cases. If one digs into the archive of
> > bug-grub, I guess several cases would be found easily. With GRUB 2, we
> > can avoid embedding BIOS drive numbers in many cases, using UUIDs or
> > labels or files. But this does not always work, so I am afraid that we
> > need to support device.map, even if it is an evil necessity.
>
> Which cases are there that can't be fixed by using UUIDs?

In any case where UUIDs are not used.

I am totally against ripping off device.map. Pavel's idea is too idealistic, 
and that regresses the flexibility.

Regards,
Okuji

Re: Next release?

[Robert Millan]

On Sat, Jul 19, 2008 at 10:16:23PM +0200, Yoshinori K. Okuji wrote:
> > Which cases are there that can't be fixed by using UUIDs?
> 
> In any case where UUIDs are not used.
> 
> I am totally against ripping off device.map. Pavel's idea is too idealistic, 
> and that regresses the flexibility.

I don't think having an approach that is based on guessing (plus, maybe, user
input) is harmful, but at least we could try to rely less on it, so that it
generates less trouble without having to remove it.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Next release?

[Christian Franke]

Pavel Roskin wrote:
> On Mon, 2008-07-14 at 18:55 +0200, Christian Franke wrote:
>
>   
>> Thanks for considering Cygwin support for the next release.
>>
>> The first (and last) grub package released in the Cygwin distribution 
>> was based on grub codebase from 2008-03-26. My latest reasonably tested 
>> merge is ~2 month old. If desired, I can merge & test all remaining 
>> changes to current HEAD and post the patches for review soon.
>>     
>
> I'm sorry, I forgot that story.  It may be a more radical change than I
> expected.  Anyway, let's see what remains to be done.
>
>   

Meantime, I merged the Cygwin version to HEAD. First tests were successful.

LZMA works and is a real benefit. Core.img size was critical due to the 
large ntfs.mod. With LZMA, there is enough space for ntfscomp.mod and 
even more.

I posted the 3 remaining patches today, see:
[PATCH] grub-probe -t prefix, fix update-grub_lib for Cygwin
[PATCH] Kernel fixes for Cygwin
[PATCH] Build fixes for Cygwin

With these patches, Grub compiles & works out-of-the-box on Cygwin.

If there are no complaints, I will commit these in a week or so.

Christian

Re: Next release?

[Pavel Roskin]

On Sat, 2008-07-19 at 22:16 +0200, Yoshinori K. Okuji wrote:

> I am totally against ripping off device.map. Pavel's idea is too
> idealistic, 
> and that regresses the flexibility.

Actually, it could be said that having device.map regresses flexibility.
Suppose I want to install GRUB on a flash drive that is seen
as /dev/sdb.  I need to add /dev/sdb to device.map even though I'm not
going to see that flash drive again.  I also need to check the options
to ensure that everything is installed on the flash drive and nothing is
installed elsewhere.

Suppose that we don't have device.map.  Then I don't need to add entries
for temporary devices.  Also, I won't be able to create a cross-drive
configuration by accident, simple because it won't be allowed by
default.

If you think that device.map is beneficial for cross-device installs,
then we can have an option to enable cross-device installs, that would
also enable device.map.  Single-drive installs don't need to make any
assumptions about the BIOS numbers, and thus won't use device.map in any
way.

Actually, I think that even cross-device installs should rely on probing
the relevant drives rather than on cached information.  But we can
discuss and implement it separately.

-- 
Regards,
Pavel Roskin

Re: device.map (Re: Next release?)

[Pavel Roskin]

On Sat, 2008-07-19 at 17:14 +0200, Robert Millan wrote:

> > As I understand it, there are two cases where we have to hardcode the
> > drive number.
> > 
> > 1) MBR and core.img (embedded or not) are on different drives.
> 
> If embedded, then they're not different drives (core.img is put right after
> MBR).
> 
> Otherwise it's a no-go, and device.map won't solve your problem since it's
> merely guessing which drive it'll be.  I think it's better to detect this at
> install time and fail, than make the user rely on our guesswork.

We could do it in theory.  Even with device.map.  It's not much more
insane than having /boot/grub on a different drive.  The boot drive can
have a "bad" geometry with too few sectors per track.  Or it could be
formatted as one partition.  Or it could be a slow floppy drive.  Or it
could be in ROM.  Or BIOS may not report the boot drive correctly.

How can we fail to support this configuration and claim that the
elimination of device.map would reduce flexibility?  If we card about
flexibility, let's support hardcoding the boot drive into the
bootloader.

Actually, the groundwork is already present in grub-setup and the
bootsector, but grub-install doesn't have an option to force a drive for
core.img embedding.

> > 2) core.img and /boot/grub are on different drives.
> > 
> > The second case can be mitigated because core.img can search all
> > available drives.  We can even tell it whether to search only hard
> > drives or only floppies.  After switching to lzma, we have some space in
> > core.img we can use for that logic.
> 
> This is mostly implemented already.  I sent a proof of concept in a mail
> titled "[PATCH] disk/fs_uuid.c".
> 
> It will only search hard drives unless no match is found (in that case your
> boot is broken, so you wouldn't care much that floppy is being probed ;-)

Then all be need it to have an option in grub-install to enable this
logic.

-- 
Regards,
Pavel Roskin

Re: device.map (Re: Next release?)

[Robert Millan]

On Mon, Jul 21, 2008 at 05:26:17PM -0400, Pavel Roskin wrote:
> > 
> > This is mostly implemented already.  I sent a proof of concept in a mail
> > titled "[PATCH] disk/fs_uuid.c".
> > 
> > It will only search hard drives unless no match is found (in that case your
> > boot is broken, so you wouldn't care much that floppy is being probed ;-)
> 
> Then all be need it to have an option in grub-install to enable this
> logic.

Why an option?  Is there any situation in which it is known that boot will
be unreliable (because embed disk != /boot/grub disk), and in spite of that
you don't want UUIDs to be used?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Next release?

[Robert Millan]

On Mon, Jul 21, 2008 at 04:48:33PM -0400, Pavel Roskin wrote:
> On Sat, 2008-07-19 at 22:16 +0200, Yoshinori K. Okuji wrote:
> 
> > I am totally against ripping off device.map. Pavel's idea is too
> > idealistic, 
> > and that regresses the flexibility.
> 
> Actually, it could be said that having device.map regresses flexibility.
> Suppose I want to install GRUB on a flash drive that is seen
> as /dev/sdb.  I need to add /dev/sdb to device.map even though I'm not
> going to see that flash drive again.  I also need to check the options
> to ensure that everything is installed on the flash drive and nothing is
> installed elsewhere.
> 
> Suppose that we don't have device.map.  Then I don't need to add entries
> for temporary devices.  Also, I won't be able to create a cross-drive
> configuration by accident, simple because it won't be allowed by
> default.

But we could have device.map _and_ a fallback mechanism for when there's
no match (e.g. give it a "(dummy0)" drive).

I even STR I implemented that in some patch.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: device.map (Re: Next release?)

[Pavel Roskin]

On Tue, 2008-07-22 at 23:36 +0200, Robert Millan wrote:
> On Mon, Jul 21, 2008 at 05:26:17PM -0400, Pavel Roskin wrote:
> > > 
> > > This is mostly implemented already.  I sent a proof of concept in a mail
> > > titled "[PATCH] disk/fs_uuid.c".
> > > 
> > > It will only search hard drives unless no match is found (in that case your
> > > boot is broken, so you wouldn't care much that floppy is being probed ;-)
> > 
> > Then all be need it to have an option in grub-install to enable this
> > logic.
> 
> Why an option?  Is there any situation in which it is known that boot will
> be unreliable (because embed disk != /boot/grub disk), and in spite of that
> you don't want UUIDs to be used?

I guess I was unclear.  It's not like I don't want the UUIDs to be used.
I don't want users to create a cross-drive setup unknowingly or without
knowing the consequences.  Failure of any of the drives will break the
loading.  There are many temporary portable drives and flash devices.
The intention of the user may be to make that drive bootable, not to
create a configuration that would require both the internal and the
external drives.

If the user is OK with the cross-drive install, using UUIDs should be
either the default or the only option.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Pavel Roskin]

On Tue, 2008-07-22 at 23:37 +0200, Robert Millan wrote:

> > Suppose that we don't have device.map.  Then I don't need to add entries
> > for temporary devices.  Also, I won't be able to create a cross-drive
> > configuration by accident, simple because it won't be allowed by
> > default.
> 
> But we could have device.map _and_ a fallback mechanism for when there's
> no match (e.g. give it a "(dummy0)" drive).
> 
> I even STR I implemented that in some patch.

The problem was that the reliable information was called "dummy" and
used as fallback, whereas the unreliable information was used by
default.

-- 
Regards,
Pavel Roskin

Re: device.map (Re: Next release?)

[Robert Millan]

On Tue, Jul 22, 2008 at 05:57:56PM -0400, Pavel Roskin wrote:
> On Tue, 2008-07-22 at 23:36 +0200, Robert Millan wrote:
> > On Mon, Jul 21, 2008 at 05:26:17PM -0400, Pavel Roskin wrote:
> > > > 
> > > > This is mostly implemented already.  I sent a proof of concept in a mail
> > > > titled "[PATCH] disk/fs_uuid.c".
> > > > 
> > > > It will only search hard drives unless no match is found (in that case your
> > > > boot is broken, so you wouldn't care much that floppy is being probed ;-)
> > > 
> > > Then all be need it to have an option in grub-install to enable this
> > > logic.
> > 
> > Why an option?  Is there any situation in which it is known that boot will
> > be unreliable (because embed disk != /boot/grub disk), and in spite of that
> > you don't want UUIDs to be used?
> 
> I guess I was unclear.  It's not like I don't want the UUIDs to be used.
> I don't want users to create a cross-drive setup unknowingly or without
> knowing the consequences.  Failure of any of the drives will break the
> loading.  There are many temporary portable drives and flash devices.
> The intention of the user may be to make that drive bootable, not to
> create a configuration that would require both the internal and the
> external drives.
> 
> If the user is OK with the cross-drive install, using UUIDs should be
> either the default or the only option.

I don't think it's that unusual.  Let's take an example.  We have 1 PATA
disk and 1 SATA disk, and are installing GNU/Linux in it.  We only have
one /boot partition (or directory in /).  Turns out we have no idea which
of the two disks the BIOS will want to use for boot.  So the safest option
is to grub-install in both.  A consequence of this is that one of the two
installs is cross-disk.

And (provided that UUIDs are used), this setup is _completely_ reliable and
there's no reason to prevent the user from doing it, IMHO.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Next release?

[Robert Millan]

On Tue, Jul 22, 2008 at 06:01:41PM -0400, Pavel Roskin wrote:
> On Tue, 2008-07-22 at 23:37 +0200, Robert Millan wrote:
> 
> > > Suppose that we don't have device.map.  Then I don't need to add entries
> > > for temporary devices.  Also, I won't be able to create a cross-drive
> > > configuration by accident, simple because it won't be allowed by
> > > default.
> > 
> > But we could have device.map _and_ a fallback mechanism for when there's
> > no match (e.g. give it a "(dummy0)" drive).
> > 
> > I even STR I implemented that in some patch.
> 
> The problem was that the reliable information was called "dummy" and
> used as fallback, whereas the unreliable information was used by
> default.

None of it was intended to be reliable for anything.  It was simply a way of
making GRUB non-util code handle system devices that haven't been registered
in device.map, like a temporary USB disk.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Next release?

[Pavel Roskin]

On Wed, 2008-07-23 at 00:09 +0200, Robert Millan wrote:
> On Tue, Jul 22, 2008 at 06:01:41PM -0400, Pavel Roskin wrote:
> > On Tue, 2008-07-22 at 23:37 +0200, Robert Millan wrote:
> > 
> > > > Suppose that we don't have device.map.  Then I don't need to add entries
> > > > for temporary devices.  Also, I won't be able to create a cross-drive
> > > > configuration by accident, simple because it won't be allowed by
> > > > default.
> > > 
> > > But we could have device.map _and_ a fallback mechanism for when there's
> > > no match (e.g. give it a "(dummy0)" drive).
> > > 
> > > I even STR I implemented that in some patch.
> > 
> > The problem was that the reliable information was called "dummy" and
> > used as fallback, whereas the unreliable information was used by
> > default.
> 
> None of it was intended to be reliable for anything.  It was simply a way of
> making GRUB non-util code handle system devices that haven't been registered
> in device.map, like a temporary USB disk.

The boot driver information from BIOS should be considered as more
reliable than guess by a userspace program how BIOS might see the
drives.

There are known buggy BIOSes, but we know how to work it around.

-- 
Regards,
Pavel Roskin

Re: device.map (Re: Next release?)

[Pavel Roskin]

On Wed, 2008-07-23 at 00:08 +0200, Robert Millan wrote:

> I don't think it's that unusual.  Let's take an example.  We have 1 PATA
> disk and 1 SATA disk, and are installing GNU/Linux in it.  We only have
> one /boot partition (or directory in /).  Turns out we have no idea which
> of the two disks the BIOS will want to use for boot.  So the safest option
> is to grub-install in both.  A consequence of this is that one of the two
> installs is cross-disk.

I'm not saying it's unusual.  GRUB cannot distinguish an external SATA
from an internal SATA.  In case on an external SATA, the user may expect
to create a drive that would boot everywhere.

> And (provided that UUIDs are used), this setup is _completely_ reliable and
> there's no reason to prevent the user from doing it, IMHO.

My point is not to prevent it, just to make it clear to the user.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Marco Gerards]

Hi,

Pavel Roskin <proski@gnu.org> writes:

> Now that we have lzma compression, we should probably consider making
> another release in a week or two.
>
> Before it happens, I'd like to add a configure test for working target
> compiler, so that users of pure x86_64 machines are not surprised by a
> message that "start" and "_start" are not found.
>
> By the way, that check could be generalized to allow more symbols, such
> as "Ltext0" on Cygwin.  It would be great if GRUB 1.97 compiled on
> Cygwin out-of-box.  Cygwin lacked some lzo development files, but we
> don't need them anymore.
>
> All warnings for i386-pc have been fixed, but perhaps I'll look at other
> targets if I have time.  Also, Valgrind finds some memory leaks, that we
> may want to fix now.

Did all this happen?  Is NEWS and distlist and whatever is required
for a release up-to-date?  In that case, Okuji, do you object to a
release?

Bean suggested we release before we continue with his ideas regarding
the modularity of the code.  Okuji (and others), can you please review
the patch I sent in for handlers in the thread related to the
elimination of normal mode?

--
Marco

Re: Next release?

[Pavel Roskin]

On Thu, 2008-07-24 at 19:02 +0200, Marco Gerards wrote:
> Hi,
> 
> Pavel Roskin <proski@gnu.org> writes:
> 
> > Now that we have lzma compression, we should probably consider making
> > another release in a week or two.
> >
> > Before it happens, I'd like to add a configure test for working target
> > compiler, so that users of pure x86_64 machines are not surprised by a
> > message that "start" and "_start" are not found.

Done.

> > By the way, that check could be generalized to allow more symbols, such
> > as "Ltext0" on Cygwin.  It would be great if GRUB 1.97 compiled on
> > Cygwin out-of-box.  Cygwin lacked some lzo development files, but we
> > don't need them anymore.

I was wrong here, but the progress is being made.  I'm just not sure it
should be in 1.97.

> > All warnings for i386-pc have been fixed, but perhaps I'll look at other
> > targets if I have time.  Also, Valgrind finds some memory leaks, that we
> > may want to fix now.

The absolute majority of warnings have been fixed, including all for
i386-pc.  There are some hard warnings for other architectures.  The can
be suppressed, but they indicate that we need e.g. some reorganization
of the include files.  So it's better to keep them for now.

I was unable to check sparc64-ieee1275.  I'm afraid it's broken.  I
don't have hardware to test it on, but the real problem is lack of time.
But with x86_64-efi, I guess most issues with 64-bit bootloaders have
been resolved.

> Did all this happen?  Is NEWS and distlist and whatever is required
> for a release up-to-date?  In that case, Okuji, do you object to a
> release?
> 
> Bean suggested we release before we continue with his ideas regarding
> the modularity of the code.  Okuji (and others), can you please review
> the patch I sent in for handlers in the thread related to the
> elimination of normal mode?

We need at least a simple fix for cross-drive installs.  Perhaps we need
a PC specific function to translate device names like hd1 into BIOS IDs
like 0x81.

Also it would be nice to make grub-install ask users to check only
relevant entries from device map.  By showing the whole file, the risk
is that everybody would just ignore the message.  I think grub-setup
should have a "semi-verbose" mode, in which it would print something
like:

device.map: /dev/sda is (hd0)

And then grub-setup would ask the user to verify those assumptions.

We can make a more fundamental fix later, unless it's already
implemented, tested and meets no objections.

-- 
Regards,
Pavel Roskin

Re: Next release?

[Christian Franke]

Pavel Roskin wrote:
> On Thu, 2008-07-24 at 19:02 +0200, Marco Gerards wrote:
>   
>> Hi,
>>
>> Pavel Roskin <proski@gnu.org> writes:
>>
>>     
> ...
>>> By the way, that check could be generalized to allow more symbols, such
>>> as "Ltext0" on Cygwin.  It would be great if GRUB 1.97 compiled on
>>> Cygwin out-of-box.  Cygwin lacked some lzo development files, but we
>>> don't need them anymore.
>>>       
>
> I was wrong here, but the progress is being made.  I'm just not sure it
> should be in 1.97.
>
>   

Thanks to grub-pe2elf provided by Bean, Cygwin support will be in 1.97 :-)
It works with current SVN, except one open issue which breaks 
grub-install, see "[PATCH] ... fix update-grub_lib for Cygwin".

Christian