multiboot

multiboot

[Johan Rydberg]

What's the status of the multiboot module in GRUB2?  Has anyone
(Jeroen?) tried to boot for example the Hurd (or Hurd/L4)?

~j

Re: multiboot

[Marco Gerards]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> What's the status of the multiboot module in GRUB2?  Has anyone
> (Jeroen?) tried to boot for example the Hurd (or Hurd/L4)?

AFAIK it can be used to boot GNU/Hurd.  IIRC I used that.  But there
were some problems with the multiboot scripts as they are used.  The
variables are parsed by GRUB 2 as well so you have to escape them.

Just have a look at my script to create boot floppies, IIRC that one
worked for booting GNU/Hurd.

I did not test :4 Hurd yet.

--
Marco

Re: multiboot

[Johan Rydberg]

Marco Gerards <metgerards@student.han.nl> writes:

> AFAIK it can be used to boot GNU/Hurd.  IIRC I used that.  But there
> were some problems with the multiboot scripts as they are used.  The
> variables are parsed by GRUB 2 as well so you have to escape them.
>
> Just have a look at my script to create boot floppies, IIRC that one
> worked for booting GNU/Hurd.

Great to hear.

This brings me to the next issue; have someone tried to boot GRUB2/PPC
with U-Boot?  It would be a fun experiment to try to run GRUB on a
development board like Ebony or Atlantis.

So, what do I need to get going?  A powerpc-elf (or should I use some
other target?) cross-compiler?

~j

Re: multiboot

[Marco Gerards]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> This brings me to the next issue; have someone tried to boot GRUB2/PPC
> with U-Boot?  It would be a fun experiment to try to run GRUB on a
> development board like Ebony or Atlantis.

U-Boot is another PPC firmware, right?  Something like OF, but not
compatible?

> So, what do I need to get going?  A powerpc-elf (or should I use some
> other target?) cross-compiler?

You can use a cross-compiler to compile grubof.

--
Marco

Re: multiboot

[Marco Gerards]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> I don't think it has a firmware layer at all, so I suppose you must
> implement things like IDE and console in GRUB itself.  Which of course
> can be rather boring and error prune.
>
> The real question I guess is whether or not things like drivers should
> be included in GRUB2 at all, or should GRUB always depend on an
> underlying layer to provide those facilities?

In some cases, yes.  For example a framebuffer driver like we have on
the PC.  Perhaps we can do the same for the PPC.  The disadvantage is
that we need several drivers (ATI, nvidia, matrox, etc).

An IDE/ATAPI driver for the PC would be nice too IMHO.  In that case
we can support CDROM drives without relying on the BIOS.

>>> So, what do I need to get going?  A powerpc-elf (or should I use some
>>> other target?) cross-compiler?
>>
>> You can use a cross-compiler to compile grubof.
>
> I didn't manage to do so.  I'll research it a bit and probably send in
> a patch.

Nice.

Thanks,
Marco

Re: multiboot

[Johan Rydberg]

Marco Gerards <metgerards@student.han.nl> writes:

>> This brings me to the next issue; have someone tried to boot GRUB2/PPC
>> with U-Boot?  It would be a fun experiment to try to run GRUB on a
>> development board like Ebony or Atlantis.
>
> U-Boot is another PPC firmware, right?  Something like OF, but not
> compatible?

It's a boot loader that is used to boot GNU/Linux on PPC (and other)
platforms.  More information can be found at http://ppcboot.sourceforge.net/

I don't think it has a firmware layer at all, so I suppose you must
implement things like IDE and console in GRUB itself.  Which of course
can be rather boring and error prune.

The real question I guess is whether or not things like drivers should
be included in GRUB2 at all, or should GRUB always depend on an
underlying layer to provide those facilities?

>> So, what do I need to get going?  A powerpc-elf (or should I use some
>> other target?) cross-compiler?
>
> You can use a cross-compiler to compile grubof.

I didn't manage to do so.  I'll research it a bit and probably send in
a patch.

~j

Re: multiboot

[Johan Rydberg]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

Marco Gerards <metgerards@student.han.nl> writes:

>> I didn't manage to do so.  I'll research it a bit and probably send in
>> a patch.
>
> Nice.

The initial problem was that AC_CHECK_SIZEOF doesn't work if you're
cross-compiling.  I fixed this by using AC_COMPILE_CHECK_SIZEOF [1]
instead.

But the major problem is that parts of GRUB2 requires a C library to
be installed, at least headers such as alloca.h and stdint.h.  Is this
really a valid demand?  

Another issue regarding BUILD_CC.  The normal convention is that if
you want to build something that should run on the build machine, you
use something similar to BUILD_CC.  That's why I think the following
patch is self-explaining.

--- configure.ac        4 Apr 2004 13:45:59 -0000       1.9
+++ configure.ac        8 Dec 2004 12:57:07 -0000
@@ -109,7 +109,7 @@
 AC_PATH_PROG(RUBY, ruby)
 
 # For cross-compiling.
-if test "x$build" = "x$host"; then
+if test "x$build" != "x$host"; then
   AC_CHECK_PROGS(BUILD_CC, [gcc egcs cc],
                 [AC_MSG_ERROR([none of gcc, egcs and cc is found. set BUILD_CC manually.])])
 else
@@ -127,8 +127,8 @@

That brings us the next issue; BUILD_CC is used to build _everything_,
including the host-specific object files, and since BUILD_CC is set to
/usr/bin/gcc in my cross-compiler environment it will try to compile
PPC assembler sources with a i386 compiler.  Not good.

My suggestion is to create a new class for programs that should be
host-specific, and let Utilities be compiled on the build machine.
See the patch.  

~j

 [1] http://ac-archive.sourceforge.net/Miscellaneous/ac_compile_check_sizeof.html


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: grub-build-cc.patch --]
[-- Type: text/x-patch, Size: 2747 bytes --]

Index: genmk.rb
===================================================================
RCS file: /cvsroot/grub/grub2/genmk.rb,v
retrieving revision 1.8
diff -u -r1.8 genmk.rb
--- genmk.rb	4 Apr 2004 13:45:59 -0000	1.8
+++ genmk.rb	8 Dec 2004 13:11:19 -0000
@@ -205,9 +205,55 @@
   end
 end
 
+class Program
+  def initialize(dir, name)
+    @dir = dir
+    @name = name
+  end
+  attr_reader :dir, :name
+
+  def rule(sources)
+    prefix = @name.to_var
+    objs = sources.collect do |src|
+      raise "unknown source file `#{src}'" if /\.[cS]$/ !~ src
+      prefix + '-' + src.to_obj
+    end
+    objs_str = objs.join(' ');
+    deps = objs.collect {|obj| obj.suffix('d')}
+    deps_str = deps.join(' ');
+
+    "CLEANFILES += #{@name} #{objs_str}
+MOSTLYCLEANFILES += #{deps_str}
+
+#{@name}: #{objs_str}
+	$(BUILD_CC) -o $@ $^ $(BUILD_LDFLAGS) $(#{prefix}_LDFLAGS)
+
+" + objs.collect_with_index do |obj, i|
+      src = sources[i]
+      fake_obj = File.basename(src).suffix('o')
+      dep = deps[i]
+      dir = File.dirname(src)
+
+      "#{obj}: #{src}
+	$(CC) -I#{dir} -I$(srcdir)/#{dir} $(CPPFLAGS) $(CFLAGS) -DGRUB_UTIL=1 $(#{prefix}_CFLAGS) -c -o $@ $<
+
+#{dep}: #{src}
+	set -e; \
+	  $(CC) -I#{dir} -I$(srcdir)/#{dir} $(CPPFLAGS) $(CFLAGS) -DGRUB_UTIL=1 $(#{prefix}_CFLAGS) -M $< \
+	  | sed 's,#{Regexp.quote(fake_obj)}[ :]*,#{obj} $@ : ,g' > $@; \
+	  [ -s $@ ] || rm -f $@
+
+-include #{dep}
+
+"
+    end.join('')
+  end
+end
+
 images = []
 utils = []
 pmodules = []
+programs = []
 
 cont = false
 s = nil
@@ -245,6 +291,11 @@
 	    Utility.new(prefix, util)
 	  end
 
+	when 'PROGRAMS'
+	  programs += args.split(/\s+/).collect do |util|
+	    Program.new(prefix, util)
+	  end
+
 	when 'SOURCES'
 	  if img = images.detect() {|i| i.name.to_var == prefix}
 	    print img.rule(args.split(/\s+/))
@@ -252,6 +303,8 @@
 	    print pmod.rule(args.split(/\s+/))
 	  elsif util = utils.detect() {|u| u.name.to_var == prefix}
 	    print util.rule(args.split(/\s+/))
+	  elsif program = programs.detect() {|u| u.name.to_var == prefix}
+	    print program.rule(args.split(/\s+/))
 	  end
 	end
       end
Index: conf/powerpc-ieee1275.rmk
===================================================================
RCS file: /cvsroot/grub/grub2/conf/powerpc-ieee1275.rmk,v
retrieving revision 1.15
diff -u -r1.15 powerpc-ieee1275.rmk
--- conf/powerpc-ieee1275.rmk	4 Dec 2004 18:45:45 -0000	1.15
+++ conf/powerpc-ieee1275.rmk	8 Dec 2004 13:11:19 -0000
@@ -15,8 +15,10 @@
 kernel_syms.lst: $(addprefix include/grub/,$(kernel_img_HEADERS)) genkernsyms.sh
 	sh $(srcdir)/genkernsyms.sh $(filter %h,$^) > $@
 
+# Programs.
+sbin_PROGRAMS = grubof
+
 # Utilities.
-sbin_UTILITIES = grubof
 bin_UTILITIES = grub-emu
 noinst_UTILITIES = genmoddep
 

Re: multiboot

[Johan Rydberg]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> +#{@name}: #{objs_str}
> +	$(BUILD_CC) -o $@ $^ $(BUILD_LDFLAGS) $(#{prefix}_LDFLAGS)

Should be $(CC) of course.  Anyway, the patch was just to promote my
idea.

~j

Re: multiboot

[Marco Gerards]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> The initial problem was that AC_CHECK_SIZEOF doesn't work if you're
> cross-compiling.  I fixed this by using AC_COMPILE_CHECK_SIZEOF [1]
> instead.

This sounds right to me, but I do not see this in the patch...

> But the major problem is that parts of GRUB2 requires a C library to
> be installed, at least headers such as alloca.h and stdint.h.  Is this
> really a valid demand?  

Is this required for grubof?  IIRC those headers are not a part of the
C library, but a part of gcc or libgcc or so.  Am I right about that?

Instead of alloca we can also use something like:

...

{
 int foo[size]
 ...
}

In most cases that will remove the dependency on alloca.  IMHO it is
better not to use alloca.

> Another issue regarding BUILD_CC.  The normal convention is that if
> you want to build something that should run on the build machine, you
> use something similar to BUILD_CC.  That's why I think the following
> patch is self-explaining.

That sounds sane to me.  I assume this fixes Timothy's problem as
well.

> That brings us the next issue; BUILD_CC is used to build _everything_,
> including the host-specific object files, and since BUILD_CC is set to
> /usr/bin/gcc in my cross-compiler environment it will try to compile
> PPC assembler sources with a i386 compiler.  Not good.
>
> My suggestion is to create a new class for programs that should be
> host-specific, and let Utilities be compiled on the build machine.
> See the patch.  

Ok, that sounds sane.  But what if you really want to cross-compile a
complete system (including libc, GRUB, etc)?

Can you make one big patch with a changelog entry so it can be
committed after Okuji agrees with the patch?

Thanks,
Marco

Re: multiboot

[Johan Rydberg]

Marco Gerards <metgerards@student.han.nl> writes:

> Johan Rydberg <jrydberg@night.trouble.net> writes:
>
>> The initial problem was that AC_CHECK_SIZEOF doesn't work if you're
>> cross-compiling.  I fixed this by using AC_COMPILE_CHECK_SIZEOF [1]
>> instead.
>
> This sounds right to me, but I do not see this in the patch...

Of course not.  I didn't include it :) I'm sure you can download the
m4 macro, install it in acinclude.m4 and change AC_CHECK_SIZEOF to
AC_COMPILE_CHECK_SIZEOF.  If you're as lazy as I am, I can include it
in a later patch.

>> But the major problem is that parts of GRUB2 requires a C library to
>> be installed, at least headers such as alloca.h and stdint.h.  Is this
>> really a valid demand?  
>
> Is this required for grubof?  IIRC those headers are not a part of the
> C library, but a part of gcc or libgcc or so.  Am I right about that?

My GCC installation (3.2.2 configured for powerpc-elf) installed the
following headers:

/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/syslimits.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/stdarg.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/stddef.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/varargs.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/stdbool.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/iso646.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/ppc-asm.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/altivec.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/limits.h
/gnu/lib/gcc-lib/powerpc-elf/3.2.2/include/float.h

And my native installation of GCC (3.3.4 configured for i486-linux)
includes:

/usr/lib/gcc-lib/i486-linux/3.3.4/include/iso646.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/float.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/limits.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/stdarg.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/stdbool.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/stddef.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/syslimits.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/unwind.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/varargs.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/mmintrin.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/emmintrin.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/pmmintrin.h
/usr/lib/gcc-lib/i486-linux/3.3.4/include/xmmintrin.h

A little more research shows that both stdint.h and alloca.h is part
of glibc:

$ head -2 /usr/include/stdint.h 
/* Copyright (C) 1997, 1998, 1999, 2000, 2001 Free Software Foundation, Inc.
   This file is part of the GNU C Library.
$ head -2 /usr/include/alloca.h 
/* Copyright (C) 1992, 1996, 1997, 1998, 1999 Free Software Foundation, Inc.
   This file is part of the GNU C Library.

So either we stop using then, or write our own versions.

> Instead of alloca we can also use something like:
> [...]
> In most cases that will remove the dependency on alloca.  IMHO it is
> better not to use alloca.

It is also possible to use __builtin_alloca if alloca.h is missing,
provided that there's a requirement for GCC.  

>> That brings us the next issue; BUILD_CC is used to build _everything_,
>> including the host-specific object files, and since BUILD_CC is set to
>> /usr/bin/gcc in my cross-compiler environment it will try to compile
>> PPC assembler sources with a i386 compiler.  Not good.
>>
>> My suggestion is to create a new class for programs that should be
>> host-specific, and let Utilities be compiled on the build machine.
>> See the patch.  
>
> Ok, that sounds sane.  But what if you really want to cross-compile a
> complete system (including libc, GRUB, etc)?

Good call.  I'm not sure what to do.  Got any ideas? 

> Can you make one big patch with a changelog entry so it can be
> committed after Okuji agrees with the patch?

I would prefer Okuji looked at the changes to genmk.rb before that.

~j

Re: multiboot

[Marco Gerards]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> Marco Gerards <metgerards@student.han.nl> writes:
>
>> Johan Rydberg <jrydberg@night.trouble.net> writes:
>>
>>> The initial problem was that AC_CHECK_SIZEOF doesn't work if you're
>>> cross-compiling.  I fixed this by using AC_COMPILE_CHECK_SIZEOF [1]
>>> instead.
>>
>> This sounds right to me, but I do not see this in the patch...
>
> Of course not.  I didn't include it :) I'm sure you can download the
> m4 macro, install it in acinclude.m4 and change AC_CHECK_SIZEOF to
> AC_COMPILE_CHECK_SIZEOF.  If you're as lazy as I am, I can include it
> in a later patch.

As you know, I am lazy... ;)

>>> But the major problem is that parts of GRUB2 requires a C library to
>>> be installed, at least headers such as alloca.h and stdint.h.  Is this
>>> really a valid demand?  
>>
>> Is this required for grubof?  IIRC those headers are not a part of the
>> C library, but a part of gcc or libgcc or so.  Am I right about that?
>
> My GCC installation (3.2.2 configured for powerpc-elf) installed the
> following headers:

[...]

> A little more research shows that both stdint.h and alloca.h is part
> of glibc:

[...]

> So either we stop using then, or write our own versions.

Ok, agreed.

>> Instead of alloca we can also use something like:
>> [...]
>> In most cases that will remove the dependency on alloca.  IMHO it is
>> better not to use alloca.
>
> It is also possible to use __builtin_alloca if alloca.h is missing,
> provided that there's a requirement for GCC.  

There is a gcc requirement already, that is not a problem.

Thanks,
Marco

Re: multiboot

[Yoshinori K. Okuji]

On Wednesday 08 December 2004 14:24, Johan Rydberg wrote:
> The initial problem was that AC_CHECK_SIZEOF doesn't work if you're
> cross-compiling.  I fixed this by using AC_COMPILE_CHECK_SIZEOF [1]
> instead.

Ok.

> But the major problem is that parts of GRUB2 requires a C library to
> be installed, at least headers such as alloca.h and stdint.h.  Is
> this really a valid demand?

Why do we use stdint.h? As far as I see, it is used only for 
powerpc-specific code. This should be removed. GRUB must use its own 
types defined in include/grub/types.h.

alloca should not be used. It must be replaced with something else (e.g. 
grub_malloc). Note that alloca requires a C library in some 
architectures.

> Another issue regarding BUILD_CC.  The normal convention is that if
> you want to build something that should run on the build machine, you
> use something similar to BUILD_CC.  That's why I think the following
> patch is self-explaining.

You are right. I didn't notice the bug, because I usually specify CC and 
BUILD_CC manually.

> That brings us the next issue; BUILD_CC is used to build
> _everything_, including the host-specific object files, and since
> BUILD_CC is set to /usr/bin/gcc in my cross-compiler environment it
> will try to compile PPC assembler sources with a i386 compiler.  Not
> good.

Why? Host-specific objects are compiled by CC instead of BUILD_CC.

Okuji

Re: multiboot

[Johan Rydberg]

"Yoshinori K. Okuji" <okuji@enbug.org> writes:

>> That brings us the next issue; BUILD_CC is used to build
>> _everything_, including the host-specific object files, and since
>> BUILD_CC is set to /usr/bin/gcc in my cross-compiler environment it
>> will try to compile PPC assembler sources with a i386 compiler.  Not
>> good.
>
> Why? Host-specific objects are compiled by CC instead of BUILD_CC.

Not for the powerpc target, where everything is "utilities" including
grubof.

~j

Re: multiboot

[Marco Gerards]

Johan Rydberg <jrydberg@night.trouble.net> writes:

> "Yoshinori K. Okuji" <okuji@enbug.org> writes:
>
>>> That brings us the next issue; BUILD_CC is used to build
>>> _everything_, including the host-specific object files, and since
>>> BUILD_CC is set to /usr/bin/gcc in my cross-compiler environment it
>>> will try to compile PPC assembler sources with a i386 compiler.  Not
>>> good.
>>
>> Why? Host-specific objects are compiled by CC instead of BUILD_CC.
>
> Not for the powerpc target, where everything is "utilities" including
> grubof.

Right, but that is just a "bug" that should be fixed after module
loading works.

As for module loading.  It already works, but in grubof it crashes
when jumping into the module.  This is most likely a problem with the
protection of the memory.  Johan told me to have a look at the ibat
registers, etc.  I'll do that.

I can check in the kern/powerpc/dl.c + powerpc-ieee1275.rmk changes so
it becomes easy for Hollis to finish grub-mkimage, IIRC he still
wanted to do some things there, although it worked fine for me.

That way we can get module loading quite soon and fix the problem with
utilities as well...  Hacking around this problem while it is not
crucial does not seem right to me.

Thanks,
Marco