[PATCH] Build issues with parallel make

[PATCH] Build issues with parallel make

[Kevin Lacquement]

[-- Attachment #1.1: Type: text/plain, Size: 386 bytes --]

Greetings all,

A bug was recently filed on Gentoo (https://bugs.gentoo.org/show_bug.cgi?id=185361), showing that for a parallel make headers aren't necessarily built before their object files.  I didn't look much into the problem, I just made a quick band-aid patch, which only handles the reported problem but it could probably bear some looking into.

Cheers,
Kevin Lacquement

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: grub-1.95-headers.patch --]
[-- Type: text/x-patch; name="grub-1.95-headers.patch", Size: 622 bytes --]

diff -ru grub-1.95.clean/conf/i386-pc.mk grub-1.95/conf/i386-pc.mk
--- grub-1.95.clean/conf/i386-pc.mk	2007-07-16 11:39:47.000000000 -0300
+++ grub-1.95/conf/i386-pc.mk	2007-07-16 11:49:25.000000000 -0300
@@ -739,7 +739,7 @@
 	$(CC) -Iutil -I$(srcdir)/util $(CPPFLAGS) $(CFLAGS) -DGRUB_UTIL=1 $(grub_emu_CFLAGS) -MD -c -o $@ $<
 -include grub_emu-util_console.d
 
-grub_emu-util_grub_emu.o: util/grub-emu.c
+grub_emu-util_grub_emu.o: util/grub-emu.c grub_modules_init.h
 	$(CC) -Iutil -I$(srcdir)/util $(CPPFLAGS) $(CFLAGS) -DGRUB_UTIL=1 $(grub_emu_CFLAGS) -MD -c -o $@ $<
 -include grub_emu-util_grub_emu.d
 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Build issues with parallel make

[Robert Millan]

On Tue, Jul 17, 2007 at 06:29:25AM -0300, Kevin Lacquement wrote:
> Greetings all,
> 
> A bug was recently filed on Gentoo (https://bugs.gentoo.org/show_bug.cgi?id=185361), showing that for a parallel make headers aren't necessarily built before their object files.  I didn't look much into the problem, I just made a quick band-aid patch, which only handles the reported problem but it could probably bear some looking into.

Thank you for your help, but it is a bit more complicated than that.  The file
you're modifiing is dynamicaly generated with:

  ruby1.8 genmk.rb < conf/i386-pc.rmk > conf/i386-pc.mk

Making this work properly might be a bit trickier.  I myself don't understand
it (plus, I'm ruby illiterate), so I'm afraid I can't be of help on this.

> diff -ru grub-1.95.clean/conf/i386-pc.mk grub-1.95/conf/i386-pc.mk
> --- grub-1.95.clean/conf/i386-pc.mk	2007-07-16 11:39:47.000000000 -0300
> +++ grub-1.95/conf/i386-pc.mk	2007-07-16 11:49:25.000000000 -0300
> @@ -739,7 +739,7 @@
>  	$(CC) -Iutil -I$(srcdir)/util $(CPPFLAGS) $(CFLAGS) -DGRUB_UTIL=1 $(grub_emu_CFLAGS) -MD -c -o $@ $<
>  -include grub_emu-util_console.d
>  
> -grub_emu-util_grub_emu.o: util/grub-emu.c
> +grub_emu-util_grub_emu.o: util/grub-emu.c grub_modules_init.h
>  	$(CC) -Iutil -I$(srcdir)/util $(CPPFLAGS) $(CFLAGS) -DGRUB_UTIL=1 $(grub_emu_CFLAGS) -MD -c -o $@ $<
>  -include grub_emu-util_grub_emu.d

-- 
Robert Millan

My spam trap is honeypot@aybabtu.com.  Note: this address is only intended
for spam harvesters.  Writing to it will get you added to my black list.

Re: [PATCH] Build issues with parallel make

[Kevin Lacquement]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Fri, 20 Jul 2007 19:21:05 +0200
Robert Millan <rmh@aybabtu.com> wrote:


> Thank you for your help, but it is a bit more complicated than that.  The file
> you're modifiing is dynamicaly generated with:
> 
>   ruby1.8 genmk.rb < conf/i386-pc.rmk > conf/i386-pc.mk
> 
> Making this work properly might be a bit trickier.  I myself don't understand
> it (plus, I'm ruby illiterate), so I'm afraid I can't be of help on this.
> 

Thank you for your response.  As I said, it's pretty much a band-aid response.  I'll take deeper look at the build process.

Cheers,
Kevin

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] Build issues with parallel make

[Simon Peter]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

Hi!

> > Making this work properly might be a bit trickier.  I myself don't
> > understand it (plus, I'm ruby illiterate), so I'm afraid I can't be
> > of help on this.
> Thank you for your response.  As I said, it's pretty much a band-aid
> response. I'll take deeper look at the build process.

I encountered the same problem and fixed it by allowing finer-grained
dependency control in the rmk-file (source files are allowed to have
dependencies now) and correcting the two header file dependencies (not
the binaries depend on the headers, but one of the source files,
respectively). I changed the Ruby script for that. My patch is attached.

On a whole, I would like to encourage the grub folks to move to
Automake. Automake has automatic dependency tracking, so problems like
this cannot arise. Plus it's one of your own tools (it's GNU)! And I'm
sure you could drop that awkward Ruby build-dependency by using some
Autoconf magic for the rest. I hope I did not offend anyone. ;)

Best regards,
Simon

[-- Attachment #2: correct_dependencies.dif --]
[-- Type: video/dv, Size: 1539 bytes --]

Re: [PATCH] Build issues with parallel make

[Robert Millan]

On Sun, Aug 26, 2007 at 09:36:01PM +0200, Simon Peter wrote:
> Hi!
> 
> > > Making this work properly might be a bit trickier.  I myself don't
> > > understand it (plus, I'm ruby illiterate), so I'm afraid I can't be
> > > of help on this.
> > Thank you for your response.  As I said, it's pretty much a band-aid
> > response. I'll take deeper look at the build process.
> 
> I encountered the same problem and fixed it by allowing finer-grained
> dependency control in the rmk-file (source files are allowed to have
> dependencies now)

Great!

> and correcting the two header file dependencies (not
> the binaries depend on the headers, but one of the source files,
> respectively).

This doesn't sound right.  Since the source files (grub-setup.c, grub-probe.c)
are not dynamicaly generated, make doesn't need a rule to "build" them.  Why
did you need to change this?

Though, if the rmk needs to be modified, take into account there are other
targets which probably share the same problem (powerpc-ieee1275, i386-efi).

Btw, could you please include a ChangeLog entry with your patch?

> On a whole, I would like to encourage the grub folks to move to
> Automake. Automake has automatic dependency tracking, so problems like
> this cannot arise. Plus it's one of your own tools (it's GNU)! And I'm
> sure you could drop that awkward Ruby build-dependency by using some
> Autoconf magic for the rest. I hope I did not offend anyone. ;)

You should take this up with Okuji.  I don't think you offended him, but
I assume he had his own reasons to put it this way :-)

Thank you,

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)

Re: [PATCH] Build issues with parallel make

[Simon Peter]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

> > and correcting the two header file dependencies (not
> > the binaries depend on the headers, but one of the source files,
> > respectively).
> This doesn't sound right.  Since the source files (grub-setup.c,
> grub-probe.c) are not dynamicaly generated, make doesn't need a rule
> to "build" them.  Why did you need to change this?

The dependencies are actually generated for the object files. Before,
the Ruby script would make the objects only depend on their respective
sources. However, you still define the dependency for the source files.
I think this is better because the Ruby script generates the rules to
build the object files and object files might have different names on
different architectures, whereas the source file names always stay
identical. And it's also not totally illogical: Since the source files
include the headers, they also depend on them in a sense.

> Though, if the rmk needs to be modified, take into account there are
> other targets which probably share the same problem
> (powerpc-ieee1275, i386-efi).

You're right. I fixed them as well.

> Btw, could you please include a ChangeLog entry with your patch?

Sure. The new version (including aforementioned fixes) against latest
CVS head is attached.

Simon

[-- Attachment #2: new-correct_dependencies.dif --]
[-- Type: video/dv, Size: 3537 bytes --]

Re: [PATCH] Build issues with parallel make

[Robert Millan]

On Tue, Aug 28, 2007 at 05:54:47PM +0200, Simon Peter wrote:
> > Btw, could you please include a ChangeLog entry with your patch?
> 
> Sure. The new version (including aforementioned fixes) against latest
> CVS head is attached.

+       * genmk.rb: Added support for finer-grained dependency
+       control: Source files can depend on headers now. Fixes parallel
+       make dependency problems.
+       * conf/i386-pc.rmk: Added proper source file dependencies for
+       grub-setup and grub-probe.
+       * conf/i386-efi.rmk: Dito.
+       * conf/powerpc-ieee1275.rmk: Dito.

There was a minor problem.  ChangeLog entries should describe the actual
changes, rather than their effect.  I fixed this for you.

Thank you!

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)