[PATCH] stubdom: disable parallel build

[PATCH] stubdom: disable parallel build

[Olaf Hering]

# HG changeset patch
# User Olaf Hering <olaf@aepfle.de>
# Date 1343666089 -7200
# Node ID ccbebdbe44da0460408172338af9daab31e56788
# Parent  db8adce4f09307a90f96103f7fd67efa97fc9ac0
stubdom: disable parallel build

Recently I changed my rpm xen.spec file from doing
'make -C tools -j N && make stubdom' to 'make -j N stubdom' because
stubdom depends on tools, so both get built.
The result was the failure below.

....
mkdir -p grub-x86_64
CPPFLAGS="-isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include -D__MINIOS__ -DHAVE_LIBC -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/posix -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../tools/xenstore  -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/x86 -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/x86/x86_64 -U __linux__ -U __FreeBSD__ -U __sun__ -nostdinc -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/posix -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/cross-root-x86_64/x86_64-xen-elf/include -isystem /usr/lib64/gcc/x86_64
 -suse-linux/4.7/include -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/lwip-x86_64/src/include -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/lwip-x86_64/src/include/ipv4 -I/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/include -I/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../xen/include" CFLAGS="-mno-red-zone -O1 -fno-omit-frame-pointer  -m64 -mno-red-zone -fno-reorder-blocks -fno-asynchronous-unwind-tables -m64 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable   -fno-stack-protector -fno-exceptions" make DESTDIR= -C grub OBJ_DIR=/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/grub-x86_64
make[2]: Entering directory `/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/grub'
make[2]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
make[2]: *** INTERNAL: readdir: Bad file descriptor
.  Stop.
make[2]: Makefile: Field 'stem' not cached: Makefile

make[2]: Leaving directory `/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/grub'
make[1]: *** [grub] Error 2
[ -d mini-os-x86_64-xenstore ] || \
for i in $(cd /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os ; find . -type d) ; do \
                mkdir -p mini-os-x86_64-xenstore/$i ; \
done
....

Disable parallel build for stubdom until the Makefiles are updated to handle
it properly.

Signed-off-by: Olaf Hering <olaf@aepfle.de>

diff -r db8adce4f093 -r ccbebdbe44da Makefile
--- a/Makefile
+++ b/Makefile
@@ -21,7 +21,7 @@ install: install-xen install-kernels ins
 build: kernels
 	$(MAKE) -C xen build
 	$(MAKE) -C tools build
-	$(MAKE) -C stubdom build
+	$(MAKE) -C stubdom build -j 1
 ifeq (x86_64,$(XEN_TARGET_ARCH))
 	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom pv-grub
 endif
@@ -78,7 +78,7 @@ install-kernels:
 
 .PHONY: install-stubdom
 install-stubdom: tools/qemu-xen-traditional-dir install-tools
-	$(MAKE) -C stubdom install
+	$(MAKE) -C stubdom install -j 1
 ifeq (x86_64,$(XEN_TARGET_ARCH))
 	XEN_TARGET_ARCH=x86_32 $(MAKE) -C stubdom install-grub
 endif

Re: [PATCH] stubdom: disable parallel build

[Ian Campbell]

On Mon, 2012-07-30 at 17:35 +0100, Olaf Hering wrote:
> # HG changeset patch
> # User Olaf Hering <olaf@aepfle.de>
> # Date 1343666089 -7200
> # Node ID ccbebdbe44da0460408172338af9daab31e56788
> # Parent  db8adce4f09307a90f96103f7fd67efa97fc9ac0
> stubdom: disable parallel build
> 
> Recently I changed my rpm xen.spec file from doing
> 'make -C tools -j N && make stubdom' to 'make -j N stubdom' because
> stubdom depends on tools, so both get built.
> The result was the failure below.

I always do -j12 builds of the stubdom sub=tree and I don't see
parallelisation issues. The stubdom build is pretty big and time
consuming so it would be a shame to needlessly linearise it.

I suspect something is not quite right in the top level Makefile WRT the
dependencies between tools and stubdom builds.

Why does stubdom-install depend on tools-install anyhow? Experimentally
it appears to only need "make -C tools/include/" before "make DESTDIR=
$(pwd)/dist/install -C stubdom/ install" succeeds. I can also see things
like -I$(XEN_ROOT)/tools/libxc but that doesn't require us to have built
the tools.

> make[2]: *** INTERNAL: readdir: Bad file descriptor

This is an internal error inside make? That's surely more critical than
any problem with our build system and needs to be solved before adding
workarounds to our tree.

Ian.

Re: [PATCH] stubdom: disable parallel build

[Olaf Hering]

On Tue, Jul 31, Ian Campbell wrote:

> On Mon, 2012-07-30 at 17:35 +0100, Olaf Hering wrote:
> > # HG changeset patch
> > # User Olaf Hering <olaf@aepfle.de>
> > # Date 1343666089 -7200
> > # Node ID ccbebdbe44da0460408172338af9daab31e56788
> > # Parent  db8adce4f09307a90f96103f7fd67efa97fc9ac0
> > stubdom: disable parallel build
> > 
> > Recently I changed my rpm xen.spec file from doing
> > 'make -C tools -j N && make stubdom' to 'make -j N stubdom' because
> > stubdom depends on tools, so both get built.
> > The result was the failure below.
> 
> I always do -j12 builds of the stubdom sub=tree and I don't see
> parallelisation issues. The stubdom build is pretty big and time
> consuming so it would be a shame to needlessly linearise it.

Is it always a clean build?
For me nothing in stubdom is built in parallel, if I interpret the
repated output of 'ps fax' right.

Olaf

> > make[2]: *** INTERNAL: readdir: Bad file descriptor
> 
> This is an internal error inside make? That's surely more critical than
> any problem with our build system and needs to be solved before adding
> workarounds to our tree.

Perhaps there is an issue, yes. I will see if I find something.

Olaf

Re: [PATCH] stubdom: disable parallel build

[Ian Campbell]

On Tue, 2012-07-31 at 09:44 +0100, Olaf Hering wrote:
> On Tue, Jul 31, Ian Campbell wrote:
> 
> > On Mon, 2012-07-30 at 17:35 +0100, Olaf Hering wrote:
> > > # HG changeset patch
> > > # User Olaf Hering <olaf@aepfle.de>
> > > # Date 1343666089 -7200
> > > # Node ID ccbebdbe44da0460408172338af9daab31e56788
> > > # Parent  db8adce4f09307a90f96103f7fd67efa97fc9ac0
> > > stubdom: disable parallel build
> > > 
> > > Recently I changed my rpm xen.spec file from doing
> > > 'make -C tools -j N && make stubdom' to 'make -j N stubdom' because
> > > stubdom depends on tools, so both get built.
> > > The result was the failure below.
> > 
> > I always do -j12 builds of the stubdom sub=tree and I don't see
> > parallelisation issues. The stubdom build is pretty big and time
> > consuming so it would be a shame to needlessly linearise it.
> 
> Is it always a clean build?

It always works if that is what you are asking.
> For me nothing in stubdom is built in parallel, if I interpret the
> repated output of 'ps fax' right.

I see at least some parallelism, although I don't know what proportion
of the build is.

> 
> Olaf
> 
> > > make[2]: *** INTERNAL: readdir: Bad file descriptor
> > 
> > This is an internal error inside make? That's surely more critical than
> > any problem with our build system and needs to be solved before adding
> > workarounds to our tree.
> 
> Perhaps there is an issue, yes. I will see if I find something.
> 
> Olaf

Re: [PATCH] stubdom: disable parallel build

[Olaf Hering]

On Tue, Jul 31, Ian Campbell wrote:

> I see at least some parallelism, although I don't know what proportion
> of the build is.

The extraction happens in parallel for me.

Olaf

Re: [PATCH] stubdom: disable parallel build

[Olaf Hering]

On Tue, Jul 31, Ian Campbell wrote:

> > make[2]: *** INTERNAL: readdir: Bad file descriptor
> 
> This is an internal error inside make? That's surely more critical than
> any problem with our build system and needs to be solved before adding
> workarounds to our tree.

I see the errors only with make 3.82, which is shipped with openSuSE
11.4 and later. make 3.81 from SLES11 shows no issues.

Olaf

Re: [PATCH] stubdom: disable parallel build

[Olaf Hering]

On Tue, Jul 31, Ian Campbell wrote:

> I suspect something is not quite right in the top level Makefile WRT the
> dependencies between tools and stubdom builds.

Doing a s@$(CROSS_MAKE)@$(MAKE) DESTDIR=@g in stubdom/Makefile fixes it
for me:


# HG changeset patch
# Parent db8adce4f09307a90f96103f7fd67efa97fc9ac0
stubdom: disable parallel build

Recently I changed my rpm xen.spec file from doing
'make -C tools -j N && make stubdom' to 'make -j N stubdom' because
stubdom depends on tools, so both get built.
The result was the failure below.

....
mkdir -p grub-x86_64
CPPFLAGS="-isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include -D__MINIOS__ -DHAVE_LIBC -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/posix -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../tools/xenstore  -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/x86 -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/x86/x86_64 -U __linux__ -U __FreeBSD__ -U __sun__ -nostdinc -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os/include/posix -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/cross-root-x86_64/x86_64-xen-elf/include -isystem /usr/lib64/gcc/x86_64
 -suse-linux/4.7/include -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/lwip-x86_64/src/include -isystem /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/lwip-x86_64/src/include/ipv4 -I/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/include -I/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../xen/include" CFLAGS="-mno-red-zone -O1 -fno-omit-frame-pointer  -m64 -mno-red-zone -fno-reorder-blocks -fno-asynchronous-unwind-tables -m64 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable   -fno-stack-protector -fno-exceptions" make DESTDIR= -C grub OBJ_DIR=/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/grub-x86_64
make[2]: Entering directory `/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/grub'
make[2]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
make[2]: *** INTERNAL: readdir: Bad file descriptor
.  Stop.
make[2]: Makefile: Field 'stem' not cached: Makefile

make[2]: Leaving directory `/home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/grub'
make[1]: *** [grub] Error 2
[ -d mini-os-x86_64-xenstore ] || \
for i in $(cd /home/abuild/rpmbuild/BUILD/xen-4.2.25602/non-dbg/stubdom/../extras/mini-os ; find . -type d) ; do \
                mkdir -p mini-os-x86_64-xenstore/$i ; \
done
....

Expanding every occurrence of CROSS_MAKE seems to fix this error. It also has
the nice side effect of actually enabling parallel build for stubdom.
According to the GNU make documentation $(MAKE) becomes its special meaning
only if it appears directly in the recipe.

Signed-off-by: Olaf Hering <olaf@aepfle.de>

diff -r db8adce4f093 stubdom/Makefile
--- a/stubdom/Makefile
+++ b/stubdom/Makefile
@@ -76,8 +76,6 @@ TARGET_LDFLAGS += -nostdlib -L$(CROSS_PR
 
 TARGETS=ioemu c caml grub xenstore
 
-CROSS_MAKE := $(MAKE) DESTDIR=
-
 .PHONY: all
 all: build
 ifeq ($(STUBDOM_SUPPORTED),1)
@@ -113,8 +111,8 @@ cross-newlib: $(NEWLIB_STAMPFILE)
 	mkdir -p newlib-$(XEN_TARGET_ARCH)
 	( cd newlib-$(XEN_TARGET_ARCH) && \
 	  CC_FOR_TARGET="$(CC) $(TARGET_CPPFLAGS) $(TARGET_CFLAGS) $(NEWLIB_CFLAGS)" AR_FOR_TARGET=$(AR) LD_FOR_TARGET=$(LD) RANLIB_FOR_TARGET=$(RANLIB) ../newlib-$(NEWLIB_VERSION)/configure --prefix=$(CROSS_PREFIX) --verbose --target=$(GNU_TARGET_ARCH)-xen-elf --enable-newlib-io-long-long --disable-multilib && \
-	  $(CROSS_MAKE) && \
-	  $(CROSS_MAKE) install )
+	  $(MAKE) DESTDIR= && \
+	  $(MAKE) DESTDIR= install )
 
 ############
 # Cross-zlib
@@ -133,8 +131,8 @@ cross-zlib: $(ZLIB_STAMPFILE)
 $(ZLIB_STAMPFILE): zlib-$(XEN_TARGET_ARCH) $(NEWLIB_STAMPFILE)
 	( cd $< && \
 	  CFLAGS="$(TARGET_CPPFLAGS) $(TARGET_CFLAGS)" CC=$(CC) ./configure --prefix=$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf && \
-	  $(CROSS_MAKE) libz.a && \
-	  $(CROSS_MAKE) install )
+	  $(MAKE) DESTDIR= libz.a && \
+	  $(MAKE) DESTDIR= install )
 
 ##############
 # Cross-libpci
@@ -158,7 +156,7 @@ cross-libpci: $(LIBPCI_STAMPFILE)
 	  chmod u+w lib/config.h && \
 	  echo '#define PCILIB_VERSION "$(LIBPCI_VERSION)"' >> lib/config.h && \
 	  ln -sf ../../libpci.config.mak lib/config.mk && \
-	  $(CROSS_MAKE) CC="$(CC) $(TARGET_CPPFLAGS) $(TARGET_CFLAGS) -I$(call realpath,$(MINI_OS)/include)" lib/libpci.a && \
+	  $(MAKE) DESTDIR= CC="$(CC) $(TARGET_CPPFLAGS) $(TARGET_CFLAGS) -I$(call realpath,$(MINI_OS)/include)" lib/libpci.a && \
 	  $(INSTALL_DATA) lib/libpci.a $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/lib/ && \
 	  $(INSTALL_DIR) $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/include/pci && \
 	  $(INSTALL_DATA) lib/config.h lib/header.h lib/pci.h lib/types.h $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/include/pci/ \
@@ -203,8 +201,8 @@ cross-ocaml: $(OCAML_STAMPFILE)
 		-no-pthread -no-shared-libs -no-tk -no-curses \
 		-cc "$(CC) -U_FORTIFY_SOURCE -fno-stack-protector -mno-red-zone"
 	$(foreach i,$(MINIOS_HASNOT),sed -i 's,^\(#define HAS_$(i)\),//\1,' ocaml-$(XEN_TARGET_ARCH)/config/s.h ; )
-	$(CROSS_MAKE) -C ocaml-$(XEN_TARGET_ARCH) world
-	$(CROSS_MAKE) -C ocaml-$(XEN_TARGET_ARCH) opt
+	$(MAKE) DESTDIR= -C ocaml-$(XEN_TARGET_ARCH) world
+	$(MAKE) DESTDIR= -C ocaml-$(XEN_TARGET_ARCH) opt
 	$(MAKE) -C ocaml-$(XEN_TARGET_ARCH) install
 	touch $@
 
@@ -219,7 +217,7 @@ QEMU_ROOT := $(shell if [ -d "$(CONFIG_Q
 
 ifeq ($(QEMU_ROOT),.)
 $(XEN_ROOT)/tools/qemu-xen-traditional-dir:
-	$(CROSS_MAKE) -C $(XEN_ROOT)/tools qemu-xen-traditional-dir-find
+	$(MAKE) DESTDIR= -C $(XEN_ROOT)/tools qemu-xen-traditional-dir-find
 
 ioemu/linkfarm.stamp: $(XEN_ROOT)/tools/qemu-xen-traditional-dir
 	mkdir -p ioemu
@@ -250,7 +248,7 @@ mk-headers-$(XEN_TARGET_ARCH): ioemu/lin
           ( [ -h include/xen/libelf ] || ln -sf $(XEN_ROOT)/tools/include/xen/libelf include/xen/libelf ) && \
 	  mkdir -p include/xen-foreign && \
 	  ln -sf $(wildcard $(XEN_ROOT)/tools/include/xen-foreign/*) include/xen-foreign/ && \
-	  $(CROSS_MAKE) -C include/xen-foreign/ && \
+	  $(MAKE) DESTDIR= -C include/xen-foreign/ && \
 	  ( [ -h include/xen/foreign ] || ln -sf ../xen-foreign include/xen/foreign )
 	mkdir -p libxc-$(XEN_TARGET_ARCH)
 	[ -h libxc-$(XEN_TARGET_ARCH)/Makefile ] || ( cd libxc-$(XEN_TARGET_ARCH) && \
@@ -267,7 +265,7 @@ mk-headers-$(XEN_TARGET_ARCH): ioemu/lin
 	  ln -sf $(XEN_ROOT)/tools/xenstore/*.c . && \
 	  ln -sf $(XEN_ROOT)/tools/xenstore/*.h . && \
 	  ln -sf $(XEN_ROOT)/tools/xenstore/Makefile . )
-	$(CROSS_MAKE) -C $(MINI_OS) links
+	$(MAKE) DESTDIR= -C $(MINI_OS) links
 	touch mk-headers-$(XEN_TARGET_ARCH)
 
 TARGETS_MINIOS=$(addprefix mini-os-$(XEN_TARGET_ARCH)-,$(TARGETS))
@@ -284,7 +282,7 @@ TARGETS_MINIOS=$(addprefix mini-os-$(XEN
 .PHONY: libxc
 libxc: libxc-$(XEN_TARGET_ARCH)/libxenctrl.a libxc-$(XEN_TARGET_ARCH)/libxenguest.a
 libxc-$(XEN_TARGET_ARCH)/libxenctrl.a: cross-zlib
-	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C libxc-$(XEN_TARGET_ARCH)
+	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(MAKE) DESTDIR= -C libxc-$(XEN_TARGET_ARCH)
 
  libxc-$(XEN_TARGET_ARCH)/libxenguest.a: libxc-$(XEN_TARGET_ARCH)/libxenctrl.a
 
@@ -302,7 +300,7 @@ ioemu: cross-zlib cross-libpci libxc
 	    TARGET_CFLAGS="$(TARGET_CFLAGS)" \
 	    TARGET_LDFLAGS="$(TARGET_LDFLAGS)" \
 	    $(QEMU_ROOT)/xen-setup-stubdom )
-	$(CROSS_MAKE) -C ioemu -f $(QEMU_ROOT)/Makefile
+	$(MAKE) DESTDIR= -C ioemu -f $(QEMU_ROOT)/Makefile
 
 ######
 # caml
@@ -310,7 +308,7 @@ ioemu: cross-zlib cross-libpci libxc
 
 .PHONY: caml
 caml: $(CROSS_ROOT)
-	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) OCAMLC_CROSS_PREFIX=$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/bin/
+	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(MAKE) DESTDIR= -C $@ LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) OCAMLC_CROSS_PREFIX=$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/bin/
 
 ###
 # C
@@ -318,7 +316,7 @@ caml: $(CROSS_ROOT)
 
 .PHONY: c
 c: $(CROSS_ROOT)
-	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) 
+	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(MAKE) DESTDIR= -C $@ LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) 
 
 ######
 # Grub
@@ -337,7 +335,7 @@ grub-upstream: grub-$(GRUB_VERSION).tar.
 .PHONY: grub
 grub: grub-upstream $(CROSS_ROOT)
 	mkdir -p grub-$(XEN_TARGET_ARCH)
-	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ OBJ_DIR=$(CURDIR)/grub-$(XEN_TARGET_ARCH)
+	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(MAKE) DESTDIR= -C $@ OBJ_DIR=$(CURDIR)/grub-$(XEN_TARGET_ARCH)
 
 ##########
 # xenstore
@@ -345,7 +343,7 @@ grub: grub-upstream $(CROSS_ROOT)
 
 .PHONY: xenstore
 xenstore: $(CROSS_ROOT)
-	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(CROSS_MAKE) -C $@ xenstored.a CONFIG_STUBDOM=y
+	CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(MAKE) DESTDIR= -C $@ xenstored.a CONFIG_STUBDOM=y
 
 ########
 # minios
@@ -354,23 +352,23 @@ xenstore: $(CROSS_ROOT)
 .PHONY: ioemu-stubdom
 ioemu-stubdom: APP_OBJS=$(CURDIR)/ioemu/i386-stubdom/qemu.a $(CURDIR)/ioemu/i386-stubdom/libqemu.a $(CURDIR)/ioemu/libqemu_common.a
 ioemu-stubdom: mini-os-$(XEN_TARGET_ARCH)-ioemu lwip-$(XEN_TARGET_ARCH) libxc ioemu
-	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/ioemu-minios.cfg" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS="$(APP_OBJS)"
+	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/ioemu-minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS="$(APP_OBJS)"
 
 .PHONY: caml-stubdom
 caml-stubdom: mini-os-$(XEN_TARGET_ARCH)-caml lwip-$(XEN_TARGET_ARCH) libxc cross-ocaml caml
-	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/caml/minios.cfg" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS="$(CURDIR)/caml/main-caml.o $(CURDIR)/caml/caml.o $(CAMLLIB)/libasmrun.a"
+	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/caml/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS="$(CURDIR)/caml/main-caml.o $(CURDIR)/caml/caml.o $(CAMLLIB)/libasmrun.a"
 
 .PHONY: c-stubdom
 c-stubdom: mini-os-$(XEN_TARGET_ARCH)-c lwip-$(XEN_TARGET_ARCH) libxc c
-	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/c/minios.cfg" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS=$(CURDIR)/c/main.a
+	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/c/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< LWIPDIR=$(CURDIR)/lwip-$(XEN_TARGET_ARCH) APP_OBJS=$(CURDIR)/c/main.a
 
 .PHONY: pv-grub
 pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxc grub
-	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/grub/minios.cfg" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a
+	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/grub/minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a
 
 .PHONY: xenstore-stubdom
 xenstore-stubdom: mini-os-$(XEN_TARGET_ARCH)-xenstore libxc xenstore
-	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/xenstore-minios.cfg" $(CROSS_MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/xenstore/xenstored.a
+	DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="$(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" MINIOS_CONFIG="$(CURDIR)/xenstore-minios.cfg" $(MAKE) DESTDIR= -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/xenstore/xenstored.a
 
 #########
 # install
@@ -412,13 +410,13 @@ clean:
 	rm -fr mini-os-$(XEN_TARGET_ARCH)-caml
 	rm -fr mini-os-$(XEN_TARGET_ARCH)-grub
 	rm -fr mini-os-$(XEN_TARGET_ARCH)-xenstore
-	$(CROSS_MAKE) -C caml clean
-	$(CROSS_MAKE) -C c clean
+	$(MAKE) DESTDIR= -C caml clean
+	$(MAKE) DESTDIR= -C c clean
 	rm -fr grub-$(XEN_TARGET_ARCH)
 	rm -f $(STUBDOMPATH)
-	[ ! -d libxc-$(XEN_TARGET_ARCH) ] || $(CROSS_MAKE) -C libxc-$(XEN_TARGET_ARCH) clean
-	-[ ! -d ioemu ] || $(CROSS_MAKE) -C ioemu clean
-	-[ ! -d xenstore ] || $(CROSS_MAKE) -C xenstore clean
+	[ ! -d libxc-$(XEN_TARGET_ARCH) ] || $(MAKE) DESTDIR= -C libxc-$(XEN_TARGET_ARCH) clean
+	-[ ! -d ioemu ] || $(MAKE) DESTDIR= -C ioemu clean
+	-[ ! -d xenstore ] || $(MAKE) DESTDIR= -C xenstore clean
 
 # clean the cross-compilation result
 .PHONY: crossclean

Re: [PATCH] stubdom: disable parallel build

[Ian Campbell]

On Tue, 2012-07-31 at 13:50 +0100, Olaf Hering wrote:
> On Tue, Jul 31, Ian Campbell wrote:
> 
> > I suspect something is not quite right in the top level Makefile WRT the
> > dependencies between tools and stubdom builds.
> 
> Doing a s@$(CROSS_MAKE)@$(MAKE) DESTDIR=@g in stubdom/Makefile fixes it
> for me:

Can anyone who understands Make explain why this should make a
difference?

I would have expected that
	FOO := $(BAR) baz
...
	$(FOO) bif
ought to be identical to
	$(BAR) baz bif

Is that not the case?

Ian.

Re: [PATCH] stubdom: disable parallel build

[Olaf Hering]

On Wed, Aug 01, Ian Campbell wrote:

> On Tue, 2012-07-31 at 13:50 +0100, Olaf Hering wrote:
> > On Tue, Jul 31, Ian Campbell wrote:
> > 
> > > I suspect something is not quite right in the top level Makefile WRT the
> > > dependencies between tools and stubdom builds.
> > 
> > Doing a s@$(CROSS_MAKE)@$(MAKE) DESTDIR=@g in stubdom/Makefile fixes it
> > for me:
> 
> Can anyone who understands Make explain why this should make a
> difference?
> 
> I would have expected that
> 	FOO := $(BAR) baz
> ...
> 	$(FOO) bif
> ought to be identical to
> 	$(BAR) baz bif
> 
> Is that not the case?

MAKE is handled special if written in a receipe:
http://www.gnu.org/software/make/manual/html_node/MAKE-Variable.html

That does not really explain why it makes a difference in the case of
stubdom.

Olaf