libxenstore.so Makefile dependency issue

libxenstore.so Makefile dependency issue

[Olaf Hering]

This is the second time I run into this linking issue. A few days ago it
happend with a make -j 4 or 8 during automated rpm package build. Now it
happend with a fresh tree. My local system is openSuSE 11.4 with make
3.82, the failed automated build was either openSUSE 11.4 or 12.1.

For some reason the libxenstore.so symlink is created after
init-xenstore-domain is about to be linked. The ln command is not
visible in this output, I saw the ln invocation in the rpm build log.

Hmm, and for some reason the symlink was not created anyway in my local
build. A second make run worked ok, libxenstore.so was created.  It
looks like I can reproduce it by running make clean in tools/xenstore.

Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE,
and libxenstore.so is also a target. So its not clear to me how make can
miss that, or how the dependencies should be listed.

Any idea whats going on here?

Olaf

...
gcc  -O1 -fno-omit-frame-pointer -m64 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement   -D__XEN_TOOLS__ -MMD -MF .xenstored_posix.o.d  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -Werror -I. -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/include  -c -o xenstored_posix.o xenstored_posix.c
gcc  -O1 -fno-omit-frame-pointer -m64 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement   -D__XEN_TOOLS__ -MMD -MF .init-xenstore-domain.o.d  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -Werror -I. -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/include -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/include  -c -o init-xenstore-domain.o init-xenstore-domain.c
gcc     -Wl,-soname -Wl,libxenstore.so.3.0 -shared -o libxenstore.so.3.0.1 xs.opic xs_lib.opic  -lpthread
ar rcs libxenstore.a xs.o xs_lib.o
gcc     xs_tdb_dump.o utils.o tdb.o talloc.o -o xs_tdb_dump
gcc     xenstored_core.o xenstored_watch.o xenstored_domain.o xenstored_transaction.o xs_lib.o talloc.o utils.o tdb.o hashtable.o xenstored_linux.o xenstored_posix.o /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc/libxenctrl.so  -o xenstored
gcc     init-xenstore-domain.o libxenstore.so /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc/libxenctrl.so /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc/libxenguest.so /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/xenstore/libxenstore.so -o init-xenstore-domain
gcc: libxenstore.so: Datei oder Verzeichnis nicht gefunden
gcc: /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/xenstore/libxenstore.so: Datei oder Verzeichnis nicht gefunden
make[3]: *** [init-xenstore-domain] Fehler 1
make[3]: *** Warte auf noch nicht beendete Prozesse...
make[3]: Leaving directory `/work/xen/xendebug/xen-unstable.hg/tools/xenstore'
make[2]: *** [subdir-install-xenstore] Fehler 2
make[2]: Leaving directory `/work/xen/xendebug/xen-unstable.hg/tools'
make[1]: *** [subdirs-install] Fehler 2
make[1]: Leaving directory `/work/xen/xendebug/xen-unstable.hg/tools'
make: *** [install-tools] Fehler 2
make: *** Warte auf noch nicht beendete Prozesse...

Re: libxenstore.so Makefile dependency issue

[Olaf Hering]

On Mon, Feb 20, Olaf Hering wrote:

> Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE,
> and libxenstore.so is also a target. So its not clear to me how make can
> miss that, or how the dependencies should be listed.

With this change, putting init-xenstore-domain first, I can not reproduce it anymore.


-ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored init-xenstore-domain
+ALL_TARGETS = init-xenstore-domain libxenstore.so libxenstore.a clients xs_tdb_dump xenstored

Is this correct?

Olaf

Re: libxenstore.so Makefile dependency issue

[Ian Campbell]

On Mon, 2012-02-20 at 19:26 +0000, Olaf Hering wrote:
> This is the second time I run into this linking issue. A few days ago it
> happend with a make -j 4 or 8 during automated rpm package build. Now it
> happend with a fresh tree. My local system is openSuSE 11.4 with make
> 3.82, the failed automated build was either openSUSE 11.4 or 12.1.
> 
> For some reason the libxenstore.so symlink is created after
> init-xenstore-domain is about to be linked. The ln command is not
> visible in this output, I saw the ln invocation in the rpm build log.
> 
> Hmm, and for some reason the symlink was not created anyway in my local
> build. A second make run worked ok, libxenstore.so was created.  It
> looks like I can reproduce it by running make clean in tools/xenstore.
> 
> Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE,
> and libxenstore.so is also a target. So its not clear to me how make can
> miss that, or how the dependencies should be listed.
> 
> Any idea whats going on here?

It's pretty odd isn't it.

I tried:
        $ make -C tools/xenstore/ clean
        $ make -C tools/xenstore/ -j12
and couldn't reproduce. I see the ln before the link lines, even with
bigger and smaller -jN.

"make -d" will tell you make's thought processes, might give a hint?

Could it be your filesystem? Something odd to do with timestamps on
symlinks which upsets your version of make perhaps? (I'm on ext3)

I'm really grasping at straws there...

The chain of rules for making the symlinks is a bit convoluted. I'd be
inclined to have:
        libxenstore.so libxenstore.so.$(MAJOR) libxenstore.so.$(MAJOR).$(MINOR): xs.opic xs_lib.opic
        	$(CC)....
        	ln -sf libxenstore.so.$(MAJOR).$(MINOR) libxenstore.so.$(MAJOR)
        	ln -sf libxenstore.so.$(MAJOR) libxenstore.so

But I don't see why it would help or matter in practice.

Ian.

Re: libxenstore.so Makefile dependency issue

[Olaf Hering]

On Tue, Feb 21, Ian Campbell wrote:

> On Mon, 2012-02-20 at 19:26 +0000, Olaf Hering wrote:
> > Any idea whats going on here?
> 
> It's pretty odd isn't it.
> 
> I tried:
>         $ make -C tools/xenstore/ clean
>         $ make -C tools/xenstore/ -j12
> and couldn't reproduce. I see the ln before the link lines, even with
> bigger and smaller -jN.

I use this script 'time bash ../build.sh -d xen tools &> output.txt':

#!/bin/bash
set -x
unset LANG
unset ${!LC_*}
tgt="xen tools"
if test -n "$1"
then
	tgt=$@
fi
if pushd tools/xenstore
then
	make clean
	popd
fi
time XEN_DOMAIN=localhost DISTDIR=/dev/shm/install-${PWD//\//_} make -j 3 ${tgt} ; echo $?

It seems to trigger with either 'make -d' or 'make -d xen tools', but
seldom with 'make -d tools'.
It does not trigger with -j 1, but with -j 2 or 3.

> "make -d" will tell you make's thought processes, might give a hint?

Thanks for that hint, I will try to make sense of the (huge) output.

> Could it be your filesystem? Something odd to do with timestamps on
> symlinks which upsets your version of make perhaps? (I'm on ext3)

I'm on ext3.  libxenstore.so is not there, thats the issue.

I wonder why libxenstore.so and libxenstore.a is listed at all in
ALL_TARGETS? Perhaps that confuses make? But removing both does not help
either. To me it looks like $(LIBXENSTORE) does not serve as a proper
dependency.

Olaf

Re: libxenstore.so Makefile dependency issue

[Olaf Hering]

On Tue, Feb 21, Ian Campbell wrote:

> On Mon, 2012-02-20 at 19:26 +0000, Olaf Hering wrote:
> > Any idea whats going on here?
> 
> It's pretty odd isn't it.

I can not reproduce the link failure with make 3.81.

Olaf

Re: libxenstore.so Makefile dependency issue

[Ian Jackson]

Olaf Hering writes ("[Xen-devel] libxenstore.so Makefile dependency issue"):
> 
> This is the second time I run into this linking issue. A few days ago it
> happend with a make -j 4 or 8 during automated rpm package build. Now it
> happend with a fresh tree. My local system is openSuSE 11.4 with make
> 3.82, the failed automated build was either openSUSE 11.4 or 12.1.
> 
> For some reason the libxenstore.so symlink is created after
> init-xenstore-domain is about to be linked. The ln command is not
> visible in this output, I saw the ln invocation in the rpm build log.
> 
> Hmm, and for some reason the symlink was not created anyway in my local
> build. A second make run worked ok, libxenstore.so was created.  It
> looks like I can reproduce it by running make clean in tools/xenstore.
> 
> Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE,
> and libxenstore.so is also a target. So its not clear to me how make can
> miss that, or how the dependencies should be listed.

This is indeed mysterious.  I haven't managed to reproduce it either.

I tried the patch below, with make -j1000, and everything worked the
way it should do.

Ian.

diff -r 6a34a42a2b5d tools/xenstore/Makefile
--- a/tools/xenstore/Makefile	Tue Feb 21 18:01:04 2012 +0000
+++ b/tools/xenstore/Makefile	Tue Feb 21 18:23:46 2012 +0000
@@ -82,7 +82,7 @@ libxenstore.so.$(MAJOR): libxenstore.so.
 xs.opic: CFLAGS += -DUSE_PTHREAD
 
 libxenstore.so.$(MAJOR).$(MINOR): xs.opic xs_lib.opic
-	$(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenstore.so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $^ $(SOCKET_LIBS) -lpthread $(APPEND_LDFLAGS)
+	set -x; while test ! -f go$$$$; do sleep 10; done; $(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenstore.so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $^ $(SOCKET_LIBS) -lpthread $(APPEND_LDFLAGS)
 
 libxenstore.a: xs.o xs_lib.o
 	$(AR) rcs $@ $^

Re: libxenstore.so Makefile dependency issue

[Olaf Hering]

On Tue, Feb 21, Ian Jackson wrote:

> This is indeed mysterious.  I haven't managed to reproduce it either.
> 
> I tried the patch below, with make -j1000, and everything worked the
> way it should do.

Is this with make 3.82?

I think its time to ask on a make mailing list for advice. The NEWS file
for 3.82 did not contain any (obvious related) incompatible change.

Olaf

Re: libxenstore.so Makefile dependency issue

[Ian Jackson]

Olaf Hering writes ("Re: [Xen-devel] libxenstore.so Makefile dependency issue"):
> Is this with make 3.82?
> 
> I think its time to ask on a make mailing list for advice. The NEWS file
> for 3.82 did not contain any (obvious related) incompatible change.

3.81.

You could try the patch from my last mail which I think should enable
to you repro the bug every time.

Ian.

Re: libxenstore.so Makefile dependency issue

[Olaf Hering]

On Tue, Feb 21, Ian Jackson wrote:

> Olaf Hering writes ("Re: [Xen-devel] libxenstore.so Makefile dependency issue"):
> > Is this with make 3.82?
> > 
> > I think its time to ask on a make mailing list for advice. The NEWS file
> > for 3.82 did not contain any (obvious related) incompatible change.
> 
> 3.81.
> 
> You could try the patch from my last mail which I think should enable
> to you repro the bug every time.

Not really. If I put a simple sleep 1 or 2 in there libxenstore.so is
not created in time, but sleep 3 and more helps.

Olaf