[Buildroot] [PATCH] dmalloc: fix parallel builds

[Buildroot] [PATCH] dmalloc: fix parallel builds

[Alex Suykov]

Signed-off-by: Alex Suykov <alex.suykov@gmail.com>
---
 This patch should fix
 http://autobuild.buildroot.net/results/2cc40ae3fc8b7a287c43528b3e4ffdbcd5033c09/

 package/dmalloc/dmalloc-deps.patch | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 package/dmalloc/dmalloc-deps.patch

diff --git a/package/dmalloc/dmalloc-deps.patch b/package/dmalloc/dmalloc-deps.patch
new file mode 100644
index 0000000..c717e1a
--- /dev/null
+++ b/package/dmalloc/dmalloc-deps.patch
@@ -0,0 +1,14 @@
+Missing dmallocc.o <- dmalloc.h dependency may break parallel builds.
+dmalloc.h is generated, and may not be around by the time gcc starts
+parsing dmallocc.cc.
+
+--- a/Makefile.in
++++ b/Makefile.in
+@@ -399,6 +399,7 @@
+ compat.o: compat.c dmalloc.h conf.h settings.h compat.h dmalloc_loc.h
+ dmalloc.o: dmalloc.c dmalloc_argv.h conf.h settings.h dmalloc.h \
+  compat.h debug_tok.h dmalloc_loc.h env.h error_val.h version.h
++dmallocc.o: dmallocc.cc dmalloc.h return.h conf.h settings.h
+ dmalloc_argv.o: dmalloc_argv.c conf.h settings.h dmalloc_argv.h \
+  dmalloc_argv_loc.h compat.h
+ dmalloc_rand.o: dmalloc_rand.c conf.h settings.h dmalloc_rand.h
-- 
2.0.3

[Buildroot] [PATCH] dmalloc: fix parallel builds

[Thomas Petazzoni]

Dear Alex Suykov,

On Sat, 3 Jan 2015 13:05:03 +0200, Alex Suykov wrote:
> Signed-off-by: Alex Suykov <alex.suykov@gmail.com>
> ---
>  This patch should fix
>  http://autobuild.buildroot.net/results/2cc40ae3fc8b7a287c43528b3e4ffdbcd5033c09/
> 
>  package/dmalloc/dmalloc-deps.patch | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>  create mode 100644 package/dmalloc/dmalloc-deps.patch

While it indeed looks like a parallel build issue, I haven't been able
to reproduce it: I've started a loop building dmalloc over and over
again on my build server (on which the issue was reproduced by the
autobuilders), and never managed to see it after an entire night
looping on dmalloc builds. So I'm a bit confused on this one.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] dmalloc: fix parallel builds

[Alex Suykov]

Sun, Jan 04, 2015 at 06:35:13PM +0100, Thomas Petazzoni wrote:

> On Sat, 3 Jan 2015 13:05:03 +0200, Alex Suykov wrote:
> > Signed-off-by: Alex Suykov <alex.suykov@gmail.com>
> > ---
> >  This patch should fix
> >  http://autobuild.buildroot.net/results/2cc40ae3fc8b7a287c43528b3e4ffdbcd5033c09/
> > 
> >  package/dmalloc/dmalloc-deps.patch | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> >  create mode 100644 package/dmalloc/dmalloc-deps.patch
> 
> While it indeed looks like a parallel build issue, I haven't been able
> to reproduce it: I've started a loop building dmalloc over and over
> again on my build server (on which the issue was reproduced by the
> autobuilders), and never managed to see it after an entire night
> looping on dmalloc builds. So I'm a bit confused on this one.

The following sequence seems to trigger it more or less reliably for me:

    ./configure --disable-static --enable-shared --enable-cxx --enable-threads
    sed -i -e "s at cat \$(srcdir)/dmalloc.h.1.*@&; sleep 1@" Makefile
    make clean
    make -j4

What happens is a race condition between "cat ...; mv ... dmalloc.h"
and "g++ -c dmallocc.cc". As long as cat-mv pair is faster, it works well.
But if there's a delay, it breaks.

This should be rare, I can only reproduce it with an artificial delay,
and only with --disable-static. But on the other hand, it did happen
on autobuild somehow.

[Buildroot] [PATCH] dmalloc: fix parallel builds

[Thomas Petazzoni]

Dear Alex Suykov,

On Sun, 4 Jan 2015 21:02:04 +0200, Alex Suykov wrote:

> The following sequence seems to trigger it more or less reliably for me:
> 
>     ./configure --disable-static --enable-shared --enable-cxx --enable-threads
>     sed -i -e "s at cat \$(srcdir)/dmalloc.h.1.*@&; sleep 1@" Makefile
>     make clean
>     make -j4
> 
> What happens is a race condition between "cat ...; mv ... dmalloc.h"
> and "g++ -c dmallocc.cc". As long as cat-mv pair is faster, it works well.
> But if there's a delay, it breaks.
> 
> This should be rare, I can only reproduce it with an artificial delay,
> and only with --disable-static. But on the other hand, it did happen
> on autobuild somehow.

Ok, thanks a lot for the explanation! I've committed your patch, with a
few minor tweaks:

    [Thomas:
     - Rename patch to the new naming convention.
     - Add SoB line from Alex inside the patch itself.
     - Adjust the commit log to contain the reference to the autobuilder
       failure.]

If you're using dmalloc, I just noticed that the version we have in
Buildroot is 5.4.3, while upstream has a 5.5.2 release. If you want to
send a patch to bump the dmalloc version, that would be great.

Also, don't hesitate to send your patch to the upstream dmalloc
developers.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] dmalloc: fix parallel builds

[Alex Suykov]

Sun, Jan 04, 2015 at 09:41:57PM +0100, Thomas Petazzoni wrote:

> If you're using dmalloc, I just noticed that the version we have in
> Buildroot is 5.4.3, while upstream has a 5.5.2 release. If you want to
> send a patch to bump the dmalloc version, that would be great.

No, I don't use it myself. I was just looking through autobuild logs
for unrelated reasons and kind of decided to fix what was clearly fixable.

Anyway, I'll send a version bump patch now, it looks like the changes
are minor, and both Gentoo and OpenWRT have 5.5.2 already.

I wonder if it makes sense to contact whoever sent that config
to autobuild and ask for some feedback. Since dmalloc was enabled
there, it was probably used for something.

[Buildroot] [PATCH] dmalloc: fix parallel builds

[Thomas Petazzoni]

Dear Alex Suykov,

On Wed, 7 Jan 2015 15:02:10 +0200, Alex Suykov wrote:

> No, I don't use it myself. I was just looking through autobuild logs
> for unrelated reasons and kind of decided to fix what was clearly fixable.
> 
> Anyway, I'll send a version bump patch now, it looks like the changes
> are minor, and both Gentoo and OpenWRT have 5.5.2 already.

Good, thanks!

> I wonder if it makes sense to contact whoever sent that config
> to autobuild and ask for some feedback. Since dmalloc was enabled
> there, it was probably used for something.

The configs tested in the autobuilders are randomly generated:

 * A random architecture/toolchain configuration is selected amongst
   http://autobuild.buildroot.org/toolchains/configs/.

 * A random selection of packages is generated using
   "randpackageconfig".

So the configuration tested in the autobuilders are not submitted by
people, they are randomly generated by a script. That's how we test so
many combinations of packages/platforms.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com