[Buildroot] [Bug 4808] New: ccache may build against wrong zlib

[Buildroot] [Bug 4808] New: ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

           Summary: ccache may build against wrong zlib
           Product: buildroot
           Version: 2011.08
          Platform: All
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
        AssignedTo: unassigned at buildroot.uclibc.org
        ReportedBy: rsanchezs at infoglobal.es
                CC: buildroot at uclibc.org
   Estimated Hours: 0.0


Created attachment 4058
  --> https://bugs.busybox.net/attachment.cgi?id=4058
Patch against git head.

Hello:

  In one of my compilations I was startled that host-lzo failed to build. The
host architecture was incorrectly detected and configuration stage failed. From
this I found out that ccache was unable to run, due to a mismatch in the zlib
versions. I found out the reason a little later: buildroot ccache built against
system zlib, whereas by the moment when ccache failed host-zlib was installed
in the buildroot host directory.

  The solution I propose is adding host-zlib dependency to ccache, so cacche is
always build against host-zlib and not against system zlib.

  I'm attaching a patch against current git head.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #1 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2012-02-28 22:57:32 UTC ---
Unfortunately, I don't think the patch works properly: host-zlib will try to
use ccache before it has been built. If you want to do this, you have to ensure
that host-zlib is built with HOSTCC_NOCCACHE instead of HOSTCC, just like
ccache itself.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #2 from Ra?l S?nchez Siles <rsanchezs@infoglobal.es> 2012-02-29 13:40:27 UTC ---
Thanks for answering, Thomas. You are right. It's not working.
After a little bit more of research, I've found out the problem is that ccache
is build using rpath as LDFLAGS. This LDFLAGS are generic for all host packages
(as stated in package/Makefile.in)

I came accross the issue by chance when my distro upgrade zlib version so it
didn't match with the one provided by buildroot.

The solution could be either voiding/fixing LDFLAGS for host-ccache compilation
or removing embedded rpath in ccache binary. I think this second is the easier
way. But what do you think?

I'll provide a patch for the second.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #3 from Ra?l S?nchez Siles <rsanchezs@infoglobal.es> 2012-02-29 14:42:12 UTC ---
..and now I'm not sure if ccache is exactly to blame. I removed the rpath from
cache and tried to make again. I had the same error as before. This is the
result:
/repos/mcv_buildroot/output/build/host-lzo-2.05$
PATH=/home/rasasi/repos/mcv_buildroot/output/host/bin:/home/rasasi/repos/mcv_buildroot/output/host/usr/bin:/home/rasasi/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
AR="/usr/bin/ar" AS="/usr/bin/as" LD="/usr/bin/ld" NM="/usr/bin/nm"
CC="/home/rasasi/repos/mcv_buildroot/output/host/usr/bin/ccache /usr/bin/gcc"
GCC="/home/rasasi/repos/mcv_buildroot/output/host/usr/bin/ccache /usr/bin/gcc"
CXX="/home/rasasi/repos/mcv_buildroot/output/host/usr/bin/ccache /usr/bin/g++"
CPP="/usr/bin/cpp" CFLAGS="-O2
-I/home/rasasi/repos/mcv_buildroot/output/host/include
-I/home/rasasi/repos/mcv_buildroot/output/host/usr/include"
CXXFLAGS="-I/home/rasasi/repos/mcv_buildroot/output/host/include
-I/home/rasasi/repos/mcv_buildroot/output/host/usr/include"
LDFLAGS="-L/home/rasasi/repos/mcv_buildroot/output/host/lib
-L/home/rasasi/repos/mcv_buildroot/output/host/usr/lib
-Wl,-rpath,/home/rasasi/repos/mcv_buildroot/output/host/usr/lib"
PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 PKG_CONFIG_ALLOW_SYSTEM_LIBS=1
PKG_CONFIG="/home/rasasi/repos/mcv_buildroot/output/host/usr/bin/pkg-config"
PKG_CONFIG_SYSROOT_DIR="/"
PKG_CONFIG_LIBDIR="/home/rasasi/repos/mcv_buildroot/output/host/usr/lib/pkgconfig:/home/rasasi/repos/mcv_buildroot/output/host/usr/share/pkgconfig"
PERLLIB="/home/rasasi/repos/mcv_buildroot/output/host/usr/lib/perl"
LD_LIBRARY_PATH="/home/rasasi/repos/mcv_buildroot/output/host/usr/lib:"
CFLAGS="-O2 -I/home/rasasi/repos/mcv_buildroot/output/host/include
-I/home/rasasi/repos/mcv_buildroot/output/host/usr/include"
LDFLAGS="-L/home/rasasi/repos/mcv_buildroot/output/host/lib
-L/home/rasasi/repos/mcv_buildroot/output/host/usr/lib
-Wl,-rpath,/home/rasasi/repos/mcv_buildroot/output/host/usr/lib"  ./configure
--prefix="/home/rasasi/repos/mcv_buildroot/output/host/usr"
--sysconfdir="/home/rasasi/repos/mcv_buildroot/output/host/etc" --enable-shared
--disable-static
configure: Configuring LZO 2.05
checking build system type... Invalid configuration `x86_64-unknown-linux-':
machine `x86_64-unknown-linux' not recognized
configure: error: /bin/sh autoconf/config.sub x86_64-unknown-linux- failed

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #4 from Ra?l S?nchez Siles <rsanchezs@infoglobal.es> 2012-03-01 10:46:23 UTC ---
Hello:

I think I have a more consistent theory about this problem. But I'm not sure by
now how to solve it correctly.
The thing is that as I've mentioned before, host-ccache is build using rpath
that points to the buildroot host-zlib, so buildroot zlib is preferred over the
system one. Once I removed the rpath it didn't work as well, since the
configure command enviroment includes:
LD_LIBRARY_PATH="/home/rasasi/repos/mcv_buildroot/output/host/usr/lib:" which
again preferres buildroot host-zlib.

This problem arises when your system zlib is newer than the one provided in
buildroot because ccache is built against system zlib and not against buildroot
zlib (it wasn't ready yet).

When ccache is not working, the host build system detection fails. config.guess
returned x86_64-unknown-linux-, the part missing after the last dash should
have been gnu (or uCLinux for target system in our case). Without this latter
part config.sub failed. This is what can be seen in my previous log.

The easiest solution I may think of is backporting the zlib upgrade to the
buildroot version I'm using 2011.8. But this is odd, since we are expecting
this buildroot is able to build (and work) for a certain product lifetime.

Other options would be:
  ? Forcing host-zlib to be build on a early stage without using ccache, as
Thomas suggested
  ? Rebuilding ccache once host-zlib is build

HTH,

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2012-03-01 13:43:58 UTC ---
ccache also contains its own internal version of zlib, so it would probably be
easier to just use that one. You could try doing something like

CCACHE_CONF_OPT += ccache_cv_zlib_1_2_3=no

This will force ccache to use its internal zlib. Can you try this ?

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #6 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2012-03-01 13:44:32 UTC ---
Sorry, it should have been

HOST_CCACHE_CONF_OPT += ccache_cv_zlib_1_2_3=no

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

Ra?l S?nchez Siles <rsanchezs@infoglobal.es> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #4058|0                           |1
        is obsolete|                            |

--- Comment #7 from Ra?l S?nchez Siles <rsanchezs@infoglobal.es> 2012-03-01 16:58:10 UTC ---
Created attachment 4064
  --> https://bugs.busybox.net/attachment.cgi?id=4064
Patch against git master head.

Thanks again, Thomas. It looks it has worked. Find attached a patch against the
current master head.

Once this is applied I'd say this bug report can be closed.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

--- Comment #8 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2012-03-02 15:46:37 UTC ---
Could you send a proper git-formatted patch to the mailing list, so that your
authorship can be preserved? It would also be nice to expand a little bit the
comment above the _CONF_OPT line, because the issue is not completely trivial,
and I'm quite sure we'll wonder ourselves in the future why we did that.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 4808] ccache may build against wrong zlib

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=4808

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #9 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2012-04-22 16:40:13 UTC ---
This has been fixed by 7ea11dafff37c5403432f691cf3c46d1d5566e46.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.