xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1446 bytes --]

Hi,

building xfsprogs-5.2.0 is failing with the following error:

>     [CC]     xfs_copy.o
> x86_64-pc-linux-gnu-gcc -O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches -D_FILE_OFFSET_BITS=64 -O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches  -O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches -DNDEBUG -DVERSION=\"5.2.0\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -I../libxfs -DENABLE_GETTEXT -D_GNU_SOURCE -funsigned-char -fno-strict-aliasing -Wall -DHAVE_MNTENT -DHAVE_FSETXATTR -DENABLE_BLKID -DHAVE_GETFSMAP  -c xfs_copy.c
>     [LD]     xfs_copy
> /bin/bash ../libtool --quiet --tag=CC --mode=link x86_64-pc-linux-gnu-gcc -o xfs_copy -Wl,-O1 -Wl,--as-needed  -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   xfs_copy.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libfrog/libfrog.la -luuid -lpthread -lrt  
> /usr/lib/gcc/x86_64-pc-linux-gnu/9.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'
> collect2: error: ld returned 1 exit status
> gmake[2]: *** [../include/buildrules:65: xfs_copy] Error 1
> gmake[1]: *** [include/buildrules:36: copy] Error 2
> make: *** [Makefile:92: default] Error 2


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Sun, Aug 11, 2019 at 03:06:13PM +0200, Thomas Deutschmann wrote:
> Hi,
> 
> building xfsprogs-5.2.0 is failing with the following error:
> 
> >     [CC]     xfs_copy.o
> > x86_64-pc-linux-gnu-gcc -O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches -D_FILE_OFFSET_BITS=64 -O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches  -O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches -DNDEBUG -DVERSION=\"5.2.0\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -I../libxfs -DENABLE_GETTEXT -D_GNU_SOURCE -funsigned-char -fno-strict-aliasing -Wall -DHAVE_MNTENT -DHAVE_FSETXATTR -DENABLE_BLKID -DHAVE_GETFSMAP  -c xfs_copy.c
> >     [LD]     xfs_copy
> > /bin/bash ../libtool --quiet --tag=CC --mode=link x86_64-pc-linux-gnu-gcc -o xfs_copy -Wl,-O1 -Wl,--as-needed  -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   xfs_copy.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libfrog/libfrog.la -luuid -lpthread -lrt  
> > /usr/lib/gcc/x86_64-pc-linux-gnu/9.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'
> > collect2: error: ld returned 1 exit status
> > gmake[2]: *** [../include/buildrules:65: xfs_copy] Error 1
> > gmake[1]: *** [include/buildrules:36: copy] Error 2
> > make: *** [Makefile:92: default] Error 2

That's quite a set of custom compiler options you have there. It's
complaining about a reference to a function that is dead code in
the build that is called from other dead code in the build. i.e the
linker or compiler has elided one part of the dead code, but not all
of it. The build works if nothing is elided or all the dead code is
elided, so that seems like a toolchain problem more that anything.

FWIW, the default build w. gcc-8.3.0 on debian unstable is fine:

gcc -MM -g -O2 -D_FILE_OFFSET_BITS=64   -g -O2 -DDEBUG -DVERSION=\"5.2.0\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -I../libxfs -DENABLE_GETTEXT -D_GNU_SOURCE -funsigned-char -fno-strict-aliasing -Wall -DHAVE_MNTENT -DHAVE_FSETXATTR -DENABLE_BLKID -DHAVE_GETFSMAP  xfs_copy.c > .dep
rm -f .dep
    [CC]     xfs_copy.o
gcc -g -O2 -D_FILE_OFFSET_BITS=64   -g -O2 -DDEBUG -DVERSION=\"5.2.0\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -I../libxfs -DENABLE_GETTEXT -D_GNU_SOURCE -funsigned-char -fno-strict-aliasing -Wall -DHAVE_MNTENT -DHAVE_FSETXATTR -DENABLE_BLKID -DHAVE_GETFSMAP  -c xfs_copy.c
    [LD]     xfs_copy
/bin/bash ../libtool --quiet --tag=CC --mode=link gcc -o xfs_copy   -static-libtool-libs  xfs_copy.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libfrog/libfrog.la -luuid -lpthread -lrt

And it tells me that your whacky set of options to the
linker pass (which repeats 3x "-Wl,-O1 -Wl,--as-needed  " instead of
"-static-libtool-libs") is likely the problem.

Yup, that's the problem. If I run your linker command on my
successful build to relink the xfs-copy binary:

/bin/bash ../libtool --quiet --tag=CC --mode=link gcc -o xfs_copy -Wl,-O1 -Wl,--as-needed  -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   xfs_copy.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libfrog/libfrog.la -luuid -lpthread -lrt
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_new_probe_from_filename'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_alignment_offset'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_get_topology'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_do_fullprobe'
/usr/bin/ld: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_logical_sector_size'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_optimal_io_size'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_lookup_value'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_free_probe'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_physical_sector_size'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_enable_partitions'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_minimum_io_size'

There's all sorts of problems with undefined references to code not
used by xfs_copy. Ok, so the problem is at your end with custom
build options, not a problem with the code or the default build.

How did you configure this build? Can you run a clean build without
configuring in any of the whacky compiler super-optimisations that
you have and see if that builds cleanly?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1091 bytes --]

On 2019-08-12 00:53, Dave Chinner wrote:
> How did you configure this build? Can you run a clean build without
> configuring in any of the whacky compiler super-optimisations that
> you have and see if that builds cleanly?

Just add

  --disable-static

and you should be able to reproduce. This option will cause that 3x
"-Wl,-O1 -Wl,--as-needed  " will be passed to the linker.

Complete configure we are using and which worked fine with xfsprogs-5.1.0:

--prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share
--sysconfdir=/etc --localstatedir=/var/lib
--docdir=/usr/share/doc/xfsprogs-5.2.0
--htmldir=/usr/share/doc/xfsprogs-5.2.0/html --with-sysroot=/
--libdir=/usr/lib64 --disable-lto --enable-blkid
--with-crond-dir=/etc/cron.d --with-systemd-unit-dir=/lib/systemd/system
--disable-libicu --enable-gettext --enable-readline --disable-editline
--disable-static


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Mon, Aug 12, 2019 at 01:30:46AM +0200, Thomas Deutschmann wrote:
> On 2019-08-12 00:53, Dave Chinner wrote:
> > How did you configure this build? Can you run a clean build without
> > configuring in any of the whacky compiler super-optimisations that
> > you have and see if that builds cleanly?
> 
> Just add
> 
>   --disable-static

yes, ok, that produces a link time error, but...
> 
> and you should be able to reproduce. This option will cause that 3x
> "-Wl,-O1 -Wl,--as-needed  " will be passed to the linker.

it does not produce that whacky linker command line you are getting:

    [LD]     xfs_copy
/bin/bash ../libtool --quiet --tag=CC --mode=link gcc -o xfs_copy   -static-libtool-libs  xfs_copy.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libfrog/libfrog.la -luuid -lpthread -lrt  
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_new_probe_from_filename'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_alignment_offset'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_get_topology'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_do_fullprobe'
/usr/bin/ld: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'
.....

In fact, the linker command is identical to the non-static build,
which implies the ../libxfs/libxfs.la library definition points
to something different:

# The name that we can dlopen(3).
dlname='libxfs.so.0'

# Names of this library.
library_names='libxfs.so.0.0.0 libxfs.so.0 libxfs.so'

# The name of the static archive.
old_library=

.....

And when static libraries are built:

# The name that we can dlopen(3).
dlname='libxfs.so.0'

# Names of this library.
library_names='libxfs.so.0.0.0 libxfs.so.0 libxfs.so'

# The name of the static archive.
old_library='libxfs.a'

....


Yeah, so there's teh difference - there should be no change to
the linker command line from --disable-static.

Really, I don't know why we support --disable-static at all.
Internal XFS libraries have to be statically linked because they are
not shipped as installed shared libraries. Hence not building the
static version of the library is wrong, and the shared version is
not used by anything. The default build dynamically links external
libraries, so there is nothing to be gained from disabling internal
static libraries.

Indeed, that's why we have the -static-libtool-libs option on the
linker command link, as commit ece49daeff1a ("xfsprogs: do not do
any dynamic linking of libtool libraries") explains:

    if --disable-static and --enable-shared are given on the command
    line, the link with xfsprogs's internal libraries fail because
    they have been dynamically compiled.
    
    Hence the following error:
    ld: attempted static link of dynamic object `../libxcmd/.libs/libxcmd.so'
    
    xfsprogs rely on the original behaviour of -static which was modified in
    Buildroot by [1].  But since commit [2] the build of xfsprogs tools is broken
    because they try to link statically with the static libuuid library
    (util-linux), which is not build for shared only build.
    
    The use of -static-libtool-libs allows to fallback to the dynamic linking for
    libuuid only:
    
    LD_TRACE_LOADED_OBJECTS=1 xfs_copy
            linux-gate.so.1 =>  (0xf7793000)
            libuuid.so.1 => /lib/libuuid.so.1 (0x465e1000)
            libpthread.so.0 => /lib/libpthread.so.0 (0x46db1000)
            librt.so.1 => /lib/librt.so.1 (0x46f21000)
            libc.so.6 => /lib/libc.so.6 (0x46bf1000)
            /lib/ld-linux.so.2 (0x46bce000)

So I'd be removing --disable-static from your build because it
really isn't doing anything useful...

That still doesn't explain where all the whacky gcc options are
coming from - that's got to be something specific to your build or
distro environment.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1342 bytes --]

On 2019-08-12 02:23, Dave Chinner wrote:
> That still doesn't explain where all the whacky gcc options are
> coming from - that's got to be something specific to your build or
> distro environment.

Mh, at the moment it looks like xfsprogs' build system is adding
$LDFLAGS multiple times when LDFLAGS is set in environment.

In a clear environment, do:

> tar -xaf xfsprogs-5.2.0.tar.xz
> cd xfsprogs-5.2.0
> export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
> export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
> ./configure
> make V=1

And you will end up

> /bin/bash ../libtool --quiet --tag=CC --mode=link x86_64-pc-linux-gnu-gcc -o xfs_repair -Wl,-O1 -Wl,--as-needed  -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed -static-libtool-libs  agheader.o attr_repair.o avl.o bmap.o btree.o da_util.o dino_chunks.o dinode.o dir2.o globals.o incore.o incore_bmc.o init.o incore_ext.o incore_ino.o phase1.o phase2.o phase3.o phase4.o phase5.o phase6.o phase7.o progress.o prefetch.o rmap.o rt.o sb.o scan.o slab.o threads.o versions.o xfs_repair.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libxcmd/libxcmd.la ../libfrog/libfrog.la -luuid -lrt -lpthread -lblkid


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Mon, Aug 12, 2019 at 03:21:28AM +0200, Thomas Deutschmann wrote:
> On 2019-08-12 02:23, Dave Chinner wrote:
> > That still doesn't explain where all the whacky gcc options are
> > coming from - that's got to be something specific to your build or
> > distro environment.
> 
> Mh, at the moment it looks like xfsprogs' build system is adding
> $LDFLAGS multiple times when LDFLAGS is set in environment.
> 
> In a clear environment, do:
> 
> > tar -xaf xfsprogs-5.2.0.tar.xz
> > cd xfsprogs-5.2.0
> > export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
> > export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Don't do this.

"--as-needed" is the default linker behaviour since gcc 4.x. You do
not need this. As for passing "-O1" to the linker, that's not going
to do anything measurable for you. Use --enable-lto to turn on link
time optimisations if they are supported by the compiler.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Mon, Aug 12, 2019 at 01:11:23PM +1000, Dave Chinner wrote:
> On Mon, Aug 12, 2019 at 03:21:28AM +0200, Thomas Deutschmann wrote:
> > On 2019-08-12 02:23, Dave Chinner wrote:
> > > That still doesn't explain where all the whacky gcc options are
> > > coming from - that's got to be something specific to your build or
> > > distro environment.
> > 
> > Mh, at the moment it looks like xfsprogs' build system is adding
> > $LDFLAGS multiple times when LDFLAGS is set in environment.
> > 
> > In a clear environment, do:
> > 
> > > tar -xaf xfsprogs-5.2.0.tar.xz
> > > cd xfsprogs-5.2.0
> > > export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
> > > export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Don't do this.
> 
> "--as-needed" is the default linker behaviour since gcc 4.x. You do
> not need this. As for passing "-O1" to the linker, that's not going
> to do anything measurable for you. Use --enable-lto to turn on link
> time optimisations if they are supported by the compiler.

Ok, I could reproduce your link time failure for a while with
--enable-lto, but I ran 'make distclean' and the problem went away
completely. And I can build with your options successfully, too:

$ make realclean
$ make configure
<builds new configure script>
$ LDFLAGS="-Wl,-O1 -Wl,--as-needed" ./configure
.....
$ make -j32
<builds successfully>
$ touch copy/xfs_copy.c
$ make Q=
....
Building copy
/usr/bin/make --no-print-directory Q= -q -C copy || /usr/bin/make --no-print-directory Q= -C copy
gcc -MM -g -O2 -D_FILE_OFFSET_BITS=64   -g -O2 -DDEBUG -DVERSION=\"5.2.0\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -I../libxfs -DENABLE_GETTEXT -D_GNU_SOURCE -funsigned-char -fno-strict-aliasing -Wall -DHAVE_MNTENT -DHAVE_FSETXATTR -DENABLE_BLKID -DHAVE_GETFSMAP  xfs_copy.c > .dep
rm -f .dep
    [CC]     xfs_copy.o
gcc -g -O2 -D_FILE_OFFSET_BITS=64   -g -O2 -DDEBUG -DVERSION=\"5.2.0\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -I../libxfs -DENABLE_GETTEXT -D_GNU_SOURCE -funsigned-char -fno-strict-aliasing -Wall -DHAVE_MNTENT -DHAVE_FSETXATTR -DENABLE_BLKID -DHAVE_GETFSMAP  -c xfs_copy.c
    [LD]     xfs_copy
/bin/bash ../libtool --quiet --tag=CC --mode=link gcc -o xfs_copy  -Wl,-O1 -Wl,--as-needed -static-libtool-libs  xfs_copy.o   ../libxfs/libxfs.la ../libxlog/libxlog.la ../libfrog/libfrog.la -luuid -lpthread -lrt  
Building db
....

So, before you try to build, start with

$ make realclean; make configure

to make sure you are running a configure script that is built from
the release tarball source code, then run your custom build and
see what happens.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1425 bytes --]

Hi,

On 2019-08-12 06:30, Dave Chinner wrote:
>>> In a clear environment, do:
>>>
>>>> tar -xaf xfsprogs-5.2.0.tar.xz
>>>> cd xfsprogs-5.2.0
>>>> export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
>>>> export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
>>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> Don't do this.
>>
>> "--as-needed" is the default linker behaviour since gcc 4.x. You do
>> not need this. As for passing "-O1" to the linker, that's not going
>> to do anything measurable for you. Use --enable-lto to turn on link
>> time optimisations if they are supported by the compiler.
> 
> Ok, I could reproduce your link time failure for a while with
> --enable-lto, but I ran 'make distclean' and the problem went away
> completely. And I can build with your options successfully, too:
> 
> $ make realclean
> $ make configure
> <builds new configure script>
> $ LDFLAGS="-Wl,-O1 -Wl,--as-needed" ./configure

That's not the correct way to reproduce. It's really important to
_export_ the variable to trigger the problem and _this_ is a problem in
xfsprogs' build system.

But keep in mind that 3x "-Wl,-O1 -Wl,--as-needed" don't cause a failure
without "--disable-static" for me... that's just the answer for your
question where this is coming from.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Eric Sandeen]

[-- Attachment #1.1: Type: text/plain, Size: 1656 bytes --]



On 8/12/19 5:57 AM, Thomas Deutschmann wrote:
> Hi,
> 
> On 2019-08-12 06:30, Dave Chinner wrote:
>>>> In a clear environment, do:
>>>>
>>>>> tar -xaf xfsprogs-5.2.0.tar.xz
>>>>> cd xfsprogs-5.2.0
>>>>> export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
>>>>> export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
>>>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>> Don't do this.
>>>
>>> "--as-needed" is the default linker behaviour since gcc 4.x. You do
>>> not need this. As for passing "-O1" to the linker, that's not going
>>> to do anything measurable for you. Use --enable-lto to turn on link
>>> time optimisations if they are supported by the compiler.
>>
>> Ok, I could reproduce your link time failure for a while with
>> --enable-lto, but I ran 'make distclean' and the problem went away
>> completely. And I can build with your options successfully, too:
>>
>> $ make realclean
>> $ make configure
>> <builds new configure script>
>> $ LDFLAGS="-Wl,-O1 -Wl,--as-needed" ./configure
> 
> That's not the correct way to reproduce. It's really important to
> _export_ the variable to trigger the problem and _this_ is a problem in
> xfsprogs' build system.
> 
> But keep in mind that 3x "-Wl,-O1 -Wl,--as-needed" don't cause a failure
> without "--disable-static" for me... that's just the answer for your
> question where this is coming from.

My takeaway here is that I should probably stub out some things to make
this issue go away altogether, but that it can also be remedied by adjusting
compiler/linker flags.  Correct?  I'm not sure if this warrants a 5.2.1 release.

Thanks,
-Eric


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 873 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Mon, Aug 12, 2019 at 11:34:56AM -0500, Eric Sandeen wrote:
> 
> 
> On 8/12/19 5:57 AM, Thomas Deutschmann wrote:
> > Hi,
> > 
> > On 2019-08-12 06:30, Dave Chinner wrote:
> >>>> In a clear environment, do:
> >>>>
> >>>>> tar -xaf xfsprogs-5.2.0.tar.xz
> >>>>> cd xfsprogs-5.2.0
> >>>>> export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
> >>>>> export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
> >>>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>> Don't do this.
> >>>
> >>> "--as-needed" is the default linker behaviour since gcc 4.x. You do
> >>> not need this. As for passing "-O1" to the linker, that's not going
> >>> to do anything measurable for you. Use --enable-lto to turn on link
> >>> time optimisations if they are supported by the compiler.
> >>
> >> Ok, I could reproduce your link time failure for a while with
> >> --enable-lto, but I ran 'make distclean' and the problem went away
> >> completely. And I can build with your options successfully, too:
> >>
> >> $ make realclean
> >> $ make configure
> >> <builds new configure script>
> >> $ LDFLAGS="-Wl,-O1 -Wl,--as-needed" ./configure
> > 
> > That's not the correct way to reproduce. It's really important to
> > _export_ the variable to trigger the problem and _this_ is a problem in
> > xfsprogs' build system.
> > 
> > But keep in mind that 3x "-Wl,-O1 -Wl,--as-needed" don't cause a failure
> > without "--disable-static" for me... that's just the answer for your
> > question where this is coming from.
> 
> My takeaway here is that I should probably stub out some things to make
> this issue go away altogether,

The linker should be taking care of eliding dead code from
statically linked libraries.

The issue here is that a shared library can't have undefined symbols
elided at link time because the linker has no idea if they will be
used or not.  Hence it fails to link against binaries that don't
specify all possible external library dependencies, even when the
library dependency is brought in by another library.

> but that it can also be remedied by adjusting
> compiler/linker flags.  Correct?  I'm not sure if this warrants a 5.2.1 release.

If you don't use the non-default --disable-static configure option
then there is no problem. That option appears to have been broken
for a while now, so lets just get rid of it.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Mon, Aug 12, 2019 at 12:57:32PM +0200, Thomas Deutschmann wrote:
> Hi,
> 
> On 2019-08-12 06:30, Dave Chinner wrote:
> >>> In a clear environment, do:
> >>>
> >>>> tar -xaf xfsprogs-5.2.0.tar.xz
> >>>> cd xfsprogs-5.2.0
> >>>> export CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt"
> >>>> export LDFLAGS="-Wl,-O1 -Wl,--as-needed"
> >>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >> Don't do this.
> >>
> >> "--as-needed" is the default linker behaviour since gcc 4.x. You do
> >> not need this. As for passing "-O1" to the linker, that's not going
> >> to do anything measurable for you. Use --enable-lto to turn on link
> >> time optimisations if they are supported by the compiler.
> > 
> > Ok, I could reproduce your link time failure for a while with
> > --enable-lto, but I ran 'make distclean' and the problem went away
> > completely. And I can build with your options successfully, too:
> > 
> > $ make realclean
> > $ make configure
> > <builds new configure script>
> > $ LDFLAGS="-Wl,-O1 -Wl,--as-needed" ./configure
> 
> That's not the correct way to reproduce. It's really important to
> _export_ the variable to trigger the problem and _this_ is a problem in
> xfsprogs' build system.

Which means you are overriding the LDFLAGS set by configure when
you _run make_, not just telling configure to use those LDFLAGS.

That's why _make_ is getting screwed up - it is doing exactly what
you are telling it to do, and that is to overrides every occurrence
of LDFLAGS with your exported options rather than using the correct
set configure calculated and specified.

Exporting your CFLAGS and LDFLAGS is the wrong thing to doing
- they should only ever be passed to the configure invocation and
not remain to pollute the build environment after you've run
configure.

> But keep in mind that 3x "-Wl,-O1 -Wl,--as-needed" don't cause a failure
> without "--disable-static" for me... that's just the answer for your
> question where this is coming from.

As I've already explained, --disable-static should never have worked
in the first place, because the internal XFS libraries are
statically linked and need to be build statically, which
--disable-static turns off. 

Indeed:

$ git reset --hard v5.1.0; make realclean; make configure ; ./configure --disable-static ; make -j 32
......
    [LD]     xfs_estimate
    [LD]     xfs_mdrestore
    [LD]     xfs_rtcp
    [LD]     libhandle.la
    [LD]     libxcmd.la
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_new_probe_from_filename'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_alignment_offset'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_get_topology'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_do_fullprobe'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_logical_sector_size'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_optimal_io_size'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_lookup_value'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_free_probe'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_physical_sector_size'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_probe_enable_partitions'
/usr/bin/ld: ../libfrog/.libs/libfrog.so: undefined reference to `blkid_topology_get_minimum_io_size'
$

Yup, --disable-static is broken in v5.1.0, too. I'm guessing it
hasn't worked for a long time....

So, please remove --disable-static from your build, and everything
will be fine. I'll write up a patch to remove --disable-static from
the configure script so this broken option can't be specified any
more.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1340 bytes --]

On 2019-08-12 23:44, Dave Chinner wrote:
>> That's not the correct way to reproduce. It's really important to
>> _export_ the variable to trigger the problem and _this_ is a problem in
>> xfsprogs' build system.
> Which means you are overriding the LDFLAGS set by configure when
> you _run make_, not just telling configure to use those LDFLAGS.
> 
> That's why _make_ is getting screwed up - it is doing exactly what
> you are telling it to do, and that is to overrides every occurrence
> of LDFLAGS with your exported options rather than using the correct
> set configure calculated and specified.
> 
> Exporting your CFLAGS and LDFLAGS is the wrong thing to doing
> - they should only ever be passed to the configure invocation and
> not remain to pollute the build environment after you've run
> configure.

I disagree with the conclusion. LDFLAGS in build environment shouldn't
cause any problems, especially when you are using a build system:

Normally, configure will get the value and the Makefiles will use the
value _from_ configure... but using configure _and_ reading _and adding_
values from environment _in addition_ seems to be wrong.

So you either use a build system or you don't.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Tue, Aug 13, 2019 at 12:18:04AM +0200, Thomas Deutschmann wrote:
> On 2019-08-12 23:44, Dave Chinner wrote:
> >> That's not the correct way to reproduce. It's really important to
> >> _export_ the variable to trigger the problem and _this_ is a problem in
> >> xfsprogs' build system.
> > Which means you are overriding the LDFLAGS set by configure when
> > you _run make_, not just telling configure to use those LDFLAGS.
> > 
> > That's why _make_ is getting screwed up - it is doing exactly what
> > you are telling it to do, and that is to overrides every occurrence
> > of LDFLAGS with your exported options rather than using the correct
> > set configure calculated and specified.
> > 
> > Exporting your CFLAGS and LDFLAGS is the wrong thing to doing
> > - they should only ever be passed to the configure invocation and
> > not remain to pollute the build environment after you've run
> > configure.
> 
> I disagree with the conclusion. LDFLAGS in build environment shouldn't
> cause any problems, especially when you are using a build system:
>
> Normally, configure will get the value and the Makefiles will use the
> value _from_ configure... but using configure _and_ reading _and adding_
> values from environment _in addition_ seems to be wrong.

<sigh>

xfsprogs-2.7.18 (16 May 2006)
        - Fixed a case where xfs_repair was reporting a valid used
          block as a duplicate during phase 4.
        - Fixed a case where xfs_repair could incorrectly flag extent
          b+tree nodes as corrupt.
        - Portability changes, get xfs_repair compiling on IRIX.
        - Parent pointer updates in xfs_io checker command.
        - Allow LDFLAGS to be overridden, for Gentoo punters.
	  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Back in 2006 we added the ability for LDFLAGS to be overriden
specifically because Gentoo users wanted it.

> So you either use a build system or you don't.

Yup, but Gentoo wanted it both ways, and so we gave them that
capability.  And now you're complaining that Gentoo users can shoot
themselves in the foot with it.... :/

Just pass your special CFLAGS/LDFLAGS to configure like the rest of
the world does and everything will work correctly.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 3899 bytes --]

On 2019-08-13 03:04, Dave Chinner wrote:
>> Normally, configure will get the value and the Makefiles will use the
>> value _from_ configure... but using configure _and_ reading _and adding_
>> values from environment _in addition_ seems to be wrong.
> 
> <sigh>
> 
> xfsprogs-2.7.18 (16 May 2006)
>         - Fixed a case where xfs_repair was reporting a valid used
>           block as a duplicate during phase 4.
>         - Fixed a case where xfs_repair could incorrectly flag extent
>           b+tree nodes as corrupt.
>         - Portability changes, get xfs_repair compiling on IRIX.
>         - Parent pointer updates in xfs_io checker command.
>         - Allow LDFLAGS to be overridden, for Gentoo punters.
> 	  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Back in 2006 we added the ability for LDFLAGS to be overriden
> specifically because Gentoo users wanted it.

Well, you got us :-)

Sorry, I wasn't around 2006 and don't know all details. Looks
like commit https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=4d32d744f07ce74abac7029c3ee7c6f5e4238d25 caused that
changelog entry.

The request itself, that user should be able to overwrite
CFLAGS/LDFLAGS, is valid. However, I haven't check implementation
yet and I guess that it was implemented the wrong way:

It was and it still is a common problem that people do autotools
wrong when it comes to CFLAGS/LDFLAGS handling. I.e. project
should define two values:

- Initial CFLAGS/LDFLAGS in case user didn't provide these flags.
However, this value will be _overwritten_ when user provided these
flags

- Additional, required general flags. This value will get appended
to previous value.

Autotools will now define one single variable containing these
two values which should be used. If you are now going to link
against an additional dependency you will set CFLAGS/LDFLAGS
just for this component which usually starts with the variable
from above (=the general CFLAGS/LDFLAGS variable) but will usually
contain component specific CFLAGS/LDFLAGS in addition (for
example when linking against a library which will provide
pkg-config file you will import lib specific CFLAGS/LDFLAGS).

Sorry if this will sound like I am teaching you something. I
guess you all know that (and probably far better than me).

Back to xfsprogs: When you pass your default CFLAGS/LDFLAGS
to configure and different to make, i.e.

CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches" \
LDFLAGS="-Wl,-O1 -Wl,--as-needed" \
./configure

and 

CFLAGS="-O2 -pipe -march=native" \
LDFAGS="-Wl,-fno-lto" \
make V=1

you will see

> /bin/bash ../libtool --quiet --tag=CC --mode=link gcc -o mkfs.xfs -Wl,-fno-lto  -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed -static-libtool-libs  proto.o xfs_mkfs.o   ../libxfs/libxfs.la ../libxcmd/libxcmd.la ../libfrog/libfrog.la -lrt -lpthread -lblkid -luuid

So we still have 3x "-Wl,-O1 -Wl,--as-needed" which is LDFLAGS
value passed to configure. LDFLAGS from environment is only
picked up once (which is still wrong).


> Yup, but Gentoo wanted it both ways, and so we gave them that
> capability.  And now you're complaining that Gentoo users can shoot
> themselves in the foot with it.... :/

Where is that coming from? From reading a few bugs from that time
in Gentoo and reading our ebuild sources I can say that we were
already using autotools for xfsprogs. So it looks like xfsprogs
was facing the problem I mentioned in the beginning.

To make it clear: Makefile should only use configure values (which
isn't the case at the moment) and configure should  of course
honor environment variables (which it already does).


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: xfsprogs-5.2.0 FTBFS: ../libxfs/.libs/libxfs.so: undefined reference to `xfs_ag_geom_health'

[Dave Chinner]

On Tue, Aug 13, 2019 at 03:52:31PM +0200, Thomas Deutschmann wrote:
> On 2019-08-13 03:04, Dave Chinner wrote:
> >> Normally, configure will get the value and the Makefiles will use the
> >> value _from_ configure... but using configure _and_ reading _and adding_
> >> values from environment _in addition_ seems to be wrong.
> > 
> > <sigh>
> > 
> > xfsprogs-2.7.18 (16 May 2006)
> >         - Fixed a case where xfs_repair was reporting a valid used
> >           block as a duplicate during phase 4.
> >         - Fixed a case where xfs_repair could incorrectly flag extent
> >           b+tree nodes as corrupt.
> >         - Portability changes, get xfs_repair compiling on IRIX.
> >         - Parent pointer updates in xfs_io checker command.
> >         - Allow LDFLAGS to be overridden, for Gentoo punters.
> > 	  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > 
> > Back in 2006 we added the ability for LDFLAGS to be overriden
> > specifically because Gentoo users wanted it.
> 
> Well, you got us :-)
> 
> Sorry, I wasn't around 2006 and don't know all details. Looks
> like commit https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=4d32d744f07ce74abac7029c3ee7c6f5e4238d25 caused that
> changelog entry.

I was hoping you'd look at the change and then notice and
understand that the override provided was for LDFLAGS passed by
configure, not the make environment.

And that, having looked at this hunk of the patch:

-LDFLAGS = $(LLDFLAGS)
+LDFLAGS += $(LOADERFLAGS) $(LLDFLAGS)

then, perhaps, you'd see the obvious bug introduced by that change
that resulted in LDFLAGS no longer being initialised correctly and
so could be accidentally initialised from the external
environment....

> Back to xfsprogs: When you pass your default CFLAGS/LDFLAGS
> to configure and different to make, i.e.
>
> CFLAGS="-O2 -pipe -march=ivybridge -mtune=ivybridge -mno-xsaveopt -frecord-gcc-switches" \
> LDFLAGS="-Wl,-O1 -Wl,--as-needed" \
> ./configure
> 
> and 
> 
> CFLAGS="-O2 -pipe -march=native" \
> LDFAGS="-Wl,-fno-lto" \
> make V=1
>
> you will see
> 
> > /bin/bash ../libtool --quiet --tag=CC --mode=link gcc -o mkfs.xfs -Wl,-fno-lto  -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed   -Wl,-O1 -Wl,--as-needed -static-libtool-libs  proto.o xfs_mkfs.o   ../libxfs/libxfs.la ../libxcmd/libxcmd.la ../libfrog/libfrog.la -lrt -lpthread -lblkid -luuid

I also note that you haven't observed that CFLAGS did not get
overridden in this test, despite it being specified in the
environment like LDFLAGS.

So we correctly initialise CFLAGS so it can't be overriden from
make, but we don't initialise LDFLAGS. Yeah, I noticed this a couple
of days ago, and I've been waiting for you to send the "you forgot
to initialise a varaible 13 years ago" patch after I pointed you
right at it...

Really, I should have just sent this patch when I wrote it couple of
days ago, rather than thinking that with enough information you'd
find it yourself and get credit and cudos for fixing it for us.

Gesta non verba.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

xfsprogs: LDFLAGS comes from configure, not the make environment

From: Dave Chinner <dchinner@redhat.com>

When doing:

$ LDFLAGS=foo make

bad things happen because we don't initialise LDFLAGS to an empty
string in include/builddefs.in and hence make takes wahtever is in
the environment and runs with it. This causes problems even when the
correct linker options are specified correctly through configure.

We don't support overriding build flags (like CFLAGS) though the
make environment, so it was an oversight 13 years ago to allow
LDFLAGS to be overridden when adding support to custom LDFLAGS being
passed from the the configure script. This ensures we only ever use
linker flags from configure, not the make environment.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 include/builddefs.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/builddefs.in b/include/builddefs.in
index c5b38b073e1f..e0f0ad94976e 100644
--- a/include/builddefs.in
+++ b/include/builddefs.in
@@ -16,6 +16,10 @@ LTLDFLAGS = @LDFLAGS@
 CFLAGS = @CFLAGS@ -D_FILE_OFFSET_BITS=64
 BUILD_CFLAGS = @BUILD_CFLAGS@ -D_FILE_OFFSET_BITS=64
 
+# make sure we don't pick up whacky LDFLAGS from the make environment and
+# only use what we calculate from the configured options above.
+LDFLAGS =
+
 LIBRT = @librt@
 LIBUUID = @libuuid@
 LIBPTHREAD = @libpthread@