cross compilation of xtables fails

cross compilation of xtables fails

[Mr Dash Four]

I have tried to compile xtables (1.30) on a x86_64 machine for i686 
architecture and that was not successful due to, in my opinion, errors 
in the m4-generated gcc statements for compiling/linking when make is 
executed. This is the log I get from make which I think causes the error 
(./configure passes with flying colours - all dependencies - i686 and 
x86_64 alike - are satisfied, I've made sure of that before I began this 
process):

gcc  -shared  -o libxt_CHAOS.so libxt_CHAOS.oo -L/lib -lxtables   ;
gcc  -shared  -o libxt_DELUDE.so libxt_DELUDE.oo -L/lib -lxtables   ;
gcc  -shared  -o libxt_DHCPMAC.so libxt_DHCPMAC.oo -L/lib -lxtables   ;
/usr/bin/ld: skipping incompatible /lib/libxtables.so when searching for 
-lxtables
/usr/bin/ld: i386 architecture of input file `libxt_CHAOS.oo' is 
incompatible with i386:x86-64 output
/usr/bin/ld: final link failed: Invalid operation
collect2: ld returned 1 exit status
make[3]: *** [libxt_CHAOS.so] Error 1
make[3]: *** Waiting for unfinished jobs....
/usr/bin/ld: skipping incompatible /lib/libxtables.so when searching for 
-lxtables
/usr/bin/ld: i386 architecture of input file `libxt_DHCPMAC.oo' is 
incompatible with i386:x86-64 output
/usr/bin/ld: final link failed: Invalid operation
collect2: ld returned 1 exit status
make[3]: *** [libxt_DHCPMAC.so] Error 1
/usr/bin/ld: skipping incompatible /lib/libxtables.so when searching for 
-lxtables
/usr/bin/ld: i386 architecture of input file `libxt_DELUDE.oo' is 
incompatible with i386:x86-64 output
/usr/bin/ld: final link failed: Invalid operation
collect2: ld returned 1 exit status

As evident from the above the linker does not like the format of 
libxt_CHAOS.so, libxt_DELUDE.so and libxt_DHCPMAC.so as they are for 
x86_64 (my build machine architecture).

After examining the above error and investigating further I've noticed 
the following: the C/CPPFLAGS I specified during the ./configure process 
which are very specific (and vital!) to enable cross compilation (-m32, 
-mtune, -march etc) have, seemingly, been ignored for 3 groups of 
files/objects:

1. libxt_CHAOS.so, libxt_DELUDE.so, libxt_DHCPMAC.so, libxt_dhcpmac.so, 
libxt_IPMARK.so, libxt_LOGMARK.so, libxt_RAWDNAT.so, libxt_RAWSNAT.so, 
libxt_STEAL.so, libxt_SYSRQ.so, libxt_TARPIT.so, libxt_condition.so, 
libxt_fuzzy.so, libxt_geoip.so, libxt_iface.so, libxt_ipp2p.so, 
libxt_ipv4options.so, libxt_length2.so, libxt_lscan.so, libxt_psd.so and 
libxt_quota2.so

2. libipset_iphash.so, libipset_ipmap.so, libipset_ipporthash.so, 
libipset_ipportiphash.so, libipset_ipportnethash.so, libipset_iptree.so, 
libipset_iptreemap.so, libipset_macipmap.so, libipset_nethash.so, 
libipset_portmap.so and libipset_setlist.so

3. libxt_pknock.so

Instead of executing gcc with the parameters I specified, for the above 
3 groups these are ignored completely and, instead, only the -L 
parameter is honoured!

That results in producing object files suitable for the machine on which 
xtables is currently compiled (x86_64 in my case) and that is not what I 
have requested initially and is the reason why the above errors occur in 
my opinion and needs to be corrected

Also, I need to mention that compilation (and linking!) of everything 
else is done with the right parameters and passes with flying colours, 
so for me, the problem is with the m4 files generating the gcc 
statements for make to execute - the full set of flags should be passed 
on instead of just '-L'.

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 14:54, Mr Dash Four wrote:

> I have tried to compile xtables (1.30) on a x86_64 machine for i686
> architecture and that was not successful due to, in my opinion, errors in the
> m4-generated gcc statements for compiling/linking when make is executed. This
> is the log I get from make which I think causes the error (./configure passes
> with flying colours - all dependencies - i686 and x86_64 alike - are satisfied,
> I've made sure of that before I began this process):

> After examining the above error and investigating further I've noticed the
> following: the C/CPPFLAGS I specified during the ./configure process which are
> very specific (and vital!) to enable cross compilation (-m32, -mtune, -march
> etc) have, seemingly, been ignored for 3 groups of files/objects:

So what is the actual configure command you used?

Re: cross compilation of xtables fails

[Mr Dash Four]

> So what is the actual configure command you used?
>   
I don't have it at hand as I am posting from a different office/machine, 
but this is the gcc statement compiling libxt_quota2:

gcc -Wp,-MMD,./.libxt_quota2.oo.d,-MT,libxt_quota2.oo 
-D_LARGEFILE_SOURCE=1 -D_LARGE_FILES -D_FILE_OFFSET_BITS=64     
-D_REENTRANT -Wall -Waggregate-return -Wmissing-declarations     
-Wmissing-prototypes -Wredundant-decls -Wshadow -Wstrict-prototypes     
-Winline -pipe -DXTABLES_LIBDIR=\"/lib/xtables\"     -I../include  
-I/home/zeek/rpm-builds/xtables-addons-1.30/extensions -DPIC -fPIC -O2 
-g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -m32 -march=i686 -mtune=pentium2 
-fasynchronous-unwind-tables -o libxt_quota2.oo -c libxt_quota2.c;

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 15:03, Mr Dash Four wrote:

>
>> So what is the actual configure command you used?
>>  
> I don't have it at hand as I am posting from a different office/machine, but
> this is the gcc statement compiling libxt_quota2:

The compile stage is unfortunately not useful here. The configure line 
is required. (If it is not in your shell history, you can find it at the 
top of config.log.)

Re: cross compilation of xtables fails

[Mr Dash Four]

> The compile stage is unfortunately not useful here. The configure line 
> is required. (If it is not in your shell history, you can find it at the 
> top of config.log.)
>   
As I already pointed out - I can't get it at present as it is on a 
different computer which I could only access later this evening (or 
later this afternoon - if I am lucky!).

What is evident from what I posted, though, is that for these 3 group of 
files the arch-specific options which make the cross compilation 
possible (-m32 -mtune and -march) are completely ignored and the gcc 
statement generated by the m4 macros are therefore tying the produced 
objects to the build architecture, which is wrong. This is without me 
providing you with any other logs.

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 15:32, Mr Dash Four wrote:

>
>> The compile stage is unfortunately not useful here. The configure line is
>> required. (If it is not in your shell history, you can find it at the top of
>> config.log.)
>>  
> As I already pointed out - I can't get it at present as it is on a different
> computer which I could only access later this evening (or later this afternoon
> - if I am lucky!).

Then do so when time comes :)

Re: cross compilation of xtables fails

[Mr Dash Four]

[-- Attachment #1: Type: text/plain, Size: 1378 bytes --]


> Then do so when time comes :)
>   
Here goes, the full monty: ;-)

LANG=C
export LANG
./autogen.sh
CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions 
-fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 
-mtune=pentium2 -fasynchronous-unwind-tables'
export CFLAGS
CXXFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions 
-fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 
-mtune=pentium2 -fasynchronous-unwind-tables'
export CXXFLAGS
FFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions 
-fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 
-mtune=pentium2 -fasynchronous-unwind-tables -I/usr/lib/gfortran/modules'
export FFLAGS
./configure --build=x86_64-redhat-linux-gnu --host=i686-redhat-linux-gnu 
--program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin 
--sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share 
--includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec 
--localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man 
--infodir=/usr/share/info --with-xtlibdir=/lib/xtables
make V=1 -j3

and that's where I end up with the error. I also use a patch (which 
executes prior to starting this whole process - as soon as I extract the 
code from the .xz source file) to disable all kernel probing during the 
build-up process (see attached that file as well).

[-- Attachment #2: xtables-addons-userspace.patch --]
[-- Type: text/plain, Size: 3812 bytes --]

diff -urP xtables-addons-1.30.org/configure.ac xtables-addons-1.30/configure.ac
--- xtables-addons-1.30.org/configure.ac	2010-05-31 09:47:58.385362271 +0800
+++ xtables-addons-1.30/configure.ac	2010-05-31 10:01:32.727112933 +0800
@@ -9,18 +9,6 @@
 AC_DISABLE_STATIC
 AC_PROG_LIBTOOL
 
-AC_ARG_WITH([kbuild],
-	AS_HELP_STRING([--with-kbuild=PATH],
-	[Path to kernel build directory [[/lib/modules/CURRENT/build]]]),
-	[kbuilddir="$withval"],
-	[kbuilddir="/lib/modules/$(uname -r)/build"])
-#
-# check for --without-kbuild
-#
-if [[ "$kbuilddir" == no ]]; then
-	kbuilddir="";
-fi
-
 AC_ARG_WITH([xtlibdir],
 	AS_HELP_STRING([--with-xtlibdir=PATH],
 	[Path where to install Xtables extensions [[LIBEXECDIR/xtables]]]),
@@ -37,42 +25,7 @@
 	-Winline -pipe -DXTABLES_LIBDIR=\\\"\${xtlibdir}\\\" \
 	-I\${XA_TOPSRCDIR}/include";
 
-#
-# check kernel version
-#
-if grep -q "CentOS release 5\." /etc/redhat-release 2>/dev/null ||
-    grep -q "Red Hat Enterprise Linux Server release 5" /etc/redhat-release 2>/dev/null; then
-	# ������������������������������������!
-	# Well, just a warning. Maybe the admin updated the kernel.
-	echo "WARNING: This distribution's shipped kernel is not supported.";
-fi;
-krel="$(make -sC ${kbuilddir} kernelrelease)";
-krel="${krel%%-*}";
-kmajor="${krel%%.*}";
-krel="${krel#*.}";
-kminor="${krel%%.*}";
-krel="${krel#*.}";
-kmicro="${krel%%.*}";
-if test "$kmicro" = "$krel"; then
-	kstable=0;
-else
-	kstable="${krel#*.}";
-	if test -z "$kstable"; then
-		kstable=0;
-	fi;
-fi;
-echo "Found kernel version $kmajor.$kminor.$kmicro.$kstable in $kbuilddir";
-if test "$kmajor" -gt 2 -o "$kminor" -gt 6 -o "$kmicro" -gt 36; then
-	echo "WARNING: You are trying a newer kernel. Results may vary. :-)";
-elif test \( "$kmajor" -lt 2 -o "$kminor" -lt 6 -o "$kmicro" -lt 17 \) -o \
-    \( "$kmajor" -eq 2 -a "$kminor" -eq 6 -a "$kmicro" -eq 18 -a \
-    "$kstable" -lt 5 \); then
-	echo "ERROR: That kernel version is not supported. Please see INSTALL for minimum configuration.";
-	exit 1;
-fi;
-
 AC_SUBST([regular_CFLAGS])
-AC_SUBST([kbuilddir])
 AC_SUBST([xtlibdir])
 AC_CONFIG_FILES([Makefile Makefile.iptrules Makefile.mans geoip/Makefile
	extensions/Makefile extensions/ACCOUNT/Makefile
	extensions/ipset/Makefile extensions/pknock/Makefile])
diff -urP xtables-addons-1.30.org/extensions/Makefile.am xtables-addons-1.30/extensions/Makefile.am
--- xtables-addons-1.30.org/extensions/Makefile.am	2010-05-31 09:47:58.363361712 +0800
+++ xtables-addons-1.30/extensions/Makefile.am	2010-05-31 10:02:08.845363738 +0800
@@ -1,26 +1,4 @@
 # -*- Makefile -*-
 # AUTOMAKE
 
-# Not having Kbuild in Makefile.extra because it will already recurse
-.PHONY: modules modules_install clean_modules
-
-_kcall = -C ${kbuilddir} M=${abs_srcdir}
-
-modules:
-	@echo -n "Xtables-addons ${PACKAGE_VERSION} - Linux "
-	@if [ -n "${kbuilddir}" ]; then ${MAKE} ${_kcall} --no-print-directory -s kernelrelease; fi;
-	${AM_V_silent}if [ -n "${kbuilddir}" ]; then ${MAKE} ${_kcall} modules; fi;
-
-modules_install:
-	${AM_V_silent}if [ -n "${kbuilddir}" ]; then ${MAKE} ${_kcall} INSTALL_MOD_PATH=${DESTDIR} ext-mod-dir='$${INSTALL_MOD_DIR}' modules_install; fi;
-
-clean_modules:
-	${AM_V_silent}if [ -n "${kbuilddir}" ]; then ${MAKE} ${_kcall} clean; fi;
-
-all-local: modules
-
-install-exec-local: modules_install
-
-clean-local: clean_modules
-
 include ../Makefile.extra
diff -urP xtables-addons-1.30.org/Makefile.am xtables-addons-1.30/Makefile.am
--- xtables-addons-1.30.org/Makefile.am	2010-05-31 09:47:58.385362271 +0800
+++ xtables-addons-1.30/Makefile.am	2010-05-31 09:52:19.632360246 +0800
@@ -12,7 +12,6 @@
 	${MAKE} -f Makefile.mans all;
 
 install-exec-hook:
-	depmod -a || :;
 
 config.status: Makefile.iptrules.in
 

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 20:31, Mr Dash Four wrote:
>
>> Then do so when time comes :)
>>  
> Here goes, the full monty: ;-)
>
> LANG=C
> export LANG
> ./autogen.sh
> CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
> -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=pentium2
> -fasynchronous-unwind-tables'
> export CFLAGS
> ./configure --build=x86_64-redhat-linux-gnu --host=i686-redhat-linux-gnu
> --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin
> --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share
> --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/libexec
> --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man
> --infodir=/usr/share/info --with-xtlibdir=/lib/xtables

As I thought. You forgot LDFLAGS="-m32".

Re: cross compilation of xtables fails

[Mr Dash Four]

> As I thought. You forgot LDFLAGS="-m32".
>   
Nice!!! It works now!

I presume I need to attach add "-march=i686 -mtune=pentium2" to LDFLAGS 
as well, seeing that gcc executes as "gcc -shared -m32 -o ..." if 
LDFLAGS="-m32", right?

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 20:52, Mr Dash Four wrote:

>
>> As I thought. You forgot LDFLAGS="-m32".
>>  
> Nice!!! It works now!
>
> I presume I need to attach add "-march=i686 -mtune=pentium2" to LDFLAGS as
> well, seeing that gcc executes as "gcc -shared -m32 -o ..." if LDFLAGS="-m32",
> right?

No, -march and -mtune are compile flags.

Re: cross compilation of xtables fails

[Mr Dash Four]

>> I presume I need to attach add "-march=i686 -mtune=pentium2" to LDFLAGS as
>> well, seeing that gcc executes as "gcc -shared -m32 -o ..." if LDFLAGS="-m32",
>> right?
>>     
>
> No, -march and -mtune are compile flags.
>   
OK, I am perfecting it now...Thanks a bunch for helping me out!

On a side note I see that in 1.31 --with-xlibdir is removed - how would 
xtables know what I am compiling/building for (x86_64 or i686)?

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 21:04, Mr Dash Four wrote:

>
>>> I presume I need to attach add "-march=i686 -mtune=pentium2" to LDFLAGS as
>>> well, seeing that gcc executes as "gcc -shared -m32 -o ..." if
>>> LDFLAGS="-m32",
>>> right?
>>>    
>>
>> No, -march and -mtune are compile flags.
>>  
> OK, I am perfecting it now...Thanks a bunch for helping me out!
>
> On a side note I see that in 1.31 --with-xlibdir is removed - how would xtables
> know what I am compiling/building for (x86_64 or i686)?

It never used xtlibdir to determine what you want to build for.
You tell it via --host and cflags/ldflags/etc.

Re: cross compilation of xtables fails

[Mr Dash Four]

> It never used xtlibdir to determine what you want to build for.
> You tell it via --host and cflags/ldflags/etc.
>   
Right, I will probably get a new xtables version installed if/when 
Jozsef gets the ipset kernel bug fixed and see if I can build it for 
both platforms. Thanks again.

Re: cross compilation of xtables fails

[Jan Engelhardt]

On Friday 2010-12-03 22:02, Mr Dash Four wrote:

>
>> It never used xtlibdir to determine what you want to build for.
>> You tell it via --host and cflags/ldflags/etc.
>>  
> Right, I will probably get a new xtables version installed if/when Jozsef gets
> the ipset kernel bug fixed and see if I can build it for both platforms. Thanks
> again.

Pedant note: that's xtables-addons. xtables is in 
linux/net/netfilter/...