[Buildroot] Cross-compiling Cyrus in Buildroot

[Buildroot] Cross-compiling Cyrus in Buildroot

[Luca Ceresoli]

Hi Cyrus and Buildroot developers,

I'm working to add cyrus-sasl and cyrus-imapd (http://www.cyrusimap.org/)
to buildroot (http://www.buildroot.net/) for use on embedded systems.

On my road I hit various problems with the cross-build process:
configuration, compilation and installation.
I could work around some, but not really fix them as they should.

I would like to share my findings with both communities, with the goal of
clearly identifying the issues and fixing them in the proper way, thus
contributing an improvement to both projects: two new packages for Buildroot
and improvements and/or fixes for Cyrus (both -sasl and -imapd).

I'll start a new thread for each issue, and keep in copy both mailing lists.
Please reply-to-all in order to keep the discussion alive in both places.

Here's a list of the issues I found:
  - cyrus-sasl: apparent CFLAGS mismatch on saslauthd;
  - cyrus-sasl: makemd5 is wrongly cross-compiled;
  - cyrus-sasl: installs files with "arm-linux-" prefix
  - cyrus-imapd: imtest does not cross-compile correctly;
  - cyrus-imapd: static binaries are huge, cannot build them shared;
  - cyrus-imapd: choose binaries to (build and) install;
  - others might follow as I go on with my work.

Before starting, a general question to Cyrus developers.
So far I've been working with the most recent released packages: cyrus-sasl
2.1.24rc1 and cyrus-imapd 2.4.9, but I see the cyrus-imapd git 
repository has
a master branch that diverges dramatically from the 2.4.x branch:

   $ git diff --shortstat  cyrus-imapd-2.4 master --
    229 files changed, 20358 insertions(+), 8177 deletions(-)

Do you think I should start working on the master branch?
Is it stable/usable? Is it similar to the 2.4.x branch in terms of usage and
administration? Has the build system been improved in the master branch
(especially with respect to the issues I'm facing)?

Thanks,
Luca

[Buildroot] cyrus-sasl in buildroot: apparent CFLAGS mismatch on saslauthd

[Luca Ceresoli]

Hi,
the first issue I hit while trying to cross-compile cyrus-sasl within 
buildroot
is in the configure step.

Cyrus developers might be interested to know that buildroot, in the case of
the Cyrus packages, builds packages unpacking the source package (the 
released
tarball in thi case), optionally patches them, then executes ./configure 
with
suitable environment and options, then launches make and make install.

I instructed buildroot to unpack cyrus-sasl 2.1.24rc1 and run its 
./configure
script with most options disabled to make things simple. It ran fine 
until it
entered the saslauthd subdir with the following error (the full 
compilation log
is attached).

   $ pwd
   /home/murray/devel/buildroot
   $ make cyrus-sasl-dirclean
   rm -Rf /home/murray/devel/buildroot/output/build/cyrus-sasl-2.1.24rc1
   $ make cyrus-sasl 2>&1 | tee cyrus-sasl.log
   [...]
   configure: configuring in saslauthd
   configure: running /bin/bash './configure' --prefix=/usr  
'--target=arm-linux' '--host=arm-linux' '--build=i686-pc-linux-gnu' 
'--prefix=/usr' '--exec-prefix=/usr' '--sysconfdir=/etc' 
'--disable-gtk-doc' '--enable-static' '--enable-shared' 
'--disable-dependency-tracking' '--disable-sample' '--disable-checkapop' 
'--enable-login' '--disable-otp' '--disable-srp' '--disable-gssapi' 
'--disable-cram' '--disable-anon' '--disable-digest' '--disable-ntlm' 
'--disable-passdss' '--without-des' '--without-openssl' 
'CC=/home/murray/devel/buildroot/output/host/usr/bin/ccache 
/home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gcc' 
'CFLAGS=-Os -pipe -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE 
-D_FILE_OFFSET_BITS=64' 
'CPP=/home/murray/devel/buildroot/output/host/usr/bin/arm-linux-cpp' 
'LDFLAGS=' 'build_alias=i686-pc-linux-gnu' 'host_alias=arm-linux' 
'target_alias=arm-linux' --cache-file=.././config.cache --srcdir=.
   configure: loading cache .././config.cache
   configure: error: `CFLAGS' has changed since the previous run:
   configure:   former value:  -Os -pipe -Os  -D_LARGEFILE_SOURCE 
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
   configure:   current value: -Os -pipe -Os -D_LARGEFILE_SOURCE 
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
   configure: error: changes in the environment can compromise the build
   configure: error: run `make distclean' and/or `rm .././config.cache' 
and start over
   configure: error: /bin/bash './configure' failed for saslauthd
   $

It is clear from the output that the "former" and "current" CFLAGS are
substantially identical, differing only in spacing.

After a quick look at the CFLAGS handling on both the buildroot and 
cyrus-sasl
side I gave up and, since I do not need saslauthd, I worked around with this
ugly patch:

  diff -u -r cyrus-sasl-2.1.24rc1-orig/configure 
cyrus-sasl-2.1.24rc1/configure
  --- cyrus-sasl-2.1.24rc1-orig/configure    2009-05-07 
16:24:25.000000000 +0200
  +++ cyrus-sasl-2.1.24rc1/configure    2011-05-02 15:10:36.793333728 +0200
  @@ -15542,8 +15542,10 @@
  -subdirs="$subdirs saslauthd"

  +if test "$with_saslauthd" != no; then
  +  subdirs="$subdirs saslauthd"
  +fi

and adding --disable-saslauthd to the ./configure options within buildroot's
cyrus-sasl.mk (which contains the buildroot instructions to build 
cyrus-sasl).

This leaves two questions open:
  - Why not adding a --disable-saslauthd option (the autoconf/automake 
way) thus
    allowing to build cyrus-sasl without this component?
  - How to solve the problem with the CFLAGS spacing handling, for those who
    might need saslauthd cross-compiled in buildroot?

The full compilation log is attached.

Thanks,
Luca

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-sasl-saslauthd-cflags-mismatch.log.gz
Type: application/x-gzip
Size: 3481 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110624/cd235ed1/attachment.bin>

[Buildroot] cyrus-sasl in buildroot: makemd5 is wrongly cross-compiled

[Luca Ceresoli]

Hi,
after having worked around the "apparent CFLAGS mismatch" problem, I hit
another cross-compilation error while trying to build cyrus-sasl in 
buildroot:

   $ make cyrus-sasl-dirclean
   $ make cyrus-sasl 2>&1 | tee cyrus-sasl.log
   [...]
   /home/murray/devel/buildroot/output/host/usr/bin/ccache 
/home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gcc -Wall -W 
-Os -pipe -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE 
-D_FILE_OFFSET_BITS=64 -o makemd5 makemd5.o -lresolv
   rm -f md5global.h
   ./makemd5 md5global.h
   /bin/bash: ./makemd5: cannot execute binary file
   make[3]: *** [md5global.h] Error 126
   [...]

The problem here is pretty clear: makemd5, which is meant to run on the 
host,
has been cross-compiled instead:

   $ file output/build/cyrus-sasl-2.1.24rc1/include/makemd5
   output/build/cyrus-sasl-2.1.24rc1/include/makemd5: ELF 32-bit LSB 
executable, ARM, version 1 (SYSV), dynamically linked (uses shared 
libs), for GNU/Linux 2.6.31, not stripped

Attached is my patch to allow makemd5 to correctly (non-cross) compiled.
Basically, it consists in replacing $(CC) with $(HOSTCC) in cyrus-sasl's
include/Makefile.in.
Note that I'm not the first that faced this problem. My patch has been
inspired by:
http://tech.dir.groups.yahoo.com/group/nslu2-linux/message/4044?var=0

The build log is attached too.

Luca

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-sasl-makemd5.log.gz
Type: application/x-gzip
Size: 3477 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110624/6e781892/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-sasl-hostcc.patch
Type: text/x-patch
Size: 1317 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110624/6e781892/attachment-0003.bin>

[Buildroot] cyrus-sasl in buildroot: installs files with "arm-linux-" prefix

[Luca Ceresoli]

Hi,

after somehow working around the two previous issues I reported in
cross-compiling cyrus-sasl in buildroot ("apparent CFLAGS mismatch on 
saslauthd"
and "fix HOSTCC and related variables") I finally got buildroot to complete
compilation and installation of cyrus-sasl!

Alas, not all is perfect. While libraries look correctly installed, the 
manpages
and executables are installed with an unwanted "arm-linux-" prefix:

   $ make cyrus-sasl-install-target
   [...]
   $ find output/target/ -name "*sasl*"
   output/target/usr/man/man3/arm-linux-sasl_verifyfile_t.3
   output/target/usr/man/man3/arm-linux-sasl_idle.3
   output/target/usr/man/man3/arm-linux-sasl_checkpass.3
   output/target/usr/man/man3/arm-linux-sasl_canon_user_t.3
   [...]
   output/target/usr/man/man3/arm-linux-sasl_getrealm_t.3
   output/target/usr/man/man8/arm-linux-saslpasswd2.8
   output/target/usr/man/man8/arm-linux-sasldblistusers2.8
   output/target/usr/sbin/arm-linux-sasldblistusers2
   output/target/usr/sbin/arm-linux-saslpasswd2
   output/target/usr/lib/libsasl2.so.2
   output/target/usr/lib/libsasl2.so
   output/target/usr/lib/libsasl2.la
   output/target/usr/lib/libsasl2.so.2.0.24
   output/target/usr/lib/libsasl2.a
   output/target/usr/lib/sasl2
   output/target/usr/lib/sasl2/libsasldb.so.2.0.24
   output/target/usr/lib/sasl2/libsasldb.so.2
   output/target/usr/lib/sasl2/libsasldb.la
   output/target/usr/lib/sasl2/libsasldb.a
   output/target/usr/lib/sasl2/libsasldb.so
   output/target/usr/include/sasl
   output/target/usr/include/sasl/sasl.h
   output/target/usr/include/sasl/saslutil.h
   output/target/usr/include/sasl/saslplug.h
   $

Once more, the install log is attached.

This might not be a real issue for manpages and static libs, which are 
removed
by buildroot later in the build process (unless one wants them 
installed, which
is uncommon in an embedded device).

It is more annoying for sasldblistusers2 and saslpasswd2.

I currently have no solution to this problem; instructing buildroot to 
rename
the two files in a post-install hook would be trivial, but if anybody 
came up
with a solution that fixes the problem at its root, it sure would be better.

Luca

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-sasl-install.log.gz
Type: application/x-gzip
Size: 2151 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110624/f8a84d83/attachment.bin>

[Buildroot] cyrus-sasl in buildroot: installs files with "arm-linux-" prefix

[Luca Ceresoli]

Luca Ceresoli wrote:
> Alas, not all is perfect. While libraries look correctly installed, the
> manpages
> and executables are installed with an unwanted "arm-linux-" prefix:

I could fix this problem adding --program-prefix='' to the ./configure
options.

Thanks to Yann for having unconsciously led me to the solution
(http://lists.busybox.net/pipermail/buildroot/2011-July/044757.html).

Luca

[Buildroot] cyrus-imapd in buildroot: imtest does not cross-compile correctly

[Luca Ceresoli]

Hi,

trying to cross-compile cyrus-imapd 2.4.9 in buildroot, with most options
disabled to keep things simple, I got this:

   [...]
   ### Making install in /home/murray/devel/buildroot/output/build/cyrus-imapd-2.4.9/imtest
   make[2]: Entering directory `/home/murray/devel/buildroot/output/build/cyrus-imapd-2.4.9/imtest'
   /usr/bin/install -c -s -m 755 imtest /home/murray/devel/buildroot/output/target/usr/bin
   strip: Unable to recognise the format of the input file `/home/murray/devel/buildroot/output/target/usr/bin/imtest'
   /usr/bin/install: strip process terminated abnormally
   make[2]: *** [install] Error 1
   make[2]: Leaving directory `/home/murray/devel/buildroot/output/build/cyrus-imapd-2.4.9/imtest'
   [...]

Clearly, the host install (/usr/bin/install) is being used and it is normal that
it fails in stripping cross-compiled files.
What I don't know is how to tell ./configure that it should not instruct install
to strip files, i.e. not pass the -s option.

After a very brief research, I cowardly opted for patching cyrus-imapd to
completely disable imtest compilation, as I don't need it.

Any idea on how to really fix this issue, instead of just disabling imtest?
And how about adding a --disable-imtest option?

Attached are the build log and my awful patch.

Luca


-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-imapd-imtest.log.gz
Type: application/x-gzip
Size: 6664 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110624/7b28cc1f/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-imapd-disable-imtest.patch
Type: text/x-patch
Size: 1755 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110624/7b28cc1f/attachment-0001.bin>

[Buildroot] cyrus-imapd: static binaries are huge + choose binaries to install

[Luca Ceresoli]

Hi,

the last two issues that I'm facing with cyrus-imapd are not strictly 
related to buildroot, nor to cross-compilation, but rather to using 
cyrus-imapd in an embedded system.

I'm discussing them together in this e-mail as they are closely related.

The cyrus-imapd build system apparently does not currently allow to 
build shared executables. This results in needlessly large files.
In a basic build for x86 (./configure && make), most of the executables 
in imap/ have a size between 2.2 and 2.5 MB, and the sum is 62.8 MB.

This may not be a problem on a classic mail server, but storage space 
can be a precious resource on embedded systems.
Clearly, using  shared libraries would reduce the overall space required.

It looks like I'm not the first to think about this. In
   http://bugzilla.cyrusimap.org/show_bug.cgi?id=3095
a patch was proposed and rejected. Unfortunately I could not apply it, 
possibly because it's based on an old code base (2008).

Would it be feasible to add shared build support in the cyrus-imapd 
build system?


For the same motivation (saving precious storage space), it would be 
extremely useful to disable the compilation and installation of selected 
executables.

The standard distribution installed as many as 30 executables in 
/usr/cyrus/bin/, which cross-compiled for ARM and stripped take 32 MB. 
Those that I really need take <7 MB and I wish I could get rid of them 
by passing --disable-pop3proxyd or similar to ./configure.

AFAIK this is not currently possible in cyrus-imapd. Is this correct?
Do the Cyrus developers have anything similar in mind for the future of 
cyrus-imapd? Is this feasible?

Thanks,
Luca

[Buildroot] Cross-compiling Cyrus in Buildroot

[Luca Ceresoli]

Luca Ceresoli wrote:
> Hi Cyrus and Buildroot developers,
>
> I'm working to add cyrus-sasl and cyrus-imapd (http://www.cyrusimap.org/)
> to buildroot (http://www.buildroot.net/) for use on embedded systems.
>
> On my road I hit various problems with the cross-build process:
> configuration, compilation and installation.
> I could work around some, but not really fix them as they should.

After a couple cleanup cycles my work on cyrus-imapd and cyrus-sasl has
reached a pretty stable shape. It is not perfect, but it is working.

Here's the status w.r.t. the problems I previously reported.
There is not much news, since I received almost no feedback so far.

> Here's a list of the issues I found:
> - cyrus-sasl: apparent CFLAGS mismatch on saslauthd;

I applied my proposed patch to disable saslauthd.

> - cyrus-sasl: makemd5 is wrongly cross-compiled;

I applied my proposed patch which allows to correctly host-compile
makemd5.
I wonder if it should be applied upstream.

> - cyrus-sasl: installs files with "arm-linux-" prefix

Fixed with --program-prefix=''.

> - cyrus-imapd: imtest does not cross-compile correctly;

I applied my proposed patch to disable imtest.

> - cyrus-imapd: static binaries are huge, cannot build them shared;
> - cyrus-imapd: choose binaries to (build and) install;

Not solved. The good thing is that ????? ???????? Dilyan Palauzov
volounteered to update the cyrus-imapd buildsystem
(http://asg.andrew.cmu.edu/archive/message.php?mailbox=archive.cyrus-devel&searchterm=automake&msg=3040),
which would fix these two issues, and possibly also the previous one
with imtest.

I'm going to send a patch series with the current status of my work.
I'd like to know from the buildroot developers whether it is good
enough to be applied or the mentioned problems are too big.

Luca