Compilation error crossbuilding tools.

Compilation error crossbuilding tools.

[Sander Bogaert]

Hi,

I'm running the latest xen unstable branch on an Arndale board ( and
FastModels A15 simulator ). I'm now trying to build the xen tools (
32bit ) following this guide:
http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling

The only deviation I made is adding my user to
/etc/schroot/chroot.d/raring-armhf-cross :
...
groups=root,sbuild,username
root-groups=root,sbuild,username
...

I did this because I was getting ( on chrooting in as a regular user,
schroot -c raring-armhf-cross on my debian wheezy system):
E: Access not authorised
I: You do not have permission to access the schroot service.
I: This failure will be reported.

( and yes I added myself to the sbuild group :-) ) After doing this I
can chroot in and end up in my home dir on my local system, which is a
bit strange but not an issue I think.

On compilation ( followed the wiki ) I get:
....
arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
-Wdeclaration-after-statement -Wno-unused-but-set-variable
-Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
 -Werror -Wno-unused -I../lib
-I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
-I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
-I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
-I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
-D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
block-qcow2.c
block-qcow2.c: In function 'bdrv_pread':
block-qcow2.c:238:4: error: format '%llx' expects argument of type
'long long unsigned int', but argument 5 has type '__off_t'
[-Werror=format]
cc1: all warnings being treated as errors
make[5]: *** [block-qcow2.o] Error 1
make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
make[4]: *** [subdir-install-drivers] Error 2
make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
make[3]: *** [subdirs-install] Error 2
make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
make[2]: *** [subdir-install-blktap] Error 2
make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
make[1]: *** [subdirs-install] Error 2
make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
make: *** [install-tools] Error 2

So I have two questions:
1. Did anyone else run into this and/or knows how to solve it?
2. Should the wiki be updated on chrooting as a regular user?

Thanks

Re: Compilation error crossbuilding tools.

[Julien Grall]

On 04/06/2013 03:29 PM, Sander Bogaert wrote:

> Hi,

Hello,

> I'm running the latest xen unstable branch on an Arndale board ( and
> FastModels A15 simulator ). I'm now trying to build the xen tools (
> 32bit ) following this guide:
> http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling
> 
> The only deviation I made is adding my user to
> /etc/schroot/chroot.d/raring-armhf-cross :
> ...
> groups=root,sbuild,username
> root-groups=root,sbuild,username
> ...
> 
> I did this because I was getting ( on chrooting in as a regular user,
> schroot -c raring-armhf-cross on my debian wheezy system):
> E: Access not authorised
> I: You do not have permission to access the schroot service.
> I: This failure will be reported.
> 
> ( and yes I added myself to the sbuild group :-) ) After doing this I
> can chroot in and end up in my home dir on my local system, which is a
> bit strange but not an issue I think.

I had the same issue when I added my user to the sbuild group. The new
user credentials are only take into account after the next log on. Did
you try to log off/log on?

Cheers,

Julien

Re: Compilation error crossbuilding tools.

[Stefano Stabellini]

On Sat, 6 Apr 2013, Sander Bogaert wrote:
> On compilation ( followed the wiki ) I get:
> ....
> arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
> -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> -Wdeclaration-after-statement -Wno-unused-but-set-variable
> -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
>  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
>  -Werror -Wno-unused -I../lib
> -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
> -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
> -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
> block-qcow2.c
> block-qcow2.c: In function 'bdrv_pread':
> block-qcow2.c:238:4: error: format '%llx' expects argument of type
> 'long long unsigned int', but argument 5 has type '__off_t'
> [-Werror=format]
> cc1: all warnings being treated as errors
> make[5]: *** [block-qcow2.o] Error 1
> make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
> make[4]: *** [subdir-install-drivers] Error 2
> make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> make[3]: *** [subdirs-install] Error 2
> make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> make[2]: *** [subdir-install-blktap] Error 2
> make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
> make[1]: *** [subdirs-install] Error 2
> make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
> make: *** [install-tools] Error 2

Reading the error, I think that the appended patch (completely untested)
should fix the problem:

diff --git a/tools/blktap/drivers/block-qcow2.c b/tools/blktap/drivers/block-qcow2.c
index ceda4f0..903d065 100644
--- a/tools/blktap/drivers/block-qcow2.c
+++ b/tools/blktap/drivers/block-qcow2.c
@@ -30,6 +30,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <sys/mman.h>
 #include <sys/stat.h>
 
 #include "tapdisk.h"

Re: Compilation error crossbuilding tools.

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] Compilation error crossbuilding tools."):
> On Sat, 6 Apr 2013, Sander Bogaert wrote:
...
> > block-qcow2.c: In function 'bdrv_pread':
> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
> > 'long long unsigned int', but argument 5 has type '__off_t'
> > [-Werror=format]
...
> Reading the error, I think that the appended patch (completely untested)
> should fix the problem:
...
> +#include <sys/mman.h>

???

How is that going to help ?  The problem seems to be that the return
value from lseek is being passed to printf(PRIx64).  I guess we have
FILE_OFFSET_BITS=64 or some such so that the return value from lseek
is 32-bit.

Ian.

Re: Compilation error crossbuilding tools.

[Stefano Stabellini]

On Mon, 8 Apr 2013, Ian Jackson wrote:
> Stefano Stabellini writes ("Re: [Xen-devel] Compilation error crossbuilding tools."):
> > On Sat, 6 Apr 2013, Sander Bogaert wrote:
> ...
> > > block-qcow2.c: In function 'bdrv_pread':
> > > block-qcow2.c:238:4: error: format '%llx' expects argument of type
> > > 'long long unsigned int', but argument 5 has type '__off_t'
> > > [-Werror=format]
> ...
> > Reading the error, I think that the appended patch (completely untested)
> > should fix the problem:
> ...
> > +#include <sys/mman.h>
> 
> ???
> 
> How is that going to help ?  The problem seems to be that the return
> value from lseek is being passed to printf(PRIx64).  I guess we have
> FILE_OFFSET_BITS=64 or some such so that the return value from lseek
> is 32-bit.

Yes, you are right.
Rather than reading the error I should have written misreading the
error :)

Re: Compilation error crossbuilding tools.

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] Compilation error crossbuilding tools."):
> On Mon, 8 Apr 2013, Ian Jackson wrote:
> > The problem seems to be that the return
> > value from lseek is being passed to printf(PRIx64).  I guess we have
> > FILE_OFFSET_BITS=64 or some such so that the return value from lseek
> > is 32-bit.

I meant =32 of course.

> Yes, you are right.

tools/blktap is never going to be supported on arm, is it ?  So
perhaps the right answer is just to disable the build.

But we should figure out how to set FILE_OFFSET_BITS=64 anyway perhaps ?

Ian.

Re: Compilation error crossbuilding tools.

[Stefano Stabellini]

On Mon, 8 Apr 2013, Ian Jackson wrote:
> Stefano Stabellini writes ("Re: [Xen-devel] Compilation error crossbuilding tools."):
> > On Mon, 8 Apr 2013, Ian Jackson wrote:
> > > The problem seems to be that the return
> > > value from lseek is being passed to printf(PRIx64).  I guess we have
> > > FILE_OFFSET_BITS=64 or some such so that the return value from lseek
> > > is 32-bit.
> 
> I meant =32 of course.
> 
> > Yes, you are right.
> 
> tools/blktap is never going to be supported on arm, is it ?

Correct


> So perhaps the right answer is just to disable the build.

I agree, we should just disable it.


> But we should figure out how to set FILE_OFFSET_BITS=64 anyway perhaps ?
 
indeed

Re: Compilation error crossbuilding tools.

[Sander Bogaert]

On 8 April 2013 19:08, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
>
> On Mon, 8 Apr 2013, Ian Jackson wrote:
> > Stefano Stabellini writes ("Re: [Xen-devel] Compilation error crossbuilding tools."):
> > > On Mon, 8 Apr 2013, Ian Jackson wrote:
> > > > The problem seems to be that the return
> > > > value from lseek is being passed to printf(PRIx64).  I guess we have
> > > > FILE_OFFSET_BITS=64 or some such so that the return value from lseek
> > > > is 32-bit.
> >
> > I meant =32 of course.
> >
> > > Yes, you are right.
> >
> > tools/blktap is never going to be supported on arm, is it ?
>
> Correct
>
>
> > So perhaps the right answer is just to disable the build.
>
> I agree, we should just disable it.
>
>
> > But we should figure out how to set FILE_OFFSET_BITS=64 anyway perhaps ?
>
> indeed

I disabled blktap and blktap2 in Makefile & Rules.mk, I would attach a
patch but what I did is not very clean :-) It would be disabled for
all linux builds. I didn't see a way to differentiate in architectures
in those files. The tools build fine now.

In response to Julien, after a relogin things worked as described.
I'll add this small caveat to the wiki.

A question: why is it blktap wouldn't be used on ARM?

I think I fail to see the big difference between blktap and the
'classic' blkback & blkfront split driver approach. I read the wiki
entry and the original notes from the blktap author. Is this correct:
the domU blktap driver is like dom0's blkback driver and the userspace
tools replace the classic domU's blkfront part? This provides a more
abstract interface to userspace allowing more 'freedom' in accessing
the disk from domU userspace?

Re: Compilation error crossbuilding tools.

[Stefano Stabellini]

On Tue, 9 Apr 2013, Sander Bogaert wrote:
> On 8 April 2013 19:08, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> >
> > On Mon, 8 Apr 2013, Ian Jackson wrote:
> > > Stefano Stabellini writes ("Re: [Xen-devel] Compilation error crossbuilding tools."):
> > > > On Mon, 8 Apr 2013, Ian Jackson wrote:
> > > > > The problem seems to be that the return
> > > > > value from lseek is being passed to printf(PRIx64).  I guess we have
> > > > > FILE_OFFSET_BITS=64 or some such so that the return value from lseek
> > > > > is 32-bit.
> > >
> > > I meant =32 of course.
> > >
> > > > Yes, you are right.
> > >
> > > tools/blktap is never going to be supported on arm, is it ?
> >
> > Correct
> >
> >
> > > So perhaps the right answer is just to disable the build.
> >
> > I agree, we should just disable it.
> >
> >
> > > But we should figure out how to set FILE_OFFSET_BITS=64 anyway perhaps ?
> >
> > indeed
> 
> I disabled blktap and blktap2 in Makefile & Rules.mk, I would attach a
> patch but what I did is not very clean :-) It would be disabled for
> all linux builds. I didn't see a way to differentiate in architectures
> in those files. The tools build fine now.
> 
> In response to Julien, after a relogin things worked as described.
> I'll add this small caveat to the wiki.
> 
> A question: why is it blktap wouldn't be used on ARM?
> 
> I think I fail to see the big difference between blktap and the
> 'classic' blkback & blkfront split driver approach. I read the wiki
> entry and the original notes from the blktap author. Is this correct:
> the domU blktap driver is like dom0's blkback driver and the userspace
> tools replace the classic domU's blkfront part?

blktap is just the block backend, alternative to blkback.
However blktap has a complex architecture and needs a kernel module as
well as some userspace libraries.


> This provides a more
> abstract interface to userspace allowing more 'freedom' in accessing
> the disk from domU userspace?

The blktap kernel module has never gone upstream in Linux.
xl can use QEMU as block backend, providing most of the features
bltkap provides, without the need for a kernel module.

Re: Compilation error crossbuilding tools.

[Ian Campbell]

On Mon, 2013-04-08 at 11:28 +0100, Stefano Stabellini wrote:
> On Sat, 6 Apr 2013, Sander Bogaert wrote:
> > On compilation ( followed the wiki ) I get:
> > ....
> > arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
> > -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> > -Wdeclaration-after-statement -Wno-unused-but-set-variable
> > -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
> >  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
> >  -Werror -Wno-unused -I../lib
> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> > -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
> > block-qcow2.c
> > block-qcow2.c: In function 'bdrv_pread':
> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
> > 'long long unsigned int', but argument 5 has type '__off_t'
> > [-Werror=format]
> > cc1: all warnings being treated as errors
> > make[5]: *** [block-qcow2.o] Error 1
> > make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
> > make[4]: *** [subdir-install-drivers] Error 2
> > make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> > make[3]: *** [subdirs-install] Error 2
> > make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> > make[2]: *** [subdir-install-blktap] Error 2
> > make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
> > make[1]: *** [subdirs-install] Error 2
> > make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
> > make: *** [install-tools] Error 2
> 
> Reading the error, I think that the appended patch (completely untested)
> should fix the problem:

blktap(1) shouldn't even be attempted to build on ARM. My series 
"arm: tools: build for arm64 and enable cross-compiling for both arm32
and arm64" contains "tools: only build blktap1 on x86".

That series is a prerequisite for cross building and is not yet applied.

Ian.

Re: Compilation error crossbuilding tools.

[Sander Bogaert]

On 10 April 2013 14:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2013-04-08 at 11:28 +0100, Stefano Stabellini wrote:
>> On Sat, 6 Apr 2013, Sander Bogaert wrote:
>> > On compilation ( followed the wiki ) I get:
>> > ....
>> > arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
>> > -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
>> > -Wdeclaration-after-statement -Wno-unused-but-set-variable
>> > -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
>> >  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
>> >  -Werror -Wno-unused -I../lib
>> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
>> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
>> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
>> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
>> > -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
>> > block-qcow2.c
>> > block-qcow2.c: In function 'bdrv_pread':
>> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
>> > 'long long unsigned int', but argument 5 has type '__off_t'
>> > [-Werror=format]
>> > cc1: all warnings being treated as errors
>> > make[5]: *** [block-qcow2.o] Error 1
>> > make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
>> > make[4]: *** [subdir-install-drivers] Error 2
>> > make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
>> > make[3]: *** [subdirs-install] Error 2
>> > make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
>> > make[2]: *** [subdir-install-blktap] Error 2
>> > make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
>> > make[1]: *** [subdirs-install] Error 2
>> > make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
>> > make: *** [install-tools] Error 2
>>
>> Reading the error, I think that the appended patch (completely untested)
>> should fix the problem:
>
> blktap(1) shouldn't even be attempted to build on ARM. My series
> "arm: tools: build for arm64 and enable cross-compiling for both arm32
> and arm64" contains "tools: only build blktap1 on x86".

So does this mean I'm building the tools wrong? I'm in a chroot
described by the wiki and my target arch was set correctly.

>
> That series is a prerequisite for cross building and is not yet applied.
>
> Ian.
>
>

After building the tools and copying them to the image file I again
get some error. First of all, for the newer xl tool the debian wheezy
I debootstrapped missed the correct glibc version.
The wiki says the developpers use debian wheezy so I wonder how you
fixed this issue?

root@x200:~# xl
xl: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.15' not found
(required by xl)
xl: /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.15' not found
(required by /usr/lib/libxenlight.so.2.0)

Because I wanted to move on I tried the xm tool, which complains about
missing modules:
root@x200:~# xm
Traceback (most recent call last):
  File "/usr/sbin/xm", line 5, in <module>
    from xen.xm import main
ImportError: No module named xen.xm

My python path:
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages']

Searching the filesystem the module was nowhere to be found. Shouldn't
it have build in the chroot when I issued 'make dist-tools'?  I
searched the dist/install directory without result:
...s/source/xen/xen-unstable/dist/install (git)-[guest-smp-3] % find .
| grep "xm\|main.py"
./usr/sbin/xm
./etc/xen/xmexample1
./etc/xen/xmexample2
./etc/xen/xm-config.xml
./etc/xen/xmexample.hvm-stubdom
./etc/xen/xmexample.vti
./etc/xen/xmexample3
./etc/xen/xmexample.pv-grub
./etc/xen/xmexample.nbd
./etc/xen/xmexample.hvm

The module just doesn't seem to be there? I just copied the whole
xensource/dist/install directory to my rootfs.

All of this is based on
http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling
and I'm really just following the steps and doing some additional
research when they fail. Has anyone actually build the tools following
that guide?

Thanks

Re: Compilation error crossbuilding tools.

[Ian Campbell]

On Wed, 2013-04-10 at 15:11 +0100, Sander Bogaert wrote:
> On 10 April 2013 14:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Mon, 2013-04-08 at 11:28 +0100, Stefano Stabellini wrote:
> >> On Sat, 6 Apr 2013, Sander Bogaert wrote:
> >> > On compilation ( followed the wiki ) I get:
> >> > ....
> >> > arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
> >> > -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> >> > -Wdeclaration-after-statement -Wno-unused-but-set-variable
> >> > -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
> >> >  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
> >> >  -Werror -Wno-unused -I../lib
> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> >> > -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
> >> > block-qcow2.c
> >> > block-qcow2.c: In function 'bdrv_pread':
> >> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
> >> > 'long long unsigned int', but argument 5 has type '__off_t'
> >> > [-Werror=format]
> >> > cc1: all warnings being treated as errors
> >> > make[5]: *** [block-qcow2.o] Error 1
> >> > make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
> >> > make[4]: *** [subdir-install-drivers] Error 2
> >> > make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> >> > make[3]: *** [subdirs-install] Error 2
> >> > make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> >> > make[2]: *** [subdir-install-blktap] Error 2
> >> > make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
> >> > make[1]: *** [subdirs-install] Error 2
> >> > make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
> >> > make: *** [install-tools] Error 2
> >>
> >> Reading the error, I think that the appended patch (completely untested)
> >> should fix the problem:
> >
> > blktap(1) shouldn't even be attempted to build on ARM. My series
> > "arm: tools: build for arm64 and enable cross-compiling for both arm32
> > and arm64" contains "tools: only build blktap1 on x86".
> 
> So does this mean I'm building the tools wrong? I'm in a chroot
> described by the wiki and my target arch was set correctly.

Quoting
http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling#Obtaining_a_cross-buildable_Xen_tree

        Patches to enable crossbuilding of Xen have been posted to the
        xen-devel mailing list but have not yet been accepted into the
        tree. A git branch with the necessary patches applied can be
        found at git://xenbits.xen.org/people/ianc/xen.git arm-cross.

Are you using the referenced tree? It contains the series I mention
above.

Ian.

Re: Compilation error crossbuilding tools.

[Sander Bogaert]

On 10 April 2013 16:49, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2013-04-10 at 15:11 +0100, Sander Bogaert wrote:
>> On 10 April 2013 14:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > On Mon, 2013-04-08 at 11:28 +0100, Stefano Stabellini wrote:
>> >> On Sat, 6 Apr 2013, Sander Bogaert wrote:
>> >> > On compilation ( followed the wiki ) I get:
>> >> > ....
>> >> > arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
>> >> > -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
>> >> > -Wdeclaration-after-statement -Wno-unused-but-set-variable
>> >> > -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
>> >> >  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
>> >> >  -Werror -Wno-unused -I../lib
>> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
>> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
>> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
>> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
>> >> > -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
>> >> > block-qcow2.c
>> >> > block-qcow2.c: In function 'bdrv_pread':
>> >> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
>> >> > 'long long unsigned int', but argument 5 has type '__off_t'
>> >> > [-Werror=format]
>> >> > cc1: all warnings being treated as errors
>> >> > make[5]: *** [block-qcow2.o] Error 1
>> >> > make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
>> >> > make[4]: *** [subdir-install-drivers] Error 2
>> >> > make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
>> >> > make[3]: *** [subdirs-install] Error 2
>> >> > make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
>> >> > make[2]: *** [subdir-install-blktap] Error 2
>> >> > make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
>> >> > make[1]: *** [subdirs-install] Error 2
>> >> > make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
>> >> > make: *** [install-tools] Error 2
>> >>
>> >> Reading the error, I think that the appended patch (completely untested)
>> >> should fix the problem:
>> >
>> > blktap(1) shouldn't even be attempted to build on ARM. My series
>> > "arm: tools: build for arm64 and enable cross-compiling for both arm32
>> > and arm64" contains "tools: only build blktap1 on x86".
>>
>> So does this mean I'm building the tools wrong? I'm in a chroot
>> described by the wiki and my target arch was set correctly.
>
> Quoting
> http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling#Obtaining_a_cross-buildable_Xen_tree
>
>         Patches to enable crossbuilding of Xen have been posted to the
>         xen-devel mailing list but have not yet been accepted into the
>         tree. A git branch with the necessary patches applied can be
>         found at git://xenbits.xen.org/people/ianc/xen.git arm-cross.
>
> Are you using the referenced tree? It contains the series I mention
> above.
>
> Ian.
>

The branch information in my copy-pastes was indeed wrong but I also
tried the branch you mention:
origin git://xenbits.xen.org/people/ianc/xen.git (fetch)
branch arm-cross
latest commit: 6e258f232d5c4409a549f2b03651bb289703aab7

I have the same issue. I think I miss something, I don't understand
what you reference to with 'series'? Also I think this python issue
isn't related to the glibc version mismatch, or does your build use an
older version ( one that's shipped with debian wheezy )?

Thanks a lot for you help!

Re: Compilation error crossbuilding tools.

[Ian Campbell]

On Wed, 2013-04-10 at 22:59 +0100, Sander Bogaert wrote:
> On 10 April 2013 16:49, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Wed, 2013-04-10 at 15:11 +0100, Sander Bogaert wrote:
> >> On 10 April 2013 14:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >> > On Mon, 2013-04-08 at 11:28 +0100, Stefano Stabellini wrote:
> >> >> On Sat, 6 Apr 2013, Sander Bogaert wrote:
> >> >> > On compilation ( followed the wiki ) I get:
> >> >> > ....
> >> >> > arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
> >> >> > -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> >> >> > -Wdeclaration-after-statement -Wno-unused-but-set-variable
> >> >> > -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
> >> >> >  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
> >> >> >  -Werror -Wno-unused -I../lib
> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
> >> >> > -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
> >> >> > block-qcow2.c
> >> >> > block-qcow2.c: In function 'bdrv_pread':
> >> >> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
> >> >> > 'long long unsigned int', but argument 5 has type '__off_t'
> >> >> > [-Werror=format]
> >> >> > cc1: all warnings being treated as errors
> >> >> > make[5]: *** [block-qcow2.o] Error 1
> >> >> > make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
> >> >> > make[4]: *** [subdir-install-drivers] Error 2
> >> >> > make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> >> >> > make[3]: *** [subdirs-install] Error 2
> >> >> > make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
> >> >> > make[2]: *** [subdir-install-blktap] Error 2
> >> >> > make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
> >> >> > make[1]: *** [subdirs-install] Error 2
> >> >> > make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
> >> >> > make: *** [install-tools] Error 2
> >> >>
> >> >> Reading the error, I think that the appended patch (completely untested)
> >> >> should fix the problem:
> >> >
> >> > blktap(1) shouldn't even be attempted to build on ARM. My series
> >> > "arm: tools: build for arm64 and enable cross-compiling for both arm32
> >> > and arm64" contains "tools: only build blktap1 on x86".
> >>
> >> So does this mean I'm building the tools wrong? I'm in a chroot
> >> described by the wiki and my target arch was set correctly.
> >
> > Quoting
> > http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling#Obtaining_a_cross-buildable_Xen_tree
> >
> >         Patches to enable crossbuilding of Xen have been posted to the
> >         xen-devel mailing list but have not yet been accepted into the
> >         tree. A git branch with the necessary patches applied can be
> >         found at git://xenbits.xen.org/people/ianc/xen.git arm-cross.
> >
> > Are you using the referenced tree? It contains the series I mention
> > above.
> >
> > Ian.
> >
> 
> The branch information in my copy-pastes was indeed wrong but I also
> tried the branch you mention:
> origin git://xenbits.xen.org/people/ianc/xen.git (fetch)
> branch arm-cross
> latest commit: 6e258f232d5c4409a549f2b03651bb289703aab7
> 
> I have the same issue. I think I miss something, I don't understand
> what you reference to with 'series'?

series == patch series, i.e. a sequence of patches posted to the list.

>  Also I think this python issue

python issue? The error message above is about blktap.

> isn't related to the glibc version mismatch,

What glibc version mismatch?

> or does your build use an older version ( one that's shipped with debian wheezy )?

If you are following the sbuild cross build stuff on the wiki you will
be running Ubuntu in the chroot, not Debian.

Ian

Re: Compilation error crossbuilding tools.

[Sander Bogaert]

On 11 April 2013 09:55, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2013-04-10 at 22:59 +0100, Sander Bogaert wrote:
>> On 10 April 2013 16:49, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > On Wed, 2013-04-10 at 15:11 +0100, Sander Bogaert wrote:
>> >> On 10 April 2013 14:34, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> >> > On Mon, 2013-04-08 at 11:28 +0100, Stefano Stabellini wrote:
>> >> >> On Sat, 6 Apr 2013, Sander Bogaert wrote:
>> >> >> > On compilation ( followed the wiki ) I get:
>> >> >> > ....
>> >> >> > arm-linux-gnueabihf-gcc  -O1 -fno-omit-frame-pointer -marm -g
>> >> >> > -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
>> >> >> > -Wdeclaration-after-statement -Wno-unused-but-set-variable
>> >> >> > -Wno-unused-local-typedefs   -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d
>> >> >> >  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls
>> >> >> >  -Werror -Wno-unused -I../lib
>> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/libxc
>> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
>> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/xenstore
>> >> >> > -I/home/user/source/xen/xen-main/tools/blktap/drivers/../../../tools/include
>> >> >> > -D_GNU_SOURCE -DMEMSHR -I ../../memshr -pthread  -c -o block-qcow2.o
>> >> >> > block-qcow2.c
>> >> >> > block-qcow2.c: In function 'bdrv_pread':
>> >> >> > block-qcow2.c:238:4: error: format '%llx' expects argument of type
>> >> >> > 'long long unsigned int', but argument 5 has type '__off_t'
>> >> >> > [-Werror=format]
>> >> >> > cc1: all warnings being treated as errors
>> >> >> > make[5]: *** [block-qcow2.o] Error 1
>> >> >> > make[5]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap/drivers'
>> >> >> > make[4]: *** [subdir-install-drivers] Error 2
>> >> >> > make[4]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
>> >> >> > make[3]: *** [subdirs-install] Error 2
>> >> >> > make[3]: Leaving directory `/home/user/source/xen/xen-main/tools/blktap'
>> >> >> > make[2]: *** [subdir-install-blktap] Error 2
>> >> >> > make[2]: Leaving directory `/home/user/source/xen/xen-main/tools'
>> >> >> > make[1]: *** [subdirs-install] Error 2
>> >> >> > make[1]: Leaving directory `/home/user/source/xen/xen-main/tools'
>> >> >> > make: *** [install-tools] Error 2
>> >> >>
>> >> >> Reading the error, I think that the appended patch (completely untested)
>> >> >> should fix the problem:
>> >> >
>> >> > blktap(1) shouldn't even be attempted to build on ARM. My series
>> >> > "arm: tools: build for arm64 and enable cross-compiling for both arm32
>> >> > and arm64" contains "tools: only build blktap1 on x86".
>> >>
>> >> So does this mean I'm building the tools wrong? I'm in a chroot
>> >> described by the wiki and my target arch was set correctly.
>> >
>> > Quoting
>> > http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/CrossCompiling#Obtaining_a_cross-buildable_Xen_tree
>> >
>> >         Patches to enable crossbuilding of Xen have been posted to the
>> >         xen-devel mailing list but have not yet been accepted into the
>> >         tree. A git branch with the necessary patches applied can be
>> >         found at git://xenbits.xen.org/people/ianc/xen.git arm-cross.
>> >
>> > Are you using the referenced tree? It contains the series I mention
>> > above.
>> >
>> > Ian.
>> >
>>
>> The branch information in my copy-pastes was indeed wrong but I also
>> tried the branch you mention:
>> origin git://xenbits.xen.org/people/ianc/xen.git (fetch)
>> branch arm-cross
>> latest commit: 6e258f232d5c4409a549f2b03651bb289703aab7
>>
>> I have the same issue. I think I miss something, I don't understand
>> what you reference to with 'series'?
>
> series == patch series, i.e. a sequence of patches posted to the list.
>
>>  Also I think this python issue
>
> python issue? The error message above is about blktap.
>
>> isn't related to the glibc version mismatch,
>
> What glibc version mismatch?
>
>> or does your build use an older version ( one that's shipped with debian wheezy )?
>
> If you are following the sbuild cross build stuff on the wiki you will
> be running Ubuntu in the chroot, not Debian.

I know but my dom0 is Debian wheezy so it has an older version of
glibc ( I read Wheezy as dom0 is what other have too ).

> Ian
>

for some reason my previous e-mail in the conversation got lost:

http://lists.xen.org/archives/html/xen-devel/2013-04/msg00772.html

Re: Compilation error crossbuilding tools.

[Ian Campbell]

On Thu, 2013-04-11 at 10:23 +0100, Sander Bogaert wrote:
> I know but my dom0 is Debian wheezy so it has an older version of
> glibc ( I read Wheezy as dom0 is what other have too ).

If you are building in a Ubuntu environment then you very likely need to
run in an Ubuntu environment too I'm afraid.

You could try a newer glibc from Debian's experimental repository, but
YMMV using that...

Re: Compilation error crossbuilding tools.

[Sander Bogaert]

On 11 April 2013 11:25, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Thu, 2013-04-11 at 10:23 +0100, Sander Bogaert wrote:
>> I know but my dom0 is Debian wheezy so it has an older version of
>> glibc ( I read Wheezy as dom0 is what other have too ).
>
> If you are building in a Ubuntu environment then you very likely need to
> run in an Ubuntu environment too I'm afraid.
>
> You could try a newer glibc from Debian's experimental repository, but
> YMMV using that...
>

I think that a part of my message got lost again ( I should only reply
inline I guess and not append to the mail? )
http://lists.xen.org/archives/html/xen-devel/2013-04/msg00772.html
The glibc thing shouldn't be related to the python issue I think?

I can start over and deboostrap an Ubuntu system to have the correct
glibc, I was just wondering how other solved it ( wiki states people
use Wheezy for dom0 a lot)?

Thanks

Re: Compilation error crossbuilding tools.

[Ian Campbell]

On Thu, 2013-04-11 at 11:06 +0100, Sander Bogaert wrote:
> On 11 April 2013 11:25, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Thu, 2013-04-11 at 10:23 +0100, Sander Bogaert wrote:
> >> I know but my dom0 is Debian wheezy so it has an older version of
> >> glibc ( I read Wheezy as dom0 is what other have too ).
> >
> > If you are building in a Ubuntu environment then you very likely need to
> > run in an Ubuntu environment too I'm afraid.
> >
> > You could try a newer glibc from Debian's experimental repository, but
> > YMMV using that...
> >
> 
> I think that a part of my message got lost again ( I should only reply
> inline I guess and not append to the mail? )

Trimming unnecessary quoted material would help.

> http://lists.xen.org/archives/html/xen-devel/2013-04/msg00772.html

I see you are trying to use xm here -- xm/xend are not supported on ARM,
only xl is supported.

> The glibc thing shouldn't be related to the python issue I think?

Correct, they are unrelated.

There should be no python issue because you shouldn't be trying to use
anything which uses Python (specifically xend) in the first place. The
tree referenced from the wiki page includes a patch which disables
building of the python stuff altogether when cross compiling. Perhaps
you have some stale python stuff installed from your previous attempts
with another tree?

> I can start over and deboostrap an Ubuntu system to have the correct
> glibc, I was just wondering how other solved it ( wiki states people
> use Wheezy for dom0 a lot)?

Those people likely aren't cross compiling in an Ubuntu chroot, they are
probably building natively on an armhf system running Debian.

Ian.

Re: Compilation error crossbuilding tools.

[Sander Bogaert]

On 11-04-13 12:22, Ian Campbell wrote:
> On Thu, 2013-04-11 at 11:06 +0100, Sander Bogaert wrote:
>> On 11 April 2013 11:25, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>>> On Thu, 2013-04-11 at 10:23 +0100, Sander Bogaert wrote:
>>>> I know but my dom0 is Debian wheezy so it has an older version of
>>>> glibc ( I read Wheezy as dom0 is what other have too ).
>>>
>>> If you are building in a Ubuntu environment then you very likely need to
>>> run in an Ubuntu environment too I'm afraid.
>>>
>>> You could try a newer glibc from Debian's experimental repository, but
>>> YMMV using that...
>>>
>>
>> I think that a part of my message got lost again ( I should only reply
>> inline I guess and not append to the mail? )
> 
> Trimming unnecessary quoted material would help.
> 
>> http://lists.xen.org/archives/html/xen-devel/2013-04/msg00772.html
> 
> I see you are trying to use xm here -- xm/xend are not supported on ARM,
> only xl is supported.
> 
>> The glibc thing shouldn't be related to the python issue I think?
> 
> Correct, they are unrelated.
> 
> There should be no python issue because you shouldn't be trying to use
> anything which uses Python (specifically xend) in the first place. The
> tree referenced from the wiki page includes a patch which disables
> building of the python stuff altogether when cross compiling. Perhaps
> you have some stale python stuff installed from your previous attempts
> with another tree?

I just wanted to report the following, it's not a real issue because I
opted to force my wheezy system to a newer glibc. ( which isn't ideal I
know but other solutions failed ).

I use the correct tree and the new branch ( arm-cross2 ) with which the
wiki was updated today. I issue a make mrproper first and then continue
to build the tools:

$ CONFIG_SITE=/etc/dpkg-cross/cross-config.armhf ./configure
--build=x86_64-unknown-linux-gnu --host=arm-linux-gnueabihf
$ make dist-tools CROSS_COMPILE=arm-linux-gnueabihf- XEN_TARGET_ARCH=arm32


It really seems like xend and xm are still build:

$ find ./dist/install/ | grep "xm\|xend"
./dist/install/usr/sbin/xend
./dist/install/usr/sbin/xm
./dist/install/etc/default/xendomains
./dist/install/etc/init.d/xend
./dist/install/etc/init.d/xendomains
./dist/install/etc/udev/rules.d/xend.rules
./dist/install/etc/xen/xend-pci-quirks.sxp
./dist/install/etc/xen/xmexample1
./dist/install/etc/xen/xmexample2
./dist/install/etc/xen/xm-config.xml
./dist/install/etc/xen/xmexample.hvm-stubdom
./dist/install/etc/xen/xmexample.vti
./dist/install/etc/xen/xend-pci-permissive.sxp
./dist/install/etc/xen/xmexample3
./dist/install/etc/xen/xmexample.pv-grub
./dist/install/etc/xen/xend-config.sxp
./dist/install/etc/xen/xmexample.nbd
./dist/install/etc/xen/xmexample.hvm


> 
>> I can start over and deboostrap an Ubuntu system to have the correct
>> glibc, I was just wondering how other solved it ( wiki states people
>> use Wheezy for dom0 a lot)?
> 
> Those people likely aren't cross compiling in an Ubuntu chroot, they are
> probably building natively on an armhf system running Debian.
> 
> Ian.
> 

Re: Compilation error crossbuilding tools.

[Ian Campbell]

On Thu, 2013-04-11 at 18:21 +0100, Sander Bogaert wrote:
> It really seems like xend and xm are still build:

Well spotted, thanks.

These are just the wrapper scripts and example configurations, not the
full thing, but yes they shouldn't be installed (they don't live under
tools/python is how they got missed).

The xendomains stuff is supposed to be there since it works with xl too.

I'll have a think about getting rid of the rest...

Ian.