Re: cross compiling xen/ia64

Re: cross compiling xen/ia64

[Ian Jackson]

Alex Williamson writes ("[Xen-ia64-devel] cross compiling xen/ia64"):
>    With the latest pull of the ia64 tree, it's now possible, and fairly
> easy, to cross compile xen/ia64 on an x86_32/64 system.  Aron Griffis
> has been working on this and has posted detailed instructions on the Xen
> wiki: http://wiki.xensource.com/xenwiki/CrossCompiling  These
> instructions are based on Fedora8, which should be readily available.

I've been looking into this, and with Isaku Yamahata's patch the
hypervisor does build.  I'm hoping we can get that wired into the
staging propagation system very soon.  (Sorry about the delay getting
round to this.)

However, neither the kernel nor the tools build for me.

The kernel build stops asking for a kernel config option:

  mariner:xen-unstable-2.hg> PATH=/usr/local/ia64/bin:$PATH make XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu- -j4 dist-kernels
  for i in  linux-2.6-xen  ; do make $i-install || exit 1; done
  ...
  *
  Xen hypervisor support (XEN) [Y/n/?] y
    Xen/IA64 exposure p2m table (XEN_IA64_EXPOSE_P2M) [Y/n/?] (NEW) 

Obviously that's no good for an automatic test system.

The tools build stops with some kind of linker exec format problem
which I admit I don't understand and haven't investigated:

  mariner:xen-unstable-2.hg> PATH=/usr/local/ia64/bin:$PATH make XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu- -j4 dist-tools
  make -C tools install
  ...
  make[5]: Entering directory `/u/iwj/work/xen-unstable-2.hg/tools/blktap/lib'
  ia64-linux-gnu-gcc -O2 -fomit-frame-pointer -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-value -Wdeclaration-after-statement  -D__XEN_TOOLS__ -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE  -I. -I../../../tools/libxc -I../../../tools/include -I../../../tools/xenstore -I../../../tools/include -Werror -Wno-unused -fPIC -D _GNU_SOURCE -Wp,-MD,.libblktap.so.3.0.0.d -Wl,-soname -Wl,libblktap.so.3.0 -shared \
		   -L../../../tools/xenstore -lxenstore -o libblktap.so.3.0.0 xenbus.opic blkif.opic xs_api.opic
  /usr/local/ia64/lib/gcc/ia64-linux-gnu/4.1.1/../../../../ia64-linux-gnu/bin/ld: xenbus.opic: Relocations in generic ELF (EM: 3)
  /usr/local/ia64/lib/gcc/ia64-linux-gnu/4.1.1/../../../../ia64-linux-gnu/bin/ld: xenbus.opic: Relocations in generic ELF (EM: 3)
  xenbus.opic: could not read symbols: File in wrong format
  collect2: ld returned 1 exit status
  make[5]: *** [libblktap.so.3.0.0] Error 1
  make[5]: Leaving directory `/u/iwj/work/xen-unstable-2.hg/tools/blktap/lib'

This may be some problem with my cross building installation.  I built
it roughly following the instructions on the wiki.  I used the
prebuilt ia64 sys-root and did a single-stage build of binutils 2.17
and gcc-4.1.1 with the following runes:

After unpacking the sysroot and mkaing a link to it at
/usr/ia64-linux-gnu/sys-root:

  cd binutils-2.17/
  ./configure --with-sysroot=/usr/ia64-linux-gnu/sys-root --target=ia64-linux-gnu --prefix=/usr/local/ia64
  PATH=/usr/local/ia64/bin:$PATH make -j4
  PATH=/usr/local/ia64/bin:$PATH make install
  cd ..

  cd gcc-4.1.1/
  ./configure --with-sysroot=/usr/ia64-linux-gnu/sys-root --with-headers=/usr/ia64-linux-gnu/sys-root/usr/include --target=ia64-linux-gnu --prefix=/usr/local/ia64 --enable-languages=c
  PATH=/usr/local/ia64/bin:$PATH make -j4
  PATH=/usr/local/ia64/bin:$PATH make install

Ian.

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Ian Campbell]

On Mon, 2008-06-16 at 16:22 +0100, Ian Jackson wrote:
> 
> The kernel build stops asking for a kernel config option:
> 
>   mariner:xen-unstable-2.hg> PATH=/usr/local/ia64/bin:$PATH make
> XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu- -j4 dist-kernels
>   for i in  linux-2.6-xen  ; do make $i-install || exit 1; done
>   ...
>   *
>   Xen hypervisor support (XEN) [Y/n/?] y
>     Xen/IA64 exposure p2m table (XEN_IA64_EXPOSE_P2M) [Y/n/?] (NEW) 
> 
> Obviously that's no good for an automatic test system.

You can add XEN_LINUX_NONINTERACTIVE_CONFIG=y to the command line to
accept the default for any unknown options. I don't think our automated
test system uses that directly but it does essentially the same trick
since I think we see the same issues with the x86 kernel configs, at
least occasionally.

Ian.

Re: [Xen-devel] Re: cross compiling xen/ia64

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> You can add XEN_LINUX_NONINTERACTIVE_CONFIG=y to the command line

Ah, how useful.

Sadly, it still doesn't build:

  ...
    LD      .tmp_vmlinux1
  arch/ia64/kernel/built-in.o: In function `machine_kexec_setup_load_arg':
  (.text+0x44aa1): undefined reference to `mem_map'
  ...
  drivers/built-in.o:rd.c:(.text+0xdd4a1): more undefined references to `mem_map' follow
  make[5]: *** [.tmp_vmlinux1] Error 1
  make[4]: *** [vmlinuz] Error 2

Ian.

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Aron Griffis]

Hi Ian,

Ian Jackson wrote:  [Mon Jun 16 2008, 11:22:37AM EDT]
> I've been looking into this, and with Isaku Yamahata's patch the
> hypervisor does build.  I'm hoping we can get that wired into the
> staging propagation system very soon.  (Sorry about the delay getting
> round to this.)

This is great to hear.

> However, neither the kernel nor the tools build for me.

Could you provide the changeset info for the repos you're
building?  My most recent build was:
    
    staging/xen-unstable.hg-17867
    staging/linux-2.6.18-xen.hg-577

Full output at http://free.linux.hp.com/~agriffis/17867+577.log

Thanks,
Aron

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Ian Jackson]

Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> Could you provide the changeset info for the repos you're
> building?  My most recent build was:
>     
>     staging/xen-unstable.hg-17867
>     staging/linux-2.6.18-xen.hg-577

xen-unstable 17864:4bb8a4f66eff
linux-2.6.18-xen 417:f76e90b4f7ad

Err.  Now that I look at it that latter is obviously wrong - it's from
February.  I've pulled and am trying again.

Ian.

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Aron Griffis]

Ian Jackson wrote:  [Tue Jun 17 2008, 11:46:39AM EDT]
> Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> > Could you provide the changeset info for the repos you're
> > building?  My most recent build was:
> >     
> >     staging/xen-unstable.hg-17867
> >     staging/linux-2.6.18-xen.hg-577
> 
> xen-unstable 17864:4bb8a4f66eff
> linux-2.6.18-xen 417:f76e90b4f7ad
> 
> Err.  Now that I look at it that latter is obviously wrong - it's from
> February.  I've pulled and am trying again.

Any luck?

Re: [Xen-devel] Re: cross compiling xen/ia64

[Ian Jackson]

Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> Any luck?

drivers/built-in.o: In function `make_page_uptodate':
rd.c:(.text+0xdda50): undefined reference to `mem_map'
drivers/built-in.o:rd.c:(.text+0xdda61): more undefined references to `mem_map' follow
make[5]: *** [.tmp_vmlinux1] Error 1

etc., again.

Is CONFIG_NEED_MULTIPLE_NODES supposed to be enabled ?  AFAICT mem_map
and max_mapnr are defined in mm/memory.c but inside
#ifndef CONFIG_NEED_MULTIPLE_NODES

Ian.

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Aron Griffis]

Ian Jackson wrote:  [Wed Jun 18 2008, 05:09:45AM EDT]
> Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> > Any luck?
> 
> drivers/built-in.o: In function `make_page_uptodate':
> rd.c:(.text+0xdda50): undefined reference to `mem_map'
> drivers/built-in.o:rd.c:(.text+0xdda61): more undefined references to `mem_map' follow
> make[5]: *** [.tmp_vmlinux1] Error 1

I've never seen this particular error in my cross-builds, and
I've been building all the changesets constantly since March.

> Is CONFIG_NEED_MULTIPLE_NODES supposed to be enabled ?  AFAICT mem_map
> and max_mapnr are defined in mm/memory.c but inside
> #ifndef CONFIG_NEED_MULTIPLE_NODES

Rather than try to debug this, how about getting a known-good
combination building?  I'm concerned that these errors are due to
a problem in your building process rather than issues that need
to be debugged in the tree.

I just successfully built this combination:

    xen-unstable.hg-17879
    linux-2.6.18-xen.hg-577

If this combination fails for you, could you provide the full log
of the build for perusal?

The full log of my build, including the commands used, is here:

    http://free.linux.hp.com/~agriffis/17879+577.log

In particular, the make command:

    make XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu-

This is running in a Fedora8 chroot with the cross-compiler and
sysroot installed as described on the wiki.

Thanks,
Aron

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Ian Jackson]

Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> I've never seen this particular error in my cross-builds, and
> I've been building all the changesets constantly since March.
> 
> Ian Jackson wrote:  [Wed Jun 18 2008, 05:09:45AM EDT]
> > Is CONFIG_NEED_MULTIPLE_NODES supposed to be enabled ?  AFAICT mem_map
> > and max_mapnr are defined in mm/memory.c but inside
> > #ifndef CONFIG_NEED_MULTIPLE_NODES
> 
> Rather than try to debug this, how about getting a known-good
> combination building?  I'm concerned that these errors are due to
> a problem in your building process rather than issues that need
> to be debugged in the tree.

That may be the case but I didn't specify that myself.  The automatic
configuration system did it.

> I just successfully built this combination:
>     xen-unstable.hg-17879
>     linux-2.6.18-xen.hg-577
> If this combination fails for you, could you provide the full log
> of the build for perusal?

I'll do that.

> The full log of my build, including the commands used, is here:
>     http://free.linux.hp.com/~agriffis/17879+577.log
> In particular, the make command:
>     make XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu-

My command was nearly identical.

> This is running in a Fedora8 chroot with the cross-compiler and
> sysroot installed as described on the wiki.

I'm using Debian etch with the sysroot from the wiki and a specially
built binutils and cross-compiler.

Ian.

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Aron Griffis]

Ian Jackson wrote:  [Thu Jun 19 2008, 12:38:20PM EDT]
> I'm using Debian etch with the sysroot from the wiki and a specially
> built binutils and cross-compiler.

Hm, excuse for me to build up the same.  I'll let you know how
that goes. :-)

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Aron Griffis]

[-- Attachment #1: Type: text/plain, Size: 3320 bytes --]

Hi Ian,

Sorry this took so long.  Xen Summit got in the way of work. ;-)

I've managed to get the cross-building working in an etch chroot.
I think the main difference between your build and mine is that
my build is running in a cron job, so stdin is /dev/null.  That
allows the kernel config to go through without prompting
(accepting defaults, which isn't necessarily ideal, but at least
it allows the automated build to progress).

If it's helpful, here are most of the steps I did after creating
the chroot with debootstrap, bind-mounting some dirs into it, and
installing sudo...

1. Move aside invoke-rc.d so that daemons aren't started in the
   chroot.  Using dpkg-divert makes it stick for future updates.
   See http://www.debian-administration.org/articles/118

   $ sudo dpkg-divert --add --rename --divert /usr/sbin/invoke-rc.d{.real,} 
   $ sudo ln -s /bin/true /usr/sbin/invoke-rc.d

2. Install most of the stuff we need for building the
   cross-compiler and xen.  The underscore on mawk_ causes it to
   be uninstalled so gawk can take its place.

   $ sudo aptitude install mawk_ gawk mercurial dpkg-cross fakeroot 
   $ sudo apt-get build-dep binutils gcc-4.1

3. Install the pre-built cross toolchain from the emdebian
   project.  Unfortunately the emdebian-provided toolchain
   doesn't use the sysroot, but installing it makes things easier
   since it fulfills some cross-deps...

   $ echo 'deb http://www.emdebian.org/debian/ stable main' | \
      sudo tee -a /etc/apt/sources.list
   $ wget -O- http://www.emdebian.org/0x97BB3B58.txt | sudo apt-key add -
   $ sudo aptitude update
   $ sudo aptitude install gcc-4.1-ia64-linux-gnu

4. Prepare to rebuild the toolchain.

   $ echo 'deb-src ftp://ftp.us.debian.org/debian/ stable main' | \
      sudo tee -a /etc/apt/sources.list
   $ sudo aptitude update
   $ mkdir cross-debs
   $ cd cross-debs

5. Rebuild binutils to use the sysroot.

   $ apt-get source binutils
   $ cd binutils-2.17/
   $ patch -p0 < binutils-debian-sysroot.patch  # see attachment
   $ TARGET=ia64 fakeroot debian/rules binary-cross  # see debian/README.cross
   $ cd ..
   $ sudo dpkg -i binutils-ia64-linux-gnu_2.17-3_amd64.deb

6. Unpack the chroot since it's required to build gcc with
   sysroot capability.

   $ wget http://free.linux.hp.com/~agriffis/cross/sys-roots/etch-ia64.tar.gz
   $ sudo tar xzf etch-ia64.tar.gz -C /usr/ia64-linux-gnu
   $ sudo ln -s etch-root /usr/ia64-linux-gnu/sys-root

7. Install another requirement for rebuilding gcc.

   $ wget ftp://ftp.us.debian.org/debian/pool/main/liba/libatomic-ops/libatomic-ops-dev_1.1-4_ia64.deb
   $ sudo dpkg-cross -i -a ia64 libatomic-ops-dev_1.1-4_ia64.deb 

8. Rebuild gcc-4.1 to use the sysroot.

   $ apt-get source gcc-4.1
   $ cd gcc-4.1_4.1.1ds2
   $ export GCC_TARGET=ia64 DEB_CROSS_INDEPENDENT=yes with_sysroot=yes
   $ debian/rules  # see debian/README.cross
   $ dpkg-buildpackage -rfakeroot
   $ cd ..
   $ sudo dpkg -i *amd64.deb *all.deb

9. Build xen.

   $ cd
   $ hg clone http://xenbits.xen.org/xen-unstable.hg
   $ hg clone http://xenbits.xen.org/linux-2.6.18-xen.hg
   $ cd xen-unstable.hg
   $ make XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu- world </dev/null

10. Profit!!!

I'll try to update the wiki to include both Fedora-based and
Debian-based setups.

Thanks,
Aron

[-- Attachment #2: binutils-debian-sysroot.patch --]
[-- Type: text/x-diff, Size: 364 bytes --]

--- debian/rules.sysroot	2008-06-27 17:27:46.000000000 +0000
+++ debian/rules	2008-06-27 17:27:54.000000000 +0000
@@ -501,6 +501,7 @@
 	cd builddir-$(TARGET) \
 	    && env CC="$(CC)" ../configure --host=$(DEB_HOST_GNU_TYPE) \
 	        --build=$(DEB_BUILD_GNU_TYPE) --target=$(TARGET) --prefix=/usr \
+		--with-sysroot=yes \
 		$(ADDITIONAL_TARGETS)
 	touch $@
 

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [Xen-devel] Re: cross compiling xen/ia64

[Aron Griffis]

Ian,

I've updated the wiki now with Debian Etch instructions based on
my previous email.  Let me know if you continue to have trouble
with the build.

http://wiki.xensource.com/xenwiki/CrossCompiling

Thanks,
Aron

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Ian Jackson]

Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> I've updated the wiki now with Debian Etch instructions based on
> my previous email.  Let me know if you continue to have trouble
> with the build.

Thanks.  I did try this again while I was at the Xen Summit and it
seemed to work for me, but now I'm a bit distracted with the feature
freeze looming.

I'll revisit this next week and hope to get better coverage for the
ia64 patchman compiles.

Ian.

Re: Re: [Xen-ia64-devel] cross compiling xen/ia64

[Aron Griffis]

Ian Jackson wrote:  [Wed Jul 02 2008, 06:25:23AM EDT]
> Aron Griffis writes ("Re: [Xen-devel] Re: [Xen-ia64-devel] cross compiling xen/ia64"):
> > I've updated the wiki now with Debian Etch instructions based on
> > my previous email.  Let me know if you continue to have trouble
> > with the build.
> 
> Thanks.  I did try this again while I was at the Xen Summit and it
> seemed to work for me, but now I'm a bit distracted with the feature
> freeze looming.
> 
> I'll revisit this next week and hope to get better coverage for the
> ia64 patchman compiles.

Hi Ian,

FYI the staging tree tools build has been broken on ia64 since
cset 9cf72db44ee9 which references PAGE_SIZE, and this has now
filtered down to the unstable tree.

Hope you can get the cross-builder in the ring soon.  This is the
kind of breakage it should help hold in staging rather than
letting it slip into unstable.

Thanks,
Aron