Re: Re: [Xen-staging] [xen-unstable] ioemu: Use XEN_TARGET_ARCH instead of uname -m.

[Aron Griffis <aron@hp.com>]

Keir Fraser wrote:  [Sun Jan 27 2008, 05:21:06PM EST]
> I'm not sure what particular scenario Bastian was intending to fix, but the
> patch does appear to make sense. qemu-dm is going to run in the target
> system's dom0. So the 'host CPU' type for qemu-dm is related to
> XEN_TARGET_ARCH, not XEN_COMPILE_ARCH nor 'uname -m'. This is a bit
> confusing because for qemu there are three environments to consider: build,
> host and emulated. I do believe that host CPU need not == build CPU.

Thanks Keir.  It doesn't work for me at the moment (including the ia64
follow-on patch) because it uses a custom ld script, tools/ioemu/ia64.ld

Here is the output when using that script.  You can see pretty easily
that it's ignoring the sys-root.  Jump down for a patch that fixes
this particular issue...

------------------------------------------------------------
Current output
------------------------------------------------------------
$ make -j1 XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu- ioemu
[ -f ioemu/config-host.mak ] || \
          ( cd ioemu && XEN_TARGET_ARCH=ia64 sh configure --prefix=/usr \
                --cross-prefix=ia64-linux-gnu- --interp-prefix=/usr/ia64-linux-gnu/sys-root )
make -C ioemu 
make[1]: Entering directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu'
make -C i386-dm all
*** PCI passthrough capability has been enabled ***
make[2]: Entering directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu/i386-dm'
ia64-linux-gnu-gcc  -Wl,-G0 -Wl,-T,/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu/ia64.ld -o qemu-dm vl.o osdep.o readline.o monitor.o pci.o console.o isa_mmio.o cutils.o block.o block-raw.o block-cow.o block-qcow.o aes.o block-vmdk.o block-cloop.o block-dmg.o block-bochs.o block-vpc.o block-vvfat.o block-qcow2.o pass-through.o scsi-disk.o cdrom.o lsi53c895a.o usb.o usb-hub.o usb-linux.o usb-hid.o usb-ohci.o usb-msd.o ne2000.o rtl8139.o pcnet.o e100.o ide.o pckbd.o ps2.o vga.o sb16.o es1370.o dma.o audio.o noaudio.o wavaudio.o ossaudio.o wavcapture.o extboot.o fdc.o mc146818rtc.o serial.o pc.o cirrus_vga.o mixeng.o parallel.o acpi.o usb-uhci.o smbus_eeprom.o piix4acpi.o xenstore.o xen_platform.o xen_machine_fv.o xen_machine_pv.o xenfb.o xen_console.o tpm_tis.o vnc.o d3d
 es.o libqemu.a  -lm -L../../libxc -lxenctrl -lxenguest -L../../xenstore -lxenstore -lz -lpci -lgnutls     -lutil -lrt
/usr/lib/gcc/ia64-linux-gnu/4.1.2/../../../../ia64-linux-gnu/bin/ld: warning: libpthread.so.0, needed by ../../libxc/libxenctrl.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/ia64-linux-gnu/4.1.2/../../../../ia64-linux-gnu/bin/ld: warning: libtasn1.so.3, needed by /usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/ia64-linux-gnu/4.1.2/../../../../ia64-linux-gnu/bin/ld: warning: libgcrypt.so.11, needed by /usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/ia64-linux-gnu/4.1.2/../../../../ia64-linux-gnu/bin/ld: warning: libgpg-error.so.0, needed by /usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/ia64-linux-gnu/4.1.2/../../../../ia64-linux-gnu/bin/ld: warning: ld-linux-ia64.so.2, needed by /usr/ia64-linux-gnu/sys-root/usr/lib/librt.so, not found (try using -rpath or -rpath-link)
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_array2tree@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_copy_node@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_setkey@GCRYPT_1.2'
../../libxc/libxenctrl.so: undefined reference to `pthread_getspecific@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_new@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_mod@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_sub_ui@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_release@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_mulm@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_get_algo_dlen@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_invm@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `_pthread_cleanup_pop@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_cipher_decrypt@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_cipher_open@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_number_of_elements@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_snew@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_add@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_prime_release_factors@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_pk_encrypt@GCRYPT_1.2'
../../libxc/libxenctrl.so: undefined reference to `pthread_once@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_cipher_ctl@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_copy@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_set_ui@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_dl_unmap@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_copy@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_check_version@GCRYPT_1.2'
../../libxc/libxenctrl.so: undefined reference to `pthread_key_create@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_dl_argv@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_subm@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_write@GCRYPT_1.2'
../../xenstore/libxenstore.so: undefined reference to `pthread_create@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_control@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_div@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_rtld_global_ro@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_prime_check@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_randomize@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_cmp@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `pthread_barrier_init@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_pk_sign@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_cipher_close@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `__tls_get_addr@GLIBC_2.3'
../../libxc/libxenctrl.so: undefined reference to `pthread_setspecific@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_scan@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_print@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_prime_group_generator@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_dl_lookup_address@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_randomize@GCRYPT_1.2'
../../xenstore/libxenstore.so: undefined reference to `pthread_cancel@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_write_value@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_hash_buffer@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_get_nbits@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_add_ui@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_create_element@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_pk_verify@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `pthread_attr_setstacksize@GLIBC_2.3.3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_der_decoding_startEnd@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_create_nonce@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_sexp_find_token@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_sexp_nth_mpi@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_read_value@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_addm@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_der_coding@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_pk_decrypt@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_read@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_dl_out_of_memory@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_pk_genkey@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_sexp_build@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_dl_symbol_address@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_powm@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_clear_highbit@GCRYPT_1.2'
../../xenstore/libxenstore.so: undefined reference to `pthread_join@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_ctl@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `_r_debug@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_sexp_release@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_open@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `_rtld_global@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_set_allocation_handler@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_prime_generate@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_der_decoding@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `pthread_detach@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_check_version@LIBTASN1_0_3'
../../xenstore/libxenstore.so: undefined reference to `__isoc99_sscanf@GLIBC_2.7'
/usr/ia64-linux-gnu/sys-root/lib/libc.so.6.1: undefined reference to `__libc_enable_secure@GLIBC_PRIVATE'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `pthread_sigmask@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_find_structure_from_oid@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `asn1_delete_structure@LIBTASN1_0_3'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_mul@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_md_close@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_mpi_cmp_ui@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/libgnutls.so: undefined reference to `gcry_cipher_encrypt@GCRYPT_1.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `pthread_barrier_wait@GLIBC_2.2'
/usr/ia64-linux-gnu/sys-root/usr/lib/librt.so: undefined reference to `_pthread_cleanup_push@GLIBC_2.2'
collect2: ld returned 1 exit status
make[2]: *** [qemu-dm] Error 1
make[2]: Leaving directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu/i386-dm'
make[1]: *** [subdir-i386-dm] Error 2
make[1]: Leaving directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu'
make: *** [ioemu] Error 2
------------------------------------------------------------

Here's a small patch which reduces this output significantly by fixing
the sys-root problem.  It seems that the '=' tells ld to use the
sys-root, if it was built with --with-sysroot:

diff -r 32d4dda41994 tools/ioemu/ia64.ld
--- a/tools/ioemu/ia64.ld	Sun Jan 27 16:13:41 2008 -0500
+++ b/tools/ioemu/ia64.ld	Sun Jan 27 18:02:54 2008 -0500
@@ -3,7 +3,7 @@ OUTPUT_FORMAT("elf64-ia64-little", "elf6
 	      "elf64-ia64-little")
 OUTPUT_ARCH(ia64)
 ENTRY(_start)
-SEARCH_DIR("/usr/ia64-linux/lib"); SEARCH_DIR("/usr/local/lib"); SEARCH_DIR("/lib"); SEARCH_DIR("/usr/lib");
+SEARCH_DIR("=/usr/ia64-linux/lib"); SEARCH_DIR("=/usr/local/lib"); SEARCH_DIR("=/lib"); SEARCH_DIR("=/usr/lib");
 /* Do we need any of these for elf?
    __DYNAMIC = 0;    */
 SECTIONS

Signed-off-by: Aron Griffis <aron@hp.com>

However even with this patch, the build still fails.  I guess I need
to understand the point of these custom ld-scripts...  With it
commented out in Makefile.target, the build works fine!

------------------------------------------------------------
Output after patch
------------------------------------------------------------
$ make -j1 XEN_TARGET_ARCH=ia64 CROSS_COMPILE=ia64-linux-gnu- ioemu
[ -f ioemu/config-host.mak ] || \
          ( cd ioemu && XEN_TARGET_ARCH=ia64 sh configure --prefix=/usr \
                --cross-prefix=ia64-linux-gnu- --interp-prefix=/usr/ia64-linux-gnu/sys-root )
make -C ioemu 
make[1]: Entering directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu'
make -C i386-dm all
*** PCI passthrough capability has been enabled ***
make[2]: Entering directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu/i386-dm'
ia64-linux-gnu-gcc  -Wl,-G0 -Wl,-T,/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu/ia64.ld -o qemu-dm vl.o osdep.o readline.o monitor.o pci.o console.o isa_mmio.o cutils.o block.o block-raw.o block-cow.o block-qcow.o aes.o block-vmdk.o block-cloop.o block-dmg.o block-bochs.o block-vpc.o block-vvfat.o block-qcow2.o pass-through.o scsi-disk.o cdrom.o lsi53c895a.o usb.o usb-hub.o usb-linux.o usb-hid.o usb-ohci.o usb-msd.o ne2000.o rtl8139.o pcnet.o e100.o ide.o pckbd.o ps2.o vga.o sb16.o es1370.o dma.o audio.o noaudio.o wavaudio.o ossaudio.o wavcapture.o extboot.o fdc.o mc146818rtc.o serial.o pc.o cirrus_vga.o mixeng.o parallel.o acpi.o usb-uhci.o smbus_eeprom.o piix4acpi.o xenstore.o xen_platform.o xen_machine_fv.o xen_machine_pv.o xenfb.o xen_console.o tpm_tis.o vnc.o d3d
 es.o libqemu.a  -lm -L../../libxc -lxenctrl -lxenguest -L../../xenstore -lxenstore -lz -lpci -lgnutls     -lutil -lrt
../../xenstore/libxenstore.so: undefined reference to `__isoc99_sscanf@GLIBC_2.7'
collect2: ld returned 1 exit status
make[2]: *** [qemu-dm] Error 1
make[2]: Leaving directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu/i386-dm'
make[1]: *** [subdir-i386-dm] Error 2
make[1]: Leaving directory `/home/agriffis/work/xen/20080102-cross/xen-unstable.hg/tools/ioemu'
make: *** [ioemu] Error 2
------------------------------------------------------------

Thanks,
Aron