[Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

The following changes since commit f6e3035f75e5c6a73485335765ae070304c7a110:

  Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream-smm' into staging (2015-07-06 23:37:53 +0100)

are available in the git repository at:

  git://github.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 2828a307232ffceeddec9feb6a87ac660b68b693:

  ossaudio: fix memory leak (2015-07-08 13:11:01 +0200)

----------------------------------------------------------------
Bugfixes and Daniel Berrange's crypto library.

----------------------------------------------------------------
Daniel P. Berrange (10):
      crypto: introduce new module for computing hash digests
      crypto: move built-in AES implementation into crypto/
      crypto: move built-in D3DES implementation into crypto/
      crypto: introduce generic cipher API & built-in implementation
      crypto: add a gcrypt cipher implementation
      crypto: add a nettle cipher implementation
      block: convert quorum blockdrv to use crypto APIs
      ui: convert VNC websockets to use crypto APIs
      block: convert qcow/qcow2 to use generic cipher API
      ui: convert VNC to use generic cipher API

Eric Auger (1):
      vl: move rom_load_all after machine init done

Gonglei (1):
      ossaudio: fix memory leak

 MAINTAINERS                           |   7 +
 Makefile.objs                         |   1 +
 audio/ossaudio.c                      |   1 +
 block/Makefile.objs                   |   2 +-
 block/qcow.c                          | 102 ++++++---
 block/qcow2-cluster.c                 |  46 +++-
 block/qcow2.c                         |  96 ++++----
 block/qcow2.h                         |  13 +-
 block/quorum.c                        |  39 ++--
 configure                             | 160 +++++++++-----
 crypto/Makefile.objs                  |   5 +
 {util => crypto}/aes.c                |   2 +-
 crypto/cipher-builtin.c               | 398 ++++++++++++++++++++++++++++++++++
 crypto/cipher-gcrypt.c                | 195 +++++++++++++++++
 crypto/cipher-nettle.c                | 206 ++++++++++++++++++
 crypto/cipher.c                       |  74 +++++++
 ui/d3des.c => crypto/desrfb.c         |   2 +-
 crypto/hash.c                         | 200 +++++++++++++++++
 crypto/init.c                         | 150 +++++++++++++
 hw/core/loader.c                      |   8 +-
 include/{qemu => crypto}/aes.h        |   0
 include/crypto/cipher.h               | 210 ++++++++++++++++++
 ui/d3des.h => include/crypto/desrfb.h |   0
 include/crypto/hash.h                 | 189 ++++++++++++++++
 include/crypto/init.h                 |  29 +++
 include/hw/loader.h                   |   3 +-
 target-arm/crypto_helper.c            |   2 +-
 target-i386/fpu_helper.c              |   1 -
 target-i386/ops_sse.h                 |   2 +-
 target-ppc/int_helper.c               |   2 +-
 tests/.gitignore                      |   2 +
 tests/Makefile                        |   4 +
 tests/test-crypto-cipher.c            | 290 +++++++++++++++++++++++++
 tests/test-crypto-hash.c              | 209 ++++++++++++++++++
 ui/Makefile.objs                      |   4 +-
 ui/vnc-ws.c                           |  22 +-
 ui/vnc-ws.h                           |   2 -
 ui/vnc.c                              | 119 +++++-----
 ui/vnc.h                              |   8 -
 util/Makefile.objs                    |   2 +-
 vl.c                                  |  18 +-
 41 files changed, 2546 insertions(+), 279 deletions(-)
 create mode 100644 crypto/Makefile.objs
 rename {util => crypto}/aes.c (99%)
 create mode 100644 crypto/cipher-builtin.c
 create mode 100644 crypto/cipher-gcrypt.c
 create mode 100644 crypto/cipher-nettle.c
 create mode 100644 crypto/cipher.c
 rename ui/d3des.c => crypto/desrfb.c (99%)
 create mode 100644 crypto/hash.c
 create mode 100644 crypto/init.c
 rename include/{qemu => crypto}/aes.h (100%)
 create mode 100644 include/crypto/cipher.h
 rename ui/d3des.h => include/crypto/desrfb.h (100%)
 create mode 100644 include/crypto/hash.h
 create mode 100644 include/crypto/init.h
 create mode 100644 tests/test-crypto-cipher.c
 create mode 100644 tests/test-crypto-hash.c
-- 
2.4.3

diff --git a/block/quorum.c b/block/quorum.c
index 5938864..4e66221 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -24,8 +24,6 @@
 #include "qapi-event.h"
 #include "crypto/hash.h"
 
-#include <glib/gi18n.h>
-
 #define HASH_LENGTH 32
 
 #define QUORUM_OPT_VOTE_THRESHOLD "vote-threshold"
diff --git a/crypto/cipher.c b/crypto/cipher.c
index 663bbe3..024a00c 100644
--- a/crypto/cipher.c
+++ b/crypto/cipher.c
@@ -33,8 +33,7 @@ qcrypto_cipher_validate_key_length(QCryptoCipherAlgorithm alg,
                                    size_t nkey,
                                    Error **errp)
 {
-    if (alg < 0 ||
-        alg >= QCRYPTO_CIPHER_ALG_LAST) {
+    if ((unsigned)alg >= QCRYPTO_CIPHER_ALG_LAST) {
         error_setg(errp, "Cipher algorithm %d out of range",
                    alg);
         return false;

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 8 July 2015 at 12:19, Paolo Bonzini <pbonzini@redhat.com> wrote:
> The following changes since commit f6e3035f75e5c6a73485335765ae070304c7a110:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream-smm' into staging (2015-07-06 23:37:53 +0100)
>
> are available in the git repository at:
>
>   git://github.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 2828a307232ffceeddec9feb6a87ac660b68b693:
>
>   ossaudio: fix memory leak (2015-07-08 13:11:01 +0200)
>
> ----------------------------------------------------------------
> Bugfixes and Daniel Berrange's crypto library.

I'm afraid this fails to link on my linux-static build:

c++ -I/usr/include/pixman-1    -Werror -m64 -D_GNU_SOURCE
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
-Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes
-fno-strict-aliasing -fno-common  -Wendif-labels
-Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
-Wold-style-definition -Wtype-limits -fstack-protector-all
-I/usr/include/p11-kit-1     -I/usr/include/libpng12
-I/usr/include/libusb-1.0
-I/home/petmay01/linaro/qemu-for-merges/tests -pthread
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
-g  -Wl,--warn-common -m64 -static -g  -o qemu-nbd qemu-nbd.o async.o
thread-pool.o nbd.o block.o blockjob.o main-loop.o iohandler.o
qemu-timer.o aio-posix.o qemu-io-cmds.o qemu-coroutine.o
qemu-coroutine-lock.o qemu-coroutine-io.o qemu-coroutine-sleep.o
coroutine-ucontext.o block/raw_bsd.o block/qcow.o block/vdi.o
block/vmdk.o block/cloop.o block/bochs.o block/vpc.o block/vvfat.o
block/qcow2.o block/qcow2-refcount.o block/qcow2-cluster.o
block/qcow2-snapshot.o block/qcow2-cache.o block/qed.o
block/qed-gencb.o block/qed-l2-cache.o block/qed-table.o
block/qed-cluster.o block/qed-check.o block/vhdx.o block/vhdx-endian.o
block/vhdx-log.o block/quorum.o block/parallels.o block/blkdebug.o
block/blkverify.o block/block-backend.o block/snapshot.o block/qapi.o
block/raw-posix.o block/linux-aio.o block/null.o block/mirror.o
block/io.o block/throttle-groups.o block/nbd.o block/nbd-client.o
block/sheepdog.o block/accounting.o block/write-threshold.o
block/dmg.o  libqemuutil.a libqemustub.a   -lz -laio -lm -pthread
-lgthread-2.0 -lglib-2.0 -lpcre    -lz -lrt -lz -lcap-ng -lvdeplug
-luuid -lgcrypt -R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt
-lgpg-error -ltasn1 -lz -lp11-kit    -lutil
c++: error: unrecognized command line option ‘-R’

(That's configure arguments  '--cc=ccache gcc' '--enable-debug'
'--static' '--disable-system', on an x86-64 Linux host.)

-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 08/07/2015 16:46, Peter Maydell wrote:
> I'm afraid this fails to link on my linux-static build:
> 
> c++ -I/usr/include/pixman-1    -Werror -m64 -D_GNU_SOURCE
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes
> -fno-strict-aliasing -fno-common  -Wendif-labels
> -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security
> -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
> -Wold-style-definition -Wtype-limits -fstack-protector-all
> -I/usr/include/p11-kit-1     -I/usr/include/libpng12
> -I/usr/include/libusb-1.0
> -I/home/petmay01/linaro/qemu-for-merges/tests -pthread
> -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
> -g  -Wl,--warn-common -m64 -static -g  -o qemu-nbd qemu-nbd.o async.o
> thread-pool.o nbd.o block.o blockjob.o main-loop.o iohandler.o
> qemu-timer.o aio-posix.o qemu-io-cmds.o qemu-coroutine.o
> qemu-coroutine-lock.o qemu-coroutine-io.o qemu-coroutine-sleep.o
> coroutine-ucontext.o block/raw_bsd.o block/qcow.o block/vdi.o
> block/vmdk.o block/cloop.o block/bochs.o block/vpc.o block/vvfat.o
> block/qcow2.o block/qcow2-refcount.o block/qcow2-cluster.o
> block/qcow2-snapshot.o block/qcow2-cache.o block/qed.o
> block/qed-gencb.o block/qed-l2-cache.o block/qed-table.o
> block/qed-cluster.o block/qed-check.o block/vhdx.o block/vhdx-endian.o
> block/vhdx-log.o block/quorum.o block/parallels.o block/blkdebug.o
> block/blkverify.o block/block-backend.o block/snapshot.o block/qapi.o
> block/raw-posix.o block/linux-aio.o block/null.o block/mirror.o
> block/io.o block/throttle-groups.o block/nbd.o block/nbd-client.o
> block/sheepdog.o block/accounting.o block/write-threshold.o
> block/dmg.o  libqemuutil.a libqemustub.a   -lz -laio -lm -pthread
> -lgthread-2.0 -lglib-2.0 -lpcre    -lz -lrt -lz -lcap-ng -lvdeplug
> -luuid -lgcrypt -R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt
> -lgpg-error -ltasn1 -lz -lp11-kit    -lutil
> c++: error: unrecognized command line option ‘-R’
> 
> (That's configure arguments  '--cc=ccache gcc' '--enable-debug'
> '--static' '--disable-system', on an x86-64 Linux host.)

You don't have a static gnutls installed, you need to add either
--disable-gnutls or --disable-tools.

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 8 July 2015 at 16:21, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 08/07/2015 16:46, Peter Maydell wrote:
>> I'm afraid this fails to link on my linux-static build:
>>
>> c++ -I/usr/include/pixman-1    -Werror -m64 -D_GNU_SOURCE
>> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
>> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes
>> -fno-strict-aliasing -fno-common  -Wendif-labels
>> -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security
>> -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
>> -Wold-style-definition -Wtype-limits -fstack-protector-all
>> -I/usr/include/p11-kit-1     -I/usr/include/libpng12
>> -I/usr/include/libusb-1.0
>> -I/home/petmay01/linaro/qemu-for-merges/tests -pthread
>> -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
>> -g  -Wl,--warn-common -m64 -static -g  -o qemu-nbd qemu-nbd.o async.o
>> thread-pool.o nbd.o block.o blockjob.o main-loop.o iohandler.o
>> qemu-timer.o aio-posix.o qemu-io-cmds.o qemu-coroutine.o
>> qemu-coroutine-lock.o qemu-coroutine-io.o qemu-coroutine-sleep.o
>> coroutine-ucontext.o block/raw_bsd.o block/qcow.o block/vdi.o
>> block/vmdk.o block/cloop.o block/bochs.o block/vpc.o block/vvfat.o
>> block/qcow2.o block/qcow2-refcount.o block/qcow2-cluster.o
>> block/qcow2-snapshot.o block/qcow2-cache.o block/qed.o
>> block/qed-gencb.o block/qed-l2-cache.o block/qed-table.o
>> block/qed-cluster.o block/qed-check.o block/vhdx.o block/vhdx-endian.o
>> block/vhdx-log.o block/quorum.o block/parallels.o block/blkdebug.o
>> block/blkverify.o block/block-backend.o block/snapshot.o block/qapi.o
>> block/raw-posix.o block/linux-aio.o block/null.o block/mirror.o
>> block/io.o block/throttle-groups.o block/nbd.o block/nbd-client.o
>> block/sheepdog.o block/accounting.o block/write-threshold.o
>> block/dmg.o  libqemuutil.a libqemustub.a   -lz -laio -lm -pthread
>> -lgthread-2.0 -lglib-2.0 -lpcre    -lz -lrt -lz -lcap-ng -lvdeplug
>> -luuid -lgcrypt -R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt
>> -lgpg-error -ltasn1 -lz -lp11-kit    -lutil
>> c++: error: unrecognized command line option ‘-R’
>>
>> (That's configure arguments  '--cc=ccache gcc' '--enable-debug'
>> '--static' '--disable-system', on an x86-64 Linux host.)
>
> You don't have a static gnutls installed, you need to add either
> --disable-gnutls or --disable-tools.

Shouldn't configure notice this and disable tls ?

-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 08/07/2015 17:48, Peter Maydell wrote:
> On 8 July 2015 at 16:21, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>>
>> On 08/07/2015 16:46, Peter Maydell wrote:
>>> I'm afraid this fails to link on my linux-static build:
>>>
>>> c++ -I/usr/include/pixman-1    -Werror -m64 -D_GNU_SOURCE
>>> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
>>> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes
>>> -fno-strict-aliasing -fno-common  -Wendif-labels
>>> -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security
>>> -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
>>> -Wold-style-definition -Wtype-limits -fstack-protector-all
>>> -I/usr/include/p11-kit-1     -I/usr/include/libpng12
>>> -I/usr/include/libusb-1.0
>>> -I/home/petmay01/linaro/qemu-for-merges/tests -pthread
>>> -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
>>> -g  -Wl,--warn-common -m64 -static -g  -o qemu-nbd qemu-nbd.o async.o
>>> thread-pool.o nbd.o block.o blockjob.o main-loop.o iohandler.o
>>> qemu-timer.o aio-posix.o qemu-io-cmds.o qemu-coroutine.o
>>> qemu-coroutine-lock.o qemu-coroutine-io.o qemu-coroutine-sleep.o
>>> coroutine-ucontext.o block/raw_bsd.o block/qcow.o block/vdi.o
>>> block/vmdk.o block/cloop.o block/bochs.o block/vpc.o block/vvfat.o
>>> block/qcow2.o block/qcow2-refcount.o block/qcow2-cluster.o
>>> block/qcow2-snapshot.o block/qcow2-cache.o block/qed.o
>>> block/qed-gencb.o block/qed-l2-cache.o block/qed-table.o
>>> block/qed-cluster.o block/qed-check.o block/vhdx.o block/vhdx-endian.o
>>> block/vhdx-log.o block/quorum.o block/parallels.o block/blkdebug.o
>>> block/blkverify.o block/block-backend.o block/snapshot.o block/qapi.o
>>> block/raw-posix.o block/linux-aio.o block/null.o block/mirror.o
>>> block/io.o block/throttle-groups.o block/nbd.o block/nbd-client.o
>>> block/sheepdog.o block/accounting.o block/write-threshold.o
>>> block/dmg.o  libqemuutil.a libqemustub.a   -lz -laio -lm -pthread
>>> -lgthread-2.0 -lglib-2.0 -lpcre    -lz -lrt -lz -lcap-ng -lvdeplug
>>> -luuid -lgcrypt -R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt
>>> -lgpg-error -ltasn1 -lz -lp11-kit    -lutil
>>> c++: error: unrecognized command line option ‘-R’
>>>
>>> (That's configure arguments  '--cc=ccache gcc' '--enable-debug'
>>> '--static' '--disable-system', on an x86-64 Linux host.)
>>
>> You don't have a static gnutls installed, you need to add either
>> --disable-gnutls or --disable-tools.
> 
> Shouldn't configure notice this and disable tls ?

Packaging bugs around static libraries aren't rare, and it's difficult
to say who's guilty without a before/after config.log and build log.  Is
it okay for you to pull this and fix it during the -rc period?

Also, what distro do you do this static build on?

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 8 July 2015 at 18:06, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 08/07/2015 17:48, Peter Maydell wrote:
>>> You don't have a static gnutls installed, you need to add either
>>> --disable-gnutls or --disable-tools.
>>
>> Shouldn't configure notice this and disable tls ?
>
> Packaging bugs around static libraries aren't rare, and it's difficult
> to say who's guilty without a before/after config.log and build log.  Is
> it okay for you to pull this and fix it during the -rc period?

I guess that's probably the lesser of the two evils.

> Also, what distro do you do this static build on?

Ubuntu trusty.

-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 8 July 2015 at 12:19, Paolo Bonzini <pbonzini@redhat.com> wrote:
> The following changes since commit f6e3035f75e5c6a73485335765ae070304c7a110:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream-smm' into staging (2015-07-06 23:37:53 +0100)
>
> are available in the git repository at:
>
>   git://github.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 2828a307232ffceeddec9feb6a87ac660b68b693:
>
>   ossaudio: fix memory leak (2015-07-08 13:11:01 +0200)
>
> ----------------------------------------------------------------
> Bugfixes and Daniel Berrange's crypto library.
>

Applied, thanks.

-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 08/07/2015 22:48, Peter Maydell wrote:
> On 8 July 2015 at 12:19, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> The following changes since commit f6e3035f75e5c6a73485335765ae070304c7a110:
>>
>>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream-smm' into staging (2015-07-06 23:37:53 +0100)
>>
>> are available in the git repository at:
>>
>>   git://github.com/bonzini/qemu.git tags/for-upstream
>>
>> for you to fetch changes up to 2828a307232ffceeddec9feb6a87ac660b68b693:
>>
>>   ossaudio: fix memory leak (2015-07-08 13:11:01 +0200)
>>
>> ----------------------------------------------------------------
>> Bugfixes and Daniel Berrange's crypto library.
>>
> 
> Applied, thanks.

Thank you very much!

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Daniel P. Berrange]

On Wed, Jul 08, 2015 at 04:48:27PM +0100, Peter Maydell wrote:
> On 8 July 2015 at 16:21, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> >
> > On 08/07/2015 16:46, Peter Maydell wrote:
> >> I'm afraid this fails to link on my linux-static build:
> >>
> >> c++ -I/usr/include/pixman-1    -Werror -m64 -D_GNU_SOURCE
> >> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
> >> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes
> >> -fno-strict-aliasing -fno-common  -Wendif-labels
> >> -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security
> >> -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration
> >> -Wold-style-definition -Wtype-limits -fstack-protector-all
> >> -I/usr/include/p11-kit-1     -I/usr/include/libpng12
> >> -I/usr/include/libusb-1.0
> >> -I/home/petmay01/linaro/qemu-for-merges/tests -pthread
> >> -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
> >> -g  -Wl,--warn-common -m64 -static -g  -o qemu-nbd qemu-nbd.o async.o
> >> thread-pool.o nbd.o block.o blockjob.o main-loop.o iohandler.o
> >> qemu-timer.o aio-posix.o qemu-io-cmds.o qemu-coroutine.o
> >> qemu-coroutine-lock.o qemu-coroutine-io.o qemu-coroutine-sleep.o
> >> coroutine-ucontext.o block/raw_bsd.o block/qcow.o block/vdi.o
> >> block/vmdk.o block/cloop.o block/bochs.o block/vpc.o block/vvfat.o
> >> block/qcow2.o block/qcow2-refcount.o block/qcow2-cluster.o
> >> block/qcow2-snapshot.o block/qcow2-cache.o block/qed.o
> >> block/qed-gencb.o block/qed-l2-cache.o block/qed-table.o
> >> block/qed-cluster.o block/qed-check.o block/vhdx.o block/vhdx-endian.o
> >> block/vhdx-log.o block/quorum.o block/parallels.o block/blkdebug.o
> >> block/blkverify.o block/block-backend.o block/snapshot.o block/qapi.o
> >> block/raw-posix.o block/linux-aio.o block/null.o block/mirror.o
> >> block/io.o block/throttle-groups.o block/nbd.o block/nbd-client.o
> >> block/sheepdog.o block/accounting.o block/write-threshold.o
> >> block/dmg.o  libqemuutil.a libqemustub.a   -lz -laio -lm -pthread
> >> -lgthread-2.0 -lglib-2.0 -lpcre    -lz -lrt -lz -lcap-ng -lvdeplug
> >> -luuid -lgcrypt -R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt
> >> -lgpg-error -ltasn1 -lz -lp11-kit    -lutil
> >> c++: error: unrecognized command line option ‘-R’
> >>
> >> (That's configure arguments  '--cc=ccache gcc' '--enable-debug'
> >> '--static' '--disable-system', on an x86-64 Linux host.)
> >
> > You don't have a static gnutls installed, you need to add either
> > --disable-gnutls or --disable-tools.
> 
> Shouldn't configure notice this and disable tls ?

Ok, I can reproduce this easily enough with those configure args on
Fedora too. The original configure check would use pkg-config to get
the compile/linker flags and then invoke 'compile_prog' to test
them. The new configure check uses pkg-config too, but skips the
'compile_prog' test, which is why you see a difference in behaviour
with these crypto patches applied.

To compare before and after state though, if I go back to before my
crypto patches were applied, and do

  ./configure  --enable-debug --static --disable-system && make

I get failures for many other libraries too, not least nothing seems
to check for glib2 static libs being present

  LINK  qemu-ga
/usr/bin/ld: cannot find -lgthread-2.0
/usr/bin/ld: cannot find -lglib-2.0

And if I install those, I get failures for a bunch of other storage
libraries too

  LINK  qemu-nbd
/usr/bin/ld: cannot find -liscsi
/usr/bin/ld: cannot find -lgfapi
/usr/bin/ld: cannot find -lglusterfs
/usr/bin/ld: cannot find -lgfrpc
/usr/bin/ld: cannot find -lgfxdr
/usr/bin/ld: cannot find -lssh2
/usr/bin/ld: cannot find -lssl
/usr/bin/ld: cannot find -lcrypto
/usr/bin/ld: cannot find -lgssapi_krb5
/usr/bin/ld: cannot find -lkrb5
/usr/bin/ld: cannot find -lcom_err
/usr/bin/ld: cannot find -lk5crypto
/usr/bin/ld: cannot find -lcrypto


I'm happy to fix configure to check that gnutls has working static
libs, but we do have a much broader pre-existing problem in this respect.
So it seems people wanting todo static builds currently have to deal with
installing all the correct static bits upfront or manually disabling
features in configure.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 24/07/2015 14:52, Daniel P. Berrange wrote:
> I'm happy to fix configure to check that gnutls has working static
> libs, but we do have a much broader pre-existing problem in this respect.
> So it seems people wanting todo static builds currently have to deal with
> installing all the correct static bits upfront or manually disabling
> features in configure.

Ok, then it looks like a preexisting problem indeed.  I'll add it to the
prerequisites section of the release notes.

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 24 July 2015 at 13:53, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 24/07/2015 14:52, Daniel P. Berrange wrote:
>> I'm happy to fix configure to check that gnutls has working static
>> libs, but we do have a much broader pre-existing problem in this respect.
>> So it seems people wanting todo static builds currently have to deal with
>> installing all the correct static bits upfront or manually disabling
>> features in configure.
>
> Ok, then it looks like a preexisting problem indeed.

Well, not entirely. This set of configure flags used to work,
and now it does not. That's a regression.

-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 24/07/2015 15:00, Peter Maydell wrote:
> > > I'm happy to fix configure to check that gnutls has working static
> > > libs, but we do have a much broader pre-existing problem in this respect.
> > > So it seems people wanting todo static builds currently have to deal with
> > > installing all the correct static bits upfront or manually disabling
> > > features in configure.
> >
> > Ok, then it looks like a preexisting problem indeed.
>
> Well, not entirely. This set of configure flags used to work,
> and now it does not. That's a regression.

Yes, it's indeed adding a new optional build prerequisite for the tools,
and like all other prerequisites you need to install the static version
manually.  But special-casing gnutls makes little sense when the same
problem is present for all other libraries.

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 24 July 2015 at 14:00, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 24 July 2015 at 13:53, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Ok, then it looks like a preexisting problem indeed.
>
> Well, not entirely. This set of configure flags used to work,
> and now it does not. That's a regression.

In particular, configure --static --disable-tools --disable-system
now fails, and I think that's a perfectly sensible set of options.

It fails trying to link
  LINK  tests/qemu-iotests/socket_scm_helper:

...why are we trying to build a helper program in tests/ for
a simple 'make' (not 'make check') ? Why are we building it
in a config which has disabled building tools and system exes?

Probably avoiding building this random exe would fix things.

thanks
-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Daniel P. Berrange]

On Fri, Jul 24, 2015 at 02:06:44PM +0100, Peter Maydell wrote:
> On 24 July 2015 at 14:00, Peter Maydell <peter.maydell@linaro.org> wrote:
> > On 24 July 2015 at 13:53, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >> Ok, then it looks like a preexisting problem indeed.
> >
> > Well, not entirely. This set of configure flags used to work,
> > and now it does not. That's a regression.
> 
> In particular, configure --static --disable-tools --disable-system
> now fails, and I think that's a perfectly sensible set of options.

Presumably you have previously figured out which static libs you
needed to install on your build machine previously though ?
As that command line fails to verify presence of any static
libraries at all - configure doesn't stop and tell me to install
any static libraries at all on my machine which has no static
libs present - it just fails to build later

> 
> It fails trying to link
>   LINK  tests/qemu-iotests/socket_scm_helper:
> 
> ...why are we trying to build a helper program in tests/ for
> a simple 'make' (not 'make check') ? Why are we building it
> in a config which has disabled building tools and system exes?
> 
> Probably avoiding building this random exe would fix things.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 24 July 2015 at 14:10, Daniel P. Berrange <berrange@redhat.com> wrote:
> On Fri, Jul 24, 2015 at 02:06:44PM +0100, Peter Maydell wrote:
>> On 24 July 2015 at 14:00, Peter Maydell <peter.maydell@linaro.org> wrote:
>> > On 24 July 2015 at 13:53, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> >> Ok, then it looks like a preexisting problem indeed.
>> >
>> > Well, not entirely. This set of configure flags used to work,
>> > and now it does not. That's a regression.
>>
>> In particular, configure --static --disable-tools --disable-system
>> now fails, and I think that's a perfectly sensible set of options.
>
> Presumably you have previously figured out which static libs you
> needed to install on your build machine previously though ?
> As that command line fails to verify presence of any static
> libraries at all - configure doesn't stop and tell me to install
> any static libraries at all on my machine which has no static
> libs present - it just fails to build later

Yes, but the set of required static libs for linux-user is pretty
tiny, and it doesn't generally change. In particular since we
don't want to link gnutls into linux-user we shouldn't be requiring
the user to scrape up a static lib of it to keep things working.
Requiring gnutls at all for these configure options is just a bug
(in configure or the makefile).

-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Daniel P. Berrange]

On Fri, Jul 24, 2015 at 02:31:12PM +0100, Peter Maydell wrote:
> On 24 July 2015 at 14:10, Daniel P. Berrange <berrange@redhat.com> wrote:
> > On Fri, Jul 24, 2015 at 02:06:44PM +0100, Peter Maydell wrote:
> >> On 24 July 2015 at 14:00, Peter Maydell <peter.maydell@linaro.org> wrote:
> >> > On 24 July 2015 at 13:53, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >> >> Ok, then it looks like a preexisting problem indeed.
> >> >
> >> > Well, not entirely. This set of configure flags used to work,
> >> > and now it does not. That's a regression.
> >>
> >> In particular, configure --static --disable-tools --disable-system
> >> now fails, and I think that's a perfectly sensible set of options.
> >
> > Presumably you have previously figured out which static libs you
> > needed to install on your build machine previously though ?
> > As that command line fails to verify presence of any static
> > libraries at all - configure doesn't stop and tell me to install
> > any static libraries at all on my machine which has no static
> > libs present - it just fails to build later
> 
> Yes, but the set of required static libs for linux-user is pretty
> tiny, and it doesn't generally change. In particular since we
> don't want to link gnutls into linux-user we shouldn't be requiring
> the user to scrape up a static lib of it to keep things working.
> Requiring gnutls at all for these configure options is just a bug
> (in configure or the makefile).

Ok, I'll send a patch to auto-disable it when missing

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 24/07/2015 15:48, Daniel P. Berrange wrote:
> > Yes, but the set of required static libs for linux-user is pretty
> > tiny, and it doesn't generally change. In particular since we
> > don't want to link gnutls into linux-user we shouldn't be requiring
> > the user to scrape up a static lib of it to keep things working.
> > Requiring gnutls at all for these configure options is just a bug
> > (in configure or the makefile).
>
> Ok, I'll send a patch to auto-disable it when missing

If the problem is just socket-scm-helper, which is only used to test
system emulators, we can make that depend on CONFIG_SOFTMMU.

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 24/07/2015 15:31, Peter Maydell wrote:
> Yes, but the set of required static libs for linux-user is pretty
> tiny, and it doesn't generally change. In particular since we
> don't want to link gnutls into linux-user we shouldn't be requiring
> the user to scrape up a static lib of it to keep things working.
> Requiring gnutls at all for these configure options is just a bug
> (in configure or the makefile).

Can you check if this works?

diff --git a/Makefile.objs b/Makefile.objs
index f094eff..9d988d9 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -2,7 +2,9 @@
 # Common libraries for tools and emulators
 stub-obj-y = stubs/
 util-obj-y = util/ qobject/ qapi/ qapi-types.o qapi-visit.o qapi-event.o
-util-obj-y += crypto/
+
+util-obj-$(call notempty,$(TOOLS)) += crypto/
+util-obj-$(CONFIG_SOFTMMU) += crypto/
 
 #######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Daniel P. Berrange]

On Fri, Jul 24, 2015 at 03:55:56PM +0200, Paolo Bonzini wrote:
> 
> 
> On 24/07/2015 15:31, Peter Maydell wrote:
> > Yes, but the set of required static libs for linux-user is pretty
> > tiny, and it doesn't generally change. In particular since we
> > don't want to link gnutls into linux-user we shouldn't be requiring
> > the user to scrape up a static lib of it to keep things working.
> > Requiring gnutls at all for these configure options is just a bug
> > (in configure or the makefile).
> 
> Can you check if this works?
> 
> diff --git a/Makefile.objs b/Makefile.objs
> index f094eff..9d988d9 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -2,7 +2,9 @@
>  # Common libraries for tools and emulators
>  stub-obj-y = stubs/
>  util-obj-y = util/ qobject/ qapi/ qapi-types.o qapi-visit.o qapi-event.o
> -util-obj-y += crypto/
> +
> +util-obj-$(call notempty,$(TOOLS)) += crypto/
> +util-obj-$(CONFIG_SOFTMMU) += crypto/

I have a feeling that'd cause the test suite to fail to link,
because it expects it to be unconditionally present in the
libqemuutil.a lib.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 24/07/2015 16:14, Daniel P. Berrange wrote:
>> > diff --git a/Makefile.objs b/Makefile.objs
>> > index f094eff..9d988d9 100644
>> > --- a/Makefile.objs
>> > +++ b/Makefile.objs
>> > @@ -2,7 +2,9 @@
>> >  # Common libraries for tools and emulators
>> >  stub-obj-y = stubs/
>> >  util-obj-y = util/ qobject/ qapi/ qapi-types.o qapi-visit.o qapi-event.o
>> > -util-obj-y += crypto/
>> > +
>> > +util-obj-$(call notempty,$(TOOLS)) += crypto/
>> > +util-obj-$(CONFIG_SOFTMMU) += crypto/
> I have a feeling that'd cause the test suite to fail to link,
> because it expects it to be unconditionally present in the
> libqemuutil.a lib.

Right...  We need to introduce CONFIG_CRYPTO and use it in
tests/Makefile too.

Paolo

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Daniel P. Berrange]

On Fri, Jul 24, 2015 at 04:15:31PM +0200, Paolo Bonzini wrote:
> 
> 
> On 24/07/2015 16:14, Daniel P. Berrange wrote:
> >> > diff --git a/Makefile.objs b/Makefile.objs
> >> > index f094eff..9d988d9 100644
> >> > --- a/Makefile.objs
> >> > +++ b/Makefile.objs
> >> > @@ -2,7 +2,9 @@
> >> >  # Common libraries for tools and emulators
> >> >  stub-obj-y = stubs/
> >> >  util-obj-y = util/ qobject/ qapi/ qapi-types.o qapi-visit.o qapi-event.o
> >> > -util-obj-y += crypto/
> >> > +
> >> > +util-obj-$(call notempty,$(TOOLS)) += crypto/
> >> > +util-obj-$(CONFIG_SOFTMMU) += crypto/
> > I have a feeling that'd cause the test suite to fail to link,
> > because it expects it to be unconditionally present in the
> > libqemuutil.a lib.
> 
> Right...  We need to introduce CONFIG_CRYPTO and use it in
> tests/Makefile too.

Except it was a goal of this work that we do not have such config
parameters leak outside to callers. The intent was that the APIs
are always built-in - we just conditionally disable the internal
implementation, so that code using these APIs does not need to
deal with conditionally disabling itself like this. It is
preferrable to just fix disablement of gnutls IMHO.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Paolo Bonzini]

On 24/07/2015 16:19, Daniel P. Berrange wrote:
>> > Right...  We need to introduce CONFIG_CRYPTO and use it in
>> > tests/Makefile too.
> Except it was a goal of this work that we do not have such config
> parameters leak outside to callers.

CONFIG_CRYPTO would disable all of crypto/ if neither tools nor system
emulators are compiled (e.g. using $(call lor) and $(call notempty)).
It would be an internal detail of the Makefile.

Paolo

> The intent was that the APIs
> are always built-in - we just conditionally disable the internal
> implementation, so that code using these APIs does not need to
> deal with conditionally disabling itself like this. It is
> preferrable to just fix disablement of gnutls IMHO.

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 24 July 2015 at 14:55, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 24/07/2015 15:31, Peter Maydell wrote:
>> Yes, but the set of required static libs for linux-user is pretty
>> tiny, and it doesn't generally change. In particular since we
>> don't want to link gnutls into linux-user we shouldn't be requiring
>> the user to scrape up a static lib of it to keep things working.
>> Requiring gnutls at all for these configure options is just a bug
>> (in configure or the makefile).
>
> Can you check if this works?
>
> diff --git a/Makefile.objs b/Makefile.objs
> index f094eff..9d988d9 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -2,7 +2,9 @@
>  # Common libraries for tools and emulators
>  stub-obj-y = stubs/
>  util-obj-y = util/ qobject/ qapi/ qapi-types.o qapi-visit.o qapi-event.o
> -util-obj-y += crypto/
> +
> +util-obj-$(call notempty,$(TOOLS)) += crypto/
> +util-obj-$(CONFIG_SOFTMMU) += crypto/

Unfortunately not -- we still try to build socket_scm_helper,
and it still fails with:
  LINK  tests/qemu-iotests/socket_scm_helper
c++: error: unrecognized command line option ‘-R’

The fix will need to be to either
(a) make sure we don't try to link anything which
links with LIBS_TOOLS, or
(b) make sure configure doesn't put the results of
'pkg-config --libs --static gnutls' into compiler commandlines
(it is:
-R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt -lgpg-error -ltasn1 -lz -lp11-kit
and that -R is a linker option, not a compiler option, I think.
But of course we use the compiler to do all our linking...)

thanks
-- PMM

Re: [Qemu-devel] [PULL v3 00/12] Final changes for 2.4-rc0

[Peter Maydell]

On 24 July 2015 at 15:25, Peter Maydell <peter.maydell@linaro.org> wrote:
> (b) make sure configure doesn't put the results of
> 'pkg-config --libs --static gnutls' into compiler commandlines
> (it is:
> -R/usr/lib/x86_64-linux-gnu -lgnutls -lgcrypt -lgpg-error -ltasn1 -lz -lp11-kit

FWIW, I've filed an Ubuntu bug:
https://bugs.launchpad.net/ubuntu/+source/gnutls26/+bug/1478035
because I think that pkg-config output is wrong and shouldn't
have the -R option in it at all.

-- PMM