[Qemu-devel] [PULL 0/3] NBD patches for 2019-02-25

[Qemu-devel] [PULL 0/3] NBD patches for 2019-02-25

[Eric Blake]

The following changes since commit 59a568b57848b10e8a44518a889323f12ccdd8f4:

  Merge remote-tracking branch 'remotes/kraxel/tags/vga-20190222-pull-request' into staging (2019-02-25 12:49:07 +0000)

are available in the Git repository at:

  https://repo.or.cz/qemu/ericb.git tags/pull-nbd-2019-02-25

for you to fetch changes up to f62b3c953fae63e627cfe86333efb6294bb48a14:

  iotests: avoid broken pipe with certtool (2019-02-25 08:20:17 -0600)

----------------------------------------------------------------
nbd patches for 2019-02-25

- iotest failure fixes for tests related to NBD

----------------------------------------------------------------
Andrey Shinkevich (1):
      iotests: handle TypeError for Python3 in test 242

Daniel P. Berrangé (2):
      iotests: ensure we print nbd server log on error
      iotests: avoid broken pipe with certtool

 tests/qemu-iotests/common.tls | 48 ++++++++++++++++++++++++++++---------------
 tests/qemu-iotests/233        |  3 +++
 tests/qemu-iotests/242        |  8 ++++++--
 3 files changed, 41 insertions(+), 18 deletions(-)

-- 
2.20.1

[Qemu-devel] [PULL 1/3] iotests: handle TypeError for Python3 in test 242

[Eric Blake]

From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>

The data type for bytes in Python3 differs from the one in Python2.
Those cases should be managed separately.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reported-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <1550834773-873512-1-git-send-email-andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/242 | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242
index 16c65edcd7f..446fbf8ea13 100755
--- a/tests/qemu-iotests/242
+++ b/tests/qemu-iotests/242
@@ -20,6 +20,7 @@

 import iotests
 import json
+import sys
 from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \
     file_path, img_info_log, log, filter_qemu_io

@@ -65,9 +66,12 @@ def toggle_flag(offset):
     with open(disk, "r+b") as f:
         f.seek(offset, 0)
         c = f.read(1)
-        toggled = chr(ord(c) ^ bitmap_flag_unknown)
+        toggled = ord(c) ^ bitmap_flag_unknown
         f.seek(-1, 1)
-        f.write(toggled)
+        if sys.version_info.major >= 3:
+            f.write(bytes([toggled]))
+        else:
+            f.write(chr(toggled))


 qemu_img_create('-f', iotests.imgfmt, disk, '1M')
-- 
2.20.1

[Qemu-devel] [PULL 2/3] iotests: ensure we print nbd server log on error

[Eric Blake]

From: Daniel P. Berrangé <berrange@redhat.com>

If we abort the iotest early the server.log file might contain useful
information for diagnosing the problem. Ensure its contents are
displayed in this case.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20190220145819.30969-2-berrange@redhat.com>
[eblake: fix shell quoting]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/233 | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/233 b/tests/qemu-iotests/233
index fc345a1a46e..adb742fafb0 100755
--- a/tests/qemu-iotests/233
+++ b/tests/qemu-iotests/233
@@ -30,6 +30,8 @@ _cleanup()
 {
     nbd_server_stop
     _cleanup_test_img
+    # If we aborted early we want to see this log for diagnosis
+    test -f "$TEST_DIR/server.log" && cat "$TEST_DIR/server.log"
     rm -f "$TEST_DIR/server.log"
     tls_x509_cleanup
 }
@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io
 echo
 echo "== final server log =="
 cat "$TEST_DIR/server.log"
+rm -f "$TEST_DIR/server.log"

 # success, all done
 echo "*** done"
-- 
2.20.1

[Qemu-devel] [PULL 3/3] iotests: avoid broken pipe with certtool

[Eric Blake]

From: Daniel P. Berrangé <berrange@redhat.com>

When we run "certtool 2>&1 | head -1" the latter command is likely to
complete and exit before certtool has written everything it wants to
stderr. In at least the RHEL-7 gnutls 3.3.29 this causes certtool to
quit with broken pipe before it has finished writing the desired
output file to disk. This causes non-deterministic failures of the
iotest 233 because the certs are sometimes zero length files.
If certtool fails the "head -1" means we also lose any useful error
message it would have printed.

Thus this patch gets rid of the pipe and post-processes the output in a
more flexible & reliable manner.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20190220145819.30969-3-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/common.tls | 48 +++++++++++++++++++++++------------
 1 file changed, 32 insertions(+), 16 deletions(-)

diff --git a/tests/qemu-iotests/common.tls b/tests/qemu-iotests/common.tls
index eae81789bbc..3caf989d28c 100644
--- a/tests/qemu-iotests/common.tls
+++ b/tests/qemu-iotests/common.tls
@@ -29,6 +29,17 @@ tls_x509_cleanup()
 }


+tls_certtool()
+{
+    certtool "$@" 1>"${tls_dir}"/certtool.log 2>&1
+    if test "$?" = 0; then
+      head -1 "${tls_dir}"/certtool.log
+    else
+      cat "${tls_dir}"/certtool.log
+    fi
+    rm -f "${tls_dir}"/certtool.log
+}
+
 tls_x509_init()
 {
     (certtool --help) >/dev/null 2>&1 || \
@@ -71,10 +82,11 @@ ca
 cert_signing_key
 EOF

-    certtool --generate-self-signed \
-             --load-privkey "${tls_dir}/key.pem" \
-             --template "${tls_dir}/ca.info" \
-             --outfile "${tls_dir}/$name-cert.pem" 2>&1 | head -1
+    tls_certtool \
+        --generate-self-signed \
+        --load-privkey "${tls_dir}/key.pem" \
+        --template "${tls_dir}/ca.info" \
+        --outfile "${tls_dir}/$name-cert.pem"

     rm -f "${tls_dir}/ca.info"
 }
@@ -98,12 +110,14 @@ encryption_key
 signing_key
 EOF

-    certtool --generate-certificate \
-             --load-ca-privkey "${tls_dir}/key.pem" \
-             --load-ca-certificate "${tls_dir}/$caname-cert.pem" \
-             --load-privkey "${tls_dir}/key.pem" \
-             --template "${tls_dir}/cert.info" \
-             --outfile "${tls_dir}/$name/server-cert.pem" 2>&1 | head -1
+    tls_certtool \
+        --generate-certificate \
+        --load-ca-privkey "${tls_dir}/key.pem" \
+        --load-ca-certificate "${tls_dir}/$caname-cert.pem" \
+        --load-privkey "${tls_dir}/key.pem" \
+        --template "${tls_dir}/cert.info" \
+        --outfile "${tls_dir}/$name/server-cert.pem"
+
     ln -s "${tls_dir}/$caname-cert.pem" "${tls_dir}/$name/ca-cert.pem"
     ln -s "${tls_dir}/key.pem" "${tls_dir}/$name/server-key.pem"

@@ -127,12 +141,14 @@ encryption_key
 signing_key
 EOF

-    certtool --generate-certificate \
-             --load-ca-privkey "${tls_dir}/key.pem" \
-             --load-ca-certificate "${tls_dir}/$caname-cert.pem" \
-             --load-privkey "${tls_dir}/key.pem" \
-             --template "${tls_dir}/cert.info" \
-             --outfile "${tls_dir}/$name/client-cert.pem" 2>&1 | head -1
+    tls_certtool \
+        --generate-certificate \
+        --load-ca-privkey "${tls_dir}/key.pem" \
+        --load-ca-certificate "${tls_dir}/$caname-cert.pem" \
+        --load-privkey "${tls_dir}/key.pem" \
+        --template "${tls_dir}/cert.info" \
+        --outfile "${tls_dir}/$name/client-cert.pem"
+
     ln -s "${tls_dir}/$caname-cert.pem" "${tls_dir}/$name/ca-cert.pem"
     ln -s "${tls_dir}/key.pem" "${tls_dir}/$name/client-key.pem"

-- 
2.20.1

Re: [Qemu-devel] [PULL 0/3] NBD patches for 2019-02-25

[Eric Blake]

On 2/25/19 9:20 AM, Eric Blake wrote:
> The following changes since commit 59a568b57848b10e8a44518a889323f12ccdd8f4:
> 
>   Merge remote-tracking branch 'remotes/kraxel/tags/vga-20190222-pull-request' into staging (2019-02-25 12:49:07 +0000)
> 
> are available in the Git repository at:
> 
>   https://repo.or.cz/qemu/ericb.git tags/pull-nbd-2019-02-25
> 
> for you to fetch changes up to f62b3c953fae63e627cfe86333efb6294bb48a14:
> 
>   iotests: avoid broken pipe with certtool (2019-02-25 08:20:17 -0600)
> 
> ----------------------------------------------------------------
> nbd patches for 2019-02-25
> 
> - iotest failure fixes for tests related to NBD
> 
> ----------------------------------------------------------------
> Andrey Shinkevich (1):
>       iotests: handle TypeError for Python3 in test 242

NACK - there will be a v2 pull request with an update for this patch.

> 
> Daniel P. Berrangé (2):
>       iotests: ensure we print nbd server log on error
>       iotests: avoid broken pipe with certtool
> 
>  tests/qemu-iotests/common.tls | 48 ++++++++++++++++++++++++++++---------------
>  tests/qemu-iotests/233        |  3 +++
>  tests/qemu-iotests/242        |  8 ++++++--
>  3 files changed, 41 insertions(+), 18 deletions(-)
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org