[PULL 0/3] Block patches for 5.1

[PULL 0/3] Block patches for 5.1

[Max Reitz]

The following changes since commit af3d69058e09bede9900f266a618ed11f76f49f3:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200720' into staging (2020-07-20 15:58:07 +0100)

are available in the Git repository at:

  https://github.com/XanClic/qemu.git tags/pull-block-2020-07-21

for you to fetch changes up to 1d719ddc35e9827b6e5df771555874df34301a0d:

  block: fix bdrv_aio_cancel() for ENOMEDIUM requests (2020-07-21 12:00:38 +0200)

----------------------------------------------------------------
Block patches for 5.1:
- Let LUKS images only be shared between VMs if the guest device was
  configured to allow that
- Fix abort() from bdrv_aio_cancel() for guest devices without a BDS

----------------------------------------------------------------
Maxim Levitsky (2):
  block/crypto: disallow write sharing by default
  qemu-iotests: add testcase for bz #1857490

Stefan Hajnoczi (1):
  block: fix bdrv_aio_cancel() for ENOMEDIUM requests

 block/block-backend.c      |  8 +++++++
 block/crypto.c             |  2 +-
 tests/qemu-iotests/296     | 44 +++++++++++++++++++++++++++++++++++++-
 tests/qemu-iotests/296.out | 12 +++++++++--
 4 files changed, 62 insertions(+), 4 deletions(-)

-- 
2.26.2

[PULL 1/3] block/crypto: disallow write sharing by default

[Max Reitz]

From: Maxim Levitsky <mlevitsk@redhat.com>

My commit 'block/crypto: implement the encryption key management'
accidently allowed raw luks images to be shared between different
qemu processes without share-rw=on explicit override.
Fix that.

Fixes: bbfdae91fb ("block/crypto: implement the encryption key management")
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1857490

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20200719122059.59843-2-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/crypto.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/crypto.c b/block/crypto.c
index 8725c1bc02..0807557763 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -881,7 +881,7 @@ block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
      * For backward compatibility, manually share the write
      * and resize permission
      */
-    *nshared |= (BLK_PERM_WRITE | BLK_PERM_RESIZE);
+    *nshared |= shared & (BLK_PERM_WRITE | BLK_PERM_RESIZE);
     /*
      * Since we are not fully a format driver, don't always request
      * the read/resize permission but only when explicitly
-- 
2.26.2

[PULL 2/3] qemu-iotests: add testcase for bz #1857490

[Max Reitz]

From: Maxim Levitsky <mlevitsk@redhat.com>

Test that we can't write-share raw luks images by default,
but we still can with share-rw=on

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20200719122059.59843-3-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/296     | 44 +++++++++++++++++++++++++++++++++++++-
 tests/qemu-iotests/296.out | 12 +++++++++--
 2 files changed, 53 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/296 b/tests/qemu-iotests/296
index ec69ec8974..fb7dec88aa 100755
--- a/tests/qemu-iotests/296
+++ b/tests/qemu-iotests/296
@@ -133,6 +133,21 @@ class EncryptionSetupTestCase(iotests.QMPTestCase):
         )
         self.assert_qmp(result, 'return', {})
 
+
+    ###########################################################################
+    # add virtio-blk consumer for a block device
+    def addImageUser(self, vm, id, disk_id, share_rw=False):
+        result = vm.qmp('device_add', **
+            {
+                'driver': 'virtio-blk',
+                'id': id,
+                'drive': disk_id,
+                'share-rw' : share_rw
+            }
+        )
+
+        iotests.log(result)
+
     # close the encrypted block device
     def closeImageQmp(self, vm, id):
         result = vm.qmp('blockdev-del', **{ 'node-name': id })
@@ -159,7 +174,7 @@ class EncryptionSetupTestCase(iotests.QMPTestCase):
         vm.run_job('job0')
 
     # test that when the image opened by two qemu processes,
-    # neither of them can update the image
+    # neither of them can update the encryption keys
     def test1(self):
         self.createImg(test_img, self.secrets[0]);
 
@@ -193,6 +208,9 @@ class EncryptionSetupTestCase(iotests.QMPTestCase):
         os.remove(test_img)
 
 
+    # test that when the image opened by two qemu processes,
+    # even if first VM opens it read-only, the second can't update encryption
+    # keys
     def test2(self):
         self.createImg(test_img, self.secrets[0]);
 
@@ -226,6 +244,30 @@ class EncryptionSetupTestCase(iotests.QMPTestCase):
         self.closeImageQmp(self.vm1, "testdev")
         os.remove(test_img)
 
+    # test that two VMs can't open the same luks image by default
+    # and attach it to a guest device
+    def test3(self):
+        self.createImg(test_img, self.secrets[0]);
+
+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0])
+        self.addImageUser(self.vm1, "testctrl", "testdev")
+
+        self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
+        self.addImageUser(self.vm2, "testctrl", "testdev")
+
+
+    # test that two VMs can attach the same luks image to a guest device,
+    # if both use share-rw=on
+    def test4(self):
+        self.createImg(test_img, self.secrets[0]);
+
+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0])
+        self.addImageUser(self.vm1, "testctrl", "testdev", share_rw=True)
+
+        self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
+        self.addImageUser(self.vm2, "testctrl", "testdev", share_rw=True)
+
+
 
 if __name__ == '__main__':
     # support only raw luks since luks encrypted qcow2 is a proper
diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
index afb6d2d09d..cb2859a15c 100644
--- a/tests/qemu-iotests/296.out
+++ b/tests/qemu-iotests/296.out
@@ -26,8 +26,16 @@ Job failed: Failed to get shared "consistent read" lock
 {"return": {}}
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
-..
+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+
+{"return": {}}
+{"error": {"class": "GenericError", "desc": "Failed to get \"write\" lock"}}
+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+
+{"return": {}}
+{"return": {}}
+....
 ----------------------------------------------------------------------
-Ran 2 tests
+Ran 4 tests
 
 OK
-- 
2.26.2

[PULL 3/3] block: fix bdrv_aio_cancel() for ENOMEDIUM requests

[Max Reitz]

From: Stefan Hajnoczi <stefanha@redhat.com>

bdrv_aio_cancel() calls aio_poll() on the AioContext for the given I/O
request until it has completed. ENOMEDIUM requests are special because
there is no BlockDriverState when the drive has no medium!

Define a .get_aio_context() function for BlkAioEmAIOCB requests so that
bdrv_aio_cancel() can find the AioContext where the completion BH is
pending. Without this function bdrv_aio_cancel() aborts on ENOMEDIUM
requests!

libFuzzer triggered the following assertion:

  cat << EOF | qemu-system-i386 -M pc-q35-5.0 \
    -nographic -monitor none -serial none \
    -qtest stdio -trace ide\*
  outl 0xcf8 0x8000fa24
  outl 0xcfc 0xe106c000
  outl 0xcf8 0x8000fa04
  outw 0xcfc 0x7
  outl 0xcf8 0x8000fb20
  write 0x0 0x3 0x2780e7
  write 0xe106c22c 0xd 0x1130c218021130c218021130c2
  write 0xe106c218 0x15 0x110010110010110010110010110010110010110010
  EOF
  ide_exec_cmd IDE exec cmd: bus 0x56170a77a2b8; state 0x56170a77a340; cmd 0xe7
  ide_reset IDEstate 0x56170a77a340
  Aborted (core dumped)

  (gdb) bt
  #1  0x00007ffff4f93895 in abort () at /lib64/libc.so.6
  #2  0x0000555555dc6c00 in bdrv_aio_cancel (acb=0x555556765550) at block/io.c:2745
  #3  0x0000555555dac202 in blk_aio_cancel (acb=0x555556765550) at block/block-backend.c:1546
  #4  0x0000555555b1bd74 in ide_reset (s=0x555557213340) at hw/ide/core.c:1318
  #5  0x0000555555b1e3a1 in ide_bus_reset (bus=0x5555572132b8) at hw/ide/core.c:2422
  #6  0x0000555555b2aa27 in ahci_reset_port (s=0x55555720eb50, port=2) at hw/ide/ahci.c:650
  #7  0x0000555555b29fd7 in ahci_port_write (s=0x55555720eb50, port=2, offset=44, val=16) at hw/ide/ahci.c:360
  #8  0x0000555555b2a564 in ahci_mem_write (opaque=0x55555720eb50, addr=556, val=16, size=1) at hw/ide/ahci.c:513
  #9  0x000055555598415b in memory_region_write_accessor (mr=0x55555720eb80, addr=556, value=0x7fffffffb838, size=1, shift=0, mask=255, attrs=...) at softmmu/memory.c:483

Looking at bdrv_aio_cancel:

2728 /* async I/Os */
2729
2730 void bdrv_aio_cancel(BlockAIOCB *acb)
2731 {
2732     qemu_aio_ref(acb);
2733     bdrv_aio_cancel_async(acb);
2734     while (acb->refcnt > 1) {
2735         if (acb->aiocb_info->get_aio_context) {
2736             aio_poll(acb->aiocb_info->get_aio_context(acb), true);
2737         } else if (acb->bs) {
2738             /* qemu_aio_ref and qemu_aio_unref are not thread-safe, so
2739              * assert that we're not using an I/O thread.  Thread-safe
2740              * code should use bdrv_aio_cancel_async exclusively.
2741              */
2742             assert(bdrv_get_aio_context(acb->bs) == qemu_get_aio_context());
2743             aio_poll(bdrv_get_aio_context(acb->bs), true);
2744         } else {
2745             abort();     <===============
2746         }
2747     }
2748     qemu_aio_unref(acb);
2749 }

Fixes: 02c50efe08736116048d5fc355043080f4d5859c ("block: Add bdrv_aio_cancel_async")
Reported-by: Alexander Bulekov <alxndr@bu.edu>
Buglink: https://bugs.launchpad.net/qemu/+bug/1878255
Originally-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200720100141.129739-1-stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/block-backend.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index 0bf0188133..3a13cb5f0b 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1394,8 +1394,16 @@ typedef struct BlkAioEmAIOCB {
     bool has_returned;
 } BlkAioEmAIOCB;
 
+static AioContext *blk_aio_em_aiocb_get_aio_context(BlockAIOCB *acb_)
+{
+    BlkAioEmAIOCB *acb = container_of(acb_, BlkAioEmAIOCB, common);
+
+    return blk_get_aio_context(acb->rwco.blk);
+}
+
 static const AIOCBInfo blk_aio_em_aiocb_info = {
     .aiocb_size         = sizeof(BlkAioEmAIOCB),
+    .get_aio_context    = blk_aio_em_aiocb_get_aio_context,
 };
 
 static void blk_aio_complete(BlkAioEmAIOCB *acb)
-- 
2.26.2

Re: [PULL 0/3] Block patches for 5.1

[Peter Maydell]

On Tue, 21 Jul 2020 at 12:49, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit af3d69058e09bede9900f266a618ed11f76f49f3:
>
>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200720' into staging (2020-07-20 15:58:07 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2020-07-21
>
> for you to fetch changes up to 1d719ddc35e9827b6e5df771555874df34301a0d:
>
>   block: fix bdrv_aio_cancel() for ENOMEDIUM requests (2020-07-21 12:00:38 +0200)
>
> ----------------------------------------------------------------
> Block patches for 5.1:
> - Let LUKS images only be shared between VMs if the guest device was
>   configured to allow that
> - Fix abort() from bdrv_aio_cancel() for guest devices without a BDS
>
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM

[PULL 0/3] Block patches for 5.1

[Max Reitz]

The following changes since commit 4215d3413272ad6d1c6c9d0234450b602e46a74c:

  Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.1-20200727' into staging (2020-07-27 09:33:04 +0100)

are available in the Git repository at:

  https://github.com/XanClic/qemu.git tags/pull-block-2020-07-27

for you to fetch changes up to 1855536256eb0a5708b04b85f744de69559ea323:

  iotests/197: Fix for compat=0.10 (2020-07-27 16:35:17 +0200)

----------------------------------------------------------------
Block patches for 5.1:
- Coverity fix
- iotests fix for rx and avr
- iotests fix for qcow2 -o compat=0.10

----------------------------------------------------------------
Max Reitz (2):
  block/amend: Check whether the node exists
  iotests/197: Fix for compat=0.10

Thomas Huth (1):
  iotests: Select a default machine for the rx and avr targets

 block/amend.c            |  6 +++++-
 tests/qemu-iotests/197   |  4 +++-
 tests/qemu-iotests/check | 14 +++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)

-- 
2.26.2

Re: [PULL 0/3] Block patches for 5.1

[Peter Maydell]

On Mon, 27 Jul 2020 at 15:38, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit 4215d3413272ad6d1c6c9d0234450b602e46a74c:
>
>   Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.1-20200727' into staging (2020-07-27 09:33:04 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2020-07-27
>
> for you to fetch changes up to 1855536256eb0a5708b04b85f744de69559ea323:
>
>   iotests/197: Fix for compat=0.10 (2020-07-27 16:35:17 +0200)
>
> ----------------------------------------------------------------
> Block patches for 5.1:
> - Coverity fix
> - iotests fix for rx and avr
> - iotests fix for qcow2 -o compat=0.10
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM

Re: [PULL 0/3] Block patches for 5.1

[Peter Maydell]

On Tue, 28 Jul 2020 at 11:19, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Mon, 27 Jul 2020 at 15:38, Max Reitz <mreitz@redhat.com> wrote:
> > ----------------------------------------------------------------
> > Block patches for 5.1:
> > - Coverity fix
> > - iotests fix for rx and avr
> > - iotests fix for qcow2 -o compat=0.10
> >
>
> Applied, thanks.

This seems to have broken the "tcg disabled" build on gitlab:
https://gitlab.com/qemu-project/qemu/-/jobs/659352096


197       [1m [31mfail       [0m [10:57:48] [10:59:34]
   output mismatch (see 197.out.bad)
--- /builds/qemu-project/qemu/tests/qemu-iotests/197.out 2020-07-28
10:47:16.000000000 +0000
+++ /builds/qemu-project/qemu/build/tests/qemu-iotests/197.out.bad
2020-07-28 10:59:33.000000000 +0000
@@ -26,9 +26,9 @@

 === Partial final cluster ===

-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1024
+qemu-img: TEST_DIR/t.IMGFMT: Invalid parameter 'compat'
 read 1024/1024 bytes at offset 0
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-1 KiB (0x400) bytes     allocated at offset 0 bytes (0x0)
+4 GiB (0x100000000) bytes     allocated at offset 0 bytes (0x0)
 No errors were found on the image.
 *** done


thanks
-- PMM

Re: [PULL 0/3] Block patches for 5.1

[Thomas Huth]

On 28/07/2020 18.12, Peter Maydell wrote:
> On Tue, 28 Jul 2020 at 11:19, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> On Mon, 27 Jul 2020 at 15:38, Max Reitz <mreitz@redhat.com> wrote:
>>> ----------------------------------------------------------------
>>> Block patches for 5.1:
>>> - Coverity fix
>>> - iotests fix for rx and avr
>>> - iotests fix for qcow2 -o compat=0.10
>>>
>>
>> Applied, thanks.
> 
> This seems to have broken the "tcg disabled" build on gitlab:
> https://gitlab.com/qemu-project/qemu/-/jobs/659352096

Max already sent another pull request that contains the fix for this
issue, look for "[PULL 0/3] Block patches for 5.1.0-rc2?"

 Thomas