[Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Paolo Bonzini]

The following changes since commit 39542105bbb19c690219d2f22844d8dfbd9bba05:

  Merge remote-tracking branch 'remotes/gkurz/tags/for-upstream' into staging (2016-11-01 12:48:07 +0000)

are available in the git repository at:

  git://github.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 7d175d29c9430fcba7a98f2c71925137b7870da4:

  main-loop: Suppress I/O thread warning under qtest (2016-11-02 09:28:57 +0100)

----------------------------------------------------------------
* NBD bugfix (Changlong)
* NBD write zeroes support (Eric)
* Memory backend fixes (Haozhong)
* Atomics fix (Alex)
* New AVX512 features (Luwei)
* "make check" logging fix (Paolo)
* Chardev refactoring fallout (Paolo)
* Small checkpatch improvements (Paolo, Jeff)

----------------------------------------------------------------
Alex Bennée (1):
      exec.c: ensure all AddressSpaceDispatch updates under RCU

Changlong Xie (1):
      nbd: Use CoQueue for free_sema instead of CoMutex

Christian Borntraeger (1):
      vl: exit qemu on guest panic if -no-shutdown is not set

Eric Blake (16):
      nbd: Add qemu-nbd -D for human-readable description
      nbd: Treat flags vs. command type as separate fields
      nbd: Rename NBDRequest to NBDRequestData
      nbd: Rename NbdClientSession to NBDClientSession
      nbd: Rename struct nbd_request and nbd_reply
      nbd: Share common reply-sending code in server
      nbd: Send message along with server NBD_REP_ERR errors
      nbd: Share common option-sending code in client
      nbd: Let server know when client gives up negotiation
      nbd: Let client skip portions of server reply
      nbd: Less allocation during NBD_OPT_LIST
      nbd: Support shorter handshake
      nbd: Refactor conversion to errno to silence checkpatch
      nbd: Improve server handling of shutdown requests
      nbd: Implement NBD_CMD_WRITE_ZEROES on server
      nbd: Implement NBD_CMD_WRITE_ZEROES on client

Haozhong Zhang (2):
      exec.c: do not truncate non-empty memory backend file
      exec.c: check memory backend file size with 'size' option

Jeff Cody (1):
      checkpatch: allow spaces before parenthesis for 'coroutine_fn'

Luwei Kang (1):
      x86: add AVX512_4VNNIW and AVX512_4FMAPS features

Max Reitz (1):
      main-loop: Suppress I/O thread warning under qtest

Paolo Bonzini (5):
      checkpatch: tweak "struct should normally be const" warning
      qemu-error: remove dependency of stubs on monitor
      tests: send error_report to test log
      qemu-char: do not forward events through the mux until QEMU has started
      slirp: fix CharDriver breakage

Pranith Kumar (1):
      docs/rcu.txt: Fix minor typo

 block/nbd-client.c          | 104 +++++----
 block/nbd-client.h          |  12 +-
 block/nbd.c                 |   8 +-
 docs/rcu.txt                |   2 +-
 exec.c                      |  33 ++-
 include/block/nbd.h         |  73 +++++--
 include/glib-compat.h       |  13 ++
 include/qemu/error-report.h |   1 +
 include/qemu/osdep.h        |   3 +
 main-loop.c                 |   2 +-
 monitor.c                   |  21 ++
 nbd/client.c                | 498 ++++++++++++++++++++++++--------------------
 nbd/nbd-internal.h          |  12 +-
 nbd/server.c                | 292 ++++++++++++++++++--------
 net/slirp.c                 |   3 +-
 qapi-schema.json            |   4 +-
 qemu-char.c                 |   8 +-
 qemu-nbd.c                  |  12 +-
 qemu-nbd.texi               |   5 +-
 scripts/checkpatch.pl       |   6 +-
 stubs/Makefile.objs         |   2 +-
 stubs/error-printf.c        |  19 ++
 stubs/mon-printf.c          |  11 -
 target-i386/cpu.c           |  19 +-
 target-i386/cpu.h           |   4 +
 util/qemu-error.c           |  26 +--
 vl.c                        |   5 +
 27 files changed, 763 insertions(+), 435 deletions(-)
 create mode 100644 stubs/error-printf.c
 delete mode 100644 stubs/mon-printf.c
-- 
2.7.4

[Qemu-devel] [PULL 07/30] exec.c: check memory backend file size with 'size' option

[Paolo Bonzini]

From: Haozhong Zhang <haozhong.zhang@intel.com>

If the memory backend file is not large enough to hold the required 'size',
Qemu will report error and exit.

Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
Message-Id: <20161027042300.5929-3-haozhong.zhang@intel.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20161102010551.2723-1-haozhong.zhang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/exec.c b/exec.c
index f471e73..f3c2770 100644
--- a/exec.c
+++ b/exec.c
@@ -1321,6 +1321,13 @@ static void *file_ram_alloc(RAMBlock *block,
         goto error;
     }
 
+    if (file_size > 0 && file_size < memory) {
+        error_setg(errp, "backing store %s size 0x%" PRIx64
+                   " does not match 'size' option 0x" RAM_ADDR_FMT,
+                   path, file_size, memory);
+        goto error;
+    }
+
     memory = ROUND_UP(memory, block->page_size);
 
     /*
-- 
2.7.4

Re: [Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 4470 bytes --]

On Wed, Nov 02, 2016 at 09:32:21AM +0100, Paolo Bonzini wrote:
> The following changes since commit 39542105bbb19c690219d2f22844d8dfbd9bba05:
> 
>   Merge remote-tracking branch 'remotes/gkurz/tags/for-upstream' into staging (2016-11-01 12:48:07 +0000)
> 
> are available in the git repository at:
> 
>   git://github.com/bonzini/qemu.git tags/for-upstream
> 
> for you to fetch changes up to 7d175d29c9430fcba7a98f2c71925137b7870da4:
> 
>   main-loop: Suppress I/O thread warning under qtest (2016-11-02 09:28:57 +0100)
> 
> ----------------------------------------------------------------
> * NBD bugfix (Changlong)
> * NBD write zeroes support (Eric)
> * Memory backend fixes (Haozhong)
> * Atomics fix (Alex)
> * New AVX512 features (Luwei)
> * "make check" logging fix (Paolo)
> * Chardev refactoring fallout (Paolo)
> * Small checkpatch improvements (Paolo, Jeff)
> 
> ----------------------------------------------------------------
> Alex Bennée (1):
>       exec.c: ensure all AddressSpaceDispatch updates under RCU
> 
> Changlong Xie (1):
>       nbd: Use CoQueue for free_sema instead of CoMutex
> 
> Christian Borntraeger (1):
>       vl: exit qemu on guest panic if -no-shutdown is not set
> 
> Eric Blake (16):
>       nbd: Add qemu-nbd -D for human-readable description
>       nbd: Treat flags vs. command type as separate fields
>       nbd: Rename NBDRequest to NBDRequestData
>       nbd: Rename NbdClientSession to NBDClientSession
>       nbd: Rename struct nbd_request and nbd_reply
>       nbd: Share common reply-sending code in server
>       nbd: Send message along with server NBD_REP_ERR errors
>       nbd: Share common option-sending code in client
>       nbd: Let server know when client gives up negotiation
>       nbd: Let client skip portions of server reply
>       nbd: Less allocation during NBD_OPT_LIST
>       nbd: Support shorter handshake
>       nbd: Refactor conversion to errno to silence checkpatch
>       nbd: Improve server handling of shutdown requests
>       nbd: Implement NBD_CMD_WRITE_ZEROES on server
>       nbd: Implement NBD_CMD_WRITE_ZEROES on client
> 
> Haozhong Zhang (2):
>       exec.c: do not truncate non-empty memory backend file
>       exec.c: check memory backend file size with 'size' option
> 
> Jeff Cody (1):
>       checkpatch: allow spaces before parenthesis for 'coroutine_fn'
> 
> Luwei Kang (1):
>       x86: add AVX512_4VNNIW and AVX512_4FMAPS features
> 
> Max Reitz (1):
>       main-loop: Suppress I/O thread warning under qtest
> 
> Paolo Bonzini (5):
>       checkpatch: tweak "struct should normally be const" warning
>       qemu-error: remove dependency of stubs on monitor
>       tests: send error_report to test log
>       qemu-char: do not forward events through the mux until QEMU has started
>       slirp: fix CharDriver breakage
> 
> Pranith Kumar (1):
>       docs/rcu.txt: Fix minor typo
> 
>  block/nbd-client.c          | 104 +++++----
>  block/nbd-client.h          |  12 +-
>  block/nbd.c                 |   8 +-
>  docs/rcu.txt                |   2 +-
>  exec.c                      |  33 ++-
>  include/block/nbd.h         |  73 +++++--
>  include/glib-compat.h       |  13 ++
>  include/qemu/error-report.h |   1 +
>  include/qemu/osdep.h        |   3 +
>  main-loop.c                 |   2 +-
>  monitor.c                   |  21 ++
>  nbd/client.c                | 498 ++++++++++++++++++++++++--------------------
>  nbd/nbd-internal.h          |  12 +-
>  nbd/server.c                | 292 ++++++++++++++++++--------
>  net/slirp.c                 |   3 +-
>  qapi-schema.json            |   4 +-
>  qemu-char.c                 |   8 +-
>  qemu-nbd.c                  |  12 +-
>  qemu-nbd.texi               |   5 +-
>  scripts/checkpatch.pl       |   6 +-
>  stubs/Makefile.objs         |   2 +-
>  stubs/error-printf.c        |  19 ++
>  stubs/mon-printf.c          |  11 -
>  target-i386/cpu.c           |  19 +-
>  target-i386/cpu.h           |   4 +
>  util/qemu-error.c           |  26 +--
>  vl.c                        |   5 +
>  27 files changed, 763 insertions(+), 435 deletions(-)
>  create mode 100644 stubs/error-printf.c
>  delete mode 100644 stubs/mon-printf.c
> -- 
> 2.7.4
> 
> 

Thanks, applied to my staging tree:
https://github.com/stefanha/qemu/commits/staging

Will be in qemu.git pending Travis-CI build tests.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1896 bytes --]

On 02.11.2016 09:32, Paolo Bonzini wrote:
> The following changes since commit 39542105bbb19c690219d2f22844d8dfbd9bba05:
> 
>   Merge remote-tracking branch 'remotes/gkurz/tags/for-upstream' into staging (2016-11-01 12:48:07 +0000)
> 
> are available in the git repository at:
> 
>   git://github.com/bonzini/qemu.git tags/for-upstream
> 
> for you to fetch changes up to 7d175d29c9430fcba7a98f2c71925137b7870da4:
> 
>   main-loop: Suppress I/O thread warning under qtest (2016-11-02 09:28:57 +0100)
> 
> ----------------------------------------------------------------
> * NBD bugfix (Changlong)
> * NBD write zeroes support (Eric)
> * Memory backend fixes (Haozhong)
> * Atomics fix (Alex)
> * New AVX512 features (Luwei)
> * "make check" logging fix (Paolo)
> * Chardev refactoring fallout (Paolo)
> * Small checkpatch improvements (Paolo, Jeff)
> 
> ----------------------------------------------------------------
> Alex Bennée (1):
>       exec.c: ensure all AddressSpaceDispatch updates under RCU
> 
> Changlong Xie (1):
>       nbd: Use CoQueue for free_sema instead of CoMutex
> 
> Christian Borntraeger (1):
>       vl: exit qemu on guest panic if -no-shutdown is not set
> 
> Eric Blake (16):
>       nbd: Add qemu-nbd -D for human-readable description
>       nbd: Treat flags vs. command type as separate fields
>       nbd: Rename NBDRequest to NBDRequestData
>       nbd: Rename NbdClientSession to NBDClientSession
>       nbd: Rename struct nbd_request and nbd_reply
>       nbd: Share common reply-sending code in server
>       nbd: Send message along with server NBD_REP_ERR errors
>       nbd: Share common option-sending code in client
>       nbd: Let server know when client gives up negotiation
>       nbd: Let client skip portions of server reply

This patch (7d3123e1775) breaks iotest 083 (it hangs somewhere).

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

On 11/04/2016 05:02 PM, Max Reitz wrote:

>> Eric Blake (16):
>>       nbd: Add qemu-nbd -D for human-readable description
>>       nbd: Treat flags vs. command type as separate fields
>>       nbd: Rename NBDRequest to NBDRequestData
>>       nbd: Rename NbdClientSession to NBDClientSession
>>       nbd: Rename struct nbd_request and nbd_reply
>>       nbd: Share common reply-sending code in server
>>       nbd: Send message along with server NBD_REP_ERR errors
>>       nbd: Share common option-sending code in client
>>       nbd: Let server know when client gives up negotiation
>>       nbd: Let client skip portions of server reply
> 
> This patch (7d3123e1775) breaks iotest 083 (it hangs somewhere).

Bummer. I'll investigate and submit the correction patch. (I guess my
usual './check -raw' and './check -qcow2' is insufficient since 83 is
skipped for those cases; but I just confirmed the './check -nbd 83' hang)

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

On 11/04/2016 05:24 PM, Eric Blake wrote:
> On 11/04/2016 05:02 PM, Max Reitz wrote:
> 
>>> Eric Blake (16):
>>>       nbd: Add qemu-nbd -D for human-readable description
>>>       nbd: Treat flags vs. command type as separate fields
>>>       nbd: Rename NBDRequest to NBDRequestData
>>>       nbd: Rename NbdClientSession to NBDClientSession
>>>       nbd: Rename struct nbd_request and nbd_reply
>>>       nbd: Share common reply-sending code in server
>>>       nbd: Send message along with server NBD_REP_ERR errors
>>>       nbd: Share common option-sending code in client
>>>       nbd: Let server know when client gives up negotiation
>>>       nbd: Let client skip portions of server reply
>>
>> This patch (7d3123e1775) breaks iotest 083 (it hangs somewhere).
> 
> Bummer. I'll investigate and submit the correction patch. (I guess my
> usual './check -raw' and './check -qcow2' is insufficient since 83 is
> skipped for those cases; but I just confirmed the './check -nbd 83' hang)

Not only does it hang, but hitting ^C to kill it leaves the terminal in
a weird state, where I had to use 'stty sane' to get my cursor back.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

On 11/04/2016 05:02 PM, Max Reitz wrote:

>>       nbd: Let client skip portions of server reply
> 
> This patch (7d3123e1775) breaks iotest 083 (it hangs somewhere).

Adding this breaks the infinite loop:

diff --git i/nbd/client.c w/nbd/client.c
index 5d94e34..23ad4f1 100644
--- i/nbd/client.c
+++ w/nbd/client.c
@@ -87,10 +87,11 @@ static ssize_t drop_sync(QIOChannel *ioc, size_t size)
     while (size > 0) {
         ret = read_sync(ioc, buffer, MIN(65536, size));
         if (ret < 0) {
             goto cleanup;
         }
+        assert(ret);
         assert(ret <= size);
         size -= ret;
     }
     ret = dropped;


but then fails the assertion, which means read_sync() is returning 0 in
our particular test (which sort of makes sense: the test is purposefully
truncating the server stream at various points to see how the client
reacts).  I'm trying to figure out if always treating early EOF as a
read error would be sufficient to make things work the way we want in
the client.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PULL v3 00/30] Misc patches for 2016-10-31

[Paolo Bonzini]

> but then fails the assertion, which means read_sync() is returning 0 in
> our particular test (which sort of makes sense: the test is purposefully
> truncating the server stream at various points to see how the client
> reacts).  I'm trying to figure out if always treating early EOF as a
> read error would be sufficient to make things work the way we want in
> the client.

Yeah, early EOF should become ESHUTDOWN, ECONNRESET or similar.

Paolo

[Qemu-devel] [PULL v2 00/30] Misc patches for 2016-10-31

[Paolo Bonzini]

The following changes since commit 39542105bbb19c690219d2f22844d8dfbd9bba05:

  Merge remote-tracking branch 'remotes/gkurz/tags/for-upstream' into staging (2016-11-01 12:48:07 +0000)

are available in the git repository at:

  git://github.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to a92e5ebbcea7c1d16f4aa75b08fce298fcd4cdfa:

  main-loop: Suppress I/O thread warning under qtest (2016-11-01 16:07:37 +0100)

----------------------------------------------------------------
* NBD bugfix (Changlong)
* NBD write zeroes support (Eric)
* Memory backend fixes (Haozhong)
* Atomics fix (Alex)
* New AVX512 features (Luwei)
* "make check" logging fix (Paolo)
* Chardev refactoring fallout (Paolo)
* Small checkpatch improvements (Paolo, Jeff)

----------------------------------------------------------------
Alex Bennée (1):
      exec.c: ensure all AddressSpaceDispatch updates under RCU

Changlong Xie (1):
      nbd: Use CoQueue for free_sema instead of CoMutex

Christian Borntraeger (1):
      vl: exit qemu on guest panic if -no-shutdown is not set

Eric Blake (16):
      nbd: Add qemu-nbd -D for human-readable description
      nbd: Treat flags vs. command type as separate fields
      nbd: Rename NBDRequest to NBDRequestData
      nbd: Rename NbdClientSession to NBDClientSession
      nbd: Rename struct nbd_request and nbd_reply
      nbd: Share common reply-sending code in server
      nbd: Send message along with server NBD_REP_ERR errors
      nbd: Share common option-sending code in client
      nbd: Let server know when client gives up negotiation
      nbd: Let client skip portions of server reply
      nbd: Less allocation during NBD_OPT_LIST
      nbd: Support shorter handshake
      nbd: Refactor conversion to errno to silence checkpatch
      nbd: Improve server handling of shutdown requests
      nbd: Implement NBD_CMD_WRITE_ZEROES on server
      nbd: Implement NBD_CMD_WRITE_ZEROES on client

Haozhong Zhang (2):
      exec.c: do not truncate non-empty memory backend file
      exec.c: check memory backend file size with 'size' option

Jeff Cody (1):
      checkpatch: allow spaces before parenthesis for 'coroutine_fn'

Luwei Kang (1):
      x86: add AVX512_4VNNIW and AVX512_4FMAPS features

Max Reitz (1):
      main-loop: Suppress I/O thread warning under qtest

Paolo Bonzini (5):
      checkpatch: tweak "struct should normally be const" warning
      qemu-error: remove dependency of stubs on monitor
      tests: send error_report to test log
      qemu-char: do not forward events through the mux until QEMU has started
      slirp: fix CharDriver breakage

Pranith Kumar (1):
      docs/rcu.txt: Fix minor typo

 block/nbd-client.c          | 104 +++++----
 block/nbd-client.h          |  12 +-
 block/nbd.c                 |   8 +-
 docs/rcu.txt                |   2 +-
 exec.c                      |  33 ++-
 include/block/nbd.h         |  73 +++++--
 include/glib-compat.h       |  13 ++
 include/qemu/error-report.h |   1 +
 include/qemu/osdep.h        |   3 +
 main-loop.c                 |   2 +-
 monitor.c                   |  21 ++
 nbd/client.c                | 498 ++++++++++++++++++++++++--------------------
 nbd/nbd-internal.h          |  12 +-
 nbd/server.c                | 292 ++++++++++++++++++--------
 net/slirp.c                 |   3 +-
 qapi-schema.json            |   4 +-
 qemu-char.c                 |   8 +-
 qemu-nbd.c                  |  12 +-
 qemu-nbd.texi               |   5 +-
 scripts/checkpatch.pl       |   6 +-
 stubs/Makefile.objs         |   2 +-
 stubs/error-printf.c        |  19 ++
 stubs/mon-printf.c          |  11 -
 target-i386/cpu.c           |  19 +-
 target-i386/cpu.h           |   4 +
 util/qemu-error.c           |  26 +--
 vl.c                        |   5 +
 27 files changed, 763 insertions(+), 435 deletions(-)
 create mode 100644 stubs/error-printf.c
 delete mode 100644 stubs/mon-printf.c
-- 
2.7.4

[Qemu-devel] [PULL 07/30] exec.c: check memory backend file size with 'size' option

[Paolo Bonzini]

From: Haozhong Zhang <haozhong.zhang@intel.com>

If the memory backend file is not large enough to hold the required 'size',
Qemu will report error and exit.

Signed-off-by: Haozhong Zhang <haozhong.zhang@intel.com>
Message-Id: <20161027042300.5929-3-haozhong.zhang@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 exec.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/exec.c b/exec.c
index f471e73..a3f9d8b 100644
--- a/exec.c
+++ b/exec.c
@@ -1321,6 +1321,13 @@ static void *file_ram_alloc(RAMBlock *block,
         goto error;
     }
 
+    if (file_size > 0 && file_size < memory) {
+        error_setg(errp, "backing store %s size %"PRId64
+                   " does not match 'size' option %"PRIu64,
+                   path, file_size, memory);
+        goto error;
+    }
+
     memory = ROUND_UP(memory, block->page_size);
 
     /*
-- 
2.7.4