qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PULL 0/3] Block patches
@ 2019-12-20 10:25 Stefan Hajnoczi
  2020-01-03 18:50 ` Peter Maydell
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2019-12-20 10:25 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Peter Maydell, Eduardo Habkost, qemu-block,
	libvir-list, Jason Wang, Max Reitz, Stefan Hajnoczi

The following changes since commit aceeaa69d28e6f08a24395d0aa6915b687d0a681:

  Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-12-17' into staging (2019-12-17 15:55:20 +0000)

are available in the Git repository at:

  https://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 725fe5d10dbd4259b1853b7d253cef83a3c0d22a:

  virtio-blk: fix out-of-bounds access to bitmap in notify_guest_bh (2019-12-19 16:20:25 +0000)

----------------------------------------------------------------
Pull request

----------------------------------------------------------------

Li Hangjing (1):
  virtio-blk: fix out-of-bounds access to bitmap in notify_guest_bh

Stefan Hajnoczi (2):
  virtio-blk: deprecate SCSI passthrough
  docs: fix rst syntax errors in unbuilt docs

 docs/arm-cpu-features.rst       |  6 +++---
 docs/virtio-net-failover.rst    |  4 ++--
 docs/virtio-pmem.rst            | 19 ++++++++++---------
 hw/block/dataplane/virtio-blk.c |  2 +-
 qemu-deprecated.texi            | 11 +++++++++++
 5 files changed, 27 insertions(+), 15 deletions(-)

-- 
2.23.0



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2019-12-20 10:25 Stefan Hajnoczi
@ 2020-01-03 18:50 ` Peter Maydell
  0 siblings, 0 replies; 29+ messages in thread
From: Peter Maydell @ 2020-01-03 18:50 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Eduardo Habkost, Qemu-block, Libvirt, Jason Wang,
	QEMU Developers, Max Reitz

On Fri, 20 Dec 2019 at 10:25, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit aceeaa69d28e6f08a24395d0aa6915b687d0a681:
>
>   Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-12-17' into staging (2019-12-17 15:55:20 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 725fe5d10dbd4259b1853b7d253cef83a3c0d22a:
>
>   virtio-blk: fix out-of-bounds access to bitmap in notify_guest_bh (2019-12-19 16:20:25 +0000)
>
> ----------------------------------------------------------------
> Pull request
>
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2020-03-06 14:23 Stefan Hajnoczi
  2020-03-06 17:02 ` Peter Maydell
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2020-03-06 14:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
	Alexander Bulekov, Bandan Das, Stefan Hajnoczi, Paolo Bonzini

The following changes since commit ef9f8fcbec6276414921dcd042575129a6331a2d:

  Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2020-03-04-2' into staging (2020-03-05 19:39:47 +0000)

are available in the Git repository at:

  https://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 1f40ace7b5634f93801c8474b9eb77fe2e00289c:

  tests: Fix a bug with count variables (2020-03-06 10:35:15 +0000)

----------------------------------------------------------------
Pull request

These patches would have gone through Thomas Huth but he is away on leave.

----------------------------------------------------------------

Alexander Bulekov (2):
  fuzz: fix style/typos in linker-script comments
  qtest: fix fuzzer-related 80-char limit violations

Tianjia Zhang (1):
  tests: Fix a bug with count variables

 qtest.c                       |  3 ++-
 tests/qtest/fuzz/fork_fuzz.ld | 16 ++++++++++------
 tests/qtest/fuzz/qos_fuzz.c   |  5 ++++-
 tests/test-rcu-list.c         |  2 +-
 4 files changed, 17 insertions(+), 9 deletions(-)

-- 
2.24.1


^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2020-03-06 14:23 Stefan Hajnoczi
@ 2020-03-06 17:02 ` Peter Maydell
  0 siblings, 0 replies; 29+ messages in thread
From: Peter Maydell @ 2020-03-06 17:02 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Laurent Vivier, Thomas Huth, Qemu-block, QEMU Developers,
	Alexander Bulekov, Bandan Das, Paolo Bonzini

On Fri, 6 Mar 2020 at 14:23, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit ef9f8fcbec6276414921dcd042575129a6331a2d:
>
>   Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2020-03-04-2' into staging (2020-03-05 19:39:47 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 1f40ace7b5634f93801c8474b9eb77fe2e00289c:
>
>   tests: Fix a bug with count variables (2020-03-06 10:35:15 +0000)
>
> ----------------------------------------------------------------
> Pull request
>
> These patches would have gone through Thomas Huth but he is away on leave.
>
> ----------------------------------------------------------------



Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2020-08-17 15:16 Stefan Hajnoczi
  2020-08-21 19:08 ` Peter Maydell
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2020-08-17 15:16 UTC (permalink / raw)
  To: qemu-devel, Peter Maydell
  Cc: Fam Zheng, Kevin Wolf, Eduardo Habkost, qemu-block, Max Reitz,
	Stefan Hajnoczi, Cleber Rosa, Paolo Bonzini

The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc:

  Update version for v5.1.0 release (2020-08-11 17:07:03 +0100)

are available in the Git repository at:

  https://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 44277bf914471962c9e88e09c859aae65ae109c4:

  aio-posix: keep aio_notify_me disabled during polling (2020-08-13 13:34:14 =
+0100)

----------------------------------------------------------------
Pull request

----------------------------------------------------------------

Stefan Hajnoczi (3):
  async: rename event_notifier_dummy_cb/poll()
  async: always set ctx->notified in aio_notify()
  aio-posix: keep aio_notify_me disabled during polling

 util/aio-posix.c | 47 +++++++++++++++++++++++++----------------------
 util/async.c     | 36 +++++++++++++++++++++++-------------
 2 files changed, 48 insertions(+), 35 deletions(-)

--=20
2.26.2


^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2020-08-17 15:16 Stefan Hajnoczi
@ 2020-08-21 19:08 ` Peter Maydell
  0 siblings, 0 replies; 29+ messages in thread
From: Peter Maydell @ 2020-08-21 19:08 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Fam Zheng, Kevin Wolf, Eduardo Habkost, Qemu-block,
	QEMU Developers, Max Reitz, Cleber Rosa, Paolo Bonzini

On Mon, 17 Aug 2020 at 16:16, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc:
>
>   Update version for v5.1.0 release (2020-08-11 17:07:03 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 44277bf914471962c9e88e09c859aae65ae109c4:
>
>   aio-posix: keep aio_notify_me disabled during polling (2020-08-13 13:34:14 =
> +0100)
>
> ----------------------------------------------------------------
> Pull request
>
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2
for any user-visible changes.

-- PMM


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2021-11-23 15:59 Hanna Reitz
  2021-11-23 17:58 ` Richard Henderson
  0 siblings, 1 reply; 29+ messages in thread
From: Hanna Reitz @ 2021-11-23 15:59 UTC (permalink / raw)
  To: qemu-block
  Cc: Kevin Wolf, Peter Maydell, Hanna Reitz, Richard Henderson,
	qemu-devel

The following changes since commit 73e0f70e097b7c92a5ce16ee35b53afe119b20d7:

  Merge tag 'pull-lu-20211123' of https://gitlab.com/rth7680/qemu into staging (2021-11-23 11:33:14 +0100)

are available in the Git repository at:

  https://gitlab.com/hreitz/qemu.git tags/pull-block-2021-11-23

for you to fetch changes up to 4dd218fd0717ed3cddb69c01eeb9da630107d89d:

  iotests/149: Skip on unsupported ciphers (2021-11-23 15:39:12 +0100)

----------------------------------------------------------------
Block patches for 6.2-rc2:
- Fix memory leak in vvfat when vvfat_open() fails
- iotest fixes for the gnutls crypto backend

----------------------------------------------------------------
Daniella Lee (1):
  block/vvfat.c fix leak when failure occurs

Hanna Reitz (2):
  iotests: Use aes-128-cbc
  iotests/149: Skip on unsupported ciphers

 block/vvfat.c              | 16 ++++++++++++----
 tests/qemu-iotests/149     | 23 ++++++++++++++++++-----
 tests/qemu-iotests/206     |  4 ++--
 tests/qemu-iotests/206.out |  6 +++---
 tests/qemu-iotests/210     |  4 ++--
 tests/qemu-iotests/210.out |  6 +++---
 6 files changed, 40 insertions(+), 19 deletions(-)

-- 
2.33.1



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2021-11-23 15:59 Hanna Reitz
@ 2021-11-23 17:58 ` Richard Henderson
  0 siblings, 0 replies; 29+ messages in thread
From: Richard Henderson @ 2021-11-23 17:58 UTC (permalink / raw)
  To: Hanna Reitz, qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-devel

On 11/23/21 4:59 PM, Hanna Reitz wrote:
> The following changes since commit 73e0f70e097b7c92a5ce16ee35b53afe119b20d7:
> 
>    Merge tag 'pull-lu-20211123' of https://gitlab.com/rth7680/qemu into staging (2021-11-23 11:33:14 +0100)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/hreitz/qemu.git tags/pull-block-2021-11-23
> 
> for you to fetch changes up to 4dd218fd0717ed3cddb69c01eeb9da630107d89d:
> 
>    iotests/149: Skip on unsupported ciphers (2021-11-23 15:39:12 +0100)
> 
> ----------------------------------------------------------------
> Block patches for 6.2-rc2:
> - Fix memory leak in vvfat when vvfat_open() fails
> - iotest fixes for the gnutls crypto backend
> 
> ----------------------------------------------------------------
> Daniella Lee (1):
>    block/vvfat.c fix leak when failure occurs
> 
> Hanna Reitz (2):
>    iotests: Use aes-128-cbc
>    iotests/149: Skip on unsupported ciphers
> 
>   block/vvfat.c              | 16 ++++++++++++----
>   tests/qemu-iotests/149     | 23 ++++++++++++++++++-----
>   tests/qemu-iotests/206     |  4 ++--
>   tests/qemu-iotests/206.out |  6 +++---
>   tests/qemu-iotests/210     |  4 ++--
>   tests/qemu-iotests/210.out |  6 +++---
>   6 files changed, 40 insertions(+), 19 deletions(-)

Applied, thanks.

r~



^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2021-12-22 16:52 Hanna Reitz
  2021-12-23  2:33 ` Richard Henderson
  0 siblings, 1 reply; 29+ messages in thread
From: Hanna Reitz @ 2021-12-22 16:52 UTC (permalink / raw)
  To: qemu-block; +Cc: Kevin Wolf, Hanna Reitz, Richard Henderson, qemu-devel

The following changes since commit 8c5f94cd4182753959c8be8de415120dc879d8f0:

  Merge tag 'pull-loong-20211221-2' of https://gitlab.com/rth7680/qemu into staging (2021-12-21 13:30:35 -0800)

are available in the Git repository at:

  https://gitlab.com/hreitz/qemu.git tags/pull-block-2021-12-22

for you to fetch changes up to 722f87df2545b308aec49b459b028f0802b4fd9e:

  iotests: check: multiprocessing support (2021-12-22 16:29:48 +0100)

----------------------------------------------------------------
Block patches:
- Added support to the iotests for running tests in several parallel
  jobs (using the new -j parameter)

----------------------------------------------------------------
Vladimir Sementsov-Ogievskiy (3):
  iotests/testrunner.py: add doc string for run_test()
  iotests/testrunner.py: move updating last_elapsed to run_tests
  iotests: check: multiprocessing support

 tests/qemu-iotests/check         |  4 +-
 tests/qemu-iotests/testrunner.py | 86 ++++++++++++++++++++++++++++----
 2 files changed, 80 insertions(+), 10 deletions(-)

-- 
2.33.1



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2021-12-22 16:52 Hanna Reitz
@ 2021-12-23  2:33 ` Richard Henderson
  0 siblings, 0 replies; 29+ messages in thread
From: Richard Henderson @ 2021-12-23  2:33 UTC (permalink / raw)
  To: Hanna Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel

On 12/22/21 8:52 AM, Hanna Reitz wrote:
> The following changes since commit 8c5f94cd4182753959c8be8de415120dc879d8f0:
> 
>    Merge tag 'pull-loong-20211221-2' of https://gitlab.com/rth7680/qemu into staging (2021-12-21 13:30:35 -0800)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/hreitz/qemu.git tags/pull-block-2021-12-22
> 
> for you to fetch changes up to 722f87df2545b308aec49b459b028f0802b4fd9e:
> 
>    iotests: check: multiprocessing support (2021-12-22 16:29:48 +0100)
> 
> ----------------------------------------------------------------
> Block patches:
> - Added support to the iotests for running tests in several parallel
>    jobs (using the new -j parameter)
> 
> ----------------------------------------------------------------
> Vladimir Sementsov-Ogievskiy (3):
>    iotests/testrunner.py: add doc string for run_test()
>    iotests/testrunner.py: move updating last_elapsed to run_tests
>    iotests: check: multiprocessing support
> 
>   tests/qemu-iotests/check         |  4 +-
>   tests/qemu-iotests/testrunner.py | 86 ++++++++++++++++++++++++++++----
>   2 files changed, 80 insertions(+), 10 deletions(-)

Applied, thanks.

r~


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2022-02-14 17:15 Stefan Hajnoczi
  2022-02-15 22:40 ` Peter Maydell
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-02-14 17:15 UTC (permalink / raw)
  To: qemu-devel, Peter Maydell
  Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, libvir-list,
	Dr. David Alan Gilbert, virtio-fs, Hanna Reitz, Stefan Hajnoczi

The following changes since commit cc5ce8b8b6be83e5fe3b668dbd061ad97c534e3f:

  Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220210' into staging (2022-02-13 20:33:28 +0000)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 4c41c69e05fe28c0f95f8abd2ebf407e95a4f04b:

  util: adjust coroutine pool size to virtio block queue (2022-02-14 17:11:25 +0000)

----------------------------------------------------------------
Pull request

This contains coroutine poll size scaling, virtiofsd rseq seccomp for new glibc
versions, and the QEMU C virtiofsd deprecation notice.

----------------------------------------------------------------

Christian Ehrhardt (1):
  tools/virtiofsd: Add rseq syscall to the seccomp allowlist

Dr. David Alan Gilbert (1):
  Deprecate C virtiofsd

Hiroki Narukawa (1):
  util: adjust coroutine pool size to virtio block queue

 docs/about/deprecated.rst             | 17 +++++++++++++++++
 include/qemu/coroutine.h              | 10 ++++++++++
 hw/block/virtio-blk.c                 |  5 +++++
 tools/virtiofsd/passthrough_seccomp.c |  3 +++
 util/qemu-coroutine.c                 | 20 ++++++++++++++++----
 5 files changed, 51 insertions(+), 4 deletions(-)

-- 
2.34.1




^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2022-02-14 17:15 Stefan Hajnoczi
@ 2022-02-15 22:40 ` Peter Maydell
  0 siblings, 0 replies; 29+ messages in thread
From: Peter Maydell @ 2022-02-15 22:40 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, libvir-list,
	qemu-devel, Dr. David Alan Gilbert, virtio-fs, Hanna Reitz

On Mon, 14 Feb 2022 at 17:44, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit cc5ce8b8b6be83e5fe3b668dbd061ad97c534e3f:
>
>   Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220210' into staging (2022-02-13 20:33:28 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 4c41c69e05fe28c0f95f8abd2ebf407e95a4f04b:
>
>   util: adjust coroutine pool size to virtio block queue (2022-02-14 17:11:25 +0000)
>
> ----------------------------------------------------------------
> Pull request
>
> This contains coroutine poll size scaling, virtiofsd rseq seccomp for new glibc
> versions, and the QEMU C virtiofsd deprecation notice.
>
> ----------------------------------------------------------------

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

thanks
-- PMM


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2022-04-25  8:48 Stefan Hajnoczi
  2022-04-25 20:35 ` Richard Henderson
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-04-25  8:48 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi, qemu-block

The following changes since commit 9c125d17e9402c232c46610802e5931b3639d77b:

  Merge tag 'pull-tcg-20220420' of https://gitlab.com/rth7680/qemu into staging (2022-04-20 16:43:11 -0700)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to d45c83328feab2e4083991693160f0a417cfd9b0:

  virtiofsd: Add docs/helper for killpriv_v2/no_killpriv_v2 option (2022-04-21 12:05:15 +0200)

----------------------------------------------------------------
Pull request

Small contrib/vhost-user-blk, contrib/vhost-user-scsi, and tools/virtiofsd
improvements.

----------------------------------------------------------------

Liu Yiding (1):
  virtiofsd: Add docs/helper for killpriv_v2/no_killpriv_v2 option

Sakshi Kaushik (1):
  Implements Backend Program conventions for vhost-user-scsi

Stefan Hajnoczi (1):
  contrib/vhost-user-blk: add missing GOptionEntry NULL terminator

 docs/tools/virtiofsd.rst                  |  5 ++
 contrib/vhost-user-blk/vhost-user-blk.c   |  3 +-
 contrib/vhost-user-scsi/vhost-user-scsi.c | 77 +++++++++++++++--------
 tools/virtiofsd/helper.c                  |  3 +
 4 files changed, 62 insertions(+), 26 deletions(-)

-- 
2.35.1



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2022-04-25  8:48 Stefan Hajnoczi
@ 2022-04-25 20:35 ` Richard Henderson
  0 siblings, 0 replies; 29+ messages in thread
From: Richard Henderson @ 2022-04-25 20:35 UTC (permalink / raw)
  To: Stefan Hajnoczi, qemu-devel; +Cc: Peter Maydell, qemu-block

On 4/25/22 01:48, Stefan Hajnoczi wrote:
> The following changes since commit 9c125d17e9402c232c46610802e5931b3639d77b:
> 
>    Merge tag 'pull-tcg-20220420' of https://gitlab.com/rth7680/qemu into staging (2022-04-20 16:43:11 -0700)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> 
> for you to fetch changes up to d45c83328feab2e4083991693160f0a417cfd9b0:
> 
>    virtiofsd: Add docs/helper for killpriv_v2/no_killpriv_v2 option (2022-04-21 12:05:15 +0200)
> 
> ----------------------------------------------------------------
> Pull request
> 
> Small contrib/vhost-user-blk, contrib/vhost-user-scsi, and tools/virtiofsd
> improvements.

Applied, thanks.  Please update https://wiki.qemu.org/ChangeLog/7.1 as appropriate.


r~



> 
> ----------------------------------------------------------------
> 
> Liu Yiding (1):
>    virtiofsd: Add docs/helper for killpriv_v2/no_killpriv_v2 option
> 
> Sakshi Kaushik (1):
>    Implements Backend Program conventions for vhost-user-scsi
> 
> Stefan Hajnoczi (1):
>    contrib/vhost-user-blk: add missing GOptionEntry NULL terminator
> 
>   docs/tools/virtiofsd.rst                  |  5 ++
>   contrib/vhost-user-blk/vhost-user-blk.c   |  3 +-
>   contrib/vhost-user-scsi/vhost-user-scsi.c | 77 +++++++++++++++--------
>   tools/virtiofsd/helper.c                  |  3 +
>   4 files changed, 62 insertions(+), 26 deletions(-)
> 



^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2022-05-05  8:42 Stefan Hajnoczi
  2022-05-05  8:42 ` [PULL 1/3] Introduce event-loop-base abstract class Stefan Hajnoczi
                   ` (3 more replies)
  0 siblings, 4 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-05-05  8:42 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-block, Richard Henderson, Stefan Hajnoczi

The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:

  Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:

  util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)

----------------------------------------------------------------
Pull request

Add new thread-pool-min/thread-pool-max parameters to control the thread pool
used for async I/O.

----------------------------------------------------------------

Nicolas Saenz Julienne (3):
  Introduce event-loop-base abstract class
  util/main-loop: Introduce the main loop into QOM
  util/event-loop-base: Introduce options to set the thread pool size

 qapi/qom.json                    |  43 ++++++++--
 meson.build                      |  26 +++---
 include/block/aio.h              |  10 +++
 include/block/thread-pool.h      |   3 +
 include/qemu/main-loop.h         |  10 +++
 include/sysemu/event-loop-base.h |  41 +++++++++
 include/sysemu/iothread.h        |   6 +-
 event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
 iothread.c                       |  68 +++++----------
 util/aio-posix.c                 |   1 +
 util/async.c                     |  20 +++++
 util/main-loop.c                 |  65 ++++++++++++++
 util/thread-pool.c               |  55 +++++++++++-
 13 files changed, 419 insertions(+), 69 deletions(-)
 create mode 100644 include/sysemu/event-loop-base.h
 create mode 100644 event-loop-base.c

-- 
2.35.1



^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 1/3] Introduce event-loop-base abstract class
  2022-05-05  8:42 [PULL 0/3] Block patches Stefan Hajnoczi
@ 2022-05-05  8:42 ` Stefan Hajnoczi
  2022-05-05  8:42 ` [PULL 2/3] util/main-loop: Introduce the main loop into QOM Stefan Hajnoczi
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-05-05  8:42 UTC (permalink / raw)
  To: qemu-devel
  Cc: qemu-block, Richard Henderson, Nicolas Saenz Julienne,
	Stefan Hajnoczi, Markus Armbruster

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>

Introduce the 'event-loop-base' abstract class, it'll hold the
properties common to all event loops and provide the necessary hooks for
their creation and maintenance. Then have iothread inherit from it.

EventLoopBaseClass is defined as user creatable and provides a hook for
its children to attach themselves to the user creatable class 'complete'
function. It also provides an update_params() callback to propagate
property changes onto its children.

The new 'event-loop-base' class will live in the root directory. It is
built on its own using the 'link_whole' option (there are no direct
function dependencies between the class and its children, it all happens
trough 'constructor' magic). And also imposes new compilation
dependencies:

    qom <- event-loop-base <- blockdev (iothread.c)

And in subsequent patches:

    qom <- event-loop-base <- qemuutil (util/main-loop.c)

All this forced some amount of reordering in meson.build:

 - Moved qom build definition before qemuutil. Doing it the other way
   around (i.e. moving qemuutil after qom) isn't possible as a lot of
   core libraries that live in between the two depend on it.

 - Process the 'hw' subdir earlier, as it introduces files into the
   'qom' source set.

No functional changes intended.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-2-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/qom.json                    |  22 +++++--
 meson.build                      |  23 ++++---
 include/sysemu/event-loop-base.h |  36 +++++++++++
 include/sysemu/iothread.h        |   6 +-
 event-loop-base.c                | 104 +++++++++++++++++++++++++++++++
 iothread.c                       |  65 ++++++-------------
 6 files changed, 192 insertions(+), 64 deletions(-)
 create mode 100644 include/sysemu/event-loop-base.h
 create mode 100644 event-loop-base.c

diff --git a/qapi/qom.json b/qapi/qom.json
index eeb5395ff3..a2439533c5 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -499,6 +499,20 @@
             '*repeat': 'bool',
             '*grab-toggle': 'GrabToggleKeys' } }
 
+##
+# @EventLoopBaseProperties:
+#
+# Common properties for event loops
+#
+# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
+#                 0 means that the engine will use its default.
+#                 (default: 0)
+#
+# Since: 7.1
+##
+{ 'struct': 'EventLoopBaseProperties',
+  'data': { '*aio-max-batch': 'int' } }
+
 ##
 # @IothreadProperties:
 #
@@ -516,17 +530,15 @@
 #               algorithm detects it is spending too long polling without
 #               encountering events. 0 selects a default behaviour (default: 0)
 #
-# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
-#                 0 means that the engine will use its default
-#                 (default:0, since 6.1)
+# The @aio-max-batch option is available since 6.1.
 #
 # Since: 2.0
 ##
 { 'struct': 'IothreadProperties',
+  'base': 'EventLoopBaseProperties',
   'data': { '*poll-max-ns': 'int',
             '*poll-grow': 'int',
-            '*poll-shrink': 'int',
-            '*aio-max-batch': 'int' } }
+            '*poll-shrink': 'int' } }
 
 ##
 # @MemoryBackendProperties:
diff --git a/meson.build b/meson.build
index c26aa442d4..11e9bd4824 100644
--- a/meson.build
+++ b/meson.build
@@ -2899,6 +2899,7 @@ subdir('qom')
 subdir('authz')
 subdir('crypto')
 subdir('ui')
+subdir('hw')
 
 
 if enable_modules
@@ -2906,6 +2907,18 @@ if enable_modules
   modulecommon = declare_dependency(link_whole: libmodulecommon, compile_args: '-DBUILD_DSO')
 endif
 
+qom_ss = qom_ss.apply(config_host, strict: false)
+libqom = static_library('qom', qom_ss.sources() + genh,
+                        dependencies: [qom_ss.dependencies()],
+                        name_suffix: 'fa')
+qom = declare_dependency(link_whole: libqom)
+
+event_loop_base = files('event-loop-base.c')
+event_loop_base = static_library('event-loop-base', sources: event_loop_base + genh,
+                                 build_by_default: true)
+event_loop_base = declare_dependency(link_whole: event_loop_base,
+                                     dependencies: [qom])
+
 stub_ss = stub_ss.apply(config_all, strict: false)
 
 util_ss.add_all(trace_ss)
@@ -2992,7 +3005,6 @@ subdir('monitor')
 subdir('net')
 subdir('replay')
 subdir('semihosting')
-subdir('hw')
 subdir('tcg')
 subdir('fpu')
 subdir('accel')
@@ -3117,13 +3129,6 @@ qemu_syms = custom_target('qemu.syms', output: 'qemu.syms',
                              capture: true,
                              command: [undefsym, nm, '@INPUT@'])
 
-qom_ss = qom_ss.apply(config_host, strict: false)
-libqom = static_library('qom', qom_ss.sources() + genh,
-                        dependencies: [qom_ss.dependencies()],
-                        name_suffix: 'fa')
-
-qom = declare_dependency(link_whole: libqom)
-
 authz_ss = authz_ss.apply(config_host, strict: false)
 libauthz = static_library('authz', authz_ss.sources() + genh,
                           dependencies: [authz_ss.dependencies()],
@@ -3176,7 +3181,7 @@ libblockdev = static_library('blockdev', blockdev_ss.sources() + genh,
                              build_by_default: false)
 
 blockdev = declare_dependency(link_whole: [libblockdev],
-                              dependencies: [block])
+                              dependencies: [block, event_loop_base])
 
 qmp_ss = qmp_ss.apply(config_host, strict: false)
 libqmp = static_library('qmp', qmp_ss.sources() + genh,
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
new file mode 100644
index 0000000000..8e77d8b69f
--- /dev/null
+++ b/include/sysemu/event-loop-base.h
@@ -0,0 +1,36 @@
+/*
+ * QEMU event-loop backend
+ *
+ * Copyright (C) 2022 Red Hat Inc
+ *
+ * Authors:
+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef QEMU_EVENT_LOOP_BASE_H
+#define QEMU_EVENT_LOOP_BASE_H
+
+#include "qom/object.h"
+#include "block/aio.h"
+#include "qemu/typedefs.h"
+
+#define TYPE_EVENT_LOOP_BASE         "event-loop-base"
+OBJECT_DECLARE_TYPE(EventLoopBase, EventLoopBaseClass,
+                    EVENT_LOOP_BASE)
+
+struct EventLoopBaseClass {
+    ObjectClass parent_class;
+
+    void (*init)(EventLoopBase *base, Error **errp);
+    void (*update_params)(EventLoopBase *base, Error **errp);
+};
+
+struct EventLoopBase {
+    Object parent;
+
+    /* AioContext AIO engine parameters */
+    int64_t aio_max_batch;
+};
+#endif
diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h
index 7f714bd136..8f8601d6ab 100644
--- a/include/sysemu/iothread.h
+++ b/include/sysemu/iothread.h
@@ -17,11 +17,12 @@
 #include "block/aio.h"
 #include "qemu/thread.h"
 #include "qom/object.h"
+#include "sysemu/event-loop-base.h"
 
 #define TYPE_IOTHREAD "iothread"
 
 struct IOThread {
-    Object parent_obj;
+    EventLoopBase parent_obj;
 
     QemuThread thread;
     AioContext *ctx;
@@ -37,9 +38,6 @@ struct IOThread {
     int64_t poll_max_ns;
     int64_t poll_grow;
     int64_t poll_shrink;
-
-    /* AioContext AIO engine parameters */
-    int64_t aio_max_batch;
 };
 typedef struct IOThread IOThread;
 
diff --git a/event-loop-base.c b/event-loop-base.c
new file mode 100644
index 0000000000..a924c73a7c
--- /dev/null
+++ b/event-loop-base.c
@@ -0,0 +1,104 @@
+/*
+ * QEMU event-loop base
+ *
+ * Copyright (C) 2022 Red Hat Inc
+ *
+ * Authors:
+ *  Stefan Hajnoczi <stefanha@redhat.com>
+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qom/object_interfaces.h"
+#include "qapi/error.h"
+#include "sysemu/event-loop-base.h"
+
+typedef struct {
+    const char *name;
+    ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
+} EventLoopBaseParamInfo;
+
+static EventLoopBaseParamInfo aio_max_batch_info = {
+    "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
+};
+
+static void event_loop_base_get_param(Object *obj, Visitor *v,
+        const char *name, void *opaque, Error **errp)
+{
+    EventLoopBase *event_loop_base = EVENT_LOOP_BASE(obj);
+    EventLoopBaseParamInfo *info = opaque;
+    int64_t *field = (void *)event_loop_base + info->offset;
+
+    visit_type_int64(v, name, field, errp);
+}
+
+static void event_loop_base_set_param(Object *obj, Visitor *v,
+        const char *name, void *opaque, Error **errp)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(obj);
+    EventLoopBase *base = EVENT_LOOP_BASE(obj);
+    EventLoopBaseParamInfo *info = opaque;
+    int64_t *field = (void *)base + info->offset;
+    int64_t value;
+
+    if (!visit_type_int64(v, name, &value, errp)) {
+        return;
+    }
+
+    if (value < 0) {
+        error_setg(errp, "%s value must be in range [0, %" PRId64 "]",
+                   info->name, INT64_MAX);
+        return;
+    }
+
+    *field = value;
+
+    if (bc->update_params) {
+        bc->update_params(base, errp);
+    }
+
+    return;
+}
+
+static void event_loop_base_complete(UserCreatable *uc, Error **errp)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
+    EventLoopBase *base = EVENT_LOOP_BASE(uc);
+
+    if (bc->init) {
+        bc->init(base, errp);
+    }
+}
+
+static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
+    ucc->complete = event_loop_base_complete;
+
+    object_class_property_add(klass, "aio-max-batch", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &aio_max_batch_info);
+}
+
+static const TypeInfo event_loop_base_info = {
+    .name = TYPE_EVENT_LOOP_BASE,
+    .parent = TYPE_OBJECT,
+    .instance_size = sizeof(EventLoopBase),
+    .class_size = sizeof(EventLoopBaseClass),
+    .class_init = event_loop_base_class_init,
+    .abstract = true,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};
+
+static void register_types(void)
+{
+    type_register_static(&event_loop_base_info);
+}
+type_init(register_types);
diff --git a/iothread.c b/iothread.c
index 0f98af0f2a..8fa2f3bfb8 100644
--- a/iothread.c
+++ b/iothread.c
@@ -17,6 +17,7 @@
 #include "qemu/module.h"
 #include "block/aio.h"
 #include "block/block.h"
+#include "sysemu/event-loop-base.h"
 #include "sysemu/iothread.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-misc.h"
@@ -152,10 +153,15 @@ static void iothread_init_gcontext(IOThread *iothread)
     iothread->main_loop = g_main_loop_new(iothread->worker_context, TRUE);
 }
 
-static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
+static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
 {
+    IOThread *iothread = IOTHREAD(base);
     ERRP_GUARD();
 
+    if (!iothread->ctx) {
+        return;
+    }
+
     aio_context_set_poll_params(iothread->ctx,
                                 iothread->poll_max_ns,
                                 iothread->poll_grow,
@@ -166,14 +172,15 @@ static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
     }
 
     aio_context_set_aio_params(iothread->ctx,
-                               iothread->aio_max_batch,
+                               iothread->parent_obj.aio_max_batch,
                                errp);
 }
 
-static void iothread_complete(UserCreatable *obj, Error **errp)
+
+static void iothread_init(EventLoopBase *base, Error **errp)
 {
     Error *local_error = NULL;
-    IOThread *iothread = IOTHREAD(obj);
+    IOThread *iothread = IOTHREAD(base);
     char *thread_name;
 
     iothread->stopping = false;
@@ -189,7 +196,7 @@ static void iothread_complete(UserCreatable *obj, Error **errp)
      */
     iothread_init_gcontext(iothread);
 
-    iothread_set_aio_context_params(iothread, &local_error);
+    iothread_set_aio_context_params(base, &local_error);
     if (local_error) {
         error_propagate(errp, local_error);
         aio_context_unref(iothread->ctx);
@@ -201,7 +208,7 @@ static void iothread_complete(UserCreatable *obj, Error **errp)
      * to inherit.
      */
     thread_name = g_strdup_printf("IO %s",
-                        object_get_canonical_path_component(OBJECT(obj)));
+                        object_get_canonical_path_component(OBJECT(base)));
     qemu_thread_create(&iothread->thread, thread_name, iothread_run,
                        iothread, QEMU_THREAD_JOINABLE);
     g_free(thread_name);
@@ -226,9 +233,6 @@ static IOThreadParamInfo poll_grow_info = {
 static IOThreadParamInfo poll_shrink_info = {
     "poll-shrink", offsetof(IOThread, poll_shrink),
 };
-static IOThreadParamInfo aio_max_batch_info = {
-    "aio-max-batch", offsetof(IOThread, aio_max_batch),
-};
 
 static void iothread_get_param(Object *obj, Visitor *v,
         const char *name, IOThreadParamInfo *info, Error **errp)
@@ -288,35 +292,12 @@ static void iothread_set_poll_param(Object *obj, Visitor *v,
     }
 }
 
-static void iothread_get_aio_param(Object *obj, Visitor *v,
-        const char *name, void *opaque, Error **errp)
-{
-    IOThreadParamInfo *info = opaque;
-
-    iothread_get_param(obj, v, name, info, errp);
-}
-
-static void iothread_set_aio_param(Object *obj, Visitor *v,
-        const char *name, void *opaque, Error **errp)
-{
-    IOThread *iothread = IOTHREAD(obj);
-    IOThreadParamInfo *info = opaque;
-
-    if (!iothread_set_param(obj, v, name, info, errp)) {
-        return;
-    }
-
-    if (iothread->ctx) {
-        aio_context_set_aio_params(iothread->ctx,
-                                   iothread->aio_max_batch,
-                                   errp);
-    }
-}
-
 static void iothread_class_init(ObjectClass *klass, void *class_data)
 {
-    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
-    ucc->complete = iothread_complete;
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(klass);
+
+    bc->init = iothread_init;
+    bc->update_params = iothread_set_aio_context_params;
 
     object_class_property_add(klass, "poll-max-ns", "int",
                               iothread_get_poll_param,
@@ -330,23 +311,15 @@ static void iothread_class_init(ObjectClass *klass, void *class_data)
                               iothread_get_poll_param,
                               iothread_set_poll_param,
                               NULL, &poll_shrink_info);
-    object_class_property_add(klass, "aio-max-batch", "int",
-                              iothread_get_aio_param,
-                              iothread_set_aio_param,
-                              NULL, &aio_max_batch_info);
 }
 
 static const TypeInfo iothread_info = {
     .name = TYPE_IOTHREAD,
-    .parent = TYPE_OBJECT,
+    .parent = TYPE_EVENT_LOOP_BASE,
     .class_init = iothread_class_init,
     .instance_size = sizeof(IOThread),
     .instance_init = iothread_instance_init,
     .instance_finalize = iothread_instance_finalize,
-    .interfaces = (InterfaceInfo[]) {
-        {TYPE_USER_CREATABLE},
-        {}
-    },
 };
 
 static void iothread_register_types(void)
@@ -383,7 +356,7 @@ static int query_one_iothread(Object *object, void *opaque)
     info->poll_max_ns = iothread->poll_max_ns;
     info->poll_grow = iothread->poll_grow;
     info->poll_shrink = iothread->poll_shrink;
-    info->aio_max_batch = iothread->aio_max_batch;
+    info->aio_max_batch = iothread->parent_obj.aio_max_batch;
 
     QAPI_LIST_APPEND(*tail, info);
     return 0;
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 29+ messages in thread

* [PULL 2/3] util/main-loop: Introduce the main loop into QOM
  2022-05-05  8:42 [PULL 0/3] Block patches Stefan Hajnoczi
  2022-05-05  8:42 ` [PULL 1/3] Introduce event-loop-base abstract class Stefan Hajnoczi
@ 2022-05-05  8:42 ` Stefan Hajnoczi
  2022-05-05  8:42 ` [PULL 3/3] util/event-loop-base: Introduce options to set the thread pool size Stefan Hajnoczi
  2022-05-05 16:29 ` [PULL 0/3] Block patches Richard Henderson
  3 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-05-05  8:42 UTC (permalink / raw)
  To: qemu-devel
  Cc: qemu-block, Richard Henderson, Nicolas Saenz Julienne,
	Stefan Hajnoczi, Markus Armbruster

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>

'event-loop-base' provides basic property handling for all 'AioContext'
based event loops. So let's define a new 'MainLoopClass' that inherits
from it. This will permit tweaking the main loop's properties through
qapi as well as through the command line using the '-object' keyword[1].
Only one instance of 'MainLoopClass' might be created at any time.

'EventLoopBaseClass' learns a new callback, 'can_be_deleted()' so as to
mark 'MainLoop' as non-deletable.

[1] For example:
      -object main-loop,id=main-loop,aio-max-batch=<value>

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-3-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/qom.json                    | 13 ++++++++
 meson.build                      |  3 +-
 include/qemu/main-loop.h         | 10 ++++++
 include/sysemu/event-loop-base.h |  1 +
 event-loop-base.c                | 13 ++++++++
 util/main-loop.c                 | 56 ++++++++++++++++++++++++++++++++
 6 files changed, 95 insertions(+), 1 deletion(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index a2439533c5..7d4a2ac1b9 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -540,6 +540,17 @@
             '*poll-grow': 'int',
             '*poll-shrink': 'int' } }
 
+##
+# @MainLoopProperties:
+#
+# Properties for the main-loop object.
+#
+# Since: 7.1
+##
+{ 'struct': 'MainLoopProperties',
+  'base': 'EventLoopBaseProperties',
+  'data': {} }
+
 ##
 # @MemoryBackendProperties:
 #
@@ -830,6 +841,7 @@
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
     'iothread',
+    'main-loop',
     { 'name': 'memory-backend-epc',
       'if': 'CONFIG_LINUX' },
     'memory-backend-file',
@@ -895,6 +907,7 @@
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' },
       'iothread':                   'IothreadProperties',
+      'main-loop':                  'MainLoopProperties',
       'memory-backend-epc':         { 'type': 'MemoryBackendEpcProperties',
                                       'if': 'CONFIG_LINUX' },
       'memory-backend-file':        'MemoryBackendFileProperties',
diff --git a/meson.build b/meson.build
index 11e9bd4824..b3d7ed41f9 100644
--- a/meson.build
+++ b/meson.build
@@ -2927,7 +2927,8 @@ libqemuutil = static_library('qemuutil',
                              sources: util_ss.sources() + stub_ss.sources() + genh,
                              dependencies: [util_ss.dependencies(), libm, threads, glib, socket, malloc, pixman])
 qemuutil = declare_dependency(link_with: libqemuutil,
-                              sources: genh + version_res)
+                              sources: genh + version_res,
+                              dependencies: [event_loop_base])
 
 if have_system or have_user
   decodetree = generator(find_program('scripts/decodetree.py'),
diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index d3750c8e76..20c9387654 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -26,9 +26,19 @@
 #define QEMU_MAIN_LOOP_H
 
 #include "block/aio.h"
+#include "qom/object.h"
+#include "sysemu/event-loop-base.h"
 
 #define SIG_IPI SIGUSR1
 
+#define TYPE_MAIN_LOOP  "main-loop"
+OBJECT_DECLARE_TYPE(MainLoop, MainLoopClass, MAIN_LOOP)
+
+struct MainLoop {
+    EventLoopBase parent_obj;
+};
+typedef struct MainLoop MainLoop;
+
 /**
  * qemu_init_main_loop: Set up the process so that it can run the main loop.
  *
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
index 8e77d8b69f..fced4c9fea 100644
--- a/include/sysemu/event-loop-base.h
+++ b/include/sysemu/event-loop-base.h
@@ -25,6 +25,7 @@ struct EventLoopBaseClass {
 
     void (*init)(EventLoopBase *base, Error **errp);
     void (*update_params)(EventLoopBase *base, Error **errp);
+    bool (*can_be_deleted)(EventLoopBase *base);
 };
 
 struct EventLoopBase {
diff --git a/event-loop-base.c b/event-loop-base.c
index a924c73a7c..e7f99a6ec8 100644
--- a/event-loop-base.c
+++ b/event-loop-base.c
@@ -73,10 +73,23 @@ static void event_loop_base_complete(UserCreatable *uc, Error **errp)
     }
 }
 
+static bool event_loop_base_can_be_deleted(UserCreatable *uc)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
+    EventLoopBase *backend = EVENT_LOOP_BASE(uc);
+
+    if (bc->can_be_deleted) {
+        return bc->can_be_deleted(backend);
+    }
+
+    return true;
+}
+
 static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
 {
     UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
     ucc->complete = event_loop_base_complete;
+    ucc->can_be_deleted = event_loop_base_can_be_deleted;
 
     object_class_property_add(klass, "aio-max-batch", "int",
                               event_loop_base_get_param,
diff --git a/util/main-loop.c b/util/main-loop.c
index 9afac10dff..e30f034815 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -33,6 +33,7 @@
 #include "qemu/error-report.h"
 #include "qemu/queue.h"
 #include "qemu/compiler.h"
+#include "qom/object.h"
 
 #ifndef _WIN32
 #include <sys/wait.h>
@@ -184,6 +185,61 @@ int qemu_init_main_loop(Error **errp)
     return 0;
 }
 
+static void main_loop_update_params(EventLoopBase *base, Error **errp)
+{
+    if (!qemu_aio_context) {
+        error_setg(errp, "qemu aio context not ready");
+        return;
+    }
+
+    aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+}
+
+MainLoop *mloop;
+
+static void main_loop_init(EventLoopBase *base, Error **errp)
+{
+    MainLoop *m = MAIN_LOOP(base);
+
+    if (mloop) {
+        error_setg(errp, "only one main-loop instance allowed");
+        return;
+    }
+
+    main_loop_update_params(base, errp);
+
+    mloop = m;
+    return;
+}
+
+static bool main_loop_can_be_deleted(EventLoopBase *base)
+{
+    return false;
+}
+
+static void main_loop_class_init(ObjectClass *oc, void *class_data)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(oc);
+
+    bc->init = main_loop_init;
+    bc->update_params = main_loop_update_params;
+    bc->can_be_deleted = main_loop_can_be_deleted;
+}
+
+static const TypeInfo main_loop_info = {
+    .name = TYPE_MAIN_LOOP,
+    .parent = TYPE_EVENT_LOOP_BASE,
+    .class_init = main_loop_class_init,
+    .instance_size = sizeof(MainLoop),
+};
+
+static void main_loop_register_types(void)
+{
+    type_register_static(&main_loop_info);
+}
+
+type_init(main_loop_register_types)
+
 static int max_priority;
 
 #ifndef _WIN32
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 29+ messages in thread

* [PULL 3/3] util/event-loop-base: Introduce options to set the thread pool size
  2022-05-05  8:42 [PULL 0/3] Block patches Stefan Hajnoczi
  2022-05-05  8:42 ` [PULL 1/3] Introduce event-loop-base abstract class Stefan Hajnoczi
  2022-05-05  8:42 ` [PULL 2/3] util/main-loop: Introduce the main loop into QOM Stefan Hajnoczi
@ 2022-05-05  8:42 ` Stefan Hajnoczi
  2022-05-05 16:29 ` [PULL 0/3] Block patches Richard Henderson
  3 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-05-05  8:42 UTC (permalink / raw)
  To: qemu-devel
  Cc: qemu-block, Richard Henderson, Nicolas Saenz Julienne,
	Stefan Hajnoczi, Markus Armbruster

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>

The thread pool regulates itself: when idle, it kills threads until
empty, when in demand, it creates new threads until full. This behaviour
doesn't play well with latency sensitive workloads where the price of
creating a new thread is too high. For example, when paired with qemu's
'-mlock', or using safety features like SafeStack, creating a new thread
has been measured take multiple milliseconds.

In order to mitigate this let's introduce a new 'EventLoopBase'
property to set the thread pool size. The threads will be created during
the pool's initialization or upon updating the property's value, remain
available during its lifetime regardless of demand, and destroyed upon
freeing it. A properly characterized workload will then be able to
configure the pool to avoid any latency spikes.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-4-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/qom.json                    | 10 +++++-
 include/block/aio.h              | 10 ++++++
 include/block/thread-pool.h      |  3 ++
 include/sysemu/event-loop-base.h |  4 +++
 event-loop-base.c                | 23 +++++++++++++
 iothread.c                       |  3 ++
 util/aio-posix.c                 |  1 +
 util/async.c                     | 20 ++++++++++++
 util/main-loop.c                 |  9 ++++++
 util/thread-pool.c               | 55 +++++++++++++++++++++++++++++---
 10 files changed, 133 insertions(+), 5 deletions(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index 7d4a2ac1b9..6a653c6636 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -508,10 +508,18 @@
 #                 0 means that the engine will use its default.
 #                 (default: 0)
 #
+# @thread-pool-min: minimum number of threads reserved in the thread pool
+#                   (default:0)
+#
+# @thread-pool-max: maximum number of threads the thread pool can contain
+#                   (default:64)
+#
 # Since: 7.1
 ##
 { 'struct': 'EventLoopBaseProperties',
-  'data': { '*aio-max-batch': 'int' } }
+  'data': { '*aio-max-batch': 'int',
+            '*thread-pool-min': 'int',
+            '*thread-pool-max': 'int' } }
 
 ##
 # @IothreadProperties:
diff --git a/include/block/aio.h b/include/block/aio.h
index 5634173b12..d128558f1d 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -192,6 +192,8 @@ struct AioContext {
     QSLIST_HEAD(, Coroutine) scheduled_coroutines;
     QEMUBH *co_schedule_bh;
 
+    int thread_pool_min;
+    int thread_pool_max;
     /* Thread pool for performing work and receiving completion callbacks.
      * Has its own locking.
      */
@@ -769,4 +771,12 @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
 void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch,
                                 Error **errp);
 
+/**
+ * aio_context_set_thread_pool_params:
+ * @ctx: the aio context
+ * @min: min number of threads to have readily available in the thread pool
+ * @min: max number of threads the thread pool can contain
+ */
+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
+                                        int64_t max, Error **errp);
 #endif
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index 7dd7d730a0..2020bcc92d 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -20,6 +20,8 @@
 
 #include "block/block.h"
 
+#define THREAD_POOL_MAX_THREADS_DEFAULT         64
+
 typedef int ThreadPoolFunc(void *opaque);
 
 typedef struct ThreadPool ThreadPool;
@@ -33,5 +35,6 @@ BlockAIOCB *thread_pool_submit_aio(ThreadPool *pool,
 int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg);
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+void thread_pool_update_params(ThreadPool *pool, struct AioContext *ctx);
 
 #endif
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
index fced4c9fea..2748bf6ae1 100644
--- a/include/sysemu/event-loop-base.h
+++ b/include/sysemu/event-loop-base.h
@@ -33,5 +33,9 @@ struct EventLoopBase {
 
     /* AioContext AIO engine parameters */
     int64_t aio_max_batch;
+
+    /* AioContext thread pool parameters */
+    int64_t thread_pool_min;
+    int64_t thread_pool_max;
 };
 #endif
diff --git a/event-loop-base.c b/event-loop-base.c
index e7f99a6ec8..d5be4dc6fc 100644
--- a/event-loop-base.c
+++ b/event-loop-base.c
@@ -14,6 +14,7 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 #include "qapi/error.h"
+#include "block/thread-pool.h"
 #include "sysemu/event-loop-base.h"
 
 typedef struct {
@@ -21,9 +22,22 @@ typedef struct {
     ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
 } EventLoopBaseParamInfo;
 
+static void event_loop_base_instance_init(Object *obj)
+{
+    EventLoopBase *base = EVENT_LOOP_BASE(obj);
+
+    base->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
+}
+
 static EventLoopBaseParamInfo aio_max_batch_info = {
     "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
 };
+static EventLoopBaseParamInfo thread_pool_min_info = {
+    "thread-pool-min", offsetof(EventLoopBase, thread_pool_min),
+};
+static EventLoopBaseParamInfo thread_pool_max_info = {
+    "thread-pool-max", offsetof(EventLoopBase, thread_pool_max),
+};
 
 static void event_loop_base_get_param(Object *obj, Visitor *v,
         const char *name, void *opaque, Error **errp)
@@ -95,12 +109,21 @@ static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
                               event_loop_base_get_param,
                               event_loop_base_set_param,
                               NULL, &aio_max_batch_info);
+    object_class_property_add(klass, "thread-pool-min", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &thread_pool_min_info);
+    object_class_property_add(klass, "thread-pool-max", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &thread_pool_max_info);
 }
 
 static const TypeInfo event_loop_base_info = {
     .name = TYPE_EVENT_LOOP_BASE,
     .parent = TYPE_OBJECT,
     .instance_size = sizeof(EventLoopBase),
+    .instance_init = event_loop_base_instance_init,
     .class_size = sizeof(EventLoopBaseClass),
     .class_init = event_loop_base_class_init,
     .abstract = true,
diff --git a/iothread.c b/iothread.c
index 8fa2f3bfb8..529194a566 100644
--- a/iothread.c
+++ b/iothread.c
@@ -174,6 +174,9 @@ static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
     aio_context_set_aio_params(iothread->ctx,
                                iothread->parent_obj.aio_max_batch,
                                errp);
+
+    aio_context_set_thread_pool_params(iothread->ctx, base->thread_pool_min,
+                                       base->thread_pool_max, errp);
 }
 
 
diff --git a/util/aio-posix.c b/util/aio-posix.c
index be0182a3c6..731f3826c0 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -15,6 +15,7 @@
 
 #include "qemu/osdep.h"
 #include "block/block.h"
+#include "block/thread-pool.h"
 #include "qemu/main-loop.h"
 #include "qemu/rcu.h"
 #include "qemu/rcu_queue.h"
diff --git a/util/async.c b/util/async.c
index 2ea1172f3e..554ba70cca 100644
--- a/util/async.c
+++ b/util/async.c
@@ -563,6 +563,9 @@ AioContext *aio_context_new(Error **errp)
 
     ctx->aio_max_batch = 0;
 
+    ctx->thread_pool_min = 0;
+    ctx->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
+
     return ctx;
 fail:
     g_source_destroy(&ctx->source);
@@ -696,3 +699,20 @@ void qemu_set_current_aio_context(AioContext *ctx)
     assert(!get_my_aiocontext());
     set_my_aiocontext(ctx);
 }
+
+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
+                                        int64_t max, Error **errp)
+{
+
+    if (min > max || !max || min > INT_MAX || max > INT_MAX) {
+        error_setg(errp, "bad thread-pool-min/thread-pool-max values");
+        return;
+    }
+
+    ctx->thread_pool_min = min;
+    ctx->thread_pool_max = max;
+
+    if (ctx->thread_pool) {
+        thread_pool_update_params(ctx->thread_pool, ctx);
+    }
+}
diff --git a/util/main-loop.c b/util/main-loop.c
index e30f034815..f00a25451b 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -30,6 +30,7 @@
 #include "sysemu/replay.h"
 #include "qemu/main-loop.h"
 #include "block/aio.h"
+#include "block/thread-pool.h"
 #include "qemu/error-report.h"
 #include "qemu/queue.h"
 #include "qemu/compiler.h"
@@ -187,12 +188,20 @@ int qemu_init_main_loop(Error **errp)
 
 static void main_loop_update_params(EventLoopBase *base, Error **errp)
 {
+    ERRP_GUARD();
+
     if (!qemu_aio_context) {
         error_setg(errp, "qemu aio context not ready");
         return;
     }
 
     aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+    if (*errp) {
+        return;
+    }
+
+    aio_context_set_thread_pool_params(qemu_aio_context, base->thread_pool_min,
+                                       base->thread_pool_max, errp);
 }
 
 MainLoop *mloop;
diff --git a/util/thread-pool.c b/util/thread-pool.c
index d763cea505..196835b4d3 100644
--- a/util/thread-pool.c
+++ b/util/thread-pool.c
@@ -58,7 +58,6 @@ struct ThreadPool {
     QemuMutex lock;
     QemuCond worker_stopped;
     QemuSemaphore sem;
-    int max_threads;
     QEMUBH *new_thread_bh;
 
     /* The following variables are only accessed from one AioContext. */
@@ -71,8 +70,27 @@ struct ThreadPool {
     int new_threads;     /* backlog of threads we need to create */
     int pending_threads; /* threads created but not running yet */
     bool stopping;
+    int min_threads;
+    int max_threads;
 };
 
+static inline bool back_to_sleep(ThreadPool *pool, int ret)
+{
+    /*
+     * The semaphore timed out, we should exit the loop except when:
+     *  - There is work to do, we raced with the signal.
+     *  - The max threads threshold just changed, we raced with the signal.
+     *  - The thread pool forces a minimum number of readily available threads.
+     */
+    if (ret == -1 && (!QTAILQ_EMPTY(&pool->request_list) ||
+            pool->cur_threads > pool->max_threads ||
+            pool->cur_threads <= pool->min_threads)) {
+            return true;
+    }
+
+    return false;
+}
+
 static void *worker_thread(void *opaque)
 {
     ThreadPool *pool = opaque;
@@ -91,8 +109,9 @@ static void *worker_thread(void *opaque)
             ret = qemu_sem_timedwait(&pool->sem, 10000);
             qemu_mutex_lock(&pool->lock);
             pool->idle_threads--;
-        } while (ret == -1 && !QTAILQ_EMPTY(&pool->request_list));
-        if (ret == -1 || pool->stopping) {
+        } while (back_to_sleep(pool, ret));
+        if (ret == -1 || pool->stopping ||
+            pool->cur_threads > pool->max_threads) {
             break;
         }
 
@@ -294,6 +313,33 @@ void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg)
     thread_pool_submit_aio(pool, func, arg, NULL, NULL);
 }
 
+void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
+{
+    qemu_mutex_lock(&pool->lock);
+
+    pool->min_threads = ctx->thread_pool_min;
+    pool->max_threads = ctx->thread_pool_max;
+
+    /*
+     * We either have to:
+     *  - Increase the number available of threads until over the min_threads
+     *    threshold.
+     *  - Decrease the number of available threads until under the max_threads
+     *    threshold.
+     *  - Do nothing. The current number of threads fall in between the min and
+     *    max thresholds. We'll let the pool manage itself.
+     */
+    for (int i = pool->cur_threads; i < pool->min_threads; i++) {
+        spawn_thread(pool);
+    }
+
+    for (int i = pool->cur_threads; i > pool->max_threads; i--) {
+        qemu_sem_post(&pool->sem);
+    }
+
+    qemu_mutex_unlock(&pool->lock);
+}
+
 static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
 {
     if (!ctx) {
@@ -306,11 +352,12 @@ static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
     qemu_mutex_init(&pool->lock);
     qemu_cond_init(&pool->worker_stopped);
     qemu_sem_init(&pool->sem, 0);
-    pool->max_threads = 64;
     pool->new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
 
     QLIST_INIT(&pool->head);
     QTAILQ_INIT(&pool->request_list);
+
+    thread_pool_update_params(pool, ctx);
 }
 
 ThreadPool *thread_pool_new(AioContext *ctx)
-- 
2.35.1



^ permalink raw reply related	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2022-05-05  8:42 [PULL 0/3] Block patches Stefan Hajnoczi
                   ` (2 preceding siblings ...)
  2022-05-05  8:42 ` [PULL 3/3] util/event-loop-base: Introduce options to set the thread pool size Stefan Hajnoczi
@ 2022-05-05 16:29 ` Richard Henderson
  2022-05-09 10:34   ` Stefan Hajnoczi
  3 siblings, 1 reply; 29+ messages in thread
From: Richard Henderson @ 2022-05-05 16:29 UTC (permalink / raw)
  To: Stefan Hajnoczi, qemu-devel; +Cc: qemu-block

On 5/5/22 03:42, Stefan Hajnoczi wrote:
> The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:
> 
>    Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> 
> for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:
> 
>    util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)
> 
> ----------------------------------------------------------------
> Pull request
> 
> Add new thread-pool-min/thread-pool-max parameters to control the thread pool
> used for async I/O.
> 
> ----------------------------------------------------------------
> 
> Nicolas Saenz Julienne (3):
>    Introduce event-loop-base abstract class
>    util/main-loop: Introduce the main loop into QOM
>    util/event-loop-base: Introduce options to set the thread pool size
> 
>   qapi/qom.json                    |  43 ++++++++--
>   meson.build                      |  26 +++---
>   include/block/aio.h              |  10 +++
>   include/block/thread-pool.h      |   3 +
>   include/qemu/main-loop.h         |  10 +++
>   include/sysemu/event-loop-base.h |  41 +++++++++
>   include/sysemu/iothread.h        |   6 +-
>   event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
>   iothread.c                       |  68 +++++----------
>   util/aio-posix.c                 |   1 +
>   util/async.c                     |  20 +++++
>   util/main-loop.c                 |  65 ++++++++++++++
>   util/thread-pool.c               |  55 +++++++++++-
>   13 files changed, 419 insertions(+), 69 deletions(-)
>   create mode 100644 include/sysemu/event-loop-base.h
>   create mode 100644 event-loop-base.c
> 

This appears to introduce a new error on msys2-64bit:


14/85 qemu:unit / test-aio                                          ERROR           2.14s 
   exit status 3
 >>> MALLOC_PERTURB_=82 G_TEST_SRCDIR=C:/GitLab-Runner/builds/qemu-project/qemu/tests/unit 
G_TEST_BUILDDIR=C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit 
C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit/test-aio.exe --tap -k
------------------------------------- 8< -------------------------------------
stderr:
(test program exited with status code 3)

https://gitlab.com/qemu-project/qemu/-/jobs/2418935125

Are you in a position to test this yourself locally?


r~


^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2022-05-05 16:29 ` [PULL 0/3] Block patches Richard Henderson
@ 2022-05-09 10:34   ` Stefan Hajnoczi
  2022-05-09 12:45     ` Stefan Hajnoczi
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-05-09 10:34 UTC (permalink / raw)
  To: Richard Henderson, Nicolas Saenz Julienne
  Cc: Stefan Hajnoczi, qemu-devel, qemu block

On Thu, 5 May 2022 at 17:43, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 5/5/22 03:42, Stefan Hajnoczi wrote:
> > The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:
> >
> >    Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)
> >
> > are available in the Git repository at:
> >
> >    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> >
> > for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:
> >
> >    util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)
> >
> > ----------------------------------------------------------------
> > Pull request
> >
> > Add new thread-pool-min/thread-pool-max parameters to control the thread pool
> > used for async I/O.
> >
> > ----------------------------------------------------------------
> >
> > Nicolas Saenz Julienne (3):
> >    Introduce event-loop-base abstract class
> >    util/main-loop: Introduce the main loop into QOM
> >    util/event-loop-base: Introduce options to set the thread pool size
> >
> >   qapi/qom.json                    |  43 ++++++++--
> >   meson.build                      |  26 +++---
> >   include/block/aio.h              |  10 +++
> >   include/block/thread-pool.h      |   3 +
> >   include/qemu/main-loop.h         |  10 +++
> >   include/sysemu/event-loop-base.h |  41 +++++++++
> >   include/sysemu/iothread.h        |   6 +-
> >   event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
> >   iothread.c                       |  68 +++++----------
> >   util/aio-posix.c                 |   1 +
> >   util/async.c                     |  20 +++++
> >   util/main-loop.c                 |  65 ++++++++++++++
> >   util/thread-pool.c               |  55 +++++++++++-
> >   13 files changed, 419 insertions(+), 69 deletions(-)
> >   create mode 100644 include/sysemu/event-loop-base.h
> >   create mode 100644 event-loop-base.c
> >
>
> This appears to introduce a new error on msys2-64bit:
>
>
> 14/85 qemu:unit / test-aio                                          ERROR           2.14s
>    exit status 3
>  >>> MALLOC_PERTURB_=82 G_TEST_SRCDIR=C:/GitLab-Runner/builds/qemu-project/qemu/tests/unit
> G_TEST_BUILDDIR=C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit
> C:/GitLab-Runner/builds/qemu-project/qemu/build/tests/unit/test-aio.exe --tap -k
> ------------------------------------- 8< -------------------------------------
> stderr:
> (test program exited with status code 3)
>
> https://gitlab.com/qemu-project/qemu/-/jobs/2418935125
>
> Are you in a position to test this yourself locally?

I haven't reproduced it yet but will dig a bit more.

test-aio.exe succeeds under Wine:
# random seed: R02S572ad8b9cfeac92bb23a64678114e66d
1..29
# Start of aio tests
ok 1 /aio/acquire
ok 2 /aio/external-client
# Start of bh tests
ok 3 /aio/bh/schedule
ok 4 /aio/bh/schedule10
ok 5 /aio/bh/cancel
ok 6 /aio/bh/delete
ok 7 /aio/bh/flush
# Start of callback-delete tests
ok 8 /aio/bh/callback-delete/one
ok 9 /aio/bh/callback-delete/many
# End of callback-delete tests
# End of bh tests
# Start of event tests
ok 10 /aio/event/add-remove
ok 11 /aio/event/wait
ok 12 /aio/event/flush
# Start of wait tests
ok 13 /aio/event/wait/no-flush-cb
# End of wait tests
# End of event tests
# Start of timer tests
ok 14 /aio/timer/schedule
# End of timer tests
# Start of coroutine tests
ok 15 /aio/coroutine/queue-chaining
ok 16 /aio/coroutine/worker-thread-co-enter
# End of coroutine tests
# End of aio tests
# Start of aio-gsource tests
ok 17 /aio-gsource/flush
# Start of bh tests
ok 18 /aio-gsource/bh/schedule
ok 19 /aio-gsource/bh/schedule10
ok 20 /aio-gsource/bh/cancel
ok 21 /aio-gsource/bh/delete
ok 22 /aio-gsource/bh/flush
# Start of callback-delete tests
ok 23 /aio-gsource/bh/callback-delete/one
ok 24 /aio-gsource/bh/callback-delete/many
# End of callback-delete tests
# End of bh tests
# Start of event tests
ok 25 /aio-gsource/event/add-remove
ok 26 /aio-gsource/event/wait
ok 27 /aio-gsource/event/flush
# Start of wait tests
ok 28 /aio-gsource/event/wait/no-flush-cb
# End of wait tests
# End of event tests
# Start of timer tests
ok 29 /aio-gsource/timer/schedule
# End of timer tests
# End of aio-gsource tests

Stefan


^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2022-05-09 10:34   ` Stefan Hajnoczi
@ 2022-05-09 12:45     ` Stefan Hajnoczi
  0 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-05-09 12:45 UTC (permalink / raw)
  To: Richard Henderson, Nicolas Saenz Julienne
  Cc: Stefan Hajnoczi, qemu-devel, qemu block

It looks like a race condition in the test case. The failed test case
(tests/unit/test-aio.c:test_timer_schedule()) assumes the process will
be scheduled in a timely manner. A timer is scheduled for 750 ms and
the test waits for 1 second before running the event loop. At that
point a callback schedules the timer another 750 ms in the future. The
problem is that the exact sequence of event loop (aio_poll())
invocations can be unexpected on a heavily loaded CI runner leading to
spurious test failures.

I re-ran the job and it passed:
https://gitlab.com/qemu-project/qemu/-/jobs/2430529496

I also ran it locally under Wine and didn't see a failure.

FWIW I'm about to send another block pull request and it will include
these patches again. Feel free to wait for the next pull request.

Stefan


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2022-10-31 18:51 Stefan Hajnoczi
  2022-11-01 20:04 ` Stefan Hajnoczi
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-10-31 18:51 UTC (permalink / raw)
  To: qemu-devel
  Cc: Eric Blake, Hanna Reitz, Stefan Hajnoczi, Markus Armbruster,
	qemu-block, Kevin Wolf

The following changes since commit 7208429223963c405c62fa2611398f1aa8033593:

  Merge tag 'mem-2022-10-28' of https://github.com/davidhildenbrand/qemu into staging (2022-10-30 18:31:59 -0400)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 6c32fc0df9cd901add75618c831fb26a9eb742cb:

  block/blkio: Make driver nvme-io_uring take a "path" instead of a "filename" (2022-10-31 14:35:14 -0400)

----------------------------------------------------------------
Pull request

Note that we're still discussing "block/blkio: Make driver nvme-io_uring take a
"path" instead of a "filename"". I have sent the pull request now so everything
is ready for the soft freeze tomorrow if we decide to go ahead with the patch.

----------------------------------------------------------------

Alberto Faria (3):
  block/blkio: Add virtio-blk-vfio-pci BlockDriver
  block/blkio: Tolerate device size changes
  block/blkio: Make driver nvme-io_uring take a "path" instead of a
    "filename"

 qapi/block-core.json | 22 +++++++++++++++++++--
 block/blkio.c        | 47 ++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 63 insertions(+), 6 deletions(-)

-- 
2.38.1



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2022-10-31 18:51 Stefan Hajnoczi
@ 2022-11-01 20:04 ` Stefan Hajnoczi
  0 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2022-11-01 20:04 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: qemu-devel, Eric Blake, Hanna Reitz, Stefan Hajnoczi,
	Markus Armbruster, qemu-block, Kevin Wolf

[-- Attachment #1: Type: text/plain, Size: 115 bytes --]

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/7.2 for any user-visible changes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2023-01-23 20:04 Stefan Hajnoczi
  2023-01-24 12:24 ` Peter Maydell
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2023-01-23 20:04 UTC (permalink / raw)
  To: qemu-devel
  Cc: Hanna Reitz, Michael S. Tsirkin, Kevin Wolf, Stefan Hajnoczi,
	Fam Zheng, qemu-block, Paolo Bonzini

The following changes since commit 00b1faea41d283e931256aa78aa975a369ec3ae6:

  Merge tag 'pull-target-arm-20230123' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2023-01-23 13:40:28 +0000)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 4f01a9bb0461e8c11ee0c94d90a504cb7d580a85:

  block/blkio: Fix inclusion of required headers (2023-01-23 15:02:07 -0500)

----------------------------------------------------------------
Pull request

----------------------------------------------------------------

Chao Gao (1):
  util/aio: Defer disabling poll mode as long as possible

Peter Krempa (1):
  block/blkio: Fix inclusion of required headers

Stefan Hajnoczi (1):
  virtio-blk: simplify virtio_blk_dma_restart_cb()

 include/hw/virtio/virtio-blk.h  |  2 --
 block/blkio.c                   |  2 ++
 hw/block/dataplane/virtio-blk.c | 17 +++++-------
 hw/block/virtio-blk.c           | 46 ++++++++++++++-------------------
 util/aio-posix.c                | 21 ++++++++++-----
 5 files changed, 43 insertions(+), 45 deletions(-)

-- 
2.39.0



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2023-01-23 20:04 Stefan Hajnoczi
@ 2023-01-24 12:24 ` Peter Maydell
  0 siblings, 0 replies; 29+ messages in thread
From: Peter Maydell @ 2023-01-24 12:24 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: qemu-devel, Hanna Reitz, Michael S. Tsirkin, Kevin Wolf,
	Fam Zheng, qemu-block, Paolo Bonzini

On Mon, 23 Jan 2023 at 20:04, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit 00b1faea41d283e931256aa78aa975a369ec3ae6:
>
>   Merge tag 'pull-target-arm-20230123' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2023-01-23 13:40:28 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 4f01a9bb0461e8c11ee0c94d90a504cb7d580a85:
>
>   block/blkio: Fix inclusion of required headers (2023-01-23 15:02:07 -0500)
>
> ----------------------------------------------------------------
> Pull request
>
> ----------------------------------------------------------------
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.0
for any user-visible changes.

-- PMM


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2023-11-06 17:10 Hanna Czenczek
  2023-11-07  3:03 ` Stefan Hajnoczi
  0 siblings, 1 reply; 29+ messages in thread
From: Hanna Czenczek @ 2023-11-06 17:10 UTC (permalink / raw)
  To: qemu-block; +Cc: qemu-devel, Hanna Czenczek

The following changes since commit 3e01f1147a16ca566694b97eafc941d62fa1e8d8:

  Merge tag 'pull-sp-20231105' of https://gitlab.com/rth7680/qemu into staging (2023-11-06 09:34:22 +0800)

are available in the Git repository at:

  https://gitlab.com/hreitz/qemu.git tags/pull-block-2023-11-06

for you to fetch changes up to ad4feaca61d76fecad784e6d5e7bae40d0411c46:

  file-posix: fix over-writing of returning zone_append offset (2023-11-06 16:15:07 +0100)

----------------------------------------------------------------
Block patches:
- One patch to make qcow2's discard-no-unref option do better what it is
  supposed to do (i.e. prevent fragmentation)
- Two fixes for zoned requests

----------------------------------------------------------------
Jean-Louis Dupond (1):
  qcow2: keep reference on zeroize with discard-no-unref enabled

Naohiro Aota (1):
  file-posix: fix over-writing of returning zone_append offset

Sam Li (1):
  block/file-posix: fix update_zones_wp() caller

 qapi/block-core.json  | 24 ++++++++++++++----------
 block/file-posix.c    | 23 ++++++++++++-----------
 block/qcow2-cluster.c | 22 ++++++++++++++++++----
 qemu-options.hx       | 10 +++++++---
 4 files changed, 51 insertions(+), 28 deletions(-)

-- 
2.41.0



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2023-11-06 17:10 Hanna Czenczek
@ 2023-11-07  3:03 ` Stefan Hajnoczi
  0 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2023-11-07  3:03 UTC (permalink / raw)
  To: Hanna Czenczek; +Cc: qemu-block, qemu-devel, Hanna Czenczek

[-- Attachment #1: Type: text/plain, Size: 115 bytes --]

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.2 for any user-visible changes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 29+ messages in thread

* [PULL 0/3] Block patches
@ 2025-05-08 14:22 Stefan Hajnoczi
  2025-05-10 18:36 ` Stefan Hajnoczi
  0 siblings, 1 reply; 29+ messages in thread
From: Stefan Hajnoczi @ 2025-05-08 14:22 UTC (permalink / raw)
  To: qemu-devel
  Cc: Stefan Hajnoczi, Kevin Wolf, Fam Zheng, Hanna Reitz, qemu-block,
	qemu-s390x, Thomas Huth, Philippe Mathieu-Daudé

The following changes since commit 57b6f8d07f1478375f85a4593a207e936c63ff59:

  Merge tag 'pull-target-arm-20250506' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2025-05-07 14:28:20 -0400)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 624379be3a8136db420ec3a3fee36fe91e9f789e:

  block/nvme: Use host PCI MMIO API (2025-05-08 10:21:10 -0400)

----------------------------------------------------------------
Pull request

Farhan Ali's s390x host PCI support for the block/nvme.c driver.

----------------------------------------------------------------

Farhan Ali (3):
  util: Add functions for s390x mmio read/write
  include: Add a header to define host PCI MMIO functions
  block/nvme: Use host PCI MMIO API

 include/qemu/host-pci-mmio.h  | 136 +++++++++++++++++++++++++++++++
 include/qemu/s390x_pci_mmio.h |  24 ++++++
 block/nvme.c                  |  41 +++++-----
 util/s390x_pci_mmio.c         | 146 ++++++++++++++++++++++++++++++++++
 util/meson.build              |   2 +
 5 files changed, 331 insertions(+), 18 deletions(-)
 create mode 100644 include/qemu/host-pci-mmio.h
 create mode 100644 include/qemu/s390x_pci_mmio.h
 create mode 100644 util/s390x_pci_mmio.c

-- 
2.49.0



^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [PULL 0/3] Block patches
  2025-05-08 14:22 Stefan Hajnoczi
@ 2025-05-10 18:36 ` Stefan Hajnoczi
  0 siblings, 0 replies; 29+ messages in thread
From: Stefan Hajnoczi @ 2025-05-10 18:36 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: qemu-devel, Stefan Hajnoczi, Kevin Wolf, Fam Zheng, Hanna Reitz,
	qemu-block, qemu-s390x, Thomas Huth, Philippe Mathieu-Daudé

[-- Attachment #1: Type: text/plain, Size: 116 bytes --]

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/10.1 for any user-visible changes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 29+ messages in thread

end of thread, other threads:[~2025-05-10 18:38 UTC | newest]

Thread overview: 29+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-05-05  8:42 [PULL 0/3] Block patches Stefan Hajnoczi
2022-05-05  8:42 ` [PULL 1/3] Introduce event-loop-base abstract class Stefan Hajnoczi
2022-05-05  8:42 ` [PULL 2/3] util/main-loop: Introduce the main loop into QOM Stefan Hajnoczi
2022-05-05  8:42 ` [PULL 3/3] util/event-loop-base: Introduce options to set the thread pool size Stefan Hajnoczi
2022-05-05 16:29 ` [PULL 0/3] Block patches Richard Henderson
2022-05-09 10:34   ` Stefan Hajnoczi
2022-05-09 12:45     ` Stefan Hajnoczi
  -- strict thread matches above, loose matches on Subject: below --
2025-05-08 14:22 Stefan Hajnoczi
2025-05-10 18:36 ` Stefan Hajnoczi
2023-11-06 17:10 Hanna Czenczek
2023-11-07  3:03 ` Stefan Hajnoczi
2023-01-23 20:04 Stefan Hajnoczi
2023-01-24 12:24 ` Peter Maydell
2022-10-31 18:51 Stefan Hajnoczi
2022-11-01 20:04 ` Stefan Hajnoczi
2022-04-25  8:48 Stefan Hajnoczi
2022-04-25 20:35 ` Richard Henderson
2022-02-14 17:15 Stefan Hajnoczi
2022-02-15 22:40 ` Peter Maydell
2021-12-22 16:52 Hanna Reitz
2021-12-23  2:33 ` Richard Henderson
2021-11-23 15:59 Hanna Reitz
2021-11-23 17:58 ` Richard Henderson
2020-08-17 15:16 Stefan Hajnoczi
2020-08-21 19:08 ` Peter Maydell
2020-03-06 14:23 Stefan Hajnoczi
2020-03-06 17:02 ` Peter Maydell
2019-12-20 10:25 Stefan Hajnoczi
2020-01-03 18:50 ` Peter Maydell

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).