* [PULL 0/8] Block layer patches
@ 2021-01-27 19:57 Kevin Wolf
2021-01-28 13:58 ` Peter Maydell
2021-01-28 18:19 ` Peter Maydell
0 siblings, 2 replies; 27+ messages in thread
From: Kevin Wolf @ 2021-01-27 19:57 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, peter.maydell, qemu-devel
The following changes since commit bf159f0bdc7b8e7aa8342dedb3829ca744c1b612:
Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream' into staging (2021-01-27 17:40:25 +0000)
are available in the Git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to a44be0334beae3a9affb4a3a92cc6852993d7a84:
iotests: rename and move 169 and 199 tests (2021-01-27 20:53:14 +0100)
----------------------------------------------------------------
Block layer patches:
- Fix crash on write to read-only devices
- iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
non-numeric test case names
----------------------------------------------------------------
Kevin Wolf (2):
block: Separate blk_is_writable() and blk_supports_write_perm()
virtio-scsi-test: Test writing to scsi-cd device
Vladimir Sementsov-Ogievskiy (6):
iotests: 146: drop extra whitespaces from .out file
iotests: add findtests.py
iotests: add testenv.py
iotests: add testrunner.py
iotests: rewrite check into python
iotests: rename and move 169 and 199 tests
docs/devel/testing.rst | 50 +-
include/sysemu/block-backend.h | 3 +-
block/block-backend.c | 19 +-
hw/block/dataplane/xen-block.c | 2 +-
hw/block/fdc.c | 9 +-
hw/block/m25p80.c | 6 +-
hw/block/nand.c | 2 +-
hw/block/nvme-ns.c | 7 +-
hw/block/onenand.c | 2 +-
hw/block/pflash_cfi01.c | 2 +-
hw/block/pflash_cfi02.c | 2 +-
hw/block/swim.c | 6 +-
hw/block/virtio-blk.c | 6 +-
hw/block/xen-block.c | 2 +-
hw/ide/core.c | 2 +-
hw/misc/sifive_u_otp.c | 2 +-
hw/ppc/pnv_pnor.c | 2 +-
hw/scsi/scsi-disk.c | 10 +-
hw/scsi/scsi-generic.c | 4 +-
hw/sd/sd.c | 6 +-
hw/usb/dev-storage.c | 4 +-
tests/qtest/virtio-scsi-test.c | 39 +
tests/qemu-iotests/findtests.py | 159 +++
tests/qemu-iotests/iotests.py | 8 +
tests/qemu-iotests/testenv.py | 281 +++++
tests/qemu-iotests/testrunner.py | 367 +++++++
Makefile | 1 -
tests/check-block.sh | 3 +-
tests/qemu-iotests/146.out | 780 +++++++-------
tests/qemu-iotests/check | 1095 +++-----------------
tests/qemu-iotests/common.env.in | 3 -
tests/qemu-iotests/group | 323 ------
tests/qemu-iotests/meson.build | 3 -
.../{199 => tests/migrate-bitmaps-postcopy-test} | 0
.../migrate-bitmaps-postcopy-test.out} | 0
.../{169 => tests/migrate-bitmaps-test} | 0
.../{169.out => tests/migrate-bitmaps-test.out} | 0
37 files changed, 1481 insertions(+), 1729 deletions(-)
create mode 100644 tests/qemu-iotests/findtests.py
create mode 100644 tests/qemu-iotests/testenv.py
create mode 100644 tests/qemu-iotests/testrunner.py
delete mode 100644 tests/qemu-iotests/common.env.in
delete mode 100644 tests/qemu-iotests/group
rename tests/qemu-iotests/{199 => tests/migrate-bitmaps-postcopy-test} (100%)
rename tests/qemu-iotests/{199.out => tests/migrate-bitmaps-postcopy-test.out} (100%)
rename tests/qemu-iotests/{169 => tests/migrate-bitmaps-test} (100%)
rename tests/qemu-iotests/{169.out => tests/migrate-bitmaps-test.out} (100%)
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2021-01-27 19:57 Kevin Wolf
@ 2021-01-28 13:58 ` Peter Maydell
2021-01-28 18:19 ` Peter Maydell
1 sibling, 0 replies; 27+ messages in thread
From: Peter Maydell @ 2021-01-28 13:58 UTC (permalink / raw)
To: Kevin Wolf; +Cc: QEMU Developers, Qemu-block
On Wed, 27 Jan 2021 at 19:58, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit bf159f0bdc7b8e7aa8342dedb3829ca744c1b612:
>
> Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream' into staging (2021-01-27 17:40:25 +0000)
>
> are available in the Git repository at:
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to a44be0334beae3a9affb4a3a92cc6852993d7a84:
>
> iotests: rename and move 169 and 199 tests (2021-01-27 20:53:14 +0100)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - Fix crash on write to read-only devices
> - iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
> non-numeric test case names
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/6.0
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2021-01-27 19:57 Kevin Wolf
2021-01-28 13:58 ` Peter Maydell
@ 2021-01-28 18:19 ` Peter Maydell
2021-01-28 20:13 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 27+ messages in thread
From: Peter Maydell @ 2021-01-28 18:19 UTC (permalink / raw)
To: Kevin Wolf; +Cc: QEMU Developers, Qemu-block
On Wed, 27 Jan 2021 at 19:58, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit bf159f0bdc7b8e7aa8342dedb3829ca744c1b612:
>
> Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream' into staging (2021-01-27 17:40:25 +0000)
>
> are available in the Git repository at:
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to a44be0334beae3a9affb4a3a92cc6852993d7a84:
>
> iotests: rename and move 169 and 199 tests (2021-01-27 20:53:14 +0100)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - Fix crash on write to read-only devices
> - iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
> non-numeric test case names
>
> ----------------------------------------------------------------
I somehow failed to notice before applying this, but this
breaks 'make check' on the netbsd and freebsd VMs.
As usual the build log is pretty opaque, but this looks like the
probable culprit:
env: python3: No such file or directory
gmake: *** [/home/qemu/qemu-test.nU2bcG/src/tests/Makefile.include:144:
check-block] Error 1
The python in the netbsd VM is /usr/pkg/bin/python3.7. Something
seems to be ignoring the --python= passed into our configure
and assuming that "python3" is always a valid executable.
thanks
-- PMM
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2021-01-28 18:19 ` Peter Maydell
@ 2021-01-28 20:13 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 27+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-01-28 20:13 UTC (permalink / raw)
To: Peter Maydell, Kevin Wolf; +Cc: QEMU Developers, Qemu-block
28.01.2021 21:19, Peter Maydell wrote:
> On Wed, 27 Jan 2021 at 19:58, Kevin Wolf <kwolf@redhat.com> wrote:
>>
>> The following changes since commit bf159f0bdc7b8e7aa8342dedb3829ca744c1b612:
>>
>> Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream' into staging (2021-01-27 17:40:25 +0000)
>>
>> are available in the Git repository at:
>>
>> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>>
>> for you to fetch changes up to a44be0334beae3a9affb4a3a92cc6852993d7a84:
>>
>> iotests: rename and move 169 and 199 tests (2021-01-27 20:53:14 +0100)
>>
>> ----------------------------------------------------------------
>> Block layer patches:
>>
>> - Fix crash on write to read-only devices
>> - iotests: Rewrite 'check' in Python, get rid of 'groups' and allow
>> non-numeric test case names
>>
>> ----------------------------------------------------------------
>
> I somehow failed to notice before applying this, but this
> breaks 'make check' on the netbsd and freebsd VMs.
>
> As usual the build log is pretty opaque, but this looks like the
> probable culprit:
>
> env: python3: No such file or directory
> gmake: *** [/home/qemu/qemu-test.nU2bcG/src/tests/Makefile.include:144:
> check-block] Error 1
>
> The python in the netbsd VM is /usr/pkg/bin/python3.7. Something
> seems to be ignoring the --python= passed into our configure
> and assuming that "python3" is always a valid executable.
>
Seems, that's shows that the fact that "PYTHON" variable in check-block.sh works for me doesn't mean that it works everywhere.. My fault, I look closer tomorrow.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PULL 0/8] Block layer patches
@ 2024-06-11 17:36 Kevin Wolf
2024-06-13 14:51 ` Richard Henderson
0 siblings, 1 reply; 27+ messages in thread
From: Kevin Wolf @ 2024-06-11 17:36 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit 80e8f0602168f451a93e71cbb1d59e93d745e62e:
Merge tag 'bsd-user-misc-2024q2-pull-request' of gitlab.com:bsdimp/qemu into staging (2024-06-09 11:21:55 -0700)
are available in the Git repository at:
https://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 3ab0f063e58ed9224237d69c4211ca83335164c4:
crypto/block: drop qcrypto_block_open() n_threads argument (2024-06-10 11:05:43 +0200)
----------------------------------------------------------------
Block layer patches
- crypto: Fix crash when used with multiqueue devices
- linux-aio: add IO_CMD_FDSYNC command support
- copy-before-write: Avoid integer overflows for timeout > 4s
- Fix crash with QMP block_resize and iothreads
- qemu-io: add cvtnum() error handling for zone commands
- Code cleanup
----------------------------------------------------------------
Denis V. Lunev via (1):
block: drop force_dup parameter of raw_reconfigure_getfd()
Fiona Ebner (1):
block/copy-before-write: use uint64_t for timeout in nanoseconds
Prasad J Pandit (1):
linux-aio: add IO_CMD_FDSYNC command support
Stefan Hajnoczi (5):
Revert "monitor: use aio_co_reschedule_self()"
aio: warn about iohandler_ctx special casing
qemu-io: add cvtnum() error handling for zone commands
block/crypto: create ciphers on demand
crypto/block: drop qcrypto_block_open() n_threads argument
crypto/blockpriv.h | 13 +++--
include/block/aio.h | 6 +++
include/block/raw-aio.h | 1 +
include/crypto/block.h | 2 -
block/copy-before-write.c | 2 +-
block/crypto.c | 1 -
block/file-posix.c | 17 ++++--
block/linux-aio.c | 21 +++++++-
block/qcow.c | 2 +-
block/qcow2.c | 5 +-
crypto/block-luks.c | 4 +-
crypto/block-qcow.c | 8 ++-
crypto/block.c | 114 ++++++++++++++++++++++++-----------------
qapi/qmp-dispatch.c | 7 ++-
qemu-io-cmds.c | 48 ++++++++++++++++-
tests/unit/test-crypto-block.c | 4 --
16 files changed, 176 insertions(+), 79 deletions(-)
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2024-06-11 17:36 Kevin Wolf
@ 2024-06-13 14:51 ` Richard Henderson
0 siblings, 0 replies; 27+ messages in thread
From: Richard Henderson @ 2024-06-13 14:51 UTC (permalink / raw)
To: Kevin Wolf, qemu-block; +Cc: qemu-devel
On 6/11/24 10:36, Kevin Wolf wrote:
> The following changes since commit 80e8f0602168f451a93e71cbb1d59e93d745e62e:
>
> Merge tag 'bsd-user-misc-2024q2-pull-request' of gitlab.com:bsdimp/qemu into staging (2024-06-09 11:21:55 -0700)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 3ab0f063e58ed9224237d69c4211ca83335164c4:
>
> crypto/block: drop qcrypto_block_open() n_threads argument (2024-06-10 11:05:43 +0200)
>
> ----------------------------------------------------------------
> Block layer patches
>
> - crypto: Fix crash when used with multiqueue devices
> - linux-aio: add IO_CMD_FDSYNC command support
> - copy-before-write: Avoid integer overflows for timeout > 4s
> - Fix crash with QMP block_resize and iothreads
> - qemu-io: add cvtnum() error handling for zone commands
> - Code cleanup
Applied, thanks. Please update https://wiki.qemu.org/ChangeLog/9.1 as appropriate.
r~
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PULL 0/8] Block layer patches
@ 2024-11-14 16:56 Kevin Wolf
2024-11-15 20:16 ` Peter Maydell
0 siblings, 1 reply; 27+ messages in thread
From: Kevin Wolf @ 2024-11-14 16:56 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit f0a5a31c33a8109061c2493e475c8a2f4d022432:
Update version for v9.2.0-rc0 release (2024-11-13 21:44:45 +0000)
are available in the Git repository at:
https://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 378a645b2f6125b1bdbd1fae3e8f30452d5b5934:
vl: use qmp_device_add() in qemu_create_cli_devices() (2024-11-14 17:55:51 +0100)
----------------------------------------------------------------
Block layer patches
- Fix qmp_device_add() to not throw non-scalar options away (fixes
iothread-vq-mapping being silently ignored in device_add)
- iotests: Fix mypy failure
- parallels: Avoid potential integer overflow
- Fix crash in migration_is_running()
----------------------------------------------------------------
Dmitry Frolov (1):
parallels: fix possible int overflow
John Snow (4):
iotests: reflow ReproducibleTestRunner arguments
iotests: correct resultclass type in ReproducibleTestRunner
python: disable too-many-positional-arguments warning
python: silence pylint raising-non-exception error
Peter Xu (1):
migration: Check current_migration in migration_is_running()
Stefan Hajnoczi (2):
qdev-monitor: avoid QemuOpts in QMP device_add
vl: use qmp_device_add() in qemu_create_cli_devices()
block/parallels.c | 4 ++--
migration/migration.c | 4 ++++
system/qdev-monitor.c | 42 ++++++++++++++++++++++++++++--------------
system/vl.c | 14 ++++----------
python/scripts/mkvenv.py | 3 +++
tests/qemu-iotests/iotests.py | 11 +++++++----
python/setup.cfg | 1 +
tests/qemu-iotests/pylintrc | 1 +
8 files changed, 50 insertions(+), 30 deletions(-)
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2024-11-14 16:56 Kevin Wolf
@ 2024-11-15 20:16 ` Peter Maydell
2024-11-19 11:25 ` Kevin Wolf
0 siblings, 1 reply; 27+ messages in thread
From: Peter Maydell @ 2024-11-15 20:16 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel, Stefan Hajnoczi
On Thu, 14 Nov 2024 at 16:58, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit f0a5a31c33a8109061c2493e475c8a2f4d022432:
>
> Update version for v9.2.0-rc0 release (2024-11-13 21:44:45 +0000)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 378a645b2f6125b1bdbd1fae3e8f30452d5b5934:
>
> vl: use qmp_device_add() in qemu_create_cli_devices() (2024-11-14 17:55:51 +0100)
>
> ----------------------------------------------------------------
> Block layer patches
>
> - Fix qmp_device_add() to not throw non-scalar options away (fixes
> iothread-vq-mapping being silently ignored in device_add)
> - iotests: Fix mypy failure
> - parallels: Avoid potential integer overflow
> - Fix crash in migration_is_running()
>
Hi; this seems to cause an error for the avocado test
tests/avocado/hotplug_blk.py:HotPlug.test
https://gitlab.com/qemu-project/qemu/-/jobs/8387009365
https://gitlab.com/qemu-project/qemu/-/jobs/8387009383
(12/51) tests/avocado/hotplug_blk.py:HotPlug.test: STARTED
(12/51) tests/avocado/hotplug_blk.py:HotPlug.test: ERROR: Could not
perform graceful shutdown (17.16 s)
If you dig through the build artefacts you can find the debug log:
https://qemu-project.gitlab.io/-/qemu/-/jobs/8387009383/artifacts/build/tests/results/latest/test-results/12-tests_avocado_hotplug_blk.py_HotPlug.test/debug.log
and it seems like the test sends a device_add command over
QMP and the result is that QEMU dies with an assertion failure.
The relevant device_add is
[stdlog] "execute": "device_add",
[stdlog] "arguments": {
[stdlog] "driver": "virtio-blk-pci",
[stdlog] "drive": "disk",
[stdlog] "id": "virtio-disk0",
[stdlog] "bus": "pci.1",
[stdlog] "addr": 1
[stdlog] },
[stdlog] "id": "__qmp#00002"
[stdlog] }
Avocado helpfully hides the assertion message under a rock
in a different log file:
https://qemu-project.gitlab.io/-/qemu/-/jobs/8387009383/artifacts/build/tests/results/latest/test-results/12-tests_avocado_hotplug_blk.py_HotPlug.test/7f00b63ed810.log
qemu-system-x86_64: ../qapi/qobject-input-visitor.c:143:
qobject_input_try_get_object: Assertion `removed' failed.
I'm guessing this is Stefan's patches since they touch
the device_add path.
thanks
-- PMM
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2024-11-15 20:16 ` Peter Maydell
@ 2024-11-19 11:25 ` Kevin Wolf
2024-11-19 14:41 ` Stefan Hajnoczi
0 siblings, 1 reply; 27+ messages in thread
From: Kevin Wolf @ 2024-11-19 11:25 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-block, qemu-devel, Stefan Hajnoczi
Am 15.11.2024 um 21:16 hat Peter Maydell geschrieben:
> On Thu, 14 Nov 2024 at 16:58, Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > The following changes since commit f0a5a31c33a8109061c2493e475c8a2f4d022432:
> >
> > Update version for v9.2.0-rc0 release (2024-11-13 21:44:45 +0000)
> >
> > are available in the Git repository at:
> >
> > https://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 378a645b2f6125b1bdbd1fae3e8f30452d5b5934:
> >
> > vl: use qmp_device_add() in qemu_create_cli_devices() (2024-11-14 17:55:51 +0100)
> >
> > ----------------------------------------------------------------
> > Block layer patches
> >
> > - Fix qmp_device_add() to not throw non-scalar options away (fixes
> > iothread-vq-mapping being silently ignored in device_add)
> > - iotests: Fix mypy failure
> > - parallels: Avoid potential integer overflow
> > - Fix crash in migration_is_running()
> >
>
> Hi; this seems to cause an error for the avocado test
> tests/avocado/hotplug_blk.py:HotPlug.test
>
> https://gitlab.com/qemu-project/qemu/-/jobs/8387009365
> https://gitlab.com/qemu-project/qemu/-/jobs/8387009383
>
> (12/51) tests/avocado/hotplug_blk.py:HotPlug.test: STARTED
> (12/51) tests/avocado/hotplug_blk.py:HotPlug.test: ERROR: Could not
> perform graceful shutdown (17.16 s)
>
> If you dig through the build artefacts you can find the debug log:
> https://qemu-project.gitlab.io/-/qemu/-/jobs/8387009383/artifacts/build/tests/results/latest/test-results/12-tests_avocado_hotplug_blk.py_HotPlug.test/debug.log
>
> and it seems like the test sends a device_add command over
> QMP and the result is that QEMU dies with an assertion failure.
> The relevant device_add is
>
> [stdlog] "execute": "device_add",
> [stdlog] "arguments": {
> [stdlog] "driver": "virtio-blk-pci",
> [stdlog] "drive": "disk",
> [stdlog] "id": "virtio-disk0",
> [stdlog] "bus": "pci.1",
> [stdlog] "addr": 1
> [stdlog] },
> [stdlog] "id": "__qmp#00002"
> [stdlog] }
>
> Avocado helpfully hides the assertion message under a rock
> in a different log file:
> https://qemu-project.gitlab.io/-/qemu/-/jobs/8387009383/artifacts/build/tests/results/latest/test-results/12-tests_avocado_hotplug_blk.py_HotPlug.test/7f00b63ed810.log
>
> qemu-system-x86_64: ../qapi/qobject-input-visitor.c:143:
> qobject_input_try_get_object: Assertion `removed' failed.
>
>
> I'm guessing this is Stefan's patches since they touch
> the device_add path.
Yes, this is Stefan's patches exposing a preexisting bug on a new code
path. You can already trigger the same bug on the command line with git
master:
$ ./qemu-system-x86_64 -blockdev null-co,node-name=disk -device '{ "driver": "virtio-blk-pci", "drive": "disk", "id": "virtio-disk0", "addr": 1 }'
qemu-system-x86_64: ../qapi/qobject-input-visitor.c:143: QObject *qobject_input_try_get_object(QObjectInputVisitor *, const char *, _Bool): Assertion `removed' failed.
The problem is that set_pci_devfn() visits the same field twice, which
is not allowed. Apparently the QemuOpts visitor accepts it anyway, but
the QObject one doesn't. I'll write a patch to use the proper alternate
mechanism instead, that should fix it.
Kevin
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2024-11-19 11:25 ` Kevin Wolf
@ 2024-11-19 14:41 ` Stefan Hajnoczi
0 siblings, 0 replies; 27+ messages in thread
From: Stefan Hajnoczi @ 2024-11-19 14:41 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Peter Maydell, qemu-block, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 3543 bytes --]
On Tue, Nov 19, 2024 at 12:25:26PM +0100, Kevin Wolf wrote:
> Am 15.11.2024 um 21:16 hat Peter Maydell geschrieben:
> > On Thu, 14 Nov 2024 at 16:58, Kevin Wolf <kwolf@redhat.com> wrote:
> > >
> > > The following changes since commit f0a5a31c33a8109061c2493e475c8a2f4d022432:
> > >
> > > Update version for v9.2.0-rc0 release (2024-11-13 21:44:45 +0000)
> > >
> > > are available in the Git repository at:
> > >
> > > https://repo.or.cz/qemu/kevin.git tags/for-upstream
> > >
> > > for you to fetch changes up to 378a645b2f6125b1bdbd1fae3e8f30452d5b5934:
> > >
> > > vl: use qmp_device_add() in qemu_create_cli_devices() (2024-11-14 17:55:51 +0100)
> > >
> > > ----------------------------------------------------------------
> > > Block layer patches
> > >
> > > - Fix qmp_device_add() to not throw non-scalar options away (fixes
> > > iothread-vq-mapping being silently ignored in device_add)
> > > - iotests: Fix mypy failure
> > > - parallels: Avoid potential integer overflow
> > > - Fix crash in migration_is_running()
> > >
> >
> > Hi; this seems to cause an error for the avocado test
> > tests/avocado/hotplug_blk.py:HotPlug.test
> >
> > https://gitlab.com/qemu-project/qemu/-/jobs/8387009365
> > https://gitlab.com/qemu-project/qemu/-/jobs/8387009383
> >
> > (12/51) tests/avocado/hotplug_blk.py:HotPlug.test: STARTED
> > (12/51) tests/avocado/hotplug_blk.py:HotPlug.test: ERROR: Could not
> > perform graceful shutdown (17.16 s)
> >
> > If you dig through the build artefacts you can find the debug log:
> > https://qemu-project.gitlab.io/-/qemu/-/jobs/8387009383/artifacts/build/tests/results/latest/test-results/12-tests_avocado_hotplug_blk.py_HotPlug.test/debug.log
> >
> > and it seems like the test sends a device_add command over
> > QMP and the result is that QEMU dies with an assertion failure.
> > The relevant device_add is
> >
> > [stdlog] "execute": "device_add",
> > [stdlog] "arguments": {
> > [stdlog] "driver": "virtio-blk-pci",
> > [stdlog] "drive": "disk",
> > [stdlog] "id": "virtio-disk0",
> > [stdlog] "bus": "pci.1",
> > [stdlog] "addr": 1
> > [stdlog] },
> > [stdlog] "id": "__qmp#00002"
> > [stdlog] }
> >
> > Avocado helpfully hides the assertion message under a rock
> > in a different log file:
> > https://qemu-project.gitlab.io/-/qemu/-/jobs/8387009383/artifacts/build/tests/results/latest/test-results/12-tests_avocado_hotplug_blk.py_HotPlug.test/7f00b63ed810.log
> >
> > qemu-system-x86_64: ../qapi/qobject-input-visitor.c:143:
> > qobject_input_try_get_object: Assertion `removed' failed.
> >
> >
> > I'm guessing this is Stefan's patches since they touch
> > the device_add path.
>
> Yes, this is Stefan's patches exposing a preexisting bug on a new code
> path. You can already trigger the same bug on the command line with git
> master:
>
> $ ./qemu-system-x86_64 -blockdev null-co,node-name=disk -device '{ "driver": "virtio-blk-pci", "drive": "disk", "id": "virtio-disk0", "addr": 1 }'
> qemu-system-x86_64: ../qapi/qobject-input-visitor.c:143: QObject *qobject_input_try_get_object(QObjectInputVisitor *, const char *, _Bool): Assertion `removed' failed.
>
> The problem is that set_pci_devfn() visits the same field twice, which
> is not allowed. Apparently the QemuOpts visitor accepts it anyway, but
> the QObject one doesn't. I'll write a patch to use the proper alternate
> mechanism instead, that should fix it.
Thank you, Kevin!
Stefan
>
> Kevin
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PULL 0/8] Block layer patches
@ 2026-06-08 16:51 Kevin Wolf
2026-06-08 16:52 ` [PULL 1/8] virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914) Kevin Wolf
` (8 more replies)
0 siblings, 9 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:51 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
The following changes since commit cc329c491768b2d91eb0b0984f3baa0bf805776d:
Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2026-06-08 09:30:30 -0400)
are available in the Git repository at:
https://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 296b66d05bac7eb2b86600febdbd2bd8d410f026:
qed: Don't try to flush during incoming migration (2026-06-08 17:00:47 +0200)
----------------------------------------------------------------
Block layer patches
- qcow2: Fix data loss on zero write with detect-zeroes=unmap
- qemu-img bitmap: add sub-command --remove-all
- export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
- virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914)
- qed: Don't try to flush during incoming migration
- iotests/136: Test stats-intervals with -blockdev/-device
----------------------------------------------------------------
Denis V. Lunev (1):
qemu-img: add sub-command --remove-all to 'qemu-img bitmap'
Fabiano Rosas (1):
qed: Don't try to flush during incoming migration
Fiona Ebner (3):
block/export/fuse: use struct fuse_init_in
block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
iotests: test shared mmap for fuse export
Kevin Wolf (1):
iotests/136: Test stats-intervals with -blockdev/-device
Stefan Hajnoczi (1):
virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914)
Thomas Lamprecht (1):
qcow2: Fix data loss on zero write with detect-zeroes=unmap
docs/tools/qemu-img.rst | 10 ++-
block/qcow2.h | 4 +
block/export/fuse.c | 69 +++++++++++++----
block/qcow2-cluster.c | 10 +--
block/qcow2.c | 8 +-
block/qed.c | 16 ++--
hw/block/virtio-blk.c | 8 +-
qemu-img.c | 55 ++++++++++++--
tests/qemu-iotests/046 | 23 ++++++
tests/qemu-iotests/046.out | 10 +++
tests/qemu-iotests/136 | 87 +++++++++++++++++++---
tests/qemu-iotests/136.out | 4 +-
tests/qemu-iotests/tests/fuse-mmap-shared | 103 ++++++++++++++++++++++++++
tests/qemu-iotests/tests/fuse-mmap-shared.out | 5 ++
tests/qemu-iotests/tests/qemu-img-bitmaps | 24 ++++++
tests/qemu-iotests/tests/qemu-img-bitmaps.out | 46 ++++++++++++
16 files changed, 430 insertions(+), 52 deletions(-)
create mode 100755 tests/qemu-iotests/tests/fuse-mmap-shared
create mode 100644 tests/qemu-iotests/tests/fuse-mmap-shared.out
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PULL 1/8] virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914)
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 2/8] qemu-img: add sub-command --remove-all to 'qemu-img bitmap' Kevin Wolf
` (7 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: Stefan Hajnoczi <stefanha@redhat.com>
Check that the iovec containing struct virtio_scsi_inhdr is large enough
before storing an error value there.
Feifan Qian <bea1e@proton.me> pointed out that this can be used to
corrupt heap memory when the descriptor uses an MMIO address and a
length of 1, forcing QEMU to allocate a 1-byte heap bounce buffer.
virtio_stl_p() stores 4 bytes and therefore corrupts whatever is beyond
the bounce buffer.
Fixes: CVE-2026-48914
Fixes: f34e73cd69bd ("virtio-blk: report non-zero status when failing SG_IO requests")
Reported-by: Feifan Qian <bea1e@proton.me>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20260526154957.1741622-1-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
hw/block/virtio-blk.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 9cb9f1fb2b2..6b92066aff4 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -199,10 +199,16 @@ static void virtio_blk_handle_scsi(VirtIOBlockReq *req)
/*
* The scsi inhdr is placed in the second-to-last input segment, just
- * before the regular inhdr.
+ * before the regular inhdr. VIRTIO implementations normally do not rely on
+ * the precise message framing, but legacy implementations did and so we do
+ * too for the legacy virtio-blk SCSI request type.
*
* Just put anything nonzero so that the ioctl fails in the guest.
*/
+ if (elem->in_sg[elem->in_num - 2].iov_len != sizeof(*scsi)) {
+ status = VIRTIO_BLK_S_IOERR;
+ goto fail;
+ }
scsi = (void *)elem->in_sg[elem->in_num - 2].iov_base;
virtio_stl_p(vdev, &scsi->errors, 255);
status = VIRTIO_BLK_S_UNSUPP;
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 2/8] qemu-img: add sub-command --remove-all to 'qemu-img bitmap'
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
2026-06-08 16:52 ` [PULL 1/8] virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914) Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 3/8] iotests/136: Test stats-intervals with -blockdev/-device Kevin Wolf
` (6 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: "Denis V. Lunev" <den@openvz.org>
From time to time it is needed to remove all bitmaps from the image.
Before this patch the process is not very convenient. One should
perform
qemu-img info
and parse the output to obtain all names. After that one should
sequentially call
qemu-img bitmap --remove
for each present bitmap.
The patch adds --remove-all sub-command to 'qemu-img bitmap'. The
new sub-command also composes with other bitmap actions in the same
invocation, so a common "wipe and recreate" workflow can be expressed
as
qemu-img bitmap --remove-all --add NEW FILE
instead of enumerating existing bitmaps, removing them one by one,
and only then adding the fresh one.
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Message-ID: <20260520235952.500250-1-den@openvz.org>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
docs/tools/qemu-img.rst | 10 +++-
qemu-img.c | 55 ++++++++++++++++---
tests/qemu-iotests/tests/qemu-img-bitmaps | 24 ++++++++
tests/qemu-iotests/tests/qemu-img-bitmaps.out | 46 ++++++++++++++++
4 files changed, 125 insertions(+), 10 deletions(-)
diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index 558b0eb84d6..b0c798b77a0 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -301,15 +301,19 @@ Command description:
For write tests, by default a buffer filled with zeros is written. This can be
overridden with a pattern byte specified by *PATTERN*.
-.. option:: bitmap (--merge SOURCE | --add | --remove | --clear | --enable | --disable)... [-b SOURCE_FILE [-F SOURCE_FMT]] [-g GRANULARITY] [--object OBJECTDEF] [--image-opts | -f FMT] FILENAME BITMAP
+.. option:: bitmap (--merge SOURCE | --add | --remove | --remove-all | --clear | --enable | --disable)... [-b SOURCE_FILE [-F SOURCE_FMT]] [-g GRANULARITY] [--object OBJECTDEF] [--image-opts | -f FMT] FILENAME [BITMAP]
- Perform one or more modifications of the persistent bitmap *BITMAP*
- in the disk image *FILENAME*. The various modifications are:
+ Perform one or more modifications of persistent bitmaps in the disk
+ image *FILENAME*. Most operations require *BITMAP* to be specified;
+ ``--remove-all`` operates on all bitmaps and does not take *BITMAP*.
+ The various modifications are:
``--add`` to create *BITMAP*, enabled to record future edits.
``--remove`` to remove *BITMAP*.
+ ``--remove-all`` to remove all bitmaps.
+
``--clear`` to clear *BITMAP*.
``--enable`` to change *BITMAP* to start recording future edits.
diff --git a/qemu-img.c b/qemu-img.c
index c42dd4e995e..2f63d311413 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -87,6 +87,7 @@ enum {
OPTION_FORCE = 276,
OPTION_SKIP_BROKEN = 277,
OPTION_LIMITS = 278,
+ OPTION_REMOVE_ALL = 279,
};
typedef enum OutputFormat {
@@ -5018,6 +5019,7 @@ enum ImgBitmapAct {
BITMAP_ENABLE,
BITMAP_DISABLE,
BITMAP_MERGE,
+ BITMAP_REMOVE_ALL,
};
typedef struct ImgBitmapAction {
enum ImgBitmapAct act;
@@ -5036,7 +5038,7 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
BlockDriverState *bs = NULL, *src_bs = NULL;
bool image_opts = false;
int64_t granularity = 0;
- bool add = false, merge = false;
+ bool add = false, merge = false, need_bitmap_name = false;
QSIMPLEQ_HEAD(, ImgBitmapAction) actions;
ImgBitmapAction *act, *act_next;
const char *op;
@@ -5052,6 +5054,7 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
{"add", no_argument, 0, OPTION_ADD},
{"granularity", required_argument, 0, 'g'},
{"remove", no_argument, 0, OPTION_REMOVE},
+ {"remove-all", no_argument, 0, OPTION_REMOVE_ALL},
{"clear", no_argument, 0, OPTION_CLEAR},
{"enable", no_argument, 0, OPTION_ENABLE},
{"disable", no_argument, 0, OPTION_DISABLE},
@@ -5070,9 +5073,9 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
switch (c) {
case 'h':
cmd_help(ccmd, "[-f FMT | --image-opts]\n"
-" ( --add [-g SIZE] | --remove | --clear | --enable | --disable |\n"
-" --merge SOURCE [-b SRC_FILE [-F SRC_FMT]] )..\n"
-" [--object OBJDEF] FILE BITMAP\n"
+" ( --add [-g SIZE] | --remove | --remove-all | --clear | --enable |\n"
+" --disable | --merge SOURCE [-b SRC_FILE [-F SRC_FMT]] )..\n"
+" [--object OBJDEF] FILE [BITMAP]\n"
,
" -f, --format FMT\n"
" specify FILE format explicitly (default: probing is used)\n"
@@ -5086,6 +5089,8 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
" with optional multiplier suffix (in powers of 1024)\n"
" --remove\n"
" removes BITMAP from FILE\n"
+" --remove-all\n"
+" removes all bitmaps from FILE\n"
" --clear\n"
" clears BITMAP in FILE\n"
" --enable, --disable\n"
@@ -5116,6 +5121,7 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
act->act = BITMAP_ADD;
QSIMPLEQ_INSERT_TAIL(&actions, act, next);
add = true;
+ need_bitmap_name = true;
break;
case 'g':
granularity = cvtnum("granularity", optarg, true);
@@ -5127,21 +5133,30 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
act = g_new0(ImgBitmapAction, 1);
act->act = BITMAP_REMOVE;
QSIMPLEQ_INSERT_TAIL(&actions, act, next);
+ need_bitmap_name = true;
+ break;
+ case OPTION_REMOVE_ALL:
+ act = g_new0(ImgBitmapAction, 1);
+ act->act = BITMAP_REMOVE_ALL;
+ QSIMPLEQ_INSERT_TAIL(&actions, act, next);
break;
case OPTION_CLEAR:
act = g_new0(ImgBitmapAction, 1);
act->act = BITMAP_CLEAR;
QSIMPLEQ_INSERT_TAIL(&actions, act, next);
+ need_bitmap_name = true;
break;
case OPTION_ENABLE:
act = g_new0(ImgBitmapAction, 1);
act->act = BITMAP_ENABLE;
QSIMPLEQ_INSERT_TAIL(&actions, act, next);
+ need_bitmap_name = true;
break;
case OPTION_DISABLE:
act = g_new0(ImgBitmapAction, 1);
act->act = BITMAP_DISABLE;
QSIMPLEQ_INSERT_TAIL(&actions, act, next);
+ need_bitmap_name = true;
break;
case OPTION_MERGE:
act = g_new0(ImgBitmapAction, 1);
@@ -5149,6 +5164,7 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
act->src = optarg;
QSIMPLEQ_INSERT_TAIL(&actions, act, next);
merge = true;
+ need_bitmap_name = true;
break;
case 'b':
src_filename = optarg;
@@ -5165,8 +5181,8 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
}
if (QSIMPLEQ_EMPTY(&actions)) {
- error_report("Need at least one of --add, --remove, --clear, "
- "--enable, --disable, or --merge");
+ error_report("Need at least one of --add, --remove, --remove-all, "
+ "--clear, --enable, --disable, or --merge");
goto out;
}
@@ -5184,11 +5200,22 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
goto out;
}
- if (optind != argc - 2) {
+ if (need_bitmap_name && optind != argc - 2) {
error_report("Expecting filename and bitmap name");
goto out;
}
+ /*
+ * Every action other than --remove-all sets need_bitmap_name, so
+ * !need_bitmap_name means the only action(s) given were --remove-all
+ * and the BITMAP positional argument must be omitted. Combinations
+ * like '--remove-all --add foo' remain valid via the branch above.
+ */
+ if (!need_bitmap_name && optind != argc - 1) {
+ error_report("Expecting filename");
+ goto out;
+ }
+
filename = argv[optind];
bitmap = argv[optind + 1];
@@ -5225,6 +5252,20 @@ static int img_bitmap(const img_cmd_t *ccmd, int argc, char **argv)
qmp_block_dirty_bitmap_remove(bs->node_name, bitmap, &err);
op = "remove";
break;
+ case BITMAP_REMOVE_ALL: {
+ BdrvDirtyBitmap *bm;
+ while ((bm = bdrv_dirty_bitmap_first(bs))) {
+ const char *name = bdrv_dirty_bitmap_name(bm);
+ qmp_block_dirty_bitmap_remove(bs->node_name, name, &err);
+ if (err) {
+ /* Save name for proper error reporting */
+ bitmap = name;
+ break;
+ }
+ }
+ op = "remove-all";
+ break;
+ }
case BITMAP_CLEAR:
qmp_block_dirty_bitmap_clear(bs->node_name, bitmap, &err);
op = "clear";
diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps b/tests/qemu-iotests/tests/qemu-img-bitmaps
index 7a3fe8c3d37..979bc0d6c9c 100755
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps
@@ -161,6 +161,30 @@ $QEMU_IMG convert --bitmaps -O qcow2 "$TEST_IMG" "$TEST_IMG.copy"
TEST_IMG="$TEST_IMG.copy" _img_info --format-specific \
| _filter_irrelevant_img_info
+echo
+echo "=== Check --remove-all ==="
+echo
+
+# Start from a fresh image so prior state does not bleed into the assertions
+_rm_test_img "$TEST_IMG"
+_make_test_img 10M
+$QEMU_IMG bitmap --add -f $IMGFMT "$TEST_IMG" b0
+$QEMU_IMG bitmap --add -f $IMGFMT "$TEST_IMG" b1
+$QEMU_IMG bitmap --add -f $IMGFMT "$TEST_IMG" b2
+_img_info --format-specific | _filter_irrelevant_img_info
+
+# Sweep every bitmap in a single command, no BITMAP positional
+echo
+$QEMU_IMG bitmap --remove-all -f $IMGFMT "$TEST_IMG"
+_img_info --format-specific | _filter_irrelevant_img_info
+
+# Wipe + recreate in one invocation: only 'fresh' should remain
+echo
+$QEMU_IMG bitmap --add -f $IMGFMT "$TEST_IMG" b0
+$QEMU_IMG bitmap --add -f $IMGFMT "$TEST_IMG" b1
+$QEMU_IMG bitmap --remove-all --add -f $IMGFMT "$TEST_IMG" fresh
+_img_info --format-specific | _filter_irrelevant_img_info
+
# success, all done
echo '*** done'
rm -f $seq.full
diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps.out b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
index 74b81f703b0..99290d3ba5e 100644
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps.out
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
@@ -180,4 +180,50 @@ Format specific information:
name: b2
granularity: 65536
corrupt: false
+
+=== Check --remove-all ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=10485760
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 10 MiB (10485760 bytes)
+cluster_size: 65536
+Format specific information:
+ bitmaps:
+ [0]:
+ flags:
+ [0]: auto
+ name: b0
+ granularity: 65536
+ [1]:
+ flags:
+ [0]: auto
+ name: b1
+ granularity: 65536
+ [2]:
+ flags:
+ [0]: auto
+ name: b2
+ granularity: 65536
+ corrupt: false
+
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 10 MiB (10485760 bytes)
+cluster_size: 65536
+Format specific information:
+ corrupt: false
+
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 10 MiB (10485760 bytes)
+cluster_size: 65536
+Format specific information:
+ bitmaps:
+ [0]:
+ flags:
+ [0]: auto
+ name: fresh
+ granularity: 65536
+ corrupt: false
*** done
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 3/8] iotests/136: Test stats-intervals with -blockdev/-device
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
2026-06-08 16:52 ` [PULL 1/8] virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914) Kevin Wolf
2026-06-08 16:52 ` [PULL 2/8] qemu-img: add sub-command --remove-all to 'qemu-img bitmap' Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 4/8] qcow2: Fix data loss on zero write with detect-zeroes=unmap Kevin Wolf
` (5 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
Commit 9f0c763e introduced the "stats-intervals" qdev property for
block devices, a setting that was previously only accessible with
-drive. Extend the corresponding test to include test cases that set the
property on -device instead, both with -drive and -blockdev.
We wouldn't really improve coverage with testing every combination of
account_invalid and account_failed with all modes to set up statistics,
so it seems good enough to test all combinations with the old way, and
only both True or both False with the additional ways.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20260521101854.31997-1-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/136 | 87 ++++++++++++++++++++++++++++++++------
tests/qemu-iotests/136.out | 4 +-
2 files changed, 77 insertions(+), 14 deletions(-)
diff --git a/tests/qemu-iotests/136 b/tests/qemu-iotests/136
index 8fce88bd677..58df876bafc 100755
--- a/tests/qemu-iotests/136
+++ b/tests/qemu-iotests/136
@@ -22,6 +22,7 @@
import iotests
import os
+import json
interval_length = 10
nsec_per_sec = 1000000000
@@ -45,14 +46,22 @@ class BlockDeviceStatsTestCase(iotests.QMPTestCase):
wr_highest_offset = 0
account_invalid = False
account_failed = False
+ stats_in_device = False
+ use_blockdev = False
def blockstats(self, device):
result = self.vm.qmp("query-blockstats")
for r in result['return']:
- if r['device'] == device:
+ if r['device'] == device or r['node-name'] == device:
return r['stats']
raise Exception("Device not found for blockstats: %s" % device)
+ def qemu_io(self, cmd):
+ if self.use_blockdev:
+ self.vm.hmp_qemu_io("virtio0/virtio-backend", cmd, qdev=True)
+ else:
+ self.vm.hmp_qemu_io("drive0", cmd)
+
def create_blkdebug_file(self):
file = open(blkdebug_file, 'w')
file.write('''
@@ -73,17 +82,54 @@ sector = "%d"
@iotests.skip_if_unsupported(required_drivers)
def setUp(self):
- drive_args = []
- drive_args.append("stats-intervals.0=%d" % interval_length)
- drive_args.append("stats-account-invalid=%s" %
- (self.account_invalid and "on" or "off"))
- drive_args.append("stats-account-failed=%s" %
- (self.account_failed and "on" or "off"))
- drive_args.append("file.image.read-zeroes=on")
self.create_blkdebug_file()
- self.vm = iotests.VM().add_drive('blkdebug:%s:%s://' %
- (blkdebug_file, self.test_driver),
- ','.join(drive_args))
+ self.vm = iotests.VM()
+
+ drive_args = [
+ "file.image.read-zeroes=on",
+ ]
+ if self.stats_in_device:
+ interface = "none"
+ dev_args = {
+ "driver": "virtio-blk",
+ "id": "virtio0",
+ "drive": "drive0",
+ "stats-intervals": [ interval_length ],
+ "account-invalid": "on" if self.account_invalid else "off",
+ "account-failed": "on" if self.account_failed else "off",
+ }
+ self.vm.add_device(json.dumps(dev_args))
+ else:
+ assert not self.use_blockdev
+ interface = "virtio"
+ drive_args += [
+ "stats-intervals.0=%d" % interval_length,
+ "stats-account-invalid=%s" %
+ (self.account_invalid and "on" or "off"),
+ "stats-account-failed=%s" %
+ (self.account_failed and "on" or "off"),
+ ]
+
+ if self.use_blockdev:
+ blockdev_args = {
+ "node-name": "drive0",
+ "driver": "raw",
+ "file": {
+ "driver": "blkdebug",
+ "config": blkdebug_file,
+ "image": {
+ "driver": self.test_driver,
+ "read-zeroes": True,
+ },
+ },
+ }
+ self.vm.add_blockdev(json.dumps(blockdev_args))
+ else:
+ self.vm.add_drive('blkdebug:%s:%s://' %
+ (blkdebug_file, self.test_driver),
+ ','.join(drive_args),
+ interface=interface)
+
self.vm.launch()
# Set an initial value for the clock
self.vm.qtest("clock_step %d" % nsec_per_sec)
@@ -261,7 +307,7 @@ sector = "%d"
# Now perform all operations
for op in ops:
- self.vm.hmp_qemu_io("drive0", op)
+ self.qemu_io(op)
# Update the expected totals
self.total_rd_bytes += rd_ops * rd_size
@@ -328,6 +374,12 @@ sector = "%d"
# All values must be sane before doing any I/O
self.check_values()
+class BlockDeviceStatsTestDevice(BlockDeviceStatsTestCase):
+ stats_in_device = True
+
+class BlockDeviceStatsTestBlockdev(BlockDeviceStatsTestCase):
+ stats_in_device = True
+ use_blockdev = True
class BlockDeviceStatsTestAccountInvalid(BlockDeviceStatsTestCase):
account_invalid = True
@@ -341,6 +393,17 @@ class BlockDeviceStatsTestAccountBoth(BlockDeviceStatsTestCase):
account_invalid = True
account_failed = True
+class BlockDeviceStatsTestAccountBothDevice(BlockDeviceStatsTestCase):
+ account_invalid = True
+ account_failed = True
+ stats_in_device = True
+
+class BlockDeviceStatsTestAccountBothBlockdev(BlockDeviceStatsTestCase):
+ account_invalid = True
+ account_failed = True
+ stats_in_device = True
+ use_blockdev = True
+
class BlockDeviceStatsTestCoroutine(BlockDeviceStatsTestCase):
test_driver = "null-co"
diff --git a/tests/qemu-iotests/136.out b/tests/qemu-iotests/136.out
index cfa5c0d0e66..4823c113d58 100644
--- a/tests/qemu-iotests/136.out
+++ b/tests/qemu-iotests/136.out
@@ -1,5 +1,5 @@
-...................................
+...............................................................
----------------------------------------------------------------------
-Ran 35 tests
+Ran 63 tests
OK
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 4/8] qcow2: Fix data loss on zero write with detect-zeroes=unmap
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
` (2 preceding siblings ...)
2026-06-08 16:52 ` [PULL 3/8] iotests/136: Test stats-intervals with -blockdev/-device Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 5/8] block/export/fuse: use struct fuse_init_in Kevin Wolf
` (4 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Commit b8bfb1478d ("qcow2: Fix corruption on discard during write with
COW") added a wait_for_dependencies() at the start of
qcow2_subcluster_zeroize(). That fixes the inconsistency it set out to
fix, but turns the lock-protected pre-check in the caller,
qcow2_co_pwrite_zeroes(), into a stale one: the wait yields s->lock,
so an in-flight allocating write whose QCowL2Meta is already on
s->cluster_allocs (but whose L2 entry is not yet linked) gets to link
its entry during the yield. When the zeroize wakes, the cluster is now
NORMAL, and with BDRV_REQ_MAY_UNMAP the free path in zero_in_l2_slice()
unmaps the just-written cluster, silently dropping the data write's
payload.
This is reachable with detect-zeroes=unmap (the default for VirtIO
disks with discard on in Proxmox VE), under which the block layer
auto-promotes all-zero buffers to BDRV_REQ_ZERO_WRITE |
BDRV_REQ_MAY_UNMAP. A memory-constrained Debian guest running 'apt
full-upgrade' on such a disk reproduces it as random SIGSEGVs:
swapped-out code pages come back as zero.
Wait for in-flight dependencies before the lock-protected check in
qcow2_co_pwrite_zeroes(). If a write linked its L2 entry during the
wait, the type check now fails and the block layer falls back to a
bounce-buffered zero write that only touches the requested subrange,
preserving the racing write's data. Promote wait_for_dependencies() to
qcow2_wait_for_dependencies() so qcow2.c can call it.
Fixes: b8bfb1478d ("qcow2: Fix corruption on discard during write with COW")
Cc: qemu-stable@nongnu.org
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Message-ID: <20260522151318.238064-1-t.lamprecht@proxmox.com>
[kwolf: Reverted unnecessary change to 'nr' assignment]
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/qcow2.h | 4 ++++
block/qcow2-cluster.c | 10 +++++-----
block/qcow2.c | 8 +++++++-
tests/qemu-iotests/046 | 23 +++++++++++++++++++++++
tests/qemu-iotests/046.out | 10 ++++++++++
5 files changed, 49 insertions(+), 6 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 192a45d596b..ce517040c47 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -966,6 +966,10 @@ int coroutine_fn GRAPH_RDLOCK
qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
int flags);
+void coroutine_mixed_fn
+qcow2_wait_for_dependencies(BlockDriverState *bs, uint64_t guest_offset,
+ uint64_t bytes);
+
int GRAPH_RDLOCK
qcow2_expand_zero_clusters(BlockDriverState *bs,
BlockDriverAmendStatusCB *status_cb,
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 8b1e80bd0b3..e02fae6a0c6 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1474,9 +1474,9 @@ static int coroutine_fn handle_dependencies(BlockDriverState *bs,
return 0;
}
-static void coroutine_mixed_fn wait_for_dependencies(BlockDriverState *bs,
- uint64_t guest_offset,
- uint64_t bytes)
+void coroutine_mixed_fn qcow2_wait_for_dependencies(BlockDriverState *bs,
+ uint64_t guest_offset,
+ uint64_t bytes)
{
BDRVQcow2State *s = bs->opaque;
QCowL2Meta *m = NULL;
@@ -2035,7 +2035,7 @@ int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset,
* We don't need to allocate a QCowL2Meta for the discard operation because
* s->lock is held for the duration of the whole operation.
*/
- wait_for_dependencies(bs, offset, bytes);
+ qcow2_wait_for_dependencies(bs, offset, bytes);
/* Caller must pass aligned values, except at image end */
assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
@@ -2204,7 +2204,7 @@ int coroutine_fn qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset,
* We don't need to allocate a QCowL2Meta for the zeroize operation because
* s->lock is held for the duration of the whole operation.
*/
- wait_for_dependencies(bs, offset, bytes);
+ qcow2_wait_for_dependencies(bs, offset, bytes);
/* If we have to stay in sync with an external data file, zero out
* s->data_file first. */
diff --git a/block/qcow2.c b/block/qcow2.c
index 81fd299b4c7..19271b10a49 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4234,10 +4234,16 @@ qcow2_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int64_t bytes,
}
qemu_co_mutex_lock(&s->lock);
- /* We can have new write after previous check */
offset -= head;
bytes = s->subcluster_size;
nr = s->subcluster_size;
+ /*
+ * Wait for in-flight allocating writes first: otherwise the type
+ * check below could pass on UNALLOCATED while a yet-to-link_l2 write
+ * completes during qcow2_subcluster_zeroize()'s own wait, letting the
+ * resumed MAY_UNMAP discard the just-written data.
+ */
+ qcow2_wait_for_dependencies(bs, offset, bytes);
ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type);
if (ret < 0 ||
(type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN &&
diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046
index e03dd401479..0d84b5c1c75 100755
--- a/tests/qemu-iotests/046
+++ b/tests/qemu-iotests/046
@@ -226,6 +226,26 @@ aio_write -z 0x140000 0x10000
resume A
aio_flush
EOF
+
+# Start an allocating write to a previously unallocated cluster and, before
+# its L2 update is linked, issue a concurrent sub-cluster zero write with
+# MAY_UNMAP that targets a disjoint range within the same cluster. The zero
+# write's head/tail are zero (cluster is unallocated), so qcow2_co_pwrite_zeroes
+# would expand it to the full subcluster. Without waiting for dependencies
+# before the zero write's "unallocated" type check, that check passes,
+# qcow2_subcluster_zeroize then yields in wait_for_dependencies, the allocating
+# write links its L2 entry, and the resumed zeroize unmaps the cluster -
+# silently discarding the just-written data. Waiting first makes the zero write
+# fall back to a bounce-buffered real write, which only touches its own
+# subrange.
+cat <<EOF
+break write_aio A
+aio_write -P 180 0x200000 0x4000
+wait_break A
+aio_write -z -u 0x204000 0x4000
+resume A
+aio_flush
+EOF
}
overlay_io | $QEMU_IO blkdebug::"$TEST_IMG" | _filter_qemu_io |\
@@ -310,6 +330,9 @@ verify_io()
echo read -P 0 0x120000 0x10000
echo read -P 0 0x130000 0x10000
echo read -P 0 0x140000 0x10000
+
+ echo read -P 180 0x200000 0x4000
+ echo read -P 0 0x204000 0xc000
}
verify_io | $QEMU_IO "$TEST_IMG" | _filter_qemu_io
diff --git a/tests/qemu-iotests/046.out b/tests/qemu-iotests/046.out
index 6341df335c9..137cf527f16 100644
--- a/tests/qemu-iotests/046.out
+++ b/tests/qemu-iotests/046.out
@@ -169,6 +169,12 @@ wrote XXX/XXX bytes at offset XXX
XXX KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote XXX/XXX bytes at offset XXX
XXX KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+blkdebug: Suspended request 'A'
+blkdebug: Resuming request 'A'
+wrote XXX/XXX bytes at offset XXX
+XXX KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote XXX/XXX bytes at offset XXX
+XXX KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== Verify image content ==
read 65536/65536 bytes at offset 0
@@ -275,5 +281,9 @@ read 65536/65536 bytes at offset 1245184
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 65536/65536 bytes at offset 1310720
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 16384/16384 bytes at offset 2097152
+16 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 49152/49152 bytes at offset 2113536
+48 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
No errors were found on the image.
*** done
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 5/8] block/export/fuse: use struct fuse_init_in
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
` (3 preceding siblings ...)
2026-06-08 16:52 ` [PULL 4/8] qcow2: Fix data loss on zero write with detect-zeroes=unmap Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 6/8] block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression Kevin Wolf
` (3 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: Fiona Ebner <f.ebner@proxmox.com>
The code is switched to use the current 'struct fuse_init_in' in
preparation to use the FUSE_DIRECT_IO_ALLOW_MMAP feature, which is
part of the flags2 member that got added in protocol version 5.36.
To not break compatibility with older kernels, the check for whether
the full header of an operation was read in co_read_from_fuse_fd()
needs to be adapted. In particular, for a FUSE_INIT operation, the
protocol version must be considered, because the length of the header
changed with protocol version 7.36. Always using the length of the
old, shorter struct was inaccurate, since for newer protocol versions
this might mean accepting a truncated read for FUSE_INIT.
Users of the init header that want to use parts of the extended
structure must check with the using_old_fuse_init_in() helper function
if they may do so.
Cc: qemu-stable@nongnu.org
Fixes: a94a1d7699 ("fuse: Manually process requests (without libfuse)")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20260506145424.10249-2-f.ebner@proxmox.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/export/fuse.c | 56 +++++++++++++++++++++++++++++++++------------
1 file changed, 42 insertions(+), 14 deletions(-)
diff --git a/block/export/fuse.c b/block/export/fuse.c
index a2a478d2934..35218e31976 100644
--- a/block/export/fuse.c
+++ b/block/export/fuse.c
@@ -51,23 +51,16 @@
#define FUSE_MAX_READ_BYTES (MIN(BDRV_REQUEST_MAX_BYTES, 1 * 1024 * 1024))
#define FUSE_MAX_WRITE_BYTES (64 * 1024)
-/*
- * fuse_init_in structure before 7.36. We don't need the flags2 field added
- * there, so we can work with the smaller older structure to stay compatible
- * with older kernels.
- */
-struct fuse_init_in_compat {
- uint32_t major;
- uint32_t minor;
- uint32_t max_readahead;
- uint32_t flags;
-};
-
typedef struct FuseRequestInHeader {
struct fuse_in_header common;
/* All supported requests */
union {
- struct fuse_init_in_compat init;
+ /*
+ * When using_old_fuse_init_in() is true, then the smaller older struct
+ * is used by the kernel. The flags2 member and other new members must
+ * be treated as absent then.
+ */
+ struct fuse_init_in init;
struct fuse_open_in open;
struct fuse_setattr_in setattr;
struct fuse_read_in read;
@@ -629,6 +622,16 @@ static int clone_fuse_fd(int fd, Error **errp)
return new_fd;
}
+/**
+ * Check whether the smaller older fuse_init_in structure from before protocol
+ * version 7.36 is used. The flags2 member and other new members must be treated
+ * as absent then.
+ */
+static bool using_old_fuse_init_in(const struct fuse_init_in *in)
+{
+ return in->major < 7 || (in->major == 7 && in->minor < 36);
+}
+
/**
* Try to read a single request from the FUSE FD.
* Takes a FuseQueue pointer in `opaque`.
@@ -693,6 +696,31 @@ static void coroutine_fn co_read_from_fuse_fd(void *opaque)
goto no_request;
}
+ /*
+ * If the request is of type FUSE_INIT, need to check the version to
+ * actually determine the length of the fuse_init_in structure used by the
+ * kernel. In protocol version 7.36, the structure was extended.
+ */
+ if (in_hdr->common.opcode == FUSE_INIT) {
+ /* Length of the fuse_init_in structure before 7.36. */
+ size_t old_init_hdr_len = 16;
+
+ /*
+ * Expect at least the size of the smaller older structure to ensure the
+ * version can be checked.
+ */
+ if (unlikely(ret < sizeof(in_hdr->common) + old_init_hdr_len)) {
+ error_report("FUSE_INIT request truncated, read only %zi bytes",
+ ret);
+ fuse_write_err(fuse_fd, &in_hdr->common, -EINVAL);
+ goto no_request;
+ }
+
+ if (using_old_fuse_init_in(&in_hdr->init)) {
+ op_hdr_len = old_init_hdr_len;
+ }
+ }
+
if (unlikely(ret < sizeof(in_hdr->common) + op_hdr_len)) {
error_report("FUSE request truncated, expected %zu bytes, read %zi "
"bytes",
@@ -826,7 +854,7 @@ static bool is_regular_file(const char *path, Error **errp)
*/
static ssize_t coroutine_fn GRAPH_RDLOCK
fuse_co_init(FuseExport *exp, struct fuse_init_out *out,
- const struct fuse_init_in_compat *in)
+ const struct fuse_init_in *in)
{
const uint32_t supported_flags = FUSE_ASYNC_READ | FUSE_ASYNC_DIO;
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 6/8] block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
` (4 preceding siblings ...)
2026-06-08 16:52 ` [PULL 5/8] block/export/fuse: use struct fuse_init_in Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 7/8] iotests: test shared mmap for fuse export Kevin Wolf
` (2 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: Fiona Ebner <f.ebner@proxmox.com>
Commit 8599559580 ("fuse: Set direct_io and parallel_direct_writes")
broke use cases that require mmap() with MAP_SHARED on the export. In
particular, swtpm_setup using its 'file://' protocol requires this.
From the kernel documentation [0]:
> To allow shared mmap, the FUSE_DIRECT_IO_ALLOW_MMAP flag may be
> enabled in the FUSE_INIT reply.
Set the FUSE_DIRECT_IO_ALLOW_MMAP flag to restore compatibility with
users requiring shared mmap. The FUSE_INIT_EXT flag needs to be set
for the flags2 member to have an effect.
[0]: https://www.kernel.org/doc/html/next/filesystems/fuse/fuse-io.html
Cc: qemu-stable@nongnu.org
Fixes: 8599559580 ("fuse: Set direct_io and parallel_direct_writes")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20260506145424.10249-3-f.ebner@proxmox.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/export/fuse.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/block/export/fuse.c b/block/export/fuse.c
index 35218e31976..c0e8dfb6430 100644
--- a/block/export/fuse.c
+++ b/block/export/fuse.c
@@ -856,7 +856,8 @@ static ssize_t coroutine_fn GRAPH_RDLOCK
fuse_co_init(FuseExport *exp, struct fuse_init_out *out,
const struct fuse_init_in *in)
{
- const uint32_t supported_flags = FUSE_ASYNC_READ | FUSE_ASYNC_DIO;
+ uint32_t supported_flags = FUSE_ASYNC_READ | FUSE_ASYNC_DIO;
+ uint32_t flags2 = 0;
if (in->major != 7) {
error_report("FUSE major version mismatch: We have 7, but kernel has %"
@@ -871,13 +872,21 @@ fuse_co_init(FuseExport *exp, struct fuse_init_out *out,
return -EINVAL;
}
+ if (!using_old_fuse_init_in(in)) {
+ /* The flags2 flags must be shifted down by 32 bits. */
+ const uint32_t supported_flags2 = FUSE_DIRECT_IO_ALLOW_MMAP >> 32;
+ /* flags2 is only considered if FUSE_INIT_EXT is set. */
+ supported_flags = supported_flags | FUSE_INIT_EXT;
+ flags2 = in->flags2 & supported_flags2;
+ }
+
*out = (struct fuse_init_out) {
.major = 7,
.minor = MIN(FUSE_KERNEL_MINOR_VERSION, in->minor),
.max_readahead = in->max_readahead,
.max_write = FUSE_MAX_WRITE_BYTES,
.flags = in->flags & supported_flags,
- .flags2 = 0,
+ .flags2 = flags2,
/* libfuse maximum: 2^16 - 1 */
.max_background = UINT16_MAX,
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 7/8] iotests: test shared mmap for fuse export
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
` (5 preceding siblings ...)
2026-06-08 16:52 ` [PULL 6/8] block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-08 16:52 ` [PULL 8/8] qed: Don't try to flush during incoming migration Kevin Wolf
2026-06-09 17:44 ` [PULL 0/8] Block layer patches Stefan Hajnoczi
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: Fiona Ebner <f.ebner@proxmox.com>
This test would have worked before commit 8599559580 ("fuse: Set
direct_io and parallel_direct_writes") and is working again since
commit HEAD~1 ("block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag
to fix regression").
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-ID: <20260506145424.10249-4-f.ebner@proxmox.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/tests/fuse-mmap-shared | 103 ++++++++++++++++++
tests/qemu-iotests/tests/fuse-mmap-shared.out | 5 +
2 files changed, 108 insertions(+)
create mode 100755 tests/qemu-iotests/tests/fuse-mmap-shared
create mode 100644 tests/qemu-iotests/tests/fuse-mmap-shared.out
diff --git a/tests/qemu-iotests/tests/fuse-mmap-shared b/tests/qemu-iotests/tests/fuse-mmap-shared
new file mode 100755
index 00000000000..a0a10cea6ac
--- /dev/null
+++ b/tests/qemu-iotests/tests/fuse-mmap-shared
@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+# group: rw
+#
+# Test that a FUSE export can be mmap()-ed with MAP_SHARED
+#
+# Copyright (C) 2026 Proxmox Server Solutions GmbH
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import itertools
+import mmap
+from mmap import MAP_SHARED
+from pathlib import Path
+
+import iotests
+from iotests import qemu_img, qemu_io, QemuStorageDaemon
+
+def test_fuse_support(mount_point):
+ test_qsd = QemuStorageDaemon('--blockdev', 'null-co,node-name=node0',
+ qmp=True)
+ res = test_qsd.qmp('block-export-add', {
+ 'id': 'exp0',
+ 'type': 'fuse',
+ 'node-name': 'node0',
+ 'mountpoint': mount_point,
+ 'allow-other': 'off'
+ })
+ test_qsd.stop()
+ if 'error' in res:
+ assert (res['error']['desc'] ==
+ "Parameter 'type' does not accept value 'fuse'")
+ iotests.notrun('No FUSE support')
+
+# Shared mmap when using direct IO is only supported for Linux kernels >= 6.6
+# with commit e78662e818f94 ("fuse: add a new fuse init flag to relax
+# estrictions in no cache mode").
+def test_linux_kernel_support():
+ [major, minor] = map(int, os.uname().release.split('.')[:2])
+ if major < 6 or (major == 6 and minor < 6):
+ iotests.notrun('No kernel support for shared mmap with direct IO')
+
+image_size = 1 * 1024 * 1024
+image = os.path.join(iotests.test_dir, 'image.' + iotests.imgfmt)
+fuse_mount_point = os.path.join(iotests.test_dir, 'export.fuse')
+Path(fuse_mount_point).touch()
+
+test_fuse_support(fuse_mount_point)
+test_linux_kernel_support()
+
+class TestMmapShared(iotests.QMPTestCase):
+
+ def setUp(self):
+ qemu_img('create', '-f', iotests.imgfmt, image, str(image_size))
+ qemu_io(image, '-c', f'write -P 23 0 {image_size}')
+
+ self.qsd = QemuStorageDaemon(qmp=True)
+
+ self.qsd.cmd('blockdev-add', {
+ 'node-name': 'node0',
+ 'driver': iotests.imgfmt,
+ 'file': {
+ 'driver': 'file',
+ 'filename': image
+ }
+ })
+
+ self.qsd.cmd('block-export-add', {
+ 'id': 'exp0',
+ 'type': 'fuse',
+ 'node-name': 'node0',
+ 'mountpoint': fuse_mount_point,
+ 'writable': True,
+ 'allow-other': 'off'
+ })
+
+ def tearDown(self):
+ self.stop_qsd()
+ os.remove(image)
+ os.remove(fuse_mount_point)
+
+ def stop_qsd(self):
+ if self.qsd:
+ self.qsd.stop()
+ self.qsd = None
+
+ def test_mmap_shared(self):
+ with open(fuse_mount_point, 'r+b') as file:
+ with mmap.mmap(file.fileno(), image_size, flags=MAP_SHARED) as mm:
+ buf = bytearray(image_size)
+ buf[:] = itertools.repeat(23, image_size)
+ assert mm.read(image_size) == buf
+ buf[:] = itertools.repeat(42, image_size)
+ mm.seek(0)
+ mm.write(buf)
+ mm.flush()
+ self.stop_qsd()
+ qemu_io(image, '-c', f'read -P 42 0 {image_size}')
+
+if __name__ == '__main__':
+ iotests.main(supported_fmts=['generic'],
+ supported_protocols=['file'],
+ supported_platforms=['linux'])
diff --git a/tests/qemu-iotests/tests/fuse-mmap-shared.out b/tests/qemu-iotests/tests/fuse-mmap-shared.out
new file mode 100644
index 00000000000..ae1213e6f86
--- /dev/null
+++ b/tests/qemu-iotests/tests/fuse-mmap-shared.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PULL 8/8] qed: Don't try to flush during incoming migration
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
` (6 preceding siblings ...)
2026-06-08 16:52 ` [PULL 7/8] iotests: test shared mmap for fuse export Kevin Wolf
@ 2026-06-08 16:52 ` Kevin Wolf
2026-06-09 17:44 ` [PULL 0/8] Block layer patches Stefan Hajnoczi
8 siblings, 0 replies; 27+ messages in thread
From: Kevin Wolf @ 2026-06-08 16:52 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, stefanha, qemu-devel
From: Fabiano Rosas <farosas@suse.de>
It's not possible to access the image file while there is an incoming
migration in progress, the QEMU process doesn't hold any locks to the
storage at this point so nodes are inactive. Attempting to flush leads
to an assert at bdrv_co_write_req_prepare():
assert(!(bs->open_flags & BDRV_O_INACTIVE))
The issue is reproducible by running iotest 181 on a host under cpu
load. The migration must coincide with the header already containing
the QED_F_NEED_CHECK flag.
The sequence of events is as follows, with the respective call stacks
referenced below:
During block device init, bdrv_qed_attach_aio_context() starts the
'need_check' timer. The timer will not fire during incoming migration
as it uses QEMU_CLOCK_VIRTUAL (to avoid this very issue, as the code
comment indicates). (0)
However, there's still bdrv_qed_drain_begin() which uses the fact that
the timer is live to decide whether to start the
qed_need_check_timer_entry() directly. (1)
The qed_need_check_timer_entry() eventually calls into
qed_write_header() -> bdrv_co_pwrite() leading to the assert. (2)
Skip creating the 'need_check' timer whenever the image is inactive.
The stacks:
(0) == issues timer_mod ==
#6 in qed_start_need_check_timer at ../block/qed.c:340
#7 in bdrv_qed_attach_aio_context at ../block/qed.c:373
#8 in bdrv_qed_do_open at ../block/qed.c:556
#9 in bdrv_qed_open_entry at ../block/qed.c:582
#10 in coroutine_trampoline at ../util/coroutine-ucontext.c:175
#0 in qemu_coroutine_switch<+120> at ../util/coroutine-ucontext.c:321
#1 in qemu_aio_coroutine_enter<+356> at ../util/qemu-coroutine.c:293
#2 in aio_co_enter<+179> at ../util/async.c:710
#3 in aio_co_wake<+53> at ../util/async.c:695
#4 in thread_pool_co_cb<+47> at ../util/thread-pool.c:283
#5 in thread_pool_completion_bh<+241> at ../util/thread-pool.c:202
#6 in aio_bh_call<+109> at ../util/async.c:173
#7 in aio_bh_poll<+299> at ../util/async.c:220
#8 in aio_poll<+690> at ../util/aio-posix.c:745
#9 in bdrv_qed_open<+392> at ../block/qed.c:607
#10 in bdrv_open_driver<+327> at ../block.c:1678
#11 in bdrv_open_common<+1619> at ../block.c:2008
#12 in bdrv_open_inherit<+2556> at ../block.c:4191
#13 in bdrv_open<+118> at ../block.c:4286
#14 in blk_new_open<+199> at ../block/block-backend.c:458
#15 in blockdev_init<+2011> at ../blockdev.c:612
#16 in drive_new<+3008> at ../blockdev.c:1008
#17 in drive_init_func<+51> at ../system/vl.c:662
#18 in qemu_opts_foreach<+227> at ../util/qemu-option.c:1148
#19 in configure_blockdev<+350> at ../system/vl.c:721
#20 in qemu_create_early_backends<+343> at ../system/vl.c:2076
#21 in qemu_init<+12483> at ../system/vl.c:3778
#22 in main<+46> at ../system/main.c:71
(1) == sees timer_pending ==
#6 in bdrv_qed_drain_begin at ../block/qed.c:391
#7 in bdrv_do_drained_begin at ../block/io.c:366
#8 in bdrv_do_drained_begin_quiesce at ../block/io.c:386
#9 in bdrv_child_cb_drained_begin at ../block.c:1207
#10 in bdrv_parent_drained_begin_single at ../block/io.c:133
#11 in bdrv_parent_drained_begin at ../block/io.c:64
#12 in bdrv_do_drained_begin at ../block/io.c:364
#13 in bdrv_drained_begin at ../block/io.c:393
#14 in blk_drain at ../block/block-backend.c:2101
#15 in blk_unref at ../block/block-backend.c:544
#16 in bdrv_open_inherit at ../block.c:4197
#17 in bdrv_open at ../block.c:4286
#18 in blk_new_open at ../block/block-backend.c:458
#19 in blockdev_init at ../blockdev.c:612
#20 in drive_new at ../blockdev.c:1008
#21 in drive_init_func at ../system/vl.c:662
#22 in qemu_opts_foreach at ../util/qemu-option.c:1148
#23 in configure_blockdev at ../system/vl.c:721
#24 in qemu_create_early_backends at ../system/vl.c:2076
#25 in qemu_init at ../system/vl.c:3778
#26 in main at ../system/main.c:71
(2) == crashes ==
#5 in __assert_fail (assertion="!(bs->open_flags & BDRV_O_INACTIVE)", file="../block/io.c", line=1977
#6 in bdrv_co_write_req_prepare at ../block/io.c:1977
#7 in bdrv_aligned_pwritev at ../block/io.c:2099
#8 in bdrv_co_pwritev_part at ../block/io.c:2316
#9 in bdrv_co_pwritev at ../block/io.c:2233
#10 in bdrv_co_pwrite at ../include/block/block_int-io.h:77
#11 in qed_write_header at ../block/qed.c:128
#12 in qed_need_check_timer at ../block/qed.c:305
#13 in qed_need_check_timer_entry at ../block/qed.c:319
Note that this issue is not exactly the same as what's been reported
in Gitlab, but given how easily this reproduces, I imagine it has to
be happening in that setup as well.
Link: https://gitlab.com/qemu-project/qemu/-/work_items/3515
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Message-ID: <20260603193813.2327596-1-farosas@suse.de>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/qed.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/block/qed.c b/block/qed.c
index da23a83d623..0eccfa21c98 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -351,16 +351,22 @@ static void bdrv_qed_detach_aio_context(BlockDriverState *bs)
{
BDRVQEDState *s = bs->opaque;
- qed_cancel_need_check_timer(s);
- timer_free(s->need_check_timer);
- s->need_check_timer = NULL;
+ if (s->need_check_timer) {
+ qed_cancel_need_check_timer(s);
+ timer_free(s->need_check_timer);
+ s->need_check_timer = NULL;
+ }
}
-static void bdrv_qed_attach_aio_context(BlockDriverState *bs,
- AioContext *new_context)
+static void GRAPH_RDLOCK bdrv_qed_attach_aio_context(BlockDriverState *bs,
+ AioContext *new_context)
{
BDRVQEDState *s = bs->opaque;
+ if (bdrv_is_inactive(bs)) {
+ return;
+ }
+
s->need_check_timer = aio_timer_new(new_context,
QEMU_CLOCK_VIRTUAL, SCALE_NS,
qed_need_check_timer_cb, s);
--
2.54.0
^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
` (7 preceding siblings ...)
2026-06-08 16:52 ` [PULL 8/8] qed: Don't try to flush during incoming migration Kevin Wolf
@ 2026-06-09 17:44 ` Stefan Hajnoczi
2026-06-10 10:15 ` Kevin Wolf
8 siblings, 1 reply; 27+ messages in thread
From: Stefan Hajnoczi @ 2026-06-09 17:44 UTC (permalink / raw)
To: Kevin Wolf, Fiona Ebner; +Cc: qemu-block, stefanha, qemu-devel
On Mon, Jun 8, 2026 at 12:52 PM Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit cc329c491768b2d91eb0b0984f3baa0bf805776d:
>
> Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2026-06-08 09:30:30 -0400)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 296b66d05bac7eb2b86600febdbd2bd8d410f026:
>
> qed: Don't try to flush during incoming migration (2026-06-08 17:00:47 +0200)
>
> ----------------------------------------------------------------
> Block layer patches
>
> - qcow2: Fix data loss on zero write with detect-zeroes=unmap
> - qemu-img bitmap: add sub-command --remove-all
> - export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
Hi Fiona and Kevin,
Please take a look at the CI failure below. If the root cause is in
this pull request, please send a new revision with fixed patches or
without the patches that cause the failure. Thanks!
>>> UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 LD_LIBRARY_PATH=/builds/qemu-project/qemu/build/subprojects/libvfio-user/lib MALLOC_PERTURB_=83 MESON_TEST_ITERATION=1 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 PYTHON=/builds/qemu-project/qemu/build/pyvenv/bin/python3 RUST_BACKTRACE=1 /builds/qemu-project/qemu/build/pyvenv/bin/python3 /builds/qemu-project/qemu/build/../tests/qemu-iotests/check -tap -luks fuse-mmap-shared --source-dir /builds/qemu-project/qemu/tests/qemu-iotests --build-dir /builds/qemu-project/qemu/build/tests/qemu-iotests
――――――――――――――――――――――――――――――――――――― ✀ ―――――――――――――――――――――――――――――――――――――
stderr:
--- /builds/qemu-project/qemu/tests/qemu-iotests/tests/fuse-mmap-shared.out
+++ /builds/qemu-project/qemu/build/scratch/luks-file-fuse-mmap-shared/fuse-mmap-shared.out.bad
@@ -1,5 +1,22 @@
-.
+E
+======================================================================
+ERROR: test_mmap_shared (__main__.TestMmapShared)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "/builds/qemu-project/qemu/tests/qemu-iotests/tests/fuse-mmap-shared",
line 55, in setUp
+ qemu_io(image, '-c', f'write -P 23 0 {image_size}')
+ File "/builds/qemu-project/qemu/tests/qemu-iotests/iotests.py", line
364, in qemu_io
+ return qemu_tool(*qemu_io_wrap_args(args),
+ File "/builds/qemu-project/qemu/tests/qemu-iotests/iotests.py", line
241, in qemu_tool
+ raise VerboseProcessError(
+qemu.utils.VerboseProcessError: Command
'('/builds/qemu-project/qemu/build/qemu-io', '--cache', 'writeback',
'--aio', 'threads',
'/builds/qemu-project/qemu/build/scratch/luks-file-fuse-mmap-shared/image.luks',
'-c', 'write -P 23 0 1048576')' returned non-zero exit status 1.
+ ┏━ output ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+ ┃ qemu-io: can't open device /builds/qemu-project/qemu/build/scratch/
+ ┃ luks-file-fuse-mmap-shared/image.luks: Parameter 'key-secret' is
+ ┃ required for cipher
+ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
----------------------------------------------------------------------
Ran 1 tests
-OK
+FAILED (errors=1)
https://gitlab.com/qemu-project/qemu/-/jobs/14772570524#L257
Stefan
> - virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914)
> - qed: Don't try to flush during incoming migration
> - iotests/136: Test stats-intervals with -blockdev/-device
>
> ----------------------------------------------------------------
> Denis V. Lunev (1):
> qemu-img: add sub-command --remove-all to 'qemu-img bitmap'
>
> Fabiano Rosas (1):
> qed: Don't try to flush during incoming migration
>
> Fiona Ebner (3):
> block/export/fuse: use struct fuse_init_in
> block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
> iotests: test shared mmap for fuse export
>
> Kevin Wolf (1):
> iotests/136: Test stats-intervals with -blockdev/-device
>
> Stefan Hajnoczi (1):
> virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914)
>
> Thomas Lamprecht (1):
> qcow2: Fix data loss on zero write with detect-zeroes=unmap
>
> docs/tools/qemu-img.rst | 10 ++-
> block/qcow2.h | 4 +
> block/export/fuse.c | 69 +++++++++++++----
> block/qcow2-cluster.c | 10 +--
> block/qcow2.c | 8 +-
> block/qed.c | 16 ++--
> hw/block/virtio-blk.c | 8 +-
> qemu-img.c | 55 ++++++++++++--
> tests/qemu-iotests/046 | 23 ++++++
> tests/qemu-iotests/046.out | 10 +++
> tests/qemu-iotests/136 | 87 +++++++++++++++++++---
> tests/qemu-iotests/136.out | 4 +-
> tests/qemu-iotests/tests/fuse-mmap-shared | 103 ++++++++++++++++++++++++++
> tests/qemu-iotests/tests/fuse-mmap-shared.out | 5 ++
> tests/qemu-iotests/tests/qemu-img-bitmaps | 24 ++++++
> tests/qemu-iotests/tests/qemu-img-bitmaps.out | 46 ++++++++++++
> 16 files changed, 430 insertions(+), 52 deletions(-)
> create mode 100755 tests/qemu-iotests/tests/fuse-mmap-shared
> create mode 100644 tests/qemu-iotests/tests/fuse-mmap-shared.out
>
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-09 17:44 ` [PULL 0/8] Block layer patches Stefan Hajnoczi
@ 2026-06-10 10:15 ` Kevin Wolf
2026-06-10 10:18 ` Fiona Ebner
0 siblings, 1 reply; 27+ messages in thread
From: Kevin Wolf @ 2026-06-10 10:15 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: Fiona Ebner, qemu-block, stefanha, qemu-devel
Am 09.06.2026 um 19:44 hat Stefan Hajnoczi geschrieben:
> On Mon, Jun 8, 2026 at 12:52 PM Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > The following changes since commit cc329c491768b2d91eb0b0984f3baa0bf805776d:
> >
> > Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2026-06-08 09:30:30 -0400)
> >
> > are available in the Git repository at:
> >
> > https://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 296b66d05bac7eb2b86600febdbd2bd8d410f026:
> >
> > qed: Don't try to flush during incoming migration (2026-06-08 17:00:47 +0200)
> >
> > ----------------------------------------------------------------
> > Block layer patches
> >
> > - qcow2: Fix data loss on zero write with detect-zeroes=unmap
> > - qemu-img bitmap: add sub-command --remove-all
> > - export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
>
> Hi Fiona and Kevin,
> Please take a look at the CI failure below. If the root cause is in
> this pull request, please send a new revision with fixed patches or
> without the patches that cause the failure. Thanks!
Yes, it's a new test case introduced in this series that doesn't work
with the luks driver. I just declared luks unsupported for the test for
now and pushed a v2 (will send the new cover letter in a moment, too).
Kevin
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-10 10:15 ` Kevin Wolf
@ 2026-06-10 10:18 ` Fiona Ebner
2026-06-10 11:17 ` Daniel P. Berrangé
0 siblings, 1 reply; 27+ messages in thread
From: Fiona Ebner @ 2026-06-10 10:18 UTC (permalink / raw)
To: Kevin Wolf, Stefan Hajnoczi; +Cc: qemu-block, stefanha, qemu-devel
Am 10.06.26 um 12:14 PM schrieb Kevin Wolf:
> Am 09.06.2026 um 19:44 hat Stefan Hajnoczi geschrieben:
>> On Mon, Jun 8, 2026 at 12:52 PM Kevin Wolf <kwolf@redhat.com> wrote:
>>>
>>> The following changes since commit cc329c491768b2d91eb0b0984f3baa0bf805776d:
>>>
>>> Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2026-06-08 09:30:30 -0400)
>>>
>>> are available in the Git repository at:
>>>
>>> https://repo.or.cz/qemu/kevin.git tags/for-upstream
>>>
>>> for you to fetch changes up to 296b66d05bac7eb2b86600febdbd2bd8d410f026:
>>>
>>> qed: Don't try to flush during incoming migration (2026-06-08 17:00:47 +0200)
>>>
>>> ----------------------------------------------------------------
>>> Block layer patches
>>>
>>> - qcow2: Fix data loss on zero write with detect-zeroes=unmap
>>> - qemu-img bitmap: add sub-command --remove-all
>>> - export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
>>
>> Hi Fiona and Kevin,
>> Please take a look at the CI failure below. If the root cause is in
>> this pull request, please send a new revision with fixed patches or
>> without the patches that cause the failure. Thanks!
>
> Yes, it's a new test case introduced in this series that doesn't work
> with the luks driver. I just declared luks unsupported for the test for
> now and pushed a v2 (will send the new cover letter in a moment, too).
Sorry and thanks! I'll need to remember to check with luks in the future.
Best Regards,
Fiona
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-10 10:18 ` Fiona Ebner
@ 2026-06-10 11:17 ` Daniel P. Berrangé
2026-06-10 11:39 ` Kevin Wolf
0 siblings, 1 reply; 27+ messages in thread
From: Daniel P. Berrangé @ 2026-06-10 11:17 UTC (permalink / raw)
To: Fiona Ebner; +Cc: Kevin Wolf, Stefan Hajnoczi, qemu-block, stefanha, qemu-devel
On Wed, Jun 10, 2026 at 12:18:48PM +0200, Fiona Ebner wrote:
> Am 10.06.26 um 12:14 PM schrieb Kevin Wolf:
> > Am 09.06.2026 um 19:44 hat Stefan Hajnoczi geschrieben:
> >> On Mon, Jun 8, 2026 at 12:52 PM Kevin Wolf <kwolf@redhat.com> wrote:
> >>>
> >>> The following changes since commit cc329c491768b2d91eb0b0984f3baa0bf805776d:
> >>>
> >>> Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2026-06-08 09:30:30 -0400)
> >>>
> >>> are available in the Git repository at:
> >>>
> >>> https://repo.or.cz/qemu/kevin.git tags/for-upstream
> >>>
> >>> for you to fetch changes up to 296b66d05bac7eb2b86600febdbd2bd8d410f026:
> >>>
> >>> qed: Don't try to flush during incoming migration (2026-06-08 17:00:47 +0200)
> >>>
> >>> ----------------------------------------------------------------
> >>> Block layer patches
> >>>
> >>> - qcow2: Fix data loss on zero write with detect-zeroes=unmap
> >>> - qemu-img bitmap: add sub-command --remove-all
> >>> - export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression
> >>
> >> Hi Fiona and Kevin,
> >> Please take a look at the CI failure below. If the root cause is in
> >> this pull request, please send a new revision with fixed patches or
> >> without the patches that cause the failure. Thanks!
> >
> > Yes, it's a new test case introduced in this series that doesn't work
> > with the luks driver. I just declared luks unsupported for the test for
> > now and pushed a v2 (will send the new cover letter in a moment, too).
>
> Sorry and thanks! I'll need to remember to check with luks in the future.
You just got unlucky with the new expanded CI testing introduced when
my pull request was merged a few days ago. Previously gitlab CI only
tested qcow2 and raw, and so compat with other drivers was "best effort"
after the fact.
Now the gitlab CI runs I/O tests across 10 drivers, so it needs to
work before merge, which is something contributors didn't need to
think about before now.
If you push a branch to your gitlab fork and trigger CI, you'll see
the results in the "block" job in the pipeline results.
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-10 11:17 ` Daniel P. Berrangé
@ 2026-06-10 11:39 ` Kevin Wolf
2026-06-10 11:48 ` Daniel P. Berrangé
0 siblings, 1 reply; 27+ messages in thread
From: Kevin Wolf @ 2026-06-10 11:39 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Fiona Ebner, Stefan Hajnoczi, qemu-block, stefanha, qemu-devel
Am 10.06.2026 um 13:17 hat Daniel P. Berrangé geschrieben:
> You just got unlucky with the new expanded CI testing introduced when
> my pull request was merged a few days ago. Previously gitlab CI only
> tested qcow2 and raw, and so compat with other drivers was "best effort"
> after the fact.
>
> Now the gitlab CI runs I/O tests across 10 drivers, so it needs to
> work before merge, which is something contributors didn't need to
> think about before now.
>
> If you push a branch to your gitlab fork and trigger CI, you'll see
> the results in the "block" job in the pipeline results.
Technically true, but who has the CI minutes to actually do this? I
don't think we've figured out a solution yet how people (or at least
maintainers) can use QEMU's minutes from the open source program prior
to sending a patch series or pull request. Or have we?
Kevin
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-10 11:39 ` Kevin Wolf
@ 2026-06-10 11:48 ` Daniel P. Berrangé
2026-06-10 12:21 ` Kevin Wolf
0 siblings, 1 reply; 27+ messages in thread
From: Daniel P. Berrangé @ 2026-06-10 11:48 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Fiona Ebner, Stefan Hajnoczi, qemu-block, stefanha, qemu-devel
On Wed, Jun 10, 2026 at 01:39:16PM +0200, Kevin Wolf wrote:
> Am 10.06.2026 um 13:17 hat Daniel P. Berrangé geschrieben:
> > You just got unlucky with the new expanded CI testing introduced when
> > my pull request was merged a few days ago. Previously gitlab CI only
> > tested qcow2 and raw, and so compat with other drivers was "best effort"
> > after the fact.
> >
> > Now the gitlab CI runs I/O tests across 10 drivers, so it needs to
> > work before merge, which is something contributors didn't need to
> > think about before now.
> >
> > If you push a branch to your gitlab fork and trigger CI, you'll see
> > the results in the "block" job in the pipeline results.
>
> Technically true, but who has the CI minutes to actually do this?
Pretty much everyone IMHO.
> I don't think we've figured out a solution yet how people (or at least
> maintainers) can use QEMU's minutes from the open source program prior
> to sending a patch series or pull request. Or have we?
GitLab user accounts get 400 minutes of CI credits.
Forks of QEMU though are only charged at a cost fact or 0.008 since
we are a member of the OSS program
https://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factors-of-hosted-runners-for-gitlabcom
IOW, you're charged 1 minute per 125 minutes of job time.
A single QEMU pipline run in my fork today cost 4.5 credits. That's
enough for 87 pipeline runs per month, if I was not contributing to
anything outside QEMU on gitlab.com.
If you run a pipeline to sanity check before sending a patch series
I don't think most people will ever run out of credits.
If you run multiple pipelines a day during development then you might
be pushing your luck. Better to use the local "make docker-...."
targets for day-to-day testing during dev, and just use gitlab
pipelines before submission.
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-10 11:48 ` Daniel P. Berrangé
@ 2026-06-10 12:21 ` Kevin Wolf
2026-06-10 12:39 ` Daniel P. Berrangé
0 siblings, 1 reply; 27+ messages in thread
From: Kevin Wolf @ 2026-06-10 12:21 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Fiona Ebner, Stefan Hajnoczi, qemu-block, stefanha, qemu-devel
Am 10.06.2026 um 13:48 hat Daniel P. Berrangé geschrieben:
> On Wed, Jun 10, 2026 at 01:39:16PM +0200, Kevin Wolf wrote:
> > Am 10.06.2026 um 13:17 hat Daniel P. Berrangé geschrieben:
> > > You just got unlucky with the new expanded CI testing introduced when
> > > my pull request was merged a few days ago. Previously gitlab CI only
> > > tested qcow2 and raw, and so compat with other drivers was "best effort"
> > > after the fact.
> > >
> > > Now the gitlab CI runs I/O tests across 10 drivers, so it needs to
> > > work before merge, which is something contributors didn't need to
> > > think about before now.
> > >
> > > If you push a branch to your gitlab fork and trigger CI, you'll see
> > > the results in the "block" job in the pipeline results.
> >
> > Technically true, but who has the CI minutes to actually do this?
>
> Pretty much everyone IMHO.
>
> > I don't think we've figured out a solution yet how people (or at least
> > maintainers) can use QEMU's minutes from the open source program prior
> > to sending a patch series or pull request. Or have we?
>
> GitLab user accounts get 400 minutes of CI credits.
>
> Forks of QEMU though are only charged at a cost fact or 0.008 since
> we are a member of the OSS program
>
> https://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factors-of-hosted-runners-for-gitlabcom
>
> IOW, you're charged 1 minute per 125 minutes of job time.
>
> A single QEMU pipline run in my fork today cost 4.5 credits. That's
> enough for 87 pipeline runs per month, if I was not contributing to
> anything outside QEMU on gitlab.com.
>
> If you run a pipeline to sanity check before sending a patch series
> I don't think most people will ever run out of credits.
>
> If you run multiple pipelines a day during development then you might
> be pushing your luck. Better to use the local "make docker-...."
> targets for day-to-day testing during dev, and just use gitlab
> pipelines before submission.
Did this change at some point? Because I'm quite sure I stopped doing
full CI runs only after running out of minutes with very moderate use.
Ever since then, I've only manually started individual jobs when I had
reason to suspect there could be a problem with them.
Kevin
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PULL 0/8] Block layer patches
2026-06-10 12:21 ` Kevin Wolf
@ 2026-06-10 12:39 ` Daniel P. Berrangé
0 siblings, 0 replies; 27+ messages in thread
From: Daniel P. Berrangé @ 2026-06-10 12:39 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Fiona Ebner, Stefan Hajnoczi, qemu-block, stefanha, qemu-devel
On Wed, Jun 10, 2026 at 02:21:33PM +0200, Kevin Wolf wrote:
> Am 10.06.2026 um 13:48 hat Daniel P. Berrangé geschrieben:
> > On Wed, Jun 10, 2026 at 01:39:16PM +0200, Kevin Wolf wrote:
> > > Am 10.06.2026 um 13:17 hat Daniel P. Berrangé geschrieben:
> > > > You just got unlucky with the new expanded CI testing introduced when
> > > > my pull request was merged a few days ago. Previously gitlab CI only
> > > > tested qcow2 and raw, and so compat with other drivers was "best effort"
> > > > after the fact.
> > > >
> > > > Now the gitlab CI runs I/O tests across 10 drivers, so it needs to
> > > > work before merge, which is something contributors didn't need to
> > > > think about before now.
> > > >
> > > > If you push a branch to your gitlab fork and trigger CI, you'll see
> > > > the results in the "block" job in the pipeline results.
> > >
> > > Technically true, but who has the CI minutes to actually do this?
> >
> > Pretty much everyone IMHO.
> >
> > > I don't think we've figured out a solution yet how people (or at least
> > > maintainers) can use QEMU's minutes from the open source program prior
> > > to sending a patch series or pull request. Or have we?
> >
> > GitLab user accounts get 400 minutes of CI credits.
> >
> > Forks of QEMU though are only charged at a cost fact or 0.008 since
> > we are a member of the OSS program
> >
> > https://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factors-of-hosted-runners-for-gitlabcom
> >
> > IOW, you're charged 1 minute per 125 minutes of job time.
> >
> > A single QEMU pipline run in my fork today cost 4.5 credits. That's
> > enough for 87 pipeline runs per month, if I was not contributing to
> > anything outside QEMU on gitlab.com.
> >
> > If you run a pipeline to sanity check before sending a patch series
> > I don't think most people will ever run out of credits.
> >
> > If you run multiple pipelines a day during development then you might
> > be pushing your luck. Better to use the local "make docker-...."
> > targets for day-to-day testing during dev, and just use gitlab
> > pipelines before submission.
>
> Did this change at some point? Because I'm quite sure I stopped doing
> full CI runs only after running out of minutes with very moderate use.
> Ever since then, I've only manually started individual jobs when I had
> reason to suspect there could be a problem with them.
Yes, there was a time window when QEMU was *not* part of the OSS
program and so forks would get charged at the full 1:1 rate. At the
same time gitlab was transitioning personal accounts from two different
CI limits, so those with newer accounts would run out much faster
due to a lower limit than other people with older accounts like myself.
Everyone now has the same 400 minute limit, with the 0.008 cost factor
inherited.
The only caveat is that your git repo *MUST* be a direct fork of
qemu-project/qemu.git in order to inherit the cost factor.
So, yes, there were problems in the past, but it should be much
better for most people today.
And we do have the "QEMU_CI=1" push option to trigger individual
jobs manually vs "QEMU_CI=2" run unleashes the whole pipeline,
so users can conserve CI credits if debugging a specific job
over & over again.
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2026-06-10 12:40 UTC | newest]
Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-08 16:51 [PULL 0/8] Block layer patches Kevin Wolf
2026-06-08 16:52 ` [PULL 1/8] virtio-blk: add missing VIRTIO_BLK_T_SCSI_CMD size check (CVE-2026-48914) Kevin Wolf
2026-06-08 16:52 ` [PULL 2/8] qemu-img: add sub-command --remove-all to 'qemu-img bitmap' Kevin Wolf
2026-06-08 16:52 ` [PULL 3/8] iotests/136: Test stats-intervals with -blockdev/-device Kevin Wolf
2026-06-08 16:52 ` [PULL 4/8] qcow2: Fix data loss on zero write with detect-zeroes=unmap Kevin Wolf
2026-06-08 16:52 ` [PULL 5/8] block/export/fuse: use struct fuse_init_in Kevin Wolf
2026-06-08 16:52 ` [PULL 6/8] block/export/fuse: set FUSE_DIRECT_IO_ALLOW_MMAP flag to fix regression Kevin Wolf
2026-06-08 16:52 ` [PULL 7/8] iotests: test shared mmap for fuse export Kevin Wolf
2026-06-08 16:52 ` [PULL 8/8] qed: Don't try to flush during incoming migration Kevin Wolf
2026-06-09 17:44 ` [PULL 0/8] Block layer patches Stefan Hajnoczi
2026-06-10 10:15 ` Kevin Wolf
2026-06-10 10:18 ` Fiona Ebner
2026-06-10 11:17 ` Daniel P. Berrangé
2026-06-10 11:39 ` Kevin Wolf
2026-06-10 11:48 ` Daniel P. Berrangé
2026-06-10 12:21 ` Kevin Wolf
2026-06-10 12:39 ` Daniel P. Berrangé
-- strict thread matches above, loose matches on Subject: below --
2024-11-14 16:56 Kevin Wolf
2024-11-15 20:16 ` Peter Maydell
2024-11-19 11:25 ` Kevin Wolf
2024-11-19 14:41 ` Stefan Hajnoczi
2024-06-11 17:36 Kevin Wolf
2024-06-13 14:51 ` Richard Henderson
2021-01-27 19:57 Kevin Wolf
2021-01-28 13:58 ` Peter Maydell
2021-01-28 18:19 ` Peter Maydell
2021-01-28 20:13 ` Vladimir Sementsov-Ogievskiy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.