* [PULL v2 00/51] Block layer patches
@ 2022-12-15 11:59 Kevin Wolf
2022-12-15 14:44 ` Peter Maydell
0 siblings, 1 reply; 3+ messages in thread
From: Kevin Wolf @ 2022-12-15 11:59 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, peter.maydell, qemu-devel
The following changes since commit 5204b499a6cae4dfd9fe762d5e6e82224892383b:
mailmap: Fix Stefan Weil author email (2022-12-13 15:56:57 -0500)
are available in the Git repository at:
https://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 347fe9e156a3e00c40ae1802978276a1f7d5545f:
block: GRAPH_RDLOCK for functions only called by co_wrappers (2022-12-15 10:11:45 +0100)
v2:
- Changed TSA capability name to "mutex" to work with older clang
versions. The tsan-build CI job succeeds now.
----------------------------------------------------------------
Block layer patches
- Code cleanups around block graph modification
- Simplify drain
- coroutine_fn correctness fixes, including splitting generated
coroutine wrappers into co_wrapper (to be called only from
non-coroutine context) and co_wrapper_mixed (both coroutine and
non-coroutine context)
- Introduce a block graph rwlock
----------------------------------------------------------------
Emanuele Giuseppe Esposito (21):
block-io: introduce coroutine_fn duplicates for bdrv_common_block_status_above callers
block-copy: add coroutine_fn annotations
nbd/server.c: add coroutine_fn annotations
block-backend: replace bdrv_*_above with blk_*_above
block/vmdk: add coroutine_fn annotations
block: avoid duplicating filename string in bdrv_create
block: distinguish between bdrv_create running in coroutine and not
block: bdrv_create_file is a coroutine_fn
block: rename generated_co_wrapper in co_wrapper_mixed
block-coroutine-wrapper.py: introduce co_wrapper
block-coroutine-wrapper.py: support functions without bs arg
block-coroutine-wrapper.py: support also basic return types
block: convert bdrv_create to co_wrapper
block/dirty-bitmap: convert coroutine-only functions to co_wrapper
graph-lock: Implement guard macros
async: Register/unregister aiocontext in graph lock list
block: wrlock in bdrv_replace_child_noperm
block: remove unnecessary assert_bdrv_graph_writable()
block: assert that graph read and writes are performed correctly
block-coroutine-wrapper.py: introduce annotations that take the graph rdlock
block: use co_wrapper_mixed_bdrv_rdlock in functions taking the rdlock
Kevin Wolf (25):
qed: Don't yield in bdrv_qed_co_drain_begin()
test-bdrv-drain: Don't yield in .bdrv_co_drained_begin/end()
block: Revert .bdrv_drained_begin/end to non-coroutine_fn
block: Remove drained_end_counter
block: Inline bdrv_drain_invoke()
block: Fix locking for bdrv_reopen_queue_child()
block: Drain individual nodes during reopen
block: Don't use subtree drains in bdrv_drop_intermediate()
stream: Replace subtree drain with a single node drain
block: Remove subtree drains
block: Call drain callbacks only once
block: Remove ignore_bds_parents parameter from drain_begin/end.
block: Drop out of coroutine in bdrv_do_drained_begin_quiesce()
block: Don't poll in bdrv_replace_child_noperm()
block: Remove poll parameter from bdrv_parent_drained_begin_single()
block: Factor out bdrv_drain_all_begin_nopoll()
Import clang-tsa.h
clang-tsa: Add TSA_ASSERT() macro
clang-tsa: Add macros for shared locks
configure: Enable -Wthread-safety if present
test-bdrv-drain: Fix incorrrect drain assumptions
block: Fix locking in external_snapshot_prepare()
graph-lock: TSA annotations for lock/unlock functions
Mark assert_bdrv_graph_readable/writable() GRAPH_RD/WRLOCK
block: GRAPH_RDLOCK for functions only called by co_wrappers
Paolo Bonzini (1):
graph-lock: Introduce a lock to protect block graph operations
Vladimir Sementsov-Ogievskiy (4):
block: Inline bdrv_detach_child()
block: drop bdrv_remove_filter_or_cow_child
block: bdrv_refresh_perms(): allow external tran
block: refactor bdrv_list_refresh_perms to allow any list of nodes
docs/devel/block-coroutine-wrapper.rst | 6 +-
configure | 1 +
block/block-gen.h | 11 +-
block/coroutines.h | 21 +-
include/block/aio.h | 9 +
include/block/block-common.h | 27 ++-
include/block/block-copy.h | 5 +-
include/block/block-global-state.h | 15 +-
include/block/block-io.h | 136 +++++------
include/block/block_int-common.h | 49 ++--
include/block/block_int-global-state.h | 17 --
include/block/block_int-io.h | 12 -
include/block/block_int.h | 1 +
include/block/dirty-bitmap.h | 10 +-
include/block/graph-lock.h | 280 +++++++++++++++++++++++
include/qemu/clang-tsa.h | 114 ++++++++++
include/sysemu/block-backend-io.h | 77 ++++---
block.c | 404 ++++++++++++++++++---------------
block/block-backend.c | 25 +-
block/block-copy.c | 21 +-
block/commit.c | 4 +-
block/crypto.c | 2 +-
block/dirty-bitmap.c | 88 +------
block/graph-lock.c | 275 ++++++++++++++++++++++
block/io.c | 367 ++++++++++--------------------
block/parallels.c | 2 +-
block/qcow.c | 2 +-
block/qcow2.c | 4 +-
block/qed.c | 28 ++-
block/raw-format.c | 2 +-
block/replication.c | 6 -
block/stream.c | 26 ++-
block/throttle.c | 8 +-
block/vdi.c | 2 +-
block/vhdx.c | 2 +-
block/vmdk.c | 38 ++--
block/vpc.c | 2 +-
blockdev.c | 17 +-
blockjob.c | 2 +-
nbd/server.c | 47 ++--
stubs/graph-lock.c | 10 +
tests/unit/test-bdrv-drain.c | 387 +++++++++----------------------
util/async.c | 4 +
scripts/block-coroutine-wrapper.py | 133 ++++++++---
block/meson.build | 2 +
stubs/meson.build | 1 +
46 files changed, 1575 insertions(+), 1127 deletions(-)
create mode 100644 include/block/graph-lock.h
create mode 100644 include/qemu/clang-tsa.h
create mode 100644 block/graph-lock.c
create mode 100644 stubs/graph-lock.c
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PULL v2 00/51] Block layer patches
2022-12-15 11:59 [PULL v2 00/51] Block layer patches Kevin Wolf
@ 2022-12-15 14:44 ` Peter Maydell
2022-12-15 15:13 ` Kevin Wolf
0 siblings, 1 reply; 3+ messages in thread
From: Peter Maydell @ 2022-12-15 14:44 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, qemu-devel
On Thu, 15 Dec 2022 at 11:59, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit 5204b499a6cae4dfd9fe762d5e6e82224892383b:
>
> mailmap: Fix Stefan Weil author email (2022-12-13 15:56:57 -0500)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 347fe9e156a3e00c40ae1802978276a1f7d5545f:
>
> block: GRAPH_RDLOCK for functions only called by co_wrappers (2022-12-15 10:11:45 +0100)
>
> v2:
> - Changed TSA capability name to "mutex" to work with older clang
> versions. The tsan-build CI job succeeds now.
>
> ----------------------------------------------------------------
> Block layer patches
>
> - Code cleanups around block graph modification
> - Simplify drain
> - coroutine_fn correctness fixes, including splitting generated
> coroutine wrappers into co_wrapper (to be called only from
> non-coroutine context) and co_wrapper_mixed (both coroutine and
> non-coroutine context)
> - Introduce a block graph rwlock
This fails to compile on the FreeBSD 12 and 13 jobs:
https://gitlab.com/qemu-project/qemu/-/jobs/3479763741
https://gitlab.com/qemu-project/qemu/-/jobs/3479763746
The compiler is producing -Wthread-safety-analysis
warnings on code in qemu-thread-posix.c, which are a
compile failure because of -Werror.
thanks
-- PMM
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PULL v2 00/51] Block layer patches
2022-12-15 14:44 ` Peter Maydell
@ 2022-12-15 15:13 ` Kevin Wolf
0 siblings, 0 replies; 3+ messages in thread
From: Kevin Wolf @ 2022-12-15 15:13 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-block, qemu-devel
Am 15.12.2022 um 15:44 hat Peter Maydell geschrieben:
> On Thu, 15 Dec 2022 at 11:59, Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > The following changes since commit 5204b499a6cae4dfd9fe762d5e6e82224892383b:
> >
> > mailmap: Fix Stefan Weil author email (2022-12-13 15:56:57 -0500)
> >
> > are available in the Git repository at:
> >
> > https://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 347fe9e156a3e00c40ae1802978276a1f7d5545f:
> >
> > block: GRAPH_RDLOCK for functions only called by co_wrappers (2022-12-15 10:11:45 +0100)
> >
> > v2:
> > - Changed TSA capability name to "mutex" to work with older clang
> > versions. The tsan-build CI job succeeds now.
> >
> > ----------------------------------------------------------------
> > Block layer patches
> >
> > - Code cleanups around block graph modification
> > - Simplify drain
> > - coroutine_fn correctness fixes, including splitting generated
> > coroutine wrappers into co_wrapper (to be called only from
> > non-coroutine context) and co_wrapper_mixed (both coroutine and
> > non-coroutine context)
> > - Introduce a block graph rwlock
>
> This fails to compile on the FreeBSD 12 and 13 jobs:
> https://gitlab.com/qemu-project/qemu/-/jobs/3479763741
> https://gitlab.com/qemu-project/qemu/-/jobs/3479763746
>
> The compiler is producing -Wthread-safety-analysis
> warnings on code in qemu-thread-posix.c, which are a
> compile failure because of -Werror.
Hmm... FreeBSD actually annotates it pthread locking functions for TSA,
so all callers need to be annotated as well. I guess it's nice in
theory, but hard to enable for a huge codebase like QEMU...
I'll just drop "configure: Enable -Wthread-safety if present" for now.
Maybe we can have a configure check later to enable it by default on
glibc at least. Or we really need to go through all locks in QEMU and
annotate them properly. This might be a bit too painful, though, so we
may end up leaving FreeBSD unchecked even if that seems to be the OS to
care most about it...
Kevin
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-12-15 15:14 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-12-15 11:59 [PULL v2 00/51] Block layer patches Kevin Wolf
2022-12-15 14:44 ` Peter Maydell
2022-12-15 15:13 ` Kevin Wolf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).