All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v7 00/27] btrfs: zoned: fixes for generic/747
@ 2026-11-17 18:58 Johannes Thumshirn
  2026-11-17 18:58 ` [PATCH v7 05/27] btrfs: zoned: skip the append only metadata writeback path for conventional block groups Johannes Thumshirn
                   ` (17 more replies)
  0 siblings, 18 replies; 19+ messages in thread
From: Johannes Thumshirn @ 2026-11-17 18:58 UTC (permalink / raw)
  To: linux-btrfs
  Cc: Damien Le Moal, Naohiro Aota, David Sterba, Boris Burkov,
	Johannes Thumshirn

At the moment, zoned btrfs doesn't distinguish between DATA and
METADATA/SYSTEM block-groups, when allocating a new chunk. Therefore
METADATA and SYSTEM block-groups can end up on sequential write only
zones of a drive. This adds a lot of complexity, as we can only append
to these zones (and not perform random writes) and for METADATA we need
to write with a queue-depth of 1 using the btrfs_zoned_meta_io_lock().

Also the write stream for METADATA is not allowed to have any "holes" in
it and must be aligned to the write pointer, which was a source of hard
to fix bugs in the past.

This series introduces a split in the block-group allocation path on
zoned filesystems. For METADATA and SYSTEM block-groups try to place
them on conventional zones if the device has them and enough are free.
Then change the allocation in these block-groups from the zoned
allocator to the unclustered allocator, which is less complex than
setting up a cluster for them only. The reverse is done for DATA
block-groups, which can be trivially written on sequential write
required block-groups as we use REQ_OP_ZONE_APPEND for writing DATA with
a high queue-depth.

In addition to this, this series reworks how garbage collection works in
zoned btrfs by tackling the remaining accounting problems and relocation
still being triggered not timely enough to evacuate the space the user
write needs.

The first two patches are fixes that are independent of the rest. The
first one stops allocations from waiting forever for a zone finish after
a transaction abort or shutdown, which hangs generic/388 and is much
easier to hit once metadata lives on conventional zones. As whether a
block-group is append only is derived from the zone types of its stripes,
dev-replace must not change them, so the second one rejects a replace
onto a device with a different conventional/sequential layout.

Changes to v6:
- Rebase onto current btrfs/for-next
- New patch: don't wait for zone finish after fs error
- New patch: reject dev-replace on zone type mismatch
- Rename "don't activate conventional zones of mixed block groups"
- Drop its reservation hunk, already upstream
- Clarify its active state invariants in the changelog
- Keep upstream zone finish in btrfs_relocate_chunk_finish()
- Explain lockless check in "never activate conventional metadata block groups"
- ASSERT zoned and data space-info in btrfs_zoned_calc_inflight()
- Add Boris' Reviewed-by tags
- Mention why the unclustered allocator is used

Changes to v5:
- Don't overwrite ffe_ctl->policy, pick the allocator per block-group
- Retry without empty_size in LOOP_NO_EMPTY_SIZE on zoned as well, so non
  append only block-groups don't hit a premature ENOSPC
- Perform space_info migration only on ro block-groups
- Only set data_reloc_bg under relocation_bg_lock if it isn't set yet and
  undo the space_info migration otherwise
- Skip a block-group in do_allocation_zoned() instead of ASSERT()ing when
  data_reloc_bg points to a different block-group, and only clear
  data_reloc_bg if it points to the current block-group
- Don't account bytes_super twice when moving a block-group between
  space_infos
- Return the free space of a reset block-group to its current space_info
  if promoting it to the relocation target fails
- Reword commit messages of "use the regular allocator for conventional
  metadata block groups" and "return finished relocation block groups to
  the data space_info"

Changes to v4:
- Fixup broken rebase

Changes to v3:
- Fix the active-zone reservation race
- defer RAID activity publication until all stripe write pointers and
  capacities have been validated.

Johannes Thumshirn (27):
  btrfs: zoned: don't wait for a zone finish after a filesystem error
  btrfs: zoned: reject dev-replace onto a target with different zone
    types
  btrfs: zoned: fix possible race for reserved_active_zones
  btrfs: zoned: don't use active zones for conventional stripes
  btrfs: zoned: skip the append only metadata writeback path for
    conventional block groups
  btrfs: zoned: never activate conventional metadata block groups
  btrfs: zoned: use the regular allocator for conventional metadata
    block groups
  btrfs: zoned: only take the metadata IO lock for append only block
    groups
  btrfs: zoned: allow restricting the dev extent search to a zone type
  btrfs: zoned: prefer conventional zones for metadata chunks
  btrfs: zoned: use plain mutex lock in check_bg_is_active
  btrfs: zoned: use plain mutex lock in
    btrfs_zoned_release_dirty_metadata
  btrfs: zoned: remove btrfs_zoned_meta_io_(un)lock
  btrfs: zoned: prefer sequential zones for data chunks
  btrfs: zoned: wait for a running block group reclaim in RECLAIM_ZONES
  btrfs: zoned: don't count in-flight data as available in statfs
  btrfs: zoned: don't recheck the reclaim threshold of full zoned
    block-groups
  btrfs: zoned: don't account superblock zones as available data space
  btrfs: zoned: factor out resetting a block group in place
  btrfs: zoned: prepare reserving the data relocation block group at
    runtime
  btrfs: zoned: promote reset data block groups to the relocation target
  btrfs: zoned: return finished relocation block groups to the data
    space_info
  btrfs: zoned: reuse block groups emptied by reclaim as relocation
    target
  btrfs: add btrfs_bg_type_to_num_stripes
  btrfs: zoned: keep a zone free for relocation when allocating data
    chunks
  btrfs: zoned: don't count the relocation zone reserve as available in
    statfs
  btrfs: zoned: provide zoned specific reclaim sweep

 fs/btrfs/block-group.c       |  56 ++-
 fs/btrfs/block-group.h       |   2 +
 fs/btrfs/dev-replace.c       |   9 +
 fs/btrfs/direct-io.c         |   4 +-
 fs/btrfs/extent-tree.c       |  50 +-
 fs/btrfs/extent_io.c         |  84 ++--
 fs/btrfs/extent_io.h         |   2 +
 fs/btrfs/free-space-cache.c  |  23 +-
 fs/btrfs/fs.h                |   3 +
 fs/btrfs/inode.c             |   3 +
 fs/btrfs/messages.c          |   1 +
 fs/btrfs/space-info.c        |   3 +-
 fs/btrfs/super.c             |  33 +-
 fs/btrfs/tests/zoned-tests.c |  16 +
 fs/btrfs/volumes.c           | 107 ++++-
 fs/btrfs/volumes.h           |   1 +
 fs/btrfs/zoned.c             | 893 ++++++++++++++++++++++++++++-------
 fs/btrfs/zoned.h             | 118 ++++-
 18 files changed, 1136 insertions(+), 272 deletions(-)

-- 
2.56.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2026-11-17 21:09 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 18:58 [PATCH v7 00/27] btrfs: zoned: fixes for generic/747 Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 05/27] btrfs: zoned: skip the append only metadata writeback path for conventional block groups Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 06/27] btrfs: zoned: never activate conventional metadata " Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 07/27] btrfs: zoned: use the regular allocator for " Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 08/27] btrfs: zoned: only take the metadata IO lock for append only " Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 09/27] btrfs: zoned: allow restricting the dev extent search to a zone type Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 10/27] btrfs: zoned: prefer conventional zones for metadata chunks Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 11/27] btrfs: zoned: use plain mutex lock in check_bg_is_active Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 20/27] btrfs: zoned: prepare reserving the data relocation block group at runtime Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 21/27] btrfs: zoned: promote reset data block groups to the relocation target Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 22/27] btrfs: zoned: return finished relocation block groups to the data space_info Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 23/27] btrfs: zoned: reuse block groups emptied by reclaim as relocation target Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 24/27] btrfs: add btrfs_bg_type_to_num_stripes Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 25/27] btrfs: zoned: keep a zone free for relocation when allocating data chunks Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 26/27] btrfs: zoned: don't count the relocation zone reserve as available in statfs Johannes Thumshirn
2026-11-17 18:58 ` [PATCH v7 27/27] btrfs: zoned: provide zoned specific reclaim sweep Johannes Thumshirn
     [not found] ` <20261117185850.4128157-2-johannes.thumshirn@wdc.com>
2026-11-17 21:07   ` [PATCH v7 01/27] btrfs: zoned: don't wait for a zone finish after a filesystem error Damien Le Moal
     [not found] ` <20261117185850.4128157-3-johannes.thumshirn@wdc.com>
2026-11-17 21:08   ` [PATCH v7 02/27] btrfs: zoned: reject dev-replace onto a target with different zone types Damien Le Moal
     [not found] ` <20261117185850.4128157-4-johannes.thumshirn@wdc.com>
2026-11-17 21:09   ` [PATCH v7 03/27] btrfs: zoned: fix possible race for reserved_active_zones Damien Le Moal

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.