Linux RAID subsystem development
 help / color / mirror / Atom feed
* [PATCH v4 00/25] md: improve lockless bitmap reshape support
@ 2026-08-01 17:24 Yu Kuai
  2026-08-01 17:24 ` [PATCH v4 01/25] md/md-llbitmap: clear flush state after daemon flush Yu Kuai
                   ` (24 more replies)
  0 siblings, 25 replies; 48+ messages in thread
From: Yu Kuai @ 2026-08-01 17:24 UTC (permalink / raw)
  To: Song Liu
  Cc: Li Nan, Xiao Ni, Mykola Marzhan, Su Yue, linux-raid, linux-kernel

From: Yu Kuai <yukuai@fygo.io>

This series fixes several lockless bitmap accounting issues and adds the
missing llbitmap support for online reshape in RAID10 and RAID5.

llbitmap has different constraints from the existing bitmap code: there is
one live bitmap instance, each bit state has richer semantics, and reshape can
change the mapping from logical array ranges to bitmap ranges. Without exact
old/new geometry tracking and reshape-boundary splitting, write accounting can
use the wrong bitmap mapping or merge state across ranges that should be
checkpointed separately.

The first patches fix independent llbitmap and RAID5 correctness issues that
should be safe before the reshape work. The rest of the series adds exact
bitmap mapping hooks, tracks llbitmap reshape geometry, grows the page cache
for reshape, remaps checkpointed state as reshape advances, and wires RAID10
and RAID5 to split reshape-crossing bios before bitmap accounting and to mark
llbitmap reshape checkpoints.

Changes since v3:

  - Add patch 1 to clear the sticky LLPageFlush state after daemon flush.
  - Add patch 2 to use GFP_NOIO for llbitmap cache and control allocations.
  - Add patch 3 to only checkpoint fully completed llbitmap sync chunks.
  - Add patch 4 to reject zero-sector RAID5 reshape chunks.
  - Fix patch 8 to snapshot mddev->reshape_position with READ_ONCE() while
    splitting reshape-crossing bios.
  - Fix patch 10 to keep the independently allocated page controls under the
    GFP_NOIO allocation policy added by patch 2.
  - Fix patch 11 to quiesce the array while llbitmap grows page controls
    during resize or reshape.
  - Fix patch 15 to use READ_ONCE(mddev->reshape_position) in llbitmap reshape
    range mapping helpers instead of plain lockless reads.
  - Fix patch 17 to serialize reshape checkpoint remapping against normal
    write/discard bitmap state updates with a rwlock, and copy overlapping
    remap ranges backwards when needed.
  - Fix patch 21 to end RAID10 write accounting if reshape bio splitting
    fails.
  - Fix patch 25 to resubmit the RAID5 front split bio after waiting for
    reshape instead of retrying the already-submitted remainder.

Changes since v2:

  - Fix RAID5 bitmap stripe rounding for non-power-of-two stripe widths in
    patch 1.
  - Fix partial discard chunks being marked unwritten in patch 11.
  - Fix RAID5 writes past component size being skipped by llbitmap accounting
    in patch 11.
  - Fix stopped reshape reassembly losing target llbitmap geometry in patch 9.
  - Fix the page-cache-grow patch using target-geometry fields before they are
    introduced in patch 7.

Changes since v1:

  - Add Reviewed-by tags from Su Yue.
  - Rename llbitmap_resize_chunks() to llbitmap_calculate_chunks().
  - Use an unsigned index in llbitmap_expand_pages() error cleanup.
  - Rebase on mdraid/md-7.2, including the mddev_bio_split_at_reshape_offset()
    declaration needed by the RAID5 build reported by kernel test robot.

Previous versions:
  v3: https://lore.kernel.org/linux-raid/cover.1785206690.git.yukuai@fygo.io/
  v2: https://lore.kernel.org/linux-raid/cover.1782282042.git.yukuai@kernel.org/
  v1: https://lore.kernel.org/linux-raid/20260605091527.2463539-1-yukuai@kernel.org/

Yu Kuai (25):
  md/md-llbitmap: clear flush state after daemon flush
  md/md-llbitmap: use GFP_NOIO for cache allocations
  md/md-llbitmap: only end fully synced chunks
  md/raid5: reject zero-sector reshape chunks
  md/raid5: round bitmap stripes with sector division
  md: add exact bitmap mapping and reshape hooks
  md: skip bitmap accounting for empty write ranges
  md: add helper to split bios at reshape offset
  md/md-llbitmap: track bitmap sync_size explicitly
  md/md-llbitmap: allocate page controls independently
  md/md-llbitmap: grow the page cache in place for reshape
  md/md-llbitmap: track target reshape geometry fields
  md/md-llbitmap: finish reshape geometry
  md/md-llbitmap: refuse reshape while llbitmap still needs sync
  md/md-llbitmap: add reshape range mapping helpers
  md/md-llbitmap: don't skip reshape ranges from bitmap state
  md/md-llbitmap: remap checkpointed bits as reshape progresses
  md/md-llbitmap: clamp state-machine walks to tracked bits
  md/raid10: reject llbitmap reshape when md chunk shrinks
  md/raid10: wire llbitmap reshape lifecycle
  md/raid10: split reshape bios before bitmap accounting
  md/raid5: add exact old and new llbitmap mapping helpers
  md/raid5: reject llbitmap reshape when md chunk shrinks
  md/raid5: wire llbitmap reshape lifecycle
  md/raid5: split reshape bios before bitmap accounting

 drivers/md/md-bitmap.c   |   8 +
 drivers/md/md-bitmap.h   |   9 +
 drivers/md/md-llbitmap.c | 717 +++++++++++++++++++++++++++++++++++----
 drivers/md/md.c          |  64 +++-
 drivers/md/md.h          |   8 +
 drivers/md/raid10.c      |  53 +++
 drivers/md/raid5.c       | 139 ++++++--
 7 files changed, 912 insertions(+), 86 deletions(-)


base-commit: 55b77337bdd088c77461588e5ec094421b89911b
-- 
2.51.0

^ permalink raw reply	[flat|nested] 48+ messages in thread

end of thread, other threads:[~2026-08-01 18:43 UTC | newest]

Thread overview: 48+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-01 17:24 [PATCH v4 00/25] md: improve lockless bitmap reshape support Yu Kuai
2026-08-01 17:24 ` [PATCH v4 01/25] md/md-llbitmap: clear flush state after daemon flush Yu Kuai
2026-08-01 17:43   ` sashiko-bot
2026-08-01 17:24 ` [PATCH v4 02/25] md/md-llbitmap: use GFP_NOIO for cache allocations Yu Kuai
2026-08-01 17:41   ` sashiko-bot
2026-08-01 17:24 ` [PATCH v4 03/25] md/md-llbitmap: only end fully synced chunks Yu Kuai
2026-08-01 17:42   ` sashiko-bot
2026-08-01 17:24 ` [PATCH v4 04/25] md/raid5: reject zero-sector reshape chunks Yu Kuai
2026-08-01 17:45   ` sashiko-bot
2026-08-01 17:24 ` [PATCH v4 05/25] md/raid5: round bitmap stripes with sector division Yu Kuai
2026-08-01 17:39   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 06/25] md: add exact bitmap mapping and reshape hooks Yu Kuai
2026-08-01 17:25 ` [PATCH v4 07/25] md: skip bitmap accounting for empty write ranges Yu Kuai
2026-08-01 18:05   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 08/25] md: add helper to split bios at reshape offset Yu Kuai
2026-08-01 17:41   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 09/25] md/md-llbitmap: track bitmap sync_size explicitly Yu Kuai
2026-08-01 17:44   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 10/25] md/md-llbitmap: allocate page controls independently Yu Kuai
2026-08-01 17:47   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 11/25] md/md-llbitmap: grow the page cache in place for reshape Yu Kuai
2026-08-01 18:03   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 12/25] md/md-llbitmap: track target reshape geometry fields Yu Kuai
2026-08-01 17:51   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 13/25] md/md-llbitmap: finish reshape geometry Yu Kuai
2026-08-01 17:59   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 14/25] md/md-llbitmap: refuse reshape while llbitmap still needs sync Yu Kuai
2026-08-01 17:50   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 15/25] md/md-llbitmap: add reshape range mapping helpers Yu Kuai
2026-08-01 17:47   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 16/25] md/md-llbitmap: don't skip reshape ranges from bitmap state Yu Kuai
2026-08-01 17:41   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 17/25] md/md-llbitmap: remap checkpointed bits as reshape progresses Yu Kuai
2026-08-01 18:12   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 18/25] md/md-llbitmap: clamp state-machine walks to tracked bits Yu Kuai
2026-08-01 17:55   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 19/25] md/raid10: reject llbitmap reshape when md chunk shrinks Yu Kuai
2026-08-01 17:25 ` [PATCH v4 20/25] md/raid10: wire llbitmap reshape lifecycle Yu Kuai
2026-08-01 18:28   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 21/25] md/raid10: split reshape bios before bitmap accounting Yu Kuai
2026-08-01 17:59   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 22/25] md/raid5: add exact old and new llbitmap mapping helpers Yu Kuai
2026-08-01 18:05   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 23/25] md/raid5: reject llbitmap reshape when md chunk shrinks Yu Kuai
2026-08-01 17:25 ` [PATCH v4 24/25] md/raid5: wire llbitmap reshape lifecycle Yu Kuai
2026-08-01 18:12   ` sashiko-bot
2026-08-01 17:25 ` [PATCH v4 25/25] md/raid5: split reshape bios before bitmap accounting Yu Kuai
2026-08-01 18:43   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox