[PATCH 00/17] btree write buffer & journal optimizations

[Kent Overstreet <kent.overstreet@linux.dev>]

Large stack of changes - performance optimizations and improvements to
related code.

The first big one, killing journal pre-reservations, is already merged:
besides being a performance improvement it turned out to fix a few tests
that were sporadically failing with "journal stuck" - nice. The idea
with that patch is that instead of reserving space in the journal ahead
of time (for journal reservations that must succeed or risk deadlock),
we instead rely on watermarks and tracking which operations are
necessary for reclaim - when we're low on space in the journal and
flushing things for reclaim, we're able to flush things in order, or
track when we're not flushing things in order and allow it to fail

Most of the other patches are prep work leading up to the big btree
write buffer rewrite, which changes the btree write buffer to pull keys
from the journal, instead of having the tranaction commit path directly
add keys to the btree write buffer.

btree write buffer performance is pretty important for multithreaded
update workloads - it's an amdahl's law situation, flushing it is single
threaded. More performance optimizations there would still be
worthwhile, and I've been looking at it because I'm sketching out a
rewrite of disk space accounting (which will allow for per snapshot Id
accounting) that will lean heavily on the btree write buffer.


Kent Overstreet (17):
  bcachefs: Kill journal pre-reservations
  bcachefs: track_event_change()
  bcachefs: Journal pins must always have a flush_fn
  bcachefs: BTREE_INSERT_JOURNAL_REPLAY now "don't init
    trans->journal_res"
  bcachefs: Kill BTREE_UPDATE_PREJOURNAL
  bcachefs: Go rw before journal replay
  bcachefs: Make journal replay more efficient
  bcachefs: Don't flush journal after replay
  bcachefs: Unwritten journal buffers are always dirty
  bcachefs: journal->buf_lock
  bcachefs: bch2_journal_block_reservations()
  bcachefs: Clean up btree write buffer write ref handling
  bcachefs: bch2_btree_write_buffer_flush_locked()
  bcachefs: bch2_btree_write_buffer_flush() ->
    bch2_btree_write_buffer_tryflush()
  bcachefs: Improve btree write buffer tracepoints
  bcachefs: btree write buffer now slurps keys from journal
  bcachefs: Inline btree write buffer sort

 fs/bcachefs/alloc_background.c         |   2 +-
 fs/bcachefs/bcachefs.h                 |  18 +-
 fs/bcachefs/bcachefs_format.h          |   3 +-
 fs/bcachefs/bkey_methods.h             |   2 -
 fs/bcachefs/btree_iter.c               |   2 -
 fs/bcachefs/btree_key_cache.c          |  14 -
 fs/bcachefs/btree_trans_commit.c       | 123 ++----
 fs/bcachefs/btree_types.h              |   4 -
 fs/bcachefs/btree_update.c             |  23 -
 fs/bcachefs/btree_update_interior.c    |  46 +-
 fs/bcachefs/btree_update_interior.h    |   1 -
 fs/bcachefs/btree_write_buffer.c       | 584 +++++++++++++++++--------
 fs/bcachefs/btree_write_buffer.h       |  48 +-
 fs/bcachefs/btree_write_buffer_types.h |  37 +-
 fs/bcachefs/ec.c                       |   2 +-
 fs/bcachefs/errcode.h                  |   2 -
 fs/bcachefs/inode.c                    |  10 +-
 fs/bcachefs/journal.c                  | 120 +++--
 fs/bcachefs/journal.h                  | 102 +----
 fs/bcachefs/journal_io.c               |  57 ++-
 fs/bcachefs/journal_reclaim.c          |  66 +--
 fs/bcachefs/journal_reclaim.h          |   1 +
 fs/bcachefs/journal_types.h            |  40 +-
 fs/bcachefs/move.c                     |   7 +-
 fs/bcachefs/movinggc.c                 |   4 +-
 fs/bcachefs/recovery.c                 | 106 +++--
 fs/bcachefs/super.c                    |   6 +-
 fs/bcachefs/trace.h                    |  42 +-
 fs/bcachefs/util.c                     | 140 +++---
 fs/bcachefs/util.h                     |  33 +-
 30 files changed, 925 insertions(+), 720 deletions(-)

-- 
2.42.0