public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups
@ 2024-11-14  0:16 Darrick J. Wong
  2024-11-14  0:17 ` [GIT PULL 01/10] xfs: convert perag to use xarrays Darrick J. Wong
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:16 UTC (permalink / raw)
  To: Carlos Maiolino; +Cc: xfs, Christoph Hellwig

Hi Carlos,

Please pull these fully reviewed patchsets for kernel 6.13.  This
patchbomb corrects numerous paperwork errors in the v5.5 submission so
that we can pass linux-next linter.

I have corrected my stgit wrapper program to invoke git checkpatch
before issuing pull requests.  Despite its name, this means I now have
automated checks for tagging errors in git commits.  Freeform text
fields that require a lot of parsing cleverness to check and that can be
corrupted easily buc*********.

The following excerpted range diff shows the differences between last
week's PRs and this week's:

  1:  62027820eb4486 !   1:  03326f42d6ef7a xfs: fix simplify extent lookup in xfs_can_free_eofblocks
    @@ Commit message
         this patch, we'd invoke xfs_free_eofblocks on first close if anything
         was in the CoW fork.  Now we don't do that.
     
         Fix the problem by reverting the removal of the i_delayed_blks check.
     
         Cc: <stable@vger.kernel.org> # v6.12-rc1
    -    Fixes: 11f4c3a53adde ("xfs: simplify extent lookup in xfs_can_free_eofblocks")
    +    Fixes: 11f4c3a53adde1 ("xfs: simplify extent lookup in xfs_can_free_eofblocks")
         Signed-off-by: Darrick J. Wong <djwong@kernel.org>
         Reviewed-by: Christoph Hellwig <hch@lst.de>
     
      ## fs/xfs/xfs_bmap_util.c ##
     @@ fs/xfs/xfs_bmap_util.c: xfs_can_free_eofblocks(
      		end_fsb = xfs_rtb_roundup_rtx(mp, end_fsb);
  2:  cd8ae42a82d2d7 =   2:  5bbfaf522b8c42 xfs: fix superfluous clearing of info->low in __xfs_getfsmap_datadev
...
 68:  dcfc65befb76df !  68:  3065c8cf8c7082 xfs: clean up xfs_getfsmap_helper arguments
    @@ Commit message
         fsmap irec structure that contains exactly the data we need, once.
     
         Note that we actually do need rm_startblock for rmap key comparisons
         when we're actually querying an rmap btree, so leave that field but
         document why it's there.
     
    -    Signed-off-by: Darrick J. Wong <djwong@kernel.org>
    +    Signed-off-by: Christoph Hellwig <hch@lst.de>
         Reviewed-by: Darrick J. Wong <djwong@kernel.org>
    +    [djwong: fix the SoB tag from hch, somehow my scripts replaced it...]
         Signed-off-by: Darrick J. Wong <djwong@kernel.org>
     
      ## fs/xfs/xfs_fsmap.c ##
     @@ fs/xfs/xfs_fsmap.c: xfs_fsmap_owner_to_rmap(
      	}
      	return 0;
 69:  87fe4c34a383d5 =  69:  15165713e812d9 xfs: create incore realtime group structures
...
 83:  1029f08dc53920 !  83:  08ed382bba4f52 xfs: factor out a xfs_growfs_rt_alloc_fake_mount helper
    @@ Commit message
     
         Note that this changes the rmblocks calculation method to be based
         on the passed in rblocks and extsize and not the explicitly passed
         one, but both methods will always lead to the same result.  The new
         version just does a little bit more math while being more general.
     
    -    Signed-off-by: Christoph Hellwig <hch@lst.de>
    -    Reviewed-by: Darrick J. Wong <djwong@kernel.org>
         Signed-off-by: Darrick J. Wong <djwong@kernel.org>
    +    Reviewed-by: Christoph Hellwig <hch@lst.de>
     
      ## fs/xfs/xfs_rtalloc.c ##
     @@ fs/xfs/xfs_rtalloc.c: xfs_rtginode_ensure(
      
      	if (error != -ENOENT)
      		return 0;
 84:  fc233f1fb0588a =  84:  bc1cc1849a4bfe xfs: use xfs_growfs_rt_alloc_fake_mount in xfs_growfs_rt_alloc_blocks
...
110:  b91afef724710e ! 110:  2a81329aa08d66 xfs: don't merge ioends across RTGs
    @@ Commit message
         Unlike AGs, RTGs don't always have metadata in their first blocks, and
         thus we don't get automatic protection from merging I/O completions
         across RTG boundaries.  Add code to set the IOMAP_F_BOUNDARY flag for
         ioends that start at the first block of a RTG so that they never get
         merged into the previous ioend.
     
    -    Signed-off-by: Christoph Hellwig <hch@lst.de>
    -    Reviewed-by: Darrick J. Wong <djwong@kernel.org>
         Signed-off-by: Darrick J. Wong <djwong@kernel.org>
    +    Reviewed-by: Christoph Hellwig <hch@lst.de>
     
      ## fs/xfs/libxfs/xfs_rtgroup.h ##
     @@ fs/xfs/libxfs/xfs_rtgroup.h: xfs_rtb_to_rgbno(
      	struct xfs_mount	*mp,
      	xfs_rtblock_t		rtbno)
      {
111:  d162491c5459f4 = 111:  4895d7326c2f49 xfs: make the RT allocator rtgroup aware
...
139:  13877bc79d8135 = 139:  b038df088d5bf2 xfs: port ondisk structure checks from xfs/122 to the kernel

Apologies for the last minute churn and paperwork stress.

--D

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 01/10] xfs: convert perag to use xarrays
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
@ 2024-11-14  0:17 ` Darrick J. Wong
  2024-11-14  0:18 ` [GIT PULL 02/10] xfs: create a generic allocation group structure Darrick J. Wong
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:17 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs, stable

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 59b723cd2adbac2a34fc8e12c74ae26ae45bf230:

Linux 6.12-rc6 (2024-11-03 14:05:52 -1000)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/perag-xarray-6.13_2024-11-13

for you to fetch changes up to ab2d77da259c91c00940f7638a33af57f82af0f6:

xfs: insert the pag structures into the xarray later (2024-11-13 16:05:20 -0800)

----------------------------------------------------------------
xfs: convert perag to use xarrays [v5.6 01/10]

Convert the xfs_mount perag tree to use an xarray instead of a radix
tree.  There should be no functional changes here.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Christoph Hellwig (22):
xfs: fix superfluous clearing of info->low in __xfs_getfsmap_datadev
xfs: remove the unused pagb_count field in struct xfs_perag
xfs: remove the unused pag_active_wq field in struct xfs_perag
xfs: pass a pag to xfs_difree_inode_chunk
xfs: remove the agno argument to xfs_free_ag_extent
xfs: add xfs_agbno_to_fsb and xfs_agbno_to_daddr helpers
xfs: add a xfs_agino_to_ino helper
xfs: pass a pag to xfs_extent_busy_{search,reuse}
xfs: keep a reference to the pag for busy extents
xfs: remove the mount field from struct xfs_busy_extents
xfs: remove the unused trace_xfs_iwalk_ag trace point
xfs: remove the unused xrep_bmap_walk_rmap trace point
xfs: constify pag arguments to trace points
xfs: pass a perag structure to the xfs_ag_resv_init_error trace point
xfs: pass objects to the xfs_irec_merge_{pre,post} trace points
xfs: pass the iunlink item to the xfs_iunlink_update_dinode trace point
xfs: pass objects to the xrep_ibt_walk_rmap tracepoint
xfs: pass the pag to the trace_xrep_calc_ag_resblks{,_btsize} trace points
xfs: pass the pag to the xrep_newbt_extent_class tracepoints
xfs: convert remaining trace points to pass pag structures
xfs: split xfs_initialize_perag
xfs: insert the pag structures into the xarray later

Darrick J. Wong (1):
xfs: fix simplify extent lookup in xfs_can_free_eofblocks

fs/xfs/libxfs/xfs_ag.c             | 135 ++++++++++++++------------
fs/xfs/libxfs/xfs_ag.h             |  30 +++++-
fs/xfs/libxfs/xfs_ag_resv.c        |   3 +-
fs/xfs/libxfs/xfs_alloc.c          |  32 +++----
fs/xfs/libxfs/xfs_alloc.h          |   5 +-
fs/xfs/libxfs/xfs_alloc_btree.c    |   2 +-
fs/xfs/libxfs/xfs_btree.c          |   7 +-
fs/xfs/libxfs/xfs_ialloc.c         |  67 ++++++-------
fs/xfs/libxfs/xfs_ialloc_btree.c   |   2 +-
fs/xfs/libxfs/xfs_inode_util.c     |   4 +-
fs/xfs/libxfs/xfs_refcount.c       |  11 +--
fs/xfs/libxfs/xfs_refcount_btree.c |   3 +-
fs/xfs/libxfs/xfs_rmap_btree.c     |   2 +-
fs/xfs/scrub/agheader_repair.c     |  16 +---
fs/xfs/scrub/alloc_repair.c        |  10 +-
fs/xfs/scrub/bmap.c                |   5 +-
fs/xfs/scrub/bmap_repair.c         |   4 +-
fs/xfs/scrub/common.c              |   2 +-
fs/xfs/scrub/cow_repair.c          |  18 ++--
fs/xfs/scrub/ialloc.c              |   8 +-
fs/xfs/scrub/ialloc_repair.c       |  25 ++---
fs/xfs/scrub/newbt.c               |  46 ++++-----
fs/xfs/scrub/reap.c                |   8 +-
fs/xfs/scrub/refcount_repair.c     |   5 +-
fs/xfs/scrub/repair.c              |  13 ++-
fs/xfs/scrub/rmap_repair.c         |   9 +-
fs/xfs/scrub/trace.h               | 161 +++++++++++++++----------------
fs/xfs/xfs_bmap_util.c             |   8 +-
fs/xfs/xfs_buf_item_recover.c      |   5 +-
fs/xfs/xfs_discard.c               |  20 ++--
fs/xfs/xfs_extent_busy.c           |  31 +++---
fs/xfs/xfs_extent_busy.h           |  14 ++-
fs/xfs/xfs_extfree_item.c          |   4 +-
fs/xfs/xfs_filestream.c            |   5 +-
fs/xfs/xfs_fsmap.c                 |  25 ++---
fs/xfs/xfs_health.c                |   8 +-
fs/xfs/xfs_inode.c                 |   5 +-
fs/xfs/xfs_iunlink_item.c          |  13 ++-
fs/xfs/xfs_iwalk.c                 |  17 ++--
fs/xfs/xfs_log_cil.c               |   3 +-
fs/xfs/xfs_log_recover.c           |   5 +-
fs/xfs/xfs_trace.c                 |   1 +
fs/xfs/xfs_trace.h                 | 191 ++++++++++++++++---------------------
fs/xfs/xfs_trans.c                 |   2 +-
44 files changed, 459 insertions(+), 531 deletions(-)


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 02/10] xfs: create a generic allocation group structure
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
  2024-11-14  0:17 ` [GIT PULL 01/10] xfs: convert perag to use xarrays Darrick J. Wong
@ 2024-11-14  0:18 ` Darrick J. Wong
  2024-11-14  0:18 ` [GIT PULL 03/10] xfs: metadata inode directory trees Darrick J. Wong
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:18 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit ab2d77da259c91c00940f7638a33af57f82af0f6:

xfs: insert the pag structures into the xarray later (2024-11-13 16:05:20 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/generic-groups-6.13_2024-11-13

for you to fetch changes up to 59e1e288697a96f2a9b71a87868f39699a84fbe0:

xfs: store a generic group structure in the intents (2024-11-13 16:05:23 -0800)

----------------------------------------------------------------
xfs: create a generic allocation group structure [v5.6 02/10]

Soon we'll be sharding the realtime volume into separate allocation
groups.  These rt groups will /mostly/ behave the same as the ones on
the data device, but since rt groups don't have quite the same set of
struct fields as perags, let's hoist the parts that will be shared by
both into a common xfs_group object.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Christoph Hellwig (16):
xfs: factor out a xfs_iwalk_args helper
xfs: factor out a generic xfs_group structure
xfs: add a xfs_group_next_range helper
xfs: switch perag iteration from the for_each macros to a while based iterator
xfs: move metadata health tracking to the generic group structure
xfs: mark xfs_perag_intent_{hold,rele} static
xfs: move draining of deferred operations to the generic group structure
xfs: move the online repair rmap hooks to the generic group structure
xfs: return the busy generation from xfs_extent_busy_list_empty
xfs: convert extent busy tracepoints to the generic group structure
xfs: convert busy extent tracking to the generic group structure
xfs: add a generic group pointer to the btree cursor
xfs: store a generic xfs_group pointer in xfs_getfsmap_info
xfs: add group based bno conversion helpers
xfs: remove xfs_group_intent_hold and xfs_group_intent_rele
xfs: store a generic group structure in the intents

fs/xfs/Makefile                    |   1 +
fs/xfs/libxfs/xfs_ag.c             | 149 +++++------------------
fs/xfs/libxfs/xfs_ag.h             | 165 +++++++++++++------------
fs/xfs/libxfs/xfs_ag_resv.c        |  19 +--
fs/xfs/libxfs/xfs_alloc.c          |  74 ++++++------
fs/xfs/libxfs/xfs_alloc.h          |   2 +-
fs/xfs/libxfs/xfs_alloc_btree.c    |  30 ++---
fs/xfs/libxfs/xfs_bmap.c           |   2 +-
fs/xfs/libxfs/xfs_bmap.h           |   2 +-
fs/xfs/libxfs/xfs_btree.c          |  37 ++----
fs/xfs/libxfs/xfs_btree.h          |   3 +-
fs/xfs/libxfs/xfs_btree_mem.c      |   6 +-
fs/xfs/libxfs/xfs_group.c          | 225 +++++++++++++++++++++++++++++++++++
fs/xfs/libxfs/xfs_group.h          | 131 ++++++++++++++++++++
fs/xfs/libxfs/xfs_health.h         |  45 +++----
fs/xfs/libxfs/xfs_ialloc.c         |  50 ++++----
fs/xfs/libxfs/xfs_ialloc_btree.c   |  27 +++--
fs/xfs/libxfs/xfs_refcount.c       |  26 ++--
fs/xfs/libxfs/xfs_refcount.h       |   2 +-
fs/xfs/libxfs/xfs_refcount_btree.c |  14 +--
fs/xfs/libxfs/xfs_rmap.c           |  42 +++----
fs/xfs/libxfs/xfs_rmap.h           |   6 +-
fs/xfs/libxfs/xfs_rmap_btree.c     |  28 ++---
fs/xfs/libxfs/xfs_sb.c             |  28 +++--
fs/xfs/libxfs/xfs_types.c          |   5 +-
fs/xfs/libxfs/xfs_types.h          |   8 ++
fs/xfs/scrub/agheader_repair.c     |  22 ++--
fs/xfs/scrub/alloc.c               |   2 +-
fs/xfs/scrub/alloc_repair.c        |  12 +-
fs/xfs/scrub/bmap.c                |   8 +-
fs/xfs/scrub/bmap_repair.c         |   9 +-
fs/xfs/scrub/common.c              |   4 +-
fs/xfs/scrub/common.h              |   3 +-
fs/xfs/scrub/cow_repair.c          |   9 +-
fs/xfs/scrub/fscounters.c          |  10 +-
fs/xfs/scrub/health.c              |  21 ++--
fs/xfs/scrub/ialloc.c              |  14 +--
fs/xfs/scrub/ialloc_repair.c       |   2 +-
fs/xfs/scrub/inode_repair.c        |   5 +-
fs/xfs/scrub/iscan.c               |   4 +-
fs/xfs/scrub/newbt.c               |   8 +-
fs/xfs/scrub/reap.c                |   2 +-
fs/xfs/scrub/refcount.c            |   3 +-
fs/xfs/scrub/repair.c              |   4 +-
fs/xfs/scrub/rmap.c                |   4 +-
fs/xfs/scrub/rmap_repair.c         |  16 +--
fs/xfs/scrub/trace.h               |  82 ++++++-------
fs/xfs/xfs_bmap_item.c             |   5 +-
fs/xfs/xfs_discard.c               |  20 ++--
fs/xfs/xfs_drain.c                 |  78 +++++-------
fs/xfs/xfs_drain.h                 |  22 ++--
fs/xfs/xfs_extent_busy.c           | 201 +++++++++++++++++++------------
fs/xfs/xfs_extent_busy.h           |  59 ++++-----
fs/xfs/xfs_extfree_item.c          |  14 ++-
fs/xfs/xfs_filestream.c            |   8 +-
fs/xfs/xfs_fsmap.c                 |  49 ++++----
fs/xfs/xfs_fsops.c                 |  10 +-
fs/xfs/xfs_health.c                |  99 +++++++--------
fs/xfs/xfs_icache.c                |  60 ++++------
fs/xfs/xfs_inode.c                 |   6 +-
fs/xfs/xfs_iwalk.c                 |  99 +++++++--------
fs/xfs/xfs_iwalk.h                 |   7 +-
fs/xfs/xfs_log_recover.c           |  11 +-
fs/xfs/xfs_mount.h                 |  36 +++++-
fs/xfs/xfs_refcount_item.c         |   9 +-
fs/xfs/xfs_reflink.c               |   7 +-
fs/xfs/xfs_rmap_item.c             |   9 +-
fs/xfs/xfs_super.c                 |  11 +-
fs/xfs/xfs_trace.c                 |   1 +
fs/xfs/xfs_trace.h                 | 238 ++++++++++++++++++++++---------------
70 files changed, 1380 insertions(+), 1050 deletions(-)
create mode 100644 fs/xfs/libxfs/xfs_group.c
create mode 100644 fs/xfs/libxfs/xfs_group.h


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 03/10] xfs: metadata inode directory trees
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
  2024-11-14  0:17 ` [GIT PULL 01/10] xfs: convert perag to use xarrays Darrick J. Wong
  2024-11-14  0:18 ` [GIT PULL 02/10] xfs: create a generic allocation group structure Darrick J. Wong
@ 2024-11-14  0:18 ` Darrick J. Wong
  2024-11-14  0:18 ` [GIT PULL 04/10] xfs: create incore rt allocation groups Darrick J. Wong
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:18 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 59e1e288697a96f2a9b71a87868f39699a84fbe0:

xfs: store a generic group structure in the intents (2024-11-13 16:05:23 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/metadata-directory-tree-6.13_2024-11-13

for you to fetch changes up to ffc12626fd0ee4fc7c5f1c91d33b452e6d757acd:

xfs: repair metadata directory file path connectivity (2024-11-13 16:05:28 -0800)

----------------------------------------------------------------
xfs: metadata inode directory trees [v5.6 03/10]

This series delivers a new feature -- metadata inode directories.  This
is a separate directory tree (rooted in the superblock) that contains
only inodes that contain filesystem metadata.  Different metadata
objects can be looked up with regular paths.

Start by creating xfs_imeta{dir,file}* functions to mediate access to
the metadata directory tree.  By the end of this mega series, all
existing metadata inodes (rt+quota) will use this directory tree instead
of the superblock.

Next, define the metadir on-disk format, which consists of marking
inodes with a new iflag that says they're metadata.  This prevents
bulkstat and friends from ever getting their hands on fs metadata files.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (28):
xfs: constify the xfs_sb predicates
xfs: constify the xfs_inode predicates
xfs: rename metadata inode predicates
xfs: standardize EXPERIMENTAL warning generation
xfs: define the on-disk format for the metadir feature
xfs: iget for metadata inodes
xfs: load metadata directory root at mount time
xfs: enforce metadata inode flag
xfs: read and write metadata inode directory tree
xfs: disable the agi rotor for metadata inodes
xfs: hide metadata inodes from everyone because they are special
xfs: advertise metadata directory feature
xfs: allow bulkstat to return metadata directories
xfs: don't count metadata directory files to quota
xfs: mark quota inodes as metadata files
xfs: adjust xfs_bmap_add_attrfork for metadir
xfs: record health problems with the metadata directory
xfs: refactor directory tree root predicates
xfs: do not count metadata directory files when doing online quotacheck
xfs: metadata files can have xattrs if metadir is enabled
xfs: adjust parent pointer scrubber for sb-rooted metadata files
xfs: fix di_metatype field of inodes that won't load
xfs: scrub metadata directories
xfs: check the metadata directory inumber in superblocks
xfs: move repair temporary files to the metadata directory tree
xfs: check metadata directory file path connectivity
xfs: confirm dotdot target before replacing it during a repair
xfs: repair metadata directory file path connectivity

fs/xfs/Makefile                 |   5 +-
fs/xfs/libxfs/xfs_attr.c        |   5 +-
fs/xfs/libxfs/xfs_bmap.c        |   5 +-
fs/xfs/libxfs/xfs_format.h      | 121 ++++++++--
fs/xfs/libxfs/xfs_fs.h          |  25 +-
fs/xfs/libxfs/xfs_health.h      |   6 +-
fs/xfs/libxfs/xfs_ialloc.c      |  58 +++--
fs/xfs/libxfs/xfs_inode_buf.c   |  90 ++++++-
fs/xfs/libxfs/xfs_inode_buf.h   |   3 +
fs/xfs/libxfs/xfs_inode_util.c  |   2 +
fs/xfs/libxfs/xfs_log_format.h  |   2 +-
fs/xfs/libxfs/xfs_metadir.c     | 481 +++++++++++++++++++++++++++++++++++++
fs/xfs/libxfs/xfs_metadir.h     |  47 ++++
fs/xfs/libxfs/xfs_metafile.c    |  52 ++++
fs/xfs/libxfs/xfs_metafile.h    |  31 +++
fs/xfs/libxfs/xfs_ondisk.h      |   2 +-
fs/xfs/libxfs/xfs_sb.c          |  12 +
fs/xfs/libxfs/xfs_types.c       |   4 +-
fs/xfs/libxfs/xfs_types.h       |   2 +-
fs/xfs/scrub/agheader.c         |   5 +
fs/xfs/scrub/common.c           |  65 ++++-
fs/xfs/scrub/common.h           |   5 +
fs/xfs/scrub/dir.c              |  10 +-
fs/xfs/scrub/dir_repair.c       |  20 +-
fs/xfs/scrub/dirtree.c          |  32 ++-
fs/xfs/scrub/dirtree.h          |  12 +-
fs/xfs/scrub/findparent.c       |  28 ++-
fs/xfs/scrub/health.c           |   1 +
fs/xfs/scrub/inode.c            |  35 ++-
fs/xfs/scrub/inode_repair.c     |  34 ++-
fs/xfs/scrub/metapath.c         | 521 ++++++++++++++++++++++++++++++++++++++++
fs/xfs/scrub/nlinks.c           |   4 +-
fs/xfs/scrub/nlinks_repair.c    |   4 +-
fs/xfs/scrub/orphanage.c        |   4 +-
fs/xfs/scrub/parent.c           |  39 ++-
fs/xfs/scrub/parent_repair.c    |  37 ++-
fs/xfs/scrub/quotacheck.c       |   7 +-
fs/xfs/scrub/refcount_repair.c  |   2 +-
fs/xfs/scrub/repair.c           |  14 +-
fs/xfs/scrub/repair.h           |   3 +
fs/xfs/scrub/scrub.c            |  12 +-
fs/xfs/scrub/scrub.h            |   2 +
fs/xfs/scrub/stats.c            |   1 +
fs/xfs/scrub/tempfile.c         | 105 ++++++++
fs/xfs/scrub/tempfile.h         |   3 +
fs/xfs/scrub/trace.c            |   1 +
fs/xfs/scrub/trace.h            |  42 +++-
fs/xfs/xfs_dquot.c              |   1 +
fs/xfs/xfs_fsops.c              |   4 +-
fs/xfs/xfs_health.c             |   2 +
fs/xfs/xfs_icache.c             |  74 ++++++
fs/xfs/xfs_inode.c              |  19 +-
fs/xfs/xfs_inode.h              |  36 ++-
fs/xfs/xfs_inode_item.c         |   7 +-
fs/xfs/xfs_inode_item_recover.c |   2 +-
fs/xfs/xfs_ioctl.c              |   7 +
fs/xfs/xfs_iops.c               |  15 +-
fs/xfs/xfs_itable.c             |  33 ++-
fs/xfs/xfs_itable.h             |   3 +
fs/xfs/xfs_message.c            |  51 ++++
fs/xfs/xfs_message.h            |  20 +-
fs/xfs/xfs_mount.c              |  31 ++-
fs/xfs/xfs_mount.h              |  25 +-
fs/xfs/xfs_pnfs.c               |   3 +-
fs/xfs/xfs_qm.c                 |  36 ++-
fs/xfs/xfs_quota.h              |   5 +
fs/xfs/xfs_rtalloc.c            |  38 +--
fs/xfs/xfs_super.c              |  13 +-
fs/xfs/xfs_trace.c              |   2 +
fs/xfs/xfs_trace.h              | 102 ++++++++
fs/xfs/xfs_trans_dquot.c        |   6 +
fs/xfs/xfs_xattr.c              |   3 +-
72 files changed, 2333 insertions(+), 206 deletions(-)
create mode 100644 fs/xfs/libxfs/xfs_metadir.c
create mode 100644 fs/xfs/libxfs/xfs_metadir.h
create mode 100644 fs/xfs/libxfs/xfs_metafile.c
create mode 100644 fs/xfs/libxfs/xfs_metafile.h
create mode 100644 fs/xfs/scrub/metapath.c


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 04/10] xfs: create incore rt allocation groups
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (2 preceding siblings ...)
  2024-11-14  0:18 ` [GIT PULL 03/10] xfs: metadata inode directory trees Darrick J. Wong
@ 2024-11-14  0:18 ` Darrick J. Wong
  2024-11-14  0:18 ` [GIT PULL 05/10] xfs: preparation for realtime " Darrick J. Wong
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:18 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit ffc12626fd0ee4fc7c5f1c91d33b452e6d757acd:

xfs: repair metadata directory file path connectivity (2024-11-13 16:05:28 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/incore-rtgroups-6.13_2024-11-13

for you to fetch changes up to dd2a92877987e615f5b62f872ee0d4f992362994:

xfs: make RT extent numbers relative to the rtgroup (2024-11-13 16:05:31 -0800)

----------------------------------------------------------------
xfs: create incore rt allocation groups [v5.6 04/10]

Add in-memory data structures for sharding the realtime volume into
independent allocation groups.  For existing filesystems, the entire rt
volume is modelled as having a single large group, with (potentially) a
number of rt extents exceeding 2^32 blocks, though these are not likely
to exist because the codebase has been a bit broken for decades.  The
next series fills in the ondisk format and other supporting structures.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Christoph Hellwig (14):
xfs: clean up xfs_getfsmap_helper arguments
xfs: add a xfs_bmap_free_rtblocks helper
xfs: add a xfs_qm_unmount_rt helper
xfs: factor out a xfs_growfs_rt_alloc_blocks helper
xfs: cleanup xfs_getfsmap_rtdev_rtbitmap
xfs: split xfs_trim_rtdev_extents
xfs: move RT bitmap and summary information to the rtgroup
xfs: support creating per-RTG files in growfs
xfs: calculate RT bitmap and summary blocks based on sb_rextents
xfs: use xfs_growfs_rt_alloc_fake_mount in xfs_growfs_rt_alloc_blocks
xfs: factor out a xfs_growfs_check_rtgeom helper
xfs: refactor xfs_rtbitmap_blockcount
xfs: refactor xfs_rtsummary_blockcount
xfs: make RT extent numbers relative to the rtgroup

Darrick J. Wong (7):
xfs: create incore realtime group structures
xfs: define locking primitives for realtime groups
xfs: add a lockdep class key for rtgroup inodes
xfs: support caching rtgroup metadata inodes
xfs: add rtgroup-based realtime scrubbing context management
xfs: remove XFS_ILOCK_RT*
xfs: factor out a xfs_growfs_rt_alloc_fake_mount helper

fs/xfs/Makefile                 |   1 +
fs/xfs/libxfs/xfs_bmap.c        |  46 ++--
fs/xfs/libxfs/xfs_format.h      |   3 +
fs/xfs/libxfs/xfs_rtbitmap.c    | 199 +++++++--------
fs/xfs/libxfs/xfs_rtbitmap.h    | 147 ++++++------
fs/xfs/libxfs/xfs_rtgroup.c     | 484 +++++++++++++++++++++++++++++++++++++
fs/xfs/libxfs/xfs_rtgroup.h     | 274 +++++++++++++++++++++
fs/xfs/libxfs/xfs_sb.c          |  13 +
fs/xfs/libxfs/xfs_trans_resv.c  |   2 +-
fs/xfs/libxfs/xfs_types.h       |   8 +-
fs/xfs/scrub/bmap.c             |  13 +
fs/xfs/scrub/common.c           |  78 ++++++
fs/xfs/scrub/common.h           |  30 +++
fs/xfs/scrub/fscounters.c       |  25 +-
fs/xfs/scrub/repair.c           |  24 ++
fs/xfs/scrub/repair.h           |   7 +
fs/xfs/scrub/rtbitmap.c         |  54 +++--
fs/xfs/scrub/rtsummary.c        | 111 +++++----
fs/xfs/scrub/rtsummary_repair.c |   7 +-
fs/xfs/scrub/scrub.c            |  33 ++-
fs/xfs/scrub/scrub.h            |  13 +
fs/xfs/xfs_bmap_util.c          |   3 +-
fs/xfs/xfs_buf_item_recover.c   |  25 ++
fs/xfs/xfs_discard.c            | 100 +++++---
fs/xfs/xfs_fsmap.c              | 329 ++++++++++++++-----------
fs/xfs/xfs_fsmap.h              |  15 ++
fs/xfs/xfs_inode.c              |   3 +-
fs/xfs/xfs_inode.h              |  13 +-
fs/xfs/xfs_iomap.c              |   4 +-
fs/xfs/xfs_mount.c              |  15 +-
fs/xfs/xfs_mount.h              |  26 +-
fs/xfs/xfs_qm.c                 |  27 ++-
fs/xfs/xfs_rtalloc.c            | 520 +++++++++++++++++++++++++---------------
fs/xfs/xfs_super.c              |   3 +-
fs/xfs/xfs_trace.c              |   1 +
fs/xfs/xfs_trace.h              |  74 ++++--
36 files changed, 2020 insertions(+), 710 deletions(-)
create mode 100644 fs/xfs/libxfs/xfs_rtgroup.c
create mode 100644 fs/xfs/libxfs/xfs_rtgroup.h


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 05/10] xfs: preparation for realtime allocation groups
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (3 preceding siblings ...)
  2024-11-14  0:18 ` [GIT PULL 04/10] xfs: create incore rt allocation groups Darrick J. Wong
@ 2024-11-14  0:18 ` Darrick J. Wong
  2024-11-14  0:19 ` [GIT PULL 06/10] xfs: shard the realtime section Darrick J. Wong
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:18 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-fsdevel, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit dd2a92877987e615f5b62f872ee0d4f992362994:

xfs: make RT extent numbers relative to the rtgroup (2024-11-13 16:05:31 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/rtgroups-prep-6.13_2024-11-13

for you to fetch changes up to 320c28fb42f571064ddc9b769dfaa3a59c105677:

iomap: add a merge boundary flag (2024-11-13 16:05:32 -0800)

----------------------------------------------------------------
xfs: preparation for realtime allocation groups [v5.6 05/10]

Prepare for realtime groups by adding a few bug fixes and generic code
that will be necessary.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Christoph Hellwig (1):
iomap: add a merge boundary flag

Darrick J. Wong (1):
xfs: fix rt device offset calculations for FITRIM

fs/iomap/buffered-io.c |  6 ++++++
fs/xfs/xfs_discard.c   | 19 +++++++++++--------
include/linux/iomap.h  |  4 ++++
3 files changed, 21 insertions(+), 8 deletions(-)


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 06/10] xfs: shard the realtime section
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (4 preceding siblings ...)
  2024-11-14  0:18 ` [GIT PULL 05/10] xfs: preparation for realtime " Darrick J. Wong
@ 2024-11-14  0:19 ` Darrick J. Wong
  2024-11-14  0:19 ` [GIT PULL 07/10] xfs: persist quota options with metadir Darrick J. Wong
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:19 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 320c28fb42f571064ddc9b769dfaa3a59c105677:

iomap: add a merge boundary flag (2024-11-13 16:05:32 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/realtime-groups-6.13_2024-11-13

for you to fetch changes up to 4e88a1158f8c167b47a0f115b74d4ffbc1c9901d:

xfs: use rtgroup busy extent list for FITRIM (2024-11-13 16:05:38 -0800)

----------------------------------------------------------------
xfs: shard the realtime section [v5.6 06/10]

Right now, the realtime section uses a single pair of metadata inodes to
store the free space information.  This presents a scalability problem
since every thread trying to allocate or free rt extents have to lock
these files.  Solve this problem by sharding the realtime section into
separate realtime allocation groups.

While we're at it, define a superblock to be stamped into the start of
the rt section.  This enables utilities such as blkid to identify block
devices containing realtime sections, and avoids the situation where
anything written into block 0 of the realtime extent can be
misinterpreted as file data.

The best advantage for rtgroups will become evident later when we get to
adding rmap and reflink to the realtime volume, since the geometry
constraints are the same for rt groups and AGs.  Hence we can reuse all
that code directly.

This is a very large patchset, but it catches us up with 20 years of
technical debt that have accumulated.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Christoph Hellwig (2):
xfs: add a helper to prevent bmap merges across rtgroup boundaries
xfs: make the RT allocator rtgroup aware

Darrick J. Wong (32):
xfs: define the format of rt groups
xfs: check the realtime superblock at mount time
xfs: update realtime super every time we update the primary fs super
xfs: export realtime group geometry via XFS_FSOP_GEOM
xfs: check that rtblock extents do not break rtsupers or rtgroups
xfs: add frextents to the lazysbcounters when rtgroups enabled
xfs: convert sick_map loops to use ARRAY_SIZE
xfs: record rt group metadata errors in the health system
xfs: export the geometry of realtime groups to userspace
xfs: add block headers to realtime bitmap and summary blocks
xfs: encode the rtbitmap in big endian format
xfs: encode the rtsummary in big endian format
xfs: grow the realtime section when realtime groups are enabled
xfs: store rtgroup information with a bmap intent
xfs: force swapext to a realtime file to use the file content exchange ioctl
xfs: support logging EFIs for realtime extents
xfs: support error injection when freeing rt extents
xfs: use realtime EFI to free extents when rtgroups are enabled
xfs: don't merge ioends across RTGs
xfs: don't coalesce file mappings that cross rtgroup boundaries in scrub
xfs: scrub the realtime group superblock
xfs: repair realtime group superblock
xfs: scrub metadir paths for rtgroup metadata
xfs: mask off the rtbitmap and summary inodes when metadir in use
xfs: create helpers to deal with rounding xfs_fileoff_t to rtx boundaries
xfs: create helpers to deal with rounding xfs_filblks_t to rtx boundaries
xfs: make xfs_rtblock_t a segmented address like xfs_fsblock_t
xfs: adjust min_block usage in xfs_verify_agbno
xfs: move the min and max group block numbers to xfs_group
xfs: port the perag discard code to handle generic groups
xfs: implement busy extent tracking for rtgroups
xfs: use rtgroup busy extent list for FITRIM

fs/xfs/Makefile                  |   1 +
fs/xfs/libxfs/xfs_ag.c           |  22 +-
fs/xfs/libxfs/xfs_ag.h           |  16 +-
fs/xfs/libxfs/xfs_alloc.c        |  15 +-
fs/xfs/libxfs/xfs_alloc.h        |  12 +-
fs/xfs/libxfs/xfs_bmap.c         |  84 +++++-
fs/xfs/libxfs/xfs_defer.c        |   6 +
fs/xfs/libxfs/xfs_defer.h        |   1 +
fs/xfs/libxfs/xfs_format.h       |  74 ++++-
fs/xfs/libxfs/xfs_fs.h           |  28 +-
fs/xfs/libxfs/xfs_group.h        |  33 +++
fs/xfs/libxfs/xfs_health.h       |  42 +--
fs/xfs/libxfs/xfs_ialloc_btree.c |   2 +-
fs/xfs/libxfs/xfs_log_format.h   |   6 +-
fs/xfs/libxfs/xfs_log_recover.h  |   2 +
fs/xfs/libxfs/xfs_ondisk.h       |   4 +-
fs/xfs/libxfs/xfs_rtbitmap.c     | 225 ++++++++++++---
fs/xfs/libxfs/xfs_rtbitmap.h     | 114 ++++++--
fs/xfs/libxfs/xfs_rtgroup.c      | 223 ++++++++++++++-
fs/xfs/libxfs/xfs_rtgroup.h      | 104 +++----
fs/xfs/libxfs/xfs_sb.c           | 232 ++++++++++++++--
fs/xfs/libxfs/xfs_sb.h           |   6 +-
fs/xfs/libxfs/xfs_shared.h       |   4 +
fs/xfs/libxfs/xfs_types.c        |  35 ++-
fs/xfs/scrub/agheader.c          |  15 +-
fs/xfs/scrub/agheader_repair.c   |   4 +-
fs/xfs/scrub/bmap.c              |  16 +-
fs/xfs/scrub/common.h            |   2 +
fs/xfs/scrub/fscounters_repair.c |   9 +-
fs/xfs/scrub/health.c            |  32 ++-
fs/xfs/scrub/metapath.c          |  92 +++++++
fs/xfs/scrub/repair.c            |   6 +-
fs/xfs/scrub/repair.h            |   3 +
fs/xfs/scrub/rgsuper.c           |  84 ++++++
fs/xfs/scrub/rtsummary.c         |   5 +
fs/xfs/scrub/rtsummary_repair.c  |  15 +-
fs/xfs/scrub/scrub.c             |   7 +
fs/xfs/scrub/scrub.h             |   2 +
fs/xfs/scrub/stats.c             |   1 +
fs/xfs/scrub/trace.h             |   4 +-
fs/xfs/xfs_bmap_item.c           |  25 +-
fs/xfs/xfs_bmap_util.c           |  18 +-
fs/xfs/xfs_buf_item_recover.c    |  37 ++-
fs/xfs/xfs_discard.c             | 187 ++++++++++++-
fs/xfs/xfs_exchrange.c           |   2 +-
fs/xfs/xfs_extent_busy.c         |   6 +
fs/xfs/xfs_extfree_item.c        | 270 +++++++++++++++---
fs/xfs/xfs_health.c              | 183 +++++++------
fs/xfs/xfs_ioctl.c               |  39 ++-
fs/xfs/xfs_iomap.c               |  13 +-
fs/xfs/xfs_log_recover.c         |   2 +
fs/xfs/xfs_mount.h               |  15 +-
fs/xfs/xfs_rtalloc.c             | 577 +++++++++++++++++++++++++++++++++++----
fs/xfs/xfs_rtalloc.h             |   6 +
fs/xfs/xfs_super.c               |  12 +-
fs/xfs/xfs_trace.h               | 150 +++++++---
fs/xfs/xfs_trans.c               |  64 +++--
fs/xfs/xfs_trans.h               |   2 +
fs/xfs/xfs_trans_buf.c           |  25 +-
59 files changed, 2704 insertions(+), 517 deletions(-)
create mode 100644 fs/xfs/scrub/rgsuper.c


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 07/10] xfs: persist quota options with metadir
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (5 preceding siblings ...)
  2024-11-14  0:19 ` [GIT PULL 06/10] xfs: shard the realtime section Darrick J. Wong
@ 2024-11-14  0:19 ` Darrick J. Wong
  2024-11-14  0:19 ` [GIT PULL 08/10] xfs: enable quota for realtime volumes Darrick J. Wong
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:19 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 4e88a1158f8c167b47a0f115b74d4ffbc1c9901d:

xfs: use rtgroup busy extent list for FITRIM (2024-11-13 16:05:38 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/metadir-quotas-6.13_2024-11-13

for you to fetch changes up to 7b9f1c2e842b1f493c4fd7e2c878c9fd785ec033:

xfs: persist quota flags with metadir (2024-11-13 16:05:38 -0800)

----------------------------------------------------------------
xfs: persist quota options with metadir [v5.6 07/10]

Store the quota files in the metadata directory tree instead of the
superblock.  Since we're introducing a new incompat feature flag, let's
also make the mount process bring up quotas in whatever state they were
when the filesystem was last unmounted, instead of requiring sysadmins
to remember that themselves.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (4):
xfs: refactor xfs_qm_destroy_quotainos
xfs: use metadir for quota inodes
xfs: scrub quota file metapaths
xfs: persist quota flags with metadir

fs/xfs/libxfs/xfs_dquot_buf.c  | 190 +++++++++++++++++++++++++++++++
fs/xfs/libxfs/xfs_fs.h         |   6 +-
fs/xfs/libxfs/xfs_quota_defs.h |  43 +++++++
fs/xfs/libxfs/xfs_sb.c         |   1 +
fs/xfs/scrub/metapath.c        |  76 +++++++++++++
fs/xfs/xfs_mount.c             |  15 +++
fs/xfs/xfs_mount.h             |  21 +++-
fs/xfs/xfs_qm.c                | 250 +++++++++++++++++++++++++++++++----------
fs/xfs/xfs_qm_bhv.c            |  18 +++
fs/xfs/xfs_quota.h             |   2 +
fs/xfs/xfs_super.c             |  25 ++++-
11 files changed, 586 insertions(+), 61 deletions(-)


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 08/10] xfs: enable quota for realtime volumes
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (6 preceding siblings ...)
  2024-11-14  0:19 ` [GIT PULL 07/10] xfs: persist quota options with metadir Darrick J. Wong
@ 2024-11-14  0:19 ` Darrick J. Wong
  2024-11-14  0:19 ` [GIT PULL 09/10] xfs: enable metadir Darrick J. Wong
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:19 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 7b9f1c2e842b1f493c4fd7e2c878c9fd785ec033:

xfs: persist quota flags with metadir (2024-11-13 16:05:38 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/realtime-quotas-6.13_2024-11-13

for you to fetch changes up to 72eae9ba15798bdea27ebd975323a32c3319827a:

xfs: enable realtime quota again (2024-11-13 16:05:39 -0800)

----------------------------------------------------------------
xfs: enable quota for realtime volumes [v5.6 08/10]

At some point, I realized that I've refactored enough of the quota code
in XFS that I should evaluate whether or not quota actually works on
realtime volumes.  It turns out that it nearly works: the only broken
pieces are chown and delayed allocation, and reporting of project
quotas in the statvfs output for projinherit+rtinherit directories.

Fix these things and we can have realtime quotas again after 20 years.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (6):
xfs: fix chown with rt quota
xfs: advertise realtime quota support in the xqm stat files
xfs: report realtime block quota limits on realtime directories
xfs: create quota preallocation watermarks for realtime quota
xfs: reserve quota for realtime files correctly
xfs: enable realtime quota again

fs/xfs/xfs_dquot.c       | 37 ++++++++++++++-----------
fs/xfs/xfs_dquot.h       | 18 +++++++++---
fs/xfs/xfs_iomap.c       | 37 ++++++++++++++++++++-----
fs/xfs/xfs_qm.c          | 72 +++++++++++++++++++++++++++++++-----------------
fs/xfs/xfs_qm_bhv.c      | 18 ++++++++----
fs/xfs/xfs_quota.h       | 12 ++++----
fs/xfs/xfs_rtalloc.c     |  4 ++-
fs/xfs/xfs_stats.c       |  7 +++--
fs/xfs/xfs_super.c       | 11 ++++----
fs/xfs/xfs_trans.c       | 31 +++++++++++++++++++--
fs/xfs/xfs_trans_dquot.c | 11 ++++++++
11 files changed, 182 insertions(+), 76 deletions(-)


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 09/10] xfs: enable metadir
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (7 preceding siblings ...)
  2024-11-14  0:19 ` [GIT PULL 08/10] xfs: enable quota for realtime volumes Darrick J. Wong
@ 2024-11-14  0:19 ` Darrick J. Wong
  2024-11-14  0:20 ` [GIT PULL 10/10] xfs: improve ondisk structure checks Darrick J. Wong
  2024-11-14  6:16 ` [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:19 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 72eae9ba15798bdea27ebd975323a32c3319827a:

xfs: enable realtime quota again (2024-11-13 16:05:39 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/metadir-6.13_2024-11-13

for you to fetch changes up to aca759a9b1afc6693cfaaafcad7546298423fd20:

xfs: enable metadata directory feature (2024-11-13 16:05:40 -0800)

----------------------------------------------------------------
xfs: enable metadir [v5.6 09/10]

Actually enable this very large feature, which adds metadata directory
trees, allocation groups on the realtime volume, persistent quota
options, and quota for realtime files.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (2):
xfs: update sb field checks when metadir is turned on
xfs: enable metadata directory feature

fs/xfs/libxfs/xfs_format.h |  3 ++-
fs/xfs/scrub/agheader.c    | 36 ++++++++++++++++++++++++------------
2 files changed, 26 insertions(+), 13 deletions(-)


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [GIT PULL 10/10] xfs: improve ondisk structure checks
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (8 preceding siblings ...)
  2024-11-14  0:19 ` [GIT PULL 09/10] xfs: enable metadir Darrick J. Wong
@ 2024-11-14  0:20 ` Darrick J. Wong
  2024-11-14  6:16 ` [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  0:20 UTC (permalink / raw)
  To: cem, djwong; +Cc: hch, linux-xfs

Hi Carlos,

Please pull this branch with changes for xfs for 6.13-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit aca759a9b1afc6693cfaaafcad7546298423fd20:

xfs: enable metadata directory feature (2024-11-13 16:05:40 -0800)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/better-ondisk-6.13_2024-11-13

for you to fetch changes up to b038df088d5bf221ac4f8c7fd20d7a60c9a9ddfc:

xfs: port ondisk structure checks from xfs/122 to the kernel (2024-11-13 16:05:40 -0800)

----------------------------------------------------------------
xfs: improve ondisk structure checks [v5.6 10/10]

Reorganize xfs_ondisk.h to group the build checks by type, then add a
bunch of missing checks that were in xfs/122 but not the build system.
With this, we can get rid of xfs/122.

With a bit of luck, this should all go splendidly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (3):
xfs: convert struct typedefs in xfs_ondisk.h
xfs: separate space btree structures in xfs_ondisk.h
xfs: port ondisk structure checks from xfs/122 to the kernel

fs/xfs/libxfs/xfs_ondisk.h | 186 +++++++++++++++++++++++++++++++++------------
1 file changed, 137 insertions(+), 49 deletions(-)


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups
  2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
                   ` (9 preceding siblings ...)
  2024-11-14  0:20 ` [GIT PULL 10/10] xfs: improve ondisk structure checks Darrick J. Wong
@ 2024-11-14  6:16 ` Darrick J. Wong
  10 siblings, 0 replies; 12+ messages in thread
From: Darrick J. Wong @ 2024-11-14  6:16 UTC (permalink / raw)
  To: Carlos Maiolino; +Cc: xfs, Christoph Hellwig

On Wed, Nov 13, 2024 at 04:16:37PM -0800, Darrick J. Wong wrote:
> Hi Carlos,
> 
> Please pull these fully reviewed patchsets for kernel 6.13.  This
> patchbomb corrects numerous paperwork errors in the v5.5 submission so
> that we can pass linux-next linter.
> 
> I have corrected my stgit wrapper program to invoke git checkpatch
> before issuing pull requests.  Despite its name, this means I now have
> automated checks for tagging errors in git commits.  Freeform text
> fields that require a lot of parsing cleverness to check and that can be
> corrupted easily buc*********.
> 
> The following excerpted range diff shows the differences between last
> week's PRs and this week's:
> 
>   1:  62027820eb4486 !   1:  03326f42d6ef7a xfs: fix simplify extent lookup in xfs_can_free_eofblocks
>     @@ Commit message
>          this patch, we'd invoke xfs_free_eofblocks on first close if anything
>          was in the CoW fork.  Now we don't do that.
>      
>          Fix the problem by reverting the removal of the i_delayed_blks check.
>      
>          Cc: <stable@vger.kernel.org> # v6.12-rc1
>     -    Fixes: 11f4c3a53adde ("xfs: simplify extent lookup in xfs_can_free_eofblocks")
>     +    Fixes: 11f4c3a53adde1 ("xfs: simplify extent lookup in xfs_can_free_eofblocks")
>          Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>          Reviewed-by: Christoph Hellwig <hch@lst.de>
>      
>       ## fs/xfs/xfs_bmap_util.c ##
>      @@ fs/xfs/xfs_bmap_util.c: xfs_can_free_eofblocks(
>       		end_fsb = xfs_rtb_roundup_rtx(mp, end_fsb);
>   2:  cd8ae42a82d2d7 =   2:  5bbfaf522b8c42 xfs: fix superfluous clearing of info->low in __xfs_getfsmap_datadev
> ...
>  68:  dcfc65befb76df !  68:  3065c8cf8c7082 xfs: clean up xfs_getfsmap_helper arguments
>     @@ Commit message
>          fsmap irec structure that contains exactly the data we need, once.
>      
>          Note that we actually do need rm_startblock for rmap key comparisons
>          when we're actually querying an rmap btree, so leave that field but
>          document why it's there.
>      
>     -    Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>     +    Signed-off-by: Christoph Hellwig <hch@lst.de>
>          Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>     +    [djwong: fix the SoB tag from hch, somehow my scripts replaced it...]
>          Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>      
>       ## fs/xfs/xfs_fsmap.c ##
>      @@ fs/xfs/xfs_fsmap.c: xfs_fsmap_owner_to_rmap(
>       	}
>       	return 0;
>  69:  87fe4c34a383d5 =  69:  15165713e812d9 xfs: create incore realtime group structures
> ...
>  83:  1029f08dc53920 !  83:  08ed382bba4f52 xfs: factor out a xfs_growfs_rt_alloc_fake_mount helper
>     @@ Commit message
>      
>          Note that this changes the rmblocks calculation method to be based
>          on the passed in rblocks and extsize and not the explicitly passed
>          one, but both methods will always lead to the same result.  The new
>          version just does a little bit more math while being more general.
>      
>     -    Signed-off-by: Christoph Hellwig <hch@lst.de>
>     -    Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>          Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>     +    Reviewed-by: Christoph Hellwig <hch@lst.de>

hch pointed out that I reversed the polarity on this change and the one
after it; both should be From/SOB him, and RVB me.  So I'll send a v5.7
with that fixed.  Never mind that it's 22:15 and I've been dragged back
to work because we're running the hell out of time.  I hate all this
petty bureaucratic shit that's hard to manage.

--D

>       ## fs/xfs/xfs_rtalloc.c ##
>      @@ fs/xfs/xfs_rtalloc.c: xfs_rtginode_ensure(
>       
>       	if (error != -ENOENT)
>       		return 0;
>  84:  fc233f1fb0588a =  84:  bc1cc1849a4bfe xfs: use xfs_growfs_rt_alloc_fake_mount in xfs_growfs_rt_alloc_blocks
> ...
> 110:  b91afef724710e ! 110:  2a81329aa08d66 xfs: don't merge ioends across RTGs
>     @@ Commit message
>          Unlike AGs, RTGs don't always have metadata in their first blocks, and
>          thus we don't get automatic protection from merging I/O completions
>          across RTG boundaries.  Add code to set the IOMAP_F_BOUNDARY flag for
>          ioends that start at the first block of a RTG so that they never get
>          merged into the previous ioend.
>      
>     -    Signed-off-by: Christoph Hellwig <hch@lst.de>
>     -    Reviewed-by: Darrick J. Wong <djwong@kernel.org>
>          Signed-off-by: Darrick J. Wong <djwong@kernel.org>
>     +    Reviewed-by: Christoph Hellwig <hch@lst.de>
>      
>       ## fs/xfs/libxfs/xfs_rtgroup.h ##
>      @@ fs/xfs/libxfs/xfs_rtgroup.h: xfs_rtb_to_rgbno(
>       	struct xfs_mount	*mp,
>       	xfs_rtblock_t		rtbno)
>       {
> 111:  d162491c5459f4 = 111:  4895d7326c2f49 xfs: make the RT allocator rtgroup aware
> ...
> 139:  13877bc79d8135 = 139:  b038df088d5bf2 xfs: port ondisk structure checks from xfs/122 to the kernel
> 
> Apologies for the last minute churn and paperwork stress.
> 
> --D
> 

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2024-11-14  6:16 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-11-14  0:16 [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong
2024-11-14  0:17 ` [GIT PULL 01/10] xfs: convert perag to use xarrays Darrick J. Wong
2024-11-14  0:18 ` [GIT PULL 02/10] xfs: create a generic allocation group structure Darrick J. Wong
2024-11-14  0:18 ` [GIT PULL 03/10] xfs: metadata inode directory trees Darrick J. Wong
2024-11-14  0:18 ` [GIT PULL 04/10] xfs: create incore rt allocation groups Darrick J. Wong
2024-11-14  0:18 ` [GIT PULL 05/10] xfs: preparation for realtime " Darrick J. Wong
2024-11-14  0:19 ` [GIT PULL 06/10] xfs: shard the realtime section Darrick J. Wong
2024-11-14  0:19 ` [GIT PULL 07/10] xfs: persist quota options with metadir Darrick J. Wong
2024-11-14  0:19 ` [GIT PULL 08/10] xfs: enable quota for realtime volumes Darrick J. Wong
2024-11-14  0:19 ` [GIT PULL 09/10] xfs: enable metadir Darrick J. Wong
2024-11-14  0:20 ` [GIT PULL 10/10] xfs: improve ondisk structure checks Darrick J. Wong
2024-11-14  6:16 ` [GIT PULLBOMB v5.6] xfs: metadata directories and realtime groups Darrick J. Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox