[GIT PULLBOMB] xfsprogs: catch us up to 6.10

[GIT PULLBOMB] xfsprogs: catch us up to 6.10

[Darrick J. Wong]

Hi Carlos,

Please pull these fully reviewed patchsets for xfsprogs 6.10.

--D

[GIT PULL 01/23] libxfs: fixes for 6.9

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 42fc61ebd061ffc63a0302a8422ed21f67e38b5f:

xfsprogs: Release v6.9.0 (2024-07-22 11:05:03 +0200)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/libxfs-6.9-fixes_2024-07-29

for you to fetch changes up to c6438c3e740e139adcd6c236606498d49bbb5655:

xfs_repair: don't crash on -vv (2024-07-29 17:00:59 -0700)

----------------------------------------------------------------
libxfs: fixes for 6.9 [01/28]

A couple more last minute fixes for 6.9.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Chris Hofstaedtler (1):
Remove support for split-/usr installs

Christoph Hellwig (1):
repair: btree blocks are never on the RT subvolume

Darrick J. Wong (3):
xfile: fix missing error unlock in xfile_fcb_find
xfs_repair: don't leak the rootdir inode when orphanage already exists
xfs_repair: don't crash on -vv

configure.ac                | 21 ---------------------
debian/Makefile             |  4 ++--
debian/local/initramfs.hook |  2 +-
debian/rules                |  5 ++---
fsck/Makefile               |  4 ++--
include/builddefs.in        |  2 --
include/buildmacros         | 20 ++++++++++----------
libxfs/xfile.c              |  6 +++---
mkfs/Makefile               |  4 ++--
repair/Makefile             |  4 ++--
repair/phase6.c             |  7 +++++--
repair/progress.c           |  2 +-
repair/scan.c               | 21 +++++----------------
repair/xfs_repair.c         |  2 +-
14 files changed, 36 insertions(+), 68 deletions(-)

[GIT PULL 02/23] libxfs: sync with 6.10

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit c6438c3e740e139adcd6c236606498d49bbb5655:

xfs_repair: don't crash on -vv (2024-07-29 17:00:59 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/libxfs-sync-6.10_2024-07-29

for you to fetch changes up to 7fbf8e036dc1d5b9caaf6f64ad4bc88d40c8292b:

xfs: fix direction in XFS_IOC_EXCHANGE_RANGE (2024-07-29 17:01:05 -0700)

----------------------------------------------------------------
libxfs: sync with 6.10 [02/28]

Synchronize libxfs with the kernel.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Allison Henderson (14):
xfs: Increase XFS_DEFER_OPS_NR_INODES to 5
xfs: add parent pointer support to attribute code
xfs: define parent pointer ondisk extended attribute format
xfs: add parent pointer validator functions
xfs: extend transaction reservations for parent attributes
xfs: parent pointer attribute creation
xfs: add parent attributes to link
xfs: add parent attributes to symlink
xfs: remove parent pointers in unlink
xfs: Add parent pointers to rename
xfs: don't return XFS_ATTR_PARENT attributes via listxattr
xfs: pass the attr value to put_listent when possible
xfs: don't remove the attr fork when parent pointers are enabled
xfs: add a incompat feature bit for parent pointers

Christoph Hellwig (26):
xfs: make XFS_TRANS_LOWMODE match the other XFS_TRANS_ definitions
xfs: refactor realtime inode locking
xfs: free RT extents after updating the bmap btree
xfs: move RT inode locking out of __xfs_bunmapi
xfs: split xfs_mod_freecounter
xfs: reinstate RT support in xfs_bmapi_reserve_delalloc
xfs: cleanup fdblock/frextent accounting in xfs_bmap_del_extent_delay
xfs: support RT inodes in xfs_mod_delalloc
xfs: rework splitting of indirect block reservations
xfs: stop the steal (of data blocks for RT indirect blocks)
xfs: check the flags earlier in xfs_attr_match
xfs: factor out a xfs_dir_lookup_args helper
xfs: factor out a xfs_dir_createname_args helper
xfs: factor out a xfs_dir_removename_args helper
xfs: factor out a xfs_dir_replace_args helper
xfs: refactor dir format helpers
xfs: fix error returns from xfs_bmapi_write
xfs: remove the unusued tmp_logflags variable in xfs_bmapi_allocate
xfs: lift a xfs_valid_startblock into xfs_bmapi_allocate
xfs: don't open code XFS_FILBLKS_MIN in xfs_bmapi_write
xfs: pass the actual offset and len to allocate to xfs_bmapi_allocate
xfs: remove the xfs_iext_peek_prev_extent call in xfs_bmapi_allocate
xfs: fix xfs_bmap_add_extent_delay_real for partial conversions
xfs: do not allocate the entire delalloc extent in xfs_bmapi_write
xfs: xfs_quota_unreserve_blkres can't fail
xfs: simplify iext overflow checking and upgrade

Darrick J. Wong (70):
xfs: pass xfs_buf lookup flags to xfs_*read_agi
xfs: constify xfs_bmap_is_written_extent
xfs: introduce new file range exchange ioctl
xfs: create a incompat flag for atomic file mapping exchanges
xfs: introduce a file mapping exchange log intent item
xfs: create deferred log items for file mapping exchanges
xfs: add error injection to test file mapping exchange recovery
xfs: condense extended attributes after a mapping exchange operation
xfs: condense directories after a mapping exchange operation
xfs: condense symbolic links after a mapping exchange operation
xfs: make file range exchange support realtime files
xfs: capture inode generation numbers in the ondisk exchmaps log item
xfs: enable logged file mapping exchange feature
xfs: add an explicit owner field to xfs_da_args
xfs: use the xfs_da_args owner field to set new dir/attr block owner
xfs: validate attr leaf buffer owners
xfs: validate attr remote value buffer owners
xfs: validate dabtree node buffer owners
xfs: validate directory leaf buffer owners
xfs: validate explicit directory data buffer owners
xfs: validate explicit directory block buffer owners
xfs: validate explicit directory free block owners
xfs: use atomic extent swapping to fix user file fork data
xfs: repair extended attributes
xfs: expose xfs_bmap_local_to_extents for online repair
xfs: pass the owner to xfs_symlink_write_target
xfs: check unused nlink fields in the ondisk inode
xfs: try to avoid allocating from sick inode clusters
xfs: pin inodes that would otherwise overflow link count
xfs: remove XFS_DA_OP_REMOVE
xfs: remove XFS_DA_OP_NOTIME
xfs: remove xfs_da_args.attr_flags
xfs: make attr removal an explicit operation
xfs: rearrange xfs_da_args a bit to use less space
xfs: attr fork iext must be loaded before calling xfs_attr_is_leaf
xfs: fix missing check for invalid attr flags
xfs: restructure xfs_attr_complete_op a bit
xfs: use helpers to extract xattr op from opflags
xfs: enforce one namespace per attribute
xfs: rearrange xfs_attr_match parameters
xfs: move xfs_attr_defer_add to xfs_attr_item.c
xfs: create a separate hashname function for extended attributes
xfs: allow xattr matching on name and value for parent pointers
xfs: create attr log item opcodes and formats for parent pointers
xfs: record inode generation in xattr update log intent items
xfs: create a hashname function for parent pointers
xfs: split out handle management helpers a bit
xfs: add parent pointer ioctls
xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res
xfs: drop compatibility minimum log size computations for reflink
xfs: enable parent pointers
xfs: check dirents have parent pointers
xfs: remove some boilerplate from xfs_attr_set
xfs: make the reserved block permission flag explicit in xfs_attr_set
xfs: add raw parent pointer apis to support repair
xfs: remove pointless unlocked assertion
xfs: split xfs_bmap_add_attrfork into two pieces
xfs: actually rebuild the parent pointer xattrs
xfs: teach online scrub to find directory tree structure problems
xfs: report directory tree corruption in the health information
xfs: introduce vectored scrub mode
xfs: use unsigned ints for non-negative quantities in xfs_attr_remote.c
xfs: turn XFS_ATTR3_RMT_BUF_SPACE into a function
xfs: create a helper to compute the blockcount of a max sized remote value
xfs: minor cleanups of xfs_attr3_rmt_blocks
xfs: fix xfs_init_attr_trans not handling explicit operation codes
xfs: allow symlinks with short remote targets
xfs: restrict when we try to align cow fork delalloc to cowextsz hints
xfs: allow unlinked symlinks and dirs with zero size
xfs: fix direction in XFS_IOC_EXCHANGE_RANGE

John Garry (1):
xfs: Stop using __maybe_unused in xfs_alloc.c

Ritesh Harjani (IBM) (1):
xfs: Add cond_resched to block unmap range and reflink remap path

Wengang Wang (1):
xfs: make sure sb_fdblocks is non-negative

Zhang Yi (2):
xfs: make the seq argument to xfs_bmapi_convert_delalloc() optional
xfs: make xfs_bmapi_convert_delalloc() to allocate the target offset

db/attr.c                   |    2 +-
db/attrset.c                |   11 +-
db/iunlink.c                |    4 +-
db/metadump.c               |    8 +-
db/namei.c                  |   23 +-
include/libxfs.h            |    1 +
include/xfs_inode.h         |   16 +
include/xfs_mount.h         |    2 +
include/xfs_trace.h         |   15 +
io/inject.c                 |    1 +
libxfs/Makefile             |    5 +
libxfs/defer_item.c         |  116 ++++
libxfs/defer_item.h         |   13 +
libxfs/init.c               |    3 +
libxfs/libxfs_api_defs.h    |    6 +-
libxfs/libxfs_priv.h        |   75 ++-
libxfs/util.c               |    6 +
libxfs/xfs_ag.c             |   12 +-
libxfs/xfs_ag_resv.c        |   24 +-
libxfs/xfs_ag_resv.h        |    2 +-
libxfs/xfs_alloc.c          |   10 +-
libxfs/xfs_attr.c           |  272 ++++++----
libxfs/xfs_attr.h           |   49 +-
libxfs/xfs_attr_leaf.c      |  154 ++++--
libxfs/xfs_attr_leaf.h      |    4 +-
libxfs/xfs_attr_remote.c    |  102 ++--
libxfs/xfs_attr_remote.h    |    8 +-
libxfs/xfs_attr_sf.h        |    1 +
libxfs/xfs_bmap.c           |  409 +++++++-------
libxfs/xfs_bmap.h           |   13 +-
libxfs/xfs_da_btree.c       |  189 ++++++-
libxfs/xfs_da_btree.h       |   34 +-
libxfs/xfs_da_format.h      |   37 +-
libxfs/xfs_defer.c          |   12 +-
libxfs/xfs_defer.h          |   10 +-
libxfs/xfs_dir2.c           |  283 +++++-----
libxfs/xfs_dir2.h           |   23 +-
libxfs/xfs_dir2_block.c     |   42 +-
libxfs/xfs_dir2_data.c      |   18 +-
libxfs/xfs_dir2_leaf.c      |  100 +++-
libxfs/xfs_dir2_node.c      |   44 +-
libxfs/xfs_dir2_priv.h      |   15 +-
libxfs/xfs_errortag.h       |    4 +-
libxfs/xfs_exchmaps.c       | 1232 +++++++++++++++++++++++++++++++++++++++++++
libxfs/xfs_exchmaps.h       |  124 +++++
libxfs/xfs_format.h         |   34 +-
libxfs/xfs_fs.h             |  158 +++++-
libxfs/xfs_health.h         |    4 +-
libxfs/xfs_ialloc.c         |   56 +-
libxfs/xfs_ialloc.h         |    5 +-
libxfs/xfs_ialloc_btree.c   |    4 +-
libxfs/xfs_inode_buf.c      |   55 +-
libxfs/xfs_inode_fork.c     |   63 +--
libxfs/xfs_inode_fork.h     |    6 +-
libxfs/xfs_log_format.h     |   89 +++-
libxfs/xfs_log_rlimit.c     |   46 ++
libxfs/xfs_ondisk.h         |    6 +
libxfs/xfs_parent.c         |  376 +++++++++++++
libxfs/xfs_parent.h         |  110 ++++
libxfs/xfs_rtbitmap.c       |   57 ++
libxfs/xfs_rtbitmap.h       |   17 +
libxfs/xfs_sb.c             |   15 +-
libxfs/xfs_shared.h         |    6 +-
libxfs/xfs_symlink_remote.c |   54 +-
libxfs/xfs_symlink_remote.h |    8 +-
libxfs/xfs_trans_resv.c     |  324 ++++++++++--
libxfs/xfs_trans_space.c    |  121 +++++
libxfs/xfs_trans_space.h    |   29 +-
mkfs/proto.c                |    3 +-
repair/attr_repair.c        |    7 +-
repair/phase6.c             |   35 +-
71 files changed, 4349 insertions(+), 873 deletions(-)
create mode 100644 libxfs/xfs_exchmaps.c
create mode 100644 libxfs/xfs_exchmaps.h
create mode 100644 libxfs/xfs_parent.c
create mode 100644 libxfs/xfs_parent.h
create mode 100644 libxfs/xfs_trans_space.c

[GIT PULL 03/23] xfsprogs: atomic file updates

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 7fbf8e036dc1d5b9caaf6f64ad4bc88d40c8292b:

xfs: fix direction in XFS_IOC_EXCHANGE_RANGE (2024-07-29 17:01:05 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/atomic-file-updates-6.10_2024-07-29

for you to fetch changes up to 39e346ba525c51dd2f405ed5d6368db712fac586:

mkfs: add a formatting option for exchange-range (2024-07-29 17:01:06 -0700)

----------------------------------------------------------------
xfsprogs: atomic file updates [v30.9 03/28]

This series creates a new XFS_IOC_EXCHANGE_RANGE ioctl to exchange
ranges of bytes between two files atomically.

This new functionality enables data storage programs to stage and commit
file updates such that reader programs will see either the old contents
or the new contents in their entirety, with no chance of torn writes.  A
successful call completion guarantees that the new contents will be seen
even if the system fails.

The ability to exchange file fork mappings between files in this manner
is critical to supporting online filesystem repair, which is built upon
the strategy of constructing a clean copy of a damaged structure and
committing the new structure into the metadata file atomically.  The
ioctls exist to facilitate testing of the new functionality and to
enable future application program designs.

User programs will be able to update files atomically by opening an
O_TMPFILE, reflinking the source file to it, making whatever updates
they want to make, and exchange the relevant ranges of the temp file
with the original file.  If the updates are aligned with the file block
size, a new (since v2) flag provides for exchanging only the written
areas.  Note that application software must quiesce writes to the file
while it stages an atomic update.  This will be addressed by a
subsequent series.

This mechanism solves the clunkiness of two existing atomic file update
mechanisms: for O_TRUNC + rewrite, this eliminates the brief period
where other programs can see an empty file.  For create tempfile +
rename, the need to copy file attributes and extended attributes for
each file update is eliminated.

However, this method introduces its own awkwardness -- any program
initiating an exchange now needs to have a way to signal to other
programs that the file contents have changed.  For file access mediated
via read and write, fanotify or inotify are probably sufficient.  For
mmaped files, that may not be fast enough.

The reference implementation in XFS creates a new log incompat feature
and log intent items to track high level progress of swapping ranges of
two files and finish interrupted work if the system goes down.  Sample
code can be found in the corresponding changes to xfs_io to exercise the
use case mentioned above.

Note that this function is /not/ the O_DIRECT atomic untorn file writes
concept that has also been floating around for years.  It is also not
the RWF_ATOMIC patchset that has been shared.  This RFC is constructed
entirely in software, which means that there are no limitations other
than the general filesystem limits.

As a side note, the original motivation behind the kernel functionality
is online repair of file-based metadata.  The atomic file content
exchange is implemented as an atomic exchange of file fork mappings,
which means that we can implement online reconstruction of extended
attributes and directories by building a new one in another inode and
exchanging the contents.

Subsequent patchsets adapt the online filesystem repair code to use
atomic file exchanges.  This enables repair functions to construct a
clean copy of a directory, xattr information, symbolic links, realtime
bitmaps, and realtime summary information in a temporary inode.  If this
completes successfully, the new contents can be committed atomically
into the inode being repaired.  This is essential to avoid making
corruption problems worse if the system goes down in the middle of
running repair.

For userspace, this series also includes the userspace pieces needed to
test the new functionality, and a sample implementation of atomic file
updates.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (12):
man: document the exchange-range ioctl
man: document XFS_FSOP_GEOM_FLAGS_EXCHRANGE
libhandle: add support for bulkstat v5
libfrog: add support for exchange range ioctl family
xfs_db: advertise exchange-range in the version command
xfs_logprint: support dumping exchmaps log items
xfs_fsr: convert to bulkstat v5 ioctls
xfs_fsr: skip the xattr/forkoff levering with the newer swapext implementations
xfs_io: create exchangerange command to test file range exchange ioctl
libfrog: advertise exchange-range support
xfs_repair: add exchange-range to file systems
mkfs: add a formatting option for exchange-range

db/sb.c                             |   2 +
fsr/xfs_fsr.c                       | 167 +++++++++++++---------
include/jdm.h                       |  23 +++
io/Makefile                         |  48 ++++++-
io/exchrange.c                      | 156 ++++++++++++++++++++
io/init.c                           |   1 +
io/io.h                             |   1 +
libfrog/Makefile                    |   2 +
libfrog/file_exchange.c             |  52 +++++++
libfrog/file_exchange.h             |  15 ++
libfrog/fsgeom.c                    |  49 +++++--
libfrog/fsgeom.h                    |   1 +
libhandle/jdm.c                     | 117 +++++++++++++++
logprint/log_misc.c                 |  11 ++
logprint/log_print_all.c            |  12 ++
logprint/log_redo.c                 | 128 +++++++++++++++++
logprint/logprint.h                 |   6 +
man/man2/ioctl_xfs_exchange_range.2 | 278 ++++++++++++++++++++++++++++++++++++
man/man2/ioctl_xfs_fsgeometry.2     |   3 +
man/man8/mkfs.xfs.8.in              |   7 +
man/man8/xfs_admin.8                |   7 +
man/man8/xfs_io.8                   |  40 ++++++
mkfs/lts_4.19.conf                  |   1 +
mkfs/lts_5.10.conf                  |   1 +
mkfs/lts_5.15.conf                  |   1 +
mkfs/lts_5.4.conf                   |   1 +
mkfs/lts_6.1.conf                   |   1 +
mkfs/lts_6.6.conf                   |   1 +
mkfs/xfs_mkfs.c                     |  26 +++-
repair/globals.c                    |   1 +
repair/globals.h                    |   1 +
repair/phase2.c                     |  30 ++++
repair/xfs_repair.c                 |  11 ++
33 files changed, 1114 insertions(+), 87 deletions(-)
create mode 100644 io/exchrange.c
create mode 100644 libfrog/file_exchange.c
create mode 100644 libfrog/file_exchange.h
create mode 100644 man/man2/ioctl_xfs_exchange_range.2

[GIT PULL 04/23] xfsprogs: set and validate dir/attr block owners

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 39e346ba525c51dd2f405ed5d6368db712fac586:

mkfs: add a formatting option for exchange-range (2024-07-29 17:01:06 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/dirattr-validate-owners-6.10_2024-07-29

for you to fetch changes up to 7e74984e652fab200bc7319d7c3d90f6ae36be2e:

xfs_{db,repair}: add an explicit owner field to xfs_da_args (2024-07-29 17:01:06 -0700)

----------------------------------------------------------------
xfsprogs: set and validate dir/attr block owners [v30.9 04/28]

There are a couple of significatn changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number, we instead change libxfs to allow callers of the dir
and xattr code the ability to set an explicit owner number to be written
into the header fields of any new blocks that are created.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (1):
xfs_{db,repair}: add an explicit owner field to xfs_da_args

db/namei.c      | 1 +
repair/phase6.c | 3 +++
2 files changed, 4 insertions(+)

[GIT PULL 05/23] xfsprogs: inode-related repair fixes

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 7e74984e652fab200bc7319d7c3d90f6ae36be2e:

xfs_{db,repair}: add an explicit owner field to xfs_da_args (2024-07-29 17:01:06 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/inode-repair-improvements-6.10_2024-07-29

for you to fetch changes up to ebf05a446c09336c08865dc29a6332be6ff8223c:

mkfs/repair: pin inodes that would otherwise overflow link count (2024-07-29 17:01:06 -0700)

----------------------------------------------------------------
xfsprogs: inode-related repair fixes [v30.9 05/28]

While doing QA of the online fsck code, I made a few observations:
First, nobody was checking that the di_onlink field is actually zero;
Second, that allocating a temporary file for repairs can fail (and
thus bring down the entire fs) if the inode cluster is corrupt; and
Third, that file link counts do not pin at ~0U to prevent integer
overflows.

This scattered patchset fixes those three problems.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (2):
libxfs: port the bumplink function from the kernel
mkfs/repair: pin inodes that would otherwise overflow link count

include/xfs_inode.h |  2 ++
libxfs/util.c       | 18 ++++++++++++++++++
mkfs/proto.c        |  4 ++--
repair/incore_ino.c | 14 +++++++++-----
repair/phase6.c     | 10 +++++-----
5 files changed, 36 insertions(+), 12 deletions(-)

[GIT PULL 06/23] xfs_scrub: fixes to the repair code

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit ebf05a446c09336c08865dc29a6332be6ff8223c:

mkfs/repair: pin inodes that would otherwise overflow link count (2024-07-29 17:01:06 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-repair-fixes-6.10_2024-07-29

for you to fetch changes up to 4b959abc5f353123775973cd98c94d819cc9de79:

xfs_scrub: actually try to fix summary counters ahead of repairs (2024-07-29 17:01:06 -0700)

----------------------------------------------------------------
xfs_scrub: fixes to the repair code [v30.9 06/28]

Now that we've landed the new kernel code, it's time to reorganize the
xfs_scrub code that handles repairs.  Clean up various naming warts and
misleading error messages.  Move the repair code to scrub/repair.c as
the first step.  Then, fix various issues in the repair code before we
start reorganizing things.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (5):
xfs_scrub: remove ALP_* flags namespace
xfs_scrub: move repair functions to repair.c
xfs_scrub: log when a repair was unnecessary
xfs_scrub: require primary superblock repairs to complete before proceeding
xfs_scrub: actually try to fix summary counters ahead of repairs

scrub/phase1.c        |   2 +-
scrub/phase2.c        |   3 +-
scrub/phase3.c        |   2 +-
scrub/phase4.c        |  22 ++++--
scrub/phase5.c        |   2 +-
scrub/phase7.c        |   2 +-
scrub/repair.c        | 177 ++++++++++++++++++++++++++++++++++++++++++-
scrub/repair.h        |  16 +++-
scrub/scrub.c         | 204 +-------------------------------------------------
scrub/scrub.h         |  16 +---
scrub/scrub_private.h |  55 ++++++++++++++
11 files changed, 269 insertions(+), 232 deletions(-)
create mode 100644 scrub/scrub_private.h

[GIT PULL 07/23] xfs_scrub: improve warnings about difficult repairs

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 4b959abc5f353123775973cd98c94d819cc9de79:

xfs_scrub: actually try to fix summary counters ahead of repairs (2024-07-29 17:01:06 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-better-repair-warnings-6.10_2024-07-29

for you to fetch changes up to bf15d7766e3dd63eda56f6b2f7976e529cd07575:

xfs_scrub: enable users to bump information messages to warnings (2024-07-29 17:01:07 -0700)

----------------------------------------------------------------
xfs_scrub: improve warnings about difficult repairs [v30.9 07/28]

While I was poking through the QA results for xfs_scrub, I noticed that
it doesn't warn the user when the primary and secondary realtime
metadata are so out of whack that the chances of a successful repair are
not so high.  I decided that it was worth refactoring the scrub code a
bit so that we could warn the user about these types of things, and
ended up refactoring unnecessary helpers out of existence and fixing
other reporting gaps.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (8):
xfs_scrub: fix missing scrub coverage for broken inodes
xfs_scrub: collapse trivial superblock scrub helpers
xfs_scrub: get rid of trivial fs metadata scanner helpers
xfs_scrub: split up the mustfix repairs and difficulty assessment functions
xfs_scrub: add missing repair types to the mustfix and difficulty assessment
xfs_scrub: any inconsistency in metadata should trigger difficulty warnings
xfs_scrub: warn about difficult repairs to rt and quota metadata
xfs_scrub: enable users to bump information messages to warnings

man/man8/xfs_scrub.8 | 19 +++++++++++++++++++
scrub/common.c       |  2 ++
scrub/phase1.c       |  2 +-
scrub/phase2.c       | 53 ++++++++++++++++++++++++++++++++--------------------
scrub/phase3.c       | 21 +++++++++++++++++----
scrub/phase4.c       |  9 +++++----
scrub/phase5.c       | 15 +++++++--------
scrub/repair.c       | 47 +++++++++++++++++++++++++++++++++++-----------
scrub/repair.h       | 10 +++++++---
scrub/scrub.c        | 52 +--------------------------------------------------
scrub/scrub.h        |  7 ++-----
scrub/xfs_scrub.c    | 45 +++++++++++++++++++++++++++++++++++++++++++-
scrub/xfs_scrub.h    |  1 +
13 files changed, 175 insertions(+), 108 deletions(-)

[GIT PULL 08/23] xfs_scrub: track data dependencies for repairs

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit bf15d7766e3dd63eda56f6b2f7976e529cd07575:

xfs_scrub: enable users to bump information messages to warnings (2024-07-29 17:01:07 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-repair-data-deps-6.10_2024-07-29

for you to fetch changes up to 81bfd0ad04a58892e2c153a22c361e7ff959f3fd:

xfs_scrub: remove unused action_list fields (2024-07-29 17:01:07 -0700)

----------------------------------------------------------------
xfs_scrub: track data dependencies for repairs [v30.9 08/28]

Certain kinds of XFS metadata depend on the correctness of lower level
metadata.  For example, directory indexes depends on the directory data
fork, which in turn depend on the directory inode to be correct.  The
current scrub code does not strictly preserve these dependencies if it
has to defer a repair until phase 4, because phase 4 prioritizes repairs
by type (corruption, then cross referencing, and then preening) and
loses the ordering of in the previous phases.  This leads to absurd
things like trying to repair a directory before repairing its corrupted
fork, which is absurd.

To solve this problem, introduce a repair ticket structure to track all
the repairs pending for a principal object (inode, AG, etc).  This
reduces memory requirements if an object requires more than one type of
repair and makes it very easy to track the data dependencies between
sub-objects of a principal object.  Repair dependencies between object
types (e.g.  bnobt before inodes) must still be encoded statically into
phase 4.

A secondary benefit of this new ticket structure is that we can decide
to attempt a repair of an object A that was flagged for a cross
referencing error during the scan if a different object B depends on A
but only B showed definitive signs of corruption.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (9):
xfs_scrub: track repair items by principal, not by individual repairs
xfs_scrub: use repair_item to direct repair activities
xfs_scrub: remove action lists from phaseX code
xfs_scrub: remove scrub_metadata_file
xfs_scrub: boost the repair priority of dependencies of damaged items
xfs_scrub: clean up repair_item_difficulty a little
xfs_scrub: check dependencies of a scrub type before repairing
xfs_scrub: retry incomplete repairs
xfs_scrub: remove unused action_list fields

libfrog/scrub.c       |   1 +
scrub/phase1.c        |   9 +-
scrub/phase2.c        |  46 +++--
scrub/phase3.c        |  77 ++++----
scrub/phase4.c        |  17 +-
scrub/phase5.c        |   9 +-
scrub/phase7.c        |   9 +-
scrub/repair.c        | 530 ++++++++++++++++++++++++++++++++++----------------
scrub/repair.h        |  47 +++--
scrub/scrub.c         | 136 ++++++-------
scrub/scrub.h         | 108 ++++++++--
scrub/scrub_private.h |  37 ++++
12 files changed, 664 insertions(+), 362 deletions(-)

[GIT PULL 09/23] xfs_scrub: use scrub_item to track check progress

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 81bfd0ad04a58892e2c153a22c361e7ff959f3fd:

xfs_scrub: remove unused action_list fields (2024-07-29 17:01:07 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-object-tracking-6.10_2024-07-29

for you to fetch changes up to 8dd67c8eccbe0e6b8dd7975ba53f9ccaf532aa9c:

xfs_scrub: hoist scrub retry loop to scrub_item_check_file (2024-07-29 17:01:08 -0700)

----------------------------------------------------------------
xfs_scrub: use scrub_item to track check progress [v30.9 09/28]

Now that we've introduced tickets to track the status of repairs to a
specific principal XFS object (fs, ag, file), use them to track the
scrub state of those same objects.  Ultimately, we want to make it easy
to introduce vectorized repair, where we send a batch of repair requests
to the kernel instead of making millions of ioctl calls.  For now,
however, we'll settle for easier bookkeepping.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (5):
xfs_scrub: start tracking scrub state in scrub_item
xfs_scrub: remove enum check_outcome
xfs_scrub: refactor scrub_meta_type out of existence
xfs_scrub: hoist repair retry loop to repair_item_class
xfs_scrub: hoist scrub retry loop to scrub_item_check_file

scrub/phase1.c        |   3 +-
scrub/phase2.c        |  12 +-
scrub/phase3.c        |  41 ++-----
scrub/phase4.c        |  16 ++-
scrub/phase5.c        |   5 +-
scrub/phase7.c        |   5 +-
scrub/repair.c        |  71 ++++++-----
scrub/scrub.c         | 321 +++++++++++++++++++++++---------------------------
scrub/scrub.h         |  40 +++++--
scrub/scrub_private.h |  14 +++
10 files changed, 257 insertions(+), 271 deletions(-)

[GIT PULL 10/23] xfs_scrub: improve scheduling of repair items

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 8dd67c8eccbe0e6b8dd7975ba53f9ccaf532aa9c:

xfs_scrub: hoist scrub retry loop to scrub_item_check_file (2024-07-29 17:01:08 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-repair-scheduling-6.10_2024-07-29

for you to fetch changes up to 474ff27d466c053f1cd51024e6b0c5a741a2d4bd:

xfs_scrub: try to repair space metadata before file metadata (2024-07-29 17:01:08 -0700)

----------------------------------------------------------------
xfs_scrub: improve scheduling of repair items [v30.9 10/28]

Currently, phase 4 of xfs_scrub uses per-AG repair item lists to
schedule repair work across a thread pool.  This scheme is suboptimal
when most of the repairs involve a single AG because all the work gets
dumped on a single pool thread.

Instead, we should create a thread pool with the same number of workers
as CPUs, and dispatch individual repair tickets as separate work items
to maximize parallelization.

However, we also need to ensure that repairs to space metadata and file
metadata are kept in separate queues because file repairs generally
depend on correctness of space metadata.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (4):
libfrog: enhance ptvar to support initializer functions
xfs_scrub: improve thread scheduling repair items during phase 4
xfs_scrub: recheck entire metadata objects after corruption repairs
xfs_scrub: try to repair space metadata before file metadata

libfrog/ptvar.c       |   9 +-
libfrog/ptvar.h       |   4 +-
scrub/counter.c       |   2 +-
scrub/descr.c         |   2 +-
scrub/phase1.c        |  15 +++-
scrub/phase2.c        |  23 ++++-
scrub/phase3.c        | 106 ++++++++++++++--------
scrub/phase4.c        | 244 +++++++++++++++++++++++++++++++++++++-------------
scrub/phase7.c        |   2 +-
scrub/read_verify.c   |   2 +-
scrub/repair.c        | 172 ++++++++++++++++++++++-------------
scrub/repair.h        |  37 ++++++--
scrub/scrub.c         |   5 +-
scrub/scrub.h         |  10 +++
scrub/scrub_private.h |   2 +
scrub/xfs_scrub.h     |   3 +-
16 files changed, 455 insertions(+), 183 deletions(-)

[GIT PULL 11/23] xfs_scrub: detect deceptive filename extensions

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 474ff27d466c053f1cd51024e6b0c5a741a2d4bd:

xfs_scrub: try to repair space metadata before file metadata (2024-07-29 17:01:08 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-detect-deceptive-extensions-6.10_2024-07-29

for you to fetch changes up to 746ee95b71649b4ae515893ffa3bbe7b5e815d0d:

xfs_scrub: dump unicode points (2024-07-29 17:01:09 -0700)

----------------------------------------------------------------
xfs_scrub: detect deceptive filename extensions [v30.9 11/28]

In early 2023, malware researchers disclosed a phishing attack that was
targeted at people running Linux workstations.  The attack vector
involved the use of filenames containing what looked like a file
extension but instead contained a lookalike for the full stop (".")
and a common extension ("pdf").  Enhance xfs_scrub phase 5 to detect
these types of attacks and warn the system administrator.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (13):
xfs_scrub: use proper UChar string iterators
xfs_scrub: hoist code that removes ignorable characters
xfs_scrub: add a couple of omitted invisible code points
xfs_scrub: avoid potential UAF after freeing a duplicate name entry
xfs_scrub: guard against libicu returning negative buffer lengths
xfs_scrub: hoist non-rendering character predicate
xfs_scrub: store bad flags with the name entry
xfs_scrub: rename UNICRASH_ZERO_WIDTH to UNICRASH_INVISIBLE
xfs_scrub: type-coerce the UNICRASH_* flags
xfs_scrub: reduce size of struct name_entry
xfs_scrub: rename struct unicrash.normalizer
xfs_scrub: report deceptive file extensions
xfs_scrub: dump unicode points

scrub/unicrash.c | 532 ++++++++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 426 insertions(+), 106 deletions(-)

[GIT PULL 12/23] xfs_scrub: move fstrim to a separate phase

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 746ee95b71649b4ae515893ffa3bbe7b5e815d0d:

xfs_scrub: dump unicode points (2024-07-29 17:01:09 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-fstrim-phase-6.10_2024-07-29

for you to fetch changes up to 5ccdd24dc9987b50333332b7381ff1a305e67ef7:

xfs_scrub: improve responsiveness while trimming the filesystem (2024-07-29 17:01:09 -0700)

----------------------------------------------------------------
xfs_scrub: move fstrim to a separate phase [v30.9 12/28]

Back when I originally designed xfs_scrub, all filesystem metadata
checks were complete by the end of phase 3, and phase 4 was where all
the metadata repairs occurred.  On the grounds that the filesystem
should be fully consistent by then, I made a call to FITRIM at the end
of phase 4 to discard empty space in the filesystem.

Unfortunately, that's no longer the case -- summary counters, link
counts, and quota counters are not checked until phase 7.  It's not safe
to instruct the storage to unmap "empty" areas if we don't know where
those empty areas are, so we need to create a phase 8 to trim the fs.
While we're at it, make it more obvious that fstrim only gets to run if
there are no unfixed corruptions and no other runtime errors have
occurred.

Finally, reduce the latency impacts on the rest of the system by
breaking up the fstrim work into a loop that targets only 16GB per call.
This enables better progress reporting for interactive runs and cgroup
based resource constraints for background runs.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (7):
xfs_scrub: move FITRIM to phase 8
xfs_scrub: ignore phase 8 if the user disabled fstrim
xfs_scrub: collapse trim_filesystem
xfs_scrub: fix the work estimation for phase 8
xfs_scrub: report FITRIM errors properly
xfs_scrub: don't call FITRIM after runtime errors
xfs_scrub: improve responsiveness while trimming the filesystem

scrub/Makefile    |   1 +
scrub/phase4.c    |  30 ++---------
scrub/phase8.c    | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
scrub/vfs.c       |  22 +++++---
scrub/vfs.h       |   2 +-
scrub/xfs_scrub.c |  11 +++-
scrub/xfs_scrub.h |   3 ++
7 files changed, 183 insertions(+), 37 deletions(-)
create mode 100644 scrub/phase8.c

[GIT PULL 13/23] xfs_scrub: use free space histograms to reduce fstrim runtime

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 5ccdd24dc9987b50333332b7381ff1a305e67ef7:

xfs_scrub: improve responsiveness while trimming the filesystem (2024-07-29 17:01:09 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-fstrim-minlen-freesp-histogram-6.10_2024-07-29

for you to fetch changes up to 34bed605490f936c3ead49e2e1cad78505260461:

xfs_scrub: tune fstrim minlen parameter based on free space histograms (2024-07-29 17:01:09 -0700)

----------------------------------------------------------------
xfs_scrub: use free space histograms to reduce fstrim runtime [v30.9 13/28]

This patchset dramatically reduces the runtime of the FITRIM calls made
during phase 8 of xfs_scrub.  It turns out that phase 8 can really get
bogged down if the free space contains a large number of very small
extents.  In these cases, the runtime can increase by an order of
magnitude to free less than 1%% of the free space.  This is not worth the
time, since we're spending a lot of time to do very little work.  The
FITRIM ioctl allows us to specify a minimum extent length, so we can use
statistical methods to compute a minlen parameter.

It turns out xfs_db/spaceman already have the code needed to create
histograms of free space extent lengths.  We add the ability to compute
a CDF of the extent lengths, which make it easy to pick a minimum length
corresponding to 99%% of the free space.  In most cases, this results in
dramatic reductions in phase 8 runtime.  Hence, move the histogram code
to libfrog, and wire up xfs_scrub, since phase 7 already walks the
fsmap.

We also add a new -o suboption to xfs_scrub so that people who /do/ want
to examine every free extent can do so.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (7):
libfrog: hoist free space histogram code
libfrog: print wider columns for free space histogram
libfrog: print cdf of free space buckets
xfs_scrub: don't close stdout when closing the progress bar
xfs_scrub: remove pointless spacemap.c arguments
xfs_scrub: collect free space histograms during phase 7
xfs_scrub: tune fstrim minlen parameter based on free space histograms

db/freesp.c          |  89 +++++------------
libfrog/Makefile     |   2 +
libfrog/histogram.c  | 270 +++++++++++++++++++++++++++++++++++++++++++++++++++
libfrog/histogram.h  |  77 +++++++++++++++
man/man8/xfs_scrub.8 |  16 +++
scrub/phase7.c       |  47 ++++++++-
scrub/phase8.c       |  91 ++++++++++++++++-
scrub/spacemap.c     |  11 +--
scrub/vfs.c          |   4 +-
scrub/vfs.h          |   2 +-
scrub/xfs_scrub.c    |  45 ++++++++-
scrub/xfs_scrub.h    |  16 +++
spaceman/freesp.c    |  99 +++++++------------
13 files changed, 619 insertions(+), 150 deletions(-)
create mode 100644 libfrog/histogram.c
create mode 100644 libfrog/histogram.h

[GIT PULL 14/23] xfs_scrub: tighten security of systemd services

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 34bed605490f936c3ead49e2e1cad78505260461:

xfs_scrub: tune fstrim minlen parameter based on free space histograms (2024-07-29 17:01:09 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-service-security-6.10_2024-07-29

for you to fetch changes up to 50411335572120153cc84d54213cd5ca9dd11b14:

xfs_scrub_all: tighten up the security on the background systemd service (2024-07-29 17:01:10 -0700)

----------------------------------------------------------------
xfs_scrub: tighten security of systemd services [v30.9 14/28]

To reduce the risk of the online fsck service suffering some sort of
catastrophic breach that results in attackers reconfiguring the running
system, I embarked on a security audit of the systemd service files.
The result should be that all elements of the background service
(individual scrub jobs, the scrub_all initiator, and the failure
reporting) run with as few privileges and within as strong of a sandbox
as possible.

Granted, this does nothing about the potential for the /kernel/ screwing
up, but at least we could prevent obvious container escapes.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (6):
xfs_scrub: allow auxiliary pathnames for sandboxing
xfs_scrub.service: reduce background CPU usage to less than one core if possible
xfs_scrub: use dynamic users when running as a systemd service
xfs_scrub: tighten up the security on the background systemd service
xfs_scrub_fail: tighten up the security on the background systemd service
xfs_scrub_all: tighten up the security on the background systemd service

man/man8/xfs_scrub.8             |  9 +++-
scrub/Makefile                   |  7 ++-
scrub/phase1.c                   |  4 +-
scrub/system-xfs_scrub.slice     | 30 +++++++++++++
scrub/vfs.c                      |  2 +-
scrub/xfs_scrub.c                | 11 +++--
scrub/xfs_scrub.h                |  5 ++-
scrub/xfs_scrub@.service.in      | 97 +++++++++++++++++++++++++++++++++++-----
scrub/xfs_scrub_all.service.in   | 66 +++++++++++++++++++++++++++
scrub/xfs_scrub_fail@.service.in | 59 ++++++++++++++++++++++++
10 files changed, 270 insertions(+), 20 deletions(-)
create mode 100644 scrub/system-xfs_scrub.slice

[GIT PULL 15/23] xfs_scrub_all: automatic media scan service

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 50411335572120153cc84d54213cd5ca9dd11b14:

xfs_scrub_all: tighten up the security on the background systemd service (2024-07-29 17:01:10 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-media-scan-service-6.10_2024-07-29

for you to fetch changes up to e040916f649f71fb6a695dfcc6f6c9f576c5c4db:

xfs_scrub_all: failure reporting for the xfs_scrub_all job (2024-07-29 17:01:10 -0700)

----------------------------------------------------------------
xfs_scrub_all: automatic media scan service [v30.9 15/28]

Now that we've completed the online fsck functionality, there are a few
things that could be improved in the automatic service.  Specifically,
we would like to perform a more intensive metadata + media scan once per
month, to give the user confidence that the filesystem isn't losing data
silently.  To accomplish this, enhance xfs_scrub_all to be able to
trigger media scans.  Next, add a duplicate set of system services that
start the media scans automatically.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (6):
xfs_scrub_all: only use the xfs_scrub@ systemd services in service mode
xfs_scrub_all: remove journalctl background process
xfs_scrub_all: support metadata+media scans of all filesystems
xfs_scrub_all: enable periodic file data scrubs automatically
xfs_scrub_all: trigger automatic media scans once per month
xfs_scrub_all: failure reporting for the xfs_scrub_all job

debian/rules                                     |   3 +-
include/builddefs.in                             |   3 +
man/man8/Makefile                                |   7 +-
man/man8/{xfs_scrub_all.8 => xfs_scrub_all.8.in} |  20 +++-
scrub/Makefile                                   |  21 +++-
scrub/xfs_scrub@.service.in                      |   2 +-
scrub/xfs_scrub_all.cron.in                      |   2 +-
scrub/xfs_scrub_all.in                           | 126 ++++++++++++++++++-----
scrub/xfs_scrub_all.service.in                   |   9 +-
scrub/xfs_scrub_all_fail.service.in              |  71 +++++++++++++
scrub/xfs_scrub_fail.in                          |  46 +++++++--
scrub/xfs_scrub_fail@.service.in                 |   2 +-
scrub/xfs_scrub_media@.service.in                | 100 ++++++++++++++++++
scrub/xfs_scrub_media_fail@.service.in           |  76 ++++++++++++++
14 files changed, 443 insertions(+), 45 deletions(-)
rename man/man8/{xfs_scrub_all.8 => xfs_scrub_all.8.in} (59%)
create mode 100644 scrub/xfs_scrub_all_fail.service.in
create mode 100644 scrub/xfs_scrub_media@.service.in
create mode 100644 scrub/xfs_scrub_media_fail@.service.in

[GIT PULL 16/23] xfs_scrub_all: improve systemd handling

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit e040916f649f71fb6a695dfcc6f6c9f576c5c4db:

xfs_scrub_all: failure reporting for the xfs_scrub_all job (2024-07-29 17:01:10 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-all-improve-systemd-handling-6.10_2024-07-29

for you to fetch changes up to e46249ec0b96ac5ced5b49928d3b3865b6d9546c:

xfs_scrub_all: implement retry and backoff for dbus calls (2024-07-29 17:01:10 -0700)

----------------------------------------------------------------
xfs_scrub_all: improve systemd handling [v30.9 16/28]

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (5):
xfs_scrub_all: encapsulate all the subprocess code in an object
xfs_scrub_all: encapsulate all the systemctl code in an object
xfs_scrub_all: add CLI option for easier debugging
xfs_scrub_all: convert systemctl calls to dbus
xfs_scrub_all: implement retry and backoff for dbus calls

debian/control         |   2 +-
scrub/xfs_scrub_all.in | 284 ++++++++++++++++++++++++++++++++++++-------------
2 files changed, 213 insertions(+), 73 deletions(-)

[GIT PULL 17/23] xfsprogs: improve extended attribute validation

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit e46249ec0b96ac5ced5b49928d3b3865b6d9546c:

xfs_scrub_all: implement retry and backoff for dbus calls (2024-07-29 17:01:10 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/improve-attr-validation-6.10_2024-07-29

for you to fetch changes up to 2823d8ed93da2bd3880abb52a58e91a920961e27:

xfs_repair: check for unknown flags in attr entries (2024-07-29 17:01:11 -0700)

----------------------------------------------------------------
xfsprogs: improve extended attribute validation [v13.8 17/28]

Prior to introducing parent pointer extended attributes, let's spend
some time cleaning up the attr code and strengthening the validation
that it performs on attrs coming in from the disk.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (6):
xfs_scrub_all: fail fast on masked units
xfs_scrub: automatic downgrades to dry-run mode in service mode
xfs_scrub: add an optimization-only mode
xfs_repair: check free space requirements before allowing upgrades
xfs_repair: enforce one namespace bit per extended attribute
xfs_repair: check for unknown flags in attr entries

include/libxfs.h         |   1 +
libxfs/libxfs_api_defs.h |   1 +
man/man8/xfs_scrub.8     |   6 ++-
repair/attr_repair.c     |  30 +++++++++++
repair/phase2.c          | 134 +++++++++++++++++++++++++++++++++++++++++++++++
scrub/Makefile           |   2 +-
scrub/phase1.c           |  13 +++++
scrub/phase4.c           |   6 +++
scrub/repair.c           |  37 ++++++++++++-
scrub/repair.h           |   2 +
scrub/scrub.c            |   4 +-
scrub/xfs_scrub.c        |  21 +++++++-
scrub/xfs_scrub.h        |   1 +
scrub/xfs_scrub_all.in   |  21 ++++++++
14 files changed, 272 insertions(+), 7 deletions(-)

[GIT PULL 18/23] xfsprogs: Parent Pointers

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 2823d8ed93da2bd3880abb52a58e91a920961e27:

xfs_repair: check for unknown flags in attr entries (2024-07-29 17:01:11 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/pptrs-6.10_2024-07-29

for you to fetch changes up to b2677fa4f4009abf8de4c15960a2e97dd5370d41:

mkfs: enable formatting with parent pointers (2024-07-29 17:01:12 -0700)

----------------------------------------------------------------
xfsprogs: Parent Pointers [v13.8 18/28]

This is the latest parent pointer attributes for xfs.  The goal of this
patch set is to add a parent pointer attribute to each inode.  The
attribute name containing the parent inode, generation, and directory
offset, while the  attribute value contains the file name.  This feature
will enable future optimizations for online scrub, shrink, nfs handles,
verity, or any other feature that could make use of quickly deriving an
inodes path from the mount point.

Directory parent pointers are stored as namespaced extended attributes
of a file.  Because parent pointers are an indivisible tuple of
(dirent_name, parent_ino, parent_gen) we cannot use the usual attr name
lookup functions to find a parent pointer.  This is solvable by
introducing a new lookup mode that checks both the name and the value of
the xattr.

Therefore, introduce this new name-value lookup mode that's gated on the
XFS_ATTR_PARENT namespace.  This requires the introduction of new
opcodes for the extended attribute update log intent items, which
actually means that parent pointers (itself an INCOMPAT feature) does
not depend on the LOGGED_XATTRS log incompat feature bit.

To reduce collisions on the dirent names of parent pointers, introduce a
new attr hash mode that is the dir2 namehash of the dirent name xor'd
with the parent inode number.

At this point, Allison has moved on to other things, so I've merged her
patchset into djwong-dev for merging.

Updates since v12 [djwong]:

Rebase on 6.9-rc and update the online fsck design document.
Redesign the ondisk format to use the name-value lookups to get us back
to the point where the attr is (dirent_name -> parent_ino/gen).

Updates since v11 [djwong]:

Rebase on 6.4-rc and make some tweaks and bugfixes to enable the repair
prototypes.  Merge with djwong-dev and make online repair actually work.

Updates since v10 [djwong]:

Merge in the ondisk format changes to get rid of the diroffset conflicts
with the parent pointer repair code, rebase the entire series with the
attr vlookup changes first, and merge all the other random fixes.

Updates since v9:

Reordered patches 2 and 3 to be 6 and 7

xfs: Add xfs_verify_pptr
moved parent pointer validators to xfs_parent

xfs: Add parent pointer ioctl
Extra validation checks for fs id
added missing release for the inode
use GFP_KERNEL flags for malloc/realloc
reworked ioctl to use pptr listenty and flex array

NEW
xfs: don't remove the attr fork when parent pointers are enabled

NEW
directory lookups should return diroffsets too

NEW
xfs: move/add parent pointer validators to xfs_parent

Updates since v8:

xfs: parent pointer attribute creation
Fix xfs_parent_init to release log assist on alloc fail
Add slab cache for xfs_parent_defer
Fix xfs_create to release after unlock
Add xfs_parent_start and xfs_parent_finish wrappers
removed unused xfs_parent_name_irec and xfs_init_parent_name_irec

xfs: add parent attributes to link
Start/finish wrapper updates
Fix xfs_link to disallow reservationless quotas

xfs: add parent attributes to symlink
Fix xfs_symlink to release after unlock
Start/finish wrapper updates

xfs: remove parent pointers in unlink
Start/finish wrapper updates
Add missing parent free

xfs: Add parent pointers to rename
Start/finish wrapper updates
Fix rename to only grab logged xattr once
Fix xfs_rename to disallow reservationless quotas
Fix double unlock on dqattach fail
Move parent frees to out_release_wip

xfs: Add parent pointers to xfs_cross_rename
Hoist parent pointers into rename

Questions comments and feedback appreciated!

Thanks all!
Allison

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Allison Henderson (4):
xfs_io: Add i, n and f flags to parent command
xfs_logprint: decode parent pointers in ATTRI items fully
mkfs: Add parent pointers during protofile creation
mkfs: enable formatting with parent pointers

Darrick J. Wong (20):
libxfs: create attr log item opcodes and formats for parent pointers
xfs_{db,repair}: implement new attr hash value function
xfs_logprint: dump new attr log item fields
man: document the XFS_IOC_GETPARENTS ioctl
libfrog: report parent pointers to userspace
libfrog: add parent pointer support code
xfs_io: adapt parent command to new parent pointer ioctls
xfs_spaceman: report file paths
xfs_scrub: use parent pointers when possible to report file operations
xfs_scrub: use parent pointers to report lost file data
xfs_db: report parent pointers in version command
xfs_db: report parent bit on xattrs
xfs_db: report parent pointers embedded in xattrs
xfs_db: obfuscate dirent and parent pointer names consistently
libxfs: export attr3_leaf_hdr_from_disk via libxfs_api_defs.h
xfs_db: add a parents command to list the parents of a file
xfs_db: make attr_set and attr_remove handle parent pointers
xfs_db: add link and unlink expert commands
xfs_db: compute hashes of parent pointers
libxfs: create new files with attr forks if necessary

db/attr.c                       |  33 +-
db/attrset.c                    | 202 +++++++++---
db/attrshort.c                  |  27 ++
db/field.c                      |  10 +
db/field.h                      |   3 +
db/hash.c                       |  44 ++-
db/metadump.c                   | 322 ++++++++++++++++--
db/namei.c                      | 701 ++++++++++++++++++++++++++++++++++++++++
db/sb.c                         |   2 +
include/handle.h                |   1 +
include/xfs_inode.h             |   4 +
io/parent.c                     | 541 +++++++++++--------------------
libfrog/Makefile                |   2 +
libfrog/fsgeom.c                |   6 +-
libfrog/getparents.c            | 355 ++++++++++++++++++++
libfrog/getparents.h            |  42 +++
libfrog/paths.c                 | 168 ++++++++++
libfrog/paths.h                 |  25 ++
libhandle/handle.c              |   7 +-
libxfs/defer_item.c             |  52 ++-
libxfs/init.c                   |   4 +
libxfs/libxfs_api_defs.h        |  19 ++
libxfs/util.c                   |  19 +-
logprint/log_redo.c             | 217 +++++++++++--
logprint/logprint.h             |   5 +-
man/man2/ioctl_xfs_getparents.2 | 212 ++++++++++++
man/man8/xfs_db.8               |  59 +++-
man/man8/xfs_io.8               |  32 +-
man/man8/xfs_spaceman.8         |   7 +-
mkfs/lts_4.19.conf              |   3 +
mkfs/lts_5.10.conf              |   3 +
mkfs/lts_5.15.conf              |   3 +
mkfs/lts_5.4.conf               |   3 +
mkfs/lts_6.1.conf               |   3 +
mkfs/lts_6.6.conf               |   3 +
mkfs/proto.c                    |  62 +++-
mkfs/xfs_mkfs.c                 |  45 ++-
repair/attr_repair.c            |  24 +-
scrub/common.c                  |  41 ++-
scrub/phase6.c                  |  75 ++++-
spaceman/Makefile               |  16 +-
spaceman/file.c                 |   7 +
spaceman/health.c               |  53 ++-
spaceman/space.h                |   3 +
44 files changed, 2941 insertions(+), 524 deletions(-)
create mode 100644 libfrog/getparents.c
create mode 100644 libfrog/getparents.h
create mode 100644 man/man2/ioctl_xfs_getparents.2

[GIT PULL 19/23] xfsprogs: scrubbing for parent pointers

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit b2677fa4f4009abf8de4c15960a2e97dd5370d41:

mkfs: enable formatting with parent pointers (2024-07-29 17:01:12 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-pptrs-6.10_2024-07-29

for you to fetch changes up to 4b327cc2f5d03b772dd6d3352cbe28452cd41ef0:

man2: update ioctl_xfs_scrub_metadata.2 for parent pointers (2024-07-29 17:01:12 -0700)

----------------------------------------------------------------
xfsprogs: scrubbing for parent pointers [v13.8 19/28]

Teach online fsck to use parent pointers to assist in checking
directories, parent pointers, extended attributes, and link counts.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (2):
xfs: create a blob array data structure
man2: update ioctl_xfs_scrub_metadata.2 for parent pointers

libxfs/Makefile                     |   2 +
libxfs/xfblob.c                     | 147 ++++++++++++++++++++++++++++++++++++
libxfs/xfblob.h                     |  24 ++++++
libxfs/xfile.c                      |  11 +++
libxfs/xfile.h                      |   1 +
man/man2/ioctl_xfs_scrub_metadata.2 |  20 ++++-
6 files changed, 201 insertions(+), 4 deletions(-)
create mode 100644 libxfs/xfblob.c
create mode 100644 libxfs/xfblob.h

[GIT PULL 20/23] xfsprogs: offline repair for parent pointers

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 4b327cc2f5d03b772dd6d3352cbe28452cd41ef0:

man2: update ioctl_xfs_scrub_metadata.2 for parent pointers (2024-07-29 17:01:12 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/repair-pptrs-6.10_2024-07-29

for you to fetch changes up to 7ea215189a3cac45cb9323439318fcc3410727d4:

xfs_repair: wipe ondisk parent pointers when there are none (2024-07-29 17:01:13 -0700)

----------------------------------------------------------------
xfsprogs: offline repair for parent pointers [v13.8 20/28]

This series implements online checking and repair for directory parent
pointer metadata.  The checking half is fairly straightforward -- for
each outgoing directory link (forward or backwards), grab the inode at
the other end, and confirm that there's a corresponding link.  If we
can't grab an inode or lock it, we'll save that link for a slower loop
that cycles all the locks, confirms the continued existence of the link,
and rechecks the link if it's actually still there.

Repairs are a bit more involved -- for directories, we walk the entire
filesystem to rebuild the dirents from parent pointer information.
Parent pointer repairs do the same walk but rebuild the pptrs from the
dirent information, but with the added twist that it duplicates all the
xattrs so that it can use the atomic extent swapping code to commit the
repairs atomically.

This introduces an added twist to the xattr repair code -- we use dirent
hooks to detect a colliding update to the pptr data while we're not
holding the ILOCKs; if one is detected, we restart the xattr salvaging
process but this time hold all the ILOCKs until the end of the scan.

For offline repair, the phase6 directory connectivity scan generates an
index of all the expected parent pointers in the filesystem.  Then it
walks each file and compares the parent pointers attached to that file
against the index generated, and resyncs the results as necessary.

The last patch teaches xfs_scrub to report pathnames of files that are
being repaired, when possible.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (12):
xfs_db: remove some boilerplate from xfs_attr_set
xfs_db: actually report errors from libxfs_attr_set
xfs_repair: junk parent pointer attributes when filesystem doesn't support them
xfs_repair: add parent pointers when messing with /lost+found
xfs_repair: junk duplicate hashtab entries when processing sf dirents
xfs_repair: build a parent pointer index
xfs_repair: move the global dirent name store to a separate object
xfs_repair: deduplicate strings stored in string blob
xfs_repair: check parent pointers
xfs_repair: dump garbage parent pointer attributes
xfs_repair: update ondisk parent pointer records
xfs_repair: wipe ondisk parent pointers when there are none

db/attrset.c             |   36 +-
libxfs/libxfs_api_defs.h |    6 +
libxfs/xfblob.c          |    9 +
libxfs/xfblob.h          |    2 +
repair/Makefile          |    6 +
repair/attr_repair.c     |   30 ++
repair/listxattr.c       |  271 ++++++++++
repair/listxattr.h       |   15 +
repair/phase6.c          |  121 ++++-
repair/pptr.c            | 1331 ++++++++++++++++++++++++++++++++++++++++++++++
repair/pptr.h            |   17 +
repair/strblobs.c        |  211 ++++++++
repair/strblobs.h        |   24 +
13 files changed, 2069 insertions(+), 10 deletions(-)
create mode 100644 repair/listxattr.c
create mode 100644 repair/listxattr.h
create mode 100644 repair/pptr.c
create mode 100644 repair/pptr.h
create mode 100644 repair/strblobs.c
create mode 100644 repair/strblobs.h

[GIT PULL 21/23] xfsprogs: detect and correct directory tree problems

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 7ea215189a3cac45cb9323439318fcc3410727d4:

xfs_repair: wipe ondisk parent pointers when there are none (2024-07-29 17:01:13 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/scrub-directory-tree-6.10_2024-07-29

for you to fetch changes up to 5a30504f0c60e10dc0cecd201c5afc18083fd0ac:

xfs_scrub: defer phase5 file scans if dirloop fails (2024-07-29 17:01:13 -0700)

----------------------------------------------------------------
xfsprogs: detect and correct directory tree problems [v13.8 21/28]

Historically, checking the tree-ness of the directory tree structure has
not been complete.  Cycles of subdirectories break the tree properties,
as do subdirectories with multiple parents.  It's easy enough for DFS to
detect problems as long as one of the participants is reachable from the
root, but this technique cannot find unconnected cycles.

Directory parent pointers change that, because we can discover all of
these problems from a simple walk from a subdirectory towards the root.
For each child we start with, if the walk terminates without reaching
the root, we know the path is disconnected and ought to be attached to
the lost and found.  If we find ourselves, we know this is a cycle and
can delete an incoming edge.  If we find multiple paths to the root, we
know to delete an incoming edge.

Even better, once we've finished walking paths, we've identified the
good ones and know which other path(s) to remove.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (5):
libfrog: add directory tree structure scrubber to scrub library
xfs_spaceman: report directory tree corruption in the health information
xfs_scrub: fix erroring out of check_inode_names
xfs_scrub: detect and repair directory tree corruptions
xfs_scrub: defer phase5 file scans if dirloop fails

libfrog/scrub.c                     |   5 +
man/man2/ioctl_xfs_bulkstat.2       |   3 +
man/man2/ioctl_xfs_fsbulkstat.2     |   3 +
man/man2/ioctl_xfs_scrub_metadata.2 |  14 ++
scrub/phase5.c                      | 271 ++++++++++++++++++++++++++++++++++--
scrub/repair.c                      |  13 ++
scrub/repair.h                      |   2 +
spaceman/health.c                   |   4 +
8 files changed, 301 insertions(+), 14 deletions(-)

[GIT PULL 22/23] xfs_scrub: vectorize kernel calls

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit 5a30504f0c60e10dc0cecd201c5afc18083fd0ac:

xfs_scrub: defer phase5 file scans if dirloop fails (2024-07-29 17:01:13 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/vectorized-scrub-6.10_2024-07-29

for you to fetch changes up to df914edeeb1e1919831192951d657cfc73b46418:

xfs_scrub: try spot repairs of metadata items to make scrub progress (2024-07-29 17:01:13 -0700)

----------------------------------------------------------------
xfs_scrub: vectorize kernel calls [v30.9 22/28]

Create a vectorized version of the metadata scrub and repair ioctl, and
adapt xfs_scrub to use that.  This is an experiment to measure overhead
and to try refactoring xfs_scrub.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (10):
man: document vectored scrub mode
libfrog: support vectored scrub
xfs_io: support vectored scrub
xfs_scrub: split the scrub epilogue code into a separate function
xfs_scrub: split the repair epilogue code into a separate function
xfs_scrub: convert scrub and repair epilogues to use xfs_scrub_vec
xfs_scrub: vectorize scrub calls
xfs_scrub: vectorize repair calls
xfs_scrub: use scrub barriers to reduce kernel calls
xfs_scrub: try spot repairs of metadata items to make scrub progress

io/scrub.c                           | 368 +++++++++++++++++++++++++++++++----
libfrog/fsgeom.h                     |   6 +
libfrog/scrub.c                      | 137 +++++++++++++
libfrog/scrub.h                      |  35 ++++
man/man2/ioctl_xfs_scrubv_metadata.2 | 171 ++++++++++++++++
man/man8/xfs_io.8                    |  51 +++++
scrub/phase1.c                       |   2 +
scrub/phase2.c                       |  93 +++++++--
scrub/phase3.c                       |  84 ++++++--
scrub/repair.c                       | 355 ++++++++++++++++++++-------------
scrub/scrub.c                        | 360 +++++++++++++++++++++++++---------
scrub/scrub.h                        |  19 ++
scrub/scrub_private.h                |  55 ++++--
scrub/xfs_scrub.c                    |   1 +
14 files changed, 1431 insertions(+), 306 deletions(-)
create mode 100644 man/man2/ioctl_xfs_scrubv_metadata.2

[GIT PULL 23/23] xfs_repair: fixes for kernel 6.10

[Darrick J. Wong]

Hi Carlos,

Please pull this branch with changes for xfsprogs for 6.10-rc1.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

The following changes since commit df914edeeb1e1919831192951d657cfc73b46418:

xfs_scrub: try spot repairs of metadata items to make scrub progress (2024-07-29 17:01:13 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/repair-fixes-6.10_2024-07-29

for you to fetch changes up to 5a43a00432ebe9ab8b54155703a9eb9e1a1dd4ec:

xfs_repair: allow symlinks with short remote targets (2024-07-29 17:01:13 -0700)

----------------------------------------------------------------
xfs_repair: fixes for kernel 6.10 [v30.9 23/28]

Fix some incorrect validation problems in xfs_repair.

This has been running on the djcloud for months with no problems.  Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (1):
xfs_repair: allow symlinks with short remote targets

repair/dinode.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

Re: [GIT PULLBOMB] xfsprogs: catch us up to 6.10

[Carlos Maiolino]

On Mon, Jul 29, 2024 at 06:36:26PM GMT, Darrick J. Wong wrote:
> Hi Carlos,
> 
> Please pull these fully reviewed patchsets for xfsprogs 6.10.

All pulled, thanks!

Carlos

> 
> --D
>