From: "Darrick J. Wong" <djwong@kernel.org>
To: Allison Henderson <allison.henderson@oracle.com>
Cc: xfs <linux-xfs@vger.kernel.org>, fstests <fstests@vger.kernel.org>
Subject: [PATCHSET DELUGE v11] xfs: Parent Pointers
Date: Thu, 6 Apr 2023 11:10:38 -0700 [thread overview]
Message-ID: <20230406181038.GA360889@frogsfrogsfrogs> (raw)
Hi everyone,
This submission contains all of the changes to the parent pointers
patchset that I've been working since last month's deluge. The kernel
and xfsprogs patchsets are based on Allison's v10 tag from some time
ago. To recap Allison's cover letter:
"The goal of this patch set is to add a parent pointer attribute to each
inode. The attribute name containing the parent inode, generation, and
directory offset, while the attribute value contains the file name.
This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."
v11 rebases everything against 6.3-rc5, and weaves all the changes that
I had made against v10 into Allison's original series. The new xattr
NVLOOKUP mode that I introduced for v10 is critical for handling parent
pointer attr name collisions with grace, so that has been retained from
v10. With that in place, I've replaced the diroffset in the ondisk
parent pointer with the dirent hash of the name.
Parent pointers now look like this:
(parent_ino, parent_gen, namehash) -> (name[])
I experimented with replacing the dahash with crc32c for this patchset
but left it out, having concluded that checksum operation has higher
overhead (thanks, crypto api!), would require someone to figure out crc
spoofing sufficiently well to keep metadump name obfuscation working,
and doesn't seem to improve collision resistance sufficiently to be
worth the added engineering cost.
As of this submission, I think this feature is ready to go once we've
merged the online repair code and rebased the online repair code to
actually commit the repaired directories.
If you want to pull the whole thing, use these links:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
--D
next reply other threads:[~2023-04-06 18:10 UTC|newest]
Thread overview: 121+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-06 18:10 Darrick J. Wong [this message]
2023-04-06 19:07 ` [PATCHSET DELUGE v11] xfs: Parent Pointers Darrick J. Wong
2023-04-06 19:13 ` [PATCHSET v11 00/12] xfs: name-value xattr lookups Darrick J. Wong
2023-04-06 19:16 ` [PATCH 01/12] xfs: check opcode and iovec count match in xlog_recover_attri_commit_pass2 Darrick J. Wong
2023-04-06 19:17 ` [PATCH 02/12] xfs: make xfs_attr_set require XFS_DA_OP_REMOVE Darrick J. Wong
2023-04-06 19:17 ` [PATCH 03/12] xfs: allow xattr matching on name and value for local/sf attrs Darrick J. Wong
2023-04-06 19:17 ` [PATCH 04/12] xfs: preserve NVLOOKUP in xfs_attr_set Darrick J. Wong
2023-04-06 19:17 ` [PATCH 05/12] xfs: restructure xfs_attr_complete_op a bit Darrick J. Wong
2023-04-06 19:18 ` [PATCH 06/12] xfs: use helpers to extract xattr op from opflags Darrick J. Wong
2023-04-06 19:18 ` [PATCH 07/12] xfs: validate recovered name buffers when recovering xattr items Darrick J. Wong
2023-04-06 19:18 ` [PATCH 08/12] xfs: always set args->value in xfs_attri_item_recover Darrick J. Wong
2023-04-06 19:18 ` [PATCH 09/12] xfs: use local variables for name and value length in _attri_commit_pass2 Darrick J. Wong
2023-04-06 19:19 ` [PATCH 10/12] xfs: log NVLOOKUP xattr removal operations Darrick J. Wong
2023-04-06 19:19 ` [PATCH 11/12] xfs: log NVLOOKUP xattr setting operations Darrick J. Wong
2023-04-06 19:19 ` [PATCH 12/12] xfs: log NVLOOKUP xattr nvreplace operations Darrick J. Wong
2023-04-06 19:14 ` [PATCHSET v11 0/3] xfs: hold ILOCK during deferred dir ops Darrick J. Wong
2023-04-06 19:19 ` [PATCH 1/3] xfs: Hold inode locks in xfs_ialloc Darrick J. Wong
2023-04-06 19:20 ` [PATCH 2/3] xfs: Hold inode locks in xfs_trans_alloc_dir Darrick J. Wong
2023-04-06 19:20 ` [PATCH 3/3] xfs: Hold inode locks in xfs_rename Darrick J. Wong
2023-04-06 19:14 ` [PATCHSET v11 00/23] xfs: Parent Pointers Darrick J. Wong
2023-04-06 19:20 ` [PATCH 01/23] xfs: Expose init_xattrs in xfs_create_tmpfile Darrick J. Wong
2023-04-06 19:20 ` [PATCH 02/23] xfs: Increase XFS_DEFER_OPS_NR_INODES to 5 Darrick J. Wong
2023-04-06 19:21 ` [PATCH 03/23] xfs: Increase XFS_QM_TRANS_MAXDQS " Darrick J. Wong
2023-04-06 19:21 ` [PATCH 04/23] xfs: add parent pointer support to attribute code Darrick J. Wong
2023-04-06 19:21 ` [PATCH 05/23] xfs: define parent pointer ondisk extended attribute format Darrick J. Wong
2023-04-06 19:21 ` [PATCH 06/23] xfs: add parent pointer validator functions Darrick J. Wong
2023-04-06 19:22 ` [PATCH 07/23] xfs: extend transaction reservations for parent attributes Darrick J. Wong
2023-04-06 19:22 ` [PATCH 08/23] xfs: parent pointer attribute creation Darrick J. Wong
2023-04-06 19:22 ` [PATCH 09/23] xfs: add parent attributes to link Darrick J. Wong
2023-04-06 19:23 ` [PATCH 10/23] xfs: add parent attributes to symlink Darrick J. Wong
2023-04-06 19:23 ` [PATCH 11/23] xfs: remove parent pointers in unlink Darrick J. Wong
2023-04-06 19:23 ` [PATCH 12/23] xfs: Indent xfs_rename Darrick J. Wong
2023-04-06 19:23 ` [PATCH 13/23] xfs: Add parent pointers to rename Darrick J. Wong
2023-04-06 19:24 ` [PATCH 14/23] xfs: Add parent pointers to xfs_cross_rename Darrick J. Wong
2023-04-06 19:24 ` [PATCH 15/23] xfs: Add helper function xfs_attr_list_context_init Darrick J. Wong
2023-04-06 19:24 ` [PATCH 16/23] xfs: Filter XFS_ATTR_PARENT for getfattr Darrick J. Wong
2023-04-06 19:24 ` [PATCH 17/23] xfs: pass the attr value to put_listent when possible Darrick J. Wong
2023-04-06 19:25 ` [PATCH 18/23] xfs: add a libxfs header file for staging new ioctls Darrick J. Wong
2023-04-06 19:25 ` [PATCH 19/23] xfs: Add parent pointer ioctl Darrick J. Wong
2023-04-06 19:25 ` [PATCH 20/23] xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res Darrick J. Wong
2023-04-06 19:25 ` [PATCH 21/23] xfs: drop compatibility minimum log size computations for reflink Darrick J. Wong
2023-04-06 19:26 ` [PATCH 22/23] xfs: don't remove the attr fork when parent pointers are enabled Darrick J. Wong
2023-04-06 19:26 ` [PATCH 23/23] xfs: Add the parent pointer support to the superblock version 5 Darrick J. Wong
2023-04-06 19:14 ` [PATCHSET v11 0/3] xfs: online repair of directories Darrick J. Wong
2023-04-06 19:26 ` [PATCH 1/3] xfs: reconstruct directories from parent pointers Darrick J. Wong
2023-04-06 19:26 ` [PATCH 2/3] xfs: add hooks to do directory updates Darrick J. Wong
2023-04-06 19:27 ` [PATCH 3/3] xfs: compare generated and existing dirents Darrick J. Wong
2023-04-06 19:14 ` [PATCHSET v11 0/2] xfs: online checking of parent pointers Darrick J. Wong
2023-04-06 19:27 ` [PATCH 1/2] xfs: scrub " Darrick J. Wong
2023-04-06 19:27 ` [PATCH 2/2] xfs: deferred scrub of " Darrick J. Wong
2023-04-06 19:15 ` [PATCHSET v11 0/3] xfs: online checking " Darrick J. Wong
2023-04-06 19:27 ` [PATCH 1/3] xfs: repair parent pointers by scanning directories Darrick J. Wong
2023-04-06 19:28 ` [PATCH 2/3] xfs: repair parent pointers with live scan hooks Darrick J. Wong
2023-04-06 19:28 ` [PATCH 3/3] xfs: compare generated and existing parent pointers Darrick J. Wong
2023-04-06 19:15 ` [PATCHSET v11 0/2] xfs: online checking of directories Darrick J. Wong
2023-04-06 19:28 ` [PATCH 1/2] xfs: check dirents have parent pointers Darrick J. Wong
2023-04-06 19:28 ` [PATCH 2/2] xfs: deferred scrub of dirents Darrick J. Wong
2023-04-06 19:15 ` [PATCHSET v11 00/10] xfs: name-value xattr lookups Darrick J. Wong
2023-04-06 19:29 ` [PATCH 01/10] xfs: make xfs_attr_set require XFS_DA_OP_REMOVE Darrick J. Wong
2023-04-06 19:29 ` [PATCH 02/10] xfs: allow xattr matching on name and value for local/sf attrs Darrick J. Wong
2023-04-06 19:29 ` [PATCH 03/10] xfs: preserve NVLOOKUP in xfs_attr_set Darrick J. Wong
2023-04-06 19:30 ` [PATCH 04/10] xfs: restructure xfs_attr_complete_op a bit Darrick J. Wong
2023-04-06 19:30 ` [PATCH 05/10] xfs: use helpers to extract xattr op from opflags Darrick J. Wong
2023-04-06 19:30 ` [PATCH 06/10] xfs: log NVLOOKUP xattr removal operations Darrick J. Wong
2023-04-06 19:30 ` [PATCH 07/10] xfs: log NVLOOKUP xattr setting operations Darrick J. Wong
2023-04-06 19:31 ` [PATCH 08/10] xfs: log NVLOOKUP xattr nvreplace operations Darrick J. Wong
2023-04-06 19:31 ` [PATCH 09/10] xfs_logprint: dump new attr log item fields Darrick J. Wong
2023-04-06 19:31 ` [PATCH 10/10] xfs_logprint: print missing attri header fields Darrick J. Wong
2023-04-06 19:15 ` [PATCHSET v11 00/32] xfsprogs: Parent Pointers Darrick J. Wong
2023-04-06 19:31 ` [PATCH 01/32] xfsprogs: Increase XFS_DEFER_OPS_NR_INODES to 5 Darrick J. Wong
2023-04-06 19:32 ` [PATCH 02/32] xfs: add parent pointer support to attribute code Darrick J. Wong
2023-04-06 19:32 ` [PATCH 03/32] xfs: define parent pointer ondisk extended attribute format Darrick J. Wong
2023-04-06 19:32 ` [PATCH 04/32] xfs: add parent pointer validator functions Darrick J. Wong
2023-04-06 19:32 ` [PATCH 05/32] xfs: extend transaction reservations for parent attributes Darrick J. Wong
2023-04-06 19:33 ` [PATCH 06/32] xfs: parent pointer attribute creation Darrick J. Wong
2023-04-06 19:33 ` [PATCH 07/32] xfs: add parent attributes to link Darrick J. Wong
2023-04-06 19:33 ` [PATCH 08/32] xfs: add parent attributes to symlink Darrick J. Wong
2023-04-06 19:33 ` [PATCH 09/32] xfs: remove parent pointers in unlink Darrick J. Wong
2023-04-06 19:34 ` [PATCH 10/32] xfs: Add parent pointers to rename Darrick J. Wong
2023-04-06 19:34 ` [PATCH 11/32] xfsprogs: pass the attr value to put_listent when possible Darrick J. Wong
2023-04-06 19:34 ` [PATCH 12/32] xfs: add a libxfs header file for staging new ioctls Darrick J. Wong
2023-04-06 19:34 ` [PATCH 13/32] xfs: Add parent pointer ioctl Darrick J. Wong
2023-04-06 19:35 ` [PATCH 14/32] xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res Darrick J. Wong
2023-04-06 19:35 ` [PATCH 15/32] xfs: drop compatibility minimum log size computations for reflink Darrick J. Wong
2023-04-06 19:35 ` [PATCH 16/32] xfs: don't remove the attr fork when parent pointers are enabled Darrick J. Wong
2023-04-06 19:35 ` [PATCH 17/32] xfs: Add the parent pointer support to the superblock version 5 Darrick J. Wong
2023-04-06 19:36 ` [PATCH 18/32] libfrog: add parent pointer support code Darrick J. Wong
2023-04-06 19:36 ` [PATCH 19/32] xfs_io: adapt parent command to new parent pointer ioctls Darrick J. Wong
2023-04-06 19:36 ` [PATCH 20/32] xfs_io: Add i, n and f flags to parent command Darrick J. Wong
2023-04-06 19:37 ` [PATCH 21/32] xfs_logprint: decode parent pointers in ATTRI items fully Darrick J. Wong
2023-04-06 19:37 ` [PATCH 22/32] xfs_scrub: use parent pointers when possible to report file operations Darrick J. Wong
2023-04-06 19:37 ` [PATCH 23/32] xfs_db: report parent bit on xattrs Darrick J. Wong
2023-04-06 19:37 ` [PATCH 24/32] xfs_db: report parent pointers embedded in xattrs Darrick J. Wong
2023-04-06 19:38 ` [PATCH 25/32] xfs_db: obfuscate dirent and parent pointer names consistently Darrick J. Wong
2023-04-06 19:38 ` [PATCH 26/32] xfs_db: hoist name obfuscation code out of metadump.c Darrick J. Wong
2023-04-06 19:38 ` [PATCH 27/32] xfs_db: create dirents and xattrs with colliding names Darrick J. Wong
2023-04-06 19:38 ` [PATCH 28/32] xfs_db: add a parents command to list the parents of a file Darrick J. Wong
2023-04-06 19:39 ` [PATCH 29/32] libxfs: create new files with attr forks if necessary Darrick J. Wong
2023-04-06 19:39 ` [PATCH 30/32] xfsprogs: Fix default superblock attr bits Darrick J. Wong
2023-04-06 19:39 ` [PATCH 31/32] mkfs: Add parent pointers during protofile creation Darrick J. Wong
2023-04-06 19:39 ` [PATCH 32/32] mkfs: enable formatting with parent pointers Darrick J. Wong
2023-04-06 19:16 ` [PATCHSET v11 0/7] xfs_repair: support " Darrick J. Wong
2023-04-06 19:40 ` [PATCH 1/7] xfs_repair: build a parent pointer index Darrick J. Wong
2023-04-06 19:40 ` [PATCH 2/7] xfs_repair: move the global dirent name store to a separate object Darrick J. Wong
2023-04-06 19:40 ` [PATCH 3/7] xfs_repair: deduplicate strings stored in string blob Darrick J. Wong
2023-04-06 19:40 ` [PATCH 4/7] xfs_repair: check parent pointers Darrick J. Wong
2023-04-06 19:41 ` [PATCH 5/7] xfs_repair: dump garbage parent pointer attributes Darrick J. Wong
2023-04-06 19:41 ` [PATCH 6/7] xfs_repair: update ondisk parent pointer records Darrick J. Wong
2023-04-06 19:41 ` [PATCH 7/7] xfs_repair: wipe ondisk parent pointers when there are none Darrick J. Wong
2023-04-06 19:16 ` [PATCHSET v11 00/11] fstests: adjust tests for xfs parent pointers Darrick J. Wong
2023-04-06 19:41 ` [PATCH 01/11] xfs/206: filter out the parent= status from mkfs Darrick J. Wong
2023-04-06 19:42 ` [PATCH 02/11] xfs/122: update for parent pointers Darrick J. Wong
2023-04-06 19:42 ` [PATCH 03/11] populate: create hardlinks " Darrick J. Wong
2023-04-06 19:42 ` [PATCH 04/11] xfs/021: adapt golden output files " Darrick J. Wong
2023-04-06 19:43 ` [PATCH 05/11] generic/050: adapt " Darrick J. Wong
2023-04-06 19:43 ` [PATCH 06/11] xfs/018: disable parent pointers for this test Darrick J. Wong
2023-04-06 19:43 ` [PATCH 07/11] xfs/306: fix formatting failures with parent pointers Darrick J. Wong
2023-04-06 19:43 ` [PATCH 08/11] common: add helpers for parent pointer tests Darrick J. Wong
2023-04-06 19:44 ` [PATCH 09/11] xfs: add parent pointer test Darrick J. Wong
2023-04-06 19:44 ` [PATCH 10/11] xfs: add multi link " Darrick J. Wong
2023-04-06 19:44 ` [PATCH 11/11] xfs: add parent pointer inject test Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230406181038.GA360889@frogsfrogsfrogs \
--to=djwong@kernel.org \
--cc=allison.henderson@oracle.com \
--cc=fstests@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox