[PATCHSET v5.1 3/9] xfs: metadata inode directory trees

[PATCHSET v5.1 3/9] xfs: metadata inode directory trees

[Darrick J. Wong]

Hi all,

This series delivers a new feature -- metadata inode directories.  This
is a separate directory tree (rooted in the superblock) that contains
only inodes that contain filesystem metadata.  Different metadata
objects can be looked up with regular paths.

Start by creating xfs_imeta{dir,file}* functions to mediate access to
the metadata directory tree.  By the end of this mega series, all
existing metadata inodes (rt+quota) will use this directory tree instead
of the superblock.

Next, define the metadir on-disk format, which consists of marking
inodes with a new iflag that says they're metadata.  This prevents
bulkstat and friends from ever getting their hands on fs metadata files.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=metadata-directory-tree

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=metadata-directory-tree
---
Commits in this patchset:
 * xfs: constify the xfs_sb predicates
 * xfs: constify the xfs_inode predicates
 * xfs: rename metadata inode predicates
 * xfs: standardize EXPERIMENTAL warning generation
 * xfs: define the on-disk format for the metadir feature
 * xfs: iget for metadata inodes
 * xfs: load metadata directory root at mount time
 * xfs: enforce metadata inode flag
 * xfs: read and write metadata inode directory tree
 * xfs: disable the agi rotor for metadata inodes
 * xfs: hide metadata inodes from everyone because they are special
 * xfs: advertise metadata directory feature
 * xfs: allow bulkstat to return metadata directories
 * xfs: don't count metadata directory files to quota
 * xfs: mark quota inodes as metadata files
 * xfs: adjust xfs_bmap_add_attrfork for metadir
 * xfs: record health problems with the metadata directory
 * xfs: refactor directory tree root predicates
 * xfs: do not count metadata directory files when doing online quotacheck
 * xfs: don't fail repairs on metadata files with no attr fork
 * xfs: metadata files can have xattrs if metadir is enabled
 * xfs: adjust parent pointer scrubber for sb-rooted metadata files
 * xfs: fix di_metatype field of inodes that won't load
 * xfs: scrub metadata directories
 * xfs: check the metadata directory inumber in superblocks
 * xfs: move repair temporary files to the metadata directory tree
 * xfs: check metadata directory file path connectivity
 * xfs: confirm dotdot target before replacing it during a repair
 * xfs: repair metadata directory file path connectivity
---
 fs/xfs/Makefile                 |    5 
 fs/xfs/libxfs/xfs_attr.c        |    5 
 fs/xfs/libxfs/xfs_bmap.c        |    5 
 fs/xfs/libxfs/xfs_format.h      |  121 +++++++--
 fs/xfs/libxfs/xfs_fs.h          |   25 ++
 fs/xfs/libxfs/xfs_health.h      |    6 
 fs/xfs/libxfs/xfs_ialloc.c      |   58 +++-
 fs/xfs/libxfs/xfs_inode_buf.c   |   90 ++++++-
 fs/xfs/libxfs/xfs_inode_buf.h   |    3 
 fs/xfs/libxfs/xfs_inode_util.c  |    2 
 fs/xfs/libxfs/xfs_log_format.h  |    2 
 fs/xfs/libxfs/xfs_metadir.c     |  481 ++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_metadir.h     |   47 ++++
 fs/xfs/libxfs/xfs_metafile.c    |   52 ++++
 fs/xfs/libxfs/xfs_metafile.h    |   31 ++
 fs/xfs/libxfs/xfs_ondisk.h      |    2 
 fs/xfs/libxfs/xfs_sb.c          |   12 +
 fs/xfs/libxfs/xfs_types.c       |    4 
 fs/xfs/libxfs/xfs_types.h       |    2 
 fs/xfs/scrub/agheader.c         |    5 
 fs/xfs/scrub/common.c           |   65 ++++-
 fs/xfs/scrub/common.h           |    5 
 fs/xfs/scrub/dir.c              |   10 +
 fs/xfs/scrub/dir_repair.c       |   20 +
 fs/xfs/scrub/dirtree.c          |   32 ++
 fs/xfs/scrub/dirtree.h          |   12 -
 fs/xfs/scrub/findparent.c       |   28 ++
 fs/xfs/scrub/health.c           |    1 
 fs/xfs/scrub/inode.c            |   35 ++-
 fs/xfs/scrub/inode_repair.c     |   34 ++-
 fs/xfs/scrub/metapath.c         |  521 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/scrub/nlinks.c           |    4 
 fs/xfs/scrub/nlinks_repair.c    |    4 
 fs/xfs/scrub/orphanage.c        |    4 
 fs/xfs/scrub/parent.c           |   39 ++-
 fs/xfs/scrub/parent_repair.c    |   37 ++-
 fs/xfs/scrub/quotacheck.c       |    7 -
 fs/xfs/scrub/refcount_repair.c  |    2 
 fs/xfs/scrub/repair.c           |   22 +-
 fs/xfs/scrub/repair.h           |    3 
 fs/xfs/scrub/scrub.c            |   12 +
 fs/xfs/scrub/scrub.h            |    2 
 fs/xfs/scrub/stats.c            |    1 
 fs/xfs/scrub/tempfile.c         |  105 ++++++++
 fs/xfs/scrub/tempfile.h         |    3 
 fs/xfs/scrub/trace.c            |    1 
 fs/xfs/scrub/trace.h            |   42 +++
 fs/xfs/xfs_dquot.c              |    1 
 fs/xfs/xfs_fsops.c              |    4 
 fs/xfs/xfs_health.c             |    2 
 fs/xfs/xfs_icache.c             |   74 ++++++
 fs/xfs/xfs_inode.c              |   19 +
 fs/xfs/xfs_inode.h              |   36 ++-
 fs/xfs/xfs_inode_item.c         |    7 -
 fs/xfs/xfs_inode_item_recover.c |    2 
 fs/xfs/xfs_ioctl.c              |    7 +
 fs/xfs/xfs_iops.c               |   15 +
 fs/xfs/xfs_itable.c             |   33 ++
 fs/xfs/xfs_itable.h             |    3 
 fs/xfs/xfs_message.c            |   47 ++++
 fs/xfs/xfs_message.h            |   19 +
 fs/xfs/xfs_mount.c              |   31 ++
 fs/xfs/xfs_mount.h              |   11 +
 fs/xfs/xfs_qm.c                 |   36 +++
 fs/xfs/xfs_quota.h              |    5 
 fs/xfs/xfs_rtalloc.c            |   38 ++-
 fs/xfs/xfs_super.c              |   13 -
 fs/xfs/xfs_trace.c              |    2 
 fs/xfs/xfs_trace.h              |  102 ++++++++
 fs/xfs/xfs_trans_dquot.c        |    6 
 fs/xfs/xfs_xattr.c              |    3 
 71 files changed, 2324 insertions(+), 201 deletions(-)
 create mode 100644 fs/xfs/libxfs/xfs_metadir.c
 create mode 100644 fs/xfs/libxfs/xfs_metadir.h
 create mode 100644 fs/xfs/libxfs/xfs_metafile.c
 create mode 100644 fs/xfs/libxfs/xfs_metafile.h
 create mode 100644 fs/xfs/scrub/metapath.c

[PATCH 20/29] xfs: don't fail repairs on metadata files with no attr fork

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix a minor bug where we fail repairs on metadata files that do not have
attr forks because xrep_metadata_inode_subtype doesn't filter ENOENT.

Cc: <stable@vger.kernel.org> # v6.8
Fixes: 5a8e07e799721b ("xfs: repair the inode core and forks of a metadata inode")
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/scrub/repair.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)


diff --git a/fs/xfs/scrub/repair.c b/fs/xfs/scrub/repair.c
index 3b4f6c207576a6..646ac8ade88d0b 100644
--- a/fs/xfs/scrub/repair.c
+++ b/fs/xfs/scrub/repair.c
@@ -1083,9 +1083,11 @@ xrep_metadata_inode_forks(
 		return error;
 
 	/* Make sure the attr fork looks ok before we delete it. */
-	error = xrep_metadata_inode_subtype(sc, XFS_SCRUB_TYPE_BMBTA);
-	if (error)
-		return error;
+	if (xfs_inode_hasattr(sc->ip)) {
+		error = xrep_metadata_inode_subtype(sc, XFS_SCRUB_TYPE_BMBTA);
+		if (error)
+			return error;
+	}
 
 	/* Clear the reflink flag since metadata never shares. */
 	if (xfs_is_reflink_inode(sc->ip)) {

Re: [PATCH 20/29] xfs: don't fail repairs on metadata files with no attr fork

[Greg KH]

On Thu, Oct 17, 2024 at 11:58:10AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Fix a minor bug where we fail repairs on metadata files that do not have
> attr forks because xrep_metadata_inode_subtype doesn't filter ENOENT.
> 
> Cc: <stable@vger.kernel.org> # v6.8
> Fixes: 5a8e07e799721b ("xfs: repair the inode core and forks of a metadata inode")
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> ---
>  fs/xfs/scrub/repair.c |    8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)

Why is a bugfix / stable-tagged-patch, number 20 in a 29 patch series?
Why isn't it first, or better yet, on it's own if it is fixing a bug
that people want merged "soon"?

thanks,

greg k-h

Re: [PATCH 20/29] xfs: don't fail repairs on metadata files with no attr fork

[Darrick J. Wong]

On Fri, Oct 18, 2024 at 08:00:21AM +0200, Greg KH wrote:
> On Thu, Oct 17, 2024 at 11:58:10AM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Fix a minor bug where we fail repairs on metadata files that do not have
> > attr forks because xrep_metadata_inode_subtype doesn't filter ENOENT.
> > 
> > Cc: <stable@vger.kernel.org> # v6.8
> > Fixes: 5a8e07e799721b ("xfs: repair the inode core and forks of a metadata inode")
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > ---
> >  fs/xfs/scrub/repair.c |    8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> Why is a bugfix / stable-tagged-patch, number 20 in a 29 patch series?
> Why isn't it first, or better yet, on it's own if it is fixing a bug
> that people want merged "soon"?

I have too many patches, and every time I try to get a set through the
review process I end up having to write *more* patches to appease the
reviewers, and fixes get lost.  Look at the copyrights on the other
patches, I've been trying to get this upstreamed since 2018.

This particular bugfix got lost last month probably because I forgot to
ping cem to take it for 6.12-rc1.  Thanks for pushing on this, Greg.

Hey Carlos, can you queue this one up for 6.12-rc5, please?

--D

> thanks,
> 
> greg k-h
> 

Re: [PATCH 20/29] xfs: don't fail repairs on metadata files with no attr fork

[Carlos Maiolino]

On Mon, Oct 21, 2024 at 10:27:51AM GMT, Darrick J. Wong wrote:
> On Fri, Oct 18, 2024 at 08:00:21AM +0200, Greg KH wrote:
> > On Thu, Oct 17, 2024 at 11:58:10AM -0700, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > Fix a minor bug where we fail repairs on metadata files that do not have
> > > attr forks because xrep_metadata_inode_subtype doesn't filter ENOENT.
> > > 
> > > Cc: <stable@vger.kernel.org> # v6.8
> > > Fixes: 5a8e07e799721b ("xfs: repair the inode core and forks of a metadata inode")
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > >  fs/xfs/scrub/repair.c |    8 +++++---
> > >  1 file changed, 5 insertions(+), 3 deletions(-)
> > 
> > Why is a bugfix / stable-tagged-patch, number 20 in a 29 patch series?
> > Why isn't it first, or better yet, on it's own if it is fixing a bug
> > that people want merged "soon"?
> 
> I have too many patches, and every time I try to get a set through the
> review process I end up having to write *more* patches to appease the
> reviewers, and fixes get lost.  Look at the copyrights on the other
> patches, I've been trying to get this upstreamed since 2018.
> 
> This particular bugfix got lost last month probably because I forgot to
> ping cem to take it for 6.12-rc1.  Thanks for pushing on this, Greg.
> 
> Hey Carlos, can you queue this one up for 6.12-rc5, please?

Sure, it's queued now in next-rc, I'm testing it and I hope to send it to
for-next today yet. It's the only patch for -rc5.

Cheers.
Carlos

> 
> --D
> 
> > thanks,
> > 
> > greg k-h
> > 
> 

Re: [PATCH 20/29] xfs: don't fail repairs on metadata files with no attr fork

[Carlos Maiolino]

On Thu, 17 Oct 2024 11:58:10 -0700, Darrick J. Wong wrote:
> Fix a minor bug where we fail repairs on metadata files that do not have
> attr forks because xrep_metadata_inode_subtype doesn't filter ENOENT.
> 
> 

Applied to for-next, thanks!

[20/29] xfs: don't fail repairs on metadata files with no attr fork
        commit: af8512c5277d17aae09be5305daa9118d2fa8881

Best regards,
-- 
Carlos Maiolino <cem@kernel.org>