linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCHSET v29.4 06/13] xfs: set and validate dir/attr block owners
@ 2024-02-27  2:18 Darrick J. Wong
  2024-02-27  2:26 ` [PATCH 1/9] xfs: add an explicit owner field to xfs_da_args Darrick J. Wong
                   ` (8 more replies)
  0 siblings, 9 replies; 24+ messages in thread
From: Darrick J. Wong @ 2024-02-27  2:18 UTC (permalink / raw)
  To: djwong; +Cc: linux-xfs, hch

Hi all,

There are a couple of significant changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number before the swap, we instead change libxfs to allow
callers of the dir and xattr code the ability to set an explicit owner
number to be written into the header fields of any new blocks that are
created.  For regular operation this will be the directory inode number.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dirattr-validate-owners

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=dirattr-validate-owners
---
Commits in this patchset:
 * xfs: add an explicit owner field to xfs_da_args
 * xfs: use the xfs_da_args owner field to set new dir/attr block owner
 * xfs: validate attr leaf buffer owners
 * xfs: validate attr remote value buffer owners
 * xfs: validate dabtree node buffer owners
 * xfs: validate directory leaf buffer owners
 * xfs: validate explicit directory data buffer owners
 * xfs: validate explicit directory block buffer owners
 * xfs: validate explicit directory free block owners
---
 fs/xfs/libxfs/xfs_attr.c        |   10 +-
 fs/xfs/libxfs/xfs_attr_leaf.c   |   59 +++++++++++---
 fs/xfs/libxfs/xfs_attr_leaf.h   |    4 +
 fs/xfs/libxfs/xfs_attr_remote.c |   13 +--
 fs/xfs/libxfs/xfs_bmap.c        |    1 
 fs/xfs/libxfs/xfs_da_btree.c    |  168 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_da_btree.h    |    3 +
 fs/xfs/libxfs/xfs_dir2.c        |    5 +
 fs/xfs/libxfs/xfs_dir2.h        |    4 +
 fs/xfs/libxfs/xfs_dir2_block.c  |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_data.c   |   17 ++--
 fs/xfs/libxfs/xfs_dir2_leaf.c   |   99 ++++++++++++++++++-----
 fs/xfs/libxfs/xfs_dir2_node.c   |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_priv.h   |   11 +--
 fs/xfs/libxfs/xfs_exchmaps.c    |    7 +-
 fs/xfs/scrub/attr.c             |    1 
 fs/xfs/scrub/dabtree.c          |    8 ++
 fs/xfs/scrub/dir.c              |   23 +++--
 fs/xfs/scrub/readdir.c          |    6 +
 fs/xfs/xfs_acl.c                |    2 
 fs/xfs/xfs_attr_item.c          |    1 
 fs/xfs/xfs_attr_list.c          |   35 +++++++-
 fs/xfs/xfs_dir2_readdir.c       |    6 +
 fs/xfs/xfs_ioctl.c              |    2 
 fs/xfs/xfs_iops.c               |    1 
 fs/xfs/xfs_trace.h              |    7 +-
 fs/xfs/xfs_xattr.c              |    2 
 27 files changed, 464 insertions(+), 119 deletions(-)


^ permalink raw reply	[flat|nested] 24+ messages in thread
* [PATCHSET v29.0 21/40] xfsprogs: set and validate dir/attr block owners
@ 2023-12-31 19:45 Darrick J. Wong
  2023-12-31 22:33 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
  0 siblings, 1 reply; 24+ messages in thread
From: Darrick J. Wong @ 2023-12-31 19:45 UTC (permalink / raw)
  To: djwong, cem; +Cc: linux-xfs

Hi all,

There are a couple of significatn changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number, we instead change libxfs to allow callers of the dir
and xattr code the ability to set an explicit owner number to be written
into the header fields of any new blocks that are created.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dirattr-validate-owners

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=dirattr-validate-owners
---
 db/attrset.c             |    2 +
 db/namei.c               |    6 +-
 libxfs/libxfs_api_defs.h |    3 +
 libxfs/xfs_attr.c        |   10 +--
 libxfs/xfs_attr_leaf.c   |   59 +++++++++++++---
 libxfs/xfs_attr_leaf.h   |    4 +
 libxfs/xfs_attr_remote.c |   13 ++--
 libxfs/xfs_bmap.c        |    1 
 libxfs/xfs_da_btree.c    |  168 ++++++++++++++++++++++++++++++++++++++++++++++
 libxfs/xfs_da_btree.h    |    3 +
 libxfs/xfs_dir2.c        |    5 +
 libxfs/xfs_dir2.h        |    4 +
 libxfs/xfs_dir2_block.c  |   44 +++++++-----
 libxfs/xfs_dir2_data.c   |   17 +++--
 libxfs/xfs_dir2_leaf.c   |   99 +++++++++++++++++++++------
 libxfs/xfs_dir2_node.c   |   44 +++++++-----
 libxfs/xfs_dir2_priv.h   |   11 ++-
 libxfs/xfs_swapext.c     |    7 +-
 repair/phase6.c          |    3 +
 19 files changed, 402 insertions(+), 101 deletions(-)


^ permalink raw reply	[flat|nested] 24+ messages in thread
* [PATCHSET v29.0 19/28] xfs: set and validate dir/attr block owners
@ 2023-12-31 19:30 Darrick J. Wong
  2023-12-31 20:33 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
  0 siblings, 1 reply; 24+ messages in thread
From: Darrick J. Wong @ 2023-12-31 19:30 UTC (permalink / raw)
  To: djwong; +Cc: linux-xfs

Hi all,

There are a couple of significant changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number before the swap, we instead change libxfs to allow
callers of the dir and xattr code the ability to set an explicit owner
number to be written into the header fields of any new blocks that are
created.  For regular operation this will be the directory inode number.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dirattr-validate-owners

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=dirattr-validate-owners
---
 fs/xfs/libxfs/xfs_attr.c        |   10 +-
 fs/xfs/libxfs/xfs_attr_leaf.c   |   59 +++++++++++---
 fs/xfs/libxfs/xfs_attr_leaf.h   |    4 +
 fs/xfs/libxfs/xfs_attr_remote.c |   13 +--
 fs/xfs/libxfs/xfs_bmap.c        |    1 
 fs/xfs/libxfs/xfs_da_btree.c    |  168 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_da_btree.h    |    3 +
 fs/xfs/libxfs/xfs_dir2.c        |    5 +
 fs/xfs/libxfs/xfs_dir2.h        |    4 +
 fs/xfs/libxfs/xfs_dir2_block.c  |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_data.c   |   17 ++--
 fs/xfs/libxfs/xfs_dir2_leaf.c   |   99 ++++++++++++++++++-----
 fs/xfs/libxfs/xfs_dir2_node.c   |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_priv.h   |   11 +--
 fs/xfs/libxfs/xfs_swapext.c     |    7 +-
 fs/xfs/scrub/attr.c             |    1 
 fs/xfs/scrub/dabtree.c          |    8 ++
 fs/xfs/scrub/dir.c              |   23 +++--
 fs/xfs/scrub/readdir.c          |    6 +
 fs/xfs/xfs_acl.c                |    2 
 fs/xfs/xfs_attr_item.c          |    1 
 fs/xfs/xfs_attr_list.c          |   35 +++++++-
 fs/xfs/xfs_dir2_readdir.c       |    6 +
 fs/xfs/xfs_ioctl.c              |    2 
 fs/xfs/xfs_iops.c               |    1 
 fs/xfs/xfs_trace.h              |    7 +-
 fs/xfs/xfs_xattr.c              |    2 
 27 files changed, 464 insertions(+), 119 deletions(-)


^ permalink raw reply	[flat|nested] 24+ messages in thread
* [PATCHSET v25.0 0/9] xfs: set and validate dir/attr block owners
@ 2023-05-26  0:35 Darrick J. Wong
  2023-05-26  1:31 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
  0 siblings, 1 reply; 24+ messages in thread
From: Darrick J. Wong @ 2023-05-26  0:35 UTC (permalink / raw)
  To: djwong; +Cc: linux-xfs

Hi all,

There are a couple of significant changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number before the swap, we instead change libxfs to allow
callers of the dir and xattr code the ability to set an explicit owner
number to be written into the header fields of any new blocks that are
created.  For regular operation this will be the directory inode number.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dirattr-validate-owners

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=dirattr-validate-owners
---
 fs/xfs/libxfs/xfs_attr.c        |   10 +-
 fs/xfs/libxfs/xfs_attr_leaf.c   |   59 +++++++++++---
 fs/xfs/libxfs/xfs_attr_leaf.h   |    4 +
 fs/xfs/libxfs/xfs_attr_remote.c |   13 +--
 fs/xfs/libxfs/xfs_bmap.c        |    1 
 fs/xfs/libxfs/xfs_da_btree.c    |  168 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_da_btree.h    |    3 +
 fs/xfs/libxfs/xfs_dir2.c        |    5 +
 fs/xfs/libxfs/xfs_dir2.h        |    4 +
 fs/xfs/libxfs/xfs_dir2_block.c  |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_data.c   |   17 ++--
 fs/xfs/libxfs/xfs_dir2_leaf.c   |   99 ++++++++++++++++++-----
 fs/xfs/libxfs/xfs_dir2_node.c   |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_priv.h   |   11 +--
 fs/xfs/libxfs/xfs_swapext.c     |    7 +-
 fs/xfs/scrub/attr.c             |    1 
 fs/xfs/scrub/dabtree.c          |    8 ++
 fs/xfs/scrub/dir.c              |   23 +++--
 fs/xfs/scrub/readdir.c          |    6 +
 fs/xfs/xfs_acl.c                |    2 
 fs/xfs/xfs_attr_item.c          |    1 
 fs/xfs/xfs_attr_list.c          |   35 +++++++-
 fs/xfs/xfs_dir2_readdir.c       |    6 +
 fs/xfs/xfs_ioctl.c              |    2 
 fs/xfs/xfs_iops.c               |    1 
 fs/xfs/xfs_trace.h              |    7 +-
 fs/xfs/xfs_xattr.c              |    2 
 27 files changed, 464 insertions(+), 119 deletions(-)


^ permalink raw reply	[flat|nested] 24+ messages in thread
* [PATCHSET v24.0 0/9] xfs: set and validate dir/attr block owners
@ 2022-12-30 22:14 Darrick J. Wong
  2022-12-30 22:14 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
  0 siblings, 1 reply; 24+ messages in thread
From: Darrick J. Wong @ 2022-12-30 22:14 UTC (permalink / raw)
  To: djwong; +Cc: linux-xfs

Hi all,

There are a couple of significant changes that need to be made to the
directory and xattr code before we can support online repairs of those
data structures.

The first change is because online repair is designed to use libxfs to
create a replacement dir/xattr structure in a temporary file, and use
atomic extent swapping to commit the corrected structure.  To avoid the
performance hit of walking every block of the new structure to rewrite
the owner number before the swap, we instead change libxfs to allow
callers of the dir and xattr code the ability to set an explicit owner
number to be written into the header fields of any new blocks that are
created.  For regular operation this will be the directory inode number.

The second change is to update the dir/xattr code to actually *check*
the owner number in each block that is read off the disk, since we don't
currently do that.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=dirattr-validate-owners

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=dirattr-validate-owners
---
 fs/xfs/libxfs/xfs_attr.c        |   10 +-
 fs/xfs/libxfs/xfs_attr_leaf.c   |   59 +++++++++++---
 fs/xfs/libxfs/xfs_attr_leaf.h   |    4 +
 fs/xfs/libxfs/xfs_attr_remote.c |   13 +--
 fs/xfs/libxfs/xfs_bmap.c        |    1 
 fs/xfs/libxfs/xfs_da_btree.c    |  168 +++++++++++++++++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_da_btree.h    |    3 +
 fs/xfs/libxfs/xfs_dir2.c        |    5 +
 fs/xfs/libxfs/xfs_dir2.h        |    4 +
 fs/xfs/libxfs/xfs_dir2_block.c  |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_data.c   |   17 ++--
 fs/xfs/libxfs/xfs_dir2_leaf.c   |   99 ++++++++++++++++++-----
 fs/xfs/libxfs/xfs_dir2_node.c   |   44 ++++++----
 fs/xfs/libxfs/xfs_dir2_priv.h   |   11 +--
 fs/xfs/libxfs/xfs_swapext.c     |    7 +-
 fs/xfs/scrub/attr.c             |    1 
 fs/xfs/scrub/dabtree.c          |    8 ++
 fs/xfs/scrub/dir.c              |   23 +++--
 fs/xfs/scrub/readdir.c          |    6 +
 fs/xfs/xfs_acl.c                |    2 
 fs/xfs/xfs_attr_item.c          |    1 
 fs/xfs/xfs_attr_list.c          |   35 +++++++-
 fs/xfs/xfs_dir2_readdir.c       |    6 +
 fs/xfs/xfs_ioctl.c              |    2 
 fs/xfs/xfs_iops.c               |    1 
 fs/xfs/xfs_trace.h              |    7 +-
 fs/xfs/xfs_xattr.c              |    2 
 27 files changed, 464 insertions(+), 119 deletions(-)


^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2024-02-28 17:41 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-02-27  2:18 [PATCHSET v29.4 06/13] xfs: set and validate dir/attr block owners Darrick J. Wong
2024-02-27  2:26 ` [PATCH 1/9] xfs: add an explicit owner field to xfs_da_args Darrick J. Wong
2024-02-28 15:54   ` Christoph Hellwig
2024-02-27  2:26 ` [PATCH 2/9] xfs: use the xfs_da_args owner field to set new dir/attr block owner Darrick J. Wong
2024-02-28 15:54   ` Christoph Hellwig
2024-02-27  2:27 ` [PATCH 3/9] xfs: validate attr leaf buffer owners Darrick J. Wong
2024-02-28 15:57   ` Christoph Hellwig
2024-02-28 17:41     ` Darrick J. Wong
2024-02-27  2:27 ` [PATCH 4/9] xfs: validate attr remote value " Darrick J. Wong
2024-02-28 15:57   ` Christoph Hellwig
2024-02-27  2:27 ` [PATCH 5/9] xfs: validate dabtree node " Darrick J. Wong
2024-02-28 15:58   ` Christoph Hellwig
2024-02-27  2:27 ` [PATCH 6/9] xfs: validate directory leaf " Darrick J. Wong
2024-02-28 15:58   ` Christoph Hellwig
2024-02-27  2:28 ` [PATCH 7/9] xfs: validate explicit directory data " Darrick J. Wong
2024-02-28 15:58   ` Christoph Hellwig
2024-02-27  2:28 ` [PATCH 8/9] xfs: validate explicit directory block " Darrick J. Wong
2024-02-28 15:59   ` Christoph Hellwig
2024-02-27  2:28 ` [PATCH 9/9] xfs: validate explicit directory free block owners Darrick J. Wong
2024-02-28 15:59   ` Christoph Hellwig
  -- strict thread matches above, loose matches on Subject: below --
2023-12-31 19:45 [PATCHSET v29.0 21/40] xfsprogs: set and validate dir/attr " Darrick J. Wong
2023-12-31 22:33 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
2023-12-31 19:30 [PATCHSET v29.0 19/28] xfs: set and validate dir/attr block owners Darrick J. Wong
2023-12-31 20:33 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
2023-05-26  0:35 [PATCHSET v25.0 0/9] xfs: set and validate dir/attr block owners Darrick J. Wong
2023-05-26  1:31 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong
2022-12-30 22:14 [PATCHSET v24.0 0/9] xfs: set and validate dir/attr block owners Darrick J. Wong
2022-12-30 22:14 ` [PATCH 5/9] xfs: validate dabtree node buffer owners Darrick J. Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).