public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/16] xfs: metadata CRCs, second batch
@ 2013-02-25  1:31 Dave Chinner
  2013-02-25  1:31 ` [PATCH 01/16] xfs: rearrange some code in xfs_bmap for better locality Dave Chinner
                   ` (15 more replies)
  0 siblings, 16 replies; 23+ messages in thread
From: Dave Chinner @ 2013-02-25  1:31 UTC (permalink / raw)
  To: xfs

Hi folks,

This is the second version of the kernel support for metadata CRCs.
This builds on the first batch posted here:

http://oss.sgi.com/archives/xfs/2013-01/msg00328.html

I won't write everything I wrote in that description again, so I'll
just focus on what is new in this series:

- based on xfs_bmap.c rearrangement patch
- fixed up a couple of bugs and comments noted by Ben
- fixed a significant number of bugs, including the dquot recovery
  problem and some symlink issues on 512 byte block size
  filesystems.
- added support for directories and attributes
- moved the superblock CRC addition and enablment patch to the end
  of the series.

Things that are still missing from this version:

- remote attribute CRCs
- directory/attribute buffer type tracking for log recovery
- DT_* type fields in the directory entries
- storage of attributes larger than 256 bytes in shortform attribute
  forks.

The fact that directory/attr buffer type tracking is missing means
this code simply won't work right now with CRCs enabled. My
mainfocus so far is getting all the structural changes to the code
in place without causing regressions. In general, the directory/attr
code changes are along the lines of:

- CRC enabled directories are named as "dir3", btree nodes "da3" and
  attribute trees are "attr3".

- functions that support both attr/attr3 and dir2/dir3 types are
  renamed to attr3/dir3/da3 so it is obvious that they can support
  both types. This may not be 100% consistent through the patches
  yet - I didn't settle name change until a bit into the directory
  conversion. The directory code needs cleaning up, and will be done
  as an additional patch.

- header sizes change, so access to data types after the header need
  to be abstracted. this is done by helper functions rather tan
  direct dereference of a structure type.

- header sizes change, so that access to certain header fields need
  ot be abstracted. This is done in most cases by introducing an
  "in-core header" type and to/from disk translation functions. This
  has removed a *lot* of endian swapping from the code.

- code has been modified to use abtract access functions, and in
  some cases pass in-core headers around when child functions also
  modify headers. In some cases, logging of header has been pulled
  up into the parent functions that own the incore header
  structure.

- much of the function declarations has been reformatted to be
  consistent with the rest of the XFS code. Every time I touched a
  function name I reformatted the parameters and the local variable
  declarations inside the functions.

While the changes to the directory/attr code look complex, but
really they are mostly mechanical changes. i.e. the same change done
over and over again. There are places that I know that I've missed
changing, and these will be corrected in followup patches as I start
testing with CRCs enabled.

In terms of testing, this series passes xfstests on several machines
with 512 byte and 4k block size filesystems using existing tools
(i.e. no CRC support) without any new regressions being introduced.
It survives repeated fsmark runs on 100TB filesystems, and so seems
fairly robust when used with existing tools and filesystems.

Diffstat for the series:

$ git diff --stat 2292716..
 fs/xfs/Makefile           |    5 +-
 fs/xfs/xfs_ag.h           |   56 +-
 fs/xfs/xfs_alloc.c        |  199 +-
 fs/xfs/xfs_alloc_btree.c  |  105 +-
 fs/xfs/xfs_alloc_btree.h  |   12 +-
 fs/xfs/xfs_attr.c         |  158 +-
 fs/xfs/xfs_attr_leaf.c    | 1769 +++++----
 fs/xfs/xfs_attr_leaf.h    |  121 +-
 fs/xfs/xfs_bmap.c         | 9214 +++++++++++++++++++++++------------------------
 fs/xfs/xfs_bmap_btree.c   |  110 +-
 fs/xfs/xfs_bmap_btree.h   |   19 +-
 fs/xfs/xfs_btree.c        |  256 +-
 fs/xfs/xfs_btree.h        |   64 +-
 fs/xfs/xfs_buf_item.h     |   36 +-
 fs/xfs/xfs_da_btree.c     | 1426 +++++---
 fs/xfs/xfs_da_btree.h     |  121 +-
 fs/xfs/xfs_dinode.h       |   43 +-
 fs/xfs/xfs_dir2_block.c   |  161 +-
 fs/xfs/xfs_dir2_data.c    |  257 +-
 fs/xfs/xfs_dir2_format.h  |  249 +-
 fs/xfs/xfs_dir2_leaf.c    |  845 +++--
 fs/xfs/xfs_dir2_node.c    |  972 +++--
 fs/xfs/xfs_dir2_priv.h    |   48 +-
 fs/xfs/xfs_dquot.c        |  112 +-
 fs/xfs/xfs_file.c         |    2 +-
 fs/xfs/xfs_fsops.c        |   34 +-
 fs/xfs/xfs_ialloc.c       |   80 +-
 fs/xfs/xfs_ialloc_btree.c |   87 +-
 fs/xfs/xfs_ialloc_btree.h |    9 +-
 fs/xfs/xfs_inode.c        |  212 +-
 fs/xfs/xfs_inode.h        |   31 +-
 fs/xfs/xfs_inode_item.c   |    2 +-
 fs/xfs/xfs_log_recover.c  |  118 +-
 fs/xfs/xfs_mount.c        |   97 +-
 fs/xfs/xfs_mount.h        |    1 +
 fs/xfs/xfs_qm.c           |   23 +-
 fs/xfs/xfs_qm.h           |    2 +
 fs/xfs/xfs_quota.h        |   11 +-
 fs/xfs/xfs_sb.h           |  100 +-
 fs/xfs/xfs_trace.c        |    2 +-
 fs/xfs/xfs_trans.h        |    2 +
 fs/xfs/xfs_trans_buf.c    |   34 +-
 fs/xfs/xfs_vnodeops.c     |  480 +--
 43 files changed, 9830 insertions(+), 7855 deletions(-)

So you can see most of the change is in xfs_bmap.c and the directory
code. Without the first xfs_bmap.c patch, it is about half that size
and the majority comes from the directory and attribute changes.

43 files changed, 5315 insertions(+), 3206 deletions(-)

Reviews, testing, etc. welcome!

Cheers,

Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2013-03-07 18:48 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-02-25  1:31 [PATCH 00/16] xfs: metadata CRCs, second batch Dave Chinner
2013-02-25  1:31 ` [PATCH 01/16] xfs: rearrange some code in xfs_bmap for better locality Dave Chinner
2013-02-25 14:44   ` Mark Tinguely
2013-03-07 18:48   ` Ben Myers
2013-02-25  1:31 ` [PATCH 02/16] xfs: take inode version into account in XFS_LITINO Dave Chinner
2013-02-25  1:31 ` [PATCH 03/16] xfs: add support for large btree blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 04/16] xfs: add CRC checks to the AGF Dave Chinner
2013-02-25  1:31 ` [PATCH 05/16] xfs: add CRC checks to the AGFL Dave Chinner
2013-03-01 23:56   ` Ben Myers
2013-02-25  1:31 ` [PATCH 06/16] xfs: add CRC checks to the AGI Dave Chinner
2013-02-25  1:31 ` [PATCH 07/16] xfs: add CRC checks for quota blocks Dave Chinner
2013-03-05 20:36   ` Ben Myers
2013-03-05 22:49     ` Dave Chinner
2013-02-25  1:31 ` [PATCH 08/16] xfs: add version 3 inode format with CRCs Dave Chinner
2013-02-25  1:31 ` [PATCH 09/16] xfs: add CRC checks to remote symlinks Dave Chinner
2013-02-27 23:35   ` Dave Chinner
2013-02-25  1:31 ` [PATCH 10/16] xfs: add CRC checks to block format directory blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 11/16] xfs: add CRC checking to dir2 free blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 12/16] xfs: add CRC checking to dir2 data blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 13/16] xfs: add CRC checking to dir2 leaf blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 14/16] xfs: add CRCs to dir2/da node blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 15/16] xfs: add CRCs to attr leaf blocks Dave Chinner
2013-02-25  1:31 ` [PATCH 16/16] xfs: add CRC checks to the superblock Dave Chinner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox