public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/30] xfsprogs: Initial CRC support
@ 2013-05-17 11:12 Dave Chinner
  2013-05-17 11:12 ` [PATCH 01/30] mkfs: fix realtime device initialisation Dave Chinner
                   ` (32 more replies)
  0 siblings, 33 replies; 58+ messages in thread
From: Dave Chinner @ 2013-05-17 11:12 UTC (permalink / raw)
  To: xfs

Hi Folks,

This is the first real "works ok" CRC patchset for xfsprogs. It
provides full support for mkfs.xfs and xfs_repair, and partial
read-only support for xfs_db.

For mkfs.xfs, it does everything properly, and filesystems that are
freshly made also run cleanly through xfs_repair and mount and run
just fine.

For xfs_repair, it reads and writes all metadata with CRC checks,
calculations and validation just like the kernel code does, but it
currently silently ignores the validation done in the IO layer.
Enabling that is future work - it involves adding buffer error checking to
every libxfs_readbuf() call that is made, and we do none of that
right now. It does, however, fully validate all the non-CRC format
metadata just as it does for non-CRC filesystems, and so the
coverage it has is the same for both CRC and non-CRC filesystems.

For xfs_db, there is read-only support for looking at the filesystem
as the xfs_db IO stack does not support CRCs at all. We need to
convert xfs_db to use the libxfs infrastructure to enable that.
Apart from that, xfs_db has partial support for the extended
metadata fields - the directory/attribute blocks don't have extended
support yet, but everything else does.

xfs_check is made special. It currently detects a version 5
superblock, and immediately exits with success. Hence it always says
CRC enabled filesystems are OK. This is a temporary change that
enables running xfstests without full support in xfs_db for all the
new metadata structures (like headers in remote symlink and
attribute blocks). Depending on if we want to keep xfs-check useful
for xfstests, we can revisit this bypass hack once xfs_db has been
converted to use the libxfs IO engine.

Overall, xfstests is now running enough to start to find bugs in the
kernel CRC code - I'm mainly hitting remote attribute block bugs
right now (generic/117!) but there's certainly less problems being
reported than I expected.

Oh, and I've tested it with external log devices and real time
devices, too.

Comments, thoughts, flames, and testing all welcome!

Cheers,

Dave.

$ git diff --minimal --stat --summary 8e78288..
 copy/xfs_copy.c            |    4 +-
 db/agf.c                   |    3 +
 db/agfl.c                  |   16 +
 db/agfl.h                  |    2 +
 db/agi.c                   |    3 +
 db/attr.c                  |   20 +-
 db/btblock.c               |  145 +++++
 db/btblock.h               |   10 +
 db/check.c                 |   34 +-
 db/dir2.c                  |   10 +-
 db/dir2sf.c                |    9 +-
 db/field.c                 |   16 +
 db/field.h                 |    8 +
 db/freesp.c                |   13 +-
 db/init.c                  |   19 +
 db/inode.c                 |   27 +
 db/inode.h                 |    3 +
 db/metadump.c              |    4 +-
 db/sb.c                    |   54 +-
 db/type.c                  |   34 +-
 db/type.h                  |    3 +-
 include/Makefile           |    4 +-
 include/libxfs.h           |  118 ++--
 include/libxlog.h          |    2 +-
 include/xfs_ag.h           |   54 +-
 include/xfs_alloc_btree.h  |   13 +-
 include/xfs_attr_leaf.h    |  124 +++-
 include/xfs_attr_remote.h  |   52 ++
 include/xfs_bmap_btree.h   |   20 +-
 include/xfs_btree.h        |   63 +-
 include/xfs_buf_item.h     |   64 +-
 include/xfs_da_btree.h     |  130 +++-
 include/xfs_dinode.h       |   37 +-
 include/xfs_dir2.h         |    7 +
 include/xfs_dir2_format.h  |  293 ++++++++-
 include/xfs_ialloc_btree.h |   10 +-
 include/xfs_inode.h        |   26 +
 include/xfs_mount.h        |    1 +
 include/xfs_quota.h        |   11 +-
 include/xfs_sb.h           |  166 ++++-
 include/xfs_symlink.h      |   45 ++
 libxfs/Makefile            |    4 +-
 libxfs/init.c              |   84 ++-
 libxfs/logitem.c           |    4 +-
 libxfs/rdwr.c              |  236 ++++---
 libxfs/trans.c             |   17 +-
 libxfs/util.c              |   29 +-
 libxfs/xfs.h               |   37 +-
 libxfs/xfs_alloc.c         |  202 ++++--
 libxfs/xfs_alloc_btree.c   |   99 ++-
 libxfs/xfs_attr.c          |  386 ++----------
 libxfs/xfs_attr_leaf.c     | 1553 ++++++++++++++++++++++++++--------------------
 libxfs/xfs_attr_remote.c   |  527 ++++++++++++++++
 libxfs/xfs_bmap.c          |   49 +-
 libxfs/xfs_bmap_btree.c    |  107 +++-
 libxfs/xfs_btree.c         |  255 ++++++--
 libxfs/xfs_da_btree.c      | 1499 +++++++++++++++++++++++++++-----------------
 libxfs/xfs_dir2_block.c    |  171 +++--
 libxfs/xfs_dir2_data.c     |  266 +++++---
 libxfs/xfs_dir2_leaf.c     |  880 ++++++++++++++++----------
 libxfs/xfs_dir2_node.c     | 1007 +++++++++++++++++++-----------
 libxfs/xfs_dir2_priv.h     |   46 +-
 libxfs/xfs_dir2_sf.c       |    8 +-
 libxfs/xfs_ialloc.c        |   78 ++-
 libxfs/xfs_ialloc_btree.c  |   80 ++-
 libxfs/xfs_inode.c         |  124 +++-
 libxfs/xfs_mount.c         |  137 +++-
 libxfs/xfs_symlink.c       |  154 +++++
 logprint/log_misc.c        |   51 +-
 logprint/log_print_all.c   |    3 +-
 logprint/log_print_trans.c |   18 +
 logprint/logprint.c        |    4 +-
 mdrestore/Makefile         |    2 +-
 mkfs/maxtrres.c            |    4 +-
 mkfs/proto.c               |    6 +-
 mkfs/xfs_mkfs.c            |  170 +++--
 mkfs/xfs_mkfs.h            |   12 +-
 repair/agheader.c          |   36 +-
 repair/attr_repair.c       |  173 +++---
 repair/dino_chunks.c       |   13 +-
 repair/dinode.c            |  200 +++---
 repair/dir2.c              |  167 ++---
 repair/dir2.h              |    6 +-
 repair/phase2.c            |    8 +-
 repair/phase3.c            |    2 +-
 repair/phase5.c            |  156 +++--
 repair/phase6.c            |  172 +++--
 repair/prefetch.c          |    9 +-
 repair/rt.c                |    4 +-
 repair/scan.c              |  169 ++---
 repair/scan.h              |   12 +-
 repair/versions.c          |    2 +-
 repair/xfs_repair.c        |    8 +-
 93 files changed, 7559 insertions(+), 3564 deletions(-)
 create mode 100644 include/xfs_attr_remote.h
 create mode 100644 include/xfs_symlink.h
 create mode 100644 libxfs/xfs_attr_remote.c
 create mode 100644 libxfs/xfs_symlink.c

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 58+ messages in thread

end of thread, other threads:[~2013-07-22 20:46 UTC | newest]

Thread overview: 58+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-05-17 11:12 [PATCH 00/30] xfsprogs: Initial CRC support Dave Chinner
2013-05-17 11:12 ` [PATCH 01/30] mkfs: fix realtime device initialisation Dave Chinner
2013-07-22 20:46   ` Ben Myers
2013-05-17 11:12 ` [PATCH 02/30] logprint: fix wrapped log dump issue Dave Chinner
2013-05-17 11:12 ` [PATCH 03/30] libxfs: add crc format changes to generic btrees Dave Chinner
2013-05-17 11:12 ` [PATCH 04/30] xfsprogs: add crc format chagnes to ag headers Dave Chinner
2013-05-17 11:13 ` [PATCH 05/30] xfsprogs: Support new AGFL format Dave Chinner
2013-05-17 11:13 ` [PATCH 06/30] libxfs: change quota buffer formats Dave Chinner
2013-05-17 11:13 ` [PATCH 07/30] libxfs: add version 3 inode support Dave Chinner
2013-05-17 11:13 ` [PATCH 08/30] libxfs: add support for crc headers on remote symlinks Dave Chinner
2013-05-17 11:13 ` [PATCH 09/30] xfs: add CRC checks to block format directory blocks Dave Chinner
2013-05-17 11:13 ` [PATCH 10/30] xfs: add CRC checking to dir2 free blocks Dave Chinner
2013-05-17 11:13 ` [PATCH 11/30] xfs: add CRC checking to dir2 data blocks Dave Chinner
2013-05-17 11:13 ` [PATCH 12/30] xfs: add CRC checking to dir2 leaf blocks Dave Chinner
2013-05-17 11:13 ` [PATCH 13/30] xfs: shortform directory offsets change for dir3 format Dave Chinner
2013-05-17 11:13 ` [PATCH 14/30] xfs: add CRCs to dir2/da node blocks Dave Chinner
2013-05-17 11:13 ` [PATCH 15/30] xfs: add CRCs to attr leaf blocks Dave Chinner
2013-05-17 11:13 ` [PATCH 16/30] xfs: split remote attribute code out Dave Chinner
2013-05-17 11:13 ` [PATCH 17/30] xfs: add CRC protection to remote attributes Dave Chinner
2013-05-17 11:13 ` [PATCH 18/30] xfs: add buffer types to directory and attribute buffers Dave Chinner
2013-05-17 11:13 ` [PATCH 19/30] xfs: buffer type overruns blf_flags field Dave Chinner
2013-05-17 11:13 ` [PATCH 20/30] xfs: add CRC checks to the superblock Dave Chinner
2013-05-17 11:13 ` [PATCH 21/30] xfs: implement extended feature masks Dave Chinner
2013-05-17 11:13 ` [PATCH 22/30] xfsprogs: Add verifiers to libxfs buffer interfaces Dave Chinner
2013-05-17 11:13 ` [PATCH 23/30] patch xfsprogs-mkfs-crc-support-2 Dave Chinner
2013-05-17 11:13 ` [PATCH 24/30] xfsprogs: add crc format support to repair Dave Chinner
2013-05-17 11:13 ` [PATCH 25/30] xfs_repair: update for dir/attr crc format changes Dave Chinner
2013-05-17 11:13 ` [PATCH 26/30] xfsprogs: disable xfs_check for CRC enabled filesystems Dave Chinner
2013-05-17 11:13 ` [PATCH 27/30] xfs_db: disable modification for CRC enabled filessytems Dave Chinner
2013-05-17 11:13 ` [PATCH 28/30] libxfs: determine inode size from version number, not struct xfs_dinode Dave Chinner
2013-05-17 11:13 ` [PATCH 29/30] xfsdb: support version 5 superblock in versionnum command Dave Chinner
2013-05-17 11:13 ` [PATCH 30/30] xfsprogs: add crc format support to db Dave Chinner
2013-05-17 20:54 ` [PATCH 00/30] xfsprogs: Initial CRC support Michael L. Semon
2013-05-18  3:25   ` Dave Chinner
2013-05-18  5:07     ` Jeff Liu
2013-05-18  5:39       ` Dave Chinner
2013-05-18  6:27       ` Michael L. Semon
2013-05-18  8:46         ` Jeff Liu
2013-05-18  5:40     ` Michael L. Semon
2013-05-18  6:27       ` Dave Chinner
2013-05-18  7:42         ` Michael L. Semon
2013-05-18 18:13 ` Michael L. Semon
2013-05-20  6:52 ` [PATCH 0/6] xfsprogs: more CRC support patches Dave Chinner
2013-05-20  6:52   ` [PATCH 1/6] xfs_repair: always use incore header for directory block checks Dave Chinner
2013-05-20  6:52   ` [PATCH 2/6] xfs_db: convert directory parsing to use libxfs structure Dave Chinner
2013-05-20  6:53   ` [PATCH 3/6] xfs_db: factor some common dir2 field parsing code Dave Chinner
2013-05-20  6:53   ` [PATCH 4/6] xfs_db: update field printing for dir crc format changes Dave Chinner
2013-05-20  6:53   ` [PATCH 5/6] xfs_repair: convert directory parsing to use libxfs structure Dave Chinner
2013-05-20  6:53   ` [PATCH 6/6] xfs_repair: make directory freespace table CRC format aware Dave Chinner
2013-05-20 16:11   ` [PATCH 0/6] xfsprogs: more CRC support patches Michael L. Semon
2013-05-23 12:36   ` [PATCH 0/2] xfsprogs: yet " Dave Chinner
2013-05-23 12:36     ` [PATCH 1/2] xfs_db: add CRC information to dquot output Dave Chinner
2013-05-23 12:36     ` [PATCH 2/2] xfs_db: add CRC support for attribute fork structures Dave Chinner
2013-05-27  7:14     ` [PATCH 0/4] xfsprogs: more CRC patches Dave Chinner
2013-05-27  7:14       ` [PATCH 1/4] mkfs.xfs: validate options for CRCs up front Dave Chinner
2013-05-27  7:14       ` [PATCH 2/4] xfsprogs: support CRC enabled filesystem detection Dave Chinner
2013-05-27  7:14       ` [PATCH 3/4] xfs_mdrestore: recalculate sb CRC before writing Dave Chinner
2013-05-27  7:14       ` [PATCH 4/4] xfs_metadump: requires some object CRC recalculation Dave Chinner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox