public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: xfs@oss.sgi.com
Subject: [RFC PATCH 00/12] xfs_db: write support for v5 filesystems
Date: Thu,  5 Sep 2013 22:13:21 +1000	[thread overview]
Message-ID: <1378383213-16155-1-git-send-email-david@fromorbit.com> (raw)

Hi folks,

This RFC series brings write support to xfs_db for CRC enabled
filesytsems. The series converts xfs_db to use libxfs for it's IO,
adds awareness of verifiers and CRC validation to the IO stack and
enables write support for v5 filesystem structures.

The main infrastructure change is switching to libxfs-based IO
instead of buffered block device IO. This means that xfs_db now
switches to direct IO cached in it's own address space, meaning that
when it runs it always starts with a cold cache and so we don't have
problem with it reading stale cached data out of the kernel.

With it using libxfs, we can also attach verifiers to the IO to
validate structures and CRCs before we act on them. The fact that
CRCs were passed or not is available to usres of the data that is
read, and so programs like xfs_check can be made to fail if CRC
validation fails on read of metadata.

Finally, write support is enabled because now that all data buffers
have verifiers they can recalculate CRCs on write. There's some
special support for inodes and dquots, but they will still have CRCs
recalculated before write IOs. Hence xfs_db won't corrupt v5
metadata now when you change a field value manually.

There are still some issues with the libxfs support - there are some
places where xfs_db re-reads the same block address with different
buffer lengths, resulting in libxfs throwing a cache lookup badness
warning. e.g:

$ sudo xfs_db -c "push sb" -c "p crc" /dev/vdc
7fe16ab3f740: Badness in key lookup (length)
bp=(bno 0x0, len 0 bytes) key=(bno 0x0, len 512 bytes)
Version 5 superblock detected. xfsprogs has EXPERIMENTAL support enabled!
Use of these features is at your own risk!
crc = 0xfc7b42ec (correct)
$

This shows a buffer at block 0 was created with zero length for some
reason, but when we push the superblock onto the stack, it looks up
a buffer of 512 bytes, not 0, and it throws an error. The
experimental warning is being thrown by the superblock buffer
verifier on detecting the v5 superblock version. And the CRC of the
buffer is displayed as being correct as the verifier didn't throw a
corruption error when it ran.

This output currently causes all sorts of problems for xfstests -
the root inode cluster is another that causes issues, and there are
also cases occurring where cache purging on unmount is throwing
warnings. Hence it's not yet ready for merge, but otherwise it seems
to work reliably enough to use. Early review and testing would be
helpful at this point.

These changes also mean that we should be able to turn metadump back
on, as it needs CRC write support for the obfuscation code to
work on v5 filesystems. Patches will be coming shortly to do that.

xfs_check is still an open question. I might just re-enable it and
add CRC validation checks and see what falls out from that, without
adding any extra validation for the v5 filesystem format bits...

Comments, thoughts, flames?

Cheers,

Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

             reply	other threads:[~2013-09-05 12:13 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-09-05 12:13 Dave Chinner [this message]
2013-09-05 12:13 ` [RFC PATCH 01/12] db: separate out straight buffer IO from map based IO Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 02/12] db: rewrite bbmap to use xfs_buf_map Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 03/12] db: rewrite IO engine to use libxfs Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 04/12] xfs: split dquot buffer operations out Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 05/12] xfs: move buf_ops externs to a xfs_format.h Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 06/12] db: introduce verifier support into set_cur Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 07/12] db: indicate if the CRC on a buffer is correct or not Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 08/12] db: verify and calculate inode CRCs Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 09/12] db: verify and calculate dquot CRCs Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 10/12] db: add a special directory buffer verifier Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 11/12] db: add a special attribute " Dave Chinner
2013-09-05 12:13 ` [RFC PATCH 12/12] db: re-enable write support for v5 filesystems Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1378383213-16155-1-git-send-email-david@fromorbit.com \
    --to=david@fromorbit.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox