[RFCv3 00/51] xfsprogs: add reverse-mapping, reflink, and dedupe support

Darrick J. Wong
2015-10-07T05:05:19Z

Hi all,

This is the third revision of an RFC for adding to xfsprogs support
for tracking reverse-mappings of physical blocks to file and metadata;
and support for mapping multiple file logical blocks to the same
physical block, more commonly known as reflinking. Given the
significant amount of re-engineering required to make the initial rmap
implementation compatible with reflink, I decided to publish both
features as an integrated patchset off of upstream. This means that
rmap and reflink are now compatible with each other.

The patch set is based on the current (4.2.0+) for-next branch. This
code should be relatively bug-free, and the bulk of the patches are to
teach xfs_repair how to record all mappings and to use that data both
to regenerate the reference count data (refcntbt) and the reverse
mapping index (rmapbt). There are way too many patches to discuss
them individually, but roughly speaking they're grouped by functional
area:

0. Cleanups
1. Implement reflink and dedupe in xfs_io
2. Spot-check and fuzz v5 filesystems in xfs_db
(otherwise the test/scratch fs checks in xfstests get unhappy)
3. rmapbt support
4. rmapbt rebuilding in xfs_repair
5. refcntbt support
6. refcntbt rebuilding in xfs_repair

Issues:

* I'm not 100% sure xfs_repair correctly handles rebuilding all the
XFS_RMAP_OWN_AG rmap entries (which are the bnobt, cntbt, rmapbt, and
the AGFL).

* General shakiness of the code that spots errors in the rmapbt and
refcntbt. Given that we're either readonly or rebuilding them anyway,
I wonder if it matters...

* Under certain circumstances, mkfs underestimates the minimum log
size and the kernel refuses to mount. The last patch in the set
hacks around this in an ugly way.

If you're going to start using this mess, you probably ought to just
pull from my github trees for kernel[1], xfsprogs[2], and xfstests[3].

This is an extraordinary way to eat your data. Enjoy!

Comments and questions are, as always, welcome.

--D

[1] https://github.com/djwong/linux-xfs-dev/commits/master
[2] https://github.com/djwong/xfsprogs/commits/for-next
[3] https://github.com/djwong/xfstests/commits/master