XFS reflink and development tip git trees updated

XFS reflink and development tip git trees updated

[Darrick J. Wong]

Hi everyone,

I've just rebased the reflink patches against kernel 4.8-rc5 and the xfsprogs
dev branch:

https://github.com/djwong/linux/tree/for-dave-for-4.9-1 [17f66c42de]
https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-1 [7b015920a]

These branches should contain /only/ the patches needed to get reflink working:
no fsmap ioctl, no online scrub, and no realtime rmapbt.  I'm publishing these
branches in the hopes of getting wider testing prior to the merge window.  I
haven't heard any complaints for several weeks now.

I created these two branches partly because Brian was complaining about it
being a PITA to carve out a working branch.  The patches haven't changed at all
since the giant mailing a couple of weeks ago.

-----------

For the latest development branch, check out the djwong-wtf branches on github:
https://github.com/djwong/linux/tree/djwong-wtf
https://github.com/djwong/xfsprogs/tree/djwong-wtf

Note that I'm in the middle of reworking the GETFSMAP ioctl to eliminate the
problems that Andreas pointed out last week, and reworking the getfsmap and
scrub code to eliminate buffer deadlocks when a btree loop exists, so the last
few patches are ... very scraggly.

--D

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

XFS reflink and getfsmap/scrub development git trees updated again

[Darrick J. Wong]

Hi everyone,

I've just rebased the reflink patches against kernel 4.8-rc5 and the xfsprogs
dev branch:

https://github.com/djwong/linux/tree/for-dave-for-4.9-2
https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-2

These branches should contain /only/ the patches needed to get reflink working:
no fsmap ioctl, no online scrub, and no realtime rmapbt.  I'm publishing these
branches in the hopes of getting wider testing prior to the merge window.  I
haven't heard any complaints for several weeks now.  The only changes since
4.9-1 are to take care of all the comments that Christoph has sent up to
this point.

-----------

For the latest development branch, check out the djwong-wtf branches on github:
https://github.com/djwong/linux/tree/djwong-wtf
https://github.com/djwong/xfsprogs/tree/djwong-wtf

This contains all the latest reflink patches, the reworked GETFSMAP ioctl,
and the latest online scrubbing code.  The scraggly deadlock avoidance fixes
I mentioned in the last posting are now integrated properly into the patch
series.  Also, btree left/right sibling pointer checks have been implemented.

(Adding Ted and the ext4 list because Ted expressed interest in GETFSMAP.)

--D

Re: XFS reflink and development tip git trees updated

[Christoph Hellwig]

Can you rebase (and repost the series) on top of the preparations
in for-next that Dave just pushed out?

That should make reviewing a whole lot easier.

Re: XFS reflink and development tip git trees updated

[Darrick J. Wong]

On Mon, Sep 19, 2016 at 09:11:00AM -0700, Christoph Hellwig wrote:
> Can you rebase (and repost the series) on top of the preparations
> in for-next that Dave just pushed out?
> 
> That should make reviewing a whole lot easier.

Porting is underway, though the delalloc refactoring banged things up a bit. :)

--D

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS reflink and development tip git trees updated

[Darrick J. Wong]

On Mon, Sep 19, 2016 at 09:11:00AM -0700, Christoph Hellwig wrote:
> Can you rebase (and repost the series) on top of the preparations
> in for-next that Dave just pushed out?
> 
> That should make reviewing a whole lot easier.

I'm on vacation all week (in the desert with tcp-over-cactus) but I
don't want the reflink review to stall, so here's a branch atop the
for-next branch that Dave announced last night:

https://github.com/djwong/linux/tree/for-dave-for-4.9-3

The scary parts are the rework that I had to do to integrate with hch's
delalloc rewrite.  I think all the changes to xfs_file_iomap_begin_delay
are right (the regression tests all pass, at least) but it could use
another set of eyes.

No changes to the userland tools, they're still at:

https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-2

--D

XFS reflink and development tip git trees updated

[Darrick J. Wong]

Ok, here's this week's for-4.9 branch:

https://github.com/djwong/linux/tree/for-dave-for-4.9-5
https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-5

The kernel tree includes Christoph's fixes for copy-on-write,
refactoring of a refcount btree function to use _btree_query_range (and
inclusion of the required diff_two_keys/init_high_key helpers), and
removal of the (totally broken) always_cow debug switch.  It also
contains a new error injection point to make reflink fail with ENOSPC.

The xfsprogs tree incorporates the libxfs changes in the kernel tree
but is largely the same as last week's for-4.9-2 branch.  The only
change in userspace programs is that xfs_io can trigger the
above-mentioned error injection point.

---

The WTF trees have been rebased to 4.8-rc8 and xfsprogs for-next:
https://github.com/djwong/linux/tree/djwong-wtf
https://github.com/djwong/xfsprogs/tree/djwong-wtf

These two trees contain all the changes noted above as well as in-kernel
scrubbing of directory and extended attribute data.  xfs_scrub has been
modified to take advantage of them, and can now fall back a little more
gracefully when kernel scrubbing is unavailable.

(Note that the WTF tree doesn't reflect the iomap rework that's in the
4.9 for-next tree; I'll rebase WTF against 4.9 after the merge window
closes.  Unfortunately that's the only way to pick up scrub or realtime
rmap support.)

--D

Re: XFS reflink and development tip git trees updated

[Christoph Hellwig]

On Mon, Sep 26, 2016 at 10:23:44PM -0700, Darrick J. Wong wrote:
> Ok, here's this week's for-4.9 branch:
> 
> https://github.com/djwong/linux/tree/for-dave-for-4.9-5
> https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-5
> 
> The kernel tree includes Christoph's fixes for copy-on-write,
> refactoring of a refcount btree function to use _btree_query_range (and
> inclusion of the required diff_two_keys/init_high_key helpers), and
> removal of the (totally broken) always_cow debug switch.  It also
> contains a new error injection point to make reflink fail with ENOSPC.

Are you going to send out the patches to the list?  Also it seems like
the writeback updates weren't folded in but just added to the end,
was that intentional?

Re: XFS reflink and development tip git trees updated

[Darrick J. Wong]

On Tue, Sep 27, 2016 at 07:27:26AM -0700, Christoph Hellwig wrote:
> On Mon, Sep 26, 2016 at 10:23:44PM -0700, Darrick J. Wong wrote:
> > Ok, here's this week's for-4.9 branch:
> > 
> > https://github.com/djwong/linux/tree/for-dave-for-4.9-5
> > https://github.com/djwong/xfsprogs/tree/for-dave-for-4.9-5
> > 
> > The kernel tree includes Christoph's fixes for copy-on-write,
> > refactoring of a refcount btree function to use _btree_query_range (and
> > inclusion of the required diff_two_keys/init_high_key helpers), and
> > removal of the (totally broken) always_cow debug switch.  It also
> > contains a new error injection point to make reflink fail with ENOSPC.
> 
> Are you going to send out the patches to the list?

Yeah, I'll spam^Wmail the list once more today.

> Also it seems like the writeback updates weren't folded in but just
> added to the end, was that intentional?

Yes -- both because I'm trying to limit the number of changes to the
patches in the middle that Dave might be reviewing, and also to get
all the combined fixes out for testing quicker.

(Don't worry, I'm already working on a 4.9-6 with these things cleaned
up.)

--D