[RFC, PATCH 0/2] xfs: Fast zeroing of allocated space

[RFC, PATCH 0/2] xfs: Fast zeroing of allocated space

[Dave Chinner]

These two patches allow conversion of a written range of a file into
back into unwritten state. The first patch converts page cache
helper functions to use the correct range primitives as we need to
be able to toss (invalidate) pages only within the range specified,
not to to the end of the file as it currently uses.

The actual conversion also preallocates any holes in the range, so
it turns the entire range requested into allocated, unwritten
extents.

I also have a patch that adds this functionality to fallocate(). I
haven't tested that at all, but if this is something we want to
support, I'd suggest that we want fallocate to be able to do it...

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 1/2] xfs: use range primitives for xfs page cache operations

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

While XFS passes ranges to operate on from the core code, the
functions being called ignore the either the entire range or the end
of the range. This is historical because when the function were
written linux didn't have the necessary range operations. Update the
functions to use the correct operations.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 fs/xfs/linux-2.6/xfs_fs_subr.c |   19 ++++++++++---------
 1 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/fs/xfs/linux-2.6/xfs_fs_subr.c b/fs/xfs/linux-2.6/xfs_fs_subr.c
index 1f279b0..e76d31d 100644
--- a/fs/xfs/linux-2.6/xfs_fs_subr.c
+++ b/fs/xfs/linux-2.6/xfs_fs_subr.c
@@ -32,10 +32,7 @@ xfs_tosspages(
 	xfs_off_t	last,
 	int		fiopt)
 {
-	struct address_space *mapping = VFS_I(ip)->i_mapping;
-
-	if (mapping->nrpages)
-		truncate_inode_pages(mapping, first);
+	truncate_inode_pages_range(VFS_I(ip)->i_mapping, first, last);
 }
 
 int
@@ -52,9 +49,10 @@ xfs_flushinval_pages(
 
 	if (mapping->nrpages) {
 		xfs_iflags_clear(ip, XFS_ITRUNCATED);
-		ret = filemap_write_and_wait(mapping);
+		ret = filemap_write_and_wait_range(mapping, first,
+					last == -1 ? LLONG_MAX : last);
 		if (!ret)
-			truncate_inode_pages(mapping, first);
+			truncate_inode_pages_range(mapping, first, last);
 	}
 	return -ret;
 }
@@ -73,7 +71,8 @@ xfs_flush_pages(
 
 	if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY)) {
 		xfs_iflags_clear(ip, XFS_ITRUNCATED);
-		ret = -filemap_fdatawrite(mapping);
+		ret = -filemap_fdatawrite_range(mapping, first,
+					last == -1 ? LLONG_MAX : last);
 	}
 	if (flags & XBF_ASYNC)
 		return ret;
@@ -91,7 +90,9 @@ xfs_wait_on_pages(
 {
 	struct address_space *mapping = VFS_I(ip)->i_mapping;
 
-	if (mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK))
-		return -filemap_fdatawait(mapping);
+	if (mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK)) {
+		return -filemap_fdatawait_range(mapping, first,
+					last == -1 ? ip->i_size - 1 : last);
+	}
 	return 0;
 }
-- 
1.7.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 2/2] xfs: Introduce XFS_IOC_ZERO_RANGE

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

XFS_IOC_ZERO_RANGE is the equivalent of an atomic XFS_IOC_UNRESVSP/
XFS_IOC_RESVSP call pair. It enabled ranges of written data to be
turned into zeroes without requiring IO or having to free and
reallocate the extents in the range given as would occur if we had
to punch and then preallocate them separately.  This enables
applications to zero parts of files very quickly without changing
the layout of the files in any way.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 fs/xfs/linux-2.6/xfs_ioctl.c   |    3 ++-
 fs/xfs/linux-2.6/xfs_ioctl32.c |    1 +
 fs/xfs/xfs_bmap.c              |   14 +++++++++++---
 fs/xfs/xfs_bmap.h              |    9 ++++++---
 fs/xfs/xfs_fs.h                |    1 +
 fs/xfs/xfs_vnodeops.c          |   13 +++++++++++--
 6 files changed, 32 insertions(+), 9 deletions(-)

diff --git a/fs/xfs/linux-2.6/xfs_ioctl.c b/fs/xfs/linux-2.6/xfs_ioctl.c
index 237f5ff..a9c4810 100644
--- a/fs/xfs/linux-2.6/xfs_ioctl.c
+++ b/fs/xfs/linux-2.6/xfs_ioctl.c
@@ -1292,7 +1292,8 @@ xfs_file_ioctl(
 	case XFS_IOC_ALLOCSP64:
 	case XFS_IOC_FREESP64:
 	case XFS_IOC_RESVSP64:
-	case XFS_IOC_UNRESVSP64: {
+	case XFS_IOC_UNRESVSP64:
+	case XFS_IOC_ZERO_RANGE: {
 		xfs_flock64_t		bf;
 
 		if (copy_from_user(&bf, arg, sizeof(bf)))
diff --git a/fs/xfs/linux-2.6/xfs_ioctl32.c b/fs/xfs/linux-2.6/xfs_ioctl32.c
index 6c83f7f..464bcc7 100644
--- a/fs/xfs/linux-2.6/xfs_ioctl32.c
+++ b/fs/xfs/linux-2.6/xfs_ioctl32.c
@@ -574,6 +574,7 @@ xfs_file_compat_ioctl(
 	case XFS_IOC_FSGEOMETRY_V1:
 	case XFS_IOC_FSGROWFSDATA:
 	case XFS_IOC_FSGROWFSRT:
+	case XFS_IOC_ZERO_RANGE:
 		return xfs_file_ioctl(filp, cmd, p);
 #else
 	case XFS_IOC_ALLOCSP_32:
diff --git a/fs/xfs/xfs_bmap.c b/fs/xfs/xfs_bmap.c
index 23f14e5..581e646 100644
--- a/fs/xfs/xfs_bmap.c
+++ b/fs/xfs/xfs_bmap.c
@@ -4744,8 +4744,14 @@ xfs_bmapi(
 		 * Check if writing previously allocated but
 		 * unwritten extents.
 		 */
-		if (wr && mval->br_state == XFS_EXT_UNWRITTEN &&
-		    ((flags & (XFS_BMAPI_PREALLOC|XFS_BMAPI_DELAY)) == 0)) {
+		if (wr &&
+		    ((mval->br_state == XFS_EXT_UNWRITTEN &&
+		      ((flags & (XFS_BMAPI_PREALLOC|XFS_BMAPI_DELAY)) == 0)) ||
+		     (mval->br_state == XFS_EXT_NORM &&
+		      ((flags & (XFS_BMAPI_PREALLOC|XFS_BMAPI_CONVERT)) ==
+				(XFS_BMAPI_PREALLOC|XFS_BMAPI_CONVERT))))) {
+
+			printk("extent conversion\n");
 			/*
 			 * Modify (by adding) the state flag, if writing.
 			 */
@@ -4757,7 +4763,9 @@ xfs_bmapi(
 					*firstblock;
 				cur->bc_private.b.flist = flist;
 			}
-			mval->br_state = XFS_EXT_NORM;
+			mval->br_state = (mval->br_state == XFS_EXT_UNWRITTEN)
+						? XFS_EXT_NORM
+						: XFS_EXT_UNWRITTEN;
 			error = xfs_bmap_add_extent(ip, lastx, &cur, mval,
 				firstblock, flist, &tmp_logflags,
 				whichfork, (flags & XFS_BMAPI_RSVBLOCKS));
diff --git a/fs/xfs/xfs_bmap.h b/fs/xfs/xfs_bmap.h
index b13569a..71ec9b6 100644
--- a/fs/xfs/xfs_bmap.h
+++ b/fs/xfs/xfs_bmap.h
@@ -74,9 +74,12 @@ typedef	struct xfs_bmap_free
 #define	XFS_BMAPI_IGSTATE	0x080	/* Ignore state - */
 					/* combine contig. space */
 #define	XFS_BMAPI_CONTIG	0x100	/* must allocate only one extent */
-#define XFS_BMAPI_CONVERT	0x200	/* unwritten extent conversion - */
-					/* need write cache flushing and no */
-					/* additional allocation alignments */
+/*
+ * unwritten extent conversion - this needs write cache flushing and no additional
+ * allocation alignments. When specified with XFS_BMAPI_PREALLOC it converts
+ * from written to unwritten, otherwise convert from unwritten to written.
+ */
+#define XFS_BMAPI_CONVERT	0x200
 
 #define XFS_BMAPI_FLAGS \
 	{ XFS_BMAPI_WRITE,	"WRITE" }, \
diff --git a/fs/xfs/xfs_fs.h b/fs/xfs/xfs_fs.h
index 7cf7220..6af6518 100644
--- a/fs/xfs/xfs_fs.h
+++ b/fs/xfs/xfs_fs.h
@@ -446,6 +446,7 @@ typedef struct xfs_handle {
 /*	XFS_IOC_SETBIOSIZE ---- deprecated 46	   */
 /*	XFS_IOC_GETBIOSIZE ---- deprecated 47	   */
 #define XFS_IOC_GETBMAPX	_IOWR('X', 56, struct getbmap)
+#define XFS_IOC_ZERO_RANGE	_IOW ('X', 57, struct xfs_flock64)
 
 /*
  * ioctl commands that replace IRIX syssgi()'s
diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c
index 3ac137d..9f82422 100644
--- a/fs/xfs/xfs_vnodeops.c
+++ b/fs/xfs/xfs_vnodeops.c
@@ -2270,7 +2270,7 @@ xfs_alloc_file_space(
 	count = len;
 	imapp = &imaps[0];
 	nimaps = 1;
-	bmapi_flag = XFS_BMAPI_WRITE | (alloc_type ? XFS_BMAPI_PREALLOC : 0);
+	bmapi_flag = XFS_BMAPI_WRITE | alloc_type;
 	startoffset_fsb	= XFS_B_TO_FSBT(mp, offset);
 	allocatesize_fsb = XFS_B_TO_FSB(mp, count);
 
@@ -2702,7 +2702,9 @@ xfs_change_file_space(
 	xfs_off_t	llen;
 	xfs_trans_t	*tp;
 	struct iattr	iattr;
+	int		prealloc_type;
 
+	printk("type 0x%x\n", bf->l_type);
 	if (!S_ISREG(ip->i_d.di_mode))
 		return XFS_ERROR(EINVAL);
 
@@ -2744,12 +2746,19 @@ xfs_change_file_space(
 	 * size to be changed.
 	 */
 	setprealloc = clrprealloc = 0;
+	prealloc_type = XFS_BMAPI_PREALLOC;
 
 	switch (cmd) {
+	case XFS_IOC_ZERO_RANGE:
+		prealloc_type |= XFS_BMAPI_CONVERT;
+		printk("force prealloc off 0x%llx len 0x%llx\n",
+				startoffset, bf->l_len);
+		xfs_tosspages(ip, startoffset, startoffset + bf->l_len, 0);
+		/* FALLTHRU */
 	case XFS_IOC_RESVSP:
 	case XFS_IOC_RESVSP64:
 		error = xfs_alloc_file_space(ip, startoffset, bf->l_len,
-								1, attr_flags);
+						prealloc_type, attr_flags);
 		if (error)
 			return error;
 		setprealloc = 1;
-- 
1.7.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/2] xfs: use range primitives for xfs page cache operations

[Christoph Hellwig]

On Tue, Jul 27, 2010 at 03:55:28PM +1000, Dave Chinner wrote:
> While XFS passes ranges to operate on from the core code, the
> functions being called ignore the either the entire range or the end
> of the range. This is historical because when the function were
> written linux didn't have the necessary range operations. Update the
> functions to use the correct operations.

Assuming you have actually tested this - given that we've ignore
these parameters so long that I'm really fearing some callers have
started to rely on that behaviour.

>  	if (mapping->nrpages) {

I'd drop this check ere as well - no other caller does it.

>  		xfs_iflags_clear(ip, XFS_ITRUNCATED);
> -		ret = filemap_write_and_wait(mapping);
> +		ret = filemap_write_and_wait_range(mapping, first,
> +					last == -1 ? LLONG_MAX : last);
>  		if (!ret)
> -			truncate_inode_pages(mapping, first);
> +			truncate_inode_pages_range(mapping, first, last);
>  	}
>  	return -ret;
>  }
> @@ -73,7 +71,8 @@ xfs_flush_pages(
>  
>  	if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY)) {

Same for this check.

>  		xfs_iflags_clear(ip, XFS_ITRUNCATED);
> -		ret = -filemap_fdatawrite(mapping);
> +		ret = -filemap_fdatawrite_range(mapping, first,
> +					last == -1 ? LLONG_MAX : last);

Also for the non-async case we should just use
filemap_write_and_wait_range, and kill off xfs_wait_on_pages.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/2] xfs: Introduce XFS_IOC_ZERO_RANGE

[Christoph Hellwig]

On Tue, Jul 27, 2010 at 03:55:29PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> XFS_IOC_ZERO_RANGE is the equivalent of an atomic XFS_IOC_UNRESVSP/
> XFS_IOC_RESVSP call pair. It enabled ranges of written data to be
> turned into zeroes without requiring IO or having to free and
> reallocate the extents in the range given as would occur if we had
> to punch and then preallocate them separately.  This enables
> applications to zero parts of files very quickly without changing
> the layout of the files in any way.

This looks good minus the left over printks.  It'll also needs xfstests
coverage and a description in the xfsctl manpage.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/2] xfs: use range primitives for xfs page cache operations

[Dave Chinner]

On Tue, Jul 27, 2010 at 02:53:26AM -0400, Christoph Hellwig wrote:
> On Tue, Jul 27, 2010 at 03:55:28PM +1000, Dave Chinner wrote:
> > While XFS passes ranges to operate on from the core code, the
> > functions being called ignore the either the entire range or the end
> > of the range. This is historical because when the function were
> > written linux didn't have the necessary range operations. Update the
> > functions to use the correct operations.
> 
> Assuming you have actually tested this - given that we've ignore
> these parameters so long that I'm really fearing some callers have
> started to rely on that behaviour.

Several xfstests run on different configs haven't shown any
regressions.

All current callers of xfs_tosspages() trim from and offset to EOF,
so no change in behaviour there.

All of the xfs_flush_pages() icallers except one flush the entire
file (0 to -1), except for a single call in xfs_setattr which
between on disk size and the new filesystem when extending the file
size by truncate.  It will now flush just the necessary range
instead of the whole file.

All of xfs_flushinval_pages() callers flush to the end of file,
so once again there should be no change in behaviour there.

So I don't think there really is much risk here - the only one that
I'd be concerned about is the setattr() call and we have tests
covering that specific case (138, 139 and 140)....

> >  	if (mapping->nrpages) {
> 
> I'd drop this check ere as well - no other caller does it.
> 
> >  		xfs_iflags_clear(ip, XFS_ITRUNCATED);
> > -		ret = filemap_write_and_wait(mapping);
> > +		ret = filemap_write_and_wait_range(mapping, first,
> > +					last == -1 ? LLONG_MAX : last);
> >  		if (!ret)
> > -			truncate_inode_pages(mapping, first);
> > +			truncate_inode_pages_range(mapping, first, last);
> >  	}
> >  	return -ret;
> >  }
> > @@ -73,7 +71,8 @@ xfs_flush_pages(
> >  
> >  	if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY)) {
> 
> Same for this check.

Ok, killed those checks.

> >  		xfs_iflags_clear(ip, XFS_ITRUNCATED);
> > -		ret = -filemap_fdatawrite(mapping);
> > +		ret = -filemap_fdatawrite_range(mapping, first,
> > +					last == -1 ? LLONG_MAX : last);
> 
> Also for the non-async case we should just use
> filemap_write_and_wait_range, and kill off xfs_wait_on_pages.

That can be done separately, I think. If we don't see any
problems from this patch, then I think we can kill all
the wrappers, not just one of them.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [RFC, PATCH 0/2] xfs: Fast zeroing of allocated space

[Alex Elder]

On Tue, 2010-07-27 at 15:55 +1000, Dave Chinner wrote:
> These two patches allow conversion of a written range of a file into
> back into unwritten state. The first patch converts page cache
> helper functions to use the correct range primitives as we need to
> be able to toss (invalidate) pages only within the range specified,
> not to to the end of the file as it currently uses.
> 
> The actual conversion also preallocates any holes in the range, so
> it turns the entire range requested into allocated, unwritten
> extents.
> 
> I also have a patch that adds this functionality to fallocate(). I
> haven't tested that at all, but if this is something we want to
> support, I'd suggest that we want fallocate to be able to do it...

Nice feature.  Kind of surprising to me how small the set
of changes it requires is.

I agree with Christoph's comments.  These two patches otherwise
look good to me as well.

Reviewed-by: Alex Elder <aelder@sgi.com>


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs