[PATCHSET v2 0/2] xfs: make quota reservations for directory changes

[PATCHSET v2 0/2] xfs: make quota reservations for directory changes

[Darrick J. Wong]

Hi all,

A couple of weeks ago, I was triaging an unexpected quota counter
failure report coming from xfs_repair after the online repair code
rebuilt a directory.  It turned out that the XFS implementation of
linkat does not reserve any quota, which means that a directory
expansion can increase the quota block count beyond the hard limit.
Similar problems exist in the unlink, rmdir, and rename code, so we'll
fix those problems at the same time.

Note: this series does not try to fix a similar bug in the swapext code,
because estimating the necessary quota reservation is very very tricky.
I wrote all that estimation code as the first part of the atomic extent
exchange patchset, so I'll leave that there.

v2: fix unlink and rename, since Dave suggested it

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=quota-reservations-5.18
---
 fs/xfs/xfs_inode.c |   67 +++++++++++++++++++++++++++--------------
 fs/xfs/xfs_trans.c |   86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_trans.h |    3 ++
 3 files changed, 133 insertions(+), 23 deletions(-)

[PATCH 1/2] xfs: reserve quota for dir expansion when linking/unlinking files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

XFS does not reserve quota for directory expansion when linking or
unlinking children from a directory.  This means that we don't reject
the expansion with EDQUOT when we're at or near a hard limit, which
means that unprivileged userspace can use link()/unlink() to exceed
quota.

The fix for this is nuanced -- link operations don't always expand the
directory, and we allow a link to proceed with no space reservation if
we don't need to add a block to the directory to handle the addition.
Unlink operations generally do not expand the directory (you'd have to
free a block and then cause a btree split) and we can defer the
directory block freeing if there is no space reservation.

Moreover, there is a further bug in that we do not trigger the blockgc
workers to try to clear space when we're out of quota.

To fix both cases, create a new xfs_trans_alloc_dir function that
allocates the transaction, locks and joins the inodes, and reserves
quota for the directory.  If there isn't sufficient space or quota,
we'll switch the caller to reservationless mode.  This should prevent
quota usage overruns with the least restriction in functionality.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 fs/xfs/xfs_inode.c |   30 +++++-------------
 fs/xfs/xfs_trans.c |   86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_trans.h |    3 ++
 3 files changed, 97 insertions(+), 22 deletions(-)


diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index 04bf467b1090..a131bbfe74e4 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -1217,7 +1217,7 @@ xfs_link(
 {
 	xfs_mount_t		*mp = tdp->i_mount;
 	xfs_trans_t		*tp;
-	int			error;
+	int			error, space_error;
 	int			resblks;
 
 	trace_xfs_link(tdp, target_name);
@@ -1236,19 +1236,11 @@ xfs_link(
 		goto std_return;
 
 	resblks = XFS_LINK_SPACE_RES(mp, target_name->len);
-	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, resblks, 0, 0, &tp);
-	if (error == -ENOSPC) {
-		resblks = 0;
-		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, 0, 0, 0, &tp);
-	}
+	error = xfs_trans_alloc_dir(tdp, &M_RES(mp)->tr_link, sip, &resblks,
+			&tp, &space_error);
 	if (error)
 		goto std_return;
 
-	xfs_lock_two_inodes(sip, XFS_ILOCK_EXCL, tdp, XFS_ILOCK_EXCL);
-
-	xfs_trans_ijoin(tp, sip, XFS_ILOCK_EXCL);
-	xfs_trans_ijoin(tp, tdp, XFS_ILOCK_EXCL);
-
 	error = xfs_iext_count_may_overflow(tdp, XFS_DATA_FORK,
 			XFS_IEXT_DIR_MANIP_CNT(mp));
 	if (error)
@@ -1267,6 +1259,8 @@ xfs_link(
 
 	if (!resblks) {
 		error = xfs_dir_canenter(tp, tdp, target_name);
+		if (error == -ENOSPC && space_error)
+			error = space_error;
 		if (error)
 			goto error_return;
 	}
@@ -2755,6 +2749,7 @@ xfs_remove(
 	xfs_mount_t		*mp = dp->i_mount;
 	xfs_trans_t             *tp = NULL;
 	int			is_dir = S_ISDIR(VFS_I(ip)->i_mode);
+	int			dontcare;
 	int                     error = 0;
 	uint			resblks;
 
@@ -2781,22 +2776,13 @@ xfs_remove(
 	 * block from the directory.
 	 */
 	resblks = XFS_REMOVE_SPACE_RES(mp);
-	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, resblks, 0, 0, &tp);
-	if (error == -ENOSPC) {
-		resblks = 0;
-		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, 0, 0, 0,
-				&tp);
-	}
+	error = xfs_trans_alloc_dir(dp, &M_RES(mp)->tr_remove, ip, &resblks,
+			&tp, &dontcare);
 	if (error) {
 		ASSERT(error != -ENOSPC);
 		goto std_return;
 	}
 
-	xfs_lock_two_inodes(dp, XFS_ILOCK_EXCL, ip, XFS_ILOCK_EXCL);
-
-	xfs_trans_ijoin(tp, dp, XFS_ILOCK_EXCL);
-	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
-
 	/*
 	 * If we're removing a directory perform some additional validation.
 	 */
diff --git a/fs/xfs/xfs_trans.c b/fs/xfs/xfs_trans.c
index 59e2f9031b9f..75327f2aeb99 100644
--- a/fs/xfs/xfs_trans.c
+++ b/fs/xfs/xfs_trans.c
@@ -1210,3 +1210,89 @@ xfs_trans_alloc_ichange(
 	xfs_trans_cancel(tp);
 	return error;
 }
+
+/*
+ * Allocate an transaction, lock and join the directory and child inodes to it,
+ * and reserve quota for a directory update.  If there isn't sufficient space,
+ * @dblocks will be set to zero for a reservationless directory update and
+ * @space_error will be set to a negative errno describing the space constraint
+ * we hit.
+ *
+ * The caller must ensure that the on-disk dquots attached to this inode have
+ * already been allocated and initialized.  The ILOCKs will be dropped when the
+ * transaction is committed or cancelled.
+ */
+int
+xfs_trans_alloc_dir(
+	struct xfs_inode	*dp,
+	struct xfs_trans_res	*resv,
+	struct xfs_inode	*ip,
+	unsigned int		*dblocks,
+	struct xfs_trans	**tpp,
+	int			*space_error)
+{
+	struct xfs_trans	*tp;
+	struct xfs_mount	*mp = ip->i_mount;
+	unsigned int		resblks;
+	bool			retried = false;
+	int			error;
+
+retry:
+	*space_error = 0;
+	resblks = *dblocks;
+	error = xfs_trans_alloc(mp, resv, resblks, 0, 0, &tp);
+	if (error == -ENOSPC) {
+		*space_error = error;
+		resblks = 0;
+		error = xfs_trans_alloc(mp, resv, resblks, 0, 0, &tp);
+	}
+	if (error)
+		return error;
+
+	xfs_lock_two_inodes(dp, XFS_ILOCK_EXCL, ip, XFS_ILOCK_EXCL);
+
+	xfs_trans_ijoin(tp, dp, XFS_ILOCK_EXCL);
+	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
+
+	error = xfs_qm_dqattach_locked(dp, false);
+	if (error) {
+		/* Caller should have allocated the dquots! */
+		ASSERT(error != -ENOENT);
+		goto out_cancel;
+	}
+
+	error = xfs_qm_dqattach_locked(ip, false);
+	if (error) {
+		/* Caller should have allocated the dquots! */
+		ASSERT(error != -ENOENT);
+		goto out_cancel;
+	}
+
+	if (resblks == 0)
+		goto done;
+
+	error = xfs_trans_reserve_quota_nblks(tp, dp, resblks, 0, false);
+	if (error == -EDQUOT || error == -ENOSPC) {
+		if (!retried) {
+			xfs_trans_cancel(tp);
+			xfs_blockgc_free_quota(dp, 0);
+			retried = true;
+			goto retry;
+		}
+
+		*space_error = error;
+		resblks = 0;
+		error = 0;
+	}
+	if (error)
+		goto out_cancel;
+
+done:
+	*tpp = tp;
+	*dblocks = resblks;
+	return 0;
+
+out_cancel:
+	xfs_trans_cancel(tp);
+	return error;
+}
diff --git a/fs/xfs/xfs_trans.h b/fs/xfs/xfs_trans.h
index a487b264a9eb..3b9128d6e214 100644
--- a/fs/xfs/xfs_trans.h
+++ b/fs/xfs/xfs_trans.h
@@ -259,6 +259,9 @@ int xfs_trans_alloc_icreate(struct xfs_mount *mp, struct xfs_trans_res *resv,
 int xfs_trans_alloc_ichange(struct xfs_inode *ip, struct xfs_dquot *udqp,
 		struct xfs_dquot *gdqp, struct xfs_dquot *pdqp, bool force,
 		struct xfs_trans **tpp);
+int xfs_trans_alloc_dir(struct xfs_inode *dp, struct xfs_trans_res *resv,
+		struct xfs_inode *ip, unsigned int *dblocks,
+		struct xfs_trans **tpp, int *space_error);
 
 static inline void
 xfs_trans_set_context(

Re: [PATCH 1/2] xfs: reserve quota for dir expansion when linking/unlinking files

[Dave Chinner]

On Wed, Mar 09, 2022 at 11:22:26AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> XFS does not reserve quota for directory expansion when linking or
> unlinking children from a directory.  This means that we don't reject
> the expansion with EDQUOT when we're at or near a hard limit, which
> means that unprivileged userspace can use link()/unlink() to exceed
> quota.
> 
> The fix for this is nuanced -- link operations don't always expand the
> directory, and we allow a link to proceed with no space reservation if
> we don't need to add a block to the directory to handle the addition.
> Unlink operations generally do not expand the directory (you'd have to
> free a block and then cause a btree split) and we can defer the
> directory block freeing if there is no space reservation.
> 
> Moreover, there is a further bug in that we do not trigger the blockgc
> workers to try to clear space when we're out of quota.
> 
> To fix both cases, create a new xfs_trans_alloc_dir function that
> allocates the transaction, locks and joins the inodes, and reserves
> quota for the directory.  If there isn't sufficient space or quota,
> we'll switch the caller to reservationless mode.  This should prevent
> quota usage overruns with the least restriction in functionality.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  fs/xfs/xfs_inode.c |   30 +++++-------------
>  fs/xfs/xfs_trans.c |   86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/xfs/xfs_trans.h |    3 ++
>  3 files changed, 97 insertions(+), 22 deletions(-)

Overall looks good, minor nits below:

> 
> 
> diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
> index 04bf467b1090..a131bbfe74e4 100644
> --- a/fs/xfs/xfs_inode.c
> +++ b/fs/xfs/xfs_inode.c
> @@ -1217,7 +1217,7 @@ xfs_link(
>  {
>  	xfs_mount_t		*mp = tdp->i_mount;
>  	xfs_trans_t		*tp;
> -	int			error;
> +	int			error, space_error;
>  	int			resblks;
>  
>  	trace_xfs_link(tdp, target_name);
> @@ -1236,19 +1236,11 @@ xfs_link(
>  		goto std_return;
>  
>  	resblks = XFS_LINK_SPACE_RES(mp, target_name->len);
> -	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, resblks, 0, 0, &tp);
> -	if (error == -ENOSPC) {
> -		resblks = 0;
> -		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, 0, 0, 0, &tp);
> -	}
> +	error = xfs_trans_alloc_dir(tdp, &M_RES(mp)->tr_link, sip, &resblks,
> +			&tp, &space_error);

It's the nospace_error, isn't it? The code reads a lot better when
it's called that, too.


>  	if (error)
>  		goto std_return;
>  
> -	xfs_lock_two_inodes(sip, XFS_ILOCK_EXCL, tdp, XFS_ILOCK_EXCL);
> -
> -	xfs_trans_ijoin(tp, sip, XFS_ILOCK_EXCL);
> -	xfs_trans_ijoin(tp, tdp, XFS_ILOCK_EXCL);
> -
>  	error = xfs_iext_count_may_overflow(tdp, XFS_DATA_FORK,
>  			XFS_IEXT_DIR_MANIP_CNT(mp));
>  	if (error)
> @@ -1267,6 +1259,8 @@ xfs_link(
>  
>  	if (!resblks) {
>  		error = xfs_dir_canenter(tp, tdp, target_name);
> +		if (error == -ENOSPC && space_error)
> +			error = space_error;

This  would be better in the error_return stack, I think. That way
the transformation only has to be done once, and it will be done for
all functions that can potentially return ENOSPC.

>  		if (error)
>  			goto error_return;
>  	}
> @@ -2755,6 +2749,7 @@ xfs_remove(
>  	xfs_mount_t		*mp = dp->i_mount;
>  	xfs_trans_t             *tp = NULL;
>  	int			is_dir = S_ISDIR(VFS_I(ip)->i_mode);
> +	int			dontcare;
>  	int                     error = 0;
>  	uint			resblks;
>  
> @@ -2781,22 +2776,13 @@ xfs_remove(
>  	 * block from the directory.
>  	 */
>  	resblks = XFS_REMOVE_SPACE_RES(mp);
> -	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, resblks, 0, 0, &tp);
> -	if (error == -ENOSPC) {
> -		resblks = 0;
> -		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, 0, 0, 0,
> -				&tp);
> -	}
> +	error = xfs_trans_alloc_dir(dp, &M_RES(mp)->tr_remove, ip, &resblks,
> +			&tp, &dontcare);
>  	if (error) {
>  		ASSERT(error != -ENOSPC);
>  		goto std_return;
>  	}

So we just ignore -EDQUOT when it is returned in @dontcare? I'd like
a comment to explain why we don't care about EDQUOT here, because
the next time I look at this I will have forgotten all about this...

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 1/2] xfs: reserve quota for dir expansion when linking/unlinking files

[Darrick J. Wong]

On Thu, Mar 10, 2022 at 08:48:21AM +1100, Dave Chinner wrote:
> On Wed, Mar 09, 2022 at 11:22:26AM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > XFS does not reserve quota for directory expansion when linking or
> > unlinking children from a directory.  This means that we don't reject
> > the expansion with EDQUOT when we're at or near a hard limit, which
> > means that unprivileged userspace can use link()/unlink() to exceed
> > quota.
> > 
> > The fix for this is nuanced -- link operations don't always expand the
> > directory, and we allow a link to proceed with no space reservation if
> > we don't need to add a block to the directory to handle the addition.
> > Unlink operations generally do not expand the directory (you'd have to
> > free a block and then cause a btree split) and we can defer the
> > directory block freeing if there is no space reservation.
> > 
> > Moreover, there is a further bug in that we do not trigger the blockgc
> > workers to try to clear space when we're out of quota.
> > 
> > To fix both cases, create a new xfs_trans_alloc_dir function that
> > allocates the transaction, locks and joins the inodes, and reserves
> > quota for the directory.  If there isn't sufficient space or quota,
> > we'll switch the caller to reservationless mode.  This should prevent
> > quota usage overruns with the least restriction in functionality.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  fs/xfs/xfs_inode.c |   30 +++++-------------
> >  fs/xfs/xfs_trans.c |   86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  fs/xfs/xfs_trans.h |    3 ++
> >  3 files changed, 97 insertions(+), 22 deletions(-)
> 
> Overall looks good, minor nits below:
> 
> > 
> > 
> > diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
> > index 04bf467b1090..a131bbfe74e4 100644
> > --- a/fs/xfs/xfs_inode.c
> > +++ b/fs/xfs/xfs_inode.c
> > @@ -1217,7 +1217,7 @@ xfs_link(
> >  {
> >  	xfs_mount_t		*mp = tdp->i_mount;
> >  	xfs_trans_t		*tp;
> > -	int			error;
> > +	int			error, space_error;
> >  	int			resblks;
> >  
> >  	trace_xfs_link(tdp, target_name);
> > @@ -1236,19 +1236,11 @@ xfs_link(
> >  		goto std_return;
> >  
> >  	resblks = XFS_LINK_SPACE_RES(mp, target_name->len);
> > -	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, resblks, 0, 0, &tp);
> > -	if (error == -ENOSPC) {
> > -		resblks = 0;
> > -		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_link, 0, 0, 0, &tp);
> > -	}
> > +	error = xfs_trans_alloc_dir(tdp, &M_RES(mp)->tr_link, sip, &resblks,
> > +			&tp, &space_error);
> 
> It's the nospace_error, isn't it? The code reads a lot better when
> it's called that, too.

Fixed.

> 
> >  	if (error)
> >  		goto std_return;
> >  
> > -	xfs_lock_two_inodes(sip, XFS_ILOCK_EXCL, tdp, XFS_ILOCK_EXCL);
> > -
> > -	xfs_trans_ijoin(tp, sip, XFS_ILOCK_EXCL);
> > -	xfs_trans_ijoin(tp, tdp, XFS_ILOCK_EXCL);
> > -
> >  	error = xfs_iext_count_may_overflow(tdp, XFS_DATA_FORK,
> >  			XFS_IEXT_DIR_MANIP_CNT(mp));
> >  	if (error)
> > @@ -1267,6 +1259,8 @@ xfs_link(
> >  
> >  	if (!resblks) {
> >  		error = xfs_dir_canenter(tp, tdp, target_name);
> > +		if (error == -ENOSPC && space_error)
> > +			error = space_error;
> 
> This  would be better in the error_return stack, I think. That way
> the transformation only has to be done once, and it will be done for
> all functions that can potentially return ENOSPC.

Ok.

> >  		if (error)
> >  			goto error_return;
> >  	}
> > @@ -2755,6 +2749,7 @@ xfs_remove(
> >  	xfs_mount_t		*mp = dp->i_mount;
> >  	xfs_trans_t             *tp = NULL;
> >  	int			is_dir = S_ISDIR(VFS_I(ip)->i_mode);
> > +	int			dontcare;
> >  	int                     error = 0;
> >  	uint			resblks;
> >  
> > @@ -2781,22 +2776,13 @@ xfs_remove(
> >  	 * block from the directory.
> >  	 */
> >  	resblks = XFS_REMOVE_SPACE_RES(mp);
> > -	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, resblks, 0, 0, &tp);
> > -	if (error == -ENOSPC) {
> > -		resblks = 0;
> > -		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, 0, 0, 0,
> > -				&tp);
> > -	}
> > +	error = xfs_trans_alloc_dir(dp, &M_RES(mp)->tr_remove, ip, &resblks,
> > +			&tp, &dontcare);
> >  	if (error) {
> >  		ASSERT(error != -ENOSPC);
> >  		goto std_return;
> >  	}
> 
> So we just ignore -EDQUOT when it is returned in @dontcare? I'd like
> a comment to explain why we don't care about EDQUOT here, because
> the next time I look at this I will have forgotten all about this...

Ok.  How about:

	/*
	 * We try to get the real space reservation first, allowing for
	 * directory btree deletion(s) implying possible bmap insert(s).
	 * If we can't get the space reservation then we use 0 instead,
	 * and avoid the bmap btree insert(s) in the directory code by,
	 * if the bmap insert tries to happen, instead trimming the LAST
	 * block from the directory.
	 *
	 * Ignore EDQUOT and ENOSPC being returned via nospace_error
	 * because the directory code can handle a reservationless
	 * update and we don't want to prevent a user from trying to
	 * free space by deleting things.
	 */
	error = xfs_trans_alloc_dir(...);

--D

> Cheers,
> 
> Dave.
> 
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH 1/2] xfs: reserve quota for dir expansion when linking/unlinking files

[Dave Chinner]

On Wed, Mar 09, 2022 at 03:33:02PM -0800, Darrick J. Wong wrote:
> On Thu, Mar 10, 2022 at 08:48:21AM +1100, Dave Chinner wrote:
> > On Wed, Mar 09, 2022 at 11:22:26AM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > >  		if (error)
> > >  			goto error_return;
> > >  	}
> > > @@ -2755,6 +2749,7 @@ xfs_remove(
> > >  	xfs_mount_t		*mp = dp->i_mount;
> > >  	xfs_trans_t             *tp = NULL;
> > >  	int			is_dir = S_ISDIR(VFS_I(ip)->i_mode);
> > > +	int			dontcare;
> > >  	int                     error = 0;
> > >  	uint			resblks;
> > >  
> > > @@ -2781,22 +2776,13 @@ xfs_remove(
> > >  	 * block from the directory.
> > >  	 */
> > >  	resblks = XFS_REMOVE_SPACE_RES(mp);
> > > -	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, resblks, 0, 0, &tp);
> > > -	if (error == -ENOSPC) {
> > > -		resblks = 0;
> > > -		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_remove, 0, 0, 0,
> > > -				&tp);
> > > -	}
> > > +	error = xfs_trans_alloc_dir(dp, &M_RES(mp)->tr_remove, ip, &resblks,
> > > +			&tp, &dontcare);
> > >  	if (error) {
> > >  		ASSERT(error != -ENOSPC);
> > >  		goto std_return;
> > >  	}
> > 
> > So we just ignore -EDQUOT when it is returned in @dontcare? I'd like
> > a comment to explain why we don't care about EDQUOT here, because
> > the next time I look at this I will have forgotten all about this...
> 
> Ok.  How about:
> 
> 	/*
> 	 * We try to get the real space reservation first, allowing for
> 	 * directory btree deletion(s) implying possible bmap insert(s).
> 	 * If we can't get the space reservation then we use 0 instead,
> 	 * and avoid the bmap btree insert(s) in the directory code by,
> 	 * if the bmap insert tries to happen, instead trimming the LAST
> 	 * block from the directory.
> 	 *
> 	 * Ignore EDQUOT and ENOSPC being returned via nospace_error
> 	 * because the directory code can handle a reservationless
> 	 * update and we don't want to prevent a user from trying to
> 	 * free space by deleting things.
> 	 */
> 	error = xfs_trans_alloc_dir(...);

Yeah, that looks good.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

[PATCH 2/2] xfs: reserve quota for target dir expansion when renaming files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

XFS does not reserve quota for directory expansion when renaming
children into a directory.  This means that we don't reject the
expansion with EDQUOT when we're at or near a hard limit, which means
that unprivileged userspace can use rename() to exceed quota.

Rename operations don't always expand the target directory, and we allow
a rename to proceed with no space reservation if we don't need to add a
block to the target directory to handle the addition.  Moreover, the
unlink operation on the source directory generally does not expand the
directory (you'd have to free a block and then cause a btree split) and
it's probably of little consequence to leave the corner case that
renaming a file out of a directory can increase its size.

As with link and unlink, there is a further bug in that we do not
trigger the blockgc workers to try to clear space when we're out of
quota.

Because rename is its own special tricky animal, we'll patch xfs_rename
directly to reserve quota to the rename transaction.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 fs/xfs/xfs_inode.c |   37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)


diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index a131bbfe74e4..8ff67b7aad53 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -3095,7 +3095,8 @@ xfs_rename(
 	bool			new_parent = (src_dp != target_dp);
 	bool			src_is_directory = S_ISDIR(VFS_I(src_ip)->i_mode);
 	int			spaceres;
-	int			error;
+	bool			retried = false;
+	int			error, space_error;
 
 	trace_xfs_rename(src_dp, target_dp, src_name, target_name);
 
@@ -3119,9 +3120,12 @@ xfs_rename(
 	xfs_sort_for_rename(src_dp, target_dp, src_ip, target_ip, wip,
 				inodes, &num_inodes);
 
+retry:
+	space_error = 0;
 	spaceres = XFS_RENAME_SPACE_RES(mp, target_name->len);
 	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, spaceres, 0, 0, &tp);
 	if (error == -ENOSPC) {
+		space_error = error;
 		spaceres = 0;
 		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, 0, 0, 0,
 				&tp);
@@ -3175,6 +3179,31 @@ xfs_rename(
 					target_dp, target_name, target_ip,
 					spaceres);
 
+	/*
+	 * Try to reserve quota to handle an expansion of the target directory.
+	 * We'll allow the rename to continue in reservationless mode if we hit
+	 * a space usage constraint.  If we trigger reservationless mode, save
+	 * the errno if there isn't any free space in the target directory.
+	 */
+	if (spaceres != 0) {
+		error = xfs_trans_reserve_quota_nblks(tp, target_dp, spaceres,
+				0, false);
+		if (error == -EDQUOT || error == -ENOSPC) {
+			if (!retried) {
+				xfs_trans_cancel(tp);
+				xfs_blockgc_free_quota(target_dp, 0);
+				retried = true;
+				goto retry;
+			}
+
+			space_error = error;
+			spaceres = 0;
+			error = 0;
+		}
+		if (error)
+			goto out_trans_cancel;
+	}
+
 	/*
 	 * Check for expected errors before we dirty the transaction
 	 * so we can return an error without a transaction abort.
@@ -3215,6 +3244,8 @@ xfs_rename(
 		 */
 		if (!spaceres) {
 			error = xfs_dir_canenter(tp, target_dp, target_name);
+			if (error == -ENOSPC && space_error)
+				error = space_error;
 			if (error)
 				goto out_trans_cancel;
 		} else {
@@ -3299,6 +3330,8 @@ xfs_rename(
 		 */
 		error = xfs_dir_createname(tp, target_dp, target_name,
 					   src_ip->i_ino, spaceres);
+		if (error == -ENOSPC && space_error)
+			error = space_error;
 		if (error)
 			goto out_trans_cancel;
 
@@ -3320,6 +3353,8 @@ xfs_rename(
 		 */
 		error = xfs_dir_replace(tp, target_dp, target_name,
 					src_ip->i_ino, spaceres);
+		if (error == -ENOSPC && space_error)
+			error = space_error;
 		if (error)
 			goto out_trans_cancel;
 

Re: [PATCH 2/2] xfs: reserve quota for target dir expansion when renaming files

[Dave Chinner]

On Wed, Mar 09, 2022 at 11:22:32AM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> XFS does not reserve quota for directory expansion when renaming
> children into a directory.  This means that we don't reject the
> expansion with EDQUOT when we're at or near a hard limit, which means
> that unprivileged userspace can use rename() to exceed quota.
> 
> Rename operations don't always expand the target directory, and we allow
> a rename to proceed with no space reservation if we don't need to add a
> block to the target directory to handle the addition.  Moreover, the
> unlink operation on the source directory generally does not expand the
> directory (you'd have to free a block and then cause a btree split) and
> it's probably of little consequence to leave the corner case that
> renaming a file out of a directory can increase its size.
> 
> As with link and unlink, there is a further bug in that we do not
> trigger the blockgc workers to try to clear space when we're out of
> quota.
> 
> Because rename is its own special tricky animal, we'll patch xfs_rename
> directly to reserve quota to the rename transaction.

Yeah, and this makes it even more tricky - the retry jumps back
across the RENAME_EXCHANGE callout/exit from xfs_rename. At some
point we need to clean up the spaghetti that rename has become.

> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  fs/xfs/xfs_inode.c |   37 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 36 insertions(+), 1 deletion(-)
> 
> 
> diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
> index a131bbfe74e4..8ff67b7aad53 100644
> --- a/fs/xfs/xfs_inode.c
> +++ b/fs/xfs/xfs_inode.c
> @@ -3095,7 +3095,8 @@ xfs_rename(
>  	bool			new_parent = (src_dp != target_dp);
>  	bool			src_is_directory = S_ISDIR(VFS_I(src_ip)->i_mode);
>  	int			spaceres;
> -	int			error;
> +	bool			retried = false;
> +	int			error, space_error;
>  
>  	trace_xfs_rename(src_dp, target_dp, src_name, target_name);
>  
> @@ -3119,9 +3120,12 @@ xfs_rename(
>  	xfs_sort_for_rename(src_dp, target_dp, src_ip, target_ip, wip,
>  				inodes, &num_inodes);
>  
> +retry:
> +	space_error = 0;
>  	spaceres = XFS_RENAME_SPACE_RES(mp, target_name->len);
>  	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, spaceres, 0, 0, &tp);
>  	if (error == -ENOSPC) {
> +		space_error = error;

nospace_error.

>  		spaceres = 0;
>  		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, 0, 0, 0,
>  				&tp);
> @@ -3175,6 +3179,31 @@ xfs_rename(
>  					target_dp, target_name, target_ip,
>  					spaceres);
>  
> +	/*
> +	 * Try to reserve quota to handle an expansion of the target directory.
> +	 * We'll allow the rename to continue in reservationless mode if we hit
> +	 * a space usage constraint.  If we trigger reservationless mode, save
> +	 * the errno if there isn't any free space in the target directory.
> +	 */
> +	if (spaceres != 0) {
> +		error = xfs_trans_reserve_quota_nblks(tp, target_dp, spaceres,
> +				0, false);
> +		if (error == -EDQUOT || error == -ENOSPC) {
> +			if (!retried) {
> +				xfs_trans_cancel(tp);
> +				xfs_blockgc_free_quota(target_dp, 0);
> +				retried = true;
> +				goto retry;
> +			}
> +
> +			space_error = error;
> +			spaceres = 0;
> +			error = 0;
> +		}
> +		if (error)
> +			goto out_trans_cancel;
> +	}
> +
>  	/*
>  	 * Check for expected errors before we dirty the transaction
>  	 * so we can return an error without a transaction abort.
> @@ -3215,6 +3244,8 @@ xfs_rename(
>  		 */
>  		if (!spaceres) {
>  			error = xfs_dir_canenter(tp, target_dp, target_name);
> +			if (error == -ENOSPC && space_error)
> +				error = space_error;

And move this error transformation to out_trans_cancel: so it only
has to be coded once.

Other than that, it's about as clean as rename allows it to be right
now.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 2/2] xfs: reserve quota for target dir expansion when renaming files

[Darrick J. Wong]

On Thu, Mar 10, 2022 at 09:05:53AM +1100, Dave Chinner wrote:
> On Wed, Mar 09, 2022 at 11:22:32AM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > XFS does not reserve quota for directory expansion when renaming
> > children into a directory.  This means that we don't reject the
> > expansion with EDQUOT when we're at or near a hard limit, which means
> > that unprivileged userspace can use rename() to exceed quota.
> > 
> > Rename operations don't always expand the target directory, and we allow
> > a rename to proceed with no space reservation if we don't need to add a
> > block to the target directory to handle the addition.  Moreover, the
> > unlink operation on the source directory generally does not expand the
> > directory (you'd have to free a block and then cause a btree split) and
> > it's probably of little consequence to leave the corner case that
> > renaming a file out of a directory can increase its size.
> > 
> > As with link and unlink, there is a further bug in that we do not
> > trigger the blockgc workers to try to clear space when we're out of
> > quota.
> > 
> > Because rename is its own special tricky animal, we'll patch xfs_rename
> > directly to reserve quota to the rename transaction.
> 
> Yeah, and this makes it even more tricky - the retry jumps back
> across the RENAME_EXCHANGE callout/exit from xfs_rename. At some
> point we need to clean up the spaghetti that rename has become.

Heh.  I did that as a cleanup to the directory code ahead of the
metadata directory tree patchset.

> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  fs/xfs/xfs_inode.c |   37 ++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 36 insertions(+), 1 deletion(-)
> > 
> > 
> > diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
> > index a131bbfe74e4..8ff67b7aad53 100644
> > --- a/fs/xfs/xfs_inode.c
> > +++ b/fs/xfs/xfs_inode.c
> > @@ -3095,7 +3095,8 @@ xfs_rename(
> >  	bool			new_parent = (src_dp != target_dp);
> >  	bool			src_is_directory = S_ISDIR(VFS_I(src_ip)->i_mode);
> >  	int			spaceres;
> > -	int			error;
> > +	bool			retried = false;
> > +	int			error, space_error;
> >  
> >  	trace_xfs_rename(src_dp, target_dp, src_name, target_name);
> >  
> > @@ -3119,9 +3120,12 @@ xfs_rename(
> >  	xfs_sort_for_rename(src_dp, target_dp, src_ip, target_ip, wip,
> >  				inodes, &num_inodes);
> >  
> > +retry:
> > +	space_error = 0;
> >  	spaceres = XFS_RENAME_SPACE_RES(mp, target_name->len);
> >  	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, spaceres, 0, 0, &tp);
> >  	if (error == -ENOSPC) {
> > +		space_error = error;
> 
> nospace_error.

Fixed.

> >  		spaceres = 0;
> >  		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, 0, 0, 0,
> >  				&tp);
> > @@ -3175,6 +3179,31 @@ xfs_rename(
> >  					target_dp, target_name, target_ip,
> >  					spaceres);
> >  
> > +	/*
> > +	 * Try to reserve quota to handle an expansion of the target directory.
> > +	 * We'll allow the rename to continue in reservationless mode if we hit
> > +	 * a space usage constraint.  If we trigger reservationless mode, save
> > +	 * the errno if there isn't any free space in the target directory.
> > +	 */
> > +	if (spaceres != 0) {
> > +		error = xfs_trans_reserve_quota_nblks(tp, target_dp, spaceres,
> > +				0, false);
> > +		if (error == -EDQUOT || error == -ENOSPC) {
> > +			if (!retried) {
> > +				xfs_trans_cancel(tp);
> > +				xfs_blockgc_free_quota(target_dp, 0);
> > +				retried = true;
> > +				goto retry;
> > +			}
> > +
> > +			space_error = error;
> > +			spaceres = 0;
> > +			error = 0;
> > +		}
> > +		if (error)
> > +			goto out_trans_cancel;
> > +	}
> > +
> >  	/*
> >  	 * Check for expected errors before we dirty the transaction
> >  	 * so we can return an error without a transaction abort.
> > @@ -3215,6 +3244,8 @@ xfs_rename(
> >  		 */
> >  		if (!spaceres) {
> >  			error = xfs_dir_canenter(tp, target_dp, target_name);
> > +			if (error == -ENOSPC && space_error)
> > +				error = space_error;
> 
> And move this error transformation to out_trans_cancel: so it only
> has to be coded once.
> 
> Other than that, it's about as clean as rename allows it to be right
> now.

Fixed.

--D

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

[PATCHSET v3 0/2] xfs: make quota reservations for directory changes

[Darrick J. Wong]

Hi all,

A couple of weeks ago, I was triaging an unexpected quota counter
failure report coming from xfs_repair after the online repair code
rebuilt a directory.  It turned out that the XFS implementation of
linkat does not reserve any quota, which means that a directory
expansion can increase the quota block count beyond the hard limit.
Similar problems exist in the unlink, rmdir, and rename code, so we'll
fix those problems at the same time.

Note: this series does not try to fix a similar bug in the swapext code,
because estimating the necessary quota reservation is very very tricky.
I wrote all that estimation code as the first part of the atomic extent
exchange patchset, so I'll leave that there.

v2: fix unlink and rename, since Dave suggested it
v3: enhance comments, clarify variable names

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=quota-reservations-5.18
---
 fs/xfs/xfs_inode.c |   79 ++++++++++++++++++++++++++++++------------------
 fs/xfs/xfs_trans.c |   86 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_trans.h |    3 ++
 3 files changed, 138 insertions(+), 30 deletions(-)

[PATCH 2/2] xfs: reserve quota for target dir expansion when renaming files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

XFS does not reserve quota for directory expansion when renaming
children into a directory.  This means that we don't reject the
expansion with EDQUOT when we're at or near a hard limit, which means
that unprivileged userspace can use rename() to exceed quota.

Rename operations don't always expand the target directory, and we allow
a rename to proceed with no space reservation if we don't need to add a
block to the target directory to handle the addition.  Moreover, the
unlink operation on the source directory generally does not expand the
directory (you'd have to free a block and then cause a btree split) and
it's probably of little consequence to leave the corner case that
renaming a file out of a directory can increase its size.

As with link and unlink, there is a further bug in that we do not
trigger the blockgc workers to try to clear space when we're out of
quota.

Because rename is its own special tricky animal, we'll patch xfs_rename
directly to reserve quota to the rename transaction.  We'll leave
cleaning up the rest of xfs_rename for the metadata directory tree
patchset.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 fs/xfs/xfs_inode.c |   33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)


diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index 766a621b970d..35a2489942e5 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -3097,7 +3097,8 @@ xfs_rename(
 	bool			new_parent = (src_dp != target_dp);
 	bool			src_is_directory = S_ISDIR(VFS_I(src_ip)->i_mode);
 	int			spaceres;
-	int			error;
+	bool			retried = false;
+	int			error, nospace_error = 0;
 
 	trace_xfs_rename(src_dp, target_dp, src_name, target_name);
 
@@ -3121,9 +3122,12 @@ xfs_rename(
 	xfs_sort_for_rename(src_dp, target_dp, src_ip, target_ip, wip,
 				inodes, &num_inodes);
 
+retry:
+	nospace_error = 0;
 	spaceres = XFS_RENAME_SPACE_RES(mp, target_name->len);
 	error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, spaceres, 0, 0, &tp);
 	if (error == -ENOSPC) {
+		nospace_error = error;
 		spaceres = 0;
 		error = xfs_trans_alloc(mp, &M_RES(mp)->tr_rename, 0, 0, 0,
 				&tp);
@@ -3177,6 +3181,31 @@ xfs_rename(
 					target_dp, target_name, target_ip,
 					spaceres);
 
+	/*
+	 * Try to reserve quota to handle an expansion of the target directory.
+	 * We'll allow the rename to continue in reservationless mode if we hit
+	 * a space usage constraint.  If we trigger reservationless mode, save
+	 * the errno if there isn't any free space in the target directory.
+	 */
+	if (spaceres != 0) {
+		error = xfs_trans_reserve_quota_nblks(tp, target_dp, spaceres,
+				0, false);
+		if (error == -EDQUOT || error == -ENOSPC) {
+			if (!retried) {
+				xfs_trans_cancel(tp);
+				xfs_blockgc_free_quota(target_dp, 0);
+				retried = true;
+				goto retry;
+			}
+
+			nospace_error = error;
+			spaceres = 0;
+			error = 0;
+		}
+		if (error)
+			goto out_trans_cancel;
+	}
+
 	/*
 	 * Check for expected errors before we dirty the transaction
 	 * so we can return an error without a transaction abort.
@@ -3423,6 +3452,8 @@ xfs_rename(
 out_release_wip:
 	if (wip)
 		xfs_irele(wip);
+	if (error == -ENOSPC && nospace_error)
+		error = nospace_error;
 	return error;
 }
 

Re: [PATCH 2/2] xfs: reserve quota for target dir expansion when renaming files

[Dave Chinner]

On Thu, Mar 10, 2022 at 01:53:39PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> XFS does not reserve quota for directory expansion when renaming
> children into a directory.  This means that we don't reject the
> expansion with EDQUOT when we're at or near a hard limit, which means
> that unprivileged userspace can use rename() to exceed quota.
> 
> Rename operations don't always expand the target directory, and we allow
> a rename to proceed with no space reservation if we don't need to add a
> block to the target directory to handle the addition.  Moreover, the
> unlink operation on the source directory generally does not expand the
> directory (you'd have to free a block and then cause a btree split) and
> it's probably of little consequence to leave the corner case that
> renaming a file out of a directory can increase its size.
> 
> As with link and unlink, there is a further bug in that we do not
> trigger the blockgc workers to try to clear space when we're out of
> quota.
> 
> Because rename is its own special tricky animal, we'll patch xfs_rename
> directly to reserve quota to the rename transaction.  We'll leave
> cleaning up the rest of xfs_rename for the metadata directory tree
> patchset.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  fs/xfs/xfs_inode.c |   33 ++++++++++++++++++++++++++++++++-
>  1 file changed, 32 insertions(+), 1 deletion(-)

LGTM.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com