[PATCH] Btrfs: do not allocate chunks as agressively

[PATCH] Btrfs: do not allocate chunks as agressively

[Josef Bacik]

Swinging this pendulum back the other way.  We've been allocating chunks up
to 2% of the disk no matter how much we actually have allocated.  So instead
fix this calculation to only allocate chunks if we have more than 80% of the
space available allocated.  Please test this as it will likely cause all
sorts of ENOSPC problems to pop up suddenly.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
---
 fs/btrfs/extent-tree.c |   12 +++---------
 1 files changed, 3 insertions(+), 9 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index ce494b9..eaf1a9e 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3487,7 +3487,8 @@ static int should_alloc_chunk(struct btrfs_root *root,
 	 * and purposes it's used space.  Don't worry about locking the
 	 * global_rsv, it doesn't change except when the transaction commits.
 	 */
-	num_allocated += global_rsv->size;
+	if (sinfo->flags & BTRFS_BLOCK_GROUP_METADATA)
+		num_allocated += global_rsv->size;
 
 	/*
 	 * in limited mode, we want to have some free space up to
@@ -3501,15 +3502,8 @@ static int should_alloc_chunk(struct btrfs_root *root,
 		if (num_bytes - num_allocated < thresh)
 			return 1;
 	}
-	thresh = btrfs_super_total_bytes(root->fs_info->super_copy);
 
-	/* 256MB or 2% of the FS */
-	thresh = max_t(u64, 256 * 1024 * 1024, div_factor_fine(thresh, 2));
-	/* system chunks need a much small threshold */
-	if (sinfo->flags & BTRFS_BLOCK_GROUP_SYSTEM)
-		thresh = 32 * 1024 * 1024;
-
-	if (num_bytes > thresh && sinfo->bytes_used < div_factor(num_bytes, 8))
+	if (num_allocated + alloc_bytes < div_factor(num_bytes, 8))
 		return 0;
 	return 1;
 }
-- 
1.7.7.6

Re: [PATCH] Btrfs: do not allocate chunks as agressively

[Mitch Harder]

On Tue, Aug 14, 2012 at 3:22 PM, Josef Bacik <jbacik@fusionio.com> wrote:
> Swinging this pendulum back the other way.  We've been allocating chunks up
> to 2% of the disk no matter how much we actually have allocated.  So instead
> fix this calculation to only allocate chunks if we have more than 80% of the
> space available allocated.  Please test this as it will likely cause all
> sorts of ENOSPC problems to pop up suddenly.  Thanks,
>
> Signed-off-by: Josef Bacik <jbacik@fusionio.com>

I've been testing this patch with my multiple rsync test (On a 3.5.1
kernel merged with for-linus).

I tested without compression, and with lzo compression, and I haven't
run into any ENOSPC issues.  I still have ENOSPC issues with zlib,
with or without this patch.

I made a series of  runs with and without this patch (on an
uncompressed, newly formatted partition), and some of the results were
not what I anticipated.

1) I found that *MORE* metadata space was being allocated with this
patch than when using an unpatched baseline kernel.  The total
allocated space was exactly the same in each run (I saw a slight
variation in the amount of used Metadata).

On the unpatched baseline kernel, at the end of the run, the 'btrfs fi
df' command would show:

# btrfs fi df /mnt/benchmark/
Data: total=10.01GB, used=6.99GB
System: total=4.00MB, used=4.00KB
Metadata: total=776.00MB, used=481.38MB

With this patch applied, the 'btrfs fi df' command would show:

# btrfs fi df /mnt/benchmark/
Data: total=10.01GB, used=6.99GB
System: total=4.00MB, used=4.00KB
Metadata: total=1.01GB, used=480.94MB


2)  The multiple rsync's would run significantly faster with the patched kernel.

Unpatched baseline kernel:  Time to run 7 rysncs:  348.3 sec (+/- 9.7 sec)
Patched kernel: Time to run 7 rsyncs:  316.6 sec (+/- 6.5 sec)

Perhaps the extra allocated metadata space made things run better, or
perhaps something else was going on.

Re: [PATCH] Btrfs: do not allocate chunks as agressively

[Josef Bacik]

On Wed, Aug 15, 2012 at 11:29:11AM -0600, Mitch Harder wrote:
> On Tue, Aug 14, 2012 at 3:22 PM, Josef Bacik <jbacik@fusionio.com> wrote:
> > Swinging this pendulum back the other way.  We've been allocating chunks up
> > to 2% of the disk no matter how much we actually have allocated.  So instead
> > fix this calculation to only allocate chunks if we have more than 80% of the
> > space available allocated.  Please test this as it will likely cause all
> > sorts of ENOSPC problems to pop up suddenly.  Thanks,
> >
> > Signed-off-by: Josef Bacik <jbacik@fusionio.com>
> 
> I've been testing this patch with my multiple rsync test (On a 3.5.1
> kernel merged with for-linus).
> 
> I tested without compression, and with lzo compression, and I haven't
> run into any ENOSPC issues.  I still have ENOSPC issues with zlib,
> with or without this patch.
> 
> I made a series of  runs with and without this patch (on an
> uncompressed, newly formatted partition), and some of the results were
> not what I anticipated.
> 
> 1) I found that *MORE* metadata space was being allocated with this
> patch than when using an unpatched baseline kernel.  The total
> allocated space was exactly the same in each run (I saw a slight
> variation in the amount of used Metadata).
> 
> On the unpatched baseline kernel, at the end of the run, the 'btrfs fi
> df' command would show:
> 
> # btrfs fi df /mnt/benchmark/
> Data: total=10.01GB, used=6.99GB
> System: total=4.00MB, used=4.00KB
> Metadata: total=776.00MB, used=481.38MB
> 
> With this patch applied, the 'btrfs fi df' command would show:
> 
> # btrfs fi df /mnt/benchmark/
> Data: total=10.01GB, used=6.99GB
> System: total=4.00MB, used=4.00KB
> Metadata: total=1.01GB, used=480.94MB
> 
> 
> 2)  The multiple rsync's would run significantly faster with the patched kernel.
> 
> Unpatched baseline kernel:  Time to run 7 rysncs:  348.3 sec (+/- 9.7 sec)
> Patched kernel: Time to run 7 rsyncs:  316.6 sec (+/- 6.5 sec)
> 
> Perhaps the extra allocated metadata space made things run better, or
> perhaps something else was going on.

Well that's odd, I wonder if we're doing the limited dance more often.  Once
I've finished my fsync work I'll come back to this.  I know for sure in my
tests it's allocating chunks way too often, so I imagine your test is just
tickling a different aspect of the chunk allocator.  Thanks,

Josef