Btrfs experimental branch rebased

Btrfs experimental branch rebased

[Chris Mason]

Hello everyone,

My experimental performance fixes would occasionally trigger corruptions
under load, so I've folded an incremental fix back in and rebased the
experimental branch.  This was done to prevent git bisects later on from
landing on the bad commit and corrupting some poor tester's FS.

This only affects the experimental branch of btrfs-unstable, which is
not what you get when you clone the tree unless you specifically ask for
it.

It also now includes Josef's ENOSPC work, which is a big improvement in
enospc handling.  He has one set of fixes pending to avoid early enospc
on metadata, and I'll push that out once he sends them along.

Steve, most of the performance fixes are aimed at the mail server raid
workload, and I'd be curious to see how it compares on your hardware.

I'm afraid that mount -o noatime is required to get good numbers though,
the cow  triggered by atime updates is fairly expensive compared with
ext[34].

The experimental branch can be cloned with:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git experimental

It has four main groups of changes:

1) Create an async queue of extent allocation tree modifications.  This
allows the mods to be done outside of critical locks and it allows them
to be ordered for less seeky IO.

2) Turn btrfs_unlink into a partially async operation.  This allows
unlinks to complete in the background without the directory mutex held.

3) Josef's ENOSPC work.  This focuses on better accounting of delayed
allocations, and covers the majority of the enospc problem.

4) Stack footprint reduction.  The async extent allocation tree mods and
Josef's ENOSPC work both significantly reduce the depth of the call
chains required to do given operations by doing extent allocation tree
mods at different times.

The experimental tree also has commits that reduce stack foot print on a
number of functions, and it reorders some extent allocation tree mods
for better batching at a more stack friendly time.

-chris

Re: Btrfs experimental branch rebased

[Chris Mason]

On Wed, 2009-02-18 at 10:11 -0500, Chris Mason wrote:
> Hello everyone,
> 
> My experimental performance fixes would occasionally trigger corruptions
> under load, so I've folded an incremental fix back in and rebased the
> experimental branch.  This was done to prevent git bisects later on from
> landing on the bad commit and corrupting some poor tester's FS.
> 

This seems to be a daily operation, which isn't too surprising because
right now I'm trying to use the experimental branch as a way to
integrate Josef's work and mine frequently.

Today's rebase includes some crucial fixes for my delayed reference
modifications, including crash fixes during btrfs-vol -b.

I've got Josef's latest enospc patches, and I reordered them before the
more experimental performance updates.

The goal is to get the enospc work stable for 2.6.29-rc, and pull those
over to the main branch.

-chris

Re: Btrfs experimental branch rebased

[Steven Pratt]

Chris Mason wrote:
> Hello everyone,
>
> My experimental performance fixes would occasionally trigger corruptions
> under load, so I've folded an incremental fix back in and rebased the
> experimental branch.  This was done to prevent git bisects later on from
> landing on the bad commit and corrupting some poor tester's FS.
>
> This only affects the experimental branch of btrfs-unstable, which is
> not what you get when you clone the tree unless you specifically ask for
> it.
>
> It also now includes Josef's ENOSPC work, which is a big improvement in
> enospc handling.  He has one set of fixes pending to avoid early enospc
> on metadata, and I'll push that out once he sends them along.
>
> Steve, most of the performance fixes are aimed at the mail server raid
> workload, and I'd be curious to see how it compares on your hardware.
>   

OK, I've been out of the country without internet access for  the week.  
Will try to get some runs in on Monday when I get back to the office.

> I'm afraid that mount -o noatime is required to get good numbers though,
> the cow  triggered by atime updates is fairly expensive compared with
> ext[34].
>
>   
Ok, I'll give it  a try with and without noatime.

> The experimental branch can be cloned with:
>
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git experimental
>
> It has four main groups of changes:
>
> 1) Create an async queue of extent allocation tree modifications.  This
> allows the mods to be done outside of critical locks and it allows them
> to be ordered for less seeky IO.
>
> 2) Turn btrfs_unlink into a partially async operation.  This allows
> unlinks to complete in the background without the directory mutex held.
>
> 3) Josef's ENOSPC work.  This focuses on better accounting of delayed
> allocations, and covers the majority of the enospc problem.
>
> 4) Stack footprint reduction.  The async extent allocation tree mods and
> Josef's ENOSPC work both significantly reduce the depth of the call
> chains required to do given operations by doing extent allocation tree
> mods at different times.
>
> The experimental tree also has commits that reduce stack foot print on a
> number of functions, and it reorders some extent allocation tree mods
> for better batching at a more stack friendly time.
>
> -chris
>
>   
Steve