Re: Updated performance results

Chris Mason
2009-09-14T23:13:03Z

On Mon, Sep 14, 2009 at 04:41:48PM -0500, Steven Pratt wrote:
> Chris Mason wrote:
> >On Fri, Sep 11, 2009 at 04:35:50PM -0500, Steven Pratt wrote:
> >>Chris Mason wrote:
> >>>On Mon, Aug 31, 2009 at 12:49:13PM -0500, Steven Pratt wrote:
> >>>>Better late than never. Finally got this finished up. Mixed bag on
> >>>>this one. BTRFS lags significantly on single threaded. Seems
> >>>>unable to keep IO outstanding to the device. Less that 60% busy on
> >>>>the DM device, compared to 97%+ for all other filesystems.
> >>>>nodatacow helps out, increasing utilization to about 70%, but still
> >>>>trails by a large margin.
> >>>Hi Steve,
> >>>
> >>>Jens Axboe did some profiling on his big test rig and I think we found
> >>>the biggest CPU problems. The end result is now setting in the master
> >>>branch of the btrfs-unstable repo.
> >>>
> >>>On his boxes, btrfs went from around 400MB/s streaming writes to 1GB/s
> >>>limit, and we're now tied with XFS while using less CPU time.
> >>>
> >>>Hopefully you will see similar results ;)
> >>Hmmm, well no I didn't. Throughputs at 1 and 128 threads are pretty
> >>much unchanged, although I do see a good CPU savings on the 128
> >>thread case (with cow). For 16 threads we actually regressed with
> >>cow enabled.
> >>
> >>Results are here:
> >>
> >>http://btrfs.boxacle.net/repository/raid/large_create_test/write-test/1M_odirect_create.html
> >>
> >>I'll try to look more into this next week.
> >>
> >
> >Hmmm, Jens was benchmarking buffered writes, but he was also testing on
> >his new per-bdi write back code. If your next run could be buffered
> >instead of O_DIRECT, I'd be curious to see the results.
> >
> Buffered does look a lot better. I don't have a btrfs baseline
> before these latest changes for this exact workload, but these
> results are not bad at all. With cow, beats just about everything
> except XFS, and with nocow simply screams. CPU consumption looks
> good as well. I'll probably give the full set of tests a run
> tonight.

Wow, good news at last ;) For the oops, try the patch below (I need to
push it out, but I think it'll help).

I'll try to figure out the O_DIRECT problems.

-chris

diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
index 6ea5cd0..ba28742 100644
--- a/fs/btrfs/async-thread.c
+++ b/fs/btrfs/async-thread.c
@@ -177,7 +177,7 @@ static int try_worker_shutdown(struct btrfs_worker_thread *worker)
int freeit = 0;

spin_lock_irq(&worker->lock);
- spin_lock_irq(&worker->workers->lock);
+ spin_lock(&worker->workers->lock);
if (worker->workers->num_workers > 1 &&
worker->idle &&
!worker->working &&
@@ -188,7 +188,7 @@ static int try_worker_shutdown(struct btrfs_worker_thread *worker)
list_del_init(&worker->worker_list);
worker->workers->num_workers--;
}
- spin_unlock_irq(&worker->workers->lock);
+ spin_unlock(&worker->workers->lock);
spin_unlock_irq(&worker->lock);

if (freeit)