Re: btrfs kernel workqueues performance regression

[Chris Mason <clm@fb.com>]

On 07/15/2014 11:26 AM, Morten Stevens wrote:
> Hi,
> 
> I see that btrfs is using kernel workqueues since linux 3.15. After
> some tests I noticed performance regressions with fs_mark.
> 
> mount options: rw,relatime,compress=lzo,space_cache
> 
> fs_mark on Kernel 3.14.9:
> 
> # fs_mark  -d  /mnt/btrfs/fsmark  -D  512  -t  16  -n  4096  -s  51200  -L5  -S0
> FSUse%        Count         Size    Files/sec     App Overhead
>      1        65536        51200      17731.4           723894
>      1       131072        51200      16832.6           685444
>      1       196608        51200      19604.5           652294
>      1       262144        51200      18663.6           630067
>      1       327680        51200      20112.2           692769
> 
> The results are really nice! compress=lzo performs very good.
> 
> fs_mark after upgrading to Kernel 3.15.4:
> 
> # fs_mark  -d  /mnt/btrfs/fsmark  -D  512  -t  16  -n  4096  -s  51200  -L5  -S0
> FSUse%        Count         Size    Files/sec     App Overhead
>      0        65536        51200      10718.1           749540
>      0       131072        51200       8601.2           853050
>      0       196608        51200      11623.2           558546
>      0       262144        51200      11534.2           536342
>      0       327680        51200      11167.4           578562
> 
> That's really a big performance regression :(
> 
> What do you think? It's easy to reproduce with fs_mark.

I wasn't able to trigger regressions here when we first merged it, but I
was sure that something would pop up.  fs_mark is sensitive to a few
different factors outside just the worker threads, so it could easily be
another change as well.

With 16 threads, the btree locking also has a huge impact, and we've
made change there too.

I'll reproduce here, thanks for sending it in.

-chris