[Cluster-devel] [PATCH 0/2] GFS2: inplace_reserve performance improvements

[Bob Peterson <rpeterso@redhat.com>]

----- Original Message -----
> While testing GFS2 as a storage repository for virtual machines we
> discovered a number of scenarios where the performance was being
> pathologically poor.
> 
> The scenarios are simplfied to the following -
> 
>   * On a single host in the cluster grow a number of files to a
>     significant proportion of the filesystems LUN size, exceeding the
>     hosts preferred resource group allocation. This can be replicated
>     by using fio and writing to 20 different files with a script like

Hi Mark, Tim and all,

The performance problems with rgrp contention are well known, and have
been for a very long time.

In rhel6 it's not as big of a problem because rhel6 gfs2 uses "try locks"
which distributes different processes to unique rgrps, thus keeping
them from contending. However, it results in file system fragmentation
that tends to catch up with you later.

I posted a different patch set to solve the problem a different way
by trying to keep track of both Inter-node and Intra-node contention,
and redistributed rgrps accordingly. It was similar to your first patch,
but used a more predictable distribution, whereas yours is random.
It worked very well, but it ultimately got rejected by Steve Whitehouse
in favor of a better approach:

Our current plan is to allow rgrps to be shared among many processes on
a single node. This alleviates the contention, improves throughput
and performance, and fixes the "favoritism" problems gfs2 has today.
In other words, it's better than just redistributing the rgrps.

I did a proof-of-concept set of patches and saw pretty good performance
numbers and "fairness" among simultaneous writers. I posted that a few
months ago.

Your patch would certainly work, and random distribution of rgrps
would definitely gain performance, just as the Orlov algorithm does,
however, I still want to pursue what Steve suggested.

My patch set for this still needs some work because I found some
bugs with how things are done, so it'll take time to get working
properly.

Regards,

Bob Peterson
Red Hat File Systems