Re: [PATCH v2 4/4] btrfs: statfs: Use virtual chunk allocation to calculation available data space

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 1819 bytes --]



On 2020/1/3 上午12:20, Josef Bacik wrote:
> On 1/2/20 6:27 AM, Qu Wenruo wrote:
>> Although btrfs_calc_avail_data_space() is trying to do an estimation
>> on how many data chunks it can allocate, the estimation is far from
>> perfect:
>>
>> - Metadata over-commit is not considered at all
>> - Chunk allocation doesn't take RAID5/6 into consideration
>>
>> Although current per-profile available space itself is not able to
>> handle metadata over-commit itself, the virtual chunk infrastructure can
>> be re-used to address above problems.
>>
>> This patch will change btrfs_calc_avail_data_space() to do the following
>> things:
>> - Do metadata virtual chunk allocation first
>>    This is to address the over-commit behavior.
>>    If current metadata chunks have enough free space, we can completely
>>    skip this step.
>>
>> - Allocate data virtual chunks as many as possible
>>    Just like what we did in per-profile available space estimation.
>>    Here we only need to calculate one profile, since statfs() call is
>>    a relative cold path.
>>
>> Now statfs() should be able to report near perfect estimation on
>> available data space, and can handle RAID5/6 better.
>>
>> Signed-off-by: Qu Wenruo <wqu@suse.com>
> 
> Can you put a comparison of the statfs call time for the old way vs the
> new way in your changelog?  Say make a raid5 fs for example, populate it
> a little bit, and then run statfs 10 times and take the average with and
> without your patch so we can make sure there's no performance penalty. 
> You'd be surprised how many times statfs() things have caused problems
> for us in production.  Thanks,

Sure no problem.

Never considered statfs() can be a problem.
Will keep an eye on that.

Thanks,
Qu

> 
> Josef


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]