Re: simple Chunk allocator like calculation to replace Factor based calculation

[Qu Wenruo <wqu@suse.com>]

On 2020/10/1 下午11:36, Wang Yugui wrote:
> Hi, Qu Wenruo
> 
> Chunk allocator like calculation will get the right value, but it is
> slow for big file system such as 500T.

Nope, the ballon allocator doesn't have any size limit, thus it will try
to use as much space as possible in a single run.

If the 500T fs only has say 100T used, and the remaining 400T is split
into, say 2 parts, then just two small run would finish.

On the other hand, if the 500T is mostly used, only 100T unallocated and
the 100T is in a big unallocate chunk (under most cases it's true), then
just one allocation run is enough.

So in short, it's unrelated to the fs, but how fragmented the
unallocated space is.
And normally unallocated space is not fragmented at all, thus it's very
speedy.

> 
> we can call this 'Simple Chunk allocator' as 'free_space_by_min_profile',
> it is good enough to prevent over commit, and fast enough too.
> 
> N = .devs_min + .nparity
> 
> (N-th big free space) * (.devs_min) / (.devs_min + .nparity)  / .ncopies.

Nope. Check the basic unbalance case of 10T + 1T raid1, and cases like 6
disk RAID10 with 10T, 10T, 5T, 5T, 1T, 1T.

Last but not the least, please send such feedback to the mail list.
Open-source doesn't only mean source open, but also open discussion.

Without an open environment to discuss, it will be lame open-source
practice just like almost all Chinese companies do, publish a tarball
and call it a day. That's not open at all.

Thanks,
Qu

> 
> Best Regards
> 王玉贵
> 2020/10/01
> 
>> Hi, Qu Wenruo
>>
>> https://patchwork.kernel.org/patch/11810913/
>> is a good job.
>>
>> We have two types of estimation:
>> - Factor based calculation
>> - Chunk allocator like calculation
>>
>> In factor, we can have a simple Chunk allocator like calculation to
>> replace Factor based calculation to prevent overcommit.
>>
>> -Simple Chunk allocator like calculation
>> we just use the top devs_min devices to get the free space.
>> we use this value to prevent overcommit.
>>
>> this Simple Chunk allocator like calculation is always <= Chunk allocator like calculation.
>>
>> for  this example:
>>   devid 1 unallocated:	1T
>>   devid 2 unallocated:  1T
>>   devid 3 unallocated:	10T
>>   devid 4 unallocated:	5T
>>
>> RAID1 of Simple Chunk allocator like calculation :	5T(top 2 device)
>> RAID5 of Simple Chunk allocator like calculation :	2T(top 3 device)
>>
>> Simple Chunk allocator like calculation is fast for big file system just like
>> Factor based calculation.
>>
>> Best Regards
>> 王玉贵
>> 2020/10/01
>>
>> --------------------------------------
>> 北京京垓科技有限公司
>> 王玉贵	wangyugui@e16-tech.com
>> 电话：+86-136-71123776
> 
> --------------------------------------
> 北京京垓科技有限公司
> 王玉贵	wangyugui@e16-tech.com
> 电话：+86-136-71123776
>