Re: btrfs performance, sudden drop to 0 IOPs

[Wang Shilong <wangshilong1991@gmail.com>]

Hello guys,

> 
> On Thu, Feb 12, 2015 at 05:33:41AM +0100, Kai Krakow wrote:
>> Duncan <1i5t5.duncan@cox.net> schrieb:
>> 
>>> P. Remek posted on Tue, 10 Feb 2015 18:44:33 +0100 as excerpted:
>>> 
>>>> In the test, I use --direct=1 parameter for fio which basically does
>>>> O_DIRECT on target file. The O_DIRECT should guarantee that the
>>>> filesystem cache is bypassed and IO is sent directly to the underlaying
>>>> storage. Are you saying that btrfs buffers writes despite of O_DIRECT?
>>> 
>>> I'm out of my (admin, no claims at developer) league on that.  I see
>>> someone else replied, and would defer to them on this.
>> 
>> I don't think that O_DIRECT can work efficiently on COW filesystems. It 
>> probably has a negative effect and cannot be faster as normal access. Linus 
>> itself said one time that O_DIRECT is broken and should go away, and instead 
>> cache hinting should be used.
>> 
>> Think of this: For the _unbuffered_ direct-io request to be fulfilled the 
>> file system has to go through its COW logic first which it otherwise had 
>> buffered and done in background. Bypassing the cache is probably only a 
>> side-effect of O_DIRECT, not its purpose.
> 
> Hmm, not true in btrfs, the COW logic mentioned above is nothing but to allocate
> a NEW extent, and it's not done in background.
> 
> Comparing to nocow logic, the main difference comes from 
> a) COW files' calculating checksums of the dirty data in DIO pages which nocow files don't need to.
> b) their endio handlers.
> 
> Or am I missing something?


We did benchmark Btrfs aio/dio performance before, we noticed one big differences
from COW and nocow is not only checksum but checksum cost more metadata, which will
make Btrfs performance drop suddenly for a while, because of metadata reservation.


> 
> Thanks,
> 
> -liubo
>> 
>> At least I'd try with a nocow-file for the benchmark if you still have to 
>> use O_DIRECT.
>> 
>> -- 
>> Replies to list only preferred.
>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Best Regards,
Wang Shilong