Re: 10 times higher disk load with btrfs

[Stefan Priebe - Profihost AG <s.priebe@profihost.ag>]

Hi,
Am 06.01.2015 um 04:44 schrieb Alexandre DERUMIER:
> Hi Stefan,
> 
> Do you see a difference if you force filestore journal writeahead for btrfs instead parrallel ?
> 
> filestore journal writeahead = 1
> filestore journal parallel = 0

i already tested filestore btrfs snap = false which automatically
disabled parallel write.

Stefan

> ----- Mail original -----
> De: "Stefan Priebe" <s.priebe@profihost.ag>
> À: "Mark Nelson" <mnelson@redhat.com>, "Sage Weil" <sage@newdream.net>
> Cc: "ceph-devel" <ceph-devel@vger.kernel.org>
> Envoyé: Lundi 5 Janvier 2015 21:33:22
> Objet: Re: 10 times higher disk load with btrfs
> 
> Am 05.01.2015 um 21:29 schrieb Mark Nelson: 
>>
>>
>> On 01/05/2015 02:20 PM, Stefan Priebe wrote: 
>>> Hi Sage, 
>>>
>>> Am 05.01.2015 um 20:25 schrieb Sage Weil: 
>>>> On Mon, 5 Jan 2015, Stefan Priebe wrote: 
>>>>>
>>>>> Am 05.01.2015 um 19:36 schrieb Stefan Priebe: 
>>>>>> Hi devs, 
>>>>>>
>>>>>> while btrfs is now declared as stable ;-) i wanted to retest btrfs on 
>>>>>> our production cluster on 2 out of 54 osds. So if they crash it 
>>>>>> doesn't 
>>>>>> hurt. 
>>>>>>
>>>>>> While if those OSDs run XFS have spikes of 20MB/s every 4-7s. The same 
>>>>>> OSDs after formatting them with btrfs have spikes of 190MB/s every 
>>>>>> 4-7s. 
>>>>>>
>>>>>> Why does just another filesystem raises the disk load by a factor of 
>>>>>> 10? 
>>>>>
>>>>> OK this seems to happen cause ceph is creating every 5s a new 
>>>>> subvolume / 
>>>>> snap. Is this really expected / needed? 
>>>>
>>>> You can disable it with 
>>>>
>>>> filestore btrfs snap = false 
>>>>
>>>> I'm curious how much this drops the load down; originally the 
>>>> snaps were no more expensive than a regular sync but perhaps this 
>>>> has changed... 
>>>
>>> - with XFS the average write is at 9Mb/s 
>>> - with btrfs (filestore_btrfs_snap=true) write is at 40Mb/s 
>>> - with btrfs (filestore_btrfs_snap=false) write is at 20Mb/s 
>>
>> Is that the average and not the spikes? It looks like before the spikes 
>> were 20MB/s and 190MB/s? 
> 
> Yes these are average values. 
> 
> Spikes: 
> - with XFS the spike write is at 20Mb/s 
> - with btrfs (filestore_btrfs_snap=true) spike write is 200Mb/s 
> - with btrfs (filestore_btrfs_snap=false) spike is still 185Mb/s but avg 
> is 1/2 (20Mb/s) see above 
> 
> 
>>
>>>
>>> Stefan 
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in 
>>> the body of a message to majordomo@vger.kernel.org 
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html