Re: 10 times higher disk load with btrfs

[Stefan Priebe <s.priebe@profihost.ag>]

Am 05.01.2015 um 21:29 schrieb Mark Nelson:
>
>
> On 01/05/2015 02:20 PM, Stefan Priebe wrote:
>> Hi Sage,
>>
>> Am 05.01.2015 um 20:25 schrieb Sage Weil:
>>> On Mon, 5 Jan 2015, Stefan Priebe wrote:
>>>>
>>>> Am 05.01.2015 um 19:36 schrieb Stefan Priebe:
>>>>> Hi devs,
>>>>>
>>>>> while btrfs is now declared as stable ;-) i wanted to retest btrfs on
>>>>> our production cluster on 2 out of 54 osds. So if they crash it
>>>>> doesn't
>>>>> hurt.
>>>>>
>>>>> While if those OSDs run XFS have spikes of 20MB/s every 4-7s. The same
>>>>> OSDs after formatting them with btrfs have spikes of 190MB/s every
>>>>> 4-7s.
>>>>>
>>>>> Why does just another filesystem raises the disk load by a factor of
>>>>> 10?
>>>>
>>>> OK this seems to happen cause ceph is creating every 5s a new
>>>> subvolume /
>>>> snap. Is this really expected / needed?
>>>
>>> You can disable it with
>>>
>>>   filestore btrfs snap = false
>>>
>>> I'm curious how much this drops the load down; originally the
>>> snaps were no more expensive than a regular sync but perhaps this
>>> has changed...
>>
>> - with XFS the average write is at 9Mb/s
>> - with btrfs (filestore_btrfs_snap=true) write is at 40Mb/s
>> - with btrfs (filestore_btrfs_snap=false) write is at 20Mb/s
>
> Is that the average and not the spikes?  It looks like before the spikes
> were 20MB/s and 190MB/s?

Yes these are average values.

Spikes:
- with XFS the spike write is at 20Mb/s
- with btrfs (filestore_btrfs_snap=true) spike write is 200Mb/s
- with btrfs (filestore_btrfs_snap=false) spike is still 185Mb/s but avg 
is 1/2 (20Mb/s) see above


>
>>
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html