Re: Performance results with exofs

["J. Bruce Fields" <bfields@fieldses.org>]

>> On 06/07/2010 07:07 PM, Boaz Harrosh wrote:
>>> I did not yet publish the Document. It's stuck behind my dis-talent for
>>> writing and the pnfs bugs de jur.

Untalented writing we can fix, as long as the details are there!

>>>
>>> Basically all machines:
>>> - connected by a 1 GBit link.
>>> - All clients doing a dd write of 8GB file from /dev/zero
>>> - 3of8 is the special raid-groups arrangement of exofs && objlayout
>>>   where out of 8 devices each file is striped over 3 devices in a
>>>   round robin fashion. (*With a small dirty trick)

Random stupid questions:

	- why do you think the 3of8 arrangement is scaling better than
	  the 8of8?
	- Have you tried any other workloads?  (Perfectly reasonable
	  that simple write throughput would be the first thing to
	  check--I'm just curious.)

>>>
>>
>> - All tests over an *empty* filesystem.
>>
>>> [single client]
>>> 1 - osds 40MB
>>> 2 - osds 80MB
>>> 4 - osds 114MB (saturation point of the 1 Gbit link)
>>> 8 - osds 114MB
>>>
>>> [2 clients 8of8 osds]
>>> 226 MBs
>>>
>>> [4 clients 8of8 osds]
>>> 263 MBs
>>>
>>> [8 clients 8of8 osds]
>>> 252 MBs
>>>
>>> [1 clients 3of8 osds]
>>> 114 MBs
>>>
>>> [2 clients 3of8 osds *]
>>> 226 MBs
>>>
>>> [4 clients 3of8 osds *]
>>> 417 MBs

If each osd has a single gigabit interface, and you're striping to 3, of
them, isn't that 417/3 == 139 MB/s each?

(Oh, I see: you must be writing to a different file from each client,
hence you are using all osd's even if each client is only using 3?)

--b.

>>>
>>> [8 clients 3of8 osds]
>>> 405 MBs