Re: Performance results with exofs

Benny Halevy
2010-06-08T06:54:58Z

On 2010-06-07 21:49, J. Bruce Fields wrote:
> On Mon, Jun 07, 2010 at 09:41:29PM +0300, Boaz Harrosh wrote:
>> On 06/07/2010 09:29 PM, J. Bruce Fields wrote:
>>>>> On 06/07/2010 07:07 PM, Boaz Harrosh wrote:
>>>>>> I did not yet publish the Document. It's stuck behind my dis-talent for
>>>>>> writing and the pnfs bugs de jur.
>>>
>>> Untalented writing we can fix, as long as the details are there!
>>>
>>>>>>
>>>>>> Basically all machines:
>>>>>> - connected by a 1 GBit link.
>>>>>> - All clients doing a dd write of 8GB file from /dev/zero
>>>>>> - 3of8 is the special raid-groups arrangement of exofs && objlayout
>>>>>> where out of 8 devices each file is striped over 3 devices in a
>>>>>> round robin fashion. (*With a small dirty trick)
>>>
>>> Random stupid questions:
>>>
>>> - why do you think the 3of8 arrangement is scaling better than
>>> the 8of8?
>>
>> It's a know problem with a network storage cluster. What happens is
>> that with 8of8 all the clients exercise all of the nodes at the same
>> time so they are clashing on the network.
>
> OK, so if two clients are both trying to send a stripe of data to the
> same OSD data at the same time, absent a switch that could somehow
> afford to queue up a full stripe-unit's worth of data, packets get lost?
>
> (Also, out of curiosity: do you know of any papers or documentation that
> describe that problem in more detail?)
>

A good place to start would be
http://www.pdl.cmu.edu/Incast/

Benny

> --b.
> --