Re: Performance results with exofs

Boaz Harrosh
2010-06-08T05:26:48Z

On 06/07/2010 09:49 PM, J. Bruce Fields wrote:
>>
>> It's a know problem with a network storage cluster. What happens is
>> that with 8of8 all the clients exercise all of the nodes at the same
>> time so they are clashing on the network.
>
> OK, so if two clients are both trying to send a stripe of data to the
> same OSD data at the same time, absent a switch that could somehow
> afford to queue up a full stripe-unit's worth of data, packets get lost?
>

It's tcp they don't get lost, per-se they just get queued up. And that tcp
ramp up and all that, you know.

We use a 64k stripe unit with say raid of 4-8 that's 256k-1M bytes in a stripe.
I don't think a network buffer that big will help at all. It'll just delay
everything more. The best is a sound statistical network strategy that'll let
the system even out overall. (Or not ...)

> (Also, out of curiosity: do you know of any papers or documentation that
> describe that problem in more detail?)
>

Personally, I'm privileged to learn from the best here at Panasas.

CC: Brent, Can you recommend to Bruce some good papers about raid
groups and network SAN strategies?

> --b.

Boaz