Re: Performance results with exofs

[Boaz Harrosh <bharrosh@panasas.com>]

On 06/07/2010 09:49 PM, J. Bruce Fields wrote:
>>
>> It's a know problem with a network storage cluster. What happens is
>> that with 8of8 all the clients exercise all of the nodes at the same
>> time so they are clashing on the network.
> 
> OK, so if two clients are both trying to send a stripe of data to the
> same OSD data at the same time, absent a switch that could somehow
> afford to queue up a full stripe-unit's worth of data, packets get lost?
> 

It's tcp they don't get lost, per-se they just get queued up. And that tcp
ramp up and all that, you know. 

We use a 64k stripe unit with say raid of 4-8 that's 256k-1M bytes in a stripe.
I don't think a network buffer that big will help at all. It'll just delay
everything more. The best is a sound statistical network strategy that'll let
the system even out overall. (Or not ...)

> (Also, out of curiosity: do you know of any papers or documentation that
> describe that problem in more detail?)
> 

Personally, I'm privileged to learn from the best here at Panasas. 

CC: Brent, Can you recommend to Bruce some good papers about raid
groups and network SAN strategies? 

> --b.

Boaz