Re: Performance results with exofs

["J. Bruce Fields" <bfields@fieldses.org>]

On Mon, Jun 07, 2010 at 09:41:29PM +0300, Boaz Harrosh wrote:
> On 06/07/2010 09:29 PM, J. Bruce Fields wrote:
> >>> On 06/07/2010 07:07 PM, Boaz Harrosh wrote:
> >>>> I did not yet publish the Document. It's stuck behind my dis-talent for
> >>>> writing and the pnfs bugs de jur.
> > 
> > Untalented writing we can fix, as long as the details are there!
> > 
> >>>>
> >>>> Basically all machines:
> >>>> - connected by a 1 GBit link.
> >>>> - All clients doing a dd write of 8GB file from /dev/zero
> >>>> - 3of8 is the special raid-groups arrangement of exofs && objlayout
> >>>>   where out of 8 devices each file is striped over 3 devices in a
> >>>>   round robin fashion. (*With a small dirty trick)
> > 
> > Random stupid questions:
> > 
> > 	- why do you think the 3of8 arrangement is scaling better than
> > 	  the 8of8?
> 
> It's a know problem with a network storage cluster. What happens is
> that with 8of8 all the clients exercise all of the nodes at the same
> time so they are clashing on the network.

OK, so if two clients are both trying to send a stripe of data to the
same OSD data at the same time, absent a switch that could somehow
afford to queue up a full stripe-unit's worth of data, packets get lost?

(Also, out of curiosity: do you know of any papers or documentation that
describe that problem in more detail?)

--b.