Re: RBD speed vs threads

[Stefan Priebe <s.priebe@profihost.ag>]

Am 15.06.2012 14:03, schrieb Mark Nelson:
> On 06/15/2012 12:56 AM, Stefan Priebe - Profihost AG wrote:
> Let me preface this by saying that I haven't specifically read through
> the rados bench code.  Having said that, the basic idea here is that you
> have a pipeline where a request is sent from the client to an OSD.  If
> you specify "-t 1", the client will only send a single request at a
> time, which means that the entire process is serial and you are entirely
> latency bound.  Now think about what happens when the client sends the
> request.  Before client gets an acknowledgement, the request must:
>
> 1) Go through client side processing.
> 2) Travel over the IP network to the destination OSD.
> 3) Go through all of the queue processing code on the OSD.
> 4a) Write the data to the journal (Or the faster of the journal/data
> disk when using btrfs.  Note: The journal writes may stall if the data
> disk is too slow and the journal has gotten sufficiently ahead of it)
> 4b) Complete replication to other OSDs based on the pool's replication
> level and the placement group the data gets put in. (basically steps
> 1,2,3,4a and 5 all over again with the OSD as the client).
> 5) Send the Ack back to the client over the IP network
>
> If only one request is sent at a time, most of the hardware will sit
> idle while the request is making it's way through the pipeline.  If you
> have multiple concurrent requests, the OSD(s) can better utilize all of
> the hardware (ie some requests can be coming in over the network, while
> others can be writing to disk, while others can be replicating).
>
> You can probably imagine that once you have multiple OSDs on multiple
> Nodes, having concurrent requests in flight help you even more.

Thanks for your explanation.

Stefan