Re: RBD fio Performance concerns

[Mark Kampe <mark.kampe@inktank.com>]

Sequential is faster than random on a disk, but we are not
doing I/O to a disk, but a distributed storage cluster:

   small random operations are striped over multiple objects and
   servers, and so can proceed in parallel and take advantage of
   more nodes and disks.  This parallelism can overcome the added
   latencies of network I/O to yield very good throughput.

   small sequential read and write operations are serialized on
   a single server, NIC, and drive.  This serialization eliminates
   parallelism, and the network and other queuing delays are no
   longer compensated for.

This striping is a good idea for the small random I/O that is
typical of the way Linux systems talk to their disks.  But for
other I/O patterns, it is not optimal.

On 11/21/2012 01:47 PM, Sébastien Han wrote:
> Hi Mark,
>
> Well the most concerning thing is that I have 2 Ceph clusters and both
> of them show better rand than seq...
> I don't have enough background to argue on your assomptions but I
> could try to skrink my test platform to a single OSD and how it
> performs. We keep in touch on that one.
>
> But it seems that Alexandre and I have the same results (more rand
> than seq), he has (at least) one cluster and I have 2. Thus I start to
> think that's not an isolated issue.
>
> Is it different for you? Do you usually get more seq IOPS from an RBD
> thant rand?
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html