Re: some performance issue

[Mark Kampe <mark.kampe@inktank.com>]

Writes are intrinsically more expensive (in both the file
system and hardware) but it is not uncommon for individual
small random writes to substantially outperform reads even
if O_DIRECT.

If the I/O is not massively parallel, reads are going to be
processed one at a time (e.g. ~6ms seek, ~4ms latency, and
27us transfer).  Writes, however, are commonly accepted by
the drive and then queued, enabling the drive to choose among
the competing requests to significantly (e.g. 2-3x) reduce
both average seek time and rotational latency.

If the I/O is being buffered, the performance advantages for
random writes can be even greater (due to a deeper request
queue and potential request aggregation).  Isolated random
reads (with few cache hits) get a much smaller performance
boost (if any) from buffered I/O.

With massively parallel requests, however, the write
advantage should evaporate.

On 02/04/2013 09:15 AM, sheng qiu wrote:
> Hi Xiaoxi,
>
> thanks for your reply.
>
> On Mon, Feb 4, 2013 at 10:52 AM, Chen, Xiaoxi <xiaoxi.chen@intel.com> wrote:
>> I doubt your data is correct ,even the ext4 data, have you use O_DIRECT when doing the test? It's unusual to have 2X random write IOPS than random read.
>>
>
>   i did not use O_DIRECT. so page cache is used during the test.
> one thing i guess why random write is better than random read is that
> since the io request size is 4KB, so for each write request if miss on
> page cache, it will allocate a new page and write the complete 4KB
> dirty data there (since no partitional writes, no need to fetch the
> missed data from OSDs). While for read requests, it has to wait until
> the data are fetched from the OSDs.