Re: BIO_RW_SYNCIO

[Vivek Goyal <vgoyal@redhat.com>]

On Mon, Jul 26, 2010 at 05:53:40PM -0400, Mikulas Patocka wrote:
> Hi Jens
> 
[ Jens's mail id has changed. Ccing him on different mail id ]

> I found out that when I remove BIO_RW_SYNCIO from dm-kcopyd.c, performance 
> when writing to the snapshot origin is increased twice. In this workload, 
> snapshot driver reads a lot of 8k chunks from one place on the disk and 
> writes them to the other place on the same disk. Without BIO_RW_SYNCIO, it 
> is twice faster with CFQ scheduler.
> 
> What does BIO_RW_SYNCIO exactly do?  Does it attempt to decrease latency 
> and decrease throughput? (so that disk head seeks more between the source 
> and destination and it causes performance degradation) Is there some 
> general guidance, when use BIO_RW_SYNCIO and when not?

BIO_RW_SYNC marks a request as SYNC. reads are by default sync. Writes
can be marked as SYNC so that they get priority of ASYNC WRITES.

Marking writes as SYNC has advantage that it gets its own queue, it can
preempt ASYNC WRITES and CFQ also idles on the cfq queue waiting for
more requests.

I suspect if it is idling on SYNC WRITES which is a problem here.

- Is it the same thread which is submitting both read and write requests?
- Are you interleaving reads and writes. Like reading some data, then
  writting it and reading again...

I guess after writting some data, CFQ is idling on WRITE_SYNC and next
READ does not get dispatched immediately to the disk.

Is it possible to provide some 30 second blktrace of the underlying device
where CFQ is running. You can also try setting slice_idle to 0 and see if
it helps.

If setting slice_idle=0 helps you, can you please also give a try to
following patch.

https://patchwork.kernel.org/patch/113061/

Thanks
Vivek