Question about loop device DIO/AIO and FLUSH

Question about loop device DIO/AIO and FLUSH

[Amir Goldstein]

Hi Ming,

I was reading through the code of loop DIO/AIO
and I was wondering whether lo_req_flush() really
honors REQ_FLUSH by calling  vfs_fsync() on a backing file
that is open with O_DIRECT?

Shouldn't lo_req_flush() actually wait for completion of
all the in-flight loop AIO requests to complete?
Am I missing something?

Thanks,
Amir.

Re: Question about loop device DIO/AIO and FLUSH

[Christoph Hellwig]

On Thu, Jun 30, 2016 at 11:54:48PM +0300, Amir Goldstein wrote:
> Hi Ming,
> 
> I was reading through the code of loop DIO/AIO
> and I was wondering whether lo_req_flush() really
> honors REQ_FLUSH by calling  vfs_fsync() on a backing file
> that is open with O_DIRECT?
> 
> Shouldn't lo_req_flush() actually wait for completion of
> all the in-flight loop AIO requests to complete?
> Am I missing something?

Both fsync and flush semanrics only apply to I/O requests that have already
completed.

Re: Question about loop device DIO/AIO and FLUSH

[Damien Le Moal]

Christoph,

> On Jul 1, 2016, at 13:25, Christoph Hellwig <hch@lst.de> wrote:
> 
> On Thu, Jun 30, 2016 at 11:54:48PM +0300, Amir Goldstein wrote:
>> Hi Ming,
>> 
>> I was reading through the code of loop DIO/AIO
>> and I was wondering whether lo_req_flush() really
>> honors REQ_FLUSH by calling  vfs_fsync() on a backing file
>> that is open with O_DIRECT?
>> 
>> Shouldn't lo_req_flush() actually wait for completion of
>> all the in-flight loop AIO requests to complete?
>> Am I missing something?
> 
> Both fsync and flush semanrics only apply to I/O requests that have already
> completed.

Did you mean "...I/O requests that have already been fully submitted" ?
Flushing I/Os that have already completed does not seem
to make much sense. Or I am also missing something...

Best.

---
Damien Le Moal
Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

Re: Question about loop device DIO/AIO and FLUSH

[Christoph Hellwig]

On Fri, Jul 01, 2016 at 05:00:18AM +0000, Damien Le Moal wrote:
> Did you mean "...I/O requests that have already been fully submitted" ?

No.

> Flushing I/Os that have already completed does not seem
> to make much sense. Or I am also missing something...

With a volatile write cache this makes a lot of sense.  And fsync
flushes the volatile fs metadata to disk, while flush flushes the
volatile disk write cache.

Re: Question about loop device DIO/AIO and FLUSH

[Damien Le Moal]

Christoph,

> On Jul 1, 2016, at 14:04, Christoph Hellwig <hch@lst.de> wrote:
> 
> On Fri, Jul 01, 2016 at 05:00:18AM +0000, Damien Le Moal wrote:
>> Did you mean "...I/O requests that have already been fully submitted" ?
> 
> No.
> 
>> Flushing I/Os that have already completed does not seem
>> to make much sense. Or I am also missing something...
> 
> With a volatile write cache this makes a lot of sense.  And fsync
> flushes the volatile fs metadata to disk, while flush flushes the
> volatile disk write cache.

Thank you for the clarification. Makes sense.
Just to be picky though, I think that "completed I/Os" applies only
to flush in this case (flush of the volatile device cache to media).
fsync of the host cache (page cache most of the time) to the device
will issue new BIOs, and those will be waited for until completed.
Right ?

Best.

Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

Re: Question about loop device DIO/AIO and FLUSH

[Christoph Hellwig]

On Fri, Jul 01, 2016 at 05:17:28AM +0000, Damien Le Moal wrote:
> Thank you for the clarification. Makes sense.
> Just to be picky though, I think that "completed I/Os" applies only
> to flush in this case (flush of the volatile device cache to media).
> fsync of the host cache (page cache most of the time) to the device
> will issue new BIOs, and those will be waited for until completed.
> Right ?

For fsync the completed I/O at the syscall / VFS level.  Fsync only
needs to cover read/write calls that have return to the user, be that
through a synchronous call or aio completion.

Re: Question about loop device DIO/AIO and FLUSH

[Damien Le Moal]

Christoph,

> On Jul 1, 2016, at 14:32, Christoph Hellwig <hch@lst.de> wrote:
> 
> On Fri, Jul 01, 2016 at 05:17:28AM +0000, Damien Le Moal wrote:
>> Thank you for the clarification. Makes sense.
>> Just to be picky though, I think that "completed I/Os" applies only
>> to flush in this case (flush of the volatile device cache to media).
>> fsync of the host cache (page cache most of the time) to the device
>> will issue new BIOs, and those will be waited for until completed.
>> Right ?
> 
> For fsync the completed I/O at the syscall / VFS level.  Fsync only
> needs to cover read/write calls that have return to the user, be that
> through a synchronous call or aio completion.

Got it. Thanks.

Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

Re: Question about loop device DIO/AIO and FLUSH

[Ming Lei]

On Fri, Jul 1, 2016 at 4:54 AM, Amir Goldstein <amir73il@gmail.com> wrote:
> Hi Ming,
>
> I was reading through the code of loop DIO/AIO
> and I was wondering whether lo_req_flush() really
> honors REQ_FLUSH by calling  vfs_fsync() on a backing file
> that is open with O_DIRECT?

>From view of device, REQ_FLUSH just flushs the cache inside
drive to medium.

>
> Shouldn't lo_req_flush() actually wait for completion of
> all the in-flight loop AIO requests to complete?

I think it isn't needed, just like the flush handling in blk-flush.c, and
it just works for completed I/O.

> Am I missing something?
>
> Thanks,
> Amir.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-block" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question about loop device DIO/AIO and FLUSH

[Amir Goldstein]

On Fri, Jul 1, 2016 at 9:57 AM, Ming Lei <ming.lei@canonical.com> wrote:
> On Fri, Jul 1, 2016 at 4:54 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>> Hi Ming,
>>
>> I was reading through the code of loop DIO/AIO
>> and I was wondering whether lo_req_flush() really
>> honors REQ_FLUSH by calling  vfs_fsync() on a backing file
>> that is open with O_DIRECT?
>
> From view of device, REQ_FLUSH just flushs the cache inside
> drive to medium.
>
>>
>> Shouldn't lo_req_flush() actually wait for completion of
>> all the in-flight loop AIO requests to complete?
>
> I think it isn't needed, just like the flush handling in blk-flush.c, and
> it just works for completed I/O.

Thanks Ming and Christoph.
I see what I was missing.

>
>> Am I missing something?
>>
>> Thanks,
>> Amir.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-block" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html