Direct I/O + Asynchronous I/O from kernel

Direct I/O + Asynchronous I/O from kernel

[Rajat Sharma]

Hi Al,

>From a long time, I have been facing two issues in Linux kernel
related to Direct I/O and Asynchronous I/O. Although so far I have
managed to work around them, but I feel these could be easily fixed
with a kernel patch. But before working out on a patch, I thought it
is worthy to ask for an opinion about these two issues:

1. Direct I/O on Kernel Buffers:
Almost all filesystems supporting Direct I/O lock down user space
pages in memory for DMA (or network transfer) using get_user_pages (or
equivalent) function. This function strictly assumes pages to be
locked down are in user process' (calling process) address space and
hence looks for pages in all virtual memory areas (vm_area_struct)
holding the required user space address range, finds its page mapping
in page tables and locks them in memory. This function returns -EINVAL
for kernel buffers.

2. Asynchronous I/O Completion:
AIO framework maintains an IO completion event ring buffer in address
space of the requester process. When a filesystem completes AIO
request (kiocb), it calls aio_complete function which puts a
completion event in AIO ring buffer and application can consider AIO
to be done. There is no way to track down completion.

Above two limitation do not allow Direct I/O or asynchronous I/O to be
issued from kernel, using kernel buffers. As a use-case, stackable
filesystems like ecryptfs can manipulate (encrypt/decrypt) kernel
buffers and support direct I/O + Asynchronous I/O. A database engine
heavily rely on this combination and encryption has a better use case
there.

It is possible that these issues are already known and have not been
fixed for a good reason, but would be great to know anyways.

Thanks,
Rajat

Re: Direct I/O + Asynchronous I/O from kernel

[Jeff Moyer]

Rajat Sharma <fs.rajat@gmail.com> writes:

> Hi Al,
>
> From a long time, I have been facing two issues in Linux kernel
> related to Direct I/O and Asynchronous I/O. Although so far I have
> managed to work around them, but I feel these could be easily fixed
> with a kernel patch. But before working out on a patch, I thought it
> is worthy to ask for an opinion about these two issues:

Dave Kleikamp has been working on resurrecting an old patchset from Zach
Brown:

https://lkml.org/lkml/2012/3/30/317

I think there may have been another posting since then.  Anyway, I think
Dave is due to send another soon.  Right Dave?  ;-)

-Jeff

Re: Direct I/O + Asynchronous I/O from kernel

[Rajat Sharma]

> Dave Kleikamp has been working on resurrecting an old patchset from Zach
> Brown:
>
> https://lkml.org/lkml/2012/3/30/317
>

Looks relevant. Although started with loop dev as target, should be
able to extend to any kernel component, e.g. ecryptfs and nfsd. Hope
to see these patches soon in mainstream.

Thanks,
Rajat

On Fri, Sep 21, 2012 at 7:44 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
> Rajat Sharma <fs.rajat@gmail.com> writes:
>
>> Hi Al,
>>
>> From a long time, I have been facing two issues in Linux kernel
>> related to Direct I/O and Asynchronous I/O. Although so far I have
>> managed to work around them, but I feel these could be easily fixed
>> with a kernel patch. But before working out on a patch, I thought it
>> is worthy to ask for an opinion about these two issues:
>
> Dave Kleikamp has been working on resurrecting an old patchset from Zach
> Brown:
>
> https://lkml.org/lkml/2012/3/30/317
>
> I think there may have been another posting since then.  Anyway, I think
> Dave is due to send another soon.  Right Dave?  ;-)
>
> -Jeff