Expense of read_iter

Matthew Wilcox
2021-01-08T20:36:40Z

On Thu, Jan 07, 2021 at 08:15:41AM -0500, Mikulas Patocka wrote:
> I'd like to ask about this piece of code in __kernel_read:
> if (unlikely(!file->f_op->read_iter || file->f_op->read))
> return warn_unsupported...
> and __kernel_write:
> if (unlikely(!file->f_op->write_iter || file->f_op->write))
> return warn_unsupported...
>
> - It exits with an error if both read_iter and read or write_iter and
> write are present.
>
> I found out that on NVFS, reading a file with the read method has 10%
> better performance than the read_iter method. The benchmark just reads the
> same 4k page over and over again - and the cost of creating and parsing
> the kiocb and iov_iter structures is just that high.

Which part of it is so expensive? Is it worth, eg adding an iov_iter
type that points to a single buffer instead of a single-member iov?

+++ b/include/linux/uio.h
@@ -19,6 +19,7 @@ struct kvec {

enum iter_type {
/* iter types */
+ ITER_UBUF = 2,
ITER_IOVEC = 4,
ITER_KVEC = 8,
ITER_BVEC = 16,
@@ -36,6 +36,7 @@ struct iov_iter {
size_t iov_offset;
size_t count;
union {
+ void __user *buf;
const struct iovec *iov;
const struct kvec *kvec;
const struct bio_vec *bvec;

and then doing all the appropriate changes to make that work.