[LSF/MM/BPF TOPIC] Improving iov_iter - and replacing scatterlists

David Howells
2025-01-17T21:17:03Z

Hi,

I'd like to propose a discussion of two things: firstly, how might we improve
iov_iter and, secondly, would it be possible to replace scatterlists.

[*] First: Improvements to iov_iter.

I'm trying to get rid of ITER_XARRAY; xarrays are too unstable (in the sense
that their contents can shift under you).

I'm trying to replace that with ITER_FOLIOQ instead. This is a segmented list
of folios - so it can only hold folios, but has infinite capacity. How easy
would it be to extend this to be able to handle some other types of page, such
as anon pages or stuff that's been spliced out of network receive buffers?

Would it make sense to be able to have a chain of disparate types of object?
Say a couple of kmalloc'd buffers, followed by a number of folios, followed by
another kmalloc'd buffer and mark them such we know which ones can be DMA'd
and which ones must be copied.

Currently, the core iteration functions in linux/iov_iter.h each handle a
specific type of iterable. I wonder how much performance difference it would
make to have each item in a list have its own type. Now, I know, "try it and
see" is a valid suggestion here.

Rumour has it that John Hubbard may be working along similar lines, possibly
just in the area of bio_vecs and ITER_BVEC.

[*] Second: Can we replace the uses of scatterlist with iov_iter and reduce
the number of iterator classes we have?

One reason I'd like to do this is we have iov_iter at user end of the I/O
stack, and it percolates down to various depths. For network filesystems, for
example, the socket API takes iov_iters, so we want to plumb iov_iters all the
way down if we can - and have the filesystem know at little as possible about
folios and pages if we can manage it.

However, one thing that particularly stands out for me is that network
filesystems often want to use the crypto API - and that means allocating and
constructing a scatterlist to talk to the crypto API. Having spent some time
looking at crypto API, in most places iteration functions are used that mean
that changing to use an iov_iter might not be so hard.

That said, one thing that is made use of occasionally with scatterlists is the
ability to chain something on the front. That's significantly harder to do
with iov_iter.

That that said, one reason it's hard to modify the list attached to an
iterator is that we allow iterators to be rewound, using state stored in the
list to go backwards. I wonder if it might be possible to get rid of
iov_iter_revert() and use iterator copying instead.

David