Re: [PATCH 0/4] implement vectored registered buffers for sendzc

[Pavel Begunkov <asml.silence@gmail.com>]

On 10/24/24 18:19, Jens Axboe wrote:
> On 10/24/24 10:06 AM, Pavel Begunkov wrote:
>> On 10/24/24 16:45, Jens Axboe wrote:
...
>>> bv = kmsg->bvec;
>>> for_each_iov {
>>>      struct iovec iov;
>>>
>>>      unsafe_get_user(iov.iov_base, &user_iovec->iov_base, foo);
>>>      unsafe_get_user(iov.iov_len, &user_iovec->iov_len, foo);
>>>
>>>      import_to_bvec(bv, &iov);
>>>
>>>      user_iovec++;
>>>      bv++;
>>> }
>>>
>>> if it can be done at prep time, because then there's no need to store
>>> the iovec at all, it's already stable, just in bvecs. And this avoids
>>> overlapping iovec/bvec memory, and it avoids doing two iterations for
>>> import. Purely a poc, just tossing out ideas.
>>>
>>> But I haven't looked too closely at your series yet. In any case,
>>> whatever ends up working for you, will most likely be find for the
>>> bundled zerocopy send (non-vectored) as well, and I can just put it on
>>> top of that.
>>>
>>>> And you just made one towards delaying the imu resolution, which
>>>> is conceptually the right thing to do because of the mess with
>>>> links, just like it is with fixed files. That's why it need to
>>>> copy the iovec at the prep stage and resolve at the issue time.
>>>
>>> Indeed, prep time is certainly the place to do it. And the above
>>> incremental import should work fine then, as we won't care abot
>>> user_iovec being stable being prep.
>>
>> Seems like you're agreeing but then stating the opposite, there
>> is some confusion. I'm saying that IMHO the right API wise way
>> is resolving an imu at issue time, just like it's done for fixed
>> files, and what your recent series did for send zc.
> 
> Yeah early morning confusion I guess. And I do agree in principle,
> though for registered buffers, those have to be registered upfront
> anyway, so no confusion possible with prep vs issue there. For provided
> buffers, it only matters for the legacy ones, which generally should not
> be used. Doesn't change the fact that you're technically correct, the
> right time to resolve them would be at issue time.

I'm talking about sendmsg with iovec. Registered buffers should
be registered upfront, that's right, but iovec should be copied
at prep, and finally resolved into bvecs incl the imu/buffer lookup
at the issue time. And those are two different points in time,
maybe because of links, draining or anything else. And if they
should be at different moments, there is no way to do it while
copying iovec.

-- 
Pavel Begunkov