Re: [PATCH 0/4] implement vectored registered buffers for sendzc

[Jens Axboe <axboe@kernel.dk>]

On 10/24/24 12:13 PM, Pavel Begunkov wrote:
> On 10/24/24 19:00, Jens Axboe wrote:
>> On 10/24/24 11:56 AM, Pavel Begunkov wrote:
>>> On 10/24/24 18:19, Jens Axboe wrote:
>>>> On 10/24/24 10:06 AM, Pavel Begunkov wrote:
>>>>> On 10/24/24 16:45, Jens Axboe wrote:
> ...>>>> Seems like you're agreeing but then stating the opposite, there
>>>>> is some confusion. I'm saying that IMHO the right API wise way
>>>>> is resolving an imu at issue time, just like it's done for fixed
>>>>> files, and what your recent series did for send zc.
>>>>
>>>> Yeah early morning confusion I guess. And I do agree in principle,
>>>> though for registered buffers, those have to be registered upfront
>>>> anyway, so no confusion possible with prep vs issue there. For provided
>>>> buffers, it only matters for the legacy ones, which generally should not
>>>> be used. Doesn't change the fact that you're technically correct, the
>>>> right time to resolve them would be at issue time.
>>>
>>> I'm talking about sendmsg with iovec. Registered buffers should
>>> be registered upfront, that's right, but iovec should be copied
>>> at prep, and finally resolved into bvecs incl the imu/buffer lookup
>>> at the issue time. And those are two different points in time,
>>> maybe because of links, draining or anything else. And if they
>>> should be at different moments, there is no way to do it while
>>> copying iovec.
>>
>> Oh I totally follow, the incremental approach would only work if it can
>> be done at prep time. If at issue time, then it has to turn an existing
>> iovec array into the appropriate bvec array. And that's where you'd have
>> to do some clever bits to avoid holding both a full bvec and iovec array
>> in memory, which would be pretty wasteful/inefficient. If done at issue
> 
> Why would it be wasteful and inefficient? No more than jumping
> though that incremental infra for each chunk, doubling the size
> of the array / reallocating / memcpy'ing it, instead of a tight
> loop doing the entire conversion.

Because it would prevent doing an iovec at-the-time import, then turning
it into the desired bvec. That's one loop instead of two. You would have
the space upfront, there should be no need to realloc+memcpy. And then
there's the space concern, where the initial import is an iovec, and
then you need a bvec. For 64-bit that's fine as they take up the same
amount of space, but for 32-bit it'd make incremental importing from a
stable iovec to a bvec array a bit more tricky (and would need realloc,
unless you over-alloc'ed for the iovec array upfront).

-- 
Jens Axboe