Re: Question about sendfile

[Pavel Begunkov <asml.silence@gmail.com>]

On 11/26/21 08:50, Hao Xu wrote:
> 在 2021/7/7 下午10:16, Pavel Begunkov 写道:
>> On 7/3/21 11:47 AM, Hao Xu wrote:
>>> Hi Pavel,
>>> I found this mail about sendfile in the maillist, may I ask why it's not
>>> good to have one pipe each for a io-wq thread.
>>> https://lore.kernel.org/io-uring/94dbbb15-4751-d03c-01fd-d25a0fe98e25@gmail.com/
>>
>> IIRC, it's one page allocated for each such task, which is bearable but
>> don't like yet another chunk of uncontrollable implicit state. If there
>> not a bunch of active workers, IFAIK there is no way to force them to
>> drop their pipes.
>>
>> I also don't remember the restrictions on the sendfile and what's with
>> the eternal question of "what to do if the write part of sendfile has
>> failed".
> Hi Pavel,
> Could you explain this question a little bit.., is there any special
> concern? What I thought is sendfile does what it does,when it fails,
> it will return -1 and errno is set appropriately.

I don't have much concern about this one, though interesting how
it was solved and whether you need to know the issuing task to
handle errors.

I didn't like more having uncontrollable memory, i.e. a pipe per
worker that used sendfile (IIRC it keeps 1 page), and no way to
reuse the memory or release it. In other words, a sendfile request
chooses to which worker it goes randomly. E.g. First sendfile may go
to worker 1 leaving 1 page allocated. The second sendfile goes to
worker 2, so after we have 2 pages allocated, an so on. At some
point you have N pages, where any particular one may likely be
rarely used.

Please correct me if I forgot how it works and wrong here.

>> Though, workers are now much more alike to user threads, so there
>> should be less of concern. And even though my gut feeling don't like
>> them, it may actually be useful. Do you have a good use case where
>> explicit pipes don't work well?

-- 
Pavel Begunkov