Re: [External] Re: [PATCH] block: enable per-cpu bio cache by default

[Pavel Begunkov <asml.silence@gmail.com>]

On 10/13/25 13:58, Fengnan Chang wrote:
> Christoph Hellwig <hch@infradead.org> 于2025年10月13日周一 14:28写道：
>>
>> On Mon, Oct 13, 2025 at 01:42:47PM +0800, fengnan chang wrote:
>>>> Just set the req flag in the branch instead of unconditionally setting
>>>> it and then clearing it.
>>>
>>> clearing this flag is necessary, because bio_alloc_clone will call this in
>>> boot stage, maybe the bs->cache of the new bio is not initialized yet.
>>
>> Given that we're using the flag by default and setting it here,
>> bio_alloc_clone should not inherit it.  In fact we should probably
>> figure out a way to remove it entirely, but if that is not possible
>> it should only be set when the cache was actually used.
> 
> For now bio_alloc_clone will inherit all flag of source bio, IMO if only not
> inherit REQ_ALLOC_CACHE, it's a little strange.
> The REQ_ALLOC_CACHE flag can not remove entirely.  maybe we can
> modify like this:
> 
> if (bs->cache && nr_vecs <= BIO_INLINE_VECS) {
>      opf |= REQ_ALLOC_CACHE;
>      bio = bio_alloc_percpu_cache(bdev, nr_vecs, opf,
>      gfp_mask, bs);
>      if (bio)
>          return bio;
>      /*
>       * No cached bio available, bio returned below marked with
>       * REQ_ALLOC_CACHE to participate in per-cpu alloc cache.
>      */
> } else
>          opf &= ~REQ_ALLOC_CACHE;
> 
>>
>>>>> +     /*
>>>>> +      * Even REQ_ALLOC_CACHE is enabled by default, we still need this to
>>>>> +      * mark bio is allocated by bio_alloc_bioset.
>>>>> +      */
>>>>>        if (rq->cmd_flags & REQ_ALLOC_CACHE && (nr_vecs <= BIO_INLINE_VECS)) {
>>>>
>>>> I can't really parse the comment, can you explain what you mean?
>>>
>>> This is to tell others that REQ_ALLOC_CACHE can't be deleted here, and
>>> that this flag
>>> serves other purposes here.
>>
>> So what can't it be deleted?
> 
> blk_rq_map_bio_alloc use REQ_ALLOC_CACHE to tell whether to use
> bio_alloc_bioset or bio_kmalloc, I considered removing the flag in
> blk_rq_map_bio_alloc, but then there would have to be the introduction
> of a new flag like  REQ_xx. So I keep this and comment.

That can likely be made unconditional as well. Regardless of that,
it can't be removed without additional changes because it's used to
avoid de-allocating into the pcpu cache requests that wasn't
allocated for it. i.e.

if (bio->bi_opf & REQ_ALLOC_CACHE)
	bio_put_percpu_cache(bio);
else
	bio_free(bio);

Without it under memory pressure you can end up in a situation
where bios are put into pcpu caches of other CPUs and can't be
reallocated by the current CPU, effectively loosing the mempool
forward progress guarantees. See:

commit 759aa12f19155fe4e4fb4740450b4aa4233b7d9f
Author: Pavel Begunkov <asml.silence@gmail.com>
Date:   Wed Nov 2 15:18:20 2022 +0000

     bio: don't rob starving biosets of bios

-- 
Pavel Begunkov