Re: [PATCH v3 0/5] io_uring/rsrc: coalescing multi-hugepage registered buffers

[Jens Axboe <axboe@kernel.dk>]

On 5/13/24 6:09 AM, Anuj gupta wrote:
> On Mon, May 13, 2024 at 1:59?PM Chenliang Li <cliang01.li@samsung.com> wrote:
>>
>> Registered buffers are stored and processed in the form of bvec array,
>> each bvec element typically points to a PAGE_SIZE page but can also work
>> with hugepages. Specifically, a buffer consisting of a hugepage is
>> coalesced to use only one hugepage bvec entry during registration.
>> This coalescing feature helps to save both the space and DMA-mapping time.
>>
>> However, currently the coalescing feature doesn't work for multi-hugepage
>> buffers. For a buffer with several 2M hugepages, we still split it into
>> thousands of 4K page bvec entries while in fact, we can just use a
>> handful of hugepage bvecs.
>>
>> This patch series enables coalescing registered buffers with more than
>> one hugepages. It optimizes the DMA-mapping time and saves memory for
>> these kind of buffers.
>>
>> Perf diff of 8M(4*2M) hugepage fixed buffer fio test:
>>
>> fio/t/io_uring -d64 -s32 -c32 -b8388608 -p0 -B1 -F0 -n1 -O1 -r10 \
>> -R1 /dev/nvme0n1
> 
> It seems you modified t/io_uring to allocate from hugepages. It would be nice
> to mention that part here.

Yes, please just send a separate series/patch for both liburing and fio.
This series should be strictly the kernel side changes required, then
reference/link the postings for the t/io_uring and liburing test case(s)
in the cover letter.

-- 
Jens Axboe