From: Jens Axboe <axboe@kernel.dk>
To: Ming Lei <ming.lei@redhat.com>,
Hazem Mohamed Abuelfotoh <abuehaze@amazon.com>
Cc: stable@vger.kernel.org,
"kernel test robot" <oliver.sang@intel.com>,
"Hagar Hemdan" <hagarhem@amazon.com>,
"Shaoying Xu" <shaoyi@amazon.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Jason Wang" <jasowang@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Stefan Hajnoczi" <stefanha@redhat.com>,
"Eugenio Pérez" <eperezma@redhat.com>,
"Xuan Zhuo" <xuanzhuo@linux.alibaba.com>,
"Keith Busch" <kbusch@kernel.org>,
"Christoph Hellwig" <hch@lst.de>,
"Sagi Grimberg" <sagi@grimberg.me>,
linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
virtualization@lists.linux.dev, linux-nvme@lists.infradead.org,
linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH] Revert "block: don't reorder requests in blk_add_rq_to_plug"
Date: Wed, 11 Jun 2025 09:26:41 -0600 [thread overview]
Message-ID: <afb8792b-a78d-4886-bf9a-23121510dec5@kernel.dk> (raw)
In-Reply-To: <aEmcZLGtQFWMDDXZ@fedora>
On 6/11/25 9:10 AM, Ming Lei wrote:
> On Wed, Jun 11, 2025 at 12:14:54PM +0000, Hazem Mohamed Abuelfotoh wrote:
>> This reverts commit e70c301faece15b618e54b613b1fd6ece3dd05b4.
>>
>> Commit <e70c301faece> ("block: don't reorder requests in
>> blk_add_rq_to_plug") reversed how requests are stored in the blk_plug
>> list, this had significant impact on bio merging with requests exist on
>> the plug list. This impact has been reported in [1] and could easily be
>> reproducible using 4k randwrite fio benchmark on an NVME based SSD without
>> having any filesystem on the disk.
>>
>> My benchmark is:
>>
>> fio --time_based --name=benchmark --size=50G --rw=randwrite \
>> --runtime=60 --filename="/dev/nvme1n1" --ioengine=psync \
>> --randrepeat=0 --iodepth=1 --fsync=64 --invalidate=1 \
>> --verify=0 --verify_fatal=0 --blocksize=4k --numjobs=4 \
>> --group_reporting
>>
>> On 1.9TiB SSD(180K Max IOPS) attached to i3.16xlarge AWS EC2 instance.
>>
>> Kernel | fio (B.W MiB/sec) | I/O size (iostat)
>> --------------+---------------------+--------------------
>> 6.15.1 | 362 | 2KiB
>> 6.15.1+revert | 660 (+82%) | 4KiB
>> --------------+---------------------+--------------------
>
> I just run one quick test in my test VM, but can't reproduce it.
>
> Also be curious, why does writeback produce so many 2KiB bios?
I was pondering that too, sounds like a misconfiguration of sorts. But
even without that, in a quick synthetic test here locally, I do see a
lot of missed merges that is solved with the alternative patch I sent
out. I strongly suspect it'll fix this issue too.
--
Jens Axboe
next prev parent reply other threads:[~2025-06-11 15:26 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-11 12:14 [PATCH] Revert "block: don't reorder requests in blk_add_rq_to_plug" Hazem Mohamed Abuelfotoh
2025-06-11 12:56 ` Jens Axboe
2025-06-11 15:15 ` Mohamed Abuelfotoh, Hazem
2025-06-11 15:10 ` Ming Lei
2025-06-11 15:24 ` Mohamed Abuelfotoh, Hazem
2025-06-11 15:26 ` Jens Axboe [this message]
2025-06-11 15:24 ` Bart Van Assche
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=afb8792b-a78d-4886-bf9a-23121510dec5@kernel.dk \
--to=axboe@kernel.dk \
--cc=abuehaze@amazon.com \
--cc=eperezma@redhat.com \
--cc=hagarhem@amazon.com \
--cc=hch@lst.de \
--cc=jasowang@redhat.com \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=ming.lei@redhat.com \
--cc=mst@redhat.com \
--cc=oliver.sang@intel.com \
--cc=pbonzini@redhat.com \
--cc=sagi@grimberg.me \
--cc=shaoyi@amazon.com \
--cc=stable@vger.kernel.org \
--cc=stefanha@redhat.com \
--cc=virtualization@lists.linux.dev \
--cc=xuanzhuo@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox