From: Jens Axboe <axboe@kernel.dk>
To: Qu Wenruo <quwenruo.btrfs@gmx.com>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: Weird delay introduced in v6.16-rc only, possible regression
Date: Tue, 1 Jul 2025 10:30:56 -0600 [thread overview]
Message-ID: <20f07b3b-194a-4dfe-936e-0f159bf44485@kernel.dk> (raw)
In-Reply-To: <e597d8c6-ed77-47ae-b030-1016727d6abe@gmx.com>
On 6/29/25 3:45 PM, Qu Wenruo wrote:
> Hi,
>
> Recently I'm hitting a very weird delay when doing development inside a x86_64 VM.
>
> The dmesg shows the delay (10+ sec) between virtio blk and device-mapper:
>
> [ 3.651377] virtio_blk virtio4: 10/0/0 default/read/poll queues
> [ 3.653075] virtio_scsi virtio2: 10/0/0 default/read/poll queues
> [ 3.670269] virtio_blk virtio4: [vda] 83886080 512-byte logical blocks (42.9 GB/40.0 GiB)
> [ 3.672096] scsi host6: Virtio SCSI HBA
> [ 3.708452] vda: vda1 vda2
> [ 3.711073] virtio_blk virtio5: 10/0/0 default/read/poll queues
> [ 3.729879] virtio_blk virtio5: [vdb] 167772160 512-byte logical blocks (85.9 GB/80.0 GiB)
> [ 3.737535] virtio_blk virtio8: 10/0/0 default/read/poll queues
> [ 3.747045] virtio_blk virtio8: [vdc] 83886080 512-byte logical blocks (42.9 GB/40.0 GiB)
> [ 17.453833] device-mapper: uevent: version 1.0.3
> [ 17.455689] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev
> :: performing fsck on '/dev/os/root'
> /dev/mapper/os-root: clean, 240299/1048576 files, 3372218/4194304 blocks
> :: mounting '/dev/os/root' on real root
> [ 17.871671] EXT4-fs (dm-0): mounted filesystem 00a85626-d289-4817-8183-ee828e221f76 r/w with ordered data mode. Quota mode: none.
>
> The VM is running kernel based on upstream commit 78f4e737a53e ("Merge tag 'for-6.16/dm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm"), with a lot of extra btrfs patches.
>
>
> The v6.15 kernel from Archlinux is totally fine without any delay.
>
> The v6.16-rc kernel may have some different configs, but the config is used for a long long time, way before v6.15, so it looks like it's something in the v6.16 cycle causing problems.
>
> I can definitely do a bisection, but any clue would be appreciated.
Probably a good idea to go ahead with a bisect to help pin it down.
--
Jens Axboe
next prev parent reply other threads:[~2025-07-01 16:30 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-29 21:45 Weird delay introduced in v6.16-rc only, possible regression Qu Wenruo
2025-07-01 16:30 ` Jens Axboe [this message]
2025-07-01 22:07 ` Qu Wenruo
2025-07-01 22:11 ` Jens Axboe
2025-07-02 3:52 ` Qu Wenruo
2025-07-02 4:14 ` Qu Wenruo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20f07b3b-194a-4dfe-936e-0f159bf44485@kernel.dk \
--to=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=quwenruo.btrfs@gmx.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox