From: Ming Lei <ming.lei@redhat.com>
To: Dusty Mabe <dusty@dustymabe.com>
Cc: Jens Axboe <axboe@kernel.dk>,
linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
hch@lst.de, linux-raid@vger.kernel.org
Subject: Re: regression caused by block: freeze the queue earlier in del_gendisk
Date: Thu, 1 Sep 2022 15:06:08 +0800 [thread overview]
Message-ID: <YxBZ4BBjxvAkvI2A@T590> (raw)
In-Reply-To: <017845ae-fbae-70f6-5f9e-29aff2742b8c@dustymabe.com>
Hi Dusty,
On Fri, Aug 26, 2022 at 12:15:22PM -0400, Dusty Mabe wrote:
> Hey All,
>
> I think I've found a regression introduced by:
>
> a09b314 o block: freeze the queue earlier in del_gendisk
>
> In Fedora CoreOS we have tests that set up RAID1 on the /boot/ and /root/ partitions
> and then subsequently removes one of the disks to simulate a failure. Sometime recently
Do you have test case which doesn't need raid1 over /boot or /root? such
as by create raid1 over two disks, then mount & remove one of device, ...
It isn't easy to setup/observe such test case and observe what is wrong.
> this test started timing out occasionally. Looking a bit closer it appears instances are
> getting stuck during reboot with a bunch of looping messages:
>
> ```
> [ 17.978854] block device autoloading is deprecated and will be removed.
> [ 17.982555] block device autoloading is deprecated and will be removed.
> [ 17.985537] block device autoloading is deprecated and will be removed.
> [ 17.987546] block device autoloading is deprecated and will be removed.
> [ 17.989540] block device autoloading is deprecated and will be removed.
> [ 17.991547] block device autoloading is deprecated and will be removed.
> [ 17.993555] block device autoloading is deprecated and will be removed.
> [ 17.995539] block device autoloading is deprecated and will be removed.
> [ 17.997577] block device autoloading is deprecated and will be removed.
> [ 17.999544] block device autoloading is deprecated and will be removed.
> [ 22.979465] blkdev_get_no_open: 1666 callbacks suppressed
> ...
> ...
> ...
> [ 618.221270] blkdev_get_no_open: 1664 callbacks suppressed
> [ 618.221273] block device autoloading is deprecated and will be removed.
> [ 618.224274] block device autoloading is deprecated and will be removed.
> [ 618.227267] block device autoloading is deprecated and will be removed.
> [ 618.229274] block device autoloading is deprecated and will be removed.
> [ 618.231277] block device autoloading is deprecated and will be removed.
> [ 618.233277] block device autoloading is deprecated and will be removed.
> [ 618.235282] block device autoloading is deprecated and will be removed.
> [ 618.237370] block device autoloading is deprecated and will be removed.
> [ 618.239356] block device autoloading is deprecated and will be removed.
> [ 618.241290] block device autoloading is deprecated and will be removed.
> ```
>
> Using the Fedora kernels I narrowed it down to being introduced between
> `kernel-5.19.0-0.rc3.27.fc37` (good) and `kernel-5.19.0-0.rc4.33.fc37` (bad).
>
> I then did a bisect and found:
>
> ```
> $ git bisect bad
> a09b314005f3a0956ebf56e01b3b80339df577cc is the first bad commit
> commit a09b314005f3a0956ebf56e01b3b80339df577cc
> Author: Christoph Hellwig <hch@lst.de>
> Date: Tue Jun 14 09:48:27 2022 +0200
>
> block: freeze the queue earlier in del_gendisk
It is a bit hard to associate the above commit with reported issue.
Thanks,
Ming
next prev parent reply other threads:[~2022-09-01 7:07 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-26 16:15 regression caused by block: freeze the queue earlier in del_gendisk Dusty Mabe
2022-08-28 10:24 ` Thorsten Leemhuis
2022-08-31 12:36 ` Thorsten Leemhuis
2022-09-01 7:06 ` Ming Lei [this message]
2022-09-03 13:47 ` Dusty Mabe
2022-09-07 7:33 ` Christoph Hellwig
2022-09-07 8:38 ` Ming Lei
2022-09-07 14:40 ` Chaitanya Kulkarni
2022-09-07 14:58 ` Ming Lei
2022-09-07 15:53 ` Chaitanya Kulkarni
2022-09-09 8:24 ` Ming Lei
2022-09-12 7:16 ` Christoph Hellwig
2022-09-13 1:55 ` Ming Lei
2022-09-13 2:36 ` Dusty Mabe
2022-09-20 9:11 ` Thorsten Leemhuis
2022-09-20 14:05 ` Jens Axboe
2022-09-20 14:12 ` Christoph Hellwig
2022-09-20 14:14 ` Jens Axboe
2022-09-21 9:25 ` Thorsten Leemhuis
2022-09-21 14:34 ` Jens Axboe
2022-09-21 14:47 ` Greg KH
2022-09-21 14:56 ` Jens Axboe
2022-09-26 7:09 ` Greg KH
2022-09-07 7:22 ` Christoph Hellwig
2022-09-07 14:56 ` Dusty Mabe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YxBZ4BBjxvAkvI2A@T590 \
--to=ming.lei@redhat.com \
--cc=axboe@kernel.dk \
--cc=dusty@dustymabe.com \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox