Re: regression caused by block: freeze the queue earlier in del_gendisk

Christoph Hellwig
2022-09-07T07:22:37Z

Just curious: what is the underlying device (or more speicifically
driver) under the md raids?

On Fri, Aug 26, 2022 at 12:15:22PM -0400, Dusty Mabe wrote:
> Hey All,
>
> I think I've found a regression introduced by:
>
> a09b314 o block: freeze the queue earlier in del_gendisk
>
> In Fedora CoreOS we have tests that set up RAID1 on the /boot/ and /root/ partitions
> and then subsequently removes one of the disks to simulate a failure. Sometime recently
> this test started timing out occasionally. Looking a bit closer it appears instances are
> getting stuck during reboot with a bunch of looping messages:
>
> ```
> [ 17.978854] block device autoloading is deprecated and will be removed.
> [ 17.982555] block device autoloading is deprecated and will be removed.
> [ 17.985537] block device autoloading is deprecated and will be removed.
> [ 17.987546] block device autoloading is deprecated and will be removed.
> [ 17.989540] block device autoloading is deprecated and will be removed.
> [ 17.991547] block device autoloading is deprecated and will be removed.
> [ 17.993555] block device autoloading is deprecated and will be removed.
> [ 17.995539] block device autoloading is deprecated and will be removed.
> [ 17.997577] block device autoloading is deprecated and will be removed.
> [ 17.999544] block device autoloading is deprecated and will be removed.
> [ 22.979465] blkdev_get_no_open: 1666 callbacks suppressed
> ...
> ...
> ...
> [ 618.221270] blkdev_get_no_open: 1664 callbacks suppressed
> [ 618.221273] block device autoloading is deprecated and will be removed.
> [ 618.224274] block device autoloading is deprecated and will be removed.
> [ 618.227267] block device autoloading is deprecated and will be removed.
> [ 618.229274] block device autoloading is deprecated and will be removed.
> [ 618.231277] block device autoloading is deprecated and will be removed.
> [ 618.233277] block device autoloading is deprecated and will be removed.
> [ 618.235282] block device autoloading is deprecated and will be removed.
> [ 618.237370] block device autoloading is deprecated and will be removed.
> [ 618.239356] block device autoloading is deprecated and will be removed.
> [ 618.241290] block device autoloading is deprecated and will be removed.
> ```
>
> Using the Fedora kernels I narrowed it down to being introduced between
> `kernel-5.19.0-0.rc3.27.fc37` (good) and `kernel-5.19.0-0.rc4.33.fc37` (bad).
>
> I then did a bisect and found:
>
> ```
> $ git bisect bad
> a09b314005f3a0956ebf56e01b3b80339df577cc is the first bad commit
> commit a09b314005f3a0956ebf56e01b3b80339df577cc
> Author: Christoph Hellwig <hch@lst.de>
> Date: Tue Jun 14 09:48:27 2022 +0200
>
> block: freeze the queue earlier in del_gendisk
>
> Freeze the queue earlier in del_gendisk so that the state does not
> change while we remove debugfs and sysfs files.
>
> Ming mentioned that being able to observer request in debugfs might
> be useful while the queue is being frozen in del_gendisk, which is
> made possible by this change.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Link: https://lore.kernel.org/r/20220614074827.458955-5-hch@lst.de
> Signed-off-by: Jens Axboe <axboe@kernel.dk>
>
> block/genhd.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
> ```
>
> Reverting this commit on top of latest git master (4c612826b) gave me successful results.
>
> Any ideas on what could be amiss here? Luckily the patch is tiny so hopefully it might
> be obvious.
>
> More details (including logs) in the following locations:
>
> - https://bugzilla.redhat.com/show_bug.cgi?id=2121791
> - https://github.com/coreos/fedora-coreos-tracker/issues/1282
>
>
> Thanks!
> Dusty
---end quoted text---