[PATCH v4 0/2] block: warn once for each partition in bio_check_ro()

[PATCH v4 0/2] block: warn once for each partition in bio_check_ro()

[Yu Kuai]

From: Yu Kuai <yukuai3@huawei.com>

Changes in v4:
 - remove the patch to add 'bd_flags', and add a new field 'bool
 bd_ro_warned' in patch 2. 'bd_flags' will be added once 'bd_inode' is
 removed from other thread.

Changes in v3:
 - add patch 1 from Ming, swap bd_inode layout with bd_openers and
 bd_size_lock;
 - change bd_flags from u32 to u16 in patch 2, prevent to affect layout of
 other fields;

Changes in v2:
 - don't use test/set_bit() for new field, because unsigned long will
 cause that some field can't be placed in the first cacheline(64 bytes),
 use unsigned int for new field and test/set/clear it like 'bio->bi_flags'.

Ming Lei (1):
  block: move .bd_inode into 1st cacheline of block_device

Yu Kuai (1):
  block: warn once for each partition in bio_check_ro()

 block/blk-core.c          | 14 +++++++++++---
 include/linux/blk_types.h |  4 +++-
 2 files changed, 14 insertions(+), 4 deletions(-)

-- 
2.39.2

[PATCH v4 1/2] block: move .bd_inode into 1st cacheline of block_device

[Yu Kuai]

From: Ming Lei <ming.lei@redhat.com>

The .bd_inode field of block_device is used in IO fast path of
blkdev_write_iter() and blkdev_llseek(), so it is more efficient to keep
it into the 1st cacheline.

.bd_openers is only touched in open()/close(), and .bd_size_lock is only
for updating bdev capacity, which is in slow path too.

So swap .bd_inode layout with .bd_openers & .bd_size_lock to move
.bd_inode into the 1st cache line.

Cc: Yu Kuai <yukuai3@huawei.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Yu Kuai <yukuai3@huawei.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/blk_types.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index d5c5e59ddbd2..f7d40692dd94 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -49,9 +49,10 @@ struct block_device {
 	bool			bd_write_holder;
 	bool			bd_has_submit_bio;
 	dev_t			bd_dev;
+	struct inode		*bd_inode;	/* will die */
+
 	atomic_t		bd_openers;
 	spinlock_t		bd_size_lock; /* for bd_inode->i_size updates */
-	struct inode *		bd_inode;	/* will die */
 	void *			bd_claiming;
 	void *			bd_holder;
 	const struct blk_holder_ops *bd_holder_ops;
-- 
2.39.2

[PATCH v4 2/2] block: warn once for each partition in bio_check_ro()

[Yu Kuai]

From: Yu Kuai <yukuai3@huawei.com>

Commit 1b0a151c10a6 ("blk-core: use pr_warn_ratelimited() in
bio_check_ro()") fix message storm by limit the rate, however, there
will still be lots of message in the long term. Fix it better by warn
once for each partition.

Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
 block/blk-core.c          | 14 +++++++++++---
 include/linux/blk_types.h |  1 +
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index fdf25b8d6e78..2eca76ccf4ee 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -501,9 +501,17 @@ static inline void bio_check_ro(struct bio *bio)
 	if (op_is_write(bio_op(bio)) && bdev_read_only(bio->bi_bdev)) {
 		if (op_is_flush(bio->bi_opf) && !bio_sectors(bio))
 			return;
-		pr_warn_ratelimited("Trying to write to read-only block-device %pg\n",
-				    bio->bi_bdev);
-		/* Older lvm-tools actually trigger this */
+
+		if (bio->bi_bdev->bd_ro_warned)
+			return;
+
+		bio->bi_bdev->bd_ro_warned = true;
+		/*
+		 * Use ioctl to set underlying disk of raid/dm to read-only
+		 * will trigger this.
+		 */
+		pr_warn("Trying to write to read-only block-device %pg\n",
+			bio->bi_bdev);
 	}
 }
 
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index f7d40692dd94..b29ebd53417d 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -70,6 +70,7 @@ struct block_device {
 #ifdef CONFIG_FAIL_MAKE_REQUEST
 	bool			bd_make_it_fail;
 #endif
+	bool			bd_ro_warned;
 	/*
 	 * keep this out-of-line as it's both big and not needed in the fast
 	 * path
-- 
2.39.2

Re: [PATCH v4 1/2] block: move .bd_inode into 1st cacheline of block_device

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v4 2/2] block: warn once for each partition in bio_check_ro()

[Christoph Hellwig]

On Tue, Nov 28, 2023 at 08:30:27PM +0800, Yu Kuai wrote:
> From: Yu Kuai <yukuai3@huawei.com>
> 
> Commit 1b0a151c10a6 ("blk-core: use pr_warn_ratelimited() in
> bio_check_ro()") fix message storm by limit the rate, however, there
> will still be lots of message in the long term. Fix it better by warn
> once for each partition.

The new field is in the same dword alignment as bd_make_it_fail and
could in theory corrupt it, at least on alpha.  I guess we're fine,
because if you enable CONFIG_FAIL_MAKE_REQUEST on alpha you're asking
for this.  I still hope we can clean up these non-atomic bools and
replace them with bitops soon.

Signed-off-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v4 0/2] block: warn once for each partition in bio_check_ro()

[Jens Axboe]

On Tue, 28 Nov 2023 20:30:25 +0800, Yu Kuai wrote:
> Changes in v4:
>  - remove the patch to add 'bd_flags', and add a new field 'bool
>  bd_ro_warned' in patch 2. 'bd_flags' will be added once 'bd_inode' is
>  removed from other thread.
> 
> Changes in v3:
>  - add patch 1 from Ming, swap bd_inode layout with bd_openers and
>  bd_size_lock;
>  - change bd_flags from u32 to u16 in patch 2, prevent to affect layout of
>  other fields;
> 
> [...]

Applied, thanks!

[1/2] block: move .bd_inode into 1st cacheline of block_device
      commit: fad907cffd4bde7384812cf32fcf69becab805cc
[2/2] block: warn once for each partition in bio_check_ro()
      commit: 67d995e069535c32829f5d368d919063492cec6e

Best regards,
-- 
Jens Axboe

Re: [PATCH v4 2/2] block: warn once for each partition in bio_check_ro()

[Yu Kuai]

Hi,

在 2023/11/28 21:00, Christoph Hellwig 写道:
> On Tue, Nov 28, 2023 at 08:30:27PM +0800, Yu Kuai wrote:
>> From: Yu Kuai <yukuai3@huawei.com>
>>
>> Commit 1b0a151c10a6 ("blk-core: use pr_warn_ratelimited() in
>> bio_check_ro()") fix message storm by limit the rate, however, there
>> will still be lots of message in the long term. Fix it better by warn
>> once for each partition.
> 
> The new field is in the same dword alignment as bd_make_it_fail and
> could in theory corrupt it, at least on alpha.  I guess we're fine,
> because if you enable CONFIG_FAIL_MAKE_REQUEST on alpha you're asking
> for this.  I still hope we can clean up these non-atomic bools and
> replace them with bitops soon.

Yes, I'm working on this, and thanks for the review!

Kuai
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> .
>