[PATCH -next v2] block: update io_ticks when io hang

[PATCH -next v2] block: update io_ticks when io hang

[Zhang Wensheng]

When the inflight IOs are slow and no new IOs are issued, we expect
iostat could manifest the IO hang problem. However after
commit 5b18b5a73760 ("block: delete part_round_stats and switch to less
precise counting"), io_tick and time_in_queue will not be updated until
the end of IO, and the avgqu-sz and %util columns of iostat will be zero.

Because it has using stat.nsecs accumulation to express time_in_queue
which is not suitable to change, and may %util will express the status
better when io hang occur. To fix io_ticks, we use update_io_ticks and
inflight to update io_ticks when diskstats_show and part_stat_show
been called.

Fixes: 5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting")
Signed-off-by: Zhang Wensheng <zhangwensheng5@huawei.com>
---
 block/genhd.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
---
v2:
* add part_stat_lock() & part_stat_unlock() to protect update_io_ticks().
v1: https://www.spinics.net/lists/linux-block/msg78931.html

diff --git a/block/genhd.c b/block/genhd.c
index 626c8406f21a..781dc78f97d8 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -913,12 +913,17 @@ ssize_t part_stat_show(struct device *dev,
 	struct disk_stats stat;
 	unsigned int inflight;
 
-	part_stat_read_all(bdev, &stat);
 	if (queue_is_mq(q))
 		inflight = blk_mq_in_flight(q, bdev);
 	else
 		inflight = part_in_flight(bdev);
 
+	if (inflight) {
+		part_stat_lock();
+		update_io_ticks(bdev, jiffies, true);
+		part_stat_unlock();
+	}
+	part_stat_read_all(bdev, &stat);
 	return sprintf(buf,
 		"%8lu %8lu %8llu %8u "
 		"%8lu %8lu %8llu %8u "
@@ -1174,12 +1179,17 @@ static int diskstats_show(struct seq_file *seqf, void *v)
 	xa_for_each(&gp->part_tbl, idx, hd) {
 		if (bdev_is_partition(hd) && !bdev_nr_sectors(hd))
 			continue;
-		part_stat_read_all(hd, &stat);
 		if (queue_is_mq(gp->queue))
 			inflight = blk_mq_in_flight(gp->queue, hd);
 		else
 			inflight = part_in_flight(hd);
 
+		if (inflight) {
+			part_stat_lock();
+			update_io_ticks(hd, jiffies, true);
+			part_stat_unlock();
+		}
+		part_stat_read_all(hd, &stat);
 		seq_printf(seqf, "%4d %7d %pg "
 			   "%lu %lu %lu %u "
 			   "%lu %lu %lu %u "
-- 
2.31.1

Re: [PATCH -next v2] block: update io_ticks when io hang

[zhangwensheng (E)]

friendly ping...

在 2022/2/17 14:42, Zhang Wensheng 写道:
> When the inflight IOs are slow and no new IOs are issued, we expect
> iostat could manifest the IO hang problem. However after
> commit 5b18b5a73760 ("block: delete part_round_stats and switch to less
> precise counting"), io_tick and time_in_queue will not be updated until
> the end of IO, and the avgqu-sz and %util columns of iostat will be zero.
>
> Because it has using stat.nsecs accumulation to express time_in_queue
> which is not suitable to change, and may %util will express the status
> better when io hang occur. To fix io_ticks, we use update_io_ticks and
> inflight to update io_ticks when diskstats_show and part_stat_show
> been called.
>
> Fixes: 5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting")
> Signed-off-by: Zhang Wensheng <zhangwensheng5@huawei.com>
> ---
>   block/genhd.c | 14 ++++++++++++--
>   1 file changed, 12 insertions(+), 2 deletions(-)
> ---
> v2:
> * add part_stat_lock() & part_stat_unlock() to protect update_io_ticks().
> v1: https://www.spinics.net/lists/linux-block/msg78931.html
>
> diff --git a/block/genhd.c b/block/genhd.c
> index 626c8406f21a..781dc78f97d8 100644
> --- a/block/genhd.c
> +++ b/block/genhd.c
> @@ -913,12 +913,17 @@ ssize_t part_stat_show(struct device *dev,
>   	struct disk_stats stat;
>   	unsigned int inflight;
>   
> -	part_stat_read_all(bdev, &stat);
>   	if (queue_is_mq(q))
>   		inflight = blk_mq_in_flight(q, bdev);
>   	else
>   		inflight = part_in_flight(bdev);
>   
> +	if (inflight) {
> +		part_stat_lock();
> +		update_io_ticks(bdev, jiffies, true);
> +		part_stat_unlock();
> +	}
> +	part_stat_read_all(bdev, &stat);
>   	return sprintf(buf,
>   		"%8lu %8lu %8llu %8u "
>   		"%8lu %8lu %8llu %8u "
> @@ -1174,12 +1179,17 @@ static int diskstats_show(struct seq_file *seqf, void *v)
>   	xa_for_each(&gp->part_tbl, idx, hd) {
>   		if (bdev_is_partition(hd) && !bdev_nr_sectors(hd))
>   			continue;
> -		part_stat_read_all(hd, &stat);
>   		if (queue_is_mq(gp->queue))
>   			inflight = blk_mq_in_flight(gp->queue, hd);
>   		else
>   			inflight = part_in_flight(hd);
>   
> +		if (inflight) {
> +			part_stat_lock();
> +			update_io_ticks(hd, jiffies, true);
> +			part_stat_unlock();
> +		}
> +		part_stat_read_all(hd, &stat);
>   		seq_printf(seqf, "%4d %7d %pg "
>   			   "%lu %lu %lu %u "
>   			   "%lu %lu %lu %u "

Re: [PATCH -next v2] block: update io_ticks when io hang

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH -next v2] block: update io_ticks when io hang

[Jens Axboe]

On Thu, 17 Feb 2022 14:42:47 +0800, Zhang Wensheng wrote:
> When the inflight IOs are slow and no new IOs are issued, we expect
> iostat could manifest the IO hang problem. However after
> commit 5b18b5a73760 ("block: delete part_round_stats and switch to less
> precise counting"), io_tick and time_in_queue will not be updated until
> the end of IO, and the avgqu-sz and %util columns of iostat will be zero.
> 
> Because it has using stat.nsecs accumulation to express time_in_queue
> which is not suitable to change, and may %util will express the status
> better when io hang occur. To fix io_ticks, we use update_io_ticks and
> inflight to update io_ticks when diskstats_show and part_stat_show
> been called.
> 
> [...]

Applied, thanks!

[1/1] block: update io_ticks when io hang
      commit: 86d7331299fda7634b11c1b7c911432679d525a5

Best regards,
-- 
Jens Axboe

[PATCH -next v2] block: update io_ticks when io hang

[Zhang Wensheng]

When the inflight IOs are slow and no new IOs are issued, we expect
iostat could manifest the IO hang problem. However after
commit 5b18b5a73760 ("block: delete part_round_stats and switch to less
precise counting"), io_tick and time_in_queue will not be updated until
the end of IO, and the avgqu-sz and %util columns of iostat will be zero.

Because it has using stat.nsecs accumulation to express time_in_queue
which is not suitable to change, and may %util will express the status
better when io hang occur. To fix io_ticks, we use update_io_ticks and
inflight to update io_ticks when diskstats_show and part_stat_show
been called.

Fixes: 5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting")
Signed-off-by: Zhang Wensheng <zhangwensheng5@huawei.com>
---
 block/genhd.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
---
v2:
* add part_stat_lock() & part_stat_unlock() to protect update_io_ticks().
v1: https://www.spinics.net/lists/linux-block/msg78931.html

diff --git a/block/genhd.c b/block/genhd.c
index 626c8406f21a..781dc78f97d8 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -913,12 +913,17 @@ ssize_t part_stat_show(struct device *dev,
 	struct disk_stats stat;
 	unsigned int inflight;
 
-	part_stat_read_all(bdev, &stat);
 	if (queue_is_mq(q))
 		inflight = blk_mq_in_flight(q, bdev);
 	else
 		inflight = part_in_flight(bdev);
 
+	if (inflight) {
+		part_stat_lock();
+		update_io_ticks(bdev, jiffies, true);
+		part_stat_unlock();
+	}
+	part_stat_read_all(bdev, &stat);
 	return sprintf(buf,
 		"%8lu %8lu %8llu %8u "
 		"%8lu %8lu %8llu %8u "
@@ -1174,12 +1179,17 @@ static int diskstats_show(struct seq_file *seqf, void *v)
 	xa_for_each(&gp->part_tbl, idx, hd) {
 		if (bdev_is_partition(hd) && !bdev_nr_sectors(hd))
 			continue;
-		part_stat_read_all(hd, &stat);
 		if (queue_is_mq(gp->queue))
 			inflight = blk_mq_in_flight(gp->queue, hd);
 		else
 			inflight = part_in_flight(hd);
 
+		if (inflight) {
+			part_stat_lock();
+			update_io_ticks(hd, jiffies, true);
+			part_stat_unlock();
+		}
+		part_stat_read_all(hd, &stat);
 		seq_printf(seqf, "%4d %7d %pg "
 			   "%lu %lu %lu %u "
 			   "%lu %lu %lu %u "
-- 
2.31.1