From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bart Van Assche Subject: [PATCH 08/12] dm: Fix a race condition related to stopping and starting queues Date: Wed, 26 Oct 2016 15:54:34 -0700 Message-ID: <28b3e91c-018a-0dbd-8ca9-0a7994a97a5d@sandisk.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: linux-block-owner@vger.kernel.org To: Jens Axboe Cc: Christoph Hellwig , James Bottomley , "Martin K. Petersen" , Mike Snitzer , Doug Ledford , Keith Busch , Ming Lei , Laurence Oberman , "linux-block@vger.kernel.org" , "linux-scsi@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "linux-nvme@lists.infradead.org" List-Id: linux-rdma@vger.kernel.org Ensure that all ongoing dm_mq_queue_rq() and dm_mq_requeue_request() calls have stopped before setting the "queue stopped" flag. This allows to remove the "queue stopped" test from dm_mq_queue_rq() and dm_mq_requeue_request(). This patch fixes a race condition because dm_mq_queue_rq() is called without holding the queue lock and hence BLK_MQ_S_STOPPED can be set at any time while dm_mq_queue_rq() is in progress. This patch prevents that the following hang occurs sporadically when using dm-mq: INFO: task systemd-udevd:10111 blocked for more than 480 seconds. Call Trace: [] schedule+0x37/0x90 [] schedule_timeout+0x27f/0x470 [] io_schedule_timeout+0x9f/0x110 [] bit_wait_io+0x16/0x60 [] __wait_on_bit_lock+0x49/0xa0 [] __lock_page+0xb9/0xc0 [] truncate_inode_pages_range+0x3e0/0x760 [] truncate_inode_pages+0x10/0x20 [] kill_bdev+0x30/0x40 [] __blkdev_put+0x71/0x360 [] blkdev_put+0x49/0x170 [] blkdev_close+0x20/0x30 [] __fput+0xe8/0x1f0 [] ____fput+0x9/0x10 [] task_work_run+0x83/0xb0 [] do_exit+0x3ee/0xc40 [] do_group_exit+0x4b/0xc0 [] get_signal+0x2ca/0x940 [] do_signal+0x23/0x660 [] exit_to_usermode_loop+0x73/0xb0 [] syscall_return_slowpath+0xb0/0xc0 [] entry_SYSCALL_64_fastpath+0xa6/0xa8 Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke Reviewed-by: Johannes Thumshirn Reviewed-by: Christoph Hellwig Cc: Mike Snitzer --- drivers/md/dm-rq.c | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm-rq.c b/drivers/md/dm-rq.c index d47a504..107ed19 100644 --- a/drivers/md/dm-rq.c +++ b/drivers/md/dm-rq.c @@ -105,6 +105,8 @@ static void dm_mq_stop_queue(struct request_queue *q) /* Avoid that requeuing could restart the queue. */ blk_mq_cancel_requeue_work(q); blk_mq_stop_hw_queues(q); + /* Wait until dm_mq_queue_rq() has finished. */ + blk_mq_quiesce_queue(q); } void dm_stop_queue(struct request_queue *q) @@ -887,17 +889,6 @@ static int dm_mq_queue_rq(struct blk_mq_hw_ctx *hctx, dm_put_live_table(md, srcu_idx); } - /* - * On suspend dm_stop_queue() handles stopping the blk-mq - * request_queue BUT: even though the hw_queues are marked - * BLK_MQ_S_STOPPED at that point there is still a race that - * is allowing block/blk-mq.c to call ->queue_rq against a - * hctx that it really shouldn't. The following check guards - * against this rarity (albeit _not_ race-free). - */ - if (unlikely(test_bit(BLK_MQ_S_STOPPED, &hctx->state))) - return BLK_MQ_RQ_QUEUE_BUSY; - if (ti->type->busy && ti->type->busy(ti)) return BLK_MQ_RQ_QUEUE_BUSY; -- 2.10.1