From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 846E71339B2; Sun, 28 Jan 2024 16:15:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706458549; cv=none; b=dSOxRivb64qXnk2l4InNtvgMfJwE+Shw+Eh+qPUCDj6whWQkV/LFa5x/qKKxDlwJaXG7xgbqc8dEdqAMdwuRwFctr6wzIDEc4hAqdLQJvIm53mS5u55PRSxbpB2U+H09Q497aHn7IqWtXJbuCKnzVCAHaI6hFYL7UzyRoox2LKI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706458549; c=relaxed/simple; bh=aT4xyTy/4s5/JEA/cj6T330uERtCwlNcortw7kINqRA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tzkXp/K3InjclH0q2zeUl21Q8nnK8HdKbOH51bbtOHkRU5/qiQgceD9Che3O+NDtVNqqVvQnmP3EYXeoP1uvbhJbfO2O3Fw5QsBMpRpfhB5mwJ4zHZidSxTuhluG+QzZ5vL62YMBE/l2h1pkR9GQ3t7q7QtjML7Lh0XJrbL+MEs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eYqDkcFP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eYqDkcFP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F7DEC433C7; Sun, 28 Jan 2024 16:15:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706458549; bh=aT4xyTy/4s5/JEA/cj6T330uERtCwlNcortw7kINqRA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eYqDkcFP8nM1uK3VZcqeURZNBvqNUKkv8NakRY7a7FmuPXz/H1LWxQH/EeM1HmOqj tBfy16E6TnndL+Kv3HoQaX1T+ft3hIUEug0ySN4GXioBxjPdgYzXhwOpfTP9B+X7B9 QEWV8XmyyIyJFiiIkWuHPsPWz4QCKoR9FJFWNuTzb9cUpBmTYc7rWtGBFHmth/XT5A FumK2Ia5YqnCFxej4gQd/ZF2KetwCa2Toh97okgIHTw8pgVrbl23Hqco/Upx3hYUDr f9WX0fLj7FFQQYiF3YPNAUob7KApxp8VaSwoEIxnnR/3lqfDrhXQ4nFxCFU84LcsbO fMrbfktOshf7Q== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Ming Lei , Jan Kara , Kemeng Shi , Changhui Zhong , Jens Axboe , Sasha Levin , linux-block@vger.kernel.org Subject: [PATCH AUTOSEL 5.15 15/19] blk-mq: fix IO hang from sbitmap wakeup race Date: Sun, 28 Jan 2024 11:15:12 -0500 Message-ID: <20240128161524.204182-15-sashal@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240128161524.204182-1-sashal@kernel.org> References: <20240128161524.204182-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore X-stable-base: Linux 5.15.148 Content-Transfer-Encoding: 8bit From: Ming Lei [ Upstream commit 5266caaf5660529e3da53004b8b7174cab6374ed ] In blk_mq_mark_tag_wait(), __add_wait_queue() may be re-ordered with the following blk_mq_get_driver_tag() in case of getting driver tag failure. Then in __sbitmap_queue_wake_up(), waitqueue_active() may not observe the added waiter in blk_mq_mark_tag_wait() and wake up nothing, meantime blk_mq_mark_tag_wait() can't get driver tag successfully. This issue can be reproduced by running the following test in loop, and fio hang can be observed in < 30min when running it on my test VM in laptop. modprobe -r scsi_debug modprobe scsi_debug delay=0 dev_size_mb=4096 max_queue=1 host_max_queue=1 submit_queues=4 dev=`ls -d /sys/bus/pseudo/drivers/scsi_debug/adapter*/host*/target*/*/block/* | head -1 | xargs basename` fio --filename=/dev/"$dev" --direct=1 --rw=randrw --bs=4k --iodepth=1 \ --runtime=100 --numjobs=40 --time_based --name=test \ --ioengine=libaio Fix the issue by adding one explicit barrier in blk_mq_mark_tag_wait(), which is just fine in case of running out of tag. Cc: Jan Kara Cc: Kemeng Shi Reported-by: Changhui Zhong Signed-off-by: Ming Lei Link: https://lore.kernel.org/r/20240112122626.4181044-1-ming.lei@redhat.com Signed-off-by: Jens Axboe Signed-off-by: Sasha Levin --- block/blk-mq.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/block/blk-mq.c b/block/blk-mq.c index bbbbcd2c1941..d80ebba16778 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1176,6 +1176,22 @@ static bool blk_mq_mark_tag_wait(struct blk_mq_hw_ctx *hctx, wait->flags &= ~WQ_FLAG_EXCLUSIVE; __add_wait_queue(wq, wait); + /* + * Add one explicit barrier since blk_mq_get_driver_tag() may + * not imply barrier in case of failure. + * + * Order adding us to wait queue and allocating driver tag. + * + * The pair is the one implied in sbitmap_queue_wake_up() which + * orders clearing sbitmap tag bits and waitqueue_active() in + * __sbitmap_queue_wake_up(), since waitqueue_active() is lockless + * + * Otherwise, re-order of adding wait queue and getting driver tag + * may cause __sbitmap_queue_wake_up() to wake up nothing because + * the waitqueue_active() may not observe us in wait queue. + */ + smp_mb(); + /* * It's possible that a tag was freed in the window between the * allocation failure and adding the hardware queue to the wait -- 2.43.0