From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43A3ED1039A for ; Mon, 28 Oct 2024 12:57:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BcUrdGaZLtr9ged66NRG2hWG88kIL4/g3u6k5M4mM1Q=; b=S7/k1zfa0aHk0NTQ9W5pTHiF0U jhN4XVJzDupFa7HnKcANs2pbaT2EDoT7uD3ROKn13n/GO3eTz0WPRhgVFUQzzHc79ozY8pRSzKpUM qnM0wjafSy2KpgMUcLUOdXtCR0YZA3JhFZHBnPB1JXiq+5ZkqVi+x7oKAmYKNP+lA39w2XzuvK1Yt 2EoSU4vjVGuCu3eXnLH0bogmTn0bvmkRrIdTj9i/QMAYBCP8j2P5CjygBXIx5MKrZ5gHpGK8scSwx KDHdiSbWgj3LwfslbXaPM9/TzT2z8/BKCSwBj9VExkKOUJNrsEGIh3Wf3GteIxCyxroExpRk6D20v ZrxQW4Kg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t5PIv-0000000Aqms-2FcL; Mon, 28 Oct 2024 12:57:05 +0000 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t5PA8-0000000ApIY-3Oew for linux-nvme@lists.infradead.org; Mon, 28 Oct 2024 12:48:02 +0000 Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49S4Wc5x014786; Mon, 28 Oct 2024 12:47:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=BcUrdGaZLtr9ged66 NRG2hWG88kIL4/g3u6k5M4mM1Q=; b=rf2XMLTQ6Yc2bHADZrPgHC7KjjoMCoMJ+ PXcsW/2Dn4AjrakkSmqSv9bG4tNyLA1aOJfyVFynF8gQ0Hb8wiSgbhXRJW/MI8UE 3Nf4TU3RuRn5cC/7AMqkHFpy8rrJw2G9rU7PkMOFu9TnxDMCTG27LY1gRy2oMLdq KHY2oXIi+iDDxULzNnLtRt2/GlufZmaY5wKgkyDgysVNAyXeoCATkGxSTHxio0jR 8QBafi+5JbsWRlkmm1pyEZXQnhGldu3GT5GZ2EY2oqeFZjYgFGm5Ff4U7262THdK m898B1lnk+19JDkQUu5f27Pbg5XDoaNkCpGeKmcIoH30w3T46ydEQ== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 42j3nsjj7a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 28 Oct 2024 12:47:52 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 49SB4I0q017312; Mon, 28 Oct 2024 12:47:27 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 42hars6e3w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 28 Oct 2024 12:47:27 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 49SClNDE54985062 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 28 Oct 2024 12:47:23 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A09EE2004B; Mon, 28 Oct 2024 12:47:23 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7C13920040; Mon, 28 Oct 2024 12:47:21 +0000 (GMT) Received: from li-c9696b4c-3419-11b2-a85c-f9edc3bf8a84.ibm.com.com (unknown [9.171.1.253]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 28 Oct 2024 12:47:21 +0000 (GMT) From: Nilay Shroff To: linux-nvme@lists.infradead.org Cc: kbusch@kernel.org, hch@lst.de, sagi@grimberg.me, axboe@fb.com, chaitanyak@nvidia.com, dlemoal@kernel.org, gjoyce@linux.ibm.com, Nilay Shroff Subject: [PATCHv2 1/3] Revert "nvme: make keep-alive synchronous operation" Date: Mon, 28 Oct 2024 18:17:09 +0530 Message-ID: <20241028124717.517132-2-nilay@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241028124717.517132-1-nilay@linux.ibm.com> References: <20241028124717.517132-1-nilay@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: U5SQeta3qY_S591AE5CVwKXHjoCbp-pQ X-Proofpoint-GUID: U5SQeta3qY_S591AE5CVwKXHjoCbp-pQ Content-Transfer-Encoding: 8bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-15_01,2024-10-11_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 adultscore=0 bulkscore=0 mlxlogscore=992 clxscore=1015 priorityscore=1501 suspectscore=0 lowpriorityscore=0 mlxscore=0 impostorscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2410280103 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241028_054800_945591_F35D8428 X-CRM114-Status: GOOD ( 16.04 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org This reverts commit d06923670b5a5f609603d4a9fee4dec02d38de9c. This reverts commit 599d9f3a10eec69ef28a90161763e4bd7c9c02bf. It was realized that the fix implemented to avoid the race condition between keep alive task and the fabric shutdown code path in the commit d06923670b5ia ("nvme: make keep-alive synchronous operation") is not optimal. We also found that the above race condition is regression caused due to the changes implemented in commit a54a93d0e359 ("nvme: move stopping keep-alive into nvme_uninit_ctrl()"). So we decided to first revert the commit d06923670b5a ("nvme: make keep-alive synchronous operation") and commit 599d9f3a10ee ("nvme: use helper nvme_ctrl_state in nvme_keep_alive_finish function") and then fix the regression. Link: https://lore.kernel.org/all/196f4013-3bbf-43ff-98b4-9cb2a96c20c2@grimberg.me/ Reviewed-by: Sagi Grimberg Signed-off-by: Nilay Shroff --- drivers/nvme/host/core.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 84cb859a911d..5016f69e9a15 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -1292,12 +1292,14 @@ static void nvme_queue_keep_alive_work(struct nvme_ctrl *ctrl) queue_delayed_work(nvme_wq, &ctrl->ka_work, delay); } -static void nvme_keep_alive_finish(struct request *rq, - blk_status_t status, struct nvme_ctrl *ctrl) +static enum rq_end_io_ret nvme_keep_alive_end_io(struct request *rq, + blk_status_t status) { + struct nvme_ctrl *ctrl = rq->end_io_data; + unsigned long flags; + bool startka = false; unsigned long rtt = jiffies - (rq->deadline - rq->timeout); unsigned long delay = nvme_keep_alive_work_period(ctrl); - enum nvme_ctrl_state state = nvme_ctrl_state(ctrl); /* * Subtract off the keepalive RTT so nvme_keep_alive_work runs @@ -1311,17 +1313,25 @@ static void nvme_keep_alive_finish(struct request *rq, delay = 0; } + blk_mq_free_request(rq); + if (status) { dev_err(ctrl->device, "failed nvme_keep_alive_end_io error=%d\n", status); - return; + return RQ_END_IO_NONE; } ctrl->ka_last_check_time = jiffies; ctrl->comp_seen = false; - if (state == NVME_CTRL_LIVE || state == NVME_CTRL_CONNECTING) + spin_lock_irqsave(&ctrl->lock, flags); + if (ctrl->state == NVME_CTRL_LIVE || + ctrl->state == NVME_CTRL_CONNECTING) + startka = true; + spin_unlock_irqrestore(&ctrl->lock, flags); + if (startka) queue_delayed_work(nvme_wq, &ctrl->ka_work, delay); + return RQ_END_IO_NONE; } static void nvme_keep_alive_work(struct work_struct *work) @@ -1330,7 +1340,6 @@ static void nvme_keep_alive_work(struct work_struct *work) struct nvme_ctrl, ka_work); bool comp_seen = ctrl->comp_seen; struct request *rq; - blk_status_t status; ctrl->ka_last_check_time = jiffies; @@ -1353,9 +1362,9 @@ static void nvme_keep_alive_work(struct work_struct *work) nvme_init_request(rq, &ctrl->ka_cmd); rq->timeout = ctrl->kato * HZ; - status = blk_execute_rq(rq, false); - nvme_keep_alive_finish(rq, status, ctrl); - blk_mq_free_request(rq); + rq->end_io = nvme_keep_alive_end_io; + rq->end_io_data = ctrl; + blk_execute_rq_nowait(rq, false); } static void nvme_start_keep_alive(struct nvme_ctrl *ctrl) -- 2.45.2