From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dl1-f48.google.com (mail-dl1-f48.google.com [74.125.82.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AA9B36B05F for ; Tue, 17 Feb 2026 15:35:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.48 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771342535; cv=none; b=XoSqF6KpmbcsbHKPW+3WHumGDgLjCZo4udFd29C975CxjvxRpgBJ37SX0GlS6GEqUos5sDCAmR5PJkH4SNvplVlkw5fOCrdhAbm29ESCJXTboDL0bn/+jMaP29TLB4c4oAiQ8lkiTawJ40U5tS6VAGL1ULtj07kaNOITL1ALQHw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771342535; c=relaxed/simple; bh=4cCKZrtdB5d9uR4SSDRWu683JGHEP9OaAQZ/6QRzgDc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jult/t5+f6Nm3dp4C+w92ONll844gdRApf+F46EhJyYRq9fRVVIKpcx2/A3XKjfQYCDXzc5+w6U4hJ1Cle1zjKsg4Nqf6Qot9R69sbCMMiNfLmTAQEKm4q7slBC2i4OuyBGB/i6bERuzbUf1zJFrO9Nl73Rv9GdH2YShAWxxM60= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=gN5W9CZD; arc=none smtp.client-ip=74.125.82.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="gN5W9CZD" Received: by mail-dl1-f48.google.com with SMTP id a92af1059eb24-124a95e592fso415426c88.0 for ; Tue, 17 Feb 2026 07:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1771342533; x=1771947333; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jzHPwh7gCcw3aD1stz1FwTBp8o4EpMP2ycLOZFh68ak=; b=gN5W9CZDml134WPQPwdL1YJX7yEpfGfQAOT55JOIms2R8HJzpjXDDL3UG2ZVx60mBl JBVW0go3h3GDnGYo9NV8/xiybNR2Bd8YjRX8KIcAv+uc5WGYdcSMBQdwt9zH6sQhepdu kJVt4euypORNxaj2dKHDfCmnmvdkE5AVUPPcXCBkYS/sCRwiH4UFY6bF8T4dCG3skiot XRaICIAB9KpS8iy7tnS8lm62GnWlYD2O1hMYJCg0VU43Cg9QwKlDI9OxThVXJfEzVTxu eubfMRBoxgyydVwOEpbgd2X0iSmL/64c7Yn+sFDNRlNSTyld8ChSOwDOOG3uf07urECB A2fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771342533; x=1771947333; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jzHPwh7gCcw3aD1stz1FwTBp8o4EpMP2ycLOZFh68ak=; b=nHTDgQP+HOIsbOoa3kYwsusGH6xlbB5JITAC/FPovPuLFbuLxVNjlljvYGfb04+WQS sFI2kLB7CHPW9uozC7W9b+4+8nXKbN/jPqKaKO9diwOrj6ULrIVNly9H7BvmLVdlOu6d hfiy//gUtjYTR3rVWLEIn9Ts6ySKG9B6TEW8ZSC/h8srNTQLFm7cTQH64aXQ4BlC/VGv 1buD1HbAUCurmLGRAH3+gsAjwfwDfhvBDA9TP604Klk/bpE82LNFAHCnvZgZcvnzSYfI Mks2YSixClRpwKVLvAR6NRlOwET3QezSQsmZgKq6qkM2gtUq6SRQBKundkEM1r+DyMwD iNdw== X-Forwarded-Encrypted: i=1; AJvYcCUJ6edYrBIkvKE4pttqwCnreLrhSRJ4dQzODfRuz6pMNi/g0Scye2UmGyOTK1r+LXcgNdWtb3XZ3xUbUu8=@vger.kernel.org X-Gm-Message-State: AOJu0Yw/MToXlaSYqHROIq+VUTZ9+ZxA+RL0sPFGD1cA/ZWmeN11CbdI 983Na7pL1QrqSHJPswlz1xQ7dIwH3dgIK+FtziN0ykMqFZ0yTaftIeVC0JZaQT5MlmU= X-Gm-Gg: AZuq6aI9+uJAL/Cgc4w3VvfsngX2lr3UE/H2lKN6wMzqpxow0nsRtMrCVx6TbCiV1pE bPKBY+/SSyPfnFPJbuFPtOGIEoA3+mHbyTL7DPSolN6Sn2usdPkMdu9Lg4RDH1LWwzYdWHpNht/ OqJ+oXILbqb/1F+OXCs2qPAaKxu8EnQwU1Q4ayThjVU8aDH4RWkzfp9z4mgBq86wfLrOuW1ms1w 2BhwbmveUpReH1JvJssOXreXMcwBZALZdNM8ryWG7uqWtCEFDS4j9JlBIDL+Bn6vdryJH2TH0+k i5U4YFZ7W1WTGd1g6Qd+FnCxF2TMpHfKzkKOB6DTGQFE8/Zv1cpjvFZSqMe9Qx7Sgv1CQM4kwWu vu2Catg3kA0X7+oiRLOiZX0MbDuiyMIVgHA/sTUxxQYDkw0oRZ03QbDp2PUnTWsPXKyi2QE09J0 hJ4H/FZWUy5kdOdDUfPW2h8DtMlg0= X-Received: by 2002:a05:7300:6d22:b0:2b7:f7f:6ad with SMTP id 5a478bee46e88-2babc535b2amr5645107eec.26.1771342533072; Tue, 17 Feb 2026 07:35:33 -0800 (PST) Received: from medusa.lab.kspace.sh ([2601:640:8202:6fb0::68dd]) by smtp.googlemail.com with UTF8SMTPSA id 5a478bee46e88-2bacb543d4dsm14668289eec.7.2026.02.17.07.35.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Feb 2026 07:35:32 -0800 (PST) Date: Tue, 17 Feb 2026 07:35:30 -0800 From: Mohamed Khalfella To: Hannes Reinecke Cc: Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg , James Smart , Aaron Dailey , Randy Jennings , Dhaval Giani , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 18/21] nvme: Update CCR completion wait timeout to consider CQT Message-ID: <20260217153530.GI2392949-mkhalfella@purestorage.com> References: <20260214042753.4073668-1-mkhalfella@purestorage.com> <20260214042753.4073668-19-mkhalfella@purestorage.com> <9d6cf4f9-37d8-4704-bcc1-0b849ad28955@suse.de> <20260216184515.GH2392949-mkhalfella@purestorage.com> <3bad149e-a0fa-4377-9701-7b35ef6b5b88@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3bad149e-a0fa-4377-9701-7b35ef6b5b88@suse.de> On Tue 2026-02-17 08:09:33 +0100, Hannes Reinecke wrote: > On 2/16/26 19:45, Mohamed Khalfella wrote: > > On Mon 2026-02-16 13:54:18 +0100, Hannes Reinecke wrote: > >> On 2/14/26 05:25, Mohamed Khalfella wrote: > >>> TP8028 Rapid Path Failure Recovery does not define how much time the > >>> host should wait for CCR operation to complete. It is reasonable to > >>> assume that CCR operation can take up to ctrl->cqt. Update wait time for > >>> CCR operation to be max(ctrl->cqt, ctrl->kato). > >>> > >>> Signed-off-by: Mohamed Khalfella > >>> --- > >>> drivers/nvme/host/core.c | 2 +- > >>> 1 file changed, 1 insertion(+), 1 deletion(-) > >>> > >>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c > >>> index 0680d05900c1..ff479c0263ab 100644 > >>> --- a/drivers/nvme/host/core.c > >>> +++ b/drivers/nvme/host/core.c > >>> @@ -631,7 +631,7 @@ static int nvme_issue_wait_ccr(struct nvme_ctrl *sctrl, struct nvme_ctrl *ictrl) > >>> if (result & 0x01) /* Immediate Reset Successful */ > >>> goto out; > >>> > >>> - tmo = secs_to_jiffies(ictrl->kato); > >>> + tmo = msecs_to_jiffies(max(ictrl->cqt, ictrl->kato * 1000)); > >>> if (!wait_for_completion_timeout(&ccr.complete, tmo)) { > >>> ret = -ETIMEDOUT; > >>> goto out; > >> > >> That is not my understanding. I was under the impression that CQT is the > >> _additional_ time a controller requires to clear out outstanding > >> commands once it detected a loss of communication (ie _after_ KATO). > >> Which would mean we have to wait for up to > >> (ctrl->kato * 1000) + ctrl->cqt. > > > > At this point the source controller knows about communication loss. We > > do not need kato wait. In theory we should just wait for CQT. > > max(cqt, kato) is a conservative guess I made. > > > Not quite. The source controller (on the host!) knows about the > communication loss. But the target might not, as the keep-alive > command might have arrived at the target _just_ before KATO > triggered on the host. So the target is still good, and will > be waiting for _another_ KATO interval before declaring > a loss of communication. > And only then will the CQT period start at the target. > > Randy, please correct me if I'm wrong ... > wait_for_completion_timeout(&ccr.complete, tmo)) waits for CCR operation to complete. The wait starts after CCR command completed successfully. IOW, it starts after the host received a CQE from source controller on the target telling us all is good. If the source controller on the target already know about loss of communication then there is no need to wait for KATO. We just need to wait for CCR operation to finish because we know it has been started successfully. The specs does not tell us how much time to wait for CCR operation to complete. max(cqt, kato) is an estimate I think reasonable to make.