From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dy1-f172.google.com (mail-dy1-f172.google.com [74.125.82.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A12E83191CF for ; Wed, 18 Feb 2026 12:47:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771418833; cv=none; b=PPvJHmypiOl7SAI7e003Rl+9bYTWqIcXM2A862hyjWgMCK3TusiHaKGtnMxQoeT9LobvGO+dI2VSVJwxPJNJ0NlTCscVA7SiPKFy4JXfF5umrxQoDr8X/xzJCcD1B+Rx+cNkssrscGbLqbAYcY+HQLl9d+DWrGVOLTEpLUXLFgU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771418833; c=relaxed/simple; bh=p1wMR9tFjEkiJzLkquKUGaoGLMr8mEOAjQCzbCx3Exw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=t/C712GAO8hR6/lOACbPvM/J0BQ/ytRtDkbqBlF/OijJnTBAHl8FW4MwTpoz+WKObacMefziDLusVeVoaLigqW9FR5OE9doHDV2f1UfRRKwJntNhSp44AgdSU6VKV+ckBIfrMI6LrT/VKSyyUiRMwHE4zLFQSM2pfnPf/qImac8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=TxZMWpqa; arc=none smtp.client-ip=74.125.82.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="TxZMWpqa" Received: by mail-dy1-f172.google.com with SMTP id 5a478bee46e88-2bab70f8c8aso5570291eec.1 for ; Wed, 18 Feb 2026 04:47:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1771418831; x=1772023631; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=5cDJ595EzNlIRPNEWgUvyHB8pLI4M3wyzlKqJjlAins=; b=TxZMWpqakwYTGwt66on9wHPsbnMxjstsNRVpNhb/0ecYq/GTGQuzU+ht4QT0NEWHPO mbfznJ5+HXgI4vkCDB2YgwplYwD/zRMBFU+1BQbPDOT1Y2blXGKVwkoTaJljA0cIODCl NcgYP6rXDzsRfJBxP0xOgcrjFHNAEPMTXZc9VDubC2/LXp3Vpk6sND6HQZgShuWIH1Xm uL+/NcOtMugoC6SQJC1IG9ZoPrDOLLK56x1PW+2lz5y+5O6KKiEdZyfNCJ3qsFm7FDCi v4lms3SkwLoRuJYvzFrjIjizC0EnP5GRDXC/C5RPPv/oAwjmz7YMNarEenM4Re1X9Dqv Hojg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771418831; x=1772023631; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5cDJ595EzNlIRPNEWgUvyHB8pLI4M3wyzlKqJjlAins=; b=Lj++blKCAy8ktfT7hjruB1HNiDBTu3XTWm39blzryoBkCFdjrVQH9ePI0set5A28m6 Zc1tFXj9FDv9Z/hjxzP4PfP4Ben//fWkYcdolttoXq5BIlfgsZ0Q1DBFw7G0+Fw8hRbx pk2pdJo+3g3OBCwsaPZWsbWQq9gd0ReZ08j723VyvPY+H2WCahBwYaDRVj1r7wyocDd2 ZG2HINPdHLk3dcW7GyDVPRi6/dImQe/rjUvvMgfrp/NHbQ2VlANpJvYQRQcnwCaQOLM1 q4WxYDSGyAjmr4Xnj3Gg3iH5M6w8Hnq6fZny39LQxel2gBzRDSv8xwS9g0jtPRZwc41m 6vFA== X-Forwarded-Encrypted: i=1; AJvYcCXPMMcO7WokbOrvc8y3YkdrMhf0muNjqmLnJcyvSUQ60HbQMxruI1FZHa+Wn6/TjgxuyO047UBzxoPq+q4=@vger.kernel.org X-Gm-Message-State: AOJu0YxfBU1B27AVQCsy8qQGM7M/dpRBSuoBR+kgP1ZmIHKNB2ViKa6i X/RhBpHwjRZNoHOjGpJi4uFkR+4uYrLHfHqj/6FErEwC2JQhZv8VPvT99w2BXNTCoFs= X-Gm-Gg: AZuq6aJ7FaT0T/lQ4C6BeNy8HkkyBnHlF4gFy5i4rB7dBQ8ohX/Ru012TbaGe/rp7Mb FrJAsMKaK1oULtXqA9+xZ2VvcvcsrsKlmgRsrPboBT+8Ooqee8/+BkMDiqj8Df3UpOgkiop2piY 3ywC/mltak/y91BdNVJE7oxB34BJlGmcAcD39OnRf5gjQFXSjZFVLQRsESTLPkC14CJ9sLW5BPX eV06LICusOkYLX960eJKkyPxjF8RpcKxGE/LiGLz02zBVoM5CGb1y3t1zKu71Sbf54X32YfR5yz tLMbt5KhfqVv/lk04cGFggixytG/t8zJs6iXMHbFGCjvrtjVhsSOGhxcNv+VD1xbQ2Yr2VAjUZE X7C8UBUigoiOBH4RDr+IF7Rpg/FJHbsPA+R7AjagV+e4xuR+RjvJApqNmwDr12qq8AOjANrCDKl LUPtY1OpXwHh+z+gnyes6IsLUGZ/YxFWvDng0dTQ== X-Received: by 2002:a05:7300:6d06:b0:2ba:70b6:a162 with SMTP id 5a478bee46e88-2bd502a6853mr696063eec.35.1771418830524; Wed, 18 Feb 2026 04:47:10 -0800 (PST) Received: from medusa.lab.kspace.sh ([2601:640:8202:6fb0::68dd]) by smtp.googlemail.com with UTF8SMTPSA id 5a478bee46e88-2bacb658509sm17230056eec.19.2026.02.18.04.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Feb 2026 04:47:10 -0800 (PST) Date: Wed, 18 Feb 2026 04:47:08 -0800 From: Mohamed Khalfella To: Hannes Reinecke Cc: Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg , James Smart , Aaron Dailey , Randy Jennings , Dhaval Giani , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 09/21] nvme: Implement cross-controller reset completion Message-ID: <20260218124708.GJ2392949-mkhalfella@purestorage.com> References: <20260214042753.4073668-1-mkhalfella@purestorage.com> <20260214042753.4073668-10-mkhalfella@purestorage.com> <3b21ccbd-7948-436b-8faa-a5541c65946a@suse.de> <20260217182554.GE3435530-mkhalfella@purestorage.com> <1a9feff7-5a0d-4385-b582-c9ab60d724db@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1a9feff7-5a0d-4385-b582-c9ab60d724db@suse.de> On Wed 2026-02-18 08:51:31 +0100, Hannes Reinecke wrote: > On 2/17/26 19:25, Mohamed Khalfella wrote: > > On Mon 2026-02-16 13:43:51 +0100, Hannes Reinecke wrote: > [ .. ] > >> > >> We really would need some indicator whether 'ccr' is supported at all. > > > > Why do we need this indicator, other than exporting it via sysfs? > > > To avoid false positives. We will never try CCR on a controller that does not support it. False positive of what? > > >> Using the number of available CCR commands would be an option, if though > >> that would require us to keep two counters (one for the number of > >> possible outstanding CCRs, and one for the number of actual outstanding > >> CCRs.). > > > > Like mentioned above ctrl->ccr_limit gives us the number of ccrs > > available now. It is not 100% indicator if CCR is supported or not, but > > it is enough to implement CCR. A second counter can help us skip trying > > CCR if we know impacted controller does not support it. > > > > Do you think it is worth it? > > > Yes. The problem is that we want to get towards TP8028 compliance, which > forces us to wait for 2*KATO + CQT before requests on the failed patch > can be retried. That will cause a _noticeable_ stall on the application > side. And the only way to shorten that is CCR; once we get confirmation > from CCR we can start retrying immediately. > At the same time the current implementation only waits for 1*KATO before > retrying, so there will be regression if we switch to TP8028-compliant > KATO handling for systems not supporting CCR. The statement above is not correct. Careful consideration and testing has been made to not introduce such regression. If CCR is not supported then nvme_find_ctrl_ccr() will return NULL and nvme_fence_ctrl() will return immediately. No CCR command will be sent and no wait for AEN. What happens next depends on whether ictrl->cqt is supported or not. If not supported, which will be the case for systems in the field today, then requests will be retried immediately. Requests will not be held in this case and no delay will be seen in failover case. > > So we can (and should) use CCR as the determining factor whether we > want to switch to TP8028-compliant behaviour or stick with the original > implementation. We do check CCR support and availability in nvme_find_ctrl_ccr(). Adding a second counter will spare us the loop in nvme_find_ctrl_ccr(), which is not worth it IMO. > > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Kernel Storage Architect > hare@suse.de +49 911 74053 688 > SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg > HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich