From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA3C2883F for ; Wed, 11 Feb 2026 00:12:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770768770; cv=none; b=g5+M4VaCz6/4aLzKV4tfVddU/Wd3Ng919oOmMIJNG3UjwnotkugfordK2lv1aRxe7ecW+3/xzqlqyiy4Xr3R87U2y4pqA8fAlW/G4t/Gfr3Si0HcmCgCqzktfAA8mhBlQijjqs4TEza6PNFae1OmGABXlsKW9YyhmqJ9+qWqHzU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770768770; c=relaxed/simple; bh=60FSo9MkeX+cLrvkiBcjKbRbCXfx6atBX2wykPGyNM4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=lr9YmJhyABNBfGW93kwtx172rMJAs/JS31xctauV90uUephcAwsr+uxicVBps9Uc+VxbZArqFcqVFXZEKc3ZhRu/KXDyYpHftZEaXgTn+9GvifMp3J0g2SDiGFG3nO+Qzzy6jkbvgLDlPovV+whCW5+L58PqDYUr8ALUh128hv8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=KXi0zVtd; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="KXi0zVtd" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-4801c2fae63so51471185e9.2 for ; Tue, 10 Feb 2026 16:12:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1770768767; x=1771373567; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=cujy7Px8Xtv882zc02e0/J26UmbB2bTAirwtCIJhOg8=; b=KXi0zVtduhZVDeC+GTKEU+LrTsf2z8JXdmIEv56VJTDwyrqR4QfNufa52REm8Khnsr DLVWUa184JQQvo8YDpalgfy1ILSu5r4vpeqYUSlsMhO6vVQLGkkyJDiZXTLbk5y8kzia z+FpPiot/uSNRLh9I5cXOVAcLBPFMchHWE7EfkKm1p+/gcmWLqw4nciN/4KHcj+D7ajr C7sxBZhxHqHVPGr2orRY0W4ZlTSNKpzXf1kYKk5WNukbN65JYuDMMZYMi6E7HUu4mNuV xRkdZizKzKQKKaop3xWcjICmYxBWKdAe57BotdtwefDONFsAW8Yhk0ZP9cpF2RIkJr3O aUOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770768767; x=1771373567; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cujy7Px8Xtv882zc02e0/J26UmbB2bTAirwtCIJhOg8=; b=ZF2aU/j9Ouj1a5WYoDO8K4kbQAal5kKJ5IyhckKCw3Wnq/siPT7edaz7cLuJgsmUzH 8Yll+zmI4edOieL1ud03NGWMWZpWoVf9N9/gH/8D+9w/4Klw+TqLzZaWRs/6tDw4ZP95 22bsPA/58mR5vb9toXZF8c8RG6yWfJsIOd54R9fcaxWEgmoChqLcM/B8CL5BKFQ/gF1L 284bJKJskVpBCyK4uFRGQ2ECPzMyNuIOBIIauVh6eP3bIywvYY1SAB7rVZgYq7Q/ZQxm TnKW4apvrcYU8i76NNsMVNy9U4U5bhMAEnwAe9XGsUe7XISRNcFPHiMEQdRL9N/Oyo95 peGg== X-Forwarded-Encrypted: i=1; AJvYcCVpXScahqzQccXu4AATi8warHCOkTM1q1YOeDQ5CxIhP0J1ZWbfaW7AMmcW0aB+SBWM2r+m8Ytt0c7hRbU=@vger.kernel.org X-Gm-Message-State: AOJu0YwC2TqWNCgT4Wk7Yxg/v95sZr0qfaF4qLMU5n4BrUzP+BytaIfz 4rVxFjck+4CCC6/Xr/nLIJCtrLvUQLVT+F45WY717lKGxMaNSYBeuPuJELob+Wsq/MQ= X-Gm-Gg: AZuq6aJZ1qVHOzg2Qd4L6+cpn9FwT9NClYId6OLwikesULZjv1FIWfIQOh8MX4kUuDH vDR8mJbwPMFFDLyNt31tdG3+8ybFoBowQPhtHVYRLFgHBR1yoMBi6WcUaU4Bm1rSdX17T1dsWFk sfIlQ3CZP5QmR1yM8arW16QMOXrm5xZHG310R1UgSbVcBIhe9iRF8PW9wfmP+a1o0n0j/6UfUP6 TPChA7icdN++zmeQNB2+/oOh6YdKNRsVSl72RVt6yvVe0uBktGcUJL8pdgti7XOmTFjmnPClqNF Ya0XxwgTNFri23T6BDoLG7VKpCxX3Ni/6OD117oja8CD34BnadgaNMc7Kd2s+NVACyRKRJmmL/Q ExtzOsrem4hzhWHhvVZ2mhXrltT3fUAJUYOtYI+fKTp6Ah/amnPpz7vYAK9c4cfASiIsmuWGPj4 uG1R9iEabqSFXi6p3kzZyjuGX4Q3PjVjNgg1eHDek7t4k= X-Received: by 2002:a05:600c:1da8:b0:47e:f481:24b7 with SMTP id 5b1f17b1804b1-4835dfcf995mr677145e9.17.1770768767169; Tue, 10 Feb 2026 16:12:47 -0800 (PST) Received: from medusa.lab.kspace.sh ([208.88.152.253]) by smtp.googlemail.com with UTF8SMTPSA id 5b1f17b1804b1-4834d8334a8sm88916655e9.12.2026.02.10.16.12.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Feb 2026 16:12:46 -0800 (PST) Date: Tue, 10 Feb 2026 16:12:43 -0800 From: Mohamed Khalfella To: James Smart Cc: Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg , Aaron Dailey , Randy Jennings , Dhaval Giani , Hannes Reinecke , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 08/14] nvme: Implement cross-controller reset recovery Message-ID: <20260211001243.GS3729-mkhalfella@purestorage.com> References: <20260130223531.2478849-1-mkhalfella@purestorage.com> <20260130223531.2478849-9-mkhalfella@purestorage.com> <05875e07-b908-425a-ba6f-5e060e03241e@gmail.com> <20260210222732.GQ3729-mkhalfella@purestorage.com> <5f3c9cf0-7fee-432a-b6c5-44fb2acb0b1d@gmail.com> <20260210232553.GR3729-mkhalfella@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260210232553.GR3729-mkhalfella@purestorage.com> On Tue 2026-02-10 15:25:55 -0800, Mohamed Khalfella wrote: > On Tue 2026-02-10 14:49:15 -0800, James Smart wrote: > > On 2/10/2026 2:27 PM, Mohamed Khalfella wrote: > > > On Tue 2026-02-10 14:09:27 -0800, James Smart wrote: > > >> On 1/30/2026 2:34 PM, Mohamed Khalfella wrote: > > >> ... > > >>> +unsigned long nvme_fence_ctrl(struct nvme_ctrl *ictrl) > > >>> +{ > > >>> + unsigned long deadline, now, timeout; > > >>> + struct nvme_ctrl *sctrl; > > >>> + u32 min_cntlid = 0; > > >>> + int ret; > > >>> + > > >>> + timeout = nvme_fence_timeout_ms(ictrl); > > >>> + dev_info(ictrl->device, "attempting CCR, timeout %lums\n", timeout); > > >>> + > > >>> + now = jiffies; > > >>> + deadline = now + msecs_to_jiffies(timeout); > > >>> + while (time_before(now, deadline)) { > > >> > > >> Q: don't we have something to identify the controller's subsystem > > >> supports CCR before we starting selecting controllers and sending CCR ? > > >> > > >> I would think on older devices that don't support it we should be > > >> skipping this loop. The loop could delay the Time-Based delay without > > >> any CCR. > > > > > > I do not think we have something that identifies CCR support at > > > subsystem level. The spec defines CCRL at the controller level. The loop > > > should not that bad. nvme_find_ctrl_ccr() should return NULL if CCR is > > > not supported and nvme_fence_ctrl() will return immediately. > > > > > >> > > >> -- james > > >> > > > > I would think CCRL on the failed controller would be enough to assume > > the subsystem supports it. > > ictrl->ccr_limit is a good indication that subsystem supports CCR. I do > not think it is enough though. I say that for two reasons: > > - May be this controller does not support CCR but others do on the same > subsystem. There is nothing prevents subsystem from putting a cap of > CCR at subsytem level. > - May be this controller supports CCR command but not now because all > CCR slots are used now. This can happen in the case of cascading > failure. > > > > > I'm not worried about the coding on the host is so bad. It's more the > > multiple paths that must have cmds sent to them and getting error > > responses for unknown cmds (should be responded to ok, but you never > > know) as well as creating conditions for other errors where there will > > be no return for it - e.g. other paths losing connectivity while the ccr > > outstanding, etc. yes, they all have to work, but why bother adding > > these flows to an old controller that would never do CCR ? > > If nvme_find_ctrl_ccr() returns a source controller to use then we know > the controller supports CCR and does have an available slot to process > this CCR request. I do not see how this code will send CCR request to an > old controller that does not know about CCR command. > > I am not fully opposed against using ictrl->ccr_limit to return early. I > do not see the need for it. If you feel strongly about it I can update > nvme_fence_ctrl() to do so. > I forgot to mention that ctrl->ccr_limit is initialized from id->ccrl in nvme_init_identify(). If this value is greater than zero then we know the controller does support CCR. nvme_find_ctrl_ccr() checks for that and the returned source controller must support CCR and has a slot available for it.