From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87AE5C433E0 for ; Tue, 12 Jan 2021 09:35:29 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0FE7F230FF for ; Tue, 12 Jan 2021 09:35:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0FE7F230FF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9cS+7vvSW9yganDc3XHrt2ztE0b6W0haR8FXnRDh6zc=; b=iNhjdFoA8pE4XdayCPrOOVKWh eZ0LIlu78wmHl2MvQVs0BeGENF9QxOl3PxYcoCbX9iqepyXXlwgEWalWiOP05TTYAFdHqiJahc6dZ djlbJx4ap5BWsjekFVAA5Fbkq630qy2TvOVQEIjwlYA6oaQPy3bU7ctBDojYSllP72OoS2I3buu3h c42G1KrExEE4c4Q9MLsuiQD3+m4kdoRaeq+tc76DevOiIVxiUMa0D5qSpWbelnycUidr6xqCftKCG WGdDgvp9JYkwcUeVmgntVBbt0lJhRqi636ayRo4GDFOtD4xOpiy2KeeiICjVRtGAnl+YcAZOEGpAg adt+c07dw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kzG52-0000dz-A7; Tue, 12 Jan 2021 09:35:12 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kzG4x-0000XX-6A for linux-nvme@lists.infradead.org; Tue, 12 Jan 2021 09:35:09 +0000 Received: from DGGEMM405-HUB.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4DFQPX5mBwz5FWH; Tue, 12 Jan 2021 17:33:52 +0800 (CST) Received: from dggema772-chm.china.huawei.com (10.1.198.214) by DGGEMM405-HUB.china.huawei.com (10.3.20.213) with Microsoft SMTP Server (TLS) id 14.3.498.0; Tue, 12 Jan 2021 17:34:52 +0800 Received: from [10.169.42.93] (10.169.42.93) by dggema772-chm.china.huawei.com (10.1.198.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Tue, 12 Jan 2021 17:34:49 +0800 Subject: Re: [PATCH 3/3] nvme: add parameter command_retry to enable retry To: Minwoo Im References: <20210108144659.7126-1-minwoo.im.dev@gmail.com> <20210108144659.7126-4-minwoo.im.dev@gmail.com> <20210111122748.GA6831@localhost.localdomain> From: Chao Leng Message-ID: Date: Tue, 12 Jan 2021 17:34:48 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20210111122748.GA6831@localhost.localdomain> Content-Language: en-US X-Originating-IP: [10.169.42.93] X-ClientProxiedBy: dggeme713-chm.china.huawei.com (10.1.199.109) To dggema772-chm.china.huawei.com (10.1.198.214) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210112_043507_769900_19A94CED X-CRM114-Status: GOOD ( 28.45 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Jens Axboe , Christoph Hellwig , linux-nvme@lists.infradead.org, Sagi Grimberg Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On 2021/1/11 20:27, Minwoo Im wrote: > Hello, > > On 21-01-11 11:47:56, Chao Leng wrote: >> >> >> On 2021/1/8 22:46, Minwoo Im wrote: >>> nvme_init_request() has set REQ_FAILFAST_DRIVER to make requests >>> non-retryable. This command flag value is checked in >>> nvme_decide_disposition() to decide whether to RETRY or other >>> operations. In that point, blk_noretry_request() macro will be used to >>> check if command flags have one of REQ_FAILFAST_*. If so, it just >>> decides to complete the request without retrying. >>> >>> This patch added a module parameter named command_retry to turn on the >>> command retry feature in this driver. If turning it on, >>> REQ3_FAILFAST_DRIVER will not be set to requests so that retry can be >>> reached out to nvme_retry_req() based on the module parameter. >>> >>> Signed-off-by: Minwoo Im >>> --- >>> drivers/nvme/host/core.c | 7 ++++++- >>> 1 file changed, 6 insertions(+), 1 deletion(-) >>> >>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c >>> index 6e428fdc25a8..e1836ca9956f 100644 >>> --- a/drivers/nvme/host/core.c >>> +++ b/drivers/nvme/host/core.c >>> @@ -44,6 +44,10 @@ static unsigned char shutdown_timeout = 5; >>> module_param(shutdown_timeout, byte, 0644); >>> MODULE_PARM_DESC(shutdown_timeout, "timeout in seconds for controller shutdown"); >>> +static bool command_retry; >>> +module_param(command_retry, bool, 0644); >>> +MODULE_PARM_DESC(command_retry, "retry commands up to nvme_max_retries"); >>> + >>> static u8 nvme_max_retries = 5; >>> module_param_named(max_retries, nvme_max_retries, byte, 0644); >>> MODULE_PARM_DESC(max_retries, "max number of retries a command may have"); >>> @@ -560,7 +564,8 @@ static inline void nvme_init_request(struct request *req, >>> else /* no queuedata implies admin queue */ >>> req->timeout = NVME_ADMIN_TIMEOUT; >>> - req->cmd_flags |= REQ_FAILFAST_DRIVER; >>> + if (!command_retry) >>> + req->cmd_flags |= REQ_FAILFAST_DRIVER; >> In abnormal scenarios, such as request time out, connection process may takes long time or the admin command waits for long time. >> Retry only for non-host errors may be a better choice. Maybe we can make some optimizations in nvme_decide_disposition. > > Thanks for your review! > > Oh, I agreed that it might wait for so long time in connecting process. > Restricting some of commands that should be retried would be better as > you mentiond. > > Do you mean that maybe we can check this module parameter in > nvme_decide_disposition()? Like, even if blk_noretry_request(req) says > that it's non-retriable, if this module parameter is enabled, then we > can retry rather than failfast? No, I mean that add the local preferential retry which defined in the NVMe protocol. --- drivers/nvme/host/core.c | 18 +++++++++++++----- drivers/nvme/host/nvme.h | 1 + 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 99f91efe3824..a25e9b4956b9 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -316,15 +316,20 @@ static inline enum nvme_disposition nvme_decide_disposition(struct request *req) if (likely(nvme_req(req)->status == 0)) return COMPLETE; - if (blk_noretry_request(req) || - (nvme_req(req)->status & NVME_SC_DNR) || + if ((nvme_req(req)->status & NVME_SC_DNR) || nvme_req(req)->retries >= nvme_max_retries) return COMPLETE; + if (nvme_req(req)->ctrl->acre && + !nvme_is_path_error(nvme_req(req)->status) && + !blk_queue_dying(req->q)) + return RETRY; + + if (blk_noretry_request(req)) + return COMPLETE; + if (req->cmd_flags & REQ_NVME_MPATH) { - if (nvme_is_path_error(nvme_req(req)->status) || - blk_queue_dying(req->q)) - return FAILOVER; + return FAILOVER; } else { if (blk_queue_dying(req->q)) return COMPLETE; @@ -2513,6 +2518,7 @@ static int nvme_configure_acre(struct nvme_ctrl *ctrl) struct nvme_feat_host_behavior *host; int ret; + ctrl->acre = false; /* Don't bother enabling the feature if retry delay is not reported */ if (!ctrl->crdt[0]) return 0; @@ -2524,6 +2530,8 @@ static int nvme_configure_acre(struct nvme_ctrl *ctrl) host->acre = NVME_ENABLE_ACRE; ret = nvme_set_features(ctrl, NVME_FEAT_HOST_BEHAVIOR, 0, host, sizeof(*host), NULL); + if (!ret) + ctrl->acre = true; kfree(host); return ret; } diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index bfcedfa4b057..fd914b0dec88 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -276,6 +276,7 @@ struct nvme_ctrl { #ifdef CONFIG_BLK_DEV_ZONED u32 max_zone_append; #endif + bool acre; u16 crdt[3]; u16 oncs; u16 oacs; -- > > Thanks, > . > _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme