From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1091C11D34 for ; Mon, 24 Feb 2020 16:47:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7321B21744 for ; Mon, 24 Feb 2020 16:47:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="bYTpdsqP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7321B21744 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VXvO+w2OaCcMBFNGeCTmD3k3i/xnM5qrkTSQrWhsilQ=; b=bYTpdsqPafAnWs Q9zawVuQA8ts+lU8wXKo4Ri4kgZp4yKTdbKJr8tAa/mOpo0xxewhxd/N/hKbPiAQjTDszj6QVxPhV pXRXhvpIFwEiDPbmazfseffqD59VmQaAZSZP4NamHseMSvu2ZO082I6D/sn4SAs0b9ISeOwH+1/Y9 pc9qtiSwqXB7yEveD0I0LuYAzbVmq09yBKbvfIY/kYoLPgaLwssprysqDoaY3vlX6I1A7J8AftbJd IQtaGKxJtjd6meMztdEmwEezsCLwcaYXBl9GE8OJscyCx4Ky3xzQM3x3zFESE7iswUOvPTHcVjVKB afzL6vVrIr6egj7qQAZw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6Gte-0002uX-E6; Mon, 24 Feb 2020 16:47:54 +0000 Received: from mail-il-dmz.mellanox.com ([193.47.165.129] helo=mellanox.co.il) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6Grj-0001EG-4U for linux-nvme@lists.infradead.org; Mon, 24 Feb 2020 16:46:01 +0000 Received: from Internal Mail-Server by MTLPINE1 (envelope-from maxg@mellanox.com) with ESMTPS (AES256-SHA encrypted); 24 Feb 2020 18:45:47 +0200 Received: from mtr-vdi-031.wap.labs.mlnx. (mtr-vdi-031.wap.labs.mlnx [10.209.102.136]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 01OGji9e013647; Mon, 24 Feb 2020 18:45:47 +0200 From: Max Gurtovoy To: linux-nvme@lists.infradead.org, sagi@grimberg.me, linux-rdma@vger.kernel.org, kbusch@kernel.org, hch@lst.de, martin.petersen@oracle.com Subject: [PATCH 18/19] nvmet-rdma: Implement set_mdts controller op Date: Mon, 24 Feb 2020 18:45:43 +0200 Message-Id: <20200224164544.219438-20-maxg@mellanox.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200224164544.219438-1-maxg@mellanox.com> References: <20200224164544.219438-1-maxg@mellanox.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200224_084555_597852_E37830D2 X-CRM114-Status: UNSURE ( 9.16 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: axboe@kernel.dk, vladimirk@mellanox.com, shlomin@mellanox.com, israelr@mellanox.com, idanb@mellanox.com, maxg@mellanox.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org Set the maximal data transfer size according to device capabilities. For example, for T10-DIF offload by supporting RDMA HCA, one uses RDMA/rw API that limits the IO operation to use a single MR with 256 pages at the most. Limit the mdts according to RDMA/rw API and even decrease it in order to avoid multiple splits by the local block layer for large IOs to ease on the CPU on the target side. Signed-off-by: Max Gurtovoy --- drivers/nvme/target/rdma.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 37d262a..2227adf 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -30,6 +30,7 @@ #define NVMET_RDMA_DEFAULT_INLINE_DATA_SIZE PAGE_SIZE #define NVMET_RDMA_MAX_INLINE_SGE 4 #define NVMET_RDMA_MAX_INLINE_DATA_SIZE max_t(int, SZ_16K, PAGE_SIZE) +#define NVMET_RDMA_T10_PI_MDTS 5 struct nvmet_rdma_cmd { struct ib_sge sge[NVMET_RDMA_MAX_INLINE_SGE + 1]; @@ -1602,6 +1603,21 @@ static void nvmet_rdma_disc_port_addr(struct nvmet_req *req, } } +static u8 nvmet_rdma_set_mdts(struct nvmet_ctrl *ctrl) +{ + struct nvmet_port *port = ctrl->port; + struct rdma_cm_id *cm_id = port->priv; + u32 max_pages; + + if (ctrl->pi_support) { + max_pages = rdma_rw_fr_page_list_len(cm_id->device, true); + /* Assume mpsmin == device_page_size == 4KB */ + return min(ilog2(max_pages), NVMET_RDMA_T10_PI_MDTS); + } + + return 0; +} + static const struct nvmet_fabrics_ops nvmet_rdma_ops = { .owner = THIS_MODULE, .type = NVMF_TRTYPE_RDMA, @@ -1612,6 +1628,7 @@ static void nvmet_rdma_disc_port_addr(struct nvmet_req *req, .queue_response = nvmet_rdma_queue_response, .delete_ctrl = nvmet_rdma_delete_ctrl, .disc_traddr = nvmet_rdma_disc_port_addr, + .set_mdts = nvmet_rdma_set_mdts, }; static void nvmet_rdma_remove_one(struct ib_device *ib_device, void *client_data) -- 1.8.3.1 _______________________________________________ linux-nvme mailing list linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme