From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7121BC433E7 for ; Thu, 8 Oct 2020 05:30:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1CE4B2087D for ; Thu, 8 Oct 2020 05:30:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602135007; bh=0itYT/2Oi7jerNZY1UNsS81UQ96H3ynHfeRmw1iyiVw=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=aDKbVMXHzsqBEomEEZOrr8OVZ3FEO0g1zv553AoGvE1Xz/1+YHzHWmcPu6eerKX/L CgqV4pIu1Aj5MoKepksocCapwJut7ydAQH2qyQjZ/qjS4KaugZ95gQayzvUltu73lz WQDsYpxeu70rNiC+n+Zb86Hbrv3KvIJFTXsiHqeo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726245AbgJHFaG (ORCPT ); Thu, 8 Oct 2020 01:30:06 -0400 Received: from mail.kernel.org ([198.145.29.99]:53220 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725858AbgJHFaG (ORCPT ); Thu, 8 Oct 2020 01:30:06 -0400 Received: from localhost (unknown [213.57.247.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2726F20708; Thu, 8 Oct 2020 05:30:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602135005; bh=0itYT/2Oi7jerNZY1UNsS81UQ96H3ynHfeRmw1iyiVw=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=gATTpmZNeT33Kcp6jk0G+gBm0UxIpYxI3tg5WliFFe+AwUiOSc9Ps+ucIeCefZNzq 2m0CDIkII4ZoiWEPFraJ8zJnXoVy9ODPjIen9wjvD02hJtEgMj0Bwk24wV8qx7m39Q nEEBwdm1Q1sjcBMSdlfW03n/z3OJDYR6VbU2Md8k= Date: Thu, 8 Oct 2020 08:30:02 +0300 From: Leon Romanovsky To: Sagi Grimberg Cc: Max Gurtovoy , Krishnamraju Eraparaju , linux-rdma@vger.kernel.org, Potnuri Bharat Teja , Max Gurtovoy Subject: Re: reduce iSERT Max IO size Message-ID: <20201008053002.GC13580@unreal> References: <20200922104424.GA18887@chelsio.com> <07e53835-8389-3e07-6976-505edbd94f2a@grimberg.me> <20201002171007.GA16636@chelsio.com> <4d0b1a3f-2980-c7ed-ef9a-0ed6a9c87a69@grimberg.me> <20201003033644.GA19516@chelsio.com> <4391e240-5d6d-fb59-e6fb-e7818d1d0bd2@nvidia.com> <20201007033619.GA11425@chelsio.com> <1a034761-3723-3c70-8a44-25ef2cbf786e@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Wed, Oct 07, 2020 at 04:50:27PM -0700, Sagi Grimberg wrote: > > > > I think max IO size, at iSER initiator, depends on > > > "max_fast_reg_page_list_len". > > > currently, below are the supported "max_fast_reg_page_list_len" of > > > various iwarp drivers: > > > > > > iw_cxgb4: 128 pages > > > Softiwarp: 256 pages > > > i40iw: 512 pages > > > qedr: couldn't find. > > > > > > For iwarp case, if 512 is the max pages supported by all iwarp drivers, > > > then provisioning a gigantic MR pool at target(to accommodate never used > > > 16MiB IO) wouldn't be a overkill? > > > > For RoCE/IB Mellanox HCAs we support 16MiB IO size and even more. We > > limited to 16MiB in iSER/iSERT. > > > > Sagi, > > > > what about adding a module parameter for this as we did in iSER initiator ? > > I don't think we have any other choice... Sagi, I didn't read whole thread and know little about ULPs, but wonder if isn't it possible to check device type (iWARP/RoCE) during iSERT initialization and create MR pool only after device is recognized? Thanks