From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 024ACC433F5 for ; Wed, 22 Dec 2021 03:16:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242120AbhLVDQV (ORCPT ); Tue, 21 Dec 2021 22:16:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242094AbhLVDQN (ORCPT ); Tue, 21 Dec 2021 22:16:13 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1029C061574 for ; Tue, 21 Dec 2021 19:16:13 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 43614617F8 for ; Wed, 22 Dec 2021 03:16:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 546C9C36AED; Wed, 22 Dec 2021 03:16:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1640142972; bh=JgGhDdk7FthbmgQy9cZijtUtH3TZ3TWmoQ+CryZ8hx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kJtKm57CBHIhhL6BUY2TUSbwpIM6GsHbi+h3uXvFWh0aKranVCPzFPmambzNowMGF MJs/+ma4FRMBxAk/H/6KApP9DspqLdHfxGYmfyiDsfjpF8MvkM/OdMUmj1P86MzTTP oIASHAcMKv1NyrR4bme35o3uceSUF3pDFscQwYN/cJiebjt0tdAjdPpHZ3Ll2XZb5e o65kTf7P0L1C27FHJZ1bCqxk/KuOZppEdEd3eMR+b0QZ19yu9bYUI4m3t++D3jPau4 AaqAKheGiqY4SxLly/Epp06JOMQ2kVQ9RsiM29SUVYnRZMxNAJ93sqdgIPJ3dt2q7k EkCW2G8O2C9Tg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Shay Drory , Moshe Shemesh , Saeed Mahameed Subject: [net-next v0 04/14] net/mlx5: Let user configure event_eq_size param Date: Tue, 21 Dec 2021 19:15:54 -0800 Message-Id: <20211222031604.14540-5-saeed@kernel.org> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211222031604.14540-1-saeed@kernel.org> References: <20211222031604.14540-1-saeed@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Shay Drory Event EQ is an EQ which received the notification of almost all the events generated by the NIC. Currently, each event EQ is taking 512KB of memory. This size is not needed in most use cases, and is critical with large scale. Hence, allow user to configure the size of the event EQ. For example to reduce event EQ size to 64, execute:: $ devlink dev param set pci/0000:00:0b.0 name event_eq_size value 64 \ cmode driverinit $ devlink dev reload pci/0000:00:0b.0 Signed-off-by: Shay Drory Reviewed-by: Moshe Shemesh Signed-off-by: Saeed Mahameed --- Documentation/networking/devlink/mlx5.rst | 3 +++ .../net/ethernet/mellanox/mlx5/core/devlink.c | 7 +++++++ drivers/net/ethernet/mellanox/mlx5/core/eq.c | 16 +++++++++++++++- 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/Documentation/networking/devlink/mlx5.rst b/Documentation/networking/devlink/mlx5.rst index 291e7f63af73..38089f0aefcf 100644 --- a/Documentation/networking/devlink/mlx5.rst +++ b/Documentation/networking/devlink/mlx5.rst @@ -20,6 +20,9 @@ Parameters * - ``io_eq_size`` - driverinit - The range is between 64 and 4096. + * - ``event_eq_size`` + - driverinit + - The range is between 64 and 4096. The ``mlx5`` driver also implements the following driver-specific parameters. diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 177c6e9159f8..37b7600c5545 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -579,6 +579,8 @@ static const struct devlink_param mlx5_devlink_params[] = { mlx5_devlink_enable_remote_dev_reset_set, NULL), DEVLINK_PARAM_GENERIC(IO_EQ_SIZE, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), NULL, NULL, mlx5_devlink_eq_depth_validate), + DEVLINK_PARAM_GENERIC(EVENT_EQ_SIZE, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, mlx5_devlink_eq_depth_validate), }; static void mlx5_devlink_set_params_init_values(struct devlink *devlink) @@ -622,6 +624,11 @@ static void mlx5_devlink_set_params_init_values(struct devlink *devlink) devlink_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE, value); + + value.vu32 = MLX5_NUM_ASYNC_EQE; + devlink_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE, + value); } static const struct devlink_param enable_eth_param = diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index 7686d7c9c824..b695aad71ee1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -623,6 +623,20 @@ static void cleanup_async_eq(struct mlx5_core_dev *dev, name, err); } +static u16 async_eq_depth_devlink_param_get(struct mlx5_core_dev *dev) +{ + struct devlink *devlink = priv_to_devlink(dev); + union devlink_param_value val; + int err; + + err = devlink_param_driverinit_value_get(devlink, + DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE, + &val); + if (!err) + return val.vu32; + mlx5_core_dbg(dev, "Failed to get param. using default. err = %d\n", err); + return MLX5_NUM_ASYNC_EQE; +} static int create_async_eqs(struct mlx5_core_dev *dev) { struct mlx5_eq_table *table = dev->priv.eq_table; @@ -647,7 +661,7 @@ static int create_async_eqs(struct mlx5_core_dev *dev) param = (struct mlx5_eq_param) { .irq_index = MLX5_IRQ_EQ_CTRL, - .nent = MLX5_NUM_ASYNC_EQE, + .nent = async_eq_depth_devlink_param_get(dev), }; gather_async_events_mask(dev, param.mask); -- 2.33.1