From mboxrd@z Thu Jan 1 00:00:00 1970 From: Leon Romanovsky Subject: [PATCH mlx5-next 07/10] net/mlx5: Use multi threaded workqueue for page fault handling Date: Thu, 8 Nov 2018 21:10:14 +0200 Message-ID: <20181108191017.21891-8-leon@kernel.org> References: <20181108191017.21891-1-leon@kernel.org> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Cc: Leon Romanovsky , RDMA mailing list , Artemy Kovalyov , Majd Dibbiny , Moni Shoua , Saeed Mahameed , linux-netdev To: Doug Ledford , Jason Gunthorpe Return-path: Received: from mail.kernel.org ([198.145.29.99]:50292 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726995AbeKIErf (ORCPT ); Thu, 8 Nov 2018 23:47:35 -0500 In-Reply-To: <20181108191017.21891-1-leon@kernel.org> Sender: netdev-owner@vger.kernel.org List-ID: From: Moni Shoua Page fault events are processed in a workqueue context. Since each QP can have up to two concurrent unrelated page-faults, one for requester and one for responder, page-fault handling can be done in parallel. Achieve this by changing the workqueue to be multi-threaded. The number of threads is the same as the number of command interface channels to avoid command interface bottlenecks. In addition to multi-threads, change the workqueue flags to give it high priority. Stress benchmark shows that before this change 85% of page faults were waiting in queue 8 seconds or more while after the change 98% of page faults were waiting in queue 64 milliseconds or less. The number of threads was chosen as the number of channels to the command interface. Fixes: d9aaed838765 ("{net,IB}/mlx5: Refactor page fault handling") Signed-off-by: Moni Shoua Signed-off-by: Leon Romanovsky --- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index c1e1a16a9b07..aeab0c4f60f4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -356,8 +356,9 @@ static int init_pf_ctx(struct mlx5_eq_pagefault *pf_ctx, const char *name) spin_lock_init(&pf_ctx->lock); INIT_WORK(&pf_ctx->work, eq_pf_action); - pf_ctx->wq = alloc_ordered_workqueue(name, - WQ_MEM_RECLAIM); + pf_ctx->wq = alloc_workqueue(name, + WQ_HIGHPRI | WQ_UNBOUND | WQ_MEM_RECLAIM, + MLX5_NUM_CMD_EQE); if (!pf_ctx->wq) return -ENOMEM; -- 2.19.1