From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEA01C282E7 for ; Sun, 14 Aug 2022 15:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241074AbiHNPiw (ORCPT ); Sun, 14 Aug 2022 11:38:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240914AbiHNPhB (ORCPT ); Sun, 14 Aug 2022 11:37:01 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D61D8205D5; Sun, 14 Aug 2022 08:32:16 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 68C6E60C94; Sun, 14 Aug 2022 15:32:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D6D2C433D7; Sun, 14 Aug 2022 15:32:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1660491135; bh=fAjjiAokNNwjXjiivz1hDMQr+piTtzKlQa31AnYrtBI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i8jDu8MF+KRR+lnRYnNZ5YbE0ZTdH/U1WmGxgUd7FjEWYz9F5vbqJJrYcpS/9ebeI 7E8Q7tsCR0++gRnAEo5H4hdTg81XG6l2s7r3jeyrVlrG8/QKB4xQSkGyIydEyrDCHx 5xtXzMl8ob1ztASvWlG3ModEi18gJHinfxbRRqD2Tf8rdBjVqBh9zu2fACtmOyr330 x1Lym6AmBA/wEeoTnny3b5RTha05cGcvEU3x6Z39e3zHpz8DyWulCDRkMyB0QuqDDM ENFQGLALzLCt+wh54ZTL77KtaZuh31vpgsOyytbuO8Atn4NG6djSdffj5cWzS1sbwg h3MvXEpPz/SvA== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Bob Pearson , Jason Gunthorpe , Sasha Levin , zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Subject: [PATCH AUTOSEL 5.18 42/56] RDMA/rxe: Limit the number of calls to each tasklet Date: Sun, 14 Aug 2022 11:30:12 -0400 Message-Id: <20220814153026.2377377-42-sashal@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220814153026.2377377-1-sashal@kernel.org> References: <20220814153026.2377377-1-sashal@kernel.org> MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Bob Pearson [ Upstream commit eff6d998ca297cb0b2e53b032a56cf8e04dd8b17 ] Limit the maximum number of calls to each tasklet from rxe_do_task() before yielding the cpu. When the limit is reached reschedule the tasklet and exit the calling loop. This patch prevents one tasklet from consuming 100% of a cpu core and causing a deadlock or soft lockup. Link: https://lore.kernel.org/r/20220630190425.2251-9-rpearsonhpe@gmail.com Signed-off-by: Bob Pearson Signed-off-by: Jason Gunthorpe Signed-off-by: Sasha Levin --- drivers/infiniband/sw/rxe/rxe_param.h | 6 ++++++ drivers/infiniband/sw/rxe/rxe_task.c | 16 ++++++++++++---- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index 918270e34a35..4d85c9496a95 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -107,6 +107,12 @@ enum rxe_device_param { RXE_INFLIGHT_SKBS_PER_QP_HIGH = 64, RXE_INFLIGHT_SKBS_PER_QP_LOW = 16, + /* Max number of interations of each tasklet + * before yielding the cpu to let other + * work make progress + */ + RXE_MAX_ITERATIONS = 1024, + /* Delay before calling arbiter timer */ RXE_NSEC_ARB_TIMER_DELAY = 200, diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 0c4db5bb17d7..2248cf33d776 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -8,7 +8,7 @@ #include #include -#include "rxe_task.h" +#include "rxe.h" int __rxe_do_task(struct rxe_task *task) @@ -33,6 +33,7 @@ void rxe_do_task(struct tasklet_struct *t) int cont; int ret; struct rxe_task *task = from_tasklet(task, t, tasklet); + unsigned int iterations = RXE_MAX_ITERATIONS; spin_lock_bh(&task->state_lock); switch (task->state) { @@ -61,13 +62,20 @@ void rxe_do_task(struct tasklet_struct *t) spin_lock_bh(&task->state_lock); switch (task->state) { case TASK_STATE_BUSY: - if (ret) + if (ret) { task->state = TASK_STATE_START; - else + } else if (iterations--) { cont = 1; + } else { + /* reschedule the tasklet and exit + * the loop to give up the cpu + */ + tasklet_schedule(&task->tasklet); + task->state = TASK_STATE_START; + } break; - /* soneone tried to run the task since the last time we called + /* someone tried to run the task since the last time we called * func, so we will call one more time regardless of the * return value */ -- 2.35.1