From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1ED31288C1C; Wed, 11 Jun 2025 14:10:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749651032; cv=none; b=ZZBlVAycUC+aNGt3hKX3xcARizjcYfnuzcVKta2wEhDUziYqTVfWUhxKrq1XoM2gaWY7yavydeQl+0lKcn24/G2mMTMR+0FWfuRQL1GxQoLtgRc4T2UQb32pLbMo30rzgUm/u41cERFRRs/rVk2Xiq6EvJ+P5fO8AioN7jedV/4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749651032; c=relaxed/simple; bh=O8b1iR0BorA2fQJLiWlvFsiX8GqxGChfquVY8SFKOys=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=BgxJ3YEC9wpQUgtBdP5gRvb6rESJ6doRXZw20tqsDMBxeav05bo2rOnn6dW0gsqnywh0Wn7+2uP6/xSyTYomkqz8n2KfNlRwAn1a2Iddngp91k9fm7DYzj7FXxCPwWKO7HXqfmoMPhRQh00BBqZ4wpkcWLrD1miZ9uCehK29dBc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=fuhBF8uS; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="fuhBF8uS" Received: by linux.microsoft.com (Postfix, from userid 1134) id BAF7F2115191; Wed, 11 Jun 2025 07:10:30 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com BAF7F2115191 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1749651030; bh=OvAAbL/c+Rwk797WDOKhGH2iI5RVdUOq6aNBguP1zco=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fuhBF8uSSAqNfKHjAxHp9ZSmTHWoDUTOnnskDgrB4ULHwzAm7MMSvwmAdAQAdALXo sD7Pjm1olGVP0TUW9Ce9u1FVxmdAzP6f0WYhlaQVnoM+iR3Sd65iVZBsaSjVJgPt8o p/hpzn9dqIDQhFNtp3TFG1WMtj1/thHI1quJefx4= From: Shradha Gupta To: Dexuan Cui , Wei Liu , Haiyang Zhang , "K. Y. Srinivasan" , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Konstantin Taranov , Simon Horman , Leon Romanovsky , Maxim Levitsky , Erni Sri Satya Vennela , Peter Zijlstra , Michael Kelley Cc: Yury Norov , linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Nipun Gupta , Jason Gunthorpe , Jonathan Cameron , Anna-Maria Behnsen , Shradha Gupta , Kevin Tian , Long Li , Thomas Gleixner , Bjorn Helgaas , Rob Herring , Manivannan Sadhasivam , =?UTF-8?q?Krzysztof=20Wilczy=EF=BF=BD=7EDski?= , Lorenzo Pieralisi , netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Paul Rosswurm , Shradha Gupta Subject: [PATCH v6 3/5] net: mana: explain irq_setup() algorithm Date: Wed, 11 Jun 2025 07:10:29 -0700 Message-Id: <1749651029-9790-1-git-send-email-shradhagupta@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com> References: <1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Yury Norov Commit 91bfe210e196 ("net: mana: add a function to spread IRQs per CPUs") added the irq_setup() function that distributes IRQs on CPUs according to a tricky heuristic. The corresponding commit message explains the heuristic. Duplicate it in the source code to make available for readers without digging git in history. Also, add more detailed explanation about how the heuristics is implemented. Signed-off-by: Yury Norov Signed-off-by: Shradha Gupta --- Changes in v5: * Corrected the Authur of the patch --- .../net/ethernet/microsoft/mana/gdma_main.c | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c index 3504507477c6..6c4e143972a1 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -1288,6 +1288,47 @@ void mana_gd_free_res_map(struct gdma_resource *r) r->size = 0; } +/* + * Spread on CPUs with the following heuristics: + * + * 1. No more than one IRQ per CPU, if possible; + * 2. NUMA locality is the second priority; + * 3. Sibling dislocality is the last priority. + * + * Let's consider this topology: + * + * Node 0 1 + * Core 0 1 2 3 + * CPU 0 1 2 3 4 5 6 7 + * + * The most performant IRQ distribution based on the above topology + * and heuristics may look like this: + * + * IRQ Nodes Cores CPUs + * 0 1 0 0-1 + * 1 1 1 2-3 + * 2 1 0 0-1 + * 3 1 1 2-3 + * 4 2 2 4-5 + * 5 2 3 6-7 + * 6 2 2 4-5 + * 7 2 3 6-7 + * + * The heuristics is implemented as follows. + * + * The outer for_each() loop resets the 'weight' to the actual number + * of CPUs in the hop. Then inner for_each() loop decrements it by the + * number of sibling groups (cores) while assigning first set of IRQs + * to each group. IRQs 0 and 1 above are distributed this way. + * + * Now, because NUMA locality is more important, we should walk the + * same set of siblings and assign 2nd set of IRQs (2 and 3), and it's + * implemented by the medium while() loop. We do like this unless the + * number of IRQs assigned on this hop will not become equal to number + * of CPUs in the hop (weight == 0). Then we switch to the next hop and + * do the same thing. + */ + static int irq_setup(unsigned int *irqs, unsigned int len, int node) { const struct cpumask *next, *prev = cpu_none_mask; -- 2.34.1