From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9AD6AD21 for ; Fri, 26 Jan 2024 02:40:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706236832; cv=none; b=Ou/lFF6H1KmFYADadItIcilnb0fh2EA3Jkk3ZO9mOGJaC/6CcZTlMgxzd7U5CtNwpuwkGLw8lISGGcHtiQ55wme3r8/xid9pcs1fUIthcM7hnMOQeRJ+rc80ZKTgfZaXGhh7nRFvy+Xr766Ad/43qpC6ke7Tuyx7yvfRT7LKf1k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706236832; c=relaxed/simple; bh=l7G2s0EaRPs+Mgo2IZZntnCB01GO108zkohsGxkw9XI=; h=Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iDi45nGy+SnPavmLAK5ple6Kx4F4B1kIeTvEHuPw+UxdiBPqI912i4IrZ1t8ZxYjPQXaNjyk//50VMmCjJhZ6QXnKMiFKGTyZlvj/zckxwDqfNjUuYP+9AYtG9O5xCyZ/svYa7husIgtqOsCV1yGILRigWBxBRhOnPGZVZTIB94= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QnEUqWYe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QnEUqWYe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E798DC433F1; Fri, 26 Jan 2024 02:40:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706236832; bh=l7G2s0EaRPs+Mgo2IZZntnCB01GO108zkohsGxkw9XI=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=QnEUqWYey2ydukRZO+v+QhX1xioPK0IUhaXZ0NEMBAwd5nraSvq+cdkOAL7FARq6o c90S3kSNSYHv8+vw7vBlGXfLgCxJrn35xpW57uV6IgRwbv94n17lhQH6YkJvi/NicT dTkF8SfbWDlfiULsm0kQWSJ0Ykei/LiWg50C9jmZ2BAvoWfoVVIC8TukT47nt5nDN2 bbpDmYruCpYILJa7UTVvZaDF7inBsKtGN/rNjaV1Js+KU9lR0I4U/Ij+Ta5QNQcFpe 2WVXQbwX3dHpeKP4pASrlpxFAn8E5jhXgdJiXXx5602MSUNmxU7GCIlFtqutWXWZfv 2fk85AvZpSaYg== Date: Thu, 25 Jan 2024 18:40:31 -0800 From: Jakub Kicinski To: Tariq Toukan Cc: Gal Pressman , Saeed Mahameed , "David S. Miller" , Paolo Abeni , Eric Dumazet , Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan Subject: Re: [net-next 10/15] net/mlx5e: Let channels be SD-aware Message-ID: <20240125184031.5ddc5799@kernel.org> In-Reply-To: <4bb155ee-f727-449f-bd88-ba117107a88f@gmail.com> References: <20231221005721.186607-1-saeed@kernel.org> <20231221005721.186607-11-saeed@kernel.org> <20240104145041.67475695@kernel.org> <20240108190811.3ad5d259@kernel.org> <20240109080036.65634705@kernel.org> <9d29e624-fc02-44cd-9a92-01f813e66eed@nvidia.com> <4bb155ee-f727-449f-bd88-ba117107a88f@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Thu, 25 Jan 2024 10:01:05 +0200 Tariq Toukan wrote: > Exactly. That's the desired configuration. > Our driver has the logic to set it in default. > > Here's the default XPS on my setup: > > NUMA: > NUMA node(s): 2 > NUMA node0 CPU(s): 0-11 > NUMA node1 CPU(s): 12-23 > > PF0 on node0, PF1 on node1. Okay, good that you took care of the defaults, but having a queue per CPU thread is quite inefficient. Most sensible users will reconfigure your NICs and remap IRQs and XPS. Which is fine, but we need to give them the necessary info to do this right - documentation and preferably the PCIe dev mapping in the new netlink queue API.