From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jes Sorensen Subject: Re: mlx5 broken affinity Date: Thu, 2 Nov 2017 10:48:07 -0400 Message-ID: <556f3ff5-c1d4-25c6-7bfc-9866c0d9b323@fb.com> References: <2187e555-2c4e-ef55-1c3a-17f5af54d762@fb.com> <573060a9-19a7-9133-ef52-fa947088dabb@fb.com> <3af0c164-8dde-b6f0-45e1-edbbb28e7f73@mellanox.com> <83d3944f-8a31-eb31-93db-294906630b0e@grimberg.me> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Cc: Networking , Leon Romanovsky , Saeed Mahameed , Kernel Team , Christoph Hellwig , Thomas Gleixner To: Sagi Grimberg , Tariq Toukan , Saeed Mahameed Return-path: Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:33106 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933034AbdKBOsk (ORCPT ); Thu, 2 Nov 2017 10:48:40 -0400 In-Reply-To: <83d3944f-8a31-eb31-93db-294906630b0e@grimberg.me> Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 11/02/2017 06:08 AM, Sagi Grimberg wrote: > >>>> I vaguely remember Nacking Sagi's patch as we knew it would break >>>> mlx5e netdev affinity assumptions. >> I remember that argument. Still the series found its way in. > > Of course it maid its way in, it was acked by three different > maintainers, and I addressed all of Saeed's comments. > >> That series moves affinity decisions to kernel's responsibility. >> AFAI see, what kernel does is assign IRQs to the NUMA's one by one in >> increasing indexing (starting with cores of NUMA #0), no matter what >> NUMA is closer to the NIC. > > Well, as we said before, if there is a good argument to do the home node > first we can change the generic code (as it should be given that this is > absolutely not device specific). > >> This means that if your NIC is on NUMA #1, and you reduce the number >> of channels, you might end up working only with the cores on the far >> NUMA. Not good! > We deliberated on this before, and concluded that application affinity > and device affinity are equally important. If you have a real use case > that shows otherwise, its perfectly doable to start from the device home > node. This wasn't to start a debate about which allocation method is the perfect solution. I am perfectly happy with the new default, the part that is broken is to take away the user's option to reassign the affinity. That is a bug and it needs to be fixed! Jes