From mboxrd@z Thu Jan 1 00:00:00 1970 From: Prarit Bhargava Subject: Re: [PATCH 0/2] ixgbe, fix numa issues Date: Tue, 25 Feb 2014 10:13:24 -0500 Message-ID: <530CB314.2020907@redhat.com> References: <1393267913-28212-1-git-send-email-prarit@redhat.com> <530B9C3E.1000308@intel.com> <530B9EC9.4080007@redhat.com> <530BA40E.3060008@intel.com> <530BEC90.2010705@redhat.com> <063D6719AE5E284EB5DD2968C1650D6D0F6CA75F@AcuExch.aculab.com> <530C77DE.8050409@redhat.com> <530CB279.4010307@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: David Laight , "netdev@vger.kernel.org" , Jeff Kirsher , Jesse Brandeburg , Bruce Allan , Carolyn Wyborny , Don Skidmore , Greg Rose , John Ronciak , Mitch Williams , "David S. Miller" , "nhorman@redhat.com" , "agospoda@redhat.com" , "e1000-devel@lists.sourceforge.net" To: Alexander Duyck Return-path: Received: from mx1.redhat.com ([209.132.183.28]:23828 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752521AbaBYPNr (ORCPT ); Tue, 25 Feb 2014 10:13:47 -0500 In-Reply-To: <530CB279.4010307@intel.com> Sender: netdev-owner@vger.kernel.org List-ID: On 02/25/2014 10:10 AM, Alexander Duyck wrote: > On 02/25/2014 03:00 AM, Prarit Bhargava wrote: >> >> On 02/25/2014 05:21 AM, David Laight wrote: >>> From: Prarit Bhargava >>> ... >>>> What has caused that check to be necessary is that the ixgbe driver is now >>>> allocating so many interrupts that on large systems which full sockets are taken >>>> in and out of service, it is possible that there are not enough empty vectors >>>> for all the irqs on a down'd cpu. IMO what the ixgbe driver is effectively >>>> doing is starving the system of resources. If I rmmod the ixgbe driver (and >>>> free it's irqs of course) I have no problem in taking all cpus except 1 out of >>>> service. >>> If I read that correctly it looks as though ixgbe should be allocating >>> a number of interrupts on each cpu - for the interrupts it wants to take >>> on that cpu. >> Yes, the code currently does it. >> >>> Then taking the cpu out of service would 'just' require that the interrupts >>> that are tied to that cpu be removed first? >> Yes, that would happen with a cpu notifier (I've already written a simple dummy >> one that just printk's when called). I started to implement a single queue >> teardown but hit some of these enumeration issues. I'd like to fix these first >> and then get to the teardown. >> >> P. >> >> > > What should happen if you attempt to remove the CPU the root complex is > attached to? Will that trigger a remove via the PCIe complex being removed? I haven't tried that yet :), but my understanding is that the remove will be triggered. P. > > Thanks, > > Alex