From mboxrd@z Thu Jan 1 00:00:00 1970 From: Prarit Bhargava Subject: Re: [PATCH 1/2] ixgbe, make interrupt allocations NUMA aware Date: Mon, 24 Feb 2014 14:39:58 -0500 Message-ID: <530BA00E.4070802@redhat.com> References: <1393267913-28212-1-git-send-email-prarit@redhat.com> <1393267913-28212-2-git-send-email-prarit@redhat.com> <530B9CE8.2070800@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: nhorman@redhat.com, e1000-devel@lists.sourceforge.net, netdev@vger.kernel.org, Bruce Allan , Jesse Brandeburg , agospoda@redhat.com, John Ronciak , "David S. Miller" To: Alexander Duyck Return-path: In-Reply-To: <530B9CE8.2070800@intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: e1000-devel-bounces@lists.sourceforge.net List-Id: netdev.vger.kernel.org On 02/24/2014 02:26 PM, Alexander Duyck wrote: > On 02/24/2014 10:51 AM, Prarit Bhargava wrote: >> The ixgbe driver creates one queue/cpu on the system in order to spread >> work out on all cpus rather than restricting work to a single cpu. This >> model, while efficient, does not take into account the NUMA configuration >> of the system. >> >> This patch introduces ixgbe_num_cpus() which returns >> the number of online cpus if the adapter's PCI device has no NUMA >> restrictions, and the number of cpus in the node if the PCI device is >> allocated to a specific node. >> >> Signed-off-by: Prarit Bhargava >> Cc: Jeff Kirsher >> Cc: Jesse Brandeburg >> Cc: Bruce Allan >> Cc: Carolyn Wyborny >> Cc: Don Skidmore >> Cc: Greg Rose >> Cc: Alex Duyck >> Cc: John Ronciak >> Cc: Mitch Williams >> Cc: "David S. Miller" >> Cc: nhorman@redhat.com >> Cc: agospoda@redhat.com >> Cc: e1000-devel@lists.sourceforge.net >> --- >> drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2 ++ >> drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 28 +++++++++++++++++++++--- >> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 ++--- >> drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 5 +++-- >> 4 files changed, 33 insertions(+), 8 deletions(-) >> > > [...] > >> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c >> index 18076c4..b68a6e9 100644 >> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c >> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c >> @@ -4953,13 +4953,13 @@ static int ixgbe_sw_init(struct ixgbe_adapter *adapter) >> hw->subsystem_device_id = pdev->subsystem_device; >> >> /* Set common capability flags and settings */ >> - rss = min_t(int, IXGBE_MAX_RSS_INDICES, num_online_cpus()); >> + rss = min_t(int, IXGBE_MAX_RSS_INDICES, ixgbe_num_cpus(adapter)); >> adapter->ring_feature[RING_F_RSS].limit = rss; >> adapter->flags2 |= IXGBE_FLAG2_RSC_CAPABLE; >> adapter->flags2 |= IXGBE_FLAG2_RSC_ENABLED; >> adapter->max_q_vectors = MAX_Q_VECTORS_82599; >> adapter->atr_sample_rate = 20; >> - fdir = min_t(int, IXGBE_MAX_FDIR_INDICES, num_online_cpus()); >> + fdir = min_t(int, IXGBE_MAX_FDIR_INDICES, ixgbe_num_cpus(adapter)); >> adapter->ring_feature[RING_F_FDIR].limit = fdir; >> adapter->fdir_pballoc = IXGBE_FDIR_PBALLOC_64K; >> #ifdef CONFIG_IXGBE_DCA > > This is the one bit I object to in this patch. The flow director queue > count should be equal to the number of online CPUs, or at least as close > to it as the hardware can get. Otherwise ATR is completely useless. I'm reading up on ATR now and I see your point completely. I will remove this chunk in V2. OOC, however, what about my concern with ATR & the location of the PCI device (on a different root bridge)? Isn't that a concern with ATR or am I missing something with the overall scheme of ATR? P. > > Thanks, > > Alex ------------------------------------------------------------------------------ Flow-based real-time traffic analytics software. Cisco certified tool. Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer Customize your own dashboards, set traffic alerts and generate reports. Network behavioral analysis & security monitoring. All-in-one tool. http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired