From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0BC3C31E40 for ; Fri, 9 Aug 2019 14:44:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9799720820 for ; Fri, 9 Aug 2019 14:44:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1565361870; bh=j082qK0VPCgjQ7rtto77M3UFMPxumwW+3n0+RuNDtXA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=i92n8hYZwU9FFAGaA+sAViDChP5WIymylDEQSnw818sWg3S96oIz9gc58fZNE58I2 +uCbmgMYb4tmvaxnSOfAntGPKv+CUAofNJhujgutZ8FWD8nWXo3ZRrDF2tRaFwoSgH CCRN7muZeCr0aFFdCxeOIhYCP9oRpuR1SOHh2akg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2407135AbfHIOo3 (ORCPT ); Fri, 9 Aug 2019 10:44:29 -0400 Received: from mga05.intel.com ([192.55.52.43]:50872 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726157AbfHIOo3 (ORCPT ); Fri, 9 Aug 2019 10:44:29 -0400 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Aug 2019 07:44:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,364,1559545200"; d="scan'208";a="169333583" Received: from unknown (HELO localhost.localdomain) ([10.232.112.69]) by orsmga008.jf.intel.com with ESMTP; 09 Aug 2019 07:44:27 -0700 Date: Fri, 9 Aug 2019 08:42:04 -0600 From: Keith Busch To: Ming Lei Cc: Thomas Gleixner , linux-kernel@vger.kernel.org, Christoph Hellwig , linux-nvme@lists.infradead.org, Jon Derrick Subject: Re: [PATCH 1/2] genirq/affinity: improve __irq_build_affinity_masks() Message-ID: <20190809144204.GA28515@localhost.localdomain> References: <20190809102310.27246-1-ming.lei@redhat.com> <20190809102310.27246-2-ming.lei@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190809102310.27246-2-ming.lei@redhat.com> User-Agent: Mutt/1.9.1 (2017-09-22) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Aug 09, 2019 at 06:23:09PM +0800, Ming Lei wrote: > One invariant of __irq_build_affinity_masks() is that all CPUs in the > specified masks( cpu_mask AND node_to_cpumask for each node) should be > covered during the spread. Even though all requested vectors have been > reached, we still need to spread vectors among left CPUs. The similar > policy has been taken in case of 'numvecs <= nodes'. > > So remove the following check inside the loop: > > if (done >= numvecs) > break; > > Meantime assign at least 1 vector for left nodes if 'numvecs' vectors > have been spread. > > Also, if the specified cpumask for one numa node is empty, simply not > spread vectors on this node. > > Cc: Christoph Hellwig > Cc: Keith Busch > Cc: linux-nvme@lists.infradead.org, > Cc: Jon Derrick > Signed-off-by: Ming Lei > --- > kernel/irq/affinity.c | 33 +++++++++++++++++++++------------ > 1 file changed, 21 insertions(+), 12 deletions(-) > > diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c > index 6fef48033f96..bc3652a2c61b 100644 > --- a/kernel/irq/affinity.c > +++ b/kernel/irq/affinity.c > @@ -129,21 +129,32 @@ static int __irq_build_affinity_masks(unsigned int startvec, > for_each_node_mask(n, nodemsk) { > unsigned int ncpus, v, vecs_to_assign, vecs_per_node; > > - /* Spread the vectors per node */ > - vecs_per_node = (numvecs - (curvec - firstvec)) / nodes; > - > /* Get the cpus on this node which are in the mask */ > cpumask_and(nmsk, cpu_mask, node_to_cpumask[n]); > - > - /* Calculate the number of cpus per vector */ > ncpus = cpumask_weight(nmsk); > + if (!ncpus) > + continue; This shouldn't be possible, right? The nodemsk we're looping wouldn't have had that node set if no CPUs intersect the node_to_cpu_mask for that node, so the resulting cpumask should always have a non-zero weight. > @@ -153,16 +164,14 @@ static int __irq_build_affinity_masks(unsigned int startvec, > } > irq_spread_init_one(&masks[curvec].mask, nmsk, > cpus_per_vec); > + if (++curvec >= last_affv) > + curvec = firstvec; I'm not so sure about wrapping the vector to share it across nodes. We have enough vectors in this path to ensure each compute node can have a unique one, and it's much cheaper to share these within nodes than across them. > } > > done += v; > - if (done >= numvecs) > - break; > - if (curvec >= last_affv) > - curvec = firstvec; > --nodes; > } > - return done; > + return done < numvecs ? done : numvecs; > }