From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B43EC43603 for ; Fri, 20 Dec 2019 13:01:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 37B6D2067C for ; Fri, 20 Dec 2019 13:01:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727434AbfLTNBI (ORCPT ); Fri, 20 Dec 2019 08:01:08 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:42262 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727344AbfLTNBI (ORCPT ); Fri, 20 Dec 2019 08:01:08 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xBKD0iEw002244 for ; Fri, 20 Dec 2019 08:01:07 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2x0vdyn4g5-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 20 Dec 2019 08:01:06 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 20 Dec 2019 13:01:04 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 20 Dec 2019 13:01:00 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xBKD0xL465732644 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 20 Dec 2019 13:00:59 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F0DD52065; Fri, 20 Dec 2019 13:00:59 +0000 (GMT) Received: from linux.vnet.ibm.com (unknown [9.126.150.29]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id DC35752057; Fri, 20 Dec 2019 13:00:56 +0000 (GMT) Date: Fri, 20 Dec 2019 18:30:56 +0530 From: Srikar Dronamraju To: Vincent Guittot Cc: Mel Gorman , Ingo Molnar , Peter Zijlstra , pauld@redhat.com, valentin.schneider@arm.com, quentin.perret@arm.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, hdanton@sina.com, parth@linux.ibm.com, riel@surriel.com, LKML Subject: Re: [PATCH] sched, fair: Allow a small degree of load imbalance between SD_NUMA domains Reply-To: Srikar Dronamraju References: <20191218154402.GF3178@techsingularity.net> <20191219144539.GA19614@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20191219144539.GA19614@linaro.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-TM-AS-GCONF: 00 x-cbid: 19122013-0028-0000-0000-000003CA9E41 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19122013-0029-0000-0000-0000248DF303 Message-Id: <20191220130056.GA13192@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-20_02:2019-12-17,2019-12-20 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 impostorscore=0 mlxscore=0 mlxlogscore=837 spamscore=0 clxscore=1015 adultscore=0 suspectscore=0 bulkscore=0 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912200103 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Vincent Guittot [2019-12-19 15:45:39]: > Hi Mel, > > Thanks for looking at this NUMA locality vs spreading tasks point. > > > Shouldn't you consider the number of busiest->idle_cpus instead of the busiest->sum_nr_running ? > and you could simplify by  > > > if ((env->sd->flags & SD_NUMA) && > ((100 * busiest->group_weight) <= (env->sd->imbalance_pct * (busiest->idle_cpus << 1)))) { > env->imbalance = 0; > return; > } Are idle_cpus and sum_nr_running good enough metrics to look at a NUMA level? We could have asymmetric NUMA topology where one DIE/MC/groups may have more cores than the other. In such a case looking at idle_cpus (or sum_nr_running) of the group may not always lead us to the right load balancing solution. -- Thanks and Regards Srikar Dronamraju