From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 898DDC433F5 for ; Fri, 4 Feb 2022 07:35:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Reply-To:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=V9x8x5vBi2yj4hV1TmVThp9QwmSUt7PIItDFFG7+hvo=; b=i3sV470OANMw9A WIrOqzan3K87GEv5PWZ3+MKP/F++6URPxBTkTNvJ1B1E3kKfecmTEzpIoBg6hga/6BLt4nHoRhYdC AqiLDnoNMitxG6pK7xZNs/zz/7SCFg5AaRcodc1krFGUz6aeUn64shtPMZhxMGHAXe+nDe+Z460+b g4Rf8fyfymuE2Oo4ulwYmjLtaws7Ls2j2RuGPucu/SRaj3VWvBKFNqw0V31twjwwbepcvwvvNadPZ +Y6HvjEI4PUHPUdx/Flv+y8J0MtHMtVMPj0rN5vzT3LknyAyuW6SYv/2ren9T14m4pzqe4raxEHX6 rSz0t85HgwjD+TNU6UUQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nFt6m-003fTA-IC; Fri, 04 Feb 2022 07:34:16 +0000 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nFt6h-003fSc-KP for linux-arm-kernel@lists.infradead.org; Fri, 04 Feb 2022 07:34:14 +0000 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 2143bvRP006656; Fri, 4 Feb 2022 07:33:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc : subject : message-id : reply-to : references : mime-version : content-type : in-reply-to; s=pp1; bh=UHDNs1wrqvVu14azFkNXA4EZ2VmaNN7Pxw8iym9ieoY=; b=ADhYGugVwtDTJ1NPtdV+GCHoNoVOk9XcHdBS9U262quqWvt1faWFELc892NJM3ht406v zvEdwxoCRiyU1JZtQwww37csFbCAo0XM9gn/JVeZCOrxnqYOdlr9vrE8Hko0dLgAaRcN 0Xo88bYlb+Z/BjBlJl7gRVqgrudHCnjSonrYtEOW2aue/sjnbaUsARaasEpI2WZR3iWR y6mn5VjR5+IQXkSqDNUvJYTiUeGIQiChjJyMf3O0KA/SuNA+MejjcP49kVtTx/SO7+T4 lsb2PJ8wYB38HvsupDTF+MBxsjLQO8c0vmtW5lizhBPPe0dnyLJfKKUKAU4fXsRVTDrb kg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e0qx0fcsf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Feb 2022 07:33:28 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 2147ETk4001431; Fri, 4 Feb 2022 07:33:28 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e0qx0fcr5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Feb 2022 07:33:27 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2147RK8U015280; Fri, 4 Feb 2022 07:33:25 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma06ams.nl.ibm.com with ESMTP id 3e0r0u2712-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Feb 2022 07:33:25 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2147XMvU27263336 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Feb 2022 07:33:22 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 84BC0AE051; Fri, 4 Feb 2022 07:33:22 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 389B9AE055; Fri, 4 Feb 2022 07:33:18 +0000 (GMT) Received: from linux.vnet.ibm.com (unknown [9.126.150.29]) by d06av26.portsmouth.uk.ibm.com (Postfix) with SMTP; Fri, 4 Feb 2022 07:33:18 +0000 (GMT) Date: Fri, 4 Feb 2022 13:03:17 +0530 From: Srikar Dronamraju To: Barry Song <21cnbao@gmail.com> Cc: "Gautham R. Shenoy" , Yicong Yang , Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Tim Chen , LKML , LAK , Dietmar Eggemann , Steven Rostedt , Ben Segall , Daniel Bristot de Oliveira , prime.zeng@huawei.com, Jonathan Cameron , ego@linux.vnet.ibm.com, Linuxarm , Barry Song , Guodong Xu Subject: Re: [PATCH v2 2/2] sched/fair: Scan cluster before scanning LLC in wake-up path Message-ID: <20220204073317.GG618915@linux.vnet.ibm.com> References: <20220126080947.4529-1-yangyicong@hisilicon.com> <20220126080947.4529-3-yangyicong@hisilicon.com> <20220128071337.GC618915@linux.vnet.ibm.com> <20220201093859.GE618915@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-TM-AS-GCONF: 00 X-Proofpoint-GUID: iiW6K_j8QyEpGA4PtSkUAgxuNgDr9QT5 X-Proofpoint-ORIG-GUID: NjYi9HlZBDXiu7voHv_wIXhgNWBjg2tA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-04_02,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 mlxlogscore=999 suspectscore=0 spamscore=0 malwarescore=0 phishscore=0 priorityscore=1501 impostorscore=0 adultscore=0 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202040037 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220203_233411_689870_B90DD613 X-CRM114-Status: GOOD ( 45.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Srikar Dronamraju Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org * Barry Song <21cnbao@gmail.com> [2022-02-02 09:20:32]: > On Tue, Feb 1, 2022 at 10:39 PM Srikar Dronamraju > wrote: > > > > * Barry Song <21cnbao@gmail.com> [2022-01-28 07:40:15]: > > > > > On Fri, Jan 28, 2022 at 8:13 PM Srikar Dronamraju > > > wrote: > > > > > > > > * Barry Song <21cnbao@gmail.com> [2022-01-28 09:21:08]: > > > > > > > > > On Fri, Jan 28, 2022 at 4:41 AM Gautham R. Shenoy > > > > > wrote: > > > > > > > > > > > > On Wed, Jan 26, 2022 at 04:09:47PM +0800, Yicong Yang wrote: > > > > > > > From: Barry Song > > > > > > > > > > I am sorry I didn't get your question. Currently the code works as below: > > > if task A wakes up task B, and task A is in LLC0 and task B is in LLC1. > > > we will scan the cluster of A before scanning the whole LLC0, in this case, > > > cluster of A is the closest sibling, so it is the better choice than other CPUs > > > which are in LLC0 but not in the cluster of A. > > > > Yes, this is right. > > > > > But we do scan all cpus of LLC0 > > > afterwards if we fail to find an idle CPU in the cluster. > > > > However my reading of the patch, before we can scan other clusters within > > the LLC (aka LLC0), we have a check in scan cluster which says > > > > /* Don't ping-pong tasks in and out cluster frequently */ > > if (cpus_share_resources(target, prev_cpu)) > > return target; > > > > My reading of this is, ignore other clusters (at this point, we know there > > are no idle CPUs in this cluster. We don't know if there are idle cpus in > > them or not) if the previous CPU and target CPU happen to be from the same > > cluster. This effectively means we are given preference to cache over idle > > CPU. > > Note we only ignore other cluster while prev_cpu and target are in same > cluster. if the condition is false, we are not ignoring other cpus. typically, > if waker is the target, and wakee is the prev_cpu, that means if they are > already in one cluster, we don't stupidly spread them in select_idle_cpu() path > as benchmark shows we are losing. so, yes, we are giving preference to > cache over CPU. We already figured out that there are no idle CPUs in this cluster. So dont we gain performance by picking a idle CPU/core in the neighbouring cluster. If there are no idle CPU/core in the neighbouring cluster, then it does make sense to fallback on the current cluster. > > > > > Or Am I still missing something? > > > > > > > > After a while, if the cluster of A gets an idle CPU and pulls B into the > > > cluster, we prefer not pushing B out of the cluster of A again though > > > there might be an idle CPU outside. as benchmark shows getting an > > > idle CPU out of the cluster of A doesn't bring performance improvement > > > but performance decreases as B might be getting in and getting out > > > the cluster of A very frequently, then cache coherence ping-pong. > > > > > > > The counter argument can be that Task A and Task B are related and were > > running on the same cluster. But Load balancer moved Task B to a different > > cluster. Now this check may cause them to continue to run on two different > > clusters, even though the underlying load balance issues may have changed. > > > > No? > > LB is much slower than select_idle_cpu(). select_idle_cpu() can dynamically > work afterwards. so it is always a dynamic balance and task migration. > > > > > > > -- > > Thanks and Regards > > Srikar Dronamraju > > Thanks > Barry -- Thanks and Regards Srikar Dronamraju _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel