From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CA74C433ED for ; Wed, 28 Apr 2021 12:50:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 683CF61408 for ; Wed, 28 Apr 2021 12:50:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239608AbhD1Mu4 (ORCPT ); Wed, 28 Apr 2021 08:50:56 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:30102 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239585AbhD1Mut (ORCPT ); Wed, 28 Apr 2021 08:50:49 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 13SCXH8k157896; Wed, 28 Apr 2021 08:49:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc : subject : message-id : reply-to : references : mime-version : content-type : in-reply-to; s=pp1; bh=IDjg4GaukE4TS0CR+66TfofNd5gjbr0y7g4QfU5cIn4=; b=Tsra2k9C8Z8NyR22sHpVOsLYQ7/xF1Rh7UNTWFHpWPZkMFYZ4efipoKhjLjxp8RC9ivD nZ3TtsVowhG2fvTgwll/W/xBvTo/zXyI+16W3jVfA3kEKJnOeuO1WpQzjTATKLkikrEx xa8vwEaxIaVgYNec/rHULY/vBLDUK9Nxz1vFNzJOe8sKqDwyPyqxEEV7GkGZjtV8mpGE RWek/x0OKE1LDsXLg8/9FFZQH7C1Br2QtURffJQQ7s5uMzcS8ETosHhIpM/Xfw+YFY0D qu2SYAmrhxuLYHBcdj0Ju31yIc1YfjWefIjOiGGU2VxaUI+8c9QmLCIfwsKeOqgUcEtQ pQ== Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 3877hds1dj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Apr 2021 08:49:34 -0400 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.0.43/8.16.0.43) with SMTP id 13SCSR7d000326; Wed, 28 Apr 2021 12:49:32 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma06ams.nl.ibm.com with ESMTP id 384akh9vxc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Apr 2021 12:49:31 +0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 13SCnT4X33030630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Apr 2021 12:49:29 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 123554203F; Wed, 28 Apr 2021 12:49:29 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A6EE942041; Wed, 28 Apr 2021 12:49:26 +0000 (GMT) Received: from linux.vnet.ibm.com (unknown [9.126.150.29]) by d06av24.portsmouth.uk.ibm.com (Postfix) with SMTP; Wed, 28 Apr 2021 12:49:26 +0000 (GMT) Date: Wed, 28 Apr 2021 18:19:25 +0530 From: Srikar Dronamraju To: Vincent Guittot Cc: Ingo Molnar , Peter Zijlstra , LKML , Mel Gorman , Rik van Riel , Thomas Gleixner , Valentin Schneider , Dietmar Eggemann , Michael Ellerman , Gautham R Shenoy , Parth Shah Subject: Re: [PATCH 00/10] sched/fair: wake_affine improvements Message-ID: <20210428124925.GM2633526@linux.vnet.ibm.com> Reply-To: Srikar Dronamraju References: <20210422102326.35889-1-srikar@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: X-TM-AS-GCONF: 00 X-Proofpoint-GUID: A-U07s19itrnCdzBqc1gUKaVyDEl01Zt X-Proofpoint-ORIG-GUID: A-U07s19itrnCdzBqc1gUKaVyDEl01Zt X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761 definitions=2021-04-28_06:2021-04-27,2021-04-28 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 priorityscore=1501 malwarescore=0 impostorscore=0 lowpriorityscore=0 adultscore=0 mlxlogscore=545 mlxscore=0 suspectscore=0 clxscore=1015 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104280084 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Vincent Guittot [2021-04-27 16:52:30]: > Hi Srikar, Hi Vincent, > On Thu, 22 Apr 2021 at 12:23, Srikar Dronamraju > wrote: > > > > Recently we found that some of the benchmark numbers on Power10 were lesser > > than expected. Some analysis showed that the problem lies in the fact that > > L2-Cache on Power10 is at core level i.e only 4 threads share the L2-cache. > > > > > > Summary: > > mc-llc outperforms, this patchset and upstream almost give similar performance. > > So mc-llc patch seems to be the best approach IMHO. Although the > hemisphere don't share cache, they share enough resources so > cache-snooping is as efficient as sharing cache > Yes, mc-llc helps just specific systems like Power10 but its shows better numbers than my posted patchset. However in this patchset, we are looking at areas in wakeup (aka idler llcs) we could optimize which can help other archs too. + the fallback mechanism is generic enough that we could use it for other Systems too. I know that there are valid concerns raised by Mel and I working to resolve them. Some of them are. - How hot is idle-core - Crashes when running tbench (I was able to reproduce with kernbench on x86) Also I am adding some more changes with which we are getting similar performance as mc-llc. > > Cc: LKML > > Cc: Michael Ellerman > > Cc: Gautham R Shenoy > > Cc: Parth Shah > > Cc: Ingo Molnar > > Cc: Peter Zijlstra > > Cc: Valentin Schneider > > Cc: Dietmar Eggemann > > Cc: Mel Gorman > > Cc: Vincent Guittot > > Cc: Rik van Riel > > > > Srikar Dronamraju (10): > > sched/fair: Update affine statistics when needed > > sched/fair: Maintain the identity of idle-core > > sched/fair: Update idle-core more often > > sched/fair: Prefer idle CPU to cache affinity > > sched/fair: Call wake_affine only if necessary > > sched/idle: Move busy_cpu accounting to idle callback > > sched/fair: Remove ifdefs in waker_affine_idler_llc > > sched/fair: Dont iterate if no idle CPUs > > sched/topology: Introduce fallback LLC > > powerpc/smp: Add fallback flag to powerpc MC domain > > > > arch/powerpc/kernel/smp.c | 7 +- > > include/linux/sched/sd_flags.h | 7 + > > include/linux/sched/topology.h | 3 +- > > kernel/sched/fair.c | 229 +++++++++++++++++++++++++++------ > > kernel/sched/features.h | 1 + > > kernel/sched/idle.c | 33 ++++- > > kernel/sched/sched.h | 6 + > > kernel/sched/topology.c | 54 +++++++- > > 8 files changed, 296 insertions(+), 44 deletions(-) > > > > -- > > 2.18.2 > > -- Thanks and Regards Srikar Dronamraju