From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3AA0C433EF for ; Mon, 7 Feb 2022 15:16:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eTH/mMjilYzf2N6tlSDX19MkNG8CjnDjgqLGSC1xGgY=; b=RvgnUuQ7ga2zP7 dHltbKpP9gcXGNoKOgtoZyVptHN0lVxWgS69GaAXcn8shXzQIGWWUhuB/gDMAJ+aKet48SpAViyyr BdOXq5OoabgiITSve13mUEcqOgkZAMF9z7zAM98Y9YCdUq+9Q7LNaglwnILhpAfQU+zEQKYzLWogu eKkVeTpwpXKr0qbkKwBgyp4XtLxYY90JuV8LpCCW3CX+61CgMdgKzFZG/RYhqb9bezFmdg1OHVgNB QQHD6M+6sYEnUIcHg+4Bi0zWcXJf7q1O09eHoMhqota2bge5f5pr3kdmiskF/sUw8Z6t01kPoYEzU KlAwCcMmOVvnvsxEfP9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nH5j3-00AeW8-Gh; Mon, 07 Feb 2022 15:14:45 +0000 Received: from mail-bn8nam08on20601.outbound.protection.outlook.com ([2a01:111:f400:7e8d::601] helo=NAM04-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nH5iz-00AeV7-P8 for linux-arm-kernel@lists.infradead.org; Mon, 07 Feb 2022 15:14:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Jlp7PJ7trw0mI9CIb5J+PiaXz75cSicov9rbosYHbYY8l1PucLMXT0TQDEJPSxG8b/6rg2C2PtschKIBAr7fFWmiy6dbM+GkH5Qi13wdbr1/7mfvEs4sD02aEzdQg1XSGZAsNJ/6TprAjNYWrJGi511aB64LiB4eQ2j26F0HsywOf3ub65LP58kGu256ejMl4kpky5jxW2HDuBHqSRv8kFNZR6wVNYOIvpa9D3SJ8yD2eIqa7YWQvZpi931dZV1eEoPz+cGPUG8+scsjgz9Ti8bArz1cv5qqPluEDpULlJZa1LqIVk2me/AzzFMFKWYG6YgfvCCjs9VmoFQijB+Zgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zWq5Ivu1vvt7LXD30krMpU7fMxOlcKzH9TyejkD1/24=; b=VRAJ2TAT6K6cLsHRgZhUvX62p97xHnk+hY10OLsnXZPzrtGibv3OueSoWClLEZ3d+znjDbxeDLOwCgpVWOR5Duimjv+iFbxjjFc5r6I8+vdshCwHntujqGoGqWmKSZHz+bDlS/WlS+44Ql2z6AOHHJVHTpJgwm5O/KeB8JAOebPgBZhnikAhcmTPhrIG4ymexWNvxUGzUdgUDIF3lJGpvB4T4vH03hfuNMKyy2vpMhMERunpJzDdhGQaQaDkKDmwFTVQsHNyACddD1uYXnc6rHFke2hOuCxkuFYGzWBBe6hxsEHwFjRXN0WZlde0HlFS01OsIqnmwe9iO5AAcKFV3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zWq5Ivu1vvt7LXD30krMpU7fMxOlcKzH9TyejkD1/24=; b=cOBvQq4PNFV8353+lvUjFlP8ep7l5nsmLINeGXT5e+c90JAtpLbPX97LlCMC3jIrRYPiCWcBAnFNp6DN6Fvpm9gVHKGkS7XawhWR/6TWXgHs5i5CEO5sJBsa7fhB4Y/ea7RoJ/sb3tn01ksKdcxPvb0x+otoqokV9tVS3Xgl2w0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from BYAPR12MB3286.namprd12.prod.outlook.com (2603:10b6:a03:139::15) by MN2PR12MB3168.namprd12.prod.outlook.com (2603:10b6:208:af::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.17; Mon, 7 Feb 2022 15:14:36 +0000 Received: from BYAPR12MB3286.namprd12.prod.outlook.com ([fe80::88e7:cc12:9082:743c]) by BYAPR12MB3286.namprd12.prod.outlook.com ([fe80::88e7:cc12:9082:743c%4]) with mapi id 15.20.4951.018; Mon, 7 Feb 2022 15:14:36 +0000 Date: Mon, 7 Feb 2022 20:44:22 +0530 From: "Gautham R. Shenoy" To: Barry Song <21cnbao@gmail.com> Cc: Srikar Dronamraju , Yicong Yang , Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Tim Chen , LKML , LAK , Dietmar Eggemann , Steven Rostedt , Ben Segall , Daniel Bristot de Oliveira , prime.zeng@huawei.com, Jonathan Cameron , ego@linux.vnet.ibm.com, Linuxarm , Barry Song , Guodong Xu Subject: Re: [PATCH v2 2/2] sched/fair: Scan cluster before scanning LLC in wake-up path Message-ID: References: <20220126080947.4529-1-yangyicong@hisilicon.com> <20220126080947.4529-3-yangyicong@hisilicon.com> <20220128071337.GC618915@linux.vnet.ibm.com> <20220201093859.GE618915@linux.vnet.ibm.com> <20220204073317.GG618915@linux.vnet.ibm.com> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BMXPR01CA0096.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:54::36) To BYAPR12MB3286.namprd12.prod.outlook.com (2603:10b6:a03:139::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6f93f001-5d10-4218-f17d-08d9ea4c8d2e X-MS-TrafficTypeDiagnostic: MN2PR12MB3168:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2399; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TbE7Vmy36DdWMbFdPAGA6FwKPLkgRJP8xUZR3TUxx9i9NdUcQkFuZk402sUQ2V3xbMuE6R0JjxjZPvMNxkCm3w/RGS22XCw15Cfp/JPuwUJLoLOT6pBSDEQHoOMUmBIHHWqQGpbH9qK2AYWrcXOnkNHwbG6ZBpKv5ig6L5+w5C+0w4ax8vF+qoWEmYCZL+Lpq7TIOfQBKGgdyVCEJboaVgeB7LLI1KrqJcv6cZkBF5lQKhNjh407RvwyOjqwkpQLypq9rmvjUOJF+W/b9NA1TJBLr7XteaahsEJWcEfVytm+pv/NYUDk2XuicavINJmZ0x1JYXr67G5urQuz9nBuW56NCnf0LB0YOSD1yn58zsnPIIk6koTWWvkseJdgOX3nD8+UpwA2rf8PWomjGXKn4CZbV2Wnz0WIMlvBy4tcD1pr3U1dA70UJvs4GYeCecFVcSbzZq3RahPO5PyXFA14hrBdrlFasBrcySLRFhBs+RT4VOZvMH+BW8KTDa7owOLN9oVnEp+ch/SBNGdbfh2FcAbnKdwjcvjnRqd9GyqBXRz7/RonGu9mqW+IGZCdT7Xxk4wqeLU++KXp9JFAjGe05k/8Bl5jJNqksChxmDWCT/RIhMGh7VBgO96jgFZTIwajgYRm+BLK6/p8lh4yPwi8vH0aVn0767ew99fsBIiCcSQbXiPjQfyycn3jvnct2Uf5hs9hFDICC3WanKwspNsHrajdaz0eoZfDOqZ29ZN3PdY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR12MB3286.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(508600001)(2906002)(86362001)(6486002)(186003)(5660300002)(83380400001)(66476007)(66556008)(4326008)(6506007)(8676002)(8936002)(6666004)(6512007)(66946007)(316002)(6916009)(54906003)(26005)(38100700002)(7416002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?vkxEb+fZ/LeWQJzKqEjkX0fkTCh8MI2BYxgvo8CvEmRP47ecUKcQ/d3U9elj?= =?us-ascii?Q?Z7eqUlN9LhJUAWrXoLQeuOel2AeBwQc5vYCsG+W9ng1qreUQLzFut631afKY?= =?us-ascii?Q?7n2AT6wPeiVFnI0QtYDgcIWjOpRiAnqYtjM8YhEA2msk+MZPBEHSBXOFNpCZ?= =?us-ascii?Q?uTFR+4fQAwJrW4C06/z78ijBBZn8uD52blBLK/14IIRckPW9ibVa2md8mRV6?= =?us-ascii?Q?Hsujh8WQ4DAJ6Gj8FR6yEW5Ur4ftdWqh7bOmpd3YDS9JLaI+e1tLHI+pvQ4/?= =?us-ascii?Q?xPn8MtIbn6RYpOLSCjVJJ4VpeaO1KopTyNySgukcuVmNqRtTOunAfp1RsR7C?= =?us-ascii?Q?9JIjQe5njlZgP5Z8Wewn6CmCovb9sv72uL6TWoAp0a5iIaeDFKeRhXXofVrA?= =?us-ascii?Q?HTG6mKrsn3BEptRU3QAVVGDr03B3QHffdX8jE63NJ6spoDYU0e9aCqbG8VYi?= =?us-ascii?Q?hB8JXgBZimtqWqV9gYoYLQrykTQlMIr19y/L2loPfxDIODxvWgBceiXCA2Qh?= =?us-ascii?Q?TMixxPy+5z2dEX+zdxLbd3DQcYvUPAgGcg1dWc2uRN+azKQWsfe1ZmaX8TQI?= =?us-ascii?Q?MDhb6cY+SJnpooVHXYasCR0FncdevgwKoodjO0J6VaRieUBDEjeb5A7vXf+j?= =?us-ascii?Q?bHxPgPJvfDaImYpOuZF6XRwezHpyBWivC9dbRGIA2/erRDd2cjNGxRKgIqUG?= =?us-ascii?Q?WGEuXWY/Um99I7qaVBlT8Ns8sgePJQP7LyspL8UTN0bwB0Xi3v/Ae7lUtMUF?= =?us-ascii?Q?vzsDlunyznN4ywela9/fRoTAfAmPZyU+6SXVlJABrdOGtPUVCG/IA5HLUgqv?= =?us-ascii?Q?mn9yidNgDL0axApqG8TxZ+0zWaOAFYixaoIeduLhfm/PbOPwQEPJhaig8z9Z?= =?us-ascii?Q?um2qnwulz7sReLh2iZWd10XpNurqJfrTjFeg266MPjWiC9cEwDDH8cg7Zh6R?= =?us-ascii?Q?YBqK/8d1vPnwxFZrN8f+eeo0WQo31hk4RBY3Q0K0ulKd5L3Ubm7IUZDL619M?= =?us-ascii?Q?jhhOPF7ANQ3wnMa9lqKA/lsPLca+xtenrlGFS/A9JKPMQKmWBLMQOfqJIU1Q?= =?us-ascii?Q?LvJy0kLy+jhZDiy+FbtQeFTwfCtDxr3IeHztB6HioHVpSoFHBgE1xEUcPUF+?= =?us-ascii?Q?lyxS52PyBUgqt514wlZYICoxOiZCngY5awqtEI3cSew2gvkkxvaAhIVX4l/y?= =?us-ascii?Q?9ffUn1DZTtAjR+Vnyw3ygdeB2G08OtzDLNfP+A3zQ1EBpA6CMl0FkUaMQq9g?= =?us-ascii?Q?0nBhl+UiUe52CFcvYlK14sbU7mO0fGL5wR7EuFqRJOyN2HElrcAwjQhpULIw?= =?us-ascii?Q?6D14JyedH6spxFTn9uSUovD5Xu1RoXONhF/8oHoC3O2eyVbdPwFKQ/bL4ipK?= =?us-ascii?Q?iccWf6IpITmle5aUatys7zgn5XbN7vQRveZfHntwI6p/sWIQ8uN7jKjpACTK?= =?us-ascii?Q?eGepzMp5Cuq6aVtdzb5BUcipsVJnYFV+gSnSXS1Wlk0zpmIc/OxYWT7TFY9D?= =?us-ascii?Q?5lXrauxuduClbqvUSqe9xTufq5+GorhItBMkdYOVe5yxsVGMPX3/1Z8M9ViS?= =?us-ascii?Q?AMTrAaKEtk09Knb/1THF6CRPhp6i7S1xKrCrGe4ctO5V1vSENe6/Z3/2i5wO?= =?us-ascii?Q?mONlmDPlJ+E4KHbB5lK/NNg=3D?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6f93f001-5d10-4218-f17d-08d9ea4c8d2e X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3286.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2022 15:14:36.4525 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NycS+uXuwMTLQBiJvqn6MpCyR1DkkvVXd6HMB6FgUSqLEPyyvepeaX503LzyEPjO+gFkJqJ8y803+KUVJMFkaA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3168 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220207_071441_973722_84C53381 X-CRM114-Status: GOOD ( 16.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Feb 04, 2022 at 11:28:25PM +1300, Barry Song wrote: > > We already figured out that there are no idle CPUs in this cluster. So dont > > we gain performance by picking a idle CPU/core in the neighbouring cluster. > > If there are no idle CPU/core in the neighbouring cluster, then it does make > > sense to fallback on the current cluster. > > What you suggested is exactly the approach we have tried at the first beginning > during debugging. but we didn't gain performance according to benchmark, we > were actually losing. that is why we added this line to stop ping-pong: > /* Don't ping-pong tasks in and out cluster frequently */ > if (cpus_share_resources(target, prev_cpu)) > return target; > > If we delete this, we are seeing a big loss of tbench while system > load is medium > and above. Thanks for clarifying this Barry. Indeed, if the workload is sensitive to data ping-ponging across L2 clusters, this heuristic makes sense. I was thinking of workloads that require lower tail latency, in which case exploring the larger LLC would have made more sense, assuming that the larger LLC has an idle core/CPU. In the absence of any hints from the workload, like something that Peter had previous suggested (https://lore.kernel.org/lkml/YVwnsrZWrnWHaoqN@hirez.programming.kicks-ass.net/), optimizing for cache-access seems to be the right thing to do. > > Thanks > Barry -- Thanks and Regards gautham. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel