From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB382C433F5 for ; Thu, 24 Feb 2022 23:37:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D2788D0005; Thu, 24 Feb 2022 18:37:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5824E8D0001; Thu, 24 Feb 2022 18:37:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44A468D0005; Thu, 24 Feb 2022 18:37:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 320CA8D0001 for ; Thu, 24 Feb 2022 18:37:29 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DDD0323762 for ; Thu, 24 Feb 2022 23:37:28 +0000 (UTC) X-FDA: 79179287376.09.919A160 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf16.hostedemail.com (Postfix) with ESMTP id E1483180003 for ; Thu, 24 Feb 2022 23:37:27 +0000 (UTC) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21OKrf6M014174; Thu, 24 Feb 2022 23:37:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=+4vthZ7Pmyfi2hSy5nAEFgWUJnl6DLwx6sQ3Stcr49w=; b=HKOvMJEf60QJCefIfB8i87ih5GMgdVLndZ5kSI4Y76tELxXFt9uwN6AgXIfoWfUuCIC0 L7mRSZxBYarqNBft9lZHfCh4rdodCFyElCjRziuGDUthswx7UR2ZA1U5GaWFxwzriQKQ 51sfPGL1tRHrrDrA5jQTpv9513aClb1QyplWdvf6pbPw0iqKJaYX+zg/RTEntslbAIKy 8xjEx5lBI+5Vid9BK+PMlxiB/VSMI8gzVIpyfRtRuAZ1Q4ulXi+VFabf7VYridQ/K0UG NOSzl2POxjzpimY7SHHD17qkMnLGhMNcE+N4Hvw6uravuWI9+jPq+VWORLA9R0dnAtSx Xg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3edw3ce2yb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 23:37:25 +0000 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 21ONPoMi030642; Thu, 24 Feb 2022 23:37:25 GMT Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com with ESMTP id 3edw3ce2xm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 23:37:24 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 21ONRBfs005272; Thu, 24 Feb 2022 23:37:22 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma02fra.de.ibm.com with ESMTP id 3ear69k0vp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 23:37:22 +0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 21ONbKT018678088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Feb 2022 23:37:20 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 11A855204E; Thu, 24 Feb 2022 23:37:20 +0000 (GMT) Received: from [9.43.82.21] (unknown [9.43.82.21]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id E563F5204F; Thu, 24 Feb 2022 23:37:16 +0000 (GMT) Message-ID: <4e8067e1-0574-c9d2-9d6c-d676d32071bd@linux.vnet.ibm.com> Date: Fri, 25 Feb 2022 05:07:15 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [PATCH -V11 2/9] mm/migrate: update node demotion order on hotplug events Content-Language: en-US To: Dave Hansen , Dave Hansen Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zi Yan , David Hildenbrand , Yang Shi , Huang Ying , Andrew Morton References: <20210721063926.3024591-1-ying.huang@intel.com> <20210721063926.3024591-2-ying.huang@intel.com> From: Abhishek Goel In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 9hzKxExDbYTpXwNE4mN7A5wsFDJgCAMJ X-Proofpoint-GUID: I3XXsI_SUImIYOZW-Ys2InNS1YMVbHjL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-02-24_06,2022-02-24_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 clxscore=1011 priorityscore=1501 lowpriorityscore=0 malwarescore=0 adultscore=0 phishscore=0 impostorscore=0 spamscore=0 mlxscore=0 bulkscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202240129 X-Rspamd-Queue-Id: E1483180003 X-Stat-Signature: xgxw45ifg3nmk5i6g3dkm83o83y6k47e X-Rspam-User: Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=HKOvMJEf; spf=none (imf16.hostedemail.com: domain of huntbag@linux.vnet.ibm.com has no SPF policy when checking 148.163.156.1) smtp.mailfrom=huntbag@linux.vnet.ibm.com; dmarc=pass (policy=none) header.from=ibm.com X-Rspamd-Server: rspam05 X-HE-Tag: 1645745847-667857 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 24/02/22 05:35, Dave Hansen wrote: > On 2/23/22 15:02, Abhishek Goel wrote: >> If needed, I will provide experiment results and traces that were used >> to conclude this. > It would be great if you can provide some more info. Even just a CPU > time profile would be helpful. Average total time taken for SMT=8 to SMT=1 in v5.14 : 20s Average total time taken for SMT=8 to SMT=1 in v5.15 : 36s (Observed in system with 150+ CPUs ) > > It would also be great to understand more about what "hotplug on power > systems" actually means. Is this a synthetic benchmark, or are actual > end-users running into this issue? Are entire nodes of CPUs going > offline? Or is this just doing an offline/online of CPU 22 in a 100-CPU > NUMA node? No, this is not a synthetic benchmark. This can be recreated with entire nodes of CPUs going offline. And the online/offline operations have been performed by simple scripts. The time observed can also be verified (for individual CPU or the entire system) by observing CPU- Hotplug trace which provide consistent result as observed by using the scripts.