From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D319BC67861 for ; Sat, 6 Apr 2024 18:43:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-ID:In-Reply-To: References:Message-ID:Date:CC:To:From:Subject:Reply-To:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5PXGb0IuQ4gyY/SQsOeDj+5nhaMzIVYgV4vJN0MZtY4=; b=uummeolmSWXTwg UhznUbIOkwtCJGX1XLD7aPlKdiURgSd391HsQiSeY1zZ2DvM2D/ycDJ+mfHDu+p907/q5dfLwmWsj sSmzm7lrDB/uIlsVs0nBwZLyCJZ151tyse1YYKFV2G5ezPAOWB5NoXAfMVrJVjKKrWZk/KPS/Ljsy TNhppvtp3UsqU/yZkqYhqnj6nvBW9tn/z5JM0COZnZPk7EBSGRMCQZYp+G0UMOhzQGeQK527soQbg AbFjhU53TRoo72kQxu1MDKboJnTEcqwN6LyKVDBxt/zw5sW9UsteWy2SnL214QJS+Vfa0tQ4gPer0 9RKY+OppJdarJJXN61dg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rtB0F-0000000AqDa-3KmU; Sat, 06 Apr 2024 18:42:59 +0000 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rtB0C-0000000AqAr-2AQv for linux-arm-kernel@lists.infradead.org; Sat, 06 Apr 2024 18:42:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1712428976; x=1743964976; h=from:to:cc:date:message-id:references:in-reply-to: content-id:content-transfer-encoding:mime-version:subject; bh=gLAVxu8KHoNgflhSP1o9Xg0UT8ODjoT8j/oAbHJfQyY=; b=j+uTR3A2CzdmuA8ttq9FOsukU9gxD+qkcSNPuQEOQVBejoaB/JDJiHbK X2tXGU0d2JT1czK2IePPpuB2umKtN9glz2NrJzsf6r0bJhd9IFPhS5ewj 0Zaq6x2x2YtOekImynELsColdplQrFkI6FMARVG4TDVy8Bo6kIVFoUGf1 M=; X-IronPort-AV: E=Sophos;i="6.07,183,1708387200"; d="scan'208";a="337396067" Subject: Re: [PATCH v4 7/8] cpuidle/poll_state: replace cpu_relax with smp_cond_load_relaxed Thread-Topic: [PATCH v4 7/8] cpuidle/poll_state: replace cpu_relax with smp_cond_load_relaxed Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-33001.sea14.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2024 18:42:43 +0000 Received: from EX19MTAUWB002.ant.amazon.com [10.0.7.35:27027] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.59.87:2525] with esmtp (Farcaster) id 4f2e5d76-4265-43aa-8207-7fa4ea3e82a5; Sat, 6 Apr 2024 18:42:42 +0000 (UTC) X-Farcaster-Flow-ID: 4f2e5d76-4265-43aa-8207-7fa4ea3e82a5 Received: from EX19D001UWA001.ant.amazon.com (10.13.138.214) by EX19MTAUWB002.ant.amazon.com (10.250.64.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Sat, 6 Apr 2024 18:42:39 +0000 Received: from EX19D001UWA003.ant.amazon.com (10.13.138.211) by EX19D001UWA001.ant.amazon.com (10.13.138.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.28; Sat, 6 Apr 2024 18:42:38 +0000 Received: from EX19D001UWA003.ant.amazon.com ([fe80::256a:26de:3ee6:48a2]) by EX19D001UWA003.ant.amazon.com ([fe80::256a:26de:3ee6:48a2%7]) with mapi id 15.02.1258.028; Sat, 6 Apr 2024 18:42:38 +0000 From: "Okanovic, Haris" To: "ankur.a.arora@oracle.com" CC: "joao.m.martins@oracle.com" , "kvm@vger.kernel.org" , "dianders@chromium.org" , "linux-arm-kernel@lists.infradead.org" , "pmladek@suse.com" , "wanpengli@tencent.com" , "akpm@linux-foundation.org" , "linux-kernel@vger.kernel.org" , "catalin.marinas@arm.com" , "mingo@redhat.com" , "pbonzini@redhat.com" , "tglx@linutronix.de" , "daniel.lezcano@linaro.org" , "mihai.carabas@oracle.com" , "arnd@arndb.de" , "will@kernel.org" , "hpa@zytor.com" , "peterz@infradead.org" , "mic@digikod.net" , "vkuznets@redhat.com" , "bp@alien8.de" , "npiggin@gmail.com" , "linux-pm@vger.kernel.org" , "rafael@kernel.org" , "juerg.haefliger@canonical.com" , "x86@kernel.org" , "rick.p.edgecombe@intel.com" Thread-Index: AQHahgAj0rMfU3rQtUK6VuthoD2QJ7FaOuMAgAAXOYCAAUWwAA== Date: Sat, 6 Apr 2024 18:42:38 +0000 Message-ID: References: <1707982910-27680-1-git-send-email-mihai.carabas@oracle.com> <1707982910-27680-8-git-send-email-mihai.carabas@oracle.com> <7f3e540ad30f40ae51f1abda24b1bea2c8b648ea.camel@amazon.com> <87r0fjtn9y.fsf@oracle.com> In-Reply-To: <87r0fjtn9y.fsf@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.136.65.60] Content-ID: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240406_114256_616220_6EA18A76 X-CRM114-Status: GOOD ( 36.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 2024-04-05 at 16:14 -0700, Ankur Arora wrote: > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Okanovic, Haris writes: > > > On Thu, 2024-02-15 at 09:41 +0200, Mihai Carabas wrote: > > > cpu_relax on ARM64 does a simple "yield". Thus we replace it with > > > smp_cond_load_relaxed which basically does a "wfe". > > > > > > Suggested-by: Peter Zijlstra > > > Signed-off-by: Mihai Carabas > > > --- > > > drivers/cpuidle/poll_state.c | 15 ++++++++++----- > > > 1 file changed, 10 insertions(+), 5 deletions(-) > > > > > > diff --git a/drivers/cpuidle/poll_state.c b/drivers/cpuidle/poll_state.c > > > index 9b6d90a72601..1e45be906e72 100644 > > > --- a/drivers/cpuidle/poll_state.c > > > +++ b/drivers/cpuidle/poll_state.c > > > @@ -13,6 +13,7 @@ > > > static int __cpuidle poll_idle(struct cpuidle_device *dev, > > > struct cpuidle_driver *drv, int index) > > > { > > > + unsigned long ret; > > > u64 time_start; > > > > > > time_start = local_clock_noinstr(); > > > @@ -26,12 +27,16 @@ static int __cpuidle poll_idle(struct cpuidle_device *dev, > > > > > > limit = cpuidle_poll_time(drv, dev); > > > > > > - while (!need_resched()) { > > > - cpu_relax(); > > > - if (loop_count++ < POLL_IDLE_RELAX_COUNT) > > > - continue; > > > - > > > + for (;;) { > > > loop_count = 0; > > > + > > > + ret = smp_cond_load_relaxed(¤t_thread_info()->flags, > > > + VAL & _TIF_NEED_RESCHED || > > > + loop_count++ >= POLL_IDLE_RELAX_COUNT); > > > > Is it necessary to repeat this 200 times with a wfe poll? > > The POLL_IDLE_RELAX_COUNT is there because on x86 each cpu_relax() > iteration is much shorter. > > With WFE, it makes less sense. > > > Does kvm not implement a timeout period? > > Not yet, but it does become more useful after a WFE haltpoll is > available on ARM64. Note that kvm conditionally traps WFE and WFI based on number of host CPU tasks. VMs will sometimes see hardware behavior - potentially polling for a long time before entering WFI. https://elixir.bootlin.com/linux/latest/source/arch/arm64/kvm/arm.c#L459 > > Haltpoll does have a timeout, which you should be able to tune via > /sys/module/haltpoll/parameters/ but that, of course, won't help here. > > > Could you make it configurable? This patch improves certain workloads > > on AWS Graviton instances as well, but blocks up to 6ms in 200 * 30us > > increments before going to wfi, which is a bit excessive. > > Yeah, this looks like a problem. We could solve it by making it an > architectural parameter. Though I worry about ARM platforms with > much smaller default timeouts. > The other possibility is using WFET in the primitive, but then we > have that dependency and that's a bigger change. See arm64's delay() for inspiration: https://elixir.bootlin.com/linux/v6.9-rc2/source/arch/arm64/lib/delay.c#L26 > > Will address this in the next version. > > Thanks for pointing this out. > > -- > ankur _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel