From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75DDAC636D7 for ; Tue, 31 Jan 2023 10:22:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E457A408EF; Tue, 31 Jan 2023 10:22:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E457A408EF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YEcGuaIbDrUw; Tue, 31 Jan 2023 10:22:18 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2812B40894; Tue, 31 Jan 2023 10:22:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2812B40894 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DF339C0032; Tue, 31 Jan 2023 10:22:17 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id EBBB4C002B for ; Tue, 31 Jan 2023 10:22:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C658C6079B for ; Tue, 31 Jan 2023 10:22:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C658C6079B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XxA-6afofG1Y for ; Tue, 31 Jan 2023 10:22:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A7BC960899 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp3.osuosl.org (Postfix) with ESMTP id A7BC960899 for ; Tue, 31 Jan 2023 10:22:15 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DFD102F4; Tue, 31 Jan 2023 02:22:56 -0800 (PST) Received: from FVFF77S0Q05N (unknown [10.57.12.254]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C65F63F64C; Tue, 31 Jan 2023 02:22:12 -0800 (PST) Date: Tue, 31 Jan 2023 10:22:09 +0000 From: Mark Rutland To: Josh Poimboeuf Subject: Re: [PATCH 0/2] vhost: improve livepatch switching for heavily loaded vhost worker kthreads Message-ID: References: <20230127044355.frggdswx424kd5dq@treble> <20230127165236.rjcp6jm6csdta6z3@treble> <20230127170946.zey6xbr4sm4kvh3x@treble> <20230127221131.sdneyrlxxhc4h3fa@treble> <20230130194823.6y3rc227bvsgele4@treble> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230130194823.6y3rc227bvsgele4@treble> Cc: Petr Mladek , Joe Lawrence , kvm@vger.kernel.org, "Michael S. Tsirkin" , Peter Zijlstra , netdev@vger.kernel.org, Jiri Kosina , linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, "Seth Forshee \(DigitalOcean\)" , live-patching@vger.kernel.org, Miroslav Benes X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" On Mon, Jan 30, 2023 at 11:48:23AM -0800, Josh Poimboeuf wrote: > On Mon, Jan 30, 2023 at 06:36:32PM +0000, Mark Rutland wrote: > > On Mon, Jan 30, 2023 at 01:40:18PM +0100, Peter Zijlstra wrote: > > > On Fri, Jan 27, 2023 at 02:11:31PM -0800, Josh Poimboeuf wrote: > > > > @@ -8500,8 +8502,10 @@ EXPORT_STATIC_CALL_TRAMP(might_resched); > > > > static DEFINE_STATIC_KEY_FALSE(sk_dynamic_cond_resched); > > > > int __sched dynamic_cond_resched(void) > > > > { > > > > - if (!static_branch_unlikely(&sk_dynamic_cond_resched)) > > > > + if (!static_branch_unlikely(&sk_dynamic_cond_resched)) { > > > > + klp_sched_try_switch(); > > > > return 0; > > > > + } > > > > return __cond_resched(); > > > > } > > > > EXPORT_SYMBOL(dynamic_cond_resched); > > > > > > I would make the klp_sched_try_switch() not depend on > > > sk_dynamic_cond_resched, because __cond_resched() is not a guaranteed > > > pass through __schedule(). > > > > > > But you'll probably want to check with Mark here, this all might > > > generate crap code on arm64. > > > > IIUC here klp_sched_try_switch() is a static call, so on arm64 this'll generate > > at least a load, a conditional branch, and an indirect branch. That's not > > ideal, but I'd have to benchmark it to find out whether it's a significant > > overhead relative to the baseline of PREEMPT_DYNAMIC. > > > > For arm64 it'd be a bit nicer to have another static key check, and a call to > > __klp_sched_try_switch(). That way the static key check gets turned into a NOP > > in the common case, and the call to __klp_sched_try_switch() can be a direct > > call (potentially a tail-call if we made it return 0). > > Hm, it might be nice if our out-of-line static call implementation would > automatically do a static key check as part of static_call_cond() for > NULL-type static calls. > > But the best answer is probably to just add inline static calls to > arm64. Is the lack of objtool the only thing blocking that? The major issues were branch range limitations (and needing the linker to add PLTs), and painful instruction patching requirements (e.g. the architecture's "CMODX" rules for Concurrent MODification and eXecution of instructions). We went with the static key scheme above because that was what our assembled code generation would devolve to anyway. If we knew each call-site would only call a particular function or skip the call, then we could do better (and would probably need something like objtool to NOP that out at compile time), but since we don't know the callee at build time we can't ensure we have a PLT in range when necessary. > Objtool is now modular, so all the controversial CFG reverse engineering > is now optional, so it shouldn't be too hard to just enable objtool for > static call inlines. Funnily enough, I spent some time yesterday looking at enabling a trivial objtool for arm64 as I wanted some basic ELF rewriting functionality (to manipulate the mcount_loc table). So I'll likely be looking at that soon regardless of static calls. :) Thanks, Mark. _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization