From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 247BC15E99 for ; Mon, 30 Oct 2023 12:36:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="T/eyqJHU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2B04C433C7; Mon, 30 Oct 2023 12:36:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698669373; bh=/71kbiytdp/VoW3kuCJ1kqiDxE4ZD+L4PqRlfCczo0I=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=T/eyqJHUuAonHxnuxkt6GEdACiAWxX9iI1QzI/1RvlpMqU+QIDFtVwJ77UdO0XvJC whof5OokBZX4wF9CnXOJilVIpV5+6Vm3nfMb9Dmn0iL21Ijq/oVFyOmGXToQ3b79uo +Px4uqo3N8WAbbkz+P6kLR1kTuGBpuT5r4SsJf1bH41QU76hfWLInxSyk8xNcNm5ss HbgM4IgDJY+Mr5i+VpNTJ3FnH/IQ+f10OD1v+xEPcU64+kY2ArVy3bmKZxvB7i6q4I k47oIEq2mQDF6WbGwAJZpWuAyBW/R5SZ9/+zn6LmcoBGl3Nm5hIALsSVer6hj82+ad 4rxYLEkan3pew== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qxRV5-008vI6-5Z; Mon, 30 Oct 2023 12:36:11 +0000 Date: Mon, 30 Oct 2023 12:36:09 +0000 Message-ID: <86msw01e4m.wl-maz@kernel.org> From: Marc Zyngier To: Jan Henrik Weinstock Cc: oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, Lukas =?UTF-8?B?SsO8bmdlcg==?= Subject: Re: KVM exit to userspace on WFI In-Reply-To: References: <87ttql5aq7.wl-maz@kernel.org> <86cyx250w9.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: jan@mwa.re, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, lukas@mwa.re X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false [please make an effort not to top-post] On Fri, 27 Oct 2023 18:41:44 +0100, Jan Henrik Weinstock wrote: > > Hi Marc, > > the basic idea behind this is to have a (single-threaded) execution loop, > something like this: > > vcpu-thread: vcpu-run | process-io-devices | vcpu-run | process-io... > ^ > WFX or timeout > > We switch to simulating IO devices whenever the vcpu is idle (wfi) or exceeds > a certain budget of instructions (counted via pmu). Our fallback currently is > to kick the vcpu out of its execution using a signal (via a timeout/alarm). But > of course, if the cpu is stuck at a wfi, we are wasting a lot of time. > > I understand that the proposed behavior is not desirable for most use cases, > which is why I suggest locking it behind a flag, e.g. > KVM_ARCH_FLAG_WFX_EXIT_TO_USER. But how do you reconcile the fact that exposing this to userspace breaks fundamental expectations that the guest has, such as getting its timer interrupts and directly injected LPIs? Implementing WFI in userspace breaks it. What about the case where we don't trap WFx and let the *guest* wait for an interrupt? Honestly, what you are describing seems to be a use model that doesn't fit KVM, which is a general purpose hypervisor, but more a simulation environment. Yes, the primitives are the same, but the plumbing is wildly different. *If* that's the stuff you're looking at, then I'm afraid you'll have to do it in different way, because what you are suggesting is fundamentally incompatible with the guarantees that KVM gives to guest and userspace. Because your KVM_ARCH_FLAG_WFX_EXIT_TO_USER is really a lie. It should really be named something more along the lines of KVM_ARCH_FLAG_WFX_EXIT_TO_USER_SOMETIME_AND_I_DONT_EVEN_KNOW_WHEN (probably with additional clauses related to breaking things). Overall, you are still asking for something that is not guaranteed at the architecture level, even less in KVM, and I'm not going to add support for something that can only work "sometime". M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D0980C4332F for ; Mon, 30 Oct 2023 12:36:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=C8qmyVf1E+JeBFk3mjPKMbbEc5gPQD69oZLaKZuVbF0=; b=3XC9zd/0cW0u95 jKQfjCKWOc5D/jVEIPwGsIAciO9pX95fG7yRAKQpV6AeF3RxcldN0gSk7msxzsvgbFGjMrHhmUrQO ZIZUGWGrAR61ucAqCozeI4OxndKyFIqcN0zcMhtAueWHNXA7UIs7Qhz5xCEPRXOuT3vNJ68KrulSq dNIoiM2rmlJsgZGGdLeiop5Vl4UPRcDwwGotjAlisqpYxGbMop7AMfFTfBeoYUQBQ6xsP67mt274R ccNzvxZ2A+Qpj+iPt7hKkgiNYcKDE7aaOxt99Eyg1pFNmT+8fX7c2QZjzq1aAvmwgWMz9qsgdBIlA xF+zqKTW/MTbwUjqiLFw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qxRVB-003LQV-2r; Mon, 30 Oct 2023 12:36:17 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qxRV8-003LQ8-2t for linux-arm-kernel@lists.infradead.org; Mon, 30 Oct 2023 12:36:16 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 068BF60959; Mon, 30 Oct 2023 12:36:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2B04C433C7; Mon, 30 Oct 2023 12:36:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698669373; bh=/71kbiytdp/VoW3kuCJ1kqiDxE4ZD+L4PqRlfCczo0I=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=T/eyqJHUuAonHxnuxkt6GEdACiAWxX9iI1QzI/1RvlpMqU+QIDFtVwJ77UdO0XvJC whof5OokBZX4wF9CnXOJilVIpV5+6Vm3nfMb9Dmn0iL21Ijq/oVFyOmGXToQ3b79uo +Px4uqo3N8WAbbkz+P6kLR1kTuGBpuT5r4SsJf1bH41QU76hfWLInxSyk8xNcNm5ss HbgM4IgDJY+Mr5i+VpNTJ3FnH/IQ+f10OD1v+xEPcU64+kY2ArVy3bmKZxvB7i6q4I k47oIEq2mQDF6WbGwAJZpWuAyBW/R5SZ9/+zn6LmcoBGl3Nm5hIALsSVer6hj82+ad 4rxYLEkan3pew== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qxRV5-008vI6-5Z; Mon, 30 Oct 2023 12:36:11 +0000 Date: Mon, 30 Oct 2023 12:36:09 +0000 Message-ID: <86msw01e4m.wl-maz@kernel.org> From: Marc Zyngier To: Jan Henrik Weinstock Cc: oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, Lukas =?UTF-8?B?SsO8bmdlcg==?= Subject: Re: KVM exit to userspace on WFI In-Reply-To: References: <87ttql5aq7.wl-maz@kernel.org> <86cyx250w9.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: jan@mwa.re, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, lukas@mwa.re X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231030_053615_023760_9E98A6AE X-CRM114-Status: GOOD ( 21.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org [please make an effort not to top-post] On Fri, 27 Oct 2023 18:41:44 +0100, Jan Henrik Weinstock wrote: > > Hi Marc, > > the basic idea behind this is to have a (single-threaded) execution loop, > something like this: > > vcpu-thread: vcpu-run | process-io-devices | vcpu-run | process-io... > ^ > WFX or timeout > > We switch to simulating IO devices whenever the vcpu is idle (wfi) or exceeds > a certain budget of instructions (counted via pmu). Our fallback currently is > to kick the vcpu out of its execution using a signal (via a timeout/alarm). But > of course, if the cpu is stuck at a wfi, we are wasting a lot of time. > > I understand that the proposed behavior is not desirable for most use cases, > which is why I suggest locking it behind a flag, e.g. > KVM_ARCH_FLAG_WFX_EXIT_TO_USER. But how do you reconcile the fact that exposing this to userspace breaks fundamental expectations that the guest has, such as getting its timer interrupts and directly injected LPIs? Implementing WFI in userspace breaks it. What about the case where we don't trap WFx and let the *guest* wait for an interrupt? Honestly, what you are describing seems to be a use model that doesn't fit KVM, which is a general purpose hypervisor, but more a simulation environment. Yes, the primitives are the same, but the plumbing is wildly different. *If* that's the stuff you're looking at, then I'm afraid you'll have to do it in different way, because what you are suggesting is fundamentally incompatible with the guarantees that KVM gives to guest and userspace. Because your KVM_ARCH_FLAG_WFX_EXIT_TO_USER is really a lie. It should really be named something more along the lines of KVM_ARCH_FLAG_WFX_EXIT_TO_USER_SOMETIME_AND_I_DONT_EVEN_KNOW_WHEN (probably with additional clauses related to breaking things). Overall, you are still asking for something that is not guaranteed at the architecture level, even less in KVM, and I'm not going to add support for something that can only work "sometime". M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel