From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DBDFC54E71 for ; Fri, 22 Mar 2024 14:35:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=A8bOMZP/r0YVJ/O3jb17ctoAb7QwaZSQiay9PtsxA3M=; b=oXH4yI95+KXbiT OG7aLGk+ZlgTdKafKyycvjnvExQb34SjSJG16xvESlV3lZQhBUsWTLb93USkKAnCC/YDimjlXGIqR gbn45FF+R/mvhpBHO3BnfkiudhnZ046W70F4jKeDAll1nAg3EllEJOrjzARbjTsq5/hlY3QtFdovq PEYDdf4O3BezMFlDkfxlOsc9X4IjOyFST+XGJJUTNCwqADfaz/NSvDMdV0sN9pqCZW0F6RnIt//bn yCnkXEBFf4mA3EREqhQ8Ny2ZuYmJ9YL+PdvbfrBv++q7PhOHunCmFKmp9zjELwHi6VXsH1JuNLIM3 mA5mvmvR4c7ztZiH0EOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnfyq-00000007ZoA-2BQJ; Fri, 22 Mar 2024 14:34:49 +0000 Received: from mail-ej1-x635.google.com ([2a00:1450:4864:20::635]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnfym-00000007Zim-0pV4 for linux-arm-kernel@lists.infradead.org; Fri, 22 Mar 2024 14:34:47 +0000 Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a44665605f3so260545066b.2 for ; Fri, 22 Mar 2024 07:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1711118079; x=1711722879; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=HazYimjF8xR0j5uiuwvoaW7vTpuUwkSCXCP6Apf2dQs=; b=uK3fxqximzOQfq8M0QnsDZ/TzS3TrlS1Ixyk7XcxJnUsSmB04xq0tI3/gwrEJWgNds /SJvNOM51yyitEj30GyQXxYkkkhdrsciBDlVY2tG3APSRZXnqOQgjCDZbAOBXki1m6pb fy0bct3LlEDyw/+QDz1Xm/w7b06xVhQbM0Bv+4pfjVmjUmW/Sva4tONOzNy39gQ2OW2X FmEwQCo0gc6fllcLNewjBMQ1pdm206827HxgMnkHEFM4PUQrAwK1CBJao7T1eAfiJOhK CpltkfHWZjGMRjTB2vu1TRbZ7ZlEA5toAHcFIMVEJMvzjTOXA+5XNP9YLTUCZ0NwZibt pacA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711118079; x=1711722879; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HazYimjF8xR0j5uiuwvoaW7vTpuUwkSCXCP6Apf2dQs=; b=G6VTcQ07eEjmTLaCAziDHcFc0Hvs+mlLkpuWzDPQTXwvuZBH8GWPPoikWJTvd0zV9S YtSO8j1q9+llVcIUKYGxZvrValn+Bh+pP4Py/TM9/AWcRjj25i4lgoHRp11sRGxYW2pp zlbGSdeAQZhMCH3/ts9JgfWJK5og/QMZ97gIuThxU+7o3J9G+rzip1NNIJnVoeFOwITp yGcmgQExiIy0VS8naZbjgvUt1tpxOP938gedo1/HPahjCCD0G67s/lhdkxbtlfGEsLxT gvFXd/vn/jtPRQbkQYXc1ZjYAiLY9ExCPkrIC8RuGcH1X8Zr185hNrM1Eb+Fj8WEIG2O P8yA== X-Forwarded-Encrypted: i=1; AJvYcCXyhPSfy3eNkJmb5miQnV0GHOFg5UhKBNjVCxN/7DZK+CMyAE7OJR2PO/KfTTPqPpk86Ge7QyztXPG3mxsQg4nn0/H8ELRU3wIqlfvnrVWI8BKlJHU= X-Gm-Message-State: AOJu0YwkHjf3ZCrzL2GCjMtqA2sRIAsGpMOONweBY2rhySe8t1Dj4OzP m7sTva6EWUyD5qDR4Jft93KQG+psywUQnXSruO42+7Lt9bd1+e8k8xbt2vnFmA== X-Google-Smtp-Source: AGHT+IGtReWAu4dOxnsh6dGzLvOkCiUJGth6MQj6uX9nKBz1cC9kFCHUfNkyYYI2Y/RW7tLAF1EQiw== X-Received: by 2002:a17:907:8690:b0:a47:365f:647c with SMTP id qa16-20020a170907869000b00a47365f647cmr1103902ejc.0.1711118078581; Fri, 22 Mar 2024 07:34:38 -0700 (PDT) Received: from google.com (64.227.90.34.bc.googleusercontent.com. [34.90.227.64]) by smtp.gmail.com with ESMTPSA id jg4-20020a170907970400b00a4652efd795sm1085122ejc.83.2024.03.22.07.34.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Mar 2024 07:34:37 -0700 (PDT) Date: Fri, 22 Mar 2024 14:34:35 +0000 From: Quentin Perret To: Colton Lewis Cc: kvm@vger.kernel.org, Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Paolo Bonzini , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2] KVM: arm64: Add KVM_CAP to control WFx trapping Message-ID: References: <20240319164341.1674863-1-coltonlewis@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240322_073444_298424_0399C7EC X-CRM114-Status: GOOD ( 30.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Friday 22 Mar 2024 at 14:24:35 (+0000), Quentin Perret wrote: > On Tuesday 19 Mar 2024 at 16:43:41 (+0000), Colton Lewis wrote: > > Add a KVM_CAP to control WFx (WFI or WFE) trapping based on scheduler > > runqueue depth. This is so they can be passed through if the runqueue > > is shallow or the CPU has support for direct interrupt injection. They > > may be always trapped by setting this value to 0. Technically this > > means traps will be cleared when the runqueue depth is 0, but that > > implies nothing is running anyway so there is no reason to care. The > > default value is 1 to preserve previous behavior before adding this > > option. > > I recently discovered that this was enabled by default, but it's not > obvious to me everyone will want this enabled, so I'm in favour of > figuring out a way to turn it off (in fact we might want to make this > feature opt in as the status quo used to be to always trap). > > There are a few potential issues I see with having this enabled: > > - a lone vcpu thread on a CPU will completely screw up the host > scheduler's load tracking metrics if the vCPU actually spends a > significant amount of time in WFI (the PELT signal will no longer > be a good proxy for "how much CPU time does this task need"); > > - the scheduler's decision will impact massively the behaviour of the > vcpu task itself. Co-scheduling a task with a vcpu task (or not) will > impact massively the perceived behaviour of the vcpu task in a way > that is entirely unpredictable to the scheduler; > > - while the above problems might be OK for some users, I don't think > this will always be true, e.g. when running on big.LITTLE systems the > above sounds nightmare-ish; > > - the guest spending long periods of time in WFI prevents the host from > being able to enter deeper idle states, which will impact power very > negatively; > > And probably a whole bunch of other things. > > > Think about his option as a threshold. The instruction will be trapped > > if the runqueue depth is higher than the threshold. > > So talking about the exact interface, I'm not sure exposing this to > userspace is really appropriate. The current rq depth is next to > impossible for userspace to control well. > > My gut feeling tells me we might want to gate all of this on > PREEMPT_FULL instead, since PREEMPT_FULL is pretty much a way to say > "I'm willing to give up scheduler tracking accuracy to gain throughput > when I've got a task running alone on a CPU". Thoughts? And obviously I meant s/PREEMPT_FULL/NOHZ_FULL, but hopefully that was clear :-) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel