From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E458C433DF for ; Mon, 17 Aug 2020 12:25:12 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 29ED920786 for ; Mon, 17 Aug 2020 12:25:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="h9GmAFF/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 29ED920786 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id BD3864C0FE; Mon, 17 Aug 2020 08:25:11 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nFB95dm3W6dR; Mon, 17 Aug 2020 08:25:10 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id AEF914C0EB; Mon, 17 Aug 2020 08:25:10 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id DADF14C0AB for ; Mon, 17 Aug 2020 08:25:09 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DObhcYbMw3qI for ; Mon, 17 Aug 2020 08:25:08 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id B003F4C070 for ; Mon, 17 Aug 2020 08:25:08 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 99FFA20658; Mon, 17 Aug 2020 12:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597667107; bh=RCadk8ibATBlC4OXsKfPUxVYLbnc1pQ2OO78rBZG6yc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=h9GmAFF/1R9Kaz+2nZ1gJUEdSkuolk7DrWiYGIpCxI2BkfTQ3AZlmTOWwUPSjSmQ3 FL5bikAbznQlWFNqieMVDUj5Y54rBaOuCeSvavd2eS5oepKBLM9TD6qmCeReqgmA+d 3H9UVjKoWqs6x1dNCWP7nfXphqkIY/UpohdzoDGY= Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1k7eCI-003YxY-1H; Mon, 17 Aug 2020 13:25:06 +0100 MIME-Version: 1.0 Date: Mon, 17 Aug 2020 13:25:05 +0100 From: Marc Zyngier To: yezengruan Subject: Re: [RFC][PATCH 0/4] arm64:kvm: teach guest sched that VCPUs can be preempted In-Reply-To: References: <20200721041742.197354-1-sergey.senozhatsky@gmail.com> <20200817020310.GA1210848@jagdpanzerIV.localdomain> User-Agent: Roundcube Webmail/1.4.7 Message-ID: X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: yezengruan@huawei.com, sergey.senozhatsky@gmail.com, will@kernel.org, joelaf@google.com, linux-kernel@vger.kernel.org, suleiman@google.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: joelaf@google.com, linux-kernel@vger.kernel.org, Sergey Senozhatsky , suleiman@google.com, will@kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On 2020-08-17 13:03, yezengruan wrote: > On 2020/8/17 10:03, Sergey Senozhatsky wrote: >> On (20/07/21 13:17), Sergey Senozhatsky wrote: >>> Hello, >>> >>> RFC >>> >>> We noticed that in a number of cases when we wake_up_process() >>> on arm64 guest we end up enqueuing that task on a preempted VCPU. The >>> culprit >>> appears to be the fact that arm64 guests are not aware of VCPU >>> preemption >>> as such, so when sched picks up an idle VCPU it always assumes that >>> VCPU >>> is available: >>> >>> wake_up_process() >>> try_to_wake_up() >>> select_task_rq_fair() >>> available_idle_cpu() >>> vcpu_is_preempted() // return false; >>> >>> Which is, obviously, not the case. >>> >>> This RFC patch set adds a simple vcpu_is_preempted() implementation >>> so >>> that scheduler can make better decisions when it search for the idle >>> (v)CPU. >> Hi, >> >> A gentle ping. >> >> -ss >> _______________________________________________ >> kvmarm mailing list >> kvmarm@lists.cs.columbia.edu >> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm >> . > > Hi Sergey, > > I have a set of patches similar to yours. > > https://lore.kernel.org/lkml/20191226135833.1052-1-yezengruan@huawei.com/ It really isn't the same thing at all. You are exposing PV spinlocks, while Sergey exposes preemption to vcpus. The former is a massive, and probably unnecessary superset of the later, which only impacts the scheduler (it doesn't change the way locks are implemented). You really shouldn't conflate the two (which you have done in your series). M. -- Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DFABC433E1 for ; Mon, 17 Aug 2020 12:26:29 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2BFD620658 for ; Mon, 17 Aug 2020 12:26:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="mJqOi5GT"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="h9GmAFF/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2BFD620658 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:References:In-Reply-To:Subject:To:From: Date:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eoHy8axUSzb8neEG0ISS8Cs7tky6VdC76PRnM6ZYpJk=; b=mJqOi5GTVnbUPQC3bhMtYH3Kl 0kbDWs7xgPSDFMNQx6lCsUEj8zA9R8z8HlqB8btbayqjps0INVTWbQOOUO7MaQPt1h3Y4k/eamUp9 k+fh7xBmUnzryyFv5V5Ct7KqgD3x7OADzzmaYGgHILJCYdPDFL9qhuGM5F5csFi7qyFDmQAMH6Kzq 3MKuqZLLZ+4Lg0X9FlQxYZJVqFcFfgVJJ/Ddsx3RF2E/Pi1cHUY7FnVXpvwkbEDMDhLvy3IvXYYAe ZICslkb20WORMKWRnff8maNJysfEaQtjdXgSTaO+FqIoZlqAqnDhnci/SvL6NxXS6jmTd+af9rUpv micGixJTA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k7eCO-00033d-4a; Mon, 17 Aug 2020 12:25:12 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k7eCK-00032P-Di for linux-arm-kernel@lists.infradead.org; Mon, 17 Aug 2020 12:25:09 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 99FFA20658; Mon, 17 Aug 2020 12:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597667107; bh=RCadk8ibATBlC4OXsKfPUxVYLbnc1pQ2OO78rBZG6yc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=h9GmAFF/1R9Kaz+2nZ1gJUEdSkuolk7DrWiYGIpCxI2BkfTQ3AZlmTOWwUPSjSmQ3 FL5bikAbznQlWFNqieMVDUj5Y54rBaOuCeSvavd2eS5oepKBLM9TD6qmCeReqgmA+d 3H9UVjKoWqs6x1dNCWP7nfXphqkIY/UpohdzoDGY= Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1k7eCI-003YxY-1H; Mon, 17 Aug 2020 13:25:06 +0100 MIME-Version: 1.0 Date: Mon, 17 Aug 2020 13:25:05 +0100 From: Marc Zyngier To: yezengruan Subject: Re: [RFC][PATCH 0/4] arm64:kvm: teach guest sched that VCPUs can be preempted In-Reply-To: References: <20200721041742.197354-1-sergey.senozhatsky@gmail.com> <20200817020310.GA1210848@jagdpanzerIV.localdomain> User-Agent: Roundcube Webmail/1.4.7 Message-ID: X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: yezengruan@huawei.com, sergey.senozhatsky@gmail.com, will@kernel.org, joelaf@google.com, linux-kernel@vger.kernel.org, suleiman@google.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200817_082508_559702_AA3D0F1D X-CRM114-Status: GOOD ( 15.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: joelaf@google.com, linux-kernel@vger.kernel.org, Sergey Senozhatsky , "Wanghaibin \(D\)" , suleiman@google.com, will@kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2020-08-17 13:03, yezengruan wrote: > On 2020/8/17 10:03, Sergey Senozhatsky wrote: >> On (20/07/21 13:17), Sergey Senozhatsky wrote: >>> Hello, >>> >>> RFC >>> >>> We noticed that in a number of cases when we wake_up_process() >>> on arm64 guest we end up enqueuing that task on a preempted VCPU. The >>> culprit >>> appears to be the fact that arm64 guests are not aware of VCPU >>> preemption >>> as such, so when sched picks up an idle VCPU it always assumes that >>> VCPU >>> is available: >>> >>> wake_up_process() >>> try_to_wake_up() >>> select_task_rq_fair() >>> available_idle_cpu() >>> vcpu_is_preempted() // return false; >>> >>> Which is, obviously, not the case. >>> >>> This RFC patch set adds a simple vcpu_is_preempted() implementation >>> so >>> that scheduler can make better decisions when it search for the idle >>> (v)CPU. >> Hi, >> >> A gentle ping. >> >> -ss >> _______________________________________________ >> kvmarm mailing list >> kvmarm@lists.cs.columbia.edu >> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm >> . > > Hi Sergey, > > I have a set of patches similar to yours. > > https://lore.kernel.org/lkml/20191226135833.1052-1-yezengruan@huawei.com/ It really isn't the same thing at all. You are exposing PV spinlocks, while Sergey exposes preemption to vcpus. The former is a massive, and probably unnecessary superset of the later, which only impacts the scheduler (it doesn't change the way locks are implemented). You really shouldn't conflate the two (which you have done in your series). M. -- Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44C2AC433DF for ; Mon, 17 Aug 2020 12:25:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F2BD20658 for ; Mon, 17 Aug 2020 12:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597667122; bh=RCadk8ibATBlC4OXsKfPUxVYLbnc1pQ2OO78rBZG6yc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-ID:From; b=EmCun2gHcr8fLOKeEujQn7HYsc6PdPOBsZLkaGUq3oNqPPui5skQmKkPDcNLebW14 wIif2sOd8LZ/V4UO1o1w/d+HagwN4v/tONIlHxT3wBpWKJpJ51h3fpOePey1LA+2o6 fpJ5lRNyhrO2cy7G+pKqCE/tfPW68fGk8BpSjdus= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726871AbgHQMZU (ORCPT ); Mon, 17 Aug 2020 08:25:20 -0400 Received: from mail.kernel.org ([198.145.29.99]:44010 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728317AbgHQMZI (ORCPT ); Mon, 17 Aug 2020 08:25:08 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 99FFA20658; Mon, 17 Aug 2020 12:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597667107; bh=RCadk8ibATBlC4OXsKfPUxVYLbnc1pQ2OO78rBZG6yc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=h9GmAFF/1R9Kaz+2nZ1gJUEdSkuolk7DrWiYGIpCxI2BkfTQ3AZlmTOWwUPSjSmQ3 FL5bikAbznQlWFNqieMVDUj5Y54rBaOuCeSvavd2eS5oepKBLM9TD6qmCeReqgmA+d 3H9UVjKoWqs6x1dNCWP7nfXphqkIY/UpohdzoDGY= Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1k7eCI-003YxY-1H; Mon, 17 Aug 2020 13:25:06 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 17 Aug 2020 13:25:05 +0100 From: Marc Zyngier To: yezengruan Cc: Sergey Senozhatsky , will@kernel.org, joelaf@google.com, linux-kernel@vger.kernel.org, suleiman@google.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, "Wanghaibin (D)" Subject: Re: [RFC][PATCH 0/4] arm64:kvm: teach guest sched that VCPUs can be preempted In-Reply-To: References: <20200721041742.197354-1-sergey.senozhatsky@gmail.com> <20200817020310.GA1210848@jagdpanzerIV.localdomain> User-Agent: Roundcube Webmail/1.4.7 Message-ID: X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: yezengruan@huawei.com, sergey.senozhatsky@gmail.com, will@kernel.org, joelaf@google.com, linux-kernel@vger.kernel.org, suleiman@google.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020-08-17 13:03, yezengruan wrote: > On 2020/8/17 10:03, Sergey Senozhatsky wrote: >> On (20/07/21 13:17), Sergey Senozhatsky wrote: >>> Hello, >>> >>> RFC >>> >>> We noticed that in a number of cases when we wake_up_process() >>> on arm64 guest we end up enqueuing that task on a preempted VCPU. The >>> culprit >>> appears to be the fact that arm64 guests are not aware of VCPU >>> preemption >>> as such, so when sched picks up an idle VCPU it always assumes that >>> VCPU >>> is available: >>> >>> wake_up_process() >>> try_to_wake_up() >>> select_task_rq_fair() >>> available_idle_cpu() >>> vcpu_is_preempted() // return false; >>> >>> Which is, obviously, not the case. >>> >>> This RFC patch set adds a simple vcpu_is_preempted() implementation >>> so >>> that scheduler can make better decisions when it search for the idle >>> (v)CPU. >> Hi, >> >> A gentle ping. >> >> -ss >> _______________________________________________ >> kvmarm mailing list >> kvmarm@lists.cs.columbia.edu >> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm >> . > > Hi Sergey, > > I have a set of patches similar to yours. > > https://lore.kernel.org/lkml/20191226135833.1052-1-yezengruan@huawei.com/ It really isn't the same thing at all. You are exposing PV spinlocks, while Sergey exposes preemption to vcpus. The former is a massive, and probably unnecessary superset of the later, which only impacts the scheduler (it doesn't change the way locks are implemented). You really shouldn't conflate the two (which you have done in your series). M. -- Jazz is not dead. It just smells funny...