From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60E84EEAA40 for ; Thu, 14 Sep 2023 13:38:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:References:In-Reply-To:Subject:Cc:To:From :Date:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hXHUg0IfchcQBiSEep20+pyr7PtV2qGiPVgTUF9SbPk=; b=IIGjqaysQVR0ht1qfSUxAHHeRL m/z7gqgkeQXk7ek0vOm126fBzFoNugeFKgxz9oQxv4okJdo446t6yq/JJlodn44+2oFTrLM6kT8ia 4zr9Hm0Nn7n5Te7I+0RCoKHI6RHejdQq5ydX0x+06KHDtZY+9IFmkg0YUDaZKzqLsmPW6e3NrXfku 1DW8CPI+gAmgSYsKqM8DgXIDFPBrU2K8HRGqkA4FSTdg0EM6kucBHEU7sJaJLwzO5scCsxFgzIQPK ywlZ4KqbCCQaaDWdG87fzNZYW3QFuQjo9XI5stfRLP3H0nbdZTiFd1MbFXp0TW8DeC5L6fh+Jx3Ss DYDuI3YQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qgmXd-008WkW-27; Thu, 14 Sep 2023 13:37:57 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qgmWy-008VuO-0R for linux-arm-kernel@lists.infradead.org; Thu, 14 Sep 2023 13:37:17 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 086CFB82762; Thu, 14 Sep 2023 13:37:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B97FBC433C8; Thu, 14 Sep 2023 13:37:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694698632; bh=Fl6RNpJimEuHpL5sPJLw45C8oocbmvKU729su9Dk9wo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=pt+eN9LPhV/y8mBV/XzMPHIkcxgQ/uO4qEu4xUKH+bsnFneOCl1sOpF8jmddmRMvy PdiplUPKQdJaFy9ecE1NRUTl8zODWSLmxzNiXCPVkKZcn9CMvJTW5wQFpSe9lZozJW 6lJh6edujGtDNPh4VoGYSJA1TXVYvK7Q0aBRKQwlgog1vwyeKxogjWmjKhmk1yxlsY PgHh3sHt6BpTSYNx72z6L/z9/MmjAUVOko7lti/QABrHl1A6I+ir9noq8hDxzoXkvZ JAzOTqfG6waKPiMv2TPcugjQc7H3YBo89GikUnf4FWYzrOA/Nu7QaT9F5DAs45SPvY LacycEtpqU7fw== Received: from disco-boy.misterjones.org ([217.182.43.188] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qgmWr-00CvtI-LK; Thu, 14 Sep 2023 14:37:09 +0100 MIME-Version: 1.0 Date: Thu, 14 Sep 2023 14:37:09 +0100 From: Marc Zyngier To: Ganapatrao Kulkarni Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Christoffer Dall , Alexandru Elisei , Andre Przywara , Chase Conklin , Darren Hart , Jintack Lim , Russell King , Miguel Luis , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , D Scott Phillips Subject: Re: [PATCH v10 29/59] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables In-Reply-To: <8d0f77a8-00db-93f7-aeae-bf96190b6f5b@os.amperecomputing.com> References: <20230515173103.1017669-1-maz@kernel.org> <20230515173103.1017669-30-maz@kernel.org> <8d0f77a8-00db-93f7-aeae-bf96190b6f5b@os.amperecomputing.com> User-Agent: Roundcube Webmail/1.4.13 Message-ID: <3454100b4bb927a24b380f442f02d178@kernel.org> X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 217.182.43.188 X-SA-Exim-Rcpt-To: gankulkarni@os.amperecomputing.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, christoffer.dall@arm.com, alexandru.elisei@arm.com, andre.przywara@arm.com, chase.conklin@arm.com, darren@os.amperecomputing.com, jintack@cs.columbia.edu, rmk+kernel@armlinux.org.uk, miguel.luis@oracle.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, scott@os.amperecomputing.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230914_063716_476378_FF9BEDD6 X-CRM114-Status: GOOD ( 21.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Ganapatrao, On 2023-09-14 14:10, Ganapatrao Kulkarni wrote: > Hi Marc, > > On 15-05-2023 11:00 pm, Marc Zyngier wrote: >> From: Christoffer Dall >> >> Unmap/flush shadow stage 2 page tables for the nested VMs as well as >> the >> stage 2 page table for the guest hypervisor. >> >> Note: A bunch of the code in mmu.c relating to MMU notifiers is >> currently dealt with in an extremely abrupt way, for example by >> clearing >> out an entire shadow stage-2 table. This will be handled in a more >> efficient way using the reverse mapping feature in a later version of >> the patch series. > > We are seeing spin-lock contention due to this patch when the > Guest-Hypervisor(L1) is booted with higher number of cores and > auto-numa is enabled on L0. > kvm_nested_s2_unmap is called as part of notifier call-back when numa > page migration is happening and this function which holds lock becomes > source of contention when there are more vCPUs are processing the > auto-numa page fault/migration. This is fully expected. Honestly, expecting any sort of performance at this stage is extremely premature (and I have zero sympathy for hacks like auto-numa...). [...] >> + kvm_nested_s2_unmap(kvm); > > This kvm_nested_s2_unmap/kvm_unmap_stage2_range is called for every > active L2 and page table walk-through iterates for long iterations > since kvm_phys_size(mmu) is pretty big size(atleast 48bits). > What would be the best fix if we want to avoid this unnessary long > iteration of page table lookup? The fix would be to have a reverse mapping for any canonical IPA to any shadow IPA, which would allow us to not fully tear down the shadow PTs but only the bit we need to remove. This is clearly stated in the commit message you quoted. However, this plan is pretty far down the list, and I have no plan to work on performance optimisations until we have basic support merged upstream. Thanks, M. -- Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel