From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66955C433FE for ; Tue, 1 Nov 2022 21:07:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C7FAB4BAC9; Tue, 1 Nov 2022 17:07:42 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fU0ZhfnoJiNV; Tue, 1 Nov 2022 17:07:41 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id AD6654BA86; Tue, 1 Nov 2022 17:07:41 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6479C4BA86 for ; Tue, 1 Nov 2022 17:07:40 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zrWob2RYtO1G for ; Tue, 1 Nov 2022 17:07:35 -0400 (EDT) Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 624754BA7C for ; Tue, 1 Nov 2022 17:07:35 -0400 (EDT) Received: by mail-pl1-f182.google.com with SMTP id l2so14697404pld.13 for ; Tue, 01 Nov 2022 14:07:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=oFX3TjIuKKqYykGd6uzPpB/gicyMyYF9f2ih7JzKNRQ=; b=nBxrbawSkasV5OIFis4uRsM+ydVhwdbUYy7S1Z3udwJaID7FVoII4y3e2Y6NcN+RFh lZp3bFFxhaissiuG6SA1KtcxUPcOTyxEnfs3GW5mwTdPgGenTxv71EwIcLPnJ5aI0kQL dTkinmMWMPJlBd1h4MrEs6dFRGDXU5ZWycVzIeEZRmebSIxux39oV0pjFOaqEEMwW3PI mdWnXqnGBZe7Y4JMXHIAhJSV70YC2ZZS7M8HJAu/hH0IOh/jqS9KEtS1DxLpUlDaT2HB iwxcgPifN/nh2CZXcMCvRbDSp/gibhS85slcsL0bNnD/E/hPfgU1EdRvi82v6+YKObPV 7Xzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=oFX3TjIuKKqYykGd6uzPpB/gicyMyYF9f2ih7JzKNRQ=; b=uEW91jnA0TAqTDNTtkBk3VS7Mu7UQVdEL9xiog3beSMyjffp6iD9+BG3Sw3v83A0fr iWJ6vtuySz+dPsYOrcROUBA5fRCTqtDQVPEdIN/hPN4C0ELrT/zYpiOUZ3dbZvjMl6Xk ZkcZi7H0Z08IK9HC/iXRaCr1nXaUOamuSxyAJtJiRSzGuSveq4ylhWvOqjjyrpcutsTO jp92Hl1rx6n424UqhKMkAc0uPFo3D5rs5Fc/03755L6UdLQTmJvEzvvi7cI5haj5vFN/ /mubdbekqSsdJkTN6v7UufMJxGC5qTLpohr0AKeSmUQ5QCMssxdOYcQLCB3zhdjqfnT6 TE0w== X-Gm-Message-State: ACrzQf1wCtgZEiMZTx2EdhdmL8Q0VRwBQPtTJdSH0XcApYIgQjvouvLo NxZkFdy+i5h+WtDEJVizDi3TyQ== X-Google-Smtp-Source: AMsMyM7wcyEL7piFmVnE/Id8mKnk5QFETXWa137oLbYLhxqiM7qCfR6bO73L99JU47mjSsviqexCHw== X-Received: by 2002:a17:90a:9bc7:b0:213:9d21:b0b0 with SMTP id b7-20020a17090a9bc700b002139d21b0b0mr22261185pjw.26.1667336854224; Tue, 01 Nov 2022 14:07:34 -0700 (PDT) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id q3-20020a635c03000000b00460d89df1f1sm6321288pgb.57.2022.11.01.14.07.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Nov 2022 14:07:33 -0700 (PDT) Date: Tue, 1 Nov 2022 21:07:30 +0000 From: Sean Christopherson To: Oliver Upton Subject: Re: [PATCH v3 09/15] KVM: arm64: Free removed stage-2 tables in RCU callback Message-ID: References: <20221027221752.1683510-1-oliver.upton@linux.dev> <20221027221752.1683510-10-oliver.upton@linux.dev> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Cc: kvm@vger.kernel.org, Marc Zyngier , Will Deacon , kvmarm@lists.linux.dev, Ben Gardon , David Matlack , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Tue, Nov 01, 2022, Oliver Upton wrote: > On Tue, Nov 01, 2022 at 08:28:04PM +0000, Sean Christopherson wrote: > > On Thu, Oct 27, 2022, Oliver Upton wrote: > > > There is no real urgency to free a stage-2 subtree that was pruned. > > > Nonetheless, KVM does the tear down in the stage-2 fault path while > > > holding the MMU lock. > > > > > [ copy ] > > > This is _very_ misleading. The above paints RCU as an optimization of sorts to > > avoid doing work while holding mmu_lock. Freeing page tables in an RCU callback > > is _required_ for correctness when allowing parallel page faults to remove page > > tables, as holding mmu_lock for read in that case doesn't ensure no other CPU is > > accessing and/or holds a reference to the to-be-freed page table. > > Agree, but it is still important to reason about what is changing here > too. Moving work out of the vCPU fault path _is_ valuable, though > ancillary to the correctness requirements. Sure, but that's at best a footnote. Similar to protecting freeing, RCU isn't the only option for moving work out of the vCPU fault path. In fact, it's probably one of the worst options because RCU callbacks run with soft IRQs disabled, i.e. doing _too_ much in a RCU callback is a real problem. If RCU weren't being used to protect readers, deferring freeing via a workqueue, kthread, etc... would work just as well, if not better. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA4557B for ; Tue, 1 Nov 2022 21:07:34 +0000 (UTC) Received: by mail-pl1-f179.google.com with SMTP id k7so4997191pll.6 for ; Tue, 01 Nov 2022 14:07:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=oFX3TjIuKKqYykGd6uzPpB/gicyMyYF9f2ih7JzKNRQ=; b=nBxrbawSkasV5OIFis4uRsM+ydVhwdbUYy7S1Z3udwJaID7FVoII4y3e2Y6NcN+RFh lZp3bFFxhaissiuG6SA1KtcxUPcOTyxEnfs3GW5mwTdPgGenTxv71EwIcLPnJ5aI0kQL dTkinmMWMPJlBd1h4MrEs6dFRGDXU5ZWycVzIeEZRmebSIxux39oV0pjFOaqEEMwW3PI mdWnXqnGBZe7Y4JMXHIAhJSV70YC2ZZS7M8HJAu/hH0IOh/jqS9KEtS1DxLpUlDaT2HB iwxcgPifN/nh2CZXcMCvRbDSp/gibhS85slcsL0bNnD/E/hPfgU1EdRvi82v6+YKObPV 7Xzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=oFX3TjIuKKqYykGd6uzPpB/gicyMyYF9f2ih7JzKNRQ=; b=1S61fB1We8RpG+y/iVfE4zzmtwF3gi9WaWxQzyizChPMuZKQ3wWPNnFCDD3zUtdL8f yhHjAsmeOk3Gluxhy0vT5dzkPiCbBo2nXTOEwjwwlqIWaW2m9e5DeADNdMaDXXbCF+PN qUChp3MCSGJkhnGetXsZ8ia8FCv/q4JTx8RY/MjrK4Cu82OqadMtaV/FBDYNO+K3E1xp /JLRN2y/x35Yde/yPPIqb+fQziI0gbSOjczurOjj+ncu5FT9IMGJFF9wlKd1csjWVoVR LcrTLw94T7GeklRVUcVnFusG1JZSMubdpyDzRF/w3h/3X+BL/nxjGq3WkSOhJRNgRdnD 0usw== X-Gm-Message-State: ACrzQf2n0wFTd5CMg0yTe+3BitD3MyjHX11JmIWODTOx63izL9R7fv9O KXGMOs7UcVube5RcC3hOyLktWg== X-Google-Smtp-Source: AMsMyM7wcyEL7piFmVnE/Id8mKnk5QFETXWa137oLbYLhxqiM7qCfR6bO73L99JU47mjSsviqexCHw== X-Received: by 2002:a17:90a:9bc7:b0:213:9d21:b0b0 with SMTP id b7-20020a17090a9bc700b002139d21b0b0mr22261185pjw.26.1667336854224; Tue, 01 Nov 2022 14:07:34 -0700 (PDT) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id q3-20020a635c03000000b00460d89df1f1sm6321288pgb.57.2022.11.01.14.07.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Nov 2022 14:07:33 -0700 (PDT) Date: Tue, 1 Nov 2022 21:07:30 +0000 From: Sean Christopherson To: Oliver Upton Cc: Marc Zyngier , James Morse , Alexandru Elisei , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Reiji Watanabe , Ricardo Koller , David Matlack , Quentin Perret , Ben Gardon , Gavin Shan , Peter Xu , Will Deacon , kvmarm@lists.linux.dev Subject: Re: [PATCH v3 09/15] KVM: arm64: Free removed stage-2 tables in RCU callback Message-ID: References: <20221027221752.1683510-1-oliver.upton@linux.dev> <20221027221752.1683510-10-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Message-ID: <20221101210730.E_ffQe9xEhqEqkM-DXwFttDylBDzB6Lb_679FFfMsGM@z> On Tue, Nov 01, 2022, Oliver Upton wrote: > On Tue, Nov 01, 2022 at 08:28:04PM +0000, Sean Christopherson wrote: > > On Thu, Oct 27, 2022, Oliver Upton wrote: > > > There is no real urgency to free a stage-2 subtree that was pruned. > > > Nonetheless, KVM does the tear down in the stage-2 fault path while > > > holding the MMU lock. > > > > > [ copy ] > > > This is _very_ misleading. The above paints RCU as an optimization of sorts to > > avoid doing work while holding mmu_lock. Freeing page tables in an RCU callback > > is _required_ for correctness when allowing parallel page faults to remove page > > tables, as holding mmu_lock for read in that case doesn't ensure no other CPU is > > accessing and/or holds a reference to the to-be-freed page table. > > Agree, but it is still important to reason about what is changing here > too. Moving work out of the vCPU fault path _is_ valuable, though > ancillary to the correctness requirements. Sure, but that's at best a footnote. Similar to protecting freeing, RCU isn't the only option for moving work out of the vCPU fault path. In fact, it's probably one of the worst options because RCU callbacks run with soft IRQs disabled, i.e. doing _too_ much in a RCU callback is a real problem. If RCU weren't being used to protect readers, deferring freeing via a workqueue, kthread, etc... would work just as well, if not better. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1B5F1C433FE for ; Tue, 1 Nov 2022 21:09:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JBFqbcyciwGO3A2N60gZQe+DBm4fp2+lTC6eP7ceV7I=; b=PUelI9UuOlxpQ1 hi//goadVaD+W/Agul4+IJA3LZMeJpg09PKyCzsEwCSW5P0/eMzZx+iHwOpk5AdQfYW+WVetncA5b 1nEF1qO535GeOa3qIk1srRTn4IUEIiqZlSkzEbhe58hzebCVeWb31TYlHostiptc46pCM8wbo6HY8 CIAguCURPUdOuELDcELxH2se3ClJ9w0LfblfVWQDGuXu55HPPZy1+Wd3O/aZPnfMcWGo9UeveY1ZA lpHbiESkfdQ92CPHWkOD6O7AXXWt7yPUaXb4R1QQIKPxlU/tZcJgsBJvfjv50QnW7re4tD8jQ5TbS CPXXQ/f7EnWn/67a8Low==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1opyTz-0074oa-Vz; Tue, 01 Nov 2022 21:07:40 +0000 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1opyTw-0074n3-VE for linux-arm-kernel@lists.infradead.org; Tue, 01 Nov 2022 21:07:38 +0000 Received: by mail-pl1-x62b.google.com with SMTP id u6so14702837plq.12 for ; Tue, 01 Nov 2022 14:07:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=oFX3TjIuKKqYykGd6uzPpB/gicyMyYF9f2ih7JzKNRQ=; b=nBxrbawSkasV5OIFis4uRsM+ydVhwdbUYy7S1Z3udwJaID7FVoII4y3e2Y6NcN+RFh lZp3bFFxhaissiuG6SA1KtcxUPcOTyxEnfs3GW5mwTdPgGenTxv71EwIcLPnJ5aI0kQL dTkinmMWMPJlBd1h4MrEs6dFRGDXU5ZWycVzIeEZRmebSIxux39oV0pjFOaqEEMwW3PI mdWnXqnGBZe7Y4JMXHIAhJSV70YC2ZZS7M8HJAu/hH0IOh/jqS9KEtS1DxLpUlDaT2HB iwxcgPifN/nh2CZXcMCvRbDSp/gibhS85slcsL0bNnD/E/hPfgU1EdRvi82v6+YKObPV 7Xzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=oFX3TjIuKKqYykGd6uzPpB/gicyMyYF9f2ih7JzKNRQ=; b=1kKQG5V/Igto7tUXcgQ4jdA8CUxUX2Di/zNu/omRt5IdXtNUGFhvdWcto2bVZU14n1 ydT2CzbhzbhBUQzNSuWax2coIWDiQDjYe/2Gb6BRiSYIAHs2YPs6gcrTyh609plVIG96 IlvyNVg7T/+QtiBVFioPcD9Kd6DwJ881XY/zzT2qcr8uuJ2GIzheNZCuEUR0Iapf1Bnc 8/d+7rkZWDdPm6NPWaZb5CX0lGQrg13PGmxrlK+aRMvFEY4hnYHXtsp4wyQ8Grd+pnw2 qtbjyW77BknuRzRAkLmKcNbRc/f1jWdW1jrGuUNh94uPWGiDQuJFQJOazh0GVmAf/xbf eZvQ== X-Gm-Message-State: ACrzQf2gD1a3TkWQtsLCpV8/wlty6IyFyom9BS/bWk8iQKPy/wJkzR7+ Y+uRl3hCMkn9IjvEjc9LzUTy3cX36NbsvA== X-Google-Smtp-Source: AMsMyM7wcyEL7piFmVnE/Id8mKnk5QFETXWa137oLbYLhxqiM7qCfR6bO73L99JU47mjSsviqexCHw== X-Received: by 2002:a17:90a:9bc7:b0:213:9d21:b0b0 with SMTP id b7-20020a17090a9bc700b002139d21b0b0mr22261185pjw.26.1667336854224; Tue, 01 Nov 2022 14:07:34 -0700 (PDT) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id q3-20020a635c03000000b00460d89df1f1sm6321288pgb.57.2022.11.01.14.07.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Nov 2022 14:07:33 -0700 (PDT) Date: Tue, 1 Nov 2022 21:07:30 +0000 From: Sean Christopherson To: Oliver Upton Cc: Marc Zyngier , James Morse , Alexandru Elisei , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Reiji Watanabe , Ricardo Koller , David Matlack , Quentin Perret , Ben Gardon , Gavin Shan , Peter Xu , Will Deacon , kvmarm@lists.linux.dev Subject: Re: [PATCH v3 09/15] KVM: arm64: Free removed stage-2 tables in RCU callback Message-ID: References: <20221027221752.1683510-1-oliver.upton@linux.dev> <20221027221752.1683510-10-oliver.upton@linux.dev> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221101_140737_024233_8A88A6B7 X-CRM114-Status: GOOD ( 19.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Nov 01, 2022, Oliver Upton wrote: > On Tue, Nov 01, 2022 at 08:28:04PM +0000, Sean Christopherson wrote: > > On Thu, Oct 27, 2022, Oliver Upton wrote: > > > There is no real urgency to free a stage-2 subtree that was pruned. > > > Nonetheless, KVM does the tear down in the stage-2 fault path while > > > holding the MMU lock. > > > > > [ copy ] > > > This is _very_ misleading. The above paints RCU as an optimization of sorts to > > avoid doing work while holding mmu_lock. Freeing page tables in an RCU callback > > is _required_ for correctness when allowing parallel page faults to remove page > > tables, as holding mmu_lock for read in that case doesn't ensure no other CPU is > > accessing and/or holds a reference to the to-be-freed page table. > > Agree, but it is still important to reason about what is changing here > too. Moving work out of the vCPU fault path _is_ valuable, though > ancillary to the correctness requirements. Sure, but that's at best a footnote. Similar to protecting freeing, RCU isn't the only option for moving work out of the vCPU fault path. In fact, it's probably one of the worst options because RCU callbacks run with soft IRQs disabled, i.e. doing _too_ much in a RCU callback is a real problem. If RCU weren't being used to protect readers, deferring freeing via a workqueue, kthread, etc... would work just as well, if not better. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel