From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD85AC433F5 for ; Fri, 7 Jan 2022 16:31:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 0AEF14B2AA; Fri, 7 Jan 2022 11:31:42 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XvVZm5BCrs4F; Fri, 7 Jan 2022 11:31:40 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id D7E004B281; Fri, 7 Jan 2022 11:31:40 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id EBCDF4B281 for ; Fri, 7 Jan 2022 11:31:39 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W0EVkHGVAzTr for ; Fri, 7 Jan 2022 11:31:38 -0500 (EST) Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id CAE364B274 for ; Fri, 7 Jan 2022 11:31:38 -0500 (EST) Received: by mail-pg1-f169.google.com with SMTP id a22so1467150pgd.6 for ; Fri, 07 Jan 2022 08:31:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=8x5aau+9GId4X1fkS/Ix8Wi9zq+PLZoxIQ2oHrW+IM8=; b=L9AJlkClDBCQp2d0qcZIwYZbbM0UJrWK7mAnAs7XoHi5p3eXECQk/DT7ZPdJL6dOLF jIWwg4D0FCWGc+KY752pSLb0iqOUgyV4ueCX1tRs0iyIihUOfu+/nrN1UnQgQtHzU6Q9 E2bh0Vzff3Bi2MgEvD6x1btKH7Hj+ZDktF1TswaiGXH2itZ6sMNFH0eff2bzCSZIUzPx Ljeu9mMpDh9SifesE99f/y4hFJusqSEbqQ+3QYY58RlPh7erzlf6CgVoHjBTmnzEr0gg cXvZjK196zShjNX+a2OopX31KfOYlHQPnp5tpCH8PJpwA4n+VcXTb4X9AGr99mUZ2sKJ dpBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8x5aau+9GId4X1fkS/Ix8Wi9zq+PLZoxIQ2oHrW+IM8=; b=LxqHpU5aEoGNcXNIy8Kx7j9V/HlYFMIKGz0tU41TI8H0IcDpbNhuq39txGlkAgnGuy kbZA+ak7gDw6rgJEkhfWtHvlpZEpA3j2L1VhBbXx3wuhbUeFtVda40y0JiaWrgckx8Ao KKmtHW1BxsyrsX7hpGNNs0FC/CQ3dnndXniXgjlCLdfgK7TQOD0kbsGMIbW+ZkXf8NqN hAeBuSuichOah57gbPFh4bk+avRYVClRKNdayC8oCuTBC5VupW7peWjCNOHSZ5tquT6V kBJIK/5GGUAqoDCYmAzYV98YzaZ/nz0lkI8ynmSEiixQMlM1+U4wXzDXwBih8Clg2FfV Gfyw== X-Gm-Message-State: AOAM5326LJhlVulgYxBPIweLPDju+RMyeez7WqnuU5M7dQCY2S1iL7r7 2xU8Mhj8xrAgO4wtexQHMfaJjQ== X-Google-Smtp-Source: ABdhPJzgG47GKpuKWTf6oJSn9E2fBwBv84lP3oKIaE8NOaZd04A6CCABm5d7mq013onV7fWVKolFbQ== X-Received: by 2002:a63:3f88:: with SMTP id m130mr5233049pga.413.1641573097689; Fri, 07 Jan 2022 08:31:37 -0800 (PST) Received: from google.com (157.214.185.35.bc.googleusercontent.com. [35.185.214.157]) by smtp.gmail.com with ESMTPSA id v128sm4290920pfc.36.2022.01.07.08.31.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Jan 2022 08:31:37 -0800 (PST) Date: Fri, 7 Jan 2022 16:31:33 +0000 From: Sean Christopherson To: David Stevens Subject: Re: [PATCH v5 4/4] KVM: mmu: remove over-aggressive warnings Message-ID: References: <20211129034317.2964790-1-stevensd@google.com> <20211129034317.2964790-5-stevensd@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Cc: Chia-I Wu , Wanpeng Li , kvm@vger.kernel.org, Marc Zyngier , Joerg Roedel , linux-kernel@vger.kernel.org, Paolo Bonzini , Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, Jim Mattson X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Fri, Jan 07, 2022, David Stevens wrote: > > > These are the type of pages which KVM is currently rejecting. Is this > > > something that KVM can support? > > > > I'm not opposed to it. My complaint is that this series is incomplete in that it > > allows mapping the memory into the guest, but doesn't support accessing the memory > > from KVM itself. That means for things to work properly, KVM is relying on the > > guest to use the memory in a limited capacity, e.g. isn't using the memory as > > general purpose RAM. That's not problematic for your use case, because presumably > > the memory is used only by the vGPU, but as is KVM can't enforce that behavior in > > any way. > > > > The really gross part is that failures are not strictly punted to userspace; > > the resulting error varies significantly depending on how the guest "illegally" > > uses the memory. > > > > My first choice would be to get the amdgpu driver "fixed", but that's likely an > > unreasonable request since it sounds like the non-KVM behavior is working as intended. > > > > One thought would be to require userspace to opt-in to mapping this type of memory > > by introducing a new memslot flag that explicitly states that the memslot cannot > > be accessed directly by KVM, i.e. can only be mapped into the guest. That way, > > KVM has an explicit ABI with respect to how it handles this type of memory, even > > though the semantics of exactly what will happen if userspace/guest violates the > > ABI are not well-defined. And internally, KVM would also have a clear touchpoint > > where it deliberately allows mapping such memslots, as opposed to the more implicit > > behavior of bypassing ensure_pfn_ref(). > > Is it well defined when KVM needs to directly access a memslot? Not really, there's certainly no established rule. > At least for x86, it looks like most of the use cases are related to nested > virtualization, except for the call in emulator_cmpxchg_emulated. The emulator_cmpxchg_emulated() will hopefully go away in the nearish future[*]. Paravirt features that communicate between guest and host via memory is the other case that often maps a pfn into KVM. > Without being able to specifically state what should be avoided, a flag like > that would be difficult for userspace to use. Yeah :-( I was thinking KVM could state the flag would be safe to use if and only if userspace could guarantee that the guest would use the memory for some "special" use case, but hadn't actually thought about how to word things. The best thing to do is probably to wait for for kvm_vcpu_map() to be eliminated, as described in the changelogs for commits: 357a18ad230f ("KVM: Kill kvm_map_gfn() / kvm_unmap_gfn() and gfn_to_pfn_cache") 7e2175ebd695 ("KVM: x86: Fix recording of guest steal time / preempted status") Once that is done, everything in KVM will either access guest memory through the userspace hva, or via a mechanism that is tied into the mmu_notifier, at which point accessing non-refcounted struct pages is safe and just needs to worry about not corrupting _refcount. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03482C433F5 for ; Fri, 7 Jan 2022 16:33:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=89lDP9ugr6R7ysWYyzOcSwqAla8D/R9HjX/dTKn1uE4=; b=eYIS00CpfYZ6uY eEob7fozgMbftnnNAW5T/FWxOpFDajUpkrA9v7K/3NhwVoIRH/Kmc8bKKqQRgAX55ckbZRZBkLQP3 5K+JrdUcRoQBHXdDPZi39jUtoU9qEeNtE8xR2BlpwvzviK/53W9m9JwRySUN0Cb1lnIh6wAlVpS3Z +Q3SAygQxZNz1iUkElsXFNUrVJjlppxspa3SYN5LJuUnQbQtX/en31olRn7inxh+tg0IHklgErhy1 Vln+bLEYkefKbZlXu98Zo6p4kj/i7dzXBL+/JMa0p0dcRxC6DcPxa5btzXZkksoZbDQZa0qe176yU s52J6ZtH1LN+v0mD5UOg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1n5s9a-004ePz-69; Fri, 07 Jan 2022 16:31:46 +0000 Received: from mail-pg1-x52d.google.com ([2607:f8b0:4864:20::52d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1n5s9W-004eNy-0q for linux-arm-kernel@lists.infradead.org; Fri, 07 Jan 2022 16:31:43 +0000 Received: by mail-pg1-x52d.google.com with SMTP id 200so5863944pgg.3 for ; Fri, 07 Jan 2022 08:31:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=8x5aau+9GId4X1fkS/Ix8Wi9zq+PLZoxIQ2oHrW+IM8=; b=L9AJlkClDBCQp2d0qcZIwYZbbM0UJrWK7mAnAs7XoHi5p3eXECQk/DT7ZPdJL6dOLF jIWwg4D0FCWGc+KY752pSLb0iqOUgyV4ueCX1tRs0iyIihUOfu+/nrN1UnQgQtHzU6Q9 E2bh0Vzff3Bi2MgEvD6x1btKH7Hj+ZDktF1TswaiGXH2itZ6sMNFH0eff2bzCSZIUzPx Ljeu9mMpDh9SifesE99f/y4hFJusqSEbqQ+3QYY58RlPh7erzlf6CgVoHjBTmnzEr0gg cXvZjK196zShjNX+a2OopX31KfOYlHQPnp5tpCH8PJpwA4n+VcXTb4X9AGr99mUZ2sKJ dpBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8x5aau+9GId4X1fkS/Ix8Wi9zq+PLZoxIQ2oHrW+IM8=; b=CPGHcjaoKxYNsj2Iot8kIfD2fIZF1aDOZS46gEB2SuFYctgYJFCDFE93p5PFnR7+0C oLfoShVucTKsLQuz3o4BgKjXCKsHJ2kkjju7C6X5edGqFNSnGiaJmyMDn2UhlX8G85Sr bq1DNCoQiYzLKPtTJ7W7CO1HLOQv0JxQju6F3xSu0fTJ1AKV2xhVkiPnoa2A0lXPt1By /ufCsS/yprviVipIEM29jezHvRu1YGRq/1WEQCvHSDzIZxWDFWma3fQjzVupoklV5OSq mdJLYmg025OrJQdUW04dgNM/uy0C9gk2oqa7Bmr4xCVPAOiNZLfeUGQI2exFgBc1+0KX 55vQ== X-Gm-Message-State: AOAM530UfX2lAHmrO5aeElAh6adVHEhZl37H+0Z1HVMt/a3CN3idOK9u It79V0rIcPAkM+BtGM1TWhDTyQ== X-Google-Smtp-Source: ABdhPJzgG47GKpuKWTf6oJSn9E2fBwBv84lP3oKIaE8NOaZd04A6CCABm5d7mq013onV7fWVKolFbQ== X-Received: by 2002:a63:3f88:: with SMTP id m130mr5233049pga.413.1641573097689; Fri, 07 Jan 2022 08:31:37 -0800 (PST) Received: from google.com (157.214.185.35.bc.googleusercontent.com. [35.185.214.157]) by smtp.gmail.com with ESMTPSA id v128sm4290920pfc.36.2022.01.07.08.31.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Jan 2022 08:31:37 -0800 (PST) Date: Fri, 7 Jan 2022 16:31:33 +0000 From: Sean Christopherson To: David Stevens Cc: Marc Zyngier , Paolo Bonzini , James Morse , Alexandru Elisei , Suzuki K Poulose , Will Deacon , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Chia-I Wu Subject: Re: [PATCH v5 4/4] KVM: mmu: remove over-aggressive warnings Message-ID: References: <20211129034317.2964790-1-stevensd@google.com> <20211129034317.2964790-5-stevensd@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220107_083142_093584_74B01FCF X-CRM114-Status: GOOD ( 31.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Jan 07, 2022, David Stevens wrote: > > > These are the type of pages which KVM is currently rejecting. Is this > > > something that KVM can support? > > > > I'm not opposed to it. My complaint is that this series is incomplete in that it > > allows mapping the memory into the guest, but doesn't support accessing the memory > > from KVM itself. That means for things to work properly, KVM is relying on the > > guest to use the memory in a limited capacity, e.g. isn't using the memory as > > general purpose RAM. That's not problematic for your use case, because presumably > > the memory is used only by the vGPU, but as is KVM can't enforce that behavior in > > any way. > > > > The really gross part is that failures are not strictly punted to userspace; > > the resulting error varies significantly depending on how the guest "illegally" > > uses the memory. > > > > My first choice would be to get the amdgpu driver "fixed", but that's likely an > > unreasonable request since it sounds like the non-KVM behavior is working as intended. > > > > One thought would be to require userspace to opt-in to mapping this type of memory > > by introducing a new memslot flag that explicitly states that the memslot cannot > > be accessed directly by KVM, i.e. can only be mapped into the guest. That way, > > KVM has an explicit ABI with respect to how it handles this type of memory, even > > though the semantics of exactly what will happen if userspace/guest violates the > > ABI are not well-defined. And internally, KVM would also have a clear touchpoint > > where it deliberately allows mapping such memslots, as opposed to the more implicit > > behavior of bypassing ensure_pfn_ref(). > > Is it well defined when KVM needs to directly access a memslot? Not really, there's certainly no established rule. > At least for x86, it looks like most of the use cases are related to nested > virtualization, except for the call in emulator_cmpxchg_emulated. The emulator_cmpxchg_emulated() will hopefully go away in the nearish future[*]. Paravirt features that communicate between guest and host via memory is the other case that often maps a pfn into KVM. > Without being able to specifically state what should be avoided, a flag like > that would be difficult for userspace to use. Yeah :-( I was thinking KVM could state the flag would be safe to use if and only if userspace could guarantee that the guest would use the memory for some "special" use case, but hadn't actually thought about how to word things. The best thing to do is probably to wait for for kvm_vcpu_map() to be eliminated, as described in the changelogs for commits: 357a18ad230f ("KVM: Kill kvm_map_gfn() / kvm_unmap_gfn() and gfn_to_pfn_cache") 7e2175ebd695 ("KVM: x86: Fix recording of guest steal time / preempted status") Once that is done, everything in KVM will either access guest memory through the userspace hva, or via a mechanism that is tied into the mmu_notifier, at which point accessing non-refcounted struct pages is safe and just needs to worry about not corrupting _refcount. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C42DEC433F5 for ; Fri, 7 Jan 2022 16:31:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240074AbiAGQbj (ORCPT ); Fri, 7 Jan 2022 11:31:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239599AbiAGQbi (ORCPT ); Fri, 7 Jan 2022 11:31:38 -0500 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BDD4C06173E for ; Fri, 7 Jan 2022 08:31:38 -0800 (PST) Received: by mail-pg1-x52f.google.com with SMTP id f8so5844864pgf.8 for ; Fri, 07 Jan 2022 08:31:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=8x5aau+9GId4X1fkS/Ix8Wi9zq+PLZoxIQ2oHrW+IM8=; b=L9AJlkClDBCQp2d0qcZIwYZbbM0UJrWK7mAnAs7XoHi5p3eXECQk/DT7ZPdJL6dOLF jIWwg4D0FCWGc+KY752pSLb0iqOUgyV4ueCX1tRs0iyIihUOfu+/nrN1UnQgQtHzU6Q9 E2bh0Vzff3Bi2MgEvD6x1btKH7Hj+ZDktF1TswaiGXH2itZ6sMNFH0eff2bzCSZIUzPx Ljeu9mMpDh9SifesE99f/y4hFJusqSEbqQ+3QYY58RlPh7erzlf6CgVoHjBTmnzEr0gg cXvZjK196zShjNX+a2OopX31KfOYlHQPnp5tpCH8PJpwA4n+VcXTb4X9AGr99mUZ2sKJ dpBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8x5aau+9GId4X1fkS/Ix8Wi9zq+PLZoxIQ2oHrW+IM8=; b=eyoK8dz5459wOsWvIymyroBLQ+DDBovzRlKiA8GRQdef096ZGLi0Bqkd3JmiBNmSof xegvlTk6QUwf8BrKmfHUVYm74CzhRcyLIObQocYo3ysnYXdIYxeBgpBVWLK0Q0jibhnS LFoEeoDrj9r7A8VfkWFXqTnJ7M/p3nC2hKmX3jHJVFr82M8z6UgqIO1bOelZWHWbNssw ZKiGe8r4kgYJYm3lUjYDBXfHmp1qNLMxjOkM1NsuuKima0K+9Q1+L5vlSzOiNR2UyHDA OM/QUBf0ECD0w7S4Iugl7KjAjZVUxKgfxHPlKIzj5Q3f3JFHSYFBdEjeimRuCpwn8nWS QQUQ== X-Gm-Message-State: AOAM532LaF9IPQ54HASObh7WxiCJOp8XQ/XQqSsrROJUVgxrvZJe/TE+ GJ7ku+GnHXuM2796wngqIVJgFA== X-Google-Smtp-Source: ABdhPJzgG47GKpuKWTf6oJSn9E2fBwBv84lP3oKIaE8NOaZd04A6CCABm5d7mq013onV7fWVKolFbQ== X-Received: by 2002:a63:3f88:: with SMTP id m130mr5233049pga.413.1641573097689; Fri, 07 Jan 2022 08:31:37 -0800 (PST) Received: from google.com (157.214.185.35.bc.googleusercontent.com. [35.185.214.157]) by smtp.gmail.com with ESMTPSA id v128sm4290920pfc.36.2022.01.07.08.31.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Jan 2022 08:31:37 -0800 (PST) Date: Fri, 7 Jan 2022 16:31:33 +0000 From: Sean Christopherson To: David Stevens Cc: Marc Zyngier , Paolo Bonzini , James Morse , Alexandru Elisei , Suzuki K Poulose , Will Deacon , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Chia-I Wu Subject: Re: [PATCH v5 4/4] KVM: mmu: remove over-aggressive warnings Message-ID: References: <20211129034317.2964790-1-stevensd@google.com> <20211129034317.2964790-5-stevensd@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Fri, Jan 07, 2022, David Stevens wrote: > > > These are the type of pages which KVM is currently rejecting. Is this > > > something that KVM can support? > > > > I'm not opposed to it. My complaint is that this series is incomplete in that it > > allows mapping the memory into the guest, but doesn't support accessing the memory > > from KVM itself. That means for things to work properly, KVM is relying on the > > guest to use the memory in a limited capacity, e.g. isn't using the memory as > > general purpose RAM. That's not problematic for your use case, because presumably > > the memory is used only by the vGPU, but as is KVM can't enforce that behavior in > > any way. > > > > The really gross part is that failures are not strictly punted to userspace; > > the resulting error varies significantly depending on how the guest "illegally" > > uses the memory. > > > > My first choice would be to get the amdgpu driver "fixed", but that's likely an > > unreasonable request since it sounds like the non-KVM behavior is working as intended. > > > > One thought would be to require userspace to opt-in to mapping this type of memory > > by introducing a new memslot flag that explicitly states that the memslot cannot > > be accessed directly by KVM, i.e. can only be mapped into the guest. That way, > > KVM has an explicit ABI with respect to how it handles this type of memory, even > > though the semantics of exactly what will happen if userspace/guest violates the > > ABI are not well-defined. And internally, KVM would also have a clear touchpoint > > where it deliberately allows mapping such memslots, as opposed to the more implicit > > behavior of bypassing ensure_pfn_ref(). > > Is it well defined when KVM needs to directly access a memslot? Not really, there's certainly no established rule. > At least for x86, it looks like most of the use cases are related to nested > virtualization, except for the call in emulator_cmpxchg_emulated. The emulator_cmpxchg_emulated() will hopefully go away in the nearish future[*]. Paravirt features that communicate between guest and host via memory is the other case that often maps a pfn into KVM. > Without being able to specifically state what should be avoided, a flag like > that would be difficult for userspace to use. Yeah :-( I was thinking KVM could state the flag would be safe to use if and only if userspace could guarantee that the guest would use the memory for some "special" use case, but hadn't actually thought about how to word things. The best thing to do is probably to wait for for kvm_vcpu_map() to be eliminated, as described in the changelogs for commits: 357a18ad230f ("KVM: Kill kvm_map_gfn() / kvm_unmap_gfn() and gfn_to_pfn_cache") 7e2175ebd695 ("KVM: x86: Fix recording of guest steal time / preempted status") Once that is done, everything in KVM will either access guest memory through the userspace hva, or via a mechanism that is tied into the mmu_notifier, at which point accessing non-refcounted struct pages is safe and just needs to worry about not corrupting _refcount.