From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Mon, 18 Sep 2023 08:49:57 -0700 Subject: [PATCH 05/26] vfio: KVM: Pass get/put helpers from KVM to VFIO, don't do circular lookup In-Reply-To: <20230918152110.GI13795@ziepe.ca> References: <20230916003118.2540661-1-seanjc@google.com> <20230916003118.2540661-6-seanjc@google.com> <20230918152110.GI13795@ziepe.ca> Message-ID: List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Mon, Sep 18, 2023, Jason Gunthorpe wrote: > On Fri, Sep 15, 2023 at 05:30:57PM -0700, Sean Christopherson wrote: > > Explicitly pass KVM's get/put helpers to VFIO when attaching a VM to > > VFIO instead of having VFIO do a symbol lookup back into KVM. Having both > > KVM and VFIO do symbol lookups increases the overall complexity and places > > an unnecessary dependency on KVM (from VFIO) without adding any value. > > > > Signed-off-by: Sean Christopherson > > --- > > drivers/vfio/vfio.h | 2 ++ > > drivers/vfio/vfio_main.c | 74 +++++++++++++++++++--------------------- > > include/linux/vfio.h | 4 ++- > > virt/kvm/vfio.c | 9 +++-- > > 4 files changed, 47 insertions(+), 42 deletions(-) > > I don't mind this, but Christoph had disliked my prior attempt to do > this with function pointers.. > > The get can be inlined, IIRC, what about putting a pointer to the put > inside the kvm struct? That wouldn't allow us to achieve our goal, which is to hide the details of "struct kvm" from VFIO (and the rest of the kernel). What's the objection to handing VFIO a function pointer? > The the normal kvm get/put don't have to exported symbols at all? The export of kvm_get_kvm_safe() can go away (I forgot to do that in this series), but kvm_get_kvm() will hang around as it's needed by KVM sub-modules (PPC and x86), KVMGT (x86), and drivers/s390/crypto/vfio_ap_ops.c (no idea what to call that beast). Gah, KVMGT doesn't actually need to call get/put, that can be handled by kvm_page_track_register_notifier(). I am planning on making exports for sub-modules conditional on there actually being submodules, so that's 2 of the 3 gone, but tackling the s390 crypto driver is an entirely different story. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4699F1F5EE for ; Mon, 18 Sep 2023 15:50:00 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-d8153284d6eso5253756276.3 for ; Mon, 18 Sep 2023 08:49:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695052199; x=1695656999; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=rfr6F+5OCUOvPUK/RR7Kq8/OOpnpe9Yys6Zia//mq7UU90tcCZoqvLcAxFPFR9TL5O kKL3Qxt514wNncaCyOQorn9VTy/1JKTLncXEQlCveNPZpMS78iSo2MQ8dtV8b4KzhGiz G30czmQ69ENUT/m+WqEh5zX+EFcWrwlibbMQYE08i+5d5mSponB6pwmkLfrGMocKrJ8d jJao8m6pGLP8ghZ4OoD5FD9LnF/pjagcqi17MEZtCJ3Li5doQQkzkNQ9i0QLRywtyQfV a9eAusclcaQJWlh7X2npi9v00BQm8kQTdackr6/1O6jwsZl3PNxvsRbXwTcpTkMT/2K7 vg/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695052199; x=1695656999; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=wwKRvzutIyQXfK5thVFuGiqzF37lxBswk5ncYU0sKYx0Ng9a17HWFYzrYEmKnLFYFw VoISGir/1xHUzpHMT1xqEykl84yr3YDoXlsE0Fs9H+99jwlcOM8efUkAgqpt43mfx7Wi u/mb/IpzP6OLskQ4UbRhIGkG5hXlsfphlwJZxjlKQu+vs2sBthKTs2wL/H8TKpZ7xrCA F5BuqAr40UM4PDM0C3TAPzSMNJP08fFcelqSkR3GfMEOuC6FPwNLLn4Ms8LhN11ISX4Q aCDL0g+X/DnR5EwMWOESKuMI6eOQSruE4cVAd5cGbkFMBD9N7Me4pjZXJxWxME8NYGcQ VrIg== X-Gm-Message-State: AOJu0YyKbmsYEZDiB4o3CUEy+3mkth1Fo8l5rfsvfgHJJYrV8stO+I7B hdXYOwDKHtNmHzQKG4RNWkoUnYJtrUI= X-Google-Smtp-Source: AGHT+IE1DPoRp0N2nDleDeL+nMknRqhKzdJCPwHMJrwMUBjkYLBNjcfJz91+ASKxJdGzFGGlGgloaP/fXoU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:2411:0:b0:d81:78ec:c403 with SMTP id k17-20020a252411000000b00d8178ecc403mr192905ybk.12.1695052199184; Mon, 18 Sep 2023 08:49:59 -0700 (PDT) Date: Mon, 18 Sep 2023 08:49:57 -0700 In-Reply-To: <20230918152110.GI13795@ziepe.ca> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20230916003118.2540661-1-seanjc@google.com> <20230916003118.2540661-6-seanjc@google.com> <20230918152110.GI13795@ziepe.ca> Message-ID: Subject: Re: [PATCH 05/26] vfio: KVM: Pass get/put helpers from KVM to VFIO, don't do circular lookup From: Sean Christopherson To: Jason Gunthorpe Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Tony Krowiak , Halil Pasic , Jason Herne , Harald Freudenberger , Alex Williamson , Andy Lutomirski , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, kvm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Anish Ghulati , Venkatesh Srinivas , Andrew Thornton Content-Type: text/plain; charset="us-ascii" On Mon, Sep 18, 2023, Jason Gunthorpe wrote: > On Fri, Sep 15, 2023 at 05:30:57PM -0700, Sean Christopherson wrote: > > Explicitly pass KVM's get/put helpers to VFIO when attaching a VM to > > VFIO instead of having VFIO do a symbol lookup back into KVM. Having both > > KVM and VFIO do symbol lookups increases the overall complexity and places > > an unnecessary dependency on KVM (from VFIO) without adding any value. > > > > Signed-off-by: Sean Christopherson > > --- > > drivers/vfio/vfio.h | 2 ++ > > drivers/vfio/vfio_main.c | 74 +++++++++++++++++++--------------------- > > include/linux/vfio.h | 4 ++- > > virt/kvm/vfio.c | 9 +++-- > > 4 files changed, 47 insertions(+), 42 deletions(-) > > I don't mind this, but Christoph had disliked my prior attempt to do > this with function pointers.. > > The get can be inlined, IIRC, what about putting a pointer to the put > inside the kvm struct? That wouldn't allow us to achieve our goal, which is to hide the details of "struct kvm" from VFIO (and the rest of the kernel). What's the objection to handing VFIO a function pointer? > The the normal kvm get/put don't have to exported symbols at all? The export of kvm_get_kvm_safe() can go away (I forgot to do that in this series), but kvm_get_kvm() will hang around as it's needed by KVM sub-modules (PPC and x86), KVMGT (x86), and drivers/s390/crypto/vfio_ap_ops.c (no idea what to call that beast). Gah, KVMGT doesn't actually need to call get/put, that can be handled by kvm_page_track_register_notifier(). I am planning on making exports for sub-modules conditional on there actually being submodules, so that's 2 of the 3 gone, but tackling the s390 crypto driver is an entirely different story. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0E3CDC46CA1 for ; Mon, 18 Sep 2023 15:50:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=mf978lexFv3DV8+2OirC6pAyi+c6cQgvNsG7gw4uJX4=; b=nr4z/C8YalO+GKayn9oJshJAXd 7+bfpXRkc0naYroaqk7C4njKJeSmggWyQUVEACw4e0Wr4LkoN1rX3ZMuDXVafOO/Pl+mfyo/tD2P1 3hVD1MPHMQvhpjMQgNx9kPFPi3msfNPxOSvGWLyQj8ifKNgHNRbbDhVAj/XpEhFccCpK01EqKQPLF GFo0mAr5mtFFsD8vp1cYyndHQiHj7qCvzniUiQgxjpIMr7BXWu+W0FVV8RQivZFZbQPw4Oo4XPSwU vIXkWyx99EI3Wa0UHGpBSx0uWQwpYdPdcL9am+mP+q8clmR4moCeAi/8yzVy4WyAJjQny8cBPN/Tz bW8leCaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qiGVt-00FnAg-0J; Mon, 18 Sep 2023 15:50:17 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qiGVr-00FnAG-1F for linux-riscv@bombadil.infradead.org; Mon, 18 Sep 2023 15:50:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=J5Nd0Ri/XjDZs15mUJZPAJxNR7 W8VqSw2NCwV6BV2assbQ5exg1EeuLBhhvB6StFOaS1mg2BEDP13bHAwQuesZVt1+HdgVKluFn9RFx IZlBup+hmA06P+LwsP1OD9vMKfg3e4GIq4Bp78imofeA3g5zRmf+tWiaAjgfCta+Fm4GQILOd9Q4G 0+XtITwhrPy5HFPk26Rf0jztA5p5yt1ZwEc2cdA0wN+Ppw6qo3fFkiCzro3+t/1RDaCgZx7vmaHWX kVPvg+0y69qjy7bGRwXFIn1Od/G5RN2YkumKS2zor6UOU5/26bXzwFBEcfTuWvYOQTJZGW/D1mpQS WPXh6TCA==; Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by casper.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1qiGVo-00BqOu-8K for linux-riscv@lists.infradead.org; Mon, 18 Sep 2023 15:50:14 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d7e8bb74b59so5256694276.2 for ; Mon, 18 Sep 2023 08:50:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695052199; x=1695656999; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=ec9A+DxA8RLMuBQLX+WfXQI0iplwLbC2nBWVChwpOhGwXW4Z1vRyy3Z6DdRZUpBTWs ytMFdmo9mznH8mawF/hVSmDzhxNA85qiT0CnPcP+1JOWJ0KzhnP9G9iEQMcONPXj9ojD uZJ+3q2y28tJpGbb0OyQLzaYu6uBTg220fmoX4R5B34F+XQAVaaeuXzQBzw8QCZ+4Y3U m+R+plZfesCF8r93bbiKrwqGVFtV6A3JdADqjJ31kX54Ag/8BQF/EYpv3giDTDqFFEzq 5A+VZPV9YSdnHpIccQHReP+dZdyR2r/e/BcGr1X5L+WskLVTiLg8Bqg56CywCg62ee0D cwAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695052199; x=1695656999; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=ovz9EcxI0Rgc37H2LferJM9kCV/iWYmOpf/9KJEaoiIbdgM+VVP8+V6rJ2gTC8jBtm iywHobTkvfXuJx+nDcLc7v9yAExByaEislQUcYugpA+MuqIbPJ5DNVrKe8NBDHp9F3T+ k6WlryqBYACyCz/ZF6H4NH8eEucbEh2oVUaHyaFGyavaA/8YJ6BlSwAAI4FhwMoDsaYR KZqBErw+yTG6jpPwrNfSLxAysGXvCTCx12EqBw07AmDNb5epBO20XcJ2+YaB1TvW1McO g371wG2ROTf6p/HBmmuty+E4U5J5MZUOAtBIugoiWy5cUhlZHs4AQwstodC6J27MyzTK HSWA== X-Gm-Message-State: AOJu0YzWF3Z6RmwJwq0+G3KNyA1snUFy8HnqEaUZGTW/c5uxHPzz8hxK v1m+Wj9PtUapjdIA+PeBTRwLYhVvrhI= X-Google-Smtp-Source: AGHT+IE1DPoRp0N2nDleDeL+nMknRqhKzdJCPwHMJrwMUBjkYLBNjcfJz91+ASKxJdGzFGGlGgloaP/fXoU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:2411:0:b0:d81:78ec:c403 with SMTP id k17-20020a252411000000b00d8178ecc403mr192905ybk.12.1695052199184; Mon, 18 Sep 2023 08:49:59 -0700 (PDT) Date: Mon, 18 Sep 2023 08:49:57 -0700 In-Reply-To: <20230918152110.GI13795@ziepe.ca> Mime-Version: 1.0 References: <20230916003118.2540661-1-seanjc@google.com> <20230916003118.2540661-6-seanjc@google.com> <20230918152110.GI13795@ziepe.ca> Message-ID: Subject: Re: [PATCH 05/26] vfio: KVM: Pass get/put helpers from KVM to VFIO, don't do circular lookup From: Sean Christopherson To: Jason Gunthorpe Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Tony Krowiak , Halil Pasic , Jason Herne , Harald Freudenberger , Alex Williamson , Andy Lutomirski , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, kvm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Anish Ghulati , Venkatesh Srinivas , Andrew Thornton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230918_165012_321402_69F34322 X-CRM114-Status: GOOD ( 15.49 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On Mon, Sep 18, 2023, Jason Gunthorpe wrote: > On Fri, Sep 15, 2023 at 05:30:57PM -0700, Sean Christopherson wrote: > > Explicitly pass KVM's get/put helpers to VFIO when attaching a VM to > > VFIO instead of having VFIO do a symbol lookup back into KVM. Having both > > KVM and VFIO do symbol lookups increases the overall complexity and places > > an unnecessary dependency on KVM (from VFIO) without adding any value. > > > > Signed-off-by: Sean Christopherson > > --- > > drivers/vfio/vfio.h | 2 ++ > > drivers/vfio/vfio_main.c | 74 +++++++++++++++++++--------------------- > > include/linux/vfio.h | 4 ++- > > virt/kvm/vfio.c | 9 +++-- > > 4 files changed, 47 insertions(+), 42 deletions(-) > > I don't mind this, but Christoph had disliked my prior attempt to do > this with function pointers.. > > The get can be inlined, IIRC, what about putting a pointer to the put > inside the kvm struct? That wouldn't allow us to achieve our goal, which is to hide the details of "struct kvm" from VFIO (and the rest of the kernel). What's the objection to handing VFIO a function pointer? > The the normal kvm get/put don't have to exported symbols at all? The export of kvm_get_kvm_safe() can go away (I forgot to do that in this series), but kvm_get_kvm() will hang around as it's needed by KVM sub-modules (PPC and x86), KVMGT (x86), and drivers/s390/crypto/vfio_ap_ops.c (no idea what to call that beast). Gah, KVMGT doesn't actually need to call get/put, that can be handled by kvm_page_track_register_notifier(). I am planning on making exports for sub-modules conditional on there actually being submodules, so that's 2 of the 3 gone, but tackling the s390 crypto driver is an entirely different story. _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E38AAC46CA1 for ; Mon, 18 Sep 2023 15:50:58 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=pzJ/o8mW; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4Rq8Qn43gtz3c8q for ; Tue, 19 Sep 2023 01:50:57 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=pzJ/o8mW; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=flex--seanjc.bounces.google.com (client-ip=2607:f8b0:4864:20::b4a; helo=mail-yb1-xb4a.google.com; envelope-from=3p3eizqykdnooa6jf8ckkcha.8kihejqtll8-9arheopo.kvh67o.knc@flex--seanjc.bounces.google.com; receiver=lists.ozlabs.org) Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4Rq8Pm2skVz3bZ1 for ; Tue, 19 Sep 2023 01:50:02 +1000 (AEST) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d8153284d6eso5253761276.3 for ; Mon, 18 Sep 2023 08:50:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695052199; x=1695656999; darn=lists.ozlabs.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=pzJ/o8mW7IunGPP9qVv0bFIiJZy6tpfBkOlUQlQfFg4mxIWPEPox0hsVHf+oLFsgtJ zPLIZRQCdOksxUtvPg2L4oPBbNTVAdiNDjM6cEnD05r3odUsFp1yzgBCDH0eWHjh1wk2 lO1umolrCukVPgtNfAYQ8LZ+7ce9UanMmJ98jqTjycDmPa/2ciNf3+0zISbEOtjjNA3+ fkj9HEiHsHN0zR54I2ltWugPPsq5WaphE9tFUIGqurt+X2dhYe7w6eXQULGPHlXVBi2e GJsAdAwZBK/VuGcd7xoX8UN8jCz9Honap4zTypuHPYSZM3+sPUGLtDGyBZ9tszpViEQn GTEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695052199; x=1695656999; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=qF3kHv5Wqd/U2qpHa5DNdjn4PiTPueEyuachR0K06FuZEHmjKnJcD0Unu9lKYLt5Ix 6PjBZFepflPJJzgA8hfCXx+LyuW5fq/O6w5maFdRlgSzVfKMbF4gnNo7fWMpjccpxzlw xkX0aC3TYgEhzwKt/xFEf5hZIJrjJx1LH3mRu+wbZAvRgtfqiywve5Zw5jmKBr2ZkjMt jAi5GgG/ncWpDgiuCfN0xpCijBgNSwuW/h6sHZzmWbITnQuUeq7utcDaj120fvjRb0lf RXUzTKtP38WR30jgaZRLAeB9gFpyF6IxvSgCJkQNqcdBzYTiWSk7fgTNhFCT3wz3W3En Sv5w== X-Gm-Message-State: AOJu0YypD+8S/Bx8Lwnp5GjdrkS+BWjoSj1Zrn8lKrPkZE2tLcsnRRsk 73+mv87kK3qLbwha+GXBpWL5yJRoMvU= X-Google-Smtp-Source: AGHT+IE1DPoRp0N2nDleDeL+nMknRqhKzdJCPwHMJrwMUBjkYLBNjcfJz91+ASKxJdGzFGGlGgloaP/fXoU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:2411:0:b0:d81:78ec:c403 with SMTP id k17-20020a252411000000b00d8178ecc403mr192905ybk.12.1695052199184; Mon, 18 Sep 2023 08:49:59 -0700 (PDT) Date: Mon, 18 Sep 2023 08:49:57 -0700 In-Reply-To: <20230918152110.GI13795@ziepe.ca> Mime-Version: 1.0 References: <20230916003118.2540661-1-seanjc@google.com> <20230916003118.2540661-6-seanjc@google.com> <20230918152110.GI13795@ziepe.ca> Message-ID: Subject: Re: [PATCH 05/26] vfio: KVM: Pass get/put helpers from KVM to VFIO, don't do circular lookup From: Sean Christopherson To: Jason Gunthorpe Content-Type: text/plain; charset="us-ascii" X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: x86@kernel.org, kvm@vger.kernel.org, Peter Zijlstra , Catalin Marinas , Dave Hansen , linux-kernel@vger.kernel.org, Alexander Gordeev , Claudio Imbrenda , Will Deacon , linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, Janosch Frank , Harald Freudenberger , Marc Zyngier , Huacai Chen , Halil Pasic , Andrew Thornton , Ingo Molnar , Christian Borntraeger , Jason Herne , Albert Ou , Vasily Gorbik , Venkatesh Srinivas , Heiko Carstens , Arnaldo Carvalho de Melo , Alex Williamson , Borislav Petkov , And y Lutomirski , Paul Walmsley , kvmarm@lists.linux.dev, Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Tony Krowiak , Anish Ghulati , linux-mips@vger.kernel.org, Oliver Upton , linux-perf-users@vger.kernel.org, Palmer Dabbelt , kvm-riscv@lists.infradead.org, Anup Patel , Paolo Bonzini , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, Sep 18, 2023, Jason Gunthorpe wrote: > On Fri, Sep 15, 2023 at 05:30:57PM -0700, Sean Christopherson wrote: > > Explicitly pass KVM's get/put helpers to VFIO when attaching a VM to > > VFIO instead of having VFIO do a symbol lookup back into KVM. Having both > > KVM and VFIO do symbol lookups increases the overall complexity and places > > an unnecessary dependency on KVM (from VFIO) without adding any value. > > > > Signed-off-by: Sean Christopherson > > --- > > drivers/vfio/vfio.h | 2 ++ > > drivers/vfio/vfio_main.c | 74 +++++++++++++++++++--------------------- > > include/linux/vfio.h | 4 ++- > > virt/kvm/vfio.c | 9 +++-- > > 4 files changed, 47 insertions(+), 42 deletions(-) > > I don't mind this, but Christoph had disliked my prior attempt to do > this with function pointers.. > > The get can be inlined, IIRC, what about putting a pointer to the put > inside the kvm struct? That wouldn't allow us to achieve our goal, which is to hide the details of "struct kvm" from VFIO (and the rest of the kernel). What's the objection to handing VFIO a function pointer? > The the normal kvm get/put don't have to exported symbols at all? The export of kvm_get_kvm_safe() can go away (I forgot to do that in this series), but kvm_get_kvm() will hang around as it's needed by KVM sub-modules (PPC and x86), KVMGT (x86), and drivers/s390/crypto/vfio_ap_ops.c (no idea what to call that beast). Gah, KVMGT doesn't actually need to call get/put, that can be handled by kvm_page_track_register_notifier(). I am planning on making exports for sub-modules conditional on there actually being submodules, so that's 2 of the 3 gone, but tackling the s390 crypto driver is an entirely different story. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96973CD37B0 for ; Mon, 18 Sep 2023 15:50:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qh5QcXNTcPH477PxdRZ9lSjmqnMjqSZbBg+9q2nO53A=; b=QU7YS0bTgBzwRew3dmVIXuvVaa dFuYfFgOBCngpLvyvquPpu1Lb3o/8f4fXOFymWN69MMOrTXd6skjtsjejCun3fs553giTkI01VTRE PtDLkgsYVGtFhBwUnofqFa1fHxO5eOKqLWkBTa/gAHbuN3o++U/MXoFrzgAjbMq7tns4uSIw/4ANz /PMdsvJW0osO/A9xVx9yqcYpgAWMj/6IgP90Zh4gF/avML+YyguJMuUjmJIEJLMtWItUBgl1k+sAH Qlis6qjVwMJ3LXb92fL9RUpJGgm/2C3gg+rIhBSmJukQbHeBPFJXtu2Q2E62E03k7dqbt54fHEErq AVEg+kLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qiGVt-00FnAm-26; Mon, 18 Sep 2023 15:50:17 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qiGVr-00FnAJ-2J for linux-arm-kernel@bombadil.infradead.org; Mon, 18 Sep 2023 15:50:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=J5Nd0Ri/XjDZs15mUJZPAJxNR7 W8VqSw2NCwV6BV2assbQ5exg1EeuLBhhvB6StFOaS1mg2BEDP13bHAwQuesZVt1+HdgVKluFn9RFx IZlBup+hmA06P+LwsP1OD9vMKfg3e4GIq4Bp78imofeA3g5zRmf+tWiaAjgfCta+Fm4GQILOd9Q4G 0+XtITwhrPy5HFPk26Rf0jztA5p5yt1ZwEc2cdA0wN+Ppw6qo3fFkiCzro3+t/1RDaCgZx7vmaHWX kVPvg+0y69qjy7bGRwXFIn1Od/G5RN2YkumKS2zor6UOU5/26bXzwFBEcfTuWvYOQTJZGW/D1mpQS WPXh6TCA==; Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by casper.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1qiGVo-00BqOt-Kt for linux-arm-kernel@lists.infradead.org; Mon, 18 Sep 2023 15:50:14 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d81646fcf3eso5292212276.0 for ; Mon, 18 Sep 2023 08:50:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695052199; x=1695656999; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=ec9A+DxA8RLMuBQLX+WfXQI0iplwLbC2nBWVChwpOhGwXW4Z1vRyy3Z6DdRZUpBTWs ytMFdmo9mznH8mawF/hVSmDzhxNA85qiT0CnPcP+1JOWJ0KzhnP9G9iEQMcONPXj9ojD uZJ+3q2y28tJpGbb0OyQLzaYu6uBTg220fmoX4R5B34F+XQAVaaeuXzQBzw8QCZ+4Y3U m+R+plZfesCF8r93bbiKrwqGVFtV6A3JdADqjJ31kX54Ag/8BQF/EYpv3giDTDqFFEzq 5A+VZPV9YSdnHpIccQHReP+dZdyR2r/e/BcGr1X5L+WskLVTiLg8Bqg56CywCg62ee0D cwAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695052199; x=1695656999; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sNXD76U7haNeW91VMZlpdLB8TVqFEiovhx95zes76gg=; b=IYgZmvzBbA6Z78odVcpCZa1xb9wC8f+GjgyrE6h8MDKkhMILmqasWDXsVn3j6nUCjC 3M8N9UhiEmv2rvM7MjFUf8E8R0T8f/HNDf28es0UfOfHCHC8UL/SUUYfukbjgOcJAVaE 2Sy3/2sM+UDAPQb/A59NNcSeBJkY7BhFRRkRn2fmuruE185wYCJ0zE77fqYE4/19px/H bC8tCu93oCYHSYSzywiB6wUkJLc8j3EsDY4yyd74sMxVZy7sLeo1JxpjqTBFYp5An2NK SYAy0WiyrP78ZA0mdF67E0jJYNo3MzjtgZZogmit1m/mNCgUUwfTCmCaeBByYYDWqvWw sDHQ== X-Gm-Message-State: AOJu0Yz1HqlODLuKNapx3ZVEpf8dFW5uP+iXT13o8GaqasXbgYlBERIP FNPIzjS8rzITBvvq3eRCp6ZpiAgDeL8= X-Google-Smtp-Source: AGHT+IE1DPoRp0N2nDleDeL+nMknRqhKzdJCPwHMJrwMUBjkYLBNjcfJz91+ASKxJdGzFGGlGgloaP/fXoU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:2411:0:b0:d81:78ec:c403 with SMTP id k17-20020a252411000000b00d8178ecc403mr192905ybk.12.1695052199184; Mon, 18 Sep 2023 08:49:59 -0700 (PDT) Date: Mon, 18 Sep 2023 08:49:57 -0700 In-Reply-To: <20230918152110.GI13795@ziepe.ca> Mime-Version: 1.0 References: <20230916003118.2540661-1-seanjc@google.com> <20230916003118.2540661-6-seanjc@google.com> <20230918152110.GI13795@ziepe.ca> Message-ID: Subject: Re: [PATCH 05/26] vfio: KVM: Pass get/put helpers from KVM to VFIO, don't do circular lookup From: Sean Christopherson To: Jason Gunthorpe Cc: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Tony Krowiak , Halil Pasic , Jason Herne , Harald Freudenberger , Alex Williamson , Andy Lutomirski , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, kvm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Anish Ghulati , Venkatesh Srinivas , Andrew Thornton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230918_165012_717460_9EB1DEB8 X-CRM114-Status: GOOD ( 16.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Sep 18, 2023, Jason Gunthorpe wrote: > On Fri, Sep 15, 2023 at 05:30:57PM -0700, Sean Christopherson wrote: > > Explicitly pass KVM's get/put helpers to VFIO when attaching a VM to > > VFIO instead of having VFIO do a symbol lookup back into KVM. Having both > > KVM and VFIO do symbol lookups increases the overall complexity and places > > an unnecessary dependency on KVM (from VFIO) without adding any value. > > > > Signed-off-by: Sean Christopherson > > --- > > drivers/vfio/vfio.h | 2 ++ > > drivers/vfio/vfio_main.c | 74 +++++++++++++++++++--------------------- > > include/linux/vfio.h | 4 ++- > > virt/kvm/vfio.c | 9 +++-- > > 4 files changed, 47 insertions(+), 42 deletions(-) > > I don't mind this, but Christoph had disliked my prior attempt to do > this with function pointers.. > > The get can be inlined, IIRC, what about putting a pointer to the put > inside the kvm struct? That wouldn't allow us to achieve our goal, which is to hide the details of "struct kvm" from VFIO (and the rest of the kernel). What's the objection to handing VFIO a function pointer? > The the normal kvm get/put don't have to exported symbols at all? The export of kvm_get_kvm_safe() can go away (I forgot to do that in this series), but kvm_get_kvm() will hang around as it's needed by KVM sub-modules (PPC and x86), KVMGT (x86), and drivers/s390/crypto/vfio_ap_ops.c (no idea what to call that beast). Gah, KVMGT doesn't actually need to call get/put, that can be handled by kvm_page_track_register_notifier(). I am planning on making exports for sub-modules conditional on there actually being submodules, so that's 2 of the 3 gone, but tackling the s390 crypto driver is an entirely different story. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel