From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D427C4332F for ; Fri, 4 Nov 2022 16:31:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 7B418410AD; Fri, 4 Nov 2022 12:31:36 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kBW6rHtCLbLS; Fri, 4 Nov 2022 12:31:34 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 1FDA74107F; Fri, 4 Nov 2022 12:31:34 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 5C71D40BD9 for ; Fri, 4 Nov 2022 12:31:32 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5+EDdshPubKq for ; Fri, 4 Nov 2022 12:31:31 -0400 (EDT) Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 245AE40B6C for ; Fri, 4 Nov 2022 12:31:31 -0400 (EDT) Received: by mail-pg1-f170.google.com with SMTP id b62so4842726pgc.0 for ; Fri, 04 Nov 2022 09:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=tykh0C7NQoqMsiv0FEJ8nDBTc1ar6j/4SljcpXaw+gg=; b=ZvfqWRbmiygXbQlvUXzlFPIy6pWgi3NnbKZ9axUfvTwSG1jmeczFdzfmEBk6JPul9u KtfoNDPD7+7L7ckLKxE85H3i30VZE1cLcwZqr+9Pmyt/Uvr0gsenysKMrttjRd1DNmYa taHLNc4BlGL4LuS+a5zOzje7JmI5lr7pLr83tPoJKPDxZ+8mRqo1TUsZIcKB3+pLirGc 61VotfoTC994tXlc+pHq0IWsR3SGM7VfiO/hK89Ek0CoqTLSvnWOMLSW6BMMx8STYHyn tAWojTUjFC5IOfCMzJ25ZkS53CsxfaYW+If16SDvap+gn/hFvgnPNsAIXJ/ZXr/CVgup nXnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tykh0C7NQoqMsiv0FEJ8nDBTc1ar6j/4SljcpXaw+gg=; b=sPu9PQOdaF8nfz95fOhVPBPQ0Ar65++j9++Oq3pa2gkwdmf2U1DAR+yvac6ZHunlx4 wEzwZ4S2UfgE9cGB+TJYRmcxnpYzYXOZ/ga8V4tBUbjszmTnBcGfDhqnydhmjvry7KNP Ih04ZlEIFpmjcqy21r+i8HSzLQT4xsrPivmwnyYmIsDYwYmHBGlw8IUCrHmO+KmgyvTV 0a87RB3JXzpJXJArs1UlWbvCv6jDaVdWuBtHaAMnnYZpB6pElvrpgOYf9Qu2fZuOvcRt hoyKHm/EycyC/7jAE/zYx+YtMBaIqpXFWgeCHXNDh3SF2zNpkex5yICVUJdZQRBbVidA SBEw== X-Gm-Message-State: ACrzQf1x858KAmidN1GxPfIO7SZuMSRlEacdJNm7VsCzG05jHrctqBeh TUwDCrNSEZmF4Lgxp3IYoxE6UQ== X-Google-Smtp-Source: AMsMyM4q4CNgN2lQ4PyaYyADoR+QimlVBokrxw9X6LS0+cBweFifUR+XVuuTHXr8UlnLM2zvHk12kA== X-Received: by 2002:a05:6a00:248e:b0:56e:ad31:b976 with SMTP id c14-20020a056a00248e00b0056ead31b976mr1059125pfv.51.1667579490004; Fri, 04 Nov 2022 09:31:30 -0700 (PDT) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id y133-20020a62ce8b000000b00565cbad9616sm2954667pfg.6.2022.11.04.09.31.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 09:31:29 -0700 (PDT) Date: Fri, 4 Nov 2022 16:31:25 +0000 From: Sean Christopherson To: Yuan Yao Subject: Re: [PATCH 08/44] KVM: x86: Move hardware setup/unsetup to init/exit Message-ID: References: <20221102231911.3107438-1-seanjc@google.com> <20221102231911.3107438-9-seanjc@google.com> <20221104062223.7kcrbt66mlmqxk7f@yy-desk-7060> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20221104062223.7kcrbt66mlmqxk7f@yy-desk-7060> Cc: Matthew Rosato , David Hildenbrand , Yuan Yao , Paul Walmsley , linux-kernel@vger.kernel.org, Michael Ellerman , linux-riscv@lists.infradead.org, Claudio Imbrenda , kvmarm@lists.cs.columbia.edu, linux-s390@vger.kernel.org, Janosch Frank , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Christian Borntraeger , Chao Gao , Eric Farman , Albert Ou , kvm@vger.kernel.org, Atish Patra , kvmarm@lists.linux.dev, Thomas Gleixner , linux-arm-kernel@lists.infradead.org, Isaku Yamahata , Fabiano Rosas , linux-mips@vger.kernel.org, Palmer Dabbelt , kvm-riscv@lists.infradead.org, Paolo Bonzini , Vitaly Kuznetsov , linuxppc-dev@lists.ozlabs.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Fri, Nov 04, 2022, Yuan Yao wrote: > On Wed, Nov 02, 2022 at 11:18:35PM +0000, Sean Christopherson wrote: > > To avoid having to unwind various setup, e.g registration of several > > notifiers, slot in the vendor hardware setup before the registration of > > said notifiers and callbacks. Introducing a functional change while > > moving code is less than ideal, but the alternative is adding a pile of > > unwinding code, which is much more error prone, e.g. several attempts to > > move the setup code verbatim all introduced bugs. ... > > @@ -9325,6 +9343,24 @@ int kvm_arch_init(void *opaque) > > kvm_caps.supported_xcr0 = host_xcr0 & KVM_SUPPORTED_XCR0; > > } > > > > + rdmsrl_safe(MSR_EFER, &host_efer); > > + > > + if (boot_cpu_has(X86_FEATURE_XSAVES)) > > + rdmsrl(MSR_IA32_XSS, host_xss); > > + > > + kvm_init_pmu_capability(); > > + > > + r = ops->hardware_setup(); > > + if (r != 0) > > + goto out_mmu_exit; > > The failure case of ops->hardware_setup() is unwound > by kvm_arch_exit() before this patch, do we need to > keep that old behavior ? As called out in the changelog, the call to ops->hardware_setup() was deliberately slotted in before the call to kvm_timer_init() so that kvm_arch_init() wouldn't need to unwind more stuff if harware_setup() fails. > > + /* > > + * Point of no return! DO NOT add error paths below this point unless > > + * absolutely necessary, as most operations from this point forward > > + * require unwinding. > > + */ > > + kvm_ops_update(ops); > > + > > kvm_timer_init(); > > > > if (pi_inject_timer == -1) > > @@ -9336,8 +9372,32 @@ int kvm_arch_init(void *opaque) > > set_hv_tscchange_cb(kvm_hyperv_tsc_notifier); > > #endif > > > > + kvm_register_perf_callbacks(ops->handle_intel_pt_intr); > > + > > + if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) > > + kvm_caps.supported_xss = 0; > > + > > +#define __kvm_cpu_cap_has(UNUSED_, f) kvm_cpu_cap_has(f) > > + cr4_reserved_bits = __cr4_reserved_bits(__kvm_cpu_cap_has, UNUSED_); > > +#undef __kvm_cpu_cap_has > > + > > + if (kvm_caps.has_tsc_control) { > > + /* > > + * Make sure the user can only configure tsc_khz values that > > + * fit into a signed integer. > > + * A min value is not calculated because it will always > > + * be 1 on all machines. > > + */ > > + u64 max = min(0x7fffffffULL, > > + __scale_tsc(kvm_caps.max_tsc_scaling_ratio, tsc_khz)); > > + kvm_caps.max_guest_tsc_khz = max; > > + } > > + kvm_caps.default_tsc_scaling_ratio = 1ULL << kvm_caps.tsc_scaling_ratio_frac_bits; > > + kvm_init_msr_list(); > > return 0; > > > > +out_mmu_exit: > > + kvm_mmu_vendor_module_exit(); > > out_free_percpu: > > free_percpu(user_return_msrs); > > out_free_x86_emulator_cache: _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B416379F1 for ; Fri, 4 Nov 2022 16:31:30 +0000 (UTC) Received: by mail-pf1-f172.google.com with SMTP id 130so4955773pfu.8 for ; Fri, 04 Nov 2022 09:31:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=tykh0C7NQoqMsiv0FEJ8nDBTc1ar6j/4SljcpXaw+gg=; b=ZvfqWRbmiygXbQlvUXzlFPIy6pWgi3NnbKZ9axUfvTwSG1jmeczFdzfmEBk6JPul9u KtfoNDPD7+7L7ckLKxE85H3i30VZE1cLcwZqr+9Pmyt/Uvr0gsenysKMrttjRd1DNmYa taHLNc4BlGL4LuS+a5zOzje7JmI5lr7pLr83tPoJKPDxZ+8mRqo1TUsZIcKB3+pLirGc 61VotfoTC994tXlc+pHq0IWsR3SGM7VfiO/hK89Ek0CoqTLSvnWOMLSW6BMMx8STYHyn tAWojTUjFC5IOfCMzJ25ZkS53CsxfaYW+If16SDvap+gn/hFvgnPNsAIXJ/ZXr/CVgup nXnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tykh0C7NQoqMsiv0FEJ8nDBTc1ar6j/4SljcpXaw+gg=; b=dm0/3luZgr2nb7aCUgMXhEb387mnUYDk5vUgnVcOeu4gLCLmUU/ydgxwOO+rmTOK5C 8Zd/L6o+Tx7ggyhT2pusqN9BTaEc8t/ZmFPx2PwNN02u2BrxX+itRcrnqeCCPY/oUlz5 2fQWFGVlAM5+afTPmmT3mbD1bb7RbdoksQe/xTinxp61kUc1xFW1hRwqxmlv3Dodz9Fq efTPvc2MydKp7DdqYeVCcw7o29fzuDrptRwfh2k/IGbi19gm6akEWUX98qE1gqsSnSy9 tAGLFK0MKHW05DWTtfuult9AJ+hy1d+JvwMgFIiiIkfyd8NzZswrWLC7RPHR4d0oy64m o5vQ== X-Gm-Message-State: ACrzQf38eUQ2IfNAm7uw7d/ttQ8YohO7Rh/F/1bpo3Vt9a7qT1uxTAtm kNSzwDbfI8VjlcYmbt+N5OSmaQ== X-Google-Smtp-Source: AMsMyM4q4CNgN2lQ4PyaYyADoR+QimlVBokrxw9X6LS0+cBweFifUR+XVuuTHXr8UlnLM2zvHk12kA== X-Received: by 2002:a05:6a00:248e:b0:56e:ad31:b976 with SMTP id c14-20020a056a00248e00b0056ead31b976mr1059125pfv.51.1667579490004; Fri, 04 Nov 2022 09:31:30 -0700 (PDT) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id y133-20020a62ce8b000000b00565cbad9616sm2954667pfg.6.2022.11.04.09.31.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 09:31:29 -0700 (PDT) Date: Fri, 4 Nov 2022 16:31:25 +0000 From: Sean Christopherson To: Yuan Yao Cc: Paolo Bonzini , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Matthew Rosato , Eric Farman , Vitaly Kuznetsov , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Atish Patra , David Hildenbrand , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, Isaku Yamahata , Fabiano Rosas , Michael Ellerman , Chao Gao , Thomas Gleixner , Yuan Yao Subject: Re: [PATCH 08/44] KVM: x86: Move hardware setup/unsetup to init/exit Message-ID: References: <20221102231911.3107438-1-seanjc@google.com> <20221102231911.3107438-9-seanjc@google.com> <20221104062223.7kcrbt66mlmqxk7f@yy-desk-7060> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221104062223.7kcrbt66mlmqxk7f@yy-desk-7060> Message-ID: <20221104163125.B49kywaLK73rbuEcCQQviYf_8QAQyFy-B3FFCq6uGtI@z> On Fri, Nov 04, 2022, Yuan Yao wrote: > On Wed, Nov 02, 2022 at 11:18:35PM +0000, Sean Christopherson wrote: > > To avoid having to unwind various setup, e.g registration of several > > notifiers, slot in the vendor hardware setup before the registration of > > said notifiers and callbacks. Introducing a functional change while > > moving code is less than ideal, but the alternative is adding a pile of > > unwinding code, which is much more error prone, e.g. several attempts to > > move the setup code verbatim all introduced bugs. ... > > @@ -9325,6 +9343,24 @@ int kvm_arch_init(void *opaque) > > kvm_caps.supported_xcr0 = host_xcr0 & KVM_SUPPORTED_XCR0; > > } > > > > + rdmsrl_safe(MSR_EFER, &host_efer); > > + > > + if (boot_cpu_has(X86_FEATURE_XSAVES)) > > + rdmsrl(MSR_IA32_XSS, host_xss); > > + > > + kvm_init_pmu_capability(); > > + > > + r = ops->hardware_setup(); > > + if (r != 0) > > + goto out_mmu_exit; > > The failure case of ops->hardware_setup() is unwound > by kvm_arch_exit() before this patch, do we need to > keep that old behavior ? As called out in the changelog, the call to ops->hardware_setup() was deliberately slotted in before the call to kvm_timer_init() so that kvm_arch_init() wouldn't need to unwind more stuff if harware_setup() fails. > > + /* > > + * Point of no return! DO NOT add error paths below this point unless > > + * absolutely necessary, as most operations from this point forward > > + * require unwinding. > > + */ > > + kvm_ops_update(ops); > > + > > kvm_timer_init(); > > > > if (pi_inject_timer == -1) > > @@ -9336,8 +9372,32 @@ int kvm_arch_init(void *opaque) > > set_hv_tscchange_cb(kvm_hyperv_tsc_notifier); > > #endif > > > > + kvm_register_perf_callbacks(ops->handle_intel_pt_intr); > > + > > + if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) > > + kvm_caps.supported_xss = 0; > > + > > +#define __kvm_cpu_cap_has(UNUSED_, f) kvm_cpu_cap_has(f) > > + cr4_reserved_bits = __cr4_reserved_bits(__kvm_cpu_cap_has, UNUSED_); > > +#undef __kvm_cpu_cap_has > > + > > + if (kvm_caps.has_tsc_control) { > > + /* > > + * Make sure the user can only configure tsc_khz values that > > + * fit into a signed integer. > > + * A min value is not calculated because it will always > > + * be 1 on all machines. > > + */ > > + u64 max = min(0x7fffffffULL, > > + __scale_tsc(kvm_caps.max_tsc_scaling_ratio, tsc_khz)); > > + kvm_caps.max_guest_tsc_khz = max; > > + } > > + kvm_caps.default_tsc_scaling_ratio = 1ULL << kvm_caps.tsc_scaling_ratio_frac_bits; > > + kvm_init_msr_list(); > > return 0; > > > > +out_mmu_exit: > > + kvm_mmu_vendor_module_exit(); > > out_free_percpu: > > free_percpu(user_return_msrs); > > out_free_x86_emulator_cache: