* [PATCH 0/6] Improve KVM's interaction with CPU hotplug
@ 2021-12-27 8:15 Chao Gao
2021-12-27 8:15 ` [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat Chao Gao
0 siblings, 1 reply; 6+ messages in thread
From: Chao Gao @ 2021-12-27 8:15 UTC (permalink / raw)
To: kvm, pbonzini, kevin.tian, tglx
Cc: Guo Ren, Wanpeng Li, David Hildenbrand, Maciej S. Szmigiero,
Dave Hansen, x86, linux-mips, Atish Patra, Paul Mackerras,
H. Peter Anvin, Alexander Gordeev, Claudio Imbrenda, Will Deacon,
kvmarm, linux-s390, Janosch Frank, Marc Zyngier, Joerg Roedel,
Huacai Chen, linux-riscv, Aleksandar Markovic, Ingo Molnar,
Catalin Marinas, Palmer Dabbelt, Christian Borntraeger, Chao Gao,
Ravi Bangoria, Albert Ou, Vasily Gorbik, Suzuki K Poulose,
Heiko Carstens, John Garry, Nicholas Piggin, Shaokun Zhang,
Borislav Petkov, Cédric Le Goater, Paul Walmsley,
Sumanth Korikkar, Sean Christopherson, Alexandru Elisei,
linux-arm-kernel, Jim Mattson, Thomas Bogendoerfer, Fabiano Rosas,
Mel Gorman, Anup Patel, Nick Desaulniers, linux-kernel,
Bharata B Rao, James Morse, kvm-riscv, Vitaly Kuznetsov,
linuxppc-dev, Thomas Richter
KVM registers its CPU hotplug callback to CPU starting section. And in the
callback, KVM enables hardware virtualization on hotplugged CPUs if any VM
is running on existing CPUs.
There are two problems in the process:
1. KVM doesn't do compatibility checks before enabling hardware
virtualization on hotplugged CPUs. This may cause #GP if VMX isn't
supported or vmentry failure if some in-use VMX features are missing on
hotplugged CPUs. Both break running VMs.
2. Callbacks in CPU STARTING section cannot fail. So, even if KVM finds
some incompatible CPUs, its callback cannot block CPU hotplug.
This series improves KVM's interaction with CPU hotplug to avoid
incompatible CPUs breaking running VMs. Following changes are made:
1. move KVM's CPU hotplug callback to ONLINE section (suggested by Thomas)
2. do compatibility checks on hotplugged CPUs.
3. abort onlining incompatible CPUs if there is a running VM.
This series is a follow-up to the discussion about KVM and CPU hotplug
https://lore.kernel.org/lkml/3d3296f0-9245-40f9-1b5a-efffdb082de9@redhat.com/T/
Note: this series is tested only on Intel systems.
Chao Gao (6):
KVM: x86: Move check_processor_compatibility from init ops to runtime
ops
KVM: x86: Use kvm_x86_ops in kvm_arch_check_processor_compat
KVM: Remove opaque from kvm_arch_check_processor_compat
KVM: Rename and move CPUHP_AP_KVM_STARTING to ONLINE section
KVM: x86: Remove WARN_ON in kvm_arch_check_processor_compat
KVM: Do compatibility checks on hotplugged CPUs
arch/arm64/kvm/arm.c | 2 +-
arch/mips/kvm/mips.c | 2 +-
arch/powerpc/kvm/powerpc.c | 2 +-
arch/riscv/kvm/main.c | 2 +-
arch/s390/kvm/kvm-s390.c | 2 +-
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/svm/svm.c | 4 +-
arch/x86/kvm/vmx/evmcs.c | 2 +-
arch/x86/kvm/vmx/evmcs.h | 2 +-
arch/x86/kvm/vmx/vmx.c | 12 +++---
arch/x86/kvm/x86.c | 7 +---
include/linux/cpuhotplug.h | 2 +-
include/linux/kvm_host.h | 2 +-
virt/kvm/kvm_main.c | 74 ++++++++++++++++++++++++---------
14 files changed, 74 insertions(+), 43 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat 2021-12-27 8:15 [PATCH 0/6] Improve KVM's interaction with CPU hotplug Chao Gao @ 2021-12-27 8:15 ` Chao Gao 2022-01-10 23:06 ` Sean Christopherson 0 siblings, 1 reply; 6+ messages in thread From: Chao Gao @ 2021-12-27 8:15 UTC (permalink / raw) To: kvm, pbonzini, kevin.tian, tglx Cc: x86, Wanpeng Li, David Hildenbrand, Dave Hansen, linux-mips, Atish Patra, Paul Mackerras, H. Peter Anvin, Alexander Gordeev, Claudio Imbrenda, Will Deacon, Maciej S. Szmigiero, linux-s390, Janosch Frank, Marc Zyngier, Joerg Roedel, Huacai Chen, linux-riscv, kvmarm, Aleksandar Markovic, Ingo Molnar, Catalin Marinas, Palmer Dabbelt, Christian Borntraeger, Chao Gao, Ravi Bangoria, Albert Ou, Vasily Gorbik, Suzuki K Poulose, Heiko Carstens, Nicholas Piggin, Borislav Petkov, Cédric Le Goater, Paul Walmsley, Alexandru Elisei, linux-arm-kernel, Jim Mattson, Thomas Bogendoerfer, Fabiano Rosas, Sean Christopherson, Anup Patel, linux-kernel, Bharata B Rao, James Morse, kvm-riscv, Vitaly Kuznetsov, linuxppc-dev No arch implementation uses this opaque now. Signed-off-by: Chao Gao <chao.gao@intel.com> --- arch/arm64/kvm/arm.c | 2 +- arch/mips/kvm/mips.c | 2 +- arch/powerpc/kvm/powerpc.c | 2 +- arch/riscv/kvm/main.c | 2 +- arch/s390/kvm/kvm-s390.c | 2 +- arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 14 ++------------ 8 files changed, 9 insertions(+), 19 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 14106a7c75b5..c196f005a2d3 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -73,7 +73,7 @@ int kvm_arch_hardware_setup(void *opaque) return 0; } -int kvm_arch_check_processor_compat(void *opaque) +int kvm_arch_check_processor_compat(void) { return 0; } diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index e59cb6246f76..c5dc4fe53bfc 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -140,7 +140,7 @@ int kvm_arch_hardware_setup(void *opaque) return 0; } -int kvm_arch_check_processor_compat(void *opaque) +int kvm_arch_check_processor_compat(void) { return 0; } diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c index 2ad0ccd202d5..30c817f3fa0c 100644 --- a/arch/powerpc/kvm/powerpc.c +++ b/arch/powerpc/kvm/powerpc.c @@ -423,7 +423,7 @@ int kvm_arch_hardware_setup(void *opaque) return 0; } -int kvm_arch_check_processor_compat(void *opaque) +int kvm_arch_check_processor_compat(void) { return kvmppc_core_check_processor_compat(); } diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c index 421ecf4e6360..3b0b104e443f 100644 --- a/arch/riscv/kvm/main.c +++ b/arch/riscv/kvm/main.c @@ -20,7 +20,7 @@ long kvm_arch_dev_ioctl(struct file *filp, return -EINVAL; } -int kvm_arch_check_processor_compat(void *opaque) +int kvm_arch_check_processor_compat(void) { return 0; } diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 9c6d45d0d345..99c70d881cb6 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -252,7 +252,7 @@ int kvm_arch_hardware_enable(void) return 0; } -int kvm_arch_check_processor_compat(void *opaque) +int kvm_arch_check_processor_compat(void) { return 0; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 770b68e72391..aa09c8792134 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11380,7 +11380,7 @@ void kvm_arch_hardware_unsetup(void) static_call(kvm_x86_hardware_unsetup)(); } -int kvm_arch_check_processor_compat(void *opaque) +int kvm_arch_check_processor_compat(void) { struct cpuinfo_x86 *c = &cpu_data(smp_processor_id()); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 05862176df6a..630ffd2289c6 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1307,7 +1307,7 @@ int kvm_arch_hardware_enable(void); void kvm_arch_hardware_disable(void); int kvm_arch_hardware_setup(void *opaque); void kvm_arch_hardware_unsetup(void); -int kvm_arch_check_processor_compat(void *opaque); +int kvm_arch_check_processor_compat(void); int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu); bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu); int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 168d0ab93c88..4e1e7770e984 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5600,22 +5600,14 @@ struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) return &kvm_running_vcpu; } -struct kvm_cpu_compat_check { - void *opaque; - int *ret; -}; - static void check_processor_compat(void *data) { - struct kvm_cpu_compat_check *c = data; - - *c->ret = kvm_arch_check_processor_compat(c->opaque); + *(int *)data = kvm_arch_check_processor_compat(); } int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, struct module *module) { - struct kvm_cpu_compat_check c; int r; int cpu; @@ -5643,10 +5635,8 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, if (r < 0) goto out_free_1; - c.ret = &r; - c.opaque = opaque; for_each_online_cpu(cpu) { - smp_call_function_single(cpu, check_processor_compat, &c, 1); + smp_call_function_single(cpu, check_processor_compat, &r, 1); if (r < 0) goto out_free_2; } -- 2.25.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat 2021-12-27 8:15 ` [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat Chao Gao @ 2022-01-10 23:06 ` Sean Christopherson 2022-01-11 3:19 ` Chao Gao 0 siblings, 1 reply; 6+ messages in thread From: Sean Christopherson @ 2022-01-10 23:06 UTC (permalink / raw) To: Chao Gao Cc: x86, Wanpeng Li, kvm, David Hildenbrand, Dave Hansen, linux-mips, Atish Patra, Paul Mackerras, H. Peter Anvin, Alexander Gordeev, Claudio Imbrenda, Will Deacon, Maciej S. Szmigiero, linux-s390, Janosch Frank, Marc Zyngier, Joerg Roedel, Huacai Chen, linux-riscv, kvmarm, Aleksandar Markovic, Ingo Molnar, Catalin Marinas, Palmer Dabbelt, Christian Borntraeger, Ravi Bangoria, kevin.tian, Albert Ou, Vasily Gorbik, Suzuki K Poulose, Heiko Carstens, Nicholas Piggin, Borislav Petkov, Cédric Le Goater, Paul Walmsley, tglx, Alexandru Elisei, linux-arm-kernel, Jim Mattson, Thomas Bogendoerfer, Fabiano Rosas, Anup Patel, linux-kernel, Bharata B Rao, James Morse, kvm-riscv, pbonzini, Vitaly Kuznetsov, linuxppc-dev On Mon, Dec 27, 2021, Chao Gao wrote: > No arch implementation uses this opaque now. Except for the RISC-V part, this can be a pure revert of commit b99040853738 ("KVM: Pass kvm_init()'s opaque param to additional arch funcs"). I think it makes sense to process it as a revert, with a short blurb in the changelog to note that RISC-V is manually modified as RISC-V support came along in the interim. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat 2022-01-10 23:06 ` Sean Christopherson @ 2022-01-11 3:19 ` Chao Gao 2022-01-12 17:20 ` Sean Christopherson 0 siblings, 1 reply; 6+ messages in thread From: Chao Gao @ 2022-01-11 3:19 UTC (permalink / raw) To: Sean Christopherson Cc: x86, Wanpeng Li, kvm, David Hildenbrand, Dave Hansen, linux-mips, Atish Patra, Paul Mackerras, H. Peter Anvin, Alexander Gordeev, Claudio Imbrenda, Will Deacon, Maciej S. Szmigiero, linux-s390, Janosch Frank, Marc Zyngier, Joerg Roedel, Huacai Chen, linux-riscv, kvmarm, Aleksandar Markovic, Ingo Molnar, Catalin Marinas, Palmer Dabbelt, Christian Borntraeger, Ravi Bangoria, kevin.tian, Albert Ou, Vasily Gorbik, Suzuki K Poulose, Heiko Carstens, Nicholas Piggin, Borislav Petkov, Cédric Le Goater, Paul Walmsley, tglx, Alexandru Elisei, linux-arm-kernel, Jim Mattson, Thomas Bogendoerfer, Fabiano Rosas, Anup Patel, linux-kernel, Bharata B Rao, James Morse, kvm-riscv, pbonzini, Vitaly Kuznetsov, linuxppc-dev On Mon, Jan 10, 2022 at 11:06:44PM +0000, Sean Christopherson wrote: >On Mon, Dec 27, 2021, Chao Gao wrote: >> No arch implementation uses this opaque now. > >Except for the RISC-V part, this can be a pure revert of commit b99040853738 ("KVM: >Pass kvm_init()'s opaque param to additional arch funcs"). I think it makes sense >to process it as a revert, with a short blurb in the changelog to note that RISC-V >is manually modified as RISC-V support came along in the interim. commit b99040853738 adds opaque param to kvm_arch_hardware_setup(), which isn't reverted in this patch. I.e., this patch is a partial revert of b99040853738 plus manual changes to RISC-V. Given that, "process it as a revert" means clearly say in changelog that this commit contains a partial revert of commit b99040853738 ("KVM: Pass kvm_init()'s opaque param to additional arch funcs"). Right? ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat 2022-01-11 3:19 ` Chao Gao @ 2022-01-12 17:20 ` Sean Christopherson 2022-01-12 17:21 ` Sean Christopherson 0 siblings, 1 reply; 6+ messages in thread From: Sean Christopherson @ 2022-01-12 17:20 UTC (permalink / raw) To: Chao Gao Cc: x86, Wanpeng Li, kvm, David Hildenbrand, Dave Hansen, linux-mips, Atish Patra, Paul Mackerras, H. Peter Anvin, Alexander Gordeev, Claudio Imbrenda, Will Deacon, Maciej S. Szmigiero, linux-s390, Janosch Frank, Marc Zyngier, Joerg Roedel, Huacai Chen, linux-riscv, kvmarm, Aleksandar Markovic, Ingo Molnar, Catalin Marinas, Palmer Dabbelt, Christian Borntraeger, Ravi Bangoria, kevin.tian, Albert Ou, Vasily Gorbik, Suzuki K Poulose, Heiko Carstens, Nicholas Piggin, Borislav Petkov, Cédric Le Goater, Paul Walmsley, tglx, Alexandru Elisei, linux-arm-kernel, Jim Mattson, Thomas Bogendoerfer, Fabiano Rosas, Anup Patel, linux-kernel, Bharata B Rao, James Morse, kvm-riscv, pbonzini, Vitaly Kuznetsov, linuxppc-dev On Tue, Jan 11, 2022, Chao Gao wrote: > On Mon, Jan 10, 2022 at 11:06:44PM +0000, Sean Christopherson wrote: > >On Mon, Dec 27, 2021, Chao Gao wrote: > >> No arch implementation uses this opaque now. > > > >Except for the RISC-V part, this can be a pure revert of commit b99040853738 ("KVM: > >Pass kvm_init()'s opaque param to additional arch funcs"). I think it makes sense > >to process it as a revert, with a short blurb in the changelog to note that RISC-V > >is manually modified as RISC-V support came along in the interim. > > commit b99040853738 adds opaque param to kvm_arch_hardware_setup(), which isn't > reverted in this patch. I.e., this patch is a partial revert of b99040853738 > plus manual changes to RISC-V. Given that, "process it as a revert" means > clearly say in changelog that this commit contains a partial revert of commit > b99040853738 ("KVM: Pass kvm_init()'s opaque param to additional arch funcs"). > > Right? What I meant is literally do git revert -s b99040853738 and then manually handle RISC-V. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat 2022-01-12 17:20 ` Sean Christopherson @ 2022-01-12 17:21 ` Sean Christopherson 0 siblings, 0 replies; 6+ messages in thread From: Sean Christopherson @ 2022-01-12 17:21 UTC (permalink / raw) To: Chao Gao Cc: x86, Wanpeng Li, kvm, David Hildenbrand, Dave Hansen, linux-mips, Atish Patra, Paul Mackerras, H. Peter Anvin, Alexander Gordeev, Claudio Imbrenda, Will Deacon, Maciej S. Szmigiero, linux-s390, Janosch Frank, Marc Zyngier, Joerg Roedel, Huacai Chen, linux-riscv, kvmarm, Aleksandar Markovic, Ingo Molnar, Catalin Marinas, Palmer Dabbelt, Christian Borntraeger, Ravi Bangoria, kevin.tian, Albert Ou, Vasily Gorbik, Suzuki K Poulose, Heiko Carstens, Nicholas Piggin, Borislav Petkov, Cédric Le Goater, Paul Walmsley, tglx, Alexandru Elisei, linux-arm-kernel, Jim Mattson, Thomas Bogendoerfer, Fabiano Rosas, Anup Patel, linux-kernel, Bharata B Rao, James Morse, kvm-riscv, pbonzini, Vitaly Kuznetsov, linuxppc-dev On Wed, Jan 12, 2022, Sean Christopherson wrote: > On Tue, Jan 11, 2022, Chao Gao wrote: > > On Mon, Jan 10, 2022 at 11:06:44PM +0000, Sean Christopherson wrote: > > >On Mon, Dec 27, 2021, Chao Gao wrote: > > >> No arch implementation uses this opaque now. > > > > > >Except for the RISC-V part, this can be a pure revert of commit b99040853738 ("KVM: > > >Pass kvm_init()'s opaque param to additional arch funcs"). I think it makes sense > > >to process it as a revert, with a short blurb in the changelog to note that RISC-V > > >is manually modified as RISC-V support came along in the interim. > > > > commit b99040853738 adds opaque param to kvm_arch_hardware_setup(), which isn't > > reverted in this patch. I.e., this patch is a partial revert of b99040853738 > > plus manual changes to RISC-V. Given that, "process it as a revert" means > > clearly say in changelog that this commit contains a partial revert of commit > > b99040853738 ("KVM: Pass kvm_init()'s opaque param to additional arch funcs"). > > > > Right? > > What I meant is literally do > > git revert -s b99040853738 > > and then manually handle RISC-V. Doh, to be clear, "manually handle RISC-V _in the same commit_". ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-12 17:22 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-12-27 8:15 [PATCH 0/6] Improve KVM's interaction with CPU hotplug Chao Gao 2021-12-27 8:15 ` [PATCH 3/6] KVM: Remove opaque from kvm_arch_check_processor_compat Chao Gao 2022-01-10 23:06 ` Sean Christopherson 2022-01-11 3:19 ` Chao Gao 2022-01-12 17:20 ` Sean Christopherson 2022-01-12 17:21 ` Sean Christopherson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).