From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21C08C4332F for ; Tue, 15 Nov 2022 20:21:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230492AbiKOUVG (ORCPT ); Tue, 15 Nov 2022 15:21:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230394AbiKOUVF (ORCPT ); Tue, 15 Nov 2022 15:21:05 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 024282D1C8 for ; Tue, 15 Nov 2022 12:21:05 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id v28so15117130pfi.12 for ; Tue, 15 Nov 2022 12:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=XxrH205VSL6PuZxNZy5QSHBg1esXqApruwRLQHoD1FQ=; b=QoLsrppxCqrrm8Et/yxXiG7qWE0hiOrc+RLWkwKVvGYvEXTURW54Ct7G0aFnW7vDom O3WnXZWOxSKhgS+uv00LIKaFYAiY4w1O0HuVw0sHlWnVNkXyDKqsQfpccjPy0o//76Ot 9HbC7sv+GYjRBhUyAvWsaFUrflZFb6BoKafnLIcIXy0vGyf6tKbOGR6vv68o6WUCslEQ BOT4pCgeSJnKExa2xfKvrZGwBZhzuokrCi6o4yMRfhSIGNbmSd/1S94WiYpeZI3RH1jS tMxrZXr8PoWD2Hx+ZMuaUIAmz2hZjOEiOFHGi/ELuekxNrIkmFIAeW4IyLdTJL7wajgf 6FdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XxrH205VSL6PuZxNZy5QSHBg1esXqApruwRLQHoD1FQ=; b=QaUkLUI52NEXA/Y0dSqgGdDCvuntLirXPVCFI4ucXuKnSAjhTaF7sorJsHrm8EuEIL DPpVOVRvJ4JOcAlolnLasS8iKxXKh0riZTuPKXo07fdbiQGsFDxFtnVYPGiM+lHxvbfZ JkSRF4OH2T9T1hWuG97HVNzyeR0srHjzGOJ1Xq6oDJfZtKKB0miLDVN9y7Xf8L1c/WbL jz8GCN24s8Si2SiiMnAgT0rI3PA/arp2Eozi3BTzCTlLzfv98+RU6pUVU+71D7Ock94y iC8wxW3TipNZEMwoTS7qtZrPI5HtHjHE52I1Xwd7i1y74WzCz2h/YuBH4xjQ9d9wCrQx fb2g== X-Gm-Message-State: ANoB5plRwKhrSVWgpVeuDb8ahnRk/MWAg57SI/D5OWzgVBNNYmqalkI6 eksfEgJQzdAV6AubVYXD13Cn6g== X-Google-Smtp-Source: AA0mqf7I+HCT8JLUK2SnELicaIfj+5Xh/wdDfnKOa9DuiFN7rhFS7bzZLz9XVMfRQzvW4tmCjZQXQw== X-Received: by 2002:aa7:9493:0:b0:56b:9ae8:ca05 with SMTP id z19-20020aa79493000000b0056b9ae8ca05mr19812088pfk.59.1668543664233; Tue, 15 Nov 2022 12:21:04 -0800 (PST) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id o15-20020a170902d4cf00b001754fa42065sm10448477plg.143.2022.11.15.12.21.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 12:21:03 -0800 (PST) Date: Tue, 15 Nov 2022 20:21:00 +0000 From: Sean Christopherson To: "Huang, Kai" Cc: "farman@linux.ibm.com" , "frankja@linux.ibm.com" , "mjrosato@linux.ibm.com" , "vkuznets@redhat.com" , "chenhuacai@kernel.org" , "aou@eecs.berkeley.edu" , "palmer@dabbelt.com" , "paul.walmsley@sifive.com" , "maz@kernel.org" , "anup@brainfault.org" , "imbrenda@linux.ibm.com" , "pbonzini@redhat.com" , "borntraeger@linux.ibm.com" , "aleksandar.qemu.devel@gmail.com" , "kvm@vger.kernel.org" , "atishp@atishpatra.org" , "farosas@linux.ibm.com" , "david@redhat.com" , "Yao, Yuan" , "mpe@ellerman.id.au" , "alexandru.elisei@arm.com" , "linux-s390@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "tglx@linutronix.de" , "Yamahata, Isaku" , "kvmarm@lists.linux.dev" , "james.morse@arm.com" , "kvm-riscv@lists.infradead.org" , "suzuki.poulose@arm.com" , "linuxppc-dev@lists.ozlabs.org" , "linux-arm-kernel@lists.infradead.org" , "linux-mips@vger.kernel.org" , "Gao, Chao" , "oliver.upton@linux.dev" , "kvmarm@lists.cs.columbia.edu" , "linux-riscv@lists.infradead.org" Subject: Re: [PATCH 38/44] KVM: Disable CPU hotplug during hardware enabling Message-ID: References: <20221102231911.3107438-1-seanjc@google.com> <20221102231911.3107438-39-seanjc@google.com> <88e920944de70e7d69a98f74005b49c59b5aaa3b.camel@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-s390@vger.kernel.org On Tue, Nov 15, 2022, Sean Christopherson wrote: > On Thu, Nov 10, 2022, Huang, Kai wrote: > > On Thu, 2022-11-10 at 01:33 +0000, Huang, Kai wrote: > > > > @@ -9283,7 +9283,13 @@ static int > > > > kvm_x86_check_processor_compatibility(struct kvm_x86_init_ops *ops) > > > >   int cpu = smp_processor_id(); > > > >   struct cpuinfo_x86 *c = &cpu_data(cpu); > > > >   > > > > - WARN_ON(!irqs_disabled()); > > > > + /* > > > > + * Compatibility checks are done when loading KVM and when enabling > > > > + * hardware, e.g. during CPU hotplug, to ensure all online CPUs are > > > > + * compatible, i.e. KVM should never perform a compatibility check > > > > on > > > > + * an offline CPU. > > > > + */ > > > > + WARN_ON(!irqs_disabled() && cpu_active(cpu)); > > > >   > > > > > > Also, the logic of: > > > > > > !irqs_disabled() && cpu_active(cpu) > > > > > > is quite weird. > > > > > > The original "WARN(!irqs_disabled())" is reasonable because in STARTING > > > section > > > the IRQ is indeed disabled. > > > > > > But this doesn't make sense anymore after we move to ONLINE section, in which > > > IRQ has already been enabled (see start_secondary()).  IIUC the WARN_ON() > > > doesn't get exploded is purely because there's an additional cpu_active(cpu) > > > check. > > > > > > So, a more reasonable check should be something like: > > > > > > WARN_ON(irqs_disabled() || cpu_active(cpu) || !cpu_online(cpu)); > > > > > > Or we can simply do: > > > > > > WARN_ON(!cpu_online(cpu) || cpu_active(cpu)); > > > > > > (because I don't know whether it's possible IRQ can somehow get disabled in > > > ONLINE section). > > > > > > Btw above is purely based on code analysis, but I haven't done any test. > > > > Hmm.. I wasn't thinking thoroughly. I forgot CPU compatibility check also > > happens on all online cpus when loading KVM. For this case, IRQ is disabled and > > cpu_active() is true. For the hotplug case, IRQ is enabled but cpu_active() is > > false. > > Actually, you're right (and wrong). You're right in that the WARN is flawed. And > the reason for that is because you're wrong about the hotplug case. In this version > of things, the compatibility checks are routed through hardware enabling, i.e. this > flow is used only when loading KVM. This helper should only be called via SMP function > call, which means that IRQs should always be disabled. Grr, but not routing through this helper is flawed in that KVM doesn't do the CR4 checks in the hardware enabling case. Don't think that changes the WARN, but other patches in this series need tweaks.