From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9793C433FE for ; Tue, 15 Nov 2022 20:22:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ukGJoWe4K6ekqGwY4xOG4Oim97ZV8IXBnBQ6rxsCOHk=; b=E4r+kPKtwZpqyh 4YppSVFBrd7lwUy3IeZPBY7Vb7s8YO+W/vKq6ibmQ7XUkXRotpQJ/Wk/JAC9Ycmi15/5qvgw0OAzd 3fRIpe30Vvw3h0KpvGmYtTOWSWmFDPPC6Yx1aiHdCJmsqAIeighC2sGELftfc287N2Q3MEX+zFbxD OhRyMuVUveJdQZbzCLFV8/ndWp+/Fb6+/8AmaWOi6tj8LL3LnKC29FmoJEuHSt0vQN95TtWIOas12 +lB+QS4G4Ad9iI9fLCsWu6GmRyGK0oE3Ho9HpaPJL0oteusyefmr0+o6bntMaEr30I4OXa3yEVp5i hz/VvTGmkRkrx4kRm2iA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ov2Qi-00EX7Y-5l; Tue, 15 Nov 2022 20:21:12 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ov2Qc-00EX4j-1Q for linux-arm-kernel@lists.infradead.org; Tue, 15 Nov 2022 20:21:08 +0000 Received: by mail-pf1-x429.google.com with SMTP id b29so15100734pfp.13 for ; Tue, 15 Nov 2022 12:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=XxrH205VSL6PuZxNZy5QSHBg1esXqApruwRLQHoD1FQ=; b=QoLsrppxCqrrm8Et/yxXiG7qWE0hiOrc+RLWkwKVvGYvEXTURW54Ct7G0aFnW7vDom O3WnXZWOxSKhgS+uv00LIKaFYAiY4w1O0HuVw0sHlWnVNkXyDKqsQfpccjPy0o//76Ot 9HbC7sv+GYjRBhUyAvWsaFUrflZFb6BoKafnLIcIXy0vGyf6tKbOGR6vv68o6WUCslEQ BOT4pCgeSJnKExa2xfKvrZGwBZhzuokrCi6o4yMRfhSIGNbmSd/1S94WiYpeZI3RH1jS tMxrZXr8PoWD2Hx+ZMuaUIAmz2hZjOEiOFHGi/ELuekxNrIkmFIAeW4IyLdTJL7wajgf 6FdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XxrH205VSL6PuZxNZy5QSHBg1esXqApruwRLQHoD1FQ=; b=3givj7uHvKy4jYOsOcMHzk1p4G8wXPM0x8uGlxiRjPsRHEywTohd9aZ8gVECs1PkZE CMLgCMFIrtrzz5LiTkb6gy5aYHRkbIBn0zOlaDr2ct6tE2zPuTuRG/njM7OXZJwu18YH 8g8KiU5AuXUVHQM5GvaB7seRyFIroje6nM81pAzXvp+Tls+7vrmNguUfpVGVJoo04T1k wWiF7vg7U9P95F7Xf/l92hu+fCnsZUpKY5JfywXMGih2nXw8fkABVTKHaZaAtQQEYcsd znjCgpeh6DpGkafgNsln51VQvIOTuxzkk2PyhCuEIpCfJfcyzW44a6277XEH8cz8GWPi zjaQ== X-Gm-Message-State: ANoB5plpLIakjCzMfnFrYdNy9bTc+x/eSTOx2wAAJzAsobUF39ElZ/go KeO66VGa8IxQ0aFK/7rSJHyqWQ== X-Google-Smtp-Source: AA0mqf7I+HCT8JLUK2SnELicaIfj+5Xh/wdDfnKOa9DuiFN7rhFS7bzZLz9XVMfRQzvW4tmCjZQXQw== X-Received: by 2002:aa7:9493:0:b0:56b:9ae8:ca05 with SMTP id z19-20020aa79493000000b0056b9ae8ca05mr19812088pfk.59.1668543664233; Tue, 15 Nov 2022 12:21:04 -0800 (PST) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id o15-20020a170902d4cf00b001754fa42065sm10448477plg.143.2022.11.15.12.21.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 12:21:03 -0800 (PST) Date: Tue, 15 Nov 2022 20:21:00 +0000 From: Sean Christopherson To: "Huang, Kai" Cc: "farman@linux.ibm.com" , "frankja@linux.ibm.com" , "mjrosato@linux.ibm.com" , "vkuznets@redhat.com" , "chenhuacai@kernel.org" , "aou@eecs.berkeley.edu" , "palmer@dabbelt.com" , "paul.walmsley@sifive.com" , "maz@kernel.org" , "anup@brainfault.org" , "imbrenda@linux.ibm.com" , "pbonzini@redhat.com" , "borntraeger@linux.ibm.com" , "aleksandar.qemu.devel@gmail.com" , "kvm@vger.kernel.org" , "atishp@atishpatra.org" , "farosas@linux.ibm.com" , "david@redhat.com" , "Yao, Yuan" , "mpe@ellerman.id.au" , "alexandru.elisei@arm.com" , "linux-s390@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "tglx@linutronix.de" , "Yamahata, Isaku" , "kvmarm@lists.linux.dev" , "james.morse@arm.com" , "kvm-riscv@lists.infradead.org" , "suzuki.poulose@arm.com" , "linuxppc-dev@lists.ozlabs.org" , "linux-arm-kernel@lists.infradead.org" , "linux-mips@vger.kernel.org" , "Gao, Chao" , "oliver.upton@linux.dev" , "kvmarm@lists.cs.columbia.edu" , "linux-riscv@lists.infradead.org" Subject: Re: [PATCH 38/44] KVM: Disable CPU hotplug during hardware enabling Message-ID: References: <20221102231911.3107438-1-seanjc@google.com> <20221102231911.3107438-39-seanjc@google.com> <88e920944de70e7d69a98f74005b49c59b5aaa3b.camel@intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221115_122106_108458_5F27F6FE X-CRM114-Status: GOOD ( 29.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Nov 15, 2022, Sean Christopherson wrote: > On Thu, Nov 10, 2022, Huang, Kai wrote: > > On Thu, 2022-11-10 at 01:33 +0000, Huang, Kai wrote: > > > > @@ -9283,7 +9283,13 @@ static int > > > > kvm_x86_check_processor_compatibility(struct kvm_x86_init_ops *ops) > > > > =A0 int cpu =3D smp_processor_id(); > > > > =A0 struct cpuinfo_x86 *c =3D &cpu_data(cpu); > > > > =A0 > > > > - WARN_ON(!irqs_disabled()); > > > > + /* > > > > + * Compatibility checks are done when loading KVM and when enabli= ng > > > > + * hardware, e.g. during CPU hotplug, to ensure all online CPUs a= re > > > > + * compatible, i.e. KVM should never perform a compatibility check > > > > on > > > > + * an offline CPU. > > > > + */ > > > > + WARN_ON(!irqs_disabled() && cpu_active(cpu)); > > > > =A0 > > > = > > > Also, the logic of: > > > = > > > !irqs_disabled() && cpu_active(cpu) > > > = > > > is quite weird. > > > = > > > The original "WARN(!irqs_disabled())" is reasonable because in STARTI= NG > > > section > > > the IRQ is indeed disabled. > > > = > > > But this doesn't make sense anymore after we move to ONLINE section, = in which > > > IRQ has already been enabled (see start_secondary()).=A0 IIUC the WAR= N_ON() > > > doesn't get exploded is purely because there's an additional cpu_acti= ve(cpu) > > > check. > > > = > > > So, a more reasonable check should be something like: > > > = > > > WARN_ON(irqs_disabled() || cpu_active(cpu) || !cpu_online(cpu)); > > > = > > > Or we can simply do: > > > = > > > WARN_ON(!cpu_online(cpu) || cpu_active(cpu)); > > > = > > > (because I don't know whether it's possible IRQ can somehow get disab= led in > > > ONLINE section). > > > = > > > Btw above is purely based on code analysis, but I haven't done any te= st. > > = > > Hmm.. I wasn't thinking thoroughly. I forgot CPU compatibility check a= lso > > happens on all online cpus when loading KVM. For this case, IRQ is dis= abled and > > cpu_active() is true. For the hotplug case, IRQ is enabled but cpu_ac= tive() is > > false. > = > Actually, you're right (and wrong). You're right in that the WARN is fla= wed. And > the reason for that is because you're wrong about the hotplug case. In t= his version > of things, the compatibility checks are routed through hardware enabling,= i.e. this > flow is used only when loading KVM. This helper should only be called vi= a SMP function > call, which means that IRQs should always be disabled. Grr, but not routing through this helper is flawed in that KVM doesn't do t= he CR4 checks in the hardware enabling case. Don't think that changes the WAR= N, but other patches in this series need tweaks. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel