From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB868C2B9F8 for ; Mon, 24 May 2021 16:00:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ADBC961209 for ; Mon, 24 May 2021 16:00:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235075AbhEXQCV convert rfc822-to-8bit (ORCPT ); Mon, 24 May 2021 12:02:21 -0400 Received: from mail.kernel.org ([198.145.29.99]:46618 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234717AbhEXQA6 (ORCPT ); Mon, 24 May 2021 12:00:58 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9D86461993; Mon, 24 May 2021 15:46:26 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1llCme-003GgK-G2; Mon, 24 May 2021 16:46:24 +0100 Date: Mon, 24 May 2021 16:46:23 +0100 Message-ID: <87eedww2wg.wl-maz@kernel.org> From: Marc Zyngier To: Andreas =?UTF-8?B?RsOkcmJlcg==?= Cc: linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Rob Herring , Heiko Stuebner , devicetree@vger.kernel.org Subject: Re: [PATCH RFC 5/9] arm64: dts: rockchip: rk1808k-toybrick-m0: Suppress vGIC interrupt In-Reply-To: References: <20210516230551.12469-1-afaerber@suse.de> <20210516230551.12469-6-afaerber@suse.de> <87fsylvhck.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: afaerber@suse.de, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, robh+dt@kernel.org, heiko@sntech.de, devicetree@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On Mon, 24 May 2021 15:40:22 +0100, Andreas Färber wrote: > > On 17.05.21 11:29, Marc Zyngier wrote: > > On Mon, 17 May 2021 00:05:47 +0100, > > Andreas Färber wrote: > >> > >> Avoid the kernel getting stuck after: > >> > >> [ 1.175956] kvm [1]: IPA Size Limit: 40 bits > >> [ 1.177164] kvm [1]: vgic-v2@ff320000 > >> [ 1.177545] kvm [1]: GIC system register CPU interface enabled > >> > >> or when dropping GICV reg entry: > >> > >> [ 1.176001] kvm [1]: IPA Size Limit: 40 bits > >> [ 1.177191] kvm [1]: GICv3: no GICV resource entry > >> [ 1.177664] kvm [1]: disabling GICv2 emulation > >> [ 1.178115] kvm [1]: GIC system register CPU interface enabled > >> > >> Signed-off-by: Andreas Färber > >> --- > >> arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts | 4 ++++ > >> 1 file changed, 4 insertions(+) > >> > >> diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts > >> index 2f8075d2391c..15293a8576c6 100644 > >> --- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts > >> +++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts > >> @@ -48,6 +48,10 @@ &cpu1 { > >> cpu-supply = <&vdd_cpu>; > >> }; > >> > >> +&gic { > >> + /delete-property/ interrupts; > >> +}; > >> + > >> &uart2 { > >> status = "okay"; > >> clocks = <&xin24m>; > > > > As I said in my reply to the cover letter, this is not an acceptable > > outcome. Please add some debug to kvm_vgic_hyp_init() to understand > > where this is hanging and why. > > Many thanks for that pointer. > > So, as alternative to dropping the DT interrupts property above, I could > also work around this issue by commenting out > vgic-init.c:vgic_init_cpu_starting()'s enable_percpu_irq() call. > > Otherwise I am seeing the following call flow: > > cpuhp_setup_state() -> __cpuhp_setup_state_cpuslocked() -> > cpuhp_issue_call() -> cpuhp_invoke_ap_callback() -> __cpuhp_kick_ap() -> > wait_for_ap_thread() -> wait_for_completion() --- doesn't return > > With kvm_info() / printk(): > > [ 1.244079] kvm [1]: IPA Size Limit: 40 bits > > [ 1.245205] kvm [1]: vgic-v2@ff320000 > > [ 1.245584] kvm [1]: GIC system register CPU interface enabled > > [ 1.246177] kvm [1]: before cpuhp_setup_state > > [ 1.246605] __cpuhp_setup_state_cpuslocked: kvm/arm/vgic:starting > > [ 1.247198] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0: > state 225 > > [ 1.247933] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0: > before cpuhp_issue_call > > [ 1.248745] cpuhp_issue_call: before invoke > > [ 1.249154] cpuhp_issue_call: before AP invoke > > [ 1.249585] cpuhp_invoke_ap_callback > > [ 1.249936] cpuhp_invoke_ap_callback: after cpu_online > > [ 1.250435] cpuhp_invoke_ap_callback: before st->thread > > [ 1.250944] cpuhp_invoke_ap_callback: after st->thread > > [ 1.251445] __cpuhp_kick_ap > > [ 1.251731] __cpuhp_kick_ap: not returned > > [ 1.252140] vgic_init_cpu_starting: 9 > > [ 1.252507] vgic_init_cpu_starting: done > > [ 1.255538] __cpuhp_kick_ap: wait_for_ap_thread And you never see any RCU stall after that? It looks like a CPU has disappeared in the weeds after enabling the per-CPU interrupt. Please instrument what happens in drivers/irqchip/irq-gic-v3.c::gic_unmask_irq() when d->hwirq == 9, a well as vgic_maintenance_handler(), just in case it gets called... Thanks, M. -- Without deviation from the norm, progress is not possible.