From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC403C433F5 for ; Fri, 22 Oct 2021 02:02:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC7EA61163 for ; Fri, 22 Oct 2021 02:02:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AC7EA61163 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DsfJ0RwWye6aKOJNpSOqQG5RrgPtRw8HjhtdpXtFcPQ=; b=37wuhp1KwtkCtI zkOxEVxj1xw80J1KaBNGUaFS25aMHXkKvhx9M4R91+OAmN8l6RVfZpMhNsoH/+dgrIfenHCenKI3t CbG3zzwXQsmPoq6BBtIwfSqQg5Anzpd0x1XBDtxk9l/w5ccJz/xKLqU5AMqL1u+kWm/6bHZqSC6aF BbjssUM43d4FpBOUJm3XM7n10Ofn+lOA0qJrJOIEYtElB7QN+cIohF5vtrJ3jjQ0G2zCncVbi77zq XN/DkLBI+ySwAnftw7mxwTWR2apMb2sR1Zna4ohRGobZVkNlmts7UXP+RNVPMwkIG0vSobxooIzhA C6P+P3KLoUgoMZzOwI6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mdjov-009SRJ-CU; Fri, 22 Oct 2021 01:58:09 +0000 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mdjor-009SPr-PC for linux-arm-kernel@lists.infradead.org; Fri, 22 Oct 2021 01:58:07 +0000 Received: by mail-pl1-x62f.google.com with SMTP id e10so1663943plh.8 for ; Thu, 21 Oct 2021 18:58:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=8sLhvlh8s15YkZACc0nmuzMn6IO+rS87+1k5lj2Suns=; b=Hgxc3vPOclZDyHlAlzsRKPtJU22EzuXMwvdS9C7YN9YUouc7VtBYGNqVuc+4VPYJyc 6SmyBLvK9dTAmpeRfrmTm4uwyZz8RjxAURXS5MLx7MvwcwcgcgsC5+0NcHDLtkO5j7Bv 9aCDQzxBwFb3KPhii2zCFzyDtL7IKqpU/crF3Dl0yLysNZTW4DbbAkzM39i1hPlr2bpD /PulKpKM3isNDcYPCHl5zXb3yG5APxlT5XVFAm/+l2QdWgxIF/3t3nCeQ+xMmDS6JaSl IzOMnNSCf4w0STSAyiKZxb2nQAzz3mlvbYv/kMSvr8Vh+E3G7uiZUHZ1c4FPUCNtSHke rTXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8sLhvlh8s15YkZACc0nmuzMn6IO+rS87+1k5lj2Suns=; b=WNanbU8Kh/AkWZelya26MYt1q/w/IF6liHZvY7D6WdG/46bgZHeWb6Y/WR/MvGSWF5 D5wnrJOmCCxRZFOM9VWdWAUypAMyzLQxncRb90u9l/74UXmoNfqRhR/pkjTwvZ2pVyKJ RIAGGm3HMTQ/CWEQzynVYDb8SPrQ5ushTIOKm5GpJhXVRriqxT66K6w1CHTlsQxgCaYA k34kFjRV3wbrk8osXNH1ILdN6DSBbpHdKCMjxWnECIq7dPFgOxRkgJEiIZdj55P+oYDq GmHNsT9Ej6IB6CZsm4PjsdxkSpe9FjIjj2v/Dwgf4PDUJyW9KOsRi42uvRZCSRdP63OT m44A== X-Gm-Message-State: AOAM531VMIZDcNMxiiRLG/Nt1V+s9E+pFjZj9/NAki9+0KXsAW+jCRmG AbyT1V2/jTx6ttZRFEPPmw== X-Google-Smtp-Source: ABdhPJw/E7DfMOPK/LYC3UXmbN65zv8SJaa+sW/X8ya+j/45TA/Jr0IoCQbeldKU0gIYjCHuHUx5mQ== X-Received: by 2002:a17:902:ea09:b0:13f:ac2:c5ae with SMTP id s9-20020a170902ea0900b0013f0ac2c5aemr8576150plg.3.1634867883725; Thu, 21 Oct 2021 18:58:03 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id j8sm7068876pfe.105.2021.10.21.18.57.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Oct 2021 18:58:03 -0700 (PDT) Date: Fri, 22 Oct 2021 09:57:53 +0800 From: Pingfan Liu To: Mark Rutland Cc: linux-kernel@vger.kernel.org, aou@eecs.berkeley.edu, catalin.marinas@arm.com, deanbo422@gmail.com, green.hu@gmail.com, guoren@kernel.org, jonas@southpole.se, linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk, maz@kernel.org, nickhu@andestech.com, palmer@dabbelt.com, paulmck@kernel.org, paul.walmsley@sifive.com, peterz@infradead.org, shorne@gmail.com, stefan.kristiansson@saunalahti.fi, tglx@linutronix.de, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vgupta@kernel.org, will@kernel.org Subject: Re: [PATCH 10/15] irq: arm64: perform irqentry in entry code Message-ID: References: <20211021180236.37428-1-mark.rutland@arm.com> <20211021180236.37428-11-mark.rutland@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20211021180236.37428-11-mark.rutland@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211021_185805_875452_6251CB20 X-CRM114-Status: GOOD ( 28.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Oct 21, 2021 at 07:02:31PM +0100, Mark Rutland wrote: > In preparation for removing HANDLE_DOMAIN_IRQ_IRQENTRY, have arch/arm64 > perform all the irqentry accounting in its entry code. > > As arch/arm64 already performs portions of the irqentry logic in > enter_from_kernel_mode() and exit_to_kernel_mode(), including > rcu_irq_{enter,exit}(), the only additional calls that need to be made > are to irq_{enter,exit}_rcu(). Removing the calls to > rcu_irq_{enter,exit}() from handle_domain_irq() ensures that we inform > RCU once per IRQ entry and will correctly identify quiescent periods. > > Since we should not call irq_{enter,exit}_rcu() when entering a > pseudo-NMI, el1_interrupt() is reworked to have separate __el1_irq() and > __el1_pnmi() paths for regular IRQ and psuedo-NMI entry, with > irq_{enter,exit}_irq() only called for the former. > > In preparation for removing HANDLE_DOMAIN_IRQ, the irq regs are managed > in do_interrupt_handler() for both regular IRQ and pseudo-NMI. This is > currently redundant, but not harmful. > > For clarity the preemption logic is moved into __el1_irq(). We should > never preempt within a pseudo-NMI, and arm64_enter_nmi() already > enforces this by incrementing the preempt_count, but it's clearer if we > never invoke the preemption logic when entering a pseudo-NMI. > > Signed-off-by: Mark Rutland > Cc: Catalin Marinas > Cc: Marc Zyngier > Cc: Thomas Gleixner > Cc: Will Deacon > --- > arch/arm64/Kconfig | 1 - > arch/arm64/kernel/entry-common.c | 52 ++++++++++++++++++++++++---------------- > 2 files changed, 31 insertions(+), 22 deletions(-) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 553239a5a5f7..5c7ae4c3954b 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -134,7 +134,6 @@ config ARM64 > select GENERIC_GETTIMEOFDAY > select GENERIC_VDSO_TIME_NS > select HANDLE_DOMAIN_IRQ > - select HANDLE_DOMAIN_IRQ_IRQENTRY > select HARDIRQS_SW_RESEND > select HAVE_MOVE_PMD > select HAVE_MOVE_PUD > diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c > index 32f9796c4ffe..f7408edf8571 100644 > --- a/arch/arm64/kernel/entry-common.c > +++ b/arch/arm64/kernel/entry-common.c > @@ -17,6 +17,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -219,22 +220,6 @@ static void noinstr arm64_exit_el1_dbg(struct pt_regs *regs) > lockdep_hardirqs_on(CALLER_ADDR0); > } > > -static void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs) > -{ > - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) > - arm64_enter_nmi(regs); > - else > - enter_from_kernel_mode(regs); > -} > - > -static void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs) > -{ > - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) > - arm64_exit_nmi(regs); > - else > - exit_to_kernel_mode(regs); > -} > - > static void __sched arm64_preempt_schedule_irq(void) > { > lockdep_assert_irqs_disabled(); > @@ -263,10 +248,14 @@ static void __sched arm64_preempt_schedule_irq(void) > static void do_interrupt_handler(struct pt_regs *regs, > void (*handler)(struct pt_regs *)) > { > + struct pt_regs *old_regs = set_irq_regs(regs); > + > if (on_thread_stack()) > call_on_irq_stack(regs, handler); > else > handler(regs); > + > + set_irq_regs(old_regs); > } > > extern void (*handle_arch_irq)(struct pt_regs *); > @@ -432,13 +421,22 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) > } > } > > -static void noinstr el1_interrupt(struct pt_regs *regs, > - void (*handler)(struct pt_regs *)) > +static __always_inline void __el1_pnmi(struct pt_regs *regs, > + void (*handler)(struct pt_regs *)) > { > - write_sysreg(DAIF_PROCCTX_NOIRQ, daif); > + arm64_enter_nmi(regs); > + do_interrupt_handler(regs, handler); > + arm64_exit_nmi(regs); > +} > + > +static __always_inline void __el1_irq(struct pt_regs *regs, > + void (*handler)(struct pt_regs *)) > +{ > + enter_from_kernel_mode(regs); > > - enter_el1_irq_or_nmi(regs); > + irq_enter_rcu(); > do_interrupt_handler(regs, handler); > + irq_exit_rcu(); > > /* > * Note: thread_info::preempt_count includes both thread_info::count > @@ -449,7 +447,17 @@ static void noinstr el1_interrupt(struct pt_regs *regs, > READ_ONCE(current_thread_info()->preempt_count) == 0) > arm64_preempt_schedule_irq(); > > - exit_el1_irq_or_nmi(regs); > + exit_to_kernel_mode(regs); > +} > +static void noinstr el1_interrupt(struct pt_regs *regs, > + void (*handler)(struct pt_regs *)) > +{ > + write_sysreg(DAIF_PROCCTX_NOIRQ, daif); > + > + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) > + __el1_pnmi(regs, handler); > + else > + __el1_irq(regs, handler); > } > > asmlinkage void noinstr el1h_64_irq_handler(struct pt_regs *regs) > @@ -667,7 +675,9 @@ static void noinstr el0_interrupt(struct pt_regs *regs, > if (regs->pc & BIT(55)) > arm64_apply_bp_hardening(); > > + irq_enter_rcu(); > do_interrupt_handler(regs, handler); > + irq_exit_rcu(); > > exit_to_user_mode(regs); > } Reviewed-by: Pingfan Liu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel