From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DAD1C433E0 for ; Wed, 20 Jan 2021 13:38:17 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B889C22573 for ; Wed, 20 Jan 2021 13:38:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B889C22573 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=caramail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=b1V/LBbW+ntfYDe2MWunotYX5LPINR6p68iEBpGzWQw=; b=LuMksBjgopnvlCpEqzJvCPqkn EcLZxgPXxFFroh2c65ZdKsX9nVtk4VuFyJRgC8RW8jCju821CR4g39uxuZx+zH8NcqTEXaZnyjkp4 m1gDZINxy/x96h+aiyZFi9DkCZQGV11Bo4m1s/+DUxz22uvrrKBL/UBMA0wwPfSaS1cK2PQ3RfgLw TViCO8tT3dfV80D8UB1lGyQAXyuhqz8GsJV9KQIOHvxuw9Y32dj4/QM3fDAfTk1Dq6C7WFCR6m9iY yZk3HozCNW0nuNDgCac11vhu1Qa13pU4N3R5/DgcUL/x2O84vi9Ioj1f5uqSumKYMzWAhpwUKNCbz HGkS6szRw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2DfE-0000Aw-Rn; Wed, 20 Jan 2021 13:36:48 +0000 Received: from mout.gmx.net ([212.227.15.15]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2Df3-00006J-Kk for linux-arm-kernel@lists.infradead.org; Wed, 20 Jan 2021 13:36:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1611149795; bh=+KSPF4GnD3omtIIe3i7Lyq2tvVwNmWhc6oOtmKZfGzI=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=jiAMZk8rfbGD2voG186kD4Sj6cIDGFWY6Rp5SyAcaZwsBcc6Mc1LxuZjuBKzMAiaR ozfK797V93N8OuNZkFO2cQz0j8x+uDWalLB0QwC2AGeQDFuMRrDQns5prlhVbx2lYu IAlAWbNDtS3QMDbfM/d5GXcxrD5tDPaNOmznWhEY= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from localhost.localdomain ([83.204.192.78]) by mail.gmx.net (mrgmx004 [212.227.17.184]) with ESMTPSA (Nemesis) id 1MA7GS-1l9K0M2Kew-00BZJF; Wed, 20 Jan 2021 14:27:32 +0100 From: Mohamed Mediouni To: linux-arm-kernel@lists.infradead.org Subject: [RFC PATCH 1/7] arm64: kernel: FIQ support Date: Wed, 20 Jan 2021 14:27:11 +0100 Message-Id: <20210120132717.395873-2-mohamed.mediouni@caramail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210120132717.395873-1-mohamed.mediouni@caramail.com> References: <20210120132717.395873-1-mohamed.mediouni@caramail.com> MIME-Version: 1.0 X-Provags-ID: V03:K1:Ml5n3z6MXBeROYCSnnYCVIFLVuy9eVuLxApHlTE8E1RE4GX2lBM EvZAjrVErqyUftDvvUykHP8m3molE9GngFEsNtZgjmFmH90S1gY9mWf0b7cJw0X43NNbw9N LYhpMiKOB+BUDSWIB5XVGOmMrvbAHLpKIizsVp+S5WLVYma1rgSvka3479KcBTEYRDItj39 eYtYg1q/s4hjVPoDrZe+g== X-UI-Out-Filterresults: notjunk:1;V03:K0:b2W8Gcvbrqg=:ZJgwF7VOP8rR6XflfWIj/i aBw4+FjT8Hht4lOrQIGVSFNWbMIyt03RQh2gQ1Fn8NIsLXDaXUOoVPb+HF+PoBMA5rvVzSjI5 IbPImy70wB76Gw4YRxPnebnGGpiAQ1ybIuWQVF8fwPxtqmiJqOdAtKmpKUeXfC79PUaxZJu5R 4gN00CWmw7t6OtsJVYzSYwfsAfkYwuqrkal0/iNh9ZP0cs6mcFGTLIaB7jcw76DilNTI+l2EP fm7XWDNgidp6VyVNM8YdVdgHQobUbqDQTnp1QBTEjEkVdUmU1kYjU5NIQqfPRczQTNrvg+OTq oo/5IYub8MhdzfqKqOk7aYsHN0iGsmoTY9pYnbzgU+iwprEVNleMthMOq0fNLf00tGKXtKPVx FoI8LE5Nq6ImFa7g+9pfwFlZpI8Y7YEkTbRi6UNnAAmC23YmZyBCtaEUHfJ1dBt8DD8E4iy6H /aGQs7GtokIbN10m1b+MshXChmkeeksbZMN3P2xUXO4rnV55PKvKux85qtIE5hgUp031Cxqpk kQn+5cFpyXwGDm/4MnJbAF6kLUceeIzuo17ReNZfC7bXgXIOzbrbe2n0KRu0kazstSxS7vg7R 6w2TnduDQJHCgs6dyd98LOzNQ2Nsjlxro/TSR92vXu5WbhG6jN77/q/l50Jqq/g4gH2/k+g1n bFL890eC37nqsmZDGLXhKbNUd7VNaonKW7Ej5OIzIu5xKhc0Viz8HNqcIErX3AMF2NvX3Z3tt KxIksFnzUL0SHXRxc+IqcWIlXPAIZzIcjSnOSG44ssADAku1rqcJXuJzUJVdlEALCzbiWmcRJ YYAv3AcqsBss9feomlirwm64C6w/z8KDG6j5QhBpSFY34WbodyLK4E5uimBkDj0Sq/+rk/vbT rqhMfXMYE7Xjy55l0rbCMpB5q9RLG20Qdb3srLwwo= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210120_083638_086244_263549DE X-CRM114-Status: GOOD ( 20.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Catalin Marinas , Hector Martin , linux-kernel@vger.kernel.org, Marc Zyngier , Mohamed Mediouni , Will Deacon , Stan Skowronek Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Stan Skowronek On Apple processors, the timer is wired through FIQ. As such, add FIQ support to the kernel. Signed-off-by: Stan Skowronek Signed-off-by: Mohamed Mediouni --- arch/arm64/include/asm/arch_gicv3.h | 2 +- arch/arm64/include/asm/assembler.h | 8 ++-- arch/arm64/include/asm/daifflags.h | 4 +- arch/arm64/include/asm/irq.h | 4 ++ arch/arm64/include/asm/irqflags.h | 6 +-- arch/arm64/kernel/entry.S | 74 ++++++++++++++++++++++++++--- arch/arm64/kernel/irq.c | 14 ++++++ arch/arm64/kernel/process.c | 2 +- 8 files changed, 97 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 880b9054d75c..934b9be582d2 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -173,7 +173,7 @@ static inline void gic_pmr_mask_irqs(void) static inline void gic_arch_enable_irqs(void) { - asm volatile ("msr daifclr, #2" : : : "memory"); + asm volatile ("msr daifclr, #3" : : : "memory"); } #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index bf125c591116..6fe55713dfe0 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -40,9 +40,9 @@ msr daif, \flags .endm - /* IRQ is the lowest priority flag, unconditionally unmask the rest. */ - .macro enable_da_f - msr daifclr, #(8 | 4 | 1) + /* IRQ/FIQ is the lowest priority flag, unconditionally unmask the rest. */ + .macro enable_da + msr daifclr, #(8 | 4) .endm /* @@ -50,7 +50,7 @@ */ .macro save_and_disable_irq, flags mrs \flags, daif - msr daifset, #2 + msr daifset, #3 .endm .macro restore_irq, flags diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index 1c26d7baa67f..44de96c7fb1a 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -13,8 +13,8 @@ #include #define DAIF_PROCCTX 0 -#define DAIF_PROCCTX_NOIRQ PSR_I_BIT -#define DAIF_ERRCTX (PSR_I_BIT | PSR_A_BIT) +#define DAIF_PROCCTX_NOIRQ (PSR_I_BIT | PSR_F_BIT) +#define DAIF_ERRCTX (PSR_I_BIT | PSR_F_BIT | PSR_A_BIT) #define DAIF_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h index b2b0c6405eb0..2d1537d3a245 100644 --- a/arch/arm64/include/asm/irq.h +++ b/arch/arm64/include/asm/irq.h @@ -13,5 +13,9 @@ static inline int nr_legacy_irqs(void) return 0; } +int set_handle_fiq(void (*handle_fiq)(struct pt_regs *)); + +extern void (*handle_arch_fiq)(struct pt_regs *) __ro_after_init; + #endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index ff328e5bbb75..26d7f378113e 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -35,7 +35,7 @@ static inline void arch_local_irq_enable(void) } asm volatile(ALTERNATIVE( - "msr daifclr, #2 // arch_local_irq_enable", + "msr daifclr, #3 // arch_local_irq_enable", __msr_s(SYS_ICC_PMR_EL1, "%0"), ARM64_HAS_IRQ_PRIO_MASKING) : @@ -54,7 +54,7 @@ static inline void arch_local_irq_disable(void) } asm volatile(ALTERNATIVE( - "msr daifset, #2 // arch_local_irq_disable", + "msr daifset, #3 // arch_local_irq_disable", __msr_s(SYS_ICC_PMR_EL1, "%0"), ARM64_HAS_IRQ_PRIO_MASKING) : @@ -85,7 +85,7 @@ static inline int arch_irqs_disabled_flags(unsigned long flags) int res; asm volatile(ALTERNATIVE( - "and %w0, %w1, #" __stringify(PSR_I_BIT), + "and %w0, %w1, #" __stringify(PSR_I_BIT | PSR_F_BIT), "eor %w0, %w1, #" __stringify(GIC_PRIO_IRQON), ARM64_HAS_IRQ_PRIO_MASKING) : "=&r" (res) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index c9bae73f2621..abcca0db0736 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -499,6 +499,14 @@ tsk .req x28 // current thread_info irq_stack_exit .endm + .macro fiq_handler + ldr_l x1, handle_arch_fiq + mov x0, sp + irq_stack_entry + blr x1 + irq_stack_exit + .endm + #ifdef CONFIG_ARM64_PSEUDO_NMI /* * Set res to 0 if irqs were unmasked in interrupted context. @@ -547,18 +555,18 @@ SYM_CODE_START(vectors) kernel_ventry 1, sync // Synchronous EL1h kernel_ventry 1, irq // IRQ EL1h - kernel_ventry 1, fiq_invalid // FIQ EL1h + kernel_ventry 1, fiq // FIQ EL1h kernel_ventry 1, error // Error EL1h kernel_ventry 0, sync // Synchronous 64-bit EL0 kernel_ventry 0, irq // IRQ 64-bit EL0 - kernel_ventry 0, fiq_invalid // FIQ 64-bit EL0 + kernel_ventry 0, fiq // FIQ 64-bit EL0 kernel_ventry 0, error // Error 64-bit EL0 #ifdef CONFIG_COMPAT kernel_ventry 0, sync_compat, 32 // Synchronous 32-bit EL0 kernel_ventry 0, irq_compat, 32 // IRQ 32-bit EL0 - kernel_ventry 0, fiq_invalid_compat, 32 // FIQ 32-bit EL0 + kernel_ventry 0, fiq_compat, 32 // FIQ 32-bit EL0 kernel_ventry 0, error_compat, 32 // Error 32-bit EL0 #else kernel_ventry 0, sync_invalid, 32 // Synchronous 32-bit EL0 @@ -661,7 +669,7 @@ SYM_CODE_END(el1_sync) SYM_CODE_START_LOCAL_NOALIGN(el1_irq) kernel_entry 1 gic_prio_irq_setup pmr=x20, tmp=x1 - enable_da_f + enable_da mov x0, sp bl enter_el1_irq_or_nmi @@ -689,6 +697,38 @@ alternative_else_nop_endif kernel_exit 1 SYM_CODE_END(el1_irq) + .align 6 +SYM_CODE_START_LOCAL_NOALIGN(el1_fiq) + kernel_entry 1 + gic_prio_irq_setup pmr=x20, tmp=x1 + enable_da + + mov x0, sp + bl enter_el1_irq_or_nmi + + fiq_handler + +#ifdef CONFIG_PREEMPTION + ldr x24, [tsk, #TSK_TI_PREEMPT] // get preempt count +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + /* + * DA_F were cleared at start of handling. If anything is set in DAIF, + * we come back from an NMI, so skip preemption + */ + mrs x0, daif + orr x24, x24, x0 +alternative_else_nop_endif + cbnz x24, 1f // preempt count != 0 || NMI return path + bl arm64_preempt_schedule_irq // irq en/disable is done inside +1: +#endif + + mov x0, sp + bl exit_el1_irq_or_nmi + + kernel_exit 1 +SYM_CODE_END(el1_fiq) + /* * EL0 mode handlers. */ @@ -715,6 +755,12 @@ SYM_CODE_START_LOCAL_NOALIGN(el0_irq_compat) b el0_irq_naked SYM_CODE_END(el0_irq_compat) + .align 6 +SYM_CODE_START_LOCAL_NOALIGN(el0_fiq_compat) + kernel_entry 0, 32 + b el0_fiq_naked +SYM_CODE_END(el0_fiq_compat) + SYM_CODE_START_LOCAL_NOALIGN(el0_error_compat) kernel_entry 0, 32 b el0_error_naked @@ -727,7 +773,7 @@ SYM_CODE_START_LOCAL_NOALIGN(el0_irq) el0_irq_naked: gic_prio_irq_setup pmr=x20, tmp=x0 user_exit_irqoff - enable_da_f + enable_da tbz x22, #55, 1f bl do_el0_irq_bp_hardening @@ -737,6 +783,22 @@ el0_irq_naked: b ret_to_user SYM_CODE_END(el0_irq) + .align 6 +SYM_CODE_START_LOCAL_NOALIGN(el0_fiq) + kernel_entry 0 +el0_fiq_naked: + gic_prio_irq_setup pmr=x20, tmp=x0 + user_exit_irqoff + enable_da + + tbz x22, #55, 1f + bl do_el0_irq_bp_hardening +1: + fiq_handler + + b ret_to_user +SYM_CODE_END(el0_fiq) + SYM_CODE_START_LOCAL(el1_error) kernel_entry 1 mrs x1, esr_el1 @@ -757,7 +819,7 @@ el0_error_naked: mov x0, sp mov x1, x25 bl do_serror - enable_da_f + enable_da b ret_to_user SYM_CODE_END(el0_error) diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index dfb1feab867d..4d7a9fb41d93 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -88,3 +88,17 @@ void __init init_IRQ(void) local_daif_restore(DAIF_PROCCTX_NOIRQ); } } + +/* + * Analogous to the generic handle_arch_irq / set_handle_irq + */ +void (*handle_arch_fiq)(struct pt_regs *) __ro_after_init; + +int __init set_handle_fiq(void (*handle_fiq)(struct pt_regs *)) +{ + if (handle_arch_fiq) + return -EBUSY; + + handle_arch_fiq = handle_fiq; + return 0; +} diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 6616486a58fe..34ec400288d0 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -84,7 +84,7 @@ static void noinstr __cpu_do_idle_irqprio(void) unsigned long daif_bits; daif_bits = read_sysreg(daif); - write_sysreg(daif_bits | PSR_I_BIT, daif); + write_sysreg(daif_bits | PSR_I_BIT | PSR_F_BIT, daif); /* * Unmask PMR before going idle to make sure interrupts can -- 2.29.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel