* Re: [PATCH] powerpc: Fix missing prototype problems for "arch/powerpc/kernel/setup_64.c"
From: Cédric Le Goater @ 2021-03-15 12:57 UTC (permalink / raw)
To: heying (H), Christophe Leroy, mpe, benh, paulus, npiggin, dja,
akpm, rppt, aneesh.kumar
Cc: johnny.chenyi, linuxppc-dev
In-Reply-To: <ddd03e30-9e54-66cd-8917-6f620557b795@huawei.com>
On 3/15/21 1:48 PM, heying (H) wrote:
>
> 在 2021/3/15 20:17, Christophe Leroy 写道:
>> You subject doesn't match the content of the patch.
> OK. I'll adapt that.
>>
>> Le 15/03/2021 à 13:04, He Ying a écrit :
>>> The variables 'uaccess_fulsh' and 'entry_flush' are not referenced
>>> outside the file. So define them as static to avoid the warnings.
>>>
>>> And add a prototype for the function 'panic_smp_self_stop' for the
>>> same purpose.
>>>
>>> Sparse also warns that 'rfi_flush' should be static. However, it's
>>> referenced outside the file.
>>
>> To clear that warning, you have to include asm/security_features.h, rfi_flush is declared there.
> Do you mean that I should include this header in arch/powerpc/kernel/setup_64.c?
yes.
>>
>>>
>>> The warnings about the file reported by sparse are as follows:
>>> arch/powerpc/kernel/setup_64.c:422:6: warning: symbol 'panic_smp_self_stop' was not declared. Should it be static?
>>> arch/powerpc/kernel/setup_64.c:951:6: warning: symbol 'rfi_flush' was not declared. Should it be static?
>>> arch/powerpc/kernel/setup_64.c:952:6: warning: symbol 'entry_flush' was not declared. Should it be static?
>>> arch/powerpc/kernel/setup_64.c:953:6: warning: symbol 'uaccess_flush' was not declared. Should it be static?
>>>
>>> Reported-by: Hulk Robot <hulkci@huawei.com>
>>> Signed-off-by: He Ying <heying24@huawei.com>
>>> ---
>>> arch/powerpc/kernel/setup_64.c | 6 ++++--
>>> 1 file changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
>>> index 560ed8b975e7..603aacd8527b 100644
>>> --- a/arch/powerpc/kernel/setup_64.c
>>> +++ b/arch/powerpc/kernel/setup_64.c
>>> @@ -71,6 +71,8 @@
>>> #include "setup.h"
>>> +extern void panic_smp_self_stop(void);
>>> +
>>
>> For function prototypes 'extern' is unneeded and deprecated.
>>
>> And function prototypes should go in an header file.
>>
>> panic_smp_self_stop() is called from kernel/panic.c , it should be declared in one of the generic linux header files I think.
> Yes, you're right. But I have no idea which header it should be declared in. May I have your suggestions?
arch/powerpc/include/asm/bug.h looks like a good place.
C.
>>
>>> int spinning_secondaries;
>>> u64 ppc64_pft_size;
>>> @@ -949,8 +951,8 @@ static bool no_rfi_flush;
>>> static bool no_entry_flush;
>>> static bool no_uaccess_flush;
>>> bool rfi_flush;
>>> -bool entry_flush;
>>> -bool uaccess_flush;
>>> +static bool entry_flush;
>>> +static bool uaccess_flush;
>>> DEFINE_STATIC_KEY_FALSE(uaccess_flush_key);
>>> EXPORT_SYMBOL(uaccess_flush_key);
>>>
>> .
^ permalink raw reply
* Re: [PATCH] powerpc: Fix missing prototype problems for "arch/powerpc/kernel/setup_64.c"
From: Christophe Leroy @ 2021-03-15 12:57 UTC (permalink / raw)
To: heying (H), mpe, benh, paulus, npiggin, dja, akpm, rppt,
aneesh.kumar, clg
Cc: johnny.chenyi, linuxppc-dev
In-Reply-To: <ddd03e30-9e54-66cd-8917-6f620557b795@huawei.com>
Le 15/03/2021 à 13:48, heying (H) a écrit :
>
> 在 2021/3/15 20:17, Christophe Leroy 写道:
>> You subject doesn't match the content of the patch.
> OK. I'll adapt that.
>>
>> Le 15/03/2021 à 13:04, He Ying a écrit :
>>> The variables 'uaccess_fulsh' and 'entry_flush' are not referenced
>>> outside the file. So define them as static to avoid the warnings.
>>>
>>> And add a prototype for the function 'panic_smp_self_stop' for the
>>> same purpose.
>>>
>>> Sparse also warns that 'rfi_flush' should be static. However, it's
>>> referenced outside the file.
>>
>> To clear that warning, you have to include asm/security_features.h, rfi_flush is declared there.
> Do you mean that I should include this header in arch/powerpc/kernel/setup_64.c?
Yes
>>
>>>
>>> The warnings about the file reported by sparse are as follows:
>>> arch/powerpc/kernel/setup_64.c:422:6: warning: symbol 'panic_smp_self_stop' was not declared.
>>> Should it be static?
>>> arch/powerpc/kernel/setup_64.c:951:6: warning: symbol 'rfi_flush' was not declared. Should it be
>>> static?
>>> arch/powerpc/kernel/setup_64.c:952:6: warning: symbol 'entry_flush' was not declared. Should it
>>> be static?
>>> arch/powerpc/kernel/setup_64.c:953:6: warning: symbol 'uaccess_flush' was not declared. Should it
>>> be static?
>>>
>>> Reported-by: Hulk Robot <hulkci@huawei.com>
>>> Signed-off-by: He Ying <heying24@huawei.com>
>>> ---
>>> arch/powerpc/kernel/setup_64.c | 6 ++++--
>>> 1 file changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
>>> index 560ed8b975e7..603aacd8527b 100644
>>> --- a/arch/powerpc/kernel/setup_64.c
>>> +++ b/arch/powerpc/kernel/setup_64.c
>>> @@ -71,6 +71,8 @@
>>> #include "setup.h"
>>> +extern void panic_smp_self_stop(void);
>>> +
>>
>> For function prototypes 'extern' is unneeded and deprecated.
>>
>> And function prototypes should go in an header file.
>>
>> panic_smp_self_stop() is called from kernel/panic.c , it should be declared in one of the generic
>> linux header files I think.
> Yes, you're right. But I have no idea which header it should be declared in. May I have your
> suggestions?
Maybe include/linux/smp.h ?
>>
>>> int spinning_secondaries;
>>> u64 ppc64_pft_size;
>>> @@ -949,8 +951,8 @@ static bool no_rfi_flush;
>>> static bool no_entry_flush;
>>> static bool no_uaccess_flush;
>>> bool rfi_flush;
>>> -bool entry_flush;
>>> -bool uaccess_flush;
>>> +static bool entry_flush;
>>> +static bool uaccess_flush;
>>> DEFINE_STATIC_KEY_FALSE(uaccess_flush_key);
>>> EXPORT_SYMBOL(uaccess_flush_key);
>>>
>> .
^ permalink raw reply
* Re: [PATCH v6 19/22] powerpc/book3s64/hash/kuap: Enable kuap on hash
From: Aneesh Kumar K.V @ 2021-03-15 12:59 UTC (permalink / raw)
To: Christophe Leroy, linuxppc-dev, mpe; +Cc: Sandipan Das
In-Reply-To: <2898f50c-aa42-5842-9d29-7272c7da5d00@csgroup.eu>
On 3/15/21 5:36 PM, Christophe Leroy wrote:
>
>
> Le 25/11/2020 à 06:16, Aneesh Kumar K.V a écrit :
>> Reviewed-by: Sandipan Das <sandipan@linux.ibm.com>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>
> PPC_HAVE_KUAP is only selected on book3s/64 when PPC_RADIX_MMU is
> selected. Is that normal ?
>
I guess we missed fixing that with this patch. How about
modified arch/powerpc/platforms/Kconfig.cputype
@@ -103,6 +103,8 @@ config PPC_BOOK3S_64
select ARCH_SUPPORTS_NUMA_BALANCING
select IRQ_WORK
select PPC_MM_SLICES
+ select PPC_HAVE_KUEP
+ select PPC_HAVE_KUAP
config PPC_BOOK3E_64
bool "Embedded processors"
@@ -365,8 +367,6 @@ config PPC_RADIX_MMU
bool "Radix MMU Support"
depends on PPC_BOOK3S_64
select ARCH_HAS_GIGANTIC_PAGE
- select PPC_HAVE_KUEP
- select PPC_HAVE_KUAP
default y
help
Enable support for the Power ISA 3.0 Radix style MMU. Currently this
-aneesh
^ permalink raw reply
* Re: [PATCH] powerpc: Fix missing prototype problems for "arch/powerpc/kernel/setup_64.c"
From: Christophe Leroy @ 2021-03-15 13:01 UTC (permalink / raw)
To: Cédric Le Goater, heying (H), mpe, benh, paulus, npiggin,
dja, akpm, rppt, aneesh.kumar
Cc: johnny.chenyi, linuxppc-dev
In-Reply-To: <c6af9455-9a04-c93e-ac7c-3c7d6a56953a@kaod.org>
Le 15/03/2021 à 13:57, Cédric Le Goater a écrit :
> On 3/15/21 1:48 PM, heying (H) wrote:
>>
>> 在 2021/3/15 20:17, Christophe Leroy 写道:
>>> You subject doesn't match the content of the patch.
>> OK. I'll adapt that.
>>>
>>> Le 15/03/2021 à 13:04, He Ying a écrit :
>>>> The variables 'uaccess_fulsh' and 'entry_flush' are not referenced
>>>> outside the file. So define them as static to avoid the warnings.
>>>>
>>>> And add a prototype for the function 'panic_smp_self_stop' for the
>>>> same purpose.
>>>>
>>>> Sparse also warns that 'rfi_flush' should be static. However, it's
>>>> referenced outside the file.
>>>
>>> To clear that warning, you have to include asm/security_features.h, rfi_flush is declared there.
>> Do you mean that I should include this header in arch/powerpc/kernel/setup_64.c?
>
> yes.
>
>>>
>>>>
>>>> The warnings about the file reported by sparse are as follows:
>>>> arch/powerpc/kernel/setup_64.c:422:6: warning: symbol 'panic_smp_self_stop' was not declared. Should it be static?
>>>> arch/powerpc/kernel/setup_64.c:951:6: warning: symbol 'rfi_flush' was not declared. Should it be static?
>>>> arch/powerpc/kernel/setup_64.c:952:6: warning: symbol 'entry_flush' was not declared. Should it be static?
>>>> arch/powerpc/kernel/setup_64.c:953:6: warning: symbol 'uaccess_flush' was not declared. Should it be static?
>>>>
>>>> Reported-by: Hulk Robot <hulkci@huawei.com>
>>>> Signed-off-by: He Ying <heying24@huawei.com>
>>>> ---
>>>> arch/powerpc/kernel/setup_64.c | 6 ++++--
>>>> 1 file changed, 4 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
>>>> index 560ed8b975e7..603aacd8527b 100644
>>>> --- a/arch/powerpc/kernel/setup_64.c
>>>> +++ b/arch/powerpc/kernel/setup_64.c
>>>> @@ -71,6 +71,8 @@
>>>> #include "setup.h"
>>>> +extern void panic_smp_self_stop(void);
>>>> +
>>>
>>> For function prototypes 'extern' is unneeded and deprecated.
>>>
>>> And function prototypes should go in an header file.
>>>
>>> panic_smp_self_stop() is called from kernel/panic.c , it should be declared in one of the generic linux header files I think.
>> Yes, you're right. But I have no idea which header it should be declared in. May I have your suggestions?
>
> arch/powerpc/include/asm/bug.h looks like a good place.
Why declaring it in a powerpc header ?
It's a weak function defined in core part of kernel (kernel/panic.c).
I think it should go in a common header, just like for instance arch_thaw_secondary_cpus_begin()
>
> C.
>
>>>
>>>> int spinning_secondaries;
>>>> u64 ppc64_pft_size;
>>>> @@ -949,8 +951,8 @@ static bool no_rfi_flush;
>>>> static bool no_entry_flush;
>>>> static bool no_uaccess_flush;
>>>> bool rfi_flush;
>>>> -bool entry_flush;
>>>> -bool uaccess_flush;
>>>> +static bool entry_flush;
>>>> +static bool uaccess_flush;
>>>> DEFINE_STATIC_KEY_FALSE(uaccess_flush_key);
>>>> EXPORT_SYMBOL(uaccess_flush_key);
>>>>
>>> .
^ permalink raw reply
* Re: [PATCH v6 19/22] powerpc/book3s64/hash/kuap: Enable kuap on hash
From: Christophe Leroy @ 2021-03-15 13:02 UTC (permalink / raw)
To: Aneesh Kumar K.V, linuxppc-dev, mpe; +Cc: Sandipan Das
In-Reply-To: <cc0338af-da92-f6d5-c0fb-aca67a2cccc5@linux.ibm.com>
Le 15/03/2021 à 13:59, Aneesh Kumar K.V a écrit :
> On 3/15/21 5:36 PM, Christophe Leroy wrote:
>>
>>
>> Le 25/11/2020 à 06:16, Aneesh Kumar K.V a écrit :
>>> Reviewed-by: Sandipan Das <sandipan@linux.ibm.com>
>>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
>>
>> PPC_HAVE_KUAP is only selected on book3s/64 when PPC_RADIX_MMU is selected. Is that normal ?
>>
>
>
> I guess we missed fixing that with this patch. How about
Yes, looks good to me.
>
> modified arch/powerpc/platforms/Kconfig.cputype
> @@ -103,6 +103,8 @@ config PPC_BOOK3S_64
> select ARCH_SUPPORTS_NUMA_BALANCING
> select IRQ_WORK
> select PPC_MM_SLICES
> + select PPC_HAVE_KUEP
> + select PPC_HAVE_KUAP
>
> config PPC_BOOK3E_64
> bool "Embedded processors"
> @@ -365,8 +367,6 @@ config PPC_RADIX_MMU
> bool "Radix MMU Support"
> depends on PPC_BOOK3S_64
> select ARCH_HAS_GIGANTIC_PAGE
> - select PPC_HAVE_KUEP
> - select PPC_HAVE_KUAP
> default y
> help
> Enable support for the Power ISA 3.0 Radix style MMU. Currently this
>
>
>
>
>
> -aneesh
^ permalink raw reply
* Re: Build regressions/improvements in v5.12-rc3
From: Christophe Leroy @ 2021-03-15 13:07 UTC (permalink / raw)
To: Geert Uytterhoeven; +Cc: linuxppc-dev, Linux Kernel Mailing List
In-Reply-To: <CAMuHMdUMjN9TW-ggAgOtj3V36kzNCfoG5o-Bcj=Lk9diJciS=g@mail.gmail.com>
Le 15/03/2021 à 12:02, Geert Uytterhoeven a écrit :
> Hi Christophe,
>
> On Mon, Mar 15, 2021 at 11:55 AM Christophe Leroy
> <christophe.leroy@csgroup.eu> wrote:
>> Le 15/03/2021 à 11:49, Geert Uytterhoeven a écrit :
>>> On Mon, Mar 15, 2021 at 11:46 AM Geert Uytterhoeven
>>> <geert@linux-m68k.org> wrote:
>>>> JFYI, when comparing v5.12-rc3[1] to v5.12-rc2[3], the summaries are:
>>>> - build errors: +2/-2
>>>
>>>> 2 error regressions:
>>>> + /kisskb/src/include/linux/compiler_types.h: error: call to '__compiletime_assert_248' declared with attribute error: BUILD_BUG failed: => 320:38
>>>> + /kisskb/src/include/linux/compiler_types.h: error: call to '__compiletime_assert_249' declared with attribute error: BUILD_BUG failed: => 320:38
>>>
>>> powerpc-gcc4.9/ppc64_book3e_allmodconfig
>>>
>>> So we traded implicit declaration errors:
>>>
>>> - /kisskb/src/drivers/gpu/drm/amd/amdgpu/../display/dc/calcs/dcn_calcs.c:
>>> error: implicit declaration of function 'disable_kernel_vsx'
>>> [-Werror=implicit-function-declaration]: 674:2 =>
>>> - /kisskb/src/drivers/gpu/drm/amd/amdgpu/../display/dc/calcs/dcn_calcs.c:
>>> error: implicit declaration of function 'enable_kernel_vsx'
>>> [-Werror=implicit-function-declaration]: 638:2 =>
>>>
>>> for compile-time assertions.
>>>
>>
>> You are missing https://github.com/linuxppc/linux/commit/eed5fae00593ab9d261a0c1ffc1bdb786a87a55a
>
> Which is not part of v5.12-rc3.
>
Yes, one step at a time :)
When I did the first fix, I tested it with GCC 10 and it worked.
It is only afterwards that I saw the problem in kisskb with gcc 4.9, hence the second fix.
Christophe
^ permalink raw reply
* Re: [PATCH 03/10] powerpc/64e/interrupt: use new interrupt return
From: Christophe Leroy @ 2021-03-15 13:30 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Scott Wood
In-Reply-To: <20210315031716.3940350-4-npiggin@gmail.com>
Le 15/03/2021 à 04:17, Nicholas Piggin a écrit :
> Update the new C and asm interrupt return code to account for 64e
> specifics, switch over to use it.
>
> The now-unused old ret_from_except code, that was moved to 64e after the
> 64s conversion, is removed.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
> arch/powerpc/include/asm/asm-prototypes.h | 2 -
> arch/powerpc/kernel/entry_64.S | 9 +-
> arch/powerpc/kernel/exceptions-64e.S | 321 ++--------------------
> arch/powerpc/kernel/interrupt.c | 27 +-
> arch/powerpc/kernel/irq.c | 76 -----
> 5 files changed, 56 insertions(+), 379 deletions(-)
>
> diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
> index fbabb49888d3..ae7b058b2970 100644
> --- a/arch/powerpc/kernel/interrupt.c
> +++ b/arch/powerpc/kernel/interrupt.c
> @@ -235,6 +235,10 @@ static notrace void booke_load_dbcr0(void)
> #endif
> }
>
> +/* temporary hack for context tracking, removed in later patch */
> +#include <linux/sched/debug.h>
> +asmlinkage __visible void __sched schedule_user(void);
> +
> /*
> * This should be called after a syscall returns, with r3 the return value
> * from the syscall. If this function returns non-zero, the system call
> @@ -292,7 +296,11 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
> while (unlikely(ti_flags & (_TIF_USER_WORK_MASK & ~_TIF_RESTORE_TM))) {
> local_irq_enable();
> if (ti_flags & _TIF_NEED_RESCHED) {
> +#ifdef CONFIG_PPC_BOOK3E_64
> + schedule_user();
> +#else
> schedule();
> +#endif
> } else {
> /*
> * SIGPENDING must restore signal handler function
> @@ -360,7 +368,6 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
> return ret;
> }
>
> -#ifndef CONFIG_PPC_BOOK3E_64 /* BOOK3E not yet using this */
> notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned long msr)
> {
> unsigned long ti_flags;
> @@ -372,7 +379,9 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned
> BUG_ON(!(regs->msr & MSR_PR));
> BUG_ON(!FULL_REGS(regs));
> BUG_ON(arch_irq_disabled_regs(regs));
> +#ifdef CONFIG_PPC_BOOK3S_64
Shouldn't this go away in patch 6 as well ?
Or is that needed at all ? In syscall_exit_prepare() it is not ifdefed .
> CT_WARN_ON(ct_state() == CONTEXT_USER);
> +#endif
>
> /*
> * We don't need to restore AMR on the way back to userspace for KUAP.
^ permalink raw reply
* Re: [PATCH] powerpc: Force inlining of cpu_has_feature() to avoid build failure
From: Michael Ellerman @ 2021-03-15 13:31 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Michael Ellerman, Paul Mackerras,
Christophe Leroy
Cc: linuxppc-dev, linux-kernel
In-Reply-To: <b231dfa040ce4cc37f702f5c3a595fdeabfe0462.1615378209.git.christophe.leroy@csgroup.eu>
On Wed, 10 Mar 2021 12:10:34 +0000 (UTC), Christophe Leroy wrote:
> The code relies on constant folding of cpu_has_feature() based
> on possible and always true values as defined per
> CPU_FTRS_ALWAYS and CPU_FTRS_POSSIBLE.
>
> Build failure is encountered with for instance
> book3e_all_defconfig on kisskb in the AMDGPU driver which uses
> cpu_has_feature(CPU_FTR_VSX_COMP) to decide whether calling
> kernel_enable_vsx() or not.
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc: Force inlining of cpu_has_feature() to avoid build failure
https://git.kernel.org/powerpc/c/eed5fae00593ab9d261a0c1ffc1bdb786a87a55a
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/vdso32: Add missing _restgpr_31_x to fix build failure
From: Michael Ellerman @ 2021-03-15 13:31 UTC (permalink / raw)
To: Paul Mackerras, Benjamin Herrenschmidt, Michael Ellerman, segher,
Christophe Leroy
Cc: linuxppc-dev, linux-kernel
In-Reply-To: <a7aa198a88bcd33c6e35e99f70f86c7b7f2f9440.1615270757.git.christophe.leroy@csgroup.eu>
On Tue, 9 Mar 2021 06:19:30 +0000 (UTC), Christophe Leroy wrote:
> With some defconfig including CONFIG_CC_OPTIMIZE_FOR_SIZE,
> (for instance mvme5100_defconfig and ps3_defconfig), gcc 5
> generates a call to _restgpr_31_x.
>
> Until recently it went unnoticed, but
> commit 42ed6d56ade2 ("powerpc/vdso: Block R_PPC_REL24 relocations")
> made it rise to the surface.
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/vdso32: Add missing _restgpr_31_x to fix build failure
https://git.kernel.org/powerpc/c/08c18b63d9656e0389087d1956d2b37fd7019172
cheers
^ permalink raw reply
* Re: [PATCH v3 28/32] powerpc/64s: interrupt implement exit logic in C
From: Christophe Leroy @ 2021-03-15 13:41 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Michal Suchanek
In-Reply-To: <20200225173541.1549955-29-npiggin@gmail.com>
Le 25/02/2020 à 18:35, Nicholas Piggin a écrit :
> Implement the bulk of interrupt return logic in C. The asm return code
> must handle a few cases: restoring full GPRs, and emulating stack store.
>
> The stack store emulation is significantly simplfied, rather than creating
> a new return frame and switching to that before performing the store, it
> uses the PACA to keep a scratch register around to perform thestore.
>
> The asm return code is moved into 64e for now. The new logic has made
> allowance for 64e, but I don't have a full environment that works well
> to test it, and even booting in emulated qemu is not great for stress
> testing. 64e shouldn't be too far off working with this, given a bit
> more testing and auditing of the logic.
>
> This is slightly faster on a POWER9 (page fault speed increases about
> 1.1%), probably due to reduced mtmsrd.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> Signed-off-by: Michal Suchanek <msuchanek@suse.de>
> ---
...
> +notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned long msr)
> +{
...
> +
> +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
> + local_paca->tm_scratch = regs->msr;
> +#endif
Could we define a helper for that in asm/tm.h, that voids when CONFIG_PPC_TRANSACTIONAL_MEM is not
selected ?
> +
> + kuap_check_amr();
> +
> + account_cpu_user_exit();
> +
> + return ret;
> +}
> +
> +void unrecoverable_exception(struct pt_regs *regs);
> +void preempt_schedule_irq(void);
> +
> +notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsigned long msr)
> +{
> +
...
> +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
> + local_paca->tm_scratch = regs->msr;
> +#endif
> +
> + /*
> + * We don't need to restore AMR on the way back to userspace for KUAP.
> + * The value of AMR only matters while we're in the kernel.
> + */
> + kuap_restore_amr(regs);
> +
> + if (unlikely(*ti_flagsp & _TIF_EMULATE_STACK_STORE)) {
> + clear_bits(_TIF_EMULATE_STACK_STORE, ti_flagsp);
> + return 1;
> + }
> + return 0;
> +}
> +#endif
Christophe
^ permalink raw reply
* Re: [PATCH] powerpc: Fix missing prototype problems for "arch/powerpc/kernel/setup_64.c"
From: Cédric Le Goater @ 2021-03-15 13:14 UTC (permalink / raw)
To: Christophe Leroy, heying (H), mpe, benh, paulus, npiggin, dja,
akpm, rppt, aneesh.kumar
Cc: johnny.chenyi, linuxppc-dev
In-Reply-To: <fe0638cc-28be-2e5c-9e94-ab3ff7666164@csgroup.eu>
On 3/15/21 2:01 PM, Christophe Leroy wrote:
>
>
> Le 15/03/2021 à 13:57, Cédric Le Goater a écrit :
>> On 3/15/21 1:48 PM, heying (H) wrote:
>>>
>>> 在 2021/3/15 20:17, Christophe Leroy 写道:
>>>> You subject doesn't match the content of the patch.
>>> OK. I'll adapt that.
>>>>
>>>> Le 15/03/2021 à 13:04, He Ying a écrit :
>>>>> The variables 'uaccess_fulsh' and 'entry_flush' are not referenced
>>>>> outside the file. So define them as static to avoid the warnings.
>>>>>
>>>>> And add a prototype for the function 'panic_smp_self_stop' for the
>>>>> same purpose.
>>>>>
>>>>> Sparse also warns that 'rfi_flush' should be static. However, it's
>>>>> referenced outside the file.
>>>>
>>>> To clear that warning, you have to include asm/security_features.h, rfi_flush is declared there.
>>> Do you mean that I should include this header in arch/powerpc/kernel/setup_64.c?
>>
>> yes.
>>
>>>>
>>>>>
>>>>> The warnings about the file reported by sparse are as follows:
>>>>> arch/powerpc/kernel/setup_64.c:422:6: warning: symbol 'panic_smp_self_stop' was not declared. Should it be static?
>>>>> arch/powerpc/kernel/setup_64.c:951:6: warning: symbol 'rfi_flush' was not declared. Should it be static?
>>>>> arch/powerpc/kernel/setup_64.c:952:6: warning: symbol 'entry_flush' was not declared. Should it be static?
>>>>> arch/powerpc/kernel/setup_64.c:953:6: warning: symbol 'uaccess_flush' was not declared. Should it be static?
>>>>>
>>>>> Reported-by: Hulk Robot <hulkci@huawei.com>
>>>>> Signed-off-by: He Ying <heying24@huawei.com>
>>>>> ---
>>>>> arch/powerpc/kernel/setup_64.c | 6 ++++--
>>>>> 1 file changed, 4 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
>>>>> index 560ed8b975e7..603aacd8527b 100644
>>>>> --- a/arch/powerpc/kernel/setup_64.c
>>>>> +++ b/arch/powerpc/kernel/setup_64.c
>>>>> @@ -71,6 +71,8 @@
>>>>> #include "setup.h"
>>>>> +extern void panic_smp_self_stop(void);
>>>>> +
>>>>
>>>> For function prototypes 'extern' is unneeded and deprecated.
>>>>
>>>> And function prototypes should go in an header file.
>>>>
>>>> panic_smp_self_stop() is called from kernel/panic.c , it should be declared in one of the generic linux header files I think.
>>> Yes, you're right. But I have no idea which header it should be declared in. May I have your suggestions?
>>
>> arch/powerpc/include/asm/bug.h looks like a good place.
>
> Why declaring it in a powerpc header ?
>
> It's a weak function defined in core part of kernel (kernel/panic.c).
>
> I think it should go in a common header, just like for instance arch_thaw_secondary_cpus_begin()
Indeed. include/linux/smp.h is a better place for a common routine.
C.
^ permalink raw reply
* Re: [PATCH 07/10] powerpc/64e/interrupt: handle bad_page_fault in C
From: Christophe Leroy @ 2021-03-15 14:07 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Scott Wood
In-Reply-To: <20210315031716.3940350-8-npiggin@gmail.com>
Le 15/03/2021 à 04:17, Nicholas Piggin a écrit :
> With non-volatile registers saved on interrupt, bad_page_fault
> can now be called by do_page_fault.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
> arch/powerpc/kernel/exceptions-64e.S | 6 ------
> arch/powerpc/mm/fault.c | 5 +----
> 2 files changed, 1 insertion(+), 10 deletions(-)
>
> diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
> index a059ab3542c2..b08c84e0fa56 100644
> --- a/arch/powerpc/kernel/exceptions-64e.S
> +++ b/arch/powerpc/kernel/exceptions-64e.S
> @@ -937,12 +937,6 @@ storage_fault_common:
> ld r14,PACA_EXGEN+EX_R14(r13)
> ld r15,PACA_EXGEN+EX_R15(r13)
> bl do_page_fault
> - cmpdi r3,0
> - bne- 1f
> - b interrupt_return
> - mr r4,r3
> - addi r3,r1,STACK_FRAME_OVERHEAD
> - bl __bad_page_fault
Then __bad_page_fault() can be static now.
> b interrupt_return
>
> /*
> diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
> index 2e54bac99a22..44833660b21d 100644
> --- a/arch/powerpc/mm/fault.c
> +++ b/arch/powerpc/mm/fault.c
> @@ -552,12 +552,9 @@ static long __do_page_fault(struct pt_regs *regs)
> if (likely(entry)) {
> instruction_pointer_set(regs, extable_fixup(entry));
> return 0;
> - } else if (!IS_ENABLED(CONFIG_PPC_BOOK3E_64)) {
> + } else {
> __bad_page_fault(regs, err);
> return 0;
> - } else {
> - /* 32 and 64e handle the bad page fault in asm */
> - return err;
> }
> }
> NOKPROBE_SYMBOL(__do_page_fault);
>
^ permalink raw reply
* [PATCH] powerpc/mm: Remove unneeded #ifdef CONFIG_PPC_MEM_KEYS
From: Christophe Leroy @ 2021-03-15 14:52 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman
Cc: linuxppc-dev, linux-kernel
In fault.c, #ifdef CONFIG_PPC_MEM_KEYS is not needed because all
functions are always defined, and arch_vma_access_permitted()
always returns true when CONFIG_PPC_MEM_KEYS is not defined so
access_pkey_error() will return false so bad_access_pkey()
will never be called.
Include linux/pkeys.h to get a definition of vma_pkeys() for
bad_access_pkey().
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
arch/powerpc/mm/fault.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index bb368257b55c..f1b3f5922f90 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -32,6 +32,7 @@
#include <linux/context_tracking.h>
#include <linux/hugetlb.h>
#include <linux/uaccess.h>
+#include <linux/pkeys.h>
#include <asm/firmware.h>
#include <asm/interrupt.h>
@@ -87,7 +88,6 @@ static noinline int bad_area(struct pt_regs *regs, unsigned long address)
return __bad_area(regs, address, SEGV_MAPERR);
}
-#ifdef CONFIG_PPC_MEM_KEYS
static noinline int bad_access_pkey(struct pt_regs *regs, unsigned long address,
struct vm_area_struct *vma)
{
@@ -127,7 +127,6 @@ static noinline int bad_access_pkey(struct pt_regs *regs, unsigned long address,
return 0;
}
-#endif
static noinline int bad_access(struct pt_regs *regs, unsigned long address)
{
@@ -234,7 +233,6 @@ static bool bad_kernel_fault(struct pt_regs *regs, unsigned long error_code,
return false;
}
-#ifdef CONFIG_PPC_MEM_KEYS
static bool access_pkey_error(bool is_write, bool is_exec, bool is_pkey,
struct vm_area_struct *vma)
{
@@ -248,7 +246,6 @@ static bool access_pkey_error(bool is_write, bool is_exec, bool is_pkey,
return false;
}
-#endif
static bool access_error(bool is_write, bool is_exec, struct vm_area_struct *vma)
{
@@ -492,11 +489,9 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address,
return bad_area(regs, address);
}
-#ifdef CONFIG_PPC_MEM_KEYS
if (unlikely(access_pkey_error(is_write, is_exec,
(error_code & DSISR_KEYFAULT), vma)))
return bad_access_pkey(regs, address, vma);
-#endif /* CONFIG_PPC_MEM_KEYS */
if (unlikely(access_error(is_write, is_exec, vma)))
return bad_access(regs, address);
--
2.25.0
^ permalink raw reply related
* Re: [PATCH] powerpc/numa: Fix topology_physical_package_id() on pSeries
From: Daniel Henrique Barboza @ 2021-03-15 15:12 UTC (permalink / raw)
To: Cédric Le Goater, linuxppc-dev
Cc: Nathan Lynch, Srikar Dronamraju, Greg Kurz, Vasant Hegde,
David Gibson
In-Reply-To: <20210312143154.3181109-1-clg@kaod.org>
On 3/12/21 11:31 AM, Cédric Le Goater wrote:
> Initial commit 15863ff3b8da ("powerpc: Make chip-id information
> available to userspace") introduce a cpu_to_chip_id() routine for the
> PowerNV platform using the "ibm,chip-id" property to query the chip id
> of a CPU. But PAPR does not specify such a property and the node id
> query is broken.
>
> Use cpu_to_node() instead which guarantees to have a correct value on
> all platforms, PowerNV an pSeries.
It is worth mentioning that that this patch will change how
topology_physical_package_id() represents in a QEMU guest. Right now, ibm,chip-id
in QEMU is matching the socket-id. After this patch, topology_physical_package_id()
will now match the NUMA id of the CPU.
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>
> Cc: Nathan Lynch <nathanl@linux.ibm.com>
> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
> Cc: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
> Signed-off-by: Cédric Le Goater <clg@kaod.org>
> ---
> arch/powerpc/include/asm/topology.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
> index 3beeb030cd78..887c42a4e43d 100644
> --- a/arch/powerpc/include/asm/topology.h
> +++ b/arch/powerpc/include/asm/topology.h
> @@ -123,7 +123,7 @@ static inline int cpu_to_coregroup_id(int cpu)
> #ifdef CONFIG_PPC64
> #include <asm/smp.h>
>
> -#define topology_physical_package_id(cpu) (cpu_to_chip_id(cpu))
> +#define topology_physical_package_id(cpu) (cpu_to_node(cpu))
>
> #define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
> #define topology_core_cpumask(cpu) (cpu_cpu_mask(cpu))
>
^ permalink raw reply
* Re: [PATCH] powerpc/numa: Fix topology_physical_package_id() on pSeries
From: Cédric Le Goater @ 2021-03-15 16:16 UTC (permalink / raw)
To: Daniel Henrique Barboza, linuxppc-dev
Cc: Nathan Lynch, Srikar Dronamraju, Greg Kurz, Vasant Hegde,
David Gibson
In-Reply-To: <bd0c948c-64f8-cf64-6d30-b9167b6a7629@gmail.com>
On 3/15/21 4:12 PM, Daniel Henrique Barboza wrote:
>
>
> On 3/12/21 11:31 AM, Cédric Le Goater wrote:
>> Initial commit 15863ff3b8da ("powerpc: Make chip-id information
>> available to userspace") introduce a cpu_to_chip_id() routine for the
>> PowerNV platform using the "ibm,chip-id" property to query the chip id
>> of a CPU. But PAPR does not specify such a property and the node id
>> query is broken.
>>
>> Use cpu_to_node() instead which guarantees to have a correct value on
>> all platforms, PowerNV an pSeries.
>
> It is worth mentioning that that this patch will change how
> topology_physical_package_id() represents in a QEMU guest. Right now, ibm,chip-id
> in QEMU is matching the socket-id. After this patch, topology_physical_package_id()
> will now match the NUMA id of the CPU.
yes. I should have added some more background.
LPARs are impacted by the use of ibm,chip-id because the property
does not exist under PowerVM and the topology-id in sysfs is always
-1 even if NUMA nodes are defined.
Under QEMU/KVM, ibm,chip-id is badly calculated when using uncommon
SMT configuration. This leads to a bogus topology-id value being
exported in sysfs.
The use of cpu_to_node() guarantees to have a correct NUMA node id
under both environments QEMU/KVM and PowerVM.
On the PowerNV platform, the numa node id returned by cpu_to_node()
is computed from the "ibm,associativity" property of the CPU. Its
value is built from the OPAL chip id and is equivalent to ibm,chip-id.
May be I should rephrase the commit log in a v2 ?
C.
> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
> Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>
>>
>> Cc: Nathan Lynch <nathanl@linux.ibm.com>
>> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>> Cc: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
>> Signed-off-by: Cédric Le Goater <clg@kaod.org>
>> ---
>> arch/powerpc/include/asm/topology.h | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
>> index 3beeb030cd78..887c42a4e43d 100644
>> --- a/arch/powerpc/include/asm/topology.h
>> +++ b/arch/powerpc/include/asm/topology.h
>> @@ -123,7 +123,7 @@ static inline int cpu_to_coregroup_id(int cpu)
>> #ifdef CONFIG_PPC64
>> #include <asm/smp.h>
>> -#define topology_physical_package_id(cpu) (cpu_to_chip_id(cpu))
>> +#define topology_physical_package_id(cpu) (cpu_to_node(cpu))
>> #define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
>> #define topology_core_cpumask(cpu) (cpu_cpu_mask(cpu))
>>
^ permalink raw reply
* Re: [PATCH] powerpc/vdso32: Add missing _restgpr_31_x to fix build failure
From: Rasmus Villemoes @ 2021-03-15 16:23 UTC (permalink / raw)
To: Segher Boessenkool, Christophe Leroy
Cc: linuxppc-dev, Paul Mackerras, linux-kernel
In-Reply-To: <20210312022940.GO29191@gate.crashing.org>
On 12/03/2021 03.29, Segher Boessenkool wrote:
> Hi!
>
> On Tue, Mar 09, 2021 at 06:19:30AM +0000, Christophe Leroy wrote:
>> With some defconfig including CONFIG_CC_OPTIMIZE_FOR_SIZE,
>> (for instance mvme5100_defconfig and ps3_defconfig), gcc 5
>> generates a call to _restgpr_31_x.
>
>> I don't know if there is a way to tell GCC not to emit that call, because at the end we get more instructions than needed.
>
> The function is required by the ABI, you need to have it.
>
> You get *fewer* insns statically, and that is what -Os is about: reduce
> the size of the binaries.
Is there any reason to not just always build the vdso with -O2? It's one
page/one VMA either way, and the vdso is about making certain system
calls cheaper, so if unconditional -O2 could save a few cycles compared
to -Os, why not? (And if, as it seems, there's only one user within the
DSO of _restgpr_31_x, yes, the overall size of the .text segment
probably increases slightly).
Rasmus
^ permalink raw reply
* RE: [PATCH] powerpc/vdso32: Add missing _restgpr_31_x to fix build failure
From: David Laight @ 2021-03-15 16:38 UTC (permalink / raw)
To: 'Rasmus Villemoes', Segher Boessenkool, Christophe Leroy
Cc: Paul Mackerras, linuxppc-dev@lists.ozlabs.org,
linux-kernel@vger.kernel.org
In-Reply-To: <023afd0c-dc61-5891-5145-5bcdce8227be@prevas.dk>
From: Rasmus Villemoes
> Sent: 15 March 2021 16:24
>
> On 12/03/2021 03.29, Segher Boessenkool wrote:
> > Hi!
> >
> > On Tue, Mar 09, 2021 at 06:19:30AM +0000, Christophe Leroy wrote:
> >> With some defconfig including CONFIG_CC_OPTIMIZE_FOR_SIZE,
> >> (for instance mvme5100_defconfig and ps3_defconfig), gcc 5
> >> generates a call to _restgpr_31_x.
> >
> >> I don't know if there is a way to tell GCC not to emit that call, because at the end we get more
> instructions than needed.
> >
> > The function is required by the ABI, you need to have it.
> >
> > You get *fewer* insns statically, and that is what -Os is about: reduce
> > the size of the binaries.
>
> Is there any reason to not just always build the vdso with -O2? It's one
> page/one VMA either way, and the vdso is about making certain system
> calls cheaper, so if unconditional -O2 could save a few cycles compared
> to -Os, why not? (And if, as it seems, there's only one user within the
> DSO of _restgpr_31_x, yes, the overall size of the .text segment
> probably increases slightly).
Sometimes -Os generates such horrid code you really never want to use it.
A classic is on x86 where it replaces 'load register with byte constant'
with 'push byte' 'pop register'.
The code is actually smaller but the execution time is horrid.
There are also cases where -O2 actually generates smaller code.
Although you may need to disable loop unrolling (often dubious at best)
and either force or disable some function inlining.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
^ permalink raw reply
* Re: [PATCH] rpadlpar: fix potential drc_name corruption in store functions
From: Tyrel Datwyler @ 2021-03-15 16:58 UTC (permalink / raw)
To: Michael Ellerman, Michal Suchánek
Cc: bhelgaas, linux-pci, mmc, linuxppc-dev, linux-kernel
In-Reply-To: <87o8fl3z80.fsf@mpe.ellerman.id.au>
On 3/14/21 7:52 PM, Michael Ellerman wrote:
> Tyrel Datwyler <tyreld@linux.ibm.com> writes:
>> On 3/13/21 1:17 AM, Michal Suchánek wrote:
>>> On Wed, Mar 10, 2021 at 04:30:21PM -0600, Tyrel Datwyler wrote:
>>>> Both add_slot_store() and remove_slot_store() try to fix up the drc_name
>>>> copied from the store buffer by placing a NULL terminator at nbyte + 1
>>>> or in place of a '\n' if present. However, the static buffer that we
>>>> copy the drc_name data into is not zeored and can contain anything past
>>>> the n-th byte. This is problematic if a '\n' byte appears in that buffer
>>>> after nbytes and the string copied into the store buffer was not NULL
>>>> terminated to start with as the strchr() search for a '\n' byte will mark
>>>> this incorrectly as the end of the drc_name string resulting in a drc_name
>>>> string that contains garbage data after the n-th byte. The following
>>>> debugging shows an example of the drmgr utility writing "PHB 4543" to
>>>> the add_slot sysfs attribute, but add_slot_store logging a corrupted
>>>> string value.
>>>>
>>>> [135823.702864] drmgr: drmgr: -c phb -a -s PHB 4543 -d 1
>>>> [135823.702879] add_slot_store: drc_name = PHB 4543°|<82>!, rc = -19
>>>>
>>>> Fix this by NULL terminating the string when we copy it into our static
>>>> buffer by coping nbytes + 1 of data from the store buffer. The code has
>>> Why is it OK to copy nbytes + 1 and why is it expected that the buffer
>>> contains a nul after the content?
>>
>> It is my understanding that the store function buffer is allocated as a
>> zeroed-page which the kernel copies up to at most (PAGE_SIZE - 1) of user data
>> into. Anything after nbytes would therefore be zeroed.
>
> I think that's true, but it would be nice if we didn't have to rely on
> that obscure detail in order for this code to be correct & understandable.
I think its a security guarantee, but I guess barring a comment that explicitly
outlines the correctness it probably isn't obvious.
>
>>> Isn't it much saner to just nul terminate the string after copying?
>>
>> At the cost of an extra line of code, sure.
>
> Is there a reason we can't use strscpy()? That should deal with all the
> corner cases around the string copy, and then all you have to do is look
> for a newline and turn it into nul.
Fine with me. I'll spin v2 with strscpy().
-Tyrel
>
> cheers
>
^ permalink raw reply
* Re: [PATCH] powerpc/numa: Fix topology_physical_package_id() on pSeries
From: Daniel Henrique Barboza @ 2021-03-15 17:36 UTC (permalink / raw)
To: Cédric Le Goater, linuxppc-dev
Cc: Nathan Lynch, Srikar Dronamraju, Greg Kurz, Vasant Hegde,
David Gibson
In-Reply-To: <09b298ce-a566-0542-0aa8-8c4e5248bda6@kaod.org>
On 3/15/21 1:16 PM, Cédric Le Goater wrote:
> On 3/15/21 4:12 PM, Daniel Henrique Barboza wrote:
>>
>>
>> On 3/12/21 11:31 AM, Cédric Le Goater wrote:
>>> Initial commit 15863ff3b8da ("powerpc: Make chip-id information
>>> available to userspace") introduce a cpu_to_chip_id() routine for the
>>> PowerNV platform using the "ibm,chip-id" property to query the chip id
>>> of a CPU. But PAPR does not specify such a property and the node id
>>> query is broken.
>>>
>>> Use cpu_to_node() instead which guarantees to have a correct value on
>>> all platforms, PowerNV an pSeries.
>>
>> It is worth mentioning that that this patch will change how
>> topology_physical_package_id() represents in a QEMU guest. Right now, ibm,chip-id
>> in QEMU is matching the socket-id. After this patch, topology_physical_package_id()
>> will now match the NUMA id of the CPU.
>
> yes. I should have added some more background.
>
> LPARs are impacted by the use of ibm,chip-id because the property
> does not exist under PowerVM and the topology-id in sysfs is always
> -1 even if NUMA nodes are defined.
>
> Under QEMU/KVM, ibm,chip-id is badly calculated when using uncommon
> SMT configuration. This leads to a bogus topology-id value being
> exported in sysfs.
>
> The use of cpu_to_node() guarantees to have a correct NUMA node id
> under both environments QEMU/KVM and PowerVM.
>
> On the PowerNV platform, the numa node id returned by cpu_to_node()
> is computed from the "ibm,associativity" property of the CPU. Its
> value is built from the OPAL chip id and is equivalent to ibm,chip-id.
>
> May be I should rephrase the commit log in a v2 ?
It's a fine idea, given that apparently we don't have documentation explaining
these details (well, at least I didn't find any). We can reference the commit
message later on as explanation :)
Thanks,
DHB
>
> C.
>
>
>> Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>>
>>>
>>> Cc: Nathan Lynch <nathanl@linux.ibm.com>
>>> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>>> Cc: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
>>> Signed-off-by: Cédric Le Goater <clg@kaod.org>
>>> ---
>>> arch/powerpc/include/asm/topology.h | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
>>> index 3beeb030cd78..887c42a4e43d 100644
>>> --- a/arch/powerpc/include/asm/topology.h
>>> +++ b/arch/powerpc/include/asm/topology.h
>>> @@ -123,7 +123,7 @@ static inline int cpu_to_coregroup_id(int cpu)
>>> #ifdef CONFIG_PPC64
>>> #include <asm/smp.h>
>>> -#define topology_physical_package_id(cpu) (cpu_to_chip_id(cpu))
>>> +#define topology_physical_package_id(cpu) (cpu_to_node(cpu))
>>> #define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
>>> #define topology_core_cpumask(cpu) (cpu_cpu_mask(cpu))
>>>
>
^ permalink raw reply
* Re: [PATCH] powerpc/numa: Fix topology_physical_package_id() on pSeries
From: Greg Kurz @ 2021-03-15 13:08 UTC (permalink / raw)
To: Cédric Le Goater
Cc: Nathan Lynch, Srikar Dronamraju, Daniel Henrique Barboza,
Vasant Hegde, linuxppc-dev, David Gibson
In-Reply-To: <20210312143154.3181109-1-clg@kaod.org>
On Fri, 12 Mar 2021 15:31:54 +0100
Cédric Le Goater <clg@kaod.org> wrote:
> Initial commit 15863ff3b8da ("powerpc: Make chip-id information
> available to userspace") introduce a cpu_to_chip_id() routine for the
> PowerNV platform using the "ibm,chip-id" property to query the chip id
> of a CPU. But PAPR does not specify such a property and the node id
> query is broken.
>
> Use cpu_to_node() instead which guarantees to have a correct value on
> all platforms, PowerNV an pSeries.
>
> Cc: Nathan Lynch <nathanl@linux.ibm.com>
> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
> Cc: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
> Signed-off-by: Cédric Le Goater <clg@kaod.org>
> ---
Makes sense.
FWIW
Reviewed-by: Greg Kurz <groug@kaod.org>
> arch/powerpc/include/asm/topology.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h
> index 3beeb030cd78..887c42a4e43d 100644
> --- a/arch/powerpc/include/asm/topology.h
> +++ b/arch/powerpc/include/asm/topology.h
> @@ -123,7 +123,7 @@ static inline int cpu_to_coregroup_id(int cpu)
> #ifdef CONFIG_PPC64
> #include <asm/smp.h>
>
> -#define topology_physical_package_id(cpu) (cpu_to_chip_id(cpu))
> +#define topology_physical_package_id(cpu) (cpu_to_node(cpu))
>
> #define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
> #define topology_core_cpumask(cpu) (cpu_cpu_mask(cpu))
^ permalink raw reply
* [PATCH 00/14] powerpc/64: fast interrupt exits
From: Nicholas Piggin @ 2021-03-15 21:08 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
This applies to powerpc next-test (particularly Christophe's ppc32
interrupt conversion) plus the 64e interrupt conversion patches I
recently posted.
This series attempts to improve the speed of interrupts and system calls
in three major ways.
Firstly, the SRR/HSRR registers do not need to be reloaded if they were
not used or clobbered fur the duration of the interrupt. 64e does not
implement this, but it could.
Secondly, an alternate return location facility is added for soft-masked
asynchronous interrupts and then that's used to set everything up for
return without having to disable MSR RI or EE.
Thirdly, mtmsrd and mtspr are reduced by various means. This is mostly
specific to 64s.
After this series, the entire system call / interrupt handler fast path
executes no mtsprs and one mtmsrd to enable interrupts initially, and
the system call vectored path doesn't even need to do that. This gives a
decent performance benefit. On POWER9 with a powernv_defconfig without
VIRT_CPU_ACCOUNTING_NATIVE, no meltdown workarounds, gettid sc system
call goes from 481 -> 344 cycles, gettid scv 345->299 cycles, and page
fault 1225->1064 cycles.
Since RFC, this no longer breaks 64e, several techniques for reducing
MSR/SPR updates become possible or tidier with interrupt wrappers, and
security fallback flushes aren't broken, usual bug fixes.
Thanks,
Nick
Nicholas Piggin (14):
powerpc: remove interrupt exit helpers unused argument
powerpc/64s: security fallback improvement
powerpc/64s: introduce different functions to return from SRR vs HSRR
interrupts
powerpc/64s: avoid reloading (H)SRR registers if they are still valid
powerpc/64: move interrupt return asm to interrupt_64.S
powerpc/64s: save one more register in the masked interrupt handler
powerpc/64: allow alternate return locations for soft-masked
interrupts
powerpc/64: interrupt soft-enable race fix
powerpc/64: treat low kernel text as irqs soft-masked
powerpc/64: use interrupt restart table to speed up return from
interrupt
powerpc/64e: Remove PPR from pt_regs
powerpc/64s: system call avoid setting MSR[RI] until we set MSR[EE]
powerpc/64: handle MSR EE and RI in interrupt entry wrapper
powerpc/64s: use the same default PPR for user and kernel
arch/powerpc/Kconfig.debug | 5 +
arch/powerpc/include/asm/asm-prototypes.h | 4 +-
arch/powerpc/include/asm/exception-64e.h | 6 +
arch/powerpc/include/asm/exception-64s.h | 52 +-
arch/powerpc/include/asm/feature-fixups.h | 18 +
arch/powerpc/include/asm/head-64.h | 2 +-
arch/powerpc/include/asm/interrupt.h | 41 +-
arch/powerpc/include/asm/paca.h | 9 +-
arch/powerpc/include/asm/ppc_asm.h | 8 +
arch/powerpc/include/asm/processor.h | 4 +-
arch/powerpc/include/asm/ptrace.h | 65 +-
arch/powerpc/kernel/asm-offsets.c | 7 +-
arch/powerpc/kernel/entry_64.S | 516 --------------
arch/powerpc/kernel/exceptions-64e.S | 53 +-
arch/powerpc/kernel/exceptions-64s.S | 384 +++++------
arch/powerpc/kernel/fpu.S | 2 +
arch/powerpc/kernel/head_64.S | 5 +-
arch/powerpc/kernel/interrupt.c | 319 +++++----
arch/powerpc/kernel/interrupt_64.S | 738 +++++++++++++++++++++
arch/powerpc/kernel/irq.c | 81 ++-
arch/powerpc/kernel/kgdb.c | 2 +-
arch/powerpc/kernel/kprobes-ftrace.c | 2 +-
arch/powerpc/kernel/kprobes.c | 10 +-
arch/powerpc/kernel/process.c | 20 +-
arch/powerpc/kernel/rtas.c | 13 +-
arch/powerpc/kernel/signal.c | 2 +-
arch/powerpc/kernel/signal_64.c | 14 +
arch/powerpc/kernel/syscalls.c | 2 +
arch/powerpc/kernel/traps.c | 18 +-
arch/powerpc/kernel/vector.S | 6 +-
arch/powerpc/kernel/vmlinux.lds.S | 24 +
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 +
arch/powerpc/lib/Makefile | 2 +-
arch/powerpc/lib/feature-fixups.c | 241 ++++---
arch/powerpc/lib/restart_table.c | 29 +
arch/powerpc/lib/sstep.c | 5 +-
arch/powerpc/math-emu/math.c | 2 +-
arch/powerpc/platforms/powernv/opal-call.c | 3 +
arch/powerpc/sysdev/fsl_pci.c | 2 +-
39 files changed, 1631 insertions(+), 1089 deletions(-)
create mode 100644 arch/powerpc/kernel/interrupt_64.S
create mode 100644 arch/powerpc/lib/restart_table.c
--
2.23.0
^ permalink raw reply
* [PATCH v2] rpadlpar: fix potential drc_name corruption in store functions
From: Tyrel Datwyler @ 2021-03-15 21:48 UTC (permalink / raw)
To: bhelgaas; +Cc: linux-pci, linuxppc-dev, linux-kernel, Tyrel Datwyler
Both add_slot_store() and remove_slot_store() try to fix up the drc_name
copied from the store buffer by placing a NULL terminator at nbyte + 1
or in place of a '\n' if present. However, the static buffer that we
copy the drc_name data into is not zeored and can contain anything past
the n-th byte. This is problematic if a '\n' byte appears in that buffer
after nbytes and the string copied into the store buffer was not NULL
terminated to start with as the strchr() search for a '\n' byte will mark
this incorrectly as the end of the drc_name string resulting in a drc_name
string that contains garbage data after the n-th byte. The following
debugging shows an example of the drmgr utility writing "PHB 4543" to
the add_slot sysfs attribute, but add_slot_store logging a corrupted
string value.
[135823.702864] drmgr: drmgr: -c phb -a -s PHB 4543 -d 1
[135823.702879] add_slot_store: drc_name = PHB 4543°|<82>!, rc = -19
Fix this by using strscpy() instead of memcpy() to ensure the string is
NULL terminated when copied into the static drc_name buffer. Further,
since the string is now NULL terminated the code only needs to change
'\n' to '\0' when present.
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
Changes in v2:
* use strscpy instead of memcpy (suggested by mpe)
drivers/pci/hotplug/rpadlpar_sysfs.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/drivers/pci/hotplug/rpadlpar_sysfs.c b/drivers/pci/hotplug/rpadlpar_sysfs.c
index cdbfa5df3a51..dbfa0b55d31a 100644
--- a/drivers/pci/hotplug/rpadlpar_sysfs.c
+++ b/drivers/pci/hotplug/rpadlpar_sysfs.c
@@ -34,12 +34,11 @@ static ssize_t add_slot_store(struct kobject *kobj, struct kobj_attribute *attr,
if (nbytes >= MAX_DRC_NAME_LEN)
return 0;
- memcpy(drc_name, buf, nbytes);
+ strscpy(drc_name, buf, nbytes + 1);
end = strchr(drc_name, '\n');
- if (!end)
- end = &drc_name[nbytes];
- *end = '\0';
+ if (end)
+ *end = '\0';
rc = dlpar_add_slot(drc_name);
if (rc)
@@ -65,12 +64,11 @@ static ssize_t remove_slot_store(struct kobject *kobj,
if (nbytes >= MAX_DRC_NAME_LEN)
return 0;
- memcpy(drc_name, buf, nbytes);
+ strscpy(drc_name, buf, nbytes + 1);
end = strchr(drc_name, '\n');
- if (!end)
- end = &drc_name[nbytes];
- *end = '\0';
+ if (end)
+ *end = '\0';
rc = dlpar_remove_slot(drc_name);
if (rc)
--
2.27.0
^ permalink raw reply related
* [PATCH 01/14] powerpc: remove interrupt exit helpers unused argument
From: Nicholas Piggin @ 2021-03-15 22:03 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20210315220402.260594-1-npiggin@gmail.com>
The msr argument is not used, remove it.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/include/asm/asm-prototypes.h | 4 ++--
arch/powerpc/kernel/interrupt.c | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/asm-prototypes.h b/arch/powerpc/include/asm/asm-prototypes.h
index 1c7b75834e04..95492655462e 100644
--- a/arch/powerpc/include/asm/asm-prototypes.h
+++ b/arch/powerpc/include/asm/asm-prototypes.h
@@ -71,8 +71,8 @@ void __init machine_init(u64 dt_ptr);
#endif
long system_call_exception(long r3, long r4, long r5, long r6, long r7, long r8, unsigned long r0, struct pt_regs *regs);
notrace unsigned long syscall_exit_prepare(unsigned long r3, struct pt_regs *regs, long scv);
-notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned long msr);
-notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsigned long msr);
+notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs);
+notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs);
long ppc_fadvise64_64(int fd, int advice, u32 offset_high, u32 offset_low,
u32 len_high, u32 len_low);
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 96ca27ef68ae..efeeefe6ee8f 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -359,7 +359,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
return ret;
}
-notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned long msr)
+notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs)
{
unsigned long ti_flags;
unsigned long flags;
@@ -443,7 +443,7 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned
void preempt_schedule_irq(void);
-notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsigned long msr)
+notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
{
unsigned long flags;
unsigned long ret = 0;
--
2.23.0
^ permalink raw reply related
* [PATCH 00/14] powerpc/64: fast interrupt exits
From: Nicholas Piggin @ 2021-03-15 22:03 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
This applies to powerpc next-test (particularly Christophe's ppc32
interrupt conversion) plus the 64e interrupt conversion patches I
recently posted.
This series attempts to improve the speed of interrupts and system calls
in three major ways.
Firstly, the SRR/HSRR registers do not need to be reloaded if they were
not used or clobbered fur the duration of the interrupt. 64e does not
implement this, but it could.
Secondly, an alternate return location facility is added for soft-masked
asynchronous interrupts and then that's used to set everything up for
return without having to disable MSR RI or EE.
Thirdly, mtmsrd and mtspr are reduced by various means. This is mostly
specific to 64s.
After this series, the entire system call / interrupt handler fast path
executes no mtsprs and one mtmsrd to enable interrupts initially, and
the system call vectored path doesn't even need to do that. This gives a
decent performance benefit. On POWER9 with a powernv_defconfig without
VIRT_CPU_ACCOUNTING_NATIVE, no meltdown workarounds, gettid sc system
call goes from 481 -> 344 cycles, gettid scv 345->299 cycles, and page
fault 1225->1064 cycles.
Since RFC, this no longer breaks 64e, several techniques for reducing
MSR/SPR updates become possible or tidier with interrupt wrappers, and
security fallback flushes aren't broken, usual bug fixes.
Thanks,
Nick
Nicholas Piggin (14):
powerpc: remove interrupt exit helpers unused argument
powerpc/64s: security fallback improvement
powerpc/64s: introduce different functions to return from SRR vs HSRR
interrupts
powerpc/64s: avoid reloading (H)SRR registers if they are still valid
powerpc/64: move interrupt return asm to interrupt_64.S
powerpc/64s: save one more register in the masked interrupt handler
powerpc/64: allow alternate return locations for soft-masked
interrupts
powerpc/64: interrupt soft-enable race fix
powerpc/64: treat low kernel text as irqs soft-masked
powerpc/64: use interrupt restart table to speed up return from
interrupt
powerpc/64e: Remove PPR from pt_regs
powerpc/64s: system call avoid setting MSR[RI] until we set MSR[EE]
powerpc/64: handle MSR EE and RI in interrupt entry wrapper
powerpc/64s: use the same default PPR for user and kernel
arch/powerpc/Kconfig.debug | 5 +
arch/powerpc/include/asm/asm-prototypes.h | 4 +-
arch/powerpc/include/asm/exception-64e.h | 6 +
arch/powerpc/include/asm/exception-64s.h | 52 +-
arch/powerpc/include/asm/feature-fixups.h | 18 +
arch/powerpc/include/asm/head-64.h | 2 +-
arch/powerpc/include/asm/interrupt.h | 41 +-
arch/powerpc/include/asm/paca.h | 9 +-
arch/powerpc/include/asm/ppc_asm.h | 8 +
arch/powerpc/include/asm/processor.h | 4 +-
arch/powerpc/include/asm/ptrace.h | 65 +-
arch/powerpc/kernel/asm-offsets.c | 7 +-
arch/powerpc/kernel/entry_64.S | 516 --------------
arch/powerpc/kernel/exceptions-64e.S | 53 +-
arch/powerpc/kernel/exceptions-64s.S | 384 +++++------
arch/powerpc/kernel/fpu.S | 2 +
arch/powerpc/kernel/head_64.S | 5 +-
arch/powerpc/kernel/interrupt.c | 319 +++++----
arch/powerpc/kernel/interrupt_64.S | 738 +++++++++++++++++++++
arch/powerpc/kernel/irq.c | 81 ++-
arch/powerpc/kernel/kgdb.c | 2 +-
arch/powerpc/kernel/kprobes-ftrace.c | 2 +-
arch/powerpc/kernel/kprobes.c | 10 +-
arch/powerpc/kernel/process.c | 20 +-
arch/powerpc/kernel/rtas.c | 13 +-
arch/powerpc/kernel/signal.c | 2 +-
arch/powerpc/kernel/signal_64.c | 14 +
arch/powerpc/kernel/syscalls.c | 2 +
arch/powerpc/kernel/traps.c | 18 +-
arch/powerpc/kernel/vector.S | 6 +-
arch/powerpc/kernel/vmlinux.lds.S | 24 +
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 +
arch/powerpc/lib/Makefile | 2 +-
arch/powerpc/lib/feature-fixups.c | 241 ++++---
arch/powerpc/lib/restart_table.c | 29 +
arch/powerpc/lib/sstep.c | 5 +-
arch/powerpc/math-emu/math.c | 2 +-
arch/powerpc/platforms/powernv/opal-call.c | 3 +
arch/powerpc/sysdev/fsl_pci.c | 2 +-
39 files changed, 1631 insertions(+), 1089 deletions(-)
create mode 100644 arch/powerpc/kernel/interrupt_64.S
create mode 100644 arch/powerpc/lib/restart_table.c
--
2.23.0
^ permalink raw reply
* [PATCH 02/14] powerpc/64s: security fallback improvement
From: Nicholas Piggin @ 2021-03-15 22:03 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20210315220402.260594-1-npiggin@gmail.com>
The fallback sequences for L1D flushing and store forwarding barriers
requires reloading r13, and saving and reloading registers from a
special PACA save area and SPRGs.
This is painful, and has caused a few difficult bugs (recently the scv
interrupt re-entrancy). Things would get even more hairy with planned
interrupt exit optimizations that can return without disabling
interrupts.
This patch moves those fallbacks further into the kernel, to the point
where r13 is available, and some registers are available to use. This
exposes slightly more attack surface, but not a huge amount (mainly some
stack frame and more paca). Firmware to implement the stateless security
ops has been available for several years now, which does not use this
path.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/include/asm/exception-64e.h | 6 +
arch/powerpc/include/asm/exception-64s.h | 52 +++--
arch/powerpc/include/asm/feature-fixups.h | 18 ++
arch/powerpc/include/asm/paca.h | 6 +-
arch/powerpc/kernel/asm-offsets.c | 2 +-
arch/powerpc/kernel/entry_64.S | 26 ++-
arch/powerpc/kernel/exceptions-64s.S | 186 +++++------------
arch/powerpc/kernel/vmlinux.lds.S | 14 ++
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 +
arch/powerpc/lib/feature-fixups.c | 241 ++++++++++++----------
10 files changed, 276 insertions(+), 279 deletions(-)
diff --git a/arch/powerpc/include/asm/exception-64e.h b/arch/powerpc/include/asm/exception-64e.h
index 40cdcb2fb057..bc90e872484e 100644
--- a/arch/powerpc/include/asm/exception-64e.h
+++ b/arch/powerpc/include/asm/exception-64e.h
@@ -164,5 +164,11 @@ exc_##label##_book3e:
#define RFI_TO_USER \
rfi
+#define ENTER_KERNEL_SECURITY_FALLBACK
+
+#define EXIT_KERNEL_SECURITY_FALLBACK
+
+#define ENTER_GUEST_SECURITY_FALLBACK
+
#endif /* _ASM_POWERPC_EXCEPTION_64E_H */
diff --git a/arch/powerpc/include/asm/exception-64s.h b/arch/powerpc/include/asm/exception-64s.h
index c1a8aac01cf9..9f2684488922 100644
--- a/arch/powerpc/include/asm/exception-64s.h
+++ b/arch/powerpc/include/asm/exception-64s.h
@@ -44,30 +44,21 @@
#define STF_ENTRY_BARRIER_SLOT \
STF_ENTRY_BARRIER_FIXUP_SECTION; \
- nop; \
- nop; \
nop
#define STF_EXIT_BARRIER_SLOT \
STF_EXIT_BARRIER_FIXUP_SECTION; \
- nop; \
- nop; \
- nop; \
- nop; \
- nop; \
nop
#define ENTRY_FLUSH_SLOT \
ENTRY_FLUSH_FIXUP_SECTION; \
nop; \
- nop; \
- nop;
+ nop
#define SCV_ENTRY_FLUSH_SLOT \
SCV_ENTRY_FLUSH_FIXUP_SECTION; \
nop; \
- nop; \
- nop;
+ nop
/*
* r10 must be free to use, r13 must be paca
@@ -100,7 +91,6 @@
#define RFI_FLUSH_SLOT \
RFI_FLUSH_FIXUP_SECTION; \
nop; \
- nop; \
nop
#define RFI_TO_KERNEL \
@@ -109,20 +99,17 @@
#define RFI_TO_USER \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- rfid; \
- b rfi_flush_fallback
+ rfid
#define RFI_TO_USER_OR_KERNEL \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- rfid; \
- b rfi_flush_fallback
+ rfid
#define RFI_TO_GUEST \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- rfid; \
- b rfi_flush_fallback
+ rfid
#define HRFI_TO_KERNEL \
hrfid
@@ -130,35 +117,44 @@
#define HRFI_TO_USER \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- hrfid; \
- b hrfi_flush_fallback
+ hrfid
#define HRFI_TO_USER_OR_KERNEL \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- hrfid; \
- b hrfi_flush_fallback
+ hrfid
#define HRFI_TO_GUEST \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- hrfid; \
- b hrfi_flush_fallback
+ hrfid
#define HRFI_TO_UNKNOWN \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- hrfid; \
- b hrfi_flush_fallback
+ hrfid
#define RFSCV_TO_USER \
STF_EXIT_BARRIER_SLOT; \
RFI_FLUSH_SLOT; \
- RFSCV; \
- b rfscv_flush_fallback
+ RFSCV
+
+#define ENTER_KERNEL_SECURITY_FALLBACK \
+ ENTER_SECURITY_FALLBACK_SECTION; \
+ nop
+
+#define EXIT_KERNEL_SECURITY_FALLBACK \
+ EXIT_SECURITY_FALLBACK_SECTION; \
+ nop
+
+#define ENTER_GUEST_SECURITY_FALLBACK \
+ EXIT_SECURITY_FALLBACK_SECTION; \
+ nop
#else /* __ASSEMBLY__ */
/* Prototype for function defined in exceptions-64s.S */
+void exit_security_fallback(void);
+void enter_security_fallback(void);
void do_uaccess_flush(void);
#endif /* __ASSEMBLY__ */
diff --git a/arch/powerpc/include/asm/feature-fixups.h b/arch/powerpc/include/asm/feature-fixups.h
index ac605fc369c4..4b5fab33688a 100644
--- a/arch/powerpc/include/asm/feature-fixups.h
+++ b/arch/powerpc/include/asm/feature-fixups.h
@@ -256,6 +256,22 @@ label##3: \
FTR_ENTRY_OFFSET 951b-952b; \
.popsection;
+#define ENTER_SECURITY_FALLBACK_SECTION \
+958: \
+ .pushsection __enter_security_fallback_fixup,"a"; \
+ .align 2; \
+959: \
+ FTR_ENTRY_OFFSET 958b-959b; \
+ .popsection;
+
+#define EXIT_SECURITY_FALLBACK_SECTION \
+960: \
+ .pushsection __exit_security_fallback_fixup,"a"; \
+ .align 2; \
+961: \
+ FTR_ENTRY_OFFSET 960b-961b; \
+ .popsection;
+
#define NOSPEC_BARRIER_FIXUP_SECTION \
953: \
.pushsection __barrier_nospec_fixup,"a"; \
@@ -288,6 +304,8 @@ extern long __start___uaccess_flush_fixup, __stop___uaccess_flush_fixup;
extern long __start___entry_flush_fixup, __stop___entry_flush_fixup;
extern long __start___scv_entry_flush_fixup, __stop___scv_entry_flush_fixup;
extern long __start___rfi_flush_fixup, __stop___rfi_flush_fixup;
+extern long __start___enter_security_fallback_fixup, __stop___enter_security_fallback_fixup;
+extern long __start___exit_security_fallback_fixup, __stop___exit_security_fallback_fixup;
extern long __start___barrier_nospec_fixup, __stop___barrier_nospec_fixup;
extern long __start__btb_flush_fixup, __stop__btb_flush_fixup;
diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h
index ec18ac818e3a..819db8afd425 100644
--- a/arch/powerpc/include/asm/paca.h
+++ b/arch/powerpc/include/asm/paca.h
@@ -249,11 +249,7 @@ struct paca_struct {
#endif
#endif
#ifdef CONFIG_PPC_BOOK3S_64
- /*
- * rfi fallback flush must be in its own cacheline to prevent
- * other paca data leaking into the L1d
- */
- u64 exrfi[EX_SIZE] __aligned(0x80);
+ u64 stf_fallback_scratch[2];
void *rfi_flush_fallback_area;
u64 l1d_flush_size;
#endif
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 85ba2b0bc8d8..e33f04280f77 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -275,7 +275,7 @@ int main(void)
OFFSET(PACA_IN_MCE, paca_struct, in_mce);
OFFSET(PACA_IN_NMI, paca_struct, in_nmi);
OFFSET(PACA_RFI_FLUSH_FALLBACK_AREA, paca_struct, rfi_flush_fallback_area);
- OFFSET(PACA_EXRFI, paca_struct, exrfi);
+ OFFSET(PACA_STF_FALLBACK_SCRATCH, paca_struct, stf_fallback_scratch);
OFFSET(PACA_L1D_FLUSH_SIZE, paca_struct, l1d_flush_size);
#endif
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 03727308d8cc..3632d8c56e48 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -116,6 +116,8 @@ BEGIN_FTR_SECTION
HMT_MEDIUM
END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
+ ENTER_KERNEL_SECURITY_FALLBACK
+
/*
* scv enters with MSR[EE]=1 and is immediately considered soft-masked.
* The entry vector already sets PACAIRQSOFTMASK to IRQS_ALL_DISABLED,
@@ -134,6 +136,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
li r5,1 /* scv */
bl syscall_exit_prepare
+ EXIT_KERNEL_SECURITY_FALLBACK
+
ld r2,_CCR(r1)
ld r4,_NIP(r1)
ld r5,_MSR(r1)
@@ -296,6 +300,8 @@ END_BTB_FLUSH_SECTION
stb r11,PACAIRQSOFTMASK(r13)
stb r12,PACAIRQHAPPENED(r13)
+ ENTER_KERNEL_SECURITY_FALLBACK
+
/* Calling convention has r9 = orig r0, r10 = regs */
mr r9,r0
bl system_call_exception
@@ -305,6 +311,8 @@ END_BTB_FLUSH_SECTION
li r5,0 /* !scv */
bl syscall_exit_prepare
+ EXIT_KERNEL_SECURITY_FALLBACK
+
ld r2,_CCR(r1)
ld r4,_NIP(r1)
ld r5,_MSR(r1)
@@ -642,11 +650,16 @@ _ASM_NOKPROBE_SYMBOL(fast_interrupt_return)
ld r5,_MSR(r1)
andi. r0,r5,MSR_PR
#ifdef CONFIG_PPC_BOOK3S
- bne .Lfast_user_interrupt_return_amr
- kuap_kernel_restore r3, r4
+ beq 1f
+ kuap_user_restore r3, r4
+ b .Lfast_user_interrupt_return
+1:
andi. r0,r5,MSR_RI
+ beq- 2f
+ kuap_kernel_restore r3, r4
li r3,0 /* 0 return value, no EMULATE_STACK_STORE */
- bne+ .Lfast_kernel_interrupt_return
+ b .Lfast_kernel_interrupt_return
+2:
addi r3,r1,STACK_FRAME_OVERHEAD
bl unrecoverable_exception
b . /* should not get here */
@@ -666,12 +679,9 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return)
bl interrupt_exit_user_prepare
cmpdi r3,0
bne- .Lrestore_nvgprs
-
-#ifdef CONFIG_PPC_BOOK3S
-.Lfast_user_interrupt_return_amr:
- kuap_user_restore r3, r4
-#endif
.Lfast_user_interrupt_return:
+ EXIT_KERNEL_SECURITY_FALLBACK
+
ld r11,_NIP(r1)
ld r12,_MSR(r1)
BEGIN_FTR_SECTION
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 0cdb59e8b577..0127032bc2aa 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -644,6 +644,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
ld r11,exception_marker@toc(r2)
std r10,RESULT(r1) /* clear regs->result */
std r11,STACK_FRAME_OVERHEAD-16(r1) /* mark the frame */
+
+ ENTER_KERNEL_SECURITY_FALLBACK
.endm
/*
@@ -996,6 +998,8 @@ EXC_COMMON_BEGIN(system_reset_common)
subi r10,r10,1
sth r10,PACA_IN_NMI(r13)
+ EXIT_KERNEL_SECURITY_FALLBACK
+
kuap_kernel_restore r9, r10
EXCEPTION_RESTORE_REGS
RFI_TO_USER_OR_KERNEL
@@ -2199,6 +2203,8 @@ EXC_COMMON_BEGIN(hmi_exception_early_common)
cmpdi cr0,r3,0
bne 1f
+ EXIT_KERNEL_SECURITY_FALLBACK
+
EXCEPTION_RESTORE_REGS hsrr=1
HRFI_TO_USER_OR_KERNEL
@@ -2843,26 +2849,13 @@ masked_interrupt:
b .
.endm
-TRAMP_REAL_BEGIN(stf_barrier_fallback)
- std r9,PACA_EXRFI+EX_R9(r13)
- std r10,PACA_EXRFI+EX_R10(r13)
- sync
- ld r9,PACA_EXRFI+EX_R9(r13)
- ld r10,PACA_EXRFI+EX_R10(r13)
- ori 31,31,0
- .rept 14
- b 1f
-1:
- .endr
- blr
-
-/* Clobbers r10, r11, ctr */
+/* Clobbers r11, r12, ctr */
.macro L1D_DISPLACEMENT_FLUSH
- ld r10,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
- ld r11,PACA_L1D_FLUSH_SIZE(r13)
- srdi r11,r11,(7 + 3) /* 128 byte lines, unrolled 8x */
- mtctr r11
- DCBT_BOOK3S_STOP_ALL_STREAM_IDS(r11) /* Stop prefetch streams */
+ ld r11,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
+ ld r12,PACA_L1D_FLUSH_SIZE(r13)
+ srdi r12,r12,(7 + 3) /* 128 byte lines, unrolled 8x */
+ mtctr r12
+ DCBT_BOOK3S_STOP_ALL_STREAM_IDS(r12) /* Stop prefetch streams */
/* order ld/st prior to dcbt stop all streams with flushing */
sync
@@ -2873,125 +2866,31 @@ TRAMP_REAL_BEGIN(stf_barrier_fallback)
* hurt).
*/
1:
- ld r11,(0x80 + 8)*0(r10)
- ld r11,(0x80 + 8)*1(r10)
- ld r11,(0x80 + 8)*2(r10)
- ld r11,(0x80 + 8)*3(r10)
- ld r11,(0x80 + 8)*4(r10)
- ld r11,(0x80 + 8)*5(r10)
- ld r11,(0x80 + 8)*6(r10)
- ld r11,(0x80 + 8)*7(r10)
- addi r10,r10,0x80*8
+ ld r12,(0x80 + 8)*0(r11)
+ ld r12,(0x80 + 8)*1(r11)
+ ld r12,(0x80 + 8)*2(r11)
+ ld r12,(0x80 + 8)*3(r11)
+ ld r12,(0x80 + 8)*4(r11)
+ ld r12,(0x80 + 8)*5(r11)
+ ld r12,(0x80 + 8)*6(r11)
+ ld r12,(0x80 + 8)*7(r11)
+ addi r11,r11,0x80*8
bdnz 1b
.endm
-TRAMP_REAL_BEGIN(entry_flush_fallback)
- std r9,PACA_EXRFI+EX_R9(r13)
- std r10,PACA_EXRFI+EX_R10(r13)
- std r11,PACA_EXRFI+EX_R11(r13)
- mfctr r9
- L1D_DISPLACEMENT_FLUSH
- mtctr r9
- ld r9,PACA_EXRFI+EX_R9(r13)
- ld r10,PACA_EXRFI+EX_R10(r13)
- ld r11,PACA_EXRFI+EX_R11(r13)
- blr
-
-/*
- * The SCV entry flush happens with interrupts enabled, so it must disable
- * to prevent EXRFI being clobbered by NMIs (e.g., soft_nmi_common). r10
- * (containing LR) does not need to be preserved here because scv entry
- * puts 0 in the pt_regs, CTR can be clobbered for the same reason.
- */
-TRAMP_REAL_BEGIN(scv_entry_flush_fallback)
- li r10,0
- mtmsrd r10,1
- lbz r10,PACAIRQHAPPENED(r13)
- ori r10,r10,PACA_IRQ_HARD_DIS
- stb r10,PACAIRQHAPPENED(r13)
- std r11,PACA_EXRFI+EX_R11(r13)
- L1D_DISPLACEMENT_FLUSH
- ld r11,PACA_EXRFI+EX_R11(r13)
- li r10,MSR_RI
- mtmsrd r10,1
- blr
-
-TRAMP_REAL_BEGIN(rfi_flush_fallback)
- SET_SCRATCH0(r13);
- GET_PACA(r13);
- std r1,PACA_EXRFI+EX_R12(r13)
- ld r1,PACAKSAVE(r13)
- std r9,PACA_EXRFI+EX_R9(r13)
- std r10,PACA_EXRFI+EX_R10(r13)
- std r11,PACA_EXRFI+EX_R11(r13)
- mfctr r9
- L1D_DISPLACEMENT_FLUSH
- mtctr r9
- ld r9,PACA_EXRFI+EX_R9(r13)
- ld r10,PACA_EXRFI+EX_R10(r13)
- ld r11,PACA_EXRFI+EX_R11(r13)
- ld r1,PACA_EXRFI+EX_R12(r13)
- GET_SCRATCH0(r13);
- rfid
-
-TRAMP_REAL_BEGIN(hrfi_flush_fallback)
- SET_SCRATCH0(r13);
- GET_PACA(r13);
- std r1,PACA_EXRFI+EX_R12(r13)
- ld r1,PACAKSAVE(r13)
- std r9,PACA_EXRFI+EX_R9(r13)
- std r10,PACA_EXRFI+EX_R10(r13)
- std r11,PACA_EXRFI+EX_R11(r13)
- mfctr r9
- L1D_DISPLACEMENT_FLUSH
- mtctr r9
- ld r9,PACA_EXRFI+EX_R9(r13)
- ld r10,PACA_EXRFI+EX_R10(r13)
- ld r11,PACA_EXRFI+EX_R11(r13)
- ld r1,PACA_EXRFI+EX_R12(r13)
- GET_SCRATCH0(r13);
- hrfid
-
-TRAMP_REAL_BEGIN(rfscv_flush_fallback)
- /* system call volatile */
- mr r7,r13
- GET_PACA(r13);
- mr r8,r1
- ld r1,PACAKSAVE(r13)
- mfctr r9
- ld r10,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
- ld r11,PACA_L1D_FLUSH_SIZE(r13)
- srdi r11,r11,(7 + 3) /* 128 byte lines, unrolled 8x */
- mtctr r11
- DCBT_BOOK3S_STOP_ALL_STREAM_IDS(r11) /* Stop prefetch streams */
-
- /* order ld/st prior to dcbt stop all streams with flushing */
+/* Clobbers r11, r12 */
+.macro STF_FALLBACK_BARRIER
+ std r11,PACA_STF_FALLBACK_SCRATCH+0(r13)
+ std r12,PACA_STF_FALLBACK_SCRATCH+8(r13)
sync
-
- /*
- * The load adresses are at staggered offsets within cachelines,
- * which suits some pipelines better (on others it should not
- * hurt).
- */
+ ld r11,PACA_STF_FALLBACK_SCRATCH+0(r13)
+ ld r12,PACA_STF_FALLBACK_SCRATCH+8(r13)
+ ori 31,31,0
+ .rept 14
+ b 1f
1:
- ld r11,(0x80 + 8)*0(r10)
- ld r11,(0x80 + 8)*1(r10)
- ld r11,(0x80 + 8)*2(r10)
- ld r11,(0x80 + 8)*3(r10)
- ld r11,(0x80 + 8)*4(r10)
- ld r11,(0x80 + 8)*5(r10)
- ld r11,(0x80 + 8)*6(r10)
- ld r11,(0x80 + 8)*7(r10)
- addi r10,r10,0x80*8
- bdnz 1b
-
- mtctr r9
- li r9,0
- li r10,0
- li r11,0
- mr r1,r8
- mr r13,r7
- RFSCV
+ .endr
+.endm
USE_TEXT_SECTION()
@@ -3006,6 +2905,27 @@ _GLOBAL(do_uaccess_flush)
_ASM_NOKPROBE_SYMBOL(do_uaccess_flush)
EXPORT_SYMBOL(do_uaccess_flush)
+_GLOBAL(enter_security_fallback)
+ STF_FALLBACK_BARRIER
+ ld r11,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
+ cmpdi r11,0
+ beq 10f
+ L1D_DISPLACEMENT_FLUSH
+10:
+ ld r12,_MSR(r1) // some interrupts require r12==SRR1
+ andi. r11,r12,MSR_PR // and cr0 set
+ blr
+_ASM_NOKPROBE_SYMBOL(enter_security_fallback)
+
+_GLOBAL(exit_security_fallback)
+ STF_FALLBACK_BARRIER
+ ld r11,PACA_RFI_FLUSH_FALLBACK_AREA(r13)
+ cmpdi r11,0
+ beq 10f
+ L1D_DISPLACEMENT_FLUSH
+10:
+ blr
+_ASM_NOKPROBE_SYMBOL(exit_security_fallback)
MASKED_INTERRUPT
MASKED_INTERRUPT hsrr=1
diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S
index 72fa3c00229a..582009dacef4 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -165,6 +165,20 @@ SECTIONS
*(__rfi_flush_fixup)
__stop___rfi_flush_fixup = .;
}
+
+ . = ALIGN(8);
+ __enter_security_fallback_fixup : AT(ADDR(__enter_security_fallback_fixup) - LOAD_OFFSET) {
+ __start___enter_security_fallback_fixup = .;
+ *(__enter_security_fallback_fixup)
+ __stop___enter_security_fallback_fixup = .;
+ }
+
+ . = ALIGN(8);
+ __exit_security_fallback_fixup : AT(ADDR(__exit_security_fallback_fixup) - LOAD_OFFSET) {
+ __start___exit_security_fallback_fixup = .;
+ *(__exit_security_fallback_fixup)
+ __stop___exit_security_fallback_fixup = .;
+ }
#endif /* CONFIG_PPC64 */
#ifdef CONFIG_PPC_BARRIER_NOSPEC
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 5e634db4809b..e5adfa090c6a 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -1066,6 +1066,8 @@ fast_guest_return:
bl kvmhv_accumulate_time
#endif
+ ENTER_GUEST_SECURITY_FALLBACK
+
/* Enter guest */
BEGIN_FTR_SECTION
@@ -1348,6 +1350,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
li r0, MSR_RI
mtmsrd r0, 1
+ ENTER_KERNEL_SECURITY_FALLBACK
+
#ifdef CONFIG_KVM_BOOK3S_HV_EXIT_TIMING
addi r3, r9, VCPU_TB_RMINTR
mr r4, r9
diff --git a/arch/powerpc/lib/feature-fixups.c b/arch/powerpc/lib/feature-fixups.c
index 1fd31b4b0e13..370e98dc64db 100644
--- a/arch/powerpc/lib/feature-fixups.c
+++ b/arch/powerpc/lib/feature-fixups.c
@@ -118,115 +118,169 @@ void do_feature_fixups(unsigned long value, void *fixup_start, void *fixup_end)
}
#ifdef CONFIG_PPC_BOOK3S_64
-static void do_stf_entry_barrier_fixups(enum stf_barrier_type types)
+static void do_enter_security_fallback_fixups(bool enable)
{
- unsigned int instrs[3], *dest;
+ unsigned int instr, *dest;
long *start, *end;
int i;
- start = PTRRELOC(&__start___stf_entry_barrier_fixup);
- end = PTRRELOC(&__stop___stf_entry_barrier_fixup);
+ start = PTRRELOC(&__start___enter_security_fallback_fixup);
+ end = PTRRELOC(&__stop___enter_security_fallback_fixup);
- instrs[0] = 0x60000000; /* nop */
- instrs[1] = 0x60000000; /* nop */
- instrs[2] = 0x60000000; /* nop */
+ instr = 0x60000000; /* nop */
- i = 0;
- if (types & STF_BARRIER_FALLBACK) {
- instrs[i++] = 0x7d4802a6; /* mflr r10 */
- instrs[i++] = 0x60000000; /* branch patched below */
- instrs[i++] = 0x7d4803a6; /* mtlr r10 */
- } else if (types & STF_BARRIER_EIEIO) {
- instrs[i++] = 0x7e0006ac; /* eieio + bit 6 hint */
- } else if (types & STF_BARRIER_SYNC_ORI) {
- instrs[i++] = 0x7c0004ac; /* hwsync */
- instrs[i++] = 0xe94d0000; /* ld r10,0(r13) */
- instrs[i++] = 0x63ff0000; /* ori 31,31,0 speculation barrier */
+ for (i = 0; start < end; start++, i++) {
+ dest = (void *)start + *start;
+
+ pr_devel("patching dest %lx\n", (unsigned long)dest);
+
+ if (enable) {
+ patch_branch((struct ppc_inst *)dest,
+ (unsigned long)&enter_security_fallback,
+ BRANCH_SET_LINK);
+ } else {
+ patch_instruction((struct ppc_inst *)dest,
+ ppc_inst(instr));
+ }
}
+ printk(KERN_DEBUG "enter-security-fallback: patched %d locations (%s)\n", i,
+ enable ? "enable" : "disable");
+}
+
+static void do_exit_security_fallback_fixups(bool enable)
+{
+ unsigned int instr, *dest;
+ long *start, *end;
+ int i;
+
+ start = PTRRELOC(&__start___exit_security_fallback_fixup);
+ end = PTRRELOC(&__stop___exit_security_fallback_fixup);
+
+ instr = 0x60000000; /* nop */
+
for (i = 0; start < end; start++, i++) {
dest = (void *)start + *start;
pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
-
- if (types & STF_BARRIER_FALLBACK)
- patch_branch((struct ppc_inst *)(dest + 1),
- (unsigned long)&stf_barrier_fallback,
+ if (enable) {
+ patch_branch((struct ppc_inst *)dest,
+ (unsigned long)&exit_security_fallback,
BRANCH_SET_LINK);
- else
- patch_instruction((struct ppc_inst *)(dest + 1),
- ppc_inst(instrs[1]));
+ } else {
+ patch_instruction((struct ppc_inst *)dest,
+ ppc_inst(instr));
+ }
+ }
- patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
+ printk(KERN_DEBUG "exit-security-fallback: patched %d locations (%s)\n", i,
+ enable ? "enable" : "disable");
+}
+
+static enum stf_barrier_type enter_stf_barrier_type = STF_BARRIER_NONE;
+static enum stf_barrier_type exit_stf_barrier_type = STF_BARRIER_NONE;
+static enum l1d_flush_type enter_flush_type = L1D_FLUSH_NONE;
+static enum l1d_flush_type exit_flush_type = L1D_FLUSH_NONE;
+
+static void update_fallback_calls(void)
+{
+ static bool enter_fallback_enabled = false;
+ static bool exit_fallback_enabled = false;
+
+ // This is slightly racy if called concurrently.
+
+ if (enter_stf_barrier_type == STF_BARRIER_FALLBACK ||
+ enter_flush_type == L1D_FLUSH_FALLBACK) {
+ if (!enter_fallback_enabled) {
+ do_enter_security_fallback_fixups(true);
+ enter_fallback_enabled = true;
+ }
+ } else if (enter_stf_barrier_type == STF_BARRIER_NONE &&
+ enter_flush_type == L1D_FLUSH_NONE) {
+ if (enter_fallback_enabled) {
+ do_enter_security_fallback_fixups(false);
+ enter_fallback_enabled = false;
+ }
+ }
+
+ if (exit_stf_barrier_type == STF_BARRIER_FALLBACK ||
+ exit_flush_type == L1D_FLUSH_FALLBACK) {
+ if (!exit_fallback_enabled) {
+ do_exit_security_fallback_fixups(true);
+ exit_fallback_enabled = true;
+ }
+ } else if (exit_stf_barrier_type == STF_BARRIER_NONE &&
+ exit_flush_type == L1D_FLUSH_NONE) {
+ if (exit_fallback_enabled) {
+ do_exit_security_fallback_fixups(false);
+ exit_fallback_enabled = false;
+ }
+ }
+}
+
+static void do_stf_entry_barrier_fixups(enum stf_barrier_type types)
+{
+ unsigned int instr, *dest;
+ long *start, *end;
+ int i;
+
+ start = PTRRELOC(&__start___stf_entry_barrier_fixup);
+ end = PTRRELOC(&__stop___stf_entry_barrier_fixup);
+
+ instr = 0x60000000; /* nop */
+ if (types & STF_BARRIER_EIEIO)
+ instr = 0x7e0006ac; /* eieio + bit 6 hint */
+
+ for (i = 0; start < end; start++, i++) {
+ dest = (void *)start + *start;
+
+ pr_devel("patching dest %lx\n", (unsigned long)dest);
+
+ patch_instruction((struct ppc_inst *)dest, ppc_inst(instr));
}
printk(KERN_DEBUG "stf-barrier: patched %d entry locations (%s barrier)\n", i,
(types == STF_BARRIER_NONE) ? "no" :
(types == STF_BARRIER_FALLBACK) ? "fallback" :
(types == STF_BARRIER_EIEIO) ? "eieio" :
- (types == (STF_BARRIER_SYNC_ORI)) ? "hwsync"
- : "unknown");
+ (types == STF_BARRIER_SYNC_ORI) ? "hwsync"
+ : "unknown");
+
+ enter_stf_barrier_type = types;
+ update_fallback_calls();
}
static void do_stf_exit_barrier_fixups(enum stf_barrier_type types)
{
- unsigned int instrs[6], *dest;
+ unsigned int instr, *dest;
long *start, *end;
int i;
start = PTRRELOC(&__start___stf_exit_barrier_fixup);
end = PTRRELOC(&__stop___stf_exit_barrier_fixup);
- instrs[0] = 0x60000000; /* nop */
- instrs[1] = 0x60000000; /* nop */
- instrs[2] = 0x60000000; /* nop */
- instrs[3] = 0x60000000; /* nop */
- instrs[4] = 0x60000000; /* nop */
- instrs[5] = 0x60000000; /* nop */
-
- i = 0;
- if (types & STF_BARRIER_FALLBACK || types & STF_BARRIER_SYNC_ORI) {
- if (cpu_has_feature(CPU_FTR_HVMODE)) {
- instrs[i++] = 0x7db14ba6; /* mtspr 0x131, r13 (HSPRG1) */
- instrs[i++] = 0x7db04aa6; /* mfspr r13, 0x130 (HSPRG0) */
- } else {
- instrs[i++] = 0x7db243a6; /* mtsprg 2,r13 */
- instrs[i++] = 0x7db142a6; /* mfsprg r13,1 */
- }
- instrs[i++] = 0x7c0004ac; /* hwsync */
- instrs[i++] = 0xe9ad0000; /* ld r13,0(r13) */
- instrs[i++] = 0x63ff0000; /* ori 31,31,0 speculation barrier */
- if (cpu_has_feature(CPU_FTR_HVMODE)) {
- instrs[i++] = 0x7db14aa6; /* mfspr r13, 0x131 (HSPRG1) */
- } else {
- instrs[i++] = 0x7db242a6; /* mfsprg r13,2 */
- }
- } else if (types & STF_BARRIER_EIEIO) {
- instrs[i++] = 0x7e0006ac; /* eieio + bit 6 hint */
- }
+ instr = 0x60000000; /* nop */
+ if (types == STF_BARRIER_EIEIO)
+ instr = 0x7e0006ac; /* eieio + bit 6 hint */
for (i = 0; start < end; start++, i++) {
dest = (void *)start + *start;
pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
- patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
- patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
- patch_instruction((struct ppc_inst *)(dest + 3), ppc_inst(instrs[3]));
- patch_instruction((struct ppc_inst *)(dest + 4), ppc_inst(instrs[4]));
- patch_instruction((struct ppc_inst *)(dest + 5), ppc_inst(instrs[5]));
+ patch_instruction((struct ppc_inst *)dest, ppc_inst(instr));
}
printk(KERN_DEBUG "stf-barrier: patched %d exit locations (%s barrier)\n", i,
(types == STF_BARRIER_NONE) ? "no" :
(types == STF_BARRIER_FALLBACK) ? "fallback" :
(types == STF_BARRIER_EIEIO) ? "eieio" :
- (types == (STF_BARRIER_SYNC_ORI)) ? "hwsync"
- : "unknown");
-}
+ (types == STF_BARRIER_SYNC_ORI) ? "hwsync"
+ : "unknown");
+ exit_stf_barrier_type = types;
+ update_fallback_calls();
+}
void do_stf_barrier_fixups(enum stf_barrier_type types)
{
@@ -286,28 +340,20 @@ void do_uaccess_flush_fixups(enum l1d_flush_type types)
void do_entry_flush_fixups(enum l1d_flush_type types)
{
- unsigned int instrs[3], *dest;
+ unsigned int instrs[2], *dest;
long *start, *end;
int i;
instrs[0] = 0x60000000; /* nop */
instrs[1] = 0x60000000; /* nop */
- instrs[2] = 0x60000000; /* nop */
i = 0;
- if (types == L1D_FLUSH_FALLBACK) {
- instrs[i++] = 0x7d4802a6; /* mflr r10 */
- instrs[i++] = 0x60000000; /* branch patched below */
- instrs[i++] = 0x7d4803a6; /* mtlr r10 */
- }
-
if (types & L1D_FLUSH_ORI) {
instrs[i++] = 0x63ff0000; /* ori 31,31,0 speculation barrier */
instrs[i++] = 0x63de0000; /* ori 30,30,0 L1d flush*/
- }
-
- if (types & L1D_FLUSH_MTTRIG)
+ } else if (types & L1D_FLUSH_MTTRIG) {
instrs[i++] = 0x7c12dba6; /* mtspr TRIG2,r0 (SPR #882) */
+ }
start = PTRRELOC(&__start___entry_flush_fixup);
end = PTRRELOC(&__stop___entry_flush_fixup);
@@ -316,15 +362,9 @@ void do_entry_flush_fixups(enum l1d_flush_type types)
pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
-
- if (types == L1D_FLUSH_FALLBACK)
- patch_branch((struct ppc_inst *)(dest + 1), (unsigned long)&entry_flush_fallback,
- BRANCH_SET_LINK);
- else
- patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
+ patch_instruction((struct ppc_inst *)(dest + 0), ppc_inst(instrs[0]));
- patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
+ patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
}
start = PTRRELOC(&__start___scv_entry_flush_fixup);
@@ -334,15 +374,9 @@ void do_entry_flush_fixups(enum l1d_flush_type types)
pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
-
- if (types == L1D_FLUSH_FALLBACK)
- patch_branch((struct ppc_inst *)(dest + 1), (unsigned long)&scv_entry_flush_fallback,
- BRANCH_SET_LINK);
- else
- patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
+ patch_instruction((struct ppc_inst *)(dest + 0), ppc_inst(instrs[0]));
- patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
+ patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
}
@@ -354,11 +388,14 @@ void do_entry_flush_fixups(enum l1d_flush_type types)
: "ori type" :
(types & L1D_FLUSH_MTTRIG) ? "mttrig type"
: "unknown");
+
+ enter_flush_type = types;
+ update_fallback_calls();
}
void do_rfi_flush_fixups(enum l1d_flush_type types)
{
- unsigned int instrs[3], *dest;
+ unsigned int instrs[2], *dest;
long *start, *end;
int i;
@@ -367,29 +404,22 @@ void do_rfi_flush_fixups(enum l1d_flush_type types)
instrs[0] = 0x60000000; /* nop */
instrs[1] = 0x60000000; /* nop */
- instrs[2] = 0x60000000; /* nop */
-
- if (types & L1D_FLUSH_FALLBACK)
- /* b .+16 to fallback flush */
- instrs[0] = 0x48000010;
i = 0;
if (types & L1D_FLUSH_ORI) {
instrs[i++] = 0x63ff0000; /* ori 31,31,0 speculation barrier */
instrs[i++] = 0x63de0000; /* ori 30,30,0 L1d flush*/
- }
-
- if (types & L1D_FLUSH_MTTRIG)
+ } else if (types & L1D_FLUSH_MTTRIG) {
instrs[i++] = 0x7c12dba6; /* mtspr TRIG2,r0 (SPR #882) */
+ }
for (i = 0; start < end; start++, i++) {
dest = (void *)start + *start;
pr_devel("patching dest %lx\n", (unsigned long)dest);
- patch_instruction((struct ppc_inst *)dest, ppc_inst(instrs[0]));
+ patch_instruction((struct ppc_inst *)(dest + 0), ppc_inst(instrs[0]));
patch_instruction((struct ppc_inst *)(dest + 1), ppc_inst(instrs[1]));
- patch_instruction((struct ppc_inst *)(dest + 2), ppc_inst(instrs[2]));
}
printk(KERN_DEBUG "rfi-flush: patched %d locations (%s flush)\n", i,
@@ -400,6 +430,9 @@ void do_rfi_flush_fixups(enum l1d_flush_type types)
: "ori type" :
(types & L1D_FLUSH_MTTRIG) ? "mttrig type"
: "unknown");
+
+ exit_flush_type = types;
+ update_fallback_calls();
}
void do_barrier_nospec_fixups_range(bool enable, void *fixup_start, void *fixup_end)
--
2.23.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox