* Re: [PATCH] Documentation/powerpc: fix malformed table in syscall64-abi
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: Michael Ellerman, LKML, linux-doc@vger.kernel.org,
Nicholas Piggin, Randy Dunlap, linuxppc-dev
In-Reply-To: <e06de4d3-a36f-2745-9775-467e125436cc@infradead.org>
On Sun, 23 Aug 2020 17:31:16 -0700, Randy Dunlap wrote:
> Fix malformed table warning in powerpc/syscall64-abi.rst by making
> two tables and moving the headings.
>
> Documentation/powerpc/syscall64-abi.rst:53: WARNING: Malformed table.
> Text in column margin in table line 2.
>
> =========== ============= ========================================
> --- For the sc instruction, differences with the ELF ABI ---
> r0 Volatile (System call number.)
> r3 Volatile (Parameter 1, and return value.)
> r4-r8 Volatile (Parameters 2-6.)
> cr0 Volatile (cr0.SO is the return error condition.)
> cr1, cr5-7 Nonvolatile
> lr Nonvolatile
>
> [...]
Applied to powerpc/fixes.
[1/1] Documentation/powerpc: fix malformed table in syscall64-abi
https://git.kernel.org/powerpc/c/aa661d7fab436d8a782618b3138da1a84ca28a31
cheers
^ permalink raw reply
* Re: [PATCH] Revert "powerpc/powernv/idle: Replace CPU feature check with PVR check"
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: pratik.r.sampat, linux-kernel, mpe, mikey, svaidy, ego, npiggin,
Pratik Rajesh Sampat, linuxppc-dev
In-Reply-To: <20200826082918.89306-1-psampat@linux.ibm.com>
On Wed, 26 Aug 2020 13:59:18 +0530, Pratik Rajesh Sampat wrote:
> Cpuidle stop state implementation has minor optimizations for P10
> where hardware preserves more SPR registers compared to P9.
> The current P9 driver works for P10, although does few extra
> save-restores. P9 driver can provide the required power management
> features like SMT thread folding and core level power savings
> on a P10 platform.
>
> [...]
Applied to powerpc/fixes.
[1/1] Revert "powerpc/powernv/idle: Replace CPU feature check with PVR check"
https://git.kernel.org/powerpc/c/16d83a540ca4e7f1ebb2b3756869b77451d31414
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/64s: scv entry should set PPR
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: linuxppc-dev, Nicholas Piggin
In-Reply-To: <20200825075309.224184-1-npiggin@gmail.com>
On Tue, 25 Aug 2020 17:53:09 +1000, Nicholas Piggin wrote:
> Kernel entry sets PPR to HMT_MEDIUM by convention. The scv entry
> path missed this.
Applied to powerpc/fixes.
[1/1] powerpc/64s: scv entry should set PPR
https://git.kernel.org/powerpc/c/e5fe56092e753c50093c60e757561984abff335e
cheers
^ permalink raw reply
* Re: [PATCH] video: fbdev: controlfb: Fix build for COMPILE_TEST=y && PPC_PMAC=n
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: Michael Ellerman, linuxppc-dev
Cc: linux-fbdev, b.zolnierkie, daniel.vetter, linux-kernel, dri-devel,
sam
In-Reply-To: <20200821104910.3363818-1-mpe@ellerman.id.au>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 633 bytes --]
On Fri, 21 Aug 2020 20:49:10 +1000, Michael Ellerman wrote:
> The build is currently broken, if COMPILE_TEST=y and PPC_PMAC=n:
>
> linux/drivers/video/fbdev/controlfb.c: In function ‘control_set_hardware’:
> linux/drivers/video/fbdev/controlfb.c:276:2: error: implicit declaration of function ‘btext_update_display’
> 276 | btext_update_display(p->frame_buffer_phys + CTRLFB_OFF,
> | ^~~~~~~~~~~~~~~~~~~~
>
> [...]
Applied to powerpc/fixes.
[1/1] video: fbdev: controlfb: Fix build for COMPILE_TEST=y && PPC_PMAC=n
https://git.kernel.org/powerpc/c/4d618b9f3fcab84e9ec28c180de46fb2c929d096
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/64s: Fix crash in load_fp_state() due to fpexc_mode
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: Michael Ellerman, linuxppc-dev; +Cc: miltonm, npiggin
In-Reply-To: <20200825093424.3967813-1-mpe@ellerman.id.au>
On Tue, 25 Aug 2020 19:34:24 +1000, Michael Ellerman wrote:
> The recent commit 01eb01877f33 ("powerpc/64s: Fix restore_math
> unnecessarily changing MSR") changed some of the handling of floating
> point/vector restore.
>
> In particular it caused current->thread.fpexc_mode to be copied into
> the current MSR (via msr_check_and_set()), rather than just into
> regs->msr (which is moved into MSR on return to userspace).
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/64s: Fix crash in load_fp_state() due to fpexc_mode
https://git.kernel.org/powerpc/c/b91eb5182405b01a8aeb42e9b5207831767e97ee
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/32s: Fix module loading failure when VMALLOC_END is over 0xf0000000
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: Paul Mackerras, Michael Ellerman, schwab, Christophe Leroy,
Benjamin Herrenschmidt
Cc: linuxppc-dev, linux-kernel
In-Reply-To: <09fc73fe9c7423c6b4cf93f93df9bb0ed8eefab5.1597994047.git.christophe.leroy@csgroup.eu>
On Fri, 21 Aug 2020 07:15:25 +0000 (UTC), Christophe Leroy wrote:
> In is_module_segment(), when VMALLOC_END is over 0xf0000000,
> ALIGN(VMALLOC_END, SZ_256M) has value 0.
>
> In that case, addr >= ALIGN(VMALLOC_END, SZ_256M) is always
> true then is_module_segment() always returns false.
>
> Use (ALIGN(VMALLOC_END, SZ_256M) - 1) which will have
> value 0xffffffff and will be suitable for the comparison.
Applied to powerpc/fixes.
[1/1] powerpc/32s: Fix module loading failure when VMALLOC_END is over 0xf0000000
https://git.kernel.org/powerpc/c/541cebb51f3422d4f2c6cb95c1e5cc3dcc9e5021
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/perf/hv-24x7: Move cpumask file to top folder of hv-24x7 driver
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: linuxppc-dev, mpe, Kajol Jain; +Cc: suka, maddy
In-Reply-To: <20200821080610.123997-1-kjain@linux.ibm.com>
On Fri, 21 Aug 2020 13:36:10 +0530, Kajol Jain wrote:
> Commit 792f73f747b8 ("powerpc/hv-24x7: Add sysfs files inside hv-24x7
> device to show cpumask") added cpumask file as part of hv-24x7 driver
> inside the interface folder. Cpumask file suppose to be in the top
> folder of the pmu driver inorder to make hotplug works.
>
> This patch fix that issue and create new group 'cpumask_attr_group'
> to add cpumask file and make sure it added on top folder.
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/perf/hv-24x7: Move cpumask file to top folder of hv-24x7 driver
https://git.kernel.org/powerpc/c/64ef8f2c4791940d7f3945507b6a45c20d959260
cheers
^ permalink raw reply
* Re: [PATCH kernel] powerpc/perf: Stop crashing with generic_compat_pmu
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: linuxppc-dev, Alexey Kardashevskiy; +Cc: Madhavan Srinivasan
In-Reply-To: <20200602025612.62707-1-aik@ozlabs.ru>
On Tue, 2 Jun 2020 12:56:12 +1000, Alexey Kardashevskiy wrote:
> The bhrb_filter_map ("The Branch History Rolling Buffer") callback is
> only defined in raw CPUs' power_pmu structs. The "architected" CPUs use
> generic_compat_pmu which does not have this callback and crashed occur.
>
> This add a NULL pointer check for bhrb_filter_map() which behaves as if
> the callback returned an error.
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/perf: Fix crashes with generic_compat_pmu & BHRB
https://git.kernel.org/powerpc/c/b460b512417ae9c8b51a3bdcc09020cd6c60ff69
cheers
^ permalink raw reply
* Re: [PATCH] powerpc/perf: Fix reading of MSR[HV PR] bits in trace-imc
From: Michael Ellerman @ 2020-08-27 7:46 UTC (permalink / raw)
To: mpe, Athira Rajeev; +Cc: maddy, linuxppc-dev
In-Reply-To: <1598424029-1662-1-git-send-email-atrajeev@linux.vnet.ibm.com>
On Wed, 26 Aug 2020 02:40:29 -0400, Athira Rajeev wrote:
> IMC trace-mode uses MSR[HV PR] bits to set the cpumode
> for the instruction pointer captured in each sample.
> The bits are fetched from third DW of the trace record.
> Reading third DW from IMC trace record should use be64_to_cpu
> along with READ_ONCE inorder to fetch correct MSR[HV PR] bits.
> Patch addresses this change.
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/perf: Fix reading of MSR[HV/PR] bits in trace-imc
https://git.kernel.org/powerpc/c/82715a0f332843d3a1830d7ebc9ac7c99a00c880
cheers
^ permalink raw reply
* Re: fsl_espi errors on v5.7.15
From: Nicholas Piggin @ 2020-08-27 7:12 UTC (permalink / raw)
To: benh@kernel.crashing.org, broonie@kernel.org, Chris Packham,
Heiner Kallweit, mpe@ellerman.id.au, paulus@samba.org
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
linux-spi@vger.kernel.org
In-Reply-To: <1020029e-4cb9-62ba-c6d6-e6b9bdf93aac@gmail.com>
Excerpts from Heiner Kallweit's message of August 26, 2020 4:38 pm:
> On 26.08.2020 08:07, Chris Packham wrote:
>>
>> On 26/08/20 1:48 pm, Chris Packham wrote:
>>>
>>> On 26/08/20 10:22 am, Chris Packham wrote:
>>>> On 25/08/20 7:22 pm, Heiner Kallweit wrote:
>>>>
>>>> <snip>
>>>>> I've been staring at spi-fsl-espi.c for while now and I think I've
>>>>>> identified a couple of deficiencies that may or may not be related
>>>>>> to my
>>>>>> issue.
>>>>>>
>>>>>> First I think the 'Transfer done but SPIE_DON isn't set' message
>>>>>> can be
>>>>>> generated spuriously. In fsl_espi_irq() we read the ESPI_SPIE
>>>>>> register.
>>>>>> We also write back to it to clear the current events. We re-read it in
>>>>>> fsl_espi_cpu_irq() and complain when SPIE_DON is not set. But we can
>>>>>> naturally end up in that situation if we're doing a large read.
>>>>>> Consider
>>>>>> the messages for reading a block of data from a spi-nor chip
>>>>>>
>>>>>> tx = READ_OP + ADDR
>>>>>> rx = data
>>>>>>
>>>>>> We setup the transfer and pump out the tx_buf. The first interrupt
>>>>>> goes
>>>>>> off and ESPI_SPIE has SPIM_DON and SPIM_RXT set. We empty the rx fifo,
>>>>>> clear ESPI_SPIE and wait for the next interrupt. The next interrupt
>>>>>> fires and this time we have ESPI_SPIE with just SPIM_RXT set. This
>>>>>> continues until we've received all the data and we finish with
>>>>>> ESPI_SPIE
>>>>>> having only SPIM_RXT set. When we re-read it we complain that SPIE_DON
>>>>>> isn't set.
>>>>>>
>>>>>> The other deficiency is that we only get an interrupt when the
>>>>>> amount of
>>>>>> data in the rx fifo is above FSL_ESPI_RXTHR. If there are fewer than
>>>>>> FSL_ESPI_RXTHR left to be received we will never pull them out of
>>>>>> the fifo.
>>>>>>
>>>>> SPIM_DON will trigger an interrupt once the last characters have been
>>>>> transferred, and read the remaining characters from the FIFO.
>>>>
>>>> The T2080RM that I have says the following about the DON bit
>>>>
>>>> "Last character was transmitted. The last character was transmitted
>>>> and a new command can be written for the next frame."
>>>>
>>>> That does at least seem to fit with my assertion that it's all about
>>>> the TX direction. But the fact that it doesn't happen all the time
>>>> throws some doubt on it.
>>>>
>>>>> I think the reason I'm seeing some variability is because of how fast
>>>>>> (or slow) the interrupts get processed and how fast the spi-nor
>>>>>> chip can
>>>>>> fill the CPUs rx fifo.
>>>>>>
>>>>> To rule out timing issues at high bus frequencies I initially asked
>>>>> for re-testing at lower frequencies. If you e.g. limit the bus to 1 MHz
>>>>> or even less, then timing shouldn't be an issue.
>>>> Yes I've currently got spi-max-frequency = <1000000>; in my dts. I
>>>> would also expect a slower frequency would fit my "DON is for TX"
>>>> narrative.
>>>>> Last relevant functional changes have been done almost 4 years ago.
>>>>> And yours is the first such report I see. So question is what could
>>>>> be so
>>>>> special with your setup that it seems you're the only one being
>>>>> affected.
>>>>> The scenarios you describe are standard, therefore much more people
>>>>> should be affected in case of a driver bug.
>>>> Agreed. But even on my hardware (which may have a latent issue
>>>> despite being in the field for going on 5 years) the issue only
>>>> triggers under some fairly specific circumstances.
>>>>> You said that kernel config impacts how frequently the issue happens.
>>>>> Therefore question is what's the diff in kernel config, and how could
>>>>> the differences be related to SPI.
>>>>
>>>> It did seem to be somewhat random. Things like CONFIG_PREEMPT have an
>>>> impact but every time I found something that seemed to be having an
>>>> impact I've been able to disprove it. I actually think its about how
>>>> busy the system is which may or may not affect when we get round to
>>>> processing the interrupts.
>>>>
>>>> I have managed to get the 'Transfer done but SPIE_DON isn't set!' to
>>>> occur on the T2080RDB.
>>>>
>>>> I've had to add the following to expose the environment as a mtd
>>>> partition
>>>>
>>>> diff --git a/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
>>>> b/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
>>>> index ff87e67c70da..fbf95fc1fd68 100644
>>>> --- a/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
>>>> +++ b/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
>>>> @@ -116,6 +116,15 @@ flash@0 {
>>>> compatible = "micron,n25q512ax3",
>>>> "jedec,spi-nor";
>>>> reg = <0>;
>>>> spi-max-frequency = <10000000>; /*
>>>> input clock */
>>>> +
>>>> + partition@u-boot {
>>>> + reg = <0x00000000 0x00100000>;
>>>> + label = "u-boot";
>>>> + };
>>>> + partition@u-boot-env {
>>>> + reg = <0x00100000 0x00010000>;
>>>> + label = "u-boot-env";
>>>> + };
>>>> };
>>>> };
>>>>
>>>> And I'm using the following script to poke at the environment
>>>> (warning if anyone does try this and the bug hits it can render your
>>>> u-boot environment invalid).
>>>>
>>>> cat flash/fw_env_test.sh
>>>> #!/bin/sh
>>>>
>>>> generate_fw_env_config()
>>>> {
>>>> cat /proc/mtd | sed 's/[:"]//g' | while read dev size erasesize
>>>> name ; do
>>>> echo "$dev $size $erasesize $name"
>>>> [ "$name" = "u-boot-env" ] && echo "/dev/$dev 0x0000 0x2000
>>>> $erasesize" >/flash/fw_env.config
>>>> done
>>>> }
>>>>
>>>> cycles=10
>>>> [ $# -ge 1 ] && cycles=$1
>>>>
>>>> generate_fw_env_config
>>>>
>>>> fw_printenv -c /flash/fw_env.config
>>>>
>>>> dmesg -c >/dev/null
>>>> x=0
>>>> while [ $x -lt $cycles ]; do
>>>> fw_printenv -c /flash/fw_env.config >/dev/null || break
>>>> fw_setenv -c /flash/fw_env.config foo $RANDOM || break;
>>>> dmesg -c | grep -q fsl_espi && break;
>>>> let x=x+1
>>>> done
>>>>
>>>> echo "Ran $x cycles"
>>>
>>> I've also now seen the RX FIFO not empty error on the T2080RDB
>>>
>>> fsl_espi ffe110000.spi: Transfer done but SPIE_DON isn't set!
>>> fsl_espi ffe110000.spi: Transfer done but SPIE_DON isn't set!
>>> fsl_espi ffe110000.spi: Transfer done but SPIE_DON isn't set!
>>> fsl_espi ffe110000.spi: Transfer done but SPIE_DON isn't set!
>>> fsl_espi ffe110000.spi: Transfer done but rx/tx fifo's aren't empty!
>>> fsl_espi ffe110000.spi: SPIE_RXCNT = 1, SPIE_TXCNT = 32
>>>
>>> With my current workaround of emptying the RX FIFO. It seems
>>> survivable. Interestingly it only ever seems to be 1 extra byte in the
>>> RX FIFO and it seems to be after either a READ_SR or a READ_FSR.
>>>
>>> fsl_espi ffe110000.spi: tx 70
>>> fsl_espi ffe110000.spi: rx 03
>>> fsl_espi ffe110000.spi: Extra RX 00
>>> fsl_espi ffe110000.spi: Transfer done but SPIE_DON isn't set!
>>> fsl_espi ffe110000.spi: Transfer done but rx/tx fifo's aren't empty!
>>> fsl_espi ffe110000.spi: SPIE_RXCNT = 1, SPIE_TXCNT = 32
>>> fsl_espi ffe110000.spi: tx 05
>>> fsl_espi ffe110000.spi: rx 00
>>> fsl_espi ffe110000.spi: Extra RX 03
>>> fsl_espi ffe110000.spi: Transfer done but SPIE_DON isn't set!
>>> fsl_espi ffe110000.spi: Transfer done but rx/tx fifo's aren't empty!
>>> fsl_espi ffe110000.spi: SPIE_RXCNT = 1, SPIE_TXCNT = 32
>>> fsl_espi ffe110000.spi: tx 05
>>> fsl_espi ffe110000.spi: rx 00
>>> fsl_espi ffe110000.spi: Extra RX 03
>>>
>>> From all the Micron SPI-NOR datasheets I've got access to it is
>>> possible to continually read the SR/FSR. But I've no idea why it
>>> happens some times and not others.
>>
>> So I think I've got a reproduction and I think I've bisected the problem
>> to commit 3282a3da25bd ("powerpc/64: Implement soft interrupt replay in
>> C"). My day is just finishing now so I haven't applied too much scrutiny
>> to this result. Given the various rabbit holes I've been down on this
>> issue already I'd take this information with a good degree of skepticism.
>>
> OK, so an easy test should be to re-test with a 5.4 kernel.
> It doesn't have yet the change you're referring to, and the fsl-espi driver
> is basically the same as in 5.7 (just two small changes in 5.7).
There's 6cc0c16d82f88 and maybe also other interrupt related patches
around this time that could affect book E, so it's good if that exact
patch is confirmed.
I've been staring at 3282a3da25bd for a while and nothing immediately
stands out. It doesn't look like the low level handlers do anything
special (well 0x900 does ack the decrementer, but so does the masked
handler).
Can you try this patch and also enable CONFIG_PPC_IRQ_SOFT_MASK_DEBUG?
Thanks,
Nick
---
diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c
index bf21ebd36190..10d339042330 100644
--- a/arch/powerpc/kernel/irq.c
+++ b/arch/powerpc/kernel/irq.c
@@ -214,7 +214,7 @@ void replay_soft_interrupts(void)
struct pt_regs regs;
ppc_save_regs(®s);
- regs.softe = IRQS_ALL_DISABLED;
+ regs.softe = IRQS_ENABLED;
again:
if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG))
@@ -349,6 +349,7 @@ notrace void arch_local_irq_restore(unsigned long mask)
if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG))
WARN_ON_ONCE(!(mfmsr() & MSR_EE));
__hard_irq_disable();
+ local_paca->irq_happened |= PACA_IRQ_HARD_DIS;
} else {
/*
* We should already be hard disabled here. We had bugs
@@ -368,6 +369,7 @@ notrace void arch_local_irq_restore(unsigned long mask)
}
}
+ preempt_disable();
irq_soft_mask_set(IRQS_ALL_DISABLED);
trace_hardirqs_off();
@@ -377,6 +379,7 @@ notrace void arch_local_irq_restore(unsigned long mask)
trace_hardirqs_on();
irq_soft_mask_set(IRQS_ENABLED);
__hard_irq_enable();
+ preempt_enable();
}
EXPORT_SYMBOL(arch_local_irq_restore);
^ permalink raw reply related
* [PATCH v3 4/6] powerpc: Introduce temporary mm
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
x86 supports the notion of a temporary mm which restricts access to
temporary PTEs to a single CPU. A temporary mm is useful for situations
where a CPU needs to perform sensitive operations (such as patching a
STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing
said mappings to other CPUs. A side benefit is that other CPU TLBs do
not need to be flushed when the temporary mm is torn down.
Mappings in the temporary mm can be set in the userspace portion of the
address-space.
Interrupts must be disabled while the temporary mm is in use. HW
breakpoints, which may have been set by userspace as watchpoints on
addresses now within the temporary mm, are saved and disabled when
loading the temporary mm. The HW breakpoints are restored when unloading
the temporary mm. All HW breakpoints are indiscriminately disabled while
the temporary mm is in use.
Based on x86 implementation:
commit cefa929c034e
("x86/mm: Introduce temporary mm structs")
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/powerpc/include/asm/debug.h | 1 +
arch/powerpc/kernel/process.c | 5 +++
arch/powerpc/lib/code-patching.c | 65 ++++++++++++++++++++++++++++++++
3 files changed, 71 insertions(+)
diff --git a/arch/powerpc/include/asm/debug.h b/arch/powerpc/include/asm/debug.h
index ec57daf87f40..827350c9bcf3 100644
--- a/arch/powerpc/include/asm/debug.h
+++ b/arch/powerpc/include/asm/debug.h
@@ -46,6 +46,7 @@ static inline int debugger_fault_handler(struct pt_regs *regs) { return 0; }
#endif
void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk);
+void __get_breakpoint(int nr, struct arch_hw_breakpoint *brk);
bool ppc_breakpoint_available(void);
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
extern void do_send_trap(struct pt_regs *regs, unsigned long address,
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 016bd831908e..0758a8db6342 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -843,6 +843,11 @@ static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk)
return 0;
}
+void __get_breakpoint(int nr, struct arch_hw_breakpoint *brk)
+{
+ memcpy(brk, this_cpu_ptr(¤t_brk[nr]), sizeof(*brk));
+}
+
void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk)
{
memcpy(this_cpu_ptr(¤t_brk[nr]), brk, sizeof(*brk));
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 85d3fdca9452..89b37ece6d2f 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -17,6 +17,7 @@
#include <asm/code-patching.h>
#include <asm/setup.h>
#include <asm/inst.h>
+#include <asm/mmu_context.h>
static int __patch_instruction(struct ppc_inst *exec_addr, struct ppc_inst instr,
struct ppc_inst *patch_addr)
@@ -44,6 +45,70 @@ int raw_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
}
#ifdef CONFIG_STRICT_KERNEL_RWX
+
+struct temp_mm {
+ struct mm_struct *temp;
+ struct mm_struct *prev;
+ bool is_kernel_thread;
+ struct arch_hw_breakpoint brk[HBP_NUM_MAX];
+};
+
+static inline void init_temp_mm(struct temp_mm *temp_mm, struct mm_struct *mm)
+{
+ temp_mm->temp = mm;
+ temp_mm->prev = NULL;
+ temp_mm->is_kernel_thread = false;
+ memset(&temp_mm->brk, 0, sizeof(temp_mm->brk));
+}
+
+static inline void use_temporary_mm(struct temp_mm *temp_mm)
+{
+ lockdep_assert_irqs_disabled();
+
+ temp_mm->is_kernel_thread = current->mm == NULL;
+ if (temp_mm->is_kernel_thread)
+ temp_mm->prev = current->active_mm;
+ else
+ temp_mm->prev = current->mm;
+
+ /*
+ * Hash requires a non-NULL current->mm to allocate a userspace address
+ * when handling a page fault. Does not appear to hurt in Radix either.
+ */
+ current->mm = temp_mm->temp;
+ switch_mm_irqs_off(NULL, temp_mm->temp, current);
+
+ if (ppc_breakpoint_available()) {
+ struct arch_hw_breakpoint null_brk = {0};
+ int i = 0;
+
+ for (; i < nr_wp_slots(); ++i) {
+ __get_breakpoint(i, &temp_mm->brk[i]);
+ if (temp_mm->brk[i].type != 0)
+ __set_breakpoint(i, &null_brk);
+ }
+ }
+}
+
+static inline void unuse_temporary_mm(struct temp_mm *temp_mm)
+{
+ lockdep_assert_irqs_disabled();
+
+ if (temp_mm->is_kernel_thread)
+ current->mm = NULL;
+ else
+ current->mm = temp_mm->prev;
+ switch_mm_irqs_off(NULL, temp_mm->prev, current);
+
+ if (ppc_breakpoint_available()) {
+ int i = 0;
+
+ for (; i < nr_wp_slots(); ++i)
+ if (temp_mm->brk[i].type != 0)
+ __set_breakpoint(i, &temp_mm->brk[i]);
+ }
+}
+
static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
#ifdef CONFIG_LKDTM
--
2.28.0
^ permalink raw reply related
* [PATCH v3 3/6] Add LKDTM test to hijack a patch mapping (powerpc, x86_64)
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
When live patching with STRICT_KERNEL_RWX, the CPU doing the patching
must temporarily remap the page(s) containing the patch site with +W
permissions. While this temporary mapping is in use another CPU could
write to the same mapping and maliciously alter kernel text. Implement a
LKDTM test to attempt to exploit such an opening from another (ie. not
the patching) CPU. The test is implemented on x86_64 and powerpc only.
The LKDTM "hijack" test works as follows:
1. A CPU executes an infinite loop to patch an instruction.
This is the "patching" CPU.
2. Another CPU attempts to write to the address of the temporary
mapping used by the "patching" CPU. This other CPU is the
"hijacker" CPU. The hijack either fails with a segfault or
succeeds, in which case some kernel text is now overwritten.
How to run the test:
mount -t debugfs none /sys/kernel/debug
(echo HIJACK_PATCH > /sys/kernel/debug/provoke-crash/DIRECT)
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
drivers/misc/lkdtm/core.c | 1 +
drivers/misc/lkdtm/lkdtm.h | 1 +
drivers/misc/lkdtm/perms.c | 146 +++++++++++++++++++++++++++++++++++++
3 files changed, 148 insertions(+)
diff --git a/drivers/misc/lkdtm/core.c b/drivers/misc/lkdtm/core.c
index a5e344df9166..482e72f6a1e1 100644
--- a/drivers/misc/lkdtm/core.c
+++ b/drivers/misc/lkdtm/core.c
@@ -145,6 +145,7 @@ static const struct crashtype crashtypes[] = {
CRASHTYPE(WRITE_RO),
CRASHTYPE(WRITE_RO_AFTER_INIT),
CRASHTYPE(WRITE_KERN),
+ CRASHTYPE(HIJACK_PATCH),
CRASHTYPE(REFCOUNT_INC_OVERFLOW),
CRASHTYPE(REFCOUNT_ADD_OVERFLOW),
CRASHTYPE(REFCOUNT_INC_NOT_ZERO_OVERFLOW),
diff --git a/drivers/misc/lkdtm/lkdtm.h b/drivers/misc/lkdtm/lkdtm.h
index 8878538b2c13..8bd98e8f0443 100644
--- a/drivers/misc/lkdtm/lkdtm.h
+++ b/drivers/misc/lkdtm/lkdtm.h
@@ -60,6 +60,7 @@ void lkdtm_EXEC_USERSPACE(void);
void lkdtm_EXEC_NULL(void);
void lkdtm_ACCESS_USERSPACE(void);
void lkdtm_ACCESS_NULL(void);
+void lkdtm_HIJACK_PATCH(void);
/* lkdtm_refcount.c */
void lkdtm_REFCOUNT_INC_OVERFLOW(void);
diff --git a/drivers/misc/lkdtm/perms.c b/drivers/misc/lkdtm/perms.c
index 2dede2ef658f..0ed32aba5216 100644
--- a/drivers/misc/lkdtm/perms.c
+++ b/drivers/misc/lkdtm/perms.c
@@ -9,6 +9,7 @@
#include <linux/vmalloc.h>
#include <linux/mman.h>
#include <linux/uaccess.h>
+#include <linux/kthread.h>
#include <asm/cacheflush.h>
/* Whether or not to fill the target memory area with do_nothing(). */
@@ -222,6 +223,151 @@ void lkdtm_ACCESS_NULL(void)
pr_err("FAIL: survived bad write\n");
}
+#if defined(CONFIG_PPC) || defined(CONFIG_X86_64)
+#if defined(CONFIG_STRICT_KERNEL_RWX) && defined(CONFIG_SMP)
+/*
+ * This is just a dummy location to patch-over.
+ */
+static void patching_target(void)
+{
+ return;
+}
+
+#ifdef CONFIG_PPC
+#include <asm/code-patching.h>
+struct ppc_inst * const patch_site = (struct ppc_inst *)&patching_target;
+#endif
+
+#ifdef CONFIG_X86_64
+#include <asm/text-patching.h>
+int * const patch_site = (int *)&patching_target;
+#endif
+
+static inline int lkdtm_do_patch(int data)
+{
+#ifdef CONFIG_PPC
+ return patch_instruction(patch_site, ppc_inst(data));
+#endif
+#ifdef CONFIG_X86_64
+ text_poke(patch_site, &data, sizeof(int));
+ return 0;
+#endif
+}
+
+static inline bool lkdtm_verify_patch(int data)
+{
+#ifdef CONFIG_PPC
+ return ppc_inst_equal(ppc_inst_read(READ_ONCE(patch_site)),
+ ppc_inst(data));
+#endif
+#ifdef CONFIG_X86_64
+ return READ_ONCE(*patch_site) == data;
+#endif
+}
+
+static int lkdtm_patching_cpu(void *data)
+{
+ int err = 0;
+ int val = 0xdeadbeef;
+
+ pr_info("starting patching_cpu=%d\n", smp_processor_id());
+ do {
+ err = lkdtm_do_patch(val);
+ } while (lkdtm_verify_patch(val) && !err && !kthread_should_stop());
+
+ if (err)
+ pr_warn("patch_instruction returned error: %d\n", err);
+
+ set_current_state(TASK_INTERRUPTIBLE);
+ while (!kthread_should_stop()) {
+ schedule();
+ set_current_state(TASK_INTERRUPTIBLE);
+ }
+
+ return err;
+}
+
+void lkdtm_HIJACK_PATCH(void)
+{
+#ifdef CONFIG_PPC
+ struct ppc_inst original_insn = ppc_inst_read(READ_ONCE(patch_site));
+#endif
+#ifdef CONFIG_X86_64
+ int original_insn = READ_ONCE(*patch_site);
+#endif
+ struct task_struct *patching_kthrd;
+ int patching_cpu, hijacker_cpu, attempts;
+ unsigned long addr;
+ bool hijacked;
+ const int bad_data = 0xbad00bad;
+
+ if (num_online_cpus() < 2) {
+ pr_warn("need at least two cpus\n");
+ return;
+ }
+
+ hijacker_cpu = smp_processor_id();
+ patching_cpu = cpumask_any_but(cpu_online_mask, hijacker_cpu);
+
+ patching_kthrd = kthread_create_on_node(&lkdtm_patching_cpu, NULL,
+ cpu_to_node(patching_cpu),
+ "lkdtm_patching_cpu");
+ kthread_bind(patching_kthrd, patching_cpu);
+ wake_up_process(patching_kthrd);
+
+ addr = offset_in_page(patch_site) | read_cpu_patching_addr(patching_cpu);
+
+ pr_info("starting hijacker_cpu=%d\n", hijacker_cpu);
+ for (attempts = 0; attempts < 100000; ++attempts) {
+ /* Use __put_user to catch faults without an Oops */
+ hijacked = !__put_user(bad_data, (int *)addr);
+
+ if (hijacked) {
+ if (kthread_stop(patching_kthrd))
+ pr_err("error trying to stop patching thread\n");
+ break;
+ }
+ }
+ pr_info("hijack attempts: %d\n", attempts);
+
+ if (hijacked) {
+ if (lkdtm_verify_patch(bad_data))
+ pr_err("overwrote kernel text\n");
+ /*
+ * There are window conditions where the hijacker cpu manages to
+ * write to the patch site but the site gets overwritten again by
+ * the patching cpu. We still consider that a "successful" hijack
+ * since the hijacker cpu did not fault on the write.
+ */
+ pr_err("FAIL: wrote to another cpu's patching area\n");
+ } else {
+ kthread_stop(patching_kthrd);
+ }
+
+ /* Restore the original insn for any future lkdtm tests */
+#ifdef CONFIG_PPC
+ patch_instruction(patch_site, original_insn);
+#endif
+#ifdef CONFIG_X86_64
+ lkdtm_do_patch(original_insn);
+#endif
+}
+
+#else
+
+void lkdtm_HIJACK_PATCH(void)
+{
+ if (!IS_ENABLED(CONFIG_PPC) && !IS_ENABLED(CONFIG_X86_64))
+ pr_err("XFAIL: this test only runs on x86_64 or powerpc\n");
+ if (!IS_ENABLED(CONFIG_STRICT_KERNEL_RWX))
+ pr_err("XFAIL: this test requires CONFIG_STRICT_KERNEL_RWX\n");
+ if (!IS_ENABLED(CONFIG_SMP))
+ pr_err("XFAIL: this test requires CONFIG_SMP\n");
+}
+
+#endif /* CONFIG_STRICT_KERNEL_RWX && CONFIG_SMP */
+#endif /* CONFIG_PPC || CONFIG_X86_64 */
+
void __init lkdtm_perms_init(void)
{
/* Make sure we can write to __ro_after_init values during __init */
--
2.28.0
^ permalink raw reply related
* Re: [PATCH v1 4/9] powerpc/vdso: Remove unnecessary ifdefs in vdso_pagelist initialization
From: Christophe Leroy @ 2020-08-27 6:47 UTC (permalink / raw)
To: Michael Ellerman, Benjamin Herrenschmidt, Paul Mackerras
Cc: linuxppc-dev, linux-kernel
In-Reply-To: <87ft89h2st.fsf@mpe.ellerman.id.au>
On 08/26/2020 02:58 PM, Michael Ellerman wrote:
> Christophe Leroy <christophe.leroy@csgroup.eu> writes:
>> diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c
>> index daef14a284a3..bbb69832fd46 100644
>> --- a/arch/powerpc/kernel/vdso.c
>> +++ b/arch/powerpc/kernel/vdso.c
>> @@ -718,16 +710,14 @@ static int __init vdso_init(void)
> ...
>>
>> -
>> -#ifdef CONFIG_VDSO32
>> vdso32_kbase = &vdso32_start;
>>
>> /*
>> @@ -735,8 +725,6 @@ static int __init vdso_init(void)
>> */
>> vdso32_pages = (&vdso32_end - &vdso32_start) >> PAGE_SHIFT;
>> DBG("vdso32_kbase: %p, 0x%x pages\n", vdso32_kbase, vdso32_pages);
>> -#endif
>
> This didn't build for ppc64le:
>
> /opt/cross/gcc-8.20_binutils-2.32/powerpc64-unknown-linux-gnu/bin/powerpc64-unknown-linux-gnu-ld: arch/powerpc/kernel/vdso.o:(.toc+0x0): undefined reference to `vdso32_end'
> /opt/cross/gcc-8.20_binutils-2.32/powerpc64-unknown-linux-gnu/bin/powerpc64-unknown-linux-gnu-ld: arch/powerpc/kernel/vdso.o:(.toc+0x8): undefined reference to `vdso32_start'
> make[1]: *** [/scratch/michael/build/maint/Makefile:1166: vmlinux] Error 1
> make: *** [Makefile:185: __sub-make] Error 2
>
> So I just put that ifdef back.
>
The problem is because is_32bit() can still return true even when
CONFIG_VDSO32 is not set.
The change below fixes the problem:
diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c
index bbb69832fd46..38abff60cbe2 100644
--- a/arch/powerpc/kernel/vdso.c
+++ b/arch/powerpc/kernel/vdso.c
@@ -132,11 +132,7 @@ int arch_setup_additional_pages(struct linux_binprm
*bprm, int uses_interp)
if (!vdso_ready)
return 0;
- if (is_32bit_task()) {
- vdso_pagelist = vdso32_pagelist;
- vdso_pages = vdso32_pages;
- vdso_base = VDSO32_MBASE;
- } else {
+ if (!is_32bit_task()) {
vdso_pagelist = vdso64_pagelist;
vdso_pages = vdso64_pages;
/*
@@ -145,6 +141,12 @@ int arch_setup_additional_pages(struct linux_binprm
*bprm, int uses_interp)
* and most likely share a SLB entry.
*/
vdso_base = 0;
+ } else if (IS_ENABLED(CONFIG_VDSO32)) {
+ vdso_pagelist = vdso32_pagelist;
+ vdso_pages = vdso32_pages;
+ vdso_base = VDSO32_MBASE;
+ } else {
+ vdso_pages = 0;
}
current->mm->context.vdso_base = 0;
With this change all vdso32 static objects (functions and vars) go away
as expected.
We get a simple conflict with the following patch.
Do you prefer an updated series or a follow up patch, or you take the
above change yourself ?
Christophe
^ permalink raw reply related
* Re: [PATCHv5 2/2] powerpc/pseries: update device tree before ejecting hotplug uevents
From: Pingfan Liu @ 2020-08-27 6:36 UTC (permalink / raw)
To: linuxppc-dev
Cc: Nathan Lynch, Kexec Mailing List, Nathan Fontenot, Laurent Dufour,
Hari Bathini
In-Reply-To: <1597049570-19536-2-git-send-email-kernelfans@gmail.com>
Hello guys. Do you have further comments on this version?
Thanks,
Pingfan
On Mon, Aug 10, 2020 at 4:53 PM Pingfan Liu <kernelfans@gmail.com> wrote:
>
> A bug is observed on pseries by taking the following steps on rhel:
> -1. drmgr -c mem -r -q 5
> -2. echo c > /proc/sysrq-trigger
>
> And then, the failure looks like:
> kdump: saving to /sysroot//var/crash/127.0.0.1-2020-01-16-02:06:14/
> kdump: saving vmcore-dmesg.txt
> kdump: saving vmcore-dmesg.txt complete
> kdump: saving vmcore
> Checking for memory holes : [ 0.0 %] / Checking for memory holes : [100.0 %] | Excluding unnecessary pages : [100.0 %] \ Copying data : [ 0.3 %] - eta: 38s[ 44.337636] hash-mmu: mm: Hashing failure ! EA=0x7fffba400000 access=0x8000000000000004 current=makedumpfile
> [ 44.337663] hash-mmu: trap=0x300 vsid=0x13a109c ssize=1 base psize=2 psize 2 pte=0xc000000050000504
> [ 44.337677] hash-mmu: mm: Hashing failure ! EA=0x7fffba400000 access=0x8000000000000004 current=makedumpfile
> [ 44.337692] hash-mmu: trap=0x300 vsid=0x13a109c ssize=1 base psize=2 psize 2 pte=0xc000000050000504
> [ 44.337708] makedumpfile[469]: unhandled signal 7 at 00007fffba400000 nip 00007fffbbc4d7fc lr 000000011356ca3c code 2
> [ 44.338548] Core dump to |/bin/false pipe failed
> /lib/kdump-lib-initramfs.sh: line 98: 469 Bus error $CORE_COLLECTOR /proc/vmcore $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete
> kdump: saving vmcore failed
>
> * Root cause *
> After analyzing, it turns out that in the current implementation,
> when hot-removing lmb, the KOBJ_REMOVE event ejects before the dt updating as
> the code __remove_memory() comes before drmem_update_dt().
> So in kdump kernel, when read_from_oldmem() resorts to
> pSeries_lpar_hpte_insert() to install hpte, but fails with -2 due to
> non-exist pfn. And finally, low_hash_fault() raise SIGBUS to process, as it
> can be observed "Bus error"
>
> From a viewpoint of listener and publisher, the publisher notifies the
> listener before data is ready. This introduces a problem where udev
> launches kexec-tools (due to KOBJ_REMOVE) and loads a stale dt before
> updating. And in capture kernel, makedumpfile will access the memory based
> on the stale dt info, and hit a SIGBUS error due to an un-existed lmb.
>
> * Fix *
> This bug is introduced by commit 063b8b1251fd
> ("powerpc/pseries/memory-hotplug: Only update DT once per memory DLPAR
> request"), which tried to combine all the dt updating into one.
>
> To fix this issue, meanwhile not to introduce a quadratic runtime
> complexity by the model:
> dlpar_memory_add_by_count
> for_each_drmem_lmb <--
> dlpar_add_lmb
> drmem_update_dt(_v1|_v2)
> for_each_drmem_lmb <--
> The dt should still be only updated once, and just before the last memory
> online/offline event is ejected to user space. Achieve this by tracing the
> num of lmb added or removed.
>
> Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Nathan Lynch <nathanl@linux.ibm.com>
> Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> Cc: Laurent Dufour <ldufour@linux.ibm.com>
> To: linuxppc-dev@lists.ozlabs.org
> Cc: kexec@lists.infradead.org
> ---
> v4 -> v5: change dlpar_add_lmb()/dlpar_remove_lmb() prototype to report
> whether dt is updated successfully.
> Fix a condition boundary check bug
> v3 -> v4: resolve a quadratic runtime complexity issue.
> This series is applied on next-test branch
> arch/powerpc/platforms/pseries/hotplug-memory.c | 102 +++++++++++++++++++-----
> 1 file changed, 80 insertions(+), 22 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
> index 46cbcd1..1567d9f 100644
> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c
> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
> @@ -350,13 +350,22 @@ static bool lmb_is_removable(struct drmem_lmb *lmb)
> return true;
> }
>
> -static int dlpar_add_lmb(struct drmem_lmb *);
> +enum dt_update_status {
> + DT_NOUPDATE,
> + DT_TOUPDATE,
> + DT_UPDATED,
> +};
> +
> +/* "*dt_update" returns DT_UPDATED if updated */
> +static int dlpar_add_lmb(struct drmem_lmb *lmb,
> + enum dt_update_status *dt_update);
>
> -static int dlpar_remove_lmb(struct drmem_lmb *lmb)
> +static int dlpar_remove_lmb(struct drmem_lmb *lmb,
> + enum dt_update_status *dt_update)
> {
> unsigned long block_sz;
> phys_addr_t base_addr;
> - int rc, nid;
> + int rc, ret, nid;
>
> if (!lmb_is_removable(lmb))
> return -EINVAL;
> @@ -372,6 +381,13 @@ static int dlpar_remove_lmb(struct drmem_lmb *lmb)
> invalidate_lmb_associativity_index(lmb);
> lmb_clear_nid(lmb);
> lmb->flags &= ~DRCONF_MEM_ASSIGNED;
> + if (*dt_update) {
> + ret = drmem_update_dt();
> + if (ret)
> + pr_warn("%s fail to update dt, but continue\n", __func__);
> + else
> + *dt_update = DT_UPDATED;
> + }
>
> __remove_memory(nid, base_addr, block_sz);
>
> @@ -387,6 +403,7 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
> int lmbs_removed = 0;
> int lmbs_available = 0;
> int rc;
> + enum dt_update_status dt_update = DT_NOUPDATE;
>
> pr_info("Attempting to hot-remove %d LMB(s)\n", lmbs_to_remove);
>
> @@ -409,7 +426,11 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
> }
>
> for_each_drmem_lmb(lmb) {
> - rc = dlpar_remove_lmb(lmb);
> +
> + /* combine dt updating for all LMBs */
> + if (lmbs_to_remove - lmbs_removed <= 1)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_remove_lmb(lmb, &dt_update);
> if (rc)
> continue;
>
> @@ -424,13 +445,17 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
> }
>
> if (lmbs_removed != lmbs_to_remove) {
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
> +
> pr_err("Memory hot-remove failed, adding LMB's back\n");
>
> for_each_drmem_lmb(lmb) {
> if (!drmem_lmb_reserved(lmb))
> continue;
>
> - rc = dlpar_add_lmb(lmb);
> + if (--lmbs_removed == 0 && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to add LMB back, drc index %x\n",
> lmb->drc_index);
> @@ -458,6 +483,7 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
>
> static int dlpar_memory_remove_by_index(u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_TOUPDATE;
> struct drmem_lmb *lmb;
> int lmb_found;
> int rc;
> @@ -468,7 +494,7 @@ static int dlpar_memory_remove_by_index(u32 drc_index)
> for_each_drmem_lmb(lmb) {
> if (lmb->drc_index == drc_index) {
> lmb_found = 1;
> - rc = dlpar_remove_lmb(lmb);
> + rc = dlpar_remove_lmb(lmb, &dt_update);
> if (!rc)
> dlpar_release_drc(lmb->drc_index);
>
> @@ -490,6 +516,7 @@ static int dlpar_memory_remove_by_index(u32 drc_index)
>
> static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_NOUPDATE;
> struct drmem_lmb *lmb, *start_lmb, *end_lmb;
> int lmbs_available = 0;
> int rc;
> @@ -519,7 +546,9 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> if (!(lmb->flags & DRCONF_MEM_ASSIGNED))
> continue;
>
> - rc = dlpar_remove_lmb(lmb);
> + if (lmb == end_lmb)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_remove_lmb(lmb, &dt_update);
> if (rc)
> break;
>
> @@ -527,14 +556,16 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> }
>
> if (rc) {
> - pr_err("Memory indexed-count-remove failed, adding any removed LMBs\n");
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
>
> + pr_err("Memory indexed-count-remove failed, adding any removed LMBs\n");
>
> for_each_drmem_lmb_in_range(lmb, start_lmb, end_lmb) {
> if (!drmem_lmb_reserved(lmb))
> continue;
> -
> - rc = dlpar_add_lmb(lmb);
> + if (lmb == end_lmb && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to add LMB, drc index %x\n",
> lmb->drc_index);
> @@ -572,7 +603,7 @@ static inline int dlpar_memory_remove(struct pseries_hp_errorlog *hp_elog)
> {
> return -EOPNOTSUPP;
> }
> -static int dlpar_remove_lmb(struct drmem_lmb *lmb)
> +static int dlpar_remove_lmb(struct drmem_lmb *lmb, bool dt_update)
> {
> return -EOPNOTSUPP;
> }
> @@ -591,10 +622,11 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> }
> #endif /* CONFIG_MEMORY_HOTREMOVE */
>
> -static int dlpar_add_lmb(struct drmem_lmb *lmb)
> +static int dlpar_add_lmb(struct drmem_lmb *lmb,
> + enum dt_update_status *dt_update)
> {
> unsigned long block_sz;
> - int rc;
> + int rc, ret;
>
> if (lmb->flags & DRCONF_MEM_ASSIGNED)
> return -EINVAL;
> @@ -607,6 +639,13 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
>
> lmb_set_nid(lmb);
> lmb->flags |= DRCONF_MEM_ASSIGNED;
> + if (*dt_update) {
> + ret = drmem_update_dt();
> + if (ret)
> + pr_warn("%s fail to update dt, but continue\n", __func__);
> + else
> + *dt_update = DT_UPDATED;
> + }
>
> block_sz = memory_block_size_bytes();
>
> @@ -616,6 +655,8 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
> invalidate_lmb_associativity_index(lmb);
> lmb_clear_nid(lmb);
> lmb->flags &= ~DRCONF_MEM_ASSIGNED;
> + if (*dt_update == DT_UPDATED)
> + drmem_update_dt();
> return rc;
> }
>
> @@ -627,7 +668,11 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
> invalidate_lmb_associativity_index(lmb);
> lmb_clear_nid(lmb);
> lmb->flags &= ~DRCONF_MEM_ASSIGNED;
> -
> + if (*dt_update == DT_UPDATED) {
> + ret = drmem_update_dt();
> + if (ret)
> + pr_warn("%s fail to update dt during rollback, but continue\n", __func__);
> + }
> __remove_memory(nid, base_addr, block_sz);
> }
>
> @@ -636,6 +681,7 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
>
> static int dlpar_memory_add_by_count(u32 lmbs_to_add)
> {
> + enum dt_update_status dt_update = DT_NOUPDATE;
> struct drmem_lmb *lmb;
> int lmbs_available = 0;
> int lmbs_added = 0;
> @@ -666,7 +712,9 @@ static int dlpar_memory_add_by_count(u32 lmbs_to_add)
> if (rc)
> continue;
>
> - rc = dlpar_add_lmb(lmb);
> + if (lmbs_to_add - lmbs_added <= 1)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &dt_update);
> if (rc) {
> dlpar_release_drc(lmb->drc_index);
> continue;
> @@ -683,13 +731,18 @@ static int dlpar_memory_add_by_count(u32 lmbs_to_add)
> }
>
> if (lmbs_added != lmbs_to_add) {
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
> +
> pr_err("Memory hot-add failed, removing any added LMBs\n");
>
> for_each_drmem_lmb(lmb) {
> if (!drmem_lmb_reserved(lmb))
> continue;
>
> - rc = dlpar_remove_lmb(lmb);
> + if (--lmbs_added == 0 && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> +
> + rc = dlpar_remove_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to remove LMB, drc index %x\n",
> lmb->drc_index);
> @@ -716,6 +769,7 @@ static int dlpar_memory_add_by_count(u32 lmbs_to_add)
>
> static int dlpar_memory_add_by_index(u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_TOUPDATE;
> struct drmem_lmb *lmb;
> int rc, lmb_found;
>
> @@ -727,7 +781,7 @@ static int dlpar_memory_add_by_index(u32 drc_index)
> lmb_found = 1;
> rc = dlpar_acquire_drc(lmb->drc_index);
> if (!rc) {
> - rc = dlpar_add_lmb(lmb);
> + rc = dlpar_add_lmb(lmb, &dt_update);
> if (rc)
> dlpar_release_drc(lmb->drc_index);
> }
> @@ -750,6 +804,7 @@ static int dlpar_memory_add_by_index(u32 drc_index)
>
> static int dlpar_memory_add_by_ic(u32 lmbs_to_add, u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_NOUPDATE;
> struct drmem_lmb *lmb, *start_lmb, *end_lmb;
> int lmbs_available = 0;
> int rc;
> @@ -783,7 +838,9 @@ static int dlpar_memory_add_by_ic(u32 lmbs_to_add, u32 drc_index)
> if (rc)
> break;
>
> - rc = dlpar_add_lmb(lmb);
> + if (lmb == end_lmb)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &dt_update);
> if (rc) {
> dlpar_release_drc(lmb->drc_index);
> break;
> @@ -796,10 +853,14 @@ static int dlpar_memory_add_by_ic(u32 lmbs_to_add, u32 drc_index)
> pr_err("Memory indexed-count-add failed, removing any added LMBs\n");
>
> for_each_drmem_lmb_in_range(lmb, start_lmb, end_lmb) {
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
> +
> if (!drmem_lmb_reserved(lmb))
> continue;
>
> - rc = dlpar_remove_lmb(lmb);
> + if (lmb == end_lmb && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> + rc = dlpar_remove_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to remove LMB, drc index %x\n",
> lmb->drc_index);
> @@ -879,9 +940,6 @@ int dlpar_memory(struct pseries_hp_errorlog *hp_elog)
> break;
> }
>
> - if (!rc)
> - rc = drmem_update_dt();
> -
> unlock_device_hotplug();
> return rc;
> }
> --
> 2.7.5
>
^ permalink raw reply
* [PATCH v3 0/6] Use per-CPU temporary mappings for patching
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
When compiled with CONFIG_STRICT_KERNEL_RWX, the kernel must create
temporary mappings when patching itself. These mappings temporarily
override the strict RWX text protections to permit a write. Currently,
powerpc allocates a per-CPU VM area for patching. Patching occurs as
follows:
1. Map page of text to be patched to per-CPU VM area w/
PAGE_KERNEL protection
2. Patch text
3. Remove the temporary mapping
While the VM area is per-CPU, the mapping is actually inserted into the
kernel page tables. Presumably, this could allow another CPU to access
the normally write-protected text - either malicously or accidentally -
via this same mapping if the address of the VM area is known. Ideally,
the mapping should be kept local to the CPU doing the patching (or any
other sensitive operations requiring temporarily overriding memory
protections) [0].
x86 introduced "temporary mm" structs which allow the creation of
mappings local to a particular CPU [1]. This series intends to bring the
notion of a temporary mm to powerpc and harden powerpc by using such a
mapping for patching a kernel with strict RWX permissions.
The first, second, and third patches implement an LKDTM test
"proof-of-concept" which exploits the potential vulnerability (ie. the
mapping during patching is exposed in kernel page tables and accessible
by other CPUS). The LKDTM test is somewhat "rough" in that it uses a
brute-force approach - I am open to any suggestions and/or ideas to
improve this. Currently, the LKDTM test passes with this series on
POWER8 (hash) and POWER9 (radix, hash) and fails without this series
(ie. the temporary mapping for patching is exposed to CPUs other than
the patching CPU).
The test is also implemented on x86_64 where it passes with a current
tree and fails only when using a tree prior to the temporary mappings. I
have such a tree here which intentionally fails:
https://github.com/cmr-informatik/linux/tree/x86_64-non-percpu-lkdtm
The fourth patch introduces the temporary mm struct and API for powerpc
along with a new function to retrieve a current hw breakpoint.
The fifth patch uses the `poking_init` init hook added by the x86
patches to initialize a temporary mm and patching address. The patching
address is randomized between PAGE_SIZE and DEFAULT_MAP_WINDOW-PAGE_SIZE.
The upper limit is necessary due to how the hash MMU operates - by default
the space above DEFAULT_MAP_WINDOW is not available. For now, both hash
and radix randomize inside this range. The number of possible random
addresses is dependent on PAGE_SIZE and limited by DEFAULT_MAP_WINDOW.
Bits of entropy with 64K page size on BOOK3S_64:
bits of entropy = log2(DEFAULT_MAP_WINDOW_USER64 / PAGE_SIZE)
PAGE_SIZE=64K, DEFAULT_MAP_WINDOW_USER64=128TB
bits of entropy = log2(128TB / 64K)
bits of entropy = 31
Randomization occurs only once during initialization at boot.
The sixth patch replaces the VM area with the temporary mm in the
patching code. The page for patching has to be mapped PAGE_SHARED with
the hash MMU since hash prevents the kernel from accessing userspace
pages with PAGE_PRIVILEGED bit set. On the radix MMU the page is mapped
with PAGE_KERNEL which has the added benefit that we can skip KUAP.
Tested on Blackbird (8-core POWER9) w/ Hash (`disable_radix`) and Radix
MMUs, QEMU (TCG) POWER8 and POWER9, POWER8 VM.
Tested LKDTM test (passing and failing situations) on QEMU x86_64.
v3: * Rebase on linuxppc/next:
commit 9123e3a74ec7 ("Linux 5.9-rc1")
* Move temporary mm implementation into code-patching.c where it
belongs
* Implement LKDTM hijacker test on x86_64 (on IBM time oof)
* Do not use address zero for the patching address in the
temporary mm (thanks @dja for pointing this out!)
* Wrap the LKDTM test w/ CONFIG_SMP as suggested by Christophe
Leroy
* Comments to clarify PTE pre-allocation and patching addr
selection
v2: * Rebase on linuxppc/next:
commit 105fb38124a4 ("powerpc/8xx: Modify ptep_get()")
* Always dirty pte when mapping patch
* Use `ppc_inst_len` instead of `sizeof` on instructions
* Declare LKDTM patching addr accessor in header where it
belongs
v1: * Rebase on linuxppc/next (4336b9337824)
* Save and restore second hw watchpoint
* Use new ppc_inst_* functions for patching check and in LKDTM
test
rfc-v2: * Many fixes and improvements mostly based on extensive feedback and
testing by Christophe Leroy (thanks!).
* Make patching_mm and patching_addr static and mode '__ro_after_init'
to after the variable name (more common in other parts of the kernel)
* Use 'asm/debug.h' header instead of 'asm/hw_breakpoint.h' to fix
PPC64e compile
* Add comment explaining why we use BUG_ON() during the init call to
setup for patching later
* Move ptep into patch_mapping to avoid walking page tables a second
time when unmapping the temporary mapping
* Use KUAP under non-radix, also manually dirty the PTE for patch
mapping on non-BOOK3S_64 platforms
* Properly return any error from __patch_instruction
* Do not use 'memcmp' where a simple comparison is appropriate
* Simplify expression for patch address by removing pointer maths
* Add LKDTM test
[0]: https://github.com/linuxppc/issues/issues/224
[1]: https://lore.kernel.org/kernel-hardening/20190426232303.28381-1-nadav.amit@gmail.com/
Christopher M. Riedl (6):
powerpc: Add LKDTM accessor for patching addr
x86: Add LKDTM accessor for patching addr
Add LKDTM test to hijack a patch mapping (powerpc,x86_64)
powerpc: Introduce temporary mm
powerpc: Initialize a temporary mm for code patching
powerpc: Use a temporary mm for code patching
arch/powerpc/include/asm/code-patching.h | 4 +
arch/powerpc/include/asm/debug.h | 1 +
arch/powerpc/kernel/process.c | 5 +
arch/powerpc/lib/code-patching.c | 239 +++++++++++++++--------
arch/x86/include/asm/text-patching.h | 4 +
arch/x86/kernel/alternative.c | 7 +
drivers/misc/lkdtm/core.c | 1 +
drivers/misc/lkdtm/lkdtm.h | 1 +
drivers/misc/lkdtm/perms.c | 146 ++++++++++++++
9 files changed, 322 insertions(+), 86 deletions(-)
--
2.28.0
^ permalink raw reply
* Re: [PATCH v2] powerpc: Update documentation of ISA versions for Power10
From: Michael Ellerman @ 2020-08-27 6:18 UTC (permalink / raw)
To: Jordan Niethe, linuxppc-dev; +Cc: Jordan Niethe
In-Reply-To: <20200827040556.1783-1-jniethe5@gmail.com>
Jordan Niethe <jniethe5@gmail.com> writes:
> Update the CPU to ISA Version Mapping document to include Power10 and
> ISA v3.1.
>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v2: Transactional Memory = No
> ---
> Documentation/powerpc/isa-versions.rst | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/Documentation/powerpc/isa-versions.rst b/Documentation/powerpc/isa-versions.rst
> index a363d8c1603c..3873bbba183a 100644
> --- a/Documentation/powerpc/isa-versions.rst
> +++ b/Documentation/powerpc/isa-versions.rst
> @@ -62,6 +65,7 @@ PPC970 No
> ========== ====================================
> CPU Transactional Memory
> ========== ====================================
> +Power10 No (* see Power ISA v3.1 Appendix A.)
There's three "Appendix A"s in ISA v3.1.
There's one in Book I, and one in Book II.
And then the one you're referring to is not actually in Book III, it's
listed after Book III, and is apparently an appendix to all three books?
Which is just utterly confusing.
So I'll change it to say:
"Appendix A. Notes on the Removal of Transactional Memory from the Architecture"
Which is very long, but at least you can search for it.
cheers
^ permalink raw reply
* [PATCH v3 2/6] x86: Add LKDTM accessor for patching addr
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
When live patching a STRICT_RWX kernel, a mapping is installed at a
"patching address" with temporary write permissions. Provide a
LKDTM-only accessor function for this address in preparation for a LKDTM
test which attempts to "hijack" this mapping by writing to it from
another CPU.
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/x86/include/asm/text-patching.h | 4 ++++
arch/x86/kernel/alternative.c | 7 +++++++
2 files changed, 11 insertions(+)
diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h
index 6593b42cb379..f67b4dd30bf8 100644
--- a/arch/x86/include/asm/text-patching.h
+++ b/arch/x86/include/asm/text-patching.h
@@ -148,4 +148,8 @@ void int3_emulate_call(struct pt_regs *regs, unsigned long func)
}
#endif /* !CONFIG_UML_X86 */
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu);
+#endif
+
#endif /* _ASM_X86_TEXT_PATCHING_H */
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index c3daf0aaa0ee..c16833f35a1f 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -843,6 +843,13 @@ static inline void unuse_temporary_mm(temp_mm_state_t prev_state)
__ro_after_init struct mm_struct *poking_mm;
__ro_after_init unsigned long poking_addr;
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu)
+{
+ return poking_addr;
+}
+#endif
+
static void *__text_poke(void *addr, const void *opcode, size_t len)
{
bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
--
2.28.0
^ permalink raw reply related
* Re: [PATCH] Revert "powerpc/powernv/idle: Replace CPU feature check with PVR check"
From: Michael Ellerman @ 2020-08-27 6:07 UTC (permalink / raw)
To: Pratik Sampat, Christophe Leroy, linuxppc-dev, npiggin, mikey,
ego, svaidy, linux-kernel, pratik.r.sampat
In-Reply-To: <170e1919-bc45-6b99-dc4d-713418c98be1@linux.ibm.com>
Pratik Sampat <psampat@linux.ibm.com> writes:
> On 26/08/20 2:07 pm, Christophe Leroy wrote:
>> Le 26/08/2020 à 10:29, Pratik Rajesh Sampat a écrit :
>>> Cpuidle stop state implementation has minor optimizations for P10
>>> where hardware preserves more SPR registers compared to P9.
>>> The current P9 driver works for P10, although does few extra
>>> save-restores. P9 driver can provide the required power management
>>> features like SMT thread folding and core level power savings
>>> on a P10 platform.
>>>
>>> Until the P10 stop driver is available, revert the commit which
>>> allows for only P9 systems to utilize cpuidle and blocks all
>>> idle stop states for P10.
>>> Cpu idle states are enabled and tested on the P10 platform
>>> with this fix.
>>>
>>> This reverts commit 8747bf36f312356f8a295a0c39ff092d65ce75ae.
>>>
>>> Fixes: 8747bf36f312 ("powerpc/powernv/idle: Replace CPU feature check
>>> with PVR check")
>>> Signed-off-by: Pratik Rajesh Sampat <psampat@linux.ibm.com>
>>> ---
>>> @mpe: This revert would resolve a staging issue wherein the P10 stop
>>> driver is not yet ready while cpuidle stop states need not be blocked
>>> on 5.9 for Power10 systems which could cause SMT folding related
>>> performance issues.
>>>
>>> The P10 stop driver is in the works here:
>>> https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216773.html
>>>
>>> arch/powerpc/platforms/powernv/idle.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/platforms/powernv/idle.c
>>> b/arch/powerpc/platforms/powernv/idle.c
>>> index 77513a80cef9..345ab062b21a 100644
>>> --- a/arch/powerpc/platforms/powernv/idle.c
>>> +++ b/arch/powerpc/platforms/powernv/idle.c
>>> @@ -1223,7 +1223,7 @@ static void __init pnv_probe_idle_states(void)
>>> return;
>>> }
>>> - if (pvr_version_is(PVR_POWER9))
>>> + if (cpu_has_feature(CPU_FTR_ARCH_300))
>>
>> Why not something like:
>>
>> if (pvr_version_is(PVR_POWER9) || pvr_version_is(PVR_POWER10))
>> pnv_power9_idle_init();
>
> In order to use PVR_POWER10 I would need to define it under
> arch/powerpc/include/asm/reg.h, which is not present in 5.9 yet.
>
> However, if it okay with @mpe I could split out Nick's P10 stop driver
> (https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216773.html)
> into two parts:
I'll just take this for now, it's the simplest option.
cheers
^ permalink raw reply
* [PATCH v3 6/6] powerpc: Use a temporary mm for code patching
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
Currently, code patching a STRICT_KERNEL_RWX exposes the temporary
mappings to other CPUs. These mappings should be kept local to the CPU
doing the patching. Use the pre-initialized temporary mm and patching
address for this purpose. Also add a check after patching to ensure the
patch succeeded.
Toggle KUAP on non-radix MMU platforms when patching since the temporary
mapping for patching uses a userspace address. With a radix MMU this is
not required since mapping a page with PAGE_KERNEL sets EAA[0] for the
PTE which means the AMR (KUAP) protection is ignored (see PowerISA
v3.0b, Fig. 35).
Based on x86 implementation:
commit b3fd8e83ada0
("x86/alternatives: Use temporary mm for text poking")
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/powerpc/lib/code-patching.c | 153 +++++++++++--------------------
1 file changed, 54 insertions(+), 99 deletions(-)
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 051d7ae6d8ee..9fb098680da8 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -149,113 +149,64 @@ void __init poking_init(void)
pte_unmap_unlock(ptep, ptl);
}
-static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
-
#ifdef CONFIG_LKDTM
unsigned long read_cpu_patching_addr(unsigned int cpu)
{
- return (unsigned long)(per_cpu(text_poke_area, cpu))->addr;
+ return patching_addr;
}
#endif
-static int text_area_cpu_up(unsigned int cpu)
-{
- struct vm_struct *area;
-
- area = get_vm_area(PAGE_SIZE, VM_ALLOC);
- if (!area) {
- WARN_ONCE(1, "Failed to create text area for cpu %d\n",
- cpu);
- return -1;
- }
- this_cpu_write(text_poke_area, area);
-
- return 0;
-}
-
-static int text_area_cpu_down(unsigned int cpu)
-{
- free_vm_area(this_cpu_read(text_poke_area));
- return 0;
-}
-
-/*
- * Run as a late init call. This allows all the boot time patching to be done
- * simply by patching the code, and then we're called here prior to
- * mark_rodata_ro(), which happens after all init calls are run. Although
- * BUG_ON() is rude, in this case it should only happen if ENOMEM, and we judge
- * it as being preferable to a kernel that will crash later when someone tries
- * to use patch_instruction().
- */
-static int __init setup_text_poke_area(void)
-{
- BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
- "powerpc/text_poke:online", text_area_cpu_up,
- text_area_cpu_down));
-
- return 0;
-}
-late_initcall(setup_text_poke_area);
+struct patch_mapping {
+ spinlock_t *ptl; /* for protecting pte table */
+ pte_t *ptep;
+ struct temp_mm temp_mm;
+};
/*
* This can be called for kernel text or a module.
*/
-static int map_patch_area(void *addr, unsigned long text_poke_addr)
+static int map_patch(const void *addr, struct patch_mapping *patch_mapping)
{
- unsigned long pfn;
- int err;
+ struct page *page;
+ pte_t pte;
+ pgprot_t pgprot;
if (is_vmalloc_or_module_addr(addr))
- pfn = vmalloc_to_pfn(addr);
+ page = vmalloc_to_page(addr);
else
- pfn = __pa_symbol(addr) >> PAGE_SHIFT;
+ page = virt_to_page(addr);
- err = map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL);
+ if (radix_enabled())
+ pgprot = PAGE_KERNEL;
+ else
+ pgprot = PAGE_SHARED;
- pr_devel("Mapped addr %lx with pfn %lx:%d\n", text_poke_addr, pfn, err);
- if (err)
+ patch_mapping->ptep = get_locked_pte(patching_mm, patching_addr,
+ &patch_mapping->ptl);
+ if (unlikely(!patch_mapping->ptep)) {
+ pr_warn("map patch: failed to allocate pte for patching\n");
return -1;
+ }
+
+ pte = mk_pte(page, pgprot);
+ pte = pte_mkdirty(pte);
+ set_pte_at(patching_mm, patching_addr, patch_mapping->ptep, pte);
+
+ init_temp_mm(&patch_mapping->temp_mm, patching_mm);
+ use_temporary_mm(&patch_mapping->temp_mm);
return 0;
}
-static inline int unmap_patch_area(unsigned long addr)
+static void unmap_patch(struct patch_mapping *patch_mapping)
{
- pte_t *ptep;
- pmd_t *pmdp;
- pud_t *pudp;
- p4d_t *p4dp;
- pgd_t *pgdp;
-
- pgdp = pgd_offset_k(addr);
- if (unlikely(!pgdp))
- return -EINVAL;
-
- p4dp = p4d_offset(pgdp, addr);
- if (unlikely(!p4dp))
- return -EINVAL;
-
- pudp = pud_offset(p4dp, addr);
- if (unlikely(!pudp))
- return -EINVAL;
-
- pmdp = pmd_offset(pudp, addr);
- if (unlikely(!pmdp))
- return -EINVAL;
+ /* In hash, pte_clear flushes the tlb */
+ pte_clear(patching_mm, patching_addr, patch_mapping->ptep);
+ unuse_temporary_mm(&patch_mapping->temp_mm);
- ptep = pte_offset_kernel(pmdp, addr);
- if (unlikely(!ptep))
- return -EINVAL;
-
- pr_devel("clearing mm %p, pte %p, addr %lx\n", &init_mm, ptep, addr);
-
- /*
- * In hash, pte_clear flushes the tlb, in radix, we have to
- */
- pte_clear(&init_mm, addr, ptep);
- flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
-
- return 0;
+ /* In radix, we have to explicitly flush the tlb (no-op in hash) */
+ local_flush_tlb_mm(patching_mm);
+ pte_unmap_unlock(patch_mapping->ptep, patch_mapping->ptl);
}
static int do_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
@@ -263,32 +214,36 @@ static int do_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
int err;
struct ppc_inst *patch_addr = NULL;
unsigned long flags;
- unsigned long text_poke_addr;
- unsigned long kaddr = (unsigned long)addr;
+ struct patch_mapping patch_mapping;
/*
- * During early early boot patch_instruction is called
- * when text_poke_area is not ready, but we still need
- * to allow patching. We just do the plain old patching
+ * The patching_mm is initialized before calling mark_rodata_ro. Prior
+ * to this, patch_instruction is called when we don't have (and don't
+ * need) the patching_mm so just do plain old patching.
*/
- if (!this_cpu_read(text_poke_area))
+ if (!patching_mm)
return raw_patch_instruction(addr, instr);
local_irq_save(flags);
- text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr;
- if (map_patch_area(addr, text_poke_addr)) {
- err = -1;
+ err = map_patch(addr, &patch_mapping);
+ if (err)
goto out;
- }
- patch_addr = (struct ppc_inst *)(text_poke_addr + (kaddr & ~PAGE_MASK));
+ patch_addr = (struct ppc_inst *)(patching_addr | offset_in_page(addr));
- __patch_instruction(addr, instr, patch_addr);
+ if (!radix_enabled())
+ allow_write_to_user(patch_addr, ppc_inst_len(instr));
+ err = __patch_instruction(addr, instr, patch_addr);
+ if (!radix_enabled())
+ prevent_write_to_user(patch_addr, ppc_inst_len(instr));
- err = unmap_patch_area(text_poke_addr);
- if (err)
- pr_warn("failed to unmap %lx\n", text_poke_addr);
+ unmap_patch(&patch_mapping);
+ /*
+ * Something is wrong if what we just wrote doesn't match what we
+ * think we just wrote.
+ */
+ WARN_ON(!ppc_inst_equal(ppc_inst_read(addr), instr));
out:
local_irq_restore(flags);
--
2.28.0
^ permalink raw reply related
* [PATCH v3 1/6] powerpc: Add LKDTM accessor for patching addr
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
When live patching a STRICT_RWX kernel, a mapping is installed at a
"patching address" with temporary write permissions. Provide a
LKDTM-only accessor function for this address in preparation for a LKDTM
test which attempts to "hijack" this mapping by writing to it from
another CPU.
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/powerpc/include/asm/code-patching.h | 4 ++++
arch/powerpc/lib/code-patching.c | 7 +++++++
2 files changed, 11 insertions(+)
diff --git a/arch/powerpc/include/asm/code-patching.h b/arch/powerpc/include/asm/code-patching.h
index eacc9102c251..7216d6a6bb0a 100644
--- a/arch/powerpc/include/asm/code-patching.h
+++ b/arch/powerpc/include/asm/code-patching.h
@@ -187,4 +187,8 @@ static inline unsigned long ppc_kallsyms_lookup_name(const char *name)
___PPC_RA(__REG_R1) | PPC_LR_STKOFF)
#endif /* CONFIG_PPC64 */
+#if defined(CONFIG_LKDTM) && defined(CONFIG_STRICT_KERNEL_RWX)
+unsigned long read_cpu_patching_addr(unsigned int cpu);
+#endif
+
#endif /* _ASM_POWERPC_CODE_PATCHING_H */
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 8c3934ea6220..85d3fdca9452 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -46,6 +46,13 @@ int raw_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
#ifdef CONFIG_STRICT_KERNEL_RWX
static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu)
+{
+ return (unsigned long)(per_cpu(text_poke_area, cpu))->addr;
+}
+#endif
+
static int text_area_cpu_up(unsigned int cpu)
{
struct vm_struct *area;
--
2.28.0
^ permalink raw reply related
* Re: [PATCH v2] powerpc: Update documentation of ISA versions for Power10
From: Jordan Niethe @ 2020-08-27 5:00 UTC (permalink / raw)
To: Christophe Leroy; +Cc: linuxppc-dev
In-Reply-To: <d71a27b8-f12d-7485-23ec-99d36ff1b0ea@csgroup.eu>
On Thu, Aug 27, 2020 at 2:49 PM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
>
>
>
> Le 27/08/2020 à 06:05, Jordan Niethe a écrit :
> > Update the CPU to ISA Version Mapping document to include Power10 and
> > ISA v3.1.
>
> Maybe Documentation/powerpc/cpu_families.rst should be updated as well.
Good idea it still needs Power9 too.
>
> Christophe
>
>
>
> >
> > Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> > ---
> > v2: Transactional Memory = No
> > ---
> > Documentation/powerpc/isa-versions.rst | 4 ++++
> > 1 file changed, 4 insertions(+)
> >
> > diff --git a/Documentation/powerpc/isa-versions.rst b/Documentation/powerpc/isa-versions.rst
> > index a363d8c1603c..3873bbba183a 100644
> > --- a/Documentation/powerpc/isa-versions.rst
> > +++ b/Documentation/powerpc/isa-versions.rst
> > @@ -7,6 +7,7 @@ Mapping of some CPU versions to relevant ISA versions.
> > ========= ====================================================================
> > CPU Architecture version
> > ========= ====================================================================
> > +Power10 Power ISA v3.1
> > Power9 Power ISA v3.0B
> > Power8 Power ISA v2.07
> > Power7 Power ISA v2.06
> > @@ -32,6 +33,7 @@ Key Features
> > ========== ==================
> > CPU VMX (aka. Altivec)
> > ========== ==================
> > +Power10 Yes
> > Power9 Yes
> > Power8 Yes
> > Power7 Yes
> > @@ -47,6 +49,7 @@ PPC970 Yes
> > ========== ====
> > CPU VSX
> > ========== ====
> > +Power10 Yes
> > Power9 Yes
> > Power8 Yes
> > Power7 Yes
> > @@ -62,6 +65,7 @@ PPC970 No
> > ========== ====================================
> > CPU Transactional Memory
> > ========== ====================================
> > +Power10 No (* see Power ISA v3.1 Appendix A.)
> > Power9 Yes (* see transactional_memory.txt)
> > Power8 Yes
> > Power7 No
> >
^ permalink raw reply
* Re: [PATCH v5 08/23] powerpc/book3s64/kuap: Rename MMU_FTR_RADIX_KUAP to MMU_FTR_KUAP
From: Christophe Leroy @ 2020-08-27 4:57 UTC (permalink / raw)
To: Aneesh Kumar K.V, linuxppc-dev, mpe
In-Reply-To: <20200827040931.297759-9-aneesh.kumar@linux.ibm.com>
Le 27/08/2020 à 06:09, Aneesh Kumar K.V a écrit :
> This is in preparate to adding support for kuap with hash translation.
> In preparation for that rename/move kuap related functions to
> non radix names. Also move the feature bit closer to MMU_FTR_KUEP.
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> ---
> arch/powerpc/include/asm/book3s/64/kup.h | 18 +++++++++---------
> arch/powerpc/include/asm/mmu.h | 16 ++++++++--------
> arch/powerpc/mm/book3s64/pkeys.c | 2 +-
> 3 files changed, 18 insertions(+), 18 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
> index 918a2fcceee7..5cec202dc42f 100644
> --- a/arch/powerpc/include/asm/book3s/64/kup.h
> +++ b/arch/powerpc/include/asm/book3s/64/kup.h
> @@ -24,7 +24,7 @@
> mtspr SPRN_AMR, \gpr2
> /* No isync required, see kuap_restore_amr() */
> 998:
> - END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
> + END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
> #endif
> .endm
>
> @@ -36,7 +36,7 @@
> sldi \gpr2, \gpr2, AMR_KUAP_SHIFT
> 999: tdne \gpr1, \gpr2
> EMIT_BUG_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING | BUGFLAG_ONCE)
> - END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
> + END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
> #endif
> .endm
>
> @@ -56,7 +56,7 @@
> mtspr SPRN_AMR, \gpr2
> isync
> 99:
> - END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
> + END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
> #endif
> .endm
>
> @@ -69,7 +69,7 @@
>
> static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
> {
> - if (mmu_has_feature(MMU_FTR_RADIX_KUAP) && unlikely(regs->kuap != amr)) {
> + if (mmu_has_feature(MMU_FTR_KUAP) && unlikely(regs->kuap != amr)) {
> isync();
> mtspr(SPRN_AMR, regs->kuap);
> /*
> @@ -82,7 +82,7 @@ static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
>
> static inline unsigned long kuap_get_and_check_amr(void)
> {
> - if (mmu_has_feature(MMU_FTR_RADIX_KUAP)) {
> + if (mmu_has_feature(MMU_FTR_KUAP)) {
> unsigned long amr = mfspr(SPRN_AMR);
> if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG)) /* kuap_check_amr() */
> WARN_ON_ONCE(amr != AMR_KUAP_BLOCKED);
> @@ -93,7 +93,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
>
> static inline void kuap_check_amr(void)
> {
> - if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && mmu_has_feature(MMU_FTR_RADIX_KUAP))
> + if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && mmu_has_feature(MMU_FTR_KUAP))
> WARN_ON_ONCE(mfspr(SPRN_AMR) != AMR_KUAP_BLOCKED);
> }
>
> @@ -122,7 +122,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
>
> static inline unsigned long get_kuap(void)
> {
> - if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
> + if (!early_mmu_has_feature(MMU_FTR_KUAP))
> return 0;
>
> return mfspr(SPRN_AMR);
> @@ -130,7 +130,7 @@ static inline unsigned long get_kuap(void)
>
> static inline void set_kuap(unsigned long value)
> {
> - if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
> + if (!early_mmu_has_feature(MMU_FTR_KUAP))
> return;
>
> /*
> @@ -180,7 +180,7 @@ static inline void restore_user_access(unsigned long flags)
> static inline bool
> bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
> {
> - return WARN(mmu_has_feature(MMU_FTR_RADIX_KUAP) &&
> + return WARN(mmu_has_feature(MMU_FTR_KUAP) &&
> (regs->kuap & (is_write ? AMR_KUAP_BLOCK_WRITE : AMR_KUAP_BLOCK_READ)),
> "Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
> }
> diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
> index 255a1837e9f7..04e7a65637fb 100644
> --- a/arch/powerpc/include/asm/mmu.h
> +++ b/arch/powerpc/include/asm/mmu.h
> @@ -29,7 +29,12 @@
> */
>
> /*
> - * Support for KUEP feature.
> + * Supports KUAP (key 0 controlling userspace addresses) on radix
> + */
> +#define MMU_FTR_KUAP ASM_CONST(0x00000200)
> +
> +/*
> + * Suppor for KUEP feature.
Unexpected change I guess. Suppor ==> Support
Christophe
> */
> #define MMU_FTR_KUEP ASM_CONST(0x00000400)
>
> @@ -120,11 +125,6 @@
> */
> #define MMU_FTR_1T_SEGMENT ASM_CONST(0x40000000)
>
> -/*
> - * Supports KUAP (key 0 controlling userspace addresses) on radix
> - */
> -#define MMU_FTR_RADIX_KUAP ASM_CONST(0x80000000)
> -
> /* MMU feature bit sets for various CPUs */
> #define MMU_FTRS_DEFAULT_HPTE_ARCH_V2 \
> MMU_FTR_HPTE_TABLE | MMU_FTR_PPCAS_ARCH_V2
> @@ -187,10 +187,10 @@ enum {
> #ifdef CONFIG_PPC_RADIX_MMU
> MMU_FTR_TYPE_RADIX |
> MMU_FTR_GTSE |
> +#endif /* CONFIG_PPC_RADIX_MMU */
> #ifdef CONFIG_PPC_KUAP
> - MMU_FTR_RADIX_KUAP |
> + MMU_FTR_KUAP |
> #endif /* CONFIG_PPC_KUAP */
> -#endif /* CONFIG_PPC_RADIX_MMU */
> #ifdef CONFIG_PPC_MEM_KEYS
> MMU_FTR_PKEY |
> #endif
> diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
> index 82c722fbce52..bfc27f1f0ab0 100644
> --- a/arch/powerpc/mm/book3s64/pkeys.c
> +++ b/arch/powerpc/mm/book3s64/pkeys.c
> @@ -258,7 +258,7 @@ void __init setup_kuap(bool disabled)
>
> if (smp_processor_id() == boot_cpuid) {
> pr_info("Activating Kernel Userspace Access Prevention\n");
> - cur_cpu_spec->mmu_features |= MMU_FTR_RADIX_KUAP;
> + cur_cpu_spec->mmu_features |= MMU_FTR_KUAP;
> }
>
> /*
>
^ permalink raw reply
* Re: [PATCH v2] powerpc: Update documentation of ISA versions for Power10
From: Christophe Leroy @ 2020-08-27 4:49 UTC (permalink / raw)
To: Jordan Niethe, linuxppc-dev
In-Reply-To: <20200827040556.1783-1-jniethe5@gmail.com>
Le 27/08/2020 à 06:05, Jordan Niethe a écrit :
> Update the CPU to ISA Version Mapping document to include Power10 and
> ISA v3.1.
Maybe Documentation/powerpc/cpu_families.rst should be updated as well.
Christophe
>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v2: Transactional Memory = No
> ---
> Documentation/powerpc/isa-versions.rst | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/Documentation/powerpc/isa-versions.rst b/Documentation/powerpc/isa-versions.rst
> index a363d8c1603c..3873bbba183a 100644
> --- a/Documentation/powerpc/isa-versions.rst
> +++ b/Documentation/powerpc/isa-versions.rst
> @@ -7,6 +7,7 @@ Mapping of some CPU versions to relevant ISA versions.
> ========= ====================================================================
> CPU Architecture version
> ========= ====================================================================
> +Power10 Power ISA v3.1
> Power9 Power ISA v3.0B
> Power8 Power ISA v2.07
> Power7 Power ISA v2.06
> @@ -32,6 +33,7 @@ Key Features
> ========== ==================
> CPU VMX (aka. Altivec)
> ========== ==================
> +Power10 Yes
> Power9 Yes
> Power8 Yes
> Power7 Yes
> @@ -47,6 +49,7 @@ PPC970 Yes
> ========== ====
> CPU VSX
> ========== ====
> +Power10 Yes
> Power9 Yes
> Power8 Yes
> Power7 Yes
> @@ -62,6 +65,7 @@ PPC970 No
> ========== ====================================
> CPU Transactional Memory
> ========== ====================================
> +Power10 No (* see Power ISA v3.1 Appendix A.)
> Power9 Yes (* see transactional_memory.txt)
> Power8 Yes
> Power7 No
>
^ permalink raw reply
* [PATCH v5 23/23] powerpc/book3s64/pkeys: Optimize FTR_KUAP and FTR_KUEP disabled case
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
If FTR_KUAP is disabled kernel will continue to run with the same AMR
value with which it was entered. Hence there is a high chance that
we can return without restoring the AMR value. This also helps the case
when applications are not using the pkey feature. In this case, different
applications will have the same AMR values and hence we can avoid restoring
AMR in this case too.
Also avoid isync() if not really needed.
Do the same for IAMR.
null-syscall benchmark results:
With smap/smep disabled:
Without patch:
957.95 ns 2778.17 cycles
With patch:
858.38 ns 2489.30 cycles
With smap/smep enabled:
Without patch:
1017.26 ns 2950.36 cycles
With patch:
1021.51 ns 2962.44 cycles
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 61 +++++++++++++++++++++---
arch/powerpc/kernel/entry_64.S | 2 +-
arch/powerpc/kernel/syscall_64.c | 12 +++--
3 files changed, 65 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 34a412d2a65b..d71e9a958eb5 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -12,28 +12,54 @@
#ifdef __ASSEMBLY__
-.macro kuap_restore_user_amr gpr1
+.macro kuap_restore_user_amr gpr1, gpr2
#if defined(CONFIG_PPC_PKEY)
BEGIN_MMU_FTR_SECTION_NESTED(67)
+ b 100f // skip_restore_amr
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_PKEY, 67)
/*
* AMR and IAMR are going to be different when
* returning to userspace.
*/
ld \gpr1, STACK_REGS_KUAP(r1)
+
+ /*
+ * If kuap feature is not enabled, do the mtspr
+ * only if AMR value is different.
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(68)
+ mfspr \gpr2, SPRN_AMR
+ cmpd \gpr1, \gpr2
+ beq 99f
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_KUAP, 68)
+
isync
mtspr SPRN_AMR, \gpr1
+99:
/*
* Restore IAMR only when returning to userspace
*/
ld \gpr1, STACK_REGS_KUEP(r1)
+
+ /*
+ * If kuep feature is not enabled, do the mtspr
+ * only if IAMR value is different.
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(69)
+ mfspr \gpr2, SPRN_IAMR
+ cmpd \gpr1, \gpr2
+ beq 100f
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_KUEP, 69)
+
+ isync
mtspr SPRN_IAMR, \gpr1
+100: //skip_restore_amr
/* No isync required, see kuap_restore_user_amr() */
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_PKEY, 67)
#endif
.endm
-.macro kuap_restore_kernel_amr gpr1, gpr2
+.macro kuap_restore_kernel_amr gpr1, gpr2
#if defined(CONFIG_PPC_PKEY)
BEGIN_MMU_FTR_SECTION_NESTED(67)
@@ -190,18 +216,41 @@ static inline u64 current_thread_iamr(void)
static inline void kuap_restore_user_amr(struct pt_regs *regs)
{
+ bool restore_amr = false, restore_iamr = false;
+ unsigned long amr, iamr;
+
if (!mmu_has_feature(MMU_FTR_PKEY))
return;
- isync();
- mtspr(SPRN_AMR, regs->kuap);
- mtspr(SPRN_IAMR, regs->kuep);
+ if (!mmu_has_feature(MMU_FTR_KUAP)) {
+ amr = mfspr(SPRN_AMR);
+ if (amr != regs->kuap)
+ restore_amr = true;
+ } else
+ restore_amr = true;
+
+ if (!mmu_has_feature(MMU_FTR_KUEP)) {
+ iamr = mfspr(SPRN_IAMR);
+ if (iamr != regs->kuep)
+ restore_iamr = true;
+ } else
+ restore_iamr = true;
+
+
+ if (restore_amr || restore_iamr) {
+ isync();
+ if (restore_amr)
+ mtspr(SPRN_AMR, regs->kuap);
+ if (restore_iamr)
+ mtspr(SPRN_IAMR, regs->kuep);
+ }
/*
* No isync required here because we are about to rfi
* back to previous context before any user accesses
* would be made, which is a CSI.
*/
}
+
static inline void kuap_restore_kernel_amr(struct pt_regs *regs,
unsigned long amr)
{
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 68171689db5d..ac6c84a53ff8 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -667,7 +667,7 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return)
bne- .Lrestore_nvgprs
.Lfast_user_interrupt_return_amr:
- kuap_restore_user_amr r3
+ kuap_restore_user_amr r3, r4
.Lfast_user_interrupt_return:
ld r11,_NIP(r1)
ld r12,_MSR(r1)
diff --git a/arch/powerpc/kernel/syscall_64.c b/arch/powerpc/kernel/syscall_64.c
index e49d604b811b..945a14e41898 100644
--- a/arch/powerpc/kernel/syscall_64.c
+++ b/arch/powerpc/kernel/syscall_64.c
@@ -38,6 +38,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
#ifdef CONFIG_PPC_PKEY
if (mmu_has_feature(MMU_FTR_PKEY)) {
unsigned long amr, iamr;
+ bool flush_needed = false;
/*
* When entering from userspace we mostly have the AMR/IAMR
* different from kernel default values. Hence don't compare.
@@ -46,11 +47,16 @@ notrace long system_call_exception(long r3, long r4, long r5,
iamr = mfspr(SPRN_IAMR);
regs->kuap = amr;
regs->kuep = iamr;
- if (mmu_has_feature(MMU_FTR_KUAP))
+ if (mmu_has_feature(MMU_FTR_KUAP)) {
mtspr(SPRN_AMR, AMR_KUAP_BLOCKED);
- if (mmu_has_feature(MMU_FTR_KUEP))
+ flush_needed = true;
+ }
+ if (mmu_has_feature(MMU_FTR_KUEP)) {
mtspr(SPRN_IAMR, AMR_KUEP_BLOCKED);
- isync();
+ flush_needed = true;
+ }
+ if (flush_needed)
+ isync();
} else
#endif
kuap_check_amr();
--
2.26.2
^ permalink raw reply related
* [PATCH v5 22/23] powerpc/book3s64/hash/kup: Don't hardcode kup key
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Make KUAP/KUEP key a variable and also check whether the platform
limit the max key such that we can't use the key for KUAP/KEUP.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
.../powerpc/include/asm/book3s/64/hash-pkey.h | 22 +-------
arch/powerpc/include/asm/book3s/64/pkeys.h | 1 +
arch/powerpc/mm/book3s64/pkeys.c | 53 ++++++++++++++++---
3 files changed, 49 insertions(+), 27 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/hash-pkey.h b/arch/powerpc/include/asm/book3s/64/hash-pkey.h
index 9f44e208f036..ff9907c72ee3 100644
--- a/arch/powerpc/include/asm/book3s/64/hash-pkey.h
+++ b/arch/powerpc/include/asm/book3s/64/hash-pkey.h
@@ -2,9 +2,7 @@
#ifndef _ASM_POWERPC_BOOK3S_64_HASH_PKEY_H
#define _ASM_POWERPC_BOOK3S_64_HASH_PKEY_H
-/* We use key 3 for KERNEL */
-#define HASH_DEFAULT_KERNEL_KEY (HPTE_R_KEY_BIT0 | HPTE_R_KEY_BIT1)
-
+u64 pte_to_hpte_pkey_bits(u64 pteflags, unsigned long flags);
static inline u64 hash__vmflag_to_pte_pkey_bits(u64 vm_flags)
{
return (((vm_flags & VM_PKEY_BIT0) ? H_PTE_PKEY_BIT0 : 0x0UL) |
@@ -14,24 +12,6 @@ static inline u64 hash__vmflag_to_pte_pkey_bits(u64 vm_flags)
((vm_flags & VM_PKEY_BIT4) ? H_PTE_PKEY_BIT4 : 0x0UL));
}
-static inline u64 pte_to_hpte_pkey_bits(u64 pteflags, unsigned long flags)
-{
- unsigned long pte_pkey;
-
- pte_pkey = (((pteflags & H_PTE_PKEY_BIT4) ? HPTE_R_KEY_BIT4 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT3) ? HPTE_R_KEY_BIT3 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT2) ? HPTE_R_KEY_BIT2 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT1) ? HPTE_R_KEY_BIT1 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT0) ? HPTE_R_KEY_BIT0 : 0x0UL));
-
- if (mmu_has_feature(MMU_FTR_KUAP) || mmu_has_feature(MMU_FTR_KUEP)) {
- if ((pte_pkey == 0) && (flags & HPTE_USE_KERNEL_KEY))
- return HASH_DEFAULT_KERNEL_KEY;
- }
-
- return pte_pkey;
-}
-
static inline u16 hash__pte_to_pkey_bits(u64 pteflags)
{
return (((pteflags & H_PTE_PKEY_BIT4) ? 0x10 : 0x0UL) |
diff --git a/arch/powerpc/include/asm/book3s/64/pkeys.h b/arch/powerpc/include/asm/book3s/64/pkeys.h
index 3b8640498f5b..a2b6c4a7275f 100644
--- a/arch/powerpc/include/asm/book3s/64/pkeys.h
+++ b/arch/powerpc/include/asm/book3s/64/pkeys.h
@@ -8,6 +8,7 @@
extern u64 __ro_after_init default_uamor;
extern u64 __ro_after_init default_amr;
extern u64 __ro_after_init default_iamr;
+extern int kup_key;
static inline u64 vmflag_to_pte_pkey_bits(u64 vm_flags)
{
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index b862d5cd78ff..cb1d7d39e801 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -37,7 +37,10 @@ u64 default_uamor __ro_after_init;
*/
static int execute_only_key = 2;
static bool pkey_execute_disable_supported;
-
+/*
+ * key used to implement KUAP/KUEP with hash translation.
+ */
+int kup_key = 3;
#define AMR_BITS_PER_PKEY 2
#define AMR_RD_BIT 0x1UL
@@ -185,6 +188,25 @@ void __init pkey_early_init_devtree(void)
default_uamor &= ~(0x3ul << pkeyshift(execute_only_key));
}
+ if (unlikely(num_pkey <= kup_key)) {
+ /*
+ * Insufficient number of keys to support
+ * KUAP/KUEP feature.
+ */
+ kup_key = -1;
+ } else {
+ /* handle key which is used by kernel for KAUP */
+ reserved_allocation_mask |= (0x1 << kup_key);
+ /*
+ * Mark access for kup_key in default amr so that
+ * we continue to operate with that AMR in
+ * copy_to/from_user().
+ */
+ default_amr &= ~(0x3ul << pkeyshift(kup_key));
+ default_iamr &= ~(0x1ul << pkeyshift(kup_key));
+ default_uamor &= ~(0x3ul << pkeyshift(kup_key));
+ }
+
/*
* Allow access for only key 0. And prevent any other modification.
*/
@@ -205,9 +227,6 @@ void __init pkey_early_init_devtree(void)
reserved_allocation_mask |= (0x1 << 1);
default_uamor &= ~(0x3ul << pkeyshift(1));
- /* handle key 3 which is used by kernel for KAUP */
- reserved_allocation_mask |= (0x1 << 3);
- default_uamor &= ~(0x3ul << pkeyshift(3));
/*
* Prevent the usage of OS reserved keys. Update UAMOR
@@ -236,7 +255,7 @@ void __init pkey_early_init_devtree(void)
#ifdef CONFIG_PPC_KUEP
void __init setup_kuep(bool disabled)
{
- if (disabled)
+ if (disabled || kup_key == -1)
return;
/*
* On hash if PKEY feature is not enabled, disable KUAP too.
@@ -262,7 +281,7 @@ void __init setup_kuep(bool disabled)
#ifdef CONFIG_PPC_KUAP
void __init setup_kuap(bool disabled)
{
- if (disabled)
+ if (disabled || kup_key == -1)
return;
/*
* On hash if PKEY feature is not enabled, disable KUAP too.
@@ -458,4 +477,26 @@ void arch_dup_pkeys(struct mm_struct *oldmm, struct mm_struct *mm)
mm->context.execute_only_pkey = oldmm->context.execute_only_pkey;
}
+u64 pte_to_hpte_pkey_bits(u64 pteflags, unsigned long flags)
+{
+ unsigned long pte_pkey;
+
+ pte_pkey = (((pteflags & H_PTE_PKEY_BIT4) ? HPTE_R_KEY_BIT4 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT3) ? HPTE_R_KEY_BIT3 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT2) ? HPTE_R_KEY_BIT2 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT1) ? HPTE_R_KEY_BIT1 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT0) ? HPTE_R_KEY_BIT0 : 0x0UL));
+
+ if (mmu_has_feature(MMU_FTR_KUAP) || mmu_has_feature(MMU_FTR_KUEP)) {
+ if ((pte_pkey == 0) &&
+ (flags & HPTE_USE_KERNEL_KEY) && (kup_key != -1)) {
+ u64 vm_flag = pkey_to_vmflag_bits(kup_key);
+ u64 pte_flag = hash__vmflag_to_pte_pkey_bits(vm_flag);
+ return pte_to_hpte_pkey_bits(pte_flag, 0);
+ }
+ }
+
+ return pte_pkey;
+}
+
#endif /* CONFIG_PPC_MEM_KEYS */
--
2.26.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox