* Re: [PATCHv5 2/2] powerpc/pseries: update device tree before ejecting hotplug uevents
From: Pingfan Liu @ 2020-08-27 6:36 UTC (permalink / raw)
To: linuxppc-dev
Cc: Nathan Lynch, Kexec Mailing List, Nathan Fontenot, Laurent Dufour,
Hari Bathini
In-Reply-To: <1597049570-19536-2-git-send-email-kernelfans@gmail.com>
Hello guys. Do you have further comments on this version?
Thanks,
Pingfan
On Mon, Aug 10, 2020 at 4:53 PM Pingfan Liu <kernelfans@gmail.com> wrote:
>
> A bug is observed on pseries by taking the following steps on rhel:
> -1. drmgr -c mem -r -q 5
> -2. echo c > /proc/sysrq-trigger
>
> And then, the failure looks like:
> kdump: saving to /sysroot//var/crash/127.0.0.1-2020-01-16-02:06:14/
> kdump: saving vmcore-dmesg.txt
> kdump: saving vmcore-dmesg.txt complete
> kdump: saving vmcore
> Checking for memory holes : [ 0.0 %] / Checking for memory holes : [100.0 %] | Excluding unnecessary pages : [100.0 %] \ Copying data : [ 0.3 %] - eta: 38s[ 44.337636] hash-mmu: mm: Hashing failure ! EA=0x7fffba400000 access=0x8000000000000004 current=makedumpfile
> [ 44.337663] hash-mmu: trap=0x300 vsid=0x13a109c ssize=1 base psize=2 psize 2 pte=0xc000000050000504
> [ 44.337677] hash-mmu: mm: Hashing failure ! EA=0x7fffba400000 access=0x8000000000000004 current=makedumpfile
> [ 44.337692] hash-mmu: trap=0x300 vsid=0x13a109c ssize=1 base psize=2 psize 2 pte=0xc000000050000504
> [ 44.337708] makedumpfile[469]: unhandled signal 7 at 00007fffba400000 nip 00007fffbbc4d7fc lr 000000011356ca3c code 2
> [ 44.338548] Core dump to |/bin/false pipe failed
> /lib/kdump-lib-initramfs.sh: line 98: 469 Bus error $CORE_COLLECTOR /proc/vmcore $_mp/$KDUMP_PATH/$HOST_IP-$DATEDIR/vmcore-incomplete
> kdump: saving vmcore failed
>
> * Root cause *
> After analyzing, it turns out that in the current implementation,
> when hot-removing lmb, the KOBJ_REMOVE event ejects before the dt updating as
> the code __remove_memory() comes before drmem_update_dt().
> So in kdump kernel, when read_from_oldmem() resorts to
> pSeries_lpar_hpte_insert() to install hpte, but fails with -2 due to
> non-exist pfn. And finally, low_hash_fault() raise SIGBUS to process, as it
> can be observed "Bus error"
>
> From a viewpoint of listener and publisher, the publisher notifies the
> listener before data is ready. This introduces a problem where udev
> launches kexec-tools (due to KOBJ_REMOVE) and loads a stale dt before
> updating. And in capture kernel, makedumpfile will access the memory based
> on the stale dt info, and hit a SIGBUS error due to an un-existed lmb.
>
> * Fix *
> This bug is introduced by commit 063b8b1251fd
> ("powerpc/pseries/memory-hotplug: Only update DT once per memory DLPAR
> request"), which tried to combine all the dt updating into one.
>
> To fix this issue, meanwhile not to introduce a quadratic runtime
> complexity by the model:
> dlpar_memory_add_by_count
> for_each_drmem_lmb <--
> dlpar_add_lmb
> drmem_update_dt(_v1|_v2)
> for_each_drmem_lmb <--
> The dt should still be only updated once, and just before the last memory
> online/offline event is ejected to user space. Achieve this by tracing the
> num of lmb added or removed.
>
> Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Nathan Lynch <nathanl@linux.ibm.com>
> Cc: Nathan Fontenot <nfont@linux.vnet.ibm.com>
> Cc: Laurent Dufour <ldufour@linux.ibm.com>
> To: linuxppc-dev@lists.ozlabs.org
> Cc: kexec@lists.infradead.org
> ---
> v4 -> v5: change dlpar_add_lmb()/dlpar_remove_lmb() prototype to report
> whether dt is updated successfully.
> Fix a condition boundary check bug
> v3 -> v4: resolve a quadratic runtime complexity issue.
> This series is applied on next-test branch
> arch/powerpc/platforms/pseries/hotplug-memory.c | 102 +++++++++++++++++++-----
> 1 file changed, 80 insertions(+), 22 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
> index 46cbcd1..1567d9f 100644
> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c
> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
> @@ -350,13 +350,22 @@ static bool lmb_is_removable(struct drmem_lmb *lmb)
> return true;
> }
>
> -static int dlpar_add_lmb(struct drmem_lmb *);
> +enum dt_update_status {
> + DT_NOUPDATE,
> + DT_TOUPDATE,
> + DT_UPDATED,
> +};
> +
> +/* "*dt_update" returns DT_UPDATED if updated */
> +static int dlpar_add_lmb(struct drmem_lmb *lmb,
> + enum dt_update_status *dt_update);
>
> -static int dlpar_remove_lmb(struct drmem_lmb *lmb)
> +static int dlpar_remove_lmb(struct drmem_lmb *lmb,
> + enum dt_update_status *dt_update)
> {
> unsigned long block_sz;
> phys_addr_t base_addr;
> - int rc, nid;
> + int rc, ret, nid;
>
> if (!lmb_is_removable(lmb))
> return -EINVAL;
> @@ -372,6 +381,13 @@ static int dlpar_remove_lmb(struct drmem_lmb *lmb)
> invalidate_lmb_associativity_index(lmb);
> lmb_clear_nid(lmb);
> lmb->flags &= ~DRCONF_MEM_ASSIGNED;
> + if (*dt_update) {
> + ret = drmem_update_dt();
> + if (ret)
> + pr_warn("%s fail to update dt, but continue\n", __func__);
> + else
> + *dt_update = DT_UPDATED;
> + }
>
> __remove_memory(nid, base_addr, block_sz);
>
> @@ -387,6 +403,7 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
> int lmbs_removed = 0;
> int lmbs_available = 0;
> int rc;
> + enum dt_update_status dt_update = DT_NOUPDATE;
>
> pr_info("Attempting to hot-remove %d LMB(s)\n", lmbs_to_remove);
>
> @@ -409,7 +426,11 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
> }
>
> for_each_drmem_lmb(lmb) {
> - rc = dlpar_remove_lmb(lmb);
> +
> + /* combine dt updating for all LMBs */
> + if (lmbs_to_remove - lmbs_removed <= 1)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_remove_lmb(lmb, &dt_update);
> if (rc)
> continue;
>
> @@ -424,13 +445,17 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
> }
>
> if (lmbs_removed != lmbs_to_remove) {
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
> +
> pr_err("Memory hot-remove failed, adding LMB's back\n");
>
> for_each_drmem_lmb(lmb) {
> if (!drmem_lmb_reserved(lmb))
> continue;
>
> - rc = dlpar_add_lmb(lmb);
> + if (--lmbs_removed == 0 && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to add LMB back, drc index %x\n",
> lmb->drc_index);
> @@ -458,6 +483,7 @@ static int dlpar_memory_remove_by_count(u32 lmbs_to_remove)
>
> static int dlpar_memory_remove_by_index(u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_TOUPDATE;
> struct drmem_lmb *lmb;
> int lmb_found;
> int rc;
> @@ -468,7 +494,7 @@ static int dlpar_memory_remove_by_index(u32 drc_index)
> for_each_drmem_lmb(lmb) {
> if (lmb->drc_index == drc_index) {
> lmb_found = 1;
> - rc = dlpar_remove_lmb(lmb);
> + rc = dlpar_remove_lmb(lmb, &dt_update);
> if (!rc)
> dlpar_release_drc(lmb->drc_index);
>
> @@ -490,6 +516,7 @@ static int dlpar_memory_remove_by_index(u32 drc_index)
>
> static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_NOUPDATE;
> struct drmem_lmb *lmb, *start_lmb, *end_lmb;
> int lmbs_available = 0;
> int rc;
> @@ -519,7 +546,9 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> if (!(lmb->flags & DRCONF_MEM_ASSIGNED))
> continue;
>
> - rc = dlpar_remove_lmb(lmb);
> + if (lmb == end_lmb)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_remove_lmb(lmb, &dt_update);
> if (rc)
> break;
>
> @@ -527,14 +556,16 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> }
>
> if (rc) {
> - pr_err("Memory indexed-count-remove failed, adding any removed LMBs\n");
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
>
> + pr_err("Memory indexed-count-remove failed, adding any removed LMBs\n");
>
> for_each_drmem_lmb_in_range(lmb, start_lmb, end_lmb) {
> if (!drmem_lmb_reserved(lmb))
> continue;
> -
> - rc = dlpar_add_lmb(lmb);
> + if (lmb == end_lmb && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to add LMB, drc index %x\n",
> lmb->drc_index);
> @@ -572,7 +603,7 @@ static inline int dlpar_memory_remove(struct pseries_hp_errorlog *hp_elog)
> {
> return -EOPNOTSUPP;
> }
> -static int dlpar_remove_lmb(struct drmem_lmb *lmb)
> +static int dlpar_remove_lmb(struct drmem_lmb *lmb, bool dt_update)
> {
> return -EOPNOTSUPP;
> }
> @@ -591,10 +622,11 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
> }
> #endif /* CONFIG_MEMORY_HOTREMOVE */
>
> -static int dlpar_add_lmb(struct drmem_lmb *lmb)
> +static int dlpar_add_lmb(struct drmem_lmb *lmb,
> + enum dt_update_status *dt_update)
> {
> unsigned long block_sz;
> - int rc;
> + int rc, ret;
>
> if (lmb->flags & DRCONF_MEM_ASSIGNED)
> return -EINVAL;
> @@ -607,6 +639,13 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
>
> lmb_set_nid(lmb);
> lmb->flags |= DRCONF_MEM_ASSIGNED;
> + if (*dt_update) {
> + ret = drmem_update_dt();
> + if (ret)
> + pr_warn("%s fail to update dt, but continue\n", __func__);
> + else
> + *dt_update = DT_UPDATED;
> + }
>
> block_sz = memory_block_size_bytes();
>
> @@ -616,6 +655,8 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
> invalidate_lmb_associativity_index(lmb);
> lmb_clear_nid(lmb);
> lmb->flags &= ~DRCONF_MEM_ASSIGNED;
> + if (*dt_update == DT_UPDATED)
> + drmem_update_dt();
> return rc;
> }
>
> @@ -627,7 +668,11 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
> invalidate_lmb_associativity_index(lmb);
> lmb_clear_nid(lmb);
> lmb->flags &= ~DRCONF_MEM_ASSIGNED;
> -
> + if (*dt_update == DT_UPDATED) {
> + ret = drmem_update_dt();
> + if (ret)
> + pr_warn("%s fail to update dt during rollback, but continue\n", __func__);
> + }
> __remove_memory(nid, base_addr, block_sz);
> }
>
> @@ -636,6 +681,7 @@ static int dlpar_add_lmb(struct drmem_lmb *lmb)
>
> static int dlpar_memory_add_by_count(u32 lmbs_to_add)
> {
> + enum dt_update_status dt_update = DT_NOUPDATE;
> struct drmem_lmb *lmb;
> int lmbs_available = 0;
> int lmbs_added = 0;
> @@ -666,7 +712,9 @@ static int dlpar_memory_add_by_count(u32 lmbs_to_add)
> if (rc)
> continue;
>
> - rc = dlpar_add_lmb(lmb);
> + if (lmbs_to_add - lmbs_added <= 1)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &dt_update);
> if (rc) {
> dlpar_release_drc(lmb->drc_index);
> continue;
> @@ -683,13 +731,18 @@ static int dlpar_memory_add_by_count(u32 lmbs_to_add)
> }
>
> if (lmbs_added != lmbs_to_add) {
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
> +
> pr_err("Memory hot-add failed, removing any added LMBs\n");
>
> for_each_drmem_lmb(lmb) {
> if (!drmem_lmb_reserved(lmb))
> continue;
>
> - rc = dlpar_remove_lmb(lmb);
> + if (--lmbs_added == 0 && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> +
> + rc = dlpar_remove_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to remove LMB, drc index %x\n",
> lmb->drc_index);
> @@ -716,6 +769,7 @@ static int dlpar_memory_add_by_count(u32 lmbs_to_add)
>
> static int dlpar_memory_add_by_index(u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_TOUPDATE;
> struct drmem_lmb *lmb;
> int rc, lmb_found;
>
> @@ -727,7 +781,7 @@ static int dlpar_memory_add_by_index(u32 drc_index)
> lmb_found = 1;
> rc = dlpar_acquire_drc(lmb->drc_index);
> if (!rc) {
> - rc = dlpar_add_lmb(lmb);
> + rc = dlpar_add_lmb(lmb, &dt_update);
> if (rc)
> dlpar_release_drc(lmb->drc_index);
> }
> @@ -750,6 +804,7 @@ static int dlpar_memory_add_by_index(u32 drc_index)
>
> static int dlpar_memory_add_by_ic(u32 lmbs_to_add, u32 drc_index)
> {
> + enum dt_update_status dt_update = DT_NOUPDATE;
> struct drmem_lmb *lmb, *start_lmb, *end_lmb;
> int lmbs_available = 0;
> int rc;
> @@ -783,7 +838,9 @@ static int dlpar_memory_add_by_ic(u32 lmbs_to_add, u32 drc_index)
> if (rc)
> break;
>
> - rc = dlpar_add_lmb(lmb);
> + if (lmb == end_lmb)
> + dt_update = DT_TOUPDATE;
> + rc = dlpar_add_lmb(lmb, &dt_update);
> if (rc) {
> dlpar_release_drc(lmb->drc_index);
> break;
> @@ -796,10 +853,14 @@ static int dlpar_memory_add_by_ic(u32 lmbs_to_add, u32 drc_index)
> pr_err("Memory indexed-count-add failed, removing any added LMBs\n");
>
> for_each_drmem_lmb_in_range(lmb, start_lmb, end_lmb) {
> + enum dt_update_status rollback_dt_update = DT_NOUPDATE;
> +
> if (!drmem_lmb_reserved(lmb))
> continue;
>
> - rc = dlpar_remove_lmb(lmb);
> + if (lmb == end_lmb && dt_update == DT_UPDATED)
> + rollback_dt_update = DT_TOUPDATE;
> + rc = dlpar_remove_lmb(lmb, &rollback_dt_update);
> if (rc)
> pr_err("Failed to remove LMB, drc index %x\n",
> lmb->drc_index);
> @@ -879,9 +940,6 @@ int dlpar_memory(struct pseries_hp_errorlog *hp_elog)
> break;
> }
>
> - if (!rc)
> - rc = drmem_update_dt();
> -
> unlock_device_hotplug();
> return rc;
> }
> --
> 2.7.5
>
^ permalink raw reply
* [PATCH v3 0/6] Use per-CPU temporary mappings for patching
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
When compiled with CONFIG_STRICT_KERNEL_RWX, the kernel must create
temporary mappings when patching itself. These mappings temporarily
override the strict RWX text protections to permit a write. Currently,
powerpc allocates a per-CPU VM area for patching. Patching occurs as
follows:
1. Map page of text to be patched to per-CPU VM area w/
PAGE_KERNEL protection
2. Patch text
3. Remove the temporary mapping
While the VM area is per-CPU, the mapping is actually inserted into the
kernel page tables. Presumably, this could allow another CPU to access
the normally write-protected text - either malicously or accidentally -
via this same mapping if the address of the VM area is known. Ideally,
the mapping should be kept local to the CPU doing the patching (or any
other sensitive operations requiring temporarily overriding memory
protections) [0].
x86 introduced "temporary mm" structs which allow the creation of
mappings local to a particular CPU [1]. This series intends to bring the
notion of a temporary mm to powerpc and harden powerpc by using such a
mapping for patching a kernel with strict RWX permissions.
The first, second, and third patches implement an LKDTM test
"proof-of-concept" which exploits the potential vulnerability (ie. the
mapping during patching is exposed in kernel page tables and accessible
by other CPUS). The LKDTM test is somewhat "rough" in that it uses a
brute-force approach - I am open to any suggestions and/or ideas to
improve this. Currently, the LKDTM test passes with this series on
POWER8 (hash) and POWER9 (radix, hash) and fails without this series
(ie. the temporary mapping for patching is exposed to CPUs other than
the patching CPU).
The test is also implemented on x86_64 where it passes with a current
tree and fails only when using a tree prior to the temporary mappings. I
have such a tree here which intentionally fails:
https://github.com/cmr-informatik/linux/tree/x86_64-non-percpu-lkdtm
The fourth patch introduces the temporary mm struct and API for powerpc
along with a new function to retrieve a current hw breakpoint.
The fifth patch uses the `poking_init` init hook added by the x86
patches to initialize a temporary mm and patching address. The patching
address is randomized between PAGE_SIZE and DEFAULT_MAP_WINDOW-PAGE_SIZE.
The upper limit is necessary due to how the hash MMU operates - by default
the space above DEFAULT_MAP_WINDOW is not available. For now, both hash
and radix randomize inside this range. The number of possible random
addresses is dependent on PAGE_SIZE and limited by DEFAULT_MAP_WINDOW.
Bits of entropy with 64K page size on BOOK3S_64:
bits of entropy = log2(DEFAULT_MAP_WINDOW_USER64 / PAGE_SIZE)
PAGE_SIZE=64K, DEFAULT_MAP_WINDOW_USER64=128TB
bits of entropy = log2(128TB / 64K)
bits of entropy = 31
Randomization occurs only once during initialization at boot.
The sixth patch replaces the VM area with the temporary mm in the
patching code. The page for patching has to be mapped PAGE_SHARED with
the hash MMU since hash prevents the kernel from accessing userspace
pages with PAGE_PRIVILEGED bit set. On the radix MMU the page is mapped
with PAGE_KERNEL which has the added benefit that we can skip KUAP.
Tested on Blackbird (8-core POWER9) w/ Hash (`disable_radix`) and Radix
MMUs, QEMU (TCG) POWER8 and POWER9, POWER8 VM.
Tested LKDTM test (passing and failing situations) on QEMU x86_64.
v3: * Rebase on linuxppc/next:
commit 9123e3a74ec7 ("Linux 5.9-rc1")
* Move temporary mm implementation into code-patching.c where it
belongs
* Implement LKDTM hijacker test on x86_64 (on IBM time oof)
* Do not use address zero for the patching address in the
temporary mm (thanks @dja for pointing this out!)
* Wrap the LKDTM test w/ CONFIG_SMP as suggested by Christophe
Leroy
* Comments to clarify PTE pre-allocation and patching addr
selection
v2: * Rebase on linuxppc/next:
commit 105fb38124a4 ("powerpc/8xx: Modify ptep_get()")
* Always dirty pte when mapping patch
* Use `ppc_inst_len` instead of `sizeof` on instructions
* Declare LKDTM patching addr accessor in header where it
belongs
v1: * Rebase on linuxppc/next (4336b9337824)
* Save and restore second hw watchpoint
* Use new ppc_inst_* functions for patching check and in LKDTM
test
rfc-v2: * Many fixes and improvements mostly based on extensive feedback and
testing by Christophe Leroy (thanks!).
* Make patching_mm and patching_addr static and mode '__ro_after_init'
to after the variable name (more common in other parts of the kernel)
* Use 'asm/debug.h' header instead of 'asm/hw_breakpoint.h' to fix
PPC64e compile
* Add comment explaining why we use BUG_ON() during the init call to
setup for patching later
* Move ptep into patch_mapping to avoid walking page tables a second
time when unmapping the temporary mapping
* Use KUAP under non-radix, also manually dirty the PTE for patch
mapping on non-BOOK3S_64 platforms
* Properly return any error from __patch_instruction
* Do not use 'memcmp' where a simple comparison is appropriate
* Simplify expression for patch address by removing pointer maths
* Add LKDTM test
[0]: https://github.com/linuxppc/issues/issues/224
[1]: https://lore.kernel.org/kernel-hardening/20190426232303.28381-1-nadav.amit@gmail.com/
Christopher M. Riedl (6):
powerpc: Add LKDTM accessor for patching addr
x86: Add LKDTM accessor for patching addr
Add LKDTM test to hijack a patch mapping (powerpc,x86_64)
powerpc: Introduce temporary mm
powerpc: Initialize a temporary mm for code patching
powerpc: Use a temporary mm for code patching
arch/powerpc/include/asm/code-patching.h | 4 +
arch/powerpc/include/asm/debug.h | 1 +
arch/powerpc/kernel/process.c | 5 +
arch/powerpc/lib/code-patching.c | 239 +++++++++++++++--------
arch/x86/include/asm/text-patching.h | 4 +
arch/x86/kernel/alternative.c | 7 +
drivers/misc/lkdtm/core.c | 1 +
drivers/misc/lkdtm/lkdtm.h | 1 +
drivers/misc/lkdtm/perms.c | 146 ++++++++++++++
9 files changed, 322 insertions(+), 86 deletions(-)
--
2.28.0
^ permalink raw reply
* Re: [PATCH v2] powerpc: Update documentation of ISA versions for Power10
From: Michael Ellerman @ 2020-08-27 6:18 UTC (permalink / raw)
To: Jordan Niethe, linuxppc-dev; +Cc: Jordan Niethe
In-Reply-To: <20200827040556.1783-1-jniethe5@gmail.com>
Jordan Niethe <jniethe5@gmail.com> writes:
> Update the CPU to ISA Version Mapping document to include Power10 and
> ISA v3.1.
>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v2: Transactional Memory = No
> ---
> Documentation/powerpc/isa-versions.rst | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/Documentation/powerpc/isa-versions.rst b/Documentation/powerpc/isa-versions.rst
> index a363d8c1603c..3873bbba183a 100644
> --- a/Documentation/powerpc/isa-versions.rst
> +++ b/Documentation/powerpc/isa-versions.rst
> @@ -62,6 +65,7 @@ PPC970 No
> ========== ====================================
> CPU Transactional Memory
> ========== ====================================
> +Power10 No (* see Power ISA v3.1 Appendix A.)
There's three "Appendix A"s in ISA v3.1.
There's one in Book I, and one in Book II.
And then the one you're referring to is not actually in Book III, it's
listed after Book III, and is apparently an appendix to all three books?
Which is just utterly confusing.
So I'll change it to say:
"Appendix A. Notes on the Removal of Transactional Memory from the Architecture"
Which is very long, but at least you can search for it.
cheers
^ permalink raw reply
* [PATCH v3 2/6] x86: Add LKDTM accessor for patching addr
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
When live patching a STRICT_RWX kernel, a mapping is installed at a
"patching address" with temporary write permissions. Provide a
LKDTM-only accessor function for this address in preparation for a LKDTM
test which attempts to "hijack" this mapping by writing to it from
another CPU.
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/x86/include/asm/text-patching.h | 4 ++++
arch/x86/kernel/alternative.c | 7 +++++++
2 files changed, 11 insertions(+)
diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h
index 6593b42cb379..f67b4dd30bf8 100644
--- a/arch/x86/include/asm/text-patching.h
+++ b/arch/x86/include/asm/text-patching.h
@@ -148,4 +148,8 @@ void int3_emulate_call(struct pt_regs *regs, unsigned long func)
}
#endif /* !CONFIG_UML_X86 */
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu);
+#endif
+
#endif /* _ASM_X86_TEXT_PATCHING_H */
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index c3daf0aaa0ee..c16833f35a1f 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -843,6 +843,13 @@ static inline void unuse_temporary_mm(temp_mm_state_t prev_state)
__ro_after_init struct mm_struct *poking_mm;
__ro_after_init unsigned long poking_addr;
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu)
+{
+ return poking_addr;
+}
+#endif
+
static void *__text_poke(void *addr, const void *opcode, size_t len)
{
bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
--
2.28.0
^ permalink raw reply related
* Re: [PATCH] Revert "powerpc/powernv/idle: Replace CPU feature check with PVR check"
From: Michael Ellerman @ 2020-08-27 6:07 UTC (permalink / raw)
To: Pratik Sampat, Christophe Leroy, linuxppc-dev, npiggin, mikey,
ego, svaidy, linux-kernel, pratik.r.sampat
In-Reply-To: <170e1919-bc45-6b99-dc4d-713418c98be1@linux.ibm.com>
Pratik Sampat <psampat@linux.ibm.com> writes:
> On 26/08/20 2:07 pm, Christophe Leroy wrote:
>> Le 26/08/2020 à 10:29, Pratik Rajesh Sampat a écrit :
>>> Cpuidle stop state implementation has minor optimizations for P10
>>> where hardware preserves more SPR registers compared to P9.
>>> The current P9 driver works for P10, although does few extra
>>> save-restores. P9 driver can provide the required power management
>>> features like SMT thread folding and core level power savings
>>> on a P10 platform.
>>>
>>> Until the P10 stop driver is available, revert the commit which
>>> allows for only P9 systems to utilize cpuidle and blocks all
>>> idle stop states for P10.
>>> Cpu idle states are enabled and tested on the P10 platform
>>> with this fix.
>>>
>>> This reverts commit 8747bf36f312356f8a295a0c39ff092d65ce75ae.
>>>
>>> Fixes: 8747bf36f312 ("powerpc/powernv/idle: Replace CPU feature check
>>> with PVR check")
>>> Signed-off-by: Pratik Rajesh Sampat <psampat@linux.ibm.com>
>>> ---
>>> @mpe: This revert would resolve a staging issue wherein the P10 stop
>>> driver is not yet ready while cpuidle stop states need not be blocked
>>> on 5.9 for Power10 systems which could cause SMT folding related
>>> performance issues.
>>>
>>> The P10 stop driver is in the works here:
>>> https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216773.html
>>>
>>> arch/powerpc/platforms/powernv/idle.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/platforms/powernv/idle.c
>>> b/arch/powerpc/platforms/powernv/idle.c
>>> index 77513a80cef9..345ab062b21a 100644
>>> --- a/arch/powerpc/platforms/powernv/idle.c
>>> +++ b/arch/powerpc/platforms/powernv/idle.c
>>> @@ -1223,7 +1223,7 @@ static void __init pnv_probe_idle_states(void)
>>> return;
>>> }
>>> - if (pvr_version_is(PVR_POWER9))
>>> + if (cpu_has_feature(CPU_FTR_ARCH_300))
>>
>> Why not something like:
>>
>> if (pvr_version_is(PVR_POWER9) || pvr_version_is(PVR_POWER10))
>> pnv_power9_idle_init();
>
> In order to use PVR_POWER10 I would need to define it under
> arch/powerpc/include/asm/reg.h, which is not present in 5.9 yet.
>
> However, if it okay with @mpe I could split out Nick's P10 stop driver
> (https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216773.html)
> into two parts:
I'll just take this for now, it's the simplest option.
cheers
^ permalink raw reply
* [PATCH v3 6/6] powerpc: Use a temporary mm for code patching
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
Currently, code patching a STRICT_KERNEL_RWX exposes the temporary
mappings to other CPUs. These mappings should be kept local to the CPU
doing the patching. Use the pre-initialized temporary mm and patching
address for this purpose. Also add a check after patching to ensure the
patch succeeded.
Toggle KUAP on non-radix MMU platforms when patching since the temporary
mapping for patching uses a userspace address. With a radix MMU this is
not required since mapping a page with PAGE_KERNEL sets EAA[0] for the
PTE which means the AMR (KUAP) protection is ignored (see PowerISA
v3.0b, Fig. 35).
Based on x86 implementation:
commit b3fd8e83ada0
("x86/alternatives: Use temporary mm for text poking")
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/powerpc/lib/code-patching.c | 153 +++++++++++--------------------
1 file changed, 54 insertions(+), 99 deletions(-)
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 051d7ae6d8ee..9fb098680da8 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -149,113 +149,64 @@ void __init poking_init(void)
pte_unmap_unlock(ptep, ptl);
}
-static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
-
#ifdef CONFIG_LKDTM
unsigned long read_cpu_patching_addr(unsigned int cpu)
{
- return (unsigned long)(per_cpu(text_poke_area, cpu))->addr;
+ return patching_addr;
}
#endif
-static int text_area_cpu_up(unsigned int cpu)
-{
- struct vm_struct *area;
-
- area = get_vm_area(PAGE_SIZE, VM_ALLOC);
- if (!area) {
- WARN_ONCE(1, "Failed to create text area for cpu %d\n",
- cpu);
- return -1;
- }
- this_cpu_write(text_poke_area, area);
-
- return 0;
-}
-
-static int text_area_cpu_down(unsigned int cpu)
-{
- free_vm_area(this_cpu_read(text_poke_area));
- return 0;
-}
-
-/*
- * Run as a late init call. This allows all the boot time patching to be done
- * simply by patching the code, and then we're called here prior to
- * mark_rodata_ro(), which happens after all init calls are run. Although
- * BUG_ON() is rude, in this case it should only happen if ENOMEM, and we judge
- * it as being preferable to a kernel that will crash later when someone tries
- * to use patch_instruction().
- */
-static int __init setup_text_poke_area(void)
-{
- BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
- "powerpc/text_poke:online", text_area_cpu_up,
- text_area_cpu_down));
-
- return 0;
-}
-late_initcall(setup_text_poke_area);
+struct patch_mapping {
+ spinlock_t *ptl; /* for protecting pte table */
+ pte_t *ptep;
+ struct temp_mm temp_mm;
+};
/*
* This can be called for kernel text or a module.
*/
-static int map_patch_area(void *addr, unsigned long text_poke_addr)
+static int map_patch(const void *addr, struct patch_mapping *patch_mapping)
{
- unsigned long pfn;
- int err;
+ struct page *page;
+ pte_t pte;
+ pgprot_t pgprot;
if (is_vmalloc_or_module_addr(addr))
- pfn = vmalloc_to_pfn(addr);
+ page = vmalloc_to_page(addr);
else
- pfn = __pa_symbol(addr) >> PAGE_SHIFT;
+ page = virt_to_page(addr);
- err = map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL);
+ if (radix_enabled())
+ pgprot = PAGE_KERNEL;
+ else
+ pgprot = PAGE_SHARED;
- pr_devel("Mapped addr %lx with pfn %lx:%d\n", text_poke_addr, pfn, err);
- if (err)
+ patch_mapping->ptep = get_locked_pte(patching_mm, patching_addr,
+ &patch_mapping->ptl);
+ if (unlikely(!patch_mapping->ptep)) {
+ pr_warn("map patch: failed to allocate pte for patching\n");
return -1;
+ }
+
+ pte = mk_pte(page, pgprot);
+ pte = pte_mkdirty(pte);
+ set_pte_at(patching_mm, patching_addr, patch_mapping->ptep, pte);
+
+ init_temp_mm(&patch_mapping->temp_mm, patching_mm);
+ use_temporary_mm(&patch_mapping->temp_mm);
return 0;
}
-static inline int unmap_patch_area(unsigned long addr)
+static void unmap_patch(struct patch_mapping *patch_mapping)
{
- pte_t *ptep;
- pmd_t *pmdp;
- pud_t *pudp;
- p4d_t *p4dp;
- pgd_t *pgdp;
-
- pgdp = pgd_offset_k(addr);
- if (unlikely(!pgdp))
- return -EINVAL;
-
- p4dp = p4d_offset(pgdp, addr);
- if (unlikely(!p4dp))
- return -EINVAL;
-
- pudp = pud_offset(p4dp, addr);
- if (unlikely(!pudp))
- return -EINVAL;
-
- pmdp = pmd_offset(pudp, addr);
- if (unlikely(!pmdp))
- return -EINVAL;
+ /* In hash, pte_clear flushes the tlb */
+ pte_clear(patching_mm, patching_addr, patch_mapping->ptep);
+ unuse_temporary_mm(&patch_mapping->temp_mm);
- ptep = pte_offset_kernel(pmdp, addr);
- if (unlikely(!ptep))
- return -EINVAL;
-
- pr_devel("clearing mm %p, pte %p, addr %lx\n", &init_mm, ptep, addr);
-
- /*
- * In hash, pte_clear flushes the tlb, in radix, we have to
- */
- pte_clear(&init_mm, addr, ptep);
- flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
-
- return 0;
+ /* In radix, we have to explicitly flush the tlb (no-op in hash) */
+ local_flush_tlb_mm(patching_mm);
+ pte_unmap_unlock(patch_mapping->ptep, patch_mapping->ptl);
}
static int do_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
@@ -263,32 +214,36 @@ static int do_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
int err;
struct ppc_inst *patch_addr = NULL;
unsigned long flags;
- unsigned long text_poke_addr;
- unsigned long kaddr = (unsigned long)addr;
+ struct patch_mapping patch_mapping;
/*
- * During early early boot patch_instruction is called
- * when text_poke_area is not ready, but we still need
- * to allow patching. We just do the plain old patching
+ * The patching_mm is initialized before calling mark_rodata_ro. Prior
+ * to this, patch_instruction is called when we don't have (and don't
+ * need) the patching_mm so just do plain old patching.
*/
- if (!this_cpu_read(text_poke_area))
+ if (!patching_mm)
return raw_patch_instruction(addr, instr);
local_irq_save(flags);
- text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr;
- if (map_patch_area(addr, text_poke_addr)) {
- err = -1;
+ err = map_patch(addr, &patch_mapping);
+ if (err)
goto out;
- }
- patch_addr = (struct ppc_inst *)(text_poke_addr + (kaddr & ~PAGE_MASK));
+ patch_addr = (struct ppc_inst *)(patching_addr | offset_in_page(addr));
- __patch_instruction(addr, instr, patch_addr);
+ if (!radix_enabled())
+ allow_write_to_user(patch_addr, ppc_inst_len(instr));
+ err = __patch_instruction(addr, instr, patch_addr);
+ if (!radix_enabled())
+ prevent_write_to_user(patch_addr, ppc_inst_len(instr));
- err = unmap_patch_area(text_poke_addr);
- if (err)
- pr_warn("failed to unmap %lx\n", text_poke_addr);
+ unmap_patch(&patch_mapping);
+ /*
+ * Something is wrong if what we just wrote doesn't match what we
+ * think we just wrote.
+ */
+ WARN_ON(!ppc_inst_equal(ppc_inst_read(addr), instr));
out:
local_irq_restore(flags);
--
2.28.0
^ permalink raw reply related
* [PATCH v3 1/6] powerpc: Add LKDTM accessor for patching addr
From: Christopher M. Riedl @ 2020-08-27 5:26 UTC (permalink / raw)
To: linuxppc-dev; +Cc: kernel-hardening
In-Reply-To: <20200827052659.24922-1-cmr@codefail.de>
When live patching a STRICT_RWX kernel, a mapping is installed at a
"patching address" with temporary write permissions. Provide a
LKDTM-only accessor function for this address in preparation for a LKDTM
test which attempts to "hijack" this mapping by writing to it from
another CPU.
Signed-off-by: Christopher M. Riedl <cmr@codefail.de>
---
arch/powerpc/include/asm/code-patching.h | 4 ++++
arch/powerpc/lib/code-patching.c | 7 +++++++
2 files changed, 11 insertions(+)
diff --git a/arch/powerpc/include/asm/code-patching.h b/arch/powerpc/include/asm/code-patching.h
index eacc9102c251..7216d6a6bb0a 100644
--- a/arch/powerpc/include/asm/code-patching.h
+++ b/arch/powerpc/include/asm/code-patching.h
@@ -187,4 +187,8 @@ static inline unsigned long ppc_kallsyms_lookup_name(const char *name)
___PPC_RA(__REG_R1) | PPC_LR_STKOFF)
#endif /* CONFIG_PPC64 */
+#if defined(CONFIG_LKDTM) && defined(CONFIG_STRICT_KERNEL_RWX)
+unsigned long read_cpu_patching_addr(unsigned int cpu);
+#endif
+
#endif /* _ASM_POWERPC_CODE_PATCHING_H */
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 8c3934ea6220..85d3fdca9452 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -46,6 +46,13 @@ int raw_patch_instruction(struct ppc_inst *addr, struct ppc_inst instr)
#ifdef CONFIG_STRICT_KERNEL_RWX
static DEFINE_PER_CPU(struct vm_struct *, text_poke_area);
+#ifdef CONFIG_LKDTM
+unsigned long read_cpu_patching_addr(unsigned int cpu)
+{
+ return (unsigned long)(per_cpu(text_poke_area, cpu))->addr;
+}
+#endif
+
static int text_area_cpu_up(unsigned int cpu)
{
struct vm_struct *area;
--
2.28.0
^ permalink raw reply related
* Re: [PATCH v2] powerpc: Update documentation of ISA versions for Power10
From: Jordan Niethe @ 2020-08-27 5:00 UTC (permalink / raw)
To: Christophe Leroy; +Cc: linuxppc-dev
In-Reply-To: <d71a27b8-f12d-7485-23ec-99d36ff1b0ea@csgroup.eu>
On Thu, Aug 27, 2020 at 2:49 PM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
>
>
>
> Le 27/08/2020 à 06:05, Jordan Niethe a écrit :
> > Update the CPU to ISA Version Mapping document to include Power10 and
> > ISA v3.1.
>
> Maybe Documentation/powerpc/cpu_families.rst should be updated as well.
Good idea it still needs Power9 too.
>
> Christophe
>
>
>
> >
> > Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> > ---
> > v2: Transactional Memory = No
> > ---
> > Documentation/powerpc/isa-versions.rst | 4 ++++
> > 1 file changed, 4 insertions(+)
> >
> > diff --git a/Documentation/powerpc/isa-versions.rst b/Documentation/powerpc/isa-versions.rst
> > index a363d8c1603c..3873bbba183a 100644
> > --- a/Documentation/powerpc/isa-versions.rst
> > +++ b/Documentation/powerpc/isa-versions.rst
> > @@ -7,6 +7,7 @@ Mapping of some CPU versions to relevant ISA versions.
> > ========= ====================================================================
> > CPU Architecture version
> > ========= ====================================================================
> > +Power10 Power ISA v3.1
> > Power9 Power ISA v3.0B
> > Power8 Power ISA v2.07
> > Power7 Power ISA v2.06
> > @@ -32,6 +33,7 @@ Key Features
> > ========== ==================
> > CPU VMX (aka. Altivec)
> > ========== ==================
> > +Power10 Yes
> > Power9 Yes
> > Power8 Yes
> > Power7 Yes
> > @@ -47,6 +49,7 @@ PPC970 Yes
> > ========== ====
> > CPU VSX
> > ========== ====
> > +Power10 Yes
> > Power9 Yes
> > Power8 Yes
> > Power7 Yes
> > @@ -62,6 +65,7 @@ PPC970 No
> > ========== ====================================
> > CPU Transactional Memory
> > ========== ====================================
> > +Power10 No (* see Power ISA v3.1 Appendix A.)
> > Power9 Yes (* see transactional_memory.txt)
> > Power8 Yes
> > Power7 No
> >
^ permalink raw reply
* Re: [PATCH v5 08/23] powerpc/book3s64/kuap: Rename MMU_FTR_RADIX_KUAP to MMU_FTR_KUAP
From: Christophe Leroy @ 2020-08-27 4:57 UTC (permalink / raw)
To: Aneesh Kumar K.V, linuxppc-dev, mpe
In-Reply-To: <20200827040931.297759-9-aneesh.kumar@linux.ibm.com>
Le 27/08/2020 à 06:09, Aneesh Kumar K.V a écrit :
> This is in preparate to adding support for kuap with hash translation.
> In preparation for that rename/move kuap related functions to
> non radix names. Also move the feature bit closer to MMU_FTR_KUEP.
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> ---
> arch/powerpc/include/asm/book3s/64/kup.h | 18 +++++++++---------
> arch/powerpc/include/asm/mmu.h | 16 ++++++++--------
> arch/powerpc/mm/book3s64/pkeys.c | 2 +-
> 3 files changed, 18 insertions(+), 18 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
> index 918a2fcceee7..5cec202dc42f 100644
> --- a/arch/powerpc/include/asm/book3s/64/kup.h
> +++ b/arch/powerpc/include/asm/book3s/64/kup.h
> @@ -24,7 +24,7 @@
> mtspr SPRN_AMR, \gpr2
> /* No isync required, see kuap_restore_amr() */
> 998:
> - END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
> + END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
> #endif
> .endm
>
> @@ -36,7 +36,7 @@
> sldi \gpr2, \gpr2, AMR_KUAP_SHIFT
> 999: tdne \gpr1, \gpr2
> EMIT_BUG_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING | BUGFLAG_ONCE)
> - END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
> + END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
> #endif
> .endm
>
> @@ -56,7 +56,7 @@
> mtspr SPRN_AMR, \gpr2
> isync
> 99:
> - END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
> + END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
> #endif
> .endm
>
> @@ -69,7 +69,7 @@
>
> static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
> {
> - if (mmu_has_feature(MMU_FTR_RADIX_KUAP) && unlikely(regs->kuap != amr)) {
> + if (mmu_has_feature(MMU_FTR_KUAP) && unlikely(regs->kuap != amr)) {
> isync();
> mtspr(SPRN_AMR, regs->kuap);
> /*
> @@ -82,7 +82,7 @@ static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
>
> static inline unsigned long kuap_get_and_check_amr(void)
> {
> - if (mmu_has_feature(MMU_FTR_RADIX_KUAP)) {
> + if (mmu_has_feature(MMU_FTR_KUAP)) {
> unsigned long amr = mfspr(SPRN_AMR);
> if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG)) /* kuap_check_amr() */
> WARN_ON_ONCE(amr != AMR_KUAP_BLOCKED);
> @@ -93,7 +93,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
>
> static inline void kuap_check_amr(void)
> {
> - if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && mmu_has_feature(MMU_FTR_RADIX_KUAP))
> + if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && mmu_has_feature(MMU_FTR_KUAP))
> WARN_ON_ONCE(mfspr(SPRN_AMR) != AMR_KUAP_BLOCKED);
> }
>
> @@ -122,7 +122,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
>
> static inline unsigned long get_kuap(void)
> {
> - if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
> + if (!early_mmu_has_feature(MMU_FTR_KUAP))
> return 0;
>
> return mfspr(SPRN_AMR);
> @@ -130,7 +130,7 @@ static inline unsigned long get_kuap(void)
>
> static inline void set_kuap(unsigned long value)
> {
> - if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
> + if (!early_mmu_has_feature(MMU_FTR_KUAP))
> return;
>
> /*
> @@ -180,7 +180,7 @@ static inline void restore_user_access(unsigned long flags)
> static inline bool
> bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
> {
> - return WARN(mmu_has_feature(MMU_FTR_RADIX_KUAP) &&
> + return WARN(mmu_has_feature(MMU_FTR_KUAP) &&
> (regs->kuap & (is_write ? AMR_KUAP_BLOCK_WRITE : AMR_KUAP_BLOCK_READ)),
> "Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
> }
> diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
> index 255a1837e9f7..04e7a65637fb 100644
> --- a/arch/powerpc/include/asm/mmu.h
> +++ b/arch/powerpc/include/asm/mmu.h
> @@ -29,7 +29,12 @@
> */
>
> /*
> - * Support for KUEP feature.
> + * Supports KUAP (key 0 controlling userspace addresses) on radix
> + */
> +#define MMU_FTR_KUAP ASM_CONST(0x00000200)
> +
> +/*
> + * Suppor for KUEP feature.
Unexpected change I guess. Suppor ==> Support
Christophe
> */
> #define MMU_FTR_KUEP ASM_CONST(0x00000400)
>
> @@ -120,11 +125,6 @@
> */
> #define MMU_FTR_1T_SEGMENT ASM_CONST(0x40000000)
>
> -/*
> - * Supports KUAP (key 0 controlling userspace addresses) on radix
> - */
> -#define MMU_FTR_RADIX_KUAP ASM_CONST(0x80000000)
> -
> /* MMU feature bit sets for various CPUs */
> #define MMU_FTRS_DEFAULT_HPTE_ARCH_V2 \
> MMU_FTR_HPTE_TABLE | MMU_FTR_PPCAS_ARCH_V2
> @@ -187,10 +187,10 @@ enum {
> #ifdef CONFIG_PPC_RADIX_MMU
> MMU_FTR_TYPE_RADIX |
> MMU_FTR_GTSE |
> +#endif /* CONFIG_PPC_RADIX_MMU */
> #ifdef CONFIG_PPC_KUAP
> - MMU_FTR_RADIX_KUAP |
> + MMU_FTR_KUAP |
> #endif /* CONFIG_PPC_KUAP */
> -#endif /* CONFIG_PPC_RADIX_MMU */
> #ifdef CONFIG_PPC_MEM_KEYS
> MMU_FTR_PKEY |
> #endif
> diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
> index 82c722fbce52..bfc27f1f0ab0 100644
> --- a/arch/powerpc/mm/book3s64/pkeys.c
> +++ b/arch/powerpc/mm/book3s64/pkeys.c
> @@ -258,7 +258,7 @@ void __init setup_kuap(bool disabled)
>
> if (smp_processor_id() == boot_cpuid) {
> pr_info("Activating Kernel Userspace Access Prevention\n");
> - cur_cpu_spec->mmu_features |= MMU_FTR_RADIX_KUAP;
> + cur_cpu_spec->mmu_features |= MMU_FTR_KUAP;
> }
>
> /*
>
^ permalink raw reply
* Re: [PATCH v2] powerpc: Update documentation of ISA versions for Power10
From: Christophe Leroy @ 2020-08-27 4:49 UTC (permalink / raw)
To: Jordan Niethe, linuxppc-dev
In-Reply-To: <20200827040556.1783-1-jniethe5@gmail.com>
Le 27/08/2020 à 06:05, Jordan Niethe a écrit :
> Update the CPU to ISA Version Mapping document to include Power10 and
> ISA v3.1.
Maybe Documentation/powerpc/cpu_families.rst should be updated as well.
Christophe
>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v2: Transactional Memory = No
> ---
> Documentation/powerpc/isa-versions.rst | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/Documentation/powerpc/isa-versions.rst b/Documentation/powerpc/isa-versions.rst
> index a363d8c1603c..3873bbba183a 100644
> --- a/Documentation/powerpc/isa-versions.rst
> +++ b/Documentation/powerpc/isa-versions.rst
> @@ -7,6 +7,7 @@ Mapping of some CPU versions to relevant ISA versions.
> ========= ====================================================================
> CPU Architecture version
> ========= ====================================================================
> +Power10 Power ISA v3.1
> Power9 Power ISA v3.0B
> Power8 Power ISA v2.07
> Power7 Power ISA v2.06
> @@ -32,6 +33,7 @@ Key Features
> ========== ==================
> CPU VMX (aka. Altivec)
> ========== ==================
> +Power10 Yes
> Power9 Yes
> Power8 Yes
> Power7 Yes
> @@ -47,6 +49,7 @@ PPC970 Yes
> ========== ====
> CPU VSX
> ========== ====
> +Power10 Yes
> Power9 Yes
> Power8 Yes
> Power7 Yes
> @@ -62,6 +65,7 @@ PPC970 No
> ========== ====================================
> CPU Transactional Memory
> ========== ====================================
> +Power10 No (* see Power ISA v3.1 Appendix A.)
> Power9 Yes (* see transactional_memory.txt)
> Power8 Yes
> Power7 No
>
^ permalink raw reply
* [PATCH v5 23/23] powerpc/book3s64/pkeys: Optimize FTR_KUAP and FTR_KUEP disabled case
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
If FTR_KUAP is disabled kernel will continue to run with the same AMR
value with which it was entered. Hence there is a high chance that
we can return without restoring the AMR value. This also helps the case
when applications are not using the pkey feature. In this case, different
applications will have the same AMR values and hence we can avoid restoring
AMR in this case too.
Also avoid isync() if not really needed.
Do the same for IAMR.
null-syscall benchmark results:
With smap/smep disabled:
Without patch:
957.95 ns 2778.17 cycles
With patch:
858.38 ns 2489.30 cycles
With smap/smep enabled:
Without patch:
1017.26 ns 2950.36 cycles
With patch:
1021.51 ns 2962.44 cycles
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 61 +++++++++++++++++++++---
arch/powerpc/kernel/entry_64.S | 2 +-
arch/powerpc/kernel/syscall_64.c | 12 +++--
3 files changed, 65 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 34a412d2a65b..d71e9a958eb5 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -12,28 +12,54 @@
#ifdef __ASSEMBLY__
-.macro kuap_restore_user_amr gpr1
+.macro kuap_restore_user_amr gpr1, gpr2
#if defined(CONFIG_PPC_PKEY)
BEGIN_MMU_FTR_SECTION_NESTED(67)
+ b 100f // skip_restore_amr
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_PKEY, 67)
/*
* AMR and IAMR are going to be different when
* returning to userspace.
*/
ld \gpr1, STACK_REGS_KUAP(r1)
+
+ /*
+ * If kuap feature is not enabled, do the mtspr
+ * only if AMR value is different.
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(68)
+ mfspr \gpr2, SPRN_AMR
+ cmpd \gpr1, \gpr2
+ beq 99f
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_KUAP, 68)
+
isync
mtspr SPRN_AMR, \gpr1
+99:
/*
* Restore IAMR only when returning to userspace
*/
ld \gpr1, STACK_REGS_KUEP(r1)
+
+ /*
+ * If kuep feature is not enabled, do the mtspr
+ * only if IAMR value is different.
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(69)
+ mfspr \gpr2, SPRN_IAMR
+ cmpd \gpr1, \gpr2
+ beq 100f
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_KUEP, 69)
+
+ isync
mtspr SPRN_IAMR, \gpr1
+100: //skip_restore_amr
/* No isync required, see kuap_restore_user_amr() */
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_PKEY, 67)
#endif
.endm
-.macro kuap_restore_kernel_amr gpr1, gpr2
+.macro kuap_restore_kernel_amr gpr1, gpr2
#if defined(CONFIG_PPC_PKEY)
BEGIN_MMU_FTR_SECTION_NESTED(67)
@@ -190,18 +216,41 @@ static inline u64 current_thread_iamr(void)
static inline void kuap_restore_user_amr(struct pt_regs *regs)
{
+ bool restore_amr = false, restore_iamr = false;
+ unsigned long amr, iamr;
+
if (!mmu_has_feature(MMU_FTR_PKEY))
return;
- isync();
- mtspr(SPRN_AMR, regs->kuap);
- mtspr(SPRN_IAMR, regs->kuep);
+ if (!mmu_has_feature(MMU_FTR_KUAP)) {
+ amr = mfspr(SPRN_AMR);
+ if (amr != regs->kuap)
+ restore_amr = true;
+ } else
+ restore_amr = true;
+
+ if (!mmu_has_feature(MMU_FTR_KUEP)) {
+ iamr = mfspr(SPRN_IAMR);
+ if (iamr != regs->kuep)
+ restore_iamr = true;
+ } else
+ restore_iamr = true;
+
+
+ if (restore_amr || restore_iamr) {
+ isync();
+ if (restore_amr)
+ mtspr(SPRN_AMR, regs->kuap);
+ if (restore_iamr)
+ mtspr(SPRN_IAMR, regs->kuep);
+ }
/*
* No isync required here because we are about to rfi
* back to previous context before any user accesses
* would be made, which is a CSI.
*/
}
+
static inline void kuap_restore_kernel_amr(struct pt_regs *regs,
unsigned long amr)
{
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 68171689db5d..ac6c84a53ff8 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -667,7 +667,7 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return)
bne- .Lrestore_nvgprs
.Lfast_user_interrupt_return_amr:
- kuap_restore_user_amr r3
+ kuap_restore_user_amr r3, r4
.Lfast_user_interrupt_return:
ld r11,_NIP(r1)
ld r12,_MSR(r1)
diff --git a/arch/powerpc/kernel/syscall_64.c b/arch/powerpc/kernel/syscall_64.c
index e49d604b811b..945a14e41898 100644
--- a/arch/powerpc/kernel/syscall_64.c
+++ b/arch/powerpc/kernel/syscall_64.c
@@ -38,6 +38,7 @@ notrace long system_call_exception(long r3, long r4, long r5,
#ifdef CONFIG_PPC_PKEY
if (mmu_has_feature(MMU_FTR_PKEY)) {
unsigned long amr, iamr;
+ bool flush_needed = false;
/*
* When entering from userspace we mostly have the AMR/IAMR
* different from kernel default values. Hence don't compare.
@@ -46,11 +47,16 @@ notrace long system_call_exception(long r3, long r4, long r5,
iamr = mfspr(SPRN_IAMR);
regs->kuap = amr;
regs->kuep = iamr;
- if (mmu_has_feature(MMU_FTR_KUAP))
+ if (mmu_has_feature(MMU_FTR_KUAP)) {
mtspr(SPRN_AMR, AMR_KUAP_BLOCKED);
- if (mmu_has_feature(MMU_FTR_KUEP))
+ flush_needed = true;
+ }
+ if (mmu_has_feature(MMU_FTR_KUEP)) {
mtspr(SPRN_IAMR, AMR_KUEP_BLOCKED);
- isync();
+ flush_needed = true;
+ }
+ if (flush_needed)
+ isync();
} else
#endif
kuap_check_amr();
--
2.26.2
^ permalink raw reply related
* [PATCH v5 22/23] powerpc/book3s64/hash/kup: Don't hardcode kup key
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Make KUAP/KUEP key a variable and also check whether the platform
limit the max key such that we can't use the key for KUAP/KEUP.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
.../powerpc/include/asm/book3s/64/hash-pkey.h | 22 +-------
arch/powerpc/include/asm/book3s/64/pkeys.h | 1 +
arch/powerpc/mm/book3s64/pkeys.c | 53 ++++++++++++++++---
3 files changed, 49 insertions(+), 27 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/hash-pkey.h b/arch/powerpc/include/asm/book3s/64/hash-pkey.h
index 9f44e208f036..ff9907c72ee3 100644
--- a/arch/powerpc/include/asm/book3s/64/hash-pkey.h
+++ b/arch/powerpc/include/asm/book3s/64/hash-pkey.h
@@ -2,9 +2,7 @@
#ifndef _ASM_POWERPC_BOOK3S_64_HASH_PKEY_H
#define _ASM_POWERPC_BOOK3S_64_HASH_PKEY_H
-/* We use key 3 for KERNEL */
-#define HASH_DEFAULT_KERNEL_KEY (HPTE_R_KEY_BIT0 | HPTE_R_KEY_BIT1)
-
+u64 pte_to_hpte_pkey_bits(u64 pteflags, unsigned long flags);
static inline u64 hash__vmflag_to_pte_pkey_bits(u64 vm_flags)
{
return (((vm_flags & VM_PKEY_BIT0) ? H_PTE_PKEY_BIT0 : 0x0UL) |
@@ -14,24 +12,6 @@ static inline u64 hash__vmflag_to_pte_pkey_bits(u64 vm_flags)
((vm_flags & VM_PKEY_BIT4) ? H_PTE_PKEY_BIT4 : 0x0UL));
}
-static inline u64 pte_to_hpte_pkey_bits(u64 pteflags, unsigned long flags)
-{
- unsigned long pte_pkey;
-
- pte_pkey = (((pteflags & H_PTE_PKEY_BIT4) ? HPTE_R_KEY_BIT4 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT3) ? HPTE_R_KEY_BIT3 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT2) ? HPTE_R_KEY_BIT2 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT1) ? HPTE_R_KEY_BIT1 : 0x0UL) |
- ((pteflags & H_PTE_PKEY_BIT0) ? HPTE_R_KEY_BIT0 : 0x0UL));
-
- if (mmu_has_feature(MMU_FTR_KUAP) || mmu_has_feature(MMU_FTR_KUEP)) {
- if ((pte_pkey == 0) && (flags & HPTE_USE_KERNEL_KEY))
- return HASH_DEFAULT_KERNEL_KEY;
- }
-
- return pte_pkey;
-}
-
static inline u16 hash__pte_to_pkey_bits(u64 pteflags)
{
return (((pteflags & H_PTE_PKEY_BIT4) ? 0x10 : 0x0UL) |
diff --git a/arch/powerpc/include/asm/book3s/64/pkeys.h b/arch/powerpc/include/asm/book3s/64/pkeys.h
index 3b8640498f5b..a2b6c4a7275f 100644
--- a/arch/powerpc/include/asm/book3s/64/pkeys.h
+++ b/arch/powerpc/include/asm/book3s/64/pkeys.h
@@ -8,6 +8,7 @@
extern u64 __ro_after_init default_uamor;
extern u64 __ro_after_init default_amr;
extern u64 __ro_after_init default_iamr;
+extern int kup_key;
static inline u64 vmflag_to_pte_pkey_bits(u64 vm_flags)
{
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index b862d5cd78ff..cb1d7d39e801 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -37,7 +37,10 @@ u64 default_uamor __ro_after_init;
*/
static int execute_only_key = 2;
static bool pkey_execute_disable_supported;
-
+/*
+ * key used to implement KUAP/KUEP with hash translation.
+ */
+int kup_key = 3;
#define AMR_BITS_PER_PKEY 2
#define AMR_RD_BIT 0x1UL
@@ -185,6 +188,25 @@ void __init pkey_early_init_devtree(void)
default_uamor &= ~(0x3ul << pkeyshift(execute_only_key));
}
+ if (unlikely(num_pkey <= kup_key)) {
+ /*
+ * Insufficient number of keys to support
+ * KUAP/KUEP feature.
+ */
+ kup_key = -1;
+ } else {
+ /* handle key which is used by kernel for KAUP */
+ reserved_allocation_mask |= (0x1 << kup_key);
+ /*
+ * Mark access for kup_key in default amr so that
+ * we continue to operate with that AMR in
+ * copy_to/from_user().
+ */
+ default_amr &= ~(0x3ul << pkeyshift(kup_key));
+ default_iamr &= ~(0x1ul << pkeyshift(kup_key));
+ default_uamor &= ~(0x3ul << pkeyshift(kup_key));
+ }
+
/*
* Allow access for only key 0. And prevent any other modification.
*/
@@ -205,9 +227,6 @@ void __init pkey_early_init_devtree(void)
reserved_allocation_mask |= (0x1 << 1);
default_uamor &= ~(0x3ul << pkeyshift(1));
- /* handle key 3 which is used by kernel for KAUP */
- reserved_allocation_mask |= (0x1 << 3);
- default_uamor &= ~(0x3ul << pkeyshift(3));
/*
* Prevent the usage of OS reserved keys. Update UAMOR
@@ -236,7 +255,7 @@ void __init pkey_early_init_devtree(void)
#ifdef CONFIG_PPC_KUEP
void __init setup_kuep(bool disabled)
{
- if (disabled)
+ if (disabled || kup_key == -1)
return;
/*
* On hash if PKEY feature is not enabled, disable KUAP too.
@@ -262,7 +281,7 @@ void __init setup_kuep(bool disabled)
#ifdef CONFIG_PPC_KUAP
void __init setup_kuap(bool disabled)
{
- if (disabled)
+ if (disabled || kup_key == -1)
return;
/*
* On hash if PKEY feature is not enabled, disable KUAP too.
@@ -458,4 +477,26 @@ void arch_dup_pkeys(struct mm_struct *oldmm, struct mm_struct *mm)
mm->context.execute_only_pkey = oldmm->context.execute_only_pkey;
}
+u64 pte_to_hpte_pkey_bits(u64 pteflags, unsigned long flags)
+{
+ unsigned long pte_pkey;
+
+ pte_pkey = (((pteflags & H_PTE_PKEY_BIT4) ? HPTE_R_KEY_BIT4 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT3) ? HPTE_R_KEY_BIT3 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT2) ? HPTE_R_KEY_BIT2 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT1) ? HPTE_R_KEY_BIT1 : 0x0UL) |
+ ((pteflags & H_PTE_PKEY_BIT0) ? HPTE_R_KEY_BIT0 : 0x0UL));
+
+ if (mmu_has_feature(MMU_FTR_KUAP) || mmu_has_feature(MMU_FTR_KUEP)) {
+ if ((pte_pkey == 0) &&
+ (flags & HPTE_USE_KERNEL_KEY) && (kup_key != -1)) {
+ u64 vm_flag = pkey_to_vmflag_bits(kup_key);
+ u64 pte_flag = hash__vmflag_to_pte_pkey_bits(vm_flag);
+ return pte_to_hpte_pkey_bits(pte_flag, 0);
+ }
+ }
+
+ return pte_pkey;
+}
+
#endif /* CONFIG_PPC_MEM_KEYS */
--
2.26.2
^ permalink raw reply related
* [PATCH v5 21/23] powerpc/book3s64/hash/kuep: Enable KUEP on hash
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/mm/book3s64/pkeys.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index 16ea0b2f0ea5..b862d5cd78ff 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -236,7 +236,12 @@ void __init pkey_early_init_devtree(void)
#ifdef CONFIG_PPC_KUEP
void __init setup_kuep(bool disabled)
{
- if (disabled || !early_radix_enabled())
+ if (disabled)
+ return;
+ /*
+ * On hash if PKEY feature is not enabled, disable KUAP too.
+ */
+ if (!early_radix_enabled() && !early_mmu_has_feature(MMU_FTR_PKEY))
return;
if (smp_processor_id() == boot_cpuid) {
--
2.26.2
^ permalink raw reply related
* [PATCH v5 20/23] powerpc/book3s64/hash/kuap: Enable kuap on hash
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/mm/book3s64/pkeys.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index 391230f93da2..16ea0b2f0ea5 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -257,7 +257,12 @@ void __init setup_kuep(bool disabled)
#ifdef CONFIG_PPC_KUAP
void __init setup_kuap(bool disabled)
{
- if (disabled || !early_radix_enabled())
+ if (disabled)
+ return;
+ /*
+ * On hash if PKEY feature is not enabled, disable KUAP too.
+ */
+ if (!early_radix_enabled() && !early_mmu_has_feature(MMU_FTR_PKEY))
return;
if (smp_processor_id() == boot_cpuid) {
--
2.26.2
^ permalink raw reply related
* [PATCH v5 19/23] powerpc/book3s64/kuep: Use Key 3 to implement KUEP with hash translation.
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Radix use IAMR Key 0 and hash translation use IAMR key 3.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index f326be9e0db7..34a412d2a65b 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -7,7 +7,7 @@
#define AMR_KUAP_BLOCK_READ UL(0x5455555555555555)
#define AMR_KUAP_BLOCK_WRITE UL(0xa8aaaaaaaaaaaaaa)
-#define AMR_KUEP_BLOCKED (1UL << 62)
+#define AMR_KUEP_BLOCKED UL(0x5455555555555555)
#define AMR_KUAP_BLOCKED (AMR_KUAP_BLOCK_READ | AMR_KUAP_BLOCK_WRITE)
#ifdef __ASSEMBLY__
--
2.26.2
^ permalink raw reply related
* [PATCH v5 18/23] powerpc/book3s64/kuap: Use Key 3 to implement KUAP with hash translation.
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Radix use AMR Key 0 and hash translation use AMR key 3.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 9c85e4397b2d..f326be9e0db7 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -5,11 +5,10 @@
#include <linux/const.h>
#include <asm/reg.h>
-#define AMR_KUAP_BLOCK_READ UL(0x4000000000000000)
-#define AMR_KUAP_BLOCK_WRITE UL(0x8000000000000000)
+#define AMR_KUAP_BLOCK_READ UL(0x5455555555555555)
+#define AMR_KUAP_BLOCK_WRITE UL(0xa8aaaaaaaaaaaaaa)
#define AMR_KUEP_BLOCKED (1UL << 62)
#define AMR_KUAP_BLOCKED (AMR_KUAP_BLOCK_READ | AMR_KUAP_BLOCK_WRITE)
-#define AMR_KUAP_SHIFT 62
#ifdef __ASSEMBLY__
@@ -61,8 +60,8 @@
#ifdef CONFIG_PPC_KUAP_DEBUG
BEGIN_MMU_FTR_SECTION_NESTED(67)
mfspr \gpr1, SPRN_AMR
- li \gpr2, (AMR_KUAP_BLOCKED >> AMR_KUAP_SHIFT)
- sldi \gpr2, \gpr2, AMR_KUAP_SHIFT
+ /* Prevent access to userspace using any key values */
+ LOAD_REG_IMMEDIATE(\gpr2, AMR_KUAP_BLOCKED)
999: tdne \gpr1, \gpr2
EMIT_BUG_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING | BUGFLAG_ONCE)
END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
--
2.26.2
^ permalink raw reply related
* [PATCH v5 16/23] powerpc/book3s64/kuap: Restrict access to userspace based on userspace AMR
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
If an application has configured address protection such that read/write is
denied using pkey even the kernel should receive a FAULT on accessing the same.
This patch use user AMR value stored in pt_regs.kuap to achieve the same.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 4e1d666032f6..878cd84922d8 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -292,14 +292,20 @@ static inline void set_kuap(unsigned long value)
static __always_inline void allow_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir)
{
+ unsigned long thread_amr = 0;
+
// This is written so we can resolve to a single case at build time
BUILD_BUG_ON(!__builtin_constant_p(dir));
+
+ if (mmu_has_feature(MMU_FTR_PKEY))
+ thread_amr = current_thread_amr();
+
if (dir == KUAP_READ)
- set_kuap(AMR_KUAP_BLOCK_WRITE);
+ set_kuap(thread_amr | AMR_KUAP_BLOCK_WRITE);
else if (dir == KUAP_WRITE)
- set_kuap(AMR_KUAP_BLOCK_READ);
+ set_kuap(thread_amr | AMR_KUAP_BLOCK_READ);
else if (dir == KUAP_READ_WRITE)
- set_kuap(0);
+ set_kuap(thread_amr);
else
BUILD_BUG();
}
--
2.26.2
^ permalink raw reply related
* [PATCH v5 17/23] powerpc/book3s64/kuap: Improve error reporting with KUAP
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
With hash translation use DSISR_KEYFAULT to identify a wrong access.
With Radix we look at the AMR value and type of fault.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/32/kup.h | 4 +--
arch/powerpc/include/asm/book3s/64/kup.h | 27 ++++++++++++++++----
arch/powerpc/include/asm/kup.h | 4 +--
arch/powerpc/include/asm/nohash/32/kup-8xx.h | 4 +--
arch/powerpc/mm/fault.c | 2 +-
5 files changed, 29 insertions(+), 12 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/32/kup.h b/arch/powerpc/include/asm/book3s/32/kup.h
index 32fd4452e960..b18cd931e325 100644
--- a/arch/powerpc/include/asm/book3s/32/kup.h
+++ b/arch/powerpc/include/asm/book3s/32/kup.h
@@ -177,8 +177,8 @@ static inline void restore_user_access(unsigned long flags)
allow_user_access(to, to, end - addr, KUAP_READ_WRITE);
}
-static inline bool
-bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+static inline bool bad_kuap_fault(struct pt_regs *regs, unsigned long address,
+ bool is_write, unsigned long error_code)
{
unsigned long begin = regs->kuap & 0xf0000000;
unsigned long end = regs->kuap << 28;
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 878cd84922d8..9c85e4397b2d 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -330,12 +330,29 @@ static inline void restore_user_access(unsigned long flags)
set_kuap(flags);
}
-static inline bool
-bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+#define RADIX_KUAP_BLOCK_READ UL(0x4000000000000000)
+#define RADIX_KUAP_BLOCK_WRITE UL(0x8000000000000000)
+
+static inline bool bad_kuap_fault(struct pt_regs *regs, unsigned long address,
+ bool is_write, unsigned long error_code)
{
- return WARN(mmu_has_feature(MMU_FTR_KUAP) &&
- (regs->kuap & (is_write ? AMR_KUAP_BLOCK_WRITE : AMR_KUAP_BLOCK_READ)),
- "Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
+ if (!mmu_has_feature(MMU_FTR_KUAP))
+ return false;
+
+ if (radix_enabled()) {
+ /*
+ * Will be a storage protection fault.
+ * Only check the details of AMR[0]
+ */
+ return WARN((regs->kuap & (is_write ? RADIX_KUAP_BLOCK_WRITE : RADIX_KUAP_BLOCK_READ)),
+ "Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
+ }
+ /*
+ * We don't want to WARN here because userspace can setup
+ * keys such that a kernel access to user address can cause
+ * fault
+ */
+ return !!(error_code & DSISR_KEYFAULT);
}
#endif /* CONFIG_PPC_KUAP */
diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h
index 6c3ee976ee15..8f5e2d820723 100644
--- a/arch/powerpc/include/asm/kup.h
+++ b/arch/powerpc/include/asm/kup.h
@@ -67,8 +67,8 @@ static inline void prevent_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir) { }
static inline unsigned long prevent_user_access_return(void) { return 0UL; }
static inline void restore_user_access(unsigned long flags) { }
-static inline bool
-bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+static inline bool bad_kuap_fault(struct pt_regs *regs, unsigned long address,
+ bool is_write, unsigned long error_code)
{
return false;
}
diff --git a/arch/powerpc/include/asm/nohash/32/kup-8xx.h b/arch/powerpc/include/asm/nohash/32/kup-8xx.h
index 85ed2390fb99..c401e4e404d4 100644
--- a/arch/powerpc/include/asm/nohash/32/kup-8xx.h
+++ b/arch/powerpc/include/asm/nohash/32/kup-8xx.h
@@ -60,8 +60,8 @@ static inline void restore_user_access(unsigned long flags)
mtspr(SPRN_MD_AP, flags);
}
-static inline bool
-bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+static inline bool bad_kuap_fault(struct pt_regs *regs, unsigned long address,
+ bool is_write, unsigned long error_code)
{
return WARN(!((regs->kuap ^ MD_APG_KUAP) & 0xf0000000),
"Bug: fault blocked by AP register !");
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 0add963a849b..c91621df0c61 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -227,7 +227,7 @@ static bool bad_kernel_fault(struct pt_regs *regs, unsigned long error_code,
// Read/write fault in a valid region (the exception table search passed
// above), but blocked by KUAP is bad, it can never succeed.
- if (bad_kuap_fault(regs, address, is_write))
+ if (bad_kuap_fault(regs, address, is_write, error_code))
return true;
// What's left? Kernel fault on user in well defined regions (extable
--
2.26.2
^ permalink raw reply related
* [PATCH v5 15/23] powerpc/book3s64/pkeys: Don't update SPRN_AMR when in kernel mode.
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Now that kernel correctly store/restore userspace AMR/IAMR values, avoid
manipulating AMR and IAMR from the kernel on behalf of userspace.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 18 ++++++++
arch/powerpc/include/asm/processor.h | 4 --
arch/powerpc/kernel/process.c | 4 --
arch/powerpc/kernel/traps.c | 6 ---
arch/powerpc/mm/book3s64/pkeys.c | 57 +++++-------------------
5 files changed, 28 insertions(+), 61 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 3f5b97b2a3d8..4e1d666032f6 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -171,6 +171,24 @@
#include <asm/mmu.h>
#include <asm/ptrace.h>
+/*
+ * For kernel thread that doesn't have thread.regs return
+ * default AMR/IAMR values.
+ */
+static inline u64 current_thread_amr(void)
+{
+ if (current->thread.regs)
+ return current->thread.regs->kuap;
+ return AMR_KUAP_BLOCKED;
+}
+
+static inline u64 current_thread_iamr(void)
+{
+ if (current->thread.regs)
+ return current->thread.regs->kuep;
+ return AMR_KUEP_BLOCKED;
+}
+
static inline void kuap_restore_user_amr(struct pt_regs *regs)
{
if (!mmu_has_feature(MMU_FTR_PKEY))
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index ed0d633ab5aa..8adf44a7e54f 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -234,10 +234,6 @@ struct thread_struct {
struct thread_vr_state ckvr_state; /* Checkpointed VR state */
unsigned long ckvrsave; /* Checkpointed VRSAVE */
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
-#ifdef CONFIG_PPC_MEM_KEYS
- unsigned long amr;
- unsigned long iamr;
-#endif
#ifdef CONFIG_KVM_BOOK3S_32_HANDLER
void* kvm_shadow_vcpu; /* KVM internal data */
#endif /* CONFIG_KVM_BOOK3S_32_HANDLER */
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 75fd30e023bd..c8f57afba3a0 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -603,7 +603,6 @@ static void save_all(struct task_struct *tsk)
__giveup_spe(tsk);
msr_check_and_clear(msr_all_available);
- thread_pkey_regs_save(&tsk->thread);
}
void flush_all_to_thread(struct task_struct *tsk)
@@ -1127,8 +1126,6 @@ static inline void save_sprs(struct thread_struct *t)
t->tar = mfspr(SPRN_TAR);
}
#endif
-
- thread_pkey_regs_save(t);
}
static inline void restore_sprs(struct thread_struct *old_thread,
@@ -1169,7 +1166,6 @@ static inline void restore_sprs(struct thread_struct *old_thread,
mtspr(SPRN_TIDR, new_thread->tidr);
#endif
- thread_pkey_regs_restore(new_thread, old_thread);
}
struct task_struct *__switch_to(struct task_struct *prev,
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index d1ebe152f210..5bda54454a2d 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -347,12 +347,6 @@ static bool exception_common(int signr, struct pt_regs *regs, int code,
current->thread.trap_nr = code;
- /*
- * Save all the pkey registers AMR/IAMR/UAMOR. Eg: Core dumps need
- * to capture the content, if the task gets killed.
- */
- thread_pkey_regs_save(¤t->thread);
-
return true;
}
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index f47d11f2743d..391230f93da2 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -273,30 +273,17 @@ void __init setup_kuap(bool disabled)
}
#endif
-static inline u64 read_amr(void)
+static inline void update_current_thread_amr(u64 value)
{
- return mfspr(SPRN_AMR);
+ current->thread.regs->kuap = value;
}
-static inline void write_amr(u64 value)
-{
- mtspr(SPRN_AMR, value);
-}
-
-static inline u64 read_iamr(void)
-{
- if (!likely(pkey_execute_disable_supported))
- return 0x0UL;
-
- return mfspr(SPRN_IAMR);
-}
-
-static inline void write_iamr(u64 value)
+static inline void update_current_thread_iamr(u64 value)
{
if (!likely(pkey_execute_disable_supported))
return;
- mtspr(SPRN_IAMR, value);
+ current->thread.regs->kuep = value;
}
#ifdef CONFIG_PPC_MEM_KEYS
@@ -311,17 +298,17 @@ void pkey_mm_init(struct mm_struct *mm)
static inline void init_amr(int pkey, u8 init_bits)
{
u64 new_amr_bits = (((u64)init_bits & 0x3UL) << pkeyshift(pkey));
- u64 old_amr = read_amr() & ~((u64)(0x3ul) << pkeyshift(pkey));
+ u64 old_amr = current_thread_amr() & ~((u64)(0x3ul) << pkeyshift(pkey));
- write_amr(old_amr | new_amr_bits);
+ update_current_thread_amr(old_amr | new_amr_bits);
}
static inline void init_iamr(int pkey, u8 init_bits)
{
u64 new_iamr_bits = (((u64)init_bits & 0x1UL) << pkeyshift(pkey));
- u64 old_iamr = read_iamr() & ~((u64)(0x1ul) << pkeyshift(pkey));
+ u64 old_iamr = current_thread_iamr() & ~((u64)(0x1ul) << pkeyshift(pkey));
- write_iamr(old_iamr | new_iamr_bits);
+ update_current_thread_iamr(old_iamr | new_iamr_bits);
}
/*
@@ -364,30 +351,6 @@ int __arch_set_user_pkey_access(struct task_struct *tsk, int pkey,
return 0;
}
-void thread_pkey_regs_save(struct thread_struct *thread)
-{
- if (!mmu_has_feature(MMU_FTR_PKEY))
- return;
-
- /*
- * TODO: Skip saving registers if @thread hasn't used any keys yet.
- */
- thread->amr = read_amr();
- thread->iamr = read_iamr();
-}
-
-void thread_pkey_regs_restore(struct thread_struct *new_thread,
- struct thread_struct *old_thread)
-{
- if (!mmu_has_feature(MMU_FTR_PKEY))
- return;
-
- if (old_thread->amr != new_thread->amr)
- write_amr(new_thread->amr);
- if (old_thread->iamr != new_thread->iamr)
- write_iamr(new_thread->iamr);
-}
-
int execute_only_pkey(struct mm_struct *mm)
{
return mm->context.execute_only_pkey;
@@ -436,9 +399,9 @@ static bool pkey_access_permitted(int pkey, bool write, bool execute)
pkey_shift = pkeyshift(pkey);
if (execute)
- return !(read_iamr() & (IAMR_EX_BIT << pkey_shift));
+ return !(current_thread_iamr() & (IAMR_EX_BIT << pkey_shift));
- amr = read_amr();
+ amr = current_thread_amr();
if (write)
return !(amr & (AMR_WR_BIT << pkey_shift));
--
2.26.2
^ permalink raw reply related
* [PATCH v5 14/23] powerpc/ptrace-view: Use pt_regs values instead of thread_struct based one.
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
We will remove thread.amr/iamr/uamor in a later patch
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/kernel/ptrace/ptrace-view.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/ptrace/ptrace-view.c b/arch/powerpc/kernel/ptrace/ptrace-view.c
index 7e6478e7ed07..c719e29aff76 100644
--- a/arch/powerpc/kernel/ptrace/ptrace-view.c
+++ b/arch/powerpc/kernel/ptrace/ptrace-view.c
@@ -470,12 +470,12 @@ static int pkey_active(struct task_struct *target, const struct user_regset *reg
static int pkey_get(struct task_struct *target, const struct user_regset *regset,
struct membuf to)
{
- BUILD_BUG_ON(TSO(amr) + sizeof(unsigned long) != TSO(iamr));
if (!arch_pkeys_enabled())
return -ENODEV;
- membuf_write(&to, &target->thread.amr, 2 * sizeof(unsigned long));
+ membuf_store(&to, target->thread.regs->kuap);
+ membuf_store(&to, target->thread.regs->kuep);
return membuf_store(&to, default_uamor);
}
@@ -508,7 +508,8 @@ static int pkey_set(struct task_struct *target, const struct user_regset *regset
* Pick the AMR values for the keys that kernel is using. This
* will be indicated by the ~default_uamor bits.
*/
- target->thread.amr = (new_amr & default_uamor) | (target->thread.amr & ~default_uamor);
+ target->thread.regs->kuap = (new_amr & default_uamor) |
+ (target->thread.regs->kuap & ~default_uamor);
return 0;
}
--
2.26.2
^ permalink raw reply related
* [PATCH v5 13/23] powerpc/book3s64/pkeys: Reset userspace AMR correctly on exec
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
On fork, we inherit from the parent and on exec, we should switch to default_amr values.
Also, avoid changing the AMR register value within the kernel. The kernel now runs with
different AMR values.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/pkeys.h | 2 ++
arch/powerpc/kernel/process.c | 6 +++++-
arch/powerpc/mm/book3s64/pkeys.c | 16 ++--------------
3 files changed, 9 insertions(+), 15 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/pkeys.h b/arch/powerpc/include/asm/book3s/64/pkeys.h
index b7d9f4267bcd..3b8640498f5b 100644
--- a/arch/powerpc/include/asm/book3s/64/pkeys.h
+++ b/arch/powerpc/include/asm/book3s/64/pkeys.h
@@ -6,6 +6,8 @@
#include <asm/book3s/64/hash-pkey.h>
extern u64 __ro_after_init default_uamor;
+extern u64 __ro_after_init default_amr;
+extern u64 __ro_after_init default_iamr;
static inline u64 vmflag_to_pte_pkey_bits(u64 vm_flags)
{
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 725fd1bed2b6..75fd30e023bd 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1506,6 +1506,11 @@ void arch_setup_new_exec(void)
current->thread.regs = regs - 1;
}
+#ifdef CONFIG_PPC_MEM_KEYS
+ current->thread.regs->kuap = default_amr;
+ current->thread.regs->kuep = default_iamr;
+#endif
+
}
#else
void arch_setup_new_exec(void)
@@ -1866,7 +1871,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
current->thread.load_tm = 0;
#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
- thread_pkey_regs_init(¤t->thread);
}
EXPORT_SYMBOL(start_thread);
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index 640f090b9f9d..f47d11f2743d 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -28,8 +28,8 @@ static u32 initial_allocation_mask __ro_after_init;
* Even if we allocate keys with sys_pkey_alloc(), we need to make sure
* other thread still find the access denied using the same keys.
*/
-static u64 default_amr = ~0x0UL;
-static u64 default_iamr = 0x5555555555555555UL;
+u64 default_amr __ro_after_init = ~0x0UL;
+u64 default_iamr __ro_after_init = 0x5555555555555555UL;
u64 default_uamor __ro_after_init;
/*
* Key used to implement PROT_EXEC mmap. Denies READ/WRITE
@@ -388,18 +388,6 @@ void thread_pkey_regs_restore(struct thread_struct *new_thread,
write_iamr(new_thread->iamr);
}
-void thread_pkey_regs_init(struct thread_struct *thread)
-{
- if (!mmu_has_feature(MMU_FTR_PKEY))
- return;
-
- thread->amr = default_amr;
- thread->iamr = default_iamr;
-
- write_amr(default_amr);
- write_iamr(default_iamr);
-}
-
int execute_only_pkey(struct mm_struct *mm)
{
return mm->context.execute_only_pkey;
--
2.26.2
^ permalink raw reply related
* [PATCH v5 12/23] powerpc/book3s64/pkeys: Inherit correctly on fork.
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
Child thread.kuap value is inherited from the parent in copy_thread_tls. We still
need to make sure when the child returns from a fork in the kernel we start with the kernel
default AMR value.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/kernel/process.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 4633924ea77f..725fd1bed2b6 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1732,6 +1732,15 @@ int copy_thread(unsigned long clone_flags, unsigned long usp,
childregs->ppr = DEFAULT_PPR;
p->thread.tidr = 0;
+#endif
+ /*
+ * Run with the current AMR value of the kernel
+ */
+#if defined(CONFIG_PPC_MEM_KEYS)
+ if (mmu_has_feature(MMU_FTR_KUAP))
+ kregs->kuap = AMR_KUAP_BLOCKED;
+ if (mmu_has_feature(MMU_FTR_KUEP))
+ kregs->kuep = AMR_KUEP_BLOCKED;
#endif
kregs->nip = ppc_function_entry(f);
return 0;
--
2.26.2
^ permalink raw reply related
* [PATCH v5 11/23] powerpc/book3s64/pkeys: Store/restore userspace AMR/IAMR correctly on entry and exit from kernel
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
This prepare kernel to operate with a different value than userspace AMR/IAMR.
For this, AMR/IAMR need to be saved and restored on entry and return from the
kernel.
With KUAP we modify kernel AMR when accessing user address from the kernel
via copy_to/from_user interfaces. We don't need to modify IAMR value in
similar fashion.
If MMU_FTR_PKEY is enabled we need to save AMR/IAMR in pt_regs on entering
kernel from userspace. If not we can assume that AMR/IAMR is not modified
from userspace.
We need to save AMR if we have MMU_FTR_KUAP feature enabled and we are
interrupted within kernel. This is required so that if we get interrupted
within copy_to/from_user we continue with the right AMR value.
If we hae MMU_FTR_KUEP enabled we need to restore IAMR on return to userspace
beause kernel will be running with a different IAMR value.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 177 ++++++++++++++++++++---
arch/powerpc/include/asm/ptrace.h | 4 +-
arch/powerpc/kernel/asm-offsets.c | 2 +
arch/powerpc/kernel/entry_64.S | 6 +-
arch/powerpc/kernel/exceptions-64s.S | 4 +-
arch/powerpc/kernel/syscall_64.c | 30 +++-
6 files changed, 192 insertions(+), 31 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 5cec202dc42f..3f5b97b2a3d8 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -13,17 +13,46 @@
#ifdef __ASSEMBLY__
-.macro kuap_restore_amr gpr1, gpr2
-#ifdef CONFIG_PPC_KUAP
+.macro kuap_restore_user_amr gpr1
+#if defined(CONFIG_PPC_PKEY)
BEGIN_MMU_FTR_SECTION_NESTED(67)
- mfspr \gpr1, SPRN_AMR
+ /*
+ * AMR and IAMR are going to be different when
+ * returning to userspace.
+ */
+ ld \gpr1, STACK_REGS_KUAP(r1)
+ isync
+ mtspr SPRN_AMR, \gpr1
+ /*
+ * Restore IAMR only when returning to userspace
+ */
+ ld \gpr1, STACK_REGS_KUEP(r1)
+ mtspr SPRN_IAMR, \gpr1
+
+ /* No isync required, see kuap_restore_user_amr() */
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_PKEY, 67)
+#endif
+.endm
+
+.macro kuap_restore_kernel_amr gpr1, gpr2
+#if defined(CONFIG_PPC_PKEY)
+
+ BEGIN_MMU_FTR_SECTION_NESTED(67)
+ /*
+ * AMR is going to be mostly the same since we are
+ * returning to the kernel. Compare and do a mtspr.
+ */
ld \gpr2, STACK_REGS_KUAP(r1)
+ mfspr \gpr1, SPRN_AMR
cmpd \gpr1, \gpr2
- beq 998f
+ beq 100f
isync
mtspr SPRN_AMR, \gpr2
- /* No isync required, see kuap_restore_amr() */
-998:
+ /*
+ * No isync required, see kuap_restore_amr()
+ * No need to restore IAMR when returning to kernel space.
+ */
+100:
END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
#endif
.endm
@@ -40,23 +69,98 @@
#endif
.endm
+/*
+ * if (pkey) {
+ *
+ * save AMR -> stack;
+ * if (kuap) {
+ * if (AMR != BLOCKED)
+ * KUAP_BLOCKED -> AMR;
+ * }
+ * if (from_user) {
+ * save IAMR -> stack;
+ * if (kuep) {
+ * KUEP_BLOCKED ->IAMR
+ * }
+ * }
+ * return;
+ * }
+ *
+ * if (kuap) {
+ * if (from_kernel) {
+ * save AMR -> stack;
+ * if (AMR != BLOCKED)
+ * KUAP_BLOCKED -> AMR;
+ * }
+ *
+ * }
+ */
.macro kuap_save_amr_and_lock gpr1, gpr2, use_cr, msr_pr_cr
-#ifdef CONFIG_PPC_KUAP
+#if defined(CONFIG_PPC_PKEY)
+
+ /*
+ * if both pkey and kuap is disabled, nothing to do
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(68)
+ b 100f // skip_save_amr
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_PKEY | MMU_FTR_KUAP, 68)
+
+ /*
+ * if pkey is disabled and we are entering from userspace
+ * don't do anything.
+ */
BEGIN_MMU_FTR_SECTION_NESTED(67)
.ifnb \msr_pr_cr
- bne \msr_pr_cr, 99f
+ /*
+ * Without pkey we are not changing AMR outside the kernel
+ * hence skip this completely.
+ */
+ bne \msr_pr_cr, 100f // from userspace
.endif
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_PKEY, 67)
+
+ /*
+ * pkey is enabled or pkey is disabled but entering from kernel
+ */
mfspr \gpr1, SPRN_AMR
std \gpr1, STACK_REGS_KUAP(r1)
- li \gpr2, (AMR_KUAP_BLOCKED >> AMR_KUAP_SHIFT)
- sldi \gpr2, \gpr2, AMR_KUAP_SHIFT
+
+ /*
+ * update kernel AMR with AMR_KUAP_BLOCKED only
+ * if KUAP feature is enabled
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(69)
+ LOAD_REG_IMMEDIATE(\gpr2, AMR_KUAP_BLOCKED)
cmpd \use_cr, \gpr1, \gpr2
- beq \use_cr, 99f
- // We don't isync here because we very recently entered via rfid
+ beq \use_cr, 102f
+ /*
+ * We don't isync here because we very recently entered via an interrupt
+ */
mtspr SPRN_AMR, \gpr2
isync
-99:
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
+102:
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 69)
+
+ /*
+ * if entering from kernel we don't need save IAMR
+ */
+ .ifnb \msr_pr_cr
+ beq \msr_pr_cr, 100f // from kernel space
+ mfspr \gpr1, SPRN_IAMR
+ std \gpr1, STACK_REGS_KUEP(r1)
+
+ /*
+ * update kernel IAMR with AMR_KUEP_BLOCKED only
+ * if KUEP feature is enabled
+ */
+ BEGIN_MMU_FTR_SECTION_NESTED(70)
+ LOAD_REG_IMMEDIATE(\gpr2, AMR_KUEP_BLOCKED)
+ mtspr SPRN_IAMR, \gpr2
+ isync
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUEP, 70)
+ .endif
+
+100: // skip_save_amr
#endif
.endm
@@ -67,17 +171,37 @@
#include <asm/mmu.h>
#include <asm/ptrace.h>
-static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
+static inline void kuap_restore_user_amr(struct pt_regs *regs)
+{
+ if (!mmu_has_feature(MMU_FTR_PKEY))
+ return;
+
+ isync();
+ mtspr(SPRN_AMR, regs->kuap);
+ mtspr(SPRN_IAMR, regs->kuep);
+ /*
+ * No isync required here because we are about to rfi
+ * back to previous context before any user accesses
+ * would be made, which is a CSI.
+ */
+}
+static inline void kuap_restore_kernel_amr(struct pt_regs *regs,
+ unsigned long amr)
{
- if (mmu_has_feature(MMU_FTR_KUAP) && unlikely(regs->kuap != amr)) {
- isync();
- mtspr(SPRN_AMR, regs->kuap);
- /*
- * No isync required here because we are about to RFI back to
- * previous context before any user accesses would be made,
- * which is a CSI.
- */
+ if (mmu_has_feature(MMU_FTR_KUAP)) {
+ if (unlikely(regs->kuap != amr)) {
+ isync();
+ mtspr(SPRN_AMR, regs->kuap);
+ /*
+ * No isync required here because we are about to rfi
+ * back to previous context before any user accesses
+ * would be made, which is a CSI.
+ */
+ }
}
+ /*
+ * No need to restore IAMR when returning to kernel space.
+ */
}
static inline unsigned long kuap_get_and_check_amr(void)
@@ -99,7 +223,11 @@ static inline void kuap_check_amr(void)
#else /* CONFIG_PPC_PKEY */
-static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
+static inline void kuap_restore_user_amr(struct pt_regs *regs)
+{
+}
+
+static inline void kuap_restore_kernel_amr(struct pt_regs *regs, unsigned long amr)
{
}
@@ -111,6 +239,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
{
return 0;
}
+
#endif /* CONFIG_PPC_PKEY */
diff --git a/arch/powerpc/include/asm/ptrace.h b/arch/powerpc/include/asm/ptrace.h
index 5f62ce579a8b..83f3276fab7b 100644
--- a/arch/powerpc/include/asm/ptrace.h
+++ b/arch/powerpc/include/asm/ptrace.h
@@ -55,9 +55,11 @@ struct pt_regs
#endif
#ifdef CONFIG_PPC_PKEY
unsigned long kuap;
+ unsigned long kuep;
#endif
+
};
- unsigned long __pad[2]; /* Maintain 16 byte interrupt stack alignment */
+ unsigned long __pad[4]; /* Maintain 16 byte interrupt stack alignment */
};
};
#endif
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 63548992b5ab..2f413aaa29cb 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -355,8 +355,10 @@ int main(void)
#ifdef CONFIG_PPC_PKEY
STACK_PT_REGS_OFFSET(STACK_REGS_KUAP, kuap);
+ STACK_PT_REGS_OFFSET(STACK_REGS_KUEP, kuep);
#endif
+
#if defined(CONFIG_PPC32)
#if defined(CONFIG_BOOKE) || defined(CONFIG_40x)
DEFINE(EXC_LVL_SIZE, STACK_EXC_LVL_FRAME_SIZE);
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 33a42e42c56f..68171689db5d 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -645,8 +645,8 @@ _ASM_NOKPROBE_SYMBOL(fast_interrupt_return)
kuap_check_amr r3, r4
ld r5,_MSR(r1)
andi. r0,r5,MSR_PR
- bne .Lfast_user_interrupt_return
- kuap_restore_amr r3, r4
+ bne .Lfast_user_interrupt_return_amr
+ kuap_restore_kernel_amr r3, r4
andi. r0,r5,MSR_RI
li r3,0 /* 0 return value, no EMULATE_STACK_STORE */
bne+ .Lfast_kernel_interrupt_return
@@ -666,6 +666,8 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return)
cmpdi r3,0
bne- .Lrestore_nvgprs
+.Lfast_user_interrupt_return_amr:
+ kuap_restore_user_amr r3
.Lfast_user_interrupt_return:
ld r11,_NIP(r1)
ld r12,_MSR(r1)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index f7d748b88705..93c7840ea8b9 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -1061,7 +1061,7 @@ EXC_COMMON_BEGIN(system_reset_common)
ld r10,SOFTE(r1)
stb r10,PACAIRQSOFTMASK(r13)
- kuap_restore_amr r9, r10
+ kuap_restore_kernel_amr r9, r10
EXCEPTION_RESTORE_REGS
RFI_TO_USER_OR_KERNEL
@@ -2874,7 +2874,7 @@ EXC_COMMON_BEGIN(soft_nmi_common)
ld r10,SOFTE(r1)
stb r10,PACAIRQSOFTMASK(r13)
- kuap_restore_amr r9, r10
+ kuap_restore_kernel_amr r9, r10
EXCEPTION_RESTORE_REGS hsrr=0
RFI_TO_KERNEL
diff --git a/arch/powerpc/kernel/syscall_64.c b/arch/powerpc/kernel/syscall_64.c
index 22a31a988264..e49d604b811b 100644
--- a/arch/powerpc/kernel/syscall_64.c
+++ b/arch/powerpc/kernel/syscall_64.c
@@ -35,7 +35,25 @@ notrace long system_call_exception(long r3, long r4, long r5,
BUG_ON(!FULL_REGS(regs));
BUG_ON(regs->softe != IRQS_ENABLED);
- kuap_check_amr();
+#ifdef CONFIG_PPC_PKEY
+ if (mmu_has_feature(MMU_FTR_PKEY)) {
+ unsigned long amr, iamr;
+ /*
+ * When entering from userspace we mostly have the AMR/IAMR
+ * different from kernel default values. Hence don't compare.
+ */
+ amr = mfspr(SPRN_AMR);
+ iamr = mfspr(SPRN_IAMR);
+ regs->kuap = amr;
+ regs->kuep = iamr;
+ if (mmu_has_feature(MMU_FTR_KUAP))
+ mtspr(SPRN_AMR, AMR_KUAP_BLOCKED);
+ if (mmu_has_feature(MMU_FTR_KUEP))
+ mtspr(SPRN_IAMR, AMR_KUEP_BLOCKED);
+ isync();
+ } else
+#endif
+ kuap_check_amr();
account_cpu_user_entry();
@@ -245,6 +263,10 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3,
account_cpu_user_exit();
+ /*
+ * We do this at the end so that we do context switch with KERNEL AMR
+ */
+ kuap_restore_user_amr(regs);
return ret;
}
@@ -330,6 +352,10 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned
account_cpu_user_exit();
+ /*
+ * We do this at the end so that we do context switch with KERNEL AMR
+ */
+ kuap_restore_user_amr(regs);
return ret;
}
@@ -400,7 +426,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsign
* which would cause Read-After-Write stalls. Hence, we take the AMR
* value from the check above.
*/
- kuap_restore_amr(regs, amr);
+ kuap_restore_kernel_amr(regs, amr);
return ret;
}
--
2.26.2
^ permalink raw reply related
* [PATCH v5 10/23] powerpc/exec: Set thread.regs early during exec
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
In later patches during exec, we would like to access default regs.kuap to
control access to the user mapping. Having thread.regs set early makes the
code changes simpler.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/thread_info.h | 2 --
arch/powerpc/kernel/process.c | 37 +++++++++++++++++---------
2 files changed, 25 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/include/asm/thread_info.h b/arch/powerpc/include/asm/thread_info.h
index ca6c97025704..9418dff1cfe1 100644
--- a/arch/powerpc/include/asm/thread_info.h
+++ b/arch/powerpc/include/asm/thread_info.h
@@ -77,10 +77,8 @@ struct thread_info {
/* how to get the thread information struct from C */
extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
-#ifdef CONFIG_PPC_BOOK3S_64
void arch_setup_new_exec(void);
#define arch_setup_new_exec arch_setup_new_exec
-#endif
#endif /* __ASSEMBLY__ */
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 016bd831908e..4633924ea77f 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1494,10 +1494,32 @@ void flush_thread(void)
#ifdef CONFIG_PPC_BOOK3S_64
void arch_setup_new_exec(void)
{
- if (radix_enabled())
- return;
- hash__setup_new_exec();
+ if (!radix_enabled())
+ hash__setup_new_exec();
+
+ /*
+ * If we exec out of a kernel thread then thread.regs will not be
+ * set. Do it now.
+ */
+ if (!current->thread.regs) {
+ struct pt_regs *regs = task_stack_page(current) + THREAD_SIZE;
+ current->thread.regs = regs - 1;
+ }
+
}
+#else
+void arch_setup_new_exec(void)
+{
+ /*
+ * If we exec out of a kernel thread then thread.regs will not be
+ * set. Do it now.
+ */
+ if (!current->thread.regs) {
+ struct pt_regs *regs = task_stack_page(current) + THREAD_SIZE;
+ current->thread.regs = regs - 1;
+ }
+}
+
#endif
#ifdef CONFIG_PPC64
@@ -1731,15 +1753,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
#endif
#endif
- /*
- * If we exec out of a kernel thread then thread.regs will not be
- * set. Do it now.
- */
- if (!current->thread.regs) {
- struct pt_regs *regs = task_stack_page(current) + THREAD_SIZE;
- current->thread.regs = regs - 1;
- }
-
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
/*
* Clear any transactional state, we're exec()ing. The cause is
--
2.26.2
^ permalink raw reply related
* [PATCH v5 08/23] powerpc/book3s64/kuap: Rename MMU_FTR_RADIX_KUAP to MMU_FTR_KUAP
From: Aneesh Kumar K.V @ 2020-08-27 4:09 UTC (permalink / raw)
To: linuxppc-dev, mpe; +Cc: Aneesh Kumar K.V
In-Reply-To: <20200827040931.297759-1-aneesh.kumar@linux.ibm.com>
This is in preparate to adding support for kuap with hash translation.
In preparation for that rename/move kuap related functions to
non radix names. Also move the feature bit closer to MMU_FTR_KUEP.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
arch/powerpc/include/asm/book3s/64/kup.h | 18 +++++++++---------
arch/powerpc/include/asm/mmu.h | 16 ++++++++--------
arch/powerpc/mm/book3s64/pkeys.c | 2 +-
3 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/64/kup.h b/arch/powerpc/include/asm/book3s/64/kup.h
index 918a2fcceee7..5cec202dc42f 100644
--- a/arch/powerpc/include/asm/book3s/64/kup.h
+++ b/arch/powerpc/include/asm/book3s/64/kup.h
@@ -24,7 +24,7 @@
mtspr SPRN_AMR, \gpr2
/* No isync required, see kuap_restore_amr() */
998:
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
#endif
.endm
@@ -36,7 +36,7 @@
sldi \gpr2, \gpr2, AMR_KUAP_SHIFT
999: tdne \gpr1, \gpr2
EMIT_BUG_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING | BUGFLAG_ONCE)
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
#endif
.endm
@@ -56,7 +56,7 @@
mtspr SPRN_AMR, \gpr2
isync
99:
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_RADIX_KUAP, 67)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
#endif
.endm
@@ -69,7 +69,7 @@
static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
{
- if (mmu_has_feature(MMU_FTR_RADIX_KUAP) && unlikely(regs->kuap != amr)) {
+ if (mmu_has_feature(MMU_FTR_KUAP) && unlikely(regs->kuap != amr)) {
isync();
mtspr(SPRN_AMR, regs->kuap);
/*
@@ -82,7 +82,7 @@ static inline void kuap_restore_amr(struct pt_regs *regs, unsigned long amr)
static inline unsigned long kuap_get_and_check_amr(void)
{
- if (mmu_has_feature(MMU_FTR_RADIX_KUAP)) {
+ if (mmu_has_feature(MMU_FTR_KUAP)) {
unsigned long amr = mfspr(SPRN_AMR);
if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG)) /* kuap_check_amr() */
WARN_ON_ONCE(amr != AMR_KUAP_BLOCKED);
@@ -93,7 +93,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
static inline void kuap_check_amr(void)
{
- if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && mmu_has_feature(MMU_FTR_RADIX_KUAP))
+ if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG) && mmu_has_feature(MMU_FTR_KUAP))
WARN_ON_ONCE(mfspr(SPRN_AMR) != AMR_KUAP_BLOCKED);
}
@@ -122,7 +122,7 @@ static inline unsigned long kuap_get_and_check_amr(void)
static inline unsigned long get_kuap(void)
{
- if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
+ if (!early_mmu_has_feature(MMU_FTR_KUAP))
return 0;
return mfspr(SPRN_AMR);
@@ -130,7 +130,7 @@ static inline unsigned long get_kuap(void)
static inline void set_kuap(unsigned long value)
{
- if (!early_mmu_has_feature(MMU_FTR_RADIX_KUAP))
+ if (!early_mmu_has_feature(MMU_FTR_KUAP))
return;
/*
@@ -180,7 +180,7 @@ static inline void restore_user_access(unsigned long flags)
static inline bool
bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
{
- return WARN(mmu_has_feature(MMU_FTR_RADIX_KUAP) &&
+ return WARN(mmu_has_feature(MMU_FTR_KUAP) &&
(regs->kuap & (is_write ? AMR_KUAP_BLOCK_WRITE : AMR_KUAP_BLOCK_READ)),
"Bug: %s fault blocked by AMR!", is_write ? "Write" : "Read");
}
diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
index 255a1837e9f7..04e7a65637fb 100644
--- a/arch/powerpc/include/asm/mmu.h
+++ b/arch/powerpc/include/asm/mmu.h
@@ -29,7 +29,12 @@
*/
/*
- * Support for KUEP feature.
+ * Supports KUAP (key 0 controlling userspace addresses) on radix
+ */
+#define MMU_FTR_KUAP ASM_CONST(0x00000200)
+
+/*
+ * Suppor for KUEP feature.
*/
#define MMU_FTR_KUEP ASM_CONST(0x00000400)
@@ -120,11 +125,6 @@
*/
#define MMU_FTR_1T_SEGMENT ASM_CONST(0x40000000)
-/*
- * Supports KUAP (key 0 controlling userspace addresses) on radix
- */
-#define MMU_FTR_RADIX_KUAP ASM_CONST(0x80000000)
-
/* MMU feature bit sets for various CPUs */
#define MMU_FTRS_DEFAULT_HPTE_ARCH_V2 \
MMU_FTR_HPTE_TABLE | MMU_FTR_PPCAS_ARCH_V2
@@ -187,10 +187,10 @@ enum {
#ifdef CONFIG_PPC_RADIX_MMU
MMU_FTR_TYPE_RADIX |
MMU_FTR_GTSE |
+#endif /* CONFIG_PPC_RADIX_MMU */
#ifdef CONFIG_PPC_KUAP
- MMU_FTR_RADIX_KUAP |
+ MMU_FTR_KUAP |
#endif /* CONFIG_PPC_KUAP */
-#endif /* CONFIG_PPC_RADIX_MMU */
#ifdef CONFIG_PPC_MEM_KEYS
MMU_FTR_PKEY |
#endif
diff --git a/arch/powerpc/mm/book3s64/pkeys.c b/arch/powerpc/mm/book3s64/pkeys.c
index 82c722fbce52..bfc27f1f0ab0 100644
--- a/arch/powerpc/mm/book3s64/pkeys.c
+++ b/arch/powerpc/mm/book3s64/pkeys.c
@@ -258,7 +258,7 @@ void __init setup_kuap(bool disabled)
if (smp_processor_id() == boot_cpuid) {
pr_info("Activating Kernel Userspace Access Prevention\n");
- cur_cpu_spec->mmu_features |= MMU_FTR_RADIX_KUAP;
+ cur_cpu_spec->mmu_features |= MMU_FTR_KUAP;
}
/*
--
2.26.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox