* Re: [PATCH 2/2] powerpc/64s/hash: add torture_hpt kernel boot option to increase hash faults
From: Aneesh Kumar K.V @ 2020-05-04 7:36 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20200503082236.17991-2-npiggin@gmail.com>
Nicholas Piggin <npiggin@gmail.com> writes:
> This option increases the number of hash misses by limiting the number of
> kernel HPT entries. This helps stress test difficult to hit paths in the
> kernel.
>
It would nice if we can explain in commit message how we are limiting
the number of HPT entries.
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
> .../admin-guide/kernel-parameters.txt | 9 +++
> arch/powerpc/include/asm/book3s/64/mmu-hash.h | 10 +++
> arch/powerpc/mm/book3s64/hash_4k.c | 3 +
> arch/powerpc/mm/book3s64/hash_64k.c | 8 +++
> arch/powerpc/mm/book3s64/hash_utils.c | 66 ++++++++++++++++++-
> 5 files changed, 95 insertions(+), 1 deletion(-)
....
> +void hpt_do_torture(unsigned long ea, unsigned long access,
> + unsigned long rflags, unsigned long hpte_group)
> +{
> + unsigned long last_group;
> + int cpu = raw_smp_processor_id();
> +
> + last_group = torture_hpt_last_group[cpu];
> + if (last_group != -1UL) {
> + while (mmu_hash_ops.hpte_remove(last_group) != -1)
> + ;
> + torture_hpt_last_group[cpu] = -1UL;
> + }
> +
> +#define QEMU_WORKAROUND 0
> +
> + if (ea >= PAGE_OFFSET) {
> + if (!QEMU_WORKAROUND && (access & (_PAGE_READ|_PAGE_WRITE)) &&
> + !(rflags & (HPTE_R_I|HPTE_R_G))) {
> + /* prefetch / prefetchw does not seem to set up a TLB
> + * entry with the powerpc systemsim (mambo) emulator,
> + * though it works with real hardware. An alternative
> + * approach that would work more reliably on quirky
> + * emulators like QEMU may be to remember the last
> + * insertion and remove that, rather than removing the
> + * current insertion. Then no prefetch is required.
> + */
> + if ((access & _PAGE_WRITE) && (access & _PAGE_READ))
> + atomic_add(0, (atomic_t *)(ea & ~0x3));
> + else if (access & _PAGE_READ)
> + *(volatile char *)ea;
> +
> + mb();
> +
> + while (mmu_hash_ops.hpte_remove(hpte_group) != -1)
> + ;
Do we get similar hpte faults rate, if we remove everything except the
current inserted entry?. If so that would largely simplify the code.
> + } else {
> + /* Can't prefetch cache-inhibited so clear next time. */
> + torture_hpt_last_group[cpu] = hpte_group;
> + }
> + }
> +}
> +
> +
> #ifdef CONFIG_DEBUG_PAGEALLOC
> static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi)
> {
> --
> 2.23.0
-aneesh
^ permalink raw reply
* Re: Build regressions/improvements in v5.7-rc4
From: Geert Uytterhoeven @ 2020-05-04 7:32 UTC (permalink / raw)
To: Linux Kernel Mailing List
Cc: Deucher, Alexander, Timothy Pearson, linuxppc-dev
In-Reply-To: <20200504072229.31214-1-geert@linux-m68k.org>
On Mon, May 4, 2020 at 9:24 AM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> JFYI, when comparing v5.7-rc4[1] to v5.7-rc3[3], the summaries are:
> - build errors: +3/-123
> [1] http://kisskb.ellerman.id.au/kisskb/branch/linus/head/0e698dfa282211e414076f9dc7e83c1c288314fd/ (all 239 configs)
> [3] http://kisskb.ellerman.id.au/kisskb/branch/linus/head/6a8b55ed4056ea5559ebe4f6a4b247f627870d4c/ (all 239 configs)
It's back:
+ /kisskb/src/drivers/gpu/drm/amd/amdgpu/../display/dc/calcs/dcn_calcs.c:
error: implicit declaration of function 'cpu_has_feature'
[-Werror=implicit-function-declaration]: => 626:2
+ /kisskb/src/drivers/gpu/drm/amd/amdgpu/../display/dc/calcs/dcn_calcs.c:
error: implicit declaration of function 'disable_kernel_vsx'
[-Werror=implicit-function-declaration]: => 662:2
+ /kisskb/src/drivers/gpu/drm/amd/amdgpu/../display/dc/calcs/dcn_calcs.c:
error: implicit declaration of function 'enable_kernel_vsx'
[-Werror=implicit-function-declaration]: => 626:2
powerpc-gcc4.6/ppc64_book3e_allmodconfig
powerpc-gcc9/ppc64_book3e_allmodconfig builds fine, as it doesn't have
DRM_AMD_DC_DCN, enabled due to:
select DRM_AMD_DC_DCN if (X86 || PPC64) && !(KCOV_INSTRUMENT_ALL
&& KCOV_ENABLE_COMPARISONS)
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH 1/2] powerpc/64s/hash: add torture_slb kernel boot option to increase SLB faults
From: Aneesh Kumar K.V @ 2020-05-04 7:27 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20200503082236.17991-1-npiggin@gmail.com>
Nicholas Piggin <npiggin@gmail.com> writes:
> This option increases the number of SLB misses by limiting the number of
> kernel SLB entries, and increased flushing of cached lookaside information.
> This helps stress test difficult to hit paths in the kernel.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
....
> +{
> + unsigned long slbie_data = get_paca()->slb_cache[index];
> + unsigned long ksp = get_paca()->kstack;
> +
> + slbie_data <<= SID_SHIFT;
> + slbie_data |= 0xc000000000000000ULL;
> + if ((ksp & slb_esid_mask(mmu_kernel_ssize)) == slbie_data)
> + return;
> + slbie_data |= mmu_kernel_ssize << SLBIE_SSIZE_SHIFT;
> +
> + asm volatile("slbie %0" : : "r" (slbie_data));
> +}
> +
> +static void slb_cache_slbie(unsigned int index)
May be slb_cache_slbie_user()? Similar to _kernel above?
> +{
> + unsigned long slbie_data = get_paca()->slb_cache[index];
> +
> + slbie_data <<= SID_SHIFT;
> + slbie_data |= user_segment_size(slbie_data) << SLBIE_SSIZE_SHIFT;
> + slbie_data |= SLBIE_C; /* user slbs have C=1 */
> +
> + asm volatile("slbie %0" : : "r" (slbie_data));
> +}
>
> /* Flush all user entries from the segment table of the current processor. */
> void switch_slb(struct task_struct *tsk, struct mm_struct *mm)
> @@ -414,8 +449,14 @@ void switch_slb(struct task_struct *tsk, struct mm_struct *mm)
> * which would update the slb_cache/slb_cache_ptr fields in the PACA.
> */
> hard_irq_disable();
> - asm volatile("isync" : : : "memory");
> - if (cpu_has_feature(CPU_FTR_ARCH_300)) {
> + isync();
> + if (torture_slb()) {
> + __slb_flush_and_restore_bolted(0);
s/0/SLIBA_IH_ALL or something like that?
> + isync();
> + get_paca()->slb_cache_ptr = 0;
> + get_paca()->slb_kern_bitmap = (1U << SLB_NUM_BOLTED) - 1;
> +
> + } else if (cpu_has_feature(CPU_FTR_ARCH_300)) {
> /*
> * SLBIA IH=3 invalidates all Class=1 SLBEs and their
> * associated lookaside structures, which matches what
> @@ -423,47 +464,36 @@ void switch_slb(struct task_struct *tsk, struct mm_struct *mm)
> * cache.
> */
> asm volatile(PPC_SLBIA(3));
-aneesh
^ permalink raw reply
* Re: [PATCH] powerpc/mce: Add MCE notification chain
From: Ganesh @ 2020-05-04 6:39 UTC (permalink / raw)
To: mpe, linuxppc-dev; +Cc: mahesh, santosh, arbab, npiggin, aneesh.kumar
In-Reply-To: <20200330071219.12284-1-ganeshgr@linux.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 470 bytes --]
On 3/30/20 12:42 PM, Ganesh Goudar wrote:
> From: Santosh S <santosh@fossix.org>
>
> Introduce notification chain which lets know about uncorrected memory
> errors(UE). This would help prospective users in pmem or nvdimm subsystem
> to track bad blocks for better handling of persistent memory allocations.
>
> Signed-off-by: Santosh S <santosh@fossix.org>
> Signed-off-by: Ganesh Goudar <ganeshgr@linux.ibm.com>
> ---
Hi mpe, Do you have any comments on this patch?
[-- Attachment #2: Type: text/html, Size: 1030 bytes --]
^ permalink raw reply
* Re: [PATCH v7 06/28] powerpc: Use a macro for creating instructions from u32s
From: Alistair Popple @ 2020-05-04 5:54 UTC (permalink / raw)
To: Jordan Niethe; +Cc: npiggin, bala24, naveen.n.rao, linuxppc-dev, dja
In-Reply-To: <20200501034220.8982-7-jniethe5@gmail.com>
I haven't reviewed our existing code to figure out if we've caught all places a
unsigned int is used as an instruction, but what's here looks correct and I
assume any missed cases will likely get caught by the compiler when the
datatype is actually introduced. At least past experience trying to build this
series suggests that assumption is reasonably accurate :-)
Michael - this requires a minor fix up when merged on top of Ravi's 2nd DAWR
series, let me know if you need it.
Reviewed-by: Alistair Popple <alistair@popple.id.au>
On Friday, 1 May 2020 1:41:58 PM AEST Jordan Niethe wrote:
> In preparation for instructions having a more complex data type start
> using a macro, ppc_inst(), for making an instruction out of a u32. A
> macro is used so that instructions can be used as initializer elements.
> Currently this does nothing, but it will allow for creating a data type
> that can represent prefixed instructions.
>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v4: New to series
> v5: - Rename PPC_INST() -> ppc_inst().
> - Use on epapr_paravirt.c, kgdb.c
> v6: - Use in setup_32.c
> - epapr_paravirt.c: early_init_dt_scan_epapr(): move the use of
> ppc_inst() earlier.
> ---
> arch/powerpc/include/asm/code-patching.h | 3 +-
> arch/powerpc/include/asm/inst.h | 11 +++++
> arch/powerpc/kernel/align.c | 1 +
> arch/powerpc/kernel/epapr_paravirt.c | 3 +-
> arch/powerpc/kernel/hw_breakpoint.c | 3 +-
> arch/powerpc/kernel/jump_label.c | 3 +-
> arch/powerpc/kernel/kgdb.c | 5 ++-
> arch/powerpc/kernel/kprobes.c | 5 ++-
> arch/powerpc/kernel/module_64.c | 3 +-
> arch/powerpc/kernel/optprobes.c | 31 ++++++-------
> arch/powerpc/kernel/security.c | 9 ++--
> arch/powerpc/kernel/setup_32.c | 2 +-
> arch/powerpc/kernel/trace/ftrace.c | 25 ++++++-----
> arch/powerpc/kernel/uprobes.c | 1 +
> arch/powerpc/kvm/emulate_loadstore.c | 2 +-
> arch/powerpc/lib/code-patching.c | 57 ++++++++++++------------
> arch/powerpc/lib/feature-fixups.c | 39 ++++++++--------
> arch/powerpc/lib/test_emulate_step.c | 39 ++++++++--------
> arch/powerpc/xmon/xmon.c | 7 +--
> 19 files changed, 138 insertions(+), 111 deletions(-)
> create mode 100644 arch/powerpc/include/asm/inst.h
>
> diff --git a/arch/powerpc/include/asm/code-patching.h
> b/arch/powerpc/include/asm/code-patching.h index 351dda7215b6..48e021957ee5
> 100644
> --- a/arch/powerpc/include/asm/code-patching.h
> +++ b/arch/powerpc/include/asm/code-patching.h
> @@ -11,6 +11,7 @@
> #include <linux/string.h>
> #include <linux/kallsyms.h>
> #include <asm/asm-compat.h>
> +#include <asm/inst.h>
>
> /* Flags for create_branch:
> * "b" == create_branch(addr, target, 0);
> @@ -48,7 +49,7 @@ static inline int patch_branch_site(s32 *site, unsigned
> long target, int flags) static inline int modify_instruction(unsigned int
> *addr, unsigned int clr, unsigned int set)
> {
> - return patch_instruction(addr, (*addr & ~clr) | set);
> + return patch_instruction(addr, ppc_inst((*addr & ~clr) | set));
> }
>
> static inline int modify_instruction_site(s32 *site, unsigned int clr,
> unsigned int set) diff --git a/arch/powerpc/include/asm/inst.h
> b/arch/powerpc/include/asm/inst.h new file mode 100644
> index 000000000000..5298ba33b6e5
> --- /dev/null
> +++ b/arch/powerpc/include/asm/inst.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +#ifndef _ASM_INST_H
> +#define _ASM_INST_H
> +
> +/*
> + * Instruction data type for POWER
> + */
> +
> +#define ppc_inst(x) (x)
> +
> +#endif /* _ASM_INST_H */
> diff --git a/arch/powerpc/kernel/align.c b/arch/powerpc/kernel/align.c
> index 92045ed64976..86e9bf62f18c 100644
> --- a/arch/powerpc/kernel/align.c
> +++ b/arch/powerpc/kernel/align.c
> @@ -24,6 +24,7 @@
> #include <asm/disassemble.h>
> #include <asm/cpu_has_feature.h>
> #include <asm/sstep.h>
> +#include <asm/inst.h>
>
> struct aligninfo {
> unsigned char len;
> diff --git a/arch/powerpc/kernel/epapr_paravirt.c
> b/arch/powerpc/kernel/epapr_paravirt.c index 9d32158ce36f..e8eb72a65572
> 100644
> --- a/arch/powerpc/kernel/epapr_paravirt.c
> +++ b/arch/powerpc/kernel/epapr_paravirt.c
> @@ -11,6 +11,7 @@
> #include <asm/cacheflush.h>
> #include <asm/code-patching.h>
> #include <asm/machdep.h>
> +#include <asm/inst.h>
>
> #if !defined(CONFIG_64BIT) || defined(CONFIG_PPC_BOOK3E_64)
> extern void epapr_ev_idle(void);
> @@ -36,7 +37,7 @@ static int __init early_init_dt_scan_epapr(unsigned long
> node, return -1;
>
> for (i = 0; i < (len / 4); i++) {
> - u32 inst = be32_to_cpu(insts[i]);
> + u32 inst = ppc_inst(be32_to_cpu(insts[i]));
> patch_instruction(epapr_hypercall_start + i, inst);
> #if !defined(CONFIG_64BIT) || defined(CONFIG_PPC_BOOK3E_64)
> patch_instruction(epapr_ev_idle_start + i, inst);
> diff --git a/arch/powerpc/kernel/hw_breakpoint.c
> b/arch/powerpc/kernel/hw_breakpoint.c index 72f461bd70fb..46e09ac8b84a
> 100644
> --- a/arch/powerpc/kernel/hw_breakpoint.c
> +++ b/arch/powerpc/kernel/hw_breakpoint.c
> @@ -24,6 +24,7 @@
> #include <asm/debug.h>
> #include <asm/debugfs.h>
> #include <asm/hvcall.h>
> +#include <asm/inst.h>
> #include <linux/uaccess.h>
>
> /*
> @@ -243,7 +244,7 @@ dar_range_overlaps(unsigned long dar, int size, struct
> arch_hw_breakpoint *info) static bool stepping_handler(struct pt_regs
> *regs, struct perf_event *bp, struct arch_hw_breakpoint *info)
> {
> - unsigned int instr = 0;
> + unsigned int instr = ppc_inst(0);
> int ret, type, size;
> struct instruction_op op;
> unsigned long addr = info->address;
> diff --git a/arch/powerpc/kernel/jump_label.c
> b/arch/powerpc/kernel/jump_label.c index ca37702bde97..daa4afce7ec8 100644
> --- a/arch/powerpc/kernel/jump_label.c
> +++ b/arch/powerpc/kernel/jump_label.c
> @@ -6,6 +6,7 @@
> #include <linux/kernel.h>
> #include <linux/jump_label.h>
> #include <asm/code-patching.h>
> +#include <asm/inst.h>
>
> void arch_jump_label_transform(struct jump_entry *entry,
> enum jump_label_type type)
> @@ -15,5 +16,5 @@ void arch_jump_label_transform(struct jump_entry *entry,
> if (type == JUMP_LABEL_JMP)
> patch_branch(addr, entry->target, 0);
> else
> - patch_instruction(addr, PPC_INST_NOP);
> + patch_instruction(addr, ppc_inst(PPC_INST_NOP));
> }
> diff --git a/arch/powerpc/kernel/kgdb.c b/arch/powerpc/kernel/kgdb.c
> index 7dd55eb1259d..a6b38a19133f 100644
> --- a/arch/powerpc/kernel/kgdb.c
> +++ b/arch/powerpc/kernel/kgdb.c
> @@ -26,6 +26,7 @@
> #include <asm/debug.h>
> #include <asm/code-patching.h>
> #include <linux/slab.h>
> +#include <asm/inst.h>
>
> /*
> * This table contains the mapping between PowerPC hardware trap types, and
> @@ -424,7 +425,7 @@ int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) if
> (err)
> return err;
>
> - err = patch_instruction(addr, BREAK_INSTR);
> + err = patch_instruction(addr, ppc_inst(BREAK_INSTR));
> if (err)
> return -EFAULT;
>
> @@ -439,7 +440,7 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt)
> unsigned int instr = *(unsigned int *)bpt->saved_instr;
> unsigned int *addr = (unsigned int *)bpt->bpt_addr;
>
> - err = patch_instruction(addr, instr);
> + err = patch_instruction(addr, ppc_inst(instr));
> if (err)
> return -EFAULT;
>
> diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c
> index 81efb605113e..2378a7ed4438 100644
> --- a/arch/powerpc/kernel/kprobes.c
> +++ b/arch/powerpc/kernel/kprobes.c
> @@ -23,6 +23,7 @@
> #include <asm/cacheflush.h>
> #include <asm/sstep.h>
> #include <asm/sections.h>
> +#include <asm/inst.h>
> #include <linux/uaccess.h>
>
> DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
> @@ -138,13 +139,13 @@ NOKPROBE_SYMBOL(arch_prepare_kprobe);
>
> void arch_arm_kprobe(struct kprobe *p)
> {
> - patch_instruction(p->addr, BREAKPOINT_INSTRUCTION);
> + patch_instruction(p->addr, ppc_inst(BREAKPOINT_INSTRUCTION));
> }
> NOKPROBE_SYMBOL(arch_arm_kprobe);
>
> void arch_disarm_kprobe(struct kprobe *p)
> {
> - patch_instruction(p->addr, p->opcode);
> + patch_instruction(p->addr, ppc_inst(p->opcode));
> }
> NOKPROBE_SYMBOL(arch_disarm_kprobe);
>
> diff --git a/arch/powerpc/kernel/module_64.c
> b/arch/powerpc/kernel/module_64.c index 007606a48fd9..7fd6b29edcb2 100644
> --- a/arch/powerpc/kernel/module_64.c
> +++ b/arch/powerpc/kernel/module_64.c
> @@ -20,6 +20,7 @@
> #include <linux/sort.h>
> #include <asm/setup.h>
> #include <asm/sections.h>
> +#include <asm/inst.h>
>
> /* FIXME: We don't do .init separately. To do this, we'd need to have
> a separate r2 value in the init and core section, and stub between
> @@ -506,7 +507,7 @@ static int restore_r2(const char *name, u32
> *instruction, struct module *me) * "link" branches and they don't return,
> so they don't need the r2 * restore afterwards.
> */
> - if (!instr_is_relative_link_branch(*prev_insn))
> + if (!instr_is_relative_link_branch(ppc_inst(*prev_insn)))
> return 1;
>
> if (*instruction != PPC_INST_NOP) {
> diff --git a/arch/powerpc/kernel/optprobes.c
> b/arch/powerpc/kernel/optprobes.c index 445b3dad82dc..3b33ebf18859 100644
> --- a/arch/powerpc/kernel/optprobes.c
> +++ b/arch/powerpc/kernel/optprobes.c
> @@ -16,6 +16,7 @@
> #include <asm/code-patching.h>
> #include <asm/sstep.h>
> #include <asm/ppc-opcode.h>
> +#include <asm/inst.h>
>
> #define TMPL_CALL_HDLR_IDX \
> (optprobe_template_call_handler - optprobe_template_entry)
> @@ -147,13 +148,13 @@ void arch_remove_optimized_kprobe(struct
> optimized_kprobe *op) void patch_imm32_load_insns(unsigned int val,
> kprobe_opcode_t *addr) {
> /* addis r4,0,(insn)@h */
> - patch_instruction(addr, PPC_INST_ADDIS | ___PPC_RT(4) |
> - ((val >> 16) & 0xffff));
> + patch_instruction(addr, ppc_inst(PPC_INST_ADDIS | ___PPC_RT(4) |
> + ((val >> 16) & 0xffff)));
> addr++;
>
> /* ori r4,r4,(insn)@l */
> - patch_instruction(addr, PPC_INST_ORI | ___PPC_RA(4) |
> - ___PPC_RS(4) | (val & 0xffff));
> + patch_instruction(addr, ppc_inst(PPC_INST_ORI | ___PPC_RA(4) |
> + ___PPC_RS(4) | (val & 0xffff)));
> }
>
> /*
> @@ -163,28 +164,28 @@ void patch_imm32_load_insns(unsigned int val,
> kprobe_opcode_t *addr) void patch_imm64_load_insns(unsigned long val,
> kprobe_opcode_t *addr) {
> /* lis r3,(op)@highest */
> - patch_instruction(addr, PPC_INST_ADDIS | ___PPC_RT(3) |
> - ((val >> 48) & 0xffff));
> + patch_instruction(addr, ppc_inst(PPC_INST_ADDIS | ___PPC_RT(3) |
> + ((val >> 48) & 0xffff)));
> addr++;
>
> /* ori r3,r3,(op)@higher */
> - patch_instruction(addr, PPC_INST_ORI | ___PPC_RA(3) |
> - ___PPC_RS(3) | ((val >> 32) & 0xffff));
> + patch_instruction(addr, ppc_inst(PPC_INST_ORI | ___PPC_RA(3) |
> + ___PPC_RS(3) | ((val >> 32) & 0xffff)));
> addr++;
>
> /* rldicr r3,r3,32,31 */
> - patch_instruction(addr, PPC_INST_RLDICR | ___PPC_RA(3) |
> - ___PPC_RS(3) | __PPC_SH64(32) | __PPC_ME64(31));
> + patch_instruction(addr, ppc_inst(PPC_INST_RLDICR | ___PPC_RA(3) |
> + ___PPC_RS(3) | __PPC_SH64(32) | __PPC_ME64(31)));
> addr++;
>
> /* oris r3,r3,(op)@h */
> - patch_instruction(addr, PPC_INST_ORIS | ___PPC_RA(3) |
> - ___PPC_RS(3) | ((val >> 16) & 0xffff));
> + patch_instruction(addr, ppc_inst(PPC_INST_ORIS | ___PPC_RA(3) |
> + ___PPC_RS(3) | ((val >> 16) & 0xffff)));
> addr++;
>
> /* ori r3,r3,(op)@l */
> - patch_instruction(addr, PPC_INST_ORI | ___PPC_RA(3) |
> - ___PPC_RS(3) | (val & 0xffff));
> + patch_instruction(addr, ppc_inst(PPC_INST_ORI | ___PPC_RA(3) |
> + ___PPC_RS(3) | (val & 0xffff)));
> }
>
> int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct
> kprobe *p) @@ -230,7 +231,7 @@ int arch_prepare_optimized_kprobe(struct
> optimized_kprobe *op, struct kprobe *p) size = (TMPL_END_IDX *
> sizeof(kprobe_opcode_t)) / sizeof(int);
> pr_devel("Copying template to %p, size %lu\n", buff, size);
> for (i = 0; i < size; i++) {
> - rc = patch_instruction(buff + i, *(optprobe_template_entry + i));
> + rc = patch_instruction(buff + i, ppc_inst(*(optprobe_template_entry +
> i))); if (rc < 0)
> goto error;
> }
> diff --git a/arch/powerpc/kernel/security.c b/arch/powerpc/kernel/security.c
> index bd70f5be1c27..81a288b1a603 100644
> --- a/arch/powerpc/kernel/security.c
> +++ b/arch/powerpc/kernel/security.c
> @@ -14,6 +14,7 @@
> #include <asm/debugfs.h>
> #include <asm/security_features.h>
> #include <asm/setup.h>
> +#include <asm/inst.h>
>
>
> u64 powerpc_security_features __read_mostly = SEC_FTR_DEFAULT;
> @@ -403,9 +404,9 @@ static void toggle_count_cache_flush(bool enable)
> enable = false;
>
> if (!enable) {
> - patch_instruction_site(&patch__call_flush_count_cache, PPC_INST_NOP);
> + patch_instruction_site(&patch__call_flush_count_cache,
> ppc_inst(PPC_INST_NOP)); #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
> - patch_instruction_site(&patch__call_kvm_flush_link_stack,
PPC_INST_NOP);
> + patch_instruction_site(&patch__call_kvm_flush_link_stack,
> ppc_inst(PPC_INST_NOP)); #endif
> pr_info("link-stack-flush: software flush disabled.\n");
> link_stack_flush_enabled = false;
> @@ -428,7 +429,7 @@ static void toggle_count_cache_flush(bool enable)
>
> // If we just need to flush the link stack, patch an early return
> if (!security_ftr_enabled(SEC_FTR_FLUSH_COUNT_CACHE)) {
> - patch_instruction_site(&patch__flush_link_stack_return, PPC_INST_BLR);
> + patch_instruction_site(&patch__flush_link_stack_return,
> ppc_inst(PPC_INST_BLR)); no_count_cache_flush();
> return;
> }
> @@ -439,7 +440,7 @@ static void toggle_count_cache_flush(bool enable)
> return;
> }
>
> - patch_instruction_site(&patch__flush_count_cache_return, PPC_INST_BLR);
> + patch_instruction_site(&patch__flush_count_cache_return,
> ppc_inst(PPC_INST_BLR)); count_cache_flush_type = COUNT_CACHE_FLUSH_HW;
> pr_info("count-cache-flush: hardware assisted flush sequence enabled\n");
> }
> diff --git a/arch/powerpc/kernel/setup_32.c b/arch/powerpc/kernel/setup_32.c
> index 3a43e8e847c8..0536e4aed330 100644
> --- a/arch/powerpc/kernel/setup_32.c
> +++ b/arch/powerpc/kernel/setup_32.c
> @@ -85,7 +85,7 @@ notrace void __init machine_init(u64 dt_ptr)
> /* Enable early debugging if any specified (see udbg.h) */
> udbg_early_init();
>
> - patch_instruction_site(&patch__memcpy_nocache, PPC_INST_NOP);
> + patch_instruction_site(&patch__memcpy_nocache, ppc_inst(PPC_INST_NOP));
>
> create_cond_branch(&insn, addr, branch_target(addr), 0x820000);
> patch_instruction(addr, insn); /* replace b by bne cr0 */
> diff --git a/arch/powerpc/kernel/trace/ftrace.c
> b/arch/powerpc/kernel/trace/ftrace.c index 828c5f64ca1e..0318e1ed6248
> 100644
> --- a/arch/powerpc/kernel/trace/ftrace.c
> +++ b/arch/powerpc/kernel/trace/ftrace.c
> @@ -27,6 +27,7 @@
> #include <asm/code-patching.h>
> #include <asm/ftrace.h>
> #include <asm/syscall.h>
> +#include <asm/inst.h>
>
>
> #ifdef CONFIG_DYNAMIC_FTRACE
> @@ -161,7 +162,7 @@ __ftrace_make_nop(struct module *mod,
>
> #ifdef CONFIG_MPROFILE_KERNEL
> /* When using -mkernel_profile there is no load to jump over */
> - pop = PPC_INST_NOP;
> + pop = ppc_inst(PPC_INST_NOP);
>
> if (probe_kernel_read(&op, (void *)(ip - 4), 4)) {
> pr_err("Fetching instruction at %lx failed.\n", ip - 4);
> @@ -169,7 +170,7 @@ __ftrace_make_nop(struct module *mod,
> }
>
> /* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
> - if (op != PPC_INST_MFLR && op != PPC_INST_STD_LR) {
> + if (op != ppc_inst(PPC_INST_MFLR) && op != ppc_inst(PPC_INST_STD_LR)) {
> pr_err("Unexpected instruction %08x around bl _mcount\n", op);
> return -EINVAL;
> }
> @@ -188,7 +189,7 @@ __ftrace_make_nop(struct module *mod,
> * Use a b +8 to jump over the load.
> */
>
> - pop = PPC_INST_BRANCH | 8; /* b +8 */
> + pop = ppc_inst(PPC_INST_BRANCH | 8); /* b +8 */
>
> /*
> * Check what is in the next instruction. We can see ld r2,40(r1), but
> @@ -199,7 +200,7 @@ __ftrace_make_nop(struct module *mod,
> return -EFAULT;
> }
>
> - if (op != PPC_INST_LD_TOC) {
> + if (op != ppc_inst(PPC_INST_LD_TOC)) {
> pr_err("Expected %08x found %08x\n", PPC_INST_LD_TOC, op);
> return -EINVAL;
> }
> @@ -275,7 +276,7 @@ __ftrace_make_nop(struct module *mod,
> return -EINVAL;
> }
>
> - op = PPC_INST_NOP;
> + op = ppc_inst(PPC_INST_NOP);
>
> if (patch_instruction((unsigned int *)ip, op))
> return -EPERM;
> @@ -420,7 +421,7 @@ static int __ftrace_make_nop_kernel(struct dyn_ftrace
> *rec, unsigned long addr) }
> }
>
> - if (patch_instruction((unsigned int *)ip, PPC_INST_NOP)) {
> + if (patch_instruction((unsigned int *)ip, ppc_inst(PPC_INST_NOP))) {
> pr_err("Patching NOP failed.\n");
> return -EPERM;
> }
> @@ -442,7 +443,7 @@ int ftrace_make_nop(struct module *mod,
> if (test_24bit_addr(ip, addr)) {
> /* within range */
> old = ftrace_call_replace(ip, addr, 1);
> - new = PPC_INST_NOP;
> + new = ppc_inst(PPC_INST_NOP);
> return ftrace_modify_code(ip, old, new);
> } else if (core_kernel_text(ip))
> return __ftrace_make_nop_kernel(rec, addr);
> @@ -496,7 +497,7 @@ expected_nop_sequence(void *ip, unsigned int op0,
> unsigned int op1) * The load offset is different depending on the ABI. For
> simplicity * just mask it out when doing the compare.
> */
> - if ((op0 != 0x48000008) || ((op1 & 0xffff0000) != 0xe8410000))
> + if ((op0 != ppc_inst(0x48000008)) || ((op1 & 0xffff0000) != 0xe8410000))
> return 0;
> return 1;
> }
> @@ -505,7 +506,7 @@ static int
> expected_nop_sequence(void *ip, unsigned int op0, unsigned int op1)
> {
> /* look for patched "NOP" on ppc64 with -mprofile-kernel */
> - if (op0 != PPC_INST_NOP)
> + if (op0 != ppc_inst(PPC_INST_NOP))
> return 0;
> return 1;
> }
> @@ -588,7 +589,7 @@ __ftrace_make_call(struct dyn_ftrace *rec, unsigned long
> addr) return -EFAULT;
>
> /* It should be pointing to a nop */
> - if (op != PPC_INST_NOP) {
> + if (op != ppc_inst(PPC_INST_NOP)) {
> pr_err("Expected NOP but have %x\n", op);
> return -EINVAL;
> }
> @@ -645,7 +646,7 @@ static int __ftrace_make_call_kernel(struct dyn_ftrace
> *rec, unsigned long addr) return -EFAULT;
> }
>
> - if (op != PPC_INST_NOP) {
> + if (op != ppc_inst(PPC_INST_NOP)) {
> pr_err("Unexpected call sequence at %p: %x\n", ip, op);
> return -EINVAL;
> }
> @@ -676,7 +677,7 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned
> long addr) */
> if (test_24bit_addr(ip, addr)) {
> /* within range */
> - old = PPC_INST_NOP;
> + old = ppc_inst(PPC_INST_NOP);
> new = ftrace_call_replace(ip, addr, 1);
> return ftrace_modify_code(ip, old, new);
> } else if (core_kernel_text(ip))
> diff --git a/arch/powerpc/kernel/uprobes.c b/arch/powerpc/kernel/uprobes.c
> index 1cfef0e5fec5..31c870287f2b 100644
> --- a/arch/powerpc/kernel/uprobes.c
> +++ b/arch/powerpc/kernel/uprobes.c
> @@ -14,6 +14,7 @@
> #include <linux/kdebug.h>
>
> #include <asm/sstep.h>
> +#include <asm/inst.h>
>
> #define UPROBE_TRAP_NR UINT_MAX
>
> diff --git a/arch/powerpc/kvm/emulate_loadstore.c
> b/arch/powerpc/kvm/emulate_loadstore.c index 1139bc56e004..135d0e686622
> 100644
> --- a/arch/powerpc/kvm/emulate_loadstore.c
> +++ b/arch/powerpc/kvm/emulate_loadstore.c
> @@ -95,7 +95,7 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
>
> emulated = EMULATE_FAIL;
> vcpu->arch.regs.msr = vcpu->arch.shared->msr;
> - if (analyse_instr(&op, &vcpu->arch.regs, inst) == 0) {
> + if (analyse_instr(&op, &vcpu->arch.regs, ppc_inst(inst)) == 0) {
> int type = op.type & INSTR_TYPE_MASK;
> int size = GETSIZE(op.type);
>
> diff --git a/arch/powerpc/lib/code-patching.c
> b/arch/powerpc/lib/code-patching.c index 6ed3301c0582..6c30ddadd971 100644
> --- a/arch/powerpc/lib/code-patching.c
> +++ b/arch/powerpc/lib/code-patching.c
> @@ -17,6 +17,7 @@
> #include <asm/page.h>
> #include <asm/code-patching.h>
> #include <asm/setup.h>
> +#include <asm/inst.h>
>
> static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
> unsigned int *patch_addr)
> @@ -414,37 +415,37 @@ static void __init test_branch_iform(void)
> addr = (unsigned long)&instr;
>
> /* The simplest case, branch to self, no flags */
> - check(instr_is_branch_iform(0x48000000));
> + check(instr_is_branch_iform(ppc_inst(0x48000000)));
> /* All bits of target set, and flags */
> - check(instr_is_branch_iform(0x4bffffff));
> + check(instr_is_branch_iform(ppc_inst(0x4bffffff)));
> /* High bit of opcode set, which is wrong */
> - check(!instr_is_branch_iform(0xcbffffff));
> + check(!instr_is_branch_iform(ppc_inst(0xcbffffff)));
> /* Middle bits of opcode set, which is wrong */
> - check(!instr_is_branch_iform(0x7bffffff));
> + check(!instr_is_branch_iform(ppc_inst(0x7bffffff)));
>
> /* Simplest case, branch to self with link */
> - check(instr_is_branch_iform(0x48000001));
> + check(instr_is_branch_iform(ppc_inst(0x48000001)));
> /* All bits of targets set */
> - check(instr_is_branch_iform(0x4bfffffd));
> + check(instr_is_branch_iform(ppc_inst(0x4bfffffd)));
> /* Some bits of targets set */
> - check(instr_is_branch_iform(0x4bff00fd));
> + check(instr_is_branch_iform(ppc_inst(0x4bff00fd)));
> /* Must be a valid branch to start with */
> - check(!instr_is_branch_iform(0x7bfffffd));
> + check(!instr_is_branch_iform(ppc_inst(0x7bfffffd)));
>
> /* Absolute branch to 0x100 */
> - instr = 0x48000103;
> + instr = ppc_inst(0x48000103);
> check(instr_is_branch_to_addr(&instr, 0x100));
> /* Absolute branch to 0x420fc */
> - instr = 0x480420ff;
> + instr = ppc_inst(0x480420ff);
> check(instr_is_branch_to_addr(&instr, 0x420fc));
> /* Maximum positive relative branch, + 20MB - 4B */
> - instr = 0x49fffffc;
> + instr = ppc_inst(0x49fffffc);
> check(instr_is_branch_to_addr(&instr, addr + 0x1FFFFFC));
> /* Smallest negative relative branch, - 4B */
> - instr = 0x4bfffffc;
> + instr = ppc_inst(0x4bfffffc);
> check(instr_is_branch_to_addr(&instr, addr - 4));
> /* Largest negative relative branch, - 32 MB */
> - instr = 0x4a000000;
> + instr = ppc_inst(0x4a000000);
> check(instr_is_branch_to_addr(&instr, addr - 0x2000000));
>
> /* Branch to self, with link */
> @@ -478,7 +479,7 @@ static void __init test_branch_iform(void)
> /* Check flags are masked correctly */
> err = create_branch(&instr, &instr, addr, 0xFFFFFFFC);
> check(instr_is_branch_to_addr(&instr, addr));
> - check(instr == 0x48000000);
> + check(instr == ppc_inst(0x48000000));
> }
>
> static void __init test_create_function_call(void)
> @@ -505,28 +506,28 @@ static void __init test_branch_bform(void)
> addr = (unsigned long)iptr;
>
> /* The simplest case, branch to self, no flags */
> - check(instr_is_branch_bform(0x40000000));
> + check(instr_is_branch_bform(ppc_inst(0x40000000)));
> /* All bits of target set, and flags */
> - check(instr_is_branch_bform(0x43ffffff));
> + check(instr_is_branch_bform(ppc_inst(0x43ffffff)));
> /* High bit of opcode set, which is wrong */
> - check(!instr_is_branch_bform(0xc3ffffff));
> + check(!instr_is_branch_bform(ppc_inst(0xc3ffffff)));
> /* Middle bits of opcode set, which is wrong */
> - check(!instr_is_branch_bform(0x7bffffff));
> + check(!instr_is_branch_bform(ppc_inst(0x7bffffff)));
>
> /* Absolute conditional branch to 0x100 */
> - instr = 0x43ff0103;
> + instr = ppc_inst(0x43ff0103);
> check(instr_is_branch_to_addr(&instr, 0x100));
> /* Absolute conditional branch to 0x20fc */
> - instr = 0x43ff20ff;
> + instr = ppc_inst(0x43ff20ff);
> check(instr_is_branch_to_addr(&instr, 0x20fc));
> /* Maximum positive relative conditional branch, + 32 KB - 4B */
> - instr = 0x43ff7ffc;
> + instr = ppc_inst(0x43ff7ffc);
> check(instr_is_branch_to_addr(&instr, addr + 0x7FFC));
> /* Smallest negative relative conditional branch, - 4B */
> - instr = 0x43fffffc;
> + instr = ppc_inst(0x43fffffc);
> check(instr_is_branch_to_addr(&instr, addr - 4));
> /* Largest negative relative conditional branch, - 32 KB */
> - instr = 0x43ff8000;
> + instr = ppc_inst(0x43ff8000);
> check(instr_is_branch_to_addr(&instr, addr - 0x8000));
>
> /* All condition code bits set & link */
> @@ -563,7 +564,7 @@ static void __init test_branch_bform(void)
> /* Check flags are masked correctly */
> err = create_cond_branch(&instr, iptr, addr, 0xFFFFFFFC);
> check(instr_is_branch_to_addr(&instr, addr));
> - check(instr == 0x43FF0000);
> + check(instr == ppc_inst(0x43FF0000));
> }
>
> static void __init test_translate_branch(void)
> @@ -597,7 +598,7 @@ static void __init test_translate_branch(void)
> patch_instruction(q, instr);
> check(instr_is_branch_to_addr(p, addr));
> check(instr_is_branch_to_addr(q, addr));
> - check(*q == 0x4a000000);
> + check(*q == ppc_inst(0x4a000000));
>
> /* Maximum positive case, move x to x - 32 MB + 4 */
> p = buf + 0x2000000;
> @@ -608,7 +609,7 @@ static void __init test_translate_branch(void)
> patch_instruction(q, instr);
> check(instr_is_branch_to_addr(p, addr));
> check(instr_is_branch_to_addr(q, addr));
> - check(*q == 0x49fffffc);
> + check(*q == ppc_inst(0x49fffffc));
>
> /* Jump to x + 16 MB moved to x + 20 MB */
> p = buf;
> @@ -654,7 +655,7 @@ static void __init test_translate_branch(void)
> patch_instruction(q, instr);
> check(instr_is_branch_to_addr(p, addr));
> check(instr_is_branch_to_addr(q, addr));
> - check(*q == 0x43ff8000);
> + check(*q == ppc_inst(0x43ff8000));
>
> /* Maximum positive case, move x to x - 32 KB + 4 */
> p = buf + 0x8000;
> @@ -666,7 +667,7 @@ static void __init test_translate_branch(void)
> patch_instruction(q, instr);
> check(instr_is_branch_to_addr(p, addr));
> check(instr_is_branch_to_addr(q, addr));
> - check(*q == 0x43ff7ffc);
> + check(*q == ppc_inst(0x43ff7ffc));
>
> /* Jump to x + 12 KB moved to x + 20 KB */
> p = buf;
> diff --git a/arch/powerpc/lib/feature-fixups.c
> b/arch/powerpc/lib/feature-fixups.c index b129d7b4e7dd..6e7479b8887a 100644
> --- a/arch/powerpc/lib/feature-fixups.c
> +++ b/arch/powerpc/lib/feature-fixups.c
> @@ -21,6 +21,7 @@
> #include <asm/setup.h>
> #include <asm/security_features.h>
> #include <asm/firmware.h>
> +#include <asm/inst.h>
>
> struct fixup_entry {
> unsigned long mask;
> @@ -89,7 +90,7 @@ static int patch_feature_section(unsigned long value,
> struct fixup_entry *fcur) }
>
> for (; dest < end; dest++)
> - raw_patch_instruction(dest, PPC_INST_NOP);
> + raw_patch_instruction(dest, ppc_inst(PPC_INST_NOP));
>
> return 0;
> }
> @@ -146,15 +147,15 @@ static void do_stf_entry_barrier_fixups(enum
> stf_barrier_type types)
>
> pr_devel("patching dest %lx\n", (unsigned long)dest);
>
> - patch_instruction(dest, instrs[0]);
> + patch_instruction(dest, ppc_inst(instrs[0]));
>
> if (types & STF_BARRIER_FALLBACK)
> patch_branch(dest + 1, (unsigned long)&stf_barrier_fallback,
> BRANCH_SET_LINK);
> else
> - patch_instruction(dest + 1, instrs[1]);
> + patch_instruction(dest + 1, ppc_inst(instrs[1]));
>
> - patch_instruction(dest + 2, instrs[2]);
> + patch_instruction(dest + 2, ppc_inst(instrs[2]));
> }
>
> printk(KERN_DEBUG "stf-barrier: patched %d entry locations (%s
> barrier)\n", i, @@ -207,12 +208,12 @@ static void
> do_stf_exit_barrier_fixups(enum stf_barrier_type types)
>
> pr_devel("patching dest %lx\n", (unsigned long)dest);
>
> - patch_instruction(dest, instrs[0]);
> - patch_instruction(dest + 1, instrs[1]);
> - patch_instruction(dest + 2, instrs[2]);
> - patch_instruction(dest + 3, instrs[3]);
> - patch_instruction(dest + 4, instrs[4]);
> - patch_instruction(dest + 5, instrs[5]);
> + patch_instruction(dest, ppc_inst(instrs[0]));
> + patch_instruction(dest + 1, ppc_inst(instrs[1]));
> + patch_instruction(dest + 2, ppc_inst(instrs[2]));
> + patch_instruction(dest + 3, ppc_inst(instrs[3]));
> + patch_instruction(dest + 4, ppc_inst(instrs[4]));
> + patch_instruction(dest + 5, ppc_inst(instrs[5]));
> }
> printk(KERN_DEBUG "stf-barrier: patched %d exit locations (%s barrier)
\n",
> i, (types == STF_BARRIER_NONE) ? "no" :
> @@ -260,9 +261,9 @@ void do_rfi_flush_fixups(enum l1d_flush_type types)
>
> pr_devel("patching dest %lx\n", (unsigned long)dest);
>
> - patch_instruction(dest, instrs[0]);
> - patch_instruction(dest + 1, instrs[1]);
> - patch_instruction(dest + 2, instrs[2]);
> + patch_instruction(dest, ppc_inst(instrs[0]));
> + patch_instruction(dest + 1, ppc_inst(instrs[1]));
> + patch_instruction(dest + 2, ppc_inst(instrs[2]));
> }
>
> printk(KERN_DEBUG "rfi-flush: patched %d locations (%s flush)\n", i,
> @@ -295,7 +296,7 @@ void do_barrier_nospec_fixups_range(bool enable, void
> *fixup_start, void *fixup_ dest = (void *)start + *start;
>
> pr_devel("patching dest %lx\n", (unsigned long)dest);
> - patch_instruction(dest, instr);
> + patch_instruction(dest, ppc_inst(instr));
> }
>
> printk(KERN_DEBUG "barrier-nospec: patched %d locations\n", i);
> @@ -338,8 +339,8 @@ void do_barrier_nospec_fixups_range(bool enable, void
> *fixup_start, void *fixup_ dest = (void *)start + *start;
>
> pr_devel("patching dest %lx\n", (unsigned long)dest);
> - patch_instruction(dest, instr[0]);
> - patch_instruction(dest + 1, instr[1]);
> + patch_instruction(dest, ppc_inst(instr[0]));
> + patch_instruction(dest + 1, ppc_inst(instr[1]));
> }
>
> printk(KERN_DEBUG "barrier-nospec: patched %d locations\n", i);
> @@ -353,7 +354,7 @@ static void patch_btb_flush_section(long *curr)
> end = (void *)curr + *(curr + 1);
> for (; start < end; start++) {
> pr_devel("patching dest %lx\n", (unsigned long)start);
> - patch_instruction(start, PPC_INST_NOP);
> + patch_instruction(start, ppc_inst(PPC_INST_NOP));
> }
> }
>
> @@ -382,7 +383,7 @@ void do_lwsync_fixups(unsigned long value, void
> *fixup_start, void *fixup_end)
>
> for (; start < end; start++) {
> dest = (void *)start + *start;
> - raw_patch_instruction(dest, PPC_INST_LWSYNC);
> + raw_patch_instruction(dest, ppc_inst(PPC_INST_LWSYNC));
> }
> }
>
> @@ -400,7 +401,7 @@ static void do_final_fixups(void)
> length = (__end_interrupts - _stext) / sizeof(int);
>
> while (length--) {
> - raw_patch_instruction(dest, *src);
> + raw_patch_instruction(dest, ppc_inst(*src));
> src++;
> dest++;
> }
> diff --git a/arch/powerpc/lib/test_emulate_step.c
> b/arch/powerpc/lib/test_emulate_step.c index 53df4146dd32..85d62f16d07a
> 100644
> --- a/arch/powerpc/lib/test_emulate_step.c
> +++ b/arch/powerpc/lib/test_emulate_step.c
> @@ -11,6 +11,7 @@
> #include <asm/sstep.h>
> #include <asm/ppc-opcode.h>
> #include <asm/code-patching.h>
> +#include <asm/inst.h>
>
> #define IMM_L(i) ((uintptr_t)(i) & 0xffff)
> #define IMM_DS(i) ((uintptr_t)(i) & 0xfffc)
> @@ -19,40 +20,40 @@
> * Defined with TEST_ prefix so it does not conflict with other
> * definitions.
> */
> -#define TEST_LD(r, base, i) (PPC_INST_LD | ___PPC_RT(r) | \
> +#define TEST_LD(r, base, i) ppc_inst(PPC_INST_LD | ___PPC_RT(r) | \
> ___PPC_RA(base) | IMM_DS(i))
> -#define TEST_LWZ(r, base, i) (PPC_INST_LWZ | ___PPC_RT(r) | \
> +#define TEST_LWZ(r, base, i) ppc_inst(PPC_INST_LWZ | ___PPC_RT(r) | \
> ___PPC_RA(base) | IMM_L(i))
> -#define TEST_LWZX(t, a, b) (PPC_INST_LWZX | ___PPC_RT(t) | \
> +#define TEST_LWZX(t, a, b) ppc_inst(PPC_INST_LWZX | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_STD(r, base, i) (PPC_INST_STD | ___PPC_RS(r) | \
> +#define TEST_STD(r, base, i) ppc_inst(PPC_INST_STD | ___PPC_RS(r) | \
> ___PPC_RA(base) | IMM_DS(i))
> -#define TEST_LDARX(t, a, b, eh) (PPC_INST_LDARX | ___PPC_RT(t) | \
> +#define TEST_LDARX(t, a, b, eh) ppc_inst(PPC_INST_LDARX | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b) | \
> __PPC_EH(eh))
> -#define TEST_STDCX(s, a, b) (PPC_INST_STDCX | ___PPC_RS(s) | \
> +#define TEST_STDCX(s, a, b) ppc_inst(PPC_INST_STDCX | ___PPC_RS(s) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_LFSX(t, a, b) (PPC_INST_LFSX | ___PPC_RT(t) | \
> +#define TEST_LFSX(t, a, b) ppc_inst(PPC_INST_LFSX | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_STFSX(s, a, b) (PPC_INST_STFSX | ___PPC_RS(s) | \
> +#define TEST_STFSX(s, a, b) ppc_inst(PPC_INST_STFSX | ___PPC_RS(s) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_LFDX(t, a, b) (PPC_INST_LFDX | ___PPC_RT(t) | \
> +#define TEST_LFDX(t, a, b) ppc_inst(PPC_INST_LFDX | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_STFDX(s, a, b) (PPC_INST_STFDX | ___PPC_RS(s) | \
> +#define TEST_STFDX(s, a, b) ppc_inst(PPC_INST_STFDX | ___PPC_RS(s) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_LVX(t, a, b) (PPC_INST_LVX | ___PPC_RT(t) | \
> +#define TEST_LVX(t, a, b) ppc_inst(PPC_INST_LVX | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_STVX(s, a, b) (PPC_INST_STVX | ___PPC_RS(s) | \
> +#define TEST_STVX(s, a, b) ppc_inst(PPC_INST_STVX | ___PPC_RS(s) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_LXVD2X(s, a, b) (PPC_INST_LXVD2X | VSX_XX1((s), R##a, R##b))
> -#define TEST_STXVD2X(s, a, b) (PPC_INST_STXVD2X | VSX_XX1((s), R##a, R##b))
> -#define TEST_ADD(t, a, b) (PPC_INST_ADD | ___PPC_RT(t) | \
> +#define TEST_LXVD2X(s, a, b) ppc_inst(PPC_INST_LXVD2X | VSX_XX1((s), R##a,
> R##b)) +#define TEST_STXVD2X(s, a, b) ppc_inst(PPC_INST_STXVD2X |
> VSX_XX1((s), R##a, R##b)) +#define TEST_ADD(t, a, b) ppc_inst(PPC_INST_ADD
> | ___PPC_RT(t) | \ ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_ADD_DOT(t, a, b) (PPC_INST_ADD | ___PPC_RT(t) | \
> +#define TEST_ADD_DOT(t, a, b) ppc_inst(PPC_INST_ADD | ___PPC_RT(t) |
\
> ___PPC_RA(a) | ___PPC_RB(b) | 0x1)
> -#define TEST_ADDC(t, a, b) (PPC_INST_ADDC | ___PPC_RT(t) | \
> +#define TEST_ADDC(t, a, b) ppc_inst(PPC_INST_ADDC | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b))
> -#define TEST_ADDC_DOT(t, a, b) (PPC_INST_ADDC | ___PPC_RT(t) | \
> +#define TEST_ADDC_DOT(t, a, b) ppc_inst(PPC_INST_ADDC | ___PPC_RT(t) | \
> ___PPC_RA(a) | ___PPC_RB(b) | 0x1)
>
> #define MAX_SUBTESTS 16
> @@ -472,7 +473,7 @@ static struct compute_test compute_tests[] = {
> .subtests = {
> {
> .descr = "R0 = LONG_MAX",
> - .instr = PPC_INST_NOP,
> + .instr = ppc_inst(PPC_INST_NOP),
> .regs = {
> .gpr[0] = LONG_MAX,
> }
> diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
> index e122f0c8a044..7e60327a9483 100644
> --- a/arch/powerpc/xmon/xmon.c
> +++ b/arch/powerpc/xmon/xmon.c
> @@ -54,6 +54,7 @@
> #include <asm/firmware.h>
> #include <asm/code-patching.h>
> #include <asm/sections.h>
> +#include <asm/inst.h>
>
> #ifdef CONFIG_PPC64
> #include <asm/hvcall.h>
> @@ -946,7 +947,7 @@ static void remove_bpts(void)
> if ((bp->enabled & (BP_TRAP|BP_CIABR)) != BP_TRAP)
> continue;
> if (mread(bp->address, &instr, 4) == 4
> - && instr == bpinstr
> + && instr == ppc_inst(bpinstr)
> && patch_instruction(
> (unsigned int *)bp->address, bp->instr[0]) != 0)
> printf("Couldn't remove breakpoint at %lx\n",
> @@ -2847,7 +2848,7 @@ generic_inst_dump(unsigned long adr, long count, int
> praddr, {
> int nr, dotted;
> unsigned long first_adr;
> - unsigned int inst, last_inst = 0;
> + unsigned int inst, last_inst = ppc_inst(0);
> unsigned char val[4];
>
> dotted = 0;
> @@ -2860,7 +2861,7 @@ generic_inst_dump(unsigned long adr, long count, int
> praddr, }
> break;
> }
> - inst = GETWORD(val);
> + inst = ppc_inst(GETWORD(val));
> if (adr > first_adr && inst == last_inst) {
> if (!dotted) {
> printf(" ...\n");
^ permalink raw reply
* Re: [PATCH v7 02/28] powerpc/xmon: Move breakpoint instructions to own array
From: Jordan Niethe @ 2020-05-04 5:52 UTC (permalink / raw)
To: Alistair Popple
Cc: Nicholas Piggin, Balamuruhan S, naveen.n.rao, linuxppc-dev,
Daniel Axtens
In-Reply-To: <2060873.nxaIYQu1l1@townsend>
On Mon, May 4, 2020 at 3:41 PM Alistair Popple <alistair@popple.id.au> wrote:
>
> On Friday, 1 May 2020 1:41:54 PM AEST Jordan Niethe wrote:
> > To execute an instruction out of line after a breakpoint, the NIP is set
> > to the address of struct bpt::instr. Here a copy of the instruction that
> > was replaced with a breakpoint is kept, along with a trap so normal flow
> > can be resumed after XOLing. The struct bpt's are located within the
> > data section. This is problematic as the data section may be marked as
> > no execute.
> >
> > Instead of each struct bpt holding the instructions to be XOL'd, make a
> > new array, bpt_table[], with enough space to hold instructions for the
> > number of supported breakpoints. A later patch will move this to the
> > text section.
> > Make struct bpt::instr a pointer to the instructions in bpt_table[]
> > associated with that breakpoint. This association is a simple mapping:
> > bpts[n] -> bpt_table[n * words per breakpoint]. Currently we only need
> > the copied instruction followed by a trap, so 2 words per breakpoint.
> >
> > Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> > ---
> > v4: New to series
> > v5: - Do not use __section(), use a .space directive in .S file
> > - Simplify in_breakpoint_table() calculation
> > - Define BPT_SIZE
> > v6: - Seperate moving to text section
> > ---
> > arch/powerpc/xmon/xmon.c | 21 ++++++++++++---------
> > 1 file changed, 12 insertions(+), 9 deletions(-)
> >
> > diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
> > index f91ae2c9adbe..6ba7f66c1dd0 100644
> > --- a/arch/powerpc/xmon/xmon.c
> > +++ b/arch/powerpc/xmon/xmon.c
> > @@ -98,7 +98,7 @@ static long *xmon_fault_jmp[NR_CPUS];
> > /* Breakpoint stuff */
> > struct bpt {
> > unsigned long address;
> > - unsigned int instr[2];
> > + unsigned int *instr;
> > atomic_t ref_count;
> > int enabled;
> > unsigned long pad;
> > @@ -117,6 +117,10 @@ static unsigned bpinstr = 0x7fe00008; /* trap */
> >
> > #define BP_NUM(bp) ((bp) - bpts + 1)
> >
> > +#define BPT_SIZE (sizeof(unsigned int) * 2)
> > +#define BPT_WORDS (BPT_SIZE / sizeof(unsigned int))
>
> Minor nit-pick but IMHO this would be more logical if you defined BPT_WORDS
> first like so:
>
> #define BPT_WORDS (2)
> #define BPT_SIZE (sizeof(unsigned int) * BPT_WORDS)
>
> Otherwise this looks good and I think the offset calculations below are correct
> so:
What I was thinking was BPT_SIZE would later be defined in terms the
instruction type and by doing it this way
BPT_WORDS would be correct for the the instruction type if like on
ppc32, it did not include a suffix.
>
> Reviewed-by: Alistair Popple <alistair@popple.id.au>
>
> > +static unsigned int bpt_table[NBPTS * BPT_WORDS];
> > +
> > /* Prototypes */
> > static int cmds(struct pt_regs *);
> > static int mread(unsigned long, void *, int);
> > @@ -854,15 +858,13 @@ static struct bpt *in_breakpoint_table(unsigned long
> > nip, unsigned long *offp) {
> > unsigned long off;
> >
> > - off = nip - (unsigned long) bpts;
> > - if (off >= sizeof(bpts))
> > + off = nip - (unsigned long) bpt_table;
> > + if (off >= sizeof(bpt_table))
> > return NULL;
> > - off %= sizeof(struct bpt);
> > - if (off != offsetof(struct bpt, instr[0])
> > - && off != offsetof(struct bpt, instr[1]))
> > + *offp = off % BPT_SIZE;
> > + if (*offp != 0 && *offp != 4)
> > return NULL;
> > - *offp = off - offsetof(struct bpt, instr[0]);
> > - return (struct bpt *) (nip - off);
> > + return bpts + (off / BPT_SIZE);
> > }
> >
> > static struct bpt *new_breakpoint(unsigned long a)
> > @@ -877,7 +879,8 @@ static struct bpt *new_breakpoint(unsigned long a)
> > for (bp = bpts; bp < &bpts[NBPTS]; ++bp) {
> > if (!bp->enabled && atomic_read(&bp->ref_count) == 0) {
> > bp->address = a;
> > - patch_instruction(&bp->instr[1], bpinstr);
> > + bp->instr = bpt_table + ((bp - bpts) * BPT_WORDS);
> > + patch_instruction(bp->instr + 1, bpinstr);
> > return bp;
> > }
> > }
>
>
>
>
^ permalink raw reply
* Re: [PATCH v7 04/28] powerpc/xmon: Use bitwise calculations in_breakpoint_table()
From: Alistair Popple @ 2020-05-04 5:41 UTC (permalink / raw)
To: Jordan Niethe; +Cc: npiggin, bala24, naveen.n.rao, linuxppc-dev, dja
In-Reply-To: <20200501034220.8982-5-jniethe5@gmail.com>
I probably would have just folded this change into patch 2 but it looks fine to
me.
Reviewed-by: Alistair Popple <alistair@popple.id.au>
On Friday, 1 May 2020 1:41:56 PM AEST Jordan Niethe wrote:
> A modulo operation is used for calculating the current offset from a
> breakpoint within the breakpoint table. As instruction lengths are
> always a power of 2, this can be replaced with a bitwise 'and'. The
> current check for word alignment can be replaced with checking that the
> lower 2 bits are not set.
>
> Suggested-by: Christophe Leroy <christophe.leroy@c-s.fr>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v6: New to series
> ---
> arch/powerpc/xmon/xmon.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
> index bbfea22f4a96..e122f0c8a044 100644
> --- a/arch/powerpc/xmon/xmon.c
> +++ b/arch/powerpc/xmon/xmon.c
> @@ -857,8 +857,8 @@ static struct bpt *in_breakpoint_table(unsigned long
> nip, unsigned long *offp) off = nip - (unsigned long) bpt_table;
> if (off >= sizeof(bpt_table))
> return NULL;
> - *offp = off % BPT_SIZE;
> - if (*offp != 0 && *offp != 4)
> + *offp = off & (BPT_SIZE - 1);
> + if (off & 3)
> return NULL;
> return bpts + (off / BPT_SIZE);
> }
^ permalink raw reply
* Re: [PATCH v7 02/28] powerpc/xmon: Move breakpoint instructions to own array
From: Alistair Popple @ 2020-05-04 5:41 UTC (permalink / raw)
To: Jordan Niethe; +Cc: npiggin, bala24, naveen.n.rao, linuxppc-dev, dja
In-Reply-To: <20200501034220.8982-3-jniethe5@gmail.com>
On Friday, 1 May 2020 1:41:54 PM AEST Jordan Niethe wrote:
> To execute an instruction out of line after a breakpoint, the NIP is set
> to the address of struct bpt::instr. Here a copy of the instruction that
> was replaced with a breakpoint is kept, along with a trap so normal flow
> can be resumed after XOLing. The struct bpt's are located within the
> data section. This is problematic as the data section may be marked as
> no execute.
>
> Instead of each struct bpt holding the instructions to be XOL'd, make a
> new array, bpt_table[], with enough space to hold instructions for the
> number of supported breakpoints. A later patch will move this to the
> text section.
> Make struct bpt::instr a pointer to the instructions in bpt_table[]
> associated with that breakpoint. This association is a simple mapping:
> bpts[n] -> bpt_table[n * words per breakpoint]. Currently we only need
> the copied instruction followed by a trap, so 2 words per breakpoint.
>
> Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
> ---
> v4: New to series
> v5: - Do not use __section(), use a .space directive in .S file
> - Simplify in_breakpoint_table() calculation
> - Define BPT_SIZE
> v6: - Seperate moving to text section
> ---
> arch/powerpc/xmon/xmon.c | 21 ++++++++++++---------
> 1 file changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
> index f91ae2c9adbe..6ba7f66c1dd0 100644
> --- a/arch/powerpc/xmon/xmon.c
> +++ b/arch/powerpc/xmon/xmon.c
> @@ -98,7 +98,7 @@ static long *xmon_fault_jmp[NR_CPUS];
> /* Breakpoint stuff */
> struct bpt {
> unsigned long address;
> - unsigned int instr[2];
> + unsigned int *instr;
> atomic_t ref_count;
> int enabled;
> unsigned long pad;
> @@ -117,6 +117,10 @@ static unsigned bpinstr = 0x7fe00008; /* trap */
>
> #define BP_NUM(bp) ((bp) - bpts + 1)
>
> +#define BPT_SIZE (sizeof(unsigned int) * 2)
> +#define BPT_WORDS (BPT_SIZE / sizeof(unsigned int))
Minor nit-pick but IMHO this would be more logical if you defined BPT_WORDS
first like so:
#define BPT_WORDS (2)
#define BPT_SIZE (sizeof(unsigned int) * BPT_WORDS)
Otherwise this looks good and I think the offset calculations below are correct
so:
Reviewed-by: Alistair Popple <alistair@popple.id.au>
> +static unsigned int bpt_table[NBPTS * BPT_WORDS];
> +
> /* Prototypes */
> static int cmds(struct pt_regs *);
> static int mread(unsigned long, void *, int);
> @@ -854,15 +858,13 @@ static struct bpt *in_breakpoint_table(unsigned long
> nip, unsigned long *offp) {
> unsigned long off;
>
> - off = nip - (unsigned long) bpts;
> - if (off >= sizeof(bpts))
> + off = nip - (unsigned long) bpt_table;
> + if (off >= sizeof(bpt_table))
> return NULL;
> - off %= sizeof(struct bpt);
> - if (off != offsetof(struct bpt, instr[0])
> - && off != offsetof(struct bpt, instr[1]))
> + *offp = off % BPT_SIZE;
> + if (*offp != 0 && *offp != 4)
> return NULL;
> - *offp = off - offsetof(struct bpt, instr[0]);
> - return (struct bpt *) (nip - off);
> + return bpts + (off / BPT_SIZE);
> }
>
> static struct bpt *new_breakpoint(unsigned long a)
> @@ -877,7 +879,8 @@ static struct bpt *new_breakpoint(unsigned long a)
> for (bp = bpts; bp < &bpts[NBPTS]; ++bp) {
> if (!bp->enabled && atomic_read(&bp->ref_count) == 0) {
> bp->address = a;
> - patch_instruction(&bp->instr[1], bpinstr);
> + bp->instr = bpt_table + ((bp - bpts) * BPT_WORDS);
> + patch_instruction(bp->instr + 1, bpinstr);
> return bp;
> }
> }
^ permalink raw reply
* Re: [PATCH V2 00/11] Subject: Remove duplicated kmap code
From: Al Viro @ 2020-05-04 5:33 UTC (permalink / raw)
To: Ira Weiny
Cc: Peter Zijlstra, Dave Hansen, dri-devel, linux-mips,
James E.J. Bottomley, Max Filippov, Huang Rui, Paul Mackerras,
H. Peter Anvin, sparclinux, Dan Williams, Helge Deller, x86,
linux-csky, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
linux-arm-kernel, Chris Zankel, Thomas Bogendoerfer, linux-parisc,
linux-kernel, Christian Koenig, Andrew Morton, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504050447.GA979899@iweiny-DESK2.sc.intel.com>
On Sun, May 03, 2020 at 10:04:47PM -0700, Ira Weiny wrote:
> Grepping for 'asm/highmem.h' and investigations don't reveal any issues... But
> you do have me worried. That said 0-day has been crunching on multiple
> versions of this series without issues such as this (save the mips issue
> above).
>
> I have to say it would be nice if the relation between linux/highmem.h and
> asm/highmem.h was more straightforward.
IIRC, the headache was due to asm/pgtable.h on several architectures and
asm/cacheflush.h on parisc.
<digs the notes out>
|| IOW, there's one in linux/highmem.h (conditional on !CONFIG_HIGHMEM,
|| !ARCH_HAS_KMAP) and several per-architecture variants, usually declared in
|| their asm/highmem.h. In three of those (microblaze, parisc and powerpc) these
|| are inlines (parisc one identical to linux/highmem.h, lives in asm/cacheflush.h,
|| powerpc and microblaze ones calling kmap_atomic_prot() which is defined in
|| arch/$ARCH/mm/highmem.c).
||
|| parisc case is weird - essentially, they want to call
|| flush_kernel_dcache_page_addr() at the places where kunmap/kunmap_atomic
|| is done. And they do so despite not selecting HIGHMEM, with definitions
|| in usual place. They do have ARCH_HAS_KMAP defined, which prevents
|| getting buggered in linux/highmem.h. ARCH_HAS_KMAP is parisc-unique,
|| BTW, and checked only in linux/highmem.h.
||
|| All genuine arch-specific variants are defined in (or call functions
|| defined in) translation units that are only included CONFIG_HIGHMEM builds.
||
|| It would be tempting to consolidate those, e.g. by adding __kmap_atomic()
|| and __kmap_atomic_prot() without that boilerplate, with universal kmap_atomic()
|| and kmap_atomic_prot() in linux/highmem.h. Potential problem with that would
|| be something that pulls ash/highmem.h (or asm/cacheflush.h in case of parisc)
|| directly and uses kmap_atomic/kmap_atomic_prot. There's not a lot places
|| pulling asm/highmem.h, and many of them are not even in includes:
||
|| arch/arm/include/asm/efi.h:13:#include <asm/highmem.h>
|| arch/arm/mm/dma-mapping.c:31:#include <asm/highmem.h>
|| arch/arm/mm/flush.c:14:#include <asm/highmem.h>
|| arch/arm/mm/mmu.c:27:#include <asm/highmem.h>
|| arch/mips/include/asm/pgtable-32.h:22:#include <asm/highmem.h>
|| arch/mips/mm/cache.c:19:#include <asm/highmem.h>
|| arch/mips/mm/fault.c:28:#include <asm/highmem.h> /* For VMALLOC_END */
|| arch/nds32/include/asm/pgtable.h:60:#include <asm/highmem.h>
|| arch/x86/kernel/setup_percpu.c:20:#include <asm/highmem.h>
|| include/linux/highmem.h:35:#include <asm/highmem.h>
||
|| Users of asm/cacheflush.h are rather more numerous; however, anything
|| outside of parisc-specific code has to pull linux/highmem.h, or it won't see
|| the definitions of kmap_atomic/kmap_atomic_prot at all. arch/parisc itself
|| has no callers of those.
||
|| Outside of arch/* there is a plenty of callers. However, the following is
|| true: all instances of kmap_atomic or kmap_atomic_prot outside of arch/*
|| are either inside the linux/highmem.h or are preceded by include of
|| linux/highmem.h on any build that sees them (there is a common include
|| chain that is conditional upon CONFIG_BLOCK, but it's only needed in
|| drivers that are BLOCK-dependent). It was not fun to verify, to put
|| it mildly...
||
|| So for parisc we have no problem - leaving __kmap_atomic()/__kmap_atomic_prot()
|| in asm/cachefile.h and adding universal wrappers in linux/highmem.h will be
|| fine. For other architectures the things might be trickier.
||
|| * arc: all users in arch/arc/ are within arch/arc/mm/{cache,highmem}.c;
|| both pull linux/highmem.h. We are fine.
||
|| * arm: much, much worse. We have several files that pull linux/highmem.h:
|| arch/arm/mm/cache-feroceon-l2.c, arch/arm/mm/cache-xsc3l2.c,
|| arch/arm/mm/copypage-*.c, arch/arm/mm/dma-mapping.c, arch/arm/mm/flush.c,
|| arch/arm/mm/highmem.c, arch/arm/probes/uprobes/core.c,
|| arch/arm/include/asm/kvm_mmu.h (kmap_atomic_pfn()).
|| Those are fine, but we also have this:
|| arch/arm/include/asm/pgtable.h:200:#define __pte_map(pmd) (pte_t *)kmap_atomic(pmd_page(*(pmd)))
|| arch/arm/include/asm/pgtable.h:208:#define pte_offset_map(pmd,addr) (__pte_map(pmd) + pte_index(addr))
|| and sure as hell, asm/pgtable.h does *NOT* pull linux/highmem.h.
||
|| Fortunately, the users of pte_offset_map() (__pte_map() has no other users)
|| are few, both in arch/arm and outside of arch. All arm ones are pulling
|| linux/highmem (arch/arm/mm/{pgd,fault*}.c). Outside of arch we have several
|| that pull highmem.h (by way of rmap.h or pagemap.h, usually):
|| fs/userfaultfd.c, mm/gup.c, mm/hmm.c, mm/huge_memory.c,
|| mm/khugepaged.c, mm/memory-failure.c, mm/memory.c, mm/migrate.c,
|| mm/mremap.c, mm/page_vma_mapped.c, mm/swap_state.c, mm/swapfile.c,
|| mm/vmalloc.c
|| and then there are these in linux/mm.h:
||
|| #define pte_offset_map_lock(mm, pmd, address, ptlp) \
|| ({ \
|| spinlock_t *__ptl = pte_lockptr(mm, pmd); \
|| pte_t *__pte = pte_offset_map(pmd, address); \
|| *(ptlp) = __ptl; \
|| spin_lock(__ptl); \
|| __pte; \
|| })
|| #define pte_alloc_map(mm, pmd, address) \
|| (pte_alloc(mm, pmd) ? NULL : pte_offset_map(pmd, address))
|| #define pte_alloc_map_lock(mm, pmd, address, ptlp) \
|| (pte_alloc(mm, pmd) ? \
|| NULL : pte_offset_map_lock(mm, pmd, address, ptlp))
||
|| These have two users in arch/arm (arch/arm/mm/pgd.c and
|| arch/arm/lib/uaccess_with_memcpy.c, both pulling highmem.h). Outside of
|| arch there are several new files (plus a lot of what we'd already seen
|| in mm/*.c, unsurprisingly):
|| fs/proc/task_mmu.c, mm/ksm.c, mm/madvise.c, mm/memcontrol.c,
|| mm/mempolicy.c, mm/mincore.c, mm/mprotect.c, mm/pagewalk.c,
|| mm/shmem.c, mm/userfaultfd.c,
|| all pulling linux/highmem.h, as pretty much all core VM does. So we are
|| still fine.
||
|| * csky: users in arch/csky/abiv2/cacheflush.c, arch/csky/mm/dma-mapping.c,
|| arch/csky/mm/highmem.c, all pulling linux/highmem.h
||
|| * microblaze: users in arch/microblaze/mm/highmem.c (pulls linux/highmem.h) and,
|| arch/microblaze/include/asm/pgtable.h, this:
|| #define pte_offset_map(dir, addr) \
|| ((pte_t *) kmap_atomic(pmd_page(*(dir))) + pte_index(addr))
|| One pte_offset_map user in arch/microblaze:
|| arch/microblaze/kernel/signal.c:207: ptep = pte_offset_map(pmdp, address);
|| Messy, but doesn't require any changes (we have asm/pgalloc.h included
|| there, and that pull linux/highmem.h).
|| Outside of arch we'd already sorted it out when looking at arm.
||
|| * mips: users in arch/mips/kernel/crash_dump.c, arch/mips/kernel/uprobes.c,
|| arch/mips/mm/c-r4k.c, arch/mips/mm/dma-noncoherent.c, arch/mips/mm/highmem.c,
|| and arch/mips/mm/init.c (all pulling linux/highmem.h) plus this
|| arch/mips/mm/cache.c, which relies upon asm/highmem.h. This can be switched
|| to linux/highmem.h. On !CONFIG_HIGHMEM builds the call of kmap_atomic() in
|| there is eliminated, since it's conditional upon PageHighMem(). IOW, even
|| though we get a call of (inexistent) out-of-line version, it's not going to
|| survive into object file. With linux/highmem.h use it will be an equally
|| eliminated call of inlined version.
|| XXX: arch/mips/mm/cache.c
||
|| * nds32: users in arch/nds32/kernel/dma.c, arch/nds32/mm/cacheflush.c and
|| arch/nds32/mm/highmem.c, all pulling linux/highmem.h
||
|| * powerpc: users in arch/powerpc/kvm/book3s_pr.c,
|| arch/powerpc/kvm/e500_mmu_host.c, arch/powerpc/mm/dma-noncoherent.c,
|| arch/powerpc/mm/highmem.c and arch/powerpc/mm/mem.c, all pulling
|| linux/highmem.h, a user in arch/powerpc/mm/hugetlbpage.c pulling it
|| via asm/tlb.h -> linux/pagemap.h -> linux/highmem.h and
|| macros for pte_offset_map in arch/powerpc/include/asm/*/32/pgtable.h.
|| Users of that within arch/powerpc are either 64bit-only or
|| pull linux/highmem.h (arch/powerpc/mm/pgtable_32.c and
|| arch/powerpc/xmon/xmon.c). Users outside of arch - same as for arm.
||
|| * sparc: users in arch/sparc/kernel/uprobes.c and arch/sparc/mm/highmem.c
|| (both pulling linux/highmem.h directly) + arch/sparc/mm/init_64.c pulling
|| it via linux/pagemap.h. Strangely, arch/sparc/mm/io-unit.c and
|| arch/sparc/mm/iommu.c both include linux/highmem.h with odd comment
|| that seems to indicate that once upon a time pte_offset_map() used to
|| requite kmap_atomic() there... Right, it used to - until 2002.
|| These includes are pointless, then...
||
|| * x86: users in arch/x86/kernel/crash_dump_32.c, arch/x86/kvm/svm.c,
|| arch/x86/lib/usercopy_64.c, arch/x86/mm/highmem_32.c and arch/x86/mm/iomap_32.c,
|| all pulling linux/highmem.h, users in paging_tmpl.h (included from
|| arch/x86/kvm/mmu/mmu.c, which has pulled linux/highmem.h prior to that)
|| and definition of pte_offset_map() (in asm/pgtable_32.h)
|| Users of pte_offset_map() and friends in arch/x86 are in
|| arch/x86/kernel/vm86_32.c and arch/x86/mm/dump_pagetables.c (both
|| pulling linux/highmem.h), in arch/x86/mm/mem_encrypt_identity.c
|| (64bit-only, pte_offset_map() doesn't use kmap_atomic() there) and
|| arch/x86/kernel/tboot.c (pulls linux/highmem.h via asm/pgalloc.h
|| and linux/pagemap.h)
||
|| * xtensa: users in arch/xtensa/kernel/pci-dma.c, arch/xtensa/mm/highmem.c,
|| arch/xtensa/mm/cache.c and arch/xtensa/platforms/iss/simdisk.c (all pull
|| linux/highmem.h).
^ permalink raw reply
* Re: [PATCH] powerpc/powernv: Fix a warning message
From: Gautham R Shenoy @ 2020-05-04 5:05 UTC (permalink / raw)
To: Christophe JAILLET
Cc: ego, maddy, Rafael J. Wysocki, kernel-janitors, cclaudio, npiggin,
linux-kernel, zhangshaokun, Viresh Kumar, atrajeev, paulus, tglx,
linuxppc-dev, akshay.adiga
In-Reply-To: <20200502115949.139000-1-christophe.jaillet@wanadoo.fr>
Hello Christophe,
On Sat, May 02, 2020 at 01:59:49PM +0200, Christophe JAILLET wrote:
> Fix a cut'n'paste error in a warning message. This should be
> 'cpu-idle-state-residency-ns' to match the property searched in the
> previous 'of_property_read_u32_array()'
>
> Fixes: 9c7b185ab2fe ("powernv/cpuidle: Parse dt idle properties into global structure")
> Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Thanks for catching this.
Reviewed-by: Gautham R. Shenoy <ego@linux.vnet.ibm.com>
> ---
> arch/powerpc/platforms/powernv/idle.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
> index 78599bca66c2..2dd467383a88 100644
> --- a/arch/powerpc/platforms/powernv/idle.c
> +++ b/arch/powerpc/platforms/powernv/idle.c
> @@ -1270,7 +1270,7 @@ static int pnv_parse_cpuidle_dt(void)
> /* Read residencies */
> if (of_property_read_u32_array(np, "ibm,cpu-idle-state-residency-ns",
> temp_u32, nr_idle_states)) {
> - pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-latencies-ns in DT\n");
> + pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-residency-ns in DT\n");
> rc = -EINVAL;
> goto out;
> }
> --
> 2.25.1
>
^ permalink raw reply
* Re: [PATCH V2 00/11] Subject: Remove duplicated kmap code
From: Ira Weiny @ 2020-05-04 5:04 UTC (permalink / raw)
To: Al Viro
Cc: Peter Zijlstra, Dave Hansen, dri-devel, linux-mips,
James E.J. Bottomley, Max Filippov, Huang Rui, Paul Mackerras,
H. Peter Anvin, sparclinux, Dan Williams, Helge Deller, x86,
linux-csky, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
linux-arm-kernel, Chris Zankel, Thomas Bogendoerfer, linux-parisc,
linux-kernel, Christian Koenig, Andrew Morton, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504013509.GU23230@ZenIV.linux.org.uk>
On Mon, May 04, 2020 at 02:35:09AM +0100, Al Viro wrote:
> On Sun, May 03, 2020 at 06:09:01PM -0700, ira.weiny@intel.com wrote:
> > From: Ira Weiny <ira.weiny@intel.com>
> >
> > The kmap infrastructure has been copied almost verbatim to every architecture.
> > This series consolidates obvious duplicated code by defining core functions
> > which call into the architectures only when needed.
> >
> > Some of the k[un]map_atomic() implementations have some similarities but the
> > similarities were not sufficient to warrant further changes.
> >
> > In addition we remove a duplicate implementation of kmap() in DRM.
> >
> > Testing was done by 0day to cover all the architectures I can't readily
> > build/test.
>
> OK... Looking through my old notes on kmap unification (this winter, never
> went anywhere),
>
> * arch/mips/mm/cache.c ought to use linux/highmem.h, not asm/highmem.h
> I suspect that your series doesn't build on some configs there. Hadn't
> verified that, though.
Yes patch 6 makes the change because kmap_atomic() was no longer declared in
asm/highmem.h. I'm pretty sure 0-day caught that ... but I seem to remember
noticing some oddness in that file and I did go through it by hand.
>
> * kmap_atomic_to_page() is dead, but not quite gone - csky and nds32 brought
> the damn thing back (nds32 - only an extern). It needs killin'...
Easy enough. Added as a follow on patch.
>
> * parisc is (arguably) abusing kunmap()/kunmap_atomic() for cache flushing.
> Replace the bulk of its highmem.h with
> #define ARCH_HAS_FLUSH_ON_KUNMAP
> #define arch_before_kunmap flush_kernel_dcache_page_addr
> and have default kunmap()/kunmap_atomic() do
> #ifdef ARCH_HAS_FLUSH_ON_KUNMAP
> arch_before_kunmap(page_address(page));
> #endif
> and
> #ifdef ARCH_HAS_FLUSH_ON_KUNMAP
> arch_before_kunmap(addr);
> #endif
> resp. Kills ARCH_HAS_KMAP along with ifdefs on it, makes parisc use somewhat
> less hacky.
Agreed. Done in a follow on patch.
>
> I'd suggest checking various configs on mips - that's likely to cause headache.
> Said that, my analysis of include chains back then is pretty much worthless
> by now - I really hate the amount of indirect include chains leading to that
> sucker on some, but not all configs ;-/ IIRC, the proof that everything
> using kmap*/kunmap* would pull linux/highmem.h regardless of config took several
> hours of digging, ran for several pages and had been hopelessly brittle.
> arch/mips/mm/cache.c was the only exception caught by it, but these days
> there might be more.
Grepping for 'asm/highmem.h' and investigations don't reveal any issues... But
you do have me worried. That said 0-day has been crunching on multiple
versions of this series without issues such as this (save the mips issue
above).
I have to say it would be nice if the relation between linux/highmem.h and
asm/highmem.h was more straightforward.
Ira
^ permalink raw reply
* [PATCH V18 2/2] mm/debug: Add tests validating architecture page table helpers
From: Anshuman Khandual @ 2020-05-04 4:01 UTC (permalink / raw)
To: linux-mm
Cc: Heiko Carstens, Paul Mackerras, H. Peter Anvin, linux-riscv,
Will Deacon, linux-arch, linux-s390, x86, Mike Rapoport,
Christian Borntraeger, Ingo Molnar, Catalin Marinas,
linux-snps-arc, Vasily Gorbik, Anshuman Khandual, Borislav Petkov,
Paul Walmsley, Kirill A . Shutemov, Thomas Gleixner,
linux-arm-kernel, Vineet Gupta, linux-kernel, Palmer Dabbelt,
Qian Cai, Andrew Morton, linuxppc-dev
In-Reply-To: <1588564865-31160-1-git-send-email-anshuman.khandual@arm.com>
This adds tests which will validate architecture page table helpers and
other accessors in their compliance with expected generic MM semantics.
This will help various architectures in validating changes to existing
page table helpers or addition of new ones.
This test covers basic page table entry transformations including but not
limited to old, young, dirty, clean, write, write protect etc at various
level along with populating intermediate entries with next page table page
and validating them.
Test page table pages are allocated from system memory with required size
and alignments. The mapped pfns at page table levels are derived from a
real pfn representing a valid kernel text symbol. This test gets called
via late_initcall().
This test gets built and run when CONFIG_DEBUG_VM_PGTABLE is selected. Any
architecture, which is willing to subscribe this test will need to select
ARCH_HAS_DEBUG_VM_PGTABLE. For now this is limited to arc, arm64, x86, s390
and powerpc platforms where the test is known to build and run successfully
Going forward, other architectures too can subscribe the test after fixing
any build or runtime problems with their page table helpers.
Folks interested in making sure that a given platform's page table helpers
conform to expected generic MM semantics should enable the above config
which will just trigger this test during boot. Any non conformity here will
be reported as an warning which would need to be fixed. This test will help
catch any changes to the agreed upon semantics expected from generic MM and
enable platforms to accommodate it thereafter.
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mike Rapoport <rppt@linux.ibm.com>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Kirill A. Shutemov <kirill@shutemov.name>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: linux-snps-arc@lists.infradead.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: linux-riscv@lists.infradead.org
Cc: x86@kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> # s390
Tested-by: Christophe Leroy <christophe.leroy@c-s.fr> # ppc32
Signed-off-by: Qian Cai <cai@lca.pw>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
.../debug/debug-vm-pgtable/arch-support.txt | 34 ++
arch/arc/Kconfig | 1 +
arch/arm64/Kconfig | 1 +
arch/powerpc/Kconfig | 1 +
arch/s390/Kconfig | 1 +
arch/x86/Kconfig | 1 +
lib/Kconfig.debug | 22 +
mm/Makefile | 1 +
mm/debug_vm_pgtable.c | 382 ++++++++++++++++++
9 files changed, 444 insertions(+)
create mode 100644 Documentation/features/debug/debug-vm-pgtable/arch-support.txt
create mode 100644 mm/debug_vm_pgtable.c
diff --git a/Documentation/features/debug/debug-vm-pgtable/arch-support.txt b/Documentation/features/debug/debug-vm-pgtable/arch-support.txt
new file mode 100644
index 000000000000..c527d05c0459
--- /dev/null
+++ b/Documentation/features/debug/debug-vm-pgtable/arch-support.txt
@@ -0,0 +1,34 @@
+#
+# Feature name: debug-vm-pgtable
+# Kconfig: ARCH_HAS_DEBUG_VM_PGTABLE
+# description: arch supports pgtable tests for semantics compliance
+#
+ -----------------------
+ | arch |status|
+ -----------------------
+ | alpha: | TODO |
+ | arc: | ok |
+ | arm: | TODO |
+ | arm64: | ok |
+ | c6x: | TODO |
+ | csky: | TODO |
+ | h8300: | TODO |
+ | hexagon: | TODO |
+ | ia64: | TODO |
+ | m68k: | TODO |
+ | microblaze: | TODO |
+ | mips: | TODO |
+ | nds32: | TODO |
+ | nios2: | TODO |
+ | openrisc: | TODO |
+ | parisc: | TODO |
+ | powerpc: | ok |
+ | riscv: | TODO |
+ | s390: | ok |
+ | sh: | TODO |
+ | sparc: | TODO |
+ | um: | TODO |
+ | unicore32: | TODO |
+ | x86: | ok |
+ | xtensa: | TODO |
+ -----------------------
diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig
index ff306246d0f8..471ef22216c4 100644
--- a/arch/arc/Kconfig
+++ b/arch/arc/Kconfig
@@ -6,6 +6,7 @@
config ARC
def_bool y
select ARC_TIMERS
+ select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DMA_PREP_COHERENT
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_SETUP_DMA_OPS
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 40fb05d96c60..0efb46abaf3d 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -10,6 +10,7 @@ config ARM64
select ACPI_SPCR_TABLE if ACPI
select ACPI_PPTT if ACPI
select ARCH_HAS_DEBUG_VIRTUAL
+ select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_DMA_PREP_COHERENT
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 924c541a9260..4e89f22cdd27 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -116,6 +116,7 @@ config PPC
#
select ARCH_32BIT_OFF_T if PPC32
select ARCH_HAS_DEBUG_VIRTUAL
+ select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_FORTIFY_SOURCE
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 2167bce993ff..8206b2c19aa8 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -59,6 +59,7 @@ config KASAN_SHADOW_OFFSET
config S390
def_bool y
select ARCH_BINFMT_ELF_STATE
+ select ARCH_HAS_DEBUG_VM_PGTABLE
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_FORTIFY_SOURCE
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 1197b5596d5a..65f545c130f6 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -59,6 +59,7 @@ config X86
select ARCH_CLOCKSOURCE_INIT
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
select ARCH_HAS_DEBUG_VIRTUAL
+ select ARCH_HAS_DEBUG_VM_PGTABLE if !X86_PAE
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_FAST_MULTIPLIER
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 21d9c5f6e7ec..f37d01054bc3 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -653,6 +653,12 @@ config SCHED_STACK_END_CHECK
data corruption or a sporadic crash at a later stage once the region
is examined. The runtime overhead introduced is minimal.
+config ARCH_HAS_DEBUG_VM_PGTABLE
+ bool
+ help
+ An architecture should select this when it can successfully
+ build and run DEBUG_VM_PGTABLE.
+
config DEBUG_VM
bool "Debug VM"
depends on DEBUG_KERNEL
@@ -688,6 +694,22 @@ config DEBUG_VM_PGFLAGS
If unsure, say N.
+config DEBUG_VM_PGTABLE
+ bool "Debug arch page table for semantics compliance"
+ depends on MMU
+ depends on ARCH_HAS_DEBUG_VM_PGTABLE
+ default y if DEBUG_VM
+ help
+ This option provides a debug method which can be used to test
+ architecture page table helper functions on various platforms in
+ verifying if they comply with expected generic MM semantics. This
+ will help architecture code in making sure that any changes or
+ new additions of these helpers still conform to expected
+ semantics of the generic MM. Platforms will have to opt in for
+ this through ARCH_HAS_DEBUG_VM_PGTABLE.
+
+ If unsure, say N.
+
config ARCH_HAS_DEBUG_VIRTUAL
bool
diff --git a/mm/Makefile b/mm/Makefile
index fccd3756b25f..662fd1504646 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -88,6 +88,7 @@ obj-$(CONFIG_HWPOISON_INJECT) += hwpoison-inject.o
obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o
obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak-test.o
obj-$(CONFIG_DEBUG_RODATA_TEST) += rodata_test.o
+obj-$(CONFIG_DEBUG_VM_PGTABLE) += debug_vm_pgtable.o
obj-$(CONFIG_PAGE_OWNER) += page_owner.o
obj-$(CONFIG_CLEANCACHE) += cleancache.o
obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
new file mode 100644
index 000000000000..188c18908964
--- /dev/null
+++ b/mm/debug_vm_pgtable.c
@@ -0,0 +1,382 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This kernel test validates architecture page table helpers and
+ * accessors and helps in verifying their continued compliance with
+ * expected generic MM semantics.
+ *
+ * Copyright (C) 2019 ARM Ltd.
+ *
+ * Author: Anshuman Khandual <anshuman.khandual@arm.com>
+ */
+#define pr_fmt(fmt) "debug_vm_pgtable: %s: " fmt, __func__
+
+#include <linux/gfp.h>
+#include <linux/highmem.h>
+#include <linux/hugetlb.h>
+#include <linux/kernel.h>
+#include <linux/kconfig.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mm_types.h>
+#include <linux/module.h>
+#include <linux/pfn_t.h>
+#include <linux/printk.h>
+#include <linux/random.h>
+#include <linux/spinlock.h>
+#include <linux/swap.h>
+#include <linux/swapops.h>
+#include <linux/start_kernel.h>
+#include <linux/sched/mm.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+
+#define VMFLAGS (VM_READ|VM_WRITE|VM_EXEC)
+
+/*
+ * On s390 platform, the lower 4 bits are used to identify given page table
+ * entry type. But these bits might affect the ability to clear entries with
+ * pxx_clear() because of how dynamic page table folding works on s390. So
+ * while loading up the entries do not change the lower 4 bits. It does not
+ * have affect any other platform.
+ */
+#define S390_MASK_BITS 4
+#define RANDOM_ORVALUE GENMASK(BITS_PER_LONG - 1, S390_MASK_BITS)
+#define RANDOM_NZVALUE GENMASK(7, 0)
+
+static void __init pte_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+ pte_t pte = pfn_pte(pfn, prot);
+
+ WARN_ON(!pte_same(pte, pte));
+ WARN_ON(!pte_young(pte_mkyoung(pte_mkold(pte))));
+ WARN_ON(!pte_dirty(pte_mkdirty(pte_mkclean(pte))));
+ WARN_ON(!pte_write(pte_mkwrite(pte_wrprotect(pte))));
+ WARN_ON(pte_young(pte_mkold(pte_mkyoung(pte))));
+ WARN_ON(pte_dirty(pte_mkclean(pte_mkdirty(pte))));
+ WARN_ON(pte_write(pte_wrprotect(pte_mkwrite(pte))));
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+ pmd_t pmd = pfn_pmd(pfn, prot);
+
+ WARN_ON(!pmd_same(pmd, pmd));
+ WARN_ON(!pmd_young(pmd_mkyoung(pmd_mkold(pmd))));
+ WARN_ON(!pmd_dirty(pmd_mkdirty(pmd_mkclean(pmd))));
+ WARN_ON(!pmd_write(pmd_mkwrite(pmd_wrprotect(pmd))));
+ WARN_ON(pmd_young(pmd_mkold(pmd_mkyoung(pmd))));
+ WARN_ON(pmd_dirty(pmd_mkclean(pmd_mkdirty(pmd))));
+ WARN_ON(pmd_write(pmd_wrprotect(pmd_mkwrite(pmd))));
+ /*
+ * A huge page does not point to next level page table
+ * entry. Hence this must qualify as pmd_bad().
+ */
+ WARN_ON(!pmd_bad(pmd_mkhuge(pmd)));
+}
+
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+ pud_t pud = pfn_pud(pfn, prot);
+
+ WARN_ON(!pud_same(pud, pud));
+ WARN_ON(!pud_young(pud_mkyoung(pud_mkold(pud))));
+ WARN_ON(!pud_write(pud_mkwrite(pud_wrprotect(pud))));
+ WARN_ON(pud_write(pud_wrprotect(pud_mkwrite(pud))));
+ WARN_ON(pud_young(pud_mkold(pud_mkyoung(pud))));
+
+ if (mm_pmd_folded(mm))
+ return;
+
+ /*
+ * A huge page does not point to next level page table
+ * entry. Hence this must qualify as pud_bad().
+ */
+ WARN_ON(!pud_bad(pud_mkhuge(pud)));
+}
+#else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
+static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { }
+#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
+#else /* !CONFIG_TRANSPARENT_HUGEPAGE */
+static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { }
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+ p4d_t p4d;
+
+ memset(&p4d, RANDOM_NZVALUE, sizeof(p4d_t));
+ WARN_ON(!p4d_same(p4d, p4d));
+}
+
+static void __init pgd_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+ pgd_t pgd;
+
+ memset(&pgd, RANDOM_NZVALUE, sizeof(pgd_t));
+ WARN_ON(!pgd_same(pgd, pgd));
+}
+
+#ifndef __PAGETABLE_PUD_FOLDED
+static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp)
+{
+ pud_t pud = READ_ONCE(*pudp);
+
+ if (mm_pmd_folded(mm))
+ return;
+
+ pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
+ WRITE_ONCE(*pudp, pud);
+ pud_clear(pudp);
+ pud = READ_ONCE(*pudp);
+ WARN_ON(!pud_none(pud));
+}
+
+static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
+ pmd_t *pmdp)
+{
+ pud_t pud;
+
+ if (mm_pmd_folded(mm))
+ return;
+ /*
+ * This entry points to next level page table page.
+ * Hence this must not qualify as pud_bad().
+ */
+ pmd_clear(pmdp);
+ pud_clear(pudp);
+ pud_populate(mm, pudp, pmdp);
+ pud = READ_ONCE(*pudp);
+ WARN_ON(pud_bad(pud));
+}
+#else /* !__PAGETABLE_PUD_FOLDED */
+static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp) { }
+static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
+ pmd_t *pmdp)
+{
+}
+#endif /* PAGETABLE_PUD_FOLDED */
+
+#ifndef __PAGETABLE_P4D_FOLDED
+static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp)
+{
+ p4d_t p4d = READ_ONCE(*p4dp);
+
+ if (mm_pud_folded(mm))
+ return;
+
+ p4d = __p4d(p4d_val(p4d) | RANDOM_ORVALUE);
+ WRITE_ONCE(*p4dp, p4d);
+ p4d_clear(p4dp);
+ p4d = READ_ONCE(*p4dp);
+ WARN_ON(!p4d_none(p4d));
+}
+
+static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
+ pud_t *pudp)
+{
+ p4d_t p4d;
+
+ if (mm_pud_folded(mm))
+ return;
+
+ /*
+ * This entry points to next level page table page.
+ * Hence this must not qualify as p4d_bad().
+ */
+ pud_clear(pudp);
+ p4d_clear(p4dp);
+ p4d_populate(mm, p4dp, pudp);
+ p4d = READ_ONCE(*p4dp);
+ WARN_ON(p4d_bad(p4d));
+}
+
+static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp)
+{
+ pgd_t pgd = READ_ONCE(*pgdp);
+
+ if (mm_p4d_folded(mm))
+ return;
+
+ pgd = __pgd(pgd_val(pgd) | RANDOM_ORVALUE);
+ WRITE_ONCE(*pgdp, pgd);
+ pgd_clear(pgdp);
+ pgd = READ_ONCE(*pgdp);
+ WARN_ON(!pgd_none(pgd));
+}
+
+static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
+ p4d_t *p4dp)
+{
+ pgd_t pgd;
+
+ if (mm_p4d_folded(mm))
+ return;
+
+ /*
+ * This entry points to next level page table page.
+ * Hence this must not qualify as pgd_bad().
+ */
+ p4d_clear(p4dp);
+ pgd_clear(pgdp);
+ pgd_populate(mm, pgdp, p4dp);
+ pgd = READ_ONCE(*pgdp);
+ WARN_ON(pgd_bad(pgd));
+}
+#else /* !__PAGETABLE_P4D_FOLDED */
+static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp) { }
+static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp) { }
+static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
+ pud_t *pudp)
+{
+}
+static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
+ p4d_t *p4dp)
+{
+}
+#endif /* PAGETABLE_P4D_FOLDED */
+
+static void __init pte_clear_tests(struct mm_struct *mm, pte_t *ptep,
+ unsigned long vaddr)
+{
+ pte_t pte = READ_ONCE(*ptep);
+
+ pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
+ set_pte_at(mm, vaddr, ptep, pte);
+ barrier();
+ pte_clear(mm, vaddr, ptep);
+ pte = READ_ONCE(*ptep);
+ WARN_ON(!pte_none(pte));
+}
+
+static void __init pmd_clear_tests(struct mm_struct *mm, pmd_t *pmdp)
+{
+ pmd_t pmd = READ_ONCE(*pmdp);
+
+ pmd = __pmd(pmd_val(pmd) | RANDOM_ORVALUE);
+ WRITE_ONCE(*pmdp, pmd);
+ pmd_clear(pmdp);
+ pmd = READ_ONCE(*pmdp);
+ WARN_ON(!pmd_none(pmd));
+}
+
+static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pgtable)
+{
+ pmd_t pmd;
+
+ /*
+ * This entry points to next level page table page.
+ * Hence this must not qualify as pmd_bad().
+ */
+ pmd_clear(pmdp);
+ pmd_populate(mm, pmdp, pgtable);
+ pmd = READ_ONCE(*pmdp);
+ WARN_ON(pmd_bad(pmd));
+}
+
+static unsigned long __init get_random_vaddr(void)
+{
+ unsigned long random_vaddr, random_pages, total_user_pages;
+
+ total_user_pages = (TASK_SIZE - FIRST_USER_ADDRESS) / PAGE_SIZE;
+
+ random_pages = get_random_long() % total_user_pages;
+ random_vaddr = FIRST_USER_ADDRESS + random_pages * PAGE_SIZE;
+
+ return random_vaddr;
+}
+
+static int __init debug_vm_pgtable(void)
+{
+ struct mm_struct *mm;
+ pgd_t *pgdp;
+ p4d_t *p4dp, *saved_p4dp;
+ pud_t *pudp, *saved_pudp;
+ pmd_t *pmdp, *saved_pmdp, pmd;
+ pte_t *ptep;
+ pgtable_t saved_ptep;
+ pgprot_t prot;
+ phys_addr_t paddr;
+ unsigned long vaddr, pte_aligned, pmd_aligned;
+ unsigned long pud_aligned, p4d_aligned, pgd_aligned;
+ spinlock_t *uninitialized_var(ptl);
+
+ pr_info("Validating architecture page table helpers\n");
+ prot = vm_get_page_prot(VMFLAGS);
+ vaddr = get_random_vaddr();
+ mm = mm_alloc();
+ if (!mm) {
+ pr_err("mm_struct allocation failed\n");
+ return 1;
+ }
+
+ /*
+ * PFN for mapping at PTE level is determined from a standard kernel
+ * text symbol. But pfns for higher page table levels are derived by
+ * masking lower bits of this real pfn. These derived pfns might not
+ * exist on the platform but that does not really matter as pfn_pxx()
+ * helpers will still create appropriate entries for the test. This
+ * helps avoid large memory block allocations to be used for mapping
+ * at higher page table levels.
+ */
+ paddr = __pa_symbol(&start_kernel);
+
+ pte_aligned = (paddr & PAGE_MASK) >> PAGE_SHIFT;
+ pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
+ pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
+ p4d_aligned = (paddr & P4D_MASK) >> PAGE_SHIFT;
+ pgd_aligned = (paddr & PGDIR_MASK) >> PAGE_SHIFT;
+ WARN_ON(!pfn_valid(pte_aligned));
+
+ pgdp = pgd_offset(mm, vaddr);
+ p4dp = p4d_alloc(mm, pgdp, vaddr);
+ pudp = pud_alloc(mm, p4dp, vaddr);
+ pmdp = pmd_alloc(mm, pudp, vaddr);
+ ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+
+ /*
+ * Save all the page table page addresses as the page table
+ * entries will be used for testing with random or garbage
+ * values. These saved addresses will be used for freeing
+ * page table pages.
+ */
+ pmd = READ_ONCE(*pmdp);
+ saved_p4dp = p4d_offset(pgdp, 0UL);
+ saved_pudp = pud_offset(p4dp, 0UL);
+ saved_pmdp = pmd_offset(pudp, 0UL);
+ saved_ptep = pmd_pgtable(pmd);
+
+ pte_basic_tests(pte_aligned, prot);
+ pmd_basic_tests(pmd_aligned, prot);
+ pud_basic_tests(pud_aligned, prot);
+ p4d_basic_tests(p4d_aligned, prot);
+ pgd_basic_tests(pgd_aligned, prot);
+
+ pte_clear_tests(mm, ptep, vaddr);
+ pmd_clear_tests(mm, pmdp);
+ pud_clear_tests(mm, pudp);
+ p4d_clear_tests(mm, p4dp);
+ pgd_clear_tests(mm, pgdp);
+
+ pte_unmap_unlock(ptep, ptl);
+
+ pmd_populate_tests(mm, pmdp, saved_ptep);
+ pud_populate_tests(mm, pudp, saved_pmdp);
+ p4d_populate_tests(mm, p4dp, saved_pudp);
+ pgd_populate_tests(mm, pgdp, saved_p4dp);
+
+ p4d_free(mm, saved_p4dp);
+ pud_free(mm, saved_pudp);
+ pmd_free(mm, saved_pmdp);
+ pte_free(mm, saved_ptep);
+
+ mm_dec_nr_puds(mm);
+ mm_dec_nr_pmds(mm);
+ mm_dec_nr_ptes(mm);
+ mmdrop(mm);
+ return 0;
+}
+late_initcall(debug_vm_pgtable);
--
2.20.1
^ permalink raw reply related
* [PATCH V18 0/2] mm/debug: Add tests validating architecture page table helpers
From: Anshuman Khandual @ 2020-05-04 4:01 UTC (permalink / raw)
To: linux-mm
Cc: Heiko Carstens, Paul Mackerras, H. Peter Anvin, linux-riscv,
Will Deacon, linux-arch, linux-s390, x86, Mike Rapoport,
Christian Borntraeger, Ingo Molnar, Catalin Marinas,
linux-snps-arc, Vasily Gorbik, Anshuman Khandual, Borislav Petkov,
Paul Walmsley, Kirill A . Shutemov, Thomas Gleixner,
linux-arm-kernel, Vineet Gupta, linux-kernel, Palmer Dabbelt,
Andrew Morton, linuxppc-dev
This adds a test validation for architecture exported page table helpers.
Patch adds basic transformation tests at various levels of the page table.
This test was originally suggested by Catalin during arm64 THP migration
RFC discussion earlier. Going forward it can include more specific tests
with respect to various generic MM functions like THP, HugeTLB etc and
platform specific tests.
https://lore.kernel.org/linux-mm/20190628102003.GA56463@arrakis.emea.arm.com/
Needs to be applied on linux V5.7-rc4
Changes in V18:
- Stopped enabling CONFIG_DEBUG_VM_PGTABLE via CONFIG_EXPERT
- Dropped the exclude list (IA64 and ARM) as CONFIG_EXPERT route no longer available
- Updated CONFIG_DEBUG_VM_PGTABLE's help section as required
- Updated the commit message for [PATCH 2/2] as required
Changes in V17: (https://patchwork.kernel.org/project/linux-mm/list/?series=274401)
- debug_vm_pgtable() is now called from late_initcall() per Linus
- Explicitly enable DEBUG_VM_PGTABLE when ARCH_HAS_DEBUG_VM_PGTABLE and DEBUG_VM
- Added #ifdef documentation per Gerald
- Dropped page table helper semantics documentation (will be added via later patches)
- Split the X86 changes defining mm_p4d_folded() into a new prerequisite patch
Changes in V16: (https://patchwork.kernel.org/patch/11431277/)
- Replaced WRITE_ONCE() with set_pte_at() with a new barrier() in pte_clear_tests() per Qian
- Enabled all powerpc platforms and updated the feature list
Changes in V15: (https://patchwork.kernel.org/patch/11422803/)
- Replaced __pa() with __pa_symbol() (https://patchwork.kernel.org/patch/11407715/)
- Replaced pte_alloc_map() with pte_alloc_map_lock() per Qian
- Replaced pte_unmap() with pte_unmap_unlock() per Qian
- Added address to pte_clear_tests() and passed it down till pte_clear() per Qian
Changes in V14: (https://patchwork.kernel.org/project/linux-mm/list/?series=242305)
- Disabled DEBUG_VM_PGTABLE for IA64 and ARM (32 Bit) per Andrew and Christophe
- Updated DEBUG_VM_PGTABLE documentation wrt EXPERT and disabled platforms
- Updated RANDOM_[OR|NZ]VALUE open encodings with GENMASK() per Catalin
- Updated s390 constraint bits from 12 to 4 (S390_MASK_BITS) per Gerald
- Updated in-code documentation for RANDOM_ORVALUE per Gerald
- Updated pxx_basic_tests() to use invert functions first per Catalin
- Dropped ARCH_HAS_4LEVEL_HACK check from pud_basic_tests()
- Replaced __ARCH_HAS_[4|5]LEVEL_HACK with __PAGETABLE_[PUD|P4D]_FOLDED per Catalin
- Trimmed the CC list on the commit message per Catalin
Changes in V13: (https://patchwork.kernel.org/project/linux-mm/list/?series=237125)
- Subscribed s390 platform and updated debug-vm-pgtable/arch-support.txt per Gerald
- Dropped keyword 'extern' from debug_vm_pgtable() declaration per Christophe
- Moved debug_vm_pgtable() declarations to <linux/mmdebug.h> per Christophe
- Moved debug_vm_pgtable() call site into kernel_init() per Christophe
- Changed CONFIG_DEBUG_VM_PGTABLE rules per Christophe
- Updated commit to include new supported platforms and changed config selection
Changes in V12: (https://patchwork.kernel.org/project/linux-mm/list/?series=233905)
- Replaced __mmdrop() with mmdrop()
- Enable ARCH_HAS_DEBUG_VM_PGTABLE on X86 for non CONFIG_X86_PAE platforms as the
test procedure interfere with pre-allocated PMDs attached to the PGD resulting
in runtime failures with VM_BUG_ON()
Changes in V11: (https://patchwork.kernel.org/project/linux-mm/list/?series=221135)
- Rebased the patch on V5.4
Changes in V10: (https://patchwork.kernel.org/project/linux-mm/list/?series=205529)
- Always enable DEBUG_VM_PGTABLE when DEBUG_VM is enabled per Ingo
- Added tags from Ingo
Changes in V9: (https://patchwork.kernel.org/project/linux-mm/list/?series=201429)
- Changed feature support enumeration for powerpc platforms per Christophe
- Changed config wrapper for basic_[pmd|pud]_tests() to enable ARC platform
- Enabled the test on ARC platform
Changes in V8: (https://patchwork.kernel.org/project/linux-mm/list/?series=194297)
- Enabled ARCH_HAS_DEBUG_VM_PGTABLE on PPC32 platform per Christophe
- Updated feature documentation as DEBUG_VM_PGTABLE is now enabled on PPC32 platform
- Moved ARCH_HAS_DEBUG_VM_PGTABLE earlier to indent it with DEBUG_VM per Christophe
- Added an information message in debug_vm_pgtable() per Christophe
- Dropped random_vaddr boundary condition checks per Christophe and Qian
- Replaced virt_addr_valid() check with pfn_valid() check in debug_vm_pgtable()
- Slightly changed pr_fmt(fmt) information
Changes in V7: (https://patchwork.kernel.org/project/linux-mm/list/?series=193051)
- Memory allocation and free routines for mapped pages have been droped
- Mapped pfns are derived from standard kernel text symbol per Matthew
- Moved debug_vm_pgtaable() after page_alloc_init_late() per Michal and Qian
- Updated the commit message per Michal
- Updated W=1 GCC warning problem on x86 per Qian Cai
- Addition of new alloc_contig_pages() helper has been submitted separately
Changes in V6: (https://patchwork.kernel.org/project/linux-mm/list/?series=187589)
- Moved alloc_gigantic_page_order() into mm/page_alloc.c per Michal
- Moved alloc_gigantic_page_order() within CONFIG_CONTIG_ALLOC in the test
- Folded Andrew's include/asm-generic/pgtable.h fix into the test patch 2/2
Changes in V5: (https://patchwork.kernel.org/project/linux-mm/list/?series=185991)
- Redefined and moved X86 mm_p4d_folded() into a different header per Kirill/Ingo
- Updated the config option comment per Ingo and dropped 'kernel module' reference
- Updated the commit message and dropped 'kernel module' reference
- Changed DEBUG_ARCH_PGTABLE_TEST into DEBUG_VM_PGTABLE per Ingo
- Moved config option from mm/Kconfig.debug into lib/Kconfig.debug
- Renamed core test function arch_pgtable_tests() as debug_vm_pgtable()
- Renamed mm/arch_pgtable_test.c as mm/debug_vm_pgtable.c
- debug_vm_pgtable() gets called from kernel_init_freeable() after init_mm_internals()
- Added an entry in Documentation/features/debug/ per Ingo
- Enabled the test on arm64 and x86 platforms for now
Changes in V4: (https://patchwork.kernel.org/project/linux-mm/list/?series=183465)
- Disable DEBUG_ARCH_PGTABLE_TEST for ARM and IA64 platforms
Changes in V3: (https://lore.kernel.org/patchwork/project/lkml/list/?series=411216)
- Changed test trigger from module format into late_initcall()
- Marked all functions with __init to be freed after completion
- Changed all __PGTABLE_PXX_FOLDED checks as mm_pxx_folded()
- Folded in PPC32 fixes from Christophe
Changes in V2:
https://lore.kernel.org/linux-mm/1568268173-31302-1-git-send-email-anshuman.khandual@arm.com/T/#t
- Fixed small typo error in MODULE_DESCRIPTION()
- Fixed m64k build problems for lvalue concerns in pmd_xxx_tests()
- Fixed dynamic page table level folding problems on x86 as per Kirril
- Fixed second pointers during pxx_populate_tests() per Kirill and Gerald
- Allocate and free pte table with pte_alloc_one/pte_free per Kirill
- Modified pxx_clear_tests() to accommodate s390 lower 12 bits situation
- Changed RANDOM_NZVALUE value from 0xbe to 0xff
- Changed allocation, usage, free sequence for saved_ptep
- Renamed VMA_FLAGS as VMFLAGS
- Implemented a new method for random vaddr generation
- Implemented some other cleanups
- Dropped extern reference to mm_alloc()
- Created and exported new alloc_gigantic_page_order()
- Dropped the custom allocator and used new alloc_gigantic_page_order()
Changes in V1:
https://lore.kernel.org/linux-mm/1567497706-8649-1-git-send-email-anshuman.khandual@arm.com/
- Added fallback mechanism for PMD aligned memory allocation failure
Changes in RFC V2:
https://lore.kernel.org/linux-mm/1565335998-22553-1-git-send-email-anshuman.khandual@arm.com/T/#u
- Moved test module and it's config from lib/ to mm/
- Renamed config TEST_ARCH_PGTABLE as DEBUG_ARCH_PGTABLE_TEST
- Renamed file from test_arch_pgtable.c to arch_pgtable_test.c
- Added relevant MODULE_DESCRIPTION() and MODULE_AUTHOR() details
- Dropped loadable module config option
- Basic tests now use memory blocks with required size and alignment
- PUD aligned memory block gets allocated with alloc_contig_range()
- If PUD aligned memory could not be allocated it falls back on PMD aligned
memory block from page allocator and pud_* tests are skipped
- Clear and populate tests now operate on real in memory page table entries
- Dummy mm_struct gets allocated with mm_alloc()
- Dummy page table entries get allocated with [pud|pmd|pte]_alloc_[map]()
- Simplified [p4d|pgd]_basic_tests(), now has random values in the entries
Original RFC V1:
https://lore.kernel.org/linux-mm/1564037723-26676-1-git-send-email-anshuman.khandual@arm.com/
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Mike Rapoport <rppt@linux.ibm.com>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Kirill A. Shutemov <kirill@shutemov.name>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: linux-snps-arc@lists.infradead.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: linux-riscv@lists.infradead.org
Cc: x86@kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Anshuman Khandual (2):
x86/mm: Define mm_p4d_folded()
mm/debug: Add tests validating architecture page table helpers
.../debug/debug-vm-pgtable/arch-support.txt | 34 ++
arch/arc/Kconfig | 1 +
arch/arm64/Kconfig | 1 +
arch/powerpc/Kconfig | 1 +
arch/s390/Kconfig | 1 +
arch/x86/Kconfig | 1 +
arch/x86/include/asm/pgtable_64.h | 6 +
lib/Kconfig.debug | 22 +
mm/Makefile | 1 +
mm/debug_vm_pgtable.c | 382 ++++++++++++++++++
10 files changed, 450 insertions(+)
create mode 100644 Documentation/features/debug/debug-vm-pgtable/arch-support.txt
create mode 100644 mm/debug_vm_pgtable.c
--
2.20.1
^ permalink raw reply
* Re: [PATCH v7 05/28] powerpc: Change calling convention for create_branch() et. al.
From: Alistair Popple @ 2020-05-04 2:55 UTC (permalink / raw)
To: Jordan Niethe; +Cc: npiggin, bala24, naveen.n.rao, linuxppc-dev, dja
In-Reply-To: <20200501034220.8982-6-jniethe5@gmail.com>
On Friday, 1 May 2020 1:41:57 PM AEST Jordan Niethe wrote:
> create_branch(), create_cond_branch() and translate_branch() return the
> instruction that they create, or return 0 to signal an error. Separate
> these concerns in preparation for an instruction type that is not just
> an unsigned int. Fill the created instruction to a pointer passed as
> the first parameter to the function and use a non-zero return value to
> signify an error.
We're not adding any new checks for error cases here, but this patch doesn't
change existing behaviour as far as I can tell and adding checks would be
difficult (and could introduce other bugs) so would be best done as a separate
series anyway if required at all.
> @@ -403,6 +407,7 @@ static void __init test_trampoline(void)
>
> static void __init test_branch_iform(void)
> {
> + int err;
> unsigned int instr;
> unsigned long addr;
>
> @@ -443,35 +448,35 @@ static void __init test_branch_iform(void)
> check(instr_is_branch_to_addr(&instr, addr - 0x2000000));
>
> /* Branch to self, with link */
> - instr = create_branch(&instr, addr, BRANCH_SET_LINK);
> + err = create_branch(&instr, &instr, addr, BRANCH_SET_LINK);
> check(instr_is_branch_to_addr(&instr, addr));
The pointer alias above initially caught my eye, but it's ok because
create_branch() doesn't actually dereference the second instance. Arguably the
argument type could be changed to an unsigned long but then we'd just end up
with more casts so this is ok to me at least.
Reviewed-by: Alistair Popple <alistair@popple.id.au>
^ permalink raw reply
* Re: [PATCH V2 00/11] Subject: Remove duplicated kmap code
From: Al Viro @ 2020-05-04 1:35 UTC (permalink / raw)
To: ira.weiny
Cc: Peter Zijlstra, Dave Hansen, dri-devel, linux-mips,
James E.J. Bottomley, Max Filippov, Huang Rui, Paul Mackerras,
H. Peter Anvin, sparclinux, Dan Williams, Helge Deller, x86,
linux-csky, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
linux-arm-kernel, Chris Zankel, Thomas Bogendoerfer, linux-parisc,
linux-kernel, Christian Koenig, Andrew Morton, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
On Sun, May 03, 2020 at 06:09:01PM -0700, ira.weiny@intel.com wrote:
> From: Ira Weiny <ira.weiny@intel.com>
>
> The kmap infrastructure has been copied almost verbatim to every architecture.
> This series consolidates obvious duplicated code by defining core functions
> which call into the architectures only when needed.
>
> Some of the k[un]map_atomic() implementations have some similarities but the
> similarities were not sufficient to warrant further changes.
>
> In addition we remove a duplicate implementation of kmap() in DRM.
>
> Testing was done by 0day to cover all the architectures I can't readily
> build/test.
OK... Looking through my old notes on kmap unification (this winter, never
went anywhere),
* arch/mips/mm/cache.c ought to use linux/highmem.h, not asm/highmem.h
I suspect that your series doesn't build on some configs there. Hadn't
verified that, though.
* kmap_atomic_to_page() is dead, but not quite gone - csky and nds32 brought
the damn thing back (nds32 - only an extern). It needs killin'...
* parisc is (arguably) abusing kunmap()/kunmap_atomic() for cache flushing.
Replace the bulk of its highmem.h with
#define ARCH_HAS_FLUSH_ON_KUNMAP
#define arch_before_kunmap flush_kernel_dcache_page_addr
and have default kunmap()/kunmap_atomic() do
#ifdef ARCH_HAS_FLUSH_ON_KUNMAP
arch_before_kunmap(page_address(page));
#endif
and
#ifdef ARCH_HAS_FLUSH_ON_KUNMAP
arch_before_kunmap(addr);
#endif
resp. Kills ARCH_HAS_KMAP along with ifdefs on it, makes parisc use somewhat
less hacky.
I'd suggest checking various configs on mips - that's likely to cause headache.
Said that, my analysis of include chains back then is pretty much worthless
by now - I really hate the amount of indirect include chains leading to that
sucker on some, but not all configs ;-/ IIRC, the proof that everything
using kmap*/kunmap* would pull linux/highmem.h regardless of config took several
hours of digging, ran for several pages and had been hopelessly brittle.
arch/mips/mm/cache.c was the only exception caught by it, but these days
there might be more.
^ permalink raw reply
* [PATCH V2 02/11] arch/xtensa: Move kmap build bug out of the way
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Christoph Hellwig, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Dan Williams, linux-arm-kernel,
Chris Zankel, Thomas Bogendoerfer, linux-parisc, linux-mips,
linuxppc-dev, David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
Move the kmap() build bug to kmap_init() to facilitate patches to lift
kmap() to the core.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes from V1:
combine code onto 1 line.
---
arch/xtensa/include/asm/highmem.h | 5 -----
arch/xtensa/mm/highmem.c | 4 ++++
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/arch/xtensa/include/asm/highmem.h b/arch/xtensa/include/asm/highmem.h
index 413848cc1e56..a9587c85be85 100644
--- a/arch/xtensa/include/asm/highmem.h
+++ b/arch/xtensa/include/asm/highmem.h
@@ -68,11 +68,6 @@ void kunmap_high(struct page *page);
static inline void *kmap(struct page *page)
{
- /* Check if this memory layout is broken because PKMAP overlaps
- * page table.
- */
- BUILD_BUG_ON(PKMAP_BASE <
- TLBTEMP_BASE_1 + TLBTEMP_SIZE);
might_sleep();
if (!PageHighMem(page))
return page_address(page);
diff --git a/arch/xtensa/mm/highmem.c b/arch/xtensa/mm/highmem.c
index 184ceadccc1a..da734a2ed641 100644
--- a/arch/xtensa/mm/highmem.c
+++ b/arch/xtensa/mm/highmem.c
@@ -88,6 +88,10 @@ void __init kmap_init(void)
{
unsigned long kmap_vstart;
+ /* Check if this memory layout is broken because PKMAP overlaps
+ * page table.
+ */
+ BUILD_BUG_ON(PKMAP_BASE < TLBTEMP_BASE_1 + TLBTEMP_SIZE);
/* cache the first kmap pte */
kmap_vstart = __fix_to_virt(FIX_KMAP_BEGIN);
kmap_pte = kmap_get_fixmap_pte(kmap_vstart);
--
2.25.1
^ permalink raw reply related
* [PATCH V2 09/11] arch/kmap: Don't hard code kmap_prot values
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Christoph Hellwig, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Dan Williams, linux-arm-kernel,
Chris Zankel, Thomas Bogendoerfer, linux-parisc, linux-mips,
linuxppc-dev, David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
To support kmap_atomic_prot() on all architectures each arch must
support protections passed in to them.
Change csky, mips, nds32 and xtensa to use their global constant
kmap_prot rather than a hard coded value which was equal.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
changes from V1:
Mention that kmap_prot is a constant in commit message
---
arch/csky/mm/highmem.c | 2 +-
arch/mips/mm/highmem.c | 2 +-
arch/nds32/mm/highmem.c | 2 +-
arch/xtensa/mm/highmem.c | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/csky/mm/highmem.c b/arch/csky/mm/highmem.c
index 0aafbbbe651c..f4311669b5bb 100644
--- a/arch/csky/mm/highmem.c
+++ b/arch/csky/mm/highmem.c
@@ -32,7 +32,7 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte - idx)));
#endif
- set_pte(kmap_pte-idx, mk_pte(page, PAGE_KERNEL));
+ set_pte(kmap_pte-idx, mk_pte(page, kmap_prot));
flush_tlb_one((unsigned long)vaddr);
return (void *)vaddr;
diff --git a/arch/mips/mm/highmem.c b/arch/mips/mm/highmem.c
index 155fbb107b35..87023bd1a33c 100644
--- a/arch/mips/mm/highmem.c
+++ b/arch/mips/mm/highmem.c
@@ -29,7 +29,7 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte - idx)));
#endif
- set_pte(kmap_pte-idx, mk_pte(page, PAGE_KERNEL));
+ set_pte(kmap_pte-idx, mk_pte(page, kmap_prot));
local_flush_tlb_one((unsigned long)vaddr);
return (void*) vaddr;
diff --git a/arch/nds32/mm/highmem.c b/arch/nds32/mm/highmem.c
index f6e6915c0d31..809f8c830f06 100644
--- a/arch/nds32/mm/highmem.c
+++ b/arch/nds32/mm/highmem.c
@@ -21,7 +21,7 @@ void *kmap_atomic_high(struct page *page)
idx = type + KM_TYPE_NR * smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
- pte = (page_to_pfn(page) << PAGE_SHIFT) | (PAGE_KERNEL);
+ pte = (page_to_pfn(page) << PAGE_SHIFT) | (kmap_prot);
ptep = pte_offset_kernel(pmd_off_k(vaddr), vaddr);
set_pte(ptep, pte);
diff --git a/arch/xtensa/mm/highmem.c b/arch/xtensa/mm/highmem.c
index 4de323e43682..50168b09510a 100644
--- a/arch/xtensa/mm/highmem.c
+++ b/arch/xtensa/mm/highmem.c
@@ -48,7 +48,7 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte + idx)));
#endif
- set_pte(kmap_pte + idx, mk_pte(page, PAGE_KERNEL_EXEC));
+ set_pte(kmap_pte + idx, mk_pte(page, kmap_prot));
return (void *)vaddr;
}
--
2.25.1
^ permalink raw reply related
* [PATCH V2 07/11] arch/kunmap_atomic: Consolidate duplicate code
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Ingo Molnar, linux-snps-arc, linux-xtensa, Borislav Petkov,
Andy Lutomirski, Dan Williams, linux-arm-kernel, Chris Zankel,
Thomas Bogendoerfer, linux-parisc, linux-mips, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
Every single architecture (including !CONFIG_HIGHMEM) calls...
pagefault_enable();
preempt_enable();
... before returning from __kunmap_atomic(). Lift this code into the
kunmap_atomic() macro.
While we are at it rename __kunmap_atomic() to kunmap_atomic_high() to
be consistent.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes from V1:
Adjust to preserve bisect-ability
Remove uneeded kunmap_atomic_high() declarations
---
arch/arc/include/asm/highmem.h | 2 --
arch/arc/mm/highmem.c | 7 ++-----
arch/arm/include/asm/highmem.h | 1 -
arch/arm/mm/highmem.c | 6 ++----
arch/csky/include/asm/highmem.h | 1 -
arch/csky/mm/highmem.c | 9 +++------
arch/microblaze/include/asm/highmem.h | 1 -
arch/microblaze/mm/highmem.c | 6 ++----
arch/mips/include/asm/highmem.h | 1 -
arch/mips/mm/cache.c | 4 ++--
arch/mips/mm/highmem.c | 6 ++----
arch/nds32/include/asm/highmem.h | 1 -
arch/nds32/mm/highmem.c | 6 ++----
arch/parisc/include/asm/cacheflush.h | 4 +---
arch/powerpc/include/asm/highmem.h | 1 -
arch/powerpc/mm/highmem.c | 6 ++----
arch/sparc/include/asm/highmem.h | 2 --
arch/sparc/mm/highmem.c | 6 ++----
arch/x86/include/asm/highmem.h | 1 -
arch/x86/mm/highmem_32.c | 7 ++-----
arch/xtensa/include/asm/highmem.h | 2 --
arch/xtensa/mm/highmem.c | 7 ++-----
include/linux/highmem.h | 11 +++++++----
23 files changed, 31 insertions(+), 67 deletions(-)
diff --git a/arch/arc/include/asm/highmem.h b/arch/arc/include/asm/highmem.h
index db425cd38545..70900a73bfc8 100644
--- a/arch/arc/include/asm/highmem.h
+++ b/arch/arc/include/asm/highmem.h
@@ -30,8 +30,6 @@
#include <asm/cacheflush.h>
-extern void __kunmap_atomic(void *kvaddr);
-
extern void kmap_init(void);
static inline void flush_cache_kmaps(void)
diff --git a/arch/arc/mm/highmem.c b/arch/arc/mm/highmem.c
index 0964b011c29f..5d3eab4ac0b0 100644
--- a/arch/arc/mm/highmem.c
+++ b/arch/arc/mm/highmem.c
@@ -65,7 +65,7 @@ void *kmap_atomic_high(struct page *page)
}
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kv)
+void kunmap_atomic_high(void *kv)
{
unsigned long kvaddr = (unsigned long)kv;
@@ -87,11 +87,8 @@ void __kunmap_atomic(void *kv)
kmap_atomic_idx_pop();
}
-
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
static noinline pte_t * __init alloc_kmap_pgtable(unsigned long kvaddr)
{
diff --git a/arch/arm/include/asm/highmem.h b/arch/arm/include/asm/highmem.h
index 8c80bfe18a34..b0d4bd8dc3c1 100644
--- a/arch/arm/include/asm/highmem.h
+++ b/arch/arm/include/asm/highmem.h
@@ -60,7 +60,6 @@ static inline void *kmap_high_get(struct page *page)
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
-extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
#endif
diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c
index 075fdc235091..ac8394655a6e 100644
--- a/arch/arm/mm/highmem.c
+++ b/arch/arm/mm/highmem.c
@@ -73,7 +73,7 @@ void *kmap_atomic_high(struct page *page)
}
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int idx, type;
@@ -95,10 +95,8 @@ void __kunmap_atomic(void *kvaddr)
/* this address was obtained through kmap_high_get() */
kunmap_high(pte_page(pkmap_page_table[PKMAP_NR(vaddr)]));
}
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
void *kmap_atomic_pfn(unsigned long pfn)
{
diff --git a/arch/csky/include/asm/highmem.h b/arch/csky/include/asm/highmem.h
index 8ceee12f9bc1..263fbddcd0a3 100644
--- a/arch/csky/include/asm/highmem.h
+++ b/arch/csky/include/asm/highmem.h
@@ -32,7 +32,6 @@ extern pte_t *pkmap_page_table;
#define ARCH_HAS_KMAP_FLUSH_TLB
extern void kmap_flush_tlb(unsigned long addr);
-extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
extern struct page *kmap_atomic_to_page(void *ptr);
diff --git a/arch/csky/mm/highmem.c b/arch/csky/mm/highmem.c
index 63d74b47eee6..0aafbbbe651c 100644
--- a/arch/csky/mm/highmem.c
+++ b/arch/csky/mm/highmem.c
@@ -39,13 +39,13 @@ void *kmap_atomic_high(struct page *page)
}
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int idx;
if (vaddr < FIXADDR_START)
- goto out;
+ return;
#ifdef CONFIG_DEBUG_HIGHMEM
idx = KM_TYPE_NR*smp_processor_id() + kmap_atomic_idx();
@@ -58,11 +58,8 @@ void __kunmap_atomic(void *kvaddr)
(void) idx; /* to kill a warning */
#endif
kmap_atomic_idx_pop();
-out:
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
/*
* This is the same as kmap_atomic() but can map memory that doesn't
diff --git a/arch/microblaze/include/asm/highmem.h b/arch/microblaze/include/asm/highmem.h
index 00c75a423ac4..1b8a3c5102bd 100644
--- a/arch/microblaze/include/asm/highmem.h
+++ b/arch/microblaze/include/asm/highmem.h
@@ -61,7 +61,6 @@ void *kmap_atomic_prot(struct page *page, pgprot_t prot)
return kmap_atomic_high_prot(page, prot);
}
-extern void __kunmap_atomic(void *kvaddr);
static inline void *kmap_atomic_high(struct page *page)
{
diff --git a/arch/microblaze/mm/highmem.c b/arch/microblaze/mm/highmem.c
index 0e3efaa8a004..ee8a422b2b76 100644
--- a/arch/microblaze/mm/highmem.c
+++ b/arch/microblaze/mm/highmem.c
@@ -51,7 +51,7 @@ void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int type;
@@ -77,7 +77,5 @@ void __kunmap_atomic(void *kvaddr)
local_flush_tlb_page(NULL, vaddr);
kmap_atomic_idx_pop();
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
diff --git a/arch/mips/include/asm/highmem.h b/arch/mips/include/asm/highmem.h
index 8bdbbfc322ad..76dec0bd4f59 100644
--- a/arch/mips/include/asm/highmem.h
+++ b/arch/mips/include/asm/highmem.h
@@ -48,7 +48,6 @@ extern pte_t *pkmap_page_table;
#define ARCH_HAS_KMAP_FLUSH_TLB
extern void kmap_flush_tlb(unsigned long addr);
-extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
#define flush_cache_kmaps() BUG_ON(cpu_has_dc_aliases)
diff --git a/arch/mips/mm/cache.c b/arch/mips/mm/cache.c
index f015bb51fab0..1873c2a01fdb 100644
--- a/arch/mips/mm/cache.c
+++ b/arch/mips/mm/cache.c
@@ -104,7 +104,7 @@ void __flush_dcache_page(struct page *page)
flush_data_cache_page(addr);
if (PageHighMem(page))
- __kunmap_atomic((void *)addr);
+ kunmap_atomic((void *)addr);
}
EXPORT_SYMBOL(__flush_dcache_page);
@@ -147,7 +147,7 @@ void __update_cache(unsigned long address, pte_t pte)
flush_data_cache_page(addr);
if (PageHighMem(page))
- __kunmap_atomic((void *)addr);
+ kunmap_atomic((void *)addr);
ClearPageDcacheDirty(page);
}
diff --git a/arch/mips/mm/highmem.c b/arch/mips/mm/highmem.c
index 2bda56372995..155fbb107b35 100644
--- a/arch/mips/mm/highmem.c
+++ b/arch/mips/mm/highmem.c
@@ -36,7 +36,7 @@ void *kmap_atomic_high(struct page *page)
}
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int type __maybe_unused;
@@ -63,10 +63,8 @@ void __kunmap_atomic(void *kvaddr)
}
#endif
kmap_atomic_idx_pop();
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
/*
* This is the same as kmap_atomic() but can map memory that doesn't
diff --git a/arch/nds32/include/asm/highmem.h b/arch/nds32/include/asm/highmem.h
index a3970e566ede..4d21308549c9 100644
--- a/arch/nds32/include/asm/highmem.h
+++ b/arch/nds32/include/asm/highmem.h
@@ -51,7 +51,6 @@ extern void kmap_init(void);
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
-extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
extern struct page *kmap_atomic_to_page(void *ptr);
#endif
diff --git a/arch/nds32/mm/highmem.c b/arch/nds32/mm/highmem.c
index f5f3a21460c4..f6e6915c0d31 100644
--- a/arch/nds32/mm/highmem.c
+++ b/arch/nds32/mm/highmem.c
@@ -34,7 +34,7 @@ void *kmap_atomic_high(struct page *page)
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
if (kvaddr >= (void *)FIXADDR_START) {
unsigned long vaddr = (unsigned long)kvaddr;
@@ -45,8 +45,6 @@ void __kunmap_atomic(void *kvaddr)
ptep = pte_offset_kernel(pmd_off_k(vaddr), vaddr);
set_pte(ptep, 0);
}
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
diff --git a/arch/parisc/include/asm/cacheflush.h b/arch/parisc/include/asm/cacheflush.h
index 0c83644bfa5c..119c9a7681bc 100644
--- a/arch/parisc/include/asm/cacheflush.h
+++ b/arch/parisc/include/asm/cacheflush.h
@@ -122,11 +122,9 @@ static inline void *kmap_atomic(struct page *page)
return page_address(page);
}
-static inline void __kunmap_atomic(void *addr)
+static inline void kunmap_atomic_high(void *addr)
{
flush_kernel_dcache_page_addr(addr);
- pagefault_enable();
- preempt_enable();
}
#define kmap_atomic_prot(page, prot) kmap_atomic(page)
diff --git a/arch/powerpc/include/asm/highmem.h b/arch/powerpc/include/asm/highmem.h
index 74fa2c726fde..373a470df205 100644
--- a/arch/powerpc/include/asm/highmem.h
+++ b/arch/powerpc/include/asm/highmem.h
@@ -69,7 +69,6 @@ static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
return kmap_atomic_high_prot(page, prot);
}
-extern void __kunmap_atomic(void *kvaddr);
static inline void *kmap_atomic_high(struct page *page)
{
diff --git a/arch/powerpc/mm/highmem.c b/arch/powerpc/mm/highmem.c
index 67aaa5217f7f..35071c2913f1 100644
--- a/arch/powerpc/mm/highmem.c
+++ b/arch/powerpc/mm/highmem.c
@@ -40,7 +40,7 @@ void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
@@ -66,7 +66,5 @@ void __kunmap_atomic(void *kvaddr)
}
kmap_atomic_idx_pop();
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
diff --git a/arch/sparc/include/asm/highmem.h b/arch/sparc/include/asm/highmem.h
index 458210c5bc38..f4babe67cb5d 100644
--- a/arch/sparc/include/asm/highmem.h
+++ b/arch/sparc/include/asm/highmem.h
@@ -50,8 +50,6 @@ void kmap_init(void) __init;
#define PKMAP_END (PKMAP_ADDR(LAST_PKMAP))
-void __kunmap_atomic(void *kvaddr);
-
#define flush_cache_kmaps() flush_cache_all()
#endif /* __KERNEL__ */
diff --git a/arch/sparc/mm/highmem.c b/arch/sparc/mm/highmem.c
index b53070ab6a31..469786bc430f 100644
--- a/arch/sparc/mm/highmem.c
+++ b/arch/sparc/mm/highmem.c
@@ -84,7 +84,7 @@ void *kmap_atomic_high(struct page *page)
}
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
int type;
@@ -126,7 +126,5 @@ void __kunmap_atomic(void *kvaddr)
#endif
kmap_atomic_idx_pop();
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
diff --git a/arch/x86/include/asm/highmem.h b/arch/x86/include/asm/highmem.h
index 9393d55a2adb..be66b77885a0 100644
--- a/arch/x86/include/asm/highmem.h
+++ b/arch/x86/include/asm/highmem.h
@@ -72,7 +72,6 @@ static inline void *kmap_atomic_high(struct page *page)
{
return kmap_atomic_high_prot(page, kmap_prot);
}
-void __kunmap_atomic(void *kvaddr);
void *kmap_atomic_pfn(unsigned long pfn);
void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot);
diff --git a/arch/x86/mm/highmem_32.c b/arch/x86/mm/highmem_32.c
index c3e272a759e0..075fe51317b0 100644
--- a/arch/x86/mm/highmem_32.c
+++ b/arch/x86/mm/highmem_32.c
@@ -30,7 +30,7 @@ void *kmap_atomic_pfn(unsigned long pfn)
}
EXPORT_SYMBOL_GPL(kmap_atomic_pfn);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
unsigned long vaddr = (unsigned long) kvaddr & PAGE_MASK;
@@ -60,11 +60,8 @@ void __kunmap_atomic(void *kvaddr)
BUG_ON(vaddr >= (unsigned long)high_memory);
}
#endif
-
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
void __init set_highmem_pages_init(void)
{
diff --git a/arch/xtensa/include/asm/highmem.h b/arch/xtensa/include/asm/highmem.h
index 1e6aa15c4bdf..d6a10704307a 100644
--- a/arch/xtensa/include/asm/highmem.h
+++ b/arch/xtensa/include/asm/highmem.h
@@ -68,8 +68,6 @@ static inline void flush_cache_kmaps(void)
flush_cache_all();
}
-void __kunmap_atomic(void *kvaddr);
-
void kmap_init(void);
#endif
diff --git a/arch/xtensa/mm/highmem.c b/arch/xtensa/mm/highmem.c
index 90b85a897cb0..4de323e43682 100644
--- a/arch/xtensa/mm/highmem.c
+++ b/arch/xtensa/mm/highmem.c
@@ -54,7 +54,7 @@ void *kmap_atomic_high(struct page *page)
}
EXPORT_SYMBOL(kmap_atomic_high);
-void __kunmap_atomic(void *kvaddr)
+void kunmap_atomic_high(void *kvaddr)
{
if (kvaddr >= (void *)FIXADDR_START &&
kvaddr < (void *)FIXADDR_TOP) {
@@ -73,11 +73,8 @@ void __kunmap_atomic(void *kvaddr)
kmap_atomic_idx_pop();
}
-
- pagefault_enable();
- preempt_enable();
}
-EXPORT_SYMBOL(__kunmap_atomic);
+EXPORT_SYMBOL(kunmap_atomic_high);
void __init kmap_init(void)
{
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index 86b93dee758a..c36c0ee09423 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -33,6 +33,7 @@ static inline void invalidate_kernel_vmap_range(void *vaddr, int size)
#ifdef CONFIG_HIGHMEM
extern void *kmap_atomic_high(struct page *page);
+extern void kunmap_atomic_high(void *kvaddr);
#include <asm/highmem.h>
#ifndef ARCH_HAS_KMAP_FLUSH_TLB
@@ -146,10 +147,10 @@ static inline void *kmap_atomic(struct page *page)
}
#define kmap_atomic_prot(page, prot) kmap_atomic(page)
-static inline void __kunmap_atomic(void *addr)
+static inline void kunmap_atomic_high(void *addr)
{
- pagefault_enable();
- preempt_enable();
+ /* Nothing to do in the CONFIG_HIGHMEM=n case as kunmap_atomic()
+ * handles re-enabling faults + preemption */
}
#define kmap_atomic_pfn(pfn) kmap_atomic(pfn_to_page(pfn))
@@ -199,7 +200,9 @@ static inline void kmap_atomic_idx_pop(void)
#define kunmap_atomic(addr) \
do { \
BUILD_BUG_ON(__same_type((addr), struct page *)); \
- __kunmap_atomic(addr); \
+ kunmap_atomic_high(addr); \
+ pagefault_enable(); \
+ preempt_enable(); \
} while (0)
--
2.25.1
^ permalink raw reply related
* [PATCH V2 01/11] arch/kmap: Remove BUG_ON()
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Dan Williams, Helge Deller, x86, linux-csky,
Christoph Hellwig, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
linux-arm-kernel, Chris Zankel, Thomas Bogendoerfer, linux-parisc,
linux-mips, linuxppc-dev, David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
Replace the use of BUG_ON(in_interrupt()) in the kmap() and kunmap()
in favor of might_sleep().
Besides the benefits of might_sleep(), this normalizes the
implementations such that they can be made generic in subsequent
patches.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
arch/arc/include/asm/highmem.h | 2 +-
arch/arc/mm/highmem.c | 2 +-
arch/arm/mm/highmem.c | 2 +-
arch/csky/mm/highmem.c | 2 +-
arch/microblaze/include/asm/highmem.h | 2 +-
arch/mips/mm/highmem.c | 2 +-
arch/nds32/mm/highmem.c | 2 +-
arch/powerpc/include/asm/highmem.h | 2 +-
arch/sparc/include/asm/highmem.h | 4 ++--
arch/x86/mm/highmem_32.c | 3 +--
arch/xtensa/include/asm/highmem.h | 4 ++--
11 files changed, 13 insertions(+), 14 deletions(-)
diff --git a/arch/arc/include/asm/highmem.h b/arch/arc/include/asm/highmem.h
index 1af00accb37f..042e92921c4c 100644
--- a/arch/arc/include/asm/highmem.h
+++ b/arch/arc/include/asm/highmem.h
@@ -45,7 +45,7 @@ static inline void flush_cache_kmaps(void)
static inline void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/arc/mm/highmem.c b/arch/arc/mm/highmem.c
index fc8849e4f72e..39ef7b9a3aa9 100644
--- a/arch/arc/mm/highmem.c
+++ b/arch/arc/mm/highmem.c
@@ -51,7 +51,7 @@ static pte_t * fixmap_page_table;
void *kmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return page_address(page);
diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c
index a76f8ace9ce6..cc6eb79ef20c 100644
--- a/arch/arm/mm/highmem.c
+++ b/arch/arm/mm/highmem.c
@@ -42,7 +42,7 @@ EXPORT_SYMBOL(kmap);
void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/csky/mm/highmem.c b/arch/csky/mm/highmem.c
index 813129145f3d..690d678649d1 100644
--- a/arch/csky/mm/highmem.c
+++ b/arch/csky/mm/highmem.c
@@ -29,7 +29,7 @@ EXPORT_SYMBOL(kmap);
void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/microblaze/include/asm/highmem.h b/arch/microblaze/include/asm/highmem.h
index 332c78e15198..99ced7278b5c 100644
--- a/arch/microblaze/include/asm/highmem.h
+++ b/arch/microblaze/include/asm/highmem.h
@@ -66,7 +66,7 @@ static inline void *kmap(struct page *page)
static inline void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/mips/mm/highmem.c b/arch/mips/mm/highmem.c
index d08e6d7d533b..edd889f6cede 100644
--- a/arch/mips/mm/highmem.c
+++ b/arch/mips/mm/highmem.c
@@ -28,7 +28,7 @@ EXPORT_SYMBOL(kmap);
void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/nds32/mm/highmem.c b/arch/nds32/mm/highmem.c
index 022779af6148..4c7c28e994ea 100644
--- a/arch/nds32/mm/highmem.c
+++ b/arch/nds32/mm/highmem.c
@@ -24,7 +24,7 @@ EXPORT_SYMBOL(kmap);
void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/powerpc/include/asm/highmem.h b/arch/powerpc/include/asm/highmem.h
index a4b65b186ec6..529512f6d65a 100644
--- a/arch/powerpc/include/asm/highmem.h
+++ b/arch/powerpc/include/asm/highmem.h
@@ -74,7 +74,7 @@ static inline void *kmap(struct page *page)
static inline void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/sparc/include/asm/highmem.h b/arch/sparc/include/asm/highmem.h
index 18d776925c45..7dd2d4b3f980 100644
--- a/arch/sparc/include/asm/highmem.h
+++ b/arch/sparc/include/asm/highmem.h
@@ -55,7 +55,7 @@ void kunmap_high(struct page *page);
static inline void *kmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return page_address(page);
return kmap_high(page);
@@ -63,7 +63,7 @@ static inline void *kmap(struct page *page)
static inline void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/x86/mm/highmem_32.c b/arch/x86/mm/highmem_32.c
index 0a1898b8552e..8af66382672b 100644
--- a/arch/x86/mm/highmem_32.c
+++ b/arch/x86/mm/highmem_32.c
@@ -15,8 +15,7 @@ EXPORT_SYMBOL(kmap);
void kunmap(struct page *page)
{
- if (in_interrupt())
- BUG();
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
diff --git a/arch/xtensa/include/asm/highmem.h b/arch/xtensa/include/asm/highmem.h
index 04e9340eac4b..413848cc1e56 100644
--- a/arch/xtensa/include/asm/highmem.h
+++ b/arch/xtensa/include/asm/highmem.h
@@ -73,7 +73,7 @@ static inline void *kmap(struct page *page)
*/
BUILD_BUG_ON(PKMAP_BASE <
TLBTEMP_BASE_1 + TLBTEMP_SIZE);
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return page_address(page);
return kmap_high(page);
@@ -81,7 +81,7 @@ static inline void *kmap(struct page *page)
static inline void kunmap(struct page *page)
{
- BUG_ON(in_interrupt());
+ might_sleep();
if (!PageHighMem(page))
return;
kunmap_high(page);
--
2.25.1
^ permalink raw reply related
* [PATCH V2 05/11] {x86,powerpc,microblaze}/kmap: Move preempt disable
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Ingo Molnar, linux-snps-arc, linux-xtensa, Borislav Petkov,
Andy Lutomirski, Dan Williams, linux-arm-kernel, Chris Zankel,
Thomas Bogendoerfer, linux-parisc, linux-mips, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
During this kmap() conversion series we must maintain bisect-ability.
To do this, kmap_atomic_prot() in x86, powerpc, and microblaze need to
remain functional.
Create a temporary inline version of kmap_atomic_prot within these
architectures so we can rework their kmap_atomic() calls and then lift
kmap_atomic_prot() to the core.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes from V1:
New patch
---
arch/microblaze/include/asm/highmem.h | 11 ++++++++++-
arch/microblaze/mm/highmem.c | 10 ++--------
arch/powerpc/include/asm/highmem.h | 11 ++++++++++-
arch/powerpc/mm/highmem.c | 9 ++-------
arch/x86/include/asm/highmem.h | 11 ++++++++++-
arch/x86/mm/highmem_32.c | 10 ++--------
6 files changed, 36 insertions(+), 26 deletions(-)
diff --git a/arch/microblaze/include/asm/highmem.h b/arch/microblaze/include/asm/highmem.h
index 0c94046f2d58..ec9954b091e1 100644
--- a/arch/microblaze/include/asm/highmem.h
+++ b/arch/microblaze/include/asm/highmem.h
@@ -51,7 +51,16 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt - PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_atomic_prot(struct page *page, pgprot_t prot);
+extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
+void *kmap_atomic_prot(struct page *page, pgprot_t prot)
+{
+ preempt_disable();
+ pagefault_disable();
+ if (!PageHighMem(page))
+ return page_address(page);
+
+ return kmap_atomic_high_prot(page, prot);
+}
extern void __kunmap_atomic(void *kvaddr);
static inline void *kmap_atomic(struct page *page)
diff --git a/arch/microblaze/mm/highmem.c b/arch/microblaze/mm/highmem.c
index d7569f77fa15..0e3efaa8a004 100644
--- a/arch/microblaze/mm/highmem.c
+++ b/arch/microblaze/mm/highmem.c
@@ -32,18 +32,12 @@
*/
#include <asm/tlbflush.h>
-void *kmap_atomic_prot(struct page *page, pgprot_t prot)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -55,7 +49,7 @@ void *kmap_atomic_prot(struct page *page, pgprot_t prot)
return (void *) vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_prot);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/powerpc/include/asm/highmem.h b/arch/powerpc/include/asm/highmem.h
index ba3371977d49..d049806a8354 100644
--- a/arch/powerpc/include/asm/highmem.h
+++ b/arch/powerpc/include/asm/highmem.h
@@ -59,7 +59,16 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_atomic_prot(struct page *page, pgprot_t prot);
+extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
+static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
+{
+ preempt_disable();
+ pagefault_disable();
+ if (!PageHighMem(page))
+ return page_address(page);
+
+ return kmap_atomic_high_prot(page, prot);
+}
extern void __kunmap_atomic(void *kvaddr);
static inline void *kmap_atomic(struct page *page)
diff --git a/arch/powerpc/mm/highmem.c b/arch/powerpc/mm/highmem.c
index 320c1672b2ae..f075cef6d663 100644
--- a/arch/powerpc/mm/highmem.c
+++ b/arch/powerpc/mm/highmem.c
@@ -30,16 +30,11 @@
* be used in IRQ contexts, so in some (very limited) cases we need
* it.
*/
-void *kmap_atomic_prot(struct page *page, pgprot_t prot)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -49,7 +44,7 @@ void *kmap_atomic_prot(struct page *page, pgprot_t prot)
return (void*) vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_prot);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/x86/include/asm/highmem.h b/arch/x86/include/asm/highmem.h
index 90b96594d6c5..61f47fef40e5 100644
--- a/arch/x86/include/asm/highmem.h
+++ b/arch/x86/include/asm/highmem.h
@@ -58,7 +58,16 @@ extern unsigned long highstart_pfn, highend_pfn;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-void *kmap_atomic_prot(struct page *page, pgprot_t prot);
+extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
+static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
+{
+ preempt_disable();
+ pagefault_disable();
+ if (!PageHighMem(page))
+ return page_address(page);
+
+ return kmap_atomic_high_prot(page, prot);
+}
void *kmap_atomic(struct page *page);
void __kunmap_atomic(void *kvaddr);
void *kmap_atomic_pfn(unsigned long pfn);
diff --git a/arch/x86/mm/highmem_32.c b/arch/x86/mm/highmem_32.c
index c4ebfd0ae401..48b56b1af902 100644
--- a/arch/x86/mm/highmem_32.c
+++ b/arch/x86/mm/highmem_32.c
@@ -12,17 +12,11 @@
* However when holding an atomic kmap it is not legal to sleep, so atomic
* kmaps are appropriate for short, tight code paths only.
*/
-void *kmap_atomic_prot(struct page *page, pgprot_t prot)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
- preempt_disable();
- pagefault_disable();
-
- if (!PageHighMem(page))
- return page_address(page);
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -32,7 +26,7 @@ void *kmap_atomic_prot(struct page *page, pgprot_t prot)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_prot);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void *kmap_atomic(struct page *page)
{
--
2.25.1
^ permalink raw reply related
* [PATCH V2 00/11] Subject: Remove duplicated kmap code
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Ingo Molnar, linux-snps-arc, linux-xtensa, Borislav Petkov,
Andy Lutomirski, Dan Williams, linux-arm-kernel, Chris Zankel,
Thomas Bogendoerfer, linux-parisc, linux-mips, linuxppc-dev,
David S. Miller
From: Ira Weiny <ira.weiny@intel.com>
The kmap infrastructure has been copied almost verbatim to every architecture.
This series consolidates obvious duplicated code by defining core functions
which call into the architectures only when needed.
Some of the k[un]map_atomic() implementations have some similarities but the
similarities were not sufficient to warrant further changes.
In addition we remove a duplicate implementation of kmap() in DRM.
Testing was done by 0day to cover all the architectures I can't readily
build/test.
---
Changes from V1:
Fix bisect-ability
Update commit message and fix line lengths
Remove unneded kunmap_atomic_high() declarations
Remove unneded kmap_atomic_high() declarations
collect reviews
rebase to 5.7-rc4
Changes from V0:
Define kmap_flush_tlb() and make kmap() truely arch independent.
Redefine the k[un]map_atomic_* code to call into the architectures for
high mem pages
Ensure all architectures define kmap_prot, use it appropriately, and
define kmap_atomic_prot()
Remove drm implementation of kmap_atomic()
Ira Weiny (11):
arch/kmap: Remove BUG_ON()
arch/xtensa: Move kmap build bug out of the way
arch/kmap: Remove redundant arch specific kmaps
arch/kunmap: Remove duplicate kunmap implementations
{x86,powerpc,microblaze}/kmap: Move preempt disable
arch/kmap_atomic: Consolidate duplicate code
arch/kunmap_atomic: Consolidate duplicate code
arch/kmap: Ensure kmap_prot visibility
arch/kmap: Don't hard code kmap_prot values
arch/kmap: Define kmap_atomic_prot() for all arch's
drm: Remove drm specific kmap_atomic code
arch/arc/include/asm/highmem.h | 15 -------
arch/arc/mm/highmem.c | 28 +++---------
arch/arm/include/asm/highmem.h | 7 ---
arch/arm/mm/highmem.c | 35 +++------------
arch/csky/include/asm/highmem.h | 9 +---
arch/csky/mm/highmem.c | 43 +++++--------------
arch/microblaze/include/asm/highmem.h | 28 +-----------
arch/microblaze/mm/highmem.c | 16 ++-----
arch/microblaze/mm/init.c | 3 --
arch/mips/include/asm/highmem.h | 9 +---
arch/mips/mm/cache.c | 6 +--
arch/mips/mm/highmem.c | 49 ++++-----------------
arch/nds32/include/asm/highmem.h | 7 ---
arch/nds32/mm/highmem.c | 39 +++--------------
arch/parisc/include/asm/cacheflush.h | 4 +-
arch/powerpc/include/asm/highmem.h | 29 +------------
arch/powerpc/mm/highmem.c | 21 ++-------
arch/powerpc/mm/mem.c | 3 --
arch/sparc/include/asm/highmem.h | 22 ----------
arch/sparc/mm/highmem.c | 18 +++-----
arch/x86/include/asm/highmem.h | 9 ----
arch/x86/mm/highmem_32.c | 50 ++-------------------
arch/xtensa/include/asm/highmem.h | 27 ------------
arch/xtensa/mm/highmem.c | 22 ++++------
drivers/gpu/drm/ttm/ttm_bo_util.c | 56 ++----------------------
drivers/gpu/drm/vmwgfx/vmwgfx_blit.c | 16 +++----
include/drm/ttm/ttm_bo_api.h | 4 --
include/linux/highmem.h | 62 +++++++++++++++++++++++++--
28 files changed, 140 insertions(+), 497 deletions(-)
--
2.25.1
^ permalink raw reply
* [PATCH V2 11/11] drm: Remove drm specific kmap_atomic code
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Christoph Hellwig, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Dan Williams, linux-arm-kernel,
Chris Zankel, Thomas Bogendoerfer, linux-parisc, linux-mips,
linuxppc-dev, David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
kmap_atomic_prot() is now exported by all architectures. Use this
function rather than open coding a driver specific kmap_atomic.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
drivers/gpu/drm/ttm/ttm_bo_util.c | 56 ++--------------------------
drivers/gpu/drm/vmwgfx/vmwgfx_blit.c | 16 ++++----
include/drm/ttm/ttm_bo_api.h | 4 --
3 files changed, 12 insertions(+), 64 deletions(-)
diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c
index 52d2b71f1588..f09b096ba4fd 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_util.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_util.c
@@ -257,54 +257,6 @@ static int ttm_copy_io_page(void *dst, void *src, unsigned long page)
return 0;
}
-#ifdef CONFIG_X86
-#define __ttm_kmap_atomic_prot(__page, __prot) kmap_atomic_prot(__page, __prot)
-#define __ttm_kunmap_atomic(__addr) kunmap_atomic(__addr)
-#else
-#define __ttm_kmap_atomic_prot(__page, __prot) vmap(&__page, 1, 0, __prot)
-#define __ttm_kunmap_atomic(__addr) vunmap(__addr)
-#endif
-
-
-/**
- * ttm_kmap_atomic_prot - Efficient kernel map of a single page with
- * specified page protection.
- *
- * @page: The page to map.
- * @prot: The page protection.
- *
- * This function maps a TTM page using the kmap_atomic api if available,
- * otherwise falls back to vmap. The user must make sure that the
- * specified page does not have an aliased mapping with a different caching
- * policy unless the architecture explicitly allows it. Also mapping and
- * unmapping using this api must be correctly nested. Unmapping should
- * occur in the reverse order of mapping.
- */
-void *ttm_kmap_atomic_prot(struct page *page, pgprot_t prot)
-{
- if (pgprot_val(prot) == pgprot_val(PAGE_KERNEL))
- return kmap_atomic(page);
- else
- return __ttm_kmap_atomic_prot(page, prot);
-}
-EXPORT_SYMBOL(ttm_kmap_atomic_prot);
-
-/**
- * ttm_kunmap_atomic_prot - Unmap a page that was mapped using
- * ttm_kmap_atomic_prot.
- *
- * @addr: The virtual address from the map.
- * @prot: The page protection.
- */
-void ttm_kunmap_atomic_prot(void *addr, pgprot_t prot)
-{
- if (pgprot_val(prot) == pgprot_val(PAGE_KERNEL))
- kunmap_atomic(addr);
- else
- __ttm_kunmap_atomic(addr);
-}
-EXPORT_SYMBOL(ttm_kunmap_atomic_prot);
-
static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
unsigned long page,
pgprot_t prot)
@@ -316,13 +268,13 @@ static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
return -ENOMEM;
src = (void *)((unsigned long)src + (page << PAGE_SHIFT));
- dst = ttm_kmap_atomic_prot(d, prot);
+ dst = kmap_atomic_prot(d, prot);
if (!dst)
return -ENOMEM;
memcpy_fromio(dst, src, PAGE_SIZE);
- ttm_kunmap_atomic_prot(dst, prot);
+ kunmap_atomic(dst);
return 0;
}
@@ -338,13 +290,13 @@ static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
return -ENOMEM;
dst = (void *)((unsigned long)dst + (page << PAGE_SHIFT));
- src = ttm_kmap_atomic_prot(s, prot);
+ src = kmap_atomic_prot(s, prot);
if (!src)
return -ENOMEM;
memcpy_toio(dst, src, PAGE_SIZE);
- ttm_kunmap_atomic_prot(src, prot);
+ kunmap_atomic(src);
return 0;
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c
index bb46ca0c458f..94d456a1d1a9 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c
@@ -374,12 +374,12 @@ static int vmw_bo_cpu_blit_line(struct vmw_bo_blit_line_data *d,
copy_size = min_t(u32, copy_size, PAGE_SIZE - src_page_offset);
if (unmap_src) {
- ttm_kunmap_atomic_prot(d->src_addr, d->src_prot);
+ kunmap_atomic(d->src_addr);
d->src_addr = NULL;
}
if (unmap_dst) {
- ttm_kunmap_atomic_prot(d->dst_addr, d->dst_prot);
+ kunmap_atomic(d->dst_addr);
d->dst_addr = NULL;
}
@@ -388,8 +388,8 @@ static int vmw_bo_cpu_blit_line(struct vmw_bo_blit_line_data *d,
return -EINVAL;
d->dst_addr =
- ttm_kmap_atomic_prot(d->dst_pages[dst_page],
- d->dst_prot);
+ kmap_atomic_prot(d->dst_pages[dst_page],
+ d->dst_prot);
if (!d->dst_addr)
return -ENOMEM;
@@ -401,8 +401,8 @@ static int vmw_bo_cpu_blit_line(struct vmw_bo_blit_line_data *d,
return -EINVAL;
d->src_addr =
- ttm_kmap_atomic_prot(d->src_pages[src_page],
- d->src_prot);
+ kmap_atomic_prot(d->src_pages[src_page],
+ d->src_prot);
if (!d->src_addr)
return -ENOMEM;
@@ -499,9 +499,9 @@ int vmw_bo_cpu_blit(struct ttm_buffer_object *dst,
}
out:
if (d.src_addr)
- ttm_kunmap_atomic_prot(d.src_addr, d.src_prot);
+ kunmap_atomic(d.src_addr);
if (d.dst_addr)
- ttm_kunmap_atomic_prot(d.dst_addr, d.dst_prot);
+ kunmap_atomic(d.dst_addr);
return ret;
}
diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h
index 0a9d042e075a..de1ccdcd5703 100644
--- a/include/drm/ttm/ttm_bo_api.h
+++ b/include/drm/ttm/ttm_bo_api.h
@@ -668,10 +668,6 @@ int ttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo);
int ttm_bo_mmap(struct file *filp, struct vm_area_struct *vma,
struct ttm_bo_device *bdev);
-void *ttm_kmap_atomic_prot(struct page *page, pgprot_t prot);
-
-void ttm_kunmap_atomic_prot(void *addr, pgprot_t prot);
-
/**
* ttm_bo_io
*
--
2.25.1
^ permalink raw reply related
* [PATCH V2 03/11] arch/kmap: Remove redundant arch specific kmaps
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Christoph Hellwig, Ingo Molnar, linux-snps-arc, linux-xtensa,
Borislav Petkov, Andy Lutomirski, Dan Williams, linux-arm-kernel,
Chris Zankel, Thomas Bogendoerfer, linux-parisc, linux-mips,
linuxppc-dev, David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
The kmap code for all the architectures is almost 100% identical.
Lift the common code to the core. Use ARCH_HAS_KMAP_FLUSH_TLB to
indicate if an arch defines kmap_flush_tlb() and call if if needed.
This also has the benefit of changing kmap() on a number of
architectures to be an inline call rather than an actual function.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
arch/arc/include/asm/highmem.h | 2 --
arch/arc/mm/highmem.c | 10 ----------
arch/arm/include/asm/highmem.h | 2 --
arch/arm/mm/highmem.c | 9 ---------
arch/csky/include/asm/highmem.h | 4 ++--
arch/csky/mm/highmem.c | 14 ++++----------
arch/microblaze/include/asm/highmem.h | 9 ---------
arch/mips/include/asm/highmem.h | 4 ++--
arch/mips/mm/highmem.c | 14 +++-----------
arch/nds32/include/asm/highmem.h | 2 --
arch/nds32/mm/highmem.c | 12 ------------
arch/powerpc/include/asm/highmem.h | 9 ---------
arch/sparc/include/asm/highmem.h | 9 ---------
arch/x86/include/asm/highmem.h | 2 --
arch/x86/mm/highmem_32.c | 9 ---------
arch/xtensa/include/asm/highmem.h | 9 ---------
include/linux/highmem.h | 18 ++++++++++++++++++
17 files changed, 29 insertions(+), 109 deletions(-)
diff --git a/arch/arc/include/asm/highmem.h b/arch/arc/include/asm/highmem.h
index 042e92921c4c..96eb67c86961 100644
--- a/arch/arc/include/asm/highmem.h
+++ b/arch/arc/include/asm/highmem.h
@@ -30,8 +30,6 @@
#include <asm/cacheflush.h>
-extern void *kmap(struct page *page);
-extern void *kmap_high(struct page *page);
extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void kunmap_high(struct page *page);
diff --git a/arch/arc/mm/highmem.c b/arch/arc/mm/highmem.c
index 39ef7b9a3aa9..4db13a6b9f3b 100644
--- a/arch/arc/mm/highmem.c
+++ b/arch/arc/mm/highmem.c
@@ -49,16 +49,6 @@
extern pte_t * pkmap_page_table;
static pte_t * fixmap_page_table;
-void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
-
- return kmap_high(page);
-}
-EXPORT_SYMBOL(kmap);
-
void *kmap_atomic(struct page *page)
{
int idx, cpu_idx;
diff --git a/arch/arm/include/asm/highmem.h b/arch/arm/include/asm/highmem.h
index eb4e4207cd3c..c917522541de 100644
--- a/arch/arm/include/asm/highmem.h
+++ b/arch/arm/include/asm/highmem.h
@@ -20,7 +20,6 @@
extern pte_t *pkmap_page_table;
-extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
/*
@@ -63,7 +62,6 @@ static inline void *kmap_high_get(struct page *page)
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
-extern void *kmap(struct page *page);
extern void kunmap(struct page *page);
extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c
index cc6eb79ef20c..e8ba37c36590 100644
--- a/arch/arm/mm/highmem.c
+++ b/arch/arm/mm/highmem.c
@@ -31,15 +31,6 @@ static inline pte_t get_fixmap_pte(unsigned long vaddr)
return *ptep;
}
-void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- return kmap_high(page);
-}
-EXPORT_SYMBOL(kmap);
-
void kunmap(struct page *page)
{
might_sleep();
diff --git a/arch/csky/include/asm/highmem.h b/arch/csky/include/asm/highmem.h
index a345a2f2c22e..9d0516e38110 100644
--- a/arch/csky/include/asm/highmem.h
+++ b/arch/csky/include/asm/highmem.h
@@ -30,10 +30,10 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
-extern void *kmap(struct page *page);
+#define ARCH_HAS_KMAP_FLUSH_TLB
+extern void kmap_flush_tlb(unsigned long addr);
extern void kunmap(struct page *page);
extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
diff --git a/arch/csky/mm/highmem.c b/arch/csky/mm/highmem.c
index 690d678649d1..4a3c273bc8b9 100644
--- a/arch/csky/mm/highmem.c
+++ b/arch/csky/mm/highmem.c
@@ -13,18 +13,12 @@ static pte_t *kmap_pte;
unsigned long highstart_pfn, highend_pfn;
-void *kmap(struct page *page)
+void kmap_flush_tlb(unsigned long addr)
{
- void *addr;
-
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- addr = kmap_high(page);
- flush_tlb_one((unsigned long)addr);
-
- return addr;
+ flush_tlb_one(addr);
}
+EXPORT_SYMBOL(kmap_flush_tlb);
+
EXPORT_SYMBOL(kmap);
void kunmap(struct page *page)
diff --git a/arch/microblaze/include/asm/highmem.h b/arch/microblaze/include/asm/highmem.h
index 99ced7278b5c..8c5bfd228bd8 100644
--- a/arch/microblaze/include/asm/highmem.h
+++ b/arch/microblaze/include/asm/highmem.h
@@ -51,19 +51,10 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt - PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
extern void *kmap_atomic_prot(struct page *page, pgprot_t prot);
extern void __kunmap_atomic(void *kvaddr);
-static inline void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- return kmap_high(page);
-}
-
static inline void kunmap(struct page *page)
{
might_sleep();
diff --git a/arch/mips/include/asm/highmem.h b/arch/mips/include/asm/highmem.h
index 9d84aafc33d0..1f741e3ecabf 100644
--- a/arch/mips/include/asm/highmem.h
+++ b/arch/mips/include/asm/highmem.h
@@ -46,10 +46,10 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void * kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
-extern void *kmap(struct page *page);
+#define ARCH_HAS_KMAP_FLUSH_TLB
+extern void kmap_flush_tlb(unsigned long addr);
extern void kunmap(struct page *page);
extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
diff --git a/arch/mips/mm/highmem.c b/arch/mips/mm/highmem.c
index edd889f6cede..c72058bfead6 100644
--- a/arch/mips/mm/highmem.c
+++ b/arch/mips/mm/highmem.c
@@ -12,19 +12,11 @@ static pte_t *kmap_pte;
unsigned long highstart_pfn, highend_pfn;
-void *kmap(struct page *page)
+void kmap_flush_tlb(unsigned long addr)
{
- void *addr;
-
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- addr = kmap_high(page);
- flush_tlb_one((unsigned long)addr);
-
- return addr;
+ flush_tlb_one(addr);
}
-EXPORT_SYMBOL(kmap);
+EXPORT_SYMBOL(kmap_flush_tlb);
void kunmap(struct page *page)
{
diff --git a/arch/nds32/include/asm/highmem.h b/arch/nds32/include/asm/highmem.h
index b3a82c97ded3..b13654a79069 100644
--- a/arch/nds32/include/asm/highmem.h
+++ b/arch/nds32/include/asm/highmem.h
@@ -44,7 +44,6 @@ extern unsigned long highstart_pfn, highend_pfn;
extern pte_t *pkmap_page_table;
-extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
extern void kmap_init(void);
@@ -54,7 +53,6 @@ extern void kmap_init(void);
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
-extern void *kmap(struct page *page);
extern void kunmap(struct page *page);
extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
diff --git a/arch/nds32/mm/highmem.c b/arch/nds32/mm/highmem.c
index 4c7c28e994ea..d0cde53b84ae 100644
--- a/arch/nds32/mm/highmem.c
+++ b/arch/nds32/mm/highmem.c
@@ -10,18 +10,6 @@
#include <asm/fixmap.h>
#include <asm/tlbflush.h>
-void *kmap(struct page *page)
-{
- unsigned long vaddr;
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- vaddr = (unsigned long)kmap_high(page);
- return (void *)vaddr;
-}
-
-EXPORT_SYMBOL(kmap);
-
void kunmap(struct page *page)
{
might_sleep();
diff --git a/arch/powerpc/include/asm/highmem.h b/arch/powerpc/include/asm/highmem.h
index 529512f6d65a..f14e4feef6d5 100644
--- a/arch/powerpc/include/asm/highmem.h
+++ b/arch/powerpc/include/asm/highmem.h
@@ -59,19 +59,10 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
extern void *kmap_atomic_prot(struct page *page, pgprot_t prot);
extern void __kunmap_atomic(void *kvaddr);
-static inline void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- return kmap_high(page);
-}
-
static inline void kunmap(struct page *page)
{
might_sleep();
diff --git a/arch/sparc/include/asm/highmem.h b/arch/sparc/include/asm/highmem.h
index 7dd2d4b3f980..2ff1192047f7 100644
--- a/arch/sparc/include/asm/highmem.h
+++ b/arch/sparc/include/asm/highmem.h
@@ -50,17 +50,8 @@ void kmap_init(void) __init;
#define PKMAP_END (PKMAP_ADDR(LAST_PKMAP))
-void *kmap_high(struct page *page);
void kunmap_high(struct page *page);
-static inline void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- return kmap_high(page);
-}
-
static inline void kunmap(struct page *page)
{
might_sleep();
diff --git a/arch/x86/include/asm/highmem.h b/arch/x86/include/asm/highmem.h
index a8059930056d..c916a28a9738 100644
--- a/arch/x86/include/asm/highmem.h
+++ b/arch/x86/include/asm/highmem.h
@@ -58,10 +58,8 @@ extern unsigned long highstart_pfn, highend_pfn;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_high(struct page *page);
extern void kunmap_high(struct page *page);
-void *kmap(struct page *page);
void kunmap(struct page *page);
void *kmap_atomic_prot(struct page *page, pgprot_t prot);
diff --git a/arch/x86/mm/highmem_32.c b/arch/x86/mm/highmem_32.c
index 8af66382672b..12591a81b85c 100644
--- a/arch/x86/mm/highmem_32.c
+++ b/arch/x86/mm/highmem_32.c
@@ -4,15 +4,6 @@
#include <linux/swap.h> /* for totalram_pages */
#include <linux/memblock.h>
-void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- return kmap_high(page);
-}
-EXPORT_SYMBOL(kmap);
-
void kunmap(struct page *page)
{
might_sleep();
diff --git a/arch/xtensa/include/asm/highmem.h b/arch/xtensa/include/asm/highmem.h
index a9587c85be85..2546b88ddecf 100644
--- a/arch/xtensa/include/asm/highmem.h
+++ b/arch/xtensa/include/asm/highmem.h
@@ -63,17 +63,8 @@ static inline wait_queue_head_t *get_pkmap_wait_queue_head(unsigned int color)
extern pte_t *pkmap_page_table;
-void *kmap_high(struct page *page);
void kunmap_high(struct page *page);
-static inline void *kmap(struct page *page)
-{
- might_sleep();
- if (!PageHighMem(page))
- return page_address(page);
- return kmap_high(page);
-}
-
static inline void kunmap(struct page *page)
{
might_sleep();
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index ea5cdbd8c2c3..fc3adc51254a 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -34,6 +34,24 @@ static inline void invalidate_kernel_vmap_range(void *vaddr, int size)
#ifdef CONFIG_HIGHMEM
#include <asm/highmem.h>
+#ifndef ARCH_HAS_KMAP_FLUSH_TLB
+static inline void kmap_flush_tlb(unsigned long addr) { }
+#endif
+
+void *kmap_high(struct page *page);
+static inline void *kmap(struct page *page)
+{
+ void *addr;
+
+ might_sleep();
+ if (!PageHighMem(page))
+ addr = page_address(page);
+ else
+ addr = kmap_high(page);
+ kmap_flush_tlb((unsigned long)addr);
+ return addr;
+}
+
/* declarations for linux/mm/highmem.c */
unsigned int nr_free_highpages(void);
extern atomic_long_t _totalhigh_pages;
--
2.25.1
^ permalink raw reply related
* [PATCH V2 10/11] arch/kmap: Define kmap_atomic_prot() for all arch's
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Ingo Molnar, linux-snps-arc, linux-xtensa, Borislav Petkov,
Andy Lutomirski, Dan Williams, linux-arm-kernel, Chris Zankel,
Thomas Bogendoerfer, linux-parisc, linux-mips, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
To support kmap_atomic_prot(), all architectures need to support
protections passed to their kmap_atomic_high() function. Pass
protections into kmap_atomic_high() and change the name to
kmap_atomic_high_prot() to match.
Then define kmap_atomic_prot() as a core function which calls
kmap_atomic_high_prot() when needed.
Finally, redefine kmap_atomic() as a wrapper of kmap_atomic_prot() with
the default kmap_prot exported by the architectures.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes from V1:
Adjust for bisect-ability
Adjust for removing kunmap_atomic_high
Remove kmap_atomic_high_prot declarations
---
arch/arc/mm/highmem.c | 6 +++---
arch/arm/mm/highmem.c | 6 +++---
arch/csky/mm/highmem.c | 6 +++---
arch/microblaze/include/asm/highmem.h | 16 ----------------
arch/mips/mm/highmem.c | 6 +++---
arch/nds32/mm/highmem.c | 6 +++---
arch/powerpc/include/asm/highmem.h | 17 -----------------
arch/sparc/mm/highmem.c | 6 +++---
arch/x86/include/asm/highmem.h | 14 --------------
arch/xtensa/mm/highmem.c | 6 +++---
include/linux/highmem.h | 7 ++++---
11 files changed, 25 insertions(+), 71 deletions(-)
diff --git a/arch/arc/mm/highmem.c b/arch/arc/mm/highmem.c
index 5d3eab4ac0b0..479b0d72d3cf 100644
--- a/arch/arc/mm/highmem.c
+++ b/arch/arc/mm/highmem.c
@@ -49,7 +49,7 @@
extern pte_t * pkmap_page_table;
static pte_t * fixmap_page_table;
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
int idx, cpu_idx;
unsigned long vaddr;
@@ -59,11 +59,11 @@ void *kmap_atomic_high(struct page *page)
vaddr = FIXMAP_ADDR(idx);
set_pte_at(&init_mm, vaddr, fixmap_page_table + idx,
- mk_pte(page, kmap_prot));
+ mk_pte(page, prot));
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kv)
{
diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c
index ac8394655a6e..e013f6b81328 100644
--- a/arch/arm/mm/highmem.c
+++ b/arch/arm/mm/highmem.c
@@ -31,7 +31,7 @@ static inline pte_t get_fixmap_pte(unsigned long vaddr)
return *ptep;
}
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned int idx;
unsigned long vaddr;
@@ -67,11 +67,11 @@ void *kmap_atomic_high(struct page *page)
* in place, so the contained TLB flush ensures the TLB is updated
* with the new mapping.
*/
- set_fixmap_pte(idx, mk_pte(page, kmap_prot));
+ set_fixmap_pte(idx, mk_pte(page, prot));
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
diff --git a/arch/csky/mm/highmem.c b/arch/csky/mm/highmem.c
index f4311669b5bb..3ae5c8cd7619 100644
--- a/arch/csky/mm/highmem.c
+++ b/arch/csky/mm/highmem.c
@@ -21,7 +21,7 @@ EXPORT_SYMBOL(kmap_flush_tlb);
EXPORT_SYMBOL(kmap);
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
@@ -32,12 +32,12 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte - idx)));
#endif
- set_pte(kmap_pte-idx, mk_pte(page, kmap_prot));
+ set_pte(kmap_pte-idx, mk_pte(page, prot));
flush_tlb_one((unsigned long)vaddr);
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
diff --git a/arch/microblaze/include/asm/highmem.h b/arch/microblaze/include/asm/highmem.h
index 033ac5b5c2da..d7c55cfd27bd 100644
--- a/arch/microblaze/include/asm/highmem.h
+++ b/arch/microblaze/include/asm/highmem.h
@@ -51,22 +51,6 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt - PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
-void *kmap_atomic_prot(struct page *page, pgprot_t prot)
-{
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
- return kmap_atomic_high_prot(page, prot);
-}
-
-static inline void *kmap_atomic_high(struct page *page)
-{
- return kmap_atomic_high_prot(page, kmap_prot);
-}
-
#define flush_cache_kmaps() { flush_icache(); flush_dcache(); }
#endif /* __KERNEL__ */
diff --git a/arch/mips/mm/highmem.c b/arch/mips/mm/highmem.c
index 87023bd1a33c..37e244cdb14e 100644
--- a/arch/mips/mm/highmem.c
+++ b/arch/mips/mm/highmem.c
@@ -18,7 +18,7 @@ void kmap_flush_tlb(unsigned long addr)
}
EXPORT_SYMBOL(kmap_flush_tlb);
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
int idx, type;
@@ -29,12 +29,12 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte - idx)));
#endif
- set_pte(kmap_pte-idx, mk_pte(page, kmap_prot));
+ set_pte(kmap_pte-idx, mk_pte(page, prot));
local_flush_tlb_one((unsigned long)vaddr);
return (void*) vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
diff --git a/arch/nds32/mm/highmem.c b/arch/nds32/mm/highmem.c
index 809f8c830f06..63ded527c1e8 100644
--- a/arch/nds32/mm/highmem.c
+++ b/arch/nds32/mm/highmem.c
@@ -10,7 +10,7 @@
#include <asm/fixmap.h>
#include <asm/tlbflush.h>
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned int idx;
unsigned long vaddr, pte;
@@ -21,7 +21,7 @@ void *kmap_atomic_high(struct page *page)
idx = type + KM_TYPE_NR * smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
- pte = (page_to_pfn(page) << PAGE_SHIFT) | (kmap_prot);
+ pte = (page_to_pfn(page) << PAGE_SHIFT) | prot;
ptep = pte_offset_kernel(pmd_off_k(vaddr), vaddr);
set_pte(ptep, pte);
@@ -32,7 +32,7 @@ void *kmap_atomic_high(struct page *page)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
diff --git a/arch/powerpc/include/asm/highmem.h b/arch/powerpc/include/asm/highmem.h
index ee5de974c5ef..8d8ee3fcd800 100644
--- a/arch/powerpc/include/asm/highmem.h
+++ b/arch/powerpc/include/asm/highmem.h
@@ -59,23 +59,6 @@ extern pte_t *pkmap_page_table;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
-static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
-{
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
- return kmap_atomic_high_prot(page, prot);
-}
-
-static inline void *kmap_atomic_high(struct page *page)
-{
- return kmap_atomic_high_prot(page, kmap_prot);
-}
-
-
#define flush_cache_kmaps() flush_cache_all()
#endif /* __KERNEL__ */
diff --git a/arch/sparc/mm/highmem.c b/arch/sparc/mm/highmem.c
index 9f06d75e88e1..414f578d1e57 100644
--- a/arch/sparc/mm/highmem.c
+++ b/arch/sparc/mm/highmem.c
@@ -54,7 +54,7 @@ void __init kmap_init(void)
kmap_prot = __pgprot(SRMMU_ET_PTE | SRMMU_PRIV | SRMMU_CACHE);
}
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
long idx, type;
@@ -73,7 +73,7 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte-idx)));
#endif
- set_pte(kmap_pte-idx, mk_pte(page, kmap_prot));
+ set_pte(kmap_pte-idx, mk_pte(page, prot));
/* XXX Fix - Anton */
#if 0
__flush_tlb_one(vaddr);
@@ -83,7 +83,7 @@ void *kmap_atomic_high(struct page *page)
return (void*) vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
diff --git a/arch/x86/include/asm/highmem.h b/arch/x86/include/asm/highmem.h
index be66b77885a0..0f420b24e0fc 100644
--- a/arch/x86/include/asm/highmem.h
+++ b/arch/x86/include/asm/highmem.h
@@ -58,20 +58,6 @@ extern unsigned long highstart_pfn, highend_pfn;
#define PKMAP_NR(virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)
#define PKMAP_ADDR(nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
-extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
-static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
-{
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
- return kmap_atomic_high_prot(page, prot);
-}
-static inline void *kmap_atomic_high(struct page *page)
-{
- return kmap_atomic_high_prot(page, kmap_prot);
-}
void *kmap_atomic_pfn(unsigned long pfn);
void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot);
diff --git a/arch/xtensa/mm/highmem.c b/arch/xtensa/mm/highmem.c
index 50168b09510a..99b5ad137ab5 100644
--- a/arch/xtensa/mm/highmem.c
+++ b/arch/xtensa/mm/highmem.c
@@ -37,7 +37,7 @@ static inline enum fixed_addresses kmap_idx(int type, unsigned long color)
color;
}
-void *kmap_atomic_high(struct page *page)
+void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
enum fixed_addresses idx;
unsigned long vaddr;
@@ -48,11 +48,11 @@ void *kmap_atomic_high(struct page *page)
#ifdef CONFIG_DEBUG_HIGHMEM
BUG_ON(!pte_none(*(kmap_pte + idx)));
#endif
- set_pte(kmap_pte + idx, mk_pte(page, kmap_prot));
+ set_pte(kmap_pte + idx, mk_pte(page, prot));
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic_high);
+EXPORT_SYMBOL(kmap_atomic_high_prot);
void kunmap_atomic_high(void *kvaddr)
{
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index c36c0ee09423..89838306f50d 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -32,7 +32,7 @@ static inline void invalidate_kernel_vmap_range(void *vaddr, int size)
#include <asm/kmap_types.h>
#ifdef CONFIG_HIGHMEM
-extern void *kmap_atomic_high(struct page *page);
+extern void *kmap_atomic_high_prot(struct page *page, pgprot_t prot);
extern void kunmap_atomic_high(void *kvaddr);
#include <asm/highmem.h>
@@ -76,14 +76,15 @@ static inline void kunmap(struct page *page)
* be used in IRQ contexts, so in some (very limited) cases we need
* it.
*/
-static inline void *kmap_atomic(struct page *page)
+static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
{
preempt_disable();
pagefault_disable();
if (!PageHighMem(page))
return page_address(page);
- return kmap_atomic_high(page);
+ return kmap_atomic_high_prot(page, prot);
}
+#define kmap_atomic(page) kmap_atomic_prot(page, kmap_prot)
/* declarations for linux/mm/highmem.c */
unsigned int nr_free_highpages(void);
--
2.25.1
^ permalink raw reply related
* [PATCH V2 06/11] arch/kmap_atomic: Consolidate duplicate code
From: ira.weiny @ 2020-05-04 1:09 UTC (permalink / raw)
To: linux-kernel, Andrew Morton, Christian Koenig, Huang Rui
Cc: Peter Zijlstra, Dave Hansen, dri-devel, James E.J. Bottomley,
Max Filippov, Paul Mackerras, H. Peter Anvin, sparclinux,
Ira Weiny, Thomas Gleixner, Helge Deller, x86, linux-csky,
Ingo Molnar, linux-snps-arc, linux-xtensa, Borislav Petkov,
Andy Lutomirski, Dan Williams, linux-arm-kernel, Chris Zankel,
Thomas Bogendoerfer, linux-parisc, linux-mips, linuxppc-dev,
David S. Miller
In-Reply-To: <20200504010912.982044-1-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
Every arch has the same code to ensure atomic operations and a check for
!HIGHMEM page.
Remove the duplicate code by defining a core kmap_atomic() which only
calls the arch specific kmap_atomic_high() when the page is high memory.
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes from V1:
Adjust to preserve bisect-ability
Remove unneeded kmap_atomic_high declarations
---
arch/arc/include/asm/highmem.h | 1 -
arch/arc/mm/highmem.c | 9 ++-------
arch/arm/include/asm/highmem.h | 1 -
arch/arm/mm/highmem.c | 9 ++-------
arch/csky/include/asm/highmem.h | 1 -
arch/csky/mm/highmem.c | 9 ++-------
arch/microblaze/include/asm/highmem.h | 4 ++--
arch/mips/include/asm/highmem.h | 1 -
arch/mips/mm/cache.c | 2 +-
arch/mips/mm/highmem.c | 18 ++----------------
arch/nds32/include/asm/highmem.h | 1 -
arch/nds32/mm/highmem.c | 9 ++-------
arch/powerpc/include/asm/highmem.h | 4 ++--
arch/powerpc/mm/highmem.c | 6 ------
arch/sparc/include/asm/highmem.h | 1 -
arch/sparc/mm/highmem.c | 9 ++-------
arch/x86/include/asm/highmem.h | 5 ++++-
arch/x86/mm/highmem_32.c | 14 --------------
arch/xtensa/include/asm/highmem.h | 1 -
arch/xtensa/mm/highmem.c | 9 ++-------
include/linux/highmem.h | 23 +++++++++++++++++++++++
21 files changed, 46 insertions(+), 91 deletions(-)
diff --git a/arch/arc/include/asm/highmem.h b/arch/arc/include/asm/highmem.h
index 8387a5596a91..db425cd38545 100644
--- a/arch/arc/include/asm/highmem.h
+++ b/arch/arc/include/asm/highmem.h
@@ -30,7 +30,6 @@
#include <asm/cacheflush.h>
-extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void kmap_init(void);
diff --git a/arch/arc/mm/highmem.c b/arch/arc/mm/highmem.c
index 4db13a6b9f3b..0964b011c29f 100644
--- a/arch/arc/mm/highmem.c
+++ b/arch/arc/mm/highmem.c
@@ -49,16 +49,11 @@
extern pte_t * pkmap_page_table;
static pte_t * fixmap_page_table;
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
int idx, cpu_idx;
unsigned long vaddr;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
cpu_idx = kmap_atomic_idx_push();
idx = cpu_idx + KM_TYPE_NR * smp_processor_id();
vaddr = FIXMAP_ADDR(idx);
@@ -68,7 +63,7 @@ void *kmap_atomic(struct page *page)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kv)
{
diff --git a/arch/arm/include/asm/highmem.h b/arch/arm/include/asm/highmem.h
index 736f65283e7b..8c80bfe18a34 100644
--- a/arch/arm/include/asm/highmem.h
+++ b/arch/arm/include/asm/highmem.h
@@ -60,7 +60,6 @@ static inline void *kmap_high_get(struct page *page)
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
-extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
#endif
diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c
index c700b32350ee..075fdc235091 100644
--- a/arch/arm/mm/highmem.c
+++ b/arch/arm/mm/highmem.c
@@ -31,18 +31,13 @@ static inline pte_t get_fixmap_pte(unsigned long vaddr)
return *ptep;
}
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
unsigned int idx;
unsigned long vaddr;
void *kmap;
int type;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
#ifdef CONFIG_DEBUG_HIGHMEM
/*
* There is no cache coherency issue when non VIVT, so force the
@@ -76,7 +71,7 @@ void *kmap_atomic(struct page *page)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/csky/include/asm/highmem.h b/arch/csky/include/asm/highmem.h
index be11c5b67122..8ceee12f9bc1 100644
--- a/arch/csky/include/asm/highmem.h
+++ b/arch/csky/include/asm/highmem.h
@@ -32,7 +32,6 @@ extern pte_t *pkmap_page_table;
#define ARCH_HAS_KMAP_FLUSH_TLB
extern void kmap_flush_tlb(unsigned long addr);
-extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
extern struct page *kmap_atomic_to_page(void *ptr);
diff --git a/arch/csky/mm/highmem.c b/arch/csky/mm/highmem.c
index e9952211264b..63d74b47eee6 100644
--- a/arch/csky/mm/highmem.c
+++ b/arch/csky/mm/highmem.c
@@ -21,16 +21,11 @@ EXPORT_SYMBOL(kmap_flush_tlb);
EXPORT_SYMBOL(kmap);
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
unsigned long vaddr;
int idx, type;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -42,7 +37,7 @@ void *kmap_atomic(struct page *page)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/microblaze/include/asm/highmem.h b/arch/microblaze/include/asm/highmem.h
index ec9954b091e1..00c75a423ac4 100644
--- a/arch/microblaze/include/asm/highmem.h
+++ b/arch/microblaze/include/asm/highmem.h
@@ -63,9 +63,9 @@ void *kmap_atomic_prot(struct page *page, pgprot_t prot)
}
extern void __kunmap_atomic(void *kvaddr);
-static inline void *kmap_atomic(struct page *page)
+static inline void *kmap_atomic_high(struct page *page)
{
- return kmap_atomic_prot(page, kmap_prot);
+ return kmap_atomic_high_prot(page, kmap_prot);
}
#define flush_cache_kmaps() { flush_icache(); flush_dcache(); }
diff --git a/arch/mips/include/asm/highmem.h b/arch/mips/include/asm/highmem.h
index 24e7e7e5cc7b..8bdbbfc322ad 100644
--- a/arch/mips/include/asm/highmem.h
+++ b/arch/mips/include/asm/highmem.h
@@ -48,7 +48,6 @@ extern pte_t *pkmap_page_table;
#define ARCH_HAS_KMAP_FLUSH_TLB
extern void kmap_flush_tlb(unsigned long addr);
-extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
diff --git a/arch/mips/mm/cache.c b/arch/mips/mm/cache.c
index 33b409391ddb..f015bb51fab0 100644
--- a/arch/mips/mm/cache.c
+++ b/arch/mips/mm/cache.c
@@ -14,9 +14,9 @@
#include <linux/sched.h>
#include <linux/syscalls.h>
#include <linux/mm.h>
+#include <linux/highmem.h>
#include <asm/cacheflush.h>
-#include <asm/highmem.h>
#include <asm/processor.h>
#include <asm/cpu.h>
#include <asm/cpu-features.h>
diff --git a/arch/mips/mm/highmem.c b/arch/mips/mm/highmem.c
index eb8ec8493f2f..2bda56372995 100644
--- a/arch/mips/mm/highmem.c
+++ b/arch/mips/mm/highmem.c
@@ -18,25 +18,11 @@ void kmap_flush_tlb(unsigned long addr)
}
EXPORT_SYMBOL(kmap_flush_tlb);
-/*
- * kmap_atomic/kunmap_atomic is significantly faster than kmap/kunmap because
- * no global lock is needed and because the kmap code must perform a global TLB
- * invalidation when the kmap pool wraps.
- *
- * However when holding an atomic kmap is is not legal to sleep, so atomic
- * kmaps are appropriate for short, tight code paths only.
- */
-
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
unsigned long vaddr;
int idx, type;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -48,7 +34,7 @@ void *kmap_atomic(struct page *page)
return (void*) vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/nds32/include/asm/highmem.h b/arch/nds32/include/asm/highmem.h
index c93c7368bb3f..a3970e566ede 100644
--- a/arch/nds32/include/asm/highmem.h
+++ b/arch/nds32/include/asm/highmem.h
@@ -51,7 +51,6 @@ extern void kmap_init(void);
* when CONFIG_HIGHMEM is not set.
*/
#ifdef CONFIG_HIGHMEM
-extern void *kmap_atomic(struct page *page);
extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
extern struct page *kmap_atomic_to_page(void *ptr);
diff --git a/arch/nds32/mm/highmem.c b/arch/nds32/mm/highmem.c
index f9348bec0ecb..f5f3a21460c4 100644
--- a/arch/nds32/mm/highmem.c
+++ b/arch/nds32/mm/highmem.c
@@ -10,18 +10,13 @@
#include <asm/fixmap.h>
#include <asm/tlbflush.h>
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
unsigned int idx;
unsigned long vaddr, pte;
int type;
pte_t *ptep;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR * smp_processor_id();
@@ -37,7 +32,7 @@ void *kmap_atomic(struct page *page)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/powerpc/include/asm/highmem.h b/arch/powerpc/include/asm/highmem.h
index d049806a8354..74fa2c726fde 100644
--- a/arch/powerpc/include/asm/highmem.h
+++ b/arch/powerpc/include/asm/highmem.h
@@ -71,9 +71,9 @@ static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
}
extern void __kunmap_atomic(void *kvaddr);
-static inline void *kmap_atomic(struct page *page)
+static inline void *kmap_atomic_high(struct page *page)
{
- return kmap_atomic_prot(page, kmap_prot);
+ return kmap_atomic_high_prot(page, kmap_prot);
}
diff --git a/arch/powerpc/mm/highmem.c b/arch/powerpc/mm/highmem.c
index f075cef6d663..67aaa5217f7f 100644
--- a/arch/powerpc/mm/highmem.c
+++ b/arch/powerpc/mm/highmem.c
@@ -24,12 +24,6 @@
#include <linux/highmem.h>
#include <linux/module.h>
-/*
- * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
- * gives a more generic (and caching) interface. But kmap_atomic can
- * be used in IRQ contexts, so in some (very limited) cases we need
- * it.
- */
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
diff --git a/arch/sparc/include/asm/highmem.h b/arch/sparc/include/asm/highmem.h
index 4bdb79fed02c..458210c5bc38 100644
--- a/arch/sparc/include/asm/highmem.h
+++ b/arch/sparc/include/asm/highmem.h
@@ -50,7 +50,6 @@ void kmap_init(void) __init;
#define PKMAP_END (PKMAP_ADDR(LAST_PKMAP))
-void *kmap_atomic(struct page *page);
void __kunmap_atomic(void *kvaddr);
#define flush_cache_kmaps() flush_cache_all()
diff --git a/arch/sparc/mm/highmem.c b/arch/sparc/mm/highmem.c
index d4a80adea7e5..b53070ab6a31 100644
--- a/arch/sparc/mm/highmem.c
+++ b/arch/sparc/mm/highmem.c
@@ -53,16 +53,11 @@ void __init kmap_init(void)
kmap_prot = __pgprot(SRMMU_ET_PTE | SRMMU_PRIV | SRMMU_CACHE);
}
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
unsigned long vaddr;
long idx, type;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
type = kmap_atomic_idx_push();
idx = type + KM_TYPE_NR*smp_processor_id();
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -87,7 +82,7 @@ void *kmap_atomic(struct page *page)
return (void*) vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/arch/x86/include/asm/highmem.h b/arch/x86/include/asm/highmem.h
index 61f47fef40e5..9393d55a2adb 100644
--- a/arch/x86/include/asm/highmem.h
+++ b/arch/x86/include/asm/highmem.h
@@ -68,7 +68,10 @@ static inline void *kmap_atomic_prot(struct page *page, pgprot_t prot)
return kmap_atomic_high_prot(page, prot);
}
-void *kmap_atomic(struct page *page);
+static inline void *kmap_atomic_high(struct page *page)
+{
+ return kmap_atomic_high_prot(page, kmap_prot);
+}
void __kunmap_atomic(void *kvaddr);
void *kmap_atomic_pfn(unsigned long pfn);
void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot);
diff --git a/arch/x86/mm/highmem_32.c b/arch/x86/mm/highmem_32.c
index 48b56b1af902..c3e272a759e0 100644
--- a/arch/x86/mm/highmem_32.c
+++ b/arch/x86/mm/highmem_32.c
@@ -4,14 +4,6 @@
#include <linux/swap.h> /* for totalram_pages */
#include <linux/memblock.h>
-/*
- * kmap_atomic/kunmap_atomic is significantly faster than kmap/kunmap because
- * no global lock is needed and because the kmap code must perform a global TLB
- * invalidation when the kmap pool wraps.
- *
- * However when holding an atomic kmap it is not legal to sleep, so atomic
- * kmaps are appropriate for short, tight code paths only.
- */
void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
{
unsigned long vaddr;
@@ -28,12 +20,6 @@ void *kmap_atomic_high_prot(struct page *page, pgprot_t prot)
}
EXPORT_SYMBOL(kmap_atomic_high_prot);
-void *kmap_atomic(struct page *page)
-{
- return kmap_atomic_prot(page, kmap_prot);
-}
-EXPORT_SYMBOL(kmap_atomic);
-
/*
* This is the same as kmap_atomic() but can map memory that doesn't
* have a struct page associated with it.
diff --git a/arch/xtensa/include/asm/highmem.h b/arch/xtensa/include/asm/highmem.h
index 5a481f7def0b..1e6aa15c4bdf 100644
--- a/arch/xtensa/include/asm/highmem.h
+++ b/arch/xtensa/include/asm/highmem.h
@@ -68,7 +68,6 @@ static inline void flush_cache_kmaps(void)
flush_cache_all();
}
-void *kmap_atomic(struct page *page);
void __kunmap_atomic(void *kvaddr);
void kmap_init(void);
diff --git a/arch/xtensa/mm/highmem.c b/arch/xtensa/mm/highmem.c
index da734a2ed641..90b85a897cb0 100644
--- a/arch/xtensa/mm/highmem.c
+++ b/arch/xtensa/mm/highmem.c
@@ -37,16 +37,11 @@ static inline enum fixed_addresses kmap_idx(int type, unsigned long color)
color;
}
-void *kmap_atomic(struct page *page)
+void *kmap_atomic_high(struct page *page)
{
enum fixed_addresses idx;
unsigned long vaddr;
- preempt_disable();
- pagefault_disable();
- if (!PageHighMem(page))
- return page_address(page);
-
idx = kmap_idx(kmap_atomic_idx_push(),
DCACHE_ALIAS(page_to_phys(page)));
vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
@@ -57,7 +52,7 @@ void *kmap_atomic(struct page *page)
return (void *)vaddr;
}
-EXPORT_SYMBOL(kmap_atomic);
+EXPORT_SYMBOL(kmap_atomic_high);
void __kunmap_atomic(void *kvaddr)
{
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index ae6e8cb81043..86b93dee758a 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -32,6 +32,7 @@ static inline void invalidate_kernel_vmap_range(void *vaddr, int size)
#include <asm/kmap_types.h>
#ifdef CONFIG_HIGHMEM
+extern void *kmap_atomic_high(struct page *page);
#include <asm/highmem.h>
#ifndef ARCH_HAS_KMAP_FLUSH_TLB
@@ -61,6 +62,28 @@ static inline void kunmap(struct page *page)
kunmap_high(page);
}
+/*
+ * kmap_atomic/kunmap_atomic is significantly faster than kmap/kunmap because
+ * no global lock is needed and because the kmap code must perform a global TLB
+ * invalidation when the kmap pool wraps.
+ *
+ * However when holding an atomic kmap is is not legal to sleep, so atomic
+ * kmaps are appropriate for short, tight code paths only.
+ *
+ * The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap
+ * gives a more generic (and caching) interface. But kmap_atomic can
+ * be used in IRQ contexts, so in some (very limited) cases we need
+ * it.
+ */
+static inline void *kmap_atomic(struct page *page)
+{
+ preempt_disable();
+ pagefault_disable();
+ if (!PageHighMem(page))
+ return page_address(page);
+ return kmap_atomic_high(page);
+}
+
/* declarations for linux/mm/highmem.c */
unsigned int nr_free_highpages(void);
extern atomic_long_t _totalhigh_pages;
--
2.25.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox