* Re: switch the remaining architectures to use generic GUP v3
From: Jason Gunthorpe @ 2019-06-21 14:43 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-1-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:46PM +0200, Christoph Hellwig wrote:
> Hi Linus and maintainers,
>
> below is a series to switch mips, sh and sparc64 to use the generic
> GUP code so that we only have one codebase to touch for further
> improvements to this code. I don't have hardware for any of these
> architectures, and generally no clue about their page table
> management, so handle with care.
I like this series, ther are alot of people talking about work for GUP
and this will make any of that so much easier to do.
Jason
^ permalink raw reply
* Re: [PATCH 6/7] kprobes/ftrace: Use ftrace_location() when [dis]arming probes
From: Masami Hiramatsu @ 2019-06-21 14:41 UTC (permalink / raw)
To: Naveen N. Rao
Cc: linux-kernel, Nicholas Piggin, Steven Rostedt, linuxppc-dev,
Ingo Molnar
In-Reply-To: <4fedad69107b7fd81b9324315ce4fbf6287e5084.1560868106.git.naveen.n.rao@linux.vnet.ibm.com>
On Tue, 18 Jun 2019 20:17:05 +0530
"Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
> Ftrace location could include more than a single instruction in case of
> some architectures (powerpc64, for now). In this case, kprobe is
> permitted on any of those instructions, and uses ftrace infrastructure
> for functioning.
>
> However, [dis]arm_kprobe_ftrace() uses the kprobe address when setting
> up ftrace filter IP. This won't work if the address points to any
> instruction apart from the one that has a branch to _mcount(). To
> resolve this, have [dis]arm_kprobe_ftrace() use ftrace_function() to
> identify the filter IP.
This looks good to me.
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Thank you!
>
> Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> ---
> kernel/kprobes.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> index 445337c107e0..282ee704e2d8 100644
> --- a/kernel/kprobes.c
> +++ b/kernel/kprobes.c
> @@ -978,10 +978,10 @@ static int prepare_kprobe(struct kprobe *p)
> /* Caller must lock kprobe_mutex */
> static int arm_kprobe_ftrace(struct kprobe *p)
> {
> + unsigned long ftrace_ip = ftrace_location((unsigned long)p->addr);
> int ret = 0;
>
> - ret = ftrace_set_filter_ip(&kprobe_ftrace_ops,
> - (unsigned long)p->addr, 0, 0);
> + ret = ftrace_set_filter_ip(&kprobe_ftrace_ops, ftrace_ip, 0, 0);
> if (ret) {
> pr_debug("Failed to arm kprobe-ftrace at %pS (%d)\n",
> p->addr, ret);
> @@ -1005,13 +1005,14 @@ static int arm_kprobe_ftrace(struct kprobe *p)
> * non-empty filter_hash for IPMODIFY ops, we're safe from an accidental
> * empty filter_hash which would undesirably trace all functions.
> */
> - ftrace_set_filter_ip(&kprobe_ftrace_ops, (unsigned long)p->addr, 1, 0);
> + ftrace_set_filter_ip(&kprobe_ftrace_ops, ftrace_ip, 1, 0);
> return ret;
> }
>
> /* Caller must lock kprobe_mutex */
> static int disarm_kprobe_ftrace(struct kprobe *p)
> {
> + unsigned long ftrace_ip = ftrace_location((unsigned long)p->addr);
> int ret = 0;
>
> if (kprobe_ftrace_enabled == 1) {
> @@ -1022,8 +1023,7 @@ static int disarm_kprobe_ftrace(struct kprobe *p)
>
> kprobe_ftrace_enabled--;
>
> - ret = ftrace_set_filter_ip(&kprobe_ftrace_ops,
> - (unsigned long)p->addr, 1, 0);
> + ret = ftrace_set_filter_ip(&kprobe_ftrace_ops, ftrace_ip, 1, 0);
> WARN_ONCE(ret < 0, "Failed to disarm kprobe-ftrace at %pS (%d)\n",
> p->addr, ret);
> return ret;
> --
> 2.22.0
>
--
Masami Hiramatsu <mhiramat@kernel.org>
^ permalink raw reply
* Re: [PATCH 11/16] mm: consolidate the get_user_pages* implementations
From: Jason Gunthorpe @ 2019-06-21 14:41 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-12-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:57PM +0200, Christoph Hellwig wrote:
> @@ -2168,7 +2221,7 @@ static void gup_pgd_range(unsigned long addr, unsigned long end,
> */
> static bool gup_fast_permitted(unsigned long start, unsigned long end)
> {
> - return true;
> + return IS_ENABLED(CONFIG_HAVE_FAST_GUP) ? true : false;
The ?: is needed with IS_ENABLED?
> }
> #endif
Oh, you fixed the util.c this way instead of the headerfile
#ifdef..
I'd suggest to revise this block a tiny bit:
-#ifndef gup_fast_permitted
+#if !IS_ENABLED(CONFIG_HAVE_FAST_GUP) || !defined(gup_fast_permitted)
/*
* Check if it's allowed to use __get_user_pages_fast() for the range, or
* we need to fall back to the slow version:
*/
-bool gup_fast_permitted(unsigned long start, int nr_pages)
+static bool gup_fast_permitted(unsigned long start, int nr_pages)
{
Just in case some future arch code mismatches the header and kconfig..
Regards,
Jason
^ permalink raw reply
* Re: [PATCH 10/16] mm: rename CONFIG_HAVE_GENERIC_GUP to CONFIG_HAVE_FAST_GUP
From: Jason Gunthorpe @ 2019-06-21 14:28 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-11-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:56PM +0200, Christoph Hellwig wrote:
> We only support the generic GUP now, so rename the config option to
> be more clear, and always use the mm/Kconfig definition of the
> symbol and select it from the arch Kconfigs.
Looks OK to me
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
But could you also roll something like this in to the series? There is
no longer any reason for the special __weak stuff that I can see -
just follow the normal pattern for stubbing config controlled
functions through the header file.
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 0e8834ac32b76c..13b1cb573383d5 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1561,8 +1561,17 @@ long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
struct page **pages, unsigned int gup_flags);
+#ifdef CONFIG_HAVE_FAST_GUP
int get_user_pages_fast(unsigned long start, int nr_pages,
unsigned int gup_flags, struct page **pages);
+#else
+static inline int get_user_pages_fast(unsigned long start, int nr_pages,
+ unsigned int gup_flags,
+ struct page **pages)
+{
+ return get_user_pages_unlocked(start, nr_pages, pages, gup_flags);
+}
+#endif
/* Container for pinned pfns / pages */
struct frame_vector {
@@ -1668,8 +1677,17 @@ extern int mprotect_fixup(struct vm_area_struct *vma,
/*
* doesn't attempt to fault and will return short.
*/
+#ifdef CONFIG_HAVE_FAST_GUP
int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages);
+#else
+static inline int __get_user_pages_fast(unsigned long start, int nr_pages,
+ int write, struct page **pages)
+{
+ return 0;
+}
+#endif
+
/*
* per-process(per-mm_struct) statistics.
*/
diff --git a/mm/util.c b/mm/util.c
index 9834c4ab7d8e86..68575a315dc5ad 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -300,53 +300,6 @@ void arch_pick_mmap_layout(struct mm_struct *mm, struct rlimit *rlim_stack)
}
#endif
-/*
- * Like get_user_pages_fast() except its IRQ-safe in that it won't fall
- * back to the regular GUP.
- * Note a difference with get_user_pages_fast: this always returns the
- * number of pages pinned, 0 if no pages were pinned.
- * If the architecture does not support this function, simply return with no
- * pages pinned.
- */
-int __weak __get_user_pages_fast(unsigned long start,
- int nr_pages, int write, struct page **pages)
-{
- return 0;
-}
-EXPORT_SYMBOL_GPL(__get_user_pages_fast);
-
-/**
- * get_user_pages_fast() - pin user pages in memory
- * @start: starting user address
- * @nr_pages: number of pages from start to pin
- * @gup_flags: flags modifying pin behaviour
- * @pages: array that receives pointers to the pages pinned.
- * Should be at least nr_pages long.
- *
- * get_user_pages_fast provides equivalent functionality to get_user_pages,
- * operating on current and current->mm, with force=0 and vma=NULL. However
- * unlike get_user_pages, it must be called without mmap_sem held.
- *
- * get_user_pages_fast may take mmap_sem and page table locks, so no
- * assumptions can be made about lack of locking. get_user_pages_fast is to be
- * implemented in a way that is advantageous (vs get_user_pages()) when the
- * user memory area is already faulted in and present in ptes. However if the
- * pages have to be faulted in, it may turn out to be slightly slower so
- * callers need to carefully consider what to use. On many architectures,
- * get_user_pages_fast simply falls back to get_user_pages.
- *
- * Return: number of pages pinned. This may be fewer than the number
- * requested. If nr_pages is 0 or negative, returns 0. If no pages
- * were pinned, returns -errno.
- */
-int __weak get_user_pages_fast(unsigned long start,
- int nr_pages, unsigned int gup_flags,
- struct page **pages)
-{
- return get_user_pages_unlocked(start, nr_pages, pages, gup_flags);
-}
-EXPORT_SYMBOL_GPL(get_user_pages_fast);
-
unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
unsigned long flag, unsigned long pgoff)
^ permalink raw reply related
* Re: [PATCH 04/16] MIPS: use the generic get_user_pages_fast code
From: Jason Gunthorpe @ 2019-06-21 14:05 UTC (permalink / raw)
To: Christoph Hellwig, Kamal Dasu, Ralf Baechle
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-5-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:50PM +0200, Christoph Hellwig wrote:
> diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
> index 4ccb465ef3f2..7d27194e3b45 100644
> +++ b/arch/mips/include/asm/pgtable.h
> @@ -20,6 +20,7 @@
> #include <asm/cmpxchg.h>
> #include <asm/io.h>
> #include <asm/pgtable-bits.h>
> +#include <asm/cpu-features.h>
>
> struct mm_struct;
> struct vm_area_struct;
> @@ -626,6 +627,8 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
>
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
> +#define gup_fast_permitted(start, end) (!cpu_has_dc_aliases)
> +
Today this check is only being done on the get_user_pages_fast() -
after this patch it is also done for __get_user_pages_fast().
Which means __get_user_pages_fast is now non-functional on a range of
MIPS CPUs, but that seems OK as far as I can tell, so:
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
However, looks to me like this patch is also a bug fix for this:
commit 5b167c123b3c3582f62cf1896465019bc40fe526
Author: Kamal Dasu <kdasu.kdev@gmail.com>
Date: Fri Jun 14 17:10:03 2013 +0000
MIPS: Fix get_user_page_fast() for mips with cache alias
get_user_pages_fast() is missing cache flushes for MIPS platforms with
cache aliases. Filesystem failures observed with DirectIO operations due
to missing flush_anon_page() that use page coloring logic to work with
cache aliases. This fix falls through to take slow_irqon path that calls
get_user_pages() that has required logic for platforms where
cpu_has_dc_aliases is true.
> - pgdp = pgd_offset(mm, addr);
> - do {
> - pgd_t pgd = *pgdp;
> -
> - next = pgd_addr_end(addr, end);
> - if (pgd_none(pgd))
> - goto slow;
> - if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE,
> - pages, &nr))
This is different too, the core code has a p4d layer, but I see that
whole thing gets NOP'd by the compiler as mips uses pgtable-nop4d.h -
right?
Jason
^ permalink raw reply
* Re: [PATCH 03/16] mm: lift the x86_32 PAE version of gup_get_pte to common code
From: Jason Gunthorpe @ 2019-06-21 13:45 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-4-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:49PM +0200, Christoph Hellwig wrote:
> The split low/high access is the only non-READ_ONCE version of
> gup_get_pte that did show up in the various arch implemenations.
> Lift it to common code and drop the ifdef based arch override.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/x86/Kconfig | 1 +
> arch/x86/include/asm/pgtable-3level.h | 47 ------------------------
> arch/x86/kvm/mmu.c | 2 +-
> mm/Kconfig | 3 ++
> mm/gup.c | 51 ++++++++++++++++++++++++---
> 5 files changed, 52 insertions(+), 52 deletions(-)
Yep, the sh and mips conversions look right too.
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
> diff --git a/mm/Kconfig b/mm/Kconfig
> index f0c76ba47695..fe51f104a9e0 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -762,6 +762,9 @@ config GUP_BENCHMARK
>
> See tools/testing/selftests/vm/gup_benchmark.c
>
> +config GUP_GET_PTE_LOW_HIGH
> + bool
> +
The config name seems a bit out of place though, should it be prefixed
with GENERIC_ or ARCH_?
Jason
^ permalink raw reply
* Re: [PATCH 02/16] mm: simplify gup_fast_permitted
From: Jason Gunthorpe @ 2019-06-21 13:40 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-3-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:48PM +0200, Christoph Hellwig wrote:
> Pass in the already calculated end value instead of recomputing it, and
> leave the end > start check in the callers instead of duplicating them
> in the arch code.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> arch/s390/include/asm/pgtable.h | 8 +-------
> arch/x86/include/asm/pgtable_64.h | 8 +-------
> mm/gup.c | 17 +++++++----------
> 3 files changed, 9 insertions(+), 24 deletions(-)
Much cleaner
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
Jason
^ permalink raw reply
* Re: [PATCH 01/16] mm: use untagged_addr() for get_user_pages_fast addresses
From: Jason Gunthorpe @ 2019-06-21 13:39 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-2-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:47PM +0200, Christoph Hellwig wrote:
> This will allow sparc64 to override its ADI tags for
> get_user_pages and get_user_pages_fast.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> mm/gup.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/gup.c b/mm/gup.c
> index ddde097cf9e4..6bb521db67ec 100644
> +++ b/mm/gup.c
> @@ -2146,7 +2146,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
> unsigned long flags;
> int nr = 0;
>
> - start &= PAGE_MASK;
> + start = untagged_addr(start) & PAGE_MASK;
> len = (unsigned long) nr_pages << PAGE_SHIFT;
> end = start + len;
Hmm, this function, and the other, goes on to do:
if (unlikely(!access_ok((void __user *)start, len)))
return 0;
and I thought that access_ok takes in the tagged pointer?
How about re-order it a bit?
diff --git a/mm/gup.c b/mm/gup.c
index ddde097cf9e410..f48747ced4723b 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2148,11 +2148,12 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
start &= PAGE_MASK;
len = (unsigned long) nr_pages << PAGE_SHIFT;
- end = start + len;
-
if (unlikely(!access_ok((void __user *)start, len)))
return 0;
+ start = untagged_ptr(start);
+ end = start + len;
+
/*
* Disable interrupts. We use the nested form as we can already have
* interrupts disabled by get_futex_key.
^ permalink raw reply related
* Re: [PATCH 01/16] mm: use untagged_addr() for get_user_pages_fast addresses
From: Jason Gunthorpe @ 2019-06-21 13:16 UTC (permalink / raw)
To: Christoph Hellwig
Cc: x86, Rich Felker, Yoshinori Sato, linux-sh, James Hogan,
linuxppc-dev, Khalid Aziz, Nicholas Piggin, linux-mips, linux-mm,
Paul Burton, Paul Mackerras, Andrey Konovalov, sparclinux,
Linus Torvalds, David S. Miller, linux-kernel
In-Reply-To: <20190611144102.8848-2-hch@lst.de>
On Tue, Jun 11, 2019 at 04:40:47PM +0200, Christoph Hellwig wrote:
> This will allow sparc64 to override its ADI tags for
> get_user_pages and get_user_pages_fast.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> mm/gup.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
^ permalink raw reply
* Re: [PATCH 11/13] powerpc/64s: Save r13 in machine_check_common_early
From: Mahesh J Salgaonkar @ 2019-06-21 11:47 UTC (permalink / raw)
To: Santosh Sivaraj
Cc: Aneesh Kumar K.V, linuxppc-dev, Nicholas Piggin,
Mahesh Salgaonkar, Chandan Rajendra, Reza Arbab
In-Reply-To: <d6ae7dd59966ee1c7593b8fd936774c0b13e6dd4.1561020760.git.santosh@fossix.org>
On 2019-06-21 06:27:15 Fri, Santosh Sivaraj wrote:
> From: Reza Arbab <arbab@linux.ibm.com>
>
> Testing my memcpy_mcsafe() work in progress with an injected UE, I get
> an error like this immediately after the function returns:
>
> BUG: Unable to handle kernel data access at 0x7fff84dec8f8
> Faulting instruction address: 0xc0080000009c00b0
> Oops: Kernel access of bad area, sig: 11 [#1]
> LE PAGE_SIZE=64K MMU=Radix MMU=Hash SMP NR_CPUS=2048 NUMA PowerNV
> Modules linked in: mce(O+) vmx_crypto crc32c_vpmsum
> CPU: 0 PID: 1375 Comm: modprobe Tainted: G O 5.1.0-rc6 #267
> NIP: c0080000009c00b0 LR: c0080000009c00a8 CTR: c000000000095f90
> REGS: c0000000ee197790 TRAP: 0300 Tainted: G O (5.1.0-rc6)
> MSR: 900000000280b033 <SF,HV,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 88002826 XER: 00040000
> CFAR: c000000000095f8c DAR: 00007fff84dec8f8 DSISR: 40000000 IRQMASK: 0
> GPR00: 000000006c6c6568 c0000000ee197a20 c0080000009c8400 fffffffffffffff2
> GPR04: c0080000009c02e0 0000000000000006 0000000000000000 c000000003c834c8
> GPR08: 0080000000000000 776a6681b7fb5100 0000000000000000 c0080000009c01c8
> GPR12: c000000000095f90 00007fff84debc00 000000004d071440 0000000000000000
> GPR16: 0000000100000601 c0080000009e0000 c000000000c98dd8 c000000000c98d98
> GPR20: c000000003bba970 c0080000009c04d0 c0080000009c0618 c0000000001e5820
> GPR24: 0000000000000000 0000000000000100 0000000000000001 c000000003bba958
> GPR28: c0080000009c02e8 c0080000009c0318 c0080000009c02e0 0000000000000000
> NIP [c0080000009c00b0] cause_ue+0xa8/0xe8 [mce]
> LR [c0080000009c00a8] cause_ue+0xa0/0xe8 [mce]
>
> To fix, ensure that r13 is properly restored after an MCE.
>
> Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
> ---
> arch/powerpc/kernel/exceptions-64s.S | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
> index 311f1392a2ec..932d8d05892c 100644
> --- a/arch/powerpc/kernel/exceptions-64s.S
> +++ b/arch/powerpc/kernel/exceptions-64s.S
> @@ -265,6 +265,7 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE)
> EXC_REAL_END(machine_check, 0x200, 0x100)
> EXC_VIRT_NONE(0x4200, 0x100)
> TRAMP_REAL_BEGIN(machine_check_common_early)
> + SET_SCRATCH0(r13) /* save r13 */
> EXCEPTION_PROLOG_1(PACA_EXMC, NOTEST, 0x200)
> /*
> * Register contents:
We do save r13 before we call machine_check_common_early(). I don't
think I understood clearly how this change fixes the issue you are
seeing. What am I missing here ?
Above change will push the paca pointer to scratch0 overwriting the
original saved r13.
EXC_REAL_BEGIN(machine_check, 0x200, 0x100)
/* This is moved out of line as it can be patched by FW, but
* some code path might still want to branch into the original
* vector
*/
SET_SCRATCH0(r13) /* save r13 */
EXCEPTION_PROLOG_0(PACA_EXMC)
BEGIN_FTR_SECTION
b machine_check_common_early
Thanks,
-Mahesh.
^ permalink raw reply
* Re: [PATCH 2/4] powerpc/powernv: remove the unused tunneling exports
From: Christoph Hellwig @ 2019-06-21 9:24 UTC (permalink / raw)
To: Frederic Barrat
Cc: Andrew Donnellan, Linux Kernel Mailing List,
Oliver O'Halloran, Paul Mackerras, linuxppc-dev,
Christoph Hellwig
In-Reply-To: <048e1242-a6ea-5d56-dc9a-e16f9eedf6d9@linux.ibm.com>
On Fri, Jun 21, 2019 at 11:21:38AM +0200, Frederic Barrat wrote:
> The as-notify can be used in both CAPI mode and PCI mode. In capi mode,
> it's integrated in the capi protocol, so the cxl driver doesn't need to do
> extra setup, compared to what's already done to activate capi.
> As mentioned in a previous iteration of that patchset, those APIs are to be
> used by the Mellanox CX5 driver. The in-tree driver is always a step behind
> their latest, but word is they are working on upstreaming those
> interactions.
We can review them together with the driver. Especially as we need to
consider if we even want to support it if there is no generic platform
independent inferface.
^ permalink raw reply
* Re: [PATCH 2/4] powerpc/powernv: remove the unused tunneling exports
From: Frederic Barrat @ 2019-06-21 9:21 UTC (permalink / raw)
To: Oliver O'Halloran, Christoph Hellwig
Cc: Andrew Donnellan, Linux Kernel Mailing List, Paul Mackerras,
linuxppc-dev
In-Reply-To: <CAOSf1CFu_T=7weW0eagzjTc8474ntVx1uCKdQh8sX85bfaPxCQ@mail.gmail.com>
Le 21/06/2019 à 03:47, Oliver O'Halloran a écrit :
> On Thu, May 23, 2019 at 5:51 PM Christoph Hellwig <hch@lst.de> wrote:
>>
>> These have been unused ever since they've been added to the kernel.
>>
>> Signed-off-by: Christoph Hellwig <hch@lst.de>
>> ---
>> arch/powerpc/include/asm/pnv-pci.h | 4 --
>> arch/powerpc/platforms/powernv/pci-ioda.c | 4 +-
>> arch/powerpc/platforms/powernv/pci.c | 71 -----------------------
>> arch/powerpc/platforms/powernv/pci.h | 1 -
>> 4 files changed, 3 insertions(+), 77 deletions(-)
>>
>> diff --git a/arch/powerpc/include/asm/pnv-pci.h b/arch/powerpc/include/asm/pnv-pci.h
>> index 9fcb0bc462c6..1ab4b0111abc 100644
>> --- a/arch/powerpc/include/asm/pnv-pci.h
>> +++ b/arch/powerpc/include/asm/pnv-pci.h
>> @@ -27,12 +27,8 @@ extern int pnv_pci_get_power_state(uint64_t id, uint8_t *state);
>> extern int pnv_pci_set_power_state(uint64_t id, uint8_t state,
>> struct opal_msg *msg);
>>
>> -extern int pnv_pci_enable_tunnel(struct pci_dev *dev, uint64_t *asnind);
>> -extern int pnv_pci_disable_tunnel(struct pci_dev *dev);
>> extern int pnv_pci_set_tunnel_bar(struct pci_dev *dev, uint64_t addr,
>> int enable);
>> -extern int pnv_pci_get_as_notify_info(struct task_struct *task, u32 *lpid,
>> - u32 *pid, u32 *tid);
>
> IIRC as-notify was for CAPI which has an in-tree driver (cxl). Fred or
> Andrew (+cc), what's going on with this? Will it ever see the light of
> day?
The as-notify can be used in both CAPI mode and PCI mode. In capi mode,
it's integrated in the capi protocol, so the cxl driver doesn't need to
do extra setup, compared to what's already done to activate capi.
As mentioned in a previous iteration of that patchset, those APIs are to
be used by the Mellanox CX5 driver. The in-tree driver is always a step
behind their latest, but word is they are working on upstreaming those
interactions.
Fred
>> int pnv_phb_to_cxl_mode(struct pci_dev *dev, uint64_t mode);
>> int pnv_cxl_ioda_msi_setup(struct pci_dev *dev, unsigned int hwirq,
>> unsigned int virq);
>> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
>> index 126602b4e399..6b0caa2d0425 100644
>> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
>> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
>> @@ -54,6 +54,8 @@
>> static const char * const pnv_phb_names[] = { "IODA1", "IODA2", "NPU_NVLINK",
>> "NPU_OCAPI" };
>>
>> +static void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable);
>> +
>> void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
>> const char *fmt, ...)
>> {
>> @@ -2360,7 +2362,7 @@ static long pnv_pci_ioda2_set_window(struct iommu_table_group *table_group,
>> return 0;
>> }
>>
>> -void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable)
>> +static void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable)
>> {
>> uint16_t window_id = (pe->pe_number << 1 ) + 1;
>> int64_t rc;
>> diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
>> index 8d28f2932c3b..fc69f5611020 100644
>> --- a/arch/powerpc/platforms/powernv/pci.c
>> +++ b/arch/powerpc/platforms/powernv/pci.c
>> @@ -868,54 +868,6 @@ struct device_node *pnv_pci_get_phb_node(struct pci_dev *dev)
>> }
>> EXPORT_SYMBOL(pnv_pci_get_phb_node);
>>
>> -int pnv_pci_enable_tunnel(struct pci_dev *dev, u64 *asnind)
>> -{
>> - struct device_node *np;
>> - const __be32 *prop;
>> - struct pnv_ioda_pe *pe;
>> - uint16_t window_id;
>> - int rc;
>> -
>> - if (!radix_enabled())
>> - return -ENXIO;
>> -
>> - if (!(np = pnv_pci_get_phb_node(dev)))
>> - return -ENXIO;
>> -
>> - prop = of_get_property(np, "ibm,phb-indications", NULL);
>> - of_node_put(np);
>> -
>> - if (!prop || !prop[1])
>> - return -ENXIO;
>> -
>> - *asnind = (u64)be32_to_cpu(prop[1]);
>> - pe = pnv_ioda_get_pe(dev);
>> - if (!pe)
>> - return -ENODEV;
>> -
>> - /* Increase real window size to accept as_notify messages. */
>> - window_id = (pe->pe_number << 1 ) + 1;
>> - rc = opal_pci_map_pe_dma_window_real(pe->phb->opal_id, pe->pe_number,
>> - window_id, pe->tce_bypass_base,
>> - (uint64_t)1 << 48);
>> - return opal_error_code(rc);
>> -}
>> -EXPORT_SYMBOL_GPL(pnv_pci_enable_tunnel);
>> -
>> -int pnv_pci_disable_tunnel(struct pci_dev *dev)
>> -{
>> - struct pnv_ioda_pe *pe;
>> -
>> - pe = pnv_ioda_get_pe(dev);
>> - if (!pe)
>> - return -ENODEV;
>> -
>> - /* Restore default real window size. */
>> - pnv_pci_ioda2_set_bypass(pe, true);
>> - return 0;
>> -}
>> -EXPORT_SYMBOL_GPL(pnv_pci_disable_tunnel);
>> -
>> int pnv_pci_set_tunnel_bar(struct pci_dev *dev, u64 addr, int enable)
>> {
>> __be64 val;
>> @@ -970,29 +922,6 @@ int pnv_pci_set_tunnel_bar(struct pci_dev *dev, u64 addr, int enable)
>> }
>> EXPORT_SYMBOL_GPL(pnv_pci_set_tunnel_bar);
>>
>> -#ifdef CONFIG_PPC64 /* for thread.tidr */
>> -int pnv_pci_get_as_notify_info(struct task_struct *task, u32 *lpid, u32 *pid,
>> - u32 *tid)
>> -{
>> - struct mm_struct *mm = NULL;
>> -
>> - if (task == NULL)
>> - return -EINVAL;
>> -
>> - mm = get_task_mm(task);
>> - if (mm == NULL)
>> - return -EINVAL;
>> -
>> - *pid = mm->context.id;
>> - mmput(mm);
>> -
>> - *tid = task->thread.tidr;
>> - *lpid = mfspr(SPRN_LPID);
>> - return 0;
>> -}
>> -EXPORT_SYMBOL_GPL(pnv_pci_get_as_notify_info);
>> -#endif
>> -
>> void pnv_pci_shutdown(void)
>> {
>> struct pci_controller *hose;
>> diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
>> index 4f11c077af62..469c24463247 100644
>> --- a/arch/powerpc/platforms/powernv/pci.h
>> +++ b/arch/powerpc/platforms/powernv/pci.h
>> @@ -195,7 +195,6 @@ extern int pnv_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type);
>> extern void pnv_teardown_msi_irqs(struct pci_dev *pdev);
>> extern struct pnv_ioda_pe *pnv_ioda_get_pe(struct pci_dev *dev);
>> extern void pnv_set_msi_irq_chip(struct pnv_phb *phb, unsigned int virq);
>> -extern void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable);
>> extern unsigned long pnv_pci_ioda2_get_table_size(__u32 page_shift,
>> __u64 window_size, __u32 levels);
>> extern int pnv_eeh_post_init(void);
>> --
>> 2.20.1
>>
>
^ permalink raw reply
* [PATCH] powerpc/lib/xor_vmx: Relax frame size for clang
From: Mathieu Malaterre @ 2019-06-21 8:58 UTC (permalink / raw)
To: Michael Ellerman
Cc: Mathieu Malaterre, linux-kernel, Paul Mackerras, Joel Stanley,
linuxppc-dev
When building with clang-8 the frame size limit is hit:
../arch/powerpc/lib/xor_vmx.c:119:6: error: stack frame size of 1200 bytes in function '__xor_altivec_5' [-Werror,-Wframe-larger-than=]
Follow the same approach as commit 9c87156cce5a ("powerpc/xmon: Relax
frame size for clang") until a proper fix is implemented upstream in
clang and relax requirement for clang.
Link: https://github.com/ClangBuiltLinux/linux/issues/563
Cc: Joel Stanley <joel@jms.id.au>
Signed-off-by: Mathieu Malaterre <malat@debian.org>
---
arch/powerpc/lib/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/powerpc/lib/Makefile b/arch/powerpc/lib/Makefile
index c55f9c27bf79..b3f7d64caaf0 100644
--- a/arch/powerpc/lib/Makefile
+++ b/arch/powerpc/lib/Makefile
@@ -58,5 +58,9 @@ obj-$(CONFIG_FTR_FIXUP_SELFTEST) += feature-fixups-test.o
obj-$(CONFIG_ALTIVEC) += xor_vmx.o xor_vmx_glue.o
CFLAGS_xor_vmx.o += -maltivec $(call cc-option,-mabi=altivec)
+ifdef CONFIG_CC_IS_CLANG
+# See https://github.com/ClangBuiltLinux/linux/issues/563
+CFLAGS_xor_vmx.o += -Wframe-larger-than=4096
+endif
obj-$(CONFIG_PPC64) += $(obj64-y)
--
2.20.1
^ permalink raw reply related
* Re: [PATCH 16/16] mm: pass get_user_pages_fast iterator arguments in a structure
From: Nicholas Piggin @ 2019-06-21 8:29 UTC (permalink / raw)
To: Linus Torvalds
Cc: Rich Felker, Yoshinori Sato, Linux-sh list, James Hogan,
the arch/x86 maintainers, Linux List Kernel Mailing,
David S. Miller, Linux-MM, Khalid Aziz, Paul Burton,
Andrey Konovalov, sparclinux, Paul Mackerras, linux-mips,
linuxppc-dev, Christoph Hellwig
In-Reply-To: <CAHk-=wh46y3x5O0HkR=R4ETh6e5pDCrEsJ94CtC0fyQiYYAf6A@mail.gmail.com>
Linus Torvalds's on June 21, 2019 3:21 am:
> On Thu, Jun 20, 2019 at 5:19 AM Nicholas Piggin <npiggin@gmail.com> wrote:
>>
>> The processor aliasing problem happens because the struct will
>> be initialised with stores using one base register (e.g., stack
>> register), and then same memory is loaded using a different
>> register (e.g., parameter register).
>
> Hmm. Honestly, I've never seen anything like that in any kernel profiles.
>
> Compared to the problems I _do_ see (which is usually the obvious
> cache misses, and locking), it must either be in the noise or it's
> some problem specific to whatever CPU you are doing performance work
> on?
No you're right, the performance hit from these flushes is not a
big hit that stands out in cycle counts. I just look at kernel code
for various flushes. Branches not surprisingly are usually the main
culprit, but they're normally not so interesting.
Static alias prediction seems to work well outside this case. It's
interesting, you need both a store ; load sequence that does not
predict well (e.g., using a different base register), and you also
need that load to be executed ahead of the store.
The small stack structure for arguments is the perfect case. Bad
pattern, and load executed right after store. Even then you also need
a reason to delay the store (e.g., source not ready or store queue
full), but those hazards do show up.
Now, even when all that goes wrong, there are dynamic heuristics that
can take over. So if you run a repetitive microbenchmark you won't
see it.
Some CPUs seem to be quite aggressive about giving up and turning off
the alias prediction globally if you take misses (Intel x86 used to do
that IIRC, not sure if they still do). So in that case you wouldn't
even see it show up in one place, everything will just run slightly
slower.
What I worry about is high rate direct IO workloads that see single
flushes in these paths as significant. Or if this thing creeps in to
the kernel too much and just slightly raises global misses enough,
then it will cause disambiguation to be significantly shut down.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH 16/16] mm: pass get_user_pages_fast iterator arguments in a structure
From: Christoph Hellwig @ 2019-06-21 8:15 UTC (permalink / raw)
To: Linus Torvalds
Cc: sparclinux, Rich Felker, Yoshinori Sato, Linux-sh list,
Andrey Konovalov, the arch/x86 maintainers,
Linux List Kernel Mailing, Nicholas Piggin, David S. Miller,
Linux-MM, Khalid Aziz, Paul Burton, James Hogan, Paul Mackerras,
linux-mips, linuxppc-dev, Christoph Hellwig
In-Reply-To: <CAHk-=wh46y3x5O0HkR=R4ETh6e5pDCrEsJ94CtC0fyQiYYAf6A@mail.gmail.com>
On Thu, Jun 20, 2019 at 10:21:46AM -0700, Linus Torvalds wrote:
> Hmm. Honestly, I've never seen anything like that in any kernel profiles.
>
> Compared to the problems I _do_ see (which is usually the obvious
> cache misses, and locking), it must either be in the noise or it's
> some problem specific to whatever CPU you are doing performance work
> on?
>
> I've occasionally seen pipeline hiccups in profiles, but it's usually
> been either some serious glass jaw of the core, or it's been something
> really stupid we did (or occasionally that the compiler did: one in
> particular I remember was how there was a time when gcc would narrow
> stores when it could, so if you set a bit in a word, it would do it
> with a byte store, and then when you read the whole word afterwards
> you'd get a major pipeline stall and it happened to show up in some
> really hot paths).
I've not seen any difference in the GUP bench output here ar all.
But I'm fine with skipping this patch for now, I have a potential
series I'm looking into that would benefit a lot from it, but we
can discusss it in that context and make sure all the other works gets in
in time.
^ permalink raw reply
* Re: [PATCH 05/13] powerpc/mce: Allow notifier callback to handle MCE
From: Mahesh Jagannath Salgaonkar @ 2019-06-21 7:05 UTC (permalink / raw)
To: Santosh Sivaraj, linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <196df6a74f259c041a4269e6cba026a1248ed4af.1561020760.git.santosh@fossix.org>
On 6/21/19 6:27 AM, Santosh Sivaraj wrote:
> From: Reza Arbab <arbab@linux.ibm.com>
>
> If a notifier returns NOTIFY_STOP, consider the MCE handled, just as we
> do when machine_check_early() returns 1.
>
> Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
> ---
> arch/powerpc/include/asm/asm-prototypes.h | 2 +-
> arch/powerpc/kernel/exceptions-64s.S | 3 +++
> arch/powerpc/kernel/mce.c | 28 ++++++++++++++++-------
> 3 files changed, 24 insertions(+), 9 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/asm-prototypes.h b/arch/powerpc/include/asm/asm-prototypes.h
> index f66f26ef3ce0..49ee8f08de2a 100644
> --- a/arch/powerpc/include/asm/asm-prototypes.h
> +++ b/arch/powerpc/include/asm/asm-prototypes.h
> @@ -72,7 +72,7 @@ void machine_check_exception(struct pt_regs *regs);
> void emulation_assist_interrupt(struct pt_regs *regs);
> long do_slb_fault(struct pt_regs *regs, unsigned long ea);
> void do_bad_slb_fault(struct pt_regs *regs, unsigned long ea, long err);
> -void machine_check_notify(struct pt_regs *regs);
> +long machine_check_notify(struct pt_regs *regs);
>
> /* signals, syscalls and interrupts */
> long sys_swapcontext(struct ucontext __user *old_ctx,
> diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
> index 2e56014fca21..c83e38a403fd 100644
> --- a/arch/powerpc/kernel/exceptions-64s.S
> +++ b/arch/powerpc/kernel/exceptions-64s.S
> @@ -460,6 +460,9 @@ EXC_COMMON_BEGIN(machine_check_handle_early)
>
> addi r3,r1,STACK_FRAME_OVERHEAD
> bl machine_check_notify
> + ld r11,RESULT(r1)
> + or r3,r3,r11
> + std r3,RESULT(r1)
>
> ld r12,_MSR(r1)
> BEGIN_FTR_SECTION
> diff --git a/arch/powerpc/kernel/mce.c b/arch/powerpc/kernel/mce.c
> index 0ab171b41ede..912efe58e0b1 100644
> --- a/arch/powerpc/kernel/mce.c
> +++ b/arch/powerpc/kernel/mce.c
> @@ -647,16 +647,28 @@ long hmi_exception_realmode(struct pt_regs *regs)
> return 1;
> }
>
> -void machine_check_notify(struct pt_regs *regs)
> +long machine_check_notify(struct pt_regs *regs)
> {
> - struct machine_check_event evt;
> + int index = __this_cpu_read(mce_nest_count) - 1;
> + struct machine_check_event *evt;
> + int rc;
>
> - if (!get_mce_event(&evt, MCE_EVENT_DONTRELEASE))
> - return;
> + if (index < 0 || index >= MAX_MC_EVT)
> + return 0;
> +
> + evt = this_cpu_ptr(&mce_event[index]);
>
> - blocking_notifier_call_chain(&mce_notifier_list, 0, &evt);
> + rc = blocking_notifier_call_chain(&mce_notifier_list, 0, evt);
> + if (rc & NOTIFY_STOP_MASK) {
> + evt->disposition = MCE_DISPOSITION_RECOVERED;
> + regs->msr |= MSR_RI;
What is the reason for setting MSR_RI ? I don't think this is a good
idea. MSR_RI = 0 means system got MCE interrupt when SRR0 and SRR1
contents were live and was overwritten by MCE interrupt. Hence this
interrupt is unrecoverable irrespective of whether machine check handler
recovers from it or not.
Thanks,
-Mahesh.
^ permalink raw reply
* [PATCH] powerpc/rtas: retry when cpu offline races with suspend/migration
From: Nathan Lynch @ 2019-06-21 6:05 UTC (permalink / raw)
To: linuxppc-dev; +Cc: ego, mmc, julietk
The protocol for suspending or migrating an LPAR requires all present
processor threads to enter H_JOIN. So if we have threads offline, we
have to temporarily bring them up. This can race with administrator
actions such as SMT state changes. As of dfd718a2ed1f ("powerpc/rtas:
Fix a potential race between CPU-Offline & Migration"),
rtas_ibm_suspend_me() accounts for this, but errors out with -EBUSY
for what almost certainly is a transient condition in any reasonable
scenario.
Callers of rtas_ibm_suspend_me() already retry when -EAGAIN is
returned, and it is typical during a migration for that to happen
repeatedly for several minutes polling the H_VASI_STATE hcall result
before proceeding to the next stage.
So return -EAGAIN instead of -EBUSY when this race is
encountered. Additionally: logging this event is still appropriate but
use pr_info instead of pr_err; and remove use of unlikely() while here
as this is not a hot path at all.
Fixes: dfd718a2ed1f ("powerpc/rtas: Fix a potential race between CPU-Offline & Migration")
Signed-off-by: Nathan Lynch <nathanl@linux.ibm.com>
---
arch/powerpc/kernel/rtas.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/kernel/rtas.c b/arch/powerpc/kernel/rtas.c
index fbc676160adf..9b4d2a2ffb4f 100644
--- a/arch/powerpc/kernel/rtas.c
+++ b/arch/powerpc/kernel/rtas.c
@@ -984,10 +984,9 @@ int rtas_ibm_suspend_me(u64 handle)
cpu_hotplug_disable();
/* Check if we raced with a CPU-Offline Operation */
- if (unlikely(!cpumask_equal(cpu_present_mask, cpu_online_mask))) {
- pr_err("%s: Raced against a concurrent CPU-Offline\n",
- __func__);
- atomic_set(&data.error, -EBUSY);
+ if (!cpumask_equal(cpu_present_mask, cpu_online_mask)) {
+ pr_info("%s: Raced against a concurrent CPU-Offline\n", __func__);
+ atomic_set(&data.error, -EAGAIN);
goto out_hotplug_enable;
}
--
2.20.1
^ permalink raw reply related
* [PATCH] powerpc/powernv: Rename pe_level_printk to pe_printk and embed KERN_LEVEL in format
From: Joe Perches @ 2019-06-21 5:36 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman
Cc: linuxppc-dev, linux-kernel
Remove the separate KERN_<LEVEL> from each pe_level_printk and
instead add the KERN_<LEVEL> to the format.
pfix in pe_level_printk could also be used uninitialized so
add a new else and set pfx to the hex value of pe->flags.
Rename pe_level_printk to pe_printk and update the pe_<level>
macros.
Signed-off-by: Joe Perches <joe@perches.com>
---
arch/powerpc/platforms/powernv/pci-ioda.c | 14 ++++++++++++--
arch/powerpc/platforms/powernv/pci.h | 11 +++++------
2 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 10cc42b9e541..60fc36ae626a 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -50,15 +50,23 @@
static const char * const pnv_phb_names[] = { "IODA1", "IODA2", "NPU_NVLINK",
"NPU_OCAPI" };
-void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
- const char *fmt, ...)
+void pe_printk(const struct pnv_ioda_pe *pe, const char *fmt, ...)
{
struct va_format vaf;
va_list args;
char pfix[32];
+ char level[PRINTK_MAX_SINGLE_HEADER_LEN + 1] = "\0";
va_start(args, fmt);
+ while (printk_get_level(fmt)) {
+ size_t size = printk_skip_level(fmt) - fmt;
+
+ memcpy(level, fmt, size);
+ level[size] = '\0';
+ fmt += size;
+ }
+
vaf.fmt = fmt;
vaf.va = &args;
@@ -74,6 +82,8 @@ void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
(pe->rid & 0xff00) >> 8,
PCI_SLOT(pe->rid), PCI_FUNC(pe->rid));
#endif /* CONFIG_PCI_IOV*/
+ else
+ sprintf(pfix, "(flags: 0x%lx)", pe->flags);
printk("%spci %s: [PE# %.2x] %pV",
level, pfix, pe->pe_number, &vaf);
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index be26ab3d99e0..870b21f55b3f 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -205,15 +205,14 @@ extern unsigned long pnv_pci_ioda2_get_table_size(__u32 page_shift,
__u64 window_size, __u32 levels);
extern int pnv_eeh_post_init(void);
-__printf(3, 4)
-extern void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
- const char *fmt, ...);
+__printf(2, 3)
+extern void pe_printk(const struct pnv_ioda_pe *pe, const char *fmt, ...);
#define pe_err(pe, fmt, ...) \
- pe_level_printk(pe, KERN_ERR, fmt, ##__VA_ARGS__)
+ pe_printk(pe, KERN_ERR fmt, ##__VA_ARGS__)
#define pe_warn(pe, fmt, ...) \
- pe_level_printk(pe, KERN_WARNING, fmt, ##__VA_ARGS__)
+ pe_printk(pe, KERN_WARNING fmt, ##__VA_ARGS__)
#define pe_info(pe, fmt, ...) \
- pe_level_printk(pe, KERN_INFO, fmt, ##__VA_ARGS__)
+ pe_printk(pe, KERN_INFO fmt, ##__VA_ARGS__)
/* Nvlink functions */
extern void pnv_npu_try_dma_set_bypass(struct pci_dev *gpdev, bool bypass);
^ permalink raw reply related
* [PATCH kernel] powerpc/of/pci: Rewrite pci_parse_of_flags
From: Alexey Kardashevskiy @ 2019-06-21 5:11 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, Oliver O'Halloran, Paul Mackerras,
Sam Bobroff
The existing code uses bunch of hardcoded values from the PCI Bus Binding
to IEEE Std 1275 spec; and it does so in quite non-obvious way.
This defines fields from the cell#0 of the "reg" property of a PCI device
and uses them for parsing.
This should cause no behavioral change.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/kernel/pci_of_scan.c | 53 ++++++++++++++++++++++++++-----
1 file changed, 45 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c
index 8078bce89bec..fc55ee710eb3 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -42,13 +42,50 @@ unsigned int pci_parse_of_flags(u32 addr0, int bridge)
{
unsigned int flags = 0;
- if (addr0 & 0x02000000) {
+/*
+ * PCI Bus Binding to IEEE Std 1275-1994
+ *
+ * Bit# 33222222 22221111 11111100 00000000
+ * 10987654 32109876 54321098 76543210
+ * phys.hi cell: npt000ss bbbbbbbb dddddfff rrrrrrrr
+ * phys.mid cell: hhhhhhhh hhhhhhhh hhhhhhhh hhhhhhhh
+ * phys.lo cell: llllllll llllllll llllllll llllllll
+ *
+ * where:
+ * n is 0 if the address is relocatable, 1 otherwise
+ * p is 1 if the addressable region is "prefetchable", 0 otherwise
+ * t is 1 if the address is aliased (for non-relocatable I/O),
+ * below 1 MB (for Memory),or below 64 KB (for relocatable I/O).
+ * ss is the space code, denoting the address space
+ * bbbbbbbb is the 8-bit Bus Number
+ * ddddd is the 5-bit Device Number
+ * fff is the 3-bit Function Number
+ * rrrrrrrr is the 8-bit Register Number
+ */
+#define OF_PCI_ADDR0_SPACE_CODE(ss) (((ss)&0x3UL)<<24)
+#define OF_PCI_ADDR0_SPACE_CFG OF_PCI_ADDR0_SPACE_CODE(0)
+#define OF_PCI_ADDR0_SPACE_IO OF_PCI_ADDR0_SPACE_CODE(1)
+#define OF_PCI_ADDR0_SPACE_MMIO32 OF_PCI_ADDR0_SPACE_CODE(2)
+#define OF_PCI_ADDR0_SPACE_MMIO64 OF_PCI_ADDR0_SPACE_CODE(3)
+#define OF_PCI_ADDR0_SPACE_MMIO (OF_PCI_ADDR0_SPACE_MMIO32 | \
+ OF_PCI_ADDR0_SPACE_MMIO64)
+#define OF_PCI_ADDR0_RELOC (1UL<<31)
+#define OF_PCI_ADDR0_PREFETCH (1UL<<30)
+#define OF_PCI_ADDR0_ALIAS (1UL<<29)
+#define OF_PCI_ADDR0_BUS 0x00FF0000UL
+#define OF_PCI_ADDR0_DEV 0x0000F800UL
+#define OF_PCI_ADDR0_FN 0x00000700UL
+#define OF_PCI_ADDR0_BARREG 0x000000FFUL
+
+ if (addr0 & OF_PCI_ADDR0_SPACE_MMIO) {
flags = IORESOURCE_MEM | PCI_BASE_ADDRESS_SPACE_MEMORY;
- flags |= (addr0 >> 22) & PCI_BASE_ADDRESS_MEM_TYPE_64;
- if (flags & PCI_BASE_ADDRESS_MEM_TYPE_64)
- flags |= IORESOURCE_MEM_64;
- flags |= (addr0 >> 28) & PCI_BASE_ADDRESS_MEM_TYPE_1M;
- if (addr0 & 0x40000000)
+ if ((addr0 & OF_PCI_ADDR0_SPACE_MMIO) ==
+ OF_PCI_ADDR0_SPACE_MMIO64)
+ flags |= PCI_BASE_ADDRESS_MEM_TYPE_64 |
+ IORESOURCE_MEM_64;
+ if (addr0 & OF_PCI_ADDR0_ALIAS)
+ flags |= PCI_BASE_ADDRESS_MEM_TYPE_1M;
+ if (addr0 & OF_PCI_ADDR0_PREFETCH)
flags |= IORESOURCE_PREFETCH
| PCI_BASE_ADDRESS_MEM_PREFETCH;
/* Note: We don't know whether the ROM has been left enabled
@@ -56,9 +93,9 @@ unsigned int pci_parse_of_flags(u32 addr0, int bridge)
* not set the IORESOURCE_ROM_ENABLE flag) for now rather than
* do a config space read, it will be force-enabled if needed
*/
- if (!bridge && (addr0 & 0xff) == 0x30)
+ if (!bridge && (addr0 & OF_PCI_ADDR0_BARREG) == PCI_ROM_ADDRESS)
flags |= IORESOURCE_READONLY;
- } else if (addr0 & 0x01000000)
+ } else if (addr0 & OF_PCI_ADDR0_SPACE_IO)
flags = IORESOURCE_IO | PCI_BASE_ADDRESS_SPACE_IO;
if (flags)
flags |= IORESOURCE_SIZEALIGN;
--
2.17.1
^ permalink raw reply related
* [PATCH 13/13] powerpc: add machine check safe copy_to_user
From: Santosh Sivaraj @ 2019-06-21 4:55 UTC (permalink / raw)
To: linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <cover.1561020760.git.santosh@fossix.org>
Use memcpy_mcsafe() implementation to define copy_to_user_mcsafe()
Signed-off-by: Santosh Sivaraj <santosh@fossix.org>
---
arch/powerpc/Kconfig | 1 +
arch/powerpc/include/asm/uaccess.h | 12 ++++++++++++
2 files changed, 13 insertions(+)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 8c1c636308c8..a173b392c272 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -134,6 +134,7 @@ config PPC
select ARCH_HAS_STRICT_KERNEL_RWX if ((PPC_BOOK3S_64 || PPC32) && !RELOCATABLE && !HIBERNATION)
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
select ARCH_HAS_UACCESS_FLUSHCACHE if PPC64
+ select ARCH_HAS_UACCESS_MCSAFE if PPC64
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARCH_HAS_ZONE_DEVICE if PPC_BOOK3S_64
select ARCH_HAVE_NMI_SAFE_CMPXCHG
diff --git a/arch/powerpc/include/asm/uaccess.h b/arch/powerpc/include/asm/uaccess.h
index 76f34346b642..f8fcaab4c5bc 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -386,6 +386,18 @@ static inline unsigned long raw_copy_to_user(void __user *to,
return ret;
}
+static __always_inline unsigned long __must_check
+copy_to_user_mcsafe(void __user *to, const void *from, unsigned long n)
+{
+ if (likely(check_copy_size(from, n, true))) {
+ allow_write_to_user(to, n);
+ n = memcpy_mcsafe(to, from, n);
+ prevent_write_to_user(to, n);
+ }
+
+ return n;
+}
+
extern unsigned long __clear_user(void __user *addr, unsigned long size);
static inline unsigned long clear_user(void __user *addr, unsigned long size)
--
2.20.1
^ permalink raw reply related
* [PATCH 12/13] powerpc/memcpy_mcsafe: return remaining bytes
From: Santosh Sivaraj @ 2019-06-21 4:55 UTC (permalink / raw)
To: linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <cover.1561020760.git.santosh@fossix.org>
memcpy_mcsafe currently return -EFAULT on a machine check exception, change
it to return the remaining bytes that needs to be copied, so that machine
check safe copy_to_user can maintain the same behavior as copy_to_user.
Signed-off-by: Santosh Sivaraj <santosh@fossix.org>
---
arch/powerpc/lib/memcpy_mcsafe_64.S | 129 +++++++++++++++-------------
1 file changed, 70 insertions(+), 59 deletions(-)
diff --git a/arch/powerpc/lib/memcpy_mcsafe_64.S b/arch/powerpc/lib/memcpy_mcsafe_64.S
index 50f865db0338..566c664aa640 100644
--- a/arch/powerpc/lib/memcpy_mcsafe_64.S
+++ b/arch/powerpc/lib/memcpy_mcsafe_64.S
@@ -30,11 +30,12 @@
ld r14,STK_REG(R14)(r1)
addi r1,r1,STACKFRAMESIZE
.Ldo_err1:
- li r3,-EFAULT
+ mr r3,r7
blr
_GLOBAL(memcpy_mcsafe)
+ mr r7,r5
cmpldi r5,16
blt .Lshort_copy
@@ -49,18 +50,21 @@ err1; lbz r0,0(r4)
addi r4,r4,1
err1; stb r0,0(r3)
addi r3,r3,1
+ subi r7,r7,1
1: bf cr7*4+2,2f
err1; lhz r0,0(r4)
addi r4,r4,2
err1; sth r0,0(r3)
addi r3,r3,2
+ subi r7,r7,2
2: bf cr7*4+1,3f
err1; lwz r0,0(r4)
addi r4,r4,4
err1; stw r0,0(r3)
addi r3,r3,4
+ subi r7,r7,4
3: sub r5,r5,r6
cmpldi r5,128
@@ -87,43 +91,69 @@ err1; stw r0,0(r3)
4:
err2; ld r0,0(r4)
err2; ld r6,8(r4)
-err2; ld r7,16(r4)
-err2; ld r8,24(r4)
-err2; ld r9,32(r4)
-err2; ld r10,40(r4)
-err2; ld r11,48(r4)
-err2; ld r12,56(r4)
-err2; ld r14,64(r4)
-err2; ld r15,72(r4)
-err2; ld r16,80(r4)
-err2; ld r17,88(r4)
-err2; ld r18,96(r4)
-err2; ld r19,104(r4)
-err2; ld r20,112(r4)
-err2; ld r21,120(r4)
+err2; ld r8,16(r4)
+err2; ld r9,24(r4)
+err2; ld r10,32(r4)
+err2; ld r11,40(r4)
+err2; ld r12,48(r4)
+err2; ld r14,56(r4)
+err2; ld r15,64(r4)
+err2; ld r16,72(r4)
+err2; ld r17,80(r4)
+err2; ld r18,88(r4)
+err2; ld r19,96(r4)
+err2; ld r20,104(r4)
+err2; ld r21,112(r4)
+err2; ld r22,120(r4)
addi r4,r4,128
err2; std r0,0(r3)
err2; std r6,8(r3)
-err2; std r7,16(r3)
-err2; std r8,24(r3)
-err2; std r9,32(r3)
-err2; std r10,40(r3)
-err2; std r11,48(r3)
-err2; std r12,56(r3)
-err2; std r14,64(r3)
-err2; std r15,72(r3)
-err2; std r16,80(r3)
-err2; std r17,88(r3)
-err2; std r18,96(r3)
-err2; std r19,104(r3)
-err2; std r20,112(r3)
-err2; std r21,120(r3)
+err2; std r8,16(r3)
+err2; std r9,24(r3)
+err2; std r10,32(r3)
+err2; std r11,40(r3)
+err2; std r12,48(r3)
+err2; std r14,56(r3)
+err2; std r15,64(r3)
+err2; std r16,72(r3)
+err2; std r17,80(r3)
+err2; std r18,88(r3)
+err2; std r19,96(r3)
+err2; std r20,104(r3)
+err2; std r21,112(r3)
+err2; std r22,120(r3)
addi r3,r3,128
+ subi r7,r7,128
bdnz 4b
clrldi r5,r5,(64-7)
- ld r14,STK_REG(R14)(r1)
+ /* Up to 127B to go */
+5: srdi r6,r5,4
+ mtocrf 0x01,r6
+
+6: bf cr7*4+1,7f
+err2; ld r0,0(r4)
+err2; ld r6,8(r4)
+err2; ld r8,16(r4)
+err2; ld r9,24(r4)
+err2; ld r10,32(r4)
+err2; ld r11,40(r4)
+err2; ld r12,48(r4)
+err2; ld r14,56(r4)
+ addi r4,r4,64
+err2; std r0,0(r3)
+err2; std r6,8(r3)
+err2; std r8,16(r3)
+err2; std r9,24(r3)
+err2; std r10,32(r3)
+err2; std r11,40(r3)
+err2; std r12,48(r3)
+err2; std r14,56(r3)
+ addi r3,r3,64
+ subi r7,r7,64
+
+7: ld r14,STK_REG(R14)(r1)
ld r15,STK_REG(R15)(r1)
ld r16,STK_REG(R16)(r1)
ld r17,STK_REG(R17)(r1)
@@ -134,42 +164,19 @@ err2; std r21,120(r3)
ld r22,STK_REG(R22)(r1)
addi r1,r1,STACKFRAMESIZE
- /* Up to 127B to go */
-5: srdi r6,r5,4
- mtocrf 0x01,r6
-
-6: bf cr7*4+1,7f
-err1; ld r0,0(r4)
-err1; ld r6,8(r4)
-err1; ld r7,16(r4)
-err1; ld r8,24(r4)
-err1; ld r9,32(r4)
-err1; ld r10,40(r4)
-err1; ld r11,48(r4)
-err1; ld r12,56(r4)
- addi r4,r4,64
-err1; std r0,0(r3)
-err1; std r6,8(r3)
-err1; std r7,16(r3)
-err1; std r8,24(r3)
-err1; std r9,32(r3)
-err1; std r10,40(r3)
-err1; std r11,48(r3)
-err1; std r12,56(r3)
- addi r3,r3,64
-
/* Up to 63B to go */
-7: bf cr7*4+2,8f
+ bf cr7*4+2,8f
err1; ld r0,0(r4)
err1; ld r6,8(r4)
-err1; ld r7,16(r4)
-err1; ld r8,24(r4)
+err1; ld r8,16(r4)
+err1; ld r9,24(r4)
addi r4,r4,32
err1; std r0,0(r3)
err1; std r6,8(r3)
-err1; std r7,16(r3)
-err1; std r8,24(r3)
+err1; std r8,16(r3)
+err1; std r9,24(r3)
addi r3,r3,32
+ subi r7,r7,32
/* Up to 31B to go */
8: bf cr7*4+3,9f
@@ -179,6 +186,7 @@ err1; ld r6,8(r4)
err1; std r0,0(r3)
err1; std r6,8(r3)
addi r3,r3,16
+ subi r7,r7,16
9: clrldi r5,r5,(64-4)
@@ -192,18 +200,21 @@ err1; lwz r6,4(r4)
err1; stw r0,0(r3)
err1; stw r6,4(r3)
addi r3,r3,8
+ subi r7,r7,8
12: bf cr7*4+1,13f
err1; lwz r0,0(r4)
addi r4,r4,4
err1; stw r0,0(r3)
addi r3,r3,4
+ subi r7,r7,4
13: bf cr7*4+2,14f
err1; lhz r0,0(r4)
addi r4,r4,2
err1; sth r0,0(r3)
addi r3,r3,2
+ subi r7,r7,2
14: bf cr7*4+3,15f
err1; lbz r0,0(r4)
--
2.20.1
^ permalink raw reply related
* [PATCH 11/13] powerpc/64s: Save r13 in machine_check_common_early
From: Santosh Sivaraj @ 2019-06-21 4:55 UTC (permalink / raw)
To: linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <cover.1561020760.git.santosh@fossix.org>
From: Reza Arbab <arbab@linux.ibm.com>
Testing my memcpy_mcsafe() work in progress with an injected UE, I get
an error like this immediately after the function returns:
BUG: Unable to handle kernel data access at 0x7fff84dec8f8
Faulting instruction address: 0xc0080000009c00b0
Oops: Kernel access of bad area, sig: 11 [#1]
LE PAGE_SIZE=64K MMU=Radix MMU=Hash SMP NR_CPUS=2048 NUMA PowerNV
Modules linked in: mce(O+) vmx_crypto crc32c_vpmsum
CPU: 0 PID: 1375 Comm: modprobe Tainted: G O 5.1.0-rc6 #267
NIP: c0080000009c00b0 LR: c0080000009c00a8 CTR: c000000000095f90
REGS: c0000000ee197790 TRAP: 0300 Tainted: G O (5.1.0-rc6)
MSR: 900000000280b033 <SF,HV,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 88002826 XER: 00040000
CFAR: c000000000095f8c DAR: 00007fff84dec8f8 DSISR: 40000000 IRQMASK: 0
GPR00: 000000006c6c6568 c0000000ee197a20 c0080000009c8400 fffffffffffffff2
GPR04: c0080000009c02e0 0000000000000006 0000000000000000 c000000003c834c8
GPR08: 0080000000000000 776a6681b7fb5100 0000000000000000 c0080000009c01c8
GPR12: c000000000095f90 00007fff84debc00 000000004d071440 0000000000000000
GPR16: 0000000100000601 c0080000009e0000 c000000000c98dd8 c000000000c98d98
GPR20: c000000003bba970 c0080000009c04d0 c0080000009c0618 c0000000001e5820
GPR24: 0000000000000000 0000000000000100 0000000000000001 c000000003bba958
GPR28: c0080000009c02e8 c0080000009c0318 c0080000009c02e0 0000000000000000
NIP [c0080000009c00b0] cause_ue+0xa8/0xe8 [mce]
LR [c0080000009c00a8] cause_ue+0xa0/0xe8 [mce]
To fix, ensure that r13 is properly restored after an MCE.
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
---
arch/powerpc/kernel/exceptions-64s.S | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 311f1392a2ec..932d8d05892c 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -265,6 +265,7 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE)
EXC_REAL_END(machine_check, 0x200, 0x100)
EXC_VIRT_NONE(0x4200, 0x100)
TRAMP_REAL_BEGIN(machine_check_common_early)
+ SET_SCRATCH0(r13) /* save r13 */
EXCEPTION_PROLOG_1(PACA_EXMC, NOTEST, 0x200)
/*
* Register contents:
--
2.20.1
^ permalink raw reply related
* [PATCH 10/13] powerpc/mce: Enable MCE notifiers in external modules
From: Santosh Sivaraj @ 2019-06-21 4:55 UTC (permalink / raw)
To: linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <cover.1561020760.git.santosh@fossix.org>
From: Reza Arbab <arbab@linux.ibm.com>
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
---
arch/powerpc/kernel/exceptions-64s.S | 6 ++++++
arch/powerpc/kernel/mce.c | 2 ++
2 files changed, 8 insertions(+)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index c83e38a403fd..311f1392a2ec 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -458,6 +458,12 @@ EXC_COMMON_BEGIN(machine_check_handle_early)
bl machine_check_early
std r3,RESULT(r1) /* Save result */
+ /* Notifiers may be in a module, so enable virtual addressing. */
+ mfmsr r11
+ ori r11,r11,MSR_IR
+ ori r11,r11,MSR_DR
+ mtmsr r11
+
addi r3,r1,STACK_FRAME_OVERHEAD
bl machine_check_notify
ld r11,RESULT(r1)
diff --git a/arch/powerpc/kernel/mce.c b/arch/powerpc/kernel/mce.c
index 9cb5a731377b..413f7866a9c4 100644
--- a/arch/powerpc/kernel/mce.c
+++ b/arch/powerpc/kernel/mce.c
@@ -50,11 +50,13 @@ int mce_register_notifier(struct notifier_block *nb)
{
return blocking_notifier_chain_register(&mce_notifier_list, nb);
}
+EXPORT_SYMBOL_GPL(mce_register_notifier);
int mce_unregister_notifier(struct notifier_block *nb)
{
return blocking_notifier_chain_unregister(&mce_notifier_list, nb);
}
+EXPORT_SYMBOL_GPL(mce_unregister_notifier);
static int check_memcpy_mcsafe(struct notifier_block *nb, unsigned long val,
void *data)
--
2.20.1
^ permalink raw reply related
* [PATCH 09/13] powerpc/mce: Handle memcpy_mcsafe()
From: Santosh Sivaraj @ 2019-06-21 4:55 UTC (permalink / raw)
To: linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <cover.1561020760.git.santosh@fossix.org>
From: Reza Arbab <arbab@linux.ibm.com>
Add an mce notifier intended to service memcpy_mcsafe().
The notifier uses this heuristic; if a UE occurs when accessing device
memory, and the faulting instruction had a fixup entry, the callback
will return NOTIFY_STOP.
This causes the notification mechanism to consider the MCE handled and
continue execution at the fixup address, which returns -EFAULT from the
memcpy_mcsafe() call.
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
---
arch/powerpc/kernel/mce.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/arch/powerpc/kernel/mce.c b/arch/powerpc/kernel/mce.c
index 8afda1ab7358..9cb5a731377b 100644
--- a/arch/powerpc/kernel/mce.c
+++ b/arch/powerpc/kernel/mce.c
@@ -56,6 +56,40 @@ int mce_unregister_notifier(struct notifier_block *nb)
return blocking_notifier_chain_unregister(&mce_notifier_list, nb);
}
+static int check_memcpy_mcsafe(struct notifier_block *nb, unsigned long val,
+ void *data)
+{
+ struct machine_check_event *evt = data;
+ unsigned long pfn;
+ struct page *page;
+
+ if (evt->error_type != MCE_ERROR_TYPE_UE ||
+ !evt->u.ue_error.physical_address_provided)
+ return NOTIFY_DONE;
+
+ pfn = evt->u.ue_error.physical_address >> PAGE_SHIFT;
+ page = pfn_to_page(pfn);
+ if (!page)
+ return NOTIFY_DONE;
+
+ /* HMM and PMEM */
+ if (is_zone_device_page(page) && evt->u.ue_error.fixup_address_provided)
+ return NOTIFY_STOP;
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block memcpy_mcsafe_nb = {
+ .notifier_call = check_memcpy_mcsafe
+};
+
+static int __init mce_mcsafe_register(void)
+{
+ mce_register_notifier(&memcpy_mcsafe_nb);
+ return 0;
+}
+arch_initcall(mce_mcsafe_register);
+
static void mce_set_error_info(struct machine_check_event *mce,
struct mce_error_info *mce_err)
{
--
2.20.1
^ permalink raw reply related
* [PATCH 08/13] powerpc/memcpy: Add memcpy_mcsafe for pmem
From: Santosh Sivaraj @ 2019-06-21 4:55 UTC (permalink / raw)
To: linuxppc-dev
Cc: Aneesh Kumar K.V, Mahesh Salgaonkar, Nicholas Piggin,
Chandan Rajendra, Reza Arbab
In-Reply-To: <cover.1561020760.git.santosh@fossix.org>
From: Balbir Singh <bsingharora@gmail.com>
The pmem infrastructure uses memcpy_mcsafe in the pmem
layer so as to convert machine check exceptions into
a return value on failure in case a machine check
exception is encountered during the memcpy.
This patch largely borrows from the copyuser_power7
logic and does not add the VMX optimizations, largely
to keep the patch simple. If needed those optimizations
can be folded in.
Signed-off-by: Balbir Singh <bsingharora@gmail.com>
Acked-by: Nicholas Piggin <npiggin@gmail.com>
[arbab@linux.ibm.com: Added symbol export]
---
arch/powerpc/include/asm/string.h | 2 +
arch/powerpc/lib/Makefile | 2 +-
arch/powerpc/lib/memcpy_mcsafe_64.S | 215 ++++++++++++++++++++++++++++
3 files changed, 218 insertions(+), 1 deletion(-)
create mode 100644 arch/powerpc/lib/memcpy_mcsafe_64.S
diff --git a/arch/powerpc/include/asm/string.h b/arch/powerpc/include/asm/string.h
index 9bf6dffb4090..b72692702f35 100644
--- a/arch/powerpc/include/asm/string.h
+++ b/arch/powerpc/include/asm/string.h
@@ -53,7 +53,9 @@ void *__memmove(void *to, const void *from, __kernel_size_t n);
#ifndef CONFIG_KASAN
#define __HAVE_ARCH_MEMSET32
#define __HAVE_ARCH_MEMSET64
+#define __HAVE_ARCH_MEMCPY_MCSAFE
+extern int memcpy_mcsafe(void *dst, const void *src, __kernel_size_t sz);
extern void *__memset16(uint16_t *, uint16_t v, __kernel_size_t);
extern void *__memset32(uint32_t *, uint32_t v, __kernel_size_t);
extern void *__memset64(uint64_t *, uint64_t v, __kernel_size_t);
diff --git a/arch/powerpc/lib/Makefile b/arch/powerpc/lib/Makefile
index c55f9c27bf79..529d6536eb4a 100644
--- a/arch/powerpc/lib/Makefile
+++ b/arch/powerpc/lib/Makefile
@@ -39,7 +39,7 @@ obj-$(CONFIG_PPC_BOOK3S_64) += copyuser_power7.o copypage_power7.o \
memcpy_power7.o
obj64-y += copypage_64.o copyuser_64.o mem_64.o hweight_64.o \
- memcpy_64.o pmem.o
+ memcpy_64.o pmem.o memcpy_mcsafe_64.o
obj64-$(CONFIG_SMP) += locks.o
obj64-$(CONFIG_ALTIVEC) += vmx-helper.o
diff --git a/arch/powerpc/lib/memcpy_mcsafe_64.S b/arch/powerpc/lib/memcpy_mcsafe_64.S
new file mode 100644
index 000000000000..50f865db0338
--- /dev/null
+++ b/arch/powerpc/lib/memcpy_mcsafe_64.S
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) IBM Corporation, 2011
+ * Derived from copyuser_power7.s by Anton Blanchard <anton@au.ibm.com>
+ * Author - Balbir Singh <bsingharora@gmail.com>
+ */
+#include <asm/ppc_asm.h>
+#include <asm/errno.h>
+#include <asm/export.h>
+
+ .macro err1
+100:
+ EX_TABLE(100b,.Ldo_err1)
+ .endm
+
+ .macro err2
+200:
+ EX_TABLE(200b,.Ldo_err2)
+ .endm
+
+.Ldo_err2:
+ ld r22,STK_REG(R22)(r1)
+ ld r21,STK_REG(R21)(r1)
+ ld r20,STK_REG(R20)(r1)
+ ld r19,STK_REG(R19)(r1)
+ ld r18,STK_REG(R18)(r1)
+ ld r17,STK_REG(R17)(r1)
+ ld r16,STK_REG(R16)(r1)
+ ld r15,STK_REG(R15)(r1)
+ ld r14,STK_REG(R14)(r1)
+ addi r1,r1,STACKFRAMESIZE
+.Ldo_err1:
+ li r3,-EFAULT
+ blr
+
+
+_GLOBAL(memcpy_mcsafe)
+ cmpldi r5,16
+ blt .Lshort_copy
+
+.Lcopy:
+ /* Get the source 8B aligned */
+ neg r6,r4
+ mtocrf 0x01,r6
+ clrldi r6,r6,(64-3)
+
+ bf cr7*4+3,1f
+err1; lbz r0,0(r4)
+ addi r4,r4,1
+err1; stb r0,0(r3)
+ addi r3,r3,1
+
+1: bf cr7*4+2,2f
+err1; lhz r0,0(r4)
+ addi r4,r4,2
+err1; sth r0,0(r3)
+ addi r3,r3,2
+
+2: bf cr7*4+1,3f
+err1; lwz r0,0(r4)
+ addi r4,r4,4
+err1; stw r0,0(r3)
+ addi r3,r3,4
+
+3: sub r5,r5,r6
+ cmpldi r5,128
+ blt 5f
+
+ mflr r0
+ stdu r1,-STACKFRAMESIZE(r1)
+ std r14,STK_REG(R14)(r1)
+ std r15,STK_REG(R15)(r1)
+ std r16,STK_REG(R16)(r1)
+ std r17,STK_REG(R17)(r1)
+ std r18,STK_REG(R18)(r1)
+ std r19,STK_REG(R19)(r1)
+ std r20,STK_REG(R20)(r1)
+ std r21,STK_REG(R21)(r1)
+ std r22,STK_REG(R22)(r1)
+ std r0,STACKFRAMESIZE+16(r1)
+
+ srdi r6,r5,7
+ mtctr r6
+
+ /* Now do cacheline (128B) sized loads and stores. */
+ .align 5
+4:
+err2; ld r0,0(r4)
+err2; ld r6,8(r4)
+err2; ld r7,16(r4)
+err2; ld r8,24(r4)
+err2; ld r9,32(r4)
+err2; ld r10,40(r4)
+err2; ld r11,48(r4)
+err2; ld r12,56(r4)
+err2; ld r14,64(r4)
+err2; ld r15,72(r4)
+err2; ld r16,80(r4)
+err2; ld r17,88(r4)
+err2; ld r18,96(r4)
+err2; ld r19,104(r4)
+err2; ld r20,112(r4)
+err2; ld r21,120(r4)
+ addi r4,r4,128
+err2; std r0,0(r3)
+err2; std r6,8(r3)
+err2; std r7,16(r3)
+err2; std r8,24(r3)
+err2; std r9,32(r3)
+err2; std r10,40(r3)
+err2; std r11,48(r3)
+err2; std r12,56(r3)
+err2; std r14,64(r3)
+err2; std r15,72(r3)
+err2; std r16,80(r3)
+err2; std r17,88(r3)
+err2; std r18,96(r3)
+err2; std r19,104(r3)
+err2; std r20,112(r3)
+err2; std r21,120(r3)
+ addi r3,r3,128
+ bdnz 4b
+
+ clrldi r5,r5,(64-7)
+
+ ld r14,STK_REG(R14)(r1)
+ ld r15,STK_REG(R15)(r1)
+ ld r16,STK_REG(R16)(r1)
+ ld r17,STK_REG(R17)(r1)
+ ld r18,STK_REG(R18)(r1)
+ ld r19,STK_REG(R19)(r1)
+ ld r20,STK_REG(R20)(r1)
+ ld r21,STK_REG(R21)(r1)
+ ld r22,STK_REG(R22)(r1)
+ addi r1,r1,STACKFRAMESIZE
+
+ /* Up to 127B to go */
+5: srdi r6,r5,4
+ mtocrf 0x01,r6
+
+6: bf cr7*4+1,7f
+err1; ld r0,0(r4)
+err1; ld r6,8(r4)
+err1; ld r7,16(r4)
+err1; ld r8,24(r4)
+err1; ld r9,32(r4)
+err1; ld r10,40(r4)
+err1; ld r11,48(r4)
+err1; ld r12,56(r4)
+ addi r4,r4,64
+err1; std r0,0(r3)
+err1; std r6,8(r3)
+err1; std r7,16(r3)
+err1; std r8,24(r3)
+err1; std r9,32(r3)
+err1; std r10,40(r3)
+err1; std r11,48(r3)
+err1; std r12,56(r3)
+ addi r3,r3,64
+
+ /* Up to 63B to go */
+7: bf cr7*4+2,8f
+err1; ld r0,0(r4)
+err1; ld r6,8(r4)
+err1; ld r7,16(r4)
+err1; ld r8,24(r4)
+ addi r4,r4,32
+err1; std r0,0(r3)
+err1; std r6,8(r3)
+err1; std r7,16(r3)
+err1; std r8,24(r3)
+ addi r3,r3,32
+
+ /* Up to 31B to go */
+8: bf cr7*4+3,9f
+err1; ld r0,0(r4)
+err1; ld r6,8(r4)
+ addi r4,r4,16
+err1; std r0,0(r3)
+err1; std r6,8(r3)
+ addi r3,r3,16
+
+9: clrldi r5,r5,(64-4)
+
+ /* Up to 15B to go */
+.Lshort_copy:
+ mtocrf 0x01,r5
+ bf cr7*4+0,12f
+err1; lwz r0,0(r4) /* Less chance of a reject with word ops */
+err1; lwz r6,4(r4)
+ addi r4,r4,8
+err1; stw r0,0(r3)
+err1; stw r6,4(r3)
+ addi r3,r3,8
+
+12: bf cr7*4+1,13f
+err1; lwz r0,0(r4)
+ addi r4,r4,4
+err1; stw r0,0(r3)
+ addi r3,r3,4
+
+13: bf cr7*4+2,14f
+err1; lhz r0,0(r4)
+ addi r4,r4,2
+err1; sth r0,0(r3)
+ addi r3,r3,2
+
+14: bf cr7*4+3,15f
+err1; lbz r0,0(r4)
+err1; stb r0,0(r3)
+
+15: li r3,0
+ blr
+
+EXPORT_SYMBOL_GPL(memcpy_mcsafe);
--
2.20.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox