* Re: [PATCH] selftests/powerpc: Add a test of SEGV error behaviour
From: John Sperbeck @ 2018-01-02 19:41 UTC (permalink / raw)
To: Michael Ellerman; +Cc: linuxppc-dev, Benjamin Herrenschmidt
In-Reply-To: <20180102110324.31978-1-mpe@ellerman.id.au>
On Tue, Jan 2, 2018 at 3:03 AM, Michael Ellerman <mpe@ellerman.id.au> wrote:
> Add a test case of the error code reported when we take a SEGV on a
> mapped but inaccessible area. We broke this recently.
>
> Based on a test case from John Sperbeck <jsperbeck@google.com>.
>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> ---
> tools/testing/selftests/powerpc/mm/.gitignore | 3 +-
> tools/testing/selftests/powerpc/mm/Makefile | 2 +-
> tools/testing/selftests/powerpc/mm/segv_errors.c | 78 ++++++++++++++++++++++++
> 3 files changed, 81 insertions(+), 2 deletions(-)
> create mode 100644 tools/testing/selftests/powerpc/mm/segv_errors.c
>
> diff --git a/tools/testing/selftests/powerpc/mm/.gitignore b/tools/testing/selftests/powerpc/mm/.gitignore
> index e715a3f2fbf4..7d7c42ed6de9 100644
> --- a/tools/testing/selftests/powerpc/mm/.gitignore
> +++ b/tools/testing/selftests/powerpc/mm/.gitignore
> @@ -1,4 +1,5 @@
> hugetlb_vs_thp_test
> subpage_prot
> tempfile
> -prot_sao
> \ No newline at end of file
> +prot_sao
> +segv_errors
> \ No newline at end of file
> diff --git a/tools/testing/selftests/powerpc/mm/Makefile b/tools/testing/selftests/powerpc/mm/Makefile
> index bf315bcbe663..8ebbe96d80a8 100644
> --- a/tools/testing/selftests/powerpc/mm/Makefile
> +++ b/tools/testing/selftests/powerpc/mm/Makefile
> @@ -2,7 +2,7 @@
> noarg:
> $(MAKE) -C ../
>
> -TEST_GEN_PROGS := hugetlb_vs_thp_test subpage_prot prot_sao
> +TEST_GEN_PROGS := hugetlb_vs_thp_test subpage_prot prot_sao segv_errors
> TEST_GEN_FILES := tempfile
>
> include ../../lib.mk
> diff --git a/tools/testing/selftests/powerpc/mm/segv_errors.c b/tools/testing/selftests/powerpc/mm/segv_errors.c
> new file mode 100644
> index 000000000000..06ae76ee3ea1
> --- /dev/null
> +++ b/tools/testing/selftests/powerpc/mm/segv_errors.c
> @@ -0,0 +1,78 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +/*
> + * Copyright 2017 John Sperbeck
> + *
> + * Test that an access to a mapped but inaccessible area causes a SEGV and
> + * reports si_code == SEGV_ACCERR.
> + */
> +
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <signal.h>
> +#include <sys/mman.h>
> +#include <assert.h>
> +#include <ucontext.h>
> +
> +#include "utils.h"
> +
> +static bool faulted;
> +static int si_code;
> +
> +static void segv_handler(int n, siginfo_t *info, void *ctxt_v)
> +{
> + ucontext_t *ctxt = (ucontext_t *)ctxt_v;
> + struct pt_regs *regs = ctxt->uc_mcontext.regs;
> +
> + faulted = true;
> + si_code = info->si_code;
> + regs->nip += 4;
> +}
> +
> +int test_segv_errors(void)
> +{
> + struct sigaction act = {
> + .sa_sigaction = segv_handler,
> + .sa_flags = SA_SIGINFO,
> + };
> + char c, *p = NULL;
> +
> + p = mmap(NULL, getpagesize(), 0, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
> + FAIL_IF(p == MAP_FAILED);
> +
> + FAIL_IF(sigaction(SIGSEGV, &act, NULL) != 0);
> +
> + faulted = false;
> + si_code = 0;
> +
> + /*
> + * We just need a compiler barrier, but mb() works and has the nice
> + * property of being easy to spot in the disassembly.
> + */
> + mb();
> + c = *p;
> + mb();
> +
> + FAIL_IF(!faulted);
> + FAIL_IF(si_code != SEGV_ACCERR);
> +
> + faulted = false;
> + si_code = 0;
> +
> + mb();
> + *p = c;
> + mb();
> +
> + FAIL_IF(!faulted);
> + FAIL_IF(si_code != SEGV_ACCERR);
> +
> + return 0;
> +}
> +
> +int main(void)
> +{
> + return test_harness(test_segv_errors, "segv_errors");
> +}
> --
> 2.14.3
>
Looks good to me.
Acked-by: John Sperbeck <jsperbeck@google.com>
^ permalink raw reply
* [PATCH v7 00/10] add support for relative references in special sections
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
This adds support for emitting special sections such as initcall arrays,
PCI fixups and tracepoints as relative references rather than absolute
references. This reduces the size by 50% on 64-bit architectures, but
more importantly, it removes the need for carrying relocation metadata
for these sections in relocatables kernels (e.g., for KASLR) that need
to fix up these absolute references at boot time. On arm64, this reduces
the vmlinux footprint of such a reference by 8x (8 byte absolute reference
+ 24 byte RELA entry vs 4 byte relative reference)
Patch #3 was sent out before as a single patch. This series supersedes
the previous submission. This version makes relative ksymtab entries
dependent on the new Kconfig symbol HAVE_ARCH_PREL32_RELOCATIONS rather
than trying to infer from kbuild test robot replies for which architectures
it should be blacklisted.
Patch #1 introduces the new Kconfig symbol HAVE_ARCH_PREL32_RELOCATIONS,
and sets it for the main architectures that are expected to benefit the
most from this feature, i.e., 64-bit architectures or ones that use
runtime relocations.
Patches #4 - #6 implement relative references for initcalls, PCI fixups
and tracepoints, respectively, all of which produce sections with order
~1000 entries on an arm64 defconfig kernel with tracing enabled. This
means we save about 28 KB of vmlinux space for each of these patches.
Patches #7 - #10 have been added in v5, and implement relative references
in jump tables for arm64 and x86. On arm64, this results in significant
space savings (650+ KB on a typical distro kernel). On x86, the savings
are not as impressive, but still worthwhile. (Note that these patches
do not rely on CONFIG_HAVE_ARCH_PREL32_RELOCATIONS, given that the
inline asm that is emitted is already per-arch)
For the arm64 kernel, all patches combined reduce the memory footprint of
vmlinux by about 1.3 MB (using a config copied from Ubuntu that has KASLR
enabled), of which ~1 MB is the size reduction of the RELA section in .init,
and the remaining 300 KB is reduction of .text/.data.
Branch:
git://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git relative-special-sections-v7
Changes since v6:
- drop S390 from patch #1 introducing HAVE_ARCH_PREL32_RELOCATIONS: kbuild
robot threw me some s390 curveballs, and given that s390 does not define
CONFIG_RELOCATABLE in the first place, it does not benefit as much from
relative references as arm64, x86 and power do
- add patch to allow symbol exports to be disabled at compilation unit
granularity (#2)
- get rid of arm64 vmlinux.lds.S hunk to ensure code generated by __ADDRESSABLE
gets discarded from the EFI stub - it is no longer needed after adding #2 (#1)
- change _ADDRESSABLE() to emit a data reference, not a code reference - this
is another simplification made possible by patch #2 (#3)
- add Steven's ack to #6
- split x86 jump_label patch into two (#9, #10)
Changes since v5:
- add missing jump_label prototypes to s390 jump_label.h (#6)
- fix inverted condition in call to jump_entry_is_module_init() (#6)
Changes since v4:
- add patches to convert x86 and arm64 to use relative references for jump
tables (#6 - #8)
- rename PCI patch and add Bjorn's ack (#4)
- rebase onto v4.15-rc5
Changes since v3:
- fix module unload issue in patch #5 reported by Jessica, by reusing the
updated routine for_each_tracepoint_range() for the quiescent check at
module unload time; this requires this routine to be moved before
tracepoint_module_going() in kernel/tracepoint.c
- add Jessica's ack to #2
- rebase onto v4.14-rc1
Changes since v2:
- Revert my slightly misguided attempt to appease checkpatch, which resulted
in needless churn and worse code. This v3 is based on v1 with a few tweaks
that were actually reasonable checkpatch warnings: unnecessary braces (as
pointed out by Ingo) and other minor whitespace misdemeanors.
Changes since v1:
- Remove checkpatch errors to the extent feasible: in some cases, this
involves moving extern declarations into C files, and switching to
struct definitions rather than typedefs. Some errors are impossible
to fix: please find the remaining ones after the diffstat.
- Used 'int' instead if 'signed int' for the various offset fields: there
is no ambiguity between architectures regarding its signedness (unlike
'char')
- Refactor the different patches to be more uniform in the way they define
the section entry type and accessors in the .h file, and avoid the need to
add #ifdefs to the C code.
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Thomas Garnier <thgarnie@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Morris <james.l.morris@oracle.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Nicolas Pitre <nico@linaro.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jessica Yu <jeyu@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: x86@kernel.org
Ard Biesheuvel (10):
arch: enable relative relocations for arm64, power and x86
module: allow symbol exports to be disabled
module: use relative references for __ksymtab entries
init: allow initcall tables to be emitted using relative references
PCI: Add support for relative addressing in quirk tables
kernel: tracepoints: add support for relative references
kernel/jump_label: abstract jump_entry member accessors
arm64/kernel: jump_label: use relative references
x86: jump_label: switch to jump_entry accessors
x86/kernel: jump_table: use relative references
arch/Kconfig | 10 ++++
arch/arm/include/asm/jump_label.h | 27 +++++++++
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/jump_label.h | 48 +++++++++++++---
arch/arm64/kernel/jump_label.c | 22 +++++++-
arch/mips/include/asm/jump_label.h | 27 +++++++++
arch/powerpc/Kconfig | 1 +
arch/powerpc/include/asm/jump_label.h | 27 +++++++++
arch/s390/include/asm/jump_label.h | 27 +++++++++
arch/sparc/include/asm/jump_label.h | 27 +++++++++
arch/tile/include/asm/jump_label.h | 27 +++++++++
arch/x86/Kconfig | 1 +
arch/x86/boot/compressed/kaslr.c | 5 +-
arch/x86/include/asm/Kbuild | 1 +
arch/x86/include/asm/export.h | 5 --
arch/x86/include/asm/jump_label.h | 56 +++++++++++++++----
arch/x86/kernel/jump_label.c | 59 ++++++++++++++------
drivers/firmware/efi/libstub/Makefile | 3 +-
drivers/pci/quirks.c | 13 ++++-
include/asm-generic/export.h | 12 +++-
include/linux/compiler.h | 10 ++++
include/linux/export.h | 55 ++++++++++++++----
include/linux/init.h | 44 +++++++++++----
include/linux/pci.h | 20 +++++++
include/linux/tracepoint.h | 19 +++++--
init/main.c | 32 +++++------
kernel/jump_label.c | 38 ++++++-------
kernel/module.c | 33 +++++++++--
kernel/printk/printk.c | 4 +-
kernel/tracepoint.c | 50 +++++++++--------
security/security.c | 4 +-
tools/objtool/special.c | 4 +-
32 files changed, 560 insertions(+), 152 deletions(-)
delete mode 100644 arch/x86/include/asm/export.h
--
2.11.0
^ permalink raw reply
* [PATCH v7 01/10] arch: enable relative relocations for arm64, power and x86
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
Before updating certain subsystems to use place relative 32-bit
relocations in special sections, to save space and reduce the
number of absolute relocations that need to be processed at runtime
by relocatable kernels, introduce the Kconfig symbol and define it
for some architectures that should be able to support and benefit
from it.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/Kconfig | 10 ++++++++++
arch/arm64/Kconfig | 1 +
arch/powerpc/Kconfig | 1 +
arch/x86/Kconfig | 1 +
4 files changed, 13 insertions(+)
diff --git a/arch/Kconfig b/arch/Kconfig
index 400b9e1b2f27..dbc036a7bd1b 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -959,4 +959,14 @@ config REFCOUNT_FULL
against various use-after-free conditions that can be used in
security flaw exploits.
+config HAVE_ARCH_PREL32_RELOCATIONS
+ bool
+ help
+ May be selected by an architecture if it supports place-relative
+ 32-bit relocations, both in the toolchain and in the module loader,
+ in which case relative references can be used in special sections
+ for PCI fixup, initcalls etc which are only half the size on 64 bit
+ architectures, and don't require runtime relocation on relocatable
+ kernels.
+
source "kernel/gcov/Kconfig"
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index c9a7e9e1414f..66c7b9ab2a3d 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -89,6 +89,7 @@ config ARM64
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
+ select HAVE_ARCH_PREL32_RELOCATIONS
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c51e6ce42e7a..e172478e2ae7 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -177,6 +177,7 @@ config PPC
select HAVE_ARCH_KGDB
select HAVE_ARCH_MMAP_RND_BITS
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
+ select HAVE_ARCH_PREL32_RELOCATIONS
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select ARCH_HAS_STRICT_KERNEL_RWX if ((PPC_BOOK3S_64 || PPC32) && !RELOCATABLE && !HIBERNATION)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index d4fc98c50378..9f2bb853aedb 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -115,6 +115,7 @@ config X86
select HAVE_ARCH_MMAP_RND_BITS if MMU
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT
select HAVE_ARCH_COMPAT_MMAP_BASES if MMU && COMPAT
+ select HAVE_ARCH_PREL32_RELOCATIONS
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
--
2.11.0
^ permalink raw reply related
* [PATCH v7 02/10] module: allow symbol exports to be disabled
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86, matt
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
To allow existing C code to be incorporated into the decompressor or
the UEFI stub, introduce a CPP macro that turns all EXPORT_SYMBOL_xxx
declarations into nops, and #define it in places where such exports
are undesirable. Note that this gets rid of a rather dodgy redefine
of linux/export.h's header guard.
Cc: matt@codeblueprint.co.uk
Cc: keescook@chromium.org
Cc: jeyu@kernel.org
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/x86/boot/compressed/kaslr.c | 5 +----
drivers/firmware/efi/libstub/Makefile | 3 ++-
include/linux/export.h | 9 +++++++++
3 files changed, 12 insertions(+), 5 deletions(-)
diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 8199a6187251..3a2a6d7049e4 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -23,11 +23,8 @@
* _ctype[] in lib/ctype.c is needed by isspace() of linux/ctype.h.
* While both lib/ctype.c and lib/cmdline.c will bring EXPORT_SYMBOL
* which is meaningless and will cause compiling error in some cases.
- * So do not include linux/export.h and define EXPORT_SYMBOL(sym)
- * as empty.
*/
-#define _LINUX_EXPORT_H
-#define EXPORT_SYMBOL(sym)
+#define __DISABLE_EXPORTS
#include "misc.h"
#include "error.h"
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index adaa4a964f0c..312bd0b64a61 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -20,7 +20,8 @@ cflags-$(CONFIG_EFI_ARMSTUB) += -I$(srctree)/scripts/dtc/libfdt
KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \
-D__NO_FORTIFY \
$(call cc-option,-ffreestanding) \
- $(call cc-option,-fno-stack-protector)
+ $(call cc-option,-fno-stack-protector) \
+ -D__DISABLE_EXPORTS
GCOV_PROFILE := n
KASAN_SANITIZE := n
diff --git a/include/linux/export.h b/include/linux/export.h
index 1a1dfdb2a5c6..6dba2fb08f77 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -83,6 +83,15 @@ extern struct module __this_module;
*/
#define __EXPORT_SYMBOL(sym, sec) === __KSYM_##sym ===
+#elif defined(__DISABLE_EXPORTS)
+
+/*
+ * Allow symbol exports to be disabled completely so that C code may
+ * be reused in other execution contexts such as the UEFI stub or the
+ * decompressor.
+ */
+#define __EXPORT_SYMBOL(sym, sec)
+
#elif defined(CONFIG_TRIM_UNUSED_KSYMS)
#include <generated/autoksyms.h>
--
2.11.0
^ permalink raw reply related
* [PATCH v7 03/10] module: use relative references for __ksymtab entries
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86, Ingo Molnar
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
An ordinary arm64 defconfig build has ~64 KB worth of __ksymtab
entries, each consisting of two 64-bit fields containing absolute
references, to the symbol itself and to a char array containing
its name, respectively.
When we build the same configuration with KASLR enabled, we end
up with an additional ~192 KB of relocations in the .init section,
i.e., one 24 byte entry for each absolute reference, which all need
to be processed at boot time.
Given how the struct kernel_symbol that describes each entry is
completely local to module.c (except for the references emitted
by EXPORT_SYMBOL() itself), we can easily modify it to contain
two 32-bit relative references instead. This reduces the size of
the __ksymtab section by 50% for all 64-bit architectures, and
gets rid of the runtime relocations entirely for architectures
implementing KASLR, either via standard PIE linking (arm64) or
using custom host tools (x86).
Note that the binary search involving __ksymtab contents relies
on each section being sorted by symbol name. This is implemented
based on the input section names, not the names in the ksymtab
entries, so this patch does not interfere with that.
Given that the use of place-relative relocations requires support
both in the toolchain and in the module loader, we cannot enable
this feature for all architectures. So make it dependent on whether
CONFIG_HAVE_ARCH_PREL32_RELOCATIONS is defined.
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Thomas Garnier <thgarnie@google.com>
Cc: Nicolas Pitre <nico@linaro.org>
Acked-by: Jessica Yu <jeyu@kernel.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/x86/include/asm/Kbuild | 1 +
arch/x86/include/asm/export.h | 5 ---
include/asm-generic/export.h | 12 ++++-
include/linux/compiler.h | 10 +++++
include/linux/export.h | 46 +++++++++++++++-----
kernel/module.c | 33 +++++++++++---
6 files changed, 83 insertions(+), 24 deletions(-)
diff --git a/arch/x86/include/asm/Kbuild b/arch/x86/include/asm/Kbuild
index 5d6a53fd7521..3e8a88dcaa1d 100644
--- a/arch/x86/include/asm/Kbuild
+++ b/arch/x86/include/asm/Kbuild
@@ -9,5 +9,6 @@ generated-y += xen-hypercalls.h
generic-y += clkdev.h
generic-y += dma-contiguous.h
generic-y += early_ioremap.h
+generic-y += export.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
diff --git a/arch/x86/include/asm/export.h b/arch/x86/include/asm/export.h
deleted file mode 100644
index 2a51d66689c5..000000000000
--- a/arch/x86/include/asm/export.h
+++ /dev/null
@@ -1,5 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifdef CONFIG_64BIT
-#define KSYM_ALIGN 16
-#endif
-#include <asm-generic/export.h>
diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 719db1968d81..97ce606459ae 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -5,12 +5,10 @@
#define KSYM_FUNC(x) x
#endif
#ifdef CONFIG_64BIT
-#define __put .quad
#ifndef KSYM_ALIGN
#define KSYM_ALIGN 8
#endif
#else
-#define __put .long
#ifndef KSYM_ALIGN
#define KSYM_ALIGN 4
#endif
@@ -25,6 +23,16 @@
#define KSYM(name) name
#endif
+.macro __put, val, name
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ .long \val - ., \name - .
+#elif defined(CONFIG_64BIT)
+ .quad \val, \name
+#else
+ .long \val, \name
+#endif
+.endm
+
/*
* note on .section use: @progbits vs %progbits nastiness doesn't matter,
* since we immediately emit into those sections anyway.
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 52e611ab9a6c..79db4aa87d75 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -327,4 +327,14 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s
compiletime_assert(__native_word(t), \
"Need native word sized stores/loads for atomicity.")
+/*
+ * Force the compiler to emit 'sym' as a symbol, so that we can reference
+ * it from inline assembler. Necessary in case 'sym' could be inlined
+ * otherwise, or eliminated entirely due to lack of references that are
+ * visible to the compiler.
+ */
+#define __ADDRESSABLE(sym) \
+ static void * const __attribute__((section(".discard"), used)) \
+ __PASTE(__addressable_##sym, __LINE__) = (void *)&sym;
+
#endif /* __LINUX_COMPILER_H */
diff --git a/include/linux/export.h b/include/linux/export.h
index 6dba2fb08f77..4744cf4736b0 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -24,12 +24,6 @@
#define VMLINUX_SYMBOL_STR(x) __VMLINUX_SYMBOL_STR(x)
#ifndef __ASSEMBLY__
-struct kernel_symbol
-{
- unsigned long value;
- const char *name;
-};
-
#ifdef MODULE
extern struct module __this_module;
#define THIS_MODULE (&__this_module)
@@ -60,17 +54,47 @@ extern struct module __this_module;
#define __CRC_SYMBOL(sym, sec)
#endif
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#include <linux/compiler.h>
+/*
+ * Emit the ksymtab entry as a pair of relative references: this reduces
+ * the size by half on 64-bit architectures, and eliminates the need for
+ * absolute relocations that require runtime processing on relocatable
+ * kernels.
+ */
+#define __KSYMTAB_ENTRY(sym, sec) \
+ __ADDRESSABLE(sym) \
+ asm(" .section \"___ksymtab" sec "+" #sym "\", \"a\" \n" \
+ " .balign 8 \n" \
+ VMLINUX_SYMBOL_STR(__ksymtab_##sym) ": \n" \
+ " .long " VMLINUX_SYMBOL_STR(sym) "- . \n" \
+ " .long " VMLINUX_SYMBOL_STR(__kstrtab_##sym) "- .\n" \
+ " .previous \n")
+
+struct kernel_symbol {
+ int value_offset;
+ int name_offset;
+};
+#else
+#define __KSYMTAB_ENTRY(sym, sec) \
+ static const struct kernel_symbol __ksymtab_##sym \
+ __attribute__((section("___ksymtab" sec "+" #sym), used)) \
+ = { (unsigned long)&sym, __kstrtab_##sym }
+
+struct kernel_symbol {
+ unsigned long value;
+ const char *name;
+};
+#endif
+
/* For every exported symbol, place a struct in the __ksymtab section */
#define ___EXPORT_SYMBOL(sym, sec) \
extern typeof(sym) sym; \
__CRC_SYMBOL(sym, sec) \
static const char __kstrtab_##sym[] \
- __attribute__((section("__ksymtab_strings"), aligned(1))) \
+ __attribute__((section("__ksymtab_strings"), used, aligned(1))) \
= VMLINUX_SYMBOL_STR(sym); \
- static const struct kernel_symbol __ksymtab_##sym \
- __used \
- __attribute__((section("___ksymtab" sec "+" #sym), used)) \
- = { (unsigned long)&sym, __kstrtab_##sym }
+ __KSYMTAB_ENTRY(sym, sec)
#if defined(__KSYM_DEPS__)
diff --git a/kernel/module.c b/kernel/module.c
index dea01ac9cb74..d3a908ffc42c 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -549,12 +549,31 @@ static bool check_symbol(const struct symsearch *syms,
return true;
}
+static unsigned long kernel_symbol_value(const struct kernel_symbol *sym)
+{
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ return (unsigned long)&sym->value_offset + sym->value_offset;
+#else
+ return sym->value;
+#endif
+}
+
+static const char *kernel_symbol_name(const struct kernel_symbol *sym)
+{
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ return (const char *)((unsigned long)&sym->name_offset +
+ sym->name_offset);
+#else
+ return sym->name;
+#endif
+}
+
static int cmp_name(const void *va, const void *vb)
{
const char *a;
const struct kernel_symbol *b;
a = va; b = vb;
- return strcmp(a, b->name);
+ return strcmp(a, kernel_symbol_name(b));
}
static bool find_symbol_in_section(const struct symsearch *syms,
@@ -2198,7 +2217,7 @@ void *__symbol_get(const char *symbol)
sym = NULL;
preempt_enable();
- return sym ? (void *)sym->value : NULL;
+ return sym ? (void *)kernel_symbol_value(sym) : NULL;
}
EXPORT_SYMBOL_GPL(__symbol_get);
@@ -2228,10 +2247,12 @@ static int verify_export_symbols(struct module *mod)
for (i = 0; i < ARRAY_SIZE(arr); i++) {
for (s = arr[i].sym; s < arr[i].sym + arr[i].num; s++) {
- if (find_symbol(s->name, &owner, NULL, true, false)) {
+ if (find_symbol(kernel_symbol_name(s), &owner, NULL,
+ true, false)) {
pr_err("%s: exports duplicate symbol %s"
" (owned by %s)\n",
- mod->name, s->name, module_name(owner));
+ mod->name, kernel_symbol_name(s),
+ module_name(owner));
return -ENOEXEC;
}
}
@@ -2280,7 +2301,7 @@ static int simplify_symbols(struct module *mod, const struct load_info *info)
ksym = resolve_symbol_wait(mod, info, name);
/* Ok if resolved. */
if (ksym && !IS_ERR(ksym)) {
- sym[i].st_value = ksym->value;
+ sym[i].st_value = kernel_symbol_value(ksym);
break;
}
@@ -2540,7 +2561,7 @@ static int is_exported(const char *name, unsigned long value,
ks = lookup_symbol(name, __start___ksymtab, __stop___ksymtab);
else
ks = lookup_symbol(name, mod->syms, mod->syms + mod->num_syms);
- return ks != NULL && ks->value == value;
+ return ks != NULL && kernel_symbol_value(ks) == value;
}
/* As per nm */
--
2.11.0
^ permalink raw reply related
* [PATCH v7 04/10] init: allow initcall tables to be emitted using relative references
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
Allow the initcall tables to be emitted using relative references that
are only half the size on 64-bit architectures and don't require fixups
at runtime on relocatable kernels.
Cc: Petr Mladek <pmladek@suse.com>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: James Morris <james.l.morris@oracle.com>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
include/linux/init.h | 44 +++++++++++++++-----
init/main.c | 32 +++++++-------
kernel/printk/printk.c | 4 +-
security/security.c | 4 +-
4 files changed, 53 insertions(+), 31 deletions(-)
diff --git a/include/linux/init.h b/include/linux/init.h
index ea1b31101d9e..cef8e817e5a5 100644
--- a/include/linux/init.h
+++ b/include/linux/init.h
@@ -109,8 +109,24 @@
typedef int (*initcall_t)(void);
typedef void (*exitcall_t)(void);
-extern initcall_t __con_initcall_start[], __con_initcall_end[];
-extern initcall_t __security_initcall_start[], __security_initcall_end[];
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+typedef int initcall_entry_t;
+
+static inline initcall_t initcall_from_entry(initcall_entry_t *entry)
+{
+ return (initcall_t)((unsigned long)entry + *entry);
+}
+#else
+typedef initcall_t initcall_entry_t;
+
+static inline initcall_t initcall_from_entry(initcall_entry_t *entry)
+{
+ return *entry;
+}
+#endif
+
+extern initcall_entry_t __con_initcall_start[], __con_initcall_end[];
+extern initcall_entry_t __security_initcall_start[], __security_initcall_end[];
/* Used for contructor calls. */
typedef void (*ctor_fn_t)(void);
@@ -160,9 +176,20 @@ extern bool initcall_debug;
* as KEEP() in the linker script.
*/
-#define __define_initcall(fn, id) \
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#define ___define_initcall(fn, id, __sec) \
+ __ADDRESSABLE(fn) \
+ asm(".section \"" #__sec ".init\", \"a\" \n" \
+ "__initcall_" #fn #id ": \n" \
+ ".long " VMLINUX_SYMBOL_STR(fn) " - . \n" \
+ ".previous \n");
+#else
+#define ___define_initcall(fn, id, __sec) \
static initcall_t __initcall_##fn##id __used \
- __attribute__((__section__(".initcall" #id ".init"))) = fn;
+ __attribute__((__section__(#__sec ".init"))) = fn;
+#endif
+
+#define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
/*
* Early initcalls run before initializing SMP.
@@ -201,13 +228,8 @@ extern bool initcall_debug;
#define __exitcall(fn) \
static exitcall_t __exitcall_##fn __exit_call = fn
-#define console_initcall(fn) \
- static initcall_t __initcall_##fn \
- __used __section(.con_initcall.init) = fn
-
-#define security_initcall(fn) \
- static initcall_t __initcall_##fn \
- __used __section(.security_initcall.init) = fn
+#define console_initcall(fn) ___define_initcall(fn,, .con_initcall)
+#define security_initcall(fn) ___define_initcall(fn,, .security_initcall)
struct obs_kernel_param {
const char *str;
diff --git a/init/main.c b/init/main.c
index a8100b954839..d81487cc126d 100644
--- a/init/main.c
+++ b/init/main.c
@@ -848,18 +848,18 @@ int __init_or_module do_one_initcall(initcall_t fn)
}
-extern initcall_t __initcall_start[];
-extern initcall_t __initcall0_start[];
-extern initcall_t __initcall1_start[];
-extern initcall_t __initcall2_start[];
-extern initcall_t __initcall3_start[];
-extern initcall_t __initcall4_start[];
-extern initcall_t __initcall5_start[];
-extern initcall_t __initcall6_start[];
-extern initcall_t __initcall7_start[];
-extern initcall_t __initcall_end[];
-
-static initcall_t *initcall_levels[] __initdata = {
+extern initcall_entry_t __initcall_start[];
+extern initcall_entry_t __initcall0_start[];
+extern initcall_entry_t __initcall1_start[];
+extern initcall_entry_t __initcall2_start[];
+extern initcall_entry_t __initcall3_start[];
+extern initcall_entry_t __initcall4_start[];
+extern initcall_entry_t __initcall5_start[];
+extern initcall_entry_t __initcall6_start[];
+extern initcall_entry_t __initcall7_start[];
+extern initcall_entry_t __initcall_end[];
+
+static initcall_entry_t *initcall_levels[] __initdata = {
__initcall0_start,
__initcall1_start,
__initcall2_start,
@@ -885,7 +885,7 @@ static char *initcall_level_names[] __initdata = {
static void __init do_initcall_level(int level)
{
- initcall_t *fn;
+ initcall_entry_t *fn;
strcpy(initcall_command_line, saved_command_line);
parse_args(initcall_level_names[level],
@@ -895,7 +895,7 @@ static void __init do_initcall_level(int level)
NULL, &repair_env_string);
for (fn = initcall_levels[level]; fn < initcall_levels[level+1]; fn++)
- do_one_initcall(*fn);
+ do_one_initcall(initcall_from_entry(fn));
}
static void __init do_initcalls(void)
@@ -926,10 +926,10 @@ static void __init do_basic_setup(void)
static void __init do_pre_smp_initcalls(void)
{
- initcall_t *fn;
+ initcall_entry_t *fn;
for (fn = __initcall_start; fn < __initcall0_start; fn++)
- do_one_initcall(*fn);
+ do_one_initcall(initcall_from_entry(fn));
}
/*
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index b9006617710f..0516005261c7 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -2611,7 +2611,7 @@ EXPORT_SYMBOL(unregister_console);
*/
void __init console_init(void)
{
- initcall_t *call;
+ initcall_entry_t *call;
/* Setup the default TTY line discipline. */
n_tty_init();
@@ -2622,7 +2622,7 @@ void __init console_init(void)
*/
call = __con_initcall_start;
while (call < __con_initcall_end) {
- (*call)();
+ initcall_from_entry(call)();
call++;
}
}
diff --git a/security/security.c b/security/security.c
index 1cd8526cb0b7..f648eeff06de 100644
--- a/security/security.c
+++ b/security/security.c
@@ -45,10 +45,10 @@ static __initdata char chosen_lsm[SECURITY_NAME_MAX + 1] =
static void __init do_security_initcalls(void)
{
- initcall_t *call;
+ initcall_entry_t *call;
call = __security_initcall_start;
while (call < __security_initcall_end) {
- (*call) ();
+ initcall_from_entry(call)();
call++;
}
}
--
2.11.0
^ permalink raw reply related
* [PATCH v7 05/10] PCI: Add support for relative addressing in quirk tables
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
Allow the PCI quirk tables to be emitted in a way that avoids absolute
references to the hook functions. This reduces the size of the entries,
and, more importantly, makes them invariant under runtime relocation
(e.g., for KASLR)
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
drivers/pci/quirks.c | 13 ++++++++++---
include/linux/pci.h | 20 ++++++++++++++++++++
2 files changed, 30 insertions(+), 3 deletions(-)
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 10684b17d0bd..b6d51b4d5ce1 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3556,9 +3556,16 @@ static void pci_do_fixups(struct pci_dev *dev, struct pci_fixup *f,
f->vendor == (u16) PCI_ANY_ID) &&
(f->device == dev->device ||
f->device == (u16) PCI_ANY_ID)) {
- calltime = fixup_debug_start(dev, f->hook);
- f->hook(dev);
- fixup_debug_report(dev, calltime, f->hook);
+ void (*hook)(struct pci_dev *dev);
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ hook = (void *)((unsigned long)&f->hook_offset +
+ f->hook_offset);
+#else
+ hook = f->hook;
+#endif
+ calltime = fixup_debug_start(dev, hook);
+ hook(dev);
+ fixup_debug_report(dev, calltime, hook);
}
}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index c170c9250c8b..086c3965710b 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1792,7 +1792,11 @@ struct pci_fixup {
u16 device; /* You can use PCI_ANY_ID here of course */
u32 class; /* You can use PCI_ANY_ID here too */
unsigned int class_shift; /* should be 0, 8, 16 */
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+ int hook_offset;
+#else
void (*hook)(struct pci_dev *dev);
+#endif
};
enum pci_fixup_pass {
@@ -1806,12 +1810,28 @@ enum pci_fixup_pass {
pci_fixup_suspend_late, /* pci_device_suspend_late() */
};
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#define __DECLARE_PCI_FIXUP_SECTION(sec, name, vendor, device, class, \
+ class_shift, hook) \
+ __ADDRESSABLE(hook) \
+ asm(".section " #sec ", \"a\" \n" \
+ ".balign 16 \n" \
+ ".short " #vendor ", " #device " \n" \
+ ".long " #class ", " #class_shift " \n" \
+ ".long " VMLINUX_SYMBOL_STR(hook) " - . \n" \
+ ".previous \n");
+#define DECLARE_PCI_FIXUP_SECTION(sec, name, vendor, device, class, \
+ class_shift, hook) \
+ __DECLARE_PCI_FIXUP_SECTION(sec, name, vendor, device, class, \
+ class_shift, hook)
+#else
/* Anonymous variables would be nice... */
#define DECLARE_PCI_FIXUP_SECTION(section, name, vendor, device, class, \
class_shift, hook) \
static const struct pci_fixup __PASTE(__pci_fixup_##name,__LINE__) __used \
__attribute__((__section__(#section), aligned((sizeof(void *))))) \
= { vendor, device, class, class_shift, hook };
+#endif
#define DECLARE_PCI_FIXUP_CLASS_EARLY(vendor, device, class, \
class_shift, hook) \
--
2.11.0
^ permalink raw reply related
* [PATCH v7 06/10] kernel: tracepoints: add support for relative references
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
To avoid the need for relocating absolute references to tracepoint
structures at boot time when running relocatable kernels (which may
take a disproportionate amount of space), add the option to emit
these tables as relative references instead.
Cc: Ingo Molnar <mingo@redhat.com>
Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
include/linux/tracepoint.h | 19 ++++++--
kernel/tracepoint.c | 50 +++++++++++---------
2 files changed, 42 insertions(+), 27 deletions(-)
diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h
index a26ffbe09e71..d02bf1a695e8 100644
--- a/include/linux/tracepoint.h
+++ b/include/linux/tracepoint.h
@@ -228,6 +228,19 @@ extern void syscall_unregfunc(void);
return static_key_false(&__tracepoint_##name.key); \
}
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#define __TRACEPOINT_ENTRY(name) \
+ asm(" .section \"__tracepoints_ptrs\", \"a\" \n" \
+ " .balign 4 \n" \
+ " .long " VMLINUX_SYMBOL_STR(__tracepoint_##name) " - .\n" \
+ " .previous \n")
+#else
+#define __TRACEPOINT_ENTRY(name) \
+ static struct tracepoint * const __tracepoint_ptr_##name __used \
+ __attribute__((section("__tracepoints_ptrs"))) = \
+ &__tracepoint_##name
+#endif
+
/*
* We have no guarantee that gcc and the linker won't up-align the tracepoint
* structures, so we create an array of pointers that will be used for iteration
@@ -237,11 +250,9 @@ extern void syscall_unregfunc(void);
static const char __tpstrtab_##name[] \
__attribute__((section("__tracepoints_strings"))) = #name; \
struct tracepoint __tracepoint_##name \
- __attribute__((section("__tracepoints"))) = \
+ __attribute__((section("__tracepoints"), used)) = \
{ __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
- static struct tracepoint * const __tracepoint_ptr_##name __used \
- __attribute__((section("__tracepoints_ptrs"))) = \
- &__tracepoint_##name;
+ __TRACEPOINT_ENTRY(name);
#define DEFINE_TRACE(name) \
DEFINE_TRACE_FN(name, NULL, NULL);
diff --git a/kernel/tracepoint.c b/kernel/tracepoint.c
index 685c50ae6300..05649fef106c 100644
--- a/kernel/tracepoint.c
+++ b/kernel/tracepoint.c
@@ -327,6 +327,28 @@ int tracepoint_probe_unregister(struct tracepoint *tp, void *probe, void *data)
}
EXPORT_SYMBOL_GPL(tracepoint_probe_unregister);
+static void for_each_tracepoint_range(struct tracepoint * const *begin,
+ struct tracepoint * const *end,
+ void (*fct)(struct tracepoint *tp, void *priv),
+ void *priv)
+{
+ if (!begin)
+ return;
+
+ if (IS_ENABLED(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)) {
+ const int *iter;
+
+ for (iter = (const int *)begin; iter < (const int *)end; iter++)
+ fct((struct tracepoint *)((unsigned long)iter + *iter),
+ priv);
+ } else {
+ struct tracepoint * const *iter;
+
+ for (iter = begin; iter < end; iter++)
+ fct(*iter, priv);
+ }
+}
+
#ifdef CONFIG_MODULES
bool trace_module_has_bad_taint(struct module *mod)
{
@@ -391,15 +413,9 @@ EXPORT_SYMBOL_GPL(unregister_tracepoint_module_notifier);
* Ensure the tracer unregistered the module's probes before the module
* teardown is performed. Prevents leaks of probe and data pointers.
*/
-static void tp_module_going_check_quiescent(struct tracepoint * const *begin,
- struct tracepoint * const *end)
+static void tp_module_going_check_quiescent(struct tracepoint *tp, void *priv)
{
- struct tracepoint * const *iter;
-
- if (!begin)
- return;
- for (iter = begin; iter < end; iter++)
- WARN_ON_ONCE((*iter)->funcs);
+ WARN_ON_ONCE(tp->funcs);
}
static int tracepoint_module_coming(struct module *mod)
@@ -450,8 +466,9 @@ static void tracepoint_module_going(struct module *mod)
* Called the going notifier before checking for
* quiescence.
*/
- tp_module_going_check_quiescent(mod->tracepoints_ptrs,
- mod->tracepoints_ptrs + mod->num_tracepoints);
+ for_each_tracepoint_range(mod->tracepoints_ptrs,
+ mod->tracepoints_ptrs + mod->num_tracepoints,
+ tp_module_going_check_quiescent, NULL);
break;
}
}
@@ -503,19 +520,6 @@ static __init int init_tracepoints(void)
__initcall(init_tracepoints);
#endif /* CONFIG_MODULES */
-static void for_each_tracepoint_range(struct tracepoint * const *begin,
- struct tracepoint * const *end,
- void (*fct)(struct tracepoint *tp, void *priv),
- void *priv)
-{
- struct tracepoint * const *iter;
-
- if (!begin)
- return;
- for (iter = begin; iter < end; iter++)
- fct(*iter, priv);
-}
-
/**
* for_each_kernel_tracepoint - iteration on all kernel tracepoints
* @fct: callback
--
2.11.0
^ permalink raw reply related
* [PATCH v7 07/10] kernel/jump_label: abstract jump_entry member accessors
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
In preparation of allowing architectures to use relative references
in jump_label entries [which can dramatically reduce the memory
footprint], introduce abstractions for references to the 'code' and
'key' members of struct jump_entry.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm/include/asm/jump_label.h | 27 ++++++++++++++
arch/arm64/include/asm/jump_label.h | 27 ++++++++++++++
arch/mips/include/asm/jump_label.h | 27 ++++++++++++++
arch/powerpc/include/asm/jump_label.h | 27 ++++++++++++++
arch/s390/include/asm/jump_label.h | 27 ++++++++++++++
arch/sparc/include/asm/jump_label.h | 27 ++++++++++++++
arch/tile/include/asm/jump_label.h | 27 ++++++++++++++
arch/x86/include/asm/jump_label.h | 27 ++++++++++++++
kernel/jump_label.c | 38 +++++++++-----------
9 files changed, 232 insertions(+), 22 deletions(-)
diff --git a/arch/arm/include/asm/jump_label.h b/arch/arm/include/asm/jump_label.h
index e12d7d096fc0..7b05b404063a 100644
--- a/arch/arm/include/asm/jump_label.h
+++ b/arch/arm/include/asm/jump_label.h
@@ -45,5 +45,32 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h
index 1b5e0e843c3a..9d6e46355c89 100644
--- a/arch/arm64/include/asm/jump_label.h
+++ b/arch/arm64/include/asm/jump_label.h
@@ -62,5 +62,32 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#endif /* __ASSEMBLY__ */
#endif /* __ASM_JUMP_LABEL_H */
diff --git a/arch/mips/include/asm/jump_label.h b/arch/mips/include/asm/jump_label.h
index e77672539e8e..70df9293dc49 100644
--- a/arch/mips/include/asm/jump_label.h
+++ b/arch/mips/include/asm/jump_label.h
@@ -66,5 +66,32 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#endif /* __ASSEMBLY__ */
#endif /* _ASM_MIPS_JUMP_LABEL_H */
diff --git a/arch/powerpc/include/asm/jump_label.h b/arch/powerpc/include/asm/jump_label.h
index 9a287e0ac8b1..412b2699c9f6 100644
--- a/arch/powerpc/include/asm/jump_label.h
+++ b/arch/powerpc/include/asm/jump_label.h
@@ -59,6 +59,33 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#else
#define ARCH_STATIC_BRANCH(LABEL, KEY) \
1098: nop; \
diff --git a/arch/s390/include/asm/jump_label.h b/arch/s390/include/asm/jump_label.h
index 40f651292aa7..1ecfd46835d9 100644
--- a/arch/s390/include/asm/jump_label.h
+++ b/arch/s390/include/asm/jump_label.h
@@ -50,5 +50,32 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/sparc/include/asm/jump_label.h b/arch/sparc/include/asm/jump_label.h
index 94eb529dcb77..18e893687f7c 100644
--- a/arch/sparc/include/asm/jump_label.h
+++ b/arch/sparc/include/asm/jump_label.h
@@ -48,5 +48,32 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/tile/include/asm/jump_label.h b/arch/tile/include/asm/jump_label.h
index cde7573f397b..86acaa6ff33d 100644
--- a/arch/tile/include/asm/jump_label.h
+++ b/arch/tile/include/asm/jump_label.h
@@ -55,4 +55,31 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#endif /* _ASM_TILE_JUMP_LABEL_H */
diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_label.h
index 8c0de4282659..009ff2699d07 100644
--- a/arch/x86/include/asm/jump_label.h
+++ b/arch/x86/include/asm/jump_label.h
@@ -74,6 +74,33 @@ struct jump_entry {
jump_label_t key;
};
+static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
+{
+ return entry->code;
+}
+
+static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
+{
+ return (struct static_key *)((unsigned long)entry->key & ~1UL);
+}
+
+static inline bool jump_entry_is_branch(const struct jump_entry *entry)
+{
+ return (unsigned long)entry->key & 1UL;
+}
+
+static inline bool jump_entry_is_module_init(const struct jump_entry *entry)
+{
+ return entry->code == 0;
+}
+
+static inline void jump_entry_set_module_init(struct jump_entry *entry)
+{
+ entry->code = 0;
+}
+
+#define jump_label_swap NULL
+
#else /* __ASSEMBLY__ */
.macro STATIC_JUMP_IF_TRUE target, key, def
diff --git a/kernel/jump_label.c b/kernel/jump_label.c
index 8594d24e4adc..4f44db58d981 100644
--- a/kernel/jump_label.c
+++ b/kernel/jump_label.c
@@ -37,10 +37,12 @@ static int jump_label_cmp(const void *a, const void *b)
const struct jump_entry *jea = a;
const struct jump_entry *jeb = b;
- if (jea->key < jeb->key)
+ if ((unsigned long)jump_entry_key(jea) <
+ (unsigned long)jump_entry_key(jeb))
return -1;
- if (jea->key > jeb->key)
+ if ((unsigned long)jump_entry_key(jea) >
+ (unsigned long)jump_entry_key(jeb))
return 1;
return 0;
@@ -53,7 +55,8 @@ jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop)
size = (((unsigned long)stop - (unsigned long)start)
/ sizeof(struct jump_entry));
- sort(start, size, sizeof(struct jump_entry), jump_label_cmp, NULL);
+ sort(start, size, sizeof(struct jump_entry), jump_label_cmp,
+ jump_label_swap);
}
static void jump_label_update(struct static_key *key);
@@ -254,8 +257,8 @@ EXPORT_SYMBOL_GPL(jump_label_rate_limit);
static int addr_conflict(struct jump_entry *entry, void *start, void *end)
{
- if (entry->code <= (unsigned long)end &&
- entry->code + JUMP_LABEL_NOP_SIZE > (unsigned long)start)
+ if (jump_entry_code(entry) <= (unsigned long)end &&
+ jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE > (unsigned long)start)
return 1;
return 0;
@@ -314,16 +317,6 @@ static inline void static_key_set_linked(struct static_key *key)
key->type |= JUMP_TYPE_LINKED;
}
-static inline struct static_key *jump_entry_key(struct jump_entry *entry)
-{
- return (struct static_key *)((unsigned long)entry->key & ~1UL);
-}
-
-static bool jump_entry_branch(struct jump_entry *entry)
-{
- return (unsigned long)entry->key & 1UL;
-}
-
/***
* A 'struct static_key' uses a union such that it either points directly
* to a table of 'struct jump_entry' or to a linked list of modules which in
@@ -348,7 +341,7 @@ static enum jump_label_type jump_label_type(struct jump_entry *entry)
{
struct static_key *key = jump_entry_key(entry);
bool enabled = static_key_enabled(key);
- bool branch = jump_entry_branch(entry);
+ bool branch = jump_entry_is_branch(entry);
/* See the comment in linux/jump_label.h */
return enabled ^ branch;
@@ -364,7 +357,8 @@ static void __jump_label_update(struct static_key *key,
* kernel_text_address() verifies we are not in core kernel
* init code, see jump_label_invalidate_module_init().
*/
- if (entry->code && kernel_text_address(entry->code))
+ if (!jump_entry_is_module_init(entry) &&
+ kernel_text_address(jump_entry_code(entry)))
arch_jump_label_transform(entry, jump_label_type(entry));
}
}
@@ -417,7 +411,7 @@ static enum jump_label_type jump_label_init_type(struct jump_entry *entry)
{
struct static_key *key = jump_entry_key(entry);
bool type = static_key_type(key);
- bool branch = jump_entry_branch(entry);
+ bool branch = jump_entry_is_branch(entry);
/* See the comment in linux/jump_label.h */
return type ^ branch;
@@ -541,7 +535,7 @@ static int jump_label_add_module(struct module *mod)
continue;
key = iterk;
- if (within_module(iter->key, mod)) {
+ if (within_module((unsigned long)key, mod)) {
static_key_set_entries(key, iter);
continue;
}
@@ -591,7 +585,7 @@ static void jump_label_del_module(struct module *mod)
key = jump_entry_key(iter);
- if (within_module(iter->key, mod))
+ if (within_module((unsigned long)key, mod))
continue;
/* No memory during module load */
@@ -634,8 +628,8 @@ static void jump_label_invalidate_module_init(struct module *mod)
struct jump_entry *iter;
for (iter = iter_start; iter < iter_stop; iter++) {
- if (within_module_init(iter->code, mod))
- iter->code = 0;
+ if (within_module_init(jump_entry_code(iter), mod))
+ jump_entry_set_module_init(iter);
}
}
--
2.11.0
^ permalink raw reply related
* [PATCH v7 08/10] arm64/kernel: jump_label: use relative references
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
On a randomly chosen distro kernel build for arm64, vmlinux.o shows the
following sections, containing jump label entries, and the associated
RELA relocation records, respectively:
...
[38088] __jump_table PROGBITS 0000000000000000 00e19f30
000000000002ea10 0000000000000000 WA 0 0 8
[38089] .rela__jump_table RELA 0000000000000000 01fd8bb0
000000000008be30 0000000000000018 I 38178 38088 8
...
In other words, we have 190 KB worth of 'struct jump_entry' instances,
and 573 KB worth of RELA entries to relocate each entry's code, target
and key members. This means the RELA section occupies 10% of the .init
segment, and the two sections combined represent 5% of vmlinux's entire
memory footprint.
So let's switch from 64-bit absolute references to 32-bit relative
references: this reduces the size of the __jump_table by 50%, and gets
rid of the RELA section entirely.
Note that this requires some extra care in the sorting routine, given
that the offsets change when entries are moved around in the jump_entry
table.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/include/asm/jump_label.h | 27 ++++++++++++--------
arch/arm64/kernel/jump_label.c | 22 +++++++++++++---
2 files changed, 36 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h
index 9d6e46355c89..8f82adeb7b0b 100644
--- a/arch/arm64/include/asm/jump_label.h
+++ b/arch/arm64/include/asm/jump_label.h
@@ -30,8 +30,8 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran
{
asm goto("1: nop\n\t"
".pushsection __jump_table, \"aw\"\n\t"
- ".align 3\n\t"
- ".quad 1b, %l[l_yes], %c0\n\t"
+ ".align 2\n\t"
+ ".long 1b - ., %l[l_yes] - ., %c0 - .\n\t"
".popsection\n\t"
: : "i"(&((char *)key)[branch]) : : l_yes);
@@ -44,8 +44,8 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool
{
asm goto("1: b %l[l_yes]\n\t"
".pushsection __jump_table, \"aw\"\n\t"
- ".align 3\n\t"
- ".quad 1b, %l[l_yes], %c0\n\t"
+ ".align 2\n\t"
+ ".long 1b - ., %l[l_yes] - ., %c0 - .\n\t"
".popsection\n\t"
: : "i"(&((char *)key)[branch]) : : l_yes);
@@ -57,19 +57,26 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool
typedef u64 jump_label_t;
struct jump_entry {
- jump_label_t code;
- jump_label_t target;
- jump_label_t key;
+ s32 code;
+ s32 target;
+ s32 key;
};
static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
{
- return entry->code;
+ return (unsigned long)&entry->code + entry->code;
+}
+
+static inline jump_label_t jump_entry_target(const struct jump_entry *entry)
+{
+ return (unsigned long)&entry->target + entry->target;
}
static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
{
- return (struct static_key *)((unsigned long)entry->key & ~1UL);
+ unsigned long key = (unsigned long)&entry->key + entry->key;
+
+ return (struct static_key *)(key & ~1UL);
}
static inline bool jump_entry_is_branch(const struct jump_entry *entry)
@@ -87,7 +94,7 @@ static inline void jump_entry_set_module_init(struct jump_entry *entry)
entry->code = 0;
}
-#define jump_label_swap NULL
+void jump_label_swap(void *a, void *b, int size);
#endif /* __ASSEMBLY__ */
#endif /* __ASM_JUMP_LABEL_H */
diff --git a/arch/arm64/kernel/jump_label.c b/arch/arm64/kernel/jump_label.c
index c2dd1ad3e648..2b8e459e91f7 100644
--- a/arch/arm64/kernel/jump_label.c
+++ b/arch/arm64/kernel/jump_label.c
@@ -25,12 +25,12 @@
void arch_jump_label_transform(struct jump_entry *entry,
enum jump_label_type type)
{
- void *addr = (void *)entry->code;
+ void *addr = (void *)jump_entry_code(entry);
u32 insn;
if (type == JUMP_LABEL_JMP) {
- insn = aarch64_insn_gen_branch_imm(entry->code,
- entry->target,
+ insn = aarch64_insn_gen_branch_imm(jump_entry_code(entry),
+ jump_entry_target(entry),
AARCH64_INSN_BRANCH_NOLINK);
} else {
insn = aarch64_insn_gen_nop();
@@ -50,4 +50,20 @@ void arch_jump_label_transform_static(struct jump_entry *entry,
*/
}
+void jump_label_swap(void *a, void *b, int size)
+{
+ long delta = (unsigned long)a - (unsigned long)b;
+ struct jump_entry *jea = a;
+ struct jump_entry *jeb = b;
+ struct jump_entry tmp = *jea;
+
+ jea->code = jeb->code - delta;
+ jea->target = jeb->target - delta;
+ jea->key = jeb->key - delta;
+
+ jeb->code = tmp.code + delta;
+ jeb->target = tmp.target + delta;
+ jeb->key = tmp.key + delta;
+}
+
#endif /* HAVE_JUMP_LABEL */
--
2.11.0
^ permalink raw reply related
* [PATCH v7 09/10] x86: jump_label: switch to jump_entry accessors
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
In preparation of switching x86 to use place-relative references for
the code, target and key members of struct jump_entry, replace direct
references to the struct member with invocations of the new accessors.
This will allow us to make the switch by modifying the accessors only.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/x86/kernel/jump_label.c | 43 ++++++++++++--------
1 file changed, 26 insertions(+), 17 deletions(-)
diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c
index e56c95be2808..d64296092ef5 100644
--- a/arch/x86/kernel/jump_label.c
+++ b/arch/x86/kernel/jump_label.c
@@ -52,22 +52,24 @@ static void __jump_label_transform(struct jump_entry *entry,
* Jump label is enabled for the first time.
* So we expect a default_nop...
*/
- if (unlikely(memcmp((void *)entry->code, default_nop, 5)
- != 0))
- bug_at((void *)entry->code, __LINE__);
+ if (unlikely(memcmp((void *)jump_entry_code(entry),
+ default_nop, 5) != 0))
+ bug_at((void *)jump_entry_code(entry),
+ __LINE__);
} else {
/*
* ...otherwise expect an ideal_nop. Otherwise
* something went horribly wrong.
*/
- if (unlikely(memcmp((void *)entry->code, ideal_nop, 5)
- != 0))
- bug_at((void *)entry->code, __LINE__);
+ if (unlikely(memcmp((void *)jump_entry_code(entry),
+ ideal_nop, 5) != 0))
+ bug_at((void *)jump_entry_code(entry),
+ __LINE__);
}
code.jump = 0xe9;
- code.offset = entry->target -
- (entry->code + JUMP_LABEL_NOP_SIZE);
+ code.offset = jump_entry_target(entry) -
+ (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE);
} else {
/*
* We are disabling this jump label. If it is not what
@@ -76,14 +78,18 @@ static void __jump_label_transform(struct jump_entry *entry,
* are converting the default nop to the ideal nop.
*/
if (init) {
- if (unlikely(memcmp((void *)entry->code, default_nop, 5) != 0))
- bug_at((void *)entry->code, __LINE__);
+ if (unlikely(memcmp((void *)jump_entry_code(entry),
+ default_nop, 5) != 0))
+ bug_at((void *)jump_entry_code(entry),
+ __LINE__);
} else {
code.jump = 0xe9;
- code.offset = entry->target -
- (entry->code + JUMP_LABEL_NOP_SIZE);
- if (unlikely(memcmp((void *)entry->code, &code, 5) != 0))
- bug_at((void *)entry->code, __LINE__);
+ code.offset = jump_entry_target(entry) -
+ (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE);
+ if (unlikely(memcmp((void *)jump_entry_code(entry),
+ &code, 5) != 0))
+ bug_at((void *)jump_entry_code(entry),
+ __LINE__);
}
memcpy(&code, ideal_nops[NOP_ATOMIC5], JUMP_LABEL_NOP_SIZE);
}
@@ -97,10 +103,13 @@ static void __jump_label_transform(struct jump_entry *entry,
*
*/
if (poker)
- (*poker)((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE);
+ (*poker)((void *)jump_entry_code(entry), &code,
+ JUMP_LABEL_NOP_SIZE);
else
- text_poke_bp((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE,
- (void *)entry->code + JUMP_LABEL_NOP_SIZE);
+ text_poke_bp((void *)jump_entry_code(entry), &code,
+ JUMP_LABEL_NOP_SIZE,
+ (void *)jump_entry_code(entry) +
+ JUMP_LABEL_NOP_SIZE);
}
void arch_jump_label_transform(struct jump_entry *entry,
--
2.11.0
^ permalink raw reply related
* [PATCH v7 10/10] x86/kernel: jump_table: use relative references
From: Ard Biesheuvel @ 2018-01-02 20:05 UTC (permalink / raw)
To: linux-kernel
Cc: Ard Biesheuvel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Nicolas Pitre, Josh Poimboeuf,
Steven Rostedt, Martin Schwidefsky, Sergey Senozhatsky,
Linus Torvalds, Jessica Yu, linux-arm-kernel, linux-mips,
linuxppc-dev, linux-s390, sparclinux, x86
In-Reply-To: <20180102200549.22984-1-ard.biesheuvel@linaro.org>
Similar to the arm64 case, 64-bit x86 can benefit from using 32-bit
relative references rather than 64-bit absolute ones when emitting
struct jump_entry instances. Not only does this reduce the memory
footprint of the entries themselves by 50%, it also removes the need
for carrying relocation metadata on relocatable builds (i.e., for KASLR)
which saves a fair chunk of .init space as well (although the savings
are not as dramatic as on arm64)
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/x86/include/asm/jump_label.h | 35 ++++++++++++--------
arch/x86/kernel/jump_label.c | 16 +++++++++
tools/objtool/special.c | 4 +--
3 files changed, 39 insertions(+), 16 deletions(-)
diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_label.h
index 009ff2699d07..35fc2c5ec846 100644
--- a/arch/x86/include/asm/jump_label.h
+++ b/arch/x86/include/asm/jump_label.h
@@ -36,8 +36,8 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran
asm_volatile_goto("1:"
".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t"
".pushsection __jump_table, \"aw\" \n\t"
- _ASM_ALIGN "\n\t"
- _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t"
+ ".balign 4\n\t"
+ ".long 1b - ., %l[l_yes] - ., %c0 + %c1 - .\n\t"
".popsection \n\t"
: : "i" (key), "i" (branch) : : l_yes);
@@ -52,8 +52,8 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool
".byte 0xe9\n\t .long %l[l_yes] - 2f\n\t"
"2:\n\t"
".pushsection __jump_table, \"aw\" \n\t"
- _ASM_ALIGN "\n\t"
- _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t"
+ ".balign 4\n\t"
+ ".long 1b - ., %l[l_yes] - ., %c0 + %c1 - .\n\t"
".popsection \n\t"
: : "i" (key), "i" (branch) : : l_yes);
@@ -69,19 +69,26 @@ typedef u32 jump_label_t;
#endif
struct jump_entry {
- jump_label_t code;
- jump_label_t target;
- jump_label_t key;
+ s32 code;
+ s32 target;
+ s32 key;
};
static inline jump_label_t jump_entry_code(const struct jump_entry *entry)
{
- return entry->code;
+ return (unsigned long)&entry->code + entry->code;
+}
+
+static inline jump_label_t jump_entry_target(const struct jump_entry *entry)
+{
+ return (unsigned long)&entry->target + entry->target;
}
static inline struct static_key *jump_entry_key(const struct jump_entry *entry)
{
- return (struct static_key *)((unsigned long)entry->key & ~1UL);
+ unsigned long key = (unsigned long)&entry->key + entry->key;
+
+ return (struct static_key *)(key & ~1UL);
}
static inline bool jump_entry_is_branch(const struct jump_entry *entry)
@@ -99,7 +106,7 @@ static inline void jump_entry_set_module_init(struct jump_entry *entry)
entry->code = 0;
}
-#define jump_label_swap NULL
+void jump_label_swap(void *a, void *b, int size);
#else /* __ASSEMBLY__ */
@@ -114,8 +121,8 @@ static inline void jump_entry_set_module_init(struct jump_entry *entry)
.byte STATIC_KEY_INIT_NOP
.endif
.pushsection __jump_table, "aw"
- _ASM_ALIGN
- _ASM_PTR .Lstatic_jump_\@, \target, \key
+ .balign 4
+ .long .Lstatic_jump_\@ - ., \target - ., \key - .
.popsection
.endm
@@ -130,8 +137,8 @@ static inline void jump_entry_set_module_init(struct jump_entry *entry)
.Lstatic_jump_after_\@:
.endif
.pushsection __jump_table, "aw"
- _ASM_ALIGN
- _ASM_PTR .Lstatic_jump_\@, \target, \key + 1
+ .balign 4
+ .long .Lstatic_jump_\@ - ., \target - ., \key + 1 - .
.popsection
.endm
diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c
index d64296092ef5..cc5034b42335 100644
--- a/arch/x86/kernel/jump_label.c
+++ b/arch/x86/kernel/jump_label.c
@@ -149,4 +149,20 @@ __init_or_module void arch_jump_label_transform_static(struct jump_entry *entry,
__jump_label_transform(entry, type, text_poke_early, 1);
}
+void jump_label_swap(void *a, void *b, int size)
+{
+ long delta = (unsigned long)a - (unsigned long)b;
+ struct jump_entry *jea = a;
+ struct jump_entry *jeb = b;
+ struct jump_entry tmp = *jea;
+
+ jea->code = jeb->code - delta;
+ jea->target = jeb->target - delta;
+ jea->key = jeb->key - delta;
+
+ jeb->code = tmp.code + delta;
+ jeb->target = tmp.target + delta;
+ jeb->key = tmp.key + delta;
+}
+
#endif
diff --git a/tools/objtool/special.c b/tools/objtool/special.c
index 84f001d52322..98ae55b39037 100644
--- a/tools/objtool/special.c
+++ b/tools/objtool/special.c
@@ -30,9 +30,9 @@
#define EX_ORIG_OFFSET 0
#define EX_NEW_OFFSET 4
-#define JUMP_ENTRY_SIZE 24
+#define JUMP_ENTRY_SIZE 12
#define JUMP_ORIG_OFFSET 0
-#define JUMP_NEW_OFFSET 8
+#define JUMP_NEW_OFFSET 4
#define ALT_ENTRY_SIZE 13
#define ALT_ORIG_OFFSET 0
--
2.11.0
^ permalink raw reply related
* Re: [RFC PATCH 2/2] KVM: PPC: Book3S HV: Work around transactional memory bugs in POWER9
From: Suraj Jitindar Singh @ 2018-01-02 23:15 UTC (permalink / raw)
To: Paul Mackerras, linuxppc-dev, kvm, kvm-ppc; +Cc: David Gibson
In-Reply-To: <20171208061113.sm2cuug2uypdduw5@rohan>
On Fri, 2017-12-08 at 17:11 +1100, Paul Mackerras wrote:
> POWER9 has hardware bugs relating to transactional memory and thread
> reconfiguration (changes to hardware SMT mode). Specifically, the
> core
> does not have enough storage to store a complete checkpoint of all
> the
> architected state for all four threads. The DD2.2 version of POWER9
> includes hardware modifications designed to allow hypervisor software
> to implement workarounds for these problems. This patch implements
> those workarounds in KVM code so that KVM guests see a full, working
> transactional memory implementation.
>
> The problems center around the use of TM suspended state, where the
> CPU has a checkpointed state but execution is not transactional. The
> workaround is to implement a "fake suspend" state, which looks to the
> guest like suspended state but the CPU does not store a checkpoint.
> In this state, any instruction that would cause a transition to
> transactional state (rfid, rfebb, mtmsrd, tresume) or would use the
> checkpointed state (treclaim) causes a "soft patch" interrupt (vector
> 0x1500) to the hypervisor so that it can be emulated. The trechkpt
> instruction also causes a soft patch interrupt.
>
> On POWER9 DD2.2, we avoid returning to the guest in any state which
> would require a checkpoint to be present. The trechkpt in the guest
> entry path which would normally create that checkpoint is replaced by
> either a transition to fake suspend state, if the guest is in suspend
> state, or a rollback to the pre-transactional state if the guest is
> in
> transactional state. Fake suspend state is indicated by a flag in
> the
> PACA plus a new bit in the PSSCR. The new PSSCR bit is write-only
> and
> reads back as 0.
>
> On exit from the guest, if the guest is in fake suspend state, we
> still
> do the treclaim instruction as we would in real suspend state, in
> order
> to get into non-transactional state, but we do not save the resulting
> register state since there was no checkpoint.
>
> Emulation of the instructions that cause a softpath interrupt is
> handled
> in two paths. If the guest is in real suspend mode, we call
> kvmhv_p9_tm_emulation_early() to handle the cases where the guest is
> transitioning to transactional state. This is called before we do
> the treclaim in the guest exit path; because we haven't done
> treclaim,
> we can get back to the guest with the transaction still active.
> If the instruction is a case that kvmhv_p9_tm_emulation_early()
> doesn't
> handle, or if the guest is in fake suspend state, then we proceed to
> do the complete guest exit path and subsequently call
> kvmhv_p9_tm_emulation() in host context with the MMU on. This
> handles all the cases including the cases that generate program
> interrupts (illegal instruction or TM Bad Thing) and facility
> unavailable interrupts.
>
> The emulation is reasonably straightforward and is mostly concerned
> with checking for exception conditions and updating the state of
> registers such as MSR and CR0. The treclaim emulation takes care to
> ensure that the TEXASR register gets updated as if it were the guest
> treclaim instruction that had done failure recording, not the
> treclaim
> done in hypervisor state in the guest exit path.
>
> Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
>
With the following patch applied on top of the TM emulation code I was
able to get at least a basic test to run on the guest on real hardware.
[snip]
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index c7fe377ff6bc..adf2da6b2211 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -3049,6 +3049,7 @@ BEGIN_FTR_SECTION
li r0, PSSCR_FAKE_SUSPEND
andc r3, r3, r0
mtspr SPRN_PSSCR, r3
+ ld r9, HSTATE_KVM_VCPU(r13)
b 1f
2:
END_FTR_SECTION_IFSET(CPU_FTR_P9_TM_EMUL)
@@ -3273,8 +3274,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_P9_TM_EMUL)
b 9b /* and return */
10: stdu r1, -PPC_MIN_STKFRM(r1)
/* guest is in transactional state, so simulate rollback */
+ mr r3, r4
bl kvmhv_emulate_tm_rollback
nop
+ ld r4, HSTATE_KVM_VCPU(r13) /* our vcpu pointer has been
trashed */
addi r1, r1, PPC_MIN_STKFRM
b 9b
#endif
^ permalink raw reply related
* Re: [PATCH v1 00/15] ASoC: fsl_ssi: Clean up - program flow level
From: Caleb Crome @ 2018-01-02 23:28 UTC (permalink / raw)
To: Nicolin Chen
Cc: Timur Tabi, Mark Brown, linux-kernel, linuxppc-dev, alsa-devel,
Liam Girdwood, Fabio Estevam, mail, Arnaud Mouiche, lukma,
Sascha Hauer
In-Reply-To: <1513702819-42310-1-git-send-email-nicoleotsuka@gmail.com>
On Tue, Dec 19, 2017 at 9:00 AM, Nicolin Chen <nicoleotsuka@gmail.com> wrote:
>
> ==Background==
> The fsl_ssi driver was designed for PPC originally and then it has
> been updated to support different modes for i.MX Series, including
> SDMA, I2S Master mode, AC97 and older i.MXs with FIQ, by different
> contributors for different use cases in different coding styles.
>
> Additionally, in order to fix/work-around hardware bugs and design
> flaws, the driver made a lot of compromise so now its program flow
> looks very complicated and it's getting hard to maintain or update.
>
> So I am going to clean up the driver on both coding style level and
> program flow level.
>
> ==Introduction==
> This series of patches is the second set to clean up fsl_ssi driver
> in the program flow level. Any patch here may impact a fundamental
> test case like playback or record.
>
> ==Verification==
> This series of patches require fully tested. I have done such tests
> on i.MX6SoloX with WM8962 using imx_v6_v7_defconfig as:
> - Playback via I2S Master and Slave mode
> - Record via I2S Master and Slave mode
> - Simultaneous playback and record via I2S Master and Slave mode
> - Background playback with foreground record (starting at different
> time) via I2S Master and Slave mode
> - Background record with foreground playback (starting at different
> time) via I2S Master and Slave mode
> * All tests above by hacking offline_config to true in imx51.
>
> Example of uncovered tests: TDM, AC97, PowerPC and FIQ.
>
> Nicolin Chen (15):
> ASoC: fsl_ssi: Clean up set_dai_tdm_slot()
> ASoC: fsl_ssi: Maintain a mask of active streams
> ASoC: fsl_ssi: Rename fsl_ssi_disable_val macro
> ASoC: fsl_ssi: Clear FIFO directly in fsl_ssi_config()
> ASoC: fsl_ssi: Clean up helper functions of trigger()
> ASoC: fsl_ssi: Add DAIFMT define for AC97
> ASoC: fsl_ssi: Clean up fsl_ssi_setup_regvals()
> ASoC: fsl_ssi: Set xFEN0 and xFEN1 together
> ASoC: fsl_ssi: Use snd_soc_init_dma_data instead
> ASoC: fsl_ssi: Move one-time configurations to dai_probe()
> ASoC: fsl_ssi: Setup AC97 in dai_probe()
> ASoC: fsl_ssi: Clean up _fsl_ssi_set_dai_fmt()
> ASoC: fsl_ssi: Remove cpu_dai_drv from fsl_ssi structure
> ASoC: fsl_ssi: Move DT related code to a separate probe()
> ASoC: fsl_ssi: Use ssi->streams instead of reading register
>
> sound/soc/fsl/fsl_ssi.c | 710 ++++++++++++++++++++++++------------------------
> 1 file changed, 348 insertions(+), 362 deletions(-)
>
> --
> 2.7.4
>
tested this patch set on MX6 SSI against broonie for-next (4.15-rc5),
no problems.
Do I send a separate Tested-by for each patch, or just the 00/15 one?
Tested-by: Caleb Crome <caleb@crome.org>
^ permalink raw reply
* Re: [PATCH v7 02/10] module: allow symbol exports to be disabled
From: Nicolas Pitre @ 2018-01-02 23:47 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: linux-kernel, H. Peter Anvin, Ralf Baechle, Arnd Bergmann,
Heiko Carstens, Kees Cook, Will Deacon, Michael Ellerman,
Thomas Garnier, Thomas Gleixner, Serge E. Hallyn, Bjorn Helgaas,
Benjamin Herrenschmidt, Russell King, Paul Mackerras,
Catalin Marinas, David S. Miller, Petr Mladek, Ingo Molnar,
James Morris, Andrew Morton, Josh Poimboeuf, Steven Rostedt,
Martin Schwidefsky, Sergey Senozhatsky, Linus Torvalds,
Jessica Yu, linux-arm-kernel, linux-mips, linuxppc-dev,
linux-s390, sparclinux, x86, matt
In-Reply-To: <20180102200549.22984-3-ard.biesheuvel@linaro.org>
On Tue, 2 Jan 2018, Ard Biesheuvel wrote:
> To allow existing C code to be incorporated into the decompressor or
> the UEFI stub, introduce a CPP macro that turns all EXPORT_SYMBOL_xxx
> declarations into nops, and #define it in places where such exports
> are undesirable. Note that this gets rid of a rather dodgy redefine
> of linux/export.h's header guard.
[...]
> --- a/include/linux/export.h
> +++ b/include/linux/export.h
> @@ -83,6 +83,15 @@ extern struct module __this_module;
> */
> #define __EXPORT_SYMBOL(sym, sec) === __KSYM_##sym ===
>
> +#elif defined(__DISABLE_EXPORTS)
> +
> +/*
> + * Allow symbol exports to be disabled completely so that C code may
> + * be reused in other execution contexts such as the UEFI stub or the
> + * decompressor.
> + */
> +#define __EXPORT_SYMBOL(sym, sec)
> +
I think you should rather put this first thing in the #if sequence so to
override the defined(__KSYM_DEPS__) case too. No need to create build
dependencies for module symbols that you're going to stub out
afterwards anyway.
Nicolas
^ permalink raw reply
* Re: [PATCH v7 02/10] module: allow symbol exports to be disabled
From: Ard Biesheuvel @ 2018-01-02 23:55 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Linux Kernel Mailing List, H. Peter Anvin, Ralf Baechle,
Arnd Bergmann, Heiko Carstens, Kees Cook, Will Deacon,
Michael Ellerman, Thomas Garnier, Thomas Gleixner,
Serge E. Hallyn, Bjorn Helgaas, Benjamin Herrenschmidt,
Russell King, Paul Mackerras, Catalin Marinas, David S. Miller,
Petr Mladek, Ingo Molnar, James Morris, Andrew Morton,
Josh Poimboeuf, Steven Rostedt, Martin Schwidefsky,
Sergey Senozhatsky, Linus Torvalds, Jessica Yu, linux-arm-kernel,
linux-mips, linuxppc-dev, linux-s390, sparclinux,
the arch/x86 maintainers, Matt Fleming
In-Reply-To: <nycvar.YSQ.7.76.1801021841410.8567@knanqh.ubzr>
On 2 January 2018 at 23:47, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Tue, 2 Jan 2018, Ard Biesheuvel wrote:
>
>> To allow existing C code to be incorporated into the decompressor or
>> the UEFI stub, introduce a CPP macro that turns all EXPORT_SYMBOL_xxx
>> declarations into nops, and #define it in places where such exports
>> are undesirable. Note that this gets rid of a rather dodgy redefine
>> of linux/export.h's header guard.
> [...]
>
>> --- a/include/linux/export.h
>> +++ b/include/linux/export.h
>> @@ -83,6 +83,15 @@ extern struct module __this_module;
>> */
>> #define __EXPORT_SYMBOL(sym, sec) === __KSYM_##sym ===
>>
>> +#elif defined(__DISABLE_EXPORTS)
>> +
>> +/*
>> + * Allow symbol exports to be disabled completely so that C code may
>> + * be reused in other execution contexts such as the UEFI stub or the
>> + * decompressor.
>> + */
>> +#define __EXPORT_SYMBOL(sym, sec)
>> +
>
> I think you should rather put this first thing in the #if sequence so to
> override the defined(__KSYM_DEPS__) case too. No need to create build
> dependencies for module symbols that you're going to stub out
> afterwards anyway.
>
I wasn't sure, so thanks for clearing that up.
^ permalink raw reply
* Re: [PATCH 2/2] powerpc/pseries,ps3: panic flush kernel messages before halting system
From: David Gibson @ 2018-01-03 0:49 UTC (permalink / raw)
To: Nicholas Piggin; +Cc: linuxppc-dev
In-Reply-To: <20171223164923.10587-3-npiggin@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 6301 bytes --]
On Sun, Dec 24, 2017 at 02:49:23AM +1000, Nicholas Piggin wrote:
> Platforms with a panic handler that halts the system can have problems
> getting kernel messages out, because the panic notifiers are called
> before kernel/panic.c does its flushing of printk buffers an console
> etc.
>
> This was attempted to be solved with commit a3b2cb30f252 ("powerpc: Do
> not call ppc_md.panic in fadump panic notifier"), but that wasn't the
> right approach and caused other problems, and was reverted by commit
> ab9dbf771ff9.
>
> Instead, the powernv shutdown paths have already had a similar
> problem, fixed by taking the message flushing sequence from
> kernel/panic.c. That's a little bit ugly, but while we have the code
> duplicated, it will work for this case as well. So have ppc panic
> handlers do the same flushing before they terminate.
>
> Without this patch, a qemu pseries_le_defconfig guest stops silently
> when issued the nmi command when xmon is off and no crash dumpers
> enabled. Afterwards, an oops is printed by each CPU as expected.
>
> Fixes: ab9dbf771ff9 ("Revert "powerpc: Do not call ppc_md.panic in fadump panic notifier"")
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> arch/powerpc/include/asm/bug.h | 3 ++-
> arch/powerpc/kernel/traps.c | 24 ++++++++++++++++++++++++
> arch/powerpc/platforms/powernv/opal.c | 18 ++++--------------
> arch/powerpc/platforms/ps3/setup.c | 1 +
> arch/powerpc/platforms/pseries/setup.c | 8 +++++++-
> 5 files changed, 38 insertions(+), 16 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/bug.h b/arch/powerpc/include/asm/bug.h
> index 3c04249bcf39..bca101ee1f32 100644
> --- a/arch/powerpc/include/asm/bug.h
> +++ b/arch/powerpc/include/asm/bug.h
> @@ -135,7 +135,8 @@ extern void bad_page_fault(struct pt_regs *, unsigned long, int);
> extern void _exception(int, struct pt_regs *, int, unsigned long);
> extern void die(const char *, struct pt_regs *, long);
> extern bool die_will_crash(void);
> -
> +extern void panic_flush_kmsg_start(void);
> +extern void panic_flush_kmsg_end(void);
> #endif /* !__ASSEMBLY__ */
>
> #endif /* __KERNEL__ */
> diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
> index 109989676776..37c1ea9b0642 100644
> --- a/arch/powerpc/kernel/traps.c
> +++ b/arch/powerpc/kernel/traps.c
> @@ -38,6 +38,8 @@
> #include <linux/ratelimit.h>
> #include <linux/context_tracking.h>
> #include <linux/smp.h>
> +#include <linux/console.h>
> +#include <linux/kmsg_dump.h>
>
> #include <asm/emulated_ops.h>
> #include <asm/pgtable.h>
> @@ -142,6 +144,28 @@ static int die_owner = -1;
> static unsigned int die_nest_count;
> static int die_counter;
>
> +extern void panic_flush_kmsg_start(void)
> +{
> + /*
> + * These are mostly taken from kernel/panic.c, but tries to do
> + * relatively minimal work. Don't use delay functions (TB may
> + * be broken), don't crash dump (need to set a firmware log),
> + * don't run notifiers. We do want to get some information to
> + * Linux console.
> + */
> + console_verbose();
> + bust_spinlocks(1);
> +}
> +
> +extern void panic_flush_kmsg_end(void)
> +{
> + printk_safe_flush_on_panic();
> + kmsg_dump(KMSG_DUMP_PANIC);
> + bust_spinlocks(0);
> + debug_locks_off();
> + console_flush_on_panic();
> +}
> +
> static unsigned long oops_begin(struct pt_regs *regs)
> {
> int cpu;
> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
> index 69b5263fc9e3..c15182765ff5 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -461,24 +461,14 @@ static int opal_recover_mce(struct pt_regs *regs,
>
> void pnv_platform_error_reboot(struct pt_regs *regs, const char *msg)
> {
> - /*
> - * This is mostly taken from kernel/panic.c, but tries to do
> - * relatively minimal work. Don't use delay functions (TB may
> - * be broken), don't crash dump (need to set a firmware log),
> - * don't run notifiers. We do want to get some information to
> - * Linux console.
> - */
> - console_verbose();
> - bust_spinlocks(1);
> + panic_flush_kmsg_start();
> +
> pr_emerg("Hardware platform error: %s\n", msg);
> if (regs)
> show_regs(regs);
> smp_send_stop();
> - printk_safe_flush_on_panic();
> - kmsg_dump(KMSG_DUMP_PANIC);
> - bust_spinlocks(0);
> - debug_locks_off();
> - console_flush_on_panic();
> +
> + panic_flush_kmsg_end();
>
> /*
> * Don't bother to shut things down because this will
> diff --git a/arch/powerpc/platforms/ps3/setup.c b/arch/powerpc/platforms/ps3/setup.c
> index 6244bc849469..77a37520068d 100644
> --- a/arch/powerpc/platforms/ps3/setup.c
> +++ b/arch/powerpc/platforms/ps3/setup.c
> @@ -113,6 +113,7 @@ static void ps3_panic(char *str)
> printk(" System does not reboot automatically.\n");
> printk(" Please press POWER button.\n");
> printk("\n");
> + panic_flush_kmsg_end();
>
> while(1)
> lv1_pause(1);
> diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
> index 28b286df0e91..a65843059c38 100644
> --- a/arch/powerpc/platforms/pseries/setup.c
> +++ b/arch/powerpc/platforms/pseries/setup.c
> @@ -498,6 +498,12 @@ static void __init pSeries_setup_arch(void)
> ppc_md.pcibios_root_bridge_prepare = pseries_root_bridge_prepare;
> }
>
> +static void pSeries_panic(char *str)
> +{
> + panic_flush_kmsg_end();
> + rtas_os_term(str);
> +}
> +
> static int __init pSeries_init_panel(void)
> {
> /* Manually leave the kernel version on the panel. */
> @@ -726,7 +732,7 @@ define_machine(pseries) {
> .pcibios_fixup = pSeries_final_fixup,
> .restart = rtas_restart,
> .halt = rtas_halt,
> - .panic = rtas_os_term,
> + .panic = pSeries_panic,
> .get_boot_time = rtas_get_boot_time,
> .get_rtc_time = rtas_get_rtc_time,
> .set_rtc_time = rtas_set_rtc_time,
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH 1/2] powerpc: System reset avoid interleaving oops using die synchronisation
From: David Gibson @ 2018-01-03 0:49 UTC (permalink / raw)
To: Nicholas Piggin; +Cc: linuxppc-dev
In-Reply-To: <20171223164923.10587-2-npiggin@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1744 bytes --]
On Sun, Dec 24, 2017 at 02:49:22AM +1000, Nicholas Piggin wrote:
> The die() oops path contains a serializing lock to prevent oops
> messages from being interleaved. In the case of a system reset
> initiated oops (e.g., qemu nmi command), __die was being called
> which lacks that synchronisation and oops reports could be
> interleaved across CPUs.
>
> A recent patch 4388c9b3a6ee7 ("powerpc: Do not send system reset
> request through the oops path") changed this to __die to avoid
> the debugger() call, but there is no real harm to calling it twice
> if the first time fell through. So go back to using die() here.
> This was observed to fix the problem.
>
> Fixes: 4388c9b3a6ee7 ("powerpc: Do not send system reset request through the oops path")
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
> ---
> arch/powerpc/kernel/traps.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
> index f3eb61be0d30..109989676776 100644
> --- a/arch/powerpc/kernel/traps.c
> +++ b/arch/powerpc/kernel/traps.c
> @@ -337,7 +337,7 @@ void system_reset_exception(struct pt_regs *regs)
> * No debugger or crash dump registered, print logs then
> * panic.
> */
> - __die("System Reset", regs, SIGABRT);
> + die("System Reset", regs, SIGABRT);
>
> mdelay(2*MSEC_PER_SEC); /* Wait a little while for others to print */
> add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE);
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH 01/13] powerpc/powernv: Introduce new PHB type for opencapi links
From: Andrew Donnellan @ 2018-01-03 3:53 UTC (permalink / raw)
To: Frederic Barrat, linuxppc-dev, linux-kernel; +Cc: arnd, gregkh, mpe, alastair
In-Reply-To: <6870a754d346e7d8a47325e5ba4327853aea9226.1513608243.git.fbarrat@linux.vnet.ibm.com>
On 19/12/17 02:21, Frederic Barrat wrote:
> The NPU was already abstracted by opal as a virtual PHB for nvlink,
> but it helps to be able to differentiate between a nvlink or opencapi
> PHB, as it's not completely transparent to linux. In particular, PE
> assignment differs and we'll also need the information in later
> patches.
>
> So rename existing PNV_PHB_NPU type to PNV_PHB_NPU_NVLINK and add a
> new type PNV_PHB_NPU_OCAPI.
>
> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
> Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> ---
> arch/powerpc/platforms/powernv/npu-dma.c | 2 +-
> arch/powerpc/platforms/powernv/pci-ioda.c | 46 +++++++++++++++++++++++--------
> arch/powerpc/platforms/powernv/pci.c | 4 +++
> arch/powerpc/platforms/powernv/pci.h | 8 ++++--
> 4 files changed, 45 insertions(+), 15 deletions(-)
>
> diff --git a/arch/powerpc/platforms/powernv/npu-dma.c b/arch/powerpc/platforms/powernv/npu-dma.c
> index f6cbc1a71472..c5899c107d59 100644
> --- a/arch/powerpc/platforms/powernv/npu-dma.c
> +++ b/arch/powerpc/platforms/powernv/npu-dma.c
> @@ -277,7 +277,7 @@ static int pnv_npu_dma_set_bypass(struct pnv_ioda_pe *npe)
> int64_t rc = 0;
> phys_addr_t top = memblock_end_of_DRAM();
>
> - if (phb->type != PNV_PHB_NPU || !npe->pdev)
> + if (phb->type != PNV_PHB_NPU_NVLINK || !npe->pdev)
> return -EINVAL;
>
> rc = pnv_npu_unset_window(npe, 0);
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> index 749055553064..c37b5d288f9c 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -54,7 +54,8 @@
> #define POWERNV_IOMMU_DEFAULT_LEVELS 1
> #define POWERNV_IOMMU_MAX_LEVELS 5
>
> -static const char * const pnv_phb_names[] = { "IODA1", "IODA2", "NPU" };
> +static const char * const pnv_phb_names[] = { "IODA1", "IODA2", "NPU_NVLINK",
> + "NPU_OCAPI" };
> static void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl);
>
> void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
> @@ -924,7 +925,7 @@ static int pnv_ioda_configure_pe(struct pnv_phb *phb, struct pnv_ioda_pe *pe)
> * Configure PELTV. NPUs don't have a PELTV table so skip
> * configuration on them.
> */
> - if (phb->type != PNV_PHB_NPU)
> + if (phb->type != PNV_PHB_NPU_NVLINK && phb->type != PNV_PHB_NPU_OCAPI)
> pnv_ioda_set_peltv(phb, pe, true);
>
> /* Setup reverse map */
> @@ -1260,12 +1261,13 @@ static struct pnv_ioda_pe *pnv_ioda_setup_npu_PE(struct pci_dev *npu_pdev)
> return pe;
> }
>
> -static void pnv_ioda_setup_npu_PEs(struct pci_bus *bus)
> +static void pnv_ioda_setup_npu_PEs(struct pci_bus *bus,
> + struct pnv_ioda_pe *fn(struct pci_dev *npu_pdev))
> {
> struct pci_dev *pdev;
>
> list_for_each_entry(pdev, &bus->devices, bus_list)
> - pnv_ioda_setup_npu_PE(pdev);
> + fn(pdev);
> }
I think adding a function pointer here is rather ugly, at this point you
might as well just do this directly in pnv_pci_ioda_setup_PEs()
>
> static void pnv_pci_ioda_setup_PEs(void)
> @@ -1275,13 +1277,18 @@ static void pnv_pci_ioda_setup_PEs(void)
>
> list_for_each_entry_safe(hose, tmp, &hose_list, list_node) {
> phb = hose->private_data;
> - if (phb->type == PNV_PHB_NPU) {
> + if (phb->type == PNV_PHB_NPU_NVLINK) {
> /* PE#0 is needed for error reporting */
> pnv_ioda_reserve_pe(phb, 0);
> - pnv_ioda_setup_npu_PEs(hose->bus);
> + pnv_ioda_setup_npu_PEs(hose->bus,
> + pnv_ioda_setup_npu_PE);
> if (phb->model == PNV_PHB_MODEL_NPU2)
> pnv_npu2_init(phb);
> }
> + if (phb->type == PNV_PHB_NPU_OCAPI) {
> + pnv_ioda_setup_npu_PEs(hose->bus,
> + pnv_ioda_setup_dev_PE);
> + }
> }
> }
>
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com IBM Australia Limited
^ permalink raw reply
* Re: [PATCH 10/13] ocxl: Add Makefile and Kconfig
From: Andrew Donnellan @ 2018-01-03 5:48 UTC (permalink / raw)
To: Frederic Barrat, linuxppc-dev, linux-kernel; +Cc: arnd, gregkh, mpe, alastair
In-Reply-To: <ddc10e7f73d714bb02bcac4958fc22ab10f7e129.1513608243.git.fbarrat@linux.vnet.ibm.com>
On 19/12/17 02:21, Frederic Barrat wrote:
> OCXL_BASE triggers the platform support needed by the driver.
>
> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
> ---
> drivers/misc/Kconfig | 1 +
> drivers/misc/Makefile | 1 +
> drivers/misc/ocxl/Kconfig | 25 +++++++++++++++++++++++++
> drivers/misc/ocxl/Makefile | 10 ++++++++++
> 4 files changed, 37 insertions(+)
> create mode 100644 drivers/misc/ocxl/Kconfig
> create mode 100644 drivers/misc/ocxl/Makefile
>
> diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
> index f1a5c2357b14..0534f338c84a 100644
> --- a/drivers/misc/Kconfig
> +++ b/drivers/misc/Kconfig
> @@ -508,4 +508,5 @@ source "drivers/misc/mic/Kconfig"
> source "drivers/misc/genwqe/Kconfig"
> source "drivers/misc/echo/Kconfig"
> source "drivers/misc/cxl/Kconfig"
> +source "drivers/misc/ocxl/Kconfig"
> endmenu
> diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
> index 5ca5f64df478..73326d54e246 100644
> --- a/drivers/misc/Makefile
> +++ b/drivers/misc/Makefile
> @@ -55,6 +55,7 @@ obj-$(CONFIG_CXL_BASE) += cxl/
> obj-$(CONFIG_ASPEED_LPC_CTRL) += aspeed-lpc-ctrl.o
> obj-$(CONFIG_ASPEED_LPC_SNOOP) += aspeed-lpc-snoop.o
> obj-$(CONFIG_PCI_ENDPOINT_TEST) += pci_endpoint_test.o
> +obj-$(CONFIG_OCXL) += ocxl/
>
> lkdtm-$(CONFIG_LKDTM) += lkdtm_core.o
> lkdtm-$(CONFIG_LKDTM) += lkdtm_bugs.o
> diff --git a/drivers/misc/ocxl/Kconfig b/drivers/misc/ocxl/Kconfig
> new file mode 100644
> index 000000000000..4496b61f48db
> --- /dev/null
> +++ b/drivers/misc/ocxl/Kconfig
> @@ -0,0 +1,25 @@
> +#
> +# Open Coherent Accelerator (OCXL) compatible devices
> +#
> +
> +config OCXL_BASE
> + bool
> + default n
> + select PPC_COPRO_BASE
> +
> +config OCXL
> + tristate "Support for Open Coherent Accelerators (OCXL)"
> + depends on PPC_POWERNV && PCI && EEH
> + select OCXL_BASE
> + default m
> + help
> +
> + Select this option to enable driver support for Open
> + Coherent Accelerators (OCXL). OCXL is otherwise known as
> + Open Coherent Accelerator Processor Interface (OCAPI).
> + OCAPI allows accelerators in FPGAs to be coherently attached
> + to a CPU through a Open CAPI link. This driver enables
> + userspace programs to access these accelerators through
> + devices found in /dev/ocxl/
I'd prefer more consistency in how we refer to OpenCAPI. "ocxl" is a
driver name that we have purely for historical reasons, it's not really
the name of anything else. I know throughout the various specs and code,
we use "OCAPI" a lot, but that's not really an abbreviation that should
be "user-facing".
Something like:
config OCXL
tristate "OpenCAPI coherent accelerator support"
help
Select this option to enable the ocxl driver for Open Coherent
Accelerator Processor Interface (OpenCAPI) devices.
OpenCAPI allows FPGA and ASIC accelerators to be coherently
attached to a CPU over an OpenCAPI link.
The ocxl driver enables userspace programs to access these
accelerators through devices in /dev/ocxl/.
For more information, see http://opencapi.org.
If unsure, say N.
> +
> + If unsure, say N.
> diff --git a/drivers/misc/ocxl/Makefile b/drivers/misc/ocxl/Makefile
> new file mode 100644
> index 000000000000..f75853411cfd
> --- /dev/null
> +++ b/drivers/misc/ocxl/Makefile
> @@ -0,0 +1,10 @@
> +ccflags-$(CONFIG_PPC_WERROR) += -Werror
> +
> +ocxl-y += main.o pci.o config.o file.o pasid.o
> +ocxl-y += link.o context.o afu_irq.o sysfs.o trace.o
> +obj-$(CONFIG_OCXL) += ocxl.o
> +
> +# For tracepoints to include our trace.h from tracepoint infrastructure:
> +CFLAGS_trace.o := -I$(src)
> +
> +# ccflags-y += -DDEBUG
>
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com IBM Australia Limited
^ permalink raw reply
* Re: [PATCH] selftests/powerpc: Add a test of SEGV error behaviour
From: Michael Ellerman @ 2018-01-03 6:14 UTC (permalink / raw)
To: John Sperbeck; +Cc: linuxppc-dev, Benjamin Herrenschmidt
In-Reply-To: <CAFNjLiWrz_=epy8BE6-EVDsr6hbUO5=ykOL1gV1asc=WMOoFFQ@mail.gmail.com>
John Sperbeck <jsperbeck@google.com> writes:
> On Tue, Jan 2, 2018 at 3:03 AM, Michael Ellerman <mpe@ellerman.id.au> wrote:
>> Add a test case of the error code reported when we take a SEGV on a
>> mapped but inaccessible area. We broke this recently.
>>
>> Based on a test case from John Sperbeck <jsperbeck@google.com>.
>>
>> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
...
>
> Looks good to me.
>
> Acked-by: John Sperbeck <jsperbeck@google.com>
Thanks.
cheers
^ permalink raw reply
* Re: [PATCH 16/67] powerpc: rename dma_direct_ to dma_nommu_
From: Michael Ellerman @ 2018-01-03 6:24 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Christoph Hellwig, iommu, Linux MIPS Mailing List, linux-ia64,
Linux-sh list, sparclinux, Guan Xuetao, Linux-Arch, linux-s390,
linux-c6x-dev, open list:QUALCOMM HEXAGON...,
the arch/x86 maintainers, arcml, adi-buildroot-devel, linux-m68k,
patches, open list:METAG ARCHITECTURE, linux-arm-kernel,
Michal Simek, Parisc List, Cris, Linux Kernel Mailing List, alpha,
linuxppc-dev
In-Reply-To: <CAMuHMdWWus2kNSOzS94k-3678826W1YjKwCWTquu3hBLZ80cvw@mail.gmail.com>
Geert Uytterhoeven <geert@linux-m68k.org> writes:
> On Tue, Jan 2, 2018 at 10:45 AM, Michael Ellerman <mpe@ellerman.id.au> wrote:
>> Christoph Hellwig <hch@lst.de> writes:
>>
>>> We want to use the dma_direct_ namespace for a generic implementation,
>>> so rename powerpc to the second best choice: dma_nommu_.
>>
>> I'm not a fan of "nommu". Some of the users of direct ops *are* using an
>> IOMMU, they're just setting up a 1:1 mapping once at init time, rather
>> than mapping dynamically.
>>
>> Though I don't have a good idea for a better name, maybe "1to1",
>> "linear", "premapped" ?
>
> "identity"?
I think that would be wrong, but thanks for trying to help :)
The address on the device side is sometimes (often?) offset from the CPU
address. So eg. the device can DMA to RAM address 0x0 using address
0x800000000000000.
Identity would imply 0 == 0 etc.
I think "bijective" is the correct term, but that's probably a bit
esoteric.
cheers
^ permalink raw reply
* Re: [PATCH v3] powerpc/64s: Improve local TLB flush for boot and MCE on POWER9
From: Aneesh Kumar K.V @ 2018-01-03 7:04 UTC (permalink / raw)
To: Nicholas Piggin, linuxppc-dev; +Cc: Nicholas Piggin
In-Reply-To: <20171223151550.30612-1-npiggin@gmail.com>
Nicholas Piggin <npiggin@gmail.com> writes:
> There are several cases outside the normal address space management
> where a CPU's entire local TLB is to be flushed:
>
> 1. Booting the kernel, in case something has left stale entries in
> the TLB (e.g., kexec).
>
> 2. Machine check, to clean corrupted TLB entries.
>
> One other place where the TLB is flushed, is waking from deep idle
> states. The flush is a side-effect of calling ->cpu_restore with the
> intention of re-setting various SPRs. The flush itself is unnecessary
> because in the first case, the TLB should not acquire new corrupted
> TLB entries as part of sleep/wake (though they may be lost).
>
> This type of TLB flush is coded inflexibly, several times for each CPU
> type, and they have a number of problems with ISA v3.0B:
>
> - The current radix mode of the MMU is not taken into account, it is
> always done as a hash flushn For IS=2 (LPID-matching flush from host)
> and IS=3 with HV=0 (guest kernel flush), tlbie(l) is undefined if
> the R field does not match the current radix mode.
>
> - ISA v3.0B hash must flush the partition and process table caches as
> well.
>
> - ISA v3.0B radix must flush partition and process scoped translations,
> partition and process table caches, and also the page walk cache.
>
> So consolidate the flushing code and implement it in C and inline asm
> under the mm/ directory with the rest of the flush code. Add ISA v3.0B
> cases for radix and hash, and use the radix flush in radix environment.
>
> Provide a way for IS=2 (LPID flush) to specify the radix mode of the
> partition. Have KVM pass in the radix mode of the guest.
>
> Take out the flushes from early cputable/dt_cpu_ftrs detection hooks,
> and move it later in the boot process after, the MMU registers are set
> up and before relocation is first turned on.
>
> The TLB flush is no longer called when restoring from deep idle states.
> This was not be done as a separate step because booting secondaries
> uses the same cpu_restore as idle restore, which needs the TLB flush.
>
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
......
> diff --git a/arch/powerpc/kvm/book3s_hv_ras.c b/arch/powerpc/kvm/book3s_hv_ras.c
> index c356f9a40b24..e61066bb6725 100644
> --- a/arch/powerpc/kvm/book3s_hv_ras.c
> +++ b/arch/powerpc/kvm/book3s_hv_ras.c
> @@ -87,8 +87,7 @@ static long kvmppc_realmode_mc_power7(struct kvm_vcpu *vcpu)
> DSISR_MC_SLB_PARITY | DSISR_MC_DERAT_MULTI);
> }
> if (dsisr & DSISR_MC_TLB_MULTI) {
> - if (cur_cpu_spec && cur_cpu_spec->flush_tlb)
> - cur_cpu_spec->flush_tlb(TLB_INVAL_SCOPE_LPID);
> + tlbiel_all_lpid(vcpu->kvm->arch.radix);
Why use vcpu->kvm-arch.radix? why not TLB_INVAL_SCOPE_LPID?
> dsisr &= ~DSISR_MC_TLB_MULTI;
> }
> /* Any other errors we don't understand? */
> @@ -105,8 +104,7 @@ static long kvmppc_realmode_mc_power7(struct kvm_vcpu *vcpu)
> reload_slb(vcpu);
> break;
> case SRR1_MC_IFETCH_TLBMULTI:
> - if (cur_cpu_spec && cur_cpu_spec->flush_tlb)
> - cur_cpu_spec->flush_tlb(TLB_INVAL_SCOPE_LPID);
> + tlbiel_all_lpid(vcpu->kvm->arch.radix);
> break;
> default:
> handled = 0;
-aneesh
^ permalink raw reply
* Re: [PATCH 06/13] ocxl: Driver code for 'generic' opencapi devices
From: Andrew Donnellan @ 2018-01-03 7:30 UTC (permalink / raw)
To: Frederic Barrat, linuxppc-dev, linux-kernel; +Cc: arnd, gregkh, mpe, alastair
In-Reply-To: <599cc92264dfdceb7d62ff65cc9aa9e2c2f1fd35.1513608243.git.fbarrat@linux.vnet.ibm.com>
On 19/12/17 02:21, Frederic Barrat wrote:
> Add an ocxl driver to handle generic opencapi devices. Of course, it's
> not meant to be the only opencapi driver, any device is free to
> implement its own. But if a host application only needs basic services
> like attaching to an opencapi adapter, have translation faults handled
> or allocate AFU interrupts, it should suffice.
>
> The AFU config space must follow the opencapi specification and use
> the expected vendor/device ID to be seen by the generic driver.
>
> The driver exposes the device AFUs as a char device in /dev/ocxl/
>
> Note that the driver currently doesn't handle memory attached to the
> opencapi device.
>
> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
> Signed-off-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
A bunch of sparse warnings we should look at. (there's a few more that
appear in later patches too)
> ---
> drivers/misc/ocxl/config.c | 718 ++++++++++++++++++++++++++++++++++++++
> drivers/misc/ocxl/context.c | 237 +++++++++++++
> drivers/misc/ocxl/file.c | 405 +++++++++++++++++++++
> drivers/misc/ocxl/link.c | 610 ++++++++++++++++++++++++++++++++
> drivers/misc/ocxl/main.c | 40 +++
> drivers/misc/ocxl/ocxl_internal.h | 200 +++++++++++
> drivers/misc/ocxl/pasid.c | 114 ++++++
> drivers/misc/ocxl/pci.c | 592 +++++++++++++++++++++++++++++++
> drivers/misc/ocxl/sysfs.c | 150 ++++++++
> include/uapi/misc/ocxl.h | 47 +++
> 10 files changed, 3113 insertions(+)
> create mode 100644 drivers/misc/ocxl/config.c
> create mode 100644 drivers/misc/ocxl/context.c
> create mode 100644 drivers/misc/ocxl/file.c
> create mode 100644 drivers/misc/ocxl/link.c
> create mode 100644 drivers/misc/ocxl/main.c
> create mode 100644 drivers/misc/ocxl/ocxl_internal.h
> create mode 100644 drivers/misc/ocxl/pasid.c
> create mode 100644 drivers/misc/ocxl/pci.c
> create mode 100644 drivers/misc/ocxl/sysfs.c
> create mode 100644 include/uapi/misc/ocxl.h
>
> diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c
> new file mode 100644
> index 000000000000..bb2fde5967e2
> --- /dev/null
> +++ b/drivers/misc/ocxl/config.c
> @@ -0,0 +1,718 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/pci.h>
> +#include <asm/pnv-ocxl.h>
> +#include <misc/ocxl-config.h>
> +#include "ocxl_internal.h"
> +
> +#define EXTRACT_BIT(val, bit) (!!(val & BIT(bit)))
> +#define EXTRACT_BITS(val, s, e) ((val & GENMASK(e, s)) >> s)
> +
> +#define OCXL_DVSEC_AFU_IDX_MASK GENMASK(5, 0)
> +#define OCXL_DVSEC_ACTAG_MASK GENMASK(11, 0)
> +#define OCXL_DVSEC_PASID_MASK GENMASK(19, 0)
> +#define OCXL_DVSEC_PASID_LOG_MASK GENMASK(4, 0)
> +
> +#define OCXL_DVSEC_TEMPL_VERSION 0x0
> +#define OCXL_DVSEC_TEMPL_NAME 0x4
> +#define OCXL_DVSEC_TEMPL_AFU_VERSION 0x1C
> +#define OCXL_DVSEC_TEMPL_MMIO_GLOBAL 0x20
> +#define OCXL_DVSEC_TEMPL_MMIO_GLOBAL_SZ 0x28
> +#define OCXL_DVSEC_TEMPL_MMIO_PP 0x30
> +#define OCXL_DVSEC_TEMPL_MMIO_PP_SZ 0x38
> +#define OCXL_DVSEC_TEMPL_MEM_SZ 0x3C
> +#define OCXL_DVSEC_TEMPL_WWID 0x40
> +
> +#define OCXL_MAX_AFU_PER_FUNCTION 64
> +#define OCXL_TEMPL_LEN 0x58
> +#define OCXL_TEMPL_NAME_LEN 24
> +#define OCXL_CFG_TIMEOUT 3
> +
> +static int find_dvsec(struct pci_dev *dev, int dvsec_id)
> +{
> + int vsec = 0;
> + u16 vendor, id;
> +
> + while ((vsec = pci_find_next_ext_capability(dev, vsec,
> + OCXL_EXT_CAP_ID_DVSEC))) {
> + pci_read_config_word(dev, vsec + OCXL_DVSEC_VENDOR_OFFSET,
> + &vendor);
> + pci_read_config_word(dev, vsec + OCXL_DVSEC_ID_OFFSET, &id);
> + if (vendor == PCI_VENDOR_ID_IBM && id == dvsec_id)
> + return vsec;
> + }
> + return 0;
> +}
> +
> +static int find_dvsec_afu_ctrl(struct pci_dev *dev, u8 afu_idx)
> +{
> + int vsec = 0;
> + u16 vendor, id;
> + u8 idx;
> +
> + while ((vsec = pci_find_next_ext_capability(dev, vsec,
> + OCXL_EXT_CAP_ID_DVSEC))) {
> + pci_read_config_word(dev, vsec + OCXL_DVSEC_VENDOR_OFFSET,
> + &vendor);
> + pci_read_config_word(dev, vsec + OCXL_DVSEC_ID_OFFSET, &id);
> +
> + if (vendor == PCI_VENDOR_ID_IBM &&
> + id == OCXL_DVSEC_AFU_CTRL_ID) {
> + pci_read_config_byte(dev,
> + vsec + OCXL_DVSEC_AFU_CTRL_AFU_IDX,
> + &idx);
> + if (idx == afu_idx)
> + return vsec;
> + }
> + }
> + return 0;
> +}
> +
> +static int read_pasid(struct pci_dev *dev, struct ocxl_fn_config *fn)
> +{
> + u16 val;
> + int pos;
> +
> + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_PASID);
> + if (!pos) {
> + /*
> + * PASID capability is not mandatory, but there
> + * shouldn't be any AFU
> + */
> + dev_dbg(&dev->dev, "Function doesn't require any PASID\n");
> + fn->max_pasid_log = -1;
> + goto out;
> + }
> + pci_read_config_word(dev, pos + PCI_PASID_CAP, &val);
> + fn->max_pasid_log = EXTRACT_BITS(val, 8, 12);
> +
> +out:
> + dev_dbg(&dev->dev, "PASID capability:\n");
> + dev_dbg(&dev->dev, " Max PASID log = %d\n", fn->max_pasid_log);
> + return 0;
> +}
> +
> +static int read_dvsec_tl(struct pci_dev *dev, struct ocxl_fn_config *fn)
> +{
> + int pos;
> +
> + pos = find_dvsec(dev, OCXL_DVSEC_TL_ID);
> + if (!pos && PCI_FUNC(dev->devfn) == 0) {
> + dev_err(&dev->dev, "Can't find TL DVSEC\n");
> + return -ENODEV;
> + }
> + if (pos && PCI_FUNC(dev->devfn) != 0) {
> + dev_err(&dev->dev, "TL DVSEC is only allowed on function 0\n");
> + return -ENODEV;
> + }
> + fn->dvsec_tl_pos = pos;
> + return 0;
> +}
> +
> +static int read_dvsec_function(struct pci_dev *dev, struct ocxl_fn_config *fn)
> +{
> + int pos, afu_present;
> + u32 val;
> +
> + pos = find_dvsec(dev, OCXL_DVSEC_FUNC_ID);
> + if (!pos) {
> + dev_err(&dev->dev, "Can't find function DVSEC\n");
> + return -ENODEV;
> + }
> + fn->dvsec_function_pos = pos;
> +
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_FUNC_OFF_INDEX, &val);
> + afu_present = EXTRACT_BIT(val, 31);
> + if (!afu_present) {
> + fn->max_afu_index = -1;
> + dev_dbg(&dev->dev, "Function doesn't define any AFU\n");
> + goto out;
> + }
> + fn->max_afu_index = EXTRACT_BITS(val, 24, 29);
> +
> +out:
> + dev_dbg(&dev->dev, "Function DVSEC:\n");
> + dev_dbg(&dev->dev, " Max AFU index = %d\n", fn->max_afu_index);
> + return 0;
> +}
> +
> +static int read_dvsec_afu_info(struct pci_dev *dev, struct ocxl_fn_config *fn)
> +{
> + int pos;
> +
> + if (fn->max_afu_index < 0) {
> + fn->dvsec_afu_info_pos = -1;
> + return 0;
> + }
> +
> + pos = find_dvsec(dev, OCXL_DVSEC_AFU_INFO_ID);
> + if (!pos) {
> + dev_err(&dev->dev, "Can't find AFU information DVSEC\n");
> + return -ENODEV;
> + }
> + fn->dvsec_afu_info_pos = pos;
> + return 0;
> +}
> +
> +static int read_dvsec_vendor(struct pci_dev *dev)
> +{
> + int pos;
> + u32 cfg, tlx, dlx;
> +
> + /*
> + * vendor specific DVSEC is optional
> + *
> + * It's currently only used on function 0 to specify the
> + * version of some logic blocks. Some older images may not
> + * even have it so we ignore any errors
> + */
> + if (PCI_FUNC(dev->devfn) != 0)
> + return 0;
> +
> + pos = find_dvsec(dev, OCXL_DVSEC_VENDOR_ID);
> + if (!pos)
> + return 0;
> +
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_VENDOR_CFG_VERS, &cfg);
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_VENDOR_TLX_VERS, &tlx);
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_VENDOR_DLX_VERS, &dlx);
> +
> + dev_dbg(&dev->dev, "Vendor specific DVSEC:\n");
> + dev_dbg(&dev->dev, " CFG version = 0x%x\n", cfg);
> + dev_dbg(&dev->dev, " TLX version = 0x%x\n", tlx);
> + dev_dbg(&dev->dev, " DLX version = 0x%x\n", dlx);
> + return 0;
> +}
> +
> +static int validate_function(struct pci_dev *dev, struct ocxl_fn_config *fn)
> +{
> + if (fn->max_pasid_log == -1 && fn->max_afu_index >= 0) {
> + dev_err(&dev->dev,
> + "AFUs are defined but no PASIDs are requested\n");
> + return -EINVAL;
> + }
> +
> + if (fn->max_afu_index > OCXL_MAX_AFU_PER_FUNCTION) {
> + dev_err(&dev->dev,
> + "Max AFU index out of architectural limit (%d vs %d)\n",
> + fn->max_afu_index, OCXL_MAX_AFU_PER_FUNCTION);
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> +int ocxl_config_read_function(struct pci_dev *dev, struct ocxl_fn_config *fn)
> +{
> + int rc;
> +
> + rc = read_pasid(dev, fn);
> + if (rc) {
> + dev_err(&dev->dev, "Invalid PASID configuration: %d\n", rc);
> + return -ENODEV;
> + }
> +
> + rc = read_dvsec_tl(dev, fn);
> + if (rc) {
> + dev_err(&dev->dev,
> + "Invalid Transaction Layer DVSEC configuration: %d\n",
> + rc);
> + return -ENODEV;
> + }
> +
> + rc = read_dvsec_function(dev, fn);
> + if (rc) {
> + dev_err(&dev->dev,
> + "Invalid Function DVSEC configuration: %d\n", rc);
> + return -ENODEV;
> + }
> +
> + rc = read_dvsec_afu_info(dev, fn);
> + if (rc) {
> + dev_err(&dev->dev, "Invalid AFU configuration: %d\n", rc);
> + return -ENODEV;
> + }
> +
> + rc = read_dvsec_vendor(dev);
> + if (rc) {
> + dev_err(&dev->dev,
> + "Invalid vendor specific DVSEC configuration: %d\n",
> + rc);
> + return -ENODEV;
> + }
> +
> + rc = validate_function(dev, fn);
> + return rc;
> +}
> +
> +static int read_afu_info(struct pci_dev *dev, struct ocxl_fn_config *fn,
> + int offset, u32 *data)
> +{
> + u32 val;
> + unsigned long timeout = jiffies + (HZ * OCXL_CFG_TIMEOUT);
> + int pos = fn->dvsec_afu_info_pos;
> +
> + /* Protect 'data valid' bit */
> + if (EXTRACT_BIT(offset, 31)) {
> + dev_err(&dev->dev, "Invalid offset in AFU info DVSEC\n");
> + return -EINVAL;
> + }
> +
> + pci_write_config_dword(dev, pos + OCXL_DVSEC_AFU_INFO_OFF, offset);
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_AFU_INFO_OFF, &val);
> + while (!EXTRACT_BIT(val, 31)) {
> + if (time_after_eq(jiffies, timeout)) {
> + dev_err(&dev->dev,
> + "Timeout while reading AFU info DVSEC (offset=%d)\n",
> + offset);
> + return -EBUSY;
> + }
> + cpu_relax();
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_AFU_INFO_OFF, &val);
> + }
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_AFU_INFO_DATA, data);
> + return 0;
> +}
> +
> +int ocxl_config_check_afu_index(struct pci_dev *dev,
> + struct ocxl_fn_config *fn, int afu_idx)
> +{
> + u32 val;
> + int rc, templ_major, templ_minor, len;
> +
> + pci_write_config_word(dev, fn->dvsec_afu_info_pos, afu_idx);
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_VERSION, &val);
> + if (rc)
> + return rc;
> +
> + /* AFU index map can have holes */
> + if (!val)
> + return 0;
> +
> + templ_major = EXTRACT_BITS(val, 8, 15);
> + templ_minor = EXTRACT_BITS(val, 0, 7);
> + dev_dbg(&dev->dev, "AFU descriptor template version %d.%d\n",
> + templ_major, templ_minor);
> +
> + len = EXTRACT_BITS(val, 16, 31);
> + if (len != OCXL_TEMPL_LEN) {
> + dev_warn(&dev->dev,
> + "Unexpected template length in AFU information (%#x)\n",
> + len);
> + }
> + return 1;
> +}
> +
> +static int read_afu_name(struct pci_dev *dev, struct ocxl_fn_config *fn,
> + struct ocxl_afu_config *afu)
> +{
> + int i, rc;
> + u32 val, *ptr;
> +
> + BUILD_BUG_ON(OCXL_AFU_NAME_SZ < OCXL_TEMPL_NAME_LEN);
> + for (i = 0; i < OCXL_TEMPL_NAME_LEN; i += 4) {
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_NAME + i, &val);
> + if (rc)
> + return rc;
> + ptr = (u32 *) &afu->name[i];
> + *ptr = val;
> + }
> + afu->name[OCXL_AFU_NAME_SZ - 1] = '\0'; /* play safe */
> + return 0;
> +}
> +
> +static int read_afu_mmio(struct pci_dev *dev, struct ocxl_fn_config *fn,
> + struct ocxl_afu_config *afu)
> +{
> + int rc;
> + u32 val;
> +
> + /*
> + * Global MMIO
> + */
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MMIO_GLOBAL, &val);
> + if (rc)
> + return rc;
> + afu->global_mmio_bar = EXTRACT_BITS(val, 0, 2);
> + afu->global_mmio_offset = EXTRACT_BITS(val, 16, 31) << 16;
> +
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MMIO_GLOBAL + 4, &val);
> + if (rc)
> + return rc;
> + afu->global_mmio_offset += (u64) val << 32;
> +
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MMIO_GLOBAL_SZ, &val);
> + if (rc)
> + return rc;
> + afu->global_mmio_size = val;
> +
> + /*
> + * Per-process MMIO
> + */
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MMIO_PP, &val);
> + if (rc)
> + return rc;
> + afu->pp_mmio_bar = EXTRACT_BITS(val, 0, 2);
> + afu->pp_mmio_offset = EXTRACT_BITS(val, 16, 31) << 16;
> +
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MMIO_PP + 4, &val);
> + if (rc)
> + return rc;
> + afu->pp_mmio_offset += (u64) val << 32;
> +
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MMIO_PP_SZ, &val);
> + if (rc)
> + return rc;
> + afu->pp_mmio_stride = val;
> +
> + return 0;
> +}
> +
> +static int read_afu_control(struct pci_dev *dev, struct ocxl_afu_config *afu)
> +{
> + int pos;
> + u8 val8;
> + u16 val16;
> +
> + pos = find_dvsec_afu_ctrl(dev, afu->idx);
> + if (!pos) {
> + dev_err(&dev->dev, "Can't find AFU control DVSEC for AFU %d\n",
> + afu->idx);
> + return -ENODEV;
> + }
> + afu->dvsec_afu_control_pos = pos;
> +
> + pci_read_config_byte(dev, pos + OCXL_DVSEC_AFU_CTRL_PASID_SUP, &val8);
> + afu->pasid_supported_log = EXTRACT_BITS(val8, 0, 4);
> +
> + pci_read_config_word(dev, pos + OCXL_DVSEC_AFU_CTRL_ACTAG_SUP, &val16);
> + afu->actag_supported = EXTRACT_BITS(val16, 0, 11);
> + return 0;
> +}
> +
> +static bool char_allowed(int c)
> +{
> + /*
> + * Permitted Characters : Alphanumeric, hyphen, underscore, comma
> + */
> + if ((c >= 0x30 && c <= 0x39) /* digits */ ||
> + (c >= 0x41 && c <= 0x5A) /* upper case */ ||
> + (c >= 0x61 && c <= 0x7A) /* lower case */ ||
> + c == 0 /* NULL */ ||
> + c == 0x2D /* - */ ||
> + c == 0x5F /* _ */ ||
> + c == 0x2C /* , */)
> + return true;
> + return false;
> +}
> +
> +static int validate_afu(struct pci_dev *dev, struct ocxl_afu_config *afu)
> +{
> + int i;
> +
> + if (!afu->name[0]) {
> + dev_err(&dev->dev, "Empty AFU name\n");
> + return -EINVAL;
> + }
> + for (i = 0; i < OCXL_TEMPL_NAME_LEN; i++) {
> + if (!char_allowed(afu->name[i])) {
> + dev_err(&dev->dev,
> + "Invalid character in AFU name\n");
> + return -EINVAL;
> + }
> + }
> +
> + if (afu->global_mmio_bar != 0 &&
> + afu->global_mmio_bar != 2 &&
> + afu->global_mmio_bar != 4) {
> + dev_err(&dev->dev, "Invalid global MMIO bar number\n");
> + return -EINVAL;
> + }
> + if (afu->pp_mmio_bar != 0 &&
> + afu->pp_mmio_bar != 2 &&
> + afu->pp_mmio_bar != 4) {
> + dev_err(&dev->dev, "Invalid per-process MMIO bar number\n");
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> +int ocxl_config_read_afu(struct pci_dev *dev, struct ocxl_fn_config *fn,
> + struct ocxl_afu_config *afu, u8 afu_idx)
> +{
> + int rc;
> + u32 val32;
> +
> + /*
> + * First, we need to write the AFU idx for the AFU we want to
> + * access.
> + */
> + WARN_ON((afu_idx & OCXL_DVSEC_AFU_IDX_MASK) != afu_idx);
> + afu->idx = afu_idx;
> + pci_write_config_byte(dev,
> + fn->dvsec_afu_info_pos + OCXL_DVSEC_AFU_INFO_AFU_IDX,
> + afu->idx);
> +
> + rc = read_afu_name(dev, fn, afu);
> + if (rc)
> + return rc;
> +
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_AFU_VERSION, &val32);
> + if (rc)
> + return rc;
> + afu->version_major = EXTRACT_BITS(val32, 24, 31);
> + afu->version_minor = EXTRACT_BITS(val32, 16, 23);
> + afu->afuc_type = EXTRACT_BITS(val32, 14, 15);
> + afu->afum_type = EXTRACT_BITS(val32, 12, 13);
> + afu->profile = EXTRACT_BITS(val32, 0, 7);
> +
> + rc = read_afu_mmio(dev, fn, afu);
> + if (rc)
> + return rc;
> +
> + rc = read_afu_info(dev, fn, OCXL_DVSEC_TEMPL_MEM_SZ, &val32);
> + if (rc)
> + return rc;
> + afu->log_mem_size = EXTRACT_BITS(val32, 0, 7);
> +
> + rc = read_afu_control(dev, afu);
> + if (rc)
> + return rc;
> +
> + dev_dbg(&dev->dev, "AFU configuration:\n");
> + dev_dbg(&dev->dev, " name = %s\n", afu->name);
> + dev_dbg(&dev->dev, " version = %d.%d\n", afu->version_major,
> + afu->version_minor);
> + dev_dbg(&dev->dev, " global mmio bar = %hhu\n", afu->global_mmio_bar);
> + dev_dbg(&dev->dev, " global mmio offset = %#llx\n",
> + afu->global_mmio_offset);
> + dev_dbg(&dev->dev, " global mmio size = %#x\n", afu->global_mmio_size);
> + dev_dbg(&dev->dev, " pp mmio bar = %hhu\n", afu->pp_mmio_bar);
> + dev_dbg(&dev->dev, " pp mmio offset = %#llx\n", afu->pp_mmio_offset);
> + dev_dbg(&dev->dev, " pp mmio stride = %#x\n", afu->pp_mmio_stride);
> + dev_dbg(&dev->dev, " mem size (log) = %hhu\n", afu->log_mem_size);
> + dev_dbg(&dev->dev, " pasid supported (log) = %u\n",
> + afu->pasid_supported_log);
> + dev_dbg(&dev->dev, " actag supported = %u\n",
> + afu->actag_supported);
> +
> + rc = validate_afu(dev, afu);
> + return rc;
> +}
> +
> +int ocxl_config_get_actag_info(struct pci_dev *dev, u16 *base, u16 *enabled,
> + u16 *supported)
> +{
> + int rc;
> +
> + /*
> + * This is really a simple wrapper for the kernel API, to
> + * avoid an external driver using ocxl as a library to call
> + * platform-dependent code
> + */
> + rc = pnv_ocxl_get_actag(dev, base, enabled, supported);
> + if (rc) {
> + dev_err(&dev->dev, "Can't get actag for device: %d\n", rc);
> + return rc;
> + }
> + return 0;
> +}
> +
> +void ocxl_config_set_afu_actag(struct pci_dev *dev, int pos, int actag_base,
> + int actag_count)
> +{
> + u16 val;
> +
> + val = actag_count & OCXL_DVSEC_ACTAG_MASK;
> + pci_write_config_byte(dev, pos + OCXL_DVSEC_AFU_CTRL_ACTAG_EN, val);
> +
> + val = actag_base & OCXL_DVSEC_ACTAG_MASK;
> + pci_write_config_dword(dev, pos + OCXL_DVSEC_AFU_CTRL_ACTAG_BASE, val);
> +}
> +
> +int ocxl_config_get_pasid_info(struct pci_dev *dev, int *count)
> +{
> + return pnv_ocxl_get_pasid_count(dev, count);
> +}
> +
> +void ocxl_config_set_afu_pasid(struct pci_dev *dev, int pos, int pasid_base,
> + u32 pasid_count_log)
> +{
> + u8 val8;
> + u32 val32;
> +
> + val8 = pasid_count_log & OCXL_DVSEC_PASID_LOG_MASK;
> + pci_write_config_byte(dev, pos + OCXL_DVSEC_AFU_CTRL_PASID_EN, val8);
> +
> + pci_read_config_dword(dev, pos + OCXL_DVSEC_AFU_CTRL_PASID_BASE,
> + &val32);
> + val32 &= ~OCXL_DVSEC_PASID_MASK;
> + val32 |= pasid_base & OCXL_DVSEC_PASID_MASK;
> + pci_write_config_dword(dev, pos + OCXL_DVSEC_AFU_CTRL_PASID_BASE,
> + val32);
> +}
> +
> +void ocxl_config_set_afu_state(struct pci_dev *dev, int pos, int enable)
> +{
> + u8 val;
> +
> + pci_read_config_byte(dev, pos + OCXL_DVSEC_AFU_CTRL_ENABLE, &val);
> + if (enable)
> + val |= 1;
> + else
> + val &= 0xFE;
> + pci_write_config_byte(dev, pos + OCXL_DVSEC_AFU_CTRL_ENABLE, val);
> +}
> +
> +int ocxl_config_set_TL(struct pci_dev *dev, int tl_dvsec)
> +{
> + u32 val, *ptr32;
> + u8 timers;
> + int i, rc;
> + long recv_cap;
> + char *recv_rate;
> +
> + /*
> + * Skip on function != 0, as the TL can only be defined on 0
> + */
> + if (PCI_FUNC(dev->devfn) != 0)
> + return 0;
> +
> + recv_rate = kzalloc(PNV_OCXL_TL_RATE_BUF_SIZE, GFP_KERNEL);
> + if (!recv_rate)
> + return -ENOMEM;
> + /*
> + * The spec defines 64 templates for messages in the
> + * Transaction Layer (TL).
> + *
> + * The host and device each support a subset, so we need to
> + * configure the transmitters on each side to send only
> + * templates the receiver understands, at a rate the receiver
> + * can process. Per the spec, template 0 must be supported by
> + * everybody. That's the template which has been used by the
> + * host and device so far.
> + *
> + * The sending rate limit must be set before the template is
> + * enabled.
> + */
> +
> + /*
> + * Device -> host
> + */
> + rc = pnv_ocxl_get_tl_cap(dev, &recv_cap, recv_rate,
> + PNV_OCXL_TL_RATE_BUF_SIZE);
> + if (rc)
> + goto out;
> +
> + for (i = 0; i < PNV_OCXL_TL_RATE_BUF_SIZE; i += 4) {
> + ptr32 = (u32 *) &recv_rate[i];
> + pci_write_config_dword(dev,
> + tl_dvsec + OCXL_DVSEC_TL_SEND_RATE + i,
> + be32_to_cpu(*ptr32));
drivers/misc/ocxl/config.c:618:33: warning: cast to restricted __be32
> + }
> + val = recv_cap >> 32;
> + pci_write_config_dword(dev, tl_dvsec + OCXL_DVSEC_TL_SEND_CAP, val);
> + val = recv_cap & GENMASK(31, 0);
> + pci_write_config_dword(dev, tl_dvsec + OCXL_DVSEC_TL_SEND_CAP + 4, val);
> +
> + /*
> + * Host -> device
> + */
> + for (i = 0; i < PNV_OCXL_TL_RATE_BUF_SIZE; i += 4) {
> + pci_read_config_dword(dev,
> + tl_dvsec + OCXL_DVSEC_TL_RECV_RATE + i,
> + &val);
> + ptr32 = (u32 *) &recv_rate[i];
> + *ptr32 = cpu_to_be32(val);
drivers/misc/ocxl/config.c:633:24: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/config.c:633:24: expected unsigned int [unsigned]
[usertype] <noident>
drivers/misc/ocxl/config.c:633:24: got restricted __be32 [usertype]
<noident>
> + }
> + pci_read_config_dword(dev, tl_dvsec + OCXL_DVSEC_TL_RECV_CAP, &val);
> + recv_cap = (long) val << 32;
> + pci_read_config_dword(dev, tl_dvsec + OCXL_DVSEC_TL_RECV_CAP + 4, &val);
> + recv_cap |= val;
> +
> + rc = pnv_ocxl_set_tl_conf(dev, recv_cap, __pa(recv_rate),
> + PNV_OCXL_TL_RATE_BUF_SIZE);
> + if (rc)
> + goto out;
> +
> + /*
> + * Opencapi commands needing to be retried are classified per
> + * the TL in 2 groups: short and long commands.
> + *
> + * The short back off timer it not used for now. It will be
> + * for opencapi 4.0.
> + *
> + * The long back off timer is typically used when an AFU hits
> + * a page fault but the NPU is already processing one. So the
> + * AFU needs to wait before it can resubmit. Having a value
> + * too low doesn't break anything, but can generate extra
> + * traffic on the link.
> + * We set it to 1.6 us for now. It's shorter than, but in the
> + * same order of magnitude as the time spent to process a page
> + * fault.
> + */
> + timers = 0x2 << 4; /* long timer = 1.6 us */
> + pci_write_config_byte(dev, tl_dvsec + OCXL_DVSEC_TL_BACKOFF_TIMERS,
> + timers);
> +
> + rc = 0;
> +out:
> + kfree(recv_rate);
> + return rc;
> +}
> +
> +int ocxl_config_terminate_pasid(struct pci_dev *dev, int afu_control, int pasid)
> +{
> + u32 val;
> + unsigned long timeout;
> +
> + pci_read_config_dword(dev, afu_control + OCXL_DVSEC_AFU_CTRL_TERM_PASID,
> + &val);
> + if (EXTRACT_BIT(val, 20)) {
> + dev_err(&dev->dev,
> + "Can't terminate PASID %#x, previous termination didn't complete\n",
> + pasid);
> + return -EBUSY;
> + }
> +
> + val &= ~OCXL_DVSEC_PASID_MASK;
> + val |= pasid & OCXL_DVSEC_PASID_MASK;
> + val |= BIT(20);
> + pci_write_config_dword(dev,
> + afu_control + OCXL_DVSEC_AFU_CTRL_TERM_PASID,
> + val);
> +
> + timeout = jiffies + (HZ * OCXL_CFG_TIMEOUT);
> + pci_read_config_dword(dev, afu_control + OCXL_DVSEC_AFU_CTRL_TERM_PASID,
> + &val);
> + while (EXTRACT_BIT(val, 20)) {
> + if (time_after_eq(jiffies, timeout)) {
> + dev_err(&dev->dev,
> + "Timeout while waiting for AFU to terminate PASID %#x\n",
> + pasid);
> + return -EBUSY;
> + }
> + cpu_relax();
> + pci_read_config_dword(dev,
> + afu_control + OCXL_DVSEC_AFU_CTRL_TERM_PASID,
> + &val);
> + }
> + return 0;
> +}
> +
> +void ocxl_config_set_actag(struct pci_dev *dev, int func_dvsec, u32 tag_first,
> + u32 tag_count)
> +{
> + u32 val;
> +
> + val = (tag_first & OCXL_DVSEC_ACTAG_MASK) << 16;
> + val |= tag_count & OCXL_DVSEC_ACTAG_MASK;
> + pci_write_config_dword(dev, func_dvsec + OCXL_DVSEC_FUNC_OFF_ACTAG,
> + val);
> +}
> diff --git a/drivers/misc/ocxl/context.c b/drivers/misc/ocxl/context.c
> new file mode 100644
> index 000000000000..0bc0dd97d784
> --- /dev/null
> +++ b/drivers/misc/ocxl/context.c
> @@ -0,0 +1,237 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/sched/mm.h>
> +#include "ocxl_internal.h"
> +
> +struct ocxl_context *ocxl_context_alloc(void)
> +{
> + return kzalloc(sizeof(struct ocxl_context), GFP_KERNEL);
> +}
> +
> +int ocxl_context_init(struct ocxl_context *ctx, struct ocxl_afu *afu,
> + struct address_space *mapping)
> +{
> + int pasid;
> +
> + ctx->afu = afu;
> + mutex_lock(&afu->contexts_lock);
> + pasid = idr_alloc(&afu->contexts_idr, ctx, afu->pasid_base,
> + afu->pasid_base + afu->pasid_max, GFP_KERNEL);
> + if (pasid < 0) {
> + mutex_unlock(&afu->contexts_lock);
> + return pasid;
> + }
> + afu->pasid_count++;
> + mutex_unlock(&afu->contexts_lock);
> +
> + ctx->pasid = pasid;
> + ctx->status = OPENED;
> + mutex_init(&ctx->status_mutex);
> + ctx->mapping = mapping;
> + mutex_init(&ctx->mapping_lock);
> + init_waitqueue_head(&ctx->events_wq);
> + mutex_init(&ctx->xsl_error_lock);
> + /*
> + * Keep a reference on the AFU to make sure it's valid for the
> + * duration of the life of the context
> + */
> + ocxl_afu_get(afu);
> + return 0;
> +}
> +
> +/*
> + * Callback for when a translation fault triggers an error
> + * data: a pointer to the context which triggered the fault
> + * addr: the address that triggered the error
> + * dsisr: the value of the PPC64 dsisr register
> + */
> +static void xsl_fault_error(void *data, u64 addr, u64 dsisr)
> +{
> + struct ocxl_context *ctx = (struct ocxl_context *) data;
> +
> + mutex_lock(&ctx->xsl_error_lock);
> + ctx->xsl_error.addr = addr;
> + ctx->xsl_error.dsisr = dsisr;
> + ctx->xsl_error.count++;
> + mutex_unlock(&ctx->xsl_error_lock);
> +
> + wake_up_all(&ctx->events_wq);
> +}
> +
> +int ocxl_context_attach(struct ocxl_context *ctx, u64 amr)
> +{
> + int rc;
> +
> + mutex_lock(&ctx->status_mutex);
> + if (ctx->status != OPENED) {
> + rc = -EIO;
> + goto out;
> + }
> +
> + rc = ocxl_link_add_pe(ctx->afu->fn->link, ctx->pasid,
> + current->mm->context.id, 0, amr, current->mm,
> + xsl_fault_error, ctx);
> + if (rc)
> + goto out;
> +
> + ctx->status = ATTACHED;
> +out:
> + mutex_unlock(&ctx->status_mutex);
> + return rc;
> +}
> +
> +static int map_pp_mmio(struct vm_area_struct *vma, unsigned long address,
> + u64 offset, struct ocxl_context *ctx)
> +{
> + u64 pp_mmio_addr;
> + int pasid_off;
> +
> + if (offset >= ctx->afu->config.pp_mmio_stride)
> + return VM_FAULT_SIGBUS;
> +
> + mutex_lock(&ctx->status_mutex);
> + if (ctx->status != ATTACHED) {
> + mutex_unlock(&ctx->status_mutex);
> + pr_debug("%s: Context not attached, failing mmio mmap\n",
> + __func__);
> + return VM_FAULT_SIGBUS;
> + }
> +
> + pasid_off = ctx->pasid - ctx->afu->pasid_base;
> + pp_mmio_addr = ctx->afu->pp_mmio_start +
> + pasid_off * ctx->afu->config.pp_mmio_stride +
> + offset;
> +
> + vm_insert_pfn(vma, address, pp_mmio_addr >> PAGE_SHIFT);
> + mutex_unlock(&ctx->status_mutex);
> + return VM_FAULT_NOPAGE;
> +}
> +
> +static int ocxl_mmap_fault(struct vm_fault *vmf)
> +{
> + struct vm_area_struct *vma = vmf->vma;
> + struct ocxl_context *ctx = vma->vm_file->private_data;
> + u64 offset;
> + int rc;
> +
> + offset = vmf->pgoff << PAGE_SHIFT;
> + pr_debug("%s: pasid %d address 0x%lx offset 0x%llx\n", __func__,
> + ctx->pasid, vmf->address, offset);
> +
> + rc = map_pp_mmio(vma, vmf->address, offset, ctx);
> + return rc;
> +}
> +
> +static const struct vm_operations_struct ocxl_vmops = {
> + .fault = ocxl_mmap_fault,
> +};
> +
> +static int check_mmap_mmio(struct ocxl_context *ctx,
> + struct vm_area_struct *vma)
> +{
> + if ((vma_pages(vma) + vma->vm_pgoff) >
> + (ctx->afu->config.pp_mmio_stride >> PAGE_SHIFT))
> + return -EINVAL;
> + return 0;
> +}
> +
> +int ocxl_context_mmap(struct ocxl_context *ctx, struct vm_area_struct *vma)
> +{
> + int rc;
> +
> + rc = check_mmap_mmio(ctx, vma);
> + if (rc)
> + return rc;
> +
> + vma->vm_flags |= VM_IO | VM_PFNMAP;
> + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> + vma->vm_ops = &ocxl_vmops;
> + return 0;
> +}
> +
> +int ocxl_context_detach(struct ocxl_context *ctx)
> +{
> + struct pci_dev *dev;
> + int afu_control_pos;
> + enum ocxl_context_status status;
> + int rc;
> +
> + mutex_lock(&ctx->status_mutex);
> + status = ctx->status;
> + ctx->status = CLOSED;
> + mutex_unlock(&ctx->status_mutex);
> + if (status != ATTACHED)
> + return 0;
> +
> + dev = to_pci_dev(ctx->afu->fn->dev.parent);
> + afu_control_pos = ctx->afu->config.dvsec_afu_control_pos;
> +
> + mutex_lock(&ctx->afu->afu_control_lock);
> + rc = ocxl_config_terminate_pasid(dev, afu_control_pos, ctx->pasid);
> + mutex_unlock(&ctx->afu->afu_control_lock);
> + if (rc) {
> + /*
> + * If we timeout waiting for the AFU to terminate the
> + * pasid, then it's dangerous to clean up the Process
> + * Element entry in the SPA, as it may be referenced
> + * in the future by the AFU. In which case, we would
> + * checkstop because of an invalid PE access (FIR
> + * register 2, bit 42). So leave the PE
> + * defined. Caller shouldn't free the context so that
> + * PASID remains allocated.
> + *
> + * A link reset will be required to cleanup the AFU
> + * and the SPA.
> + */
> + if (rc == -EBUSY)
> + return rc;
> + }
> + rc = ocxl_link_remove_pe(ctx->afu->fn->link, ctx->pasid);
> + if (rc) {
> + dev_warn(&ctx->afu->dev,
> + "Couldn't remove PE entry cleanly: %d\n", rc);
> + }
> + return 0;
> +}
> +
> +void ocxl_context_detach_all(struct ocxl_afu *afu)
> +{
> + struct ocxl_context *ctx;
> + int tmp;
> +
> + mutex_lock(&afu->contexts_lock);
> + idr_for_each_entry(&afu->contexts_idr, ctx, tmp) {
> + ocxl_context_detach(ctx);
> + /*
> + * We are force detaching - remove any active mmio
> + * mappings so userspace cannot interfere with the
> + * card if it comes back. Easiest way to exercise
> + * this is to unbind and rebind the driver via sysfs
> + * while it is in use.
> + */
> + mutex_lock(&ctx->mapping_lock);
> + if (ctx->mapping)
> + unmap_mapping_range(ctx->mapping, 0, 0, 1);
> + mutex_unlock(&ctx->mapping_lock);
> + }
> + mutex_unlock(&afu->contexts_lock);
> +}
> +
> +void ocxl_context_free(struct ocxl_context *ctx)
> +{
> + mutex_lock(&ctx->afu->contexts_lock);
> + ctx->afu->pasid_count--;
> + idr_remove(&ctx->afu->contexts_idr, ctx->pasid);
> + mutex_unlock(&ctx->afu->contexts_lock);
> +
> + /* reference to the AFU taken in ocxl_context_init */
> + ocxl_afu_put(ctx->afu);
> + kfree(ctx);
> +}
> diff --git a/drivers/misc/ocxl/file.c b/drivers/misc/ocxl/file.c
> new file mode 100644
> index 000000000000..a51386eff4f5
> --- /dev/null
> +++ b/drivers/misc/ocxl/file.c
> @@ -0,0 +1,405 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/fs.h>
> +#include <linux/poll.h>
> +#include <linux/sched/signal.h>
> +#include <linux/uaccess.h>
> +#include <uapi/misc/ocxl.h>
> +#include "ocxl_internal.h"
> +
> +
> +#define OCXL_NUM_MINORS 256 /* Total to reserve */
> +
> +static dev_t ocxl_dev;
> +static struct class *ocxl_class;
> +static struct mutex minors_idr_lock;
> +static struct idr minors_idr;
> +
> +static struct ocxl_afu *find_and_get_afu(dev_t devno)
> +{
> + struct ocxl_afu *afu;
> + int afu_minor;
> +
> + afu_minor = MINOR(devno);
> + /*
> + * We don't declare an RCU critical section here, as our AFU
> + * is protected by a reference counter on the device. By the time the
> + * minor number of a device is removed from the idr, the ref count of
> + * the device is already at 0, so no user API will access that AFU and
> + * this function can't return it.
> + */
> + afu = idr_find(&minors_idr, afu_minor);
> + if (afu)
> + ocxl_afu_get(afu);
> + return afu;
> +}
> +
> +static int allocate_afu_minor(struct ocxl_afu *afu)
> +{
> + int minor;
> +
> + mutex_lock(&minors_idr_lock);
> + minor = idr_alloc(&minors_idr, afu, 0, OCXL_NUM_MINORS, GFP_KERNEL);
> + mutex_unlock(&minors_idr_lock);
> + return minor;
> +}
> +
> +static void free_afu_minor(struct ocxl_afu *afu)
> +{
> + mutex_lock(&minors_idr_lock);
> + idr_remove(&minors_idr, MINOR(afu->dev.devt));
> + mutex_unlock(&minors_idr_lock);
> +}
> +
> +static int afu_open(struct inode *inode, struct file *file)
> +{
> + struct ocxl_afu *afu;
> + struct ocxl_context *ctx;
> + int rc;
> +
> + pr_debug("%s for device %x\n", __func__, inode->i_rdev);
> +
> + afu = find_and_get_afu(inode->i_rdev);
> + if (!afu)
> + return -ENODEV;
> +
> + ctx = ocxl_context_alloc();
> + if (!ctx) {
> + rc = -ENOMEM;
> + goto put_afu;
> + }
> +
> + rc = ocxl_context_init(ctx, afu, inode->i_mapping);
> + if (rc)
> + goto put_afu;
> + file->private_data = ctx;
> + ocxl_afu_put(afu);
> + return 0;
> +
> +put_afu:
> + ocxl_afu_put(afu);
> + return rc;
> +}
> +
> +static long afu_ioctl_attach(struct ocxl_context *ctx,
> + struct ocxl_ioctl_attach __user *uarg)
> +{
> + struct ocxl_ioctl_attach arg;
> + u64 amr = 0;
> + int rc;
> +
> + pr_debug("%s for context %d\n", __func__, ctx->pasid);
> +
> + if (copy_from_user(&arg, uarg, sizeof(arg)))
> + return -EFAULT;
> +
> + /* Make sure reserved fields are not set for forward compatibility */
> + if (arg.reserved1 || arg.reserved2 || arg.reserved3)
> + return -EINVAL;
> +
> + amr = arg.amr & mfspr(SPRN_UAMOR);
> + rc = ocxl_context_attach(ctx, amr);
> + return rc;
> +}
> +
> +#define CMD_STR(x) (x == OCXL_IOCTL_ATTACH ? "ATTACH" : \
> + "UNKNOWN")
> +
> +static long afu_ioctl(struct file *file, unsigned int cmd,
> + unsigned long args)
> +{
> + struct ocxl_context *ctx = file->private_data;
> + long rc;
> +
> + pr_debug("%s for context %d, command %s\n", __func__, ctx->pasid,
> + CMD_STR(cmd));
> +
> + if (ctx->status == CLOSED)
> + return -EIO;
> +
> + switch (cmd) {
> + case OCXL_IOCTL_ATTACH:
> + rc = afu_ioctl_attach(ctx,
> + (struct ocxl_ioctl_attach __user *) args);
> + break;
> +
> + default:
> + rc = -EINVAL;
> + }
> + return rc;
> +}
> +
> +static long afu_compat_ioctl(struct file *file, unsigned int cmd,
> + unsigned long args)
> +{
> + return afu_ioctl(file, cmd, args);
> +}
> +
> +static int afu_mmap(struct file *file, struct vm_area_struct *vma)
> +{
> + struct ocxl_context *ctx = file->private_data;
> +
> + pr_debug("%s for context %d\n", __func__, ctx->pasid);
> + return ocxl_context_mmap(ctx, vma);
> +}
> +
> +static bool has_xsl_error(struct ocxl_context *ctx)
> +{
> + bool ret;
> +
> + mutex_lock(&ctx->xsl_error_lock);
> + ret = !!ctx->xsl_error.addr;
> + mutex_unlock(&ctx->xsl_error_lock);
> +
> + return ret;
> +}
> +
> +/*
> + * Are there any events pending on the AFU
> + * ctx: The AFU context
> + * Returns: true if there are events pending
> + */
> +static bool afu_events_pending(struct ocxl_context *ctx)
> +{
> + if (has_xsl_error(ctx))
> + return true;
> + return false;
> +}
> +
> +static unsigned int afu_poll(struct file *file, struct poll_table_struct *wait)
> +{
> + struct ocxl_context *ctx = file->private_data;
> + unsigned int mask = 0;
> + bool closed;
> +
> + pr_debug("%s for context %d\n", __func__, ctx->pasid);
> +
> + poll_wait(file, &ctx->events_wq, wait);
> +
> + mutex_lock(&ctx->status_mutex);
> + closed = (ctx->status == CLOSED);
> + mutex_unlock(&ctx->status_mutex);
> +
> + if (afu_events_pending(ctx))
> + mask = POLLIN | POLLRDNORM;
> + else if (closed)
> + mask = POLLERR;
> +
> + return mask;
> +}
> +
> +/*
> + * Populate the supplied buffer with a single XSL error
> + * ctx: The AFU context to report the error from
> + * header: the event header to populate
> + * buf: The buffer to write the body into (should be at least
> + * AFU_EVENT_BODY_XSL_ERROR_SIZE)
> + * Return: the amount of buffer that was populated
> + */
> +static ssize_t append_xsl_error(struct ocxl_context *ctx,
> + struct ocxl_kernel_event_header *header,
> + char __user *buf)
> +{
> + struct ocxl_kernel_event_xsl_fault_error body;
> +
> + memset(&body, 0, sizeof(body));
> +
> + mutex_lock(&ctx->xsl_error_lock);
> + if (!ctx->xsl_error.addr) {
> + mutex_unlock(&ctx->xsl_error_lock);
> + return 0;
> + }
> +
> + body.addr = ctx->xsl_error.addr;
> + body.dsisr = ctx->xsl_error.dsisr;
> + body.count = ctx->xsl_error.count;
> +
> + ctx->xsl_error.addr = 0;
> + ctx->xsl_error.dsisr = 0;
> + ctx->xsl_error.count = 0;
> +
> + mutex_unlock(&ctx->xsl_error_lock);
> +
> + header->type = OCXL_AFU_EVENT_XSL_FAULT_ERROR;
> +
> + if (copy_to_user(buf, &body, sizeof(body)))
> + return -EFAULT;
> +
> + return sizeof(body);
> +}
> +
> +#define AFU_EVENT_BODY_MAX_SIZE sizeof(struct ocxl_kernel_event_xsl_fault_error)
> +
> +/*
> + * Reports events on the AFU
> + * Format:
> + * Header (struct ocxl_kernel_event_header)
> + * Body (struct ocxl_kernel_event_*)
> + * Header...
> + */
> +static ssize_t afu_read(struct file *file, char __user *buf, size_t count,
> + loff_t *off)
> +{
> + struct ocxl_context *ctx = file->private_data;
> + struct ocxl_kernel_event_header header;
> + ssize_t rc;
> + size_t used = 0;
> + DEFINE_WAIT(event_wait);
> +
> + memset(&header, 0, sizeof(header));
> +
> + /* Require offset to be 0 */
> + if (*off != 0)
> + return -EINVAL;
> +
> + if (count < (sizeof(struct ocxl_kernel_event_header) +
> + AFU_EVENT_BODY_MAX_SIZE))
> + return -EINVAL;
> +
> + for (;;) {
> + prepare_to_wait(&ctx->events_wq, &event_wait,
> + TASK_INTERRUPTIBLE);
> +
> + if (afu_events_pending(ctx))
> + break;
> +
> + if (ctx->status == CLOSED)
> + break;
> +
> + if (file->f_flags & O_NONBLOCK) {
> + finish_wait(&ctx->events_wq, &event_wait);
> + return -EAGAIN;
> + }
> +
> + if (signal_pending(current)) {
> + finish_wait(&ctx->events_wq, &event_wait);
> + return -ERESTARTSYS;
> + }
> +
> + schedule();
> + }
> +
> + finish_wait(&ctx->events_wq, &event_wait);
> +
> + if (has_xsl_error(ctx)) {
> + used = append_xsl_error(ctx, &header, buf + sizeof(header));
> + if (used < 0)
> + return used;
> + }
> +
> + if (!afu_events_pending(ctx))
> + header.flags |= OCXL_KERNEL_EVENT_FLAG_LAST;
> +
> + if (copy_to_user(buf, &header, sizeof(header)))
> + return -EFAULT;
> +
> + used += sizeof(header);
> +
> + rc = (ssize_t) used;
> + return rc;
> +}
> +
> +static int afu_release(struct inode *inode, struct file *file)
> +{
> + struct ocxl_context *ctx = file->private_data;
> + int rc;
> +
> + pr_debug("%s for device %x\n", __func__, inode->i_rdev);
> + rc = ocxl_context_detach(ctx);
> + mutex_lock(&ctx->mapping_lock);
> + ctx->mapping = NULL;
> + mutex_unlock(&ctx->mapping_lock);
> + wake_up_all(&ctx->events_wq);
> + if (rc != -EBUSY)
> + ocxl_context_free(ctx);
> + return 0;
> +}
> +
> +static const struct file_operations ocxl_afu_fops = {
> + .owner = THIS_MODULE,
> + .open = afu_open,
> + .unlocked_ioctl = afu_ioctl,
> + .compat_ioctl = afu_compat_ioctl,
> + .mmap = afu_mmap,
> + .poll = afu_poll,
> + .read = afu_read,
> + .release = afu_release,
> +};
> +
> +int ocxl_create_cdev(struct ocxl_afu *afu)
> +{
> + int rc;
> +
> + cdev_init(&afu->cdev, &ocxl_afu_fops);
> + rc = cdev_add(&afu->cdev, afu->dev.devt, 1);
> + if (rc) {
> + dev_err(&afu->dev, "Unable to add afu char device: %d\n", rc);
> + return rc;
> + }
> + return 0;
> +}
> +
> +void ocxl_destroy_cdev(struct ocxl_afu *afu)
> +{
> + cdev_del(&afu->cdev);
> +}
> +
> +int ocxl_register_afu(struct ocxl_afu *afu)
> +{
> + int minor;
> +
> + minor = allocate_afu_minor(afu);
> + if (minor < 0)
> + return minor;
> + afu->dev.devt = MKDEV(MAJOR(ocxl_dev), minor);
> + afu->dev.class = ocxl_class;
> + return device_register(&afu->dev);
> +}
> +
> +void ocxl_unregister_afu(struct ocxl_afu *afu)
> +{
> + free_afu_minor(afu);
> +}
> +
> +static char *ocxl_devnode(struct device *dev, umode_t *mode)
> +{
> + return kasprintf(GFP_KERNEL, "ocxl/%s", dev_name(dev));
> +}
> +
> +int ocxl_file_init(void)
> +{
> + int rc;
> +
> + mutex_init(&minors_idr_lock);
> + idr_init(&minors_idr);
> +
> + rc = alloc_chrdev_region(&ocxl_dev, 0, OCXL_NUM_MINORS, "ocxl");
> + if (rc) {
> + pr_err("Unable to allocate ocxl major number: %d\n", rc);
> + return rc;
> + }
> +
> + ocxl_class = class_create(THIS_MODULE, "ocxl");
> + if (IS_ERR(ocxl_class)) {
> + pr_err("Unable to create ocxl class\n");
> + unregister_chrdev_region(ocxl_dev, OCXL_NUM_MINORS);
> + return PTR_ERR(ocxl_class);
> + }
> +
> + ocxl_class->devnode = ocxl_devnode;
> + return 0;
> +}
> +
> +void ocxl_file_exit(void)
> +{
> + class_destroy(ocxl_class);
> + unregister_chrdev_region(ocxl_dev, OCXL_NUM_MINORS);
> + idr_destroy(&minors_idr);
> +}
> diff --git a/drivers/misc/ocxl/link.c b/drivers/misc/ocxl/link.c
> new file mode 100644
> index 000000000000..6b184cd7d2a6
> --- /dev/null
> +++ b/drivers/misc/ocxl/link.c
> @@ -0,0 +1,610 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/sched/mm.h>
> +#include <linux/mutex.h>
> +#include <linux/mmu_context.h>
> +#include <asm/copro.h>
> +#include <asm/pnv-ocxl.h>
> +#include "ocxl_internal.h"
> +
> +
> +#define SPA_PASID_BITS 15
> +#define SPA_PASID_MAX ((1 << SPA_PASID_BITS) - 1)
> +#define SPA_PE_MASK SPA_PASID_MAX
> +#define SPA_SPA_SIZE_LOG 22 /* Each SPA is 4 Mb */
> +
> +#define SPA_CFG_SF (1ull << (63-0))
> +#define SPA_CFG_TA (1ull << (63-1))
> +#define SPA_CFG_HV (1ull << (63-3))
> +#define SPA_CFG_UV (1ull << (63-4))
> +#define SPA_CFG_XLAT_hpt (0ull << (63-6)) /* Hashed page table (HPT) mode */
> +#define SPA_CFG_XLAT_roh (2ull << (63-6)) /* Radix on HPT mode */
> +#define SPA_CFG_XLAT_ror (3ull << (63-6)) /* Radix on Radix mode */
> +#define SPA_CFG_PR (1ull << (63-49))
> +#define SPA_CFG_TC (1ull << (63-54))
> +#define SPA_CFG_DR (1ull << (63-59))
> +
> +#define SPA_XSL_TF (1ull << (63-3)) /* Translation fault */
> +#define SPA_XSL_S (1ull << (63-38)) /* Store operation */
> +
> +#define SPA_PE_VALID 0x80000000
> +
> +
> +struct pe_data {
> + struct mm_struct *mm;
> + /* callback to trigger when a translation fault occurs */
> + void (*xsl_err_cb)(void *data, u64 addr, u64 dsisr);
> + /* opaque pointer to be passed to the above callback */
> + void *xsl_err_data;
> + struct rcu_head rcu;
> +};
> +
> +struct spa {
> + struct ocxl_process_element *spa_mem;
> + int spa_order;
> + struct mutex spa_lock;
> + struct radix_tree_root pe_tree; /* Maps PE handles to pe_data */
> + char *irq_name;
> + int virq;
> + void __iomem *reg_dsisr;
> + void __iomem *reg_dar;
> + void __iomem *reg_tfc;
> + void __iomem *reg_pe_handle;
> + /*
> + * The following field are used by the memory fault
> + * interrupt handler. We can only have one interrupt at a
> + * time. The NPU won't raise another interrupt until the
> + * previous one has been ack'd by writing to the TFC register
> + */
> + struct xsl_fault {
> + struct work_struct fault_work;
> + u64 pe;
> + u64 dsisr;
> + u64 dar;
> + struct pe_data pe_data;
> + } xsl_fault;
> +};
> +
> +/*
> + * A opencapi link can be used be by several PCI functions. We have
> + * one link per device slot.
> + *
> + * A linked list of opencapi links should suffice, as there's a
> + * limited number of opencapi slots on a system and lookup is only
> + * done when the device is probed
> + */
> +struct link {
> + struct list_head list;
> + struct kref ref;
> + int domain;
> + int bus;
> + int dev;
> + atomic_t irq_available;
> + struct spa *spa;
> + void *platform_data;
> +};
> +static struct list_head links_list = LIST_HEAD_INIT(links_list);
> +static DEFINE_MUTEX(links_list_lock);
> +
> +enum xsl_response {
> + CONTINUE,
> + ADDRESS_ERROR,
> + RESTART,
> +};
> +
> +
> +static void read_irq(struct spa *spa, u64 *dsisr, u64 *dar, u64 *pe)
> +{
> + u64 reg;
> +
> + *dsisr = in_be64(spa->reg_dsisr);
> + *dar = in_be64(spa->reg_dar);
> + reg = in_be64(spa->reg_pe_handle);
> + *pe = reg & SPA_PE_MASK;
> +}
> +
> +static void ack_irq(struct spa *spa, enum xsl_response r)
> +{
> + u64 reg = 0;
> +
> + /* continue is not supported */
> + if (r == RESTART)
> + reg = PPC_BIT(31);
> + else if (r == ADDRESS_ERROR)
> + reg = PPC_BIT(30);
> + else
> + WARN(1, "Invalid irq response %d\n", r);
> +
> + if (reg)
> + out_be64(spa->reg_tfc, reg);
> +}
> +
> +static void xsl_fault_handler_bh(struct work_struct *fault_work)
> +{
> + unsigned int flt = 0;
> + unsigned long access, flags, inv_flags = 0;
> + enum xsl_response r;
> + struct xsl_fault *fault = container_of(fault_work, struct xsl_fault,
> + fault_work);
> + struct spa *spa = container_of(fault, struct spa, xsl_fault);
> +
> + int rc;
> +
> + /*
> + * We need to release a reference on the mm whenever exiting this
> + * function (taken in the memory fault interrupt handler)
> + */
> + rc = copro_handle_mm_fault(fault->pe_data.mm, fault->dar, fault->dsisr,
> + &flt);
> + if (rc) {
> + pr_debug("copro_handle_mm_fault failed: %d\n", rc);
> + if (fault->pe_data.xsl_err_cb) {
> + fault->pe_data.xsl_err_cb(
> + fault->pe_data.xsl_err_data,
> + fault->dar, fault->dsisr);
> + }
> + r = ADDRESS_ERROR;
> + goto ack;
> + }
> +
> + if (!radix_enabled()) {
> + /*
> + * update_mmu_cache() will not have loaded the hash
> + * since current->trap is not a 0x400 or 0x300, so
> + * just call hash_page_mm() here.
> + */
> + access = _PAGE_PRESENT | _PAGE_READ;
> + if (fault->dsisr & SPA_XSL_S)
> + access |= _PAGE_WRITE;
> +
> + if (REGION_ID(fault->dar) != USER_REGION_ID)
> + access |= _PAGE_PRIVILEGED;
> +
> + local_irq_save(flags);
> + hash_page_mm(fault->pe_data.mm, fault->dar, access, 0x300,
> + inv_flags);
> + local_irq_restore(flags);
> + }
> + r = RESTART;
> +ack:
> + mmdrop(fault->pe_data.mm);
> + ack_irq(spa, r);
> +}
> +
> +static irqreturn_t xsl_fault_handler(int irq, void *data)
> +{
> + struct link *link = (struct link *) data;
> + struct spa *spa = link->spa;
> + u64 dsisr, dar, pe_handle;
> + struct pe_data *pe_data;
> + struct ocxl_process_element *pe;
> + int lpid, pid, tid;
> +
> + read_irq(spa, &dsisr, &dar, &pe_handle);
> +
> + WARN_ON(pe_handle > SPA_PE_MASK);
> + pe = spa->spa_mem + pe_handle;
> + lpid = be32_to_cpu(pe->lpid);
> + pid = be32_to_cpu(pe->pid);
> + tid = be32_to_cpu(pe->tid);
drivers/misc/ocxl/link.c:193:16: warning: cast to restricted __be32
drivers/misc/ocxl/link.c:194:15: warning: cast to restricted __be32
drivers/misc/ocxl/link.c:195:15: warning: cast to restricted __be32
> + /* We could be reading all null values here if the PE is being
> + * removed while an interrupt kicks in. It's not supposed to
> + * happen if the driver notified the AFU to terminate the
> + * PASID, and the AFU waited for pending operations before
> + * acknowledging. But even if it happens, we won't find a
> + * memory context below and fail silently, so it should be ok.
> + */
> + if (!(dsisr & SPA_XSL_TF)) {
> + WARN(1, "Invalid xsl interrupt fault register %#llx\n", dsisr);
> + ack_irq(spa, ADDRESS_ERROR);
> + return IRQ_HANDLED;
> + }
> +
> + rcu_read_lock();
> + pe_data = radix_tree_lookup(&spa->pe_tree, pe_handle);
> + if (!pe_data) {
> + /*
> + * Could only happen if the driver didn't notify the
> + * AFU about PASID termination before removing the PE,
> + * or the AFU didn't wait for all memory access to
> + * have completed.
> + *
> + * Either way, we fail early, but we shouldn't log an
> + * error message, as it is a valid (if unexpected)
> + * scenario
> + */
> + rcu_read_unlock();
> + pr_debug("Unknown mm context for xsl interrupt\n");
> + ack_irq(spa, ADDRESS_ERROR);
> + return IRQ_HANDLED;
> + }
> + WARN_ON(pe_data->mm->context.id != pid);
> +
> + spa->xsl_fault.pe = pe_handle;
> + spa->xsl_fault.dar = dar;
> + spa->xsl_fault.dsisr = dsisr;
> + spa->xsl_fault.pe_data = *pe_data;
> + mmgrab(pe_data->mm); /* mm count is released by bottom half */
> +
> + rcu_read_unlock();
> + schedule_work(&spa->xsl_fault.fault_work);
> + return IRQ_HANDLED;
> +}
> +
> +static void unmap_irq_registers(struct spa *spa)
> +{
> + pnv_ocxl_unmap_xsl_regs(spa->reg_dsisr, spa->reg_dar, spa->reg_tfc,
> + spa->reg_pe_handle);
> +}
> +
> +static int map_irq_registers(struct pci_dev *dev, struct spa *spa)
> +{
> + return pnv_ocxl_map_xsl_regs(dev, &spa->reg_dsisr, &spa->reg_dar,
> + &spa->reg_tfc, &spa->reg_pe_handle);
> +}
> +
> +static int setup_xsl_irq(struct pci_dev *dev, struct link *link)
> +{
> + struct spa *spa = link->spa;
> + int rc;
> + int hwirq;
> +
> + rc = pnv_ocxl_get_xsl_irq(dev, &hwirq);
> + if (rc)
> + return rc;
> +
> + rc = map_irq_registers(dev, spa);
> + if (rc)
> + return rc;
> +
> + spa->irq_name = kasprintf(GFP_KERNEL, "ocxl-xsl-%x-%x-%x",
> + link->domain, link->bus, link->dev);
> + if (!spa->irq_name) {
> + unmap_irq_registers(spa);
> + dev_err(&dev->dev, "Can't allocate name for xsl interrupt\n");
> + return -ENOMEM;
> + }
> + /*
> + * At some point, we'll need to look into allowing a higher
> + * number of interrupts. Could we have an IRQ domain per link?
> + */
> + spa->virq = irq_create_mapping(NULL, hwirq);
> + if (!spa->virq) {
> + kfree(spa->irq_name);
> + unmap_irq_registers(spa);
> + dev_err(&dev->dev,
> + "irq_create_mapping failed for translation interrupt\n");
> + return -EINVAL;
> + }
> +
> + dev_dbg(&dev->dev, "hwirq %d mapped to virq %d\n", hwirq, spa->virq);
> +
> + rc = request_irq(spa->virq, xsl_fault_handler, 0, spa->irq_name,
> + link);
> + if (rc) {
> + irq_dispose_mapping(spa->virq);
> + kfree(spa->irq_name);
> + unmap_irq_registers(spa);
> + dev_err(&dev->dev,
> + "request_irq failed for translation interrupt: %d\n",
> + rc);
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> +static void release_xsl_irq(struct link *link)
> +{
> + struct spa *spa = link->spa;
> +
> + if (spa->virq) {
> + free_irq(spa->virq, link);
> + irq_dispose_mapping(spa->virq);
> + }
> + kfree(spa->irq_name);
> + unmap_irq_registers(spa);
> +}
> +
> +static int alloc_spa(struct pci_dev *dev, struct link *link)
> +{
> + struct spa *spa;
> +
> + spa = kzalloc(sizeof(struct spa), GFP_KERNEL);
> + if (!spa)
> + return -ENOMEM;
> +
> + mutex_init(&spa->spa_lock);
> + INIT_RADIX_TREE(&spa->pe_tree, GFP_KERNEL);
> + INIT_WORK(&spa->xsl_fault.fault_work, xsl_fault_handler_bh);
> +
> + spa->spa_order = SPA_SPA_SIZE_LOG - PAGE_SHIFT;
> + spa->spa_mem = (struct ocxl_process_element *)
> + __get_free_pages(GFP_KERNEL | __GFP_ZERO, spa->spa_order);
> + if (!spa->spa_mem) {
> + dev_err(&dev->dev, "Can't allocate Shared Process Area\n");
> + kfree(spa);
> + return -ENOMEM;
> + }
> + pr_debug("Allocated SPA for %x:%x:%x at %p\n", link->domain, link->bus,
> + link->dev, spa->spa_mem);
> +
> + link->spa = spa;
> + return 0;
> +}
> +
> +static void free_spa(struct link *link)
> +{
> + struct spa *spa = link->spa;
> +
> + pr_debug("Freeing SPA for %x:%x:%x\n", link->domain, link->bus,
> + link->dev);
> +
> + if (spa && spa->spa_mem) {
> + free_pages((unsigned long) spa->spa_mem, spa->spa_order);
> + kfree(spa);
> + link->spa = NULL;
> + }
> +}
> +
> +static int alloc_link(struct pci_dev *dev, int PE_mask, struct link **out_link)
> +{
> + struct link *link;
> + int rc;
> +
> + link = kzalloc(sizeof(struct link), GFP_KERNEL);
> + if (!link)
> + return -ENOMEM;
> +
> + kref_init(&link->ref);
> + link->domain = pci_domain_nr(dev->bus);
> + link->bus = dev->bus->number;
> + link->dev = PCI_SLOT(dev->devfn);
> + atomic_set(&link->irq_available, MAX_IRQ_PER_LINK);
> +
> + rc = alloc_spa(dev, link);
> + if (rc)
> + goto err_free;
> +
> + rc = setup_xsl_irq(dev, link);
> + if (rc)
> + goto err_spa;
> +
> + /* platform specific hook */
> + rc = pnv_ocxl_spa_setup(dev, link->spa->spa_mem, PE_mask,
> + &link->platform_data);
> + if (rc)
> + goto err_xsl_irq;
> +
> + *out_link = link;
> + return 0;
> +
> +err_xsl_irq:
> + release_xsl_irq(link);
> +err_spa:
> + free_spa(link);
> +err_free:
> + kfree(link);
> + return rc;
> +}
> +
> +static void free_link(struct link *link)
> +{
> + release_xsl_irq(link);
> + free_spa(link);
> + kfree(link);
> +}
> +
> +int ocxl_link_setup(struct pci_dev *dev, int PE_mask, void **link_handle)
> +{
> + int rc = 0;
> + struct link *link;
> +
> + mutex_lock(&links_list_lock);
> + list_for_each_entry(link, &links_list, list) {
> + /* The functions of a device all share the same link */
> + if (link->domain == pci_domain_nr(dev->bus) &&
> + link->bus == dev->bus->number &&
> + link->dev == PCI_SLOT(dev->devfn)) {
> + kref_get(&link->ref);
> + *link_handle = link;
> + goto unlock;
> + }
> + }
> + rc = alloc_link(dev, PE_mask, &link);
> + if (rc)
> + goto unlock;
> +
> + list_add(&link->list, &links_list);
> + *link_handle = link;
> +unlock:
> + mutex_unlock(&links_list_lock);
> + return rc;
> +}
> +
> +static void release_xsl(struct kref *ref)
> +{
> + struct link *link = container_of(ref, struct link, ref);
> +
> + list_del(&link->list);
> + /* call platform code before releasing data */
> + pnv_ocxl_spa_release(link->platform_data);
> + free_link(link);
> +}
> +
> +void ocxl_link_release(struct pci_dev *dev, void *link_handle)
> +{
> + struct link *link = (struct link *) link_handle;
> +
> + mutex_lock(&links_list_lock);
> + kref_put(&link->ref, release_xsl);
> + mutex_unlock(&links_list_lock);
> +}
> +
> +static u64 calculate_cfg_state(bool kernel)
> +{
> + u64 state;
> +
> + state = SPA_CFG_DR;
> + if (mfspr(SPRN_LPCR) & LPCR_TC)
> + state |= SPA_CFG_TC;
> + if (radix_enabled())
> + state |= SPA_CFG_XLAT_ror;
> + else
> + state |= SPA_CFG_XLAT_hpt;
> + state |= SPA_CFG_HV;
> + if (kernel) {
> + if (mfmsr() & MSR_SF)
> + state |= SPA_CFG_SF;
> + } else {
> + state |= SPA_CFG_PR;
> + if (!test_tsk_thread_flag(current, TIF_32BIT))
> + state |= SPA_CFG_SF;
> + }
> + return state;
> +}
> +
> +int ocxl_link_add_pe(void *link_handle, int pasid, u32 pidr, u32 tidr,
> + u64 amr, struct mm_struct *mm,
> + void (*xsl_err_cb)(void *data, u64 addr, u64 dsisr),
> + void *xsl_err_data)
> +{
> + struct link *link = (struct link *) link_handle;
> + struct spa *spa = link->spa;
> + struct ocxl_process_element *pe;
> + int pe_handle, rc = 0;
> + struct pe_data *pe_data;
> +
> + BUILD_BUG_ON(sizeof(struct ocxl_process_element) != 128);
> + if (pasid > SPA_PASID_MAX)
> + return -EINVAL;
> +
> + mutex_lock(&spa->spa_lock);
> + pe_handle = pasid & SPA_PE_MASK;
> + pe = spa->spa_mem + pe_handle;
> +
> + if (pe->software_state) {
> + rc = -EBUSY;
> + goto unlock;
> + }
> +
> + pe_data = kmalloc(sizeof(*pe_data), GFP_KERNEL);
> + if (!pe_data) {
> + rc = -ENOMEM;
> + goto unlock;
> + }
> +
> + pe_data->mm = mm;
> + pe_data->xsl_err_cb = xsl_err_cb;
> + pe_data->xsl_err_data = xsl_err_data;
> +
> + memset(pe, 0, sizeof(struct ocxl_process_element));
> + pe->config_state = cpu_to_be64(calculate_cfg_state(pidr == 0));
> + pe->lpid = cpu_to_be32(mfspr(SPRN_LPID));
> + pe->pid = cpu_to_be32(pidr);
> + pe->tid = cpu_to_be32(tidr);
> + pe->amr = cpu_to_be64(amr);
> + pe->software_state = cpu_to_be32(SPA_PE_VALID);
drivers/misc/ocxl/link.c:509:26: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/link.c:509:26: expected unsigned long long
[unsigned] [usertype] config_state
drivers/misc/ocxl/link.c:509:26: got restricted __be64 [usertype]
<noident>
drivers/misc/ocxl/link.c:510:18: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/link.c:510:18: expected unsigned int [unsigned]
[usertype] lpid
drivers/misc/ocxl/link.c:510:18: got restricted __be32 [usertype]
<noident>
drivers/misc/ocxl/link.c:511:17: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/link.c:511:17: expected unsigned int [unsigned]
[usertype] pid
drivers/misc/ocxl/link.c:511:17: got restricted __be32 [usertype]
<noident>
drivers/misc/ocxl/link.c:512:17: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/link.c:512:17: expected unsigned int [unsigned]
[usertype] tid
drivers/misc/ocxl/link.c:512:17: got restricted __be32 [usertype]
<noident>
drivers/misc/ocxl/link.c:513:17: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/link.c:513:17: expected unsigned long long
[unsigned] [usertype] amr
drivers/misc/ocxl/link.c:513:17: got restricted __be64 [usertype]
<noident>
drivers/misc/ocxl/link.c:514:28: warning: incorrect type in assignment
(different base types)
drivers/misc/ocxl/link.c:514:28: expected unsigned int [unsigned]
[usertype] software_state
drivers/misc/ocxl/link.c:514:28: got restricted __be32 [usertype]
<noident>
> +
> + mm_context_add_copro(mm);
> + /*
> + * Barrier is to make sure PE is visible in the SPA before it
> + * is used by the device. It also helps with the global TLBI
> + * invalidation
> + */
> + mb();
> + radix_tree_insert(&spa->pe_tree, pe_handle, pe_data);
> +
> + /*
> + * The mm must stay valid for as long as the device uses it. We
> + * lower the count when the context is removed from the SPA.
> + *
> + * We grab mm_count (and not mm_users), as we don't want to
> + * end up in a circular dependency if a process mmaps its
> + * mmio, therefore incrementing the file ref count when
> + * calling mmap(), and forgets to unmap before exiting. In
> + * that scenario, when the kernel handles the death of the
> + * process, the file is not cleaned because unmap was not
> + * called, and the mm wouldn't be freed because we would still
> + * have a reference on mm_users. Incrementing mm_count solves
> + * the problem.
> + */
> + mmgrab(mm);
> +unlock:
> + mutex_unlock(&spa->spa_lock);
> + return rc;
> +}
> +
> +int ocxl_link_remove_pe(void *link_handle, int pasid)
> +{
> + struct link *link = (struct link *) link_handle;
> + struct spa *spa = link->spa;
> + struct ocxl_process_element *pe;
> + struct pe_data *pe_data;
> + int pe_handle, rc;
> +
> + if (pasid > SPA_PASID_MAX)
> + return -EINVAL;
> +
> + /*
> + * About synchronization with our memory fault handler:
> + *
> + * Before removing the PE, the driver is supposed to have
> + * notified the AFU, which should have cleaned up and make
> + * sure the PASID is no longer in use, including pending
> + * interrupts. However, there's no way to be sure...
> + *
> + * We clear the PE and remove the context from our radix
> + * tree. From that point on, any new interrupt for that
> + * context will fail silently, which is ok. As mentioned
> + * above, that's not expected, but it could happen if the
> + * driver or AFU didn't do the right thing.
> + *
> + * There could still be a bottom half running, but we don't
> + * need to wait/flush, as it is managing a reference count on
> + * the mm it reads from the radix tree.
> + */
> + pe_handle = pasid & SPA_PE_MASK;
> + pe = spa->spa_mem + pe_handle;
> +
> + mutex_lock(&spa->spa_lock);
> +
> + if (!(pe->software_state & cpu_to_be32(SPA_PE_VALID))) {
drivers/misc/ocxl/link.c:581:36: warning: restricted __be32 degrades to
integer
> + rc = -EINVAL;
> + goto unlock;
> + }
> +
> + memset(pe, 0, sizeof(struct ocxl_process_element));
> + /*
> + * The barrier makes sure the PE is removed from the SPA
> + * before we clear the NPU context cache below, so that the
> + * old PE cannot be reloaded erroneously.
> + */
> + mb();
> +
> + /*
> + * hook to platform code
> + * On powerpc, the entry needs to be cleared from the context
> + * cache of the NPU.
> + */
> + rc = pnv_ocxl_spa_remove_pe(link->platform_data, pe_handle);
> + WARN_ON(rc);
> +
> + pe_data = radix_tree_delete(&spa->pe_tree, pe_handle);
> + if (!pe_data) {
> + WARN(1, "Couldn't find pe data when removing PE\n");
> + } else {
> + mm_context_remove_copro(pe_data->mm);
> + mmdrop(pe_data->mm);
> + kfree_rcu(pe_data, rcu);
> + }
> +unlock:
> + mutex_unlock(&spa->spa_lock);
> + return rc;
> +}
> diff --git a/drivers/misc/ocxl/main.c b/drivers/misc/ocxl/main.c
> new file mode 100644
> index 000000000000..be34b8fae97a
> --- /dev/null
> +++ b/drivers/misc/ocxl/main.c
> @@ -0,0 +1,40 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include "ocxl_internal.h"
> +
> +static int __init init_ocxl(void)
> +{
> + int rc = 0;
> +
> + rc = ocxl_file_init();
> + if (rc)
> + return rc;
> +
> + rc = pci_register_driver(&ocxl_pci_driver);
> + if (rc) {
> + ocxl_file_exit();
> + return rc;
> + }
> + return 0;
> +}
> +
> +static void exit_ocxl(void)
> +{
> + pci_unregister_driver(&ocxl_pci_driver);
> + ocxl_file_exit();
> +}
> +
> +module_init(init_ocxl);
> +module_exit(exit_ocxl);
> +
> +MODULE_DESCRIPTION("Open Coherent Accelerator");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h
> new file mode 100644
> index 000000000000..e07f7d523275
> --- /dev/null
> +++ b/drivers/misc/ocxl/ocxl_internal.h
> @@ -0,0 +1,200 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#ifndef _OCXL_INTERNAL_H_
> +#define _OCXL_INTERNAL_H_
> +
> +#include <linux/pci.h>
> +#include <linux/cdev.h>
> +#include <linux/list.h>
> +
> +#define OCXL_AFU_NAME_SZ (24+1) /* add 1 for NULL termination */
> +#define MAX_IRQ_PER_LINK 2000
> +#define MAX_IRQ_PER_CONTEXT MAX_IRQ_PER_LINK
> +
> +#define to_ocxl_function(d) container_of(d, struct ocxl_fn, dev)
> +#define to_ocxl_afu(d) container_of(d, struct ocxl_afu, dev)
> +
> +extern struct pci_driver ocxl_pci_driver;
> +
> +/*
> + * The following 2 structures are a fairly generic way of representing
> + * the configuration data for a function and AFU, as read from the
> + * configuration space.
> + */
> +struct ocxl_afu_config {
> + u8 idx;
> + int dvsec_afu_control_pos;
> + char name[OCXL_AFU_NAME_SZ];
> + u8 version_major;
> + u8 version_minor;
> + u8 afuc_type;
> + u8 afum_type;
> + u8 profile;
> + u8 global_mmio_bar;
> + u64 global_mmio_offset;
> + u32 global_mmio_size;
> + u8 pp_mmio_bar;
> + u64 pp_mmio_offset;
> + u32 pp_mmio_stride;
> + u8 log_mem_size;
> + u8 pasid_supported_log;
> + u16 actag_supported;
> +};
> +
> +struct ocxl_fn_config {
> + int dvsec_tl_pos;
> + int dvsec_function_pos;
> + int dvsec_afu_info_pos;
> + s8 max_pasid_log;
> + s8 max_afu_index;
> +};
> +
> +struct ocxl_fn {
> + struct device dev;
> + int bar_used[3];
> + struct ocxl_fn_config config;
> + struct list_head afu_list;
> + int pasid_base;
> + int actag_base;
> + int actag_enabled;
> + int actag_supported;
> + struct list_head pasid_list;
> + struct list_head actag_list;
> + void *link;
> +};
> +
> +struct ocxl_afu {
> + struct ocxl_fn *fn;
> + struct list_head list;
> + struct device dev;
> + struct cdev cdev;
> + struct ocxl_afu_config config;
> + int pasid_base;
> + int pasid_count; /* opened contexts */
> + int pasid_max; /* maximum number of contexts */
> + int actag_base;
> + int actag_enabled;
> + struct mutex contexts_lock;
> + struct idr contexts_idr;
> + struct mutex afu_control_lock;
> + u64 global_mmio_start;
> + u64 irq_base_offset;
> + void __iomem *global_mmio_ptr;
> + u64 pp_mmio_start;
> + struct bin_attribute attr_global_mmio;
> +};
> +
> +enum ocxl_context_status {
> + CLOSED,
> + OPENED,
> + ATTACHED,
> +};
> +
> +// Contains metadata about a translation fault
> +struct ocxl_xsl_error {
> + u64 addr; // The address that triggered the fault
> + u64 dsisr; // the value of the dsisr register
> + u64 count; // The number of times this fault has been triggered
> +};
> +
> +struct ocxl_context {
> + struct ocxl_afu *afu;
> + int pasid;
> + struct mutex status_mutex;
> + enum ocxl_context_status status;
> + struct address_space *mapping;
> + struct mutex mapping_lock;
> + wait_queue_head_t events_wq;
> + struct mutex xsl_error_lock;
> + struct ocxl_xsl_error xsl_error;
> + struct mutex irq_lock;
> + struct idr irq_idr;
> +};
> +
> +struct ocxl_process_element {
> + u64 config_state;
> + u32 reserved1[11];
> + u32 lpid;
> + u32 tid;
> + u32 pid;
> + u32 reserved2[10];
> + u64 amr;
> + u32 reserved3[3];
> + u32 software_state;
> +};
> +
> +
> +extern struct ocxl_afu *ocxl_afu_get(struct ocxl_afu *afu);
> +extern void ocxl_afu_put(struct ocxl_afu *afu);
> +
> +extern int ocxl_create_cdev(struct ocxl_afu *afu);
> +extern void ocxl_destroy_cdev(struct ocxl_afu *afu);
> +extern int ocxl_register_afu(struct ocxl_afu *afu);
> +extern void ocxl_unregister_afu(struct ocxl_afu *afu);
> +
> +extern int ocxl_file_init(void);
> +extern void ocxl_file_exit(void);
> +
> +extern int ocxl_config_read_function(struct pci_dev *dev,
> + struct ocxl_fn_config *fn);
> +
> +extern int ocxl_config_check_afu_index(struct pci_dev *dev,
> + struct ocxl_fn_config *fn, int afu_idx);
> +extern int ocxl_config_read_afu(struct pci_dev *dev,
> + struct ocxl_fn_config *fn,
> + struct ocxl_afu_config *afu,
> + u8 afu_idx);
> +extern int ocxl_config_get_pasid_info(struct pci_dev *dev, int *count);
> +extern void ocxl_config_set_afu_pasid(struct pci_dev *dev,
> + int afu_control,
> + int pasid_base, u32 pasid_count_log);
> +extern int ocxl_config_get_actag_info(struct pci_dev *dev,
> + u16 *base, u16 *enabled, u16 *supported);
> +extern void ocxl_config_set_actag(struct pci_dev *dev, int func_dvsec,
> + u32 tag_first, u32 tag_count);
> +extern void ocxl_config_set_afu_actag(struct pci_dev *dev, int afu_control,
> + int actag_base, int actag_count);
> +extern void ocxl_config_set_afu_state(struct pci_dev *dev, int afu_control,
> + int enable);
> +extern int ocxl_config_set_TL(struct pci_dev *dev, int tl_dvsec);
> +extern int ocxl_config_terminate_pasid(struct pci_dev *dev, int afu_control,
> + int pasid);
> +
> +extern int ocxl_link_setup(struct pci_dev *dev, int PE_mask,
> + void **link_handle);
> +extern void ocxl_link_release(struct pci_dev *dev, void *link_handle);
> +extern int ocxl_link_add_pe(void *link_handle, int pasid, u32 pidr, u32 tidr,
> + u64 amr, struct mm_struct *mm,
> + void (*xsl_err_cb)(void *data, u64 addr, u64 dsisr),
> + void *xsl_err_data);
> +extern int ocxl_link_remove_pe(void *link_handle, int pasid);
> +extern int ocxl_link_irq_alloc(void *link_handle, int *hw_irq,
> + u64 *addr);
> +extern void ocxl_link_free_irq(void *link_handle, int hw_irq);
> +
> +extern int ocxl_pasid_afu_alloc(struct ocxl_fn *fn, u32 size);
> +extern void ocxl_pasid_afu_free(struct ocxl_fn *fn, u32 start, u32 size);
> +extern int ocxl_actag_afu_alloc(struct ocxl_fn *fn, u32 size);
> +extern void ocxl_actag_afu_free(struct ocxl_fn *fn, u32 start, u32 size);
> +
> +extern struct ocxl_context *ocxl_context_alloc(void);
> +extern int ocxl_context_init(struct ocxl_context *ctx, struct ocxl_afu *afu,
> + struct address_space *mapping);
> +extern int ocxl_context_attach(struct ocxl_context *ctx, u64 amr);
> +extern int ocxl_context_mmap(struct ocxl_context *ctx,
> + struct vm_area_struct *vma);
> +extern int ocxl_context_detach(struct ocxl_context *ctx);
> +extern void ocxl_context_detach_all(struct ocxl_afu *afu);
> +extern void ocxl_context_free(struct ocxl_context *ctx);
> +
> +extern int ocxl_sysfs_add_afu(struct ocxl_afu *afu);
> +extern void ocxl_sysfs_remove_afu(struct ocxl_afu *afu);
> +
> +#endif /* _OCXL_INTERNAL_H_ */
> diff --git a/drivers/misc/ocxl/pasid.c b/drivers/misc/ocxl/pasid.c
> new file mode 100644
> index 000000000000..ea999a3a99b4
> --- /dev/null
> +++ b/drivers/misc/ocxl/pasid.c
> @@ -0,0 +1,114 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include "ocxl_internal.h"
> +
> +
> +struct id_range {
> + struct list_head list;
> + u32 start;
> + u32 end;
> +};
> +
> +#ifdef DEBUG
> +static void dump_list(struct list_head *head, char *type_str)
> +{
> + struct id_range *cur;
> +
> + pr_debug("%s ranges allocated:\n", type_str);
> + list_for_each_entry(cur, head, list) {
> + pr_debug("Range %d->%d\n", cur->start, cur->end);
> + }
> +}
> +#endif
> +
> +static int range_alloc(struct list_head *head, u32 size, int max_id,
> + char *type_str)
> +{
> + struct list_head *pos;
> + struct id_range *cur, *new;
> + int rc, last_end;
> +
> + new = kmalloc(sizeof(struct id_range), GFP_KERNEL);
> + if (!new)
> + return -ENOMEM;
> +
> + pos = head;
> + last_end = -1;
> + list_for_each_entry(cur, head, list) {
> + if ((cur->start - last_end) > size)
> + break;
> + last_end = cur->end;
> + pos = &cur->list;
> + }
> +
> + new->start = last_end + 1;
> + new->end = new->start + size - 1;
> +
> + if (new->end > max_id) {
> + kfree(new);
> + rc = -ENOSPC;
> + } else {
> + list_add(&new->list, pos);
> + rc = new->start;
> + }
> +
> +#ifdef DEBUG
> + dump_list(head, type_str);
> +#endif
> + return rc;
> +}
> +
> +static void range_free(struct list_head *head, u32 start, u32 size,
> + char *type_str)
> +{
> + bool found = false;
> + struct id_range *cur, *tmp;
> +
> + list_for_each_entry_safe(cur, tmp, head, list) {
> + if (cur->start == start && cur->end == (start + size - 1)) {
> + found = true;
> + list_del(&cur->list);
> + kfree(cur);
> + break;
> + }
> + }
> + WARN_ON(!found);
> +#ifdef DEBUG
> + dump_list(head, type_str);
> +#endif
> +}
> +
> +int ocxl_pasid_afu_alloc(struct ocxl_fn *fn, u32 size)
> +{
> + int max_pasid;
> +
> + if (fn->config.max_pasid_log < 0)
> + return -ENOSPC;
> + max_pasid = 1 << fn->config.max_pasid_log;
> + return range_alloc(&fn->pasid_list, size, max_pasid, "afu pasid");
> +}
> +
> +void ocxl_pasid_afu_free(struct ocxl_fn *fn, u32 start, u32 size)
> +{
> + return range_free(&fn->pasid_list, start, size, "afu pasid");
> +}
> +
> +int ocxl_actag_afu_alloc(struct ocxl_fn *fn, u32 size)
> +{
> + int max_actag;
> +
> + max_actag = fn->actag_enabled;
> + return range_alloc(&fn->actag_list, size, max_actag, "afu actag");
> +}
> +
> +void ocxl_actag_afu_free(struct ocxl_fn *fn, u32 start, u32 size)
> +{
> + return range_free(&fn->actag_list, start, size, "afu actag");
> +}
> diff --git a/drivers/misc/ocxl/pci.c b/drivers/misc/ocxl/pci.c
> new file mode 100644
> index 000000000000..39e7bdd48215
> --- /dev/null
> +++ b/drivers/misc/ocxl/pci.c
> @@ -0,0 +1,592 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include <linux/idr.h>
> +#include <asm/pnv-ocxl.h>
> +#include "ocxl_internal.h"
> +
> +/*
> + * Any opencapi device which wants to use this 'generic' driver should
> + * use the 0x062B device ID. Vendors should define the subsystem
> + * vendor/device ID to help differentiate devices.
> + */
> +static const struct pci_device_id ocxl_pci_tbl[] = {
> + { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x062B), },
> + { }
> +};
> +MODULE_DEVICE_TABLE(pci, ocxl_pci_tbl);
> +
> +
> +static struct ocxl_fn *ocxl_fn_get(struct ocxl_fn *fn)
> +{
> + return (get_device(&fn->dev) == NULL) ? NULL : fn;
> +}
> +
> +static void ocxl_fn_put(struct ocxl_fn *fn)
> +{
> + put_device(&fn->dev);
> +}
> +
> +struct ocxl_afu *ocxl_afu_get(struct ocxl_afu *afu)
> +{
> + return (get_device(&afu->dev) == NULL) ? NULL : afu;
> +}
> +
> +void ocxl_afu_put(struct ocxl_afu *afu)
> +{
> + put_device(&afu->dev);
> +}
> +
> +static struct ocxl_afu *alloc_afu(struct ocxl_fn *fn)
> +{
> + struct ocxl_afu *afu;
> +
> + afu = kzalloc(sizeof(struct ocxl_afu), GFP_KERNEL);
> + if (!afu)
> + return NULL;
> +
> + mutex_init(&afu->contexts_lock);
> + mutex_init(&afu->afu_control_lock);
> + idr_init(&afu->contexts_idr);
> + afu->fn = fn;
> + ocxl_fn_get(fn);
> + return afu;
> +}
> +
> +static void free_afu(struct ocxl_afu *afu)
> +{
> + idr_destroy(&afu->contexts_idr);
> + ocxl_fn_put(afu->fn);
> + kfree(afu);
> +}
> +
> +static void free_afu_dev(struct device *dev)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(dev);
> +
> + ocxl_unregister_afu(afu);
> + free_afu(afu);
> +}
> +
> +static int set_afu_device(struct ocxl_afu *afu, const char *location)
> +{
> + struct ocxl_fn *fn = afu->fn;
> + int rc;
> +
> + afu->dev.parent = &fn->dev;
> + afu->dev.release = free_afu_dev;
> + rc = dev_set_name(&afu->dev, "%s.%s.%hhu", afu->config.name, location,
> + afu->config.idx);
> + return rc;
> +}
> +
> +static int assign_afu_actag(struct ocxl_afu *afu, struct pci_dev *dev)
> +{
> + struct ocxl_fn *fn = afu->fn;
> + int actag_count, actag_offset;
> +
> + /*
> + * if there were not enough actags for the function, each afu
> + * reduces its count as well
> + */
> + actag_count = afu->config.actag_supported *
> + fn->actag_enabled / fn->actag_supported;
> + actag_offset = ocxl_actag_afu_alloc(fn, actag_count);
> + if (actag_offset < 0) {
> + dev_err(&afu->dev, "Can't allocate %d actags for AFU: %d\n",
> + actag_count, actag_offset);
> + return actag_offset;
> + }
> + afu->actag_base = fn->actag_base + actag_offset;
> + afu->actag_enabled = actag_count;
> +
> + ocxl_config_set_afu_actag(dev, afu->config.dvsec_afu_control_pos,
> + afu->actag_base, afu->actag_enabled);
> + dev_dbg(&afu->dev, "actag base=%d enabled=%d\n",
> + afu->actag_base, afu->actag_enabled);
> + return 0;
> +}
> +
> +static void reclaim_afu_actag(struct ocxl_afu *afu)
> +{
> + struct ocxl_fn *fn = afu->fn;
> + int start_offset, size;
> +
> + start_offset = afu->actag_base - fn->actag_base;
> + size = afu->actag_enabled;
> + ocxl_actag_afu_free(afu->fn, start_offset, size);
> +}
> +
> +static int assign_afu_pasid(struct ocxl_afu *afu, struct pci_dev *dev)
> +{
> + struct ocxl_fn *fn = afu->fn;
> + int pasid_count, pasid_offset;
> +
> + /*
> + * We only support the case where the function configuration
> + * requested enough PASIDs to cover all AFUs.
> + */
> + pasid_count = 1 << afu->config.pasid_supported_log;
> + pasid_offset = ocxl_pasid_afu_alloc(fn, pasid_count);
> + if (pasid_offset < 0) {
> + dev_err(&afu->dev, "Can't allocate %d PASIDs for AFU: %d\n",
> + pasid_count, pasid_offset);
> + return pasid_offset;
> + }
> + afu->pasid_base = fn->pasid_base + pasid_offset;
> + afu->pasid_count = 0;
> + afu->pasid_max = pasid_count;
> +
> + ocxl_config_set_afu_pasid(dev, afu->config.dvsec_afu_control_pos,
> + afu->pasid_base,
> + afu->config.pasid_supported_log);
> + dev_dbg(&afu->dev, "PASID base=%d, enabled=%d\n",
> + afu->pasid_base, pasid_count);
> + return 0;
> +}
> +
> +static void reclaim_afu_pasid(struct ocxl_afu *afu)
> +{
> + struct ocxl_fn *fn = afu->fn;
> + int start_offset, size;
> +
> + start_offset = afu->pasid_base - fn->pasid_base;
> + size = 1 << afu->config.pasid_supported_log;
> + ocxl_pasid_afu_free(afu->fn, start_offset, size);
> +}
> +
> +static int reserve_fn_bar(struct ocxl_fn *fn, int bar)
> +{
> + struct pci_dev *dev = to_pci_dev(fn->dev.parent);
> + int rc, idx;
> +
> + if (bar != 0 && bar != 2 && bar != 4)
> + return -EINVAL;
> +
> + idx = bar >> 1;
> + if (fn->bar_used[idx]++ == 0) {
> + rc = pci_request_region(dev, bar, "ocxl");
> + if (rc)
> + return rc;
> + }
> + return 0;
> +}
> +
> +static void release_fn_bar(struct ocxl_fn *fn, int bar)
> +{
> + struct pci_dev *dev = to_pci_dev(fn->dev.parent);
> + int idx;
> +
> + if (bar != 0 && bar != 2 && bar != 4)
> + return;
> +
> + idx = bar >> 1;
> + if (--fn->bar_used[idx] == 0)
> + pci_release_region(dev, bar);
> + WARN_ON(fn->bar_used[idx] < 0);
> +}
> +
> +static int map_mmio_areas(struct ocxl_afu *afu, struct pci_dev *dev)
> +{
> + int rc;
> +
> + rc = reserve_fn_bar(afu->fn, afu->config.global_mmio_bar);
> + if (rc)
> + return rc;
> +
> + rc = reserve_fn_bar(afu->fn, afu->config.pp_mmio_bar);
> + if (rc) {
> + release_fn_bar(afu->fn, afu->config.global_mmio_bar);
> + return rc;
> + }
> +
> + afu->global_mmio_start =
> + pci_resource_start(dev, afu->config.global_mmio_bar) +
> + afu->config.global_mmio_offset;
> + afu->pp_mmio_start =
> + pci_resource_start(dev, afu->config.pp_mmio_bar) +
> + afu->config.pp_mmio_offset;
> +
> + afu->global_mmio_ptr = ioremap(afu->global_mmio_start,
> + afu->config.global_mmio_size);
> + if (!afu->global_mmio_ptr) {
> + release_fn_bar(afu->fn, afu->config.pp_mmio_bar);
> + release_fn_bar(afu->fn, afu->config.global_mmio_bar);
> + dev_err(&dev->dev, "Error mapping global mmio area\n");
> + return -ENOMEM;
> + }
> +
> + /*
> + * Leave an empty page between the per-process mmio area and
> + * the AFU interrupt mappings
> + */
> + afu->irq_base_offset = afu->config.pp_mmio_stride + PAGE_SIZE;
> + return 0;
> +}
> +
> +static void unmap_mmio_areas(struct ocxl_afu *afu)
> +{
> + if (afu->global_mmio_ptr) {
> + iounmap(afu->global_mmio_ptr);
> + afu->global_mmio_ptr = NULL;
> + }
> + afu->global_mmio_start = 0;
> + afu->pp_mmio_start = 0;
> + release_fn_bar(afu->fn, afu->config.pp_mmio_bar);
> + release_fn_bar(afu->fn, afu->config.global_mmio_bar);
> +}
> +
> +static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev)
> +{
> + int rc;
> +
> + rc = ocxl_config_read_afu(dev, &afu->fn->config, &afu->config, afu_idx);
> + if (rc)
> + return rc;
> +
> + rc = set_afu_device(afu, dev_name(&dev->dev));
> + if (rc)
> + return rc;
> +
> + rc = assign_afu_actag(afu, dev);
> + if (rc)
> + return rc;
> +
> + rc = assign_afu_pasid(afu, dev);
> + if (rc) {
> + reclaim_afu_actag(afu);
> + return rc;
> + }
> +
> + rc = map_mmio_areas(afu, dev);
> + if (rc) {
> + reclaim_afu_pasid(afu);
> + reclaim_afu_actag(afu);
> + return rc;
> + }
> + return 0;
> +}
> +
> +static void deconfigure_afu(struct ocxl_afu *afu)
> +{
> + unmap_mmio_areas(afu);
> + reclaim_afu_pasid(afu);
> + reclaim_afu_actag(afu);
> +}
> +
> +static int activate_afu(struct pci_dev *dev, struct ocxl_afu *afu)
> +{
> + int rc;
> +
> + ocxl_config_set_afu_state(dev, afu->config.dvsec_afu_control_pos, 1);
> + /*
> + * Char device creation is the last step, as processes can
> + * call our driver immediately, so all our inits must be finished.
> + */
> + rc = ocxl_create_cdev(afu);
> + if (rc)
> + return rc;
> + return 0;
> +}
> +
> +static void deactivate_afu(struct ocxl_afu *afu)
> +{
> + struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent);
> +
> + ocxl_destroy_cdev(afu);
> + ocxl_config_set_afu_state(dev, afu->config.dvsec_afu_control_pos, 0);
> +}
> +
> +static int init_afu(struct pci_dev *dev, struct ocxl_fn *fn, u8 afu_idx)
> +{
> + int rc;
> + struct ocxl_afu *afu;
> +
> + afu = alloc_afu(fn);
> + if (!afu)
> + return -ENOMEM;
> +
> + rc = configure_afu(afu, afu_idx, dev);
> + if (rc) {
> + free_afu(afu);
> + return rc;
> + }
> +
> + rc = ocxl_register_afu(afu);
> + if (rc)
> + goto err;
> +
> + rc = ocxl_sysfs_add_afu(afu);
> + if (rc)
> + goto err;
> +
> + rc = activate_afu(dev, afu);
> + if (rc)
> + goto err_sys;
> +
> + list_add_tail(&afu->list, &fn->afu_list);
> + return 0;
> +
> +err_sys:
> + ocxl_sysfs_remove_afu(afu);
> +err:
> + deconfigure_afu(afu);
> + device_unregister(&afu->dev);
> + return rc;
> +}
> +
> +static void remove_afu(struct ocxl_afu *afu)
> +{
> + list_del(&afu->list);
> + ocxl_context_detach_all(afu);
> + deactivate_afu(afu);
> + ocxl_sysfs_remove_afu(afu);
> + deconfigure_afu(afu);
> + device_unregister(&afu->dev);
> +}
> +
> +static struct ocxl_fn *alloc_function(struct pci_dev *dev)
> +{
> + struct ocxl_fn *fn;
> +
> + fn = kzalloc(sizeof(struct ocxl_fn), GFP_KERNEL);
> + if (!fn)
> + return NULL;
> +
> + INIT_LIST_HEAD(&fn->afu_list);
> + INIT_LIST_HEAD(&fn->pasid_list);
> + INIT_LIST_HEAD(&fn->actag_list);
> + return fn;
> +}
> +
> +static void free_function(struct ocxl_fn *fn)
> +{
> + WARN_ON(!list_empty(&fn->afu_list));
> + WARN_ON(!list_empty(&fn->pasid_list));
> + kfree(fn);
> +}
> +
> +static void free_function_dev(struct device *dev)
> +{
> + struct ocxl_fn *fn = to_ocxl_function(dev);
> +
> + free_function(fn);
> +}
> +
> +static int set_function_device(struct ocxl_fn *fn, struct pci_dev *dev)
> +{
> + int rc;
> +
> + fn->dev.parent = &dev->dev;
> + fn->dev.release = free_function_dev;
> + rc = dev_set_name(&fn->dev, "ocxlfn.%s", dev_name(&dev->dev));
> + if (rc)
> + return rc;
> + pci_set_drvdata(dev, fn);
> + return 0;
> +}
> +
> +static int assign_function_actag(struct ocxl_fn *fn)
> +{
> + struct pci_dev *dev = to_pci_dev(fn->dev.parent);
> + u16 base, enabled, supported;
> + int rc;
> +
> + rc = ocxl_config_get_actag_info(dev, &base, &enabled, &supported);
> + if (rc)
> + return rc;
> +
> + fn->actag_base = base;
> + fn->actag_enabled = enabled;
> + fn->actag_supported = supported;
> +
> + ocxl_config_set_actag(dev, fn->config.dvsec_function_pos,
> + fn->actag_base, fn->actag_enabled);
> + dev_dbg(&fn->dev, "actag range starting at %d, enabled %d\n",
> + fn->actag_base, fn->actag_enabled);
> + return 0;
> +}
> +
> +static int set_function_pasid(struct ocxl_fn *fn)
> +{
> + struct pci_dev *dev = to_pci_dev(fn->dev.parent);
> + int rc, desired_count, max_count;
> +
> + /* A function may not require any PASID */
> + if (fn->config.max_pasid_log < 0)
> + return 0;
> +
> + rc = ocxl_config_get_pasid_info(dev, &max_count);
> + if (rc)
> + return rc;
> +
> + desired_count = 1 << fn->config.max_pasid_log;
> +
> + if (desired_count > max_count) {
> + dev_err(&fn->dev,
> + "Function requires more PASIDs than is available (%d vs. %d)\n",
> + desired_count, max_count);
> + return -ENOSPC;
> + }
> +
> + fn->pasid_base = 0;
> + return 0;
> +}
> +
> +static int configure_function(struct ocxl_fn *fn, struct pci_dev *dev)
> +{
> + int rc;
> +
> + rc = pci_enable_device(dev);
> + if (rc) {
> + dev_err(&dev->dev, "pci_enable_device failed: %d\n", rc);
> + return rc;
> + }
> +
> + /*
> + * Once it has been confirmed to work on our hardware, we
> + * should reset the function, to force the adapter to restart
> + * from scratch.
> + * A function reset would also reset all its AFUs.
> + *
> + * Some hints for implementation:
> + *
> + * - there's not status bit to know when the reset is done. We
> + * should try reading the config space to know when it's
> + * done.
> + * - probably something like:
> + * Reset
> + * wait 100ms
> + * issue config read
> + * allow device up to 1 sec to return success on config
> + * read before declaring it broken
> + *
> + * Some shared logic on the card (CFG, TLX) won't be reset, so
> + * there's no guarantee that it will be enough.
> + */
> + rc = ocxl_config_read_function(dev, &fn->config);
> + if (rc)
> + return rc;
> +
> + rc = set_function_device(fn, dev);
> + if (rc)
> + return rc;
> +
> + rc = assign_function_actag(fn);
> + if (rc)
> + return rc;
> +
> + rc = set_function_pasid(fn);
> + if (rc)
> + return rc;
> +
> + rc = ocxl_link_setup(dev, 0, &fn->link);
> + if (rc)
> + return rc;
> +
> + rc = ocxl_config_set_TL(dev, fn->config.dvsec_tl_pos);
> + if (rc) {
> + ocxl_link_release(dev, fn->link);
> + return rc;
> + }
> + return 0;
> +}
> +
> +static void deconfigure_function(struct ocxl_fn *fn)
> +{
> + struct pci_dev *dev = to_pci_dev(fn->dev.parent);
> +
> + ocxl_link_release(dev, fn->link);
> + pci_disable_device(dev);
> +}
> +
> +static struct ocxl_fn *init_function(struct pci_dev *dev)
> +{
> + struct ocxl_fn *fn;
> + int rc;
> +
> + fn = alloc_function(dev);
> + if (!fn)
> + return ERR_PTR(-ENOMEM);
> +
> + rc = configure_function(fn, dev);
> + if (rc) {
> + free_function(fn);
> + return ERR_PTR(rc);
> + }
> +
> + rc = device_register(&fn->dev);
> + if (rc) {
> + deconfigure_function(fn);
> + device_unregister(&fn->dev);
> + return ERR_PTR(rc);
> + }
> + return fn;
> +}
> +
> +static void remove_function(struct ocxl_fn *fn)
> +{
> + deconfigure_function(fn);
> + device_unregister(&fn->dev);
> +}
> +
> +static int ocxl_probe(struct pci_dev *dev, const struct pci_device_id *id)
> +{
> + int rc, afu_count = 0;
> + u8 afu;
> + struct ocxl_fn *fn;
> +
> + if (!radix_enabled()) {
> + dev_err(&dev->dev, "Unsupported memory model (hash)\n");
> + return -ENODEV;
> + }
> +
> + fn = init_function(dev);
> + if (IS_ERR(fn)) {
> + dev_err(&dev->dev, "function init failed: %li\n",
> + PTR_ERR(fn));
> + return PTR_ERR(fn);
> + }
> +
> + for (afu = 0; afu <= fn->config.max_afu_index; afu++) {
> + rc = ocxl_config_check_afu_index(dev, &fn->config, afu);
> + if (rc > 0) {
> + rc = init_afu(dev, fn, afu);
> + if (rc) {
> + dev_err(&dev->dev,
> + "Can't initialize AFU index %d\n", afu);
> + continue;
> + }
> + afu_count++;
> + }
> + }
> + dev_info(&dev->dev, "%d AFU(s) configured\n", afu_count);
> + return 0;
> +}
> +
> +static void ocxl_remove(struct pci_dev *dev)
> +{
> + struct ocxl_afu *afu, *tmp;
> + struct ocxl_fn *fn = pci_get_drvdata(dev);
> +
> + list_for_each_entry_safe(afu, tmp, &fn->afu_list, list) {
> + remove_afu(afu);
> + }
> + remove_function(fn);
> +}
> +
> +struct pci_driver ocxl_pci_driver = {
> + .name = "ocxl",
> + .id_table = ocxl_pci_tbl,
> + .probe = ocxl_probe,
> + .remove = ocxl_remove,
> + .shutdown = ocxl_remove,
> +};
> diff --git a/drivers/misc/ocxl/sysfs.c b/drivers/misc/ocxl/sysfs.c
> new file mode 100644
> index 000000000000..b7b1d1735c07
> --- /dev/null
> +++ b/drivers/misc/ocxl/sysfs.c
> @@ -0,0 +1,150 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/sysfs.h>
> +#include "ocxl_internal.h"
> +
> +static ssize_t global_mmio_size_show(struct device *device,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(device);
> +
> + return scnprintf(buf, PAGE_SIZE, "%d\n",
> + afu->config.global_mmio_size);
> +}
> +
> +static ssize_t pp_mmio_size_show(struct device *device,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(device);
> +
> + return scnprintf(buf, PAGE_SIZE, "%d\n",
> + afu->config.pp_mmio_stride);
> +}
> +
> +static ssize_t afu_version_show(struct device *device,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(device);
> +
> + return scnprintf(buf, PAGE_SIZE, "%hhu:%hhu\n",
> + afu->config.version_major,
> + afu->config.version_minor);
> +}
> +
> +static ssize_t contexts_show(struct device *device,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(device);
> +
> + return scnprintf(buf, PAGE_SIZE, "%d/%d\n",
> + afu->pasid_count, afu->pasid_max);
> +}
> +
> +static struct device_attribute afu_attrs[] = {
> + __ATTR_RO(global_mmio_size),
> + __ATTR_RO(pp_mmio_size),
> + __ATTR_RO(afu_version),
> + __ATTR_RO(contexts),
> +};
> +
> +static ssize_t global_mmio_read(struct file *filp, struct kobject *kobj,
> + struct bin_attribute *bin_attr, char *buf,
> + loff_t off, size_t count)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(kobj_to_dev(kobj));
> +
> + if (count == 0 || off < 0 ||
> + off >= afu->config.global_mmio_size)
> + return 0;
> +
> + memcpy(buf, afu->global_mmio_ptr + off, count);
drivers/misc/ocxl/sysfs.c:64:42: warning: incorrect type in argument 2
(different address spaces)
drivers/misc/ocxl/sysfs.c:64:42: expected void const *<noident>
drivers/misc/ocxl/sysfs.c:64:42: got void [noderef] <asn:2>*
> + return count;
> +}
> +
> +static int global_mmio_fault(struct vm_fault *vmf)
> +{
> + struct vm_area_struct *vma = vmf->vma;
> + struct ocxl_afu *afu = vma->vm_private_data;
> + unsigned long offset;
> +
> + if (vmf->pgoff >= (afu->config.global_mmio_size >> PAGE_SHIFT))
> + return VM_FAULT_SIGBUS;
> +
> + offset = vmf->pgoff;
> + offset += (afu->global_mmio_start >> PAGE_SHIFT);
> + vm_insert_pfn(vma, vmf->address, offset);
> + return VM_FAULT_NOPAGE;
> +}
> +
> +static const struct vm_operations_struct global_mmio_vmops = {
> + .fault = global_mmio_fault,
> +};
> +
> +static int global_mmio_mmap(struct file *filp, struct kobject *kobj,
> + struct bin_attribute *bin_attr,
> + struct vm_area_struct *vma)
> +{
> + struct ocxl_afu *afu = to_ocxl_afu(kobj_to_dev(kobj));
> +
> + if ((vma_pages(vma) + vma->vm_pgoff) >
> + (afu->config.global_mmio_size >> PAGE_SHIFT))
> + return -EINVAL;
> +
> + vma->vm_flags |= VM_IO | VM_PFNMAP;
> + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> + vma->vm_ops = &global_mmio_vmops;
> + vma->vm_private_data = afu;
> + return 0;
> +}
> +
> +int ocxl_sysfs_add_afu(struct ocxl_afu *afu)
> +{
> + int i, rc;
> +
> + for (i = 0; i < ARRAY_SIZE(afu_attrs); i++) {
> + rc = device_create_file(&afu->dev, &afu_attrs[i]);
> + if (rc)
> + goto err;
> + }
> +
> + sysfs_attr_init(&afu->attr_global_mmio.attr);
> + afu->attr_global_mmio.attr.name = "global_mmio_area";
> + afu->attr_global_mmio.attr.mode = 0600;
> + afu->attr_global_mmio.size = afu->config.global_mmio_size;
> + afu->attr_global_mmio.read = global_mmio_read;
> + afu->attr_global_mmio.mmap = global_mmio_mmap;
> + rc = device_create_bin_file(&afu->dev, &afu->attr_global_mmio);
> + if (rc) {
> + dev_err(&afu->dev,
> + "Unable to create global mmio attr for afu: %d\n",
> + rc);
> + goto err;
> + }
> +
> + return 0;
> +
> +err:
> + for (i--; i >= 0; i--)
> + device_remove_file(&afu->dev, &afu_attrs[i]);
> + return rc;
> +}
> +
> +void ocxl_sysfs_remove_afu(struct ocxl_afu *afu)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(afu_attrs); i++)
> + device_remove_file(&afu->dev, &afu_attrs[i]);
> + device_remove_bin_file(&afu->dev, &afu->attr_global_mmio);
> +}
> diff --git a/include/uapi/misc/ocxl.h b/include/uapi/misc/ocxl.h
> new file mode 100644
> index 000000000000..71fa387f2efd
> --- /dev/null
> +++ b/include/uapi/misc/ocxl.h
> @@ -0,0 +1,47 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#ifndef _UAPI_MISC_OCXL_H
> +#define _UAPI_MISC_OCXL_H
> +
> +#include <linux/types.h>
> +#include <linux/ioctl.h>
> +
> +enum ocxl_event_type {
> + OCXL_AFU_EVENT_XSL_FAULT_ERROR = 0,
> +};
> +
> +#define OCXL_KERNEL_EVENT_FLAG_LAST 0x0001 /* This is the last event pending */
> +
> +struct ocxl_kernel_event_header {
> + __u16 type;
> + __u16 flags;
> + __u32 reserved;
> +};
> +
> +struct ocxl_kernel_event_xsl_fault_error {
> + __u64 addr;
> + __u64 dsisr;
> + __u64 count;
> + __u64 reserved;
> +};
> +
> +struct ocxl_ioctl_attach {
> + __u64 amr;
> + __u64 reserved1;
> + __u64 reserved2;
> + __u64 reserved3;
> +};
> +
> +/* ioctl numbers */
> +#define OCXL_MAGIC 0xCA
> +/* AFU devices */
> +#define OCXL_IOCTL_ATTACH _IOW(OCXL_MAGIC, 0x10, struct ocxl_ioctl_attach)
> +
> +#endif /* _UAPI_MISC_OCXL_H */
>
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com IBM Australia Limited
^ permalink raw reply
* Re: [PATCH 04/13] powerpc/powernv: Add platform-specific services for opencapi
From: Andrew Donnellan @ 2018-01-03 7:31 UTC (permalink / raw)
To: Frederic Barrat, linuxppc-dev, linux-kernel; +Cc: arnd, gregkh, mpe, alastair
In-Reply-To: <daaa7e08af012047baed61a79ed5dc62e5738766.1513608243.git.fbarrat@linux.vnet.ibm.com>
On 19/12/17 02:21, Frederic Barrat wrote:
> Implement a few platform-specific calls which can be used by drivers:
>
> - provide the Transaction Layer capabilities of the host, so that the
> driver can find some common ground and configure the device and host
> appropriately.
>
> - provide the hw interrupt to be used for translation faults raised by
> the NPU
>
> - map/unmap some NPU mmio registers to get the fault context when the
> NPU raises an address translation fault
>
> The rest are wrappers around the previously-introduced opal calls.
>
>
> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
> ---
> arch/powerpc/include/asm/pnv-ocxl.h | 36 ++++++
> arch/powerpc/platforms/powernv/Makefile | 1 +
> arch/powerpc/platforms/powernv/ocxl.c | 187 ++++++++++++++++++++++++++++++++
> 3 files changed, 224 insertions(+)
> create mode 100644 arch/powerpc/include/asm/pnv-ocxl.h
> create mode 100644 arch/powerpc/platforms/powernv/ocxl.c
>
> diff --git a/arch/powerpc/include/asm/pnv-ocxl.h b/arch/powerpc/include/asm/pnv-ocxl.h
> new file mode 100644
> index 000000000000..b9ab3f0a9634
> --- /dev/null
> +++ b/arch/powerpc/include/asm/pnv-ocxl.h
> @@ -0,0 +1,36 @@
> +/*
> + * Copyright 2017 IBM Corp.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#ifndef _ASM_PVN_OCXL_H
> +#define _ASM_PVN_OCXL_H
I assume you meant "PNV" here.
> +
> +#include <linux/pci.h>
> +
> +#define PNV_OCXL_TL_MAX_TEMPLATE 63
> +#define PNV_OCXL_TL_BITS_PER_RATE 4
> +#define PNV_OCXL_TL_RATE_BUF_SIZE ((PNV_OCXL_TL_MAX_TEMPLATE+1) * PNV_OCXL_TL_BITS_PER_RATE / 8)
> +
> +extern int pnv_ocxl_get_tl_cap(struct pci_dev *dev, long *cap,
> + char *rate_buf, int rate_buf_size);
> +extern int pnv_ocxl_set_tl_conf(struct pci_dev *dev, long cap,
> + uint64_t rate_buf_phys, int rate_buf_size);
> +
> +extern int pnv_ocxl_get_xsl_irq(struct pci_dev *dev, int *hwirq);
> +extern void pnv_ocxl_unmap_xsl_regs(void __iomem *dsisr, void __iomem *dar,
> + void __iomem *tfc, void __iomem *pe_handle);
> +extern int pnv_ocxl_map_xsl_regs(struct pci_dev *dev, void __iomem **dsisr,
> + void __iomem **dar, void __iomem **tfc,
> + void __iomem **pe_handle);
> +
> +extern int pnv_ocxl_spa_setup(struct pci_dev *dev, void *spa_mem, int PE_mask,
> + void **platform_data);
> +extern void pnv_ocxl_spa_release(void *platform_data);
> +extern int pnv_ocxl_spa_remove_pe(void *platform_data, int pe_handle);
> +
> +#endif /* _ASM_PVN_OCXL_H */
And here
> diff --git a/arch/powerpc/platforms/powernv/Makefile b/arch/powerpc/platforms/powernv/Makefile
> index 3732118a0482..6c9d5199a7e2 100644
> --- a/arch/powerpc/platforms/powernv/Makefile
> +++ b/arch/powerpc/platforms/powernv/Makefile
> @@ -17,3 +17,4 @@ obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
> obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
> obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
> obj-$(CONFIG_PPC_FTW) += nx-ftw.o
> +obj-$(CONFIG_OCXL_BASE) += ocxl.o
> diff --git a/arch/powerpc/platforms/powernv/ocxl.c b/arch/powerpc/platforms/powernv/ocxl.c
> new file mode 100644
> index 000000000000..3378b75cf5e5
> --- /dev/null
> +++ b/arch/powerpc/platforms/powernv/ocxl.c
> +int pnv_ocxl_get_xsl_irq(struct pci_dev *dev, int *hwirq)
> +{
> + int rc;
> +
> + rc = of_property_read_u32(dev->dev.of_node, "ibm,opal-xsl-irq", hwirq);
> + if (rc) {
> + dev_err(&dev->dev,
> + "Can't translation xsl interrupt for device\n");
Can't get?
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com IBM Australia Limited
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox