[GIT PULL] Locking: Add the Kernel Concurrency Sanitizer (KCSAN) subsystem

Ingo Molnar
2019-11-25T12:11:39Z

Linus,

Please pull the latest locking-kcsan-for-linus git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking-kcsan-for-linus

# HEAD: 5cbaefe9743bf14c9d3106db0cc19f8cb0a3ca22 kcsan: Improve various small stylistic details

This tree adds the Kernel Concurrency Sanitizer (KCSAN) subsystem
authored by Marco Elver, which is a debugging facility that uses compiler
instrumentation of data accesses to statistically detect and report data
races on live kernels.

KCSAN uses the -fsanitize=thread build time instrumentation features of
both GCC and Clang, which transforms all memory reads/writes into
__tsan_*callbacks with addresses and access type flags passed in that
KCSAN can process and turn into a global array of 'watchpoints' that
denote ongoing accesses. If two CPUs happen upon each other via an unsafe
(non-atomic) access then a warning is generated.

Most users probably don't want to enable CONFIG_KCSAN due to the
significant runtime overhead, bloat and the intentional udelay()s added
to widen data races:

text data bss
19769486 5201776 1613896 vmlinux.defconfig
30181608 5313912 1675336 vmlinux.defconfig.CONFIG_KCSAN=y

... but for kernel developers it's a powerful facility that has already
found a number of concurrency bugs.

There's a significant set of false positives and early and low level code
complications that require the whitelisting and blacklisting of various
pieces of kernel code - but in practice the code is already functional
enough to find races.

Thanks,

Ingo

------------------>
Ingo Molnar (1):
kcsan: Improve various small stylistic details

Marco Elver (9):
kcsan: Add Kernel Concurrency Sanitizer infrastructure
include/linux/compiler.h: Introduce data_race(expr) macro
kcsan: Add Documentation entry in dev-tools
objtool, kcsan: Add KCSAN runtime functions to whitelist
build, kcsan: Add KCSAN build exceptions
seqlock, kcsan: Add annotations for KCSAN
seqlock: Require WRITE_ONCE surrounding raw_seqcount_barrier
locking/atomics, kcsan: Add KCSAN instrumentation
x86, kcsan: Enable KCSAN for x86

Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kcsan.rst | 256 ++++++++++++
MAINTAINERS | 11 +
Makefile | 3 +-
arch/x86/Kconfig | 1 +
arch/x86/boot/Makefile | 2 +
arch/x86/boot/compressed/Makefile | 2 +
arch/x86/entry/vdso/Makefile | 3 +
arch/x86/include/asm/bitops.h | 6 +-
arch/x86/kernel/Makefile | 4 +
arch/x86/kernel/cpu/Makefile | 3 +
arch/x86/lib/Makefile | 4 +
arch/x86/mm/Makefile | 4 +
arch/x86/purgatory/Makefile | 2 +
arch/x86/realmode/Makefile | 3 +
arch/x86/realmode/rm/Makefile | 3 +
drivers/firmware/efi/libstub/Makefile | 2 +
include/asm-generic/atomic-instrumented.h | 393 ++++++++++---------
include/linux/compiler-clang.h | 11 +-
include/linux/compiler-gcc.h | 7 +
include/linux/compiler.h | 57 ++-
include/linux/kcsan-checks.h | 93 +++++
include/linux/kcsan.h | 108 ++++++
include/linux/sched.h | 4 +
include/linux/seqlock.h | 51 ++-
init/init_task.c | 8 +
init/main.c | 2 +
kernel/Makefile | 6 +
kernel/kcsan/Makefile | 11 +
kernel/kcsan/atomic.h | 27 ++
kernel/kcsan/core.c | 621 ++++++++++++++++++++++++++++++
kernel/kcsan/debugfs.c | 271 +++++++++++++
kernel/kcsan/encoding.h | 95 +++++
kernel/kcsan/kcsan.h | 109 ++++++
kernel/kcsan/report.c | 318 +++++++++++++++
kernel/kcsan/test.c | 121 ++++++
kernel/sched/Makefile | 6 +
lib/Kconfig.debug | 2 +
lib/Kconfig.kcsan | 116 ++++++
lib/Makefile | 3 +
mm/Makefile | 8 +
scripts/Makefile.kcsan | 6 +
scripts/Makefile.lib | 10 +
scripts/atomic/gen-atomic-instrumented.sh | 17 +-
tools/objtool/check.c | 18 +
45 files changed, 2602 insertions(+), 207 deletions(-)
create mode 100644 Documentation/dev-tools/kcsan.rst
create mode 100644 include/linux/kcsan-checks.h
create mode 100644 include/linux/kcsan.h
create mode 100644 kernel/kcsan/Makefile
create mode 100644 kernel/kcsan/atomic.h
create mode 100644 kernel/kcsan/core.c
create mode 100644 kernel/kcsan/debugfs.c
create mode 100644 kernel/kcsan/encoding.h
create mode 100644 kernel/kcsan/kcsan.h
create mode 100644 kernel/kcsan/report.c
create mode 100644 kernel/kcsan/test.c
create mode 100644 lib/Kconfig.kcsan
create mode 100644 scripts/Makefile.kcsan