From: Nathan Chancellor <natechancellor@gmail.com>
To: Mark Rutland <mark.rutland@arm.com>
Cc: clang-built-linux@googlegroups.com,
Nick Desaulniers <ndesaulniers@google.com>,
linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: Enabling UBSAN breaks KCOV in clang (8.0.*) on arm64
Date: Tue, 20 Aug 2019 00:28:09 -0700 [thread overview]
Message-ID: <20190820072809.GA62296@archlinux-threadripper> (raw)
In-Reply-To: <20190819165947.GA30292@lakrids.cambridge.arm.com>
On Mon, Aug 19, 2019 at 05:59:48PM +0100, Mark Rutland wrote:
> Hi,
>
> I found that when I enable both KCOV and UBSAN on arm64, clang fails to
> emit any __sanitizer_cov_trace_*() calls in the resulting binary,
> rendering KCOV useless.
>
> For example, when building v5.3-rc3's arch/arm64/kernel/setup.o:
>
> * With defconfig + CONFIG KCOV:
>
> clang -Wp,-MD,arch/arm64/kernel/.setup.o.d -nostdinc -isystem
> /mnt/data/opt/toolchain/llvm/8.0.0/clang+llvm-8.0.0-x86_64-linux-sles11.3/lib/clang/8.0.0/include
> -I./arch/arm64/include -I./arch/arm64/include/generated -I./include
> -I./arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi
> -I./include/uapi -I./include/generated/uapi -include
> ./include/linux/kconfig.h -include ./include/linux/compiler_types.h
> -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3
> -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes
> -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE
> -Werror=implicit-function-declaration -Werror=implicit-int
> -Wno-format-security -std=gnu89 --target=aarch64-linux
> --prefix=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux/bin/
> --gcc-toolchain=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux
> -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only
> -DCONFIG_AS_LSE=1 -fno-asynchronous-unwind-tables
> -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks
> -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048
> -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu
> -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable
> -fno-omit-frame-pointer -fno-optimize-sibling-calls -g
> -Wdeclaration-after-statement -Wvla -Wno-pointer-sign
> -fno-strict-overflow -fno-merge-all-constants -fno-stack-check
> -Werror=date-time -Werror=incompatible-pointer-types
> -Wno-initializer-overrides -Wno-format -Wno-sign-compare
> -Wno-format-zero-length -fsanitize-coverage=trace-pc
> -DKBUILD_BASENAME='"setup"' -DKBUILD_MODNAME='"setup"' -c -o
> arch/arm64/kernel/setup.o arch/arm64/kernel/setup.c
>
> ... and there are 44 calls to __sanitizer_cov_trace_pc in the
> resulting setup.o
>
> * with defconfig + CONFIG_KCOV + CONFIG_UBSAN:
>
> clang -Wp,-MD,arch/arm64/kernel/.setup.o.d -nostdinc -isystem
> /mnt/data/opt/toolchain/llvm/8.0.0/clang+llvm-8.0.0-x86_64-linux-sles11.3/lib/clang/8.0.0/include
> -I./arch/arm64/include -I./arch/arm64/include/generated -I./include
> -I./arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi
> -I./include/uapi -I./include/generated/uapi -include
> ./include/linux/kconfig.h -include ./include/linux/compiler_types.h
> -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3
> -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes
> -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE
> -Werror=implicit-function-declaration -Werror=implicit-int
> -Wno-format-security -std=gnu89 --target=aarch64-linux
> --prefix=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux/bin/
> --gcc-toolchain=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux
> -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only
> -DCONFIG_AS_LSE=1 -fno-asynchronous-unwind-tables
> -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks
> -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048
> -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu
> -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable
> -fno-omit-frame-pointer -fno-optimize-sibling-calls -g
> -Wdeclaration-after-statement -Wvla -Wno-pointer-sign
> -fno-strict-overflow -fno-merge-all-constants -fno-stack-check
> -Werror=date-time -Werror=incompatible-pointer-types
> -Wno-initializer-overrides -Wno-format -Wno-sign-compare
> -Wno-format-zero-length -fsanitize=shift
> -fsanitize=integer-divide-by-zero -fsanitize=unreachable
> -fsanitize=signed-integer-overflow -fsanitize=bounds
> -fsanitize=object-size -fsanitize=bool -fsanitize=enum
> -fsanitize-coverage=trace-pc -DKBUILD_BASENAME='"setup"'
> -DKBUILD_MODNAME='"setup"' -c -o arch/arm64/kernel/setup.o
> arch/arm64/kernel/setup.c
>
> ... and there are 0 calls to __sanitizer_cov_trace_pc in the resulting
> setup.o, even though -fsanitize-coverage=trace-pc was passed to clang.
>
> If I remove -fsanitize=bounds, there are 121 calls to
> __sanitizer_cov_trace_pc in setup.o. Removing the other options enabled
> by UBSAN didn't have any effect on setup.o.
>
> I'm using the llvm.org 8.0.{0,1} binaries [1,2], along with the
> kernel.org crosstool 8.1.0 binaries [3].
>
> Any ideas as to what's going on?
>
> Thanks,
> Mark.
>
> [1] http://releases.llvm.org/download.html#8.0.0
> [2] http://releases.llvm.org/download.html#8.0.1
> [3] https://mirrors.edge.kernel.org/pub/tools/crosstool/
Hi Mark,
I've narrowed it down further; it seems that the combination of
-fsanitize-coverage=trace-pc and -fsanitize=local-bounds prevents the
emission of __sanitizer_cov_trace_pc. -fsanitize=bounds is the same as
-fsanitize=local-bounds and -fsanitize=array-bounds, the latter of which
has no issues.
https://godbolt.org/z/YdF-he
This reproducer was taken from a somewhat related bug report in April.
https://bugs.llvm.org/show_bug.cgi?id=41387
What's also interesting is when you remove -Qunused-arguments from the
clang command, the following warning appears (also visible in the
godbolt link):
clang-10: warning: argument unused during compilation:
'-fsanitize-coverage=trace-pc' [-Wunused-command-line-argument]
I have no idea why this combination is special, I've been searching the
source trying to see what I can find and I am currently not coming up
with anything (I'm sure a good night's rest will give me a fresh set of
eyes).
This is still an issue on Clang trunk.
Cheers,
Nathan
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Nathan Chancellor <natechancellor@gmail.com>
To: Mark Rutland <mark.rutland@arm.com>
Cc: linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
clang-built-linux@googlegroups.com,
Nick Desaulniers <ndesaulniers@google.com>
Subject: Re: Enabling UBSAN breaks KCOV in clang (8.0.*) on arm64
Date: Tue, 20 Aug 2019 00:28:09 -0700 [thread overview]
Message-ID: <20190820072809.GA62296@archlinux-threadripper> (raw)
In-Reply-To: <20190819165947.GA30292@lakrids.cambridge.arm.com>
On Mon, Aug 19, 2019 at 05:59:48PM +0100, Mark Rutland wrote:
> Hi,
>
> I found that when I enable both KCOV and UBSAN on arm64, clang fails to
> emit any __sanitizer_cov_trace_*() calls in the resulting binary,
> rendering KCOV useless.
>
> For example, when building v5.3-rc3's arch/arm64/kernel/setup.o:
>
> * With defconfig + CONFIG KCOV:
>
> clang -Wp,-MD,arch/arm64/kernel/.setup.o.d -nostdinc -isystem
> /mnt/data/opt/toolchain/llvm/8.0.0/clang+llvm-8.0.0-x86_64-linux-sles11.3/lib/clang/8.0.0/include
> -I./arch/arm64/include -I./arch/arm64/include/generated -I./include
> -I./arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi
> -I./include/uapi -I./include/generated/uapi -include
> ./include/linux/kconfig.h -include ./include/linux/compiler_types.h
> -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3
> -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes
> -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE
> -Werror=implicit-function-declaration -Werror=implicit-int
> -Wno-format-security -std=gnu89 --target=aarch64-linux
> --prefix=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux/bin/
> --gcc-toolchain=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux
> -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only
> -DCONFIG_AS_LSE=1 -fno-asynchronous-unwind-tables
> -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks
> -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048
> -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu
> -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable
> -fno-omit-frame-pointer -fno-optimize-sibling-calls -g
> -Wdeclaration-after-statement -Wvla -Wno-pointer-sign
> -fno-strict-overflow -fno-merge-all-constants -fno-stack-check
> -Werror=date-time -Werror=incompatible-pointer-types
> -Wno-initializer-overrides -Wno-format -Wno-sign-compare
> -Wno-format-zero-length -fsanitize-coverage=trace-pc
> -DKBUILD_BASENAME='"setup"' -DKBUILD_MODNAME='"setup"' -c -o
> arch/arm64/kernel/setup.o arch/arm64/kernel/setup.c
>
> ... and there are 44 calls to __sanitizer_cov_trace_pc in the
> resulting setup.o
>
> * with defconfig + CONFIG_KCOV + CONFIG_UBSAN:
>
> clang -Wp,-MD,arch/arm64/kernel/.setup.o.d -nostdinc -isystem
> /mnt/data/opt/toolchain/llvm/8.0.0/clang+llvm-8.0.0-x86_64-linux-sles11.3/lib/clang/8.0.0/include
> -I./arch/arm64/include -I./arch/arm64/include/generated -I./include
> -I./arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi
> -I./include/uapi -I./include/generated/uapi -include
> ./include/linux/kconfig.h -include ./include/linux/compiler_types.h
> -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3
> -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes
> -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE
> -Werror=implicit-function-declaration -Werror=implicit-int
> -Wno-format-security -std=gnu89 --target=aarch64-linux
> --prefix=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux/bin/
> --gcc-toolchain=/mnt/data/opt/toolchain/kernel-org-crosstool/gcc-8.1.0-nolibc/aarch64-linux
> -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only
> -DCONFIG_AS_LSE=1 -fno-asynchronous-unwind-tables
> -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks
> -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048
> -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu
> -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable
> -fno-omit-frame-pointer -fno-optimize-sibling-calls -g
> -Wdeclaration-after-statement -Wvla -Wno-pointer-sign
> -fno-strict-overflow -fno-merge-all-constants -fno-stack-check
> -Werror=date-time -Werror=incompatible-pointer-types
> -Wno-initializer-overrides -Wno-format -Wno-sign-compare
> -Wno-format-zero-length -fsanitize=shift
> -fsanitize=integer-divide-by-zero -fsanitize=unreachable
> -fsanitize=signed-integer-overflow -fsanitize=bounds
> -fsanitize=object-size -fsanitize=bool -fsanitize=enum
> -fsanitize-coverage=trace-pc -DKBUILD_BASENAME='"setup"'
> -DKBUILD_MODNAME='"setup"' -c -o arch/arm64/kernel/setup.o
> arch/arm64/kernel/setup.c
>
> ... and there are 0 calls to __sanitizer_cov_trace_pc in the resulting
> setup.o, even though -fsanitize-coverage=trace-pc was passed to clang.
>
> If I remove -fsanitize=bounds, there are 121 calls to
> __sanitizer_cov_trace_pc in setup.o. Removing the other options enabled
> by UBSAN didn't have any effect on setup.o.
>
> I'm using the llvm.org 8.0.{0,1} binaries [1,2], along with the
> kernel.org crosstool 8.1.0 binaries [3].
>
> Any ideas as to what's going on?
>
> Thanks,
> Mark.
>
> [1] http://releases.llvm.org/download.html#8.0.0
> [2] http://releases.llvm.org/download.html#8.0.1
> [3] https://mirrors.edge.kernel.org/pub/tools/crosstool/
Hi Mark,
I've narrowed it down further; it seems that the combination of
-fsanitize-coverage=trace-pc and -fsanitize=local-bounds prevents the
emission of __sanitizer_cov_trace_pc. -fsanitize=bounds is the same as
-fsanitize=local-bounds and -fsanitize=array-bounds, the latter of which
has no issues.
https://godbolt.org/z/YdF-he
This reproducer was taken from a somewhat related bug report in April.
https://bugs.llvm.org/show_bug.cgi?id=41387
What's also interesting is when you remove -Qunused-arguments from the
clang command, the following warning appears (also visible in the
godbolt link):
clang-10: warning: argument unused during compilation:
'-fsanitize-coverage=trace-pc' [-Wunused-command-line-argument]
I have no idea why this combination is special, I've been searching the
source trying to see what I can find and I am currently not coming up
with anything (I'm sure a good night's rest will give me a fresh set of
eyes).
This is still an issue on Clang trunk.
Cheers,
Nathan
next prev parent reply other threads:[~2019-08-20 7:28 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-19 16:59 Enabling UBSAN breaks KCOV in clang (8.0.*) on arm64 Mark Rutland
2019-08-19 16:59 ` Mark Rutland
2019-08-20 7:28 ` Nathan Chancellor [this message]
2019-08-20 7:28 ` Nathan Chancellor
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190820072809.GA62296@archlinux-threadripper \
--to=natechancellor@gmail.com \
--cc=clang-built-linux@googlegroups.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=ndesaulniers@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.