* Re: UBSAN: run-time undefined behavior sanity checker [not found] <20160121205717.AF61F661293@gitolite.kernel.org> @ 2016-01-22 5:15 ` Dave Jones 2016-01-22 16:43 ` Andrey Ryabinin 0 siblings, 1 reply; 6+ messages in thread From: Dave Jones @ 2016-01-22 5:15 UTC (permalink / raw) To: Linux Kernel Mailing List; +Cc: Andrey Ryabinin On Thu, Jan 21, 2016 at 08:57:17PM +0000, Linux Kernel wrote: > Web: https://git.kernel.org/torvalds/c/c6d308534aef6c99904bf5862066360ae067abc4 > Commit: c6d308534aef6c99904bf5862066360ae067abc4 > Parent: 68920c973254c5b71a684645c5f6f82d6732c5d6 > Refname: refs/heads/master > Author: Andrey Ryabinin <aryabinin@virtuozzo.com> > AuthorDate: Wed Jan 20 15:00:55 2016 -0800 > Committer: Linus Torvalds <torvalds@linux-foundation.org> > CommitDate: Wed Jan 20 17:09:18 2016 -0800 > > UBSAN: run-time undefined behavior sanity checker > > UBSAN uses compile-time instrumentation to catch undefined behavior > (UB). Compiler inserts code that perform certain kinds of checks before > operations that could cause UB. If check fails (i.e. UB detected) > __ubsan_handle_* function called to print error message. > > So the most of the work is done by compiler. This patch just implements > ubsan handlers printing errors. > > GCC has this capability since 4.9.x [1] (see -fsanitize=undefined > option and its suboptions). > However GCC 5.x has more checkers implemented [2]. > Article [3] has a bit more details about UBSAN in the GCC. If I enable this and CONFIG_UBSAN_ALIGNMENT, the kernel doesn't boot, and hangs really early (pretty much as soon as I hit return in grub) far too early for serial console or even tty output. Compiler is debian unstable's 5.3.1 20160114 I don't know if this is worth chasing down, I chose to just disable it, but figured I'd post in case other people stumble across the same issue. That aside though, neat feature. I look forward to breaking kernels with it :) Dave ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: UBSAN: run-time undefined behavior sanity checker 2016-01-22 5:15 ` UBSAN: run-time undefined behavior sanity checker Dave Jones @ 2016-01-22 16:43 ` Andrey Ryabinin 2016-01-22 17:00 ` Dave Jones 0 siblings, 1 reply; 6+ messages in thread From: Andrey Ryabinin @ 2016-01-22 16:43 UTC (permalink / raw) To: Dave Jones, Linux Kernel Mailing List On 01/22/2016 08:15 AM, Dave Jones wrote: > On Thu, Jan 21, 2016 at 08:57:17PM +0000, Linux Kernel wrote: > > Web: https://git.kernel.org/torvalds/c/c6d308534aef6c99904bf5862066360ae067abc4 > > Commit: c6d308534aef6c99904bf5862066360ae067abc4 > > Parent: 68920c973254c5b71a684645c5f6f82d6732c5d6 > > Refname: refs/heads/master > > Author: Andrey Ryabinin <aryabinin@virtuozzo.com> > > AuthorDate: Wed Jan 20 15:00:55 2016 -0800 > > Committer: Linus Torvalds <torvalds@linux-foundation.org> > > CommitDate: Wed Jan 20 17:09:18 2016 -0800 > > > > UBSAN: run-time undefined behavior sanity checker > > > > UBSAN uses compile-time instrumentation to catch undefined behavior > > (UB). Compiler inserts code that perform certain kinds of checks before > > operations that could cause UB. If check fails (i.e. UB detected) > > __ubsan_handle_* function called to print error message. > > > > So the most of the work is done by compiler. This patch just implements > > ubsan handlers printing errors. > > > > GCC has this capability since 4.9.x [1] (see -fsanitize=undefined > > option and its suboptions). > > However GCC 5.x has more checkers implemented [2]. > > Article [3] has a bit more details about UBSAN in the GCC. > > If I enable this and CONFIG_UBSAN_ALIGNMENT, the kernel doesn't boot, > and hangs really early (pretty much as soon as I hit return in grub) > far too early for serial console or even tty output. > > Compiler is debian unstable's 5.3.1 20160114 > > I don't know if this is worth chasing down, I chose to just disable it, > but figured I'd post in case other people stumble across the same issue. > Likely caused by unaligned access in very early code, which ends up in too early printk() call. You could try to disable instrumentation (UBSAN_SANITIZE := n) in early code. Be aware that CONFIG_UBSAN_ALIGNMENT causes a *lot* of spam in dmesg. Since x86 supports unaligned accesses, the significant amount of that spam just a false-positive reports. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: UBSAN: run-time undefined behavior sanity checker 2016-01-22 16:43 ` Andrey Ryabinin @ 2016-01-22 17:00 ` Dave Jones 2016-01-25 14:03 ` Andrey Ryabinin 0 siblings, 1 reply; 6+ messages in thread From: Dave Jones @ 2016-01-22 17:00 UTC (permalink / raw) To: Andrey Ryabinin; +Cc: Linux Kernel Mailing List On Fri, Jan 22, 2016 at 07:43:55PM +0300, Andrey Ryabinin wrote: > On 01/22/2016 08:15 AM, Dave Jones wrote: > > On Thu, Jan 21, 2016 at 08:57:17PM +0000, Linux Kernel wrote: > > > Web: https://git.kernel.org/torvalds/c/c6d308534aef6c99904bf5862066360ae067abc4 > > > Commit: c6d308534aef6c99904bf5862066360ae067abc4 > > > Parent: 68920c973254c5b71a684645c5f6f82d6732c5d6 > > > Refname: refs/heads/master > > > Author: Andrey Ryabinin <aryabinin@virtuozzo.com> > > > AuthorDate: Wed Jan 20 15:00:55 2016 -0800 > > > Committer: Linus Torvalds <torvalds@linux-foundation.org> > > > CommitDate: Wed Jan 20 17:09:18 2016 -0800 > > > > > > UBSAN: run-time undefined behavior sanity checker > > > > > > UBSAN uses compile-time instrumentation to catch undefined behavior > > > (UB). Compiler inserts code that perform certain kinds of checks before > > > operations that could cause UB. If check fails (i.e. UB detected) > > > __ubsan_handle_* function called to print error message. > > > > > > So the most of the work is done by compiler. This patch just implements > > > ubsan handlers printing errors. > > > > > > GCC has this capability since 4.9.x [1] (see -fsanitize=undefined > > > option and its suboptions). > > > However GCC 5.x has more checkers implemented [2]. > > > Article [3] has a bit more details about UBSAN in the GCC. > > > > If I enable this and CONFIG_UBSAN_ALIGNMENT, the kernel doesn't boot, > > and hangs really early (pretty much as soon as I hit return in grub) > > far too early for serial console or even tty output. > > > > Compiler is debian unstable's 5.3.1 20160114 > > > > I don't know if this is worth chasing down, I chose to just disable it, > > but figured I'd post in case other people stumble across the same issue. > > > > Likely caused by unaligned access in very early code, which ends up in too early printk() call. > You could try to disable instrumentation (UBSAN_SANITIZE := n) in early code. > > Be aware that CONFIG_UBSAN_ALIGNMENT causes a *lot* of spam in dmesg. Since x86 supports unaligned > accesses, the significant amount of that spam just a false-positive reports. So disabling that option fixed booting on one machine, but every other I've tried it on hangs the same way, really early. Any thoughts on how to chase this down ? Dave ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: UBSAN: run-time undefined behavior sanity checker 2016-01-22 17:00 ` Dave Jones @ 2016-01-25 14:03 ` Andrey Ryabinin [not found] ` <20160126165322.GA24364@codemonkey.org.uk> 0 siblings, 1 reply; 6+ messages in thread From: Andrey Ryabinin @ 2016-01-25 14:03 UTC (permalink / raw) To: Dave Jones, Linux Kernel Mailing List On 01/22/2016 08:00 PM, Dave Jones wrote: > On Fri, Jan 22, 2016 at 07:43:55PM +0300, Andrey Ryabinin wrote: > > On 01/22/2016 08:15 AM, Dave Jones wrote: > > > On Thu, Jan 21, 2016 at 08:57:17PM +0000, Linux Kernel wrote: > > > > Web: https://git.kernel.org/torvalds/c/c6d308534aef6c99904bf5862066360ae067abc4 > > > > Commit: c6d308534aef6c99904bf5862066360ae067abc4 > > > > Parent: 68920c973254c5b71a684645c5f6f82d6732c5d6 > > > > Refname: refs/heads/master > > > > Author: Andrey Ryabinin <aryabinin@virtuozzo.com> > > > > AuthorDate: Wed Jan 20 15:00:55 2016 -0800 > > > > Committer: Linus Torvalds <torvalds@linux-foundation.org> > > > > CommitDate: Wed Jan 20 17:09:18 2016 -0800 > > > > > > > > UBSAN: run-time undefined behavior sanity checker > > > > > > > > UBSAN uses compile-time instrumentation to catch undefined behavior > > > > (UB). Compiler inserts code that perform certain kinds of checks before > > > > operations that could cause UB. If check fails (i.e. UB detected) > > > > __ubsan_handle_* function called to print error message. > > > > > > > > So the most of the work is done by compiler. This patch just implements > > > > ubsan handlers printing errors. > > > > > > > > GCC has this capability since 4.9.x [1] (see -fsanitize=undefined > > > > option and its suboptions). > > > > However GCC 5.x has more checkers implemented [2]. > > > > Article [3] has a bit more details about UBSAN in the GCC. > > > > > > If I enable this and CONFIG_UBSAN_ALIGNMENT, the kernel doesn't boot, > > > and hangs really early (pretty much as soon as I hit return in grub) > > > far too early for serial console or even tty output. > > > > > > Compiler is debian unstable's 5.3.1 20160114 > > > > > > I don't know if this is worth chasing down, I chose to just disable it, > > > but figured I'd post in case other people stumble across the same issue. > > > > > > > Likely caused by unaligned access in very early code, which ends up in too early printk() call. > > You could try to disable instrumentation (UBSAN_SANITIZE := n) in early code. > > > > Be aware that CONFIG_UBSAN_ALIGNMENT causes a *lot* of spam in dmesg. Since x86 supports unaligned > > accesses, the significant amount of that spam just a false-positive reports. > > So disabling that option fixed booting on one machine, but every other I've > tried it on hangs the same way, really early. Any thoughts on how to chase this down ? > Try to disable instrumentation for early code, like in the patch bellow. Also send me you .config please. Perhaps I will be able to reproduce this. diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index b1b78ff..d39a954 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -20,6 +20,8 @@ KASAN_SANITIZE_head$(BITS).o := n KASAN_SANITIZE_dumpstack.o := n KASAN_SANITIZE_dumpstack_$(BITS).o := n +UBSAN_SANITIZE := n + CFLAGS_irq.o := -I$(src)/../include/asm/trace obj-y := process_$(BITS).o signal.o ^ permalink raw reply related [flat|nested] 6+ messages in thread
[parent not found: <20160126165322.GA24364@codemonkey.org.uk>]
* Re: UBSAN: run-time undefined behavior sanity checker [not found] ` <20160126165322.GA24364@codemonkey.org.uk> @ 2016-02-01 16:08 ` Andrey Ryabinin 2016-02-01 22:18 ` Dave Jones 0 siblings, 1 reply; 6+ messages in thread From: Andrey Ryabinin @ 2016-02-01 16:08 UTC (permalink / raw) To: Dave Jones, LKML On 01/26/2016 07:53 PM, Dave Jones wrote: > <off-list because of huge config> > > On Mon, Jan 25, 2016 at 05:03:48PM +0300, Andrey Ryabinin wrote: > > > > So disabling that option fixed booting on one machine, but every other I've > > > tried it on hangs the same way, really early. Any thoughts on how to chase this down ? > > > > > Try to disable instrumentation for early code, like in the patch bellow. > > > > > > diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile > > index b1b78ff..d39a954 100644 > > --- a/arch/x86/kernel/Makefile > > +++ b/arch/x86/kernel/Makefile > > @@ -20,6 +20,8 @@ KASAN_SANITIZE_head$(BITS).o := n > > KASAN_SANITIZE_dumpstack.o := n > > KASAN_SANITIZE_dumpstack_$(BITS).o := n > > > > +UBSAN_SANITIZE := n > > + > > CFLAGS_irq.o := -I$(src)/../include/asm/trace > > > > obj-y := process_$(BITS).o signal.o > > This didn't help. > > > Also send me you .config please. Perhaps I will be able to reproduce this. > > below. Though I diffed a similar config from a machine where UBSAN works, > and the only differences seemed to be mostly benign stuff or hw specific drivers. > So after I enabled UBSAN_ALIGNMENT in your config, the kernel didn't boot. That is because unaligned access happens before lockdep_init() so ubsan callback takes the spinlock before locked_init() which is not allowed. As far as I understood most of your machines doesn't boot even without UBSAN_ALIGNMENT. So I'm guessing it might be similar problem. Could you try it without CONFIG_DEBUG_LOCKDEP? Or alternatively with patch like this: diff --git a/lib/ubsan.c b/lib/ubsan.c index 8799ae5..220e9d9 100644 --- a/lib/ubsan.c +++ b/lib/ubsan.c @@ -146,13 +146,13 @@ static bool location_is_valid(struct source_location *loc) return loc->file_name != NULL; } -static DEFINE_SPINLOCK(report_lock); +//static DEFINE_SPINLOCK(report_lock); static void ubsan_prologue(struct source_location *location, unsigned long *flags) { current->in_ubsan++; - spin_lock_irqsave(&report_lock, *flags); +// spin_lock_irqsave(&report_lock, *flags); pr_err("========================================" "========================================\n"); @@ -164,7 +164,7 @@ static void ubsan_epilogue(unsigned long *flags) dump_stack(); pr_err("========================================" "========================================\n"); - spin_unlock_irqrestore(&report_lock, *flags); +// spin_unlock_irqrestore(&report_lock, *flags); current->in_ubsan--; } ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: UBSAN: run-time undefined behavior sanity checker 2016-02-01 16:08 ` Andrey Ryabinin @ 2016-02-01 22:18 ` Dave Jones 0 siblings, 0 replies; 6+ messages in thread From: Dave Jones @ 2016-02-01 22:18 UTC (permalink / raw) To: Andrey Ryabinin; +Cc: LKML On Mon, Feb 01, 2016 at 07:08:46PM +0300, Andrey Ryabinin wrote: > > > > tried it on hangs the same way, really early. Any thoughts on how to chase this down ? > > > Try to disable instrumentation for early code, like in the patch bellow. > > This didn't help. > > So after I enabled UBSAN_ALIGNMENT in your config, the kernel didn't boot. > That is because unaligned access happens before lockdep_init() so ubsan callback takes the > spinlock before locked_init() which is not allowed. > Could you try it without CONFIG_DEBUG_LOCKDEP? > Or alternatively with patch like this: Tried with the patch, seems to work fine now. thanks, Dave ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-02-01 22:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20160121205717.AF61F661293@gitolite.kernel.org>
2016-01-22 5:15 ` UBSAN: run-time undefined behavior sanity checker Dave Jones
2016-01-22 16:43 ` Andrey Ryabinin
2016-01-22 17:00 ` Dave Jones
2016-01-25 14:03 ` Andrey Ryabinin
[not found] ` <20160126165322.GA24364@codemonkey.org.uk>
2016-02-01 16:08 ` Andrey Ryabinin
2016-02-01 22:18 ` Dave Jones
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).