[PATCH v2] compiler/gcc4: make quirk for asm_volatile_goto unconditional

[PATCH v2] compiler/gcc4: make quirk for asm_volatile_goto unconditional

[Steven Noonan]

I started noticing problems with KVM guest destruction on Linux 3.12+, where
guest memory wasn't being cleaned up. I bisected it down to the commit
introducing the new 'asm goto'-based atomics, and found this quirk was later
applied to those.

Unfortunately, even with GCC 4.8.2 (which ostensibly fixed the known 'asm goto'
bug) I am still getting some kind of miscompilation. If I enable the
asm_volatile_goto quirk for my compiler, KVM guests are destroyed correctly and
the memory is cleaned up.

[1] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58670
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: stable@vger.kernel.org
Signed-off-by: Steven Noonan <steven@uplinklabs.net>
---

v2: Adding stable@vger.kernel.org to Cc.

 include/linux/compiler-gcc4.h | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/include/linux/compiler-gcc4.h b/include/linux/compiler-gcc4.h
index ded4299..2507fd2 100644
--- a/include/linux/compiler-gcc4.h
+++ b/include/linux/compiler-gcc4.h
@@ -75,11 +75,7 @@
  *
  * (asm goto is automatically volatile - the naming reflects this.)
  */
-#if GCC_VERSION <= 40801
-# define asm_volatile_goto(x...)	do { asm goto(x); asm (""); } while (0)
-#else
-# define asm_volatile_goto(x...)	do { asm goto(x); } while (0)
-#endif
+#define asm_volatile_goto(x...)	do { asm goto(x); asm (""); } while (0)
 
 #ifdef CONFIG_ARCH_USE_BUILTIN_BSWAP
 #if GCC_VERSION >= 40400
-- 
1.8.5.4

[tip:x86/urgent] compiler/gcc4: Make quirk for asm_volatile_goto( ) unconditional

[tip-bot for Steven Noonan]

Commit-ID:  a9f180345f5378ac87d80ed0bea55ba421d83859
Gitweb:     http://git.kernel.org/tip/a9f180345f5378ac87d80ed0bea55ba421d83859
Author:     Steven Noonan <steven@uplinklabs.net>
AuthorDate: Wed, 12 Feb 2014 23:01:07 -0800
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 13 Feb 2014 12:34:05 +0100

compiler/gcc4: Make quirk for asm_volatile_goto() unconditional

I started noticing problems with KVM guest destruction on Linux
3.12+, where guest memory wasn't being cleaned up. I bisected it
down to the commit introducing the new 'asm goto'-based atomics,
and found this quirk was later applied to those.

Unfortunately, even with GCC 4.8.2 (which ostensibly fixed the
known 'asm goto' bug) I am still getting some kind of
miscompilation. If I enable the asm_volatile_goto quirk for my
compiler, KVM guests are destroyed correctly and the memory is
cleaned up.

So make the quirk unconditional for now, until bug is found
and fixed.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Steven Noonan <steven@uplinklabs.net>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Jakub Jelinek <jakub@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/1392274867-15236-1-git-send-email-steven@uplinklabs.net
Link: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58670
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 include/linux/compiler-gcc4.h | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/include/linux/compiler-gcc4.h b/include/linux/compiler-gcc4.h
index ded4299..2507fd2 100644
--- a/include/linux/compiler-gcc4.h
+++ b/include/linux/compiler-gcc4.h
@@ -75,11 +75,7 @@
  *
  * (asm goto is automatically volatile - the naming reflects this.)
  */
-#if GCC_VERSION <= 40801
-# define asm_volatile_goto(x...)	do { asm goto(x); asm (""); } while (0)
-#else
-# define asm_volatile_goto(x...)	do { asm goto(x); } while (0)
-#endif
+#define asm_volatile_goto(x...)	do { asm goto(x); asm (""); } while (0)
 
 #ifdef CONFIG_ARCH_USE_BUILTIN_BSWAP
 #if GCC_VERSION >= 40400

Re: [tip:x86/urgent] compiler/gcc4: Make quirk for asm_volatile_goto( ) unconditional

[Jakub Jelinek]

On Thu, Feb 13, 2014 at 03:37:08AM -0800, tip-bot for Steven Noonan wrote:
> Commit-ID:  a9f180345f5378ac87d80ed0bea55ba421d83859
> Gitweb:     http://git.kernel.org/tip/a9f180345f5378ac87d80ed0bea55ba421d83859
> Author:     Steven Noonan <steven@uplinklabs.net>
> AuthorDate: Wed, 12 Feb 2014 23:01:07 -0800
> Committer:  Ingo Molnar <mingo@kernel.org>
> CommitDate: Thu, 13 Feb 2014 12:34:05 +0100
> 
> compiler/gcc4: Make quirk for asm_volatile_goto() unconditional
> 
> I started noticing problems with KVM guest destruction on Linux
> 3.12+, where guest memory wasn't being cleaned up. I bisected it
> down to the commit introducing the new 'asm goto'-based atomics,
> and found this quirk was later applied to those.
> 
> Unfortunately, even with GCC 4.8.2 (which ostensibly fixed the
> known 'asm goto' bug) I am still getting some kind of
> miscompilation. If I enable the asm_volatile_goto quirk for my
> compiler, KVM guests are destroyed correctly and the memory is
> cleaned up.

BTW, which exact 4.8.2 were you using?
The last known asm goto bug has been fixed on October, 10th, 2013:
http://gcc.gnu.org/PR58670
so before the October, 16th, 2013 4.8.2 release.  But already since
May 31th, 2013 the tip of the 4.8 GCC branch has been announcing itself
as 4.8.2 prerelease.  While some distribution versions of GCC announce
themselves as the new version only starting from the release date,
i.e. snapshots in between 4.8.1 release and 4.8.2 release announce
themselves as 4.8.1, in other distributions or upstream it announces itself
as 4.8.2.  So, if you are using the latter and a snapshot in between May
31th, 2013 and October, 10th, 2013, then you could see gcc patchlevel 2,
yet have a gcc with that bug unfixed.
So, if the kernel doesn't use a runtime test/configure test to check for
this issue, but instead just relies on the patchlevel version, the only
safe way would be to look for GCC >= 4.9 or GCC 4.8 with patchlevel > 2
rather than > 1.

	Jakub

Re: [tip:x86/urgent] compiler/gcc4: Make quirk for asm_volatile_goto( ) unconditional

[Steven Noonan]

It's the current Arch Linux GCC package:

$ gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: /build/gcc/src/gcc-4.8-20140206/configure
--prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib
--mandir=/usr/share/man --infodir=/usr/share/info
--with-bugurl=https://bugs.archlinux.org/
 --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++
--enable-shared --enable-threads=posix --with-system-zlib
--enable-__cxa_atexit --disable-libunwind-exceptions
--enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp
--enable-gnu-unique-object --enable-linker-build-id
--enable-cloog-backend=isl --disable-cloog-version-check --enable-lto
--enable-plugin --enable-install-libiberty
--with-linker-hash-style=gnu --disable-multilib --disable-werror
--enable-checking=release
Thread model: posix
gcc version 4.8.2 20140206 (prerelease) (GCC)

https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/gcc&id=53e7ca8e616ebf6530f5dc43e86381dff92f136c

(Resending this message because gmail keeps defaulting to HTML email
and vger.kernel.org keeps rejecting those... Thanks Google.)

On Thu, Feb 13, 2014 at 3:55 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, Feb 13, 2014 at 03:37:08AM -0800, tip-bot for Steven Noonan wrote:
>> Commit-ID:  a9f180345f5378ac87d80ed0bea55ba421d83859
>> Gitweb:     http://git.kernel.org/tip/a9f180345f5378ac87d80ed0bea55ba421d83859
>> Author:     Steven Noonan <steven@uplinklabs.net>
>> AuthorDate: Wed, 12 Feb 2014 23:01:07 -0800
>> Committer:  Ingo Molnar <mingo@kernel.org>
>> CommitDate: Thu, 13 Feb 2014 12:34:05 +0100
>>
>> compiler/gcc4: Make quirk for asm_volatile_goto() unconditional
>>
>> I started noticing problems with KVM guest destruction on Linux
>> 3.12+, where guest memory wasn't being cleaned up. I bisected it
>> down to the commit introducing the new 'asm goto'-based atomics,
>> and found this quirk was later applied to those.
>>
>> Unfortunately, even with GCC 4.8.2 (which ostensibly fixed the
>> known 'asm goto' bug) I am still getting some kind of
>> miscompilation. If I enable the asm_volatile_goto quirk for my
>> compiler, KVM guests are destroyed correctly and the memory is
>> cleaned up.
>
> BTW, which exact 4.8.2 were you using?
> The last known asm goto bug has been fixed on October, 10th, 2013:
> http://gcc.gnu.org/PR58670
> so before the October, 16th, 2013 4.8.2 release.  But already since
> May 31th, 2013 the tip of the 4.8 GCC branch has been announcing itself
> as 4.8.2 prerelease.  While some distribution versions of GCC announce
> themselves as the new version only starting from the release date,
> i.e. snapshots in between 4.8.1 release and 4.8.2 release announce
> themselves as 4.8.1, in other distributions or upstream it announces itself
> as 4.8.2.  So, if you are using the latter and a snapshot in between May
> 31th, 2013 and October, 10th, 2013, then you could see gcc patchlevel 2,
> yet have a gcc with that bug unfixed.
> So, if the kernel doesn't use a runtime test/configure test to check for
> this issue, but instead just relies on the patchlevel version, the only
> safe way would be to look for GCC >= 4.9 or GCC 4.8 with patchlevel > 2
> rather than > 1.
>
>         Jakub

Re: [tip:x86/urgent] compiler/gcc4: Make quirk for asm_volatile_goto( ) unconditional

[Vlastimil Babka]

On 02/13/2014 12:55 PM, Jakub Jelinek wrote:
> On Thu, Feb 13, 2014 at 03:37:08AM -0800, tip-bot for Steven Noonan wrote:
>> Commit-ID:  a9f180345f5378ac87d80ed0bea55ba421d83859
>> Gitweb:     http://git.kernel.org/tip/a9f180345f5378ac87d80ed0bea55ba421d83859
>> Author:     Steven Noonan <steven@uplinklabs.net>
>> AuthorDate: Wed, 12 Feb 2014 23:01:07 -0800
>> Committer:  Ingo Molnar <mingo@kernel.org>
>> CommitDate: Thu, 13 Feb 2014 12:34:05 +0100
>> 
>> compiler/gcc4: Make quirk for asm_volatile_goto() unconditional
>> 
>> I started noticing problems with KVM guest destruction on Linux
>> 3.12+, where guest memory wasn't being cleaned up. I bisected it
>> down to the commit introducing the new 'asm goto'-based atomics,
>> and found this quirk was later applied to those.
>> 
>> Unfortunately, even with GCC 4.8.2 (which ostensibly fixed the
>> known 'asm goto' bug) I am still getting some kind of
>> miscompilation. If I enable the asm_volatile_goto quirk for my
>> compiler, KVM guests are destroyed correctly and the memory is
>> cleaned up.
> 
> BTW, which exact 4.8.2 were you using?
> The last known asm goto bug has been fixed on October, 10th, 2013:
> http://gcc.gnu.org/PR58670

FYI, we have hit a very similar kind of memory leak (orphaned THP pages staying on LRU
with elevated page_count) due to the quirk patch missing in a backport, and tracked the
problem down to put_compound_page() which contains this:

                if (put_page_testzero(page_head))
                        VM_BUG_ON_PAGE(1, page_head);

The problem is that with DEBUG_VM disabled, the 'then' part of this 'if' is a no-op which
makes gcc optimize out the whole put_page_testzero operation. The quirk happens to prevent
this.

There is a new gcc bug filed for this:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61772	

> so before the October, 16th, 2013 4.8.2 release.  But already since
> May 31th, 2013 the tip of the 4.8 GCC branch has been announcing itself
> as 4.8.2 prerelease.  While some distribution versions of GCC announce
> themselves as the new version only starting from the release date,
> i.e. snapshots in between 4.8.1 release and 4.8.2 release announce
> themselves as 4.8.1, in other distributions or upstream it announces itself
> as 4.8.2.  So, if you are using the latter and a snapshot in between May
> 31th, 2013 and October, 10th, 2013, then you could see gcc patchlevel 2,
> yet have a gcc with that bug unfixed.
> So, if the kernel doesn't use a runtime test/configure test to check for
> this issue, but instead just relies on the patchlevel version, the only
> safe way would be to look for GCC >= 4.9 or GCC 4.8 with patchlevel > 2
> rather than > 1.
> 
> 	Jakub
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>