Segmentation fault on all golang executables

Segmentation fault on all golang executables

[Kui Zhang]

Hello,

This commit seem to cause Segmentation fault on any of my golang executables.

commit 6f121e548f83674ab4920a4e60afb58d4f61b829
Author: Andy Lutomirski <luto@amacapital.net>
Date:   Mon May 5 12:19:34 2014 -0700

x86, vdso: Reimplement vdso.so preparation in build-time C

golang-go                             2:1.2.1-2ubuntu1
     amd64
golang-go-linux-amd64                 2:1.2.1-2ubuntu1
     amd64


/tmp]> go
Segmentation fault

go_seg]> cat a.go
package main

func main(){
        println("test")

}

################

(gdb) run
Starting program: go_seg/a

Program received signal SIGSEGV, Segmentation fault.
vdso_parse_symbols (vdso_info=<error reading variable: Attempt to
dereference a generic pointer.>,
    version=<error reading variable: Attempt to dereference a generic pointer.>)
    at /usr/lib/go/src/pkg/runtime/vdso_linux_amd64.c:279
279                     if(ELF64_ST_TYPE(sym->st_info) != STT_FUNC)
(gdb) bt
#0  vdso_parse_symbols (vdso_info=<error reading variable: Attempt to
dereference a generic pointer.>,
    version=<error reading variable: Attempt to dereference a generic pointer.>)
    at /usr/lib/go/src/pkg/runtime/vdso_linux_amd64.c:279
#1  0x0000000000417228 in runtime.linux_setup_vdso (
    argc=<error reading variable: Attempt to dereference a generic pointer.>,
    argv=<error reading variable: Attempt to dereference a generic pointer.>)
    at /usr/lib/go/src/pkg/runtime/vdso_linux_amd64.c:326
#2  0x000000000041358c in runtime.args (c=<error reading variable:
Attempt to dereference a generic pointer.>,
    v=<error reading variable: Attempt to dereference a generic
pointer.>) at /usr/lib/go/src/pkg/runtime/runtime.c:86
#3  0x000000000041a01b in _rt0_go () at
/usr/lib/go/src/pkg/runtime/asm_amd64.s:85
#4  0x0000000000000001 in ?? ()
#5  0x00007fffffffe7b8 in ?? ()
#6  0x0000000000000001 in ?? ()
#7  0x00007fffffffe7b8 in ?? ()
#8  0x0000000000000000 in ?? ()

###########################################

Starting program: /usr/bin/go
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
vdso_parse_symbols (vdso_info=<error reading variable: Attempt to
dereference a generic pointer.>,
    version=<error reading variable: Attempt to dereference a generic pointer.>)
    at /usr/lib/go/src/pkg/runtime/vdso_linux_amd64.c:279
279                     if(ELF64_ST_TYPE(sym->st_info) != STT_FUNC)
(gdb) bt
#0  vdso_parse_symbols (vdso_info=<error reading variable: Attempt to
dereference a generic pointer.>,
    version=<error reading variable: Attempt to dereference a generic pointer.>)
    at /usr/lib/go/src/pkg/runtime/vdso_linux_amd64.c:279
#1  0x0000000000470288 in runtime.linux_setup_vdso (
    argc=<error reading variable: Attempt to dereference a generic pointer.>,
    argv=<error reading variable: Attempt to dereference a generic pointer.>)
    at /usr/lib/go/src/pkg/runtime/vdso_linux_amd64.c:326
#2  0x000000000046bf6c in runtime.args (c=<error reading variable:
Attempt to dereference a generic pointer.>,
    v=<error reading variable: Attempt to dereference a generic
pointer.>) at /usr/lib/go/src/pkg/runtime/runtime.c:86
#3  0x000000000047558b in _rt0_go () at
/usr/lib/go/src/pkg/runtime/asm_amd64.s:85
#4  0x0000000000000001 in ?? ()
#5  0x00007fffffffe7d8 in ?? ()
#6  0x0000000000000001 in ?? ()
#7  0x00007fffffffe7d8 in ?? ()
#8  0x0000000000000000 in ?? ()



thanks
Kui.Z

Re: Segmentation fault on all golang executables

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 9:59 AM, Kui Zhang <kuizhang@gmail.com> wrote:
> Hello,
>
> This commit seem to cause Segmentation fault on any of my golang executables.

Well, crap.  It looks like the Go people took my sample vDSO parser...
and broke it.  WTF were they thinking?  I should have noticed that
they screwed it up when I was cc'd on this thing:

https://code.google.com/p/go/source/detail?r=56ea40aac72b

but I didn't.  Sorry, everyone.  And the issue isn't even something
sensibly broken like trying to find vdso symbols in the symbol table.
No, they're using the size of the SHT_DYNSYM section to deduce the
number of entries *in the dynamic table*.  This is just completely
wrong.

But even that's not all.  They borrowed by error handling, so they
should have silently failed to parse the vdso instead of crashing.
But then they broke it completely by failing to zero the state, so
they're just reading from initialized memory.  Grr.

The current offending code is here:

https://code.google.com/p/go/source/browse/src/pkg/runtime/vdso_linux_amd64.c

Since we don't get to break all Go executables, here are two options:

1. Try to keep the whole symbol table intact.  This is annoying:
there's a reason I removed it.  The linker script doesn't know how big
it is, so it's hard to make it compatible with the vvar table.

2. Write a dummy section table that contains a single empty section of
type SHT_DYNSYM.  Hopefully the Go runtime will then get far enough to
fail cleanly.  And they can go and write a real ELF parser or copy my
reference parser correctly.  (hpa, can you apply my patches to make
the reference parser 32-bit clean?)

--Andy

Re: Segmentation fault on all golang executables

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 10:20 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Thu, Jun 12, 2014 at 9:59 AM, Kui Zhang <kuizhang@gmail.com> wrote:
>> Hello,
>>
>> This commit seem to cause Segmentation fault on any of my golang executables.

Can you send me a golang executable that doesn't work.  I just build
the "hello world" example and it works fine -- my copy seems to be
getting lucky and reading zeros in the uninitialized memory :-/

--Andy

Re: Segmentation fault on all golang executables

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 10:26 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Thu, Jun 12, 2014 at 10:20 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Thu, Jun 12, 2014 at 9:59 AM, Kui Zhang <kuizhang@gmail.com> wrote:
>>> Hello,
>>>
>>> This commit seem to cause Segmentation fault on any of my golang executables.
>
> Can you send me a golang executable that doesn't work.  I just build
> the "hello world" example and it works fine -- my copy seems to be
> getting lucky and reading zeros in the uninitialized memory :-/
>

I filed this:

https://code.google.com/p/go/issues/detail?id=8197

--Andy

Re: Segmentation fault on all golang executables

[Kui Zhang]

Thanks for the patches. The workaround works.

Stupid idea, maybe something in dmesg to help spark conversions, when
this workaround is hit?

Looks like golang people are close.


Thanks
Kuiz

Re: Segmentation fault on all golang executables

[H. Peter Anvin]

On 06/12/2014 10:47 PM, Kui Zhang wrote:
> Thanks for the patches. The workaround works.
> 
> Stupid idea, maybe something in dmesg to help spark conversions, when
> this workaround is hit?
> 
> Looks like golang people are close.
> 

The kernel won't even know.

	-hpa

Re: Segmentation fault on all golang executables

[Andy Lutomirski]

[-- Attachment #1: Type: text/plain, Size: 1941 bytes --]

On Thu, Jun 12, 2014 at 10:20 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Thu, Jun 12, 2014 at 9:59 AM, Kui Zhang <kuizhang@gmail.com> wrote:
>> Hello,
>>
>> This commit seem to cause Segmentation fault on any of my golang executables.
>
> Well, crap.  It looks like the Go people took my sample vDSO parser...
> and broke it.  WTF were they thinking?  I should have noticed that
> they screwed it up when I was cc'd on this thing:
>
> https://code.google.com/p/go/source/detail?r=56ea40aac72b
>
> but I didn't.  Sorry, everyone.  And the issue isn't even something
> sensibly broken like trying to find vdso symbols in the symbol table.
> No, they're using the size of the SHT_DYNSYM section to deduce the
> number of entries *in the dynamic table*.  This is just completely
> wrong.
>
> But even that's not all.  They borrowed by error handling, so they
> should have silently failed to parse the vdso instead of crashing.
> But then they broke it completely by failing to zero the state, so
> they're just reading from initialized memory.  Grr.
>
> The current offending code is here:
>
> https://code.google.com/p/go/source/browse/src/pkg/runtime/vdso_linux_amd64.c
>
> Since we don't get to break all Go executables, here are two options:
>
> 1. Try to keep the whole symbol table intact.  This is annoying:
> there's a reason I removed it.  The linker script doesn't know how big
> it is, so it's hard to make it compatible with the vvar table.
>
> 2. Write a dummy section table that contains a single empty section of
> type SHT_DYNSYM.  Hopefully the Go runtime will then get far enough to
> fail cleanly.  And they can go and write a real ELF parser or copy my
> reference parser correctly.  (hpa, can you apply my patches to make
> the reference parser 32-bit clean?)

Sigh.

See attached.  It seems to work for me.  Can you test it?  It'll hurt
performance for Go programs, but I don't feel too bad about that.

--Andy

[-- Attachment #2: 0001-x86-vdso-Hack-to-keep-64-bit-Go-programs-working.patch --]
[-- Type: text/x-patch, Size: 3966 bytes --]

From ad9f313026949247ed2067631b4e5c27c1935064 Mon Sep 17 00:00:00 2001
Message-Id: <ad9f313026949247ed2067631b4e5c27c1935064.1402598233.git.luto@amacapital.net>
From: Andy Lutomirski <luto@amacapital.net>
Date: Thu, 12 Jun 2014 11:31:46 -0700
Subject: [PATCH] x86,vdso: Hack to keep 64-bit Go programs working

The Go runtime has a buggy vDSO parser that currently segfaults.
This writes an empty SHT_DYNSYM entry that causes Go's runtime to
malfunction by thinking that the vDSO is empty rather than
malfunctioning by running off the end and segfaulting.

This is currently broken for big-endian build hosts.  The hack
should also be disabled for x32, but I'm not sure what the right way
to do that is.

Signed-off-by: Andy Lutomirski <luto@amacapital.net>
---
 arch/x86/vdso/Makefile            |  2 +-
 arch/x86/vdso/vdso-fakesections.c | 20 ++++++++++++++++++++
 arch/x86/vdso/vdso2c.h            | 21 +++++++++++++++++----
 3 files changed, 38 insertions(+), 5 deletions(-)
 create mode 100644 arch/x86/vdso/vdso-fakesections.c

diff --git a/arch/x86/vdso/Makefile b/arch/x86/vdso/Makefile
index 9769df0..ffdeb5b 100644
--- a/arch/x86/vdso/Makefile
+++ b/arch/x86/vdso/Makefile
@@ -15,7 +15,7 @@ vdso-install-$(VDSO32-y)	+= $(vdso32-images)
 
 
 # files to link into the vdso
-vobjs-y := vdso-note.o vclock_gettime.o vgetcpu.o
+vobjs-y := vdso-note.o vclock_gettime.o vgetcpu.o vdso-fakesections.o
 
 vobjs-$(VDSOX32-y) += $(vobjx32s-compat)
 
diff --git a/arch/x86/vdso/vdso-fakesections.c b/arch/x86/vdso/vdso-fakesections.c
new file mode 100644
index 0000000..ef75ece
--- /dev/null
+++ b/arch/x86/vdso/vdso-fakesections.c
@@ -0,0 +1,20 @@
+/*
+ * Copyright 2014 Andy Lutomirski
+ * Subject to the GNU Public License, v.2
+ *
+ * Hack to keep broken Go programs working.
+ */
+
+#if __x86_64__  /* hack only needed for the 64-bit vDSO */
+
+#include <linux/elf.h>
+
+extern const __visible struct elf64_shdr vdso_fake_sections[];
+const __visible struct elf64_shdr vdso_fake_sections[] = {
+	{
+		.sh_type = SHT_DYNSYM,
+		.sh_entsize = sizeof(Elf64_Sym),
+	}
+};
+
+#endif
diff --git a/arch/x86/vdso/vdso2c.h b/arch/x86/vdso/vdso2c.h
index d9f6f61..43d9ab1 100644
--- a/arch/x86/vdso/vdso2c.h
+++ b/arch/x86/vdso/vdso2c.h
@@ -18,6 +18,8 @@ static void GOFUNC(void *addr, size_t len, FILE *outfile, const char *name)
 	const char *secstrings;
 	uint64_t syms[NSYMS] = {};
 
+	uint64_t fake_sections_value = 0, fake_sections_size = 0;
+
 	Elf_Phdr *pt = (Elf_Phdr *)(addr + GET_LE(&hdr->e_phoff));
 
 	/* Walk the segment table. */
@@ -84,6 +86,7 @@ static void GOFUNC(void *addr, size_t len, FILE *outfile, const char *name)
 			GET_LE(&symtab_hdr->sh_entsize) * i;
 		const char *name = addr + GET_LE(&strtab_hdr->sh_offset) +
 			GET_LE(&sym->st_name);
+
 		for (k = 0; k < NSYMS; k++) {
 			if (!strcmp(name, required_syms[k])) {
 				if (syms[k]) {
@@ -93,6 +96,13 @@ static void GOFUNC(void *addr, size_t len, FILE *outfile, const char *name)
 				syms[k] = GET_LE(&sym->st_value);
 			}
 		}
+
+		if (!strcmp(name, "vdso_fake_sections")) {
+			if (fake_sections_value)
+				fail("duplicate vdso_fake_sections\n");
+			fake_sections_value = GET_LE(&sym->st_value);
+			fake_sections_size = GET_LE(&sym->st_size);
+		}
 	}
 
 	/* Validate mapping addresses. */
@@ -112,10 +122,13 @@ static void GOFUNC(void *addr, size_t len, FILE *outfile, const char *name)
 	if (syms[sym_end_mapping] % 4096)
 		fail("end_mapping must be a multiple of 4096\n");
 
-	/* Remove sections. */
-	hdr->e_shoff = 0;
-	hdr->e_shentsize = 0;
-	hdr->e_shnum = 0;
+	/* Remove sections or use fakes */
+	if (fake_sections_size % sizeof(Elf_Shdr))
+		fail("vdso_fake_sections size is not a multiple of %ld\n",
+		     (long)sizeof(Elf_Shdr));
+	hdr->e_shoff = fake_sections_value;
+	hdr->e_shentsize = fake_sections_value ? sizeof(Elf_Shdr) : 0;
+	hdr->e_shnum = fake_sections_size / sizeof(Elf_Shdr);
 	hdr->e_shstrndx = SHN_UNDEF; /* SHN_UNDEF == 0 */
 
 	if (!name) {
-- 
1.9.3

Re: Segmentation fault on all golang executables

[H. Peter Anvin]

On 06/12/2014 11:38 AM, Andy Lutomirski wrote:
>>
>> 2. Write a dummy section table that contains a single empty section of
>> type SHT_DYNSYM.  Hopefully the Go runtime will then get far enough to
>> fail cleanly.  And they can go and write a real ELF parser or copy my
>> reference parser correctly.  (hpa, can you apply my patches to make
>> the reference parser 32-bit clean?)
> 

Could you resend it please?

> Sigh.
> 
> See attached.  It seems to work for me.  Can you test it?  It'll hurt
> performance for Go programs, but I don't feel too bad about that.
> 

If Google fixes Go, will that address the performance problem?

	-hpa

Re: Segmentation fault on all golang executables

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 11:41 AM, H. Peter Anvin <hpa@linux.intel.com> wrote:
> On 06/12/2014 11:38 AM, Andy Lutomirski wrote:
>>>
>>> 2. Write a dummy section table that contains a single empty section of
>>> type SHT_DYNSYM.  Hopefully the Go runtime will then get far enough to
>>> fail cleanly.  And they can go and write a real ELF parser or copy my
>>> reference parser correctly.  (hpa, can you apply my patches to make
>>> the reference parser 32-bit clean?)
>>
>
> Could you resend it please?

Yes.  I'll send the whole series.

>
>> Sigh.
>>
>> See attached.  It seems to work for me.  Can you test it?  It'll hurt
>> performance for Go programs, but I don't feel too bad about that.
>>
>
> If Google fixes Go, will that address the performance problem?

Yes.

Note that my hack will build a buggy vdso on big-endian hosts.  Is
there a way to convert host -> BE yet?

--Andy

Re: Segmentation fault on all golang executables

[H. Peter Anvin]

On 06/12/2014 11:42 AM, Andy Lutomirski wrote:
> 
> Note that my hack will build a buggy vdso on big-endian hosts.  Is
> there a way to convert host -> BE yet?
> 

Yes, use <tools/be_byteshift.h>.

	-hpa

Re: Segmentation fault on all golang executables

[Andy Lutomirski]

On Thu, Jun 12, 2014 at 11:48 AM, H. Peter Anvin <hpa@linux.intel.com> wrote:
> On 06/12/2014 11:42 AM, Andy Lutomirski wrote:
>>
>> Note that my hack will build a buggy vdso on big-endian hosts.  Is
>> there a way to convert host -> BE yet?
>>
>
> Yes, use <tools/be_byteshift.h>.

What should I rebase on for that?  And wasn't there some header that
was supposed to choose the right include file given the host runtime?

Also, I meant host -> LE, of course, but that should be trivial.

--Andy

>
>         -hpa
>
>



-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: Segmentation fault on all golang executables

[H. Peter Anvin]

On 06/12/2014 11:51 AM, Andy Lutomirski wrote:
> On Thu, Jun 12, 2014 at 11:48 AM, H. Peter Anvin <hpa@linux.intel.com> wrote:
>> On 06/12/2014 11:42 AM, Andy Lutomirski wrote:
>>>
>>> Note that my hack will build a buggy vdso on big-endian hosts.  Is
>>> there a way to convert host -> BE yet?
>>>
>>
>> Yes, use <tools/be_byteshift.h>.
> 
> What should I rebase on for that?  And wasn't there some header that
> was supposed to choose the right include file given the host runtime?
> 
> Also, I meant host -> LE, of course, but that should be trivial.
> 

No need to rebase.  My patchset proposes replacing
<tools/be_byteshift.h> and <tools/le_byteshift.h> by a unified
<tools/unaligned.h> but that is not important for this purpose.

host -> LE uses <tools/le_byteshift.h> which is already in use by
vdso2c.  Use put_unaligned_leXX().

	-hpa