Re: [OE-core] Strange segfault on native Go binaries

["Khem Raj" <raj.khem@gmail.com>]

On 12/14/20 4:12 PM, Khem Raj wrote:
> On Mon, Dec 14, 2020 at 3:50 PM Otavio Salvador
> <otavio.salvador@ossystems.com.br> wrote:
>>
>> Em seg., 14 de dez. de 2020 às 19:59, Khem Raj <raj.khem@gmail.com> escreveu:
>>> On Sat, Dec 12, 2020 at 08:52:34PM -0300, Otavio Salvador wrote:
>>>> Hello all,
>>>>
>>>> I am trying to debug a strange issue on native Go binaries. We're
>>>> seeing it on 'glide' for example.
>>>>
>>>> If I build 'glide-native' and I go to its tmp/work/.../image/bin/glide
>>>> it works just fine. For example, running it we see the expected error
>>>> as I am running inside a clean environment without any extra tools:
>>>>
>>>> otavio@yocto-env ~/.../build %
>>>> ./tmp/work/x86_64-linux/glide-native/0.13.3-r0/build/bin/glide
>>>> [ERROR] Error getting version: exec: "go": executable file not found in $PATH.
>>>>
>>>> Now, if we run it from the sysroot-native, it segfaults:
>>>>
>>>> otavio@yocto-env ~/.../build %
>>>> ./tmp/work/cortexa9hf-neon-oel-linux-gnueabi/updatehub/1.1.1-r0/recipe-sysroot-native/usr/bin/glide
>>>> [1]    26085 segmentation fault (core dumped)
>>>>
>>>> And this segfault is also happening inside the build. I ran the
>>>> readelf on them and it seems to have been modified to use the
>>>> uninative libc loader but I am in doubt how if this is the culprit.
>>>> Can someone shed any light on this?
>>>>
>>>
>>> It looks like a known regression in 1.15.5
>>>
>>> There was a linker regression which crept into go compiler in 1.15.5
>>> which was immediately patched and we also backported that to OE-core
>>> however, recently we have switched to using go-binary-native to
>>> bootstrap the toolchain which means the bug was still not plugged in the
>>> native version even though we patched our go versions, we cant help the
>>> prebuilts.
>>>
>>> Luckily we now have .6 release which came out last week, which means we
>>> can upgrade to it and that should take care of the segfault, I have
>>> improvised on the patch which was sent for upgrades and sent a v2 [1]
>>>
>>> Please try it out and let me know.
>>>
>>> [1] https://lists.openembedded.org/g/openembedded-core/topic/patch_v2_go_update_1_15_5/78964072?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,78964072
>>
>> I tried it but it did not solve the problem. The behavior is the same
>> for this specific issue. The cgo issue documented was indeed fixed but
>> the relocation one is still happening :(
> 
> The issue seems to be in stripped glide binary in sysroot but normal
> unstripped binary is ok. So the issue is indeed orthogonal. I am
> looking into what changes during strip
> step here
> 

The problem seems to obscure editing of the glide binary which is 
resulting in a single byte in ELF header being modified between 
do_install and do_populate_sysroot

easy reproducer is

bitbake -cinstall glide-native

then make a copy of glide installed in image/ dir ( where do_install 
puts it ) then run

bitbake -cpopulate_sysroot glide-native

% cmp 
../../image/mnt/b/yoe/master/build/tmp/work/x86_64-linux/glide-native/0.13.3-r0/recipe-sysroot-native/usr/bin/glide 
glide

so they are identical.

and now compare the installed binary and the saved copy.

% cmp 
../../image/mnt/b/yoe/master/build/tmp/work/x86_64-linux/glide-native/0.13.3-r0/recipe-sysroot-native/usr/bin/glide 
glide
../../image/mnt/b/yoe/master/build/tmp/work/x86_64-linux/glide-native/0.13.3-r0/recipe-sysroot-native/usr/bin/glide 
glide differ: byte 57, line 1

it seems installed binary is fine until do_install but then gets edited 
during populate_sysroot

I have shunted many functions, including patchelf operations and the 
binary still gets modified in place

I wonder what other operations could be put under scanner here, Let me 
know if anyone has ideas..



>>
>>
>> --
>> Otavio Salvador                             O.S. Systems
>> http://www.ossystems.com.br        http://code.ossystems.com.br
>> Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750