glibc updated

glibc updated

[John David Anglin]

Glibc is no longer is broken on hppa.  See:
https://sourceware.org/ml/libc-alpha/2017-07/msg00595.html

Dave
--
John David Anglin	dave.anglin@bell.net

Re: glibc updated

[Carlos O'Donell]

On Mon, Jul 17, 2017 at 7:54 AM, John David Anglin <dave.anglin@bell.net> wrote:
> Glibc is no longer is broken on hppa.  See:
> https://sourceware.org/ml/libc-alpha/2017-07/msg00595.html

Thanks  for all the upstream fixes!

Cheers,
Carlos.

Re: glibc updated

[John David Anglin]

On 2017-07-17 12:00 PM, Carlos O'Donell wrote:
> On Mon, Jul 17, 2017 at 7:54 AM, John David Anglin<dave.anglin@bell.net>  wrote:
>> Glibc is no longer broken on hppa.  See:
>> https://sourceware.org/ml/libc-alpha/2017-07/msg00595.html
> Thanks  for all the upstream fixes!
Appreciated.  It took a fair bit of time last weekend.  As a result, we 
got a bit of help in
fixing a couple more bugs :-)

I would like to know if the gentoo folks would consider fixing the 
__gmon_start__ bug:
https://sourceware.org/bugzilla/show_bug.cgi?id=19170

There is some risk in applying the patch as rebuilding a library package 
may break other
packages which depend on the library.  This could break critical tools 
such as binutils and
gcc.  In which case, some manual intervention may be needed. However, 
the transition
on Debian went fairly smoothly.  As a result, we no longer have the 
external symbol
__gmon_start exposed and we have correct library dependencies.

The issues with _init referred to in the BZ report are fixed.  It is now 
PIC; and PIE applications
work on hppa thanks to Alan Modra.

Although not ideal, we could keep the __gmon_start__ patch in Debian.

Dave

-- 
John David Anglin  dave.anglin@bell.net

Re: glibc updated

[John David Anglin]

On 2017-07-17 12:33 PM, John David Anglin wrote:
> I would like to know if the gentoo folks would consider fixing the 
> __gmon_start__ bug:
> https://sourceware.org/bugzilla/show_bug.cgi?id=19170
>
> There is some risk in applying the patch as rebuilding a library 
> package may break other
> packages which depend on the library.  This could break critical tools 
> such as binutils and
> gcc.  In which case, some manual intervention may be needed. However, 
> the transition
> on Debian went fairly smoothly.  As a result, we no longer have the 
> external symbol
> __gmon_start exposed and we have correct library dependencies.
>
> The issues with _init referred to in the BZ report are fixed.  It is 
> now PIC; and PIE applications
> work on hppa thanks to Alan Modra.
Helge: we need to add PIE load address to the kernel TODO list if it's 
not already there.
>
> Although not ideal, we could keep the __gmon_start__ patch in Debian.
>
The other approach is to install the __gmon_start__ patch and let gentoo 
revert it.  I'm starting
to think this is best.

Dave

-- 
John David Anglin  dave.anglin@bell.net

Re: glibc updated

[Rolf Eike Beer]

Am 2017-07-18 16:32, schrieb John David Anglin:
> On 2017-07-17 12:33 PM, John David Anglin wrote:
>> I would like to know if the gentoo folks would consider fixing the 
>> __gmon_start__ bug:
>> https://sourceware.org/bugzilla/show_bug.cgi?id=19170
>> 
>> There is some risk in applying the patch as rebuilding a library 
>> package may break other
>> packages which depend on the library.  This could break critical tools 
>> such as binutils and
>> gcc.  In which case, some manual intervention may be needed. However, 
>> the transition
>> on Debian went fairly smoothly.  As a result, we no longer have the 
>> external symbol
>> __gmon_start exposed and we have correct library dependencies.
>> 
>> The issues with _init referred to in the BZ report are fixed.  It is 
>> now PIC; and PIE applications
>> work on hppa thanks to Alan Modra.
> Helge: we need to add PIE load address to the kernel TODO list if it's
> not already there.
>> 
>> Although not ideal, we could keep the __gmon_start__ patch in Debian.
>> 
> The other approach is to install the __gmon_start__ patch and let
> gentoo revert it.  I'm starting
> to think this is best.

I don't think there will be a big problem for Gentoo to accept it, as 
long as there is a working upgrade path like "build glibc with flag 
-special-foo, rebuild system, remove flag and rebuild glibc again". And 
of course a hint in the release notes so it will be obvious to the 
packagers what they have to take care of. That flag thing is something 
that Gentoo probably can add to their build scripts, something that e.g. 
keeps the symbol in the lib without exporting it during linking, so it 
would be resolvable first and the reference goes away on rebuild. Or 
whatever ;)

Eike

Re: glibc updated

[John David Anglin]

Hi Eike,

On 2017-07-18 11:13 AM, Rolf Eike Beer wrote:
> Am 2017-07-18 16:32, schrieb John David Anglin:
>> On 2017-07-17 12:33 PM, John David Anglin wrote:
>>> I would like to know if the gentoo folks would consider fixing the 
>>> __gmon_start__ bug:
>>> https://sourceware.org/bugzilla/show_bug.cgi?id=19170
>>>
>>> There is some risk in applying the patch as rebuilding a library 
>>> package may break other
>>> packages which depend on the library.  This could break critical 
>>> tools such as binutils and
>>> gcc.  In which case, some manual intervention may be needed. 
>>> However, the transition
>>> on Debian went fairly smoothly.  As a result, we no longer have the 
>>> external symbol
>>> __gmon_start exposed and we have correct library dependencies.
>>>
>>> The issues with _init referred to in the BZ report are fixed. It is 
>>> now PIC; and PIE applications
>>> work on hppa thanks to Alan Modra.
>> Helge: we need to add PIE load address to the kernel TODO list if it's
>> not already there.
>>>
>>> Although not ideal, we could keep the __gmon_start__ patch in Debian.
>>>
>> The other approach is to install the __gmon_start__ patch and let
>> gentoo revert it.  I'm starting
>> to think this is best.
>
> I don't think there will be a big problem for Gentoo to accept it, as 
> long as there is a working upgrade path like "build glibc with flag 
> -special-foo, rebuild system, remove flag and rebuild glibc again". 
> And of course a hint in the release notes so it will be obvious to the 
> packagers what they have to take care of. That flag thing is something 
> that Gentoo probably can add to their build scripts, something that 
> e.g. keeps the symbol in the lib without exporting it during linking, 
> so it would be resolvable first and the reference goes away on 
> rebuild. Or whatever ;)
I'm not sure you fully understand the issue.  It doesn't involve doing 
anything special in building glibc.

The issue is this code in glibc's crtn.S:

/* Note that we cannot have a weak undefined __gmon_start__, because
    that would require this to be PIC, and the linker is currently not
    able to generate a proper procedure descriptor for _init.  Sad but
    true.  Anyway, HPPA is one of those horrible architectures where
    making the comparison and indirect call is quite expensive (see the
    comment in sysdeps/generic/initfini.c). */
         .text
         .align 4
         .weak   __gmon_start__
         .type    __gmon_start__,@function
__gmon_start__:
         .proc
         .callinfo
         .entry
         bv,n %r0(%r2)
         .exit
         .procend

The above hunk of code results in a weak __gmon_start__ function in 
every shared library.
The function needs to be removed.

As can be seen, the function does nothing.  It is called once when a 
shared library is loaded.

In an application link, __gmon_start__ is resolved to one of the shared 
libraries used by
the application.  If this library is relinked against the new glibc, the 
application may break if
the dynamic linker isn't able to resolve __gmon_start__.  In that case, 
the application needs
to be rebuilt.  The function pointer used to call __gmon_start__ becomes 
a NULL pointer
except when one does a profiled link requiring __gmon_start__.

The difficulty is in knowing which applications need rebuilding and it's 
possible something
critical might break in the upgrade process.  Rebuilding went okay on 
Debian but a few
applications needed rebuilding.

A patch to change __gmon_start__ to undefined weak is referenced in the 
above BZ link.
Gentoo would have to revert the change to keep the current behavior 
where __gmon_start__
is a defined weak symbol.  I don't think a define to allow switching 
behavior is a good idea.

The issues referred to in the comment for  __gmon_start__ are fixed.

The presence of __gmon_start__ is a security issue, it messes up the 
linker --as-needed
option, and a small performance issue.  Applications end up needing 
unnecessary libraries.

Dave

-- 
John David Anglin  dave.anglin@bell.net

Re: glibc updated

[Rolf Eike Beer]

[-- Attachment #1: Type: text/plain, Size: 3435 bytes --]

John David Anglin wrote:
> On 2017-07-18 11:13 AM, Rolf Eike Beer wrote:
> > Am 2017-07-18 16:32, schrieb John David Anglin:
> >> On 2017-07-17 12:33 PM, John David Anglin wrote:
> >>> I would like to know if the gentoo folks would consider fixing the
> >>> __gmon_start__ bug:
> >>> https://sourceware.org/bugzilla/show_bug.cgi?id=19170
> >>> 
> >>> There is some risk in applying the patch as rebuilding a library
> >>> package may break other
> >>> packages which depend on the library.  This could break critical
> >>> tools such as binutils and
> >>> gcc.  In which case, some manual intervention may be needed.
> >>> However, the transition
> >>> on Debian went fairly smoothly.  As a result, we no longer have the
> >>> external symbol
> >>> __gmon_start exposed and we have correct library dependencies.
> >>> 
> >>> The issues with _init referred to in the BZ report are fixed. It is
> >>> now PIC; and PIE applications
> >>> work on hppa thanks to Alan Modra.
> >> 
> >> Helge: we need to add PIE load address to the kernel TODO list if it's
> >> not already there.
> >> 
> >>> Although not ideal, we could keep the __gmon_start__ patch in Debian.
> >> 
> >> The other approach is to install the __gmon_start__ patch and let
> >> gentoo revert it.  I'm starting
> >> to think this is best.
> > 
> > I don't think there will be a big problem for Gentoo to accept it, as
> > long as there is a working upgrade path like "build glibc with flag
> > -special-foo, rebuild system, remove flag and rebuild glibc again".
> > And of course a hint in the release notes so it will be obvious to the
> > packagers what they have to take care of. That flag thing is something
> > that Gentoo probably can add to their build scripts, something that
> > e.g. keeps the symbol in the lib without exporting it during linking,
> > so it would be resolvable first and the reference goes away on
> > rebuild. Or whatever ;)
> 
> I'm not sure you fully understand the issue.  It doesn't involve doing
> anything special in building glibc.
> 
> The issue is this code in glibc's crtn.S:
> 
> /* Note that we cannot have a weak undefined __gmon_start__, because
>     that would require this to be PIC, and the linker is currently not
>     able to generate a proper procedure descriptor for _init.  Sad but
>     true.  Anyway, HPPA is one of those horrible architectures where
>     making the comparison and indirect call is quite expensive (see the
>     comment in sysdeps/generic/initfini.c). */
>          .text
>          .align 4
>          .weak   __gmon_start__
>          .type    __gmon_start__,@function
> __gmon_start__:
>          .proc
>          .callinfo
>          .entry
>          bv,n %r0(%r2)
>          .exit
>          .procend
> 
> The above hunk of code results in a weak __gmon_start__ function in
> every shared library.
> The function needs to be removed.
> 
> As can be seen, the function does nothing.  It is called once when a
> shared library is loaded.
> 
> In an application link, __gmon_start__ is resolved to one of the shared
> libraries used by the application.

So, this would be a problem if the application only links to glibc (and it's 
libraries), no? The problem is not rebuilding, Gentoo is actually totally 
about rebuilding your system every now and then ;) The point is: what must be 
done to have an upgrade path from the old glibc to the new one that will not 
break the system on the way.

Eike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: glibc updated

[John David Anglin]

On 2017-07-18 3:39 PM, Rolf Eike Beer wrote:
> John David Anglin wrote:
>> On 2017-07-18 11:13 AM, Rolf Eike Beer wrote:
>>> Am 2017-07-18 16:32, schrieb John David Anglin:
>>>> On 2017-07-17 12:33 PM, John David Anglin wrote:
>>>>> I would like to know if the gentoo folks would consider fixing the
>>>>> __gmon_start__ bug:
>>>>> https://sourceware.org/bugzilla/show_bug.cgi?id=19170
>>>>>
>>>>> There is some risk in applying the patch as rebuilding a library
>>>>> package may break other
>>>>> packages which depend on the library.  This could break critical
>>>>> tools such as binutils and
>>>>> gcc.  In which case, some manual intervention may be needed.
>>>>> However, the transition
>>>>> on Debian went fairly smoothly.  As a result, we no longer have the
>>>>> external symbol
>>>>> __gmon_start exposed and we have correct library dependencies.
>>>>>
>>>>> The issues with _init referred to in the BZ report are fixed. It is
>>>>> now PIC; and PIE applications
>>>>> work on hppa thanks to Alan Modra.
>>>> Helge: we need to add PIE load address to the kernel TODO list if it's
>>>> not already there.
>>>>
>>>>> Although not ideal, we could keep the __gmon_start__ patch in Debian.
>>>> The other approach is to install the __gmon_start__ patch and let
>>>> gentoo revert it.  I'm starting
>>>> to think this is best.
>>> I don't think there will be a big problem for Gentoo to accept it, as
>>> long as there is a working upgrade path like "build glibc with flag
>>> -special-foo, rebuild system, remove flag and rebuild glibc again".
>>> And of course a hint in the release notes so it will be obvious to the
>>> packagers what they have to take care of. That flag thing is something
>>> that Gentoo probably can add to their build scripts, something that
>>> e.g. keeps the symbol in the lib without exporting it during linking,
>>> so it would be resolvable first and the reference goes away on
>>> rebuild. Or whatever ;)
>> I'm not sure you fully understand the issue.  It doesn't involve doing
>> anything special in building glibc.
>>
>> The issue is this code in glibc's crtn.S:
>>
>> /* Note that we cannot have a weak undefined __gmon_start__, because
>>      that would require this to be PIC, and the linker is currently not
>>      able to generate a proper procedure descriptor for _init.  Sad but
>>      true.  Anyway, HPPA is one of those horrible architectures where
>>      making the comparison and indirect call is quite expensive (see the
>>      comment in sysdeps/generic/initfini.c). */
>>           .text
>>           .align 4
>>           .weak   __gmon_start__
>>           .type    __gmon_start__,@function
>> __gmon_start__:
>>           .proc
>>           .callinfo
>>           .entry
>>           bv,n %r0(%r2)
>>           .exit
>>           .procend
>>
>> The above hunk of code results in a weak __gmon_start__ function in
>> every shared library.
>> The function needs to be removed.
>>
>> As can be seen, the function does nothing.  It is called once when a
>> shared library is loaded.
>>
>> In an application link, __gmon_start__ is resolved to one of the shared
>> libraries used by the application.
> So, this would be a problem if the application only links to glibc (and it's
> libraries), no? The problem is not rebuilding, Gentoo is actually totally
> about rebuilding your system every now and then ;) The point is: what must be
> done to have an upgrade path from the old glibc to the new one that will not
> break the system on the way.
It's not a problem for applications that only link against libc, but for 
other libraries.  I'm
not aware of any way to avoid some breakage other than retaining the old 
behavior.  The new
glibc doesn't break anything directly.  The breakage occurs when 
libraries are linked with
the new crt* files.

One could build up a minimal system in a chroot.  When you are sure that 
it works, then
these tools could be used to get past any breakage.  I don't know the 
gentoo build system
so I don't know how easy this would be.

Dave

-- 
John David Anglin  dave.anglin@bell.net

Re: glibc updated

[John David Anglin]

On 2017-07-18 4:26 PM, John David Anglin wrote:
> On 2017-07-18 3:39 PM, Rolf Eike Beer wrote:
>> John David Anglin wrote:
>>> On 2017-07-18 11:13 AM, Rolf Eike Beer wrote:
>>>> Am 2017-07-18 16:32, schrieb John David Anglin:
>>>>> On 2017-07-17 12:33 PM, John David Anglin wrote:
>>>>>> I would like to know if the gentoo folks would consider fixing the
>>>>>> __gmon_start__ bug:
>>>>>> https://sourceware.org/bugzilla/show_bug.cgi?id=19170
>>>>>>
>>>>>> There is some risk in applying the patch as rebuilding a library
>>>>>> package may break other
>>>>>> packages which depend on the library.  This could break critical
>>>>>> tools such as binutils and
>>>>>> gcc.  In which case, some manual intervention may be needed.
>>>>>> However, the transition
>>>>>> on Debian went fairly smoothly.  As a result, we no longer have the
>>>>>> external symbol
>>>>>> __gmon_start exposed and we have correct library dependencies.
>>>>>>
>>>>>> The issues with _init referred to in the BZ report are fixed. It is
>>>>>> now PIC; and PIE applications
>>>>>> work on hppa thanks to Alan Modra.
>>>>> Helge: we need to add PIE load address to the kernel TODO list if 
>>>>> it's
>>>>> not already there.
>>>>>
>>>>>> Although not ideal, we could keep the __gmon_start__ patch in 
>>>>>> Debian.
>>>>> The other approach is to install the __gmon_start__ patch and let
>>>>> gentoo revert it.  I'm starting
>>>>> to think this is best.
>>>> I don't think there will be a big problem for Gentoo to accept it, as
>>>> long as there is a working upgrade path like "build glibc with flag
>>>> -special-foo, rebuild system, remove flag and rebuild glibc again".
>>>> And of course a hint in the release notes so it will be obvious to the
>>>> packagers what they have to take care of. That flag thing is something
>>>> that Gentoo probably can add to their build scripts, something that
>>>> e.g. keeps the symbol in the lib without exporting it during linking,
>>>> so it would be resolvable first and the reference goes away on
>>>> rebuild. Or whatever ;)
>>> I'm not sure you fully understand the issue.  It doesn't involve doing
>>> anything special in building glibc.
>>>
>>> The issue is this code in glibc's crtn.S:
>>>
>>> /* Note that we cannot have a weak undefined __gmon_start__, because
>>>      that would require this to be PIC, and the linker is currently not
>>>      able to generate a proper procedure descriptor for _init.  Sad but
>>>      true.  Anyway, HPPA is one of those horrible architectures where
>>>      making the comparison and indirect call is quite expensive (see 
>>> the
>>>      comment in sysdeps/generic/initfini.c). */
>>>           .text
>>>           .align 4
>>>           .weak   __gmon_start__
>>>           .type    __gmon_start__,@function
>>> __gmon_start__:
>>>           .proc
>>>           .callinfo
>>>           .entry
>>>           bv,n %r0(%r2)
>>>           .exit
>>>           .procend
>>>
>>> The above hunk of code results in a weak __gmon_start__ function in
>>> every shared library.
>>> The function needs to be removed.
>>>
>>> As can be seen, the function does nothing.  It is called once when a
>>> shared library is loaded.
>>>
>>> In an application link, __gmon_start__ is resolved to one of the shared
>>> libraries used by the application.
>> So, this would be a problem if the application only links to glibc 
>> (and it's
>> libraries), no? The problem is not rebuilding, Gentoo is actually 
>> totally
>> about rebuilding your system every now and then ;) The point is: what 
>> must be
>> done to have an upgrade path from the old glibc to the new one that 
>> will not
>> break the system on the way.
> It's not a problem for applications that only link against libc, but 
> for other libraries.  I'm
> not aware of any way to avoid some breakage other than retaining the 
> old behavior.  The new
> glibc doesn't break anything directly.  The breakage occurs when 
> libraries are linked with
> the new crt* files.
>
> One could build up a minimal system in a chroot.  When you are sure 
> that it works, then
> these tools could be used to get past any breakage.  I don't know the 
> gentoo build system
> so I don't know how easy this would be.
The patch will not be applied until after the 2.26 branch is created so 
there is plenty of time to
consider what needs to be done.

There is one addition issue to consider.  Linuxthreads is now removed 
and hppa uses the
generic nptl thread support.  So, old linuxthread applications will no 
longer work.  Changes
to nptl broke 2.25 on hppa and no doubt the code that tried to handle 
both methodologies
was buggy.  Most architectures dropped linuxthreads years ago.

Dave

-- 
John David Anglin  dave.anglin@bell.net