Re: hppa nptl switch

Re: hppa nptl switch

[Carlos O'Donell]

On Wed, Aug 19, 2009 at 8:48 AM, Aurelien Jarno<aurelien@aurel32.net> wrote:
> [Changing debian-bsd to debian-glibc, probably more appropriate to
> discuss about the internal glibc code ;-)]
>> > may I ask you for status of hppa nptl switch ?

Petr, Aurelian,

I spent the last two days rewriting the pthread structure layouts for
hppa's nptl implementation.

Thanks to Petr's insight, I was able to restructure both
pthread_mutex_t, and pthread_rwlock_t to be 100% ABI compatible with
Linuxthreads.

In the case of pthread_cond_t, the structure grew so much, that I had
to use some of the old lock words (non-zero initialized words) for new
fields, and this meant that all of the pthread_cond_* functions will
quickly test and zero pthread_cond_t before use. The new NPTL
PTHREAD_COND_INITIALIZER uses the fast-path in the code, thus as
applications are rebuilt they will run faster.

The prototype patch in testing is here:
http://www.parisc-linux.org/~carlos/2009-08-31-glibc-ports-hppa-nptl.diff

This patch is not the final version, I need to remove the
old_pthread_cond_* functions, and cleanup the pthread_cond_* functions
to use the same version number. At the end of the day I'm not going to
version any of the functions, because technically they will be ABI
compatible. I will however, surround the checking code with some
version checks so it can be removed in the future (or generate a
warning).

Please note that the new code *will* support partial upgrades for
debian, since this is a 100% ABI compatible change.

I will report back over the next few days on the status of testing the
new patch.

Cheers,
Carlos.

Re: hppa nptl switch

[Petr Salinger]

Hi.

> I spent the last two days rewriting the pthread structure layouts for
> hppa's nptl implementation.

It looks very nice and promising now, thanks.

> I was able to restructure both pthread_mutex_t, and pthread_rwlock_t to 
> be 100% ABI compatible with Linuxthreads.

Strictly speaking, currently they are not,
they are only 100% backward compatible.
But only step away, they could be 100% compatible.
Just the new NPTL initializers should initilize
the four ones as previously in LT.
In practice it shouldn't be problem at all.
Debian should make sure that binary/library compiled
against NPTL-hppa-glibc will require NPTL-hppa-glibc
by proper Depends: line like "libc6 (>= 2.10)".

> The prototype patch in testing is here:
> http://www.parisc-linux.org/~carlos/2009-08-31-glibc-ports-hppa-nptl.diff

It is not possible to use "#include_next <pthread.h>" in
sysdeps/unix/sysv/linux/hppa/nptl/pthread.h, as this file
is installed in libc6-dev for general use.
The proper way would be to convince Mr. Drepper to split
<pthread.h> into <pthread.h> and <bits/pthreadinit.h>.
Good luck with it.

The pthread_cond_init.c should call cond_compat_clear()
instead of cond_compat_check_and_clear() or may be just
"cond->__data.__initializer = 0"

Thanks for all your work.

Petr

Re: hppa nptl switch

[Carlos O'Donell]

On Mon, Aug 31, 2009 at 4:16 AM, Petr Salinger<Petr.Salinger@seznam.cz> wrote:
> Hi.
>
>> I spent the last two days rewriting the pthread structure layouts for
>> hppa's nptl implementation.
>
> It looks very nice and promising now, thanks.
>
>> I was able to restructure both pthread_mutex_t, and pthread_rwlock_t to be
>> 100% ABI compatible with Linuxthreads.
>
> Strictly speaking, currently they are not,
> they are only 100% backward compatible.
> But only step away, they could be 100% compatible.
> Just the new NPTL initializers should initilize
> the four ones as previously in LT.

Yes, my intent was to do exactly that, I had not yet fixed pthread.h
to set the old lock words to all one. I have done exactly this in the
most recent version of my patch. The pthread_mutex_t and
pthread_rwlock_t are now 100% ABI compatible with Linuxthreads.

> In practice it shouldn't be problem at all.
> Debian should make sure that binary/library compiled
> against NPTL-hppa-glibc will require NPTL-hppa-glibc
> by proper Depends: line like "libc6 (>= 2.10)".

Does every package have to do this? I'm not very familiar with all the
packaging requirements.

>> The prototype patch in testing is here:
>> http://www.parisc-linux.org/~carlos/2009-08-31-glibc-ports-hppa-nptl.diff

Second version here:
http://www.parisc-linux.org/~carlos/2009-08-31-glibc-ports-hppa-nptl-v2.diff

The next version is currently testing. It still does not include code
to remove cond_*_clear calls when the ABI level is raised to 2.10.

> It is not possible to use "#include_next <pthread.h>" in
> sysdeps/unix/sysv/linux/hppa/nptl/pthread.h, as this file
> is installed in libc6-dev for general use.

Fixed.

> The pthread_cond_init.c should call cond_compat_clear()
> instead of cond_compat_check_and_clear() or may be just
> "cond->__data.__initializer = 0"

Fixed.

Thanks for the review.

Cheers,
Carlos.

Re: hppa nptl switch

[Frans Pop]

Carlos O'Donell wrote:
>> In practice it shouldn't be problem at all.
>> Debian should make sure that binary/library compiled
>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
>> by proper Depends: line like "libc6 (>= 2.10)".
> 
> Does every package have to do this? I'm not very familiar with all the
> packaging requirements.

It is something that should automatically get done correctly as long as 
the libc-dev package defines the minimum version that way.

The mechanism that determines this is in /var/lib/dpkg/info/libc6.shlibs. 
Currently this has lines like:
   libc 6 libc6 (>= 2.9)

It's virtually certain that the shlibs file will be updated to read
'(>= 2.10)' when the glibc maintainers switch to that version.

Cheers,
FJP

Re: hppa nptl switch

[Carlos O'Donell]

On Mon, Aug 31, 2009 at 12:46 PM, Frans Pop<elendil@planet.nl> wrote:
> Carlos O'Donell wrote:
>>> In practice it shouldn't be problem at all.
>>> Debian should make sure that binary/library compiled
>>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
>>> by proper Depends: line like "libc6 (>=3D 2.10)".
>>
>> Does every package have to do this? I'm not very familiar with all t=
he
>> packaging requirements.
>
> It is something that should automatically get done correctly as long =
as
> the libc-dev package defines the minimum version that way.
>
> The mechanism that determines this is in /var/lib/dpkg/info/libc6.shl=
ibs.
> Currently this has lines like:
> =A0 libc 6 libc6 (>=3D 2.9)
>
> It's virtually certain that the shlibs file will be updated to read
> '(>=3D 2.10)' when the glibc maintainers switch to that version.

Thanks Frans. This is what I expected.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hppa nptl switch

[Aurelien Jarno]

Frans Pop a =E9crit :
> Carlos O'Donell wrote:
>>> In practice it shouldn't be problem at all.
>>> Debian should make sure that binary/library compiled
>>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
>>> by proper Depends: line like "libc6 (>=3D 2.10)".
>> Does every package have to do this? I'm not very familiar with all the
>> packaging requirements.
>=20
> It is something that should automatically get done correctly as long as=
=20
> the libc-dev package defines the minimum version that way.
>=20
> The mechanism that determines this is in /var/lib/dpkg/info/libc6.shlib=
s.=20
> Currently this has lines like:
>    libc 6 libc6 (>=3D 2.9)
>=20

No, as glibc uses symbols files, this file is actually not used.
Nevertheless it is still possible to resolve all symbols to libc6 (>=3D 2=
.10).

--=20
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net


--=20
To UNSUBSCRIBE, email to debian-hppa-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian=
.org

Re: hppa nptl switch

[Carlos O'Donell]

On Tue, Sep 1, 2009 at 2:08 AM, Aurelien Jarno<aurelien@aurel32.net> wrote:
> Frans Pop a =E9crit :
>> Carlos O'Donell wrote:
>>>> In practice it shouldn't be problem at all.
>>>> Debian should make sure that binary/library compiled
>>>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
>>>> by proper Depends: line like "libc6 (>=3D 2.10)".
>>> Does every package have to do this? I'm not very familiar with all the
>>> packaging requirements.
>>
>> It is something that should automatically get done correctly as long as
>> the libc-dev package defines the minimum version that way.
>>
>> The mechanism that determines this is in /var/lib/dpkg/info/libc6.shlibs=
.
>> Currently this has lines like:
>> =A0 =A0libc 6 libc6 (>=3D 2.9)
>>
>
> No, as glibc uses symbols files, this file is actually not used.
> Nevertheless it is still possible to resolve all symbols to libc6 (>=3D 2=
.10).

Once an application is rebuilt against a new libc, what prevents the
user from downgrading libc and breaking the application?

Cheers,
Carlos.

Re: hppa nptl switch

[Thibaut VARENE]

On Tue, Sep 1, 2009 at 3:55 PM, Carlos O'Donell<carlos@systemhalted.org=
> wrote:
> On Tue, Sep 1, 2009 at 2:08 AM, Aurelien Jarno<aurelien@aurel32.net> =
wrote:
>> Frans Pop a =C3=A9crit :
>>> Carlos O'Donell wrote:
>>>>> In practice it shouldn't be problem at all.
>>>>> Debian should make sure that binary/library compiled
>>>>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
>>>>> by proper Depends: line like "libc6 (>=3D 2.10)".
>>>> Does every package have to do this? I'm not very familiar with all=
 the
>>>> packaging requirements.
>>>
>>> It is something that should automatically get done correctly as lon=
g as
>>> the libc-dev package defines the minimum version that way.
>>>
>>> The mechanism that determines this is in /var/lib/dpkg/info/libc6.s=
hlibs.
>>> Currently this has lines like:
>>> =C2=A0 =C2=A0libc 6 libc6 (>=3D 2.9)
>>>
>>
>> No, as glibc uses symbols files, this file is actually not used.
>> Nevertheless it is still possible to resolve all symbols to libc6 (>=
=3D 2.10).
>
> Once an application is rebuilt against a new libc, what prevents the
> user from downgrading libc and breaking the application?

I'd say "common sense" but I recall we don't have protection against
silliness ;^)

T-Bone

--=20
Thibaut VARENE
http://www.parisc-linux.org/~varenet/
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hppa nptl switch

[Aurelien Jarno]

Carlos O'Donell a =E9crit :
> On Tue, Sep 1, 2009 at 2:08 AM, Aurelien Jarno<aurelien@aurel32.net> =
wrote:
>> Frans Pop a =E9crit :
>>> Carlos O'Donell wrote:
>>>>> In practice it shouldn't be problem at all.
>>>>> Debian should make sure that binary/library compiled
>>>>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
>>>>> by proper Depends: line like "libc6 (>=3D 2.10)".
>>>> Does every package have to do this? I'm not very familiar with all=
 the
>>>> packaging requirements.
>>> It is something that should automatically get done correctly as lon=
g as
>>> the libc-dev package defines the minimum version that way.
>>>
>>> The mechanism that determines this is in /var/lib/dpkg/info/libc6.s=
hlibs.
>>> Currently this has lines like:
>>>    libc 6 libc6 (>=3D 2.9)
>>>
>> No, as glibc uses symbols files, this file is actually not used.
>> Nevertheless it is still possible to resolve all symbols to libc6 (>=
=3D 2.10).
>=20
> Once an application is rebuilt against a new libc, what prevents the
> user from downgrading libc and breaking the application?
>=20

If we make sure that the new programs are using symbols from version
GLIBC_2.10, the program should refuse to start with a lower version of
the glibc.

--=20
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hppa nptl switch

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1759 bytes --]

On Tuesday 01 September 2009 11:50:27 Aurelien Jarno wrote:
> Carlos O'Donell a écrit :
> > On Tue, Sep 1, 2009 at 2:08 AM, Aurelien Jarno<aurelien@aurel32.net> 
wrote:
> >> Frans Pop a écrit :
> >>> Carlos O'Donell wrote:
> >>>>> In practice it shouldn't be problem at all.
> >>>>> Debian should make sure that binary/library compiled
> >>>>> against NPTL-hppa-glibc will require NPTL-hppa-glibc
> >>>>> by proper Depends: line like "libc6 (>= 2.10)".
> >>>>
> >>>> Does every package have to do this? I'm not very familiar with all the
> >>>> packaging requirements.
> >>>
> >>> It is something that should automatically get done correctly as long as
> >>> the libc-dev package defines the minimum version that way.
> >>>
> >>> The mechanism that determines this is in
> >>> /var/lib/dpkg/info/libc6.shlibs. Currently this has lines like:
> >>>    libc 6 libc6 (>= 2.9)
> >>
> >> No, as glibc uses symbols files, this file is actually not used.
> >> Nevertheless it is still possible to resolve all symbols to libc6 (>=
> >> 2.10).
> >
> > Once an application is rebuilt against a new libc, what prevents the
> > user from downgrading libc and breaking the application?
>
> If we make sure that the new programs are using symbols from version
> GLIBC_2.10, the program should refuse to start with a lower version of
> the glibc.

i think the question was one about packaging rather than general use ?  if you 
build a package against a newer glibc version but it only uses older symbols, 
then in theory it should work fine with older glibc versions.  if the symbol 
changes between versions, then it should have corresponding symbol version 
changes as well (which will automatically be recorded in the binary).
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: hppa nptl switch

[Carlos O'Donell]

On Tue, Sep 1, 2009 at 8:03 PM, Mike Frysinger<vapier@gentoo.org> wrote:
> i think the question was one about packaging rather than general use ? =
=A0if you
> build a package against a newer glibc version but it only uses older symb=
ols,
> then in theory it should work fine with older glibc versions. =A0if the s=
ymbol
> changes between versions, then it should have corresponding symbol versio=
n
> changes as well (which will automatically be recorded in the binary).

Yes, the question is specifically about packaging.

If the answer is "Debian does not prevent you from downgrading glibc,
even if you have new packages built against the new glibc", then I
accept that.

However, I have another question. What prevents applications built on
a buildd against a newer glibc, from being installed on a system with
an older glibc? Glibc provides backwards compatibility, but *not*
forwards. What happens in the packaging subsystem to prevent this?

How does an application encode "I was built against glibc X.Y,
therefore I should only be run on such a glibc or newer."

Cheers,
Carlos.

Re: hppa nptl switch

[Aurelien Jarno]

On Wed, Sep 02, 2009 at 10:09:22AM -0400, Carlos O'Donell wrote:
> On Tue, Sep 1, 2009 at 8:03 PM, Mike Frysinger<vapier@gentoo.org> wro=
te:
> > i think the question was one about packaging rather than general us=
e ? =A0if you
> > build a package against a newer glibc version but it only uses olde=
r symbols,
> > then in theory it should work fine with older glibc versions. =A0if=
 the symbol
> > changes between versions, then it should have corresponding symbol =
version
> > changes as well (which will automatically be recorded in the binary=
).
>=20
> Yes, the question is specifically about packaging.
>=20
> If the answer is "Debian does not prevent you from downgrading glibc,
> even if you have new packages built against the new glibc", then I
> accept that.
>=20

With the correct shlibs and symbol files, all packages built against th=
e
new glibc will depends on libc6 (>=3D 2.10). This way it won't be possi=
ble
to downgrade the libc6 packages is packages compiled against the new
glibc are installed.

--=20
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hppa nptl switch

[Carlos O'Donell]

On Wed, Sep 2, 2009 at 10:55 AM, Aurelien Jarno<aurelien@aurel32.net> w=
rote:
> On Wed, Sep 02, 2009 at 10:09:22AM -0400, Carlos O'Donell wrote:
>> On Tue, Sep 1, 2009 at 8:03 PM, Mike Frysinger<vapier@gentoo.org> wr=
ote:
>> > i think the question was one about packaging rather than general u=
se ? =A0if you
>> > build a package against a newer glibc version but it only uses old=
er symbols,
>> > then in theory it should work fine with older glibc versions. =A0i=
f the symbol
>> > changes between versions, then it should have corresponding symbol=
 version
>> > changes as well (which will automatically be recorded in the binar=
y).
>>
>> Yes, the question is specifically about packaging.
>>
>> If the answer is "Debian does not prevent you from downgrading glibc=
,
>> even if you have new packages built against the new glibc", then I
>> accept that.
>>
>
> With the correct shlibs and symbol files, all packages built against =
the
> new glibc will depends on libc6 (>=3D 2.10). This way it won't be pos=
sible
> to downgrade the libc6 packages is packages compiled against the new
> glibc are installed.

Is the shlibs sufficient? For example, data structures aren't
versioned. In my new NPTL patches, I change PTHREAD_COND_INITIALIZER,
but I do not version anything (not required because the current
functions support both old and new style initializers), therefore the
symbol files will be identical?

Thanks a lot for your help in answering my debian related packaging que=
stions.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hppa nptl switch

[Aurelien Jarno]

Carlos O'Donell a =E9crit :
> On Wed, Sep 2, 2009 at 10:55 AM, Aurelien Jarno<aurelien@aurel32.net>=
 wrote:
>> On Wed, Sep 02, 2009 at 10:09:22AM -0400, Carlos O'Donell wrote:
>>> On Tue, Sep 1, 2009 at 8:03 PM, Mike Frysinger<vapier@gentoo.org> w=
rote:
>>>> i think the question was one about packaging rather than general u=
se ?  if you
>>>> build a package against a newer glibc version but it only uses old=
er symbols,
>>>> then in theory it should work fine with older glibc versions.  if =
the symbol
>>>> changes between versions, then it should have corresponding symbol=
 version
>>>> changes as well (which will automatically be recorded in the binar=
y).
>>> Yes, the question is specifically about packaging.
>>>
>>> If the answer is "Debian does not prevent you from downgrading glib=
c,
>>> even if you have new packages built against the new glibc", then I
>>> accept that.
>>>
>> With the correct shlibs and symbol files, all packages built against=
 the
>> new glibc will depends on libc6 (>=3D 2.10). This way it won't be po=
ssible
>> to downgrade the libc6 packages is packages compiled against the new
>> glibc are installed.
>=20
> Is the shlibs sufficient? For example, data structures aren't
> versioned. In my new NPTL patches, I change PTHREAD_COND_INITIALIZER,
> but I do not version anything (not required because the current
> functions support both old and new style initializers), therefore the
> symbol files will be identical?
>=20

Yes, but we can change the symbol files so that all versions of all
symbols (for current symbols) resolve to libc6 (>=3D 2.10). This has
already been done for example for the sparc v8 to sparc v8plus ABI chan=
ge.

--=20
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html