Re: hppa release status

Re: hppa release status

[Helge Deller]

CC'ed: parisc-linux kernel development list

Andreas Barth wrote:
> during the upload of python2.5, the build failed on hppa due to stalls
> in the test suite, see http://bugs.debian.org/483042 and
> http://buildd.debian.org/fetch.cgi?&pkg=python2.5&ver=2.5.2-5&arch=hppa&stamp=1211583145&file=log
> (Matthias "fixed" that bug by disabling the testsuite, not something that makes
> us happy.)
> 
> After that happened, we asked on #parisc if someone could take a look,
> and we were told that linuxthreads is currently unmaintained for hppa,
> and the issue could only be fixed by moving to nptl and we need to do an
> (incompatible) abi change in glibc. Such a change would be really
> unfortunate, and we hope that every other roads have been evaluated
> first (like trying to understand why python on linuxthreads fails on
> hppa but not on e.g. kfreebsd). We also would like to be sure that ntpl
> is really better than linuxthreads for python2.5 before a transition.

My personal feeling is, that a switch to NPTL is probably the best 
solution. Even if this involves a abi change.
Maybe experts on NPTL could comment here?

> In addition to the python2.5 issue, there are two other issues that are
> quite concerning:
>   * a problem with ruby1.9 which likely is kernel related #478717.

Hmm..

>   * dirmngr that segfaults, likely because of some signalstack issues
>     #459567.

Yes, we need to implement makecontext()/getcontext() in glibc.

> We've seen no porter activity on those bugs yet.

I'd volunteer to try on thedirmngr/makecontext() issue. (At least as far 
as my time permits).

> On further discussing that within the release team, we noticed that the
> Qualification page on http://wiki.debian.org/hppaLennyReleaseRecertification
> is not really complete, e.g. it says:
> | The installer is being maintained by ... and it's currently working
> | effectively. Successful installation reports are available at: ...
> 
> It would really be great (read: it is necessary) that the Qualification
> Page is filled with the missing information, and that we actually have
> enough porters for hppa.

I've added myself there in a few items.
I'd be willing to look into issues with the installer, but not being a 
active debian developer I'd need help from a debian guy if necessary.

> So, with respect to the python2.5 issue, what now?
> 
> 
> At the technical side, best of course would be if linuxthreads would
> continue to work at least enough for lenny, this was the case for a few
> years already, it should be able to survive a few months more, and
> python2.5 can build with the test-suite on hppa.  Of course not breaking
> the API during a linuxthreads -> NPTL switch would be even better.

I can't comment on that.

> If really you see no other option than switching to NPTL, even at the
> current unfortunate moment, the only way how this could be done in a
> timely fashion would be to exempt hppa from the list of architectures
> our testing migration scripts look at for updateness and non-breakness.
> Then upload glibc ASAP, and schedule an archive-wide binNMU campaign.
> 
> Of course, this demands enough buildd power, and wanna-build access by
> (some of) the porters (or whatever else you consider appropriate).
> Moreover it needs quite a lot of time to track that closely, which the
> Release Team probably won't have on its own, so we will need hppa
> buildd-admin and hppa porters time, a lot.
> 
> After the transition is done (and we can only hope it is in time for
> lenny), hppa could be added back to the normal architectures. Downside
> of this is of course that in case hppa is slower than lenny, lenny would
> be released without hppa.

which would be sad...


> Of course, we also need plans for the ruby and dirmngr issues.

Yes.

> So, after that long mail, what's your take on this? How do we continue?

Any other comments?

Helge

Re: hppa release status

[dann frazier]

On Tue, Jun 10, 2008 at 11:44:55PM +0200, Helge Deller wrote:
> CC'ed: parisc-linux kernel development list
>
> Andreas Barth wrote:
>> during the upload of python2.5, the build failed on hppa due to stalls
>> in the test suite, see http://bugs.debian.org/483042 and
>> http://buildd.debian.org/fetch.cgi?&pkg=python2.5&ver=2.5.2-5&arch=hppa&stamp=1211583145&file=log
>> (Matthias "fixed" that bug by disabling the testsuite, not something that 
>> makes
>> us happy.)
>> After that happened, we asked on #parisc if someone could take a look,
>> and we were told that linuxthreads is currently unmaintained for hppa,
>> and the issue could only be fixed by moving to nptl and we need to do an
>> (incompatible) abi change in glibc. Such a change would be really
>> unfortunate, and we hope that every other roads have been evaluated
>> first (like trying to understand why python on linuxthreads fails on
>> hppa but not on e.g. kfreebsd). We also would like to be sure that ntpl
>> is really better than linuxthreads for python2.5 before a transition.
>
> My personal feeling is, that a switch to NPTL is probably the best 
> solution. Even if this involves a abi change.
> Maybe experts on NPTL could comment here?
>
>> In addition to the python2.5 issue, there are two other issues that are
>> quite concerning:
>>   * a problem with ruby1.9 which likely is kernel related #478717.
>
> Hmm..
>
>>   * dirmngr that segfaults, likely because of some signalstack issues
>>     #459567.
>
> Yes, we need to implement makecontext()/getcontext() in glibc.
>
>> We've seen no porter activity on those bugs yet.
>
> I'd volunteer to try on thedirmngr/makecontext() issue. (At least as far as 
> my time permits).
>
>> On further discussing that within the release team, we noticed that the
>> Qualification page on 
>> http://wiki.debian.org/hppaLennyReleaseRecertification
>> is not really complete, e.g. it says:
>> | The installer is being maintained by ... and it's currently working
>> | effectively. Successful installation reports are available at: ...
>> It would really be great (read: it is necessary) that the Qualification
>> Page is filled with the missing information, and that we actually have
>> enough porters for hppa.
>
> I've added myself there in a few items.
> I'd be willing to look into issues with the installer, but not being a 
> active debian developer I'd need help from a debian guy if necessary.

hey Helge,
 Feel free to contact me if you need something done that requires DD
privs.

>> So, with respect to the python2.5 issue, what now?
>> At the technical side, best of course would be if linuxthreads would
>> continue to work at least enough for lenny, this was the case for a few
>> years already, it should be able to survive a few months more, and
>> python2.5 can build with the test-suite on hppa.  Of course not breaking
>> the API during a linuxthreads -> NPTL switch would be even better.
>
> I can't comment on that.
>
>> If really you see no other option than switching to NPTL, even at the
>> current unfortunate moment, the only way how this could be done in a
>> timely fashion would be to exempt hppa from the list of architectures
>> our testing migration scripts look at for updateness and non-breakness.
>> Then upload glibc ASAP, and schedule an archive-wide binNMU campaign.
>> Of course, this demands enough buildd power, and wanna-build access by
>> (some of) the porters (or whatever else you consider appropriate).
>> Moreover it needs quite a lot of time to track that closely, which the
>> Release Team probably won't have on its own, so we will need hppa
>> buildd-admin and hppa porters time, a lot.
>> After the transition is done (and we can only hope it is in time for
>> lenny), hppa could be added back to the normal architectures. Downside
>> of this is of course that in case hppa is slower than lenny, lenny would
>> be released without hppa.
>
> which would be sad...
>
>
>> Of course, we also need plans for the ruby and dirmngr issues.
>
> Yes.
>
>> So, after that long mail, what's your take on this? How do we continue?
>
> Any other comments?
>
> Helge
>
>
>

-- 
dann frazier

Re: hppa release status

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]

On Tue, Jun 10, 2008 at 09:44:55PM +0000, Helge Deller wrote:
> CC'ed: parisc-linux kernel development list
> 
> Andreas Barth wrote:
> >during the upload of python2.5, the build failed on hppa due to stalls
> >in the test suite, see http://bugs.debian.org/483042 and
> >http://buildd.debian.org/fetch.cgi?&pkg=python2.5&ver=2.5.2-5&arch=hppa&stamp=1211583145&file=log
> >(Matthias "fixed" that bug by disabling the testsuite, not something 
> >that makes
> >us happy.)
> >After that happened, we asked on #parisc if someone could take a look,
> >and we were told that linuxthreads is currently unmaintained for hppa,
> >and the issue could only be fixed by moving to nptl and we need to do an
> >(incompatible) abi change in glibc. Such a change would be really
> >unfortunate, and we hope that every other roads have been evaluated
> >first (like trying to understand why python on linuxthreads fails on
> >hppa but not on e.g. kfreebsd). We also would like to be sure that ntpl
> >is really better than linuxthreads for python2.5 before a transition.
> 
> My personal feeling is, that a switch to NPTL is probably the best
> solution. Even if this involves a abi change.
> Maybe experts on NPTL could comment here?

  The NPTL in glibc works well afaict, the downside is the ABI bump,
because doing the archive-wide rebuild as a 66% chance to hurt the port
so bad that it will not be ready in time for lenny. This is a last
resort measure, and everything _must_ be tried before, because even
"losing" a week trying to fix linuxthreads is way faster than rebuilding
the whole archive which is rather a matter of months and risking no HPPA
at all in the end.

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: hppa release status

[James Bottomley]

On Tue, 2008-06-10 at 23:44 +0200, Helge Deller wrote:
> CC'ed: parisc-linux kernel development list
> 
> Andreas Barth wrote:
> > during the upload of python2.5, the build failed on hppa due to stalls
> > in the test suite, see http://bugs.debian.org/483042 and
> > http://buildd.debian.org/fetch.cgi?&pkg=python2.5&ver=2.5.2-5&arch=hppa&stamp=1211583145&file=log
> > (Matthias "fixed" that bug by disabling the testsuite, not something that makes
> > us happy.)
> > 
> > After that happened, we asked on #parisc if someone could take a look,
> > and we were told that linuxthreads is currently unmaintained for hppa,
> > and the issue could only be fixed by moving to nptl and we need to do an
> > (incompatible) abi change in glibc. Such a change would be really
> > unfortunate, and we hope that every other roads have been evaluated
> > first (like trying to understand why python on linuxthreads fails on
> > hppa but not on e.g. kfreebsd). We also would like to be sure that ntpl
> > is really better than linuxthreads for python2.5 before a transition.
> 
> My personal feeling is, that a switch to NPTL is probably the best 
> solution. Even if this involves a abi change.
> Maybe experts on NPTL could comment here?

Well ... we asked for a switch to NPTL over a year ago, raising the ABI
change issue (and requesting glibc6.1 or something similar).  At that
time there was a resounding lack of interest from Debian.  Ordinary
release logic does say that you shouldn't rev an ABI just before a
release.  However debian release logic seems to require some type of
crisis before we can get nptl in, so if this is it ...

> > In addition to the python2.5 issue, there are two other issues that are
> > quite concerning:
> >   * a problem with ruby1.9 which likely is kernel related #478717.
> 
> Hmm..

Actually, I can't reproduce this on ion, which is my debian testing
build box.  The only difference from a normal testing system is that
it's running 2.6.26-rc1 (it's also a pa8800 which makes its coherency a
bit more stringent).  Building python 2.5.2-6 and running all the built
in tests except the two parisc exceptions runs.  Ryan Murray stated that
the failing test was test_sys, so this is what I get running it alone:

jejb@ion> pwd
/home/jejb/sources/python2.5-2.5.2/build-shared
jejb@ion> ./python  -E -tt ../Lib/test/regrtest.py -w -l -uall -s
test_sys
test_sys
1 test OK.

So I think more investigation of the actual alleged failure is
warranted.  At this time, if it is a real failure, I'm not sure it's
necessarily threads related.

> >   * dirmngr that segfaults, likely because of some signalstack issues
> >     #459567.
> 
> Yes, we need to implement makecontext()/getcontext() in glibc.
> 
> > We've seen no porter activity on those bugs yet.
> 
> I'd volunteer to try on thedirmngr/makecontext() issue. (At least as far 
> as my time permits).
> 
> > On further discussing that within the release team, we noticed that the
> > Qualification page on http://wiki.debian.org/hppaLennyReleaseRecertification
> > is not really complete, e.g. it says:
> > | The installer is being maintained by ... and it's currently working
> > | effectively. Successful installation reports are available at: ...
> > 
> > It would really be great (read: it is necessary) that the Qualification
> > Page is filled with the missing information, and that we actually have
> > enough porters for hppa.
> 
> I've added myself there in a few items.
> I'd be willing to look into issues with the installer, but not being a 
> active debian developer I'd need help from a debian guy if necessary.
> 
> > So, with respect to the python2.5 issue, what now?
> > 
> > 
> > At the technical side, best of course would be if linuxthreads would
> > continue to work at least enough for lenny, this was the case for a few
> > years already, it should be able to survive a few months more, and
> > python2.5 can build with the test-suite on hppa.  Of course not breaking
> > the API during a linuxthreads -> NPTL switch would be even better.
> 
> I can't comment on that.

I'll see if I can fix it whatever it is, but right now I need a
reproducible test case.  It looks like the current failure might be tied
to whatever the buildd system was doing or some weird installation
dependency it happens to have that I don't.

James

Re: hppa release status

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 3200 bytes --]

<answer with my glibc hat on>

On Thu, Jun 12, 2008 at 03:39:07PM +0000, James Bottomley wrote:
> On Tue, 2008-06-10 at 23:44 +0200, Helge Deller wrote:
> > My personal feeling is, that a switch to NPTL is probably the best 
> > solution. Even if this involves a abi change.
> > Maybe experts on NPTL could comment here?
> 
> Well ... we asked for a switch to NPTL over a year ago, raising the ABI
> change issue (and requesting glibc6.1 or something similar).  At that
> time there was a resounding lack of interest from Debian.  Ordinary
> release logic does say that you shouldn't rev an ABI just before a
> release.  However debian release logic seems to require some type of
> crisis before we can get nptl in, so if this is it ...

  This is totally unfair, at the time Carlos said that it was probable
that the ABI bump wasn't necessary but that he needed some time to
figure it out and. I've seen absolutely no hard push for NPTL at the
time, and except python2.5 (and again it's still remain to be proven
it's LT related) we've seen no issue on hppa with linuxthreads so far.
So I remain convinced that it's better to see if the ABI bump can be
avoided or not, and if it can't, we can do it in lenny+1 OK.

> Actually, I can't reproduce this on ion, which is my debian testing
> build box.  The only difference from a normal testing system is that
> it's running 2.6.26-rc1 (it's also a pa8800 which makes its coherency a
> bit more stringent).  Building python 2.5.2-6 and running all the built
> in tests except the two parisc exceptions runs.  Ryan Murray stated that
> the failing test was test_sys, so this is what I get running it alone:
> 
> jejb@ion> pwd
> /home/jejb/sources/python2.5-2.5.2/build-shared
> jejb@ion> ./python  -E -tt ../Lib/test/regrtest.py -w -l -uall -s
> test_sys
> test_sys
> 1 test OK.
> 
> So I think more investigation of the actual alleged failure is
> warranted.  At this time, if it is a real failure, I'm not sure it's
> necessarily threads related.

  This is really coherent with our observations: LT is almost pure C,
and kfreebsd uses LT and python works really well. So either it's a bug
in the LT hppa specific code, or a kernel issue. Your tests tends to
show the latter.

> > > So, with respect to the python2.5 issue, what now?
> > >
> > > At the technical side, best of course would be if linuxthreads would
> > > continue to work at least enough for lenny, this was the case for a few
> > > years already, it should be able to survive a few months more, and
> > > python2.5 can build with the test-suite on hppa.  Of course not breaking
> > > the API during a linuxthreads -> NPTL switch would be even better.
> > 
> > I can't comment on that.
> 
> I'll see if I can fix it whatever it is, but right now I need a
> reproducible test case.  It looks like the current failure might be tied
> to whatever the buildd system was doing or some weird installation
> dependency it happens to have that I don't.


-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: hppa release status

[Thiemo Seufer]

Pierre Habouzit wrote:
[snip]
> > Actually, I can't reproduce this on ion, which is my debian testing
> > build box.  The only difference from a normal testing system is that
> > it's running 2.6.26-rc1 (it's also a pa8800 which makes its coherency a
> > bit more stringent).  Building python 2.5.2-6 and running all the built
> > in tests except the two parisc exceptions runs.  Ryan Murray stated that
> > the failing test was test_sys, so this is what I get running it alone:
> > 
> > jejb@ion> pwd
> > /home/jejb/sources/python2.5-2.5.2/build-shared
> > jejb@ion> ./python  -E -tt ../Lib/test/regrtest.py -w -l -uall -s
> > test_sys
> > test_sys
> > 1 test OK.
> > 
> > So I think more investigation of the actual alleged failure is
> > warranted.  At this time, if it is a real failure, I'm not sure it's
> > necessarily threads related.
> 
>   This is really coherent with our observations: LT is almost pure C,
> and kfreebsd uses LT and python works really well. So either it's a bug
> in the LT hppa specific code, or a kernel issue. Your tests tends to
> show the latter.

I recall a similiar failure on mips/mipsel went away after upgrading the
buildd kernels to 2.6.24+. (I have no idea what version the hppa buildds
use.)


Thiemo