[parisc-linux] Users staying up to date.

[parisc-linux] Users staying up to date.

[Carlos O'Donell Jr.]

parisc,

I would like to say that you guys have done a great job
with this port!

I was just looking to bounce some ideas off the list members.
As an avid user, I was just musing to myself, how can I keep
everything up to date (kernel/glibc/packages)?

Obviously the latest kernel/glibc/gcc... will reside in the
CVS. It's a matter of slurping/building and testing on one
of our boxen.

The packages are a slightly different matter. Is the 0.9.2
iso the latest collection of packages? Is anyone in debain-hppa
compiling/testing packages?

We have a cluster of 30x 715/50's running.
And for one reason or another have not quite gotten the debian
installer to run to completetion (nor can I spend the time to
install on each node... that's what scripts and root tarballs
are for).

Instead we have used the baseplus tar from sid to install the nodes.
The only extra packages we required were NIS(yp*) and MPICH(1.2.1).

I recently found though, that when compiling more complicated
MPI programs, cc and ld start dropping SIG11's like there was
no tommorow (more on this after I do some testing).

Is there a fundamental flaw in using sid's baseplus tarball for
the 715/50 systems?

We were doing:
1- Unpack sid baseplus root tarball
2- x-compiled and installed NIS(yp*) and MPICH
3- Use this as my new root tarball for the rest of the cluster

Should I be doing:

1- Unpack sid baseplus and export via nfs
2- Download latest debs and export via nfs
3- Boot "clone" box with nfs root
4- Point apt to the debs on the nfs share
5- Update packages like mad
6- Use "clone" as my new root tarball for the rest of the cluster

Really, I'm just looking for a simple way to stay up to date
with all the latest packages and kernel changes :)

Cheers,
Carlos O'Donell Jr.
-------------------------
Baldric Project
http://www.baldric.uwo.ca
-------------------------

Re: [parisc-linux] Users staying up to date.

[Bdale Garbee]

carlos@baldric.uwo.ca (Carlos O'Donell Jr.) writes:

> Obviously the latest kernel/glibc/gcc... will reside in the
> CVS. It's a matter of slurping/building and testing on one
> of our boxen.

Actually, I just uploaded gcc to the Debian archive built entirely from the
Debian source package, and am working on glibc right now.  The only thing that
there is still any need to use CVS for is the kernel, and I'll be working on
that too once the merge up to 2.4.8 is done.

> The packages are a slightly different matter. Is the 0.9.2
> iso the latest collection of packages? Is anyone in debain-hppa
> compiling/testing packages?

We have a full-time Debian autobuilder running, so on any given day in excess
of 70% of all Debian packages are fully up to date in the unstable tree of
the Debian mirrors.

For a cluster config such as yours, the ideal situation is probably to set 
up a caching proxy like a squid server, and then configure all of your machines
to do their apt-get's from the Debian mirror network through the cache.  That
keeps you as current as you want to be, eliminates any manual steps involved
in maintaining a local cache of packages, and means that you only pull each
package once from the mirror network.

Bdale

Re: [parisc-linux] Users staying up to date.

[Matt Taggart]

"Carlos O'Donell Jr." writes...

> I was just looking to bounce some ideas off the list members.
> As an avid user, I was just musing to myself, how can I keep
> everything up to date (kernel/glibc/packages)?

apt-get upgrade

> Obviously the latest kernel/glibc/gcc... will reside in the
> CVS. It's a matter of slurping/building and testing on one
> of our boxen.

All of the binutils/gcc/glibc/palo changes are in the Debian packages in the 
archive. The kernel-image packages should go in soon, but you'll have to 
upgrade your kernel by hand for now.

> The packages are a slightly different matter. Is the 0.9.2
> iso the latest collection of packages?

No it is a snapshot in time of the Debian unstable archive. After installing 
you should apt-get upgrade to get the latest versions of the things you have 
installed. We'll update the packages on the next iso.

> Is anyone in debain-hppa compiling/testing packages?

Yes, lots of people are building and uploading Debian packages for hppa, 
including the Debian autobuilder that automatically attempts to build new 
debian source packages as they are uploaded. There are currently Debian 5747 
packages available for hppa (including the architecture indepenent ones).

> We have a cluster of 30x 715/50's running.
> And for one reason or another have not quite gotten the debian
> installer to run to completetion (nor can I spend the time to
> install on each node... that's what scripts and root tarballs
> are for).
> 
> Instead we have used the baseplus tar from sid to install the nodes.
> The only extra packages we required were NIS(yp*) and MPICH(1.2.1).

baseplus is really old. You should really use the Debian installer (or at 
least roll a new base tarball).

> I recently found though, that when compiling more complicated
> MPI programs, cc and ld start dropping SIG11's like there was
> no tommorow (more on this after I do some testing).

I wouldn't trust the old base tarball(or any system assembled by hand for that 
matter). Try a fresh install and post the results to the list.

> Is there a fundamental flaw in using sid's baseplus tarball for
> the 715/50 systems?

Yes, the old base tarball probably isn't total clean wrt the alignment needed 
for 715/50.

> Really, I'm just looking for a simple way to stay up to date
> with all the latest packages and kernel changes :)

man apt

Bdale's comments were good too.

-- 
Matt Taggart        Linux Development Lab
taggart@fc.hp.com   HP Linux Systems Operation

Re: [parisc-linux] Users staying up to date.

[Matthew Wilcox]

On Tue, Aug 14, 2001 at 03:40:37PM -0400, Carlos O'Donell Jr. wrote:
> Obviously the latest kernel/glibc/gcc... will reside in the
> CVS. It's a matter of slurping/building and testing on one
> of our boxen.

Actually, the debian glibc package has more uptodate bits than our CVS tree.

> The packages are a slightly different matter. Is the 0.9.2
> iso the latest collection of packages? Is anyone in debain-hppa
> compiling/testing packages?

we're compiling packages constantly.  we have a lack of testing though,
which is what we rely on people like you for :-)

> Instead we have used the baseplus tar from sid to install the nodes.
> The only extra packages we required were NIS(yp*) and MPICH(1.2.1).

We seem to have those compiled already in debian...

> I recently found though, that when compiling more complicated
> MPI programs, cc and ld start dropping SIG11's like there was
> no tommorow (more on this after I do some testing).

OK, I'd be interested to see more detail once we're sure you're on
an uptodate system.

> We were doing:
> 1- Unpack sid baseplus root tarball
> 2- x-compiled and installed NIS(yp*) and MPICH
> 3- Use this as my new root tarball for the rest of the cluster
> 
> Should I be doing:
> 
> 1- Unpack sid baseplus and export via nfs
> 2- Download latest debs and export via nfs
> 3- Boot "clone" box with nfs root
> 4- Point apt to the debs on the nfs share
> 5- Update packages like mad
> 6- Use "clone" as my new root tarball for the rest of the cluster

You should be doing:

Run debian installer to completion on one machine
apt-get dist-upgrade
apt-get install nis mpich
tar up / and copy it to all other machines

If the latest debian installer (never mind what's on the CDs) doesn't
work, then please report the problems.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] Users staying up to date.

[Carlos O'Donell Jr.]

> > I recently found though, that when compiling more complicated
> > MPI programs, cc and ld start dropping SIG11's like there was
> > no tommorow (more on this after I do some testing).
> 
> OK, I'd be interested to see more detail once we're sure you're on
> an uptodate system.
> 
> Run debian installer to completion on one machine
> apt-get dist-upgrade
> apt-get install nis mpich
> tar up / and copy it to all other machines
> 
> If the latest debian installer (never mind what's on the CDs) doesn't
> work, then please report the problems.
>

Thanks for all the feedback.

In general, I don't want to submit a bugreport and have the
list say "That's already fixed in the latest updated package" :)

Which is the reason for my original post.

"How does one stay ontop of a _moving_ target 
 like the PARISC development team?"
- Anonymous Coward :)


I will try out the latest debain intaller and feedback to the list.

I'm currently running apt-get on one of our nodes, and trying
to get it to an updated state.

Hrm. apg-get segfaults randomly. 

If at first you don't succeed, run, run and run again
(until it stops segfaulting).

<Carlos looks at terminal>
Ohhhhh.... it's working!
And traveling through time as tar seems to be reminiscing about how
it's unpacking a file it hasn't made yet, because the obviously correct 
system clock has warned it against time travel :}

Ouch...

--------------
[Nearing the end of apt-get -f dist-update]

Writing passwd-file to /etc/passwd.upwd-write
Replacing "/etc/passwd" with "/etc/passwd.upwd-write"
Writing group-file to /etc/group.upwd-write
Replacing "/etc/group" with "/etc/group.upwd-write"
dpkg: error processing base-passwd (--configure):
 subprocess post-installation script returned error exit status 127
Errors were encountered while processing:
 base-passwd
E: Sub-process /usr/bin/dpkg returned an error code (1)

[So I run it again]

root@node10:/# apt-get -f dist-upgrade
Segmentation faultsts... 43%
root@node10:/# 

[Ouch...]
[With a dump in /var/log/messages]

[dmesg]

do_page_fault() pid=13164 command='apt-get' type=6 address=0x40040dd7
vm_start = 0x3ff13000, vm_end = 0x400a4000

     YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00000000000001001111110000001111
r0-3	 00000000 000002f4 40040dd7 00022350
r4-7	 400c98e0 00022330 faf00948 faf00924
r8-11	 faf00958 00000001 0001dbe5 faf00888
r12-15	 faf00920 00000030 faf008b8 faf008b0
r16-19	 faf00760 faf008a4 faf00748 4030c57c
r20-23	 3ff13000 00000000 00000005 00000001
r24-27	 00000001 00190eb9 00000000 0001d900
r28-31	 3ff13000 00000000 faf00c80 4029fdab
sr0-3	 00000000 000002f4 00000000 000002f4
sr4-7	 000002f4 000002f4 000002f4 000002f4

IASQ: 000002f4 000002f4 IAOQ: 40040dd7 40040ddb
 IIR: 081c0254    ISR: 000002f4  IOR: 404a0004
ORIG_R28: 00190eb9

-------------

We're running a hand rolled:
Linux node10 2.4.0 #37 Tue Jul 17 22:33:26 EDT 2001 parisc unknown

This apt-update was done from a system based on the unstable
baseplus tar.

-------------

[However, I ran it again, like the good robot I am.]
[I kept running it infact, until I get int the following situation:]

root@node10:/# apt-get -f dist-upgrade
Reading Package Lists... Done
Building Dependency Tree... Done
Calculating Upgrade... Done
The following NEW packages will be installed:
  autoconf2.13 autotools-dev groff-base libdb3 
118 packages upgraded, 4 newly installed, 0 to remove and 0  not upgraded.
8 packages not fully installed or removed.
Need to get 1105kB/47.5MB of archives. After unpacking 6158kB will be used.
Do you want to continue? [Y/n] Y
Get:1 http://http.us.debian.org unstable/main dpkg 1.9.16 [1105kB]
Fetched 1105kB in 25s (42.7kB/s)                                                        
debconf: cannot preconfigure packages -- apt-utils is not installed
tar: ./conffiles: time stamp 2001-04-23 18:17:20 is 980225879 s in the future
tar: ./preinst: time stamp 1999-11-24 19:14:54 is 935650533 s in the future
tar: ./prerm: time stamp 1999-11-24 19:14:54 is 935650533 s in the future
tar: ./postinst: time stamp 2000-12-23 15:25:50 is 969764789 s in the future
tar: ./control: time stamp 2001-07-06 14:33:14 is 986606033 s in the future
(Reading database ... 17525 files and directories currently installed.)
Preparing to replace dpkg 1.8.3 (using .../archives/dpkg_1.9.16_hppa.deb) ...
Unpacking replacement dpkg ...
dpkg: error processing /var/cache/apt/archives/dpkg_1.9.16_hppa.deb (--unpack):
 trying to overwrite directory `/usr/share/locale/cs' in package texinfo with nondirectory
dpkg-deb: subprocess paste killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/dpkg_1.9.16_hppa.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@node10:/# 

--------------


In light of these problems.

I think I'm going to:

- Build a newer kernel.
- Try the latest debian installer.
- Give apt another shot at upgrading the system.

--------------

[Musing]

I wonder if running the 715/50's as diskless nodes is as bad an
idea as I originally thought. Would 10Mbit switched be faster than
the 1gig SCSI drive? (HP C2247M1 0BW4)... probably not.

However, it would give us many more active nodes, since many of
our systems are just missing drives :}

Cheers,
Carlos.
-------------------------
Baldric Project
http://www.baldric.uwo.ca
-------------------------

Re: [parisc-linux] Users staying up to date.

[Matthew Wilcox]

On Tue, Aug 14, 2001 at 09:50:30PM -0400, Carlos O'Donell Jr. wrote:
> In general, I don't want to submit a bugreport and have the
> list say "That's already fixed in the latest updated package" :)

That's fair enough.

> I'm currently running apt-get on one of our nodes, and trying
> to get it to an updated state.
> 
> Hrm. apg-get segfaults randomly. 

Yes, it does.  Until you've managed to get sully synced up to latest
toolchain/libs/kernel at which point, it magically stops.

> We're running a hand rolled:
> Linux node10 2.4.0 #37 Tue Jul 17 22:33:26 EDT 2001 parisc unknown

OK, you _definitely_ need to update this.  You could pull from CVS or
grab a snapshot tarball, see the webpages on how to do this.

> (Reading database ... 17525 files and directories currently installed.)
> Preparing to replace dpkg 1.8.3 (using .../archives/dpkg_1.9.16_hppa.deb) ...
> Unpacking replacement dpkg ...
> dpkg: error processing /var/cache/apt/archives/dpkg_1.9.16_hppa.deb (--unpack):
>  trying to overwrite directory `/usr/share/locale/cs' in package texinfo with nondirectory
> dpkg-deb: subprocess paste killed by signal (Broken pipe)

Umm.  That's rather interesting.  an excerpt from dpkg --listfiles dpkg
on paer.debian.org:

/usr/share/locale
/usr/share/locale/cs
/usr/share/locale/cs/LC_MESSAGES
/usr/share/locale/cs/LC_MESSAGES/dpkg.mo

I seem to remember someone saying this was a bug in an old version
of dpkg.  So one way around this may be to dpkg --remove texinfo, then
install dpkg, then install texinfo again.

-- 
Revolutions do not require corporate support.

[parisc-linux] Users staying up to late. ;)

[Carlos O'Donell Jr.]

> > In general, I don't want to submit a bugreport and have the
> > list say "That's already fixed in the latest updated package" :)
> 
> That's fair enough.

Thanks for all the comments!

> 
> Yes, it does.  Until you've managed to get sully synced up to latest
> toolchain/libs/kernel at which point, it magically stops.
> 

Leading back to my reasoning at the start.
Things magically fix themselves when you get the latest package.
Atleast that's what the developers always say ;)

> > Linux node10 2.4.0 #37 Tue Jul 17 22:33:26 EDT 2001 parisc unknown
>
> OK, you _definitely_ need to update this.  You could pull from CVS or
> grab a snapshot tarball, see the webpages on how to do this.

True. Our nodes aren't running the latest kernel, but now that I see
the myriad of page_faults that many programs are causing, I will
be rolling a new one quite soon.

I had the auto-builder perl script working at one point.
Then I mucked about with it, broke it, and then never used it again.
I should get it going again, have been doing most of the kernel builds
by doing a quick CVS slurp and xcompile (Yes my PIII 450 is much
faster at compiling kernels ;) ... then again so is our newly acquired Dual
PIII 1.0GHz mmmmm.... -j16. god bless funding from the engineering
undergraduate society!)

> 
> I seem to remember someone saying this was a bug in an old version
> of dpkg.  So one way around this may be to dpkg --remove texinfo, then
> install dpkg, then install texinfo again.
>

I will try removing texinfo and then installing dpkg, and tell you 
if that was the fix.

TODO:

- Latest kernel.
- dpkg unintall-install-install.

Muchas gracias senores y senoras!

Cheers,
Carlos
--------------------------
Baldric Project
http://www.baldric.uwo.ca
--------------------------

Re: [parisc-linux] Users staying up to date.

[Carlos O'Donell Jr.]

> 
> /usr/share/locale
> /usr/share/locale/cs
> /usr/share/locale/cs/LC_MESSAGES
> /usr/share/locale/cs/LC_MESSAGES/dpkg.mo
> 
> I seem to remember someone saying this was a bug in an old version
> of dpkg.  So one way around this may be to dpkg --remove texinfo, then
> install dpkg, then install texinfo again.
>

I tried removing texinfo. Sadly, what proceeds to occur is a 
relative chain of packages that continually depend on 
/usr/share/locale/cs, all the way down to 'tar.' At which point
I had lost most of my system functionality and was scp'ing 
binaries into /bin to keep it alive.

A fruitless effort, but interesting none the less. ;)

When I get home tonight I'll roll a new kernel and do more testing.

How does the installer get around this problem?
Since it too must start with a base system?
(Maybe it's just my misunderstanding of the complete inner workings
of hte installer).

Cheers,
Carlos.


 

Re: [parisc-linux] Users staying up to date.

[Richard Hirst]

On Fri, Aug 17, 2001 at 03:53:40PM -0400, Carlos O'Donell Jr. wrote:
> > 
> > /usr/share/locale
> > /usr/share/locale/cs
> > /usr/share/locale/cs/LC_MESSAGES
> > /usr/share/locale/cs/LC_MESSAGES/dpkg.mo
> > 
> > I seem to remember someone saying this was a bug in an old version
> > of dpkg.  So one way around this may be to dpkg --remove texinfo, then
> > install dpkg, then install texinfo again.
> >
> 
> I tried removing texinfo. Sadly, what proceeds to occur is a 
> relative chain of packages that continually depend on 
> /usr/share/locale/cs, all the way down to 'tar.' At which point
> I had lost most of my system functionality and was scp'ing 
> binaries into /bin to keep it alive.
> 
> A fruitless effort, but interesting none the less. ;)
> 
> When I get home tonight I'll roll a new kernel and do more testing.
> 
> How does the installer get around this problem?
> Since it too must start with a base system?
> (Maybe it's just my misunderstanding of the complete inner workings
> of hte installer).

The installer starts with a bunch of .debs which are sufficiently new
that they don't have the apt-segv problem.  That is the case for the
0.9.2 ISO, anyway.

You might me able to do a fresh install from the 0.9.2 ISO to get a
more stable system.  If you don't want to do that, you could try using
dpkg -i to install .debs from that ISO.  Possibly a dpkg -i *.deb in
the binary-hppa/base dir might just work.  You should probably be running
a new(ish) kernel before you do that.

Richard

Re: [parisc-linux] Users staying up to date.

[Carlos O'Donell Jr.]

> 
> The installer starts with a bunch of .debs which are sufficiently new
> that they don't have the apt-segv problem.  That is the case for the
> 0.9.2 ISO, anyway.
> 
> You might me able to do a fresh install from the 0.9.2 ISO to get a
> more stable system.  If you don't want to do that, you could try using
> dpkg -i to install .debs from that ISO.  Possibly a dpkg -i *.deb in
> the binary-hppa/base dir might just work.  You should probably be running
> a new(ish) kernel before you do that.
> 
> Richard
>

Richard,

I think the main issue was that we were using a _really_ old kernel.
Which didn't have a lot of the later patches for the older 715 boxes.

I'm currently trying to roll the toolchain from start to end, and 
automate the process a little. BTW, is there any automation setup for
building the toolchain? Does each developer just have their own way(tm)
of doing the build?

The problem I have now lies in gathering up-to-date src for glibc,
and as Matthew pointed out, it is only available from the debian packages.

I'm currently trying this out.

Cheers,
Carlos.

Re: [parisc-linux] Users staying up to date.

[Richard Hirst]

On Tue, Aug 28, 2001 at 12:22:51AM -0400, Carlos O'Donell Jr. wrote:
> I'm currently trying to roll the toolchain from start to end, and 
> automate the process a little. BTW, is there any automation setup for
> building the toolchain? Does each developer just have their own way(tm)
> of doing the build?

Mostly we use the prebuild cross-compilers from the parisc-linux
website, or we use the prebuilt native pkgs from a debian archive.

Richard

Re: [parisc-linux] Users staying up to date.

[Matt Taggart]

"Carlos O'Donell Jr." writes...

> I think the main issue was that we were using a _really_ old kernel.
> Which didn't have a lot of the later patches for the older 715 boxes.
> 
> I'm currently trying to roll the toolchain from start to end, and 
> automate the process a little. BTW, is there any automation setup for
> building the toolchain?

For an x86 -> hppa/hppa64 cross toolchain I use the palinux-autobuilder script 
in the build-tools section of pehc cvs.

For a native hppa toolchain use the debian packages(which several people 
worked very hard on in order to get working so we didn't have to deal with 
building by hand).

For hppa -> hppa64 cross compilers I still do it by hand.

Are you rolling your own toolchain for fun, or is the stuff on pehc not 
meeting your needs?

-- 
Matt Taggart        Linux Development Lab
taggart@fc.hp.com   HP Linux Systems Operation

Re: [parisc-linux] Users staying up to date.

[Carlos O'Donell Jr.]

> 
> For an x86 -> hppa/hppa64 cross toolchain I use the palinux-autobuilder script 
> in the build-tools section of pehc cvs.
> 
> For a native hppa toolchain use the debian packages(which several people 
> worked very hard on in order to get working so we didn't have to deal with 
> building by hand).
> 

True, the native hppa toolchain is excellent. But in our case the systems we
are compiling for are much too slow.  Our cluster of 715's is great for 
proving and testing that your code works in parallel, but nothing short of
dead slow...

Thus we make extensive use of the toolchain on x86.
Ever try building MPICH on a 715/50? :}
Compared to make -j16 on our dual PIII...

> For hppa -> hppa64 cross compilers I still do it by hand.
> 
> Are you rolling your own toolchain for fun, or is the stuff on pehc not 
> meeting your needs?
>

The palinux-autobuilder, atleast the last one I saw, could not compile
the latest set of sources.  I haven't tried in a while.  This is really
the whole reason this thread got started... that palinux-autobuilder 
wasn't working properly for me :} And now Matthew almost has me signed on
as the maintainer for glibc on hppa!

Also, the notes on http://www.parisc-linux.org/toolchain/index.html should
be changed to reflect the fact that it's not possible to build a system
using said instructions ;)

It should probably be changed to include the steps required to:

a) Roll it by hand (what I was attempting)
b) Use the palinux-autobuilder

It goes without saying that I really _want_ the autobuilder to work.
Maybe I should grab it out of CVS... I had the ftp version.

I noted in my other email, that there are a few finiky things that
the autobuilder must do in order to work (set certain environment variables,
patch glibc, etc...).

Cheers,
Carlos.