[parisc-linux] Applications in 64 bits userspace

[parisc-linux] Applications in 64 bits userspace

[Rodrigo Colao Merlo]

 Hello,
 I work for a software company in Brazil specialized in Linux 
 development under contract by HP.
 
 We're currently working in a project that needs investigate what are missing 
in kernel to run applications in 64 bits userspace. I need only known what 
are missing in kernel and what kernel do you recomend ? 2.4 or 2.5 ? 

The patchs for both kernels(2.4.x and 2.5.x) are in the same stage?

Thank you!

Rodrigo

Re: [parisc-linux] Applications in 64 bits userspace

[Matthew Wilcox]

On Thu, Apr 03, 2003 at 03:12:35PM -0300, Rodrigo Colao Merlo wrote:
>  Hello,
>  I work for a software company in Brazil specialized in Linux 
>  development under contract by HP.
>  
>  We're currently working in a project that needs investigate what are missing 
> in kernel to run applications in 64 bits userspace. I need only known what 
> are missing in kernel and what kernel do you recomend ? 2.4 or 2.5 ? 

Yes, Leandro from HP Brazil kept posting asking that question.
Unfortunately, he kept on asking even after he'd been given a fairly
comprehensive list of things to do, so I think people couldn't be bothered
to tell him any more.

Probably the best list of things to do would be the thread starting at
http://lists.parisc-linux.org/pipermail/parisc-linux/2003-January/018911.html

As to which kernel to base it on; most of the work is kernel-independent.
It's all about making the toolchain work properly.  There's a certain
amount of work to do in the kernel, but there's some design work to do
up-front first.  It shouldn't be hard to do it to either kernel and then
port to the other.

-- 
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Re: [parisc-linux] Applications in 64 bits userspace

[John David Anglin]

> As to which kernel to base it on; most of the work is kernel-independent.
> It's all about making the toolchain work properly.  There's a certain
> amount of work to do in the kernel, but there's some design work to do
> up-front first.  It shouldn't be hard to do it to either kernel and then
> port to the other.

With respect to the toolchain, you should be aware that there are
two different GNU linkers.  The 64-bit linker was designed for HP-UX
operation and was written more or less from scratch.  The 32-bit
linker was ported I believe from the i386 port and is better
suited to linux use.  It has had more testing.  If the 32-bit linker
is ported to 64 bits, you will have to deal with the different
handling of function descriptors (OPDs).  This is also an issue
for the parisc specific code in glibc used for the dynamic loader.
Code to handle various 64-bit relocations would need to be added
to the 32-bit linker.  Management of stub groups might also be an
issue although the 32-bit linker currently does a better job of this.

The current 64-bit linker is broken in a number of ways.  These
problems might get fixed in the coming months but it would be nice 
if we could dispense with having two different linkers.  If
the current 64-bit linker is used for linux, then the problems
with link-once sections, symbol overloading, symbol visibility,
PHDRs, static links, etc., will need fixing.  Fixing these problems
could be 2-3 months work, or more, for someone who knows their way
around the linker.

What you need to do is to look at the parisc specific code in the
kernel, binutils and glibc.  Carlos O'Donell <carlos@baldric.uwo.ca>
did some hacking a few months ago to try to get a simple static linked
application to run in 64-bit mode.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Applications in 64 bits userspace

[Carlos O'Donell]

John,

> With respect to the toolchain, you should be aware that there are
> two different GNU linkers.  The 64-bit linker was designed for HP-UX
> operation and was written more or less from scratch.  The 32-bit
> linker was ported I believe from the i386 port and is better
> suited to linux use.  It has had more testing.  If the 32-bit linker
> is ported to 64 bits, you will have to deal with the different
> handling of function descriptors (OPDs).  This is also an issue
> for the parisc specific code in glibc used for the dynamic loader.
> Code to handle various 64-bit relocations would need to be added
> to the 32-bit linker.  Management of stub groups might also be an
> issue although the 32-bit linker currently does a better job of this.
 
Why was a 64-bit linker written for HPUX, couldn't the normal HPUX
linker have been used?

As for function descriptors, I think that I'll be reworking that code 
(glibc) in the next few months.... following somewhat what H.J.Lu 
has suggested and ia64 / PowerPC64 implement. This should make things 
easier to work with.

> The current 64-bit linker is broken in a number of ways.  These
> problems might get fixed in the coming months but it would be nice 
> if we could dispense with having two different linkers.  If
> the current 64-bit linker is used for linux, then the problems
> with link-once sections, symbol overloading, symbol visibility,
> PHDRs, static links, etc., will need fixing.  Fixing these problems
> could be 2-3 months work, or more, for someone who knows their way
> around the linker.

I think extending the 32-bit linker would be a better use of resources. 

> What you need to do is to look at the parisc specific code in the
> kernel, binutils and glibc.  Carlos O'Donell <carlos@baldric.uwo.ca>
> did some hacking a few months ago to try to get a simple static linked
> application to run in 64-bit mode.

Not true 64-bit since leaving the kernel doesn't mean the processor
stays in wide mode. I need to make some more kernel patches to
enable the start of a static 64-bit userspace.

I've pulled glibc into cvs.parisc-linux.org again, and I'll be 
updating frequently and championing our patches back to libc-alpha 
in the next month.

I think that before a 64-bit userspace gets started, I'm going to
be working at getting glibc to pass "make -k check" without failure,
followed by lots of TLS work so we keep up with the changes in glibc
for our 32-bit userspace :)

c.

Re: [parisc-linux] Applications in 64 bits userspace

[John David Anglin]

> Why was a 64-bit linker written for HPUX, couldn't the normal HPUX
> linker have been used?

I can't answer that.  Presumably, HP funded the development.  Possibly,
they were considering using the GNU linker but then decided to go with
a different port.  I do most of my 64-bit testing now with the HPUX
linker.  There were some rough edges in using the HP linker with GCC
a few months ago but these have been resolved.

The HP linker is not compliant with the sysv ELF ABI in its handling
of weak symbols.  Undefined weak symbols are supposed to resolve
to a value of 0, and the linker is not supposed to search archive
libraries for undefined weaks.  I tried hacking GNU ld to see if
this could be fixed but this isn't possible since the dynamic loader
has the same behavior.  Basically, weak symbols appear to behave
like secondary definition symbols with the SOM runtime.  The lack
of proper weak support impacts GCC thread support.

HP doesn't support DT_INIT and DT_FINI.  They also used a non
standard implementation for DT_ARRAY_INIT and DT_ARRAY_FINI.
Their section flags for these are non compliant and we needed an
assembler hack to handle initializers and finalizers.

Finally, the HP linker doesn't know about gnu linkonce sections but
I haven't hit any situation where it causes a functional problem.
Probably, it only affect program startup time.

> As for function descriptors, I think that I'll be reworking that code 
> (glibc) in the next few months.... following somewhat what H.J.Lu 
> has suggested and ia64 / PowerPC64 implement. This should make things 
> easier to work with.

I presume you are talking about the implementation for lazy linking.
I know the ia64 implementation of function descriptors differs from
what is done for the 32-bit hppa ports.

> Not true 64-bit since leaving the kernel doesn't mean the processor
> stays in wide mode. I need to make some more kernel patches to
> enable the start of a static 64-bit userspace.

Ok.  I should note that the current 64-bit GNU ld doesn't know how
to do a true static link.

> I think that before a 64-bit userspace gets started, I'm going to
> be working at getting glibc to pass "make -k check" without failure,
> followed by lots of TLS work so we keep up with the changes in glibc
> for our 32-bit userspace :)

I certainly agree with that plan.  64-bit code is always going to
be slower than 32-bit code, so it shouldn't be used unless really
needed.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Applications in 64 bits userspace

[Carlos O'Donell]

John,

> I can't answer that.  Presumably, HP funded the development.  Possibly,
> they were considering using the GNU linker but then decided to go with
> a different port.  I do most of my 64-bit testing now with the HPUX
> linker.  There were some rough edges in using the HP linker with GCC
> a few months ago but these have been resolved.

I get random email from people who've seen my libc-alpha postings, and
they want to know how to fix their busted gcc (hppa2.0w) + HPUX linker
compiles... looks like floating point loads and stores with bad 
relocations (the reason we disable fpregs in the rtld code for glibc).

> The HP linker is not compliant with the sysv ELF ABI in its handling
> of weak symbols.  Undefined weak symbols are supposed to resolve
> to a value of 0, and the linker is not supposed to search archive
> libraries for undefined weaks.  I tried hacking GNU ld to see if
> this could be fixed but this isn't possible since the dynamic loader
> has the same behavior.  Basically, weak symbols appear to behave
> like secondary definition symbols with the SOM runtime.  The lack
> of proper weak support impacts GCC thread support.

Ick! Okay, I see why the initial intent to transition toolchains was
made. Though it still probably kicks our arse in terms of performance.

> I presume you are talking about the implementation for lazy linking.
> I know the ia64 implementation of function descriptors differs from
> what is done for the 32-bit hppa ports.

See Lu's patch at:
http://sources.redhat.com/ml/libc-alpha/2003-04/msg00048.html

It's a unification effort to try get arches that use func. desc. onto
the same page... hopefully lockless lookups and lazy linking :)
 
> Ok.  I should note that the current 64-bit GNU ld doesn't know how
> to do a true static link.

How so? -static and -nostdlib and add all the bits yourself?
 
> I certainly agree with that plan.  64-bit code is always going to
> be slower than 32-bit code, so it shouldn't be used unless really
> needed.

It would still be a nice treat to get more testing on the 64-bit
compiler, and be able to run bigger apps on the bigger PARISC boxes once
they become available :)

Though, yes, the current plan is, without fail, in the order I feel like
fixing them today:
- make -k check passes
- function descriptor fixes
- TLS support
- Static 64-bit userspace

c.

Re: [parisc-linux] Applications in 64 bits userspace

[John David Anglin]

> I get random email from people who've seen my libc-alpha postings, and
> they want to know how to fix their busted gcc (hppa2.0w) + HPUX linker
> compiles... looks like floating point loads and stores with bad 
> relocations (the reason we disable fpregs in the rtld code for glibc).

This sounds like a configuration mixup but I would have to see
a real bug report to be sure.  Regarding disabling fpregs, this
might not be a good idea.  Integer multiplication uses the xmpyu
instruction.  There is millicode support for 32-bit multiplication
but not for 64-bit multiplication.  On the 64-bit port, loop
unrolling can cause a multiply instruction to emitted after
virtual registers are instantiated.  If fpregs are disabled,
the multiply requires a libcall.  Emitting a libcall, requires
setting the arg pointer using the virtual outgoing args register.
There is a small chance that we might not have reserved enough
space for the outgoing arguments when this is done after virtual
register instantiation.  You need -nostdlib when linking as
various functions in libgcc uses the xmpyu instruction.

> How so? -static and -nostdlib and add all the bits yourself?

No.  The dynamic loader is still required to resolve some special
symbols.  I also think the file format is not quite right for
a static executable.  I'm sure that it would be possible to generate
a truly static binary but I'm not sure how much work is involved.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Applications in 64 bits userspace

[Carlos O'Donell]

> This sounds like a configuration mixup but I would have to see
> a real bug report to be sure.  Regarding disabling fpregs, this
> might not be a good idea.  

It's currently the only solution for getting the rtld code working, since
the floating point code was trying to load a non-existant linkage table pointer...

> Integer multiplication uses the xmpyu
> instruction.  There is millicode support for 32-bit multiplication
> but not for 64-bit multiplication.  On the 64-bit port, loop
> unrolling can cause a multiply instruction to emitted after
> virtual registers are instantiated.  If fpregs are disabled,
> the multiply requires a libcall.  Emitting a libcall, requires
> setting the arg pointer using the virtual outgoing args register.
> There is a small chance that we might not have reserved enough
> space for the outgoing arguments when this is done after virtual
> register instantiation.  You need -nostdlib when linking as
> various functions in libgcc uses the xmpyu instruction.

I'll enable it again for the 64-bit static port and see what happens :)

> > How so? -static and -nostdlib and add all the bits yourself?
> 
> No.  The dynamic loader is still required to resolve some special
> symbols.  I also think the file format is not quite right for
> a static executable.  I'm sure that it would be possible to generate
> a truly static binary but I'm not sure how much work is involved.

Special symbols?

c.

Re: [parisc-linux] Applications in 64 bits userspace

[John David Anglin]

> > No.  The dynamic loader is still required to resolve some special
> > symbols.  I also think the file format is not quite right for
> > a static executable.  I'm sure that it would be possible to generate
> > a truly static binary but I'm not sure how much work is involved.
> 
> Special symbols?

See chapter 6 in PA-RISC 64-Bit Runtime Architecture Supplement.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Applications in 64 bits userspace

[Carlos O'Donell]

> See chapter 6 in PA-RISC 64-Bit Runtime Architecture Supplement.

Stack unwinding and exception handling?
Does this code have to call into another library? (Or just millicode).

c.

Re: [parisc-linux] Applications in 64 bits userspace

[John David Anglin]

> > See chapter 6 in PA-RISC 64-Bit Runtime Architecture Supplement.
> 
> Stack unwinding and exception handling?

No, you are looking the main document.  The supplement is available
here <http://devresource.hp.com/STK/partner/pa64supp.pdf>.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)