[parisc-linux] HPUX binary compatibility

[parisc-linux] HPUX binary compatibility

[Matthew Wilcox]

How far do we want to or are we able to go with making constants identical
between HPUX and Linux?

Some syscalls are going to have to be different between the two OSes --
ioctl is the obvious example -- but it would be nice to have to do as
little work as possible.

I've just been through errno.h making the Linux error numbers the same as
the HPUX ones.  This seems pretty sane; I can't imagine that it will have
any negative effect on anything.  Where it gets a little more thorny is
in signal.h -- HPUX uses more than 32 signals.  Is it going to negatively
impact Linux at all to use more than 32?  I see there is space reserved
for them, but I'd like someone to reassure me.

The real thing which bothers me about this is that doing this means that
HPUX and Linux are then allocating from the same numberspace, so where
Linux has things which HPUX doesn't, HPUX might later allocate the same
number for a different extension.

(Of course, if HP want to add Linux binary compatibility to HPUX, now
would be a great time to mention it :-)

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] HPUX binary compatibility

[Stan Sieler]

Re:

> How far do we want to or are we able to go with making constants identical
> between HPUX and Linux?
> 
> Some syscalls are going to have to be different between the two OSes --
> ioctl is the obvious example -- but it would be nice to have to do as
> little work as possible.

that's why I suggested shifting the Linux system call magic numbers
upwards, to avoid the HP range.  If the Linux ioctl is different than the
HP-UX one, then it should have a different syscall number.
 
> Linux has things which HPUX doesn't, HPUX might later allocate the same
> number for a different extension.

Not if HP agrees to reserve a range for Linux use.
 
> (Of course, if HP want to add Linux binary compatibility to HPUX, now
> would be a great time to mention it :-)

that sounds fair!
 
-- 
Stan Sieler                                          sieler@allegro.com
                                         http://www.allegro.com/sieler/

Re: [parisc-linux] HPUX binary compatibility

[Jeffrey A Law]

  In message <19990620194015.J30362@mencheca.ch.genedata.com>you write:
  > 
  > How far do we want to or are we able to go with making constants identical
  > between HPUX and Linux?
We never bothered at the UofU.  It didn't seem worth the headache to try and
get either HP or Berkeley to change syscall, errno or other constants.

  > I've just been through errno.h making the Linux error numbers the same as
  > the HPUX ones.  This seems pretty sane; I can't imagine that it will have
  > any negative effect on anything.  Where it gets a little more thorny is
  > in signal.h -- HPUX uses more than 32 signals.  Is it going to negatively
  > impact Linux at all to use more than 32?  I see there is space reserved
  > for them, but I'd like someone to reassure me.
Just write translators on the hpux emulation side.

ie, consider the linux values the canonical internal form.

You then have two external representations.

  First is linux.  Do nothing here since the internal & external representation
  is the same.

  Second is hpux.  Translate values as you enter/exit the kernel.  So for
  example, you may need a second syscall table to do handoff of syscalls
  after translating the syscall #.  On the exit path you probably need to
  translate errno values.  Similarly for signals.

jeff

Re: [parisc-linux] HPUX binary compatibility

[Matthew Wilcox]

On Sun, Jun 20, 1999 at 02:21:36PM -0600, Jeffrey A Law wrote:
> 
>   In message <19990620194015.J30362@mencheca.ch.genedata.com>you write:
>   > 
>   > How far do we want to or are we able to go with making constants identical
>   > between HPUX and Linux?
> We never bothered at the UofU.  It didn't seem worth the headache to try and
> get either HP or Berkeley to change syscall, errno or other constants.

syscall and errno values are already different on different Linux
architectures anyway, so we have a free hand here.

> Just write translators on the hpux emulation side.

Yes, this is certainly what we'll do.  But if we can be clever and get a
bunch of compatibility for free, then we should.

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

[parisc-linux] Assembly error: entry.S

[John David Anglin]

The following error occurs building the kernel:

gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
entry.S:219: macro `ENTRY_NAME' used without args
make[1]: *** [entry.o] Error 1

-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] Assembly error: entry.S

[Matthew Wilcox]

On Mon, Jun 21, 1999 at 11:06:27AM -0400, John David Anglin wrote:
> The following error occurs building the kernel:
> 
> gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
> entry.S:219: macro `ENTRY_NAME' used without args
> make[1]: *** [entry.o] Error 1

Delete line 219.  Oops.

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] Assembly error: entry.S

[John David Anglin]

> 
> On Mon, Jun 21, 1999 at 11:06:27AM -0400, John David Anglin wrote:
> > The following error occurs building the kernel:
> > 
> > gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
> > entry.S:219: macro `ENTRY_NAME' used without args
> > make[1]: *** [entry.o] Error 1
> 
> Delete line 219.  Oops.

and then

gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
entry.S: Assembler messages:
entry.S:228: Error: bad or irreducible absolute expression; zero assumed
make[1]: *** [entry.o] Error 1

-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] Assembly error: entry.S

[Matthew Wilcox]

On Mon, Jun 21, 1999 at 11:27:31AM -0400, John David Anglin wrote:
> > 
> > On Mon, Jun 21, 1999 at 11:06:27AM -0400, John David Anglin wrote:
> > > The following error occurs building the kernel:
> > > 
> > > gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
> > > entry.S:219: macro `ENTRY_NAME' used without args
> > > make[1]: *** [entry.o] Error 1
> > 
> > Delete line 219.  Oops.
> 
> and then
> 
> gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
> entry.S: Assembler messages:
> entry.S:228: Error: bad or irreducible absolute expression; zero assumed
> make[1]: *** [entry.o] Error 1

I don't know how to fix this one since I'm not sure what constants are
available.  The snippet is:

        .rept NR_syscalls-NR_Linux_syscalls
                ENTRY_NAME(sys_ni_syscall)
        .endr

which just pads the table out to the right size.  Evidently the number
256-182 = 74 is not representable.  What numbers are?  I'd guess 64 is.

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] Assembly error: entry.S

[John David Anglin]

> 
> On Mon, Jun 21, 1999 at 11:27:31AM -0400, John David Anglin wrote:
> > > 
> > > On Mon, Jun 21, 1999 at 11:06:27AM -0400, John David Anglin wrote:
> > > > The following error occurs building the kernel:
> > > > 
> > > > gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
> > > > entry.S:219: macro `ENTRY_NAME' used without args
> > > > make[1]: *** [entry.o] Error 1
> > > 
> > > Delete line 219.  Oops.
> > 
> > and then
> > 
> > gcc -D__KERNEL__ -I/ehic/a/pa/linux/include    -c -o entry.o entry.S
> > entry.S: Assembler messages:
> > entry.S:228: Error: bad or irreducible absolute expression; zero assumed
> > make[1]: *** [entry.o] Error 1
> 
> I don't know how to fix this one since I'm not sure what constants are
> available.  The snippet is:
> 
>         .rept NR_syscalls-NR_Linux_syscalls
>                 ENTRY_NAME(sys_ni_syscall)
>         .endr
> 
> which just pads the table out to the right size.  Evidently the number
> 256-182 = 74 is not representable.  What numbers are?  I'd guess 64 is.

The problem is NR_Linux_syscalls is not defined.  Define this to 182
and the file will assemble.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] Assembly error: entry.S

[Matthew Wilcox]

On Mon, Jun 21, 1999 at 11:55:42AM -0400, John David Anglin wrote:
> The problem is NR_Linux_syscalls is not defined.  Define this to 182
> and the file will assemble.

Ah.  It was supposed to be __NR_Linux_syscalls, of course.

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] Assembly error: entry.S

[John David Anglin]

> 
> On Mon, Jun 21, 1999 at 11:55:42AM -0400, John David Anglin wrote:
> > The problem is NR_Linux_syscalls is not defined.  Define this to 182
> > and the file will assemble.
> 
> Ah.  It was supposed to be __NR_Linux_syscalls, of course.

This is defined in asm/unistd.h.  However, it is not included.  I changed
the asm/segment.h include to asm/unistd.h since the segment.h include
is obsolete.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

[parisc-linux] Hack to head.S

[John David Anglin]

A recent change to head.S was made in order to get the kernel to
successfully link.  This change places the kernel entry points
stext and _stext in the 'data' subspace $FLUFFY$.  Aside from the
dubious practice of placing code in the data space, it looks to
me that the change is wrong.

The module init/main.c uses the symbol _stext to determine the
starting address of the kernel text.  Placing the start in the
data region will give a totally erroneous result for the length
of kernel text to profile.  Probably, init/main.c should use
the origin selected for text when linking (currently, 0x8000)
as the start of the kernel text.  Then, the kernel entry points
can be moved back into the standard text space.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] HPUX binary compatibility

[Stan Sieler]

Hi,

> > Just write translators on the hpux emulation side.
> 
> Yes, this is certainly what we'll do.  But if we can be clever and get a
> bunch of compatibility for free, then we should.

The problem with translators is that they don't work in all cases.

The most important case is where you want to link some .o files
together: some are compiled for Linux, some for HP-UX. 

*That's* why having different system call numbers is important.

-- 
Stan Sieler                                          sieler@allegro.com
                                         http://www.allegro.com/sieler/

Re: [parisc-linux] HPUX binary compatibility

[Jeffrey A Law]

  In message <199906211723.KAA39308@bart.allegro.com>you write:
  > The problem with translators is that they don't work in all cases.
  > 
  > The most important case is where you want to link some .o files
  > together: some are compiled for Linux, some for HP-UX. 
  > 
  > *That's* why having different system call numbers is important.
You really don't want to do that.  I strongly recommend against it.  In
5 years of working on a system which had hpux compatibility we had zero
need to do this kind of stuff and it just makes things a lot more complicated
than they need to be.

If we wanted to build an hpux app, we had a set of tools to build hpux apps.

jeff

Re: [parisc-linux] HPUX binary compatibility

[Stan Sieler]

Jeff writes:

>   > The problem with translators is that they don't work in all cases.
>   > 
>   > The most important case is where you want to link some .o files
>   > together: some are compiled for Linux, some for HP-UX. 
>   > 
>   > *That's* why having different system call numbers is important.

> You really don't want to do that.  I strongly recommend against it.  In

(Presuming "that" is "have different system call numbers")

> 5 years of working on a system which had hpux compatibility we had zero
> need to do this kind of stuff and it just makes things a lot more complicated
> than they need to be.

You're "compatible" if you can run an HP-UX app "out of the box" (e.g.,
restore it and run).  If you have to run it through a translator of some kind,
you aren't compatible.   (If the translator is part of the OS, and invoked 
automatically & invisibly, that's ok...been there, used that for 15 years 
on MPE)

So...how do you achieve this level of compatibility?  Simple...the HP-UX
system call numbers *must* be honored.  What does that mean, then, if
Linux has an identically named (not numbered) system call whose
semantics differ?  Precisely one thing: the kernel must be able to
differentiate between them.  There are two basic methods of doing this:

  1) different system call numbers

or 

  2) different gateway page for the system calls (e.g., 0xc0000000 for
     HP-UX, and 0xc0000040 for Linux).  (Note: I don't recall the exact
     digits HP-UX uses...the values above are for illustrative purposes)
     (The kernel could check to see which address had been used for the
     call)

Without such a mechanism, you can't tell the system calls apart.

So, the only unanswered question is: is HP-UX compatibility desired?  
If the answer is "yes", the subsequent question (how) was answered above.

-- 
Stan Sieler                                          sieler@allegro.com
                                         http://www.allegro.com/sieler/

Re: [parisc-linux] HPUX binary compatibility

[Jeffrey A Law]

  In message <199906211807.LAA29277@bart.allegro.com>you write:
  > > You really don't want to do that.  I strongly recommend against it.  In
  > 
  > (Presuming "that" is "have different system call numbers")
Correct.

  > You're "compatible" if you can run an HP-UX app "out of the box" (e.g.,
  > restore it and run).  If you have to run it through a translator of some ki
  > nd,
  > you aren't compatible.   (If the translator is part of the OS, and invoked 
  > automatically & invisibly, that's ok...been there, used that for 15 years 
  > on MPE)
The translator is entirely in the kernel.  It does not require hacking the
binary in any way shape or form.

  > Without such a mechanism, you can't tell the system calls apart.
Yes you can.  The exec headers have different magic #s  which you can use
to identify hpux vs bsd vs osf vs lites vs linux binaries.

If the process is not a linux process, then you look up the syscall number
in the appropriate translation table.  Similarly for signal #s, errno values,
etc.

I strongly recommend you actually look at the code Utah wrote.  It handles
hpux compatibility well enough to run X servers, compilers, shells, etc etc
right off an hpux distribution tape/cd.

jeff

Re: [parisc-linux] HPUX binary compatibility

[Alan Cox]

> any negative effect on anything.  Where it gets a little more thorny is
> in signal.h -- HPUX uses more than 32 signals.  Is it going to negatively
> impact Linux at all to use more than 32?  I see there is space reserved
> for them, but I'd like someone to reassure me.

How many non RT signals has HP/UX got. If its more than 32 we should allow
for it, otherwise whoever does HP/UX compatibility will hate you 

Re: [parisc-linux] HPUX binary compatibility

[Matthew Wilcox]

On Sun, Jun 20, 1999 at 10:05:16PM +0100, Alan Cox wrote:
> > any negative effect on anything.  Where it gets a little more thorny is
> > in signal.h -- HPUX uses more than 32 signals.  Is it going to negatively
> > impact Linux at all to use more than 32?  I see there is space reserved
> > for them, but I'd like someone to reassure me.
> 
> How many non RT signals has HP/UX got. If its more than 32 we should allow
> for it, otherwise whoever does HP/UX compatibility will hate you 

1-36 are non-RT, 37-44 are RT.  The header file contains the ominous
comment:

/* New signals: assign numbers before _SIGRTMIN, and increase _SIGRTMIN
   and _SIGRTMAX to keep at end. */

The signal.h I committed last night gives Linux/PARISC ordinary signals
from 1-36 and RT signals from 37-64.  I reused _SIGGFAULT (Graphics
framebuffer fault) to be SIGSTKFLT for which HPUX has no equivalent.  My
question was, will having more than 32 signals annoy any Linux people?

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] HPUX binary compatibility

[Alan Cox]

> framebuffer fault) to be SIGSTKFLT for which HPUX has no equivalent.  My
> question was, will having more than 32 signals annoy any Linux people?

No. It just makes the signal delivery code a microscopic bit trickier


Alan

Re: [parisc-linux] HPUX binary compatibility

[Larry Dwyer]

At 07:40 PM 6/20/99 +0200, Matthew Wilcox wrote:
>
>How far do we want to or are we able to go with making constants identical
>between HPUX and Linux?
>
> ...
>The real thing which bothers me about this is that doing this means that
>HPUX and Linux are then allocating from the same numberspace, so where
>Linux has things which HPUX doesn't, HPUX might later allocate the same
>number for a different extension.
>

I would encourage converging the vector constants for Linux on PA with the
same constants for HP-UX on PA.

If there are some extra constants in Linux, I can arrange to have them
reserved in HP-UX.

Do not assume they are reserved until I post to this group (in other words,
every enq needs an ack).  It is best to send mail directly to me since I do
not read this mailing list as often as I ought to.


>(Of course, if HP want to add Linux binary compatibility to HPUX, now
>would be a great time to mention it :-)

Let's just say that we do not want to preclude binary compatibility at some
time in the future.


		Cheers,
		Larry

Re: [parisc-linux] HPUX binary compatibility

[Matthew Wilcox]

On Mon, Jun 21, 1999 at 02:39:29PM -0700, Larry Dwyer wrote:
> I would encourage converging the vector constants for Linux on PA with the
> same constants for HP-UX on PA.
> 
> If there are some extra constants in Linux, I can arrange to have them
> reserved in HP-UX.
> 
> Do not assume they are reserved until I post to this group (in other words,
> every enq needs an ack).  It is best to send mail directly to me since I do
> not read this mailing list as often as I ought to.

That's perfect.  We're pretty flexible for the moment as we don't
have any userland yet.  There are 24 error codes defined in Linux which
are not defined in HPUX.  How about reserving the block from 129-192
for Linux error numbers?

(if you're interested, these are defined in linux/asm-parisc/errno.h).

I'm not entirely sure what to do about signals.  Linux has only one signal
which HP/UX does not -- SIGSTKFLT.  As I said, I reused _SIGGFAULT as
we don't have a use for it in Linux (yet?).  What do you think is the
right thing to do here?  I'm extremely reluctant to place it after the
realtime signals, or move the realtime signals, and there aren't any gaps
(unless you count _SIGRESERVE.  hmm.. but Linux has a SIGRESERVE too,
maybe that will get used officially one day.)

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] HPUX binary compatibility

[Alan Cox]

> I'm not entirely sure what to do about signals.  Linux has only one signal
> which HP/UX does not -- SIGSTKFLT.  As I said, I reused _SIGGFAULT as
> we don't have a use for it in Linux (yet?).  What do you think is the

I don't think we use SIGSTKFLT either do we ...

Re: [parisc-linux] HPUX binary compatibility

[Matthew Wilcox]

On Tue, Jun 22, 1999 at 10:49:10AM +0100, Alan Cox wrote:
> > I'm not entirely sure what to do about signals.  Linux has only one signal
> > which HP/UX does not -- SIGSTKFLT.  As I said, I reused _SIGGFAULT as
> > we don't have a use for it in Linux (yet?).  What do you think is the
> 
> I don't think we use SIGSTKFLT either do we ...

Oh, you're right.

mrw@mencheca:/usr/src/parisc/linux$ find . -type f |xargs grep SIGSTKFLT
./include/asm-arm/signal.h:#define SIGSTKFLT    16
./include/asm-i386/signal.h:#define SIGSTKFLT   16
./include/asm-m68k/signal.h:#define SIGSTKFLT   16
./include/asm-parisc/signal.h:#define SIGSTKFLT 36
./include/asm-ppc/signal.h:#define SIGSTKFLT    16

So not only don't we currently use it, the alpha, mips, sparc and sparc64
ports don't have it.  Would a patch to remove it from all architectures
be accepted, do you think?

-- 
Matthew Wilcox <willy@bofh.ai>
"Windows and MacOS are products, contrived by engineers in the service of
specific companies. Unix, by contrast, is not so much a product as it is a
painstakingly compiled oral history of the hacker subculture." - N Stephenson

Re: [parisc-linux] HPUX binary compatibility

[Mike Hibler]

> From: Stan Sieler <sieler@allegro.com>
> Subject: Re: [parisc-linux] HPUX binary compatibility
> To: Matthew.Wilcox@genedata.com (Matthew Wilcox)
> Date: Mon, 21 Jun 1999 10:23:17 -0700 (PDT)
> 
> Hi,
> 
> > > Just write translators on the hpux emulation side.
> > 
> > Yes, this is certainly what we'll do.  But if we can be clever and get a
> > bunch of compatibility for free, then we should.
> 
> The problem with translators is that they don't work in all cases.
> 
> The most important case is where you want to link some .o files
> together: some are compiled for Linux, some for HP-UX. 
> 
> *That's* why having different system call numbers is important.
> 
> -- 
> Stan Sieler                                          sieler@allegro.com
>                                          http://www.allegro.com/sieler/

Yow, that is an ambitious goal!  I can see the desirability of doing it
this way, but I am afraid it may not work.  No example comes to mind but
I worry about subtle assumptions in code written for HP-UX; i.e., merely
defining your structures and constants the same as HP-UX may not be enough.

Like Jeff said, we got by (and still do) by either building such applications
on an HP-UX box or using an emulated HP-UX environment to create real HP-UX
binaries.

Hmm...just read Stan's last message.  There seems to be some confusion about
what we do for compatibility.  When we exec a binary, it is identified as
either being HP-UX or "native" (BSD).  That determines which syscall mapping
is used.  There is no need to have distinct name spaces this way.

Does no other Linux port provide native OS compatibility?

Re: [parisc-linux] HPUX binary compatibility

[Stan Sieler]

Re:

> Yow, that is an ambitious goal!  I can see the desirability of doing it
 
Yes...I've been trying to get MPE/iX to support HP-UX binaries via this
method for a number of years, but HP hasn't shown much interest.

The reason I like it is that it removes the slightly obscene distinction
of a "linux process" vs. "HP-UX process".

(MPE/iX, which also runs on PA-RISC, and which has an internal emulator
for the Classic (CISC) HP 3000 instruction set, was originally going to
have a similar distinction: Native Mode process vs. Compatibility Mode process ...
but people pointed out, quite correctly, that a process might want
to switch from NM to CM to NM ... so a process is just a process ... the
NM/CM mode is no longer a type of process.)

What does the MPE example have to do with Linux?  Everything...what happens
when your "linux process" wants to dynamically load code from an HP-UX
library?  Or vice versa?  What about shared libraries?

Take it from someone who's been there: having a "process type" isn't
a 100% solution.  Having non-overlapping system call mechanism (either
different numbers or a different address) provides a much closer to
100% solution.  You may still need to translate things to provide
data back to an HP-UX system call (e.g., process structure for pstat?),
but that's *doable* ... and the translator method won't get that far!

-- 
Stan Sieler                                          sieler@allegro.com
                                         http://www.allegro.com/sieler/