[Linux-ia64] One little, two little, three little endian...

[Linux-ia64] One little, two little, three little endian...

[Gavin Scott]

Hello list,

I want a big endian version of Linux for IPF.  Just how crazy am I?

I'm working with a group of customers and software developers who are
migrating away from HP's proprietary MPE/iX operating system on the HPe3000
systems that HP announced the discontinuance of one year ago today.  Many of
these people are already using HP-UX to some degree, and most are interested
in the possibility of using Linux in the future.

Of course both MPE/iX and HP-UX are big-endian environments, and many of the
people I talk to would be very interested in having an endian-compatible
Linux that would run on big (i.e. IA-64) HP servers.  For this group, x86
compatibility is probably a non-issue.

I'm interested in any comments that come to mind.  I'm most interested in
just how complex the task would be from a technical point of view, either to
make the system buildable either way, or possibly supporting a per-process
endian bit (which would be cool, but probably a lot more work).

I'm reasonably familiar with the IA-64 architecture and know most of what
there is to know about PA-RISC.

Thanks,

Gavin
--
Gavin Scott
Vice President
Allegro Consultants, Inc.
gavin@allegro.com

Re: [Linux-ia64] One little, two little, three little endian...

[David Mosberger]

Hi Gavin,

>>>>> On Thu, 14 Nov 2002 16:52:13 -0800, "Gavin Scott" <gavin@allegro.com> said:

  Gavin> Hello list, I want a big endian version of Linux for IPF.
  Gavin> Just how crazy am I?

Very! ;-)

Seriously though: running big-endian processes is the easy part, the
hard part is developing (& maintaining) big-endian versions of all
libraries.  Not only is it a lot of work, but it would also fragment
the ia64 linux market, so I'd consider it a Bad Thing (TM).

  Gavin> Of course both MPE/iX and HP-UX are big-endian environments,
  Gavin> and many of the people I talk to would be very interested in
  Gavin> having an endian-compatible Linux that would run on big
  Gavin> (i.e. IA-64) HP servers.  For this group, x86 compatibility
  Gavin> is probably a non-issue.

It would be a lot easier to add an hp-ux compatibility layer on top of
ia64 linux.  The layer would emulate the (big-endian) hp-ux system
calls and then you would install the big-endian hp-ux libraries on top
of that (e.g., in the /emul/ia64-hpux/ tree).  Of course, I'm ignoring
legal issues here as copying the hp-ux libraries would presumably
require some sort of license.  But legal issues aside, this is doable
and has been done many times before (e.g., alpha linux can run (some)
Tru64 binaries, sparc linux can run sunos/solaris(?) binaries, etc.).

  Gavin> I'm interested in any comments that come to mind.  I'm most
  Gavin> interested in just how complex the task would be from a
  Gavin> technical point of view, either to make the system buildable
  Gavin> either way, or possibly supporting a per-process endian bit
  Gavin> (which would be cool, but probably a lot more work).

Actually, adding a per-process endian bit would be easy.  In fact,
even without, a little-endian program can turn on the big-endian bit
as long as it's prepared to deal with the consequences.  The hard part
is really the libraries.

	--david

Re: [Linux-ia64] One little, two little, three little endian...

[Don Dugger]

My reaction is why?

If the HP-UX people are converting to Linux you're talking about porting
the applications anyway.  Why don't you just bite the bullet and port
the applications to little endian Linux and be done with it.  I know
there's a tendency to shudder when contemplating an endian port but it's
really not that difficult these days.  GCC is pretty good about flagging
most of the issues that get you in trouble so if you resolve all warnings
carefully most of the job is done.  It's actually easier to port from
big endian to little than vice versa so you'd be faced with the simpler
problem.

If the apps have assembly language it's going to be a complete port
anyway so porting to little endian doesn't add any work there.

Data compatibility is the only real gotcha I can think of and that's just
a translation issue.  Potentially creating the translation utilities will
be a problem but I predict it's a smaller problem than coming up with a
big endian IA64 Linux.

On Thu, Nov 14, 2002 at 04:52:13PM -0800, Gavin Scott wrote:
> Hello list,
> 
> I want a big endian version of Linux for IPF.  Just how crazy am I?
> 
> I'm working with a group of customers and software developers who are
> migrating away from HP's proprietary MPE/iX operating system on the HPe3000
> systems that HP announced the discontinuance of one year ago today.  Many of
> these people are already using HP-UX to some degree, and most are interested
> in the possibility of using Linux in the future.
> 
> Of course both MPE/iX and HP-UX are big-endian environments, and many of the
> people I talk to would be very interested in having an endian-compatible
> Linux that would run on big (i.e. IA-64) HP servers.  For this group, x86
> compatibility is probably a non-issue.
> 
> I'm interested in any comments that come to mind.  I'm most interested in
> just how complex the task would be from a technical point of view, either to
> make the system buildable either way, or possibly supporting a per-process
> endian bit (which would be cool, but probably a lot more work).
> 
> I'm reasonably familiar with the IA-64 architecture and know most of what
> there is to know about PA-RISC.
> 
> Thanks,
> 
> Gavin
> --
> Gavin Scott
> Vice President
> Allegro Consultants, Inc.
> gavin@allegro.com
> 
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

-- 
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
n0ano@n0ano.com

Re: [Linux-ia64] One little, two little, three little endian...

[Randolph Chung]

> I want a big endian version of Linux for IPF.  Just how crazy am I?
> 
> I'm working with a group of customers and software developers who are
> migrating away from HP's proprietary MPE/iX operating system on the HPe3000
> systems that HP announced the discontinuance of one year ago today.  Many of
> these people are already using HP-UX to some degree, and most are interested
> in the possibility of using Linux in the future.

how about running parisc-linux? :^)

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/

Re: [Linux-ia64] One little, two little, three little endian...

[Grant Grundler]

Don Dugger wrote:
> Data compatibility is the only real gotcha I can think of and that's just
> a translation issue.  Potentially creating the translation utilities will
> be a problem but I predict it's a smaller problem than coming up with a
> big endian IA64 Linux.

Have you ever tried to import wrong-endian data?

I thought people gave up trying to do that on 16-bit machines.
I've been in more endian flip-flops than I care to remember.
(Olivetti flip-flopped about every 2 years in the early 90s, talk
about customer nightmares...)

Data conversion has to be re-adressed by *every* application
because of padding. I think the work mostly cannot be amortized over
lots of customers.

I agree with david. User space libs need to be migrated from the
desired environment. See how far a per-process approach gets first.

grant

Re: [Linux-ia64] One little, two little, three little endian...

[Don Dugger]

Two points:

1)  I didn't say data conversion was easy just that the one time effort
of converting would be easier in the long run than the on-going effort
of creating and maintaining, for ever, big endian user libraries.

2)  I thought PA-RISC was a 32-bit processor which means porting to
IA64, assuming you're porting to 64-bit addressing (and if not why are
you porting to IA64 in the first place), will entail data incompatibilities
independent of endian-ness so you still have to do the data conversion.

It comes down to engineering trade offs and I still say that the one time
port will be easier in the long run.

On Thu, Nov 14, 2002 at 07:41:28PM -0800, Grant Grundler wrote:
> Don Dugger wrote:
> > Data compatibility is the only real gotcha I can think of and that's just
> > a translation issue.  Potentially creating the translation utilities will
> > be a problem but I predict it's a smaller problem than coming up with a
> > big endian IA64 Linux.
> 
> Have you ever tried to import wrong-endian data?
> 
> I thought people gave up trying to do that on 16-bit machines.
> I've been in more endian flip-flops than I care to remember.
> (Olivetti flip-flopped about every 2 years in the early 90s, talk
> about customer nightmares...)
> 
> Data conversion has to be re-adressed by *every* application
> because of padding. I think the work mostly cannot be amortized over
> lots of customers.
> 
> I agree with david. User space libs need to be migrated from the
> desired environment. See how far a per-process approach gets first.
> 
> grant
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

-- 
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
n0ano@n0ano.com

Re: [Linux-ia64] One little, two little, three little endian...

[Randolph Chung]

> 2)  I thought PA-RISC was a 32-bit processor which means porting to
> IA64, assuming you're porting to 64-bit addressing (and if not why are
> you porting to IA64 in the first place), will entail data incompatibilities
> independent of endian-ness so you still have to do the data conversion.

this is not true, many (most?) pa20 processors are 64-bit.

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/

Re: [Linux-ia64] One little, two little, three little endian...

[Matthew Wilcox]

On Thu, Nov 14, 2002 at 08:47:00PM -0800, Randolph Chung wrote:
> this is not true, many (most?) pa20 processors are 64-bit.

all PA2.0 processors are 64-bit.  it's possible, though not recommended,
to run a 64-bit Linux kernel on all PA2.0 boxes HP produced.  however
there's no 64-bit userspace for the Linux/PA port yet, so this is a
rather academic discussion.

-- 
Revolutions do not require corporate support.

Re: [Linux-ia64] One little, two little, three little endian...

[Mario Smarduch]

David Mosberger wrote:

> Hi Gavin,
>
> >>>>> On Thu, 14 Nov 2002 16:52:13 -0800, "Gavin Scott" <gavin@allegro.com> said:
>
>   Gavin> Hello list, I want a big endian version of Linux for IPF.
>   Gavin> Just how crazy am I?
>
> Very! ;-)
>
> Seriously though: running big-endian processes is the easy part, the
> hard part is developing (& maintaining) big-endian versions of all
> libraries.  Not only is it a lot of work, but it would also fragment
> the ia64 linux market, so I'd consider it a Bad Thing (TM).
>
>   Gavin> Of course both MPE/iX and HP-UX are big-endian environments,
>   Gavin> and many of the people I talk to would be very interested in
>   Gavin> having an endian-compatible Linux that would run on big
>   Gavin> (i.e. IA-64) HP servers.  For this group, x86 compatibility
>   Gavin> is probably a non-issue.
>
> It would be a lot easier to add an hp-ux compatibility layer on top of
> ia64 linux.  The layer would emulate the (big-endian) hp-ux system
> calls and then you would install the big-endian hp-ux libraries on top
> of that (e.g., in the /emul/ia64-hpux/ tree).  Of course, I'm ignoring
> legal issues here as copying the hp-ux libraries would presumably
> require some sort of license.  But legal issues aside, this is doable
> and has been done many times before (e.g., alpha linux can run (some)
> Tru64 binaries, sparc linux can run sunos/solaris(?) binaries, etc.).

David a  quick question. Even though the library problem is overcomed,
how would signal handling be accomplished by an application
running in big-endian. Doesn't the kernel fill in the siginfo and context
while runing in little endian?

- Mario.

>
>
>   Gavin> I'm interested in any comments that come to mind.  I'm most
>   Gavin> interested in just how complex the task would be from a
>   Gavin> technical point of view, either to make the system buildable
>   Gavin> either way, or possibly supporting a per-process endian bit
>   Gavin> (which would be cool, but probably a lot more work).
>
> Actually, adding a per-process endian bit would be easy.  In fact,
> even without, a little-endian program can turn on the big-endian bit
> as long as it's prepared to deal with the consequences.  The hard part
> is really the libraries.
>
>         --david
>
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

Re: [Linux-ia64] One little, two little, three little endian...

[David Mosberger]

>>>>> On Fri, 15 Nov 2002 10:11:56 -0600, Mario Smarduch <cms063@email.mot.com> said:

  Mario> David a quick question. Even though the library problem is
  Mario> overcomed, how would signal handling be accomplished by an
  Mario> application running in big-endian. Doesn't the kernel fill in
  Mario> the siginfo and context while runing in little endian?

Yes, signal-handling would have to be adjusted, just like we do for an
x86 process (the signal stack format is somewhat different for hp-ux
at any rate, so it's more than just endianness).

	--david

Re: [Linux-ia64] One little, two little, three little endian...

[Rich Altmaier]

FYI,
the Intel Fortran compiler runtime
has features for endianness conversion. 
See the environment variable  F_UFMTENDIAN. 
See  Intel Fortran Compiler Users's Guide, Customizing Compilation Environment,
Environment Variables,  Little-endian to Big-endian Conversion.

Of course ISVs today have faced the endianness problem for many years.
So most aps have already designed in features for data conversion.
You should push back very hard as to exactly what application really
has a problem here.  Clearly Fortran formatted data is not an issue, for e.g.
It would be pretty surprising if data generated for enterprise applications
were not already routinely interchanging with the little endian Windows environment.

To create an entirely new ABI is not a small deal.

Hopefully this is helpful,
Rich Altmaier, SGI



> 
> On Thu, Nov 14, 2002 at 04:52:13PM -0800, Gavin Scott wrote:
> > Hello list,
> >
> > I want a big endian version of Linux for IPF.  Just how crazy am I?
> >
> > I'm working with a group of customers and software developers who are
> > migrating away from HP's proprietary MPE/iX operating system on the HPe3000
> > systems that HP announced the discontinuance of one year ago today.  Many of
> > these people are already using HP-UX to some degree, and most are interested
> > in the possibility of using Linux in the future.
> >
> > Of course both MPE/iX and HP-UX are big-endian environments, and many of the
> > people I talk to would be very interested in having an endian-compatible
> > Linux that would run on big (i.e. IA-64) HP servers.  For this group, x86
> > compatibility is probably a non-issue.
> >
> > I'm interested in any comments that come to mind.  I'm most interested in
> > just how complex the task would be from a technical point of view, either to
> > make the system buildable either way, or possibly supporting a per-process
> > endian bit (which would be cool, but probably a lot more work).
> >
> > I'm reasonably familiar with the IA-64 architecture and know most of what
> > there is to know about PA-RISC.
> >
> > Thanks,
> >
> > Gavin
> > --
> > Gavin Scott
> > Vice President
> > Allegro Consultants, Inc.
> > gavin@allegro.com
> >
> >

RE: [Linux-ia64] One little, two little, three little endian...

[Gavin Scott]

Don writes:
> It comes down to engineering trade offs and I still say that the one time
> port will be easier in the long run.

The largest problem is the fuzziness of the "one time".  Customers
performing these migrations are likely to want to run in parallel through
their development and testing processes, and these can take years in
commercial environments.

During this period the issue of data compatibility can be quite important,
especially if they are doing any kid of periodic or real-time replication of
transactions between the old and new environments.

It's (somewhat) easier to migrate binary data for an application if you
don't have to identify all fields and their types in order to byte-juggle
them during conversion.  At least in cases where storage-compatible
compilers and architecture exist.

And then there's the technically-less-relevant but
the-customer-is-always-right issue that many people ask for, or feel better
with, or simply demand, a target system that matches their own personal
endian bigotry[1].

Gavin

[1] For example preferring "big endian" (a.k.a. God's Own Byte Order) over
"little endian" (a.k.a. Satan's Humorous Little Joke)

RE: [Linux-ia64] One little, two little, three little endian...

[Gavin Scott]

Thanks for all the comments.  They have been quite helpful.

Gavin