forestalling GNU incompatibility - proposal for binary relative dynamic linking

forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

hey all,

Forgive the crosspost in advance, but I had an idea that touched many 
areas, and would need input from multiple groups associated with the 
gnu build chain, and perhaps the kernel itself.

After spending *two weeks* on various ways of building glibc, 
I'm convinced that the gnu/linux toolchain is in great danger of 
losing interoperability.

The main problem is that the glibc's supplied with each commercial 
system are *heavily* patched. My Suse 9.2 system has a rpm for 
glibc with fifty patches.  Fifty patches! 

Fifty patches which make the SuSE glibc binarily incompatible 
with the redhat, and so on.  And everything is incompatible
with the vanilla flavor.

All this makes me very skeptical that I'm ever going to get 
to a 'standard' out of the box glibc build. I build a standard 
glibc, and then all the supplied programs that come with SuSE 
break. And each vendor has the same problem - new that they are 
stuck with a heavily patched glibc, what chance do *they* have 
of getting back to a standard with the need to have old programs 
still work with the upgrades?









What IMO is desperately needed is a backwards compatibility 
hack or hacks. And of course the will of the different linux 
distribution providers to migrate back to the standard gnu 
toolchain.

Below is such a hack that lets providers do this, and 
solves quite a few problems in the process - the basic problem 
being that one currently can't get from a nonstandard glibc to
a standard one without quite a bit of pain - for migrating back 
breaks all the legacy binaries out there.


What is needed is the ability to reference multiple versions of the 
glibc WITHOUT changing my environment to do so.


Ie: If I set up my LD_LIBRARY_PATH to reference:

	setenv LD_LIBRARY_PATH /system/path/to/libc:.....


then the SuSE executables work fine but my new executables break, 
and if I set up my LD_LIBRARY_PATH to reference:

	setenv LD_LIBRARY_PATH /new/path/to/libc:.....

then my new executables work, but my *old* executables break.


What I'd like to do is be able to set up my LD_LIBRARY_PATH 
so that I can reference it from the point of view of the 
*executable*:

	setenv LD_LIBRARY_PATH   "*/../lib:....."




Here, read "* == full path of dirname of executable".

So if gcc was installed in say, "/opt/tools/bin/gcc", 
LD_LIBRARY_PATH would become at runtime "/opt/tools/bin/../lib" 
or "/opt/tools/lib". And hence if I had a libc.so installed 
there, it would pick it up and use it. 



Anyways, I have no idea whether or not this idea is being considered
or has been considered in the past, but AFAICT it would save me the 
trouble of having hundreds of wrapper scripts. And I would like to 
get an idea of what adding something like this to the gnu toolset would 
require. Discussion is welcome..


Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

(
    ps -

    just to forestall a particular objection that was pointed out to me 
    - simply use chroot - I don't want a chroot environment. I want to 
    use both old binaries and new binaries seamlessly.

    Ed
)

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Mike Frysinger]

On Monday 24 January 2005 05:24 pm, Edward Peschko wrote:
> After spending *two weeks* on various ways of building glibc,
> I'm convinced that the gnu/linux toolchain is in great danger of
> losing interoperability.

sounds like what you want is already being tackled by OSDL and their Binary 
Regression Testing group ...
http://groups.osdl.org/apps/group_public/workgroup.php?wg_abbrev=binary_sig
http://www.osdl.org/docs/isv_issues_and_binary_testing.pdf
-mike

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Richard Henderson]

On Mon, Jan 24, 2005 at 02:24:49PM -0800, Edward Peschko wrote:
> What I'd like to do is be able to set up my LD_LIBRARY_PATH 
> so that I can reference it from the point of view of the 
> *executable*:
> 
> 	setenv LD_LIBRARY_PATH   "*/../lib:....."
> 
> Here, read "* == full path of dirname of executable".

See -Wl,-rpath,'$ORIGIN/../lib/'


r~

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 06:08:52PM -0500, Mike Frysinger wrote:
> On Monday 24 January 2005 05:24 pm, Edward Peschko wrote:
> > After spending *two weeks* on various ways of building glibc,
> > I'm convinced that the gnu/linux toolchain is in great danger of
> > losing interoperability.
> 
> sounds like what you want is already being tackled by OSDL and their Binary 
> Regression Testing group ...
> http://groups.osdl.org/apps/group_public/workgroup.php?wg_abbrev=binary_sig
> http://www.osdl.org/docs/isv_issues_and_binary_testing.pdf
> -mike

well of course the osdl is going to focus on this, but they need tools
and functionality to do it correctly.. 

In particular, the statement 'Vendor lock-in (at any level) is not 
desirable' is false - there *is* vendor lock-in, and the suggestion 
of relative pathing for LD_LIBRARY_PATH is just one way to migrate 
back to doing things the right way.


Distributions are basically hoist by their own petard - they need to 
support old legacy executables, which have nonstandard glibc's. And since
they need to support legacy executables in the past, they need 
to support them in the future.


What I'd envision is that the distributions split basically into two: 
executables using the old style glibc/libraries, and executables using the 
new, standard glibcs. There would be two paths,


	/usr/bin

and

	/usr/standard/bin


The first directory would contain old executables which haven't been ported to
the standard glibc. The second would contain executables that have. The relative
pathing in LD_LIBRARY_PATH would insure that each tree used the correct libraries.


In the process of making /usr/standard/bin, linux vendors would need
to make their rpms both path-neutral and build clean. However, they would
*not* need to hold up their release process until everything is ported - 
they could pick and choose which applications were the most important to 
port. Ultimately, /usr/standard/bin would go away, and be moved back to 
/usr/bin, and perhaps the cycle could begin again, with upgrades going into 
a new /usr/standard/bin.

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 03:10:47PM -0800, Richard Henderson wrote:
> On Mon, Jan 24, 2005 at 02:24:49PM -0800, Edward Peschko wrote:
> > What I'd like to do is be able to set up my LD_LIBRARY_PATH 
> > so that I can reference it from the point of view of the 
> > *executable*:
> > 
> > 	setenv LD_LIBRARY_PATH   "*/../lib:....."
> > 
> > Here, read "* == full path of dirname of executable".
> 
> See -Wl,-rpath,'$ORIGIN/../lib/'
> 
> 
> r~

cool.. any chance for some syntactic sugar so me (and other 
users/vendors) wouldn't need to change any of their build scripts 
and compilation processes?

The only thing I would see as a drawback would be backwards compatibility,
but how often do people have directories named '*'?

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Richard Henderson]

On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> cool.. any chance for some syntactic sugar so me (and other 
> users/vendors) wouldn't need to change any of their build scripts 
> and compilation processes?

Uh, like what?  That's about as simple as you can get.


r~

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Edward Peschko]

On Mon, Jan 24, 2005 at 03:38:49PM -0800, Richard Henderson wrote:
> On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> > cool.. any chance for some syntactic sugar so me (and other 
> > users/vendors) wouldn't need to change any of their build scripts 
> > and compilation processes?
> 
> Uh, like what?  That's about as simple as you can get.
> 
> 
> r~

I don't understand. 

Which is simpler, changing an environmental variable, or adding extra 
CFLAGS to every single compile and recompiling?

In addition, in your --rpath example, the relative pathing is hardcoded
into the executable, wheras with "*" you could modify the runtime behavior
of the executable at runtime. I suppose you could change this with chrpath,
but why bother? What if you want to test out two versions of relative
libraries side by side? 

And in any case, I'm not even sure if you can change the runtime path 
to something longer than what currently exists in the executable using
chrunpath.


And finally, certain programs (glibc, for example) seem to get into to 
trouble (ie: not compile) when you use --rpath flags. So, what's the 
issue with "*"?

Ed

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Daniel Jacobowitz]

On Mon, Jan 24, 2005 at 03:53:11PM -0800, Edward Peschko wrote:
> On Mon, Jan 24, 2005 at 03:38:49PM -0800, Richard Henderson wrote:
> > On Mon, Jan 24, 2005 at 03:16:36PM -0800, Edward Peschko wrote:
> > > cool.. any chance for some syntactic sugar so me (and other 
> > > users/vendors) wouldn't need to change any of their build scripts 
> > > and compilation processes?
> > 
> > Uh, like what?  That's about as simple as you can get.
> > 
> > 
> > r~
> 
> I don't understand. 
> 
> Which is simpler, changing an environmental variable, or adding extra 
> CFLAGS to every single compile and recompiling?
> 
> In addition, in your --rpath example, the relative pathing is hardcoded
> into the executable, wheras with "*" you could modify the runtime behavior
> of the executable at runtime. I suppose you could change this with chrpath,
> but why bother? What if you want to test out two versions of relative
> libraries side by side? 

You might want to take a look at Richard's suggestion again.  The
string '$ORIGIN' gets hardcoded into the binary and handled by the
dynamic linker.

But really, RPATH is a good solution to almost no problems.

-- 
Daniel Jacobowitz

Re: forestalling GNU incompatibility - proposal for binary relative dynamic linking

[Adrian von Bidder]

[-- Attachment #1: Type: text/plain, Size: 2130 bytes --]

On Monday 24 January 2005 23.24, Edward Peschko wrote:
[ Linux interoperability in danger - libc != libc ]

Hi,

[this may be just ignorance - I only use one distro (Debian), and don't try 
to mix binaries]

Before discussing solutions: Is there a survey on how bad the problem really 
is?  IIRC some minor API changes were done between some gcc versions, but 
mostly on the C++ side.  I know Debian, too, ships with a quite heavily 
patched toolchain, but the Debian libc/binutils/gcc packagers work together 
with the upstream developers, and many Debian specific patches end up in 
the official releases sooner or later. (And most of Debian's patches 
concern architectures like ARM, m68k, HPPA etc., which the big 
distributions don't support anyway.)

So, I guess, the survey would need to compare only C programs (not C++), 
since you explicitly are talking about libc - C++ has always been more 
difficult.  And, it would be important to differentiate between 
incompatibilites caused by gcc versions from incompatibilities really 
caused by vendor specific modifications.

cheers
-- vbi

P.S.: looking at Debian's libc6 glibc package (2.3.2.ds1-20):
 - 106 patches total
Going by the name of the patch files only:
 - 8 hurd specific
 - 4 arm specific
 - 9 hppa specific
 - 4 m68k specific
 - 3 alpha specific
 - 1 amd64 specific
 - 1 x86 specific
 - 4 sparc specific
 - 5 mips specific
 - 3 ppc specific
 - 1 ia64 specific
 - 3 s390 specific
 - 4 correcting paths/build system only (wouldn't affect the libc API as 
such)
 - 15 locale specific (dito)

So, over half of the patches will likely not affect 95% of Linux users.  Of 
the rest, a good number clearly affect multithreaded programs only (which, 
admittedly, are much more widespread now than a few years back.)  I didn't 
look at any patches at all, so I can't judge how much the patches do really 
change the libc.  Also, I can't say how intrusive the patches of other 
distributions are.

-- 
We are not loved by our friends for what we are; rather, we are loved in
spite of what we are.
  -- Victor Hugo

[-- Attachment #2: Type: application/pgp-signature, Size: 388 bytes --]