dynamic libraries

dynamic libraries

[Andrei Petrov]

While looking at tools build I noticed that same object files are used to build both
static and dynamic libraries (libxc, libxutil). It's normally required to compile in
PIC form for dynamic ones. At first I thought that no applcations use those libraries,
but no, they are used by all of them. Surprisingly it doens't cause any problems (at
least nobody's complaining, ld.so even).

Well, anyway it doesn't look correct for me, I'd like to come up with a patch
but it'll take some time, so I wouldn't mind if some Makefile savvy beat me on it.
Short term solution would be always set -fPIC.

--Andrei Petrov


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Keir Fraser]

On 23 Feb 2005, at 21:28, Andrei Petrov wrote:

> While looking at tools build I noticed that same object files are used 
> to build both
> static and dynamic libraries (libxc, libxutil). It's normally required 
> to compile in
> PIC form for dynamic ones. At first I thought that no applcations use 
> those libraries,
> but no, they are used by all of them. Surprisingly it doens't cause 
> any problems (at
> least nobody's complaining, ld.so even).
>
> Well, anyway it doesn't look correct for me, I'd like to come up with 
> a patch
> but it'll take some time, so I wouldn't mind if some Makefile savvy 
> beat me on it.
> Short term solution would be always set -fPIC.

Well, we have the functionality in the build system already; it just 
isn't the default.
Where is it stated that dynamic libraries must be compiled -fPIC? They 
contain
relocation info and so can be relocated wherever needed at run time. 
Potentially
less efficient (library relocated to different addresses cannot have 
its text pages
shared across processes) but I don't see why it is *disallowed*.

If there's a suitably convincing explanation/reason then I will change 
the default setting. :-)

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Adam Heath]

On Wed, 23 Feb 2005, Keir Fraser wrote:

>
> On 23 Feb 2005, at 21:28, Andrei Petrov wrote:
>
> > While looking at tools build I noticed that same object files are used
> > to build both
> > static and dynamic libraries (libxc, libxutil). It's normally required
> > to compile in
> > PIC form for dynamic ones. At first I thought that no applcations use
> > those libraries,
> > but no, they are used by all of them. Surprisingly it doens't cause
> > any problems (at
> > least nobody's complaining, ld.so even).
> >
> > Well, anyway it doesn't look correct for me, I'd like to come up with
> > a patch
> > but it'll take some time, so I wouldn't mind if some Makefile savvy
> > beat me on it.
> > Short term solution would be always set -fPIC.
>
> Well, we have the functionality in the build system already; it just
> isn't the default.
> Where is it stated that dynamic libraries must be compiled -fPIC? They
> contain
> relocation info and so can be relocated wherever needed at run time.
> Potentially
> less efficient (library relocated to different addresses cannot have
> its text pages
> shared across processes) but I don't see why it is *disallowed*.
>
> If there's a suitably convincing explanation/reason then I will change
> the default setting. :-)

Well, it just doesn't work at all on amd64, as well as other arches.  It just
happens to work on i386, so most people never notice it.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Keir Fraser]

On 23 Feb 2005, at 23:14, Adam Heath wrote:

>> Well, we have the functionality in the build system already; it just
>> isn't the default.
>> Where is it stated that dynamic libraries must be compiled -fPIC? They
>> contain
>> relocation info and so can be relocated wherever needed at run time.
>> Potentially
>> less efficient (library relocated to different addresses cannot have
>> its text pages
>> shared across processes) but I don't see why it is *disallowed*.
>>
>> If there's a suitably convincing explanation/reason then I will change
>> the default setting. :-)
>
> Well, it just doesn't work at all on amd64, as well as other arches.  
> It just
> happens to work on i386, so most people never notice it.

Okay, can we just build -fPIC for static libraries as well, or do they 
have to be non-PIC?
Building the libraries twice seems weird to me. :-)

Another option is not to bother building dynamic libraries but to 
statically link libxc and
libxutil. Avoids versioning problems and it's not like there's some big 
sharing win to
be had. Any strong opinions on this?

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Adam Heath]

On Wed, 23 Feb 2005, Keir Fraser wrote:

> Okay, can we just build -fPIC for static libraries as well, or do they
> have to be non-PIC?
> Building the libraries twice seems weird to me. :-)
>
> Another option is not to bother building dynamic libraries but to
> statically link libxc and
> libxutil. Avoids versioning problems and it's not like there's some big
> sharing win to
> be had. Any strong opinions on this?

Versioning is still a problem in the static case.

libxc requires code from libxutil.  If you have both as .a, then when a
third-party app links to libxc, the third-party app also has to link to
libxutil.  This is normally done with a helper script(ala gnome, etc).
Dynamic linking solves this.

Additionally, if libfoo links to libxc.a(and libxutil.a), and libbar also
links to libxc.a(and libxutil.a), then both libfoo and libbar will not only
have duplicate code, but may have different incompatible versions of it.  This
is a recipe for disaster.

Dynamic linking is really the way to go.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Keir Fraser]

On 23 Feb 2005, at 23:26, Adam Heath wrote:

> libxc requires code from libxutil.  If you have both as .a, then when a
> third-party app links to libxc, the third-party app also has to link to
> libxutil.  This is normally done with a helper script(ala gnome, etc).
> Dynamic linking solves this.
>
> Additionally, if libfoo links to libxc.a(and libxutil.a), and libbar 
> also
> links to libxc.a(and libxutil.a), then both libfoo and libbar will not 
> only
> have duplicate code, but may have different incompatible versions of 
> it.  This
> is a recipe for disaster.
>
> Dynamic linking is really the way to go.

Okay, well I shall change to -fPIC as the default and just not build 
static libraries. Seems pointless having both static and dynamic.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Adam Heath]

On Wed, 23 Feb 2005, Keir Fraser wrote:

>
> On 23 Feb 2005, at 23:26, Adam Heath wrote:
>
> > libxc requires code from libxutil.  If you have both as .a, then when a
> > third-party app links to libxc, the third-party app also has to link to
> > libxutil.  This is normally done with a helper script(ala gnome, etc).
> > Dynamic linking solves this.
> >
> > Additionally, if libfoo links to libxc.a(and libxutil.a), and libbar
> > also
> > links to libxc.a(and libxutil.a), then both libfoo and libbar will not
> > only
> > have duplicate code, but may have different incompatible versions of
> > it.  This
> > is a recipe for disaster.
> >
> > Dynamic linking is really the way to go.
>
> Okay, well I shall change to -fPIC as the default and just not build
> static libraries. Seems pointless having both static and dynamic.

Hehe, wrong again.

You can't debug dynamic libraries.  gdb can't load their symbols.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[smohekey]

On Wed, Feb 23, 2005 at 06:00:56PM -0600, Adam Heath wrote:
> 
> You can't debug dynamic libraries.  gdb can't load their symbols.
> 

Are you sure about this? I've been debugging dynamic libraries for some time now. Best results are seen with no optimisation and -ggdb to enable all debugging symbols.

Scott.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Adam Heath]

On Wed, 23 Feb 2005, smohekey@insightful.co.nz wrote:

>
> On Wed, Feb 23, 2005 at 06:00:56PM -0600, Adam Heath wrote:
> >
> > You can't debug dynamic libraries.  gdb can't load their symbols.
> >
>
> Are you sure about this? I've been debugging dynamic libraries for some
> time now. Best results are seen with no optimisation and -ggdb to enable all
> debugging symbols.

There's something that can't work with dynamic libs.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Andrei Petrov]

On Wed, Feb 23, 2005 at 05:14:14PM -0600, Adam Heath wrote:
> On Wed, 23 Feb 2005, Keir Fraser wrote:
> 
> >
> > On 23 Feb 2005, at 21:28, Andrei Petrov wrote:
> >
> > > While looking at tools build I noticed that same object files are used
> > > to build both
> > > static and dynamic libraries (libxc, libxutil). It's normally required
> > > to compile in
> > > PIC form for dynamic ones. At first I thought that no applcations use
> > > those libraries,
> > > but no, they are used by all of them. Surprisingly it doens't cause
> > > any problems (at
> > > least nobody's complaining, ld.so even).
> > >
> > > Well, anyway it doesn't look correct for me, I'd like to come up with
> > > a patch
> > > but it'll take some time, so I wouldn't mind if some Makefile savvy
> > > beat me on it.
> > > Short term solution would be always set -fPIC.
> >
> > Well, we have the functionality in the build system already; it just
> > isn't the default.
> > Where is it stated that dynamic libraries must be compiled -fPIC? They
> > contain
> > relocation info and so can be relocated wherever needed at run time.
> > Potentially
> > less efficient (library relocated to different addresses cannot have
> > its text pages
> > shared across processes) but I don't see why it is *disallowed*.
> >

AFAIK, you need position independant code for shared library, so -fPIC,
as man(1) says. I can imagine cases when code migth be PIC without
the flag, say pure code. But if you have data segment which supposed to
be relocatable and your code access it in position dependant mode, I'm
kind of doubt that would work correctly,  I need to experiment with that.

> > If there's a suitably convincing explanation/reason then I will change
> > the default setting. :-)
> 
> Well, it just doesn't work at all on amd64, as well as other arches.  It just
> happens to work on i386, so most people never notice it.

Yep, that how I noticed it.

And, mapping to different addresses doesn't hurt page sharing.

--Andrei Petrov


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Andrei Petrov]

On Wed, Feb 23, 2005 at 11:19:18PM +0000, Keir Fraser wrote:
> >
> >Well, it just doesn't work at all on amd64, as well as other arches.  
> >It just
> >happens to work on i386, so most people never notice it.
> 
> Okay, can we just build -fPIC for static libraries as well, or do they 
> have to be non-PIC?
> Building the libraries twice seems weird to me. :-)
> 

The most common case I saw is indeed build them twice. So if both library
are needed I'd do the same. PIC implies performance hit you can avoid in
static library.

> Another option is not to bother building dynamic libraries but to 
> statically link libxc and
> libxutil. Avoids versioning problems and it's not like there's some big 
> sharing win to
> be had. Any strong opinions on this?
> 

In production environment it is enought to have only dynamic library,
for development - static one sometimes is more convinient.

--Andrei Petrov


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: dynamic libraries

[Jacob Gorm Hansen]

Keir Fraser wrote:

> Another option is not to bother building dynamic libraries but to 
> statically link libxc and
> libxutil. Avoids versioning problems and it's not like there's some big 
> sharing win to
> be had. Any strong opinions on this?

Sounds like a good idea, I have been doing similar stuff for a while, it 
saves some installation/versioning headaches, though I guess the real 
win here is if/when the python stuff goes away.

Jacob


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: [Xen-devel][PATCH] dynamic libraries

[Vincent Hanquez]

On Wed, Feb 23, 2005 at 11:43:23PM +0000, Keir Fraser wrote:
> Okay, well I shall change to -fPIC as the default and just not build 
> static libraries. Seems pointless having both static and dynamic.

Hi Keir,

Without the following patch %ebx is spilled in fpic mode.
(%ebx hold the GOT in this mode)

--- xeno-unstable.bk/tools/libxc/xc_vmx_build.c.orig	2005-02-09 13:50:48 +0100
+++ xeno-unstable.bk/tools/libxc/xc_vmx_build.c	2005-02-09 13:51:33 +0100
@@ -426,10 +426,10 @@
 {
     int eax, ecx;
 
-    __asm__ __volatile__ ("cpuid" 
+    __asm__ __volatile__ ("pushl %%ebx; cpuid; popl %%ebx" 
 			  : "=a" (eax), "=c" (ecx) 
 			  : "0" (1) 
-			  : "bx", "dx");
+			  : "dx");
     if (!(ecx & VMX_FEATURE_FLAG)) {
         return -1;
     }


-- 
Vincent Hanquez


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: [Xen-devel][PATCH] dynamic libraries

[Keir Fraser]

On 24 Feb 2005, at 01:40, Vincent Hanquez wrote:

> Without the following patch %ebx is spilled in fpic mode.
> (%ebx hold the GOT in this mode)

Thank you. This spill is what originally caused me to disable -fPIC. :-)

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click