Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Carsten Haitzler]

On Fri, 31 Oct 2008 12:00:45 +0100 (CET) git@amethyst.openembedded.net (GIT
User account) babbled:

is there any reason to run ldconfig on boot? this should be handled by opkg on
package install/upgrade or by the flash image creation (this slows boot
significantly and i've patched out/disabled ldconfig run in bootmisc.sh with no
ill effects).

> Module: openembedded.git
> Branch: org.openembedded.dev
> Commit: 6628886e858d77a8f9f94dd3229143e144d39f0e
> URL:
> http://gitweb.openembedded.net/?p=openembedded.git&a=commit;h=6628886e858d77a8f9f94dd3229143e144d39f0e
> 
> Author: Koen Kooi <koen@openembedded.org>
> Date:   Fri Oct 31 11:57:54 2008 +0100
> 
> initscripts: only run ldconfig on boot when ld.so.conf is present
> 
> ---
> 
>  packages/initscripts/initscripts-1.0/bootmisc.sh |    6 ++++--
>  packages/initscripts/initscripts_1.0.bb          |    2 +-
>  2 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/packages/initscripts/initscripts-1.0/bootmisc.sh
> b/packages/initscripts/initscripts-1.0/bootmisc.sh index 2a40e0c..dde1209
> 100755
> --- a/packages/initscripts/initscripts-1.0/bootmisc.sh
> +++ b/packages/initscripts/initscripts-1.0/bootmisc.sh
> @@ -57,9 +57,11 @@ then
>  fi
>  
>  #
> -# Update dynamic library cache
> +# Update dynamic library cache, but only if ld.so.conf is present
>  #
> -/sbin/ldconfig
> +if [ -e /etc/ld.so.conf ] ; then
> +	/sbin/ldconfig
> +fi
>  
>  # Set the system clock from hardware clock
>  # If the timestamp is 1 day or more recent than the current time,
> diff --git a/packages/initscripts/initscripts_1.0.bb
> b/packages/initscripts/initscripts_1.0.bb index b1c4317..8daefc5 100644
> --- a/packages/initscripts/initscripts_1.0.bb
> +++ b/packages/initscripts/initscripts_1.0.bb
> @@ -4,7 +4,7 @@ PRIORITY = "required"
>  DEPENDS = "makedevs"
>  RDEPENDS = "makedevs"
>  LICENSE = "GPL"
> -PR = "r109"
> +PR = "r110"
>  
>  SRC_URI = "file://functions \
>             file://halt \
> 
> 
> _______________________________________________
> Openembedded-commits mailing list
> Openembedded-commits@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-commits
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@rasterman.com

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Koen Kooi]

On 31-10-2008 13:37, Carsten Haitzler (The Rasterman) wrote:
> On Fri, 31 Oct 2008 12:00:45 +0100 (CET) git@amethyst.openembedded.net (GIT
> User account) babbled:
>
> is there any reason to run ldconfig on boot? this should be handled by opkg on
> package install/upgrade or by the flash image creation (this slows boot
> significantly and i've patched out/disabled ldconfig run in bootmisc.sh with no
> ill effects).

Not all images have a package management tool inside, so depending on 
that is a no-go. Recently OE stopped shipping ld.so.conf so the ldconfig 
on boot won't get run. Doing it this way seemed to me the safest way to 
get a nice speedup and keep things working for the funky people needing 
ld.so.conf without a package manager :)

regards,

Koen


>
>> Module: openembedded.git
>> Branch: org.openembedded.dev
>> Commit: 6628886e858d77a8f9f94dd3229143e144d39f0e
>> URL:
>> http://gitweb.openembedded.net/?p=openembedded.git&a=commit;h=6628886e858d77a8f9f94dd3229143e144d39f0e
>>
>> Author: Koen Kooi<koen@openembedded.org>
>> Date:   Fri Oct 31 11:57:54 2008 +0100
>>
>> initscripts: only run ldconfig on boot when ld.so.conf is present
>>
>> ---
>>
>>   packages/initscripts/initscripts-1.0/bootmisc.sh |    6 ++++--
>>   packages/initscripts/initscripts_1.0.bb          |    2 +-
>>   2 files changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/packages/initscripts/initscripts-1.0/bootmisc.sh
>> b/packages/initscripts/initscripts-1.0/bootmisc.sh index 2a40e0c..dde1209
>> 100755
>> --- a/packages/initscripts/initscripts-1.0/bootmisc.sh
>> +++ b/packages/initscripts/initscripts-1.0/bootmisc.sh
>> @@ -57,9 +57,11 @@ then
>>   fi
>>
>>   #
>> -# Update dynamic library cache
>> +# Update dynamic library cache, but only if ld.so.conf is present
>>   #
>> -/sbin/ldconfig
>> +if [ -e /etc/ld.so.conf ] ; then
>> +	/sbin/ldconfig
>> +fi
>>
>>   # Set the system clock from hardware clock
>>   # If the timestamp is 1 day or more recent than the current time,
>> diff --git a/packages/initscripts/initscripts_1.0.bb
>> b/packages/initscripts/initscripts_1.0.bb index b1c4317..8daefc5 100644
>> --- a/packages/initscripts/initscripts_1.0.bb
>> +++ b/packages/initscripts/initscripts_1.0.bb
>> @@ -4,7 +4,7 @@ PRIORITY = "required"
>>   DEPENDS = "makedevs"
>>   RDEPENDS = "makedevs"
>>   LICENSE = "GPL"
>> -PR = "r109"
>> +PR = "r110"
>>
>>   SRC_URI = "file://functions \
>>              file://halt \
>>
>>
>> _______________________________________________
>> Openembedded-commits mailing list
>> Openembedded-commits@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-commits
>>
>
>

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Carsten Haitzler]

On Fri, 31 Oct 2008 16:55:50 +0100 Koen Kooi <k.kooi@student.utwente.nl>
babbled:

> On 31-10-2008 13:37, Carsten Haitzler (The Rasterman) wrote:
> > On Fri, 31 Oct 2008 12:00:45 +0100 (CET) git@amethyst.openembedded.net (GIT
> > User account) babbled:
> >
> > is there any reason to run ldconfig on boot? this should be handled by opkg
> > on package install/upgrade or by the flash image creation (this slows boot
> > significantly and i've patched out/disabled ldconfig run in bootmisc.sh
> > with no ill effects).
> 
> Not all images have a package management tool inside, so depending on 
> that is a no-go. Recently OE stopped shipping ld.so.conf so the ldconfig 
> on boot won't get run. Doing it this way seemed to me the safest way to 
> get a nice speedup and keep things working for the funky people needing 
> ld.so.conf without a package manager :)

no package manager? gads. well - i'd lean to the "then you're on your own
buddy" line there - u install libs, you remember to run ldconfig, or suffer :)
maybe i'm just too harsh :) though your way is clean. why don't we make it

if [ ! -e /etc/ld.handled ]; then
  /sbin/ldconfig
fi

and opkg adds a /etc/ld.handled to its package?

similar effect to what you did butbasically if someone knows they will handle
ldconfig then they just need to throw in that file into their system one way or
another?

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@rasterman.com

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Koen Kooi]

On 31-10-2008 22:53, Carsten Haitzler (The Rasterman) wrote:
> On Fri, 31 Oct 2008 16:55:50 +0100 Koen Kooi<k.kooi@student.utwente.nl>
> babbled:
>
>> On 31-10-2008 13:37, Carsten Haitzler (The Rasterman) wrote:
>>> On Fri, 31 Oct 2008 12:00:45 +0100 (CET) git@amethyst.openembedded.net (GIT
>>> User account) babbled:
>>>
>>> is there any reason to run ldconfig on boot? this should be handled by opkg
>>> on package install/upgrade or by the flash image creation (this slows boot
>>> significantly and i've patched out/disabled ldconfig run in bootmisc.sh
>>> with no ill effects).
>> Not all images have a package management tool inside, so depending on
>> that is a no-go. Recently OE stopped shipping ld.so.conf so the ldconfig
>> on boot won't get run. Doing it this way seemed to me the safest way to
>> get a nice speedup and keep things working for the funky people needing
>> ld.so.conf without a package manager :)
>
> no package manager? gads. well - i'd lean to the "then you're on your own
> buddy" line there - u install libs, you remember to run ldconfig, or suffer :)
> maybe i'm just too harsh :) though your way is clean. why don't we make it
>
> if [ ! -e /etc/ld.handled ]; then
>    /sbin/ldconfig
> fi
>
> and opkg adds a /etc/ld.handled to its package?
>
> similar effect to what you did butbasically if someone knows they will handle
> ldconfig then they just need to throw in that file into their system one way or
> another?

I don't have a strong opinion on that :) People more knowledgable about 
ldconfig should chip in on this topic.

regards,

Koen

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Stanislav Brabec]

Koen Kooi wrote:

> I don't have a strong opinion on that :) People more knowledgable about 
> ldconfig should chip in on this topic.

And what about something like:
if ..../ld.so.cache -ot /usr/lib -o /etc/ld.so.cache -ot /lib ; then
    /sbin/ldconfig
fi

Call ldconfig only if ld.so cache is outdated. Fast for users of package
managers, working for others, fast in subsequent reboots.

I guess that all package managers use method "fill new file in target
directory, then replace existing", so timestamp of library directory is
updated on any change.

Note that /etc/ld.so.conf contain also /usr/local/lib (not exists by
default, but defined by FHS) and /usr/X11R6/lib (obsolete).


________________________________________________________________________
Stanislav Brabec
http://www.penguin.cz/~utx/zaurus

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Phil Blundell]

On Sat, 2008-11-01 at 12:38 +0100, Stanislav Brabec wrote:
> And what about something like:
> if ..../ld.so.cache -ot /usr/lib -o /etc/ld.so.cache -ot /lib ; then
>     /sbin/ldconfig
> fi

That would work fine for DISTROs which only put libraries in those two
directories.  But, frankly, if you only have those two library paths you
are probably better off not using ldconfig and ld.so.conf at all.

Even in the simple case I think a better solution would probably be to
teach ldconfig to do this checking of mtimes, rather than having to wrap
every call to it in a shell conditional.

> Call ldconfig only if ld.so cache is outdated. Fast for users of package
> managers, working for others, fast in subsequent reboots.

Just as an aside, there isn't really any need to have complicated
runtime logic to handle all possible combinations of DISTRO policy.  Any
single DISTRO would clearly have only one policy (for example, either it
uses a package manager or it doesn't) and hence it can make these
determinations at build time.  As an example of that, DISTROs which use
package management but don't require ldconfig can just set
USE_LDCONFIG=0 and suppress all those postinsts altogether, rather than
emitting a postinst with convoluted tests for file existence that ends
up doing nothing.

p.

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Koen Kooi]

On 01-11-2008 20:20, Phil Blundell wrote:

> Just as an aside, there isn't really any need to have complicated
> runtime logic to handle all possible combinations of DISTRO policy.  Any
> single DISTRO would clearly have only one policy (for example, either it
> uses a package manager or it doesn't) and hence it can make these
> determinations at build time.

That's awfully shortsighted and not the direction OE has been going in 
for the past years. A small example:

* kernel
* initramfs/initrd
* rootfs

Only the rootfs has a packagemanager, but the initramfs/initrd doesn't. 
This shows that a single distro can build image both with and without 
packagemanagement.
Saying "distros should only do <foo>" is for the distros to decide.

regards,

Koen

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Carsten Haitzler]

On Sat, 01 Nov 2008 12:38:16 +0100 Stanislav Brabec <utx@penguin.cz> babbled:

> Koen Kooi wrote:
> 
> > I don't have a strong opinion on that :) People more knowledgable about 
> > ldconfig should chip in on this topic.
> 
> And what about something like:
> if ..../ld.so.cache -ot /usr/lib -o /etc/ld.so.cache -ot /lib ; then
>     /sbin/ldconfig
> fi

you'd need to include all dirs in ld.so.conf in addition... also doesn't cover
me overwriting existing .so's in the lib dir with new ones. :)

really - if you install a lib from source or some tarball - run ldconfig. it's
been this way since the dawn of time. if you don't know this then the only sure
way to "fix" this to work is to reboot which is incredibly stupid. :) imho
running it on boot at all is just a silly idea full-stop. it should be run
immediately post-install/upgrade etc. of any shared lib(s) and they will
immediately be usable. a reboot is a horrible way to trigger that. :)

> Call ldconfig only if ld.so cache is outdated. Fast for users of package
> managers, working for others, fast in subsequent reboots.
> 
> I guess that all package managers use method "fill new file in target
> directory, then replace existing", so timestamp of library directory is
> updated on any change.

yes. as do most source installs using "install" (unlink old file, write new).
BUT if you are using a package manager.. it runs ldconfig for you.. and not on
boot - after install! if it doesn't - it is a fundamentally broken package
manager. source installs do not. as do simple untars of binaries. but if you
are doing this i'd expect you to have enough clue to run ldconfig - and not
just "reboot" to fix it :)

basically - i am trying to dig for a "sane justification" for putting ldconfig
in on boot. the only one i can think of is:

at some point when oe was building images, it was unable to run ldconfig using
qemu natively within the rootfs before converting to a file system image (ext2,
jffs2, or tar etc.) and thus as a workaround to not being able to do this, it
was run on boot to populate ld.so.cache. it's possible to do this now via qemu
before packaging up the rootfs.

ooh wait. i found out...

/var/run/ld.so.cache... it's in /var/run ... which is /var/volatile - which is
ramfs. which means it's not persistent. this... is a problem. well a problem
for systems that can do persistent storage and modify the fs runtime (and keep
the changes - ie anything using jffs2).

perhaps.. this is what needs changing - distros that have non-writable rootfs's
use ldconfig on boot (makes sense now) and those with, do it at image creating
time with qemu (and henceforth let the package manager handle it).

> Note that /etc/ld.so.conf contain also /usr/local/lib (not exists by
> default, but defined by FHS) and /usr/X11R6/lib (obsolete).
> 
> 
> ________________________________________________________________________
> Stanislav Brabec
> http://www.penguin.cz/~utx/zaurus
> 
> 
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@rasterman.com

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Denys Dmytriyenko]

On Sun, Nov 02, 2008 at 10:08:19AM +1100, Carsten Haitzler wrote:
> at some point when oe was building images, it was unable to run ldconfig using
> qemu natively within the rootfs before converting to a file system image (ext2,
> jffs2, or tar etc.) and thus as a workaround to not being able to do this, it
> was run on boot to populate ld.so.cache. it's possible to do this now via qemu
> before packaging up the rootfs.
> 
> ooh wait. i found out...
> 
> /var/run/ld.so.cache... it's in /var/run ... which is /var/volatile - which is
> ramfs. which means it's not persistent. this... is a problem. well a problem
> for systems that can do persistent storage and modify the fs runtime (and keep
> the changes - ie anything using jffs2).
> 
> perhaps.. this is what needs changing - distros that have non-writable rootfs's
> use ldconfig on boot (makes sense now) and those with, do it at image creating
> time with qemu (and henceforth let the package manager handle it).

I personally wouldn't want to rely on qemu for this. I'm trying to avoid it 
for my setups so I wouldn't need to deal with its weirdness and support my 
developers. And currently I only had to disable locales generation to achive 
that peace of mind...

-- 
Denys

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Carsten Haitzler]

On Sun, 02 Nov 2008 00:51:09 -0400 Denys Dmytriyenko <denis@denix.org> babbled:

> On Sun, Nov 02, 2008 at 10:08:19AM +1100, Carsten Haitzler wrote:
> > at some point when oe was building images, it was unable to run ldconfig
> > using qemu natively within the rootfs before converting to a file system
> > image (ext2, jffs2, or tar etc.) and thus as a workaround to not being able
> > to do this, it was run on boot to populate ld.so.cache. it's possible to do
> > this now via qemu before packaging up the rootfs.
> > 
> > ooh wait. i found out...
> > 
> > /var/run/ld.so.cache... it's in /var/run ... which is /var/volatile - which
> > is ramfs. which means it's not persistent. this... is a problem. well a
> > problem for systems that can do persistent storage and modify the fs
> > runtime (and keep the changes - ie anything using jffs2).
> > 
> > perhaps.. this is what needs changing - distros that have non-writable
> > rootfs's use ldconfig on boot (makes sense now) and those with, do it at
> > image creating time with qemu (and henceforth let the package manager
> > handle it).
> 
> I personally wouldn't want to rely on qemu for this. I'm trying to avoid it 
> for my setups so I wouldn't need to deal with its weirdness and support my 
> developers. And currently I only had to disable locales generation to achive 
> that peace of mind...

ld.so.cache doesn't NEED to be generated before packing the image together - it
works quite happily without it... :) so same would apply here.


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@rasterman.com

Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Stanislav Brabec]

Carsten Haitzler wrote:
> On Sat, 01 Nov 2008 12:38:16 +0100 Stanislav Brabec <utx@penguin.cz> babbled:
> 
> > Koen Kooi wrote:
> > 
> > > I don't have a strong opinion on that :) People more knowledgable about 
> > > ldconfig should chip in on this topic.
> > 
> > And what about something like:
> > if ..../ld.so.cache -ot /usr/lib -o /etc/ld.so.cache -ot /lib ; then
> >     /sbin/ldconfig
> > fi
> 
> you'd need to include all dirs in ld.so.conf in addition...

Yes.

> also doesn't cover
> me overwriting existing .so's in the lib dir with new ones. :)

Overwriting of existing .so is an extremely risky operation and I guess
that install command nor any packaging system do it. To safely
replace .so file, you have to unlink existing instance and use a
different inode for the new instance. This operation updates directory
mtime.

> run ldconfig.

I agree with the rest. Ldconfig should be called whenever any package is
installed.

> at some point when oe was building images, it was unable to run ldconfig using
> qemu natively within the rootfs before converting to a file system image (ext2,
> jffs2, or tar etc.) and thus as a workaround to not being able to do this, it
> was run on boot to populate ld.so.cache. it's possible to do this now via qemu
> before packaging up the rootfs.

It's not generally true. Some platforms have no support in qemu, some
platforms are broken in qemu and commands fail there. The command
proposed above may be a work-around for this case.

I can imagine one-time script for this purpose. Platform has no working
qemu support => packaging scripts will place one-time commands
somewhere.

Being inspired by DejaGnu, I can also imagine a native host as an
alternative solution for platforms without qemu support.

- Build (minimal) environment not needing any native run and being able
  to run on target.
- Mount a target root as a chroot on native host via a NFS.
- Using a ssh and chroot command, execute required code and get result
  back.

(Well, it would complicate life to Koen a bit.)

It could perform any task not possible on build system:
- run ldconfig
- generate locales (may need swap on native host)
- compile emacs macros
...


________________________________________________________________________
Stanislav Brabec
http://www.penguin.cz/~utx/zaurus

Running one-off target binaries, was: Re: [oe-commits] Koen Kooi : initscripts: only run ldconfig on boot when ld.so. conf is present

[Koen Kooi]

On 02-11-2008 11:52, Stanislav Brabec wrote:

> Being inspired by DejaGnu, I can also imagine a native host as an
> alternative solution for platforms without qemu support.
>
> - Build (minimal) environment not needing any native run and being able
>    to run on target.
> - Mount a target root as a chroot on native host via a NFS.
> - Using a ssh and chroot command, execute required code and get result
>    back.
>
> (Well, it would complicate life to Koen a bit.)
>
> It could perform any task not possible on build system:
> - run ldconfig
> - generate locales (may need swap on native host)
> - compile emacs macros
> ...

We do have sbrsh (from scratchbox fame) in OE, so one could write a 
class reusing scratchbox code to run such binaries on a network-attached 
target. Any volunteers for doing a proof-of-concept bbclass?

regards,

Koen