eglibc configurability

eglibc configurability

[Gary Thomas]

I often run small (slow) embedded systems with only a ramdisk
based file system.  When I use Poky for this, one side effect
is that some packages need to be "configured" on bootup, which
in the case of a ramdisk based operation means every time.

I notice that the eglibc package brings in a couple of these
which are problematic (mostly in how long they take to run)
Looking at meta/conf/distro/include/poky-eglibc.inc:

LIBC_DEPENDENCIES = "libsegfault \
                      eglibc \
                      eglibc-dbg \
                      eglibc-dev \
                      eglibc-utils \
                      eglibc-thread-db \
                      eglibc-localedata-i18n \
                      eglibc-gconv-ibm850 \
                      eglibc-gconv-cp1252 \
                      eglibc-gconv-iso8859-1 \
                      eglibc-gconv-iso8859-15 \
                      locale-base-en-us \
                      locale-base-en-gb "

On my OMAP-L138 target, configuring locale-base-* takes
a long time, upwards of 35 seconds each.

Are multiple locale-base packages really necessary?
How could I best (in the Poky spirit) limit this?  In the
minimum, I'd like to only have one locale, saving at least
35 seconds of boot time.

Ideas?  Comments?

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: eglibc configurability

[Kamble, Nitin A]

> -----Original Message-----
> From: poky-bounces@yoctoproject.org [mailto:poky-
> bounces@yoctoproject.org] On Behalf Of Gary Thomas
> Sent: Monday, February 07, 2011 6:44 AM
> To: Poky
> Subject: [poky] eglibc configurability
> 
> I often run small (slow) embedded systems with only a ramdisk
> based file system.  When I use Poky for this, one side effect
> is that some packages need to be "configured" on bootup, which
> in the case of a ramdisk based operation means every time.
> 
> I notice that the eglibc package brings in a couple of these
> which are problematic (mostly in how long they take to run)
> Looking at meta/conf/distro/include/poky-eglibc.inc:
> 
> LIBC_DEPENDENCIES = "libsegfault \
>                       eglibc \
>                       eglibc-dbg \
>                       eglibc-dev \
>                       eglibc-utils \
>                       eglibc-thread-db \
>                       eglibc-localedata-i18n \
>                       eglibc-gconv-ibm850 \
>                       eglibc-gconv-cp1252 \
>                       eglibc-gconv-iso8859-1 \
>                       eglibc-gconv-iso8859-15 \
>                       locale-base-en-us \
>                       locale-base-en-gb "
> 
> On my OMAP-L138 target, configuring locale-base-* takes
> a long time, upwards of 35 seconds each.
> 
> Are multiple locale-base packages really necessary?
> How could I best (in the Poky spirit) limit this?  In the
> minimum, I'd like to only have one locale, saving at least
> 35 seconds of boot time.
> 
> Ideas?  Comments?

Gary,
  Look at the LIMIT_BUILT_LOCALES variable in local.conf. I think that would give you what are you looking for. Also you can enable the mklibs optimizations for such images, which would save image size.


> 
> Thanks
> 
> --
> ------------------------------------------------------------
> Gary Thomas                 |  Consulting for the
> MLB Associates              |    Embedded world
> ------------------------------------------------------------
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky

Re: eglibc configurability

[Gary Thomas]

On 02/07/2011 08:45 AM, Kamble, Nitin A wrote:
>> -----Original Message-----
>> From: poky-bounces@yoctoproject.org [mailto:poky-
>> bounces@yoctoproject.org] On Behalf Of Gary Thomas
>> Sent: Monday, February 07, 2011 6:44 AM
>> To: Poky
>> Subject: [poky] eglibc configurability
>>
>> I often run small (slow) embedded systems with only a ramdisk
>> based file system.  When I use Poky for this, one side effect
>> is that some packages need to be "configured" on bootup, which
>> in the case of a ramdisk based operation means every time.
>>
>> I notice that the eglibc package brings in a couple of these
>> which are problematic (mostly in how long they take to run)
>> Looking at meta/conf/distro/include/poky-eglibc.inc:
>>
>> LIBC_DEPENDENCIES = "libsegfault \
>>                        eglibc \
>>                        eglibc-dbg \
>>                        eglibc-dev \
>>                        eglibc-utils \
>>                        eglibc-thread-db \
>>                        eglibc-localedata-i18n \
>>                        eglibc-gconv-ibm850 \
>>                        eglibc-gconv-cp1252 \
>>                        eglibc-gconv-iso8859-1 \
>>                        eglibc-gconv-iso8859-15 \
>>                        locale-base-en-us \
>>                        locale-base-en-gb "
>>
>> On my OMAP-L138 target, configuring locale-base-* takes
>> a long time, upwards of 35 seconds each.
>>
>> Are multiple locale-base packages really necessary?
>> How could I best (in the Poky spirit) limit this?  In the
>> minimum, I'd like to only have one locale, saving at least
>> 35 seconds of boot time.
>>
>> Ideas?  Comments?
>
> Gary,
>    Look at the LIMIT_BUILT_LOCALES variable in local.conf. I think that would give you what are you looking for. Also you can enable the mklibs optimizations for such images, which would save image size.

That variable does not seem to be used in the eglibc recipes,
only the older [deprecated] glibc.  Or did I miss something?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: eglibc configurability

[Richard Purdie]

On Mon, 2011-02-07 at 07:44 -0700, Gary Thomas wrote:
> I often run small (slow) embedded systems with only a ramdisk
> based file system.  When I use Poky for this, one side effect
> is that some packages need to be "configured" on bootup, which
> in the case of a ramdisk based operation means every time.
> 
> I notice that the eglibc package brings in a couple of these
> which are problematic (mostly in how long they take to run)
> Looking at meta/conf/distro/include/poky-eglibc.inc:
> 
> LIBC_DEPENDENCIES = "libsegfault \
>                       eglibc \
>                       eglibc-dbg \
>                       eglibc-dev \
>                       eglibc-utils \
>                       eglibc-thread-db \
>                       eglibc-localedata-i18n \
>                       eglibc-gconv-ibm850 \
>                       eglibc-gconv-cp1252 \
>                       eglibc-gconv-iso8859-1 \
>                       eglibc-gconv-iso8859-15 \
>                       locale-base-en-us \
>                       locale-base-en-gb "
> 
> On my OMAP-L138 target, configuring locale-base-* takes
> a long time, upwards of 35 seconds each.
> 
> Are multiple locale-base packages really necessary?
> How could I best (in the Poky spirit) limit this?  In the
> minimum, I'd like to only have one locale, saving at least
> 35 seconds of boot time.
> 
> Ideas?  Comments?

Shouldn't the cross locale generation be generating the locales at build
time meaning the locales shouldn't be generated on the device?

Cheers,

Richard

Re: eglibc configurability

[Gary Thomas]

On 02/07/2011 09:06 AM, Richard Purdie wrote:
> On Mon, 2011-02-07 at 07:44 -0700, Gary Thomas wrote:
>> I often run small (slow) embedded systems with only a ramdisk
>> based file system.  When I use Poky for this, one side effect
>> is that some packages need to be "configured" on bootup, which
>> in the case of a ramdisk based operation means every time.
>>
>> I notice that the eglibc package brings in a couple of these
>> which are problematic (mostly in how long they take to run)
>> Looking at meta/conf/distro/include/poky-eglibc.inc:
>>
>> LIBC_DEPENDENCIES = "libsegfault \
>>                        eglibc \
>>                        eglibc-dbg \
>>                        eglibc-dev \
>>                        eglibc-utils \
>>                        eglibc-thread-db \
>>                        eglibc-localedata-i18n \
>>                        eglibc-gconv-ibm850 \
>>                        eglibc-gconv-cp1252 \
>>                        eglibc-gconv-iso8859-1 \
>>                        eglibc-gconv-iso8859-15 \
>>                        locale-base-en-us \
>>                        locale-base-en-gb "
>>
>> On my OMAP-L138 target, configuring locale-base-* takes
>> a long time, upwards of 35 seconds each.
>>
>> Are multiple locale-base packages really necessary?
>> How could I best (in the Poky spirit) limit this?  In the
>> minimum, I'd like to only have one locale, saving at least
>> 35 seconds of boot time.
>>
>> Ideas?  Comments?
>
> Shouldn't the cross locale generation be generating the locales at build
> time meaning the locales shouldn't be generated on the device?

I'm not sure the details, I just know that it prints these
messages on boot:
   Configuring dbus-1.
    Adding system startup for /etc/init.d/dbus-1.
   Configuring locale-base-en-gb.
   Configuring locale-base-en-us.

The last two are my concern since they take 35 seconds each to run.
It's a deeply embedded system and does not need multi-locale support
at any level.

In any case, my comment was that there doesn't seem to be any way
to control/configure this at initial build time.  Per the previous
reply, I don't see how LIMIT_BUILT_LOCALES affects this at all.


-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: eglibc configurability

[Mark Hatle]

Could you find the related init script and point me to the contents.  I don't
see any initscripts in the eglibc integration.  The only thing I see is a switch
in the locale generation between on target, on host and via QEMU.  I'm wondering
if maybe this is being triggered?

--Mark

On 2/7/11 10:19 AM, Gary Thomas wrote:
> On 02/07/2011 09:06 AM, Richard Purdie wrote:
>> On Mon, 2011-02-07 at 07:44 -0700, Gary Thomas wrote:
>>> I often run small (slow) embedded systems with only a ramdisk
>>> based file system.  When I use Poky for this, one side effect
>>> is that some packages need to be "configured" on bootup, which
>>> in the case of a ramdisk based operation means every time.
>>>
>>> I notice that the eglibc package brings in a couple of these
>>> which are problematic (mostly in how long they take to run)
>>> Looking at meta/conf/distro/include/poky-eglibc.inc:
>>>
>>> LIBC_DEPENDENCIES = "libsegfault \
>>>                        eglibc \
>>>                        eglibc-dbg \
>>>                        eglibc-dev \
>>>                        eglibc-utils \
>>>                        eglibc-thread-db \
>>>                        eglibc-localedata-i18n \
>>>                        eglibc-gconv-ibm850 \
>>>                        eglibc-gconv-cp1252 \
>>>                        eglibc-gconv-iso8859-1 \
>>>                        eglibc-gconv-iso8859-15 \
>>>                        locale-base-en-us \
>>>                        locale-base-en-gb "
>>>
>>> On my OMAP-L138 target, configuring locale-base-* takes
>>> a long time, upwards of 35 seconds each.
>>>
>>> Are multiple locale-base packages really necessary?
>>> How could I best (in the Poky spirit) limit this?  In the
>>> minimum, I'd like to only have one locale, saving at least
>>> 35 seconds of boot time.
>>>
>>> Ideas?  Comments?
>>
>> Shouldn't the cross locale generation be generating the locales at build
>> time meaning the locales shouldn't be generated on the device?
> 
> I'm not sure the details, I just know that it prints these
> messages on boot:
>    Configuring dbus-1.
>     Adding system startup for /etc/init.d/dbus-1.
>    Configuring locale-base-en-gb.
>    Configuring locale-base-en-us.
> 
> The last two are my concern since they take 35 seconds each to run.
> It's a deeply embedded system and does not need multi-locale support
> at any level.
> 
> In any case, my comment was that there doesn't seem to be any way
> to control/configure this at initial build time.  Per the previous
> reply, I don't see how LIMIT_BUILT_LOCALES affects this at all.
> 
> 

Re: eglibc configurability

[Gary Thomas]

On 02/07/2011 09:29 AM, Mark Hatle wrote:
> Could you find the related init script and point me to the contents.  I don't
> see any initscripts in the eglibc integration.  The only thing I see is a switch
> in the locale generation between on target, on host and via QEMU.  I'm wondering
> if maybe this is being triggered?

This could be what I'm seeing.  I have ENABLE_BINARY_LOCALE_GENERATION disabled
in my local.conf (I had troubles with QEMU-ARM on Fedora in the past and this
was the way around it).  It looks like that may be pushing the locale compilation
to the target.  I see these scripts on my target:
-rwxr-xr-x    1 root     root           456 Feb  4 22:18 /var/lib/opkg/info/locale-base-en-gb.postinst
-rwxr-xr-x    1 root     root           456 Feb  4 22:19 /var/lib/opkg/info/locale-base-en-us.postinst

One of which looks like this:
------------------------------------------------------------------------------------------
# cat /var/lib/opkg/info/locale-base-en-gb.postinst
#!/bin/sh

if [ "x$D" != "x" ]; then
         exit 1
fi

rm -rf /tmp/locale/usr/lib/locale
mkdir -p /tmp/locale/usr/lib/locale
if [ -f /usr/lib/locale/locale-archive ]; then
         cp /usr/lib/locale/locale-archive /tmp/locale/usr/lib/locale/
fi
localedef --inputfile=/usr/share/i18n/locales/en_GB --charmap=UTF-8 --prefix=/tmp/locale en_GB
mkdir -p /usr/lib/locale/
mv /tmp/locale/usr/lib/locale/locale-archive /usr/lib/locale/
rm -rf /tmp/locale/usr/lib/locale
------------------------------------------------------------------------------------------

I'd like to use this (from libc-package.bbclass)
# Caller should set GLIBC_INTERNAL_USE_BINARY_LOCALE to one of:
#  "compile" - Use QEMU to generate the binary locale files
#  "precompiled" - The binary locale files are pregenerated and already present
#  "ondevice" - The device will build the locale files upon first boot through the postinst

If I can figure out how to make it use "precompiled", that would be the best.  Since
the default is "ondevice", this explains _what_ it's doing.

> On 2/7/11 10:19 AM, Gary Thomas wrote:
>> On 02/07/2011 09:06 AM, Richard Purdie wrote:
>>> On Mon, 2011-02-07 at 07:44 -0700, Gary Thomas wrote:
>>>> I often run small (slow) embedded systems with only a ramdisk
>>>> based file system.  When I use Poky for this, one side effect
>>>> is that some packages need to be "configured" on bootup, which
>>>> in the case of a ramdisk based operation means every time.
>>>>
>>>> I notice that the eglibc package brings in a couple of these
>>>> which are problematic (mostly in how long they take to run)
>>>> Looking at meta/conf/distro/include/poky-eglibc.inc:
>>>>
>>>> LIBC_DEPENDENCIES = "libsegfault \
>>>>                         eglibc \
>>>>                         eglibc-dbg \
>>>>                         eglibc-dev \
>>>>                         eglibc-utils \
>>>>                         eglibc-thread-db \
>>>>                         eglibc-localedata-i18n \
>>>>                         eglibc-gconv-ibm850 \
>>>>                         eglibc-gconv-cp1252 \
>>>>                         eglibc-gconv-iso8859-1 \
>>>>                         eglibc-gconv-iso8859-15 \
>>>>                         locale-base-en-us \
>>>>                         locale-base-en-gb "
>>>>
>>>> On my OMAP-L138 target, configuring locale-base-* takes
>>>> a long time, upwards of 35 seconds each.
>>>>
>>>> Are multiple locale-base packages really necessary?
>>>> How could I best (in the Poky spirit) limit this?  In the
>>>> minimum, I'd like to only have one locale, saving at least
>>>> 35 seconds of boot time.
>>>>
>>>> Ideas?  Comments?
>>>
>>> Shouldn't the cross locale generation be generating the locales at build
>>> time meaning the locales shouldn't be generated on the device?
>>
>> I'm not sure the details, I just know that it prints these
>> messages on boot:
>>     Configuring dbus-1.
>>      Adding system startup for /etc/init.d/dbus-1.
>>     Configuring locale-base-en-gb.
>>     Configuring locale-base-en-us.
>>
>> The last two are my concern since they take 35 seconds each to run.
>> It's a deeply embedded system and does not need multi-locale support
>> at any level.
>>
>> In any case, my comment was that there doesn't seem to be any way
>> to control/configure this at initial build time.  Per the previous
>> reply, I don't see how LIMIT_BUILT_LOCALES affects this at all.
>>
>>

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: eglibc configurability

[Richard Purdie]

On Mon, 2011-02-07 at 09:54 -0700, Gary Thomas wrote:
> On 02/07/2011 09:29 AM, Mark Hatle wrote:
> > Could you find the related init script and point me to the contents.  I don't
> > see any initscripts in the eglibc integration.  The only thing I see is a switch
> > in the locale generation between on target, on host and via QEMU.  I'm wondering
> > if maybe this is being triggered?
> 
> This could be what I'm seeing.  I have ENABLE_BINARY_LOCALE_GENERATION disabled
> in my local.conf (I had troubles with QEMU-ARM on Fedora in the past and this
> was the way around it).  It looks like that may be pushing the locale compilation
> to the target.

Correct. The locales have to be generated somewhere!

You turned them off on the build machine so they run on the target. All
is functioning as it should.

FWIW, cross locale gen doesn't use qemu now and is a lot faster so
perhaps you could turn it back on ;-).

Cheers,

Richard

Re: eglibc configurability

[Mark Hatle]

On 2/7/11 10:57 AM, Richard Purdie wrote:
> On Mon, 2011-02-07 at 09:54 -0700, Gary Thomas wrote:
>> On 02/07/2011 09:29 AM, Mark Hatle wrote:
>>> Could you find the related init script and point me to the contents.  I don't
>>> see any initscripts in the eglibc integration.  The only thing I see is a switch
>>> in the locale generation between on target, on host and via QEMU.  I'm wondering
>>> if maybe this is being triggered?
>>
>> This could be what I'm seeing.  I have ENABLE_BINARY_LOCALE_GENERATION disabled
>> in my local.conf (I had troubles with QEMU-ARM on Fedora in the past and this
>> was the way around it).  It looks like that may be pushing the locale compilation
>> to the target.
> 
> Correct. The locales have to be generated somewhere!
> 
> You turned them off on the build machine so they run on the target. All
> is functioning as it should.
> 
> FWIW, cross locale gen doesn't use qemu now and is a lot faster so
> perhaps you could turn it back on ;-).

Just to be clear, I'm running builds on machines where QEMU is unhappy (and I've
disabled the QEMU check...)  I have locales being generated and they are working
fine in the testing I have done with them.

This is using eglibc and the default conf file configuration setting.

> Cheers,
> 
> Richard
> 
> 
> 

Re: eglibc configurability

[Gary Thomas]

On 02/07/2011 10:04 AM, Mark Hatle wrote:
> On 2/7/11 10:57 AM, Richard Purdie wrote:
>> On Mon, 2011-02-07 at 09:54 -0700, Gary Thomas wrote:
>>> On 02/07/2011 09:29 AM, Mark Hatle wrote:
>>>> Could you find the related init script and point me to the contents.  I don't
>>>> see any initscripts in the eglibc integration.  The only thing I see is a switch
>>>> in the locale generation between on target, on host and via QEMU.  I'm wondering
>>>> if maybe this is being triggered?
>>>
>>> This could be what I'm seeing.  I have ENABLE_BINARY_LOCALE_GENERATION disabled
>>> in my local.conf (I had troubles with QEMU-ARM on Fedora in the past and this
>>> was the way around it).  It looks like that may be pushing the locale compilation
>>> to the target.
>>
>> Correct. The locales have to be generated somewhere!
>>
>> You turned them off on the build machine so they run on the target. All
>> is functioning as it should.
>>
>> FWIW, cross locale gen doesn't use qemu now and is a lot faster so
>> perhaps you could turn it back on ;-).
>
> Just to be clear, I'm running builds on machines where QEMU is unhappy (and I've
> disabled the QEMU check...)  I have locales being generated and they are working
> fine in the testing I have done with them.
>
> This is using eglibc and the default conf file configuration setting.

I enabled ENABLE_BINARY_LOCALE_GENERATION and now the locale
generation is being done at build time - hurray :-)  Boot time
is much improved.

Now to figure out how to control the ones that are installed.

Thanks

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: eglibc configurability

[Mark Hatle]

On 2/7/11 11:57 AM, Gary Thomas wrote:
> On 02/07/2011 10:04 AM, Mark Hatle wrote:
>> On 2/7/11 10:57 AM, Richard Purdie wrote:
>>> On Mon, 2011-02-07 at 09:54 -0700, Gary Thomas wrote:
>>>> On 02/07/2011 09:29 AM, Mark Hatle wrote:
>>>>> Could you find the related init script and point me to the contents.  I don't
>>>>> see any initscripts in the eglibc integration.  The only thing I see is a switch
>>>>> in the locale generation between on target, on host and via QEMU.  I'm wondering
>>>>> if maybe this is being triggered?
>>>>
>>>> This could be what I'm seeing.  I have ENABLE_BINARY_LOCALE_GENERATION disabled
>>>> in my local.conf (I had troubles with QEMU-ARM on Fedora in the past and this
>>>> was the way around it).  It looks like that may be pushing the locale compilation
>>>> to the target.
>>>
>>> Correct. The locales have to be generated somewhere!
>>>
>>> You turned them off on the build machine so they run on the target. All
>>> is functioning as it should.
>>>
>>> FWIW, cross locale gen doesn't use qemu now and is a lot faster so
>>> perhaps you could turn it back on ;-).
>>
>> Just to be clear, I'm running builds on machines where QEMU is unhappy (and I've
>> disabled the QEMU check...)  I have locales being generated and they are working
>> fine in the testing I have done with them.
>>
>> This is using eglibc and the default conf file configuration setting.
> 
> I enabled ENABLE_BINARY_LOCALE_GENERATION and now the locale
> generation is being done at build time - hurray :-)  Boot time
> is much improved.
> 
> Now to figure out how to control the ones that are installed.

The value is "IMAGE_LINGUAS".  It defaults to "en-us en-gb".

(default is located in meta/conf/distro/poky.conf.. but you should override this
in your build/conf/local.conf)

--Mark

> Thanks
> 

Re: eglibc configurability

[Richard Purdie]

On Mon, 2011-02-07 at 10:29 -0600, Mark Hatle wrote:
> Could you find the related init script and point me to the contents.  I don't
> see any initscripts in the eglibc integration.  The only thing I see is a switch
> in the locale generation between on target, on host and via QEMU.  I'm wondering
> if maybe this is being triggered?

See the .bbclass file, that contains the postinsts script code but it
should be getting disabled.

Cheers,

Richard