Reducing glibc with mklibs.sh

Reducing glibc with mklibs.sh

[Michael Habermann]

I tried to reduce the size of my self-compiled glibc 2.2.2, with the debian
script mklibs.sh, but it terminates with the following output:

---
++ echo :/usr/local/powerpc-linux/glibc-2.2.2
++ sed -e 's/::*/:/g' -e 's/^://' -e 's/:$//' -e 's/:/ -L/g'
++ cat /tmp/,mklibs.21079/lib-dependencies
++ sed 's/^/-l/'
+ powerpc-linux-gcc -nostdlib -nostartfiles -shared -Wl,-soname=libc.so
-uabort -ualarm -uatoi -uatol -ubcopy -ubindtextdomain -ubzero -uchdir
-uclearerr -uclose -uclosedir -uconfstr -u__ctype_b -u__ctype_tolower
-udcgettext -udup -udup2 -uendpwent -uenviron -u_environ -u__environ
-u__errno_location -uexecve -uexit -ufclose@GLIBC_2.0 -ufclose@@GLIBC_2.1
-ufcntl -ufdopen@GLIBC_2.0 -ufdopen@@GLIBC_2.1 -uferror -ufflush -ufileno
-ufnmatch -ufork -ufprintf -ufputs -ufree -ufwrite -u__fxstat64@GLIBC_2.1
-u__fxstat64@@GLIBC_2.2 -ugetcwd -ugetdtablesize -ugetegid -ugetenv
-ugeteuid -ugetgid -ugetgroups -ugethostname -ugetpeername -ugetpid
-ugetppid -ugetpwnam -ugetpwuid -ugetrlimit64@GLIBC_2.1
-ugetrlimit64@@GLIBC_2.2 -ugetrusage -ugetuid -uglob_pattern_p
-ugroup_member -u_init -u_IO_putc -uisalpha -uisatty -uisdigit -ukill
-u__libc_start_main -ulongjmp -ulseek64 -u__lxstat64@GLIBC_2.1
-u__lxstat64@@GLIBC_2.2 -umalloc -umemcpy -uopen64 -uopendir -upathconf
-upipe -uprintf -uputchar -uputs -uqsort -uread -ureaddir64@GLIBC_2.1
-ureaddir64@@GLIBC_2.2 -ureadlink -urealloc -usbrk -usetgid -u_setjmp
-usetlocale -usetrlimit64 -usetuid -usetvbuf -usigaction -usigaddset
-usigdelset -usigemptyset -usiginterrupt -usigprocmask -usprintf -ustderr
-ustdin -ustdout -ustrcasecmp -ustrcat -ustrchr -ustrcmp -ustrcpy
-ustrerror -ustrlen -ustrncmp -ustrncpy -ustrrchr -ustrsignal -ustrtod
-ustrtol -ustrtoul -usysconf -utcgetattr -utcsetattr -utextdomain -utime
-uttyname -uumask -uunlink -uvfprintf -uwaitpid -uwrite
-u__xstat64@GLIBC_2.1 -u__xstat64@@GLIBC_2.2 -o /tmp/,mklibs.21079/lib-so
/usr/local/powerpc-linux/glibc-2.2.2/libc_pic.a -L/ppcroot/smalllib
-L/usr/local/powerpc-linux/glibc-2.2.2
/usr/local/powerpc-linux/bin/ld: /tmp/,mklibs.21079/lib-so: undefined
versioned symbol name sys_siglist@@GLIBC_2.1
/usr/local/powerpc-linux/bin/ld: failed to set dynamic section sizes: Bad value
collect2: ld returned 1 exit status
+ echo ./mklibs.sh: install-small-lib: powerpc-linux-gcc or
powerpc-linux-objcopy failed.
./mklibs.sh: install-small-lib: powerpc-linux-gcc or powerpc-linux-objcopy
failed.
+ exit 1
+ rm -fr /tmp/,mklibs.21079
---

The program call was:
----
#!/bin/sh
GCC=powerpc-linux-gcc \
OBJDUMP=powerpc-linux-objdump \
OBJCOPY=powerpc-linux-objcopy \
./mklibs.sh --verbose -L /usr/local/powerpc-linux/glibc-2.2.2 \
--dest-dir /ppcroot/smalllib \
/ppcroot/bin/bash
---

By default, only the libc_pic.a file is created. How can I create the other
PICs?

If you know any other tools to do this task, please let me know. Seems that
LIPO (Lineo) and LOT(HardHat) are not free.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Wolfgang Denk]

In message <5.1.0.14.0.20011017155249.03548ec0@pop.gmx.de> you wrote:
>
> If you know any other tools to do this task, please let me know. Seems that
> LIPO (Lineo) and LOT(HardHat) are not free.

What makes you think so?

The MV Library Optimizer at
http://sourceforge.net/projects/libraryopt/ it's under GPL.

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
There are two ways to write error-free programs. Only the  third  one
works.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Joe Green]

Wolfgang Denk wrote:
> In message <5.1.0.14.0.20011017155249.03548ec0@pop.gmx.de> you wrote:
> >
> > If you know any other tools to do this task, please let me know. Seems that
> > LIPO (Lineo) and LOT(HardHat) are not free.
>
> What makes you think so?
>
> The MV Library Optimizer at
> http://sourceforge.net/projects/libraryopt/ it's under GPL.

Yes, but it's not well documented yet...I need to improve that and add
a mailing list.

Michael Habermann wrote:
> I tried to reduce the size of my self-compiled glibc 2.2.2, with the debian
> script mklibs.sh, but it terminates with the following output:
> ---
> /usr/local/powerpc-linux/bin/ld: /tmp/,mklibs.21079/lib-so: undefined
> versioned symbol name sys_siglist@@GLIBC_2.1

This looks like a problem with COPY relocation types, which was seen and
fixed for the LOT.

--
Joe Green <jgreen@mvista.com>
MontaVista Software, Inc.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Michael Habermann]

At 09:28 AM 10/17/2001 -0700, Joe Green wrote:
> > The MV Library Optimizer at
> > http://sourceforge.net/projects/libraryopt/ it's under GPL.
>
>Yes, but it's not well documented yet...I need to improve that and add
>a mailing list.

libopt looks great, but I don't know how to handle it.
After setting the toolprefix manually, I call the software in this way:
         ./powerpc-linux-libopt -d /ppcroot.smalllib /ppcroot

/ppcroot/ contains my RAMDisk. The program executes up to the end of main
without error messages, but /ppcroot.smallib/ is empty after execution.

Is it possible, that the script runs only with HardHat Linux?
It seems that when DynamicLibrary.generate(...) is called, self.optimizer
is always None.
The optimizers are read from an optinfo file, which I don't have. Where to
get one?

I have also no idea, what libindex is good for.


> > /usr/local/powerpc-linux/bin/ld: /tmp/,mklibs.21079/lib-so: undefined
> > versioned symbol name sys_siglist@@GLIBC_2.1
>
>This looks like a problem with COPY relocation types, which was seen and
>fixed for the LOT.

Any hint how to fix it?


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Reducing glibc with mklibs.sh

[Jean-Denis Boyer]

A couple of things are missing in your configuration for mklibs.sh
to properly work. I spent many hours finding what was the exact
configuration to have mklibs.sh properly works.
I understood when I installed Debian Linux on a PC,
and looked how the boot-floppies package was installed.
That was a lot easier than understanding the script.

First, the map files should be present along with the pic files.
Next, to properly reduce libc, mklibs.sh needs two or three
(depending on the glibc version) objects files:
    soinit.o sofini.o interp.o
AFAIK, mklibs.sh needs extra object files only for libc.

As an example, I successfully cross-reduced glibc-2.2.4 for powerpc-linux,
on my ix86 Redhat.
The files were installed in the following hierarchy:
  /usr/local/powerpc-linux/lib/libc_pic.a
  /usr/local/powerpc-linux/lib/libc_pic.map
  /usr/local/powerpc-linux/lib/libc_pic/interp.o
  /usr/local/powerpc-linux/lib/libc_pic/soinit.o
  /usr/local/powerpc-linux/lib/libc_pic/sofini.o

The map file is generated along with the pic file, when building the libc
package.
It may have a different name, like 'libc.map', but mklibs.sh looks for
'libc_pic.map'.
The object files are found under the directory elf/ (also in glibc build)
with the extension .os.

Note: I successfully reduced libpthread.so, also.

Good luck.
--------------------------------------------
 Jean-Denis Boyer, B.Eng., Technical Leader
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Michael Habermann]

Thank you all for your answers, also the private mails. I will check out
each solution and feedback to you.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Reducing glibc with mklibs.sh

[Michael Habermann]

At 09:07 AM 10/18/2001 -0400, Jean-Denis Boyer wrote:

>A couple of things are missing in your configuration for mklibs.sh
>to properly work. I spent many hours finding what was the exact
>configuration to have mklibs.sh properly works.

Thank you very much, it worked in exactly this way. I could reduce my
libc-2.2.2 from 1.1 MB to 500 kB now.

For the other people trying this here is my script I execute after
compiling and installing  glibc 2.2.2:

---
#!/bin/sh
cp libc_pic.a ../lib/
cp libc.map ../lib/libc_pic.map
mkdir ../lib/libc_pic
cp elf/interp.os ../lib/libc_pic/interp.o
cp elf/soinit.os ../lib/libc_pic/soinit.o
cp elf/sofini.os ../lib/libc_pic/sofini.o
---



>Note: I successfully reduced libpthread.so, also.


How have you done the libthread? Standard compilation for glibc produces
_pic only for libc on my system. My libm is still quite large.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Michael Habermann]

At 08:08 AM 10/18/2001 -0700, Joe Green wrote:
>You have to create them.  This is why the documentation is needed :-).
>The packages come with Hard Hat Linux, but for other distributions you
>need to create them yourself.

Thanks for the quick docu. I could create the optinfo path and it is used
by libopt. But my buildscript fails and I've no idea how to solve it.

I can use mklibs.sh now, so I wait until the docu has been finished.

>   ppc_8xx-libindex -r interp.o sofini.o soinit.o -o <other_objs> > index

I think there was a typo, the library is not specified.
Assuming that other_objs is empty in my case, I've used this command:

ppc_8xx-libindex libc-2.2.2.so -r interp.o sofini.o soinit.o > index


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Mark Hatle]

> >Note: I successfully reduced libpthread.so, also.
>
> How have you done the libthread? Standard compilation for glibc produces
> _pic only for libc on my system. My libm is still quite large.

I suggest you download the Hard Hat Linux (ftp.mvista.com) source rpm
for glibc.  I believe there is either a patch or a special "step" in the
%install section of the rpm spec file.

(It works for LOT, but is probably applicable to mklibs.sh)

--Mark

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Reducing glibc with mklibs.sh

[Jean-Denis Boyer]

> How have you done the libthread? Standard compilation for
> glibc produces
> _pic only for libc on my system. My libm is still quite large.

If the linuxthreads add-on is enabled in your glibc configuration, that
should produce pic & map files, in the following manner:
  ./linuxthreads/libpthread_pic.a
  ./libpthread.map

For libm, it is created in the same way:
  ./math/libm_pic.a
  ./libm.map

And it seems to work very weel for libm too.
No additional object files seems to be necessary, as for libc.

Regards,
--------------------------------------------
 Jean-Denis Boyer, B.Eng., Technical Leader
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Reducing glibc with mklibs.sh

[Michael Habermann]

At 10:16 AM 10/24/2001 -0400, Jean-Denis Boyer wrote:
>If the linuxthreads add-on is enabled in your glibc configuration, that
>should produce pic & map files, in the following manner:
>   ./linuxthreads/libpthread_pic.a
>   ./libpthread.map

I must apologize, they have been there. (I used accidentally find -name
*_pic.a)

Maybe you can help me another time (I'm lost when it get's to the linker).

It works when having only the libc pic file, but when adding libm_pic.a, I
get the following error:
++ echo :/usr/local/powerpc-linux/lib
++ sed -e 's/::*/:/g' -e 's/^://' -e 's/:$//' -e 's/:/ -L/g'
++ cat /tmp/,mklibs.11755/lib-dependencies
++ sed 's/^/-l/'
+ powerpc-linux-gcc -nostdlib -nostartfiles -shared
-Wl,-soname=libm-2.2.2.so -uceil -ufloor -ulog10 -upow -o
/tmp/,mklibs.11755/lib-so /usr/local/powerpc-linux/lib/libm_pic.a
-L/ppcroot/lib -L/usr/local/powerpc-linux/lib -lc
/usr/local/powerpc-linux/bin/ld: /tmp/,mklibs.11755/lib-so: undefined
versioned symbol name feraiseexcept@GLIBC_2.1
/usr/local/powerpc-linux/bin/ld: failed to set dynamic section sizes: Bad value
collect2: ld returned 1 exit status
+ echo ./mklibs.sh: install-small-lib: powerpc-linux-gcc or
powerpc-linux-objcopy failed.
./mklibs.sh: install-small-lib: powerpc-linux-gcc or powerpc-linux-objcopy
failed.
+ exit 1

That's my build script:
---
#!/bin/sh

make clean
CC=powerpc-linux-gcc \
AR=powerpc-linux-ar \
RANLIB=powerpc-linux-ranlib \
CFLAGS="-Os -DNDEBUG" \
./configure \
--enable-kernel=2.4.4 \
--disable-profile \
--host=powerpc-linux \
--enable-add-ons=linuxthreads \
--with-headers=/usr/local/powerpc-linux/include \
--prefix=

make
make install_root=/usr/local/powerpc-linux install

#Prepare the lib directory for reducing the standard library size with
#mklibs.sh
cp *.map ../lib/
cp `find -name "*_pic.a"` ../lib/
#libc requires special handling
mkdir ../lib/libc_pic
cp elf/interp.os ../lib/libc_pic/interp.o
cp elf/soinit.os ../lib/libc_pic/soinit.o
cp elf/sofini.os ../lib/libc_pic/sofini.o
---


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Reducing glibc with mklibs.sh

[Joe Green]

On Tuesday 23 October 2001 19:18, Michael Habermann wrote:
> > ppc_8xx-libindex -r interp.o sofini.o soinit.o -o <other_objs> > index
>
> I think there was a typo, the library is not specified.

D'oh.  Yes, you're right.

> Assuming that other_objs is empty in my case, I've used this command:
>
> ppc_8xx-libindex libc-2.2.2.so -r interp.o sofini.o soinit.o > index

Well, that indicates a library with no optional objects.  There's no
point in using the optimizer on such a library.

It's more common to have a library with no required objects.

--
Joe Green <jgreen@mvista.com>
MontaVista Software, Inc.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Reducing glibc with mklibs.sh

[Jean-Denis Boyer]

It looks like mklibs.sh does not find the map file for libm.

In your script:
> cp *.map ../lib/
might be bad, because the makefile of glibc produces libm.map,
while mklibs.sh looks for libm_pic.map (the name differs).

Either you rename the file when copying it, or you modify mklibs.sh.

--------------------------------------------
 Jean-Denis Boyer, B.Eng., Technical Leader
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Reducing glibc with mklibs.sh

[Michael Habermann]

At 05:36 PM 10/29/2001 -0500, Jean-Denis Boyer wrote:
>In your script:
> > cp *.map ../lib/
>might be bad, because the makefile of glibc produces libm.map,
>while mklibs.sh looks for libm_pic.map (the name differs).

Sorry for having posted this buggy script. Here's the correct one:
---
CC=powerpc-linux-gcc \
AR=powerpc-linux-ar \
RANLIB=powerpc-linux-ranlib \
CFLAGS="-Os -DNDEBUG" \
./configure \
--enable-kernel=2.4.4 \
--disable-profile \
--host=powerpc-linux \
--enable-add-ons=linuxthreads \
--with-headers=/usr/local/powerpc-linux/include \
--prefix=

make
make install_root=/usr/local/powerpc-linux install

#Prepare the lib directory for reducing the standard library size with
#mklibs.shcp
#copy pic files to ../lib/
cp `find -name "*_pic.a"` ../lib/

#rename *.map file to ../lib/*_pic.map
for i  in *.map; do
     cp $i ../lib/${i%%.map}_pic.map
done

#libc requires special handling
mkdir ../lib/libc_pic
cp elf/interp.os ../lib/libc_pic/interp.o
cp elf/soinit.os ../lib/libc_pic/soinit.o
cp elf/sofini.os ../lib/libc_pic/sofini.o

#libm doesn't work
rm -f ../lib/libc.map
rm -f ../lib/libc_pic.a
---


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Reducing glibc with mklibs.sh

[Orlov, Leonid]

[-- Attachment #1: Type: text/plain, Size: 274 bytes --]

Do you have an example of library reduction experience using mklibs.
I created a library with 3 functions and use only 2 in main.
I have hard time to make it work in this simple environment.
I must be missing something.
Thanks for your help.
Leonid Orlov
Los Angeles

[-- Attachment #2: Type: text/html, Size: 1085 bytes --]

Reducing glibc with mklibs.sh

[Orlov, Leonid]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

Hi,
Does anyone has an example of successful use of either libopt or
mklib.sh in redhat environmet?
I am having hard time to make it work.
I created a myLib library with 3 1 statement function and a main that
use only 2 of the myLib functions.
I am trying to have either tool to detect that only 2 out of 3 functions
are used and reduce myLib correspondingly. So far I have hard time to
have it work.
BTW does mklib dependent on Debian environment. I am using redhat
I am also wondering if Lineo released LIPO for public use.
I called them trying to get a commercial version, they excluded LIPO
from their offerings.
Thanks for your help.
Leonid Orlov

[-- Attachment #2: Type: text/html, Size: 1813 bytes --]