Moving zlib so that others may use it

Moving zlib so that others may use it

[Corey Minyard]

I'm working on a function that uses zlib in the kernel, and I know of 
other places zlib is used (ppp_deflate, jffs2, mcore).  I would expect 
more users to come along.

I would like to propose putting zlib in the lib directory and making it 
optionally compile if it is needed.  It's pretty easy to move the files 
around and make a few small changes to the code.  However, how do I 
configure such a thing?  I could add something like:

   if [ "$CONFIG_PPP_DEFLATE" = "y" ]; then
      define_tristate CONFIG_ZLIB y
   fi
   if [ "$CONFIG_PPP_DEFLATE" = "m" ]; then
      if [ "$CONFIG_ZLIB" != "y" ]; then
         define_tristate CONFIG_ZLIB m
      fi
   fi

to every place that uses it, or I could put something like:

   if [ "$CONFIG_JFFS2_FS" = "y" \
        -o "$CONFIG_PPP_DEFLATE" = "y" ]; then
      define_tristate CONFIG_ZLIB y
   else
      if [ "$CONFIG_JFFS2_FS" = "m" \
          -o "$CONFIG_PPP_DEFLATE" = "m" ]; then
         define_tristate CONFIG_ZLIB m
      fi
   fi

at the end of the config.  I would prefer the latter, but there is not 
an "end of config" place, you would have to put it at the end of every 
config.in (about 15 architectures right now).  I propose to add a 
Config.in to the lib directory that is sourced at the end of every 
config.in for the architectures.  I'll do the work if it's willing to be 
accepted into the kernel.

-Corey

Re: Moving zlib so that others may use it

[H. Peter Anvin]

Followup to:  <3C3CD304.9070704@acm.org>
By author:    Corey Minyard <minyard@acm.org>
In newsgroup: linux.dev.kernel
>
> I'm working on a function that uses zlib in the kernel, and I know of 
> other places zlib is used (ppp_deflate, jffs2, mcore).  I would expect 
> more users to come along.
> 

CAREFUL.  First of all, don't mix up the deflate and inflate
functions, second of all, make sure you get the memory management
right.  It's not trivial to do the latter, since the default zlib
memory management is unusable for at least some users in kernelspace.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Moving zlib so that others may use it

[Keith Owens]

On Wed, 09 Jan 2002 17:32:20 -0600, 
Corey Minyard <minyard@acm.org> wrote:
>I'm working on a function that uses zlib in the kernel, and I know of 
>other places zlib is used (ppp_deflate, jffs2, mcore).  I would expect 
>more users to come along.
>
>I would like to propose putting zlib in the lib directory and making it 
>optionally compile if it is needed.  It's pretty easy to move the files 
>around and make a few small changes to the code.  However, how do I 
>configure such a thing?  I could add something like:

I am assuming that you can satisfy hpa's concerns about using a single
version of zlib for everybody.  Also note that arch/ppc/boot/lib has
its own version of zlib which is quite different to the others.  First
make sure that you can build one version of zlib that works for
everybody.

Configuring and linking zlib is a problem.  lib/lib.a is an archive
which means that the linker will only include zlib from lib.a if some
other code refers to zlib.  That breaks when all users of zlib are in
modules and nothing in vmlinux refers to zlib, the zlib code is not
linked into the kernel and modules get unresolved errors.  Linking
lib.a into the affected modules is not the solution either, the modules
would pick up all the code from lib.a instead of using the code in the
kernel with undefined results.

The best option is to build zlib.o for the kernel (not module) and
store it in lib.a.  Compile zlib.o if any consumer of zlib has been
selected and add a dummy reference to zlib code in vmlinux to ensure
that zlib is pulled from the archive if anybody needs it, even if all
the consumers are in modules.  Some of the zlib symbols will need to be
exported, I will leave that to you.

This should do the trick, completely untested.  It assumes that zlib.h
has been moved to to include/linux and zlib.c to lib.  Against 2.4.17.

Index: 17.1/lib/Makefile
--- 17.1/lib/Makefile Tue, 18 Sep 2001 13:43:44 +1000 kaos (linux-2.4/j/28_Makefile 1.1.4.3.3.2 644)
+++ 17.1(w)/lib/Makefile Thu, 10 Jan 2002 12:26:18 +1100 kaos (linux-2.4/j/28_Makefile 1.1.4.3.3.2 644)
@@ -8,12 +8,13 @@
 
 L_TARGET := lib.a
 
-export-objs := cmdline.o dec_and_lock.o rwsem-spinlock.o rwsem.o
+export-objs := cmdline.o dec_and_lock.o rwsem-spinlock.o rwsem.o zlib.o
 
 obj-y := errno.o ctype.o string.o vsprintf.o brlock.o cmdline.o bust_spinlocks.o rbtree.o
 
 obj-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o
 obj-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
+obj-$(CONFIG_ZLIB) += zlib.o
 
 ifneq ($(CONFIG_HAVE_DEC_LOCK),y) 
   obj-y += dec_and_lock.o
Index: 17.1/init/main.c
--- 17.1/init/main.c Sat, 01 Dec 2001 11:29:21 +1100 kaos (linux-2.4/k/11_main.c 1.1.5.1.1.8.1.3.1.8 644)
+++ 17.1(w)/init/main.c Thu, 10 Jan 2002 12:24:49 +1100 kaos (linux-2.4/k/11_main.c 1.1.5.1.1.8.1.3.1.8 644)
@@ -269,6 +269,11 @@ static struct dev_name_struct {
 	{ NULL, 0 }
 };
 
+#ifdef CONFIG_ZLIB
+#include <linux/zlib.h>
+static void *dummy_zlib_reference __initdata = &deflate;
+#endif
+
 kdev_t __init name_to_kdev_t(char *line)
 {
 	int base = 0;
Index: 17.1/Makefile
--- 17.1/Makefile Sat, 22 Dec 2001 12:56:52 +1100 kaos (linux-2.4/T/c/50_Makefile 1.1.2.15.1.2.2.25.2.2.1.17.1.4.1.29.1.40 644)
+++ 17.1(w)/Makefile Thu, 10 Jan 2002 12:42:09 +1100 kaos (linux-2.4/T/c/50_Makefile 1.1.2.15.1.2.2.25.2.2.1.17.1.4.1.29.1.40 644)
@@ -124,6 +124,21 @@ NETWORKS	=net/network.o
 LIBS		=$(TOPDIR)/lib/lib.a
 SUBDIRS		=kernel drivers mm fs net ipc lib
 
+# FIXME: make CONFIG_ZLIB a CML2 derived rule later and remove all references to
+# zlib from this file.  There is no clean place to handle this in CML1.  KAO
+export CONFIG_ZLIB
+ifneq ($(subst n,,$(CONFIG_JFFS2)$(CONFIG_PPP_DEFLATE)),)
+  CONFIG_ZLIB := y
+endif
+# FIXME: Use this if ppc/boot has been changed to use the common zlib.
+# ifeq ($(ARCH),ppc)
+#   CONFIG_ZLIB := y
+# endif
+# Kludge because CONFIG_ZLIB is not in include/linux/autoconf.h in CML1.
+ifeq ($(CONFIG_ZLIB),y)
+  CFLAGS += -DCONFIG_ZLIB=1
+endif
+
 DRIVERS-n :=
 DRIVERS-y :=
 DRIVERS-m :=

Re: Moving zlib so that others may use it

[Corey Minyard]

Keith Owens wrote:

>On Wed, 09 Jan 2002 17:32:20 -0600, 
>Corey Minyard <minyard@acm.org> wrote:
>
>>I'm working on a function that uses zlib in the kernel, and I know of 
>>other places zlib is used (ppp_deflate, jffs2, mcore).  I would expect 
>>more users to come along.
>>
>>I would like to propose putting zlib in the lib directory and making it 
>>optionally compile if it is needed.  It's pretty easy to move the files 
>>around and make a few small changes to the code.  However, how do I 
>>configure such a thing?  I could add something like:
>>
>
>I am assuming that you can satisfy hpa's concerns about using a single
>version of zlib for everybody.  Also note that arch/ppc/boot/lib has
>its own version of zlib which is quite different to the others.  First
>make sure that you can build one version of zlib that works for
>everybody.
>
Every version I found was practically the same (only comment differences 
and one that variable that was initialized in one and not initialized in 
the other).  I think (actually, I know, because I have tested it) 
everyone can use the same one.  I'm not talking about the ones in the 
boot directories, nor the one for inflate_fs, either, just the ones in 
the kernel proper that are named zlib.c and zlib.h.

>Configuring and linking zlib is a problem.  lib/lib.a is an archive
>which means that the linker will only include zlib from lib.a if some
>other code refers to zlib.  That breaks when all users of zlib are in
>modules and nothing in vmlinux refers to zlib, the zlib code is not
>linked into the kernel and modules get unresolved errors.  Linking
>lib.a into the affected modules is not the solution either, the modules
>would pick up all the code from lib.a instead of using the code in the
>kernel with undefined result
>
>The best option is to build zlib.o for the kernel (not module) and
>store it in lib.a.  Compile zlib.o if any consumer of zlib has been
>selected and add a dummy reference to zlib code in vmlinux to ensure
>that zlib is pulled from the archive if anybody needs it, even if all
>the consumers are in modules.  Some of the zlib symbols will need to be
>exported, I will leave that to you.
>
Why not just create zlib as a module if all the users are modules (so 
depmod and modprobe load it)?  That's what everything else does.  And 
that way, if it's already in the kernel, the module just won't get 
loaded, but if it's not the module gets loaded.  What you are suggesting 
seems rather convoluted.

I guess one other option would be to have an explicit user-set tristate 
like CONFIG_ZLIB, and if anything uses zlib, it could only be modules if 
CONFIG_ZLIB was a module, etc.

>
>
>This should do the trick, completely untested.  It assumes that zlib.h
>has been moved to to include/linux and zlib.c to lib.  Against 2.4.17.
>
I already have patches to do all this, actually, I was just trying to 
get some feedback on it and see if the kernel maintainers would be 
interested in this.  I should have said that earlier.

Thanks,

-Corey

Re: Moving zlib so that others may use it

[Keith Owens]

On Wed, 09 Jan 2002 22:23:31 -0600, 
Corey Minyard <minyard@acm.org> wrote:
>Keith Owens wrote:
>>On Wed, 09 Jan 2002 17:32:20 -0600, 
>>Corey Minyard <minyard@acm.org> wrote:
>>>I would like to propose putting zlib in the lib directory and making it 
>>>optionally compile if it is needed.
>>
>>The best option is to build zlib.o for the kernel (not module) and
>>store it in lib.a.  Compile zlib.o if any consumer of zlib has been
>>selected and add a dummy reference to zlib code in vmlinux to ensure
>>that zlib is pulled from the archive if anybody needs it, even if all
>>the consumers are in modules.  Some of the zlib symbols will need to be
>>exported, I will leave that to you.
>>
>Why not just create zlib as a module if all the users are modules (so 
>depmod and modprobe load it)?  That's what everything else does.  And 
>that way, if it's already in the kernel, the module just won't get 
>loaded, but if it's not the module gets loaded.  What you are suggesting 
>seems rather convoluted.

If zlib is a module then it cannot be part of lib/lib.a, it has to be
separate, with changes to the top level Makefile to conditionally
include lib/zlib.o.  I did that originally but the changes to
lib/Makefile and the top level Makefile were worse.  Building zlib as a
module guarantees that you cannot use it in a boot loader, forcing you
to maintain multiple versions of zlib.c.  If you are going to use one
version of zlib then you should try to handle bootloaders as well.

What is convoluted about my solution?  The derivation of CONFIG_ZLIB in
the top level Makefile is ugly but that ugliness is a side effect of
CML1.  CONFIG_ZLIB has to be derived somewhere, it is a smaller patch
to do it in Makefile than to patch 15 arch/*/config.in files.  Apart
from that, the only other niggle is the dummy reference in init/main.c.

>I guess one other option would be to have an explicit user-set tristate 
>like CONFIG_ZLIB, and if anything uses zlib, it could only be modules if 
>CONFIG_ZLIB was a module, etc.

Don't ask the user, they will not understand the problem.  CONFIG_ZLIB
is derived from other configs and possibly ARCH variables, users have
no direct control over CONFIG_ZLIB.

Re: Moving zlib so that others may use it

[Corey Minyard]

Keith Owens wrote:

>On Wed, 09 Jan 2002 22:23:31 -0600, 
>Corey Minyard <minyard@acm.org> wrote:
>
>>Keith Owens wrote:
>>
>>>On Wed, 09 Jan 2002 17:32:20 -0600, 
>>>Corey Minyard <minyard@acm.org> wrote:
>>>
>>>>I would like to propose putting zlib in the lib directory and making it 
>>>>optionally compile if it is needed.
>>>>
>>>The best option is to build zlib.o for the kernel (not module) and
>>>store it in lib.a.  Compile zlib.o if any consumer of zlib has been
>>>selected and add a dummy reference to zlib code in vmlinux to ensure
>>>that zlib is pulled from the archive if anybody needs it, even if all
>>>the consumers are in modules.  Some of the zlib symbols will need to be
>>>exported, I will leave that to you.
>>>
>>Why not just create zlib as a module if all the users are modules (so 
>>depmod and modprobe load it)?  That's what everything else does.  And 
>>that way, if it's already in the kernel, the module just won't get 
>>loaded, but if it's not the module gets loaded.  What you are suggesting 
>>seems rather convoluted.
>>
>
>If zlib is a module then it cannot be part of lib/lib.a, it has to be
>separate, with changes to the top level Makefile to conditionally
>include lib/zlib.o.  I did that originally but the changes to
>lib/Makefile and the top level Makefile were worse.  Building zlib as a
>module guarantees that you cannot use it in a boot loader, forcing you
>to maintain multiple versions of zlib.c.  If you are going to use one
>version of zlib then you should try to handle bootloaders as well.
>
Hmm.  It worked fine for me.  I made it a module, and it put it into 
kernel/lib in
/lib/modules/2.4.17 and it did not put it in lib/lib.a  I make it a 
non-module, and
it gets included in lib/lib.a. My diff was the same as yours for the 
Makefile.

I don't know about the bootloaders.  I'm not sure you can make the 
requirement
to have them compiled the same as the kernel, since they may have different
compilation requirements in the boot loader.

>What is convoluted about my solution?  The derivation of CONFIG_ZLIB in
>the top level Makefile is ugly but that ugliness is a side effect of
>CML1.  CONFIG_ZLIB has to be derived somewhere, it is a smaller patch
>to do it in Makefile than to patch 15 arch/*/config.in files.  Apart
>from that, the only other niggle is the dummy reference in init/main.c.
>
The problem is that if you come along later and compile a new module that
needs it, it won't work.  That's a fairly common thing I do, I expect 
other kernel
developers do the same.  And the dummy ref thing is a little ugly.

>>I guess one other option would be to have an explicit user-set tristate 
>>like CONFIG_ZLIB, and if anything uses zlib, it could only be modules if 
>>CONFIG_ZLIB was a module, etc.
>>
>
>Don't ask the user, they will not understand the problem.  CONFIG_ZLIB
>is derived from other configs and possibly ARCH variables, users have
>no direct control over CONFIG_ZLIB.
>
Yeah, I agree, it was just a random thought that occured to me.

Thanks,

-Corey

Re: Moving zlib so that others may use it

[Keith Owens]

On Wed, 09 Jan 2002 23:13:28 -0600, 
Corey Minyard <minyard@acm.org> wrote:
>Hmm.  It worked fine for me.  I made it a module, and it put it into 
>kernel/lib in
>/lib/modules/2.4.17 and it did not put it in lib/lib.a  I make it a 
>non-module, and
>it gets included in lib/lib.a. My diff was the same as yours for the 
>Makefile.

Worked for me this time as well.  I had a typo the first time then did
an ugly fix to a non-existent problem :(

>I don't know about the bootloaders.  I'm not sure you can make the 
>requirement
>to have them compiled the same as the kernel, since they may have different
>compilation requirements in the boot loader.

Probably true, but I don't want to rule it out completely.  In any case
it is easily catered for in lib/Makefile.

obj-$(CONFIG_JFFS2_FS) += zlib.o
obj-$(CONFIG_PPP_DEFLATE) += zlib.o
# Uncomment these if ppc bootloader can use the common zlib
# ifeq ($(ARCH),ppc)
#   obj-y += zlib.o
# endif

>The problem is that if you come along later and compile a new module that
>needs it, it won't work.  That's a fairly common thing I do, I expect 
>other kernel
>developers do the same.  And the dummy ref thing is a little ugly.

Any new module that requires zlib requires a change to the zlib
selection.  Whether you do it in the top level Makefile, in
lib/Config.in or lib/Makefile is irrelevant, zlib has to be selected
somewhere and the criteria must be updated for new modules.

Since lib/zlib.c works for both built in and modules, there is no need
to change the top level Makefile.  AFAICT the above lines in
lib/Makefile are the minimal change.

Re: Moving zlib so that others may use it

[Thomas Capricelli]

On Thursday 10 January 2002 06:40, Keith Owens wrote:
> On Wed, 09 Jan 2002 23:13:28 -0600,
>
> Corey Minyard <minyard@acm.org> wrote:
> >Hmm.  It worked fine for me.  I made it a module, and it put it into
> >kernel/lib in
> >/lib/modules/2.4.17 and it did not put it in lib/lib.a  I make it a
> >non-module, and
> >it gets included in lib/lib.a. My diff was the same as yours for the
> >Makefile.
>
> Worked for me this time as well.  I had a typo the first time then did
> an ugly fix to a non-existent problem :(


	Could you give a patch for all of this ? either Keith or Corey ?


Thomas

Re: Moving zlib so that others may use it

[Corey Minyard]

I'll do it, since I already have most of it, unless Keith already has or 
wants to.

-Corey

Thomas Capricelli wrote:

>On Thursday 10 January 2002 06:40, Keith Owens wrote:
>
>>On Wed, 09 Jan 2002 23:13:28 -0600,
>>
>>Corey Minyard <minyard@acm.org> wrote:
>>
>>>Hmm.  It worked fine for me.  I made it a module, and it put it into
>>>kernel/lib in
>>>/lib/modules/2.4.17 and it did not put it in lib/lib.a  I make it a
>>>non-module, and
>>>it gets included in lib/lib.a. My diff was the same as yours for the
>>>Makefile.
>>>
>>Worked for me this time as well.  I had a typo the first time then did
>>an ugly fix to a non-existent problem :(
>>
>
>
>	Could you give a patch for all of this ? either Keith or Corey ?
>
>
>Thomas
>

Re: Moving zlib so that others may use it

[Corey Minyard]

Ok, the patch is done.  It's large, so get it at: 
http://home.attbi.com/~minyard/linux-genzlib.gz.

-Corey

Thomas Capricelli wrote:

>
>	Could you give a patch for all of this ? either Keith or Corey ?
>

Re: Moving zlib so that others may use it

[Zygo Blaxell]

In article <3C3D22F8.1080201@acm.org>, Corey Minyard  <minyard@acm.org> wrote:
>Keith Owens wrote:
>
>>On Wed, 09 Jan 2002 22:23:31 -0600, 
>>Corey Minyard <minyard@acm.org> wrote:
>>>Keith Owens wrote:
[...]
>Building zlib as a
>>module guarantees that you cannot use it in a boot loader, forcing you
>>to maintain multiple versions of zlib.c.  If you are going to use one
>>version of zlib then you should try to handle bootloaders as well.
[...]
>I don't know about the bootloaders.  I'm not sure you can make the 
>requirement
>to have them compiled the same as the kernel, since they may have different
>compilation requirements in the boot loader.

Ummm, you can't use an in-kernel anything in a bootloader.  How do you
uncompress an in-kernel zlib.o without an out-of-kernel zlib.o lying
around somewhere?

The closest thing to a zlib.o shared between bootloader and kernel would
be to build one zlib.o and then perhaps copy the compiled binary from the
kernel to the bootloader (thus having only one zlib.c but two zlib.o) or
link it from the bootloader to the kernel once the kernel is uncompressed.

-- 
Zygo Blaxell (Laptop) <zblaxell@feedme.hungrycats.org>
GPG = D13D 6651 F446 9787 600B AD1E CCF3 6F93 2823 44AD

Re: Moving zlib so that others may use it

[Tom Rini]

On Thu, Jan 10, 2002 at 04:37:00AM -0500, Zygo Blaxell wrote:
> In article <3C3D22F8.1080201@acm.org>, Corey Minyard  <minyard@acm.org> wrote:
> >Keith Owens wrote:
> >
> >>On Wed, 09 Jan 2002 22:23:31 -0600, 
> >>Corey Minyard <minyard@acm.org> wrote:
> >>>Keith Owens wrote:
> [...]
> >Building zlib as a
> >>module guarantees that you cannot use it in a boot loader, forcing you
> >>to maintain multiple versions of zlib.c.  If you are going to use one
> >>version of zlib then you should try to handle bootloaders as well.
> [...]
> >I don't know about the bootloaders.  I'm not sure you can make the 
> >requirement
> >to have them compiled the same as the kernel, since they may have different
> >compilation requirements in the boot loader.
> 
> Ummm, you can't use an in-kernel anything in a bootloader.  How do you
> uncompress an in-kernel zlib.o without an out-of-kernel zlib.o lying
> around somewhere?

Well, when your 'bootloader' is in arch/$(ARCH)/boot, you can rather
easily.  I think there's a few archs which grab lib/lib.a right now.

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

Re: Moving zlib so that others may use it

[Tom Rini]

On Thu, Jan 10, 2002 at 03:44:47PM +1100, Keith Owens wrote:
> On Wed, 09 Jan 2002 22:23:31 -0600, 
> Corey Minyard <minyard@acm.org> wrote:
> >Keith Owens wrote:
> >>On Wed, 09 Jan 2002 17:32:20 -0600, 
> >>Corey Minyard <minyard@acm.org> wrote:
> >>>I would like to propose putting zlib in the lib directory and making it 
> >>>optionally compile if it is needed.
> >>
> >>The best option is to build zlib.o for the kernel (not module) and
> >>store it in lib.a.  Compile zlib.o if any consumer of zlib has been
> >>selected and add a dummy reference to zlib code in vmlinux to ensure
> >>that zlib is pulled from the archive if anybody needs it, even if all
> >>the consumers are in modules.  Some of the zlib symbols will need to be
> >>exported, I will leave that to you.
> >>
> >Why not just create zlib as a module if all the users are modules (so 
> >depmod and modprobe load it)?  That's what everything else does.  And 
> >that way, if it's already in the kernel, the module just won't get 
> >loaded, but if it's not the module gets loaded.  What you are suggesting 
> >seems rather convoluted.
> 
> If zlib is a module then it cannot be part of lib/lib.a, it has to be
> separate, with changes to the top level Makefile to conditionally
> include lib/zlib.o.  I did that originally but the changes to
> lib/Makefile and the top level Makefile were worse.  Building zlib as a
> module guarantees that you cannot use it in a boot loader, forcing you
> to maintain multiple versions of zlib.c.  If you are going to use one
> version of zlib then you should try to handle bootloaders as well.


It's possible they can share, but the bootloaders (PPC & MIPS) need a
slight change to the zlib.c code to to allow using zero as a real
address to store the uncompressed data.  So we'd want to guard the
changes with __BOOTER__ or so, and then cp the file or do
#define __BOOTER__
#include "zlib.c"

And do -I$(TOPDIR)/lib, or something along those lines.

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

Re: Moving zlib so that others may use it

[Corey Minyard]

Tom Rini wrote:

>
>It's possible they can share, but the bootloaders (PPC & MIPS) need a
>slight change to the zlib.c code to to allow using zero as a real
>address to store the uncompressed data.  So we'd want to guard the
>changes with __BOOTER__ or so, and then cp the file or do
>#define __BOOTER__
>#include "zlib.c"
>
>And do -I$(TOPDIR)/lib, or something along those lines.
>

I agree, but I think it would be better to do this one step at a time. 
 Let's get the things in the kernel working first to get rid of the 
namespace crash, then get the bootloaders to share the code.  I can't 
test out a lot of the code, so I can't really do the bootloader changes, 
and the bootloader changes are completely independent, anyway, once zlib 
is moved.

-Corey

Re: Moving zlib so that others may use it

[Tom Rini]

On Thu, Jan 10, 2002 at 10:20:47AM -0600, Corey Minyard wrote:
> Tom Rini wrote:
> 
> >
> >It's possible they can share, but the bootloaders (PPC & MIPS) need a
> >slight change to the zlib.c code to to allow using zero as a real
> >address to store the uncompressed data.  So we'd want to guard the
> >changes with __BOOTER__ or so, and then cp the file or do
> >#define __BOOTER__
> >#include "zlib.c"
> >
> >And do -I$(TOPDIR)/lib, or something along those lines.
>
> I agree, but I think it would be better to do this one step at a time. 

So do I.  So it can work, but lets get this in and I'll make the PPC
stuff use it (since I've got lots of other updates for it I'm getting
ready to push out).

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

Re: Moving zlib so that others may use it

[Corey Minyard]

H. Peter Anvin wrote:

>>
>> I'm working on a function that uses zlib in the kernel, and I know of 
>> other places zlib is used (ppp_deflate, jffs2, mcore).  I would expect 
>> more users to come along.
>> 
>
>CAREFUL.  First of all, don't mix up the deflate and inflate
>functions, second of all, make sure you get the memory management
>right.  It's not trivial to do the latter, since the default zlib
>memory management is unusable for at least some users in kernelspace.

I'm not sure I follow you here.  Do you want to completely separate the
inflate and deflate stuff (so if something only needs one, it only has
to include one)?  I'm not sure of the value, and it would be kind of a
pain for maintenance (since zlib is from an external source).

As far as memory management, all the versions I am talking about are
almost exactly the same, so that shouldn't be a problem.

-Corey

Re: Moving zlib so that others may use it

[H. Peter Anvin]

Corey Minyard wrote:

> 
> I'm not sure I follow you here.  Do you want to completely separate the
> inflate and deflate stuff (so if something only needs one, it only has
> to include one)?  I'm not sure of the value, and it would be kind of a
> pain for maintenance (since zlib is from an external source).
> 
> As far as memory management, all the versions I am talking about are
> almost exactly the same, so that shouldn't be a problem.
> 


It is external, but it's quite well separated anyway.

Look at the inflate_fs one (fs/inflate_fs); it has both the memory 
management dealt with and decompression factored out...

	-hpa

Re: Moving zlib so that others may use it

[David Woodhouse]

kaos@ocs.com.au said:
>  I am assuming that you can satisfy hpa's concerns about using a
> single version of zlib for everybody.  Also note that arch/ppc/boot/
> lib has its own version of zlib which is quite different to the
> others.  First make sure that you can build one version of zlib that
> works for everybody. 

I can confirm that the JFFS2 and PPP zlib are compatible - differing in 
cosmetics only. Moving it to lib/zlib would be a good thing.

We can verify compatibility for other zlib users as an when those other
users are converted to use lib/zlib instead of their own private copy.


> The best option is to build zlib.o for the kernel (not module) and
> store it in lib.a.  Compile zlib.o if any consumer of zlib has been
> selected and add a dummy reference to zlib code in vmlinux to ensure
> that zlib is pulled from the archive if anybody needs it, even if all
> the consumers are in modules. 

AUIU you've since decided this isn't necessary - which is good. Making the
static kernel image differ according to which modules happened to be
compiled at the time is not a good thing. Sometimes we do it, but we should
avoid it when we can.

If zlib.o is used in modules only, compile it as a module. Don't put it 
into the kernel.

--
dwmw2