problem with compressing the kernel

problem with compressing the kernel

[Rolf Liu]

Hi, everyone,
The linux kernel is compressed into a zip file, which is done under linux
using the utility
named gzip. now, I am doing such work under Windows NT. I  am wondering if
there is
a utility that can do the same job , compressing the linux kernel ,which
still can be
uncompressed at running time.
Thanks a lot.

Rolf Liu
Phone: 86-10-65642023            Fax: 86-10-65668458
-----------------------------------------------------------------


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Steven Hanley]

> Hi, everyone,
> The linux kernel is compressed into a zip file, which is done under linux
> using the utility
> named gzip.

and thus a gzip file not a zip file, but this is being picky

> now, I am doing such work under Windows NT. I  am wondering if
> there is
> a utility that can do the same job , compressing the linux kernel ,which
> still can be
> uncompressed at running time.

well my initial thought is why would you need this?

the answer is no.

you seem to be confused anyway, yes the kernel is compressed for x86
machiens (it is however not compressed for other architectures, my ppc
box uses a normal uncompresased vmlinux image.

however it is not just a compressed gzip file

using the file tool a gzip file will look like

[16:06:53] 23 sjhmmj sjh ~>file
/home/asd/sources/pmac/linux-pmac-2.2.15-pre14.tar.gz
/home/asd/sources/pmac/linux-pmac-2.2.15-pre14.tar.gz: gzip compressed
data, deflated, last modified: Mon Mar 27 01:27:14 2000, os: Unix

wheras a kernel image (for ppc) will look like

[17:18:53] 1 innuendo sjh ~>file /mac/System\ Folder/Linux\
Kernels/vmlinux-2.2.15pre14
/mac/System Folder/Linux Kernels/vmlinux-2.2.15pre14: ELF 32-bit MSB
executable, PowerPC or cisco 4500, version 1, statically linked, not
stripped

which is a plain all ordinary elf binary pretty much

the compressed kernels found on x86 boxes will look like
[15:56:34] 22 sjhmmj sjh ~>file /boot/devel
/boot/devel: Linux kernel x86 boot executable bzImage, version 2.2.16
(sjh@sjhmmj) #46 SMP Thu, RO-rootFS, root_dev=0x301, Normal VGA

it may seem you can just gzip an uncompressed vmlinux and get a vmlinuz
that will boot, except if you notice the devel image just there is not a
gzip file. There is some executable code added on the front of the image
so the x86 box will run it on boot up, some of this code uncompresses
the image.

So although gzip is available on windows (see prep.ai.mit.edu or the
cygwin site for more details (www.cygnus.com will have links)) it will
not help making a kernel image.

Now of course I wonder why you would need the ability as after all a
linux box is all you need to compile and make the image, or heck a
compatible gcc on any unix can in theory compile the kernel, the utility
to compress and uncompress the kernel is in the source tree so long as
you can compile the source tree it should work fine. gzip is not listed
as a requirement in kernel compiling in the file
/usr/src/linux/Documentation/Changes for this reason, if you can compile
the kernel on a machine it can create the compressed images if needed.

And of course remember you dont need otr use compressed images on ppc
machines.

	See You
	    Steve

--
sjh@wibble.net http://wibble.net/~sjh/
Look Up In The Sky
   Is it a bird?  No
      Is it a plane?  No
         Is it a small blue banana?
YES

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Wolfgang Denk]

In message <395998E0.65DD0D9B@wibble.net> Steven Hanley wrote:
>
> you seem to be confused anyway, yes the kernel is compressed for x86
> machiens (it is however not compressed for other architectures, my ppc
> box uses a normal uncompresased vmlinux image.

Well, on the  (embedded)  PowerPC  systems  we  are  running  we  use
compressed  kernel  images  - saving lots of space in expensive FLASH
memory is reason enough.

> wheras a kernel image (for ppc) will look like
>
> [17:18:53] 1 innuendo sjh ~>file /mac/System\ Folder/Linux\
> Kernels/vmlinux-2.2.15pre14
> /mac/System Folder/Linux Kernels/vmlinux-2.2.15pre14: ELF 32-bit MSB
> executable, PowerPC or cisco 4500, version 1, statically linked, not
> stripped

This is what you see. Bat some things look different from  what  they
really are.

I see this:

->  file linux-2.2.13/arch/ppc/mbxboot/zvmlinux
linux-2.2.13/arch/ppc/mbxboot/zvmlinux: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1, statically linked, not stripped

Nevertheless, this  file  _contains_  a  compressed  (gzipped)  Linux
kernel image as a special section "image":

-> powerpc-linux-objdump -h linux-2.2.13/arch/ppc/mbxboot/zvmlinux

linux-2.2.13/arch/ppc/mbxboot/zvmlinux:     file format elf32-powerpc

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00004a90  00100000  00100000  00010000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .rodata       000006a0  00104a90  00104a90  00014a90  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .data         0000031c  00106000  00106000  00016000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  3 .bss          000031fc  00107000  00107000  00017000  2**2
                  ALLOC
  4 image         0005d3f8  00000000  00000000  00017000  2**0
                  CONTENTS, READONLY



> which is a plain all ordinary elf binary pretty much
>
> the compressed kernels found on x86 boxes will look like

There are more things in heaven and earth, Steven, / Than are dreamt
of in your philosophy. :-)

> it may seem you can just gzip an uncompressed vmlinux and get a vmlinuz
> that will boot, except if you notice the devel image just there is not a
> gzip file. There is some executable code added on the front of the image
> so the x86 box will run it on boot up, some of this code uncompresses
> the image.

Same on (embedded) PowerPC

> Now of course I wonder why you would need the ability as after all a
> linux box is all you need to compile and make the image, or heck a

There are systems which are too slow and/or too limited to compile  a
kernel (especially embedded systems).

> And of course remember you dont need otr use compressed images on ppc
> machines.

You do, in some cases.

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
Do not follow where the path may lead....go instead where there is no
path and leave a trail.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Gregorio Gervasio Jr.]

>>>>> On Wed, 28 Jun 2000 16:19:12 +1000, Steven Hanley <sjh@wibble.net> said:

s> And of course remember you dont need otr use compressed images on
s> ppc machines.

	In the days before BootX, it was sometimes useful to boot from
OF using a floppy disk containing vmlinux.coff (which is what "make
zImage" builds):

-rwxr-xr-x   1 root     root       998439 Jun 21 14:36 arch/ppc/coffboot/vmlinux.coff
-rwxr-xr-x   1 root     root      2889970 Jun 21 14:36 vmlinux

> file arch/ppc/coffboot/vmlinux.coff
arch/ppc/coffboot/vmlinux.coff: executable (RISC System/6000 V3.1) or obj module not stripped

I suppose it's still useful in case the MacOS partition gets hosed,
although I haven't tried it in a while.

Gregorio Gervasio, Jr.
gtgj@pacbell.net

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Gabriel Paubert]

On Wed, 28 Jun 2000, Steven Hanley wrote:

> you seem to be confused anyway, yes the kernel is compressed for x86
> machiens (it is however not compressed for other architectures, my ppc
> box uses a normal uncompresased vmlinux image.

Perhaps on a Mac, but _all_ the kernels I have ever built for _all_ my PPC
machines have _always_ been compressed, without a _single_ exception.
Browsing the tree you'll easily find that there the kernel image is
compressed (maybe depending on some CONFIG option) on:

- alpha
- arm
- i386
- m68k
- ppc

so please don't claim that it is x86 specific without checking the facts
first.

> Now of course I wonder why you would need the ability as after all a
> linux box is all you need to compile and make the image, or heck a
> compatible gcc on any unix can in theory compile the kernel, the utility
> to compress and uncompress the kernel is in the source tree so long as
> you can compile the source tree it should work fine. gzip is not listed
> as a requirement in kernel compiling in the file
> /usr/src/linux/Documentation/Changes for this reason, if you can compile
> the kernel on a machine it can create the compressed images if needed.

This looks more like a documentation error. Nowadays you can get your
sources with bzip2, but for very long gzip/gunzip was an implicit
prerequisite to uncompress the kernel tarball (and the patches), otherwise
you could not even have a look at the requirements. Chicken and egg...

>
> And of course remember you dont need otr use compressed images on ppc
> machines.

I do need them. Not all PPC machines are macs, it seems :-)

	Regards,
	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Steven Hanley]

in response to a few other people, yes I realise you may wish to use
compressed images on non x86 and that it is poossible. Thankyou for
further claryfying some details about the process though. (embedded
machiens hadnt crossed my mind for example)

Gabriel Paubert wrote:
> machines have _always_ been compressed, without a _single_ exception.
> Browsing the tree you'll easily find that there the kernel image is
> compressed (maybe depending on some CONFIG option) on:
>
> - alpha
> - arm
> - i386
> - m68k
> - ppc

well it is how you build them really, you probably type make zImage or
make bzImage or some such, I personally just type make vmlinux for my
ppc kernels as I dont need the compressed kernel so dont use it.

the output is dictated by the command you use to make the kernel, and
the setup in the makefile.

> so please don't claim that it is x86 specific without checking the facts
> first.

it is specific in that x86 is braindead enough that for years the option
was necessary, I am unsure if x86 still needs the compressed image
(anyone?) however as has been pointed out you may use compressed images
on other architectures. I still strongly suspect the person asking on a
ppc list how to compress a kernel for linux on an NT box was somewhat
confused at least.

> This looks more like a documentation error. Nowadays you can get your
> sources with bzip2, but for very long gzip/gunzip was an implicit
> prerequisite to uncompress the kernel tarball (and the patches), otherwise
> you could not even have a look at the requirements. Chicken and egg...

not at all, this list is in the kernel tree, it has nothing to do with
what format you get your kernel in, (rsync or cvs are common, you may
notice, as examples of retrival of both patch updates and full kernel
updates that have no need of gzip (ignoring that both can have and
commonly do have, zlib compiled in and use it during transfer) the
Changes file lists what utilities/tools are needed to actually compile
the kernel. There is no need for bzip2 or gzip in the compilation
process.

> > And of course remember you dont need otr use compressed images on ppc
> > machines.
>
> I do need them. Not all PPC machines are macs, it seems :-)

well if the hardware is embedded or similarly lacking in space as
someone suggested I can understand the need, or possible x86. Otherwise
I would be surprised if the hardware cant handle large images. There is
a difference between "need" and "common use"

>         Regards,
>         Gabriel.

	See You
	    Steve

--
sjh@wibble.net http://wibble.net/~sjh/
Look Up In The Sky
   Is it a bird?  No
      Is it a plane?  No
         Is it a small blue banana?
YES

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Tony Mantler]

At 4:05 AM -0500 6/28/2000, Gabriel Paubert wrote:
>On Wed, 28 Jun 2000, Steven Hanley wrote:
>
>> you seem to be confused anyway, yes the kernel is compressed for x86
>> machiens (it is however not compressed for other architectures, my ppc
>> box uses a normal uncompresased vmlinux image.
>
>Perhaps on a Mac, but _all_ the kernels I have ever built for _all_ my PPC
>machines have _always_ been compressed, without a _single_ exception.
>Browsing the tree you'll easily find that there the kernel image is
>compressed (maybe depending on some CONFIG option) on:
>
>- alpha
>- arm
>- i386
>- m68k
>- ppc

Just to confuse the issue a bit, I'd like to note that on Mac68k (maybe
m68k in general, I don't recall), you *can* take an uncompressed kernel
image, run it through gzip or bzip2, and boot with it. Sometimes always
needing a seperate bootloader can be a luxury. :)


Cheers - Tony :)


--
Tony Mantler       Renaissance Nerd Extraordinaire     nicoya@apia.dhs.org
Winnipeg, Manitoba, Canada                     http://nicoya.feline.pp.se/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Gabriel Paubert]

On Wed, 28 Jun 2000, Steven Hanley wrote:

> the output is dictated by the command you use to make the kernel, and
> the setup in the makefile.

Indeed.

>
> > so please don't claim that it is x86 specific without checking the facts
> > first.
>
> it is specific in that x86 is braindead enough that for years the option
> was necessary, I am unsure if x86 still needs the compressed image
> (anyone?) however as has been pointed out you may use compressed images
> on other architectures. I still strongly suspect the person asking on a
> ppc list how to compress a kernel for linux on an NT box was somewhat
> confused at least.

Try to build a ppc boot floppy without compression (of for any achitecture
if you have enough drivers).

> > This looks more like a documentation error. Nowadays you can get your
> > sources with bzip2, but for very long gzip/gunzip was an implicit
> > prerequisite to uncompress the kernel tarball (and the patches), otherwise
> > you could not even have a look at the requirements. Chicken and egg...
>
> not at all, this list is in the kernel tree, it has nothing to do with
> what format you get your kernel in, (rsync or cvs are common, you may
> notice, as examples of retrival of both patch updates and full kernel
> updates that have no need of gzip (ignoring that both can have and
> commonly do have, zlib compiled in and use it during transfer) the
> Changes file lists what utilities/tools are needed to actually compile
> the kernel. There is no need for bzip2 or gzip in the compilation
> process.

Not in the compilation, but since when do you use
cvs/rsync/bitkeeper/whatever ?

The official source tree (the mirrors of ftp.kernel.org) is still only
available as gz and more recently with bz2. So historically gzip has been
a prerequisite to look at the kernel source and build your kernel, that's
all what I claimed (did you use Linux and build your kernels from
ftp.funet.fi in 1995 ?).

For my needs I only use ftp.kernel.org kernels with my own patches since I
was severely burnt the day vger cvs disappeared, but that's clearly a
personal choice (simply that I don't trust anything else anymore).

>
> > > And of course remember you dont need otr use compressed images on ppc
> > > machines.
> >
> > I do need them. Not all PPC machines are macs, it seems :-)
>
> well if the hardware is embedded or similarly lacking in space as
> someone suggested I can understand the need, or possible x86. Otherwise
> I would be surprised if the hardware cant handle large images. There is
> a difference between "need" and "common use"

The HW/FW I have can handle fairly large images (several Mb at least),
floppies may have trouble however and I prefer to transfer smaller images
for network loads (besides the fact that the raw vmlinux will never boot
on my machines, I need to setup the HW through a first stage loader in a
way the kernel likes before giving it control).

That's common use BTW, there are probably more PPC processors running
Linux in embedded environments than in PowerMacs. Besides that, you
claimed that "you don't need to use compressed images on PPC machines"
which is definitely wrong without knowing the kind of machine and of boot
media.

	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Wolfgang Denk]

In message <3959E550.3CC61CD@wibble.net> Steven Hanley wrote:
>
> on other architectures. I still strongly suspect the person asking on a
> ppc list how to compress a kernel for linux on an NT box was somewhat
> confused at least.

No, I don't think so. He probably just wants (or has) to use NT as  a
cross  development environment for building Linux images. It's pretty
normal to do that usinf Linux boxen, but  I  guess  his  employer  is
still preferring NT installations :-(

> well if the hardware is embedded or similarly lacking in space as
> someone suggested I can understand the need, or possible x86. Otherwise
> I would be surprised if the hardware cant handle large images. There is
> a difference between "need" and "common use"

Not all things that "can" be done are practical. I _can_ compile  the
Linux  kernel  on  a  MPC860  system with 16 MB RAM running with root
filesystem over NFS. It takes well over two hours  on  that  board  -
it's  a  good  regression  test,  but it's not practical for software
development.

For some of us, embedded systems are common use.

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
"Deliver yesterday, code today, think tomorrow."

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: problem with compressing the kernel

[Steven Hanley]

> Not in the compilation, but since when do you use
> cvs/rsync/bitkeeper/whatever ?

cvs since Tridge showed it to me a few years ago and how to use it well,
thus I used to get kernel source from samba.anu.edu.au's cvs tree often
(depending on what was using said source for), as for rsync, since I
started using ppc as Paul keeps his kernels on linuxcare.com.au
available over rsync. Anyway I was just using them as current examples
where gzip and bzip2 are not required, just as you used the fact
generally kernel's are obtained in tar.{gz,bz2} format as an argument
for needing the appropriate tool. I still would say the Changes file is
not out of date or anythng (as you previously suggested) as there is no
requirement for gzip or bzip2 to build the kernel from source, this is
what the file claims to cover. However this is just being picky.

> The official source tree (the mirrors of ftp.kernel.org) is still only
> available as gz and more recently with bz2. So historically gzip has been
> a prerequisite to look at the kernel source and build your kernel, that's
> all what I claimed (did you use Linux and build your kernels from
> ftp.funet.fi in 1995 ?).

well until recently I only had x86 boxes, but admittedly the argument is
academic as you wont find a linux box on which you compile kernels (ie
not some small embedded system, etc etc) without gzip installed.


--
sjh@wibble.net http://wibble.net/~sjh/
Look Up In The Sky
   Is it a bird?  No
      Is it a plane?  No
         Is it a small blue banana?
YES

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/