[GIT PULL] Squashfs updates for 2.6.34

[GIT PULL] Squashfs updates for 2.6.34

[Phillip Lougher]

Hi Linus,

Please pull the following Squashfs updates.  They add support
for lzma compressed file systems, plus there's also a couple of
trivial code tidy patches.

Thanks

Phillip


-------

The following changes since commit 7284ce6c9f6153d1777df5f310c959724d1bd446:
   Linus Torvalds (1):
         Linux 2.6.33-rc4

are available in the git repository at:

   git://git.kernel.org/pub/scm/linux/kernel/git/pkl/squashfs-linus.git master

Phillip Lougher (11):
       Squashfs: move zlib decompression wrapper code into a separate file
       Squashfs: factor out remaining zlib dependencies into separate wrapper file
       Squashfs: add a decompressor framework
       Squashfs: add decompressor entries for lzma and lzo
       Squashfs: add support for LZMA compressed filesystems
       lzma: Make lzma available to non initramfs/initrd code
       Squashfs: select DECOMPRESS_LZMA_NEEDED when including support for lzma
       lzma: make lzma reentrant
       Squashfs: get rid of obsolete variable in struct squashfs_sb_info
       Squashfs: get rid of obsolete definition in header file
       Squashfs: update Kconfig help to include lzma support

Randy Dunlap (1):
       libs: force lzma_wrapper to be retained

  fs/squashfs/Kconfig                   |   23 +++--
  fs/squashfs/Makefile                  |    3 +-
  fs/squashfs/block.c                   |   76 +---------------
  fs/squashfs/cache.c                   |    1 -
  fs/squashfs/decompressor.c            |   72 ++++++++++++++++
  fs/squashfs/decompressor.h            |   55 ++++++++++++
  fs/squashfs/dir.c                     |    1 -
  fs/squashfs/export.c                  |    1 -
  fs/squashfs/file.c                    |    1 -
  fs/squashfs/fragment.c                |    1 -
  fs/squashfs/id.c                      |    1 -
  fs/squashfs/inode.c                   |    1 -
  fs/squashfs/lzma_wrapper.c            |  151 +++++++++++++++++++++++++++++++++
  fs/squashfs/namei.c                   |    1 -
  fs/squashfs/squashfs.h                |   11 ++-
  fs/squashfs/squashfs_fs.h             |    6 +-
  fs/squashfs/squashfs_fs_sb.h          |   40 +++++-----
  fs/squashfs/super.c                   |   49 ++++++-----
  fs/squashfs/symlink.c                 |    1 -
  fs/squashfs/zlib_wrapper.c            |  150 ++++++++++++++++++++++++++++++++
  include/linux/decompress/bunzip2_mm.h |   13 +++
  include/linux/decompress/inflate_mm.h |   13 +++
  include/linux/decompress/mm.h         |    4 -
  include/linux/decompress/unlzma_mm.h  |   20 +++++
  include/linux/decompress/unlzo_mm.h   |   13 +++
  lib/Kconfig                           |    3 +
  lib/Makefile                          |    2 +-
  lib/decompress_bunzip2.c              |    1 +
  lib/decompress_inflate.c              |    1 +
  lib/decompress_unlzma.c               |   88 ++++++++++---------
  lib/decompress_unlzo.c                |    1 +
  31 files changed, 621 insertions(+), 183 deletions(-)
  create mode 100644 fs/squashfs/decompressor.c
  create mode 100644 fs/squashfs/decompressor.h
  create mode 100644 fs/squashfs/lzma_wrapper.c
  create mode 100644 fs/squashfs/zlib_wrapper.c
  create mode 100644 include/linux/decompress/bunzip2_mm.h
  create mode 100644 include/linux/decompress/inflate_mm.h
  create mode 100644 include/linux/decompress/unlzma_mm.h
  create mode 100644 include/linux/decompress/unlzo_mm.h

Re: [GIT PULL] Squashfs updates for 2.6.34

[Linus Torvalds]

On Mon, 1 Mar 2010, Phillip Lougher wrote:

> 
> Please pull the following Squashfs updates.  They add support
> for lzma compressed file systems, plus there's also a couple of
> trivial code tidy patches.

No. I'm not pulling this. 

>  include/linux/decompress/bunzip2_mm.h |   13 +++
>  include/linux/decompress/inflate_mm.h |   13 +++
>  include/linux/decompress/mm.h         |    4 -
>  include/linux/decompress/unlzma_mm.h  |   20 +++++
>  include/linux/decompress/unlzo_mm.h   |   13 +++

What the _h*ll_ is the point of these totally illogical and ugly files? 
The name makes no sense, and the _contents_ make zero sense either.

Please. Don't add more ugly generic header files into the main include 
directory. I realize that the ugliness is partly historical (that 
"<linux/decompress/mm.h>" file sure as hell was messy to begin with), but 
this just makes that crazy situation worse.

Clean up the mess instead of making it even less understandable. Please. 
That whole "#ifdef STATIC" crap needs to go - not get spread out even 
more.

		Linus

Re: [GIT PULL] Squashfs updates for 2.6.34

[Phillip Lougher]

Linus Torvalds wrote:

> 
> Please. Don't add more ugly generic header files into the main include 
> directory. I realize that the ugliness is partly historical (that 
> "<linux/decompress/mm.h>" file sure as hell was messy to begin with), but 
> this just makes that crazy situation worse.
> 
> Clean up the mess instead of making it even less understandable. Please. 
> That whole "#ifdef STATIC" crap needs to go - not get spread out even 
> more.
> 

OK.  The decompressor code (inflate/bunzip/lzma) is a "pile
of crap" and mm.h more so.  In the past six months I've fixed a lot
of truly bad problems with that code (NULL pointers, buffer over-runs etc.).

In my defence with these patches I was trying for the best solution while
touching as little of the mess as possible.  As that's not acceptable, I'll
go away and try and fix some of the mess, before trying for another merge.

Thanks

Phillip

Re: [GIT PULL] Squashfs updates for 2.6.34

[Linus Torvalds]

On Thu, 4 Mar 2010, Phillip Lougher wrote:
> 
> In my defence with these patches I was trying for the best solution while
> touching as little of the mess as possible.  As that's not acceptable, I'll
> go away and try and fix some of the mess, before trying for another merge.

I know, I understand. The whole thing is messy as hell, I just don't want 
that mess to spread even more.

Let's ask Peter for help. He's been involved with that whole crazy init 
loader decompressor thing.

Peter, see the thread and my unhappiness on lkml (if you don't have any 
other archives, see for example http://lkml.org/lkml/2010/3/3/275).

That whole crazy "let's hide a crappy malloc implementation in a header 
file, and then make things depend on '#ifdef STATIC'" just makes me puke. 
And I'm sure Phillip has been gouging his eyes out too, while he was 
spreading it out into those _new_ crazy header file fragments for each 
decompressor.

So rather than have those crazy header file tricks, can we just add a 
kmalloc() _library_ file for the crazy bootloader crud, so that the 
bootloader could just share header files with the regular kernel, and not 
do that crazy thing?

It's insane having that malloc() implementation, and those STATIC games, 
in a header file, and the games we play with "error()" sometimes being a 
function pointer and sometimes being a function. Crazy, crazy.

So Phillip split up these things:

	#ifdef STATIC
	/* Code active when included from pre-boot environment: */
	#define INIT
	#else
	/* Compile for initramfs/initrd code only */
	#define INIT __init
	static void(*error)(char *m);
	#endif

and I refuse to see three new copies of that disgusting thing (bunzip, 
inflate, lzma). So I'd really _really_ want for the pre-boot environment 
to have that same __init declaration, and the _same_ indirect error() 
handler model, so that we don't have this kind of thing spreading.

It was ugly enough in <compress/mm.h> (which really should be nuked from 
orbit - it's the only way to be sure), but when I see it spreading, I go 
into full zombie-attack mode, and want to start up the chainsaw and run 
around naked.

		Linus

Re: [GIT PULL] Squashfs updates for 2.6.34

[H. Peter Anvin]

On 03/03/2010 07:49 PM, Linus Torvalds wrote:
> 
> So rather than have those crazy header file tricks, can we just add a 
> kmalloc() _library_ file for the crazy bootloader crud, so that the 
> bootloader could just share header files with the regular kernel, and not 
> do that crazy thing?
> 
> It's insane having that malloc() implementation, and those STATIC games, 
> in a header file, and the games we play with "error()" sometimes being a 
> function pointer and sometimes being a function. Crazy, crazy.
> 

If with library you mean a linking library, that won't work everywhere,
because on some platforms the early-kernel decompressor needs to be
compiled with differently than the kernel as a whole.  However, it would
be easy enough to have a source-level library (I don't mean an include
file, obviously) and a common include file to wrap this all up in.

> So Phillip split up these things:
> 
> 	#ifdef STATIC
> 	/* Code active when included from pre-boot environment: */
> 	#define INIT
> 	#else
> 	/* Compile for initramfs/initrd code only */
> 	#define INIT __init
> 	static void(*error)(char *m);
> 	#endif
> 
> and I refuse to see three new copies of that disgusting thing (bunzip, 
> inflate, lzma). So I'd really _really_ want for the pre-boot environment 
> to have that same __init declaration, and the _same_ indirect error() 
> handler model, so that we don't have this kind of thing spreading.

Such a common header file for the preboot environment could simply do:

#undef __init

... and be done with it.  That way we don't have to worry about mucking
with $DEITY knows how many linker scripts... it seems the easiest, and
would certainly eliminate any uglies outside the common file.

> It was ugly enough in <compress/mm.h> (which really should be nuked from 
> orbit - it's the only way to be sure), but when I see it spreading, I go 
> into full zombie-attack mode, and want to start up the chainsaw and run 
> around naked.

OK, we really really don't want that... simply because there just aren't
enough zombies to go around already.  Last I heard, the EPA was
considering classifying them as an endangered species, only to get stuck
in a bureaucratic mess if they can be classified as a "species" at all,
or if they should be classified together with bread mold, toxic waste
and Microsoft salesmen.

The only problem with this is that it'll touch nearly every
architecture, so it probably is going to have to be .35 material.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.