New compression method for JFFS2

New compression method for JFFS2

[Ferenc Havasi]

Dear All,

We would like to introduce our new work concerning JFFS2 compression.

New feautres:
- it uses the kernel's CryptoAPI interface, and will allow not only
compressors but also crypto algorithms
- advanced compressor configuration which allows to compress/encode a
block with more algorithm sequencially.
- certainly full backward compatibility for old JFFS2 images

The new compressor configuration is a regular expression like string to
specify the mode for JFFS2 how to compress blocks:
- "c": means to use algorithm c
- "[c1,c2,c3,...]": means to try algotithm c1, c2, c3,... and use the
first one which was successful.
- "{c1,c2,c3,...}": means to try all the algorithm c1, c2, c3, ... and
use the one which has the best compression ratio for that block.
- "x,y,..": means to apply algorithm x, and after apply algorithm y on
the result of x, and so on. Here x and y can be not only an algorithm
but also a subexpression.

At recent JFFS2 there is a compr field in the struct jffs2_raw_inode to
identify the compression method used in that node. In most cases it is
JFFS2_COMPR_ZLIB which means ZLIB.

Using this new method the compression/encoding method is stored not in
that compr field (that will be JFFS2_COMPR_ENGINE) but in the begining
of the compressed/encoded data. It allows to use not only one algorithm
for one block. At decompressing/deconding there will be a loop which
will terminated if the begining of the result is 0 and not a valid
algorithm name.

About user-space tools:
- there is an old mkfs.jffs2 which can be compiled and used as before
- there is a new mkfs.jffs2 which needs CryptoDev kernel patch which
allows to use the kernel's CryptoAPI from user space, too (patch is also
provided)

More info and patches are on our website at
http://www.inf.u-szeged.hu/jffs2/compression.php

Any comments are welcome.

Ferenc

Re: New compression method for JFFS2

[Holger Schurig]

> - it uses the kernel's CryptoAPI interface, and will allow not only
> compressors but also crypto algorithms

Does this make the zImage bigger or smaller as compared to the current JFFS2 ?

Re: New compression method for JFFS2

[Ferenc Havasi]

Holger Schurig wrote:

>>- it uses the kernel's CryptoAPI interface, and will allow not only
>>compressors but also crypto algorithms
>>    
>>
>
>Does this make the zImage bigger or smaller as compared to the current JFFS2 ?
>  
>
These is no relevant change in this point of view.

It is an other kind of (more flexible) compression framework than the
recent JFFS2 uses. It uses ZLIB via CryptoAPI which can cause smaller
JFFS2 image, but the kernel size is almost the same as before.

Ferenc

Re: New compression method for JFFS2

[Tehn Yit Chin]

> New feautres:
> - it uses the kernel's CryptoAPI interface, and will allow not only
> compressors but also crypto algorithms
> - advanced compressor configuration which allows to compress/encode a
> block with more algorithm sequencially.
> - certainly full backward compatibility for old JFFS2 images
> 

With the crypto stuff being used, is there much of a performance hit?

Tehn Yit Chin
Software Engineer, Grey Innovation Pty. Ltd.

Re: New compression method for JFFS2

[Holger Schurig]

> With the crypto stuff being used, is there much of a performance hit?

You don't have to do crypto, you can. If you don't use it, it won't hurt 
performance.

Kernel Crypto API is not just crypto (like aes.c, des.c, ...), but also about 
compression (compress.c, deflate.c, ...) and HMACs (md5.c, sha1.c, ...).