[patch 0/3] [RFC] zlib crypto module

[patch 0/3] [RFC] zlib crypto module

[Geert Uytterhoeven]

These patches add a (de)compression module for the "zlib" format using the
crypto API:
  [1] crypto: Add a zlib crypto module
  [2] tcrypt: Add a self test for the zlib crypto module
  [3] squashfs: Switch from zlib/inflate to "zlib" crypto module

The last patch is a proof-of-concept to make SquashFS 3.4 use this new zlib
crypto module. This makes it easier to e.g. change the decompression algorithm
in SquashFS or to make use of a hardware-accelerated zlib crypto module.

It can be extended to other compressed file systems, like e.g. AxFS and CRAMFS.

All comments are welcome. Thanks!

With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

[patch 1/3] crypto: Add a zlib crypto module

[Geert Uytterhoeven]

[-- Attachment #1: crypto/add-zlib-crypto-module.diff --]
[-- Type: TEXT/PLAIN, Size: 8561 bytes --]

From: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>

Add a (de)compression module for the "zlib" format using the crypto API.

While both the "zlib" and "deflate" crypto modules are implemented on top of
the zlib library, they differ in the following aspects:
  - The "deflate" crypto module (used by IPSec and UBIFS) does not support
    partial decompression, i.e. all compressed data has to be passed at once.
    The "zlib" crypto module does support partial decompression;
    zlib_decompress() will return -EAGAIN if not all compressed data has been
    passed.
  - The deflate crypto module uses the raw deflate data format (zlib is
    initialized with a windowBits parameter of -DEFLATE_DEF_WINBITS = -11),
    while e.g. squashfs and axfs use the zlib data format, with the default
    windowBits parameter DEF_WBITS = 15.
    Both parameters are incompatible with each other due to the different data
    formats, as indicated by the sign of the windowbits parameter.
    The absolute value of this parameter is the base two logarithm of the
    maximum window size, and larger values are backwards compatible with
    smaller values (as far as decompression is concerned).

TODO:
  - As the crypto wrapper around the zlib library supports both compression and
    decompression (and always allocates temporary space for both during
    initialization; lazy allocation was removed in 2004 because IPComp calls
    the crypto routines in non-process context), more memory is needed:
      * decompression (inflate) needs only ca. 41 KiB
      * compression (deflate) needs ca. 262 KiB! 
    It would be nice to allow having support for decompression only.

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
---
 crypto/Kconfig  |    8 ++
 crypto/Makefile |    1 
 crypto/zlib.c   |  216 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 225 insertions(+)

--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -666,6 +666,14 @@ config CRYPTO_LZO
 	help
 	  This is the LZO algorithm.
 
+config CRYPTO_ZLIB
+	tristate "Zlib compression algorithm"
+	select CRYPTO_ALGAPI
+	select ZLIB_INFLATE
+	select ZLIB_DEFLATE
+	help
+	  This is the Zlib algorithm.
+
 source "drivers/crypto/Kconfig"
 
 endif	# if CRYPTO
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -69,6 +69,7 @@ obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += mich
 obj-$(CONFIG_CRYPTO_CRC32C) += crc32c.o
 obj-$(CONFIG_CRYPTO_AUTHENC) += authenc.o
 obj-$(CONFIG_CRYPTO_LZO) += lzo.o
+obj-$(CONFIG_CRYPTO_ZLIB) += zlib.o
 
 obj-$(CONFIG_CRYPTO_TEST) += tcrypt.o
 
--- /dev/null
+++ b/crypto/zlib.c
@@ -0,0 +1,216 @@
+/*
+ * Cryptographic API.
+ *
+ * "zlib" crypto module, based on the "deflate" crypto module
+ *
+ * Copyright (c) 2003 James Morris <jmorris@intercode.com.au>
+ * Copyright 2008 Sony Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ * FIXME: deflate transforms will require up to a total of about 436k of kernel
+ * memory on i386 (390k for compression, the rest for decompression), as the
+ * current zlib kernel code uses a worst case pre-allocation system by default.
+ * This needs to be fixed so that the amount of memory required is properly
+ * related to the  winbits and memlevel parameters.
+ */
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/crypto.h>
+#include <linux/zlib.h>
+#include <linux/vmalloc.h>
+#include <linux/interrupt.h>
+#include <linux/mm.h>
+#include <linux/net.h>
+#include <linux/slab.h>
+
+#define ZLIB_DEF_LEVEL		Z_DEFAULT_COMPRESSION
+
+struct zlib_ctx {
+	struct z_stream_s comp_stream;
+	struct z_stream_s decomp_stream;
+	bool decomp_needs_reset;
+};
+
+static int zlib_comp_init(struct zlib_ctx *ctx)
+{
+	int ret = 0;
+	struct z_stream_s *stream = &ctx->comp_stream;
+
+	stream->workspace = vmalloc(zlib_deflate_workspacesize());
+	if (!stream->workspace ) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	memset(stream->workspace, 0, zlib_deflate_workspacesize());
+	ret = zlib_deflateInit(stream, ZLIB_DEF_LEVEL);
+	if (ret != Z_OK) {
+		ret = -EINVAL;
+		goto out_free;
+	}
+out:
+	return ret;
+out_free:
+	vfree(stream->workspace);
+	goto out;
+}
+
+static int zlib_decomp_init(struct zlib_ctx *ctx)
+{
+	int ret = 0;
+	struct z_stream_s *stream = &ctx->decomp_stream;
+
+	stream->workspace = kzalloc(zlib_inflate_workspacesize(), GFP_KERNEL);
+	if (!stream->workspace ) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	ret = zlib_inflateInit(stream);
+	if (ret != Z_OK) {
+		ret = -EINVAL;
+		goto out_free;
+	}
+	ctx->decomp_needs_reset = true;
+out:
+	return ret;
+out_free:
+	kfree(stream->workspace);
+	goto out;
+}
+
+static void zlib_comp_exit(struct zlib_ctx *ctx)
+{
+	zlib_deflateEnd(&ctx->comp_stream);
+	vfree(ctx->comp_stream.workspace);
+}
+
+static void zlib_decomp_exit(struct zlib_ctx *ctx)
+{
+	zlib_inflateEnd(&ctx->decomp_stream);
+	kfree(ctx->decomp_stream.workspace);
+}
+
+static int zlib_init(struct crypto_tfm *tfm)
+{
+	struct zlib_ctx *ctx = crypto_tfm_ctx(tfm);
+	int ret;
+
+	ret = zlib_comp_init(ctx);
+	if (ret)
+		goto out;
+	ret = zlib_decomp_init(ctx);
+	if (ret)
+		zlib_comp_exit(ctx);
+out:
+	return ret;
+}
+
+static void zlib_exit(struct crypto_tfm *tfm)
+{
+	struct zlib_ctx *ctx = crypto_tfm_ctx(tfm);
+
+	zlib_comp_exit(ctx);
+	zlib_decomp_exit(ctx);
+}
+
+static int zlib_compress(struct crypto_tfm *tfm, const u8 *src,
+			 unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+	int ret = 0;
+	struct zlib_ctx *dctx = crypto_tfm_ctx(tfm);
+	struct z_stream_s *stream = &dctx->comp_stream;
+
+	ret = zlib_deflateReset(stream);
+	if (ret != Z_OK) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	stream->next_in = (u8 *)src;
+	stream->avail_in = slen;
+	stream->next_out = (u8 *)dst;
+	stream->avail_out = *dlen;
+
+	ret = zlib_deflate(stream, Z_FINISH);
+	if (ret != Z_STREAM_END) {
+		ret = -EINVAL;
+		goto out;
+	}
+	ret = 0;
+	*dlen = stream->total_out;
+out:
+	return ret;
+}
+
+static int zlib_decompress(struct crypto_tfm *tfm, const u8 *src,
+			   unsigned int slen, u8 *dst, unsigned int *dlen)
+{
+	int ret = 0;
+	struct zlib_ctx *dctx = crypto_tfm_ctx(tfm);
+	struct z_stream_s *stream = &dctx->decomp_stream;
+	unsigned long old_total_out;
+
+	if (dctx->decomp_needs_reset) {
+		ret = zlib_inflateReset(stream);
+		if (ret != Z_OK)
+			return -EINVAL;
+		dctx->decomp_needs_reset = false;
+	}
+
+	stream->next_in = (u8 *)src;
+	stream->avail_in = slen;
+	stream->next_out = (u8 *)dst;
+	stream->avail_out = *dlen;
+	old_total_out = stream->total_out;
+
+	ret = zlib_inflate(stream, Z_SYNC_FLUSH);
+	switch (ret) {
+	case Z_STREAM_END:
+		dctx->decomp_needs_reset = true;
+		ret = 0;
+		break;
+
+	case Z_OK:
+		ret = -EAGAIN;
+		break;
+
+	default:
+		return -EINVAL;
+	}
+
+	*dlen = stream->total_out - old_total_out;
+	return ret;
+}
+
+static struct crypto_alg alg = {
+	.cra_name	= "zlib",
+	.cra_flags	= CRYPTO_ALG_TYPE_COMPRESS,
+	.cra_ctxsize	= sizeof(struct zlib_ctx),
+	.cra_module	= THIS_MODULE,
+	.cra_list	= LIST_HEAD_INIT(alg.cra_list),
+	.cra_init	= zlib_init,
+	.cra_exit	= zlib_exit,
+	.cra_u		= { .compress = {
+	.coa_compress	= zlib_compress,
+	.coa_decompress	= zlib_decompress } }
+};
+
+static int __init zlib_mod_init(void)
+{
+	return crypto_register_alg(&alg);
+}
+
+static void __exit zlib_mod_fini(void)
+{
+	crypto_unregister_alg(&alg);
+}
+
+module_init(zlib_mod_init);
+module_exit(zlib_mod_fini);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Zlib Compression Algorithm");
+

-- 
With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[patch 2/3] tcrypt: Add a self test for the zlib crypto module

[Geert Uytterhoeven]

[-- Attachment #1: crypto/add-zlib-crypto-test.diff --]
[-- Type: TEXT/PLAIN, Size: 4872 bytes --]

From: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
---
 crypto/tcrypt.c |    9 ++++++
 crypto/tcrypt.h |   81 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 90 insertions(+)

--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -1400,6 +1400,9 @@ static void do_test(void)
 			  DEFLATE_DECOMP_TEST_VECTORS);
 		test_comp("lzo", lzo_comp_tv_template, lzo_decomp_tv_template,
 			  LZO_COMP_TEST_VECTORS, LZO_DECOMP_TEST_VECTORS);
+		test_comp("zlib", zlib_comp_tv_template,
+			  zlib_decomp_tv_template, ZLIB_COMP_TEST_VECTORS,
+			  ZLIB_DECOMP_TEST_VECTORS);
 		test_hash("crc32c", crc32c_tv_template, CRC32C_TEST_VECTORS);
 		test_hash("hmac(md5)", hmac_md5_tv_template,
 			  HMAC_MD5_TEST_VECTORS);
@@ -1701,6 +1704,12 @@ static void do_test(void)
 		test_hash("rmd320", rmd320_tv_template, RMD320_TEST_VECTORS);
 		break;
 
+	case 43:
+		test_comp("zlib", zlib_comp_tv_template,
+			  zlib_decomp_tv_template, ZLIB_COMP_TEST_VECTORS,
+			  ZLIB_DECOMP_TEST_VECTORS);
+		break;
+
 	case 100:
 		test_hash("hmac(md5)", hmac_md5_tv_template,
 			  HMAC_MD5_TEST_VECTORS);
--- a/crypto/tcrypt.h
+++ b/crypto/tcrypt.h
@@ -8512,6 +8512,87 @@ static struct comp_testvec lzo_decomp_tv
 };
 
 /*
+ * Zlib test vectors (null-terminated strings).
+ * Params: winbits=DEF_WBITS, Z_DEFAULT_COMPRESSION, MAX_MEM_LEVEL.
+ */
+#define ZLIB_COMP_TEST_VECTORS 2
+#define ZLIB_DECOMP_TEST_VECTORS 2
+
+static struct comp_testvec zlib_comp_tv_template[] = {
+	{
+		.inlen	= 70,
+		.outlen	= 44,
+		.input	= "Join us now and share the software "
+			"Join us now and share the software ",
+		.output	= "\x78\x9c\xf3\xca\xcf\xcc\x53\x28"
+			  "\x2d\x56\xc8\xcb\x2f\x57\x48\xcc"
+			  "\x4b\x51\x28\xce\x48\x2c\x4a\x55"
+			  "\x28\xc9\x48\x55\x28\xce\x4f\x2b"
+			  "\x29\x07\x71\xbc\x08\x2b\x01\x00"
+			  "\x7c\x65\x19\x3d",
+	}, {
+		.inlen	= 191,
+		.outlen	= 128,
+		.input	= "This document describes a compression method based on the DEFLATE"
+			"compression algorithm.  This document defines the application of "
+			"the DEFLATE algorithm to the IP Payload Compression Protocol.",
+		.output	= "\x78\x9c\x5d\x8d\x31\x0e\xc2\x30"
+			  "\x10\x04\xbf\xb2\x2f\xc8\x1f\x10"
+			  "\x04\x09\x89\xc2\x85\x3f\x70\xb1"
+			  "\x2f\xf8\x24\xdb\x67\xd9\x47\xc1"
+			  "\xef\x49\x68\x12\x51\xae\x76\x67"
+			  "\xd6\x27\x19\x88\x1a\xde\x85\xab"
+			  "\x21\xf2\x08\x5d\x16\x1e\x20\x04"
+			  "\x2d\xad\xf3\x18\xa2\x15\x85\x2d"
+			  "\x69\xc4\x42\x83\x23\xb6\x6c\x89"
+			  "\x71\x9b\xef\xcf\x8b\x9f\xcf\x33"
+			  "\xca\x2f\xed\x62\xa9\x4c\x80\xff"
+			  "\x13\xaf\x52\x37\xed\x0e\x52\x6b"
+			  "\x59\x02\xd9\x4e\xe8\x7a\x76\x1d"
+			  "\x02\x98\xfe\x8a\x87\x83\xa3\x4f"
+			  "\x56\x8a\xb8\x9e\x8e\x5c\x57\xd3"
+			  "\xa0\x79\xfa\x02\x2e\x32\x45\x4e",
+	},
+};
+
+static struct comp_testvec zlib_decomp_tv_template[] = {
+	{
+		.inlen	= 128,
+		.outlen	= 191,
+		.input	= "\x78\x9c\x5d\x8d\x31\x0e\xc2\x30"
+			  "\x10\x04\xbf\xb2\x2f\xc8\x1f\x10"
+			  "\x04\x09\x89\xc2\x85\x3f\x70\xb1"
+			  "\x2f\xf8\x24\xdb\x67\xd9\x47\xc1"
+			  "\xef\x49\x68\x12\x51\xae\x76\x67"
+			  "\xd6\x27\x19\x88\x1a\xde\x85\xab"
+			  "\x21\xf2\x08\x5d\x16\x1e\x20\x04"
+			  "\x2d\xad\xf3\x18\xa2\x15\x85\x2d"
+			  "\x69\xc4\x42\x83\x23\xb6\x6c\x89"
+			  "\x71\x9b\xef\xcf\x8b\x9f\xcf\x33"
+			  "\xca\x2f\xed\x62\xa9\x4c\x80\xff"
+			  "\x13\xaf\x52\x37\xed\x0e\x52\x6b"
+			  "\x59\x02\xd9\x4e\xe8\x7a\x76\x1d"
+			  "\x02\x98\xfe\x8a\x87\x83\xa3\x4f"
+			  "\x56\x8a\xb8\x9e\x8e\x5c\x57\xd3"
+			  "\xa0\x79\xfa\x02\x2e\x32\x45\x4e",
+		.output	= "This document describes a compression method based on the DEFLATE"
+			"compression algorithm.  This document defines the application of "
+			"the DEFLATE algorithm to the IP Payload Compression Protocol.",
+	}, {
+		.inlen	= 44,
+		.outlen	= 70,
+		.input	= "\x78\x9c\xf3\xca\xcf\xcc\x53\x28"
+			  "\x2d\x56\xc8\xcb\x2f\x57\x48\xcc"
+			  "\x4b\x51\x28\xce\x48\x2c\x4a\x55"
+			  "\x28\xc9\x48\x55\x28\xce\x4f\x2b"
+			  "\x29\x07\x71\xbc\x08\x2b\x01\x00"
+			  "\x7c\x65\x19\x3d",
+		.output	= "Join us now and share the software "
+			"Join us now and share the software ",
+	},
+};
+
+/*
  * Michael MIC test vectors from IEEE 802.11i
  */
 #define MICHAEL_MIC_TEST_VECTORS 6

-- 
With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

[patch 3/3] squashfs: Switch from zlib/inflate to "zlib" crypto module

[Geert Uytterhoeven]

[-- Attachment #1: crypto/squashfs-use-crypto-api-for-orig.diff --]
[-- Type: TEXT/PLAIN, Size: 6165 bytes --]

From: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>

Modify SquashFS 3.4 to use the "zlib" crypto module instead of making direct
calls to the zlib/inflate library

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
---
 fs/Kconfig                     |    3 +
 fs/squashfs/inode.c            |   65 +++++++++++++++++++++--------------------
 include/linux/squashfs_fs_sb.h |    3 +
 3 files changed, 38 insertions(+), 33 deletions(-)

--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -1350,7 +1350,8 @@ config CRAMFS
 
 config SQUASHFS
 	tristate "SquashFS 3.4 - Squashed file system support"
-	select ZLIB_INFLATE
+	select CRYPTO
+	select CRYPTO_ZLIB
 	help
 	  Saying Y here includes support for SquashFS 3.4 (a Compressed
 	  Read-Only File System).  Squashfs is a highly compressed read-only
--- a/fs/squashfs/inode.c
+++ b/fs/squashfs/inode.c
@@ -23,7 +23,6 @@
 
 #include <linux/squashfs_fs.h>
 #include <linux/module.h>
-#include <linux/zlib.h>
 #include <linux/fs.h>
 #include <linux/squashfs_fs_sb.h>
 #include <linux/squashfs_fs_i.h>
@@ -36,6 +35,9 @@
 
 #include "squashfs.h"
 
+#define SQUASHFS_CRYPTO_ALG	"zlib"
+
+
 static struct dentry *squashfs_fh_to_dentry(struct super_block *s,
 		struct fid *fid, int fh_len, int fh_type);
 static struct dentry *squashfs_fh_to_parent(struct super_block *s,
@@ -236,7 +238,10 @@ SQSH_EXTERN unsigned int squashfs_read_d
 	}
 
 	if (compressed) {
-		int zlib_err = 0;
+		int error = 0;
+		void *next_out;
+		int avail_out;
+		unsigned int total_out = 0, dlen;
 
 		/*
 	 	* uncompress block
@@ -244,8 +249,8 @@ SQSH_EXTERN unsigned int squashfs_read_d
 
 		mutex_lock(&msblk->read_data_mutex);
 
-		msblk->stream.next_out = buffer;
-		msblk->stream.avail_out = srclength;
+		next_out = buffer;
+		avail_out = srclength;
 
 		for (bytes = 0; k < b; k++) {
 			avail_bytes = min(c_byte - bytes, msblk->devblksize - offset);
@@ -254,16 +259,8 @@ SQSH_EXTERN unsigned int squashfs_read_d
 			if (!buffer_uptodate(bh[k]))
 				goto release_mutex;
 
-			msblk->stream.next_in = bh[k]->b_data + offset;
-			msblk->stream.avail_in = avail_bytes;
-
 			if (k == 0) {
-				zlib_err = zlib_inflateInit(&msblk->stream);
-				if (zlib_err != Z_OK) {
-					ERROR("zlib_inflateInit returned unexpected result 0x%x,"
-						" srclength %d\n", zlib_err, srclength);
-					goto release_mutex;
-				}
+				total_out = 0;
 
 				if (avail_bytes == 0) {
 					offset = 0;
@@ -272,29 +269,32 @@ SQSH_EXTERN unsigned int squashfs_read_d
 				}
 			}
 
-			zlib_err = zlib_inflate(&msblk->stream, Z_NO_FLUSH);
-			if (zlib_err != Z_OK && zlib_err != Z_STREAM_END) {
-				ERROR("zlib_inflate returned unexpected result 0x%x,"
-					" srclength %d, avail_in %d, avail_out %d\n", zlib_err,
-					srclength, msblk->stream.avail_in, msblk->stream.avail_out);
+			dlen = avail_out;
+			error = crypto_comp_decompress(msblk->tfm,
+						       bh[k]->b_data + offset,
+						       avail_bytes, next_out,
+						       &dlen);
+			if (error && error != -EAGAIN) {
+				ERROR("crypto_comp_decompress returned "
+				      "unexpected result %d, srclength %d, "
+				      "avail_bytes %d, avail_out %d\n",
+				      error, srclength, avail_bytes,
+				      avail_out);
 				goto release_mutex;
 			}
+			next_out += dlen;
+			avail_out -= dlen;
+			total_out += dlen;
 
 			bytes += avail_bytes;
 			offset = 0;
 			brelse(bh[k]);
 		}
 
-		if (zlib_err != Z_STREAM_END)
+		if (error)
 			goto release_mutex;
 
-		zlib_err = zlib_inflateEnd(&msblk->stream);
-		if (zlib_err != Z_OK) {
-			ERROR("zlib_inflateEnd returned unexpected result 0x%x,"
-				" srclength %d\n", zlib_err, srclength);
-			goto release_mutex;
-		}
-		bytes = msblk->stream.total_out;
+		bytes = total_out;
 		mutex_unlock(&msblk->read_data_mutex);
 	} else {
 		int i;
@@ -1104,9 +1104,12 @@ static int squashfs_fill_super(struct su
 	}
 	msblk = s->s_fs_info;
 
-	msblk->stream.workspace = vmalloc(zlib_inflate_workspacesize());
-	if (msblk->stream.workspace == NULL) {
-		ERROR("Failed to allocate zlib workspace\n");
+	msblk->tfm = crypto_alloc_comp(SQUASHFS_CRYPTO_ALG, 0,
+				       CRYPTO_ALG_ASYNC);
+	if (IS_ERR(msblk->tfm)) {
+		ERROR("Failed to load %s crypto module\n",
+		      SQUASHFS_CRYPTO_ALG);
+		msblk->tfm = NULL;
 		goto failure;
 	}
 	sblk = &msblk->sblk;
@@ -1273,7 +1276,7 @@ failed_mount:
 	vfree(msblk->read_page);
 	squashfs_cache_delete(msblk->block_cache);
 	kfree(msblk->fragment_index_2);
-	vfree(msblk->stream.workspace);
+	crypto_free_comp(msblk->tfm);
 	kfree(s->s_fs_info);
 	s->s_fs_info = NULL;
 	return -EINVAL;
@@ -2084,7 +2087,7 @@ static void squashfs_put_super(struct su
 		kfree(sbi->fragment_index);
 		kfree(sbi->fragment_index_2);
 		kfree(sbi->meta_index);
-		vfree(sbi->stream.workspace);
+		crypto_free_comp(sbi->tfm);
 		kfree(s->s_fs_info);
 		s->s_fs_info = NULL;
 	}
--- a/include/linux/squashfs_fs_sb.h
+++ b/include/linux/squashfs_fs_sb.h
@@ -24,6 +24,7 @@
  */
 
 #include <linux/squashfs_fs.h>
+#include <linux/crypto.h>
 
 struct squashfs_cache_entry {
 	long long	block;
@@ -67,7 +68,7 @@ struct squashfs_sb_info {
 	struct mutex		read_page_mutex;
 	struct mutex		meta_index_mutex;
 	struct meta_index	*meta_index;
-	z_stream		stream;
+	struct crypto_comp	*tfm;
 	long long		*inode_lookup_table;
 	int			(*read_inode)(struct inode *i,  squashfs_inode_t \
 				inode);

-- 
With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [patch 1/3] crypto: Add a zlib crypto module

[Herbert Xu]

On Fri, Aug 29, 2008 at 01:41:59PM +0000, Geert Uytterhoeven wrote:
> From: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
> 
> Add a (de)compression module for the "zlib" format using the crypto API.

I think we can safely conclude that our current compression
interface sucks for what you're trying to achieve :)

> While both the "zlib" and "deflate" crypto modules are implemented on top of
> the zlib library, they differ in the following aspects:
>   - The "deflate" crypto module (used by IPSec and UBIFS) does not support
>     partial decompression, i.e. all compressed data has to be passed at once.
>     The "zlib" crypto module does support partial decompression;
>     zlib_decompress() will return -EAGAIN if not all compressed data has been
>     passed.
>   - The deflate crypto module uses the raw deflate data format (zlib is
>     initialized with a windowBits parameter of -DEFLATE_DEF_WINBITS = -11),
>     while e.g. squashfs and axfs use the zlib data format, with the default
>     windowBits parameter DEF_WBITS = 15.
>     Both parameters are incompatible with each other due to the different data
>     formats, as indicated by the sign of the windowbits parameter.
>     The absolute value of this parameter is the base two logarithm of the
>     maximum window size, and larger values are backwards compatible with
>     smaller values (as far as decompression is concerned).

Therefore I suggest that we change the interface rather than
mutilate the implementations.

So here are a few things we should add:

1) Separate alloc functions for comp/decomp to avoid the memory
   wastage you've identified.

2) Provide parameters to these alloc functions through an opaque
   pointer.  The format of the paramters will be determined by the
   name of the algorithm, i.e., if you want to change the parameters
   then you should change the name as well (e.g., deflate => deflate2).

   This removes the need to dupliate the implementation just because
   you want 15 instead of -11.

3) Provide init/update/final (one set each for comp and decomp)
   functions similar to crypto_hash, in addition to the current
   comp/decomp functions.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [patch 1/3] crypto: Add a zlib crypto module

[David Woodhouse]

On Sat, 2008-08-30 at 16:23 +1000, Herbert Xu wrote:
> On Fri, Aug 29, 2008 at 01:41:59PM +0000, Geert Uytterhoeven wrote:
> > From: Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
> > 
> > Add a (de)compression module for the "zlib" format using the crypto API.
> 
> I think we can safely conclude that our current compression
> interface sucks for what you're trying to achieve :)

The main thing that's missing for JFFS2 is 
 "Compress as much of this as you can into X bytes"

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: [Squashfs-devel] [patch 0/3] [RFC] zlib crypto module

[Phillip Lougher]

Geert Uytterhoeven wrote:
> These patches add a (de)compression module for the "zlib" format using the
> crypto API:
>   [1] crypto: Add a zlib crypto module
>   [2] tcrypt: Add a self test for the zlib crypto module
>   [3] squashfs: Switch from zlib/inflate to "zlib" crypto module
> 
> The last patch is a proof-of-concept to make SquashFS 3.4 use this new zlib
> crypto module. This makes it easier to e.g. change the decompression algorithm
> in SquashFS or to make use of a hardware-accelerated zlib crypto module.
> 
> It can be extended to other compressed file systems, like e.g. AxFS and CRAMFS.
> 
> All comments are welcome. Thanks!
> 

Moving Squashfs over to the crypto API is a good idea.  When the zlib 
crypto module code is in the mainline kernel moving Squashfs, AxFS and 
CRAMFS over will be easy.

Are you planning to do the necessary work to get this (or a subsequent 
version following the comments from Herbert Xu) into mainline?

Cheers

Phillip