* Re: [PATCH 1/9] Squashfs: move zlib decompression wrapper code into a separate file
From: Phillip Lougher @ 2009-12-10 23:20 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: dedekind1, Andrew Morton, linux-embedded, linux-fsdevel,
linux-kernel, phillip.lougher, Tim Bird, Geert Uytterhoeven,
Felix Fietkau
In-Reply-To: <10f740e80912100201l6d83588evff46e95ec96d0aed@mail.gmail.com>
Geert Uytterhoeven wrote:
> Hi Phillip,
>
> On Thu, Dec 10, 2009 at 01:38, Phillip Lougher
> <phillip@lougher.demon.co.uk> wrote:
>> Artem Bityutskiy wrote:
>>> Did you consider using cryptoapi? UBIFS uses zlib/lzo in cryptoapi - it
>>> is a very clean way.
>
> Exactly my question, as that's why the Crypto API was extended with
> support for partial (de)compression in the first place ;-)
>
Your cryptoapi patches are on my "must do something about them" list, I've
not forgotten them :-) The lack of lzma in the crypotapi made moving
solely to the cryptoapi difficult, when I wanted to add lzma decompression
support.
The decompressor framework is my _solution_ to this problem. This allows
cryptoapi support to be added as additional decompression_wrapper, with
multiple compression types using the crptoapi wrapper as appropriate.
This has the advantage it allows Squashfs to use the cryptoapi (with
its advantages), without needlessly restricting Squashfs to the
compression algorithms supported by the cryptoapi.
Let's get this mainlined, and then we can work on getting your
cryptoapi patches integrated into that. That'll make me happy,
and hopefully you'll be happy too?
Thanks
Phillip
^ permalink raw reply
* [PATCH V2 0/9] Squashfs: Add support for LZMA compressed filesystems
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Hi,
This a respin of my LZMA decompression support patches, taking into
account Andrew Morton's comments regarding poorly named globals:
Changes:
1. zlib_uncompress() renamed to squashfs_zlib_uncompress() (patch 1)
2. zlib_init() and zlib_free() renamed to squashfs_zlib_init() and
squashfs_zlib_free() (patch 2)
3. Merged patch 7 (make decompressor init function pass superblock
info) into patch 3. This was a development commit which
should not have been a separate patch for review
Original patch series info:
The following patches add LZMA decompression support to Squashfs, using the
in-kernel LZMA decompression library.
The patches also add a decompression framework to Squashfs.
This allows LZMA decompression to be added cleanly, and it allows
additional decompressors to be easily added in the future.
To enable the in-kernel LZMA decompression code to be used by Squashfs,
there are two patches to the lzma code itself: one to make lzma available
to non-init code, and one to make lzma reentrant. These are obviously
not restricted to Squashfs, but are needed by any non-init code that
may wish to use lzma compression.
These patches are available in my squashfs-devel git tree in a slightly
different format (these patches have been refactored for posting, they'll
be put into a git tree ASAP).
http://git.kernel.org/?p=linux/kernel/git/pkl/squashfs-devel.git;a=summary
I would like to thank the CE Linux Forum (CELF) for supporting this work.
Thanks
Phillip
^ permalink raw reply
* [PATCH V2 7/8] Squashfs: select DECOMPRESS_LZMA_NEEDED when including support for lzma
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
fs/squashfs/Kconfig | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index 0294aa2..7ec5d7e 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -30,6 +30,7 @@ config SQUASHFS_LZMA
bool "Include support for LZMA compressed file systems"
depends on SQUASHFS
select DECOMPRESS_LZMA
+ select DECOMPRESS_LZMA_NEEDED
config SQUASHFS_EMBEDDED
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 3/8] Squashfs: add a decompressor framework
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
This adds a decompressor framework which allows multiple compression
algorithms to be cleanly supported.
Also update zlib wrapper and other code to use the new framework.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
fs/squashfs/Makefile | 2 +-
fs/squashfs/block.c | 3 +-
fs/squashfs/decompressor.c | 58 ++++++++++++++++++++++++++++++++++++++++++
fs/squashfs/decompressor.h | 55 +++++++++++++++++++++++++++++++++++++++
fs/squashfs/squashfs.h | 14 +++++-----
fs/squashfs/squashfs_fs_sb.h | 41 +++++++++++++++--------------
fs/squashfs/super.c | 45 ++++++++++++++++++-------------
fs/squashfs/zlib_wrapper.c | 17 ++++++++++--
8 files changed, 184 insertions(+), 51 deletions(-)
create mode 100644 fs/squashfs/decompressor.c
create mode 100644 fs/squashfs/decompressor.h
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index a397e6f..df8a19e 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -4,4 +4,4 @@
obj-$(CONFIG_SQUASHFS) += squashfs.o
squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o
-squashfs-y += namei.o super.o symlink.o zlib_wrapper.o
+squashfs-y += namei.o super.o symlink.o zlib_wrapper.o decompressor.o
diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c
index 3f836e1..1cb0d81 100644
--- a/fs/squashfs/block.c
+++ b/fs/squashfs/block.c
@@ -36,6 +36,7 @@
#include "squashfs_fs_sb.h"
#include "squashfs_fs_i.h"
#include "squashfs.h"
+#include "decompressor.h"
/*
* Read the metadata block length, this is stored in the first two
@@ -151,7 +152,7 @@ int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
}
if (compressed) {
- length = squashfs_zlib_uncompress(msblk, buffer, bh, b, offset,
+ length = squashfs_decompress(msblk, buffer, bh, b, offset,
length, srclength, pages);
if (length < 0)
goto read_failure;
diff --git a/fs/squashfs/decompressor.c b/fs/squashfs/decompressor.c
new file mode 100644
index 0000000..0072ccd
--- /dev/null
+++ b/fs/squashfs/decompressor.c
@@ -0,0 +1,58 @@
+/*
+ * Squashfs - a compressed read only filesystem for Linux
+ *
+ * Copyright (c) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
+ * Phillip Lougher <phillip@lougher.demon.co.uk>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ *
+ * decompressor.c
+ */
+
+#include <linux/types.h>
+#include <linux/mutex.h>
+#include <linux/buffer_head.h>
+
+#include "squashfs_fs.h"
+#include "squashfs_fs_sb.h"
+#include "squashfs_fs_i.h"
+#include "decompressor.h"
+#include "squashfs.h"
+
+/*
+ * This file (and decompressor.h) implements a decompressor framework for
+ * Squashfs, allowing multiple decompressors to be easily supported
+ */
+
+static const struct squashfs_decompressor squashfs_unknown_comp_ops = {
+ NULL, NULL, NULL, 0, "unknown", 0
+};
+
+static const struct squashfs_decompressor *decompressor[] = {
+ &squashfs_zlib_comp_ops,
+ &squashfs_unknown_comp_ops
+};
+
+
+const struct squashfs_decompressor *squashfs_lookup_decompressor(int id)
+{
+ int i;
+
+ for (i = 0; decompressor[i]->id; i++)
+ if (id == decompressor[i]->id)
+ break;
+
+ return decompressor[i];
+}
diff --git a/fs/squashfs/decompressor.h b/fs/squashfs/decompressor.h
new file mode 100644
index 0000000..7425f80
--- /dev/null
+++ b/fs/squashfs/decompressor.h
@@ -0,0 +1,55 @@
+#ifndef DECOMPRESSOR_H
+#define DECOMPRESSOR_H
+/*
+ * Squashfs - a compressed read only filesystem for Linux
+ *
+ * Copyright (c) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
+ * Phillip Lougher <phillip@lougher.demon.co.uk>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ *
+ * decompressor.h
+ */
+
+struct squashfs_decompressor {
+ void *(*init)(struct squashfs_sb_info *);
+ void (*free)(void *);
+ int (*decompress)(struct squashfs_sb_info *, void **,
+ struct buffer_head **, int, int, int, int, int);
+ int id;
+ char *name;
+ int supported;
+};
+
+static inline void *squashfs_decompressor_init(struct squashfs_sb_info *msblk)
+{
+ return msblk->decompressor->init(msblk);
+}
+
+static inline void squashfs_decompressor_free(struct squashfs_sb_info *msblk,
+ void *s)
+{
+ if (msblk->decompressor)
+ msblk->decompressor->free(s);
+}
+
+static inline int squashfs_decompress(struct squashfs_sb_info *msblk,
+ void **buffer, struct buffer_head **bh, int b, int offset, int length,
+ int srclength, int pages)
+{
+ return msblk->decompressor->decompress(msblk, buffer, bh, b, offset,
+ length, srclength, pages);
+}
+#endif
diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index 9c2f76a..fe2587a 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -51,6 +51,9 @@ extern struct squashfs_cache_entry *squashfs_get_datablock(struct super_block *,
u64, int);
extern int squashfs_read_table(struct super_block *, void *, u64, int);
+/* decompressor.c */
+extern const struct squashfs_decompressor *squashfs_lookup_decompressor(int);
+
/* export.c */
extern __le64 *squashfs_read_inode_lookup_table(struct super_block *, u64,
unsigned int);
@@ -70,14 +73,8 @@ extern struct inode *squashfs_iget(struct super_block *, long long,
unsigned int);
extern int squashfs_read_inode(struct inode *, long long);
-/* zlib_wrapper.c */
-extern void *squashfs_zlib_init(void);
-extern void squashfs_zlib_free(void *);
-extern int squashfs_zlib_uncompress(struct squashfs_sb_info *, void **,
- struct buffer_head **, int, int, int, int, int);
-
/*
- * Inodes and files operations
+ * Inodes, files and decompressor operations
*/
/* dir.c */
@@ -94,3 +91,6 @@ extern const struct inode_operations squashfs_dir_inode_ops;
/* symlink.c */
extern const struct address_space_operations squashfs_symlink_aops;
+
+/* zlib_wrapper.c */
+extern const struct squashfs_decompressor squashfs_zlib_comp_ops;
diff --git a/fs/squashfs/squashfs_fs_sb.h b/fs/squashfs/squashfs_fs_sb.h
index 23a67fa..7533350 100644
--- a/fs/squashfs/squashfs_fs_sb.h
+++ b/fs/squashfs/squashfs_fs_sb.h
@@ -52,25 +52,26 @@ struct squashfs_cache_entry {
};
struct squashfs_sb_info {
- int devblksize;
- int devblksize_log2;
- struct squashfs_cache *block_cache;
- struct squashfs_cache *fragment_cache;
- struct squashfs_cache *read_page;
- int next_meta_index;
- __le64 *id_table;
- __le64 *fragment_index;
- unsigned int *fragment_index_2;
- struct mutex read_data_mutex;
- struct mutex meta_index_mutex;
- struct meta_index *meta_index;
- void *stream;
- __le64 *inode_lookup_table;
- u64 inode_table;
- u64 directory_table;
- unsigned int block_size;
- unsigned short block_log;
- long long bytes_used;
- unsigned int inodes;
+ const struct squashfs_decompressor *decompressor;
+ int devblksize;
+ int devblksize_log2;
+ struct squashfs_cache *block_cache;
+ struct squashfs_cache *fragment_cache;
+ struct squashfs_cache *read_page;
+ int next_meta_index;
+ __le64 *id_table;
+ __le64 *fragment_index;
+ unsigned int *fragment_index_2;
+ struct mutex read_data_mutex;
+ struct mutex meta_index_mutex;
+ struct meta_index *meta_index;
+ void *stream;
+ __le64 *inode_lookup_table;
+ u64 inode_table;
+ u64 directory_table;
+ unsigned int block_size;
+ unsigned short block_log;
+ long long bytes_used;
+ unsigned int inodes;
};
#endif
diff --git a/fs/squashfs/super.c b/fs/squashfs/super.c
index b9f8c6a..3550aec 100644
--- a/fs/squashfs/super.c
+++ b/fs/squashfs/super.c
@@ -41,27 +41,35 @@
#include "squashfs_fs_sb.h"
#include "squashfs_fs_i.h"
#include "squashfs.h"
+#include "decompressor.h"
static struct file_system_type squashfs_fs_type;
static const struct super_operations squashfs_super_ops;
-static int supported_squashfs_filesystem(short major, short minor, short comp)
+static const struct squashfs_decompressor *supported_squashfs_filesystem(short
+ major, short minor, short id)
{
+ const struct squashfs_decompressor *decompressor;
+
if (major < SQUASHFS_MAJOR) {
ERROR("Major/Minor mismatch, older Squashfs %d.%d "
"filesystems are unsupported\n", major, minor);
- return -EINVAL;
+ return NULL;
} else if (major > SQUASHFS_MAJOR || minor > SQUASHFS_MINOR) {
ERROR("Major/Minor mismatch, trying to mount newer "
"%d.%d filesystem\n", major, minor);
ERROR("Please update your kernel\n");
- return -EINVAL;
+ return NULL;
}
- if (comp != ZLIB_COMPRESSION)
- return -EINVAL;
+ decompressor = squashfs_lookup_decompressor(id);
+ if (!decompressor->supported) {
+ ERROR("Filesystem uses \"%s\" compression. This is not "
+ "supported\n", decompressor->name);
+ return NULL;
+ }
- return 0;
+ return decompressor;
}
@@ -86,10 +94,6 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
}
msblk = sb->s_fs_info;
- msblk->stream = squashfs_zlib_init();
- if (msblk->stream == NULL)
- goto failure;
-
sblk = kzalloc(sizeof(*sblk), GFP_KERNEL);
if (sblk == NULL) {
ERROR("Failed to allocate squashfs_super_block\n");
@@ -116,25 +120,25 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
goto failed_mount;
}
+ err = -EINVAL;
+
/* Check it is a SQUASHFS superblock */
sb->s_magic = le32_to_cpu(sblk->s_magic);
if (sb->s_magic != SQUASHFS_MAGIC) {
if (!silent)
ERROR("Can't find a SQUASHFS superblock on %s\n",
bdevname(sb->s_bdev, b));
- err = -EINVAL;
goto failed_mount;
}
- /* Check the MAJOR & MINOR versions and compression type */
- err = supported_squashfs_filesystem(le16_to_cpu(sblk->s_major),
+ /* Check the MAJOR & MINOR versions and lookup compression type */
+ msblk->decompressor = supported_squashfs_filesystem(
+ le16_to_cpu(sblk->s_major),
le16_to_cpu(sblk->s_minor),
le16_to_cpu(sblk->compression));
- if (err < 0)
+ if (msblk->decompressor == NULL)
goto failed_mount;
- err = -EINVAL;
-
/*
* Check if there's xattrs in the filesystem. These are not
* supported in this version, so warn that they will be ignored.
@@ -201,6 +205,10 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
err = -ENOMEM;
+ msblk->stream = squashfs_decompressor_init(msblk);
+ if (msblk->stream == NULL)
+ goto failed_mount;
+
msblk->block_cache = squashfs_cache_init("metadata",
SQUASHFS_CACHED_BLKS, SQUASHFS_METADATA_SIZE);
if (msblk->block_cache == NULL)
@@ -288,7 +296,7 @@ failed_mount:
squashfs_cache_delete(msblk->block_cache);
squashfs_cache_delete(msblk->fragment_cache);
squashfs_cache_delete(msblk->read_page);
- squashfs_zlib_free(msblk->stream);
+ squashfs_decompressor_free(msblk, msblk->stream);
kfree(msblk->inode_lookup_table);
kfree(msblk->fragment_index);
kfree(msblk->id_table);
@@ -298,7 +306,6 @@ failed_mount:
return err;
failure:
- squashfs_zlib_free(msblk->stream);
kfree(sb->s_fs_info);
sb->s_fs_info = NULL;
return -ENOMEM;
@@ -342,7 +349,7 @@ static void squashfs_put_super(struct super_block *sb)
squashfs_cache_delete(sbi->block_cache);
squashfs_cache_delete(sbi->fragment_cache);
squashfs_cache_delete(sbi->read_page);
- squashfs_zlib_free(sbi->stream);
+ squashfs_decompressor_free(sbi, sbi->stream);
kfree(sbi->id_table);
kfree(sbi->fragment_index);
kfree(sbi->meta_index);
diff --git a/fs/squashfs/zlib_wrapper.c b/fs/squashfs/zlib_wrapper.c
index c814594..4dd70e0 100644
--- a/fs/squashfs/zlib_wrapper.c
+++ b/fs/squashfs/zlib_wrapper.c
@@ -30,8 +30,9 @@
#include "squashfs_fs_sb.h"
#include "squashfs_fs_i.h"
#include "squashfs.h"
+#include "decompressor.h"
-void *squashfs_zlib_init()
+static void *zlib_init(struct squashfs_sb_info *dummy)
{
z_stream *stream = kmalloc(sizeof(z_stream), GFP_KERNEL);
if (stream == NULL)
@@ -50,7 +51,7 @@ failed:
}
-void squashfs_zlib_free(void *strm)
+static void zlib_free(void *strm)
{
z_stream *stream = strm;
@@ -60,7 +61,7 @@ void squashfs_zlib_free(void *strm)
}
-int squashfs_zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
+static int zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
struct buffer_head **bh, int b, int offset, int length, int srclength,
int pages)
{
@@ -137,3 +138,13 @@ release_mutex:
return -EIO;
}
+
+const struct squashfs_decompressor squashfs_zlib_comp_ops = {
+ .init = zlib_init,
+ .free = zlib_free,
+ .decompress = zlib_uncompress,
+ .id = ZLIB_COMPRESSION,
+ .name = "zlib",
+ .supported = 1
+};
+
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 6/8] lzma: Make lzma available to non initramfs/initrd code
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Add a config option DECOMPRESS_LZMA_NEEDED which allows subsystems to
specify they need the unlzma code. Normally decompress_unlzma.c is
compiled with __init and unlzma is not exported to modules.
Move INIT definition into separate header files for bzip2/lzma/inflate
so it can be defined differently for each decompressor.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
include/linux/decompress/bunzip2_mm.h | 12 ++++++++++++
include/linux/decompress/inflate_mm.h | 12 ++++++++++++
include/linux/decompress/mm.h | 3 ---
include/linux/decompress/unlzma_mm.h | 20 ++++++++++++++++++++
lib/Kconfig | 3 +++
lib/decompress_bunzip2.c | 1 +
lib/decompress_inflate.c | 1 +
lib/decompress_unlzma.c | 6 +++++-
8 files changed, 54 insertions(+), 4 deletions(-)
create mode 100644 include/linux/decompress/bunzip2_mm.h
create mode 100644 include/linux/decompress/inflate_mm.h
create mode 100644 include/linux/decompress/unlzma_mm.h
diff --git a/include/linux/decompress/bunzip2_mm.h b/include/linux/decompress/bunzip2_mm.h
new file mode 100644
index 0000000..cac6fef
--- /dev/null
+++ b/include/linux/decompress/bunzip2_mm.h
@@ -0,0 +1,12 @@
+#ifndef BUNZIP2_MM_H
+#define BUNZIP2_MM_H
+
+#ifdef STATIC
+/* Code active when included from pre-boot environment: */
+#define INIT
+#else
+/* Compile for initramfs/initrd code only */
+#define INIT __init
+#endif
+
+#endif
diff --git a/include/linux/decompress/inflate_mm.h b/include/linux/decompress/inflate_mm.h
new file mode 100644
index 0000000..ca4a2ae
--- /dev/null
+++ b/include/linux/decompress/inflate_mm.h
@@ -0,0 +1,12 @@
+#ifndef INFLATE_MM_H
+#define INFLATE_MM_H
+
+#ifdef STATIC
+/* Code active when included from pre-boot environment: */
+#define INIT
+#else
+/* Compile for initramfs/initrd code only */
+#define INIT __init
+#endif
+
+#endif
diff --git a/include/linux/decompress/mm.h b/include/linux/decompress/mm.h
index 12ff8c3..80f5ba4 100644
--- a/include/linux/decompress/mm.h
+++ b/include/linux/decompress/mm.h
@@ -53,8 +53,6 @@ static void free(void *where)
#define set_error_fn(x)
-#define INIT
-
#else /* STATIC */
/* Code active when compiled standalone for use when loading ramdisk: */
@@ -77,7 +75,6 @@ static void free(void *where)
static void(*error)(char *m);
#define set_error_fn(x) error = x;
-#define INIT __init
#define STATIC
#include <linux/init.h>
diff --git a/include/linux/decompress/unlzma_mm.h b/include/linux/decompress/unlzma_mm.h
new file mode 100644
index 0000000..859287e
--- /dev/null
+++ b/include/linux/decompress/unlzma_mm.h
@@ -0,0 +1,20 @@
+#ifndef UNLZMA_MM_H
+#define UNLZMA_MM_H
+
+#ifdef STATIC
+
+/* Code active when included from pre-boot environment: */
+#define INIT
+
+#elif defined(CONFIG_DECOMPRESS_LZMA_NEEDED)
+
+/* Make it available to non initramfs/initrd code */
+#define INIT
+#include <linux/module.h>
+#else
+
+/* Compile for initramfs/initrd code only */
+#define INIT __init
+#endif
+
+#endif
diff --git a/lib/Kconfig b/lib/Kconfig
index bb1326d..25e7f28 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -117,6 +117,9 @@ config DECOMPRESS_BZIP2
config DECOMPRESS_LZMA
tristate
+config DECOMPRESS_LZMA_NEEDED
+ boolean
+
#
# Generic allocator support is selected if needed
#
diff --git a/lib/decompress_bunzip2.c b/lib/decompress_bunzip2.c
index 600f473..6eb6433 100644
--- a/lib/decompress_bunzip2.c
+++ b/lib/decompress_bunzip2.c
@@ -52,6 +52,7 @@
#include <linux/slab.h>
#endif /* STATIC */
+#include <linux/decompress/bunzip2_mm.h>
#include <linux/decompress/mm.h>
#ifndef INT_MAX
diff --git a/lib/decompress_inflate.c b/lib/decompress_inflate.c
index fc686c7..cb6bcab 100644
--- a/lib/decompress_inflate.c
+++ b/lib/decompress_inflate.c
@@ -23,6 +23,7 @@
#endif /* STATIC */
+#include <linux/decompress/inflate_mm.h>
#include <linux/decompress/mm.h>
#define GZIP_IOBUF_SIZE (16*1024)
diff --git a/lib/decompress_unlzma.c b/lib/decompress_unlzma.c
index ca82fde..a614b26 100644
--- a/lib/decompress_unlzma.c
+++ b/lib/decompress_unlzma.c
@@ -36,6 +36,7 @@
#include <linux/slab.h>
#endif /* STATIC */
+#include <linux/decompress/unlzma_mm.h>
#include <linux/decompress/mm.h>
#define MIN(a, b) (((a) < (b)) ? (a) : (b))
@@ -531,7 +532,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
-STATIC inline int INIT unlzma(unsigned char *buf, int in_len,
+STATIC int INIT unlzma(unsigned char *buf, int in_len,
int(*fill)(void*, unsigned int),
int(*flush)(void*, unsigned int),
unsigned char *output,
@@ -652,6 +653,9 @@ exit_1:
exit_0:
return ret;
}
+#if defined(CONFIG_DECOMPRESS_LZMA_NEEDED) && !defined(PREBOOT)
+EXPORT_SYMBOL(unlzma);
+#endif
#ifdef PREBOOT
STATIC int INIT decompress(unsigned char *buf, int in_len,
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 1/8] Squashfs: move zlib decompression wrapper code into a separate file
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
fs/squashfs/Makefile | 2 +-
fs/squashfs/block.c | 74 ++----------------------------
fs/squashfs/squashfs.h | 4 ++
fs/squashfs/zlib_wrapper.c | 109 ++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 118 insertions(+), 71 deletions(-)
create mode 100644 fs/squashfs/zlib_wrapper.c
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index 70e3244..a397e6f 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -4,4 +4,4 @@
obj-$(CONFIG_SQUASHFS) += squashfs.o
squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o
-squashfs-y += namei.o super.o symlink.o
+squashfs-y += namei.o super.o symlink.o zlib_wrapper.o
diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c
index 2a79603..b8addfd 100644
--- a/fs/squashfs/block.c
+++ b/fs/squashfs/block.c
@@ -29,7 +29,6 @@
#include <linux/fs.h>
#include <linux/vfs.h>
#include <linux/slab.h>
-#include <linux/mutex.h>
#include <linux/string.h>
#include <linux/buffer_head.h>
#include <linux/zlib.h>
@@ -153,72 +152,10 @@ int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
}
if (compressed) {
- int zlib_err = 0, zlib_init = 0;
-
- /*
- * Uncompress block.
- */
-
- mutex_lock(&msblk->read_data_mutex);
-
- msblk->stream.avail_out = 0;
- msblk->stream.avail_in = 0;
-
- bytes = length;
- do {
- if (msblk->stream.avail_in == 0 && k < b) {
- avail = min(bytes, msblk->devblksize - offset);
- bytes -= avail;
- wait_on_buffer(bh[k]);
- if (!buffer_uptodate(bh[k]))
- goto release_mutex;
-
- if (avail == 0) {
- offset = 0;
- put_bh(bh[k++]);
- continue;
- }
-
- msblk->stream.next_in = bh[k]->b_data + offset;
- msblk->stream.avail_in = avail;
- offset = 0;
- }
-
- if (msblk->stream.avail_out == 0 && page < pages) {
- msblk->stream.next_out = buffer[page++];
- msblk->stream.avail_out = PAGE_CACHE_SIZE;
- }
-
- if (!zlib_init) {
- zlib_err = zlib_inflateInit(&msblk->stream);
- if (zlib_err != Z_OK) {
- ERROR("zlib_inflateInit returned"
- " unexpected result 0x%x,"
- " srclength %d\n", zlib_err,
- srclength);
- goto release_mutex;
- }
- zlib_init = 1;
- }
-
- zlib_err = zlib_inflate(&msblk->stream, Z_SYNC_FLUSH);
-
- if (msblk->stream.avail_in == 0 && k < b)
- put_bh(bh[k++]);
- } while (zlib_err == Z_OK);
-
- if (zlib_err != Z_STREAM_END) {
- ERROR("zlib_inflate error, data probably corrupt\n");
- goto release_mutex;
- }
-
- zlib_err = zlib_inflateEnd(&msblk->stream);
- if (zlib_err != Z_OK) {
- ERROR("zlib_inflate error, data probably corrupt\n");
- goto release_mutex;
- }
- length = msblk->stream.total_out;
- mutex_unlock(&msblk->read_data_mutex);
+ length = squashfs_zlib_uncompress(msblk, buffer, bh, b, offset,
+ length, srclength, pages);
+ if (length < 0)
+ goto read_failure;
} else {
/*
* Block is uncompressed.
@@ -255,9 +192,6 @@ int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
kfree(bh);
return length;
-release_mutex:
- mutex_unlock(&msblk->read_data_mutex);
-
block_release:
for (; k < b; k++)
put_bh(bh[k]);
diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index 0e9feb6..ba87db6 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -70,6 +70,10 @@ extern struct inode *squashfs_iget(struct super_block *, long long,
unsigned int);
extern int squashfs_read_inode(struct inode *, long long);
+/* zlib_wrapper.c */
+extern int squashfs_zlib_uncompress(struct squashfs_sb_info *, void **,
+ struct buffer_head **, int, int, int, int, int);
+
/*
* Inodes and files operations
*/
diff --git a/fs/squashfs/zlib_wrapper.c b/fs/squashfs/zlib_wrapper.c
new file mode 100644
index 0000000..3be9964
--- /dev/null
+++ b/fs/squashfs/zlib_wrapper.c
@@ -0,0 +1,109 @@
+/*
+ * Squashfs - a compressed read only filesystem for Linux
+ *
+ * Copyright (c) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
+ * Phillip Lougher <phillip@lougher.demon.co.uk>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ *
+ * zlib_wrapper.c
+ */
+
+
+#include <linux/mutex.h>
+#include <linux/buffer_head.h>
+#include <linux/zlib.h>
+
+#include "squashfs_fs.h"
+#include "squashfs_fs_sb.h"
+#include "squashfs_fs_i.h"
+#include "squashfs.h"
+
+int squashfs_zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
+ struct buffer_head **bh, int b, int offset, int length, int srclength,
+ int pages)
+{
+ int zlib_err = 0, zlib_init = 0;
+ int avail, bytes, k = 0, page = 0;
+
+ mutex_lock(&msblk->read_data_mutex);
+
+ msblk->stream.avail_out = 0;
+ msblk->stream.avail_in = 0;
+
+ bytes = length;
+ do {
+ if (msblk->stream.avail_in == 0 && k < b) {
+ avail = min(bytes, msblk->devblksize - offset);
+ bytes -= avail;
+ wait_on_buffer(bh[k]);
+ if (!buffer_uptodate(bh[k]))
+ goto release_mutex;
+
+ if (avail == 0) {
+ offset = 0;
+ put_bh(bh[k++]);
+ continue;
+ }
+
+ msblk->stream.next_in = bh[k]->b_data + offset;
+ msblk->stream.avail_in = avail;
+ offset = 0;
+ }
+
+ if (msblk->stream.avail_out == 0 && page < pages) {
+ msblk->stream.next_out = buffer[page++];
+ msblk->stream.avail_out = PAGE_CACHE_SIZE;
+ }
+
+ if (!zlib_init) {
+ zlib_err = zlib_inflateInit(&msblk->stream);
+ if (zlib_err != Z_OK) {
+ ERROR("zlib_inflateInit returned unexpected "
+ "result 0x%x, srclength %d\n",
+ zlib_err, srclength);
+ goto release_mutex;
+ }
+ zlib_init = 1;
+ }
+
+ zlib_err = zlib_inflate(&msblk->stream, Z_SYNC_FLUSH);
+
+ if (msblk->stream.avail_in == 0 && k < b)
+ put_bh(bh[k++]);
+ } while (zlib_err == Z_OK);
+
+ if (zlib_err != Z_STREAM_END) {
+ ERROR("zlib_inflate error, data probably corrupt\n");
+ goto release_mutex;
+ }
+
+ zlib_err = zlib_inflateEnd(&msblk->stream);
+ if (zlib_err != Z_OK) {
+ ERROR("zlib_inflate error, data probably corrupt\n");
+ goto release_mutex;
+ }
+
+ mutex_unlock(&msblk->read_data_mutex);
+ return msblk->stream.total_out;
+
+release_mutex:
+ mutex_unlock(&msblk->read_data_mutex);
+
+ for (; k < b; k++)
+ put_bh(bh[k]);
+
+ return -EIO;
+}
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 4/8] Squashfs: add decompressor entries for lzma and lzo
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Add knowledge of lzma/lzo compression formats to the decompressor
framework. For now these are added as unsupported. Without
these entries lzma/lzo compressed filesystems will be flagged as
having unknown compression which is undesirable.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
fs/squashfs/decompressor.c | 10 ++++++++++
fs/squashfs/squashfs_fs.h | 4 +++-
2 files changed, 13 insertions(+), 1 deletions(-)
diff --git a/fs/squashfs/decompressor.c b/fs/squashfs/decompressor.c
index 0072ccd..157478d 100644
--- a/fs/squashfs/decompressor.c
+++ b/fs/squashfs/decompressor.c
@@ -36,12 +36,22 @@
* Squashfs, allowing multiple decompressors to be easily supported
*/
+static const struct squashfs_decompressor squashfs_lzma_unsupported_comp_ops = {
+ NULL, NULL, NULL, LZMA_COMPRESSION, "lzma", 0
+};
+
+static const struct squashfs_decompressor squashfs_lzo_unsupported_comp_ops = {
+ NULL, NULL, NULL, LZO_COMPRESSION, "lzo", 0
+};
+
static const struct squashfs_decompressor squashfs_unknown_comp_ops = {
NULL, NULL, NULL, 0, "unknown", 0
};
static const struct squashfs_decompressor *decompressor[] = {
&squashfs_zlib_comp_ops,
+ &squashfs_lzma_unsupported_comp_ops,
+ &squashfs_lzo_unsupported_comp_ops,
&squashfs_unknown_comp_ops
};
diff --git a/fs/squashfs/squashfs_fs.h b/fs/squashfs/squashfs_fs.h
index 283daaf..36e1604 100644
--- a/fs/squashfs/squashfs_fs.h
+++ b/fs/squashfs/squashfs_fs.h
@@ -211,7 +211,9 @@ struct meta_index {
/*
* definitions for structures on disk
*/
-#define ZLIB_COMPRESSION 1
+#define ZLIB_COMPRESSION 1
+#define LZMA_COMPRESSION 2
+#define LZO_COMPRESSION 3
struct squashfs_super_block {
__le32 s_magic;
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 5/8] Squashfs: add support for LZMA compressed filesystems
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
fs/squashfs/Kconfig | 5 ++
fs/squashfs/Makefile | 1 +
fs/squashfs/decompressor.c | 4 +
fs/squashfs/lzma_wrapper.c | 151 ++++++++++++++++++++++++++++++++++++++++++++
fs/squashfs/squashfs.h | 3 +
5 files changed, 164 insertions(+), 0 deletions(-)
create mode 100644 fs/squashfs/lzma_wrapper.c
diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index 25a00d1..0294aa2 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -26,6 +26,11 @@ config SQUASHFS
If unsure, say N.
+config SQUASHFS_LZMA
+ bool "Include support for LZMA compressed file systems"
+ depends on SQUASHFS
+ select DECOMPRESS_LZMA
+
config SQUASHFS_EMBEDDED
bool "Additional option for memory-constrained systems"
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index df8a19e..45aaefd 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -5,3 +5,4 @@
obj-$(CONFIG_SQUASHFS) += squashfs.o
squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o
squashfs-y += namei.o super.o symlink.o zlib_wrapper.o decompressor.o
+squashfs-$(CONFIG_SQUASHFS_LZMA) += lzma_wrapper.o
diff --git a/fs/squashfs/decompressor.c b/fs/squashfs/decompressor.c
index 157478d..0b6ad9b 100644
--- a/fs/squashfs/decompressor.c
+++ b/fs/squashfs/decompressor.c
@@ -50,7 +50,11 @@ static const struct squashfs_decompressor squashfs_unknown_comp_ops = {
static const struct squashfs_decompressor *decompressor[] = {
&squashfs_zlib_comp_ops,
+#ifdef CONFIG_SQUASHFS_LZMA
+ &squashfs_lzma_comp_ops,
+#else
&squashfs_lzma_unsupported_comp_ops,
+#endif
&squashfs_lzo_unsupported_comp_ops,
&squashfs_unknown_comp_ops
};
diff --git a/fs/squashfs/lzma_wrapper.c b/fs/squashfs/lzma_wrapper.c
new file mode 100644
index 0000000..cef06d6
--- /dev/null
+++ b/fs/squashfs/lzma_wrapper.c
@@ -0,0 +1,151 @@
+/*
+ * Squashfs - a compressed read only filesystem for Linux
+ *
+ * Copyright (c) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
+ * Phillip Lougher <phillip@lougher.demon.co.uk>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ *
+ * lzma_wrapper.c
+ */
+
+#include <asm/unaligned.h>
+#include <linux/buffer_head.h>
+#include <linux/mutex.h>
+#include <linux/vmalloc.h>
+#include <linux/decompress/unlzma.h>
+
+#include "squashfs_fs.h"
+#include "squashfs_fs_sb.h"
+#include "squashfs_fs_i.h"
+#include "squashfs.h"
+#include "decompressor.h"
+
+struct squashfs_lzma {
+ void *input;
+ void *output;
+};
+
+/* decompress_unlzma.c is currently non re-entrant... */
+DEFINE_MUTEX(lzma_mutex);
+
+/* decompress_unlzma.c doesn't provide any context in its callbacks... */
+static int lzma_error;
+
+static void error(char *m)
+{
+ ERROR("unlzma error: %s\n", m);
+ lzma_error = 1;
+}
+
+
+static void *lzma_init(struct squashfs_sb_info *msblk)
+{
+ struct squashfs_lzma *stream = kzalloc(sizeof(*stream), GFP_KERNEL);
+ if (stream == NULL)
+ goto failed;
+ stream->input = vmalloc(msblk->block_size);
+ if (stream->input == NULL)
+ goto failed;
+ stream->output = vmalloc(msblk->block_size);
+ if (stream->output == NULL)
+ goto failed2;
+
+ return stream;
+
+failed2:
+ vfree(stream->input);
+failed:
+ ERROR("failed to allocate lzma workspace\n");
+ kfree(stream);
+ return NULL;
+}
+
+
+static void lzma_free(void *strm)
+{
+ struct squashfs_lzma *stream = strm;
+
+ if (stream) {
+ vfree(stream->input);
+ vfree(stream->output);
+ }
+ kfree(stream);
+}
+
+
+static int lzma_uncompress(struct squashfs_sb_info *msblk, void **buffer,
+ struct buffer_head **bh, int b, int offset, int length, int srclength,
+ int pages)
+{
+ struct squashfs_lzma *stream = msblk->stream;
+ void *buff = stream->input;
+ int avail, i, bytes = length, res;
+
+ mutex_lock(&lzma_mutex);
+
+ for (i = 0; i < b; i++) {
+ wait_on_buffer(bh[i]);
+ if (!buffer_uptodate(bh[i]))
+ goto block_release;
+
+ avail = min(bytes, msblk->devblksize - offset);
+ memcpy(buff, bh[i]->b_data + offset, avail);
+ buff += avail;
+ bytes -= avail;
+ offset = 0;
+ put_bh(bh[i]);
+ }
+
+ lzma_error = 0;
+ res = unlzma(stream->input, length, NULL, NULL, stream->output, NULL,
+ error);
+ if (res || lzma_error)
+ goto failed;
+
+ /* uncompressed size is stored in the LZMA header (5 byte offset) */
+ res = bytes = get_unaligned_le32(stream->input + 5);
+ for (i = 0, buff = stream->output; bytes && i < pages; i++) {
+ avail = min_t(int, bytes, PAGE_CACHE_SIZE);
+ memcpy(buffer[i], buff, avail);
+ buff += avail;
+ bytes -= avail;
+ }
+ if (bytes)
+ goto failed;
+
+ mutex_unlock(&lzma_mutex);
+ return res;
+
+block_release:
+ for (; i < b; i++)
+ put_bh(bh[i]);
+
+failed:
+ mutex_unlock(&lzma_mutex);
+
+ ERROR("lzma decompression failed, data probably corrupt\n");
+ return -EIO;
+}
+
+const struct squashfs_decompressor squashfs_lzma_comp_ops = {
+ .init = lzma_init,
+ .free = lzma_free,
+ .decompress = lzma_uncompress,
+ .id = LZMA_COMPRESSION,
+ .name = "lzma",
+ .supported = 1
+};
+
diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index fe2587a..d094886 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -94,3 +94,6 @@ extern const struct address_space_operations squashfs_symlink_aops;
/* zlib_wrapper.c */
extern const struct squashfs_decompressor squashfs_zlib_comp_ops;
+
+/* lzma wrapper.c */
+extern const struct squashfs_decompressor squashfs_lzma_comp_ops;
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 2/8] Squashfs: Factor out remaining zlib dependencies into separate wrapper file
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
Move zlib buffer init/destroy code into separate wrapper file. Also
make zlib z_stream field a void * removing the need to include zlib.h
for most files.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
fs/squashfs/block.c | 1 -
fs/squashfs/cache.c | 1 -
fs/squashfs/dir.c | 1 -
fs/squashfs/export.c | 1 -
fs/squashfs/file.c | 1 -
fs/squashfs/fragment.c | 1 -
fs/squashfs/id.c | 1 -
fs/squashfs/inode.c | 1 -
fs/squashfs/namei.c | 1 -
fs/squashfs/squashfs.h | 2 +
fs/squashfs/squashfs_fs_sb.h | 2 +-
fs/squashfs/super.c | 14 +++------
fs/squashfs/symlink.c | 1 -
fs/squashfs/zlib_wrapper.c | 56 ++++++++++++++++++++++++++++++++---------
14 files changed, 51 insertions(+), 33 deletions(-)
diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c
index b8addfd..3f836e1 100644
--- a/fs/squashfs/block.c
+++ b/fs/squashfs/block.c
@@ -31,7 +31,6 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/buffer_head.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/cache.c b/fs/squashfs/cache.c
index 40c98fa..57314be 100644
--- a/fs/squashfs/cache.c
+++ b/fs/squashfs/cache.c
@@ -51,7 +51,6 @@
#include <linux/sched.h>
#include <linux/spinlock.h>
#include <linux/wait.h>
-#include <linux/zlib.h>
#include <linux/pagemap.h>
#include "squashfs_fs.h"
diff --git a/fs/squashfs/dir.c b/fs/squashfs/dir.c
index 566b0ea..12b933a 100644
--- a/fs/squashfs/dir.c
+++ b/fs/squashfs/dir.c
@@ -30,7 +30,6 @@
#include <linux/fs.h>
#include <linux/vfs.h>
#include <linux/slab.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/export.c b/fs/squashfs/export.c
index 2b1b8fe..7f93d5a 100644
--- a/fs/squashfs/export.c
+++ b/fs/squashfs/export.c
@@ -39,7 +39,6 @@
#include <linux/vfs.h>
#include <linux/dcache.h>
#include <linux/exportfs.h>
-#include <linux/zlib.h>
#include <linux/slab.h>
#include "squashfs_fs.h"
diff --git a/fs/squashfs/file.c b/fs/squashfs/file.c
index 717767d..a25c506 100644
--- a/fs/squashfs/file.c
+++ b/fs/squashfs/file.c
@@ -47,7 +47,6 @@
#include <linux/string.h>
#include <linux/pagemap.h>
#include <linux/mutex.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/fragment.c b/fs/squashfs/fragment.c
index b5a2c15..7c90bbd 100644
--- a/fs/squashfs/fragment.c
+++ b/fs/squashfs/fragment.c
@@ -36,7 +36,6 @@
#include <linux/fs.h>
#include <linux/vfs.h>
#include <linux/slab.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/id.c b/fs/squashfs/id.c
index 3795b83..b7f64bc 100644
--- a/fs/squashfs/id.c
+++ b/fs/squashfs/id.c
@@ -34,7 +34,6 @@
#include <linux/fs.h>
#include <linux/vfs.h>
#include <linux/slab.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/inode.c b/fs/squashfs/inode.c
index 9101dbd..49daaf6 100644
--- a/fs/squashfs/inode.c
+++ b/fs/squashfs/inode.c
@@ -40,7 +40,6 @@
#include <linux/fs.h>
#include <linux/vfs.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/namei.c b/fs/squashfs/namei.c
index 9e39865..5266bd8 100644
--- a/fs/squashfs/namei.c
+++ b/fs/squashfs/namei.c
@@ -57,7 +57,6 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/dcache.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index ba87db6..9c2f76a 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -71,6 +71,8 @@ extern struct inode *squashfs_iget(struct super_block *, long long,
extern int squashfs_read_inode(struct inode *, long long);
/* zlib_wrapper.c */
+extern void *squashfs_zlib_init(void);
+extern void squashfs_zlib_free(void *);
extern int squashfs_zlib_uncompress(struct squashfs_sb_info *, void **,
struct buffer_head **, int, int, int, int, int);
diff --git a/fs/squashfs/squashfs_fs_sb.h b/fs/squashfs/squashfs_fs_sb.h
index c8c6561..23a67fa 100644
--- a/fs/squashfs/squashfs_fs_sb.h
+++ b/fs/squashfs/squashfs_fs_sb.h
@@ -64,7 +64,7 @@ struct squashfs_sb_info {
struct mutex read_data_mutex;
struct mutex meta_index_mutex;
struct meta_index *meta_index;
- z_stream stream;
+ void *stream;
__le64 *inode_lookup_table;
u64 inode_table;
u64 directory_table;
diff --git a/fs/squashfs/super.c b/fs/squashfs/super.c
index 6c197ef..b9f8c6a 100644
--- a/fs/squashfs/super.c
+++ b/fs/squashfs/super.c
@@ -35,7 +35,6 @@
#include <linux/pagemap.h>
#include <linux/init.h>
#include <linux/module.h>
-#include <linux/zlib.h>
#include <linux/magic.h>
#include "squashfs_fs.h"
@@ -87,12 +86,9 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
}
msblk = sb->s_fs_info;
- msblk->stream.workspace = kmalloc(zlib_inflate_workspacesize(),
- GFP_KERNEL);
- if (msblk->stream.workspace == NULL) {
- ERROR("Failed to allocate zlib workspace\n");
+ msblk->stream = squashfs_zlib_init();
+ if (msblk->stream == NULL)
goto failure;
- }
sblk = kzalloc(sizeof(*sblk), GFP_KERNEL);
if (sblk == NULL) {
@@ -292,17 +288,17 @@ failed_mount:
squashfs_cache_delete(msblk->block_cache);
squashfs_cache_delete(msblk->fragment_cache);
squashfs_cache_delete(msblk->read_page);
+ squashfs_zlib_free(msblk->stream);
kfree(msblk->inode_lookup_table);
kfree(msblk->fragment_index);
kfree(msblk->id_table);
- kfree(msblk->stream.workspace);
kfree(sb->s_fs_info);
sb->s_fs_info = NULL;
kfree(sblk);
return err;
failure:
- kfree(msblk->stream.workspace);
+ squashfs_zlib_free(msblk->stream);
kfree(sb->s_fs_info);
sb->s_fs_info = NULL;
return -ENOMEM;
@@ -346,10 +342,10 @@ static void squashfs_put_super(struct super_block *sb)
squashfs_cache_delete(sbi->block_cache);
squashfs_cache_delete(sbi->fragment_cache);
squashfs_cache_delete(sbi->read_page);
+ squashfs_zlib_free(sbi->stream);
kfree(sbi->id_table);
kfree(sbi->fragment_index);
kfree(sbi->meta_index);
- kfree(sbi->stream.workspace);
kfree(sb->s_fs_info);
sb->s_fs_info = NULL;
}
diff --git a/fs/squashfs/symlink.c b/fs/squashfs/symlink.c
index 83d8788..e80be20 100644
--- a/fs/squashfs/symlink.c
+++ b/fs/squashfs/symlink.c
@@ -36,7 +36,6 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/pagemap.h>
-#include <linux/zlib.h>
#include "squashfs_fs.h"
#include "squashfs_fs_sb.h"
diff --git a/fs/squashfs/zlib_wrapper.c b/fs/squashfs/zlib_wrapper.c
index 3be9964..c814594 100644
--- a/fs/squashfs/zlib_wrapper.c
+++ b/fs/squashfs/zlib_wrapper.c
@@ -31,21 +31,51 @@
#include "squashfs_fs_i.h"
#include "squashfs.h"
+void *squashfs_zlib_init()
+{
+ z_stream *stream = kmalloc(sizeof(z_stream), GFP_KERNEL);
+ if (stream == NULL)
+ goto failed;
+ stream->workspace = kmalloc(zlib_inflate_workspacesize(),
+ GFP_KERNEL);
+ if (stream->workspace == NULL)
+ goto failed;
+
+ return stream;
+
+failed:
+ ERROR("Failed to allocate zlib workspace\n");
+ kfree(stream);
+ return NULL;
+}
+
+
+void squashfs_zlib_free(void *strm)
+{
+ z_stream *stream = strm;
+
+ if (stream)
+ kfree(stream->workspace);
+ kfree(stream);
+}
+
+
int squashfs_zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
struct buffer_head **bh, int b, int offset, int length, int srclength,
int pages)
{
int zlib_err = 0, zlib_init = 0;
int avail, bytes, k = 0, page = 0;
+ z_stream *stream = msblk->stream;
mutex_lock(&msblk->read_data_mutex);
- msblk->stream.avail_out = 0;
- msblk->stream.avail_in = 0;
+ stream->avail_out = 0;
+ stream->avail_in = 0;
bytes = length;
do {
- if (msblk->stream.avail_in == 0 && k < b) {
+ if (stream->avail_in == 0 && k < b) {
avail = min(bytes, msblk->devblksize - offset);
bytes -= avail;
wait_on_buffer(bh[k]);
@@ -58,18 +88,18 @@ int squashfs_zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
continue;
}
- msblk->stream.next_in = bh[k]->b_data + offset;
- msblk->stream.avail_in = avail;
+ stream->next_in = bh[k]->b_data + offset;
+ stream->avail_in = avail;
offset = 0;
}
- if (msblk->stream.avail_out == 0 && page < pages) {
- msblk->stream.next_out = buffer[page++];
- msblk->stream.avail_out = PAGE_CACHE_SIZE;
+ if (stream->avail_out == 0 && page < pages) {
+ stream->next_out = buffer[page++];
+ stream->avail_out = PAGE_CACHE_SIZE;
}
if (!zlib_init) {
- zlib_err = zlib_inflateInit(&msblk->stream);
+ zlib_err = zlib_inflateInit(stream);
if (zlib_err != Z_OK) {
ERROR("zlib_inflateInit returned unexpected "
"result 0x%x, srclength %d\n",
@@ -79,9 +109,9 @@ int squashfs_zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
zlib_init = 1;
}
- zlib_err = zlib_inflate(&msblk->stream, Z_SYNC_FLUSH);
+ zlib_err = zlib_inflate(stream, Z_SYNC_FLUSH);
- if (msblk->stream.avail_in == 0 && k < b)
+ if (stream->avail_in == 0 && k < b)
put_bh(bh[k++]);
} while (zlib_err == Z_OK);
@@ -90,14 +120,14 @@ int squashfs_zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
goto release_mutex;
}
- zlib_err = zlib_inflateEnd(&msblk->stream);
+ zlib_err = zlib_inflateEnd(stream);
if (zlib_err != Z_OK) {
ERROR("zlib_inflate error, data probably corrupt\n");
goto release_mutex;
}
mutex_unlock(&msblk->read_data_mutex);
- return msblk->stream.total_out;
+ return stream->total_out;
release_mutex:
mutex_unlock(&msblk->read_data_mutex);
--
1.6.3.3
^ permalink raw reply related
* [PATCH V2 8/8] lzma: make lzma reentrant
From: Phillip Lougher @ 2009-12-11 1:34 UTC (permalink / raw)
To: akpm, linux-embedded, linux-fsdevel, linux-kernel,
phillip.lougher, tim.bird
The error function pointer used by lzma is global (file scope) which
prevents it being used concurrently. This patch removes the global
error pointer use, and instead passes it to all functions that need it.
The error function pointer is still used by bzip2 and inflate.
This patch moves the definition into the separate bzip2/inflate
header files. This prevents gcc from complaining about an
unused definition compiling lzma.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
include/linux/decompress/bunzip2_mm.h | 1 +
include/linux/decompress/inflate_mm.h | 1 +
include/linux/decompress/mm.h | 1 -
lib/decompress_unlzma.c | 82 +++++++++++++++++----------------
4 files changed, 44 insertions(+), 41 deletions(-)
diff --git a/include/linux/decompress/bunzip2_mm.h b/include/linux/decompress/bunzip2_mm.h
index cac6fef..863efd0 100644
--- a/include/linux/decompress/bunzip2_mm.h
+++ b/include/linux/decompress/bunzip2_mm.h
@@ -7,6 +7,7 @@
#else
/* Compile for initramfs/initrd code only */
#define INIT __init
+static void(*error)(char *m);
#endif
#endif
diff --git a/include/linux/decompress/inflate_mm.h b/include/linux/decompress/inflate_mm.h
index ca4a2ae..87a742b 100644
--- a/include/linux/decompress/inflate_mm.h
+++ b/include/linux/decompress/inflate_mm.h
@@ -7,6 +7,7 @@
#else
/* Compile for initramfs/initrd code only */
#define INIT __init
+static void(*error)(char *m);
#endif
#endif
diff --git a/include/linux/decompress/mm.h b/include/linux/decompress/mm.h
index 80f5ba4..32651e4 100644
--- a/include/linux/decompress/mm.h
+++ b/include/linux/decompress/mm.h
@@ -72,7 +72,6 @@ static void free(void *where)
#define large_malloc(a) vmalloc(a)
#define large_free(a) vfree(a)
-static void(*error)(char *m);
#define set_error_fn(x) error = x;
#define STATIC
diff --git a/lib/decompress_unlzma.c b/lib/decompress_unlzma.c
index a614b26..3e85763 100644
--- a/lib/decompress_unlzma.c
+++ b/lib/decompress_unlzma.c
@@ -89,7 +89,7 @@ static int nofill(void *buffer, unsigned int len)
}
/* Called twice: once at startup and once in rc_normalize() */
-static void INIT rc_read(struct rc *rc)
+static void INIT rc_read(struct rc *rc, void(*error)(char *x))
{
rc->buffer_size = rc->fill((char *)rc->buffer, LZMA_IOBUF_SIZE);
if (rc->buffer_size <= 0)
@@ -116,13 +116,13 @@ static inline void INIT rc_init(struct rc *rc,
rc->range = 0xFFFFFFFF;
}
-static inline void INIT rc_init_code(struct rc *rc)
+static inline void INIT rc_init_code(struct rc *rc, void(*error)(char *x))
{
int i;
for (i = 0; i < 5; i++) {
if (rc->ptr >= rc->buffer_end)
- rc_read(rc);
+ rc_read(rc, error);
rc->code = (rc->code << 8) | *rc->ptr++;
}
}
@@ -135,32 +135,33 @@ static inline void INIT rc_free(struct rc *rc)
}
/* Called twice, but one callsite is in inline'd rc_is_bit_0_helper() */
-static void INIT rc_do_normalize(struct rc *rc)
+static void INIT rc_do_normalize(struct rc *rc, void(*error)(char *x))
{
if (rc->ptr >= rc->buffer_end)
- rc_read(rc);
+ rc_read(rc, error);
rc->range <<= 8;
rc->code = (rc->code << 8) | *rc->ptr++;
}
-static inline void INIT rc_normalize(struct rc *rc)
+static inline void INIT rc_normalize(struct rc *rc, void(*error)(char *x))
{
if (rc->range < (1 << RC_TOP_BITS))
- rc_do_normalize(rc);
+ rc_do_normalize(rc, error);
}
/* Called 9 times */
/* Why rc_is_bit_0_helper exists?
*Because we want to always expose (rc->code < rc->bound) to optimizer
*/
-static inline uint32_t INIT rc_is_bit_0_helper(struct rc *rc, uint16_t *p)
+static inline uint32_t INIT rc_is_bit_0_helper(struct rc *rc, uint16_t *p,
+ void (*error)(char *x))
{
- rc_normalize(rc);
+ rc_normalize(rc, error);
rc->bound = *p * (rc->range >> RC_MODEL_TOTAL_BITS);
return rc->bound;
}
-static inline int INIT rc_is_bit_0(struct rc *rc, uint16_t *p)
+static inline int INIT rc_is_bit_0(struct rc *rc, uint16_t *p, void(*error)(char *x))
{
- uint32_t t = rc_is_bit_0_helper(rc, p);
+ uint32_t t = rc_is_bit_0_helper(rc, p, error);
return rc->code < t;
}
@@ -178,9 +179,9 @@ static inline void rc_update_bit_1(struct rc *rc, uint16_t *p)
}
/* Called 4 times in unlzma loop */
-static int INIT rc_get_bit(struct rc *rc, uint16_t *p, int *symbol)
+static int INIT rc_get_bit(struct rc *rc, uint16_t *p, int *symbol, void(*error)(char *x))
{
- if (rc_is_bit_0(rc, p)) {
+ if (rc_is_bit_0(rc, p, error)) {
rc_update_bit_0(rc, p);
*symbol *= 2;
return 0;
@@ -192,9 +193,9 @@ static int INIT rc_get_bit(struct rc *rc, uint16_t *p, int *symbol)
}
/* Called once */
-static inline int INIT rc_direct_bit(struct rc *rc)
+static inline int INIT rc_direct_bit(struct rc *rc , void(*error)(char *x))
{
- rc_normalize(rc);
+ rc_normalize(rc, error);
rc->range >>= 1;
if (rc->code >= rc->range) {
rc->code -= rc->range;
@@ -205,13 +206,14 @@ static inline int INIT rc_direct_bit(struct rc *rc)
/* Called twice */
static inline void INIT
-rc_bit_tree_decode(struct rc *rc, uint16_t *p, int num_levels, int *symbol)
+rc_bit_tree_decode(struct rc *rc, uint16_t *p, int num_levels, int *symbol,
+ void(*error)(char *x))
{
int i = num_levels;
*symbol = 1;
while (i--)
- rc_get_bit(rc, p + *symbol, symbol);
+ rc_get_bit(rc, p + *symbol, symbol, error);
*symbol -= 1 << num_levels;
}
@@ -348,7 +350,8 @@ static inline void INIT copy_bytes(struct writer *wr,
static inline void INIT process_bit0(struct writer *wr, struct rc *rc,
struct cstate *cst, uint16_t *p,
int pos_state, uint16_t *prob,
- int lc, uint32_t literal_pos_mask) {
+ int lc, uint32_t literal_pos_mask,
+ void(*error)(char *x)) {
int mi = 1;
rc_update_bit_0(rc, prob);
prob = (p + LZMA_LITERAL +
@@ -366,7 +369,7 @@ static inline void INIT process_bit0(struct writer *wr, struct rc *rc,
match_byte <<= 1;
bit = match_byte & 0x100;
prob_lit = prob + 0x100 + bit + mi;
- if (rc_get_bit(rc, prob_lit, &mi)) {
+ if (rc_get_bit(rc, prob_lit, &mi, error)) {
if (!bit)
break;
} else {
@@ -377,7 +380,7 @@ static inline void INIT process_bit0(struct writer *wr, struct rc *rc,
}
while (mi < 0x100) {
uint16_t *prob_lit = prob + mi;
- rc_get_bit(rc, prob_lit, &mi);
+ rc_get_bit(rc, prob_lit, &mi, error);
}
write_byte(wr, mi);
if (cst->state < 4)
@@ -390,7 +393,8 @@ static inline void INIT process_bit0(struct writer *wr, struct rc *rc,
static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
struct cstate *cst, uint16_t *p,
- int pos_state, uint16_t *prob) {
+ int pos_state, uint16_t *prob,
+ void(*error)(char *x)) {
int offset;
uint16_t *prob_len;
int num_bits;
@@ -398,7 +402,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
rc_update_bit_1(rc, prob);
prob = p + LZMA_IS_REP + cst->state;
- if (rc_is_bit_0(rc, prob)) {
+ if (rc_is_bit_0(rc, prob, error)) {
rc_update_bit_0(rc, prob);
cst->rep3 = cst->rep2;
cst->rep2 = cst->rep1;
@@ -408,13 +412,13 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
} else {
rc_update_bit_1(rc, prob);
prob = p + LZMA_IS_REP_G0 + cst->state;
- if (rc_is_bit_0(rc, prob)) {
+ if (rc_is_bit_0(rc, prob, error)) {
rc_update_bit_0(rc, prob);
prob = (p + LZMA_IS_REP_0_LONG
+ (cst->state <<
LZMA_NUM_POS_BITS_MAX) +
pos_state);
- if (rc_is_bit_0(rc, prob)) {
+ if (rc_is_bit_0(rc, prob, error)) {
rc_update_bit_0(rc, prob);
cst->state = cst->state < LZMA_NUM_LIT_STATES ?
@@ -429,13 +433,13 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
rc_update_bit_1(rc, prob);
prob = p + LZMA_IS_REP_G1 + cst->state;
- if (rc_is_bit_0(rc, prob)) {
+ if (rc_is_bit_0(rc, prob, error)) {
rc_update_bit_0(rc, prob);
distance = cst->rep1;
} else {
rc_update_bit_1(rc, prob);
prob = p + LZMA_IS_REP_G2 + cst->state;
- if (rc_is_bit_0(rc, prob)) {
+ if (rc_is_bit_0(rc, prob, error)) {
rc_update_bit_0(rc, prob);
distance = cst->rep2;
} else {
@@ -453,7 +457,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
}
prob_len = prob + LZMA_LEN_CHOICE;
- if (rc_is_bit_0(rc, prob_len)) {
+ if (rc_is_bit_0(rc, prob_len, error)) {
rc_update_bit_0(rc, prob_len);
prob_len = (prob + LZMA_LEN_LOW
+ (pos_state <<
@@ -463,7 +467,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
} else {
rc_update_bit_1(rc, prob_len);
prob_len = prob + LZMA_LEN_CHOICE_2;
- if (rc_is_bit_0(rc, prob_len)) {
+ if (rc_is_bit_0(rc, prob_len, error)) {
rc_update_bit_0(rc, prob_len);
prob_len = (prob + LZMA_LEN_MID
+ (pos_state <<
@@ -479,7 +483,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
}
}
- rc_bit_tree_decode(rc, prob_len, num_bits, &len);
+ rc_bit_tree_decode(rc, prob_len, num_bits, &len, error);
len += offset;
if (cst->state < 4) {
@@ -494,7 +498,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
<< LZMA_NUM_POS_SLOT_BITS);
rc_bit_tree_decode(rc, prob,
LZMA_NUM_POS_SLOT_BITS,
- &pos_slot);
+ &pos_slot, error);
if (pos_slot >= LZMA_START_POS_MODEL_INDEX) {
int i, mi;
num_bits = (pos_slot >> 1) - 1;
@@ -507,7 +511,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
num_bits -= LZMA_NUM_ALIGN_BITS;
while (num_bits--)
cst->rep0 = (cst->rep0 << 1) |
- rc_direct_bit(rc);
+ rc_direct_bit(rc, error);
prob = p + LZMA_ALIGN;
cst->rep0 <<= LZMA_NUM_ALIGN_BITS;
num_bits = LZMA_NUM_ALIGN_BITS;
@@ -515,7 +519,7 @@ static inline void INIT process_bit1(struct writer *wr, struct rc *rc,
i = 1;
mi = 1;
while (num_bits--) {
- if (rc_get_bit(rc, prob + mi, &mi))
+ if (rc_get_bit(rc, prob + mi, &mi, error))
cst->rep0 |= i;
i <<= 1;
}
@@ -537,7 +541,7 @@ STATIC int INIT unlzma(unsigned char *buf, int in_len,
int(*flush)(void*, unsigned int),
unsigned char *output,
int *posp,
- void(*error_fn)(char *x)
+ void(*error)(char *x)
)
{
struct lzma_header header;
@@ -553,8 +557,6 @@ STATIC int INIT unlzma(unsigned char *buf, int in_len,
unsigned char *inbuf;
int ret = -1;
- set_error_fn(error_fn);
-
if (buf)
inbuf = buf;
else
@@ -577,7 +579,7 @@ STATIC int INIT unlzma(unsigned char *buf, int in_len,
for (i = 0; i < sizeof(header); i++) {
if (rc.ptr >= rc.buffer_end)
- rc_read(&rc);
+ rc_read(&rc, error);
((unsigned char *)&header)[i] = *rc.ptr++;
}
@@ -622,17 +624,17 @@ STATIC int INIT unlzma(unsigned char *buf, int in_len,
for (i = 0; i < num_probs; i++)
p[i] = (1 << RC_MODEL_TOTAL_BITS) >> 1;
- rc_init_code(&rc);
+ rc_init_code(&rc, error);
while (get_pos(&wr) < header.dst_size) {
int pos_state = get_pos(&wr) & pos_state_mask;
uint16_t *prob = p + LZMA_IS_MATCH +
(cst.state << LZMA_NUM_POS_BITS_MAX) + pos_state;
- if (rc_is_bit_0(&rc, prob))
+ if (rc_is_bit_0(&rc, prob, error))
process_bit0(&wr, &rc, &cst, p, pos_state, prob,
- lc, literal_pos_mask);
+ lc, literal_pos_mask, error);
else {
- process_bit1(&wr, &rc, &cst, p, pos_state, prob);
+ process_bit1(&wr, &rc, &cst, p, pos_state, prob, error);
if (cst.rep0 == 0)
break;
}
--
1.6.3.3
^ permalink raw reply related
* Re: Squashfs performance (3.3 vs 4.0 in mainline)
From: Phillip Lougher @ 2009-12-11 11:55 UTC (permalink / raw)
To: Simon Kagstrom; +Cc: linux-embedded, Phillip Lougher
In-Reply-To: <20091210150243.2ed788f4@marrow.netinsight.se>
On Thu, Dec 10, 2009 at 2:02 PM, Simon Kagstrom
<simon.kagstrom@netinsight.net> wrote:
> Squashfs works well, but I've noted a performance regression compared
> to the out-of-tree squashfs 3.3 which we ran on 2.6.23. The new kernel
> is faster until the squashfs root filesystem is mounted, but after that
> it goes downhill. Especially mounting takes a lot more time with the
> new:
> (The first column is the time difference compared to last timestamp).
> Are there any general changes which could have caused this difference?
>
There's nothing which should have caused such a slow-down in Squashfs.
Unfortunately, you're comparing Squashfs 3.3 on 2.6.23, with Squashfs
4.0 on 2.6.31, there's a huge amount of kernel differences between the
two. Anything broken in your 2.6.31 kernel (caching, memory handling,
interrupts etc.) could be the underlying cause.
Personally, I would try and isolate the problem more before blaming
Squashfs. A couple of things which you should try:
1. Try a different filesystem, i.e. cramfs or ext2, on both 2.6.23 and
2.6.31. Do you get the same slowdown? Preferable to use cramfs, as
this has similar compute intensive decompression overhead.
2. Enable tracing in Squashfs and send the output. Squashfs 3.3/4.0
should behave the same way. This will pinpoint such issues.
3. Do some compute intensive, and memory/IO intensive application
benchmarking, on both your 2.6.23 and 2.6.31 kernels.
4. Test Squashfs 3.3 in your 2.6.31 kernel. Do you get the same
slowdown? If the above tests still tend to show Squashfs 4.0 is the
issue, I'm prepared to produce a Squashfs 3.3 patch for 2.6.31 for
you.
>
> I've tried with various mksquashfs options, and -noI makes it somewhat
> faster (and larger), but not near the old version.
>
OK. This shows less compression makes it go faster. If you have a
slow CPU this is to be expected, and would happen with or without any
underlying problems somewhere.
Phillip
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Pavel Machek @ 2009-12-13 13:19 UTC (permalink / raw)
To: Linus Walleij; +Cc: Mark Brown, cbou, dwmw2, LKML, linux-embedded
In-Reply-To: <A6D19A13FE030A409EC4362C172E091F0E0A1FA1@eseldmw101.eemea.ericsson.se>
> One of the things we're facing is Android, which has
> its userspace in plain Java JNI at the end of this link:
> http://android.git.kernel.org/?p=platform/frameworks/base.git;a=blob;f=s
> ervices/jni/com_android_server_BatteryService.cpp;h=8e7cadc6b680fc420d34
> 1faa094c71922946fdab;hb=HEAD
>
> If you browse down to line 275 you can see it parse the sysfs
> attribute "capacity", then this propagates up to the battery
> status indicator on *all* Android phones out there. So if
> you want to run Android unmodified, this is what you need to
> provide. They are effectively using the power sysfs as
> their hardware abstraction layer in this case.
>
> Note: I'm not claiming that Android is doing it right or that
> we can't modify this code or so, it's just that this is the way
> a few million Android devices out there are actually doing it.
Users can't modify them, so they don't count...
...but then there are all the systems that rely on /proc/apm
emulation, like openembedded popular on sharp zaurus...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Pavel Machek @ 2009-12-13 13:24 UTC (permalink / raw)
To: Mark Brown
Cc: Linus Walleij, cbou, dwmw2, LKML, linux-embedded, Brian Swetland,
rpurdie, lenz, Dirk, arminlitzel, Cyril Hrubis, thommycheck,
linux-arm-kernel, dbaryshkov, omegamoon, eric.y.miao, utx,
zaurus-devel
In-Reply-To: <20091207165628.GA24981@rakim.wolfsonmicro.main>
Hi!
> > If you browse down to line 275 you can see it parse the sysfs
> > attribute "capacity", then this propagates up to the battery
> > status indicator on *all* Android phones out there. So if
> > you want to run Android unmodified, this is what you need to
> > provide. They are effectively using the power sysfs as
> > their hardware abstraction layer in this case.
>
> Oh dear. Using the power sysfs as the hardware abstraction seems
> perfectly reasonable but assuming that a given battery driver is going
> to have this level of information doesn't match up with an awful lot of
> actual charger hardware. My main concern here is that battery
> performance monitoring has no pressing need to be in kernel and that
> pushing it into the kernel creates a barrier to implementing more
> advanced schemes in userspace, which is especially serious given how
> involved this needs to be in order to be accurate.
Well, kernel provides /proc/apm emulation and many systems still rely
on it. So it would be nice to provide something halfway-decent there.
Plus you need to shutdown/suspend machine on battery critical. That
has to be in kernel and already needs those tricky parts.
(Sharp got it wrong in collie kernel, and you get 5hours instead of 10
with old battery :-(().
> I'm not sure how familiar you are with the issues surrounding trying to
> do a voltage to charge mapping for a battery but it's much more complex
> than a simple table if you want to get it accurate. There's a lot
> of
Well... current zaurus kernels use _huge_ table that maps voltage to
battery %... and that table is linear function :-(.
Do you have some papers on that?
> dependence on particular operating conditions and things do change as
> the batteries age. There are systems out there that do the work
> required to gather the information in hardware and it's definitely good
> to report the information from them but that doesn't mean it's a good
> idea to try to synthesise the information for other systems.
So... on zaurus I plan to:
1) provide better voltage -- %age map
2) estimate current
3) estimate internal battery resistance as constant
4) estimate internal battery volltage using ohm's law and base %age
estmate on that.
Now... I realize that internal resistance depends on charge left. Nasty
but probably can be ignored. Then it depends on temperature. Does
anyone have better idea how?
Then... I need a way to measure internal resistance. I know it is in
200mOhm to 400mOhm range, on my device. Is there easy way to measure
it more accurately?
Pavel
#!/bin/bash
#
# Copyright 2009 Pavel Machek <pavel@ucw.cz>, GPLv2
#
getval() {
SETTLETIME=5
echo Run this on idle, unplugged system, with expansion cards
echo removed and backlight enabled
echo
echo 1 > /sys/class/backlight/corgi?bl/brightness
echo Backlight 1, waiting for power to settle
sleep $SETTLETIME
VBMIN=`cat /sys/class/power*/*battery/voltage_now`
VBMIN=$[$VBMIN/1000]
echo Voltage = $VBMIN mV
echo
echo 47 > /sys/class/backlight/corgi?bl/brightness
echo Backlight 47, waiting for power to settle
sleep $SETTLETIME
VBMAX=`cat /sys/class/power*/*battery/voltage_now`
VBMAX=$[$VBMAX/1000]
echo Voltage = $VBMAX mV
echo 1 > /sys/class/backlight/corgi?bl/brightness
}
fake1() {
# Very old 1000mAh battery from collie: 703 mOhm
VBMIN=3638
VBMAX=3543
}
fake2() {
# Old 2000mAh battery, nearly charged, 4C: 274 mOhm
VBMIN=3732
VBMAX=3695
}
fake3() {
# Same old 2000mAh battery, nearly charged, 4C: 140 mOhm
# temp: 155.
VBMIN=3714
VBMAX=3695
# Next try: temp 151 -- little warmer: 422 mOhm.
# Next try: temp 151 -- little warmer: 1266 mOhm.
# Next try: temp 148 -- getting warmer: 281 mOhm.
# Next try: temp 148 -- getting warmer, full load: 422 mOhm.
# Next try: temp 148 -- getting warmer, full load: 140 mOhm.
# Next try: temp 148 -- getting warmer, full load: 422 mOhm.
# Next try: temp 138 -- getting warmer, full load: 422 mOhm.
# Next try: temp 139 -- getting warmer, full load: 422 mOhm.
# Next try: temp 136 -- getting warmer, full load: 562 mOhm.
# Next try: temp 132 -- getting warmer, full load: 703 mOhm.
# Next try: temp 132 -- getting warmer, full load: 281 mOhm.
# Next try: temp 134 -- getting warmer, full load: 281 mOhm.
# Next try: temp 134 -- getting warmer, full load: 562 mOhm.
# Next try: temp 129 -- getting warmer, full load: 562 mOhm.
# hugh, I''m getting n*140, wtf?
# ...voltmeters have sensitivity limits...
# temp 118 -- metro, venku zima -- full load: 281 mOhm.
# temp 118 -- metro, venku zima, baterie poloprazdna -- full load: 281 mOhm.
# temp 120 -- metro, venku zima, baterie poloprazdna -- full load: 281 mOhm.
# temp 120 -- metro, venku zima, baterie poloprazdna -- full load: 281 mOhm.
# temp 120 -- metro, venku zima, baterie poloprazdna -- full load: 414 mOhm.
# temp 120 -- metro, venku zima, baterie poloprazdna -- full load: 555 mOhm.
# temp 120 -- metro, venku zima, baterie poloprazdna -- full load: 422 mOhm.
# temp 124 -- metro, venku zima, baterie poloprazdna -- full load: 422 mOhm.
# temp 127 -- metro, venku zima, baterie poloprazdna -- full load: 422 mOhm.
}
getval
#fake1
BLDIFF=135
echo Assuming $BLDIFF mA difference
echo $VBMIN $VBMAX $BLDIFF
# With RESIST in mOhm
# VINSIDE = VBMIN + RESIST * BASECURRENT / 1000
# VINSIDE = VBMAX + RESIST * (BASECURRENT + BLDIFF) / 1000
# Therefore
# RESIST = 1000*(VBMIN-VBMAX)/BLDIFF
RESIST=$[(1000*($VBMIN-$VBMAX))/$BLDIFF]
echo Resistance is $RESIST mOhm
BASECURRENT=280
echo This should be equal:
echo During low backlight $[$VBMIN + ($RESIST * $BASECURRENT) / 1000] mV
echo During high backlight $[$VBMAX + ($RESIST * ($BASECURRENT + $BLDIFF)) / 1000] mV
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* Re: [PATCH 0/9] Squashfs: Add support for LZMA compressed filesystems
From: Arkadiusz Miśkiewicz @ 2009-12-13 20:13 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-fsdevel, linux-embedded, linux-fsdevel
In-Reply-To: <4B1E93E5.8090104@lougher.demon.co.uk>
Phillip Lougher wrote:
> Peter Korsgaard wrote:
>> Nice, do you also have an mksquashfs with lzma support?
>>
>
> Yes, in Squashfs CVS (http://sourceforge.net/projects/squashfs/develop).
Any reason for not using much nicer xz library instead of lzma sdk ?
http://tukaani.org/xz/
> Phillip
--
Arkadiusz Mi≈õkiewicz PLD/Linux Team
arekm / maven.pl http://ftp.pld-linux.org/
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Mark Brown @ 2009-12-14 11:50 UTC (permalink / raw)
To: Pavel Machek; +Cc: Linus Walleij, cbou, dwmw2, LKML, linux-embedded
In-Reply-To: <20091213131921.GA1437@ucw.cz>
On Sun, Dec 13, 2009 at 02:19:22PM +0100, Pavel Machek wrote:
> ...but then there are all the systems that rely on /proc/apm
> emulation, like openembedded popular on sharp zaurus...
OpenEmbedded is a meta-distribution so doesn't use any particular
software here - I suspect you're referring to things like the GPE stack
which isn't so actively developed these days.
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Pavel Machek @ 2009-12-14 11:58 UTC (permalink / raw)
To: Mark Brown; +Cc: Linus Walleij, cbou, dwmw2, LKML, linux-embedded
In-Reply-To: <20091214115023.GA22388@rakim.wolfsonmicro.main>
On Mon 2009-12-14 11:50:24, Mark Brown wrote:
> On Sun, Dec 13, 2009 at 02:19:22PM +0100, Pavel Machek wrote:
>
> > ...but then there are all the systems that rely on /proc/apm
> > emulation, like openembedded popular on sharp zaurus...
>
> OpenEmbedded is a meta-distribution so doesn't use any particular
> software here - I suspect you're referring to things like the GPE stack
> which isn't so actively developed these days.
Both Opie and GPE have the problem IIRC...
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Mark Brown @ 2009-12-14 12:12 UTC (permalink / raw)
To: Pavel Machek
Cc: Linus Walleij, cbou, dwmw2, LKML, linux-embedded, Brian Swetland,
rpurdie, lenz, Dirk, arminlitzel, Cyril Hrubis, thommycheck,
linux-arm-kernel, dbaryshkov, omegamoon, eric.y.miao, utx,
zaurus-devel
In-Reply-To: <20091213132413.GB1437@ucw.cz>
On Sun, Dec 13, 2009 at 02:24:14PM +0100, Pavel Machek wrote:
> > actual charger hardware. My main concern here is that battery
> > performance monitoring has no pressing need to be in kernel and that
> > pushing it into the kernel creates a barrier to implementing more
> > advanced schemes in userspace, which is especially serious given how
> > involved this needs to be in order to be accurate.
> Well, kernel provides /proc/apm emulation and many systems still rely
> on it. So it would be nice to provide something halfway-decent there.
Unfortunately that's really painful in kernel since you really need to
do state tracking over reboots, and even if you do that it's really
not trivial.
> Plus you need to shutdown/suspend machine on battery critical. That
> has to be in kernel and already needs those tricky parts.
Power failure detection based on voltage drop is much more reasonable
but it's a very different thing to general battery capacity estimation.
Normally you'd want to do the power failure detection separately anyway,
monitoring the system supply voltage rather than the battery voltage.
Supply failure is not only an issue in battery operation, it's also an
issue for example in systems systems powered over USB which may be
drawing more than the 500mA that USB delivers and need to supplement the
USB supply with the battery.
> > I'm not sure how familiar you are with the issues surrounding trying to
> > do a voltage to charge mapping for a battery but it's much more complex
> > than a simple table if you want to get it accurate. There's a lot
> > of
> Well... current zaurus kernels use _huge_ table that maps voltage to
> battery %... and that table is linear function :-(.
> Do you have some papers on that?
Something like "Measure Battery Capacity Precisely in Medical Design"
by Bernd Krafthoefer in Power Electronics Technology Jan 2005 might be
useful here.
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Mark Brown @ 2009-12-14 12:14 UTC (permalink / raw)
To: Pavel Machek; +Cc: Linus Walleij, cbou, dwmw2, LKML, linux-embedded
In-Reply-To: <20091214115826.GA2280@elf.ucw.cz>
On Mon, Dec 14, 2009 at 12:58:26PM +0100, Pavel Machek wrote:
> On Mon 2009-12-14 11:50:24, Mark Brown wrote:
> > OpenEmbedded is a meta-distribution so doesn't use any particular
> > software here - I suspect you're referring to things like the GPE stack
> > which isn't so actively developed these days.
> Both Opie and GPE have the problem IIRC...
Yeah, if you're working on older systems like that you'll probably have
to do some userspace work as well as kernel work.
^ permalink raw reply
* Re: [POWER] battery calibration parameters from sysfs
From: Pavel Machek @ 2009-12-14 21:22 UTC (permalink / raw)
To: Mark Brown
Cc: Linus Walleij, cbou, dwmw2, LKML, linux-embedded, Brian Swetland,
rpurdie, lenz, Dirk, arminlitzel, Cyril Hrubis, thommycheck,
linux-arm-kernel, dbaryshkov, omegamoon, eric.y.miao, utx,
zaurus-devel
In-Reply-To: <20091214121247.GB22388@rakim.wolfsonmicro.main>
Hi!
> > > I'm not sure how familiar you are with the issues surrounding trying to
> > > do a voltage to charge mapping for a battery but it's much more complex
> > > than a simple table if you want to get it accurate. There's a lot
> > > of
>
> > Well... current zaurus kernels use _huge_ table that maps voltage to
> > battery %... and that table is linear function :-(.
>
> > Do you have some papers on that?
>
> Something like "Measure Battery Capacity Precisely in Medical Design"
> by Bernd Krafthoefer in Power Electronics Technology Jan 2005 might be
> useful here.
Well, that would really require extensive hardware modifications: it
needs _way_ more accurate ADCs on battery for a start, and a way to
generate huge load.
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* [PATCH 1/3] bzip2/lzma/gzip: pre-boot malloc doesn't return NULL on failure
From: Phillip Lougher @ 2009-12-14 21:45 UTC (permalink / raw)
To: linux-kernel, linux-embedded, akpm; +Cc: phillip, mingo, albin.tonnerre
The trivial malloc implementation used in the pre-boot environment by the
decompressors returns a bad pointer on failure (falling through after
calling error). This is doubly wrong - the callers expect malloc to
return NULL on failure, second the error function is intended to be
used by the decompressors to propagate errors to *their* callers. The
decompressors have no access to any state set by the error function.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
include/linux/decompress/mm.h | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/decompress/mm.h b/include/linux/decompress/mm.h
index 12ff8c3..5032b9a 100644
--- a/include/linux/decompress/mm.h
+++ b/include/linux/decompress/mm.h
@@ -25,7 +25,7 @@ static void *malloc(int size)
void *p;
if (size < 0)
- error("Malloc error");
+ return NULL;
if (!malloc_ptr)
malloc_ptr = free_mem_ptr;
@@ -35,7 +35,7 @@ static void *malloc(int size)
malloc_ptr += size;
if (free_mem_end_ptr && malloc_ptr >= free_mem_end_ptr)
- error("Out of memory");
+ return NULL;
malloc_count++;
return p;
--
1.6.3.3
^ permalink raw reply related
* [PATCH 0/3] bzip2/lzma/gzip/initramfs: NULL pointer bugs and missing error checks
From: Phillip Lougher @ 2009-12-14 21:45 UTC (permalink / raw)
To: linux-kernel, linux-embedded, akpm; +Cc: phillip, mingo, albin.tonnerre
Hi,
The following patches fix up a number of NULL pointer bugs amd missing
error checks.
Thanks
Phillip
bzip2/lzma/gzip: pre-boot malloc doesn't return NULL on failure
bzip2: Add missing checks for malloc returning NULL
initramfs: add missing decompressor error check
---
include/linux/decompress/mm.h | 4 ++--
init/initramfs.c | 10 ++++++----
lib/decompress_bunzip2.c | 10 +++++++++-
3 files changed, 17 insertions(+), 7 deletions(-)
^ permalink raw reply
* [PATCH 2/3] bzip2: Add missing checks for malloc returning NULL
From: Phillip Lougher @ 2009-12-14 21:45 UTC (permalink / raw)
To: linux-kernel, linux-embedded, akpm; +Cc: phillip, mingo, albin.tonnerre
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
lib/decompress_bunzip2.c | 10 +++++++++-
1 files changed, 9 insertions(+), 1 deletions(-)
diff --git a/lib/decompress_bunzip2.c b/lib/decompress_bunzip2.c
index 600f473..a6b836b 100644
--- a/lib/decompress_bunzip2.c
+++ b/lib/decompress_bunzip2.c
@@ -637,6 +637,8 @@ static int INIT start_bunzip(struct bunzip_data **bdp, void *inbuf, int len,
/* Allocate bunzip_data. Most fields initialize to zero. */
bd = *bdp = malloc(i);
+ if (!bd)
+ return RETVAL_OUT_OF_MEMORY;
memset(bd, 0, sizeof(struct bunzip_data));
/* Setup input buffer */
bd->inbuf = inbuf;
@@ -664,6 +666,8 @@ static int INIT start_bunzip(struct bunzip_data **bdp, void *inbuf, int len,
bd->dbufSize = 100000*(i-BZh0);
bd->dbuf = large_malloc(bd->dbufSize * sizeof(int));
+ if (!bd->dbuf)
+ return RETVAL_OUT_OF_MEMORY;
return RETVAL_OK;
}
@@ -686,7 +690,7 @@ STATIC int INIT bunzip2(unsigned char *buf, int len,
if (!outbuf) {
error("Could not allocate output bufer");
- return -1;
+ return RETVAL_OUT_OF_MEMORY;
}
if (buf)
inbuf = buf;
@@ -694,6 +698,7 @@ STATIC int INIT bunzip2(unsigned char *buf, int len,
inbuf = malloc(BZIP2_IOBUF_SIZE);
if (!inbuf) {
error("Could not allocate input bufer");
+ i = RETVAL_OUT_OF_MEMORY;
goto exit_0;
}
i = start_bunzip(&bd, inbuf, len, fill);
@@ -720,11 +725,14 @@ STATIC int INIT bunzip2(unsigned char *buf, int len,
} else if (i == RETVAL_UNEXPECTED_OUTPUT_EOF) {
error("Compressed file ends unexpectedly");
}
+ if (!bd)
+ goto exit_1;
if (bd->dbuf)
large_free(bd->dbuf);
if (pos)
*pos = bd->inbufPos;
free(bd);
+exit_1:
if (!buf)
free(inbuf);
exit_0:
--
1.6.3.3
^ permalink raw reply related
* [PATCH 3/3] initramfs: add missing decompressor error check
From: Phillip Lougher @ 2009-12-14 21:45 UTC (permalink / raw)
To: linux-kernel, linux-embedded, akpm; +Cc: phillip, mingo, albin.tonnerre
The decompressors return error by calling a supplied error function, and/or
by returning an error return value. The initramfs code, however, fails to
check the exit code returned by the decompressor, and only checks the error
status set by calling the error function.
This patch adds a return code check and calls the error function.
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
---
init/initramfs.c | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/init/initramfs.c b/init/initramfs.c
index 4c00edc..b37d34b 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -413,7 +413,7 @@ static unsigned my_inptr; /* index of next byte to be processed in inbuf */
static char * __init unpack_to_rootfs(char *buf, unsigned len)
{
- int written;
+ int written, res;
decompress_fn decompress;
const char *compress_name;
static __initdata char msg_buf[64];
@@ -445,10 +445,12 @@ static char * __init unpack_to_rootfs(char *buf, unsigned len)
}
this_header = 0;
decompress = decompress_method(buf, len, &compress_name);
- if (decompress)
- decompress(buf, len, NULL, flush_buffer, NULL,
+ if (decompress) {
+ res = decompress(buf, len, NULL, flush_buffer, NULL,
&my_inptr, error);
- else if (compress_name) {
+ if (res)
+ error("decompressor failed");
+ } else if (compress_name) {
if (!message) {
snprintf(msg_buf, sizeof msg_buf,
"compression method %s not configured",
--
1.6.3.3
^ permalink raw reply related
* Re: [POWER] battery calibration parameters from sysfs
From: Aras Vaichas @ 2009-12-14 23:43 UTC (permalink / raw)
To: Pavel Machek
Cc: Mark Brown, Linus Walleij, cbou, dwmw2, LKML, linux-embedded,
Brian Swetland, rpurdie, lenz, Dirk, arminlitzel, Cyril Hrubis,
thommycheck, linux-arm-kernel, dbaryshkov, omegamoon, eric.y.miao,
utx, zaurus-devel
In-Reply-To: <20091214212257.GA9213@elf.ucw.cz>
2009/12/15 Pavel Machek <pavel@ucw.cz>:
> Hi!
>
>> > > I'm not sure how familiar you are with the issues surrounding trying to
>> > > do a voltage to charge mapping for a battery but it's much more complex
>> > > than a simple table if you want to get it accurate. There's a lot
>> > > of
>>
>> > Well... current zaurus kernels use _huge_ table that maps voltage to
>> > battery %... and that table is linear function :-(.
>>
>> > Do you have some papers on that?
>>
>> Something like "Measure Battery Capacity Precisely in Medical Design"
>> by Bernd Krafthoefer in Power Electronics Technology Jan 2005 might be
>> useful here.
>
> Well, that would really require extensive hardware modifications: it
> needs _way_ more accurate ADCs on battery for a start, and a way to
> generate huge load.
I've worked on several battery powered devices and battery monitoring
is very hard to achieve without a deep understanding of the chemistry,
circuitry and the algorithms. Large current loads can cause the
battery voltage to drop and therefore there can be a significant
voltage drop between measuring the quiescent and the loaded battery
voltages. I once did try to implement an algorithm which took into
account the different battery voltage during heavy current drain but
it's hard to know how to interpret that data. One of the RFID systems
I worked on could run the user interface for days on a low battery but
a single RFID read would cause the system to die due to the internal
resistance current drop ...
One of the easiest ways to monitor the battery is with a coulomb
counting method (accumulate current going into and out of the battery)
e.g. "LTC4150 - Coulomb Counter/Battery Gas Gauge"
http://www.linear.com/pc/productDetail.jsp?navId=H0,C1,C1003,C1037,C1134,P2354
As long as you aren't draining very large amounts of current from the
battery, the accumulated current measurement should be almost equal to
the amp-hour rating of the battery.
Unfortunately the simple coulomb counting chips have the disadvantage
that the CPU has to be running to accumulate the pulses. Of course,
the pulses could wake the CPU from a suspend mode, but I'd rather not
do that just to add "one" to a counter ...
Finally we decided on a fully integrated "smart" battery monitor
device like the DS2782
http://www.maxim-ic.com/quick_view2.cfm/qv_pk/4779. This chip has
Linux device driver support, I got the driver from Ryan Mallon at
Bluewater Systems. It was fairly complex to set up and calibrate but
we now use it in a commercial product.
I recommend having a look at the datasheet for the DS2782 to see what
methods they use for determining the remaining charge in a battery and
to get an idea of the complexity of the problem.
Good luck!
Aras Vaichas
^ permalink raw reply
* CELF Project proposal: UBIFS mount-time speedups
From: Tim Bird @ 2009-12-15 1:56 UTC (permalink / raw)
To: CE Linux Developers List, linux-embedded
Summary: Improve UBIFS mounting time
Proposer: Tim Bird
Description:
UBIFS is a next-generation flash-based file system for Linux.
It is a read/write file system, which supports compression
and has good performance. However, it's mount times are not
very good. This affects overall Linux boot time, for a UBIFS-based
embedded system.
The purpose of this project would be to investigate mount performance
issues, and try to resolve them. One suggestion is to keep the
bad block table on flash, instead of re-scanning it on every boot.
It is not known if algorithms to do this are covered by existing
flash management patents.
Related work:
* UBIFS
** http://en.wikipedia.org/wiki/UBIFS
** http://lwn.net/Articles/276025/
* UBI2
** see portions of http://www.mjmwired.net/kernel/Documentation/filesystems/ubifs.txt
* Mount times
** http://osl.sed.hu/wiki/ubifs/index.php/Mount_results
** http://elinux.org/images/f/f8/CELFJamboree30-UBIFS_update.pdf
Scope: This project might take about 4 months
Notes:
Toshiba may already be working on this project.
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox