* [PATCH 2/2] Squashfs: Add LZ4 compression configuration option
2013-07-22 2:21 Phillip Lougher
@ 2013-07-22 2:21 ` Phillip Lougher
0 siblings, 0 replies; 12+ messages in thread
From: Phillip Lougher @ 2013-07-22 2:21 UTC (permalink / raw)
To: linux-kernel, linux-fsdevel, linux-embedded; +Cc: Phillip Lougher
Add the glue code, and also update the documentation.
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
Documentation/filesystems/squashfs.txt | 8 ++++----
fs/squashfs/Kconfig | 15 +++++++++++++++
fs/squashfs/Makefile | 1 +
fs/squashfs/decompressor.c | 7 +++++++
fs/squashfs/decompressor.h | 4 ++++
5 files changed, 31 insertions(+), 4 deletions(-)
diff --git a/Documentation/filesystems/squashfs.txt b/Documentation/filesystems/squashfs.txt
index 403c090..e5274f8 100644
--- a/Documentation/filesystems/squashfs.txt
+++ b/Documentation/filesystems/squashfs.txt
@@ -2,10 +2,10 @@ SQUASHFS 4.0 FILESYSTEM
=======================
Squashfs is a compressed read-only filesystem for Linux.
-It uses zlib/lzo/xz compression to compress files, inodes and directories.
-Inodes in the system are very small and all blocks are packed to minimise
-data overhead. Block sizes greater than 4K are supported up to a maximum
-of 1Mbytes (default block size 128K).
+It uses zlib, lz4, lzo, or xz compression to compress files, inodes and
+directories. Inodes in the system are very small and all blocks are packed to
+minimise data overhead. Block sizes greater than 4K are supported up to a
+maximum of 1Mbytes (default block size 128K).
Squashfs is intended for general read-only filesystem use, for archival
use (i.e. in cases where a .tar.gz file may be used), and in constrained
diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index c70111e..257f934 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -48,6 +48,21 @@ config SQUASHFS_ZLIB
If unsure, say Y.
+config SQUASHFS_LZ4
+ bool "Include support for LZ4 compressed file systems"
+ depends on SQUASHFS
+ select LZ4_DECOMPRESS
+ help
+ Saying Y here includes support for reading Squashfs file systems
+ compressed with LZ4 compression. LZ4 compression is mainly
+ aimed at embedded systems with slower CPUs where the overheads
+ of zlib are too high.
+
+ LZ4 is not the standard compression used in Squashfs and so most
+ file systems will be readable without selecting this option.
+
+ If unsure, say N.
+
config SQUASHFS_LZO
bool "Include support for LZO compressed file systems"
depends on SQUASHFS
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index 110b047..4a80ca7 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -6,6 +6,7 @@ obj-$(CONFIG_SQUASHFS) += squashfs.o
squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o
squashfs-y += namei.o super.o symlink.o decompressor.o
squashfs-$(CONFIG_SQUASHFS_XATTR) += xattr.o xattr_id.o
+squashfs-$(CONFIG_SQUASHFS_LZ4) += lz4_wrapper.o
squashfs-$(CONFIG_SQUASHFS_LZO) += lzo_wrapper.o
squashfs-$(CONFIG_SQUASHFS_XZ) += xz_wrapper.o
squashfs-$(CONFIG_SQUASHFS_ZLIB) += zlib_wrapper.o
diff --git a/fs/squashfs/decompressor.c b/fs/squashfs/decompressor.c
index 3f6271d..ae60211 100644
--- a/fs/squashfs/decompressor.c
+++ b/fs/squashfs/decompressor.c
@@ -40,6 +40,12 @@ static const struct squashfs_decompressor squashfs_lzma_unsupported_comp_ops = {
NULL, NULL, NULL, LZMA_COMPRESSION, "lzma", 0
};
+#ifndef CONFIG_SQUASHFS_LZ4
+static const struct squashfs_decompressor squashfs_lz4_comp_ops = {
+ NULL, NULL, NULL, LZ4_COMPRESSION, "lz4", 0
+};
+#endif
+
#ifndef CONFIG_SQUASHFS_LZO
static const struct squashfs_decompressor squashfs_lzo_comp_ops = {
NULL, NULL, NULL, LZO_COMPRESSION, "lzo", 0
@@ -64,6 +70,7 @@ static const struct squashfs_decompressor squashfs_unknown_comp_ops = {
static const struct squashfs_decompressor *decompressor[] = {
&squashfs_zlib_comp_ops,
+ &squashfs_lz4_comp_ops,
&squashfs_lzo_comp_ops,
&squashfs_xz_comp_ops,
&squashfs_lzma_unsupported_comp_ops,
diff --git a/fs/squashfs/decompressor.h b/fs/squashfs/decompressor.h
index 330073e..a9fbdd2 100644
--- a/fs/squashfs/decompressor.h
+++ b/fs/squashfs/decompressor.h
@@ -52,6 +52,10 @@ static inline int squashfs_decompress(struct squashfs_sb_info *msblk,
extern const struct squashfs_decompressor squashfs_xz_comp_ops;
#endif
+#ifdef CONFIG_SQUASHFS_LZ4
+extern const struct squashfs_decompressor squashfs_lz4_comp_ops;
+#endif
+
#ifdef CONFIG_SQUASHFS_LZO
extern const struct squashfs_decompressor squashfs_lzo_comp_ops;
#endif
--
1.7.10.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 0/2] Squashfs: add LZ4 compression
@ 2014-11-27 8:00 Phillip Lougher
2014-11-27 8:00 ` [PATCH 1/2] Squashfs: add LZ4 compression support Phillip Lougher
` (5 more replies)
0 siblings, 6 replies; 12+ messages in thread
From: Phillip Lougher @ 2014-11-27 8:00 UTC (permalink / raw)
To: linux-kernel, linux-fsdevel
Cc: texstar, martin, guanx.bac, bruno, dave, szycha, blyons,
tokiclover, afm404, hugochevrain, Phillip Lougher
Hi,
The following is a repost of the Squashfs LZ4 compression patches
updated to the latest -rc kernel. These patches were originally sent
last year, but they did not receive much interest at the time.
These patches are being reposted by popular demand. I have received
many private and public requests to repost these patches and to get
them mainlined.
My intention is to submit them in the next kernel merge window.
If you want LZ4 support in Squashfs now is a good time to publically
support the inclusion of these patches.
Phillip Lougher (2):
Squashfs: add LZ4 compression support
Squashfs: Add LZ4 compression configuration option
Documentation/filesystems/squashfs.txt | 8 +-
fs/squashfs/Kconfig | 15 ++++
fs/squashfs/Makefile | 1 +
fs/squashfs/decompressor.c | 7 ++
fs/squashfs/decompressor.h | 4 +
fs/squashfs/lz4_wrapper.c | 142 ++++++++++++++++++++++++++++++++
fs/squashfs/squashfs_fs.h | 1 +
7 files changed, 174 insertions(+), 4 deletions(-)
create mode 100644 fs/squashfs/lz4_wrapper.c
These patches are also available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/pkl/squashfs-lz4.git master
Squashfs-tools 4.3 already supports LZ4 compression.
Phillip
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 1/2] Squashfs: add LZ4 compression support
2014-11-27 8:00 [PATCH 0/2] Squashfs: add LZ4 compression Phillip Lougher
@ 2014-11-27 8:00 ` Phillip Lougher
2014-11-27 8:00 ` [PATCH 2/2] Squashfs: Add LZ4 compression configuration option Phillip Lougher
` (4 subsequent siblings)
5 siblings, 0 replies; 12+ messages in thread
From: Phillip Lougher @ 2014-11-27 8:00 UTC (permalink / raw)
To: linux-kernel, linux-fsdevel
Cc: texstar, martin, guanx.bac, bruno, dave, szycha, blyons,
tokiclover, afm404, hugochevrain, Phillip Lougher
Add support for reading file systems compressed with the
LZ4 compression algorithm.
This patch adds the LZ4 decompressor wrapper code.
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
fs/squashfs/lz4_wrapper.c | 142 +++++++++++++++++++++++++++++++++++++++++++++
fs/squashfs/squashfs_fs.h | 1 +
2 files changed, 143 insertions(+)
create mode 100644 fs/squashfs/lz4_wrapper.c
diff --git a/fs/squashfs/lz4_wrapper.c b/fs/squashfs/lz4_wrapper.c
new file mode 100644
index 0000000..c31e2bc
--- /dev/null
+++ b/fs/squashfs/lz4_wrapper.c
@@ -0,0 +1,142 @@
+/*
+ * Copyright (c) 2013, 2014
+ * Phillip Lougher <phillip@squashfs.org.uk>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ */
+
+#include <linux/buffer_head.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+#include <linux/lz4.h>
+
+#include "squashfs_fs.h"
+#include "squashfs_fs_sb.h"
+#include "squashfs.h"
+#include "decompressor.h"
+#include "page_actor.h"
+
+#define LZ4_LEGACY 1
+
+struct lz4_comp_opts {
+ __le32 version;
+ __le32 flags;
+};
+
+struct squashfs_lz4 {
+ void *input;
+ void *output;
+};
+
+
+static void *lz4_comp_opts(struct squashfs_sb_info *msblk,
+ void *buff, int len)
+{
+ struct lz4_comp_opts *comp_opts = buff;
+
+ /* LZ4 compressed filesystems always have compression options */
+ if (comp_opts == NULL || len < sizeof(*comp_opts))
+ return ERR_PTR(-EIO);
+
+ if (le32_to_cpu(comp_opts->version) != LZ4_LEGACY) {
+ /* LZ4 format currently used by the kernel is the 'legacy'
+ * format */
+ ERROR("Unknown LZ4 version\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ return NULL;
+}
+
+
+static void *lz4_init(struct squashfs_sb_info *msblk, void *buff)
+{
+ int block_size = max_t(int, msblk->block_size, SQUASHFS_METADATA_SIZE);
+ struct squashfs_lz4 *stream;
+
+ stream = kzalloc(sizeof(*stream), GFP_KERNEL);
+ if (stream == NULL)
+ goto failed;
+ stream->input = vmalloc(block_size);
+ if (stream->input == NULL)
+ goto failed2;
+ stream->output = vmalloc(block_size);
+ if (stream->output == NULL)
+ goto failed3;
+
+ return stream;
+
+failed3:
+ vfree(stream->input);
+failed2:
+ kfree(stream);
+failed:
+ ERROR("Failed to initialise LZ4 decompressor\n");
+ return ERR_PTR(-ENOMEM);
+}
+
+
+static void lz4_free(void *strm)
+{
+ struct squashfs_lz4 *stream = strm;
+
+ if (stream) {
+ vfree(stream->input);
+ vfree(stream->output);
+ }
+ kfree(stream);
+}
+
+
+static int lz4_uncompress(struct squashfs_sb_info *msblk, void *strm,
+ struct buffer_head **bh, int b, int offset, int length,
+ struct squashfs_page_actor *output)
+{
+ struct squashfs_lz4 *stream = strm;
+ void *buff = stream->input, *data;
+ int avail, i, bytes = length, res;
+ size_t dest_len = output->length;
+
+ for (i = 0; i < b; i++) {
+ avail = min(bytes, msblk->devblksize - offset);
+ memcpy(buff, bh[i]->b_data + offset, avail);
+ buff += avail;
+ bytes -= avail;
+ offset = 0;
+ put_bh(bh[i]);
+ }
+
+ res = lz4_decompress_unknownoutputsize(stream->input, length,
+ stream->output, &dest_len);
+ if (res)
+ return -EIO;
+
+ bytes = dest_len;
+ data = squashfs_first_page(output);
+ buff = stream->output;
+ while (data) {
+ if (bytes <= PAGE_CACHE_SIZE) {
+ memcpy(data, buff, bytes);
+ break;
+ }
+ memcpy(data, buff, PAGE_CACHE_SIZE);
+ buff += PAGE_CACHE_SIZE;
+ bytes -= PAGE_CACHE_SIZE;
+ data = squashfs_next_page(output);
+ }
+ squashfs_finish_page(output);
+
+ return dest_len;
+}
+
+const struct squashfs_decompressor squashfs_lz4_comp_ops = {
+ .init = lz4_init,
+ .comp_opts = lz4_comp_opts,
+ .free = lz4_free,
+ .decompress = lz4_uncompress,
+ .id = LZ4_COMPRESSION,
+ .name = "lz4",
+ .supported = 1
+};
diff --git a/fs/squashfs/squashfs_fs.h b/fs/squashfs/squashfs_fs.h
index 4b2beda..506f4ba 100644
--- a/fs/squashfs/squashfs_fs.h
+++ b/fs/squashfs/squashfs_fs.h
@@ -240,6 +240,7 @@ struct meta_index {
#define LZMA_COMPRESSION 2
#define LZO_COMPRESSION 3
#define XZ_COMPRESSION 4
+#define LZ4_COMPRESSION 5
struct squashfs_super_block {
__le32 s_magic;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 2/2] Squashfs: Add LZ4 compression configuration option
2014-11-27 8:00 [PATCH 0/2] Squashfs: add LZ4 compression Phillip Lougher
2014-11-27 8:00 ` [PATCH 1/2] Squashfs: add LZ4 compression support Phillip Lougher
@ 2014-11-27 8:00 ` Phillip Lougher
2014-11-27 13:37 ` [PATCH 0/2] Squashfs: add LZ4 compression Bruno Wolff III
` (3 subsequent siblings)
5 siblings, 0 replies; 12+ messages in thread
From: Phillip Lougher @ 2014-11-27 8:00 UTC (permalink / raw)
To: linux-kernel, linux-fsdevel
Cc: texstar, martin, guanx.bac, bruno, dave, szycha, blyons,
tokiclover, afm404, hugochevrain, Phillip Lougher
Add the glue code, and also update the documentation.
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
Documentation/filesystems/squashfs.txt | 8 ++++----
fs/squashfs/Kconfig | 15 +++++++++++++++
fs/squashfs/Makefile | 1 +
fs/squashfs/decompressor.c | 7 +++++++
fs/squashfs/decompressor.h | 4 ++++
5 files changed, 31 insertions(+), 4 deletions(-)
diff --git a/Documentation/filesystems/squashfs.txt b/Documentation/filesystems/squashfs.txt
index 403c090..e5274f8 100644
--- a/Documentation/filesystems/squashfs.txt
+++ b/Documentation/filesystems/squashfs.txt
@@ -2,10 +2,10 @@ SQUASHFS 4.0 FILESYSTEM
=======================
Squashfs is a compressed read-only filesystem for Linux.
-It uses zlib/lzo/xz compression to compress files, inodes and directories.
-Inodes in the system are very small and all blocks are packed to minimise
-data overhead. Block sizes greater than 4K are supported up to a maximum
-of 1Mbytes (default block size 128K).
+It uses zlib, lz4, lzo, or xz compression to compress files, inodes and
+directories. Inodes in the system are very small and all blocks are packed to
+minimise data overhead. Block sizes greater than 4K are supported up to a
+maximum of 1Mbytes (default block size 128K).
Squashfs is intended for general read-only filesystem use, for archival
use (i.e. in cases where a .tar.gz file may be used), and in constrained
diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index b6fa865..ffb093e 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -120,6 +120,21 @@ config SQUASHFS_ZLIB
If unsure, say Y.
+config SQUASHFS_LZ4
+ bool "Include support for LZ4 compressed file systems"
+ depends on SQUASHFS
+ select LZ4_DECOMPRESS
+ help
+ Saying Y here includes support for reading Squashfs file systems
+ compressed with LZ4 compression. LZ4 compression is mainly
+ aimed at embedded systems with slower CPUs where the overheads
+ of zlib are too high.
+
+ LZ4 is not the standard compression used in Squashfs and so most
+ file systems will be readable without selecting this option.
+
+ If unsure, say N.
+
config SQUASHFS_LZO
bool "Include support for LZO compressed file systems"
depends on SQUASHFS
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index 4132520..246a6f3 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -11,6 +11,7 @@ squashfs-$(CONFIG_SQUASHFS_DECOMP_SINGLE) += decompressor_single.o
squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI) += decompressor_multi.o
squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU) += decompressor_multi_percpu.o
squashfs-$(CONFIG_SQUASHFS_XATTR) += xattr.o xattr_id.o
+squashfs-$(CONFIG_SQUASHFS_LZ4) += lz4_wrapper.o
squashfs-$(CONFIG_SQUASHFS_LZO) += lzo_wrapper.o
squashfs-$(CONFIG_SQUASHFS_XZ) += xz_wrapper.o
squashfs-$(CONFIG_SQUASHFS_ZLIB) += zlib_wrapper.o
diff --git a/fs/squashfs/decompressor.c b/fs/squashfs/decompressor.c
index ac22fe7..505d6aa 100644
--- a/fs/squashfs/decompressor.c
+++ b/fs/squashfs/decompressor.c
@@ -41,6 +41,12 @@ static const struct squashfs_decompressor squashfs_lzma_unsupported_comp_ops = {
NULL, NULL, NULL, NULL, LZMA_COMPRESSION, "lzma", 0
};
+#ifndef CONFIG_SQUASHFS_LZ4
+static const struct squashfs_decompressor squashfs_lz4_comp_ops = {
+ NULL, NULL, NULL, LZ4_COMPRESSION, "lz4", 0
+};
+#endif
+
#ifndef CONFIG_SQUASHFS_LZO
static const struct squashfs_decompressor squashfs_lzo_comp_ops = {
NULL, NULL, NULL, NULL, LZO_COMPRESSION, "lzo", 0
@@ -65,6 +71,7 @@ static const struct squashfs_decompressor squashfs_unknown_comp_ops = {
static const struct squashfs_decompressor *decompressor[] = {
&squashfs_zlib_comp_ops,
+ &squashfs_lz4_comp_ops,
&squashfs_lzo_comp_ops,
&squashfs_xz_comp_ops,
&squashfs_lzma_unsupported_comp_ops,
diff --git a/fs/squashfs/decompressor.h b/fs/squashfs/decompressor.h
index af09853..a25713c 100644
--- a/fs/squashfs/decompressor.h
+++ b/fs/squashfs/decompressor.h
@@ -46,6 +46,10 @@ static inline void *squashfs_comp_opts(struct squashfs_sb_info *msblk,
extern const struct squashfs_decompressor squashfs_xz_comp_ops;
#endif
+#ifdef CONFIG_SQUASHFS_LZ4
+extern const struct squashfs_decompressor squashfs_lz4_comp_ops;
+#endif
+
#ifdef CONFIG_SQUASHFS_LZO
extern const struct squashfs_decompressor squashfs_lzo_comp_ops;
#endif
--
1.7.10.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
2014-11-27 8:00 [PATCH 0/2] Squashfs: add LZ4 compression Phillip Lougher
2014-11-27 8:00 ` [PATCH 1/2] Squashfs: add LZ4 compression support Phillip Lougher
2014-11-27 8:00 ` [PATCH 2/2] Squashfs: Add LZ4 compression configuration option Phillip Lougher
@ 2014-11-27 13:37 ` Bruno Wolff III
[not found] ` <CAJZww4gzTnbyB4jbFQc0N4G6GJ31pVwv-PNZtDMbUxXMoH-orQ@mail.gmail.com>
2014-12-12 7:44 ` Dave Vasilevsky
` (2 subsequent siblings)
5 siblings, 1 reply; 12+ messages in thread
From: Bruno Wolff III @ 2014-11-27 13:37 UTC (permalink / raw)
To: Phillip Lougher
Cc: linux-kernel, linux-fsdevel, texstar, martin, guanx.bac, dave,
szycha, blyons, tokiclover, afm404, hugochevrain
On Thu, Nov 27, 2014 at 08:00:47 +0000,
Phillip Lougher <phillip@squashfs.org.uk> wrote:
>
>My intention is to submit them in the next kernel merge window.
>If you want LZ4 support in Squashfs now is a good time to publically
>support the inclusion of these patches.
Fedora has been supporting LZ4 functionallity in mksquashfs and unsquashfs
for about 6 months (in F19+), but the Fedora kernel team won't add support
in the kernel until LZ4 support is accepted for the upstream kernel.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Fwd: [PATCH 0/2] Squashfs: add LZ4 compression
[not found] ` <CAJZww4gzTnbyB4jbFQc0N4G6GJ31pVwv-PNZtDMbUxXMoH-orQ@mail.gmail.com>
@ 2014-11-27 15:54 ` Marcin Szychowski
2014-11-27 16:55 ` Martin Vath
1 sibling, 0 replies; 12+ messages in thread
From: Marcin Szychowski @ 2014-11-27 15:54 UTC (permalink / raw)
To: linux-kernel, linux-fsdevel
---------- Forwarded message ----------
From: Marcin Szychowski <szycha@gmail.com>
Date: 2014-11-27 15:50 GMT+00:00
Subject: Re: [PATCH 0/2] Squashfs: add LZ4 compression
To: Bruno Wolff III <bruno@wolff.to>
Cc: Phillip Lougher <phillip@squashfs.org.uk>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
texstar@gmail.com, martin@mvath.de, guanx.bac@gmail.com,
dave@vasilevsky.ca, blyons@students.naropa.edu, tokiclover@gmail.com,
afm404@gmail.com, hugochevrain@gmail.com
2014-11-27 13:37 GMT+00:00 Bruno Wolff III <bruno@wolff.to>:
>
> On Thu, Nov 27, 2014 at 08:00:47 +0000,
> Phillip Lougher <phillip@squashfs.org.uk> wrote:
>>
>>
>> My intention is to submit them in the next kernel merge window.
>> If you want LZ4 support in Squashfs now is a good time to publically
>> support the inclusion of these patches.
>
>
> Fedora has been supporting LZ4 functionallity in mksquashfs and unsquashfs for about 6 months (in F19+), but the Fedora kernel team won't add support in the kernel until LZ4 support is accepted for the upstream kernel.
Hello everyone,
Squashfs is awesome piece of code. It’s potential usage pattern
analysis could turn into pretty decent PhD thesis – from simple
archival purposes through embedded systems, to dynamic nearly-instant
multi-server software deployment systems, and many others.
On the other hand, LZ4 is fantastic compression algorithm with
well-known list of advantages.
As I wrote to Phillip earlier, I have been using squashfs+lz4 / aufs
to speed up my laptop (and reclaim some space, too). That would not
be possible without LZ4: I could have gain some speed with LZO or disk
space with gzip or xz, but not both. LZ4 is comparable with LZO in
terms of decompression speed and with gzip in terms of compression
ratios. LZ4 is much faster in both compression and in particular
decompression speed than gzip (1-9) and xz (1-9). It is just the best
choice for many storage-related purposes (see ZFS on Linux
[https://github.com/zfsonlinux/zfs/commit/9759c60f1a1503e48dc5c45a209c3edd5758319f]
and [http://wiki.illumos.org/display/illumos/LZ4+Compression])
I don’t quite get why does anyone need to be convinced to merge
advantages of two fine pieces of code, since both have been already in
the mainline kernel for some time now. It’s like being forced to
choose between big car and safe car, or delicious food and
good-looking food.
It is not about forcing anyone to use something, it is about ease of choice.
Having squashfs with lz4 in mainline kernel sounds like next natural
step. Please do not hesitate to take it.
--
Regards,
Marcin Szychowski
+48.791460067
http://adastor.pl
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
[not found] ` <CAJZww4gzTnbyB4jbFQc0N4G6GJ31pVwv-PNZtDMbUxXMoH-orQ@mail.gmail.com>
2014-11-27 15:54 ` Fwd: " Marcin Szychowski
@ 2014-11-27 16:55 ` Martin Vath
1 sibling, 0 replies; 12+ messages in thread
From: Martin Vath @ 2014-11-27 16:55 UTC (permalink / raw)
To: Marcin Szychowski
Cc: Bruno Wolff III, Phillip Lougher, linux-kernel, linux-fsdevel,
texstar, Martin Vaeth, guanx.bac, dave, blyons, tokiclover,
afm404, hugochevrain
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1351 bytes --]
Hi:
> squashfs+lz4 / aufs to speed up my laptop
Squashfs proves its usefulness especially in combination
with aufs or overlayfs; I suppose that this combination
will become more popular with the spreading of overlayfs
(e.g. to save disk space etc.)
For users with such a combination, it is especially important
to get a fast (re)compression of huge directories.
The speed of LZ4 for compression in such a setting is really incredible.
Just for orientation for myself, I made a list of times/sizes
on some machines I had access to:
https://github.com/vaeth/squashmount/blob/master/compress.txt
(I do not claim that this is a scientific benchmark - just a
straightforward average over several runs).
The results for the kernel source and libreoffice are really
unbelievable, but I repeated them several times (and,
as you can see, for the kernel sources on different machines).
>From the user perspective, it is mainly important that
the files compressed in this way can be read by the kernel -
the decompression speed is here secondary, so I made no comparison
for this case, although lz4 is known to be very fast also
for decompression.
I would really like to see lz4 support included in the
squashfs kernel driver. I cannot imagine any negative consequences,
especially since squashfs and lz4 are in the kernel, anyway.
Sincerely
Martin Väth
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
2014-11-27 8:00 [PATCH 0/2] Squashfs: add LZ4 compression Phillip Lougher
` (2 preceding siblings ...)
2014-11-27 13:37 ` [PATCH 0/2] Squashfs: add LZ4 compression Bruno Wolff III
@ 2014-12-12 7:44 ` Dave Vasilevsky
2014-12-12 8:31 ` Anthony F McInerney
2014-12-12 10:27 ` Guan Xin
[not found] ` <CAGw_SrN6tZyXRx6pxkpnhWc2U5845HRbfzhOK5v9qkjWejFpmA@mail.gmail.com>
5 siblings, 1 reply; 12+ messages in thread
From: Dave Vasilevsky @ 2014-12-12 7:44 UTC (permalink / raw)
To: Phillip Lougher
Cc: linux-kernel, linux-fsdevel, texstar, martin, Guan Xin, bruno,
szycha, blyons, tokiclover, afm404, hugochevrain
I'd like to support the inclusion of these patches.
There's enough demand for LZ4-compressed squashfs that I received an
unsolicited patch adding support to my FUSE implementation,
squashfuse: https://github.com/vasi/squashfuse/pull/2
At least a few people are currently using squashfuse to mount their
LZ4-compressed squashfs images. Ideally these users would be able to
use the in-kernel squashfs for improved performance.
Cheers,
Dave Vasilevsky
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
2014-12-12 7:44 ` Dave Vasilevsky
@ 2014-12-12 8:31 ` Anthony F McInerney
0 siblings, 0 replies; 12+ messages in thread
From: Anthony F McInerney @ 2014-12-12 8:31 UTC (permalink / raw)
To: Dave Vasilevsky
Cc: Phillip Lougher, linux-kernel, linux-fsdevel, texstar, martin,
Guan Xin, bruno, szycha, blyons, tokiclover, hugochevrain
Thanks very much for keeping me in the loop.
I'd like to put a big +1 on this patch inclusion.
Looking forward to in-kernel support for LZ4 squashfs images.
Thanks
Anthony
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
2014-11-27 8:00 [PATCH 0/2] Squashfs: add LZ4 compression Phillip Lougher
` (3 preceding siblings ...)
2014-12-12 7:44 ` Dave Vasilevsky
@ 2014-12-12 10:27 ` Guan Xin
[not found] ` <CAGw_SrN6tZyXRx6pxkpnhWc2U5845HRbfzhOK5v9qkjWejFpmA@mail.gmail.com>
5 siblings, 0 replies; 12+ messages in thread
From: Guan Xin @ 2014-12-12 10:27 UTC (permalink / raw)
To: Phillip Lougher
Cc: linux-kernel, linux-fsdevel, texstar, martin, Bruno Wolff III,
dave, szycha, blyons, tokiclover, afm404, hugochevrain
Hi Philip,
Thank you for your work on squashfs and your persistent effort on mainlining
squashfs-lz4 support!
I've been using squashfs with lz4 compression (together with aufs by Okajima)
for my desktop, laptops, home router and bluetooth music player for about
half a year. There was not a single issue. All I got were reduced latency
compared to lzo compression, and drastically increased compression speed.
The later is important to me because my application requires re-mksquashfs of
the whole system in order to make software updates permanent.
lz4 support (especially with -Xhc) is not only helpful, but also a
valuable addition
to squashfs.
Regards,
Guan
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
[not found] ` <CAGw_SrN6tZyXRx6pxkpnhWc2U5845HRbfzhOK5v9qkjWejFpmA@mail.gmail.com>
@ 2014-12-12 15:56 ` Bruno Wolff III
2014-12-12 22:45 ` Phillip Lougher
0 siblings, 1 reply; 12+ messages in thread
From: Bruno Wolff III @ 2014-12-12 15:56 UTC (permalink / raw)
To: toki clover
Cc: Phillip Lougher, linux-kernel, linux-fsdevel, texstar, martin,
Guan Xin, dave, szycha, blyons, afm404, hugochevrain
On Fri, Dec 12, 2014 at 13:23:19 +0100,
toki clover <tokiclover@gmail.com> wrote:
>
>Now, I did not see any Linux FS devs activity/response to this... What
>a waste of time because if those patch don't make it for this merge
>window, rebasing/reposting will be, again, necessary.
The patches got pulled into linux-next. For XZ, the patches sat in
linux-next for a full release cycle. I don't know whether or not
if that is the plan for the lz4 patches.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/2] Squashfs: add LZ4 compression
2014-12-12 15:56 ` Bruno Wolff III
@ 2014-12-12 22:45 ` Phillip Lougher
0 siblings, 0 replies; 12+ messages in thread
From: Phillip Lougher @ 2014-12-12 22:45 UTC (permalink / raw)
To: Bruno Wolff III, toki clover
Cc: Phillip Lougher, linux-kernel, linux-fsdevel, texstar, martin,
Guan Xin, dave, szycha, blyons, afm404, hugochevrain
On 12/12/14 15:56, Bruno Wolff III wrote:
> On Fri, Dec 12, 2014 at 13:23:19 +0100,
> toki clover <tokiclover@gmail.com> wrote:
>>
>> Now, I did not see any Linux FS devs activity/response to this... What
>> a waste of time because if those patch don't make it for this merge
>> window, rebasing/reposting will be, again, necessary.
>
> The patches got pulled into linux-next. For XZ, the patches sat in linux-next for a full release cycle. I don't know whether or not if that is the plan for the lz4 patches.
> .
>
The plan is to send out the merge request this weekend.
I've been too busy this week with my main job to do the merge request,
sorry (I'll probably be working past midnight again tonight, the
weeks leading up to Christmas are always really busy).
Phillip
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2014-12-12 22:45 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-11-27 8:00 [PATCH 0/2] Squashfs: add LZ4 compression Phillip Lougher
2014-11-27 8:00 ` [PATCH 1/2] Squashfs: add LZ4 compression support Phillip Lougher
2014-11-27 8:00 ` [PATCH 2/2] Squashfs: Add LZ4 compression configuration option Phillip Lougher
2014-11-27 13:37 ` [PATCH 0/2] Squashfs: add LZ4 compression Bruno Wolff III
[not found] ` <CAJZww4gzTnbyB4jbFQc0N4G6GJ31pVwv-PNZtDMbUxXMoH-orQ@mail.gmail.com>
2014-11-27 15:54 ` Fwd: " Marcin Szychowski
2014-11-27 16:55 ` Martin Vath
2014-12-12 7:44 ` Dave Vasilevsky
2014-12-12 8:31 ` Anthony F McInerney
2014-12-12 10:27 ` Guan Xin
[not found] ` <CAGw_SrN6tZyXRx6pxkpnhWc2U5845HRbfzhOK5v9qkjWejFpmA@mail.gmail.com>
2014-12-12 15:56 ` Bruno Wolff III
2014-12-12 22:45 ` Phillip Lougher
-- strict thread matches above, loose matches on Subject: below --
2013-07-22 2:21 Phillip Lougher
2013-07-22 2:21 ` [PATCH 2/2] Squashfs: Add LZ4 compression configuration option Phillip Lougher
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).