* [PATCH 11/16] lib: enable lzma decompression support for SPL build @ 2020-01-08 3:01 Weijie Gao 2020-01-08 7:22 ` Heinrich Schuchardt 0 siblings, 1 reply; 6+ messages in thread From: Weijie Gao @ 2020-01-08 3:01 UTC (permalink / raw) To: u-boot This patch enables LZMA decompression support for SPL build Signed-off-by: Weijie Gao <weijie.gao@mediatek.com> --- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+) diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot. + config SPL_LZO bool "Enable LZO decompression support in SPL" help diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o obj-$(CONFIG_LIBAVB) += libavb/ -- 2.17.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build 2020-01-08 3:01 [PATCH 11/16] lib: enable lzma decompression support for SPL build Weijie Gao @ 2020-01-08 7:22 ` Heinrich Schuchardt 2020-01-09 8:10 ` Weijie Gao 0 siblings, 1 reply; 6+ messages in thread From: Heinrich Schuchardt @ 2020-01-08 7:22 UTC (permalink / raw) To: u-boot On 1/8/20 4:01 AM, Weijie Gao wrote: > This patch enables LZMA decompression support for SPL build > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com> > --- > lib/Kconfig | 5 +++++ > lib/Makefile | 1 + > 2 files changed, 6 insertions(+) > > diff --git a/lib/Kconfig b/lib/Kconfig > index 965cf7bc03..2f38e37802 100644 > --- a/lib/Kconfig > +++ b/lib/Kconfig > @@ -418,6 +418,11 @@ config SPL_LZ4 > fast compression and decompression speed. It belongs to the LZ77 > family of byte-oriented compression schemes. > > +config SPL_LZMA > + bool "Enable LZMA decompression support for SPL build" > + help > + This enables support for LZMA compression altorithm for SPL boot. %s/altorithm/algorithm/g Why do we need yet another algorithm for compression? Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression Best regards Heinrich > + > config SPL_LZO > bool "Enable LZO decompression support in SPL" > help > diff --git a/lib/Makefile b/lib/Makefile > index 1fb650cd90..8e51eec01c 100644 > --- a/lib/Makefile > +++ b/lib/Makefile > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > obj-$(CONFIG_LIBAVB) += libavb/ > ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build 2020-01-08 7:22 ` Heinrich Schuchardt @ 2020-01-09 8:10 ` Weijie Gao 2020-01-09 17:45 ` Tom Rini 0 siblings, 1 reply; 6+ messages in thread From: Weijie Gao @ 2020-01-09 8:10 UTC (permalink / raw) To: u-boot On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > This patch enables LZMA decompression support for SPL build > > > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com> > > --- > > lib/Kconfig | 5 +++++ > > lib/Makefile | 1 + > > 2 files changed, 6 insertions(+) > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > index 965cf7bc03..2f38e37802 100644 > > --- a/lib/Kconfig > > +++ b/lib/Kconfig > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > fast compression and decompression speed. It belongs to the LZ77 > > family of byte-oriented compression schemes. > > > > +config SPL_LZMA > > + bool "Enable LZMA decompression support for SPL build" > > + help > > + This enables support for LZMA compression altorithm for SPL boot. > > %s/altorithm/algorithm/g > > Why do we need yet another algorithm for compression? > > Doesn't Zstandard provide a much faster decompression? > Cf. https://sites.google.com/site/powturbo/home/web-compression > > Best regards > > Heinrich > > > + > > config SPL_LZO > > bool "Enable LZO decompression support in SPL" > > help > > diff --git a/lib/Makefile b/lib/Makefile > > index 1fb650cd90..8e51eec01c 100644 > > --- a/lib/Makefile > > +++ b/lib/Makefile > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > Hi Heinrich, I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible. I believe currently lzma and xz are the best choices. Refer to: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time. Best Regards, Weijie ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build 2020-01-09 8:10 ` Weijie Gao @ 2020-01-09 17:45 ` Tom Rini 2020-01-10 8:21 ` Weijie Gao 0 siblings, 1 reply; 6+ messages in thread From: Tom Rini @ 2020-01-09 17:45 UTC (permalink / raw) To: u-boot On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote: > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > > This patch enables LZMA decompression support for SPL build > > > > > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com> > > > --- > > > lib/Kconfig | 5 +++++ > > > lib/Makefile | 1 + > > > 2 files changed, 6 insertions(+) > > > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > > index 965cf7bc03..2f38e37802 100644 > > > --- a/lib/Kconfig > > > +++ b/lib/Kconfig > > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > > fast compression and decompression speed. It belongs to the LZ77 > > > family of byte-oriented compression schemes. > > > > > > +config SPL_LZMA > > > + bool "Enable LZMA decompression support for SPL build" > > > + help > > > + This enables support for LZMA compression altorithm for SPL boot. > > > > %s/altorithm/algorithm/g > > > > Why do we need yet another algorithm for compression? > > > > Doesn't Zstandard provide a much faster decompression? > > Cf. https://sites.google.com/site/powturbo/home/web-compression > > > > Best regards > > > > Heinrich > > > > > + > > > config SPL_LZO > > > bool "Enable LZO decompression support in SPL" > > > help > > > diff --git a/lib/Makefile b/lib/Makefile > > > index 1fb650cd90..8e51eec01c 100644 > > > --- a/lib/Makefile > > > +++ b/lib/Makefile > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > > > > Hi Heinrich, > > I need an algorithm with the maximum compression ratio to reduce the > size of the u-boot payload as much as possible. > > I believe currently lzma and xz are the best choices. > Refer to: > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO > > Although they will take longer time for compression and decompression, > considering that the u-boot payload is usually small (<= 1MiB), there > will not be significant effect on decompression time. Where does zstd fall in the list however? Can you please try that and report back? Thanks! -- Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200109/744b4b1c/attachment.sig> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build 2020-01-09 17:45 ` Tom Rini @ 2020-01-10 8:21 ` Weijie Gao 2020-01-10 14:17 ` Tom Rini 0 siblings, 1 reply; 6+ messages in thread From: Weijie Gao @ 2020-01-10 8:21 UTC (permalink / raw) To: u-boot On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote: > On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote: > > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > > > > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > > > This patch enables LZMA decompression support for SPL build > > > > > > > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com> > > > > --- > > > > lib/Kconfig | 5 +++++ > > > > lib/Makefile | 1 + > > > > 2 files changed, 6 insertions(+) > > > > > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > > > index 965cf7bc03..2f38e37802 100644 > > > > --- a/lib/Kconfig > > > > +++ b/lib/Kconfig > > > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > > > fast compression and decompression speed. It belongs to the LZ77 > > > > family of byte-oriented compression schemes. > > > > > > > > +config SPL_LZMA > > > > + bool "Enable LZMA decompression support for SPL build" > > > > + help > > > > + This enables support for LZMA compression altorithm for SPL boot. > > > > > > %s/altorithm/algorithm/g > > > > > > Why do we need yet another algorithm for compression? > > > > > > Doesn't Zstandard provide a much faster decompression? > > > Cf. https://sites.google.com/site/powturbo/home/web-compression > > > > > > Best regards > > > > > > Heinrich > > > > > > > + > > > > config SPL_LZO > > > > bool "Enable LZO decompression support in SPL" > > > > help > > > > diff --git a/lib/Makefile b/lib/Makefile > > > > index 1fb650cd90..8e51eec01c 100644 > > > > --- a/lib/Makefile > > > > +++ b/lib/Makefile > > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > > > > > > > Hi Heinrich, > > > > I need an algorithm with the maximum compression ratio to reduce the > > size of the u-boot payload as much as possible. > > > > I believe currently lzma and xz are the best choices. > > Refer to: > > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO > > > > Although they will take longer time for compression and decompression, > > considering that the u-boot payload is usually small (<= 1MiB), there > > will not be significant effect on decompression time. > > Where does zstd fall in the list however? Can you please try that and > report back? Thanks! > My purpose is to reduce the size of u-boot binary, and my plan is to compress the payload and the SPL decompress it at runtime. I need smallest size here, not fastest decompression speed. So there are two points must be taken into consideration: the size of the decompressor (i.e. u-boot-spl.bin) and the size of the compressed payload (u-boot.bin.gz/bz2/...). Below is my test results, including all compression algorithms supported by uImage, plus zstd: Platform: mips - mt7628 Uncompressed size of u-boot.bin: 302736 (295KiB) Commands for compression: gzip -c -f -k -9 bzip2 -c -f -k -9 lzma -c -z -k -9 lzop -c -f -9 lz4 -f -9 zstd --ultra -f Results: algo. | spl.bin size | compressed size | % of uncomp | total size ------------------------------------------------------------------------ gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB) ------------------------------------------------------------------------ bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB) ------------------------------------------------------------------------ lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB) ------------------------------------------------------------------------ lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB) ------------------------------------------------------------------------ lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB) ------------------------------------------------------------------------ zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB) As you can see lzma has the best compression ratio. The is the reason I chose lzma here. Best Regards ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 11/16] lib: enable lzma decompression support for SPL build 2020-01-10 8:21 ` Weijie Gao @ 2020-01-10 14:17 ` Tom Rini 0 siblings, 0 replies; 6+ messages in thread From: Tom Rini @ 2020-01-10 14:17 UTC (permalink / raw) To: u-boot On Fri, Jan 10, 2020 at 04:21:35PM +0800, Weijie Gao wrote: > On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote: > > On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote: > > > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > > > > > > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > > > > This patch enables LZMA decompression support for SPL build > > > > > > > > > > Signed-off-by: Weijie Gao <weijie.gao@mediatek.com> > > > > > --- > > > > > lib/Kconfig | 5 +++++ > > > > > lib/Makefile | 1 + > > > > > 2 files changed, 6 insertions(+) > > > > > > > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > > > > index 965cf7bc03..2f38e37802 100644 > > > > > --- a/lib/Kconfig > > > > > +++ b/lib/Kconfig > > > > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > > > > fast compression and decompression speed. It belongs to the LZ77 > > > > > family of byte-oriented compression schemes. > > > > > > > > > > +config SPL_LZMA > > > > > + bool "Enable LZMA decompression support for SPL build" > > > > > + help > > > > > + This enables support for LZMA compression altorithm for SPL boot. > > > > > > > > %s/altorithm/algorithm/g > > > > > > > > Why do we need yet another algorithm for compression? > > > > > > > > Doesn't Zstandard provide a much faster decompression? > > > > Cf. https://sites.google.com/site/powturbo/home/web-compression > > > > > > > > Best regards > > > > > > > > Heinrich > > > > > > > > > + > > > > > config SPL_LZO > > > > > bool "Enable LZO decompression support in SPL" > > > > > help > > > > > diff --git a/lib/Makefile b/lib/Makefile > > > > > index 1fb650cd90..8e51eec01c 100644 > > > > > --- a/lib/Makefile > > > > > +++ b/lib/Makefile > > > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > > > > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > > > > > > > > > > Hi Heinrich, > > > > > > I need an algorithm with the maximum compression ratio to reduce the > > > size of the u-boot payload as much as possible. > > > > > > I believe currently lzma and xz are the best choices. > > > Refer to: > > > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO > > > > > > Although they will take longer time for compression and decompression, > > > considering that the u-boot payload is usually small (<= 1MiB), there > > > will not be significant effect on decompression time. > > > > Where does zstd fall in the list however? Can you please try that and > > report back? Thanks! > > > > My purpose is to reduce the size of u-boot binary, and my plan is to > compress the payload and the SPL decompress it at runtime. > I need smallest size here, not fastest decompression speed. > > So there are two points must be taken into consideration: > the size of the decompressor (i.e. u-boot-spl.bin) and the size of the > compressed payload (u-boot.bin.gz/bz2/...). > > Below is my test results, including all compression algorithms supported > by uImage, plus zstd: > > Platform: mips - mt7628 > > Uncompressed size of u-boot.bin: 302736 (295KiB) > > Commands for compression: > gzip -c -f -k -9 > bzip2 -c -f -k -9 > lzma -c -z -k -9 > lzop -c -f -9 > lz4 -f -9 > zstd --ultra -f > > Results: > algo. | spl.bin size | compressed size | % of uncomp | total size > ------------------------------------------------------------------------ > gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB) > ------------------------------------------------------------------------ > bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB) > ------------------------------------------------------------------------ > lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB) > ------------------------------------------------------------------------ > lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB) > ------------------------------------------------------------------------ > lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB) > ------------------------------------------------------------------------ > zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB) > > As you can see lzma has the best compression ratio. The is the reason I > chose lzma here. Thanks, that's exactly the kind of data I was looking for. Reviewed-by: Tom Rini <trini@konsulko.com> -- Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200110/e5894b33/attachment.sig> ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-01-10 14:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-01-08 3:01 [PATCH 11/16] lib: enable lzma decompression support for SPL build Weijie Gao 2020-01-08 7:22 ` Heinrich Schuchardt 2020-01-09 8:10 ` Weijie Gao 2020-01-09 17:45 ` Tom Rini 2020-01-10 8:21 ` Weijie Gao 2020-01-10 14:17 ` Tom Rini
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox