[RFC PATCH] ubifs: Add zstd support

[RFC PATCH] ubifs: Add zstd support

[Yufen Yu]

Add zstd compression and decompression support to ubifs. zstd at its
fastest level compresses almost as well as zlib, while offering much
faster compression and decompression, approaching lzo speeds.

This patch is based on the patch:
	https://patchwork.kernel.org/patch/9892631/

zstd source repository: https://github.com/facebook/zstd

Signed-off-by: Yufen Yu <yuyufen@huawei.com>
---
 fs/ubifs/Kconfig       | 10 ++++++++++
 fs/ubifs/compress.c    | 25 +++++++++++++++++++++++++
 fs/ubifs/ubifs-media.h |  2 ++
 3 files changed, 37 insertions(+)

diff --git a/fs/ubifs/Kconfig b/fs/ubifs/Kconfig
index 83a961b..056a81b 100644
--- a/fs/ubifs/Kconfig
+++ b/fs/ubifs/Kconfig
@@ -5,8 +5,10 @@ config UBIFS_FS
 	select CRYPTO if UBIFS_FS_ADVANCED_COMPR
 	select CRYPTO if UBIFS_FS_LZO
 	select CRYPTO if UBIFS_FS_ZLIB
+	select CRYPTO if UBIFS_FS_ZSTD
 	select CRYPTO_LZO if UBIFS_FS_LZO
 	select CRYPTO_DEFLATE if UBIFS_FS_ZLIB
+	select CRYPTO_ZSTD if UBIFS_FS_XZ
 	depends on MTD_UBI
 	help
 	  UBIFS is a file system for flash devices which works on top of UBI.
@@ -36,6 +38,14 @@ config UBIFS_FS_ZLIB
 	help
 	  Zlib compresses better than LZO but it is slower. Say 'Y' if unsure.
 
+config UBIFS_FS_ZSTD
+	bool "ZSTD compression support" if UBIFS_FS_ADVANCED_COMPR
+	depends on UBIFS_FS
+	default y
+	help
+	  ZSTD compresses is a big win in speed over Zlib and
+	  in compression ratio over LZO. Say 'Y' if unsure.
+
 config UBIFS_ATIME_SUPPORT
 	bool "Access time support" if UBIFS_FS
 	depends on UBIFS_FS
diff --git a/fs/ubifs/compress.c b/fs/ubifs/compress.c
index 565cb56..b310d6f 100644
--- a/fs/ubifs/compress.c
+++ b/fs/ubifs/compress.c
@@ -71,6 +71,24 @@ static struct ubifs_compressor zlib_compr = {
 };
 #endif
 
+#ifdef CONFIG_UBIFS_FS_ZSTD
+static DEFINE_MUTEX(zstd_enc_mutex);
+static DEFINE_MUTEX(zstd_dec_mutex);
+
+static struct ubifs_compressor zstd_compr = {
+	.compr_type = UBIFS_COMPR_ZSTD,
+	.comp_mutex = &zstd_enc_mutex,
+	.decomp_mutex = &zstd_dec_mutex,
+	.name = "zstd",
+	.capi_name = "zstd",
+};
+#else
+static struct ubifs_compressor zstd_compr = {
+	.compr_type = UBIFS_COMPR_ZSTD,
+	.name = "zstd",
+};
+#endif
+
 /* All UBIFS compressors */
 struct ubifs_compressor *ubifs_compressors[UBIFS_COMPR_TYPES_CNT];
 
@@ -228,6 +246,10 @@ int __init ubifs_compressors_init(void)
 	if (err)
 		return err;
 
+	err = compr_init(&zstd_compr);
+	if (err)
+		goto out_zstd;
+
 	err = compr_init(&zlib_compr);
 	if (err)
 		goto out_lzo;
@@ -235,6 +257,8 @@ int __init ubifs_compressors_init(void)
 	ubifs_compressors[UBIFS_COMPR_NONE] = &none_compr;
 	return 0;
 
+out_zstd:
+	compr_exit(&zstd_compr);
 out_lzo:
 	compr_exit(&lzo_compr);
 	return err;
@@ -247,4 +271,5 @@ void ubifs_compressors_exit(void)
 {
 	compr_exit(&lzo_compr);
 	compr_exit(&zlib_compr);
+	compr_exit(&zstd_compr);
 }
diff --git a/fs/ubifs/ubifs-media.h b/fs/ubifs/ubifs-media.h
index e8c23c9..9f869c7 100644
--- a/fs/ubifs/ubifs-media.h
+++ b/fs/ubifs/ubifs-media.h
@@ -341,12 +341,14 @@ enum {
  * UBIFS_COMPR_NONE: no compression
  * UBIFS_COMPR_LZO: LZO compression
  * UBIFS_COMPR_ZLIB: ZLIB compression
+ * UBIFS_COMPR_ZSTD: ZSTD compression
  * UBIFS_COMPR_TYPES_CNT: count of supported compression types
  */
 enum {
 	UBIFS_COMPR_NONE,
 	UBIFS_COMPR_LZO,
 	UBIFS_COMPR_ZLIB,
+	UBIFS_COMPR_ZSTD,
 	UBIFS_COMPR_TYPES_CNT,
 };
 
-- 
2.9.5

Re: [RFC PATCH] ubifs: Add zstd support

[Richard Weinberger]

Yufen Yu,

Am Freitag, 12. Januar 2018, 10:24:21 CET schrieb Yufen Yu:
> Add zstd compression and decompression support to ubifs. zstd at its
> fastest level compresses almost as well as zlib, while offering much
> faster compression and decompression, approaching lzo speeds.
> 
> This patch is based on the patch:
> 	https://patchwork.kernel.org/patch/9892631/
> 
> zstd source repository: https://github.com/facebook/zstd

Do you have numbers on how much we gain for UBIFS on a typical embedded 
device?

Thanks,
//richard

Re: [RFC PATCH] ubifs: Add zstd support

[yuyufen]

On 2018/1/12 18:45, Richard Weinberger wrote:
> Yufen Yu,
>
> Am Freitag, 12. Januar 2018, 10:24:21 CET schrieb Yufen Yu:
>> Add zstd compression and decompression support to ubifs. zstd at its
>> fastest level compresses almost as well as zlib, while offering much
>> faster compression and decompression, approaching lzo speeds.
>>
>> This patch is based on the patch:
>> 	https://patchwork.kernel.org/patch/9892631/
>>
>> zstd source repository: https://github.com/facebook/zstd
> Do you have numbers on how much we gain for UBIFS on a typical embedded
> device?
>
> Thanks,
> //richard
>
> .
>

This is a RFC patch. I will post the test results after the full 
function completing.

Thanks a lot.

Yufen Yu

Re: [RFC PATCH] ubifs: Add zstd support

[Richard Weinberger]

Am Dienstag, 16. Januar 2018, 07:16:44 CET schrieb yuyufen:
> On 2018/1/12 18:45, Richard Weinberger wrote:
> > Yufen Yu,
> > 
> > Am Freitag, 12. Januar 2018, 10:24:21 CET schrieb Yufen Yu:
> >> Add zstd compression and decompression support to ubifs. zstd at its
> >> fastest level compresses almost as well as zlib, while offering much
> >> faster compression and decompression, approaching lzo speeds.
> >> 
> >> This patch is based on the patch:
> >> 	https://patchwork.kernel.org/patch/9892631/
> >> 
> >> zstd source repository: https://github.com/facebook/zstd
> > 
> > Do you have numbers on how much we gain for UBIFS on a typical embedded
> > device?
> > 
> > Thanks,
> > //richard
> > 
> > .
> 
> This is a RFC patch. I will post the test results after the full
> function completing.

Well, I have nothing against adding zstd support to UBIFS, if you can show 
that it is useful and can explain why we should use zstd and not another 
modern compressor.

Thanks,
//richard