[f2fs-dev] 答复: 答复: [PATCH v4] f2fs: compress: add nocompress extensions support

[<changfengnan@vivo.com>]

Ok, thanks.

-----邮件原件-----
发件人: Chao Yu <yuchao0@huawei.com> 
发送时间: 2021年5月19日 15:58
收件人: changfengnan@vivo.com; jaegeuk@kernel.org; chao@kernel.org;
linux-f2fs-devel@lists.sourceforge.net
主题: Re: 答复: [f2fs-dev] [PATCH v4] f2fs: compress: add nocompress
extensions support

On 2021/5/19 15:47, changfengnan@vivo.com wrote:
> Do you mean the case of don't allow use '*' to specifie all file in 
> nocompress extension? It's handled by f2fs_test_compress_extension.
> Or other case?

I've pointed out in below thread:

https://lore.kernel.org/linux-f2fs-devel/c7ee799d-a8de-4671-bab7-2c87146ad53
0@huawei.com/T/#m2ee997221a095845c07aa8bca412cd34d3c6c02c\

Quoted:
"
 > +static int f2fs_test_compress_extension(struct f2fs_sb_info *sbi)  > +{
 > +	unsigned char (*ext)[F2FS_EXTENSION_LEN];
 > +	unsigned char (*noext)[F2FS_EXTENSION_LEN];
 > +	int ext_cnt, noext_cnt, index = 0, no_index = 0;
 > +
 > +	ext = F2FS_OPTION(sbi).extensions;
 > +	ext_cnt = F2FS_OPTION(sbi).compress_ext_cnt;
 > +	noext = F2FS_OPTION(sbi).noextensions;
 > +	noext_cnt = F2FS_OPTION(sbi).nocompress_ext_cnt;
 > +
 > +	if (!ext_cnt || !noext_cnt)

It doesn't cover the case: ext_cnt = 0, and there is "*" in noext[].
"

Thanks,

> 
> -----邮件原件-----
> 发件人: Chao Yu <yuchao0@huawei.com>
> 发送时间: 2021年5月19日 15:01
> 收件人: Fengnan Chang <changfengnan@vivo.com>; jaegeuk@kernel.org; 
> chao@kernel.org; linux-f2fs-devel@lists.sourceforge.net
> 主题: Re: [f2fs-dev] [PATCH v4] f2fs: compress: add nocompress 
> extensions support
> 
> On 2021/5/18 20:07, Fengnan Chang wrote:
>> When we create a directory with enable compression, all file write 
>> into directory will try to compress.But sometimes we may know, new 
>> file cannot meet compression ratio requirements.
>> We need a nocompress extension to skip those files to avoid 
>> unnecessary compress page test.
>>
>> After add nocompress_extension, the priority should be:
>> dir_flag < comp_extention,nocompress_extension < comp_file_flag, 
>> no_comp_file_flag.
>>
>> For example:
>> 1.If dir is set compress, the default file and compress_extension 
>> specified file will compress, and nocompress_extension specified file 
>> will nocompress.
>> 2.If dir is set not compress, the default file and 
>> nocompress_extension specified file will not compress, but 
>> compress_extension specified file will compress.
>> 3.We can change compress attribute regardless of the file in which 
>> type of dir and  specified or not.
>>
>> Signed-off-by: Fengnan Chang <changfengnan@vivo.com>
>> ---
>>    Documentation/filesystems/f2fs.rst | 18 +++++++
>>    fs/f2fs/f2fs.h                     |  2 +
>>    fs/f2fs/namei.c                    | 18 +++++--
>>    fs/f2fs/super.c                    | 79 +++++++++++++++++++++++++++++-
>>    4 files changed, 113 insertions(+), 4 deletions(-)
>>
>> diff --git a/Documentation/filesystems/f2fs.rst
>> b/Documentation/filesystems/f2fs.rst
>> index 992bf91eeec8..cf162409fc01 100644
>> --- a/Documentation/filesystems/f2fs.rst
>> +++ b/Documentation/filesystems/f2fs.rst
>> @@ -281,6 +281,24 @@ compress_extension=%s	 Support adding specified
> extension, so that f2fs can enab
>>    			 For other files, we can still enable compression
> via ioctl.
>>    			 Note that, there is one reserved special extension
> '*', it
>>    			 can be set to enable compression for all files.
>> +nocompress_extension=%s	   Support adding specified extension, so
> that f2fs can disable
>> +			 compression on those corresponding files, just
> contrary to compression extension.
>> +			 If you know exactly which files cannot be
> compressed, you can use this.
>> +			 The same extension name can't appear in both
> compress and nocompress
>> +			 extension at the same time.
>> +			 If the compress extension specifies all files, the
> types specified by the
>> +			 nocompress extension will be treated as special
> cases and will not be compressed.
>> +			 Don't allow use '*' to specifie all file in
> nocompress extension.
> 
> IIUC, it seems the code doesn't cover this case?
> 
> Thanks,
> 
>> +			 After add nocompress_extension, the priority should
> be:
>> +			 dir_flag < comp_extention,nocompress_extension <
> comp_file_flag,no_comp_file_flag.
>> +			 For example:
>> +			 1.If dir is set compress, the default file and
> compress_extension specified file
>> +			 will compress, and nocompress_extension specified
> file will nocompress.
>> +			 2.If dir is set not compress, the default file and
> nocompress_extension specified
>> +			 file will not compress, but compress_extension
> specified file will compress.
>> +			 3.We can change compress attribute regardless of
> the file in which type of dir
>> +			 and specified or not.
>> +
>>    compress_chksum		 Support verifying chksum of raw data in
> compressed cluster.
>>    compress_mode=%s	 Control file compression mode. This supports "fs"
> and "user"
>>    			 modes. In "fs" mode (default), f2fs does automatic
> compression
>> diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index
>> c83d90125ebd..1c1215f06f68 100644
>> --- a/fs/f2fs/f2fs.h
>> +++ b/fs/f2fs/f2fs.h
>> @@ -150,8 +150,10 @@ struct f2fs_mount_info {
>>    	unsigned char compress_level;		/* compress level */
>>    	bool compress_chksum;			/* compressed data chksum */
>>    	unsigned char compress_ext_cnt;		/* extension count */
>> +	unsigned char nocompress_ext_cnt;		/* nocompress
> extension count */
>>    	int compress_mode;			/* compression mode */
>>    	unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN];	/*
> extensions */
>> +	unsigned char noextensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN]; 
>> +/*
> 
>> +extensions */
>>    };
>>    
>>    #define F2FS_FEATURE_ENCRYPT		0x0001
>> diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c index 
>> a9cd9cf97229..b57b41d7799e 100644
>> --- a/fs/f2fs/namei.c
>> +++ b/fs/f2fs/namei.c
>> @@ -279,14 +279,16 @@ static void set_compress_inode(struct 
>> f2fs_sb_info
> *sbi, struct inode *inode,
>>    						const unsigned char *name)
>>    {
>>    	__u8 (*extlist)[F2FS_EXTENSION_LEN] =
>> sbi->raw_super->extension_list;
>> +	unsigned char (*noext)[F2FS_EXTENSION_LEN] = 
>> +F2FS_OPTION(sbi).noextensions;
>>    	unsigned char (*ext)[F2FS_EXTENSION_LEN];
>> -	unsigned int ext_cnt = F2FS_OPTION(sbi).compress_ext_cnt;
>> +	unsigned char ext_cnt = F2FS_OPTION(sbi).compress_ext_cnt;
>> +	unsigned char noext_cnt = F2FS_OPTION(sbi).nocompress_ext_cnt;
>>    	int i, cold_count, hot_count;
>>    
>>    	if (!f2fs_sb_has_compression(sbi) ||
>> -			is_inode_flag_set(inode, FI_COMPRESSED_FILE) ||
>>    			F2FS_I(inode)->i_flags & F2FS_NOCOMP_FL ||
>> -			!f2fs_may_compress(inode))
>> +			!f2fs_may_compress(inode) ||
>> +			(!ext_cnt && !noext_cnt))
>>    		return;
>>    
>>    	down_read(&sbi->sb_lock);
>> @@ -303,6 +305,16 @@ static void set_compress_inode(struct 
>> f2fs_sb_info *sbi, struct inode *inode,
>>    
>>    	up_read(&sbi->sb_lock);
>>    
>> +	for (i = 0; i < noext_cnt; i++) {
>> +		if (is_extension_exist(name, noext[i])) {
>> +			f2fs_disable_compressed_file(inode);
>> +			return;
>> +		}
>> +	}
>> +
>> +	if (is_inode_flag_set(inode, FI_COMPRESSED_FILE))
>> +		return;
>> +
>>    	ext = F2FS_OPTION(sbi).extensions;
>>    
>>    	for (i = 0; i < ext_cnt; i++) {
>> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index
>> 7d325bfaf65a..d38e2e4d7da3 100644
>> --- a/fs/f2fs/super.c
>> +++ b/fs/f2fs/super.c
>> @@ -148,6 +148,7 @@ enum {
>>    	Opt_compress_algorithm,
>>    	Opt_compress_log_size,
>>    	Opt_compress_extension,
>> +	Opt_nocompress_extension,
>>    	Opt_compress_chksum,
>>    	Opt_compress_mode,
>>    	Opt_atgc,
>> @@ -222,6 +223,7 @@ static match_table_t f2fs_tokens = {
>>    	{Opt_compress_algorithm, "compress_algorithm=%s"},
>>    	{Opt_compress_log_size, "compress_log_size=%u"},
>>    	{Opt_compress_extension, "compress_extension=%s"},
>> +	{Opt_nocompress_extension, "nocompress_extension=%s"},
>>    	{Opt_compress_chksum, "compress_chksum"},
>>    	{Opt_compress_mode, "compress_mode=%s"},
>>    	{Opt_atgc, "atgc"},
>> @@ -473,6 +475,43 @@ static int f2fs_set_test_dummy_encryption(struct
> super_block *sb,
>>    }
>>    
>>    #ifdef CONFIG_F2FS_FS_COMPRESSION
>> +/*
>> + * 1. The same extension name cannot not appear in both compress and 
>> +non-compress extension
>> + * at the same time.
>> + * 2. If the compress extension specifies all files, the types 
>> +specified by the non-compress
>> + * extension will be treated as special cases and will not be
compressed.
>> + * 3. Don't allow the non-compress extension specifies all files.
>> + */
>> +static int f2fs_test_compress_extension(struct f2fs_sb_info *sbi) {
>> +	unsigned char (*ext)[F2FS_EXTENSION_LEN];
>> +	unsigned char (*noext)[F2FS_EXTENSION_LEN];
>> +	int ext_cnt, noext_cnt, index = 0, no_index = 0;
>> +
>> +	ext = F2FS_OPTION(sbi).extensions;
>> +	ext_cnt = F2FS_OPTION(sbi).compress_ext_cnt;
>> +	noext = F2FS_OPTION(sbi).noextensions;
>> +	noext_cnt = F2FS_OPTION(sbi).nocompress_ext_cnt;
>> +
>> +	if (!ext_cnt || !noext_cnt)
>> +		return 0;
>> +
>> +	for (no_index = 0; no_index < noext_cnt; no_index++) {
>> +		if (!strcasecmp("*", noext[no_index])) {
>> +			f2fs_info(sbi, "Don't allow the non-compress
> extension specifies all files");
>> +			return -EINVAL;
>> +		}
>> +		for (index = 0; index < ext_cnt; index++) {
>> +			if (!strcasecmp(ext[index], noext[no_index])) {
>> +				f2fs_info(sbi, "Don't allow the same
> extension %s appear in both compress and non-compress extension",
>> +						ext[index]);
>> +				return -EINVAL;
>> +			}
>> +		}
>> +	}
>> +	return 0;
>> +}
>> +
>>    #ifdef CONFIG_F2FS_FS_LZ4
>>    static int f2fs_set_lz4hc_level(struct f2fs_sb_info *sbi, const 
>> char
> *str)
>>    {
>> @@ -546,7 +585,8 @@ static int parse_options(struct super_block *sb, 
>> char
> *options, bool is_remount)
>>    	substring_t args[MAX_OPT_ARGS];
>>    #ifdef CONFIG_F2FS_FS_COMPRESSION
>>    	unsigned char (*ext)[F2FS_EXTENSION_LEN];
>> -	int ext_cnt;
>> +	unsigned char (*noext)[F2FS_EXTENSION_LEN];
>> +	int ext_cnt, noext_cnt;
>>    #endif
>>    	char *p, *name;
>>    	int arg = 0;
>> @@ -1049,6 +1089,30 @@ static int parse_options(struct super_block 
>> *sb,
> char *options, bool is_remount)
>>    			F2FS_OPTION(sbi).compress_ext_cnt++;
>>    			kfree(name);
>>    			break;
>> +		case Opt_nocompress_extension:
>> +			if (!f2fs_sb_has_compression(sbi)) {
>> +				f2fs_info(sbi, "Image doesn't support
> compression");
>> +				break;
>> +			}
>> +			name = match_strdup(&args[0]);
>> +			if (!name)
>> +				return -ENOMEM;
>> +
>> +			noext = F2FS_OPTION(sbi).noextensions;
>> +			noext_cnt = F2FS_OPTION(sbi).nocompress_ext_cnt;
>> +
>> +			if (strlen(name) >= F2FS_EXTENSION_LEN ||
>> +				noext_cnt >= COMPRESS_EXT_NUM) {
>> +				f2fs_err(sbi,
>> +					"invalid extension length/number");
>> +				kfree(name);
>> +				return -EINVAL;
>> +			}
>> +
>> +			strcpy(noext[noext_cnt], name);
>> +			F2FS_OPTION(sbi).nocompress_ext_cnt++;
>> +			kfree(name);
>> +			break;
>>    		case Opt_compress_chksum:
>>    			F2FS_OPTION(sbi).compress_chksum = true;
>>    			break;
>> @@ -1070,6 +1134,7 @@ static int parse_options(struct super_block 
>> *sb,
> char *options, bool is_remount)
>>    		case Opt_compress_algorithm:
>>    		case Opt_compress_log_size:
>>    		case Opt_compress_extension:
>> +		case Opt_nocompress_extension:
>>    		case Opt_compress_chksum:
>>    		case Opt_compress_mode:
>>    			f2fs_info(sbi, "compression options not supported");
> @@ -1122,6
>> +1187,13 @@ static int parse_options(struct super_block *sb, char
> *options, bool is_remount)
>>    	}
>>    #endif
>>    
>> +#ifdef CONFIG_F2FS_FS_COMPRESSION
>> +	if (f2fs_test_compress_extension(sbi)) {
>> +		f2fs_err(sbi, "invalid compress or nocompress extension");
>> +		return -EINVAL;
>> +	}
>> +#endif
>> +
>>    	if (F2FS_IO_SIZE_BITS(sbi) && !f2fs_lfs_mode(sbi)) {
>>    		f2fs_err(sbi, "Should set mode=lfs with %uKB-sized IO",
>>    			 F2FS_IO_SIZE_KB(sbi));
>> @@ -1665,6 +1737,11 @@ static inline void
> f2fs_show_compress_options(struct seq_file *seq,
>>    			F2FS_OPTION(sbi).extensions[i]);
>>    	}
>>    
>> +	for (i = 0; i < F2FS_OPTION(sbi).nocompress_ext_cnt; i++) {
>> +		seq_printf(seq, ",nocompress_extension=%s",
>> +			F2FS_OPTION(sbi).noextensions[i]);
>> +	}
>> +
>>    	if (F2FS_OPTION(sbi).compress_chksum)
>>    		seq_puts(seq, ",compress_chksum");
>>    
>>
> 
> 
> .
> 




_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel