Re: [PATCH 2/2] f2fs/007: add testcase to check consistency of compressed inode metadata

[Chao Yu <chao@kernel.org>]

On 2024/10/23 11:07, Zorro Lang wrote:
> On Tue, Oct 15, 2024 at 10:51:06AM +0800, Chao Yu wrote:
>> metadata of compressed inode should always be consistent after file
>> compression, reservation, releasement and decompression, let's add
>> a testcase to check it.
>>
>> Cc: Jaegeuk Kim <jaegeuk@kernel.org>
>> Cc: Qi Han <hanqi@vivo.com>
>> Signed-off-by: Chao Yu <chao@kernel.org>
>> ---
>>   tests/f2fs/007     | 116 +++++++++++++++++++++++++++++++++++++++++++++
>>   tests/f2fs/007.out |   4 ++
>>   2 files changed, 120 insertions(+)
>>   create mode 100755 tests/f2fs/007
>>   create mode 100644 tests/f2fs/007.out
>>
>> diff --git a/tests/f2fs/007 b/tests/f2fs/007
>> new file mode 100755
>> index 00000000..274be806
>> --- /dev/null
>> +++ b/tests/f2fs/007
>> @@ -0,0 +1,116 @@
>> +#! /bin/bash
>> +# SPDX-License-Identifier: GPL-2.0
>> +# Copyright (c) 2024 Oppo.  All Rights Reserved.
>> +#
>> +# FS QA Test No. f2fs/007
>> +#
>> +# This is a regression test to check whether compressed metadata
> 
> Hi Chao,
> 
> Thanks for the new f2fs test cases.
> 
> If there's a known fix for this regression, please mark it by:
> 
> _fixed_by_kernel_commit .....

Zorro,

Yes, but not been merged yet, so let me add as below and replace
commit_id after upstreaming the fix.

_fixed_by_kernel_commit xxxxxxxxxxxx \
         "f2fs: compress: fix inconsistent update of i_blocks in release_compress_blocks and reserve_compress_blocks"

Thanks for all comments below, will fix all.

Thanks,

> 
>> +# can become inconsistent after file compression, reservation
>> +# releasement, and decompression.
>> +#
>> +. ./common/preamble
>> +_begin_fstest auto quick
>                              ^^^^
>                              rw compress ?
> 
>> +
>> +_cleanup()
>> +{
>> +	cd /
>> +	rm -r -f tmp.*
>> +}
> 
> This's same with default _cleanup, so it can be removed.
> 
>> +
>> +testfile_prefix=$SCRATCH_MNT/testfile
>> +fio_config=$tmp.fio
>> +
>> +cat >$fio_config <<EOF
>> +[write_compressed_data_30]
>> +name=mytest
>> +ioengine=psync
>> +rw=write
>> +direct=0
>> +bs=1M
>> +filesize=1M
>> +numjobs=1
>> +filename=/mnt/scratch_f2fs/testfile30
>              ^^^^^^^^^^
> 
> $SCRATCH_MNT or ${testfile_prefix}30 ?
> 
>> +buffer_compress_percentage=30
>> +
>> +[write_compressed_data_60]
>> +name=mytest
>> +ioengine=psync
>> +rw=write
>> +direct=0
>> +bs=1M
>> +filesize=1M
>> +numjobs=1
>> +filename=/mnt/scratch_f2fs/testfile60
> 
> Same as above.
> 
>> +buffer_compress_percentage=60
>> +
>> +[write_compressed_data_90]
>> +name=mytest
>> +ioengine=psync
>> +rw=write
>> +direct=0
>> +bs=1M
>> +filesize=1M
>> +numjobs=1
>> +filename=/mnt/scratch_f2fs/testfile90
> 
> Same as above
> 
>> +buffer_compress_percentage=90
>> +EOF
>> +
>> +_require_fio $fio_config
>> +_require_scratch
> 
> I'd like to call _require_scratch at the beginning, especially you use
> SCRATCH_* things in $fio_config. You can refer to generic/095.
> 
>> +_scratch_mkfs "-f -O extra_attr,compression" >> $seqres.full
> 
> Can you make sure this mkfs never fail? Maybe:
>      || _fail "...."
> 
>> +_scratch_mount "-o compress_mode=user,compress_extension=*" >> $seqres.full
>> +
>> +echo -e "Run fio to initialize file w/ specified compress ratio" >> $seqres.full
>> +cat $fio_config >> $seqres.full
>> +$FIO_PROG $fio_config >> $seqres.full
>> +_scratch_unmount
>> +
>> +for i in 30 60 90; do
>> +	testfile=$testfile_prefix$i
>> +
>> +	_scratch_mount "-o compress_mode=user" >> $seqres.full
>> +	f2fs_io compress $testfile >> $seqres.full
>          ^^^^^^^
> $F2FS_IO_PROG
> 
>> +	cblocks=`f2fs_io get_cblocks $testfile`
>                   ^^^^^^^
> $F2FS_IO_PROG
> 
>> +	echo "compression ratio is: "$cblocks" / 256"
>> +
>> +	_scratch_unmount
>> +
>> +	# 1. check after compression
>> +	fsck -t $FSTYP -f $SCRATCH_DEV >> $seqres.full
> 
> _check_scratch_fs ?
> 
> If f2fs needs some custom operations, please change the _check_scratch_fs
> or _check_generic_filesystem.
> 
>> +	if [ $? -ne 0 ]; then
>> +		_fail "filesystem becomes corrupted after compress"
>> +	fi
>> +
>> +	_scratch_mount >> $seqres.full
>> +	f2fs_io release_cblocks $testfile >> $seqres.full
> 
> $F2FS_IO_PROG
> 
>> +	_scratch_unmount
>> +
>> +	# 2. check after releasement
>> +	fsck -t $FSTYP -f $SCRATCH_DEV >> $seqres.full
> 
> _check_scratch_fs
> 
>> +	if [ $? -ne 0 ]; then
>> +		_fail "filesystem becomes corrupted after release_cblocks"
>> +	fi
>> +
>> +	_scratch_mount >> $seqres.full
>> +	f2fs_io reserve_cblocks $testfile >> $seqres.full
> 
> $F2FS_IO_PROG
> 
>> +	_scratch_unmount
>> +
>> +	# 3. check after rservation
>> +	fsck -t $FSTYP -f $SCRATCH_DEV >> $seqres.full
> 
> _check_scratch_fs
> 
>> +	if [ $? -ne 0 ]; then
>> +		_fail "filesystem becomes corrupted after reserve_cblocks"
>> +	fi
>> +
>> +	_scratch_mount "-o compress_mode=user" >> $seqres.full
>> +	f2fs_io decompress $testfile >> $seqres.full
> 
> $F2FS_IO_PROG
> 
>> +	_scratch_unmount
>> +
>> +	# 4. check after decompression
>> +	fsck -t $FSTYP -f $SCRATCH_DEV >> $seqres.full
> 
> _check_scratch_fs
> 
> Thanks,
> Zorro
> 
>> +	if [ $? -ne 0 ]; then
>> +		_fail "filesystem becomes corrupted after decompress"
>> +	fi
>> +done
>> +
>> +status=0
>> +exit
>> diff --git a/tests/f2fs/007.out b/tests/f2fs/007.out
>> new file mode 100644
>> index 00000000..2ea71c18
>> --- /dev/null
>> +++ b/tests/f2fs/007.out
>> @@ -0,0 +1,4 @@
>> +QA output created by 007
>> +compression ratio is: 64 / 256
>> +compression ratio is: 128 / 256
>> +compression ratio is: 192 / 256
>> -- 
>> 2.40.1
>>
>