From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from mail-pg1-f195.google.com ([209.85.215.195]:38548 "EHLO
        mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S2390614AbeKWVgI (ORCPT
        <rfc822;fstests@vger.kernel.org>); Fri, 23 Nov 2018 16:36:08 -0500
Received: by mail-pg1-f195.google.com with SMTP id g189so2943969pgc.5
        for <fstests@vger.kernel.org>; Fri, 23 Nov 2018 02:52:21 -0800 (PST)
Date: Fri, 23 Nov 2018 18:52:14 +0800
From: Eryu Guan <guaneryu@gmail.com>
Subject: Re: [PATCH] fstest: CrashMonkey 'hard link' tests ported to xfstest
Message-ID: <20181123105214.GL3889@desktop>
References: <1542757864-14391-1-git-send-email-jaya@cs.utexas.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <1542757864-14391-1-git-send-email-jaya@cs.utexas.edu>
Sender: fstests-owner@vger.kernel.org
Content-Transfer-Encoding: quoted-printable
To: Jayashree <jaya@cs.utexas.edu>
Cc: fstests@vger.kernel.org, tytso@mit.edu, david@fromorbit.com, fdmanana@gmail.com, vijay@cs.utexas.edu
List-ID: <fstests@vger.kernel.org>

On Tue, Nov 20, 2018 at 05:51:04PM -0600, Jayashree wrote:
> This patch aims to add more tests to the xfstest suite to check
> whether the target file system recovers correctly after a crash.
> These test cases are generated by CrashMonkey, a
> crash-consistency testing framework built at the SASLab at UT Austin.
>=20
> This patch batches 37 crash-consistency tests into a xfstest test,
> each of which checks the hard link behavior under different scenarios.
> This test creates hard-links between files in the same directory or
> across directories, while allowing fsync of either the files involved,
> their parent directories, or unrelated sibling files. After each sub
> test, the metadata of the persisted file is checked for the correct
> link count. Additionally, each sub test is followed by fsck to check
> for inconsistencies. The tests run on a 256MB file system, and
> the working directory is cleaned up after every sub test.

Looks good to me! Thanks for the revision!

>=20
> This test has been added to the =E2=80=9Cauto=E2=80=9D group as advised=
 by Eryu.
> Once a decision is made about the addition of the new =E2=80=9Cregress=E2=
=80=9D
> group, this test can be moved to the appropriate group.

For now, I think we could also add 'quick' (it runs less than 30s on my
test vm) and 'metadata' group.

The two issues Filipe mentioned are easy to fix, I can fix them on
commit. But I'll let others take a look at this new version first, so
I'll leave it on the list for a while.

Thanks,
Eryu

>=20
> Signed-off-by: Jayashree Mohan <jaya@cs.utexas.edu>
> ---
>  tests/generic/520     | 183 ++++++++++++++++++++++++++++++++++++++++++=
++++++++
>  tests/generic/520.out |  75 +++++++++++++++++++++
>  tests/generic/group   |   1 +
>  3 files changed, 259 insertions(+)
>  create mode 100755 tests/generic/520
>  create mode 100644 tests/generic/520.out
>=20
> diff --git a/tests/generic/520 b/tests/generic/520
> new file mode 100755
> index 0000000..46e82ac
> --- /dev/null
> +++ b/tests/generic/520
> @@ -0,0 +1,183 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2018 The University of Texas at Austin.  All Rights Re=
served.
> +#
> +# FS QA Test 520
> +#
> +# Test case created by CrashMonkey
> +#
> +# Test if we create a hard link to a file and persist either of the fi=
les, all the names persist.
> +#
> +seq=3D`basename $0`
> +seqres=3D$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=3D`pwd`
> +tmp=3D/tmp/$$
> +status=3D1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	_cleanup_flakey
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/dmflakey
> +
> +# 256MB in byte
> +fssize=3D$((2**20 * 256))
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch_nocheck
> +_require_dm_target flakey
> +
> +# initialize scratch device
> +_scratch_mkfs_sized $fssize >> $seqres.full 2>&1
> +_require_metadata_journaling $SCRATCH_DEV
> +_init_flakey
> +
> +stat_opt=3D'-c "blocks: %b size: %s inode: %i links: %h"'
> +before=3D""
> +after=3D""
> +
> +
> +# Using _scratch_mkfs instead of cleaning up the  working directory,
> +# adds about 10 seconds of delay in total for the 37 tests.
> +clean_dir()
> +{
> +	_mount_flakey
> +	rm -rf $SCRATCH_MNT/*
> +	sync
> +	_unmount_flakey
> +}
> +
> +check_consistency()
> +{
> +	_flakey_drop_and_remount | tee -a $seqres.full
> +
> +	if [ -f $1 ]; then
> +		after=3D`stat "$stat_opt" $1`
> +	fi
> +
> +	if [ "$before" !=3D "$after" ] && [ $2 -ne 1 ]; then
> +		echo "Before: $before"
> +		echo "After: $after"
> +	fi
> +
> +	_unmount_flakey
> +	_check_scratch_fs $FLAKEY_DEV
> +	[ $? -ne 0 ] && _fatal "fsck failed"
> +}
> +
> +# create a hard link $2 to file $1, and fsync $3, followed by power-cu=
t
> +test_link_fsync()
> +{
> +	local sibling=3D0
> +	local src=3D$SCRATCH_MNT/$1
> +	local dest=3D$SCRATCH_MNT/$2
> +	before=3D""
> +	after=3D""
> +
> +	if [ "$3" =3D=3D "./" ]; then
> +		fsync=3D$SCRATCH_MNT
> +	else
> +		fsync=3D$SCRATCH_MNT/$3
> +	fi
> +
> +	echo -ne "\n=3D=3D=3D link $src $dest  with fsync $fsync =3D=3D=3D\n"=
 | _filter_scratch
> +	_mount_flakey
> +
> +	# Now execute the workload
> +	# Create the directory in which the source and destination files
> +	# will be created
> +	mkdir -p "${src%/*}"
> +	mkdir -p "${dest%/*}"
> +	touch $src
> +	ln $src $dest
> +
> +	# If the file being persisted is a sibling, create it first
> +	if [ ! -f $fsync ]; then
> +		sibling=3D1
> +		touch $fsync
> +	fi
> +
> +	$XFS_IO_PROG -c "fsync" $fsync
> +
> +	if [ $sibling -ne 1 ]; then
> +		before=3D`stat "$stat_opt" $src`
> +	fi
> +
> +	check_consistency $src $sibling
> +	clean_dir
> +}
> +
> +# create a hard link $2 to file $1, and sync, followed by power-cut
> +test_link_sync()
> +{
> +	local src=3D$SCRATCH_MNT/$1
> +	local dest=3D$SCRATCH_MNT/$2
> +	before=3D""
> +	after=3D""
> +	echo -ne "\n=3D=3D=3D link $src $dest  with sync =3D=3D=3D\n" | _filt=
er_scratch
> +	_mount_flakey
> +
> +	# now execute the workload
> +	# Create the directory in which the source and destination files
> +	# will be created
> +	mkdir -p "${src%/*}"
> +	mkdir -p "${dest%/*}"
> +	touch $src
> +	ln $src $dest
> +	sync
> +	before=3D`stat "$stat_opt" $src`
> +
> +	check_consistency $src 0
> +	clean_dir
> +}
> +
> +
> +# Create different combinations to run the link test
> +# Group 0: Both files within root directory
> +file_names[0]=3D"foo bar"
> +fsync_names[0]=3D"./ foo bar"
> +
> +# Group 1: Create hard link in a sub directory
> +file_names[1]=3D"foo A/bar"
> +fsync_names[1]=3D"./ foo bar A A/bar A/foo"
> +
> +# Group 2: Create hard link in parent directory
> +file_names[2]=3D"A/foo bar"
> +fsync_names[2]=3D"./ foo bar A A/bar A/foo"
> +
> +# Group 3: Both files within a directory other than root
> +file_names[3]=3D"A/foo A/bar"
> +fsync_names[3]=3D"./ A A/bar A/foo"
> +
> +#Group 4: Exercise name reuse : Link file in sub-directory
> +file_names[4]=3D"bar A/bar"
> +fsync_names[4]=3D"./ foo bar A A/bar A/foo"
> +
> +#Group 5: Exercise name reuse : Link file in parent directory
> +file_names[5]=3D"A/bar bar"
> +fsync_names[5]=3D"./ foo bar A A/bar A/foo"
> +
> +for ((test_group=3D0; test_group<6; test_group++)); do
> +	for file in ${fsync_names[$test_group]}; do
> +		test_link_fsync ${file_names[$test_group]} $file
> +	done
> +	test_link_sync ${file_names[$test_group]}
> +done
> +
> +# success, all done
> +status=3D0
> +exit
> diff --git a/tests/generic/520.out b/tests/generic/520.out
> new file mode 100644
> index 0000000..b0d75a1
> --- /dev/null
> +++ b/tests/generic/520.out
> @@ -0,0 +1,75 @@
> +QA output created by 520
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/bar  with fsync SCRATCH_MNT=
 =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/bar  with fsync SCRATCH_MNT=
/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/bar  with fsync SCRATCH_MNT=
/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/bar  with sync =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/A =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/A/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/A/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/foo SCRATCH_MNT/A/bar  with sync =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/A =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/A/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/A/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/bar  with sync =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/A/bar  with fsync SCRATCH=
_MNT =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/A/bar  with fsync SCRATCH=
_MNT/A =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/A/bar  with fsync SCRATCH=
_MNT/A/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/A/bar  with fsync SCRATCH=
_MNT/A/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/foo SCRATCH_MNT/A/bar  with sync =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/A =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/A/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with fsync SCRATCH_M=
NT/A/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/bar SCRATCH_MNT/A/bar  with sync =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/A =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/A/bar =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with fsync SCRATCH_M=
NT/A/foo =3D=3D=3D
> +
> +=3D=3D=3D link SCRATCH_MNT/A/bar SCRATCH_MNT/bar  with sync =3D=3D=3D
> --=20
> 2.7.4
>=20