Re: [PATCH v2] check: Fix fs specfic imports when $FSTYPE!=$OLD_FSTYPE

["Nirjhar Roy (IBM)" <nirjhar.roy.lists@gmail.com>]

On 2/21/25 11:17, Zorro Lang wrote:
> On Fri, Feb 21, 2025 at 09:44:19AM +0530, Nirjhar Roy (IBM) wrote:
>> On 2/10/25 19:53, Zorro Lang wrote:
>>> On Thu, Feb 06, 2025 at 11:32:43PM +0530, Nirjhar Roy (IBM) wrote:
>>>> On 2/1/25 12:05, Zorro Lang wrote:
>>>>> On Fri, Jan 31, 2025 at 08:24:57AM -0800, Darrick J. Wong wrote:
>>>>>> On Fri, Jan 31, 2025 at 06:49:50PM +0530, Nirjhar Roy (IBM) wrote:
>>>>>>> On 1/29/25 21:32, Darrick J. Wong wrote:
>>>>>>>> On Wed, Jan 29, 2025 at 04:48:10PM +0530, Nirjhar Roy (IBM) wrote:
>>>>>>>>> On 1/28/25 23:39, Darrick J. Wong wrote:
>>>>>>>>>> On Tue, Jan 28, 2025 at 05:00:22AM +0000, Nirjhar Roy (IBM) wrote:
>>>>>>>>>>> Bug Description:
>>>>>>>>>>>
>>>>>>>>>>> _test_mount function is failing with the following error:
>>>>>>>>>>> ./common/rc: line 4716: _xfs_prepare_for_eio_shutdown: command not found
>>>>>>>>>>> check: failed to mount /dev/loop0 on /mnt1/test
>>>>>>>>>>>
>>>>>>>>>>> when the second section in local.config file is xfs and the first section
>>>>>>>>>>> is non-xfs.
>>>>>>>>>>>
>>>>>>>>>>> It can be easily reproduced with the following local.config file
>>>>>>>>>>>
>>>>>>>>>>> [s2]
>>>>>>>>>>> export FSTYP=ext4
>>>>>>>>>>> export TEST_DEV=/dev/loop0
>>>>>>>>>>> export TEST_DIR=/mnt1/test
>>>>>>>>>>> export SCRATCH_DEV=/dev/loop1
>>>>>>>>>>> export SCRATCH_MNT=/mnt1/scratch
>>>>>>>>>>>
>>>>>>>>>>> [s1]
>>>>>>>>>>> export FSTYP=xfs
>>>>>>>>>>> export TEST_DEV=/dev/loop0
>>>>>>>>>>> export TEST_DIR=/mnt1/test
>>>>>>>>>>> export SCRATCH_DEV=/dev/loop1
>>>>>>>>>>> export SCRATCH_MNT=/mnt1/scratch
>>>>>>>>>>>
>>>>>>>>>>> ./check selftest/001
>>>>>>>>>>>
>>>>>>>>>>> Root cause:
>>>>>>>>>>> When _test_mount() is executed for the second section, the FSTYPE has
>>>>>>>>>>> already changed but the new fs specific common/$FSTYP has not yet
>>>>>>>>>>> been done. Hence _xfs_prepare_for_eio_shutdown() is not found and
>>>>>>>>>>> the test run fails.
>>>>>>>>>>>
>>>>>>>>>>> Fix:
>>>>>>>>>>> Remove the additional _test_mount in check file just before ". commom/rc"
>>>>>>>>>>> since ". commom/rc" is already sourcing fs specific imports and doing a
>>>>>>>>>>> _test_mount.
>>>>>>>>>>>
>>>>>>>>>>> Fixes: 1a49022fab9b4 ("fstests: always use fail-at-unmount semantics for XFS")
>>>>>>>>>>> Signed-off-by: Nirjhar Roy (IBM) <nirjhar.roy.lists@gmail.com>
>>>>>>>>>>> ---
>>>>>>>>>>>       check | 12 +++---------
>>>>>>>>>>>       1 file changed, 3 insertions(+), 9 deletions(-)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/check b/check
>>>>>>>>>>> index 607d2456..5cb4e7eb 100755
>>>>>>>>>>> --- a/check
>>>>>>>>>>> +++ b/check
>>>>>>>>>>> @@ -784,15 +784,9 @@ function run_section()
>>>>>>>>>>>       			status=1
>>>>>>>>>>>       			exit
>>>>>>>>>>>       		fi
>>>>>>>>>>> -		if ! _test_mount
>>>>>>>>>> Don't we want to _test_mount the newly created filesystem still?  But
>>>>>>>>>> perhaps after sourcing common/rc ?
>>>>>>>>>>
>>>>>>>>>> --D
>>>>>>>>> common/rc calls init_rc() in the end and init_rc() already does a
>>>>>>>>> _test_mount. _test_mount after sourcing common/rc will fail, won't it? Does
>>>>>>>>> that make sense?
>>>>>>>>>
>>>>>>>>> init_rc()
>>>>>>>>> {
>>>>>>>>>         # make some further configuration checks here
>>>>>>>>>         if [ "$TEST_DEV" = ""  ]
>>>>>>>>>         then
>>>>>>>>>             echo "common/rc: Error: \$TEST_DEV is not set"
>>>>>>>>>             exit 1
>>>>>>>>>         fi
>>>>>>>>>
>>>>>>>>>         # if $TEST_DEV is not mounted, mount it now as XFS
>>>>>>>>>         if [ -z "`_fs_type $TEST_DEV`" ]
>>>>>>>>>         then
>>>>>>>>>             # $TEST_DEV is not mounted
>>>>>>>>>             if ! _test_mount
>>>>>>>>>             then
>>>>>>>>>                 echo "common/rc: retrying test device mount with external set"
>>>>>>>>>                 [ "$USE_EXTERNAL" != "yes" ] && export USE_EXTERNAL=yes
>>>>>>>>>                 if ! _test_mount
>>>>>>>>>                 then
>>>>>>>>>                     echo "common/rc: could not mount $TEST_DEV on $TEST_DIR"
>>>>>>>>>                     exit 1
>>>>>>>>>                 fi
>>>>>>>>>             fi
>>>>>>>>>         fi
>>>>>>>>> ...
>>>>>>>> ahahahaha yes it does.
>>>>>>>>
>>>>>>>> /commit message reading comprehension fail, sorry about that.
>>>>>>>>
>>>>>>>> Though now that you point it out, should check elide the init_rc call
>>>>>>>> about 12 lines down if it re-sourced common/rc ?
>>>>>>> Yes, it should. init_rc() is getting called twice when common/rc is getting
>>>>>>> re-sourced. Maybe I can do like
>>>>>>>
>>>>>>>
>>>>>>> if $RECREATE_TEST_DEV || [ "$OLD_FSTYP" != "$FSTYP" ]; then
>>>>>>>
>>>>>>>        <...>
>>>>>>>
>>>>>>>        . common/rc # changes in this patch
>>>>>>>
>>>>>>>        <...>
>>>>>>>
>>>>>>> elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then
>>>>>>>
>>>>>>>        ...
>>>>>>>
>>>>>>>        init_rc() # explicitly adding an init_rc() for this condition
>>>>>>>
>>>>>>> else
>>>>>>>
>>>>>>>        init_rc() # # explicitly adding an init_rc() for all other conditions.
>>>>>>> This will prevent init_rc() from getting called twice during re-sourcing
>>>>>>> common/rc
>>>>>>>
>>>>>>> fi
>>>>>>>
>>>>>>> What do you think?
>>>>>> Sounds fine as a mechanical change, but I wonder, should calling init_rc
>>>>>> be explicit?  There are not so many places that source common/rc:
>>>>>>
>>>>>> $ git grep 'common/rc'
>>>>>> check:362:if ! . ./common/rc; then
>>>>>> check:836:              . common/rc
>>>>>> common/preamble:52:     . ./common/rc
>>>>>> soak:7:. ./common/rc
>>>>>> tests/generic/749:18:. ./common/rc
>>>>>>
>>>>>> (I filtered out the non-executable matches)
>>>>>>
>>>>>> I think the call in generic/749 is unnecessary and I don't know what
>>>>>> soak does.  But that means that one could insert an explicit call to
>>>>>> init_rc at line 366 and 837 in check and at line 53 in common/preamble,
>>>>>> and we can clean up one more of those places where sourcing a common/
>>>>>> file actually /does/ something quietly under the covers.
>>>>>>
>>>>>> (Unless the maintainer is ok with the status quo...?)
>>>>> I think people just hope to import the helpers in common/rc mostly, don't
>>>>> want to run init_rc again. Maybe we can make sure the init_rc is only run
>>>>> once each time?
>>>>>
>>>>> E.g.
>>>>>
>>>>>      if [ _INIT_RC != "done" ];then
>>>>> 	init_rc
>>>>> 	_INIT_RC="done"
>>>>>      fi
>>>>>
>>>>> Or any better idea.
>>>> Yes, this idea looks good too. However, after thinking a bit more, I like
>>>> Darrick's idea to remove the call to init_rc from common/rc and explicitly
>>>> calling them explicitly whenever necessary makes more sense. This also makes
>>>> the interface/reason to source common/rc more meaningful, and also not
>>>> making common/rc do something via init_rc silently. What do you think?
>>> Sorry I'm on a travel, reply you late. I don't like to run codes in include
>>> files either :) If we remove the init_rc calling from common/rc we might
>>> need to do 2 things:
>>> 1) xfstests/check needs to run init_rc, calls it in check properly.
>>> 2) Now each sub-cases run init_rc when they import common/rc, I think
>>> we can call init_rc in common/preamble:_begin_fstest().
>> Sorry for my delayed reply, I got caught up with some other work items.
>> Thank you for your above suggestions. Let me go through them, look for some
>> edge cases and I can come up with a patch after some proper testing.
> No problem :) I just suggested, but the thing is we must figure out which
> ". common/rc" hopes to run init_rc, and which not :) Thanks for looking
> into it and test it.

Yes, I will look into this. Thank you.

--NR

>
>> Regards,
>>
>> --NR
>>
>>> If I miss other things, please feel free to remind me:)
>>>
>>> Thanks,
>>> Zorro
>>>
>>>> --NR
>>>>
>>>>> Thanks,
>>>>> Zorro
>>>>>
>>>>>> --D
>>>>>>
>>>>>>>> Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
>>>>>>>>
>>>>>>>> --D
>>>>>>>>
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> --NR
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> -		then
>>>>>>>>>>> -			echo "check: failed to mount $TEST_DEV on $TEST_DIR"
>>>>>>>>>>> -			status=1
>>>>>>>>>>> -			exit
>>>>>>>>>>> -		fi
>>>>>>>>>>> -		# TEST_DEV has been recreated, previous FSTYP derived from
>>>>>>>>>>> -		# TEST_DEV could be changed, source common/rc again with
>>>>>>>>>>> -		# correct FSTYP to get FSTYP specific configs, e.g. common/xfs
>>>>>>>>>>> +		# Previous FSTYP derived from TEST_DEV could be changed, source
>>>>>>>>>>> +		# common/rc again with correct FSTYP to get FSTYP specific configs,
>>>>>>>>>>> +		# e.g. common/xfs
>>>>>>>>>>>       		. common/rc
>>>>>>>>>>>       		_prepare_test_list
>>>>>>>>>>>       	elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then
>>>>>>>>>>> -- 
>>>>>>>>>>> 2.34.1
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Nirjhar Roy
>>>>>>>>> Linux Kernel Developer
>>>>>>>>> IBM, Bangalore
>>>>>>>>>
>>>>>>> -- 
>>>>>>> Nirjhar Roy
>>>>>>> Linux Kernel Developer
>>>>>>> IBM, Bangalore
>>>>>>>
>>>>>>>
>>>> -- 
>>>> Nirjhar Roy
>>>> Linux Kernel Developer
>>>> IBM, Bangalore
>>>>
>> -- 
>> Nirjhar Roy
>> Linux Kernel Developer
>> IBM, Bangalore
>>
-- 
Nirjhar Roy
Linux Kernel Developer
IBM, Bangalore