Re: [PATCH] fstests: disable generic duperemove tests for NFS and others

Re: [PATCH] fstests: disable generic duperemove tests for NFS and others

[Zorro Lang]

On Sun, Dec 08, 2024 at 01:07:18PM -0500, cel@kernel.org wrote:
> From: Chuck Lever <chuck.lever@oracle.com>
> 
> On NFS mounts, at least, generic/559, 560, and 561 run for a very
> long time, and usually fail.
> 
> The above tests already gate on whether duperemove is installed on
> the test system, but when fstests is installed as part of an
> automated workflow designed to handle many filesystem types,
> duperemove is installed by default.
> 
> duperemove(8) states:
> 
>   Deduplication is currently only supported by the btrfs and xfs
>   filesystem.

If so, I'm good to limit this test on btrfs and xfs. It might be better to
add this comment to "_supported_fs btrfs xfs". Anyway,

Reviewed-by: Zorro Lang <zlang@redhat.com>

(This's a fstests patch, send to fstests@vger.kernel.org.)

Thanks,
Zorro

> 
> Ensure that the generic dedupe tests are run on only filesystems
> where duperemove is known to work.
> 
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> ---
>  tests/generic/559 | 1 +
>  tests/generic/560 | 1 +
>  tests/generic/561 | 1 +
>  3 files changed, 3 insertions(+)
> 
> diff --git a/tests/generic/559 b/tests/generic/559
> index 28cf2e1a32c2..cf80be92142d 100755
> --- a/tests/generic/559
> +++ b/tests/generic/559
> @@ -13,6 +13,7 @@ _begin_fstest auto stress dedupe
>  . ./common/filter
>  . ./common/reflink
>  
> +_supported_fs btrfs xfs
>  _require_scratch_duperemove
>  
>  fssize=$((2 * 1024 * 1024 * 1024))
> diff --git a/tests/generic/560 b/tests/generic/560
> index 067d3ec0049e..a94b512efda1 100755
> --- a/tests/generic/560
> +++ b/tests/generic/560
> @@ -15,6 +15,7 @@ _begin_fstest auto stress dedupe
>  . ./common/filter
>  . ./common/reflink
>  
> +_supported_fs btrfs xfs
>  _require_scratch_duperemove
>  
>  _scratch_mkfs > $seqres.full 2>&1
> diff --git a/tests/generic/561 b/tests/generic/561
> index afe727ac56cb..da5f111c5b23 100755
> --- a/tests/generic/561
> +++ b/tests/generic/561
> @@ -28,6 +28,7 @@ _cleanup()
>  . ./common/filter
>  . ./common/reflink
>  
> +_supported_fs btrfs xfs
>  _require_scratch_duperemove
>  
>  _scratch_mkfs > $seqres.full 2>&1
> -- 
> 2.47.0
> 
> 

Re: [PATCH] fstests: disable generic duperemove tests for NFS and others

[Chuck Lever]

On 12/9/24 1:45 PM, Zorro Lang wrote:
> On Sun, Dec 08, 2024 at 01:07:18PM -0500, cel@kernel.org wrote:
>> From: Chuck Lever <chuck.lever@oracle.com>
>>
>> On NFS mounts, at least, generic/559, 560, and 561 run for a very
>> long time, and usually fail.
>>
>> The above tests already gate on whether duperemove is installed on
>> the test system, but when fstests is installed as part of an
>> automated workflow designed to handle many filesystem types,
>> duperemove is installed by default.
>>
>> duperemove(8) states:
>>
>>    Deduplication is currently only supported by the btrfs and xfs
>>    filesystem.
> 
> If so, I'm good to limit this test on btrfs and xfs. It might be better to
> add this comment to "_supported_fs btrfs xfs". Anyway,
> 
> Reviewed-by: Zorro Lang <zlang@redhat.com>
> 
> (This's a fstests patch, send to fstests@vger.kernel.org.)

Noted.

Christoph requested additional root cause analysis to see why the
existing feature check utilities are not blocking these tests on NFS.
I plan to look into that and repost if needed.


> Thanks,
> Zorro
> 
>>
>> Ensure that the generic dedupe tests are run on only filesystems
>> where duperemove is known to work.
>>
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>   tests/generic/559 | 1 +
>>   tests/generic/560 | 1 +
>>   tests/generic/561 | 1 +
>>   3 files changed, 3 insertions(+)
>>
>> diff --git a/tests/generic/559 b/tests/generic/559
>> index 28cf2e1a32c2..cf80be92142d 100755
>> --- a/tests/generic/559
>> +++ b/tests/generic/559
>> @@ -13,6 +13,7 @@ _begin_fstest auto stress dedupe
>>   . ./common/filter
>>   . ./common/reflink
>>   
>> +_supported_fs btrfs xfs
>>   _require_scratch_duperemove
>>   
>>   fssize=$((2 * 1024 * 1024 * 1024))
>> diff --git a/tests/generic/560 b/tests/generic/560
>> index 067d3ec0049e..a94b512efda1 100755
>> --- a/tests/generic/560
>> +++ b/tests/generic/560
>> @@ -15,6 +15,7 @@ _begin_fstest auto stress dedupe
>>   . ./common/filter
>>   . ./common/reflink
>>   
>> +_supported_fs btrfs xfs
>>   _require_scratch_duperemove
>>   
>>   _scratch_mkfs > $seqres.full 2>&1
>> diff --git a/tests/generic/561 b/tests/generic/561
>> index afe727ac56cb..da5f111c5b23 100755
>> --- a/tests/generic/561
>> +++ b/tests/generic/561
>> @@ -28,6 +28,7 @@ _cleanup()
>>   . ./common/filter
>>   . ./common/reflink
>>   
>> +_supported_fs btrfs xfs
>>   _require_scratch_duperemove
>>   
>>   _scratch_mkfs > $seqres.full 2>&1
>> -- 
>> 2.47.0
>>
>>
> 


-- 
Chuck Lever

Re: [PATCH] fstests: disable generic duperemove tests for NFS and others

[Chuck Lever]

On 12/9/24 10:07 AM, Christoph Hellwig wrote:
> On Mon, Dec 09, 2024 at 10:00:58AM -0500, Chuck Lever wrote:
>> 2. How to disable this test on filesystems (like NFS) where duperemove
>> is not supported or where the test is not meaningful. The current check
>> for the presence of the duperemove executable is IMO inadequate.
> 
> I've not looked at dupremove, but the name suggested it's de-duplicating
> and thus needs a working FIDEDUPERANGE ioctl.  So the test should check
> for that using the _require_test_dedupe / _require_scratch_dedupe
> helpers. generic/{559,560,561} do that, so something either in these
> helpers is probably broken given that NFS rejects remap_file_range
> with REMAP_FILE_DEDUP, although a bit of tracing would be useful if
> there actually is something broken in NFS or the VFS.

It doesn't appear to be an NFS problem; duperemove is not even issuing
the ioctl as a toe-test to see if it will work.

I opened https://github.com/markfasheh/duperemove/issues/363


-- 
Chuck Lever