* Golden output mismatch from generic/228, fs independent
@ 2025-09-02 10:30 Qu Wenruo
2025-09-02 12:09 ` Theodore Ts'o
0 siblings, 1 reply; 5+ messages in thread
From: Qu Wenruo @ 2025-09-02 10:30 UTC (permalink / raw)
To: fstests@vger.kernel.org, linux-btrfs, linux-xfs
Hi,
Recently I updated my arm64 VM, and now several test cases are failing
due to golden output mismatch.
This time it's fs independent, and I haven't yet updated fstests itself,
so it looks like some updates in my environment is breaking the test.
E.g, generic/228 on ext4 (the same on btrfs)
FSTYP -- ext4
PLATFORM -- Linux/aarch64 btrfs-aarch64 6.17.0-rc3-custom+ #132 SMP
PREEMPT_DYNAMIC Mon Sep 1 08:38:46 ACST 2025
MKFS_OPTIONS -- -F /dev/mapper/test-scratch1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/mapper/test-scratch1 /mnt/scratch
generic/228 0s ... - output mismatch (see
/home/adam/xfstests-dev/results//generic/228.out.bad)
--- tests/generic/228.out 2024-04-25 18:13:45.126552201 +0930
+++ /home/adam/xfstests-dev/results//generic/228.out.bad 2025-09-02
19:51:22.806635177 +0930
@@ -1,6 +1,6 @@
QA output created by 228
File size limit is now set to 100 MB.
Let us try to preallocate 101 MB. This should fail.
-File size limit exceeded
+File size limit exceeded $XFS_IO_PROG -f -c 'falloc 0 101m'
$TEST_DIR/ouch
Let us now try to preallocate 50 MB. This should succeed.
Test over.
...
(Run 'diff -u /home/adam/xfstests-dev/tests/generic/228.out
/home/adam/xfstests-dev/results//generic/228.out.bad' to see the entire
diff)
Ran: generic/228
Failures: generic/228
Failed 1 of 1 tests
Or generic/733 on btrfs:
generic/733 7s ... - output mismatch (see
/home/adam/xfstests-dev/results//generic/733.out.bad)
--- tests/generic/733.out 2024-04-25 18:13:45.203549435 +0930
+++ /home/adam/xfstests-dev/results//generic/733.out.bad 2025-09-02
19:59:07.858861039 +0930
@@ -2,5 +2,5 @@
Format and mount
Create a many-block file
Reflink the big file
-Terminated
+Terminated $here/src/t_reflink_read_race
"$testdir/file1" "$testdir/file2" "$testdir/outcome" &>> $seqres.full
test completed successfully
...
(Run 'diff -u /home/adam/xfstests-dev/tests/generic/733.out
/home/adam/xfstests-dev/results//generic/733.out.bad' to see the entire
diff)
HINT: You _MAY_ be missing kernel fix:
5d6f0e9890ed btrfs: stop locking the source extent range during
reflink
I didn't know why, but the when those commands failed, the full script
line triggering it is also shown.
I checked my log, bash/xfsprogs and a lot of other packages are all
updated, and unfortunately my distro doesn't provide older packages for
me to bisect...
Thanks,
Qu
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Golden output mismatch from generic/228, fs independent
2025-09-02 10:30 Golden output mismatch from generic/228, fs independent Qu Wenruo
@ 2025-09-02 12:09 ` Theodore Ts'o
2025-09-02 16:48 ` Theodore Ts'o
2025-09-02 21:40 ` Qu Wenruo
0 siblings, 2 replies; 5+ messages in thread
From: Theodore Ts'o @ 2025-09-02 12:09 UTC (permalink / raw)
To: Qu Wenruo; +Cc: fstests@vger.kernel.org, linux-btrfs, linux-xfs
On Tue, Sep 02, 2025 at 08:00:32PM +0930, Qu Wenruo wrote:
> Hi,
>
> Recently I updated my arm64 VM, and now several test cases are failing due
> to golden output mismatch.
>
> This time it's fs independent, and I haven't yet updated fstests itself, so
> it looks like some updates in my environment is breaking the test.
>
> E.g, generic/228 on ext4 (the same on btrfs)
>
> I checked my log, bash/xfsprogs and a lot of other packages are all updated,
> and unfortunately my distro doesn't provide older packages for me to
> bisect...
I don't know if this helps, but here's a kvm-xfstests using Debian
Trixie and certain updated critical packages (fio, quota, xfsprogs,
util-linux, etc.) overriden. How does that differ from your distro
package versions?
I haven't updated my arm64 image in a bit (this was from dated July 20th).
I can try doing an arm64 rebuild and see if a newer version still
works on arm64, but here's a data point....
- TEd
KERNEL: kernel 6.17.0-rc3-xfstests #1 SMP Tue Sep 2 07:57:28 EDT 2025 aarch64
CMDLINE: --arm64 -c ext4/4k generic/228
CPUS: 2
MEM: 1977.09
ext4/4k: 1 tests, 5 seconds
generic/228 Pass 2s
Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 2s
FSTESTVER: blktests 401420a (Fri, 6 Jun 2025 22:12:43 +0900)
FSTESTVER: fio fio-3.40 (Tue, 20 May 2025 12:23:01 -0600)
FSTESTVER: fsverity v1.6-2-gee7d74d (Mon, 17 Feb 2025 11:41:58 -0800)
FSTESTVER: ima-evm-utils v1.5 (Mon, 6 Mar 2023 07:40:07 -0500)
FSTESTVER: libaio libaio-0.3.108-82-gb8eadc9 (Thu, 2 Jun 2022 13:33:11 +0200)
FSTESTVER: ltp 20250130-280-g60656cbbb (Wed, 28 May 2025 15:04:44 +0200)
FSTESTVER: quota v4.05-71-g4cd93fc (Sun, 27 Apr 2025 08:24:24 -0400)
FSTESTVER: util-linux v2.41-40-g22b91501d (Mon, 26 May 2025 11:27:31 +0200)
FSTESTVER: xfsprogs v6.15.0 (Mon, 23 Jun 2025 13:56:41 +0200)
FSTESTVER: xfstests v2025.07.13-12-gef63d1368 (Sat, 19 Jul 2025 18:14:29 -0400)
FSTESTVER: xfstests-bld gce-xfstests-202504292206-20-g905451c1 (Sun, 20 Jul 2025 03:04:27 +0000)
FSTESTVER: zz_build-distro trixie
FSTESTCFG: ext4/4k
FSTESTSET: generic/228
FSTESTOPT: aex
Truncating test artifacts in /results to 31k
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Golden output mismatch from generic/228, fs independent
2025-09-02 12:09 ` Theodore Ts'o
@ 2025-09-02 16:48 ` Theodore Ts'o
2025-09-02 21:40 ` Qu Wenruo
1 sibling, 0 replies; 5+ messages in thread
From: Theodore Ts'o @ 2025-09-02 16:48 UTC (permalink / raw)
To: Qu Wenruo; +Cc: fstests@vger.kernel.org, linux-btrfs, linux-xfs
On Tue, Sep 02, 2025 at 08:09:32AM -0400, Theodore Ts'o wrote:
> I haven't updated my arm64 image in a bit (this was from dated July 20th).
> I can try doing an arm64 rebuild and see if a newer version still
> works on arm64, but here's a data point....
And here's an update with the latest xfstests for-next branch, and
6.16 xfsprogs, etc. Still no problems for me....
- Ted
KERNEL: kernel 6.17.0-rc3-xfstests #1 SMP Tue Sep 2 07:57:28 EDT 2025 aarch64
CMDLINE: --arm64 -c ext4/4k generic/228
CPUS: 2
MEM: 1977.09
ext4/4k: 1 tests, 5 seconds
generic/228 Pass 2s
Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 2s
FSTESTVER: blktests 8ac6c4f (Sun, 31 Aug 2025 18:34:41 +0900)
FSTESTVER: fio fio-3.40 (Tue, 20 May 2025 12:23:01 -0600)
FSTESTVER: fsverity v1.6-2-gee7d74d (Mon, 17 Feb 2025 11:41:58 -0800)
FSTESTVER: ima-evm-utils v1.5 (Mon, 6 Mar 2023 07:40:07 -0500)
FSTESTVER: libaio libaio-0.3.108-82-gb8eadc9 (Thu, 2 Jun 2022 13:33:11 +0200)
FSTESTVER: ltp 20250530-178-g9691c4b2b (Mon, 1 Sep 2025 12:01:57 +0200)
FSTESTVER: quota v4.05-77-g22ff3d9 (Tue, 2 Sep 2025 08:12:02 -0400)
FSTESTVER: util-linux v2.41.1 (Tue, 24 Jun 2025 09:55:28 +0200)
FSTESTVER: xfsprogs v6.16.0 (Tue, 26 Aug 2025 20:27:25 +0200)
FSTESTVER: xfstests v2025.08.17-10-g057be3ead-dirty (Mon, 25 Aug 2025 11:59:43 -0400)
FSTESTVER: xfstests-bld gce-xfstests-202504292206-22-ge10dc029-dirty (Mon, 25 Aug 2025 12:33:01 -0400)
FSTESTVER: zz_build-distro trixie
FSTESTCFG: ext4/4k
FSTESTSET: generic/228
FSTESTOPT: aex
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Golden output mismatch from generic/228, fs independent
2025-09-02 12:09 ` Theodore Ts'o
2025-09-02 16:48 ` Theodore Ts'o
@ 2025-09-02 21:40 ` Qu Wenruo
2025-09-04 1:29 ` Qu Wenruo
1 sibling, 1 reply; 5+ messages in thread
From: Qu Wenruo @ 2025-09-02 21:40 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: fstests@vger.kernel.org, linux-btrfs, linux-xfs
在 2025/9/2 21:39, Theodore Ts'o 写道:
> On Tue, Sep 02, 2025 at 08:00:32PM +0930, Qu Wenruo wrote:
>> Hi,
>>
>> Recently I updated my arm64 VM, and now several test cases are failing due
>> to golden output mismatch.
>>
>> This time it's fs independent, and I haven't yet updated fstests itself, so
>> it looks like some updates in my environment is breaking the test.
>>
>> E.g, generic/228 on ext4 (the same on btrfs)
>>
>> I checked my log, bash/xfsprogs and a lot of other packages are all updated,
>> and unfortunately my distro doesn't provide older packages for me to
>> bisect...
>
> I don't know if this helps, but here's a kvm-xfstests using Debian
> Trixie and certain updated critical packages (fio, quota, xfsprogs,
> util-linux, etc.) overriden. How does that differ from your distro
> package versions?
>
> I haven't updated my arm64 image in a bit (this was from dated July 20th).
> I can try doing an arm64 rebuild and see if a newer version still
> works on arm64, but here's a data point....
>
> - TEd
>
>
> KERNEL: kernel 6.17.0-rc3-xfstests #1 SMP Tue Sep 2 07:57:28 EDT 2025 aarch64
> CMDLINE: --arm64 -c ext4/4k generic/228
> CPUS: 2
> MEM: 1977.09
>
> ext4/4k: 1 tests, 5 seconds
> generic/228 Pass 2s
> Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 2s
>
> FSTESTVER: blktests 401420a (Fri, 6 Jun 2025 22:12:43 +0900)
> FSTESTVER: fio fio-3.40 (Tue, 20 May 2025 12:23:01 -0600)
Fio is in fact newer than mine. Mine is 3.39.
> FSTESTVER: fsverity v1.6-2-gee7d74d (Mon, 17 Feb 2025 11:41:58 -0800)
> FSTESTVER: ima-evm-utils v1.5 (Mon, 6 Mar 2023 07:40:07 -0500)
> FSTESTVER: libaio libaio-0.3.108-82-gb8eadc9 (Thu, 2 Jun 2022 13:33:11 +0200)
Mine is a little newer, 0.3.113.
> FSTESTVER: ltp 20250130-280-g60656cbbb (Wed, 28 May 2025 15:04:44 +0200)
> FSTESTVER: quota v4.05-71-g4cd93fc (Sun, 27 Apr 2025 08:24:24 -0400)
> FSTESTVER: util-linux v2.41-40-g22b91501d (Mon, 26 May 2025 11:27:31 +0200)
> FSTESTVER: xfsprogs v6.15.0 (Mon, 23 Jun 2025 13:56:41 +0200)
> FSTESTVER: xfstests v2025.07.13-12-gef63d1368 (Sat, 19 Jul 2025 18:14:29 -0400)
> FSTESTVER: xfstests-bld gce-xfstests-202504292206-20-g905451c1 (Sun, 20 Jul 2025 03:04:27 +0000)
> FSTESTVER: zz_build-distro trixie
Others are mostly the same version.
And indeed it's bash, after rolling back to v5.2.037 bash, everything is
fine.
It looks like it's bash 5.3.x starting to fail (5.3.0 tested and failed).
I'll try dig a little deeper, and this is why I'm always running
Archlinux (ArchlinuxARM in this case), to give some early warnings of
some unexpected updates breaking the test cases.
Thanks,
Qu
> FSTESTCFG: ext4/4k
> FSTESTSET: generic/228
> FSTESTOPT: aex
> Truncating test artifacts in /results to 31k
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Golden output mismatch from generic/228, fs independent
2025-09-02 21:40 ` Qu Wenruo
@ 2025-09-04 1:29 ` Qu Wenruo
0 siblings, 0 replies; 5+ messages in thread
From: Qu Wenruo @ 2025-09-04 1:29 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: fstests@vger.kernel.org, linux-btrfs, linux-xfs
在 2025/9/3 07:10, Qu Wenruo 写道:
>
>
> 在 2025/9/2 21:39, Theodore Ts'o 写道:
>> On Tue, Sep 02, 2025 at 08:00:32PM +0930, Qu Wenruo wrote:
>>> Hi,
>>>
>>> Recently I updated my arm64 VM, and now several test cases are
>>> failing due
>>> to golden output mismatch.
>>>
>>> This time it's fs independent, and I haven't yet updated fstests
>>> itself, so
>>> it looks like some updates in my environment is breaking the test.
>>>
>>> E.g, generic/228 on ext4 (the same on btrfs)
>>>
>>> I checked my log, bash/xfsprogs and a lot of other packages are all
>>> updated,
>>> and unfortunately my distro doesn't provide older packages for me to
>>> bisect...
>>
>> I don't know if this helps, but here's a kvm-xfstests using Debian
>> Trixie and certain updated critical packages (fio, quota, xfsprogs,
>> util-linux, etc.) overriden. How does that differ from your distro
>> package versions?
>>
>> I haven't updated my arm64 image in a bit (this was from dated July
>> 20th).
>> I can try doing an arm64 rebuild and see if a newer version still
>> works on arm64, but here's a data point....
>>
>> - TEd
>>
>>
>> KERNEL: kernel 6.17.0-rc3-xfstests #1 SMP Tue Sep 2 07:57:28
>> EDT 2025 aarch64
>> CMDLINE: --arm64 -c ext4/4k generic/228
>> CPUS: 2
>> MEM: 1977.09
>>
>> ext4/4k: 1 tests, 5 seconds
>> generic/228 Pass 2s
>> Totals: 1 tests, 0 skipped, 0 failures, 0 errors, 2s
>>
>> FSTESTVER: blktests 401420a (Fri, 6 Jun 2025 22:12:43 +0900)
>> FSTESTVER: fio fio-3.40 (Tue, 20 May 2025 12:23:01 -0600)
>
> Fio is in fact newer than mine. Mine is 3.39.
>
>> FSTESTVER: fsverity v1.6-2-gee7d74d (Mon, 17 Feb 2025 11:41:58 -0800)
>> FSTESTVER: ima-evm-utils v1.5 (Mon, 6 Mar 2023 07:40:07 -0500)
>> FSTESTVER: libaio libaio-0.3.108-82-gb8eadc9 (Thu, 2 Jun 2022
>> 13:33:11 +0200)
>
> Mine is a little newer, 0.3.113.
>
>> FSTESTVER: ltp 20250130-280-g60656cbbb (Wed, 28 May 2025
>> 15:04:44 +0200)
>> FSTESTVER: quota v4.05-71-g4cd93fc (Sun, 27 Apr 2025
>> 08:24:24 -0400)
>> FSTESTVER: util-linux v2.41-40-g22b91501d (Mon, 26 May 2025 11:27:31
>> +0200)
>> FSTESTVER: xfsprogs v6.15.0 (Mon, 23 Jun 2025 13:56:41 +0200)
>> FSTESTVER: xfstests v2025.07.13-12-gef63d1368 (Sat, 19 Jul 2025
>> 18:14:29 -0400)
>> FSTESTVER: xfstests-bld gce-xfstests-202504292206-20-g905451c1 (Sun,
>> 20 Jul 2025 03:04:27 +0000)
>> FSTESTVER: zz_build-distro trixie
>
> Others are mostly the same version.
>
>
> And indeed it's bash, after rolling back to v5.2.037 bash, everything is
> fine.
>
> It looks like it's bash 5.3.x starting to fail (5.3.0 tested and failed).
>
> I'll try dig a little deeper, and this is why I'm always running
> Archlinux (ArchlinuxARM in this case), to give some early warnings of
> some unexpected updates breaking the test cases.
It turns out to be the core dump behavior, the following script can
easily reproduce it:
```
#!/bin/bash
dev="/dev/test/scratch1"
mnt="/mnt/btrfs"
mkfs.btrfs -f $dev > /dev/null
mount $dev $mnt
sysctl -w kernel.core_pattern=core &>/dev/null
ulimit -c 0
ulimit -f 102400
xfs_io -f -c "falloc 0 101m" /mnt/btrfs/foobar
umount $mnt
```
Xfs_io will trigger coredump, and the full cmdline dump belongs to the
core dump handling of bash.
Thanks,
Qu
>
> Thanks,
> Qu
>
>> FSTESTCFG: ext4/4k
>> FSTESTSET: generic/228
>> FSTESTOPT: aex
>> Truncating test artifacts in /results to 31k
>>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2025-09-04 1:29 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-02 10:30 Golden output mismatch from generic/228, fs independent Qu Wenruo
2025-09-02 12:09 ` Theodore Ts'o
2025-09-02 16:48 ` Theodore Ts'o
2025-09-02 21:40 ` Qu Wenruo
2025-09-04 1:29 ` Qu Wenruo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox