* generic/346 fails on ext4 DAX mount
@ 2017-06-02 5:33 Eryu Guan
2017-06-02 6:03 ` Ross Zwisler
2017-06-05 19:30 ` Ross Zwisler
0 siblings, 2 replies; 4+ messages in thread
From: Eryu Guan @ 2017-06-02 5:33 UTC (permalink / raw)
To: linux-fsdevel, linux-ext4; +Cc: Jan Kara, Ross Zwisler, Xiong Zhou
Hi all,
I occasionally hit generic/344 and generic/346 failures when testing
4.12-rc[1-3] kernels on ext4 DAX mount.
FSTYP -- ext4
PLATFORM -- Linux/x86_64 hp-xl420gen9-01 4.12.0-rc3
MKFS_OPTIONS -- /dev/pmem2
MOUNT_OPTIONS -- -o dax -o context=system_u:object_r:root_t:s0 /dev/pmem2 /mnt/testarea/scratch
generic/344 1s ... 1s
generic/346 1s ... - output mismatch (see /var/lib/xfstests/results//generic/346.out.bad)
--- tests/generic/346.out 2017-05-24 10:13:38.592436565 -0400
+++ /var/lib/xfstests/results//generic/346.out.bad 2017-06-01 12:46:50.122007818 -0400
@@ -10,7 +10,8 @@
INFO: sz = 1048576
INFO: thread 0 created
INFO: thread 1 created
-INFO: 0 error(s) detected
+ERROR: thread 0, offset 000ff400, 00000000 != 7f1068063700
+INFO: 1 error(s) detected
INFO: ftruncate test...
INFO: sz = 1048576
And it seems generic/346 is easer to hit, usually it can be reproduced
within 20 iterations on 4.12-rc kernels.
At first I thought it was a regression introduced in 4.12-rc1, but after
two failed bisects (pointed first bad to unrelated networking patch), I
enlarged the iteration count to 5000 and found that generic/346 failure
can also be seen on 4.11 and 4.10 kernel. I haven't tried other old
kernels yet. It's just much harder to hit on 4.10/4.11 kernels (need
hundreds of iterations).
But the failure could only be reproduced with ext4 DAX mount, XFS DAX
mount survived 5000 runs of generic/346 on 4.12-rc3 kernel.
I was testing with pmem device created by memmap kernel param
"memmap=10G!5G memmap=15G!15G", but it can be reproduced with brd
ramdisk too.
If more info is needed please let me know.
Thanks,
Eryu
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: generic/346 fails on ext4 DAX mount
2017-06-02 5:33 generic/346 fails on ext4 DAX mount Eryu Guan
@ 2017-06-02 6:03 ` Ross Zwisler
2017-06-05 19:30 ` Ross Zwisler
1 sibling, 0 replies; 4+ messages in thread
From: Ross Zwisler @ 2017-06-02 6:03 UTC (permalink / raw)
To: Eryu Guan; +Cc: linux-fsdevel, linux-ext4, Jan Kara, Ross Zwisler, Xiong Zhou
On Fri, Jun 02, 2017 at 01:33:19PM +0800, Eryu Guan wrote:
> Hi all,
>
> I occasionally hit generic/344 and generic/346 failures when testing
> 4.12-rc[1-3] kernels on ext4 DAX mount.
>
> FSTYP -- ext4
> PLATFORM -- Linux/x86_64 hp-xl420gen9-01 4.12.0-rc3
> MKFS_OPTIONS -- /dev/pmem2
> MOUNT_OPTIONS -- -o dax -o context=system_u:object_r:root_t:s0 /dev/pmem2 /mnt/testarea/scratch
>
> generic/344 1s ... 1s
> generic/346 1s ... - output mismatch (see /var/lib/xfstests/results//generic/346.out.bad)
> --- tests/generic/346.out 2017-05-24 10:13:38.592436565 -0400
> +++ /var/lib/xfstests/results//generic/346.out.bad 2017-06-01 12:46:50.122007818 -0400
> @@ -10,7 +10,8 @@
> INFO: sz = 1048576
> INFO: thread 0 created
> INFO: thread 1 created
> -INFO: 0 error(s) detected
> +ERROR: thread 0, offset 000ff400, 00000000 != 7f1068063700
> +INFO: 1 error(s) detected
>
> INFO: ftruncate test...
> INFO: sz = 1048576
>
> And it seems generic/346 is easer to hit, usually it can be reproduced
> within 20 iterations on 4.12-rc kernels.
>
> At first I thought it was a regression introduced in 4.12-rc1, but after
> two failed bisects (pointed first bad to unrelated networking patch), I
> enlarged the iteration count to 5000 and found that generic/346 failure
> can also be seen on 4.11 and 4.10 kernel. I haven't tried other old
> kernels yet. It's just much harder to hit on 4.10/4.11 kernels (need
> hundreds of iterations).
>
> But the failure could only be reproduced with ext4 DAX mount, XFS DAX
> mount survived 5000 runs of generic/346 on 4.12-rc3 kernel.
>
> I was testing with pmem device created by memmap kernel param
> "memmap=10G!5G memmap=15G!15G", but it can be reproduced with brd
> ramdisk too.
>
> If more info is needed please let me know.
>
> Thanks,
> Eryu
Thank you for the report, I'll take a look.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: generic/346 fails on ext4 DAX mount
2017-06-02 5:33 generic/346 fails on ext4 DAX mount Eryu Guan
2017-06-02 6:03 ` Ross Zwisler
@ 2017-06-05 19:30 ` Ross Zwisler
2017-06-05 21:50 ` Theodore Ts'o
1 sibling, 1 reply; 4+ messages in thread
From: Ross Zwisler @ 2017-06-05 19:30 UTC (permalink / raw)
To: Eryu Guan
Cc: linux-fsdevel, linux-ext4, Jan Kara, Ross Zwisler, Xiong Zhou,
Theodore Ts'o
On Fri, Jun 02, 2017 at 01:33:19PM +0800, Eryu Guan wrote:
> Hi all,
>
> I occasionally hit generic/344 and generic/346 failures when testing
> 4.12-rc[1-3] kernels on ext4 DAX mount.
>
> FSTYP -- ext4
> PLATFORM -- Linux/x86_64 hp-xl420gen9-01 4.12.0-rc3
> MKFS_OPTIONS -- /dev/pmem2
> MOUNT_OPTIONS -- -o dax -o context=system_u:object_r:root_t:s0 /dev/pmem2 /mnt/testarea/scratch
>
> generic/344 1s ... 1s
> generic/346 1s ... - output mismatch (see /var/lib/xfstests/results//generic/346.out.bad)
> --- tests/generic/346.out 2017-05-24 10:13:38.592436565 -0400
> +++ /var/lib/xfstests/results//generic/346.out.bad 2017-06-01 12:46:50.122007818 -0400
> @@ -10,7 +10,8 @@
> INFO: sz = 1048576
> INFO: thread 0 created
> INFO: thread 1 created
> -INFO: 0 error(s) detected
> +ERROR: thread 0, offset 000ff400, 00000000 != 7f1068063700
> +INFO: 1 error(s) detected
>
> INFO: ftruncate test...
> INFO: sz = 1048576
>
> And it seems generic/346 is easer to hit, usually it can be reproduced
> within 20 iterations on 4.12-rc kernels.
>
> At first I thought it was a regression introduced in 4.12-rc1, but after
> two failed bisects (pointed first bad to unrelated networking patch), I
> enlarged the iteration count to 5000 and found that generic/346 failure
> can also be seen on 4.11 and 4.10 kernel. I haven't tried other old
> kernels yet. It's just much harder to hit on 4.10/4.11 kernels (need
> hundreds of iterations).
>
> But the failure could only be reproduced with ext4 DAX mount, XFS DAX
> mount survived 5000 runs of generic/346 on 4.12-rc3 kernel.
>
> I was testing with pmem device created by memmap kernel param
> "memmap=10G!5G memmap=15G!15G", but it can be reproduced with brd
> ramdisk too.
>
> If more info is needed please let me know.
>
> Thanks,
> Eryu
It turns out that this has already been fixed by Jan, with this commit:
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git/commit/?h=dev&id=4f8caa60a5a13a78f26198618f21774bd6aa6498
I have a test setup that was failing with this error very constantly within
100 iterations using v4.12-rc4. I was able to run 10,000+ iterations without
issue with v4.12-rc4 + that one patch.
That patch clearly isn't upstream yet, but I think it's headed for v4.12?
(Ted, Jan?) It's marked for stable as well.
It's currently in the ext4/dev tree:
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git/log/?h=dev
- Ross
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: generic/346 fails on ext4 DAX mount
2017-06-05 19:30 ` Ross Zwisler
@ 2017-06-05 21:50 ` Theodore Ts'o
0 siblings, 0 replies; 4+ messages in thread
From: Theodore Ts'o @ 2017-06-05 21:50 UTC (permalink / raw)
To: Ross Zwisler, Eryu Guan, linux-fsdevel, linux-ext4, Jan Kara,
Xiong Zhou
On Mon, Jun 05, 2017 at 01:30:22PM -0600, Ross Zwisler wrote:
>
> That patch clearly isn't upstream yet, but I think it's headed for v4.12?
> (Ted, Jan?) It's marked for stable as well.
Yes, it should be going to Linus this week, probably in the next day
or two.
- Ted
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-06-05 21:50 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-06-02 5:33 generic/346 fails on ext4 DAX mount Eryu Guan
2017-06-02 6:03 ` Ross Zwisler
2017-06-05 19:30 ` Ross Zwisler
2017-06-05 21:50 ` Theodore Ts'o
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).