Re: [bug report] fstests generic/774 hang

["Darrick J. Wong" <djwong@kernel.org>]

On Fri, Nov 07, 2025 at 02:27:50AM +0000, Shinichiro Kawasaki wrote:
> On Nov 06, 2025 / 08:53, John Garry wrote:
> > > > > > 
> > > > > > Shinichiro, do the other atomic writes tests run ok, like 775, 767? You
> > > > > > can check group "atomicwrites" to know which tests they are.
> > > > > > 
> > > > > > 774 is the fio test.
> > > 
> > > I tried the other "atomicwrites" test. I found g778 took very long time.
> > > I think it implies that g778 may have similar problem as g774.
> > > 
> > >    g765: [not run] write atomic not supported by this block device
> > >    g767: 11s
> > >    g768: 13s
> > >    g769: 13s
> > >    g770: 35s
> > >    g773: [not run] write atomic not supported by this block device
> > >    g774: did not completed after 3 hours run (and kernel reported the INFO messages)
> > >    g775: 48s
> > >    g776: [not run] write atomic not supported by this block device
> > >    g778: did not completed after 50 minutes run
> > >    x838: [not run] External volumes not in use, skipped this test
> > >    x839: [not run] XFS error injection requires CONFIG_XFS_DEBUG
> > >    x840: [not run] write atomic not supported by this block device
> > 
> > This is testing software-based atomic writes, and they are just slow. Very
> > slow, relative to HW-based atomic writes. And having bs=1M will make things
> > worse, as we are locking out other threads for longer (when doing the
> > write).
> 
> I see, thanks for the explanation.
> 
> > So I think that we should limit the file size which we try to write.
> 
> This sounds reasonable, and it will make fstests run maintenance work easier.
> 
> > 
> > > 
> > > > > > 
> > > > > > Some things to try:
> > > > > > - use a physical disk for the TEST_DEV
> > > 
> > > I tried using a real HDD for TEST_DEV, but still observed the hang and INFO
> > > messages at g774.
> > > 
> > > > > > - Don't set LOAD_FACTOR (if you were setting it). If not, bodge 774 to
> > > > > > reduce $threads to a low value, say, 2
> > > 
> > > I do not set LOAD_FACTOR. I changed g775 script to set threads=2, then the
> > > test case completed quickly, within a few minutes. I'm suspecting that this
> > > short test time might hide the hang/INFO problem.
> > > 
> > > > > > - trying turning on XFS_DEBUG config
> > > 
> > > I turned on XFS_DEBUG, and still observed the hang and the INFO messages.
> > > 
> > 
> > I don't think that this will help.
> > 
> > > > > > 
> > > > > > BTW, Darrick has posted some xfs atomics fixes @ https://urldefense.com/
> > > > > > v3/__https://urldefense.com/v3/__https://lore.kernel.org/linux-__;!!ACWV5N9M2RV99hQ!J3HKTWLF8Qx-j42OOJ4o1YAttSSoqOCm9ymJtisUYoOtGgOyNNGqHnjjl1Zd9DQXJvCz8zqPMG-kgeVdo9MQuupMlcAo$
> > > > > > xfs/20251105001200.GV196370@frogsfrogsfrogs/T/*t__;Iw!!ACWV5N9M2RV99hQ! IuEPY6yJ1ZEQu7dpfjUplkPJucOHMQ9cpPvIC4fiJhTi_X_7ImN0t6wGqxg9_GM6gWe4B1OBiBjEI8Gz_At0595tIQ$
> > > > > > . I doubt that they will help this, but worth trying.
> > > 
> > > I have not yet tried this. Will try it tomorrow.
> > 
> > Nor this.
> 
> I confirmed it. I applied the patches to v6.18-rc4 kernel. With this kernel, the
> hang and the INFO messages are recreated.
> 
> > 
> > Having a hang - even for the conditions set - should not produce a hang. I
> > can check on whether we can improve the software-based atomic writes in xfs
> > to avoid this.
> 
> Thanks. Will sysrq-t output help? If it helps, I can take it from the hanging
> test node and share.

Yes, anything you can share would be helpful.  FWIW the test runs in 51
seconds here, but I only have 4 CPUs in the VM and fast storage so its
filesize is "only" 800MB.

--D