* a deadloop(?) when mkfs.xfs -o rtdev
@ 2025-07-17 0:13 Wang Yugui
2025-07-17 0:52 ` Wang Yugui
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Wang Yugui @ 2025-07-17 0:13 UTC (permalink / raw)
To: linux-xfs
Hi,
There seems a deadloop(?) when mkfs.xfs -o rtdev.
# mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512 agcount=20, agsize=1221072 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=0 bigtime=1 inobtcount=1 nrext64=1
= exchange=0
data = bsize=4096 blocks=24421440, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
log =internal log bsize=4096 blocks=41799, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096 blocks=390703446, rtextents=390703446
Discarding blocks...Done.
Discarding blocks...Done.
It did not fishish after 10 mins.
# pstack 5785
#0 0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
#1 0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
#2 0x0000557b7bbc3030 in libxfs_buf_read_map ()
#3 0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
#4 0x0000557b7bbee7ef in xfs_rtbuf_get ()
#5 0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
#6 0x0000557b7bbbb322 in parseproto.lto_priv ()
#7 0x0000557b7bbb7643 in main ()
And,
1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/17
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: a deadloop(?) when mkfs.xfs -o rtdev
2025-07-17 0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
@ 2025-07-17 0:52 ` Wang Yugui
2025-07-17 4:59 ` Christoph Hellwig
2025-07-17 16:03 ` Darrick J. Wong
2 siblings, 0 replies; 6+ messages in thread
From: Wang Yugui @ 2025-07-17 0:52 UTC (permalink / raw)
To: linux-xfs
Hi,
> There seems a deadloop(?) when mkfs.xfs -o rtdev.
>
> # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512 agcount=20, agsize=1221072 blks
> = sectsz=4096 attr=2, projid32bit=1
> = crc=1 finobt=1, sparse=1, rmapbt=0
> = reflink=0 bigtime=1 inobtcount=1 nrext64=1
> = exchange=0
> data = bsize=4096 blocks=24421440, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
> log =internal log bsize=4096 blocks=41799, version=2
> = sectsz=4096 sunit=1 blks, lazy-count=1
> realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096 blocks=390703446, rtextents=390703446
> Discarding blocks...Done.
> Discarding blocks...Done.
>
> It did not fishish after 10 mins.
>
> # pstack 5785
> #0 0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> #1 0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> #2 0x0000557b7bbc3030 in libxfs_buf_read_map ()
> #3 0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> #4 0x0000557b7bbee7ef in xfs_rtbuf_get ()
> #5 0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> #6 0x0000557b7bbbb322 in parseproto.lto_priv ()
> #7 0x0000557b7bbb7643 in main ()
>
> And,
> 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
it happen on xfsprogs-6.15.0 too.
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/17
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: a deadloop(?) when mkfs.xfs -o rtdev
2025-07-17 0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
2025-07-17 0:52 ` Wang Yugui
@ 2025-07-17 4:59 ` Christoph Hellwig
2025-07-17 16:03 ` Darrick J. Wong
2 siblings, 0 replies; 6+ messages in thread
From: Christoph Hellwig @ 2025-07-17 4:59 UTC (permalink / raw)
To: Wang Yugui; +Cc: linux-xfs
On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
>
> # pstack 5785
> #0 0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> #1 0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> #2 0x0000557b7bbc3030 in libxfs_buf_read_map ()
> #3 0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> #4 0x0000557b7bbee7ef in xfs_rtbuf_get ()
> #5 0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> #6 0x0000557b7bbbb322 in parseproto.lto_priv ()
> #7 0x0000557b7bbb7643 in main ()
This is reading the data from the main device for updating the bits on
the rt bitmap.
If you have a very larger rt device this can take a long time and might
not be stuck. And easy way to test that hypothesis would be to try
formatting with a very large RT exent size.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: a deadloop(?) when mkfs.xfs -o rtdev
2025-07-17 0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
2025-07-17 0:52 ` Wang Yugui
2025-07-17 4:59 ` Christoph Hellwig
@ 2025-07-17 16:03 ` Darrick J. Wong
2025-07-18 0:58 ` Wang Yugui
2 siblings, 1 reply; 6+ messages in thread
From: Darrick J. Wong @ 2025-07-17 16:03 UTC (permalink / raw)
To: Wang Yugui; +Cc: linux-xfs
On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> Hi,
>
> There seems a deadloop(?) when mkfs.xfs -o rtdev.
>
> # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512 agcount=20, agsize=1221072 blks
> = sectsz=4096 attr=2, projid32bit=1
> = crc=1 finobt=1, sparse=1, rmapbt=0
> = reflink=0 bigtime=1 inobtcount=1 nrext64=1
> = exchange=0
> data = bsize=4096 blocks=24421440, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
> log =internal log bsize=4096 blocks=41799, version=2
> = sectsz=4096 sunit=1 blks, lazy-count=1
> realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096 blocks=390703446, rtextents=390703446
> Discarding blocks...Done.
> Discarding blocks...Done.
>
> It did not fishish after 10 mins.
>
> # pstack 5785
> #0 0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> #1 0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> #2 0x0000557b7bbc3030 in libxfs_buf_read_map ()
> #3 0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> #4 0x0000557b7bbee7ef in xfs_rtbuf_get ()
> #5 0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> #6 0x0000557b7bbbb322 in parseproto.lto_priv ()
> #7 0x0000557b7bbb7643 in main ()
>
> And,
> 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
You /could/ strace the mkfs process to see if it's really stuck, or just
issuing IOs really slowly.
--D
>
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2025/07/17
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: a deadloop(?) when mkfs.xfs -o rtdev
2025-07-17 16:03 ` Darrick J. Wong
@ 2025-07-18 0:58 ` Wang Yugui
2025-07-18 1:08 ` Wang Yugui
0 siblings, 1 reply; 6+ messages in thread
From: Wang Yugui @ 2025-07-18 0:58 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs
Hi,
> On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> > Hi,
> >
> > There seems a deadloop(?) when mkfs.xfs -o rtdev.
> >
> > # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> > isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> > meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512 agcount=20, agsize=1221072 blks
> > = sectsz=4096 attr=2, projid32bit=1
> > = crc=1 finobt=1, sparse=1, rmapbt=0
> > = reflink=0 bigtime=1 inobtcount=1 nrext64=1
> > = exchange=0
> > data = bsize=4096 blocks=24421440, imaxpct=25
> > = sunit=0 swidth=0 blks
> > naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
> > log =internal log bsize=4096 blocks=41799, version=2
> > = sectsz=4096 sunit=1 blks, lazy-count=1
> > realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096 blocks=390703446, rtextents=390703446
> > Discarding blocks...Done.
> > Discarding blocks...Done.
> >
> > It did not fishish after 10 mins.
> >
> > # pstack 5785
> > #0 0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> > #1 0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> > #2 0x0000557b7bbc3030 in libxfs_buf_read_map ()
> > #3 0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> > #4 0x0000557b7bbee7ef in xfs_rtbuf_get ()
> > #5 0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> > #6 0x0000557b7bbbb322 in parseproto.lto_priv ()
> > #7 0x0000557b7bbb7643 in main ()
> >
> > And,
> > 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> > 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
>
> You /could/ strace the mkfs process to see if it's really stuck, or just
> issuing IOs really slowly.
Today the mkfs.xfs with rtdev all finished in 110s-160s on kernel 6.6.93/6.12.38.
the result of strace show that pread64() with 4K size so it is just slow.
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 868352) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 864256) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 860160) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 856064) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 851968) = 4096
Thanks a lot.
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/18
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: a deadloop(?) when mkfs.xfs -o rtdev
2025-07-18 0:58 ` Wang Yugui
@ 2025-07-18 1:08 ` Wang Yugui
0 siblings, 0 replies; 6+ messages in thread
From: Wang Yugui @ 2025-07-18 1:08 UTC (permalink / raw)
To: Darrick J. Wong, linux-xfs
Hi,
> Hi,
>
> > On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> > > Hi,
> > >
> > > There seems a deadloop(?) when mkfs.xfs -o rtdev.
> > >
> > > # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> > > isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> > > meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512 agcount=20, agsize=1221072 blks
> > > = sectsz=4096 attr=2, projid32bit=1
> > > = crc=1 finobt=1, sparse=1, rmapbt=0
> > > = reflink=0 bigtime=1 inobtcount=1 nrext64=1
> > > = exchange=0
> > > data = bsize=4096 blocks=24421440, imaxpct=25
> > > = sunit=0 swidth=0 blks
> > > naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
> > > log =internal log bsize=4096 blocks=41799, version=2
> > > = sectsz=4096 sunit=1 blks, lazy-count=1
> > > realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096 blocks=390703446, rtextents=390703446
> > > Discarding blocks...Done.
> > > Discarding blocks...Done.
> > >
> > > It did not fishish after 10 mins.
> > >
> > > # pstack 5785
> > > #0 0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> > > #1 0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> > > #2 0x0000557b7bbc3030 in libxfs_buf_read_map ()
> > > #3 0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> > > #4 0x0000557b7bbee7ef in xfs_rtbuf_get ()
> > > #5 0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> > > #6 0x0000557b7bbbb322 in parseproto.lto_priv ()
> > > #7 0x0000557b7bbb7643 in main ()
> > >
> > > And,
> > > 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> > > 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
> >
> > You /could/ strace the mkfs process to see if it's really stuck, or just
> > issuing IOs really slowly.
>
> Today the mkfs.xfs with rtdev all finished in 110s-160s on kernel 6.6.93/6.12.38.
>
> the result of strace show that pread64() with 4K size so it is just slow.
>
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 868352) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 864256) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 860160) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 856064) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 851968) = 4096
>
the offset of pread64() is in DECE order, not in ASCE order , so the readahead
does not help too.
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/18
> Thanks a lot.
>
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2025/07/18
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-07-18 1:39 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-17 0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
2025-07-17 0:52 ` Wang Yugui
2025-07-17 4:59 ` Christoph Hellwig
2025-07-17 16:03 ` Darrick J. Wong
2025-07-18 0:58 ` Wang Yugui
2025-07-18 1:08 ` Wang Yugui
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).