a deadloop(?) when mkfs.xfs -o rtdev

linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* a deadloop(?) when mkfs.xfs -o rtdev
@ 2025-07-17  0:13 Wang Yugui
  2025-07-17  0:52 ` Wang Yugui
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Wang Yugui @ 2025-07-17  0:13 UTC (permalink / raw)
  To: linux-xfs

Hi,

There seems a deadloop(?) when mkfs.xfs -o rtdev.

# mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512    agcount=20, agsize=1221072 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
         =                       exchange=0
data     =                       bsize=4096   blocks=24421440, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=0
log      =internal log           bsize=4096   blocks=41799, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096   blocks=390703446, rtextents=390703446
Discarding blocks...Done.
Discarding blocks...Done.

It did not fishish after 10 mins.

# pstack 5785
#0  0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
#1  0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
#2  0x0000557b7bbc3030 in libxfs_buf_read_map ()
#3  0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
#4  0x0000557b7bbee7ef in xfs_rtbuf_get ()
#5  0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
#6  0x0000557b7bbbb322 in parseproto.lto_priv ()
#7  0x0000557b7bbb7643 in main ()

And,
1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/17



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: a deadloop(?) when mkfs.xfs -o rtdev
  2025-07-17  0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
@ 2025-07-17  0:52 ` Wang Yugui
  2025-07-17  4:59 ` Christoph Hellwig
  2025-07-17 16:03 ` Darrick J. Wong
  2 siblings, 0 replies; 6+ messages in thread
From: Wang Yugui @ 2025-07-17  0:52 UTC (permalink / raw)
  To: linux-xfs

Hi,

> There seems a deadloop(?) when mkfs.xfs -o rtdev.
> 
> # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512    agcount=20, agsize=1221072 blks
>          =                       sectsz=4096  attr=2, projid32bit=1
>          =                       crc=1        finobt=1, sparse=1, rmapbt=0
>          =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
>          =                       exchange=0
> data     =                       bsize=4096   blocks=24421440, imaxpct=25
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=0
> log      =internal log           bsize=4096   blocks=41799, version=2
>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096   blocks=390703446, rtextents=390703446
> Discarding blocks...Done.
> Discarding blocks...Done.
> 
> It did not fishish after 10 mins.
> 
> # pstack 5785
> #0  0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> #1  0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> #2  0x0000557b7bbc3030 in libxfs_buf_read_map ()
> #3  0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> #4  0x0000557b7bbee7ef in xfs_rtbuf_get ()
> #5  0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> #6  0x0000557b7bbbb322 in parseproto.lto_priv ()
> #7  0x0000557b7bbb7643 in main ()
> 
> And,
> 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.

it happen on xfsprogs-6.15.0 too.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/17


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: a deadloop(?) when mkfs.xfs -o rtdev
  2025-07-17  0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
  2025-07-17  0:52 ` Wang Yugui
@ 2025-07-17  4:59 ` Christoph Hellwig
  2025-07-17 16:03 ` Darrick J. Wong
  2 siblings, 0 replies; 6+ messages in thread
From: Christoph Hellwig @ 2025-07-17  4:59 UTC (permalink / raw)
  To: Wang Yugui; +Cc: linux-xfs

On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> 
> # pstack 5785
> #0  0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> #1  0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> #2  0x0000557b7bbc3030 in libxfs_buf_read_map ()
> #3  0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> #4  0x0000557b7bbee7ef in xfs_rtbuf_get ()
> #5  0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> #6  0x0000557b7bbbb322 in parseproto.lto_priv ()
> #7  0x0000557b7bbb7643 in main ()

This is reading the data from the main device for updating the bits on
the rt bitmap.

If you have a very larger rt device this can take a long time and might
not be stuck.  And easy way to test that hypothesis would be to try
formatting with a very large RT exent size.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: a deadloop(?) when mkfs.xfs -o rtdev
  2025-07-17  0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
  2025-07-17  0:52 ` Wang Yugui
  2025-07-17  4:59 ` Christoph Hellwig
@ 2025-07-17 16:03 ` Darrick J. Wong
  2025-07-18  0:58   ` Wang Yugui
  2 siblings, 1 reply; 6+ messages in thread
From: Darrick J. Wong @ 2025-07-17 16:03 UTC (permalink / raw)
  To: Wang Yugui; +Cc: linux-xfs

On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> Hi,
> 
> There seems a deadloop(?) when mkfs.xfs -o rtdev.
> 
> # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512    agcount=20, agsize=1221072 blks
>          =                       sectsz=4096  attr=2, projid32bit=1
>          =                       crc=1        finobt=1, sparse=1, rmapbt=0
>          =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
>          =                       exchange=0
> data     =                       bsize=4096   blocks=24421440, imaxpct=25
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=0
> log      =internal log           bsize=4096   blocks=41799, version=2
>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096   blocks=390703446, rtextents=390703446
> Discarding blocks...Done.
> Discarding blocks...Done.
> 
> It did not fishish after 10 mins.
> 
> # pstack 5785
> #0  0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> #1  0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> #2  0x0000557b7bbc3030 in libxfs_buf_read_map ()
> #3  0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> #4  0x0000557b7bbee7ef in xfs_rtbuf_get ()
> #5  0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> #6  0x0000557b7bbbb322 in parseproto.lto_priv ()
> #7  0x0000557b7bbb7643 in main ()
> 
> And,
> 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.

You /could/ strace the mkfs process to see if it's really stuck, or just
issuing IOs really slowly.

--D

> 
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2025/07/17
> 
> 
> 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: a deadloop(?) when mkfs.xfs -o rtdev
  2025-07-17 16:03 ` Darrick J. Wong
@ 2025-07-18  0:58   ` Wang Yugui
  2025-07-18  1:08     ` Wang Yugui
  0 siblings, 1 reply; 6+ messages in thread
From: Wang Yugui @ 2025-07-18  0:58 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: linux-xfs

Hi,

> On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> > Hi,
> > 
> > There seems a deadloop(?) when mkfs.xfs -o rtdev.
> > 
> > # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> > isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> > meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512    agcount=20, agsize=1221072 blks
> >          =                       sectsz=4096  attr=2, projid32bit=1
> >          =                       crc=1        finobt=1, sparse=1, rmapbt=0
> >          =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
> >          =                       exchange=0
> > data     =                       bsize=4096   blocks=24421440, imaxpct=25
> >          =                       sunit=0      swidth=0 blks
> > naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=0
> > log      =internal log           bsize=4096   blocks=41799, version=2
> >          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> > realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096   blocks=390703446, rtextents=390703446
> > Discarding blocks...Done.
> > Discarding blocks...Done.
> > 
> > It did not fishish after 10 mins.
> > 
> > # pstack 5785
> > #0  0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> > #1  0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> > #2  0x0000557b7bbc3030 in libxfs_buf_read_map ()
> > #3  0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> > #4  0x0000557b7bbee7ef in xfs_rtbuf_get ()
> > #5  0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> > #6  0x0000557b7bbbb322 in parseproto.lto_priv ()
> > #7  0x0000557b7bbb7643 in main ()
> > 
> > And,
> > 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> > 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
> 
> You /could/ strace the mkfs process to see if it's really stuck, or just
> issuing IOs really slowly.

Today the mkfs.xfs with rtdev all finished in 110s-160s on kernel 6.6.93/6.12.38.

the result of strace show that pread64() with 4K size so it is just slow.

pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 868352) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 864256) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 860160) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 856064) = 4096
pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 851968) = 4096

Thanks a lot.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/18



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: a deadloop(?) when mkfs.xfs -o rtdev
  2025-07-18  0:58   ` Wang Yugui
@ 2025-07-18  1:08     ` Wang Yugui
  0 siblings, 0 replies; 6+ messages in thread
From: Wang Yugui @ 2025-07-18  1:08 UTC (permalink / raw)
  To: Darrick J. Wong, linux-xfs

Hi,


> Hi,
> 
> > On Thu, Jul 17, 2025 at 08:13:33AM +0800, Wang Yugui wrote:
> > > Hi,
> > > 
> > > There seems a deadloop(?) when mkfs.xfs -o rtdev.
> > > 
> > > # mkfs.xfs -r rtdev=/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 /dev/d
> > > isk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA -f
> > > meta-data=/dev/disk/by-id/scsi-SHITACHI_HUSMH842_CLAR100_0LX0JWVA isize=512    agcount=20, agsize=1221072 blks
> > >          =                       sectsz=4096  attr=2, projid32bit=1
> > >          =                       crc=1        finobt=1, sparse=1, rmapbt=0
> > >          =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
> > >          =                       exchange=0
> > > data     =                       bsize=4096   blocks=24421440, imaxpct=25
> > >          =                       sunit=0      swidth=0 blks
> > > naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=0
> > > log      =internal log           bsize=4096   blocks=41799, version=2
> > >          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> > > realtime =/dev/disk/by-id/ata-MZ7KM1T6HAJM00D3_S2CXNAAH600026 extsz=4096   blocks=390703446, rtextents=390703446
> > > Discarding blocks...Done.
> > > Discarding blocks...Done.
> > > 
> > > It did not fishish after 10 mins.
> > > 
> > > # pstack 5785
> > > #0  0x00007f8df5efc01a in pread64 () from /lib64/libc.so.6
> > > #1  0x0000557b7bbfe9c3 in __read_buf.constprop.0 ()
> > > #2  0x0000557b7bbc3030 in libxfs_buf_read_map ()
> > > #3  0x0000557b7bbfe1ae in libxfs_trans_read_buf_map.constprop ()
> > > #4  0x0000557b7bbee7ef in xfs_rtbuf_get ()
> > > #5  0x0000557b7bbf03d8 in libxfs_rtfree_extent ()
> > > #6  0x0000557b7bbbb322 in parseproto.lto_priv ()
> > > #7  0x0000557b7bbb7643 in main ()
> > > 
> > > And,
> > > 1) more chance happen when kernel 6.12.36, but yet not happen when kernel 6.6.93.
> > > 2) it happen on both xfsprogs-6.11.0 and xfsprogs-6.4.
> > 
> > You /could/ strace the mkfs process to see if it's really stuck, or just
> > issuing IOs really slowly.
> 
> Today the mkfs.xfs with rtdev all finished in 110s-160s on kernel 6.6.93/6.12.38.
> 
> the result of strace show that pread64() with 4K size so it is just slow.
> 
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 868352) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 864256) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 860160) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 856064) = 4096
> pread64(3, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096, 851968) = 4096
> 

the offset of pread64() is in DECE order, not in ASCE order , so the readahead
does not help too.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/07/18



> Thanks a lot.
> 
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2025/07/18
> 



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2025-07-18  1:39 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-17  0:13 a deadloop(?) when mkfs.xfs -o rtdev Wang Yugui
2025-07-17  0:52 ` Wang Yugui
2025-07-17  4:59 ` Christoph Hellwig
2025-07-17 16:03 ` Darrick J. Wong
2025-07-18  0:58   ` Wang Yugui
2025-07-18  1:08     ` Wang Yugui

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).