From: "Geir A. Myrestrand" <geir.myrestrand@falconstor.com>
To: xfs@oss.sgi.com
Cc: Eric Sandeen <sandeen@sandeen.net>
Subject: Re: New CentOS4/RHEL4-compatible xfs module rpms
Date: Thu, 07 Dec 2006 13:17:32 -0500 [thread overview]
Message-ID: <45785ABC.20208@falconstor.com> (raw)
In-Reply-To: <457854CB.5030507@sandeen.net>
Eric Sandeen wrote:
> Geir A. Myrestrand wrote:
>
>> However, I run into issues with xfs_freeze as it often locks up when I
>> try to freeze a file system where there is I/O activity. Sometimes it
>> happen on the first xfs_freeze invocation to freeze the file system,
>> other times I have to unfreeze and then it happens on the second time I
>> freeze. xfs_freeze never returns when this happens.
>>
>> Looks like xfs_io get stuck --see partial output from `ps auxf`:
>>
>> strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
>> \_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
>> \_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs
>>
>> Anyone else encountering this issue?
>>
>
> Did you have this problem on the previous version?
>
> You might do sysrq-t (echo t > /proc/sysrq-trigger) and see where the
> thread is stuck.
I reproduced it and dumped the task list.
The user-mode XFS processes were:
root 2816 0.0 0.1 2812 628 pts/3 S+ 12:58 0:00 |
\_ strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
root 2847 0.0 0.2 52752 1048 pts/3 T+ 12:58 0:00 |
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
root 2891 0.0 0.1 49976 600 pts/3 D+ 12:58 0:00 |
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs
My I/O generating processes were:
root 3832 1.9 0.2 59644 1376 pts/1 S+ 12:57 0:17 |
\_ /usr/bin/perl ./test.pl
root 2992 0.0 0.0 49912 404 pts/1 D+ 12:58 0:00 |
\_ touch 857
Here are some relevant information from /var/log/messages (I stuck in a
divider when I pasted in sections that weren't just right after the
previous section):
pdflush D ffffffff8014a190 0 48 6 50 47
(L-TLB)
000001001f99dcb8 0000000000000046 0000000080134722 ffffffffa0054a51
000000001f99dc30 0000000000000000 000001001fa2e310 0000000000000000
000001001f9737f0 00000000000000d9
Call Trace:<ffffffffa0054a51>{:ext3:ext3_ordered_writepage+0}
<ffffffffa01d96b5>{:xfs:xlog_state_sync_all+456}
<ffffffffa01f4b9c>{:xfs:pagebuf_rele+54}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff80302637>{__down+147}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80303c13>{__down_failed+53}
<ffffffffa01e59da>{:xfs:xfs_sync+0}
<ffffffffa01f64ad>{:xfs:.text.lock.xfs_buf+15}
<ffffffffa01df1fa>{:xfs:xfs_getsb+37}
<ffffffffa01e5817>{:xfs:xfs_syncsub+2507}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8010f459>{__down_trylock+68}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffffa01f347a>{:xfs:linvfs_write_super+33}
<ffffffff8017ca9e>{sync_supers+167}
<ffffffff8015d8e8>{wb_kupdate+36}
<ffffffff8015e38c>{pdflush+323} <ffffffff8015d8c4>{wb_kupdate+0}
<ffffffff8015e249>{pdflush+0} <ffffffff8014a167>{kthread+200}
<ffffffff80110ca3>{child_rip+8}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a09f>{kthread+0} <ffffffff80110c9b>{child_rip+0}
----------------------------------------------------------------------------
xfs S 0000000000000006 0 2723 1 2742 2697
(NOTLB)
00000100156e9d78 0000000000000002 00000100156e9d98 0000010000000074
000001001eb87880 000001001eb878f0 000000d000000000 0000000100000246
00000100154cc030 0000000000005f88
Call Trace:<ffffffff8013ea74>{__mod_timer+293}
<ffffffff803039b3>{schedule_timeout+244}
<ffffffff8013f49e>{process_timeout+0}
<ffffffff8018902b>{do_select+939}
<ffffffff80188bc5>{__pollwait+0} <ffffffff801893aa>{sys_select+820}
<ffffffff801908fc>{dnotify_parent+34}
<ffffffff80110052>{system_call+126}
----------------------------------------------------------------------------
xfslogd/0 S ffffffffa01f54b0 0 3818 6 3819 1981
(L-TLB)
00000100104e5e68 0000000000000046 0000000000000283 0000010011a268e0
ffffffffa01f54b0 0000000000000246 ffffffff80303cc1 00000000a01bb7a8
0000010011d377f0 000000000000023e
Call Trace:<ffffffffa01f54b0>{:xfs:pagebuf_iodone_work+0}
<ffffffff80303cc1>{__up_wakeup+53}
<ffffffffa01f54b0>{:xfs:pagebuf_iodone_work+0}
<ffffffff80146431>{worker_thread+226}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014634f>{worker_thread+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a167>{kthread+200}
<ffffffff80110ca3>{child_rip+8}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a09f>{kthread+0} <ffffffff80110c9b>{child_rip+0}
xfslogd/1 S ffffffffa01f54b0 0 3819 6 3820 3818
(L-TLB)
00000100105d7e68 0000000000000046 0000000000000287 ffffffffa01e2960
000001000e48eb20 000001000e48eb20 0000010012d7e260 0000000100000246
000001001053f7f0 000000000000022f
Call Trace:<ffffffffa01e2960>{:xfs:xfs_trans_delete_ail+45}
<ffffffffa01f54b0>{:xfs:pagebuf_iodone_work+0}
<ffffffff80146431>{worker_thread+226}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014634f>{worker_thread+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a167>{kthread+200} <ffffffff80110ca3>{child_rip+8}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a09f>{kthread+0}
<ffffffff80110c9b>{child_rip+0}
xfsdatad/0 S 00000100164aa9c0 0 3820 6 3821 3819
(L-TLB)
00000100104e1e68 0000000000000046 000000000032a7bd 000000190000006a
000001001053e7f0 000000000000006a 0000010001707840 0000000000187397
000001001053f030 0000000000000d8a
Call Trace:<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014634f>{worker_thread+0}
<ffffffff80146431>{worker_thread+226}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014634f>{worker_thread+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a167>{kthread+200} <ffffffff80110ca3>{child_rip+8}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a09f>{kthread+0}
<ffffffff80110c9b>{child_rip+0}
xfsdatad/1 S 00000100164aaa40 0 3821 6 3820
(L-TLB)
00000100104e3e68 0000000000000046 000000260000000a 0000001900000074
000001001f9bd030 0000000000000074 000001000170f840 000000010032ba4b
000001001053e7f0 0000000000000b24
Call Trace:<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014634f>{worker_thread+0}
<ffffffff80146431>{worker_thread+226}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014634f>{worker_thread+0}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a167>{kthread+200} <ffffffff80110ca3>{child_rip+8}
<ffffffff8014a190>{keventd_create_kthread+0}
<ffffffff8014a09f>{kthread+0}
<ffffffff80110c9b>{child_rip+0}
xfsbufd S 000000010027d0c0 0 3822 1 3827 3815
(L-TLB)
00000100111fdea8 0000000000000046 ffffffff803d9920 0000001900000073
000001001f9bd030 0000000000000073 0000010001707840 0000000080138348
000001001053e030 000000000000019b
Call Trace:<ffffffff8013ea74>{__mod_timer+293}
<ffffffff803039b3>{schedule_timeout+244}
<ffffffff8013f49e>{process_timeout+0}
<ffffffffa01f62db>{:xfs:xfsbufd+172}
<ffffffff80110ca3>{child_rip+8}
<ffffffff8011de14>{flat_send_IPI_mask+0}
<ffffffffa01f622f>{:xfs:xfsbufd+0} <ffffffff80110c9b>{child_rip+0}
xfssyncd S ffffffffa01f36b9 0 3827 1 3822
(L-TLB)
0000010011f0fea8 0000000000000046 ffffffffa01f36b9 0000000000000202
ffffffff8010f459 00000100126e2d40 000001001d5e3980 0000000100000000
000001001e4dd7f0 00000000000025eb
Call Trace:<ffffffffa01f36b9>{:xfs:linvfs_fill_super+0}
<ffffffff8010f459>{__down_trylock+68}
<ffffffff8013ea74>{__mod_timer+293}
<ffffffffa01f36b9>{:xfs:linvfs_fill_super+0}
<ffffffff803039b3>{schedule_timeout+244}
<ffffffff8013f49e>{process_timeout+0}
<ffffffffa01f323c>{:xfs:xfssyncd+120}
<ffffffff80110ca3>{child_rip+8}
<ffffffffa01f36b9>{:xfs:linvfs_fill_super+0}
<ffffffff801cc053>{dummy_d_instantiate+0}
<ffffffffa01f31c4>{:xfs:xfssyncd+0} <ffffffff80110c9b>{child_rip+0}
----------------------------------------------------------------------------
xfs_freeze t 00000000006c51e0 0 2847 2816 2891
(NOTLB)
00000100074d7e78 0000000000000002 0000010015654030 00000100111fec40
0000000000000206 ffffffff801410a6 0000000000000011 0000000000040004
0000010015654030 0000000000001a97
Call Trace:<ffffffff801410a6>{do_notify_parent_cldstop+243}
<ffffffff801418d8>{ptrace_stop+315}
<ffffffff801419c4>{ptrace_notify+139}
<ffffffff8011419b>{syscall_trace+37}
<ffffffff8011420f>{syscall_trace_enter+43}
<ffffffff80110156>{tracesys+113}
xfs_io D 00000100146af380 0 2891 2816 2847
(NOTLB)
00000100075c9af8 0000000000000002 000001000a415b80 0000000000000064
000000000007d000 ffffffff802499ca ffffffff80134722 00000001075c9a90
000001000e83f7f0 0000000001a7ca67
Call Trace:<ffffffff802499ca>{generic_make_request+355}
<ffffffff80134722>{autoremove_wake_function+0}
<ffffffff80302637>{__down+147}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80303c13>{__down_failed+53}
<ffffffffa01f2a45>{:xfs:xfs_bdstrat_cb+0}
<ffffffffa01f64ad>{:xfs:.text.lock.xfs_buf+15}
<ffffffffa01f60f6>{:xfs:xfs_flush_buftarg+213}
<ffffffffa01e59af>{:xfs:xfs_quiesce_fs+69}
<ffffffffa01f34c1>{:xfs:linvfs_sync_super+65}
<ffffffff80178367>{freeze_bdev+215}
<ffffffffa01f0cd1>{:xfs:xfs_ioctl+5161}
<ffffffff801588f0>{find_get_page+65}
<ffffffff801593d2>{filemap_nopage+378}
<ffffffff80131c6d>{finish_task_switch+55}
<ffffffff80303358>{thread_return+42}
<ffffffff8014191f>{ptrace_stop+386}
<ffffffff801419e0>{ptrace_notify+167}
<ffffffffa01ef380>{:xfs:linvfs_ioctl+112}
<ffffffff80188631>{sys_ioctl+853}
<ffffffff801101b6>{tracesys+209}
touch D 0000000000239000 0 2992 3832
(NOTLB)
0000010008dd76e8 0000000000000006 0000000000000001 0000000000000001
0000000000000016 ffffffff80131931 0000000100000000 0000000000000003
000001001ed54030 00000000000078de
Call Trace:<ffffffff80131931>{try_to_wake_up+734}
<ffffffffa01f4dc2>{:xfs:_pagebuf_find+327}
<ffffffff80302637>{__down+147}
<ffffffff80132e8d>{default_wake_function+0}
<ffffffff80303c13>{__down_failed+53}
<ffffffffa01f64ad>{:xfs:.text.lock.xfs_buf+15}
<ffffffffa01f4de1>{:xfs:_pagebuf_find+358}
<ffffffffa01f5065>{:xfs:xfs_buf_get_flags+100}
<ffffffffa01b1be0>{:xfs:xfs_bmap_search_extents+92}
<ffffffffa01f5b01>{:xfs:xfs_buf_read_flags+16}
<ffffffffa01e30b3>{:xfs:xfs_trans_read_buf+428}
<ffffffffa01bc37c>{:xfs:xfs_da_do_buf+984}
<ffffffffa01bc605>{:xfs:xfs_da_read_buf+21}
<ffffffffa01bc87b>{:xfs:xfs_da_node_lookup_int+145}
<ffffffffa01bc87b>{:xfs:xfs_da_node_lookup_int+145}
<ffffffffa01c58a2>{:xfs:xfs_dir2_node_addname+83}
<ffffffffa01d26bc>{:xfs:xfs_ichgtime+93}
<ffffffff8018f975>{wake_up_inode+6}
<ffffffffa01b286b>{:xfs:xfs_bmap_last_offset+179}
<ffffffffa01c056e>{:xfs:xfs_dir2_createname+283}
<ffffffffa01e8818>{:xfs:xfs_create+992}
<ffffffff801cbecc>{dummy_inode_permission+0}
<ffffffffa01f0fb7>{:xfs:linvfs_mknod+453}
<ffffffffa01bbf7c>{:xfs:xfs_da_brelse+116}
<ffffffffa01c6167>{:xfs:xfs_dir2_node_lookup+170}
<ffffffffa01c042d>{:xfs:xfs_dir2_lookup+248}
<ffffffff80184d4b>{vfs_create+214}
<ffffffff80185179>{open_namei+430} <ffffffff80176558>{filp_open+39}
<ffffffff801ea045>{strncpy_from_user+74}
<ffffffff80176661>{get_unused_fd+230}
<ffffffff80176750>{sys_open+57} <ffffffff80110052>{system_call+126}
--
Geir A. Myrestrand
next prev parent reply other threads:[~2006-12-07 18:19 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-11-19 19:07 New CentOS4/RHEL4-compatible xfs module rpms Eric Sandeen
2006-12-07 17:25 ` Geir A. Myrestrand
2006-12-07 17:52 ` Eric Sandeen
2006-12-07 18:17 ` Geir A. Myrestrand [this message]
2006-12-07 23:26 ` David Chinner
2006-12-08 2:58 ` Geir A. Myrestrand
2006-12-07 18:19 ` Geir A. Myrestrand
2006-12-07 20:51 ` Geir A. Myrestrand
2006-12-07 21:11 ` Nathan Scott
2006-12-07 21:35 ` Geir A. Myrestrand
2006-12-07 21:40 ` Nathan Scott
2006-12-07 21:51 ` Geir A. Myrestrand
2006-12-07 22:06 ` Nathan Scott
2006-12-07 22:16 ` Geir A. Myrestrand
2006-12-07 22:18 ` Eric Sandeen
2006-12-07 22:25 ` Geir A. Myrestrand
2006-12-07 22:30 ` Eric Sandeen
2006-12-07 22:52 ` Geir A. Myrestrand
2006-12-07 22:53 ` Eric Sandeen
2006-12-07 23:08 ` Geir A. Myrestrand
2007-07-19 14:37 ` James Braid
2007-07-19 15:15 ` Eric Sandeen
2007-07-21 3:34 ` James Braid
2007-07-21 14:58 ` Eric Sandeen
2007-07-21 15:36 ` James Braid
-- strict thread matches above, loose matches on Subject: below --
2007-08-08 16:04 James Pearson
2007-08-08 16:09 ` Eric Sandeen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=45785ABC.20208@falconstor.com \
--to=geir.myrestrand@falconstor.com \
--cc=sandeen@sandeen.net \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox