public inbox for linux-nfs@vger.kernel.org
 help / color / mirror / Atom feed
From: "J. Bruce Fields" <bfields@fieldses.org>
To: Artem.Bityutskiy@nokia.com
Cc: bjschuma@netapp.com, jaxboe@fusionio.com,
	linux-nfs@vger.kernel.org, trond@netapp.com, hch@lst.de
Subject: [PATCH] Fix lost wake-up shutting down writeback thread
Date: Fri, 27 Aug 2010 17:28:42 -0400	[thread overview]
Message-ID: <20100827212842.GC27694@fieldses.org> (raw)
In-Reply-To: <20100827210626.GB27694@fieldses.org>

From: J. Bruce Fields <bfields@redhat.com>

Setting the task state here may cause us to miss the wake up from
kthread_stop(), so we need to recheck kthread_should_stop() or risk
sleeping forever in the following schedule().

Symptom was an indefinite hang on an NFSv4 mount.  (NFSv4 may create
multiple mounts in a temporary namespace while traversing the mount
path, and since the temporary namespace is immediately destroyed, it may
end up destroying a mount very soon after it was created, possibly
making this race more likely.)

INFO: task mount.nfs4:4314 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
mount.nfs4    D 0000000000000000  2880  4314   4313 0x00000000
 ffff88001ed6da28 0000000000000046 ffff88001ed6dfd8 ffff88001ed6dfd8
 ffff88001ed6c000 ffff88001ed6c000 ffff88001ed6c000 ffff88001e5003a0
 ffff88001ed6dfd8 ffff88001e5003a8 ffff88001ed6c000 ffff88001ed6dfd8
Call Trace:
 [<ffffffff8196090d>] schedule_timeout+0x1cd/0x2e0
 [<ffffffff8106a31c>] ? mark_held_locks+0x6c/0xa0
 [<ffffffff819639a0>] ? _raw_spin_unlock_irq+0x30/0x60
 [<ffffffff8106a5fd>] ? trace_hardirqs_on_caller+0x14d/0x190
 [<ffffffff819671fe>] ? sub_preempt_count+0xe/0xd0
 [<ffffffff8195fc80>] wait_for_common+0x120/0x190
 [<ffffffff81033c70>] ? default_wake_function+0x0/0x20
 [<ffffffff8195fdcd>] wait_for_completion+0x1d/0x20
 [<ffffffff810595fa>] kthread_stop+0x4a/0x150
 [<ffffffff81061a60>] ? thaw_process+0x70/0x80
 [<ffffffff810cc68a>] bdi_unregister+0x10a/0x1a0
 [<ffffffff81229dc9>] nfs_put_super+0x19/0x20
 [<ffffffff810ee8c4>] generic_shutdown_super+0x54/0xe0
 [<ffffffff810ee9b6>] kill_anon_super+0x16/0x60
 [<ffffffff8122d3b9>] nfs4_kill_super+0x39/0x90
 [<ffffffff810eda45>] deactivate_locked_super+0x45/0x60
 [<ffffffff810edfb9>] deactivate_super+0x49/0x70
 [<ffffffff81108294>] mntput_no_expire+0x84/0xe0
 [<ffffffff811084ef>] release_mounts+0x9f/0xc0
 [<ffffffff81108575>] put_mnt_ns+0x65/0x80
 [<ffffffff8122cc56>] nfs_follow_remote_path+0x1e6/0x420
 [<ffffffff8122cfbf>] nfs4_try_mount+0x6f/0xd0
 [<ffffffff8122d0c2>] nfs4_get_sb+0xa2/0x360
 [<ffffffff810edcb8>] vfs_kern_mount+0x88/0x1f0
 [<ffffffff810ede92>] do_kern_mount+0x52/0x130
 [<ffffffff81963d9a>] ? _lock_kernel+0x6a/0x170
 [<ffffffff81108e9e>] do_mount+0x26e/0x7f0
 [<ffffffff81106b3a>] ? copy_mount_options+0xea/0x190
 [<ffffffff811094b8>] sys_mount+0x98/0xf0
 [<ffffffff810024d8>] system_call_fastpath+0x16/0x1b
1 lock held by mount.nfs4/4314:
 #0:  (&type->s_umount_key#24){+.+...}, at: [<ffffffff810edfb1>] deactivate_super+0x41/0x70

Signed-off-by: J. Bruce Fields <bfields@redhat.com>
---
 fs/fs-writeback.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

On Fri, Aug 27, 2010 at 05:06:26PM -0400, J. Bruce Fields wrote:
> Maybe the following?

I confirmed that my hang does go away after doing this.

Please apply if you think this makes sense....

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 7d9d06b..81e086d 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -808,7 +808,7 @@ int bdi_writeback_thread(void *data)
 			wb->last_active = jiffies;
 
 		set_current_state(TASK_INTERRUPTIBLE);
-		if (!list_empty(&bdi->work_list)) {
+		if (!list_empty(&bdi->work_list) || kthread_should_stop()) {
 			__set_current_state(TASK_RUNNING);
 			continue;
 		}
-- 
1.7.0.4


  reply	other threads:[~2010-08-27 21:28 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-08-25  2:34 hang in writeback code on nfsv4 mount J. Bruce Fields
2010-08-25  4:09 ` Artem.Bityutskiy
2010-08-25  6:32 ` Artem Bityutskiy
2010-08-25 15:25   ` Bryan Schumaker
2010-08-25 15:44   ` J. Bruce Fields
2010-08-25 18:46     ` Artem Bityutskiy
2010-08-26 11:24     ` Artem Bityutskiy
2010-08-26 13:20       ` Artem Bityutskiy
2010-08-27  6:13 ` Artem Bityutskiy
2010-08-27  7:12   ` Jens Axboe
2010-08-27  9:36   ` Artem Bityutskiy
2010-08-27 13:06     ` Bryan Schumaker
2010-08-27 16:09       ` J. Bruce Fields
2010-08-27 16:17         ` Artem.Bityutskiy
     [not found]           ` <10B234E0D3A1CA469E00963BF106CA392D0DB78354-xJW1crHCIS+8kqYwC468Frtp2NbXvJi8gfoxzgwHRXE@public.gmane.org>
2010-08-27 16:21             ` Artem.Bityutskiy
2010-08-27 21:06           ` J. Bruce Fields
2010-08-27 21:28             ` J. Bruce Fields [this message]
2010-08-28  1:17               ` [PATCH] Fix lost wake-up shutting down writeback thread Artem.Bityutskiy
2010-08-28  6:50               ` Jens Axboe
2010-08-29 12:21                 ` Artem.Bityutskiy
     [not found]                   ` <10B234E0D3A1CA469E00963BF106CA392D0DB78357-xJW1crHCIS+8kqYwC468Frtp2NbXvJi8gfoxzgwHRXE@public.gmane.org>
2010-08-30 11:56                     ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100827212842.GC27694@fieldses.org \
    --to=bfields@fieldses.org \
    --cc=Artem.Bityutskiy@nokia.com \
    --cc=bjschuma@netapp.com \
    --cc=hch@lst.de \
    --cc=jaxboe@fusionio.com \
    --cc=linux-nfs@vger.kernel.org \
    --cc=trond@netapp.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox