public inbox for cip-dev@lists.cip-project.org
 help / color / mirror / Atom feed
From: mosipov@ilbers.de (Maxim Yu. Osipov)
To: cip-dev@lists.cip-project.org
Subject: [cip-dev] [PATCH 05/10] watchdog: core: Fix circular locking dependency
Date: Wed,  4 Oct 2017 16:40:59 +0200	[thread overview]
Message-ID: <20171004144104.14403-6-mosipov@ilbers.de> (raw)
In-Reply-To: <20171004144104.14403-1-mosipov@ilbers.de>

From: Guenter Roeck <linux@roeck-us.net>

Backport from kernel.org, upstream commit e1f30282a1d3

lockdep reports the following circular locking dependency.

======================================================
INFO: possible circular locking dependency detected ]
4.6.0-rc3-00191-gfabf418 #162 Not tainted
-------------------------------------------------------
systemd/1 is trying to acquire lock:
((&(&wd_data->work)->work)){+.+...}, at: [<80141650>] flush_work+0x0/0x280

but task is already holding lock:

(&wd_data->lock){+.+...}, at: [<804acfa8>] watchdog_release+0x18/0x190

which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:

-> #1 (&wd_data->lock){+.+...}:
>-------[<80662310>] mutex_lock_nested+0x64/0x4a8
>-------[<804aca4c>] watchdog_ping_work+0x18/0x4c
>-------[<80143128>] process_one_work+0x1ac/0x500
>-------[<801434b4>] worker_thread+0x38/0x554
>-------[<80149510>] kthread+0xf4/0x108
>-------[<80107c10>] ret_from_fork+0x14/0x24

-> #0 ((&(&wd_data->work)->work)){+.+...}:
>-------[<8017c4e8>] lock_acquire+0x70/0x90
>-------[<8014169c>] flush_work+0x4c/0x280
>-------[<801440f8>] __cancel_work_timer+0x9c/0x1e0
>-------[<804acfcc>] watchdog_release+0x3c/0x190
>-------[<8022c5e8>] __fput+0x80/0x1c8
>-------[<80147b28>] task_work_run+0x94/0xc8
>-------[<8010b998>] do_work_pending+0x8c/0xb4
>-------[<80107ba8>] slow_work_pending+0xc/0x20

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0                    CPU1
----                    ----
lock(&wd_data->lock);
                        lock((&(&wd_data->work)->work));
                        lock(&wd_data->lock);
lock((&(&wd_data->work)->work));

*** DEADLOCK ***

1 lock held by systemd/1:

stack backtrace:
CPU: 2 PID: 1 Comm: systemd Not tainted 4.6.0-rc3-00191-gfabf418 #162
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[<8010f5e4>] (unwind_backtrace) from [<8010c038>] (show_stack+0x10/0x14)
[<8010c038>] (show_stack) from [<8039d7fc>] (dump_stack+0xa8/0xd4)
[<8039d7fc>] (dump_stack) from [<80177ee0>] (print_circular_bug+0x214/0x334)
[<80177ee0>] (print_circular_bug) from [<80179230>] (check_prevs_add+0x4dc/0x8e8)
[<80179230>] (check_prevs_add) from [<8017b3d8>] (__lock_acquire+0xc6c/0x14ec)
[<8017b3d8>] (__lock_acquire) from [<8017c4e8>] (lock_acquire+0x70/0x90)
[<8017c4e8>] (lock_acquire) from [<8014169c>] (flush_work+0x4c/0x280)
[<8014169c>] (flush_work) from [<801440f8>] (__cancel_work_timer+0x9c/0x1e0)
[<801440f8>] (__cancel_work_timer) from [<804acfcc>] (watchdog_release+0x3c/0x190)
[<804acfcc>] (watchdog_release) from [<8022c5e8>] (__fput+0x80/0x1c8)
[<8022c5e8>] (__fput) from [<80147b28>] (task_work_run+0x94/0xc8)
[<80147b28>] (task_work_run) from [<8010b998>] (do_work_pending+0x8c/0xb4)
[<8010b998>] (do_work_pending) from [<80107ba8>] (slow_work_pending+0xc/0x20)

Turns out the call to cancel_delayed_work_sync() in watchdog_release()
is not necessary and can be dropped. If the worker is no longer necessary,
the subsequent call to watchdog_update_worker() will cancel it. If it is
already running, it won't do anything, since the worker function checks
if it needs to ping the watchdog or not.

Reported-by: Clemens Gruber <clemens.gruber@pqgruber.com>
Tested-by: Clemens Gruber <clemens.gruber@pqgruber.com>
Fixes: 11d7aba9ceb7 ("watchdog: imx2: Convert to use infrastructure triggered keepalives")
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Wim Van Sebroeck <wim@iguana.be>
Cc: stable <stable@vger.kernel.org>
[mosipov at ilbers.de backported to 4.4.y]
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 drivers/watchdog/watchdog_dev.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/watchdog/watchdog_dev.c b/drivers/watchdog/watchdog_dev.c
index ca64724bf59a..2eca04d869d9 100644
--- a/drivers/watchdog/watchdog_dev.c
+++ b/drivers/watchdog/watchdog_dev.c
@@ -596,7 +596,6 @@ static int watchdog_release(struct inode *inode, struct file *file)
 	if (!watchdog_hw_running(wdd))
 		module_put(wdd->ops->owner);
 
-	cancel_delayed_work_sync(&wdd->work);
 	watchdog_update_worker(wdd);
 
 	/* make sure that /dev/watchdog can be re-opened */
-- 
2.11.0

  parent reply	other threads:[~2017-10-04 14:40 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-10-04 14:40 [cip-dev] [PATCH 00/10] Backport of watchdog core triggered keepalive infrastructure Maxim Yu. Osipov
2017-10-04 14:40 ` [cip-dev] [PATCH 01/10] watchdog: Introduce hardware maximum heartbeat in watchdog core Maxim Yu. Osipov
2017-10-04 14:40 ` [cip-dev] [PATCH 02/10] watchdog: Introduce WDOG_HW_RUNNING flag Maxim Yu. Osipov
2017-10-25  9:43   ` Ben Hutchings
2017-10-04 14:40 ` [cip-dev] [PATCH 03/10] watchdog: Make stop function optional Maxim Yu. Osipov
2017-10-04 14:40 ` [cip-dev] [PATCH 04/10] watchdog: imx2: Convert to use infrastructure triggered keepalives Maxim Yu. Osipov
2017-10-04 14:40 ` Maxim Yu. Osipov [this message]
2017-10-04 14:41 ` [cip-dev] [PATCH 06/10] watchdog: skip min and max timeout validity check when max_hw_heartbeat_ms is defined Maxim Yu. Osipov
2017-10-04 14:41 ` [cip-dev] [PATCH 07/10] watchdog: change watchdog_need_worker logic Maxim Yu. Osipov
2017-10-04 14:41 ` [cip-dev] [PATCH 08/10] watchdog: core: Fix error handling of watchdog_dev_init() Maxim Yu. Osipov
2017-10-04 14:41 ` [cip-dev] [PATCH 09/10] watchdog: core: Clear WDOG_HW_RUNNING before calling the stop function Maxim Yu. Osipov
2017-10-04 14:41 ` [cip-dev] [PATCH 10/10] watchdog: core: add option to avoid early handling of watchdog Maxim Yu. Osipov
2017-10-09 15:00 ` [cip-dev] [PATCH 00/10] Backport of watchdog core triggered keepalive infrastructure Ben Hutchings
2017-10-09 15:26   ` Jan Kiszka
2017-10-25  9:46 ` Ben Hutchings
  -- strict thread matches above, loose matches on Subject: below --
2017-11-10 12:09 [cip-dev] [PATCH 00/10] Backport of watchdog core triggered keepalive infrastructure v2 Maxim Yu. Osipov
2017-11-10 12:09 ` [cip-dev] [PATCH 05/10] watchdog: core: Fix circular locking dependency Maxim Yu. Osipov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171004144104.14403-6-mosipov@ilbers.de \
    --to=mosipov@ilbers.de \
    --cc=cip-dev@lists.cip-project.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox