stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org,
	Gerlando Falauto <gerlando.falauto@keymile.com>,
	Lin Ming <minggr@gmail.com>, John Stultz <john.stultz@linaro.org>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	Ingo Molnar <mingo@kernel.org>
Subject: [ 06/49] timekeeping: Fix HRTICK related deadlock from ntp lock changes
Date: Sun, 29 Sep 2013 12:26:40 -0700	[thread overview]
Message-ID: <20130929191603.444266684@linuxfoundation.org> (raw)
In-Reply-To: <20130929191603.008825488@linuxfoundation.org>

3.10-stable review patch.  If anyone has any objections, please let me know.

------------------

From: John Stultz <john.stultz@linaro.org>

commit 7bd36014460f793c19e7d6c94dab67b0afcfcb7f upstream.

Gerlando Falauto reported that when HRTICK is enabled, it is
possible to trigger system deadlocks. These were hard to
reproduce, as HRTICK has been broken in the past, but seemed
to be connected to the timekeeping_seq lock.

Since seqlock/seqcount's aren't supported w/ lockdep, I added
some extra spinlock based locking and triggered the following
lockdep output:

[   15.849182] ntpd/4062 is trying to acquire lock:
[   15.849765]  (&(&pool->lock)->rlock){..-...}, at: [<ffffffff810aa9b5>] __queue_work+0x145/0x480
[   15.850051]
[   15.850051] but task is already holding lock:
[   15.850051]  (timekeeper_lock){-.-.-.}, at: [<ffffffff810df6df>] do_adjtimex+0x7f/0x100

<snip>

[   15.850051] Chain exists of: &(&pool->lock)->rlock --> &p->pi_lock --> timekeeper_lock
[   15.850051]  Possible unsafe locking scenario:
[   15.850051]
[   15.850051]        CPU0                    CPU1
[   15.850051]        ----                    ----
[   15.850051]   lock(timekeeper_lock);
[   15.850051]                                lock(&p->pi_lock);
[   15.850051] lock(timekeeper_lock);
[   15.850051] lock(&(&pool->lock)->rlock);
[   15.850051]
[   15.850051]  *** DEADLOCK ***

The deadlock was introduced by 06c017fdd4dc48451a ("timekeeping:
Hold timekeepering locks in do_adjtimex and hardpps") in 3.10

This patch avoids this deadlock, by moving the call to
schedule_delayed_work() outside of the timekeeper lock
critical section.

Reported-by: Gerlando Falauto <gerlando.falauto@keymile.com>
Tested-by: Lin Ming <minggr@gmail.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: http://lkml.kernel.org/r/1378943457-27314-1-git-send-email-john.stultz@linaro.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 include/linux/timex.h     |    1 +
 kernel/time/ntp.c         |    6 ++----
 kernel/time/timekeeping.c |    2 ++
 3 files changed, 5 insertions(+), 4 deletions(-)

--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -141,6 +141,7 @@ extern int do_adjtimex(struct timex *);
 extern void hardpps(const struct timespec *, const struct timespec *);
 
 int read_current_timer(unsigned long *timer_val);
+void ntp_notify_cmos_timer(void);
 
 /* The clock frequency of the i8253/i8254 PIT */
 #define PIT_TICK_RATE 1193182ul
--- a/kernel/time/ntp.c
+++ b/kernel/time/ntp.c
@@ -516,13 +516,13 @@ static void sync_cmos_clock(struct work_
 	schedule_delayed_work(&sync_cmos_work, timespec_to_jiffies(&next));
 }
 
-static void notify_cmos_timer(void)
+void ntp_notify_cmos_timer(void)
 {
 	schedule_delayed_work(&sync_cmos_work, 0);
 }
 
 #else
-static inline void notify_cmos_timer(void) { }
+void ntp_notify_cmos_timer(void) { }
 #endif
 
 
@@ -687,8 +687,6 @@ int __do_adjtimex(struct timex *txc, str
 	if (!(time_status & STA_NANO))
 		txc->time.tv_usec /= NSEC_PER_USEC;
 
-	notify_cmos_timer();
-
 	return result;
 }
 
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1682,6 +1682,8 @@ int do_adjtimex(struct timex *txc)
 	write_seqcount_end(&timekeeper_seq);
 	raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
 
+	ntp_notify_cmos_timer();
+
 	return ret;
 }
 



  parent reply	other threads:[~2013-09-29 19:26 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-09-29 19:26 [ 00/49] 3.10.14-stable review Greg Kroah-Hartman
2013-09-29 19:26 ` [ 01/49] PCI / ACPI / PM: Clear pme_poll for devices in D3cold on wakeup Greg Kroah-Hartman
2013-09-29 19:26 ` [ 02/49] net: usb: cdc_ether: Use wwan interface for Telit modules Greg Kroah-Hartman
2013-09-29 19:26 ` [ 03/49] cifs: fix filp leak in cifs_atomic_open() Greg Kroah-Hartman
2013-09-29 19:26 ` [ 04/49] bgmac: fix internal switch initialization Greg Kroah-Hartman
2013-09-29 19:26 ` [ 05/49] rt2800: fix wrong TX power compensation Greg Kroah-Hartman
2013-09-29 19:26 ` Greg Kroah-Hartman [this message]
2013-09-29 19:26 ` [ 07/49] sched/cputime: Do not scale when utime == 0 Greg Kroah-Hartman
2013-09-29 19:26 ` [ 08/49] sched/fair: Fix small race where child->se.parent,cfs_rq might point to invalid ones Greg Kroah-Hartman
2013-09-29 19:26 ` [ 09/49] HID: provide a helper for validating hid reports Greg Kroah-Hartman
2013-09-29 19:26 ` [ 10/49] HID: validate feature and input report details Greg Kroah-Hartman
2013-09-29 19:26 ` [ 11/49] HID: multitouch: validate indexes details Greg Kroah-Hartman
2013-09-29 19:26 ` [ 12/49] HID: LG: validate HID output report details Greg Kroah-Hartman
2013-09-29 19:26 ` [ 13/49] HID: zeroplus: validate " Greg Kroah-Hartman
2013-09-29 19:26 ` [ 14/49] HID: lenovo-tpkbd: fix leak if tpkbd_probe_tp fails Greg Kroah-Hartman
2013-09-29 19:26 ` [ 15/49] HID: steelseries: validate output report details Greg Kroah-Hartman
2013-09-29 19:26 ` [ 16/49] HID: lenovo-tpkbd: " Greg Kroah-Hartman
2013-09-29 19:26 ` [ 17/49] HID: logitech-dj: " Greg Kroah-Hartman
2013-09-29 19:26 ` [ 18/49] usb: gadget: fix a bug and a WARN_ON in dummy-hcd Greg Kroah-Hartman
2013-09-29 19:26 ` [ 19/49] drm/i915: fix gpu hang vs. flip stall deadlocks Greg Kroah-Hartman
2013-09-29 19:26 ` [ 20/49] drm/i915: fix wait_for_pending_flips vs gpu hang deadlock Greg Kroah-Hartman
2013-09-29 19:26 ` [ 21/49] drm: fix DRM_IOCTL_MODE_GETFB handle-leak Greg Kroah-Hartman
2013-09-29 19:26 ` [ 22/49] drm/ast: fix the ast open key function Greg Kroah-Hartman
2013-09-29 19:26 ` [ 23/49] drm/ttm: fix the tt_populated check in ttm_tt_destroy() Greg Kroah-Hartman
2013-09-29 19:26 ` [ 24/49] drm/nv50/disp: prevent false output detection on the original nv50 Greg Kroah-Hartman
2013-09-29 19:26 ` [ 25/49] drm/radeon: fix LCD record parsing Greg Kroah-Hartman
2013-09-29 19:27 ` [ 26/49] drm/radeon: fix endian bugs in hw i2c atom routines Greg Kroah-Hartman
2013-09-29 19:27 ` [ 27/49] drm/radeon/si: Add support for CP DMA to CS checker for compute v2 Greg Kroah-Hartman
2013-09-29 19:27 ` [ 28/49] drm/radeon: update line buffer allocation for dce4.1/5 Greg Kroah-Hartman
2013-09-29 19:27 ` [ 29/49] drm/radeon: update line buffer allocation for dce6 Greg Kroah-Hartman
2013-09-29 19:27 ` [ 30/49] drm/radeon: fix resume on some rs4xx boards (v2) Greg Kroah-Hartman
2013-09-29 19:27 ` [ 31/49] drm/radeon: fix handling of variable sized arrays for router objects Greg Kroah-Hartman
2013-09-29 19:27 ` [ 32/49] rt2800: change initialization sequence to fix system freeze Greg Kroah-Hartman
2013-09-29 19:27 ` [ 33/49] drm/radeon/atom: workaround vbios bug in transmitter table on rs880 (v2) Greg Kroah-Hartman
2013-09-29 19:27 ` [ 34/49] drm/radeon: fix panel scaling with eDP and LVDS bridges Greg Kroah-Hartman
2013-09-29 19:27 ` [ 35/49] drm/radeon: avoid UVD corruptions on AGP cards Greg Kroah-Hartman
2013-09-29 19:27 ` [ 36/49] media: media/usb: fix kconfig dependencies Greg Kroah-Hartman
2013-09-29 19:27 ` [ 37/49] Properly handle tristate dependencies on USB/PCI menus Greg Kroah-Hartman
2013-09-29 19:27 ` [ 38/49] udf: Standardize return values in mount sequence Greg Kroah-Hartman
2013-09-29 19:27 ` [ 39/49] udf: Refuse RW mount of the filesystem instead of making it RO Greg Kroah-Hartman
2013-09-29 19:27 ` [ 40/49] audit: fix endless wait in audit_log_start() Greg Kroah-Hartman
2013-09-29 19:27 ` [ 41/49] mm: fix aio performance regression for database caused by THP Greg Kroah-Hartman
2013-09-29 19:27 ` [ 42/49] perf tools: Handle JITed code in shared memory Greg Kroah-Hartman
2013-09-29 19:27 ` [ 43/49] bio-integrity: Fix use of bs->bio_integrity_pool after free Greg Kroah-Hartman
2013-09-29 19:27 ` [ 44/49] cfq: explicitly use 64bit divide operation for 64bit arguments Greg Kroah-Hartman
2013-09-29 19:27 ` [ 45/49] rpc: clean up decoding of gssproxy linux creds Greg Kroah-Hartman
2013-09-29 19:27 ` [ 46/49] rpc: comment on linux_cred encoding, treat all as unsigned Greg Kroah-Hartman
2013-09-29 19:27 ` [ 47/49] rpc: fix huge kmallocs in gss-proxy Greg Kroah-Hartman
2013-09-29 19:27 ` [ 48/49] rpc: let xdr layer allocate gssproxy receieve pages Greg Kroah-Hartman
2013-09-29 19:27 ` [ 49/49] netfilter: ipset: Fix serious failure in CIDR tracking Greg Kroah-Hartman
2013-09-30  1:27 ` [ 00/49] 3.10.14-stable review Guenter Roeck
2013-09-30  1:50   ` Greg Kroah-Hartman
2013-10-01 19:22 ` Shuah Khan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130929191603.444266684@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=gerlando.falauto@keymile.com \
    --cc=john.stultz@linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=minggr@gmail.com \
    --cc=mingo@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).