stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Junxiao Bi <junxiao.bi@oracle.com>,
	Wengang Wang <wen.gang.wang@oracle.com>,
	Sunil Mushran <sunil.mushran@gmail.com>,
	Srinivas Eeda <srinivas.eeda@oracle.com>,
	Joel Becker <jlbec@evilplan.org>, Mark Fasheh <mfasheh@suse.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: [PATCH 3.4 39/43] ocfs2: dlm: fix lock migration crash
Date: Sun,  4 May 2014 11:42:33 -0400	[thread overview]
Message-ID: <20140504154229.569609201@linuxfoundation.org> (raw)
In-Reply-To: <20140504154224.211508175@linuxfoundation.org>

3.4-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Junxiao Bi <junxiao.bi@oracle.com>

commit 34aa8dac482f1358d59110d5e3a12f4351f6acaa upstream.

This issue was introduced by commit 800deef3f6f8 ("ocfs2: use
list_for_each_entry where benefical") in 2007 where it replaced
list_for_each with list_for_each_entry.  The variable "lock" will point
to invalid data if "tmpq" list is empty and a panic will be triggered
due to this.  Sunil advised reverting it back, but the old version was
also not right.  At the end of the outer for loop, that
list_for_each_entry will also set "lock" to an invalid data, then in the
next loop, if the "tmpq" list is empty, "lock" will be an stale invalid
data and cause the panic.  So reverting the list_for_each back and reset
"lock" to NULL to fix this issue.

Another concern is that this seemes can not happen because the "tmpq"
list should not be empty.  Let me describe how.

old lock resource owner(node 1):                                  migratation target(node 2):
image there's lockres with a EX lock from node 2 in
granted list, a NR lock from node x with convert_type
EX in converting list.
dlm_empty_lockres() {
 dlm_pick_migration_target() {
   pick node 2 as target as its lock is the first one
   in granted list.
 }
 dlm_migrate_lockres() {
   dlm_mark_lockres_migrating() {
     res->state |= DLM_LOCK_RES_BLOCK_DIRTY;
     wait_event(dlm->ast_wq, !dlm_lockres_is_dirty(dlm, res));
	 //after the above code, we can not dirty lockres any more,
     // so dlm_thread shuffle list will not run
                                                                   downconvert lock from EX to NR
                                                                   upconvert lock from NR to EX
<<< migration may schedule out here, then
<<< node 2 send down convert request to convert type from EX to
<<< NR, then send up convert request to convert type from NR to
<<< EX, at this time, lockres granted list is empty, and two locks
<<< in the converting list, node x up convert lock followed by
<<< node 2 up convert lock.

	 // will set lockres RES_MIGRATING flag, the following
	 // lock/unlock can not run
     dlm_lockres_release_ast(dlm, res);
   }

   dlm_send_one_lockres()
                                                                 dlm_process_recovery_data()
                                                                   for (i=0; i<mres->num_locks; i++)
                                                                     if (ml->node == dlm->node_num)
                                                                       for (j = DLM_GRANTED_LIST; j <= DLM_BLOCKED_LIST; j++) {
                                                                        list_for_each_entry(lock, tmpq, list)
                                                                        if (lock) break; <<< lock is invalid as grant list is empty.
                                                                       }
                                                                       if (lock->ml.node != ml->node)
                                                                         BUG() >>> crash here
 }

I see the above locks status from a vmcore of our internal bug.

Signed-off-by: Junxiao Bi <junxiao.bi@oracle.com>
Reviewed-by: Wengang Wang <wen.gang.wang@oracle.com>
Cc: Sunil Mushran <sunil.mushran@gmail.com>
Reviewed-by: Srinivas Eeda <srinivas.eeda@oracle.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Mark Fasheh <mfasheh@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 fs/ocfs2/dlm/dlmrecovery.c |   14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

--- a/fs/ocfs2/dlm/dlmrecovery.c
+++ b/fs/ocfs2/dlm/dlmrecovery.c
@@ -1752,13 +1752,13 @@ static int dlm_process_recovery_data(str
 				     struct dlm_migratable_lockres *mres)
 {
 	struct dlm_migratable_lock *ml;
-	struct list_head *queue;
+	struct list_head *queue, *iter;
 	struct list_head *tmpq = NULL;
 	struct dlm_lock *newlock = NULL;
 	struct dlm_lockstatus *lksb = NULL;
 	int ret = 0;
 	int i, j, bad;
-	struct dlm_lock *lock = NULL;
+	struct dlm_lock *lock;
 	u8 from = O2NM_MAX_NODES;
 	unsigned int added = 0;
 	__be64 c;
@@ -1793,14 +1793,16 @@ static int dlm_process_recovery_data(str
 			/* MIGRATION ONLY! */
 			BUG_ON(!(mres->flags & DLM_MRES_MIGRATION));
 
+			lock = NULL;
 			spin_lock(&res->spinlock);
 			for (j = DLM_GRANTED_LIST; j <= DLM_BLOCKED_LIST; j++) {
 				tmpq = dlm_list_idx_to_ptr(res, j);
-				list_for_each_entry(lock, tmpq, list) {
-					if (lock->ml.cookie != ml->cookie)
-						lock = NULL;
-					else
+				list_for_each(iter, tmpq) {
+					lock = list_entry(iter,
+						  struct dlm_lock, list);
+					if (lock->ml.cookie == ml->cookie)
 						break;
+					lock = NULL;
 				}
 				if (lock)
 					break;



  parent reply	other threads:[~2014-05-04 15:42 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-05-04 15:41 [PATCH 3.4 00/43] 3.4.89-stable review Greg Kroah-Hartman
2014-05-04 15:41 ` [PATCH 3.4 01/43] ASoC: cs42l73: Fix mask bits for SOC_VALUE_ENUM_SINGLE Greg Kroah-Hartman
2014-05-04 15:41 ` [PATCH 3.4 02/43] ARM: OMAP2+: INTC: Acknowledge stuck active interrupts Greg Kroah-Hartman
2014-05-04 15:41 ` [PATCH 3.4 03/43] ARM: OMAP3: hwmod data: Correct clock domains for USB modules Greg Kroah-Hartman
2014-05-04 15:41 ` [PATCH 3.4 04/43] ARM: 8027/1: fix do_div() bug in big-endian systems Greg Kroah-Hartman
2014-05-04 15:41 ` [PATCH 3.4 05/43] ARM: 8030/1: ARM : kdump : add arch_crash_save_vmcoreinfo Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 06/43] ALSA: hda - Enable beep for ASUS 1015E Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 07/43] ALSA: ice1712: Fix boundary checks in PCM pointer ops Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 08/43] mfd: max8925: Fix possible NULL pointer dereference on i2c_new_dummy error Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 09/43] mfd: max8998: " Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 10/43] mfd: max8997: " Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 11/43] w1: fix w1_send_slave dropping a slave id Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 12/43] staging:serqt_usb2: Fix sparse warning restricted __le16 degrades to integer Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 13/43] staging: r8712u: Fix case where ethtype was never obtained and always be checked against 0 Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 14/43] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 15/43] USB: fix crash during hotplug of PCI USB controller card Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 16/43] nfsd4: session needs room for following op to error out Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 17/43] nfsd4: buffer-length check for SUPPATTR_EXCLCREAT Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 18/43] nfsd4: fix test_stateid error reply encoding Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 19/43] nfsd: notify_change needs elevated write count Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 20/43] nfsd4: fix setclientid encode size Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 21/43] IB/ipath: Fix potential buffer overrun in sending diag packet routine Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 22/43] IB/nes: Return an error on ib_copy_from_udata() failure instead of NULL Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 23/43] IB/mthca: Return an error on ib_copy_to_udata() failure Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 24/43] IB/ehca: Returns " Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 25/43] ib_srpt: Use correct ib_sg_dma primitives Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 26/43] SCSI: arcmsr: upper 32 of dma address lost Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 27/43] iscsi-target: Fix ERL=2 ASYNC_EVENT connection pointer bug Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 28/43] target/tcm_fc: Fix use-after-free of ft_tpg Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 29/43] reiserfs: fix race in readdir Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 30/43] usb: musb: set TXMAXP and AUTOSET for full speed bulk in device mode Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 31/43] xhci: extend quirk for Renesas cards Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 32/43] usb/xhci: fix compilation warning when !CONFIG_PCI && !CONFIG_PM Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 33/43] usb: dwc3: fix wrong bit mask in dwc3_event_devt Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 34/43] hvc: ensure hvc_init is only ever called once in hvc_console.c Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 35/43] USB: unbind all interfaces before rebinding any Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 36/43] sh: fix format string bug in stack tracer Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 37/43] mm: hugetlb: fix softlockup when a large number of hugepages are freed Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 38/43] hung_task: check the value of "sysctl_hung_task_timeout_sec" Greg Kroah-Hartman
2014-05-04 15:42 ` Greg Kroah-Hartman [this message]
2014-05-04 15:42 ` [PATCH 3.4 40/43] ocfs2: dlm: fix recovery hung Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 41/43] ocfs2: do not put bh when buffer_uptodate failed Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 42/43] ext4: use i_size_read in ext4_unaligned_aio() Greg Kroah-Hartman
2014-05-04 15:42 ` [PATCH 3.4 43/43] USB: pl2303: add ids for Hewlett-Packard HP POS pole displays Greg Kroah-Hartman
2014-05-04 15:55 ` [PATCH 3.4 00/43] 3.4.89-stable review Guenter Roeck
2014-05-04 16:09   ` Greg Kroah-Hartman
2014-05-06 14:00 ` Shuah Khan
2014-05-06 14:50   ` Greg Kroah-Hartman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140504154229.569609201@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=akpm@linux-foundation.org \
    --cc=jlbec@evilplan.org \
    --cc=junxiao.bi@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mfasheh@suse.com \
    --cc=srinivas.eeda@oracle.com \
    --cc=stable@vger.kernel.org \
    --cc=sunil.mushran@gmail.com \
    --cc=torvalds@linux-foundation.org \
    --cc=wen.gang.wang@oracle.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).