From: Alex Elder <elder@ieee.org>
To: Hannes Landeholm <hannes@jumpstarter.io>,
Ceph Development <ceph-devel@vger.kernel.org>
Subject: Re: rbd unmap deadlock
Date: Fri, 02 May 2014 11:09:28 -0500 [thread overview]
Message-ID: <5363C338.1090506@ieee.org> (raw)
In-Reply-To: <CABt9es9QGaVKT7KNtnXhrGNSSdhn12Kgn1e2nUSuUys3A-3Ysg@mail.gmail.com>
On 05/02/2014 11:04 AM, Hannes Landeholm wrote:
> Hi, I just had a rbd unmap operation deadlock on my development
> machine. The file system was in heavy use before I did it but I have a
> sync barrier before the umount and unmap so it shouldn't matter. The
> rbd unmap hanged in "State: D (disk sleep)". I have so far waited
> over 10 minutes, this normally takes < 1 sec.
>
> Here is the /proc/pid/stack output:
>
> [<ffffffff8107e23a>] flush_workqueue+0x11a/0x5a0
> [<ffffffffa031b415>] ceph_msgr_flush+0x15/0x20 [libceph]
> [<ffffffffa03219c6>] ceph_monc_stop+0x46/0x120 [libceph]
> [<ffffffffa031af28>] ceph_destroy_client+0x38/0xa0 [libceph]
> [<ffffffffa0359b88>] rbd_client_release+0x68/0xa0 [rbd]
> [<ffffffffa0359bec>] rbd_put_client+0x2c/0x30 [rbd]
> [<ffffffffa0359c06>] rbd_dev_destroy+0x16/0x30 [rbd]
> [<ffffffffa0359c77>] rbd_dev_image_release+0x57/0x60 [rbd]
> [<ffffffffa035adc7>] do_rbd_remove.isra.25+0x167/0x1b0 [rbd]
> [<ffffffffa035ae54>] rbd_remove+0x24/0x30 [rbd]
> [<ffffffff8136ea67>] bus_attr_store+0x27/0x30
> [<ffffffff81218d4d>] sysfs_kf_write+0x3d/0x50
> [<ffffffff8121c982>] kernfs_fop_write+0xd2/0x140
> [<ffffffff811a67fa>] vfs_write+0xba/0x1e0
> [<ffffffff811a7206>] SyS_write+0x46/0xc0
> [<ffffffff814e66e9>] system_call_fastpath+0x16/0x1b
> [<ffffffffffffffff>] 0xffffffffffffffff
>
> Unfortunately our rbd.ko does not appear to have any debug symbols.
>
> Other unmaps also hanged after this that have the same parent. (We are
> using layering.) Linux version: 3.14.1.
Is this "stock" 3.14.1? Can you provide the full output of "uname -a"?
And if possible, either /proc/config.gz or /boot/config-3.13.1 (or
whichever file seems to match the currently-running kernel)?
Thank you.
-Alex
>
> Thank you for your time,
> --
> Hannes Landeholm
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
next prev parent reply other threads:[~2014-05-02 16:09 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-02 16:04 rbd unmap deadlock Hannes Landeholm
2014-05-02 16:09 ` Hannes Landeholm
2014-05-02 16:09 ` Alex Elder [this message]
2014-05-02 16:23 ` Hannes Landeholm
2014-05-02 16:30 ` Ilya Dryomov
2014-05-02 16:40 ` Hannes Landeholm
2014-05-02 16:52 ` Ilya Dryomov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5363C338.1090506@ieee.org \
--to=elder@ieee.org \
--cc=ceph-devel@vger.kernel.org \
--cc=hannes@jumpstarter.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.