All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qiang <wangqiang.hunan@gmail.com>
To: ceph-devel@vger.kernel.org
Subject: qemu-kvm guests hang on disk write with rbd storage
Date: Tue, 28 Oct 2014 21:32:37 +0800	[thread overview]
Message-ID: <544F9AF5.1020609@gmail.com> (raw)


Hi, Dear All

I got an issue in my environment: qemu-kvm guests hang on disk write 
with rbd storage.

My environment:
ceph version: 0.80.7
ceph osds: 11(hosts) * 10(osd) = 110
qemu version: 2.0 +

my operating steps:
ceph osd crush add-bucket ssd root
ceph osd getcrushmap -o mycrushmap
crushtool -d mycrushmap -o mycrushmap_v1

#modify mycrushmap_v1
#add 4 of 11 hosts into root=ssd .
#meanwhile the 11 hosts are still in root=default.

crushtool -c mycrushmap_v1 -o mycrushmap_input
ceph osd setcrushmap -i mycrushmap_input
After I doing above steps

In my environment, qemu-kvm VMs which attached ceph rbd storage all 
hung.  The kernel log shows:
kernel: INFO: task jbd2/sdb1-8:623 blocked for more than 120 seconds.
kernel: Not tainted 2.6.32-431.3.1.el6.x86_64 #1
kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
kernel: jbd2/sdb1-8 D 0000000000000001 0 623 2 0x00000000
kernel: ffff88011c44dc20 0000000000000046 ffff8801ffffffff 00000000cc70801d
kernel: ffff88011c44db90 ffff880119466980 00000000d127ef64 ffffffffac2de373
kernel: ffff880119538638 ffff88011c44dfd8 000000000000fbc8 ffff880119538638
kernel: Call Trace:

In the meantime the ceph.log shows everything working fine and the ceph 
health is ok. And The other guest VMs are fine which without ceph rbd 
storage.

I tried many times in my testing environment, But I cannot reproduce it. 
  So that maybe not a problem.

Is there any defect/bug relates to this issue?  Or any suggestion to 
help me find the root cause?

Thanks very much.

                 reply	other threads:[~2014-10-28 13:32 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=544F9AF5.1020609@gmail.com \
    --to=wangqiang.hunan@gmail.com \
    --cc=ceph-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.