From: Heiko Wundram <modelnine-EqIAFqbRPK3NLxjTenLetw@public.gmane.org>
To: linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Hangs on bcache on HEAD and bcache-testing when stacking LVM on top (?)
Date: Thu, 04 Apr 2013 12:27:30 +0200 [thread overview]
Message-ID: <515D5592.60906@modelnine.org> (raw)
Hey!
I've already written previously about write hangs with bcache in "bcache
hangs with continuous write I/O to SSD device, bcache device stops
working", and I thought, that the problem was solved by statically
building bcache into the kernel, but that does not seem to be the case.
I've now encountered something similar, which seems to occur after
around 3-4 days of usage of the bcache device (I've had it twice now).
The bcache device (or rather, the devices stacked on the bcache device
through LVM) hang, and show 100% CPU utilization. All other devices
which are not on the bcache volume still work properly. There's no
obvious kernel thread which is taking CPU time and blocking the device
(as was with the last bug report I noted).
When the bcache volume is blocked, iostat reports:
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
sdb 0,00 0,50 0,00 8,50 0,00 0,03
6,53 0,06 6,88 0,00 6,88 4,88 4,15
sdc 0,00 0,50 0,00 8,50 0,00 0,03
6,53 0,06 7,24 0,00 7,24 5,29 4,50
md127 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
md126 0,00 0,00 0,00 6,50 0,00 0,03
8,00 0,00 0,00 0,00 0,00 0,00 0,00
md125 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-0 0,00 0,00 0,00 6,50 0,00 0,03
8,00 0,13 20,00 0,00 20,00 5,15 3,35
dm-1 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-2 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
bcache0 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-3 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-4 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-5 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-6 0,00 0,00 0,00 0,00 0,00 0,00
0,00 1,00 0,00 0,00 0,00 0,00 100,00
dm-7 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-8 0,00 0,00 0,00 0,00 0,00 0,00
0,00 1,00 0,00 0,00 0,00 0,00 100,00
dm-9 0,00 0,00 0,00 0,00 0,00 0,00
0,00 0,00 0,00 0,00 0,00 0,00 0,00
dm-10 0,00 0,00 0,00 0,00 0,00 0,00
0,00 3,00 0,00 0,00 0,00 0,00 100,00
The devices starting with dm-3 are logical volumes in a volume group
which resides on (the only) physical volume /dev/bcache0, which in turn
is made up of /dev/sda (SSD device) and /dev/md127 (backing volume,
kernel RAID1). /dev/md127 is made up of GPT partitions on /dev/sdb and
/dev/sdc.
The bcache volume and the cache is set to use 512byte logical sectors,
which both devices support (although they both have 4k physical sectors).
There's no real clue as to where I might start to look from here. Is
this a "known" problem when using bcache together with LVM? Memory usage
does not seem to be a culprit.
I'm now trying to boot the system using kernel 3.2 (i.e., the bcache-3.2
tree), and can report about that.
Thanks for any pointers!
--
--- Heiko.
next reply other threads:[~2013-04-04 10:27 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-04 10:27 Heiko Wundram [this message]
[not found] ` <515D5592.60906-EqIAFqbRPK3NLxjTenLetw@public.gmane.org>
2013-04-05 21:29 ` Hangs on bcache on HEAD and bcache-testing when stacking LVM on top (?) Kent Overstreet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=515D5592.60906@modelnine.org \
--to=modelnine-eqiafqbrpk3nlxjtenletw@public.gmane.org \
--cc=linux-bcache-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox