From: Chris Mason <chris.mason@oracle.com>
To: Dave Cundiff <syshackmin@gmail.com>
Cc: "K. Richard Pixley" <rich@noir.com>,
"linux-btrfs@vger.kernel.org" <linux-btrfs@vger.kernel.org>
Subject: Re: Performance degradation
Date: Tue, 21 Sep 2010 20:02:17 -0400 [thread overview]
Message-ID: <20100922000217.GP27633@think> (raw)
In-Reply-To: <AANLkTi=569wzOheD_eeai6O7uaWBxMhRv5_YuUE1dJZb@mail.gmail.com>
On Tue, Sep 21, 2010 at 12:27:03AM -0400, Dave Cundiff wrote:
> On Mon, Sep 20, 2010 at 9:19 PM, K. Richard Pixley <rich@noir.com> wr=
ote:
> > Do you have any kernel processes running constantly? =A0Any taking =
near 100% of a CPU?
> >
> > --rich
> >
>=20
> Now that I look again, it seems that the btrfs-cleaner process is
> still running. CPU isn't much of an issue on this box. Its a dual qua=
d
> E5620. 8 cores, 16 threads.
> Looks like most of my load is wait from the cleaner process hammering
> the disks. I deleted about 10 snapshots I think. Should this take 4
> days to clean up from that?
It shouldn't but it depends on how much metadata we have to read in to
process the snapshots. Could you do a few sysrq-w? We'll see where yo=
u
are spending your time based on the traces.
-chris
>=20
> Cpu0 : 0.0%us, 0.2%sy, 0.0%ni, 85.1%id, 14.7%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu1 : 0.0%us, 0.4%sy, 0.0%ni, 66.9%id, 32.6%wa, 0.0%hi, 0.1%si=
, 0.0%st
> Cpu2 : 0.0%us, 0.2%sy, 0.0%ni, 88.7%id, 11.1%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu3 : 0.0%us, 0.1%sy, 0.0%ni, 96.7%id, 3.2%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu4 : 0.0%us, 0.2%sy, 0.0%ni, 92.3%id, 7.5%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu5 : 0.0%us, 0.1%sy, 0.0%ni, 97.6%id, 2.4%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu6 : 0.0%us, 0.1%sy, 0.0%ni, 98.5%id, 1.5%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu7 : 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu8 : 0.0%us, 0.1%sy, 0.0%ni, 95.7%id, 4.3%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu9 : 0.0%us, 1.6%sy, 0.0%ni, 30.2%id, 68.2%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu10 : 0.0%us, 0.0%sy, 0.0%ni, 95.1%id, 4.8%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu11 : 0.0%us, 0.0%sy, 0.0%ni, 99.6%id, 0.4%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu12 : 0.0%us, 0.4%sy, 0.0%ni, 83.8%id, 15.7%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu13 : 0.0%us, 0.0%sy, 0.0%ni, 98.1%id, 1.8%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu14 : 0.0%us, 0.0%sy, 0.0%ni, 99.7%id, 0.3%wa, 0.0%hi, 0.0%si=
, 0.0%st
> Cpu15 : 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si=
, 0.0%st
>=20
> Time: 12:24:01 AM
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-s=
z
> avgqu-sz await svctm %util
> sdb 16.25 6.49 127.45 330.65 1173.03 3946.46
> 11.18 70.55 154.01 2.13 97.71
>=20
> Time: 12:24:21 AM
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-s=
z
> avgqu-sz await svctm %util
> sdb 5.20 2.70 92.55 270.70 782.00 2795.20
> 9.85 70.12 189.42 2.69 97.85
>=20
> Time: 12:24:41 AM
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-s=
z
> avgqu-sz await svctm %util
> sdb 0.25 11.90 5.20 863.35 43.60 9391.60
> 10.86 144.84 168.01 1.15 100.00
>=20
> Time: 12:25:01 AM
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-s=
z
> avgqu-sz await svctm %util
> sdb 17.50 0.60 204.35 49.70 1776.00 524.00
> 9.05 18.64 74.23 3.74 95.03
>=20
> --=20
> Dave Cundiff
> System Administrator
> A2Hosting, Inc
> http://www.a2hosting.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2010-09-22 0:02 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-09-20 21:24 Performance degradation Dave Cundiff
2010-09-21 1:19 ` K. Richard Pixley
2010-09-21 4:27 ` Dave Cundiff
2010-09-22 0:02 ` Chris Mason [this message]
2010-09-22 12:03 ` Daniel J Blueman
2010-09-24 19:39 ` Dave Cundiff
2010-09-26 14:05 ` Daniel J Blueman
-- strict thread matches above, loose matches on Subject: below --
2017-10-02 16:57 performance degradation k
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100922000217.GP27633@think \
--to=chris.mason@oracle.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=rich@noir.com \
--cc=syshackmin@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.