From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff Layton Subject: Re: [PATCH v9 0/5] ceph: add perf metrics support Date: Mon, 09 Mar 2020 08:09:21 -0400 Message-ID: <5f1c8c66f573ce499a1232c2346f5baaff413a57.camel@kernel.org> References: <1583739430-4928-1-git-send-email-xiubli@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: Received: from mail.kernel.org ([198.145.29.99]:43162 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726215AbgCIMJX (ORCPT ); Mon, 9 Mar 2020 08:09:23 -0400 In-Reply-To: <1583739430-4928-1-git-send-email-xiubli@redhat.com> Sender: ceph-devel-owner@vger.kernel.org List-ID: To: xiubli@redhat.com, idryomov@gmail.com Cc: sage@redhat.com, zyan@redhat.com, pdonnell@redhat.com, ceph-devel@vger.kernel.org On Mon, 2020-03-09 at 03:37 -0400, xiubli@redhat.com wrote: > From: Xiubo Li > > Changed in V9: > - add an r_ended field to the mds request struct and use that to calculate the metric > - fix some commit comments > > We can get the metrics from the debugfs: > > $ cat /sys/kernel/debug/ceph/0c93a60d-5645-4c46-8568-4c8f63db4c7f.client4267/metrics > item total sum_lat(us) avg_lat(us) > ----------------------------------------------------- > read 13 417000 32076 > write 42 131205000 3123928 > metadata 104 493000 4740 > > item total miss hit > ------------------------------------------------- > d_lease 204 0 918 > caps 204 213 368218 > Thanks Xiubo! This looks good. One minor issue with the cap patch, but I can just fix that up before merging if you're ok with my proposed change. Beyond this...while average latency is a good metric, it's often not enough to help diagnose problems. I wonder if we ought to be at least tracking min/max latency for all calls too. I wonder if there's way to track standard deviation too? That would be really nice to have. Cheers, -- Jeff Layton