From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A01FC433DB for ; Tue, 23 Mar 2021 18:04:16 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id EFA346192B for ; Tue, 23 Mar 2021 18:04:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFA346192B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 7F14F4B2A6; Tue, 23 Mar 2021 14:04:15 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eXPlDuJ4IN6p; Tue, 23 Mar 2021 14:04:14 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 516F54B2AD; Tue, 23 Mar 2021 14:04:14 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 3AD194B298 for ; Tue, 23 Mar 2021 14:04:13 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nDdu2c3YAOpD for ; Tue, 23 Mar 2021 14:04:12 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id D90104B2A2 for ; Tue, 23 Mar 2021 14:04:11 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A377061920; Tue, 23 Mar 2021 18:04:10 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lOlNw-003MUB-Mv; Tue, 23 Mar 2021 18:04:08 +0000 Date: Tue, 23 Mar 2021 18:04:08 +0000 Message-ID: <87k0pxkaqv.wl-maz@kernel.org> From: Marc Zyngier To: Yoan Picchi Subject: Re: [PATCH 0/7] KVM: arm64: add more event counters for kvm_stat In-Reply-To: <20210319161711.24972-1-yoan.picchi@arm.com> References: <20210319161711.24972-1-yoan.picchi@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: yoan.picchi@arm.com, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, catalin.marinas@arm.com, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: catalin.marinas@arm.com, will@kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hi Yoan, On Fri, 19 Mar 2021 16:17:04 +0000, Yoan Picchi wrote: > > Hi all, > > As mentioned in the KVM forum talk from 2019 > (https://kvmforum2019.sched.com/event/Tmwf/kvmstat-and-beyond-past-present-and-future-of-performance-monitoring-christian-borntrager-ibm > page 10), there is few event counters for kvm_stat in the arm64 > version of kvm when you compare it to something like the x86 > version. Crucially, these differences exist because there is no universal equivalence between architecture features. A TLB invalidation on x86 doesn't do the same thing as on PPC nor arm64. > Those counters are used in kvm_stat by kernel/driver developers to > have a rough idea of the impact of their patches on the general performance. > An example would be to make sure a patch don't increase to much the amount > of interruptions. Those patches aim to add more counters to make the use of > kvm_stat more relevant when measuring performance impact. Adding more counters only make sense if the semantic of these counters is clearly defined. In this series, you have sprayed a bunch of x++ in random places, without defining what you were trying to count. > I am new in working on kernel-related things and I am learning kvm as I go. > Some of the counter I added early (memory_slot_unmaped, stage2_unmap_vm) > no longer seems relevant because while they do interesting things, they > happens in very specific scenarios. Instead of just deleting them, I prefer > to ask weither a little-used counter or no counter is the best. It works the other way around: the onus is on you to explain *why* we should even consider a counter or another. May I suggest that you start by picking exactly *one* metric, work out exactly what it is supposed to count, what significance it has at the architectural level, what it actually brings to the user? Then try and make a good job at implementing these semantics. You will learn about the arm64 architecture and KVM in one swift go, one area at a time. > I can also use some suggestion on how to test those counters as some like > remote_tlb_flush which mostly happen when fixing up a race condition; or > what uncovered event could be worth adding in a future patch set. remote_tlb_flush is a great example of something that really *doesn't* make much sense on KVM/arm64. We don't deal with remote TLBs independently of the local ones outside of vcpu migration (which you don't cover either). I can only urge you to focus on the architectural meaning of the metric you picked, and see how it maps across the hypervisor. Finally, there is the question of the general availability of these counters. They live in debugfs, which isn't a proper userspace ABI. There is work going on around making this a more palatable interface, and I'd rather see where this is going before expanding the number of counters. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E5B0C433E1 for ; Tue, 23 Mar 2021 18:05:50 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 186BF619C0 for ; Tue, 23 Mar 2021 18:05:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 186BF619C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0qJbIosffqzJEoYeqfqAxtn4MxKKU3h1BPGiCeCyPXk=; b=rbG9UXSHcZRFtRJRo1VitjcgE L0E1x+DOA4cjElZLSY/+hVVOe+HvPw+FVqZAh5xXhkgcxsCWJeS8ZsHNVnZ26K7JLtTq4gwhTCmXN W0M71gaz/GNaesVI6PSrr2PUfwU1vPbgIEIfigz6pipxkEVB8huIWx/m9zaUyXvvwxaRrovLlTn9W SEmtVykeSF/qSP+Id4qHNDVPrzC+//E2VQDIWcmtbpi6VhDFEX/f1QREAGuSSAfEQbA4PBw3K659H T1NDAWxn7a/bjRN4HzZylN9BW98Rp7DGPjatp0HM2SrJ1qQSTt2xKPUjYCPCbqmCdLJK6aCzQFbhZ +c2cb47jQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOlO4-00FUAj-Q8; Tue, 23 Mar 2021 18:04:17 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOlO0-00FUA5-CA for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 18:04:14 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A377061920; Tue, 23 Mar 2021 18:04:10 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lOlNw-003MUB-Mv; Tue, 23 Mar 2021 18:04:08 +0000 Date: Tue, 23 Mar 2021 18:04:08 +0000 Message-ID: <87k0pxkaqv.wl-maz@kernel.org> From: Marc Zyngier To: Yoan Picchi Cc: james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, catalin.marinas@arm.com, will@kernel.org Subject: Re: [PATCH 0/7] KVM: arm64: add more event counters for kvm_stat In-Reply-To: <20210319161711.24972-1-yoan.picchi@arm.com> References: <20210319161711.24972-1-yoan.picchi@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: yoan.picchi@arm.com, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, catalin.marinas@arm.com, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_180412_918149_CDFE0A42 X-CRM114-Status: GOOD ( 27.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Yoan, On Fri, 19 Mar 2021 16:17:04 +0000, Yoan Picchi wrote: > > Hi all, > > As mentioned in the KVM forum talk from 2019 > (https://kvmforum2019.sched.com/event/Tmwf/kvmstat-and-beyond-past-present-and-future-of-performance-monitoring-christian-borntrager-ibm > page 10), there is few event counters for kvm_stat in the arm64 > version of kvm when you compare it to something like the x86 > version. Crucially, these differences exist because there is no universal equivalence between architecture features. A TLB invalidation on x86 doesn't do the same thing as on PPC nor arm64. > Those counters are used in kvm_stat by kernel/driver developers to > have a rough idea of the impact of their patches on the general performance. > An example would be to make sure a patch don't increase to much the amount > of interruptions. Those patches aim to add more counters to make the use of > kvm_stat more relevant when measuring performance impact. Adding more counters only make sense if the semantic of these counters is clearly defined. In this series, you have sprayed a bunch of x++ in random places, without defining what you were trying to count. > I am new in working on kernel-related things and I am learning kvm as I go. > Some of the counter I added early (memory_slot_unmaped, stage2_unmap_vm) > no longer seems relevant because while they do interesting things, they > happens in very specific scenarios. Instead of just deleting them, I prefer > to ask weither a little-used counter or no counter is the best. It works the other way around: the onus is on you to explain *why* we should even consider a counter or another. May I suggest that you start by picking exactly *one* metric, work out exactly what it is supposed to count, what significance it has at the architectural level, what it actually brings to the user? Then try and make a good job at implementing these semantics. You will learn about the arm64 architecture and KVM in one swift go, one area at a time. > I can also use some suggestion on how to test those counters as some like > remote_tlb_flush which mostly happen when fixing up a race condition; or > what uncovered event could be worth adding in a future patch set. remote_tlb_flush is a great example of something that really *doesn't* make much sense on KVM/arm64. We don't deal with remote TLBs independently of the local ones outside of vcpu migration (which you don't cover either). I can only urge you to focus on the architectural meaning of the metric you picked, and see how it maps across the hypervisor. Finally, there is the question of the general availability of these counters. They live in debugfs, which isn't a proper userspace ABI. There is work going on around making this a more palatable interface, and I'd rather see where this is going before expanding the number of counters. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel