From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DA778C433F5 for ; Mon, 18 Apr 2022 12:03:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SlCXi0LRIpMz5M7gu0cjOgBDtvP7z+oBU5HqMWOpwuo=; b=pWmf3kt/3wDFv2 aDgdcxEubH/GNsoWVFUH66VT8FrvUaqJmd+152xtn3Mhq9vQcwhYWsCoTC8a36hvtUDXG57q6pvRt wCfqv6lZmnD93QfFaf2zRoX/bZTcc38m5Zonkw7wjcZ4jyC6owj/ZX1OuR5lMyspgHUaTlwhL64y+ kcu90XKvdRHr64I8PaaF5AVQEZwrFsheowVI4r7Oz2EPPFsm7ff7HcfimoPjPtP2k8btn/c2IpMRM CG1NcFtAcXT0L+XQVcvMhzkhgdPGqf6USvhKOx8gQc6lwkxq4kiUxDYC8aQvcSr01hyNm1Qez0vly b/RHG8WGRRPuegSKoXtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ngQ57-00Gb2P-E4; Mon, 18 Apr 2022 12:02:13 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ngQ53-00Gb0e-EX for linux-arm-kernel@lists.infradead.org; Mon, 18 Apr 2022 12:02:11 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id AE23BCE107E; Mon, 18 Apr 2022 12:01:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBB05C385A1; Mon, 18 Apr 2022 12:01:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650283317; bh=rDjmoK8WPSGStOZqE1cIQ0+lncSatC5LwnyG6mZ6eg0=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=I4zDDjisIsN/uTj2yAwSPe0yzyLC0iv8r3EkCoyE7jEtPN8wAHyhX3l2BZvH7kbee xd1C5pKDzT3kQvclF5GmaM8vjRF5kh9E7d2KRfXmxpybMIfmTK6PIi/xbIhx2naLkM YFNh0EbESNPTux/vN5yeEQDLrfQQJ1u9mZXUK/VO1BPN5GKw4SLCH4Kk71DWKE044J MnQUPzWNcW33/UFlBHBMuT5P1PJfTGJfw1TvsWApkDAkC4UjwBBp3bMM6UXDNspLns zBdCOes/u4Odq9f6CGCkDn8fZe+4C/0j8Sd1tncmFbgr+1ajaO5OHo/foPqpEw1d/6 0mJZoFsaagREQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ngQ4o-004zfb-9G; Mon, 18 Apr 2022 13:01:54 +0100 Date: Mon, 18 Apr 2022 13:01:53 +0100 Message-ID: <87o80yior2.wl-maz@kernel.org> From: Marc Zyngier To: Yichao Yu Cc: linux-arm-kernel@lists.infradead.org, khuong@os.amperecomputing.com, will@kernel.org, mark.rutland@arm.com, Frank.li@nxp.com, zhangshaokun@hisilicon.com, liuqi115@huawei.com, john.garry@huawei.com, mathieu.poirier@linaro.org, leo.yan@linaro.org Subject: Re: Kernel perf counter support (for apple M1 and others) In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: yyc1992@gmail.com, linux-arm-kernel@lists.infradead.org, khuong@os.amperecomputing.com, will@kernel.org, mark.rutland@arm.com, Frank.li@nxp.com, zhangshaokun@hisilicon.com, liuqi115@huawei.com, john.garry@huawei.com, mathieu.poirier@linaro.org, leo.yan@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220418_050209_910618_48F2C1E7 X-CRM114-Status: GOOD ( 56.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, Please make sure you use current email addresses (the MAINTAINERS file should be accurate for any recent kernel version). On Fri, 01 Apr 2022 02:39:39 +0100, Yichao Yu wrote: > > Hi, > > I am playing with the performance counters on the apple M1 chip from > linux with the hope that it could help making userspace tools like > perf and rr works on the M1. However, I was told that none of these > info should go into the kernel (not even raw event names) and the > userspace should only use the raw event numbers instead of > PERF_TYPE_HARDWARE even for events that have a canonical counterpart. Since I was the one who had a brief chat with you on IRC, let me clarify what I said exactly: - I don't think there is any value in stashing any of these HW events in the kernel. In most cases, the kernel definition only matches the x86 definition, and doesn't accurately describe the vast majority of the events implemented on an ARM CPU. The ARM architecture mentions a handful of architectural events that actually match the kernel definition, and for these CPUs the kernel carries the in-kernel description. - For the M1, none of the above applies, because there is *NO* architectural description for the events reported by the (non architectural) PMU, and there is no guarantee that they actually match the common understanding we have of these events. - The correct place for these non-architectural events is in a JSON description that would be built into perf, which would give you symbolic events. Bloating the kernel for something we're not sure about seems counterproductive. > Although I'm not planning to submit any kernel patches anytime soon > and I'm mostly interested in running the test right now, I do want to > know what I should expect in the long term on the userspace side. I > was told to ask about this on "the list" (and I'm hoping this is the > right one after browsing through MAINTAINERS) instead. There are a few > issues/questions, not all of which are related to M1/asymmetric > systems. For context, see > https://oftc.irclog.whitequark.org/asahi-dev/2022-03-30 (there also > happens to be no other discussion on the channel that day) > > 1. Is it acceptable (to either kernel or perf source) to submit > patches that are based on a14.plist from macOS. I have personally > never looked at it but if it is acceptable then there's little point > doing the experiment I was doing (apart from the fun doing so and as a > practice to understand the system). My take on this is "I am not a lawyer". The MacOS file is Apple's intellectual property, and I'm not prepared to use it, transform it, or interpret it in any way. At best, giving people a way to use this file on their own system without distributing it would be a step in the right direction (it should be rather simple to turn this file into the JSON format that perf uses). Now, if someone with the right level of IP law wants to take responsibility for this, I'm not going to get in the way. I'm just not going to be the one looking at it or taking the patch. > 2. Should the kernel provide names for hardware events? Here I'm > talking about things under > `/sys/bus/event_source/devices//events` which I assume is > provided by the kernel (that or my understanding of sysfs has been > fundamentally wrong/out-of-date...). Based on the fact that the > current pmu kernel driver for the M1 does provide this and this > comment https://github.com/torvalds/linux/blob/e8b767f5e04097aaedcd6e06e2270f9fe5282696/drivers/perf/apple_m1_cpu_pmu.c#L31 > I assume it's desired. This would also agree with what I've observed > on other (including non-x86) systems. If this is the case, I assume > the kernel driver for the M1 PMU isn't fully "done" yet. See my reply above: there are no architectural descriptions for these events, and we don't know how closely they match the definition Linux has. If one day Apple shows up and tells us how close these events are from their Linux (and thus x86) definition, we can expand this. Until then, the interpretation belongs, IMHO, to userspace. I'd rather *remove* CYCLES and INSTRUCTIONS definitions from the kernel than add any other. > 3. For counting events on a system with asymmetric cores. > I understand that if the system contains multiple processors of > different characteristics, it may not make sense to provide a counter > that counts events on both (or all) types of cores. However, there are > events (PERF_COUNT_HW_INSTRUCTIONS and > PERF_COUNT_HW_BRANCH_INSTRUCTIONS at the least) that shouldn't really > be affected by this (and in fact, any counters that counts events > visible directly to the software/userspace). I want to even say that > branch misses/cache reference/misses might be in this category as well > although certainly not as clear cut. That boat has sailed a long time ago, when the BL PMU support was introduced, and all counters are treated equally: they are *NOT* counted globally. Changing this would be an ABI break, and I seriously doubt we want to go there. It would also mean that the kernel would need to know which counters it can accumulate over the various CPU types (which is often more than 2, these days). All of that to save userspace adding things? I doubt this is worth it. > 4. There are other events that may not make as much sense to combine > (cycles for example). However, I feel like a combined cycle count > isn't going to be much tricker to use given that the cycle count on a > single core is still affected by frequency scaling and it can still be > used correctly by pinning the thread. I don't understand what frequency scaling has anything to do with this (a cycle is still a cycle at any frequency). > > The main reasons I'm asking about 3 and 4 is that > 1. Right now, even to just count instructions without pinning the > thread, I need to create two counters. How bad is that? I mean, the counters are per-CPU anyway, so there *are* N counters (N being the number of CPUs). You only have to create a counter per PMU. > 2. Even if the number isn't exactly accurate, it can still be useful > as a general guideline. Right now, even if I just want to do a quick > check, I still need to manually specify a dozen of events in `perf > stat -e` rather than simply using `perf stat` (to make it worse, perf > doesn't even provide any useful warning about it). It is also much > harder to do things generically (which is at least partially because > of the lack of documentation....). I see this as a potential perf-tool improvement. Being able to say 'Count this event on all CPU PMUs' would certainly be valuable to all asymmetric systems. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel