From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84D7FC04FFE for ; Fri, 17 May 2024 09:44:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:CC:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1ZFTQqudPP0bZKuWCyPZ6VREdGFS0GZxN3vAjmazAFo=; b=4eubb0YbmpB9ZWeS1z1MN0M/VX WJoMEThQUfeXujKuK4tW48vRfcykIjQkMClJjKVbsXNFea4dnUaPrBhnG16oPUywWYZ1uV4Qeww3W fuNa8m9/ZMgqVGH7jQFbbn05gBq/bZt0e/93t+cAHFgdiSEgL0dT//93lJ50dxtsex1XwpTovM+tX bwrpJ+XPEIZzOa9fAufNsX+Wi9WeZWw2SktrtBo8EirvkTkh4TylRGqVfy0fHTvhanUukoCDl70Ev 9OEZAzPMDAuwAEK1RLYkZFCvKMt9pWFJJ3HLMhQLzQCcre6uVFv6V/FLKu5YvmZG6R06OI+7RRUD+ HW8fdHYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s7u8M-00000007Ovc-0dDE; Fri, 17 May 2024 09:44:14 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s7u8H-00000007OrP-2fbE for linux-arm-kernel@lists.infradead.org; Fri, 17 May 2024 09:44:12 +0000 Received: from mail.maildlp.com (unknown [172.19.88.163]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Vghlj1L1xz1j59x; Fri, 17 May 2024 17:40:33 +0800 (CST) Received: from kwepemd200014.china.huawei.com (unknown [7.221.188.8]) by mail.maildlp.com (Postfix) with ESMTPS id 79FE618005F; Fri, 17 May 2024 17:43:57 +0800 (CST) Received: from [10.67.121.177] (10.67.121.177) by kwepemd200014.china.huawei.com (7.221.188.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.28; Fri, 17 May 2024 17:43:56 +0800 CC: Will Deacon , , , , , , , Subject: Re: [PATCH 1/2] perf: arm_pmu: Only show online CPUs in device's "cpus" attribute To: Dongli Zhang References: <20240410095833.63934-1-yangyicong@huawei.com> <20240410153419.GA25171@willie-the-truck> <109dada9-3164-9a04-5b7e-1031ff399017@oracle.com> <87a01991-3631-431c-8654-5b757b03e2e0@oracle.com> From: Yicong Yang Message-ID: <1a67439f-6418-203a-37d8-1682ceb1cb93@huawei.com> Date: Fri, 17 May 2024 17:43:56 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 In-Reply-To: <87a01991-3631-431c-8654-5b757b03e2e0@oracle.com> X-Originating-IP: [10.67.121.177] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemd200014.china.huawei.com (7.221.188.8) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240517_024410_071208_6B23A7C3 X-CRM114-Status: GOOD ( 23.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Dongli, Since it's merge window now, I can resend this along with the userspace perf handling in next cycle. We can continue the discussion then. Thanks. On 2024/5/16 6:10, Dongli Zhang wrote: > Ping? Is there any plan to move forward with the patch from Yicong? > > Thank you very much! > > Dongli Zhang > > On 4/18/24 9:32 AM, Dongli Zhang wrote: >> >> >> On 4/11/24 01:55, Yicong Yang wrote: >>> On 2024/4/10 23:34, Will Deacon wrote: >>>> On Wed, Apr 10, 2024 at 05:58:32PM +0800, Yicong Yang wrote: >>>>> From: Yicong Yang >>>>> >>>>> When there're CPUs offline after system booting, perf will failed: >>>>> [root@localhost ~]# /home/yang/perf stat -a -e armv8_pmuv3_0/cycles/ >>>>> Error: >>>>> The sys_perf_event_open() syscall returned with 19 (No such device) for event (cpu-clock). >>>>> /bin/dmesg | grep -i perf may provide additional information. >>>>> >>>>> This is due to PMU's "cpus" is not updated and still contains offline >>>>> CPUs and perf will try to open perf event on the offlined CPUs. >>>>> >>>>> Make "cpus" attribute only shows online CPUs and introduced a new >>>>> "supported_cpus" where users can get the range of the CPUs this >>>>> PMU supported monitoring. >>>>> >>>>> Signed-off-by: Yicong Yang >>>>> --- >>>>> drivers/perf/arm_pmu.c | 24 +++++++++++++++++++++++- >>>>> 1 file changed, 23 insertions(+), 1 deletion(-) >>>> >>>> Hmm. Is the complexity in the driver really worth it here? CPUs can be >>>> onlined and offlined after the perf_event_open() syscall has been >>>> executed, >>> >>> Yes. So we have cpuhp callbacks to handle the cpu online/offline >>> and migrate the perf context. >>> >>>> so this feels like something userspace should be aware of and >>>> handle on a best-effort basis anyway. >>>> >>> >>> Looks like it's a convention for a PMU device to provide a "cpus" attribute (for core >>> PMUs) or "cpumask" attribute (for uncore PMUs) to indicates the CPUs on which the >>> events can be opened. If no such attributes provided, all online CPUs indicated. Perf >>> will check this and if user doesn't specify a certian range of CPUs the events will >>> be opened on all the CPUs PMU indicated. >>> >>>> Does x86 get away with this because CPU0 is never offlined? >>>> >>> >>> Checked on my x86 server there's no "cpus" or "cpumask" provided so perf will try >>> to open the events on all the online CPUs if no CPU range specified. But for their >>> hybrid platform there do have a "cpus" attribute[1] and it'll be updated when CPU >>> offline[2]. >>> >>> The arm-cspmu also provides a "cpumask" to indicate supported online CPUs and an >>> "associated_cpus" to indicated the CPUs related to the PMU. >>> >>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/events/intel/core.c?h=v6.9-rc1#n5931 >>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/events/intel/core.c?h=v6.9-rc1#n4949 >>> >>> Thanks. >>> >>> >> >> >> The arm_dsu has the concepts of 'cpumask' as well. It also has 'associated_cpus'. >> >> When the current cpumask offline, the cpuhp handler will migrate the cpumask to >> other associated_cpus. >> >> # cat /sys/devices/arm_dsu_26/associated_cpus >> 4-5 >> [root@lse-aarch64-bm-ol8 opc]# cat /sys/devices/arm_dsu_26/cpumask >> 4 >> >> 812 static int dsu_pmu_cpu_online(unsigned int cpu, struct hlist_node *node) >> 813 { >> 814 struct dsu_pmu *dsu_pmu = hlist_entry_safe(node, struct dsu_pmu, >> 815 cpuhp_node); >> 816 >> 817 if (!cpumask_test_cpu(cpu, &dsu_pmu->associated_cpus)) >> 818 return 0; >> 819 >> 820 /* If the PMU is already managed, there is nothing to do */ >> 821 if (!cpumask_empty(&dsu_pmu->active_cpu)) >> 822 return 0; >> 823 >> 824 dsu_pmu_init_pmu(dsu_pmu); >> 825 dsu_pmu_set_active_cpu(cpu, dsu_pmu); >> 826 >> 827 return 0; >> 828 } >> 829 >> 830 static int dsu_pmu_cpu_teardown(unsigned int cpu, struct hlist_node *node) >> 831 { >> 832 int dst; >> 833 struct dsu_pmu *dsu_pmu = hlist_entry_safe(node, struct dsu_pmu, >> 834 cpuhp_node); >> 835 >> 836 if (!cpumask_test_and_clear_cpu(cpu, &dsu_pmu->active_cpu)) >> 837 return 0; >> 838 >> 839 dst = dsu_pmu_get_online_cpu_any_but(dsu_pmu, cpu); >> 840 /* If there are no active CPUs in the DSU, leave IRQ disabled */ >> 841 if (dst >= nr_cpu_ids) >> 842 return 0; >> 843 >> 844 perf_pmu_migrate_context(&dsu_pmu->pmu, cpu, dst); >> 845 dsu_pmu_set_active_cpu(dst, dsu_pmu); >> 846 >> 847 return 0; >> 848 } >> >> >> However, I think the userspace perf tool looks more friendly (just return > supported>) in this case when I offline all CPUs from cpumask of a DSU. Perhaps >> because it is NULL now. >> >> # perf stat -e arm_dsu_26/l3d_cache_wb/ >> ^C >> Performance counter stats for 'system wide': >> >> arm_dsu_26/l3d_cache_wb/ >> >> 0.553294766 seconds time elapsed >> >> >> # cat /sys/devices/arm_dsu_26/associated_cpus >> 4-5 >> # cat /sys/devices/arm_dsu_26/cpumask >> 4 >> # echo 0 > /sys/devices/system/cpu/cpu4/online >> # cat /sys/devices/arm_dsu_26/cpumask >> 5 >> # echo 0 > /sys/devices/system/cpu/cpu5/online >> # cat /sys/devices/arm_dsu_26/cpumask >> >> # >> >> Dongli Zhang > . > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel