From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7651AC433FE for ; Tue, 25 Jan 2022 07:22:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vaKEvCjM+h4SjsHRQEdMX6/phf+N0SmHjJiZyQkXmUU=; b=khUNaGEJwLKdWB K0eIsCBrJxUc7YppGjqzb8gAvBOHoPX7SBxQ3uitkcfLVFARp1D/PtspH70PO4chikRjCPaBhKMTk wbLawimjoIALtT0NuLp9Xue8YNKISUhlQWg+SkkXvoL1PAexKPuEsYKBr41pIbSEM6xIS55MrNO8+ 1zScR1ditDr5y6w0ZWlBnC7awruO0J+jNYnQZl2JJcilps1sJGPAHL2l2aWnc3pl+l9E34wzXr9bt N/iv6X7Jitd53fVpOQZkHOMUepb/yg8vd0e33gkAx5OGu7WN3zLlMJ3NoC+snT5YeFS3V5Wim74v6 vKNEigAS605UiRhILEgw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nCG7v-006gcZ-6O; Tue, 25 Jan 2022 07:20:27 +0000 Received: from esa6.hc1455-7.c3s2.iphmx.com ([68.232.139.139]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nCG22-006eOr-0R for linux-arm-kernel@lists.infradead.org; Tue, 25 Jan 2022 07:14:24 +0000 IronPort-SDR: 8AAwvX9OT8dbn//frT9ugJgKbWnoyUERymXG6RMtVRfzzY0oW812px/KfQaplpGJOln+vGMYl8 HP1daLs+tcP4Fl27wzNGJHRjB4mkIcAaO8JTr23R7WnIpefO+F4RL0Y9aJtdIY1sXI1RL9oygt nb1986WDe83/BL4+y3lxPEDz88vcsdyR3/jYXnwLZUH3eleOXstLXOLHVq7tYS6AnHZMxe0nK6 jXwoAB3Jw/wU9tq7y2txTF0T1YcL9mB62P4tJFBjrZnAjxtBsozMfmVgkeRx6TkE2puCXJmMpQ re/NvHzU0JK8niDnPYWXMx9V X-IronPort-AV: E=McAfee;i="6200,9189,10237"; a="60807253" X-IronPort-AV: E=Sophos;i="5.88,314,1635174000"; d="scan'208";a="60807253" Received: from unknown (HELO oym-r3.gw.nic.fujitsu.com) ([210.162.30.91]) by esa6.hc1455-7.c3s2.iphmx.com with ESMTP; 25 Jan 2022 16:14:17 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r3.gw.nic.fujitsu.com (Postfix) with ESMTP id 097F9CA242 for ; Tue, 25 Jan 2022 16:14:18 +0900 (JST) Received: from yto-om4.fujitsu.com (yto-om4.o.css.fujitsu.com [10.128.89.165]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 31D88D95EF for ; Tue, 25 Jan 2022 16:14:17 +0900 (JST) Received: from localhost.localdomain (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om4.fujitsu.com (Postfix) with ESMTP id DB25C4007ED6C; Tue, 25 Jan 2022 16:14:16 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH 3/8] arm64: Add hardware prefetch control support for ARM64 Date: Tue, 25 Jan 2022 16:14:09 +0900 Message-Id: <20220125071414.811344-4-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220125071414.811344-1-tarumizu.kohei@fujitsu.com> References: <20220125071414.811344-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220124_231422_400205_772B9E57 X-CRM114-Status: GOOD ( 21.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This adds module init/exit code, and creates sysfs attribute files for "prefetch_control". This driver works only if part number is FUJITSU_CPU_PART_A64FX at this point. The details of the registers to be read and written in this patch are described below. "https://github.com/fujitsu/A64FX/tree/master/doc/" A64FX_Specification_HPC_Extension_v1_EN.pdf Signed-off-by: Kohei Tarumizu --- arch/arm64/kernel/pfctl.c | 324 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 324 insertions(+) create mode 100644 arch/arm64/kernel/pfctl.c diff --git a/arch/arm64/kernel/pfctl.c b/arch/arm64/kernel/pfctl.c new file mode 100644 index 000000000000..14f4b8248280 --- /dev/null +++ b/arch/arm64/kernel/pfctl.c @@ -0,0 +1,324 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2022 FUJITSU LIMITED + * + * ARM64 Hardware Prefetch Control support + */ + +#include +#include +#include +#include +#include +#include + +struct pfctl_driver arm64_pfctl_driver; + +/************************************** + * FUJITSU A64FX support + **************************************/ + +/* + * Constants for these add the "A64FX_SDPF" prefix to the name described in + * section "1.3.4.2. IMP_PF_STREAM_DETECT_CTRL_EL0" of "A64FX specification". + * (https://github.com/fujitsu/A64FX/tree/master/doc/A64FX_Specification_HPC_Extension_v1_EN.pdf") + * See this document for register specification details. + */ +#define A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0 sys_reg(3, 3, 11, 4, 0) +#define A64FX_SDPF_V BIT_ULL(63) +#define A64FX_SDPF_L1PF_DIS BIT_ULL(59) +#define A64FX_SDPF_L2PF_DIS BIT_ULL(58) +#define A64FX_SDPF_L1W BIT_ULL(55) +#define A64FX_SDPF_L2W BIT_ULL(54) +#define A64FX_SDPF_L1_DIST GENMASK_ULL(27, 24) +#define A64FX_SDPF_L2_DIST GENMASK_ULL(19, 16) + +#define A64FX_SDPF_MIN_DIST_L1 256 +#define A64FX_SDPF_MIN_DIST_L2 1024 + +struct a64fx_read_info { + struct prefetcher_options *opts; + unsigned int level; + int ret; +}; + +struct a64fx_write_info { + struct prefetcher_options *opts; + unsigned int level; + int ret; +}; + +static int a64fx_get_sdpf_enable(u64 reg, unsigned int level) +{ + switch (level) { + case 1: + return FIELD_GET(A64FX_SDPF_L1PF_DIS, reg); + case 2: + return FIELD_GET(A64FX_SDPF_L2PF_DIS, reg); + default: + return -EINVAL; + } +} + +static int a64fx_modify_sdpf_enable(u64 *reg, unsigned int level, int val) +{ + switch (level) { + case 1: + *reg &= ~A64FX_SDPF_L1PF_DIS; + *reg |= FIELD_PREP(A64FX_SDPF_L1PF_DIS, val); + break; + case 2: + *reg &= ~A64FX_SDPF_L2PF_DIS; + *reg |= FIELD_PREP(A64FX_SDPF_L2PF_DIS, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int a64fx_get_sdpf_dist(u64 reg, unsigned int level) +{ + switch (level) { + case 1: + return FIELD_GET(A64FX_SDPF_L1_DIST, reg) * + A64FX_SDPF_MIN_DIST_L1; + case 2: + return FIELD_GET(A64FX_SDPF_L2_DIST, reg) * + A64FX_SDPF_MIN_DIST_L2; + default: + return -EINVAL; + } +} + +static int a64fx_modify_sdpf_dist(u64 *reg, unsigned int level, int val) +{ + switch (level) { + case 1: + val = roundup(val, A64FX_SDPF_MIN_DIST_L1) / + A64FX_SDPF_MIN_DIST_L1; + if (!FIELD_FIT(A64FX_SDPF_L1_DIST, val)) + return -EINVAL; + *reg &= ~A64FX_SDPF_L1_DIST; + *reg |= FIELD_PREP(A64FX_SDPF_L1_DIST, val); + break; + case 2: + val = roundup(val, A64FX_SDPF_MIN_DIST_L2) / + A64FX_SDPF_MIN_DIST_L2; + if (!FIELD_FIT(A64FX_SDPF_L2_DIST, val)) + return -EINVAL; + *reg &= ~A64FX_SDPF_L2_DIST; + *reg |= FIELD_PREP(A64FX_SDPF_L2_DIST, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int a64fx_get_sdpf_strong(u64 reg, unsigned int level) +{ + switch (level) { + case 1: + return FIELD_GET(A64FX_SDPF_L1W, reg); + case 2: + return FIELD_GET(A64FX_SDPF_L2W, reg); + default: + return -EINVAL; + } +} + +static int a64fx_modify_sdpf_strong(u64 *reg, unsigned int level, int val) +{ + switch (level) { + case 1: + *reg &= ~A64FX_SDPF_L1W; + *reg |= FIELD_PREP(A64FX_SDPF_L1W, val); + break; + case 2: + *reg &= ~A64FX_SDPF_L2W; + *reg |= FIELD_PREP(A64FX_SDPF_L2W, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static void a64fx_enable_sdpf_verify(u64 *reg) +{ + *reg &= ~A64FX_SDPF_V; + *reg |= FIELD_PREP(A64FX_SDPF_V, 1); +} + +static int a64fx_get_sdpf_params(struct prefetcher_options *opts, u64 reg, + unsigned int level) +{ + int ret; + + ret = a64fx_get_sdpf_enable(reg, level); + if (ret < 0) + return ret; + opts->sdpf_enable = ret; + + ret = a64fx_get_sdpf_dist(reg, level); + if (ret < 0) + return ret; + opts->sdpf_dist = ret; + + ret = a64fx_get_sdpf_strong(reg, level); + if (ret < 0) + return ret; + opts->sdpf_strong = ret; + + return 0; +} + +static int a64fx_modify_pfreg_val(u64 *reg, struct prefetcher_options *opts, + unsigned int level) +{ + int ret; + + if (opts->sdpf_enable != PFCTL_PARAM_UNSET) { + ret = a64fx_modify_sdpf_enable(reg, level, opts->sdpf_enable); + if (ret < 0) + return ret; + } + + if (opts->sdpf_dist != PFCTL_PARAM_UNSET) { + ret = a64fx_modify_sdpf_dist(reg, level, opts->sdpf_dist); + if (ret < 0) + return ret; + } + + if (opts->sdpf_strong != PFCTL_PARAM_UNSET) { + ret = a64fx_modify_sdpf_strong(reg, level, opts->sdpf_strong); + if (ret < 0) + return ret; + } + + a64fx_enable_sdpf_verify(reg); + + return 0; +} + +static void _a64fx_read_pfreg(void *info) +{ + u64 reg; + struct a64fx_read_info *rinfo = info; + + reg = read_sysreg_s(A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0); + + rinfo->ret = a64fx_get_sdpf_params(rinfo->opts, reg, rinfo->level); +} + +static int a64fx_read_pfreg(unsigned int cpu, unsigned int level, + struct prefetcher_options *opt) +{ + struct a64fx_read_info info = { + .level = level, + .opts = opt, + }; + + smp_call_function_single(cpu, _a64fx_read_pfreg, &info, true); + return info.ret; +} + +static void _a64fx_write_pfreg(void *info) +{ + int ret; + u64 reg; + struct a64fx_write_info *winfo = info; + + reg = read_sysreg_s(A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0); + + ret = a64fx_modify_pfreg_val(®, winfo->opts, winfo->level); + if (ret < 0) { + winfo->ret = ret; + return; + } + + write_sysreg_s(reg, A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0); + + winfo->ret = 0; +} + +static int a64fx_write_pfreg(unsigned int cpu, unsigned int level, + struct prefetcher_options *opt) +{ + struct a64fx_write_info info = { + .level = level, + .opts = opt, + }; + + smp_call_function_single(cpu, _a64fx_write_pfreg, &info, true); + return info.ret; +} + +/***** end of FUJITSU A64FX support *****/ + +/* + * This driver returns a negative value if it does not support the Hardware + * Prefetch Control or if it is running on a VM guest. + */ +static int __init setup_pfctl_driver_params(void) +{ + unsigned long implementor = read_cpuid_implementor(); + unsigned long part_number = read_cpuid_part_number(); + + if (!is_kernel_in_hyp_mode()) + return -EINVAL; + + switch (implementor) { + case ARM_CPU_IMP_FUJITSU: + switch (part_number) { + case FUJITSU_CPU_PART_A64FX: + /* A64FX register requires EL2 access */ + if (!has_vhe()) + return -EINVAL; + + arm64_pfctl_driver.supported_l1d_prefetcher = SDPF; + arm64_pfctl_driver.supported_l2_prefetcher = SDPF; + arm64_pfctl_driver.read_pfreg = a64fx_read_pfreg; + arm64_pfctl_driver.write_pfreg = a64fx_write_pfreg; + break; + default: + return -ENODEV; + } + break; + default: + return -ENODEV; + } + + return 0; +} + +static int __init arm64_pfctl_init(void) +{ + int ret; + + ret = setup_pfctl_driver_params(); + if (ret < 0) + return ret; + + ret = pfctl_register_driver(&arm64_pfctl_driver); + if (ret < 0) + return ret; + + return 0; +} + +static void __exit arm64_pfctl_exit(void) +{ + pfctl_unregister_driver(&arm64_pfctl_driver); +} + +late_initcall(arm64_pfctl_init); +module_exit(arm64_pfctl_exit); + +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("FUJITSU LIMITED"); +MODULE_DESCRIPTION("ARM64 Prefetch Control Driver"); -- 2.27.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel