* Re: [PATCH v2 6/6] nvme-apple: Add initial Apple SoC NVMe driver
From: hch @ 2022-04-20 4:34 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Sven Peter, hch@lst.de, Keith Busch, axboe@fb.com,
sagi@grimberg.me, Hector Martin, Alyssa Rosenzweig, Rob Herring,
Krzysztof Kozlowski, Marc Zyngier, DTML, Linux ARM,
Linux Kernel Mailing List, linux-nvme, linux-spdx
In-Reply-To: <CAK8P3a2CWZb3vdHQvseJZQwPbUFDRo5Z9aGb7iVgodT1YeB5Yw@mail.gmail.com>
On Tue, Apr 19, 2022 at 11:52:15AM +0200, Arnd Bergmann wrote:
> > I just checked again and 64-bit accesses seem to work fine.
> > I'll remove the lo_hi_* calls and this include.
>
> If you remove the #include, it is no longer possible to compile-test
> this on all 32-bit architectures, though that is probably fine as long
> as the Kconfig file has the right dependencies, like
>
> depends on ARCH_APPLE || (COMPILE_TEST && 64BIT)
>
> I'd prefer to keep the #include here, but I don't mind the dependency
> if Christoph prefers it that way.
So thre's really two steps here:
1) stop uing lo_hi_readq diretly which forces 32-bit access even on
64-bit platforms
2) stop using the io-64-nonatomic headers entirely
I definitively want 1) done if the hardware does not require it. Trying
to cater to 32-bit build tests on hardware that has no chance of ever
being used there by including the header seems a bit silly, but if
it makes folks happy I can live with it.
>
> Arnd
---end quoted text---
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH 2/2 v3] arm64: dts: fsl-ls1028a: add dspi2 support
From: Changming Huang @ 2022-04-20 3:50 UTC (permalink / raw)
To: broonie, robh+dt, krzysztof.kozlowski+dt, linux-spi, devicetree,
linux-kernel, shawnguo, leoyang.li, linux-arm-kernel
Cc: Changming Huang
In-Reply-To: <20220420035045.33940-1-jerry.huang@nxp.com>
Enable MikroBUS SPI port.
Signed-off-by: Changming Huang <jerry.huang@nxp.com>
---
changes in v3:
- add the dt-bindings
arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts
index 68c31cb8eead..6d0508d5abd0 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a-rdb.dts
@@ -117,6 +117,19 @@ &duart1 {
status = "okay";
};
+&dspi2 {
+ bus-num = <2>;
+ status = "okay";
+
+ mikrobus@0 {
+ compatible = "semtech,sx1301";
+ reg = <0>;
+ spi-max-frequency = <2000000>;
+ fsl,spi-cs-sck-delay = <1000000>;
+ fsl,spi-sck-cs-delay = <50>;
+ };
+};
+
&enetc_mdio_pf3 {
sgmii_phy0: ethernet-phy@2 {
reg = <0x2>;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH 1/2 v3] dt-bindings: dspi: added for semtech sx1301
From: Changming Huang @ 2022-04-20 3:50 UTC (permalink / raw)
To: broonie, robh+dt, krzysztof.kozlowski+dt, linux-spi, devicetree,
linux-kernel, shawnguo, leoyang.li, linux-arm-kernel
Cc: Changming Huang
Add DT Binding doc for semtech sx1301
Signed-off-by: Changming Huang <jerry.huang@nxp.com>
---
changes in v3:
- add the dt-bindings
.../bindings/spi/semtech,sx1301.yaml | 45 +++++++++++++++++++
1 file changed, 45 insertions(+)
create mode 100644 Documentation/devicetree/bindings/spi/semtech,sx1301.yaml
diff --git a/Documentation/devicetree/bindings/spi/semtech,sx1301.yaml b/Documentation/devicetree/bindings/spi/semtech,sx1301.yaml
new file mode 100644
index 000000000000..f65fb5809218
--- /dev/null
+++ b/Documentation/devicetree/bindings/spi/semtech,sx1301.yaml
@@ -0,0 +1,45 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/spi/semtech,sx1301.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Semtech sx1301 devicetree bindings
+
+allOf:
+ - $ref: "spi-controller.yaml"
+
+maintainers:
+ - Changming Huang <jerry.huang@nxp.com>
+
+properties:
+ compatible:
+ const: semtech,sx1301
+
+ reg:
+ maxItems: 1
+
+ spi-max-frequency: true
+
+ fsl,spi-cs-sck-delay: true
+
+ fsl,spi-sck-cs-delay: true
+
+required:
+ - compatible
+ - reg
+ - spi-max-frequency
+
+additionalProperties: false
+
+examples:
+ - |
+ mikrobus@0 {
+ compatible = "semtech,sx1301";
+ reg = <0>;
+ spi-max-frequency = <2000000>;
+ fsl,spi-cs-sck-delay = <1000000>;
+ fsl,spi-sck-cs-delay = <50>;
+ };
+
+...
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* Re: [PATCH v4] ARM: module: Add all unwind tables when load module
From: Chen Zhongjin @ 2022-04-20 3:37 UTC (permalink / raw)
To: Russell King (Oracle)
Cc: alexander.sverdlin, ardb, linus.walleij, nico, linux-arm-kernel,
linux-kernel
In-Reply-To: <20220401131534.241205-1-chenzhongjin@huawei.com>
ping for review, thanks!
On 2022/4/1 21:15, Chen Zhongjin wrote:
> For EABI stack unwinding, when loading .ko module
> the EXIDX sections will be added to a unwind_table list.
>
> However not all EXIDX sections are added because EXIDX
> sections are searched by hardcoded section names.
>
> For functions in other sections such as .ref.text
> or .kprobes.text, gcc generates seprated EXIDX sections
> (such as .ARM.exidx.ref.text or .ARM.exidx.kprobes.text).
>
> These extra EXIDX sections are not loaded, so when unwinding
> functions in these sections, we will failed with:
>
> unwind: Index not found xxx
>
> To fix that, I refactor the code for searching and adding
> EXIDX sections:
>
> - Check section type to search EXIDX tables (0x70000001)
> instead of strcmp() the hardcoded names. Then find the
> corresponding text sections by their section names.
>
> - Add a unwind_table list in module->arch to save their own
> unwind_table instead of the fixed-lenth array.
>
> - Save .ARM.exidx.init.text section ptr, because it should
> be cleaned after module init.
>
> Now all EXIDX sections of .ko can be added correctly.
>
> Signed-off-by: Chen Zhongjin <chenzhongjin@huawei.com>
> ---
> Changes v3 -> v4:
> - Fix indent by replace tab with code.
>
> Changes v2 -> v3:
> - Unwind "txtname" and add some blank, to make code more clear.
>
> Changes v1 -> v2:
> - Rename "table_list" to "unwind_list" for consistency.
> - Drop unnecessary locals variable "txtname" and "sectype".
> - Merge condition checks for sh_flags and sh_type.
> ---
> arch/arm/include/asm/module.h | 17 ++------
> arch/arm/include/asm/unwind.h | 1 +
> arch/arm/kernel/module.c | 78 ++++++++++++++++++-----------------
> 3 files changed, 45 insertions(+), 51 deletions(-)
>
> diff --git a/arch/arm/include/asm/module.h b/arch/arm/include/asm/module.h
> index cfffae67c04e..8139b6a33a22 100644
> --- a/arch/arm/include/asm/module.h
> +++ b/arch/arm/include/asm/module.h
> @@ -3,20 +3,10 @@
> #define _ASM_ARM_MODULE_H
>
> #include <asm-generic/module.h>
> -
> -struct unwind_table;
> +#include <asm/unwind.h>
>
> #ifdef CONFIG_ARM_UNWIND
> -enum {
> - ARM_SEC_INIT,
> - ARM_SEC_DEVINIT,
> - ARM_SEC_CORE,
> - ARM_SEC_EXIT,
> - ARM_SEC_DEVEXIT,
> - ARM_SEC_HOT,
> - ARM_SEC_UNLIKELY,
> - ARM_SEC_MAX,
> -};
> +#define ELF_SECTION_UNWIND 0x70000001
> #endif
>
> #define PLT_ENT_STRIDE L1_CACHE_BYTES
> @@ -36,7 +26,8 @@ struct mod_plt_sec {
>
> struct mod_arch_specific {
> #ifdef CONFIG_ARM_UNWIND
> - struct unwind_table *unwind[ARM_SEC_MAX];
> + struct unwind_table unwind_list;
> + struct unwind_table *init_table;
> #endif
> #ifdef CONFIG_ARM_MODULE_PLTS
> struct mod_plt_sec core;
> diff --git a/arch/arm/include/asm/unwind.h b/arch/arm/include/asm/unwind.h
> index 0f8a3439902d..b51f85417f58 100644
> --- a/arch/arm/include/asm/unwind.h
> +++ b/arch/arm/include/asm/unwind.h
> @@ -24,6 +24,7 @@ struct unwind_idx {
>
> struct unwind_table {
> struct list_head list;
> + struct list_head mod_list;
> const struct unwind_idx *start;
> const struct unwind_idx *origin;
> const struct unwind_idx *stop;
> diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
> index 549abcedf795..b98bcb47e337 100644
> --- a/arch/arm/kernel/module.c
> +++ b/arch/arm/kernel/module.c
> @@ -459,46 +459,41 @@ int module_finalize(const Elf32_Ehdr *hdr, const Elf_Shdr *sechdrs,
> #ifdef CONFIG_ARM_UNWIND
> const char *secstrs = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
> const Elf_Shdr *sechdrs_end = sechdrs + hdr->e_shnum;
> - struct mod_unwind_map maps[ARM_SEC_MAX];
> - int i;
> + struct unwind_table *unwind_list = &mod->arch.unwind_list;
>
> - memset(maps, 0, sizeof(maps));
> + INIT_LIST_HEAD(&unwind_list->mod_list);
> + mod->arch.init_table = NULL;
>
> for (s = sechdrs; s < sechdrs_end; s++) {
> const char *secname = secstrs + s->sh_name;
> + const char *txtname;
> + const Elf_Shdr *txt_sec;
>
> - if (!(s->sh_flags & SHF_ALLOC))
> + if (!strcmp(".ARM.exidx", secname))
> + txtname = ".text";
> + else
> + txtname = secname + strlen(".ARM.exidx");
> +
> + txt_sec = find_mod_section(hdr, sechdrs, txtname);
> +
> + if (!(s->sh_flags & SHF_ALLOC) ||
> + s->sh_type != ELF_SECTION_UNWIND)
> continue;
>
> - if (strcmp(".ARM.exidx.init.text", secname) == 0)
> - maps[ARM_SEC_INIT].unw_sec = s;
> - else if (strcmp(".ARM.exidx", secname) == 0)
> - maps[ARM_SEC_CORE].unw_sec = s;
> - else if (strcmp(".ARM.exidx.exit.text", secname) == 0)
> - maps[ARM_SEC_EXIT].unw_sec = s;
> - else if (strcmp(".ARM.exidx.text.unlikely", secname) == 0)
> - maps[ARM_SEC_UNLIKELY].unw_sec = s;
> - else if (strcmp(".ARM.exidx.text.hot", secname) == 0)
> - maps[ARM_SEC_HOT].unw_sec = s;
> - else if (strcmp(".init.text", secname) == 0)
> - maps[ARM_SEC_INIT].txt_sec = s;
> - else if (strcmp(".text", secname) == 0)
> - maps[ARM_SEC_CORE].txt_sec = s;
> - else if (strcmp(".exit.text", secname) == 0)
> - maps[ARM_SEC_EXIT].txt_sec = s;
> - else if (strcmp(".text.unlikely", secname) == 0)
> - maps[ARM_SEC_UNLIKELY].txt_sec = s;
> - else if (strcmp(".text.hot", secname) == 0)
> - maps[ARM_SEC_HOT].txt_sec = s;
> - }
> + if (txt_sec) {
> + struct unwind_table *table =
> + unwind_table_add(s->sh_addr,
> + s->sh_size,
> + txt_sec->sh_addr,
> + txt_sec->sh_size);
> +
> + list_add(&table->mod_list, &unwind_list->mod_list);
>
> - for (i = 0; i < ARM_SEC_MAX; i++)
> - if (maps[i].unw_sec && maps[i].txt_sec)
> - mod->arch.unwind[i] =
> - unwind_table_add(maps[i].unw_sec->sh_addr,
> - maps[i].unw_sec->sh_size,
> - maps[i].txt_sec->sh_addr,
> - maps[i].txt_sec->sh_size);
> + /* save init table for module_arch_freeing_init */
> + if (strcmp(".ARM.exidx.init.text", secname) == 0)
> + mod->arch.init_table = table;
> + }
> + }
> #endif
> #ifdef CONFIG_ARM_PATCH_PHYS_VIRT
> s = find_mod_section(hdr, sechdrs, ".pv_table");
> @@ -519,11 +514,13 @@ void
> module_arch_cleanup(struct module *mod)
> {
> #ifdef CONFIG_ARM_UNWIND
> - int i;
> + struct unwind_table *tmp;
> + struct unwind_table *n;
>
> - for (i = 0; i < ARM_SEC_MAX; i++) {
> - unwind_table_del(mod->arch.unwind[i]);
> - mod->arch.unwind[i] = NULL;
> + list_for_each_entry_safe(tmp, n,
> + &mod->arch.unwind_list.mod_list, mod_list) {
> + list_del(&tmp->mod_list);
> + unwind_table_del(tmp);
> }
> #endif
> }
> @@ -531,7 +528,12 @@ module_arch_cleanup(struct module *mod)
> void __weak module_arch_freeing_init(struct module *mod)
> {
> #ifdef CONFIG_ARM_UNWIND
> - unwind_table_del(mod->arch.unwind[ARM_SEC_INIT]);
> - mod->arch.unwind[ARM_SEC_INIT] = NULL;
> + struct unwind_table *init = mod->arch.init_table;
> +
> + if (init) {
> + mod->arch.init_table = NULL;
> + list_del(&init->mod_list);
> + unwind_table_del(init);
> + }
> #endif
> }
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [soc:omap1-multiplatform-5.18] BUILD SUCCESS 80fcf471cc8b7e205365c14d320343b7da4067f6
From: kernel test robot @ 2022-04-20 3:16 UTC (permalink / raw)
To: Arnd Bergmann; +Cc: arm, linux-arm-kernel
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git omap1-multiplatform-5.18
branch HEAD: 80fcf471cc8b7e205365c14d320343b7da4067f6 [TO BE REBASED] ARM: omap1: enable multiplatform
elapsed time: 723m
configs tested: 97
configs skipped: 3
The following configs have been built successfully.
More configs may be tested in the coming days.
gcc tested configs:
arm64 defconfig
arm64 allyesconfig
arm allmodconfig
arm defconfig
arm allyesconfig
i386 randconfig-c001
powerpc eiger_defconfig
nios2 10m50_defconfig
arm pxa910_defconfig
mips jazz_defconfig
arc tb10x_defconfig
arm gemini_defconfig
mips cobalt_defconfig
sh titan_defconfig
sh shx3_defconfig
sh sh7757lcr_defconfig
x86_64 randconfig-c001
arm randconfig-c002-20220419
ia64 allmodconfig
ia64 allyesconfig
ia64 defconfig
m68k allyesconfig
m68k allmodconfig
m68k defconfig
csky defconfig
nios2 allyesconfig
alpha defconfig
alpha allyesconfig
h8300 allyesconfig
xtensa allyesconfig
arc defconfig
sh allmodconfig
s390 defconfig
s390 allmodconfig
parisc defconfig
parisc64 defconfig
parisc allyesconfig
s390 allyesconfig
sparc defconfig
i386 allyesconfig
sparc allyesconfig
i386 defconfig
i386 debian-10.3-kselftests
i386 debian-10.3
nios2 defconfig
arc allyesconfig
mips allyesconfig
mips allmodconfig
powerpc allyesconfig
powerpc allnoconfig
powerpc allmodconfig
x86_64 randconfig-a013
x86_64 randconfig-a011
x86_64 randconfig-a015
i386 randconfig-a012
i386 randconfig-a014
i386 randconfig-a016
x86_64 randconfig-a006
x86_64 randconfig-a004
x86_64 randconfig-a002
arc randconfig-r043-20220419
riscv randconfig-r042-20220419
s390 randconfig-r044-20220419
riscv defconfig
riscv nommu_virt_defconfig
riscv rv32_defconfig
riscv nommu_k210_defconfig
riscv allnoconfig
riscv allmodconfig
riscv allyesconfig
x86_64 rhel-8.3-kselftests
um x86_64_defconfig
um i386_defconfig
x86_64 rhel-8.3-kunit
x86_64 rhel-8.3-func
x86_64 kexec
x86_64 defconfig
x86_64 allyesconfig
x86_64 rhel-8.3
clang tested configs:
x86_64 randconfig-c007
powerpc randconfig-c003-20220419
arm randconfig-c002-20220419
i386 randconfig-c001
riscv randconfig-c006-20220419
mips randconfig-c004-20220419
s390 randconfig-c005-20220419
mips lemote2f_defconfig
powerpc ppc64e_defconfig
i386 randconfig-a002
i386 randconfig-a006
i386 randconfig-a004
x86_64 randconfig-a012
x86_64 randconfig-a014
x86_64 randconfig-a016
x86_64 randconfig-a005
x86_64 randconfig-a003
x86_64 randconfig-a001
hexagon randconfig-r041-20220419
hexagon randconfig-r045-20220419
--
0-DAY CI Kernel Test Service
https://01.org/lkp
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* Re: [PATCH 3/5] dt-bindings: mediatek: add vdosys1 RDMA definition for mt8195
From: Rex-BC Chen @ 2022-04-20 3:15 UTC (permalink / raw)
To: Chun-Kuang Hu, Matthias Brugger
Cc: Rob Herring, krzysztof.kozlowski+dt@linaro.org, Philipp Zabel,
David Airlie, AngeloGioacchino Del Regno,
Jason-JH Lin (林睿祥),
Nancy Lin (林欣螢), DTML, linux-kernel,
DRI Development, moderated list:ARM/Mediatek SoC support,
Linux ARM, Project_Global_Chrome_Upstream_Group
In-Reply-To: <CAAOTY__u3q1YcNwGpyEUpRbThsg6U1-gYtaqtGgy2J4jMwSOUg@mail.gmail.com>
On Tue, 2022-04-19 at 23:51 +0800, Chun-Kuang Hu wrote:
> Matthias Brugger <matthias.bgg@gmail.com> 於 2022年4月19日 週二 下午10:57寫道：
> >
> >
> >
> > On 19/04/2022 05:32, Rex-BC Chen wrote:
> > > From: "Nancy.Lin" <nancy.lin@mediatek.com>
> > >
> > > Add vdosys1 RDMA definition.
> > >
> > > Signed-off-by: Nancy.Lin <nancy.lin@mediatek.com>
> > > Reviewed-by: AngeloGioacchino Del Regno <
> > > angelogioacchino.delregno@collabora.com>
> > > ---
> > > .../display/mediatek/mediatek,mdp-rdma.yaml | 86
> > > +++++++++++++++++++
> > > 1 file changed, 86 insertions(+)
> > > create mode 100644
> > > Documentation/devicetree/bindings/display/mediatek/mediatek,mdp-
> > > rdma.yaml
> > >
> > > diff --git
> > > a/Documentation/devicetree/bindings/display/mediatek/mediatek,mdp
> > > -rdma.yaml
> > > b/Documentation/devicetree/bindings/display/mediatek/mediatek,mdp
> > > -rdma.yaml
> > > new file mode 100644
> > > index 000000000000..6ab773569462
> > > --- /dev/null
> > > +++
> > > b/Documentation/devicetree/bindings/display/mediatek/mediatek,mdp
> > > -rdma.yaml
> > > @@ -0,0 +1,86 @@
> > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > > +%YAML 1.2
> > > +---
> > > +$id:
> > > https://urldefense.com/v3/__http://devicetree.org/schemas/arm/mediatek/mediatek,mdp-rdma.yaml*__;Iw!!CTRNKA9wMg0ARbw!2Ig4llRcam253qgvT99ty3TWC4Yo6D6Dy1DgFiNuA_fMhtu1lJHERS1f4pzOBELsqIl__FAiHl5bJCAJqNc7FAWGTw$
> > >
> > > +$schema:
> > > https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!2Ig4llRcam253qgvT99ty3TWC4Yo6D6Dy1DgFiNuA_fMhtu1lJHERS1f4pzOBELsqIl__FAiHl5bJCAJqNdU9sgsvg$
> > >
> > > +
> > > +title: MediaTek MDP RDMA
> > > +
> > > +maintainers:
> > > + - Matthias Brugger <matthias.bgg@gmail.com>
> >
> > I don't think I would be the correct person to maintain this. This
> > should be the
> > person that is maintaining the driver.
>
> Agree. This should be
>
> Chun-Kuang Hu <chunkuang.hu@kernel.org>
> Philipp Zabel <p.zabel@pengutronix.de>
>
> Regards,
> Chun-Kuang.
>
> >
> > Regards,
> > Matthias
> >
Hello Chun-Kuang and Matthias,
OK, I will update the list in next version.
BRs,
Rex
> > > +
> > > +description: |
> > > + The mediatek MDP RDMA stands for Read Direct Memory Access.
> > > + It provides real time data to the back-end panel driver, such
> > > as DSI,
> > > + DPI and DP_INTF.
> > > + It contains one line buffer to store the sufficient pixel
> > > data.
> > > + RDMA device node must be siblings to the central MMSYS_CONFIG
> > > node.
> > > + For a description of the MMSYS_CONFIG binding, see
> > > + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.
> > > yaml for details.
> > > +
> > > +properties:
> > > + compatible:
> > > + oneOf:
> > > + - items:
> > > + - const: mediatek,mt8195-vdo1-rdma
> > > +
> > > + reg:
> > > + maxItems: 1
> > > +
> > > + interrupts:
> > > + maxItems: 1
> > > +
> > > + power-domains:
> > > + description: A phandle and PM domain specifier as defined by
> > > bindings of
> > > + the power controller specified by phandle. See
> > > + Documentation/devicetree/bindings/power/power-domain.yaml
> > > for details.
> > > +
> > > + clocks:
> > > + items:
> > > + - description: RDMA Clock
> > > +
> > > + iommus:
> > > + description:
> > > + This property should point to the respective IOMMU block
> > > with master port as argument,
> > > + see
> > > Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for
> > > details.
> > > +
> > > + mediatek,gce-client-reg:
> > > + description:
> > > + The register of display function block to be set by gce.
> > > There are 4 arguments,
> > > + such as gce node, subsys id, offset and register size. The
> > > subsys id that is
> > > + mapping to the register of display function blocks is
> > > defined in the gce header
> > > + include/include/dt-bindings/gce/<chip>-gce.h of each
> > > chips.
> > > + $ref: /schemas/types.yaml#/definitions/phandle-array
> > > + maxItems: 1
> > > +
> > > +required:
> > > + - compatible
> > > + - reg
> > > + - power-domains
> > > + - clocks
> > > + - iommus
> > > +
> > > +additionalProperties: false
> > > +
> > > +examples:
> > > + - |
> > > + #include <dt-bindings/interrupt-controller/arm-gic.h>
> > > + #include <dt-bindings/clock/mt8195-clk.h>
> > > + #include <dt-bindings/power/mt8195-power.h>
> > > + #include <dt-bindings/gce/mt8195-gce.h>
> > > + #include <dt-bindings/memory/mt8195-memory-port.h>
> > > +
> > > + soc {
> > > + #address-cells = <2>;
> > > + #size-cells = <2>;
> > > +
> > > + vdo1_rdma0: mdp-rdma@1c104000 {
> > > + compatible = "mediatek,mt8195-vdo1-rdma";
> > > + reg = <0 0x1c104000 0 0x1000>;
> > > + interrupts = <GIC_SPI 495 IRQ_TYPE_LEVEL_HIGH 0>;
> > > + clocks = <&vdosys1 CLK_VDO1_MDP_RDMA0>;
> > > + power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>;
> > > + iommus = <&iommu_vdo M4U_PORT_L2_MDP_RDMA0>;
> > > + mediatek,gce-client-reg = <&gce0 SUBSYS_1c10XXXX
> > > 0x4000 0x1000>;
> > > + };
> > > + };
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH v3 8/9] x86: Add Kconfig/Makefile to build hardware prefetch control driver
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This adds Kconfig/Makefile to build hardware prefetch control driver
for x86 support. This also adds a MAINTAINERS entry.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
MAINTAINERS | 1 +
arch/x86/Kconfig | 6 ++++++
arch/x86/kernel/cpu/Makefile | 2 ++
3 files changed, 9 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index b359dcc38be3..4de219599e52 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8618,6 +8618,7 @@ K: (devm_)?hwmon_device_(un)?register(|_with_groups|_with_info)
HARDWARE PREFETCH CONTROL DRIVERS
M: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
S: Maintained
+F: arch/x86/kernel/cpu/x86-pfctl.c
F: drivers/base/pfctl.c
F: drivers/soc/fujitsu/a64fx-pfctl.c
F: include/linux/pfctl.h
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index b0142e01002e..1ef47c29c338 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1359,6 +1359,12 @@ config X86_CPUID
with major 203 and minors 0 to 31 for /dev/cpu/0/cpuid to
/dev/cpu/31/cpuid.
+config X86_HWPF_CONTROL
+ tristate "x86 Hardware Prefetch Control support"
+ depends on HWPF_CONTROL
+ help
+ This adds Hardware Prefetch driver control support for X86.
+
choice
prompt "High Memory Support"
default HIGHMEM4G
diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
index 9661e3e802be..1aa13dad17a3 100644
--- a/arch/x86/kernel/cpu/Makefile
+++ b/arch/x86/kernel/cpu/Makefile
@@ -56,6 +56,8 @@ obj-$(CONFIG_X86_LOCAL_APIC) += perfctr-watchdog.o
obj-$(CONFIG_HYPERVISOR_GUEST) += vmware.o hypervisor.o mshyperv.o
obj-$(CONFIG_ACRN_GUEST) += acrn.o
+obj-$(CONFIG_X86_HWPF_CONTROL) += x86-pfctl.o
+
ifdef CONFIG_X86_FEATURE_NAMES
quiet_cmd_mkcapflags = MKCAP $@
cmd_mkcapflags = $(CONFIG_SHELL) $(srctree)/$(src)/mkcapflags.sh $@ $^
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 9/9] docs: ABI: Add sysfs documentation interface of hardware prefetch control driver
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This describes the sysfs interface implemented by the hardware prefetch
control driver.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
.../ABI/testing/sysfs-devices-system-cpu | 98 +++++++++++++++++++
1 file changed, 98 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 2ad01cad7f1c..0da4c1bac51e 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -688,3 +688,101 @@ Description:
(RO) the list of CPUs that are isolated and don't
participate in load balancing. These CPUs are set by
boot parameter "isolcpus=".
+
+What: /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/hardware_prefetcher_enable
+ /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/ip_prefetcher_enable
+ /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/adjacent_cache_line_prefetcher_enable
+Date: March 2022
+Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org>
+Description: Parameters for some Intel CPU's hardware prefetch control
+
+ This sysfs interface provides Hardware Prefetch control
+ attribute for some Intel processors. Attributes are only
+ present if the particular cache implements the relevant
+ prefetcher controls.
+
+ *_prefetcher_enable:
+ (RW) control this prefetcher's enablement state.
+ Read returns current status:
+ 0: this prefetcher is disabled
+ 1: this prefetcher is enabled
+
+ - Attribute mapping
+
+ Some Intel processors have MSR 0x1a4. This register has several
+ specifications depending on the model. This interface provides
+ a one-to-one attribute file to control all the tunable
+ parameters the CPU provides of the following.
+
+ - "* Hardware Prefetcher Disable (R/W)"
+ corresponds to the "hardware_prefetcher_enable"
+
+ - "* Adjacent Cache Line Prefetcher Disable (R/W)"
+ corresponds to the "adjacent_cache_line_prefetcher_enable"
+
+ - "* IP Prefetcher Disable (R/W)"
+ corresponds to the "ip_prefetcher_enable"
+
+What: /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect_prefetcher_enable
+ /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect_prefetcher_strength
+ /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect_prefetcher_strength_available
+ /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect_prefetcher_dist
+Date: March 2022
+Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org>
+Description: Parameters for A64FX's hardware prefetch control
+
+ This sysfs interface provides Hardware Prefetch control
+ attribute for the processor A64FX. Attributes are only
+ present if the particular cache implements the relevant
+ prefetcher controls.
+
+ stream_detect_prefetcher_enable:
+ (RW) control the prefetcher's enablement state.
+ Read returns current status:
+ 0: this prefetcher is disabled
+ 1: this prefetcher is enabled
+
+ stream_detect_prefetcher_strength:
+ (RW) control the prefetcher operation's strongness state.
+ Read returns current status:
+ weak: prefetch operation is weak
+ strong: prefetch operation is strong
+
+ Strong prefetch operation is surely executed, if there is
+ no corresponding data in cache.
+ Weak prefetch operation allows the hardware not to execute
+ operation depending on hardware state.
+
+
+ stream_detect_prefetcher_strength_available:
+ (RO) displays a space separated list of available strongness
+ state.
+
+ stream_detect_prefetcher_dist:
+ (RW) control the prefetcher distance value.
+ Read return current prefetcher distance value in bytes
+ or the string "auto".
+
+ Write either a value in byte or the string "auto" to this
+ parameter. If you write a value less than multiples of a
+ specific value, it is rounded up.
+
+ The string "auto" have a special meaning. This means that
+ instead of setting dist to a user-specified value, it
+ operates using hardware-specific values.
+
+ - Attribute mapping
+
+ The processor A64FX has register IMP_PF_STREAM_DETECT_CTRL_EL0
+ for Hardware Prefetch Control. This attribute maps each
+ specification to the following.
+
+ - "L*PF_DIS": enablement of hardware prefetcher
+ corresponds to the "stream_detect_prefetcher_enable"
+
+ - "L*W": strongness of hardware prefetcher
+ corresponds to "stream_detect_prefetcher_strength"
+ and "stream_detect_prefetcher_strength_available"
+
+ - "L*_DIST": distance of hardware prefetcher
+ corresponds to the "stream_detect_prefetcher_dist"
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 6/9] x86: resctrl: pseudo_lock: Fix to restore to original value when re-enabling hardware prefetch register
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
The current pseudo_lock.c code overwrittes the value of the
MSR_MISC_FEATURE_CONTROL to 0 even if the original value is not 0.
Therefore, modify it to save and restore the original values.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
index db813f819ad6..2d713c20f55f 100644
--- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
+++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
@@ -420,6 +420,7 @@ static int pseudo_lock_fn(void *_rdtgrp)
struct pseudo_lock_region *plr = rdtgrp->plr;
u32 rmid_p, closid_p;
unsigned long i;
+ u64 saved_msr;
#ifdef CONFIG_KASAN
/*
* The registers used for local register variables are also used
@@ -463,6 +464,7 @@ static int pseudo_lock_fn(void *_rdtgrp)
* the buffer and evict pseudo-locked memory read earlier from the
* cache.
*/
+ saved_msr = __rdmsr(MSR_MISC_FEATURE_CONTROL);
__wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
closid_p = this_cpu_read(pqr_state.cur_closid);
rmid_p = this_cpu_read(pqr_state.cur_rmid);
@@ -514,7 +516,7 @@ static int pseudo_lock_fn(void *_rdtgrp)
__wrmsr(IA32_PQR_ASSOC, rmid_p, closid_p);
/* Re-enable the hardware prefetcher(s) */
- wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
+ wrmsrl(MSR_MISC_FEATURE_CONTROL, saved_msr);
local_irq_enable();
plr->thread_done = 1;
@@ -873,12 +875,14 @@ static int measure_cycles_lat_fn(void *_plr)
struct pseudo_lock_region *plr = _plr;
unsigned long i;
u64 start, end;
+ u32 saved_low, saved_high;
void *mem_r;
local_irq_disable();
/*
* Disable hardware prefetchers.
*/
+ rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
mem_r = READ_ONCE(plr->kmem);
/*
@@ -895,7 +899,7 @@ static int measure_cycles_lat_fn(void *_plr)
end = rdtsc_ordered();
trace_pseudo_lock_mem_latency((u32)(end - start));
}
- wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
+ wrmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
local_irq_enable();
plr->thread_done = 1;
wake_up_interruptible(&plr->lock_thread_wq);
@@ -945,6 +949,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
unsigned long i;
void *mem_r;
u64 tmp;
+ u32 saved_low, saved_high;
miss_event = perf_event_create_kernel_counter(miss_attr, plr->cpu,
NULL, NULL, NULL);
@@ -973,6 +978,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
/*
* Disable hardware prefetchers.
*/
+ rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
/* Initialize rest of local variables */
@@ -1031,7 +1037,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
*/
rmb();
/* Re-enable hardware prefetchers */
- wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
+ wrmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
local_irq_enable();
out_hit:
perf_event_release_kernel(hit_event);
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 7/9] x86: Add hardware prefetch control support for x86
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This adds module init/exit code, and creates sysfs attribute file
"hardware_prefetcher_enable", "ip_prefetcher_enable" and
"adjacent_cache_line_prefetcher_enable" for x86. This driver works
only if the model is INTEL_FAM6_BROADWELL_X at this point.
If you would like to support a new model with the same register
specifications as INTEL_FAM6_BROADWELL_X, it is possible to add the
model settings to array of broadwell_cpu_ids[].
The details of the registers to be read and written in this patch are
described below:
"https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html"
Volume 4
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
arch/x86/kernel/cpu/x86-pfctl.c | 347 ++++++++++++++++++++++++++++++++
1 file changed, 347 insertions(+)
create mode 100644 arch/x86/kernel/cpu/x86-pfctl.c
diff --git a/arch/x86/kernel/cpu/x86-pfctl.c b/arch/x86/kernel/cpu/x86-pfctl.c
new file mode 100644
index 000000000000..153b7a46ba80
--- /dev/null
+++ b/arch/x86/kernel/cpu/x86-pfctl.c
@@ -0,0 +1,347 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 FUJITSU LIMITED
+ *
+ * x86 Hardware Prefetch Control support
+ */
+
+#include <linux/bitfield.h>
+#include <linux/cacheinfo.h>
+#include <linux/pfctl.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <asm/cpu_device_id.h>
+#include <asm/intel-family.h>
+#include <asm/msr.h>
+
+struct pfctl_driver x86_pfctl_driver;
+
+/**************************************
+ * Intle BROADWELL support
+ **************************************/
+
+/*
+ * The register specification for each bits of Intel BROADWELL is as
+ * follow:
+ *
+ * [0] L2 Hardware Prefetcher Disable (R/W)
+ * [1] L2 Adjacent Cache Line Prefetcher Disable (R/W)
+ * [2] DCU Hardware Prefetcher Disable (R/W)
+ * [3] DCU IP Prefetcher Disable (R/W)
+ * [63:4] Reserved
+ *
+ * See "Intel 64 and IA-32 Architectures Software Developer's Manual"
+ * (https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html)
+ * for register specification details.
+ */
+#define BROADWELL_L2_HWPF_FIELD BIT_ULL(0)
+#define BROADWELL_L2_ACLPF_FIELD BIT_ULL(1)
+#define BROADWELL_DCU_HWPF_FIELD BIT_ULL(2)
+#define BROADWELL_DCU_IPPF_FIELD BIT_ULL(3)
+
+struct broadwell_read_info {
+ enum pfctl_attr pattr;
+ u64 val;
+ unsigned int level;
+ int ret;
+};
+
+struct broadwell_write_info {
+ enum pfctl_attr pattr;
+ u64 val;
+ unsigned int level;
+ int ret;
+};
+
+static int broadwell_get_hwpf_enable(u64 reg, unsigned int level)
+{
+ u64 val;
+
+ switch (level) {
+ case 1:
+ val = FIELD_GET(BROADWELL_DCU_HWPF_FIELD, reg);
+ break;
+ case 2:
+ val = FIELD_GET(BROADWELL_L2_HWPF_FIELD, reg);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (val == 0)
+ return PFCTL_ENABLE_VAL;
+ else if (val == 1)
+ return PFCTL_DISABLE_VAL;
+ else
+ return -EINVAL;
+}
+
+static int broadwell_modify_hwpf_enable(u64 *reg, unsigned int level, u64 val)
+{
+ if (val == PFCTL_ENABLE_VAL)
+ val = 0;
+ else
+ val = 1;
+
+ switch (level) {
+ case 1:
+ *reg &= ~BROADWELL_DCU_HWPF_FIELD;
+ *reg |= FIELD_PREP(BROADWELL_DCU_HWPF_FIELD, val);
+ break;
+ case 2:
+ *reg &= ~BROADWELL_L2_HWPF_FIELD;
+ *reg |= FIELD_PREP(BROADWELL_L2_HWPF_FIELD, val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int broadwell_get_ippf_enable(u64 reg, unsigned int level)
+{
+ u64 val;
+
+ switch (level) {
+ case 1:
+ val = FIELD_GET(BROADWELL_DCU_IPPF_FIELD, reg);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (val == 0)
+ return PFCTL_ENABLE_VAL;
+ else if (val == 1)
+ return PFCTL_DISABLE_VAL;
+ else
+ return -EINVAL;
+}
+
+static int broadwell_modify_ippf_enable(u64 *reg, unsigned int level, u64 val)
+{
+ if (val == PFCTL_ENABLE_VAL)
+ val = 0;
+ else
+ val = 1;
+
+ switch (level) {
+ case 1:
+ *reg &= ~BROADWELL_DCU_IPPF_FIELD;
+ *reg |= FIELD_PREP(BROADWELL_DCU_IPPF_FIELD, val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int broadwell_get_aclpf_enable(u64 reg, unsigned int level)
+{
+ u64 val;
+
+ switch (level) {
+ case 2:
+ val = FIELD_GET(BROADWELL_L2_ACLPF_FIELD, reg);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (val == 0)
+ return PFCTL_ENABLE_VAL;
+ else if (val == 1)
+ return PFCTL_DISABLE_VAL;
+ else
+ return -EINVAL;
+}
+
+static int broadwell_modify_aclpf_enable(u64 *reg, unsigned int level, u64 val)
+{
+ if (val == PFCTL_ENABLE_VAL)
+ val = 0;
+ else
+ val = 1;
+
+ switch (level) {
+ case 2:
+ *reg &= ~BROADWELL_L2_ACLPF_FIELD;
+ *reg |= FIELD_PREP(BROADWELL_L2_ACLPF_FIELD, val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int broadwell_get_pfctl_params(enum pfctl_attr pattr, u64 reg,
+ unsigned int level, u64 *val)
+{
+ int ret;
+
+ switch (pattr) {
+ case HWPF_ENABLE:
+ ret = broadwell_get_hwpf_enable(reg, level);
+ break;
+ case IPPF_ENABLE:
+ ret = broadwell_get_ippf_enable(reg, level);
+ break;
+ case ACLPF_ENABLE:
+ ret = broadwell_get_aclpf_enable(reg, level);
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ if (ret < 0)
+ return ret;
+ *val = ret;
+
+ return 0;
+}
+
+static int broadwell_modify_pfreg(enum pfctl_attr pattr, u64 *reg,
+ unsigned int level, u64 val)
+{
+ int ret;
+
+ switch (pattr) {
+ case HWPF_ENABLE:
+ ret = broadwell_modify_hwpf_enable(reg, level, val);
+ break;
+ case IPPF_ENABLE:
+ ret = broadwell_modify_ippf_enable(reg, level, val);
+ break;
+ case ACLPF_ENABLE:
+ ret = broadwell_modify_aclpf_enable(reg, level, val);
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static void _broadwell_read_pfreg(void *info)
+{
+ u64 reg;
+ struct broadwell_read_info *rinfo = info;
+
+ rdmsrl(MSR_MISC_FEATURE_CONTROL, reg);
+
+ rinfo->ret = broadwell_get_pfctl_params(rinfo->pattr, reg, rinfo->level,
+ &rinfo->val);
+ if (rinfo->ret < 0)
+ return;
+}
+
+static int broadwell_read_pfreg(enum pfctl_attr pattr, unsigned int cpu,
+ unsigned int level, u64 *val)
+{
+ struct broadwell_read_info info = {
+ .level = level,
+ .pattr = pattr,
+ };
+
+ smp_call_function_single(cpu, _broadwell_read_pfreg, &info, true);
+ if (info.ret < 0)
+ return info.ret;
+
+ *val = info.val;
+ return 0;
+}
+
+static void _broadwell_write_pfreg(void *info)
+{
+ u64 reg;
+ struct broadwell_write_info *winfo = info;
+
+ rdmsrl(MSR_MISC_FEATURE_CONTROL, reg);
+
+ winfo->ret = broadwell_modify_pfreg(winfo->pattr, ®, winfo->level,
+ winfo->val);
+ if (winfo->ret < 0)
+ return;
+
+ wrmsrl(MSR_MISC_FEATURE_CONTROL, reg);
+}
+
+static int broadwell_write_pfreg(enum pfctl_attr pattr, unsigned int cpu,
+ unsigned int level, u64 val)
+{
+ struct broadwell_write_info info = {
+ .level = level,
+ .pattr = pattr,
+ .val = val,
+ };
+
+ smp_call_function_single(cpu, _broadwell_write_pfreg, &info, true);
+ return info.ret;
+}
+
+/*
+ * In addition to BROADWELL_X, NEHALEM and others have same register
+ * specifications as those represented by BROADWELL_XXX_FIELD.
+ * If you want to add support for these processor, add the new target model
+ * here.
+ */
+static const struct x86_cpu_id broadwell_cpu_ids[] = {
+ X86_MATCH_INTEL_FAM6_MODEL(BROADWELL_X, NULL),
+ {}
+};
+
+/***** end of Intel BROADWELL support *****/
+
+/*
+ * This driver returns a negative value if it does not support the Hardware
+ * Prefetch Control or if it is running on a VM guest.
+ */
+static int __init setup_pfctl_driver_params(void)
+{
+ if (boot_cpu_has(X86_FEATURE_HYPERVISOR))
+ return -EINVAL;
+
+ if (x86_match_cpu(broadwell_cpu_ids)) {
+ x86_pfctl_driver.supported_l1d_prefetcher = HWPF|IPPF;
+ x86_pfctl_driver.supported_l2_prefetcher = HWPF|ACLPF;
+ x86_pfctl_driver.read_pfreg = broadwell_read_pfreg;
+ x86_pfctl_driver.write_pfreg = broadwell_write_pfreg;
+ } else {
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static int __init x86_pfctl_init(void)
+{
+ int ret;
+
+ ret = setup_pfctl_driver_params();
+ if (ret < 0)
+ return ret;
+
+ ret = pfctl_register_driver(&x86_pfctl_driver);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static void __exit x86_pfctl_exit(void)
+{
+ pfctl_unregister_driver(&x86_pfctl_driver);
+}
+
+late_initcall(x86_pfctl_init);
+module_exit(x86_pfctl_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("FUJITSU LIMITED");
+MODULE_DESCRIPTION("x86 Hardware Prefetch Control Driver");
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 3/9] soc: fujitsu: Add hardware prefetch control support for A64FX
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This adds module init/exit code, and creates sysfs attribute files for
"stream_detect_prefetcher_enable", "stream_detect_prefetcher_strong"
and "stream_detect_prefetcher_dist". This driver works only if part
number is FUJITSU_CPU_PART_A64FX at this point. The details of the
registers to be read and written in this patch are described below.
"https://github.com/fujitsu/A64FX/tree/master/doc/"
A64FX_Specification_HPC_Extension_v1_EN.pdf
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
drivers/soc/fujitsu/a64fx-pfctl.c | 356 ++++++++++++++++++++++++++++++
1 file changed, 356 insertions(+)
create mode 100644 drivers/soc/fujitsu/a64fx-pfctl.c
diff --git a/drivers/soc/fujitsu/a64fx-pfctl.c b/drivers/soc/fujitsu/a64fx-pfctl.c
new file mode 100644
index 000000000000..0e072592be73
--- /dev/null
+++ b/drivers/soc/fujitsu/a64fx-pfctl.c
@@ -0,0 +1,356 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 FUJITSU LIMITED
+ *
+ * A64FX Hardware Prefetch Control support
+ */
+
+#include <asm/cputype.h>
+#include <linux/bitfield.h>
+#include <linux/cacheinfo.h>
+#include <linux/module.h>
+#include <linux/pfctl.h>
+#include <linux/parser.h>
+
+struct pfctl_driver a64fx_pfctl_driver;
+
+/*
+ * Constants for these add the "A64FX_SDPF" prefix to the name described in
+ * section "1.3.4.2. IMP_PF_STREAM_DETECT_CTRL_EL0" of "A64FX specification".
+ * (https://github.com/fujitsu/A64FX/tree/master/doc/A64FX_Specification_HPC_Extension_v1_EN.pdf")
+ * See this document for register specification details.
+ */
+#define A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0 sys_reg(3, 3, 11, 4, 0)
+#define A64FX_SDPF_V BIT_ULL(63)
+#define A64FX_SDPF_L1PF_DIS BIT_ULL(59)
+#define A64FX_SDPF_L2PF_DIS BIT_ULL(58)
+#define A64FX_SDPF_L1W BIT_ULL(55)
+#define A64FX_SDPF_L2W BIT_ULL(54)
+#define A64FX_SDPF_L1_DIST GENMASK_ULL(27, 24)
+#define A64FX_SDPF_L2_DIST GENMASK_ULL(19, 16)
+
+#define A64FX_SDPF_MIN_DIST_L1 256
+#define A64FX_SDPF_MIN_DIST_L2 1024
+
+struct a64fx_read_info {
+ enum pfctl_attr pattr;
+ u64 val;
+ unsigned int level;
+ int ret;
+};
+
+struct a64fx_write_info {
+ enum pfctl_attr pattr;
+ u64 val;
+ unsigned int level;
+ int ret;
+};
+
+static int a64fx_get_sdpf_enable(u64 reg, unsigned int level)
+{
+ u64 val;
+
+ switch (level) {
+ case 1:
+ val = FIELD_GET(A64FX_SDPF_L1PF_DIS, reg);
+ break;
+ case 2:
+ val = FIELD_GET(A64FX_SDPF_L2PF_DIS, reg);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (val == 0)
+ return PFCTL_ENABLE_VAL;
+ else if (val == 1)
+ return PFCTL_DISABLE_VAL;
+ else
+ return -EINVAL;
+}
+
+static int a64fx_modify_sdpf_enable(u64 *reg, unsigned int level, u64 val)
+{
+ if (val == PFCTL_ENABLE_VAL)
+ val = 0;
+ else
+ val = 1;
+
+ switch (level) {
+ case 1:
+ *reg &= ~A64FX_SDPF_L1PF_DIS;
+ *reg |= FIELD_PREP(A64FX_SDPF_L1PF_DIS, val);
+ break;
+ case 2:
+ *reg &= ~A64FX_SDPF_L2PF_DIS;
+ *reg |= FIELD_PREP(A64FX_SDPF_L2PF_DIS, val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int a64fx_get_sdpf_strength(u64 reg, unsigned int level)
+{
+ u64 val;
+
+ switch (level) {
+ case 1:
+ val = FIELD_GET(A64FX_SDPF_L1W, reg);
+ break;
+ case 2:
+ val = FIELD_GET(A64FX_SDPF_L2W, reg);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (val == 0)
+ return PFCTL_STRONG_VAL;
+ else if (val == 1)
+ return PFCTL_WEAK_VAL;
+ else
+ return -EINVAL;
+}
+
+static int a64fx_modify_sdpf_strength(u64 *reg, unsigned int level, u64 val)
+{
+ if (val == PFCTL_STRONG_VAL)
+ val = 0;
+ else
+ val = 1;
+
+ switch (level) {
+ case 1:
+ *reg &= ~A64FX_SDPF_L1W;
+ *reg |= FIELD_PREP(A64FX_SDPF_L1W, val);
+ break;
+ case 2:
+ *reg &= ~A64FX_SDPF_L2W;
+ *reg |= FIELD_PREP(A64FX_SDPF_L2W, val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int a64fx_get_sdpf_dist(u64 reg, unsigned int level)
+{
+ switch (level) {
+ case 1:
+ return FIELD_GET(A64FX_SDPF_L1_DIST, reg) *
+ A64FX_SDPF_MIN_DIST_L1;
+ case 2:
+ return FIELD_GET(A64FX_SDPF_L2_DIST, reg) *
+ A64FX_SDPF_MIN_DIST_L2;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int a64fx_modify_sdpf_dist(u64 *reg, unsigned int level, u64 val)
+{
+ switch (level) {
+ case 1:
+ val = roundup(val, A64FX_SDPF_MIN_DIST_L1) /
+ A64FX_SDPF_MIN_DIST_L1;
+ if (!FIELD_FIT(A64FX_SDPF_L1_DIST, val))
+ return -EINVAL;
+ *reg &= ~A64FX_SDPF_L1_DIST;
+ *reg |= FIELD_PREP(A64FX_SDPF_L1_DIST, val);
+ break;
+ case 2:
+ val = roundup(val, A64FX_SDPF_MIN_DIST_L2) /
+ A64FX_SDPF_MIN_DIST_L2;
+ if (!FIELD_FIT(A64FX_SDPF_L2_DIST, val))
+ return -EINVAL;
+ *reg &= ~A64FX_SDPF_L2_DIST;
+ *reg |= FIELD_PREP(A64FX_SDPF_L2_DIST, val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void a64fx_enable_sdpf_verify(u64 *reg)
+{
+ *reg |= FIELD_PREP(A64FX_SDPF_V, 1);
+}
+
+static int a64fx_get_sdpf_params(enum pfctl_attr pattr, u64 reg,
+ unsigned int level, u64 *val)
+{
+ int ret;
+
+ switch (pattr) {
+ case SDPF_ENABLE:
+ ret = a64fx_get_sdpf_enable(reg, level);
+ break;
+ case SDPF_STRENGTH:
+ ret = a64fx_get_sdpf_strength(reg, level);
+ break;
+ case SDPF_DIST:
+ ret = a64fx_get_sdpf_dist(reg, level);
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ if (ret < 0)
+ return ret;
+ *val = ret;
+
+ return 0;
+}
+
+static int a64fx_modify_pfreg_val(enum pfctl_attr pattr, u64 *reg,
+ unsigned int level, u64 val)
+{
+ int ret;
+
+ switch (pattr) {
+ case SDPF_ENABLE:
+ ret = a64fx_modify_sdpf_enable(reg, level, val);
+ break;
+ case SDPF_STRENGTH:
+ ret = a64fx_modify_sdpf_strength(reg, level, val);
+ break;
+ case SDPF_DIST:
+ ret = a64fx_modify_sdpf_dist(reg, level, val);
+ break;
+ default:
+ return -ENOENT;
+ }
+
+ if (ret < 0)
+ return ret;
+
+ a64fx_enable_sdpf_verify(reg);
+
+ return 0;
+}
+
+static void _a64fx_read_pfreg(void *info)
+{
+ u64 reg;
+ struct a64fx_read_info *rinfo = info;
+
+ reg = read_sysreg_s(A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0);
+
+ rinfo->ret = a64fx_get_sdpf_params(rinfo->pattr, reg, rinfo->level,
+ &rinfo->val);
+}
+
+static int a64fx_read_pfreg(enum pfctl_attr pattr, unsigned int cpu,
+ unsigned int level, u64 *val)
+{
+ struct a64fx_read_info info = {
+ .level = level,
+ .pattr = pattr,
+ };
+
+ smp_call_function_single(cpu, _a64fx_read_pfreg, &info, true);
+ if (info.ret < 0)
+ return info.ret;
+
+ *val = info.val;
+ return 0;
+}
+
+static void _a64fx_write_pfreg(void *info)
+{
+ u64 reg;
+ struct a64fx_write_info *winfo = info;
+
+ reg = read_sysreg_s(A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0);
+
+ winfo->ret = a64fx_modify_pfreg_val(winfo->pattr, ®, winfo->level,
+ winfo->val);
+ if (winfo->ret < 0)
+ return;
+
+ write_sysreg_s(reg, A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0);
+
+ winfo->ret = 0;
+}
+
+static int a64fx_write_pfreg(enum pfctl_attr pattr, unsigned int cpu,
+ unsigned int level, u64 val)
+{
+ struct a64fx_write_info info = {
+ .level = level,
+ .pattr = pattr,
+ .val = val,
+ };
+
+ smp_call_function_single(cpu, _a64fx_write_pfreg, &info, true);
+ return info.ret;
+}
+
+/*
+ * This driver returns a negative value if it does not support the Hardware
+ * Prefetch Control or if it is running on a VM guest.
+ */
+static int __init setup_pfctl_driver_params(void)
+{
+ unsigned long implementor = read_cpuid_implementor();
+ unsigned long part_number = read_cpuid_part_number();
+
+ if (!is_kernel_in_hyp_mode())
+ return -EINVAL;
+
+ if (implementor != ARM_CPU_IMP_FUJITSU)
+ return -ENODEV;
+
+ switch (part_number) {
+ case FUJITSU_CPU_PART_A64FX:
+ /* A64FX register requires EL2 access */
+ if (!has_vhe())
+ return -EINVAL;
+
+ a64fx_pfctl_driver.supported_l1d_prefetcher = SDPF;
+ a64fx_pfctl_driver.supported_l2_prefetcher = SDPF;
+ a64fx_pfctl_driver.supported_sdpf_strength =
+ STRENGTH_STRONG|STRENGTH_WEAK;
+ a64fx_pfctl_driver.read_pfreg = a64fx_read_pfreg;
+ a64fx_pfctl_driver.write_pfreg = a64fx_write_pfreg;
+ break;
+ default:
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static int __init a64fx_pfctl_init(void)
+{
+ int ret;
+
+ ret = setup_pfctl_driver_params();
+ if (ret < 0)
+ return ret;
+
+ ret = pfctl_register_driver(&a64fx_pfctl_driver);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static void __exit a64fx_pfctl_exit(void)
+{
+ pfctl_unregister_driver(&a64fx_pfctl_driver);
+}
+
+late_initcall(a64fx_pfctl_init);
+module_exit(a64fx_pfctl_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("FUJITSU LIMITED");
+MODULE_DESCRIPTION("A64FX Hardware Prefetch Control Driver");
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 4/9] soc: fujitsu: Add Kconfig/Makefile to build hardware prefetch control driver
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This adds Kconfig/Makefile to build hardware prefetch control driver
for A64FX support. This also adds a MAINTAINERS entry.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
MAINTAINERS | 1 +
drivers/soc/Kconfig | 1 +
drivers/soc/Makefile | 1 +
drivers/soc/fujitsu/Kconfig | 11 +++++++++++
drivers/soc/fujitsu/Makefile | 2 ++
5 files changed, 16 insertions(+)
create mode 100644 drivers/soc/fujitsu/Kconfig
create mode 100644 drivers/soc/fujitsu/Makefile
diff --git a/MAINTAINERS b/MAINTAINERS
index f6640dc053c0..b359dcc38be3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8619,6 +8619,7 @@ HARDWARE PREFETCH CONTROL DRIVERS
M: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
S: Maintained
F: drivers/base/pfctl.c
+F: drivers/soc/fujitsu/a64fx-pfctl.c
F: include/linux/pfctl.h
HARDWARE RANDOM NUMBER GENERATOR CORE
diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index c5aae42673d3..d87754799d90 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -9,6 +9,7 @@ source "drivers/soc/atmel/Kconfig"
source "drivers/soc/bcm/Kconfig"
source "drivers/soc/canaan/Kconfig"
source "drivers/soc/fsl/Kconfig"
+source "drivers/soc/fujitsu/Kconfig"
source "drivers/soc/imx/Kconfig"
source "drivers/soc/ixp4xx/Kconfig"
source "drivers/soc/litex/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index 904eec2a7871..6c8ff1792cda 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_SOC_CANAAN) += canaan/
obj-$(CONFIG_ARCH_DOVE) += dove/
obj-$(CONFIG_MACH_DOVE) += dove/
obj-y += fsl/
+obj-y += fujitsu/
obj-$(CONFIG_ARCH_GEMINI) += gemini/
obj-y += imx/
obj-y += ixp4xx/
diff --git a/drivers/soc/fujitsu/Kconfig b/drivers/soc/fujitsu/Kconfig
new file mode 100644
index 000000000000..d9db05d5055d
--- /dev/null
+++ b/drivers/soc/fujitsu/Kconfig
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+menu "Fujitsu SoC drivers"
+
+config A64FX_HWPF_CONTROL
+ tristate "A64FX Hardware Prefetch Control driver"
+ depends on ARM64 || HWPF_CONTROL
+ help
+ This adds Hardware Prefetch driver control support for A64FX.
+
+endmenu
diff --git a/drivers/soc/fujitsu/Makefile b/drivers/soc/fujitsu/Makefile
new file mode 100644
index 000000000000..35e284a548bb
--- /dev/null
+++ b/drivers/soc/fujitsu/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_A64FX_HWPF_CONTROL) += a64fx-pfctl.o
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH] crypto: sun8i-ce: using pm_runtime_resume_and_get instead of pm_runtime_get_sync
From: cgel.zte @ 2022-04-20 3:02 UTC (permalink / raw)
To: clabbe.montjoie
Cc: herbert, davem, linux-crypto, linux-arm-kernel, linux-sunxi,
linux-kernel, Minghao Chi, Zeal Robot
From: Minghao Chi <chi.minghao@zte.com.cn>
Using pm_runtime_resume_and_get() to replace pm_runtime_get_sync and
pm_runtime_put_noidle. This change is just to simplify the code, no
actual functional changes.
Reported-by: Zeal Robot <zealci@zte.com.cn>
Signed-off-by: Minghao Chi <chi.minghao@zte.com.cn>
---
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c
index b3a9bbfb8831..b3cc43ea6c8a 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c
@@ -108,11 +108,9 @@ int sun8i_ce_prng_generate(struct crypto_rng *tfm, const u8 *src,
goto err_dst;
}
- err = pm_runtime_get_sync(ce->dev);
- if (err < 0) {
- pm_runtime_put_noidle(ce->dev);
+ err = pm_runtime_resume_and_get(ce->dev);
+ if (err < 0)
goto err_pm;
- }
mutex_lock(&ce->rnglock);
chan = &ce->chanlist[flow];
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 5/9] arm64: Create cache sysfs directory without ACPI PPTT for hardware prefetch control
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This patch create a cache sysfs directory without ACPI PPTT if the
CONFIG_HWPF_CONTROL is true. This patch use only the level/type
information obtained from CLIDR_EL1, and don't use CCSIDR information.
Hardware prefetch control driver need cache sysfs directory and cache
level/type information. In ARM processor, these information can be
obtained from the register even without PPTT. Therefore, we set the
cpu_map_populated to true to create cache sysfs directory if the
machine doesn't have PPTT.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
arch/arm64/kernel/cacheinfo.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/arch/arm64/kernel/cacheinfo.c b/arch/arm64/kernel/cacheinfo.c
index 587543c6c51c..039ec32d0b3d 100644
--- a/arch/arm64/kernel/cacheinfo.c
+++ b/arch/arm64/kernel/cacheinfo.c
@@ -43,6 +43,21 @@ static void ci_leaf_init(struct cacheinfo *this_leaf,
this_leaf->type = type;
}
+#if defined(CONFIG_HWPF_CONTROL)
+static bool acpi_has_pptt(void)
+{
+ struct acpi_table_header *table;
+ acpi_status status;
+
+ status = acpi_get_table(ACPI_SIG_PPTT, 0, &table);
+ if (ACPI_FAILURE(status))
+ return false;
+
+ acpi_put_table(table);
+ return true;
+}
+#endif
+
int init_cache_level(unsigned int cpu)
{
unsigned int ctype, level, leaves, fw_level;
@@ -95,5 +110,19 @@ int populate_cache_leaves(unsigned int cpu)
ci_leaf_init(this_leaf++, type, level);
}
}
+
+#if defined(CONFIG_HWPF_CONTROL)
+ /*
+ * Hardware prefetch functions need cache sysfs directory and cache
+ * level/type information. In ARM processor, these information can be
+ * obtained from registers even without PPTT. Therefore, we set the
+ * cpu_map_populated to true to create cache sysfs directory, if the
+ * machine doesn't have PPTT.
+ **/
+ if (!acpi_disabled)
+ if (!acpi_has_pptt())
+ this_cpu_ci->cpu_map_populated = true;
+#endif
+
return 0;
}
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 2/9] drivers: base: Add Kconfig/Makefile to build hardware prefetch control core driver
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This adds Kconfig/Makefile to build hardware prefetch control core
driver. This also adds a MAINTAINERS entry.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
MAINTAINERS | 6 ++++++
drivers/base/Kconfig | 9 +++++++++
drivers/base/Makefile | 1 +
3 files changed, 16 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 40fa1955ca3f..f6640dc053c0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8615,6 +8615,12 @@ F: include/linux/hwmon*.h
F: include/trace/events/hwmon*.h
K: (devm_)?hwmon_device_(un)?register(|_with_groups|_with_info)
+HARDWARE PREFETCH CONTROL DRIVERS
+M: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
+S: Maintained
+F: drivers/base/pfctl.c
+F: include/linux/pfctl.h
+
HARDWARE RANDOM NUMBER GENERATOR CORE
M: Matt Mackall <mpm@selenic.com>
M: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 6f04b831a5c0..8f8a69e7f645 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -230,4 +230,13 @@ config GENERIC_ARCH_NUMA
Enable support for generic NUMA implementation. Currently, RISC-V
and ARM64 use it.
+config HWPF_CONTROL
+ bool "Hardware Prefetch Control driver"
+ help
+ This driver allows user to control CPU's Hardware Prefetch behavior.
+ If the machine supports this behavior, it provides a sysfs interface.
+
+ See Documentation/ABI/testing/sysfs-devices-system-cpu for more
+ information.
+
endmenu
diff --git a/drivers/base/Makefile b/drivers/base/Makefile
index 02f7f1358e86..13f3a0ddf3d1 100644
--- a/drivers/base/Makefile
+++ b/drivers/base/Makefile
@@ -25,6 +25,7 @@ obj-$(CONFIG_DEV_COREDUMP) += devcoredump.o
obj-$(CONFIG_GENERIC_MSI_IRQ_DOMAIN) += platform-msi.o
obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) += arch_topology.o
obj-$(CONFIG_GENERIC_ARCH_NUMA) += arch_numa.o
+obj-$(CONFIG_HWPF_CONTROL) += pfctl.o
obj-y += test/
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 1/9] drivers: base: Add hardware prefetch control core driver
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
This driver adds the register/unregister function to create the
"prefetch_control" directory and some attribute files. Attributes are
only present if the particular cache implements the relevant
prefetcher controls
If the architecture has control of the CPU's hardware prefetcher
behavior, use this function to create sysfs. When registering, it is
necessary to provide what type of hardware prefetcher is supported
and how to read/write to the register.
Following patches add support for A64FX and x86.
Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
---
drivers/base/pfctl.c | 458 ++++++++++++++++++++++++++++++++++++++++++
include/linux/pfctl.h | 49 +++++
2 files changed, 507 insertions(+)
create mode 100644 drivers/base/pfctl.c
create mode 100644 include/linux/pfctl.h
diff --git a/drivers/base/pfctl.c b/drivers/base/pfctl.c
new file mode 100644
index 000000000000..035d51c64e33
--- /dev/null
+++ b/drivers/base/pfctl.c
@@ -0,0 +1,458 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 FUJITSU LIMITED
+ *
+ * This driver provides tunable sysfs interface for Hardware Prefetch Control.
+ * See Documentation/ABI/testing/sysfs-devices-system-cpu for more information.
+ *
+ * This code provides architecture-independent functions such as create and
+ * remove attribute file.
+ * The implementation of reads and writes to the Hardware Prefetch Control
+ * register is architecture-dependent. Therefore, each architecture register
+ * a callback to read and write the register via pfctl_register_driver().
+ */
+
+#include <linux/cacheinfo.h>
+#include <linux/cpu.h>
+#include <linux/device.h>
+#include <linux/pfctl.h>
+#include <linux/parser.h>
+#include <linux/slab.h>
+
+#ifdef pr_fmt
+#undef pr_fmt
+#endif
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+static DEFINE_PER_CPU(struct device *, cache_device_pcpu);
+#define per_cpu_cache_device(cpu) (per_cpu(cache_device_pcpu, cpu))
+
+struct pfctl_driver *pdriver;
+enum cpuhp_state hp_online;
+
+static const char dist_auto_string[] = "auto";
+static const char strength_strong_string[] = "strong";
+static const char strength_weak_string[] = "weak";
+
+static bool prefetcher_is_available(unsigned int level, enum cache_type type,
+ int prefetcher)
+{
+ if ((level == 1) && (type == CACHE_TYPE_DATA)) {
+ if (pdriver->supported_l1d_prefetcher & prefetcher)
+ return true;
+ } else if ((level == 2) && (type == CACHE_TYPE_UNIFIED)) {
+ if (pdriver->supported_l2_prefetcher & prefetcher)
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Returns the cpu number of the cpu_device(/sys/devices/system/cpu/cpuX)
+ * in the ancestor directory of prefetch_control.
+ *
+ * When initializing this driver, it is verified that the cache directory exists
+ * under cpu_device. Therefore, the third level up from prefetch_control is
+ * cpu_device as shown below.
+ *
+ * /sys/devices/system/cpu/cpuX/cache/indexX/prefetch_control
+ */
+static inline unsigned int get_pfctl_device_cpunum(struct device *pfctl_dev)
+{
+ return pfctl_dev->parent->parent->parent->id;
+}
+
+#define pfctl_enable_show(prefetcher, pattr) \
+static ssize_t \
+prefetcher##_enable_show(struct device *dev, \
+ struct device_attribute *attr, char *buf) \
+{ \
+ int ret; \
+ u64 val; \
+ unsigned int cpu; \
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent); \
+ \
+ cpu = get_pfctl_device_cpunum(dev); \
+ \
+ ret = pdriver->read_pfreg(pattr, cpu, this_leaf->level, &val); \
+ if (ret < 0) \
+ return ret; \
+ \
+ if ((val == PFCTL_ENABLE_VAL) || (val == PFCTL_DISABLE_VAL)) \
+ return sysfs_emit(buf, "%llu\n", val); \
+ else \
+ return -EINVAL; \
+}
+
+pfctl_enable_show(hardware_prefetcher, HWPF_ENABLE);
+pfctl_enable_show(ip_prefetcher, IPPF_ENABLE);
+pfctl_enable_show(adjacent_cache_line_prefetcher, ACLPF_ENABLE);
+pfctl_enable_show(stream_detect_prefetcher, SDPF_ENABLE);
+
+static ssize_t
+stream_detect_prefetcher_strength_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int ret;
+ u64 val;
+ unsigned int cpu;
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent);
+
+ cpu = get_pfctl_device_cpunum(dev);
+
+ ret = pdriver->read_pfreg(SDPF_STRENGTH, cpu, this_leaf->level, &val);
+ if (ret < 0)
+ return ret;
+
+ if (val == PFCTL_STRONG_VAL)
+ return sysfs_emit(buf, "%s\n", strength_strong_string);
+ if (val == PFCTL_WEAK_VAL)
+ return sysfs_emit(buf, "%s\n", strength_weak_string);
+ else
+ return -EINVAL;
+}
+
+static ssize_t
+stream_detect_prefetcher_strength_available_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ ssize_t len = 0;
+
+ if (pdriver->supported_sdpf_strength & STRENGTH_STRONG)
+ len += sysfs_emit_at(buf, len, "%s ", strength_strong_string);
+
+ if (pdriver->supported_sdpf_strength & STRENGTH_WEAK)
+ len += sysfs_emit_at(buf, len, "%s ", strength_weak_string);
+
+ len += sysfs_emit_at(buf, len, "\n");
+
+ return len;
+}
+
+static ssize_t
+stream_detect_prefetcher_dist_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int ret;
+ u64 val;
+ unsigned int cpu;
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent);
+
+ cpu = get_pfctl_device_cpunum(dev);
+
+ ret = pdriver->read_pfreg(SDPF_DIST, cpu, this_leaf->level, &val);
+ if (ret < 0)
+ return ret;
+
+ if (val == PFCTL_DIST_AUTO_VAL)
+ return sysfs_emit(buf, "%s\n", dist_auto_string);
+ else
+ return sysfs_emit(buf, "%llu\n", val);
+}
+
+#define pfctl_enable_store(prefetcher, pattr) \
+static ssize_t \
+prefetcher##_enable_store(struct device *dev, \
+ struct device_attribute *attr, \
+ const char *buf, size_t count) \
+{ \
+ int ret; \
+ u64 val; \
+ unsigned int cpu; \
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent); \
+ \
+ ret = kstrtoull(buf, 10, &val); \
+ if (ret < 0) \
+ return -EINVAL; \
+ \
+ if ((val != PFCTL_ENABLE_VAL) && (val != PFCTL_DISABLE_VAL)) \
+ return -EINVAL; \
+ \
+ cpu = get_pfctl_device_cpunum(dev); \
+ \
+ ret = pdriver->write_pfreg(pattr, cpu, this_leaf->level, val); \
+ if (ret < 0) \
+ return ret; \
+ \
+ return count; \
+}
+
+pfctl_enable_store(hardware_prefetcher, HWPF_ENABLE);
+pfctl_enable_store(ip_prefetcher, IPPF_ENABLE);
+pfctl_enable_store(adjacent_cache_line_prefetcher, ACLPF_ENABLE);
+pfctl_enable_store(stream_detect_prefetcher, SDPF_ENABLE);
+
+static ssize_t
+stream_detect_prefetcher_strength_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ int ret;
+ u64 val;
+ unsigned int cpu;
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent);
+
+ if (sysfs_streq(buf, strength_strong_string))
+ val = PFCTL_STRONG_VAL;
+ else if (sysfs_streq(buf, strength_weak_string))
+ val = PFCTL_WEAK_VAL;
+ else
+ return -EINVAL;
+
+ cpu = get_pfctl_device_cpunum(dev);
+
+ ret = pdriver->write_pfreg(SDPF_STRENGTH, cpu, this_leaf->level, val);
+ if (ret < 0)
+ return ret;
+
+ return count;
+}
+
+static ssize_t
+stream_detect_prefetcher_dist_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ int ret;
+ u64 val;
+ unsigned int cpu;
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent);
+
+ if (sysfs_streq(buf, dist_auto_string)) {
+ val = PFCTL_DIST_AUTO_VAL;
+ } else {
+ ret = kstrtoull(buf, 10, &val);
+ if (ret < 0 || val < 1)
+ return -EINVAL;
+ }
+
+ cpu = get_pfctl_device_cpunum(dev);
+
+ ret = pdriver->write_pfreg(SDPF_DIST, cpu, this_leaf->level, val);
+ if (ret < 0)
+ return ret;
+
+ return count;
+}
+
+static DEVICE_ATTR_ADMIN_RW(hardware_prefetcher_enable);
+static DEVICE_ATTR_ADMIN_RW(ip_prefetcher_enable);
+static DEVICE_ATTR_ADMIN_RW(adjacent_cache_line_prefetcher_enable);
+static DEVICE_ATTR_ADMIN_RW(stream_detect_prefetcher_enable);
+static DEVICE_ATTR_ADMIN_RW(stream_detect_prefetcher_strength);
+static DEVICE_ATTR_ADMIN_RO(stream_detect_prefetcher_strength_available);
+static DEVICE_ATTR_ADMIN_RW(stream_detect_prefetcher_dist);
+
+static umode_t
+pfctl_attrs_is_visible(struct kobject *kobj, struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev->parent);
+ umode_t mode = attr->mode;
+
+ if ((attr == &dev_attr_hardware_prefetcher_enable.attr) &&
+ (prefetcher_is_available(this_leaf->level, this_leaf->type, HWPF)))
+ return mode;
+
+ if ((attr == &dev_attr_ip_prefetcher_enable.attr) &&
+ (prefetcher_is_available(this_leaf->level, this_leaf->type, IPPF)))
+ return mode;
+
+ if ((attr == &dev_attr_adjacent_cache_line_prefetcher_enable.attr) &&
+ (prefetcher_is_available(this_leaf->level, this_leaf->type, ACLPF)))
+ return mode;
+
+ if (((attr == &dev_attr_stream_detect_prefetcher_enable.attr) ||
+ (attr == &dev_attr_stream_detect_prefetcher_strength.attr) ||
+ (attr == &dev_attr_stream_detect_prefetcher_strength_available.attr)
+ || (attr == &dev_attr_stream_detect_prefetcher_dist.attr)) &&
+ (prefetcher_is_available(this_leaf->level, this_leaf->type, SDPF)))
+ return mode;
+
+ return 0;
+}
+
+static struct attribute *pfctl_attrs[] = {
+ &dev_attr_hardware_prefetcher_enable.attr,
+ &dev_attr_ip_prefetcher_enable.attr,
+ &dev_attr_adjacent_cache_line_prefetcher_enable.attr,
+ &dev_attr_stream_detect_prefetcher_enable.attr,
+ &dev_attr_stream_detect_prefetcher_strength.attr,
+ &dev_attr_stream_detect_prefetcher_strength_available.attr,
+ &dev_attr_stream_detect_prefetcher_dist.attr,
+ NULL,
+};
+
+static const struct attribute_group pfctl_group = {
+ .attrs = pfctl_attrs,
+ .is_visible = pfctl_attrs_is_visible,
+};
+
+static const struct attribute_group *pfctl_groups[] = {
+ &pfctl_group,
+ NULL,
+};
+
+static int find_cache_device(unsigned int cpu)
+{
+ struct device *cpu_dev = get_cpu_device(cpu);
+ struct device *cache_dev;
+
+ cache_dev = device_find_child_by_name(cpu_dev, "cache");
+ if (!cache_dev)
+ return -ENODEV;
+
+ per_cpu_cache_device(cpu) = cache_dev;
+ return 0;
+}
+
+static int _remove_pfctl_attr(struct device *dev, void *data)
+{
+ struct cacheinfo *leaf = dev_get_drvdata(dev);
+ struct device *pfctl_dev;
+
+ if (!prefetcher_is_available(leaf->level, leaf->type, ANYPF))
+ return 0;
+
+ pfctl_dev = device_find_child_by_name(dev, "prefetch_control");
+ if (!pfctl_dev)
+ return 0;
+
+ device_unregister(pfctl_dev);
+ put_device(pfctl_dev);
+ return 0;
+}
+
+static void remove_pfctl_attr(unsigned int cpu)
+{
+ struct device *cache_dev = per_cpu_cache_device(cpu);
+
+ if (!cache_dev)
+ return;
+
+ device_for_each_child(cache_dev, NULL, _remove_pfctl_attr);
+ put_device(cache_dev);
+}
+
+static int _create_pfctl_attr(struct device *dev, void *data)
+{
+ struct cacheinfo *leaf = dev_get_drvdata(dev);
+ struct device *pfctl_dev;
+
+ if (!prefetcher_is_available(leaf->level, leaf->type, ANYPF))
+ return 0;
+
+ pfctl_dev = cpu_device_create(dev, NULL, pfctl_groups,
+ "prefetch_control");
+ if (IS_ERR(pfctl_dev))
+ return PTR_ERR(pfctl_dev);
+
+ return 0;
+}
+
+static int create_pfctl_attr(unsigned int cpu)
+{
+ int ret;
+ struct device *cache_dev = per_cpu_cache_device(cpu);
+
+ if (!cache_dev)
+ return -ENODEV;
+
+ ret = device_for_each_child(cache_dev, NULL, _create_pfctl_attr);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int pfctl_online(unsigned int cpu)
+{
+ int ret;
+
+ ret = find_cache_device(cpu);
+ if (ret < 0)
+ return ret;
+
+ ret = create_pfctl_attr(cpu);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int pfctl_prepare_down(unsigned int cpu)
+{
+ remove_pfctl_attr(cpu);
+
+ return 0;
+}
+
+/**
+ * pfctl_register_driver - register a Hardware Prefetch Control driver
+ * @driver_data: struct pfctl_driver must contain the supported prefetcher type
+ * and function pointer for reading and writing hardware prefetch
+ * register. If these are not defined this function return error.
+ *
+ * Note: This function must be called after the cache device is initialized
+ * because it requires access to the cache device.
+ * (e.g. Call at the late_initcall)
+ *
+ * Context: Any context.
+ * Return: 0 on success, negative error code on failure.
+ */
+int pfctl_register_driver(struct pfctl_driver *driver_data)
+{
+ int ret;
+
+ if (pdriver)
+ return -EEXIST;
+
+ if (!(driver_data->supported_l1d_prefetcher & ANYPF) &&
+ !(driver_data->supported_l2_prefetcher & ANYPF))
+ return -EINVAL;
+
+ if ((driver_data->supported_l1d_prefetcher & SDPF) ||
+ (driver_data->supported_l2_prefetcher & SDPF))
+ if (!(driver_data->supported_sdpf_strength & ANY_STRENGTH))
+ return -EINVAL;
+
+ if (!driver_data->read_pfreg || !driver_data->write_pfreg)
+ return -EINVAL;
+
+ pdriver = driver_data;
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "base/pfctl:online",
+ pfctl_online, pfctl_prepare_down);
+ if (ret < 0) {
+ pr_err("failed to register hotplug callbacks\n");
+ pdriver = NULL;
+ return ret;
+ }
+
+ hp_online = ret;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pfctl_register_driver);
+
+/**
+ * pfctl_unregister_driver - unregister the Hardware Prefetch Control driver
+ * @driver_data: Used to verify that this function is called by the driver that
+ * called pfctl_register_driver by determining if driver_data is
+ * the same.
+ *
+ * Context: Any context.
+ * Return: nothing.
+ */
+void pfctl_unregister_driver(struct pfctl_driver *driver_data)
+{
+ if (!pdriver || (driver_data != pdriver))
+ return;
+
+ cpuhp_remove_state(hp_online);
+
+ pdriver = NULL;
+}
+EXPORT_SYMBOL_GPL(pfctl_unregister_driver);
diff --git a/include/linux/pfctl.h b/include/linux/pfctl.h
new file mode 100644
index 000000000000..9340b23f3d03
--- /dev/null
+++ b/include/linux/pfctl.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_PFCTL_H
+#define _LINUX_PFCTL_H
+
+#define PFCTL_ENABLE_VAL 1
+#define PFCTL_DISABLE_VAL 0
+#define PFCTL_STRONG_VAL 0
+#define PFCTL_WEAK_VAL 1
+#define PFCTL_DIST_AUTO_VAL 0
+
+enum pfctl_attr {
+ HWPF_ENABLE,
+ IPPF_ENABLE,
+ ACLPF_ENABLE,
+ SDPF_ENABLE,
+ SDPF_STRENGTH,
+ SDPF_DIST,
+};
+
+enum prefetcher {
+ HWPF = BIT(0), /* Hardware Prefetcher */
+ IPPF = BIT(1), /* IP Prefetcher */
+ ACLPF = BIT(2), /* Adjacent Cache Line Prefetcher */
+ SDPF = BIT(3), /* Stream Detect Prefetcher */
+ ANYPF = HWPF|IPPF|ACLPF|SDPF,
+};
+
+enum strength {
+ STRENGTH_STRONG = BIT(0),
+ STRENGTH_WEAK = BIT(1),
+ ANY_STRENGTH = STRENGTH_STRONG|STRENGTH_WEAK,
+};
+
+struct pfctl_driver {
+ unsigned int supported_l1d_prefetcher;
+ unsigned int supported_l2_prefetcher;
+
+ unsigned int supported_sdpf_strength; /* Set this if support SDPF */
+
+ int (*read_pfreg)(enum pfctl_attr pattr, unsigned int cpu,
+ unsigned int level, u64 *val);
+ int (*write_pfreg)(enum pfctl_attr pattr, unsigned int cpu,
+ unsigned int level, u64 val);
+};
+
+int pfctl_register_driver(struct pfctl_driver *driver_data);
+void pfctl_unregister_driver(struct pfctl_driver *driver_data);
+
+#endif
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH v3 0/9] Add hardware prefetch control driver for A64FX and x86
From: Kohei Tarumizu @ 2022-04-20 3:02 UTC (permalink / raw)
To: catalin.marinas, will, tglx, mingo, bp, dave.hansen, x86, hpa,
linux-arm-kernel, linux-kernel, fenghua.yu, reinette.chatre
Cc: tarumizu.kohei
This patch series add sysfs interface to control CPU's hardware
prefetch behavior for performance tuning from userspace for the
processor A64FX and x86 (on supported CPU).
Changes from v2:
- move arm64 driver (arch/arm64) to A64FX only (drivers/soc/fujitsu)
- prohibit writing 0 to stream_detect_prefetcher_dist
- change the type of strongness state handled from bool to string
(e.g. "strong"), and rename to stream_detect_prefetcher_strength
- change x86 code to work correctly with resctrl's pseudo lock
- read and write registers in one smp_call_function_single() to
prevent context switch when writing registers in x86-pfctl.c
- restore to original value when re-enabling the register in
pseudo_lock.c
- add prefix to driver's name for A64FX(a64fx-) and x86(x86-)
- modify the document
- split the description of pfctl into blocks for x86 and A64FX
- remove unnecessary descriptions
https://lore.kernel.org/lkml/20220311101940.3403607-1-tarumizu.kohei@fujitsu.com/
[Background]
============
A64FX and some Intel processors have implementation-dependent register
for controlling CPU's hardware prefetch behavior. A64FX has
IMP_PF_STREAM_DETECT_CTRL_EL0[1], and Intel processors have MSR 0x1a4
(MSR_MISC_FEATURE_CONTROL)[2]. These registers cannot be accessed from
userspace.
[1]https://github.com/fujitsu/A64FX/tree/master/doc/
A64FX_Specification_HPC_Extension_v1_EN.pdf
[2]https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html
Volume 4
The advantage of using this is improved performance. As an example of
performance improvements, the results of running the Stream benchmark
on the A64FX are described in section [Merit].
For MSR 0x1a4, it is also possible to change the value from userspace
via the MSR driver. However, using MSR driver is not recommended, so
it needs a proper kernel interface[3].
[3]https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/about/
For these reasons, we provide a new proper kernel interface to control
both IMP_PF_STREAM_DETECT_CTRL_EL0 and MSR 0x1a4.
[Overall design]
================
The source code for this driver is divided into common parts
(driver/base/pfctl.c) and architecture parts (arch/XXX/XXX/pfctl.c).
Common parts is described architecture-independent processing, such as
creating sysfs.
Architecture parts is described architecture-dependent processing. It
must contain at least the what type of hardware prefetcher is supported
and how to read/write to the register. These information are set
through registration function in common parts.
This driver creates "prefetch_control" directory and some attribute
files in every CPU's cache/index[0,2] directory, if CPU supports
hardware prefetch control behavior. Each attribute file corresponds to
the cache level of the parent index directory.
Detailed description of this sysfs interface is in
Documentation/ABI/testing/sysfs-devices-system-cpu (patch8).
This driver needs cache sysfs directory and cache level/type
information. In ARM processor, these information can be obtained
from registers even without ACPI PPTT.
We add processing to create a cache/index directory using only the
information from the register if the machine does not support ACPI
PPTT and Kconfig for hardware prefetch control (CONFIG_HWPF_CONTROL)
is true in patch5.
This action caused a problem and is described in [Known problem].
[Examples]
==========
This section provides an example of using this sysfs interface at the
x86's model of INTEL_FAM6_BROADWELL_X.
This model has the following register specifications:
[0] L2 Hardware Prefetcher Disable (R/W)
[1] L2 Adjacent Cache Line Prefetcher Disable (R/W)
[2] DCU Hardware Prefetcher Disable (R/W)
[3] DCU IP Prefetcher Disable (R/W)
[63:4] Reserved
In this case, index0 (L1d cache) corresponds to bit[2,3] and index2
(L2 cache) corresponds to bit [0,1]. A list of attribute files of
index0 and index2 in CPU1 at BROADWELL_X is following:
```
# ls /sys/devices/system/cpu/cpu1/cache/index0/prefetch_control/
hardware_prefetcher_enable
ip_prefetcher_enable
# ls /sys/devices/system/cpu/cpu1/cache/index2/prefetch_control/
adjacent_cache_line_prefetcher_enable
hardware_prefetcher_enable
```
If user would like to disable the setting of "L2 Adjacent Cache Line
Prefetcher Disable (R/W)" in CPU1, do the following:
```
# echo 0 > /sys/devices/system/cpu/cpu1/cache/index2/prefetch_control/adjacent_cache_line_prefetcher_enable
```
In another example, a list of index0 at A64FX is following:
```
# ls /sys/devices/system/cpu/cpu1/cache/index0/prefetch_control/
stream_detect_prefetcher_dist
stream_detect_prefetcher_enable
stream_detect_prefetcher_strength
stream_detect_prefetcher_strength_available
```
[Patch organizations]
=====================
This patch series add hardware prefetch control core driver for A64FX
and x86. Also, we add support for A64FX and BROADWELL_X at x86.
- patch1: Add hardware prefetch core driver
This driver provides a register/unregister function to create the
"prefetch_control" directory and some attribute files in every CPU's
cache/index[0,2] directory.
If the architecture has control of the CPU's hardware prefetch
behavior, use this function to create sysfs. When registering, it
is necessary to provide what type of Hardware Prefetcher is
supported and how to read/write to the register.
- patch2: Add Kconfig/Makefile to build hardware prefetch control core
driver
- patch3: Add support for A64FX
This adds module init/exit code, and creates sysfs attribute file
"stream_detect_prefetcher_enable", "stream_detect_prefetcher_strong"
and "stream_detect_prefetcher_dist" for A64FX. This driver works only
if part number is FUJITSU_CPU_PART_A64FX at this point.
- patch4: Add Kconfig/Makefile to build driver for A64FX
- patch5: Create cache sysfs directory without ACPI PPTT for hardware
prefetch control
Hardware Prefetch control driver needs cache sysfs directory and cache
level/type information. In ARM processor, these information can be
obtained from registers(CLIDR_EL1) even without PPTT. Therefore, we
set the cpu_map_populated to true to create cache sysfs directory, if
the machine doesn't have PPTT.
- patch6: Fix to restore to original value when re-enabling hardware
prefetch register in pseudo_lock.c
The current pseudo_lock.c code overwrittes the value of the
MSR_MISC_FEATURE_CONTROL to 0 even if the original value is not 0.
Therefore, modify it to save and restore the original values.
- patch7: Add support for x86
This adds module init/exit code, and creates sysfs attribute file
"hardware_prefetcher_enable", "ip_prefetcher_enable" and
"adjacent_cache_line_prefetcher_enable" for x86. This driver works
only if the model is INTEL_FAM6_BROADWELL_X at this point.
- patch8: Add Kconfig/Makefile to build driver for x86
- patch9: Add documentation for the new sysfs interface
[Known problem]
===============
- `lscpu` command terminates with -ENOENT because cache/index directory
is exists but shared_cpu_map file does not exist. This is due to
patch5, which creates a cache/index directory containing only level
and type without ACPI PPTT.
[Merit]
=======
For reference, here is the result of STREAM Triad when tuning with
the "s file in L1 and L2 cache on A64FX.
| dist combination | Pattern A | Pattern B |
|-------------------|-------------|-------------|
| L1:256, L2:1024 | 234505.2144 | 114600.0801 |
| L1:1536, L2:1024 | 279172.8742 | 118979.4542 |
| L1:256, L2:10240 | 247716.7757 | 127364.1533 |
| L1:1536, L2:10240 | 283675.6625 | 125950.6847 |
In pattern A, we set the size of the array to 174720, which is about
half the size of the L1d cache. In pattern B, we set the size of the
array to 10485120, which is about twice the size of the L2 cache.
In pattern A, a change of dist at L1 has a larger effect. On the other
hand, in pattern B, the change of dist at L2 has a larger effect.
As described above, the optimal dist combination depends on the
characteristics of the application. Therefore, such a sysfs interface
is useful for performance tuning.
Best regards,
Kohei Tarumizu
Kohei Tarumizu (9):
drivers: base: Add hardware prefetch control core driver
drivers: base: Add Kconfig/Makefile to build hardware prefetch control
core driver
soc: fujitsu: Add hardware prefetch control support for A64FX
soc: fujitsu: Add Kconfig/Makefile to build hardware prefetch control
driver
arm64: Create cache sysfs directory without ACPI PPTT for hardware
prefetch control
x86: resctrl: pseudo_lock: Fix to restore to original value when
re-enabling hardware prefetch register
x86: Add hardware prefetch control support for x86
x86: Add Kconfig/Makefile to build hardware prefetch control driver
docs: ABI: Add sysfs documentation interface of hardware prefetch
control driver
.../ABI/testing/sysfs-devices-system-cpu | 98 ++++
MAINTAINERS | 8 +
arch/arm64/kernel/cacheinfo.c | 29 ++
arch/x86/Kconfig | 6 +
arch/x86/kernel/cpu/Makefile | 2 +
arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 12 +-
arch/x86/kernel/cpu/x86-pfctl.c | 347 +++++++++++++
drivers/base/Kconfig | 9 +
drivers/base/Makefile | 1 +
drivers/base/pfctl.c | 458 ++++++++++++++++++
drivers/soc/Kconfig | 1 +
drivers/soc/Makefile | 1 +
drivers/soc/fujitsu/Kconfig | 11 +
drivers/soc/fujitsu/Makefile | 2 +
drivers/soc/fujitsu/a64fx-pfctl.c | 356 ++++++++++++++
include/linux/pfctl.h | 49 ++
16 files changed, 1387 insertions(+), 3 deletions(-)
create mode 100644 arch/x86/kernel/cpu/x86-pfctl.c
create mode 100644 drivers/base/pfctl.c
create mode 100644 drivers/soc/fujitsu/Kconfig
create mode 100644 drivers/soc/fujitsu/Makefile
create mode 100644 drivers/soc/fujitsu/a64fx-pfctl.c
create mode 100644 include/linux/pfctl.h
--
2.27.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH -next v4 7/7] arm64: add cow to machine check safe
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
In the cow(copy on write) processing, the data of the user process is
copied, when hardware memory error is encountered during copy, only the
relevant processes are affected, so killing the user process and isolate
the user page with hardware memory errors is a more reasonable choice than
kernel panic.
Add new helper copy_page_mc() which provide a page copy implementation with
machine check safe. At present, only used in cow. In future, we can expand
more scenes. As long as the consequences of page copy failure are not
fatal(eg: only affect user process), we can use this helper.
The copy_page_mc() in copy_page_mc.S is largely borrows from copy_page()
in copy_page.S and the main difference is copy_page_mc() add extable entry
to every load/store insn to support machine check safe. largely to keep the
patch simple. If needed those optimizations can be folded in.
Add new extable type EX_TYPE_COPY_PAGE_MC which used in copy_page_mc().
This type only be processed in fixup_exception_mc(), The reason is that
copy_page_mc() is consistent with copy_page() except machine check safe is
considered, and copy_page() do not need to consider exception fixup.
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
---
arch/arm64/include/asm/asm-extable.h | 5 ++
arch/arm64/include/asm/page.h | 10 ++++
arch/arm64/lib/Makefile | 2 +
arch/arm64/lib/copy_page_mc.S | 86 ++++++++++++++++++++++++++++
arch/arm64/mm/copypage.c | 36 ++++++++++--
arch/arm64/mm/extable.c | 2 +
include/linux/highmem.h | 8 +++
mm/memory.c | 2 +-
8 files changed, 144 insertions(+), 7 deletions(-)
create mode 100644 arch/arm64/lib/copy_page_mc.S
diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h
index 80410899a9ad..74c056ddae15 100644
--- a/arch/arm64/include/asm/asm-extable.h
+++ b/arch/arm64/include/asm/asm-extable.h
@@ -14,6 +14,7 @@
/* _MC indicates that can fixup from machine check errors */
#define EX_TYPE_UACCESS_MC 5
#define EX_TYPE_UACCESS_MC_ERR_ZERO 6
+#define EX_TYPE_COPY_PAGE_MC 7
#ifdef __ASSEMBLY__
@@ -42,6 +43,10 @@
__ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_UACCESS_MC, 0)
.endm
+ .macro _asm_extable_copy_page_mc, insn, fixup
+ __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_COPY_PAGE_MC, 0)
+ .endm
+
/*
* Create an exception table entry for `insn` if `fixup` is provided. Otherwise
* do nothing.
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 993a27ea6f54..832571a7dddb 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -29,6 +29,16 @@ void copy_user_highpage(struct page *to, struct page *from,
void copy_highpage(struct page *to, struct page *from);
#define __HAVE_ARCH_COPY_HIGHPAGE
+#ifdef CONFIG_ARCH_HAS_COPY_MC
+extern void copy_page_mc(void *to, const void *from);
+void copy_highpage_mc(struct page *to, struct page *from);
+#define __HAVE_ARCH_COPY_HIGHPAGE_MC
+
+void copy_user_highpage_mc(struct page *to, struct page *from,
+ unsigned long vaddr, struct vm_area_struct *vma);
+#define __HAVE_ARCH_COPY_USER_HIGHPAGE_MC
+#endif
+
struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma,
unsigned long vaddr);
#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index 29490be2546b..0d9f292ef68a 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -15,6 +15,8 @@ endif
lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o
+lib-$(CONFIG_ARCH_HAS_COPY_MC) += copy_page_mc.o
+
obj-$(CONFIG_CRC32) += crc32.o
obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
diff --git a/arch/arm64/lib/copy_page_mc.S b/arch/arm64/lib/copy_page_mc.S
new file mode 100644
index 000000000000..655161363dcf
--- /dev/null
+++ b/arch/arm64/lib/copy_page_mc.S
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ */
+
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/assembler.h>
+#include <asm/page.h>
+#include <asm/cpufeature.h>
+#include <asm/alternative.h>
+#include <asm/asm-extable.h>
+
+#define CPY_MC(l, x...) \
+9999: x; \
+ _asm_extable_copy_page_mc 9999b, l
+
+/*
+ * Copy a page from src to dest (both are page aligned) with machine check
+ *
+ * Parameters:
+ * x0 - dest
+ * x1 - src
+ */
+SYM_FUNC_START(__pi_copy_page_mc)
+alternative_if ARM64_HAS_NO_HW_PREFETCH
+ // Prefetch three cache lines ahead.
+ prfm pldl1strm, [x1, #128]
+ prfm pldl1strm, [x1, #256]
+ prfm pldl1strm, [x1, #384]
+alternative_else_nop_endif
+
+CPY_MC(9998f, ldp x2, x3, [x1])
+CPY_MC(9998f, ldp x4, x5, [x1, #16])
+CPY_MC(9998f, ldp x6, x7, [x1, #32])
+CPY_MC(9998f, ldp x8, x9, [x1, #48])
+CPY_MC(9998f, ldp x10, x11, [x1, #64])
+CPY_MC(9998f, ldp x12, x13, [x1, #80])
+CPY_MC(9998f, ldp x14, x15, [x1, #96])
+CPY_MC(9998f, ldp x16, x17, [x1, #112])
+
+ add x0, x0, #256
+ add x1, x1, #128
+1:
+ tst x0, #(PAGE_SIZE - 1)
+
+alternative_if ARM64_HAS_NO_HW_PREFETCH
+ prfm pldl1strm, [x1, #384]
+alternative_else_nop_endif
+
+CPY_MC(9998f, stnp x2, x3, [x0, #-256])
+CPY_MC(9998f, ldp x2, x3, [x1])
+CPY_MC(9998f, stnp x4, x5, [x0, #16 - 256])
+CPY_MC(9998f, ldp x4, x5, [x1, #16])
+CPY_MC(9998f, stnp x6, x7, [x0, #32 - 256])
+CPY_MC(9998f, ldp x6, x7, [x1, #32])
+CPY_MC(9998f, stnp x8, x9, [x0, #48 - 256])
+CPY_MC(9998f, ldp x8, x9, [x1, #48])
+CPY_MC(9998f, stnp x10, x11, [x0, #64 - 256])
+CPY_MC(9998f, ldp x10, x11, [x1, #64])
+CPY_MC(9998f, stnp x12, x13, [x0, #80 - 256])
+CPY_MC(9998f, ldp x12, x13, [x1, #80])
+CPY_MC(9998f, stnp x14, x15, [x0, #96 - 256])
+CPY_MC(9998f, ldp x14, x15, [x1, #96])
+CPY_MC(9998f, stnp x16, x17, [x0, #112 - 256])
+CPY_MC(9998f, ldp x16, x17, [x1, #112])
+
+ add x0, x0, #128
+ add x1, x1, #128
+
+ b.ne 1b
+
+CPY_MC(9998f, stnp x2, x3, [x0, #-256])
+CPY_MC(9998f, stnp x4, x5, [x0, #16 - 256])
+CPY_MC(9998f, stnp x6, x7, [x0, #32 - 256])
+CPY_MC(9998f, stnp x8, x9, [x0, #48 - 256])
+CPY_MC(9998f, stnp x10, x11, [x0, #64 - 256])
+CPY_MC(9998f, stnp x12, x13, [x0, #80 - 256])
+CPY_MC(9998f, stnp x14, x15, [x0, #96 - 256])
+CPY_MC(9998f, stnp x16, x17, [x0, #112 - 256])
+
+9998: ret
+
+SYM_FUNC_END(__pi_copy_page_mc)
+SYM_FUNC_ALIAS(copy_page_mc, __pi_copy_page_mc)
+EXPORT_SYMBOL(copy_page_mc)
diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c
index 0dea80bf6de4..0f28edfcb234 100644
--- a/arch/arm64/mm/copypage.c
+++ b/arch/arm64/mm/copypage.c
@@ -14,13 +14,8 @@
#include <asm/cpufeature.h>
#include <asm/mte.h>
-void copy_highpage(struct page *to, struct page *from)
+static void do_mte(struct page *to, struct page *from, void *kto, void *kfrom)
{
- void *kto = page_address(to);
- void *kfrom = page_address(from);
-
- copy_page(kto, kfrom);
-
if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) {
set_bit(PG_mte_tagged, &to->flags);
page_kasan_tag_reset(to);
@@ -35,6 +30,15 @@ void copy_highpage(struct page *to, struct page *from)
mte_copy_page_tags(kto, kfrom);
}
}
+
+void copy_highpage(struct page *to, struct page *from)
+{
+ void *kto = page_address(to);
+ void *kfrom = page_address(from);
+
+ copy_page(kto, kfrom);
+ do_mte(to, from, kto, kfrom);
+}
EXPORT_SYMBOL(copy_highpage);
void copy_user_highpage(struct page *to, struct page *from,
@@ -44,3 +48,23 @@ void copy_user_highpage(struct page *to, struct page *from,
flush_dcache_page(to);
}
EXPORT_SYMBOL_GPL(copy_user_highpage);
+
+#ifdef CONFIG_ARCH_HAS_COPY_MC
+void copy_highpage_mc(struct page *to, struct page *from)
+{
+ void *kto = page_address(to);
+ void *kfrom = page_address(from);
+
+ copy_page_mc(kto, kfrom);
+ do_mte(to, from, kto, kfrom);
+}
+EXPORT_SYMBOL(copy_highpage_mc);
+
+void copy_user_highpage_mc(struct page *to, struct page *from,
+ unsigned long vaddr, struct vm_area_struct *vma)
+{
+ copy_highpage_mc(to, from);
+ flush_dcache_page(to);
+}
+EXPORT_SYMBOL_GPL(copy_user_highpage_mc);
+#endif
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index 1023ccdb2f89..4c882d36dd64 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -110,6 +110,8 @@ bool fixup_exception_mc(struct pt_regs *regs)
return ex_handler_uaccess_type(ex, regs, FIXUP_TYPE_MC);
case EX_TYPE_UACCESS_MC_ERR_ZERO:
return ex_handler_uaccess_err_zero(ex, regs);
+ case EX_TYPE_COPY_PAGE_MC:
+ return ex_handler_fixup(ex, regs);
}
diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index 39bb9b47fa9c..a9dbf331b038 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -283,6 +283,10 @@ static inline void copy_user_highpage(struct page *to, struct page *from,
#endif
+#ifndef __HAVE_ARCH_COPY_USER_HIGHPAGE_MC
+#define copy_user_highpage_mc copy_user_highpage
+#endif
+
#ifndef __HAVE_ARCH_COPY_HIGHPAGE
static inline void copy_highpage(struct page *to, struct page *from)
@@ -298,6 +302,10 @@ static inline void copy_highpage(struct page *to, struct page *from)
#endif
+#ifndef __HAVE_ARCH_COPY_HIGHPAGE_MC
+#define cop_highpage_mc copy_highpage
+#endif
+
static inline void memcpy_page(struct page *dst_page, size_t dst_off,
struct page *src_page, size_t src_off,
size_t len)
diff --git a/mm/memory.c b/mm/memory.c
index 76e3af9639d9..d5f62234152d 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2767,7 +2767,7 @@ static inline bool cow_user_page(struct page *dst, struct page *src,
unsigned long addr = vmf->address;
if (likely(src)) {
- copy_user_highpage(dst, src, addr, vma);
+ copy_user_highpage_mc(dst, src, addr, vma);
return true;
}
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH -next v4 5/7] arm64: mte: Clean up user tag accessors
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
From: Robin Murphy <robin.murphy@arm.com>
Invoking user_ldst to explicitly add a post-increment of 0 is silly.
Just use a normal USER() annotation and save the redundant instruction.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tong Tiangen <tongtiangen@huawei.com>
---
arch/arm64/lib/mte.S | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S
index 8590af3c98c0..eeb9e45bcce8 100644
--- a/arch/arm64/lib/mte.S
+++ b/arch/arm64/lib/mte.S
@@ -93,7 +93,7 @@ SYM_FUNC_START(mte_copy_tags_from_user)
mov x3, x1
cbz x2, 2f
1:
- user_ldst 2f, ldtrb, w4, x1, 0
+USER(2f, ldtrb w4, [x1])
lsl x4, x4, #MTE_TAG_SHIFT
stg x4, [x0], #MTE_GRANULE_SIZE
add x1, x1, #1
@@ -120,7 +120,7 @@ SYM_FUNC_START(mte_copy_tags_to_user)
1:
ldg x4, [x1]
ubfx x4, x4, #MTE_TAG_SHIFT, #MTE_TAG_SIZE
- user_ldst 2f, sttrb, w4, x0, 0
+USER(2f, sttrb w4, [x0])
add x0, x0, #1
add x1, x1, #MTE_GRANULE_SIZE
subs x2, x2, #1
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH -next v4 6/7] arm64: add {get, put}_user to machine check safe
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
Add {get, put}_user() to machine check safe.
If get/put fail due to hardware memory error, only the relevant processes
are affected, so killing the user process and isolate the user page with
hardware memory errors is a more reasonable choice than kernel panic.
Add new extable type EX_TYPE_UACCESS_MC_ERR_ZERO which can be used for
uaccess that can be recovered from hardware memory errors. The difference
from EX_TYPE_UACCESS_MC is that this type also sets additional two target
register which save error code and value needs to be set zero.
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
---
arch/arm64/include/asm/asm-extable.h | 14 ++++++++++++++
arch/arm64/include/asm/uaccess.h | 4 ++--
arch/arm64/mm/extable.c | 4 ++++
3 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h
index 75b2c00e9523..80410899a9ad 100644
--- a/arch/arm64/include/asm/asm-extable.h
+++ b/arch/arm64/include/asm/asm-extable.h
@@ -13,6 +13,7 @@
/* _MC indicates that can fixup from machine check errors */
#define EX_TYPE_UACCESS_MC 5
+#define EX_TYPE_UACCESS_MC_ERR_ZERO 6
#ifdef __ASSEMBLY__
@@ -78,6 +79,15 @@
#define EX_DATA_REG(reg, gpr) \
"((.L__gpr_num_" #gpr ") << " __stringify(EX_DATA_REG_##reg##_SHIFT) ")"
+#define _ASM_EXTABLE_UACCESS_MC_ERR_ZERO(insn, fixup, err, zero) \
+ __DEFINE_ASM_GPR_NUMS \
+ __ASM_EXTABLE_RAW(#insn, #fixup, \
+ __stringify(EX_TYPE_UACCESS_MC_ERR_ZERO), \
+ "(" \
+ EX_DATA_REG(ERR, err) " | " \
+ EX_DATA_REG(ZERO, zero) \
+ ")")
+
#define _ASM_EXTABLE_UACCESS_ERR_ZERO(insn, fixup, err, zero) \
__DEFINE_ASM_GPR_NUMS \
__ASM_EXTABLE_RAW(#insn, #fixup, \
@@ -90,6 +100,10 @@
#define _ASM_EXTABLE_UACCESS_ERR(insn, fixup, err) \
_ASM_EXTABLE_UACCESS_ERR_ZERO(insn, fixup, err, wzr)
+
+#define _ASM_EXTABLE_UACCESS_MC_ERR(insn, fixup, err) \
+ _ASM_EXTABLE_UACCESS_MC_ERR_ZERO(insn, fixup, err, wzr)
+
#define EX_DATA_REG_DATA_SHIFT 0
#define EX_DATA_REG_DATA GENMASK(4, 0)
#define EX_DATA_REG_ADDR_SHIFT 5
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index e8dce0cc5eaa..e41b47df48b0 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -236,7 +236,7 @@ static inline void __user *__uaccess_mask_ptr(const void __user *ptr)
asm volatile( \
"1: " load " " reg "1, [%2]\n" \
"2:\n" \
- _ASM_EXTABLE_UACCESS_ERR_ZERO(1b, 2b, %w0, %w1) \
+ _ASM_EXTABLE_UACCESS_MC_ERR_ZERO(1b, 2b, %w0, %w1) \
: "+r" (err), "=&r" (x) \
: "r" (addr))
@@ -325,7 +325,7 @@ do { \
asm volatile( \
"1: " store " " reg "1, [%2]\n" \
"2:\n" \
- _ASM_EXTABLE_UACCESS_ERR(1b, 2b, %w0) \
+ _ASM_EXTABLE_UACCESS_MC_ERR(1b, 2b, %w0) \
: "+r" (err) \
: "r" (x), "r" (addr))
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index 525876c3ebf4..1023ccdb2f89 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -88,6 +88,7 @@ bool fixup_exception(struct pt_regs *regs)
case EX_TYPE_BPF:
return ex_handler_bpf(ex, regs);
case EX_TYPE_UACCESS_ERR_ZERO:
+ case EX_TYPE_UACCESS_MC_ERR_ZERO:
return ex_handler_uaccess_err_zero(ex, regs);
case EX_TYPE_LOAD_UNALIGNED_ZEROPAD:
return ex_handler_load_unaligned_zeropad(ex, regs);
@@ -107,6 +108,9 @@ bool fixup_exception_mc(struct pt_regs *regs)
switch (ex->type) {
case EX_TYPE_UACCESS_MC:
return ex_handler_uaccess_type(ex, regs, FIXUP_TYPE_MC);
+ case EX_TYPE_UACCESS_MC_ERR_ZERO:
+ return ex_handler_uaccess_err_zero(ex, regs);
+
}
return false;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH -next v4 4/7] arm64: add copy_{to, from}_user to machine check safe
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
Add copy_{to, from}_user() to machine check safe.
If copy fail due to hardware memory error, only the relevant processes are
affected, so killing the user process and isolate the user page with
hardware memory errors is a more reasonable choice than kernel panic.
Add new extable type EX_TYPE_UACCESS_MC which can be used for uaccess that
can be recovered from hardware memory errors.
The x16 register is used to save the fixup type in copy_xxx_user which
used extable type EX_TYPE_UACCESS_MC.
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
---
arch/arm64/include/asm/asm-extable.h | 14 ++++++++++++++
arch/arm64/include/asm/asm-uaccess.h | 15 ++++++++++-----
arch/arm64/lib/copy_from_user.S | 18 +++++++++++-------
arch/arm64/lib/copy_to_user.S | 18 +++++++++++-------
arch/arm64/mm/extable.c | 18 ++++++++++++++----
5 files changed, 60 insertions(+), 23 deletions(-)
diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h
index c39f2437e08e..75b2c00e9523 100644
--- a/arch/arm64/include/asm/asm-extable.h
+++ b/arch/arm64/include/asm/asm-extable.h
@@ -2,12 +2,18 @@
#ifndef __ASM_ASM_EXTABLE_H
#define __ASM_ASM_EXTABLE_H
+#define FIXUP_TYPE_NORMAL 0
+#define FIXUP_TYPE_MC 1
+
#define EX_TYPE_NONE 0
#define EX_TYPE_FIXUP 1
#define EX_TYPE_BPF 2
#define EX_TYPE_UACCESS_ERR_ZERO 3
#define EX_TYPE_LOAD_UNALIGNED_ZEROPAD 4
+/* _MC indicates that can fixup from machine check errors */
+#define EX_TYPE_UACCESS_MC 5
+
#ifdef __ASSEMBLY__
#define __ASM_EXTABLE_RAW(insn, fixup, type, data) \
@@ -27,6 +33,14 @@
__ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_FIXUP, 0)
.endm
+/*
+ * Create an exception table entry for `insn`, which will branch to `fixup`
+ * when an unhandled fault(include sea fault) is taken.
+ */
+ .macro _asm_extable_uaccess_mc, insn, fixup
+ __ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_UACCESS_MC, 0)
+ .endm
+
/*
* Create an exception table entry for `insn` if `fixup` is provided. Otherwise
* do nothing.
diff --git a/arch/arm64/include/asm/asm-uaccess.h b/arch/arm64/include/asm/asm-uaccess.h
index 0557af834e03..6c23c138e1fc 100644
--- a/arch/arm64/include/asm/asm-uaccess.h
+++ b/arch/arm64/include/asm/asm-uaccess.h
@@ -63,6 +63,11 @@ alternative_else_nop_endif
9999: x; \
_asm_extable 9999b, l
+
+#define USER_MC(l, x...) \
+9999: x; \
+ _asm_extable_uaccess_mc 9999b, l
+
/*
* Generate the assembly for LDTR/STTR with exception table entries.
* This is complicated as there is no post-increment or pair versions of the
@@ -73,8 +78,8 @@ alternative_else_nop_endif
8889: ldtr \reg2, [\addr, #8];
add \addr, \addr, \post_inc;
- _asm_extable 8888b,\l;
- _asm_extable 8889b,\l;
+ _asm_extable_uaccess_mc 8888b, \l;
+ _asm_extable_uaccess_mc 8889b, \l;
.endm
.macro user_stp l, reg1, reg2, addr, post_inc
@@ -82,14 +87,14 @@ alternative_else_nop_endif
8889: sttr \reg2, [\addr, #8];
add \addr, \addr, \post_inc;
- _asm_extable 8888b,\l;
- _asm_extable 8889b,\l;
+ _asm_extable_uaccess_mc 8888b,\l;
+ _asm_extable_uaccess_mc 8889b,\l;
.endm
.macro user_ldst l, inst, reg, addr, post_inc
8888: \inst \reg, [\addr];
add \addr, \addr, \post_inc;
- _asm_extable 8888b,\l;
+ _asm_extable_uaccess_mc 8888b, \l;
.endm
#endif
diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S
index 34e317907524..480cc5ac0a8d 100644
--- a/arch/arm64/lib/copy_from_user.S
+++ b/arch/arm64/lib/copy_from_user.S
@@ -25,7 +25,7 @@
.endm
.macro strb1 reg, ptr, val
- strb \reg, [\ptr], \val
+ USER_MC(9998f, strb \reg, [\ptr], \val)
.endm
.macro ldrh1 reg, ptr, val
@@ -33,7 +33,7 @@
.endm
.macro strh1 reg, ptr, val
- strh \reg, [\ptr], \val
+ USER_MC(9998f, strh \reg, [\ptr], \val)
.endm
.macro ldr1 reg, ptr, val
@@ -41,7 +41,7 @@
.endm
.macro str1 reg, ptr, val
- str \reg, [\ptr], \val
+ USER_MC(9998f, str \reg, [\ptr], \val)
.endm
.macro ldp1 reg1, reg2, ptr, val
@@ -49,11 +49,12 @@
.endm
.macro stp1 reg1, reg2, ptr, val
- stp \reg1, \reg2, [\ptr], \val
+ USER_MC(9998f, stp \reg1, \reg2, [\ptr], \val)
.endm
-end .req x5
-srcin .req x15
+end .req x5
+srcin .req x15
+fixup_type .req x16
SYM_FUNC_START(__arch_copy_from_user)
add end, x0, x2
mov srcin, x1
@@ -62,7 +63,10 @@ SYM_FUNC_START(__arch_copy_from_user)
ret
// Exception fixups
-9997: cmp dst, dstin
+ // x16: fixup type written by ex_handler_uaccess_mc
+9997: cmp fixup_type, #FIXUP_TYPE_MC
+ b.eq 9998f
+ cmp dst, dstin
b.ne 9998f
// Before being absolutely sure we couldn't copy anything, try harder
USER(9998f, ldtrb tmp1w, [srcin])
diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S
index 802231772608..021a7d27b3a4 100644
--- a/arch/arm64/lib/copy_to_user.S
+++ b/arch/arm64/lib/copy_to_user.S
@@ -20,7 +20,7 @@
* x0 - bytes not copied
*/
.macro ldrb1 reg, ptr, val
- ldrb \reg, [\ptr], \val
+ USER_MC(9998f, ldrb \reg, [\ptr], \val)
.endm
.macro strb1 reg, ptr, val
@@ -28,7 +28,7 @@
.endm
.macro ldrh1 reg, ptr, val
- ldrh \reg, [\ptr], \val
+ USER_MC(9998f, ldrh \reg, [\ptr], \val)
.endm
.macro strh1 reg, ptr, val
@@ -36,7 +36,7 @@
.endm
.macro ldr1 reg, ptr, val
- ldr \reg, [\ptr], \val
+ USER_MC(9998f, ldr \reg, [\ptr], \val)
.endm
.macro str1 reg, ptr, val
@@ -44,15 +44,16 @@
.endm
.macro ldp1 reg1, reg2, ptr, val
- ldp \reg1, \reg2, [\ptr], \val
+ USER_MC(9998f, ldp \reg1, \reg2, [\ptr], \val)
.endm
.macro stp1 reg1, reg2, ptr, val
user_stp 9997f, \reg1, \reg2, \ptr, \val
.endm
-end .req x5
-srcin .req x15
+end .req x5
+srcin .req x15
+fixup_type .req x16
SYM_FUNC_START(__arch_copy_to_user)
add end, x0, x2
mov srcin, x1
@@ -61,7 +62,10 @@ SYM_FUNC_START(__arch_copy_to_user)
ret
// Exception fixups
-9997: cmp dst, dstin
+ // x16: fixup type written by ex_handler_uaccess_mc
+9997: cmp fixup_type, #FIXUP_TYPE_MC
+ b.eq 9998f
+ cmp dst, dstin
b.ne 9998f
// Before being absolutely sure we couldn't copy anything, try harder
ldrb tmp1w, [srcin]
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index 4f0083a550d4..525876c3ebf4 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -24,6 +24,14 @@ static bool ex_handler_fixup(const struct exception_table_entry *ex,
return true;
}
+static bool ex_handler_uaccess_type(const struct exception_table_entry *ex,
+ struct pt_regs *regs,
+ unsigned long fixup_type)
+{
+ regs->regs[16] = fixup_type;
+ return ex_handler_fixup(ex, regs);
+}
+
static bool ex_handler_uaccess_err_zero(const struct exception_table_entry *ex,
struct pt_regs *regs)
{
@@ -75,6 +83,8 @@ bool fixup_exception(struct pt_regs *regs)
switch (ex->type) {
case EX_TYPE_FIXUP:
return ex_handler_fixup(ex, regs);
+ case EX_TYPE_UACCESS_MC:
+ return ex_handler_uaccess_type(ex, regs, FIXUP_TYPE_NORMAL);
case EX_TYPE_BPF:
return ex_handler_bpf(ex, regs);
case EX_TYPE_UACCESS_ERR_ZERO:
@@ -94,10 +104,10 @@ bool fixup_exception_mc(struct pt_regs *regs)
if (!ex)
return false;
- /*
- * This is not complete, More Machine check safe extable type can
- * be processed here.
- */
+ switch (ex->type) {
+ case EX_TYPE_UACCESS_MC:
+ return ex_handler_uaccess_type(ex, regs, FIXUP_TYPE_MC);
+ }
return false;
}
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH -next v4 3/7] arm64: add support for machine check error safe
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
During the processing of arm64 kernel hardware memory errors(do_sea()), if
the errors is consumed in the kernel, the current processing is panic.
However, it is not optimal.
Take uaccess for example, if the uaccess operation fails due to memory
error, only the user process will be affected, kill the user process
and isolate the user page with hardware memory errors is a better choice.
This patch only enable machine error check framework, it add exception
fixup before kernel panic in do_sea() and only limit the consumption of
hardware memory errors in kernel mode triggered by user mode processes.
If fixup successful, panic can be avoided.
Consistent with PPC/x86, it is implemented by CONFIG_ARCH_HAS_COPY_MC.
Also add copy_mc_to_user() in include/linux/uaccess.h, this helper is
called when CONFIG_ARCH_HAS_COPOY_MC is open.
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/extable.h | 1 +
arch/arm64/mm/extable.c | 17 +++++++++++++++++
arch/arm64/mm/fault.c | 27 ++++++++++++++++++++++++++-
include/linux/uaccess.h | 9 +++++++++
5 files changed, 54 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d9325dd95eba..012e38309955 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -19,6 +19,7 @@ config ARM64
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2
select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
select ARCH_HAS_CACHE_LINE_SIZE
+ select ARCH_HAS_COPY_MC if ACPI_APEI_GHES
select ARCH_HAS_CURRENT_STACK_POINTER
select ARCH_HAS_DEBUG_VIRTUAL
select ARCH_HAS_DEBUG_VM_PGTABLE
diff --git a/arch/arm64/include/asm/extable.h b/arch/arm64/include/asm/extable.h
index 72b0e71cc3de..f80ebd0addfd 100644
--- a/arch/arm64/include/asm/extable.h
+++ b/arch/arm64/include/asm/extable.h
@@ -46,4 +46,5 @@ bool ex_handler_bpf(const struct exception_table_entry *ex,
#endif /* !CONFIG_BPF_JIT */
bool fixup_exception(struct pt_regs *regs);
+bool fixup_exception_mc(struct pt_regs *regs);
#endif
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
index 489455309695..4f0083a550d4 100644
--- a/arch/arm64/mm/extable.c
+++ b/arch/arm64/mm/extable.c
@@ -9,6 +9,7 @@
#include <asm/asm-extable.h>
#include <asm/ptrace.h>
+#include <asm/esr.h>
static inline unsigned long
get_ex_fixup(const struct exception_table_entry *ex)
@@ -84,3 +85,19 @@ bool fixup_exception(struct pt_regs *regs)
BUG();
}
+
+bool fixup_exception_mc(struct pt_regs *regs)
+{
+ const struct exception_table_entry *ex;
+
+ ex = search_exception_tables(instruction_pointer(regs));
+ if (!ex)
+ return false;
+
+ /*
+ * This is not complete, More Machine check safe extable type can
+ * be processed here.
+ */
+
+ return false;
+}
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 77341b160aca..a9e6fb1999d1 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -695,6 +695,29 @@ static int do_bad(unsigned long far, unsigned int esr, struct pt_regs *regs)
return 1; /* "fault" */
}
+static bool arm64_do_kernel_sea(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs, int sig, int code)
+{
+ if (!IS_ENABLED(CONFIG_ARCH_HAS_COPY_MC))
+ return false;
+
+ if (user_mode(regs) || !current->mm)
+ return false;
+
+ if (apei_claim_sea(regs) < 0)
+ return false;
+
+ if (!fixup_exception_mc(regs))
+ return false;
+
+ set_thread_esr(0, esr);
+
+ arm64_force_sig_fault(sig, code, addr,
+ "Uncorrected hardware memory error in kernel-access\n");
+
+ return true;
+}
+
static int do_sea(unsigned long far, unsigned int esr, struct pt_regs *regs)
{
const struct fault_info *inf;
@@ -720,7 +743,9 @@ static int do_sea(unsigned long far, unsigned int esr, struct pt_regs *regs)
*/
siaddr = untagged_addr(far);
}
- arm64_notify_die(inf->name, regs, inf->sig, inf->code, siaddr, esr);
+
+ if (!arm64_do_kernel_sea(siaddr, esr, regs, inf->sig, inf->code))
+ arm64_notify_die(inf->name, regs, inf->sig, inf->code, siaddr, esr);
return 0;
}
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 546179418ffa..884661b29c17 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -174,6 +174,15 @@ copy_mc_to_kernel(void *dst, const void *src, size_t cnt)
}
#endif
+#ifndef copy_mc_to_user
+static inline unsigned long __must_check
+copy_mc_to_user(void *dst, const void *src, size_t cnt)
+{
+ check_object_size(src, cnt, true);
+ return raw_copy_to_user(dst, src, cnt);
+}
+#endif
+
static __always_inline void pagefault_disabled_inc(void)
{
current->pagefault_disabled++;
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH -next v4 2/7] arm64: fix types in copy_highpage()
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
In copy_highpage() the `kto` and `kfrom` local variables are pointers to
struct page, but these are used to hold arbitrary pointers to kernel memory
. Each call to page_address() returns a void pointer to memory associated
with the relevant page, and copy_page() expects void pointers to this
memory.
This inconsistency was introduced in commit 2563776b41c3 ("arm64: mte:
Tags-aware copy_{user_,}highpage() implementations") and while this
doesn't appear to be harmful in practice it is clearly wrong.
Correct this by making `kto` and `kfrom` void pointers.
Fixes: 2563776b41c3 ("arm64: mte: Tags-aware copy_{user_,}highpage() implementations")
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
arch/arm64/mm/copypage.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c
index b5447e53cd73..0dea80bf6de4 100644
--- a/arch/arm64/mm/copypage.c
+++ b/arch/arm64/mm/copypage.c
@@ -16,8 +16,8 @@
void copy_highpage(struct page *to, struct page *from)
{
- struct page *kto = page_address(to);
- struct page *kfrom = page_address(from);
+ void *kto = page_address(to);
+ void *kfrom = page_address(from);
copy_page(kto, kfrom);
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
* [PATCH -next v4 0/7]arm64: add machine check safe support
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
With the increase of memory capacity and density, the probability of
memory error increases. The increasing size and density of server RAM
in the data center and cloud have shown increased uncorrectable memory
errors.
Currently, the kernel has a mechanism to recover from hardware memory
errors. This patchset provides an new recovery mechanism.
For arm64, the hardware memory error handling is do_sea() which divided
into two cases:
1. The user state consumed the memory errors, the solution is kill the
user process and isolate the error page.
2. The kernel state consumed the memory errors, the solution is panic.
For case 2, Undifferentiated panic maybe not the optimal choice, it can be
handled better, in some scenes, we can avoid panic, such as uaccess, if the
uaccess fails due to memory error, only the user process will be affected,
kill the user process and isolate the user page with hardware memory errors
is a better choice.
This patchset can be divided into three parts:
1. Patch 0/1/4 - make some minor fixes to the associated code.
2. Patch 3 - arm64 add support for machine check safe framework.
3. Pathc 5/6/7 - arm64 add uaccess and cow to machine check safe.
Since V4:
1. According to Robin's suggestion, direct modify user_ldst and
user_ldp in asm-uaccess.h and modify mte.S.
2. Add new macro USER_MC in asm-uaccess.h, used in copy_from_user.S
and copy_to_user.S.
3. According to Robin's suggestion, using micro in copy_page_mc.S to
simplify code.
4. According to KeFeng's suggestion, modify powerpc code in patch1.
5. According to KeFeng's suggestion, modify mm/extable.c and some code
optimization.
Since V3:
1. According to Mark's suggestion, all uaccess can be recovered due to
memory error.
2. Scenario pagecache reading is also supported as part of uaccess
(copy_to_user()) and duplication code problem is also solved.
Thanks for Robin's suggestion.
3. According Mark's suggestion, update commit message of patch 2/5.
4. According Borisllav's suggestion, update commit message of patch 1/5.
Since V2:
1.Consistent with PPC/x86, Using CONFIG_ARCH_HAS_COPY_MC instead of
ARM64_UCE_KERNEL_RECOVERY.
2.Add two new scenes, cow and pagecache reading.
3.Fix two small bug(the first two patch).
V1 in here:
https://lore.kernel.org/lkml/20220323033705.3966643-1-tongtiangen@huawei.com/
Robin Murphy (1):
arm64: mte: Clean up user tag accessors
Tong Tiangen (6):
x86, powerpc: fix function define in copy_mc_to_user
arm64: fix types in copy_highpage()
arm64: add support for machine check error safe
arm64: add copy_{to, from}_user to machine check safe
arm64: add {get, put}_user to machine check safe
arm64: add cow to machine check safe
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/asm-extable.h | 33 +++++++++++
arch/arm64/include/asm/asm-uaccess.h | 15 +++--
arch/arm64/include/asm/extable.h | 1 +
arch/arm64/include/asm/page.h | 10 ++++
arch/arm64/include/asm/uaccess.h | 4 +-
arch/arm64/lib/Makefile | 2 +
arch/arm64/lib/copy_from_user.S | 18 +++---
arch/arm64/lib/copy_page_mc.S | 86 ++++++++++++++++++++++++++++
arch/arm64/lib/copy_to_user.S | 18 +++---
arch/arm64/lib/mte.S | 4 +-
arch/arm64/mm/copypage.c | 36 ++++++++++--
arch/arm64/mm/extable.c | 33 +++++++++++
arch/arm64/mm/fault.c | 27 ++++++++-
arch/powerpc/include/asm/uaccess.h | 1 +
arch/x86/include/asm/uaccess.h | 1 +
include/linux/highmem.h | 8 +++
include/linux/uaccess.h | 9 +++
mm/memory.c | 2 +-
19 files changed, 278 insertions(+), 31 deletions(-)
create mode 100644 arch/arm64/lib/copy_page_mc.S
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH -next v4 1/7] x86, powerpc: fix function define in copy_mc_to_user
From: Tong Tiangen @ 2022-04-20 3:04 UTC (permalink / raw)
To: Mark Rutland, James Morse, Andrew Morton, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Robin Murphy, Dave Hansen,
Catalin Marinas, Will Deacon, Alexander Viro, Michael Ellerman,
Benjamin Herrenschmidt, Paul Mackerras, x86, H . Peter Anvin
Cc: linuxppc-dev, linux-arm-kernel, linux-kernel, linux-mm,
Kefeng Wang, Xie XiuQi, Guohanjun, Tong Tiangen
In-Reply-To: <20220420030418.3189040-1-tongtiangen@huawei.com>
x86/powerpc has it's implementation of copy_mc_to_user but not use #define
to declare.
This may cause problems, for example, if other architectures open
CONFIG_ARCH_HAS_COPY_MC, but want to use copy_mc_to_user() outside the
architecture, the code add to include/linux/uaddess.h is as follows:
#ifndef copy_mc_to_user
static inline unsigned long __must_check
copy_mc_to_user(void *dst, const void *src, size_t cnt)
{
...
}
#endif
Then this definition will conflict with the implementation of x86/powerpc
and cause compilation errors as follow:
Fixes: ec6347bb4339 ("x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}()")
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
---
arch/powerpc/include/asm/uaccess.h | 1 +
arch/x86/include/asm/uaccess.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/powerpc/include/asm/uaccess.h b/arch/powerpc/include/asm/uaccess.h
index 9b82b38ff867..58dbe8e2e318 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -358,6 +358,7 @@ copy_mc_to_user(void __user *to, const void *from, unsigned long n)
return n;
}
+#define copy_mc_to_user copy_mc_to_user
#endif
extern long __copy_from_user_flushcache(void *dst, const void __user *src,
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index f78e2b3501a1..e18c5f098025 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -415,6 +415,7 @@ copy_mc_to_kernel(void *to, const void *from, unsigned len);
unsigned long __must_check
copy_mc_to_user(void *to, const void *from, unsigned len);
+#define copy_mc_to_user copy_mc_to_user
#endif
/*
--
2.25.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox