[PATCH 0/3] soc: Enable cache lockdown for HiSilicon L3 cache

[PATCH 0/3] soc: Enable cache lockdown for HiSilicon L3 cache

[Yushan Wang]

Provide support for L3 cache lockdown on HiSilicon L3 caches.

Cache lockdown means to make a memory region locked inside the L3 cache
for better access latency. The data stored in L3 cache will behave like
any other data (i.e. it still follows cache coherency protocol etc.)
except it won't be evicted unless explicitly asked to by deallocation.
This way a process may have an option to maintain QoS even with noisy
neighbors sharing the same L3 cache.

Ideally locked data will have stable low access latency despite high
background stress. It is also useful for scenarios that have especially
high cache miss penalty. However, while enhancing some processes,
reserving cache resource will raise the performance problem to other
processes running on the CPUs that share the L3 cache that carries
locked data, users should be careful to do so.

---

Changes since RFC v2:
- Fixed compile errors for x86 building.
- Link to v2: https://lore.kernel.org/all/20251217102357.1730573-1-wangyushan12@huawei.com/

Changes since RFC v1:
- Fixed compile errors of types specified in uapi header files.
- Changed ioctl numbers to avoid confliction.
- Link to v1: https://lore.kernel.org/all/20251125080542.3721829-1-wangyushan12@huawei.com/

Yushan Wang (3):
  soc cache: L3 cache driver for HiSilicon SoC
  soc cache: L3 cache lockdown support for HiSilicon SoC
  Documentation: soc cache: Add documentation to HiSilicon SoC cache

 Documentation/driver-api/hisi-soc-cache.rst   |  64 ++
 Documentation/driver-api/index.rst            |   1 +
 .../userspace-api/ioctl/ioctl-number.rst      |   1 +
 MAINTAINERS                                   |   7 +
 drivers/soc/hisilicon/Kconfig                 |  11 +
 drivers/soc/hisilicon/Makefile                |   2 +
 drivers/soc/hisilicon/hisi_soc_l3c.c          | 883 ++++++++++++++++++
 include/uapi/misc/hisi_l3c.h                  |  28 +
 8 files changed, 997 insertions(+)
 create mode 100644 Documentation/driver-api/hisi-soc-cache.rst
 create mode 100644 drivers/soc/hisilicon/hisi_soc_l3c.c
 create mode 100644 include/uapi/misc/hisi_l3c.h

-- 
2.33.0

[PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Yushan Wang]

The driver will create a file of `/dev/hisi_l3c` on init, mmap
operations to it will allocate a memory region that is guaranteed to be
placed in L3 cache.

The driver also provides unmap() to deallocated the locked memory.

The driver also provides an ioctl interface for user to get cache lock
information, such as lock restrictions and locked sizes.

Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
---
 .../userspace-api/ioctl/ioctl-number.rst      |   1 +
 MAINTAINERS                                   |   6 +
 drivers/soc/hisilicon/Kconfig                 |  11 +
 drivers/soc/hisilicon/Makefile                |   2 +
 drivers/soc/hisilicon/hisi_soc_l3c.c          | 357 ++++++++++++++++++
 include/uapi/misc/hisi_l3c.h                  |  28 ++
 6 files changed, 405 insertions(+)
 create mode 100644 drivers/soc/hisilicon/hisi_soc_l3c.c
 create mode 100644 include/uapi/misc/hisi_l3c.h

diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 7232b3544cec..439c5bcbfa94 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -387,6 +387,7 @@ Code  Seq#    Include File                                             Comments
                                                                        <mailto:linux-hyperv@vger.kernel.org>
 0xBA  00-0F  uapi/linux/liveupdate.h                                   Pasha Tatashin
                                                                        <mailto:pasha.tatashin@soleen.com>
+0xBB  all    uapi/misc/hisi_soc_cache.h                                HiSilicon SoC cache driver
 0xC0  00-0F  linux/usb/iowarrior.h
 0xCA  00-0F  uapi/misc/cxl.h                                           Dead since 6.15
 0xCA  10-2F  uapi/misc/ocxl.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 0efa8cc6775b..247df6e69c10 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11375,6 +11375,12 @@ F:	Documentation/ABI/testing/sysfs-devices-platform-kunpeng_hccs
 F:	drivers/soc/hisilicon/kunpeng_hccs.c
 F:	drivers/soc/hisilicon/kunpeng_hccs.h
 
+HISILICON SOC L3C DRIVER
+M:	Yushan Wang <wangyushan12@huawei.com>
+S:	Maintained
+F:	drivers/soc/hisilicon/hisi_soc_l3c.c
+F:	include/uapi/misc/hisi_l3c.h
+
 HISILICON LPC BUS DRIVER
 M:	Jay Fang <f.fangjian@huawei.com>
 S:	Maintained
diff --git a/drivers/soc/hisilicon/Kconfig b/drivers/soc/hisilicon/Kconfig
index 6d7c244d2e78..8f4202e2d8d9 100644
--- a/drivers/soc/hisilicon/Kconfig
+++ b/drivers/soc/hisilicon/Kconfig
@@ -21,4 +21,15 @@ config KUNPENG_HCCS
 	  health status and port information of HCCS, or reducing system
 	  power consumption on Kunpeng SoC.
 
+config HISI_SOC_L3C
+	bool "HiSilicon L3 Cache device driver"
+	depends on ACPI
+	depends on ARM64 || COMPILE_TEST
+	help
+	  This driver provides the functions to lock L3 cache entries from
+	  being evicted for better performance.
+
+	  This driver can be built as a module. If so, the module will be
+	  called hisi_soc_l3c.
+
 endmenu
diff --git a/drivers/soc/hisilicon/Makefile b/drivers/soc/hisilicon/Makefile
index 226e747e70d6..16ff2c73c4a5 100644
--- a/drivers/soc/hisilicon/Makefile
+++ b/drivers/soc/hisilicon/Makefile
@@ -1,2 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-$(CONFIG_KUNPENG_HCCS)	+= kunpeng_hccs.o
+
+obj-$(CONFIG_HISI_SOC_L3C)	+= hisi_soc_l3c.o
diff --git a/drivers/soc/hisilicon/hisi_soc_l3c.c b/drivers/soc/hisilicon/hisi_soc_l3c.c
new file mode 100644
index 000000000000..b6f6d5bdd4e5
--- /dev/null
+++ b/drivers/soc/hisilicon/hisi_soc_l3c.c
@@ -0,0 +1,357 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Driver for HiSilicon L3 cache.
+ *
+ * Copyright (c) 2025 HiSilicon Technologies Co., Ltd.
+ * Author: Yushan Wang <wangyushan12@huawei.com>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/cleanup.h>
+#include <linux/init.h>
+#include <linux/mm.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/spinlock_types.h>
+#include <linux/types.h>
+
+#include <uapi/misc/hisi_l3c.h>
+
+#define to_hisi_l3c(p) container_of((p), struct hisi_l3c, comp)
+
+/**
+ * struct hisi_soc_comp - Struct of HiSilicon SoC cache components.
+ *
+ * @node: list node of hisi_soc_comp_list.
+ * @ops: possible operations a component may perform.
+ * @affinity_mask: cpus that associate with this component.
+ * @private: component specific data.
+ */
+struct hisi_soc_comp {
+	struct list_head node;
+	struct hisi_soc_comp_ops *ops;
+	cpumask_t affinity_mask;
+	void *private;
+};
+
+/**
+ * struct hisi_soc_comp_ops - Callbacks for SoC cache drivers to handle
+ *			      operation requests.
+ *
+ * @do_lock: lock certain region of L3 cache from being evicted.
+ * @poll_lock_done: check if the lock operation has succeeded.
+ * @do_unlock: unlock the locked region of L3 cache back to normal.
+ * @poll_unlock_done: check if the unlock operation has succeeded.
+	      operation requests.
+ *
+ * Operations are decoupled into two phases so that framework does not have
+ * to wait for one operation to finish before calling the next when multiple
+ * hardwares onboard.
+ *
+ * Implementers must implement the functions in pairs.  Implementation should
+ * return -EBUSY when:
+ * - insufficient resources are available to perform the operation.
+ * - previously raised operation is not finished.
+ * - new operations (do_lock(), do_unlock() etc.) to the same address
+ *   before corresponding done functions being called.
+ */
+struct hisi_soc_comp_ops {
+	int (*do_lock)(struct hisi_soc_comp *comp, phys_addr_t addr, size_t size);
+	int (*poll_lock_done)(struct hisi_soc_comp *comp, phys_addr_t addr, size_t size);
+	int (*do_unlock)(struct hisi_soc_comp *comp, phys_addr_t addr);
+	int (*poll_unlock_done)(struct hisi_soc_comp *comp, phys_addr_t addr);
+};
+
+struct hisi_l3c_lock_region {
+	/* physical address of the arena allocated for aligned address */
+	unsigned long arena_start;
+	/* VMA region of locked memory for future release */
+	unsigned long vm_start;
+	unsigned long vm_end;
+	phys_addr_t addr;
+	size_t size;
+	/* Return value of cache lock call */
+	int status;
+	int cpu;
+};
+
+struct hisi_soc_comp_list {
+	struct list_head node;
+	/* protects list of HiSilicon SoC cache components */
+	spinlock_t lock;
+};
+
+static struct hisi_soc_comp_list l3c_devs;
+
+static int hisi_l3c_lock(int cpu, phys_addr_t addr, size_t size)
+{
+	struct hisi_soc_comp *comp;
+	int ret;
+
+	guard(spinlock)(&l3c_devs.lock);
+
+	/* When there is no instance onboard, no locked memory is available. */
+	if (list_empty(&l3c_devs.node))
+		return -ENOMEM;
+
+	/* Lock need to be performed on each channel of associated L3 cache. */
+	list_for_each_entry(comp, &l3c_devs.node, node) {
+		if (!cpumask_test_cpu(cpu, &comp->affinity_mask))
+			continue;
+		ret = comp->ops->do_lock(comp, addr, size);
+		if (ret)
+			return ret;
+	}
+
+	list_for_each_entry(comp, &l3c_devs.node, node) {
+		if (!cpumask_test_cpu(cpu, &comp->affinity_mask))
+			continue;
+		ret = comp->ops->poll_lock_done(comp, addr, size);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int hisi_l3c_unlock(int cpu, phys_addr_t addr)
+{
+	struct hisi_soc_comp *comp;
+	int ret;
+
+	guard(spinlock)(&l3c_devs.lock);
+
+	if (list_empty(&l3c_devs.node))
+		return -EINVAL;
+
+	/* Perform unlock on each channel of associated L3 cache. */
+	list_for_each_entry(comp, &l3c_devs.node, node) {
+		if (!cpumask_test_cpu(cpu, &comp->affinity_mask))
+			continue;
+		ret = comp->ops->do_unlock(comp, addr);
+		if (ret)
+			return ret;
+	}
+
+	list_for_each_entry(comp, &l3c_devs.node, node) {
+		if (!cpumask_test_cpu(cpu, &comp->affinity_mask))
+			continue;
+		ret = comp->ops->poll_unlock_done(comp, addr);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static void hisi_soc_comp_add(struct hisi_soc_comp *comp)
+{
+	guard(spinlock)(&l3c_devs.lock);
+	list_add_tail(&comp->node, &l3c_devs.node);
+}
+
+/* Null @comp means to delete all instances. */
+static int hisi_soc_comp_del(struct hisi_soc_comp *comp)
+{
+	struct hisi_soc_comp *entry, *tmp;
+
+	guard(spinlock)(&l3c_devs.lock);
+	list_for_each_entry_safe(entry, tmp, &l3c_devs.node, node) {
+		if (comp && comp != entry)
+			continue;
+
+		list_del(&entry->node);
+
+		/* Only continue to delete nodes when @comp is NULL */
+		if (comp)
+			break;
+	}
+
+	return 0;
+}
+
+static void hisi_l3c_vm_open(struct vm_area_struct *vma)
+{
+	struct hisi_l3c_lock_region *clr = vma->vm_private_data;
+
+	/*
+	 * Only perform cache lock when the vma passed in is created in
+	 * hisi_l3c_mmap.
+	 */
+	if (clr->vm_start != vma->vm_start || clr->vm_end != vma->vm_end)
+		return;
+
+	clr->status = hisi_l3c_lock(clr->cpu, clr->addr, clr->size);
+}
+
+static void hisi_l3c_vm_close(struct vm_area_struct *vma)
+{
+	struct hisi_l3c_lock_region *clr = vma->vm_private_data;
+	int order = get_order(clr->size);
+
+	/*
+	 * Only perform cache unlock when the vma passed in is created
+	 * in hisi_l3c_mmap.
+	 */
+	if (clr->vm_start != vma->vm_start || clr->vm_end != vma->vm_end)
+		return;
+
+	hisi_l3c_unlock(clr->cpu, clr->addr);
+
+	free_contig_range(PHYS_PFN(clr->addr), 1 << order);
+	kfree(clr);
+	vma->vm_private_data = NULL;
+}
+
+/* mremap operation is not supported for HiSilicon SoC cache. */
+static int hisi_l3c_vm_mremap(struct vm_area_struct *vma)
+{
+	struct hisi_l3c_lock_region *clr = vma->vm_private_data;
+
+	/*
+	 * vma region size will be changed as requested by mremap despite the
+	 * callback failure in this function.  Thus, change the vma region
+	 * stored in clr according to the parameters to verify if the pages
+	 * should be freed when unmapping.
+	 */
+	clr->vm_end = clr->vm_start + (vma->vm_end - vma->vm_start);
+	pr_err("mremap for HiSilicon SoC locked cache is not supported\n");
+
+	return -EOPNOTSUPP;
+}
+
+static int hisi_l3c_may_split(struct vm_area_struct *area, unsigned long addr)
+{
+	pr_err("HiSilicon SoC locked cache may not be split.\n");
+	return -EINVAL;
+}
+
+static const struct vm_operations_struct hisi_l3c_vm_ops = {
+	.open = hisi_l3c_vm_open,
+	.close = hisi_l3c_vm_close,
+	.may_split = hisi_l3c_may_split,
+	.mremap = hisi_l3c_vm_mremap,
+};
+
+static int hisi_l3c_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	unsigned long size = vma->vm_end - vma->vm_start;
+	int order = get_order(size);
+	unsigned long addr;
+	struct page *pg;
+	int ret;
+
+	struct hisi_l3c_lock_region *clr __free(kfree) = kzalloc(sizeof(*clr), GFP_KERNEL);
+	if (!clr)
+		return -ENOMEM;
+
+	/* Continuous physical memory is required for L3 cache lock. */
+	pg = alloc_contig_pages(1 << order, GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO,
+				cpu_to_node(smp_processor_id()), NULL);
+	if (!pg)
+		return -ENOMEM;
+
+	addr = page_to_phys(pg);
+	*clr = (struct hisi_l3c_lock_region) {
+		.addr = addr,
+		.size = size,
+		.cpu = smp_processor_id(),
+		/* vma should not be moved, store here for validation */
+		.vm_start = vma->vm_start,
+		.vm_end = vma->vm_end,
+	};
+
+	vm_flags_set(vma, VM_DONTCOPY | VM_DONTEXPAND);
+	vma->vm_ops = &hisi_l3c_vm_ops;
+	vma->vm_private_data = clr;
+
+	hisi_l3c_vm_ops.open(vma);
+	if (clr->status) {
+		ret = clr->status;
+		goto out_page;
+	}
+
+	ret = remap_pfn_range(vma, vma->vm_start, PFN_DOWN(addr), size,
+			      vma->vm_page_prot);
+	if (ret)
+		goto out_page;
+
+	/* Save clr from being freed when lock succeeds. */
+	vma->vm_private_data = no_free_ptr(clr);
+
+	return 0;
+
+out_page:
+	free_contig_range(PHYS_PFN(clr->addr), 1 << order);
+	return ret;
+}
+
+static int hisi_l3c_lock_restriction(unsigned long arg)
+{
+	void __user *uarg = (void __user *)arg;
+	int cpu = smp_processor_id();
+	struct hisi_soc_comp *comp;
+
+	if (list_empty(&l3c_devs.node))
+		return -ENODEV;
+
+	list_for_each_entry(comp, &l3c_devs.node, node) {
+		if (!cpumask_test_cpu(cpu, &comp->affinity_mask))
+			continue;
+
+		if (!comp->private)
+			return -ENOENT;
+
+		if (copy_to_user(uarg, comp->private, sizeof(struct hisi_l3c_lock_info)))
+			return -EFAULT;
+
+		return 0;
+	}
+
+	return -ENODEV;
+}
+
+static long hisi_l3c_ioctl(struct file *file, u32 cmd, unsigned long arg)
+{
+	switch (cmd) {
+	case HISI_L3C_LOCK_INFO:
+		return hisi_l3c_lock_restriction(arg);
+	default:
+		return -EINVAL;
+	}
+}
+
+static const struct file_operations l3c_dev_fops = {
+	.owner = THIS_MODULE,
+	.unlocked_ioctl = hisi_l3c_ioctl,
+	.mmap = hisi_l3c_mmap,
+};
+
+static struct miscdevice l3c_miscdev = {
+	.minor = MISC_DYNAMIC_MINOR,
+	.name = "hisi_l3c",
+	.fops = &l3c_dev_fops,
+	.mode = 0600,
+};
+
+static int __init hisi_l3c_init(void)
+{
+	spin_lock_init(&l3c_devs.lock);
+	INIT_LIST_HEAD(&l3c_devs.node);
+
+	return misc_register(&l3c_miscdev);
+}
+module_init(hisi_l3c_init);
+
+static void __exit hisi_l3c_exit(void)
+{
+	misc_deregister(&l3c_miscdev);
+	hisi_soc_comp_del(NULL);
+}
+module_exit(hisi_l3c_exit);
+
+MODULE_DESCRIPTION("Hisilicon L3 Cache Driver");
+MODULE_AUTHOR("Yushan Wang <wangyushan12@huawei.com>");
+MODULE_LICENSE("GPL");
diff --git a/include/uapi/misc/hisi_l3c.h b/include/uapi/misc/hisi_l3c.h
new file mode 100644
index 000000000000..6555be18aa1c
--- /dev/null
+++ b/include/uapi/misc/hisi_l3c.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later WITH Linux-syscall-note */
+/* Copyright (c) 2024 HiSilicon Technologies Co., Ltd. */
+#ifndef _UAPI_HISI_SOC_L3C_H
+#define _UAPI_HISI_SOC_L3C_H
+
+#include <linux/types.h>
+
+/* HISI_L3C_INFO: cache lock info for HiSilicon SoC */
+#define HISI_L3C_LOCK_INFO	_IOW(0xBB, 1, unsigned long)
+
+/**
+ * struct hisi_l3c_info - User data for hisi cache operates.
+ * @lock_region_num: available locked memory region on a L3C instance
+ * @lock_size: available size to be locked of the L3C instance.
+ * @address_alignment: if the L3C lock requires locked region physical start
+ *		       address to be aligned with the memory region size.
+ * @max_lock_size: maximum locked memory size on a L3C instance.
+ * @min_lock_size: minimum locked memory size on a L3C instance.
+ */
+struct hisi_l3c_lock_info {
+	__u32 lock_region_num;
+	__u64 lock_size;
+	__u8 address_alignment;
+	__u64 max_lock_size;
+	__u64 min_lock_size;
+};
+
+#endif
-- 
2.33.0

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Arnd Bergmann]

On Tue, Feb 3, 2026, at 17:18, Yushan Wang wrote:
> The driver will create a file of `/dev/hisi_l3c` on init, mmap
> operations to it will allocate a memory region that is guaranteed to be
> placed in L3 cache.
>
> The driver also provides unmap() to deallocated the locked memory.
>
> The driver also provides an ioctl interface for user to get cache lock
> information, such as lock restrictions and locked sizes.
>
> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>

Hi Yushan,

Thanks for your submission. Since we are in the last week of
the merge window, this is not going to be linux-7.0 material,
but I'll have a quick look for now.

>  .../userspace-api/ioctl/ioctl-number.rst      |   1 +
>  MAINTAINERS                                   |   6 +
>  drivers/soc/hisilicon/Kconfig                 |  11 +
>  drivers/soc/hisilicon/Makefile                |   2 +
>  drivers/soc/hisilicon/hisi_soc_l3c.c          | 357 ++++++++++++++++++
>  include/uapi/misc/hisi_l3c.h                  |  28 ++

I don't think this should be in drivers/soc/, since I want
to reserve that for internal drivers without a user visible
interface other than the soc_device information. (yes, there
are a few historic counterexamples)

I also don't think this should be a hilicon specific interface,
if possible. The functionality is not that unusual in the end.

We had similar concepts using the numactl system calls in the
part, but I don't think we should do that here because you may
need the numa interfaces for other purposes as well, and it
may be confusing to existing callers.

Having a generic madvise() based interface would be great,
not sure if hardware support for that is common enough for that.

> +	/* Continuous physical memory is required for L3 cache lock. */
> +	pg = alloc_contig_pages(1 << order, GFP_KERNEL | __GFP_NOWARN | 
> __GFP_ZERO,
> +				cpu_to_node(smp_processor_id()), NULL);

Since this is a user allocation, should that be GFP_USER instead
of GFP_KERNEL?

> +/* HISI_L3C_INFO: cache lock info for HiSilicon SoC */
> +#define HISI_L3C_LOCK_INFO	_IOW(0xBB, 1, unsigned long)

The specification here looks wrong, please see
Documentation/driver-api/ioctl.rst

I think for your implementation it should be

#define HISI_L3C_LOCK_INFO	_IOR(0xBB, 1, hisi_l3c_lock_info)

> +struct hisi_l3c_lock_info {
> +	__u32 lock_region_num;
> +	__u64 lock_size;
> +	__u8 address_alignment;
> +	__u64 max_lock_size;
> +	__u64 min_lock_size;
> +};

You are leaking kernel data because of the padding in this structure,
please rearrange the members to avoid padding.

It may be better to use a different interface instead of ioctl(),
possibly exporting global data in sysfs.

       Arnd

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[wangyushan]

On 2/4/2026 1:19 AM, Arnd Bergmann wrote:
> On Tue, Feb 3, 2026, at 17:18, Yushan Wang wrote:
>> The driver will create a file of `/dev/hisi_l3c` on init, mmap
>> operations to it will allocate a memory region that is guaranteed to be
>> placed in L3 cache.
>>
>> The driver also provides unmap() to deallocated the locked memory.
>>
>> The driver also provides an ioctl interface for user to get cache lock
>> information, such as lock restrictions and locked sizes.
>>
>> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
> Hi Yushan,
>
> Thanks for your submission. Since we are in the last week of
> the merge window, this is not going to be linux-7.0 material,
> but I'll have a quick look for now.

Many thanks for review!
>>  .../userspace-api/ioctl/ioctl-number.rst      |   1 +
>>  MAINTAINERS                                   |   6 +
>>  drivers/soc/hisilicon/Kconfig                 |  11 +
>>  drivers/soc/hisilicon/Makefile                |   2 +
>>  drivers/soc/hisilicon/hisi_soc_l3c.c          | 357 ++++++++++++++++++
>>  include/uapi/misc/hisi_l3c.h                  |  28 ++
> I don't think this should be in drivers/soc/, since I want
> to reserve that for internal drivers without a user visible
> interface other than the soc_device information. (yes, there
> are a few historic counterexamples)
>
> I also don't think this should be a hilicon specific interface,
> if possible. The functionality is not that unusual in the end.

I hesitated about the directory as well, exporting a user aware
interface directly from driver may not be a good practice and I
am hoping for advice :)
The driver itself doesn't provide much more than a mere register configuration which is platform specific, what do you think about a kernel space interface somewhere else which implemented here?
> We had similar concepts using the numactl system calls in the
> part, but I don't think we should do that here because you may
> need the numa interfaces for other purposes as well, and it
> may be confusing to existing callers.
>
> Having a generic madvise() based interface would be great,
> not sure if hardware support for that is common enough for that.
As above, cache lock could be an option for optimization of numactl or madvise(), maybe a more generic interface in kernel space which is speculatively called by other infrastructure is better, and of course extra support of the infrastructures is needed. I can try to propose an interface and maybe an example of the caller in the next version.
>> +	/* Continuous physical memory is required for L3 cache lock. */
>> +	pg = alloc_contig_pages(1 << order, GFP_KERNEL | __GFP_NOWARN | 
>> __GFP_ZERO,
>> +				cpu_to_node(smp_processor_id()), NULL);
> Since this is a user allocation, should that be GFP_USER instead
> of GFP_KERNEL?

Yes, it should be GFP_USER in this version since it is used in userspace.
>> +/* HISI_L3C_INFO: cache lock info for HiSilicon SoC */
>> +#define HISI_L3C_LOCK_INFO	_IOW(0xBB, 1, unsigned long)
> The specification here looks wrong, please see
> Documentation/driver-api/ioctl.rst
>
> I think for your implementation it should be
>
> #define HISI_L3C_LOCK_INFO	_IOR(0xBB, 1, hisi_l3c_lock_info)

Sorry, I will correct that in the next version.
>> +struct hisi_l3c_lock_info {
>> +	__u32 lock_region_num;
>> +	__u64 lock_size;
>> +	__u8 address_alignment;
>> +	__u64 max_lock_size;
>> +	__u64 min_lock_size;
>> +};
> You are leaking kernel data because of the padding in this structure,
> please rearrange the members to avoid padding.
>
> It may be better to use a different interface instead of ioctl(),
> possibly exporting global data in sysfs.

Yes, information through sysfs should do the job.
I will fix that in the next version.

>        Arnd

Thanks,
Yushan

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Krzysztof Kozlowski]

On 03/02/2026 18:19, Arnd Bergmann wrote:
> On Tue, Feb 3, 2026, at 17:18, Yushan Wang wrote:
>> The driver will create a file of `/dev/hisi_l3c` on init, mmap
>> operations to it will allocate a memory region that is guaranteed to be
>> placed in L3 cache.
>>
>> The driver also provides unmap() to deallocated the locked memory.
>>
>> The driver also provides an ioctl interface for user to get cache lock
>> information, such as lock restrictions and locked sizes.
>>
>> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
> 
> Hi Yushan,
> 
> Thanks for your submission. Since we are in the last week of
> the merge window, this is not going to be linux-7.0 material,
> but I'll have a quick look for now.


To be clear - this is a v3 but with removed previous history...

Previous version:
https://lore.kernel.org/all/20251217102357.1730573-2-wangyushan12@huawei.com/

Or even v4?

https://lore.kernel.org/all/20250122065803.3363926-2-wangyushan12@huawei.com/

Yushan, please start versioning your patches correctly. Use b4 or git
format-patch -vx

Otherwise, please explain us how can we compare it with `b4 diff` with
previous version?

Sending something AGAIN as v1 ignoring entire previous submission is
clear no go. Like you are trying till it succeeds. Negative review?
Let's try from v1 this time...

This is not correct and it should not be my task to find your previous
discussions and decipher this v1.


Best regards,
Krzysztof

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[wangyushan]

On 2/5/2026 5:37 PM, Krzysztof Kozlowski wrote:
> On 03/02/2026 18:19, Arnd Bergmann wrote:
>> On Tue, Feb 3, 2026, at 17:18, Yushan Wang wrote:
>>> The driver will create a file of `/dev/hisi_l3c` on init, mmap
>>> operations to it will allocate a memory region that is guaranteed to be
>>> placed in L3 cache.
>>>
>>> The driver also provides unmap() to deallocated the locked memory.
>>>
>>> The driver also provides an ioctl interface for user to get cache lock
>>> information, such as lock restrictions and locked sizes.
>>>
>>> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
>>
>> Hi Yushan,
>>
>> Thanks for your submission. Since we are in the last week of
>> the merge window, this is not going to be linux-7.0 material,
>> but I'll have a quick look for now.
>
>
> To be clear - this is a v3 but with removed previous history...
>
> Previous version:
> https://lore.kernel.org/all/20251217102357.1730573-2-wangyushan12@huawei.com/
>
> Or even v4?
>
> https://lore.kernel.org/all/20250122065803.3363926-2-wangyushan12@huawei.com/
>
> Yushan, please start versioning your patches correctly. Use b4 or git
> format-patch -vx

Hi Krzysztof,

Sorry about the confusing versions, the complete history is as below:

Link to v1: https://lore.kernel.org/all/20250107132907.3521574-1-wangyushan12@huawei.com

Link to v2: https://lore.kernel.org/all/20250122065803.3363926-1-wangyushan12@huawei.com/

Link to RFC v1: https://lore.kernel.org/all/20251125080542.3721829-1-wangyushan12@huawei.com/

Link to RFC v2: https://lore.kernel.org/all/20251217102357.1730573-1-wangyushan12@huawei.com/

Link to v1 again (this message): https://lore.kernel.org/all/20260203161843.649417-1-wangyushan12@huawei.com/

>
> Otherwise, please explain us how can we compare it with `b4 diff` with
> previous version?
>
> Sending something AGAIN as v1 ignoring entire previous submission is
> clear no go. Like you are trying till it succeeds. Negative review?
> Let's try from v1 this time...
>
> This is not correct and it should not be my task to find your previous
> discussions and decipher this v1.

I did spin 2 versions to mainline as the actual v1, the thread was quiet.
Then I made a major refactor to it and sent it as RFC, the thread went
quiet again but some compile check issues popped up. I spinned 2
versions of RFC for the compile issues and removed RFC in this version
since no strong objection showed up.

Apologize that I broke the rules and any inconvenience caused by it.
As there's little discussion in previous patches, is it OK that we start
here as v1?

Anyway I will include the whole history in the coverletter to prevent
more confusion.

>
>
> Best regards,
> Krzysztof
>

Thanks,
Yushan

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Krzysztof Kozlowski]

On 05/02/2026 12:19, wangyushan wrote:
> 
> On 2/5/2026 5:37 PM, Krzysztof Kozlowski wrote:
>> On 03/02/2026 18:19, Arnd Bergmann wrote:
>>> On Tue, Feb 3, 2026, at 17:18, Yushan Wang wrote:
>>>> The driver will create a file of `/dev/hisi_l3c` on init, mmap
>>>> operations to it will allocate a memory region that is guaranteed to be
>>>> placed in L3 cache.
>>>>
>>>> The driver also provides unmap() to deallocated the locked memory.
>>>>
>>>> The driver also provides an ioctl interface for user to get cache lock
>>>> information, such as lock restrictions and locked sizes.
>>>>
>>>> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
>>>
>>> Hi Yushan,
>>>
>>> Thanks for your submission. Since we are in the last week of
>>> the merge window, this is not going to be linux-7.0 material,
>>> but I'll have a quick look for now.
>>
>>
>> To be clear - this is a v3 but with removed previous history...
>>
>> Previous version:
>> https://lore.kernel.org/all/20251217102357.1730573-2-wangyushan12@huawei.com/
>>
>> Or even v4?
>>
>> https://lore.kernel.org/all/20250122065803.3363926-2-wangyushan12@huawei.com/
>>
>> Yushan, please start versioning your patches correctly. Use b4 or git
>> format-patch -vx
> 
> Hi Krzysztof,
> 
> Sorry about the confusing versions, the complete history is as below:
> 
> Link to v1: https://lore.kernel.org/all/20250107132907.3521574-1-wangyushan12@huawei.com
> 
> Link to v2: https://lore.kernel.org/all/20250122065803.3363926-1-wangyushan12@huawei.com/
> 
> Link to RFC v1: https://lore.kernel.org/all/20251125080542.3721829-1-wangyushan12@huawei.com/
> 
> Link to RFC v2: https://lore.kernel.org/all/20251217102357.1730573-1-wangyushan12@huawei.com/
> 
> Link to v1 again (this message): https://lore.kernel.org/all/20260203161843.649417-1-wangyushan12@huawei.com/
> 
>>
>> Otherwise, please explain us how can we compare it with `b4 diff` with
>> previous version?
>>
>> Sending something AGAIN as v1 ignoring entire previous submission is
>> clear no go. Like you are trying till it succeeds. Negative review?
>> Let's try from v1 this time...
>>
>> This is not correct and it should not be my task to find your previous
>> discussions and decipher this v1.
> 
> I did spin 2 versions to mainline as the actual v1, the thread was quiet.
> Then I made a major refactor to it and sent it as RFC, the thread went
> quiet again but some compile check issues popped up. I spinned 2
> versions of RFC for the compile issues and removed RFC in this version
> since no strong objection showed up.
> 
> Apologize that I broke the rules and any inconvenience caused by it.
> As there's little discussion in previous patches, is it OK that we start
> here as v1?

No, it is not okay. Your patchset continues and entire previous feedback
and history is important. Otherwise why would I review this if I can as
well ignore it and wait for next year you sending another v1?

Best regards,
Krzysztof

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[wangyushan]

On 2/5/2026 7:23 PM, Krzysztof Kozlowski wrote:
> On 05/02/2026 12:19, wangyushan wrote:
>>
>> On 2/5/2026 5:37 PM, Krzysztof Kozlowski wrote:
>>> On 03/02/2026 18:19, Arnd Bergmann wrote:
>>>> On Tue, Feb 3, 2026, at 17:18, Yushan Wang wrote:
>>>>> The driver will create a file of `/dev/hisi_l3c` on init, mmap
>>>>> operations to it will allocate a memory region that is guaranteed to be
>>>>> placed in L3 cache.
>>>>>
>>>>> The driver also provides unmap() to deallocated the locked memory.
>>>>>
>>>>> The driver also provides an ioctl interface for user to get cache lock
>>>>> information, such as lock restrictions and locked sizes.
>>>>>
>>>>> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
>>>>
>>>> Hi Yushan,
>>>>
>>>> Thanks for your submission. Since we are in the last week of
>>>> the merge window, this is not going to be linux-7.0 material,
>>>> but I'll have a quick look for now.
>>>
>>>
>>> To be clear - this is a v3 but with removed previous history...
>>>
>>> Previous version:
>>> https://lore.kernel.org/all/20251217102357.1730573-2-wangyushan12@huawei.com/
>>>
>>> Or even v4?
>>>
>>> https://lore.kernel.org/all/20250122065803.3363926-2-wangyushan12@huawei.com/
>>>
>>> Yushan, please start versioning your patches correctly. Use b4 or git
>>> format-patch -vx
>>
>> Hi Krzysztof,
>>
>> Sorry about the confusing versions, the complete history is as below:
>>
>> Link to v1: https://lore.kernel.org/all/20250107132907.3521574-1-wangyushan12@huawei.com
>>
>> Link to v2: https://lore.kernel.org/all/20250122065803.3363926-1-wangyushan12@huawei.com/
>>
>> Link to RFC v1: https://lore.kernel.org/all/20251125080542.3721829-1-wangyushan12@huawei.com/
>>
>> Link to RFC v2: https://lore.kernel.org/all/20251217102357.1730573-1-wangyushan12@huawei.com/
>>
>> Link to v1 again (this message): https://lore.kernel.org/all/20260203161843.649417-1-wangyushan12@huawei.com/
>>
>>>
>>> Otherwise, please explain us how can we compare it with `b4 diff` with
>>> previous version?
>>>
>>> Sending something AGAIN as v1 ignoring entire previous submission is
>>> clear no go. Like you are trying till it succeeds. Negative review?
>>> Let's try from v1 this time...
>>>
>>> This is not correct and it should not be my task to find your previous
>>> discussions and decipher this v1.
>>
>> I did spin 2 versions to mainline as the actual v1, the thread was quiet.
>> Then I made a major refactor to it and sent it as RFC, the thread went
>> quiet again but some compile check issues popped up. I spinned 2
>> versions of RFC for the compile issues and removed RFC in this version
>> since no strong objection showed up.
>>
>> Apologize that I broke the rules and any inconvenience caused by it.
>> As there's little discussion in previous patches, is it OK that we start
>> here as v1?
>
> No, it is not okay. Your patchset continues and entire previous feedback
> and history is important. Otherwise why would I review this if I can as
> well ignore it and wait for next year you sending another v1?

Sorry, I will correct the version numbers in next versions, with whole
history and explanations about the mess.

Sincerely apologies, I won't hide the versions again.

Yushan

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Linus Walleij]

Hi Yushan,

thanks for your patch!

On Tue, Feb 3, 2026 at 5:18 PM Yushan Wang <wangyushan12@huawei.com> wrote:
>
> The driver will create a file of `/dev/hisi_l3c` on init, mmap
> operations to it will allocate a memory region that is guaranteed to be
> placed in L3 cache.
>
> The driver also provides unmap() to deallocated the locked memory.
>
> The driver also provides an ioctl interface for user to get cache lock
> information, such as lock restrictions and locked sizes.
>
> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>

The commit message does not say *why* you are doing this?

> +config HISI_SOC_L3C
> +       bool "HiSilicon L3 Cache device driver"
> +       depends on ACPI
> +       depends on ARM64 || COMPILE_TEST
> +       help
> +         This driver provides the functions to lock L3 cache entries from
> +         being evicted for better performance.

Here is the reason though.

Things like this need to be CC to linux-mm@vger.kernel.org.

I don't see why userspace would be so well informed as to make decisions
about what should be locked in the L3 cache and not?

I see the memory hierarchy as any other hardware: a resource that is
allocated and arbitrated by the kernel.

The MM subsytem knows which memory is most cache hot.
Especially when you use DAMON DAMOS, which has the sole
purpose of executing actions like that. Here is a good YouTube.
https://www.youtube.com/watch?v=xKJO4kLTHOI

Shouldn't the MM subsystem be in charge of determining, locking
down and freeing up hot regions in L3 cache?

This looks more like userspace is going to determine that but
how exactly? By running DAMON? Then it's better to keep the
whole mechanism in the kernel where it belongs and let the
MM subsystem adapt locked L3 cache to the usage patterns.

Yours,
Linus Walleij

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[wangyushan]

On 2/4/2026 8:10 AM, Linus Walleij wrote:
> Hi Yushan,
>
> thanks for your patch!

Thanks for review!

> On Tue, Feb 3, 2026 at 5:18 PM Yushan Wang <wangyushan12@huawei.com> wrote:
>> The driver will create a file of `/dev/hisi_l3c` on init, mmap
>> operations to it will allocate a memory region that is guaranteed to be
>> placed in L3 cache.
>>
>> The driver also provides unmap() to deallocated the locked memory.
>>
>> The driver also provides an ioctl interface for user to get cache lock
>> information, such as lock restrictions and locked sizes.
>>
>> Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
> The commit message does not say *why* you are doing this?
>> +config HISI_SOC_L3C
>> +       bool "HiSilicon L3 Cache device driver"
>> +       depends on ACPI
>> +       depends on ARM64 || COMPILE_TEST
>> +       help
>> +         This driver provides the functions to lock L3 cache entries from
>> +         being evicted for better performance.
> Here is the reason though.

Sorry, I will include this into the commit message.
> Things like this need to be CC to linux-mm@vger.kernel.org.
>
> I don't see why userspace would be so well informed as to make decisions
> about what should be locked in the L3 cache and not?

This question is actually: should it be kernel or user space
application to decide if a cache lock should be applied?

Maybe the ideal situation is that this capability should be reserved into kernel space as a vendor specific optimization option. With the lack of knowledge of memory interleave etc the best move of an application might be allocate cache lock as much as possible.
> I see the memory hierarchy as any other hardware: a resource that is
> allocated and arbitrated by the kernel.
>
> The MM subsytem knows which memory is most cache hot.
> Especially when you use DAMON DAMOS, which has the sole
> purpose of executing actions like that. Here is a good YouTube.
> https://www.youtube.com/watch?v=xKJO4kLTHOI
>
> Shouldn't the MM subsystem be in charge of determining, locking
> down and freeing up hot regions in L3 cache?
Thanks for the link, I will see if there's any chance this can
cooperate with DAMON.

Gaps still exists here because DAMON operate with pages, but cache
works with cachelines, though the cache lock here supports cache
lock size larger than a page.
> This looks more like userspace is going to determine that but
> how exactly? By running DAMON? Then it's better to keep the
> whole mechanism in the kernel where it belongs and let the
> MM subsystem adapt locked L3 cache to the usage patterns.
Currently the patchset simply trusts that user knows well what
he is doing, which might not be good enough.

I will try to see if this could work with DAMON or madvice()
maybe :)

> Yours,
> Linus Walleij

Thanks,
Yushan

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Linus Walleij]

On Wed, Feb 4, 2026 at 10:53 AM wangyushan <wangyushan12@huawei.com> wrote:

> > I don't see why userspace would be so well informed as to make decisions
> > about what should be locked in the L3 cache and not?
>
> This question is actually: should it be kernel or user space
> application to decide if a cache lock should be applied?
>
> Maybe the ideal situation is that this capability should be reserved into kernel
> space as a vendor specific optimization option. With the lack of knowledge
> of memory interleave etc the best move of an application might be allocate
> cache lock as much as possible.

If it is a vendor-specific optimization that has no generic applicability
outside of this specific system, dependent on a specific userspace
that only exist on this system, what is the value for the generic
kernel to carry and maintain this code?

In that case maybe the code should be maintained outside of the
mainline kernel tree.

What we want to see as maintainers are things that are reusable
across several systems.

Integrating this with DAMOS in a generic way is what will help the
next silicon that comes down the road.

I have already seen similar things from Fujitsu (IIRC). We need this
mechanism to be kernel-driven and generic, not custom and
system-specific, least of all driven from userspace by sysfs.

Yours,
Linus Walleij

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Jonathan Cameron]

On Wed, 4 Feb 2026 01:10:01 +0100
Linus Walleij <linusw@kernel.org> wrote:

> Hi Yushan,
> 
> thanks for your patch!
> 
> On Tue, Feb 3, 2026 at 5:18 PM Yushan Wang <wangyushan12@huawei.com> wrote:
> >
> > The driver will create a file of `/dev/hisi_l3c` on init, mmap
> > operations to it will allocate a memory region that is guaranteed to be
> > placed in L3 cache.
> >
> > The driver also provides unmap() to deallocated the locked memory.
> >
> > The driver also provides an ioctl interface for user to get cache lock
> > information, such as lock restrictions and locked sizes.
> >
> > Signed-off-by: Yushan Wang <wangyushan12@huawei.com>  
> 
> The commit message does not say *why* you are doing this?
> 
> > +config HISI_SOC_L3C
> > +       bool "HiSilicon L3 Cache device driver"
> > +       depends on ACPI
> > +       depends on ARM64 || COMPILE_TEST
> > +       help
> > +         This driver provides the functions to lock L3 cache entries from
> > +         being evicted for better performance.  
> 
> Here is the reason though.
> 
> Things like this need to be CC to linux-mm@vger.kernel.org.
> 
> I don't see why userspace would be so well informed as to make decisions
> about what should be locked in the L3 cache and not?
> 
> I see the memory hierarchy as any other hardware: a resource that is
> allocated and arbitrated by the kernel.
> 
> The MM subsytem knows which memory is most cache hot.
> Especially when you use DAMON DAMOS, which has the sole
> purpose of executing actions like that. Here is a good YouTube.
> https://www.youtube.com/watch?v=xKJO4kLTHOI
Hi Linus,

This typically isn't about cache hot.  It it were, the data would
be in the cache without this. It's about ensuring something that would
otherwise unlikely to be there is in the cache.

Normally that's a latency critical region.  In general the kernel
has no chance of figuring out what those are ahead of time, only
userspace can know (based on profiling etc) that is per workload.
The first hit matters in these use cases and it's not something
the prefetchers can help with.

The only thing we could do if this was in kernel would be to
have userspace pass some hints and then let the kernel actually
kick off the process. That just boils down to using a different
interface to do what this driver is doing (and that's the conversaion
this series is trying to get going)  It's a finite resource
and you absolutely need userspace to be able to tell if it
got what it asked for or not.

Damon might be useful for that preanalysis though but it can't do
anything for the infrequent extremely latency sensitive accesses.
Normally this is fleet wide stuff based on intensive benchmarking
of a few nodes.  Same sort of approach as the original warehouse
scale computing paper on tuning zswap capacity across a fleet.
Its an extreme form of profile guided optimization (and not
currently automatic I think?). If we are putting code in this
locked region, the program has been carefully recompiled / linked
to group the critical parts so that we can use the minimum number
of these locked regions. Data is a little simpler.

It's kind of similar to resctl but at a sub process granularity.

> 
> Shouldn't the MM subsystem be in charge of determining, locking
> down and freeing up hot regions in L3 cache?
> 
> This looks more like userspace is going to determine that but
> how exactly? By running DAMON? Then it's better to keep the
> whole mechanism in the kernel where it belongs and let the
> MM subsystem adapt locked L3 cache to the usage patterns.

I haven't yet come up with any plausible scheme by which the MM
subsystem could do this.

I think what we need here Yushan, is more detail on end to end
use cases for this.  Some examples etc as clearer motivation.

Jonathan

> 
> Yours,
> Linus Walleij
> 

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Jonathan Cameron]

Fixed linux-mm address that got added a few emails back.

On Wed, 4 Feb 2026 13:40:20 +0000
Jonathan Cameron <jonathan.cameron@huawei.com> wrote:

> On Wed, 4 Feb 2026 01:10:01 +0100
> Linus Walleij <linusw@kernel.org> wrote:
> 
> > Hi Yushan,
> > 
> > thanks for your patch!
> > 
> > On Tue, Feb 3, 2026 at 5:18 PM Yushan Wang <wangyushan12@huawei.com> wrote:  
> > >
> > > The driver will create a file of `/dev/hisi_l3c` on init, mmap
> > > operations to it will allocate a memory region that is guaranteed to be
> > > placed in L3 cache.
> > >
> > > The driver also provides unmap() to deallocated the locked memory.
> > >
> > > The driver also provides an ioctl interface for user to get cache lock
> > > information, such as lock restrictions and locked sizes.
> > >
> > > Signed-off-by: Yushan Wang <wangyushan12@huawei.com>    
> > 
> > The commit message does not say *why* you are doing this?
> >   
> > > +config HISI_SOC_L3C
> > > +       bool "HiSilicon L3 Cache device driver"
> > > +       depends on ACPI
> > > +       depends on ARM64 || COMPILE_TEST
> > > +       help
> > > +         This driver provides the functions to lock L3 cache entries from
> > > +         being evicted for better performance.    
> > 
> > Here is the reason though.
> > 
> > Things like this need to be CC to linux-mm@vger.kernel.org.
> > 
> > I don't see why userspace would be so well informed as to make decisions
> > about what should be locked in the L3 cache and not?
> > 
> > I see the memory hierarchy as any other hardware: a resource that is
> > allocated and arbitrated by the kernel.
> > 
> > The MM subsytem knows which memory is most cache hot.
> > Especially when you use DAMON DAMOS, which has the sole
> > purpose of executing actions like that. Here is a good YouTube.
> > https://www.youtube.com/watch?v=xKJO4kLTHOI  
> Hi Linus,
> 
> This typically isn't about cache hot.  It it were, the data would
> be in the cache without this. It's about ensuring something that would
> otherwise unlikely to be there is in the cache.
> 
> Normally that's a latency critical region.  In general the kernel
> has no chance of figuring out what those are ahead of time, only
> userspace can know (based on profiling etc) that is per workload.
> The first hit matters in these use cases and it's not something
> the prefetchers can help with.
> 
> The only thing we could do if this was in kernel would be to
> have userspace pass some hints and then let the kernel actually
> kick off the process. That just boils down to using a different
> interface to do what this driver is doing (and that's the conversaion
> this series is trying to get going)  It's a finite resource
> and you absolutely need userspace to be able to tell if it
> got what it asked for or not.
> 
> Damon might be useful for that preanalysis though but it can't do
> anything for the infrequent extremely latency sensitive accesses.
> Normally this is fleet wide stuff based on intensive benchmarking
> of a few nodes.  Same sort of approach as the original warehouse
> scale computing paper on tuning zswap capacity across a fleet.
> Its an extreme form of profile guided optimization (and not
> currently automatic I think?). If we are putting code in this
> locked region, the program has been carefully recompiled / linked
> to group the critical parts so that we can use the minimum number
> of these locked regions. Data is a little simpler.
> 
> It's kind of similar to resctl but at a sub process granularity.
> 
> > 
> > Shouldn't the MM subsystem be in charge of determining, locking
> > down and freeing up hot regions in L3 cache?
> > 
> > This looks more like userspace is going to determine that but
> > how exactly? By running DAMON? Then it's better to keep the
> > whole mechanism in the kernel where it belongs and let the
> > MM subsystem adapt locked L3 cache to the usage patterns.  
> 
> I haven't yet come up with any plausible scheme by which the MM
> subsystem could do this.
> 
> I think what we need here Yushan, is more detail on end to end
> use cases for this.  Some examples etc as clearer motivation.
> 
> Jonathan
> 
> > 
> > Yours,
> > Linus Walleij
> >   
> 

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[SeongJae Park]

On Wed, 4 Feb 2026 13:44:47 +0000 Jonathan Cameron <jonathan.cameron@huawei.com> wrote:

> 
> Fixed linux-mm address that got added a few emails back.
> 
> On Wed, 4 Feb 2026 13:40:20 +0000
> Jonathan Cameron <jonathan.cameron@huawei.com> wrote:
> 
> > On Wed, 4 Feb 2026 01:10:01 +0100
> > Linus Walleij <linusw@kernel.org> wrote:
> > 
> > > Hi Yushan,
> > > 
> > > thanks for your patch!
> > > 
> > > On Tue, Feb 3, 2026 at 5:18 PM Yushan Wang <wangyushan12@huawei.com> wrote:  
> > > >
> > > > The driver will create a file of `/dev/hisi_l3c` on init, mmap
> > > > operations to it will allocate a memory region that is guaranteed to be
> > > > placed in L3 cache.
> > > >
> > > > The driver also provides unmap() to deallocated the locked memory.
> > > >
> > > > The driver also provides an ioctl interface for user to get cache lock
> > > > information, such as lock restrictions and locked sizes.
> > > >
> > > > Signed-off-by: Yushan Wang <wangyushan12@huawei.com>    
> > > 
> > > The commit message does not say *why* you are doing this?
> > >   
> > > > +config HISI_SOC_L3C
> > > > +       bool "HiSilicon L3 Cache device driver"
> > > > +       depends on ACPI
> > > > +       depends on ARM64 || COMPILE_TEST
> > > > +       help
> > > > +         This driver provides the functions to lock L3 cache entries from
> > > > +         being evicted for better performance.    
> > > 
> > > Here is the reason though.
> > > 
> > > Things like this need to be CC to linux-mm@vger.kernel.org.
> > > 
> > > I don't see why userspace would be so well informed as to make decisions
> > > about what should be locked in the L3 cache and not?
> > > 
> > > I see the memory hierarchy as any other hardware: a resource that is
> > > allocated and arbitrated by the kernel.
> > > 
> > > The MM subsytem knows which memory is most cache hot.
> > > Especially when you use DAMON DAMOS, which has the sole
> > > purpose of executing actions like that. Here is a good YouTube.
> > > https://www.youtube.com/watch?v=xKJO4kLTHOI  

Thank you for Cc-ing me, Linus.

> > Hi Linus,
> > 
> > This typically isn't about cache hot.  It it were, the data would
> > be in the cache without this. It's about ensuring something that would
> > otherwise unlikely to be there is in the cache.
> > 
> > Normally that's a latency critical region.  In general the kernel
> > has no chance of figuring out what those are ahead of time, only
> > userspace can know (based on profiling etc) that is per workload.
> > The first hit matters in these use cases and it's not something
> > the prefetchers can help with.
> > 
> > The only thing we could do if this was in kernel would be to
> > have userspace pass some hints and then let the kernel actually
> > kick off the process. That just boils down to using a different
> > interface to do what this driver is doing (and that's the conversaion
> > this series is trying to get going)  It's a finite resource
> > and you absolutely need userspace to be able to tell if it
> > got what it asked for or not.

And thank you for clarifying, Jonathan.

> > 
> > Damon might be useful for that preanalysis though but it can't do
> > anything for the infrequent extremely latency sensitive accesses.

I also find no good idea to let DAMON help in this scenario.

If I have to make a brain storming idea off the top of my humble head, though.
Maybe we can ask DAMON to monitor address ranges that assumed to have the
latency sensitive data.  And further ask DAMOS to find sub regions of the area
that getting colder than desired, and make an access to cache lines of the sub
regions so that they can be in the cache for "most cases".

It is just a brain storming idea off the top of my head and probably not work
for your case, since...  It ain't work if there is no good way to know or
guarantee the address ranges for the latency sensitive data.  It ain't work for
extremely latency sensitive case, as DAMON is just a best effort.  It ain't
work with DAMON of today because DAMOS doesn't support such kind of
cache-granularity access generation action.  So, it sounds like not a good
idea.

Nonetheless, if you get any question for DAMON in future, please feel free to
reach out :)


Thanks,
SJ

[...]

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Linus Walleij]

Hi Jonathan,

thanks for stepping in, I'm trying to be healthy sceptical here...

What you and others need to do is to tell me if I'm being too
critical. But right now it feels like I need some more senior
MM developers to tell me to be a good boy and let this
hack patch slip before I shut up ;)

On Wed, Feb 4, 2026 at 2:40 PM Jonathan Cameron
<jonathan.cameron@huawei.com> wrote:

> > The MM subsytem knows which memory is most cache hot.
> > Especially when you use DAMON DAMOS, which has the sole
> > purpose of executing actions like that. Here is a good YouTube.
> > https://www.youtube.com/watch?v=xKJO4kLTHOI
>
> This typically isn't about cache hot.  It it were, the data would
> be in the cache without this. It's about ensuring something that would
> otherwise unlikely to be there is in the cache.

OK I get it.

> Normally that's a latency critical region.  In general the kernel
> has no chance of figuring out what those are ahead of time, only
> userspace can know (based on profiling etc) that is per workload.
(...)
> The only thing we could do if this was in kernel would be to
> have userspace pass some hints and then let the kernel actually
> kick off the process.
(...)
> and you absolutely need userspace to be able to tell if it
> got what it asked for or not.
(...)
> Its an extreme form of profile guided optimization (and not
> currently automatic I think?). If we are putting code in this
> locked region, the program has been carefully recompiled / linked
> to group the critical parts so that we can use the minimum number
> of these locked regions. Data is a little simpler.

OK so the argument is "only userspace knows what cache lines
are performance critical, and therefore this info must be passed
from userspace". Do I understand correctly?

What I'm worried about here is that "an extreme form of profile
guided optimization" is a bit handwavy. I would accept if it is
based on simulation or simply human know-how, such as
if a developer puts signal-processing algorithm kernels
there because they know it is going to be the hard kernel
of the process.

But does the developer know if that hard kernel is importantest
taken into account all other processes running on the system,
and what happens if several processes say they have
such hard kernels? Who will arbitrate? That is usually the
kernels job.

> I haven't yet come up with any plausible scheme by which the MM
> subsystem could do this.

I find it kind of worrying if userspace knows which lines are most
performance-critical but the kernel MM subsystem does not.

That strongly inidicates that if only userspace knows that, then
madvise() is the way to go. The MM might need and use this
information for other reasons than just locking down lines in
the L3 cache.

In my mind:

Userspace madvise -> Linux MM -> arch cache-line lockdown

So the MM needs to take the decision that this indication from
userspace is something that should result in asking the arch
to lock down these cache lines, as well as re-evaluate it if
new processes start sending the same madise() calls and we
run out in lock-downable cache lines.

L3 lock-downs is a finite resource after all, and it needs to be
arbitrated. Just OTOMH, maybe if several processes ask for this
simultaneously and we run out of lockdownable cache lines,
who wins? First come first served? The process with the highest
nice value or realtime priority? Etc.

I.e. the kernel MM needs to arbitrate any cache lockdown.

Bypassing the whole MM like this patch does is a hack designed
for one single process that the user "knows" is "importantest"
and will be the only process asking for cache lines to be locked
down.

And this isn't abstract and it does not scale. We can't do that.

That's the kind of resource management we expect from the
kernel.

MM might want to use that information for other things.

> I think what we need here Yushan, is more detail on end to end
> use cases for this.  Some examples etc as clearer motivation.

I agree.

Yours,
Linus Walleij

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Jonathan Cameron]

On Thu, 5 Feb 2026 10:12:33 +0100
Linus Walleij <linusw@kernel.org> wrote:

> Hi Jonathan,
> 
> thanks for stepping in, I'm trying to be healthy sceptical here...
> 
> What you and others need to do is to tell me if I'm being too
> critical. But right now it feels like I need some more senior
> MM developers to tell me to be a good boy and let this
> hack patch slip before I shut up ;)

It's good to have these discussions as it makes us actually
explain what they want to do much more clearly!
wangyushan and I have both been taking about this for too long so
it's easy to miss that it's not been explained properly.

Note I was absolutely expecting a non trivial discussion on how to do
this and in particular how generic it should be.
 
+CC a various resctl / mpam related people.

> 
> On Wed, Feb 4, 2026 at 2:40 PM Jonathan Cameron
> <jonathan.cameron@huawei.com> wrote:
> 
> > > The MM subsytem knows which memory is most cache hot.
> > > Especially when you use DAMON DAMOS, which has the sole
> > > purpose of executing actions like that. Here is a good YouTube.
> > > https://www.youtube.com/watch?v=xKJO4kLTHOI  
> >
> > This typically isn't about cache hot.  It it were, the data would
> > be in the cache without this. It's about ensuring something that would
> > otherwise unlikely to be there is in the cache.  
> 
> OK I get it.
> 
> > Normally that's a latency critical region.  In general the kernel
> > has no chance of figuring out what those are ahead of time, only
> > userspace can know (based on profiling etc) that is per workload.  
> (...)
> > The only thing we could do if this was in kernel would be to
> > have userspace pass some hints and then let the kernel actually
> > kick off the process.  
> (...)
> > and you absolutely need userspace to be able to tell if it
> > got what it asked for or not.  
> (...)
> > Its an extreme form of profile guided optimization (and not
> > currently automatic I think?). If we are putting code in this
> > locked region, the program has been carefully recompiled / linked
> > to group the critical parts so that we can use the minimum number
> > of these locked regions. Data is a little simpler.  
> 
> OK so the argument is "only userspace knows what cache lines
> are performance critical, and therefore this info must be passed
> from userspace". Do I understand correctly?

Yes.

> 
> What I'm worried about here is that "an extreme form of profile
> guided optimization" is a bit handwavy. I would accept if it is
> based on simulation or simply human know-how, such as
> if a developer puts signal-processing algorithm kernels
> there because they know it is going to be the hard kernel
> of the process.

Those methods are part of what I'd consider profile guided optimization.
I wasn't meaning to only including the automatic methods.
For all the ways to tune this, you get lots of data from
simulation or real hardware and use that to understand what makes
sense to lock in cache.  The human involved is often going to guide
those simulations - but follow that with a lot of testing and
data gathering.

One existing user I'm aware did a lot of work to identify exactly
what they needed to pin. It's an appliance type situation where
they know exactly what the workloads are on that server. I'm not
sure how much more we can share on that customer use case / case study
beyond this vague description, so will leave it to Yushan to maybe
provide more info.

> 
> But does the developer know if that hard kernel is importantest
> taken into account all other processes running on the system,
> and what happens if several processes say they have
> such hard kernels? Who will arbitrate? That is usually the
> kernels job.

Take the closest example to this which is resctl (mpam on arm).
This actually has a feature that smells a bit like this.
Pseudo-cache locking. 

https://docs.kernel.org/filesystems/resctrl.html#cache-pseudo-locking

My understanding is that the semantics of that don't align perfectly
with what we have here.  Yushan can you add more on why we didn't
try to fit into that scheme?  Other than the obvious bit that more
general upstream support for the arch definitions of MPAM is a work in
progress and fitting vendor specific features on top will be tricky
for a while at least.  The hardware here is also independent of the
MPAM support.

Resctl puts the control on resource allocation into the hands of
userspace (in that case via cgroups etc as it's process level controls).
The cache lockdown is a weird because you have go through a dance of
creating a temporary setup, demand fetching the lines into cache and
then rely on various operations not occuring that might push them out
again.

Resctl provides many footguns and is (I believe) used by administrators
who are very careful in how they use it.  Note that there are some guards
in this new code to only allow locking a portion of the l3. We also rely
somewhat on the uarch and cache design to ensure it is safe to do this
type of locking (other than reducing perf of other tasks).
I'm dancing around uarch details here that I would need to go seek
agreement to share more on.

> 
> > I haven't yet come up with any plausible scheme by which the MM
> > subsystem could do this.  
> 
> I find it kind of worrying if userspace knows which lines are most
> performance-critical but the kernel MM subsystem does not.
> 
> That strongly inidicates that if only userspace knows that, then
> madvise() is the way to go. The MM might need and use this
> information for other reasons than just locking down lines in
> the L3 cache.

I agree that something like madvise() may well be more suitable.
We do need paths to know how many regions are left etc though so
it will need a few other bits of interface.

I'm also not sure what appetite there will be for an madvise()
for something that today we have no idea if anyone else actually
has hardware for.  If people do, then please shout and we can
look at how something like this can be generalized.

> 
> In my mind:
> 
> Userspace madvise -> Linux MM -> arch cache-line lockdown
> 
> So the MM needs to take the decision that this indication from
> userspace is something that should result in asking the arch
> to lock down these cache lines, as well as re-evaluate it if
> new processes start sending the same madise() calls and we
> run out in lock-downable cache lines.
> 
> L3 lock-downs is a finite resource after all, and it needs to be
> arbitrated. Just OTOMH, maybe if several processes ask for this
> simultaneously and we run out of lockdownable cache lines,
> who wins? First come first served? The process with the highest
> nice value or realtime priority? Etc.

My current thinking is first come first served with a path to
clearly tell an application it didn't get what it wanted.
Scheduling, priority etc being involved would all interfere
with the strong guarantees lock down provides. 

That's kind of why we ended up with a device type model as
it's common to have finite resources and just say no if
they have run out (accelerator queues etc).
It's up to the userspace code to know what to do if they
can't get what they asked for.


> 
> I.e. the kernel MM needs to arbitrate any cache lockdown.
> 
> Bypassing the whole MM like this patch does is a hack designed
> for one single process that the user "knows" is "importantest"
> and will be the only process asking for cache lines to be locked
> down.
> 
> And this isn't abstract and it does not scale. We can't do that.
> 
> That's the kind of resource management we expect from the
> kernel.

I'm with you in many ways on this, but there are other
things for which we absolutely do allocate from a finite resource
and don't let the kernel make decisions - typically because there is no
right way to arbitrate.  If we can invent a scheme for arbitration
for this then great, right now I can't envision anything other than
1st come 1st served being appropriate. Maybe there are other use cases
where other schemes work (e.g. if we ever figure out how to use this
as a form of live optimization)

> 
> MM might want to use that information for other things.

Absolutely, though I'm doubtful about trying to design a generic
way of conveying latency criticality without knowing more of those
use cases from the start.

Thanks,
Jonathan


> 
> > I think what we need here Yushan, is more detail on end to end
> > use cases for this.  Some examples etc as clearer motivation.  
> 
> I agree.
> 
> Yours,
> Linus Walleij
> 

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Linus Walleij]

On Thu, Feb 5, 2026 at 11:18 AM Jonathan Cameron
<jonathan.cameron@huawei.com> wrote:

> Take the closest example to this which is resctl (mpam on arm).
> This actually has a feature that smells a bit like this.
> Pseudo-cache locking.
>
> https://docs.kernel.org/filesystems/resctrl.html#cache-pseudo-locking

That was very interesting. And more than a little bit complex.
IIUC MPAM is mostly about requesting bandwidth to/from the
memory.

But maybe cache lockdown can build on top?

> I'm also not sure what appetite there will be for an madvise()
> for something that today we have no idea if anyone else actually
> has hardware for.  If people do, then please shout and we can
> look at how something like this can be generalized.

Cache lockdown is an ages old concept, I think others can do
it too, you are just the first to try to support it upstream.
Personally I'm all for this, as long as we can come up with
something generic for others to use as well. No custom device
+ ioctl stuff.

There are adjacent stuff that vendors are doing is about prefetch
and which I mentioned briefly:

Fujitus prefetch:
https://lore.kernel.org/linux-arm-kernel/20220607120530.2447112-1-tarumizu.kohei@fujitsu.com/

AmpereOne prefetch:
https://lore.kernel.org/linux-arm-kernel/20231122092855.4440-1-shijie@os.amperecomputing.com/

Maybe that is more related to MPAM actually. What it has in common
with cache lockdown is "significatly indicate memore areas of special
interest". But notice Will Deacons reply:
https://lore.kernel.org/linux-arm-kernel/ZV3omRGtVS9l-tKk@FVFF77S0Q05N/

 "We tend to shy away from micro-architecture specific optimisations in
 the arm64 kernel as they're pretty unmaintainable, hard to test properly,
 generally lead to bloat and add additional obstacles to updating our
 library routines."

> My current thinking is first come first served with a path to
> clearly tell an application it didn't get what it wanted.
> Scheduling, priority etc being involved would all interfere
> with the strong guarantees lock down provides.

That sounds more like mdemand() than madvise() doesn't it ;)

But surely an all-or-nothing ABI can be specified, and maybe
a please-if-you-can ABI as well.

> > MM might want to use that information for other things.
>
> Absolutely, though I'm doubtful about trying to design a generic
> way of conveying latency criticality without knowing more of those
> use cases from the start.

Well, abstracting is about boiling the world down to a few facts
that can be used for making general decisions.

But for one I suppose if someone locks down some cache lines
in L3 and then not actually use them much at long intervals
because of misc, I suppose it's not very nice if the kernel decide
to swap out the page with these cache lines in it, because that
would have adverse impact on the performace once it hits for example?

Or did someone think about that already? Is that avoided in the
current patch set? (Maybe a stupid question...)

Likewise I see that this code is keeping track of which CPU
the l3 cache line were locked from, but I don't see anything in
this code blocking task migration for whoever called this ABI
or am I wrong? What happens if the scheduler moves the
process to another CPU? Or is it implicit that this is nailed to
the current CPU? Then surely that need to be enforced?

I just get the overall feeling that this was just tested on a scenario
such as:

1. Boot
2. Run a process calling this code, hey it works
3. Terminate process

No sleeping and swapping under memory pressure etc happing.

Designing for the generic case and in a central part of the kernel
(inside MM not in drivers/soc...) would avoid such snags I think.

Yours,
Linus Walleij

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Arnd Bergmann]

On Thu, Feb 5, 2026, at 14:47, Linus Walleij wrote:
> On Thu, Feb 5, 2026 at 11:18 AM Jonathan Cameron
> <jonathan.cameron@huawei.com> wrote:
>
>> Take the closest example to this which is resctl (mpam on arm).
>> This actually has a feature that smells a bit like this.
>> Pseudo-cache locking.
>>
>> https://docs.kernel.org/filesystems/resctrl.html#cache-pseudo-locking
>
> That was very interesting. And more than a little bit complex.
> IIUC MPAM is mostly about requesting bandwidth to/from the
> memory.
>
> But maybe cache lockdown can build on top?

The CoreLink interconnect blocks also have a way to lock
cache lines, e.g. for the CMN-600:
https://developer.arm.com/documentation/100180/0200/SLC-Memory-System/Software-configurable-memory-region-locking?lang=en

These are fairly common, but so far we never had the need to
expose this in a low-level driver. If we add driver interface
for HiSilicon chips that use a custom interconnect, we should
probably check at least that the user ABI would theoretically
also work on Arm interconnects if one wrote a driver.

Another similar technology is memory that has already been locked
by firmware (or hardware design, i.e. not a cache), and there are
a few I remember:

- drivers/misc/sram.c exports sram from physical addresses to
  userspace. For a deeply embedded system with known amounts
  of locked-down L3 cache, the firmware could just pre-lock
  the cache and expose it to the kernel as an sram.
- your own arch/arm/kernel/tcm.c, which does not currently have
  any upstream users. I don't remember if it ever had.
- arch/sh/mm/numa.c supports locked SRAM through numactl, originally
  added by Paul Mundt. The idea of using numactl to move page
  cache pages is somwhat appealing, but it really messes
  with the assumptions about NUMA interfaces, and I think this
  is the only 32-bit target that exposes numactl. There are
  probably no applications left using this in modern kernels.
- arch/arc/ had an elaborate downstream patch for wireless
  network SoCs from (IIRC) Quantenna that would link mark
  any performance-sensitive .text and .data parts of the network
  stack to be in on-chip SRAM, but no user interface.

     Arnd

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Linus Walleij]

On Thu, Feb 5, 2026 at 3:39 PM Arnd Bergmann <arnd@arndb.de> wrote:

> Another similar technology is memory that has already been locked
> by firmware (or hardware design, i.e. not a cache), and there are
> a few I remember:
>
> - drivers/misc/sram.c exports sram from physical addresses to
>   userspace. For a deeply embedded system with known amounts
>   of locked-down L3 cache, the firmware could just pre-lock
>   the cache and expose it to the kernel as an sram.

I think that's possible but not practical. To use the special on-chip
RAMs for speedy execution the affected code (hard kernel) has to
be compiled as a separate PIC file and put into that memory (IIRC).

I think I heard of a platform (OMAP?) that would lock down the L2
cache and use as some kinde of on-chip memory during
retention.

> - your own arch/arm/kernel/tcm.c, which does not currently have
>   any upstream users. I don't remember if it ever had.

Nope, but it is just a very fast SRAM, the only thing that is special
with TCM is that it can actually be moved around in physical
memory (!) by altering some special registers.

> - arch/arc/ had an elaborate downstream patch for wireless
>   network SoCs from (IIRC) Quantenna that would link mark
>   any performance-sensitive .text and .data parts of the network
>   stack to be in on-chip SRAM, but no user interface.

That's how people mostly use SRAM for executable code I
think. Some use it for data too especially things like linked
scatterlists for DMA.

The Huawei patch is different because it doesn't require any
special compilation, instead it identifies the performance-critical
code in userspace, then tell the kernel to lock down those
specific lines. It's a more practical and generic way to deal with
this problem, admittedly.

What this patch doesn't solve at all is the situation when the
*kernel* want to lock down some cache lines for whatever valid
reason. A generic API would make this possible when we
identify some performance-critical code inside the kernel as
well.

Yours,
Linus Walleij

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Ben Horgan]

On 2/5/26 10:18, Jonathan Cameron wrote:
> On Thu, 5 Feb 2026 10:12:33 +0100
> Linus Walleij <linusw@kernel.org> wrote:
> 
>> Hi Jonathan,
>>
>> thanks for stepping in, I'm trying to be healthy sceptical here...
>>
>> What you and others need to do is to tell me if I'm being too
>> critical. But right now it feels like I need some more senior
>> MM developers to tell me to be a good boy and let this
>> hack patch slip before I shut up ;)
> 
> It's good to have these discussions as it makes us actually
> explain what they want to do much more clearly!
> wangyushan and I have both been taking about this for too long so
> it's easy to miss that it's not been explained properly.
> 
> Note I was absolutely expecting a non trivial discussion on how to do
> this and in particular how generic it should be.
>  
> +CC a various resctl / mpam related people.
[...]
> 
>>
>> But does the developer know if that hard kernel is importantest
>> taken into account all other processes running on the system,
>> and what happens if several processes say they have
>> such hard kernels? Who will arbitrate? That is usually the
>> kernels job.
> 
> Take the closest example to this which is resctl (mpam on arm).
> This actually has a feature that smells a bit like this.
> Pseudo-cache locking. 
> 
> https://docs.kernel.org/filesystems/resctrl.html#cache-pseudo-locking
> 
> My understanding is that the semantics of that don't align perfectly
> with what we have here.  Yushan can you add more on why we didn't
> try to fit into that scheme?  Other than the obvious bit that more
> general upstream support for the arch definitions of MPAM is a work in
> progress and fitting vendor specific features on top will be tricky
> for a while at least.  The hardware here is also independent of the
> MPAM support.
> 
> Resctl puts the control on resource allocation into the hands of
> userspace (in that case via cgroups etc as it's process level controls).
> The cache lockdown is a weird because you have go through a dance of
> creating a temporary setup, demand fetching the lines into cache and
> then rely on various operations not occuring that might push them out
> again.
> 
> Resctl provides many footguns and is (I believe) used by administrators
> who are very careful in how they use it.  Note that there are some guards
> in this new code to only allow locking a portion of the l3. We also rely
> somewhat on the uarch and cache design to ensure it is safe to do this
> type of locking (other than reducing perf of other tasks).
> I'm dancing around uarch details here that I would need to go seek
> agreement to share more on.
> 

Just wondering about the compatiblity of cache lockdown and
resctrl/mpam. If this is done outside resctrl then how would this
interact with the cache portion bitmaps used in resctrl/mpam? For
instance, how would a user know whether or not a resctrl/mpam cache
portion is unusable because it has been locked?
Thanks,

Ben

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[wangyushan]

On 2/5/2026 6:18 PM, Jonathan Cameron wrote:
> On Thu, 5 Feb 2026 10:12:33 +0100
> Linus Walleij <linusw@kernel.org> wrote:
>
>> But does the developer know if that hard kernel is importantest
>> taken into account all other processes running on the system,
>> and what happens if several processes say they have
>> such hard kernels? Who will arbitrate? That is usually the
>> kernels job.
>
> Take the closest example to this which is resctl (mpam on arm).
> This actually has a feature that smells a bit like this.
> Pseudo-cache locking.
>
> https://docs.kernel.org/filesystems/resctrl.html#cache-pseudo-locking
>
> My understanding is that the semantics of that don't align perfectly
> with what we have here.  Yushan can you add more on why we didn't
> try to fit into that scheme?  Other than the obvious bit that more
> general upstream support for the arch definitions of MPAM is a work in
> progress and fitting vendor specific features on top will be tricky
> for a while at least.  The hardware here is also independent of the
> MPAM support.

Intel cache pseudo lock requires help of IA32_PQR_ASSOC MSR, according
to [1], that register can save necessary information for processes acquired
cache pseudo locks, but Arm64 does not have the equivalent register.

[1]: https://www.intel.com/content/www/us/en/developer/articles/technical/cache-allocation-technology-usage-models.html

>
> Resctl puts the control on resource allocation into the hands of
> userspace (in that case via cgroups etc as it's process level controls).
> The cache lockdown is a weird because you have go through a dance of
> creating a temporary setup, demand fetching the lines into cache and
> then rely on various operations not occuring that might push them out
> again.
>
> Resctl provides many footguns and is (I believe) used by administrators
> who are very careful in how they use it.  Note that there are some guards
> in this new code to only allow locking a portion of the l3. We also rely
> somewhat on the uarch and cache design to ensure it is safe to do this
> type of locking (other than reducing perf of other tasks).
> I'm dancing around uarch details here that I would need to go seek
> agreement to share more on.
>
>>
>>> I haven't yet come up with any plausible scheme by which the MM
>>> subsystem could do this.
>>
>> I find it kind of worrying if userspace knows which lines are most
>> performance-critical but the kernel MM subsystem does not.
>>
>> That strongly inidicates that if only userspace knows that, then
>> madvise() is the way to go. The MM might need and use this
>> information for other reasons than just locking down lines in
>> the L3 cache.
>
> I agree that something like madvise() may well be more suitable.
> We do need paths to know how many regions are left etc though so
> it will need a few other bits of interface.
>
> I'm also not sure what appetite there will be for an madvise()
> for something that today we have no idea if anyone else actually
> has hardware for.  If people do, then please shout and we can
> look at how something like this can be generalized.

Currently madvise() "only operates on whole pages", maybe
madvise() will not be happy with the semantic change of
page / cacheline.

Cache size available for lock may be far less than the size
madvise() can handle. Though madvise() can always speculatively call
cache lock once appropriate and get back to original track if refused,
but that's a hack that need more deep discussion.

I think resctl is more suitable for this, as this serves the same
purpose as MPAM etc, to save QoS of a task, and the way to achieve it,
by tweaking hardware capability.

Thanks,
Yushan

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Ben Horgan]

Hi Yushan,

On 2/6/26 09:54, wangyushan wrote:
> 
> On 2/5/2026 6:18 PM, Jonathan Cameron wrote:
>> On Thu, 5 Feb 2026 10:12:33 +0100
>> Linus Walleij <linusw@kernel.org> wrote:
>>
>>> But does the developer know if that hard kernel is importantest
>>> taken into account all other processes running on the system,
>>> and what happens if several processes say they have
>>> such hard kernels? Who will arbitrate? That is usually the
>>> kernels job.
>>
>> Take the closest example to this which is resctl (mpam on arm).
>> This actually has a feature that smells a bit like this.
>> Pseudo-cache locking.
>>
>> https://docs.kernel.org/filesystems/resctrl.html#cache-pseudo-locking
>>
>> My understanding is that the semantics of that don't align perfectly
>> with what we have here.  Yushan can you add more on why we didn't
>> try to fit into that scheme?  Other than the obvious bit that more
>> general upstream support for the arch definitions of MPAM is a work in
>> progress and fitting vendor specific features on top will be tricky
>> for a while at least.  The hardware here is also independent of the
>> MPAM support.
> 
> Intel cache pseudo lock requires help of IA32_PQR_ASSOC MSR, according
> to [1], that register can save necessary information for processes acquired
> cache pseudo locks, but Arm64 does not have the equivalent register.

If you have MPAM, the per exception level MPAMx_ELy registers are
somewhat equivalent. They tell you which partid and pmg identifiers the
CPU is using and IA32_PQR_ASSOC tells you the closid and rmid which are
much the same thing. Is there a difference that stops being equivalent
in this scenario?

> 
> [1]: https://www.intel.com/content/www/us/en/developer/articles/technical/cache-allocation-technology-usage-models.html
> 
[...]

Thanks,

Ben

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[wangyushan]

On 2/4/2026 9:40 PM, Jonathan Cameron wrote:
> On Wed, 4 Feb 2026 01:10:01 +0100
> Linus Walleij <linusw@kernel.org> wrote:
>
>> Shouldn't the MM subsystem be in charge of determining, locking
>> down and freeing up hot regions in L3 cache?
>>
>> This looks more like userspace is going to determine that but
>> how exactly? By running DAMON? Then it's better to keep the
>> whole mechanism in the kernel where it belongs and let the
>> MM subsystem adapt locked L3 cache to the usage patterns.
> I haven't yet come up with any plausible scheme by which the MM
> subsystem could do this.
>
> I think what we need here Yushan, is more detail on end to end
> use cases for this.  Some examples etc as clearer motivation.
>

Hi,

Let me try to explain the use case here.

The idea is similar to this article:
https://www.cl.cam.ac.uk/~rnw24/papers/201708-sigcomm-diskcryptnet.pdf

Suppose we have data on SSD that need to be transferred through network.
We have technologies like DDIO and IO stash to make data flow through
L3 cache instead of DDR to avoid the influence of DDR bandwidth.

But if something is to be done to the data instead of merely copying,
and cores needs to participate, we'd like to make data to climb a bit
higher up through the memory hierarchy and stay there before data
processing is done. That is, correct amount of data being fetched to
L3 cache, and consumed just in time, then free L3 for next batch.
It is more of a userspace defined pipeline that utilizes capability
provided by kernel, where cache locks are allocated and freed quickly
with batches.

In above use case, C2C latency is chosen to avoid DDR latency, precisely
which L3 cache to store the data is not required. (For this part maybe
including steering tag as the hint to choose the correct L3 is a smarter
way, like AMD SDCIAE).

Memory management is, in many way, independent to architecture and
vendors, we might not want to take hardware specific feature into
account when kernel makes decisions of, say, swapping a page or not,
but we can control the hardware resource to lean more on a process,
like resctl.

Thanks,
Yushan

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Arnd Bergmann]

On Fri, Feb 6, 2026, at 11:07, wangyushan wrote:
>
> Let me try to explain the use case here.
>
> The idea is similar to this article:
> https://www.cl.cam.ac.uk/~rnw24/papers/201708-sigcomm-diskcryptnet.pdf
>
> Suppose we have data on SSD that need to be transferred through network.
> We have technologies like DDIO and IO stash to make data flow through
> L3 cache instead of DDR to avoid the influence of DDR bandwidth.
>
> But if something is to be done to the data instead of merely copying,
> and cores needs to participate, we'd like to make data to climb a bit
> higher up through the memory hierarchy and stay there before data
> processing is done. That is, correct amount of data being fetched to
> L3 cache, and consumed just in time, then free L3 for next batch.
> It is more of a userspace defined pipeline that utilizes capability
> provided by kernel, where cache locks are allocated and freed quickly
> with batches.
>
> In above use case, C2C latency is chosen to avoid DDR latency, precisely
> which L3 cache to store the data is not required. (For this part maybe
> including steering tag as the hint to choose the correct L3 is a smarter
> way, like AMD SDCIAE).
>
> Memory management is, in many way, independent to architecture and
> vendors, we might not want to take hardware specific feature into
> account when kernel makes decisions of, say, swapping a page or not,
> but we can control the hardware resource to lean more on a process,
> like resctl.

Ah, so if the main purpose here is to access the memory from
devices, I wonder if this should be structured as a dma-buf
driver. This would still allow you to mmap() a character
device, but in addition allow passing the file descriptor
to driver interfaces that take a dmabuf instead of a user
memory pointer.

    Arnd

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[Linus Walleij]

On Fri, Feb 6, 2026 at 11:08 AM wangyushan <wangyushan12@huawei.com> wrote:

> Suppose we have data on SSD that need to be transferred through network.
> We have technologies like DDIO and IO stash to make data flow through
> L3 cache instead of DDR to avoid the influence of DDR bandwidth.
[https://www.cl.cam.ac.uk/~rnw24/papers/201708-sigcomm-diskcryptnet.pdf]

So as to decode, encrypt or run some AI training/inference stuff on the
data, I get it.

The paper immediately gives at hand a use case the Linux kernel
(not userspace) could use: lock down the code and constants used
by in-kernel cipher algorithms to reduce latency on encrypted disk
or networks.

[Added in Ard and Herbert who may be interested]

Which means that if this could actually be used for these "hard
kernels" in Linux the proper way to abstract this is to give the kernel
a generic interface to request L3 cacheline lockdown no matter
if that is employed by the kernel or userspace.

> But if something is to be done to the data instead of merely copying,
> and cores needs to participate,

When you say this, is it "CPU cores" or others cores such as DSPs or
GPU/NPUs you are thinking of, or any kind of data processing core
(all of them)?

This surely need to be abstracted in such a way that either of these
can use it, Arnd mentions dma-buf which is a way devices think about
data that the CPU cores doesn't necessarily (but may) touch,
and resctrl could very well integrate into that I think.

What I think is important is that the modeling in the kernel is consistent
and that l3 cache lockdown is something any part of the kernel
needing it can request.

Yours,
Linus Walleij

Re: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC

[kernel test robot]

Hi Yushan,

kernel test robot noticed the following build errors:

[auto build test ERROR on linus/master]
[also build test ERROR on v6.19-rc8]
[cannot apply to soc/for-next next-20260203]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Yushan-Wang/soc-cache-L3-cache-driver-for-HiSilicon-SoC/20260204-004656
base:   linus/master
patch link:    https://lore.kernel.org/r/20260203161843.649417-2-wangyushan12%40huawei.com
patch subject: [PATCH 1/3] soc cache: L3 cache driver for HiSilicon SoC
config: loongarch-randconfig-r131-20260204 (https://download.01.org/0day-ci/archive/20260204/202602041006.7Hb46Sl8-lkp@intel.com/config)
compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 9b8addffa70cee5b2acc5454712d9cf78ce45710)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260204/202602041006.7Hb46Sl8-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202602041006.7Hb46Sl8-lkp@intel.com/

All errors (new ones prefixed by >>):

>> drivers/soc/hisilicon/hisi_soc_l3c.c:251:7: error: call to undeclared function 'alloc_contig_pages'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     251 |         pg = alloc_contig_pages(1 << order, GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO,
         |              ^
>> drivers/soc/hisilicon/hisi_soc_l3c.c:251:5: error: incompatible integer to pointer conversion assigning to 'struct page *' from 'int' [-Wint-conversion]
     251 |         pg = alloc_contig_pages(1 << order, GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO,
         |            ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     252 |                                 cpu_to_node(smp_processor_id()), NULL);
         |                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   2 errors generated.


vim +/alloc_contig_pages +251 drivers/soc/hisilicon/hisi_soc_l3c.c

   237	
   238	static int hisi_l3c_mmap(struct file *file, struct vm_area_struct *vma)
   239	{
   240		unsigned long size = vma->vm_end - vma->vm_start;
   241		int order = get_order(size);
   242		unsigned long addr;
   243		struct page *pg;
   244		int ret;
   245	
   246		struct hisi_l3c_lock_region *clr __free(kfree) = kzalloc(sizeof(*clr), GFP_KERNEL);
   247		if (!clr)
   248			return -ENOMEM;
   249	
   250		/* Continuous physical memory is required for L3 cache lock. */
 > 251		pg = alloc_contig_pages(1 << order, GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO,
   252					cpu_to_node(smp_processor_id()), NULL);
   253		if (!pg)
   254			return -ENOMEM;
   255	
   256		addr = page_to_phys(pg);
   257		*clr = (struct hisi_l3c_lock_region) {
   258			.addr = addr,
   259			.size = size,
   260			.cpu = smp_processor_id(),
   261			/* vma should not be moved, store here for validation */
   262			.vm_start = vma->vm_start,
   263			.vm_end = vma->vm_end,
   264		};
   265	
   266		vm_flags_set(vma, VM_DONTCOPY | VM_DONTEXPAND);
   267		vma->vm_ops = &hisi_l3c_vm_ops;
   268		vma->vm_private_data = clr;
   269	
   270		hisi_l3c_vm_ops.open(vma);
   271		if (clr->status) {
   272			ret = clr->status;
   273			goto out_page;
   274		}
   275	
   276		ret = remap_pfn_range(vma, vma->vm_start, PFN_DOWN(addr), size,
   277				      vma->vm_page_prot);
   278		if (ret)
   279			goto out_page;
   280	
   281		/* Save clr from being freed when lock succeeds. */
   282		vma->vm_private_data = no_free_ptr(clr);
   283	
   284		return 0;
   285	
   286	out_page:
   287		free_contig_range(PHYS_PFN(clr->addr), 1 << order);
   288		return ret;
   289	}
   290	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

[PATCH 2/3] soc cache: L3 cache lockdown support for HiSilicon SoC

[Yushan Wang]

This driver implements the interface exposed by framework, passes cache
lock/unlock requests to hardware.

The number of locked memory region is limited according to firmware,
and the total size of locked memory regions must be less than 70% of
cache size.

Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
---
 drivers/soc/hisilicon/hisi_soc_l3c.c | 530 ++++++++++++++++++++++++++-
 1 file changed, 528 insertions(+), 2 deletions(-)

diff --git a/drivers/soc/hisilicon/hisi_soc_l3c.c b/drivers/soc/hisilicon/hisi_soc_l3c.c
index b6f6d5bdd4e5..d21728085143 100644
--- a/drivers/soc/hisilicon/hisi_soc_l3c.c
+++ b/drivers/soc/hisilicon/hisi_soc_l3c.c
@@ -8,17 +8,51 @@
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/acpi.h>
 #include <linux/cleanup.h>
+#include <linux/cpuhotplug.h>
 #include <linux/init.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
 #include <linux/mm.h>
 #include <linux/miscdevice.h>
 #include <linux/module.h>
+#include <linux/platform_device.h>
 #include <linux/spinlock.h>
 #include <linux/spinlock_types.h>
 #include <linux/types.h>
+#include <linux/xarray.h>
+
+#ifdef CONFIG_ARM64
+#include <asm/cputype.h>
+#endif
 
 #include <uapi/misc/hisi_l3c.h>
 
+#define HISI_L3C_LOCK_CTRL	0x0530
+#define HISI_L3C_LOCK_AREA	0x0534
+#define HISI_L3C_LOCK_START_L	0x0538
+#define HISI_L3C_LOCK_START_H	0x053C
+
+#define HISI_L3C_DYNAMIC_AUCTRL	0x0404
+
+#define HISI_L3C_LOCK_CTRL_POLL_GAP_US	10
+#define HISI_L3C_LOCK_CTRL_POLL_MAX_US	10000
+
+/* L3C control register bit definition */
+#define HISI_L3C_LOCK_CTRL_LOCK_EN		BIT(0)
+#define HISI_L3C_LOCK_CTRL_LOCK_DONE		BIT(1)
+#define HISI_L3C_LOCK_CTRL_UNLOCK_EN		BIT(2)
+#define HISI_L3C_LOCK_CTRL_UNLOCK_DONE		BIT(3)
+
+#define HISI_L3C_LOCK_MIN_SIZE		(1 * 1024 * 1024)
+#define HISI_L3_CACHE_LINE_SIZE		64
+
+/* Allow maximum 70% of cache locked. */
+#define HISI_L3C_MAX_LOCK_SIZE(size)	((size) / 10 * 7)
+
+#define l3c_lock_reg_offset(reg, set)	((reg) + 16 * (set))
+
 #define to_hisi_l3c(p) container_of((p), struct hisi_l3c, comp)
 
 /**
@@ -44,7 +78,6 @@ struct hisi_soc_comp {
  * @poll_lock_done: check if the lock operation has succeeded.
  * @do_unlock: unlock the locked region of L3 cache back to normal.
  * @poll_unlock_done: check if the unlock operation has succeeded.
-	      operation requests.
  *
  * Operations are decoupled into two phases so that framework does not have
  * to wait for one operation to finish before calling the next when multiple
@@ -83,8 +116,280 @@ struct hisi_soc_comp_list {
 	spinlock_t lock;
 };
 
+struct hisi_l3c {
+	struct hisi_soc_comp comp;
+	cpumask_t associated_cpus;
+
+	/* Stores the first address locked by each register sets. */
+	struct xarray lock_sets;
+	/* Locks lock_sets to forbid overlapping access. */
+	spinlock_t reg_lock;
+
+	struct hlist_node node;
+	void __iomem *base;
+
+	/* ID of Super CPU cluster on where the L3 cache locates. */
+	int sccl_id;
+	/* ID of CPU cluster where L3 cache is located. */
+	int ccl_id;
+};
+
+static int hisi_l3c_cpuhp_state;
+
 static struct hisi_soc_comp_list l3c_devs;
 
+/**
+ * hisi_l3c_alloc_lock_reg_set - Allocate an available control register set
+ *				     of L3 cache for lock & unlock operations.
+ * @l3c:	The L3C instance on which the register set will be allocated.
+ * @addr:	The address to be locked.
+ * @size:	The size to be locked.
+ *
+ * @return:
+ *   - -EBUSY: If there is no available register sets.
+ *   - -ENOMEM: If there is no available memory for lock region struct.
+ *   - -EINVAL: If there is no available cache size for lock.
+ *   - 0: If allocation succeeds.
+ *
+ * Maintains the resource of control registers of L3 cache.  On allocation,
+ * the index of a spare set of registers is returned, then the address is
+ * stored inside for future match of unlock operation.
+ */
+static int hisi_l3c_alloc_lock_reg_set(struct hisi_l3c *l3c, phys_addr_t addr, size_t size)
+{
+	struct hisi_l3c_lock_info *info = l3c->comp.private;
+	struct hisi_l3c_lock_region *lr;
+	void *entry;
+	int idx, ret;
+
+	if (size > info->lock_size)
+		return -EINVAL;
+
+	for (idx = 0; idx < info->lock_region_num; ++idx) {
+		entry = xa_load(&l3c->lock_sets, idx);
+		if (!entry)
+			break;
+	}
+
+	if (idx > info->lock_region_num)
+		return -EBUSY;
+
+	lr = kzalloc(sizeof(*lr), GFP_KERNEL);
+	if (!lr)
+		return -ENOMEM;
+
+	lr->addr = addr;
+	lr->size = size;
+
+	ret = xa_alloc(&l3c->lock_sets, &idx, lr, xa_limit_31b, GFP_KERNEL);
+	if (ret) {
+		kfree(lr);
+		return ret;
+	}
+
+	info->lock_size -= size;
+	info->lock_region_num -= 1;
+
+	return idx;
+}
+
+/**
+ * hisi_l3c_get_locked_reg_set - Get the index of an allocated register set
+ *				     by locked address.
+ * @l3c:	The L3C instance on which the register set is allocated.
+ * @addr:	The locked address.
+ *
+ * @return:
+ *   - >= 0: index of register set which controls locked memory region of @addr.
+ *   - -EINVAL: If @addr is not locked in this cache.
+ */
+static int hisi_l3c_get_locked_reg_set(struct hisi_l3c *l3c, phys_addr_t addr)
+{
+	struct hisi_l3c_lock_region *entry;
+	unsigned long idx;
+
+	xa_for_each(&l3c->lock_sets, idx, entry) {
+		if (entry->addr == addr)
+			return idx;
+	}
+	return -EINVAL;
+}
+
+/**
+ * hisi_l3c_free_lock_reg_set - Free an allocated register set by locked
+ *				    address.
+ *
+ * @l3c:	The L3C instance on which the register set is allocated.
+ * @regset:	ID of Register set to be freed.
+ */
+static void hisi_l3c_free_lock_reg_set(struct hisi_l3c *l3c, int regset)
+{
+	struct hisi_l3c_lock_info *info = l3c->comp.private;
+	struct hisi_l3c_lock_region *entry;
+
+	if (regset < 0)
+		return;
+
+	entry = xa_erase(&l3c->lock_sets, regset);
+	if (!entry)
+		return;
+
+	info->lock_size += entry->size;
+	info->lock_region_num += 1;
+	kfree(entry);
+}
+
+static bool hisi_l3c_lock_wait_finished(struct hisi_l3c *l3c, int regset)
+{
+	void *reg = l3c->base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset);
+	/* Wait until neither lock or unlock operation is going on. */
+	u32 mask = HISI_L3C_LOCK_CTRL_LOCK_DONE | HISI_L3C_LOCK_CTRL_UNLOCK_DONE;
+	u32 val;
+
+	/*
+	 * Lock/unlock done bits are initially 0 if no lock operation was ever
+	 * issued, and will be set until next operation comes.
+	 * Check if this is the first lock operation after boot by checking if
+	 * the register is 0. If so, proceed with the operation.
+	 */
+	val = readl(reg);
+	if (!val)
+		return true;
+
+	return !readl_poll_timeout_atomic(reg, val, val & mask,
+			HISI_L3C_LOCK_CTRL_POLL_GAP_US,
+			HISI_L3C_LOCK_CTRL_POLL_MAX_US);
+}
+
+static int hisi_l3c_do_lock(struct hisi_soc_comp *comp, phys_addr_t addr, size_t size)
+{
+	struct hisi_l3c *l3c = to_hisi_l3c(comp);
+	struct hisi_l3c_lock_info *info = l3c->comp.private;
+	void *base = l3c->base;
+	int regset;
+	u32 ctrl;
+
+	if (info->address_alignment && do_div(addr, size))
+		return -EINVAL;
+
+	if (size < info->min_lock_size || size > info->max_lock_size)
+		return -EINVAL;
+
+	guard(spinlock)(&l3c->reg_lock);
+
+	regset = hisi_l3c_alloc_lock_reg_set(l3c, addr, size);
+	if (regset < 0)
+		return -EBUSY;
+
+	if (!hisi_l3c_lock_wait_finished(l3c, regset)) {
+		hisi_l3c_free_lock_reg_set(l3c, regset);
+		return -EBUSY;
+	}
+
+	writel(lower_32_bits(addr),
+	       base + l3c_lock_reg_offset(HISI_L3C_LOCK_START_L, regset));
+	writel(upper_32_bits(addr),
+	       base + l3c_lock_reg_offset(HISI_L3C_LOCK_START_H, regset));
+	writel(size, base + l3c_lock_reg_offset(HISI_L3C_LOCK_AREA, regset));
+
+	ctrl = readl(base + HISI_L3C_DYNAMIC_AUCTRL);
+	ctrl |= BIT(regset);
+	writel(ctrl, base + HISI_L3C_DYNAMIC_AUCTRL);
+
+	ctrl = readl(base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset));
+	ctrl = (ctrl | HISI_L3C_LOCK_CTRL_LOCK_EN) &
+		~HISI_L3C_LOCK_CTRL_UNLOCK_EN;
+	writel(ctrl, base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset));
+
+	return 0;
+}
+
+static int hisi_l3c_poll_lock_done(struct hisi_soc_comp *comp, phys_addr_t addr, size_t size)
+{
+	struct hisi_l3c *l3c = to_hisi_l3c(comp);
+	int regset;
+
+	guard(spinlock)(&l3c->reg_lock);
+
+	regset = hisi_l3c_get_locked_reg_set(l3c, addr);
+	if (regset < 0)
+		return -EINVAL;
+
+	if (!hisi_l3c_lock_wait_finished(l3c, regset))
+		return -ETIMEDOUT;
+
+	return 0;
+}
+
+static int hisi_l3c_do_unlock(struct hisi_soc_comp *comp, phys_addr_t addr)
+{
+	struct hisi_l3c *l3c = to_hisi_l3c(comp);
+	void *base = l3c->base;
+	int regset;
+	u32 ctrl;
+
+	guard(spinlock)(&l3c->reg_lock);
+
+	regset = hisi_l3c_get_locked_reg_set(l3c, addr);
+	if (regset < 0)
+		return -EINVAL;
+
+	if (!hisi_l3c_lock_wait_finished(l3c, regset))
+		return -EBUSY;
+
+	ctrl = readl(base + HISI_L3C_DYNAMIC_AUCTRL);
+	ctrl &= ~BIT(regset);
+	writel(ctrl, base + HISI_L3C_DYNAMIC_AUCTRL);
+
+	ctrl = readl(base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset));
+	ctrl = (ctrl | HISI_L3C_LOCK_CTRL_UNLOCK_EN) &
+		~HISI_L3C_LOCK_CTRL_LOCK_EN;
+	writel(ctrl, base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset));
+
+	return 0;
+}
+
+static int hisi_l3c_poll_unlock_done(struct hisi_soc_comp *comp, phys_addr_t addr)
+{
+	struct hisi_l3c *l3c = to_hisi_l3c(comp);
+	int regset;
+
+	guard(spinlock)(&l3c->reg_lock);
+
+	regset = hisi_l3c_get_locked_reg_set(l3c, addr);
+	if (regset < 0)
+		return -EINVAL;
+
+	if (!hisi_l3c_lock_wait_finished(l3c, regset))
+		return -ETIMEDOUT;
+
+	hisi_l3c_free_lock_reg_set(l3c, regset);
+
+	return 0;
+}
+
+static void hisi_l3c_remove_locks(struct hisi_l3c *l3c)
+{
+	void *base = l3c->base;
+	unsigned long regset;
+	void *entry;
+
+	guard(spinlock)(&l3c->reg_lock);
+
+	xa_for_each(&l3c->lock_sets, regset, entry) {
+		int timeout;
+		u32 ctrl;
+
+		ctrl = readl(base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset));
+		ctrl = (ctrl | HISI_L3C_LOCK_CTRL_UNLOCK_EN) & ~HISI_L3C_LOCK_CTRL_LOCK_EN;
+		writel(ctrl, base + l3c_lock_reg_offset(HISI_L3C_LOCK_CTRL, regset));
+
+		timeout = hisi_l3c_lock_wait_finished(l3c, regset);
+		if (timeout)
+			pr_err("failed to remove %lu-th cache lock.\n", regset);
+	}
+}
+
 static int hisi_l3c_lock(int cpu, phys_addr_t addr, size_t size)
 {
 	struct hisi_soc_comp *comp;
@@ -313,6 +618,201 @@ static int hisi_l3c_lock_restriction(unsigned long arg)
 	return -ENODEV;
 }
 
+static int hisi_l3c_init_lock_capacity(struct hisi_l3c *l3c, struct device *dev)
+{
+	int ret;
+	u32 val;
+
+	struct hisi_l3c_lock_info *info __free(kfree) = kmalloc(sizeof(*info), GFP_KERNEL);
+	if (!info)
+		return -ENOMEM;
+
+	ret = device_property_read_u32(dev, "hisilicon,l3c-lockregion-num", &val);
+	if (ret || val <= 0)
+		return -EINVAL;
+
+	info->lock_region_num = val;
+
+	ret = device_property_read_u32(dev, "hisilicon,l3c-max-single-lockregion-size", &val);
+	if (ret || val <= 0)
+		return -EINVAL;
+
+	info->lock_size = HISI_L3C_MAX_LOCK_SIZE(val);
+	info->address_alignment = info->lock_region_num == 1;
+	info->max_lock_size = HISI_L3C_MAX_LOCK_SIZE(val);
+	info->min_lock_size = info->lock_region_num == 1
+					? HISI_L3C_LOCK_MIN_SIZE
+					: HISI_L3_CACHE_LINE_SIZE;
+
+	l3c->comp.private = no_free_ptr(info);
+
+	return 0;
+}
+
+static int hisi_l3c_init_topology(struct hisi_l3c *l3c, struct device *dev)
+{
+	l3c->sccl_id = -1;
+	l3c->ccl_id = -1;
+
+	if (device_property_read_u32(dev, "hisilicon,scl-id", &l3c->sccl_id) ||
+	    l3c->sccl_id < 0)
+		return -EINVAL;
+
+	if (device_property_read_u32(dev, "hisilicon,ccl-id", &l3c->ccl_id) ||
+	    l3c->ccl_id < 0)
+		return -EINVAL;
+
+	return 0;
+}
+
+static void hisi_init_associated_cpus(struct hisi_l3c *l3c)
+{
+	if (!cpumask_empty(&l3c->associated_cpus))
+		return;
+	cpumask_clear(&l3c->associated_cpus);
+	cpumask_copy(&l3c->comp.affinity_mask, &l3c->associated_cpus);
+}
+
+static struct hisi_soc_comp_ops hisi_comp_ops = {
+	.do_lock = hisi_l3c_do_lock,
+	.poll_lock_done = hisi_l3c_poll_lock_done,
+	.do_unlock = hisi_l3c_do_unlock,
+	.poll_unlock_done = hisi_l3c_poll_unlock_done,
+};
+
+static struct hisi_soc_comp hisi_comp = {
+	.ops = &hisi_comp_ops,
+};
+
+static int hisi_l3c_probe(struct platform_device *pdev)
+{
+	struct hisi_l3c *l3c;
+	struct resource *mem;
+	int ret;
+
+	l3c = devm_kzalloc(&pdev->dev, sizeof(*l3c), GFP_KERNEL);
+	if (!l3c)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, l3c);
+
+	mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	if (!mem)
+		return -ENODEV;
+
+	l3c->base = devm_ioremap(&pdev->dev, mem->start, resource_size(mem));
+	if (IS_ERR_OR_NULL(l3c->base))
+		return PTR_ERR(l3c->base);
+
+	l3c->comp = hisi_comp;
+	spin_lock_init(&l3c->reg_lock);
+	xa_init_flags(&l3c->lock_sets, XA_FLAGS_ALLOC);
+
+	ret = hisi_l3c_init_lock_capacity(l3c, &pdev->dev);
+	if (ret)
+		goto err_xa;
+
+	hisi_init_associated_cpus(l3c);
+
+	ret = hisi_l3c_init_topology(l3c, &pdev->dev);
+	if (ret)
+		goto err_xa;
+
+	ret = cpuhp_state_add_instance(hisi_l3c_cpuhp_state, &l3c->node);
+	if (ret)
+		goto err_xa;
+
+	hisi_soc_comp_add(&l3c->comp);
+
+	return 0;
+
+err_xa:
+	xa_destroy(&l3c->lock_sets);
+	return ret;
+}
+
+static void hisi_l3c_remove(struct platform_device *pdev)
+{
+	struct hisi_l3c *l3c = platform_get_drvdata(pdev);
+	unsigned long idx;
+	struct hisi_l3c_lock_region *entry;
+
+	hisi_l3c_remove_locks(l3c);
+
+	hisi_soc_comp_del(&l3c->comp);
+
+	cpuhp_state_remove_instance_nocalls(hisi_l3c_cpuhp_state, &l3c->node);
+
+	xa_for_each(&l3c->lock_sets, idx, entry)
+		entry = xa_erase(&l3c->lock_sets, idx);
+
+	xa_destroy(&l3c->lock_sets);
+}
+
+#ifdef CONFIG_ARM64
+static void hisi_read_sccl_and_ccl_id(int *scclp, int *cclp)
+{
+	u64 mpidr = read_cpuid_mpidr();
+	int aff3 = MPIDR_AFFINITY_LEVEL(mpidr, 3);
+	int aff2 = MPIDR_AFFINITY_LEVEL(mpidr, 2);
+	int aff1 = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	int sccl, ccl;
+
+	if (mpidr & MPIDR_MT_BITMASK) {
+		sccl = aff3;
+		ccl = aff2;
+	} else {
+		sccl = aff2;
+		ccl = aff1;
+	}
+
+	*scclp = sccl;
+	*cclp = ccl;
+}
+#else
+#define hisi_read_sccl_and_ccl_id(s, c)	do { } while (0)
+#endif
+
+static bool hisi_l3c_is_associated(struct hisi_l3c *l3c)
+{
+	int sccl_id = 0;
+	int ccl_id = 0;
+
+	hisi_read_sccl_and_ccl_id(&sccl_id, &ccl_id);
+
+	return sccl_id == l3c->sccl_id && ccl_id == l3c->ccl_id;
+}
+
+static int hisi_l3c_online_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct hisi_l3c *l3c = hlist_entry_safe(node, struct hisi_l3c, node);
+
+	if (!cpumask_test_cpu(cpu, &l3c->associated_cpus)) {
+		if (!(hisi_l3c_is_associated(l3c)))
+			return 0;
+
+		cpumask_set_cpu(cpu, &l3c->associated_cpus);
+		cpumask_copy(&l3c->comp.affinity_mask,
+			     &l3c->associated_cpus);
+	}
+	return 0;
+}
+
+static const struct acpi_device_id hisi_l3c_acpi_match[] = {
+	{ "HISI0501", },
+	{ }
+};
+MODULE_DEVICE_TABLE(acpi, hisi_l3c_acpi_match);
+
+static struct platform_driver hisi_l3c_driver = {
+	.driver = {
+		.name = "hisi_l3c",
+		.acpi_match_table = hisi_l3c_acpi_match,
+	},
+	.probe = hisi_l3c_probe,
+	.remove = hisi_l3c_remove,
+};
+
 static long hisi_l3c_ioctl(struct file *file, u32 cmd, unsigned long arg)
 {
 	switch (cmd) {
@@ -338,15 +838,41 @@ static struct miscdevice l3c_miscdev = {
 
 static int __init hisi_l3c_init(void)
 {
+	int ret;
+
 	spin_lock_init(&l3c_devs.lock);
 	INIT_LIST_HEAD(&l3c_devs.node);
 
-	return misc_register(&l3c_miscdev);
+	ret = misc_register(&l3c_miscdev);
+	if (ret)
+		return ret;
+
+	ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "hisi_l3c",
+				      hisi_l3c_online_cpu, NULL);
+	if (ret < 0)
+		goto err_hp;
+
+	hisi_l3c_cpuhp_state = ret;
+
+	ret = platform_driver_register(&hisi_l3c_driver);
+	if (ret)
+		goto err_plat;
+
+	return 0;
+
+err_plat:
+	cpuhp_remove_multi_state(CPUHP_AP_ONLINE_DYN);
+err_hp:
+	misc_deregister(&l3c_miscdev);
+
+	return ret;
 }
 module_init(hisi_l3c_init);
 
 static void __exit hisi_l3c_exit(void)
 {
+	platform_driver_unregister(&hisi_l3c_driver);
+	cpuhp_remove_multi_state(CPUHP_AP_ONLINE_DYN);
 	misc_deregister(&l3c_miscdev);
 	hisi_soc_comp_del(NULL);
 }
-- 
2.33.0

[PATCH 3/3] Documentation: soc cache: Add documentation to HiSilicon SoC cache

[Yushan Wang]

Add necessary documentation to HiSilicon SoC cache for reference.

Signed-off-by: Yushan Wang <wangyushan12@huawei.com>
---
 Documentation/driver-api/hisi-soc-cache.rst | 64 +++++++++++++++++++++
 Documentation/driver-api/index.rst          |  1 +
 MAINTAINERS                                 |  1 +
 3 files changed, 66 insertions(+)
 create mode 100644 Documentation/driver-api/hisi-soc-cache.rst

diff --git a/Documentation/driver-api/hisi-soc-cache.rst b/Documentation/driver-api/hisi-soc-cache.rst
new file mode 100644
index 000000000000..d6312b6f9ed6
--- /dev/null
+++ b/Documentation/driver-api/hisi-soc-cache.rst
@@ -0,0 +1,64 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================
+HiSilicon SoC Cache Driver
+==========================
+
+Introduction
+============
+
+HiSilicon SoC cache provides the capabilities of preventing given range of
+memory from being evicted from L3 cache. The driver exports the lockdown API to
+userspace, allowing allocation of memory that is guranteed to be placed in L3
+cache, thus decreasing average memory access latency.
+
+Usage
+=====
+
+Kernel built with CONFIG_HISI_SOC_CACHE on will have the device file at
+`/dev/hisi_l3c`, cache operations can be performed through it.
+
+mmap():
+-------
+
+This interface can be used to allocate memory that is guranteed to not be
+evicted out of HiSilicon L3 cache. Newly allocated memory will be prefetched to
+L3 cache automatically.
+
+Users should set `PROT_READ` or `PROT_WRITE` to enable read/write to the memory
+region. Once mmap call succeeds, read and write can be applied to the memory
+region indicated by the returned pointer.
+
+Calling `munmap()` to the pointer can be used to unlock the memory regions.
+
+Restrictions of the cache lockdown are listed below:
+  - Only limited number of memory regions are supported, the exact number is
+    reported by firmware.
+  - Sum of the sizes of locked memory regions should be less than 70% of the
+    total size of cache instance.
+  - Lock/unlock can only be performed during allocation/deallocation, locking
+    existing memory is not supported yet.
+
+ioctl():
+--------
+
+This interface provides useful information of HiSilicon L3 cache.
+
+HISI_L3C_LOCK_INFO
+  - struct hisi_l3c_lock_info (read)
+
+  Gets detailed information of L3 cache lock restrictions.
+
+This ioctl call returns the detailed information of HiSilicon L3 cache lock
+restriction. Information will be presented in the form of::
+
+        struct hisi_l3c_lock_info {
+                unsigned int lock_region_num;
+                size_t lock_size;
+                bool address_alignment;
+                size_t max_lock_size;
+                size_t min_lock_size;
+        };
+
+User may perform a query before issueing cache lock to check for available
+resource.
diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
index 1833e6a0687e..a1d4bf2a22de 100644
--- a/Documentation/driver-api/index.rst
+++ b/Documentation/driver-api/index.rst
@@ -95,6 +95,7 @@ Subsystem-specific APIs
    generic-counter
    generic_pt
    gpio/index
+   hisi-soc-cache
    hsi
    hte/index
    hw-recoverable-errors
diff --git a/MAINTAINERS b/MAINTAINERS
index 247df6e69c10..d9f9452c8602 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11378,6 +11378,7 @@ F:	drivers/soc/hisilicon/kunpeng_hccs.h
 HISILICON SOC L3C DRIVER
 M:	Yushan Wang <wangyushan12@huawei.com>
 S:	Maintained
+F:	Documentation/driver-api/hisi-soc-cache.rst
 F:	drivers/soc/hisilicon/hisi_soc_l3c.c
 F:	include/uapi/misc/hisi_l3c.h
 
-- 
2.33.0