* [PATCH v15 1/9] uaccess: add generic fallback version of copy_mc_to_user()
From: Ruidong Tian @ 2026-06-18 9:21 UTC (permalink / raw)
To: catalin.marinas, will, rafael, tony.luck, guohanjun, mchehab,
xueshuai, tongtiangen, james.morse, robin.murphy, andreyknvl,
dvyukov, vincenzo.frascino, mpe, npiggin, ryabinin.a.a, glider,
christophe.leroy, aneesh.kumar, naveen.n.rao, tglx, mingo
Cc: linux-arm-kernel, linux-mm, linuxppc-dev, linux-kernel, kasan-dev,
tianruidong, Mauro Carvalho Chehab, Jonathan Cameron
In-Reply-To: <20260618092124.3901230-1-tianruidong@linux.alibaba.com>
From: Tong Tiangen <tongtiangen@huawei.com>
x86/powerpc has it's implementation of copy_mc_to_user(), we add generic
fallback in include/linux/uaccess.h prepare for other architechures to
enable CONFIG_ARCH_HAS_COPY_MC.
Signed-off-by: Tong Tiangen <tongtiangen@huawei.com>
Acked-by: Michael Ellerman <mpe@ellerman.id.au>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Shuai Xue <xueshuai@linux.alibaba.com>
---
arch/powerpc/include/asm/uaccess.h | 1 +
arch/x86/include/asm/uaccess.h | 1 +
include/linux/uaccess.h | 8 ++++++++
3 files changed, 10 insertions(+)
diff --git a/arch/powerpc/include/asm/uaccess.h b/arch/powerpc/include/asm/uaccess.h
index e98c628e3899..073de098d45a 100644
--- a/arch/powerpc/include/asm/uaccess.h
+++ b/arch/powerpc/include/asm/uaccess.h
@@ -432,6 +432,7 @@ copy_mc_to_user(void __user *to, const void *from, unsigned long n)
return n;
}
+#define copy_mc_to_user copy_mc_to_user
#endif
extern size_t copy_from_user_flushcache(void *dst, const void __user *src, size_t size);
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 3a0dd3c2b233..308b0854d1d5 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -496,6 +496,7 @@ copy_mc_to_kernel(void *to, const void *from, unsigned len);
unsigned long __must_check
copy_mc_to_user(void __user *to, const void *from, unsigned len);
+#define copy_mc_to_user copy_mc_to_user
#endif
/*
diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 56328601218c..13b4a3a15437 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -250,6 +250,14 @@ copy_mc_to_kernel(void *dst, const void *src, size_t cnt)
}
#endif
+#ifndef copy_mc_to_user
+static inline unsigned long __must_check
+copy_mc_to_user(void __user *dst, const void *src, unsigned long cnt)
+{
+ return copy_to_user(dst, src, cnt);
+}
+#endif
+
static __always_inline void pagefault_disabled_inc(void)
{
current->pagefault_disabled++;
--
2.39.3
^ permalink raw reply related
* [PATCH v15 0/8] arm64: add ARCH_HAS_COPY_MC support
From: Ruidong Tian @ 2026-06-18 9:21 UTC (permalink / raw)
To: catalin.marinas, will, rafael, tony.luck, guohanjun, mchehab,
xueshuai, tongtiangen, james.morse, robin.murphy, andreyknvl,
dvyukov, vincenzo.frascino, mpe, npiggin, ryabinin.a.a, glider,
christophe.leroy, aneesh.kumar, naveen.n.rao, tglx, mingo
Cc: linux-arm-kernel, linux-mm, linuxppc-dev, linux-kernel, kasan-dev,
tianruidong
This series continues Tong Tiangen's work on arm64 ARCH_HAS_COPY_MC
support. We encounter the same problem, and from a forward-looking
perspective, large-memory ARM machines will suffer more from this class
of issues, which motivates us to push this feature upstream.
Problem
=========
With the increase of memory capacity and density, the probability of memory
error also increases. The increasing size and density of server RAM in data
centers and clouds have shown increased uncorrectable memory errors.
Currently, more and more scenarios that can tolerate memory errors, such as
COW[1,2,8,9], KSM copy[3], coredump copy[4], khugepaged[5,6], uaccess copy[7],
page migration[10,11], etc.
Solution
=========
This patchset introduces a new processing framework on ARM64, which enables
ARM64 to support error recovery in the above scenarios, and more scenarios
can be expanded based on this in the future.
In arm64, memory error handling in do_sea(), which is divided into two cases:
1. If the user state consumed the memory errors, the solution is to kill
the user process and isolate the error page.
2. If the kernel state consumed the memory errors, the solution is to
panic.
For case 2, Undifferentiated panic may not be the optimal choice, as it can
be handled better. In some scenarios, we can avoid panic, such as uaccess,
if the uaccess fails due to memory error, only the user process will be
affected, returning an error to the caller and isolating the user page with
hardware memory errors is a better choice.
[1] commit d302c2398ba2 ("mm, hwpoison: when copy-on-write hits poison, take page offline")
[2] commit 1cb9dc4b475c ("mm: hwpoison: support recovery from HugePage copy-on-write faults")
[3] commit 6b970599e807 ("mm: hwpoison: support recovery from ksm_might_need_to_copy()")
[4] commit 245f09226893 ("mm: hwpoison: coredump: support recovery from dump_user_range()")
[5] commit 98c76c9f1ef7 ("mm/khugepaged: recover from poisoned anonymous memory")
[6] commit 12904d953364 ("mm/khugepaged: recover from poisoned file-backed memory")
[7] commit 278b917f8cb9 ("x86/mce: Add _ASM_EXTABLE_CPY for copy user access")
[8] commit 658be46520ce ("mm: support poison recovery from copy_present_page()")
[9] commit aa549f923f5e ("mm: support poison recovery from do_cow_fault()")
[10] commit f00b295b9b61 ("fs: hugetlbfs: support poisoned recover from hugetlbfs_migrate_folio()")
[11] commit 060913999d7a ("mm: migrate: support poisoned recover from migrate folio")
------------------
Test result:
Tested on Kunpeng 920.
1. copy_page(), copy_mc_page() basic function test pass, and the disassembly
contents remains the same before and after refactor.
2. copy_to/from_user() access kernel NULL pointer raise translation fault
and dump error message then die(), test pass.
3. Test following scenarios: copy_from_user(), get_user(), COW.
Before patched: trigger a hardware memory error then panic.
After patched: trigger a hardware memory error without panic.
Testing step:
step1. start an user-process.
step2. poison(einj) the user-process's page.
step3: user-process access the poison page in kernel mode, then trigger SEA.
step4: the kernel will not panic, only the user process is killed, the poison
page is isolated. (before patched, the kernel will panic in do_sea())
The above tests can also be reproduced using ras-tools with extra patch[1],
which provides einj-based injection and validation for all MC-safe recovery paths.
MM subsystem (hwpoison recovery via copy_mc_[user_]highpage / copy_mc_to_kernel):
einj_mem_uc cow_anon # wp_page_copy
einj_mem_uc cow_anon_pinned # copy_present_page (DMA-pinned)
einj_mem_uc cow_hugetlb # hugetlb CoW
einj_mem_uc cow_private_filemap # do_cow_fault
einj_mem_uc khugepaged_anon # MADV_COLLAPSE anon
einj_mem_uc khugepaged_file # MADV_COLLAPSE file
einj_mem_uc move_pages_numa # migrate_folio
einj_mem_uc migrate_pages_numa # migrate_pages cross-node
einj_mem_uc mbind_move # mbind MPOL_MF_MOVE
einj_mem_uc migrate_hugetlb # hugetlbfs_migrate_folio
einj_mem_uc coredump # dump_page_copy -> copy_mc_to_kernel
uaccess (copy_from_user direction):
einj_mem_uc pwrite_uc # pwrite(2)
einj_mem_uc writev_uc # writev(2)
einj_mem_uc send_uc # send(2) AF_UNIX
einj_mem_uc sendmsg_uc # sendmsg(2)
einj_mem_uc setsockopt_uc # setsockopt(2)
einj_mem_uc netlink_send_uc # AF_NETLINK sendto(2)
einj_mem_uc msgsnd_uc # SysV msgsnd(2)
einj_mem_uc mq_send_uc # POSIX mq_send(3)
einj_mem_uc semop_uc # semop(2)
einj_mem_uc process_vm_writev_uc # process_vm_writev(2)
Repo: https://git.kernel.org/pub/scm/linux/kernel/git/aegl/ras-tools.git
[1] https://lore.kernel.org/all/20260617015211.3962419-1-tianruidong@linux.alibaba.com/
------------------
Benefits
=========
According to Huawei's statistics from their storage products, memory errors
triggered in kernel-mode by COW and page cache read (uaccess) scenarios
account for more than 50%. With this patchset deployed, all kernel panics
caused by COW and page cache memory errors are eliminated.
Alibaba Cloud has also observed memory errors occurring in uaccess contexts.
Since V14:
1. Added additional recoverable scenarios (copy_present_page, do_cow_fault,
hugetlbfs_migrate_folio, migrate_folio) to the description, as reminded
by Kefeng Wang.
2. Renamed EX_TYPE_KACCESS_ERR_ZERO_MEM_ERR to EX_TYPE_KACCESS_SEA.
3. Applied review comments from Xueshuai Liu.
Since V13:
1. Changed MC-safe functions to return an error rather than kill the user
process. When a user program invokes a syscall and the kernel encounters
a memory error during uaccess, killing the process is unexpected; the
syscall should return an error.
2. Added FEAT_MOPS support for the copy_page_mc paths.
3. Refactored copy_page() and memcpy() on top of the shared memcpy_template,
reducing duplicated assembly code.
Since v12:
Thanks to the suggestions of Jonathan, Mark, and Mauro, the following modifications
are made:
1. Rebase to latest kernel version.
2. Patch1, add Jonathan's and Mauro's review-by.
3. Patch2, modified do_apei_claim_sea() according to Mark's and Jonathan's suggestions,
and optimized the commit message according to Mark's suggestions(Added description of
the impact on regular copy_to_user()).
4. Patch3, optimized the commit message according to Mauro's suggestions and add Jonathan's
review-by.
5. Patch4, modified copy_mc_user_highpage() and Optimized the commit message according to
Jonathan's suggestions(no functional changes).
6. Patch5, optimized the commit message according to Mauro's suggestions.
7. Patch4/5, FEAT_MOPS is added to the code logic. Currently, the fixup is not performed
on the MOPS instruction.
8. Remove patch6 in v12 according to Jonathan's suggestions.
Since v11:
1. Rebase to latest kernel version 6.9-rc1.
2. Add patch 5, Since the problem described in "Since V10 Besides 3" has
been solved in a50026bdb867 ('iov_iter: get rid of 'copy_mc' flag').
3. Add the benefit of applying the patch set to our company to the description of patch0.
Since V10:
Accroding Mark's suggestion:
1. Merge V10's patch2 and patch3 to V11's patch2.
2. Patch2(V11): use new fixup_type for ld* in copy_to_user(), fix fatal
issues (NULL kernel pointeraccess) been fixup incorrectly.
3. Patch2(V11): refactoring the logic of do_sea().
4. Patch4(V11): Remove duplicate assembly logic and remove do_mte().
Besides:
1. Patch2(V11): remove st* insn's fixup, st* generally not trigger memory error.
2. Split a part of the logic of patch2(V11) to patch5(V11), for detail,
see patch5(V11)'s commit msg.
3. Remove patch6(v10) “arm64: introduce copy_mc_to_kernel() implementation”.
During modification, some problems that cannot be solved in a short
period are found. The patch will be released after the problems are
solved.
4. Add test result in this patch.
5. Modify patchset title, do not use machine check and remove "-next".
Since V9:
1. Rebase to latest kernel version 6.8-rc2.
2. Add patch 6/6 to support copy_mc_to_kernel().
Since V8:
1. Rebase to latest kernel version and fix topo in some of the patches.
2. According to the suggestion of Catalin, I attempted to modify the
return value of function copy_mc_[user]_highpage() to bytes not copied.
During the modification process, I found that it would be more
reasonable to return -EFAULT when copy error occurs (referring to the
newly added patch 4).
For ARM64, the implementation of copy_mc_[user]_highpage() needs to
consider MTE. Considering the scenario where data copying is successful
but the MTE tag copying fails, it is also not reasonable to return
bytes not copied.
3. Considering the recent addition of machine check safe support for
multiple scenarios, modify commit message for patch 5 (patch 4 for V8).
Since V7:
Currently, there are patches supporting recover from poison
consumption for the cow scenario[1]. Therefore, Supporting cow
scenario under the arm64 architecture only needs to modify the relevant
code under the arch/.
[1]https://lore.kernel.org/lkml/20221031201029.102123-1-tony.luck@intel.com/
Since V6:
Resend patches that are not merged into the mainline in V6.
Since V5:
1. Add patch2/3 to add uaccess assembly helpers.
2. Optimize the implementation logic of arm64_do_kernel_sea() in patch8.
3. Remove kernel access fixup in patch9.
All suggestion are from Mark.
Since V4:
1. According Michael's suggestion, add patch5.
2. According Mark's suggestiog, do some restructuring to arm64
extable, then a new adaptation of machine check safe support is made based
on this.
3. According Mark's suggestion, support machine check safe in do_mte() in
cow scene.
4. In V4, two patches have been merged into -next, so V5 not send these
two patches.
Since V3:
1. According to Robin's suggestion, direct modify user_ldst and
user_ldp in asm-uaccess.h and modify mte.S.
2. Add new macro USER_MC in asm-uaccess.h, used in copy_from_user.S
and copy_to_user.S.
3. According to Robin's suggestion, using micro in copy_page_mc.S to
simplify code.
4. According to KeFeng's suggestion, modify powerpc code in patch1.
5. According to KeFeng's suggestion, modify mm/extable.c and some code
optimization.
Since V2:
1. According to Mark's suggestion, all uaccess can be recovered due to
memory error.
2. Scenario pagecache reading is also supported as part of uaccess
(copy_to_user()) and duplication code problem is also solved.
Thanks for Robin's suggestion.
3. According Mark's suggestion, update commit message of patch 2/5.
4. According Borisllav's suggestion, update commit message of patch 1/5.
Since V1:
1.Consistent with PPC/x86, Using CONFIG_ARCH_HAS_COPY_MC instead of
ARM64_UCE_KERNEL_RECOVERY.
2.Add two new scenes, cow and pagecache reading.
3.Fix two small bug(the first two patch).
V1 in here:
https://lore.kernel.org/lkml/20220323033705.3966643-1-tongtiangen@huawei.com/
Ruidong Tian (5):
ACPI: APEI: GHES: use exception context to gate SIGBUS on poison
consumption
arm64: extable: merge UACCESS_ERR_ZERO and KACCESS_ERR_ZERO into
ACCESS_ERR_ZERO
arm64: enable recover from synchronous external abort in kernel
context
lib/test: memcpy_kunit: add copy_page() and copy_mc_page() tests
lib/tests: memcpy_kunit: add memcpy_mc() and memcpy_mc_large() test
Tong Tiangen (4):
uaccess: add generic fallback version of copy_mc_to_user()
mm/hwpoison: return -EFAULT when copy fail in
copy_mc_[user]_highpage()
arm64: support copy_mc_[user]_highpage()
arm64: introduce copy_mc_to_kernel() implementation
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/asm-extable.h | 24 ++-
arch/arm64/include/asm/asm-uaccess.h | 4 +
arch/arm64/include/asm/extable.h | 1 +
arch/arm64/include/asm/mte.h | 9 +
arch/arm64/include/asm/page.h | 12 ++
arch/arm64/include/asm/string.h | 5 +
arch/arm64/include/asm/uaccess.h | 17 ++
arch/arm64/kernel/acpi.c | 2 +-
arch/arm64/lib/Makefile | 2 +
arch/arm64/lib/copy_mc_page.S | 44 +++++
arch/arm64/lib/copy_page.S | 67 ++-----
arch/arm64/lib/copy_page_template.S | 70 ++++++++
arch/arm64/lib/copy_to_user.S | 10 +-
arch/arm64/lib/memcpy.S | 251 ++-------------------------
arch/arm64/lib/memcpy_mc.S | 56 ++++++
arch/arm64/lib/memcpy_template.S | 250 ++++++++++++++++++++++++++
arch/arm64/lib/mte.S | 29 ++++
arch/arm64/mm/copypage.c | 80 +++++++++
arch/arm64/mm/extable.c | 35 +++-
arch/arm64/mm/fault.c | 30 +++-
arch/powerpc/include/asm/uaccess.h | 1 +
arch/x86/include/asm/uaccess.h | 1 +
drivers/acpi/apei/ghes.c | 36 ++--
include/acpi/ghes.h | 15 +-
include/linux/highmem.h | 16 +-
include/linux/uaccess.h | 8 +
lib/tests/memcpy_kunit.c | 186 +++++++++++++++++++-
mm/kasan/shadow.c | 12 ++
mm/khugepaged.c | 4 +-
30 files changed, 938 insertions(+), 340 deletions(-)
create mode 100644 arch/arm64/lib/copy_mc_page.S
create mode 100644 arch/arm64/lib/copy_page_template.S
create mode 100644 arch/arm64/lib/memcpy_mc.S
create mode 100644 arch/arm64/lib/memcpy_template.S
--
2.39.3
^ permalink raw reply
* [PATCH v7 2/3] arm64: dts: imx95: Add dma, intr, aer and pme interrupts for PCIe
From: hongxing.zhu @ 2026-06-18 9:20 UTC (permalink / raw)
To: robh, krzk+dt, conor+dt, bhelgaas, frank.li, l.stach, lpieralisi,
kwilczynski, mani, s.hauer, kernel, festevam
Cc: linux-pci, linux-arm-kernel, devicetree, imx, linux-kernel,
Richard Zhu
In-Reply-To: <20260618092100.3669556-1-hongxing.zhu@oss.nxp.com>
From: Richard Zhu <hongxing.zhu@nxp.com>
The current PCIe device tree configuration only defines the MSI
interrupt, which is sufficient for basic PCIe operation but limits
advanced functionality.
Add the following interrupt lines to pcie0 and pcie1 nodes:
- dma: DMA interrupt for PCIe DMA operations
- intr: General controller events and link state changes
- aer: Advanced Error Reporting interrupt
- pme: Power Management Event interrupt
This enables enhanced PCIe features and capabilities that were
previously unavailable due to missing interrupt definitions.
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
---
arch/arm64/boot/dts/freescale/imx95.dtsi | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/boot/dts/freescale/imx95.dtsi b/arch/arm64/boot/dts/freescale/imx95.dtsi
index 3e35c956a4d7a..1a9803f967901 100644
--- a/arch/arm64/boot/dts/freescale/imx95.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx95.dtsi
@@ -1945,8 +1945,12 @@ pcie0: pcie@4c300000 {
bus-range = <0x00 0xff>;
num-lanes = <1>;
num-viewport = <8>;
- interrupts = <GIC_SPI 310 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "msi";
+ interrupts = <GIC_SPI 310 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 311 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 310 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 310 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 310 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "msi", "dma", "intr", "aer", "pme";
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &gic 0 0 GIC_SPI 306 IRQ_TYPE_LEVEL_HIGH>,
@@ -2020,8 +2024,12 @@ pcie1: pcie@4c380000 {
bus-range = <0x00 0xff>;
num-lanes = <1>;
num-viewport = <8>;
- interrupts = <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "msi";
+ interrupts = <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 317 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "msi", "dma", "intr", "aer", "pme";
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &gic 0 0 GIC_SPI 312 IRQ_TYPE_LEVEL_HIGH>,
--
2.34.1
^ permalink raw reply related
* [PATCH v7 3/3] PCI: imx6: Add root port reset to support link recovery
From: hongxing.zhu @ 2026-06-18 9:21 UTC (permalink / raw)
To: robh, krzk+dt, conor+dt, bhelgaas, frank.li, l.stach, lpieralisi,
kwilczynski, mani, s.hauer, kernel, festevam
Cc: linux-pci, linux-arm-kernel, devicetree, imx, linux-kernel,
Richard Zhu
In-Reply-To: <20260618092100.3669556-1-hongxing.zhu@oss.nxp.com>
From: Richard Zhu <hongxing.zhu@nxp.com>
The PCIe link can go down due to various unexpected circumstances. Add
root port reset support to enable link recovery for the i.MX PCIe
controller when the optional "intr" interrupt is present.
When a link down event occurs, reset the root port by: uninitializing the
PCIe controller, re-initializing it, and restarting the link.
On i.MX95 platforms, link events and PME share the same interrupt line.
The link event interrupt cannot use a threaded-only IRQ handler because
the PME driver uses request_irq() with only the IRQF_SHARED flag set,
which requires a primary handler.
To handle this shared interrupt scenario, register a primary interrupt
handler with IRQF_SHARED for link events and manipulate the link event
enable bits to ensure the shared interrupt source triggers only one
handler at a time.
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
---
drivers/pci/controller/dwc/pci-imx6.c | 132 ++++++++++++++++++++++++++
1 file changed, 132 insertions(+)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 773ab65b2afac..3de70f41b0b85 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -79,6 +79,11 @@
#define IMX95_SID_MASK GENMASK(5, 0)
#define IMX95_MAX_LUT 32
+#define IMX95_LINK_INT_CTRL_STS 0x1040
+#define IMX95_PE0_INT_STS 0x10e8
+#define IMX95_LINK_DOWN_INT_STS BIT(11)
+#define IMX95_LINK_DOWN_INT_EN BIT(10)
+
#define IMX95_PCIE_RST_CTRL 0x3010
#define IMX95_PCIE_COLD_RST BIT(0)
@@ -126,6 +131,8 @@ enum imx_pcie_variants {
#define IMX_PCIE_MAX_INSTANCES 2
struct imx_pcie;
+static int imx_pcie_reset_root_port(struct pci_host_bridge *bridge,
+ struct pci_dev *pdev);
struct imx_pcie_drvdata {
enum imx_pcie_variants variant;
@@ -158,6 +165,7 @@ struct imx_pcie {
bool supports_clkreq;
bool enable_ext_refclk;
struct regmap *iomuxc_gpr;
+ int lnk_intr;
u16 msi_ctrl;
u32 controller_id;
struct reset_control *pciephy_reset;
@@ -1394,6 +1402,13 @@ static int imx_pcie_host_init(struct dw_pcie_rp *pp)
imx_setup_phy_mpll(imx_pcie);
+ /*
+ * Callback invoked by PCI core when link down is detected and
+ * recovery is needed.
+ */
+ if (pp->bridge)
+ pp->bridge->reset_root_port = imx_pcie_reset_root_port;
+
return 0;
err_phy_off:
@@ -1661,6 +1676,9 @@ static int imx_pcie_suspend_noirq(struct device *dev)
if (!(imx_pcie->drvdata->flags & IMX_PCIE_FLAG_SUPPORTS_SUSPEND))
return 0;
+ if (imx_pcie->lnk_intr > 0)
+ regmap_clear_bits(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_EN);
imx_pcie_msi_save_restore(imx_pcie, true);
if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_LUT))
imx_pcie_lut_save(imx_pcie);
@@ -1711,6 +1729,9 @@ static int imx_pcie_resume_noirq(struct device *dev)
if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_LUT))
imx_pcie_lut_restore(imx_pcie);
imx_pcie_msi_save_restore(imx_pcie, false);
+ if (imx_pcie->lnk_intr > 0)
+ regmap_set_bits(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_EN);
return 0;
}
@@ -1720,6 +1741,86 @@ static const struct dev_pm_ops imx_pcie_pm_ops = {
imx_pcie_resume_noirq)
};
+static irqreturn_t imx_pcie_lnk_irq_isr(int irq, void *priv)
+{
+ struct imx_pcie *imx_pcie = priv;
+ struct dw_pcie *pci = imx_pcie->pci;
+ struct device *dev = pci->dev;
+ u32 val;
+
+ regmap_read(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS, &val);
+ if (val & IMX95_LINK_DOWN_INT_STS) {
+ dev_dbg(dev, "PCIe link down detected, initiating recovery\n");
+ /* Clear link down interrupt status by writing 1b'1 to it */
+ regmap_set_bits(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_STS);
+ if (!(val & IMX95_LINK_DOWN_INT_EN))
+ return IRQ_NONE;
+ regmap_clear_bits(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_EN);
+
+ return IRQ_WAKE_THREAD;
+ }
+
+ regmap_read(imx_pcie->iomuxc_gpr, IMX95_PE0_INT_STS, &val);
+ if (unlikely(val))
+ regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_INT_STS, val);
+
+ return IRQ_NONE;
+}
+
+static irqreturn_t imx_pcie_lnk_irq_thread(int irq, void *priv)
+{
+ struct imx_pcie *imx_pcie = priv;
+ struct dw_pcie *pci = imx_pcie->pci;
+ struct dw_pcie_rp *pp = &pci->pp;
+ struct pci_dev *port;
+
+ for_each_pci_bridge(port, pp->bridge->bus)
+ if (pci_pcie_type(port) == PCI_EXP_TYPE_ROOT_PORT)
+ pci_host_handle_link_down(port);
+
+ regmap_set_bits(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_EN);
+
+ return IRQ_HANDLED;
+}
+
+static int imx_pcie_reset_root_port(struct pci_host_bridge *bridge,
+ struct pci_dev *pdev)
+{
+ struct pci_bus *bus = bridge->bus;
+ struct dw_pcie_rp *pp = bus->sysdata;
+ struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
+ struct imx_pcie *imx_pcie = to_imx_pcie(pci);
+ int ret;
+
+ imx_pcie_msi_save_restore(imx_pcie, true);
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_LUT))
+ imx_pcie_lut_save(imx_pcie);
+ imx_pcie_stop_link(pci);
+ imx_pcie_host_exit(pp);
+
+ ret = imx_pcie_host_init(pp);
+ if (ret) {
+ dev_err(pci->dev, "Failed to re-init PCIe\n");
+ return ret;
+ }
+ ret = dw_pcie_setup_rc(pp);
+ if (ret)
+ return ret;
+
+ imx_pcie_start_link(pci);
+ dw_pcie_wait_for_link(pci);
+
+ if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_LUT))
+ imx_pcie_lut_restore(imx_pcie);
+ imx_pcie_msi_save_restore(imx_pcie, false);
+
+ dev_dbg(pci->dev, "Root port reset completed\n");
+ return 0;
+}
+
static int imx_pcie_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -1919,15 +2020,46 @@ static int imx_pcie_probe(struct platform_device *pdev)
val |= PCI_MSI_FLAGS_ENABLE;
dw_pcie_writew_dbi(pci, offset + PCI_MSI_FLAGS, val);
}
+
+ /* Get link event irq if it is present */
+ imx_pcie->lnk_intr = platform_get_irq_byname_optional(pdev, "intr");
+ if (imx_pcie->lnk_intr == -EPROBE_DEFER) {
+ ret = -EPROBE_DEFER;
+ goto err_host_deinit;
+ }
+ if (imx_pcie->lnk_intr > 0) {
+ ret = devm_request_threaded_irq(dev, imx_pcie->lnk_intr,
+ imx_pcie_lnk_irq_isr,
+ imx_pcie_lnk_irq_thread,
+ IRQF_SHARED,
+ "lnk", imx_pcie);
+ if (ret) {
+ dev_err_probe(dev, ret,
+ "unable to request LNK IRQ\n");
+ goto err_host_deinit;
+ }
+
+ regmap_set_bits(imx_pcie->iomuxc_gpr,
+ IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_EN);
+ }
}
return 0;
+
+err_host_deinit:
+ dw_pcie_host_deinit(&pci->pp);
+
+ return ret;
}
static void imx_pcie_shutdown(struct platform_device *pdev)
{
struct imx_pcie *imx_pcie = platform_get_drvdata(pdev);
+ if (imx_pcie->lnk_intr > 0)
+ regmap_clear_bits(imx_pcie->iomuxc_gpr, IMX95_LINK_INT_CTRL_STS,
+ IMX95_LINK_DOWN_INT_EN);
/* bring down link, so bootloader gets clean state in case of reboot */
imx_pcie_assert_core_reset(imx_pcie);
imx_pcie_assert_perst(imx_pcie, true);
--
2.34.1
^ permalink raw reply related
* [PATCH v7 1/3] dt-bindings: imx6q-pcie: Add optional intr/aer/pme interrupts for i.MX95
From: hongxing.zhu @ 2026-06-18 9:20 UTC (permalink / raw)
To: robh, krzk+dt, conor+dt, bhelgaas, frank.li, l.stach, lpieralisi,
kwilczynski, mani, s.hauer, kernel, festevam
Cc: linux-pci, linux-arm-kernel, devicetree, imx, linux-kernel,
Richard Zhu, Frank Li
In-Reply-To: <20260618092100.3669556-1-hongxing.zhu@oss.nxp.com>
From: Richard Zhu <hongxing.zhu@nxp.com>
The i.MX95 PCIe controller introduces three additional dedicated hardware
interrupt lines for specific events:
- intr: general controller events
- aer: Advanced Error Reporting events
- pme: Power Management Events
These interrupts are optional on i.MX95. PCIe basic functionality
(enumeration, configuration, and data transfer) works correctly without
them, as the controller can operate using only the existing msi interrupt.
Earlier i.MX PCIe variants (imx6q, imx6sx, imx6qp, imx7d, imx8mm, imx8mp,
imx8mq, imx8q) do not have these three dedicated interrupt lines.
Update the binding to allow up to 5 interrupts for i.MX95, while
restricting earlier variants to a maximum of 2 interrupts using
conditional constraints (if/then schema). This ensures the schema
accurately reflects the hardware capabilities of each SoC variant.
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
---
.../bindings/pci/fsl,imx6q-pcie.yaml | 25 +++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml b/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml
index e8b8131f5f23b..4f56e8e4f1008 100644
--- a/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml
@@ -58,12 +58,18 @@ properties:
items:
- description: builtin MSI controller.
- description: builtin DMA controller.
+ - description: PCIe event interrupt.
+ - description: builtin AER SPI standalone interrupt line.
+ - description: builtin PME SPI standalone interrupt line.
interrupt-names:
minItems: 1
items:
- const: msi
- const: dma
+ - const: intr
+ - const: aer
+ - const: pme
reset-gpio:
deprecated: true
@@ -249,6 +255,25 @@ allOf:
- const: ref
- const: extref # Optional
+ - if:
+ properties:
+ compatible:
+ enum:
+ - fsl,imx6q-pcie
+ - fsl,imx6sx-pcie
+ - fsl,imx6qp-pcie
+ - fsl,imx7d-pcie
+ - fsl,imx8mm-pcie
+ - fsl,imx8mp-pcie
+ - fsl,imx8mq-pcie
+ - fsl,imx8q-pcie
+ then:
+ properties:
+ interrupts:
+ maxItems: 2
+ interrupt-names:
+ maxItems: 2
+
unevaluatedProperties: false
examples:
--
2.34.1
^ permalink raw reply related
* [PATCH v7 0/3] Add root port reset to support link recovery
From: hongxing.zhu @ 2026-06-18 9:20 UTC (permalink / raw)
To: robh, krzk+dt, conor+dt, bhelgaas, frank.li, l.stach, lpieralisi,
kwilczynski, mani, s.hauer, kernel, festevam
Cc: linux-pci, linux-arm-kernel, devicetree, imx, linux-kernel
Based on the following patch-set[1] issued by Mani.
Add support for resetting the Root Port for i.MX PCIe to enable link recovery.
[1] [PATCH v8 0/5] PCI: Add support for resetting the Root Ports in a platform specific way
PCIe links can go down due to various unexpected circumstances. This patch series
adds root port reset support for link recovery on i.MX PCIe controllers when the
optional "intr" interrupt is present.
When a link down event is detected, the root port reset uninitializes and
reinitializes the PCIe controller, then restarts the PCIe link.
On i.MX95 platforms, link events and PME share the same interrupt line.
Link event interrupts cannot use only an IRQ thread handler because the PME
driver uses request_irq() to bind the PME interrupt directly with only the
IRQF_SHARED flag set.
To address this, we register one handler with IRQF_SHARED for link event
interrupts and manipulate the enable bits of link events to ensure the same
interrupt source is triggered only once at a time.
Additionally, this series adds 'intr', 'aer', and 'pme' interrupt entries to
the i.MX6Q PCIe binding to support PCIe event-based interrupts for general
controller events, Advanced Error Reporting, and Power Management Events
respectively.
Changes in v7:
- Remove the redundant maxItem setting of interrupt property.
- Update driver codes refer to sashiko-reviews
Changes in v6:
- Use conditional constraints (if/then schema) to specify that these three
optional interrupts are only valid for the i.MX95 variant, while other
variants like imx6q should not have them.
- Change lnk_intr data type from u32 to int to properly handle negative
error codes returned by platform_get_irq_byname_optional().
- Replace platform_get_irq_byname() with platform_get_irq_byname_optional()
to suppress unnecessary error messages when the optional link event IRQ is
not present in the device tree.
- To avoid inadvertently clear the pending W1C status bit, clear the W1C
bit firstly, then do the regmap_clear_bits().
Changes in v5:
- Update the commit message of the first dt-binding patch for clarity.
- Add explicit comment explaining that writing 1 to IMX95_LINK_DOWN_INT_STS
clears the bit
Changes in v4:
- Set these new added three interrupts as optional interrupt.
Changes in v3:
- Don't add a new if:block; Drop the maxItems constraint of the interrupts
property for i.MX95 PCIe.
- Add constraints for the interrupts property for other variants.
- Regarding the ABI break: add descriptions explaining why these new
interrupts are mandatory and required by i.MX95 PCIe.
Changes in v2:
- Constrain the new added three interrupt entries to be valid only for the
i.MX95 variant using conditional schemas
[PATCH v7 1/3] dt-bindings: imx6q-pcie: Add optional intr/aer/pme
[PATCH v7 2/3] arm64: dts: imx95: Add dma, intr, aer and pme
[PATCH v7 3/3] PCI: imx6: Add root port reset to support link
Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.yaml | 25 +++++++++++++++++
arch/arm64/boot/dts/freescale/imx95.dtsi | 16 ++++++++---
drivers/pci/controller/dwc/pci-imx6.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 169 insertions(+), 4 deletions(-)
^ permalink raw reply
* Re: [PATCH net] net: ethernet: ti: icssg: guard PA stat lookups
From: Simon Horman @ 2026-06-18 9:10 UTC (permalink / raw)
To: Philippe Schenker
Cc: netdev, Philippe Schenker, danishanwar, rogerq, linux-arm-kernel,
stable, Andrew Lunn, David Carlier, David S. Miller, Eric Dumazet,
Jacob Keller, Jakub Kicinski, Kevin Hao, Meghana Malladi,
Paolo Abeni, Vadim Fedorenko, linux-kernel
In-Reply-To: <20260616143642.1972071-1-dev@pschenker.ch>
On Tue, Jun 16, 2026 at 04:35:34PM +0200, Philippe Schenker wrote:
> From: Philippe Schenker <philippe.schenker@impulsing.ch>
>
> icssg_ndo_get_stats64() unconditionally calls emac_get_stat_by_name()
> with FW PA stat names regardless of whether the PA stats block is
> present on the hardware. emac_get_stat_by_name() already guards the
> PA stats lookup with `if (emac->prueth->pa_stats)`; when that pointer
> is NULL the lookup falls through to netdev_err() and returns -EINVAL.
> Because ndo_get_stats64 is polled regularly by the networking stack
> this produces thousands of log entries of the form:
>
> icssg-prueth icssg1-eth end0: Invalid stats FW_RX_ERROR
>
> A secondary consequence is that the int(-EINVAL) return value is
> implicitly widened to a near-ULLONG_MAX unsigned value when accumulated
> into the __u64 fields of rtnl_link_stats64, silently corrupting the
> rx_errors, rx_dropped and tx_dropped counters reported by `ip -s link`.
>
> Every other PA-aware code path in the driver is already guarded with
> the same `if (emac->prueth->pa_stats)` check. Apply the same guard
> here.
>
> Fixes: 0d15a26b247d ("net: ti: icssg-prueth: Add ICSSG FW Stats")
nit: no blank line between tags
>
> Signed-off-by: Philippe Schenker <philippe.schenker@impulsing.ch>
>
> Cc: danishanwar@ti.com
> Cc: rogerq@kernel.org
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: stable@vger.kernel.org
Reviewed-by: Simon Horman <horms@kernel.org>
^ permalink raw reply
* Re: [RFC PATCH v2 1/3] mm/huge_memory: make persistent huge zero folio read-only
From: David Hildenbrand (Arm) @ 2026-06-18 9:06 UTC (permalink / raw)
To: Xueyuan Chen
Cc: dave.hansen, akpm, linux-mm, linux-kernel, linux-arm-kernel, x86,
catalin.marinas, will, tglx, mingo, bp, dave.hansen, luto, peterz,
hpa, ljs, liam, vbabka, rppt, surenb, mhocko, ziy, baolin.wang,
npache, ryan.roberts, dev.jain, baohua, lance.yang, yang, jannh
In-Reply-To: <20260617141547.144275-1-xueyuan.chen21@gmail.com>
On 6/17/26 16:15, Xueyuan Chen wrote:
>
> On Wed, Jun 17, 2026 at 01:50:08PM +0200, David Hildenbrand (Arm) wrote:
>
> Hi, David
>
>> Yes, kerneldoc please.
>
> Ack.
>
>>
>> We're adjusting the directmap, remapping a r/w page to be r/o. I think we should
>> be very clear about which transition we expect+support.
>>
>> Also, I rather hate the "set_memory" naming scheme ... "set_direct_map" is
>> clearer. Anyhow ...
>>
>> Now we are throwing a "arch_make_pages_*" into the mix.
>>
>> Should it really contain the "arch"?
>> Should it really contain the "make" ?
>>
>> Why can't we just reuse set_memory_ro and pass address+nr_pages? (highmem check?
>> Could that be moved in there?)
>>
>> Or do we want a "change_direct_map_ro()" / "remap_direct_map_ro" interface?
>>
>>
>
> How about naming it int set_direct_map_ro(struct page *page, unsigned nr)?
To distinguish it from "set_memory*" cruft, maybe best to use "remap" or
"adjust" instead.
--
Cheers,
David
^ permalink raw reply
* [PATCH] iommu/io-pgtable-arm: Add support for contiguous hint bit
From: Vijayanand Jitta @ 2026-06-18 9:02 UTC (permalink / raw)
To: Joerg Roedel (AMD), Will Deacon, Robin Murphy
Cc: linux-arm-msm, iommu, linux-kernel, linux-arm-kernel,
Prakash Gupta, Vijayanand Jitta
From: Prakash Gupta <prakash.gupta@oss.qualcomm.com>
Add support for the contiguous hint (CONT) bit in ARM LPAE page tables.
When a set of consecutive PTEs map a naturally-aligned contiguous block
of memory, the CONT bit can be set on all entries in the group to allow
the hardware to combine them into a single TLB entry, improving TLB
utilization.
The contiguous hint sizes per granule are:
Page Size | CONT PTE | PMD | CONT PMD
----------+----------+-------+---------
4K | 64K | 2M | 32M
16K | 2M | 32M | 1G
64K | 2M | 512M | 16G
Contiguous hint sizes are advertised in pgsize_bitmap, analogous to
how the CPU MMU advertises them via hugetlb hstates, so that IOMMU API
users (e.g. __iommu_dma_alloc_pages()) can align allocations to these
sizes and benefit from the TLB optimization automatically.
Support is gated behind CONFIG_IOMMU_IO_PGTABLE_CONTIG_HINT, which
provides a compile-time opt-out for hardware affected by SMMU errata
related to the contiguous bit.
On the mapping side, __arm_lpae_map() detects when the requested size
matches a contiguous range at the next level, sets the CONT bit on all
PTEs in the group, then recurses with the base block size and an
adjusted pgcount.
On the unmapping side, the CONT bit is cleared from all PTEs in the
affected contiguous group before any individual entry is invalidated,
following the Break-Before-Make requirement of the architecture.
Tested on QEMU (arm64/SMMUv3) with iommu_map()/iommu_unmap() of
contiguous hint sizes; verified the CONT bit is correctly set on map
and cleared on unmap via page table walk.
Co-developed-by: Vijayanand Jitta <vijayanand.jitta@oss.qualcomm.com>
Signed-off-by: Vijayanand Jitta <vijayanand.jitta@oss.qualcomm.com>
Signed-off-by: Prakash Gupta <prakash.gupta@oss.qualcomm.com>
---
drivers/iommu/Kconfig | 16 +++
drivers/iommu/io-pgtable-arm.c | 216 +++++++++++++++++++++++++++++++++++++++--
2 files changed, 226 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 6e07bd69467a3..1c514361c5c9e 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -50,6 +50,22 @@ config IOMMU_IO_PGTABLE_LPAE_KUNIT_TEST
If unsure, say N here.
+config IOMMU_IO_PGTABLE_CONTIG_HINT
+ bool "Enable contiguous hint"
+ depends on IOMMU_IO_PGTABLE_LPAE
+ default y
+ help
+ Enable contiguous hint (CONT bit) support for the ARM LPAE page
+ table allocator. Contiguous hint sizes are advertised in the
+ pgsize_bitmap so that IOMMU API users can align allocations to
+ these sizes and benefit from improved TLB utilization, analogous
+ to how the CPU MMU advertises contiguous sizes via hugetlb.
+
+ Disabling this option provides a compile-time opt-out for
+ hardware affected by SMMU errata related to the contiguous bit.
+
+ If unsure, say Y here.
+
config IOMMU_IO_PGTABLE_ARMV7S
bool "ARMv7/v8 Short Descriptor Format"
select IOMMU_IO_PGTABLE
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 476c0e25631af..9fc60520177f1 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -86,6 +86,21 @@
/* Software bit for solving coherency races */
#define ARM_LPAE_PTE_SW_SYNC (((arm_lpae_iopte)1) << 55)
+/* PTE Contiguous Bit */
+#define ARM_LPAE_PTE_CONT (((arm_lpae_iopte)1) << 52)
+
+/*
+ * CONTIG HINT SUPPORT TABLE
+ *
+ *---------------------------------------------------
+ *| Page Size | CONT PTE | PMD | CONT PMD | PUD |
+ *---------------------------------------------------
+ *| 4K | 64K | 2M | 32M | 1G |
+ *| 16K | 2M | 32M | 1G | |
+ *| 64K | 2M | 512M | 16G | |
+ *---------------------------------------------------
+ */
+
/* Stage-1 PTE */
#define ARM_LPAE_PTE_AP_UNPRIV (((arm_lpae_iopte)1) << 6)
#define ARM_LPAE_PTE_AP_RDONLY_BIT 7
@@ -453,6 +468,111 @@ static arm_lpae_iopte arm_lpae_install_table(arm_lpae_iopte *table,
return old;
}
+#ifdef CONFIG_IOMMU_IO_PGTABLE_CONTIG_HINT
+static inline int arm_lpae_cont_ptes(unsigned long size)
+{
+ if (size == SZ_4K)
+ return 16;
+ if (size == SZ_16K)
+ return 128;
+ if (size == SZ_64K)
+ return 32;
+ return 1;
+}
+
+static inline unsigned long arm_lpae_cont_pte_size(unsigned long size)
+{
+ return arm_lpae_cont_ptes(size) * size;
+}
+
+static inline int arm_lpae_cont_pmds(unsigned long size)
+{
+ if (size == SZ_2M)
+ return 16;
+ if (size == SZ_32M)
+ return 32;
+ if (size == SZ_512M)
+ return 32;
+ return 1;
+}
+
+static inline unsigned long arm_lpae_cont_pmd_size(unsigned long size)
+{
+ return arm_lpae_cont_pmds(size) * size;
+}
+
+static unsigned long arm_lpae_get_cont_sizes(struct io_pgtable_cfg *cfg)
+{
+ unsigned long pg_size, pmd_size;
+ int pg_shift, bits_per_level;
+
+ if (!cfg->pgsize_bitmap)
+ return 0;
+
+ pg_shift = __ffs(cfg->pgsize_bitmap);
+ bits_per_level = pg_shift - ilog2(sizeof(arm_lpae_iopte));
+ pg_size = (1UL << pg_shift);
+ pmd_size = (pg_size << bits_per_level);
+
+ return (arm_lpae_cont_pte_size(pg_size) | arm_lpae_cont_pmd_size(pmd_size));
+}
+
+static u32 arm_lpae_find_num_cont(struct arm_lpae_io_pgtable *data, int lvl)
+{
+ if (lvl == ARM_LPAE_MAX_LEVELS - 2)
+ return arm_lpae_cont_pmds(ARM_LPAE_BLOCK_SIZE(lvl, data));
+ else if (lvl == ARM_LPAE_MAX_LEVELS - 1)
+ return arm_lpae_cont_ptes(ARM_LPAE_BLOCK_SIZE(lvl, data));
+ else
+ return 1;
+}
+
+static u32 arm_lpae_check_num_cont(struct arm_lpae_io_pgtable *data, size_t size, int lvl)
+{
+ int num_cont;
+
+ num_cont = arm_lpae_find_num_cont(data, lvl);
+ if (size == num_cont * ARM_LPAE_BLOCK_SIZE(lvl, data))
+ return num_cont;
+ else
+ return 1;
+}
+
+static bool arm_lpae_pte_is_contiguous_range(struct arm_lpae_io_pgtable *data,
+ unsigned long size,
+ int lvl, u32 *num_cont)
+{
+ unsigned long block_size;
+
+ *num_cont = arm_lpae_find_num_cont(data, lvl);
+ block_size = ARM_LPAE_BLOCK_SIZE(lvl, data);
+
+ return (size == ((*num_cont) * block_size));
+}
+#else
+static unsigned long arm_lpae_get_cont_sizes(struct io_pgtable_cfg *cfg)
+{
+ return 0;
+}
+
+static u32 arm_lpae_find_num_cont(struct arm_lpae_io_pgtable *data, int lvl)
+{
+ return 1;
+}
+
+static u32 arm_lpae_check_num_cont(struct arm_lpae_io_pgtable *data, size_t size, int lvl)
+{
+ return 1;
+}
+
+static bool arm_lpae_pte_is_contiguous_range(struct arm_lpae_io_pgtable *data,
+ unsigned long size,
+ int lvl, u32 *num_cont)
+{
+ return false;
+}
+#endif
+
static int __arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova,
phys_addr_t paddr, size_t size, size_t pgcount,
arm_lpae_iopte prot, int lvl, arm_lpae_iopte *ptep,
@@ -463,6 +583,7 @@ static int __arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova,
size_t tblsz = ARM_LPAE_GRANULE(data);
struct io_pgtable_cfg *cfg = &data->iop.cfg;
int ret = 0, num_entries, max_entries, map_idx_start;
+ u32 num_cont = 1;
/* Find our entry at the current level */
map_idx_start = ARM_LPAE_LVL_IDX(iova, lvl, data);
@@ -505,6 +626,24 @@ static int __arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova,
return -EEXIST;
}
+ if (arm_lpae_pte_is_contiguous_range(data, size, lvl + 1, &num_cont)) {
+ size_t ct_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+
+ /* Set cont bit */
+ prot |= ARM_LPAE_PTE_CONT;
+
+ /*
+ * Since size here would be of CONT_PTE or CONT_PMD (e.g. SZ_64K/SZ_32M
+ * in case of 4K PAGE_SIZE), but actual mappings are in multiples of
+ * SZ_4K/SZ_2M, call __arm_lpae_map with ct_size and update pgcount
+ * accordingly by num_cont * pgcount.
+ */
+ ret = __arm_lpae_map(data, iova, paddr, ct_size,
+ num_cont * pgcount,
+ prot, lvl + 1, cptep, gfp, mapped);
+ return ret;
+ }
+
/* Rinse, repeat */
return __arm_lpae_map(data, iova, paddr, size, pgcount, prot, lvl + 1,
cptep, gfp, mapped);
@@ -653,6 +792,48 @@ static void arm_lpae_free_pgtable(struct io_pgtable *iop)
kfree(data);
}
+#ifdef CONFIG_IOMMU_IO_PGTABLE_CONTIG_HINT
+static void arm_lpae_cont_clear(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, int lvl,
+ arm_lpae_iopte *ptep, size_t num_entries)
+{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ u32 num_cont = arm_lpae_find_num_cont(data, lvl);
+ arm_lpae_iopte *cont_ptep;
+ arm_lpae_iopte *cont_ptep_start;
+ unsigned long cont_iova;
+ int offset, itr;
+
+ cont_ptep = ptep - ARM_LPAE_LVL_IDX(iova, lvl, data);
+ cont_iova = round_down(iova,
+ ARM_LPAE_BLOCK_SIZE(lvl, data) * num_cont);
+ cont_ptep += ARM_LPAE_LVL_IDX(cont_iova, lvl, data);
+ cont_ptep_start = cont_ptep;
+
+ /*
+ * iova may not be aligned to the contiguous group boundary; include
+ * any leading entries so round_up() covers all overlapping groups.
+ */
+ offset = ARM_LPAE_LVL_IDX(iova, lvl, data) -
+ ARM_LPAE_LVL_IDX(cont_iova, lvl, data);
+ num_entries = round_up(offset + num_entries, num_cont);
+
+ for (itr = 0; itr < num_entries; itr++) {
+ WRITE_ONCE(*cont_ptep, READ_ONCE(*cont_ptep) & ~ARM_LPAE_PTE_CONT);
+ cont_ptep++;
+ }
+
+ if (!cfg->coherent_walk)
+ __arm_lpae_sync_pte(cont_ptep_start, num_entries, cfg);
+}
+#else
+static void arm_lpae_cont_clear(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, int lvl,
+ arm_lpae_iopte *ptep, size_t num_entries)
+{
+}
+#endif
+
static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data,
struct iommu_iotlb_gather *gather,
unsigned long iova, size_t size, size_t pgcount,
@@ -660,7 +841,7 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data,
{
arm_lpae_iopte pte;
struct io_pgtable *iop = &data->iop;
- int i = 0, num_entries, max_entries, unmap_idx_start;
+ int i = 0, num_cont = 1, num_entries, max_entries, unmap_idx_start;
/* Something went horribly wrong and we ran out of page table */
if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
@@ -675,9 +856,15 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data,
}
/* If the size matches this level, we're in the right place */
- if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data) ||
+ (size == arm_lpae_find_num_cont(data, lvl) *
+ ARM_LPAE_BLOCK_SIZE(lvl, data))) {
+ size_t pte_size;
+
max_entries = arm_lpae_max_entries(unmap_idx_start, data);
- num_entries = min_t(int, pgcount, max_entries);
+ num_cont = arm_lpae_check_num_cont(data, size, lvl);
+ num_entries = min_t(int, num_cont * pgcount, max_entries);
+ pte_size = size / num_cont;
/* Find and handle non-leaf entries */
for (i = 0; i < num_entries; i++) {
@@ -687,11 +874,27 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data,
break;
}
+ /*
+ * Break-Before-Make: before invalidating any leaf
+ * entry, clear the CONT bit from every entry in the
+ * contiguous group(s) and flush the TLB, as required
+ * by the architecture. arm_lpae_cont_clear() covers
+ * the full [iova, iova + num_entries * pte_size) range
+ * via round_up(), so subsequent entries read back
+ * CONT=0 and skip this block.
+ */
+ if (pte & ARM_LPAE_PTE_CONT) {
+ arm_lpae_cont_clear(data, iova, lvl, ptep, num_entries);
+ io_pgtable_tlb_flush_walk(iop, iova,
+ num_entries * pte_size,
+ ARM_LPAE_GRANULE(data));
+ }
+
if (!iopte_leaf(pte, lvl, iop->fmt)) {
__arm_lpae_clear_pte(&ptep[i], &iop->cfg, 1);
/* Also flush any partial walks */
- io_pgtable_tlb_flush_walk(iop, iova + i * size, size,
+ io_pgtable_tlb_flush_walk(iop, iova + i * pte_size, pte_size,
ARM_LPAE_GRANULE(data));
__arm_lpae_free_pgtable(data, lvl + 1, iopte_deref(pte, data));
}
@@ -702,9 +905,9 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data,
if (gather && !iommu_iotlb_gather_queued(gather))
for (int j = 0; j < i; j++)
- io_pgtable_tlb_add_page(iop, gather, iova + j * size, size);
+ io_pgtable_tlb_add_page(iop, gather, iova + j * pte_size, pte_size);
- return i * size;
+ return i * pte_size;
} else if (iopte_leaf(pte, lvl, iop->fmt)) {
WARN_ONCE(true, "Unmap of a partial large IOPTE is not allowed");
return 0;
@@ -943,6 +1146,7 @@ static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
}
cfg->pgsize_bitmap &= page_sizes;
+ cfg->pgsize_bitmap |= arm_lpae_get_cont_sizes(cfg);
cfg->ias = min(cfg->ias, max_addr_bits);
cfg->oas = min(cfg->oas, max_addr_bits);
}
---
base-commit: 4fa3f5fabb30bf00d7475d5a33459ea83d639bf9
change-id: 20260618-iommu_contig_hint-71ae491fbb52
Best regards,
--
Vijayanand Jitta <vijayanand.jitta@oss.qualcomm.com>
^ permalink raw reply related
* [PATCH 2/3] KVM: arm64: Remove unreachable early checks in pkvm_init_host_vm()
From: Fuad Tabba @ 2026-06-18 9:01 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton, Catalin Marinas, Will Deacon
Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu,
Vincent Donnefort, Keir Fraser, Hyunwoo Kim, Fuad Tabba,
linux-arm-kernel, kvmarm, linux-kernel
In-Reply-To: <20260618090128.3913688-1-tabba@google.com>
pkvm_init_host_vm() runs once from kvm_arch_init_vm(), while the VM is
still being allocated and is not yet reachable by another thread. Both
early checks therefore test impossible state: is_created is still false
(it is only set on first vCPU run) and the handle is still zero (this
function is what reserves it). Neither branch can be taken.
Remove them.
Signed-off-by: Fuad Tabba <tabba@google.com>
---
arch/arm64/kvm/pkvm.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 053e4f733e4b..67b90a58fbea 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -230,13 +230,6 @@ int pkvm_init_host_vm(struct kvm *kvm, unsigned long type)
int ret;
bool protected = type & KVM_VM_TYPE_ARM_PROTECTED;
- if (pkvm_hyp_vm_is_created(kvm))
- return -EINVAL;
-
- /* VM is already reserved, no need to proceed. */
- if (kvm->arch.pkvm.handle)
- return 0;
-
/* Reserve the VM in hyp and obtain a hyp handle for the VM. */
ret = kvm_call_hyp_nvhe(__pkvm_reserve_vm);
if (ret < 0)
--
2.54.0.1189.g8c84645362-goog
^ permalink raw reply related
* [PATCH 3/3] KVM: arm64: Drop redundant READ_ONCE() in pkvm_hyp_vm_is_created()
From: Fuad Tabba @ 2026-06-18 9:01 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton, Catalin Marinas, Will Deacon
Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu,
Vincent Donnefort, Keir Fraser, Hyunwoo Kim, Fuad Tabba,
linux-arm-kernel, kvmarm, linux-kernel
In-Reply-To: <20260618090128.3913688-1-tabba@google.com>
is_created is written under config_lock. Every concurrent reader is
serialised against that write: pkvm_create_hyp_vm() under config_lock,
and the memslot path (kvm_arch_prepare_memory_region) via slots_lock,
which the creation writer also holds. The teardown-path accesses have no
concurrent writer. The read is therefore serialised, and the READ_ONCE()
is unnecessary.
Signed-off-by: Fuad Tabba <tabba@google.com>
---
arch/arm64/kvm/pkvm.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 67b90a58fbea..008766273912 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -185,7 +185,11 @@ static int __pkvm_create_hyp_vm(struct kvm *kvm)
bool pkvm_hyp_vm_is_created(struct kvm *kvm)
{
- return READ_ONCE(kvm->arch.pkvm.is_created);
+ /*
+ * Serialised by config_lock/slots_lock, or by VM lifecycle at
+ * teardown, so a plain read suffices.
+ */
+ return kvm->arch.pkvm.is_created;
}
int pkvm_create_hyp_vm(struct kvm *kvm)
--
2.54.0.1189.g8c84645362-goog
^ permalink raw reply related
* [PATCH 1/3] KVM: arm64: Drop the unused EL2-side is_created write
From: Fuad Tabba @ 2026-06-18 9:01 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton, Catalin Marinas, Will Deacon
Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu,
Vincent Donnefort, Keir Fraser, Hyunwoo Kim, Fuad Tabba,
linux-arm-kernel, kvmarm, linux-kernel
In-Reply-To: <20260618090128.3913688-1-tabba@google.com>
init_pkvm_hyp_vm() sets is_created on the EL2-private VM struct, but the
hypervisor never reads it: pkvm_hyp_vm_is_created() and every other
consumer operate on the host's struct kvm, a distinct allocation from
the EL2-private copy. The field is write-only at EL2.
Remove the store; host-side is_created tracking is unaffected.
Signed-off-by: Fuad Tabba <tabba@google.com>
---
arch/arm64/kvm/hyp/nvhe/pkvm.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index eb1c10120f9f..30dd4b2afc26 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -433,7 +433,6 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp_vm,
hyp_vm->host_kvm = host_kvm;
hyp_vm->kvm.created_vcpus = nr_vcpus;
hyp_vm->kvm.arch.pkvm.is_protected = READ_ONCE(host_kvm->arch.pkvm.is_protected);
- hyp_vm->kvm.arch.pkvm.is_created = true;
hyp_vm->kvm.arch.flags = 0;
pkvm_init_features_from_host(hyp_vm, host_kvm);
--
2.54.0.1189.g8c84645362-goog
^ permalink raw reply related
* [PATCH 0/3] KVM: arm64: pKVM is_created cleanup
From: Fuad Tabba @ 2026-06-18 9:01 UTC (permalink / raw)
To: Marc Zyngier, Oliver Upton, Catalin Marinas, Will Deacon
Cc: Joey Gouly, Steffen Eiden, Suzuki K Poulose, Zenghui Yu,
Vincent Donnefort, Keir Fraser, Hyunwoo Kim, Fuad Tabba,
linux-arm-kernel, kvmarm, linux-kernel
This small series tidies up the host-side kvm->arch.pkvm.is_created flag,
which tracks whether the hypervisor-side (EL2) VM has been instantiated.
It comes out of the ongoing pKVM (protected KVM) upstreaming work and runs
in parallel with it. The changes only remove dead or redundant code around
the flag, not any of the functional paths that work touches, so there is no
dependency in either direction and the two can be applied in any order.
is_created stays: the pKVM handle is reserved early (so host MMU-notifier
TLB invalidations have a valid handle before the first vCPU run), so a
non-zero handle no longer implies the EL2 VM exists. is_created is what
distinguishes "reserved" from "created", and the teardown path relies on it.
Only the cruft around it goes.
Cheers,
/fuad
Fuad Tabba (3):
KVM: arm64: Drop the unused EL2-side is_created write
KVM: arm64: Remove unreachable early checks in pkvm_init_host_vm()
KVM: arm64: Drop redundant READ_ONCE() in pkvm_hyp_vm_is_created()
arch/arm64/kvm/hyp/nvhe/pkvm.c | 1 -
arch/arm64/kvm/pkvm.c | 13 +++++--------
2 files changed, 5 insertions(+), 9 deletions(-)
--
2.54.0.1189.g8c84645362-goog
^ permalink raw reply
* Re: [PATCH v4 resend 2/5] reset: cix: add audss support to sky1 reset driver
From: Philipp Zabel @ 2026-06-18 8:49 UTC (permalink / raw)
To: joakim.zhang, mturquette, sboyd, bmasney, robh, krzk+dt, conor+dt,
gary.yang
Cc: cix-kernel-upstream, linux-clk, devicetree, linux-kernel,
linux-arm-kernel
In-Reply-To: <20260617064100.1504617-3-joakim.zhang@cixtech.com>
On Mi, 2026-06-17 at 14:40 +0800, joakim.zhang@cixtech.com wrote:
> From: Joakim Zhang <joakim.zhang@cixtech.com>
>
> Extend the Sky1 reset controller driver for the AUDSS CRU syscon. The
> AUDSS block provides sixteen active-low software reset bits in one
> register for audio subsystem peripherals, reusing the existing
> regmap-based reset ops used by the FCH and S5 system control variants.
>
> Signed-off-by: Joakim Zhang <joakim.zhang@cixtech.com>
> ---
> drivers/reset/reset-sky1.c | 86 ++++++++++++++++++++++++++++++++++++--
> 1 file changed, 83 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/reset/reset-sky1.c b/drivers/reset/reset-sky1.c
> index 78e80a533c39..af32ee005ebc 100644
> --- a/drivers/reset/reset-sky1.c
> +++ b/drivers/reset/reset-sky1.c
[...]
> @@ -343,21 +379,65 @@ static int sky1_reset_probe(struct platform_device *pdev)
> sky1src->rcdev.of_node = dev->of_node;
> sky1src->rcdev.dev = dev;
>
> - return devm_reset_controller_register(dev, &sky1src->rcdev);
> + ret = devm_reset_controller_register(dev, &sky1src->rcdev);
> + if (ret)
> + return ret;
> +
> + platform_set_drvdata(pdev, sky1src);
> +
> + if (of_device_is_compatible(dev->of_node, "cix,sky1-audss-system-control")) {
The compatible was already evaluated by of_device_get_match_data(), you
could check (variant == &variant_sky1_audss) here.
regards
Philipp
^ permalink raw reply
* [PATCH v4 6/6] mm/vmalloc: align vm_area so vmap() can batch mappings
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
In-Reply-To: <20260618084726.1070022-1-jiangwen6@xiaomi.com>
From: "Barry Song (Xiaomi)" <baohua@kernel.org>
Try to align the vmap virtual address to PMD_SHIFT or a
larger PTE mapping size hinted by the architecture, so
contiguous pages can be batch-mapped when setting PMD or
PTE entries.
Add __get_vm_area_node_aligned_caller() as a wrapper over
__get_vm_area_node() to simplify repeated calls with fixed
arguments.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com>
Tested-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
mm/vmalloc.c | 37 ++++++++++++++++++++++++++++++++++++-
1 file changed, 36 insertions(+), 1 deletion(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index fffb885cb2158..bc9fa93e2bdc6 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -3628,6 +3628,41 @@ static int vmap_batched(unsigned long addr, unsigned long end,
return err;
}
+static struct vm_struct *__get_vm_area_node_aligned_caller(unsigned long size,
+ unsigned long align, unsigned long flags, const void *caller)
+{
+ return __get_vm_area_node(size, align, PAGE_SHIFT, flags,
+ VMALLOC_START, VMALLOC_END,
+ NUMA_NO_NODE, GFP_KERNEL, caller);
+}
+
+static struct vm_struct *vmap_get_aligned_vm_area(unsigned long size,
+ unsigned long flags, const void *caller)
+{
+ struct vm_struct *vm_area;
+ unsigned int shift;
+
+ /* Try PMD alignment for large sizes */
+ if (size >= PMD_SIZE) {
+ vm_area = __get_vm_area_node_aligned_caller(size, PMD_SIZE,
+ flags, caller);
+ if (vm_area)
+ return vm_area;
+ }
+
+ /* Try CONT_PTE alignment */
+ shift = arch_vmap_pte_supported_shift(size);
+ if (shift > PAGE_SHIFT) {
+ vm_area = __get_vm_area_node_aligned_caller(size, 1UL << shift,
+ flags, caller);
+ if (vm_area)
+ return vm_area;
+ }
+
+ /* Fall back to page alignment */
+ return __get_vm_area_node_aligned_caller(size, PAGE_SIZE, flags, caller);
+}
+
/**
* vmap - map an array of pages into virtually contiguous space
* @pages: array of page pointers
@@ -3666,7 +3701,7 @@ void *vmap(struct page **pages, unsigned int count,
return NULL;
size = (unsigned long)count << PAGE_SHIFT;
- area = get_vm_area_caller(size, flags, __builtin_return_address(0));
+ area = vmap_get_aligned_vm_area(size, flags, __builtin_return_address(0));
if (!area)
return NULL;
--
2.34.1
^ permalink raw reply related
* [PATCH v4 5/6] mm/vmalloc: map contiguous pages in batches for vmap() if possible
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
In-Reply-To: <20260618084726.1070022-1-jiangwen6@xiaomi.com>
From: "Barry Song (Xiaomi)" <baohua@kernel.org>
In many cases, the pages passed to vmap() may include high-order
pages. For example, the systemheap often allocates pages in descending
order: order 8, then 4, then 0. Currently, vmap() iterates over every
page individually—even pages inside a high-order block are handled
one by one.
This patch detects physically contiguous pages (regardless of whether
they are compound or non-compound) by scanning with
num_pages_contiguous(), and maps them as a single contiguous block
whenever possible. The mapping order is determined by taking the
minimum of the contiguous page count and the pfn alignment, allowing
graceful degradation when pfn alignment is less than the contiguous
range.
Pages with the same page_shift are coalesced and mapped via
vmap_pages_range_noflush_walk() to avoid page table rewalk.
As users typically allocate memory in descending orders (e.g.
8 → 4 → 0), once an order-0 page is encountered, we stop scanning
for contiguous pages since subsequent pages are likely order-0 as well.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
Co-developed-by: Dev Jain <dev.jain@arm.com>
Signed-off-by: Dev Jain <dev.jain@arm.com>
Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com>
Tested-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
mm/vmalloc.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 85 insertions(+), 2 deletions(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 253e017130e09..fffb885cb2158 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -3545,6 +3545,89 @@ void vunmap(const void *addr)
}
EXPORT_SYMBOL(vunmap);
+static inline unsigned int vm_shift(pgprot_t prot, unsigned long size)
+{
+ if (arch_vmap_pmd_supported(prot) && size >= PMD_SIZE)
+ return PMD_SHIFT;
+
+ return arch_vmap_pte_supported_shift(size);
+}
+
+static inline int get_vmap_batch_order(struct page **pages,
+ pgprot_t prot, unsigned int max_steps, unsigned int idx)
+{
+ unsigned int nr_contig;
+ int order;
+
+ if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+ return 0;
+
+ nr_contig = num_pages_contiguous(&pages[idx], max_steps);
+ if (nr_contig < 2)
+ return 0;
+
+ order = ilog2(nr_contig);
+
+ /* Limit order by pfn alignment */
+ order = min_t(int, order, __ffs(page_to_pfn(pages[idx])));
+
+ if (vm_shift(prot, PAGE_SIZE << order) == PAGE_SHIFT)
+ return 0;
+
+ return order;
+}
+
+static int vmap_batched(unsigned long addr, unsigned long end,
+ pgprot_t prot, struct page **pages)
+{
+ unsigned int count = (end - addr) >> PAGE_SHIFT;
+ unsigned int prev_shift = 0, idx = 0;
+ unsigned long start = addr, map_addr = addr;
+ int err;
+
+ err = kmsan_vmap_pages_range_noflush(addr, end, prot, pages,
+ PAGE_SHIFT, GFP_KERNEL);
+ if (err)
+ goto out;
+
+ for (unsigned int i = 0; i < count; ) {
+ unsigned int shift = PAGE_SHIFT +
+ get_vmap_batch_order(pages, prot, count - i, i);
+
+ if (!i)
+ prev_shift = shift;
+
+ if (shift != prev_shift) {
+ err = vmap_pages_range_noflush_walk(map_addr, addr,
+ prot, pages + idx, prev_shift);
+ if (err)
+ goto out;
+ prev_shift = shift;
+ map_addr = addr;
+ idx = i;
+ }
+
+ /*
+ * Once small pages are encountered, the remaining pages
+ * are likely small as well.
+ */
+ if (shift == PAGE_SHIFT)
+ break;
+
+ addr += 1UL << shift;
+ i += 1U << (shift - PAGE_SHIFT);
+ }
+
+ /* Remaining */
+ if (map_addr < end)
+ err = vmap_pages_range_noflush_walk(map_addr, end,
+ prot, pages + idx, prev_shift);
+
+out:
+ flush_cache_vmap(start, end);
+ return err;
+}
+
/**
* vmap - map an array of pages into virtually contiguous space
* @pages: array of page pointers
@@ -3588,8 +3671,8 @@ void *vmap(struct page **pages, unsigned int count,
return NULL;
addr = (unsigned long)area->addr;
- if (vmap_pages_range(addr, addr + size, pgprot_nx(prot),
- pages, PAGE_SHIFT) < 0) {
+ if (vmap_batched(addr, addr + size, pgprot_nx(prot),
+ pages) < 0) {
vunmap(area->addr);
return NULL;
}
--
2.34.1
^ permalink raw reply related
* [PATCH v4 4/6] mm/vmalloc: Extend page table walk to support larger page_shift sizes and eliminate page table rewalk
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
In-Reply-To: <20260618084726.1070022-1-jiangwen6@xiaomi.com>
From: "Barry Song (Xiaomi)" <baohua@kernel.org>
vmap_pages_range_noflush_walk() (formerly vmap_small_pages_range_noflush())
provides a clean interface by taking struct page **pages and mapping them
via direct PTE iteration. This avoids the page table rewalk seen when
using vmap_range_noflush() for page_shift values other than PAGE_SHIFT.
Extend it to support larger page_shift values, and add PMD- and
contiguous-PTE mappings as well. Rename it to vmap_pages_range_noflush_walk()
since it now handles more than just small pages.
For vmalloc() allocations with VM_ALLOW_HUGE_VMAP, we no longer need to
iterate over pages one by one via vmap_range_noflush(), which would
otherwise lead to page table rewalk. The code is now unified with the
PAGE_SHIFT case by simply calling vmap_pages_range_noflush_walk().
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com>
Tested-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
mm/vmalloc.c | 81 ++++++++++++++++++++++++++++++----------------------
1 file changed, 47 insertions(+), 34 deletions(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 6660f240d27c9..253e017130e09 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -127,7 +127,8 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
pte_t *pte;
u64 pfn;
struct page *page;
- unsigned long size = PAGE_SIZE;
+ unsigned long size;
+ unsigned int steps;
if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr)))
return -EINVAL;
@@ -149,8 +150,8 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
}
size = vmap_set_ptes(pte, addr, end, pfn, prot, max_page_shift);
- pfn += PFN_DOWN(size);
- } while (pte += PFN_DOWN(size), addr += size, addr != end);
+ steps = PFN_DOWN(size);
+ } while (pte += steps, pfn += steps, addr += size, addr != end);
lazy_mmu_mode_disable();
*mask |= PGTBL_PTE_MODIFIED;
@@ -542,8 +543,10 @@ void vunmap_range(unsigned long addr, unsigned long end)
static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr,
unsigned long end, pgprot_t prot, struct page **pages, int *nr,
- pgtbl_mod_mask *mask)
+ pgtbl_mod_mask *mask, unsigned int shift)
{
+ unsigned long pfn, size;
+ unsigned int steps;
int err = 0;
pte_t *pte;
@@ -574,9 +577,10 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr,
break;
}
- set_pte_at(&init_mm, addr, pte, mk_pte(page, prot));
- (*nr)++;
- } while (pte++, addr += PAGE_SIZE, addr != end);
+ pfn = page_to_pfn(page);
+ size = vmap_set_ptes(pte, addr, end, pfn, prot, shift);
+ steps = PFN_DOWN(size);
+ } while (pte += steps, *nr += steps, addr += size, addr != end);
lazy_mmu_mode_disable();
*mask |= PGTBL_PTE_MODIFIED;
@@ -586,7 +590,7 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr,
static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr,
unsigned long end, pgprot_t prot, struct page **pages, int *nr,
- pgtbl_mod_mask *mask)
+ pgtbl_mod_mask *mask, unsigned int shift)
{
pmd_t *pmd;
unsigned long next;
@@ -596,7 +600,27 @@ static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr,
return -ENOMEM;
do {
next = pmd_addr_end(addr, end);
- if (vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask))
+
+ if (shift >= PMD_SHIFT) {
+ struct page *page = pages[*nr];
+ phys_addr_t phys_addr;
+
+ if (WARN_ON(!page))
+ return -ENOMEM;
+ if (WARN_ON(!pfn_valid(page_to_pfn(page))))
+ return -EINVAL;
+
+ phys_addr = page_to_phys(page);
+
+ if (vmap_try_huge_pmd(pmd, addr, next, phys_addr, prot,
+ shift)) {
+ *mask |= PGTBL_PMD_MODIFIED;
+ *nr += 1 << (PMD_SHIFT - PAGE_SHIFT);
+ continue;
+ }
+ }
+
+ if (vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask, shift))
return -ENOMEM;
} while (pmd++, addr = next, addr != end);
return 0;
@@ -604,7 +628,7 @@ static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr,
static int vmap_pages_pud_range(p4d_t *p4d, unsigned long addr,
unsigned long end, pgprot_t prot, struct page **pages, int *nr,
- pgtbl_mod_mask *mask)
+ pgtbl_mod_mask *mask, unsigned int shift)
{
pud_t *pud;
unsigned long next;
@@ -614,7 +638,7 @@ static int vmap_pages_pud_range(p4d_t *p4d, unsigned long addr,
return -ENOMEM;
do {
next = pud_addr_end(addr, end);
- if (vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask))
+ if (vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask, shift))
return -ENOMEM;
} while (pud++, addr = next, addr != end);
return 0;
@@ -622,7 +646,7 @@ static int vmap_pages_pud_range(p4d_t *p4d, unsigned long addr,
static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr,
unsigned long end, pgprot_t prot, struct page **pages, int *nr,
- pgtbl_mod_mask *mask)
+ pgtbl_mod_mask *mask, unsigned int shift)
{
p4d_t *p4d;
unsigned long next;
@@ -632,14 +656,18 @@ static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr,
return -ENOMEM;
do {
next = p4d_addr_end(addr, end);
- if (vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask))
+ if (vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask, shift))
return -ENOMEM;
} while (p4d++, addr = next, addr != end);
return 0;
}
-static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end,
- pgprot_t prot, struct page **pages)
+/*
+ * It can take an array of pages which are not all contiguous, but it
+ * may have contiguous chunks, as hinted by @shift.
+ */
+static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end,
+ pgprot_t prot, struct page **pages, unsigned int shift)
{
unsigned long start = addr;
pgd_t *pgd;
@@ -654,7 +682,7 @@ static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end,
next = pgd_addr_end(addr, end);
if (pgd_bad(*pgd))
mask |= PGTBL_PGD_MODIFIED;
- err = vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask);
+ err = vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask, shift);
if (err)
break;
} while (pgd++, addr = next, addr != end);
@@ -677,27 +705,12 @@ static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end,
int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
pgprot_t prot, struct page **pages, unsigned int page_shift)
{
- unsigned int i, nr = (end - addr) >> PAGE_SHIFT;
-
WARN_ON(page_shift < PAGE_SHIFT);
- if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC) ||
- page_shift == PAGE_SHIFT)
- return vmap_small_pages_range_noflush(addr, end, prot, pages);
+ if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC))
+ page_shift = PAGE_SHIFT;
- for (i = 0; i < nr; i += 1U << (page_shift - PAGE_SHIFT)) {
- int err;
-
- err = vmap_range_noflush(addr, addr + (1UL << page_shift),
- page_to_phys(pages[i]), prot,
- page_shift);
- if (err)
- return err;
-
- addr += 1UL << page_shift;
- }
-
- return 0;
+ return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift);
}
int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
--
2.34.1
^ permalink raw reply related
* [PATCH v4 3/6] mm/vmalloc: Extract vmap_set_ptes() to consolidate PTE mapping logic
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
In-Reply-To: <20260618084726.1070022-1-jiangwen6@xiaomi.com>
Extract the common PTE mapping logic from vmap_pte_range() into a
shared helper vmap_set_ptes(). This handles both CONT_PTE and regular
PTE mappings in a single function, preparing for the next patch which
will extend vmap_pages_pte_range() to also use this helper.
The #ifdef CONFIG_HUGETLB_PAGE guard is moved inside vmap_set_ptes(),
so callers no longer need to handle the conditional compilation.
No functional change.
Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com>
Tested-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
mm/vmalloc.c | 44 +++++++++++++++++++++++++++++++-------------
1 file changed, 31 insertions(+), 13 deletions(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 2c2f74a07f396..6660f240d27c9 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -91,6 +91,35 @@ struct vfree_deferred {
static DEFINE_PER_CPU(struct vfree_deferred, vfree_deferred);
/*** Page table manipulation functions ***/
+
+/*
+ * Set PTE mappings for the given PFN. Try CONT_PTE mappings first when
+ * supported, otherwise fall back to PAGE_SIZE mappings.
+ *
+ * Return: mapping size.
+ */
+static __always_inline unsigned long vmap_set_ptes(pte_t *pte,
+ unsigned long addr, unsigned long end, u64 pfn,
+ pgprot_t prot, unsigned int max_page_shift)
+{
+#ifdef CONFIG_HUGETLB_PAGE
+ if (max_page_shift > PAGE_SHIFT) {
+ unsigned long size;
+
+ size = arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift);
+ if (size != PAGE_SIZE) {
+ pte_t entry = pfn_pte(pfn, prot);
+
+ entry = arch_make_huge_pte(entry, ilog2(size), 0);
+ set_huge_pte_at(&init_mm, addr, pte, entry, size);
+ return size;
+ }
+ }
+#endif
+ set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot));
+ return PAGE_SIZE;
+}
+
static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
phys_addr_t phys_addr, pgprot_t prot,
unsigned int max_page_shift, pgtbl_mod_mask *mask)
@@ -119,19 +148,8 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
BUG();
}
-#ifdef CONFIG_HUGETLB_PAGE
- size = arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift);
- if (size != PAGE_SIZE) {
- pte_t entry = pfn_pte(pfn, prot);
-
- entry = arch_make_huge_pte(entry, ilog2(size), 0);
- set_huge_pte_at(&init_mm, addr, pte, entry, size);
- pfn += PFN_DOWN(size);
- continue;
- }
-#endif
- set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot));
- pfn++;
+ size = vmap_set_ptes(pte, addr, end, pfn, prot, max_page_shift);
+ pfn += PFN_DOWN(size);
} while (pte += PFN_DOWN(size), addr += size, addr != end);
lazy_mmu_mode_disable();
--
2.34.1
^ permalink raw reply related
* [PATCH v4 2/6] arm64/vmalloc: Allow arch_vmap_pte_range_map_size to batch multiple CONT_PTE
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
In-Reply-To: <20260618084726.1070022-1-jiangwen6@xiaomi.com>
From: "Barry Song (Xiaomi)" <baohua@kernel.org>
Allow arch_vmap_pte_range_map_size to batch across multiple CONT_PTE
blocks, reducing both PTE setup and TLB flush iterations.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com>
Tested-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
arch/arm64/include/asm/vmalloc.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/vmalloc.h b/arch/arm64/include/asm/vmalloc.h
index 4ec1acd3c1b34..787fd17b48e2c 100644
--- a/arch/arm64/include/asm/vmalloc.h
+++ b/arch/arm64/include/asm/vmalloc.h
@@ -23,6 +23,8 @@ static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr,
unsigned long end, u64 pfn,
unsigned int max_page_shift)
{
+ unsigned long size;
+
/*
* If the block is at least CONT_PTE_SIZE in size, and is naturally
* aligned in both virtual and physical space, then we can pte-map the
@@ -40,7 +42,9 @@ static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr,
if (!IS_ALIGNED(PFN_PHYS(pfn), CONT_PTE_SIZE))
return PAGE_SIZE;
- return CONT_PTE_SIZE;
+ size = min3(end - addr, 1UL << max_page_shift, PMD_SIZE >> 1);
+ size = 1UL << __fls(size);
+ return size;
}
#define arch_vmap_pte_range_unmap_size arch_vmap_pte_range_unmap_size
--
2.34.1
^ permalink raw reply related
* [PATCH v4 1/6] arm64/hugetlb: Extend batching of multiple CONT_PTE in a single PTE setup
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
In-Reply-To: <20260618084726.1070022-1-jiangwen6@xiaomi.com>
From: "Barry Song (Xiaomi)" <baohua@kernel.org>
For sizes aligned to CONT_PTE_SIZE and smaller than PMD_SIZE,
we can handle CONT_PTE_SIZE groups together.
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com>
Tested-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
arch/arm64/mm/hugetlbpage.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index a42c05cf56408..c4d8b226126cb 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -110,6 +110,12 @@ static inline int num_contig_ptes(unsigned long size, size_t *pgsize)
contig_ptes = CONT_PTES;
break;
default:
+ if (size > 0 && size < PMD_SIZE &&
+ IS_ALIGNED(size, CONT_PTE_SIZE)) {
+ contig_ptes = size >> PAGE_SHIFT;
+ *pgsize = PAGE_SIZE;
+ break;
+ }
WARN_ON(!__hugetlb_valid_size(size));
}
@@ -359,6 +365,10 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags)
case CONT_PTE_SIZE:
return pte_mkcont(entry);
default:
+ if (pagesize > 0 && pagesize < PMD_SIZE &&
+ IS_ALIGNED(pagesize, CONT_PTE_SIZE))
+ return pte_mkcont(entry);
+
break;
}
pr_warn("%s: unrecognized huge page size 0x%lx\n",
--
2.34.1
^ permalink raw reply related
* [PATCH v4 0/6] mm/vmalloc: Speed up ioremap, vmalloc and vmap with contiguous memory
From: Wen Jiang @ 2026-06-18 8:47 UTC (permalink / raw)
To: linux-mm, linux-arm-kernel, catalin.marinas, will, akpm, urezki
Cc: baohua, Xueyuan.chen21, dev.jain, rppt, david, ryan.roberts,
anshuman.khandual, ajd, linux-kernel, jiangwen6, shanghaoqiang
This patchset accelerates ioremap, vmalloc, and vmap when the memory
is physically fully or partially contiguous. Two techniques are used:
1. Avoid page table rewalk when setting PTEs/PMDs for multiple memory
segments
2. Use batched mappings wherever possible in both vmalloc and ARM64
layers
Besides accelerating the mapping path, this also enables large
mappings (PMD and cont-PTE) for vmap, which are currently not
supported.
Patches 1-2 extend ARM64 vmalloc CONT-PTE mapping to support multiple
CONT-PTE regions instead of just one.
Patch 3 extracts a common helper vmap_set_ptes() that consolidates PTE
mapping logic between the ioremap and vmalloc/vmap paths, handling both
CONT_PTE and regular PTE mappings. This prepares for the next patch.
Patch 4 extends the page table walk path to support page shifts other
than PAGE_SHIFT and eliminates the page table rewalk for huge vmalloc
mappings. The function is renamed from vmap_small_pages_range_noflush()
to vmap_pages_range_noflush_walk().
Patches 5-6 add huge vmap support for contiguous pages, including
support for non-compound pages with pfn alignment verification.
On the RK3588 8-core ARM64 SoC, with tasks pinned to a little core and
the performance CPUfreq policy enabled, benchmark results:
* ioremap(1 MB): 1.35x faster (3407 ns -> 2526 ns)
* vmalloc(1 MB) mapping time (excluding allocation) with
VM_ALLOW_HUGE_VMAP: 1.42x faster (5.00 us -> 3.53us)
* vmap(100MB) with order-8 pages: 8.3x faster (1235 us -> 149 us)
Many thanks to Xueyuan Chen for his testing efforts on RK3588 boards.
Changes since v3:
- Squash vmap_pte_range() loop variable fix into patch 4 (patch 3, 4)
- Use shift >= PMD_SHIFT and fix *nr increment in
vmap_pages_pmd_range() (patch 4)
- Pass page_shift directly without capping at PMD_SHIFT (patch 4, 5)
- Add vm_shift() helper and pass pgprot_t to get_vmap_batch_order()
(patch 5)
- Use min(order, __ffs(pfn)) for graceful pfn alignment degradation,
replacing IS_ALIGNED check (patch 5)
- Remove irrelevant ioremap_max_page_shift early-exit (patch 5)
- Add __get_vm_area_node_aligned_caller() wrapper, rename to
vmap_get_aligned_vm_area() (patch 6)
Changes since v2:
- Use __fls instead of fls in arch_vmap_pte_range_map_size (patch 2)
- Add WARN_ON checks in vmap_pages_pmd_range (patch 4)
- Fix flush_cache_vmap to use saved start address instead of the
already-advanced addr (patch 5)
- Rename __vmap_huge() to vmap_batched() (patch 5)
- Add caller parameter and unroll while(1) loop (patch 5)
- Squash patch 7 into patch 5 (stop scanning for compound pages after
encountering small pages)
Changes since v1:
- Fix condition order and use PMD_SIZE instead of CONT_PMD_SIZE in
patch 1 (Dev Jain)
- Squash patch 3+4 and patch 5+7 (Dev Jain)
- Replace "zigzag" with "page table rewalk" in commit messages
(Dev Jain)
- Rename vmap_small_pages_range_noflush() to
vmap_pages_range_noflush_walk() (Dev Jain)
- Extract vmap_set_ptes() as a new patch to consolidate PTE mapping
logic between vmap_pte_range() and vmap_pages_pte_range(), handling
both CONT_PTE and regular mappings (Mike Rapoport)
- Support non-compound pages in get_vmap_batch_order() by falling
back to physical contiguity scanning with pfn alignment check
(Dev Jain, Uladzislau Rezki)
- In get_vmap_batch_order(), filter out orders that the architecture
cannot batch by checking arch_vmap_pte_supported_shift() directly.
This avoids overhead for orders 1-3 on ARM64 CONT_PTE with 4K
pages. (patch 5)
Barry Song (Xiaomi) (5):
arm64/hugetlb: Extend batching of multiple CONT_PTE in a single PTE
setup
arm64/vmalloc: Allow arch_vmap_pte_range_map_size to batch multiple
CONT_PTE
mm/vmalloc: Extend page table walk to support larger page_shift sizes
and eliminate page table rewalk
mm/vmalloc: map contiguous pages in batches for vmap() if possible
mm/vmalloc: align vm_area so vmap() can batch mappings
Wen Jiang (1):
mm/vmalloc: Extract vmap_set_ptes() to consolidate PTE mapping logic
arch/arm64/include/asm/vmalloc.h | 6 +-
arch/arm64/mm/hugetlbpage.c | 10 ++
mm/vmalloc.c | 247 +++++++++++++++++++++++++------
3 files changed, 213 insertions(+), 50 deletions(-)
--
2.34.1
^ permalink raw reply
* Re: [PATCH 1/9] dt-bindings: display: vop2: Add missing reset properties
From: Cristian Ciocaltea @ 2026-06-18 8:39 UTC (permalink / raw)
To: Diederik de Haas, Sandy Huang, Heiko Stübner, Andy Yan,
David Airlie, Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Philipp Zabel, Andrzej Hajda, Neil Armstrong, Robert Foss,
Laurent Pinchart, Jonas Karlman, Jernej Skrabec, Luca Ceresoli
Cc: kernel, Andy Yan, dri-devel, devicetree, linux-arm-kernel,
linux-rockchip, linux-kernel
In-Reply-To: <DJC0L3CRJ0WL.IZEYVLPROMM1@cknow-tech.com>
Hi Diederik,
On 6/18/26 10:58 AM, Diederik de Haas wrote:
> Hi Cristian,
>
> Thanks for this series :-) Just 1 nit (at the end) ...
>
> On Wed Jun 17, 2026 at 8:52 PM CEST, Cristian Ciocaltea wrote:
>> Document the VOP2 resets corresponding to the AXI, AHB and DCLK_VP0..2
>> clocks, which are common to all supported SoCs, plus DCLK_VP3 which is
>> provided only on RK3588.
>>
>> Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
>> ---
>> .../bindings/display/rockchip/rockchip-vop2.yaml | 42 ++++++++++++++++++++++
>> 1 file changed, 42 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/display/rockchip/rockchip-vop2.yaml b/Documentation/devicetree/bindings/display/rockchip/rockchip-vop2.yaml
>> index 93da1fb9adc4..d3bc5380f910 100644
>> --- a/Documentation/devicetree/bindings/display/rockchip/rockchip-vop2.yaml
>> +++ b/Documentation/devicetree/bindings/display/rockchip/rockchip-vop2.yaml
[...]
>> @@ -289,6 +321,16 @@ examples:
>> "dclk_vp0",
>> "dclk_vp1",
>> "dclk_vp2";
>> + resets = <&cru SRST_A_VOP>,
>> + <&cru SRST_H_VOP>,
>> + <&cru SRST_VOP0>,
>> + <&cru SRST_VOP1>,
>> + <&cru SRST_VOP2>;
>> + reset-names = "axi",
>> + "ahb",
>> + "dclk_vp0",
>> + "dclk_vp1",
>> + "dclk_vp2";
>> power-domains = <&power RK3568_PD_VO>;
>
> Place reset* props below power-domains (like in patch 9) ?
> So everyone who copies your example has the correct sorting order.
The example doesn't strictly follow that ordering either — see e.g. the iommus
property — so I placed the resets right after the clocks, which keeps the
related properties grouped together.
That said, I don't have a strong preference.
Heiko, is there a convention you'd like the Rockchip bindings to follow here?
Happy to reorder if so.
Regards,
Cristian
>
> Cheers,
> Diederik
>
>> rockchip,grf = <&grf>;
>> iommus = <&vop_mmu>;
>
^ permalink raw reply
* Re: [PATCH 2/3] irqchip/gic-v3: Add Renesas R-Car Gen4 erratum workaround
From: Marc Zyngier @ 2026-06-18 8:38 UTC (permalink / raw)
To: Marek Vasut
Cc: Marek Vasut, linux-pci, Yoshihiro Shimoda,
Krzysztof Wilczyński, Bjorn Helgaas, Catalin Marinas,
Conor Dooley, Geert Uytterhoeven, Krzysztof Kozlowski,
Lorenzo Pieralisi, Manivannan Sadhasivam, Rob Herring, devicetree,
linux-arm-kernel, linux-doc, linux-kernel, linux-renesas-soc
In-Reply-To: <0935eb67-83d2-49ea-89ab-0d0aa51ead8a@mailbox.org>
On Thu, 18 Jun 2026 03:50:29 +0100,
Marek Vasut <marek.vasut@mailbox.org> wrote:
>
> On 6/17/26 9:24 AM, Marc Zyngier wrote:
>
> Hello Marc,
>
> >> Renesas R-Car S4/V4H/V4M GIC600 integration has address width for AXI
> >> or APB interface configured to 32 bit, it can therefore access only
> >> the first 4 GiB of physical address space. This information comes from
> >> R-Car V4H Interface Specification sheet, there is currently no technical
> >> update number assigned to this limitation. Further input from hardware
> >> engineer indicates that this limitation also applies to R-Car S4 and V4M.
> >> Name the limitation GEN4GICITS1, and add a driver quirk to mitigate this
> >> limitation.
>
> My concern is this ^ , I do not have an erratum number, because there
> isn't one. I am in touch with the hardware engineer and I did get a
> glimpse at internal details of the three SoC, which confirm the
> limitations. Is this sufficient ?
To be honest, this is between you and the SoC vendor. I'll take
whatever symbol you come up with at face value, and will assume that
the vendor agrees with it. After all, they are on Cc and have their
SoB on the patch.
>
> >> Note that the 0x0201743b GIC600 ID is not Renesas-specific, it is
> >> common for many ARM GICv3 implementations. Therefore, add an extra
> >
> > Not quite. It designates GIC600 unambiguously.
>
> What I am trying to communicate is, that the 0x0201743b ID is not ID
> of the Renesas GIC implementation, but it is a generic ARM GIC600
> ID. That is why we cannot match the quirk on the ID (it is generic ARM
> GIC600 ID), and instead we have to match the quirk on the [ ID
> combined with of_machine_is_compatible("renesas,...") ].
This is understood, and is no different from the other broken
platforms in the tree.
>
> > It is just that GIC600
> > is integrated in zillions of SoCs, most of which don't have this
> > problem (the machine I'm typing this from has a GIC600 *and* 96GB of
> > RAM).
>
> Right.
>
> Shall I reword this paragraph somehow to make it clearer ?
I'd simply say that the workaround is keyed on the combination of the
GIC implementation and the platform identification in the device tree.
>
> >> of_machine_is_compatible() check.
> >>
> >> The GIC600 implementation in R-Car S4/V4H/V4M is r1p6.
> >
> > Is this relevant?
>
> I included it for the sake of completeness and to provide all relevant
> information, based on previous discussions about similar limitations
> that I could find on lore.k.o
This information is already contained in the ID you quote (bits
[19:12]), and can be decoded using the public TRM [1].
Thanks,
M.
[1] https://documentation-service.arm.com/static/5e7ddddacbfe76649ba53034
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply
* Re: [PATCH v6 00/20] dma-mapping: Use DMA_ATTR_CC_SHARED through direct, pool and swiotlb paths
From: Aneesh Kumar K.V @ 2026-06-18 8:37 UTC (permalink / raw)
To: Alexey Kardashevskiy, Jason Gunthorpe, Catalin Marinas
Cc: iommu, linux-arm-kernel, linux-kernel, linux-coco, Robin Murphy,
Marek Szyprowski, Will Deacon, Marc Zyngier, Steven Price,
Suzuki K Poulose, Jiri Pirko, Mostafa Saleh, Petr Tesarik,
Dan Williams, Xu Yilun, linuxppc-dev, linux-s390,
Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
Christophe Leroy (CS GROUP), Alexander Gordeev, Gerald Schaefer,
Heiko Carstens, Vasily Gorbik, Christian Borntraeger,
Sven Schnelle, x86
In-Reply-To: <2ecfa1a8-6202-4319-9692-a6ffeb5a3dbf@amd.com>
Alexey Kardashevskiy <aik@amd.com> writes:
> On 10/6/26 00:47, Jason Gunthorpe wrote:
>> On Tue, Jun 09, 2026 at 02:43:08PM +0100, Catalin Marinas wrote:
>>> On Thu, Jun 04, 2026 at 02:09:39PM +0530, Aneesh Kumar K.V (Arm) wrote:
>>>> This series propagates DMA_ATTR_CC_SHARED through the dma-direct,
>>>> dma-pool, and swiotlb paths so that encrypted and decrypted DMA buffers
>>>> are handled consistently.
>>>>
>>>> Today, the direct DMA path mostly relies on force_dma_unencrypted() for
>>>> shared/decrypted buffer handling. This series consolidates the
>>>> force_dma_unencrypted() checks in the top-level functions and ensures
>>>> that the remaining DMA interfaces use DMA attributes to make the correct
>>>> decisions.
>>>
>>> Please check Sashiko's reports, it has some good points:
>>>
>>> https://sashiko.dev/#/patchset/20260604083959.1265923-1-aneesh.kumar@kernel.org
>>>
>>> I think the main one is the swiotlb_tbl_map_single() changes which break
>>> AMD SME host support. There cc_platform_has(CC_ATTR_MEM_ENCRYPT) is true
>>> but force_dma_unencrypted() is false. Normally you'd not end up on this
>>> path but you can have swiotlb=force.
>>
>> IMHO that's an AMD issue, not with the design of this series..
>>
>> The series is right, a device that is !force_dma_decrypted() must be
>> considerd to be a trusted device and we must never place any DMA
>> mappings for a trusted device into shared memory.
>
>
> swiotlb=force forces swiotlb, not decryption.
>
>> That AMD has done somethine insane:
>>
>> bool force_dma_unencrypted(struct device *dev)
>> {
>> /*
>> * For SEV, all DMA must be to unencrypted addresses.
>> */
>> if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
>> return true;
>>
>> /*
>> * For SME, all DMA must be to unencrypted addresses if the
>> * device does not support DMA to addresses that include the
>> * encryption mask.
>> */
>> if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) {
>> u64 dma_enc_mask = DMA_BIT_MASK(__ffs64(sme_me_mask));
>> u64 dma_dev_mask = min_not_zero(dev->coherent_dma_mask,
>> dev->bus_dma_limit);
>>
>> if (dma_dev_mask <= dma_enc_mask)
>> return true;
>> }
>
>
> So when I try "mem_encrypt=on iommu=pt swiotlb=force" with this patchset, it fails to boot. But it boots with a hack like this:
>
> ===
> @@ -39,7 +41,7 @@ bool force_dma_unencrypted(struct device *dev)
> return true;
> }
>
> - return false;
> + return swiotlb_force_bounce;
> }
> ===
>
> Or we say "mem_encrypt=on iommu=pt swiotlb=force" combo is just weird and we won't be supporting which bit in this? Thanks,
>
Something like?
modified arch/x86/mm/mem_encrypt.c
@@ -34,6 +34,13 @@ bool force_dma_unencrypted(struct device *dev)
u64 dma_enc_mask = DMA_BIT_MASK(__ffs64(sme_me_mask));
u64 dma_dev_mask = min_not_zero(dev->coherent_dma_mask,
dev->bus_dma_limit);
+ /*
+ * With memory encryption enabled, SWIOTLB is marked decrypted.
+ * If SWIOTLB bouncing is forced, treat the device as requiring
+ * decrypted DMA.
+ */
+ if (is_swiotlb_force_bounce(dev))
+ return true;
if (dma_dev_mask <= dma_enc_mask)
return true;
-aneesh
^ permalink raw reply
* Re: [PATCH v4 3/5] dt-bindings: clock: cix,sky1-audss-clock: add audss clock controller
From: Philipp Zabel @ 2026-06-18 8:30 UTC (permalink / raw)
To: Joakim Zhang, Conor Dooley
Cc: mturquette@baylibre.com, sboyd@kernel.org, bmasney@redhat.com,
robh@kernel.org, krzk+dt@kernel.org, conor+dt@kernel.org,
Gary Yang, cix-kernel-upstream, linux-clk@vger.kernel.org,
devicetree@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org
In-Reply-To: <SEYPR06MB62262C0F7823337CA9496DE982E32@SEYPR06MB6226.apcprd06.prod.outlook.com>
On Do, 2026-06-18 at 01:43 +0000, Joakim Zhang wrote:
> Hello,
>
>
> > -----Original Message-----
> > From: Conor Dooley <conor@kernel.org>
> > Sent: Wednesday, June 17, 2026 11:56 PM
> > To: Joakim Zhang <joakim.zhang@cixtech.com>
> > Cc: mturquette@baylibre.com; sboyd@kernel.org; bmasney@redhat.com;
> > robh@kernel.org; krzk+dt@kernel.org; conor+dt@kernel.org;
> > p.zabel@pengutronix.de; Gary Yang <gary.yang@cixtech.com>; cix-kernel-
> > upstream <cix-kernel-upstream@cixtech.com>; linux-clk@vger.kernel.org;
> > devicetree@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
> > kernel@lists.infradead.org
> > Subject: Re: [PATCH v4 3/5] dt-bindings: clock: cix,sky1-audss-clock: add audss
> > clock controller
> >
> > On Wed, Jun 17, 2026 at 02:04:35PM +0800, joakim.zhang@cixtech.com wrote:
> > > From: Joakim Zhang <joakim.zhang@cixtech.com>
> > >
> > > The AUDSS CRU contains an internal clock tree of muxes, dividers and
> > > gates for DSP, I2S, HDA, DMAC and related blocks. The clock provider
> > > is a child node of the cix,sky1-audss-system-control syscon and
> > > accesses registers through the parent MMIO region.
> >
> > Why can this not just be part of the parent syscon node?
>
> The clock and reset blocks are handled by different subsystems and maintainers (clk vs reset). Putting the clock provider on the parent syscon node would mean a single driver has to register both the reset controller and the clock provider on one device, which doesn't fit well.
There are many examples of clock and reset drivers sharing the same
node, by using platform_driver for one (usually clk) and
auxiliary_driver for the other (usually reset).
regards
Philipp
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox