* Re: [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs
From: Simon Horman @ 2026-07-14 13:01 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
In-Reply-To: <20260706193603.8039-13-mmc@linux.ibm.com>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Expose per-queue buffer pool details via sysfs
This commit adds a read-only sysfs attribute buffer_pools to the VIO device
to expose per-queue buffer pool details dynamically. It outputs runtime per-queue
buffer pressure metrics like size, active, and available buffers.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 4e3f49b6346fd..ecc472ee8f71f 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -2896,6 +2896,52 @@ static const struct net_device_ops ibmveth_netdev_ops = {
[ ... ]
> +static ssize_t buffer_pools_show(struct device *dev,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct net_device *netdev = dev_get_drvdata(dev);
> + struct ibmveth_adapter *adapter = netdev_priv(netdev);
> + int len = 0;
> + int i, j;
> +
> + len += scnprintf(buf + len, PAGE_SIZE - len,
> + "Queue Pool Size BuffSize Active Available\n");
> + len += scnprintf(buf + len, PAGE_SIZE - len,
> + "----- ---- ---- -------- ------ ---------\n");
[Severity: Medium]
Does this buffer_pools_show() implementation violate the sysfs "one value
per file" design rule?
The attribute appears to dump a complex, multi-line ASCII table with headers
and columns rather than exposing a single value per sysfs file.
[ ... ]
> @@ -3056,6 +3102,14 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>
> netdev_dbg(netdev, "registered\n");
>
> + rc = sysfs_create_group(&dev->dev.kobj, &ibmveth_attr_group);
> + if (rc) {
> + netdev_err(netdev, "failed to create sysfs attributes rc=%d\n", rc);
> + unregister_netdev(netdev);
> + free_netdev(netdev);
> + return rc;
> + }
[Severity: High]
If userspace brings up the interface immediately after register_netdev()
succeeds, could adapter->work be scheduled before sysfs_create_group() fails?
If this race occurs, unregister_netdev() does not synchronously cancel the
work.
Could the global workqueue execute the pending adapter->work item on freed
memory after free_netdev() is called in this ibmveth_probe() error path?
[Severity: High]
This is a pre-existing issue in older error paths of ibmveth_probe(), but it
is duplicated here: does this error path leak the adapter->rx_buff_pool
kobjects?
Since kobject_put() is not called on the pool kobjects before free_netdev()
is executed, the kobjects remain attached to the VIO device with dangling
pointers to the freed adapter memory.
Could subsequent sysfs accesses to those pool kobjects trigger a
use-after-free?
^ permalink raw reply
* Re: [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize
From: Simon Horman @ 2026-07-14 13:03 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
In-Reply-To: <20260706193603.8039-14-mmc@linux.ibm.com>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
Date: Mon Jul 6 19:36:01 2026 +0000
ibmveth: Implement incremental MQ RX queue resize
Add helpers to register/deregister individual RX queues and scale
queue count up or down while the netdev stays up. Surviving queues
keep PHYP handles, buffer pools, and IRQ state across resize.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index ecc472ee8f71f..ac4d89a66a8db 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -1080,6 +1132,138 @@ static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
> +static int
> +ibmveth_alloc_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx,
> + int rxq_entries)
> +{
[ ... ]
> + for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> + adapter->rx_buff_pool[queue_idx][i].size =
> + adapter->rx_buff_pool[0][i].size;
> + adapter->rx_buff_pool[queue_idx][i].buff_size =
> + adapter->rx_buff_pool[0][i].buff_size;
> + adapter->rx_buff_pool[queue_idx][i].threshold =
> + adapter->rx_buff_pool[0][i].threshold;
> + adapter->rx_buff_pool[queue_idx][i].active =
> + adapter->rx_buff_pool[0][i].active;
> + }
[Severity: Critical]
Does this loop miss initializing pool->index?
If pool->index remains 0 for all buffer pools on the new queue, when
replenishing, 0 is shifted into the correlator.
Then when receiving, ibmveth_remove_buffer_from_pool() will extract pool=0,
which fetches from the wrong pool and could corrupt the free map of pool 0,
leading to a use-after-free or leak.
[ ... ]
> @@ -1192,6 +1376,49 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
> +static int
> +ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
> +{
[ ... ]
> + while (drained < limit &&
> + ibmveth_rxq_pending_buffer(adapter, queue_index)) {
> + rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
Simon says: I think we can consider the issue below in the context of
possible follow-up, because I believe the use of this pattern
predates this patch.
[Severity: High]
Is a read memory barrier needed between reading the toggle bit in
ibmveth_rxq_pending_buffer() and reading the correlator in
ibmveth_rxq_harvest_buffer()?
Since the hypervisor could be actively writing new descriptors while this is
called, could a weakly-ordered CPU reorder the correlator load before the
toggle bit load, causing us to recycle the wrong buffer?
[ ... ]
> @@ -1491,6 +1718,175 @@ ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
> +static int
> +ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
> + int new_count, int rxq_entries)
> +{
[ ... ]
> + rc = ibmveth_enable_irq(adapter, i);
> + if (rc) {
> + netdev_err(netdev,
> + "Failed to enable IRQ for queue %d: %d\n",
> + i, rc);
> + ibmveth_cleanup_single_rx_interrupt(adapter, i);
> + ibmveth_deregister_single_rx_queue(adapter, i);
> + ibmveth_free_single_rx_queue(adapter, i);
> + goto cleanup_new_queues;
> + }
> +
> + napi_enable(&adapter->napi[i]);
> + }
> +
> + adapter->num_rx_queues = new_count;
[Severity: High]
What happens if an interrupt fires on the newly added queue before
adapter->num_rx_queues is updated?
If an interrupt fires between ibmveth_enable_irq() and the update of
adapter->num_rx_queues, ibmveth_interrupt() might evaluate:
WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues)
to true and return IRQ_NONE without masking the interrupt.
Could this lead to an unhandled interrupt storm?
[ ... ]
> + } else {
> + netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
> + new_count, old_count - 1);
> +
> + for (i = new_count; i < old_count; i++)
> + napi_disable(&adapter->napi[i]);
> +
> + for (i = new_count; i < old_count; i++)
> + ibmveth_drain_rx_queue(adapter, i);
> +
> + synchronize_net();
[Severity: High]
Can disabling NAPI before disabling the hardware interrupts create an
interrupt storm window?
If the hypervisor asserts an interrupt during synchronize_net(), the hardware
IRQ handler runs. Since NAPI is disabled, it returns IRQ_HANDLED without
masking the interrupt source at the hypervisor, leading immediately to an
IRQ storm and CPU lockup.
> +
> + rc = netif_set_real_num_rx_queues(netdev, new_count);
> + if (rc) {
> + netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
> + new_count, rc);
> + for (i = new_count; i < old_count; i++)
> + napi_enable(&adapter->napi[i]);
> + return rc;
> + }
[Severity: High]
Is it possible for this error path to permanently stall the RX queues?
It calls napi_enable() but doesn't call ibmveth_replenish_task(). Since the
queues were completely drained of their buffers by ibmveth_drain_rx_queue(),
will the hypervisor have any buffers to place incoming packets?
Without buffers, no interrupts will fire for these queues, and NAPI will
never run again.
^ permalink raw reply
* Re: [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to MQ RX queue resize
From: Simon Horman @ 2026-07-14 13:07 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
In-Reply-To: <20260706193603.8039-15-mmc@linux.ibm.com>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Wire ethtool set_channels to MQ RX queue resize
Expose incremental RX resize through ethtool channel control.
get_channels() reports rx_count from adapter->num_rx_queues and max_rx
as IBMVETH_MAX_RX_QUEUES when MQ firmware is enabled, else 1.
set_channels() validates rx_count is within 1..IBMVETH_MAX_RX_QUEUES.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index ac4d89a66a8db..50a332ab83fd3 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -2534,19 +2534,69 @@ static int ibmveth_set_channels(struct net_device *netdev,
> struct ethtool_channels *channels)
> {
> struct ibmveth_adapter *adapter = netdev_priv(netdev);
[ ... ]
> - if (!(netdev->flags & IFF_UP))
> + if (!(netdev->flags & IFF_UP)) {
> + if (goal_rx > 1 && !adapter->multi_queue) {
[ ... ]
> + /* Stash desired RX count; open() publishes it via
> + * netif_set_real_num_rx_queues() after queue registration.
> + */
> + if (goal_rx != adapter->num_rx_queues)
> + adapter->num_rx_queues = goal_rx;
> +
> return netif_set_real_num_tx_queues(netdev, goal);
[Severity: Medium]
If netif_set_real_num_tx_queues() fails and returns an error, the TX
configuration command will fail, but the RX queue count has already been
permanently modified in adapter->num_rx_queues. Will this leave the device in
an inconsistent state where the new RX count is applied on the next open
despite the command failing?
> + }
[ ... ]
> + if (goal_rx != old_rx) {
> + rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
> + rxq_entries);
[Severity: High]
Does this new ethtool scale-down path expose an interrupt storm?
Looking at ibmveth_resize_rx_queues_incremental(), when scaling down queues it
disables NAPI before disabling the hardware interrupt:
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_resize_rx_queues_incremental() {
...
for (i = new_count; i < old_count; i++)
napi_disable(&adapter->napi[i]);
for (i = new_count; i < old_count; i++)
ibmveth_drain_rx_queue(adapter, i);
synchronize_net();
...
/* Disable hypervisor interrupts and wait for handlers to complete
* before updating num_rx_queues.
*/
for (i = new_count; i < old_count; i++) {
ibmveth_disable_irq(adapter, i);
...
}
If the hypervisor delivers a packet during this window, ibmveth_interrupt()
will execute. Since NAPI is disabled, napi_schedule_prep() fails, causing the
handler to return IRQ_HANDLED without disabling the hardware interrupt. Will
this leave the virtual interrupt asserted and force an infinite interrupt loop?
> + if (rc) {
> + netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
> + return rc;
> + }
> + }
>
> /* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
> * but we may need to alloc/free the ltb's.
> */
[Severity: High]
This is a pre-existing issue, but does this function use an uninitialized
variable in the error path?
When decreasing the number of TX queues (goal < old), the allocation loop for
(i = old; i < goal; i++) is skipped, leaving 'i' uninitialized. If
netif_set_real_num_tx_queues() then fails, the error path assigns old = i;.
The cleanup loop then uses this garbage value:
for (i = old; i > goal; i--) {
if (adapter->tx_ltb_ptr[i - 1])
Will this access adapter->tx_ltb_ptr out of bounds and pass arbitrary pointers
to ibmveth_free_tx_ltb()?
> @@ -2580,7 +2630,7 @@ static int ibmveth_set_channels(struct net_device *netdev,
>
> netif_tx_wake_all_queues(netdev);
>
> - return rc;
> + return 0;
[Severity: High]
Does this unconditional return of 0 mask TX queue configuration failures?
If applying the new TX queue count via netif_set_real_num_tx_queues() fails,
the code handles it locally, but the function exits with 0 regardless. Will
this mislead userspace into believing the TX channel count was successfully
modified?
> }
^ permalink raw reply
* Re: [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
From: Simon Horman @ 2026-07-14 13:10 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
In-Reply-To: <20260706193603.8039-16-mmc@linux.ibm.com>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
This commit fixes polling hangs in the ibmveth driver after queue resize
by skipping over bad correlators instead of retrying endlessly. It also
ensures NAPI completes when the interface is down or being disabled.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 50a332ab83fd3..eb84b4e5f69fd 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -2959,8 +2978,23 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
> __sum16 iph_check = 0;
>
> skb = ibmveth_rxq_get_buffer(adapter, queue_index);
> - if (unlikely(!skb))
> - break;
> + if (unlikely(!skb)) {
> + if (net_ratelimit())
> + netdev_err(netdev,
[Severity: High]
This isn't a bug introduced by this patch, but can the unvalidated packet
lengths and offsets from hardware descriptors trigger an skb_over_panic()
here?
Further down in ibmveth_poll(), the driver reads length and offset directly
from the hardware receive descriptor and passes them to skb_reserve() and
skb_put() on the reused SKB:
int length = ibmveth_rxq_frame_length(adapter, queue_index);
int offset = ibmveth_rxq_frame_offset(adapter, queue_index);
...
} else {
rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, false);
if (unlikely(rc))
break;
skb_reserve(skb, offset);
}
skb_put(skb, length);
If a faulty hypervisor provides an abnormally large length or offset,
skb_put() will extend the tail past the end of the buffer, which triggers an
skb_over_panic() and causes a crash.
Should the driver validate that offset + length is within the known buffer
size before modifying the boundaries?
[ ... ]
> @@ -3034,8 +3068,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>
> ibmveth_replenish_task(adapter, queue_index);
>
> - if (frames_processed == budget)
> + if (frames_processed == budget) {
> + if (!netif_running(netdev) || napi_disable_pending(napi))
> + napi_complete_done(napi, frames_processed);
> goto out;
> + }
[Severity: High]
Does calling napi_complete_done() here conflict with the NAPI core contract?
If the polling loop exhausts its budget (frames_processed == budget), returning
the full budget after calling napi_complete_done() might cause issues with the
NAPI state machine. The NAPI core (__napi_poll) will operate on the instance
assuming it still holds the NAPIF_STATE_SCHED bit.
If napi_disable_pending() is true, the core calls napi_complete(), which will
hit a WARN_ON_ONCE.
If napi_disable_pending() is false but !netif_running() is true, the core will
add the NAPI instance back to the poll_list. Since the driver cleared
NAPIF_STATE_SCHED, a subsequent interrupt can trigger another napi_schedule(),
causing a double list_add_tail() and potentially leading to list corruption.
^ permalink raw reply
* [PATCH 00/22] Simplify special kernel page table handling
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
Page table constructors and destructors were originally introduced
specifically for user PTE pages. Over time, their use became more
widespread, and starting from v6.16 they are called at all levels
and for both user and kernel page tables [1]. Or so I thought. Turns
out, we still have various special kernel page tables where the
ctor/dtor are not called; vmemmap page tables is one such case.
Looking closer, we do not actually need per-level ctors for kernel
page tables. In fact we do not need ctors for them at all, because
nothing is specific to kernel page tables in the ctor/dtor and
nowadays they should be allocated with pagetable_alloc().
Here's the plan: all the generic page table handling moves to
pagetable_{alloc,free}(). Whatever is left is specific to user page
tables; the ctor/dtor are only called for user page tables at the
required levels (PTE/PMD), and renamed to make that clear.
Doing this will allow us to remove a good amount of special casing,
but there's quite a bit of churn to get there. Here's what this
series does:
1. Introduce the concept of "kernel mm" and replace most comparisons
with &init_mm with a flag check (patch 1-12); no functional change
so far.
2. Move the page table accounting/type setting to pagetable_{alloc,free}
(patch 13-15). Kernel mm's do not need to call the ctor/dtor any
more.
3. Mark special mm's (efi_mm, tboot_mm on x86) as kernel mm's (patch
16-19). They now abide by the same rules as init_mm, which means
in particular that split page table locks are no longer used.
4. Remove the ctor/dtor calls (now no-ops) for special kernel page
tables on arm/arm64/riscv (patch 20-22).
[1] https://lore.kernel.org/all/20250408095222.860601-1-kevin.brodsky@arm.com/
---
I stopped there for part 1 to keep it digestible. Part 2 will then
ensure that the ctor/dtor are only called for user PTE/PMD pages and
rename them accordingly. This is comparatively simpler but also
involves a lot of churn.
This series was suggested in reaction to [2]; removing dtor calls
entirely for kernel page tables systematically solves such issues.
Worth noting that some comparisons against &init_mm have been left
untouched, as they are truly specific to init_mm. Reviewers are
encouraged to take a look at the remaining cases:
$ git grep -nE '([=!]= *&init_mm|&init_mm *[=!]=)'
Documentation/mm/active_mm.rst:78: if (current->mm == &init_mm)
Documentation/translations/zh_CN/mm/active_mm.rst:76: if (current->mm == &init_mm)
arch/arm/include/asm/mmu_context.h:144: if (mm != &init_mm)
arch/arm64/include/asm/mmu_context.h:100: if (mm != &init_mm && !system_uses_ttbr0_pan())
arch/arm64/include/asm/mmu_context.h:210: if (mm == &init_mm)
arch/arm64/include/asm/mmu_context.h:242: if (next == &init_mm) {
arch/arm64/mm/fault.c:121: if (mm == &init_mm)
arch/arm64/mm/fault.c:139: if (mm == &init_mm) {
arch/arm64/mm/fault.c:154: mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K,
arch/arm64/mm/mmu.c:997: BUG_ON(mm == &init_mm);
arch/powerpc/xmon/xmon.c:3314: if (mm == &init_mm)
arch/riscv/mm/fault.c:42: mm == &init_mm ? (u64)__pa_symbol(mm->pgd) : virt_to_phys(mm->pgd));
kernel/cpu.c:917: WARN_ON(mm != &init_mm);
kernel/fork.c:725: BUG_ON(mm == &init_mm);
kernel/sched/core.c:8390: if (mm != &init_mm) {
mm/ptdump.c:182: if (mm != &init_mm)
mm/ptdump.c:191: if (mm != &init_mm)
Testing:
* Built all modified architectures
* arm64
- mm kselftests (FVP)
- Hot adding/removing memory + memory-hotplug kselftests (QEMU)
- Reading /sys/kernel/debug/{efi,kernel}_page_tables
- Reading from /sys/firmware/efi/efivars/
*riscv
- memory-hotplug kselftests (QEMU)
* x86
- mm kselftests (lkvm)
- tboot page table setup using a fake tboot shared page (QEMU)
- CONFIG_EFI_PGT_DUMP=y (QEMU)
Patches based on mm-unstable.
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
[2] https://lore.kernel.org/all/20260521032730.2104017-1-apopple@nvidia.com/
---
To: linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: loongarch@lists.linux.dev
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: linux-parisc@vger.kernel.org
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: Paul Walmsley <pjw@kernel.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: linux-riscv@lists.infradead.org
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: linux-s390@vger.kernel.org
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andreas Larsson <andreas@gaisler.com>
Cc: sparclinux@vger.kernel.org
Cc: Richard Weinberger <richard@nod.at>
Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc: Johannes Berg <johannes@sipsolutions.net>
Cc: linux-um@lists.infradead.org
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ning Sun <ning.sun@intel.com>
Cc: x86@kernel.org
Cc: tboot-devel@lists.sourceforge.net
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Cc: linux-efi@vger.kernel.org
Cc: Vishal Moola <vishal.moola@gmail.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: linux-kernel@vger.kernel.org
Cc: linux-arch@vger.kernel.org
---
Kevin Brodsky (22):
mm: drop unused __mm_flags_set_mask_bits_word()
mm: move mm_flags helpers to mm_types.h
mm: introduce MMF_KERNEL flag and set it for init_mm
mm: use mm_is_kernel() in generic page table code
arm64: mm: use mm_is_kernel() for kernel mm checks
loongarch: mm: use mm_is_kernel() in switch_mm_irqs_off()
parisc: mm: use mm_is_kernel() for kernel mm checks
powerpc: mm: use mm_is_kernel() for kernel mm checks
s390: mm: use mm_is_kernel() for kernel mm checks
sparc: mm: use mm_is_kernel() for kernel mm checks
um: mm: use mm_is_kernel() in TLB sync
x86/mm: use mm_is_kernel() for kernel mm checks
mm: account page table pages when allocated
mm: set page table page type when allocated
mm: only initialise pt_share_count for user pgtables
efi: mark efi_mm as a kernel mm
mm: pagewalk: drop redundant address check for kernel mm walks
arm64: mm: drop explicit mm_is_efi() check in contpte
x86/tboot: mark tboot_mm as a kernel mm
arm64: mm: drop ctor/dtor calls for kernel page tables
arm: mm: drop ctor call for kernel page tables
riscv: mm: drop ctor/dtor calls for kernel page tables
arch/arm/mm/mmu.c | 3 +-
arch/arm64/include/asm/pgalloc.h | 10 ++--
arch/arm64/mm/contpte.c | 26 ++++------
arch/arm64/mm/hugetlbpage.c | 2 +-
arch/arm64/mm/mmu.c | 82 +++++++++-----------------------
arch/loongarch/include/asm/mmu_context.h | 2 +-
arch/parisc/include/asm/mmu_context.h | 2 +-
arch/parisc/include/asm/pgalloc.h | 2 +-
arch/parisc/include/asm/tlbflush.h | 2 +-
arch/powerpc/include/asm/pgalloc.h | 2 +-
arch/powerpc/mm/book3s64/pgtable.c | 2 +-
arch/powerpc/mm/mmu_context.c | 2 +-
arch/powerpc/mm/pgtable.c | 2 +-
arch/riscv/mm/init.c | 63 ++++++------------------
arch/s390/include/asm/mmu_context.h | 2 +-
arch/s390/include/asm/pgtable.h | 6 +--
arch/s390/mm/pgalloc.c | 4 +-
arch/sparc/include/asm/mmu_context_64.h | 2 +-
arch/sparc/include/asm/pgtable_64.h | 6 +--
arch/sparc/mm/init_64.c | 2 +-
arch/sparc/mm/tlb.c | 2 +-
arch/um/kernel/tlb.c | 2 +-
arch/x86/include/asm/tlbflush.h | 8 ++--
arch/x86/kernel/tboot.c | 4 +-
arch/x86/mm/pgtable.c | 2 +-
arch/x86/mm/tlb.c | 9 ++--
drivers/firmware/efi/efi.c | 1 +
include/asm-generic/pgalloc.h | 16 +++----
include/linux/mm.h | 69 +++++++--------------------
include/linux/mm_types.h | 52 +++++++++++++++-----
mm/init-mm.c | 1 +
mm/memory.c | 6 +--
mm/page_table_check.c | 14 +++---
mm/pagewalk.c | 7 +--
34 files changed, 163 insertions(+), 254 deletions(-)
---
base-commit: a531f0cf705af3374388eafe4115f651c625d8db
change-id: 20260611-remove_pgtable_cdtor-3b123938f8cd
^ permalink raw reply
* [PATCH 01/22] mm: drop unused __mm_flags_set_mask_bits_word()
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
Commit 39f8049cd49f ("mm: update coredump logic to correctly
use bitmap mm flags") added coredump-specific helpers for the
remaining masked mm-flags update users.
That left __mm_flags_set_mask_bits_word() without any callers.
This is a special-purpose helper that is unlikely to be useful for
other purposes; best to remove it.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/linux/mm_types.h | 12 ------------
1 file changed, 12 deletions(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index ae9bca4eda5c..b4129beb9c4a 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1432,18 +1432,6 @@ static inline unsigned long __mm_flags_get_word(const struct mm_struct *mm)
return *__mm_flags_get_bitmap(mm);
}
-/*
- * Update the first system word of mm flags ONLY, applying the specified mask to
- * it, then setting all flags specified by bits.
- */
-static inline void __mm_flags_set_mask_bits_word(struct mm_struct *mm,
- unsigned long mask, unsigned long bits)
-{
- unsigned long *bitmap = ACCESS_PRIVATE(&mm->flags, __mm_flags);
-
- set_mask_bits(bitmap, mask, bits);
-}
-
#define MM_MT_FLAGS (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_LOCK_EXTERN | \
MT_FLAGS_USE_RCU)
extern struct mm_struct init_mm;
--
2.51.2
^ permalink raw reply related
* [PATCH 02/22] mm: move mm_flags helpers to mm_types.h
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
Some of the mm_flags helpers are defined in <linux/mm_types.h>,
while others are defined in <linux/mm.h>. We will soon need to call
one of the latter helpers from mm_types.h.
Regroup all the helpers in mm_types.h to keep things simple.
Also remove #include's that were added specifically for these
helpers in mm.h.
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/linux/mm.h | 32 --------------------------------
include/linux/mm_types.h | 30 ++++++++++++++++++++++++++++++
2 files changed, 30 insertions(+), 32 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 32bb723ffbb9..7a4b8e7ac8f4 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -34,8 +34,6 @@
#include <linux/slab.h>
#include <linux/cacheinfo.h>
#include <linux/rcuwait.h>
-#include <linux/bitmap.h>
-#include <linux/bitops.h>
#include <linux/iommu-debug-pagealloc.h>
#include <linux/kcsan-checks.h>
@@ -948,36 +946,6 @@ static inline void assert_fault_locked(const struct vm_fault *vmf)
}
#endif /* CONFIG_PER_VMA_LOCK */
-static inline bool mm_flags_test(int flag, const struct mm_struct *mm)
-{
- return test_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
-}
-
-static inline bool mm_flags_test_and_set(int flag, struct mm_struct *mm)
-{
- return test_and_set_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
-}
-
-static inline bool mm_flags_test_and_clear(int flag, struct mm_struct *mm)
-{
- return test_and_clear_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
-}
-
-static inline void mm_flags_set(int flag, struct mm_struct *mm)
-{
- set_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
-}
-
-static inline void mm_flags_clear(int flag, struct mm_struct *mm)
-{
- clear_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
-}
-
-static inline void mm_flags_clear_all(struct mm_struct *mm)
-{
- bitmap_zero(ACCESS_PRIVATE(&mm->flags, __mm_flags), NUM_MM_FLAG_BITS);
-}
-
extern const struct vm_operations_struct vma_dummy_vm_ops;
static inline void vma_init(struct vm_area_struct *vma, struct mm_struct *mm)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index b4129beb9c4a..a2e0dc5892ff 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1414,6 +1414,36 @@ struct mm_struct {
char flexible_array[] __aligned(__alignof__(unsigned long));
};
+static inline bool mm_flags_test(int flag, const struct mm_struct *mm)
+{
+ return test_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
+}
+
+static inline bool mm_flags_test_and_set(int flag, struct mm_struct *mm)
+{
+ return test_and_set_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
+}
+
+static inline bool mm_flags_test_and_clear(int flag, struct mm_struct *mm)
+{
+ return test_and_clear_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
+}
+
+static inline void mm_flags_set(int flag, struct mm_struct *mm)
+{
+ set_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
+}
+
+static inline void mm_flags_clear(int flag, struct mm_struct *mm)
+{
+ clear_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
+}
+
+static inline void mm_flags_clear_all(struct mm_struct *mm)
+{
+ bitmap_zero(ACCESS_PRIVATE(&mm->flags, __mm_flags), NUM_MM_FLAG_BITS);
+}
+
/* Copy value to the first system word of mm flags, non-atomically. */
static inline void __mm_flags_overwrite_word(struct mm_struct *mm, unsigned long value)
{
--
2.51.2
^ permalink raw reply related
* [PATCH 03/22] mm: introduce MMF_KERNEL flag and set it for init_mm
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
mm code often needs to know whether some mm represents a kernel or
user address space. This is currently done by comparing the mm
pointer with &init_mm; besides not being particularly elegant, this
ignores the fact that other mm's (e.g. efi_mm) may also represent
parts of the kernel address space.
Introduce a new mm flag MMF_KERNEL and set it for init_mm.
Subsequent patches will use this flag to replace comparisons with
&init_mm. No functional change is introduced for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/linux/mm_types.h | 10 ++++++++++
mm/init-mm.c | 1 +
2 files changed, 11 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index a2e0dc5892ff..7838ea3aac00 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1414,6 +1414,8 @@ struct mm_struct {
char flexible_array[] __aligned(__alignof__(unsigned long));
};
+#define MM_FLAGS_INIT(flags) { .__mm_flags = { BITMAP_FROM_U64(flags) } }
+
static inline bool mm_flags_test(int flag, const struct mm_struct *mm)
{
return test_bit(flag, ACCESS_PRIVATE(&mm->flags, __mm_flags));
@@ -2001,6 +2003,9 @@ enum {
#define MMF_TOPDOWN 31 /* mm searches top down by default */
#define MMF_TOPDOWN_MASK BIT(MMF_TOPDOWN)
+#define MMF_KERNEL 32 /* mm belongs to the kernel */
+#define MMF_KERNEL_MASK BIT_ULL(MMF_KERNEL)
+
#define MMF_INIT_LEGACY_MASK (MMF_DUMP_FILTER_MASK |\
MMF_DISABLE_THP_MASK | MMF_HAS_MDWE_MASK |\
MMF_VM_MERGE_ANY_MASK | MMF_TOPDOWN_MASK)
@@ -2008,6 +2013,11 @@ enum {
/* Legacy flags must fit within 32 bits. */
static_assert((u64)MMF_INIT_LEGACY_MASK <= (u64)UINT_MAX);
+static inline bool mm_is_kernel(const struct mm_struct *mm)
+{
+ return mm && mm_flags_test(MMF_KERNEL, mm);
+}
+
/*
* Initialise legacy flags according to masks, propagating selected flags on
* fork. Further flag manipulation can be performed by the caller.
diff --git a/mm/init-mm.c b/mm/init-mm.c
index 3e792aad7626..93773269bf87 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -34,6 +34,7 @@ struct mm_struct init_mm = {
.pgd = swapper_pg_dir,
.mm_users = ATOMIC_INIT(2),
.mm_count = ATOMIC_INIT(1),
+ .flags = MM_FLAGS_INIT(MMF_KERNEL_MASK),
.write_protect_seq = SEQCNT_ZERO(init_mm.write_protect_seq),
MMAP_LOCK_INITIALIZER(init_mm)
.page_table_lock = __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock),
--
2.51.2
^ permalink raw reply related
* [PATCH 04/22] mm: use mm_is_kernel() in generic page table code
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
Generic page-table allocation, constructor and walker code uses
init_mm comparisons to select kernel page table handling. Use
mm_is_kernel() instead so kernel-owned mm's can share that handling
once they are marked with MMF_KERNEL.
No functional change as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/asm-generic/pgalloc.h | 16 ++++++++--------
include/linux/mm.h | 4 ++--
mm/memory.c | 6 +++---
mm/page_table_check.c | 14 +++++++-------
mm/pagewalk.c | 4 ++--
5 files changed, 22 insertions(+), 22 deletions(-)
diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h
index 051aa1331051..a283280e3140 100644
--- a/include/asm-generic/pgalloc.h
+++ b/include/asm-generic/pgalloc.h
@@ -138,7 +138,7 @@ static inline pmd_t *pmd_alloc_one_noprof(struct mm_struct *mm, unsigned long ad
struct ptdesc *ptdesc;
gfp_t gfp = GFP_PGTABLE_USER;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp = GFP_PGTABLE_KERNEL;
ptdesc = pagetable_alloc_noprof(gfp, 0);
if (!ptdesc)
@@ -148,7 +148,7 @@ static inline pmd_t *pmd_alloc_one_noprof(struct mm_struct *mm, unsigned long ad
return NULL;
}
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
ptdesc_set_kernel(ptdesc);
return ptdesc_address(ptdesc);
@@ -175,7 +175,7 @@ static inline pud_t *__pud_alloc_one_noprof(struct mm_struct *mm, unsigned long
gfp_t gfp = GFP_PGTABLE_USER;
struct ptdesc *ptdesc;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp = GFP_PGTABLE_KERNEL;
ptdesc = pagetable_alloc_noprof(gfp, 0);
@@ -184,7 +184,7 @@ static inline pud_t *__pud_alloc_one_noprof(struct mm_struct *mm, unsigned long
pagetable_pud_ctor(ptdesc);
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
ptdesc_set_kernel(ptdesc);
return ptdesc_address(ptdesc);
@@ -232,7 +232,7 @@ static inline p4d_t *__p4d_alloc_one_noprof(struct mm_struct *mm, unsigned long
gfp_t gfp = GFP_PGTABLE_USER;
struct ptdesc *ptdesc;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp = GFP_PGTABLE_KERNEL;
ptdesc = pagetable_alloc_noprof(gfp, 0);
@@ -241,7 +241,7 @@ static inline p4d_t *__p4d_alloc_one_noprof(struct mm_struct *mm, unsigned long
pagetable_p4d_ctor(ptdesc);
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
ptdesc_set_kernel(ptdesc);
return ptdesc_address(ptdesc);
@@ -279,7 +279,7 @@ static inline pgd_t *__pgd_alloc_noprof(struct mm_struct *mm, unsigned int order
gfp_t gfp = GFP_PGTABLE_USER;
struct ptdesc *ptdesc;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp = GFP_PGTABLE_KERNEL;
ptdesc = pagetable_alloc_noprof(gfp, order);
@@ -288,7 +288,7 @@ static inline pgd_t *__pgd_alloc_noprof(struct mm_struct *mm, unsigned int order
pagetable_pgd_ctor(ptdesc);
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
ptdesc_set_kernel(ptdesc);
return ptdesc_address(ptdesc);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7a4b8e7ac8f4..b336eb32faa0 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3818,7 +3818,7 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc)
static inline bool pagetable_pte_ctor(struct mm_struct *mm,
struct ptdesc *ptdesc)
{
- if (mm != &init_mm && !ptlock_init(ptdesc))
+ if (!mm_is_kernel(mm) && !ptlock_init(ptdesc))
return false;
__pagetable_ctor(ptdesc);
return true;
@@ -3909,7 +3909,7 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd)
static inline bool pagetable_pmd_ctor(struct mm_struct *mm,
struct ptdesc *ptdesc)
{
- if (mm != &init_mm && !pmd_ptlock_init(ptdesc))
+ if (!mm_is_kernel(mm) && !pmd_ptlock_init(ptdesc))
return false;
ptdesc_pmd_pts_init(ptdesc);
__pagetable_ctor(ptdesc);
diff --git a/mm/memory.c b/mm/memory.c
index d5e87624f692..c0244c0b0756 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3394,13 +3394,13 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,
spinlock_t *ptl;
if (create) {
- mapped_pte = pte = (mm == &init_mm) ?
+ mapped_pte = pte = mm_is_kernel(mm) ?
pte_alloc_kernel_track(pmd, addr, mask) :
pte_alloc_map_lock(mm, pmd, addr, &ptl);
if (!pte)
return -ENOMEM;
} else {
- mapped_pte = pte = (mm == &init_mm) ?
+ mapped_pte = pte = mm_is_kernel(mm) ?
pte_offset_kernel(pmd, addr) :
pte_offset_map_lock(mm, pmd, addr, &ptl);
if (!pte)
@@ -3422,7 +3422,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,
lazy_mmu_mode_disable();
- if (mm != &init_mm)
+ if (!mm_is_kernel(mm))
pte_unmap_unlock(mapped_pte, ptl);
return err;
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 3fb995e5d40d..a5382c245b32 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -148,7 +148,7 @@ void __page_table_check_zero(struct page *page, unsigned int order)
void __page_table_check_pte_clear(struct mm_struct *mm, unsigned long addr,
pte_t pte)
{
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
if (pte_user_accessible_page(mm, addr, pte))
@@ -159,7 +159,7 @@ EXPORT_SYMBOL(__page_table_check_pte_clear);
void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
pmd_t pmd)
{
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
if (pmd_user_accessible_page(mm, addr, pmd))
@@ -170,7 +170,7 @@ EXPORT_SYMBOL(__page_table_check_pmd_clear);
void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
pud_t pud)
{
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
if (pud_user_accessible_page(mm, addr, pud))
@@ -201,7 +201,7 @@ void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
{
unsigned int i;
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
page_table_check_pte_flags(pte);
@@ -231,7 +231,7 @@ void __page_table_check_pmds_set(struct mm_struct *mm, unsigned long addr,
unsigned long stride = PMD_SIZE >> PAGE_SHIFT;
unsigned int i;
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
page_table_check_pmd_flags(pmd);
@@ -249,7 +249,7 @@ void __page_table_check_puds_set(struct mm_struct *mm, unsigned long addr,
unsigned long stride = PUD_SIZE >> PAGE_SHIFT;
unsigned int i;
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
for (i = 0; i < nr; i++)
@@ -263,7 +263,7 @@ void __page_table_check_pte_clear_range(struct mm_struct *mm,
unsigned long addr,
pmd_t pmd)
{
- if (&init_mm == mm)
+ if (mm_is_kernel(mm))
return;
if (!pmd_bad(pmd) && !pmd_leaf(pmd)) {
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index ed4860c01936..a3f47d1de3c5 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -72,13 +72,13 @@ static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
* fit its pmd_bad() check (_PAGE_NX set and _PAGE_RW clear),
* and CONFIG_EFI_PGT_DUMP efi_mm goes so far as to walk them.
*/
- if (walk->mm == &init_mm || addr >= TASK_SIZE)
+ if (mm_is_kernel(walk->mm) || addr >= TASK_SIZE)
pte = pte_offset_kernel(pmd, addr);
else
pte = pte_offset_map(pmd, addr);
if (pte) {
err = walk_pte_range_inner(pte, addr, end, walk);
- if (walk->mm != &init_mm && addr < TASK_SIZE)
+ if (!mm_is_kernel(walk->mm) && addr < TASK_SIZE)
pte_unmap(pte);
}
} else {
--
2.51.2
^ permalink raw reply related
* [PATCH 05/22] arm64: mm: use mm_is_kernel() for kernel mm checks
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
In some cases, such as show_pte(), the handling is truly specific to
init_mm and we leave those comparisons with &init_mm unchanged.
No functional change, as only init_mm has MMF_KERNEL set for now.
mm_is_user() will eventually be replaced with !mm_is_kernel(), once
efi_mm is also marked with MMF_KERNEL.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/arm64/include/asm/pgalloc.h | 10 +++++-----
arch/arm64/mm/contpte.c | 2 +-
arch/arm64/mm/hugetlbpage.c | 2 +-
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h
index 1b4509d3382c..33db4fa3c619 100644
--- a/arch/arm64/include/asm/pgalloc.h
+++ b/arch/arm64/include/asm/pgalloc.h
@@ -30,7 +30,7 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmdp)
{
pudval_t pudval = PUD_TYPE_TABLE | PUD_TABLE_AF;
- pudval |= (mm == &init_mm) ? PUD_TABLE_UXN : PUD_TABLE_PXN;
+ pudval |= mm_is_kernel(mm) ? PUD_TABLE_UXN : PUD_TABLE_PXN;
__pud_populate(pudp, __pa(pmdp), pudval);
}
#else
@@ -52,7 +52,7 @@ static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4dp, pud_t *pudp)
{
p4dval_t p4dval = P4D_TYPE_TABLE | P4D_TABLE_AF;
- p4dval |= (mm == &init_mm) ? P4D_TABLE_UXN : P4D_TABLE_PXN;
+ p4dval |= mm_is_kernel(mm) ? P4D_TABLE_UXN : P4D_TABLE_PXN;
__p4d_populate(p4dp, __pa(pudp), p4dval);
}
@@ -81,7 +81,7 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgdp, p4d_t *p4dp)
{
pgdval_t pgdval = PGD_TYPE_TABLE | PGD_TABLE_AF;
- pgdval |= (mm == &init_mm) ? PGD_TABLE_UXN : PGD_TABLE_PXN;
+ pgdval |= mm_is_kernel(mm) ? PGD_TABLE_UXN : PGD_TABLE_PXN;
__pgd_populate(pgdp, __pa(p4dp), pgdval);
}
@@ -108,7 +108,7 @@ static inline void __pmd_populate(pmd_t *pmdp, phys_addr_t ptep,
static inline void
pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep)
{
- VM_BUG_ON(mm && mm != &init_mm);
+ VM_BUG_ON(mm && !mm_is_kernel(mm));
__pmd_populate(pmdp, __pa(ptep),
PMD_TYPE_TABLE | PMD_TABLE_AF | PMD_TABLE_UXN);
}
@@ -116,7 +116,7 @@ pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep)
static inline void
pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep)
{
- VM_BUG_ON(mm == &init_mm);
+ VM_BUG_ON(mm_is_kernel(mm));
__pmd_populate(pmdp, page_to_phys(ptep),
PMD_TYPE_TABLE | PMD_TABLE_AF | PMD_TABLE_PXN);
}
diff --git a/arch/arm64/mm/contpte.c b/arch/arm64/mm/contpte.c
index 2de12656b4d8..ad5c04cf66ff 100644
--- a/arch/arm64/mm/contpte.c
+++ b/arch/arm64/mm/contpte.c
@@ -18,7 +18,7 @@ static inline bool mm_is_user(struct mm_struct *mm)
*/
if (unlikely(mm_is_efi(mm)))
return false;
- return mm != &init_mm;
+ return !mm_is_kernel(mm);
}
static inline pte_t *contpte_align_down(pte_t *ptep)
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 25f04d01aabe..4d054fadf51c 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -212,7 +212,7 @@ static void clear_flush(struct mm_struct *mm,
for (i = 0; i < ncontig; i++, addr += pgsize, ptep++)
__ptep_get_and_clear_anysz(mm, addr, ptep, pgsize);
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
flush_tlb_kernel_range(saddr, addr);
else
__flush_hugetlb_tlb_range(&vma, saddr, addr, pgsize, TLBF_NOWALKCACHE);
--
2.51.2
^ permalink raw reply related
* [PATCH 06/22] loongarch: mm: use mm_is_kernel() in switch_mm_irqs_off()
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/loongarch/include/asm/mmu_context.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/loongarch/include/asm/mmu_context.h b/arch/loongarch/include/asm/mmu_context.h
index 304363bd3935..6d51f11427b4 100644
--- a/arch/loongarch/include/asm/mmu_context.h
+++ b/arch/loongarch/include/asm/mmu_context.h
@@ -95,7 +95,7 @@ static inline void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *
if (!asid_valid(next, cpu))
get_new_mmu_context(next, cpu, &need_flush);
- if (next != &init_mm)
+ if (!mm_is_kernel(next))
atomic_update_pgd_asid(cpu_asid(cpu, next), (unsigned long)next->pgd);
else
atomic_update_pgd_asid(cpu_asid(cpu, next), (unsigned long)invalid_pg_dir);
--
2.51.2
^ permalink raw reply related
* [PATCH 07/22] parisc: mm: use mm_is_kernel() for kernel mm checks
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/parisc/include/asm/mmu_context.h | 2 +-
arch/parisc/include/asm/pgalloc.h | 2 +-
arch/parisc/include/asm/tlbflush.h | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/parisc/include/asm/mmu_context.h b/arch/parisc/include/asm/mmu_context.h
index c9187fe836a3..e265da67559e 100644
--- a/arch/parisc/include/asm/mmu_context.h
+++ b/arch/parisc/include/asm/mmu_context.h
@@ -87,7 +87,7 @@ static inline void activate_mm(struct mm_struct *prev, struct mm_struct *next)
* already, so we should be OK.
*/
- BUG_ON(next == &init_mm); /* Should never happen */
+ BUG_ON(mm_is_kernel(next)); /* Should never happen */
if (next->context.space_id == 0)
next->context.space_id = alloc_sid();
diff --git a/arch/parisc/include/asm/pgalloc.h b/arch/parisc/include/asm/pgalloc.h
index 3b84ee93edaa..9f09cc4fa5dd 100644
--- a/arch/parisc/include/asm/pgalloc.h
+++ b/arch/parisc/include/asm/pgalloc.h
@@ -34,7 +34,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
struct ptdesc *ptdesc;
gfp_t gfp = GFP_PGTABLE_USER;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp = GFP_PGTABLE_KERNEL;
ptdesc = pagetable_alloc(gfp, PMD_TABLE_ORDER);
if (!ptdesc)
diff --git a/arch/parisc/include/asm/tlbflush.h b/arch/parisc/include/asm/tlbflush.h
index 5ffd7c17f593..a5a6cfff295c 100644
--- a/arch/parisc/include/asm/tlbflush.h
+++ b/arch/parisc/include/asm/tlbflush.h
@@ -35,7 +35,7 @@ int __flush_tlb_range(unsigned long sid,
static inline void flush_tlb_mm(struct mm_struct *mm)
{
- BUG_ON(mm == &init_mm); /* Should never happen */
+ BUG_ON(mm_is_kernel(mm)); /* Should never happen */
#if 1 || defined(CONFIG_SMP)
/* Except for very small threads, flushing the whole TLB is
--
2.51.2
^ permalink raw reply related
* [PATCH 08/22] powerpc: mm: use mm_is_kernel() for kernel mm checks
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/powerpc/include/asm/pgalloc.h | 2 +-
arch/powerpc/mm/book3s64/pgtable.c | 2 +-
arch/powerpc/mm/mmu_context.c | 2 +-
arch/powerpc/mm/pgtable.c | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/pgalloc.h b/arch/powerpc/include/asm/pgalloc.h
index 3a971e2a8c73..88acccc5b06e 100644
--- a/arch/powerpc/include/asm/pgalloc.h
+++ b/arch/powerpc/include/asm/pgalloc.h
@@ -7,7 +7,7 @@
#ifndef MODULE
static inline gfp_t pgtable_gfp_flags(struct mm_struct *mm, gfp_t gfp)
{
- if (unlikely(mm == &init_mm))
+ if (unlikely(mm_is_kernel(mm)))
return gfp;
return gfp | __GFP_ACCOUNT;
}
diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c
index 85ab6723c8f2..eb365f7c8907 100644
--- a/arch/powerpc/mm/book3s64/pgtable.c
+++ b/arch/powerpc/mm/book3s64/pgtable.c
@@ -395,7 +395,7 @@ static pmd_t *__alloc_for_pmdcache(struct mm_struct *mm)
struct ptdesc *ptdesc;
gfp_t gfp = GFP_KERNEL_ACCOUNT | __GFP_ZERO;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp &= ~__GFP_ACCOUNT;
ptdesc = pagetable_alloc(gfp, 0);
if (!ptdesc)
diff --git a/arch/powerpc/mm/mmu_context.c b/arch/powerpc/mm/mmu_context.c
index 3e3af29b4523..b51934c511f3 100644
--- a/arch/powerpc/mm/mmu_context.c
+++ b/arch/powerpc/mm/mmu_context.c
@@ -48,7 +48,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
/* Mark this context has been used on the new CPU */
if (!cpumask_test_cpu(cpu, mm_cpumask(next))) {
- VM_WARN_ON_ONCE(next == &init_mm);
+ VM_WARN_ON_ONCE(mm_is_kernel(next));
cpumask_set_cpu(cpu, mm_cpumask(next));
inc_mm_active_cpus(next);
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index a9be337be3e4..d04f4fecfc7c 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -393,7 +393,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr)
pte_t *pte;
spinlock_t *ptl;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
return;
pgd = mm->pgd + pgd_index(addr);
BUG_ON(pgd_none(*pgd));
--
2.51.2
^ permalink raw reply related
* [PATCH 09/22] s390: mm: use mm_is_kernel() for kernel mm checks
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/s390/include/asm/mmu_context.h | 2 +-
arch/s390/include/asm/pgtable.h | 6 +++---
arch/s390/mm/pgalloc.c | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/s390/include/asm/mmu_context.h b/arch/s390/include/asm/mmu_context.h
index bd1ef5e2d2eb..675f270d0b70 100644
--- a/arch/s390/include/asm/mmu_context.h
+++ b/arch/s390/include/asm/mmu_context.h
@@ -69,7 +69,7 @@ static inline void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *
{
int cpu = smp_processor_id();
- if (next == &init_mm)
+ if (mm_is_kernel(next))
get_lowcore()->user_asce = s390_invalid_asce;
else
get_lowcore()->user_asce.val = next->context.asce;
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index e882663a58e7..f7741e6b417e 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -1836,21 +1836,21 @@ static inline int has_transparent_hugepage(void)
#ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
- VM_BUG_ON(mm == &init_mm);
+ VM_BUG_ON(mm_is_kernel(mm));
return pte_present(pte);
}
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
- VM_BUG_ON(mm == &init_mm);
+ VM_BUG_ON(mm_is_kernel(mm));
return pmd_leaf(pmd) && (pmd_val(pmd) & _SEGMENT_ENTRY_READ);
}
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
{
- VM_BUG_ON(mm == &init_mm);
+ VM_BUG_ON(mm_is_kernel(mm));
return pud_leaf(pud);
}
diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c
index 7ac44543e051..d0456b7e73d4 100644
--- a/arch/s390/mm/pgalloc.c
+++ b/arch/s390/mm/pgalloc.c
@@ -20,7 +20,7 @@ unsigned long *crst_table_alloc_noprof(struct mm_struct *mm)
struct ptdesc *ptdesc;
unsigned long *table;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp &= ~__GFP_ACCOUNT;
ptdesc = pagetable_alloc_noprof(gfp, CRST_ALLOC_ORDER);
if (!ptdesc)
@@ -120,7 +120,7 @@ unsigned long *page_table_alloc_noprof(struct mm_struct *mm)
struct ptdesc *ptdesc;
unsigned long *table;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp &= ~__GFP_ACCOUNT;
ptdesc = pagetable_alloc_noprof(gfp, 0);
if (!ptdesc)
--
2.51.2
^ permalink raw reply related
* [PATCH 10/22] sparc: mm: use mm_is_kernel() for kernel mm checks
From: Kevin Brodsky @ 2026-07-14 14:03 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
pgtable_64.h previously forward-declared init_mm to avoid pulling
in mm_types.h. It seems that including mm_types.h is not causing
troubles any more, so we can also use mm_is_kernel() there and drop
the forward declaration.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/sparc/include/asm/mmu_context_64.h | 2 +-
arch/sparc/include/asm/pgtable_64.h | 6 ++----
arch/sparc/mm/init_64.c | 2 +-
arch/sparc/mm/tlb.c | 2 +-
4 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/arch/sparc/include/asm/mmu_context_64.h b/arch/sparc/include/asm/mmu_context_64.h
index 78bbacc14d2d..e88bc61aeb04 100644
--- a/arch/sparc/include/asm/mmu_context_64.h
+++ b/arch/sparc/include/asm/mmu_context_64.h
@@ -83,7 +83,7 @@ static inline void switch_mm(struct mm_struct *old_mm, struct mm_struct *mm, str
int cpu = smp_processor_id();
per_cpu(per_cpu_secondary_mm, cpu) = mm;
- if (unlikely(mm == &init_mm))
+ if (unlikely(mm_is_kernel(mm)))
return;
spin_lock_irqsave(&mm->context.lock, flags);
diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h
index 0837ebbc5dce..960f8bf1ec5a 100644
--- a/arch/sparc/include/asm/pgtable_64.h
+++ b/arch/sparc/include/asm/pgtable_64.h
@@ -86,6 +86,7 @@ extern unsigned long VMALLOC_END;
#define vmemmap ((struct page *)VMEMMAP_BASE)
#include <linux/sched.h>
+#include <linux/mm_types.h>
#include <asm/tlbflush.h>
bool kern_addr_valid(unsigned long addr);
@@ -885,9 +886,6 @@ static inline unsigned long pud_pfn(pud_t pud)
#define p4d_set(p4dp, pudp) \
(p4d_val(*(p4dp)) = (__pa((unsigned long) (pudp))))
-/* We cannot include <linux/mm_types.h> at this point yet: */
-extern struct mm_struct init_mm;
-
/* Actual page table PTE updates. */
void tlb_batch_add(struct mm_struct *mm, unsigned long vaddr,
pte_t *ptep, pte_t orig, int fullmm,
@@ -903,7 +901,7 @@ static void maybe_tlb_batch_add(struct mm_struct *mm, unsigned long vaddr,
* SUN4V NOTE: _PAGE_VALID is the same value in both the SUN4U
* and SUN4V pte layout, so this inline test is fine.
*/
- if (likely(mm != &init_mm) && pte_accessible(mm, orig))
+ if (likely(!mm_is_kernel(mm)) && pte_accessible(mm, orig))
tlb_batch_add(mm, vaddr, ptep, orig, fullmm, hugepage_shift);
}
diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c
index 103db4683b16..e0b678349e3a 100644
--- a/arch/sparc/mm/init_64.c
+++ b/arch/sparc/mm/init_64.c
@@ -788,7 +788,7 @@ static void mmu_context_wrap(void)
*/
mm = per_cpu(per_cpu_secondary_mm, cpu);
- if (unlikely(!mm || mm == &init_mm))
+ if (unlikely(!mm || mm_is_kernel(mm)))
continue;
old_ctx = mm->context.sparc64_ctx_val;
diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c
index 6d9dd5eb1328..47083c287a94 100644
--- a/arch/sparc/mm/tlb.c
+++ b/arch/sparc/mm/tlb.c
@@ -179,7 +179,7 @@ static void tlb_batch_pmd_scan(struct mm_struct *mm, unsigned long vaddr,
static void __set_pmd_acct(struct mm_struct *mm, unsigned long addr,
pmd_t orig, pmd_t pmd)
{
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
return;
if ((pmd_val(pmd) ^ pmd_val(orig)) & _PAGE_PMD_HUGE) {
--
2.51.2
^ permalink raw reply related
* [PATCH 11/22] um: mm: use mm_is_kernel() in TLB sync
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/um/kernel/tlb.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/um/kernel/tlb.c b/arch/um/kernel/tlb.c
index 1f175716b474..fa34a710203e 100644
--- a/arch/um/kernel/tlb.c
+++ b/arch/um/kernel/tlb.c
@@ -171,7 +171,7 @@ int um_tlb_sync(struct mm_struct *mm)
return 0;
ops.mm_idp = &mm->context.id;
- if (mm == &init_mm) {
+ if (mm_is_kernel(mm)) {
ops.mmap = kern_map;
ops.unmap = kern_unmap;
} else {
--
2.51.2
^ permalink raw reply related
* [PATCH 12/22] x86/mm: use mm_is_kernel() for kernel mm checks
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
The new MMF_KERNEL flag identifies kernel-owned mm's. Checking the
flag with mm_is_kernel() is preferred over comparing directly
against &init_mm.
No functional change, as only init_mm has MMF_KERNEL set for now.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/x86/include/asm/tlbflush.h | 8 ++++----
arch/x86/mm/pgtable.c | 2 +-
arch/x86/mm/tlb.c | 9 +++++----
3 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 0545fe75c3fa..4af1cf711a60 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -159,10 +159,10 @@ struct tlb_state_shared {
* - Actively using an mm. Our CPU's bit will be set in
* mm_cpumask(loaded_mm) and is_lazy == false;
*
- * - Not using a real mm. loaded_mm == &init_mm. Our CPU's bit
- * will not be set in mm_cpumask(&init_mm) and is_lazy == false.
+ * - Not using a real mm. loaded_mm is a kernel mm. Our CPU's bit
+ * will not be set in mm_cpumask(loaded_mm) and is_lazy == false.
*
- * - Lazily using a real mm. loaded_mm != &init_mm, our bit
+ * - Lazily using a real mm. loaded_mm is not a kernel mm, our bit
* is set in mm_cpumask(loaded_mm), but is_lazy == true.
* We're heuristically guessing that the CR3 load we
* skipped more than makes up for the overhead added by
@@ -188,7 +188,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state_shared, cpu_tlbstate_shared);
#define enter_lazy_tlb enter_lazy_tlb
static __always_inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
{
- if (this_cpu_read(cpu_tlbstate.loaded_mm) == &init_mm)
+ if (mm_is_kernel(this_cpu_read(cpu_tlbstate.loaded_mm)))
return;
this_cpu_write(cpu_tlbstate_shared.is_lazy, true);
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index f32facdb3035..91d2a6f2ac5a 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -168,7 +168,7 @@ static int preallocate_pmds(struct mm_struct *mm, pmd_t *pmds[], int count)
bool failed = false;
gfp_t gfp = GFP_PGTABLE_USER;
- if (mm == &init_mm)
+ if (mm_is_kernel(mm))
gfp &= ~__GFP_ACCOUNT;
gfp &= ~__GFP_HIGHMEM;
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 1023acadd8f8..5517cab8ad00 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -594,7 +594,7 @@ void leave_mm(void)
* This needs to happen before any other sanity checks due to
* intel_idle's shenanigans.
*/
- if (loaded_mm == &init_mm)
+ if (mm_is_kernel(loaded_mm))
return;
/* Warn if we're not lazy. */
@@ -855,7 +855,7 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next,
* cpu_tlbstate_shared.is_lazy whether or not to send an IPI.
*/
if (IS_ENABLED(CONFIG_DEBUG_VM) &&
- WARN_ON_ONCE(prev != &init_mm && !is_notrack_mm(prev) &&
+ WARN_ON_ONCE(!mm_is_kernel(prev) && !is_notrack_mm(prev) &&
!cpumask_test_cpu(cpu, mm_cpumask(next))))
cpumask_set_cpu(cpu, mm_cpumask(next));
@@ -932,7 +932,8 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next,
* This way switch_mm() must see the new tlb_gen or
* flush_tlb_mm_range() must see the new loaded_mm, or both.
*/
- if (next != &init_mm && !cpumask_test_cpu(cpu, mm_cpumask(next)))
+ if (!mm_is_kernel(next) &&
+ !cpumask_test_cpu(cpu, mm_cpumask(next)))
cpumask_set_cpu(cpu, mm_cpumask(next));
else
smp_mb();
@@ -1134,7 +1135,7 @@ static void flush_tlb_func(void *info)
return;
}
- if (unlikely(loaded_mm == &init_mm))
+ if (unlikely(mm_is_kernel(loaded_mm)))
return;
/* Reload the ASID if transitioning into or out of a global ASID */
--
2.51.2
^ permalink raw reply related
* [PATCH 13/22] mm: account page table pages when allocated
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
As a step towards removing pagetable ctor/dtor calls for kernel
mm's, move the memory accounting from the ctor/dtor to
pagetable_{alloc,free}.
The ctor is normally called after pagetable_alloc() (and dtor before
pagetable_free()) so this patch should have no functional effect in
most cases. Where calls to ctor/dtor are currently missing (e.g.
x86's split_large_page()), this patch should improve the accounting.
This patch is based on Matthew's earlier implementation linked
below, with the difference that when freeing the accounting is
updated in __pagetable_free(), meaning that the update is delayed
until the pages are actually freed in the
CONFIG_ASYNC_KERNEL_PGTABLE_FREE case. This shouldn't matter in
practice.
Suggested-by: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Link: https://lore.kernel.org/r/20251113140448.1814860-3-willy@infradead.org
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/linux/mm.h | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index b336eb32faa0..361a0d7b74c5 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3681,6 +3681,11 @@ static inline struct ptdesc *pagetable_alloc_noprof(gfp_t gfp, unsigned int orde
{
struct page *page = alloc_pages_noprof(gfp | __GFP_COMP, order);
+ if (!page)
+ return NULL;
+
+ lruvec_stat_add_folio(page_folio(page), NR_PAGETABLE);
+
return page_ptdesc(page);
}
#define pagetable_alloc(...) alloc_hooks(pagetable_alloc_noprof(__VA_ARGS__))
@@ -3689,6 +3694,8 @@ static inline void __pagetable_free(struct ptdesc *pt)
{
struct page *page = ptdesc_page(pt);
+ lruvec_stat_sub_folio(page_folio(page), NR_PAGETABLE);
+
__free_pages(page, compound_order(page));
}
@@ -3797,7 +3804,6 @@ static inline void __pagetable_ctor(struct ptdesc *ptdesc)
struct folio *folio = ptdesc_folio(ptdesc);
__folio_set_pgtable(folio);
- lruvec_stat_add_folio(folio, NR_PAGETABLE);
}
static inline void pagetable_dtor(struct ptdesc *ptdesc)
@@ -3806,7 +3812,6 @@ static inline void pagetable_dtor(struct ptdesc *ptdesc)
ptlock_free(ptdesc);
__folio_clear_pgtable(folio);
- lruvec_stat_sub_folio(folio, NR_PAGETABLE);
}
static inline void pagetable_dtor_free(struct ptdesc *ptdesc)
--
2.51.2
^ permalink raw reply related
* [PATCH 14/22] mm: set page table page type when allocated
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
As a step towards removing pagetable ctor/dtor calls for kernel
mm's, move the page type updates from the ctor/dtor to
pagetable_{alloc,free}.
__pagetable_ctor() is now empty so remove it;
pagetable_{pud,p4d,pgd}_ctor() will be removed at a later stage.
The ctor is normally called after pagetable_alloc() (and dtor before
pagetable_free()) so this patch should have no functional effect in
most cases. Where calls to ctor/dtor are currently missing (e.g.
x86's split_large_page()), this patch will ensure the page type is
set correctly.
This patch is based on Matthew's earlier implementation linked
below, with the difference that when freeing the page type is
cleared in __pagetable_free(), meaning that the update is delayed
until the pages are actually freed in the
CONFIG_ASYNC_KERNEL_PGTABLE_FREE case. This shouldn't matter in
practice.
Suggested-by: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Link: https://lore.kernel.org/r/20251113140448.1814860-4-willy@infradead.org
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/linux/mm.h | 17 ++---------------
1 file changed, 2 insertions(+), 15 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 361a0d7b74c5..94f0fb1c662a 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3684,6 +3684,7 @@ static inline struct ptdesc *pagetable_alloc_noprof(gfp_t gfp, unsigned int orde
if (!page)
return NULL;
+ __folio_set_pgtable(page_folio(page));
lruvec_stat_add_folio(page_folio(page), NR_PAGETABLE);
return page_ptdesc(page);
@@ -3694,6 +3695,7 @@ static inline void __pagetable_free(struct ptdesc *pt)
{
struct page *page = ptdesc_page(pt);
+ __folio_clear_pgtable(page_folio(page));
lruvec_stat_sub_folio(page_folio(page), NR_PAGETABLE);
__free_pages(page, compound_order(page));
@@ -3799,19 +3801,9 @@ static inline bool ptlock_init(struct ptdesc *ptdesc) { return true; }
static inline void ptlock_free(struct ptdesc *ptdesc) {}
#endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */
-static inline void __pagetable_ctor(struct ptdesc *ptdesc)
-{
- struct folio *folio = ptdesc_folio(ptdesc);
-
- __folio_set_pgtable(folio);
-}
-
static inline void pagetable_dtor(struct ptdesc *ptdesc)
{
- struct folio *folio = ptdesc_folio(ptdesc);
-
ptlock_free(ptdesc);
- __folio_clear_pgtable(folio);
}
static inline void pagetable_dtor_free(struct ptdesc *ptdesc)
@@ -3825,7 +3817,6 @@ static inline bool pagetable_pte_ctor(struct mm_struct *mm,
{
if (!mm_is_kernel(mm) && !ptlock_init(ptdesc))
return false;
- __pagetable_ctor(ptdesc);
return true;
}
@@ -3917,7 +3908,6 @@ static inline bool pagetable_pmd_ctor(struct mm_struct *mm,
if (!mm_is_kernel(mm) && !pmd_ptlock_init(ptdesc))
return false;
ptdesc_pmd_pts_init(ptdesc);
- __pagetable_ctor(ptdesc);
return true;
}
@@ -3942,17 +3932,14 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud)
static inline void pagetable_pud_ctor(struct ptdesc *ptdesc)
{
- __pagetable_ctor(ptdesc);
}
static inline void pagetable_p4d_ctor(struct ptdesc *ptdesc)
{
- __pagetable_ctor(ptdesc);
}
static inline void pagetable_pgd_ctor(struct ptdesc *ptdesc)
{
- __pagetable_ctor(ptdesc);
}
extern void __init pagecache_init(void);
--
2.51.2
^ permalink raw reply related
* [PATCH 15/22] mm: only initialise pt_share_count for user pgtables
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
ptdesc_pmd_pts_init() initialises the pt_share_count ptdesc field.
That field is only used for hugetlb tracking purposes, so there is
no need to initialise it for kernel page tables. Skip the call for
kernel page tables, like pmd_ptlock_init().
From now on pagetable_*_ctor() and pagetable_dtor() do nothing for
kernel page tables.
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
include/linux/mm.h | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 94f0fb1c662a..b217ccbed5a3 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3905,9 +3905,12 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd)
static inline bool pagetable_pmd_ctor(struct mm_struct *mm,
struct ptdesc *ptdesc)
{
- if (!mm_is_kernel(mm) && !pmd_ptlock_init(ptdesc))
- return false;
- ptdesc_pmd_pts_init(ptdesc);
+ if (!mm_is_kernel(mm)) {
+ if (!pmd_ptlock_init(ptdesc))
+ return false;
+ ptdesc_pmd_pts_init(ptdesc);
+ }
+
return true;
}
--
2.51.2
^ permalink raw reply related
* [PATCH 16/22] efi: mark efi_mm as a kernel mm
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
efi_mm is a kernel-owned address space used for EFI runtime
mappings, but it is currently treated like a user address space when
manipulating its page tables. There does not seem to be a good
reason for this, and it creates unnecessary complications when
creating its page tables.
Mark it with the new MMF_KERNEL flag so that the page table handling
treats it like init_mm. In particular, this means that:
* Split page table locks are no longer initialised or used for
efi_mm's page tables.
* pagetable_*_ctor() and pagetable_dtor() no longer need to be
called for efi_mm's page tables.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
drivers/firmware/efi/efi.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 0327a39d31fa..e5ca45bdc890 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -69,6 +69,7 @@ struct mm_struct efi_mm = {
.mm_mt = MTREE_INIT_EXT(mm_mt, MM_MT_FLAGS, efi_mm.mmap_lock),
.mm_users = ATOMIC_INIT(2),
.mm_count = ATOMIC_INIT(1),
+ .flags = MM_FLAGS_INIT(MMF_KERNEL_MASK),
.write_protect_seq = SEQCNT_ZERO(efi_mm.write_protect_seq),
MMAP_LOCK_INITIALIZER(efi_mm)
.page_table_lock = __SPIN_LOCK_UNLOCKED(efi_mm.page_table_lock),
--
2.51.2
^ permalink raw reply related
* [PATCH 17/22] mm: pagewalk: drop redundant address check for kernel mm walks
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
Commit 8b1cb4a2e819 ("mm/pagewalk: fix EFI_PGT_DUMP of
espfix area") made no-VMA page table walks use
pte_offset_kernel() for any address above TASK_SIZE. This avoided
pte_offset_map() validation on EFI page table dumps, where efi_mm
could walk x86 espfix PMDs that intentionally look bad to pmd_bad().
efi_mm is now marked with MMF_KERNEL, and no-VMA walks use
mm_is_kernel() to select pte_offset_kernel() for kernel mm's. As a
result we no longer need to handle the EFI_PGT_DUMP case explicitly.The
address check is therefore redundant for EFI_PGT_DUMP and makes the
condition harder to reason about.
Remove the address check and associated comment; mm_is_kernel()
should be sufficient on its own.
Tested by booting x86 QEMU with CONFIG_EFI_PGT_DUMP=y and reading
/sys/kernel/debug/page_tables/efi.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
mm/pagewalk.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index a3f47d1de3c5..0dd9d4ad9b68 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -68,17 +68,14 @@ static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
if (walk->no_vma) {
/*
* pte_offset_map() might apply user-specific validation.
- * Indeed, on x86_64 the pmd entries set up by init_espfix_ap()
- * fit its pmd_bad() check (_PAGE_NX set and _PAGE_RW clear),
- * and CONFIG_EFI_PGT_DUMP efi_mm goes so far as to walk them.
*/
- if (mm_is_kernel(walk->mm) || addr >= TASK_SIZE)
+ if (mm_is_kernel(walk->mm))
pte = pte_offset_kernel(pmd, addr);
else
pte = pte_offset_map(pmd, addr);
if (pte) {
err = walk_pte_range_inner(pte, addr, end, walk);
- if (!mm_is_kernel(walk->mm) && addr < TASK_SIZE)
+ if (!mm_is_kernel(walk->mm))
pte_unmap(pte);
}
} else {
--
2.51.2
^ permalink raw reply related
* [PATCH 18/22] arm64: mm: drop explicit mm_is_efi() check in contpte
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
efi_mm is now marked as a kernel mm, like init_mm. mm_is_kernel()
therefore gives the right answer for efi_mm and mm_is_user() is now
equivalent to !mm_is_kernel().
Remove mm_is_user() and directly use mm_is_kernel() instead.
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/arm64/mm/contpte.c | 26 +++++++++-----------------
1 file changed, 9 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/mm/contpte.c b/arch/arm64/mm/contpte.c
index ad5c04cf66ff..e21e279101ca 100644
--- a/arch/arm64/mm/contpte.c
+++ b/arch/arm64/mm/contpte.c
@@ -4,23 +4,9 @@
*/
#include <linux/mm.h>
-#include <linux/efi.h>
#include <linux/export.h>
#include <asm/tlbflush.h>
-static inline bool mm_is_user(struct mm_struct *mm)
-{
- /*
- * Don't attempt to apply the contig bit to kernel mappings, because
- * dynamically adding/removing the contig bit can cause page faults.
- * These racing faults are ok for user space, since they get serialized
- * on the PTL. But kernel mappings can't tolerate faults.
- */
- if (unlikely(mm_is_efi(mm)))
- return false;
- return !mm_is_kernel(mm);
-}
-
static inline pte_t *contpte_align_down(pte_t *ptep)
{
return PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * CONT_PTES);
@@ -263,7 +249,13 @@ void __contpte_try_fold(struct mm_struct *mm, unsigned long addr,
int i;
- if (!mm_is_user(mm))
+ /*
+ * Don't attempt to apply the contig bit to kernel mappings, because
+ * dynamically adding/removing the contig bit can cause page faults.
+ * These racing faults are ok for user space, since they get serialized
+ * on the PTL. But kernel mappings can't tolerate faults.
+ */
+ if (mm_is_kernel(mm))
return;
page = pte_page(pte);
@@ -302,7 +294,7 @@ void __contpte_try_unfold(struct mm_struct *mm, unsigned long addr,
* We have already checked that the ptes are contiguous in
* contpte_try_unfold(), so just check that the mm is user space.
*/
- if (!mm_is_user(mm))
+ if (mm_is_kernel(mm))
return;
pte = pte_mknoncont(pte);
@@ -465,7 +457,7 @@ void contpte_set_ptes(struct mm_struct *mm, unsigned long addr,
*/
VM_WARN_ON(nr == 1);
- if (!mm_is_user(mm))
+ if (mm_is_kernel(mm))
return __set_ptes(mm, addr, ptep, pte, nr);
end = addr + (nr << PAGE_SHIFT);
--
2.51.2
^ permalink raw reply related
* [PATCH 19/22] x86/tboot: mark tboot_mm as a kernel mm
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
tboot_mm is a kernel-owned address space used to build the
identity map for tboot shutdown. Mark it with MMF_KERNEL so
page table allocation and initialisation code treats its page tables
as kernel page tables.
Having marked tboot_mm as a kernel mm, we also need to convert
pte_alloc_map() (user PTE) to pte_alloc_kernel() (kernel PTE), and
drop the matching pte_unmap().
Assisted-by: Codex:GPT-5.5
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/x86/kernel/tboot.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/tboot.c b/arch/x86/kernel/tboot.c
index 46b8f1f16676..08c899f88e41 100644
--- a/arch/x86/kernel/tboot.c
+++ b/arch/x86/kernel/tboot.c
@@ -99,6 +99,7 @@ static struct mm_struct tboot_mm = {
.pgd = swapper_pg_dir,
.mm_users = ATOMIC_INIT(2),
.mm_count = ATOMIC_INIT(1),
+ .flags = MM_FLAGS_INIT(MMF_KERNEL_MASK),
.write_protect_seq = SEQCNT_ZERO(tboot_mm.write_protect_seq),
MMAP_LOCK_INITIALIZER(init_mm)
.page_table_lock = __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock),
@@ -129,11 +130,10 @@ static int map_tboot_page(unsigned long vaddr, unsigned long pfn,
pmd = pmd_alloc(&tboot_mm, pud, vaddr);
if (!pmd)
return -1;
- pte = pte_alloc_map(&tboot_mm, pmd, vaddr);
+ pte = pte_alloc_kernel(pmd, vaddr);
if (!pte)
return -1;
set_pte_at(&tboot_mm, vaddr, pte, pfn_pte(pfn, prot));
- pte_unmap(pte);
/*
* PTI poisons low addresses in the kernel page tables in the
--
2.51.2
^ permalink raw reply related
* [PATCH 20/22] arm64: mm: drop ctor/dtor calls for kernel page tables
From: Kevin Brodsky @ 2026-07-14 14:04 UTC (permalink / raw)
To: linux-mm
Cc: Kevin Brodsky, Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Pasha Tatashin, Russell King,
Catalin Marinas, Will Deacon, Ryan Roberts, linux-arm-kernel,
Huacai Chen, loongarch, James E.J. Bottomley, Helge Deller,
linux-parisc, Madhavan Srinivasan, Michael Ellerman, linuxppc-dev,
Paul Walmsley, Palmer Dabbelt, Albert Ou, linux-riscv,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev, Gerald Schaefer,
linux-s390, David S. Miller, Andreas Larsson, sparclinux,
Richard Weinberger, Anton Ivanov, Johannes Berg, linux-um,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Andy Lutomirski, Peter Zijlstra, Ning Sun, x86,
tboot-devel, Ard Biesheuvel, Ilias Apalodimas, linux-efi,
Vishal Moola, Alistair Popple, Matthew Wilcox (Oracle),
linux-kernel, linux-arch
In-Reply-To: <20260714-remove_pgtable_cdtor-v1-0-44be8a7685d7@arm.com>
Thanks to recent changes:
* pagetable_*_ctor() and pagetable_dtor() do nothing for kernel page
tables.
* efi_mm is also treated like a kernel mm by the page table logic,
including ctors.
We can therefore drop the calls to ctors when allocating special
kernel page tables in mmu.c, as well as the dtor call when freeing
hotplug page tables.
Since we no longer need the owning mm or page table level when
allocating them, we can also remove both the init_mm/special_mm
wrappers and the pgtable_level callback argument.
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
---
arch/arm64/mm/mmu.c | 82 +++++++++++++++--------------------------------------
1 file changed, 23 insertions(+), 59 deletions(-)
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index f723bcf68174..f1400c6f68ba 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -103,7 +103,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
}
EXPORT_SYMBOL(phys_mem_access_prot);
-static phys_addr_t __init early_pgtable_alloc(enum pgtable_level pgtable_level)
+static phys_addr_t __init early_pgtable_alloc(void)
{
phys_addr_t phys;
@@ -195,7 +195,7 @@ static bool pte_range_has_valid_noncont(pte_t *ptep)
static int alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
unsigned long end, phys_addr_t phys,
pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
unsigned long next;
@@ -210,7 +210,7 @@ static int alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
if (flags & NO_EXEC_MAPPINGS)
pmdval |= PMD_TABLE_PXN;
BUG_ON(!pgtable_alloc);
- pte_phys = pgtable_alloc(PGTABLE_LEVEL_PTE);
+ pte_phys = pgtable_alloc();
if (pte_phys == INVALID_PHYS_ADDR)
return -ENOMEM;
ptep = pte_set_fixmap(pte_phys);
@@ -251,7 +251,7 @@ static int alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
static int init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end,
phys_addr_t phys, pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level), int flags)
+ phys_addr_t (*pgtable_alloc)(void), int flags)
{
unsigned long next;
@@ -303,7 +303,7 @@ static bool pmd_range_has_valid_noncont(pmd_t *pmdp)
static int alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
unsigned long end, phys_addr_t phys,
pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
int ret;
@@ -322,7 +322,7 @@ static int alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
if (flags & NO_EXEC_MAPPINGS)
pudval |= PUD_TABLE_PXN;
BUG_ON(!pgtable_alloc);
- pmd_phys = pgtable_alloc(PGTABLE_LEVEL_PMD);
+ pmd_phys = pgtable_alloc();
if (pmd_phys == INVALID_PHYS_ADDR)
return -ENOMEM;
pmdp = pmd_set_fixmap(pmd_phys);
@@ -361,7 +361,7 @@ static int alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
static int alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end,
phys_addr_t phys, pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
int ret = 0;
@@ -376,7 +376,7 @@ static int alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end,
if (flags & NO_EXEC_MAPPINGS)
p4dval |= P4D_TABLE_PXN;
BUG_ON(!pgtable_alloc);
- pud_phys = pgtable_alloc(PGTABLE_LEVEL_PUD);
+ pud_phys = pgtable_alloc();
if (pud_phys == INVALID_PHYS_ADDR)
return -ENOMEM;
pudp = pud_set_fixmap(pud_phys);
@@ -428,7 +428,7 @@ static int alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end,
static int alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end,
phys_addr_t phys, pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
int ret;
@@ -443,7 +443,7 @@ static int alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end,
if (flags & NO_EXEC_MAPPINGS)
pgdval |= PGD_TABLE_PXN;
BUG_ON(!pgtable_alloc);
- p4d_phys = pgtable_alloc(PGTABLE_LEVEL_P4D);
+ p4d_phys = pgtable_alloc();
if (p4d_phys == INVALID_PHYS_ADDR)
return -ENOMEM;
p4dp = p4d_set_fixmap(p4d_phys);
@@ -480,7 +480,7 @@ static int alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end,
static int __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys,
unsigned long virt, phys_addr_t size,
pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
int ret;
@@ -513,7 +513,7 @@ static int __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys,
static int __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
unsigned long virt, phys_addr_t size,
pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
int ret;
@@ -529,7 +529,7 @@ static int __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
static void early_create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
unsigned long virt, phys_addr_t size,
pgprot_t prot,
- phys_addr_t (*pgtable_alloc)(enum pgtable_level),
+ phys_addr_t (*pgtable_alloc)(void),
int flags)
{
int ret;
@@ -540,55 +540,20 @@ static void early_create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,
panic("Failed to create page tables\n");
}
-static phys_addr_t __pgd_pgtable_alloc(struct mm_struct *mm, gfp_t gfp,
- enum pgtable_level pgtable_level)
+static phys_addr_t pgd_pgtable_alloc_gfp(gfp_t gfp)
{
/* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */
struct ptdesc *ptdesc = pagetable_alloc(gfp & ~__GFP_ZERO, 0);
- phys_addr_t pa;
if (!ptdesc)
return INVALID_PHYS_ADDR;
- pa = page_to_phys(ptdesc_page(ptdesc));
-
- switch (pgtable_level) {
- case PGTABLE_LEVEL_PTE:
- BUG_ON(!pagetable_pte_ctor(mm, ptdesc));
- break;
- case PGTABLE_LEVEL_PMD:
- BUG_ON(!pagetable_pmd_ctor(mm, ptdesc));
- break;
- case PGTABLE_LEVEL_PUD:
- pagetable_pud_ctor(ptdesc);
- break;
- case PGTABLE_LEVEL_P4D:
- pagetable_p4d_ctor(ptdesc);
- break;
- case PGTABLE_LEVEL_PGD:
- VM_WARN_ON(1);
- break;
- }
-
- return pa;
-}
-
-static phys_addr_t
-pgd_pgtable_alloc_init_mm_gfp(enum pgtable_level pgtable_level, gfp_t gfp)
-{
- return __pgd_pgtable_alloc(&init_mm, gfp, pgtable_level);
-}
-
-static phys_addr_t __maybe_unused
-pgd_pgtable_alloc_init_mm(enum pgtable_level pgtable_level)
-{
- return pgd_pgtable_alloc_init_mm_gfp(pgtable_level, GFP_PGTABLE_KERNEL);
+ return page_to_phys(ptdesc_page(ptdesc));
}
-static phys_addr_t
-pgd_pgtable_alloc_special_mm(enum pgtable_level pgtable_level)
+static phys_addr_t pgd_pgtable_alloc(void)
{
- return __pgd_pgtable_alloc(NULL, GFP_PGTABLE_KERNEL, pgtable_level);
+ return pgd_pgtable_alloc_gfp(GFP_PGTABLE_KERNEL);
}
static void split_contpte(pte_t *ptep)
@@ -609,7 +574,7 @@ static int split_pmd(pmd_t *pmdp, pmd_t pmd, gfp_t gfp, bool to_cont)
pte_t *ptep;
int i;
- pte_phys = pgd_pgtable_alloc_init_mm_gfp(PGTABLE_LEVEL_PTE, gfp);
+ pte_phys = pgd_pgtable_alloc_gfp(gfp);
if (pte_phys == INVALID_PHYS_ADDR)
return -ENOMEM;
ptep = (pte_t *)phys_to_virt(pte_phys);
@@ -656,7 +621,7 @@ static int split_pud(pud_t *pudp, pud_t pud, gfp_t gfp, bool to_cont)
pmd_t *pmdp;
int i;
- pmd_phys = pgd_pgtable_alloc_init_mm_gfp(PGTABLE_LEVEL_PMD, gfp);
+ pmd_phys = pgd_pgtable_alloc_gfp(gfp);
if (pmd_phys == INVALID_PHYS_ADDR)
return -ENOMEM;
pmdp = (pmd_t *)phys_to_virt(pmd_phys);
@@ -1035,7 +1000,7 @@ void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys,
flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
early_create_pgd_mapping(mm->pgd, phys, virt, size, prot,
- pgd_pgtable_alloc_special_mm, flags);
+ pgd_pgtable_alloc, flags);
}
static void update_mapping_prot(phys_addr_t phys, unsigned long virt,
@@ -1249,7 +1214,7 @@ static void __init declare_vma(struct vm_struct *vma,
static phys_addr_t kpti_ng_temp_alloc __initdata;
-static phys_addr_t __init kpti_ng_pgd_alloc(enum pgtable_level pgtable_level)
+static phys_addr_t __init kpti_ng_pgd_alloc(void)
{
kpti_ng_temp_alloc -= PAGE_SIZE;
return kpti_ng_temp_alloc;
@@ -1357,7 +1322,7 @@ static int __init map_entry_trampoline(void)
memset(tramp_pg_dir, 0, PGD_SIZE);
early_create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS,
entry_tramp_text_size(), prot,
- pgd_pgtable_alloc_init_mm, NO_BLOCK_MAPPINGS);
+ pgd_pgtable_alloc, NO_BLOCK_MAPPINGS);
/* Map both the text and data into the kernel page table */
for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++)
@@ -1445,7 +1410,6 @@ static void free_hotplug_page_range(struct page *page, size_t size,
static void free_hotplug_pgtable_page(struct page *page)
{
- pagetable_dtor(page_ptdesc(page));
free_hotplug_page_range(page, PAGE_SIZE, NULL);
}
@@ -1958,7 +1922,7 @@ int arch_add_memory(int nid, u64 start, u64 size,
flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
ret = __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start),
- size, params->pgprot, pgd_pgtable_alloc_init_mm,
+ size, params->pgprot, pgd_pgtable_alloc,
flags);
if (ret)
goto err;
--
2.51.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox