* Re: [PATCH v3] docs: dt: maintainer: Add Devicetree and OF maintainer profile document
From: Frank Li @ 2026-07-20 20:34 UTC (permalink / raw)
To: Rob Herring
Cc: Krzysztof Kozlowski, Krzysztof Kozlowski, Conor Dooley,
Jonathan Corbet, Shuah Khan, devicetree, workflows, linux-doc,
linux-kernel, Saravana Kannan
In-Reply-To: <CAL_JsqKkxVTVE=o-uar=JEVvqw9NhBAQcBrJ0ukf+RBXCwFZJA@mail.gmail.com>
On Mon, Jul 20, 2026 at 03:12:17PM -0500, Rob Herring wrote:
> On Mon, Jul 20, 2026 at 1:40 PM Krzysztof Kozlowski
> <krzysztof.kozlowski@oss.qualcomm.com> wrote:
> >
> > Document how Devicetree and Open Firmware maintainers handle their
> > subsystem, especially focusing on two caveats:
> >
> > Devicetree subsystem handles patches with a minor difference comparing
> > to other subsystems: while DT maintainers pick up OF code, they only
> > provide review of DT bindings without applying these.
> >
> > All three DT bindings maintainers rely currently on Patchwork and due to
> > enormous amount of emails per day, regardless how much DT maintainers
> > try, they cannot read all the emails.
> >
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Conor Dooley <conor+dt@kernel.org>
> > Cc: Saravana Kannan <saravanak@kernel.org>
> > Cc: devicetree@vger.kernel.org
> > Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
> >
> > ---
> >
> > I expect patch to be picked up by Rob, after review.
> >
> > Changes in v3:
> > 1. Add also F: entry
> >
> > Changes in v2:
> > 1. Correct typos and trailing white spaces.
> > 2. Fix order of P: after C: in maintainers.
> > ---
> > .../process/maintainer-devicetree.rst | 70 +++++++++++++++++++
> > MAINTAINERS | 3 +
> > 2 files changed, 73 insertions(+)
> > create mode 100644 Documentation/process/maintainer-devicetree.rst
> >
> > diff --git a/Documentation/process/maintainer-devicetree.rst b/Documentation/process/maintainer-devicetree.rst
> > new file mode 100644
> > index 000000000000..d8ffe752bf5d
> > --- /dev/null
> > +++ b/Documentation/process/maintainer-devicetree.rst
> > @@ -0,0 +1,70 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +======================================
> > +Devicetree and Open Firmware Subsystem
> > +======================================
> > +
> > +Other Process Documents
> > +-----------------------
> > +
> > +Please see the documents in Documentation/devicetree/bindings/ for information
> > +on how to write proper Devicetree bindings and how to submit patches.
> > +
> > +Patch Review and Handling
> > +-------------------------
> > +
> > +Patches handled by Devicetree maintainers are processed differently depending
> > +on the patch type:
> > +
> > +1. Core OF driver code, e.g. drivers/of/:
> > + patches are reviewed and applied by DT maintainers.
> > +
> > +2. Devicetree bindings:
> > + patches are reviewed by DT maintainers but, except in certain cases, should
> > + be applied by subsystem maintainers. See also *For kernel maintainers* in
> > + Documentation/devicetree/bindings/submitting-patches.rst.
>
> I would reword:
>
> patches are reviewed by DT maintainers, but should be applied by
> subsystem maintainers except in certain cases.
It would be wonderful if provide a few sample *certain* cases.
Anyways, nice document.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
>
> > +
> > +3. DTS and drivers:
> > + DT maintainers might provide comments, but review is generally not expected.
>
> For DTS, we expect to pass schema checks or at least not add new warnings.
>
> I can address these 2 when applying.
>
> Rob
^ permalink raw reply
* Re: [PATCH v18 05/13] PCI/AER: Introduce AER-CXL protocol error kfifo
From: Dave Jiang @ 2026-07-20 20:29 UTC (permalink / raw)
To: Terry Bowman, Bjorn Helgaas, Dan Williams, Ira Weiny,
Jonathan Cameron, Len Brown, Rafael J . Wysocki, Robert Richter
Cc: linux-acpi, linux-cxl, linux-doc, linux-kernel, linux-pci,
linuxppc-dev, Alejandro Lucero, Alison Schofield, Ankit Agrawal,
Ard Biesheuvel, Ben Cheatham, Borislav Petkov, Breno Leitao,
Davidlohr Bueso, Fabio M . De Francesco, Gregory Price,
Hanjun Guo, Jonathan Corbet, Kees Cook,
Kuppuswamy Sathyanarayanan, Li Ming, Mahesh J Salgaonkar,
Mauro Carvalho Chehab, Oliver O'Halloran, Shiju Jose,
Shuah Khan, Shuai Xue, Smita Koralahalli, Tony Luck, Vishal Verma
In-Reply-To: <20260717222706.3540281-6-terry.bowman@amd.com>
On 7/17/26 3:26 PM, Terry Bowman wrote:
> CXL VH RAS handling requires a path for the AER driver to hand off
> CXL protocol errors to cxl_core for logging and recovery before PCIe
> AER recovery tears down the device. Add
> drivers/pci/pcie/aer_cxl_vh.c to implement this handoff via a kfifo-backed
> work item.
>
> Introduce is_aer_internal_error() to identify CXL protocol errors
> from AER internal error status bits across both correctable and
> uncorrectable severities.
>
> Introduce is_cxl_error() to gate the VH kfifo path.
>
> Introduce struct cxl_proto_err_work_data to carry the error source
> PCI device and severity through the kfifo. Encapsulate the kfifo,
> per-producer spinlock, registration rwsem, and work pointer in struct
> cxl_proto_err_kfifo. Initialize the embedded kfifo via INIT_KFIFO()
> from a subsys_initcall so its metadata is populated before any
> producer or consumer runs.
>
> Introduce cxl_forward_error() to enqueue a CXL protocol error. A
> reference is taken on the PCI device; the consumer releases it via
> for_each_cxl_proto_err(). On enqueue failure the reference is
> released immediately, the error is dropped, and the consumer is
> scheduled to drain existing entries. A subsequent patch wires
> cxl_forward_error() into handle_error_source() where correctable and
> uncorrectable status clearing is left to pci_aer_handle_error().
>
> Introduce cxl_proto_err_flush() to synchronously wait for the
> consumer worker to drain the kfifo. A subsequent patch wires this
> into handle_error_source() for UCE events so the CXL plane completes
> error handling and panic policy before pci_aer_handle_error() drives
> PCIe recovery.
>
> Introduce cxl_register_proto_err_work() and
> cxl_unregister_proto_err_work() for cxl_core to register and
> deregister its work handler. On unregistration, pending kfifo entries
> are drained and their pdev references released before
> cancel_work_sync() runs. Export these and for_each_cxl_proto_err()
> via EXPORT_SYMBOL_FOR_MODULES restricted to cxl_core.
>
> Protect the work pointer with a rwsem to correctly serialize
> registration, deregistration, enqueue, and dequeue against concurrent
> AER IRQ threads. Serialize concurrent kfifo writers with a spinlock.
>
> Add MAINTAINERS entries for aer_cxl_vh.c and aer_cxl_rch.c under
> the CXL entry so CXL maintainers are CC'd on changes to the AER-CXL
> bridging code.
>
> Co-developed-by: Dan Williams <djbw@kernel.org>
> Signed-off-by: Dan Williams <djbw@kernel.org>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>
> ---
>
> Changes in v17->v18:
> - Remove correctable status clear from cxl_forward_error(); the AER core
> clears all status bits via pci_aer_handle_error() info->status writeback
> - Schedule consumer on kfifo overflow so existing entries can be drained
>
> Changes in v16->v17:
> - Reword "kfifo semaphore" to "kfifo spinlock" to match fifo_lock.
> - Defer the handle_error_source() is_cxl_error() switch to the patch that
> registers the kfifo consumer to keep each commit bisect-safe.
> - Rename rwsema to rwsem
> - Change CPER exports to use EXPORT_SYMBOL_FOR_MODULES.
> - Add work cancel function.
> - Replace kfifo_put() with kfifo_in_spinlocked() for multiple producers
> - Add fifo_lock spinlock for concurrent producer serialisation
> - Initialize the embedded kfifo with INIT_KFIFO() in a subsys_initcall so
> kfifo->mask, ->esize and ->data are set before first use.
> - Clear PCI_ERR_COR_STATUS in cxl_forward_error() after enqueue so the
> device is acked for correctable events even when the consumer drops the
> event. Uncorrectable status is left for cxl_do_recovery() to clear after
> recovery completes, mirroring the AER core convention.
> - WARN on double-registration in cxl_register_proto_err_work() to make an
> unintended second consumer visible at runtime.
> - Add direct rwsem.h, cleanup.h and workqueue.h includes for symbols used
> in aer_cxl_vh.c
> - Add MAINTAINERS entries for drivers/pci/pcie/aer_cxl_*.c
> - Update message
> ---
> MAINTAINERS | 2 +
> drivers/pci/pcie/Makefile | 1 +
> drivers/pci/pcie/aer.c | 10 --
> drivers/pci/pcie/aer_cxl_vh.c | 221 ++++++++++++++++++++++++++++++++++
> drivers/pci/pcie/portdrv.h | 6 +
> include/linux/aer.h | 24 ++++
> 6 files changed, 254 insertions(+), 10 deletions(-)
> create mode 100644 drivers/pci/pcie/aer_cxl_vh.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 806bd2d80d153..39007aa90b20f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -6526,6 +6526,8 @@ S: Maintained
> F: Documentation/driver-api/cxl
> F: Documentation/userspace-api/fwctl/fwctl-cxl.rst
> F: drivers/cxl/
> +F: drivers/pci/pcie/aer_cxl_rch.c
> +F: drivers/pci/pcie/aer_cxl_vh.c
> F: include/cxl/
> F: include/uapi/linux/cxl_mem.h
> F: tools/testing/cxl/
> diff --git a/drivers/pci/pcie/Makefile b/drivers/pci/pcie/Makefile
> index b0b43a18c304b..62d3d3c69a5df 100644
> --- a/drivers/pci/pcie/Makefile
> +++ b/drivers/pci/pcie/Makefile
> @@ -9,6 +9,7 @@ obj-$(CONFIG_PCIEPORTBUS) += pcieportdrv.o bwctrl.o
> obj-y += aspm.o
> obj-$(CONFIG_PCIEAER) += aer.o err.o tlp.o
> obj-$(CONFIG_CXL_RAS) += aer_cxl_rch.o
> +obj-$(CONFIG_CXL_RAS) += aer_cxl_vh.o
> obj-$(CONFIG_PCIEAER_INJECT) += aer_inject.o
> obj-$(CONFIG_PCIE_PME) += pme.o
> obj-$(CONFIG_PCIE_DPC) += dpc.o
> diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
> index c4fd9c0b2a548..c5bce25df51cb 100644
> --- a/drivers/pci/pcie/aer.c
> +++ b/drivers/pci/pcie/aer.c
> @@ -1150,16 +1150,6 @@ void pci_aer_unmask_internal_errors(struct pci_dev *dev)
> */
> EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core");
>
> -#ifdef CONFIG_CXL_RAS
> -bool is_aer_internal_error(struct aer_err_info *info)
> -{
> - if (info->severity == AER_CORRECTABLE)
> - return info->status & PCI_ERR_COR_INTERNAL;
> -
> - return info->status & PCI_ERR_UNC_INTN;
> -}
> -#endif
> -
> /**
> * pci_aer_handle_error - handle logging error into an event log
> * @dev: pointer to pci_dev data structure of error source device
> diff --git a/drivers/pci/pcie/aer_cxl_vh.c b/drivers/pci/pcie/aer_cxl_vh.c
> new file mode 100644
> index 0000000000000..93bed07936100
> --- /dev/null
> +++ b/drivers/pci/pcie/aer_cxl_vh.c
> @@ -0,0 +1,221 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright(c) 2026 AMD Corporation. All rights reserved. */
> +
> +#include <linux/aer.h>
> +#include <linux/atomic.h>
> +#include <linux/cleanup.h>
> +#include <linux/init.h>
> +#include <linux/kfifo.h>
> +#include <linux/rwsem.h>
> +#include <linux/wait_bit.h>
> +#include <linux/workqueue.h>
> +#include "../pci.h"
> +#include "portdrv.h"
> +
> +#define CXL_ERROR_SOURCES_MAX 128
> +
> +struct cxl_proto_err_kfifo {
> + struct work_struct *work;
> + void (*flush)(void);
> + struct rw_semaphore rwsem;
> + spinlock_t fifo_lock;
> + atomic_t flush_inflight;
> + DECLARE_KFIFO(fifo, struct cxl_proto_err_work_data,
> + CXL_ERROR_SOURCES_MAX);
> +};
> +
> +static struct cxl_proto_err_kfifo cxl_proto_err_kfifo = {
> + .rwsem = __RWSEM_INITIALIZER(cxl_proto_err_kfifo.rwsem),
> + .fifo_lock = __SPIN_LOCK_UNLOCKED(cxl_proto_err_kfifo.fifo_lock),
> +};
> +
> +static int __init cxl_proto_err_kfifo_init(void)
> +{
> + INIT_KFIFO(cxl_proto_err_kfifo.fifo);
> + return 0;
> +}
> +subsys_initcall(cxl_proto_err_kfifo_init);
> +
> +bool is_aer_internal_error(struct aer_err_info *info)
> +{
> + if (info->severity == AER_CORRECTABLE)
> + return info->status & PCI_ERR_COR_INTERNAL;
> +
> + return info->status & PCI_ERR_UNC_INTN;
> +}
> +
> +bool is_cxl_error(struct pci_dev *pdev, struct aer_err_info *info)
> +{
> + if (!info || !info->is_cxl)
> + return false;
> +
> + if (pci_pcie_type(pdev) != PCI_EXP_TYPE_ENDPOINT)
> + return false;
> +
> + return is_aer_internal_error(info);
> +}
> +
> +/**
> + * cxl_forward_error - Forward a CXL protocol error to the CXL subsystem via kfifo
> + * @pdev: PCI device that reported the AER error
> + * @info: AER error info containing severity and status
> + *
> + * Producer side of the AER-CXL kfifo. Enqueues a CXL protocol error work
> + * item and schedules the consumer workqueue. Takes a reference on @pdev
> + * that the consumer releases after handling.
> + *
> + * Return: true if the caller must flush the kfifo before AER recovery,
> + * false if no CXL error handling was initiated due to early return on
> + * error.
> + */
> +bool cxl_forward_error(struct pci_dev *pdev, struct aer_err_info *info)
> +{
> + struct cxl_proto_err_work_data wd = {
> + .severity = info->severity,
> + .pdev = pdev,
> + };
> +
> + guard(rwsem_read)(&cxl_proto_err_kfifo.rwsem);
> +
> + if (!cxl_proto_err_kfifo.work) {
> + dev_err_ratelimited(&pdev->dev, "AER-CXL kfifo reader not registered\n");
> + return false;
> + }
> +
> + /*
> + * Reference discipline: the AER caller (handle_error_source())
> + * holds a ref on @pdev for the duration of this call and releases
> + * it on return. Take a fresh ref here so the pdev stays live while
> + * queued in the kfifo; the consumer (for_each_cxl_proto_err())
> + * drops that ref after handling. On enqueue failure below, drop
> + * the ref we just took to avoid a leak.
> + */
> + pci_dev_get(pdev);
> +
> + /* Serialize concurrent kfifo writers: multiple AER threaded IRQs */
> + if (!kfifo_in_spinlocked(&cxl_proto_err_kfifo.fifo, &wd, 1,
> + &cxl_proto_err_kfifo.fifo_lock)) {
> + /* Dropped; no panic - UCE unconfirmed without RAS read */
> + dev_err_ratelimited(&pdev->dev, "AER-CXL kfifo add failed\n");
> + pci_dev_put(pdev);
> + schedule_work(cxl_proto_err_kfifo.work);
> + return true;
> + }
> +
> + schedule_work(cxl_proto_err_kfifo.work);
> + return true;
> +}
> +
> +void cxl_register_proto_err_work(struct work_struct *work,
> + void (*flush)(void))
> +{
> + guard(rwsem_write)(&cxl_proto_err_kfifo.rwsem);
> +
> + /*
> + * Warn on double-registration to surface driver bugs (e.g. missing
> + * cxl_unregister_proto_err_work() on module exit)
> + */
> + if (WARN(cxl_proto_err_kfifo.work,
> + "AER-CXL kfifo consumer already registered\n"))
> + return;
> + cxl_proto_err_kfifo.work = work;
> + cxl_proto_err_kfifo.flush = flush;
> +}
> +EXPORT_SYMBOL_FOR_MODULES(cxl_register_proto_err_work, "cxl_core");
> +
> +static struct work_struct *cancel_cxl_proto_err(void)
> +{
> + struct work_struct *work;
> + struct cxl_proto_err_work_data wd;
> +
> + guard(rwsem_write)(&cxl_proto_err_kfifo.rwsem);
> + work = cxl_proto_err_kfifo.work;
> + cxl_proto_err_kfifo.work = NULL;
> + cxl_proto_err_kfifo.flush = NULL;
> +
> + /* rwsem_write excludes all producers; fifo_lock not needed */
> + while (kfifo_get(&cxl_proto_err_kfifo.fifo, &wd)) {
> + dev_err_ratelimited(&wd.pdev->dev,
> + "AER-CXL error report canceled\n");
> + pci_dev_put(wd.pdev);
> + }
> + return work;
> +}
> +
> +void cxl_unregister_proto_err_work(void)
> +{
> + struct work_struct *work;
> +
> + lockdep_assert_not_held(&cxl_proto_err_kfifo.rwsem);
> +
> + work = cancel_cxl_proto_err();
> +
> + /* Wait for any in-flight cxl_proto_err_flush() calls to complete */
> + wait_var_event(&cxl_proto_err_kfifo.flush_inflight,
> + atomic_read(&cxl_proto_err_kfifo.flush_inflight) == 0);
> +
> + if (work)
> + cancel_work_sync(work);
> +}
> +EXPORT_SYMBOL_FOR_MODULES(cxl_unregister_proto_err_work, "cxl_core");
> +
> +/**
> + * for_each_cxl_proto_err - Call a function for each kfifo work item
> + *
> + * Single-consumer invariant: this function is only called from
> + * cxl_proto_err_work_fn() via a single DECLARE_WORK.
> + *
> + * Holds rwsem_read internally; fn() must not call cxl_register_proto_err_work()
> + * or cxl_unregister_proto_err_work().
> + */
> +void for_each_cxl_proto_err(struct cxl_proto_err_work_data *wd,
> + cxl_proto_err_fn_t fn)
> +{
> + guard(rwsem_read)(&cxl_proto_err_kfifo.rwsem);
> + while (kfifo_get(&cxl_proto_err_kfifo.fifo, wd)) {
> + fn(wd);
> + pci_dev_put(wd->pdev);
> + }
> +}
> +EXPORT_SYMBOL_FOR_MODULES(for_each_cxl_proto_err, "cxl_core");
> +
> +/**
> + * cxl_proto_err_flush - drain pending AER-CXL kfifo work synchronously
> + *
> + * Wait for the consumer worker to finish processing all entries
> + * currently in the kfifo. Used by handle_error_source() for UCE so
> + * the CXL plane can read CXL RAS, apply panic policy, and clear CXL
> + * state before pci_aer_handle_error() drives PCIe recovery.
> + *
> + * Snapshots the flush callback under rwsem_read and releases the rwsem
> + * before calling it. This avoids holding rwsem_read across flush_work(),
> + * which would deadlock via the rwsem HANDOFF mechanism when a concurrent
> + * rwsem_write waiter (cxl_unregister_proto_err_work) blocks new readers
> + * including the worker's for_each_cxl_proto_err() rwsem_read acquisition.
> + *
> + * The flush_inflight counter prevents cxl_core module unload while a
> + * flush is in progress outside the rwsem. The counter is incremented
> + * under rwsem_read (mutually exclusive with the rwsem_write in
> + * cancel_cxl_proto_err() that NULLs the flush pointer) and decremented
> + * after the flush completes. cxl_unregister_proto_err_work() waits for
> + * the counter to reach zero before proceeding with cancel_work_sync().
> + *
> + * For correctable events the consumer can run asynchronously; AER
> + * does not need to call this helper for AER_CORRECTABLE.
> + */
> +void cxl_proto_err_flush(void)
> +{
> + void (*flush)(void);
> +
> + scoped_guard(rwsem_read, &cxl_proto_err_kfifo.rwsem) {
> + flush = cxl_proto_err_kfifo.flush;
> + if (flush)
> + atomic_inc(&cxl_proto_err_kfifo.flush_inflight);
> + }
> +
> + if (flush) {
> + flush();
> + if (atomic_dec_and_test(&cxl_proto_err_kfifo.flush_inflight))
> + wake_up_var(&cxl_proto_err_kfifo.flush_inflight);
> + }
> +}
> diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h
> index cc58bf2f2c844..fd203010877bf 100644
> --- a/drivers/pci/pcie/portdrv.h
> +++ b/drivers/pci/pcie/portdrv.h
> @@ -130,9 +130,15 @@ struct aer_err_info;
> bool is_aer_internal_error(struct aer_err_info *info);
> void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info);
> void cxl_rch_enable_rcec(struct pci_dev *rcec);
> +bool is_cxl_error(struct pci_dev *pdev, struct aer_err_info *info);
> +bool cxl_forward_error(struct pci_dev *pdev, struct aer_err_info *info);
> +void cxl_proto_err_flush(void);
> #else
> static inline bool is_aer_internal_error(struct aer_err_info *info) { return false; }
> static inline void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info) { }
> static inline void cxl_rch_enable_rcec(struct pci_dev *rcec) { }
> +static inline bool is_cxl_error(struct pci_dev *pdev, struct aer_err_info *info) { return false; }
> +static inline bool cxl_forward_error(struct pci_dev *pdev, struct aer_err_info *info) { return false; }
> +static inline void cxl_proto_err_flush(void) { }
> #endif /* CONFIG_CXL_RAS */
> #endif /* _PORTDRV_H_ */
> diff --git a/include/linux/aer.h b/include/linux/aer.h
> index df0f5c382286f..8eba3192e2d15 100644
> --- a/include/linux/aer.h
> +++ b/include/linux/aer.h
> @@ -25,6 +25,7 @@
> #define PCIE_STD_MAX_TLP_HEADERLOG (PCIE_STD_NUM_TLP_HEADERLOG + 10)
>
> struct pci_dev;
> +struct work_struct;
>
> struct pcie_tlp_log {
> union {
> @@ -66,6 +67,29 @@ static inline int pcie_aer_is_native(struct pci_dev *dev) { return 0; }
> static inline void pci_aer_unmask_internal_errors(struct pci_dev *dev) { }
> #endif
>
> +#ifdef CONFIG_CXL_RAS
> +/**
> + * struct cxl_proto_err_work_data - Error information used in CXL error handling
> + * @pdev: PCI device detecting the error
> + * @severity: AER severity
> + */
> +struct cxl_proto_err_work_data {
> + struct pci_dev *pdev;
> + int severity;
> +};
> +
> +/**
> + * Callback for processing a CXL protocol error from the AER-CXL kfifo.
> + */
> +typedef void (*cxl_proto_err_fn_t)(struct cxl_proto_err_work_data *wd);
> +
> +void cxl_register_proto_err_work(struct work_struct *work,
> + void (*flush)(void));
> +void for_each_cxl_proto_err(struct cxl_proto_err_work_data *wd,
> + cxl_proto_err_fn_t fn);
> +void cxl_unregister_proto_err_work(void);
> +#endif
> +
> void pci_print_aer(struct pci_dev *dev, int aer_severity,
> struct aer_capability_regs *aer);
> int cper_severity_to_aer(int cper_severity);
^ permalink raw reply
* Re: [PATCH v18 04/13] cxl: Rename find_cxl_port() to find_cxl_port_by_dport()
From: Dave Jiang @ 2026-07-20 20:25 UTC (permalink / raw)
To: Terry Bowman, Bjorn Helgaas, Dan Williams, Ira Weiny,
Jonathan Cameron, Len Brown, Rafael J . Wysocki, Robert Richter
Cc: linux-acpi, linux-cxl, linux-doc, linux-kernel, linux-pci,
linuxppc-dev, Alejandro Lucero, Alison Schofield, Ankit Agrawal,
Ard Biesheuvel, Ben Cheatham, Borislav Petkov, Breno Leitao,
Davidlohr Bueso, Fabio M . De Francesco, Gregory Price,
Hanjun Guo, Jonathan Corbet, Kees Cook,
Kuppuswamy Sathyanarayanan, Li Ming, Mahesh J Salgaonkar,
Mauro Carvalho Chehab, Oliver O'Halloran, Shiju Jose,
Shuah Khan, Shuai Xue, Smita Koralahalli, Tony Luck, Vishal Verma
In-Reply-To: <20260717222706.3540281-5-terry.bowman@amd.com>
On 7/17/26 3:26 PM, Terry Bowman wrote:
> From: Dan Williams <djbw@kernel.org>
>
> find_cxl_port() and find_cxl_port_by_uport() are internal port lookup
> functions that search the CXL bus by dport and uport respectively, but
> their names do not make the lookup method clear.
>
> Rename find_cxl_port() to find_cxl_port_by_dport() to make the lookup
> method explicit and consistent with find_cxl_port_by_uport(). Both
> functions remain static to port.c; the upcoming patch that adds the
> first cross-file caller will widen their scope.
>
> Co-developed-by: Terry Bowman <terry.bowman@amd.com>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Signed-off-by: Dan Williams <djbw@kernel.org>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>
> ---
>
> Changes in v17 -> v18:
> - New commit
> ---
> drivers/cxl/core/port.c | 20 ++++++++++++++------
> 1 file changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> index f90f899c31d07..cadb51f70f854 100644
> --- a/drivers/cxl/core/port.c
> +++ b/drivers/cxl/core/port.c
> @@ -1379,7 +1379,7 @@ static int match_port_by_dport(struct device *dev, const void *data)
> return dport != NULL;
> }
>
> -static struct cxl_port *__find_cxl_port(struct cxl_find_port_ctx *ctx)
> +static struct cxl_port *__find_cxl_port_by_dport(struct cxl_find_port_ctx *ctx)
> {
> struct device *dev;
>
> @@ -1392,8 +1392,16 @@ static struct cxl_port *__find_cxl_port(struct cxl_find_port_ctx *ctx)
> return NULL;
> }
>
> -static struct cxl_port *find_cxl_port(struct device *dport_dev,
> - struct cxl_dport **dport)
> +/**
> + * find_cxl_port_by_dport - find a cxl_port by one of its targets
> + * @dport_dev: device representing the dport target
> + * @dport: optional output of the 'struct cxl_dport' companion of the @dport_dev
> + *
> + * Return a 'struct cxl_port' with an elevated reference if found. Use
> + * __free(put_cxl_port) to release.
> + */
> +static struct cxl_port *find_cxl_port_by_dport(struct device *dport_dev,
> + struct cxl_dport **dport)
> {
> struct cxl_find_port_ctx ctx = {
> .dport_dev = dport_dev,
> @@ -1401,7 +1409,7 @@ static struct cxl_port *find_cxl_port(struct device *dport_dev,
> };
> struct cxl_port *port;
>
> - port = __find_cxl_port(&ctx);
> + port = __find_cxl_port_by_dport(&ctx);
> return port;
> }
>
> @@ -1895,14 +1903,14 @@ EXPORT_SYMBOL_NS_GPL(devm_cxl_enumerate_ports, "CXL");
> struct cxl_port *cxl_pci_find_port(struct pci_dev *pdev,
> struct cxl_dport **dport)
> {
> - return find_cxl_port(pdev->dev.parent, dport);
> + return find_cxl_port_by_dport(pdev->dev.parent, dport);
> }
> EXPORT_SYMBOL_NS_GPL(cxl_pci_find_port, "CXL");
>
> struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd,
> struct cxl_dport **dport)
> {
> - return find_cxl_port(grandparent(&cxlmd->dev), dport);
> + return find_cxl_port_by_dport(grandparent(&cxlmd->dev), dport);
> }
> EXPORT_SYMBOL_NS_GPL(cxl_mem_find_port, "CXL");
>
^ permalink raw reply
* Re: [PATCH v3] docs: dt: maintainer: Add Devicetree and OF maintainer profile document
From: Jonathan Corbet @ 2026-07-20 20:20 UTC (permalink / raw)
To: Krzysztof Kozlowski, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Shuah Khan, devicetree, workflows, linux-doc,
linux-kernel
Cc: Saravana Kannan
In-Reply-To: <3e797ffc-8cf6-437c-81f2-9f82d946d042@oss.qualcomm.com>
Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com> writes:
> On 20/07/2026 21:19, Jonathan Corbet wrote:
>>> I expect patch to be picked up by Rob, after review.
>>>
>>> Changes in v3:
>>> 1. Add also F: entry
>>>
>>> Changes in v2:
>>> 1. Correct typos and trailing white spaces.
>>> 2. Fix order of P: after C: in maintainers.
>>> ---
>>> .../process/maintainer-devicetree.rst | 70 +++++++++++++++++++
>>> MAINTAINERS | 3 +
>>> 2 files changed, 73 insertions(+)
>>> create mode 100644 Documentation/process/maintainer-devicetree.rst
>>
>> You'll need to add this new file to index.rst as well.
>
> Hm, where? In docs/process/index.rst? None of the other maintainer
> profiles are mentioned there.
Sigh...I stand corrected. I'd forgotten that we merged the
maintainers-profile-toc Sphinx magic that picks them all up from the
MAINTAINERS file entries.
Sorry for the noise,
jon
^ permalink raw reply
* Re: [PATCH v18 03/13] cxl: Tighten CPER kfifo registration API and symbol visibility
From: Dave Jiang @ 2026-07-20 20:15 UTC (permalink / raw)
To: Terry Bowman, Bjorn Helgaas, Dan Williams, Ira Weiny,
Jonathan Cameron, Len Brown, Rafael J . Wysocki, Robert Richter
Cc: linux-acpi, linux-cxl, linux-doc, linux-kernel, linux-pci,
linuxppc-dev, Alejandro Lucero, Alison Schofield, Ankit Agrawal,
Ard Biesheuvel, Ben Cheatham, Borislav Petkov, Breno Leitao,
Davidlohr Bueso, Fabio M . De Francesco, Gregory Price,
Hanjun Guo, Jonathan Corbet, Kees Cook,
Kuppuswamy Sathyanarayanan, Li Ming, Mahesh J Salgaonkar,
Mauro Carvalho Chehab, Oliver O'Halloran, Shiju Jose,
Shuah Khan, Shuai Xue, Smita Koralahalli, Tony Luck, Vishal Verma
In-Reply-To: <20260717222706.3540281-4-terry.bowman@amd.com>
On 7/17/26 3:26 PM, Terry Bowman wrote:
> From: Dan Williams <djbw@kernel.org>
>
> Tighten the CPER protocol error kfifo registration API and symbol
> visibility.
>
> Use EXPORT_SYMBOL_FOR_MODULES() instead of EXPORT_SYMBOL_NS_GPL() for
> the CPER kfifo registration symbols. This names the consuming module
> explicitly and gives compile-time enforcement.
>
> Drop the work_struct argument from the unregister path. Change the
> WARN_ONCE condition to a NULL check since there is no caller pointer
> to compare against anymore.
>
> Change register/unregister return types to void. Flag double registration
> with WARN_ONCE() inside the lock instead of returning an error.
>
> Change cxl_ras_init() to void because there is one consumer and one producer
> so the error return was unnecessary. Remove the now-dead error check in
> cxl_core_init().
>
> Add a diagnostic log when the driver is not bound in
> cxl_cper_handle_prot_err().
>
> Co-developed-by: Terry Bowman <terry.bowman@amd.com>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Signed-off-by: Dan Williams <djbw@kernel.org>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
>
> ---
>
> Changes in v17 -> v18:
> - New patch.
> ---
> drivers/acpi/apei/ghes.c | 32 +++++++++++++++-----------------
> drivers/cxl/core/core.h | 7 ++-----
> drivers/cxl/core/port.c | 6 +-----
> drivers/cxl/core/ras.c | 12 +++++++-----
> include/cxl/event.h | 17 ++++++-----------
> 5 files changed, 31 insertions(+), 43 deletions(-)
>
> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
> index ca7a138c1ff2e..187f54e31c33e 100644
> --- a/drivers/acpi/apei/ghes.c
> +++ b/drivers/acpi/apei/ghes.c
> @@ -778,41 +778,41 @@ static void cxl_cper_post_prot_err(struct cxl_cper_sec_prot_err *prot_err,
> #endif
> }
>
> -int cxl_cper_register_prot_err_work(struct work_struct *work)
> +void cxl_cper_register_prot_err_work(struct work_struct *work)
> {
> guard(raw_spinlock_irqsave)(&cxl_cper_prot_err_work_lock);
>
> if (WARN_ONCE(cxl_cper_prot_err_work,
> "CPER-CXL kfifo consumer already registered\n"))
> - return -EINVAL;
> + return;
> cxl_cper_prot_err_work = work;
> - return 0;
> }
> -EXPORT_SYMBOL_NS_GPL(cxl_cper_register_prot_err_work, "CXL");
> +EXPORT_SYMBOL_FOR_MODULES(cxl_cper_register_prot_err_work, "cxl_core");
>
> -int cxl_cper_unregister_prot_err_work(struct work_struct *work)
> +void cxl_cper_unregister_prot_err_work(void)
> {
> + struct work_struct *old;
> +
> scoped_guard(raw_spinlock_irqsave, &cxl_cper_prot_err_work_lock) {
> - if (WARN_ONCE(cxl_cper_prot_err_work != work,
> - "CPER-CXL kfifo consumer mismatch on unregister\n"))
> - return -EINVAL;
> + WARN_ONCE(!cxl_cper_prot_err_work,
> + "CPER-CXL kfifo consumer not registered on unregister\n");
> + old = cxl_cper_prot_err_work;
> cxl_cper_prot_err_work = NULL;
> }
>
> - cancel_work_sync(work);
> + if (old)
> + cancel_work_sync(old);
>
> /* Discard stale entries so they are not replayed on next module load */
> kfifo_reset(&cxl_cper_prot_err_fifo);
> -
> - return 0;
> }
> -EXPORT_SYMBOL_NS_GPL(cxl_cper_unregister_prot_err_work, "CXL");
> +EXPORT_SYMBOL_FOR_MODULES(cxl_cper_unregister_prot_err_work, "cxl_core");
>
> int cxl_cper_prot_err_kfifo_get(struct cxl_cper_prot_err_work_data *wd)
> {
> return kfifo_get(&cxl_cper_prot_err_fifo, wd);
> }
> -EXPORT_SYMBOL_NS_GPL(cxl_cper_prot_err_kfifo_get, "CXL");
> +EXPORT_SYMBOL_FOR_MODULES(cxl_cper_prot_err_kfifo_get, "cxl_core");
>
> /* Room for 8 entries for each of the 4 event log queues */
> #define CXL_CPER_FIFO_DEPTH 32
> @@ -867,12 +867,12 @@ int cxl_cper_register_work(struct work_struct *work)
> }
> EXPORT_SYMBOL_NS_GPL(cxl_cper_register_work, "CXL");
>
> -int cxl_cper_unregister_work(struct work_struct *work)
> +void cxl_cper_unregister_work(struct work_struct *work)
> {
> scoped_guard(raw_spinlock_irqsave, &cxl_cper_work_lock) {
> if (WARN_ONCE(cxl_cper_work != work,
> "CXL CPER kfifo consumer mismatch on unregister\n"))
> - return -EINVAL;
> + return;
> cxl_cper_work = NULL;
> }
>
> @@ -880,8 +880,6 @@ int cxl_cper_unregister_work(struct work_struct *work)
>
> /* Discard stale entries so they are not replayed on next module load */
> kfifo_reset(&cxl_cper_fifo);
> -
> - return 0;
> }
> EXPORT_SYMBOL_NS_GPL(cxl_cper_unregister_work, "CXL");
>
> diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
> index 07555ae638594..23fe40ddf4c6b 100644
> --- a/drivers/cxl/core/core.h
> +++ b/drivers/cxl/core/core.h
> @@ -183,7 +183,7 @@ static inline struct device *dport_to_host(struct cxl_dport *dport)
> return &port->dev;
> }
> #ifdef CONFIG_CXL_RAS
> -int cxl_ras_init(void);
> +void cxl_ras_init(void);
> void cxl_ras_exit(void);
> bool cxl_handle_ras(struct device *dev, void __iomem *ras_base);
> void cxl_handle_cor_ras(struct device *dev, void __iomem *ras_base);
> @@ -192,10 +192,7 @@ void cxl_disable_rch_root_ints(struct cxl_dport *dport);
> void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds);
> void devm_cxl_dport_ras_setup(struct cxl_dport *dport);
> #else
> -static inline int cxl_ras_init(void)
> -{
> - return 0;
> -}
> +static inline void cxl_ras_init(void) { }
> static inline void cxl_ras_exit(void) { }
> static inline bool cxl_handle_ras(struct device *dev, void __iomem *ras_base)
> {
> diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> index 1215ee4f40351..f90f899c31d07 100644
> --- a/drivers/cxl/core/port.c
> +++ b/drivers/cxl/core/port.c
> @@ -2531,14 +2531,10 @@ static __init int cxl_core_init(void)
> if (rc)
> goto err_region;
>
> - rc = cxl_ras_init();
> - if (rc)
> - goto err_ras;
> + cxl_ras_init();
>
> return 0;
>
> -err_ras:
> - cxl_region_exit();
> err_region:
> bus_unregister(&cxl_bus_type);
> err_bus:
> diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c
> index 99fb00949c2fa..135f1997e6f4f 100644
> --- a/drivers/cxl/core/ras.c
> +++ b/drivers/cxl/core/ras.c
> @@ -104,8 +104,11 @@ void cxl_cper_handle_prot_err(struct cxl_cper_prot_err_work_data *data)
> }
>
> guard(device)(&pdev->dev);
> - if (!pdev->dev.driver)
> + if (!pdev->dev.driver) {
> + dev_warn_ratelimited(&pdev->dev,
> + "Device is unbound, abort CPER error handling\n");
> return;
> + }
>
> struct device *mem_dev __free(put_device) = bus_find_device(
> &cxl_bus_type, NULL, pdev, match_memdev_by_parent);
> @@ -129,15 +132,14 @@ static void cxl_cper_prot_err_work_fn(struct work_struct *work)
> }
> static DECLARE_WORK(cxl_cper_prot_err_work, cxl_cper_prot_err_work_fn);
>
> -int cxl_ras_init(void)
> +void cxl_ras_init(void)
> {
> - return cxl_cper_register_prot_err_work(&cxl_cper_prot_err_work);
> + cxl_cper_register_prot_err_work(&cxl_cper_prot_err_work);
> }
>
> void cxl_ras_exit(void)
> {
> - cxl_cper_unregister_prot_err_work(&cxl_cper_prot_err_work);
> - cancel_work_sync(&cxl_cper_prot_err_work);
> + cxl_cper_unregister_prot_err_work();
> }
>
> static void cxl_dport_map_ras(struct cxl_dport *dport)
> diff --git a/include/cxl/event.h b/include/cxl/event.h
> index ff97fea718d2c..3471d4f75c025 100644
> --- a/include/cxl/event.h
> +++ b/include/cxl/event.h
> @@ -287,10 +287,10 @@ struct cxl_cper_prot_err_work_data {
>
> #ifdef CONFIG_ACPI_APEI_GHES
> int cxl_cper_register_work(struct work_struct *work);
> -int cxl_cper_unregister_work(struct work_struct *work);
> +void cxl_cper_unregister_work(struct work_struct *work);
> int cxl_cper_kfifo_get(struct cxl_cper_work_data *wd);
> -int cxl_cper_register_prot_err_work(struct work_struct *work);
> -int cxl_cper_unregister_prot_err_work(struct work_struct *work);
> +void cxl_cper_register_prot_err_work(struct work_struct *work);
> +void cxl_cper_unregister_prot_err_work(void);
> int cxl_cper_prot_err_kfifo_get(struct cxl_cper_prot_err_work_data *wd);
> #else
> static inline int cxl_cper_register_work(struct work_struct *work)
> @@ -298,21 +298,16 @@ static inline int cxl_cper_register_work(struct work_struct *work)
> return 0;
> }
>
> -static inline int cxl_cper_unregister_work(struct work_struct *work)
> -{
> - return 0;
> -}
> +static inline void cxl_cper_unregister_work(struct work_struct *work) { }
> static inline int cxl_cper_kfifo_get(struct cxl_cper_work_data *wd)
> {
> return 0;
> }
> -static inline int cxl_cper_register_prot_err_work(struct work_struct *work)
> +static inline void cxl_cper_register_prot_err_work(struct work_struct *work)
> {
> - return 0;
> }
> -static inline int cxl_cper_unregister_prot_err_work(struct work_struct *work)
> +static inline void cxl_cper_unregister_prot_err_work(void)
> {
> - return 0;
> }
> static inline int cxl_cper_prot_err_kfifo_get(struct cxl_cper_prot_err_work_data *wd)
> {
^ permalink raw reply
* Re: [PATCH v18 02/13] acpi/apei/ghes: Use raw_spinlock_t for CXL CPER work locks
From: Dave Jiang @ 2026-07-20 20:12 UTC (permalink / raw)
To: Terry Bowman, Bjorn Helgaas, Dan Williams, Ira Weiny,
Jonathan Cameron, Len Brown, Rafael J . Wysocki, Robert Richter
Cc: linux-acpi, linux-cxl, linux-doc, linux-kernel, linux-pci,
linuxppc-dev, Alejandro Lucero, Alison Schofield, Ankit Agrawal,
Ard Biesheuvel, Ben Cheatham, Borislav Petkov, Breno Leitao,
Davidlohr Bueso, Fabio M . De Francesco, Gregory Price,
Hanjun Guo, Jonathan Corbet, Kees Cook,
Kuppuswamy Sathyanarayanan, Li Ming, Mahesh J Salgaonkar,
Mauro Carvalho Chehab, Oliver O'Halloran, Shiju Jose,
Shuah Khan, Shuai Xue, Smita Koralahalli, Tony Luck, Vishal Verma
In-Reply-To: <20260717222706.3540281-3-terry.bowman@amd.com>
On 7/17/26 3:26 PM, Terry Bowman wrote:
> The CXL CPER work registration and unregistration helpers in
> drivers/acpi/apei/ghes.c acquire cxl_cper_work_lock and
> cxl_cper_prot_err_work_lock with guard(spinlock), which leaves local
> interrupts enabled. The corresponding post paths
> (cxl_cper_post_event(), cxl_cper_post_prot_err()) execute in hard IRQ
> context (they are called from the GHES error notification path) and
> acquire the same locks via guard(spinlock_irqsave).
>
> If a CPU is holding one of these locks via guard(spinlock) when a
> GHES interrupt arrives on the same CPU, the IRQ handler spins on the
> held lock waiting for it to release, while the lock holder is
> preempted by the IRQ. The result is a deadlock.
>
> Convert both locks from spinlock_t to raw_spinlock_t and use
> guard(raw_spinlock_irqsave) at all call sites. On PREEMPT_RT kernels
> spinlock_t is backed by rt_mutex and sleeping from hard IRQ context is
> not permitted; raw_spinlock_t is safe in both contexts.
>
> Add WARN_ONCE to both register functions to surface double-registration
> bugs at runtime.
>
> Restructure both unregister functions to clear the global work pointer
> under the lock before calling cancel_work_sync(), closing the window
> where a CPER interrupt could schedule work on a pointer about to be
> freed. Add kfifo_reset() after cancel_work_sync() so stale entries
> are not replayed on next module load.
>
> Both kfifos are single-consumer: only one work_struct is registered at
> a time, enforced by the WARN_ONCE guard in the register functions.
> kfifo_reset() is safe outside the lock because cancel_work_sync() has
> already quiesced the consumer, and no new consumer can register until
> the current module exit completes and a fresh module init runs.
>
> Remove the now-redundant cancel_work_sync() call from
> cxl_pci_driver_exit() - cxl_cper_unregister_work() handles quiescing
> internally.
>
> Reported-by: Sashiko <sashiko@linuxfoundation.org>
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
> Fixes: 5e4a264bf8b5 ("acpi/ghes: Process CXL Component Events")
> Fixes: 36f257e3b0ba ("acpi/ghes, cxl/pci: Process CXL CPER Protocol Errors")
> Cc: stable@vger.kernel.org
With the minor sashiko issue addressed,
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
This can probably be picked up ahead of the series.
>
> ---
>
> Changes in v17 -> v18:
> - New patch.
> ---
> drivers/acpi/apei/ghes.c | 50 ++++++++++++++++++++++++++--------------
> drivers/cxl/pci.c | 1 -
> 2 files changed, 33 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
> index 3236a3ce79d6b..ca7a138c1ff2e 100644
> --- a/drivers/acpi/apei/ghes.c
> +++ b/drivers/acpi/apei/ghes.c
> @@ -749,7 +749,7 @@ static DEFINE_KFIFO(cxl_cper_prot_err_fifo, struct cxl_cper_prot_err_work_data,
> CXL_CPER_PROT_ERR_FIFO_DEPTH);
>
> /* Synchronize schedule_work() with cxl_cper_prot_err_work changes */
> -static DEFINE_SPINLOCK(cxl_cper_prot_err_work_lock);
> +static DEFINE_RAW_SPINLOCK(cxl_cper_prot_err_work_lock);
> struct work_struct *cxl_cper_prot_err_work;
>
> static void cxl_cper_post_prot_err(struct cxl_cper_sec_prot_err *prot_err,
> @@ -761,7 +761,7 @@ static void cxl_cper_post_prot_err(struct cxl_cper_sec_prot_err *prot_err,
> if (cxl_cper_sec_prot_err_valid(prot_err))
> return;
>
> - guard(spinlock_irqsave)(&cxl_cper_prot_err_work_lock);
> + guard(raw_spinlock_irqsave)(&cxl_cper_prot_err_work_lock);
>
> if (!cxl_cper_prot_err_work)
> return;
> @@ -780,10 +780,11 @@ static void cxl_cper_post_prot_err(struct cxl_cper_sec_prot_err *prot_err,
>
> int cxl_cper_register_prot_err_work(struct work_struct *work)
> {
> - if (cxl_cper_prot_err_work)
> - return -EINVAL;
> + guard(raw_spinlock_irqsave)(&cxl_cper_prot_err_work_lock);
>
> - guard(spinlock)(&cxl_cper_prot_err_work_lock);
> + if (WARN_ONCE(cxl_cper_prot_err_work,
> + "CPER-CXL kfifo consumer already registered\n"))
> + return -EINVAL;
> cxl_cper_prot_err_work = work;
> return 0;
> }
> @@ -791,11 +792,18 @@ EXPORT_SYMBOL_NS_GPL(cxl_cper_register_prot_err_work, "CXL");
>
> int cxl_cper_unregister_prot_err_work(struct work_struct *work)
> {
> - if (cxl_cper_prot_err_work != work)
> - return -EINVAL;
> + scoped_guard(raw_spinlock_irqsave, &cxl_cper_prot_err_work_lock) {
> + if (WARN_ONCE(cxl_cper_prot_err_work != work,
> + "CPER-CXL kfifo consumer mismatch on unregister\n"))
> + return -EINVAL;
> + cxl_cper_prot_err_work = NULL;
> + }
> +
> + cancel_work_sync(work);
> +
> + /* Discard stale entries so they are not replayed on next module load */
> + kfifo_reset(&cxl_cper_prot_err_fifo);
>
> - guard(spinlock)(&cxl_cper_prot_err_work_lock);
> - cxl_cper_prot_err_work = NULL;
> return 0;
> }
> EXPORT_SYMBOL_NS_GPL(cxl_cper_unregister_prot_err_work, "CXL");
> @@ -811,7 +819,7 @@ EXPORT_SYMBOL_NS_GPL(cxl_cper_prot_err_kfifo_get, "CXL");
> DEFINE_KFIFO(cxl_cper_fifo, struct cxl_cper_work_data, CXL_CPER_FIFO_DEPTH);
>
> /* Synchronize schedule_work() with cxl_cper_work changes */
> -static DEFINE_SPINLOCK(cxl_cper_work_lock);
> +static DEFINE_RAW_SPINLOCK(cxl_cper_work_lock);
> struct work_struct *cxl_cper_work;
>
> static void cxl_cper_post_event(enum cxl_event_type event_type,
> @@ -831,7 +839,7 @@ static void cxl_cper_post_event(enum cxl_event_type event_type,
> return;
> }
>
> - guard(spinlock_irqsave)(&cxl_cper_work_lock);
> + guard(raw_spinlock_irqsave)(&cxl_cper_work_lock);
>
> if (!cxl_cper_work)
> return;
> @@ -849,10 +857,11 @@ static void cxl_cper_post_event(enum cxl_event_type event_type,
>
> int cxl_cper_register_work(struct work_struct *work)
> {
> - if (cxl_cper_work)
> + guard(raw_spinlock_irqsave)(&cxl_cper_work_lock);
> + if (WARN_ONCE(cxl_cper_work,
> + "CXL CPER kfifo consumer already registered\n"))
> return -EINVAL;
>
> - guard(spinlock)(&cxl_cper_work_lock);
> cxl_cper_work = work;
> return 0;
> }
> @@ -860,11 +869,18 @@ EXPORT_SYMBOL_NS_GPL(cxl_cper_register_work, "CXL");
>
> int cxl_cper_unregister_work(struct work_struct *work)
> {
> - if (cxl_cper_work != work)
> - return -EINVAL;
> + scoped_guard(raw_spinlock_irqsave, &cxl_cper_work_lock) {
> + if (WARN_ONCE(cxl_cper_work != work,
> + "CXL CPER kfifo consumer mismatch on unregister\n"))
> + return -EINVAL;
> + cxl_cper_work = NULL;
> + }
> +
> + cancel_work_sync(work);
> +
> + /* Discard stale entries so they are not replayed on next module load */
> + kfifo_reset(&cxl_cper_fifo);
>
> - guard(spinlock)(&cxl_cper_work_lock);
> - cxl_cper_work = NULL;
> return 0;
> }
> EXPORT_SYMBOL_NS_GPL(cxl_cper_unregister_work, "CXL");
> diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
> index 267c679b0b3c2..7c6faee7f85ed 100644
> --- a/drivers/cxl/pci.c
> +++ b/drivers/cxl/pci.c
> @@ -1083,7 +1083,6 @@ static int __init cxl_pci_driver_init(void)
> static void __exit cxl_pci_driver_exit(void)
> {
> cxl_cper_unregister_work(&cxl_cper_work);
> - cancel_work_sync(&cxl_cper_work);
> pci_unregister_driver(&cxl_pci_driver);
> }
>
^ permalink raw reply
* Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot
From: Babu Moger @ 2026-07-20 20:12 UTC (permalink / raw)
To: tony.luck, reinette.chatre, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
In-Reply-To: <8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger@amd.com>
On 7/17/26 16:13, Babu Moger wrote:
> The kernel currently enables the ABMC-based "mbm_event" mode by default on
> hardware that supports it. However, this can cause bandwidth monitoring
> failures with existing userspace tools such as pqos.
>
> The pqos tool mounts the resctrl filesystem and creates 16 or more resctrl
> groups by default. On systems with 32 or fewer ABMC counters, this default
> configuration can consume all available counters, since each group requires
> one counter for local MBM and another for total MBM. If additional
> monitoring groups are created, counter resources are exhausted and pqos
> tool reports memory bandwidth counters as zero for those groups.
>
> Avoid this compatibility issue by leaving mbm_assign_mode in the "default"
> mode during initialization. Users who want to use ABMC can continue to
> enable it explicitly:
>
> echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
>
> Update the resctrl documentation to reflect the new boot-time default and
> adjust the mbm_assign_mode examples accordingly.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> There are plans to enable "mbm_event" by default once additional counters
> are available. For now, keep the default mode to maintain compatibility
> with existing tools.
> ---
> Documentation/filesystems/resctrl.rst | 71 ++++++++++++++++-----------
> arch/x86/kernel/cpu/resctrl/monitor.c | 1 -
> 2 files changed, 42 insertions(+), 30 deletions(-)
>
> diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
> index e4b66af55ffb..e38bfd15fc3d 100644
> --- a/Documentation/filesystems/resctrl.rst
> +++ b/Documentation/filesystems/resctrl.rst
> @@ -355,8 +355,8 @@ with the following files:
> ::
>
> # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> - [mbm_event]
> - default
> + [default]
> + mbm_event
>
> "mbm_event":
>
> @@ -375,19 +375,23 @@ with the following files:
> to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
>
> The mode is beneficial for AMD platforms that support more CTRL_MON
> - and MON groups than available hardware counters. By default, this
> - feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
> - Monitoring Counters) capability, ensuring counters remain assigned even
> - when the corresponding RMID is not actively used by any processor.
> + and MON groups than available hardware counters. On platforms with the
> + ABMC (Assignable Bandwidth Monitoring Counters) capability, mbm_event
> + mode ensures counters remain assigned even when the corresponding RMID
> + is not actively used by any processor.
>
> "default":
>
> In default mode, resctrl assumes there is a hardware counter for each
> - event within every CTRL_MON and MON group. On AMD platforms, it is
> - recommended to use the mbm_event mode, if supported, to prevent reset of MBM
> - events between reads resulting from hardware re-allocating counters. This can
> - result in misleading values or display "Unavailable" if no counter is assigned
> - to the event.
> + event within every CTRL_MON and MON group. This mode is enabled by default.
> +
> + On AMD platforms with more CTRL_MON and MON groups than the available
> + hardware counters, hardware may re-allocate counters between reads
> + while in default mode. This can result in misleading memory bandwidth values
> + or display "Unavailable" if no counter is allocated to the event. In such
> + cases, it is recommended to use the mbm_event mode, if supported, to prevent
> + reset of MBM events between reads resulting from hardware re-allocating
> + counters.
>
> * To enable "mbm_event" counter assignment mode:
> ::
> @@ -471,8 +475,8 @@ with the following files:
>
> Determines if a counter will automatically be assigned to an RMID, MBM event
> pair when its associated monitor group is created via mkdir. Enabled by default
> - on boot, also when switched from "default" mode to "mbm_event" counter assignment
> - mode. Users can disable this capability by writing to the interface.
> + when switched to "mbm_event" counter assignment mode. Users can disable this
> + capability by writing to the interface.
>
> "0":
> Auto assignment is disabled.
> @@ -1788,32 +1792,41 @@ a. Check if MBM counter assignment mode is supported.
>
> # mount -t resctrl resctrl /sys/fs/resctrl/
>
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> + [default]
> + mbm_event
> +
> +The "mbm_event" and "default" modes are supported. The "default" mode
> +is enabled by default.
> +
> +b. Enable "mbm_event" counter assignment mode.
> +::
> +
> + # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> [mbm_event]
> default
>
> -The "mbm_event" mode is detected and enabled.
> -
> -b. Check how many assignable counters are supported.
> +c. Check how many assignable counters are supported.
> ::
>
> # cat /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs
> 0=32;1=32
>
> -c. Check how many assignable counters are available for assignment in each domain.
> +d. Check how many assignable counters are available for assignment in each domain.
> ::
>
> # cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
> 0=30;1=30
>
> -d. To list the default group's assign states.
> +e. To list the default group's assign states.
> ::
>
> # cat /sys/fs/resctrl/mbm_L3_assignments
> mbm_total_bytes:0=e;1=e
> mbm_local_bytes:0=e;1=e
Shasiko reported.
https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
--- Patch [1]: [PATCH 1/2] x86/resctrl, Documentation: Keep
mbm_assign_mode "default" on boot ---
commit 41dfae2edbe7538cef040aa9e0f1f0f5a5873564
Author: Babu Moger <babu.moger@amd.com>
x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot
This commit changes the default MBM assignment mode to "default" on boot to
maintain compatibility with existing userspace tools like pqos. Users
who want
ABMC features must now explicitly enable the "mbm_event" mode.
> diff --git a/Documentation/filesystems/resctrl.rst
b/Documentation/filesystems/resctrl.rst
> index e4b66af55ffba..e38bfd15fc3d0 100644
> --- a/Documentation/filesystems/resctrl.rst
> +++ b/Documentation/filesystems/resctrl.rst
[ ... ]
> @@ -1788,32 +1792,41 @@ a. Check if MBM counter assignment mode is
supported.
>
> # mount -t resctrl resctrl /sys/fs/resctrl/
>
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> + [default]
> + mbm_event
> +
> +The "mbm_event" and "default" modes are supported. The "default" mode
> +is enabled by default.
> +
> +b. Enable "mbm_event" counter assignment mode.
> +::
> +
> + # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> [mbm_event]
> default
[ ... ]
> -d. To list the default group's assign states.
> +e. To list the default group's assign states.
> ::
>
> # cat /sys/fs/resctrl/mbm_L3_assignments
> mbm_total_bytes:0=e;1=e
> mbm_local_bytes:0=e;1=e
Does switching to "mbm_event" mode automatically assign counters to the
default group as shown here?
Because this patch changes the boot default to "default", users must now
transition to "mbm_event" dynamically. Looking at the transition path:
fs/resctrl/monitor.c:resctrl_mbm_assign_mode_write() {
...
/* Enable auto assignment when switching to "mbm_event" mode */
if (enable)
r->mon.mbm_assign_on_mkdir = true;
/*
* Reset all the non-achitectural RMID state and assignable counters.
*/
list_for_each_entry_rcu(d, &r->mon_domains, hdr.list,
lockdep_is_cpus_held()) {
mbm_cntr_free_all(r, d);
resctrl_reset_rmid_all(r, d);
}
...
}
This path clears all existing counters and sets mbm_assign_on_mkdir for
new groups, but does not appear to assign counters for the already-existing
default group.
Prior to this patch, booting directly into "mbm_event" mode handled the
default group assignment during rdt_get_tree().
Will the default group actually show _ (unassigned) states here instead
of the documented e (assigned) states, causing bandwidth monitoring to
return "Unassigned" until manually fixed?
===========================================================================
Yes. It does not automatically assign the counters to already created
groups. User need to assign the counters manually to read the events.
The documentation needs to updated.
Thanks
Babu
^ permalink raw reply
* Re: [PATCH v3] docs: dt: maintainer: Add Devicetree and OF maintainer profile document
From: Rob Herring @ 2026-07-20 20:12 UTC (permalink / raw)
To: Krzysztof Kozlowski
Cc: Krzysztof Kozlowski, Conor Dooley, Jonathan Corbet, Shuah Khan,
devicetree, workflows, linux-doc, linux-kernel, Saravana Kannan
In-Reply-To: <20260720184040.9870-2-krzysztof.kozlowski@oss.qualcomm.com>
On Mon, Jul 20, 2026 at 1:40 PM Krzysztof Kozlowski
<krzysztof.kozlowski@oss.qualcomm.com> wrote:
>
> Document how Devicetree and Open Firmware maintainers handle their
> subsystem, especially focusing on two caveats:
>
> Devicetree subsystem handles patches with a minor difference comparing
> to other subsystems: while DT maintainers pick up OF code, they only
> provide review of DT bindings without applying these.
>
> All three DT bindings maintainers rely currently on Patchwork and due to
> enormous amount of emails per day, regardless how much DT maintainers
> try, they cannot read all the emails.
>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Conor Dooley <conor+dt@kernel.org>
> Cc: Saravana Kannan <saravanak@kernel.org>
> Cc: devicetree@vger.kernel.org
> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
>
> ---
>
> I expect patch to be picked up by Rob, after review.
>
> Changes in v3:
> 1. Add also F: entry
>
> Changes in v2:
> 1. Correct typos and trailing white spaces.
> 2. Fix order of P: after C: in maintainers.
> ---
> .../process/maintainer-devicetree.rst | 70 +++++++++++++++++++
> MAINTAINERS | 3 +
> 2 files changed, 73 insertions(+)
> create mode 100644 Documentation/process/maintainer-devicetree.rst
>
> diff --git a/Documentation/process/maintainer-devicetree.rst b/Documentation/process/maintainer-devicetree.rst
> new file mode 100644
> index 000000000000..d8ffe752bf5d
> --- /dev/null
> +++ b/Documentation/process/maintainer-devicetree.rst
> @@ -0,0 +1,70 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======================================
> +Devicetree and Open Firmware Subsystem
> +======================================
> +
> +Other Process Documents
> +-----------------------
> +
> +Please see the documents in Documentation/devicetree/bindings/ for information
> +on how to write proper Devicetree bindings and how to submit patches.
> +
> +Patch Review and Handling
> +-------------------------
> +
> +Patches handled by Devicetree maintainers are processed differently depending
> +on the patch type:
> +
> +1. Core OF driver code, e.g. drivers/of/:
> + patches are reviewed and applied by DT maintainers.
> +
> +2. Devicetree bindings:
> + patches are reviewed by DT maintainers but, except in certain cases, should
> + be applied by subsystem maintainers. See also *For kernel maintainers* in
> + Documentation/devicetree/bindings/submitting-patches.rst.
I would reword:
patches are reviewed by DT maintainers, but should be applied by
subsystem maintainers except in certain cases.
> +
> +3. DTS and drivers:
> + DT maintainers might provide comments, but review is generally not expected.
For DTS, we expect to pass schema checks or at least not add new warnings.
I can address these 2 when applying.
Rob
^ permalink raw reply
* Re: [PATCH v18 01/13] cxl/ras: Fix cxl_rch_get_aer_severity() wrong severity register
From: Dave Jiang @ 2026-07-20 20:09 UTC (permalink / raw)
To: Terry Bowman, Bjorn Helgaas, Dan Williams, Ira Weiny,
Jonathan Cameron, Len Brown, Rafael J . Wysocki, Robert Richter
Cc: linux-acpi, linux-cxl, linux-doc, linux-kernel, linux-pci,
linuxppc-dev, Alejandro Lucero, Alison Schofield, Ankit Agrawal,
Ard Biesheuvel, Ben Cheatham, Borislav Petkov, Breno Leitao,
Davidlohr Bueso, Fabio M . De Francesco, Gregory Price,
Hanjun Guo, Jonathan Corbet, Kees Cook,
Kuppuswamy Sathyanarayanan, Li Ming, Mahesh J Salgaonkar,
Mauro Carvalho Chehab, Oliver O'Halloran, Shiju Jose,
Shuah Khan, Shuai Xue, Smita Koralahalli, Tony Luck, Vishal Verma
In-Reply-To: <20260717222706.3540281-2-terry.bowman@amd.com>
On 7/17/26 3:26 PM, Terry Bowman wrote:
> cxl_rch_get_aer_severity() classifies RCH Downstream Port uncorrectable
> errors as fatal or non-fatal by ANDing uncorrectable status with
> PCI_ERR_ROOT_FATAL_RCV. This is wrong because PCI_ERR_ROOT_FATAL_RCV is a
> Root Error Status register bit (bit 6), not a severity bit. ANDing it
> against uncorrectable status tests a reserved bit and produces incorrect
> severity classification.
>
> Fix by ANDing the unmasked uncor_status against uncor_severity. Per
> PCIe Base Spec r6.0 Section 7.8.4.4, each bit in the Uncorrectable
> Error Severity register indicates whether the corresponding error is
> fatal (1) or non-fatal (0).
>
> Fixes: 6ac07883dbb5 ("cxl/pci: Add RCH downstream port error logging")
> Cc: stable@vger.kernel.org
> Signed-off-by: Terry Bowman <terry.bowman@amd.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
This can be picked up ahead looks like?
>
> ---
>
> Changes in v17 -> v18:
> - New patch.
> ---
> drivers/cxl/core/ras_rch.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/cxl/core/ras_rch.c b/drivers/cxl/core/ras_rch.c
> index 0a8b3b9b63884..44b335d560708 100644
> --- a/drivers/cxl/core/ras_rch.c
> +++ b/drivers/cxl/core/ras_rch.c
> @@ -80,7 +80,8 @@ static bool cxl_rch_get_aer_severity(struct aer_capability_regs *aer_regs,
> int *severity)
> {
> if (aer_regs->uncor_status & ~aer_regs->uncor_mask) {
> - if (aer_regs->uncor_status & PCI_ERR_ROOT_FATAL_RCV)
> + if ((aer_regs->uncor_status & ~aer_regs->uncor_mask) &
> + aer_regs->uncor_severity)
> *severity = AER_FATAL;
> else
> *severity = AER_NONFATAL;
^ permalink raw reply
* Re: [PATCH v3] docs: dt: maintainer: Add Devicetree and OF maintainer profile document
From: Krzysztof Kozlowski @ 2026-07-20 20:07 UTC (permalink / raw)
To: Jonathan Corbet, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Shuah Khan, devicetree, workflows, linux-doc, linux-kernel
Cc: Saravana Kannan
In-Reply-To: <87y0f531uz.fsf@trenco.lwn.net>
On 20/07/2026 21:19, Jonathan Corbet wrote:
>> I expect patch to be picked up by Rob, after review.
>>
>> Changes in v3:
>> 1. Add also F: entry
>>
>> Changes in v2:
>> 1. Correct typos and trailing white spaces.
>> 2. Fix order of P: after C: in maintainers.
>> ---
>> .../process/maintainer-devicetree.rst | 70 +++++++++++++++++++
>> MAINTAINERS | 3 +
>> 2 files changed, 73 insertions(+)
>> create mode 100644 Documentation/process/maintainer-devicetree.rst
>
> You'll need to add this new file to index.rst as well.
Hm, where? In docs/process/index.rst? None of the other maintainer
profiles are mentioned there.
Best regards,
Krzysztof
^ permalink raw reply
* Re: [PATCH v5 21/36] proc/kcore: include private-node RAM in the kcore RAM map
From: Omar Sandoval @ 2026-07-20 20:05 UTC (permalink / raw)
To: Gregory Price
Cc: linux-mm, Zhigang.Luo, arun.george, balbirs, brendan.jackman,
yuzenghui, apopple, alucerop, matthew.brost, akpm, david, ljs,
liam, vbabka, rppt, surenb, mhocko, corbet, skhan, gregkh, rafael,
dakr, djbw, vishal.l.verma, dave.jiang, alison.schofield, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, ying.huang, kasong, qi.zheng,
shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc, yury.norov,
linux, longman, ridong.chen, tj, mkoutny, sj, jgg, jhubbard,
peterx, baolin.wang, npache, ryan.roberts, dev.jain, lance.yang,
usama.arif, xu.xin16, chengming.zhou, roman.gushchin, muchun.song,
linux-kernel, linux-doc, driver-core, nvdimm, linux-cxl,
linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-22-gourry@gourry.net>
On Mon, Jul 20, 2026 at 03:34:15PM -0400, Gregory Price wrote:
> The kcore_ram_list bounds its scan at the highest pfn in N_MEMORY.
>
> As a result, private nodes placed above the last N_MEMORY node, which is
> the common case for device/CXL memory - is truncated out of /proc/kcore.
>
> Include N_MEMORY_PRIVATE nodes in the scan boundary. /proc/kcore is
> root-only (CAP_SYS_RAWIO), so this exposes nothing new - it just lets
> a debugger read private-node memory like any other RAM.
>
> Signed-off-by: Gregory Price <gourry@gourry.net>
> ---
> fs/proc/kcore.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c
> index 390547b992acd..f67194dc2a9b8 100644
> --- a/fs/proc/kcore.c
> +++ b/fs/proc/kcore.c
> @@ -251,11 +251,14 @@ static int kcore_ram_list(struct list_head *list)
> {
> int nid, ret;
> unsigned long end_pfn;
> + nodemask_t mem;
>
> /* Not initialized....update now */
> /* find out "max pfn" */
> end_pfn = 0;
> - for_each_node_state(nid, N_MEMORY) {
> + /* Include private node memory in the scan */
> + nodes_or(mem, node_states[N_MEMORY], node_states[N_MEMORY_PRIVATE]);
> + for_each_node_mask(nid, mem) {
> unsigned long node_end;
> node_end = node_end_pfn(nid);
> if (end_pfn < node_end)
> --
> 2.53.0-Meta
>
I'm sure this is the least of your concern with this series, but FWIW:
Acked-by: Omar Sandoval <osandov@osandov.com>
If everything goes well, kcore.c is quiet enough that this can probably
just be merged through whatever tree takes the rest of this work.
^ permalink raw reply
* [PATCH v5 35/36] KVM: guest_memfd: bind backing memory to a NUMA node at creation
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
guest_memfd presently allocates its page-cache folios through a
per-inode shared mempolicy (kvm_gmem_get_folio()).
Today that policy can only be set after the fact, via mbind() on
a host mmap of the fd. This requires the fd to be mmap-able and
cannot reach folios that are only ever guest-faulted (no host VMA).
Neither holds for a non-mappable (confidential) guest_memfd.
Add GUEST_MEMFD_FLAG_BIND_NODE.
When set, KVM builds an MPOL_BIND policy for the requested node and
installs it over the whole inode's shared policy, so every folio
is allocated on the requested node with no userspace mbind().
The flag is advertised through KVM_CAP_GUEST_MEMFD_FLAGS only when
CONFIG_NUMA is enabled.
Suggested-by: Dave Jiang <dave.jiang@intel.com>
Co-developed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Gregory Price <gourry@gourry.net>
Assisted-by: Claude:claude-opus-4-8
---
include/linux/kvm_host.h | 3 +++
include/uapi/linux/kvm.h | 5 ++++-
virt/kvm/guest_memfd.c | 39 +++++++++++++++++++++++++++++++++++++--
3 files changed, 44 insertions(+), 3 deletions(-)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index ab8cfaec82d31..f6448a73d464c 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -739,6 +739,9 @@ static inline u64 kvm_gmem_get_supported_flags(struct kvm *kvm)
if (!kvm || kvm_arch_supports_gmem_init_shared(kvm))
flags |= GUEST_MEMFD_FLAG_INIT_SHARED;
+ if (IS_ENABLED(CONFIG_NUMA))
+ flags |= GUEST_MEMFD_FLAG_BIND_NODE;
+
return flags;
}
#endif
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 419011097fa8e..a8ff150e2cbb7 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1654,11 +1654,14 @@ struct kvm_memory_attributes {
#define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd)
#define GUEST_MEMFD_FLAG_MMAP (1ULL << 0)
#define GUEST_MEMFD_FLAG_INIT_SHARED (1ULL << 1)
+#define GUEST_MEMFD_FLAG_BIND_NODE (1ULL << 2)
struct kvm_create_guest_memfd {
__u64 size;
__u64 flags;
- __u64 reserved[6];
+ __u32 node;
+ __u32 pad;
+ __u64 reserved[5];
};
#define KVM_PRE_FAULT_MEMORY _IOWR(KVMIO, 0xd5, struct kvm_pre_fault_memory)
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index f0e5da4908660..de320df08e0ff 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -456,6 +456,26 @@ static struct mempolicy *kvm_gmem_get_policy(struct vm_area_struct *vma,
*/
return mpol_shared_policy_lookup(&GMEM_I(inode)->policy, pgoff);
}
+
+static int kvm_gmem_bind_node(struct inode *inode, int node)
+{
+ struct mempolicy *pol;
+ int err;
+
+ pol = mpol_bind_node(node);
+ if (IS_ERR(pol))
+ return PTR_ERR(pol);
+
+ err = mpol_set_shared_policy_range(&GMEM_I(inode)->policy, 0,
+ MAX_LFS_FILESIZE >> PAGE_SHIFT, pol);
+ mpol_put(pol);
+ return err;
+}
+#else
+static int kvm_gmem_bind_node(struct inode *inode, int node)
+{
+ return -EINVAL;
+}
#endif /* CONFIG_NUMA */
static const struct vm_operations_struct kvm_gmem_vm_ops = {
@@ -557,7 +577,7 @@ bool __weak kvm_arch_supports_gmem_init_shared(struct kvm *kvm)
return true;
}
-static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
+static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags, int node)
{
static const char *name = "[kvm-gmem]";
struct gmem_file *f;
@@ -598,6 +618,12 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
GMEM_I(inode)->flags = flags;
+ if (flags & GUEST_MEMFD_FLAG_BIND_NODE) {
+ err = kvm_gmem_bind_node(inode, node);
+ if (err)
+ goto err_inode;
+ }
+
file = alloc_file_pseudo(inode, kvm_gmem_mnt, name, O_RDWR, &kvm_gmem_fops);
if (IS_ERR(file)) {
err = PTR_ERR(file);
@@ -630,6 +656,7 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
{
loff_t size = args->size;
u64 flags = args->flags;
+ int node = NUMA_NO_NODE;
if (flags & ~kvm_gmem_get_supported_flags(kvm))
return -EINVAL;
@@ -637,7 +664,15 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
if (size <= 0 || !PAGE_ALIGNED(size))
return -EINVAL;
- return __kvm_gmem_create(kvm, size, flags);
+ if (flags & GUEST_MEMFD_FLAG_BIND_NODE) {
+ if (args->pad || args->node >= MAX_NUMNODES)
+ return -EINVAL;
+ node = args->node;
+ } else if (args->node || args->pad) {
+ return -EINVAL;
+ }
+
+ return __kvm_gmem_create(kvm, size, flags, node);
}
int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 34/36] mm/mempolicy: add mpol_set_shared_policy_range()
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
mpol_set_shared_policy() installs a policy over a VMA's page-offset
range, which is the only programmatic way to populate an inode's
shared policy after init.
Two limitations make it unusable for binding an entire backing inode
from in-kernel code:
- It requires a VMA, so it cannot cover unmapped offsets.
(e.g. unmapped file folios faulted by pagecache)
- mpol_shared_policy_init(), the only no-VMA installer, reconstructs
the policy from mpol->w.user_nodemask. That field is only populated
for static/relative or mount-string policies.
a policy built programmatically (e.g. by mpol_bind_node()) leaves it
empty and stores w.cpuset_mems_allowed instead, so _init mangles it.
Add mpol_set_shared_policy_range(), which installs an already-built,
fully contextualised policy verbatim over an arbitrary [start, end)
page range with no VMA.
Reimplement mpol_set_shared_policy() as a thin wrapper that derives
the range from the VMA, so both share a single underlying path.
Suggested-by: Dave Jiang <dave.jiang@intel.com>
Co-developed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Gregory Price <gourry@gourry.net>
Assisted-by: Claude:claude-opus-4-8
---
include/linux/mempolicy.h | 2 ++
mm/mempolicy.c | 35 +++++++++++++++++++++++++++++------
2 files changed, 31 insertions(+), 6 deletions(-)
diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
index 715951a5b03c1..1348e9f5f2cf9 100644
--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -125,6 +125,8 @@ int vma_dup_policy(struct vm_area_struct *src, struct vm_area_struct *dst);
void mpol_shared_policy_init(struct shared_policy *sp, struct mempolicy *mpol);
int mpol_set_shared_policy(struct shared_policy *sp,
struct vm_area_struct *vma, struct mempolicy *mpol);
+int mpol_set_shared_policy_range(struct shared_policy *sp, pgoff_t start,
+ pgoff_t end, struct mempolicy *mpol);
void mpol_free_shared_policy(struct shared_policy *sp);
struct mempolicy *mpol_shared_policy_lookup(struct shared_policy *sp,
pgoff_t idx);
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 4daba81fff7c7..23c4c25097450 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -3424,24 +3424,47 @@ void mpol_shared_policy_init(struct shared_policy *sp, struct mempolicy *mpol)
}
EXPORT_SYMBOL_FOR_MODULES(mpol_shared_policy_init, "kvm");
-int mpol_set_shared_policy(struct shared_policy *sp,
- struct vm_area_struct *vma, struct mempolicy *pol)
+/**
+ * mpol_set_shared_policy_range - install @pol over [@start, @end) of @sp
+ * @sp: the shared policy tree
+ * @start: first page offset (inclusive)
+ * @end: last page offset (exclusive)
+ * @pol: a fully-built, validated policy, or NULL to clear the range
+ *
+ * Installs @pol over the given range, replacing any overlapping policy.
+ * @sp takes its own reference, the caller retains its reference on @pol.
+ *
+ * The policy is not reconstructed, so the policy is preserved exactly.
+ *
+ * Unlike mpol_set_shared_policy(), no VMA is required, so a range that
+ * is never mapped into a VMA can be covered, including the whole file.
+ *
+ * Return: 0 on success, -ENOMEM on allocation failure.
+ */
+int mpol_set_shared_policy_range(struct shared_policy *sp, pgoff_t start,
+ pgoff_t end, struct mempolicy *pol)
{
- const pgoff_t pgoff = vma_start_pgoff(vma);
- const pgoff_t pgoff_end = vma_end_pgoff(vma);
struct sp_node *new = NULL;
int err;
if (pol) {
- new = sp_alloc(pgoff, pgoff_end, pol);
+ new = sp_alloc(start, end, pol);
if (!new)
return -ENOMEM;
}
- err = shared_policy_replace(sp, pgoff, pgoff_end, new);
+ err = shared_policy_replace(sp, start, end, new);
if (err && new)
sp_free(new);
return err;
}
+EXPORT_SYMBOL_FOR_MODULES(mpol_set_shared_policy_range, "kvm");
+
+int mpol_set_shared_policy(struct shared_policy *sp,
+ struct vm_area_struct *vma, struct mempolicy *pol)
+{
+ return mpol_set_shared_policy_range(sp, vma->vm_pgoff,
+ vma->vm_pgoff + vma_pages(vma), pol);
+}
EXPORT_SYMBOL_FOR_MODULES(mpol_set_shared_policy, "kvm");
/* Free a backing policy store on inode delete. */
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 33/36] Documentation/mm: describe private (N_MEMORY_PRIVATE) memory nodes
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Add a design overview of private memory nodes:
- the isolation model (structural zonelist exclusion)
- ZONELIST_PRIVATE
- driver provisioning API
- capability opt-in model and its dependency rules
- observability surfaces
Signed-off-by: Gregory Price <gourry@gourry.net>
---
Documentation/mm/index.rst | 1 +
Documentation/mm/numa_private_nodes.rst | 160 ++++++++++++++++++++++++
2 files changed, 161 insertions(+)
create mode 100644 Documentation/mm/numa_private_nodes.rst
diff --git a/Documentation/mm/index.rst b/Documentation/mm/index.rst
index 13a79f5d092c0..f60704df6104c 100644
--- a/Documentation/mm/index.rst
+++ b/Documentation/mm/index.rst
@@ -65,6 +65,7 @@ documentation, or deleted if it has served its purpose.
mmu_notifier
multigen_lru
numa
+ numa_private_nodes
overcommit-accounting
page_migration
page_frags
diff --git a/Documentation/mm/numa_private_nodes.rst b/Documentation/mm/numa_private_nodes.rst
new file mode 100644
index 0000000000000..3b27a2e24b086
--- /dev/null
+++ b/Documentation/mm/numa_private_nodes.rst
@@ -0,0 +1,160 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+====================
+Private memory nodes
+====================
+
+A *private memory node* is a NUMA node whose memory is hotplugged by a driver
+and deliberately hidden from the kernel's normal memory management. Such a
+node is marked ``N_MEMORY_PRIVATE`` instead of ``N_MEMORY``; the two states
+are mutually exclusive, so a private node is never considered by the page
+allocator's normal or fallback paths.
+
+The intent is to give a driver a block of NUMA-addressable memory that the rest
+of the kernel will not allocate from on its own, while still letting that memory
+be mapped into processes as ordinary, struct-page, LRU-managed folios -- and to
+let the driver re-enable individual mm services it is capable of allowing.
+
+Preconditions
+=============
+
+``N_MEMORY_PRIVATE`` and ``N_MEMORY`` are mutually exclusive, so the backing
+memory must come up on a node that has no DRAM of its own (otherwise the node
+would already be ``N_MEMORY``).
+
+In practice the memory is provided by a device driver or a DAX device whose
+target node has no other memory, and usually no CPUs.
+
+Isolation model
+===============
+
+Isolation is *opt-in by exclusion* and is **structural**: by default nothing in
+the kernel can place memory on a private node because the node is absent from the
+zonelists an ordinary allocation walks.
+
+Zonelist exclusion
+ The kernel page allocator depends on the ``FALLBACK`` and ``NOFALLBACK``
+ zonelists to allocate memory. A normal ``N_MEMORY`` node's zones (except
+ ``ZONE_DEVICE``) appear in these lists and allow allocations to fall-back
+ to less preferable locations if the preferred location is pressured.
+
+ ``__GFP_THISNODE`` is used during normal operation to switch between
+ ``FALLBACK`` and ``NOFALLBACK``, where ``NOFALLBACK`` only contains the
+ zonelists of the preferred node.
+
+ ``N_MEMORY_PRIVATE`` nodes are **excluded** from both ``FALLBACK`` and
+ ``NOFALLBACK`` zonelists. Instead they are added to ``ZONELIST_PRIVATE``,
+ which includes both ``N_MEMORY`` and ``N_MEMORY_PRIVATE`` nodes. This is
+ the only zonelist that contains private-node zones, and so the only way
+ to acquire private node allocations is to explicitly request that zonelist.
+
+ Even an allocation carrying ``__GFP_THISNODE`` cannot access the node's
+ memory without also explicitly passing the private zonelist. This prevents
+ incidental allocation of private memory by users of possible/online
+ nodelists.
+
+ When ``CONFIG_NUMA`` is disabled ``ZONELIST_PRIVATE`` aliases
+ ``ZONELIST_FALLBACK`` and is never selected.
+
+The user_numa path
+
+ ``MPOL_F_PRIVATE`` is an internal user_numa flag (never accepted from
+ userspace) marking that a mempolicy has a private node in its nodemask.
+
+ When ``CAP_USER_NUMA`` for a private node is set, user-sourced mempolicy
+ (``set_mempolicy(2)``) and migration (``move_pages(2)``) operations are
+ allowed to include that node in nodemasks and targets respectively.
+
+ ``mbind(MPOL_MF_MOVE)`` is both a mempolicy and a migration operation,
+ so placement and migration share the same capability.
+
+ The mempolicy component uses ``MPOL_F_PRIVATE`` at fault-time to select
+ ``ZONELIST_PRIVATE`` and makes the node's memory available for allocation.
+ It is otherwise an ordinary, relaxable mempolicy: an unsatisfiable request
+ (an unmovable allocation on a movable-only private node) simply falls back.
+
+
+cpuset interaction
+==================
+
+cpuset.mems does **not** partition private nodes. cpuset neither grants nor
+denies access, and rebinding cpuset.mems nodemasks do not affect a private node's
+residency in any nodemask.
+
+Likewise, a private node's inclusion in a nodemask does not affect cpuset.mems'
+filtering of any ``N_MEMORY`` - they remain partitioned according to cpuset.
+
+
+Provisioning
+============
+
+A driver brings memory up as private with::
+
+ add_private_memory_driver_managed(nid, start, size, resource_name,
+ mhp_flags, online_type, np)
+
+which onlines the range and registers the driver-owned ``struct node_private``
+(``np``) describing the node, including its capability bitmap (see below).
+
+Only one driver/service may register a ``struct node_private``, which
+heavily implies a "one-node-per-device" design of the system.
+
+The node leaves ``N_MEMORY_PRIVATE`` only when the last range is offlined.
+
+.. kernel-doc:: mm/memory_hotplug.c
+ :identifiers: __add_memory_driver_managed
+
+.. kernel-doc:: drivers/base/node.c
+ :identifiers: node_private_register node_private_unregister
+
+Capabilities (per-service opt-ins)
+==================================
+
+Because the default is "no mm service touches the node", each service a driver
+wants back is requested explicitly through a capability bit in
+``np->caps``. The mm side checks the matching ``node_allows_*()`` /
+``folio_allows_*()`` predicate before acting:
+
+.. list-table::
+ :header-rows: 1
+ :widths: 35 65
+
+ * - Capability
+ - Re-enables
+ * - ``NODE_PRIVATE_CAP_RECLAIM``
+ - reclaim of the node's folios, by the mm and by userspace
+ ``MADV_COLD`` / ``PAGEOUT`` / ``FREE`` (userland-driven reclaim)
+ * - ``NODE_PRIVATE_CAP_USER_NUMA``
+ - all userspace-directed placement and migration: ``mbind()`` /
+ ``set_mempolicy()`` / home node, and ``move_pages()`` /
+ ``migrate_pages()`` to/from the node
+ * - ``NODE_PRIVATE_CAP_HOTUNPLUG``
+ - hot-unplug via migration
+ * - ``NODE_PRIVATE_CAP_DEMOTION``
+ - reclaim-driven tiering demotion onto the node (the node joins the
+ demotion hierarchy)
+ * - ``NODE_PRIVATE_CAP_NUMA_BALANCING``
+ - access-based NUMA balancing scan/migration of the node's folios
+ * - ``NODE_PRIVATE_CAP_LTPIN``
+ - ``FOLL_LONGTERM`` GUP pins
+
+khugepaged never operates on private-node folios (like ZONE_DEVICE), and DAMON
+does not act on them; ``MADV_COLLAPSE`` is covered by ``CAP_USER_NUMA``.
+
+Dependencies between capabilities are enforced **once**, by
+``node_private_register()`` at hotplug, rather than by whatever sets the bits:
+
+* ``DEMOTION`` requires ``RECLAIM`` (a demotion target accumulates demoted
+ pages, so without reclaim as a safety valve it would just fill up).
+
+Capability flags are expected to be stable at runtime.
+
+Observability
+=============
+
+A private node is reported through:
+
+* ``/sys/devices/system/node/has_private_memory``
+* ``/proc/<pid>/numa_maps`` -- per-node residency includes private nodes
+* ``/proc/kcore`` -- private-node RAM appears in the kcore RAM map
+* memcg per-node statistics account private-node memory.
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 32/36] mm/khugepaged: base private node collapse eligiblity on actor/cap bits
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Collapse can move base pages across nodes. The khugepaged daemon does
this transparently, while MADV_COLLAPSE is a userland construct.
Isolate private nodes from khugepaged exactly like ZONE_DEVICE.
Never collapse private node folios silently (the owner may not
support migration).
MADV_COLLAPSE is user-initiated, so it is allowed on a private node
opted into CAP_USER_NUMA, consistent with other userspace interfaces.
folio_allows_collapse() encodes this: false for khugepaged on a private
node, CAP_USER_NUMA for MADV_COLLAPSE.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
mm/internal.h | 26 ++++++++++++++++++++++++++
mm/khugepaged.c | 16 +++++++++++++---
2 files changed, 39 insertions(+), 3 deletions(-)
diff --git a/mm/internal.h b/mm/internal.h
index 9621eb127c28c..9dbc5752cd3fe 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -136,6 +136,32 @@ static inline bool folio_allows_numa_balance(struct folio *folio)
node_allows_numa_balancing(folio_nid(folio));
}
+/*
+ * folio_allows_collapse() - may collapse fold this folio into a THP?
+ * @is_khugepaged: true for the khugepaged, false for MADV_COLLAPSE.
+ *
+ * Collapse is a residency operation gated by the actor calling it:
+ * - khugepaged never operates on private-node folios (like ZONE_DEVICE)
+ * - MADV_COLLAPSE is gated by CAP_USER_NUMA
+ *
+ * Never true for ZONE_DEVICE.
+ */
+static inline bool folio_allows_collapse(struct folio *folio, bool is_khugepaged)
+{
+ int nid = folio_nid(folio);
+
+ if (folio_is_zone_device(folio))
+ return false;
+ if (is_khugepaged)
+ return !node_is_private(nid);
+ return node_allows_user_numa(nid);
+}
+
+static inline bool page_allows_collapse(struct page *page, bool is_khugepaged)
+{
+ return folio_allows_collapse(page_folio(page), is_khugepaged);
+}
+
/*
* folio_allows_longterm_pin() - may this folio be long-term GUP-pinned?
*
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index fb4378cc17b10..5f20839857738 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -700,7 +700,8 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
goto out;
}
page = vm_normal_page(vma, addr, pteval);
- if (unlikely(!page) || unlikely(page_is_private_managed(page))) {
+ if (unlikely(!page) ||
+ unlikely(!page_allows_collapse(page, cc->is_khugepaged))) {
result = SCAN_PAGE_NULL;
goto out;
}
@@ -1241,9 +1242,17 @@ static enum scan_result alloc_charge_folio(struct folio **foliop, struct mm_stru
gfp_t gfp = (cc->is_khugepaged ? alloc_hugepage_khugepaged_gfpmask() :
GFP_TRANSHUGE);
int node = collapse_find_target_node(cc);
+ unsigned int aflags;
struct folio *folio;
+ bool allow;
- folio = __folio_alloc(gfp, order, node, &cc->alloc_nmask, ALLOC_DEFAULT);
+ /* Private node access: khugepaged never, madvise with CAP_USER_NUMA */
+ allow = cc->is_khugepaged ? !node_is_private(node)
+ : node_allows_user_numa(node);
+ aflags = (allow && node_is_private(node)) ?
+ ALLOC_ZONELIST_PRIVATE : ALLOC_DEFAULT;
+
+ folio = __folio_alloc(gfp, order, node, &cc->alloc_nmask, aflags);
if (!folio) {
*foliop = NULL;
if (is_pmd_order(order))
@@ -1687,7 +1696,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
}
page = vm_normal_page(vma, addr, pteval);
- if (unlikely(!page) || unlikely(page_is_private_managed(page))) {
+ if (unlikely(!page) ||
+ unlikely(!page_allows_collapse(page, cc->is_khugepaged))) {
result = SCAN_PAGE_NULL;
goto out_unmap;
}
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 31/36] mm: add NODE_PRIVATE_CAP_LTPIN for private node folio pinning
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Add NODE_PRIVATE_CAP_LTPIN so a node can opt back into gup pins.
An opted-in node is then pinned exactly like ordinary memory, or
migrated and pinned if in ZONE_MOVABLE.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/linux/node_private.h | 29 +++++++++++++++++++++++++++++
mm/gup.c | 3 ++-
mm/internal.h | 9 +++++----
3 files changed, 36 insertions(+), 5 deletions(-)
diff --git a/include/linux/node_private.h b/include/linux/node_private.h
index 5c3e070ed0deb..6f568bc0d46c7 100644
--- a/include/linux/node_private.h
+++ b/include/linux/node_private.h
@@ -17,6 +17,7 @@ struct page;
#define NODE_PRIVATE_CAP_HOTUNPLUG (1UL << 2) /* allow hot-unplug */
#define NODE_PRIVATE_CAP_DEMOTION (1UL << 3) /* allow tiering demotion */
#define NODE_PRIVATE_CAP_NUMA_BALANCING (1UL << 4) /* allow NUMA balancing */
+#define NODE_PRIVATE_CAP_LTPIN (1UL << 5) /* allow GUP pins */
/**
* struct node_private - Per-node container for N_MEMORY_PRIVATE nodes
@@ -167,6 +168,29 @@ static inline bool node_allows_numa_balancing(int nid)
return ret;
}
+/**
+ * node_allows_ltpin - may a folio on this node be long-term GUP-pinned?
+ * @nid: the node to test
+ *
+ * Opted-out private nodes cause longterm pins to outright fail regardless
+ * of ZONE placement (NORMAL would allow, MOVABLE would migrate first).
+ *
+ * Opted-in private nodes allow longterm pins to operate normally.
+ */
+static inline bool node_allows_ltpin(int nid)
+{
+ struct node_private *np;
+ bool ret;
+
+ if (!node_state(nid, N_MEMORY_PRIVATE))
+ return true;
+ rcu_read_lock();
+ np = rcu_dereference(NODE_DATA(nid)->node_private);
+ ret = np && (np->caps & NODE_PRIVATE_CAP_LTPIN);
+ rcu_read_unlock();
+ return ret;
+}
+
#else /* !CONFIG_NUMA */
static inline bool folio_is_private_node(struct folio *folio)
@@ -209,6 +233,11 @@ static inline bool node_allows_numa_balancing(int nid)
return true;
}
+static inline bool node_allows_ltpin(int nid)
+{
+ return true;
+}
+
#endif /* CONFIG_NUMA */
#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
diff --git a/mm/gup.c b/mm/gup.c
index a7d4de223785c..1e313aa590bc1 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -547,7 +547,8 @@ static struct folio *try_grab_folio_fast(struct page *page, int refs,
/*
* Can't do FOLL_LONGTERM + FOLL_PIN gup fast path if not in a
* right zone, so fail and let the caller fall back to the slow
- * path. Fail for private-node folios here so slow path rejects.
+ * path. Fail for non-opted-in private node folios here so the
+ * slow path can reject the pin entirely.
*/
if (unlikely((flags & FOLL_LONGTERM) &&
!folio_allows_longterm_pin(folio))) {
diff --git a/mm/internal.h b/mm/internal.h
index 01ab8b32b0bd8..9621eb127c28c 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -146,19 +146,20 @@ static inline bool folio_allows_numa_balance(struct folio *folio)
static inline bool folio_allows_longterm_pin(struct folio *folio)
{
return folio_is_longterm_pinnable(folio) &&
- !folio_is_private_node(folio);
+ node_allows_ltpin(folio_nid(folio));
}
/*
* folio_longterm_pin_forbidden() - must a longterm pin of this folio fail
* outright (neither pinned in place nor migrated off the node)?
*
- * True for any folio on a private node: such memory can be neither pinned
- * nor migrated, so the pin must be rejected with the folio left in place.
+ * True only for a folio on a private node that did not opt into longterm
+ * pinning (NODE_PRIVATE_CAP_LTPIN); node_allows_ltpin() is true for ordinary
+ * nodes and for opted-in private nodes, so this never trips them.
*/
static inline bool folio_longterm_pin_forbidden(struct folio *folio)
{
- return folio_is_private_node(folio);
+ return !node_allows_ltpin(folio_nid(folio));
}
/*
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 30/36] mm: add NODE_PRIVATE_CAP_NUMA_BALANCING for private-node NUMA balancing
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
By default NUMA balancing does not scan or prot_none private-node
folios, so the kernel never migrates them via access sampling.
Add NODE_PRIVATE_CAP_NUMA_BALANCING to opt a private node in.
Add folio_allows_numa_balance() to gate change_prot_numa() scans on
node eligibility. Opted-in private node participate like normal.
Unlike demotion, NUMA balancing is not reclaim-driven, so this
capability stands alone and does not require CAP_RECLAIM.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/linux/node_private.h | 29 +++++++++++++++++++++++++++++
mm/internal.h | 13 +++++++++++++
mm/mempolicy.c | 2 +-
3 files changed, 43 insertions(+), 1 deletion(-)
diff --git a/include/linux/node_private.h b/include/linux/node_private.h
index 87b03444b2c97..5c3e070ed0deb 100644
--- a/include/linux/node_private.h
+++ b/include/linux/node_private.h
@@ -16,6 +16,7 @@ struct page;
#define NODE_PRIVATE_CAP_USER_NUMA (1UL << 1) /* allow mempolicy */
#define NODE_PRIVATE_CAP_HOTUNPLUG (1UL << 2) /* allow hot-unplug */
#define NODE_PRIVATE_CAP_DEMOTION (1UL << 3) /* allow tiering demotion */
+#define NODE_PRIVATE_CAP_NUMA_BALANCING (1UL << 4) /* allow NUMA balancing */
/**
* struct node_private - Per-node container for N_MEMORY_PRIVATE nodes
@@ -143,6 +144,29 @@ static inline bool node_allows_demotion(int nid)
return ret;
}
+/**
+ * node_allows_numa_balancing - may NUMA balancing scan/migrate this node?
+ * @nid: the node to test
+ *
+ * Access-based promotion/migration. Unlike demotion this is not reclaim-driven,
+ * so CAP_NUMA_BALANCING stands alone (no CAP_RECLAIM dependency).
+ *
+ * return: true for normal nodes and private nodes opted into CAP_NUMA_BALANCING.
+ */
+static inline bool node_allows_numa_balancing(int nid)
+{
+ struct node_private *np;
+ bool ret;
+
+ if (!node_state(nid, N_MEMORY_PRIVATE))
+ return true;
+ rcu_read_lock();
+ np = rcu_dereference(NODE_DATA(nid)->node_private);
+ ret = np && (np->caps & NODE_PRIVATE_CAP_NUMA_BALANCING);
+ rcu_read_unlock();
+ return ret;
+}
+
#else /* !CONFIG_NUMA */
static inline bool folio_is_private_node(struct folio *folio)
@@ -180,6 +204,11 @@ static inline bool node_allows_demotion(int nid)
return true;
}
+static inline bool node_allows_numa_balancing(int nid)
+{
+ return true;
+}
+
#endif /* CONFIG_NUMA */
#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
diff --git a/mm/internal.h b/mm/internal.h
index 62e68acae08a3..01ab8b32b0bd8 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -123,6 +123,19 @@ static inline bool folio_allows_madvise(struct folio *folio)
node_allows_reclaim(folio_nid(folio));
}
+/*
+ * folio_allows_numa_balance() - may NUMA balancing scan/migrate this folio?
+ *
+ * NUMA balancing is access-aware tiering migration, so it follows the tiering
+ * opt-in: false for ZONE_DEVICE and for N_MEMORY_PRIVATE nodes without
+ * CAP_NUMA_BALANCING, true for all other folios.
+ */
+static inline bool folio_allows_numa_balance(struct folio *folio)
+{
+ return !folio_is_zone_device(folio) &&
+ node_allows_numa_balancing(folio_nid(folio));
+}
+
/*
* folio_allows_longterm_pin() - may this folio be long-term GUP-pinned?
*
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index fe42a510590a2..4daba81fff7c7 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -874,7 +874,7 @@ bool folio_can_map_prot_numa(struct folio *folio, struct vm_area_struct *vma,
{
int nid;
- if (!folio || folio_is_private_managed(folio) || folio_test_ksm(folio))
+ if (!folio || !folio_allows_numa_balance(folio) || folio_test_ksm(folio))
return false;
/* Also skip shared copy-on-write folios */
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 29/36] mm: add NODE_PRIVATE_CAP_DEMOTION for private-node tiering demotion
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
A private node is invisible to the tiering/demotion hierarchy by default:
memory-tiers.c never includes it, so reclaim never demotes onto it.
Add NODE_PRIVATE_CAP_DEMOTION to opt a private node into demotion.
When set, memory-tiers adds the node to the demotion set and reclaim's
demote path may target it (allocating from the private zonelist).
Demotion is driven by reclaim, so CAP_DEMOTION requires CAP_RECLAIM.
Otherwise the node can either fill up and drive odd system-wide OOM
behavior, or demotion doesn't work (nothing can demote from the node).
Signed-off-by: Gregory Price <gourry@gourry.net>
---
drivers/base/node.c | 5 +++++
include/linux/node_private.h | 30 +++++++++++++++++++++++++++++
mm/memory-tiers.c | 37 +++++++++++++++++++++++++++---------
mm/vmscan.c | 4 ++++
4 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 94cd51f51b7e8..3d61ca1b805dc 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -896,6 +896,11 @@ int node_private_register(int nid, struct node_private *np)
if (!np || !node_possible(nid))
return -EINVAL;
+ /* Demotion is driven by reclaim, so it requires reclaim. */
+ if ((np->caps & NODE_PRIVATE_CAP_DEMOTION) &&
+ !(np->caps & NODE_PRIVATE_CAP_RECLAIM))
+ return -EINVAL;
+
mutex_lock(&node_private_lock);
mem_hotplug_begin();
diff --git a/include/linux/node_private.h b/include/linux/node_private.h
index 7b617b1fa9c28..87b03444b2c97 100644
--- a/include/linux/node_private.h
+++ b/include/linux/node_private.h
@@ -15,6 +15,7 @@ struct page;
#define NODE_PRIVATE_CAP_RECLAIM (1UL << 0) /* allow mm reclaim */
#define NODE_PRIVATE_CAP_USER_NUMA (1UL << 1) /* allow mempolicy */
#define NODE_PRIVATE_CAP_HOTUNPLUG (1UL << 2) /* allow hot-unplug */
+#define NODE_PRIVATE_CAP_DEMOTION (1UL << 3) /* allow tiering demotion */
/**
* struct node_private - Per-node container for N_MEMORY_PRIVATE nodes
@@ -118,6 +119,30 @@ static inline bool node_allows_hotunplug(int nid)
return ret;
}
+/**
+ * node_allows_demotion - may kernel tiering demote to this node?
+ * @nid: the node to test
+ *
+ * Governs whether a private node participates in the demotion hierarchy.
+ * Demotion accumulates pages on the node, so CAP_DEMOTION requires CAP_RECLAIM
+ * (enforced at registration) as a safety valve.
+ *
+ * return: true for normal nodes and private nodes opted into CAP_DEMOTION.
+ */
+static inline bool node_allows_demotion(int nid)
+{
+ struct node_private *np;
+ bool ret;
+
+ if (!node_state(nid, N_MEMORY_PRIVATE))
+ return true;
+ rcu_read_lock();
+ np = rcu_dereference(NODE_DATA(nid)->node_private);
+ ret = np && (np->caps & NODE_PRIVATE_CAP_DEMOTION);
+ rcu_read_unlock();
+ return ret;
+}
+
#else /* !CONFIG_NUMA */
static inline bool folio_is_private_node(struct folio *folio)
@@ -150,6 +175,11 @@ static inline bool node_allows_hotunplug(int nid)
return true;
}
+static inline bool node_allows_demotion(int nid)
+{
+ return true;
+}
+
#endif /* CONFIG_NUMA */
#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 25e121851b586..c673080d153e4 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -7,6 +7,7 @@
#include <linux/memory-tiers.h>
#include <linux/notifier.h>
#include <linux/sched/sysctl.h>
+#include <linux/node_private.h>
#include "internal.h"
@@ -317,6 +318,21 @@ void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets)
rcu_read_unlock();
}
+/* Tiering set: N_MEMORY | (N_MEMORY_PRIVATE w/ CAP_DEMOTION) */
+static nodemask_t tierable_nodes;
+
+static void update_tierable_nodes(void)
+{
+ int node;
+
+ lockdep_assert_held_once(&memory_tier_lock);
+
+ tierable_nodes = node_states[N_MEMORY];
+ for_each_node_state(node, N_MEMORY_PRIVATE)
+ if (node_allows_demotion(node))
+ node_set(node, tierable_nodes);
+}
+
/**
* next_demotion_node() - Get the next node in the demotion path
* @node: The starting node to lookup the next node
@@ -330,7 +346,7 @@ void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets)
int next_demotion_node(int node, const nodemask_t *allowed_mask)
{
struct demotion_nodes *nd;
- nodemask_t mask;
+ nodemask_t mask, tierable;
if (!node_demotion)
return NUMA_NO_NODE;
@@ -370,7 +386,8 @@ int next_demotion_node(int node, const nodemask_t *allowed_mask)
* closest demotion target.
*/
nodes_complement(mask, *allowed_mask);
- return find_next_best_node_in(node, &mask, &node_states[N_MEMORY]);
+ tierable = tierable_nodes;
+ return find_next_best_node_in(node, &mask, &tierable);
}
static void disable_all_demotion_targets(void)
@@ -378,7 +395,7 @@ static void disable_all_demotion_targets(void)
struct memory_tier *memtier;
int node;
- for_each_node_state(node, N_MEMORY) {
+ for_each_node_mask(node, tierable_nodes) {
node_demotion[node].preferred = NODE_MASK_NONE;
/*
* We are holding memory_tier_lock, it is safe
@@ -401,7 +418,7 @@ static void dump_demotion_targets(void)
{
int node;
- for_each_node_state(node, N_MEMORY) {
+ for_each_node_mask(node, tierable_nodes) {
struct memory_tier *memtier = __node_get_memory_tier(node);
nodemask_t preferred = node_demotion[node].preferred;
@@ -435,9 +452,10 @@ static void establish_demotion_targets(void)
if (!node_demotion)
return;
+ update_tierable_nodes();
disable_all_demotion_targets();
- for_each_node_state(node, N_MEMORY) {
+ for_each_node_mask(node, tierable_nodes) {
best_distance = -1;
nd = &node_demotion[node];
@@ -455,7 +473,7 @@ static void establish_demotion_targets(void)
* nodelist to skip list so that we find the best node from the
* memtier nodelist.
*/
- nodes_andnot(tier_nodes, node_states[N_MEMORY], tier_nodes);
+ nodes_andnot(tier_nodes, tierable_nodes, tier_nodes);
/*
* Find all the nodes in the memory tier node list of same best distance.
@@ -464,7 +482,7 @@ static void establish_demotion_targets(void)
*/
do {
target = find_next_best_node_in(node, &tier_nodes,
- &node_states[N_MEMORY]);
+ &tierable_nodes);
if (target == NUMA_NO_NODE)
break;
@@ -503,7 +521,7 @@ static void establish_demotion_targets(void)
* allocation to a set of nodes that is closer the above selected
* preferred node.
*/
- lower_tier = node_states[N_MEMORY];
+ lower_tier = tierable_nodes;
list_for_each_entry(memtier, &memory_tiers, list) {
/*
* Keep removing current tier from lower_tier nodes,
@@ -550,7 +568,8 @@ static struct memory_tier *set_node_memory_tier(int node)
lockdep_assert_held_once(&memory_tier_lock);
- if (!node_state(node, N_MEMORY))
+ /* Include N_MEMORY and N_MEMORY_PRIVATE with CAP_DEMOTION */
+ if (!node_state(node, N_MEMORY) && !node_allows_demotion(node))
return ERR_PTR(-EINVAL);
mt_calc_adistance(node, &adist);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f1722693ac2db..b617f7cd1e716 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -963,6 +963,10 @@ static struct folio *alloc_demote_folio(struct folio *src,
mtc = (struct migration_target_control *)private;
+ if (mtc->nmask &&
+ nodes_intersects(*mtc->nmask, node_states[N_MEMORY_PRIVATE]))
+ mtc->alloc_flags = ALLOC_ZONELIST_PRIVATE;
+
/*
* make sure we allocate from the target node first also trying to
* demote or reclaim pages from the target node via kswapd if we are
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 28/36] mm: add NODE_PRIVATE_CAP_HOTUNPLUG for opted-in private nodes
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Add CAP_HOTUNPLUG, which allows memory_hotplug to migrate a private
node's folios for the purpose of hotunplugging memory.
Without this, hotunplug fails if any folio on the node is allocated.
Add node_allows_hotunplug() - which returns true for any normal node
and private nodes with CAP_HOTUNPLUG.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/linux/node_private.h | 26 ++++++++++++++++++++++++++
mm/memory_hotplug.c | 4 ++--
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/include/linux/node_private.h b/include/linux/node_private.h
index 655fe9ec5cb61..7b617b1fa9c28 100644
--- a/include/linux/node_private.h
+++ b/include/linux/node_private.h
@@ -14,6 +14,7 @@ struct page;
*/
#define NODE_PRIVATE_CAP_RECLAIM (1UL << 0) /* allow mm reclaim */
#define NODE_PRIVATE_CAP_USER_NUMA (1UL << 1) /* allow mempolicy */
+#define NODE_PRIVATE_CAP_HOTUNPLUG (1UL << 2) /* allow hot-unplug */
/**
* struct node_private - Per-node container for N_MEMORY_PRIVATE nodes
@@ -97,6 +98,26 @@ static inline bool node_allows_user_numa(int nid)
return ret;
}
+/**
+ * node_allows_hotunplug - may hot-unplug migrate this node's folios?
+ * @nid: the node to test
+ *
+ * True for normal nodes and private nodes opted into CAP_HOTUNPLUG.
+ */
+static inline bool node_allows_hotunplug(int nid)
+{
+ struct node_private *np;
+ bool ret;
+
+ if (!node_state(nid, N_MEMORY_PRIVATE))
+ return true;
+ rcu_read_lock();
+ np = rcu_dereference(NODE_DATA(nid)->node_private);
+ ret = np && (np->caps & NODE_PRIVATE_CAP_HOTUNPLUG);
+ rcu_read_unlock();
+ return ret;
+}
+
#else /* !CONFIG_NUMA */
static inline bool folio_is_private_node(struct folio *folio)
@@ -124,6 +145,11 @@ static inline bool node_allows_user_numa(int nid)
return true;
}
+static inline bool node_allows_hotunplug(int nid)
+{
+ return true;
+}
+
#endif /* CONFIG_NUMA */
#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 1f42ed303366c..e9573f90e0dcd 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1943,8 +1943,8 @@ static int do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
goto put_folio;
}
- /* Private node folios cannot migrate, fail outright */
- if (folio_is_private_node(folio)) {
+ /* Fail outright on private nodes w/o hotunplug support */
+ if (!node_allows_hotunplug(folio_nid(folio))) {
pr_info_ratelimited("memory offline refused: node %d pfn %lx\n",
folio_nid(folio), pfn);
folio_put(folio);
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 27/36] mm: add NODE_PRIVATE_CAP_USER_NUMA for userland numa controls
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Provide a mechanism to opt private nodes into userland numa management.
Add node_allows_user_numa() to encapsulate whether a node supports
userland NUMA controls (always true for normal nodes).
Placement - setting mempolicy via:
- mbind()
- set_mempolicy()
- set_mempolicy_home_node()
For mempolicy, enforcement lives in one place: mpol_set_nodemask()
Private nodes are not N_MEMORY, so they are trimmed from a nodemask
like a cpuset-trimmed node. All-private nodemasks without a valid
node collapse to empty and the mempolicy fails.
home_node is not special-cased - it is only a preferred-nid hint, and
placement is governed by the bind nodemask, so a home node pointed at
an invalid node simply falls back via the normal fallback zonelists.
(All the same behavior as a node/mask not intersecting cpuset.mems).
Migration - relocation of pages to/from a private node:
- mbind(MPOL_MF_MOVE)
- move_pages()
- migrate_pages()
mbind(MPOL_MF_MOVE) is a migration, so mempolicy and migration share
a single opt-in control.
The migration interfaces all check node eligibility and use
ALLOC_ZONELIST_PRIVATE to allow eligible migration requests to
move a folio to a private node..
alloc_migration_target() carries mtc->zlsel into the allocator
via __folio_alloc_zonelist().
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/linux/node_private.h | 33 +++++++++++++++++++++++++++++++++
mm/internal.h | 1 +
mm/mempolicy.c | 27 ++++++++++++++++++++-------
mm/migrate.c | 19 ++++++++++++++-----
4 files changed, 68 insertions(+), 12 deletions(-)
diff --git a/include/linux/node_private.h b/include/linux/node_private.h
index f7cbae1309904..655fe9ec5cb61 100644
--- a/include/linux/node_private.h
+++ b/include/linux/node_private.h
@@ -13,6 +13,7 @@ struct page;
* to let specific services operate on its node.
*/
#define NODE_PRIVATE_CAP_RECLAIM (1UL << 0) /* allow mm reclaim */
+#define NODE_PRIVATE_CAP_USER_NUMA (1UL << 1) /* allow mempolicy */
/**
* struct node_private - Per-node container for N_MEMORY_PRIVATE nodes
@@ -69,6 +70,33 @@ static inline bool node_allows_reclaim(int nid)
return ret;
}
+/**
+ * node_allows_user_numa - may userspace place or migrate memory here?
+ * @nid: the node to test
+ *
+ * Gate all userspace-directed memory operations on a private node.
+ * - mbind()/set_mempolicy()
+ * - move_pages()/migrate_pages()
+ *
+ * return: true for N_MEMORY and N_MEMORY_PRIVATE with CAP_USER_NUMA.
+ * false for memoryless or opted-out private node.
+ */
+static inline bool node_allows_user_numa(int nid)
+{
+ struct node_private *np;
+ bool ret;
+
+ if (node_state(nid, N_MEMORY))
+ return true;
+ if (!node_state(nid, N_MEMORY_PRIVATE))
+ return false;
+ rcu_read_lock();
+ np = rcu_dereference(NODE_DATA(nid)->node_private);
+ ret = np && (np->caps & NODE_PRIVATE_CAP_USER_NUMA);
+ rcu_read_unlock();
+ return ret;
+}
+
#else /* !CONFIG_NUMA */
static inline bool folio_is_private_node(struct folio *folio)
@@ -91,6 +119,11 @@ static inline bool node_allows_reclaim(int nid)
return true;
}
+static inline bool node_allows_user_numa(int nid)
+{
+ return true;
+}
+
#endif /* CONFIG_NUMA */
#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
diff --git a/mm/internal.h b/mm/internal.h
index 8329034ae561f..62e68acae08a3 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -1250,6 +1250,7 @@ struct migration_target_control {
nodemask_t *nmask;
gfp_t gfp_mask;
enum migrate_reason reason;
+ unsigned int alloc_flags;
};
/*
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index a3ffb09897489..fe42a510590a2 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -434,13 +434,14 @@ static int mpol_set_nodemask(struct mempolicy *pol,
/*
* Private nodes are not in cpuset.mems, so they're always stripped.
- * Driver-allocated policies will already have MPOL_F_PRIVATE set,
- * if that's the case, add back in the requested set of private nodes.
+ * Driver-allocated policies (MPOL_F_PRIVATE) and CAP_USER_NUMA private
+ * nodes should be added back into the nodemask.
*/
for_each_node_mask(nid, *nodes) {
if (!node_is_private(nid))
continue;
- if (pol->flags & MPOL_F_PRIVATE)
+ if ((pol->flags & MPOL_F_PRIVATE) ||
+ node_allows_user_numa(nid))
node_set(nid, nsc->mask2);
}
@@ -696,7 +697,7 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
}
if (!queue_folio_required(folio, qp))
return;
- if (folio_is_private_node(folio))
+ if (!node_allows_user_numa(folio_nid(folio)))
return;
if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
!vma_migratable(walk->vma) ||
@@ -752,7 +753,8 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
continue;
}
folio = vm_normal_folio(vma, addr, ptent);
- if (!folio || folio_is_private_managed(folio))
+ if (!folio || folio_is_zone_device(folio) ||
+ !node_allows_user_numa(folio_nid(folio)))
continue;
if (folio_test_large(folio) && max_nr != 1)
nr = folio_pte_batch(folio, pte, ptent, max_nr);
@@ -827,7 +829,7 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask,
folio = pfn_folio(pte_pfn(ptep));
if (!queue_folio_required(folio, qp))
goto unlock;
- if (folio_is_private_node(folio))
+ if (!node_allows_user_numa(folio_nid(folio)))
goto unlock;
if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
!vma_migratable(walk->vma)) {
@@ -1412,6 +1414,8 @@ static long migrate_to_node(struct mm_struct *mm, int source, int dest,
.nid = dest,
.gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_THISNODE,
.reason = MR_SYSCALL,
+ .alloc_flags = node_is_private(dest) ?
+ ALLOC_ZONELIST_PRIVATE : ALLOC_DEFAULT,
};
nodes_clear(nmask);
@@ -1985,9 +1989,10 @@ static int kernel_migrate_pages(pid_t pid, unsigned long maxnode,
struct mm_struct *mm = NULL;
struct task_struct *task;
nodemask_t task_nodes;
- int err;
+ nodemask_t priv_ok;
nodemask_t *old;
nodemask_t *new;
+ int err, nid;
NODEMASK_SCRATCH(scratch);
if (!scratch)
@@ -2027,7 +2032,14 @@ static int kernel_migrate_pages(pid_t pid, unsigned long maxnode,
}
rcu_read_unlock();
+ /* Private nodes are stripped by cpuset checks. Allow eligible ones. */
+ nodes_clear(priv_ok);
+ for_each_node_mask(nid, *new)
+ if (node_is_private(nid) && node_allows_user_numa(nid))
+ node_set(nid, priv_ok);
+
task_nodes = cpuset_mems_allowed(task);
+ nodes_or(task_nodes, task_nodes, priv_ok);
/* Is the user allowed to access the target nodes? */
if (!nodes_subset(*new, task_nodes) && !capable(CAP_SYS_NICE)) {
err = -EPERM;
@@ -2035,6 +2047,7 @@ static int kernel_migrate_pages(pid_t pid, unsigned long maxnode,
}
task_nodes = cpuset_mems_allowed(current);
+ nodes_or(task_nodes, task_nodes, priv_ok);
if (!nodes_and(*new, *new, task_nodes))
goto out_put;
diff --git a/mm/migrate.c b/mm/migrate.c
index d20674c07b947..b548d79352a38 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2231,7 +2231,8 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private)
if (is_highmem_idx(zidx) || zidx == ZONE_MOVABLE)
gfp_mask |= __GFP_HIGHMEM;
- return __folio_alloc(gfp_mask, order, nid, mtc->nmask, ALLOC_DEFAULT);
+ return __folio_alloc(gfp_mask, order, nid, mtc->nmask,
+ mtc->alloc_flags);
}
#ifdef CONFIG_NUMA_MIGRATION
@@ -2253,6 +2254,8 @@ static int do_move_pages_to_node(struct list_head *pagelist, int node)
.nid = node,
.gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_THISNODE,
.reason = MR_SYSCALL,
+ .alloc_flags = node_is_private(node) ?
+ ALLOC_ZONELIST_PRIVATE : ALLOC_DEFAULT,
};
err = migrate_pages(pagelist, alloc_migration_target, NULL,
@@ -2268,7 +2271,8 @@ static int __add_folio_for_migration(struct folio *folio, int node,
if (is_zero_folio(folio) || is_huge_zero_folio(folio))
return -EFAULT;
- if (folio_is_private_managed(folio))
+ if (folio_is_zone_device(folio) ||
+ !node_allows_user_numa(folio_nid(folio)))
return -ENOENT;
if (folio_nid(folio) == node)
@@ -2392,11 +2396,14 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes,
err = -ENODEV;
if (node < 0 || node >= MAX_NUMNODES)
goto out_flush;
- if (!node_state(node, N_MEMORY))
+
+ if (!node_allows_user_numa(node))
goto out_flush;
err = -EACCES;
- if (!node_isset(node, task_nodes))
+ /* Private nodes are not partitioned by cpuset.mem */
+ if (!node_is_private(node) &&
+ !node_isset(node, task_nodes))
goto out_flush;
if (current_node == NUMA_NO_NODE) {
@@ -2477,7 +2484,9 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages,
if (folio) {
if (is_zero_folio(folio) || is_huge_zero_folio(folio))
err = -EFAULT;
- else if (folio_is_private_managed(folio))
+ else if (folio_is_zone_device(folio) ||
+ (folio_is_private_node(folio) &&
+ !node_allows_user_numa(folio_nid(folio))))
err = -ENOENT;
else
err = folio_nid(folio);
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 26/36] mm: add NODE_PRIVATE_CAP_RECLAIM for opted-in private node reclaim
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Provide a mechanism to opt private nodes into the reclaim process.
Reclaim as a "singular service" is actually made up of:
- kswapd reclaim
- direct reclaim
- kcompactd compaction
- direct compaction
- both mglru / lru paths
- madvise reclaim hints
- damon reclaim operations
CAP_RECLAIM gates whether the kernel may do inter-node placement
(compaction) or swap for the folios on that private node.
node_allows_reclaim() encapsulates the policy as a whole. Ordinary
nodes are always reclaimable, private nodes only when opted in.
With the exception of madvise and DAMON, reclaim operations are
highly integrated with one another, so they are opted in/out of
together - otherwise reclaim becomes unpredictable.
To prevent bisect debugging failures, this stays in a single commit.
For example: reclaim without compaction will OOM on high-order
allocation failure despite a large amount of free memory. Normally
compaction would (potentially) resolve this issue.
If a private node opts into reclaim, we create normal watermarks for
that node - otherwise pgdat_balanced() is always true and reclaim
thinks there is no work to do.
Cross-node operations (demotion, promotion, khugepaged) are NOT
included in CAP_RECLAIM because some workflows may desire different
behaviors for this kind of operation:
- prefer direct-to-swap, do not demote
- remain resident and OOM
- __GFP_THISNODE: fail and let the driver augment reclaim
Normal swap-out is allowed because there are clear userland controls
(not registering swap, cgroup.swap, etc) to control that per-workload.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/linux/node_private.h | 33 +++++++++++++++++++++++++
mm/compaction.c | 9 ++++---
mm/damon/paddr.c | 4 +--
mm/huge_memory.c | 2 +-
mm/internal.h | 13 ++++++++++
mm/madvise.c | 6 ++---
mm/memory_hotplug.c | 3 +--
mm/page_alloc.c | 2 +-
mm/vmscan.c | 48 ++++++++++++++++++++++++++++++------
9 files changed, 100 insertions(+), 20 deletions(-)
diff --git a/include/linux/node_private.h b/include/linux/node_private.h
index 475496c84249f..f7cbae1309904 100644
--- a/include/linux/node_private.h
+++ b/include/linux/node_private.h
@@ -7,6 +7,13 @@
struct page;
+/*
+ * Per-node service opt-ins (node_private.caps). A private node is isolated
+ * from all general mm services by default; the registering driver sets these
+ * to let specific services operate on its node.
+ */
+#define NODE_PRIVATE_CAP_RECLAIM (1UL << 0) /* allow mm reclaim */
+
/**
* struct node_private - Per-node container for N_MEMORY_PRIVATE nodes
*
@@ -41,6 +48,27 @@ static inline bool node_is_private(int nid)
return node_state(nid, N_MEMORY_PRIVATE);
}
+/**
+ * node_allows_reclaim - may the mm reclaim from this node?
+ * @nid: the node to test
+ *
+ * Only a private node is ever excluded. Every other node can safely
+ * be operated on by reclaim.
+ */
+static inline bool node_allows_reclaim(int nid)
+{
+ struct node_private *np;
+ bool ret;
+
+ if (!node_state(nid, N_MEMORY_PRIVATE))
+ return true;
+ rcu_read_lock();
+ np = rcu_dereference(NODE_DATA(nid)->node_private);
+ ret = np && (np->caps & NODE_PRIVATE_CAP_RECLAIM);
+ rcu_read_unlock();
+ return ret;
+}
+
#else /* !CONFIG_NUMA */
static inline bool folio_is_private_node(struct folio *folio)
@@ -58,6 +86,11 @@ static inline bool node_is_private(int nid)
return false;
}
+static inline bool node_allows_reclaim(int nid)
+{
+ return true;
+}
+
#endif /* CONFIG_NUMA */
#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
diff --git a/mm/compaction.c b/mm/compaction.c
index 8c1351cce7bcc..b9c25c599732f 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -25,6 +25,7 @@
#include <linux/psi.h>
#include <linux/cpuset.h>
#include "page_alloc.h"
+#include <linux/node_private.h>
#include "internal.h"
#ifdef CONFIG_COMPACTION
@@ -2462,7 +2463,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
!__cpuset_zone_allowed(zone, gfp_mask))
continue;
- if (node_is_private(zone_to_nid(zone)))
+ if (!node_allows_reclaim(zone_to_nid(zone)))
continue;
/*
@@ -2856,7 +2857,7 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order,
!__cpuset_zone_allowed(zone, gfp_mask))
continue;
- if (node_is_private(zone_to_nid(zone)))
+ if (!node_allows_reclaim(zone_to_nid(zone)))
continue;
if (prio > MIN_COMPACT_PRIORITY
@@ -2928,7 +2929,7 @@ static int compact_node(pg_data_t *pgdat, bool proactive)
.proactive_compaction = proactive,
};
- if (node_is_private(pgdat->node_id))
+ if (!node_allows_reclaim(pgdat->node_id))
return 0;
for (zoneid = 0; zoneid < MAX_NR_ZONES; zoneid++) {
@@ -3026,7 +3027,7 @@ static ssize_t compact_store(struct device *dev,
{
int nid = dev->id;
- if (node_is_private(nid))
+ if (!node_allows_reclaim(nid))
return -EINVAL;
if (nid >= 0 && nid < nr_node_ids && node_online(nid)) {
diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c
index c741a94319750..b668cf55d1b67 100644
--- a/mm/damon/paddr.c
+++ b/mm/damon/paddr.c
@@ -251,8 +251,8 @@ static unsigned long damon_pa_pageout(struct damon_region *r,
continue;
}
- /* private node memory is not reclaimable by default */
- if (folio_is_private_node(folio))
+ /* DAMOS pageout is reclaim; gate a private node on CAP_RECLAIM */
+ if (!node_allows_reclaim(folio_nid(folio)))
goto put_folio;
if (damos_pa_filter_out(s, folio))
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 1df91b4e5c2bc..cebc89eb0541f 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2340,7 +2340,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
folio = pmd_folio(orig_pmd);
- if (folio_is_private_node(folio))
+ if (!node_allows_reclaim(folio_nid(folio)))
goto out;
/*
diff --git a/mm/internal.h b/mm/internal.h
index 85c460296cea1..8329034ae561f 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -110,6 +110,19 @@ static inline bool page_is_private_managed(struct page *page)
return folio_is_private_managed(page_folio(page));
}
+/*
+ * folio_allows_madvise() - may madvise reclaim hints act on this folio?
+ *
+ * madvise reclaim hints (COLD/PAGEOUT/FREE) are userland-driven reclaim, so
+ * they follow reclaim opt-in: false for ZONE_DEVICE and for N_MEMORY_PRIVATE
+ * nodes without CAP_RECLAIM, true for all other normal folios.
+ */
+static inline bool folio_allows_madvise(struct folio *folio)
+{
+ return !folio_is_zone_device(folio) &&
+ node_allows_reclaim(folio_nid(folio));
+}
+
/*
* folio_allows_longterm_pin() - may this folio be long-term GUP-pinned?
*
diff --git a/mm/madvise.c b/mm/madvise.c
index 29f35a23919a0..56ca974542707 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -396,7 +396,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
folio = pmd_folio(orig_pmd);
- if (folio_is_private_node(folio))
+ if (!node_allows_reclaim(folio_nid(folio)))
goto huge_unlock;
/* Do not interfere with other mappings of this folio */
@@ -478,7 +478,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
continue;
folio = vm_normal_folio(vma, addr, ptent);
- if (!folio || folio_is_private_managed(folio))
+ if (!folio || !folio_allows_madvise(folio))
continue;
/*
@@ -707,7 +707,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
}
folio = vm_normal_folio(vma, addr, ptent);
- if (!folio || folio_is_private_managed(folio))
+ if (!folio || !folio_allows_madvise(folio))
continue;
/*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index be230ac9efe5a..1f42ed303366c 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1237,8 +1237,7 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
/* reinitialise watermarks and update pcp limits */
init_per_zone_wmark_min();
- /* Private nodes opt-out of reclaim/compaction by default */
- if (!node_is_private(nid)) {
+ if (node_allows_reclaim(nid)) {
kswapd_run(nid);
kcompactd_run(nid);
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2b08bea2379a9..2667a4564b7ac 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -6667,7 +6667,7 @@ static void __setup_per_zone_wmarks(void)
u64 tmp;
spin_lock_irqsave(&zone->lock, flags);
- if (node_is_private(zone_to_nid(zone))) {
+ if (!node_allows_reclaim(zone_to_nid(zone))) {
zone->_watermark[WMARK_MIN] = 0;
zone->_watermark[WMARK_LOW] = 0;
zone->_watermark[WMARK_HIGH] = 0;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 86b2334c23b98..f1722693ac2db 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -5396,6 +5396,21 @@ static const struct attribute_group lru_gen_attr_group = {
* debugfs interface
******************************************************************************/
+/*
+ * Nodes the lru_gen debugfs interface lists: ordinary memory nodes plus any
+ * N_MEMORY_PRIVATE nodes opted into reclaim. run_cmd() already accepts the
+ * latter, so keep the listing in sync with what it accepts.
+ */
+static void lru_gen_seq_nodes(nodemask_t *nodes)
+{
+ int nid;
+
+ *nodes = node_states[N_MEMORY];
+ for_each_node_state(nid, N_MEMORY_PRIVATE)
+ if (node_allows_reclaim(nid))
+ node_set(nid, *nodes);
+}
+
static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos)
{
struct mem_cgroup *memcg;
@@ -5407,9 +5422,11 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos)
memcg = mem_cgroup_iter(NULL, NULL, NULL);
do {
+ nodemask_t nodes;
int nid;
- for_each_node_state(nid, N_MEMORY) {
+ lru_gen_seq_nodes(&nodes);
+ for_each_node_mask(nid, nodes) {
if (!nr_to_skip--)
return get_lruvec(memcg, nid);
}
@@ -5431,16 +5448,18 @@ static void *lru_gen_seq_next(struct seq_file *m, void *v, loff_t *pos)
{
int nid = lruvec_pgdat(v)->node_id;
struct mem_cgroup *memcg = lruvec_memcg(v);
+ nodemask_t nodes;
++*pos;
- nid = next_memory_node(nid);
+ lru_gen_seq_nodes(&nodes);
+ nid = next_node(nid, nodes);
if (nid == MAX_NUMNODES) {
memcg = mem_cgroup_iter(NULL, memcg, NULL);
if (!memcg)
return NULL;
- nid = first_memory_node;
+ nid = first_node(nodes);
}
return get_lruvec(memcg, nid);
@@ -5509,10 +5528,12 @@ static int lru_gen_seq_show(struct seq_file *m, void *v)
struct lru_gen_folio *lrugen = &lruvec->lrugen;
int nid = lruvec_pgdat(lruvec)->node_id;
struct mem_cgroup *memcg = lruvec_memcg(lruvec);
+ nodemask_t nodes;
DEFINE_MAX_SEQ(lruvec);
DEFINE_MIN_SEQ(lruvec);
- if (nid == first_memory_node) {
+ lru_gen_seq_nodes(&nodes);
+ if (nid == first_node(nodes)) {
const char *path = memcg ? m->private : "";
#ifdef CONFIG_MEMCG
@@ -5612,7 +5633,9 @@ static int run_cmd(char cmd, u64 memcg_id, int nid, unsigned long seq,
int err = -EINVAL;
struct mem_cgroup *memcg = NULL;
- if (nid < 0 || nid >= MAX_NUMNODES || !node_state(nid, N_MEMORY))
+ if (nid < 0 || nid >= MAX_NUMNODES ||
+ !(node_state(nid, N_MEMORY) ||
+ (node_is_private(nid) && node_allows_reclaim(nid))))
return -EINVAL;
if (!mem_cgroup_disabled()) {
@@ -6145,7 +6168,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc)
* Private nodes do not support reclaim by default, filtering here
* captures all normal reclaim paths that may attempt eviction.
*/
- if (node_is_private(pgdat->node_id))
+ if (!node_allows_reclaim(pgdat->node_id))
return;
if ((lru_gen_enabled() || lru_gen_switching()) && root_reclaim(sc)) {
@@ -6758,6 +6781,16 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg,
return sc.nr_reclaimed;
}
+static struct zonelist *memcg_reclaim_zonelist(int nid, gfp_t gfp_mask)
+{
+ unsigned int aflags = ALLOC_DEFAULT;
+
+ if (unlikely(!nodes_empty(node_states[N_MEMORY_PRIVATE])))
+ aflags = ALLOC_ZONELIST_PRIVATE;
+
+ return select_zonelist(nid, gfp_mask, aflags);
+}
+
unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
unsigned long nr_pages,
gfp_t gfp_mask,
@@ -6784,7 +6817,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
* equal pressure on all the nodes. This is based on the assumption that
* the reclaim does not bail out early.
*/
- struct zonelist *zonelist = node_zonelist(numa_node_id(), sc.gfp_mask);
+ struct zonelist *zonelist = memcg_reclaim_zonelist(numa_node_id(),
+ sc.gfp_mask);
set_task_reclaim_state(current, &sc.reclaim_state);
trace_mm_vmscan_memcg_reclaim_begin(sc.gfp_mask, 0, memcg);
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 25/36] mm/memory_hotplug: support N_MEMORY_PRIVATE node hotplug
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Add add_private_memory_driver_managed() to let modules hotplug
memory onto an N_MEMORY_PRIVATE node they control.
Ownership is denoted via `struct node_private` and a private node
can only have a single owner.
The common __add_memory_driver_managed function is only exported to
in-tree modules that allow hotplug of N_MEMORY nodes. The new private
wrapper does not allow N_MEMORY hotplug, only N_MEMORY_PRIVATE.
When node_private is NULL hotplug behaviour is unchanged.
When node_private is non-NULL the node owner is registered before any
memory is added. Later during hotplug, when memory is actually onlined,
hotplug marks the node N_MEMORY_PRIVATE instead of N_MEMORY.
If the add fails the registration is undone.
offline_pages() clears N_MEMORY_PRIVATE the same as N_MEMORY, and
offline_and_remove_memory_ranges() drops private node registration
once the last node's memory is removed.
Private nodes are opted out of starting kswapd/kcompactd by default.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
drivers/dax/kmem.c | 2 +-
include/linux/memory_hotplug.h | 5 +-
mm/memory_hotplug.c | 99 ++++++++++++++++++++++++++++++----
3 files changed, 94 insertions(+), 12 deletions(-)
diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
index a48d699cf344c..c3faf18e7f85d 100644
--- a/drivers/dax/kmem.c
+++ b/drivers/dax/kmem.c
@@ -126,7 +126,7 @@ static int dax_kmem_do_hotplug(struct dev_dax *dev_dax,
*/
rc = __add_memory_driver_managed(data->mgid, range.start,
range_len(&range), kmem_name, mhp_flags,
- online_type);
+ online_type, NULL);
if (rc) {
dev_warn(dev, "mapping%d: %#llx-%#llx memory add failed\n",
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index b39605d308963..fcceb0a423579 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -305,7 +305,10 @@ extern int add_memory_resource(int nid, struct resource *resource,
mhp_t mhp_flags);
int __add_memory_driver_managed(int nid, u64 start, u64 size,
const char *resource_name, mhp_t mhp_flags,
- enum mmop online_type);
+ enum mmop online_type, struct node_private *np);
+int add_private_memory_driver_managed(int nid, u64 start, u64 size,
+ const char *resource_name, mhp_t mhp_flags,
+ enum mmop online_type, struct node_private *np);
extern int add_memory_driver_managed(int nid, u64 start, u64 size,
const char *resource_name,
mhp_t mhp_flags);
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 2b9c0830821e6..be230ac9efe5a 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1169,7 +1169,7 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_MOVABLE,
true);
- if (!node_state(nid, N_MEMORY)) {
+ if (!node_state(nid, N_MEMORY) && !node_is_private(nid)) {
/* Adding memory to the node for the first time */
node_arg.nid = nid;
ret = node_notify(NODE_ADDING_FIRST_MEMORY, &node_arg);
@@ -1204,13 +1204,20 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
online_pages_range(pfn, nr_pages);
adjust_present_page_count(pfn_to_page(pfn), group, nr_pages);
+ /*
+ * N_MEMORY and N_MEMORY_PRIVATE are mutually exclusive, determine
+ * which is correct based on whether the pgdat->private is set.
+ */
if (node_arg.nid >= 0)
- node_set_state(nid, N_MEMORY);
+ node_set_state(nid, pgdat_is_private(NODE_DATA(nid)) ?
+ N_MEMORY_PRIVATE : N_MEMORY);
/*
* Check whether we are adding normal memory to the node for the first
* time.
*/
- if (!node_state(nid, N_NORMAL_MEMORY) && zone_idx(zone) <= ZONE_NORMAL)
+ if (!node_is_private(nid) &&
+ !node_state(nid, N_NORMAL_MEMORY) &&
+ zone_idx(zone) <= ZONE_NORMAL)
node_set_state(nid, N_NORMAL_MEMORY);
if (need_zonelists_rebuild)
@@ -1230,8 +1237,11 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
/* reinitialise watermarks and update pcp limits */
init_per_zone_wmark_min();
- kswapd_run(nid);
- kcompactd_run(nid);
+ /* Private nodes opt-out of reclaim/compaction by default */
+ if (!node_is_private(nid)) {
+ kswapd_run(nid);
+ kcompactd_run(nid);
+ }
if (node_arg.nid >= 0)
/* First memory added successfully. Notify consumers. */
@@ -1649,6 +1659,8 @@ EXPORT_SYMBOL_GPL(add_memory);
* @resource_name: Resource name in format "System RAM ($DRIVER)"
* @mhp_flags: Memory hotplug flags
* @online_type: Auto-Online behavior (offline, online, kernel, movable)
+ * @np: driver-owned node_private to online the node as N_MEMORY_PRIVATE, or
+ * NULL for ordinary system RAM
*
* Add special, driver-managed memory to the system as system RAM. Such
* memory is not exposed via the raw firmware-provided memmap as system
@@ -1675,9 +1687,11 @@ EXPORT_SYMBOL_GPL(add_memory);
*/
int __add_memory_driver_managed(int nid, u64 start, u64 size,
const char *resource_name, mhp_t mhp_flags,
- enum mmop online_type)
+ enum mmop online_type, struct node_private *np)
{
+ int real_nid = nid;
struct resource *res;
+ struct memory_group *group;
int rc;
if (!resource_name ||
@@ -1688,6 +1702,19 @@ int __add_memory_driver_managed(int nid, u64 start, u64 size,
if (online_type < MMOP_OFFLINE || online_type > MMOP_ONLINE_MOVABLE)
return -EINVAL;
+ /* Register a private-node owner before adding memory. */
+ if (np) {
+ if (mhp_flags & MHP_NID_IS_MGID) {
+ group = memory_group_find_by_id(nid);
+ if (!group)
+ return -EINVAL;
+ real_nid = group->nid;
+ }
+ rc = node_private_register(real_nid, np);
+ if (rc)
+ return rc;
+ }
+
lock_device_hotplug();
res = register_memory_resource(start, size, resource_name);
@@ -1702,10 +1729,41 @@ int __add_memory_driver_managed(int nid, u64 start, u64 size,
out_unlock:
unlock_device_hotplug();
+ if (rc < 0 && np)
+ node_private_unregister(real_nid);
return rc;
}
EXPORT_SYMBOL_FOR_MODULES(__add_memory_driver_managed, "kmem");
+/**
+ * add_private_memory_driver_managed - add private driver-managed memory
+ * @nid: NUMA node ID where the memory will be added
+ * @start: Start physical address of the memory range
+ * @size: Size of the memory range in bytes
+ * @resource_name: Resource name in format "System RAM ($DRIVER)"
+ * @mhp_flags: Memory hotplug flags
+ * @online_type: Auto-Online behavior (offline, online, kernel, movable)
+ * @np: private node context for the owner - mandatory.
+ *
+ * Add private driver-managed memory. Private node context is required to
+ * avoid the memory being added as a normal N_MEMORY node.
+ *
+ * See __add_memory_driver_managed for more details.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+int add_private_memory_driver_managed(int nid, u64 start, u64 size,
+ const char *resource_name, mhp_t mhp_flags,
+ enum mmop online_type, struct node_private *np)
+{
+ if (!np)
+ return -EINVAL;
+
+ return __add_memory_driver_managed(nid, start, size, resource_name,
+ mhp_flags, online_type, np);
+}
+EXPORT_SYMBOL_GPL(add_private_memory_driver_managed);
+
/**
* add_memory_driver_managed - add driver-managed memory
* @nid: NUMA node ID where the memory will be added
@@ -1726,7 +1784,7 @@ int add_memory_driver_managed(int nid, u64 start, u64 size,
{
return __add_memory_driver_managed(nid, start, size, resource_name,
mhp_flags,
- mhp_get_default_online_type());
+ mhp_get_default_online_type(), NULL);
}
EXPORT_SYMBOL_GPL(add_memory_driver_managed);
@@ -2041,7 +2099,7 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
/*
* Check whether the node will have no present pages after we offline
* 'nr_pages' more. If so, we know that the node will become empty, and
- * so we will clear N_MEMORY for it.
+ * so we will clear N_MEMORY(_PRIVATE) for it.
*/
if (nr_pages >= pgdat->node_present_pages) {
node_arg.nid = node;
@@ -2146,8 +2204,10 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
* Make sure to mark the node as memory-less before rebuilding the zone
* list. Otherwise this node would still appear in the fallback lists.
*/
- if (node_arg.nid >= 0)
+ if (node_arg.nid >= 0) {
node_clear_state(node, N_MEMORY);
+ node_clear_state(node, N_MEMORY_PRIVATE);
+ }
if (!populated_zone(zone)) {
zone_pcp_reset(zone);
build_all_zonelists(NULL);
@@ -2211,6 +2271,15 @@ static int count_memory_range_altmaps_cb(struct memory_block *mem, void *arg)
return 0;
}
+static int collect_memblock_nodes_cb(struct memory_block *mem, void *arg)
+{
+ nodemask_t *nodes = arg;
+
+ if (mem->nid != NUMA_NO_NODE)
+ node_set(mem->nid, *nodes);
+ return 0;
+}
+
static int check_cpu_on_node(int nid)
{
int cpu;
@@ -2474,10 +2543,11 @@ EXPORT_SYMBOL_GPL(offline_and_remove_memory);
int offline_and_remove_memory_ranges(const struct range *ranges,
unsigned int nr_ranges)
{
+ nodemask_t nodes = NODE_MASK_NONE;
unsigned long mb_count = 0;
uint8_t *online_types, *tmp;
unsigned int i;
- int rc = 0;
+ int nid, rc = 0;
if (!ranges || !nr_ranges)
return -EINVAL;
@@ -2505,6 +2575,11 @@ int offline_and_remove_memory_ranges(const struct range *ranges,
lock_device_hotplug();
+ /* Record the nodes for the blocks to drop private-node data after */
+ for (i = 0; i < nr_ranges; i++)
+ walk_memory_blocks(ranges[i].start, range_len(&ranges[i]),
+ &nodes, collect_memblock_nodes_cb);
+
/*
* Phase 1: offline every block in every range. An already-offline
* block folds to success, so out-of-band offlining never blocks unplug.
@@ -2535,6 +2610,10 @@ int offline_and_remove_memory_ranges(const struct range *ranges,
out_unlock:
unlock_device_hotplug();
+ /* Drop private node registration if they are now memoryless */
+ for_each_node_mask(nid, nodes)
+ node_private_unregister(nid);
+
kfree(online_types);
return rc;
}
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 24/36] mm/mempolicy: add in-kernel MPOL_BIND interfaces for drivers/services
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Add and export interfaces to enable modules to build a mempolicy with
(MPOL_BIND | MPOL_F_PRIVATE) pinned to a private NUMA node, so the
drivers can stamp it onto VMAs they control (via vma->vm_policy).
mpol_bind_node() - acts the same as a userland mbind()
mpol_private_bind() - bypasses the CAP_USER_NUMA check.
Export it to modules (private:kmem, bind:kvm).
Widen __mpol_put()'s module export to kmem so the driver can
release the policy.
Adjust mpol_set_nodemask to check for a pre-set MPOL_F_PRIVATE flag
to allow the mpol_private_bind() check to bypass the N_MEMORY filter.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/linux/mempolicy.h | 14 ++++++
mm/mempolicy.c | 102 +++++++++++++++++++++++++++++++++++++-
2 files changed, 114 insertions(+), 2 deletions(-)
diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
index 65c732d440d2f..715951a5b03c1 100644
--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -7,6 +7,7 @@
#define _LINUX_MEMPOLICY_H 1
#include <linux/sched.h>
+#include <linux/err.h>
#include <linux/mmzone.h>
#include <linux/slab.h>
#include <linux/rbtree.h>
@@ -128,6 +129,9 @@ void mpol_free_shared_policy(struct shared_policy *sp);
struct mempolicy *mpol_shared_policy_lookup(struct shared_policy *sp,
pgoff_t idx);
+struct mempolicy *mpol_private_bind(int nid);
+struct mempolicy *mpol_bind_node(int nid);
+
struct mempolicy *get_task_policy(struct task_struct *p);
struct mempolicy *__get_vma_policy(struct vm_area_struct *vma,
unsigned long addr, pgoff_t *ilx);
@@ -226,6 +230,16 @@ mpol_shared_policy_lookup(struct shared_policy *sp, pgoff_t idx)
return NULL;
}
+static inline struct mempolicy *mpol_private_bind(int nid)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline struct mempolicy *mpol_bind_node(int nid)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+
static inline struct mempolicy *get_vma_policy(struct vm_area_struct *vma,
unsigned long addr, int order, pgoff_t *ilx)
{
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index e83c2c7a94c1d..a3ffb09897489 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -406,7 +406,7 @@ static int mpol_new_preferred(struct mempolicy *pol, const nodemask_t *nodes)
static int mpol_set_nodemask(struct mempolicy *pol,
const nodemask_t *nodes, struct nodemask_scratch *nsc)
{
- int ret;
+ int ret, nid;
/*
* Default (pol==NULL) resp. local memory policies are not a
@@ -432,6 +432,18 @@ static int mpol_set_nodemask(struct mempolicy *pol,
else
pol->w.cpuset_mems_allowed = cpuset_current_mems_allowed;
+ /*
+ * Private nodes are not in cpuset.mems, so they're always stripped.
+ * Driver-allocated policies will already have MPOL_F_PRIVATE set,
+ * if that's the case, add back in the requested set of private nodes.
+ */
+ for_each_node_mask(nid, *nodes) {
+ if (!node_is_private(nid))
+ continue;
+ if (pol->flags & MPOL_F_PRIVATE)
+ node_set(nid, nsc->mask2);
+ }
+
/* If any private nodes left in the nodemask - add the private flag */
if (nodes_intersects(nsc->mask2, node_states[N_MEMORY_PRIVATE]))
pol->flags |= MPOL_F_PRIVATE;
@@ -501,7 +513,7 @@ void __mpol_put(struct mempolicy *pol)
*/
kfree_rcu(pol, rcu);
}
-EXPORT_SYMBOL_FOR_MODULES(__mpol_put, "kvm");
+EXPORT_SYMBOL_FOR_MODULES(__mpol_put, "kvm,kmem");
static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes)
{
@@ -1126,6 +1138,92 @@ static long do_set_mempolicy(unsigned short mode, unsigned short flags,
return ret;
}
+/*
+ * Build a refcounted MPOL_BIND policy targeting the single node @nid,
+ * contextualised to the caller's cpuset like a userspace mbind().
+ *
+ * @flags is MPOL_F_PRIVATE for the an explicit in-kernel user, letting
+ * a private node be bound without checking CAP_USER_NUMA. Otherwise,
+ * CAP_USER_NUMA is enforced.
+ *
+ * The caller owns the reference and frees it with mpol_put().
+ */
+static struct mempolicy *__mpol_bind_node(int nid, unsigned short flags)
+{
+ struct mempolicy *pol;
+ nodemask_t nodes;
+ int err;
+
+ NODEMASK_SCRATCH(scratch);
+
+ if (!scratch)
+ return ERR_PTR(-ENOMEM);
+
+ nodes_clear(nodes);
+ node_set(nid, nodes);
+
+ pol = mpol_new(MPOL_BIND, flags, &nodes);
+ if (IS_ERR(pol)) {
+ NODEMASK_SCRATCH_FREE(scratch);
+ return pol;
+ }
+
+ err = mpol_set_nodemask(pol, &nodes, scratch);
+ NODEMASK_SCRATCH_FREE(scratch);
+ if (err) {
+ mpol_put(pol);
+ return ERR_PTR(err);
+ }
+ return pol;
+}
+
+/**
+ * mpol_private_bind - build an MPOL_BIND policy pinned to a private node
+ * @nid: an N_MEMORY_PRIVATE node
+ *
+ * Returns a refcounted mempolicy that binds allocations to @nid with the
+ * private-placement intent (MPOL_F_PRIVATE). This binds to @nid regardless
+ * of the node's CAP_USER_NUMA, providing a privileged way for node-owners
+ * to bind driver/service owned VMAs to the node.
+ *
+ * Like any MPOL_BIND it is relaxable: an unsatisfiable request falls back
+ * rather than failing.
+ *
+ * Must be called while @nid is N_MEMORY_PRIVATE.
+ *
+ * The caller owns the reference and frees it with mpol_put().
+ *
+ * Return: the policy, or an ERR_PTR on failure.
+ */
+struct mempolicy *mpol_private_bind(int nid)
+{
+ if (!node_is_private(nid))
+ return ERR_PTR(-EINVAL);
+ return __mpol_bind_node(nid, MPOL_F_PRIVATE);
+}
+EXPORT_SYMBOL_FOR_MODULES(mpol_private_bind, "kmem");
+
+/**
+ * mpol_bind_node - build an MPOL_BIND policy targeting @nid for in-kernel use
+ * @nid: the node to bind to
+ *
+ * Returns a refcounted MPOL_BIND policy that places allocations on @nid,
+ * contextualised to the caller's cpuset exactly like a userspace mbind().
+ *
+ * This interface should be used by services implenting mempolicy support with
+ * user-provided node bindings. N_MEMORY_PRIVATE node bindings are honored if
+ * the node has CAP_USER_NUMA, otherwise return -EINVAL.
+ *
+ * The caller owns the reference and frees it with mpol_put().
+ *
+ * Return: the policy, or an ERR_PTR on failure.
+ */
+struct mempolicy *mpol_bind_node(int nid)
+{
+ return __mpol_bind_node(nid, 0);
+}
+EXPORT_SYMBOL_FOR_MODULES(mpol_bind_node, "kvm");
+
/*
* Return nodemask for policy for get_mempolicy() query
*
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 23/36] mm/mempolicy: apply policy at the kernel zone for private-node binds
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
apply_policy_zone() raises dynamic_policy_zone to ZONE_MOVABLE when none
of a policy's nodes have non-movable memory, so a bind to movable-only
node does not force kernel-zone allocations onto a zone that cannot
satisfy them - resulting retries forever (livelock).
The current code uses N_HIGH_MEMORY to denote eligibility, but private
nodes are deliberately not in N_HIGH_MEMORY nor N_NORMAL_MEMORY. This
means a private node onlined into ZONE_NORMAL is judged as movable-only.
This means a policy set with set_mempolicy() binding a private node with
ZONE_NORMAL memory will shunt otherwise eligible allocations (like the
task's page tables) back to N_MEMORY nodes.
This also becomes relevant for in-kernel users that want non-movable
memory (such as KVM placing pinned VM memory solely on a private node).
Add policy_private_has_kernel_zone() to consult the actual zones of
the policy's private nodes and keep the policy applied at lower zones
when a kernel-zoned private node can satisfy the allocation.
Only a mempolicy binding a private node pays the zone scan, all other
mempolicies preserve fast-path performance.
Additionally: update the comments in apply_policy_zone for clarity,
because this particular policy mechanism is very confusing.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
mm/mempolicy.c | 48 ++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 42 insertions(+), 6 deletions(-)
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 90110e9761122..e83c2c7a94c1d 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2091,6 +2091,34 @@ bool vma_policy_mof(struct vm_area_struct *vma)
return mof;
}
+/*
+ * true if any policy *private* node can satisfy a !__GFP_MOVABLE allocation.
+ * Policies without private nodes (~MPOL_F_PRIVATE) always return false.
+ *
+ * This provides a way for otherwise-bound kernel allocations to make their
+ * way onto private nodes with ZONE_NORMAL memory without having to audit
+ * every caller. The cost is a zone scan for private nodes in the mask.
+ */
+static bool policy_private_has_kernel_zone(const struct mempolicy *pol)
+{
+ int nid;
+
+ if (!(pol->flags & MPOL_F_PRIVATE))
+ return false;
+
+ for_each_node_mask(nid, pol->nodes) {
+ pg_data_t *pgdat = NODE_DATA(nid);
+ enum zone_type zt;
+
+ if (!node_is_private(nid))
+ continue;
+ for (zt = ZONE_NORMAL; zt < ZONE_MOVABLE; zt++)
+ if (managed_zone(&pgdat->node_zones[zt]))
+ return true;
+ }
+ return false;
+}
+
bool apply_policy_zone(struct mempolicy *policy, enum zone_type zone)
{
enum zone_type dynamic_policy_zone = policy_zone;
@@ -2098,14 +2126,22 @@ bool apply_policy_zone(struct mempolicy *policy, enum zone_type zone)
BUG_ON(dynamic_policy_zone == ZONE_MOVABLE);
/*
- * if policy->nodes has movable memory only,
- * we apply policy when gfp_zone(gfp) = ZONE_MOVABLE only.
+ * dynamic_policy_zone is the lowest zone this policy is enforced for,
+ * allocations below it ignore the nodemask and fall back freely.
+ *
+ * If all policy-nodes are movable-only (all ZONE_MOVABLE), raise the
+ * dynamic_policy_zone to ZONE_MOVABLE so that only movable allocations
+ * stay bound - otherwise this would force kernel-zone allocations
+ * (e.g. page tables) onto a zone that cannot satisfy them. This
+ * results in allocation failure and retry loop (livelock).
*
- * policy->nodes is intersect with node_states[N_MEMORY].
- * so if the following test fails, it implies
- * policy->nodes has movable memory only.
+ * N_HIGH_MEMORY tells us "has a non-movable zone" for ordinary nodes.
+ * Private nodes are deliberately excluded from N_HIGH_MEMORY, so we
+ * need to check their actual zones - a kernel-zoned private node can
+ * hold the allocation and must keep the policy applied.
*/
- if (!nodes_intersects(policy->nodes, node_states[N_HIGH_MEMORY]))
+ if (!nodes_intersects(policy->nodes, node_states[N_HIGH_MEMORY]) &&
+ !policy_private_has_kernel_zone(policy))
dynamic_policy_zone = ZONE_MOVABLE;
return zone >= dynamic_policy_zone;
--
2.53.0-Meta
^ permalink raw reply related
* [PATCH v5 22/36] mm/mempolicy: add MPOL_F_PRIVATE and zonelist selection
From: Gregory Price @ 2026-07-20 19:34 UTC (permalink / raw)
To: linux-mm
Cc: Zhigang.Luo, arun.george, balbirs, brendan.jackman, yuzenghui,
apopple, alucerop, matthew.brost, akpm, david, ljs, liam, vbabka,
rppt, surenb, mhocko, corbet, skhan, gregkh, rafael, dakr, djbw,
vishal.l.verma, dave.jiang, alison.schofield, osandov, jannh,
pfalcato, jackmanb, hannes, ziy, pbonzini, osalvador,
joshua.hahnjy, rakie.kim, byungchul, gourry, ying.huang, kasong,
qi.zheng, shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, longman, ridong.chen, tj, mkoutny, sj, jgg,
jhubbard, peterx, baolin.wang, npache, ryan.roberts, dev.jain,
lance.yang, usama.arif, xu.xin16, chengming.zhou, roman.gushchin,
muchun.song, linux-kernel, linux-doc, driver-core, nvdimm,
linux-cxl, linux-debuggers, linux-fsdevel, kvm, cgroups, damon,
linux-kselftest, kernel-team
In-Reply-To: <20260720193431.3841992-1-gourry@gourry.net>
Add MPOL_F_PRIVATE, the internal flag noting that a policy contains
a nodemask with a private node, and making private node memory
reachable via standard mempolicies.
Plumb zonelist selection into the mpol allocator interfaces via the
allocator's alloc_flags, using the alloc_flags-carrying page_alloc
interfaces.
mpol_alloc_flags() maps MPOL_F_PRIVATE to the alloc_flags an
allocation uses:
ALLOC_DEFAULT - normal allocation
ALLOC_ZONELIST_PRIVATE - private node allocation
With this, a VMA with (mpol->flags & MPOL_F_PRIVATE) can successfully
services faults like any other mbind() from the private node:
buf = mmap(..., MAP_ANON);
mbind(buf, ..., {private_node});
buf[0] = 0xdeadbeef; // Page faulted from private node memory
Like any other bind, the nodemask relaxes if it is unsatisfiable.
For example: apply_policy_zone() allows an unmovable allocation
targeting the VMA to fallback to a viable node instead of failing.
cpuset rebinding does not affect private nodes - the nodes in the
original mask are preserved, and we enforce two remap rules:
1) never allow N_MEMORY nodes to remap to N_MEMORY_PRIVATE
2) never remap N_MEMORY_PRIVATE nodes at all (they stay in place)
In the case of an empty nodemask as a result of rebind, revert to
cpuset - which never includes N_MEMORY_PRIVATE and always guarantees
at least one N_MEMORY node.
As of this patch, nothing can actually sets MPOL_F_PRIVATE, so this
is a no-op that simply adds the plumbing throughout mempolicy.
Signed-off-by: Gregory Price <gourry@gourry.net>
---
include/uapi/linux/mempolicy.h | 1 +
mm/mempolicy.c | 79 +++++++++++++++++++++++-----------
2 files changed, 55 insertions(+), 25 deletions(-)
diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
index 7f6fc9599693b..87af18c84b947 100644
--- a/include/uapi/linux/mempolicy.h
+++ b/include/uapi/linux/mempolicy.h
@@ -67,6 +67,7 @@ enum mempolicy_mode {
#define MPOL_F_SHARED (1 << 0) /* identify shared policies */
#define MPOL_F_MOF (1 << 3) /* this policy wants migrate on fault */
#define MPOL_F_MORON (1 << 4) /* Migrate On protnone Reference On Node */
+#define MPOL_F_PRIVATE (1 << 5) /* nodemask contains private nodes */
/*
* Enabling zone reclaim means the page allocator will attempt to fulfill
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 2b76c57a460c9..90110e9761122 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -432,6 +432,10 @@ static int mpol_set_nodemask(struct mempolicy *pol,
else
pol->w.cpuset_mems_allowed = cpuset_current_mems_allowed;
+ /* If any private nodes left in the nodemask - add the private flag */
+ if (nodes_intersects(nsc->mask2, node_states[N_MEMORY_PRIVATE]))
+ pol->flags |= MPOL_F_PRIVATE;
+
ret = mpol_ops[pol->mode].create(pol, &nsc->mask2);
return ret;
}
@@ -505,22 +509,33 @@ static void mpol_rebind_default(struct mempolicy *pol, const nodemask_t *nodes)
static void mpol_rebind_nodemask(struct mempolicy *pol, const nodemask_t *nodes)
{
- nodemask_t tmp;
+ nodemask_t tmp, priv;
+
+ /* preserve online private nodes to re-add later */
+ nodes_and(priv, pol->nodes, node_states[N_MEMORY_PRIVATE]);
if (pol->flags & MPOL_F_STATIC_NODES)
nodes_and(tmp, pol->w.user_nodemask, *nodes);
- else if (pol->flags & MPOL_F_RELATIVE_NODES)
- mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes);
- else {
+ else if (pol->flags & MPOL_F_RELATIVE_NODES) {
+ /* fold only the public part: a private node must not take a slot */
+ nodes_and(tmp, pol->w.user_nodemask, node_states[N_MEMORY]);
+ mpol_relative_nodemask(&tmp, &tmp, nodes);
+ } else {
nodes_remap(tmp, pol->nodes, pol->w.cpuset_mems_allowed,
*nodes);
pol->w.cpuset_mems_allowed = *nodes;
}
- if (nodes_empty(tmp))
+ /* private nodes are identity-mapped during remap, drop them here */
+ nodes_and(tmp, tmp, node_states[N_MEMORY]);
+ if (nodes_empty(tmp) && nodes_empty(priv))
tmp = *nodes;
- pol->nodes = tmp;
+ /* If any online private nodes remain, add them back */
+ nodes_or(pol->nodes, tmp, priv);
+ /* If no online private nodes remain, strip the private flag */
+ if (nodes_empty(priv))
+ pol->flags &= ~MPOL_F_PRIVATE;
}
static void mpol_rebind_preferred(struct mempolicy *pol,
@@ -2411,7 +2426,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk,
}
static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
- int nid, nodemask_t *nodemask)
+ int nid, nodemask_t *nodemask,
+ unsigned int aflags)
{
struct page *page;
gfp_t preferred_gfp;
@@ -2425,14 +2441,21 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
preferred_gfp = gfp | __GFP_NOWARN;
preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
page = __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask,
- ALLOC_DEFAULT);
+ aflags);
if (!page)
page = __alloc_frozen_pages_noprof(gfp, order, nid, NULL,
- ALLOC_DEFAULT);
+ aflags);
return page;
}
+/* A private policy allocates from the private zonelist. */
+static inline unsigned int mpol_alloc_flags(struct mempolicy *pol)
+{
+ return (pol->flags & MPOL_F_PRIVATE) ? ALLOC_ZONELIST_PRIVATE :
+ ALLOC_DEFAULT;
+}
+
/**
* alloc_pages_mpol - Allocate pages according to NUMA mempolicy.
* @gfp: GFP flags.
@@ -2448,11 +2471,13 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
{
nodemask_t *nodemask;
struct page *page;
+ unsigned int aflags = mpol_alloc_flags(pol);
nodemask = policy_nodemask(gfp, pol, ilx, &nid);
if (pol->mode == MPOL_PREFERRED_MANY)
- return alloc_pages_preferred_many(gfp, order, nid, nodemask);
+ return alloc_pages_preferred_many(gfp, order, nid, nodemask,
+ aflags);
if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
/* filter "hugepage" allocation, unless from alloc_pages() */
@@ -2476,7 +2501,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
*/
page = __alloc_frozen_pages_noprof(
gfp | __GFP_THISNODE | __GFP_NORETRY, order,
- nid, NULL, ALLOC_DEFAULT);
+ nid, NULL, aflags);
if (page || !(gfp & __GFP_DIRECT_RECLAIM))
return page;
/*
@@ -2488,7 +2513,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
}
}
- page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, ALLOC_DEFAULT);
+ page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, aflags);
if (unlikely(pol->mode == MPOL_INTERLEAVE ||
pol->mode == MPOL_WEIGHTED_INTERLEAVE) && page) {
@@ -2597,6 +2622,7 @@ static unsigned long alloc_pages_bulk_interleave(gfp_t gfp,
struct mempolicy *pol, unsigned long nr_pages,
struct page **page_array)
{
+ unsigned int aflags = mpol_alloc_flags(pol);
int nodes;
unsigned long nr_pages_per_node;
int delta;
@@ -2610,14 +2636,14 @@ static unsigned long alloc_pages_bulk_interleave(gfp_t gfp,
for (i = 0; i < nodes; i++) {
if (delta) {
- nr_allocated = alloc_pages_bulk_noprof(gfp,
- interleave_nodes(pol), NULL,
+ nr_allocated = __alloc_pages_bulk_noprof(gfp,
+ aflags, interleave_nodes(pol), NULL,
nr_pages_per_node + 1,
page_array);
delta--;
} else {
- nr_allocated = alloc_pages_bulk_noprof(gfp,
- interleave_nodes(pol), NULL,
+ nr_allocated = __alloc_pages_bulk_noprof(gfp,
+ aflags, interleave_nodes(pol), NULL,
nr_pages_per_node, page_array);
}
@@ -2632,6 +2658,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
struct mempolicy *pol, unsigned long nr_pages,
struct page **page_array)
{
+ unsigned int aflags = mpol_alloc_flags(pol);
struct weighted_interleave_state *state;
struct task_struct *me = current;
unsigned int cpuset_mems_cookie;
@@ -2667,8 +2694,8 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
weight = me->il_weight;
if (weight && node_isset(node, nodes)) {
node_pages = min(rem_pages, weight);
- nr_allocated = __alloc_pages_bulk(gfp, node, NULL, node_pages,
- page_array);
+ nr_allocated = __alloc_pages_bulk_noprof(gfp, aflags, node,
+ NULL, node_pages, page_array);
page_array += nr_allocated;
total_allocated += nr_allocated;
/* if that's all the pages, no need to interleave */
@@ -2732,8 +2759,8 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
/* node_pages can be 0 if an allocation fails and rounds == 0 */
if (!node_pages)
break;
- nr_allocated = __alloc_pages_bulk(gfp, node, NULL, node_pages,
- page_array);
+ nr_allocated = __alloc_pages_bulk_noprof(gfp, aflags, node,
+ NULL, node_pages, page_array);
page_array += nr_allocated;
total_allocated += nr_allocated;
if (total_allocated == nr_pages)
@@ -2750,17 +2777,19 @@ static unsigned long alloc_pages_bulk_preferred_many(gfp_t gfp, int nid,
struct mempolicy *pol, unsigned long nr_pages,
struct page **page_array)
{
+ unsigned int aflags = mpol_alloc_flags(pol);
gfp_t preferred_gfp;
unsigned long nr_allocated = 0;
preferred_gfp = gfp | __GFP_NOWARN;
preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
- nr_allocated = alloc_pages_bulk_noprof(preferred_gfp, nid, &pol->nodes,
- nr_pages, page_array);
+ nr_allocated = __alloc_pages_bulk_noprof(preferred_gfp, aflags,
+ nid, &pol->nodes, nr_pages, page_array);
if (nr_allocated < nr_pages)
- nr_allocated += alloc_pages_bulk_noprof(gfp, numa_node_id(), NULL,
+ nr_allocated += __alloc_pages_bulk_noprof(gfp, aflags,
+ numa_node_id(), NULL,
nr_pages - nr_allocated,
page_array + nr_allocated);
return nr_allocated;
@@ -2796,8 +2825,8 @@ unsigned long alloc_pages_bulk_mempolicy_noprof(gfp_t gfp,
nid = numa_node_id();
nodemask = policy_nodemask(gfp, pol, NO_INTERLEAVE_INDEX, &nid);
- return alloc_pages_bulk_noprof(gfp, nid, nodemask,
- nr_pages, page_array);
+ return __alloc_pages_bulk_noprof(gfp, mpol_alloc_flags(pol), nid,
+ nodemask, nr_pages, page_array);
}
int vma_dup_policy(struct vm_area_struct *src, struct vm_area_struct *dst)
--
2.53.0-Meta
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox