* Re: [PATCH net-next v2 4/7] net: mana: validate hardware-supplied values in the HWC RX path
From: sashiko-bot @ 2026-07-22 23:46 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
In-Reply-To: <20260721234339.1476932-5-longli@microsoft.com>
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] This problem wasn't introduced by this patch, but a race condition between the timeout path in `mana_hwc_send_request` and the hardware response handler leads to use-after-free stack corruption or tag reuse.
--
--- Patch [4]: [PATCH net-next v2 4/7] net: mana: validate hardware-supplied values in the HWC RX path ---
commit 20acf32024b20aecf06a023b18198388ad46e67f
Author: Long Li <longli@microsoft.com>
net: mana: validate hardware-supplied values in the HWC RX path
This commit adds validation to the hardware-supplied values in the MANA HWC
RX path. It prevents out-of-bounds access and stale response processing by
checking the SGE pointer, address, and response length against posted buffers.
> diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
> index 2239fdeda57cb..515c45c4a890f 100644
> --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
> +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
> @@ -83,6 +83,17 @@ static void mana_hwc_handle_resp(struct hw_channel_context *hwc, u32 resp_len,
> struct hwc_caller_ctx *ctx;
> int err;
>
> + /* Validate msg_id is in range before using it to index bitmap
> + * and caller_ctx array. Malicious firmware could send
> + * out-of-range msg_id causing out-of-bounds access.
> + */
> + if (msg_id >= hwc->num_inflight_msg) {
> + dev_err(hwc->dev, "hwc_rx: msg_id %u >= max %u\n",
> + msg_id, hwc->num_inflight_msg);
> + mana_hwc_post_rx_wqe(hwc->rxq, rx_req);
> + return;
> + }
> +
> if (!test_bit(msg_id, hwc->inflight_msg_res.map)) {
> dev_err(hwc->dev, "hwc_rx: invalid msg_id = %u\n", msg_id);
> mana_hwc_post_rx_wqe(hwc->rxq, rx_req);
> @@ -90,6 +101,18 @@ static void mana_hwc_handle_resp(struct hw_channel_context *hwc, u32 resp_len,
> }
>
> ctx = hwc->caller_ctx + msg_id;
> +
> + /* Reject responses larger than the RX DMA buffer — the SGE
> + * limits what hardware can DMA, so an oversized resp_len
> + * indicates a firmware bug. Fail rather than silently
> + * truncating.
> + */
> + if (resp_len > rx_req->buf_len) {
> + dev_err(hwc->dev, "HWC RX: resp_len %u > buf_len %u\n",
> + resp_len, rx_req->buf_len);
> + resp_len = 0;
> + }
> +
> err = mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len);
[Severity: Critical]
This isn't a bug introduced by this patch, but is it possible for a delayed
hardware response to corrupt the stack here?
If a request times out in mana_hwc_send_request(), wait_for_completion_timeout()
expires and the function jumps to the "out" label where it calls
mana_hwc_put_msg_index(hwc, msg_id). This releases the msg_id for reuse without
clearing ctx->output_buf (which points to a buffer on the caller's stack).
If another thread subsequently allocates this same msg_id, it will set the
in-flight bit. If the delayed hardware response for the original request arrives
at this exact moment, mana_hwc_handle_resp() will pass the test_bit() check
above.
It will then proceed to copy the response data into ctx->output_buf:
drivers/net/ethernet/microsoft/mana/hw_channel.c:mana_hwc_handle_resp() {
...
if (!err && ctx->output_buf) {
ctx->status_code = resp_msg->status;
memcpy(ctx->output_buf, resp_msg, resp_len);
ctx->error = 0;
}
...
}
If the new thread hasn't yet updated ctx->output_buf, wouldn't this memcpy()
write DMA data directly into the timed-out caller's freed stack frame, causing
a use-after-free stack corruption?
Alternatively, if the new thread has already updated ctx->output_buf, could this
prematurely complete the new sender's request with stale data from the delayed
response?
[ ... ]
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260721234339.1476932-1-longli@microsoft.com?part=4
^ permalink raw reply
* Re: [PATCH net-next v2 5/7] net: mana: fix HWC teardown safety with setup_active flag and destroy ordering
From: sashiko-bot @ 2026-07-22 23:46 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
In-Reply-To: <20260721234339.1476932-6-longli@microsoft.com>
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [High] Improper PCI device reset via `pcie_flr()` leaves PCI core state out of sync.
Pre-existing issues:
- [Critical] Use-After-Free in EQ interrupt handler because the CQ is destroyed before the EQ is deregistered.
- [Critical] Tag reuse causing logical corruption when HWC commands time out.
- [High] Race condition leading to Use-After-Free on module unbind due to un-cancelled `mana_serv_work`.
--
--- Patch [5]: [PATCH net-next v2 5/7] net: mana: fix HWC teardown safety with setup_active flag and destroy ordering ---
commit 7a0e58ab3c693baa720b6e21219cbe04fb7e9fcf
Author: Long Li <longli@microsoft.com>
net: mana: fix HWC teardown safety with setup_active flag and destroy ordering
This patch adds a setup_active flag to ensure safe hardware channel teardown
on initialization failures. It also reorders queue destruction to prevent
DMA-after-free races during teardown.
> diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
> index 515c45c4a890f..5bc873d548d58 100644
> --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
> +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
[ ... ]
> @@ -902,11 +926,58 @@ void mana_hwc_destroy_channel(struct gdma_context *gc)
[ ... ]
> + /* On systems without IOMMU, freeing DMA memory with
> + * active hardware MST mappings risks memory corruption.
> + * Issue FLR to force-reset the device and invalidate
> + * all hardware state including MST entries.
> + */
> + td_err = pcie_flr(to_pci_dev(gc->dev));
[Severity: High]
Does using pcie_flr() here instead of pci_reset_function() leave the
Linux PCI core completely unaware of the reset?
Since pcie_flr() does not save and restore the PCI configuration space,
such as BARs, MSI-X vectors, and Bus Master enable, could this leave the
hardware's PCI configuration wiped and out of sync with the kernel's
cached state, resulting in a wedged device?
> + if (td_err) {
[ ... ]
> + /* Tear down the HWC CQ object first — mana_hwc_destroy_cq()
> + * both unpublishes the CQ from cq_table (+synchronize_rcu) and
> + * deregisters the HWC EQ from the interrupt handler list (via
> + * mana_gd_deregister_irq + synchronize_rcu), guaranteeing no
> + * interrupt handler can access RQ/TXQ buffers after this point.
> + */
> + if (hwc->cq)
> + mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq);
[Severity: Critical]
This is a pre-existing issue, but does destroying the CQ before the EQ
inside mana_hwc_destroy_cq() create a use-after-free window?
In mana_hwc_destroy_cq(), the CQ structure is freed first:
mana_hwc_destroy_cq() {
if (hwc_cq->gdma_cq)
mana_gd_destroy_queue(gc, hwc_cq->gdma_cq);
...
}
If the hardware posts an initialization event before the EQ is
subsequently destroyed, could the interrupt handler execute:
mana_hwc_init_event_handler() {
...
case HWC_INIT_DATA_CQID:
hwc->cq->gdma_cq->id = val;
break;
}
and write directly into the newly freed gdma_cq structure memory?
[ ... ]
[Severity: High]
This isn't a bug introduced by this patch, but is there a race condition
leading to a use-after-free on module unbind due to un-cancelled service
work?
If a hardware reset event triggers mana_schedule_serv_work(), it queues
work on the global system_wq. If the driver is unbound via sysfs,
mana_gd_remove() frees the gdma_context via vfree(gc) without cancelling
this pending work.
Could the work then execute concurrently, calling mana_serv_reset() which
fetches the freed context:
mana_serv_reset() {
struct gdma_context *gc = pci_get_drvdata(pdev);
struct hw_channel_context *hwc = gc->hwc.driver_data;
...
}
and dereference the freed memory?
[Severity: Critical]
This is a pre-existing issue, but can tag reuse cause logical data
corruption when hardware channel commands time out?
When wait_for_completion_timeout() times out in mana_hwc_send_request(),
the function calls mana_hwc_put_msg_index() to free the msg_id for
immediate reuse.
Since the hardware is not reset, the timed-out command is still pending.
If another thread sends a new command and allocates the same msg_id,
what happens when the hardware eventually responds to the original
command?
Could mana_hwc_handle_resp() match this late response to the new command:
mana_hwc_handle_resp() {
ctx = hwc->caller_ctx + msg_id;
...
memcpy(ctx->output_buf, resp_msg, resp_len);
complete(&ctx->comp_event);
}
and complete it with stale data?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260721234339.1476932-1-longli@microsoft.com?part=5
^ permalink raw reply
* Re: [PATCH net-next v2 1/7] net: mana: RCU-protect gc->cq_table lookups against concurrent CQ destroy
From: sashiko-bot @ 2026-07-22 23:46 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
In-Reply-To: <20260721234339.1476932-2-longli@microsoft.com>
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [Critical] Concurrent QP creations sharing the same CQ cause a data race on `send_cq->queue.id`, leaking CQ table entries and triggering a double free of `gdma_cq` in the error path.
- [High] The EQ event handler `mana_gd_process_eqe()` triggers a false positive `WARN_ON_ONCE(!cq)` when encountering stale EQEs for a recently destroyed CQ during normal teardown.
- [High] `mana_gd_destroy_queue()` frees EQ queues immediately after `list_del_rcu()` without waiting for an RCU grace period, causing a use-after-free if an interrupt handler is concurrently traversing the EQ list.
--
--- Patch [1]: [PATCH net-next v2 1/7] net: mana: RCU-protect gc->cq_table lookups against concurrent CQ destroy ---
commit 4df31c26faa213971de40d92e7d0a48ab9e5d300
Author: Long Li <longli@microsoft.com>
net: mana: RCU-protect gc->cq_table lookups against concurrent CQ destroy
This commit fixes a use-after-free race between the EQ interrupt handler and
CQ teardown by placing the cq_table under RCU protection. It annotates the
table, updates readers to use rcu_dereference, and ensures removers clear
the slot and call synchronize_rcu() before freeing the CQ.
> diff --git a/drivers/infiniband/hw/mana/cq.c b/drivers/infiniband/hw/mana/cq.c
> --- a/drivers/infiniband/hw/mana/cq.c
> +++ b/drivers/infiniband/hw/mana/cq.c
[ ... ]
> @@ -149,23 +157,49 @@ int mana_ib_install_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq)
> void mana_ib_remove_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq)
> {
[ ... ]
> + /* Removers for a given CQ are serialized by the IB core, so the slot
> + * is read and cleared without rcu_read_lock() or atomicity: a CQ is
> + * never torn down while a live QP references it (cq->usecnt), nor
> + * while the QP-create that installed the entry is still running (that
> + * create holds a reference on the CQ uobject across its error path,
> + * before usecnt is taken). Any double-remove is therefore sequential
> + * -- the later caller sees the NULL stored below and returns.
> + */
> + gdma_cq = rcu_dereference_protected(cq_table[cq->queue.id], true);
> + if (!gdma_cq)
> + return; /* already removed by a prior teardown path */
> +
> + rcu_assign_pointer(cq_table[cq->queue.id], NULL);
> +
> + /* Wait for in-flight EQ handlers that may have loaded the old
> + * pointer via rcu_dereference() to finish before freeing.
> + */
> + synchronize_rcu();
> + kfree(gdma_cq);
> }
[Severity: Critical]
This is a pre-existing issue, but does this actually prevent double frees
during concurrent QP creation failures?
If multiple concurrent ib_create_qp() calls share the same send_cq, the IB
core takes a read lock. In mana_ib_create_qp_raw(), the shared
send_cq->queue.id is locklessly overwritten:
drivers/infiniband/hw/mana/qp.c:mana_ib_create_qp_raw() {
...
qp->raw_sq.id = wq_spec.queue_index;
send_cq->queue.id = cq_spec.queue_index;
...
}
If both threads fail after this point (e.g., due to an invalid user pointer),
they both jump to the error path and call mana_ib_remove_cq_cb().
Since the removers are not serialized in this error path, couldn't they both
read the same gdma_cq pointer, both clear it, wait for synchronize_rcu(),
and then both call kfree(gdma_cq)?
> diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
> +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
[ ... ]
> @@ -772,10 +773,11 @@ static void mana_gd_process_eqe(struct gdma_queue *eq)
> switch (type) {
> case GDMA_EQE_COMPLETION:
> cq_id = eqe->details[0] & 0xFFFFFF;
> - if (WARN_ON_ONCE(cq_id >= gc->max_num_cqs))
> + cq_table = rcu_dereference(gc->cq_table);
> + if (WARN_ON_ONCE(cq_id >= gc->max_num_cqs || !cq_table))
> break;
>
> - cq = gc->cq_table[cq_id];
> + cq = rcu_dereference(cq_table[cq_id]);
> if (WARN_ON_ONCE(!cq || cq->type != GDMA_CQ || cq->id != cq_id))
> break;
[Severity: High]
This isn't a bug introduced by this patch, but could this WARN_ON_ONCE
trigger as a false positive during normal teardown?
During interface teardown, mana_deinit_cq() clears cq_table[id] and waits
for an RCU grace period, but the associated EQ remains active until
mana_destroy_eq() is called later.
If the EQ already contains a completion event for the destroyed CQ, the
interrupt handler will process the stale EQE, read a NULL pointer for cq,
and hit this warning.
Since panic_on_warn is common in cloud environments, could this crash the
system during a standard interface teardown?
[ ... ]
> @@ -1082,15 +1084,28 @@ static void mana_gd_create_cq(const struct gdma_queue_spec *spec,
> static void mana_gd_destroy_cq(struct gdma_context *gc,
> struct gdma_queue *queue)
> {
[ ... ]
> + /* Wait for in-flight EQ handlers that may have loaded the old
> + * pointer via rcu_dereference() to finish before the caller
> + * frees the CQ memory.
> + */
> + synchronize_rcu();
> }
[Severity: High]
This is a pre-existing issue, but does mana_gd_destroy_queue() need a similar
synchronize_rcu() to prevent a use-after-free?
While mana_gd_intr() traverses the EQ list under RCU:
drivers/net/ethernet/microsoft/mana/gdma_main.c:mana_gd_intr() {
rcu_read_lock();
list_for_each_entry_rcu(eq, &gic->eq_list, entry)
mana_gd_process_eqe(eq);
rcu_read_unlock();
}
The teardown path removes the queue via list_del_rcu() but then immediately
frees the memory without waiting for a grace period:
drivers/net/ethernet/microsoft/mana/gdma_main.c:mana_gd_destroy_queue() {
...
mana_gd_destroy_dma_region(gc, gmi->dma_region_handle);
mana_gd_free_memory(gmi);
kfree(queue);
}
Could a concurrent reader in mana_gd_intr() dereference the freed EQ if
the memory is released before the RCU grace period ends?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260721234339.1476932-1-longli@microsoft.com?part=1
^ permalink raw reply
* Re: [PATCH net-next v2 3/7] net: mana: free HWC comp_buf after destroying the EQ
From: sashiko-bot @ 2026-07-22 23:46 UTC (permalink / raw)
To: Long Li; +Cc: linux-hyperv
In-Reply-To: <20260721234339.1476932-4-longli@microsoft.com>
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
Pre-existing issues:
- [High] Use-After-Free of WQs during HWC channel destruction because WQs are destroyed before the EQ and CQ.
- [High] Immediate reuse of `msg_id` after a timeout leads to cross-request stack corruption.
- [High] Pending `mana_serv_work` items on the global workqueue are not canceled during device removal, causing a Use-After-Free.
- [High] Missing bounds check on untrusted `inline_oob_size_div4` allows out-of-bounds memory read.
- [High] Missing length check on `tx_oob_data_size` allows spoofed completions via stale DMA buffer data.
- [Medium] Destroying the CQ before the EQ triggers a `WARN_ON_ONCE(!cq)` splat if an EQ interrupt fires during the race window.
--
--- Patch [3]: [PATCH net-next v2 3/7] net: mana: free HWC comp_buf after destroying the EQ ---
commit 6326f43efebe30510f1377a79fcf8a92f5abb2b7
Author: Long Li <longli@microsoft.com>
net: mana: free HWC comp_buf after destroying the EQ
This commit changes mana_hwc_destroy_cq() to free hwc_cq->comp_buf after
destroying the CQ and EQ. This prevents a use-after-free where a late EQ
interrupt handler could touch comp_buf while the EQ was still registered.
> diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c
> index 3f011ebbe7b3e..2239fdeda57cb 100644
> --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c
> +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c
[ ... ]
> @@ -384,14 +384,20 @@ static void mana_hwc_comp_event(void *ctx, struct gdma_queue *q_self)
>
> static void mana_hwc_destroy_cq(struct gdma_context *gc, struct hwc_cq *hwc_cq)
> {
> - kfree(hwc_cq->comp_buf);
> -
> if (hwc_cq->gdma_cq)
> mana_gd_destroy_queue(gc, hwc_cq->gdma_cq);
[Severity: Medium]
This isn't a bug introduced by this patch, but destroying the CQ before the
EQ appears to leave a race window for EQ interrupts. If a hardware interrupt
fires for the EQ, could mana_gd_process_eqe() read a completion event, look
up the CQ ID in the table, find NULL, and trigger the WARN_ON_ONCE(!cq) splat?
>
> + /* comp_buf is reached only by mana_hwc_comp_event(), which the
[ ... ]
> if (hwc_cq->gdma_eq)
> mana_gd_destroy_queue(gc, hwc_cq->gdma_eq);
>
> + kfree(hwc_cq->comp_buf);
> kfree(hwc_cq);
> }
I also noticed some pre-existing issues in related code:
> drivers/net/ethernet/microsoft/mana/hw_channel.c:mana_hwc_destroy_channel()
>
> if (hwc->txq)
> mana_hwc_destroy_wq(hwc, hwc->txq);
>
> if (hwc->rxq)
> mana_hwc_destroy_wq(hwc, hwc->rxq);
>
> if (hwc->cq)
> mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq);
[Severity: High]
This is a pre-existing issue, but WQs are destroyed and freed here before
the CQ/EQ. Since the EQ and CQ are not yet destroyed and their interrupts
are still active, could an in-flight EQ interrupt process a late completion,
call mana_hwc_rx_event_handler(), and dereference hwc->rxq which points to
freed memory?
> drivers/net/ethernet/microsoft/mana/hw_channel.c:mana_hwc_send_request()
>
> if (!wait_for_completion_timeout(&ctx->comp_event,
> (msecs_to_jiffies(hwc->hwc_timeout)))) {
[ ... ]
> err = -ETIMEDOUT;
> goto out;
> }
> out:
> mana_hwc_put_msg_index(hwc, msg_id);
[Severity: High]
This is a pre-existing issue, but does the timeout path prematurely release
the msg_id without cancelling the request in hardware? If a second thread
allocates the same msg_id and the hardware sends a late response for the
first request, could mana_hwc_handle_resp() copy the response data into
the second thread's stack buffer?
> drivers/net/ethernet/microsoft/mana/hw_channel.c:mana_hwc_rx_event_handler()
>
> sge = (struct gdma_sge *)(wqe + 8 + dma_oob->inline_oob_size_div4 * 4);
>
> /* Select the RX work request for virtual address and for reposting. */
> rq_base_addr = hwc_rxq->msg_buf->mem_info.dma_handle;
> rx_req_idx = (sge->address - rq_base_addr) / hwc->max_resp_msg_size;
[Severity: High]
This is a pre-existing issue, but is there a missing bounds check on the
untrusted inline_oob_size_div4 before it is used to calculate the sge pointer?
If a malicious hypervisor supplies an arbitrarily large value, could this
lead to an out-of-bounds memory read when sge->address is dereferenced?
> drivers/net/ethernet/microsoft/mana/hw_channel.c:mana_hwc_rx_event_handler()
>
> resp = (struct gdma_resp_hdr *)rx_req->buf_va;
>
> /* Read msg_id once from DMA buffer to prevent TOCTOU:
> * DMA memory is shared/unencrypted in CVMs - host can
> * modify it between reads.
> */
> msg_id = READ_ONCE(resp->response.hwc_msg_id);
> if (msg_id >= hwc->num_inflight_msg) {
[ ... ]
> mana_hwc_handle_resp(hwc, rx_oob->tx_oob_data_size, rx_req, msg_id);
[Severity: High]
This is a pre-existing issue, but does this code read msg_id from the DMA
buffer before verifying that the response size (rx_oob->tx_oob_data_size)
is large enough to contain it? If the hardware sends a small completion,
could the buffer retain stale data containing an old msg_id, causing
mana_hwc_handle_resp() to route the response to an active, unrelated request?
> drivers/net/ethernet/microsoft/mana/gdma_main.c:mana_schedule_serv_work()
>
> INIT_WORK(&mns_wk->serv_work, mana_serv_func);
> schedule_work(&mns_wk->serv_work);
[Severity: High]
This is a pre-existing issue, but the work item is queued on the global
workqueue here. During device unbind, mana_gd_remove() frees the gc
structure via vfree(gc) but does not track or cancel pending work items.
When the delayed work item executes, could mana_serv_func() retrieve the
stale gc pointer via pci_get_drvdata(pdev) and dereference it?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260721234339.1476932-1-longli@microsoft.com?part=3
^ permalink raw reply
* Re: [PATCH v2 0/2] Drivers: hv: vmbus: Small cleanups
From: Wei Liu @ 2026-07-22 23:22 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: linux-hyperv, linux-rt-devel, linux-kernel, K. Y. Srinivasan,
Dexuan Cui, Haiyang Zhang, Jan Kiszka, Long Li, Michael Kelley,
Wei Liu
In-Reply-To: <20260721153216.382536-1-bigeasy@linutronix.de>
On Tue, Jul 21, 2026 at 05:32:14PM +0200, Sebastian Andrzej Siewior wrote:
> Replacing the lockdep_hardirq_threaded() annotation which does not
> belong in drivers and removing the vmbus_irq_initialized which seems
> redundant.
>
> v1…v2: https://lore.kernel.org/all/20260401151517.1743555-1-bigeasy@linutronix.de/
> - Added a comment next to the invocation of __vmbus_isr()
> - Added the "Drivers:" prefix to the subject line
>
> Sebastian Andrzej Siewior (2):
> Drivers: hv: vmbus: Replace lockdep_hardirq_threaded() with lockdep
> annotation
> Drivers: hv: vmbus: Remove vmbus_irq_initialized
>
Applied. Thanks.
^ permalink raw reply
* Re: [PATCH v10 0/8] mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings
From: Andrew Morton @ 2026-07-22 22:48 UTC (permalink / raw)
To: Stanislav Kinsburskii
Cc: Jason Gunthorpe, Leon Romanovsky, David Hildenbrand,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jonathan Corbet, Shuah Khan,
Shuah Khan, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
Long Li, Lyude Paul, Danilo Krummrich, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Min Ma, Lizhi Hou, Oded Gabbay, linux-mm, linux-kernel, linux-doc,
linux-kselftest, linux-hyperv, dri-devel, nouveau, linux-rdma
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
On Wed, 22 Jul 2026 14:44:22 -0700 Stanislav Kinsburskii <skinsburskii@gmail.com> wrote:
> This series extends the HMM framework to support userfaultfd-backed memory
> by allowing the mmap read lock to be dropped during hmm_range_fault().
Thanks, I've updated mm.git's mm-unstable branch to this version.
Sashiko pointed at a few things, some pre-existing. The jiffies race
in [8/8] looks legit.
https://sashiko.dev/#/patchset/20260722-hmm-v10-v1-0-606464dd601a@gmail.com
> Changes in v10:
> - Included contended mmap_lock acquisition in the
> hmm_range_fault_unlocked_timeout() retry budget.
> - Dropped the redundant top-level fatal_signal_pending() check in the HMM
> unlocked retry loop; mmap_read_lock_killable() now covers that path.
> - Restored the absolute outer timeout in drm_gpusvm_get_pages(), since it can
> run from GPU page-fault workers and must not rely on the worker task’s fatal
> signal state to stop invalidation retries.
Here's how v10 altered mm.git:
drivers/gpu/drm/drm_gpusvm.c | 11 +++++++++--
mm/hmm.c | 29 +++++++++++++----------------
2 files changed, 22 insertions(+), 18 deletions(-)
--- a/drivers/gpu/drm/drm_gpusvm.c~b
+++ a/drivers/gpu/drm/drm_gpusvm.c
@@ -1405,7 +1405,9 @@ int drm_gpusvm_get_pages(struct drm_gpus
.dev_private_owner = ctx->device_private_page_owner,
};
void *zdd;
- unsigned long timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ unsigned long timeout =
+ jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ unsigned long remaining;
unsigned long i, j;
unsigned long npages = npages_in_range(pages_start, pages_end);
unsigned long num_dma_mapped;
@@ -1420,6 +1422,11 @@ int drm_gpusvm_get_pages(struct drm_gpus
struct dma_iova_state *state = &svm_pages->state;
retry:
+ if (time_after_eq(jiffies, timeout))
+ return -EBUSY;
+
+ remaining = timeout - jiffies;
+
hmm_range.notifier_seq = mmu_interval_read_begin(notifier);
if (drm_gpusvm_pages_valid_unlocked(gpusvm, svm_pages))
goto set_seqno;
@@ -1434,7 +1441,7 @@ retry:
}
hmm_range.hmm_pfns = pfns;
- err = hmm_range_fault_unlocked_timeout(&hmm_range, timeout);
+ err = hmm_range_fault_unlocked_timeout(&hmm_range, remaining);
mmput(mm);
if (err)
goto err_free;
--- a/mm/hmm.c~b
+++ a/mm/hmm.c
@@ -790,22 +790,14 @@ int hmm_range_fault_unlocked_timeout(str
int ret;
do {
- if (fatal_signal_pending(current))
- return -EINTR;
-
- if (timeout) {
- /*
- * If the previous fault dropped mmap_lock, then the fault
- * handler made progress. Restart the retry timeout in that
- * case, but keep the existing deadline for ordinary -EBUSY
- * retries.
- */
- if (!locked)
- deadline = jiffies + timeout;
-
- if (time_after(jiffies, deadline))
- return -EBUSY;
- }
+ /*
+ * If the previous fault dropped mmap_lock, then the fault
+ * handler made progress. Restart the retry timeout in that
+ * case, but keep the existing deadline for ordinary -EBUSY
+ * retries.
+ */
+ if (timeout && !locked)
+ deadline = jiffies + timeout;
range->notifier_seq =
mmu_interval_read_begin(range->notifier);
@@ -814,6 +806,11 @@ int hmm_range_fault_unlocked_timeout(str
if (ret)
return ret;
+ if (timeout && time_after(jiffies, deadline)) {
+ mmap_read_unlock(mm);
+ return -EBUSY;
+ }
+
locked = true;
ret = hmm_range_fault_locked(range, &locked);
if (locked)
_
^ permalink raw reply
* Re: [PATCH] mshv: bounds-check cpu index in vtl mmap fault handler
From: Wei Liu @ 2026-07-22 22:22 UTC (permalink / raw)
To: Yi Xie, ssengar, namjain
Cc: kys, haiyangz, wei.liu, decui, longli, linux-hyperv, linux-kernel
In-Reply-To: <20260709021947.49436-1-xieyi@kylinos.cn>
Naman, Saurabh, please review this.
On Thu, Jul 09, 2026 at 10:19:47AM +0800, Yi Xie wrote:
> cpu is taken from pgoff & 0xffff. cpu_online() does not reject cpu >=
> nr_cpu_ids, and per_cpu_ptr() can then walk off __per_cpu_offset.
>
> Signed-off-by: Yi Xie <xieyi@kylinos.cn>
> ---
> drivers/hv/mshv_vtl_main.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/hv/mshv_vtl_main.c b/drivers/hv/mshv_vtl_main.c
> index 0d3d4161974f..fc50c44ac1bd 100644
> --- a/drivers/hv/mshv_vtl_main.c
> +++ b/drivers/hv/mshv_vtl_main.c
> @@ -801,7 +801,7 @@ static vm_fault_t mshv_vtl_fault(struct vm_fault *vmf)
> int cpu = vmf->pgoff & MSHV_PG_OFF_CPU_MASK;
> int real_off = vmf->pgoff >> MSHV_REAL_OFF_SHIFT;
>
> - if (!cpu_online(cpu))
> + if (cpu >= nr_cpu_ids || !cpu_online(cpu))
> return VM_FAULT_SIGBUS;
> /*
> * CPU Hotplug is not supported in VTL2 in OpenHCL, where this kernel driver exists.
> --
> 2.34.1
^ permalink raw reply
* Re: [PATCH] mshv: fix fd leak in mshv_ioctl_create_vtl()
From: Wei Liu @ 2026-07-22 22:20 UTC (permalink / raw)
To: Hamza Mahfooz; +Cc: Yi Xie, kys, haiyangz, wei.liu, linux-hyperv, linux-kernel
In-Reply-To: <ak5LDBRXOVYTrZ9r@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
On Wed, Jul 08, 2026 at 09:05:16AM -0400, Hamza Mahfooz wrote:
> On Wed, Jul 08, 2026 at 09:28:52AM +0800, Yi Xie wrote:
> > put_unused_fd() if anon_inode_getfile() fails.
> >
> > Signed-off-by: Yi Xie <xieyi@kylinos.cn>
>
> Reviewed-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Applied to hyperv-fixes. Thanks.
^ permalink raw reply
* Re: [PATCH v2] mshv_vtl: clear hypercall output before copyout
From: Wei Liu @ 2026-07-22 22:07 UTC (permalink / raw)
To: Yousef Alhouseen
Cc: K . Y . Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li,
Michael Kelley, linux-hyperv, linux-kernel
In-Reply-To: <20260625181314.1399-1-alhouseenyousef@gmail.com>
On Thu, Jun 25, 2026 at 08:13:14PM +0200, Yousef Alhouseen wrote:
> mshv_vtl_hvcall_call() copies output_size bytes to userspace.
>
> The output page is freshly allocated. Userspace chooses the copyout length.
>
> If the hypercall writes less, the tail can contain stale page data.
>
> Clear the copied range before issuing the hypercall.
>
> Also check both bounce page allocations before either page is used.
>
> Signed-off-by: Yousef Alhouseen <alhouseenyousef@gmail.com>
Applied. Thanks.
^ permalink raw reply
* Re: [PATCH 1/1] Drivers: hv: vmbus: Set DMA coherent mask for VMBus devices
From: Wei Liu @ 2026-07-22 22:01 UTC (permalink / raw)
To: Michael Kelley
Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, longli@microsoft.com,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <SN6PR02MB415756E14DDAFCD7ECED3971D4C72@SN6PR02MB4157.namprd02.prod.outlook.com>
On Thu, Jul 16, 2026 at 05:57:49PM +0000, Michael Kelley wrote:
> From: Michael Kelley <mhklkml@zohomail.com> Sent: Sunday, June 7, 2026 7:06 PM
> >
> > In current code, the coherent_dma_mask for VMBus devices is not set, so
> > it has the default value of 0, which essentially means "invalid". Because
> > drivers for VMBus devices do not use dma_alloc_*() functions, the usual
> > use of the coherent mask does not occur, and no errors result.
> >
> > However, a valid coherent_dma_mask may be needed even though the drivers
> > don't use dma_alloc_*() functions. In a CoCo VM, the VMBus storvsc and
> > netvsc drivers must bounce buffer DMA operations through the swiotlb
> > because the Hyper-V host can't DMA into encrypted guest memory. If the
> > kernel is built with CONFIG_SWIOTLB_DYNAMIC and the initial swiotlb size
> > is small, swiotlb code may need to grow the swiotlb in response to a DMA
> > mapping request. That growth first allocates a transient pool while the
> > swiotlb is expanded in the background. The transient pool memory is
> > allocated from the DMA atomic pools, and the allocation code checks for
> > a valid coherent_dma_mask. With current code, this check fails, then the
> > DMA mapping request from the storvsc or netvsc driver fails, and finally
> > an I/O error occurs.
> >
> > Fix this problem by setting coherent_dma_mask for VMBus devices at the
> > same time that dma_mask is set. Being a synthetic bus, VMBus does not
> > have any restrictions on coherent DMA, so the coherent mask is set to
> > the full 64 bits for all VMBus devices, just like with dma_mask.
> >
> > Signed-off-by: Michael Kelley <mhklinux@outlook.com>
>
> Gentle ping: Anyone able to review this patch? There's a
> Sashiko comment, but it's for an issue in an unrelated error path,
> so I'm not planning to respin this patch for that comment.
Applied. Thanks.
^ permalink raw reply
* Re: [PATCH v10 8/8] drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults
From: sashiko-bot @ 2026-07-22 21:59 UTC (permalink / raw)
To: Stanislav Kinsburskii; +Cc: linux-hyperv, dri-devel
In-Reply-To: <20260722-hmm-v10-v1-8-606464dd601a@gmail.com>
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [High] TOCTOU race on volatile `jiffies` can result in a 0 timeout, causing an infinite loop.
- [High] The absolute outer deadline in `drm_gpusvm_get_pages()` is defeated by the relative timeout extension semantics of `hmm_range_fault_unlocked_timeout()`.
Pre-existing issues:
- [High] Failure to allocate `pfns` in `drm_gpusvm_range_evict()` leaks the `mm_struct` reference.
--
--- Patch [8]: [PATCH v10 8/8] drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults ---
commit f0577dfbb2f6c27cfd07d3be42e1db31600b9e4d
Author: Stanislav Kinsburskii <skinsburskii@gmail.com>
Subject: drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults
This commit replaces open-coded HMM fault retry loops in the GPU SVM paths
with the new hmm_range_fault_unlocked_timeout() helper. It updates
drm_gpusvm_scan_mm(), drm_gpusvm_range_evict(), and drm_gpusvm_get_pages()
to delegate mmap_lock acquisition and retry logic to the helper.
> diff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c
> index 958cb605aedd5..b946f920b7a00 100644
> --- a/drivers/gpu/drm/drm_gpusvm.c
> +++ b/drivers/gpu/drm/drm_gpusvm.c
[ ... ]
> @@ -1408,6 +1392,7 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
> void *zdd;
> unsigned long timeout =
> jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
> + unsigned long remaining;
> unsigned long i, j;
> unsigned long npages = npages_in_range(pages_start, pages_end);
> unsigned long num_dma_mapped;
> @@ -1422,9 +1407,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
> struct dma_iova_state *state = &svm_pages->state;
>
> retry:
> - if (time_after(jiffies, timeout))
> + if (time_after_eq(jiffies, timeout))
> return -EBUSY;
>
> + remaining = timeout - jiffies;
[Severity: High]
Does this double read of jiffies introduce a race condition?
Because jiffies is volatile, it is fetched from memory for the
time_after_eq() check, and then fetched again for the subtraction. If
jiffies increments between these reads, remaining can evaluate to 0
(if jiffies equals timeout) or ULONG_MAX (if jiffies ticks past timeout).
When remaining is 0, hmm_range_fault_unlocked_timeout() disables the
deadline entirely. When it is ULONG_MAX, the timeout is effectively
infinite, which could hang the page-fault worker.
> hmm_range.notifier_seq = mmu_interval_read_begin(notifier);
> if (drm_gpusvm_pages_valid_unlocked(gpusvm, svm_pages))
> goto set_seqno;
> @@ -1439,21 +1426,7 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
> }
>
> hmm_range.hmm_pfns = pfns;
> - while (true) {
> - mmap_read_lock(mm);
> - err = hmm_range_fault(&hmm_range);
> - mmap_read_unlock(mm);
> -
> - if (err == -EBUSY) {
> - if (time_after(jiffies, timeout))
> - break;
> -
> - hmm_range.notifier_seq =
> - mmu_interval_read_begin(notifier);
> - continue;
> - }
> - break;
> - }
> + err = hmm_range_fault_unlocked_timeout(&hmm_range, remaining);
[Severity: High]
Does passing remaining defeat the absolute deadline intended for this path?
The commit message notes that drm_gpusvm_get_pages() keeps an absolute outer
deadline to prevent endless invalidation retry loops from hanging GPU
page-fault workers.
However, hmm_range_fault_unlocked_timeout() treats the passed value as a
relative timeout and recalculates its internal deadline every time the lock
is dropped:
mm/hmm.c:hmm_range_fault_unlocked_timeout() {
...
if (timeout && !locked)
deadline = jiffies + timeout;
...
}
Since remaining is passed as the full timeout budget, the helper will extend
its deadline by the full amount of remaining into the future every time it
drops the lock, which can allow an infinite loop.
> mmput(mm);
> if (err)
> goto err_free;
[ ... ]
> @@ -1736,24 +1708,12 @@ int drm_gpusvm_range_evict(struct drm_gpusvm *gpusvm,
> return -ENOMEM;
[Severity: High]
This is a pre-existing issue, but does this error path leak the mm_struct
reference?
Earlier in drm_gpusvm_range_evict(), the code acquires an mm reference:
if (!mmget_not_zero(mm))
return -EFAULT;
If kvmalloc_array() fails, the function returns -ENOMEM directly, bypassing
the mmput(mm) call at the end of the function. This permanently leaks the mm
reference, preventing the target process's memory structures from being freed
upon exit.
>
> hmm_range.hmm_pfns = pfns;
> - while (!time_after(jiffies, timeout)) {
> - hmm_range.notifier_seq = mmu_interval_read_begin(notifier);
> - if (time_after(jiffies, timeout)) {
> - err = -ETIME;
> - break;
> - }
> -
> - mmap_read_lock(mm);
> - err = hmm_range_fault(&hmm_range);
> - mmap_read_unlock(mm);
> - if (err != -EBUSY)
> - break;
> - }
> + err = hmm_range_fault_unlocked_timeout(&hmm_range, timeout);
>
> kvfree(pfns);
> mmput(mm);
>
> - return err;
> + return err == -EBUSY ? -ETIME : err;
> }
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260722-hmm-v10-v1-0-606464dd601a@gmail.com?part=8
^ permalink raw reply
* Re: [PATCH v10 6/8] RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults
From: sashiko-bot @ 2026-07-22 21:56 UTC (permalink / raw)
To: Stanislav Kinsburskii; +Cc: linux-hyperv, dri-devel
In-Reply-To: <20260722-hmm-v10-v1-6-606464dd601a@gmail.com>
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing check for unstable address space before iterating over an external mm_struct's VMAs, risking a crash if the OOM reaper has marked it MMF_UNSTABLE.
- [High] Unbounded interval invalidation retry loop in a workqueue context lacks a timeout or yield, leading to a potential CPU soft lockup.
--
--- Patch [6]: [PATCH v10 6/8] RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults ---
commit b7408a657980d2af72d45f322c843edb9faecf2e
Author: Stanislav Kinsburskii <skinsburskii@gmail.com>
RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults
ib_umem_odp_map_dma_and_lock() takes mmap_read_lock() only around
hmm_range_fault(), then retries -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT
expires.
Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns
the mmap lock and refreshes range->notifier_seq for its internal retries.
ODP keeps using HMM_RANGE_DEFAULT_TIMEOUT for each HMM fault attempt,
while interval invalidation retries continue to be handled by the existing
outer loop.
ODP still validates the interval notifier sequence while holding umem_mutex
before DMA mapping pages.
> diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c
> index 404fa1cc32542..9cc21cd762d98 100644
> --- a/drivers/infiniband/core/umem_odp.c
> +++ b/drivers/infiniband/core/umem_odp.c
[ ... ]
> @@ -363,26 +363,18 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *umem_odp, u64 user_virt,
> }
>
> range.hmm_pfns = &(umem_odp->map.pfn_list[pfn_start_idx]);
> - timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
> + timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
>
> retry:
> - current_seq = range.notifier_seq =
> - mmu_interval_read_begin(&umem_odp->notifier);
> -
> - mmap_read_lock(owning_mm);
> - ret = hmm_range_fault(&range);
> - mmap_read_unlock(owning_mm);
> - if (unlikely(ret)) {
> - if (ret == -EBUSY && !time_after(jiffies, timeout))
> - goto retry;
> + ret = hmm_range_fault_unlocked_timeout(&range, timeout);
> + if (unlikely(ret))
> goto out_put_mm;
[Severity: High]
This isn't a bug introduced by this patch, but I noticed we are passing an
external mm to the HMM helper here without checking for an unstable address
space.
When we call hmm_range_fault_unlocked_timeout(), it takes the
mmap_read_lock_killable() on the external owning_mm and traverses the VMA
tree locklessly:
mm/hmm.c:hmm_range_fault_unlocked_timeout() {
...
ret = mmap_read_lock_killable(mm);
...
ret = hmm_range_fault_locked(range, &locked);
...
}
Could this crash if the external mm is being concurrently torn down?
For instance, if the OOM reaper has marked the mm as MMF_UNSTABLE
or if a dup_mmap() failure has left XA_ZERO_ENTRY markers in the maple
tree.
Does the HMM helper or this caller need to invoke
check_stable_address_space(mm) after acquiring the mmap lock before
walking the page tables?
> - }
>
> start_idx = (range.start - ib_umem_start(umem_odp)) >> page_shift;
> dma_index = start_idx;
>
> mutex_lock(&umem_odp->umem_mutex);
> - if (mmu_interval_read_retry(&umem_odp->notifier, current_seq)) {
> + if (mmu_interval_read_retry(&umem_odp->notifier, range.notifier_seq)) {
> mutex_unlock(&umem_odp->umem_mutex);
> goto retry;
> }
[Severity: High]
This is a pre-existing issue, but does this interval invalidation retry
loop lack an overall timeout or a way to yield?
If a user-space program concurrently loops on madvise(MADV_DONTNEED) for
a memory region, it triggers the MMU notifier, causing
mmu_interval_read_retry() to repeatedly return true.
Because hmm_range_fault_unlocked_timeout() takes a relative timeout,
calling it again here on every retry resets the deadline for the HMM fault
attempt. If the pages are already faulted, the helper returns 0 quickly,
and this thread will loop unconditionally back to the retry label.
Can this unbounded loop monopolize a CPU core indefinitely and lead to a
soft lockup?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260722-hmm-v10-v1-0-606464dd601a@gmail.com?part=6
^ permalink raw reply
* Re: [PATCH v10 3/8] selftests/mm: add HMM test for mmap lock-dropping faults
From: sashiko-bot @ 2026-07-22 21:53 UTC (permalink / raw)
To: Stanislav Kinsburskii; +Cc: linux-hyperv, dri-devel
In-Reply-To: <20260722-hmm-v10-v1-3-606464dd601a@gmail.com>
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] `dmirror_range_fault_unlocked` fails to enforce an absolute timeout for its `mmu_interval_read_retry` loop, leading to potential livelocks.
--
--- Patch [3]: [PATCH v10 3/8] selftests/mm: add HMM test for mmap lock-dropping faults ---
commit e4ef68d1c56c2f5bfba882ccad54a4d7284e5403
Author: Stanislav Kinsburskii <skinsburskii@gmail.com>
selftests/mm: add HMM test for mmap lock-dropping faults
This commit adds test_hmm coverage for the HMM lock-dropping fault path.
It introduces a new ioctl that exercises the unbounded retry mode while
allowing the mmap lock to be dropped during fault handling, alongside a
corresponding userfaultfd read selftest.
> diff --git a/lib/test_hmm.c b/lib/test_hmm.c
> index 45c0cb9922184..6205fb313bd05 100644
> --- a/lib/test_hmm.c
> +++ b/lib/test_hmm.c
[ ... ]
> @@ -389,6 +389,67 @@ static int dmirror_range_fault(struct dmirror *dmirror,
> return ret;
> }
>
> +static int dmirror_range_fault_unlocked(struct dmirror *dmirror,
> + struct hmm_range *range,
> + unsigned long timeout)
> +{
> + int ret;
> +
> + while (true) {
> + ret = hmm_range_fault_unlocked_timeout(range, timeout);
> + if (ret)
> + goto out;
> +
> + mutex_lock(&dmirror->mutex);
> + if (mmu_interval_read_retry(range->notifier,
> + range->notifier_seq)) {
> + mutex_unlock(&dmirror->mutex);
> + continue;
[Severity: Medium]
Does this loop lack an absolute timeout enforcement?
When mmu_interval_read_retry() triggers a retry, the same relative timeout
value is passed back into hmm_range_fault_unlocked_timeout(). Because that
function calculates its internal deadline as jiffies + timeout, the
deadline gets pushed forward on every retry.
Could this lead to a livelock if this function is ever called with a non-zero
timeout, since the timeout resets infinitely during concurrent page table
invalidations?
> + }
> + break;
> + }
> +
> + ret = dmirror_do_fault(dmirror, range);
> +
> + mutex_unlock(&dmirror->mutex);
> +out:
> + return ret;
> +}
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260722-hmm-v10-v1-0-606464dd601a@gmail.com?part=3
^ permalink raw reply
* [PATCH v10 8/8] drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
Several GPU SVM paths take mmap_read_lock() only to call hmm_range_fault()
and open-code mmu interval sequence setup before each HMM walk. They also
retry -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT expires.
Use hmm_range_fault_unlocked_timeout() for those faults. The HMM helper now
owns mmap_lock acquisition and refreshes range->notifier_seq for its
internal retries, while GPU SVM keeps its existing driver-lock validation
with mmu_interval_read_retry() after a successful fault.
drm_gpusvm_scan_mm() and drm_gpusvm_range_evict() pass
HMM_RANGE_DEFAULT_TIMEOUT as the helper retry budget for each HMM fault
attempt. drm_gpusvm_get_pages() keeps its existing absolute outer deadline
because it can be reached from GPU page-fault workers, where fatal signals
from the faulting process cannot stop an endless invalidation retry loop.
It passes the remaining time from that deadline to HMM for each fault
attempt.
Leave drm_gpusvm_check_pages() on hmm_range_fault() because that path is
called with the mmap lock already held by its caller.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/gpu/drm/drm_gpusvm.c | 60 ++++++++------------------------------------
1 file changed, 10 insertions(+), 50 deletions(-)
diff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c
index 958cb605aedd..b946f920b7a0 100644
--- a/drivers/gpu/drm/drm_gpusvm.c
+++ b/drivers/gpu/drm/drm_gpusvm.c
@@ -773,8 +773,7 @@ enum drm_gpusvm_scan_result drm_gpusvm_scan_mm(struct drm_gpusvm_range *range,
.end = end,
.dev_private_owner = dev_private_owner,
};
- unsigned long timeout =
- jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ unsigned long timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
enum drm_gpusvm_scan_result state = DRM_GPUSVM_SCAN_UNPOPULATED, new_state;
unsigned long *pfns;
unsigned long npages = npages_in_range(start, end);
@@ -788,22 +787,7 @@ enum drm_gpusvm_scan_result drm_gpusvm_scan_mm(struct drm_gpusvm_range *range,
hmm_range.hmm_pfns = pfns;
retry:
- hmm_range.notifier_seq = mmu_interval_read_begin(notifier);
- mmap_read_lock(range->gpusvm->mm);
-
- while (true) {
- err = hmm_range_fault(&hmm_range);
- if (err == -EBUSY) {
- if (time_after(jiffies, timeout))
- break;
-
- hmm_range.notifier_seq =
- mmu_interval_read_begin(notifier);
- continue;
- }
- break;
- }
- mmap_read_unlock(range->gpusvm->mm);
+ err = hmm_range_fault_unlocked_timeout(&hmm_range, timeout);
if (err)
goto err_free;
@@ -1408,6 +1392,7 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
void *zdd;
unsigned long timeout =
jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ unsigned long remaining;
unsigned long i, j;
unsigned long npages = npages_in_range(pages_start, pages_end);
unsigned long num_dma_mapped;
@@ -1422,9 +1407,11 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
struct dma_iova_state *state = &svm_pages->state;
retry:
- if (time_after(jiffies, timeout))
+ if (time_after_eq(jiffies, timeout))
return -EBUSY;
+ remaining = timeout - jiffies;
+
hmm_range.notifier_seq = mmu_interval_read_begin(notifier);
if (drm_gpusvm_pages_valid_unlocked(gpusvm, svm_pages))
goto set_seqno;
@@ -1439,21 +1426,7 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm,
}
hmm_range.hmm_pfns = pfns;
- while (true) {
- mmap_read_lock(mm);
- err = hmm_range_fault(&hmm_range);
- mmap_read_unlock(mm);
-
- if (err == -EBUSY) {
- if (time_after(jiffies, timeout))
- break;
-
- hmm_range.notifier_seq =
- mmu_interval_read_begin(notifier);
- continue;
- }
- break;
- }
+ err = hmm_range_fault_unlocked_timeout(&hmm_range, remaining);
mmput(mm);
if (err)
goto err_free;
@@ -1720,8 +1693,7 @@ int drm_gpusvm_range_evict(struct drm_gpusvm *gpusvm,
.end = drm_gpusvm_range_end(range),
.dev_private_owner = NULL,
};
- unsigned long timeout =
- jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ unsigned long timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
unsigned long *pfns;
unsigned long npages = npages_in_range(drm_gpusvm_range_start(range),
drm_gpusvm_range_end(range));
@@ -1736,24 +1708,12 @@ int drm_gpusvm_range_evict(struct drm_gpusvm *gpusvm,
return -ENOMEM;
hmm_range.hmm_pfns = pfns;
- while (!time_after(jiffies, timeout)) {
- hmm_range.notifier_seq = mmu_interval_read_begin(notifier);
- if (time_after(jiffies, timeout)) {
- err = -ETIME;
- break;
- }
-
- mmap_read_lock(mm);
- err = hmm_range_fault(&hmm_range);
- mmap_read_unlock(mm);
- if (err != -EBUSY)
- break;
- }
+ err = hmm_range_fault_unlocked_timeout(&hmm_range, timeout);
kvfree(pfns);
mmput(mm);
- return err;
+ return err == -EBUSY ? -ETIME : err;
}
EXPORT_SYMBOL_GPL(drm_gpusvm_range_evict);
--
2.43.0
^ permalink raw reply related
* [PATCH v10 7/8] accel/amdxdna: Use hmm_range_fault_unlocked_timeout() for range population
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
aie2_populate_range() takes mmap_read_lock() only around hmm_range_fault().
It also open-codes the mmu interval sequence setup before each HMM walk and
retries -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT expires.
Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns
the mmap lock and refreshes mapp->range.notifier_seq for its internal
retries, so the driver only needs to call the helper and then validate the
sequence before marking the mapping populated.
Pass HMM_RANGE_DEFAULT_TIMEOUT as the helper retry budget for each HMM
population attempt. This scopes the timeout to repeated HMM notifier
retries while preserving the existing outer loop that moves between invalid
mappings and restarts when the interval is invalidated before the driver
updates its mapping state.
Keep returning -ETIME when the HMM retry budget expires, matching the
driver's existing timeout error convention.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/accel/amdxdna/aie2_ctx.c | 23 ++++-------------------
1 file changed, 4 insertions(+), 19 deletions(-)
diff --git a/drivers/accel/amdxdna/aie2_ctx.c b/drivers/accel/amdxdna/aie2_ctx.c
index 54486960cbf5..b5b4ca263002 100644
--- a/drivers/accel/amdxdna/aie2_ctx.c
+++ b/drivers/accel/amdxdna/aie2_ctx.c
@@ -1034,7 +1034,7 @@ static int aie2_populate_range(struct amdxdna_gem_obj *abo)
bool found;
int ret;
- timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
again:
found = false;
down_write(&xdna->notifier_lock);
@@ -1061,24 +1061,9 @@ static int aie2_populate_range(struct amdxdna_gem_obj *abo)
return -EFAULT;
}
- mapp->range.notifier_seq = mmu_interval_read_begin(&mapp->notifier);
- mmap_read_lock(mm);
- ret = hmm_range_fault(&mapp->range);
- mmap_read_unlock(mm);
- if (ret) {
- if (time_after(jiffies, timeout)) {
- ret = -ETIME;
- goto put_mm;
- }
-
- if (ret == -EBUSY) {
- amdxdna_umap_put(mapp);
- mmput(mm);
- goto again;
- }
-
+ ret = hmm_range_fault_unlocked_timeout(&mapp->range, timeout);
+ if (ret)
goto put_mm;
- }
down_write(&xdna->notifier_lock);
if (mmu_interval_read_retry(&mapp->notifier, mapp->range.notifier_seq)) {
@@ -1096,7 +1081,7 @@ static int aie2_populate_range(struct amdxdna_gem_obj *abo)
put_mm:
amdxdna_umap_put(mapp);
mmput(mm);
- return ret;
+ return ret == -EBUSY ? -ETIME : ret;
}
int aie2_cmd_submit(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job *job, u64 *seq)
--
2.43.0
^ permalink raw reply related
* [PATCH v10 6/8] RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
ib_umem_odp_map_dma_and_lock() takes mmap_read_lock() only around
hmm_range_fault(), then retries -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT
expires.
Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns
the mmap lock and refreshes range->notifier_seq for its internal retries.
ODP keeps using HMM_RANGE_DEFAULT_TIMEOUT for each HMM fault attempt,
while interval invalidation retries continue to be handled by the existing
outer loop.
ODP still validates the interval notifier sequence while holding umem_mutex
before DMA mapping pages.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/infiniband/core/umem_odp.c | 18 +++++-------------
1 file changed, 5 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c
index 404fa1cc3254..9cc21cd762d9 100644
--- a/drivers/infiniband/core/umem_odp.c
+++ b/drivers/infiniband/core/umem_odp.c
@@ -329,7 +329,7 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *umem_odp, u64 user_virt,
struct mm_struct *owning_mm = umem_odp->umem.owning_mm;
int pfn_index, dma_index, ret = 0, start_idx;
unsigned int page_shift, hmm_order, pfn_start_idx;
- unsigned long num_pfns, current_seq;
+ unsigned long num_pfns;
struct hmm_range range = {};
unsigned long timeout;
@@ -363,26 +363,18 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *umem_odp, u64 user_virt,
}
range.hmm_pfns = &(umem_odp->map.pfn_list[pfn_start_idx]);
- timeout = jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
+ timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
retry:
- current_seq = range.notifier_seq =
- mmu_interval_read_begin(&umem_odp->notifier);
-
- mmap_read_lock(owning_mm);
- ret = hmm_range_fault(&range);
- mmap_read_unlock(owning_mm);
- if (unlikely(ret)) {
- if (ret == -EBUSY && !time_after(jiffies, timeout))
- goto retry;
+ ret = hmm_range_fault_unlocked_timeout(&range, timeout);
+ if (unlikely(ret))
goto out_put_mm;
- }
start_idx = (range.start - ib_umem_start(umem_odp)) >> page_shift;
dma_index = start_idx;
mutex_lock(&umem_odp->umem_mutex);
- if (mmu_interval_read_retry(&umem_odp->notifier, current_seq)) {
+ if (mmu_interval_read_retry(&umem_odp->notifier, range.notifier_seq)) {
mutex_unlock(&umem_odp->umem_mutex);
goto retry;
}
--
2.43.0
^ permalink raw reply related
* [PATCH v10 5/8] drm/nouveau: Use hmm_range_fault_unlocked_timeout() for SVM faults
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
nouveau_range_fault() takes mmap_read_lock() only to call
hmm_range_fault(). It also keeps a single HMM_RANGE_DEFAULT_TIMEOUT
deadline across both HMM -EBUSY retries and post-fault
mmu_interval_read_retry() retries.
Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns
the mmap lock and refreshes range->notifier_seq for its internal retries.
Nouveau keeps its existing absolute deadline in the outer loop and passes
the remaining jiffies to the helper for each fault attempt, so retries
caused by mmu_interval_read_retry() do not reset the overall retry budget.
Nouveau still validates the interval notifier sequence while holding
svmm->mutex before programming the GPU mapping.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/gpu/drm/nouveau/nouveau_svm.c | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
index dcc92131488e..58735446d783 100644
--- a/drivers/gpu/drm/nouveau/nouveau_svm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
@@ -678,20 +678,22 @@ static int nouveau_range_fault(struct nouveau_svmm *svmm,
range.end = notifier->notifier.interval_tree.last + 1;
while (true) {
- if (time_after(jiffies, timeout)) {
+ long remaining = timeout - jiffies;
+
+ /*
+ * The HMM timeout only bounds retries while HMM is walking and
+ * faulting the range. This fault is handled by a kernel worker,
+ * so fatal signals from the faulting process cannot stop an
+ * endless stream of invalidations here.
+ */
+ if (time_after_eq(jiffies, timeout)) {
ret = -EBUSY;
goto out;
}
- range.notifier_seq = mmu_interval_read_begin(range.notifier);
- mmap_read_lock(mm);
- ret = hmm_range_fault(&range);
- mmap_read_unlock(mm);
- if (ret) {
- if (ret == -EBUSY)
- continue;
+ ret = hmm_range_fault_unlocked_timeout(&range, remaining);
+ if (ret)
goto out;
- }
mutex_lock(&svmm->mutex);
if (mmu_interval_read_retry(range.notifier,
--
2.43.0
^ permalink raw reply related
* [PATCH v10 4/8] mshv: Use hmm_range_fault_unlocked_timeout() for region faults
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
MSHV currently faults movable memory regions by taking mmap_read_lock()
around hmm_range_fault(). That prevents the fault path from handling VMAs
whose fault handlers need to drop mmap_lock, such as userfaultfd-backed
mappings.
Use hmm_range_fault_unlocked_timeout() instead. Passing a timeout of 0
preserves MSHV's existing unbounded retry behavior while letting the HMM
helper own mmap_lock acquisition and refresh range->notifier_seq internally
before walking the range. After the fault succeeds, MSHV still takes
mreg_mutex and checks mmu_interval_read_retry() before installing the pages
into the region, so the existing invalidation synchronization is preserved.
Fold the small fault-and-lock helper into mshv_region_range_fault(), since
the remaining retry path is just the standard "fault, take the driver lock,
check the interval notifier sequence" pattern.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
drivers/hv/mshv_regions.c | 54 +++++++++--------------------------------------
1 file changed, 10 insertions(+), 44 deletions(-)
diff --git a/drivers/hv/mshv_regions.c b/drivers/hv/mshv_regions.c
index 6d65e5b42152..dddaade31b5d 100644
--- a/drivers/hv/mshv_regions.c
+++ b/drivers/hv/mshv_regions.c
@@ -381,46 +381,6 @@ int mshv_region_get(struct mshv_mem_region *region)
return kref_get_unless_zero(®ion->mreg_refcount);
}
-/**
- * mshv_region_hmm_fault_and_lock - Handle HMM faults and lock the memory region
- * @region: Pointer to the memory region structure
- * @range: Pointer to the HMM range structure
- *
- * This function performs the following steps:
- * 1. Reads the notifier sequence for the HMM range.
- * 2. Acquires a read lock on the memory map.
- * 3. Handles HMM faults for the specified range.
- * 4. Releases the read lock on the memory map.
- * 5. If successful, locks the memory region mutex.
- * 6. Verifies if the notifier sequence has changed during the operation.
- * If it has, releases the mutex and returns -EBUSY to match with
- * hmm_range_fault() return code for repeating.
- *
- * Return: 0 on success, a negative error code otherwise.
- */
-static int mshv_region_hmm_fault_and_lock(struct mshv_mem_region *region,
- struct hmm_range *range)
-{
- int ret;
-
- range->notifier_seq = mmu_interval_read_begin(range->notifier);
- mmap_read_lock(region->mreg_mni.mm);
- ret = hmm_range_fault(range);
- mmap_read_unlock(region->mreg_mni.mm);
- if (ret)
- return ret;
-
- mutex_lock(®ion->mreg_mutex);
-
- if (mmu_interval_read_retry(range->notifier, range->notifier_seq)) {
- mutex_unlock(®ion->mreg_mutex);
- cond_resched();
- return -EBUSY;
- }
-
- return 0;
-}
-
/**
* mshv_region_range_fault - Handle memory range faults for a given region.
* @region: Pointer to the memory region structure.
@@ -452,13 +412,19 @@ static int mshv_region_range_fault(struct mshv_mem_region *region,
range.start = region->start_uaddr + page_offset * HV_HYP_PAGE_SIZE;
range.end = range.start + page_count * HV_HYP_PAGE_SIZE;
- do {
- ret = mshv_region_hmm_fault_and_lock(region, &range);
- } while (ret == -EBUSY);
-
+again:
+ ret = hmm_range_fault_unlocked_timeout(&range, 0);
if (ret)
goto out;
+ mutex_lock(®ion->mreg_mutex);
+
+ if (mmu_interval_read_retry(range.notifier, range.notifier_seq)) {
+ mutex_unlock(®ion->mreg_mutex);
+ cond_resched();
+ goto again;
+ }
+
for (i = 0; i < page_count; i++)
region->mreg_pages[page_offset + i] = hmm_pfn_to_page(pfns[i]);
--
2.43.0
^ permalink raw reply related
* [PATCH v10 3/8] selftests/mm: add HMM test for mmap lock-dropping faults
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
Add test_hmm coverage for the HMM lock-dropping fault path. The test module
gets a new HMM_DMIRROR_READ_UNLOCKED ioctl that calls
hmm_range_fault_unlocked_timeout() with a timeout of 0, exercising the
unbounded retry mode while allowing the mmap lock to be dropped during
fault handling.
Add a userfaultfd_read selftest that registers an anonymous mapping with
UFFDIO_REGISTER_MODE_MISSING, services the faults from a handler thread
with UFFDIO_COPY, and verifies that HMM can read back the data supplied by
the handler. This exercises the path where handle_mm_fault() drops
mmap_lock and hmm_range_fault_unlocked_timeout() restarts the walk
internally.
Assisted-by: GitHub-Copilot:claude-opus-4.6
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
lib/test_hmm.c | 107 ++++++++++++++++++++++-
lib/test_hmm_uapi.h | 1 +
tools/testing/selftests/mm/hmm-tests.c | 150 +++++++++++++++++++++++++++++++++
3 files changed, 257 insertions(+), 1 deletion(-)
diff --git a/lib/test_hmm.c b/lib/test_hmm.c
index 45c0cb992218..6205fb313bd0 100644
--- a/lib/test_hmm.c
+++ b/lib/test_hmm.c
@@ -389,6 +389,67 @@ static int dmirror_range_fault(struct dmirror *dmirror,
return ret;
}
+static int dmirror_range_fault_unlocked(struct dmirror *dmirror,
+ struct hmm_range *range,
+ unsigned long timeout)
+{
+ int ret;
+
+ while (true) {
+ ret = hmm_range_fault_unlocked_timeout(range, timeout);
+ if (ret)
+ goto out;
+
+ mutex_lock(&dmirror->mutex);
+ if (mmu_interval_read_retry(range->notifier,
+ range->notifier_seq)) {
+ mutex_unlock(&dmirror->mutex);
+ continue;
+ }
+ break;
+ }
+
+ ret = dmirror_do_fault(dmirror, range);
+
+ mutex_unlock(&dmirror->mutex);
+out:
+ return ret;
+}
+
+static int dmirror_fault_unlocked(struct dmirror *dmirror,
+ unsigned long start,
+ unsigned long end, bool write,
+ unsigned long timeout)
+{
+ struct mm_struct *mm = dmirror->notifier.mm;
+ unsigned long addr;
+ unsigned long pfns[32];
+ struct hmm_range range = {
+ .notifier = &dmirror->notifier,
+ .hmm_pfns = pfns,
+ .pfn_flags_mask = 0,
+ .default_flags =
+ HMM_PFN_REQ_FAULT | (write ? HMM_PFN_REQ_WRITE : 0),
+ .dev_private_owner = dmirror->mdevice,
+ };
+ int ret = 0;
+
+ if (!mmget_not_zero(mm))
+ return -EFAULT;
+
+ for (addr = start; addr < end; addr = range.end) {
+ range.start = addr;
+ range.end = min(addr + (ARRAY_SIZE(pfns) << PAGE_SHIFT), end);
+
+ ret = dmirror_range_fault_unlocked(dmirror, &range, timeout);
+ if (ret)
+ break;
+ }
+
+ mmput(mm);
+ return ret;
+}
+
static int dmirror_fault(struct dmirror *dmirror, unsigned long start,
unsigned long end, bool write)
{
@@ -488,6 +549,48 @@ static int dmirror_read(struct dmirror *dmirror, struct hmm_dmirror_cmd *cmd)
return ret;
}
+static int dmirror_read_unlocked(struct dmirror *dmirror,
+ struct hmm_dmirror_cmd *cmd,
+ unsigned long timeout)
+{
+ struct dmirror_bounce bounce;
+ unsigned long start, end;
+ unsigned long size = cmd->npages << PAGE_SHIFT;
+ int ret;
+
+ start = cmd->addr;
+ end = start + size;
+ if (end < start)
+ return -EINVAL;
+
+ ret = dmirror_bounce_init(&bounce, start, size);
+ if (ret)
+ return ret;
+
+ while (1) {
+ mutex_lock(&dmirror->mutex);
+ ret = dmirror_do_read(dmirror, start, end, &bounce);
+ mutex_unlock(&dmirror->mutex);
+ if (ret != -ENOENT)
+ break;
+
+ start = cmd->addr + (bounce.cpages << PAGE_SHIFT);
+ ret = dmirror_fault_unlocked(dmirror, start, end, false, timeout);
+ if (ret)
+ break;
+ cmd->faults++;
+ }
+
+ if (ret == 0) {
+ if (copy_to_user(u64_to_user_ptr(cmd->ptr), bounce.ptr,
+ bounce.size))
+ ret = -EFAULT;
+ }
+ cmd->cpages = bounce.cpages;
+ dmirror_bounce_fini(&bounce);
+ return ret;
+}
+
static int dmirror_do_write(struct dmirror *dmirror, unsigned long start,
unsigned long end, struct dmirror_bounce *bounce)
{
@@ -1572,7 +1675,9 @@ static long dmirror_fops_unlocked_ioctl(struct file *filp,
dmirror->flags = cmd.npages;
ret = 0;
break;
-
+ case HMM_DMIRROR_READ_UNLOCKED:
+ ret = dmirror_read_unlocked(dmirror, &cmd, 0);
+ break;
default:
return -EINVAL;
}
diff --git a/lib/test_hmm_uapi.h b/lib/test_hmm_uapi.h
index f94c6d457338..ea9b0ec404fb 100644
--- a/lib/test_hmm_uapi.h
+++ b/lib/test_hmm_uapi.h
@@ -38,6 +38,7 @@ struct hmm_dmirror_cmd {
#define HMM_DMIRROR_CHECK_EXCLUSIVE _IOWR('H', 0x06, struct hmm_dmirror_cmd)
#define HMM_DMIRROR_RELEASE _IOWR('H', 0x07, struct hmm_dmirror_cmd)
#define HMM_DMIRROR_FLAGS _IOWR('H', 0x08, struct hmm_dmirror_cmd)
+#define HMM_DMIRROR_READ_UNLOCKED _IOWR('H', 0x09, struct hmm_dmirror_cmd)
#define HMM_DMIRROR_FLAG_FAIL_ALLOC (1ULL << 0)
diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c
index 6fccbdab02ee..5acb728666f8 100644
--- a/tools/testing/selftests/mm/hmm-tests.c
+++ b/tools/testing/selftests/mm/hmm-tests.c
@@ -29,6 +29,10 @@
#include <sys/mman.h>
#include <sys/ioctl.h>
#include <sys/time.h>
+#include <sys/syscall.h>
+#include <sys/eventfd.h>
+#include <linux/userfaultfd.h>
+#include <poll.h>
/*
* This is a private UAPI to the kernel test module so it isn't exported
@@ -2952,4 +2956,150 @@ TEST_F_TIMEOUT(hmm, benchmark_thp_migration, 120)
&thp_results, ®ular_results);
}
}
+/*
+ * Test that HMM can fault in pages backed by userfaultfd using the
+ * hmm_range_fault_unlocked_timeout() path with no timeout. This exercises
+ * the lock-drop retry logic in the HMM framework.
+ */
+struct uffd_thread_args {
+ int uffd;
+ int stop_fd;
+ void *page_buffer;
+ unsigned long page_size;
+};
+
+static void *uffd_handler_thread(void *arg)
+{
+ struct uffd_thread_args *args = arg;
+ struct uffd_msg msg;
+ struct uffdio_copy copy;
+ struct pollfd pollfd[2];
+ int ret;
+
+ pollfd[0].fd = args->uffd;
+ pollfd[0].events = POLLIN;
+ pollfd[1].fd = args->stop_fd;
+ pollfd[1].events = POLLIN;
+
+ while (1) {
+ ret = poll(pollfd, 2, -1);
+ if (ret <= 0)
+ break;
+ if (pollfd[1].revents)
+ break;
+ if (!(pollfd[0].revents & POLLIN))
+ break;
+
+ ret = read(args->uffd, &msg, sizeof(msg));
+ if (ret != sizeof(msg))
+ break;
+
+ if (msg.event != UFFD_EVENT_PAGEFAULT)
+ break;
+
+ /* Fill the page with a known pattern */
+ memset(args->page_buffer, 0xAB, args->page_size);
+
+ copy.dst = msg.arg.pagefault.address & ~(args->page_size - 1);
+ copy.src = (unsigned long)args->page_buffer;
+ copy.len = args->page_size;
+ copy.mode = 0;
+ copy.copy = 0;
+
+ ret = ioctl(args->uffd, UFFDIO_COPY, ©);
+ if (ret < 0)
+ break;
+ }
+
+ return NULL;
+}
+
+TEST_F(hmm, userfaultfd_read)
+{
+ struct hmm_buffer *buffer;
+ struct uffd_thread_args uffd_args;
+ unsigned long npages;
+ unsigned long size;
+ unsigned long i;
+ unsigned char *ptr;
+ pthread_t thread;
+ int uffd;
+ int stop_fd;
+ int ret;
+ struct uffdio_api api;
+ struct uffdio_register reg;
+ uint64_t stop = 1;
+ ssize_t nwrite;
+
+ npages = 4;
+ size = npages << self->page_shift;
+
+ /* Create userfaultfd */
+ uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK);
+ if (uffd < 0)
+ SKIP(return, "userfaultfd not available");
+
+ api.api = UFFD_API;
+ api.features = 0;
+ ret = ioctl(uffd, UFFDIO_API, &api);
+ ASSERT_EQ(ret, 0);
+
+ buffer = malloc(sizeof(*buffer));
+ ASSERT_NE(buffer, NULL);
+
+ buffer->fd = -1;
+ buffer->size = size;
+ buffer->mirror = malloc(size);
+ ASSERT_NE(buffer->mirror, NULL);
+
+ /* Create anonymous mapping */
+ buffer->ptr = mmap(NULL, size,
+ PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS,
+ -1, 0);
+ ASSERT_NE(buffer->ptr, MAP_FAILED);
+
+ /* Register the region with userfaultfd */
+ reg.range.start = (unsigned long)buffer->ptr;
+ reg.range.len = size;
+ reg.mode = UFFDIO_REGISTER_MODE_MISSING;
+ ret = ioctl(uffd, UFFDIO_REGISTER, ®);
+ ASSERT_EQ(ret, 0);
+
+ /* Set up the handler thread */
+ uffd_args.uffd = uffd;
+ stop_fd = eventfd(0, EFD_CLOEXEC);
+ ASSERT_GE(stop_fd, 0);
+ uffd_args.stop_fd = stop_fd;
+ uffd_args.page_buffer = malloc(self->page_size);
+ ASSERT_NE(uffd_args.page_buffer, NULL);
+ uffd_args.page_size = self->page_size;
+
+ ret = pthread_create(&thread, NULL, uffd_handler_thread, &uffd_args);
+ ASSERT_EQ(ret, 0);
+
+ /*
+ * Use the unlocked read path which allows the mmap lock to be
+ * dropped during the fault, enabling userfaultfd resolution.
+ */
+ ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_READ_UNLOCKED,
+ buffer, npages);
+ ASSERT_EQ(ret, 0);
+ ASSERT_EQ(buffer->cpages, npages);
+
+ /* Verify the device read the data filled by the uffd handler */
+ ptr = buffer->mirror;
+ for (i = 0; i < size; ++i)
+ ASSERT_EQ(ptr[i], (unsigned char)0xAB);
+
+ nwrite = write(stop_fd, &stop, sizeof(stop));
+ ASSERT_EQ(nwrite, sizeof(stop));
+ pthread_join(thread, NULL);
+ close(stop_fd);
+ free(uffd_args.page_buffer);
+ close(uffd);
+ hmm_buffer_free(buffer);
+}
+
+
TEST_HARNESS_MAIN
--
2.43.0
^ permalink raw reply related
* [PATCH v10 2/8] mm/hmm: add hmm_range_fault_unlocked_timeout() for mmap lock-drop support
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
hmm_range_fault() requires the caller to hold the mmap read lock for the
duration of the call. This is incompatible with mappings whose fault
handler may release the mmap lock, notably userfaultfd-managed regions,
where handle_mm_fault() can return VM_FAULT_RETRY or VM_FAULT_COMPLETED
after dropping the lock. Drivers that need to populate device page tables
for such mappings have no way to do so today.
Add hmm_range_fault_unlocked_timeout() for callers that do not need to hold
mmap_lock across any work outside the HMM fault itself. The helper takes
mmap_read_lock_killable() internally, calls the common HMM fault
implementation, and releases the lock before returning if it is still held.
The timeout is specified in jiffies; passing 0 retries indefinitely, while
a non-zero timeout makes the helper return -EBUSY when the retry budget
expires. The retry deadline is set before refreshing the notifier sequence
and acquiring mmap_lock, so contended mmap_lock acquisition is included in
the retry budget.
When handle_mm_fault() drops mmap_lock, or when the range is invalidated,
hmm_range_fault_unlocked_timeout() refreshes range->notifier_seq and
retries the walk internally. If the lock was dropped, the retry deadline is
also restarted because a lock-dropping fault handler made progress.
Ordinary -EBUSY retries keep the existing deadline, preserving the caller's
timeout policy for repeated mmu-notifier invalidations.
The caller only needs to perform the usual post-success
mmu_interval_read_retry() check while holding its update lock before
consuming the pfns. If mmap_lock acquisition is interrupted or a fatal
signal is pending during retry handling, -EINTR is returned instead.
The common implementation conditionally sets FAULT_FLAG_ALLOW_RETRY and
FAULT_FLAG_KILLABLE only for hmm_range_fault_unlocked_timeout(). The
existing hmm_range_fault() path still passes no locked state, does not
allow handle_mm_fault() to drop mmap_lock, and remains a thin wrapper
preserving the existing API contract for current callers.
The previous refactor that moved page fault handling out of the page-table
walk callbacks is what makes this change small. Faults now run after
walk_page_range() has unwound, with only mmap_lock held, so dropping it
does not interact with the walker's pte spinlock or hugetlb_vma_lock.
Hugetlb regions therefore participate in the unlocked path uniformly with
PTE- and PMD-level mappings; no special case is required.
Documentation/mm/hmm.rst is updated with a description of the new API and
the recommended caller pattern.
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
Documentation/mm/hmm.rst | 79 ++++++++++++++++-------
include/linux/hmm.h | 2 +
mm/hmm.c | 162 ++++++++++++++++++++++++++++++++++++++---------
3 files changed, 191 insertions(+), 52 deletions(-)
diff --git a/Documentation/mm/hmm.rst b/Documentation/mm/hmm.rst
index 7d61b7a8b65b..e021218ada58 100644
--- a/Documentation/mm/hmm.rst
+++ b/Documentation/mm/hmm.rst
@@ -156,42 +156,57 @@ During the ops->invalidate() callback the device driver must perform the
update action to the range (mark range read only, or fully unmap, etc.). The
device must complete the update before the driver callback returns.
-When the device driver wants to populate a range of virtual addresses, it can
-use::
+When the device driver wants to populate a range of virtual addresses, the
+normal interface is::
- int hmm_range_fault(struct hmm_range *range);
+ int hmm_range_fault_unlocked_timeout(struct hmm_range *range,
+ unsigned long timeout);
It will trigger a page fault on missing or read-only entries if write access is
requested (see below). Page faults use the generic mm page fault code path just
-like a CPU page fault. The usage pattern is::
+like a CPU page fault.
+
+The caller must not hold ``mmap_read_lock`` before the call.
+``hmm_range_fault_unlocked_timeout()`` takes the mmap read lock internally and
+allows ``handle_mm_fault()`` to drop it during fault handling. This is required
+for VMAs whose fault handlers may release the mmap lock, for example regions
+managed by ``userfaultfd``.
+
+If the mmap lock is dropped or the range is invalidated, the function refreshes
+``range->notifier_seq`` and restarts the walk internally. ``-EINTR`` is returned
+if mmap lock acquisition is interrupted or a fatal signal is pending during
+retry handling.
+
+The timeout is specified in jiffies; passing ``0`` means retry indefinitely. The
+timeout exists to preserve caller policy for repeated mmu-notifier invalidation
+and is checked between retry attempts. HMM does not interrupt page fault
+handling when the timeout expires, but returns ``-EBUSY`` if the retry budget is
+exhausted before a stable range is obtained.
+
+The usage pattern is::
int driver_populate_range(...)
{
struct hmm_range range;
+ unsigned long timeout;
...
+ timeout = msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT);
range.notifier = &interval_sub;
range.start = ...;
range.end = ...;
range.hmm_pfns = ...;
- if (!mmget_not_zero(interval_sub->notifier.mm))
+ if (!mmget_not_zero(interval_sub.mm))
return -EFAULT;
again:
- range.notifier_seq = mmu_interval_read_begin(&interval_sub);
- mmap_read_lock(mm);
- ret = hmm_range_fault(&range);
- if (ret) {
- mmap_read_unlock(mm);
- if (ret == -EBUSY)
- goto again;
- return ret;
- }
- mmap_read_unlock(mm);
+ ret = hmm_range_fault_unlocked_timeout(&range, timeout);
+ if (ret)
+ goto out_put;
take_lock(driver->update);
- if (mmu_interval_read_retry(&ni, range.notifier_seq) {
+ if (mmu_interval_read_retry(range.notifier, range.notifier_seq)) {
release_lock(driver->update);
goto again;
}
@@ -200,13 +215,31 @@ like a CPU page fault. The usage pattern is::
* under the update lock */
release_lock(driver->update);
- return 0;
+ ret = 0;
+
+ out_put:
+ mmput(interval_sub.mm);
+ return ret;
}
The driver->update lock is the same lock that the driver takes inside its
invalidate() callback. That lock must be held before calling
mmu_interval_read_retry() to avoid any race with a concurrent CPU page table
-update.
+update. The retry check must use the same notifier and sequence number stored
+in ``range`` by ``hmm_range_fault_unlocked_timeout()``.
+
+Holding the mmap lock across HMM faults
+=======================================
+
+Most callers should use ``hmm_range_fault_unlocked_timeout()``. If a driver
+really needs to hold the mmap lock across work outside HMM, it can use::
+
+ int hmm_range_fault(struct hmm_range *range);
+
+The mmap lock must be held by the caller and will remain held on return. This
+interface cannot support VMAs whose fault handlers need to drop the mmap lock.
+New callers should prefer ``hmm_range_fault_unlocked_timeout()`` unless they
+have a specific requirement to keep the mmap lock held across the call.
Leverage default_flags and pfn_flags_mask
=========================================
@@ -221,8 +254,8 @@ permission, it sets::
range->default_flags = HMM_PFN_REQ_FAULT;
range->pfn_flags_mask = 0;
-and calls hmm_range_fault() as described above. This will fill fault all pages
-in the range with at least read permission.
+and calls the HMM range fault helper as described above. This will fault
+all pages in the range with at least read permission.
Now let's say the driver wants to do the same except for one page in the range for
which it wants to have write permission. Now driver set::
@@ -236,9 +269,9 @@ address == range->start + (index_of_write << PAGE_SHIFT) it will fault with
write permission i.e., if the CPU pte does not have write permission set then HMM
will call handle_mm_fault().
-After hmm_range_fault completes the flag bits are set to the current state of
-the page tables, ie HMM_PFN_VALID | HMM_PFN_WRITE will be set if the page is
-writable.
+After the HMM range fault helper completes the flag bits are set to the
+current state of the page tables, ie HMM_PFN_VALID | HMM_PFN_WRITE will be
+set if the page is writable.
Represent and manage device memory from core kernel point of view
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index db75ffc949a7..6f04e3932f5b 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -123,6 +123,8 @@ struct hmm_range {
* Please see Documentation/mm/hmm.rst for how to use the range API.
*/
int hmm_range_fault(struct hmm_range *range);
+int hmm_range_fault_unlocked_timeout(struct hmm_range *range,
+ unsigned long timeout);
/*
* HMM_RANGE_DEFAULT_TIMEOUT - default timeout (ms) when waiting for a range
diff --git a/mm/hmm.c b/mm/hmm.c
index bc9361a715fa..87952c1259c5 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -32,6 +32,7 @@
struct hmm_vma_walk {
struct hmm_range *range;
+ bool *locked;
unsigned long last;
unsigned long end;
unsigned int required_fault;
@@ -44,6 +45,14 @@ struct hmm_vma_walk {
*/
#define HMM_FAULT_PENDING -EAGAIN
+/*
+ * Internal sentinel returned by hmm_do_fault() when handle_mm_fault()
+ * completes a page fault with the mmap lock dropped. hmm_do_fault() sets
+ * *locked = false; the outer loop consumes the sentinel and never propagates
+ * it to the caller.
+ */
+#define HMM_FAULT_UNLOCKED -ENOLCK
+
enum {
HMM_NEED_FAULT = 1 << 0,
HMM_NEED_WRITE_FAULT = 1 << 1,
@@ -73,9 +82,9 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end,
*
* Called by the walk callbacks when they discover that part of the range
* needs a page fault. The callback records what to fault and returns
- * HMM_FAULT_PENDING; the outer loop in hmm_range_fault() drops back out of
- * walk_page_range() and invokes handle_mm_fault() from a context where no
- * page-table or hugetlb_vma_lock is held.
+ * HMM_FAULT_PENDING; the outer loop in hmm_range_fault_locked() drops
+ * back out of walk_page_range() and invokes handle_mm_fault() from a context
+ * where no page-table or hugetlb_vma_lock is held.
*/
static int hmm_record_fault(unsigned long addr, unsigned long end,
unsigned int required_fault,
@@ -624,7 +633,7 @@ static const struct mm_walk_ops hmm_walk_ops = {
/*
* hmm_do_fault - fault in a range recorded by a walk callback
*
- * Called from the outer loop in hmm_range_fault() after a callback
+ * Called from the outer loop in hmm_range_fault_locked() after a callback
* returned HMM_FAULT_PENDING. At this point we hold only mmap_lock;
* the page-table spinlock and any hugetlb_vma_lock acquired by the walk
* framework have already been released by the unwind.
@@ -641,6 +650,9 @@ static int hmm_do_fault(struct mm_struct *mm,
unsigned int fault_flags = FAULT_FLAG_REMOTE;
struct vm_area_struct *vma;
+ if (hmm_vma_walk->locked)
+ fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+
vma = vma_lookup(mm, addr);
if (!vma)
return -EFAULT;
@@ -651,37 +663,34 @@ static int hmm_do_fault(struct mm_struct *mm,
fault_flags |= FAULT_FLAG_WRITE;
}
- for (; addr < end; addr += PAGE_SIZE)
- if (handle_mm_fault(vma, addr, fault_flags, NULL) &
- VM_FAULT_ERROR)
- return -EFAULT;
+ for (; addr < end; addr += PAGE_SIZE) {
+ vm_fault_t ret;
+
+ ret = handle_mm_fault(vma, addr, fault_flags, NULL);
+
+ if (ret & (VM_FAULT_COMPLETED | VM_FAULT_RETRY)) {
+ *hmm_vma_walk->locked = false;
+ return HMM_FAULT_UNLOCKED;
+ }
+
+ if (ret & VM_FAULT_ERROR) {
+ int err = vm_fault_to_errno(ret, 0);
+
+ if (WARN_ON(!err))
+ err = -EINVAL;
+
+ return err;
+ }
+ }
return -EBUSY;
}
-/**
- * hmm_range_fault - try to fault some address in a virtual address range
- * @range: argument structure
- *
- * Returns 0 on success or one of the following error codes:
- *
- * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma
- * (e.g., device file vma).
- * -ENOMEM: Out of memory.
- * -EPERM: Invalid permission (e.g., asking for write and range is read
- * only).
- * -EBUSY: The range has been invalidated and the caller needs to wait for
- * the invalidation to finish.
- * -EFAULT: A page was requested to be valid and could not be made valid
- * ie it has no backing VMA or it is illegal to access
- *
- * This is similar to get_user_pages(), except that it can read the page tables
- * without mutating them (ie causing faults).
- */
-int hmm_range_fault(struct hmm_range *range)
+static int hmm_range_fault_locked(struct hmm_range *range, bool *locked)
{
struct hmm_vma_walk hmm_vma_walk = {
.range = range,
+ .locked = locked,
.last = range->start,
};
struct mm_struct *mm = range->notifier->mm;
@@ -704,8 +713,14 @@ int hmm_range_fault(struct hmm_range *range)
* returns -EBUSY so the loop re-walks and picks up the
* now-present entries.
*/
- if (ret == HMM_FAULT_PENDING)
+ if (ret == HMM_FAULT_PENDING) {
ret = hmm_do_fault(mm, &hmm_vma_walk);
+ if (ret == HMM_FAULT_UNLOCKED) {
+ if (fatal_signal_pending(current))
+ return -EINTR;
+ return -EBUSY;
+ }
+ }
/*
* When -EBUSY is returned the loop restarts with
* hmm_vma_walk.last set to an address that has not been stored
@@ -715,8 +730,97 @@ int hmm_range_fault(struct hmm_range *range)
} while (ret == -EBUSY);
return ret;
}
+
+/**
+ * hmm_range_fault - try to fault some address in a virtual address range
+ * @range: argument structure
+ *
+ * Returns 0 on success or one of the following error codes:
+ *
+ * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma
+ * (e.g., device file vma).
+ * -ENOMEM: Out of memory.
+ * -EPERM: Invalid permission (e.g., asking for write and range is read
+ * only).
+ * -EBUSY: The range has been invalidated and the caller needs to wait for
+ * the invalidation to finish.
+ * -EFAULT: A page was requested to be valid and could not be made valid
+ * ie it has no backing VMA or it is illegal to access
+ *
+ * This is similar to get_user_pages(), except that it can read the page tables
+ * without mutating them (ie causing faults).
+ *
+ * The mmap lock must be held by the caller and will remain held on return.
+ * New users should prefer hmm_range_fault_unlocked_timeout() unless they
+ * specifically need to keep the mmap lock held across the call. This helper
+ * cannot support VMAs whose fault handlers need to drop the mmap lock.
+ */
+int hmm_range_fault(struct hmm_range *range)
+{
+ return hmm_range_fault_locked(range, NULL);
+}
EXPORT_SYMBOL(hmm_range_fault);
+/**
+ * hmm_range_fault_unlocked_timeout - fault in a range with a retry timeout
+ * @range: argument structure
+ * @timeout: timeout in jiffies for internal -EBUSY retries, or 0 to retry
+ * indefinitely
+ *
+ * The caller must not hold the mmap lock. The function takes the mmap read
+ * lock internally and allows handle_mm_fault() to drop it during faults. If
+ * the mmap lock is dropped or the range is invalidated, the function refreshes
+ * range->notifier_seq and restarts the walk internally.
+ *
+ * Passing 0 for @timeout retries indefinitely. A non-zero @timeout is a caller
+ * policy limit for repeated mmu-notifier invalidation retries. HMM does not
+ * interrupt page fault handling when the timeout expires, but returns -EBUSY
+ * if the retry budget is exhausted before a stable range is obtained.
+ *
+ * Returns 0 on success or one of the error codes documented for
+ * hmm_range_fault(). -EINTR is returned if mmap_lock acquisition is
+ * interrupted or a fatal signal is pending during retry handling.
+ */
+int hmm_range_fault_unlocked_timeout(struct hmm_range *range,
+ unsigned long timeout)
+{
+ struct mm_struct *mm = range->notifier->mm;
+ unsigned long deadline = 0;
+ bool locked = false;
+ int ret;
+
+ do {
+ /*
+ * If the previous fault dropped mmap_lock, then the fault
+ * handler made progress. Restart the retry timeout in that
+ * case, but keep the existing deadline for ordinary -EBUSY
+ * retries.
+ */
+ if (timeout && !locked)
+ deadline = jiffies + timeout;
+
+ range->notifier_seq =
+ mmu_interval_read_begin(range->notifier);
+
+ ret = mmap_read_lock_killable(mm);
+ if (ret)
+ return ret;
+
+ if (timeout && time_after(jiffies, deadline)) {
+ mmap_read_unlock(mm);
+ return -EBUSY;
+ }
+
+ locked = true;
+ ret = hmm_range_fault_locked(range, &locked);
+ if (locked)
+ mmap_read_unlock(mm);
+ } while (ret == -EBUSY);
+
+ return ret;
+}
+EXPORT_SYMBOL(hmm_range_fault_unlocked_timeout);
+
/**
* hmm_dma_map_alloc - Allocate HMM map structure
* @dev: device to allocate structure for
--
2.43.0
^ permalink raw reply related
* [PATCH v10 1/8] mm/hmm: move page fault handling out of walk callbacks
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
In-Reply-To: <20260722-hmm-v10-v1-0-606464dd601a@gmail.com>
hmm_range_fault() currently triggers page faults from inside the page-table
walk callbacks: hmm_vma_walk_pmd(), hmm_vma_walk_pud(),
hmm_vma_walk_hugetlb_entry() and the pte-level helper all call
hmm_vma_fault(), which in turn calls handle_mm_fault() while the walker
still holds nested locks. The pte spinlock is dropped explicitly by each
caller, and the hugetlb path manually drops and retakes
hugetlb_vma_lock_read around the fault to dodge a deadlock against the walk
framework's unconditional unlock.
This layering does not extend cleanly to fault handlers that may release
mmap_lock (VM_FAULT_RETRY, VM_FAULT_COMPLETED). If the lock is dropped
while walk_page_range() is mid-traversal, the VMA can be freed before the
walk framework's matching hugetlb_vma_unlock_read(), turning that unlock
into a use-after-free.
Split the responsibilities the way get_user_pages() does. Walk callbacks
become inspect-only: when they detect a range that needs to be faulted in,
they record it in struct hmm_vma_walk and return a private sentinel
(HMM_FAULT_PENDING). The outer loop in hmm_range_fault() then drops out of
walk_page_range(), invokes a new helper hmm_do_fault() that calls
handle_mm_fault() with only mmap_lock held, and restarts the walk so the
now-present entries are collected into hmm_pfns.
No functional change for existing callers. As a side effect the hugetlb
callback no longer needs the hugetlb_vma_{un}lock_read dance, and every
fault-path exit from the callbacks now releases the pte spinlock on a
single, common path. This refactor is also a precursor for adding an
unlockable variant of hmm_range_fault() in a follow-up patch.
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
---
mm/hmm.c | 118 ++++++++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 75 insertions(+), 43 deletions(-)
diff --git a/mm/hmm.c b/mm/hmm.c
index e5c1f4deed24..bc9361a715fa 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -33,8 +33,17 @@
struct hmm_vma_walk {
struct hmm_range *range;
unsigned long last;
+ unsigned long end;
+ unsigned int required_fault;
};
+/*
+ * Internal sentinel returned by walk callbacks when they need a page fault.
+ * The callback stores end/required_fault in hmm_vma_walk; the outer loop
+ * consumes the sentinel and never propagates it to the caller.
+ */
+#define HMM_FAULT_PENDING -EAGAIN
+
enum {
HMM_NEED_FAULT = 1 << 0,
HMM_NEED_WRITE_FAULT = 1 << 1,
@@ -60,37 +69,25 @@ static int hmm_pfns_fill(unsigned long addr, unsigned long end,
}
/*
- * hmm_vma_fault() - fault in a range lacking valid pmd or pte(s)
- * @addr: range virtual start address (inclusive)
- * @end: range virtual end address (exclusive)
- * @required_fault: HMM_NEED_* flags
- * @walk: mm_walk structure
- * Return: -EBUSY after page fault, or page fault error
+ * hmm_record_fault() - record a range that needs to be faulted in
*
- * This function will be called whenever pmd_none() or pte_none() returns true,
- * or whenever there is no page directory covering the virtual address range.
+ * Called by the walk callbacks when they discover that part of the range
+ * needs a page fault. The callback records what to fault and returns
+ * HMM_FAULT_PENDING; the outer loop in hmm_range_fault() drops back out of
+ * walk_page_range() and invokes handle_mm_fault() from a context where no
+ * page-table or hugetlb_vma_lock is held.
*/
-static int hmm_vma_fault(unsigned long addr, unsigned long end,
- unsigned int required_fault, struct mm_walk *walk)
+static int hmm_record_fault(unsigned long addr, unsigned long end,
+ unsigned int required_fault,
+ struct mm_walk *walk)
{
struct hmm_vma_walk *hmm_vma_walk = walk->private;
- struct vm_area_struct *vma = walk->vma;
- unsigned int fault_flags = FAULT_FLAG_REMOTE;
WARN_ON_ONCE(!required_fault);
hmm_vma_walk->last = addr;
-
- if (required_fault & HMM_NEED_WRITE_FAULT) {
- if (!(vma->vm_flags & VM_WRITE))
- return -EPERM;
- fault_flags |= FAULT_FLAG_WRITE;
- }
-
- for (; addr < end; addr += PAGE_SIZE)
- if (handle_mm_fault(vma, addr, fault_flags, NULL) &
- VM_FAULT_ERROR)
- return -EFAULT;
- return -EBUSY;
+ hmm_vma_walk->end = end;
+ hmm_vma_walk->required_fault = required_fault;
+ return HMM_FAULT_PENDING;
}
static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_walk,
@@ -174,7 +171,7 @@ static int hmm_vma_walk_hole(unsigned long addr, unsigned long end,
return hmm_pfns_fill(addr, end, range, HMM_PFN_ERROR);
}
if (required_fault)
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
return hmm_pfns_fill(addr, end, range, 0);
}
@@ -209,7 +206,7 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr,
required_fault =
hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, cpu_flags);
if (required_fault)
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT);
for (i = 0; addr < end; addr += PAGE_SIZE, i++, pfn++) {
@@ -328,7 +325,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr,
fault:
pte_unmap(ptep);
/* Fault any virtual address we were asked to fault */
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
}
#ifdef CONFIG_ARCH_HAS_PMD_SOFTLEAVES
@@ -371,7 +368,7 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *walk, unsigned long start,
npages, 0);
if (required_fault) {
if (softleaf_is_device_private(entry))
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
else
return -EFAULT;
}
@@ -517,7 +514,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end,
npages, cpu_flags);
if (required_fault) {
spin_unlock(ptl);
- return hmm_vma_fault(addr, end, required_fault, walk);
+ return hmm_record_fault(addr, end, required_fault, walk);
}
pfn = pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT);
@@ -564,21 +561,8 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,
required_fault =
hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags);
if (required_fault) {
- int ret;
-
spin_unlock(ptl);
- hugetlb_vma_unlock_read(vma);
- /*
- * Avoid deadlock: drop the vma lock before calling
- * hmm_vma_fault(), which will itself potentially take and
- * drop the vma lock. This is also correct from a
- * protection point of view, because there is no further
- * use here of either pte or ptl after dropping the vma
- * lock.
- */
- ret = hmm_vma_fault(addr, end, required_fault, walk);
- hugetlb_vma_lock_read(vma);
- return ret;
+ return hmm_record_fault(addr, end, required_fault, walk);
}
pfn = pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT);
@@ -637,6 +621,44 @@ static const struct mm_walk_ops hmm_walk_ops = {
.walk_lock = PGWALK_RDLOCK,
};
+/*
+ * hmm_do_fault - fault in a range recorded by a walk callback
+ *
+ * Called from the outer loop in hmm_range_fault() after a callback
+ * returned HMM_FAULT_PENDING. At this point we hold only mmap_lock;
+ * the page-table spinlock and any hugetlb_vma_lock acquired by the walk
+ * framework have already been released by the unwind.
+ *
+ * Returns -EBUSY on success (all pages faulted, caller should re-walk).
+ * Returns a negative errno on failure.
+ */
+static int hmm_do_fault(struct mm_struct *mm,
+ struct hmm_vma_walk *hmm_vma_walk)
+{
+ unsigned long addr = hmm_vma_walk->last;
+ unsigned long end = hmm_vma_walk->end;
+ unsigned int required_fault = hmm_vma_walk->required_fault;
+ unsigned int fault_flags = FAULT_FLAG_REMOTE;
+ struct vm_area_struct *vma;
+
+ vma = vma_lookup(mm, addr);
+ if (!vma)
+ return -EFAULT;
+
+ if (required_fault & HMM_NEED_WRITE_FAULT) {
+ if (!(vma->vm_flags & VM_WRITE))
+ return -EPERM;
+ fault_flags |= FAULT_FLAG_WRITE;
+ }
+
+ for (; addr < end; addr += PAGE_SIZE)
+ if (handle_mm_fault(vma, addr, fault_flags, NULL) &
+ VM_FAULT_ERROR)
+ return -EFAULT;
+
+ return -EBUSY;
+}
+
/**
* hmm_range_fault - try to fault some address in a virtual address range
* @range: argument structure
@@ -674,6 +696,16 @@ int hmm_range_fault(struct hmm_range *range)
return -EBUSY;
ret = walk_page_range(mm, hmm_vma_walk.last, range->end,
&hmm_walk_ops, &hmm_vma_walk);
+ /*
+ * When HMM_FAULT_PENDING is returned a walk callback
+ * recorded a range that needs handle_mm_fault();
+ * hmm_do_fault() runs the fault outside walk_page_range()
+ * (so no page-table or hugetlb_vma_lock is held) and
+ * returns -EBUSY so the loop re-walks and picks up the
+ * now-present entries.
+ */
+ if (ret == HMM_FAULT_PENDING)
+ ret = hmm_do_fault(mm, &hmm_vma_walk);
/*
* When -EBUSY is returned the loop restarts with
* hmm_vma_walk.last set to an address that has not been stored
--
2.43.0
^ permalink raw reply related
* [PATCH v10 0/8] mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings
From: Stanislav Kinsburskii @ 2026-07-22 21:44 UTC (permalink / raw)
To: Jason Gunthorpe, Leon Romanovsky, Andrew Morton,
David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Jonathan Corbet, Shuah Khan, Shuah Khan, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Lyude Paul,
Danilo Krummrich, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Min Ma, Lizhi Hou,
Oded Gabbay, skinsburskii
Cc: linux-mm, linux-kernel, linux-doc, linux-kselftest, linux-hyperv,
dri-devel, nouveau, linux-rdma, Jason Gunthorpe
This series extends the HMM framework to support userfaultfd-backed memory
by allowing the mmap read lock to be dropped during hmm_range_fault().
Some page fault handlers — most notably userfaultfd — require the mmap lock
to be released so that userspace can resolve the fault. The current HMM
interface never sets FAULT_FLAG_ALLOW_RETRY, making it impossible to fault
in pages from userfaultfd-registered regions.
This series follows the established int *locked pattern from
get_user_pages_remote() in mm/gup.c. A new helper function,
hmm_range_fault_locked(), accepts an int *locked parameter. When the
mmap lock is dropped during fault resolution (VM_FAULT_RETRY or
VM_FAULT_COMPLETED), the function returns 0 with *locked = 0, signalling
the caller to restart its walk. The existing hmm_range_fault() is
refactored into a thin wrapper that passes NULL, preserving current
behavior for all existing callers.
Possible approaches to lift this limitation are documented in
Documentation/mm/hmm.rst.
Changes in v10:
- Included contended mmap_lock acquisition in the
hmm_range_fault_unlocked_timeout() retry budget.
- Dropped the redundant top-level fatal_signal_pending() check in the HMM
unlocked retry loop; mmap_read_lock_killable() now covers that path.
- Restored the absolute outer timeout in drm_gpusvm_get_pages(), since it can
run from GPU page-fault workers and must not rely on the worker task’s fatal
signal state to stop invalidation retries.
Changes in v9:
- Folded the fixups into the full 8-patch series instead of sending a
separate fixup series.
- Clarified that the HMM timeout bounds repeated HMM/mmu-notifier retry
attempts, with the helper refreshing range->notifier_seq internally.
- Kept nouveau’s explicit outer deadline because its retry loop runs in a GPU
fault worker and cannot rely on fatal signals from the faulting process.
- Converted amdxdna, and GPU SVM callers to pass the shole timeout budget to
hmm_range_fault_unlocked_timeout().
Changes in v8:
- Make hmm_range_fault_unlocked_timeout() the primary documented HMM
range-fault API, and move hmm_range_fault() into the “use only if the
caller really must hold mmap_lock” category.
- Clarify that the timeout is a retry budget for repeated mmu-notifier
invalidation retries. HMM does not interrupt an in-progress page fault
when the timeout expires.
- Restart the retry timeout only when handle_mm_fault() dropped
mmap_lock, because that indicates a lock-dropping fault handler such as
userfaultfd made progress. Ordinary -EBUSY retries keep the existing
deadline.
- Remove the attempted timeout selftest. The remaining selftest covers the
intended userfaultfd path by resolving missing-page faults through
HMM_DMIRROR_READ_UNLOCKED and hmm_range_fault_unlocked_timeout(..., 0).
Changes in v7:
- Replaced the unlocked HMM API with
hmm_range_fault_unlocked_timeout(). The helper now takes a timeout in
jiffies, with 0 meaning retry indefinitely.
- Moved -EBUSY retry handling into the HMM helper for the unlocked path.
The helper refreshes range->notifier_seq internally before each retry.
- Switched the unlocked path to mmap_read_lock_killable() and return
-EINTR if mmap lock acquisition is interrupted or a fatal signal is
pending during retry handling.
- Removed the redundant non-timeout hmm_range_fault_unlocked() interface.
- Updated Documentation/mm/hmm.rst and kernel-doc to describe the timeout API
and the intended caller pattern.
- Updated the HMM selftests to use hmm_range_fault_unlocked_timeout()
only, including coverage for the finite-timeout path.
- Added in-tree users of the new helper:
- mshv
- nouveau
- RDMA/umem
- amdxdna
- drm/gpusvm
- Preserved each converted driver’s existing timeout convention:
- unbounded retry where the old code retried indefinitely,
- HMM_RANGE_DEFAULT_TIMEOUT where the old code used that budget,
- existing driver-specific timeout return values such as -ETIME.
- Used max_t(long, timeout - jiffies, 1) when passing remaining time from
absolute jiffies deadlines to avoid unsigned underflow while keeping a
minimum one-jiffy retry window.
- Left callers on hmm_range_fault() when they already need to hold
mmap_lock across surrounding work, such as drm_gpusvm_check_pages().
Changes in v6:
- Reworked the new API from the external int *locked pattern to
hmm_range_fault_unlocked(), which owns mmap_read_lock() internally.
- Changed the dropped-lock contract: hmm_range_fault_unlocked() now returns
-EBUSY when the mmap lock is dropped, and callers restart with a fresh
mmu_interval_read_begin() sequence.
- Kept hmm_range_fault() as the locked variant for existing users, preserving
its caller-held mmap lock contract.
- Added an in-tree user by converting the MSHV region fault path to
hmm_range_fault_unlocked().
- Updated Documentation/mm/hmm.rst and kernel-doc to describe the unlocked
helper and retry pattern.
- Updated commit messages to match the new API and return semantics.
- Kept the userfaultfd HMM selftest using the test_hmm unlocked read ioctl
path.
Changes in v5:
- Rework hmm_range_fault_unlockable() retry handling to retry
VM_FAULT_RETRY internally with FAULT_FLAG_TRIED set, matching the
fixup_user_fault() pattern and avoiding repeated first-retry lock drops.
- Distinguish VM_FAULT_RETRY from VM_FAULT_COMPLETED: retry faults now
reacquire the mmap lock internally, while completed faults return to the
caller with *locked = 0 so the caller can restart with a fresh notifier
sequence.
- Document the two *locked return states, including the -EINTR case when a
fatal signal is pending after the mmap lock has already been dropped.
- Update comments around HMM_FAULT_UNLOCKED and the HMM fault loop to match
the current hmm_range_fault_unlockable() implementation.
Changes in v4:
- Rebased on 7.2-rc1
Changes in v3:
- Return -EFAULT from dmirror_fault_unlockable() when the mirrored mm can
no longer be pinned.
- Add an eventfd stop signal for the userfaultfd handler thread to avoid
waiting for the poll timeout on successful test completion.
Changes in v2:
- Split into a preparatory refactor (new patch 1) that moves
handle_mm_fault() out of the walk callbacks, plus a smaller feature
patch on top. Suggested by David Hildenbrand.
- Hugetlb regions are now supported on the unlockable path; the v1
-EFAULT short-circuit and the hugetlb_vma_lock_read drop/retake
dance are gone.
- Distinct internal sentinels for "needs fault" (HMM_FAULT_PENDING)
and "lock dropped" (HMM_FAULT_UNLOCKED).
- Outer loop now re-walks after a successful internal fault so the
faulted pfns end up in range->hmm_pfns.
- Kernel-doc on hmm_range_fault_unlockable() and the
Documentation/mm/hmm.rst example match the implementation.
- Dropped the mshv driver conversion (v1 patch 2); will post
separately.
- Selftest converted to drive the path through test_hmm with a
userfaultfd handler (new HMM_DMIRROR_READ_UNLOCKABLE ioctl).
---
Stanislav Kinsburskii (8):
mm/hmm: move page fault handling out of walk callbacks
mm/hmm: add hmm_range_fault_unlocked_timeout() for mmap lock-drop support
selftests/mm: add HMM test for mmap lock-dropping faults
mshv: Use hmm_range_fault_unlocked_timeout() for region faults
drm/nouveau: Use hmm_range_fault_unlocked_timeout() for SVM faults
RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults
accel/amdxdna: Use hmm_range_fault_unlocked_timeout() for range population
drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults
Documentation/mm/hmm.rst | 79 +++++++---
drivers/accel/amdxdna/aie2_ctx.c | 23 +--
drivers/gpu/drm/drm_gpusvm.c | 60 ++------
drivers/gpu/drm/nouveau/nouveau_svm.c | 20 +--
drivers/hv/mshv_regions.c | 54 ++-----
drivers/infiniband/core/umem_odp.c | 18 +--
include/linux/hmm.h | 2 +
lib/test_hmm.c | 107 +++++++++++++-
lib/test_hmm_uapi.h | 1 +
mm/hmm.c | 256 +++++++++++++++++++++++++--------
tools/testing/selftests/mm/hmm-tests.c | 150 +++++++++++++++++++
11 files changed, 551 insertions(+), 219 deletions(-)
---
base-commit: a4271215228d2bd2a50eea03ab0a948b36036948
change-id: 20260722-hmm-v10-727db2bb9d34
Best regards,
--
Stanislav Kinsburskii <skinsburskii@gmail.com>
^ permalink raw reply
* Re: [PATCH v2] mshv: Add dedicated ioctl for GVA to GPA translation
From: Wei Liu @ 2026-07-22 21:41 UTC (permalink / raw)
To: anirudh, paekkaladevi, avchaurasia
Cc: kys, haiyangz, wei.liu, decui, longli, linux-hyperv, linux-kernel
In-Reply-To: <177750046968.208629.5511024136301980968.stgit@skinsburskii-cloud-desktop.internal.cloudapp.net>
On Wed, Apr 29, 2026 at 10:08:06PM +0000, Stanislav Kinsburskii wrote:
> Add an MSHV_TRANSLATE_GVA ioctl on the VP fd that wraps
> HVCALL_TRANSLATE_VIRTUAL_ADDRESS_EX with transparent fault-in handling for
> movable memory regions. The passthrough path for this hypercall is retained
> for backward compatibility.
>
> When guest-backing pages reside in movable memory regions, the mmu_notifier
> invalidation path remaps them to NO_ACCESS in the hypervisor's second-level
> address translation tables. If the VMM issues a GVA translation (e.g.
> during MMIO emulation) while a page-table page is invalidated, the
> hypervisor returns HV_TRANSLATE_GVA_GPA_NO_READ_ACCESS. The VMM cannot
> resolve this on its own.
>
Pavan, Avinash and Anirudh, I think this needs further attention.
As we optimize the memory management code, this patch's premise will
likely no longer hold.
Wei
> The new ioctl detects this transient GPA access failure, faults the page
> back in via mshv_region_handle_gfn_fault(), and retries the translation
> until it succeeds or an unrecoverable error occurs.
>
> Signed-off-by: Stanislav Kinsburskii <skinsburskii@linux.microsoft.com>
> ---
> drivers/hv/mshv_root.h | 3 ++
> drivers/hv/mshv_root_hv_call.c | 37 +++++++++++++++++++++
> drivers/hv/mshv_root_main.c | 69 ++++++++++++++++++++++++++++++++++++++++
> include/hyperv/hvgdk_mini.h | 1 +
> include/hyperv/hvhdk.h | 41 ++++++++++++++++++++++++
> include/uapi/linux/mshv.h | 10 ++++++
> 6 files changed, 161 insertions(+)
>
> diff --git a/drivers/hv/mshv_root.h b/drivers/hv/mshv_root.h
> index 1f086dcb7aa1a..2e6c4414740cc 100644
> --- a/drivers/hv/mshv_root.h
> +++ b/drivers/hv/mshv_root.h
> @@ -290,6 +290,9 @@ int hv_call_delete_vp(u64 partition_id, u32 vp_index);
> int hv_call_assert_virtual_interrupt(u64 partition_id, u32 vector,
> u64 dest_addr,
> union hv_interrupt_control control);
> +int hv_call_translate_virtual_address_ex(u32 vp_index, u64 partition_id,
> + u64 flags, u64 gva, u64 *gfn,
> + struct hv_translate_gva_result_ex *result);
> int hv_call_clear_virtual_interrupt(u64 partition_id);
> int hv_call_get_gpa_access_states(u64 partition_id, u32 count, u64 gpa_base_pfn,
> union hv_gpa_page_access_state_flags state_flags,
> diff --git a/drivers/hv/mshv_root_hv_call.c b/drivers/hv/mshv_root_hv_call.c
> index 1f19a4ca824f0..f8c2341193da5 100644
> --- a/drivers/hv/mshv_root_hv_call.c
> +++ b/drivers/hv/mshv_root_hv_call.c
> @@ -677,6 +677,43 @@ int hv_call_get_partition_property_ex(u64 partition_id, u64 property_code,
> return 0;
> }
>
> +int hv_call_translate_virtual_address_ex(u32 vp_index, u64 partition_id,
> + u64 flags, u64 gva, u64 *gfn,
> + struct hv_translate_gva_result_ex *result)
> +{
> + struct hv_input_translate_virtual_address *input;
> + struct hv_output_translate_virtual_address_ex *output;
> + unsigned long irq_flags;
> + u64 status;
> +
> + local_irq_save(irq_flags);
> +
> + input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> + output = *this_cpu_ptr(hyperv_pcpu_output_arg);
> +
> + memset(input, 0, sizeof(*input));
> + input->partition_id = partition_id;
> + input->vp_index = vp_index;
> + input->control_flags = flags;
> + input->gva_page = gva >> HV_HYP_PAGE_SHIFT;
> +
> + status = hv_do_hypercall(HVCALL_TRANSLATE_VIRTUAL_ADDRESS_EX,
> + input, output);
> +
> + if (!hv_result_success(status)) {
> + local_irq_restore(irq_flags);
> + pr_err("%s: %s\n", __func__, hv_result_to_string(status));
> + return hv_result_to_errno(status);
> + }
> +
> + *result = output->translation_result;
> + *gfn = output->gpa_page;
> +
> + local_irq_restore(irq_flags);
> +
> + return 0;
> +}
> +
> int
> hv_call_clear_virtual_interrupt(u64 partition_id)
> {
> diff --git a/drivers/hv/mshv_root_main.c b/drivers/hv/mshv_root_main.c
> index bd1359eb58dd4..665d565899c15 100644
> --- a/drivers/hv/mshv_root_main.c
> +++ b/drivers/hv/mshv_root_main.c
> @@ -898,6 +898,72 @@ mshv_vp_ioctl_get_set_state(struct mshv_vp *vp,
> return 0;
> }
>
> +static bool mshv_gpa_fault_retryable(u32 result_code)
> +{
> + /*
> + * Note: HV_TRANSLATE_GVA_GPA_UNMAPPED is intentionally not handled
> + * here. The guest page table cannot be unmapped under normal
> + * operation. It may be mapped with no access during page moves,
> + * but a truly unmapped state indicates a kernel driver bug.
> + * Retrying in this case would only mask the underlying problem of
> + * an unmapped guest page table.
> + */
> + return result_code == HV_TRANSLATE_GVA_GPA_NO_READ_ACCESS;
> +}
> +
> +static long
> +mshv_vp_ioctl_translate_gva(struct mshv_vp *vp, void __user *user_args)
> +{
> + struct mshv_partition *partition = vp->vp_partition;
> + struct mshv_translate_gva args;
> + struct hv_translate_gva_result_ex result;
> + u64 gfn, gpa;
> + int ret;
> +
> + if (copy_from_user(&args, user_args, sizeof(args)))
> + return -EFAULT;
> +
> + do {
> + ret = hv_call_translate_virtual_address_ex(vp->vp_index,
> + partition->pt_id,
> + args.flags, args.gva,
> + &gfn, &result);
> + if (ret)
> + return ret;
> +
> + if (mshv_gpa_fault_retryable(result.result_code)) {
> + struct mshv_mem_region *region;
> + bool faulted;
> +
> + region = mshv_partition_region_by_gfn_get(partition,
> + gfn);
> + if (!region)
> + return -EFAULT;
> +
> + faulted = false;
> + if (region->mreg_type == MSHV_REGION_TYPE_MEM_MOVABLE)
> + faulted = mshv_region_handle_gfn_fault(region,
> + gfn);
> + mshv_region_put(region);
> +
> + if (!faulted)
> + return -EFAULT;
> +
> + cond_resched();
> + }
> + } while (mshv_gpa_fault_retryable(result.result_code));
> +
> + gpa = (gfn << PAGE_SHIFT) | (args.gva & ~PAGE_MASK);
> +
> + if (copy_to_user(args.result, &result, sizeof(*args.result)))
> + return -EFAULT;
> +
> + if (copy_to_user(args.gpa, &gpa, sizeof(*args.gpa)))
> + return -EFAULT;
> +
> + return 0;
> +}
> +
> static long
> mshv_vp_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
> {
> @@ -917,6 +983,9 @@ mshv_vp_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
> case MSHV_SET_VP_STATE:
> r = mshv_vp_ioctl_get_set_state(vp, (void __user *)arg, true);
> break;
> + case MSHV_TRANSLATE_GVA:
> + r = mshv_vp_ioctl_translate_gva(vp, (void __user *)arg);
> + break;
> case MSHV_ROOT_HVCALL:
> r = mshv_ioctl_passthru_hvcall(vp->vp_partition, false,
> (void __user *)arg);
> diff --git a/include/hyperv/hvgdk_mini.h b/include/hyperv/hvgdk_mini.h
> index 6a4e8b9d570fd..ac901801fd397 100644
> --- a/include/hyperv/hvgdk_mini.h
> +++ b/include/hyperv/hvgdk_mini.h
> @@ -484,6 +484,7 @@ union hv_vp_assist_msr_contents { /* HV_REGISTER_VP_ASSIST_PAGE */
> #define HVCALL_CONNECT_PORT 0x0096
> #define HVCALL_START_VP 0x0099
> #define HVCALL_GET_VP_INDEX_FROM_APIC_ID 0x009a
> +#define HVCALL_TRANSLATE_VIRTUAL_ADDRESS_EX 0x00ac
> #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
> #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
> #define HVCALL_SIGNAL_EVENT_DIRECT 0x00c0
> diff --git a/include/hyperv/hvhdk.h b/include/hyperv/hvhdk.h
> index 5e83d37149662..08eede666762e 100644
> --- a/include/hyperv/hvhdk.h
> +++ b/include/hyperv/hvhdk.h
> @@ -952,4 +952,45 @@ struct hv_input_modify_sparse_spa_page_host_access {
> #define HV_MODIFY_SPA_PAGE_HOST_ACCESS_LARGE_PAGE 0x4
> #define HV_MODIFY_SPA_PAGE_HOST_ACCESS_HUGE_PAGE 0x8
>
> +enum hv_translate_gva_result_code {
> + HV_TRANSLATE_GVA_SUCCESS = 0,
> +
> + /* Translation failures */
> + HV_TRANSLATE_GVA_PAGE_NOT_PRESENT = 1,
> + HV_TRANSLATE_GVA_PRIVILEGE_VIOLATION = 2,
> + HV_TRANSLATE_GVA_INVALID_PAGE_TABLE_FLAGS = 3,
> +
> + /* GPA access failures */
> + HV_TRANSLATE_GVA_GPA_UNMAPPED = 4,
> + HV_TRANSLATE_GVA_GPA_NO_READ_ACCESS = 5,
> + HV_TRANSLATE_GVA_GPA_NO_WRITE_ACCESS = 6,
> + HV_TRANSLATE_GVA_GPA_ILLEGAL_OVERLAY_ACCESS = 7,
> +
> + HV_TRANSLATE_GVA_INTERCEPT = 8,
> + HV_TRANSLATE_GVA_GPA_UNACCEPTED = 9,
> +};
> +
> +struct hv_input_translate_virtual_address {
> + u64 partition_id;
> + u32 vp_index;
> + u32 padding;
> + u64 control_flags;
> + u64 gva_page;
> +} __packed;
> +
> +struct hv_translate_gva_result_ex {
> + u32 result_code; /* enum hv_translate_gva_result_code */
> + u32 cache_type : 8;
> + u32 overlay_page : 1;
> + u32 reserved : 23;
> +#if IS_ENABLED(CONFIG_X86)
> + char event_info[40]; /* HV_X64_PENDING_EVENT */
> +#endif
> +} __packed;
> +
> +struct hv_output_translate_virtual_address_ex {
> + struct hv_translate_gva_result_ex translation_result;
> + u64 gpa_page;
> +} __packed;
> +
> #endif /* _HV_HVHDK_H */
> diff --git a/include/uapi/linux/mshv.h b/include/uapi/linux/mshv.h
> index 32ff92b6342b2..5aae848ded4ea 100644
> --- a/include/uapi/linux/mshv.h
> +++ b/include/uapi/linux/mshv.h
> @@ -318,6 +318,16 @@ struct mshv_get_set_vp_state {
> #define MSHV_RUN_VP _IOR(MSHV_IOCTL, 0x00, struct mshv_run_vp)
> #define MSHV_GET_VP_STATE _IOWR(MSHV_IOCTL, 0x01, struct mshv_get_set_vp_state)
> #define MSHV_SET_VP_STATE _IOWR(MSHV_IOCTL, 0x02, struct mshv_get_set_vp_state)
> +
> +struct mshv_translate_gva {
> + __u64 gva;
> + __u64 flags;
> + __u64 *result; /* enum hv_translate_gva_result_code */
> + __u64 *gpa;
> +};
> +
> +#define MSHV_TRANSLATE_GVA _IOWR(MSHV_IOCTL, 0xF2, struct mshv_translate_gva)
> +
> /*
> * Generic hypercall
> * Defined above in partition IOCTLs, avoid redefining it here
>
>
^ permalink raw reply
* RE: [PATCH v2 1/2] Drivers: hv: vmbus: Replace lockdep_hardirq_threaded() with lockdep annotation
From: Michael Kelley @ 2026-07-22 20:18 UTC (permalink / raw)
To: Sebastian Andrzej Siewior, linux-hyperv@vger.kernel.org,
linux-rt-devel@lists.linux.dev, linux-kernel@vger.kernel.org
Cc: K. Y. Srinivasan, Dexuan Cui, Haiyang Zhang, Jan Kiszka, Long Li,
Michael Kelley, Wei Liu
In-Reply-To: <20260721153216.382536-2-bigeasy@linutronix.de>
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Sent: Tuesday, July 21, 2026 8:32 AM
>
> lockdep_hardirq_threaded() is supposed to be used within IRQ core code
> and not within drivers. It is not obvious from within the driver, that
> this is the only interrupt service routing and that it is not shared
s/routing/routine/
> handler.
>
> Replace lockdep_hardirq_threaded() with a lockdep annotation limiting
> threaded context on PREEMPT_RT to __vmbus_isr().
>
> Fixes: f8e6343b7a89c ("Drivers: hv: vmbus: Use kthread for vmbus interrupts on PREEMPT_RT")
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> drivers/hv/vmbus_drv.c | 13 ++++++++++++-
> 1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index 23206640c6139..44877664d9d08 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -1384,8 +1384,19 @@ void vmbus_isr(void)
> if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
> vmbus_irqd_wake();
> } else {
> - lockdep_hardirq_threaded();
> + static DEFINE_WAIT_OVERRIDE_MAP(vmbus_map, LD_WAIT_CONFIG);
> +
> + /*
> + * vmbus_isr is never force-threaded and always invoked at hard
> + * IRQ level. __vmbus_isr() below can acquire a spinlock_t
> + * which becomes a sleeping lock and must not be acquired in
> + * this context. Therefore on PREEMPT_RT this will be threaded
> + * via vmbus_irqd_wake(). On non-PREEMPT the annotation lets
> + * lockdep know that acquiring a spinlock_t is not an issue.
> + */
> + lock_map_acquire_try(&vmbus_map);
> __vmbus_isr();
> + lock_map_release(&vmbus_map);
> }
> }
> EXPORT_SYMBOL_FOR_MODULES(vmbus_isr, "mshv_vtl");
> --
> 2.53.0
>
In my view, it's OK to ignore the Sashiko-reported issue about
hv_stimer0_isr() running in a thread context when
CONFIG_PREEMPT_RT is set. That scenario can only happen if the
guest is running on Windows Server/Hyper-V 2012R2 or earlier,
and that version went out of official Microsoft support in October
2023. I have an inquiry into the Microsoft team about removing
support for that Hyper-V version from the Linux kernel code, which
would resolve the issue.
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
^ permalink raw reply
* Re: [RFC PATCH 1/2] Drivers: hv: vmbus: add vmbus_establish_gpadl_caller_decrypted()
From: sashiko-bot @ 2026-07-22 19:57 UTC (permalink / raw)
To: Kameron Carr; +Cc: linux-hyperv
In-Reply-To: <20260721195633.1438361-2-kameroncarr@linux.microsoft.com>
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but vmbus_teardown_gpadl() silently overwrites the error code from vmbus_post_msg(), causing it to falsely report success when GPADL teardown fails, leading to use-after-free memory corruption.
--
--- Patch [1]: [RFC PATCH 1/2] Drivers: hv: vmbus: add vmbus_establish_gpadl_caller_decrypted() ---
commit 93ae4b1e666b7bf013113501ce3aaa80b55f1c28
Author: Kameron Carr <kameroncarr@linux.microsoft.com>
Drivers: hv: vmbus: add vmbus_establish_gpadl_caller_decrypted()
Refactor vmbus_establish_gpadl() to separate the encryption lifecycle
from the rest of the GPADL establishment logic. Add a new
vmbus_establish_gpadl_caller_decrypted() for callers that want
to decrypt their own buffers.
> diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
> index 6821f225248b1..0166367a4df37 100644
> --- a/drivers/hv/channel.c
> +++ b/drivers/hv/channel.c
[ ... ]
> +int vmbus_establish_gpadl_caller_decrypted(struct vmbus_channel *channel,
> + void *kbuffer, u32 size,
> + struct vmbus_gpadl *gpadl)
> +{
> + int ret = __vmbus_establish_gpadl(channel, HV_GPADL_BUFFER, kbuffer,
> + size, 0U, gpadl);
> +
> + /* Caller owns @kbuffer's encryption; teardown must not touch it. */
> + gpadl->decrypted = false;
[Severity: High]
This isn't a bug introduced by this patch, but does setting gpadl->decrypted
to false here interact with a pre-existing issue in vmbus_teardown_gpadl()
that masks teardown failures?
In vmbus_teardown_gpadl(), if vmbus_post_msg() fails (e.g., due to host
resource starvation), the error code in the ret variable is overwritten
unconditionally before the function returns:
drivers/hv/channel.c:vmbus_teardown_gpadl() {
...
ret = vmbus_post_msg(msg,
sizeof(struct vmbus_channel_gpadl_teardown), true);
trace_vmbus_teardown_gpadl(msg, ret);
if (ret)
goto post_msg_err;
...
post_msg_err:
...
if (gpadl->decrypted)
ret = set_memory_encrypted((unsigned long)gpadl->buffer,
PFN_UP(gpadl->size));
else
ret = 0;
...
return ret;
}
Since gpadl->decrypted is set to false for these caller-managed GPADLs,
if vmbus_post_msg() fails during teardown, the else branch will set ret = 0.
Could this cause vmbus_teardown_gpadl() to falsely report success when
the teardown actually failed? If the caller then frees the buffer while
the hypervisor still holds a mapping to it, could it lead to memory
corruption across the VM boundary?
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(vmbus_establish_gpadl_caller_decrypted);
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260721195633.1438361-1-kameroncarr@linux.microsoft.com?part=1
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox