* [PATCH 1/3] migration/postcopy: Extract page fault handler callback
2026-03-30 19:04 [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Takeru Hayasaka
@ 2026-03-30 19:04 ` Takeru Hayasaka
2026-03-30 19:04 ` [PATCH 2/3] migration/postcopy: Factor out uffd_setup_incoming() from postcopy setup Takeru Hayasaka
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Takeru Hayasaka @ 2026-03-30 19:04 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu, Fabiano Rosas, Takeru Hayasaka
Introduce a PostcopyPageHandler callback type and use it to decouple
the userfaultfd fault thread from the postcopy-specific page request
logic. This prepares for fast snapshot load, which will supply pages
from a local file instead of requesting them over the network.
The fault thread now calls mis->page_fault_handler() instead of
postcopy_request_page() directly. For postcopy, the handler is set to
postcopy_page_fault_handler() which delegates to the existing
postcopy_request_page().
No functional change for postcopy migration.
Signed-off-by: Takeru Hayasaka <hayatake396@gmail.com>
---
migration/migration.h | 16 ++++++++++++++++
migration/postcopy-ram.c | 22 +++++++++++++++++++---
2 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/migration/migration.h b/migration/migration.h
index b6888daceddf..33525402922d 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -89,6 +89,17 @@ typedef enum {
PREEMPT_THREAD_QUIT,
} PreemptThreadStatus;
+/*
+ * Callback to handle a page fault from the userfaultfd fault thread.
+ * Implementations resolve the fault by supplying the requested page,
+ * e.g., by requesting it from the migration source (postcopy) or by
+ * reading it from a snapshot file (fast snapshot load).
+ */
+typedef int (*PostcopyPageHandler)(MigrationIncomingState *mis,
+ RAMBlock *rb,
+ ram_addr_t offset,
+ void *fault_address);
+
/* State for the incoming migration */
struct MigrationIncomingState {
QEMUFile *from_src_file;
@@ -116,6 +127,11 @@ struct MigrationIncomingState {
QemuThread fault_thread;
/* Set this when we want the fault thread to quit */
bool fault_thread_quit;
+ /* Callback to resolve page faults; set before fault thread starts */
+ PostcopyPageHandler page_fault_handler;
+ void *page_fault_opaque;
+ /* ptid from current uffd fault msg, for postcopy blocktime tracking */
+ uint32_t fault_thread_ptid;
bool have_listen_thread;
QemuThread listen_thread;
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index f5ef93f1933c..8dcd8ff35e85 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -976,6 +976,20 @@ static int postcopy_request_page(MigrationIncomingState *mis, RAMBlock *rb,
return migrate_send_rp_req_pages(mis, rb, start, haddr, tid);
}
+/*
+ * Default page fault handler for postcopy live migration.
+ * Requests the faulted page from the source via the return path.
+ */
+static int postcopy_page_fault_handler(MigrationIncomingState *mis,
+ RAMBlock *rb,
+ ram_addr_t offset,
+ void *fault_address)
+{
+ return postcopy_request_page(mis, rb, offset,
+ (uint64_t)(uintptr_t)fault_address,
+ mis->fault_thread_ptid);
+}
+
/*
* Callback from shared fault handlers to ask for a page,
* the page must be specified by a RAMBlock and an offset in that rb
@@ -1392,9 +1406,9 @@ retry:
* Send the request to the source - we want to request one
* of our host page sizes (which is >= TPS)
*/
- ret = postcopy_request_page(mis, rb, rb_offset,
- msg.arg.pagefault.address,
- msg.arg.pagefault.feat.ptid);
+ mis->fault_thread_ptid = msg.arg.pagefault.feat.ptid;
+ ret = mis->page_fault_handler(mis, rb, rb_offset,
+ (void *)(uintptr_t)msg.arg.pagefault.address);
if (ret) {
/* May be network failure, try to wait for recovery */
postcopy_pause_fault_thread(mis);
@@ -1552,6 +1566,8 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
return -1;
}
+ mis->page_fault_handler = postcopy_page_fault_handler;
+
postcopy_thread_create(mis, &mis->fault_thread,
MIGRATION_THREAD_DST_FAULT,
postcopy_ram_fault_thread, QEMU_THREAD_JOINABLE);
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/3] migration/postcopy: Factor out uffd_setup_incoming() from postcopy setup
2026-03-30 19:04 [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Takeru Hayasaka
2026-03-30 19:04 ` [PATCH 1/3] migration/postcopy: Extract page fault handler callback Takeru Hayasaka
@ 2026-03-30 19:04 ` Takeru Hayasaka
2026-03-30 19:04 ` [PATCH 3/3] migration/ram: Split mapped-ram header and page loading Takeru Hayasaka
2026-03-30 20:16 ` [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Peter Xu
3 siblings, 0 replies; 6+ messages in thread
From: Takeru Hayasaka @ 2026-03-30 19:04 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu, Fabiano Rosas, Takeru Hayasaka
Extract the generic userfaultfd initialization from
postcopy_ram_incoming_setup() into a new uffd_setup_incoming() function
that can be shared between postcopy live migration and fast snapshot
load.
uffd_setup_incoming() handles:
- Opening the userfaultfd and API handshake
- Creating the eventfd for quit notification
- Setting the page fault handler callback
- Starting the fault handler thread
- Registering all RAM blocks with UFFDIO_REGISTER
- Allocating temporary page buffers for UFFDIO_COPY
postcopy_ram_incoming_setup() becomes a thin wrapper that adds
postcopy-specific logic: blocktime tracking and the preemption thread.
Also move the PostcopyPageHandler typedef from migration.h to
postcopy-ram.h where it logically belongs and is needed for the
uffd_setup_incoming() declaration.
The channels parameter of postcopy_temp_pages_setup() is now passed
by the caller instead of being determined internally, so
uffd_setup_incoming() callers can control how many temporary page
buffers are allocated.
No functional change for postcopy migration.
Signed-off-by: Takeru Hayasaka <hayatake396@gmail.com>
---
migration/migration.h | 11 --------
migration/postcopy-ram.c | 66 +++++++++++++++++++++++++++++++++---------------
migration/postcopy-ram.h | 27 ++++++++++++++++++++
3 files changed, 73 insertions(+), 31 deletions(-)
diff --git a/migration/migration.h b/migration/migration.h
index 33525402922d..43ea32c07ba7 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -89,17 +89,6 @@ typedef enum {
PREEMPT_THREAD_QUIT,
} PreemptThreadStatus;
-/*
- * Callback to handle a page fault from the userfaultfd fault thread.
- * Implementations resolve the fault by supplying the requested page,
- * e.g., by requesting it from the migration source (postcopy) or by
- * reading it from a snapshot file (fast snapshot load).
- */
-typedef int (*PostcopyPageHandler)(MigrationIncomingState *mis,
- RAMBlock *rb,
- ram_addr_t offset,
- void *fault_address);
-
/* State for the incoming migration */
struct MigrationIncomingState {
QEMUFile *from_src_file;
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 8dcd8ff35e85..7af207d6ead5 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -1478,22 +1478,15 @@ retry:
return NULL;
}
-static int postcopy_temp_pages_setup(MigrationIncomingState *mis)
+static int postcopy_temp_pages_setup(MigrationIncomingState *mis,
+ unsigned int channels)
{
PostcopyTmpPage *tmp_page;
int err;
- unsigned i, channels;
+ unsigned i;
void *temp_page;
- if (migrate_postcopy_preempt()) {
- /* If preemption enabled, need extra channel for urgent requests */
- mis->postcopy_channels = RAM_CHANNEL_MAX;
- } else {
- /* Both precopy/postcopy on the same channel */
- mis->postcopy_channels = 1;
- }
-
- channels = mis->postcopy_channels;
+ mis->postcopy_channels = channels;
mis->postcopy_tmp_pages = g_new0(PostcopyTmpPage, channels);
for (i = 0; i < channels; i++) {
@@ -1531,7 +1524,19 @@ static int postcopy_temp_pages_setup(MigrationIncomingState *mis)
return 0;
}
-int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
+/*
+ * Generic userfaultfd setup for incoming migration.
+ *
+ * Opens the userfaultfd, performs the API handshake, registers all RAM
+ * blocks for missing-page notifications, allocates temporary page buffers,
+ * and starts the fault handler thread using the given callback.
+ *
+ * Both postcopy live migration and fast snapshot load use this function;
+ * the caller supplies the appropriate page_fault_handler.
+ */
+int uffd_setup_incoming(MigrationIncomingState *mis,
+ PostcopyPageHandler handler,
+ unsigned int channels)
{
Error *local_err = NULL;
@@ -1549,15 +1554,11 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
*/
if (!ufd_check_and_apply(mis->userfault_fd, mis, &local_err)) {
error_report_err(local_err);
+ close(mis->userfault_fd);
return -1;
}
- if (migrate_postcopy_blocktime()) {
- assert(mis->blocktime_ctx == NULL);
- mis->blocktime_ctx = blocktime_context_new();
- }
-
- /* Now an eventfd we use to tell the fault-thread to quit */
+ /* An eventfd we use to tell the fault-thread to quit */
mis->userfault_event_fd = eventfd(0, EFD_CLOEXEC);
if (mis->userfault_event_fd == -1) {
error_report("%s: Opening userfault_event_fd: %s", __func__,
@@ -1566,7 +1567,7 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
return -1;
}
- mis->page_fault_handler = postcopy_page_fault_handler;
+ mis->page_fault_handler = handler;
postcopy_thread_create(mis, &mis->fault_thread,
MIGRATION_THREAD_DST_FAULT,
@@ -1579,11 +1580,29 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
return -1;
}
- if (postcopy_temp_pages_setup(mis)) {
+ if (postcopy_temp_pages_setup(mis, channels)) {
/* Error dumped in the sub-function */
return -1;
}
+ return 0;
+}
+
+int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
+{
+ unsigned int channels;
+
+ if (migrate_postcopy_blocktime()) {
+ assert(mis->blocktime_ctx == NULL);
+ mis->blocktime_ctx = blocktime_context_new();
+ }
+
+ channels = migrate_postcopy_preempt() ? RAM_CHANNEL_MAX : 1;
+
+ if (uffd_setup_incoming(mis, postcopy_page_fault_handler, channels)) {
+ return -1;
+ }
+
if (migrate_postcopy_preempt()) {
/*
* This thread needs to be created after the temp pages because
@@ -1745,6 +1764,13 @@ int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb,
g_assert_not_reached();
}
+int uffd_setup_incoming(MigrationIncomingState *mis,
+ PostcopyPageHandler handler,
+ unsigned int channels)
+{
+ g_assert_not_reached();
+}
+
int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
{
g_assert_not_reached();
diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h
index a080dd65a77a..3bbd1ed7e323 100644
--- a/migration/postcopy-ram.h
+++ b/migration/postcopy-ram.h
@@ -15,10 +15,37 @@
#include "qapi/qapi-types-migration.h"
+/*
+ * Callback to handle a page fault from the userfaultfd fault thread.
+ * Implementations resolve the fault by supplying the requested page,
+ * e.g., by requesting it from the migration source (postcopy) or by
+ * reading it from a snapshot file (fast snapshot load).
+ */
+typedef int (*PostcopyPageHandler)(MigrationIncomingState *mis,
+ RAMBlock *rb,
+ ram_addr_t offset,
+ void *fault_address);
+
/* Return true if the host supports everything we need to do postcopy-ram */
bool postcopy_ram_supported_by_host(MigrationIncomingState *mis,
Error **errp);
+/*
+ * Generic userfaultfd setup for incoming migration.
+ *
+ * Opens userfaultfd, registers all RAM blocks, allocates temporary
+ * page buffers and starts the fault handler thread. The caller
+ * supplies the page_fault_handler callback appropriate for the use
+ * case (postcopy live migration or fast snapshot load).
+ *
+ * @channels: number of temporary page channels to allocate
+ * (1 for most cases, RAM_CHANNEL_MAX when postcopy preempt
+ * is enabled).
+ */
+int uffd_setup_incoming(MigrationIncomingState *mis,
+ PostcopyPageHandler handler,
+ unsigned int channels);
+
/*
* Make all of RAM sensitive to accesses to areas that haven't yet been written
* and wire up anything necessary to deal with it.
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/3] migration/ram: Split mapped-ram header and page loading
2026-03-30 19:04 [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Takeru Hayasaka
2026-03-30 19:04 ` [PATCH 1/3] migration/postcopy: Extract page fault handler callback Takeru Hayasaka
2026-03-30 19:04 ` [PATCH 2/3] migration/postcopy: Factor out uffd_setup_incoming() from postcopy setup Takeru Hayasaka
@ 2026-03-30 19:04 ` Takeru Hayasaka
2026-03-30 20:16 ` [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Peter Xu
3 siblings, 0 replies; 6+ messages in thread
From: Takeru Hayasaka @ 2026-03-30 19:04 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu, Fabiano Rosas, Takeru Hayasaka
Split parse_ramblock_mapped_ram() into two stages:
- parse_ramblock_mapped_ram_header(): reads the MappedRamHeader and
dirty bitmap, storing pages_offset and file_bmap on the RAMBlock
for later use.
- parse_ramblock_mapped_ram_pages(): reads all page data using the
bitmap and pages_offset populated by the header stage.
The original parse_ramblock_mapped_ram() becomes a thin wrapper that
calls both in sequence, preserving the existing behavior.
The bitmap is now stored in block->file_bmap (which already exists
on RAMBlock for the save side) instead of a function-local variable.
The pages function frees the bitmap after loading is complete.
This separation prepares for fast snapshot load, which will call
only the header stage at load time and defer page loading to a
userfaultfd-based demand paging mechanism.
No functional change.
Signed-off-by: Takeru Hayasaka <hayatake396@gmail.com>
---
migration/ram.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 46 insertions(+), 6 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 979751f61b30..301ef9758e25 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4152,10 +4152,15 @@ err:
return false;
}
-static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
- ram_addr_t length, Error **errp)
+/*
+ * Read the mapped-ram header and dirty bitmap for a RAM block.
+ *
+ * Populates block->pages_offset and block->file_bmap so that page
+ * data can be loaded later (either eagerly or on demand).
+ */
+static void parse_ramblock_mapped_ram_header(QEMUFile *f, RAMBlock *block,
+ ram_addr_t length, Error **errp)
{
- g_autofree unsigned long *bitmap = NULL;
MappedRamHeader header;
size_t bitmap_size;
long num_pages;
@@ -4187,19 +4192,54 @@ static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
num_pages = length / header.page_size;
bitmap_size = BITS_TO_LONGS(num_pages) * sizeof(unsigned long);
- bitmap = g_malloc0(bitmap_size);
- if (qemu_get_buffer_at(f, (uint8_t *)bitmap, bitmap_size,
+ block->file_bmap = g_malloc0(bitmap_size);
+ if (qemu_get_buffer_at(f, (uint8_t *)block->file_bmap, bitmap_size,
header.bitmap_offset) != bitmap_size) {
error_setg(errp, "Error reading dirty bitmap");
+ g_free(block->file_bmap);
+ block->file_bmap = NULL;
return;
}
+}
+
+/*
+ * Read all page data for a mapped-ram RAM block using the bitmap
+ * and pages_offset previously populated by parse_ramblock_mapped_ram_header().
+ */
+static void parse_ramblock_mapped_ram_pages(QEMUFile *f, RAMBlock *block,
+ ram_addr_t length, Error **errp)
+{
+ long num_pages;
- if (!read_ramblock_mapped_ram(f, block, num_pages, bitmap, errp)) {
+ if (!block->file_bmap) {
+ /* Shared block that was skipped during header parsing */
return;
}
+ num_pages = length / TARGET_PAGE_SIZE;
+
+ if (!read_ramblock_mapped_ram(f, block, num_pages,
+ block->file_bmap, errp)) {
+ goto out;
+ }
+
/* Skip pages array */
qemu_set_offset(f, block->pages_offset + length, SEEK_SET);
+
+out:
+ g_free(block->file_bmap);
+ block->file_bmap = NULL;
+}
+
+static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
+ ram_addr_t length, Error **errp)
+{
+ parse_ramblock_mapped_ram_header(f, block, length, errp);
+ if (*errp) {
+ return;
+ }
+
+ parse_ramblock_mapped_ram_pages(f, block, length, errp);
}
static int parse_ramblock(QEMUFile *f, RAMBlock *block, ram_addr_t length)
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load
2026-03-30 19:04 [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Takeru Hayasaka
` (2 preceding siblings ...)
2026-03-30 19:04 ` [PATCH 3/3] migration/ram: Split mapped-ram header and page loading Takeru Hayasaka
@ 2026-03-30 20:16 ` Peter Xu
2026-03-31 2:12 ` Takeru Hayasaka
3 siblings, 1 reply; 6+ messages in thread
From: Peter Xu @ 2026-03-30 20:16 UTC (permalink / raw)
To: Takeru Hayasaka; +Cc: qemu-devel, Fabiano Rosas
Hi, Takeru,
On Mon, Mar 30, 2026 at 07:04:40PM +0000, Takeru Hayasaka wrote:
> This series refactors postcopy and mapped-ram code to make the
> userfaultfd infrastructure reusable beyond postcopy live migration.
> The goal is to prepare for fast snapshot load, which will use
> userfaultfd to load guest RAM pages on demand from a snapshot file
> instead of reading all pages before starting the VM.
>
> All three patches are pure refactoring with no functional change.
>
> Patch 1 introduces a callback for the fault thread's page request
> handler, so the page supply mechanism can be swapped (network for
> postcopy, local file for snapshot load).
>
> Patch 2 extracts generic uffd setup (open, register, fault thread,
> temp pages) from postcopy_ram_incoming_setup() into a shared
> uffd_setup_incoming() function.
>
> Patch 3 splits parse_ramblock_mapped_ram() so that the header and
> bitmap can be read without immediately loading all page data. This
> allows a future caller to read only the metadata and defer page
> loading to a fault handler.
Thanks for your interest and enthusiasm to the QEMU community. Said that,
please hold off a bit on sending patches that enable fast snapshot load for
now, even for preparations.
The problem is, this topic, as announced, is to be developed by a GSoC
student only later. We want to make sure this topic will be done during the
GSoC period by the selected candidate, and we will also need to evalute the
progress no matter who is doing it.
I'm not sure if it's fair if any of the applicants start sending patches.
I also don't hope to see multiple versions of impl floating around. IMHO
it's slightly against how collaboration works upstream when we know there
can be work collisions and we should avoid it from happening.
Feel free to send whatever patches that do not have direct involvement with
this specific topic.
>
> Tested with mapped-ram precopy file migration tests. Postcopy tests
> require userfaultfd, which was not available in my test environment,
> but no postcopy logic was changed.
When postcopy code is changed, I think it needs to be tested.
Could you explain what do you mean by "userfaultfd, which was not available
in my test environment"? Are you using Linux?
Thanks,
>
> This work is part of my preparation for the GSoC 2026 "Fast Snapshot
> Load" project.
>
> Signed-off-by: Takeru Hayasaka <hayatake396@gmail.com>
> ---
> Takeru Hayasaka (3):
> migration/postcopy: Extract page fault handler callback
> migration/postcopy: Factor out uffd_setup_incoming() from postcopy setup
> migration/ram: Split mapped-ram header and page loading
>
> migration/migration.h | 5 +++
> migration/postcopy-ram.c | 86 +++++++++++++++++++++++++++++++++++-------------
> migration/postcopy-ram.h | 27 +++++++++++++++
> migration/ram.c | 52 +++++++++++++++++++++++++----
> 4 files changed, 142 insertions(+), 28 deletions(-)
> ---
> base-commit: 8e711856d7639cbffa51405f2cc2366e3d9e3a23
> change-id: 20260327-fast-snapshot-refactor-3c6bb472b777
>
> Best regards,
> --
> Takeru Hayasaka <hayatake396@gmail.com>
>
--
Peter Xu
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load
2026-03-30 20:16 ` [PATCH 0/3] migration/postcopy: Preparatory refactoring for fast snapshot load Peter Xu
@ 2026-03-31 2:12 ` Takeru Hayasaka
0 siblings, 0 replies; 6+ messages in thread
From: Takeru Hayasaka @ 2026-03-31 2:12 UTC (permalink / raw)
To: Peter Xu; +Cc: qemu-devel, Fabiano Rosas
Hi Peter,
> Thanks for your interest and enthusiasm to the QEMU community. Said that,
> please hold off a bit on sending patches that enable fast snapshot load for
> now, even for preparations.
>
> The problem is, this topic, as announced, is to be developed by a GSoC
> student only later. We want to make sure this topic will be done during the
> GSoC period by the selected candidate, and we will also need to evalute the
> progress no matter who is doing it.
>
> I'm not sure if it's fair if any of the applicants start sending patches.
> I also don't hope to see multiple versions of impl floating around. IMHO
> it's slightly against how collaboration works upstream when we know there
> can be work collisions and we should avoid it from happening.
I understand. I was eager to demonstrate my enthusiasm for the
project, which led me to send these patches prematurely. I'll hold
off on any patches related to fast snapshot load, including
preparatory refactoring, until the GSoC period begins. I'll focus
on unrelated contributions in the meantime.
> Feel free to send whatever patches that do not have direct involvement with
> this specific topic.
Thank you, will do.
> When postcopy code is changed, I think it needs to be tested.
>
> Could you explain what do you mean by "userfaultfd, which was not available
> in my test environment"? Are you using Linux?
I apologize for the lack of rigor. I am on Linux. The postcopy tests
were skipped because I ran them as an unprivileged user
(vm.unprivileged_userfaultfd defaults to 0). I should have rerun
them with sudo. I've since confirmed the postcopy tests pass that way.
Thanks,
Takeru
^ permalink raw reply [flat|nested] 6+ messages in thread