* [PATCH v17 00/10] virtio-net: live-TAP local migration
@ 2026-05-27 19:11 Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 01/10] net/tap: move vhost-net open() calls to tap_parse_vhost_fds() Vladimir Sementsov-Ogievskiy
` (10 more replies)
0 siblings, 11 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
Hi all!
Here is a migration for TAP net backend, including its properties and
open fds.
With this new feature, management software doesn't need to initialize
new TAP and do a switch to it. Nothing should be done around
virtio-net in local migration: it just migrates and continues to use
same TAP device. So we avoid extra logic in management software, extra
allocations in kernel (for new TAP), and corresponding extra delay in
migration downtime.
v17: rework interface
02-04: new
07: - drop local-migration parameter for virtio-net and the whole
code to migrate backends (they should migrate by themselves).
- drop virtio_net_update_host_features(), and call
peer_test_vnet_hdr() and virtio_net_get_features() directly
from post-load handlers
- drop r-b
08: - add local-migratieon-supported property (instead of
local-migration that is dropped in 07)
- register own vmstate
- add check for being in incoming migration when use incoming-fds
- drop r-b, a-b
10: only move 'local_migration=local' from virtio-net device
to 'local-migration-supported": local' in netdev. keep r-bs
Based-on: <20260318113144.15697-1-vsementsov@yandex-team.ru>
"[PATCH v4 00/13] net: refactoring and fixes"
v17 is pushed to
https://gitlab.com/vsementsov/qemu.git
tag: up-tap-fd-migration-with-bk-opt-v17
To run the test, use sudo, as test needs to configure TAP device:
sudo PYTHONPATH=python:tests/functional \
QEMU_TEST_QEMU_BINARY=$PWD/build/qemu-system-x86_64 \
MESON_BUILD_ROOT=$PWD/build \
./build/pyvenv/bin/python3 tests/functional/x86_64/test_tap_migration.py
Or, to test the feature by hand, you may follow the instruction:
1. Prerequisites
----------------
QEMU=/path/to/your/build/qemu-system-x86_64
# download same image as in test
wget -O /tmp/alpine.iso "https://dl-cdn.alpinelinux.org/alpine/v3.22/releases/x86_64/alpine-standard-3.22.1-x86_64.iso"
# prepare tap device (be careful to not break your own networks)
sudo ip tuntap add dev tap0 mode tap multi_queue
sudo ip addr add 192.168.100.1/24 dev tap0
sudo ip link set tap0 up
2. Start source VM
------------------
Open terminal A, and run:
$QEMU \
-name source \
-machine q35 \
-accel kvm \
-m 1G \
-object memory-backend-file,id=ram0,size=1G,mem-path=/dev/shm/qemu_migration_test,share=on \
-machine memory-backend=ram0 \
-drive file=/tmp/alpine.iso,media=cdrom,format=raw \
-device pcie-pci-bridge,id=pci.1,bus=pcie.0 \
-netdev tap,id=netdev.1,ifname=tap0,queues=4,vnet_hdr=on,script=no,downscript=no,local-migration-supported=on \
-device virtio-net-pci,netdev=netdev.1,id=vnet.1,bus=pci.1,mq=on,vectors=18,romfile=,disable-legacy=off \
-serial stdio \
-nographic \
-qmp unix:/tmp/qmp-source.sock,server=on,wait=off
Wait for Alpine to boot. When you see the login prompt, log in as root
(no password):
localhost login: root
Configure the guest network:
ip addr add 192.168.100.2/24 dev eth0
ip link set eth0 up
Verify connectivity from the guest:
ping -c 3 192.168.100.1
And from the host (in another terminal):
ping -c 3 192.168.100.2
3. Start target VM
------------------
Open terminal B. The target VM uses the same shared memory file and
the same ISO. The NIC is configured with incoming-fds=on — it will
not open tap0 itself; instead it will receive the TAP file descriptors
from the source over the migration channel.
$QEMU \
-name target \
-machine q35 \
-accel kvm \
-m 1G \
-object memory-backend-file,id=ram0,size=1G,mem-path=/dev/shm/qemu_migration_test,share=on \
-machine memory-backend=ram0 \
-drive file=/tmp/alpine.iso,media=cdrom,format=raw \
-device pcie-pci-bridge,id=pci.1,bus=pcie.0 \
-netdev tap,id=netdev.1,queues=4,script=no,downscript=no,local-migration-supported=on,incoming-fds=on \
-device virtio-net-pci,netdev=netdev.1,id=vnet.1,bus=pci.1,mq=on,vectors=18,romfile=,disable-legacy=off \
-serial stdio \
-nographic \
-qmp unix:/tmp/qmp-target.sock,server=on,wait=off \
-incoming defer
4. Start migration
------------------
Open terminal C and connect to the source QMP socket:
socat - UNIX-CONNECT:/tmp/qmp-source.sock
Negotiate capabilities and configure migration:
{"execute": "qmp_capabilities"}
{"execute": "migrate-set-capabilities", "arguments": {
"capabilities": [
{"capability": "events", "state": true},
{"capability": "x-ignore-shared", "state": true}
]
}}
{"execute": "migrate-set-parameters", "arguments": {"local": true}}
Open terminal D and connect to the target QMP socket:
socat - UNIX-CONNECT:/tmp/qmp-target.sock
Negotiate capabilities and configure migration:
{"execute": "qmp_capabilities"}
{"execute": "migrate-set-capabilities", "arguments": {
"capabilities": [
{"capability": "events", "state": true},
{"capability": "x-ignore-shared", "state": true}
]
}}
{"execute": "migrate-set-parameters", "arguments": {"local": true}}
Tell the target to listen for the incoming migration (terminal D):
{"execute": "migrate-incoming",
"arguments": {"uri": "unix:/tmp/migration.sock"}}
Trigger the migration from the source (terminal C):
{"execute": "migrate",
"arguments": {"uri": "unix:/tmp/migration.sock"}}
Poll migration status until it completes (terminal C):
{"execute": "query-migrate"}
# repeat until "status" == "completed"
Or just wait for the MIGRATION event that QEMU emits automatically:
# {"event": "MIGRATION", "data": {"status": "completed"}, ...}
Once the source reports "completed", resume the target VM (terminal D):
{"execute": "cont"}
The target VM is now running with the migrated state and the TAP file
descriptors that were passed from the source.
Verify that the guest is still reachable from the host:
ping -c 3 192.168.100.2
And from inside the guest (terminal B, target serial console):
ping -c 3 192.168.100.1
5. Cleanup
----------
Shut down the target VM (terminal D):
{"execute": "quit"}
Shut down the source VM (terminal C):
{"execute": "quit"}
Remove the TAP device:
sudo ip tuntap del tap0 mode tap multi_queue
Remove the shared memory file:
rm /dev/shm/qemu_migration_test
Remove leftover sockets if they still exist:
rm -f /tmp/migration.sock /tmp/qmp-source.sock /tmp/qmp-target.sock
Vladimir Sementsov-Ogievskiy (10):
net/tap: move vhost-net open() calls to tap_parse_vhost_fds()
net/tap: move vhost initialization to tap_setup_vhost()
net/tap: use container_of instead of DO_UPCAST
net/tap: QOMify tap backend
net/tap: add TYPE_VMSTATE_IF interface
qapi: add local migration parameter
virtio-net: support local migration of backend
net/tap: support local migration with virtio-net
tests/functional: add skipWithoutSudo() decorator
tests/functional: add test_tap_migration
hw/core/machine.c | 2 +
hw/net/virtio-net.c | 89 +++-
include/hw/virtio/virtio-net.h | 1 +
include/migration/misc.h | 2 +
include/migration/vmstate.h | 2 +
include/net/net.h | 9 +
include/net/tap.h | 2 +
migration/options.c | 18 +-
net/net.c | 14 +-
net/tap.c | 429 +++++++++++++---
qapi/migration.json | 12 +-
qapi/net.json | 20 +-
tests/functional/qemu_test/decorators.py | 16 +
tests/functional/x86_64/meson.build | 1 +
tests/functional/x86_64/test_tap_migration.py | 458 ++++++++++++++++++
15 files changed, 986 insertions(+), 89 deletions(-)
create mode 100755 tests/functional/x86_64/test_tap_migration.py
--
2.52.0
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v17 01/10] net/tap: move vhost-net open() calls to tap_parse_vhost_fds()
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 02/10] net/tap: move vhost initialization to tap_setup_vhost() Vladimir Sementsov-Ogievskiy
` (9 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
1. Simplify code path: get vhostfds for all cases in one function.
2. Prepare for further tap-fd-migraton feature, when we'll need to
postpone vhost initialization up to post-load stage.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Ben Chaney <bchaney@akamai.com>
---
net/tap.c | 39 ++++++++++++++++++++++-----------------
1 file changed, 22 insertions(+), 17 deletions(-)
diff --git a/net/tap.c b/net/tap.c
index 57ffb09885c..d941c67895e 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -736,8 +736,7 @@ static bool net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
}
}
- if (tap->has_vhost ? tap->vhost :
- (vhostfd != -1) || (tap->has_vhostforce && tap->vhostforce)) {
+ if (vhostfd != -1) {
VhostNetOptions options;
options.backend_type = VHOST_BACKEND_TYPE_KERNEL;
@@ -747,17 +746,6 @@ static bool net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
} else {
options.busyloop_timeout = 0;
}
-
- if (vhostfd == -1) {
- vhostfd = open("/dev/vhost-net", O_RDWR);
- if (vhostfd < 0) {
- error_setg_file_open(errp, errno, "/dev/vhost-net");
- goto failed;
- }
- if (!qemu_set_blocking(vhostfd, false, errp)) {
- goto failed;
- }
- }
options.opaque = (void *)(uintptr_t)vhostfd;
options.nvqs = 2;
options.feature_bits = kernel_feature_bits;
@@ -843,14 +831,31 @@ static int tap_parse_fds_and_queues(const NetdevTapOptions *tap, int **fds,
static bool tap_parse_vhost_fds(const NetdevTapOptions *tap, int **vhost_fds,
int queues, Error **errp)
{
- if (!(tap->vhostfd || tap->vhostfds)) {
+ bool need_vhost = tap->has_vhost ? tap->vhost :
+ ((tap->vhostfd || tap->vhostfds) ||
+ (tap->has_vhostforce && tap->vhostforce));
+
+ if (!need_vhost) {
*vhost_fds = NULL;
return true;
}
- if (net_parse_fds(tap->vhostfd ?: tap->vhostfds,
- vhost_fds, queues, errp) < 0) {
- return false;
+ if (tap->vhostfd || tap->vhostfds) {
+ if (net_parse_fds(tap->vhostfd ?: tap->vhostfds,
+ vhost_fds, queues, errp) < 0) {
+ return false;
+ }
+ } else {
+ *vhost_fds = g_new(int, queues);
+ for (int i = 0; i < queues; i++) {
+ int vhostfd = open("/dev/vhost-net", O_RDWR);
+ if (vhostfd < 0) {
+ error_setg_file_open(errp, errno, "/dev/vhost-net");
+ net_free_fds(*vhost_fds, i);
+ return false;
+ }
+ (*vhost_fds)[i] = vhostfd;
+ }
}
if (!unblock_fds(*vhost_fds, queues, errp)) {
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 02/10] net/tap: move vhost initialization to tap_setup_vhost()
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 01/10] net/tap: move vhost-net open() calls to tap_parse_vhost_fds() Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 03/10] net/tap: use container_of instead of DO_UPCAST Vladimir Sementsov-Ogievskiy
` (8 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
Make a new helper function in a way it can be reused later for
TAP fd-migration feature: we'll need to initialize vhost in a later
point when we doesn't have access to QAPI parameters.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Ben Chaney <bchaney@akamai.com>
---
net/tap.c | 62 ++++++++++++++++++++++++++++++++++---------------------
1 file changed, 38 insertions(+), 24 deletions(-)
diff --git a/net/tap.c b/net/tap.c
index d941c67895e..9d6213fc3e5 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -71,6 +71,8 @@ static const int kernel_feature_bits[] = {
typedef struct TAPState {
NetClientState nc;
int fd;
+ int vhostfd;
+ uint32_t vhost_busyloop_timeout;
char down_script[1024];
char down_script_arg[128];
uint8_t buf[NET_BUFSIZE];
@@ -702,6 +704,38 @@ static int net_tap_init(const NetdevTapOptions *tap, int *vnet_hdr,
return fd;
}
+static bool tap_setup_vhost(TAPState *s, Error **errp)
+{
+ VhostNetOptions options;
+
+ if (s->vhostfd == -1) {
+ return true;
+ }
+
+ options.backend_type = VHOST_BACKEND_TYPE_KERNEL;
+ options.net_backend = &s->nc;
+ options.busyloop_timeout = s->vhost_busyloop_timeout;
+ options.opaque = (void *)(uintptr_t)s->vhostfd;
+ options.nvqs = 2;
+ options.feature_bits = kernel_feature_bits;
+ options.get_acked_features = NULL;
+ options.save_acked_features = NULL;
+ options.max_tx_queue_size = 0;
+ options.is_vhost_user = false;
+
+ s->vhost_net = vhost_net_init(&options);
+ if (!s->vhost_net) {
+ error_setg(errp,
+ "vhost-net requested but could not be initialized");
+ return false;
+ }
+
+ /* vhostfd ownership is passed to s->vhost_net */
+ s->vhostfd = -1;
+
+ return true;
+}
+
static bool net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
const char *name,
const char *ifname, const char *script,
@@ -736,30 +770,10 @@ static bool net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
}
}
- if (vhostfd != -1) {
- VhostNetOptions options;
-
- options.backend_type = VHOST_BACKEND_TYPE_KERNEL;
- options.net_backend = &s->nc;
- if (tap->has_poll_us) {
- options.busyloop_timeout = tap->poll_us;
- } else {
- options.busyloop_timeout = 0;
- }
- options.opaque = (void *)(uintptr_t)vhostfd;
- options.nvqs = 2;
- options.feature_bits = kernel_feature_bits;
- options.get_acked_features = NULL;
- options.save_acked_features = NULL;
- options.max_tx_queue_size = 0;
- options.is_vhost_user = false;
-
- s->vhost_net = vhost_net_init(&options);
- if (!s->vhost_net) {
- error_setg(errp,
- "vhost-net requested but could not be initialized");
- goto failed;
- }
+ s->vhostfd = vhostfd;
+ s->vhost_busyloop_timeout = tap->has_poll_us ? tap->poll_us : 0;
+ if (!tap_setup_vhost(s, errp)) {
+ return false;
}
return true;
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 03/10] net/tap: use container_of instead of DO_UPCAST
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 01/10] net/tap: move vhost-net open() calls to tap_parse_vhost_fds() Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 02/10] net/tap: move vhost initialization to tap_setup_vhost() Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 04/10] net/tap: QOMify tap backend Vladimir Sementsov-Ogievskiy
` (7 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
We are going to QOMify tap backend, which includes deriving TAPState
from Object. So "NetClientState nc" will not be a first member.
Let's parepare for this change, and use container_of(), which will
work regardless position of "nc" field.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
net/tap.c | 36 ++++++++++++++++++------------------
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/net/tap.c b/net/tap.c
index 9d6213fc3e5..5f45f2c6a31 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -155,7 +155,7 @@ static ssize_t tap_write_packet(TAPState *s, const struct iovec *iov, int iovcnt
static ssize_t tap_receive_iov(NetClientState *nc, const struct iovec *iov,
int iovcnt)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
const struct iovec *iovp = iov;
g_autofree struct iovec *iov_copy = NULL;
struct virtio_net_hdr hdr = { };
@@ -191,7 +191,7 @@ ssize_t tap_read_packet(int tapfd, uint8_t *buf, int maxlen)
static void tap_send_completed(NetClientState *nc, ssize_t len)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
tap_read_poll(s, true);
}
@@ -251,7 +251,7 @@ static void tap_send(void *opaque)
static bool tap_has_ufo(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
@@ -260,7 +260,7 @@ static bool tap_has_ufo(NetClientState *nc)
static bool tap_has_uso(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
@@ -269,7 +269,7 @@ static bool tap_has_uso(NetClientState *nc)
static bool tap_has_tunnel(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
return s->has_tunnel;
@@ -277,7 +277,7 @@ static bool tap_has_tunnel(NetClientState *nc)
static bool tap_has_vnet_hdr(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
@@ -291,7 +291,7 @@ static bool tap_has_vnet_hdr_len(NetClientState *nc, int len)
static void tap_set_vnet_hdr_len(NetClientState *nc, int len)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
@@ -302,21 +302,21 @@ static void tap_set_vnet_hdr_len(NetClientState *nc, int len)
static int tap_set_vnet_le(NetClientState *nc, bool is_le)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
return tap_fd_set_vnet_le(s->fd, is_le);
}
static int tap_set_vnet_be(NetClientState *nc, bool is_be)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
return tap_fd_set_vnet_be(s->fd, is_be);
}
static void tap_set_offload(NetClientState *nc, const NetOffloads *ol)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
if (s->fd < 0) {
return;
}
@@ -337,7 +337,7 @@ static void tap_exit_notify(Notifier *notifier, void *data)
static void tap_cleanup(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
if (s->vhost_net) {
vhost_net_cleanup(s->vhost_net);
@@ -361,14 +361,14 @@ static void tap_cleanup(NetClientState *nc)
static void tap_poll(NetClientState *nc, bool enable)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
tap_read_poll(s, enable);
tap_write_poll(s, enable);
}
static bool tap_set_steering_ebpf(NetClientState *nc, int prog_fd)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
return tap_fd_set_steering_ebpf(s->fd, prog_fd) == 0;
@@ -376,7 +376,7 @@ static bool tap_set_steering_ebpf(NetClientState *nc, int prog_fd)
int tap_get_fd(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
return s->fd;
}
@@ -388,7 +388,7 @@ int tap_get_fd(NetClientState *nc)
*/
static VHostNetState *tap_get_vhost_net(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
return s->vhost_net;
}
@@ -427,7 +427,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
nc = qemu_new_net_client(&net_tap_info, peer, model, name);
- s = DO_UPCAST(TAPState, nc, nc);
+ s = container_of(nc, TAPState, nc);
s->fd = fd;
s->host_vnet_hdr_len = vnet_hdr ? sizeof(struct virtio_net_hdr) : 0;
@@ -995,7 +995,7 @@ fail:
int tap_enable(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
int ret;
if (s->enabled) {
@@ -1012,7 +1012,7 @@ int tap_enable(NetClientState *nc)
int tap_disable(NetClientState *nc)
{
- TAPState *s = DO_UPCAST(TAPState, nc, nc);
+ TAPState *s = container_of(nc, TAPState, nc);
int ret;
if (s->enabled == 0) {
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 04/10] net/tap: QOMify tap backend
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (2 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 03/10] net/tap: use container_of instead of DO_UPCAST Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 05/10] net/tap: add TYPE_VMSTATE_IF interface Vladimir Sementsov-Ogievskiy
` (6 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
We prepare for being able to migrate TAP backend. We'll need a
user change-able property for it, which can be set from machine
type. So, let's QOMify it first.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
include/net/net.h | 7 +++++++
include/net/tap.h | 2 ++
net/net.c | 14 +++++++-------
net/tap.c | 48 +++++++++++++++++++++++++++++++++++++++--------
4 files changed, 56 insertions(+), 15 deletions(-)
diff --git a/include/net/net.h b/include/net/net.h
index 45bc86fc86b..bdf16bb3e64 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -160,6 +160,13 @@ char *qemu_mac_strdup_printf(const uint8_t *macaddr);
NetClientState *qemu_find_netdev(const char *id);
int qemu_find_net_clients_except(const char *id, NetClientState **ncs,
NetClientDriver type, int max);
+void qemu_net_client_setup(NetClientState *nc,
+ NetClientInfo *info,
+ NetClientState *peer,
+ const char *model,
+ const char *name,
+ NetClientDestructor *destructor,
+ bool is_datapath);
NetClientState *qemu_new_net_client(NetClientInfo *info,
NetClientState *peer,
const char *model,
diff --git a/include/net/tap.h b/include/net/tap.h
index 6f34f13eae4..268570571f4 100644
--- a/include/net/tap.h
+++ b/include/net/tap.h
@@ -28,6 +28,8 @@
#include "standard-headers/linux/virtio_net.h"
+#define TYPE_TAP_NETDEV "tap-netdev"
+
int tap_enable(NetClientState *nc);
int tap_disable(NetClientState *nc);
diff --git a/net/net.c b/net/net.c
index 2892f1730d1..a06558b75bf 100644
--- a/net/net.c
+++ b/net/net.c
@@ -261,13 +261,13 @@ static ssize_t qemu_deliver_packet_iov(NetClientState *sender,
int iovcnt,
void *opaque);
-static void qemu_net_client_setup(NetClientState *nc,
- NetClientInfo *info,
- NetClientState *peer,
- const char *model,
- const char *name,
- NetClientDestructor *destructor,
- bool is_datapath)
+void qemu_net_client_setup(NetClientState *nc,
+ NetClientInfo *info,
+ NetClientState *peer,
+ const char *model,
+ const char *name,
+ NetClientDestructor *destructor,
+ bool is_datapath)
{
nc->info = info;
nc->model = g_strdup(model);
diff --git a/net/tap.c b/net/tap.c
index 5f45f2c6a31..9c04cd4770b 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -43,6 +43,7 @@
#include "qemu/main-loop.h"
#include "qemu/sockets.h"
#include "hw/virtio/vhost.h"
+#include "qom/object.h"
#include "net/tap.h"
#include "net/util.h"
@@ -68,7 +69,11 @@ static const int kernel_feature_bits[] = {
VHOST_INVALID_FEATURE_BIT
};
-typedef struct TAPState {
+OBJECT_DECLARE_SIMPLE_TYPE(TAPState, TAP_NETDEV)
+
+struct TAPState {
+ Object parent_obj;
+
NetClientState nc;
int fd;
int vhostfd;
@@ -86,7 +91,7 @@ typedef struct TAPState {
VHostNetState *vhost_net;
unsigned host_vnet_hdr_len;
Notifier exit;
-} TAPState;
+};
static void launch_script(const char *setup_script, const char *ifname,
int fd, Error **errp);
@@ -393,6 +398,19 @@ static VHostNetState *tap_get_vhost_net(NetClientState *nc)
return s->vhost_net;
}
+
+static const TypeInfo tap_netdev_info = {
+ .name = TYPE_TAP_NETDEV,
+ .parent = TYPE_OBJECT,
+ .instance_size = sizeof(TAPState),
+};
+
+static void tap_net_client_destructor(NetClientState *nc)
+{
+ TAPState *s = container_of(nc, TAPState, nc);
+ object_unref(OBJECT(s));
+}
+
/* fd support */
static NetClientInfo net_tap_info = {
@@ -415,6 +433,18 @@ static NetClientInfo net_tap_info = {
.get_vhost_net = tap_get_vhost_net,
};
+static TAPState *new_tap(NetClientState *peer,
+ const char *model,
+ const char *name)
+{
+ TAPState *s = TAP_NETDEV(object_new(TYPE_TAP_NETDEV));
+
+ qemu_net_client_setup(&s->nc, &net_tap_info, peer, model, name,
+ tap_net_client_destructor, true);
+
+ return s;
+}
+
static TAPState *net_tap_fd_init(NetClientState *peer,
const char *model,
const char *name,
@@ -422,12 +452,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
int vnet_hdr)
{
NetOffloads ol = {};
- NetClientState *nc;
- TAPState *s;
-
- nc = qemu_new_net_client(&net_tap_info, peer, model, name);
-
- s = container_of(nc, TAPState, nc);
+ TAPState *s = new_tap(peer, model, name);
s->fd = fd;
s->host_vnet_hdr_len = vnet_hdr ? sizeof(struct virtio_net_hdr) : 0;
@@ -1027,3 +1052,10 @@ int tap_disable(NetClientState *nc)
return ret;
}
}
+
+static void tap_register_types(void)
+{
+ type_register_static(&tap_netdev_info);
+}
+
+type_init(tap_register_types)
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 05/10] net/tap: add TYPE_VMSTATE_IF interface
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (3 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 04/10] net/tap: QOMify tap backend Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 06/10] qapi: add local migration parameter Vladimir Sementsov-Ogievskiy
` (5 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
We'll need it to implement TAP backend live migration.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
net/tap.c | 43 ++++++++++++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 9 deletions(-)
diff --git a/net/tap.c b/net/tap.c
index 9c04cd4770b..529ed4b0f5b 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -43,7 +43,6 @@
#include "qemu/main-loop.h"
#include "qemu/sockets.h"
#include "hw/virtio/vhost.h"
-#include "qom/object.h"
#include "net/tap.h"
#include "net/util.h"
@@ -91,6 +90,8 @@ struct TAPState {
VHostNetState *vhost_net;
unsigned host_vnet_hdr_len;
Notifier exit;
+
+ int queue_index;
};
static void launch_script(const char *setup_script, const char *ifname,
@@ -399,10 +400,29 @@ static VHostNetState *tap_get_vhost_net(NetClientState *nc)
}
+static char *tap_vmstate_if_get_id(VMStateIf *obj)
+{
+ TAPState *s = TAP_NETDEV(obj);
+ char *res = g_strdup_printf("%s/%d", s->nc.name, s->queue_index);
+ return res;
+}
+
+static void tap_class_init(ObjectClass *klass, const void *data)
+{
+ VMStateIfClass *vc = VMSTATE_IF_CLASS(klass);
+
+ vc->get_id = tap_vmstate_if_get_id;
+}
+
static const TypeInfo tap_netdev_info = {
.name = TYPE_TAP_NETDEV,
.parent = TYPE_OBJECT,
.instance_size = sizeof(TAPState),
+ .class_init = tap_class_init,
+ .interfaces = (const InterfaceInfo[]) {
+ { TYPE_VMSTATE_IF },
+ { }
+ },
};
static void tap_net_client_destructor(NetClientState *nc)
@@ -435,13 +455,16 @@ static NetClientInfo net_tap_info = {
static TAPState *new_tap(NetClientState *peer,
const char *model,
- const char *name)
+ const char *name,
+ int queue_index)
{
TAPState *s = TAP_NETDEV(object_new(TYPE_TAP_NETDEV));
qemu_net_client_setup(&s->nc, &net_tap_info, peer, model, name,
tap_net_client_destructor, true);
+ s->queue_index = queue_index;
+
return s;
}
@@ -449,10 +472,11 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
const char *model,
const char *name,
int fd,
- int vnet_hdr)
+ int vnet_hdr,
+ int queue_index)
{
NetOffloads ol = {};
- TAPState *s = new_tap(peer, model, name);
+ TAPState *s = new_tap(peer, model, name, queue_index);
s->fd = fd;
s->host_vnet_hdr_len = vnet_hdr ? sizeof(struct virtio_net_hdr) : 0;
@@ -689,7 +713,7 @@ int net_init_bridge(const Netdev *netdev, const char *name,
close(fd);
return -1;
}
- s = net_tap_fd_init(peer, "bridge", name, fd, vnet_hdr);
+ s = net_tap_fd_init(peer, "bridge", name, fd, vnet_hdr, 0);
qemu_set_info_str(&s->nc, "helper=%s,br=%s", helper, br);
@@ -765,10 +789,11 @@ static bool net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
const char *name,
const char *ifname, const char *script,
const char *downscript, int vhostfd,
- int vnet_hdr, int fd, Error **errp)
+ int vnet_hdr, int fd, int queue_index,
+ Error **errp)
{
TAPState *s = net_tap_fd_init(peer, tap->helper ? "bridge" : "tap",
- name, fd, vnet_hdr);
+ name, fd, vnet_hdr, queue_index);
bool sndbuf_required = tap->has_sndbuf;
int sndbuf =
(tap->has_sndbuf && tap->sndbuf) ? MIN(tap->sndbuf, INT_MAX) : INT_MAX;
@@ -969,7 +994,7 @@ int net_init_tap(const Netdev *netdev, const char *name,
if (!net_init_tap_one(tap, peer, name, ifname,
NULL, NULL,
vhost_fds ? vhost_fds[i] : -1,
- vnet_hdr, fds[i], errp)) {
+ vnet_hdr, fds[i], i, errp)) {
goto fail;
}
}
@@ -1003,7 +1028,7 @@ int net_init_tap(const Netdev *netdev, const char *name,
i >= 1 ? NULL : script,
i >= 1 ? NULL : downscript,
vhost_fds ? vhost_fds[i] : -1,
- vnet_hdr, fd, errp)) {
+ vnet_hdr, fd, i, errp)) {
goto fail;
}
}
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 06/10] qapi: add local migration parameter
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (4 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 05/10] net/tap: add TYPE_VMSTATE_IF interface Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 07/10] virtio-net: support local migration of backend Vladimir Sementsov-Ogievskiy
` (4 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Eric Blake
We are going to implement local-migration feature: some devices will be
able to transfer open file descriptors through migration stream (which
must UNIX domain socket for that purpose). This allows to transfer the
whole backend state without reconnecting and restarting the backend
service. For example, virtio-net will migrate its attached TAP netdev,
together with its connected file descriptors.
In this commit we introduce a migration parameter, which enables
the feature for devices that support it (none at the moment).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: Markus Armbruster <armbru@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Ben Chaney <bchaney@akamai.com>
---
include/migration/misc.h | 2 ++
migration/options.c | 18 +++++++++++++++++-
qapi/migration.json | 12 ++++++++++--
3 files changed, 29 insertions(+), 3 deletions(-)
diff --git a/include/migration/misc.h b/include/migration/misc.h
index 3159a5e53c3..b14dc70ea3d 100644
--- a/include/migration/misc.h
+++ b/include/migration/misc.h
@@ -156,4 +156,6 @@ bool multifd_device_state_save_thread_should_exit(void);
void multifd_abort_device_state_save_threads(void);
bool multifd_join_device_state_save_threads(void);
+bool migrate_local(void);
+
#endif
diff --git a/migration/options.c b/migration/options.c
index 5cbfd29099b..ba5f9440a08 100644
--- a/migration/options.c
+++ b/migration/options.c
@@ -14,6 +14,7 @@
#include "qemu/osdep.h"
#include "qemu/error-report.h"
#include "qemu/units.h"
+#include "qapi/util.h"
#include "exec/target_page.h"
#include "qapi/clone-visitor.h"
#include "qapi/error.h"
@@ -25,6 +26,7 @@
#include "migration/colo.h"
#include "migration/cpr.h"
#include "migration/misc.h"
+#include "migration/options.h"
#include "migration.h"
#include "migration-stats.h"
#include "qemu-file.h"
@@ -348,6 +350,12 @@ bool migrate_mapped_ram(void)
return s->capabilities[MIGRATION_CAPABILITY_MAPPED_RAM];
}
+bool migrate_local(void)
+{
+ MigrationState *s = migrate_get_current();
+ return s->parameters.local;
+}
+
bool migrate_ignore_shared(void)
{
MigrationState *s = migrate_get_current();
@@ -1076,7 +1084,7 @@ static void migrate_mark_all_params_present(MigrationParameters *p)
&p->has_announce_step, &p->has_block_bitmap_mapping,
&p->has_x_vcpu_dirty_limit_period, &p->has_vcpu_dirty_limit,
&p->has_mode, &p->has_zero_page_detection, &p->has_direct_io,
- &p->has_x_rdma_chunk_size, &p->has_cpr_exec_command,
+ &p->has_x_rdma_chunk_size, &p->has_cpr_exec_command, &p->has_local,
};
len = ARRAY_SIZE(has_fields);
@@ -1424,6 +1432,10 @@ static void migrate_params_test_apply(MigrationParameters *params,
qapi_free_strList(dest->cpr_exec_command);
dest->cpr_exec_command = QAPI_CLONE(strList, params->cpr_exec_command);
}
+
+ if (params->has_local) {
+ dest->local = params->local;
+ }
}
static void migrate_params_apply(MigrationParameters *params)
@@ -1556,6 +1568,10 @@ static void migrate_params_apply(MigrationParameters *params)
s->parameters.cpr_exec_command =
QAPI_CLONE(strList, params->cpr_exec_command);
}
+
+ if (params->has_local) {
+ s->parameters.local = params->local;
+ }
}
void qmp_migrate_set_parameters(MigrationParameters *params, Error **errp)
diff --git a/qapi/migration.json b/qapi/migration.json
index 27a79705569..bc331815da0 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -836,7 +836,8 @@
'zero-page-detection',
'direct-io',
{ 'name': 'x-rdma-chunk-size', 'features': [ 'unstable' ] },
- 'cpr-exec-command'] }
+ 'cpr-exec-command',
+ 'local'] }
##
# @migrate-set-parameters:
@@ -1018,6 +1019,12 @@
# Must be set to the same value on both source and destination
# before migration starts. (Since 11.1)
#
+# @local: Enable local migration for devices that support it. Backend
+# state and its file descriptors can then be passed to the
+# destination in the migration channel. The migration channel
+# must be a Unix domain socket. Usually needs to be enabled per
+# device. (Since 11.1)
+#
# Features:
#
# @unstable: Members @x-checkpoint-delay, @x-rdma-chunk-size, and
@@ -1059,7 +1066,8 @@
'*direct-io': 'bool',
'*x-rdma-chunk-size': { 'type': 'uint64',
'features': [ 'unstable' ] },
- '*cpr-exec-command': [ 'str' ]} }
+ '*cpr-exec-command': [ 'str' ],
+ '*local': 'bool' } }
##
# @query-migrate-parameters:
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 07/10] virtio-net: support local migration of backend
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (5 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 06/10] qapi: add local migration parameter Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 08/10] net/tap: support local migration with virtio-net Vladimir Sementsov-Ogievskiy
` (3 subsequent siblings)
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland
Next commit will introduce live-migration (with fd-passing) for
TAP net backend. So, now we prepare virtio-net for it
Add virtio-net option local-migration, which is true by default,
but false for older machine types, which doesn't support the feature.
We introduce interface for live-migrating backends:
1. ->is_wait_incoming() handler, so that virtio-net knows, that
backend is not fully intialized, as it waits for incoming migration
stream.
2. MIG_PRI_BACKEND priority: backends should migrate with higher
priority than virtio-net, so that we can do final preparations
here in post-load handlers and be sure, that backends are already
prepared.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
hw/net/virtio-net.c | 89 +++++++++++++++++++++++++++++++++-
include/hw/virtio/virtio-net.h | 1 +
include/migration/vmstate.h | 2 +
include/net/net.h | 2 +
4 files changed, 93 insertions(+), 1 deletion(-)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 2a5d642a647..e23ae6052a2 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -40,6 +40,7 @@
#include "migration/misc.h"
#include "standard-headers/linux/ethtool.h"
#include "system/system.h"
+#include "system/runstate.h"
#include "system/replay.h"
#include "trace.h"
#include "monitor/qdev.h"
@@ -3060,7 +3061,17 @@ static void virtio_net_set_multiqueue(VirtIONet *n, int multiqueue)
n->multiqueue = multiqueue;
virtio_net_change_num_queues(n, max * 2 + 1);
- virtio_net_set_queue_pairs(n);
+ /*
+ * virtio_net_set_multiqueue() called from set_features(0) on early
+ * reset, when peer may wait for incoming (and is not initialized
+ * yet).
+ * Don't worry about it: virtio_net_set_queue_pairs() will be called
+ * later from virtio_net_post_load_device(), and anyway will be
+ * no-op for local incoming migration with live backend passing.
+ */
+ if (!n->peers_wait_incoming) {
+ virtio_net_set_queue_pairs(n);
+ }
}
static int virtio_net_pre_load_queues(VirtIODevice *vdev, uint32_t n)
@@ -3089,6 +3100,17 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
virtio_add_feature_ex(features, VIRTIO_NET_F_MAC);
+ if (n->peers_wait_incoming) {
+ /*
+ * Excessive feature set is OK for early initialization when
+ * we wait for local incoming migration: actual guest-negotiated
+ * features will come with migration stream anyway. And we are sure
+ * that we support same host-features as source, because the backend
+ * is the same (the same TAP device, for example).
+ */
+ return;
+ }
+
if (!peer_has_vnet_hdr(n)) {
virtio_clear_feature_ex(features, VIRTIO_NET_F_CSUM);
virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_TSO4);
@@ -3185,6 +3207,7 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
VirtIODevice *vdev = VIRTIO_DEVICE(n);
int i, link_down;
bool has_tunnel_hdr = virtio_has_tunnel_hdr(vdev->guest_features_ex);
+ Error *local_err = NULL;
trace_virtio_net_post_load_device();
virtio_net_set_mrg_rx_bufs(n, n->mergeable_rx_bufs,
@@ -3242,6 +3265,20 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
}
virtio_net_commit_rss_config(n);
+
+ /*
+ * If live-migration is enabled for some backend, than backend
+ * has already been migrated at higher priority (MIG_PRI_BACKEND)
+ * and virtio_net_vnet_post_load() has already called
+ * peer_test_vnet_hdr(). Recompute host_features so that virtio-net
+ * reflects the capabilities of the restored backend.
+ */
+ virtio_net_get_features(vdev, &vdev->host_features, &local_err);
+ if (local_err) {
+ error_report_err(local_err);
+ return -EINVAL;
+ }
+
return 0;
}
@@ -3392,6 +3429,14 @@ static int virtio_net_vnet_post_load(void *opaque, int version_id)
{
struct VirtIONetMigTmp *tmp = opaque;
+ /*
+ * If live-migration is enabled for some backend, than backend
+ * has already been migrated at higher priority (MIG_PRI_BACKEND),
+ * so n->has_vnet_hdr can be refreshed from the live backend right
+ * here.
+ */
+ peer_test_vnet_hdr(tmp->parent);
+
if (tmp->has_vnet_hdr && !peer_has_vnet_hdr(tmp->parent)) {
error_report("virtio-net: saved image requires vnet_hdr=on");
return -EINVAL;
@@ -3864,6 +3909,42 @@ static bool failover_hide_primary_device(DeviceListener *listener,
return qatomic_read(&n->failover_primary_hidden);
}
+static bool virtio_net_check_peers_wait_incoming(VirtIONet *n, bool *waiting,
+ Error **errp)
+{
+ bool has_waiting = false;
+ bool has_not_waiting = false;
+
+ for (int i = 0; i < n->max_queue_pairs; i++) {
+ NetClientState *peer = n->nic->ncs[i].peer;
+ if (!peer) {
+ continue;
+ }
+
+ if (peer->info->is_wait_incoming &&
+ peer->info->is_wait_incoming(peer)) {
+ has_waiting = true;
+ } else {
+ has_not_waiting = true;
+ }
+
+ if (has_waiting && has_not_waiting) {
+ error_setg(errp, "Mixed peer states: some peers wait for incoming "
+ "migration while others don't");
+ return false;
+ }
+ }
+
+ if (has_waiting && !runstate_check(RUN_STATE_INMIGRATE)) {
+ error_setg(errp, "Peers wait for incoming, but it's not an incoming "
+ "migration.");
+ return false;
+ }
+
+ *waiting = has_waiting;
+ return true;
+}
+
static void virtio_net_device_realize(DeviceState *dev, Error **errp)
{
VirtIODevice *vdev = VIRTIO_DEVICE(dev);
@@ -4001,6 +4082,12 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
n->nic->ncs[i].do_not_pad = true;
}
+ if (!virtio_net_check_peers_wait_incoming(n, &n->peers_wait_incoming,
+ errp)) {
+ virtio_cleanup(vdev);
+ return;
+ }
+
peer_test_vnet_hdr(n);
if (peer_has_vnet_hdr(n)) {
n->host_hdr_len = sizeof(struct virtio_net_hdr);
diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
index 371e3764282..8c967760c2a 100644
--- a/include/hw/virtio/virtio-net.h
+++ b/include/hw/virtio/virtio-net.h
@@ -230,6 +230,7 @@ struct VirtIONet {
struct EBPFRSSContext ebpf_rss;
uint32_t nr_ebpf_rss_fds;
char **ebpf_rss_fds;
+ bool peers_wait_incoming;
};
size_t virtio_net_handle_ctrl_iov(VirtIODevice *vdev,
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 0a8a2e85a63..c92c41397ae 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -178,6 +178,8 @@ typedef enum {
MIG_PRI_LOW, /* Must happen after default */
MIG_PRI_DEFAULT,
+ MIG_PRI_BACKEND, /* Must happen before emulated devices, */
+ /* e.g. virtio-net */
MIG_PRI_IOMMU, /* Must happen before PCI devices */
MIG_PRI_PCI_BUS, /* Must happen before IOMMU */
MIG_PRI_VIRTIO_MEM, /* Must happen before IOMMU */
diff --git a/include/net/net.h b/include/net/net.h
index bdf16bb3e64..658b0d0aaf1 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -81,6 +81,7 @@ typedef void (SocketReadStateFinalize)(SocketReadState *rs);
typedef void (NetAnnounce)(NetClientState *);
typedef bool (SetSteeringEBPF)(NetClientState *, int);
typedef bool (NetCheckPeerType)(NetClientState *, ObjectClass *, Error **);
+typedef bool (IsWaitIncoming)(NetClientState *);
typedef struct vhost_net *(GetVHostNet)(NetClientState *nc);
typedef struct NetClientInfo {
@@ -109,6 +110,7 @@ typedef struct NetClientInfo {
NetAnnounce *announce;
SetSteeringEBPF *set_steering_ebpf;
NetCheckPeerType *check_peer_type;
+ IsWaitIncoming *is_wait_incoming;
GetVHostNet *get_vhost_net;
} NetClientInfo;
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (6 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 07/10] virtio-net: support local migration of backend Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-28 16:03 ` Vladimir Sementsov-Ogievskiy
2026-06-02 8:02 ` Markus Armbruster
2026-05-27 19:11 ` [PATCH v17 09/10] tests/functional: add skipWithoutSudo() decorator Vladimir Sementsov-Ogievskiy
` (2 subsequent siblings)
10 siblings, 2 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake
Support transferring of TAP state (including open fd).
Add new property "local-migration-supported", which defines,
is local-migration is actually supported for this TAP device.
Starting from 11.1 MT it's enabled by default.
Note, that local-migration (including migrating opened FDs
through migration channel, which must be UNIX socket), is
enabled by global "local" migration parameters. But individual
devices may have additional options to enable/disable it
personally.
Add new option "incoming-fds", which should be set to true on
target for incoming migration work. It says "do not open any
files, but instead wait for FDs coming from migration stream".
"local-migration-supported" option is not enough, as it work in pair
with migration parameter "local", and intialization process
of TAP device should not depend on migration parameters.
For new option require explicitly unset script and downscript,
to keep possibility of implementing support for them in the
future.
Note disabling read polling on source stop for TAP migration:
otherwise, source process may steal packages from TAP fd even
after source vm STOP.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
hw/core/machine.c | 2 +
net/tap.c | 221 +++++++++++++++++++++++++++++++++++++++++++---
qapi/net.json | 20 ++++-
3 files changed, 231 insertions(+), 12 deletions(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 63baff859f3..306f01b1a68 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -36,11 +36,13 @@
#include "hw/virtio/virtio-pci.h"
#include "hw/virtio/virtio-net.h"
#include "hw/virtio/virtio-iommu.h"
+#include "net/tap.h"
#include "hw/acpi/generic_event_device.h"
#include "qemu/audio.h"
GlobalProperty hw_compat_11_0[] = {
{ "chardev-vc", "encoding", "cp437" },
+ { TYPE_TAP_NETDEV, "local-migration-supported", "false" },
};
const size_t hw_compat_11_0_len = G_N_ELEMENTS(hw_compat_11_0);
diff --git a/net/tap.c b/net/tap.c
index 529ed4b0f5b..c84daad9895 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -36,13 +36,19 @@
#include "net/net.h"
#include "clients.h"
#include "monitor/monitor.h"
+#include "system/runstate.h"
#include "system/system.h"
+#include "migration/misc.h"
#include "qapi/error.h"
#include "qemu/cutils.h"
#include "qemu/error-report.h"
#include "qemu/main-loop.h"
#include "qemu/sockets.h"
#include "hw/virtio/vhost.h"
+#include "hw/core/vmstate-if.h"
+#include "migration/vmstate.h"
+#include "qom/object.h"
+#include "qom/compat-properties.h"
#include "net/tap.h"
#include "net/util.h"
@@ -70,6 +76,8 @@ static const int kernel_feature_bits[] = {
OBJECT_DECLARE_SIMPLE_TYPE(TAPState, TAP_NETDEV)
+static const VMStateDescription vmstate_tap;
+
struct TAPState {
Object parent_obj;
@@ -92,6 +100,9 @@ struct TAPState {
Notifier exit;
int queue_index;
+ bool read_poll_detached;
+ VMChangeStateEntry *vmstate;
+ bool local_migration_supported;
};
static void launch_script(const char *setup_script, const char *ifname,
@@ -100,19 +111,25 @@ static void launch_script(const char *setup_script, const char *ifname,
static void tap_send(void *opaque);
static void tap_writable(void *opaque);
+static bool tap_is_explicit_no_script(const char *script_arg)
+{
+ return script_arg &&
+ (script_arg[0] == '\0' || strcmp(script_arg, "no") == 0);
+}
+
static char *tap_parse_script(const char *script_arg, const char *default_path)
{
g_autofree char *res = g_strdup(script_arg);
- if (!res) {
- res = get_relocated_path(default_path);
+ if (tap_is_explicit_no_script(script_arg)) {
+ return NULL;
}
- if (res[0] == '\0' || strcmp(res, "no") == 0) {
- return NULL;
+ if (!script_arg) {
+ return get_relocated_path(default_path);
}
- return g_steal_pointer(&res);
+ return g_strdup(script_arg);
}
static void tap_update_fd_handler(TAPState *s)
@@ -129,6 +146,23 @@ static void tap_read_poll(TAPState *s, bool enable)
tap_update_fd_handler(s);
}
+static void tap_vm_state_change(void *opaque, bool running, RunState state)
+{
+ TAPState *s = opaque;
+
+ if (running) {
+ if (s->read_poll_detached) {
+ tap_read_poll(s, true);
+ s->read_poll_detached = false;
+ }
+ } else if (state == RUN_STATE_FINISH_MIGRATE) {
+ if (s->read_poll) {
+ s->read_poll_detached = true;
+ tap_read_poll(s, false);
+ }
+ }
+}
+
static void tap_write_poll(TAPState *s, bool enable)
{
s->write_poll = enable;
@@ -359,10 +393,17 @@ static void tap_cleanup(NetClientState *nc)
s->exit.notify = NULL;
}
+ if (s->vmstate) {
+ qemu_del_vm_change_state_handler(s->vmstate);
+ s->vmstate = NULL;
+ }
+
tap_read_poll(s, false);
tap_write_poll(s, false);
close(s->fd);
s->fd = -1;
+
+ vmstate_unregister(VMSTATE_IF(s), &vmstate_tap, s);
}
static void tap_poll(NetClientState *nc, bool enable)
@@ -399,6 +440,73 @@ static VHostNetState *tap_get_vhost_net(NetClientState *nc)
return s->vhost_net;
}
+static bool tap_is_wait_incoming(NetClientState *nc)
+{
+ TAPState *s = container_of(nc, TAPState, nc);
+ assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
+ return s->fd == -1;
+}
+
+static int tap_pre_load(void *opaque)
+{
+ TAPState *s = opaque;
+
+ if (s->fd != -1) {
+ error_report(
+ "TAP is already initialized and cannot receive incoming fd");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static bool tap_setup_vhost(TAPState *s, Error **errp);
+
+static int tap_post_load(void *opaque, int version_id)
+{
+ TAPState *s = opaque;
+ Error *local_err = NULL;
+
+ tap_read_poll(s, true);
+
+ if (s->fd < 0) {
+ return -1;
+ }
+
+ if (!tap_setup_vhost(s, &local_err)) {
+ error_prepend(&local_err,
+ "Failed to setup vhost during TAP post-load: ");
+ error_report_err(local_err);
+ return -1;
+ }
+
+ return 0;
+}
+
+static bool tap_needed(void *opaque)
+{
+ TAPState *s = opaque;
+
+ return s->local_migration_supported && migrate_local();
+}
+
+static const VMStateDescription vmstate_tap = {
+ .name = "net-tap",
+ .priority = MIG_PRI_BACKEND,
+ .pre_load = tap_pre_load,
+ .post_load = tap_post_load,
+ .needed = tap_needed,
+ .fields = (const VMStateField[]) {
+ VMSTATE_FD(fd, TAPState),
+ VMSTATE_BOOL(using_vnet_hdr, TAPState),
+ VMSTATE_BOOL(has_ufo, TAPState),
+ VMSTATE_BOOL(has_uso, TAPState),
+ VMSTATE_BOOL(has_tunnel, TAPState),
+ VMSTATE_BOOL(enabled, TAPState),
+ VMSTATE_UINT32(host_vnet_hdr_len, TAPState),
+ VMSTATE_END_OF_LIST()
+ }
+};
static char *tap_vmstate_if_get_id(VMStateIf *obj)
{
@@ -407,17 +515,42 @@ static char *tap_vmstate_if_get_id(VMStateIf *obj)
return res;
}
+static bool tap_get_local_migration_supported_prop(Object *obj, Error **errp)
+{
+ TAPState *s = TAP_NETDEV(obj);
+ return s->local_migration_supported;
+}
+
+static void tap_set_local_migration_supported_prop(Object *obj, bool value,
+ Error **errp)
+{
+ TAPState *s = TAP_NETDEV(obj);
+ s->local_migration_supported = value;
+}
+
+static void tap_instance_init(Object *obj)
+{
+ TAPState *s = TAP_NETDEV(obj);
+ s->local_migration_supported = true;
+}
+
static void tap_class_init(ObjectClass *klass, const void *data)
{
VMStateIfClass *vc = VMSTATE_IF_CLASS(klass);
vc->get_id = tap_vmstate_if_get_id;
+
+ object_class_property_add_bool(klass, "local-migration-supported",
+ tap_get_local_migration_supported_prop,
+ tap_set_local_migration_supported_prop);
}
static const TypeInfo tap_netdev_info = {
.name = TYPE_TAP_NETDEV,
.parent = TYPE_OBJECT,
.instance_size = sizeof(TAPState),
+ .instance_init = tap_instance_init,
+ .instance_post_init = object_apply_compat_props,
.class_init = tap_class_init,
.interfaces = (const InterfaceInfo[]) {
{ TYPE_VMSTATE_IF },
@@ -450,13 +583,16 @@ static NetClientInfo net_tap_info = {
.set_vnet_le = tap_set_vnet_le,
.set_vnet_be = tap_set_vnet_be,
.set_steering_ebpf = tap_set_steering_ebpf,
+ .is_wait_incoming = tap_is_wait_incoming,
.get_vhost_net = tap_get_vhost_net,
};
static TAPState *new_tap(NetClientState *peer,
const char *model,
const char *name,
- int queue_index)
+ int queue_index,
+ bool has_local_migration_supported,
+ bool local_migration_supported)
{
TAPState *s = TAP_NETDEV(object_new(TYPE_TAP_NETDEV));
@@ -465,6 +601,12 @@ static TAPState *new_tap(NetClientState *peer,
s->queue_index = queue_index;
+ if (has_local_migration_supported) {
+ s->local_migration_supported = local_migration_supported;
+ }
+
+ vmstate_register(VMSTATE_IF(s), VMSTATE_INSTANCE_ID_ANY, &vmstate_tap, s);
+
return s;
}
@@ -473,10 +615,14 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
const char *name,
int fd,
int vnet_hdr,
- int queue_index)
+ int queue_index,
+ bool has_local_migration_supported,
+ bool local_migration_supported)
{
NetOffloads ol = {};
- TAPState *s = new_tap(peer, model, name, queue_index);
+ TAPState *s = new_tap(peer, model, name, queue_index,
+ has_local_migration_supported,
+ local_migration_supported);
s->fd = fd;
s->host_vnet_hdr_len = vnet_hdr ? sizeof(struct virtio_net_hdr) : 0;
@@ -713,7 +859,7 @@ int net_init_bridge(const Netdev *netdev, const char *name,
close(fd);
return -1;
}
- s = net_tap_fd_init(peer, "bridge", name, fd, vnet_hdr, 0);
+ s = net_tap_fd_init(peer, "bridge", name, fd, vnet_hdr, 0, true, false);
qemu_set_info_str(&s->nc, "helper=%s,br=%s", helper, br);
@@ -793,11 +939,16 @@ static bool net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
Error **errp)
{
TAPState *s = net_tap_fd_init(peer, tap->helper ? "bridge" : "tap",
- name, fd, vnet_hdr, queue_index);
+ name, fd, vnet_hdr, queue_index,
+ tap->has_local_migration_supported,
+ tap->local_migration_supported);
bool sndbuf_required = tap->has_sndbuf;
int sndbuf =
(tap->has_sndbuf && tap->sndbuf) ? MIN(tap->sndbuf, INT_MAX) : INT_MAX;
+ s->read_poll_detached = false;
+ s->vmstate = qemu_add_vm_change_state_handler(tap_vm_state_change, s);
+
if (!tap_set_sndbuf(fd, sndbuf, sndbuf_required ? errp : NULL) &&
sndbuf_required) {
goto failed;
@@ -960,6 +1111,31 @@ int net_init_tap(const Netdev *netdev, const char *name,
return -1;
}
+ if (tap->incoming_fds &&
+ (tap->fd || tap->fds || tap->helper || tap->br || tap->ifname ||
+ tap->has_sndbuf || tap->has_vnet_hdr)) {
+ error_setg(errp, "incoming-fds is incompatible with "
+ "fd=, fds=, helper=, br=, ifname=, sndbuf= and vnet_hdr=");
+ return -1;
+ }
+
+ if (tap->incoming_fds &&
+ !(tap_is_explicit_no_script(tap->script) &&
+ tap_is_explicit_no_script(tap->downscript))) {
+ /*
+ * script="" and downscript="" are silently supported to be consistent
+ * with cases without incoming_fds, but do not care to put this into
+ * error message.
+ */
+ error_setg(errp, "incoming-fds requires script=no and downscript=no");
+ return -1;
+ }
+
+ if (tap->incoming_fds && !runstate_check(RUN_STATE_INMIGRATE)) {
+ error_setg(errp, "incoming-fds requires -incoming commandline option");
+ return -1;
+ }
+
queues = tap_parse_fds_and_queues(tap, &fds, errp);
if (queues < 0) {
return -1;
@@ -978,7 +1154,30 @@ int net_init_tap(const Netdev *netdev, const char *name,
goto fail;
}
- if (fds) {
+ if (tap->incoming_fds) {
+ for (i = 0; i < queues; i++) {
+ TAPState *s = new_tap(peer, "tap", name, i,
+ tap->has_local_migration_supported,
+ tap->local_migration_supported);
+
+ if (!s->local_migration_supported) {
+ error_setg(errp,
+ "incoming-fds=true requires local-migration-supported=true");
+ qemu_del_net_client(&s->nc);
+ return -1;
+ }
+
+ qemu_set_info_str(&s->nc, "incoming");
+
+ s->fd = -1;
+ if (vhost_fds) {
+ s->vhostfd = vhost_fds[i];
+ s->vhost_busyloop_timeout = tap->has_poll_us ? tap->poll_us : 0;
+ } else {
+ s->vhostfd = -1;
+ }
+ }
+ } else if (fds) {
for (i = 0; i < queues; i++) {
if (i == 0) {
vnet_hdr = tap_probe_vnet_hdr(fds[i], errp);
diff --git a/qapi/net.json b/qapi/net.json
index 1a6382825c5..c5d87ba308b 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -425,6 +425,22 @@
# @poll-us: maximum number of microseconds that could be spent on busy
# polling for tap (since 2.7)
#
+# @incoming-fds: do not open or create any TAP devices. Prepare for
+# getting TAP file descriptors from incoming migration stream.
+# The option is incompatible with any of @fd, @fds, @helper, @br,
+# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
+# @downscript be explicitly set to nothing (empty string or "no"),
+# and requires also @local-migration-supported to be true, "local"
+# migration parameter be set as well, and QEMU being in incoming
+# migration state. (Since 11.1)
+#
+# @local-migration-supported: enable local migration for this TAP
+# backend. When set, local migration is enabled/disabled by
+# "local" migration parameter for this TAP backend. When unset,
+# "local" migration parameter is ignored for this TAP backend.
+# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
+# MT < 11.1)
+#
# Since: 1.2
##
{ 'struct': 'NetdevTapOptions',
@@ -443,7 +459,9 @@
'*vhostfds': 'str',
'*vhostforce': 'bool',
'*queues': 'uint32',
- '*poll-us': 'uint32'} }
+ '*poll-us': 'uint32',
+ '*incoming-fds': 'bool',
+ '*local-migration-supported': 'bool' } }
##
# @NetdevSocketOptions:
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 09/10] tests/functional: add skipWithoutSudo() decorator
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (7 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 08/10] net/tap: support local migration with virtio-net Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 10/10] tests/functional: add test_tap_migration Vladimir Sementsov-Ogievskiy
2026-06-03 15:24 ` [PATCH v17 00/10] virtio-net: live-TAP local migration Mark Cave-Ayland
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Thomas Huth, Lei Yang, Maksim Davydov, Thomas Huth,
Philippe Mathieu-Daudé
To be used in the next commit: that would be a test for TAP
networking, and it will need to setup TAP device.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Lei Yang <leiyang@redhat.com>
Reviewed-by: Maksim Davydov <davydov-max@yandex-team.ru>
Reviewed-by: Ben Chaney <bchaney@akamai.com>
---
tests/functional/qemu_test/decorators.py | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/tests/functional/qemu_test/decorators.py b/tests/functional/qemu_test/decorators.py
index fcf236ecfdf..aa135acc785 100644
--- a/tests/functional/qemu_test/decorators.py
+++ b/tests/functional/qemu_test/decorators.py
@@ -6,6 +6,7 @@
import os
import platform
import resource
+import subprocess
from unittest import skipIf, skipUnless
from .cmd import which
@@ -177,3 +178,18 @@ def skipLockedMemoryTest(locked_memory):
ulimit_memory == resource.RLIM_INFINITY or ulimit_memory >= locked_memory * 1024,
f'Test required {locked_memory} kB of available locked memory',
)
+
+'''
+Decorator to skip execution of a test if passwordless
+sudo command is not available.
+'''
+def skipWithoutSudo():
+ proc = subprocess.run(["sudo", "-n", "/bin/true"],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ universal_newlines=True,
+ check=False)
+
+ return skipUnless(proc.returncode == 0,
+ f'requires password-less sudo access: {proc.stdout}')
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v17 10/10] tests/functional: add test_tap_migration
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (8 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 09/10] tests/functional: add skipWithoutSudo() decorator Vladimir Sementsov-Ogievskiy
@ 2026-05-27 19:11 ` Vladimir Sementsov-Ogievskiy
2026-06-03 15:24 ` [PATCH v17 00/10] virtio-net: live-TAP local migration Mark Cave-Ayland
10 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-27 19:11 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, vsementsov,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Zhao Liu
Add test for a new local-migration migration of virtio-net/tap, with fd
passing through UNIX socket.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Ben Chaney <bchaney@akamai.com>
---
tests/functional/x86_64/meson.build | 1 +
tests/functional/x86_64/test_tap_migration.py | 458 ++++++++++++++++++
2 files changed, 459 insertions(+)
create mode 100755 tests/functional/x86_64/test_tap_migration.py
diff --git a/tests/functional/x86_64/meson.build b/tests/functional/x86_64/meson.build
index 1ed10ad6c29..c4bb14f034b 100644
--- a/tests/functional/x86_64/meson.build
+++ b/tests/functional/x86_64/meson.build
@@ -45,4 +45,5 @@ tests_x86_64_system_thorough = [
'virtio_balloon',
'virtio_gpu',
'rebuild_vmfd',
+ 'tap_migration',
]
diff --git a/tests/functional/x86_64/test_tap_migration.py b/tests/functional/x86_64/test_tap_migration.py
new file mode 100755
index 00000000000..6c99cd82834
--- /dev/null
+++ b/tests/functional/x86_64/test_tap_migration.py
@@ -0,0 +1,458 @@
+#!/usr/bin/env python3
+#
+# Functional test that tests TAP local migration
+# with fd passing
+#
+# Copyright (c) Yandex Technologies LLC, 2026
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import time
+import subprocess
+from subprocess import run
+import signal
+import ctypes
+import ctypes.util
+import unittest
+from contextlib import contextmanager, ExitStack
+from typing import Tuple
+
+from qemu_test import (
+ LinuxKernelTest,
+ Asset,
+ exec_command_and_wait_for_pattern,
+)
+from qemu_test.decorators import skipWithoutSudo
+
+
+GUEST_IP = "192.168.100.2"
+GUEST_IP_MASK = f"{GUEST_IP}/24"
+GUEST_MAC = "d6:0d:75:f8:0f:b7"
+HOST_IP = "192.168.100.1"
+HOST_IP_MASK = f"{HOST_IP}/24"
+TAP_ID = "tap0"
+TAP_ID2 = "tap1"
+TAP_MAC = "e6:1d:44:b5:03:5d"
+NETNS = f"qemu_test_ns_{os.getpid()}"
+
+
+def ip(args, check=True) -> None:
+ """Run ip command with sudo"""
+ run(["sudo", "ip"] + args, check=check)
+
+
+@contextmanager
+def switch_netns(netns_name):
+ libc = ctypes.CDLL(ctypes.util.find_library("c"))
+ netns_path = f"/var/run/netns/{netns_name}"
+
+ def switch_to_fd(fd, check: bool = False):
+ """Switch to netns by file descriptor"""
+ SYS_setns = 308
+ CLONE_NEWNET = 0x40000000
+ ret = libc.syscall(SYS_setns, fd, CLONE_NEWNET)
+ if check and ret != 0:
+ raise RuntimeError("syscall SETNS failed")
+
+ with ExitStack() as stack:
+ original_netns_fd = os.open("/proc/self/ns/net", os.O_RDONLY)
+ stack.callback(os.close, original_netns_fd)
+
+ ip(["netns", "add", netns_name])
+ stack.callback(ip, ["netns", "del", netns_name], check=False)
+
+ new_netns_fd = os.open(netns_path, os.O_RDONLY)
+ stack.callback(os.close, new_netns_fd)
+
+ switch_to_fd(new_netns_fd)
+ stack.callback(switch_to_fd, original_netns_fd, check=False)
+
+ yield
+
+
+def del_tap(tap_name: str = TAP_ID) -> None:
+ ip(["tuntap", "del", tap_name, "mode", "tap", "multi_queue"], check=False)
+
+
+def init_tap(tap_name: str = TAP_ID, with_ip: bool = True) -> None:
+ ip(["tuntap", "add", "dev", tap_name, "mode", "tap", "multi_queue"])
+ if with_ip:
+ ip(["link", "set", "dev", tap_name, "address", TAP_MAC])
+ ip(["addr", "add", HOST_IP_MASK, "dev", tap_name])
+ ip(["link", "set", tap_name, "up"])
+
+
+def switch_network_to_tap2() -> None:
+ ip(["link", "set", TAP_ID2, "down"])
+ ip(["link", "set", TAP_ID, "down"])
+ ip(["addr", "delete", HOST_IP_MASK, "dev", TAP_ID])
+ ip(["link", "set", "dev", TAP_ID2, "address", TAP_MAC])
+ ip(["addr", "add", HOST_IP_MASK, "dev", TAP_ID2])
+ ip(["link", "set", TAP_ID2, "up"])
+
+
+def parse_ping_line(line: str) -> float:
+ # suspect lines like
+ # [1748524876.590509] 64 bytes from 94.245.155.3 \
+ # (94.245.155.3): icmp_seq=1 ttl=250 time=101 ms
+ spl = line.split()
+ return float(spl[0][1:-1])
+
+
+def parse_ping_output(out) -> Tuple[bool, float, float]:
+ lines = [x for x in out.split("\n") if x.startswith("[")]
+
+ try:
+ first_no_ans = next(
+ (ind for ind in range(len(lines)) if lines[ind][20:26] == "no ans")
+ )
+ except StopIteration:
+ return False, parse_ping_line(lines[0]), parse_ping_line(lines[-1])
+
+ last_no_ans = next(
+ ind
+ for ind in range(len(lines) - 1, -1, -1)
+ if lines[ind][20:26] == "no ans"
+ )
+
+ return (
+ True,
+ parse_ping_line(lines[first_no_ans]),
+ parse_ping_line(lines[last_no_ans]),
+ )
+
+
+def wait_migration_finish(source_vm, target_vm):
+ migr_events = (
+ ("MIGRATION", {"data": {"status": "completed"}}),
+ ("MIGRATION", {"data": {"status": "failed"}}),
+ )
+
+ source_e = source_vm.events_wait(migr_events)["data"]
+ target_e = target_vm.events_wait(migr_events)["data"]
+
+ source_s = source_vm.cmd("query-status")["status"]
+ target_s = target_vm.cmd("query-status")["status"]
+
+ assert (
+ source_e["status"] == "completed"
+ and target_e["status"] == "completed"
+ and source_s == "postmigrate"
+ and target_s == "paused"
+ ), f"""Migration failed:
+ SRC status: {source_s}
+ SRC event: {source_e}
+ TGT status: {target_s}
+ TGT event:{target_e}"""
+
+
+@skipWithoutSudo()
+class TAPFdMigration(LinuxKernelTest):
+
+ ASSET_KERNEL = Asset(
+ (
+ "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases"
+ "/31/Server/x86_64/os/images/pxeboot/vmlinuz"
+ ),
+ "d4738d03dbbe083ca610d0821d0a8f1488bebbdccef54ce33e3adb35fda00129",
+ )
+
+ ASSET_INITRD = Asset(
+ (
+ "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases"
+ "/31/Server/x86_64/os/images/pxeboot/initrd.img"
+ ),
+ "277cd6c7adf77c7e63d73bbb2cded8ef9e2d3a2f100000e92ff1f8396513cd8b",
+ )
+
+ ASSET_ALPINE_ISO = Asset(
+ (
+ "https://dl-cdn.alpinelinux.org/"
+ "alpine/v3.22/releases/x86_64/alpine-standard-3.22.1-x86_64.iso"
+ ),
+ "96d1b44ea1b8a5a884f193526d92edb4676054e9fa903ad2f016441a0fe13089",
+ )
+
+ @classmethod
+ def setUpClass(cls):
+ super().setUpClass()
+
+ try:
+ cls.netns_context = switch_netns(NETNS)
+ cls.netns_context.__enter__()
+ except (OSError, subprocess.CalledProcessError) as e:
+ raise unittest.SkipTest(f"can't switch network namespace: {e}")
+
+ @classmethod
+ def tearDownClass(cls):
+ if hasattr(cls, "netns_context"):
+ cls.netns_context.__exit__(None, None, None)
+ super().tearDownClass()
+
+ def setUp(self):
+ super().setUp()
+
+ init_tap()
+
+ self.outer_ping_proc = None
+ self.shm_path = None
+
+ def tearDown(self):
+ try:
+ del_tap(TAP_ID)
+ del_tap(TAP_ID2)
+
+ if self.outer_ping_proc:
+ self.stop_outer_ping()
+
+ if self.shm_path:
+ os.unlink(self.shm_path)
+ finally:
+ super().tearDown()
+
+ def start_outer_ping(self) -> None:
+ assert self.outer_ping_proc is None
+ self.outer_ping_log = self.scratch_file("ping.log")
+ with open(self.outer_ping_log, "w") as f:
+ self.outer_ping_proc = subprocess.Popen(
+ ["ping", "-i", "0", "-O", "-D", GUEST_IP],
+ text=True,
+ stdout=f,
+ )
+
+ def stop_outer_ping(self) -> str:
+ assert self.outer_ping_proc
+ self.outer_ping_proc.send_signal(signal.SIGINT)
+
+ self.outer_ping_proc.communicate(timeout=5)
+ self.outer_ping_proc = None
+
+ with open(self.outer_ping_log) as f:
+ return f.read()
+
+ def stop_ping_and_check(self, stop_time, resume_time):
+ ping_res = self.stop_outer_ping()
+
+ discon, a, b = parse_ping_output(ping_res)
+
+ if not discon:
+ text = (
+ f"STOP: {stop_time}, RESUME: {resume_time}," f"PING: {a} - {b}"
+ )
+ if a > stop_time or b < resume_time:
+ self.fail(f"PING failed: {text}")
+ self.log.info(f"PING: no packets lost: {text}")
+ return
+
+ text = (
+ f"STOP: {stop_time}, RESUME: {resume_time},"
+ f"PING: disconnect: {a} - {b}"
+ )
+ self.log.info(text)
+ eps = 0.05
+ if a < stop_time - eps or b > resume_time + eps:
+ self.fail(text)
+
+ def one_ping_from_guest(self, vm) -> None:
+ exec_command_and_wait_for_pattern(
+ self,
+ f"ping -c 1 -W 1 {HOST_IP}",
+ "1 packets transmitted, 1 packets received",
+ "1 packets transmitted, 0 packets received",
+ vm=vm,
+ )
+ self.wait_for_console_pattern("# ", vm=vm)
+
+ def one_ping_from_host(self) -> None:
+ run(
+ ["ping", "-c", "1", "-W", "1", GUEST_IP],
+ stdout=subprocess.DEVNULL,
+ check=True,
+ )
+
+ def setup_shared_memory(self):
+ self.shm_path = f"/dev/shm/qemu_test_{os.getpid()}"
+
+ try:
+ with open(self.shm_path, "wb") as f:
+ f.write(b"\0" * (1024 * 1024 * 1024)) # 1GB
+ except Exception as e:
+ self.fail(f"Failed to create shared memory file: {e}")
+
+ def prepare_and_launch_vm(
+ self, shm_path, vhost, incoming=False, vm=None, local=True
+ ):
+ if not vm:
+ vm = self.vm
+
+ vm.set_console()
+ vm.add_args("-accel", "kvm")
+ vm.add_args("-device", "pcie-pci-bridge,id=pci.1,bus=pcie.0")
+ vm.add_args("-m", "1G")
+
+ vm.add_args(
+ "-object",
+ f"memory-backend-file,id=ram0,size=1G,mem-path={shm_path},share=on",
+ )
+ vm.add_args("-machine", "memory-backend=ram0")
+
+ vm.add_args(
+ "-drive",
+ f"file={self.ASSET_ALPINE_ISO.fetch()},media=cdrom,format=raw",
+ )
+
+ vm.add_args("-S")
+
+ if incoming:
+ vm.add_args("-incoming", "defer")
+
+ vm_s = "target" if incoming else "source"
+ self.log.info(f"Launching {vm_s} VM")
+ vm.launch()
+
+ if not local:
+ tap_name = TAP_ID2 if incoming else TAP_ID
+ else:
+ tap_name = TAP_ID
+
+ self.add_virtio_net(vm, vhost, tap_name, local, incoming)
+
+ self.set_migration_capabilities(vm, local)
+
+ def add_virtio_net(
+ self, vm, vhost: bool, tap_name: str, local: bool, incoming: bool
+ ):
+ incoming_fds = local and incoming
+ netdev_params = {
+ "id": "netdev.1",
+ "vhost": vhost,
+ "type": "tap",
+ "queues": 4,
+ "incoming-fds": incoming_fds,
+ "script": "no",
+ "downscript": "no",
+ "local-migration-supported": local,
+ }
+
+ if not incoming_fds:
+ netdev_params["vnet_hdr"] = True
+ netdev_params["ifname"] = tap_name
+
+ vm.cmd("netdev_add", netdev_params)
+
+ vm.cmd(
+ "device_add",
+ driver="virtio-net-pci",
+ romfile="",
+ id="vnet.1",
+ netdev="netdev.1",
+ mq=True,
+ vectors=18,
+ bus="pci.1",
+ mac=GUEST_MAC,
+ disable_legacy="off",
+ )
+
+ def set_migration_capabilities(self, vm, local=True):
+ vm.cmd(
+ "migrate-set-capabilities",
+ {
+ "capabilities": [
+ {"capability": "events", "state": True},
+ {"capability": "x-ignore-shared", "state": True},
+ ]
+ },
+ )
+ vm.cmd("migrate-set-parameters", {"local": local})
+
+ def setup_guest_network(self) -> None:
+ exec_command_and_wait_for_pattern(self, "ip addr", "# ")
+ exec_command_and_wait_for_pattern(
+ self,
+ f"ip addr add {GUEST_IP_MASK} dev eth0 && "
+ "ip link set eth0 up && echo OK",
+ "OK",
+ )
+ self.wait_for_console_pattern("# ")
+
+ def do_test_tap_fd_migration(self, vhost, local=True):
+ self.require_accelerator("kvm")
+ self.set_machine("q35")
+
+ socket_dir = self.socket_dir()
+ migration_socket = os.path.join(socket_dir.name, "migration.sock")
+
+ self.setup_shared_memory()
+
+ # Setup second TAP if needed
+ if not local:
+ del_tap(TAP_ID2)
+ init_tap(TAP_ID2, with_ip=False)
+
+ self.prepare_and_launch_vm(self.shm_path, vhost, local=local)
+ self.vm.cmd("cont")
+ self.wait_for_console_pattern("login:")
+ exec_command_and_wait_for_pattern(self, "root", "# ")
+
+ self.setup_guest_network()
+
+ self.one_ping_from_guest(self.vm)
+ self.one_ping_from_host()
+ self.start_outer_ping()
+
+ # Get some successful pings before migration
+ time.sleep(0.5)
+
+ target_vm = self.get_vm(name="target")
+ self.prepare_and_launch_vm(
+ self.shm_path,
+ vhost,
+ incoming=True,
+ vm=target_vm,
+ local=local,
+ )
+
+ target_vm.cmd("migrate-incoming", {"uri": f"unix:{migration_socket}"})
+
+ self.log.info("Starting migration")
+ freeze_start = time.time()
+ self.vm.cmd("migrate", {"uri": f"unix:{migration_socket}"})
+
+ self.log.info("Waiting for migration completion")
+ wait_migration_finish(self.vm, target_vm)
+
+ # Switch network to tap1 if not using local-migration
+ if not local:
+ switch_network_to_tap2()
+
+ target_vm.cmd("cont")
+ freeze_end = time.time()
+
+ self.vm.shutdown()
+
+ self.log.info("Verifying PING on target VM after migration")
+ self.one_ping_from_guest(target_vm)
+ self.one_ping_from_host()
+
+ # And a bit more pings after source shutdown
+ time.sleep(0.3)
+ self.stop_ping_and_check(freeze_start, freeze_end)
+
+ target_vm.shutdown()
+
+ def test_tap_fd_migration(self):
+ self.do_test_tap_fd_migration(False)
+
+ def test_tap_fd_migration_vhost(self):
+ self.do_test_tap_fd_migration(True)
+
+ def test_tap_new_tap_migration(self):
+ self.do_test_tap_fd_migration(False, local=False)
+
+ def test_tap_new_tap_migration_vhost(self):
+ self.do_test_tap_fd_migration(True, local=False)
+
+
+if __name__ == "__main__":
+ LinuxKernelTest.main()
--
2.52.0
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-05-27 19:11 ` [PATCH v17 08/10] net/tap: support local migration with virtio-net Vladimir Sementsov-Ogievskiy
@ 2026-05-28 16:03 ` Vladimir Sementsov-Ogievskiy
2026-06-02 8:02 ` Markus Armbruster
1 sibling, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-05-28 16:03 UTC (permalink / raw)
To: jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, qemu-devel,
berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake
On 27.05.26 22:11, Vladimir Sementsov-Ogievskiy wrote:
> diff --git a/qapi/net.json b/qapi/net.json
> index 1a6382825c5..c5d87ba308b 100644
> --- a/qapi/net.json
> +++ b/qapi/net.json
> @@ -425,6 +425,22 @@
> # @poll-us: maximum number of microseconds that could be spent on busy
> # polling for tap (since 2.7)
> #
> +# @incoming-fds: do not open or create any TAP devices. Prepare for
> +# getting TAP file descriptors from incoming migration stream.
> +# The option is incompatible with any of @fd, @fds, @helper, @br,
> +# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
> +# @downscript be explicitly set to nothing (empty string or "no"),
> +# and requires also @local-migration-supported to be true, "local"
> +# migration parameter be set as well, and QEMU being in incoming
> +# migration state. (Since 11.1)
> +#
> +# @local-migration-supported: enable local migration for this TAP
> +# backend. When set, local migration is enabled/disabled by
> +# "local" migration parameter for this TAP backend. When unset,
> +# "local" migration parameter is ignored for this TAP backend.
> +# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
> +# MT < 11.1)
> +#
> # Since: 1.2
> ##
> { 'struct': 'NetdevTapOptions',
> @@ -443,7 +459,9 @@
> '*vhostfds': 'str',
> '*vhostforce': 'bool',
> '*queues': 'uint32',
> - '*poll-us': 'uint32'} }
> + '*poll-us': 'uint32',
> + '*incoming-fds': 'bool',
> + '*local-migration-supported': 'bool' } }
>
> ##
> # @NetdevSocketOptions:
squash in, as discussed here https://lore.kernel.org/qemu-devel/ahhGBuom-GTfouRA@x1.local/
diff --git a/qapi/net.json b/qapi/net.json
index c5d87ba308b..a71dc7f1d4c 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -441,6 +441,10 @@
# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
# MT < 11.1)
#
+# Features:
+#
+# @unstable: Member @incoming-fds is experimental
+#
# Since: 1.2
##
{ 'struct': 'NetdevTapOptions',
@@ -460,7 +464,7 @@
'*vhostforce': 'bool',
'*queues': 'uint32',
'*poll-us': 'uint32',
- '*incoming-fds': 'bool',
+ '*incoming-fds': { 'type': 'bool', 'features' : [ 'unstable' ] },
'*local-migration-supported': 'bool' } }
##
--
Best regards,
Vladimir
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-05-27 19:11 ` [PATCH v17 08/10] net/tap: support local migration with virtio-net Vladimir Sementsov-Ogievskiy
2026-05-28 16:03 ` Vladimir Sementsov-Ogievskiy
@ 2026-06-02 8:02 ` Markus Armbruster
2026-06-02 12:47 ` Vladimir Sementsov-Ogievskiy
1 sibling, 1 reply; 19+ messages in thread
From: Markus Armbruster @ 2026-06-02 8:02 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: jasowang, mst, peterx, farosas, raphael.s.norwitz, bchaney,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake, John Snow
[Cc: John Snow for advice on cross-references (search for "John?")]
Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
> Support transferring of TAP state (including open fd).
>
> Add new property "local-migration-supported", which defines,
> is local-migration is actually supported for this TAP device.
which defines whether
> Starting from 11.1 MT it's enabled by default.
What does "MT" mean?
>
> Note, that local-migration (including migrating opened FDs
> through migration channel, which must be UNIX socket), is
> enabled by global "local" migration parameters. But individual
> devices may have additional options to enable/disable it
> personally.
per device
>
> Add new option "incoming-fds", which should be set to true on
> target for incoming migration work. It says "do not open any
> files, but instead wait for FDs coming from migration stream".
> "local-migration-supported" option is not enough, as it work in pair
> with migration parameter "local", and intialization process
> of TAP device should not depend on migration parameters.
>
> For new option require explicitly unset script and downscript,
> to keep possibility of implementing support for them in the
> future.
Why explicitly unset? Do we have to override a default script?
> Note disabling read polling on source stop for TAP migration:
I don't understand this sentence.
> otherwise, source process may steal packages from TAP fd even
> after source vm STOP.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
[...]
> diff --git a/qapi/net.json b/qapi/net.json
> index 1a6382825c5..c5d87ba308b 100644
> --- a/qapi/net.json
> +++ b/qapi/net.json
> @@ -425,6 +425,22 @@
##
# @NetdevTapOptions:
#
# Used to configure a host TAP network interface backend.
#
# @ifname: interface name
#
# @fd: file descriptor of an already opened tap
#
# @fds: multiple file descriptors of already opened multiqueue capable
# tap
#
# @script: script to initialize the interface
#
# @downscript: script to shut down the interface
#
# @br: bridge name (since 2.8)
#
# @helper: command to execute to configure bridge
#
# @sndbuf: send buffer limit. Understands [TGMKkb] suffixes.
#
# @vnet_hdr: enable the IFF_VNET_HDR flag on the tap interface
#
# @vhost: enable vhost-net network accelerator
#
# @vhostfd: file descriptor of an already opened vhost net device
#
# @vhostfds: file descriptors of multiple already opened vhost net
# devices
#
# @vhostforce: vhost on for non-MSIX virtio guests
#
# @queues: number of queues to be created for multiqueue capable tap
#
> # @poll-us: maximum number of microseconds that could be spent on busy
> # polling for tap (since 2.7)
> #
> +# @incoming-fds: do not open or create any TAP devices. Prepare for
> +# getting TAP file descriptors from incoming migration stream.
> +# The option is incompatible with any of @fd, @fds, @helper, @br,
> +# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
> +# @downscript be explicitly set to nothing (empty string or "no"),
WAT?!?
Special value "no" is not documented in the QAPI schema, see the
descriptions of @script and @downscript above.
It is documented for -netdev tap in qemu-options.hx:
``-netdev tap,id=id[,fd=h][,ifname=name][,script=file][,downscript=dfile][,br=bridge][,helper=helper]``
Configure a host TAP network backend with ID id.
Use the network script file to configure it and the network script
dfile to deconfigure it. If name is not provided, the OS
automatically provides one. The default network configure script is
``/etc/qemu-ifup`` and the default network deconfigure script is
``/etc/qemu-ifdown``. Use ``script=no`` or ``downscript=no`` to
disable script execution.
Special value "" is okay: you can't name have a script named "".
Special value "no" isn't: you can have a script named "no", but if you
try to use it, it's silently ignored. Yes, it's a silly name, but it's
also a silly interface.
I think we should document "no" properly, and also deprecate it.
> +# and requires also @local-migration-supported to be true, "local"
> +# migration parameter be set as well, and QEMU being in incoming
> +# migration state. (Since 11.1)
This sentence is rather long and hard to parse. Here's my attempt:
# The option is incompatible with any of @fd, @fds, @helper, @br,
# @ifname, @sndbuf and @vnet_hdr options. @script and @downscript
# must be explicitly disabled (empty string or "no"), and
# @local-migration-supported must be true. Additionally,
# migration parameter @local must be set, and QEMU mist be in
# incoming migration state. (Since 11.1)
Ideally, "migration parameter @local" would link to its description, but
I can't tell you offhand whether we can do that and how. John?
> +#
> +# @local-migration-supported: enable local migration for this TAP
> +# backend. When set, local migration is enabled/disabled by
> +# "local" migration parameter for this TAP backend. When unset,
migration parameter @local
> +# "local" migration parameter is ignored for this TAP backend.
> +# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
> +# MT < 11.1)
What's "MT"?
> +#
> # Since: 1.2
> ##
> { 'struct': 'NetdevTapOptions',
> @@ -443,7 +459,9 @@
> '*vhostfds': 'str',
> '*vhostforce': 'bool',
> '*queues': 'uint32',
> - '*poll-us': 'uint32'} }
> + '*poll-us': 'uint32',
> + '*incoming-fds': 'bool',
> + '*local-migration-supported': 'bool' } }
>
> ##
> # @NetdevSocketOptions:
Two bools mean four cases. @incoming-fd's description excludes the case
"incoming-fd": true, "local-migration-supported": false. Awkward. Does
case ""incoming-fd": false, "local-migration-supported": true make sense
and why?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-06-02 8:02 ` Markus Armbruster
@ 2026-06-02 12:47 ` Vladimir Sementsov-Ogievskiy
2026-06-03 9:33 ` Markus Armbruster
0 siblings, 1 reply; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-06-02 12:47 UTC (permalink / raw)
To: Markus Armbruster
Cc: jasowang, mst, peterx, farosas, raphael.s.norwitz, bchaney,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake, John Snow
On 02.06.26 11:02, Markus Armbruster wrote:
> [Cc: John Snow for advice on cross-references (search for "John?")]
>
> Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
>
>> Support transferring of TAP state (including open fd).
>>
>> Add new property "local-migration-supported", which defines,
>> is local-migration is actually supported for this TAP device.
>
> which defines whether
>
>> Starting from 11.1 MT it's enabled by default.
>
> What does "MT" mean?
Machine Type I mean
>
>>
>> Note, that local-migration (including migrating opened FDs
>> through migration channel, which must be UNIX socket), is
>> enabled by global "local" migration parameters. But individual
>> devices may have additional options to enable/disable it
>> personally.
>
> per device
>
>>
>> Add new option "incoming-fds", which should be set to true on
>> target for incoming migration work. It says "do not open any
>> files, but instead wait for FDs coming from migration stream".
>> "local-migration-supported" option is not enough, as it work in pair
>> with migration parameter "local", and intialization process
>> of TAP device should not depend on migration parameters.
>>
>> For new option require explicitly unset script and downscript,
>> to keep possibility of implementing support for them in the
>> future.
>
> Why explicitly unset? Do we have to override a default script?
Current version doesn't support having any scripts, as we should
also transfer a responsiblity for calling downscript to the target
(but what if migration failed, transfer it back?).
Still, several version ago Ben asked to keep a possibility to support
scripts in future. So, current version requires script=no and
downscript=no. In future we may implement support for scripts together
with fd-migration, and drop this restriction.
(Hope I answered the question)
>
>> Note disabling read polling on source stop for TAP migration:
>
> I don't understand this sentence.
That's a realization detail. I've added it, because it was a fix, appeared
several version ago. It may be dropped from commit message.
The sentence is about tap_vm_state_change() function, which disables
polling the TAP device.
>
>> otherwise, source process may steal packages from TAP fd even
>> after source vm STOP.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>
> [...]
>
>> diff --git a/qapi/net.json b/qapi/net.json
>> index 1a6382825c5..c5d87ba308b 100644
>> --- a/qapi/net.json
>> +++ b/qapi/net.json
>> @@ -425,6 +425,22 @@
> ##
> # @NetdevTapOptions:
> #
> # Used to configure a host TAP network interface backend.
> #
> # @ifname: interface name
> #
> # @fd: file descriptor of an already opened tap
> #
> # @fds: multiple file descriptors of already opened multiqueue capable
> # tap
> #
> # @script: script to initialize the interface
> #
> # @downscript: script to shut down the interface
> #
> # @br: bridge name (since 2.8)
> #
> # @helper: command to execute to configure bridge
> #
> # @sndbuf: send buffer limit. Understands [TGMKkb] suffixes.
> #
> # @vnet_hdr: enable the IFF_VNET_HDR flag on the tap interface
> #
> # @vhost: enable vhost-net network accelerator
> #
> # @vhostfd: file descriptor of an already opened vhost net device
> #
> # @vhostfds: file descriptors of multiple already opened vhost net
> # devices
> #
> # @vhostforce: vhost on for non-MSIX virtio guests
> #
> # @queues: number of queues to be created for multiqueue capable tap
> #
>> # @poll-us: maximum number of microseconds that could be spent on busy
>> # polling for tap (since 2.7)
>> #
>> +# @incoming-fds: do not open or create any TAP devices. Prepare for
>> +# getting TAP file descriptors from incoming migration stream.
>> +# The option is incompatible with any of @fd, @fds, @helper, @br,
>> +# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
>> +# @downscript be explicitly set to nothing (empty string or "no"),
>
> WAT?!?
>
> Special value "no" is not documented in the QAPI schema, see the
> descriptions of @script and @downscript above.
>
> It is documented for -netdev tap in qemu-options.hx:
>
> ``-netdev tap,id=id[,fd=h][,ifname=name][,script=file][,downscript=dfile][,br=bridge][,helper=helper]``
> Configure a host TAP network backend with ID id.
>
> Use the network script file to configure it and the network script
> dfile to deconfigure it. If name is not provided, the OS
> automatically provides one. The default network configure script is
> ``/etc/qemu-ifup`` and the default network deconfigure script is
> ``/etc/qemu-ifdown``. Use ``script=no`` or ``downscript=no`` to
> disable script execution.
>
> Special value "" is okay: you can't name have a script named "".
>
> Special value "no" isn't: you can have a script named "no", but if you
> try to use it, it's silently ignored. Yes, it's a silly name, but it's
> also a silly interface.
>
> I think we should document "no" properly, and also deprecate it.
I can make a patch, but that's actually not about this series, "no" is
preexisting.
Hmm, may be also deprecate using default scripts, when script options are
not set by user?
This way I can drop script=no & downscript=no requirements. And make
absent script option mean "no scripts" for "incoming-fds" from the start.
(or, I can do it even without deprecation, but that creates an inconsistency
in the common interface)
>
>> +# and requires also @local-migration-supported to be true, "local"
>> +# migration parameter be set as well, and QEMU being in incoming
>> +# migration state. (Since 11.1)
>
> This sentence is rather long and hard to parse. Here's my attempt:
>
> # The option is incompatible with any of @fd, @fds, @helper, @br,
> # @ifname, @sndbuf and @vnet_hdr options. @script and @downscript
> # must be explicitly disabled (empty string or "no"), and
> # @local-migration-supported must be true. Additionally,
> # migration parameter @local must be set, and QEMU mist be in
> # incoming migration state. (Since 11.1)
Sounds good, thanks!
>
> Ideally, "migration parameter @local" would link to its description, but
> I can't tell you offhand whether we can do that and how. John?
>
>> +#
>> +# @local-migration-supported: enable local migration for this TAP
>> +# backend. When set, local migration is enabled/disabled by
>> +# "local" migration parameter for this TAP backend. When unset,
>
> migration parameter @local
>
>> +# "local" migration parameter is ignored for this TAP backend.
>> +# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
>> +# MT < 11.1)
>
> What's "MT"?
Machine Type
>
>> +#
>> # Since: 1.2
>> ##
>> { 'struct': 'NetdevTapOptions',
>> @@ -443,7 +459,9 @@
>> '*vhostfds': 'str',
>> '*vhostforce': 'bool',
>> '*queues': 'uint32',
>> - '*poll-us': 'uint32'} }
>> + '*poll-us': 'uint32',
>> + '*incoming-fds': 'bool',
>> + '*local-migration-supported': 'bool' } }
>>
>> ##
>> # @NetdevSocketOptions:
>
> Two bools mean four cases. @incoming-fd's description excludes the case
> "incoming-fd": true, "local-migration-supported": false. Awkward. Does
> case ""incoming-fd": false, "local-migration-supported": true make sense
> and why?
>
Yes, as local-migration-supported enables the whole feature, and it will
be set by default starting from 11.1 machine type (read, starting from
11.1 QEMU, if not use older machine type).
"local-migration-supported" makes sense both for source and target,
"incoming-fds" only for target.
I describe, whey we can't live with only one boolean here:
https://lore.kernel.org/qemu-devel/a572dadb-6ea6-48f2-a77c-3a4531dbad49@yandex-team.ru/
and it's also a start of discussion for alternative interface (which makes possible to
realy on "local" migration parameter instead of having extra "incoming-fds").
And here further RFC from Peter: https://lore.kernel.org/qemu-devel/20260528212947.368132-1-peterx@redhat.com/
Finally, that's all why "incoming-fds" is unstable (see squash-in in parallel reply)
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-06-02 12:47 ` Vladimir Sementsov-Ogievskiy
@ 2026-06-03 9:33 ` Markus Armbruster
2026-06-03 16:50 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 19+ messages in thread
From: Markus Armbruster @ 2026-06-03 9:33 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: jasowang, mst, peterx, farosas, raphael.s.norwitz, bchaney,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake, John Snow
Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
> On 02.06.26 11:02, Markus Armbruster wrote:
>> [Cc: John Snow for advice on cross-references (search for "John?")]
>>
>> Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
>>
>>> Support transferring of TAP state (including open fd).
>>>
>>> Add new property "local-migration-supported", which defines,
>>> is local-migration is actually supported for this TAP device.
>>
>> which defines whether
>>
>>> Starting from 11.1 MT it's enabled by default.
>>
>> What does "MT" mean?
>
> Machine Type I mean
Got it. Please don't abbreviate it here.
>
>>
>>>
>>> Note, that local-migration (including migrating opened FDs
>>> through migration channel, which must be UNIX socket), is
>>> enabled by global "local" migration parameters. But individual
>>> devices may have additional options to enable/disable it
>>> personally.
>>
>> per device
>>
>>>
>>> Add new option "incoming-fds", which should be set to true on
>>> target for incoming migration work. It says "do not open any
>>> files, but instead wait for FDs coming from migration stream".
>>> "local-migration-supported" option is not enough, as it work in pair
>>> with migration parameter "local", and intialization process
>>> of TAP device should not depend on migration parameters.
>>>
>>> For new option require explicitly unset script and downscript,
>>> to keep possibility of implementing support for them in the
>>> future.
>>
>> Why explicitly unset? Do we have to override a default script?
>
> Current version doesn't support having any scripts, as we should
> also transfer a responsiblity for calling downscript to the target
> (but what if migration failed, transfer it back?).
>
> Still, several version ago Ben asked to keep a possibility to support
> scripts in future. So, current version requires script=no and
> downscript=no. In future we may implement support for scripts together
> with fd-migration, and drop this restriction.
>
> (Hope I answered the question)
I understand we can't run scripts specified with @script and @downscript
when @incoming-fds is enabled. We might be able to run them in the
future.
We obviously should reject user requests for scripts we don't run.
Thus, user specifying scripts together with @incoming-fds should be an
error. I understand it is in your code.
User specifying "no script" is obviously fine then.
Left: defaults. The doc comment doesn't tell what they are. That's a
defect in need of a fix. Peeking at the code... Looks like @script and
@downscript default to qemu-ifup and qemu-ifdown, respectively, both in
CONFIG_SYSCONFDIR. Correct?
If yes, two options:
(1) Require the user to specify "no scripts", overriding the defaults.
Simple semantics. Might be mildly annoying for human users. Management
applications won't care. I understand this is what your code does.
(2) Make the defaults depend on @incoming-fds: none if on, qemu-ifup and
qemu-ifdown if off. Acceptable if documented.
I think I prefer (1).
Please consider working some of this into your commit message.
>>> Note disabling read polling on source stop for TAP migration:
>>
>> I don't understand this sentence.
>
> That's a realization detail. I've added it, because it was a fix, appeared
> several version ago. It may be dropped from commit message.
>
> The sentence is about tap_vm_state_change() function, which disables
> polling the TAP device.
Should it be a separate patch?
>>> otherwise, source process may steal packages from TAP fd even
>>> after source vm STOP.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>>
>> [...]
>>
>>> diff --git a/qapi/net.json b/qapi/net.json
>>> index 1a6382825c5..c5d87ba308b 100644
>>> --- a/qapi/net.json
>>> +++ b/qapi/net.json
>>> @@ -425,6 +425,22 @@
>>> ##
>>> # @NetdevTapOptions:
>>> #
>>> # Used to configure a host TAP network interface backend.
>>> #
>>> # @ifname: interface name
>>> #
>>> # @fd: file descriptor of an already opened tap
>>> #
>>> # @fds: multiple file descriptors of already opened multiqueue capable
>>> # tap
>>> #
>>> # @script: script to initialize the interface
>>> #
>>> # @downscript: script to shut down the interface
>>> #
>>> # @br: bridge name (since 2.8)
>>> #
>>> # @helper: command to execute to configure bridge
>>> #
>>> # @sndbuf: send buffer limit. Understands [TGMKkb] suffixes.
>>> #
>>> # @vnet_hdr: enable the IFF_VNET_HDR flag on the tap interface
>>> #
>>> # @vhost: enable vhost-net network accelerator
>>> #
>>> # @vhostfd: file descriptor of an already opened vhost net device
>>> #
>>> # @vhostfds: file descriptors of multiple already opened vhost net
>>> # devices
>>> #
>>> # @vhostforce: vhost on for non-MSIX virtio guests
>>> #
>>> # @queues: number of queues to be created for multiqueue capable tap
>>> #
>>> # @poll-us: maximum number of microseconds that could be spent on busy
>>> # polling for tap (since 2.7)
>>> #
>>> +# @incoming-fds: do not open or create any TAP devices. Prepare for
>>> +# getting TAP file descriptors from incoming migration stream.
>>> +# The option is incompatible with any of @fd, @fds, @helper, @br,
>>> +# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
>>> +# @downscript be explicitly set to nothing (empty string or "no"),
>>
>> WAT?!?
>>
>> Special value "no" is not documented in the QAPI schema, see the
>> descriptions of @script and @downscript above.
>>
>> It is documented for -netdev tap in qemu-options.hx:
>>
>> ``-netdev tap,id=id[,fd=h][,ifname=name][,script=file][,downscript=dfile][,br=bridge][,helper=helper]``
>> Configure a host TAP network backend with ID id.
>>
>> Use the network script file to configure it and the network script
>> dfile to deconfigure it. If name is not provided, the OS
>> automatically provides one. The default network configure script is
>> ``/etc/qemu-ifup`` and the default network deconfigure script is
>> ``/etc/qemu-ifdown``. Use ``script=no`` or ``downscript=no`` to
>> disable script execution.
>>
>> Special value "" is okay: you can't name have a script named "".
>>
>> Special value "no" isn't: you can have a script named "no", but if you
>> try to use it, it's silently ignored. Yes, it's a silly name, but it's
>> also a silly interface.
>>
>> I think we should document "no" properly, and also deprecate it.
>
> I can make a patch, but that's actually not about this series, "no" is
> preexisting.
Yes, it is. Keeping the deprecation patch separate is fine.
> Hmm, may be also deprecate using default scripts, when script options are
> not set by user?
>
> This way I can drop script=no & downscript=no requirements. And make
> absent script option mean "no scripts" for "incoming-fds" from the start.
> (or, I can do it even without deprecation, but that creates an inconsistency
> in the common interface)
Changing defaults is often awkward.
Changing the meaning of "@script absent" from qemu-ifup to no script is
a compatibility break.
We can deprecate "@script absent". This pushes users to specify
@script, which is probably a good idea for management applications
anyway. It may be annoying for humans.
After the deprecation grace period, we can change the default and
undeprecate. Trap for management applications that somehow missed it
during the grace period (skipped too many QEMU versions?). Not sure
that's how our deprecation process wants to be used.
We could make @script mandatory instead. Avoids the trap.
Same for @downscript.
Thoughts?
>>> +# and requires also @local-migration-supported to be true, "local"
>>> +# migration parameter be set as well, and QEMU being in incoming
>>> +# migration state. (Since 11.1)
>>
>> This sentence is rather long and hard to parse. Here's my attempt:
>>
>> # The option is incompatible with any of @fd, @fds, @helper, @br,
>> # @ifname, @sndbuf and @vnet_hdr options. @script and @downscript
>> # must be explicitly disabled (empty string or "no"), and
>> # @local-migration-supported must be true. Additionally,
>> # migration parameter @local must be set, and QEMU mist be in
>> # incoming migration state. (Since 11.1)
>
> Sounds good, thanks!
>
>>
>> Ideally, "migration parameter @local" would link to its description, but
>> I can't tell you offhand whether we can do that and how. John?
>>
>>> +#
>>> +# @local-migration-supported: enable local migration for this TAP
>>> +# backend. When set, local migration is enabled/disabled by
>>> +# "local" migration parameter for this TAP backend. When unset,
>>
>> migration parameter @local
>>
>>> +# "local" migration parameter is ignored for this TAP backend.
>>> +# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
>>> +# MT < 11.1)
>>
>> What's "MT"?
>
> Machine Type
Got it. Please don't abbreviate it here.
>>> +#
>>> # Since: 1.2
>>> ##
>>> { 'struct': 'NetdevTapOptions',
>>> @@ -443,7 +459,9 @@
>>> '*vhostfds': 'str',
>>> '*vhostforce': 'bool',
>>> '*queues': 'uint32',
>>> - '*poll-us': 'uint32'} }
>>> + '*poll-us': 'uint32',
>>> + '*incoming-fds': 'bool',
>>> + '*local-migration-supported': 'bool' } }
>>>
>>> ##
>>> # @NetdevSocketOptions:
>>
>> Two bools mean four cases. @incoming-fd's description excludes the case
>> "incoming-fd": true, "local-migration-supported": false. Awkward. Does
>> case ""incoming-fd": false, "local-migration-supported": true make sense
>> and why?
>>
>
> Yes, as local-migration-supported enables the whole feature, and it will
> be set by default starting from 11.1 machine type (read, starting from
> 11.1 QEMU, if not use older machine type).
What are the use cases for the three valid combinations?
* "incoming-fd": false, "local-migration-supported": false
* "incoming-fd": false, "local-migration-supported": true
* "incoming-fd": true, "local-migration-supported": true
> "local-migration-supported" makes sense both for source and target,
> "incoming-fds" only for target.
>
> I describe, whey we can't live with only one boolean here:
>
> https://lore.kernel.org/qemu-devel/a572dadb-6ea6-48f2-a77c-3a4531dbad49@yandex-team.ru/
>
> and it's also a start of discussion for alternative interface (which makes possible to
> realy on "local" migration parameter instead of having extra "incoming-fds").
>
> And here further RFC from Peter: https://lore.kernel.org/qemu-devel/20260528212947.368132-1-peterx@redhat.com/
>
> Finally, that's all why "incoming-fds" is unstable (see squash-in in parallel reply)
Should @local-migration-supported also be unstable?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v17 00/10] virtio-net: live-TAP local migration
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
` (9 preceding siblings ...)
2026-05-27 19:11 ` [PATCH v17 10/10] tests/functional: add test_tap_migration Vladimir Sementsov-Ogievskiy
@ 2026-06-03 15:24 ` Mark Cave-Ayland
2026-06-03 16:17 ` Vladimir Sementsov-Ogievskiy
10 siblings, 1 reply; 19+ messages in thread
From: Mark Cave-Ayland @ 2026-06-03 15:24 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, qemu-devel,
berrange, pbonzini, yc-core
On 27/05/2026 20:11, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
>
> Here is a migration for TAP net backend, including its properties and
> open fds.
>
> With this new feature, management software doesn't need to initialize
> new TAP and do a switch to it. Nothing should be done around
> virtio-net in local migration: it just migrates and continues to use
> same TAP device. So we avoid extra logic in management software, extra
> allocations in kernel (for new TAP), and corresponding extra delay in
> migration downtime.
Hi Vladimir,
Thanks for the updated instructions - I was able to successfully test an
FD-based tap device migration using this series, although I had to make
a couple of minor changes below:
> v17: rework interface
> 02-04: new
> 07: - drop local-migration parameter for virtio-net and the whole
> code to migrate backends (they should migrate by themselves).
> - drop virtio_net_update_host_features(), and call
> peer_test_vnet_hdr() and virtio_net_get_features() directly
> from post-load handlers
> - drop r-b
> 08: - add local-migratieon-supported property (instead of
> local-migration that is dropped in 07)
> - register own vmstate
> - add check for being in incoming migration when use incoming-fds
> - drop r-b, a-b
> 10: only move 'local_migration=local' from virtio-net device
> to 'local-migration-supported": local' in netdev. keep r-bs
>
> Based-on: <20260318113144.15697-1-vsementsov@yandex-team.ru>
> "[PATCH v4 00/13] net: refactoring and fixes"
>
> v17 is pushed to
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.com_vsementsov_qemu.git&d=DwIDaQ&c=s883GpUCOChKOHiocYtGcg&r=c23RpsaH4D2MKyD3EPJTDa0BAxz6tV8aUJqVSoytEiY&m=BO9WL9PEcrRM2I2KvXYdAYQ-qD80S8kAodStSmO6ukcWGHfyhmC5oBYZnpRkE2it&s=mKXtt2cIaKN9EjEyFBVWf7-n1xDIbFzA6_5oaEK84HQ&e=
> tag: up-tap-fd-migration-with-bk-opt-v17
>
> To run the test, use sudo, as test needs to configure TAP device:
>
> sudo PYTHONPATH=python:tests/functional \
> QEMU_TEST_QEMU_BINARY=$PWD/build/qemu-system-x86_64 \
> MESON_BUILD_ROOT=$PWD/build \
> ./build/pyvenv/bin/python3 tests/functional/x86_64/test_tap_migration.py
>
> Or, to test the feature by hand, you may follow the instruction:
>
> 1. Prerequisites
> ----------------
>
> QEMU=/path/to/your/build/qemu-system-x86_64
>
> # download same image as in test
>
> wget -O /tmp/alpine.iso "https://urldefense.proofpoint.com/v2/url?u=https-3A__dl-2Dcdn.alpinelinux.org_alpine_v3.22_releases_x86-5F64_alpine-2Dstandard-2D3.22.1-2Dx86-5F64.iso&d=DwIDaQ&c=s883GpUCOChKOHiocYtGcg&r=c23RpsaH4D2MKyD3EPJTDa0BAxz6tV8aUJqVSoytEiY&m=BO9WL9PEcrRM2I2KvXYdAYQ-qD80S8kAodStSmO6ukcWGHfyhmC5oBYZnpRkE2it&s=yF_WcQ_YiVipmpp9Gs3nNSd5lHNU1lOy2GophvYXYDE&e= "
>
>
> # prepare tap device (be careful to not break your own networks)
>
> sudo ip tuntap add dev tap0 mode tap multi_queue
> sudo ip addr add 192.168.100.1/24 dev tap0
> sudo ip link set tap0 up
>
>
> 2. Start source VM
> ------------------
>
> Open terminal A, and run:
>
> $QEMU \
> -name source \
> -machine q35 \
> -accel kvm \
> -m 1G \
> -object memory-backend-file,id=ram0,size=1G,mem-path=/dev/shm/qemu_migration_test,share=on \
> -machine memory-backend=ram0 \
> -drive file=/tmp/alpine.iso,media=cdrom,format=raw \
> -device pcie-pci-bridge,id=pci.1,bus=pcie.0 \
> -netdev tap,id=netdev.1,ifname=tap0,queues=4,vnet_hdr=on,script=no,downscript=no,local-migration-supported=on \
Looks like local-migration-supported is no longer required?
> -device virtio-net-pci,netdev=netdev.1,id=vnet.1,bus=pci.1,mq=on,vectors=18,romfile=,disable-legacy=off \
> -serial stdio \
> -nographic \
> -qmp unix:/tmp/qmp-source.sock,server=on,wait=off
>
> Wait for Alpine to boot. When you see the login prompt, log in as root
> (no password):
>
> localhost login: root
>
> Configure the guest network:
>
> ip addr add 192.168.100.2/24 dev eth0
> ip link set eth0 up
>
> Verify connectivity from the guest:
>
> ping -c 3 192.168.100.1
>
> And from the host (in another terminal):
>
> ping -c 3 192.168.100.2
>
>
> 3. Start target VM
> ------------------
>
> Open terminal B. The target VM uses the same shared memory file and
> the same ISO. The NIC is configured with incoming-fds=on — it will
> not open tap0 itself; instead it will receive the TAP file descriptors
> from the source over the migration channel.
>
> $QEMU \
> -name target \
> -machine q35 \
> -accel kvm \
> -m 1G \
> -object memory-backend-file,id=ram0,size=1G,mem-path=/dev/shm/qemu_migration_test,share=on \
> -machine memory-backend=ram0 \
> -drive file=/tmp/alpine.iso,media=cdrom,format=raw \
> -device pcie-pci-bridge,id=pci.1,bus=pcie.0 \
> -netdev tap,id=netdev.1,queues=4,script=no,downscript=no,local-migration-supported=on,incoming-fds=on \
And also here?
> -device virtio-net-pci,netdev=netdev.1,id=vnet.1,bus=pci.1,mq=on,vectors=18,romfile=,disable-legacy=off \
> -serial stdio \
> -nographic \
> -qmp unix:/tmp/qmp-target.sock,server=on,wait=off \
> -incoming defer
>
>
> 4. Start migration
> ------------------
>
> Open terminal C and connect to the source QMP socket:
>
> socat - UNIX-CONNECT:/tmp/qmp-source.sock
>
> Negotiate capabilities and configure migration:
>
> {"execute": "qmp_capabilities"}
>
> {"execute": "migrate-set-capabilities", "arguments": {
> "capabilities": [
> {"capability": "events", "state": true},
> {"capability": "x-ignore-shared", "state": true}
> ]
> }}
>
> {"execute": "migrate-set-parameters", "arguments": {"local": true}}
>
> Open terminal D and connect to the target QMP socket:
>
> socat - UNIX-CONNECT:/tmp/qmp-target.sock
>
> Negotiate capabilities and configure migration:
>
> {"execute": "qmp_capabilities"}
>
> {"execute": "migrate-set-capabilities", "arguments": {
> "capabilities": [
> {"capability": "events", "state": true},
> {"capability": "x-ignore-shared", "state": true}
> ]
> }}
>
> {"execute": "migrate-set-parameters", "arguments": {"local": true}}
>
> Tell the target to listen for the incoming migration (terminal D):
>
> {"execute": "migrate-incoming",
> "arguments": {"uri": "unix:/tmp/migration.sock"}}
>
> Trigger the migration from the source (terminal C):
>
> {"execute": "migrate",
> "arguments": {"uri": "unix:/tmp/migration.sock"}}
>
> Poll migration status until it completes (terminal C):
>
> {"execute": "query-migrate"}
> # repeat until "status" == "completed"
>
> Or just wait for the MIGRATION event that QEMU emits automatically:
>
> # {"event": "MIGRATION", "data": {"status": "completed"}, ...}
>
> Once the source reports "completed", resume the target VM (terminal D):
>
> {"execute": "cont"}
>
> The target VM is now running with the migrated state and the TAP file
> descriptors that were passed from the source.
>
> Verify that the guest is still reachable from the host:
>
> ping -c 3 192.168.100.2
>
> And from inside the guest (terminal B, target serial console):
>
> ping -c 3 192.168.100.1
>
>
> 5. Cleanup
> ----------
>
> Shut down the target VM (terminal D):
>
> {"execute": "quit"}
>
> Shut down the source VM (terminal C):
>
> {"execute": "quit"}
>
> Remove the TAP device:
>
> sudo ip tuntap del tap0 mode tap multi_queue
>
> Remove the shared memory file:
>
> rm /dev/shm/qemu_migration_test
>
> Remove leftover sockets if they still exist:
>
> rm -f /tmp/migration.sock /tmp/qmp-source.sock /tmp/qmp-target.sock
>
> Vladimir Sementsov-Ogievskiy (10):
> net/tap: move vhost-net open() calls to tap_parse_vhost_fds()
> net/tap: move vhost initialization to tap_setup_vhost()
> net/tap: use container_of instead of DO_UPCAST
> net/tap: QOMify tap backend
> net/tap: add TYPE_VMSTATE_IF interface
> qapi: add local migration parameter
> virtio-net: support local migration of backend
> net/tap: support local migration with virtio-net
> tests/functional: add skipWithoutSudo() decorator
> tests/functional: add test_tap_migration
>
> hw/core/machine.c | 2 +
> hw/net/virtio-net.c | 89 +++-
> include/hw/virtio/virtio-net.h | 1 +
> include/migration/misc.h | 2 +
> include/migration/vmstate.h | 2 +
> include/net/net.h | 9 +
> include/net/tap.h | 2 +
> migration/options.c | 18 +-
> net/net.c | 14 +-
> net/tap.c | 429 +++++++++++++---
> qapi/migration.json | 12 +-
> qapi/net.json | 20 +-
> tests/functional/qemu_test/decorators.py | 16 +
> tests/functional/x86_64/meson.build | 1 +
> tests/functional/x86_64/test_tap_migration.py | 458 ++++++++++++++++++
> 15 files changed, 986 insertions(+), 89 deletions(-)
> create mode 100755 tests/functional/x86_64/test_tap_migration.py
I have a few more thoughts related to the ongoing discussions with Peter
and Fabiano which I will reply to in the relevant threads.
ATB,
Mark.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v17 00/10] virtio-net: live-TAP local migration
2026-06-03 15:24 ` [PATCH v17 00/10] virtio-net: live-TAP local migration Mark Cave-Ayland
@ 2026-06-03 16:17 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-06-03 16:17 UTC (permalink / raw)
To: Mark Cave-Ayland, jasowang, mst
Cc: armbru, peterx, farosas, raphael.s.norwitz, bchaney, qemu-devel,
berrange, pbonzini, yc-core
On 03.06.26 18:24, Mark Cave-Ayland wrote:
>> Open terminal A, and run:
>>
>> $QEMU \
>> -name source \
>> -machine q35 \
>> -accel kvm \
>> -m 1G \
>> -object memory-backend-file,id=ram0,size=1G,mem-path=/dev/shm/qemu_migration_test,share=on \
>> -machine memory-backend=ram0 \
>> -drive file=/tmp/alpine.iso,media=cdrom,format=raw \
>> -device pcie-pci-bridge,id=pci.1,bus=pcie.0 \
>> -netdev tap,id=netdev.1,ifname=tap0,queues=4,vnet_hdr=on,script=no,downscript=no,local-migration-supported=on \
>
> Looks like local-migration-supported is no longer required?
Yes in new Machine Type. So, specifying -machine q35, with current (actually not yet finished/published 11.1) qemu make local-migration-supperted to be true by default.
If you specify older machine tipe, like -machine pc-q35-11.0, local-migration-supported will be false by default.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-06-03 9:33 ` Markus Armbruster
@ 2026-06-03 16:50 ` Vladimir Sementsov-Ogievskiy
2026-06-08 8:05 ` Markus Armbruster
0 siblings, 1 reply; 19+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2026-06-03 16:50 UTC (permalink / raw)
To: Markus Armbruster
Cc: jasowang, mst, peterx, farosas, raphael.s.norwitz, bchaney,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake, John Snow
On 03.06.26 12:33, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
>
>> On 02.06.26 11:02, Markus Armbruster wrote:
>>> [Cc: John Snow for advice on cross-references (search for "John?")]
>>>
>>> Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
>>>
>>>> Support transferring of TAP state (including open fd).
>>>>
>>>> Add new property "local-migration-supported", which defines,
>>>> is local-migration is actually supported for this TAP device.
>>>
>>> which defines whether
>>>
>>>> Starting from 11.1 MT it's enabled by default.
>>>
>>> What does "MT" mean?
>>
>> Machine Type I mean
>
> Got it. Please don't abbreviate it here.
>
>>
>>>
>>>>
>>>> Note, that local-migration (including migrating opened FDs
>>>> through migration channel, which must be UNIX socket), is
>>>> enabled by global "local" migration parameters. But individual
>>>> devices may have additional options to enable/disable it
>>>> personally.
>>>
>>> per device
>>>
>>>>
>>>> Add new option "incoming-fds", which should be set to true on
>>>> target for incoming migration work. It says "do not open any
>>>> files, but instead wait for FDs coming from migration stream".
>>>> "local-migration-supported" option is not enough, as it work in pair
>>>> with migration parameter "local", and intialization process
>>>> of TAP device should not depend on migration parameters.
>>>>
>>>> For new option require explicitly unset script and downscript,
>>>> to keep possibility of implementing support for them in the
>>>> future.
>>>
>>> Why explicitly unset? Do we have to override a default script?
>>
>> Current version doesn't support having any scripts, as we should
>> also transfer a responsiblity for calling downscript to the target
>> (but what if migration failed, transfer it back?).
>>
>> Still, several version ago Ben asked to keep a possibility to support
>> scripts in future. So, current version requires script=no and
>> downscript=no. In future we may implement support for scripts together
>> with fd-migration, and drop this restriction.
>>
>> (Hope I answered the question)
>
> I understand we can't run scripts specified with @script and @downscript
> when @incoming-fds is enabled. We might be able to run them in the
> future.
>
> We obviously should reject user requests for scripts we don't run.
> Thus, user specifying scripts together with @incoming-fds should be an
> error. I understand it is in your code.
>
> User specifying "no script" is obviously fine then.
>
> Left: defaults. The doc comment doesn't tell what they are. That's a
> defect in need of a fix. Peeking at the code... Looks like @script and
> @downscript default to qemu-ifup and qemu-ifdown, respectively, both in
> CONFIG_SYSCONFDIR. Correct?
>
> If yes, two options:
>
> (1) Require the user to specify "no scripts", overriding the defaults.
> Simple semantics. Might be mildly annoying for human users. Management
> applications won't care. I understand this is what your code does.
>
> (2) Make the defaults depend on @incoming-fds: none if on, qemu-ifup and
> qemu-ifdown if off. Acceptable if documented.
This seems inconsistent. If we are not going to deprecate current defaults,
no reason to create different ones.
>
> I think I prefer (1).
That's what I do in the series
>
> Please consider working some of this into your commit message.
Ok. I think, actually adding a preparing patch, documenting current behavior
will make things a lot more clear.
>
>>>> Note disabling read polling on source stop for TAP migration:
>>>
>>> I don't understand this sentence.
>>
>> That's a realization detail. I've added it, because it was a fix, appeared
>> several version ago. It may be dropped from commit message.
>>
>> The sentence is about tap_vm_state_change() function, which disables
>> polling the TAP device.
>
> Should it be a separate patch?
Agree, that seems a good idea. Will try to split.
>
>>>> otherwise, source process may steal packages from TAP fd even
>>>> after source vm STOP.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>>>
>>> [...]
>>>
>>>> diff --git a/qapi/net.json b/qapi/net.json
>>>> index 1a6382825c5..c5d87ba308b 100644
>>>> --- a/qapi/net.json
>>>> +++ b/qapi/net.json
>>>> @@ -425,6 +425,22 @@
>>>> ##
>>>> # @NetdevTapOptions:
>>>> #
>>>> # Used to configure a host TAP network interface backend.
>>>> #
>>>> # @ifname: interface name
>>>> #
>>>> # @fd: file descriptor of an already opened tap
>>>> #
>>>> # @fds: multiple file descriptors of already opened multiqueue capable
>>>> # tap
>>>> #
>>>> # @script: script to initialize the interface
>>>> #
>>>> # @downscript: script to shut down the interface
>>>> #
>>>> # @br: bridge name (since 2.8)
>>>> #
>>>> # @helper: command to execute to configure bridge
>>>> #
>>>> # @sndbuf: send buffer limit. Understands [TGMKkb] suffixes.
>>>> #
>>>> # @vnet_hdr: enable the IFF_VNET_HDR flag on the tap interface
>>>> #
>>>> # @vhost: enable vhost-net network accelerator
>>>> #
>>>> # @vhostfd: file descriptor of an already opened vhost net device
>>>> #
>>>> # @vhostfds: file descriptors of multiple already opened vhost net
>>>> # devices
>>>> #
>>>> # @vhostforce: vhost on for non-MSIX virtio guests
>>>> #
>>>> # @queues: number of queues to be created for multiqueue capable tap
>>>> #
>>>> # @poll-us: maximum number of microseconds that could be spent on busy
>>>> # polling for tap (since 2.7)
>>>> #
>>>> +# @incoming-fds: do not open or create any TAP devices. Prepare for
>>>> +# getting TAP file descriptors from incoming migration stream.
>>>> +# The option is incompatible with any of @fd, @fds, @helper, @br,
>>>> +# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
>>>> +# @downscript be explicitly set to nothing (empty string or "no"),
>>>
>>> WAT?!?
>>>
>>> Special value "no" is not documented in the QAPI schema, see the
>>> descriptions of @script and @downscript above.
>>>
>>> It is documented for -netdev tap in qemu-options.hx:
>>>
>>> ``-netdev tap,id=id[,fd=h][,ifname=name][,script=file][,downscript=dfile][,br=bridge][,helper=helper]``
>>> Configure a host TAP network backend with ID id.
>>>
>>> Use the network script file to configure it and the network script
>>> dfile to deconfigure it. If name is not provided, the OS
>>> automatically provides one. The default network configure script is
>>> ``/etc/qemu-ifup`` and the default network deconfigure script is
>>> ``/etc/qemu-ifdown``. Use ``script=no`` or ``downscript=no`` to
>>> disable script execution.
>>>
>>> Special value "" is okay: you can't name have a script named "".
>>>
>>> Special value "no" isn't: you can have a script named "no", but if you
>>> try to use it, it's silently ignored. Yes, it's a silly name, but it's
>>> also a silly interface.
>>>
>>> I think we should document "no" properly, and also deprecate it.
>>
>> I can make a patch, but that's actually not about this series, "no" is
>> preexisting.
>
> Yes, it is. Keeping the deprecation patch separate is fine.
>
>> Hmm, may be also deprecate using default scripts, when script options are
>> not set by user?
>>
>> This way I can drop script=no & downscript=no requirements. And make
>> absent script option mean "no scripts" for "incoming-fds" from the start.
>> (or, I can do it even without deprecation, but that creates an inconsistency
>> in the common interface)
>
> Changing defaults is often awkward.
>
> Changing the meaning of "@script absent" from qemu-ifup to no script is
> a compatibility break.
>
> We can deprecate "@script absent". This pushes users to specify
> @script, which is probably a good idea for management applications
> anyway. It may be annoying for humans.
>
> After the deprecation grace period, we can change the default and
> undeprecate. Trap for management applications that somehow missed it
> during the grace period (skipped too many QEMU versions?). Not sure
> that's how our deprecation process wants to be used.
I'm afraid, skipping too many QEMU versions is existing practice
for some downstream vendors :(
>
> We could make @script mandatory instead. Avoids the trap.
>
> Same for @downscript.
>
> Thoughts?
I don't think that making "unused" options mandatory is much
better than having unobvious defaults. So, we'll get yet
another "not ideal" interface, and additional headache for
those who used this default.
I think, I'd keep things as is, just add documentation. And probably
deprecate "no" in favor of "".
>
>>>> +# and requires also @local-migration-supported to be true, "local"
>>>> +# migration parameter be set as well, and QEMU being in incoming
>>>> +# migration state. (Since 11.1)
>>>
>>> This sentence is rather long and hard to parse. Here's my attempt:
>>>
>>> # The option is incompatible with any of @fd, @fds, @helper, @br,
>>> # @ifname, @sndbuf and @vnet_hdr options. @script and @downscript
>>> # must be explicitly disabled (empty string or "no"), and
>>> # @local-migration-supported must be true. Additionally,
>>> # migration parameter @local must be set, and QEMU mist be in
>>> # incoming migration state. (Since 11.1)
>>
>> Sounds good, thanks!
>>
>>>
>>> Ideally, "migration parameter @local" would link to its description, but
>>> I can't tell you offhand whether we can do that and how. John?
>>>
>>>> +#
>>>> +# @local-migration-supported: enable local migration for this TAP
>>>> +# backend. When set, local migration is enabled/disabled by
>>>> +# "local" migration parameter for this TAP backend. When unset,
>>>
>>> migration parameter @local
>>>
>>>> +# "local" migration parameter is ignored for this TAP backend.
>>>> +# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
>>>> +# MT < 11.1)
>>>
>>> What's "MT"?
>>
>> Machine Type
>
> Got it. Please don't abbreviate it here.
>
>>>> +#
>>>> # Since: 1.2
>>>> ##
>>>> { 'struct': 'NetdevTapOptions',
>>>> @@ -443,7 +459,9 @@
>>>> '*vhostfds': 'str',
>>>> '*vhostforce': 'bool',
>>>> '*queues': 'uint32',
>>>> - '*poll-us': 'uint32'} }
>>>> + '*poll-us': 'uint32',
>>>> + '*incoming-fds': 'bool',
>>>> + '*local-migration-supported': 'bool' } }
>>>>
>>>> ##
>>>> # @NetdevSocketOptions:
>>>
>>> Two bools mean four cases. @incoming-fd's description excludes the case
>>> "incoming-fd": true, "local-migration-supported": false. Awkward. Does
>>> case ""incoming-fd": false, "local-migration-supported": true make sense
>>> and why?
>>>
>>
>> Yes, as local-migration-supported enables the whole feature, and it will
>> be set by default starting from 11.1 machine type (read, starting from
>> 11.1 QEMU, if not use older machine type).
>
> What are the use cases for the three valid combinations?
>
> * "incoming-fd": false, "local-migration-supported": false
Old Machine Type, the feature is disabled.
>
> * "incoming-fd": false, "local-migration-supported": true
New Machine type, the feature is enabled generally. But for current incoming
migration it's disabled, for example because it's a remote migration (so, we can't
migrate FDs through UNIX socket, and we don't set "local" migration parameter
of course).
Or, it's not an incoming migration, but a "first cold start" of the VM,
of course "incoming-fd" is false, to open real files/sockets, but we
are prepared for future local migrations (live-updates) with new feature
enabled.
>
> * "incoming-fd": true, "local-migration-supported": true
New Machine type, the feature is enabled generally. And it is enabled for
current incoming migration. "local" migration parameter should be set too, and
UNIX socket should be used for migration channel.
>
>> "local-migration-supported" makes sense both for source and target,
>> "incoming-fds" only for target.
>>
>> I describe, whey we can't live with only one boolean here:
>>
>> https://lore.kernel.org/qemu-devel/a572dadb-6ea6-48f2-a77c-3a4531dbad49@yandex-team.ru/
>>
>> and it's also a start of discussion for alternative interface (which makes possible to
>> realy on "local" migration parameter instead of having extra "incoming-fds").
>>
>> And here further RFC from Peter: https://lore.kernel.org/qemu-devel/20260528212947.368132-1-peterx@redhat.com/
>>
>> Finally, that's all why "incoming-fds" is unstable (see squash-in in parallel reply)
>
> Should @local-migration-supported also be unstable?
>
"incoming-fds" in unstable, as it may be removed in favor of local-migration-suppoerted && local,
if "local" becomes available earlier as suggested by Peter's RFC.
local-migration-supported will stay as is, even if RFC applied.
Still, I agree, if part of new (quite complicated) interface is unstable, the whole
interface is risky. No problem to keep all new options "unstable" for a while.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v17 08/10] net/tap: support local migration with virtio-net
2026-06-03 16:50 ` Vladimir Sementsov-Ogievskiy
@ 2026-06-08 8:05 ` Markus Armbruster
0 siblings, 0 replies; 19+ messages in thread
From: Markus Armbruster @ 2026-06-08 8:05 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: jasowang, mst, peterx, farosas, raphael.s.norwitz, bchaney,
qemu-devel, berrange, pbonzini, yc-core, mark.caveayland,
Philippe Mathieu-Daudé, Zhao Liu, Eric Blake, John Snow
Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
> On 03.06.26 12:33, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
>>
>>> On 02.06.26 11:02, Markus Armbruster wrote:
>>>> [Cc: John Snow for advice on cross-references (search for "John?")]
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> writes:
>>>>
>>>>> Support transferring of TAP state (including open fd).
>>>>>
>>>>> Add new property "local-migration-supported", which defines,
>>>>> is local-migration is actually supported for this TAP device.
>>>>
>>>> which defines whether
>>>>
>>>>> Starting from 11.1 MT it's enabled by default.
>>>>
>>>> What does "MT" mean?
>>>
>>> Machine Type I mean
>>
>> Got it. Please don't abbreviate it here.
>>
>>>
>>>>
>>>>>
>>>>> Note, that local-migration (including migrating opened FDs
>>>>> through migration channel, which must be UNIX socket), is
>>>>> enabled by global "local" migration parameters. But individual
>>>>> devices may have additional options to enable/disable it
>>>>> personally.
>>>>
>>>> per device
>>>>
>>>>>
>>>>> Add new option "incoming-fds", which should be set to true on
>>>>> target for incoming migration work. It says "do not open any
>>>>> files, but instead wait for FDs coming from migration stream".
>>>>> "local-migration-supported" option is not enough, as it work in pair
>>>>> with migration parameter "local", and intialization process
>>>>> of TAP device should not depend on migration parameters.
>>>>>
>>>>> For new option require explicitly unset script and downscript,
>>>>> to keep possibility of implementing support for them in the
>>>>> future.
>>>>
>>>> Why explicitly unset? Do we have to override a default script?
>>>
>>> Current version doesn't support having any scripts, as we should
>>> also transfer a responsiblity for calling downscript to the target
>>> (but what if migration failed, transfer it back?).
>>>
>>> Still, several version ago Ben asked to keep a possibility to support
>>> scripts in future. So, current version requires script=no and
>>> downscript=no. In future we may implement support for scripts together
>>> with fd-migration, and drop this restriction.
>>>
>>> (Hope I answered the question)
>>
>> I understand we can't run scripts specified with @script and @downscript
>> when @incoming-fds is enabled. We might be able to run them in the
>> future.
>>
>> We obviously should reject user requests for scripts we don't run.
>> Thus, user specifying scripts together with @incoming-fds should be an
>> error. I understand it is in your code.
>>
>> User specifying "no script" is obviously fine then.
>>
>> Left: defaults. The doc comment doesn't tell what they are. That's a
>> defect in need of a fix. Peeking at the code... Looks like @script and
>> @downscript default to qemu-ifup and qemu-ifdown, respectively, both in
>> CONFIG_SYSCONFDIR. Correct?
>>
>> If yes, two options:
>>
>> (1) Require the user to specify "no scripts", overriding the defaults.
>> Simple semantics. Might be mildly annoying for human users. Management
>> applications won't care. I understand this is what your code does.
>>
>> (2) Make the defaults depend on @incoming-fds: none if on, qemu-ifup and
>> qemu-ifdown if off. Acceptable if documented.
>
> This seems inconsistent. If we are not going to deprecate current defaults,
> no reason to create different ones.
>
>>
>> I think I prefer (1).
>
> That's what I do in the series
>
>>
>> Please consider working some of this into your commit message.
>
> Ok. I think, actually adding a preparing patch, documenting current behavior
> will make things a lot more clear.
That would be lovely!
>>>>> Note disabling read polling on source stop for TAP migration:
>>>>
>>>> I don't understand this sentence.
>>>
>>> That's a realization detail. I've added it, because it was a fix, appeared
>>> several version ago. It may be dropped from commit message.
>>>
>>> The sentence is about tap_vm_state_change() function, which disables
>>> polling the TAP device.
>>
>> Should it be a separate patch?
>
> Agree, that seems a good idea. Will try to split.
>
>>
>>>>> otherwise, source process may steal packages from TAP fd even
>>>>> after source vm STOP.
>>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>>>>
>>>> [...]
>>>>
>>>>> diff --git a/qapi/net.json b/qapi/net.json
>>>>> index 1a6382825c5..c5d87ba308b 100644
>>>>> --- a/qapi/net.json
>>>>> +++ b/qapi/net.json
>>>>> @@ -425,6 +425,22 @@
>>>>> ##
>>>>> # @NetdevTapOptions:
>>>>> #
>>>>> # Used to configure a host TAP network interface backend.
>>>>> #
>>>>> # @ifname: interface name
>>>>> #
>>>>> # @fd: file descriptor of an already opened tap
>>>>> #
>>>>> # @fds: multiple file descriptors of already opened multiqueue capable
>>>>> # tap
>>>>> #
>>>>> # @script: script to initialize the interface
>>>>> #
>>>>> # @downscript: script to shut down the interface
>>>>> #
>>>>> # @br: bridge name (since 2.8)
>>>>> #
>>>>> # @helper: command to execute to configure bridge
>>>>> #
>>>>> # @sndbuf: send buffer limit. Understands [TGMKkb] suffixes.
>>>>> #
>>>>> # @vnet_hdr: enable the IFF_VNET_HDR flag on the tap interface
>>>>> #
>>>>> # @vhost: enable vhost-net network accelerator
>>>>> #
>>>>> # @vhostfd: file descriptor of an already opened vhost net device
>>>>> #
>>>>> # @vhostfds: file descriptors of multiple already opened vhost net
>>>>> # devices
>>>>> #
>>>>> # @vhostforce: vhost on for non-MSIX virtio guests
>>>>> #
>>>>> # @queues: number of queues to be created for multiqueue capable tap
>>>>> #
>>>>> # @poll-us: maximum number of microseconds that could be spent on busy
>>>>> # polling for tap (since 2.7)
>>>>> #
>>>>> +# @incoming-fds: do not open or create any TAP devices. Prepare for
>>>>> +# getting TAP file descriptors from incoming migration stream.
>>>>> +# The option is incompatible with any of @fd, @fds, @helper, @br,
>>>>> +# @ifname, @sndbuf and @vnet_hdr options, and requires @script and
>>>>> +# @downscript be explicitly set to nothing (empty string or "no"),
>>>>
>>>> WAT?!?
>>>>
>>>> Special value "no" is not documented in the QAPI schema, see the
>>>> descriptions of @script and @downscript above.
>>>>
>>>> It is documented for -netdev tap in qemu-options.hx:
>>>>
>>>> ``-netdev tap,id=id[,fd=h][,ifname=name][,script=file][,downscript=dfile][,br=bridge][,helper=helper]``
>>>> Configure a host TAP network backend with ID id.
>>>>
>>>> Use the network script file to configure it and the network script
>>>> dfile to deconfigure it. If name is not provided, the OS
>>>> automatically provides one. The default network configure script is
>>>> ``/etc/qemu-ifup`` and the default network deconfigure script is
>>>> ``/etc/qemu-ifdown``. Use ``script=no`` or ``downscript=no`` to
>>>> disable script execution.
>>>>
>>>> Special value "" is okay: you can't name have a script named "".
>>>>
>>>> Special value "no" isn't: you can have a script named "no", but if you
>>>> try to use it, it's silently ignored. Yes, it's a silly name, but it's
>>>> also a silly interface.
>>>>
>>>> I think we should document "no" properly, and also deprecate it.
>>>
>>> I can make a patch, but that's actually not about this series, "no" is
>>> preexisting.
>>
>> Yes, it is. Keeping the deprecation patch separate is fine.
>>
>>> Hmm, may be also deprecate using default scripts, when script options are
>>> not set by user?
>>>
>>> This way I can drop script=no & downscript=no requirements. And make
>>> absent script option mean "no scripts" for "incoming-fds" from the start.
>>> (or, I can do it even without deprecation, but that creates an inconsistency
>>> in the common interface)
>>
>> Changing defaults is often awkward.
>>
>> Changing the meaning of "@script absent" from qemu-ifup to no script is
>> a compatibility break.
>>
>> We can deprecate "@script absent". This pushes users to specify
>> @script, which is probably a good idea for management applications
>> anyway. It may be annoying for humans.
>>
>> After the deprecation grace period, we can change the default and
>> undeprecate. Trap for management applications that somehow missed it
>> during the grace period (skipped too many QEMU versions?). Not sure
>> that's how our deprecation process wants to be used.
>
> I'm afraid, skipping too many QEMU versions is existing practice
> for some downstream vendors :(
We want to be nice to our downstreams. Even when they do things we
advise against like skipping a bunch of upstream versions without
closely examining how external interfaces evolved. But at some point
"nice" can become too expensive for us. Judgment call.
>> We could make @script mandatory instead. Avoids the trap.
>>
>> Same for @downscript.
>>
>> Thoughts?
>
> I don't think that making "unused" options mandatory is much
> better than having unobvious defaults. So, we'll get yet
> another "not ideal" interface, and additional headache for
> those who used this default.
>
>
> I think, I'd keep things as is, just add documentation. And probably
> deprecate "no" in favor of "".
Works for me.
As I said, changing defaults is awkward.
>>>>> +# and requires also @local-migration-supported to be true, "local"
>>>>> +# migration parameter be set as well, and QEMU being in incoming
>>>>> +# migration state. (Since 11.1)
>>>>
>>>> This sentence is rather long and hard to parse. Here's my attempt:
>>>>
>>>> # The option is incompatible with any of @fd, @fds, @helper, @br,
>>>> # @ifname, @sndbuf and @vnet_hdr options. @script and @downscript
>>>> # must be explicitly disabled (empty string or "no"), and
>>>> # @local-migration-supported must be true. Additionally,
>>>> # migration parameter @local must be set, and QEMU mist be in
>>>> # incoming migration state. (Since 11.1)
>>>
>>> Sounds good, thanks!
>>>
>>>>
>>>> Ideally, "migration parameter @local" would link to its description, but
>>>> I can't tell you offhand whether we can do that and how. John?
>>>>
>>>>> +#
>>>>> +# @local-migration-supported: enable local migration for this TAP
>>>>> +# backend. When set, local migration is enabled/disabled by
>>>>> +# "local" migration parameter for this TAP backend. When unset,
>>>>
>>>> migration parameter @local
>>>>
>>>>> +# "local" migration parameter is ignored for this TAP backend.
>>>>> +# (Since 11.1. Defaults to true for MT >= 11.1, and to false for
>>>>> +# MT < 11.1)
>>>>
>>>> What's "MT"?
>>>
>>> Machine Type
>>
>> Got it. Please don't abbreviate it here.
>>
>>>>> +#
>>>>> # Since: 1.2
>>>>> ##
>>>>> { 'struct': 'NetdevTapOptions',
>>>>> @@ -443,7 +459,9 @@
>>>>> '*vhostfds': 'str',
>>>>> '*vhostforce': 'bool',
>>>>> '*queues': 'uint32',
>>>>> - '*poll-us': 'uint32'} }
>>>>> + '*poll-us': 'uint32',
>>>>> + '*incoming-fds': 'bool',
>>>>> + '*local-migration-supported': 'bool' } }
>>>>>
>>>>> ##
>>>>> # @NetdevSocketOptions:
>>>>
>>>> Two bools mean four cases. @incoming-fd's description excludes the case
>>>> "incoming-fd": true, "local-migration-supported": false. Awkward. Does
>>>> case ""incoming-fd": false, "local-migration-supported": true make sense
>>>> and why?
>>>>
>>>
>>> Yes, as local-migration-supported enables the whole feature, and it will
>>> be set by default starting from 11.1 machine type (read, starting from
>>> 11.1 QEMU, if not use older machine type).
>>
>> What are the use cases for the three valid combinations?
>>
>> * "incoming-fd": false, "local-migration-supported": false
>
> Old Machine Type, the feature is disabled.
>
>>
>> * "incoming-fd": false, "local-migration-supported": true
>
> New Machine type, the feature is enabled generally. But for current incoming
> migration it's disabled, for example because it's a remote migration (so, we can't
> migrate FDs through UNIX socket, and we don't set "local" migration parameter
> of course).
>
> Or, it's not an incoming migration, but a "first cold start" of the VM,
> of course "incoming-fd" is false, to open real files/sockets, but we
> are prepared for future local migrations (live-updates) with new feature
> enabled.
>
>>
>> * "incoming-fd": true, "local-migration-supported": true
>
> New Machine type, the feature is enabled generally. And it is enabled for
> current incoming migration. "local" migration parameter should be set too, and
> UNIX socket should be used for migration channel.
Could this be worked into the docs somehow?
Would be easy if we had an enum of three values instead of two bools...
>>> "local-migration-supported" makes sense both for source and target,
>>> "incoming-fds" only for target.
>>>
>>> I describe, whey we can't live with only one boolean here:
>>>
>>> https://lore.kernel.org/qemu-devel/a572dadb-6ea6-48f2-a77c-3a4531dbad49@yandex-team.ru/
>>>
>>> and it's also a start of discussion for alternative interface (which makes possible to
>>> realy on "local" migration parameter instead of having extra "incoming-fds").
>>>
>>> And here further RFC from Peter: https://lore.kernel.org/qemu-devel/20260528212947.368132-1-peterx@redhat.com/
>>>
>>> Finally, that's all why "incoming-fds" is unstable (see squash-in in parallel reply)
>>
>> Should @local-migration-supported also be unstable?
>>
>
> "incoming-fds" in unstable, as it may be removed in favor of local-migration-suppoerted && local,
> if "local" becomes available earlier as suggested by Peter's RFC.
>
> local-migration-supported will stay as is, even if RFC applied.
>
> Still, I agree, if part of new (quite complicated) interface is unstable, the whole
> interface is risky. No problem to keep all new options "unstable" for a while.
Let's do that, nothing to lose.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-06-08 10:40 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-27 19:11 [PATCH v17 00/10] virtio-net: live-TAP local migration Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 01/10] net/tap: move vhost-net open() calls to tap_parse_vhost_fds() Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 02/10] net/tap: move vhost initialization to tap_setup_vhost() Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 03/10] net/tap: use container_of instead of DO_UPCAST Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 04/10] net/tap: QOMify tap backend Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 05/10] net/tap: add TYPE_VMSTATE_IF interface Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 06/10] qapi: add local migration parameter Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 07/10] virtio-net: support local migration of backend Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 08/10] net/tap: support local migration with virtio-net Vladimir Sementsov-Ogievskiy
2026-05-28 16:03 ` Vladimir Sementsov-Ogievskiy
2026-06-02 8:02 ` Markus Armbruster
2026-06-02 12:47 ` Vladimir Sementsov-Ogievskiy
2026-06-03 9:33 ` Markus Armbruster
2026-06-03 16:50 ` Vladimir Sementsov-Ogievskiy
2026-06-08 8:05 ` Markus Armbruster
2026-05-27 19:11 ` [PATCH v17 09/10] tests/functional: add skipWithoutSudo() decorator Vladimir Sementsov-Ogievskiy
2026-05-27 19:11 ` [PATCH v17 10/10] tests/functional: add test_tap_migration Vladimir Sementsov-Ogievskiy
2026-06-03 15:24 ` [PATCH v17 00/10] virtio-net: live-TAP local migration Mark Cave-Ayland
2026-06-03 16:17 ` Vladimir Sementsov-Ogievskiy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.