* [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel
@ 2025-07-18 8:52 Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 01/13] net: bundle all offloads in a single struct Paolo Abeni
` (12 more replies)
0 siblings, 13 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Some virtualized deployments use UDP tunnel pervasively and are impacted
negatively by the lack of GSO support for such kind of traffic in the
virtual NIC driver.
The virtio_net specification recently introduced support for GSO over
UDP tunnel, and the kernel side of the implementation has been merged
into the net-next tree; this series updates the virtio implementation to
support such a feature.
Currently the qemu virtio support limits the features space to 64 bits,
while the virtio specification allows for a larger number of features.
Specifically the GSO-over-UDP-tunnel-related virtio features use bits
65-69; the larger part of this series (patches 3-11) actually deals with
extending the features space.
The extended features are carried by fixed size uint64_t arrays,
bringing the current maximum features number to 128.
The patches use some syntactic sugar to try to minimize the otherwise
very large code churn. Specifically the extended features are boundled
in an union with 'legacy' features definition, allowing no changes in
the virtio devices not needing the extended features set.
The actual offload implementation is in patches 12 and 13 and boils down
to propagating the new offload to the tun devices and the vhost backend.
Finally patch 1 is a small pre-req refactor that ideally could enter the
tree separately; it's presented here in the same series to help
reviewers more easily getting the full picture and patch 2 is a needed
linux headers update.
Tested with basic stream transfer with all the possible permutations of
host kernel/qemu/guest kernel with/without GSO over UDP tunnel support,
vs snapshots creation and restore and vs migration.
Sharing again as RFC as the kernel bits have not entered the Linus tree
yet - but they should on next merge window.
---
v2 -> v3:
- consolidated suffixes for new fields to '_ex'
- avoid pre/post load trickery and relay on reset zeroing the features
- cleaned-up virtio store implementation deduplicating a bit of code
- many more cleanups, see the individual patches changelog for the
details
- I left patch 1 unmodified, still some hope we could live with that;)
Paolo Abeni (13):
net: bundle all offloads in a single struct
linux-headers: Update to Linux ~v6.16-rc5 net-next
virtio: introduce extended features type
virtio: serialize extended features state
virtio: add support for negotiating extended features
virtio-pci: implement support for extended features
vhost: add support for negotiating extended features
qmp: update virtio features map to support extended features
vhost-backend: implement extended features support
vhost-net: implement extended features support
virtio-net: implement extended features support
net: implement tunnel probing
net: implement UDP tunnel features offloading
hw/net/e1000e_core.c | 5 +-
hw/net/igb_core.c | 5 +-
hw/net/vhost_net-stub.c | 8 +-
hw/net/vhost_net.c | 46 ++--
hw/net/virtio-net.c | 226 +++++++++++++------
hw/net/vmxnet3.c | 13 +-
hw/virtio/vhost-backend.c | 62 ++++-
hw/virtio/vhost.c | 73 +++++-
hw/virtio/virtio-bus.c | 11 +-
hw/virtio/virtio-hmp-cmds.c | 3 +-
hw/virtio/virtio-pci.c | 54 ++++-
hw/virtio/virtio-qmp.c | 89 +++++---
hw/virtio/virtio-qmp.h | 3 +-
hw/virtio/virtio.c | 102 ++++++---
include/hw/virtio/vhost-backend.h | 6 +
include/hw/virtio/vhost.h | 33 ++-
include/hw/virtio/virtio-features.h | 123 ++++++++++
include/hw/virtio/virtio-net.h | 2 +-
include/hw/virtio/virtio-pci.h | 2 +-
include/hw/virtio/virtio.h | 11 +-
include/net/net.h | 20 +-
include/net/vhost_net.h | 33 ++-
include/standard-headers/linux/ethtool.h | 4 +-
include/standard-headers/linux/vhost_types.h | 5 +
include/standard-headers/linux/virtio_net.h | 33 +++
linux-headers/linux/vhost.h | 7 +
net/net.c | 17 +-
net/netmap.c | 3 +-
net/tap-bsd.c | 8 +-
net/tap-linux.c | 38 +++-
net/tap-linux.h | 9 +
net/tap-solaris.c | 9 +-
net/tap-stub.c | 8 +-
net/tap.c | 21 +-
net/tap_int.h | 5 +-
qapi/virtio.json | 8 +-
36 files changed, 857 insertions(+), 248 deletions(-)
create mode 100644 include/hw/virtio/virtio-features.h
--
2.50.0
^ permalink raw reply [flat|nested] 54+ messages in thread
* [PATCH RFC v3 01/13] net: bundle all offloads in a single struct
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-20 10:31 ` Akihiko Odaki
2025-07-18 8:52 ` [PATCH RFC v3 02/13] linux-headers: Update to Linux ~v6.16-rc5 net-next Paolo Abeni
` (11 subsequent siblings)
12 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
The set_offload() argument list is already pretty long and
we are going to introduce soon a bunch of additional offloads.
Replace the offload arguments with a single struct and update
all the relevant call-sites.
No functional changes intended.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
Note: I maintained the struct usage as opposed to uint64_t bitmask usage
as suggested by Akihiko, because the latter feel a bit more invasive.
v1 -> v2:
- drop unneeded 'struct' keywords
- moved to series start
---
hw/net/e1000e_core.c | 5 +++--
hw/net/igb_core.c | 5 +++--
hw/net/virtio-net.c | 19 +++++++++++--------
hw/net/vmxnet3.c | 13 +++++--------
include/net/net.h | 15 ++++++++++++---
net/net.c | 5 ++---
net/netmap.c | 3 +--
net/tap-bsd.c | 3 +--
net/tap-linux.c | 21 ++++++++++++---------
net/tap-solaris.c | 4 ++--
net/tap-stub.c | 3 +--
net/tap.c | 8 ++++----
net/tap_int.h | 4 ++--
13 files changed, 59 insertions(+), 49 deletions(-)
diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c
index 2413858790..27599a0dc2 100644
--- a/hw/net/e1000e_core.c
+++ b/hw/net/e1000e_core.c
@@ -2827,8 +2827,9 @@ e1000e_update_rx_offloads(E1000ECore *core)
trace_e1000e_rx_set_cso(cso_state);
if (core->has_vnet) {
- qemu_set_offload(qemu_get_queue(core->owner_nic)->peer,
- cso_state, 0, 0, 0, 0, 0, 0);
+ NetOffloads ol = {.csum = cso_state };
+
+ qemu_set_offload(qemu_get_queue(core->owner_nic)->peer, &ol);
}
}
diff --git a/hw/net/igb_core.c b/hw/net/igb_core.c
index 39e3ce1c8f..45d8fd795b 100644
--- a/hw/net/igb_core.c
+++ b/hw/net/igb_core.c
@@ -3058,8 +3058,9 @@ igb_update_rx_offloads(IGBCore *core)
trace_e1000e_rx_set_cso(cso_state);
if (core->has_vnet) {
- qemu_set_offload(qemu_get_queue(core->owner_nic)->peer,
- cso_state, 0, 0, 0, 0, 0, 0);
+ NetOffloads ol = {.csum = cso_state };
+
+ qemu_set_offload(qemu_get_queue(core->owner_nic)->peer, &ol);
}
}
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index c4c49b0f9c..8953c329e7 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -773,14 +773,17 @@ static uint64_t virtio_net_bad_features(VirtIODevice *vdev)
static void virtio_net_apply_guest_offloads(VirtIONet *n)
{
- qemu_set_offload(qemu_get_queue(n->nic)->peer,
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_CSUM)),
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO4)),
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO6)),
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_ECN)),
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
- !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)));
+ NetOffloads ol = {
+ .csum = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_CSUM)),
+ .tso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO4)),
+ .tso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO6)),
+ .ecn = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_ECN)),
+ .ufo = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
+ .uso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
+ .uso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)),
+ };
+
+ qemu_set_offload(qemu_get_queue(n->nic)->peer, &ol);
}
static uint64_t virtio_net_guest_offloads_by_features(uint64_t features)
diff --git a/hw/net/vmxnet3.c b/hw/net/vmxnet3.c
index af73aa8ef2..03732375a7 100644
--- a/hw/net/vmxnet3.c
+++ b/hw/net/vmxnet3.c
@@ -1322,14 +1322,11 @@ static void vmxnet3_update_features(VMXNET3State *s)
s->lro_supported, rxcso_supported,
s->rx_vlan_stripping);
if (s->peer_has_vhdr) {
- qemu_set_offload(qemu_get_queue(s->nic)->peer,
- rxcso_supported,
- s->lro_supported,
- s->lro_supported,
- 0,
- 0,
- 0,
- 0);
+ NetOffloads ol = { .csum = rxcso_supported,
+ .tso4 = s->lro_supported,
+ .tso6 = s->lro_supported };
+
+ qemu_set_offload(qemu_get_queue(s->nic)->peer, &ol);
}
}
diff --git a/include/net/net.h b/include/net/net.h
index 84ee18e0f9..48ba333d02 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -35,6 +35,16 @@ typedef struct NICConf {
int32_t bootindex;
} NICConf;
+typedef struct NetOffloads {
+ bool csum;
+ bool tso4;
+ bool tso6;
+ bool ecn;
+ bool ufo;
+ bool uso4;
+ bool uso6;
+} NetOffloads;
+
#define DEFINE_NIC_PROPERTIES(_state, _conf) \
DEFINE_PROP_MACADDR("mac", _state, _conf.macaddr), \
DEFINE_PROP_NETDEV("netdev", _state, _conf.peers)
@@ -57,7 +67,7 @@ typedef bool (HasUfo)(NetClientState *);
typedef bool (HasUso)(NetClientState *);
typedef bool (HasVnetHdr)(NetClientState *);
typedef bool (HasVnetHdrLen)(NetClientState *, int);
-typedef void (SetOffload)(NetClientState *, int, int, int, int, int, int, int);
+typedef void (SetOffload)(NetClientState *, const NetOffloads *);
typedef int (GetVnetHdrLen)(NetClientState *);
typedef void (SetVnetHdrLen)(NetClientState *, int);
typedef bool (GetVnetHashSupportedTypes)(NetClientState *, uint32_t *);
@@ -189,8 +199,7 @@ bool qemu_has_ufo(NetClientState *nc);
bool qemu_has_uso(NetClientState *nc);
bool qemu_has_vnet_hdr(NetClientState *nc);
bool qemu_has_vnet_hdr_len(NetClientState *nc, int len);
-void qemu_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
- int ecn, int ufo, int uso4, int uso6);
+void qemu_set_offload(NetClientState *nc, const NetOffloads *ol);
int qemu_get_vnet_hdr_len(NetClientState *nc);
void qemu_set_vnet_hdr_len(NetClientState *nc, int len);
bool qemu_get_vnet_hash_supported_types(NetClientState *nc, uint32_t *types);
diff --git a/net/net.c b/net/net.c
index da275db86e..63872b6855 100644
--- a/net/net.c
+++ b/net/net.c
@@ -540,14 +540,13 @@ bool qemu_has_vnet_hdr_len(NetClientState *nc, int len)
return nc->info->has_vnet_hdr_len(nc, len);
}
-void qemu_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
- int ecn, int ufo, int uso4, int uso6)
+void qemu_set_offload(NetClientState *nc, const NetOffloads *ol)
{
if (!nc || !nc->info->set_offload) {
return;
}
- nc->info->set_offload(nc, csum, tso4, tso6, ecn, ufo, uso4, uso6);
+ nc->info->set_offload(nc, ol);
}
int qemu_get_vnet_hdr_len(NetClientState *nc)
diff --git a/net/netmap.c b/net/netmap.c
index 297510e190..6cd8f2bdc5 100644
--- a/net/netmap.c
+++ b/net/netmap.c
@@ -366,8 +366,7 @@ static void netmap_set_vnet_hdr_len(NetClientState *nc, int len)
}
}
-static void netmap_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
- int ecn, int ufo, int uso4, int uso6)
+static void netmap_set_offload(NetClientState *nc, const NetOffloads *ol)
{
NetmapState *s = DO_UPCAST(NetmapState, nc, nc);
diff --git a/net/tap-bsd.c b/net/tap-bsd.c
index b4c84441ba..86b6edee94 100644
--- a/net/tap-bsd.c
+++ b/net/tap-bsd.c
@@ -231,8 +231,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
return -EINVAL;
}
-void tap_fd_set_offload(int fd, int csum, int tso4,
- int tso6, int ecn, int ufo, int uso4, int uso6)
+void tap_fd_set_offload(int fd, const NetOffloads *ol)
{
}
diff --git a/net/tap-linux.c b/net/tap-linux.c
index 22ec2f45d2..a1c58f74f5 100644
--- a/net/tap-linux.c
+++ b/net/tap-linux.c
@@ -239,8 +239,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
abort();
}
-void tap_fd_set_offload(int fd, int csum, int tso4,
- int tso6, int ecn, int ufo, int uso4, int uso6)
+void tap_fd_set_offload(int fd, const NetOffloads *ol)
{
unsigned int offload = 0;
@@ -249,20 +248,24 @@ void tap_fd_set_offload(int fd, int csum, int tso4,
return;
}
- if (csum) {
+ if (ol->csum) {
offload |= TUN_F_CSUM;
- if (tso4)
+ if (ol->tso4) {
offload |= TUN_F_TSO4;
- if (tso6)
+ }
+ if (ol->tso6) {
offload |= TUN_F_TSO6;
- if ((tso4 || tso6) && ecn)
+ }
+ if ((ol->tso4 || ol->tso6) && ol->ecn) {
offload |= TUN_F_TSO_ECN;
- if (ufo)
+ }
+ if (ol->ufo) {
offload |= TUN_F_UFO;
- if (uso4) {
+ }
+ if (ol->uso4) {
offload |= TUN_F_USO4;
}
- if (uso6) {
+ if (ol->uso6) {
offload |= TUN_F_USO6;
}
}
diff --git a/net/tap-solaris.c b/net/tap-solaris.c
index 51b7830bef..833c066bee 100644
--- a/net/tap-solaris.c
+++ b/net/tap-solaris.c
@@ -27,6 +27,7 @@
#include "tap_int.h"
#include "qemu/ctype.h"
#include "qemu/cutils.h"
+#include "net/net.h"
#include <sys/ethernet.h>
#include <sys/sockio.h>
@@ -235,8 +236,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
return -EINVAL;
}
-void tap_fd_set_offload(int fd, int csum, int tso4,
- int tso6, int ecn, int ufo, int uso4, int uso6)
+void tap_fd_set_offload(int fd, const NetOffloads *ol)
{
}
diff --git a/net/tap-stub.c b/net/tap-stub.c
index 38673434cb..67d14ad4d5 100644
--- a/net/tap-stub.c
+++ b/net/tap-stub.c
@@ -66,8 +66,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
return -EINVAL;
}
-void tap_fd_set_offload(int fd, int csum, int tso4,
- int tso6, int ecn, int ufo, int uso4, int uso6)
+void tap_fd_set_offload(int fd, const NetOffloads *ol)
{
}
diff --git a/net/tap.c b/net/tap.c
index 23536c09b4..b49db19f83 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -280,15 +280,14 @@ static int tap_set_vnet_be(NetClientState *nc, bool is_be)
return tap_fd_set_vnet_be(s->fd, is_be);
}
-static void tap_set_offload(NetClientState *nc, int csum, int tso4,
- int tso6, int ecn, int ufo, int uso4, int uso6)
+static void tap_set_offload(NetClientState *nc, const NetOffloads *ol)
{
TAPState *s = DO_UPCAST(TAPState, nc, nc);
if (s->fd < 0) {
return;
}
- tap_fd_set_offload(s->fd, csum, tso4, tso6, ecn, ufo, uso4, uso6);
+ tap_fd_set_offload(s->fd, ol);
}
static void tap_exit_notify(Notifier *notifier, void *data)
@@ -386,6 +385,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
int fd,
int vnet_hdr)
{
+ NetOffloads ol = {};
NetClientState *nc;
TAPState *s;
@@ -399,7 +399,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
s->has_ufo = tap_probe_has_ufo(s->fd);
s->has_uso = tap_probe_has_uso(s->fd);
s->enabled = true;
- tap_set_offload(&s->nc, 0, 0, 0, 0, 0, 0, 0);
+ tap_set_offload(&s->nc, &ol);
/*
* Make sure host header length is set correctly in tap:
* it might have been modified by another instance of qemu.
diff --git a/net/tap_int.h b/net/tap_int.h
index 8857ff299d..f8bbe1cb0c 100644
--- a/net/tap_int.h
+++ b/net/tap_int.h
@@ -27,6 +27,7 @@
#define NET_TAP_INT_H
#include "qapi/qapi-types-net.h"
+#include "net/net.h"
int tap_open(char *ifname, int ifname_size, int *vnet_hdr,
int vnet_hdr_required, int mq_required, Error **errp);
@@ -37,8 +38,7 @@ void tap_set_sndbuf(int fd, const NetdevTapOptions *tap, Error **errp);
int tap_probe_vnet_hdr(int fd, Error **errp);
int tap_probe_has_ufo(int fd);
int tap_probe_has_uso(int fd);
-void tap_fd_set_offload(int fd, int csum, int tso4, int tso6, int ecn, int ufo,
- int uso4, int uso6);
+void tap_fd_set_offload(int fd, const NetOffloads *ol);
void tap_fd_set_vnet_hdr_len(int fd, int len);
int tap_fd_set_vnet_le(int fd, int vnet_is_le);
int tap_fd_set_vnet_be(int fd, int vnet_is_be);
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 02/13] linux-headers: Update to Linux ~v6.16-rc5 net-next
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 01/13] net: bundle all offloads in a single struct Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 03/13] virtio: introduce extended features type Paolo Abeni
` (10 subsequent siblings)
12 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Update headers to include the virtio GSO over UDP tunnel features
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
--
Note: while the relevant change are not into Linus's tree yet, they have
been merged in the net-next tree and they should land into the vanilla
tree during the next merge window.
---
include/standard-headers/linux/ethtool.h | 4 +--
include/standard-headers/linux/vhost_types.h | 5 +++
include/standard-headers/linux/virtio_net.h | 33 ++++++++++++++++++++
| 7 +++++
4 files changed, 47 insertions(+), 2 deletions(-)
diff --git a/include/standard-headers/linux/ethtool.h b/include/standard-headers/linux/ethtool.h
index cef0d207a6..eb80314028 100644
--- a/include/standard-headers/linux/ethtool.h
+++ b/include/standard-headers/linux/ethtool.h
@@ -2314,7 +2314,7 @@ enum {
IPV6_USER_FLOW = 0x0e, /* spec only (usr_ip6_spec; nfc only) */
IPV4_FLOW = 0x10, /* hash only */
IPV6_FLOW = 0x11, /* hash only */
- ETHER_FLOW = 0x12, /* spec only (ether_spec) */
+ ETHER_FLOW = 0x12, /* hash or spec (ether_spec) */
/* Used for GTP-U IPv4 and IPv6.
* The format of GTP packets only includes
@@ -2371,7 +2371,7 @@ enum {
/* Flag to enable RSS spreading of traffic matching rule (nfc only) */
#define FLOW_RSS 0x20000000
-/* L3-L4 network traffic flow hash options */
+/* L2-L4 network traffic flow hash options */
#define RXH_L2DA (1 << 1)
#define RXH_VLAN (1 << 2)
#define RXH_L3_PROTO (1 << 3)
diff --git a/include/standard-headers/linux/vhost_types.h b/include/standard-headers/linux/vhost_types.h
index fd54044936..4a3aacc39e 100644
--- a/include/standard-headers/linux/vhost_types.h
+++ b/include/standard-headers/linux/vhost_types.h
@@ -110,6 +110,11 @@ struct vhost_msg_v2 {
};
};
+struct vhost_features_array {
+ uint64_t count; /* number of entries present in features array */
+ uint64_t features[] __counted_by(count);
+};
+
struct vhost_memory_region {
uint64_t guest_phys_addr;
uint64_t memory_size; /* bytes */
diff --git a/include/standard-headers/linux/virtio_net.h b/include/standard-headers/linux/virtio_net.h
index 982e854f14..93abaae0b9 100644
--- a/include/standard-headers/linux/virtio_net.h
+++ b/include/standard-headers/linux/virtio_net.h
@@ -70,6 +70,28 @@
* with the same MAC.
*/
#define VIRTIO_NET_F_SPEED_DUPLEX 63 /* Device set linkspeed and duplex */
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO 65 /* Driver can receive
+ * GSO-over-UDP-tunnel packets
+ */
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM 66 /* Driver handles
+ * GSO-over-UDP-tunnel
+ * packets with partial csum
+ * for the outer header
+ */
+#define VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO 67 /* Device can receive
+ * GSO-over-UDP-tunnel packets
+ */
+#define VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM 68 /* Device handles
+ * GSO-over-UDP-tunnel
+ * packets with partial csum
+ * for the outer header
+ */
+
+/* Offloads bits corresponding to VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO{,_CSUM}
+ * features
+ */
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED 46
+#define VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED 47
#ifndef VIRTIO_NET_NO_LEGACY
#define VIRTIO_NET_F_GSO 6 /* Host handles pkts w/ any GSO type */
@@ -131,12 +153,17 @@ struct virtio_net_hdr_v1 {
#define VIRTIO_NET_HDR_F_NEEDS_CSUM 1 /* Use csum_start, csum_offset */
#define VIRTIO_NET_HDR_F_DATA_VALID 2 /* Csum is valid */
#define VIRTIO_NET_HDR_F_RSC_INFO 4 /* rsc info in csum_ fields */
+#define VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM 8 /* UDP tunnel csum offload */
uint8_t flags;
#define VIRTIO_NET_HDR_GSO_NONE 0 /* Not a GSO frame */
#define VIRTIO_NET_HDR_GSO_TCPV4 1 /* GSO frame, IPv4 TCP (TSO) */
#define VIRTIO_NET_HDR_GSO_UDP 3 /* GSO frame, IPv4 UDP (UFO) */
#define VIRTIO_NET_HDR_GSO_TCPV6 4 /* GSO frame, IPv6 TCP */
#define VIRTIO_NET_HDR_GSO_UDP_L4 5 /* GSO frame, IPv4& IPv6 UDP (USO) */
+#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 0x20 /* UDPv4 tunnel present */
+#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6 0x40 /* UDPv6 tunnel present */
+#define VIRTIO_NET_HDR_GSO_UDP_TUNNEL (VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 | \
+ VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6)
#define VIRTIO_NET_HDR_GSO_ECN 0x80 /* TCP has ECN set */
uint8_t gso_type;
__virtio16 hdr_len; /* Ethernet + IP + tcp/udp hdrs */
@@ -181,6 +208,12 @@ struct virtio_net_hdr_v1_hash {
uint16_t padding;
};
+struct virtio_net_hdr_v1_hash_tunnel {
+ struct virtio_net_hdr_v1_hash hash_hdr;
+ uint16_t outer_th_offset;
+ uint16_t inner_nh_offset;
+};
+
#ifndef VIRTIO_NET_NO_LEGACY
/* This header comes first in the scatter-gather list.
* For legacy virtio, if VIRTIO_F_ANY_LAYOUT is not negotiated, it must
--git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h
index d4b3e2ae13..d6ad01fbb8 100644
--- a/linux-headers/linux/vhost.h
+++ b/linux-headers/linux/vhost.h
@@ -235,4 +235,11 @@
*/
#define VHOST_VDPA_GET_VRING_SIZE _IOWR(VHOST_VIRTIO, 0x82, \
struct vhost_vring_state)
+
+/* Extended features manipulation */
+#define VHOST_GET_FEATURES_ARRAY _IOR(VHOST_VIRTIO, 0x83, \
+ struct vhost_features_array)
+#define VHOST_SET_FEATURES_ARRAY _IOW(VHOST_VIRTIO, 0x83, \
+ struct vhost_features_array)
+
#endif
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 03/13] virtio: introduce extended features type
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 01/13] net: bundle all offloads in a single struct Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 02/13] linux-headers: Update to Linux ~v6.16-rc5 net-next Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-20 10:41 ` Akihiko Odaki
2025-07-18 8:52 ` [PATCH RFC v3 04/13] virtio: serialize extended features state Paolo Abeni
` (9 subsequent siblings)
12 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
The virtio specifications allows for up to 128 bits for the
device features. Soon we are going to use some of the 'extended'
bits features (bit 64 and above) for the virtio net driver.
Represent the virtio features bitmask with a fixes size array, and
introduce a few helpers to help manipulate them.
Most drivers will keep using only 64 bits features space: use union
to allow them access the lower part of the extended space without any
per driver change.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- fix preprocessor guard name
- use BIT_ULL
- add missing parentheses
- use memcmp()
- _is_empty() -> _empty()
- _andnot() returns a bool, true if dst has any non zero bits
- _array -> _ex
v1 -> v2:
- use a fixed size array for features instead of uint128
- use union with u64 to reduce the needed code churn
---
include/hw/virtio/virtio-features.h | 123 ++++++++++++++++++++++++++++
include/hw/virtio/virtio.h | 7 +-
2 files changed, 127 insertions(+), 3 deletions(-)
create mode 100644 include/hw/virtio/virtio-features.h
diff --git a/include/hw/virtio/virtio-features.h b/include/hw/virtio/virtio-features.h
new file mode 100644
index 0000000000..68e326e3e8
--- /dev/null
+++ b/include/hw/virtio/virtio-features.h
@@ -0,0 +1,123 @@
+/*
+ * Virtio features helpers
+ *
+ * Copyright 2025 Red Hat, Inc.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef QEMU_VIRTIO_FEATURES_H
+#define QEMU_VIRTIO_FEATURES_H
+
+#include "qemu/bitops.h"
+
+#define VIRTIO_FEATURES_FMT "%016"PRIx64"%016"PRIx64
+#define VIRTIO_FEATURES_PR(f) (f)[1], (f)[0]
+
+#define VIRTIO_FEATURES_MAX 128
+#define VIRTIO_BIT(b) BIT_ULL((b) % 64)
+#define VIRTIO_DWORD(b) ((b) >> 6)
+#define VIRTIO_FEATURES_WORDS (VIRTIO_FEATURES_MAX >> 5)
+#define VIRTIO_FEATURES_DWORDS (VIRTIO_FEATURES_WORDS >> 1)
+
+#define VIRTIO_DECLARE_FEATURES(name) \
+ union { \
+ uint64_t name; \
+ uint64_t name##_ex[VIRTIO_FEATURES_DWORDS]; \
+ }
+
+static inline void virtio_features_clear(uint64_t *features)
+{
+ memset(features, 0, sizeof(features[0]) * VIRTIO_FEATURES_DWORDS);
+}
+
+static inline void virtio_features_from_u64(uint64_t *features, uint64_t from)
+{
+ virtio_features_clear(features);
+ features[0] = from;
+}
+
+static inline bool virtio_has_feature_ex(const uint64_t *features,
+ unsigned int fbit)
+{
+ assert(fbit < VIRTIO_FEATURES_MAX);
+ return features[VIRTIO_DWORD(fbit)] & VIRTIO_BIT(fbit);
+}
+
+static inline void virtio_add_feature_ex(uint64_t *features,
+ unsigned int fbit)
+{
+ assert(fbit < VIRTIO_FEATURES_MAX);
+ features[VIRTIO_DWORD(fbit)] |= VIRTIO_BIT(fbit);
+}
+
+static inline void virtio_clear_feature_ex(uint64_t *features,
+ unsigned int fbit)
+{
+ assert(fbit < VIRTIO_FEATURES_MAX);
+ features[VIRTIO_DWORD(fbit)] &= ~VIRTIO_BIT(fbit);
+}
+
+static inline bool virtio_features_equal(const uint64_t *f1,
+ const uint64_t *f2)
+{
+ return !memcmp(f1, f2, sizeof(uint64_t) * VIRTIO_FEATURES_DWORDS);
+}
+
+static inline bool virtio_features_use_extended(const uint64_t *features)
+{
+ int i;
+
+ for (i = 1; i < VIRTIO_FEATURES_DWORDS; ++i) {
+ if (features[i]) {
+ return true;
+ }
+ }
+ return false;
+}
+
+static inline bool virtio_features_empty(const uint64_t *features)
+{
+ return !virtio_features_use_extended(features) && !features[0];
+}
+
+static inline void virtio_features_copy(uint64_t *to, const uint64_t *from)
+{
+ memcpy(to, from, sizeof(to[0]) * VIRTIO_FEATURES_DWORDS);
+}
+
+static inline bool virtio_features_andnot(uint64_t *to, const uint64_t *f1,
+ const uint64_t *f2)
+{
+ uint64_t diff = 0;
+ int i;
+
+ for (i = 0; i < VIRTIO_FEATURES_DWORDS; i++) {
+ to[i] = f1[i] & ~f2[i];
+ diff |= to[i];
+ }
+ return diff;
+}
+
+static inline void virtio_features_and(uint64_t *to, const uint64_t *f1,
+ const uint64_t *f2)
+{
+ int i;
+
+ for (i = 0; i < VIRTIO_FEATURES_DWORDS; i++) {
+ to[i] = f1[i] & f2[i];
+ }
+}
+
+static inline void virtio_features_or(uint64_t *to, const uint64_t *f1,
+ const uint64_t *f2)
+{
+ int i;
+
+ for (i = 0; i < VIRTIO_FEATURES_DWORDS; i++) {
+ to[i] = f1[i] | f2[i];
+ }
+}
+
+#endif
+
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index c594764f23..39e4059a66 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -16,6 +16,7 @@
#include "system/memory.h"
#include "hw/qdev-core.h"
+#include "hw/virtio/virtio-features.h"
#include "net/net.h"
#include "migration/vmstate.h"
#include "qemu/event_notifier.h"
@@ -121,9 +122,9 @@ struct VirtIODevice
* backend (e.g. vhost) and could potentially be a subset of the
* total feature set offered by QEMU.
*/
- uint64_t host_features;
- uint64_t guest_features;
- uint64_t backend_features;
+ VIRTIO_DECLARE_FEATURES(host_features);
+ VIRTIO_DECLARE_FEATURES(guest_features);
+ VIRTIO_DECLARE_FEATURES(backend_features);
size_t config_len;
void *config;
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 04/13] virtio: serialize extended features state
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (2 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 03/13] virtio: introduce extended features type Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 15:06 ` Stefano Garzarella
2025-07-20 10:44 ` Akihiko Odaki
2025-07-18 8:52 ` [PATCH RFC v3 05/13] virtio: add support for negotiating extended features Paolo Abeni
` (8 subsequent siblings)
12 siblings, 2 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
If the driver uses any of the extended features (i.e. 64 or above),
store the extended features range (64-127 bits).
At load time, let legacy features initialize the full features range
and pass it to the set helper; sub-states loading will have filled-up
the extended part as needed.
This is one of the few spots that need explicitly to know and set
in stone the extended features array size; add a build bug to prevent
breaking the migration should such size change again in the future:
more serialization plumbing will be needed.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- 128bit_features state load/stores only the high bits
- consolidate the load implementation to use a single set
helper for 128/64/32 bits features
- _array -> _ex
v1 -> v2:
- uint128_t -> u64[2]
---
hw/virtio/virtio.c | 88 ++++++++++++++++++++++++++++++----------------
1 file changed, 57 insertions(+), 31 deletions(-)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 2ab1d20769..2817d3a893 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2954,6 +2954,24 @@ static const VMStateDescription vmstate_virtio_disabled = {
}
};
+static bool virtio_128bit_features_needed(void *opaque)
+{
+ VirtIODevice *vdev = opaque;
+
+ return virtio_features_use_extended(vdev->host_features_ex);
+}
+
+static const VMStateDescription vmstate_virtio_128bit_features = {
+ .name = "virtio/128bit_features",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = &virtio_128bit_features_needed,
+ .fields = (const VMStateField[]) {
+ VMSTATE_UINT64(guest_features_ex[1], VirtIODevice),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
static const VMStateDescription vmstate_virtio = {
.name = "virtio",
.version_id = 1,
@@ -2963,6 +2981,7 @@ static const VMStateDescription vmstate_virtio = {
},
.subsections = (const VMStateDescription * const []) {
&vmstate_virtio_device_endian,
+ &vmstate_virtio_128bit_features,
&vmstate_virtio_64bit_features,
&vmstate_virtio_virtqueues,
&vmstate_virtio_ringsize,
@@ -3059,23 +3078,28 @@ const VMStateInfo virtio_vmstate_info = {
.put = virtio_device_put,
};
-static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val)
+static int virtio_set_features_nocheck(VirtIODevice *vdev, const uint64_t *val)
{
VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
- bool bad = (val & ~(vdev->host_features)) != 0;
+ uint64_t tmp[VIRTIO_FEATURES_DWORDS];
+ bool bad;
+
+ bad = virtio_features_andnot(tmp, val, vdev->host_features_ex);
+ virtio_features_and(tmp, val, vdev->host_features_ex);
- val &= vdev->host_features;
if (k->set_features) {
- k->set_features(vdev, val);
+ bad = bad || virtio_features_use_extended(tmp);
+ k->set_features(vdev, tmp[0]);
}
- vdev->guest_features = val;
+
+ virtio_features_copy(vdev->guest_features_ex, tmp);
return bad ? -1 : 0;
}
typedef struct VirtioSetFeaturesNocheckData {
Coroutine *co;
VirtIODevice *vdev;
- uint64_t val;
+ uint64_t val[VIRTIO_FEATURES_DWORDS];
int ret;
} VirtioSetFeaturesNocheckData;
@@ -3088,14 +3112,15 @@ static void virtio_set_features_nocheck_bh(void *opaque)
}
static int coroutine_mixed_fn
-virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev, uint64_t val)
+virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev,
+ const uint64_t *val)
{
if (qemu_in_coroutine()) {
VirtioSetFeaturesNocheckData data = {
.co = qemu_coroutine_self(),
.vdev = vdev,
- .val = val,
};
+ virtio_features_copy(data.val, val);
aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
virtio_set_features_nocheck_bh, &data);
qemu_coroutine_yield();
@@ -3107,6 +3132,7 @@ virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev, uint64_t val)
int virtio_set_features(VirtIODevice *vdev, uint64_t val)
{
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
int ret;
/*
* The driver must not attempt to set features after feature negotiation
@@ -3122,7 +3148,8 @@ int virtio_set_features(VirtIODevice *vdev, uint64_t val)
__func__, vdev->name);
}
- ret = virtio_set_features_nocheck(vdev, val);
+ virtio_features_from_u64(features, val);
+ ret = virtio_set_features_nocheck(vdev, features);
if (virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) {
/* VIRTIO_RING_F_EVENT_IDX changes the size of the caches. */
int i;
@@ -3145,6 +3172,7 @@ void virtio_reset(void *opaque)
{
VirtIODevice *vdev = opaque;
VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
int i;
virtio_set_status(vdev, 0);
@@ -3171,7 +3199,8 @@ void virtio_reset(void *opaque)
vdev->start_on_kick = false;
vdev->started = false;
vdev->broken = false;
- virtio_set_features_nocheck(vdev, 0);
+ virtio_features_clear(features);
+ virtio_set_features_nocheck(vdev, features);
vdev->queue_sel = 0;
vdev->status = 0;
vdev->disabled = false;
@@ -3254,7 +3283,7 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
* Note: devices should always test host features in future - don't create
* new dependencies like this.
*/
- vdev->guest_features = features;
+ virtio_features_from_u64(vdev->guest_features_ex, features);
config_len = qemu_get_be32(f);
@@ -3333,26 +3362,23 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
vdev->device_endian = virtio_default_endian();
}
- if (virtio_64bit_features_needed(vdev)) {
- /*
- * Subsection load filled vdev->guest_features. Run them
- * through virtio_set_features to sanity-check them against
- * host_features.
- */
- uint64_t features64 = vdev->guest_features;
- if (virtio_set_features_nocheck_maybe_co(vdev, features64) < 0) {
- error_report("Features 0x%" PRIx64 " unsupported. "
- "Allowed features: 0x%" PRIx64,
- features64, vdev->host_features);
- return -1;
- }
- } else {
- if (virtio_set_features_nocheck_maybe_co(vdev, features) < 0) {
- error_report("Features 0x%x unsupported. "
- "Allowed features: 0x%" PRIx64,
- features, vdev->host_features);
- return -1;
- }
+ /*
+ * Avoid silently breaking migration should the feature space increase
+ * even more in the (far away) future
+ */
+ QEMU_BUILD_BUG_ON(VIRTIO_FEATURES_DWORDS != 2);
+
+ /*
+ * guest_features_ex is fully initialized with u32 features and upper
+ * bits have been filled as needed by the later load.
+ */
+ if (virtio_set_features_nocheck_maybe_co(vdev,
+ vdev->guest_features_ex) < 0) {
+ error_report("Features 0x" VIRTIO_FEATURES_FMT " unsupported. "
+ "Allowed features: 0x" VIRTIO_FEATURES_FMT,
+ VIRTIO_FEATURES_PR(vdev->guest_features_ex),
+ VIRTIO_FEATURES_PR(vdev->host_features_ex));
+ return -1;
}
if (!virtio_device_started(vdev, vdev->status) &&
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 05/13] virtio: add support for negotiating extended features
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (3 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 04/13] virtio: serialize extended features state Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 06/13] virtio-pci: implement support for " Paolo Abeni
` (7 subsequent siblings)
12 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
The virtio specifications allows for up to 128 bits for the
device features. Soon we are going to use some of the 'extended'
bits features for the virtio net driver.
Add support to allow extended features negotiation on a per
devices basis. Devices willing to negotiated extended features
need to implemented a new pair of features getter/setter, the
core will conditionally use them instead of the basic one.
Note that 'bad_features' don't need to be extended, as they are
bound to the 64 bits limit.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- _array -> _ex
v1 -> v2:
- uint128_t -> uint64_t[]
---
hw/virtio/virtio-bus.c | 11 ++++++++---
hw/virtio/virtio.c | 18 +++++++++++++-----
include/hw/virtio/virtio.h | 4 ++++
3 files changed, 25 insertions(+), 8 deletions(-)
diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index 11adfbf3ab..cef944e015 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -62,9 +62,14 @@ void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp)
}
/* Get the features of the plugged device. */
- assert(vdc->get_features != NULL);
- vdev->host_features = vdc->get_features(vdev, vdev->host_features,
- &local_err);
+ if (vdc->get_features_ex) {
+ vdc->get_features_ex(vdev, vdev->host_features_ex, &local_err);
+ } else {
+ assert(vdc->get_features != NULL);
+ virtio_features_from_u64(vdev->host_features_ex,
+ vdc->get_features(vdev, vdev->host_features,
+ &local_err));
+ }
if (local_err) {
error_propagate(errp, local_err);
return;
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 2817d3a893..5b60617d33 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3087,7 +3087,9 @@ static int virtio_set_features_nocheck(VirtIODevice *vdev, const uint64_t *val)
bad = virtio_features_andnot(tmp, val, vdev->host_features_ex);
virtio_features_and(tmp, val, vdev->host_features_ex);
- if (k->set_features) {
+ if (k->set_features_ex) {
+ k->set_features_ex(vdev, val);
+ } else if (k->set_features) {
bad = bad || virtio_features_use_extended(tmp);
k->set_features(vdev, tmp[0]);
}
@@ -3130,9 +3132,8 @@ virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev,
}
}
-int virtio_set_features(VirtIODevice *vdev, uint64_t val)
+int virtio_set_features_ex(VirtIODevice *vdev, const uint64_t *features)
{
- uint64_t features[VIRTIO_FEATURES_DWORDS];
int ret;
/*
* The driver must not attempt to set features after feature negotiation
@@ -3142,13 +3143,12 @@ int virtio_set_features(VirtIODevice *vdev, uint64_t val)
return -EINVAL;
}
- if (val & (1ull << VIRTIO_F_BAD_FEATURE)) {
+ if (features[0] & (1ull << VIRTIO_F_BAD_FEATURE)) {
qemu_log_mask(LOG_GUEST_ERROR,
"%s: guest driver for %s has enabled UNUSED(30) feature bit!\n",
__func__, vdev->name);
}
- virtio_features_from_u64(features, val);
ret = virtio_set_features_nocheck(vdev, features);
if (virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) {
/* VIRTIO_RING_F_EVENT_IDX changes the size of the caches. */
@@ -3168,6 +3168,14 @@ int virtio_set_features(VirtIODevice *vdev, uint64_t val)
return ret;
}
+int virtio_set_features(VirtIODevice *vdev, uint64_t val)
+{
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
+
+ virtio_features_from_u64(features, val);
+ return virtio_set_features_ex(vdev, features);
+}
+
void virtio_reset(void *opaque)
{
VirtIODevice *vdev = opaque;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 39e4059a66..2aeb021fb3 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -178,6 +178,9 @@ struct VirtioDeviceClass {
/* This is what a VirtioDevice must implement */
DeviceRealize realize;
DeviceUnrealize unrealize;
+ void (*get_features_ex)(VirtIODevice *vdev, uint64_t *requested_features,
+ Error **errp);
+ void (*set_features_ex)(VirtIODevice *vdev, const uint64_t *val);
uint64_t (*get_features)(VirtIODevice *vdev,
uint64_t requested_features,
Error **errp);
@@ -373,6 +376,7 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
void virtio_update_irq(VirtIODevice *vdev);
int virtio_set_features(VirtIODevice *vdev, uint64_t val);
+int virtio_set_features_ex(VirtIODevice *vdev, const uint64_t *val);
/* Base devices. */
typedef struct VirtIOBlkConf VirtIOBlkConf;
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 06/13] virtio-pci: implement support for extended features
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (4 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 05/13] virtio: add support for negotiating extended features Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-22 3:28 ` Jason Wang
2025-07-18 8:52 ` [PATCH RFC v3 07/13] vhost: add support for negotiating " Paolo Abeni
` (6 subsequent siblings)
12 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Extend the features configuration space to 128 bits, and allow the
common read/write operation to access all of it.
On migration, save the 128 bit version of the features only if the
upper bits are non zero. Relay reset to clear all the feature
space before load.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- drop the pre_load/post_load trickery and relay on reset zeroing
the features instead.
- avoid union usage, just increase guest_features size and use
SUB_ARRAY.
- drop unneeded '!!'
- _array -> _ex
v1 -> v2:
- use separate VMStateDescription and pre/post load to avoid breaking
migration
- clear proxy features on device reset
---
hw/virtio/virtio-pci.c | 54 +++++++++++++++++++++++++++++-----
include/hw/virtio/virtio-pci.h | 2 +-
2 files changed, 48 insertions(+), 8 deletions(-)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 767216d795..6800c80eed 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -109,6 +109,29 @@ static const VMStateDescription vmstate_virtio_pci_modern_queue_state = {
}
};
+static bool virtio_pci_modern_state_features128_needed(void *opaque)
+{
+ VirtIOPCIProxy *proxy = opaque;
+ uint32_t features = 0;
+ int i;
+
+ for (i = 2; i < ARRAY_SIZE(proxy->guest_features); ++i) {
+ features |= proxy->guest_features[i];
+ }
+ return features;
+}
+
+static const VMStateDescription vmstate_virtio_pci_modern_state_features128 = {
+ .name = "virtio_pci/modern_state/features128",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .needed = &virtio_pci_modern_state_features128_needed,
+ .fields = (const VMStateField[]) {
+ VMSTATE_UINT32_SUB_ARRAY(guest_features, VirtIOPCIProxy, 2, 2),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
static bool virtio_pci_modern_state_needed(void *opaque)
{
VirtIOPCIProxy *proxy = opaque;
@@ -124,11 +147,15 @@ static const VMStateDescription vmstate_virtio_pci_modern_state_sub = {
.fields = (const VMStateField[]) {
VMSTATE_UINT32(dfselect, VirtIOPCIProxy),
VMSTATE_UINT32(gfselect, VirtIOPCIProxy),
- VMSTATE_UINT32_ARRAY(guest_features, VirtIOPCIProxy, 2),
+ VMSTATE_UINT32_SUB_ARRAY(guest_features, VirtIOPCIProxy, 0, 2),
VMSTATE_STRUCT_ARRAY(vqs, VirtIOPCIProxy, VIRTIO_QUEUE_MAX, 0,
vmstate_virtio_pci_modern_queue_state,
VirtIOPCIQueue),
VMSTATE_END_OF_LIST()
+ },
+ .subsections = (const VMStateDescription * const []) {
+ &vmstate_virtio_pci_modern_state_features128,
+ NULL
}
};
@@ -1494,11 +1521,14 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,
val = proxy->dfselect;
break;
case VIRTIO_PCI_COMMON_DF:
- if (proxy->dfselect <= 1) {
+ if (proxy->dfselect < VIRTIO_FEATURES_WORDS) {
VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
- val = (vdev->host_features & ~vdc->legacy_features) >>
- (32 * proxy->dfselect);
+ val = vdev->host_features_ex[proxy->dfselect >> 1] >>
+ (32 * (proxy->dfselect & 1));
+ if (proxy->dfselect <= 1) {
+ val &= (~vdc->legacy_features) >> (32 * proxy->dfselect);
+ }
}
break;
case VIRTIO_PCI_COMMON_GFSELECT:
@@ -1589,10 +1619,17 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
break;
case VIRTIO_PCI_COMMON_GF:
if (proxy->gfselect < ARRAY_SIZE(proxy->guest_features)) {
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
+ int i;
+
proxy->guest_features[proxy->gfselect] = val;
- virtio_set_features(vdev,
- (((uint64_t)proxy->guest_features[1]) << 32) |
- proxy->guest_features[0]);
+ virtio_features_clear(features);
+ for (i = 0; i < ARRAY_SIZE(proxy->guest_features); ++i) {
+ uint64_t cur = proxy->guest_features[i];
+
+ features[i >> 1] |= cur << ((i & 1) * 32);
+ }
+ virtio_set_features_ex(vdev, features);
}
break;
case VIRTIO_PCI_COMMON_MSIX:
@@ -2311,6 +2348,9 @@ static void virtio_pci_reset(DeviceState *qdev)
virtio_bus_reset(bus);
msix_unuse_all_vectors(&proxy->pci_dev);
+ QEMU_BUILD_BUG_ON(ARRAY_SIZE(proxy->guest_features) != 4);
+ memset(proxy->guest_features, 0, sizeof(proxy->guest_features));
+
for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
proxy->vqs[i].enabled = 0;
proxy->vqs[i].reset = 0;
diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h
index eab5394898..eae0fcdec3 100644
--- a/include/hw/virtio/virtio-pci.h
+++ b/include/hw/virtio/virtio-pci.h
@@ -158,7 +158,7 @@ struct VirtIOPCIProxy {
uint32_t nvectors;
uint32_t dfselect;
uint32_t gfselect;
- uint32_t guest_features[2];
+ uint32_t guest_features[4];
VirtIOPCIQueue vqs[VIRTIO_QUEUE_MAX];
VirtIOIRQFD *vector_irqfd;
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (5 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 06/13] virtio-pci: implement support for " Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 14:36 ` Stefano Garzarella
2025-07-22 3:32 ` Jason Wang
2025-07-18 8:52 ` [PATCH RFC v3 08/13] qmp: update virtio features map to support " Paolo Abeni
` (5 subsequent siblings)
12 siblings, 2 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Similar to virtio infra, vhost core maintains the features status
in the full extended format and allows the devices to implement
extended version of the getter/setter.
Note that 'protocol_features' are not extended: they are only
used by vhost-user, and the latter device is not going to implement
extended features soon.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- fix compile warning
- _array -> _ex
v1 -> v2:
- uint128_t -> uint64_t[]
- add _ex() variant of features manipulation helpers
---
hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
include/hw/virtio/vhost-backend.h | 6 +++
include/hw/virtio/vhost.h | 33 ++++++++++++--
3 files changed, 100 insertions(+), 12 deletions(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index c30ea1156e..85ae1e4d4c 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
static int vhost_dev_set_features(struct vhost_dev *dev,
bool enable_log)
{
- uint64_t features = dev->acked_features;
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
int r;
+
+ virtio_features_copy(features, dev->acked_features_ex);
if (enable_log) {
- features |= 0x1ULL << VHOST_F_LOG_ALL;
+ virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
}
if (!vhost_dev_has_iommu(dev)) {
- features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
+ virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
}
if (dev->vhost_ops->vhost_force_iommu) {
if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
- features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
+ virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
}
}
- r = dev->vhost_ops->vhost_set_features(dev, features);
+
+ if (virtio_features_use_extended(features) &&
+ !dev->vhost_ops->vhost_set_features_ex) {
+ VHOST_OPS_DEBUG(r, "extended features without device support");
+ r = -EINVAL;
+ goto out;
+ }
+
+ if (dev->vhost_ops->vhost_set_features_ex) {
+ r = dev->vhost_ops->vhost_set_features_ex(dev, features);
+ } else {
+ r = dev->vhost_ops->vhost_set_features(dev, features[0]);
+ }
if (r < 0) {
VHOST_OPS_DEBUG(r, "vhost_set_features failed");
goto out;
@@ -1506,12 +1520,27 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
}
}
+static int vhost_dev_get_features(struct vhost_dev *hdev,
+ uint64_t *features)
+{
+ uint64_t features64;
+ int r;
+
+ if (hdev->vhost_ops->vhost_get_features_ex) {
+ return hdev->vhost_ops->vhost_get_features_ex(hdev, features);
+ }
+
+ r = hdev->vhost_ops->vhost_get_features(hdev, &features64);
+ virtio_features_from_u64(features, features64);
+ return r;
+}
+
int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
VhostBackendType backend_type, uint32_t busyloop_timeout,
Error **errp)
{
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
unsigned int used, reserved, limit;
- uint64_t features;
int i, r, n_initialized_vqs = 0;
hdev->vdev = NULL;
@@ -1531,7 +1560,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
goto fail;
}
- r = hdev->vhost_ops->vhost_get_features(hdev, &features);
+ r = vhost_dev_get_features(hdev, features);
if (r < 0) {
error_setg_errno(errp, -r, "vhost_get_features failed");
goto fail;
@@ -1569,7 +1598,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
}
}
- hdev->features = features;
+ virtio_features_copy(hdev->features_ex, features);
hdev->memory_listener = (MemoryListener) {
.name = "vhost",
@@ -1592,7 +1621,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
};
if (hdev->migration_blocker == NULL) {
- if (!(hdev->features & (0x1ULL << VHOST_F_LOG_ALL))) {
+ if (!virtio_has_feature_ex(hdev->features_ex, VHOST_F_LOG_ALL)) {
error_setg(&hdev->migration_blocker,
"Migration disabled: vhost lacks VHOST_F_LOG_ALL feature.");
} else if (vhost_dev_log_is_shared(hdev) && !qemu_memfd_alloc_check()) {
@@ -1871,6 +1900,20 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
return features;
}
+void vhost_get_features_ex(struct vhost_dev *hdev,
+ const int *feature_bits,
+ uint64_t *features)
+{
+ const int *bit = feature_bits;
+
+ while (*bit != VHOST_INVALID_FEATURE_BIT) {
+ if (!virtio_has_feature_ex(hdev->features_ex, *bit)) {
+ virtio_clear_feature_ex(features, *bit);
+ }
+ bit++;
+ }
+}
+
void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
uint64_t features)
{
@@ -1884,6 +1927,18 @@ void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
}
}
+void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
+ const uint64_t *features)
+{
+ const int *bit = feature_bits;
+ while (*bit != VHOST_INVALID_FEATURE_BIT) {
+ if (virtio_has_feature_ex(features, *bit)) {
+ virtio_add_feature_ex(hdev->acked_features_ex, *bit);
+ }
+ bit++;
+ }
+}
+
int vhost_dev_get_config(struct vhost_dev *hdev, uint8_t *config,
uint32_t config_len, Error **errp)
{
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index d6df209a2f..ff94fa1734 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -95,6 +95,10 @@ typedef int (*vhost_new_worker_op)(struct vhost_dev *dev,
struct vhost_worker_state *worker);
typedef int (*vhost_free_worker_op)(struct vhost_dev *dev,
struct vhost_worker_state *worker);
+typedef int (*vhost_set_features_ex_op)(struct vhost_dev *dev,
+ const uint64_t *features);
+typedef int (*vhost_get_features_ex_op)(struct vhost_dev *dev,
+ uint64_t *features);
typedef int (*vhost_set_features_op)(struct vhost_dev *dev,
uint64_t features);
typedef int (*vhost_get_features_op)(struct vhost_dev *dev,
@@ -186,6 +190,8 @@ typedef struct VhostOps {
vhost_free_worker_op vhost_free_worker;
vhost_get_vring_worker_op vhost_get_vring_worker;
vhost_attach_vring_worker_op vhost_attach_vring_worker;
+ vhost_set_features_ex_op vhost_set_features_ex;
+ vhost_get_features_ex_op vhost_get_features_ex;
vhost_set_features_op vhost_set_features;
vhost_get_features_op vhost_get_features;
vhost_set_backend_cap_op vhost_set_backend_cap;
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 66be6afc88..39fbffc6bc 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -107,9 +107,9 @@ struct vhost_dev {
* future use should be discouraged and the variable retired as
* its easy to confuse with the VirtIO backend_features.
*/
- uint64_t features;
- uint64_t acked_features;
- uint64_t backend_features;
+ VIRTIO_DECLARE_FEATURES(features);
+ VIRTIO_DECLARE_FEATURES(acked_features);
+ VIRTIO_DECLARE_FEATURES(backend_features);
/**
* @protocol_features: is the vhost-user only feature set by
@@ -333,6 +333,21 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, VirtIODevice *vdev, int n,
uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
uint64_t features);
+/**
+ * vhost_get_features_ex() - sanitize the extended features set
+ * @hdev: common vhost_dev structure
+ * @feature_bits: pointer to terminated table of feature bits
+ * @features: original features set, filtered out on return
+ *
+ * This is the extended variant of vhost_get_features(), supporting the
+ * the extended features set. Filter it with the intersection of what is
+ * supported by the vhost backend (hdev->features) and the supported
+ * feature_bits.
+ */
+void vhost_get_features_ex(struct vhost_dev *hdev,
+ const int *feature_bits,
+ uint64_t *features);
+
/**
* vhost_ack_features() - set vhost acked_features
* @hdev: common vhost_dev structure
@@ -344,6 +359,18 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
*/
void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
uint64_t features);
+
+/**
+ * vhost_ack_features_ex() - set vhost full set of acked_features
+ * @hdev: common vhost_dev structure
+ * @feature_bits: pointer to terminated table of feature bits
+ * @features: requested feature set
+ *
+ * This sets the internal hdev->acked_features to the intersection of
+ * the backends advertised features and the supported feature_bits.
+ */
+void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
+ const uint64_t *features);
unsigned int vhost_get_max_memslots(void);
unsigned int vhost_get_free_memslots(void);
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (6 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 07/13] vhost: add support for negotiating " Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 10:18 ` Stefano Garzarella
` (2 more replies)
2025-07-18 8:52 ` [PATCH RFC v3 09/13] vhost-backend: implement extended features support Paolo Abeni
` (4 subsequent siblings)
12 siblings, 3 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Extend the VirtioDeviceFeatures struct with an additional u64
to track unknown features in the 64-127 bit range and decode
the full virtio features spaces for vhost and virtio devices.
Also add entries for the soon-to-be-supported virtio net GSO over
UDP features.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- unknown-dev-features-dword2 -> unknown-dev-features2
- _array -> _ex
- fixed typos in entries description
v1 -> v2:
- uint128_t -> uint64_t[]
---
hw/virtio/virtio-hmp-cmds.c | 3 +-
hw/virtio/virtio-qmp.c | 89 ++++++++++++++++++++++++++-----------
hw/virtio/virtio-qmp.h | 3 +-
qapi/virtio.json | 8 +++-
4 files changed, 73 insertions(+), 30 deletions(-)
diff --git a/hw/virtio/virtio-hmp-cmds.c b/hw/virtio/virtio-hmp-cmds.c
index 7d8677bcf0..1daae482d3 100644
--- a/hw/virtio/virtio-hmp-cmds.c
+++ b/hw/virtio/virtio-hmp-cmds.c
@@ -74,7 +74,8 @@ static void hmp_virtio_dump_features(Monitor *mon,
}
if (features->has_unknown_dev_features) {
- monitor_printf(mon, " unknown-features(0x%016"PRIx64")\n",
+ monitor_printf(mon, " unknown-features(0x%016"PRIx64"%016"PRIx64")\n",
+ features->unknown_dev_features2,
features->unknown_dev_features);
}
}
diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
index 3b6377cf0d..03c6163cf4 100644
--- a/hw/virtio/virtio-qmp.c
+++ b/hw/virtio/virtio-qmp.c
@@ -325,6 +325,20 @@ static const qmp_virtio_feature_map_t virtio_net_feature_map[] = {
FEATURE_ENTRY(VHOST_USER_F_PROTOCOL_FEATURES, \
"VHOST_USER_F_PROTOCOL_FEATURES: Vhost-user protocol features "
"negotiation supported"),
+ FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, \
+ "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
+ "UDP tunnel packets"),
+ FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, \
+ "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
+ "UDP tunnel packets requiring checksum offload for the outer "
+ "header"),
+ FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, \
+ "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
+ "UDP tunnel packets"),
+ FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, \
+ "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
+ "UDP tunnel packets requiring checksum offload for the outer "
+ "header"),
{ -1, "" }
};
#endif
@@ -510,6 +524,24 @@ static const qmp_virtio_feature_map_t virtio_gpio_feature_map[] = {
list; \
})
+#define CONVERT_FEATURES_EX(type, map, bitmap) \
+ ({ \
+ type *list = NULL; \
+ type *node; \
+ for (i = 0; map[i].virtio_bit != -1; i++) { \
+ bit = map[i].virtio_bit; \
+ if (!virtio_has_feature_ex(bitmap, bit)) { \
+ continue; \
+ } \
+ node = g_new0(type, 1); \
+ node->value = g_strdup(map[i].feature_desc); \
+ node->next = list; \
+ list = node; \
+ virtio_clear_feature_ex(bitmap, bit); \
+ } \
+ list; \
+ })
+
VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap)
{
VirtioDeviceStatus *status;
@@ -545,109 +577,112 @@ VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap)
return vhu_protocols;
}
-VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
+VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
+ const uint64_t *bmap)
{
+ uint64_t bitmap[VIRTIO_FEATURES_DWORDS];
VirtioDeviceFeatures *features;
uint64_t bit;
int i;
+ virtio_features_copy(bitmap, bmap);
features = g_new0(VirtioDeviceFeatures, 1);
features->has_dev_features = true;
/* transport features */
- features->transports = CONVERT_FEATURES(strList, virtio_transport_map, 0,
- bitmap);
+ features->transports = CONVERT_FEATURES_EX(strList, virtio_transport_map,
+ bitmap);
/* device features */
switch (device_id) {
#ifdef CONFIG_VIRTIO_SERIAL
case VIRTIO_ID_CONSOLE:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_serial_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_serial_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_BLK
case VIRTIO_ID_BLOCK:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_blk_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_blk_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_GPU
case VIRTIO_ID_GPU:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_gpu_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_gpu_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_NET
case VIRTIO_ID_NET:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_net_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_net_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_SCSI
case VIRTIO_ID_SCSI:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_scsi_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_scsi_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_BALLOON
case VIRTIO_ID_BALLOON:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_balloon_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_balloon_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_IOMMU
case VIRTIO_ID_IOMMU:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_iommu_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_iommu_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_INPUT
case VIRTIO_ID_INPUT:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_input_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_input_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VHOST_USER_FS
case VIRTIO_ID_FS:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_fs_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_fs_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VHOST_VSOCK
case VIRTIO_ID_VSOCK:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_vsock_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_vsock_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_CRYPTO
case VIRTIO_ID_CRYPTO:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_crypto_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_crypto_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_MEM
case VIRTIO_ID_MEM:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_mem_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_mem_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_I2C_ADAPTER
case VIRTIO_ID_I2C_ADAPTER:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_i2c_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_i2c_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VIRTIO_RNG
case VIRTIO_ID_RNG:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_rng_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_rng_feature_map, bitmap);
break;
#endif
#ifdef CONFIG_VHOST_USER_GPIO
case VIRTIO_ID_GPIO:
features->dev_features =
- CONVERT_FEATURES(strList, virtio_gpio_feature_map, 0, bitmap);
+ CONVERT_FEATURES_EX(strList, virtio_gpio_feature_map, bitmap);
break;
#endif
/* No features */
@@ -680,9 +715,10 @@ VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
g_assert_not_reached();
}
- features->has_unknown_dev_features = bitmap != 0;
+ features->has_unknown_dev_features = !virtio_features_empty(bitmap);
if (features->has_unknown_dev_features) {
- features->unknown_dev_features = bitmap;
+ features->unknown_dev_features = bitmap[0];
+ features->unknown_dev_features2 = bitmap[1];
}
return features;
@@ -743,11 +779,11 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
status->device_id = vdev->device_id;
status->vhost_started = vdev->vhost_started;
status->guest_features = qmp_decode_features(vdev->device_id,
- vdev->guest_features);
+ vdev->guest_features_ex);
status->host_features = qmp_decode_features(vdev->device_id,
- vdev->host_features);
+ vdev->host_features_ex);
status->backend_features = qmp_decode_features(vdev->device_id,
- vdev->backend_features);
+ vdev->backend_features_ex);
switch (vdev->device_endian) {
case VIRTIO_DEVICE_ENDIAN_LITTLE:
@@ -785,11 +821,12 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
status->vhost_dev->nvqs = hdev->nvqs;
status->vhost_dev->vq_index = hdev->vq_index;
status->vhost_dev->features =
- qmp_decode_features(vdev->device_id, hdev->features);
+ qmp_decode_features(vdev->device_id, hdev->features_ex);
status->vhost_dev->acked_features =
- qmp_decode_features(vdev->device_id, hdev->acked_features);
+ qmp_decode_features(vdev->device_id, hdev->acked_features_ex);
status->vhost_dev->backend_features =
- qmp_decode_features(vdev->device_id, hdev->backend_features);
+ qmp_decode_features(vdev->device_id, hdev->backend_features_ex);
+
status->vhost_dev->protocol_features =
qmp_decode_protocols(hdev->protocol_features);
status->vhost_dev->max_queues = hdev->max_queues;
diff --git a/hw/virtio/virtio-qmp.h b/hw/virtio/virtio-qmp.h
index 245a446a56..e0a1e49035 100644
--- a/hw/virtio/virtio-qmp.h
+++ b/hw/virtio/virtio-qmp.h
@@ -18,6 +18,7 @@
VirtIODevice *qmp_find_virtio_device(const char *path);
VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap);
VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap);
-VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap);
+VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
+ const uint64_t *bitmap);
#endif
diff --git a/qapi/virtio.json b/qapi/virtio.json
index 9d652fe4a8..f2e2dd6e97 100644
--- a/qapi/virtio.json
+++ b/qapi/virtio.json
@@ -490,14 +490,18 @@
# unique features)
#
# @unknown-dev-features: Virtio device features bitmap that have not
-# been decoded
+# been decoded (bits 0-63)
+#
+# @unknown-dev-features2: Virtio device features bitmap that have not
+# been decoded (bits 64-127)
#
# Since: 7.2
##
{ 'struct': 'VirtioDeviceFeatures',
'data': { 'transports': [ 'str' ],
'*dev-features': [ 'str' ],
- '*unknown-dev-features': 'uint64' } }
+ '*unknown-dev-features': 'uint64',
+ '*unknown-dev-features2': 'uint64' } }
##
# @VirtQueueStatus:
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 09/13] vhost-backend: implement extended features support
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (7 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 08/13] qmp: update virtio features map to support " Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 10/13] vhost-net: " Paolo Abeni
` (3 subsequent siblings)
12 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Leverage the kernel extended features manipulation ioctls(), if
available, and fallback to old ops otherwise. Error out when setting
extended features but kernel support is not available.
Note that extended support for get/set backend features is not needed,
as the only feature that can be changed belongs to the 64 bit range.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v1 -> v2:
- synced with kernel ioctl changes
---
hw/virtio/vhost-backend.c | 62 ++++++++++++++++++++++++++++++++-------
1 file changed, 51 insertions(+), 11 deletions(-)
diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
index 833804dd40..6fde994403 100644
--- a/hw/virtio/vhost-backend.c
+++ b/hw/virtio/vhost-backend.c
@@ -20,6 +20,11 @@
#include <linux/vhost.h>
#include <sys/ioctl.h>
+struct vhost_features {
+ uint64_t count;
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
+};
+
static int vhost_kernel_call(struct vhost_dev *dev, unsigned long int request,
void *arg)
{
@@ -182,12 +187,6 @@ static int vhost_kernel_get_vring_worker(struct vhost_dev *dev,
return vhost_kernel_call(dev, VHOST_GET_VRING_WORKER, worker);
}
-static int vhost_kernel_set_features(struct vhost_dev *dev,
- uint64_t features)
-{
- return vhost_kernel_call(dev, VHOST_SET_FEATURES, &features);
-}
-
static int vhost_kernel_set_backend_cap(struct vhost_dev *dev)
{
uint64_t features;
@@ -210,10 +209,51 @@ static int vhost_kernel_set_backend_cap(struct vhost_dev *dev)
return 0;
}
-static int vhost_kernel_get_features(struct vhost_dev *dev,
- uint64_t *features)
+static int vhost_kernel_set_features(struct vhost_dev *dev,
+ const uint64_t *features)
{
- return vhost_kernel_call(dev, VHOST_GET_FEATURES, features);
+ struct vhost_features farray;
+ bool extended_in_use;
+ int r;
+
+ farray.count = VIRTIO_FEATURES_DWORDS;
+ virtio_features_copy(farray.features, features);
+ extended_in_use = virtio_features_use_extended(farray.features);
+
+ /*
+ * Can't check for ENOTTY: for unknown ioctls the kernel interprets
+ * the argument as a virtio queue id and most likely errors out validating
+ * such id, instead of reporting an unknown operation.
+ */
+ r = vhost_kernel_call(dev, VHOST_SET_FEATURES_ARRAY, &farray);
+ if (!r) {
+ return 0;
+ }
+
+ if (extended_in_use) {
+ error_report("Trying to set extended features without kernel support");
+ return -EINVAL;
+ }
+ return vhost_kernel_call(dev, VHOST_SET_FEATURES, &farray.features[0]);
+}
+
+static int vhost_kernel_get_features(struct vhost_dev *dev, uint64_t *features)
+{
+ struct vhost_features farray;
+ int r;
+
+ farray.count = VIRTIO_FEATURES_DWORDS;
+ r = vhost_kernel_call(dev, VHOST_GET_FEATURES_ARRAY, &farray);
+ if (r) {
+ memset(&farray, 0, sizeof(farray));
+ r = vhost_kernel_call(dev, VHOST_GET_FEATURES, &farray.features[0]);
+ }
+ if (r) {
+ return r;
+ }
+
+ virtio_features_copy(features, farray.features);
+ return 0;
}
static int vhost_kernel_set_owner(struct vhost_dev *dev)
@@ -341,8 +381,8 @@ const VhostOps kernel_ops = {
.vhost_attach_vring_worker = vhost_kernel_attach_vring_worker,
.vhost_new_worker = vhost_kernel_new_worker,
.vhost_free_worker = vhost_kernel_free_worker,
- .vhost_set_features = vhost_kernel_set_features,
- .vhost_get_features = vhost_kernel_get_features,
+ .vhost_set_features_ex = vhost_kernel_set_features,
+ .vhost_get_features_ex = vhost_kernel_get_features,
.vhost_set_backend_cap = vhost_kernel_set_backend_cap,
.vhost_set_owner = vhost_kernel_set_owner,
.vhost_get_vq_index = vhost_kernel_get_vq_index,
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 10/13] vhost-net: implement extended features support
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (8 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 09/13] vhost-backend: implement extended features support Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 13:01 ` Stefano Garzarella
2025-07-18 8:52 ` [PATCH RFC v3 11/13] virtio-net: " Paolo Abeni
` (2 subsequent siblings)
12 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Provide extended version of the features manipulation helpers,
and let the device initialization deal with the full features space,
adjusting the relevant format strings accordingly.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- use virtio_features_andnot return value instead of virtio_features_empty
- rebased on top of "net: Consolidate vhost feature bits into vhost_net
structure"
- _array -> _ex
v1 -> v2:
- uint128_t -> uint64_t[]
- provide extended variant of the features manipulation helpers
---
hw/net/vhost_net-stub.c | 8 +++----
hw/net/vhost_net.c | 46 ++++++++++++++++++++++-------------------
include/net/vhost_net.h | 33 ++++++++++++++++++++++++++---
3 files changed, 58 insertions(+), 29 deletions(-)
diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index 7d49f82906..0740d5a2eb 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -46,9 +46,8 @@ void vhost_net_cleanup(struct vhost_net *net)
{
}
-uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
+void vhost_net_get_features_ex(struct vhost_net *net, uint64_t *features)
{
- return features;
}
int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
@@ -62,13 +61,12 @@ int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
return 0;
}
-void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
+void vhost_net_ack_features_ex(struct vhost_net *net, const uint64_t *features)
{
}
-uint64_t vhost_net_get_acked_features(VHostNetState *net)
+void vhost_net_get_acked_features_ex(VHostNetState *net, uint64_t *features)
{
- return 0;
}
bool vhost_net_virtqueue_pending(VHostNetState *net, int idx)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 540492b37d..33fc6ba2f9 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -35,10 +35,9 @@
#include "hw/virtio/virtio-bus.h"
#include "linux-headers/linux/vhost.h"
-uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
+void vhost_net_get_features_ex(struct vhost_net *net, uint64_t *features)
{
- return vhost_get_features(&net->dev, net->feature_bits,
- features);
+ vhost_get_features_ex(&net->dev, net->feature_bits, features);
}
int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
uint32_t config_len)
@@ -51,10 +50,11 @@ int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
return vhost_dev_set_config(&net->dev, data, offset, size, flags);
}
-void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
+void vhost_net_ack_features_ex(struct vhost_net *net, const uint64_t *features)
{
- net->dev.acked_features = net->dev.backend_features;
- vhost_ack_features(&net->dev, net->feature_bits, features);
+ virtio_features_copy(net->dev.acked_features_ex,
+ net->dev.backend_features_ex);
+ vhost_ack_features_ex(&net->dev, net->feature_bits, features);
}
uint64_t vhost_net_get_max_queues(VHostNetState *net)
@@ -62,9 +62,9 @@ uint64_t vhost_net_get_max_queues(VHostNetState *net)
return net->dev.max_queues;
}
-uint64_t vhost_net_get_acked_features(VHostNetState *net)
+void vhost_net_get_acked_features_ex(VHostNetState *net, uint64_t *features)
{
- return net->dev.acked_features;
+ virtio_features_copy(features, net->dev.acked_features_ex);
}
void vhost_net_save_acked_features(NetClientState *nc)
@@ -234,7 +234,8 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
int r;
bool backend_kernel = options->backend_type == VHOST_BACKEND_TYPE_KERNEL;
struct vhost_net *net = g_new0(struct vhost_net, 1);
- uint64_t features = 0;
+ uint64_t missing_features[VIRTIO_FEATURES_DWORDS];
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
Error *local_err = NULL;
if (!options->net_backend) {
@@ -261,7 +262,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
net->backend = r;
net->dev.protocol_features = 0;
} else {
- net->dev.backend_features = 0;
+ virtio_features_clear(net->dev.backend_features_ex);
net->dev.protocol_features = 0;
net->backend = -1;
@@ -279,28 +280,31 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
if (backend_kernel) {
if (!qemu_has_vnet_hdr_len(options->net_backend,
sizeof(struct virtio_net_hdr_mrg_rxbuf))) {
- net->dev.features &= ~(1ULL << VIRTIO_NET_F_MRG_RXBUF);
+ net->dev.features &= ~VIRTIO_BIT(VIRTIO_NET_F_MRG_RXBUF);
}
- if (~net->dev.features & net->dev.backend_features) {
- fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
- " for backend\n",
- (uint64_t)(~net->dev.features & net->dev.backend_features));
+
+ if (virtio_features_andnot(missing_features,
+ net->dev.backend_features_ex,
+ net->dev.features_ex)) {
+ fprintf(stderr, "vhost lacks feature mask 0x" VIRTIO_FEATURES_FMT
+ " for backend\n", VIRTIO_FEATURES_PR(missing_features));
goto fail;
}
}
/* Set sane init value. Override when guest acks. */
if (options->get_acked_features) {
- features = options->get_acked_features(net->nc);
- if (~net->dev.features & features) {
- fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
- " for backend\n",
- (uint64_t)(~net->dev.features & features));
+ virtio_features_from_u64(features,
+ options->get_acked_features(net->nc));
+ if (virtio_features_andnot(missing_features, features,
+ net->dev.features_ex)) {
+ fprintf(stderr, "vhost lacks feature mask 0x" VIRTIO_FEATURES_FMT
+ " for backend\n", VIRTIO_FEATURES_PR(missing_features));
goto fail;
}
}
- vhost_net_ack_features(net, features);
+ vhost_net_ack_features_ex(net, features);
return net;
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 879781dad7..1d07b437c7 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -2,6 +2,7 @@
#define VHOST_NET_H
#include "net/net.h"
+#include "hw/virtio/virtio-features.h"
#include "hw/virtio/vhost-backend.h"
struct vhost_net;
@@ -33,8 +34,26 @@ void vhost_net_stop(VirtIODevice *dev, NetClientState *ncs,
void vhost_net_cleanup(VHostNetState *net);
-uint64_t vhost_net_get_features(VHostNetState *net, uint64_t features);
-void vhost_net_ack_features(VHostNetState *net, uint64_t features);
+void vhost_net_get_features_ex(VHostNetState *net, uint64_t *features);
+static inline uint64_t vhost_net_get_features(VHostNetState *net,
+ uint64_t features)
+{
+ uint64_t features_array[VIRTIO_FEATURES_DWORDS];
+
+ virtio_features_from_u64(features_array, features);
+ vhost_net_get_features_ex(net, features_array);
+ return features_array[0];
+}
+
+void vhost_net_ack_features_ex(VHostNetState *net, const uint64_t *features);
+static inline void vhost_net_ack_features(VHostNetState *net,
+ uint64_t features)
+{
+ uint64_t features_array[VIRTIO_FEATURES_DWORDS];
+
+ virtio_features_from_u64(features_array, features);
+ vhost_net_ack_features_ex(net, features_array);
+}
int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
uint32_t config_len);
@@ -51,7 +70,15 @@ VHostNetState *get_vhost_net(NetClientState *nc);
int vhost_net_set_vring_enable(NetClientState *nc, int enable);
-uint64_t vhost_net_get_acked_features(VHostNetState *net);
+void vhost_net_get_acked_features_ex(VHostNetState *net, uint64_t *features);
+static inline uint64_t vhost_net_get_acked_features(VHostNetState *net)
+{
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
+
+ vhost_net_get_acked_features_ex(net, features);
+ assert(!virtio_features_use_extended(features));
+ return features[0];
+}
int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 11/13] virtio-net: implement extended features support
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (9 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 10/13] vhost-net: " Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 12/13] net: implement tunnel probing Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading Paolo Abeni
12 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Use the extended types and helpers to manipulate the virtio_net
features.
Note that offloads are still 64bits wide, as per specification,
and extended offloads will be mapped into such range.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- rebased on top of 2deec9ab7d ("virtio-net: Move
virtio_net_get_features() down")
- _array -> _ex
v1 -> v2:
- uint128_t -> uint64_t[]
- more verbose macro definitions
---
hw/net/virtio-net.c | 136 +++++++++++++++++++--------------
include/hw/virtio/virtio-net.h | 2 +-
2 files changed, 80 insertions(+), 58 deletions(-)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 8953c329e7..53413ec4d5 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -90,6 +90,19 @@
VIRTIO_NET_RSS_HASH_TYPE_TCP_EX | \
VIRTIO_NET_RSS_HASH_TYPE_UDP_EX)
+/*
+ * Features starting from VIRTIO_NET_FEATURES_MAP_MIN bit correspond
+ * to guest offloads in the VIRTIO_NET_OFFLOAD_MAP range
+ */
+#define VIRTIO_NET_OFFLOAD_MAP_MIN 46
+#define VIRTIO_NET_OFFLOAD_MAP_LENGTH 4
+#define VIRTIO_NET_OFFLOAD_MAP MAKE_64BIT_MASK( \
+ VIRTIO_NET_OFFLOAD_MAP_MIN, \
+ VIRTIO_NET_OFFLOAD_MAP_LENGTH)
+#define VIRTIO_NET_FEATURES_MAP_MIN 65
+#define VIRTIO_NET_F2O_SHIFT (VIRTIO_NET_OFFLOAD_MAP_MIN - \
+ VIRTIO_NET_FEATURES_MAP_MIN + 64)
+
static const VirtIOFeature feature_sizes[] = {
{.flags = 1ULL << VIRTIO_NET_F_MAC,
.end = endof(struct virtio_net_config, mac)},
@@ -786,7 +799,14 @@ static void virtio_net_apply_guest_offloads(VirtIONet *n)
qemu_set_offload(qemu_get_queue(n->nic)->peer, &ol);
}
-static uint64_t virtio_net_guest_offloads_by_features(uint64_t features)
+static uint64_t virtio_net_features_to_offload(const uint64_t *features)
+{
+ return (features[0] & ~VIRTIO_NET_OFFLOAD_MAP) |
+ ((features[1] << VIRTIO_NET_F2O_SHIFT) & VIRTIO_NET_OFFLOAD_MAP);
+}
+
+static uint64_t
+virtio_net_guest_offloads_by_features(const uint64_t *features)
{
static const uint64_t guest_offloads_mask =
(1ULL << VIRTIO_NET_F_GUEST_CSUM) |
@@ -797,13 +817,13 @@ static uint64_t virtio_net_guest_offloads_by_features(uint64_t features)
(1ULL << VIRTIO_NET_F_GUEST_USO4) |
(1ULL << VIRTIO_NET_F_GUEST_USO6);
- return guest_offloads_mask & features;
+ return guest_offloads_mask & virtio_net_features_to_offload(features);
}
uint64_t virtio_net_supported_guest_offloads(const VirtIONet *n)
{
VirtIODevice *vdev = VIRTIO_DEVICE(n);
- return virtio_net_guest_offloads_by_features(vdev->guest_features);
+ return virtio_net_guest_offloads_by_features(vdev->guest_features_ex);
}
typedef struct {
@@ -882,34 +902,39 @@ static void failover_add_primary(VirtIONet *n, Error **errp)
error_propagate(errp, err);
}
-static void virtio_net_set_features(VirtIODevice *vdev, uint64_t features)
+static void virtio_net_set_features(VirtIODevice *vdev,
+ const uint64_t *in_features)
{
+ uint64_t features[VIRTIO_FEATURES_DWORDS];
VirtIONet *n = VIRTIO_NET(vdev);
Error *err = NULL;
int i;
+ virtio_features_copy(features, in_features);
if (n->mtu_bypass_backend &&
!virtio_has_feature(vdev->backend_features, VIRTIO_NET_F_MTU)) {
- features &= ~(1ULL << VIRTIO_NET_F_MTU);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_MTU);
}
virtio_net_set_multiqueue(n,
- virtio_has_feature(features, VIRTIO_NET_F_RSS) ||
- virtio_has_feature(features, VIRTIO_NET_F_MQ));
+ virtio_has_feature_ex(features,
+ VIRTIO_NET_F_RSS) ||
+ virtio_has_feature_ex(features,
+ VIRTIO_NET_F_MQ));
virtio_net_set_mrg_rx_bufs(n,
- virtio_has_feature(features,
+ virtio_has_feature_ex(features,
VIRTIO_NET_F_MRG_RXBUF),
- virtio_has_feature(features,
+ virtio_has_feature_ex(features,
VIRTIO_F_VERSION_1),
- virtio_has_feature(features,
+ virtio_has_feature_ex(features,
VIRTIO_NET_F_HASH_REPORT));
- n->rsc4_enabled = virtio_has_feature(features, VIRTIO_NET_F_RSC_EXT) &&
- virtio_has_feature(features, VIRTIO_NET_F_GUEST_TSO4);
- n->rsc6_enabled = virtio_has_feature(features, VIRTIO_NET_F_RSC_EXT) &&
- virtio_has_feature(features, VIRTIO_NET_F_GUEST_TSO6);
- n->rss_data.redirect = virtio_has_feature(features, VIRTIO_NET_F_RSS);
+ n->rsc4_enabled = virtio_has_feature_ex(features, VIRTIO_NET_F_RSC_EXT) &&
+ virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_TSO4);
+ n->rsc6_enabled = virtio_has_feature_ex(features, VIRTIO_NET_F_RSC_EXT) &&
+ virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_TSO6);
+ n->rss_data.redirect = virtio_has_feature_ex(features, VIRTIO_NET_F_RSS);
if (n->has_vnet_hdr) {
n->curr_guest_offloads =
@@ -923,7 +948,7 @@ static void virtio_net_set_features(VirtIODevice *vdev, uint64_t features)
if (!get_vhost_net(nc->peer)) {
continue;
}
- vhost_net_ack_features(get_vhost_net(nc->peer), features);
+ vhost_net_ack_features_ex(get_vhost_net(nc->peer), features);
/*
* keep acked_features in NetVhostUserState up-to-date so it
@@ -932,11 +957,11 @@ static void virtio_net_set_features(VirtIODevice *vdev, uint64_t features)
vhost_net_save_acked_features(nc->peer);
}
- if (!virtio_has_feature(features, VIRTIO_NET_F_CTRL_VLAN)) {
+ if (!virtio_has_feature_ex(features, VIRTIO_NET_F_CTRL_VLAN)) {
memset(n->vlans, 0xff, MAX_VLAN >> 3);
}
- if (virtio_has_feature(features, VIRTIO_NET_F_STANDBY)) {
+ if (virtio_has_feature_ex(features, VIRTIO_NET_F_STANDBY)) {
qapi_event_send_failover_negotiated(n->netclient_name);
qatomic_set(&n->failover_primary_hidden, false);
failover_add_primary(n, &err);
@@ -1901,10 +1926,10 @@ static ssize_t virtio_net_receive_rcu(NetClientState *nc, const uint8_t *buf,
virtio_error(vdev, "virtio-net unexpected empty queue: "
"i %zd mergeable %d offset %zd, size %zd, "
"guest hdr len %zd, host hdr len %zd "
- "guest features 0x%" PRIx64,
+ "guest features 0x" VIRTIO_FEATURES_FMT,
i, n->mergeable_rx_bufs, offset, size,
n->guest_hdr_len, n->host_hdr_len,
- vdev->guest_features);
+ VIRTIO_FEATURES_PR(vdev->guest_features_ex));
}
err = -1;
goto err;
@@ -3011,8 +3036,8 @@ static int virtio_net_pre_load_queues(VirtIODevice *vdev, uint32_t n)
return 0;
}
-static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
- Error **errp)
+static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
+ Error **errp)
{
VirtIONet *n = VIRTIO_NET(vdev);
NetClientState *nc = qemu_get_queue(n->nic);
@@ -3026,68 +3051,67 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
(supported_hash_types & peer_hash_types) == supported_hash_types;
/* Firstly sync all virtio-net possible supported features */
- features |= n->host_features;
+ virtio_features_or(features, features, n->host_features_ex);
- virtio_add_feature(&features, VIRTIO_NET_F_MAC);
+ virtio_add_feature_ex(features, VIRTIO_NET_F_MAC);
if (!peer_has_vnet_hdr(n)) {
- virtio_clear_feature(&features, VIRTIO_NET_F_CSUM);
- virtio_clear_feature(&features, VIRTIO_NET_F_HOST_TSO4);
- virtio_clear_feature(&features, VIRTIO_NET_F_HOST_TSO6);
- virtio_clear_feature(&features, VIRTIO_NET_F_HOST_ECN);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_CSUM);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_TSO4);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_TSO6);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_ECN);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_CSUM);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_TSO4);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_TSO6);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_ECN);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_CSUM);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_TSO4);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_TSO6);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_ECN);
- virtio_clear_feature(&features, VIRTIO_NET_F_HOST_USO);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_USO4);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_USO6);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_USO);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
- virtio_clear_feature(&features, VIRTIO_NET_F_HASH_REPORT);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
}
if (!peer_has_vnet_hdr(n) || !peer_has_ufo(n)) {
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_UFO);
- virtio_clear_feature(&features, VIRTIO_NET_F_HOST_UFO);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UFO);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UFO);
}
-
if (!peer_has_uso(n)) {
- virtio_clear_feature(&features, VIRTIO_NET_F_HOST_USO);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_USO4);
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_USO6);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_USO);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
}
if (!get_vhost_net(nc->peer)) {
if (!use_own_hash) {
- virtio_clear_feature(&features, VIRTIO_NET_F_HASH_REPORT);
- virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
- } else if (virtio_has_feature(features, VIRTIO_NET_F_RSS)) {
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_RSS);
+ } else if (virtio_has_feature_ex(features, VIRTIO_NET_F_RSS)) {
virtio_net_load_ebpf(n, errp);
}
- return features;
+ return;
}
if (!use_peer_hash) {
- virtio_clear_feature(&features, VIRTIO_NET_F_HASH_REPORT);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
if (!use_own_hash || !virtio_net_attach_ebpf_to_backend(n->nic, -1)) {
if (!virtio_net_load_ebpf(n, errp)) {
- return features;
+ return;
}
- virtio_clear_feature(&features, VIRTIO_NET_F_RSS);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_RSS);
}
}
- features = vhost_net_get_features(get_vhost_net(nc->peer), features);
- vdev->backend_features = features;
+ vhost_net_get_features_ex(get_vhost_net(nc->peer), features);
+ virtio_features_copy(vdev->backend_features_ex, features);
if (n->mtu_bypass_backend &&
(n->host_features & 1ULL << VIRTIO_NET_F_MTU)) {
- features |= (1ULL << VIRTIO_NET_F_MTU);
+ virtio_add_feature_ex(features, VIRTIO_NET_F_MTU);
}
/*
@@ -3102,10 +3126,8 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
* support it.
*/
if (!virtio_has_feature(vdev->backend_features, VIRTIO_NET_F_CTRL_VQ)) {
- virtio_clear_feature(&features, VIRTIO_NET_F_GUEST_ANNOUNCE);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_ANNOUNCE);
}
-
- return features;
}
static int virtio_net_post_load_device(void *opaque, int version_id)
@@ -4237,8 +4259,8 @@ static void virtio_net_class_init(ObjectClass *klass, const void *data)
vdc->unrealize = virtio_net_device_unrealize;
vdc->get_config = virtio_net_get_config;
vdc->set_config = virtio_net_set_config;
- vdc->get_features = virtio_net_get_features;
- vdc->set_features = virtio_net_set_features;
+ vdc->get_features_ex = virtio_net_get_features;
+ vdc->set_features_ex = virtio_net_set_features;
vdc->bad_features = virtio_net_bad_features;
vdc->reset = virtio_net_reset;
vdc->queue_reset = virtio_net_queue_reset;
diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h
index 73fdefc0dc..5b8ab7bda7 100644
--- a/include/hw/virtio/virtio-net.h
+++ b/include/hw/virtio/virtio-net.h
@@ -182,7 +182,7 @@ struct VirtIONet {
uint32_t has_vnet_hdr;
size_t host_hdr_len;
size_t guest_hdr_len;
- uint64_t host_features;
+ VIRTIO_DECLARE_FEATURES(host_features);
uint32_t rsc_timeout;
uint8_t rsc4_enabled;
uint8_t rsc6_enabled;
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (10 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 11/13] virtio-net: " Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 11:17 ` Stefano Garzarella
` (2 more replies)
2025-07-18 8:52 ` [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading Paolo Abeni
12 siblings, 3 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
Tap devices support GSO over UDP tunnel offload. Probe for such
feature in a similar manner to other offloads.
GSO over UDP tunnel needs to be enabled in addition to a "plain"
offload (TSO or USO).
No need to check separately for the outer header checksum offload:
the kernel is going to support both of them or none.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- use bool type for tap_probe_has_tunnel()
- rebased on top of 2deec9ab7d ("virtio-net: Move
virtio_net_get_features() down")
- _array -> _ex
v1 -> v2:
- peer_has_tunnel return a bool
- move TUN_F definition in net/tun-linux.h
---
hw/net/virtio-net.c | 37 +++++++++++++++++++++++++++++++++++++
include/net/net.h | 3 +++
net/net.c | 9 +++++++++
net/tap-bsd.c | 5 +++++
net/tap-linux.c | 11 +++++++++++
net/tap-linux.h | 9 +++++++++
net/tap-solaris.c | 5 +++++
net/tap-stub.c | 5 +++++
net/tap.c | 11 +++++++++++
net/tap_int.h | 1 +
10 files changed, 96 insertions(+)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 53413ec4d5..70c85f7f77 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -649,6 +649,15 @@ static int peer_has_uso(VirtIONet *n)
return qemu_has_uso(qemu_get_queue(n->nic)->peer);
}
+static bool peer_has_tunnel(VirtIONet *n)
+{
+ if (!peer_has_vnet_hdr(n)) {
+ return false;
+ }
+
+ return qemu_has_tunnel(qemu_get_queue(n->nic)->peer);
+}
+
static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
int version_1, int hash_report)
{
@@ -3070,6 +3079,13 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
+ virtio_clear_feature_ex(features,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
+ virtio_clear_feature_ex(features,
+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
+
virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
}
@@ -3083,6 +3099,15 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
}
+ if (!peer_has_tunnel(n)) {
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
+ virtio_clear_feature_ex(features,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
+ virtio_clear_feature_ex(features,
+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
+ }
+
if (!get_vhost_net(nc->peer)) {
if (!use_own_hash) {
virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
@@ -4137,6 +4162,10 @@ static const VMStateDescription vmstate_virtio_net = {
.dev_unplug_pending = dev_unplug_pending,
};
+#define DEFINE_PROP_FEATURE(_name, _state, _field, _bit, _defval) \
+ DEFINE_PROP_BIT64(_name, _state, _field[VIRTIO_DWORD(_bit)], \
+ (_bit) % 64, _defval)
+
static const Property virtio_net_properties[] = {
DEFINE_PROP_BIT64("csum", VirtIONet, host_features,
VIRTIO_NET_F_CSUM, true),
@@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
rss_data.specified_hash_types,
VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
ON_OFF_AUTO_AUTO),
+ DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
+ DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
+ DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
+ DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
};
static void virtio_net_class_init(ObjectClass *klass, const void *data)
diff --git a/include/net/net.h b/include/net/net.h
index 48ba333d02..9a9084690d 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -65,6 +65,7 @@ typedef void (NetClientDestructor)(NetClientState *);
typedef RxFilterInfo *(QueryRxFilter)(NetClientState *);
typedef bool (HasUfo)(NetClientState *);
typedef bool (HasUso)(NetClientState *);
+typedef bool (HasTunnel)(NetClientState *);
typedef bool (HasVnetHdr)(NetClientState *);
typedef bool (HasVnetHdrLen)(NetClientState *, int);
typedef void (SetOffload)(NetClientState *, const NetOffloads *);
@@ -95,6 +96,7 @@ typedef struct NetClientInfo {
NetPoll *poll;
HasUfo *has_ufo;
HasUso *has_uso;
+ HasTunnel *has_tunnel;
HasVnetHdr *has_vnet_hdr;
HasVnetHdrLen *has_vnet_hdr_len;
SetOffload *set_offload;
@@ -197,6 +199,7 @@ void qemu_set_info_str(NetClientState *nc,
void qemu_format_nic_info_str(NetClientState *nc, uint8_t macaddr[6]);
bool qemu_has_ufo(NetClientState *nc);
bool qemu_has_uso(NetClientState *nc);
+bool qemu_has_tunnel(NetClientState *nc);
bool qemu_has_vnet_hdr(NetClientState *nc);
bool qemu_has_vnet_hdr_len(NetClientState *nc, int len);
void qemu_set_offload(NetClientState *nc, const NetOffloads *ol);
diff --git a/net/net.c b/net/net.c
index 63872b6855..9536184a0c 100644
--- a/net/net.c
+++ b/net/net.c
@@ -522,6 +522,15 @@ bool qemu_has_uso(NetClientState *nc)
return nc->info->has_uso(nc);
}
+bool qemu_has_tunnel(NetClientState *nc)
+{
+ if (!nc || !nc->info->has_tunnel) {
+ return false;
+ }
+
+ return nc->info->has_tunnel(nc);
+}
+
bool qemu_has_vnet_hdr(NetClientState *nc)
{
if (!nc || !nc->info->has_vnet_hdr) {
diff --git a/net/tap-bsd.c b/net/tap-bsd.c
index 86b6edee94..751d4c819c 100644
--- a/net/tap-bsd.c
+++ b/net/tap-bsd.c
@@ -217,6 +217,11 @@ int tap_probe_has_uso(int fd)
return 0;
}
+bool tap_probe_has_tunnel(int fd)
+{
+ return false;
+}
+
void tap_fd_set_vnet_hdr_len(int fd, int len)
{
}
diff --git a/net/tap-linux.c b/net/tap-linux.c
index a1c58f74f5..e2628be798 100644
--- a/net/tap-linux.c
+++ b/net/tap-linux.c
@@ -196,6 +196,17 @@ int tap_probe_has_uso(int fd)
return 1;
}
+bool tap_probe_has_tunnel(int fd)
+{
+ unsigned offload;
+
+ offload = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_UDP_TUNNEL_GSO;
+ if (ioctl(fd, TUNSETOFFLOAD, offload) < 0) {
+ return false;
+ }
+ return true;
+}
+
void tap_fd_set_vnet_hdr_len(int fd, int len)
{
if (ioctl(fd, TUNSETVNETHDRSZ, &len) == -1) {
diff --git a/net/tap-linux.h b/net/tap-linux.h
index 9a58cecb7f..8cd6b5874b 100644
--- a/net/tap-linux.h
+++ b/net/tap-linux.h
@@ -53,4 +53,13 @@
#define TUN_F_USO4 0x20 /* I can handle USO for IPv4 packets */
#define TUN_F_USO6 0x40 /* I can handle USO for IPv6 packets */
+/* I can handle TSO/USO for UDP tunneled packets */
+#define TUN_F_UDP_TUNNEL_GSO 0x080
+
+/*
+ * I can handle TSO/USO for UDP tunneled packets requiring csum offload for
+ * the outer header
+ */
+#define TUN_F_UDP_TUNNEL_GSO_CSUM 0x100
+
#endif /* QEMU_TAP_LINUX_H */
diff --git a/net/tap-solaris.c b/net/tap-solaris.c
index 833c066bee..ac1ae25761 100644
--- a/net/tap-solaris.c
+++ b/net/tap-solaris.c
@@ -222,6 +222,11 @@ int tap_probe_has_uso(int fd)
return 0;
}
+bool tap_probe_has_tunnel(int fd)
+{
+ return false;
+}
+
void tap_fd_set_vnet_hdr_len(int fd, int len)
{
}
diff --git a/net/tap-stub.c b/net/tap-stub.c
index 67d14ad4d5..f7a5e0c163 100644
--- a/net/tap-stub.c
+++ b/net/tap-stub.c
@@ -52,6 +52,11 @@ int tap_probe_has_uso(int fd)
return 0;
}
+bool tap_probe_has_tunnel(int fd)
+{
+ return false;
+}
+
void tap_fd_set_vnet_hdr_len(int fd, int len)
{
}
diff --git a/net/tap.c b/net/tap.c
index b49db19f83..23c6c118e7 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -76,6 +76,7 @@ typedef struct TAPState {
bool using_vnet_hdr;
bool has_ufo;
bool has_uso;
+ bool has_tunnel;
bool enabled;
VHostNetState *vhost_net;
unsigned host_vnet_hdr_len;
@@ -241,6 +242,14 @@ static bool tap_has_uso(NetClientState *nc)
return s->has_uso;
}
+static bool tap_has_tunnel(NetClientState *nc)
+{
+ TAPState *s = DO_UPCAST(TAPState, nc, nc);
+
+ assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
+ return s->has_tunnel;
+}
+
static bool tap_has_vnet_hdr(NetClientState *nc)
{
TAPState *s = DO_UPCAST(TAPState, nc, nc);
@@ -369,6 +378,7 @@ static NetClientInfo net_tap_info = {
.cleanup = tap_cleanup,
.has_ufo = tap_has_ufo,
.has_uso = tap_has_uso,
+ .has_tunnel = tap_has_tunnel,
.has_vnet_hdr = tap_has_vnet_hdr,
.has_vnet_hdr_len = tap_has_vnet_hdr_len,
.set_offload = tap_set_offload,
@@ -398,6 +408,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
s->using_vnet_hdr = false;
s->has_ufo = tap_probe_has_ufo(s->fd);
s->has_uso = tap_probe_has_uso(s->fd);
+ s->has_tunnel = tap_probe_has_tunnel(s->fd);
s->enabled = true;
tap_set_offload(&s->nc, &ol);
/*
diff --git a/net/tap_int.h b/net/tap_int.h
index f8bbe1cb0c..b76a05044b 100644
--- a/net/tap_int.h
+++ b/net/tap_int.h
@@ -38,6 +38,7 @@ void tap_set_sndbuf(int fd, const NetdevTapOptions *tap, Error **errp);
int tap_probe_vnet_hdr(int fd, Error **errp);
int tap_probe_has_ufo(int fd);
int tap_probe_has_uso(int fd);
+bool tap_probe_has_tunnel(int fd);
void tap_fd_set_offload(int fd, const NetOffloads *ol);
void tap_fd_set_vnet_hdr_len(int fd, int len);
int tap_fd_set_vnet_le(int fd, int vnet_is_le);
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
` (11 preceding siblings ...)
2025-07-18 8:52 ` [PATCH RFC v3 12/13] net: implement tunnel probing Paolo Abeni
@ 2025-07-18 8:52 ` Paolo Abeni
2025-07-18 13:22 ` Stefano Garzarella
2025-07-18 15:21 ` Stefano Garzarella
12 siblings, 2 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 8:52 UTC (permalink / raw)
To: qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki, Jason Wang,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
When any host or guest GSO over UDP tunnel offload is enabled the
virtio net header includes the additional tunnel-related fields,
update the size accordingly.
Push the GSO over UDP tunnel offloads all the way down to the tap
device extending the newly introduced NetFeatures struct, and
eventually enable the associated features.
As per virtio specification, to convert features bit to offload bit,
map the extended features into the reserved range.
Finally, make the vhost backend aware of the exact header layout, to
copy it correctly. The tunnel-related field are present if either
the guest or the host negotiated any UDP tunnel related feature:
add them to the kernel supported features list, to allow qemu
transfer to the backend the needed information.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2 -> v3:
- rebased on top of "net: Consolidate vhost feature bits into vhost_net
structure"
- _array -> _ex
v1 -> v2:
- squashed vhost support into this patch
- dropped tun offload consistency checks; they are implemented in
the kernel side
- virtio_has_tnl_hdr ->virtio_has_tunnel_hdr
---
hw/net/virtio-net.c | 34 ++++++++++++++++++++++++++--------
include/net/net.h | 2 ++
net/net.c | 3 ++-
net/tap-linux.c | 6 ++++++
net/tap.c | 2 ++
5 files changed, 38 insertions(+), 9 deletions(-)
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 70c85f7f77..46f7efac7d 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -103,6 +103,12 @@
#define VIRTIO_NET_F2O_SHIFT (VIRTIO_NET_OFFLOAD_MAP_MIN - \
VIRTIO_NET_FEATURES_MAP_MIN + 64)
+static bool virtio_has_tunnel_hdr(const uint64_t *features)
+{
+ return virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO) |
+ virtio_has_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
+}
+
static const VirtIOFeature feature_sizes[] = {
{.flags = 1ULL << VIRTIO_NET_F_MAC,
.end = endof(struct virtio_net_config, mac)},
@@ -659,7 +665,8 @@ static bool peer_has_tunnel(VirtIONet *n)
}
static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
- int version_1, int hash_report)
+ int version_1, int hash_report,
+ int tunnel)
{
int i;
NetClientState *nc;
@@ -667,9 +674,11 @@ static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
n->mergeable_rx_bufs = mergeable_rx_bufs;
if (version_1) {
- n->guest_hdr_len = hash_report ?
- sizeof(struct virtio_net_hdr_v1_hash) :
- sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ n->guest_hdr_len = tunnel ?
+ sizeof(struct virtio_net_hdr_v1_hash_tunnel) :
+ (hash_report ?
+ sizeof(struct virtio_net_hdr_v1_hash) :
+ sizeof(struct virtio_net_hdr_mrg_rxbuf));
n->rss_data.populate_hash = !!hash_report;
} else {
n->guest_hdr_len = n->mergeable_rx_bufs ?
@@ -803,6 +812,10 @@ static void virtio_net_apply_guest_offloads(VirtIONet *n)
.ufo = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
.uso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
.uso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)),
+ .tnl = !!(n->curr_guest_offloads &
+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED)),
+ .tnl_csum = !!(n->curr_guest_offloads &
+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED)),
};
qemu_set_offload(qemu_get_queue(n->nic)->peer, &ol);
@@ -824,7 +837,9 @@ virtio_net_guest_offloads_by_features(const uint64_t *features)
(1ULL << VIRTIO_NET_F_GUEST_ECN) |
(1ULL << VIRTIO_NET_F_GUEST_UFO) |
(1ULL << VIRTIO_NET_F_GUEST_USO4) |
- (1ULL << VIRTIO_NET_F_GUEST_USO6);
+ (1ULL << VIRTIO_NET_F_GUEST_USO6) |
+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED) |
+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED);
return guest_offloads_mask & virtio_net_features_to_offload(features);
}
@@ -937,7 +952,8 @@ static void virtio_net_set_features(VirtIODevice *vdev,
virtio_has_feature_ex(features,
VIRTIO_F_VERSION_1),
virtio_has_feature_ex(features,
- VIRTIO_NET_F_HASH_REPORT));
+ VIRTIO_NET_F_HASH_REPORT),
+ virtio_has_tunnel_hdr(features));
n->rsc4_enabled = virtio_has_feature_ex(features, VIRTIO_NET_F_RSC_EXT) &&
virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_TSO4);
@@ -3160,13 +3176,15 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
VirtIONet *n = opaque;
VirtIODevice *vdev = VIRTIO_DEVICE(n);
int i, link_down;
+ bool has_tunnel_hdr = virtio_has_tunnel_hdr(vdev->guest_features_ex);
trace_virtio_net_post_load_device();
virtio_net_set_mrg_rx_bufs(n, n->mergeable_rx_bufs,
virtio_vdev_has_feature(vdev,
VIRTIO_F_VERSION_1),
virtio_vdev_has_feature(vdev,
- VIRTIO_NET_F_HASH_REPORT));
+ VIRTIO_NET_F_HASH_REPORT),
+ has_tunnel_hdr);
/* MAC_TABLE_ENTRIES may be different from the saved image */
if (n->mac_table.in_use > MAC_TABLE_ENTRIES) {
@@ -3986,7 +4004,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
n->vqs[0].tx_waiting = 0;
n->tx_burst = n->net_conf.txburst;
- virtio_net_set_mrg_rx_bufs(n, 0, 0, 0);
+ virtio_net_set_mrg_rx_bufs(n, 0, 0, 0, 0);
n->promisc = 1; /* for compatibility */
n->mac_table.macs = g_malloc0(MAC_TABLE_ENTRIES * ETH_ALEN);
diff --git a/include/net/net.h b/include/net/net.h
index 9a9084690d..72b476ee1d 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -43,6 +43,8 @@ typedef struct NetOffloads {
bool ufo;
bool uso4;
bool uso6;
+ bool tnl;
+ bool tnl_csum;
} NetOffloads;
#define DEFINE_NIC_PROPERTIES(_state, _conf) \
diff --git a/net/net.c b/net/net.c
index 9536184a0c..27e0d27807 100644
--- a/net/net.c
+++ b/net/net.c
@@ -575,7 +575,8 @@ void qemu_set_vnet_hdr_len(NetClientState *nc, int len)
assert(len == sizeof(struct virtio_net_hdr_mrg_rxbuf) ||
len == sizeof(struct virtio_net_hdr) ||
- len == sizeof(struct virtio_net_hdr_v1_hash));
+ len == sizeof(struct virtio_net_hdr_v1_hash) ||
+ len == sizeof(struct virtio_net_hdr_v1_hash_tunnel));
nc->vnet_hdr_len = len;
nc->info->set_vnet_hdr_len(nc, len);
diff --git a/net/tap-linux.c b/net/tap-linux.c
index e2628be798..8e275d2ea4 100644
--- a/net/tap-linux.c
+++ b/net/tap-linux.c
@@ -279,6 +279,12 @@ void tap_fd_set_offload(int fd, const NetOffloads *ol)
if (ol->uso6) {
offload |= TUN_F_USO6;
}
+ if (ol->tnl) {
+ offload |= TUN_F_UDP_TUNNEL_GSO;
+ }
+ if (ol->tnl_csum) {
+ offload |= TUN_F_UDP_TUNNEL_GSO_CSUM;
+ }
}
if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) {
diff --git a/net/tap.c b/net/tap.c
index 23c6c118e7..2dfa843547 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -62,6 +62,8 @@ static const int kernel_feature_bits[] = {
VIRTIO_F_NOTIFICATION_DATA,
VIRTIO_NET_F_RSC_EXT,
VIRTIO_NET_F_HASH_REPORT,
+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO,
VHOST_INVALID_FEATURE_BIT
};
--
2.50.0
^ permalink raw reply related [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-18 8:52 ` [PATCH RFC v3 08/13] qmp: update virtio features map to support " Paolo Abeni
@ 2025-07-18 10:18 ` Stefano Garzarella
2025-07-18 10:23 ` Paolo Abeni
2025-07-19 6:57 ` Markus Armbruster
2025-07-21 7:23 ` Akihiko Odaki
2 siblings, 1 reply; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 10:18 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:34AM +0200, Paolo Abeni wrote:
>Extend the VirtioDeviceFeatures struct with an additional u64
>to track unknown features in the 64-127 bit range and decode
>the full virtio features spaces for vhost and virtio devices.
>
>Also add entries for the soon-to-be-supported virtio net GSO over
>UDP features.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - unknown-dev-features-dword2 -> unknown-dev-features2
> - _array -> _ex
> - fixed typos in entries description
>
>v1 -> v2:
> - uint128_t -> uint64_t[]
>---
> hw/virtio/virtio-hmp-cmds.c | 3 +-
> hw/virtio/virtio-qmp.c | 89 ++++++++++++++++++++++++++-----------
> hw/virtio/virtio-qmp.h | 3 +-
> qapi/virtio.json | 8 +++-
> 4 files changed, 73 insertions(+), 30 deletions(-)
>
>diff --git a/hw/virtio/virtio-hmp-cmds.c b/hw/virtio/virtio-hmp-cmds.c
>index 7d8677bcf0..1daae482d3 100644
>--- a/hw/virtio/virtio-hmp-cmds.c
>+++ b/hw/virtio/virtio-hmp-cmds.c
>@@ -74,7 +74,8 @@ static void hmp_virtio_dump_features(Monitor *mon,
> }
>
> if (features->has_unknown_dev_features) {
>- monitor_printf(mon, " unknown-features(0x%016"PRIx64")\n",
>+ monitor_printf(mon, " unknown-features(0x%016"PRIx64"%016"PRIx64")\n",
>+ features->unknown_dev_features2,
> features->unknown_dev_features);
> }
> }
>diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
>index 3b6377cf0d..03c6163cf4 100644
>--- a/hw/virtio/virtio-qmp.c
>+++ b/hw/virtio/virtio-qmp.c
>@@ -325,6 +325,20 @@ static const qmp_virtio_feature_map_t virtio_net_feature_map[] = {
> FEATURE_ENTRY(VHOST_USER_F_PROTOCOL_FEATURES, \
> "VHOST_USER_F_PROTOCOL_FEATURES: Vhost-user protocol features "
> "negotiation supported"),
>+ FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, \
>+ "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>+ "UDP tunnel packets"),
>+ FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, \
>+ "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>+ "UDP tunnel packets requiring checksum offload for the outer "
>+ "header"),
>+ FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, \
>+ "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>+ "UDP tunnel packets"),
>+ FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, \
>+ "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>+ "UDP tunnel packets requiring checksum offload for the outer "
>+ "header"),
Is this chunk supposed to be here in this patch or better in the last
patches where you add UPD tunnel features support?
Thanks,
Stefano
> { -1, "" }
> };
> #endif
>@@ -510,6 +524,24 @@ static const qmp_virtio_feature_map_t virtio_gpio_feature_map[] = {
> list; \
> })
>
>+#define CONVERT_FEATURES_EX(type, map, bitmap) \
>+ ({ \
>+ type *list = NULL; \
>+ type *node; \
>+ for (i = 0; map[i].virtio_bit != -1; i++) { \
>+ bit = map[i].virtio_bit; \
>+ if (!virtio_has_feature_ex(bitmap, bit)) { \
>+ continue; \
>+ } \
>+ node = g_new0(type, 1); \
>+ node->value = g_strdup(map[i].feature_desc); \
>+ node->next = list; \
>+ list = node; \
>+ virtio_clear_feature_ex(bitmap, bit); \
>+ } \
>+ list; \
>+ })
>+
> VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap)
> {
> VirtioDeviceStatus *status;
>@@ -545,109 +577,112 @@ VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap)
> return vhu_protocols;
> }
>
>-VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
>+VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
>+ const uint64_t *bmap)
> {
>+ uint64_t bitmap[VIRTIO_FEATURES_DWORDS];
> VirtioDeviceFeatures *features;
> uint64_t bit;
> int i;
>
>+ virtio_features_copy(bitmap, bmap);
> features = g_new0(VirtioDeviceFeatures, 1);
> features->has_dev_features = true;
>
> /* transport features */
>- features->transports = CONVERT_FEATURES(strList, virtio_transport_map, 0,
>- bitmap);
>+ features->transports = CONVERT_FEATURES_EX(strList, virtio_transport_map,
>+ bitmap);
>
> /* device features */
> switch (device_id) {
> #ifdef CONFIG_VIRTIO_SERIAL
> case VIRTIO_ID_CONSOLE:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_serial_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_serial_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_BLK
> case VIRTIO_ID_BLOCK:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_blk_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_blk_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_GPU
> case VIRTIO_ID_GPU:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_gpu_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_gpu_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_NET
> case VIRTIO_ID_NET:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_net_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_net_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_SCSI
> case VIRTIO_ID_SCSI:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_scsi_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_scsi_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_BALLOON
> case VIRTIO_ID_BALLOON:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_balloon_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_balloon_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_IOMMU
> case VIRTIO_ID_IOMMU:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_iommu_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_iommu_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_INPUT
> case VIRTIO_ID_INPUT:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_input_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_input_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VHOST_USER_FS
> case VIRTIO_ID_FS:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_fs_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_fs_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VHOST_VSOCK
> case VIRTIO_ID_VSOCK:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_vsock_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_vsock_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_CRYPTO
> case VIRTIO_ID_CRYPTO:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_crypto_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_crypto_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_MEM
> case VIRTIO_ID_MEM:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_mem_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_mem_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_I2C_ADAPTER
> case VIRTIO_ID_I2C_ADAPTER:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_i2c_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_i2c_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_RNG
> case VIRTIO_ID_RNG:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_rng_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_rng_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VHOST_USER_GPIO
> case VIRTIO_ID_GPIO:
> features->dev_features =
>- CONVERT_FEATURES(strList, virtio_gpio_feature_map, 0, bitmap);
>+ CONVERT_FEATURES_EX(strList, virtio_gpio_feature_map, bitmap);
> break;
> #endif
> /* No features */
>@@ -680,9 +715,10 @@ VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
> g_assert_not_reached();
> }
>
>- features->has_unknown_dev_features = bitmap != 0;
>+ features->has_unknown_dev_features = !virtio_features_empty(bitmap);
> if (features->has_unknown_dev_features) {
>- features->unknown_dev_features = bitmap;
>+ features->unknown_dev_features = bitmap[0];
>+ features->unknown_dev_features2 = bitmap[1];
> }
>
> return features;
>@@ -743,11 +779,11 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
> status->device_id = vdev->device_id;
> status->vhost_started = vdev->vhost_started;
> status->guest_features = qmp_decode_features(vdev->device_id,
>- vdev->guest_features);
>+ vdev->guest_features_ex);
> status->host_features = qmp_decode_features(vdev->device_id,
>- vdev->host_features);
>+ vdev->host_features_ex);
> status->backend_features = qmp_decode_features(vdev->device_id,
>- vdev->backend_features);
>+ vdev->backend_features_ex);
>
> switch (vdev->device_endian) {
> case VIRTIO_DEVICE_ENDIAN_LITTLE:
>@@ -785,11 +821,12 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
> status->vhost_dev->nvqs = hdev->nvqs;
> status->vhost_dev->vq_index = hdev->vq_index;
> status->vhost_dev->features =
>- qmp_decode_features(vdev->device_id, hdev->features);
>+ qmp_decode_features(vdev->device_id, hdev->features_ex);
> status->vhost_dev->acked_features =
>- qmp_decode_features(vdev->device_id, hdev->acked_features);
>+ qmp_decode_features(vdev->device_id, hdev->acked_features_ex);
> status->vhost_dev->backend_features =
>- qmp_decode_features(vdev->device_id, hdev->backend_features);
>+ qmp_decode_features(vdev->device_id, hdev->backend_features_ex);
>+
> status->vhost_dev->protocol_features =
> qmp_decode_protocols(hdev->protocol_features);
> status->vhost_dev->max_queues = hdev->max_queues;
>diff --git a/hw/virtio/virtio-qmp.h b/hw/virtio/virtio-qmp.h
>index 245a446a56..e0a1e49035 100644
>--- a/hw/virtio/virtio-qmp.h
>+++ b/hw/virtio/virtio-qmp.h
>@@ -18,6 +18,7 @@
> VirtIODevice *qmp_find_virtio_device(const char *path);
> VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap);
> VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap);
>-VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap);
>+VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
>+ const uint64_t *bitmap);
>
> #endif
>diff --git a/qapi/virtio.json b/qapi/virtio.json
>index 9d652fe4a8..f2e2dd6e97 100644
>--- a/qapi/virtio.json
>+++ b/qapi/virtio.json
>@@ -490,14 +490,18 @@
> # unique features)
> #
> # @unknown-dev-features: Virtio device features bitmap that have not
>-# been decoded
>+# been decoded (bits 0-63)
>+#
>+# @unknown-dev-features2: Virtio device features bitmap that have not
>+# been decoded (bits 64-127)
> #
> # Since: 7.2
> ##
> { 'struct': 'VirtioDeviceFeatures',
> 'data': { 'transports': [ 'str' ],
> '*dev-features': [ 'str' ],
>- '*unknown-dev-features': 'uint64' } }
>+ '*unknown-dev-features': 'uint64',
>+ '*unknown-dev-features2': 'uint64' } }
>
> ##
> # @VirtQueueStatus:
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-18 10:18 ` Stefano Garzarella
@ 2025-07-18 10:23 ` Paolo Abeni
2025-07-18 10:28 ` Stefano Garzarella
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 10:23 UTC (permalink / raw)
To: Stefano Garzarella
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/18/25 12:18 PM, Stefano Garzarella wrote:
> On Fri, Jul 18, 2025 at 10:52:34AM +0200, Paolo Abeni wrote:
>> diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
>> index 3b6377cf0d..03c6163cf4 100644
>> --- a/hw/virtio/virtio-qmp.c
>> +++ b/hw/virtio/virtio-qmp.c
>> @@ -325,6 +325,20 @@ static const qmp_virtio_feature_map_t virtio_net_feature_map[] = {
>> FEATURE_ENTRY(VHOST_USER_F_PROTOCOL_FEATURES, \
>> "VHOST_USER_F_PROTOCOL_FEATURES: Vhost-user protocol features "
>> "negotiation supported"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, \
>> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>> + "UDP tunnel packets"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, \
>> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>> + "UDP tunnel packets requiring checksum offload for the outer "
>> + "header"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, \
>> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>> + "UDP tunnel packets"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, \
>> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>> + "UDP tunnel packets requiring checksum offload for the outer "
>> + "header"),
>
> Is this chunk supposed to be here in this patch or better in the last
> patches where you add UPD tunnel features support?
Note that this point was discussed in the previous iteration:
https://lists.gnu.org/archive/html/qemu-devel/2025-07/msg04418.html
I think it could be both way: it could be useful for QMP to be able to
dump the (kernel/backend) features in human readable format even if qemu
does not support them.
No big objection to move the above chunk in patch 13/13, if there is
agreement :)
/P
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-18 10:23 ` Paolo Abeni
@ 2025-07-18 10:28 ` Stefano Garzarella
0 siblings, 0 replies; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 10:28 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 12:23:21PM +0200, Paolo Abeni wrote:
>On 7/18/25 12:18 PM, Stefano Garzarella wrote:
>> On Fri, Jul 18, 2025 at 10:52:34AM +0200, Paolo Abeni wrote:
>>> diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
>>> index 3b6377cf0d..03c6163cf4 100644
>>> --- a/hw/virtio/virtio-qmp.c
>>> +++ b/hw/virtio/virtio-qmp.c
>>> @@ -325,6 +325,20 @@ static const qmp_virtio_feature_map_t virtio_net_feature_map[] = {
>>> FEATURE_ENTRY(VHOST_USER_F_PROTOCOL_FEATURES, \
>>> "VHOST_USER_F_PROTOCOL_FEATURES: Vhost-user protocol features "
>>> "negotiation supported"),
>>> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, \
>>> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>>> + "UDP tunnel packets"),
>>> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, \
>>> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>>> + "UDP tunnel packets requiring checksum offload for the outer "
>>> + "header"),
>>> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, \
>>> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>>> + "UDP tunnel packets"),
>>> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, \
>>> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>>> + "UDP tunnel packets requiring checksum offload for the outer "
>>> + "header"),
>>
>> Is this chunk supposed to be here in this patch or better in the last
>> patches where you add UPD tunnel features support?
>
>Note that this point was discussed in the previous iteration:
>
>https://lists.gnu.org/archive/html/qemu-devel/2025-07/msg04418.html
Ops, sorry, I skiped that version and came directly here.
I'll check the v3.
>
>I think it could be both way: it could be useful for QMP to be able to
>dump the (kernel/backend) features in human readable format even if qemu
>does not support them.
I see.
>
>No big objection to move the above chunk in patch 13/13, if there is
>agreement :)
Not a strong opinion here.
Thanks,
Stefano
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-18 8:52 ` [PATCH RFC v3 12/13] net: implement tunnel probing Paolo Abeni
@ 2025-07-18 11:17 ` Stefano Garzarella
2025-07-21 8:48 ` Paolo Abeni
2025-07-22 3:50 ` Jason Wang
2025-07-22 4:15 ` Akihiko Odaki
2 siblings, 1 reply; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 11:17 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:38AM +0200, Paolo Abeni wrote:
>Tap devices support GSO over UDP tunnel offload. Probe for such
>feature in a similar manner to other offloads.
>
>GSO over UDP tunnel needs to be enabled in addition to a "plain"
>offload (TSO or USO).
>
>No need to check separately for the outer header checksum offload:
>the kernel is going to support both of them or none.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - use bool type for tap_probe_has_tunnel()
> - rebased on top of 2deec9ab7d ("virtio-net: Move
> virtio_net_get_features() down")
> - _array -> _ex
>
>v1 -> v2:
> - peer_has_tunnel return a bool
> - move TUN_F definition in net/tun-linux.h
>---
> hw/net/virtio-net.c | 37 +++++++++++++++++++++++++++++++++++++
> include/net/net.h | 3 +++
> net/net.c | 9 +++++++++
> net/tap-bsd.c | 5 +++++
> net/tap-linux.c | 11 +++++++++++
> net/tap-linux.h | 9 +++++++++
> net/tap-solaris.c | 5 +++++
> net/tap-stub.c | 5 +++++
> net/tap.c | 11 +++++++++++
> net/tap_int.h | 1 +
> 10 files changed, 96 insertions(+)
>
>diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
>index 53413ec4d5..70c85f7f77 100644
>--- a/hw/net/virtio-net.c
>+++ b/hw/net/virtio-net.c
>@@ -649,6 +649,15 @@ static int peer_has_uso(VirtIONet *n)
> return qemu_has_uso(qemu_get_queue(n->nic)->peer);
> }
>
>+static bool peer_has_tunnel(VirtIONet *n)
>+{
>+ if (!peer_has_vnet_hdr(n)) {
>+ return false;
>+ }
>+
>+ return qemu_has_tunnel(qemu_get_queue(n->nic)->peer);
>+}
>+
> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
> int version_1, int hash_report)
> {
>@@ -3070,6 +3079,13 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
>
>+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
>+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
>+ virtio_clear_feature_ex(features,
>+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
>+ virtio_clear_feature_ex(features,
>+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
>+
> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> }
>
>@@ -3083,6 +3099,15 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
> }
>
>+ if (!peer_has_tunnel(n)) {
>+ virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
>+ virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
>+ virtio_clear_feature_ex(features,
>+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
>+ virtio_clear_feature_ex(features,
>+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
>+ }
>+
> if (!get_vhost_net(nc->peer)) {
> if (!use_own_hash) {
> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
>@@ -4137,6 +4162,10 @@ static const VMStateDescription vmstate_virtio_net = {
> .dev_unplug_pending = dev_unplug_pending,
> };
>
>+#define DEFINE_PROP_FEATURE(_name, _state, _field, _bit, _defval) \
>+ DEFINE_PROP_BIT64(_name, _state, _field[VIRTIO_DWORD(_bit)], \
>+ (_bit) % 64, _defval)
>+
> static const Property virtio_net_properties[] = {
> DEFINE_PROP_BIT64("csum", VirtIONet, host_features,
> VIRTIO_NET_F_CSUM, true),
>@@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
> rss_data.specified_hash_types,
> VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
> ON_OFF_AUTO_AUTO),
>+ DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
>+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
>+ DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
>+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
>+ DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
>+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
>+ DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
>+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
IIUC, to avoid issue with live-migration, we should disable all of them
in `hw_compat_10_0` in hw/core/machine.c (e.g. like `host_uso`,
`guest_uso*` in hw_compat_8_1).
Thanks,
Stefano
> };
>
> static void virtio_net_class_init(ObjectClass *klass, const void *data)
>diff --git a/include/net/net.h b/include/net/net.h
>index 48ba333d02..9a9084690d 100644
>--- a/include/net/net.h
>+++ b/include/net/net.h
>@@ -65,6 +65,7 @@ typedef void (NetClientDestructor)(NetClientState *);
> typedef RxFilterInfo *(QueryRxFilter)(NetClientState *);
> typedef bool (HasUfo)(NetClientState *);
> typedef bool (HasUso)(NetClientState *);
>+typedef bool (HasTunnel)(NetClientState *);
> typedef bool (HasVnetHdr)(NetClientState *);
> typedef bool (HasVnetHdrLen)(NetClientState *, int);
> typedef void (SetOffload)(NetClientState *, const NetOffloads *);
>@@ -95,6 +96,7 @@ typedef struct NetClientInfo {
> NetPoll *poll;
> HasUfo *has_ufo;
> HasUso *has_uso;
>+ HasTunnel *has_tunnel;
> HasVnetHdr *has_vnet_hdr;
> HasVnetHdrLen *has_vnet_hdr_len;
> SetOffload *set_offload;
>@@ -197,6 +199,7 @@ void qemu_set_info_str(NetClientState *nc,
> void qemu_format_nic_info_str(NetClientState *nc, uint8_t macaddr[6]);
> bool qemu_has_ufo(NetClientState *nc);
> bool qemu_has_uso(NetClientState *nc);
>+bool qemu_has_tunnel(NetClientState *nc);
> bool qemu_has_vnet_hdr(NetClientState *nc);
> bool qemu_has_vnet_hdr_len(NetClientState *nc, int len);
> void qemu_set_offload(NetClientState *nc, const NetOffloads *ol);
>diff --git a/net/net.c b/net/net.c
>index 63872b6855..9536184a0c 100644
>--- a/net/net.c
>+++ b/net/net.c
>@@ -522,6 +522,15 @@ bool qemu_has_uso(NetClientState *nc)
> return nc->info->has_uso(nc);
> }
>
>+bool qemu_has_tunnel(NetClientState *nc)
>+{
>+ if (!nc || !nc->info->has_tunnel) {
>+ return false;
>+ }
>+
>+ return nc->info->has_tunnel(nc);
>+}
>+
> bool qemu_has_vnet_hdr(NetClientState *nc)
> {
> if (!nc || !nc->info->has_vnet_hdr) {
>diff --git a/net/tap-bsd.c b/net/tap-bsd.c
>index 86b6edee94..751d4c819c 100644
>--- a/net/tap-bsd.c
>+++ b/net/tap-bsd.c
>@@ -217,6 +217,11 @@ int tap_probe_has_uso(int fd)
> return 0;
> }
>
>+bool tap_probe_has_tunnel(int fd)
>+{
>+ return false;
>+}
>+
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> }
>diff --git a/net/tap-linux.c b/net/tap-linux.c
>index a1c58f74f5..e2628be798 100644
>--- a/net/tap-linux.c
>+++ b/net/tap-linux.c
>@@ -196,6 +196,17 @@ int tap_probe_has_uso(int fd)
> return 1;
> }
>
>+bool tap_probe_has_tunnel(int fd)
>+{
>+ unsigned offload;
>+
>+ offload = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_UDP_TUNNEL_GSO;
>+ if (ioctl(fd, TUNSETOFFLOAD, offload) < 0) {
>+ return false;
>+ }
>+ return true;
>+}
>+
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> if (ioctl(fd, TUNSETVNETHDRSZ, &len) == -1) {
>diff --git a/net/tap-linux.h b/net/tap-linux.h
>index 9a58cecb7f..8cd6b5874b 100644
>--- a/net/tap-linux.h
>+++ b/net/tap-linux.h
>@@ -53,4 +53,13 @@
> #define TUN_F_USO4 0x20 /* I can handle USO for IPv4 packets */
> #define TUN_F_USO6 0x40 /* I can handle USO for IPv6 packets */
>
>+/* I can handle TSO/USO for UDP tunneled packets */
>+#define TUN_F_UDP_TUNNEL_GSO 0x080
>+
>+/*
>+ * I can handle TSO/USO for UDP tunneled packets requiring csum offload for
>+ * the outer header
>+ */
>+#define TUN_F_UDP_TUNNEL_GSO_CSUM 0x100
>+
> #endif /* QEMU_TAP_LINUX_H */
>diff --git a/net/tap-solaris.c b/net/tap-solaris.c
>index 833c066bee..ac1ae25761 100644
>--- a/net/tap-solaris.c
>+++ b/net/tap-solaris.c
>@@ -222,6 +222,11 @@ int tap_probe_has_uso(int fd)
> return 0;
> }
>
>+bool tap_probe_has_tunnel(int fd)
>+{
>+ return false;
>+}
>+
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> }
>diff --git a/net/tap-stub.c b/net/tap-stub.c
>index 67d14ad4d5..f7a5e0c163 100644
>--- a/net/tap-stub.c
>+++ b/net/tap-stub.c
>@@ -52,6 +52,11 @@ int tap_probe_has_uso(int fd)
> return 0;
> }
>
>+bool tap_probe_has_tunnel(int fd)
>+{
>+ return false;
>+}
>+
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> }
>diff --git a/net/tap.c b/net/tap.c
>index b49db19f83..23c6c118e7 100644
>--- a/net/tap.c
>+++ b/net/tap.c
>@@ -76,6 +76,7 @@ typedef struct TAPState {
> bool using_vnet_hdr;
> bool has_ufo;
> bool has_uso;
>+ bool has_tunnel;
> bool enabled;
> VHostNetState *vhost_net;
> unsigned host_vnet_hdr_len;
>@@ -241,6 +242,14 @@ static bool tap_has_uso(NetClientState *nc)
> return s->has_uso;
> }
>
>+static bool tap_has_tunnel(NetClientState *nc)
>+{
>+ TAPState *s = DO_UPCAST(TAPState, nc, nc);
>+
>+ assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
>+ return s->has_tunnel;
>+}
>+
> static bool tap_has_vnet_hdr(NetClientState *nc)
> {
> TAPState *s = DO_UPCAST(TAPState, nc, nc);
>@@ -369,6 +378,7 @@ static NetClientInfo net_tap_info = {
> .cleanup = tap_cleanup,
> .has_ufo = tap_has_ufo,
> .has_uso = tap_has_uso,
>+ .has_tunnel = tap_has_tunnel,
> .has_vnet_hdr = tap_has_vnet_hdr,
> .has_vnet_hdr_len = tap_has_vnet_hdr_len,
> .set_offload = tap_set_offload,
>@@ -398,6 +408,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
> s->using_vnet_hdr = false;
> s->has_ufo = tap_probe_has_ufo(s->fd);
> s->has_uso = tap_probe_has_uso(s->fd);
>+ s->has_tunnel = tap_probe_has_tunnel(s->fd);
> s->enabled = true;
> tap_set_offload(&s->nc, &ol);
> /*
>diff --git a/net/tap_int.h b/net/tap_int.h
>index f8bbe1cb0c..b76a05044b 100644
>--- a/net/tap_int.h
>+++ b/net/tap_int.h
>@@ -38,6 +38,7 @@ void tap_set_sndbuf(int fd, const NetdevTapOptions *tap, Error **errp);
> int tap_probe_vnet_hdr(int fd, Error **errp);
> int tap_probe_has_ufo(int fd);
> int tap_probe_has_uso(int fd);
>+bool tap_probe_has_tunnel(int fd);
> void tap_fd_set_offload(int fd, const NetOffloads *ol);
> void tap_fd_set_vnet_hdr_len(int fd, int len);
> int tap_fd_set_vnet_le(int fd, int vnet_is_le);
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 10/13] vhost-net: implement extended features support
2025-07-18 8:52 ` [PATCH RFC v3 10/13] vhost-net: " Paolo Abeni
@ 2025-07-18 13:01 ` Stefano Garzarella
2025-07-18 14:33 ` Paolo Abeni
0 siblings, 1 reply; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 13:01 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:36AM +0200, Paolo Abeni wrote:
>Provide extended version of the features manipulation helpers,
>and let the device initialization deal with the full features space,
>adjusting the relevant format strings accordingly.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - use virtio_features_andnot return value instead of virtio_features_empty
> - rebased on top of "net: Consolidate vhost feature bits into vhost_net
> structure"
> - _array -> _ex
>
>v1 -> v2:
> - uint128_t -> uint64_t[]
> - provide extended variant of the features manipulation helpers
>---
> hw/net/vhost_net-stub.c | 8 +++----
> hw/net/vhost_net.c | 46 ++++++++++++++++++++++-------------------
> include/net/vhost_net.h | 33 ++++++++++++++++++++++++++---
> 3 files changed, 58 insertions(+), 29 deletions(-)
>
>diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
>index 7d49f82906..0740d5a2eb 100644
>--- a/hw/net/vhost_net-stub.c
>+++ b/hw/net/vhost_net-stub.c
>@@ -46,9 +46,8 @@ void vhost_net_cleanup(struct vhost_net *net)
> {
> }
>
>-uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
>+void vhost_net_get_features_ex(struct vhost_net *net, uint64_t *features)
> {
>- return features;
> }
>
> int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
>@@ -62,13 +61,12 @@ int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
> return 0;
> }
>
>-void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
>+void vhost_net_ack_features_ex(struct vhost_net *net, const uint64_t *features)
> {
> }
>
>-uint64_t vhost_net_get_acked_features(VHostNetState *net)
>+void vhost_net_get_acked_features_ex(VHostNetState *net, uint64_t *features)
> {
>- return 0;
> }
>
> bool vhost_net_virtqueue_pending(VHostNetState *net, int idx)
>diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
>index 540492b37d..33fc6ba2f9 100644
>--- a/hw/net/vhost_net.c
>+++ b/hw/net/vhost_net.c
>@@ -35,10 +35,9 @@
> #include "hw/virtio/virtio-bus.h"
> #include "linux-headers/linux/vhost.h"
>
>-uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
>+void vhost_net_get_features_ex(struct vhost_net *net, uint64_t *features)
> {
>- return vhost_get_features(&net->dev, net->feature_bits,
>- features);
>+ vhost_get_features_ex(&net->dev, net->feature_bits, features);
> }
> int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
> uint32_t config_len)
>@@ -51,10 +50,11 @@ int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
> return vhost_dev_set_config(&net->dev, data, offset, size, flags);
> }
>
>-void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
>+void vhost_net_ack_features_ex(struct vhost_net *net, const uint64_t *features)
> {
>- net->dev.acked_features = net->dev.backend_features;
>- vhost_ack_features(&net->dev, net->feature_bits, features);
>+ virtio_features_copy(net->dev.acked_features_ex,
>+ net->dev.backend_features_ex);
>+ vhost_ack_features_ex(&net->dev, net->feature_bits, features);
> }
>
> uint64_t vhost_net_get_max_queues(VHostNetState *net)
>@@ -62,9 +62,9 @@ uint64_t vhost_net_get_max_queues(VHostNetState *net)
> return net->dev.max_queues;
> }
>
>-uint64_t vhost_net_get_acked_features(VHostNetState *net)
>+void vhost_net_get_acked_features_ex(VHostNetState *net, uint64_t *features)
> {
>- return net->dev.acked_features;
>+ virtio_features_copy(features, net->dev.acked_features_ex);
> }
>
> void vhost_net_save_acked_features(NetClientState *nc)
>@@ -234,7 +234,8 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
> int r;
> bool backend_kernel = options->backend_type == VHOST_BACKEND_TYPE_KERNEL;
> struct vhost_net *net = g_new0(struct vhost_net, 1);
>- uint64_t features = 0;
>+ uint64_t missing_features[VIRTIO_FEATURES_DWORDS];
>+ uint64_t features[VIRTIO_FEATURES_DWORDS];
Should we initialize `features` (IIUC calling virtio_features_clear)
since it was set to 0 before this patch?
> Error *local_err = NULL;
>
> if (!options->net_backend) {
>@@ -261,7 +262,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
> net->backend = r;
> net->dev.protocol_features = 0;
> } else {
>- net->dev.backend_features = 0;
>+ virtio_features_clear(net->dev.backend_features_ex);
> net->dev.protocol_features = 0;
> net->backend = -1;
>
>@@ -279,28 +280,31 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
> if (backend_kernel) {
> if (!qemu_has_vnet_hdr_len(options->net_backend,
> sizeof(struct virtio_net_hdr_mrg_rxbuf))) {
>- net->dev.features &= ~(1ULL << VIRTIO_NET_F_MRG_RXBUF);
>+ net->dev.features &= ~VIRTIO_BIT(VIRTIO_NET_F_MRG_RXBUF);
> }
>- if (~net->dev.features & net->dev.backend_features) {
>- fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
>- " for backend\n",
>- (uint64_t)(~net->dev.features & net->dev.backend_features));
>+
>+ if (virtio_features_andnot(missing_features,
>+ net->dev.backend_features_ex,
>+ net->dev.features_ex)) {
>+ fprintf(stderr, "vhost lacks feature mask 0x" VIRTIO_FEATURES_FMT
>+ " for backend\n", VIRTIO_FEATURES_PR(missing_features));
> goto fail;
> }
> }
>
> /* Set sane init value. Override when guest acks. */
> if (options->get_acked_features) {
>- features = options->get_acked_features(net->nc);
>- if (~net->dev.features & features) {
>- fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
>- " for backend\n",
>- (uint64_t)(~net->dev.features & features));
>+ virtio_features_from_u64(features,
>+ options->get_acked_features(net->nc));
>+ if (virtio_features_andnot(missing_features, features,
>+ net->dev.features_ex)) {
>+ fprintf(stderr, "vhost lacks feature mask 0x" VIRTIO_FEATURES_FMT
>+ " for backend\n", VIRTIO_FEATURES_PR(missing_features));
> goto fail;
> }
> }
>
>- vhost_net_ack_features(net, features);
>+ vhost_net_ack_features_ex(net, features);
If `options->get_acked_features` is false, `features` here is not
initialized (it was set to 0 before this patch).
Thanks,
Stefano
>
> return net;
>
>diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
>index 879781dad7..1d07b437c7 100644
>--- a/include/net/vhost_net.h
>+++ b/include/net/vhost_net.h
>@@ -2,6 +2,7 @@
> #define VHOST_NET_H
>
> #include "net/net.h"
>+#include "hw/virtio/virtio-features.h"
> #include "hw/virtio/vhost-backend.h"
>
> struct vhost_net;
>@@ -33,8 +34,26 @@ void vhost_net_stop(VirtIODevice *dev, NetClientState *ncs,
>
> void vhost_net_cleanup(VHostNetState *net);
>
>-uint64_t vhost_net_get_features(VHostNetState *net, uint64_t features);
>-void vhost_net_ack_features(VHostNetState *net, uint64_t features);
>+void vhost_net_get_features_ex(VHostNetState *net, uint64_t *features);
>+static inline uint64_t vhost_net_get_features(VHostNetState *net,
>+ uint64_t features)
>+{
>+ uint64_t features_array[VIRTIO_FEATURES_DWORDS];
>+
>+ virtio_features_from_u64(features_array, features);
>+ vhost_net_get_features_ex(net, features_array);
>+ return features_array[0];
>+}
>+
>+void vhost_net_ack_features_ex(VHostNetState *net, const uint64_t *features);
>+static inline void vhost_net_ack_features(VHostNetState *net,
>+ uint64_t features)
>+{
>+ uint64_t features_array[VIRTIO_FEATURES_DWORDS];
>+
>+ virtio_features_from_u64(features_array, features);
>+ vhost_net_ack_features_ex(net, features_array);
>+}
>
> int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
> uint32_t config_len);
>@@ -51,7 +70,15 @@ VHostNetState *get_vhost_net(NetClientState *nc);
>
> int vhost_net_set_vring_enable(NetClientState *nc, int enable);
>
>-uint64_t vhost_net_get_acked_features(VHostNetState *net);
>+void vhost_net_get_acked_features_ex(VHostNetState *net, uint64_t *features);
>+static inline uint64_t vhost_net_get_acked_features(VHostNetState *net)
>+{
>+ uint64_t features[VIRTIO_FEATURES_DWORDS];
>+
>+ vhost_net_get_acked_features_ex(net, features);
>+ assert(!virtio_features_use_extended(features));
>+ return features[0];
>+}
>
> int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
>
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading
2025-07-18 8:52 ` [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading Paolo Abeni
@ 2025-07-18 13:22 ` Stefano Garzarella
2025-07-18 13:44 ` Paolo Abeni
2025-07-18 15:21 ` Stefano Garzarella
1 sibling, 1 reply; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 13:22 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:39AM +0200, Paolo Abeni wrote:
>When any host or guest GSO over UDP tunnel offload is enabled the
>virtio net header includes the additional tunnel-related fields,
>update the size accordingly.
>
>Push the GSO over UDP tunnel offloads all the way down to the tap
>device extending the newly introduced NetFeatures struct, and
>eventually enable the associated features.
>
>As per virtio specification, to convert features bit to offload bit,
>map the extended features into the reserved range.
>
>Finally, make the vhost backend aware of the exact header layout, to
>copy it correctly. The tunnel-related field are present if either
>the guest or the host negotiated any UDP tunnel related feature:
>add them to the kernel supported features list, to allow qemu
>transfer to the backend the needed information.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - rebased on top of "net: Consolidate vhost feature bits into vhost_net
> structure"
> - _array -> _ex
>
>v1 -> v2:
> - squashed vhost support into this patch
> - dropped tun offload consistency checks; they are implemented in
> the kernel side
> - virtio_has_tnl_hdr ->virtio_has_tunnel_hdr
>---
> hw/net/virtio-net.c | 34 ++++++++++++++++++++++++++--------
> include/net/net.h | 2 ++
> net/net.c | 3 ++-
> net/tap-linux.c | 6 ++++++
> net/tap.c | 2 ++
> 5 files changed, 38 insertions(+), 9 deletions(-)
>
>diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
>index 70c85f7f77..46f7efac7d 100644
>--- a/hw/net/virtio-net.c
>+++ b/hw/net/virtio-net.c
>@@ -103,6 +103,12 @@
> #define VIRTIO_NET_F2O_SHIFT (VIRTIO_NET_OFFLOAD_MAP_MIN - \
> VIRTIO_NET_FEATURES_MAP_MIN + 64)
>
>+static bool virtio_has_tunnel_hdr(const uint64_t *features)
>+{
>+ return virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO) |
>+ virtio_has_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
>+}
>+
> static const VirtIOFeature feature_sizes[] = {
> {.flags = 1ULL << VIRTIO_NET_F_MAC,
> .end = endof(struct virtio_net_config, mac)},
>@@ -659,7 +665,8 @@ static bool peer_has_tunnel(VirtIONet *n)
> }
>
> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
>- int version_1, int hash_report)
>+ int version_1, int hash_report,
>+ int tunnel)
> {
> int i;
> NetClientState *nc;
>@@ -667,9 +674,11 @@ static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
> n->mergeable_rx_bufs = mergeable_rx_bufs;
>
> if (version_1) {
>- n->guest_hdr_len = hash_report ?
>- sizeof(struct virtio_net_hdr_v1_hash) :
>- sizeof(struct virtio_net_hdr_mrg_rxbuf);
>+ n->guest_hdr_len = tunnel ?
>+ sizeof(struct virtio_net_hdr_v1_hash_tunnel) :
>+ (hash_report ?
>+ sizeof(struct virtio_net_hdr_v1_hash) :
>+ sizeof(struct virtio_net_hdr_mrg_rxbuf));
> n->rss_data.populate_hash = !!hash_report;
> } else {
> n->guest_hdr_len = n->mergeable_rx_bufs ?
>@@ -803,6 +812,10 @@ static void virtio_net_apply_guest_offloads(VirtIONet *n)
> .ufo = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
> .uso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
> .uso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)),
>+ .tnl = !!(n->curr_guest_offloads &
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED)),
>+ .tnl_csum = !!(n->curr_guest_offloads &
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED)),
> };
>
> qemu_set_offload(qemu_get_queue(n->nic)->peer, &ol);
>@@ -824,7 +837,9 @@ virtio_net_guest_offloads_by_features(const uint64_t *features)
> (1ULL << VIRTIO_NET_F_GUEST_ECN) |
> (1ULL << VIRTIO_NET_F_GUEST_UFO) |
> (1ULL << VIRTIO_NET_F_GUEST_USO4) |
>- (1ULL << VIRTIO_NET_F_GUEST_USO6);
>+ (1ULL << VIRTIO_NET_F_GUEST_USO6) |
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED) |
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED);
>
> return guest_offloads_mask & virtio_net_features_to_offload(features);
> }
>@@ -937,7 +952,8 @@ static void virtio_net_set_features(VirtIODevice *vdev,
> virtio_has_feature_ex(features,
> VIRTIO_F_VERSION_1),
> virtio_has_feature_ex(features,
>- VIRTIO_NET_F_HASH_REPORT));
>+ VIRTIO_NET_F_HASH_REPORT),
>+ virtio_has_tunnel_hdr(features));
>
> n->rsc4_enabled = virtio_has_feature_ex(features, VIRTIO_NET_F_RSC_EXT) &&
> virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_TSO4);
>@@ -3160,13 +3176,15 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
> VirtIONet *n = opaque;
> VirtIODevice *vdev = VIRTIO_DEVICE(n);
> int i, link_down;
>+ bool has_tunnel_hdr = virtio_has_tunnel_hdr(vdev->guest_features_ex);
>
> trace_virtio_net_post_load_device();
> virtio_net_set_mrg_rx_bufs(n, n->mergeable_rx_bufs,
> virtio_vdev_has_feature(vdev,
> VIRTIO_F_VERSION_1),
> virtio_vdev_has_feature(vdev,
>- VIRTIO_NET_F_HASH_REPORT));
>+ VIRTIO_NET_F_HASH_REPORT),
>+ has_tunnel_hdr);
>
> /* MAC_TABLE_ENTRIES may be different from the saved image */
> if (n->mac_table.in_use > MAC_TABLE_ENTRIES) {
>@@ -3986,7 +4004,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
>
> n->vqs[0].tx_waiting = 0;
> n->tx_burst = n->net_conf.txburst;
>- virtio_net_set_mrg_rx_bufs(n, 0, 0, 0);
>+ virtio_net_set_mrg_rx_bufs(n, 0, 0, 0, 0);
> n->promisc = 1; /* for compatibility */
>
> n->mac_table.macs = g_malloc0(MAC_TABLE_ENTRIES * ETH_ALEN);
>diff --git a/include/net/net.h b/include/net/net.h
>index 9a9084690d..72b476ee1d 100644
>--- a/include/net/net.h
>+++ b/include/net/net.h
>@@ -43,6 +43,8 @@ typedef struct NetOffloads {
> bool ufo;
> bool uso4;
> bool uso6;
>+ bool tnl;
>+ bool tnl_csum;
> } NetOffloads;
>
> #define DEFINE_NIC_PROPERTIES(_state, _conf) \
>diff --git a/net/net.c b/net/net.c
>index 9536184a0c..27e0d27807 100644
>--- a/net/net.c
>+++ b/net/net.c
>@@ -575,7 +575,8 @@ void qemu_set_vnet_hdr_len(NetClientState *nc, int len)
>
> assert(len == sizeof(struct virtio_net_hdr_mrg_rxbuf) ||
> len == sizeof(struct virtio_net_hdr) ||
>- len == sizeof(struct virtio_net_hdr_v1_hash));
>+ len == sizeof(struct virtio_net_hdr_v1_hash) ||
>+ len == sizeof(struct virtio_net_hdr_v1_hash_tunnel));
>
> nc->vnet_hdr_len = len;
> nc->info->set_vnet_hdr_len(nc, len);
>diff --git a/net/tap-linux.c b/net/tap-linux.c
>index e2628be798..8e275d2ea4 100644
>--- a/net/tap-linux.c
>+++ b/net/tap-linux.c
>@@ -279,6 +279,12 @@ void tap_fd_set_offload(int fd, const NetOffloads *ol)
> if (ol->uso6) {
> offload |= TUN_F_USO6;
> }
>+ if (ol->tnl) {
>+ offload |= TUN_F_UDP_TUNNEL_GSO;
>+ }
>+ if (ol->tnl_csum) {
>+ offload |= TUN_F_UDP_TUNNEL_GSO_CSUM;
>+ }
> }
>
> if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) {
>diff --git a/net/tap.c b/net/tap.c
>index 23c6c118e7..2dfa843547 100644
>--- a/net/tap.c
>+++ b/net/tap.c
>@@ -62,6 +62,8 @@ static const int kernel_feature_bits[] = {
> VIRTIO_F_NOTIFICATION_DATA,
> VIRTIO_NET_F_RSC_EXT,
> VIRTIO_NET_F_HASH_REPORT,
>+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
>+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO,
The *_GSO_CSUM are not supported by vhost-net, right?
(sorry, I don't know the details, it just occurred to me by looking at
the fetaures we enable in the other patch.)
Thanks,
Stefano
> VHOST_INVALID_FEATURE_BIT
> };
>
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading
2025-07-18 13:22 ` Stefano Garzarella
@ 2025-07-18 13:44 ` Paolo Abeni
2025-07-18 13:48 ` Stefano Garzarella
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 13:44 UTC (permalink / raw)
To: Stefano Garzarella
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/18/25 3:22 PM, Stefano Garzarella wrote:
> On Fri, Jul 18, 2025 at 10:52:39AM +0200, Paolo Abeni wrote:
>> diff --git a/net/tap.c b/net/tap.c
>> index 23c6c118e7..2dfa843547 100644
>> --- a/net/tap.c
>> +++ b/net/tap.c
>> @@ -62,6 +62,8 @@ static const int kernel_feature_bits[] = {
>> VIRTIO_F_NOTIFICATION_DATA,
>> VIRTIO_NET_F_RSC_EXT,
>> VIRTIO_NET_F_HASH_REPORT,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO,
>
> The *_GSO_CSUM are not supported by vhost-net, right?
> (sorry, I don't know the details, it just occurred to me by looking at
> the fetaures we enable in the other patch.)
Yes, the kernel module supports/exposes only the 2 features above:
vhost-net need only to be aware of the exact virtio_net_header size,
which in turn depends just on them. Enabling/disabling the outer header
csum offload does not change the virtio_net_header struct.
The actual csum offload is implemented by the tun device.
Please LMK if the above solves your doubt.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading
2025-07-18 13:44 ` Paolo Abeni
@ 2025-07-18 13:48 ` Stefano Garzarella
0 siblings, 0 replies; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 13:48 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 03:44:20PM +0200, Paolo Abeni wrote:
>On 7/18/25 3:22 PM, Stefano Garzarella wrote:
>> On Fri, Jul 18, 2025 at 10:52:39AM +0200, Paolo Abeni wrote:
>>> diff --git a/net/tap.c b/net/tap.c
>>> index 23c6c118e7..2dfa843547 100644
>>> --- a/net/tap.c
>>> +++ b/net/tap.c
>>> @@ -62,6 +62,8 @@ static const int kernel_feature_bits[] = {
>>> VIRTIO_F_NOTIFICATION_DATA,
>>> VIRTIO_NET_F_RSC_EXT,
>>> VIRTIO_NET_F_HASH_REPORT,
>>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
>>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO,
>>
>> The *_GSO_CSUM are not supported by vhost-net, right?
>> (sorry, I don't know the details, it just occurred to me by looking at
>> the fetaures we enable in the other patch.)
>
>Yes, the kernel module supports/exposes only the 2 features above:
>vhost-net need only to be aware of the exact virtio_net_header size,
>which in turn depends just on them. Enabling/disabling the outer header
>csum offload does not change the virtio_net_header struct.
>
>The actual csum offload is implemented by the tun device.
>
>Please LMK if the above solves your doubt.
Yes, thanks!
Stefano
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 10/13] vhost-net: implement extended features support
2025-07-18 13:01 ` Stefano Garzarella
@ 2025-07-18 14:33 ` Paolo Abeni
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-18 14:33 UTC (permalink / raw)
To: Stefano Garzarella
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/18/25 3:01 PM, Stefano Garzarella wrote:
> On Fri, Jul 18, 2025 at 10:52:36AM +0200, Paolo Abeni wrote:
>> @@ -234,7 +234,8 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
>> int r;
>> bool backend_kernel = options->backend_type == VHOST_BACKEND_TYPE_KERNEL;
>> struct vhost_net *net = g_new0(struct vhost_net, 1);
>> - uint64_t features = 0;
>> + uint64_t missing_features[VIRTIO_FEATURES_DWORDS];
>> + uint64_t features[VIRTIO_FEATURES_DWORDS];
>
> Should we initialize `features` (IIUC calling virtio_features_clear)
> since it was set to 0 before this patch?
>
>> Error *local_err = NULL;
>>
>> if (!options->net_backend) {
>> @@ -261,7 +262,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
>> net->backend = r;
>> net->dev.protocol_features = 0;
>> } else {
>> - net->dev.backend_features = 0;
>> + virtio_features_clear(net->dev.backend_features_ex);
>> net->dev.protocol_features = 0;
>> net->backend = -1;
>>
>> @@ -279,28 +280,31 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
>> if (backend_kernel) {
>> if (!qemu_has_vnet_hdr_len(options->net_backend,
>> sizeof(struct virtio_net_hdr_mrg_rxbuf))) {
>> - net->dev.features &= ~(1ULL << VIRTIO_NET_F_MRG_RXBUF);
>> + net->dev.features &= ~VIRTIO_BIT(VIRTIO_NET_F_MRG_RXBUF);
>> }
>> - if (~net->dev.features & net->dev.backend_features) {
>> - fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
>> - " for backend\n",
>> - (uint64_t)(~net->dev.features & net->dev.backend_features));
>> +
>> + if (virtio_features_andnot(missing_features,
>> + net->dev.backend_features_ex,
>> + net->dev.features_ex)) {
>> + fprintf(stderr, "vhost lacks feature mask 0x" VIRTIO_FEATURES_FMT
>> + " for backend\n", VIRTIO_FEATURES_PR(missing_features));
>> goto fail;
>> }
>> }
>>
>> /* Set sane init value. Override when guest acks. */
>> if (options->get_acked_features) {
>> - features = options->get_acked_features(net->nc);
>> - if (~net->dev.features & features) {
>> - fprintf(stderr, "vhost lacks feature mask 0x%" PRIx64
>> - " for backend\n",
>> - (uint64_t)(~net->dev.features & features));
>> + virtio_features_from_u64(features,
>> + options->get_acked_features(net->nc));
>> + if (virtio_features_andnot(missing_features, features,
>> + net->dev.features_ex)) {
>> + fprintf(stderr, "vhost lacks feature mask 0x" VIRTIO_FEATURES_FMT
>> + " for backend\n", VIRTIO_FEATURES_PR(missing_features));
>> goto fail;
>> }
>> }
>>
>> - vhost_net_ack_features(net, features);
>> + vhost_net_ack_features_ex(net, features);
>
> If `options->get_acked_features` is false, `features` here is not
> initialized (it was set to 0 before this patch).
Indeed the initialization is needed. I will fix the next revision.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-18 8:52 ` [PATCH RFC v3 07/13] vhost: add support for negotiating " Paolo Abeni
@ 2025-07-18 14:36 ` Stefano Garzarella
2025-07-21 2:53 ` Lei Yang
2025-07-21 7:00 ` Paolo Abeni
2025-07-22 3:32 ` Jason Wang
1 sibling, 2 replies; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 14:36 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:33AM +0200, Paolo Abeni wrote:
>Similar to virtio infra, vhost core maintains the features status
>in the full extended format and allows the devices to implement
>extended version of the getter/setter.
>
>Note that 'protocol_features' are not extended: they are only
>used by vhost-user, and the latter device is not going to implement
>extended features soon.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - fix compile warning
> - _array -> _ex
>
>v1 -> v2:
> - uint128_t -> uint64_t[]
> - add _ex() variant of features manipulation helpers
>---
> hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
> include/hw/virtio/vhost-backend.h | 6 +++
> include/hw/virtio/vhost.h | 33 ++++++++++++--
> 3 files changed, 100 insertions(+), 12 deletions(-)
>
>diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>index c30ea1156e..85ae1e4d4c 100644
>--- a/hw/virtio/vhost.c
>+++ b/hw/virtio/vhost.c
>@@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
> static int vhost_dev_set_features(struct vhost_dev *dev,
> bool enable_log)
> {
>- uint64_t features = dev->acked_features;
>+ uint64_t features[VIRTIO_FEATURES_DWORDS];
> int r;
>+
>+ virtio_features_copy(features, dev->acked_features_ex);
> if (enable_log) {
>- features |= 0x1ULL << VHOST_F_LOG_ALL;
>+ virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
> }
> if (!vhost_dev_has_iommu(dev)) {
>- features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
>+ virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> }
> if (dev->vhost_ops->vhost_force_iommu) {
> if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
>- features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
>+ virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> }
> }
>- r = dev->vhost_ops->vhost_set_features(dev, features);
>+
>+ if (virtio_features_use_extended(features) &&
>+ !dev->vhost_ops->vhost_set_features_ex) {
>+ VHOST_OPS_DEBUG(r, "extended features without device support");
>+ r = -EINVAL;
>+ goto out;
>+ }
>+
>+ if (dev->vhost_ops->vhost_set_features_ex) {
>+ r = dev->vhost_ops->vhost_set_features_ex(dev, features);
>+ } else {
>+ r = dev->vhost_ops->vhost_set_features(dev, features[0]);
>+ }
> if (r < 0) {
> VHOST_OPS_DEBUG(r, "vhost_set_features failed");
> goto out;
>@@ -1506,12 +1520,27 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
> }
> }
>
>+static int vhost_dev_get_features(struct vhost_dev *hdev,
>+ uint64_t *features)
>+{
>+ uint64_t features64;
>+ int r;
>+
>+ if (hdev->vhost_ops->vhost_get_features_ex) {
>+ return hdev->vhost_ops->vhost_get_features_ex(hdev, features);
>+ }
>+
>+ r = hdev->vhost_ops->vhost_get_features(hdev, &features64);
>+ virtio_features_from_u64(features, features64);
>+ return r;
>+}
>+
> int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> VhostBackendType backend_type, uint32_t busyloop_timeout,
> Error **errp)
> {
>+ uint64_t features[VIRTIO_FEATURES_DWORDS];
> unsigned int used, reserved, limit;
>- uint64_t features;
> int i, r, n_initialized_vqs = 0;
>
> hdev->vdev = NULL;
>@@ -1531,7 +1560,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> goto fail;
> }
>
>- r = hdev->vhost_ops->vhost_get_features(hdev, &features);
>+ r = vhost_dev_get_features(hdev, features);
> if (r < 0) {
> error_setg_errno(errp, -r, "vhost_get_features failed");
> goto fail;
>@@ -1569,7 +1598,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> }
> }
>
>- hdev->features = features;
>+ virtio_features_copy(hdev->features_ex, features);
>
> hdev->memory_listener = (MemoryListener) {
> .name = "vhost",
>@@ -1592,7 +1621,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void
>*opaque,
> };
>
> if (hdev->migration_blocker == NULL) {
>- if (!(hdev->features & (0x1ULL << VHOST_F_LOG_ALL))) {
>+ if (!virtio_has_feature_ex(hdev->features_ex, VHOST_F_LOG_ALL)) {
> error_setg(&hdev->migration_blocker,
> "Migration disabled: vhost lacks VHOST_F_LOG_ALL feature.");
> } else if (vhost_dev_log_is_shared(hdev) && !qemu_memfd_alloc_check()) {
>@@ -1871,6 +1900,20 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
> return features;
> }
>
>+void vhost_get_features_ex(struct vhost_dev *hdev,
>+ const int *feature_bits,
>+ uint64_t *features)
>+{
>+ const int *bit = feature_bits;
>+
>+ while (*bit != VHOST_INVALID_FEATURE_BIT) {
>+ if (!virtio_has_feature_ex(hdev->features_ex, *bit)) {
>+ virtio_clear_feature_ex(features, *bit);
>+ }
>+ bit++;
>+ }
>+}
>+
Can we do something similar of what we do in hw/virtio/virtio.c where
the old virtio_set_features() use the new virtio_set_features_ex()?
> void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
> uint64_t features)
> {
>@@ -1884,6 +1927,18 @@ void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
> }
> }
>
>+void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
>+ const uint64_t *features)
>+{
>+ const int *bit = feature_bits;
>+ while (*bit != VHOST_INVALID_FEATURE_BIT) {
>+ if (virtio_has_feature_ex(features, *bit)) {
>+ virtio_add_feature_ex(hdev->acked_features_ex, *bit);
>+ }
>+ bit++;
>+ }
>+}
>+
Ditto.
Not a strong opinion, but just to reduce code duplication.
Thanks,
Stefano
> int vhost_dev_get_config(struct vhost_dev *hdev, uint8_t *config,
> uint32_t config_len, Error **errp)
> {
>diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
>index d6df209a2f..ff94fa1734 100644
>--- a/include/hw/virtio/vhost-backend.h
>+++ b/include/hw/virtio/vhost-backend.h
>@@ -95,6 +95,10 @@ typedef int (*vhost_new_worker_op)(struct vhost_dev *dev,
> struct vhost_worker_state *worker);
> typedef int (*vhost_free_worker_op)(struct vhost_dev *dev,
> struct vhost_worker_state *worker);
>+typedef int (*vhost_set_features_ex_op)(struct vhost_dev *dev,
>+ const uint64_t *features);
>+typedef int (*vhost_get_features_ex_op)(struct vhost_dev *dev,
>+ uint64_t *features);
> typedef int (*vhost_set_features_op)(struct vhost_dev *dev,
> uint64_t features);
> typedef int (*vhost_get_features_op)(struct vhost_dev *dev,
>@@ -186,6 +190,8 @@ typedef struct VhostOps {
> vhost_free_worker_op vhost_free_worker;
> vhost_get_vring_worker_op vhost_get_vring_worker;
> vhost_attach_vring_worker_op vhost_attach_vring_worker;
>+ vhost_set_features_ex_op vhost_set_features_ex;
>+ vhost_get_features_ex_op vhost_get_features_ex;
> vhost_set_features_op vhost_set_features;
> vhost_get_features_op vhost_get_features;
> vhost_set_backend_cap_op vhost_set_backend_cap;
>diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
>index 66be6afc88..39fbffc6bc 100644
>--- a/include/hw/virtio/vhost.h
>+++ b/include/hw/virtio/vhost.h
>@@ -107,9 +107,9 @@ struct vhost_dev {
> * future use should be discouraged and the variable retired as
> * its easy to confuse with the VirtIO backend_features.
> */
>- uint64_t features;
>- uint64_t acked_features;
>- uint64_t backend_features;
>+ VIRTIO_DECLARE_FEATURES(features);
>+ VIRTIO_DECLARE_FEATURES(acked_features);
>+ VIRTIO_DECLARE_FEATURES(backend_features);
>
> /**
> * @protocol_features: is the vhost-user only feature set by
>@@ -333,6 +333,21 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, VirtIODevice *vdev, int n,
> uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
> uint64_t features);
>
>+/**
>+ * vhost_get_features_ex() - sanitize the extended features set
>+ * @hdev: common vhost_dev structure
>+ * @feature_bits: pointer to terminated table of feature bits
>+ * @features: original features set, filtered out on return
>+ *
>+ * This is the extended variant of vhost_get_features(), supporting the
>+ * the extended features set. Filter it with the intersection of what is
>+ * supported by the vhost backend (hdev->features) and the supported
>+ * feature_bits.
>+ */
>+void vhost_get_features_ex(struct vhost_dev *hdev,
>+ const int *feature_bits,
>+ uint64_t *features);
>+
> /**
> * vhost_ack_features() - set vhost acked_features
> * @hdev: common vhost_dev structure
>@@ -344,6 +359,18 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
> */
> void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
> uint64_t features);
>+
>+/**
>+ * vhost_ack_features_ex() - set vhost full set of acked_features
>+ * @hdev: common vhost_dev structure
>+ * @feature_bits: pointer to terminated table of feature bits
>+ * @features: requested feature set
>+ *
>+ * This sets the internal hdev->acked_features to the intersection of
>+ * the backends advertised features and the supported feature_bits.
>+ */
>+void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
>+ const uint64_t *features);
> unsigned int vhost_get_max_memslots(void);
> unsigned int vhost_get_free_memslots(void);
>
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 04/13] virtio: serialize extended features state
2025-07-18 8:52 ` [PATCH RFC v3 04/13] virtio: serialize extended features state Paolo Abeni
@ 2025-07-18 15:06 ` Stefano Garzarella
2025-07-20 10:44 ` Akihiko Odaki
1 sibling, 0 replies; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 15:06 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:30AM +0200, Paolo Abeni wrote:
>If the driver uses any of the extended features (i.e. 64 or above),
>store the extended features range (64-127 bits).
>
>At load time, let legacy features initialize the full features range
>and pass it to the set helper; sub-states loading will have filled-up
>the extended part as needed.
>
>This is one of the few spots that need explicitly to know and set
>in stone the extended features array size; add a build bug to prevent
>breaking the migration should such size change again in the future:
>more serialization plumbing will be needed.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - 128bit_features state load/stores only the high bits
> - consolidate the load implementation to use a single set
> helper for 128/64/32 bits features
> - _array -> _ex
>
>v1 -> v2:
> - uint128_t -> u64[2]
>---
> hw/virtio/virtio.c | 88 ++++++++++++++++++++++++++++++----------------
> 1 file changed, 57 insertions(+), 31 deletions(-)
>
>diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
>index 2ab1d20769..2817d3a893 100644
>--- a/hw/virtio/virtio.c
>+++ b/hw/virtio/virtio.c
>@@ -2954,6 +2954,24 @@ static const VMStateDescription vmstate_virtio_disabled = {
> }
> };
>
>+static bool virtio_128bit_features_needed(void *opaque)
>+{
>+ VirtIODevice *vdev = opaque;
>+
>+ return virtio_features_use_extended(vdev->host_features_ex);
Related to live-migration, IIUC if we disable the new features in the
10.0 machine type, this should allow the migration (from a new qemu
using the old machine time to an old qemu not supporting extended
features), since those features should not be set in
vdev->host_features_ex, and this will return `false`.
Right?
Thanks,
Stefano
>+}
>+
>+static const VMStateDescription vmstate_virtio_128bit_features = {
>+ .name = "virtio/128bit_features",
>+ .version_id = 1,
>+ .minimum_version_id = 1,
>+ .needed = &virtio_128bit_features_needed,
>+ .fields = (const VMStateField[]) {
>+ VMSTATE_UINT64(guest_features_ex[1], VirtIODevice),
>+ VMSTATE_END_OF_LIST()
>+ }
>+};
>+
> static const VMStateDescription vmstate_virtio = {
> .name = "virtio",
> .version_id = 1,
>@@ -2963,6 +2981,7 @@ static const VMStateDescription vmstate_virtio = {
> },
> .subsections = (const VMStateDescription * const []) {
> &vmstate_virtio_device_endian,
>+ &vmstate_virtio_128bit_features,
> &vmstate_virtio_64bit_features,
> &vmstate_virtio_virtqueues,
> &vmstate_virtio_ringsize,
>@@ -3059,23 +3078,28 @@ const VMStateInfo virtio_vmstate_info = {
> .put = virtio_device_put,
> };
>
>-static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val)
>+static int virtio_set_features_nocheck(VirtIODevice *vdev, const uint64_t *val)
> {
> VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
>- bool bad = (val & ~(vdev->host_features)) != 0;
>+ uint64_t tmp[VIRTIO_FEATURES_DWORDS];
>+ bool bad;
>+
>+ bad = virtio_features_andnot(tmp, val, vdev->host_features_ex);
>+ virtio_features_and(tmp, val, vdev->host_features_ex);
>
>- val &= vdev->host_features;
> if (k->set_features) {
>- k->set_features(vdev, val);
>+ bad = bad || virtio_features_use_extended(tmp);
>+ k->set_features(vdev, tmp[0]);
> }
>- vdev->guest_features = val;
>+
>+ virtio_features_copy(vdev->guest_features_ex, tmp);
> return bad ? -1 : 0;
> }
>
> typedef struct VirtioSetFeaturesNocheckData {
> Coroutine *co;
> VirtIODevice *vdev;
>- uint64_t val;
>+ uint64_t val[VIRTIO_FEATURES_DWORDS];
> int ret;
> } VirtioSetFeaturesNocheckData;
>
>@@ -3088,14 +3112,15 @@ static void virtio_set_features_nocheck_bh(void *opaque)
> }
>
> static int coroutine_mixed_fn
>-virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev, uint64_t val)
>+virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev,
>+ const uint64_t *val)
> {
> if (qemu_in_coroutine()) {
> VirtioSetFeaturesNocheckData data = {
> .co = qemu_coroutine_self(),
> .vdev = vdev,
>- .val = val,
> };
>+ virtio_features_copy(data.val, val);
> aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
> virtio_set_features_nocheck_bh, &data);
> qemu_coroutine_yield();
>@@ -3107,6 +3132,7 @@ virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev, uint64_t val)
>
> int virtio_set_features(VirtIODevice *vdev, uint64_t val)
> {
>+ uint64_t features[VIRTIO_FEATURES_DWORDS];
> int ret;
> /*
> * The driver must not attempt to set features after feature negotiation
>@@ -3122,7 +3148,8 @@ int virtio_set_features(VirtIODevice *vdev, uint64_t val)
> __func__, vdev->name);
> }
>
>- ret = virtio_set_features_nocheck(vdev, val);
>+ virtio_features_from_u64(features, val);
>+ ret = virtio_set_features_nocheck(vdev, features);
> if (virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) {
> /* VIRTIO_RING_F_EVENT_IDX changes the size of the caches. */
> int i;
>@@ -3145,6 +3172,7 @@ void virtio_reset(void *opaque)
> {
> VirtIODevice *vdev = opaque;
> VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
>+ uint64_t features[VIRTIO_FEATURES_DWORDS];
> int i;
>
> virtio_set_status(vdev, 0);
>@@ -3171,7 +3199,8 @@ void virtio_reset(void *opaque)
> vdev->start_on_kick = false;
> vdev->started = false;
> vdev->broken = false;
>- virtio_set_features_nocheck(vdev, 0);
>+ virtio_features_clear(features);
>+ virtio_set_features_nocheck(vdev, features);
> vdev->queue_sel = 0;
> vdev->status = 0;
> vdev->disabled = false;
>@@ -3254,7 +3283,7 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
> * Note: devices should always test host features in future - don't create
> * new dependencies like this.
> */
>- vdev->guest_features = features;
>+ virtio_features_from_u64(vdev->guest_features_ex, features);
>
> config_len = qemu_get_be32(f);
>
>@@ -3333,26 +3362,23 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
> vdev->device_endian = virtio_default_endian();
> }
>
>- if (virtio_64bit_features_needed(vdev)) {
>- /*
>- * Subsection load filled vdev->guest_features. Run them
>- * through virtio_set_features to sanity-check them against
>- * host_features.
>- */
>- uint64_t features64 = vdev->guest_features;
>- if (virtio_set_features_nocheck_maybe_co(vdev, features64) < 0) {
>- error_report("Features 0x%" PRIx64 " unsupported. "
>- "Allowed features: 0x%" PRIx64,
>- features64, vdev->host_features);
>- return -1;
>- }
>- } else {
>- if (virtio_set_features_nocheck_maybe_co(vdev, features) < 0) {
>- error_report("Features 0x%x unsupported. "
>- "Allowed features: 0x%" PRIx64,
>- features, vdev->host_features);
>- return -1;
>- }
>+ /*
>+ * Avoid silently breaking migration should the feature space increase
>+ * even more in the (far away) future
>+ */
>+ QEMU_BUILD_BUG_ON(VIRTIO_FEATURES_DWORDS != 2);
>+
>+ /*
>+ * guest_features_ex is fully initialized with u32 features and upper
>+ * bits have been filled as needed by the later load.
>+ */
>+ if (virtio_set_features_nocheck_maybe_co(vdev,
>+ vdev->guest_features_ex) < 0) {
>+ error_report("Features 0x" VIRTIO_FEATURES_FMT " unsupported. "
>+ "Allowed features: 0x" VIRTIO_FEATURES_FMT,
>+ VIRTIO_FEATURES_PR(vdev->guest_features_ex),
>+ VIRTIO_FEATURES_PR(vdev->host_features_ex));
>+ return -1;
> }
>
> if (!virtio_device_started(vdev, vdev->status) &&
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading
2025-07-18 8:52 ` [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading Paolo Abeni
2025-07-18 13:22 ` Stefano Garzarella
@ 2025-07-18 15:21 ` Stefano Garzarella
1 sibling, 0 replies; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-18 15:21 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Fri, Jul 18, 2025 at 10:52:39AM +0200, Paolo Abeni wrote:
>When any host or guest GSO over UDP tunnel offload is enabled the
>virtio net header includes the additional tunnel-related fields,
>update the size accordingly.
>
>Push the GSO over UDP tunnel offloads all the way down to the tap
>device extending the newly introduced NetFeatures struct, and
>eventually enable the associated features.
>
>As per virtio specification, to convert features bit to offload bit,
>map the extended features into the reserved range.
>
>Finally, make the vhost backend aware of the exact header layout, to
>copy it correctly. The tunnel-related field are present if either
>the guest or the host negotiated any UDP tunnel related feature:
>add them to the kernel supported features list, to allow qemu
>transfer to the backend the needed information.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>v2 -> v3:
> - rebased on top of "net: Consolidate vhost feature bits into vhost_net
> structure"
> - _array -> _ex
>
>v1 -> v2:
> - squashed vhost support into this patch
> - dropped tun offload consistency checks; they are implemented in
> the kernel side
> - virtio_has_tnl_hdr ->virtio_has_tunnel_hdr
>---
> hw/net/virtio-net.c | 34 ++++++++++++++++++++++++++--------
> include/net/net.h | 2 ++
> net/net.c | 3 ++-
> net/tap-linux.c | 6 ++++++
> net/tap.c | 2 ++
> 5 files changed, 38 insertions(+), 9 deletions(-)
>
>diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
>index 70c85f7f77..46f7efac7d 100644
>--- a/hw/net/virtio-net.c
>+++ b/hw/net/virtio-net.c
>@@ -103,6 +103,12 @@
> #define VIRTIO_NET_F2O_SHIFT (VIRTIO_NET_OFFLOAD_MAP_MIN - \
> VIRTIO_NET_FEATURES_MAP_MIN + 64)
>
>+static bool virtio_has_tunnel_hdr(const uint64_t *features)
>+{
>+ return virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO) |
I'd replace | with || since virtio_has_feature_ex() return a `bool`.
Thanks,
Stefano
>+ virtio_has_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
>+}
>+
> static const VirtIOFeature feature_sizes[] = {
> {.flags = 1ULL << VIRTIO_NET_F_MAC,
> .end = endof(struct virtio_net_config, mac)},
>@@ -659,7 +665,8 @@ static bool peer_has_tunnel(VirtIONet *n)
> }
>
> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
>- int version_1, int hash_report)
>+ int version_1, int hash_report,
>+ int tunnel)
> {
> int i;
> NetClientState *nc;
>@@ -667,9 +674,11 @@ static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
> n->mergeable_rx_bufs = mergeable_rx_bufs;
>
> if (version_1) {
>- n->guest_hdr_len = hash_report ?
>- sizeof(struct virtio_net_hdr_v1_hash) :
>- sizeof(struct virtio_net_hdr_mrg_rxbuf);
>+ n->guest_hdr_len = tunnel ?
>+ sizeof(struct virtio_net_hdr_v1_hash_tunnel) :
>+ (hash_report ?
>+ sizeof(struct virtio_net_hdr_v1_hash) :
>+ sizeof(struct virtio_net_hdr_mrg_rxbuf));
> n->rss_data.populate_hash = !!hash_report;
> } else {
> n->guest_hdr_len = n->mergeable_rx_bufs ?
>@@ -803,6 +812,10 @@ static void virtio_net_apply_guest_offloads(VirtIONet *n)
> .ufo = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
> .uso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
> .uso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)),
>+ .tnl = !!(n->curr_guest_offloads &
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED)),
>+ .tnl_csum = !!(n->curr_guest_offloads &
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED)),
> };
>
> qemu_set_offload(qemu_get_queue(n->nic)->peer, &ol);
>@@ -824,7 +837,9 @@ virtio_net_guest_offloads_by_features(const uint64_t *features)
> (1ULL << VIRTIO_NET_F_GUEST_ECN) |
> (1ULL << VIRTIO_NET_F_GUEST_UFO) |
> (1ULL << VIRTIO_NET_F_GUEST_USO4) |
>- (1ULL << VIRTIO_NET_F_GUEST_USO6);
>+ (1ULL << VIRTIO_NET_F_GUEST_USO6) |
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_MAPPED) |
>+ (1ULL << VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM_MAPPED);
>
> return guest_offloads_mask & virtio_net_features_to_offload(features);
> }
>@@ -937,7 +952,8 @@ static void virtio_net_set_features(VirtIODevice *vdev,
> virtio_has_feature_ex(features,
> VIRTIO_F_VERSION_1),
> virtio_has_feature_ex(features,
>- VIRTIO_NET_F_HASH_REPORT));
>+ VIRTIO_NET_F_HASH_REPORT),
>+ virtio_has_tunnel_hdr(features));
>
> n->rsc4_enabled = virtio_has_feature_ex(features, VIRTIO_NET_F_RSC_EXT) &&
> virtio_has_feature_ex(features, VIRTIO_NET_F_GUEST_TSO4);
>@@ -3160,13 +3176,15 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
> VirtIONet *n = opaque;
> VirtIODevice *vdev = VIRTIO_DEVICE(n);
> int i, link_down;
>+ bool has_tunnel_hdr = virtio_has_tunnel_hdr(vdev->guest_features_ex);
>
> trace_virtio_net_post_load_device();
> virtio_net_set_mrg_rx_bufs(n, n->mergeable_rx_bufs,
> virtio_vdev_has_feature(vdev,
> VIRTIO_F_VERSION_1),
> virtio_vdev_has_feature(vdev,
>- VIRTIO_NET_F_HASH_REPORT));
>+ VIRTIO_NET_F_HASH_REPORT),
>+ has_tunnel_hdr);
>
> /* MAC_TABLE_ENTRIES may be different from the saved image */
> if (n->mac_table.in_use > MAC_TABLE_ENTRIES) {
>@@ -3986,7 +4004,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
>
> n->vqs[0].tx_waiting = 0;
> n->tx_burst = n->net_conf.txburst;
>- virtio_net_set_mrg_rx_bufs(n, 0, 0, 0);
>+ virtio_net_set_mrg_rx_bufs(n, 0, 0, 0, 0);
> n->promisc = 1; /* for compatibility */
>
> n->mac_table.macs = g_malloc0(MAC_TABLE_ENTRIES * ETH_ALEN);
>diff --git a/include/net/net.h b/include/net/net.h
>index 9a9084690d..72b476ee1d 100644
>--- a/include/net/net.h
>+++ b/include/net/net.h
>@@ -43,6 +43,8 @@ typedef struct NetOffloads {
> bool ufo;
> bool uso4;
> bool uso6;
>+ bool tnl;
>+ bool tnl_csum;
> } NetOffloads;
>
> #define DEFINE_NIC_PROPERTIES(_state, _conf) \
>diff --git a/net/net.c b/net/net.c
>index 9536184a0c..27e0d27807 100644
>--- a/net/net.c
>+++ b/net/net.c
>@@ -575,7 +575,8 @@ void qemu_set_vnet_hdr_len(NetClientState *nc, int len)
>
> assert(len == sizeof(struct virtio_net_hdr_mrg_rxbuf) ||
> len == sizeof(struct virtio_net_hdr) ||
>- len == sizeof(struct virtio_net_hdr_v1_hash));
>+ len == sizeof(struct virtio_net_hdr_v1_hash) ||
>+ len == sizeof(struct virtio_net_hdr_v1_hash_tunnel));
>
> nc->vnet_hdr_len = len;
> nc->info->set_vnet_hdr_len(nc, len);
>diff --git a/net/tap-linux.c b/net/tap-linux.c
>index e2628be798..8e275d2ea4 100644
>--- a/net/tap-linux.c
>+++ b/net/tap-linux.c
>@@ -279,6 +279,12 @@ void tap_fd_set_offload(int fd, const NetOffloads *ol)
> if (ol->uso6) {
> offload |= TUN_F_USO6;
> }
>+ if (ol->tnl) {
>+ offload |= TUN_F_UDP_TUNNEL_GSO;
>+ }
>+ if (ol->tnl_csum) {
>+ offload |= TUN_F_UDP_TUNNEL_GSO_CSUM;
>+ }
> }
>
> if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) {
>diff --git a/net/tap.c b/net/tap.c
>index 23c6c118e7..2dfa843547 100644
>--- a/net/tap.c
>+++ b/net/tap.c
>@@ -62,6 +62,8 @@ static const int kernel_feature_bits[] = {
> VIRTIO_F_NOTIFICATION_DATA,
> VIRTIO_NET_F_RSC_EXT,
> VIRTIO_NET_F_HASH_REPORT,
>+ VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO,
>+ VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO,
> VHOST_INVALID_FEATURE_BIT
> };
>
>--
>2.50.0
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-18 8:52 ` [PATCH RFC v3 08/13] qmp: update virtio features map to support " Paolo Abeni
2025-07-18 10:18 ` Stefano Garzarella
@ 2025-07-19 6:57 ` Markus Armbruster
2025-07-21 7:07 ` Paolo Abeni
2025-07-21 7:23 ` Akihiko Odaki
2 siblings, 1 reply; 54+ messages in thread
From: Markus Armbruster @ 2025-07-19 6:57 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin,
Stefano Garzarella, Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri,
Vincenzo Maffione, Eric Blake, Markus Armbruster
Paolo Abeni <pabeni@redhat.com> writes:
> Extend the VirtioDeviceFeatures struct with an additional u64
> to track unknown features in the 64-127 bit range and decode
> the full virtio features spaces for vhost and virtio devices.
>
> Also add entries for the soon-to-be-supported virtio net GSO over
> UDP features.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
[...]
> diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
> index 3b6377cf0d..03c6163cf4 100644
> --- a/hw/virtio/virtio-qmp.c
> +++ b/hw/virtio/virtio-qmp.c
[...]
> @@ -680,9 +715,10 @@ VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
> g_assert_not_reached();
> }
>
> - features->has_unknown_dev_features = bitmap != 0;
> + features->has_unknown_dev_features = !virtio_features_empty(bitmap);
> if (features->has_unknown_dev_features) {
> - features->unknown_dev_features = bitmap;
> + features->unknown_dev_features = bitmap[0];
> + features->unknown_dev_features2 = bitmap[1];
> }
Why not assign unconditionally?
>
> return features;
[...]
> diff --git a/qapi/virtio.json b/qapi/virtio.json
> index 9d652fe4a8..f2e2dd6e97 100644
> --- a/qapi/virtio.json
> +++ b/qapi/virtio.json
> @@ -490,14 +490,18 @@
> # unique features)
> #
> # @unknown-dev-features: Virtio device features bitmap that have not
> -# been decoded
> +# been decoded (bits 0-63)
> +#
> +# @unknown-dev-features2: Virtio device features bitmap that have not
> +# been decoded (bits 64-127)
> #
> # Since: 7.2
> ##
> { 'struct': 'VirtioDeviceFeatures',
> 'data': { 'transports': [ 'str' ],
> '*dev-features': [ 'str' ],
> - '*unknown-dev-features': 'uint64' } }
> + '*unknown-dev-features': 'uint64',
> + '*unknown-dev-features2': 'uint64' } }
>
> ##
> # @VirtQueueStatus:
I wish we could simply widen @unknown-dev-features, but we don't have
uint128, and adding it would risk breaking QMP clients. 64 bit integers
are already troublesome in JSON.
Does the example in x-query-virtio-status's doc comment need an update?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 01/13] net: bundle all offloads in a single struct
2025-07-18 8:52 ` [PATCH RFC v3 01/13] net: bundle all offloads in a single struct Paolo Abeni
@ 2025-07-20 10:31 ` Akihiko Odaki
0 siblings, 0 replies; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-20 10:31 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/18 17:52, Paolo Abeni wrote:
> The set_offload() argument list is already pretty long and
> we are going to introduce soon a bunch of additional offloads.
>
> Replace the offload arguments with a single struct and update
> all the relevant call-sites.
>
> No functional changes intended.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> Note: I maintained the struct usage as opposed to uint64_t bitmask usage
> as suggested by Akihiko, because the latter feel a bit more invasive.
Though this statement is not going to be comitted, I think it needs an
update; I showed the level of intrusion will be about the same though
there is another downside of syntactical redundandancy. I would just
leave the URL to the previous argument:
https://lore.kernel.org/qemu-devel/f2674001-9489-44c9-b17a-c9d8a66b3935@rsg.ci.i.u-tokyo.ac.jp/
Besides I have a comment about code style below:
>
> v1 -> v2:
> - drop unneeded 'struct' keywords
> - moved to series start
> ---
> hw/net/e1000e_core.c | 5 +++--
> hw/net/igb_core.c | 5 +++--
> hw/net/virtio-net.c | 19 +++++++++++--------
> hw/net/vmxnet3.c | 13 +++++--------
> include/net/net.h | 15 ++++++++++++---
> net/net.c | 5 ++---
> net/netmap.c | 3 +--
> net/tap-bsd.c | 3 +--
> net/tap-linux.c | 21 ++++++++++++---------
> net/tap-solaris.c | 4 ++--
> net/tap-stub.c | 3 +--
> net/tap.c | 8 ++++----
> net/tap_int.h | 4 ++--
> 13 files changed, 59 insertions(+), 49 deletions(-)
>
> diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c
> index 2413858790..27599a0dc2 100644
> --- a/hw/net/e1000e_core.c
> +++ b/hw/net/e1000e_core.c
> @@ -2827,8 +2827,9 @@ e1000e_update_rx_offloads(E1000ECore *core)
> trace_e1000e_rx_set_cso(cso_state);
>
> if (core->has_vnet) {
> - qemu_set_offload(qemu_get_queue(core->owner_nic)->peer,
> - cso_state, 0, 0, 0, 0, 0, 0);
> + NetOffloads ol = {.csum = cso_state };
There should be a whitespace after '{', just like '}' which has a
preceding whitspace.
> +
> + qemu_set_offload(qemu_get_queue(core->owner_nic)->peer, &ol);
> }
> }
>
> diff --git a/hw/net/igb_core.c b/hw/net/igb_core.c
> index 39e3ce1c8f..45d8fd795b 100644
> --- a/hw/net/igb_core.c
> +++ b/hw/net/igb_core.c
> @@ -3058,8 +3058,9 @@ igb_update_rx_offloads(IGBCore *core)
> trace_e1000e_rx_set_cso(cso_state);
>
> if (core->has_vnet) {
> - qemu_set_offload(qemu_get_queue(core->owner_nic)->peer,
> - cso_state, 0, 0, 0, 0, 0, 0);
> + NetOffloads ol = {.csum = cso_state };
> +
> + qemu_set_offload(qemu_get_queue(core->owner_nic)->peer, &ol);
> }
> }
>
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index c4c49b0f9c..8953c329e7 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -773,14 +773,17 @@ static uint64_t virtio_net_bad_features(VirtIODevice *vdev)
>
> static void virtio_net_apply_guest_offloads(VirtIONet *n)
> {
> - qemu_set_offload(qemu_get_queue(n->nic)->peer,
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_CSUM)),
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO4)),
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO6)),
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_ECN)),
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
> - !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)));
> + NetOffloads ol = {
> + .csum = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_CSUM)),
> + .tso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO4)),
> + .tso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_TSO6)),
> + .ecn = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_ECN)),
> + .ufo = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_UFO)),
> + .uso4 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO4)),
> + .uso6 = !!(n->curr_guest_offloads & (1ULL << VIRTIO_NET_F_GUEST_USO6)),
> + };
> +
> + qemu_set_offload(qemu_get_queue(n->nic)->peer, &ol);
> }
>
> static uint64_t virtio_net_guest_offloads_by_features(uint64_t features)
> diff --git a/hw/net/vmxnet3.c b/hw/net/vmxnet3.c
> index af73aa8ef2..03732375a7 100644
> --- a/hw/net/vmxnet3.c
> +++ b/hw/net/vmxnet3.c
> @@ -1322,14 +1322,11 @@ static void vmxnet3_update_features(VMXNET3State *s)
> s->lro_supported, rxcso_supported,
> s->rx_vlan_stripping);
> if (s->peer_has_vhdr) {
> - qemu_set_offload(qemu_get_queue(s->nic)->peer,
> - rxcso_supported,
> - s->lro_supported,
> - s->lro_supported,
> - 0,
> - 0,
> - 0,
> - 0);
> + NetOffloads ol = { .csum = rxcso_supported,
> + .tso4 = s->lro_supported,
> + .tso6 = s->lro_supported };
> +
> + qemu_set_offload(qemu_get_queue(s->nic)->peer, &ol);
> }
> }
>
> diff --git a/include/net/net.h b/include/net/net.h
> index 84ee18e0f9..48ba333d02 100644
> --- a/include/net/net.h
> +++ b/include/net/net.h
> @@ -35,6 +35,16 @@ typedef struct NICConf {
> int32_t bootindex;
> } NICConf;
>
> +typedef struct NetOffloads {
> + bool csum;
> + bool tso4;
> + bool tso6;
> + bool ecn;
> + bool ufo;
> + bool uso4;
> + bool uso6;
> +} NetOffloads;
> +
> #define DEFINE_NIC_PROPERTIES(_state, _conf) \
> DEFINE_PROP_MACADDR("mac", _state, _conf.macaddr), \
> DEFINE_PROP_NETDEV("netdev", _state, _conf.peers)
> @@ -57,7 +67,7 @@ typedef bool (HasUfo)(NetClientState *);
> typedef bool (HasUso)(NetClientState *);
> typedef bool (HasVnetHdr)(NetClientState *);
> typedef bool (HasVnetHdrLen)(NetClientState *, int);
> -typedef void (SetOffload)(NetClientState *, int, int, int, int, int, int, int);
> +typedef void (SetOffload)(NetClientState *, const NetOffloads *);
> typedef int (GetVnetHdrLen)(NetClientState *);
> typedef void (SetVnetHdrLen)(NetClientState *, int);
> typedef bool (GetVnetHashSupportedTypes)(NetClientState *, uint32_t *);
> @@ -189,8 +199,7 @@ bool qemu_has_ufo(NetClientState *nc);
> bool qemu_has_uso(NetClientState *nc);
> bool qemu_has_vnet_hdr(NetClientState *nc);
> bool qemu_has_vnet_hdr_len(NetClientState *nc, int len);
> -void qemu_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
> - int ecn, int ufo, int uso4, int uso6);
> +void qemu_set_offload(NetClientState *nc, const NetOffloads *ol);
> int qemu_get_vnet_hdr_len(NetClientState *nc);
> void qemu_set_vnet_hdr_len(NetClientState *nc, int len);
> bool qemu_get_vnet_hash_supported_types(NetClientState *nc, uint32_t *types);
> diff --git a/net/net.c b/net/net.c
> index da275db86e..63872b6855 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -540,14 +540,13 @@ bool qemu_has_vnet_hdr_len(NetClientState *nc, int len)
> return nc->info->has_vnet_hdr_len(nc, len);
> }
>
> -void qemu_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
> - int ecn, int ufo, int uso4, int uso6)
> +void qemu_set_offload(NetClientState *nc, const NetOffloads *ol)
> {
> if (!nc || !nc->info->set_offload) {
> return;
> }
>
> - nc->info->set_offload(nc, csum, tso4, tso6, ecn, ufo, uso4, uso6);
> + nc->info->set_offload(nc, ol);
> }
>
> int qemu_get_vnet_hdr_len(NetClientState *nc)
> diff --git a/net/netmap.c b/net/netmap.c
> index 297510e190..6cd8f2bdc5 100644
> --- a/net/netmap.c
> +++ b/net/netmap.c
> @@ -366,8 +366,7 @@ static void netmap_set_vnet_hdr_len(NetClientState *nc, int len)
> }
> }
>
> -static void netmap_set_offload(NetClientState *nc, int csum, int tso4, int tso6,
> - int ecn, int ufo, int uso4, int uso6)
> +static void netmap_set_offload(NetClientState *nc, const NetOffloads *ol)
> {
> NetmapState *s = DO_UPCAST(NetmapState, nc, nc);
>
> diff --git a/net/tap-bsd.c b/net/tap-bsd.c
> index b4c84441ba..86b6edee94 100644
> --- a/net/tap-bsd.c
> +++ b/net/tap-bsd.c
> @@ -231,8 +231,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
> return -EINVAL;
> }
>
> -void tap_fd_set_offload(int fd, int csum, int tso4,
> - int tso6, int ecn, int ufo, int uso4, int uso6)
> +void tap_fd_set_offload(int fd, const NetOffloads *ol)
> {
> }
>
> diff --git a/net/tap-linux.c b/net/tap-linux.c
> index 22ec2f45d2..a1c58f74f5 100644
> --- a/net/tap-linux.c
> +++ b/net/tap-linux.c
> @@ -239,8 +239,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
> abort();
> }
>
> -void tap_fd_set_offload(int fd, int csum, int tso4,
> - int tso6, int ecn, int ufo, int uso4, int uso6)
> +void tap_fd_set_offload(int fd, const NetOffloads *ol)
> {
> unsigned int offload = 0;
>
> @@ -249,20 +248,24 @@ void tap_fd_set_offload(int fd, int csum, int tso4,
> return;
> }
>
> - if (csum) {
> + if (ol->csum) {
> offload |= TUN_F_CSUM;
> - if (tso4)
> + if (ol->tso4) {
> offload |= TUN_F_TSO4;
> - if (tso6)
> + }
> + if (ol->tso6) {
> offload |= TUN_F_TSO6;
> - if ((tso4 || tso6) && ecn)
> + }
> + if ((ol->tso4 || ol->tso6) && ol->ecn) {
> offload |= TUN_F_TSO_ECN;
> - if (ufo)
> + }
> + if (ol->ufo) {
> offload |= TUN_F_UFO;
> - if (uso4) {
> + }
> + if (ol->uso4) {
> offload |= TUN_F_USO4;
> }
> - if (uso6) {
> + if (ol->uso6) {
> offload |= TUN_F_USO6;
> }
> }
> diff --git a/net/tap-solaris.c b/net/tap-solaris.c
> index 51b7830bef..833c066bee 100644
> --- a/net/tap-solaris.c
> +++ b/net/tap-solaris.c
> @@ -27,6 +27,7 @@
> #include "tap_int.h"
> #include "qemu/ctype.h"
> #include "qemu/cutils.h"
> +#include "net/net.h"
>
> #include <sys/ethernet.h>
> #include <sys/sockio.h>
> @@ -235,8 +236,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
> return -EINVAL;
> }
>
> -void tap_fd_set_offload(int fd, int csum, int tso4,
> - int tso6, int ecn, int ufo, int uso4, int uso6)
> +void tap_fd_set_offload(int fd, const NetOffloads *ol)
> {
> }
>
> diff --git a/net/tap-stub.c b/net/tap-stub.c
> index 38673434cb..67d14ad4d5 100644
> --- a/net/tap-stub.c
> +++ b/net/tap-stub.c
> @@ -66,8 +66,7 @@ int tap_fd_set_vnet_be(int fd, int is_be)
> return -EINVAL;
> }
>
> -void tap_fd_set_offload(int fd, int csum, int tso4,
> - int tso6, int ecn, int ufo, int uso4, int uso6)
> +void tap_fd_set_offload(int fd, const NetOffloads *ol)
> {
> }
>
> diff --git a/net/tap.c b/net/tap.c
> index 23536c09b4..b49db19f83 100644
> --- a/net/tap.c
> +++ b/net/tap.c
> @@ -280,15 +280,14 @@ static int tap_set_vnet_be(NetClientState *nc, bool is_be)
> return tap_fd_set_vnet_be(s->fd, is_be);
> }
>
> -static void tap_set_offload(NetClientState *nc, int csum, int tso4,
> - int tso6, int ecn, int ufo, int uso4, int uso6)
> +static void tap_set_offload(NetClientState *nc, const NetOffloads *ol)
> {
> TAPState *s = DO_UPCAST(TAPState, nc, nc);
> if (s->fd < 0) {
> return;
> }
>
> - tap_fd_set_offload(s->fd, csum, tso4, tso6, ecn, ufo, uso4, uso6);
> + tap_fd_set_offload(s->fd, ol);
> }
>
> static void tap_exit_notify(Notifier *notifier, void *data)
> @@ -386,6 +385,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
> int fd,
> int vnet_hdr)
> {
> + NetOffloads ol = {};
> NetClientState *nc;
> TAPState *s;
>
> @@ -399,7 +399,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
> s->has_ufo = tap_probe_has_ufo(s->fd);
> s->has_uso = tap_probe_has_uso(s->fd);
> s->enabled = true;
> - tap_set_offload(&s->nc, 0, 0, 0, 0, 0, 0, 0);
> + tap_set_offload(&s->nc, &ol);
> /*
> * Make sure host header length is set correctly in tap:
> * it might have been modified by another instance of qemu.
> diff --git a/net/tap_int.h b/net/tap_int.h
> index 8857ff299d..f8bbe1cb0c 100644
> --- a/net/tap_int.h
> +++ b/net/tap_int.h
> @@ -27,6 +27,7 @@
> #define NET_TAP_INT_H
>
> #include "qapi/qapi-types-net.h"
> +#include "net/net.h"
>
> int tap_open(char *ifname, int ifname_size, int *vnet_hdr,
> int vnet_hdr_required, int mq_required, Error **errp);
> @@ -37,8 +38,7 @@ void tap_set_sndbuf(int fd, const NetdevTapOptions *tap, Error **errp);
> int tap_probe_vnet_hdr(int fd, Error **errp);
> int tap_probe_has_ufo(int fd);
> int tap_probe_has_uso(int fd);
> -void tap_fd_set_offload(int fd, int csum, int tso4, int tso6, int ecn, int ufo,
> - int uso4, int uso6);
> +void tap_fd_set_offload(int fd, const NetOffloads *ol);
> void tap_fd_set_vnet_hdr_len(int fd, int len);
> int tap_fd_set_vnet_le(int fd, int vnet_is_le);
> int tap_fd_set_vnet_be(int fd, int vnet_is_be);
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 03/13] virtio: introduce extended features type
2025-07-18 8:52 ` [PATCH RFC v3 03/13] virtio: introduce extended features type Paolo Abeni
@ 2025-07-20 10:41 ` Akihiko Odaki
2025-07-21 7:33 ` Paolo Abeni
0 siblings, 1 reply; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-20 10:41 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/18 17:52, Paolo Abeni wrote:
> The virtio specifications allows for up to 128 bits for the
> device features. Soon we are going to use some of the 'extended'
> bits features (bit 64 and above) for the virtio net driver.
>
> Represent the virtio features bitmask with a fixes size array, and
> introduce a few helpers to help manipulate them.
>
> Most drivers will keep using only 64 bits features space: use union
> to allow them access the lower part of the extended space without any
> per driver change.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2 -> v3:
> - fix preprocessor guard name
> - use BIT_ULL
> - add missing parentheses
> - use memcmp()
> - _is_empty() -> _empty()
> - _andnot() returns a bool, true if dst has any non zero bits
> - _array -> _ex
>
> v1 -> v2:
> - use a fixed size array for features instead of uint128
> - use union with u64 to reduce the needed code churn
> ---
> include/hw/virtio/virtio-features.h | 123 ++++++++++++++++++++++++++++
> include/hw/virtio/virtio.h | 7 +-
> 2 files changed, 127 insertions(+), 3 deletions(-)
> create mode 100644 include/hw/virtio/virtio-features.h
>
> diff --git a/include/hw/virtio/virtio-features.h b/include/hw/virtio/virtio-features.h
> new file mode 100644
> index 0000000000..68e326e3e8
> --- /dev/null
> +++ b/include/hw/virtio/virtio-features.h
> @@ -0,0 +1,123 @@
> +/*
> + * Virtio features helpers
> + *
> + * Copyright 2025 Red Hat, Inc.
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#ifndef QEMU_VIRTIO_FEATURES_H
> +#define QEMU_VIRTIO_FEATURES_H
> +
> +#include "qemu/bitops.h"
> +
> +#define VIRTIO_FEATURES_FMT "%016"PRIx64"%016"PRIx64
> +#define VIRTIO_FEATURES_PR(f) (f)[1], (f)[0]
> +
> +#define VIRTIO_FEATURES_MAX 128
> +#define VIRTIO_BIT(b) BIT_ULL((b) % 64)
> +#define VIRTIO_DWORD(b) ((b) >> 6)
> +#define VIRTIO_FEATURES_WORDS (VIRTIO_FEATURES_MAX >> 5)
> +#define VIRTIO_FEATURES_DWORDS (VIRTIO_FEATURES_WORDS >> 1)
These shifts are better to be replaced with division for clarity;
BIT_WORD() is a good example.
"WORD" and "DWORD" should be avoided due to contradicting definitions
common in QEMU as described at:
https://lore.kernel.org/qemu-devel/aab8c434-364e-4305-9d8b-943eb0c98406@rsg.ci.i.u-tokyo.ac.jp/
BITS_TO_U32S() is a good example this regard.
> +
> +#define VIRTIO_DECLARE_FEATURES(name) \
> + union { \
> + uint64_t name; \
> + uint64_t name##_ex[VIRTIO_FEATURES_DWORDS]; \
> + }
> +
> +static inline void virtio_features_clear(uint64_t *features)
> +{
> + memset(features, 0, sizeof(features[0]) * VIRTIO_FEATURES_DWORDS);
> +}
> +
> +static inline void virtio_features_from_u64(uint64_t *features, uint64_t from)
> +{
> + virtio_features_clear(features);
> + features[0] = from;
> +}
> +
> +static inline bool virtio_has_feature_ex(const uint64_t *features,
> + unsigned int fbit)
> +{
> + assert(fbit < VIRTIO_FEATURES_MAX);
> + return features[VIRTIO_DWORD(fbit)] & VIRTIO_BIT(fbit);
> +}
> +
> +static inline void virtio_add_feature_ex(uint64_t *features,
> + unsigned int fbit)
> +{
> + assert(fbit < VIRTIO_FEATURES_MAX);
> + features[VIRTIO_DWORD(fbit)] |= VIRTIO_BIT(fbit);
> +}
> +
> +static inline void virtio_clear_feature_ex(uint64_t *features,
> + unsigned int fbit)
> +{
> + assert(fbit < VIRTIO_FEATURES_MAX);
> + features[VIRTIO_DWORD(fbit)] &= ~VIRTIO_BIT(fbit);
> +}
> +
> +static inline bool virtio_features_equal(const uint64_t *f1,
> + const uint64_t *f2)
> +{
> + return !memcmp(f1, f2, sizeof(uint64_t) * VIRTIO_FEATURES_DWORDS);
> +}
> +
> +static inline bool virtio_features_use_extended(const uint64_t *features)
"extended" should be abbreviated as "ex" for consistency.
> +{
> + int i;
> +
> + for (i = 1; i < VIRTIO_FEATURES_DWORDS; ++i) {
> + if (features[i]) {
> + return true;
> + }
> + }
> + return false;
> +}
> +
> +static inline bool virtio_features_empty(const uint64_t *features)
> +{
> + return !virtio_features_use_extended(features) && !features[0];
> +}
> +
> +static inline void virtio_features_copy(uint64_t *to, const uint64_t *from)
> +{
> + memcpy(to, from, sizeof(to[0]) * VIRTIO_FEATURES_DWORDS);
> +}
> +
> +static inline bool virtio_features_andnot(uint64_t *to, const uint64_t *f1,
> + const uint64_t *f2)
> +{
> + uint64_t diff = 0;
> + int i;
> +
> + for (i = 0; i < VIRTIO_FEATURES_DWORDS; i++) {
> + to[i] = f1[i] & ~f2[i];
> + diff |= to[i];
> + }
> + return diff;
> +}
> +
> +static inline void virtio_features_and(uint64_t *to, const uint64_t *f1,
> + const uint64_t *f2)
> +{
> + int i;
> +
> + for (i = 0; i < VIRTIO_FEATURES_DWORDS; i++) {
> + to[i] = f1[i] & f2[i];
> + }
> +}
> +
> +static inline void virtio_features_or(uint64_t *to, const uint64_t *f1,
> + const uint64_t *f2)
> +{
> + int i;
> +
> + for (i = 0; i < VIRTIO_FEATURES_DWORDS; i++) {
> + to[i] = f1[i] | f2[i];
> + }
> +}
> +
> +#endif
> +
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index c594764f23..39e4059a66 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -16,6 +16,7 @@
>
> #include "system/memory.h"
> #include "hw/qdev-core.h"
> +#include "hw/virtio/virtio-features.h"
> #include "net/net.h"
> #include "migration/vmstate.h"
> #include "qemu/event_notifier.h"
> @@ -121,9 +122,9 @@ struct VirtIODevice
> * backend (e.g. vhost) and could potentially be a subset of the
> * total feature set offered by QEMU.
> */
> - uint64_t host_features;
> - uint64_t guest_features;
> - uint64_t backend_features;
> + VIRTIO_DECLARE_FEATURES(host_features);
> + VIRTIO_DECLARE_FEATURES(guest_features);
> + VIRTIO_DECLARE_FEATURES(backend_features);
>
> size_t config_len;
> void *config;
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 04/13] virtio: serialize extended features state
2025-07-18 8:52 ` [PATCH RFC v3 04/13] virtio: serialize extended features state Paolo Abeni
2025-07-18 15:06 ` Stefano Garzarella
@ 2025-07-20 10:44 ` Akihiko Odaki
2025-07-21 7:51 ` Paolo Abeni
1 sibling, 1 reply; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-20 10:44 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/18 17:52, Paolo Abeni wrote:
> If the driver uses any of the extended features (i.e. 64 or above),
> store the extended features range (64-127 bits).
>
> At load time, let legacy features initialize the full features range
> and pass it to the set helper; sub-states loading will have filled-up
> the extended part as needed.
>
> This is one of the few spots that need explicitly to know and set
> in stone the extended features array size; add a build bug to prevent
> breaking the migration should such size change again in the future:
> more serialization plumbing will be needed.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2 -> v3:
> - 128bit_features state load/stores only the high bits
> - consolidate the load implementation to use a single set
> helper for 128/64/32 bits features
> - _array -> _ex
>
> v1 -> v2:
> - uint128_t -> u64[2]
> ---
> hw/virtio/virtio.c | 88 ++++++++++++++++++++++++++++++----------------
> 1 file changed, 57 insertions(+), 31 deletions(-)
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 2ab1d20769..2817d3a893 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2954,6 +2954,24 @@ static const VMStateDescription vmstate_virtio_disabled = {
> }
> };
>
> +static bool virtio_128bit_features_needed(void *opaque)
> +{
> + VirtIODevice *vdev = opaque;
> +
> + return virtio_features_use_extended(vdev->host_features_ex);
> +}
> +
> +static const VMStateDescription vmstate_virtio_128bit_features = {
> + .name = "virtio/128bit_features",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .needed = &virtio_128bit_features_needed,
> + .fields = (const VMStateField[]) {
> + VMSTATE_UINT64(guest_features_ex[1], VirtIODevice),
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> static const VMStateDescription vmstate_virtio = {
> .name = "virtio",
> .version_id = 1,
> @@ -2963,6 +2981,7 @@ static const VMStateDescription vmstate_virtio = {
> },
> .subsections = (const VMStateDescription * const []) {
> &vmstate_virtio_device_endian,
> + &vmstate_virtio_128bit_features,
> &vmstate_virtio_64bit_features,
> &vmstate_virtio_virtqueues,
> &vmstate_virtio_ringsize,
> @@ -3059,23 +3078,28 @@ const VMStateInfo virtio_vmstate_info = {
> .put = virtio_device_put,
> };
>
> -static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val)
> +static int virtio_set_features_nocheck(VirtIODevice *vdev, const uint64_t *val)
> {
> VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> - bool bad = (val & ~(vdev->host_features)) != 0;
> + uint64_t tmp[VIRTIO_FEATURES_DWORDS];
> + bool bad;
> +
> + bad = virtio_features_andnot(tmp, val, vdev->host_features_ex);
> + virtio_features_and(tmp, val, vdev->host_features_ex);
>
> - val &= vdev->host_features;
> if (k->set_features) {
> - k->set_features(vdev, val);
> + bad = bad || virtio_features_use_extended(tmp);
> + k->set_features(vdev, tmp[0]);
> }
> - vdev->guest_features = val;
> +
> + virtio_features_copy(vdev->guest_features_ex, tmp);
> return bad ? -1 : 0;
> }
>
> typedef struct VirtioSetFeaturesNocheckData {
> Coroutine *co;
> VirtIODevice *vdev;
> - uint64_t val;
> + uint64_t val[VIRTIO_FEATURES_DWORDS];
> int ret;
> } VirtioSetFeaturesNocheckData;
>
> @@ -3088,14 +3112,15 @@ static void virtio_set_features_nocheck_bh(void *opaque)
> }
>
> static int coroutine_mixed_fn
> -virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev, uint64_t val)
> +virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev,
> + const uint64_t *val)
> {
> if (qemu_in_coroutine()) {
> VirtioSetFeaturesNocheckData data = {
> .co = qemu_coroutine_self(),
> .vdev = vdev,
> - .val = val,
> };
> + virtio_features_copy(data.val, val);
> aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
> virtio_set_features_nocheck_bh, &data);
> qemu_coroutine_yield();
> @@ -3107,6 +3132,7 @@ virtio_set_features_nocheck_maybe_co(VirtIODevice *vdev, uint64_t val)
>
> int virtio_set_features(VirtIODevice *vdev, uint64_t val)
> {
> + uint64_t features[VIRTIO_FEATURES_DWORDS];
> int ret;
> /*
> * The driver must not attempt to set features after feature negotiation
> @@ -3122,7 +3148,8 @@ int virtio_set_features(VirtIODevice *vdev, uint64_t val)
> __func__, vdev->name);
> }
>
> - ret = virtio_set_features_nocheck(vdev, val);
> + virtio_features_from_u64(features, val);
> + ret = virtio_set_features_nocheck(vdev, features);
> if (virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) {
> /* VIRTIO_RING_F_EVENT_IDX changes the size of the caches. */
> int i;
> @@ -3145,6 +3172,7 @@ void virtio_reset(void *opaque)
> {
> VirtIODevice *vdev = opaque;
> VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> + uint64_t features[VIRTIO_FEATURES_DWORDS];
> int i;
>
> virtio_set_status(vdev, 0);
> @@ -3171,7 +3199,8 @@ void virtio_reset(void *opaque)
> vdev->start_on_kick = false;
> vdev->started = false;
> vdev->broken = false;
> - virtio_set_features_nocheck(vdev, 0);
> + virtio_features_clear(features);
> + virtio_set_features_nocheck(vdev, features);
> vdev->queue_sel = 0;
> vdev->status = 0;
> vdev->disabled = false;
> @@ -3254,7 +3283,7 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
> * Note: devices should always test host features in future - don't create
> * new dependencies like this.
> */
> - vdev->guest_features = features;
> + virtio_features_from_u64(vdev->guest_features_ex, features);
>
> config_len = qemu_get_be32(f);
>
> @@ -3333,26 +3362,23 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
> vdev->device_endian = virtio_default_endian();
> }
>
> - if (virtio_64bit_features_needed(vdev)) {
> - /*
> - * Subsection load filled vdev->guest_features. Run them
> - * through virtio_set_features to sanity-check them against
> - * host_features.
> - */
> - uint64_t features64 = vdev->guest_features;
> - if (virtio_set_features_nocheck_maybe_co(vdev, features64) < 0) {
> - error_report("Features 0x%" PRIx64 " unsupported. "
> - "Allowed features: 0x%" PRIx64,
> - features64, vdev->host_features);
> - return -1;
> - }
> - } else {
> - if (virtio_set_features_nocheck_maybe_co(vdev, features) < 0) {
> - error_report("Features 0x%x unsupported. "
> - "Allowed features: 0x%" PRIx64,
> - features, vdev->host_features);
> - return -1;
> - }
> + /*
> + * Avoid silently breaking migration should the feature space increase
> + * even more in the (far away) future
> + */
> + QEMU_BUILD_BUG_ON(VIRTIO_FEATURES_DWORDS != 2);
This should be moved to around virtio_128bit_features_needed() and
vmstate_virtio_128bit_features as they make assumptions on the feature
space while virtio_load() doesn't.
> +
> + /*
> + * guest_features_ex is fully initialized with u32 features and upper
> + * bits have been filled as needed by the later load.
> + */
> + if (virtio_set_features_nocheck_maybe_co(vdev,
> + vdev->guest_features_ex) < 0) {
> + error_report("Features 0x" VIRTIO_FEATURES_FMT " unsupported. "
> + "Allowed features: 0x" VIRTIO_FEATURES_FMT,
> + VIRTIO_FEATURES_PR(vdev->guest_features_ex),
> + VIRTIO_FEATURES_PR(vdev->host_features_ex));
> + return -1;
> }
>
> if (!virtio_device_started(vdev, vdev->status) &&
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-18 14:36 ` Stefano Garzarella
@ 2025-07-21 2:53 ` Lei Yang
2025-07-21 8:21 ` Paolo Abeni
2025-07-21 7:00 ` Paolo Abeni
1 sibling, 1 reply; 54+ messages in thread
From: Lei Yang @ 2025-07-21 2:53 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Stefano Garzarella,
Akihiko Odaki, Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Fri, Jul 18, 2025 at 10:37 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>
> On Fri, Jul 18, 2025 at 10:52:33AM +0200, Paolo Abeni wrote:
> >Similar to virtio infra, vhost core maintains the features status
> >in the full extended format and allows the devices to implement
> >extended version of the getter/setter.
> >
> >Note that 'protocol_features' are not extended: they are only
> >used by vhost-user, and the latter device is not going to implement
> >extended features soon.
> >
> >Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> >---
> >v2 -> v3:
> > - fix compile warning
> > - _array -> _ex
> >
> >v1 -> v2:
> > - uint128_t -> uint64_t[]
> > - add _ex() variant of features manipulation helpers
> >---
> > hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
> > include/hw/virtio/vhost-backend.h | 6 +++
> > include/hw/virtio/vhost.h | 33 ++++++++++++--
> > 3 files changed, 100 insertions(+), 12 deletions(-)
> >
> >diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> >index c30ea1156e..85ae1e4d4c 100644
> >--- a/hw/virtio/vhost.c
> >+++ b/hw/virtio/vhost.c
> >@@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
> > static int vhost_dev_set_features(struct vhost_dev *dev,
> > bool enable_log)
> > {
> >- uint64_t features = dev->acked_features;
> >+ uint64_t features[VIRTIO_FEATURES_DWORDS];
> > int r;
> >+
> >+ virtio_features_copy(features, dev->acked_features_ex);
> > if (enable_log) {
> >- features |= 0x1ULL << VHOST_F_LOG_ALL;
> >+ virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
> > }
> > if (!vhost_dev_has_iommu(dev)) {
> >- features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
> >+ virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> > }
> > if (dev->vhost_ops->vhost_force_iommu) {
> > if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
> >- features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
> >+ virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> > }
> > }
> >- r = dev->vhost_ops->vhost_set_features(dev, features);
> >+
Hi Paolo
> >+ if (virtio_features_use_extended(features) &&
> >+ !dev->vhost_ops->vhost_set_features_ex) {
> >+ VHOST_OPS_DEBUG(r, "extended features without device support");
> >+ r = -EINVAL;
> >+ goto out;
> >+ }
As we discussed in version 2, this code should be changed to: [1],
otherwise the problem mentioned last time will occur when compiling.
[1] if (virtio_features_use_extended(features) &&
!dev->vhost_ops->vhost_set_features_ex) {
- VHOST_OPS_DEBUG(r, "extended features without device support");
r = -EINVAL;
+ VHOST_OPS_DEBUG(r, "extended features without device support");
goto out;
}
Thanks
Lei
> >+
> >+ if (dev->vhost_ops->vhost_set_features_ex) {
> >+ r = dev->vhost_ops->vhost_set_features_ex(dev, features);
> >+ } else {
> >+ r = dev->vhost_ops->vhost_set_features(dev, features[0]);
> >+ }
> > if (r < 0) {
> > VHOST_OPS_DEBUG(r, "vhost_set_features failed");
> > goto out;
> >@@ -1506,12 +1520,27 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
> > }
> > }
> >
> >+static int vhost_dev_get_features(struct vhost_dev *hdev,
> >+ uint64_t *features)
> >+{
> >+ uint64_t features64;
> >+ int r;
> >+
> >+ if (hdev->vhost_ops->vhost_get_features_ex) {
> >+ return hdev->vhost_ops->vhost_get_features_ex(hdev, features);
> >+ }
> >+
> >+ r = hdev->vhost_ops->vhost_get_features(hdev, &features64);
> >+ virtio_features_from_u64(features, features64);
> >+ return r;
> >+}
> >+
> > int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> > VhostBackendType backend_type, uint32_t busyloop_timeout,
> > Error **errp)
> > {
> >+ uint64_t features[VIRTIO_FEATURES_DWORDS];
> > unsigned int used, reserved, limit;
> >- uint64_t features;
> > int i, r, n_initialized_vqs = 0;
> >
> > hdev->vdev = NULL;
> >@@ -1531,7 +1560,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> > goto fail;
> > }
> >
> >- r = hdev->vhost_ops->vhost_get_features(hdev, &features);
> >+ r = vhost_dev_get_features(hdev, features);
> > if (r < 0) {
> > error_setg_errno(errp, -r, "vhost_get_features failed");
> > goto fail;
> >@@ -1569,7 +1598,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
> > }
> > }
> >
> >- hdev->features = features;
> >+ virtio_features_copy(hdev->features_ex, features);
> >
> > hdev->memory_listener = (MemoryListener) {
> > .name = "vhost",
> >@@ -1592,7 +1621,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void
> >*opaque,
> > };
> >
> > if (hdev->migration_blocker == NULL) {
> >- if (!(hdev->features & (0x1ULL << VHOST_F_LOG_ALL))) {
> >+ if (!virtio_has_feature_ex(hdev->features_ex, VHOST_F_LOG_ALL)) {
> > error_setg(&hdev->migration_blocker,
> > "Migration disabled: vhost lacks VHOST_F_LOG_ALL feature.");
> > } else if (vhost_dev_log_is_shared(hdev) && !qemu_memfd_alloc_check()) {
> >@@ -1871,6 +1900,20 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
> > return features;
> > }
> >
> >+void vhost_get_features_ex(struct vhost_dev *hdev,
> >+ const int *feature_bits,
> >+ uint64_t *features)
> >+{
> >+ const int *bit = feature_bits;
> >+
> >+ while (*bit != VHOST_INVALID_FEATURE_BIT) {
> >+ if (!virtio_has_feature_ex(hdev->features_ex, *bit)) {
> >+ virtio_clear_feature_ex(features, *bit);
> >+ }
> >+ bit++;
> >+ }
> >+}
> >+
>
> Can we do something similar of what we do in hw/virtio/virtio.c where
> the old virtio_set_features() use the new virtio_set_features_ex()?
>
> > void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
> > uint64_t features)
> > {
> >@@ -1884,6 +1927,18 @@ void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
> > }
> > }
> >
> >+void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
> >+ const uint64_t *features)
> >+{
> >+ const int *bit = feature_bits;
> >+ while (*bit != VHOST_INVALID_FEATURE_BIT) {
> >+ if (virtio_has_feature_ex(features, *bit)) {
> >+ virtio_add_feature_ex(hdev->acked_features_ex, *bit);
> >+ }
> >+ bit++;
> >+ }
> >+}
> >+
>
> Ditto.
>
> Not a strong opinion, but just to reduce code duplication.
>
> Thanks,
> Stefano
>
> > int vhost_dev_get_config(struct vhost_dev *hdev, uint8_t *config,
> > uint32_t config_len, Error **errp)
> > {
> >diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
> >index d6df209a2f..ff94fa1734 100644
> >--- a/include/hw/virtio/vhost-backend.h
> >+++ b/include/hw/virtio/vhost-backend.h
> >@@ -95,6 +95,10 @@ typedef int (*vhost_new_worker_op)(struct vhost_dev *dev,
> > struct vhost_worker_state *worker);
> > typedef int (*vhost_free_worker_op)(struct vhost_dev *dev,
> > struct vhost_worker_state *worker);
> >+typedef int (*vhost_set_features_ex_op)(struct vhost_dev *dev,
> >+ const uint64_t *features);
> >+typedef int (*vhost_get_features_ex_op)(struct vhost_dev *dev,
> >+ uint64_t *features);
> > typedef int (*vhost_set_features_op)(struct vhost_dev *dev,
> > uint64_t features);
> > typedef int (*vhost_get_features_op)(struct vhost_dev *dev,
> >@@ -186,6 +190,8 @@ typedef struct VhostOps {
> > vhost_free_worker_op vhost_free_worker;
> > vhost_get_vring_worker_op vhost_get_vring_worker;
> > vhost_attach_vring_worker_op vhost_attach_vring_worker;
> >+ vhost_set_features_ex_op vhost_set_features_ex;
> >+ vhost_get_features_ex_op vhost_get_features_ex;
> > vhost_set_features_op vhost_set_features;
> > vhost_get_features_op vhost_get_features;
> > vhost_set_backend_cap_op vhost_set_backend_cap;
> >diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> >index 66be6afc88..39fbffc6bc 100644
> >--- a/include/hw/virtio/vhost.h
> >+++ b/include/hw/virtio/vhost.h
> >@@ -107,9 +107,9 @@ struct vhost_dev {
> > * future use should be discouraged and the variable retired as
> > * its easy to confuse with the VirtIO backend_features.
> > */
> >- uint64_t features;
> >- uint64_t acked_features;
> >- uint64_t backend_features;
> >+ VIRTIO_DECLARE_FEATURES(features);
> >+ VIRTIO_DECLARE_FEATURES(acked_features);
> >+ VIRTIO_DECLARE_FEATURES(backend_features);
> >
> > /**
> > * @protocol_features: is the vhost-user only feature set by
> >@@ -333,6 +333,21 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, VirtIODevice *vdev, int n,
> > uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
> > uint64_t features);
> >
> >+/**
> >+ * vhost_get_features_ex() - sanitize the extended features set
> >+ * @hdev: common vhost_dev structure
> >+ * @feature_bits: pointer to terminated table of feature bits
> >+ * @features: original features set, filtered out on return
> >+ *
> >+ * This is the extended variant of vhost_get_features(), supporting the
> >+ * the extended features set. Filter it with the intersection of what is
> >+ * supported by the vhost backend (hdev->features) and the supported
> >+ * feature_bits.
> >+ */
> >+void vhost_get_features_ex(struct vhost_dev *hdev,
> >+ const int *feature_bits,
> >+ uint64_t *features);
> >+
> > /**
> > * vhost_ack_features() - set vhost acked_features
> > * @hdev: common vhost_dev structure
> >@@ -344,6 +359,18 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
> > */
> > void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
> > uint64_t features);
> >+
> >+/**
> >+ * vhost_ack_features_ex() - set vhost full set of acked_features
> >+ * @hdev: common vhost_dev structure
> >+ * @feature_bits: pointer to terminated table of feature bits
> >+ * @features: requested feature set
> >+ *
> >+ * This sets the internal hdev->acked_features to the intersection of
> >+ * the backends advertised features and the supported feature_bits.
> >+ */
> >+void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
> >+ const uint64_t *features);
> > unsigned int vhost_get_max_memslots(void);
> > unsigned int vhost_get_free_memslots(void);
> >
> >--
> >2.50.0
> >
>
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-18 14:36 ` Stefano Garzarella
2025-07-21 2:53 ` Lei Yang
@ 2025-07-21 7:00 ` Paolo Abeni
1 sibling, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 7:00 UTC (permalink / raw)
To: Stefano Garzarella
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/18/25 4:36 PM, Stefano Garzarella wrote:
> On Fri, Jul 18, 2025 at 10:52:33AM +0200, Paolo Abeni wrote:
>> @@ -1871,6 +1900,20 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
>> return features;
>> }
>>
>> +void vhost_get_features_ex(struct vhost_dev *hdev,
>> + const int *feature_bits,
>> + uint64_t *features)
>> +{
>> + const int *bit = feature_bits;
>> +
>> + while (*bit != VHOST_INVALID_FEATURE_BIT) {
>> + if (!virtio_has_feature_ex(hdev->features_ex, *bit)) {
>> + virtio_clear_feature_ex(features, *bit);
>> + }
>> + bit++;
>> + }
>> +}
>> +
>
> Can we do something similar of what we do in hw/virtio/virtio.c where
> the old virtio_set_features() use the new virtio_set_features_ex()?
>
>> void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
>> uint64_t features)
>> {
>> @@ -1884,6 +1927,18 @@ void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
>> }
>> }
>>
>> +void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
>> + const uint64_t *features)
>> +{
>> + const int *bit = feature_bits;
>> + while (*bit != VHOST_INVALID_FEATURE_BIT) {
>> + if (virtio_has_feature_ex(features, *bit)) {
>> + virtio_add_feature_ex(hdev->acked_features_ex, *bit);
>> + }
>> + bit++;
>> + }
>> +}
>> +
>
> Ditto.
>
> Not a strong opinion, but just to reduce code duplication.
The incremental diffstat with such cleanup looks good, so I'll include
that in the next revision, thanks!
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-19 6:57 ` Markus Armbruster
@ 2025-07-21 7:07 ` Paolo Abeni
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 7:07 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin,
Stefano Garzarella, Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri,
Vincenzo Maffione, Eric Blake
On 7/19/25 8:57 AM, Markus Armbruster wrote:
> Paolo Abeni <pabeni@redhat.com> writes:
>
>> Extend the VirtioDeviceFeatures struct with an additional u64
>> to track unknown features in the 64-127 bit range and decode
>> the full virtio features spaces for vhost and virtio devices.
>>
>> Also add entries for the soon-to-be-supported virtio net GSO over
>> UDP features.
>>
>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>
> [...]
>
>> diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
>> index 3b6377cf0d..03c6163cf4 100644
>> --- a/hw/virtio/virtio-qmp.c
>> +++ b/hw/virtio/virtio-qmp.c
>
> [...]
>
>> @@ -680,9 +715,10 @@ VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
>> g_assert_not_reached();
>> }
>>
>> - features->has_unknown_dev_features = bitmap != 0;
>> + features->has_unknown_dev_features = !virtio_features_empty(bitmap);
>> if (features->has_unknown_dev_features) {
>> - features->unknown_dev_features = bitmap;
>> + features->unknown_dev_features = bitmap[0];
>> + features->unknown_dev_features2 = bitmap[1];
>> }
>
> Why not assign unconditionally?
I stuck with the old/previous code style. I can move to unconditional
assignment in the next revision.
>> return features;
>
> [...]
>
>> diff --git a/qapi/virtio.json b/qapi/virtio.json
>> index 9d652fe4a8..f2e2dd6e97 100644
>> --- a/qapi/virtio.json
>> +++ b/qapi/virtio.json
>> @@ -490,14 +490,18 @@
>> # unique features)
>> #
>> # @unknown-dev-features: Virtio device features bitmap that have not
>> -# been decoded
>> +# been decoded (bits 0-63)
>> +#
>> +# @unknown-dev-features2: Virtio device features bitmap that have not
>> +# been decoded (bits 64-127)
>> #
>> # Since: 7.2
>> ##
>> { 'struct': 'VirtioDeviceFeatures',
>> 'data': { 'transports': [ 'str' ],
>> '*dev-features': [ 'str' ],
>> - '*unknown-dev-features': 'uint64' } }
>> + '*unknown-dev-features': 'uint64',
>> + '*unknown-dev-features2': 'uint64' } }
>>
>> ##
>> # @VirtQueueStatus:
>
> I wish we could simply widen @unknown-dev-features, but we don't have
> uint128, and adding it would risk breaking QMP clients. 64 bit integers
> are already troublesome in JSON.
>
> Does the example in x-query-virtio-status's doc comment need an update?
Yes, you are right, the output for the 'unknown features' field will
include leading zeros for the high 64 bits.
I will update it in the next revision, thanks!
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-18 8:52 ` [PATCH RFC v3 08/13] qmp: update virtio features map to support " Paolo Abeni
2025-07-18 10:18 ` Stefano Garzarella
2025-07-19 6:57 ` Markus Armbruster
@ 2025-07-21 7:23 ` Akihiko Odaki
2025-07-21 7:45 ` Stefano Garzarella
2025-07-21 8:04 ` Paolo Abeni
2 siblings, 2 replies; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-21 7:23 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/18 17:52, Paolo Abeni wrote:
> Extend the VirtioDeviceFeatures struct with an additional u64
> to track unknown features in the 64-127 bit range and decode
> the full virtio features spaces for vhost and virtio devices.
>
> Also add entries for the soon-to-be-supported virtio net GSO over
> UDP features.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2 -> v3:
> - unknown-dev-features-dword2 -> unknown-dev-features2
> - _array -> _ex
> - fixed typos in entries description
>
> v1 -> v2:
> - uint128_t -> uint64_t[]
> ---
> hw/virtio/virtio-hmp-cmds.c | 3 +-
> hw/virtio/virtio-qmp.c | 89 ++++++++++++++++++++++++++-----------
> hw/virtio/virtio-qmp.h | 3 +-
> qapi/virtio.json | 8 +++-
> 4 files changed, 73 insertions(+), 30 deletions(-)
>
> diff --git a/hw/virtio/virtio-hmp-cmds.c b/hw/virtio/virtio-hmp-cmds.c
> index 7d8677bcf0..1daae482d3 100644
> --- a/hw/virtio/virtio-hmp-cmds.c
> +++ b/hw/virtio/virtio-hmp-cmds.c
> @@ -74,7 +74,8 @@ static void hmp_virtio_dump_features(Monitor *mon,
> }
>
> if (features->has_unknown_dev_features) {
> - monitor_printf(mon, " unknown-features(0x%016"PRIx64")\n",
> + monitor_printf(mon, " unknown-features(0x%016"PRIx64"%016"PRIx64")\n",
> + features->unknown_dev_features2,
> features->unknown_dev_features);
> }
> }
> diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
> index 3b6377cf0d..03c6163cf4 100644
> --- a/hw/virtio/virtio-qmp.c
> +++ b/hw/virtio/virtio-qmp.c
> @@ -325,6 +325,20 @@ static const qmp_virtio_feature_map_t virtio_net_feature_map[] = {
> FEATURE_ENTRY(VHOST_USER_F_PROTOCOL_FEATURES, \
> "VHOST_USER_F_PROTOCOL_FEATURES: Vhost-user protocol features "
> "negotiation supported"),
> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, \
> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
> + "UDP tunnel packets"),
> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, \
> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
> + "UDP tunnel packets requiring checksum offload for the outer "
> + "header"),
> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, \
> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
> + "UDP tunnel packets"),
> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, \
> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
> + "UDP tunnel packets requiring checksum offload for the outer "
> + "header"),
> { -1, "" }
> };
> #endif
> @@ -510,6 +524,24 @@ static const qmp_virtio_feature_map_t virtio_gpio_feature_map[] = {
> list; \
> })
>
> +#define CONVERT_FEATURES_EX(type, map, bitmap) \
> + ({ \
> + type *list = NULL; \
> + type *node; \
> + for (i = 0; map[i].virtio_bit != -1; i++) { \
> + bit = map[i].virtio_bit; \
> + if (!virtio_has_feature_ex(bitmap, bit)) { \
> + continue; \
> + } \
> + node = g_new0(type, 1); \
> + node->value = g_strdup(map[i].feature_desc); \
> + node->next = list; \
> + list = node; \
> + virtio_clear_feature_ex(bitmap, bit); \
> + } \
> + list; \
> + })
> +
> VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap)
> {
> VirtioDeviceStatus *status;
> @@ -545,109 +577,112 @@ VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap)
> return vhu_protocols;
> }
>
> -VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
> +VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
> + const uint64_t *bmap)
> {
> + uint64_t bitmap[VIRTIO_FEATURES_DWORDS];
> VirtioDeviceFeatures *features;
> uint64_t bit;
> int i;
>
> + virtio_features_copy(bitmap, bmap);
> features = g_new0(VirtioDeviceFeatures, 1);
> features->has_dev_features = true;
>
> /* transport features */
> - features->transports = CONVERT_FEATURES(strList, virtio_transport_map, 0,
> - bitmap);
> + features->transports = CONVERT_FEATURES_EX(strList, virtio_transport_map,
> + bitmap);
>
> /* device features */
> switch (device_id) {
> #ifdef CONFIG_VIRTIO_SERIAL
> case VIRTIO_ID_CONSOLE:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_serial_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_serial_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_BLK
> case VIRTIO_ID_BLOCK:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_blk_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_blk_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_GPU
> case VIRTIO_ID_GPU:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_gpu_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_gpu_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_NET
> case VIRTIO_ID_NET:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_net_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_net_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_SCSI
> case VIRTIO_ID_SCSI:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_scsi_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_scsi_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_BALLOON
> case VIRTIO_ID_BALLOON:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_balloon_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_balloon_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_IOMMU
> case VIRTIO_ID_IOMMU:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_iommu_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_iommu_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_INPUT
> case VIRTIO_ID_INPUT:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_input_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_input_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VHOST_USER_FS
> case VIRTIO_ID_FS:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_fs_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_fs_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VHOST_VSOCK
> case VIRTIO_ID_VSOCK:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_vsock_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_vsock_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_CRYPTO
> case VIRTIO_ID_CRYPTO:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_crypto_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_crypto_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_MEM
> case VIRTIO_ID_MEM:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_mem_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_mem_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_I2C_ADAPTER
> case VIRTIO_ID_I2C_ADAPTER:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_i2c_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_i2c_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VIRTIO_RNG
> case VIRTIO_ID_RNG:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_rng_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_rng_feature_map, bitmap);
> break;
> #endif
> #ifdef CONFIG_VHOST_USER_GPIO
> case VIRTIO_ID_GPIO:
> features->dev_features =
> - CONVERT_FEATURES(strList, virtio_gpio_feature_map, 0, bitmap);
> + CONVERT_FEATURES_EX(strList, virtio_gpio_feature_map, bitmap);
> break;
> #endif
> /* No features */
> @@ -680,9 +715,10 @@ VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
> g_assert_not_reached();
> }
>
> - features->has_unknown_dev_features = bitmap != 0;
> + features->has_unknown_dev_features = !virtio_features_empty(bitmap);
> if (features->has_unknown_dev_features) {
> - features->unknown_dev_features = bitmap;
> + features->unknown_dev_features = bitmap[0];
> + features->unknown_dev_features2 = bitmap[1];
> }
>
> return features;
> @@ -743,11 +779,11 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
> status->device_id = vdev->device_id;
> status->vhost_started = vdev->vhost_started;
> status->guest_features = qmp_decode_features(vdev->device_id,
> - vdev->guest_features);
> + vdev->guest_features_ex);
> status->host_features = qmp_decode_features(vdev->device_id,
> - vdev->host_features);
> + vdev->host_features_ex);
> status->backend_features = qmp_decode_features(vdev->device_id,
> - vdev->backend_features);
> + vdev->backend_features_ex);
>
> switch (vdev->device_endian) {
> case VIRTIO_DEVICE_ENDIAN_LITTLE:
> @@ -785,11 +821,12 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
> status->vhost_dev->nvqs = hdev->nvqs;
> status->vhost_dev->vq_index = hdev->vq_index;
> status->vhost_dev->features =
> - qmp_decode_features(vdev->device_id, hdev->features);
> + qmp_decode_features(vdev->device_id, hdev->features_ex);
> status->vhost_dev->acked_features =
> - qmp_decode_features(vdev->device_id, hdev->acked_features);
> + qmp_decode_features(vdev->device_id, hdev->acked_features_ex);
> status->vhost_dev->backend_features =
> - qmp_decode_features(vdev->device_id, hdev->backend_features);
> + qmp_decode_features(vdev->device_id, hdev->backend_features_ex);
> +
> status->vhost_dev->protocol_features =
> qmp_decode_protocols(hdev->protocol_features);
> status->vhost_dev->max_queues = hdev->max_queues;
> diff --git a/hw/virtio/virtio-qmp.h b/hw/virtio/virtio-qmp.h
> index 245a446a56..e0a1e49035 100644
> --- a/hw/virtio/virtio-qmp.h
> +++ b/hw/virtio/virtio-qmp.h
> @@ -18,6 +18,7 @@
> VirtIODevice *qmp_find_virtio_device(const char *path);
> VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap);
> VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap);
> -VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap);
> +VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
> + const uint64_t *bitmap);
>
> #endif
> diff --git a/qapi/virtio.json b/qapi/virtio.json
> index 9d652fe4a8..f2e2dd6e97 100644
> --- a/qapi/virtio.json
> +++ b/qapi/virtio.json
> @@ -490,14 +490,18 @@
> # unique features)
> #
> # @unknown-dev-features: Virtio device features bitmap that have not
> -# been decoded
> +# been decoded (bits 0-63)
> +#
> +# @unknown-dev-features2: Virtio device features bitmap that have not
> +# been decoded (bits 64-127)
This documentation should contain "(since 10.1)" as described in:
docs/devel/qapi-code-gen.rst
> #
> # Since: 7.2
> ##
> { 'struct': 'VirtioDeviceFeatures',
> 'data': { 'transports': [ 'str' ],
> '*dev-features': [ 'str' ],
> - '*unknown-dev-features': 'uint64' } }
> + '*unknown-dev-features': 'uint64',
> + '*unknown-dev-features2': 'uint64' } }
>
> ##
> # @VirtQueueStatus:
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 03/13] virtio: introduce extended features type
2025-07-20 10:41 ` Akihiko Odaki
@ 2025-07-21 7:33 ` Paolo Abeni
2025-07-21 7:49 ` Akihiko Odaki
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 7:33 UTC (permalink / raw)
To: Akihiko Odaki, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/20/25 12:41 PM, Akihiko Odaki wrote:
> On 2025/07/18 17:52, Paolo Abeni wrote:
>> diff --git a/include/hw/virtio/virtio-features.h b/include/hw/virtio/virtio-features.h
>> new file mode 100644
>> index 0000000000..68e326e3e8
>> --- /dev/null
>> +++ b/include/hw/virtio/virtio-features.h
>> @@ -0,0 +1,123 @@
>> +/*
>> + * Virtio features helpers
>> + *
>> + * Copyright 2025 Red Hat, Inc.
>> + *
>> + * SPDX-License-Identifier: GPL-2.0-or-later
>> + */
>> +
>> +#ifndef QEMU_VIRTIO_FEATURES_H
>> +#define QEMU_VIRTIO_FEATURES_H
>> +
>> +#include "qemu/bitops.h"
>> +
>> +#define VIRTIO_FEATURES_FMT "%016"PRIx64"%016"PRIx64
>> +#define VIRTIO_FEATURES_PR(f) (f)[1], (f)[0]
>> +
>> +#define VIRTIO_FEATURES_MAX 128
>> +#define VIRTIO_BIT(b) BIT_ULL((b) % 64)
>> +#define VIRTIO_DWORD(b) ((b) >> 6)
>> +#define VIRTIO_FEATURES_WORDS (VIRTIO_FEATURES_MAX >> 5)
>> +#define VIRTIO_FEATURES_DWORDS (VIRTIO_FEATURES_WORDS >> 1)
>
> These shifts are better to be replaced with division for clarity;
> BIT_WORD() is a good example.
>
> "WORD" and "DWORD" should be avoided due to contradicting definitions
> common in QEMU as described at:
> https://lore.kernel.org/qemu-devel/aab8c434-364e-4305-9d8b-943eb0c98406@rsg.ci.i.u-tokyo.ac.jp/
>
> BITS_TO_U32S() is a good example this regard.
Ok, I'll rename:
VIRTIO_FEATURES_DWORDS -> VIRTIO_U64_PER_FEATURES
VIRTIO_FEATURES_WORDS -> VIRTIO_U32_PER_FEATURES
VIRTIO_DWORD -> VIRTIO_FEATURES_U64
(imho VIRTIO_U64 would sound ambiguous)
VIRTIO_BIT -> VIRTIO_FEATURES_BIT
(for consistency with the previous one)
>> +static inline bool virtio_features_equal(const uint64_t *f1,
>> + const uint64_t *f2)
>> +{
>> + return !memcmp(f1, f2, sizeof(uint64_t) * VIRTIO_FEATURES_DWORDS);
>> +}
>> +
>> +static inline bool virtio_features_use_extended(const uint64_t *features)
>
> "extended" should be abbreviated as "ex" for consistency.
Ok.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-21 7:23 ` Akihiko Odaki
@ 2025-07-21 7:45 ` Stefano Garzarella
2025-07-21 8:04 ` Paolo Abeni
1 sibling, 0 replies; 54+ messages in thread
From: Stefano Garzarella @ 2025-07-21 7:45 UTC (permalink / raw)
To: Akihiko Odaki
Cc: Paolo Abeni, qemu-devel, Paolo Bonzini, Dmitry Fleytman,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On Mon, Jul 21, 2025 at 04:23:14PM +0900, Akihiko Odaki wrote:
>On 2025/07/18 17:52, Paolo Abeni wrote:
>>Extend the VirtioDeviceFeatures struct with an additional u64
>>to track unknown features in the 64-127 bit range and decode
>>the full virtio features spaces for vhost and virtio devices.
>>
>>Also add entries for the soon-to-be-supported virtio net GSO over
>>UDP features.
>>
>>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>>---
>>v2 -> v3:
>> - unknown-dev-features-dword2 -> unknown-dev-features2
>> - _array -> _ex
>> - fixed typos in entries description
>>
>>v1 -> v2:
>> - uint128_t -> uint64_t[]
>>---
>> hw/virtio/virtio-hmp-cmds.c | 3 +-
>> hw/virtio/virtio-qmp.c | 89 ++++++++++++++++++++++++++-----------
>> hw/virtio/virtio-qmp.h | 3 +-
>> qapi/virtio.json | 8 +++-
>> 4 files changed, 73 insertions(+), 30 deletions(-)
>>
[...]
>>diff --git a/qapi/virtio.json b/qapi/virtio.json
>>index 9d652fe4a8..f2e2dd6e97 100644
>>--- a/qapi/virtio.json
>>+++ b/qapi/virtio.json
>>@@ -490,14 +490,18 @@
>> # unique features)
>> #
>> # @unknown-dev-features: Virtio device features bitmap that have not
>>-# been decoded
>>+# been decoded (bits 0-63)
>>+#
>>+# @unknown-dev-features2: Virtio device features bitmap that have not
>>+# been decoded (bits 64-127)
>
>This documentation should contain "(since 10.1)" as described in:
>docs/devel/qapi-code-gen.rst
Good catch!
BTW 10.1 is already in soft-freeze, so I guess this will land in the
next cycle, so I'd suggest to use 10.2
Thanks,
Stefano
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 03/13] virtio: introduce extended features type
2025-07-21 7:33 ` Paolo Abeni
@ 2025-07-21 7:49 ` Akihiko Odaki
2025-07-21 8:45 ` Paolo Abeni
0 siblings, 1 reply; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-21 7:49 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/21 16:33, Paolo Abeni wrote:
> On 7/20/25 12:41 PM, Akihiko Odaki wrote:
>> On 2025/07/18 17:52, Paolo Abeni wrote:
>>> diff --git a/include/hw/virtio/virtio-features.h b/include/hw/virtio/virtio-features.h
>>> new file mode 100644
>>> index 0000000000..68e326e3e8
>>> --- /dev/null
>>> +++ b/include/hw/virtio/virtio-features.h
>>> @@ -0,0 +1,123 @@
>>> +/*
>>> + * Virtio features helpers
>>> + *
>>> + * Copyright 2025 Red Hat, Inc.
>>> + *
>>> + * SPDX-License-Identifier: GPL-2.0-or-later
>>> + */
>>> +
>>> +#ifndef QEMU_VIRTIO_FEATURES_H
>>> +#define QEMU_VIRTIO_FEATURES_H
>>> +
>>> +#include "qemu/bitops.h"
>>> +
>>> +#define VIRTIO_FEATURES_FMT "%016"PRIx64"%016"PRIx64
>>> +#define VIRTIO_FEATURES_PR(f) (f)[1], (f)[0]
>>> +
>>> +#define VIRTIO_FEATURES_MAX 128
>>> +#define VIRTIO_BIT(b) BIT_ULL((b) % 64)
>>> +#define VIRTIO_DWORD(b) ((b) >> 6)
>>> +#define VIRTIO_FEATURES_WORDS (VIRTIO_FEATURES_MAX >> 5)
>>> +#define VIRTIO_FEATURES_DWORDS (VIRTIO_FEATURES_WORDS >> 1)
>>
>> These shifts are better to be replaced with division for clarity;
>> BIT_WORD() is a good example.
>>
>> "WORD" and "DWORD" should be avoided due to contradicting definitions
>> common in QEMU as described at:
>> https://lore.kernel.org/qemu-devel/aab8c434-364e-4305-9d8b-943eb0c98406@rsg.ci.i.u-tokyo.ac.jp/
>>
>> BITS_TO_U32S() is a good example this regard.
>
> Ok, I'll rename:
> VIRTIO_FEATURES_DWORDS -> VIRTIO_U64_PER_FEATURES
> VIRTIO_FEATURES_WORDS -> VIRTIO_U32_PER_FEATURES
U64 and U32 should be plural (i.e., rename them into
VIRTIO_U64S_PER_FEATURES and VIRTIO_U32S_PER_FEATURES)
"PER_FEATURES" also sounds a bit awkward; BITS_PER_BYTE and
BITS_PER_LONG had singular after "per" so the unit was clear, but it is
not in this case.
I could think of several options:
- VIRTIO_U64S_PER_FEATURES (what you proposed + plural U64S)
- VIRTIO_FEATURES_U64S (closer to the previous version)
- VIRTIO_FEATURES_NU64S (like CPU_TEMP_BUF_NLONGS)
- VIRTIO_U64S_PER_FEATURE_BITMASK
They have downsides and upsides, and I don't have an idea what's the best.
Regards,
Akihiko Odaki
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 04/13] virtio: serialize extended features state
2025-07-20 10:44 ` Akihiko Odaki
@ 2025-07-21 7:51 ` Paolo Abeni
2025-07-21 7:55 ` Akihiko Odaki
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 7:51 UTC (permalink / raw)
To: Akihiko Odaki, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
7/20/25 12:44 PM, Akihiko Odaki wrote:
> On 2025/07/18 17:52, Paolo Abeni wrote:
>> @@ -3333,26 +3362,23 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>> vdev->device_endian = virtio_default_endian();
>> }
>>
>> - if (virtio_64bit_features_needed(vdev)) {
>> - /*
>> - * Subsection load filled vdev->guest_features. Run them
>> - * through virtio_set_features to sanity-check them against
>> - * host_features.
>> - */
>> - uint64_t features64 = vdev->guest_features;
>> - if (virtio_set_features_nocheck_maybe_co(vdev, features64) < 0) {
>> - error_report("Features 0x%" PRIx64 " unsupported. "
>> - "Allowed features: 0x%" PRIx64,
>> - features64, vdev->host_features);
>> - return -1;
>> - }
>> - } else {
>> - if (virtio_set_features_nocheck_maybe_co(vdev, features) < 0) {
>> - error_report("Features 0x%x unsupported. "
>> - "Allowed features: 0x%" PRIx64,
>> - features, vdev->host_features);
>> - return -1;
>> - }
>> + /*
>> + * Avoid silently breaking migration should the feature space increase
>> + * even more in the (far away) future
>> + */
>> + QEMU_BUILD_BUG_ON(VIRTIO_FEATURES_DWORDS != 2);
>
> This should be moved to around virtio_128bit_features_needed() and
> vmstate_virtio_128bit_features as they make assumptions on the feature
> space while virtio_load() doesn't.
Please allow me the following dumb question to try to avoid more
iterations for small changes. I guess moving the above inside
virtio_128bit_features_needed() would suffice?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 04/13] virtio: serialize extended features state
2025-07-21 7:51 ` Paolo Abeni
@ 2025-07-21 7:55 ` Akihiko Odaki
0 siblings, 0 replies; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-21 7:55 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/21 16:51, Paolo Abeni wrote:
> 7/20/25 12:44 PM, Akihiko Odaki wrote:
>> On 2025/07/18 17:52, Paolo Abeni wrote:
>>> @@ -3333,26 +3362,23 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>>> vdev->device_endian = virtio_default_endian();
>>> }
>>>
>>> - if (virtio_64bit_features_needed(vdev)) {
>>> - /*
>>> - * Subsection load filled vdev->guest_features. Run them
>>> - * through virtio_set_features to sanity-check them against
>>> - * host_features.
>>> - */
>>> - uint64_t features64 = vdev->guest_features;
>>> - if (virtio_set_features_nocheck_maybe_co(vdev, features64) < 0) {
>>> - error_report("Features 0x%" PRIx64 " unsupported. "
>>> - "Allowed features: 0x%" PRIx64,
>>> - features64, vdev->host_features);
>>> - return -1;
>>> - }
>>> - } else {
>>> - if (virtio_set_features_nocheck_maybe_co(vdev, features) < 0) {
>>> - error_report("Features 0x%x unsupported. "
>>> - "Allowed features: 0x%" PRIx64,
>>> - features, vdev->host_features);
>>> - return -1;
>>> - }
>>> + /*
>>> + * Avoid silently breaking migration should the feature space increase
>>> + * even more in the (far away) future
>>> + */
>>> + QEMU_BUILD_BUG_ON(VIRTIO_FEATURES_DWORDS != 2);
>>
>> This should be moved to around virtio_128bit_features_needed() and
>> vmstate_virtio_128bit_features as they make assumptions on the feature
>> space while virtio_load() doesn't.
>
> Please allow me the following dumb question to try to avoid more
> iterations for small changes. I guess moving the above inside
> virtio_128bit_features_needed() would suffice?
Thinking more carefully, now I think it should be moved before or after
vmstate_virtio.
virtio_128bit_features_needed() and vmstate_virtio_128bit_features will
work fine even if the feature bitmask becomes 196-bit. But
vmstate_virtio will need to be updated to contain
vmstate_virtio_196bit_features. So this assertion should be around
vmstate_virtio.
Regards,
Akihiko Odaki
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 08/13] qmp: update virtio features map to support extended features
2025-07-21 7:23 ` Akihiko Odaki
2025-07-21 7:45 ` Stefano Garzarella
@ 2025-07-21 8:04 ` Paolo Abeni
1 sibling, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 8:04 UTC (permalink / raw)
To: Akihiko Odaki, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/21/25 9:23 AM, Akihiko Odaki wrote:
> On 2025/07/18 17:52, Paolo Abeni wrote:
>> Extend the VirtioDeviceFeatures struct with an additional u64
>> to track unknown features in the 64-127 bit range and decode
>> the full virtio features spaces for vhost and virtio devices.
>>
>> Also add entries for the soon-to-be-supported virtio net GSO over
>> UDP features.
>>
>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>> ---
>> v2 -> v3:
>> - unknown-dev-features-dword2 -> unknown-dev-features2
>> - _array -> _ex
>> - fixed typos in entries description
>>
>> v1 -> v2:
>> - uint128_t -> uint64_t[]
>> ---
>> hw/virtio/virtio-hmp-cmds.c | 3 +-
>> hw/virtio/virtio-qmp.c | 89 ++++++++++++++++++++++++++-----------
>> hw/virtio/virtio-qmp.h | 3 +-
>> qapi/virtio.json | 8 +++-
>> 4 files changed, 73 insertions(+), 30 deletions(-)
>>
>> diff --git a/hw/virtio/virtio-hmp-cmds.c b/hw/virtio/virtio-hmp-cmds.c
>> index 7d8677bcf0..1daae482d3 100644
>> --- a/hw/virtio/virtio-hmp-cmds.c
>> +++ b/hw/virtio/virtio-hmp-cmds.c
>> @@ -74,7 +74,8 @@ static void hmp_virtio_dump_features(Monitor *mon,
>> }
>>
>> if (features->has_unknown_dev_features) {
>> - monitor_printf(mon, " unknown-features(0x%016"PRIx64")\n",
>> + monitor_printf(mon, " unknown-features(0x%016"PRIx64"%016"PRIx64")\n",
>> + features->unknown_dev_features2,
>> features->unknown_dev_features);
>> }
>> }
>> diff --git a/hw/virtio/virtio-qmp.c b/hw/virtio/virtio-qmp.c
>> index 3b6377cf0d..03c6163cf4 100644
>> --- a/hw/virtio/virtio-qmp.c
>> +++ b/hw/virtio/virtio-qmp.c
>> @@ -325,6 +325,20 @@ static const qmp_virtio_feature_map_t virtio_net_feature_map[] = {
>> FEATURE_ENTRY(VHOST_USER_F_PROTOCOL_FEATURES, \
>> "VHOST_USER_F_PROTOCOL_FEATURES: Vhost-user protocol features "
>> "negotiation supported"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, \
>> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>> + "UDP tunnel packets"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, \
>> + "VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO: Driver can receive GSO over "
>> + "UDP tunnel packets requiring checksum offload for the outer "
>> + "header"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, \
>> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>> + "UDP tunnel packets"),
>> + FEATURE_ENTRY(VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, \
>> + "VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO: Device can receive GSO over "
>> + "UDP tunnel packets requiring checksum offload for the outer "
>> + "header"),
>> { -1, "" }
>> };
>> #endif
>> @@ -510,6 +524,24 @@ static const qmp_virtio_feature_map_t virtio_gpio_feature_map[] = {
>> list; \
>> })
>>
>> +#define CONVERT_FEATURES_EX(type, map, bitmap) \
>> + ({ \
>> + type *list = NULL; \
>> + type *node; \
>> + for (i = 0; map[i].virtio_bit != -1; i++) { \
>> + bit = map[i].virtio_bit; \
>> + if (!virtio_has_feature_ex(bitmap, bit)) { \
>> + continue; \
>> + } \
>> + node = g_new0(type, 1); \
>> + node->value = g_strdup(map[i].feature_desc); \
>> + node->next = list; \
>> + list = node; \
>> + virtio_clear_feature_ex(bitmap, bit); \
>> + } \
>> + list; \
>> + })
>> +
>> VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap)
>> {
>> VirtioDeviceStatus *status;
>> @@ -545,109 +577,112 @@ VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap)
>> return vhu_protocols;
>> }
>>
>> -VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
>> +VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
>> + const uint64_t *bmap)
>> {
>> + uint64_t bitmap[VIRTIO_FEATURES_DWORDS];
>> VirtioDeviceFeatures *features;
>> uint64_t bit;
>> int i;
>>
>> + virtio_features_copy(bitmap, bmap);
>> features = g_new0(VirtioDeviceFeatures, 1);
>> features->has_dev_features = true;
>>
>> /* transport features */
>> - features->transports = CONVERT_FEATURES(strList, virtio_transport_map, 0,
>> - bitmap);
>> + features->transports = CONVERT_FEATURES_EX(strList, virtio_transport_map,
>> + bitmap);
>>
>> /* device features */
>> switch (device_id) {
>> #ifdef CONFIG_VIRTIO_SERIAL
>> case VIRTIO_ID_CONSOLE:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_serial_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_serial_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_BLK
>> case VIRTIO_ID_BLOCK:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_blk_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_blk_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_GPU
>> case VIRTIO_ID_GPU:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_gpu_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_gpu_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_NET
>> case VIRTIO_ID_NET:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_net_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_net_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_SCSI
>> case VIRTIO_ID_SCSI:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_scsi_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_scsi_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_BALLOON
>> case VIRTIO_ID_BALLOON:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_balloon_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_balloon_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_IOMMU
>> case VIRTIO_ID_IOMMU:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_iommu_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_iommu_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_INPUT
>> case VIRTIO_ID_INPUT:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_input_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_input_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VHOST_USER_FS
>> case VIRTIO_ID_FS:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_fs_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_fs_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VHOST_VSOCK
>> case VIRTIO_ID_VSOCK:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_vsock_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_vsock_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_CRYPTO
>> case VIRTIO_ID_CRYPTO:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_crypto_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_crypto_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_MEM
>> case VIRTIO_ID_MEM:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_mem_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_mem_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_I2C_ADAPTER
>> case VIRTIO_ID_I2C_ADAPTER:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_i2c_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_i2c_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VIRTIO_RNG
>> case VIRTIO_ID_RNG:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_rng_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_rng_feature_map, bitmap);
>> break;
>> #endif
>> #ifdef CONFIG_VHOST_USER_GPIO
>> case VIRTIO_ID_GPIO:
>> features->dev_features =
>> - CONVERT_FEATURES(strList, virtio_gpio_feature_map, 0, bitmap);
>> + CONVERT_FEATURES_EX(strList, virtio_gpio_feature_map, bitmap);
>> break;
>> #endif
>> /* No features */
>> @@ -680,9 +715,10 @@ VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap)
>> g_assert_not_reached();
>> }
>>
>> - features->has_unknown_dev_features = bitmap != 0;
>> + features->has_unknown_dev_features = !virtio_features_empty(bitmap);
>> if (features->has_unknown_dev_features) {
>> - features->unknown_dev_features = bitmap;
>> + features->unknown_dev_features = bitmap[0];
>> + features->unknown_dev_features2 = bitmap[1];
>> }
>>
>> return features;
>> @@ -743,11 +779,11 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
>> status->device_id = vdev->device_id;
>> status->vhost_started = vdev->vhost_started;
>> status->guest_features = qmp_decode_features(vdev->device_id,
>> - vdev->guest_features);
>> + vdev->guest_features_ex);
>> status->host_features = qmp_decode_features(vdev->device_id,
>> - vdev->host_features);
>> + vdev->host_features_ex);
>> status->backend_features = qmp_decode_features(vdev->device_id,
>> - vdev->backend_features);
>> + vdev->backend_features_ex);
>>
>> switch (vdev->device_endian) {
>> case VIRTIO_DEVICE_ENDIAN_LITTLE:
>> @@ -785,11 +821,12 @@ VirtioStatus *qmp_x_query_virtio_status(const char *path, Error **errp)
>> status->vhost_dev->nvqs = hdev->nvqs;
>> status->vhost_dev->vq_index = hdev->vq_index;
>> status->vhost_dev->features =
>> - qmp_decode_features(vdev->device_id, hdev->features);
>> + qmp_decode_features(vdev->device_id, hdev->features_ex);
>> status->vhost_dev->acked_features =
>> - qmp_decode_features(vdev->device_id, hdev->acked_features);
>> + qmp_decode_features(vdev->device_id, hdev->acked_features_ex);
>> status->vhost_dev->backend_features =
>> - qmp_decode_features(vdev->device_id, hdev->backend_features);
>> + qmp_decode_features(vdev->device_id, hdev->backend_features_ex);
>> +
>> status->vhost_dev->protocol_features =
>> qmp_decode_protocols(hdev->protocol_features);
>> status->vhost_dev->max_queues = hdev->max_queues;
>> diff --git a/hw/virtio/virtio-qmp.h b/hw/virtio/virtio-qmp.h
>> index 245a446a56..e0a1e49035 100644
>> --- a/hw/virtio/virtio-qmp.h
>> +++ b/hw/virtio/virtio-qmp.h
>> @@ -18,6 +18,7 @@
>> VirtIODevice *qmp_find_virtio_device(const char *path);
>> VirtioDeviceStatus *qmp_decode_status(uint8_t bitmap);
>> VhostDeviceProtocols *qmp_decode_protocols(uint64_t bitmap);
>> -VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id, uint64_t bitmap);
>> +VirtioDeviceFeatures *qmp_decode_features(uint16_t device_id,
>> + const uint64_t *bitmap);
>>
>> #endif
>> diff --git a/qapi/virtio.json b/qapi/virtio.json
>> index 9d652fe4a8..f2e2dd6e97 100644
>> --- a/qapi/virtio.json
>> +++ b/qapi/virtio.json
>> @@ -490,14 +490,18 @@
>> # unique features)
>> #
>> # @unknown-dev-features: Virtio device features bitmap that have not
>> -# been decoded
>> +# been decoded (bits 0-63)
>> +#
>> +# @unknown-dev-features2: Virtio device features bitmap that have not
>> +# been decoded (bits 64-127)
>
> This documentation should contain "(since 10.1)" as described in:
> docs/devel/qapi-code-gen.rst
Just to be pedant, since there is a little ambiguity between the
examples in docs/devel/qapi-code-gen.rst and the actual code e.g. in
qapi/qom.json, I'll use:
# @unknown-dev-features2: Virtio device features bitmap that have not
# been decoded (bits 64-127) (since 10.2)
/P
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-21 2:53 ` Lei Yang
@ 2025-07-21 8:21 ` Paolo Abeni
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 8:21 UTC (permalink / raw)
To: Lei Yang
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Stefano Garzarella,
Akihiko Odaki, Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On 7/21/25 4:53 AM, Lei Yang wrote:
> On Fri, Jul 18, 2025 at 10:37 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>> On Fri, Jul 18, 2025 at 10:52:33AM +0200, Paolo Abeni wrote:
>>> Similar to virtio infra, vhost core maintains the features status
>>> in the full extended format and allows the devices to implement
>>> extended version of the getter/setter.
>>>
>>> Note that 'protocol_features' are not extended: they are only
>>> used by vhost-user, and the latter device is not going to implement
>>> extended features soon.
>>>
>>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>>> ---
>>> v2 -> v3:
>>> - fix compile warning
>>> - _array -> _ex
>>>
>>> v1 -> v2:
>>> - uint128_t -> uint64_t[]
>>> - add _ex() variant of features manipulation helpers
>>> ---
>>> hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
>>> include/hw/virtio/vhost-backend.h | 6 +++
>>> include/hw/virtio/vhost.h | 33 ++++++++++++--
>>> 3 files changed, 100 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>>> index c30ea1156e..85ae1e4d4c 100644
>>> --- a/hw/virtio/vhost.c
>>> +++ b/hw/virtio/vhost.c
>>> @@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
>>> static int vhost_dev_set_features(struct vhost_dev *dev,
>>> bool enable_log)
>>> {
>>> - uint64_t features = dev->acked_features;
>>> + uint64_t features[VIRTIO_FEATURES_DWORDS];
>>> int r;
>>> +
>>> + virtio_features_copy(features, dev->acked_features_ex);
>>> if (enable_log) {
>>> - features |= 0x1ULL << VHOST_F_LOG_ALL;
>>> + virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
>>> }
>>> if (!vhost_dev_has_iommu(dev)) {
>>> - features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
>>> + virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
>>> }
>>> if (dev->vhost_ops->vhost_force_iommu) {
>>> if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
>>> - features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
>>> + virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
>>> }
>>> }
>>> - r = dev->vhost_ops->vhost_set_features(dev, features);
>>> +
>
> Hi Paolo
>
>>> + if (virtio_features_use_extended(features) &&
>>> + !dev->vhost_ops->vhost_set_features_ex) {
>>> + VHOST_OPS_DEBUG(r, "extended features without device support");
>>> + r = -EINVAL;
>>> + goto out;
>>> + }
>
> As we discussed in version 2, this code should be changed to: [1],
> otherwise the problem mentioned last time will occur when compiling.
>
> [1] if (virtio_features_use_extended(features) &&
> !dev->vhost_ops->vhost_set_features_ex) {
> - VHOST_OPS_DEBUG(r, "extended features without device support");
> r = -EINVAL;
> + VHOST_OPS_DEBUG(r, "extended features without device support");
> goto out;
> }
Yes, I'm sorry, something went wrong here. Likely ENOCOFFEE or similar.
Will hopefully address for good in the next revision.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 03/13] virtio: introduce extended features type
2025-07-21 7:49 ` Akihiko Odaki
@ 2025-07-21 8:45 ` Paolo Abeni
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 8:45 UTC (permalink / raw)
To: Akihiko Odaki, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/21/25 9:49 AM, Akihiko Odaki wrote:
> On 2025/07/21 16:33, Paolo Abeni wrote:
>> On 7/20/25 12:41 PM, Akihiko Odaki wrote:
>>> On 2025/07/18 17:52, Paolo Abeni wrote:
>>>> diff --git a/include/hw/virtio/virtio-features.h b/include/hw/virtio/virtio-features.h
>>>> new file mode 100644
>>>> index 0000000000..68e326e3e8
>>>> --- /dev/null
>>>> +++ b/include/hw/virtio/virtio-features.h
>>>> @@ -0,0 +1,123 @@
>>>> +/*
>>>> + * Virtio features helpers
>>>> + *
>>>> + * Copyright 2025 Red Hat, Inc.
>>>> + *
>>>> + * SPDX-License-Identifier: GPL-2.0-or-later
>>>> + */
>>>> +
>>>> +#ifndef QEMU_VIRTIO_FEATURES_H
>>>> +#define QEMU_VIRTIO_FEATURES_H
>>>> +
>>>> +#include "qemu/bitops.h"
>>>> +
>>>> +#define VIRTIO_FEATURES_FMT "%016"PRIx64"%016"PRIx64
>>>> +#define VIRTIO_FEATURES_PR(f) (f)[1], (f)[0]
>>>> +
>>>> +#define VIRTIO_FEATURES_MAX 128
>>>> +#define VIRTIO_BIT(b) BIT_ULL((b) % 64)
>>>> +#define VIRTIO_DWORD(b) ((b) >> 6)
>>>> +#define VIRTIO_FEATURES_WORDS (VIRTIO_FEATURES_MAX >> 5)
>>>> +#define VIRTIO_FEATURES_DWORDS (VIRTIO_FEATURES_WORDS >> 1)
>>>
>>> These shifts are better to be replaced with division for clarity;
>>> BIT_WORD() is a good example.
>>>
>>> "WORD" and "DWORD" should be avoided due to contradicting definitions
>>> common in QEMU as described at:
>>> https://lore.kernel.org/qemu-devel/aab8c434-364e-4305-9d8b-943eb0c98406@rsg.ci.i.u-tokyo.ac.jp/
>>>
>>> BITS_TO_U32S() is a good example this regard.
>>
>> Ok, I'll rename:
>> VIRTIO_FEATURES_DWORDS -> VIRTIO_U64_PER_FEATURES
>> VIRTIO_FEATURES_WORDS -> VIRTIO_U32_PER_FEATURES
>
> U64 and U32 should be plural (i.e., rename them into
> VIRTIO_U64S_PER_FEATURES and VIRTIO_U32S_PER_FEATURES)
>
> "PER_FEATURES" also sounds a bit awkward; BITS_PER_BYTE and
> BITS_PER_LONG had singular after "per" so the unit was clear, but it is
> not in this case.
>
> I could think of several options:
> - VIRTIO_U64S_PER_FEATURES (what you proposed + plural U64S)
> - VIRTIO_FEATURES_U64S (closer to the previous version)
> - VIRTIO_FEATURES_NU64S (like CPU_TEMP_BUF_NLONGS)
> - VIRTIO_U64S_PER_FEATURE_BITMASK
>
> They have downsides and upsides, and I don't have an idea what's the best.
Indeed an optimal name here could be impossible to fit. IMHO
VIRTIO_FEATURES_NU64S is the best option among the above and I like it.
I'll use that one.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-18 11:17 ` Stefano Garzarella
@ 2025-07-21 8:48 ` Paolo Abeni
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-21 8:48 UTC (permalink / raw)
To: Stefano Garzarella
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Jason Wang, Sriram Yagnaraman, Michael S. Tsirkin, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 7/18/25 1:17 PM, Stefano Garzarella wrote:
> On Fri, Jul 18, 2025 at 10:52:38AM +0200, Paolo Abeni wrote:
>> @@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
>> rss_data.specified_hash_types,
>> VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
>> ON_OFF_AUTO_AUTO),
>> + DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
>> + DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
>> + DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
>> + DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
>
> IIUC, to avoid issue with live-migration, we should disable all of them
> in `hw_compat_10_0` in hw/core/machine.c (e.g. like `host_uso`,
> `guest_uso*` in hw_compat_8_1).
I guess the relevant compat entry should be 10.1, right? AFAICS such
entry should added with a separate/specific before 10.1 will be tagged.
I'll add the relevant the entries after that the hw_compat_10_1 will be
added.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 06/13] virtio-pci: implement support for extended features
2025-07-18 8:52 ` [PATCH RFC v3 06/13] virtio-pci: implement support for " Paolo Abeni
@ 2025-07-22 3:28 ` Jason Wang
2025-07-22 7:37 ` Paolo Abeni
0 siblings, 1 reply; 54+ messages in thread
From: Jason Wang @ 2025-07-22 3:28 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
>
> Extend the features configuration space to 128 bits, and allow the
> common read/write operation to access all of it.
>
> On migration, save the 128 bit version of the features only if the
> upper bits are non zero. Relay reset to clear all the feature
> space before load.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
This is a guest noticeable behaviour change. I wonder if we need a
command line option to enable and disable this feature for migration
compatibility.
Thanks
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-18 8:52 ` [PATCH RFC v3 07/13] vhost: add support for negotiating " Paolo Abeni
2025-07-18 14:36 ` Stefano Garzarella
@ 2025-07-22 3:32 ` Jason Wang
2025-07-22 16:55 ` Paolo Abeni
1 sibling, 1 reply; 54+ messages in thread
From: Jason Wang @ 2025-07-22 3:32 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
>
> Similar to virtio infra, vhost core maintains the features status
> in the full extended format and allows the devices to implement
> extended version of the getter/setter.
>
> Note that 'protocol_features' are not extended: they are only
> used by vhost-user, and the latter device is not going to implement
> extended features soon.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2 -> v3:
> - fix compile warning
> - _array -> _ex
>
> v1 -> v2:
> - uint128_t -> uint64_t[]
> - add _ex() variant of features manipulation helpers
> ---
> hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
> include/hw/virtio/vhost-backend.h | 6 +++
> include/hw/virtio/vhost.h | 33 ++++++++++++--
> 3 files changed, 100 insertions(+), 12 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index c30ea1156e..85ae1e4d4c 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
> static int vhost_dev_set_features(struct vhost_dev *dev,
> bool enable_log)
> {
> - uint64_t features = dev->acked_features;
> + uint64_t features[VIRTIO_FEATURES_DWORDS];
> int r;
> +
> + virtio_features_copy(features, dev->acked_features_ex);
> if (enable_log) {
> - features |= 0x1ULL << VHOST_F_LOG_ALL;
> + virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
> }
> if (!vhost_dev_has_iommu(dev)) {
> - features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
> + virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> }
> if (dev->vhost_ops->vhost_force_iommu) {
> if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
> - features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
> + virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> }
> }
> - r = dev->vhost_ops->vhost_set_features(dev, features);
> +
> + if (virtio_features_use_extended(features) &&
> + !dev->vhost_ops->vhost_set_features_ex) {
> + VHOST_OPS_DEBUG(r, "extended features without device support");
> + r = -EINVAL;
> + goto out;
> + }
> +
> + if (dev->vhost_ops->vhost_set_features_ex) {
> + r = dev->vhost_ops->vhost_set_features_ex(dev, features);
> + } else {
> + r = dev->vhost_ops->vhost_set_features(dev, features[0]);
> + }
> if (r < 0) {
> VHOST_OPS_DEBUG(r, "vhost_set_features failed");
> goto out;
> @@ -1506,12 +1520,27 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
> }
> }
>
> +static int vhost_dev_get_features(struct vhost_dev *hdev,
> + uint64_t *features)
> +{
> + uint64_t features64;
> + int r;
> +
> + if (hdev->vhost_ops->vhost_get_features_ex) {
> + return hdev->vhost_ops->vhost_get_features_ex(hdev, features);
> + }
> +
> + r = hdev->vhost_ops->vhost_get_features(hdev, &features64);
> + virtio_features_from_u64(features, features64);
> + return r;
> +}
Nit: let's have a vhost_dev_set_features() as well?
Thanks
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-18 8:52 ` [PATCH RFC v3 12/13] net: implement tunnel probing Paolo Abeni
2025-07-18 11:17 ` Stefano Garzarella
@ 2025-07-22 3:50 ` Jason Wang
2025-07-22 7:33 ` Paolo Abeni
2025-07-22 4:15 ` Akihiko Odaki
2 siblings, 1 reply; 54+ messages in thread
From: Jason Wang @ 2025-07-22 3:50 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Fri, Jul 18, 2025 at 4:54 PM Paolo Abeni <pabeni@redhat.com> wrote:
>
> Tap devices support GSO over UDP tunnel offload. Probe for such
> feature in a similar manner to other offloads.
>
> GSO over UDP tunnel needs to be enabled in addition to a "plain"
> offload (TSO or USO).
>
> No need to check separately for the outer header checksum offload:
> the kernel is going to support both of them or none.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2 -> v3:
> - use bool type for tap_probe_has_tunnel()
> - rebased on top of 2deec9ab7d ("virtio-net: Move
> virtio_net_get_features() down")
> - _array -> _ex
>
> v1 -> v2:
> - peer_has_tunnel return a bool
> - move TUN_F definition in net/tun-linux.h
> ---
> hw/net/virtio-net.c | 37 +++++++++++++++++++++++++++++++++++++
> include/net/net.h | 3 +++
> net/net.c | 9 +++++++++
> net/tap-bsd.c | 5 +++++
> net/tap-linux.c | 11 +++++++++++
> net/tap-linux.h | 9 +++++++++
> net/tap-solaris.c | 5 +++++
> net/tap-stub.c | 5 +++++
> net/tap.c | 11 +++++++++++
> net/tap_int.h | 1 +
> 10 files changed, 96 insertions(+)
>
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 53413ec4d5..70c85f7f77 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -649,6 +649,15 @@ static int peer_has_uso(VirtIONet *n)
> return qemu_has_uso(qemu_get_queue(n->nic)->peer);
> }
>
> +static bool peer_has_tunnel(VirtIONet *n)
Nit: it looks better to use peer_has_udp_gso_tunnel().
> +{
> + if (!peer_has_vnet_hdr(n)) {
> + return false;
> + }
> +
> + return qemu_has_tunnel(qemu_get_queue(n->nic)->peer);
> +}
> +
> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
> int version_1, int hash_report)
> {
> @@ -3070,6 +3079,13 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
>
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
> +
> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> }
>
> @@ -3083,6 +3099,15 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
> }
>
> + if (!peer_has_tunnel(n)) {
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
> + }
> +
> if (!get_vhost_net(nc->peer)) {
> if (!use_own_hash) {
> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> @@ -4137,6 +4162,10 @@ static const VMStateDescription vmstate_virtio_net = {
> .dev_unplug_pending = dev_unplug_pending,
> };
>
> +#define DEFINE_PROP_FEATURE(_name, _state, _field, _bit, _defval) \
> + DEFINE_PROP_BIT64(_name, _state, _field[VIRTIO_DWORD(_bit)], \
> + (_bit) % 64, _defval)
> +
> static const Property virtio_net_properties[] = {
> DEFINE_PROP_BIT64("csum", VirtIONet, host_features,
> VIRTIO_NET_F_CSUM, true),
> @@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
> rss_data.specified_hash_types,
> VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
> ON_OFF_AUTO_AUTO),
> + DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
> + DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
> + DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
> + DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
> };
Need compatibility work like:
d83f46d189 ("virtio-pci: compat page aligned ATS")
Thanks
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-18 8:52 ` [PATCH RFC v3 12/13] net: implement tunnel probing Paolo Abeni
2025-07-18 11:17 ` Stefano Garzarella
2025-07-22 3:50 ` Jason Wang
@ 2025-07-22 4:15 ` Akihiko Odaki
2 siblings, 0 replies; 54+ messages in thread
From: Akihiko Odaki @ 2025-07-22 4:15 UTC (permalink / raw)
To: Paolo Abeni, qemu-devel
Cc: Paolo Bonzini, Dmitry Fleytman, Jason Wang, Sriram Yagnaraman,
Michael S. Tsirkin, Stefano Garzarella, Cornelia Huck,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Eric Blake,
Markus Armbruster
On 2025/07/18 17:52, Paolo Abeni wrote:
> Tap devices support GSO over UDP tunnel offload. Probe for such
> feature in a similar manner to other offloads.
>
> GSO over UDP tunnel needs to be enabled in addition to a "plain"
> offload (TSO or USO).
>
> No need to check separately for the outer header checksum offload:
> the kernel is going to support both of them or none.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2 -> v3:
> - use bool type for tap_probe_has_tunnel()
> - rebased on top of 2deec9ab7d ("virtio-net: Move
> virtio_net_get_features() down")
> - _array -> _ex
>
> v1 -> v2:
> - peer_has_tunnel return a bool
> - move TUN_F definition in net/tun-linux.h
> ---
> hw/net/virtio-net.c | 37 +++++++++++++++++++++++++++++++++++++
> include/net/net.h | 3 +++
> net/net.c | 9 +++++++++
> net/tap-bsd.c | 5 +++++
> net/tap-linux.c | 11 +++++++++++
> net/tap-linux.h | 9 +++++++++
> net/tap-solaris.c | 5 +++++
> net/tap-stub.c | 5 +++++
> net/tap.c | 11 +++++++++++
> net/tap_int.h | 1 +
> 10 files changed, 96 insertions(+)
>
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 53413ec4d5..70c85f7f77 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -649,6 +649,15 @@ static int peer_has_uso(VirtIONet *n)
> return qemu_has_uso(qemu_get_queue(n->nic)->peer);
> }
>
> +static bool peer_has_tunnel(VirtIONet *n)
> +{
> + if (!peer_has_vnet_hdr(n)) {
> + return false;
> + }
> +
> + return qemu_has_tunnel(qemu_get_queue(n->nic)->peer);
> +}
> +
> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
> int version_1, int hash_report)
> {
> @@ -3070,6 +3079,13 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
>
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
> +
> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> }
>
> @@ -3083,6 +3099,15 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
> }
>
> + if (!peer_has_tunnel(n)) {
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
> + virtio_clear_feature_ex(features,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
> + }
> +
> if (!get_vhost_net(nc->peer)) {
> if (!use_own_hash) {
> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> @@ -4137,6 +4162,10 @@ static const VMStateDescription vmstate_virtio_net = {
> .dev_unplug_pending = dev_unplug_pending,
> };
>
> +#define DEFINE_PROP_FEATURE(_name, _state, _field, _bit, _defval) \
> + DEFINE_PROP_BIT64(_name, _state, _field[VIRTIO_DWORD(_bit)], \
> + (_bit) % 64, _defval)
I suggest moving this to: include/hw/virtio/virtio-features.h
> +
> static const Property virtio_net_properties[] = {
> DEFINE_PROP_BIT64("csum", VirtIONet, host_features,
> VIRTIO_NET_F_CSUM, true),
> @@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
> rss_data.specified_hash_types,
> VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
> ON_OFF_AUTO_AUTO),
> + DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
> + DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
> + DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
> + DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
> };
>
> static void virtio_net_class_init(ObjectClass *klass, const void *data)
> diff --git a/include/net/net.h b/include/net/net.h
> index 48ba333d02..9a9084690d 100644
> --- a/include/net/net.h
> +++ b/include/net/net.h
> @@ -65,6 +65,7 @@ typedef void (NetClientDestructor)(NetClientState *);
> typedef RxFilterInfo *(QueryRxFilter)(NetClientState *);
> typedef bool (HasUfo)(NetClientState *);
> typedef bool (HasUso)(NetClientState *);
> +typedef bool (HasTunnel)(NetClientState *);
> typedef bool (HasVnetHdr)(NetClientState *);
> typedef bool (HasVnetHdrLen)(NetClientState *, int);
> typedef void (SetOffload)(NetClientState *, const NetOffloads *);
> @@ -95,6 +96,7 @@ typedef struct NetClientInfo {
> NetPoll *poll;
> HasUfo *has_ufo;
> HasUso *has_uso;
> + HasTunnel *has_tunnel;
> HasVnetHdr *has_vnet_hdr;
> HasVnetHdrLen *has_vnet_hdr_len;
> SetOffload *set_offload;
> @@ -197,6 +199,7 @@ void qemu_set_info_str(NetClientState *nc,
> void qemu_format_nic_info_str(NetClientState *nc, uint8_t macaddr[6]);
> bool qemu_has_ufo(NetClientState *nc);
> bool qemu_has_uso(NetClientState *nc);
> +bool qemu_has_tunnel(NetClientState *nc);
> bool qemu_has_vnet_hdr(NetClientState *nc);
> bool qemu_has_vnet_hdr_len(NetClientState *nc, int len);
> void qemu_set_offload(NetClientState *nc, const NetOffloads *ol);
> diff --git a/net/net.c b/net/net.c
> index 63872b6855..9536184a0c 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -522,6 +522,15 @@ bool qemu_has_uso(NetClientState *nc)
> return nc->info->has_uso(nc);
> }
>
> +bool qemu_has_tunnel(NetClientState *nc)
> +{
> + if (!nc || !nc->info->has_tunnel) {
> + return false;
> + }
> +
> + return nc->info->has_tunnel(nc);
> +}
> +
> bool qemu_has_vnet_hdr(NetClientState *nc)
> {
> if (!nc || !nc->info->has_vnet_hdr) {
> diff --git a/net/tap-bsd.c b/net/tap-bsd.c
> index 86b6edee94..751d4c819c 100644
> --- a/net/tap-bsd.c
> +++ b/net/tap-bsd.c
> @@ -217,6 +217,11 @@ int tap_probe_has_uso(int fd)
> return 0;
> }
>
> +bool tap_probe_has_tunnel(int fd)
> +{
> + return false;
> +}
> +
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> }
> diff --git a/net/tap-linux.c b/net/tap-linux.c
> index a1c58f74f5..e2628be798 100644
> --- a/net/tap-linux.c
> +++ b/net/tap-linux.c
> @@ -196,6 +196,17 @@ int tap_probe_has_uso(int fd)
> return 1;
> }
>
> +bool tap_probe_has_tunnel(int fd)
> +{
> + unsigned offload;
> +
> + offload = TUN_F_CSUM | TUN_F_TSO4 | TUN_F_UDP_TUNNEL_GSO;
> + if (ioctl(fd, TUNSETOFFLOAD, offload) < 0) {
> + return false;
> + }
> + return true;
> +}
> +
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> if (ioctl(fd, TUNSETVNETHDRSZ, &len) == -1) {
> diff --git a/net/tap-linux.h b/net/tap-linux.h
> index 9a58cecb7f..8cd6b5874b 100644
> --- a/net/tap-linux.h
> +++ b/net/tap-linux.h
> @@ -53,4 +53,13 @@
> #define TUN_F_USO4 0x20 /* I can handle USO for IPv4 packets */
> #define TUN_F_USO6 0x40 /* I can handle USO for IPv6 packets */
>
> +/* I can handle TSO/USO for UDP tunneled packets */
> +#define TUN_F_UDP_TUNNEL_GSO 0x080
> +
> +/*
> + * I can handle TSO/USO for UDP tunneled packets requiring csum offload for
> + * the outer header
> + */
> +#define TUN_F_UDP_TUNNEL_GSO_CSUM 0x100
> +
> #endif /* QEMU_TAP_LINUX_H */
> diff --git a/net/tap-solaris.c b/net/tap-solaris.c
> index 833c066bee..ac1ae25761 100644
> --- a/net/tap-solaris.c
> +++ b/net/tap-solaris.c
> @@ -222,6 +222,11 @@ int tap_probe_has_uso(int fd)
> return 0;
> }
>
> +bool tap_probe_has_tunnel(int fd)
> +{
> + return false;
> +}
> +
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> }
> diff --git a/net/tap-stub.c b/net/tap-stub.c
> index 67d14ad4d5..f7a5e0c163 100644
> --- a/net/tap-stub.c
> +++ b/net/tap-stub.c
> @@ -52,6 +52,11 @@ int tap_probe_has_uso(int fd)
> return 0;
> }
>
> +bool tap_probe_has_tunnel(int fd)
> +{
> + return false;
> +}
> +
> void tap_fd_set_vnet_hdr_len(int fd, int len)
> {
> }
> diff --git a/net/tap.c b/net/tap.c
> index b49db19f83..23c6c118e7 100644
> --- a/net/tap.c
> +++ b/net/tap.c
> @@ -76,6 +76,7 @@ typedef struct TAPState {
> bool using_vnet_hdr;
> bool has_ufo;
> bool has_uso;
> + bool has_tunnel;
> bool enabled;
> VHostNetState *vhost_net;
> unsigned host_vnet_hdr_len;
> @@ -241,6 +242,14 @@ static bool tap_has_uso(NetClientState *nc)
> return s->has_uso;
> }
>
> +static bool tap_has_tunnel(NetClientState *nc)
> +{
> + TAPState *s = DO_UPCAST(TAPState, nc, nc);
> +
> + assert(nc->info->type == NET_CLIENT_DRIVER_TAP);
> + return s->has_tunnel;
> +}
> +
> static bool tap_has_vnet_hdr(NetClientState *nc)
> {
> TAPState *s = DO_UPCAST(TAPState, nc, nc);
> @@ -369,6 +378,7 @@ static NetClientInfo net_tap_info = {
> .cleanup = tap_cleanup,
> .has_ufo = tap_has_ufo,
> .has_uso = tap_has_uso,
> + .has_tunnel = tap_has_tunnel,
> .has_vnet_hdr = tap_has_vnet_hdr,
> .has_vnet_hdr_len = tap_has_vnet_hdr_len,
> .set_offload = tap_set_offload,
> @@ -398,6 +408,7 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
> s->using_vnet_hdr = false;
> s->has_ufo = tap_probe_has_ufo(s->fd);
> s->has_uso = tap_probe_has_uso(s->fd);
> + s->has_tunnel = tap_probe_has_tunnel(s->fd);
> s->enabled = true;
> tap_set_offload(&s->nc, &ol);
> /*
> diff --git a/net/tap_int.h b/net/tap_int.h
> index f8bbe1cb0c..b76a05044b 100644
> --- a/net/tap_int.h
> +++ b/net/tap_int.h
> @@ -38,6 +38,7 @@ void tap_set_sndbuf(int fd, const NetdevTapOptions *tap, Error **errp);
> int tap_probe_vnet_hdr(int fd, Error **errp);
> int tap_probe_has_ufo(int fd);
> int tap_probe_has_uso(int fd);
> +bool tap_probe_has_tunnel(int fd);
> void tap_fd_set_offload(int fd, const NetOffloads *ol);
> void tap_fd_set_vnet_hdr_len(int fd, int len);
> int tap_fd_set_vnet_le(int fd, int vnet_is_le);
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-22 3:50 ` Jason Wang
@ 2025-07-22 7:33 ` Paolo Abeni
2025-07-23 5:47 ` Jason Wang
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-22 7:33 UTC (permalink / raw)
To: Jason Wang
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On 7/22/25 5:50 AM, Jason Wang wrote:
> On Fri, Jul 18, 2025 at 4:54 PM Paolo Abeni <pabeni@redhat.com> wrote:
>> Tap devices support GSO over UDP tunnel offload. Probe for such
>> feature in a similar manner to other offloads.
>>
>> GSO over UDP tunnel needs to be enabled in addition to a "plain"
>> offload (TSO or USO).
>>
>> No need to check separately for the outer header checksum offload:
>> the kernel is going to support both of them or none.
>>
>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>> ---
>> v2 -> v3:
>> - use bool type for tap_probe_has_tunnel()
>> - rebased on top of 2deec9ab7d ("virtio-net: Move
>> virtio_net_get_features() down")
>> - _array -> _ex
>>
>> v1 -> v2:
>> - peer_has_tunnel return a bool
>> - move TUN_F definition in net/tun-linux.h
>> ---
>> hw/net/virtio-net.c | 37 +++++++++++++++++++++++++++++++++++++
>> include/net/net.h | 3 +++
>> net/net.c | 9 +++++++++
>> net/tap-bsd.c | 5 +++++
>> net/tap-linux.c | 11 +++++++++++
>> net/tap-linux.h | 9 +++++++++
>> net/tap-solaris.c | 5 +++++
>> net/tap-stub.c | 5 +++++
>> net/tap.c | 11 +++++++++++
>> net/tap_int.h | 1 +
>> 10 files changed, 96 insertions(+)
>>
>> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
>> index 53413ec4d5..70c85f7f77 100644
>> --- a/hw/net/virtio-net.c
>> +++ b/hw/net/virtio-net.c
>> @@ -649,6 +649,15 @@ static int peer_has_uso(VirtIONet *n)
>> return qemu_has_uso(qemu_get_queue(n->nic)->peer);
>> }
>>
>> +static bool peer_has_tunnel(VirtIONet *n)
>
> Nit: it looks better to use peer_has_udp_gso_tunnel().
>
>> +{
>> + if (!peer_has_vnet_hdr(n)) {
>> + return false;
>> + }
>> +
>> + return qemu_has_tunnel(qemu_get_queue(n->nic)->peer);
>> +}
>> +
>> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
>> int version_1, int hash_report)
>> {
>> @@ -3070,6 +3079,13 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
>> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
>> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
>>
>> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
>> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
>> + virtio_clear_feature_ex(features,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
>> + virtio_clear_feature_ex(features,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
>> +
>> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
>> }
>>
>> @@ -3083,6 +3099,15 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
>> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
>> }
>>
>> + if (!peer_has_tunnel(n)) {
>> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
>> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
>> + virtio_clear_feature_ex(features,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
>> + virtio_clear_feature_ex(features,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
>> + }
>> +
>> if (!get_vhost_net(nc->peer)) {
>> if (!use_own_hash) {
>> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
>> @@ -4137,6 +4162,10 @@ static const VMStateDescription vmstate_virtio_net = {
>> .dev_unplug_pending = dev_unplug_pending,
>> };
>>
>> +#define DEFINE_PROP_FEATURE(_name, _state, _field, _bit, _defval) \
>> + DEFINE_PROP_BIT64(_name, _state, _field[VIRTIO_DWORD(_bit)], \
>> + (_bit) % 64, _defval)
>> +
>> static const Property virtio_net_properties[] = {
>> DEFINE_PROP_BIT64("csum", VirtIONet, host_features,
>> VIRTIO_NET_F_CSUM, true),
>> @@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
>> rss_data.specified_hash_types,
>> VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
>> ON_OFF_AUTO_AUTO),
>> + DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
>> + DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
>> + DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
>> + DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
>> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
>> };
>
> Need compatibility work like:
>
> d83f46d189 ("virtio-pci: compat page aligned ATS")
If I read correctly the only missing pieces are the hw_compat_10_1
entries, am I correct?
If so, as mentioned here:
https://lists.gnu.org/archive/html/qemu-devel/2025-07/msg05032.html
I'll add the needed entries after that the hw_compat_10_1 will be created.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 06/13] virtio-pci: implement support for extended features
2025-07-22 3:28 ` Jason Wang
@ 2025-07-22 7:37 ` Paolo Abeni
2025-07-23 5:47 ` Jason Wang
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-22 7:37 UTC (permalink / raw)
To: Jason Wang
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On 7/22/25 5:28 AM, Jason Wang wrote:
> On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
>>
>> Extend the features configuration space to 128 bits, and allow the
>> common read/write operation to access all of it.
>>
>> On migration, save the 128 bit version of the features only if the
>> upper bits are non zero. Relay reset to clear all the feature
>> space before load.
>>
>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>
> This is a guest noticeable behaviour change. I wonder if we need a
> command line option to enable and disable this feature for migration
> compatibility.
This point is not clear to me, could you please elaborate a bit more? do
you mean we need i.e. a DEFINE_PROP_BOOL() or the like to enable the 128
bit space usage?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-22 3:32 ` Jason Wang
@ 2025-07-22 16:55 ` Paolo Abeni
2025-07-23 5:56 ` Jason Wang
0 siblings, 1 reply; 54+ messages in thread
From: Paolo Abeni @ 2025-07-22 16:55 UTC (permalink / raw)
To: Jason Wang
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On 7/22/25 5:32 AM, Jason Wang wrote:
> On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
>>
>> Similar to virtio infra, vhost core maintains the features status
>> in the full extended format and allows the devices to implement
>> extended version of the getter/setter.
>>
>> Note that 'protocol_features' are not extended: they are only
>> used by vhost-user, and the latter device is not going to implement
>> extended features soon.
>>
>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>> ---
>> v2 -> v3:
>> - fix compile warning
>> - _array -> _ex
>>
>> v1 -> v2:
>> - uint128_t -> uint64_t[]
>> - add _ex() variant of features manipulation helpers
>> ---
>> hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
>> include/hw/virtio/vhost-backend.h | 6 +++
>> include/hw/virtio/vhost.h | 33 ++++++++++++--
>> 3 files changed, 100 insertions(+), 12 deletions(-)
>>
>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>> index c30ea1156e..85ae1e4d4c 100644
>> --- a/hw/virtio/vhost.c
>> +++ b/hw/virtio/vhost.c
>> @@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
>> static int vhost_dev_set_features(struct vhost_dev *dev,
>> bool enable_log)
>> {
>> - uint64_t features = dev->acked_features;
>> + uint64_t features[VIRTIO_FEATURES_DWORDS];
>> int r;
>> +
>> + virtio_features_copy(features, dev->acked_features_ex);
>> if (enable_log) {
>> - features |= 0x1ULL << VHOST_F_LOG_ALL;
>> + virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
>> }
>> if (!vhost_dev_has_iommu(dev)) {
>> - features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
>> + virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
>> }
>> if (dev->vhost_ops->vhost_force_iommu) {
>> if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
>> - features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
>> + virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
>> }
>> }
>> - r = dev->vhost_ops->vhost_set_features(dev, features);
>> +
>> + if (virtio_features_use_extended(features) &&
>> + !dev->vhost_ops->vhost_set_features_ex) {
>> + VHOST_OPS_DEBUG(r, "extended features without device support");
>> + r = -EINVAL;
>> + goto out;
>> + }
>> +
>> + if (dev->vhost_ops->vhost_set_features_ex) {
>> + r = dev->vhost_ops->vhost_set_features_ex(dev, features);
>> + } else {
>> + r = dev->vhost_ops->vhost_set_features(dev, features[0]);
>> + }
>> if (r < 0) {
>> VHOST_OPS_DEBUG(r, "vhost_set_features failed");
>> goto out;
>> @@ -1506,12 +1520,27 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
>> }
>> }
>>
>> +static int vhost_dev_get_features(struct vhost_dev *hdev,
>> + uint64_t *features)
>> +{
>> + uint64_t features64;
>> + int r;
>> +
>> + if (hdev->vhost_ops->vhost_get_features_ex) {
>> + return hdev->vhost_ops->vhost_get_features_ex(hdev, features);
>> + }
>> +
>> + r = hdev->vhost_ops->vhost_get_features(hdev, &features64);
>> + virtio_features_from_u64(features, features64);
>> + return r;
>> +}
>
> Nit: let's have a vhost_dev_set_features() as well?
I guess you mean to factor out the
vhost_set_features_ex()/vhost_set_features() in a specific helper am I
correct?
Note that there is already a vhost_dev_set_features() function. It's
touched by the previous chunk. I opted for not creating the mentioned
helper to avoid some weird naming issues, as such helper would not lead
to any code deduplication.
Please LMK if you have strong opinion for a different choice.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 12/13] net: implement tunnel probing
2025-07-22 7:33 ` Paolo Abeni
@ 2025-07-23 5:47 ` Jason Wang
0 siblings, 0 replies; 54+ messages in thread
From: Jason Wang @ 2025-07-23 5:47 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Tue, Jul 22, 2025 at 3:33 PM Paolo Abeni <pabeni@redhat.com> wrote:
>
> On 7/22/25 5:50 AM, Jason Wang wrote:
> > On Fri, Jul 18, 2025 at 4:54 PM Paolo Abeni <pabeni@redhat.com> wrote:
> >> Tap devices support GSO over UDP tunnel offload. Probe for such
> >> feature in a similar manner to other offloads.
> >>
> >> GSO over UDP tunnel needs to be enabled in addition to a "plain"
> >> offload (TSO or USO).
> >>
> >> No need to check separately for the outer header checksum offload:
> >> the kernel is going to support both of them or none.
> >>
> >> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> >> ---
> >> v2 -> v3:
> >> - use bool type for tap_probe_has_tunnel()
> >> - rebased on top of 2deec9ab7d ("virtio-net: Move
> >> virtio_net_get_features() down")
> >> - _array -> _ex
> >>
> >> v1 -> v2:
> >> - peer_has_tunnel return a bool
> >> - move TUN_F definition in net/tun-linux.h
> >> ---
> >> hw/net/virtio-net.c | 37 +++++++++++++++++++++++++++++++++++++
> >> include/net/net.h | 3 +++
> >> net/net.c | 9 +++++++++
> >> net/tap-bsd.c | 5 +++++
> >> net/tap-linux.c | 11 +++++++++++
> >> net/tap-linux.h | 9 +++++++++
> >> net/tap-solaris.c | 5 +++++
> >> net/tap-stub.c | 5 +++++
> >> net/tap.c | 11 +++++++++++
> >> net/tap_int.h | 1 +
> >> 10 files changed, 96 insertions(+)
> >>
> >> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> >> index 53413ec4d5..70c85f7f77 100644
> >> --- a/hw/net/virtio-net.c
> >> +++ b/hw/net/virtio-net.c
> >> @@ -649,6 +649,15 @@ static int peer_has_uso(VirtIONet *n)
> >> return qemu_has_uso(qemu_get_queue(n->nic)->peer);
> >> }
> >>
> >> +static bool peer_has_tunnel(VirtIONet *n)
> >
> > Nit: it looks better to use peer_has_udp_gso_tunnel().
> >
> >> +{
> >> + if (!peer_has_vnet_hdr(n)) {
> >> + return false;
> >> + }
> >> +
> >> + return qemu_has_tunnel(qemu_get_queue(n->nic)->peer);
> >> +}
> >> +
> >> static void virtio_net_set_mrg_rx_bufs(VirtIONet *n, int mergeable_rx_bufs,
> >> int version_1, int hash_report)
> >> {
> >> @@ -3070,6 +3079,13 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> >> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO4);
> >> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
> >>
> >> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
> >> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
> >> + virtio_clear_feature_ex(features,
> >> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
> >> + virtio_clear_feature_ex(features,
> >> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
> >> +
> >> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> >> }
> >>
> >> @@ -3083,6 +3099,15 @@ static void virtio_net_get_features(VirtIODevice *vdev, uint64_t *features,
> >> virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_USO6);
> >> }
> >>
> >> + if (!peer_has_tunnel(n)) {
> >> + virtio_clear_feature_ex(features, VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO);
> >> + virtio_clear_feature_ex(features, VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO);
> >> + virtio_clear_feature_ex(features,
> >> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM);
> >> + virtio_clear_feature_ex(features,
> >> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM);
> >> + }
> >> +
> >> if (!get_vhost_net(nc->peer)) {
> >> if (!use_own_hash) {
> >> virtio_clear_feature_ex(features, VIRTIO_NET_F_HASH_REPORT);
> >> @@ -4137,6 +4162,10 @@ static const VMStateDescription vmstate_virtio_net = {
> >> .dev_unplug_pending = dev_unplug_pending,
> >> };
> >>
> >> +#define DEFINE_PROP_FEATURE(_name, _state, _field, _bit, _defval) \
> >> + DEFINE_PROP_BIT64(_name, _state, _field[VIRTIO_DWORD(_bit)], \
> >> + (_bit) % 64, _defval)
> >> +
> >> static const Property virtio_net_properties[] = {
> >> DEFINE_PROP_BIT64("csum", VirtIONet, host_features,
> >> VIRTIO_NET_F_CSUM, true),
> >> @@ -4245,6 +4274,14 @@ static const Property virtio_net_properties[] = {
> >> rss_data.specified_hash_types,
> >> VIRTIO_NET_HASH_REPORT_UDPv6_EX - 1,
> >> ON_OFF_AUTO_AUTO),
> >> + DEFINE_PROP_FEATURE("host_tunnel", VirtIONet, host_features_ex,
> >> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, true),
> >> + DEFINE_PROP_FEATURE("host_tunnel_csum", VirtIONet, host_features_ex,
> >> + VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM, true),
> >> + DEFINE_PROP_FEATURE("guest_tunnel", VirtIONet, host_features_ex,
> >> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO, true),
> >> + DEFINE_PROP_FEATURE("guest_tunnel_csum", VirtIONet, host_features_ex,
> >> + VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM, true),
> >> };
> >
> > Need compatibility work like:
> >
> > d83f46d189 ("virtio-pci: compat page aligned ATS")
>
> If I read correctly the only missing pieces are the hw_compat_10_1
> entries, am I correct?
>
> If so, as mentioned here:
>
> https://lists.gnu.org/archive/html/qemu-devel/2025-07/msg05032.html
Exactly.
>
> I'll add the needed entries after that the hw_compat_10_1 will be created.
>
> Thanks,
>
> Paolo
Thanks
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 06/13] virtio-pci: implement support for extended features
2025-07-22 7:37 ` Paolo Abeni
@ 2025-07-23 5:47 ` Jason Wang
2025-07-23 11:21 ` Paolo Abeni
0 siblings, 1 reply; 54+ messages in thread
From: Jason Wang @ 2025-07-23 5:47 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Tue, Jul 22, 2025 at 3:37 PM Paolo Abeni <pabeni@redhat.com> wrote:
>
> On 7/22/25 5:28 AM, Jason Wang wrote:
> > On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
> >>
> >> Extend the features configuration space to 128 bits, and allow the
> >> common read/write operation to access all of it.
> >>
> >> On migration, save the 128 bit version of the features only if the
> >> upper bits are non zero. Relay reset to clear all the feature
> >> space before load.
> >>
> >> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> >
> > This is a guest noticeable behaviour change. I wonder if we need a
> > command line option to enable and disable this feature for migration
> > compatibility.
>
> This point is not clear to me, could you please elaborate a bit more? do
> you mean we need i.e. a DEFINE_PROP_BOOL() or the like to enable the 128
> bit space usage?
Yes, or maybe have a way to let the device to enable it automatically
E.g when UDP GSO is enabled for virtio-net.
Thanks
>
> Thanks,
>
> Paolo
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 07/13] vhost: add support for negotiating extended features
2025-07-22 16:55 ` Paolo Abeni
@ 2025-07-23 5:56 ` Jason Wang
0 siblings, 0 replies; 54+ messages in thread
From: Jason Wang @ 2025-07-23 5:56 UTC (permalink / raw)
To: Paolo Abeni
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On Wed, Jul 23, 2025 at 12:55 AM Paolo Abeni <pabeni@redhat.com> wrote:
>
> On 7/22/25 5:32 AM, Jason Wang wrote:
> > On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
> >>
> >> Similar to virtio infra, vhost core maintains the features status
> >> in the full extended format and allows the devices to implement
> >> extended version of the getter/setter.
> >>
> >> Note that 'protocol_features' are not extended: they are only
> >> used by vhost-user, and the latter device is not going to implement
> >> extended features soon.
> >>
> >> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> >> ---
> >> v2 -> v3:
> >> - fix compile warning
> >> - _array -> _ex
> >>
> >> v1 -> v2:
> >> - uint128_t -> uint64_t[]
> >> - add _ex() variant of features manipulation helpers
> >> ---
> >> hw/virtio/vhost.c | 73 +++++++++++++++++++++++++++----
> >> include/hw/virtio/vhost-backend.h | 6 +++
> >> include/hw/virtio/vhost.h | 33 ++++++++++++--
> >> 3 files changed, 100 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> >> index c30ea1156e..85ae1e4d4c 100644
> >> --- a/hw/virtio/vhost.c
> >> +++ b/hw/virtio/vhost.c
> >> @@ -972,20 +972,34 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
> >> static int vhost_dev_set_features(struct vhost_dev *dev,
> >> bool enable_log)
> >> {
> >> - uint64_t features = dev->acked_features;
> >> + uint64_t features[VIRTIO_FEATURES_DWORDS];
> >> int r;
> >> +
> >> + virtio_features_copy(features, dev->acked_features_ex);
> >> if (enable_log) {
> >> - features |= 0x1ULL << VHOST_F_LOG_ALL;
> >> + virtio_add_feature_ex(features, VHOST_F_LOG_ALL);
> >> }
> >> if (!vhost_dev_has_iommu(dev)) {
> >> - features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
> >> + virtio_clear_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> >> }
> >> if (dev->vhost_ops->vhost_force_iommu) {
> >> if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
> >> - features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
> >> + virtio_add_feature_ex(features, VIRTIO_F_IOMMU_PLATFORM);
> >> }
> >> }
> >> - r = dev->vhost_ops->vhost_set_features(dev, features);
> >> +
> >> + if (virtio_features_use_extended(features) &&
> >> + !dev->vhost_ops->vhost_set_features_ex) {
> >> + VHOST_OPS_DEBUG(r, "extended features without device support");
> >> + r = -EINVAL;
> >> + goto out;
> >> + }
> >> +
> >> + if (dev->vhost_ops->vhost_set_features_ex) {
> >> + r = dev->vhost_ops->vhost_set_features_ex(dev, features);
> >> + } else {
> >> + r = dev->vhost_ops->vhost_set_features(dev, features[0]);
> >> + }
> >> if (r < 0) {
> >> VHOST_OPS_DEBUG(r, "vhost_set_features failed");
> >> goto out;
> >> @@ -1506,12 +1520,27 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
> >> }
> >> }
> >>
> >> +static int vhost_dev_get_features(struct vhost_dev *hdev,
> >> + uint64_t *features)
> >> +{
> >> + uint64_t features64;
> >> + int r;
> >> +
> >> + if (hdev->vhost_ops->vhost_get_features_ex) {
> >> + return hdev->vhost_ops->vhost_get_features_ex(hdev, features);
> >> + }
> >> +
> >> + r = hdev->vhost_ops->vhost_get_features(hdev, &features64);
> >> + virtio_features_from_u64(features, features64);
> >> + return r;
> >> +}
> >
> > Nit: let's have a vhost_dev_set_features() as well?
>
> I guess you mean to factor out the
> vhost_set_features_ex()/vhost_set_features() in a specific helper am I
> correct?
Yes.
>
> Note that there is already a vhost_dev_set_features() function. It's
> touched by the previous chunk. I opted for not creating the mentioned
> helper to avoid some weird naming issues, as such helper would not lead
> to any code deduplication.
I'm fine not having that then.
>
> Please LMK if you have strong opinion for a different choice.
>
> Thanks,
>
> Paolo
Thanks
>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: [PATCH RFC v3 06/13] virtio-pci: implement support for extended features
2025-07-23 5:47 ` Jason Wang
@ 2025-07-23 11:21 ` Paolo Abeni
0 siblings, 0 replies; 54+ messages in thread
From: Paolo Abeni @ 2025-07-23 11:21 UTC (permalink / raw)
To: Jason Wang
Cc: qemu-devel, Paolo Bonzini, Dmitry Fleytman, Akihiko Odaki,
Sriram Yagnaraman, Michael S. Tsirkin, Stefano Garzarella,
Cornelia Huck, Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione,
Eric Blake, Markus Armbruster
On 7/23/25 7:47 AM, Jason Wang wrote:
> On Tue, Jul 22, 2025 at 3:37 PM Paolo Abeni <pabeni@redhat.com> wrote:
>>
>> On 7/22/25 5:28 AM, Jason Wang wrote:
>>> On Fri, Jul 18, 2025 at 4:53 PM Paolo Abeni <pabeni@redhat.com> wrote:
>>>>
>>>> Extend the features configuration space to 128 bits, and allow the
>>>> common read/write operation to access all of it.
>>>>
>>>> On migration, save the 128 bit version of the features only if the
>>>> upper bits are non zero. Relay reset to clear all the feature
>>>> space before load.
>>>>
>>>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>>>
>>> This is a guest noticeable behaviour change. I wonder if we need a
>>> command line option to enable and disable this feature for migration
>>> compatibility.
>>
>> This point is not clear to me, could you please elaborate a bit more? do
>> you mean we need i.e. a DEFINE_PROP_BOOL() or the like to enable the 128
>> bit space usage?
>
> Yes, or maybe have a way to let the device to enable it automatically
> E.g when UDP GSO is enabled for virtio-net.
I think we can safely enable the extended space access (read/write) if
virtio_features_use_ex(vdev->host_features_ex). That should cover also
any eventual future addition of more extended features - even of
different devices.
/P
^ permalink raw reply [flat|nested] 54+ messages in thread
end of thread, other threads:[~2025-07-23 11:22 UTC | newest]
Thread overview: 54+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-18 8:52 [PATCH RFC v3 00/13] virtio: introduce support for GSO over UDP tunnel Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 01/13] net: bundle all offloads in a single struct Paolo Abeni
2025-07-20 10:31 ` Akihiko Odaki
2025-07-18 8:52 ` [PATCH RFC v3 02/13] linux-headers: Update to Linux ~v6.16-rc5 net-next Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 03/13] virtio: introduce extended features type Paolo Abeni
2025-07-20 10:41 ` Akihiko Odaki
2025-07-21 7:33 ` Paolo Abeni
2025-07-21 7:49 ` Akihiko Odaki
2025-07-21 8:45 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 04/13] virtio: serialize extended features state Paolo Abeni
2025-07-18 15:06 ` Stefano Garzarella
2025-07-20 10:44 ` Akihiko Odaki
2025-07-21 7:51 ` Paolo Abeni
2025-07-21 7:55 ` Akihiko Odaki
2025-07-18 8:52 ` [PATCH RFC v3 05/13] virtio: add support for negotiating extended features Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 06/13] virtio-pci: implement support for " Paolo Abeni
2025-07-22 3:28 ` Jason Wang
2025-07-22 7:37 ` Paolo Abeni
2025-07-23 5:47 ` Jason Wang
2025-07-23 11:21 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 07/13] vhost: add support for negotiating " Paolo Abeni
2025-07-18 14:36 ` Stefano Garzarella
2025-07-21 2:53 ` Lei Yang
2025-07-21 8:21 ` Paolo Abeni
2025-07-21 7:00 ` Paolo Abeni
2025-07-22 3:32 ` Jason Wang
2025-07-22 16:55 ` Paolo Abeni
2025-07-23 5:56 ` Jason Wang
2025-07-18 8:52 ` [PATCH RFC v3 08/13] qmp: update virtio features map to support " Paolo Abeni
2025-07-18 10:18 ` Stefano Garzarella
2025-07-18 10:23 ` Paolo Abeni
2025-07-18 10:28 ` Stefano Garzarella
2025-07-19 6:57 ` Markus Armbruster
2025-07-21 7:07 ` Paolo Abeni
2025-07-21 7:23 ` Akihiko Odaki
2025-07-21 7:45 ` Stefano Garzarella
2025-07-21 8:04 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 09/13] vhost-backend: implement extended features support Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 10/13] vhost-net: " Paolo Abeni
2025-07-18 13:01 ` Stefano Garzarella
2025-07-18 14:33 ` Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 11/13] virtio-net: " Paolo Abeni
2025-07-18 8:52 ` [PATCH RFC v3 12/13] net: implement tunnel probing Paolo Abeni
2025-07-18 11:17 ` Stefano Garzarella
2025-07-21 8:48 ` Paolo Abeni
2025-07-22 3:50 ` Jason Wang
2025-07-22 7:33 ` Paolo Abeni
2025-07-23 5:47 ` Jason Wang
2025-07-22 4:15 ` Akihiko Odaki
2025-07-18 8:52 ` [PATCH RFC v3 13/13] net: implement UDP tunnel features offloading Paolo Abeni
2025-07-18 13:22 ` Stefano Garzarella
2025-07-18 13:44 ` Paolo Abeni
2025-07-18 13:48 ` Stefano Garzarella
2025-07-18 15:21 ` Stefano Garzarella
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).