* [PATCH v4 01/14] net: introduce qemu_get_peer
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 02/14] vhost_net: use the function qemu_get_peer Cindy Lu
` (13 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
This is a small function that can get the peer
from given NetClientState and queue_index
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
include/net/net.h | 1 +
net/net.c | 7 +++++++
2 files changed, 8 insertions(+)
diff --git a/include/net/net.h b/include/net/net.h
index 39085d9444..e7ef42d62b 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -176,6 +176,7 @@ void hmp_info_network(Monitor *mon, const QDict *qdict);
void net_socket_rs_init(SocketReadState *rs,
SocketReadStateFinalize *finalize,
bool vnet_hdr);
+NetClientState *qemu_get_peer(NetClientState *nc, int queue_index);
/* NIC info */
diff --git a/net/net.c b/net/net.c
index d1130296e1..9099a327dd 100644
--- a/net/net.c
+++ b/net/net.c
@@ -325,6 +325,13 @@ void *qemu_get_nic_opaque(NetClientState *nc)
return nic->opaque;
}
+NetClientState *qemu_get_peer(NetClientState *nc, int queue_index)
+{
+ assert(nc != NULL);
+ NetClientState *ncs = nc + queue_index;
+ return ncs->peer;
+}
+
static void qemu_cleanup_net_client(NetClientState *nc)
{
QTAILQ_REMOVE(&net_clients, nc, next);
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 02/14] vhost_net: use the function qemu_get_peer
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
2020-07-01 14:55 ` [PATCH v4 01/14] net: introduce qemu_get_peer Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 03/14] virtio-bus: introduce queue_enabled method Cindy Lu
` (12 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, Laurent Vivier, shahafs, kevin.tian, parav,
vmireyno, cunming.liang, gdawar, jiri, xiao.w.wang, stefanha,
zhihong.wang, aadam, rdunlap, maxime.coquelin, lingshan.zhu
user the qemu_get_peer to replace the old process
Signed-off-by: Cindy Lu <lulu@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
hw/net/vhost_net.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 6b82803fa7..4096d64aaf 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -306,7 +306,9 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(dev)));
VirtioBusState *vbus = VIRTIO_BUS(qbus);
VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus);
+ struct vhost_net *net;
int r, e, i;
+ NetClientState *peer;
if (!k->set_guest_notifiers) {
error_report("binding does not support guest notifiers");
@@ -314,9 +316,9 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
}
for (i = 0; i < total_queues; i++) {
- struct vhost_net *net;
- net = get_vhost_net(ncs[i].peer);
+ peer = qemu_get_peer(ncs, i);
+ net = get_vhost_net(peer);
vhost_net_set_vq_index(net, i * 2);
/* Suppress the masking guest notifiers on vhost user
@@ -335,15 +337,16 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
}
for (i = 0; i < total_queues; i++) {
- r = vhost_net_start_one(get_vhost_net(ncs[i].peer), dev);
+ peer = qemu_get_peer(ncs, i);
+ r = vhost_net_start_one(get_vhost_net(peer), dev);
if (r < 0) {
goto err_start;
}
- if (ncs[i].peer->vring_enable) {
+ if (peer->vring_enable) {
/* restore vring enable state */
- r = vhost_set_vring_enable(ncs[i].peer, ncs[i].peer->vring_enable);
+ r = vhost_set_vring_enable(peer, peer->vring_enable);
if (r < 0) {
goto err_start;
@@ -355,7 +358,8 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
err_start:
while (--i >= 0) {
- vhost_net_stop_one(get_vhost_net(ncs[i].peer), dev);
+ peer = qemu_get_peer(ncs , i);
+ vhost_net_stop_one(get_vhost_net(peer), dev);
}
e = k->set_guest_notifiers(qbus->parent, total_queues * 2, false);
if (e < 0) {
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 03/14] virtio-bus: introduce queue_enabled method
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
2020-07-01 14:55 ` [PATCH v4 01/14] net: introduce qemu_get_peer Cindy Lu
2020-07-01 14:55 ` [PATCH v4 02/14] vhost_net: use the function qemu_get_peer Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 04/14] virtio-pci: implement " Cindy Lu
` (11 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
From: Jason Wang <jasowang@redhat.com>
This patch introduces queue_enabled() method which allows the
transport to implement its own way to report whether or not a queue is
enabled.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/virtio/virtio.c | 6 ++++++
include/hw/virtio/virtio-bus.h | 4 ++++
2 files changed, 10 insertions(+)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index cc9c9dc162..5bd2a2f621 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3286,6 +3286,12 @@ hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n)
bool virtio_queue_enabled(VirtIODevice *vdev, int n)
{
+ BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
+ VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
+
+ if (k->queue_enabled) {
+ return k->queue_enabled(qbus->parent, n);
+ }
return virtio_queue_get_desc_addr(vdev, n) != 0;
}
diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h
index 38c9399cd4..0f6f215925 100644
--- a/include/hw/virtio/virtio-bus.h
+++ b/include/hw/virtio/virtio-bus.h
@@ -83,6 +83,10 @@ typedef struct VirtioBusClass {
*/
int (*ioeventfd_assign)(DeviceState *d, EventNotifier *notifier,
int n, bool assign);
+ /*
+ * Whether queue number n is enabled.
+ */
+ bool (*queue_enabled)(DeviceState *d, int n);
/*
* Does the transport have variable vring alignment?
* (ie can it ever call virtio_queue_set_align()?)
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 04/14] virtio-pci: implement queue_enabled method
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (2 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 03/14] virtio-bus: introduce queue_enabled method Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 05/14] vhost: check the existence of vhost_set_iotlb_callback Cindy Lu
` (10 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
From: Jason Wang <jasowang@redhat.com>
With version 1, we can detect whether a queue is enabled via
queue_enabled.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/virtio/virtio-pci.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 7bc8c1c056..8554cf2a03 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1107,6 +1107,18 @@ static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
return pci_get_address_space(dev);
}
+static bool virtio_pci_queue_enabled(DeviceState *d, int n)
+{
+ VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
+ VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+
+ if (virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+ return proxy->vqs[vdev->queue_sel].enabled;
+ }
+
+ return virtio_queue_enabled(vdev, n);
+}
+
static int virtio_pci_add_mem_cap(VirtIOPCIProxy *proxy,
struct virtio_pci_cap *cap)
{
@@ -2064,6 +2076,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data)
k->ioeventfd_enabled = virtio_pci_ioeventfd_enabled;
k->ioeventfd_assign = virtio_pci_ioeventfd_assign;
k->get_dma_as = virtio_pci_get_dma_as;
+ k->queue_enabled = virtio_pci_queue_enabled;
}
static const TypeInfo virtio_pci_bus_info = {
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 05/14] vhost: check the existence of vhost_set_iotlb_callback
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (3 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 04/14] virtio-pci: implement " Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 06/14] vhost: introduce new VhostOps vhost_dev_start Cindy Lu
` (9 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
From: Jason Wang <jasowang@redhat.com>
Add the check of vhost_set_iotlb_callback
before calling
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/virtio/vhost.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 5fd25fe520..10304b583e 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1686,8 +1686,9 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
}
}
- if (vhost_dev_has_iommu(hdev)) {
- hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
+ if (vhost_dev_has_iommu(hdev) &&
+ hdev->vhost_ops->vhost_set_iotlb_callback) {
+ hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
/* Update used ring information for IOTLB to work correctly,
* vhost-kernel code requires for this.*/
@@ -1730,7 +1731,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
}
if (vhost_dev_has_iommu(hdev)) {
- hdev->vhost_ops->vhost_set_iotlb_callback(hdev, false);
+ if (hdev->vhost_ops->vhost_set_iotlb_callback) {
+ hdev->vhost_ops->vhost_set_iotlb_callback(hdev, false);
+ }
memory_listener_unregister(&hdev->iommu_listener);
}
vhost_log_put(hdev, true);
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 06/14] vhost: introduce new VhostOps vhost_dev_start
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (4 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 05/14] vhost: check the existence of vhost_set_iotlb_callback Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 07/14] vhost: implement vhost_dev_start method Cindy Lu
` (8 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
This patch introduces new VhostOps vhost_dev_start callback which allows the
vhost_net set the start/stop status to backend
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
include/hw/virtio/vhost-backend.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index 6f6670783f..b80f344cd6 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -112,6 +112,7 @@ typedef int (*vhost_get_inflight_fd_op)(struct vhost_dev *dev,
typedef int (*vhost_set_inflight_fd_op)(struct vhost_dev *dev,
struct vhost_inflight *inflight);
+typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);
typedef struct VhostOps {
VhostBackendType backend_type;
vhost_backend_init vhost_backend_init;
@@ -152,6 +153,7 @@ typedef struct VhostOps {
vhost_backend_mem_section_filter_op vhost_backend_mem_section_filter;
vhost_get_inflight_fd_op vhost_get_inflight_fd;
vhost_set_inflight_fd_op vhost_set_inflight_fd;
+ vhost_dev_start_op vhost_dev_start;
} VhostOps;
extern const VhostOps user_ops;
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 07/14] vhost: implement vhost_dev_start method
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (5 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 06/14] vhost: introduce new VhostOps vhost_dev_start Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 08/14] vhost: introduce new VhostOps vhost_vq_get_addr Cindy Lu
` (7 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
use the vhost_dev_start callback to send the status to backend
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/virtio/vhost.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 10304b583e..32809e54b5 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1685,7 +1685,12 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
goto fail_log;
}
}
-
+ if (hdev->vhost_ops->vhost_dev_start) {
+ r = hdev->vhost_ops->vhost_dev_start(hdev, true);
+ if (r) {
+ goto fail_log;
+ }
+ }
if (vhost_dev_has_iommu(hdev) &&
hdev->vhost_ops->vhost_set_iotlb_callback) {
hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
@@ -1723,6 +1728,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
/* should only be called after backend is connected */
assert(hdev->vhost_ops);
+ if (hdev->vhost_ops->vhost_dev_start) {
+ hdev->vhost_ops->vhost_dev_start(hdev, false);
+ }
for (i = 0; i < hdev->nvqs; ++i) {
vhost_virtqueue_stop(hdev,
vdev,
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 08/14] vhost: introduce new VhostOps vhost_vq_get_addr
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (6 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 07/14] vhost: implement vhost_dev_start method Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 09/14] vhost: implement vhost_vq_get_addr method Cindy Lu
` (6 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
This patch introduces new VhostOps vhost_vq_get_addr_op callback to get
the vring addr from the backend
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
include/hw/virtio/vhost-backend.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index b80f344cd6..fa84abac97 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -34,6 +34,7 @@ struct vhost_vring_state;
struct vhost_vring_addr;
struct vhost_scsi_target;
struct vhost_iotlb_msg;
+struct vhost_virtqueue;
typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque);
typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev);
@@ -113,6 +114,10 @@ typedef int (*vhost_set_inflight_fd_op)(struct vhost_dev *dev,
struct vhost_inflight *inflight);
typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);
+
+typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,
+ struct vhost_vring_addr *addr,
+ struct vhost_virtqueue *vq);
typedef struct VhostOps {
VhostBackendType backend_type;
vhost_backend_init vhost_backend_init;
@@ -154,6 +159,7 @@ typedef struct VhostOps {
vhost_get_inflight_fd_op vhost_get_inflight_fd;
vhost_set_inflight_fd_op vhost_set_inflight_fd;
vhost_dev_start_op vhost_dev_start;
+ vhost_vq_get_addr_op vhost_vq_get_addr;
} VhostOps;
extern const VhostOps user_ops;
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 09/14] vhost: implement vhost_vq_get_addr method
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (7 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 08/14] vhost: introduce new VhostOps vhost_vq_get_addr Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 10/14] vhost: introduce new VhostOps vhost_force_iommu Cindy Lu
` (5 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
use vhost_vq_get_addr callback to get the vq address from backend
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/virtio/vhost.c | 28 +++++++++++++++++++---------
include/hw/virtio/vhost-backend.h | 4 ++++
2 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 32809e54b5..1e083a8976 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -773,15 +773,25 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
struct vhost_virtqueue *vq,
unsigned idx, bool enable_log)
{
- struct vhost_vring_addr addr = {
- .index = idx,
- .desc_user_addr = (uint64_t)(unsigned long)vq->desc,
- .avail_user_addr = (uint64_t)(unsigned long)vq->avail,
- .used_user_addr = (uint64_t)(unsigned long)vq->used,
- .log_guest_addr = vq->used_phys,
- .flags = enable_log ? (1 << VHOST_VRING_F_LOG) : 0,
- };
- int r = dev->vhost_ops->vhost_set_vring_addr(dev, &addr);
+ struct vhost_vring_addr addr;
+ int r;
+ memset(&addr, 0, sizeof(struct vhost_vring_addr));
+
+ if (dev->vhost_ops->vhost_vq_get_addr) {
+ r = dev->vhost_ops->vhost_vq_get_addr(dev, &addr, vq);
+ if (r < 0) {
+ VHOST_OPS_DEBUG("vhost_vq_get_addr failed");
+ return -errno;
+ }
+ } else {
+ addr.desc_user_addr = (uint64_t)(unsigned long)vq->desc;
+ addr.avail_user_addr = (uint64_t)(unsigned long)vq->avail;
+ addr.used_user_addr = (uint64_t)(unsigned long)vq->used;
+ }
+ addr.index = idx;
+ addr.log_guest_addr = vq->used_phys;
+ addr.flags = enable_log ? (1 << VHOST_VRING_F_LOG) : 0;
+ r = dev->vhost_ops->vhost_set_vring_addr(dev, &addr);
if (r < 0) {
VHOST_OPS_DEBUG("vhost_set_vring_addr failed");
return -errno;
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index fa84abac97..bfc24207e2 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -118,6 +118,9 @@ typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);
typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,
struct vhost_vring_addr *addr,
struct vhost_virtqueue *vq);
+
+typedef int (*vhost_get_device_id_op)(struct vhost_dev *dev, uint32_t *dev_id);
+
typedef struct VhostOps {
VhostBackendType backend_type;
vhost_backend_init vhost_backend_init;
@@ -160,6 +163,7 @@ typedef struct VhostOps {
vhost_set_inflight_fd_op vhost_set_inflight_fd;
vhost_dev_start_op vhost_dev_start;
vhost_vq_get_addr_op vhost_vq_get_addr;
+ vhost_get_device_id_op vhost_get_device_id;
} VhostOps;
extern const VhostOps user_ops;
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 10/14] vhost: introduce new VhostOps vhost_force_iommu
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (8 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 09/14] vhost: implement vhost_vq_get_addr method Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 11/14] vhost: implement vhost_force_iommu method Cindy Lu
` (4 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
This patch introduces new VhostOps vhost_force_iommu callback
to force enable features bit VIRTIO_F_IOMMU_PLATFORM.
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
include/hw/virtio/vhost-backend.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index bfc24207e2..e7cb8d028c 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -121,6 +121,8 @@ typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,
typedef int (*vhost_get_device_id_op)(struct vhost_dev *dev, uint32_t *dev_id);
+typedef bool (*vhost_force_iommu_op)(struct vhost_dev *dev);
+
typedef struct VhostOps {
VhostBackendType backend_type;
vhost_backend_init vhost_backend_init;
@@ -164,6 +166,7 @@ typedef struct VhostOps {
vhost_dev_start_op vhost_dev_start;
vhost_vq_get_addr_op vhost_vq_get_addr;
vhost_get_device_id_op vhost_get_device_id;
+ vhost_force_iommu_op vhost_force_iommu;
} VhostOps;
extern const VhostOps user_ops;
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 11/14] vhost: implement vhost_force_iommu method
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (9 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 10/14] vhost: introduce new VhostOps vhost_force_iommu Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 12/14] vhost_net: introduce set_config & get_config Cindy Lu
` (3 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
use the vhost_force_iommu callback to force enable feature bit VIRTIO_F_IOMMU_PLATFORM
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/virtio/vhost.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 1e083a8976..1a1384e7a6 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -810,6 +810,11 @@ static int vhost_dev_set_features(struct vhost_dev *dev,
if (!vhost_dev_has_iommu(dev)) {
features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
}
+ if (dev->vhost_ops->vhost_force_iommu) {
+ if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
+ features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
+ }
+ }
r = dev->vhost_ops->vhost_set_features(dev, features);
if (r < 0) {
VHOST_OPS_DEBUG("vhost_set_features failed");
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 12/14] vhost_net: introduce set_config & get_config
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (10 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 11/14] vhost: implement vhost_force_iommu method Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 13/14] vhost-vdpa: introduce vhost-vdpa backend Cindy Lu
` (2 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
aadam, rdunlap, maxime.coquelin, lingshan.zhu
This patch introduces set_config & get_config method which allows
vhost_net set/get the config to backend
Signed-off-by: Cindy Lu <lulu@redhat.com>
---
hw/net/vhost_net-stub.c | 11 +++++++++++
hw/net/vhost_net.c | 10 ++++++++++
include/net/vhost_net.h | 5 +++++
3 files changed, 26 insertions(+)
diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index aac0e98228..a7f4252630 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -52,6 +52,17 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
return features;
}
+int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
+ uint32_t config_len)
+{
+ return 0;
+}
+int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
+ uint32_t offset, uint32_t size, uint32_t flags)
+{
+ return 0;
+}
+
void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
{
}
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 4096d64aaf..4561665f6b 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -110,6 +110,16 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
return vhost_get_features(&net->dev, vhost_net_get_feature_bits(net),
features);
}
+int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
+ uint32_t config_len)
+{
+ return vhost_dev_get_config(&net->dev, config, config_len);
+}
+int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
+ uint32_t offset, uint32_t size, uint32_t flags)
+{
+ return vhost_dev_set_config(&net->dev, data, offset, size, flags);
+}
void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
{
diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 77e47398c4..172b0051d8 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -28,6 +28,11 @@ void vhost_net_cleanup(VHostNetState *net);
uint64_t vhost_net_get_features(VHostNetState *net, uint64_t features);
void vhost_net_ack_features(VHostNetState *net, uint64_t features);
+int vhost_net_get_config(struct vhost_net *net, uint8_t *config,
+ uint32_t config_len);
+
+int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
+ uint32_t offset, uint32_t size, uint32_t flags);
bool vhost_net_virtqueue_pending(VHostNetState *net, int n);
void vhost_net_virtqueue_mask(VHostNetState *net, VirtIODevice *dev,
int idx, bool mask);
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 13/14] vhost-vdpa: introduce vhost-vdpa backend
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (11 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 12/14] vhost_net: introduce set_config & get_config Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-01 14:55 ` [PATCH v4 14/14] vhost-vdpa: introduce vhost-vdpa net client Cindy Lu
2020-07-03 7:30 ` [PATCH v4 00/14]vDPA support in qemu Jason Wang
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
Tiwei Bie, aadam, rdunlap, maxime.coquelin, lingshan.zhu
Currently we have 2 types of vhost backends in QEMU: vhost kernel and
vhost-user. The above patch provides a generic device for vDPA purpose,
this vDPA device exposes to user space a non-vendor-specific configuration
interface for setting up a vhost HW accelerator, this patch set introduces
a third vhost backend called vhost-vdpa based on the vDPA interface.
Vhost-vdpa usage:
qemu-system-x86_64 -cpu host -enable-kvm \
......
-netdev type=vhost-vdpa,vhostdev=/dev/vhost-vdpa-id,id=vhost-vdpa0 \
-device virtio-net-pci,netdev=vhost-vdpa0,page-per-vq=on \
Signed-off-by: Lingshan zhu <lingshan.zhu@intel.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Signed-off-by: Cindy Lu <lulu@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
configure | 21 ++
docs/interop/index.rst | 1 +
docs/interop/vhost-vdpa.rst | 17 ++
hw/net/vhost_net.c | 19 +-
hw/net/virtio-net.c | 19 ++
hw/virtio/Makefile.objs | 1 +
hw/virtio/vhost-backend.c | 6 +
hw/virtio/vhost-vdpa.c | 475 ++++++++++++++++++++++++++++++
include/hw/virtio/vhost-backend.h | 4 +-
include/hw/virtio/vhost-vdpa.h | 26 ++
include/hw/virtio/vhost.h | 7 +
qemu-options.hx | 12 +
12 files changed, 601 insertions(+), 7 deletions(-)
create mode 100644 docs/interop/vhost-vdpa.rst
create mode 100644 hw/virtio/vhost-vdpa.c
create mode 100644 include/hw/virtio/vhost-vdpa.h
diff --git a/configure b/configure
index 4a22dcd563..3db7f20185 100755
--- a/configure
+++ b/configure
@@ -1575,6 +1575,10 @@ for opt do
;;
--enable-vhost-user) vhost_user="yes"
;;
+ --disable-vhost-vdpa) vhost_vdpa="no"
+ ;;
+ --enable-vhost-vdpa) vhost_vdpa="yes"
+ ;;
--disable-vhost-kernel) vhost_kernel="no"
;;
--enable-vhost-kernel) vhost_kernel="yes"
@@ -1883,6 +1887,7 @@ disabled with --disable-FEATURE, default is enabled if available:
vhost-crypto vhost-user-crypto backend support
vhost-kernel vhost kernel backend support
vhost-user vhost-user backend support
+ vhost-vdpa vhost-vdpa kernel backend support
spice spice
rbd rados block device (rbd)
libiscsi iscsi support
@@ -2394,6 +2399,10 @@ test "$vhost_user" = "" && vhost_user=yes
if test "$vhost_user" = "yes" && test "$mingw32" = "yes"; then
error_exit "vhost-user isn't available on win32"
fi
+test "$vhost_vdpa" = "" && vhost_vdpa=$linux
+if test "$vhost_vdpa" = "yes" && test "$linux" != "yes"; then
+ error_exit "vhost-vdpa is only available on Linux"
+fi
test "$vhost_kernel" = "" && vhost_kernel=$linux
if test "$vhost_kernel" = "yes" && test "$linux" != "yes"; then
error_exit "vhost-kernel is only available on Linux"
@@ -2422,6 +2431,11 @@ test "$vhost_user_fs" = "" && vhost_user_fs=$vhost_user
if test "$vhost_user_fs" = "yes" && test "$vhost_user" = "no"; then
error_exit "--enable-vhost-user-fs requires --enable-vhost-user"
fi
+#vhost-vdpa backends
+test "$vhost_net_vdpa" = "" && vhost_net_vdpa=$vhost_vdpa
+if test "$vhost_net_vdpa" = "yes" && test "$vhost_vdpa" = "no"; then
+ error_exit "--enable-vhost-net-vdpa requires --enable-vhost-vdpa"
+fi
# OR the vhost-kernel and vhost-user values for simplicity
if test "$vhost_net" = ""; then
@@ -6936,6 +6950,7 @@ echo "vhost-scsi support $vhost_scsi"
echo "vhost-vsock support $vhost_vsock"
echo "vhost-user support $vhost_user"
echo "vhost-user-fs support $vhost_user_fs"
+echo "vhost-vdpa support $vhost_vdpa"
echo "Trace backends $trace_backends"
if have_backend "simple"; then
echo "Trace output file $trace_file-<pid>"
@@ -7437,6 +7452,9 @@ fi
if test "$vhost_net_user" = "yes" ; then
echo "CONFIG_VHOST_NET_USER=y" >> $config_host_mak
fi
+if test "$vhost_net_vdpa" = "yes" ; then
+ echo "CONFIG_VHOST_NET_VDPA=y" >> $config_host_mak
+fi
if test "$vhost_crypto" = "yes" ; then
echo "CONFIG_VHOST_CRYPTO=y" >> $config_host_mak
fi
@@ -7452,6 +7470,9 @@ fi
if test "$vhost_user" = "yes" ; then
echo "CONFIG_VHOST_USER=y" >> $config_host_mak
fi
+if test "$vhost_vdpa" = "yes" ; then
+ echo "CONFIG_VHOST_VDPA=y" >> $config_host_mak
+fi
if test "$vhost_user_fs" = "yes" ; then
echo "CONFIG_VHOST_USER_FS=y" >> $config_host_mak
fi
diff --git a/docs/interop/index.rst b/docs/interop/index.rst
index 049387ac6d..006f986420 100644
--- a/docs/interop/index.rst
+++ b/docs/interop/index.rst
@@ -20,3 +20,4 @@ Contents:
qemu-ga
vhost-user
vhost-user-gpu
+ vhost-vdpa
diff --git a/docs/interop/vhost-vdpa.rst b/docs/interop/vhost-vdpa.rst
new file mode 100644
index 0000000000..0c70ba01bc
--- /dev/null
+++ b/docs/interop/vhost-vdpa.rst
@@ -0,0 +1,17 @@
+=====================
+Vhost-vdpa Protocol
+=====================
+
+Introduction
+=============
+vDPA(Virtual data path acceleration) device is a device that uses
+a datapath which complies with the virtio specifications with vendor
+specific control path. vDPA devices can be both physically located on
+the hardware or emulated by software.
+
+This document describes the vDPA support in qemu
+
+Here is the kernel commit here
+https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4c8cf31885f69e86be0b5b9e6677a26797365e1d
+
+TODO : More information will add later
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 4561665f6b..e165594cf3 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -17,8 +17,10 @@
#include "net/net.h"
#include "net/tap.h"
#include "net/vhost-user.h"
+#include "net/vhost-vdpa.h"
#include "standard-headers/linux/vhost_types.h"
+#include "linux-headers/linux/vhost.h"
#include "hw/virtio/virtio-net.h"
#include "net/vhost_net.h"
#include "qemu/error-report.h"
@@ -33,12 +35,6 @@
#include "hw/virtio/vhost.h"
#include "hw/virtio/virtio-bus.h"
-struct vhost_net {
- struct vhost_dev dev;
- struct vhost_virtqueue vqs[2];
- int backend;
- NetClientState *nc;
-};
/* Features supported by host kernel. */
static const int kernel_feature_bits[] = {
@@ -96,6 +92,11 @@ static const int *vhost_net_get_feature_bits(struct vhost_net *net)
case NET_CLIENT_DRIVER_VHOST_USER:
feature_bits = user_feature_bits;
break;
+#ifdef CONFIG_VHOST_NET_VDPA
+ case NET_CLIENT_DRIVER_VHOST_VDPA:
+ feature_bits = vdpa_feature_bits;
+ break;
+#endif
default:
error_report("Feature bits not defined for this type: %d",
net->nc->info->type);
@@ -443,6 +444,12 @@ VHostNetState *get_vhost_net(NetClientState *nc)
vhost_net = vhost_user_get_vhost_net(nc);
assert(vhost_net);
break;
+#endif
+#ifdef CONFIG_VHOST_NET_VDPA
+ case NET_CLIENT_DRIVER_VHOST_VDPA:
+ vhost_net = vhost_vdpa_get_vhost_net(nc);
+ assert(vhost_net);
+ break;
#endif
default:
break;
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index aff67a92df..0a54b1910a 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -43,6 +43,7 @@
#include "monitor/qdev.h"
#include "hw/pci/pci.h"
#include "net_rx_pkt.h"
+#include "hw/virtio/vhost.h"
#define VIRTIO_NET_VM_VERSION 11
@@ -125,6 +126,8 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
VirtIONet *n = VIRTIO_NET(vdev);
struct virtio_net_config netcfg;
+ int ret = 0;
+ memset(&netcfg, 0 , sizeof(struct virtio_net_config));
virtio_stw_p(vdev, &netcfg.status, n->status);
virtio_stw_p(vdev, &netcfg.max_virtqueue_pairs, n->max_queues);
virtio_stw_p(vdev, &netcfg.mtu, n->net_conf.mtu);
@@ -138,6 +141,15 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
virtio_stl_p(vdev, &netcfg.supported_hash_types,
VIRTIO_NET_RSS_SUPPORTED_HASHES);
memcpy(config, &netcfg, n->config_size);
+
+ NetClientState *nc = qemu_get_queue(n->nic);
+ if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
+ ret = vhost_net_get_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg,
+ n->config_size);
+ if (ret != -1) {
+ memcpy(config, &netcfg, n->config_size);
+ }
+ }
}
static void virtio_net_set_config(VirtIODevice *vdev, const uint8_t *config)
@@ -153,6 +165,13 @@ static void virtio_net_set_config(VirtIODevice *vdev, const uint8_t *config)
memcpy(n->mac, netcfg.mac, ETH_ALEN);
qemu_format_nic_info_str(qemu_get_queue(n->nic), n->mac);
}
+
+ NetClientState *nc = qemu_get_queue(n->nic);
+ if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
+ vhost_net_set_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg,
+ 0, n->config_size,
+ VHOST_SET_CONFIG_TYPE_MASTER);
+ }
}
static bool virtio_net_started(VirtIONet *n, uint8_t status)
diff --git a/hw/virtio/Makefile.objs b/hw/virtio/Makefile.objs
index 13e75f171f..2b31e9b3dc 100644
--- a/hw/virtio/Makefile.objs
+++ b/hw/virtio/Makefile.objs
@@ -5,6 +5,7 @@ obj-y += virtio.o
obj-$(CONFIG_VHOST) += vhost.o vhost-backend.o
common-obj-$(call lnot,$(CONFIG_VHOST)) += vhost-stub.o
obj-$(CONFIG_VHOST_USER) += vhost-user.o
+obj-$(CONFIG_VHOST_VDPA) += vhost-vdpa.o
common-obj-$(CONFIG_VIRTIO_RNG) += virtio-rng.o
common-obj-$(CONFIG_VIRTIO_PCI) += virtio-pci.o
diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
index 48905383f8..782b1d67d9 100644
--- a/hw/virtio/vhost-backend.c
+++ b/hw/virtio/vhost-backend.c
@@ -15,6 +15,7 @@
#include "qemu/main-loop.h"
#include "standard-headers/linux/vhost_types.h"
+#include "hw/virtio/vhost-vdpa.h"
#ifdef CONFIG_VHOST_KERNEL
#include <linux/vhost.h>
#include <sys/ioctl.h>
@@ -285,6 +286,11 @@ int vhost_set_backend_type(struct vhost_dev *dev, VhostBackendType backend_type)
case VHOST_BACKEND_TYPE_USER:
dev->vhost_ops = &user_ops;
break;
+#endif
+#ifdef CONFIG_VHOST_VDPA
+ case VHOST_BACKEND_TYPE_VDPA:
+ dev->vhost_ops = &vdpa_ops;
+ break;
#endif
default:
error_report("Unknown vhost backend type");
diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
new file mode 100644
index 0000000000..6f2b01c25e
--- /dev/null
+++ b/hw/virtio/vhost-vdpa.c
@@ -0,0 +1,475 @@
+/*
+ * vhost-vdpa
+ *
+ * Copyright(c) 2017-2018 Intel Corporation.
+ * Copyright(c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include <linux/vhost.h>
+#include <linux/vfio.h>
+#include <sys/eventfd.h>
+#include <sys/ioctl.h>
+#include "hw/virtio/vhost.h"
+#include "hw/virtio/vhost-backend.h"
+#include "hw/virtio/virtio-net.h"
+#include "hw/virtio/vhost-vdpa.h"
+#include "qemu/main-loop.h"
+#include <linux/kvm.h>
+#include "sysemu/kvm.h"
+
+static bool vhost_vdpa_listener_skipped_section(MemoryRegionSection *section)
+{
+ return (!memory_region_is_ram(section->mr) &&
+ !memory_region_is_iommu(section->mr)) ||
+ /*
+ * Sizing an enabled 64-bit BAR can cause spurious mappings to
+ * addresses in the upper part of the 64-bit address space. These
+ * are never accessed by the CPU and beyond the address width of
+ * some IOMMU hardware. TODO: VDPA should tell us the IOMMU width.
+ */
+ section->offset_within_address_space & (1ULL << 63);
+}
+
+static int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size,
+ void *vaddr, bool readonly)
+{
+ struct vhost_msg_v2 msg;
+ int fd = v->device_fd;
+ int ret = 0;
+
+ msg.type = v->msg_type;
+ msg.iotlb.iova = iova;
+ msg.iotlb.size = size;
+ msg.iotlb.uaddr = (uint64_t)vaddr;
+ msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW;
+ msg.iotlb.type = VHOST_IOTLB_UPDATE;
+
+ if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) {
+ error_report("failed to write, fd=%d, errno=%d (%s)",
+ fd, errno, strerror(errno));
+ return -EIO ;
+ }
+
+ return ret;
+}
+
+static int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, hwaddr iova,
+ hwaddr size)
+{
+ struct vhost_msg_v2 msg;
+ int fd = v->device_fd;
+ int ret = 0;
+
+ msg.type = v->msg_type;
+ msg.iotlb.iova = iova;
+ msg.iotlb.size = size;
+ msg.iotlb.type = VHOST_IOTLB_INVALIDATE;
+
+ if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) {
+ error_report("failed to write, fd=%d, errno=%d (%s)",
+ fd, errno, strerror(errno));
+ return -EIO ;
+ }
+
+ return ret;
+}
+
+static void vhost_vdpa_listener_region_add(MemoryListener *listener,
+ MemoryRegionSection *section)
+{
+ struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener);
+ hwaddr iova;
+ Int128 llend, llsize;
+ void *vaddr;
+ int ret;
+
+ if (vhost_vdpa_listener_skipped_section(section)) {
+ return;
+ }
+
+ if (unlikely((section->offset_within_address_space & ~TARGET_PAGE_MASK) !=
+ (section->offset_within_region & ~TARGET_PAGE_MASK))) {
+ error_report("%s received unaligned region", __func__);
+ return;
+ }
+
+ iova = TARGET_PAGE_ALIGN(section->offset_within_address_space);
+ llend = int128_make64(section->offset_within_address_space);
+ llend = int128_add(llend, section->size);
+ llend = int128_and(llend, int128_exts64(TARGET_PAGE_MASK));
+
+ if (int128_ge(int128_make64(iova), llend)) {
+ return;
+ }
+
+ memory_region_ref(section->mr);
+
+ /* Here we assume that memory_region_is_ram(section->mr)==true */
+
+ vaddr = memory_region_get_ram_ptr(section->mr) +
+ section->offset_within_region +
+ (iova - section->offset_within_address_space);
+
+ llsize = int128_sub(llend, int128_make64(iova));
+
+ ret = vhost_vdpa_dma_map(v, iova, int128_get64(llsize),
+ vaddr, section->readonly);
+ if (ret) {
+ error_report("vhost vdpa map fail!");
+ if (memory_region_is_ram_device(section->mr)) {
+ /* Allow unexpected mappings not to be fatal for RAM devices */
+ error_report("map ram fail!");
+ return ;
+ }
+ goto fail;
+ }
+
+ return;
+
+fail:
+ if (memory_region_is_ram_device(section->mr)) {
+ error_report("failed to vdpa_dma_map. pci p2p may not work");
+ return;
+
+ }
+ /*
+ * On the initfn path, store the first error in the container so we
+ * can gracefully fail. Runtime, there's not much we can do other
+ * than throw a hardware error.
+ */
+ error_report("vhost-vdpa: DMA mapping failed, unable to continue");
+ return;
+
+}
+
+static void vhost_vdpa_listener_region_del(MemoryListener *listener,
+ MemoryRegionSection *section)
+{
+ struct vhost_vdpa *v = container_of(listener, struct vhost_vdpa, listener);
+ hwaddr iova;
+ Int128 llend, llsize;
+ int ret;
+ bool try_unmap = true;
+
+ if (vhost_vdpa_listener_skipped_section(section)) {
+ return;
+ }
+
+ if (unlikely((section->offset_within_address_space & ~TARGET_PAGE_MASK) !=
+ (section->offset_within_region & ~TARGET_PAGE_MASK))) {
+ error_report("%s received unaligned region", __func__);
+ return;
+ }
+
+ iova = TARGET_PAGE_ALIGN(section->offset_within_address_space);
+ llend = int128_make64(section->offset_within_address_space);
+ llend = int128_add(llend, section->size);
+ llend = int128_and(llend, int128_exts64(TARGET_PAGE_MASK));
+
+ if (int128_ge(int128_make64(iova), llend)) {
+ return;
+ }
+
+ llsize = int128_sub(llend, int128_make64(iova));
+
+ if (try_unmap) {
+ ret = vhost_vdpa_dma_unmap(v, iova, int128_get64(llsize));
+ if (ret) {
+ error_report("vhost_vdpa dma unmap error!");
+ }
+ }
+
+ memory_region_unref(section->mr);
+}
+/*
+ * IOTLB API is used by vhost-vpda which requires incremental updating
+ * of the mapping. So we can not use generic vhost memory listener which
+ * depends on the addnop().
+ */
+static const MemoryListener vhost_vdpa_memory_listener = {
+ .region_add = vhost_vdpa_listener_region_add,
+ .region_del = vhost_vdpa_listener_region_del,
+};
+
+static int vhost_vdpa_call(struct vhost_dev *dev, unsigned long int request,
+ void *arg)
+{
+ struct vhost_vdpa *v = dev->opaque;
+ int fd = v->device_fd;
+
+ assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA);
+
+ return ioctl(fd, request, arg);
+}
+
+static void vhost_vdpa_add_status(struct vhost_dev *dev, uint8_t status)
+{
+ uint8_t s;
+
+ if (vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &s)) {
+ return;
+ }
+
+ s |= status;
+
+ vhost_vdpa_call(dev, VHOST_VDPA_SET_STATUS, &s);
+}
+
+static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque)
+{
+ struct vhost_vdpa *v;
+ uint64_t features;
+ assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA);
+
+ v = opaque;
+ dev->opaque = opaque ;
+ vhost_vdpa_call(dev, VHOST_GET_FEATURES, &features);
+ dev->backend_features = features;
+ v->listener = vhost_vdpa_memory_listener;
+ v->msg_type = VHOST_IOTLB_MSG_V2;
+
+ vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
+ VIRTIO_CONFIG_S_DRIVER);
+
+ return 0;
+}
+
+static int vhost_vdpa_cleanup(struct vhost_dev *dev)
+{
+ struct vhost_vdpa *v;
+ assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA);
+ v = dev->opaque;
+ memory_listener_unregister(&v->listener);
+
+ dev->opaque = NULL;
+ return 0;
+}
+
+static int vhost_vdpa_memslots_limit(struct vhost_dev *dev)
+{
+ return INT_MAX;
+}
+
+static int vhost_vdpa_set_mem_table(struct vhost_dev *dev,
+ struct vhost_memory *mem)
+{
+
+ if (mem->padding) {
+ return -1;
+ }
+
+ return 0;
+}
+
+static int vhost_vdpa_set_features(struct vhost_dev *dev,
+ uint64_t features)
+{
+ int ret;
+ ret = vhost_vdpa_call(dev, VHOST_SET_FEATURES, &features);
+ uint8_t status = 0;
+ if (ret) {
+ return ret;
+ }
+ vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
+ vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status);
+
+ return !(status & VIRTIO_CONFIG_S_FEATURES_OK);
+}
+
+int vhost_vdpa_get_device_id(struct vhost_dev *dev,
+ uint32_t *device_id)
+{
+ return vhost_vdpa_call(dev, VHOST_VDPA_GET_DEVICE_ID, device_id);
+}
+
+static int vhost_vdpa_reset_device(struct vhost_dev *dev)
+{
+ uint8_t status = 0;
+
+ return vhost_vdpa_call(dev, VHOST_VDPA_SET_STATUS, &status);
+}
+
+static int vhost_vdpa_get_vq_index(struct vhost_dev *dev, int idx)
+{
+ assert(idx >= dev->vq_index && idx < dev->vq_index + dev->nvqs);
+
+ return idx - dev->vq_index;
+}
+
+static int vhost_vdpa_set_vring_ready(struct vhost_dev *dev)
+{
+ int i;
+ for (i = 0; i < dev->nvqs; ++i) {
+ struct vhost_vring_state state = {
+ .index = dev->vq_index + i,
+ .num = 1,
+ };
+ vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state);
+ }
+ return 0;
+}
+
+static int vhost_vdpa_set_config(struct vhost_dev *dev, const uint8_t *data,
+ uint32_t offset, uint32_t size,
+ uint32_t flags)
+{
+ struct vhost_vdpa_config *config;
+ int ret;
+ unsigned long config_size = offsetof(struct vhost_vdpa_config, buf);
+ config = g_malloc(size + config_size);
+ if (config == NULL) {
+ return -1;
+ }
+ config->off = offset;
+ config->len = size;
+ memcpy(config->buf, data, size);
+ ret = vhost_vdpa_call(dev, VHOST_VDPA_SET_CONFIG, config);
+ g_free(config);
+ return ret;
+}
+
+static int vhost_vdpa_get_config(struct vhost_dev *dev, uint8_t *config,
+ uint32_t config_len)
+{
+ struct vhost_vdpa_config *v_config;
+ unsigned long config_size = offsetof(struct vhost_vdpa_config, buf);
+ int ret;
+
+ v_config = g_malloc(config_len + config_size);
+ if (v_config == NULL) {
+ return -1;
+ }
+ v_config->len = config_len;
+ v_config->off = 0;
+ ret = vhost_vdpa_call(dev, VHOST_VDPA_GET_CONFIG, v_config);
+ memcpy(config, v_config->buf, config_len);
+ g_free(v_config);
+ return ret;
+ }
+
+static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started)
+{
+ struct vhost_vdpa *v = dev->opaque;
+ if (started) {
+ uint8_t status = 0;
+ memory_listener_register(&v->listener, &address_space_memory);
+ vhost_vdpa_set_vring_ready(dev);
+ vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
+ vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status);
+
+ return !(status & VIRTIO_CONFIG_S_DRIVER_OK);
+ } else {
+ vhost_vdpa_reset_device(dev);
+ vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
+ VIRTIO_CONFIG_S_DRIVER);
+ memory_listener_unregister(&v->listener);
+
+ return 0;
+ }
+}
+
+static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base,
+ struct vhost_log *log)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_LOG_BASE, &base);
+}
+
+static int vhost_vdpa_set_vring_addr(struct vhost_dev *dev,
+ struct vhost_vring_addr *addr)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_VRING_ADDR, addr);
+}
+
+static int vhost_vdpa_set_vring_num(struct vhost_dev *dev,
+ struct vhost_vring_state *ring)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_VRING_NUM, ring);
+}
+
+static int vhost_vdpa_set_vring_base(struct vhost_dev *dev,
+ struct vhost_vring_state *ring)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_VRING_BASE, ring);
+}
+
+static int vhost_vdpa_get_vring_base(struct vhost_dev *dev,
+ struct vhost_vring_state *ring)
+{
+ return vhost_vdpa_call(dev, VHOST_GET_VRING_BASE, ring);
+}
+
+static int vhost_vdpa_set_vring_kick(struct vhost_dev *dev,
+ struct vhost_vring_file *file)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_VRING_KICK, file);
+}
+
+static int vhost_vdpa_set_vring_call(struct vhost_dev *dev,
+ struct vhost_vring_file *file)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_VRING_CALL, file);
+}
+
+static int vhost_vdpa_get_features(struct vhost_dev *dev,
+ uint64_t *features)
+{
+ return vhost_vdpa_call(dev, VHOST_GET_FEATURES, features);
+}
+
+static int vhost_vdpa_set_owner(struct vhost_dev *dev)
+{
+ return vhost_vdpa_call(dev, VHOST_SET_OWNER, NULL);
+}
+
+static int vhost_vdpa_vq_get_addr(struct vhost_dev *dev,
+ struct vhost_vring_addr *addr, struct vhost_virtqueue *vq)
+{
+ assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA);
+ addr->desc_user_addr = (uint64_t)(unsigned long)vq->desc_phys;
+ addr->avail_user_addr = (uint64_t)(unsigned long)vq->avail_phys;
+ addr->used_user_addr = (uint64_t)(unsigned long)vq->used_phys;
+ return 0;
+}
+
+static bool vhost_vdpa_force_iommu(struct vhost_dev *dev)
+{
+ return true;
+}
+
+const VhostOps vdpa_ops = {
+ .backend_type = VHOST_BACKEND_TYPE_VDPA,
+ .vhost_backend_init = vhost_vdpa_init,
+ .vhost_backend_cleanup = vhost_vdpa_cleanup,
+ .vhost_set_log_base = vhost_vdpa_set_log_base,
+ .vhost_set_vring_addr = vhost_vdpa_set_vring_addr,
+ .vhost_set_vring_num = vhost_vdpa_set_vring_num,
+ .vhost_set_vring_base = vhost_vdpa_set_vring_base,
+ .vhost_get_vring_base = vhost_vdpa_get_vring_base,
+ .vhost_set_vring_kick = vhost_vdpa_set_vring_kick,
+ .vhost_set_vring_call = vhost_vdpa_set_vring_call,
+ .vhost_get_features = vhost_vdpa_get_features,
+ .vhost_set_owner = vhost_vdpa_set_owner,
+ .vhost_set_vring_endian = NULL,
+ .vhost_backend_memslots_limit = vhost_vdpa_memslots_limit,
+ .vhost_set_mem_table = vhost_vdpa_set_mem_table,
+ .vhost_set_features = vhost_vdpa_set_features,
+ .vhost_reset_device = vhost_vdpa_reset_device,
+ .vhost_get_vq_index = vhost_vdpa_get_vq_index,
+ .vhost_get_config = vhost_vdpa_get_config,
+ .vhost_set_config = vhost_vdpa_set_config,
+ .vhost_requires_shm_log = NULL,
+ .vhost_migration_done = NULL,
+ .vhost_backend_can_merge = NULL,
+ .vhost_net_set_mtu = NULL,
+ .vhost_set_iotlb_callback = NULL,
+ .vhost_send_device_iotlb_msg = NULL,
+ .vhost_dev_start = vhost_vdpa_dev_start,
+ .vhost_get_device_id = vhost_vdpa_get_device_id,
+ .vhost_vq_get_addr = vhost_vdpa_vq_get_addr,
+ .vhost_force_iommu = vhost_vdpa_force_iommu,
+};
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index e7cb8d028c..8825bd278f 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -17,7 +17,8 @@ typedef enum VhostBackendType {
VHOST_BACKEND_TYPE_NONE = 0,
VHOST_BACKEND_TYPE_KERNEL = 1,
VHOST_BACKEND_TYPE_USER = 2,
- VHOST_BACKEND_TYPE_MAX = 3,
+ VHOST_BACKEND_TYPE_VDPA = 3,
+ VHOST_BACKEND_TYPE_MAX = 4,
} VhostBackendType;
typedef enum VhostSetConfigType {
@@ -170,6 +171,7 @@ typedef struct VhostOps {
} VhostOps;
extern const VhostOps user_ops;
+extern const VhostOps vdpa_ops;
int vhost_set_backend_type(struct vhost_dev *dev,
VhostBackendType backend_type);
diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h
new file mode 100644
index 0000000000..6455663388
--- /dev/null
+++ b/include/hw/virtio/vhost-vdpa.h
@@ -0,0 +1,26 @@
+/*
+ * vhost-vdpa.h
+ *
+ * Copyright(c) 2017-2018 Intel Corporation.
+ * Copyright(c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef HW_VIRTIO_VHOST_VDPA_H
+#define HW_VIRTIO_VHOST_VDPA_H
+
+#include "hw/virtio/virtio.h"
+
+typedef struct vhost_vdpa {
+ int device_fd;
+ uint32_t msg_type;
+ MemoryListener listener;
+} VhostVDPA;
+
+extern AddressSpace address_space_memory;
+extern int vhost_vdpa_get_device_id(struct vhost_dev *dev,
+ uint32_t *device_id);
+#endif
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 085450c6f8..767a95ec0b 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -92,6 +92,13 @@ struct vhost_dev {
const VhostDevConfigOps *config_ops;
};
+struct vhost_net {
+ struct vhost_dev dev;
+ struct vhost_virtqueue vqs[2];
+ int backend;
+ NetClientState *nc;
+};
+
int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
VhostBackendType backend_type,
uint32_t busyloop_timeout);
diff --git a/qemu-options.hx b/qemu-options.hx
index 196f468786..fa1b19de4c 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2418,6 +2418,10 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
#ifdef CONFIG_POSIX
"-netdev vhost-user,id=str,chardev=dev[,vhostforce=on|off]\n"
" configure a vhost-user network, backed by a chardev 'dev'\n"
+#endif
+#ifdef __linux__
+ "-netdev vhost-vdpa,id=str,vhostdev=/path/to/dev\n"
+ " configure a vhost-vdpa network,Establish a vhost-vdpa netdev\n"
#endif
"-netdev hubport,id=str,hubid=n[,netdev=nd]\n"
" configure a hub port on the hub with ID 'n'\n", QEMU_ARCH_ALL)
@@ -2897,6 +2901,14 @@ SRST
-netdev type=vhost-user,id=net0,chardev=chr0 \
-device virtio-net-pci,netdev=net0
+``-netdev vhost-vdpa,vhostdev=/path/to/dev``
+ Establish a vhost-vdpa netdev.
+
+ vDPA device is a device that uses a datapath which complies with
+ the virtio specifications with a vendor specific control path.
+ vDPA devices can be both physically located on the hardware or
+ emulated by software.
+
``-netdev hubport,id=id,hubid=hubid[,netdev=nd]``
Create a hub port on the emulated hub with ID hubid.
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v4 14/14] vhost-vdpa: introduce vhost-vdpa net client
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (12 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 13/14] vhost-vdpa: introduce vhost-vdpa backend Cindy Lu
@ 2020-07-01 14:55 ` Cindy Lu
2020-07-03 7:30 ` [PATCH v4 00/14]vDPA support in qemu Jason Wang
14 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2020-07-01 14:55 UTC (permalink / raw)
To: mst, armbru, eblake, cohuck, jasowang
Cc: mhabets, qemu-devel, rob.miller, saugatm, lulu, hanand, hch,
eperezma, jgg, shahafs, kevin.tian, parav, vmireyno,
cunming.liang, gdawar, jiri, xiao.w.wang, stefanha, zhihong.wang,
Tiwei Bie, aadam, rdunlap, maxime.coquelin, lingshan.zhu
This patch set introduces a new net client type: vhost-vdpa.
vhost-vdpa net client will set up a vDPA device which is specified
by a "vhostdev" parameter.
Signed-off-by: Lingshan Zhu <lingshan.zhu@intel.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
Signed-off-by: Cindy Lu <lulu@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
include/net/vhost-vdpa.h | 22 ++++
net/Makefile.objs | 2 +-
net/clients.h | 2 +
net/net.c | 3 +
net/vhost-vdpa.c | 228 +++++++++++++++++++++++++++++++++++++++
qapi/net.json | 28 ++++-
6 files changed, 282 insertions(+), 3 deletions(-)
create mode 100644 include/net/vhost-vdpa.h
create mode 100644 net/vhost-vdpa.c
diff --git a/include/net/vhost-vdpa.h b/include/net/vhost-vdpa.h
new file mode 100644
index 0000000000..45e34b7cfc
--- /dev/null
+++ b/include/net/vhost-vdpa.h
@@ -0,0 +1,22 @@
+/*
+ * vhost-vdpa.h
+ *
+ * Copyright(c) 2017-2018 Intel Corporation.
+ * Copyright(c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef VHOST_VDPA_H
+#define VHOST_VDPA_H
+
+#define TYPE_VHOST_VDPA "vhost-vdpa"
+
+struct vhost_net *vhost_vdpa_get_vhost_net(NetClientState *nc);
+uint64_t vhost_vdpa_get_acked_features(NetClientState *nc);
+
+extern const int vdpa_feature_bits[];
+
+#endif /* VHOST_VDPA_H */
diff --git a/net/Makefile.objs b/net/Makefile.objs
index c5d076d19c..5ab45545db 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -26,7 +26,7 @@ tap-obj-$(CONFIG_SOLARIS) = tap-solaris.o
tap-obj-y ?= tap-stub.o
common-obj-$(CONFIG_POSIX) += tap.o $(tap-obj-y)
common-obj-$(CONFIG_WIN32) += tap-win32.o
-
+common-obj-$(CONFIG_VHOST_NET_VDPA) += vhost-vdpa.o
vde.o-libs = $(VDE_LIBS)
common-obj-$(CONFIG_CAN_BUS) += can/
diff --git a/net/clients.h b/net/clients.h
index a6ef267e19..92f9b59aed 100644
--- a/net/clients.h
+++ b/net/clients.h
@@ -61,4 +61,6 @@ int net_init_netmap(const Netdev *netdev, const char *name,
int net_init_vhost_user(const Netdev *netdev, const char *name,
NetClientState *peer, Error **errp);
+int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
+ NetClientState *peer, Error **errp);
#endif /* QEMU_NET_CLIENTS_H */
diff --git a/net/net.c b/net/net.c
index 9099a327dd..94dc546fb2 100644
--- a/net/net.c
+++ b/net/net.c
@@ -966,6 +966,9 @@ static int (* const net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
#ifdef CONFIG_VHOST_NET_USER
[NET_CLIENT_DRIVER_VHOST_USER] = net_init_vhost_user,
#endif
+#ifdef CONFIG_VHOST_NET_VDPA
+ [NET_CLIENT_DRIVER_VHOST_VDPA] = net_init_vhost_vdpa,
+#endif
#ifdef CONFIG_L2TPV3
[NET_CLIENT_DRIVER_L2TPV3] = net_init_l2tpv3,
#endif
diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
new file mode 100644
index 0000000000..bc0e0d2d35
--- /dev/null
+++ b/net/vhost-vdpa.c
@@ -0,0 +1,228 @@
+/*
+ * vhost-vdpa.c
+ *
+ * Copyright(c) 2017-2018 Intel Corporation.
+ * Copyright(c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "clients.h"
+#include "net/vhost_net.h"
+#include "net/vhost-vdpa.h"
+#include "hw/virtio/vhost-vdpa.h"
+#include "qemu/config-file.h"
+#include "qemu/error-report.h"
+#include "qemu/option.h"
+#include "qapi/error.h"
+#include <sys/ioctl.h>
+#include <err.h>
+#include "standard-headers/linux/virtio_net.h"
+#include "monitor/monitor.h"
+#include "hw/virtio/vhost.h"
+
+/* Todo:need to add the multiqueue support here */
+typedef struct VhostVDPAState {
+ NetClientState nc;
+ struct vhost_vdpa vhost_vdpa;
+ VHostNetState *vhost_net;
+ uint64_t acked_features;
+ bool started;
+} VhostVDPAState;
+
+const int vdpa_feature_bits[] = {
+ VIRTIO_F_NOTIFY_ON_EMPTY,
+ VIRTIO_RING_F_INDIRECT_DESC,
+ VIRTIO_RING_F_EVENT_IDX,
+ VIRTIO_F_ANY_LAYOUT,
+ VIRTIO_F_VERSION_1,
+ VIRTIO_NET_F_CSUM,
+ VIRTIO_NET_F_GUEST_CSUM,
+ VIRTIO_NET_F_GSO,
+ VIRTIO_NET_F_GUEST_TSO4,
+ VIRTIO_NET_F_GUEST_TSO6,
+ VIRTIO_NET_F_GUEST_ECN,
+ VIRTIO_NET_F_GUEST_UFO,
+ VIRTIO_NET_F_HOST_TSO4,
+ VIRTIO_NET_F_HOST_TSO6,
+ VIRTIO_NET_F_HOST_ECN,
+ VIRTIO_NET_F_HOST_UFO,
+ VIRTIO_NET_F_MRG_RXBUF,
+ VIRTIO_NET_F_MTU,
+ VIRTIO_F_IOMMU_PLATFORM,
+ VIRTIO_F_RING_PACKED,
+ VIRTIO_NET_F_GUEST_ANNOUNCE,
+ VHOST_INVALID_FEATURE_BIT
+};
+
+VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc)
+{
+ VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
+ assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+ return s->vhost_net;
+}
+
+uint64_t vhost_vdpa_get_acked_features(NetClientState *nc)
+{
+ VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
+ assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+ s->acked_features = vhost_net_get_acked_features(s->vhost_net);
+
+ return s->acked_features;
+}
+
+static int vhost_vdpa_net_check_device_id(struct vhost_net *net)
+{
+ uint32_t device_id;
+ int ret;
+ struct vhost_dev *hdev;
+
+ hdev = (struct vhost_dev *)&net->dev;
+ ret = hdev->vhost_ops->vhost_get_device_id(hdev, &device_id);
+ if (device_id != VIRTIO_ID_NET) {
+ return -ENOTSUP;
+ }
+ return ret;
+}
+
+static void vhost_vdpa_del(NetClientState *ncs)
+{
+ VhostVDPAState *s;
+ assert(ncs->info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+ s = DO_UPCAST(VhostVDPAState, nc, ncs);
+ if (s->vhost_net) {
+ vhost_net_cleanup(s->vhost_net);
+ }
+}
+
+static int vhost_vdpa_add(NetClientState *ncs, void *be)
+{
+ VhostNetOptions options;
+ struct vhost_net *net = NULL;
+ VhostVDPAState *s;
+ int ret;
+
+ options.backend_type = VHOST_BACKEND_TYPE_VDPA;
+ assert(ncs->info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+ s = DO_UPCAST(VhostVDPAState, nc, ncs);
+ options.net_backend = ncs;
+ options.opaque = be;
+ options.busyloop_timeout = 0;
+
+ net = vhost_net_init(&options);
+ if (!net) {
+ error_report("failed to init vhost_net for queue");
+ goto err;
+ }
+ if (s->vhost_net) {
+ vhost_net_cleanup(s->vhost_net);
+ g_free(s->vhost_net);
+ }
+ s->vhost_net = net;
+ ret = vhost_vdpa_net_check_device_id(net);
+ if (ret) {
+ goto err;
+ }
+ return 0;
+err:
+ if (net) {
+ vhost_net_cleanup(net);
+ }
+ vhost_vdpa_del(ncs);
+ return -1;
+}
+
+static void vhost_vdpa_cleanup(NetClientState *nc)
+{
+ VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
+
+ if (s->vhost_net) {
+ vhost_net_cleanup(s->vhost_net);
+ g_free(s->vhost_net);
+ s->vhost_net = NULL;
+ }
+}
+
+static bool vhost_vdpa_has_vnet_hdr(NetClientState *nc)
+{
+ assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+
+ return true;
+}
+
+static bool vhost_vdpa_has_ufo(NetClientState *nc)
+{
+ assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+ VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
+ uint64_t features = 0;
+ features |= (1ULL << VIRTIO_NET_F_HOST_UFO);
+ features = vhost_net_get_features(s->vhost_net, features);
+ return !!(features & (1ULL << VIRTIO_NET_F_HOST_UFO));
+
+}
+
+static NetClientInfo net_vhost_vdpa_info = {
+ .type = NET_CLIENT_DRIVER_VHOST_VDPA,
+ .size = sizeof(VhostVDPAState),
+ .cleanup = vhost_vdpa_cleanup,
+ .has_vnet_hdr = vhost_vdpa_has_vnet_hdr,
+ .has_ufo = vhost_vdpa_has_ufo,
+};
+
+static int net_vhost_vdpa_init(NetClientState *peer, const char *device,
+ const char *name, const char *vhostdev)
+{
+ NetClientState *nc = NULL;
+ VhostVDPAState *s;
+ int vdpa_device_fd = -1;
+ int ret = 0;
+ assert(name);
+ nc = qemu_new_net_client(&net_vhost_vdpa_info, peer, device, name);
+ snprintf(nc->info_str, sizeof(nc->info_str), TYPE_VHOST_VDPA);
+ nc->queue_index = 0;
+ s = DO_UPCAST(VhostVDPAState, nc, nc);
+ vdpa_device_fd = qemu_open(vhostdev, O_RDWR);
+ if (vdpa_device_fd == -1) {
+ return -errno;
+ }
+ s->vhost_vdpa.device_fd = vdpa_device_fd;
+ ret = vhost_vdpa_add(nc, (void *)&s->vhost_vdpa);
+ assert(s->vhost_net);
+ return ret;
+}
+
+static int net_vhost_check_net(void *opaque, QemuOpts *opts, Error **errp)
+{
+ const char *name = opaque;
+ const char *driver, *netdev;
+
+ driver = qemu_opt_get(opts, "driver");
+ netdev = qemu_opt_get(opts, "netdev");
+ if (!driver || !netdev) {
+ return 0;
+ }
+ if (strcmp(netdev, name) == 0 &&
+ !g_str_has_prefix(driver, "virtio-net-")) {
+ error_setg(errp, "vhost-vdpa requires frontend driver virtio-net-*");
+ return -1;
+ }
+ return 0;
+}
+
+int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
+ NetClientState *peer, Error **errp)
+{
+ const NetdevVhostVDPAOptions *opts;
+
+ assert(netdev->type == NET_CLIENT_DRIVER_VHOST_VDPA);
+ opts = &netdev->u.vhost_vdpa;
+ /* verify net frontend */
+ if (qemu_opts_foreach(qemu_find_opts("device"), net_vhost_check_net,
+ (char *)name, errp)) {
+ return -1;
+ }
+ return net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, opts->vhostdev);
+}
diff --git a/qapi/net.json b/qapi/net.json
index 9244c9af56..558d520a2f 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -428,16 +428,39 @@
'*vhostforce': 'bool',
'*queues': 'int' } }
+##
+# @NetdevVhostVDPAOptions:
+#
+# Vhost-vdpa network backend
+#
+# vDPA device is a device that uses a datapath which complies with the virtio
+# specifications with a vendor specific control path.
+#
+# @vhostdev: path of vhost-vdpa device
+# (default:'/dev/vhost-vdpa-0')
+#
+# @queues: number of queues to be created for multiqueue vhost-vdpa
+# (default: 1)
+#
+# Since: 5.1
+##
+{ 'struct': 'NetdevVhostVDPAOptions',
+ 'data': {
+ '*vhostdev': 'str',
+ '*queues': 'int' } }
+
##
# @NetClientDriver:
#
# Available netdev drivers.
#
# Since: 2.7
+#
+# @vhost-vdpa since 5.1
##
{ 'enum': 'NetClientDriver',
'data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',
- 'bridge', 'hubport', 'netmap', 'vhost-user' ] }
+ 'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] }
##
# @Netdev:
@@ -465,7 +488,8 @@
'bridge': 'NetdevBridgeOptions',
'hubport': 'NetdevHubPortOptions',
'netmap': 'NetdevNetmapOptions',
- 'vhost-user': 'NetdevVhostUserOptions' } }
+ 'vhost-user': 'NetdevVhostUserOptions',
+ 'vhost-vdpa': 'NetdevVhostVDPAOptions' } }
##
# @NetFilterDirection:
--
2.21.1
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [PATCH v4 00/14]vDPA support in qemu
2020-07-01 14:55 [PATCH v4 00/14]vDPA support in qemu Cindy Lu
` (13 preceding siblings ...)
2020-07-01 14:55 ` [PATCH v4 14/14] vhost-vdpa: introduce vhost-vdpa net client Cindy Lu
@ 2020-07-03 7:30 ` Jason Wang
14 siblings, 0 replies; 16+ messages in thread
From: Jason Wang @ 2020-07-03 7:30 UTC (permalink / raw)
To: Cindy Lu, mst, armbru, eblake, cohuck
Cc: mhabets, qemu-devel, rob.miller, saugatm, hanand, hch, eperezma,
jgg, shahafs, kevin.tian, parav, vmireyno, cunming.liang, gdawar,
jiri, xiao.w.wang, stefanha, zhihong.wang, aadam, rdunlap,
maxime.coquelin, lingshan.zhu
On 2020/7/1 下午10:55, Cindy Lu wrote:
> vDPA device is a device that uses a datapath which complies with the
> virtio specifications with vendor specific control path. vDPA devices
> can be both physically located on the hardware or emulated by software.
> This PATCH introduce the vDPA support in qemu
> TODO
> 1) vIOMMU support
> 2) live migration support
> 3) docs for vhost-vdpa
> 4) config interrupt support
>
> Change from v1
> separate the patch of vhost_vq_get_addr
> separate the patch of vhost_dev_start
> introduce the docmation for vhost-vdpa.rst
> other comments form last version
> github address
> https://github.com/lulu-github-name/qemutmp.git PATCHV2
>
> Change from v3
> fix the complie problem
> separate the patch of vhost_force_iommu
> other comments form last version
> github address
> https://github.com/lulu-github-name/qemutmp.git PATCHV3
>
> Change from v3
> fix the centos7 test problem
> other comments form last version
> github address
> https://github.com/lulu-github-name/qemutmp.git PATCHV4
Thanks a lot for patches.
I think we may have a more generic handling of forcing IOMMU_PLATFORM.
But it can be done on top.
So
Acked-by: Jason Wang <jasowang@redhat.com>
>
> Cindy Lu (11):
> net: introduce qemu_get_peer
> vhost_net: use the function qemu_get_peer
> vhost: introduce new VhostOps vhost_dev_start
> vhost: implement vhost_dev_start method
> vhost: introduce new VhostOps vhost_vq_get_addr
> vhost: implement vhost_vq_get_addr method
> vhost: introduce new VhostOps vhost_force_iommu
> vhost: implement vhost_force_iommu method
> vhost_net: introduce set_config & get_config
> vhost-vdpa: introduce vhost-vdpa backend
> vhost-vdpa: introduce vhost-vdpa net client
>
> Jason Wang (3):
> virtio-bus: introduce queue_enabled method
> virtio-pci: implement queue_enabled method
> vhost: check the existence of vhost_set_iotlb_callback
>
> configure | 21 ++
> docs/interop/index.rst | 1 +
> docs/interop/vhost-vdpa.rst | 17 ++
> hw/net/vhost_net-stub.c | 11 +
> hw/net/vhost_net.c | 45 ++-
> hw/net/virtio-net.c | 19 ++
> hw/virtio/Makefile.objs | 1 +
> hw/virtio/vhost-backend.c | 6 +
> hw/virtio/vhost-vdpa.c | 475 ++++++++++++++++++++++++++++++
> hw/virtio/vhost.c | 52 +++-
> hw/virtio/virtio-pci.c | 13 +
> hw/virtio/virtio.c | 6 +
> include/hw/virtio/vhost-backend.h | 19 +-
> include/hw/virtio/vhost-vdpa.h | 26 ++
> include/hw/virtio/vhost.h | 7 +
> include/hw/virtio/virtio-bus.h | 4 +
> include/net/net.h | 1 +
> include/net/vhost-vdpa.h | 22 ++
> include/net/vhost_net.h | 5 +
> net/Makefile.objs | 2 +-
> net/clients.h | 2 +
> net/net.c | 10 +
> net/vhost-vdpa.c | 228 ++++++++++++++
> qapi/net.json | 28 +-
> qemu-options.hx | 12 +
> 25 files changed, 1004 insertions(+), 29 deletions(-)
> create mode 100644 docs/interop/vhost-vdpa.rst
> create mode 100644 hw/virtio/vhost-vdpa.c
> create mode 100644 include/hw/virtio/vhost-vdpa.h
> create mode 100644 include/net/vhost-vdpa.h
> create mode 100644 net/vhost-vdpa.c
>
^ permalink raw reply [flat|nested] 16+ messages in thread