* [PATCH 0/5] Support exporting BDSs via VDUSE
@ 2022-01-25 13:17 Xie Yongji
2022-01-25 13:17 ` [PATCH 1/5] linux-headers: Add vduse.h Xie Yongji
` (4 more replies)
0 siblings, 5 replies; 19+ messages in thread
From: Xie Yongji @ 2022-01-25 13:17 UTC (permalink / raw)
To: mst, jasowang, stefanha, sgarzare, kwolf, mreitz, mlureau
Cc: qemu-devel, qemu-block
Hi all,
Last few months ago, VDUSE (vDPA Device in Userspace) [1] has
been merged into Linux kernel as a framework that make it
possible to emulate a vDPA device in userspace. This series
aimed at implementing a VDUSE block backend based on the
qemu-storage-daemon infrastructure.
To support that, we firstly introduce a VDUSE library as a
subproject (like what libvhost-user does) to help implementing
VDUSE backends in QEMU. Then a VDUSE block export is implemented
based on this library. At last, we add resize and reconnect support
to the VDUSE block export and VDUSE library.
Since we don't support vdpa-blk in QEMU currently, the VM case is
tested with my previous patchset [2].
[1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
[2] https://www.mail-archive.com/qemu-devel@nongnu.org/msg797569.html
Please review, thanks!
Xie Yongji (5):
headers: Add vduse.h
libvduse: Add VDUSE (vDPA Device in Userspace) library
vduse-blk: implements vduse-blk export
vduse-blk: Add vduse-blk resize support
libvduse: Add support for reconnecting
block/export/export.c | 6 +
block/export/meson.build | 5 +
block/export/vduse-blk.c | 448 +++++++
block/export/vduse-blk.h | 20 +
include/standard-headers/linux/vduse.h | 306 +++++
meson.build | 28 +
meson_options.txt | 4 +
qapi/block-export.json | 24 +-
scripts/meson-buildoptions.sh | 7 +
scripts/update-linux-headers.sh | 1 +
subprojects/libvduse/include/atomic.h | 1 +
subprojects/libvduse/libvduse.c | 1267 +++++++++++++++++++
subprojects/libvduse/libvduse.h | 195 +++
subprojects/libvduse/meson.build | 10 +
subprojects/libvduse/standard-headers/linux | 1 +
15 files changed, 2321 insertions(+), 2 deletions(-)
create mode 100644 block/export/vduse-blk.c
create mode 100644 block/export/vduse-blk.h
create mode 100644 include/standard-headers/linux/vduse.h
create mode 120000 subprojects/libvduse/include/atomic.h
create mode 100644 subprojects/libvduse/libvduse.c
create mode 100644 subprojects/libvduse/libvduse.h
create mode 100644 subprojects/libvduse/meson.build
create mode 120000 subprojects/libvduse/standard-headers/linux
--
2.20.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 1/5] linux-headers: Add vduse.h
2022-01-25 13:17 [PATCH 0/5] Support exporting BDSs via VDUSE Xie Yongji
@ 2022-01-25 13:17 ` Xie Yongji
2022-02-07 13:12 ` Stefan Hajnoczi
2022-01-25 13:17 ` [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library Xie Yongji
` (3 subsequent siblings)
4 siblings, 1 reply; 19+ messages in thread
From: Xie Yongji @ 2022-01-25 13:17 UTC (permalink / raw)
To: mst, jasowang, stefanha, sgarzare, kwolf, mreitz, mlureau
Cc: qemu-devel, qemu-block
This adds vduse header to standard headers so that the
relevant VDUSE API can be used in subsequent patches.
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
---
include/standard-headers/linux/vduse.h | 306 +++++++++++++++++++++++++
| 1 +
2 files changed, 307 insertions(+)
create mode 100644 include/standard-headers/linux/vduse.h
diff --git a/include/standard-headers/linux/vduse.h b/include/standard-headers/linux/vduse.h
new file mode 100644
index 0000000000..4242bc9fdf
--- /dev/null
+++ b/include/standard-headers/linux/vduse.h
@@ -0,0 +1,306 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _VDUSE_H_
+#define _VDUSE_H_
+
+#include "standard-headers/linux/types.h"
+
+#define VDUSE_BASE 0x81
+
+/* The ioctls for control device (/dev/vduse/control) */
+
+#define VDUSE_API_VERSION 0
+
+/*
+ * Get the version of VDUSE API that kernel supported (VDUSE_API_VERSION).
+ * This is used for future extension.
+ */
+#define VDUSE_GET_API_VERSION _IOR(VDUSE_BASE, 0x00, uint64_t)
+
+/* Set the version of VDUSE API that userspace supported. */
+#define VDUSE_SET_API_VERSION _IOW(VDUSE_BASE, 0x01, uint64_t)
+
+/**
+ * struct vduse_dev_config - basic configuration of a VDUSE device
+ * @name: VDUSE device name, needs to be NUL terminated
+ * @vendor_id: virtio vendor id
+ * @device_id: virtio device id
+ * @features: virtio features
+ * @vq_num: the number of virtqueues
+ * @vq_align: the allocation alignment of virtqueue's metadata
+ * @reserved: for future use, needs to be initialized to zero
+ * @config_size: the size of the configuration space
+ * @config: the buffer of the configuration space
+ *
+ * Structure used by VDUSE_CREATE_DEV ioctl to create VDUSE device.
+ */
+struct vduse_dev_config {
+#define VDUSE_NAME_MAX 256
+ char name[VDUSE_NAME_MAX];
+ uint32_t vendor_id;
+ uint32_t device_id;
+ uint64_t features;
+ uint32_t vq_num;
+ uint32_t vq_align;
+ uint32_t reserved[13];
+ uint32_t config_size;
+ uint8_t config[];
+};
+
+/* Create a VDUSE device which is represented by a char device (/dev/vduse/$NAME) */
+#define VDUSE_CREATE_DEV _IOW(VDUSE_BASE, 0x02, struct vduse_dev_config)
+
+/*
+ * Destroy a VDUSE device. Make sure there are no more references
+ * to the char device (/dev/vduse/$NAME).
+ */
+#define VDUSE_DESTROY_DEV _IOW(VDUSE_BASE, 0x03, char[VDUSE_NAME_MAX])
+
+/* The ioctls for VDUSE device (/dev/vduse/$NAME) */
+
+/**
+ * struct vduse_iotlb_entry - entry of IOTLB to describe one IOVA region [start, last]
+ * @offset: the mmap offset on returned file descriptor
+ * @start: start of the IOVA region
+ * @last: last of the IOVA region
+ * @perm: access permission of the IOVA region
+ *
+ * Structure used by VDUSE_IOTLB_GET_FD ioctl to find an overlapped IOVA region.
+ */
+struct vduse_iotlb_entry {
+ uint64_t offset;
+ uint64_t start;
+ uint64_t last;
+#define VDUSE_ACCESS_RO 0x1
+#define VDUSE_ACCESS_WO 0x2
+#define VDUSE_ACCESS_RW 0x3
+ uint8_t perm;
+};
+
+/*
+ * Find the first IOVA region that overlaps with the range [start, last]
+ * and return the corresponding file descriptor. Return -EINVAL means the
+ * IOVA region doesn't exist. Caller should set start and last fields.
+ */
+#define VDUSE_IOTLB_GET_FD _IOWR(VDUSE_BASE, 0x10, struct vduse_iotlb_entry)
+
+/*
+ * Get the negotiated virtio features. It's a subset of the features in
+ * struct vduse_dev_config which can be accepted by virtio driver. It's
+ * only valid after FEATURES_OK status bit is set.
+ */
+#define VDUSE_DEV_GET_FEATURES _IOR(VDUSE_BASE, 0x11, uint64_t)
+
+/**
+ * struct vduse_config_data - data used to update configuration space
+ * @offset: the offset from the beginning of configuration space
+ * @length: the length to write to configuration space
+ * @buffer: the buffer used to write from
+ *
+ * Structure used by VDUSE_DEV_SET_CONFIG ioctl to update device
+ * configuration space.
+ */
+struct vduse_config_data {
+ uint32_t offset;
+ uint32_t length;
+ uint8_t buffer[];
+};
+
+/* Set device configuration space */
+#define VDUSE_DEV_SET_CONFIG _IOW(VDUSE_BASE, 0x12, struct vduse_config_data)
+
+/*
+ * Inject a config interrupt. It's usually used to notify virtio driver
+ * that device configuration space has changed.
+ */
+#define VDUSE_DEV_INJECT_CONFIG_IRQ _IO(VDUSE_BASE, 0x13)
+
+/**
+ * struct vduse_vq_config - basic configuration of a virtqueue
+ * @index: virtqueue index
+ * @max_size: the max size of virtqueue
+ * @reserved: for future use, needs to be initialized to zero
+ *
+ * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue.
+ */
+struct vduse_vq_config {
+ uint32_t index;
+ uint16_t max_size;
+ uint16_t reserved[13];
+};
+
+/*
+ * Setup the specified virtqueue. Make sure all virtqueues have been
+ * configured before the device is attached to vDPA bus.
+ */
+#define VDUSE_VQ_SETUP _IOW(VDUSE_BASE, 0x14, struct vduse_vq_config)
+
+/**
+ * struct vduse_vq_state_split - split virtqueue state
+ * @avail_index: available index
+ */
+struct vduse_vq_state_split {
+ uint16_t avail_index;
+};
+
+/**
+ * struct vduse_vq_state_packed - packed virtqueue state
+ * @last_avail_counter: last driver ring wrap counter observed by device
+ * @last_avail_idx: device available index
+ * @last_used_counter: device ring wrap counter
+ * @last_used_idx: used index
+ */
+struct vduse_vq_state_packed {
+ uint16_t last_avail_counter;
+ uint16_t last_avail_idx;
+ uint16_t last_used_counter;
+ uint16_t last_used_idx;
+};
+
+/**
+ * struct vduse_vq_info - information of a virtqueue
+ * @index: virtqueue index
+ * @num: the size of virtqueue
+ * @desc_addr: address of desc area
+ * @driver_addr: address of driver area
+ * @device_addr: address of device area
+ * @split: split virtqueue state
+ * @packed: packed virtqueue state
+ * @ready: ready status of virtqueue
+ *
+ * Structure used by VDUSE_VQ_GET_INFO ioctl to get virtqueue's information.
+ */
+struct vduse_vq_info {
+ uint32_t index;
+ uint32_t num;
+ uint64_t desc_addr;
+ uint64_t driver_addr;
+ uint64_t device_addr;
+ union {
+ struct vduse_vq_state_split split;
+ struct vduse_vq_state_packed packed;
+ };
+ uint8_t ready;
+};
+
+/* Get the specified virtqueue's information. Caller should set index field. */
+#define VDUSE_VQ_GET_INFO _IOWR(VDUSE_BASE, 0x15, struct vduse_vq_info)
+
+/**
+ * struct vduse_vq_eventfd - eventfd configuration for a virtqueue
+ * @index: virtqueue index
+ * @fd: eventfd, -1 means de-assigning the eventfd
+ *
+ * Structure used by VDUSE_VQ_SETUP_KICKFD ioctl to setup kick eventfd.
+ */
+struct vduse_vq_eventfd {
+ uint32_t index;
+#define VDUSE_EVENTFD_DEASSIGN -1
+ int fd;
+};
+
+/*
+ * Setup kick eventfd for specified virtqueue. The kick eventfd is used
+ * by VDUSE kernel module to notify userspace to consume the avail vring.
+ */
+#define VDUSE_VQ_SETUP_KICKFD _IOW(VDUSE_BASE, 0x16, struct vduse_vq_eventfd)
+
+/*
+ * Inject an interrupt for specific virtqueue. It's used to notify virtio driver
+ * to consume the used vring.
+ */
+#define VDUSE_VQ_INJECT_IRQ _IOW(VDUSE_BASE, 0x17, uint32_t)
+
+/* The control messages definition for read(2)/write(2) on /dev/vduse/$NAME */
+
+/**
+ * enum vduse_req_type - request type
+ * @VDUSE_GET_VQ_STATE: get the state for specified virtqueue from userspace
+ * @VDUSE_SET_STATUS: set the device status
+ * @VDUSE_UPDATE_IOTLB: Notify userspace to update the memory mapping for
+ * specified IOVA range via VDUSE_IOTLB_GET_FD ioctl
+ */
+enum vduse_req_type {
+ VDUSE_GET_VQ_STATE,
+ VDUSE_SET_STATUS,
+ VDUSE_UPDATE_IOTLB,
+};
+
+/**
+ * struct vduse_vq_state - virtqueue state
+ * @index: virtqueue index
+ * @split: split virtqueue state
+ * @packed: packed virtqueue state
+ */
+struct vduse_vq_state {
+ uint32_t index;
+ union {
+ struct vduse_vq_state_split split;
+ struct vduse_vq_state_packed packed;
+ };
+};
+
+/**
+ * struct vduse_dev_status - device status
+ * @status: device status
+ */
+struct vduse_dev_status {
+ uint8_t status;
+};
+
+/**
+ * struct vduse_iova_range - IOVA range [start, last]
+ * @start: start of the IOVA range
+ * @last: last of the IOVA range
+ */
+struct vduse_iova_range {
+ uint64_t start;
+ uint64_t last;
+};
+
+/**
+ * struct vduse_dev_request - control request
+ * @type: request type
+ * @request_id: request id
+ * @reserved: for future use
+ * @vq_state: virtqueue state, only index field is available
+ * @s: device status
+ * @iova: IOVA range for updating
+ * @padding: padding
+ *
+ * Structure used by read(2) on /dev/vduse/$NAME.
+ */
+struct vduse_dev_request {
+ uint32_t type;
+ uint32_t request_id;
+ uint32_t reserved[4];
+ union {
+ struct vduse_vq_state vq_state;
+ struct vduse_dev_status s;
+ struct vduse_iova_range iova;
+ uint32_t padding[32];
+ };
+};
+
+/**
+ * struct vduse_dev_response - response to control request
+ * @request_id: corresponding request id
+ * @result: the result of request
+ * @reserved: for future use, needs to be initialized to zero
+ * @vq_state: virtqueue state
+ * @padding: padding
+ *
+ * Structure used by write(2) on /dev/vduse/$NAME.
+ */
+struct vduse_dev_response {
+ uint32_t request_id;
+#define VDUSE_REQ_RESULT_OK 0x00
+#define VDUSE_REQ_RESULT_FAILED 0x01
+ uint32_t result;
+ uint32_t reserved[4];
+ union {
+ struct vduse_vq_state vq_state;
+ uint32_t padding[32];
+ };
+};
+
+#endif /* _VDUSE_H_ */
--git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh
index fea4d6eb65..4c7846076f 100755
--- a/scripts/update-linux-headers.sh
+++ b/scripts/update-linux-headers.sh
@@ -198,6 +198,7 @@ for i in "$tmpdir"/include/linux/*virtio*.h \
"$tmpdir/include/linux/const.h" \
"$tmpdir/include/linux/kernel.h" \
"$tmpdir/include/linux/vhost_types.h" \
+ "$tmpdir/include/linux/vduse.h" \
"$tmpdir/include/linux/sysinfo.h"; do
cp_portable "$i" "$output/include/standard-headers/linux"
done
--
2.20.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
2022-01-25 13:17 [PATCH 0/5] Support exporting BDSs via VDUSE Xie Yongji
2022-01-25 13:17 ` [PATCH 1/5] linux-headers: Add vduse.h Xie Yongji
@ 2022-01-25 13:17 ` Xie Yongji
2022-02-07 14:00 ` Stefan Hajnoczi
2022-01-25 13:17 ` [PATCH 3/5] vduse-blk: implements vduse-blk export Xie Yongji
` (2 subsequent siblings)
4 siblings, 1 reply; 19+ messages in thread
From: Xie Yongji @ 2022-01-25 13:17 UTC (permalink / raw)
To: mst, jasowang, stefanha, sgarzare, kwolf, mreitz, mlureau
Cc: qemu-devel, qemu-block
VDUSE [1] is a linux framework that makes it possible to implement
software-emulated vDPA devices in userspace. This adds a library
as a subproject to help implementing VDUSE backends in QEMU.
[1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
---
meson.build | 15 +
meson_options.txt | 2 +
scripts/meson-buildoptions.sh | 3 +
subprojects/libvduse/include/atomic.h | 1 +
subprojects/libvduse/libvduse.c | 1025 +++++++++++++++++++
subprojects/libvduse/libvduse.h | 193 ++++
subprojects/libvduse/meson.build | 10 +
subprojects/libvduse/standard-headers/linux | 1 +
8 files changed, 1250 insertions(+)
create mode 120000 subprojects/libvduse/include/atomic.h
create mode 100644 subprojects/libvduse/libvduse.c
create mode 100644 subprojects/libvduse/libvduse.h
create mode 100644 subprojects/libvduse/meson.build
create mode 120000 subprojects/libvduse/standard-headers/linux
diff --git a/meson.build b/meson.build
index 333c61deba..864fb50ade 100644
--- a/meson.build
+++ b/meson.build
@@ -1305,6 +1305,21 @@ if not get_option('fuse_lseek').disabled()
endif
endif
+have_libvduse = (targetos == 'linux')
+if get_option('libvduse').enabled()
+ if targetos != 'linux'
+ error('libvduse requires linux')
+ endif
+elif get_option('libvduse').disabled()
+ have_libvduse = false
+endif
+
+libvduse = not_found
+if have_libvduse
+ libvduse_proj = subproject('libvduse')
+ libvduse = libvduse_proj.get_variable('libvduse_dep')
+endif
+
# libbpf
libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config')
if libbpf.found() and not cc.links('''
diff --git a/meson_options.txt b/meson_options.txt
index 921967eddb..16790d1814 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -195,6 +195,8 @@ option('virtfs', type: 'feature', value: 'auto',
description: 'virtio-9p support')
option('virtiofsd', type: 'feature', value: 'auto',
description: 'build virtiofs daemon (virtiofsd)')
+option('libvduse', type: 'feature', value: 'auto',
+ description: 'build VDUSE Library')
option('capstone', type: 'combo', value: 'auto',
choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index a4af02c527..af5c75d758 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -58,6 +58,7 @@ meson_options_help() {
printf "%s\n" ' libssh ssh block device support'
printf "%s\n" ' libudev Use libudev to enumerate host devices'
printf "%s\n" ' libusb libusb support for USB passthrough'
+ printf "%s\n" ' libvduse build VDUSE Library'
printf "%s\n" ' libxml2 libxml2 support for Parallels image format'
printf "%s\n" ' linux-aio Linux AIO support'
printf "%s\n" ' linux-io-uring Linux io_uring support'
@@ -188,6 +189,8 @@ _meson_option_parse() {
--disable-libudev) printf "%s" -Dlibudev=disabled ;;
--enable-libusb) printf "%s" -Dlibusb=enabled ;;
--disable-libusb) printf "%s" -Dlibusb=disabled ;;
+ --enable-libvduse) printf "%s" -Dlibvduse=enabled ;;
+ --disable-libvduse) printf "%s" -Dlibvduse=disabled ;;
--enable-libxml2) printf "%s" -Dlibxml2=enabled ;;
--disable-libxml2) printf "%s" -Dlibxml2=disabled ;;
--enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;;
diff --git a/subprojects/libvduse/include/atomic.h b/subprojects/libvduse/include/atomic.h
new file mode 120000
index 0000000000..8c2be64f7b
--- /dev/null
+++ b/subprojects/libvduse/include/atomic.h
@@ -0,0 +1 @@
+../../../include/qemu/atomic.h
\ No newline at end of file
diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
new file mode 100644
index 0000000000..7671864bca
--- /dev/null
+++ b/subprojects/libvduse/libvduse.c
@@ -0,0 +1,1025 @@
+/*
+ * VDUSE (vDPA Device in Userspace) library
+ *
+ * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
+ * Portions of codes and concepts borrowed from libvhost-user.c, so:
+ * Copyright IBM, Corp. 2007
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * Author:
+ * Xie Yongji <xieyongji@bytedance.com>
+ * Anthony Liguori <aliguori@us.ibm.com>
+ * Marc-André Lureau <mlureau@redhat.com>
+ * Victor Kaplansky <victork@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdbool.h>
+#include <stddef.h>
+#include <errno.h>
+#include <string.h>
+#include <assert.h>
+#include <endian.h>
+#include <unistd.h>
+#include <limits.h>
+#include <fcntl.h>
+
+#include <sys/ioctl.h>
+#include <sys/eventfd.h>
+#include <sys/mman.h>
+
+#include "include/atomic.h"
+#include "standard-headers/linux/vhost_types.h"
+#include "standard-headers/linux/vduse.h"
+#include "libvduse.h"
+
+#define VIRTQUEUE_MAX_SIZE 1024
+#define VDUSE_VQ_ALIGN 4096
+#define MAX_IOVA_REGIONS 256
+
+/* Round number down to multiple */
+#define ALIGN_DOWN(n, m) ((n) / (m) * (m))
+
+/* Round number up to multiple */
+#define ALIGN_UP(n, m) ALIGN_DOWN((n) + (m) - 1, (m))
+
+#ifndef unlikely
+#define unlikely(x) __builtin_expect(!!(x), 0)
+#endif
+
+typedef struct VduseRing {
+ unsigned int num;
+ uint64_t desc_addr;
+ uint64_t avail_addr;
+ uint64_t used_addr;
+ struct vring_desc *desc;
+ struct vring_avail *avail;
+ struct vring_used *used;
+} VduseRing;
+
+struct VduseVirtq {
+ VduseRing vring;
+ uint16_t last_avail_idx;
+ uint16_t shadow_avail_idx;
+ uint16_t used_idx;
+ uint16_t signalled_used;
+ bool signalled_used_valid;
+ int index;
+ int inuse;
+ bool ready;
+ int fd;
+ VduseDev *dev;
+};
+
+typedef struct VduseIovaRegion {
+ uint64_t iova;
+ uint64_t size;
+ uint64_t mmap_offset;
+ uint64_t mmap_addr;
+} VduseIovaRegion;
+
+struct VduseDev {
+ VduseVirtq *vqs;
+ VduseIovaRegion regions[MAX_IOVA_REGIONS];
+ int num_regions;
+ char *name;
+ uint32_t device_id;
+ uint32_t vendor_id;
+ uint16_t num_queues;
+ uint16_t queue_size;
+ uint64_t features;
+ const VduseOps *ops;
+ int fd;
+ int ctrl_fd;
+ void *priv;
+};
+
+static inline bool has_feature(uint64_t features, unsigned int fbit)
+{
+ assert(fbit < 64);
+ return !!(features & (1ULL << fbit));
+}
+
+static inline bool vduse_dev_has_feature(VduseDev *dev, unsigned int fbit)
+{
+ return has_feature(dev->features, fbit);
+}
+
+VduseDev *vduse_queue_get_dev(VduseVirtq *vq)
+{
+ return vq->dev;
+}
+
+int vduse_queue_get_fd(VduseVirtq *vq)
+{
+ return vq->fd;
+}
+
+void *vduse_dev_get_priv(VduseDev *dev)
+{
+ return dev->priv;
+}
+
+VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index)
+{
+ return &dev->vqs[index];
+}
+
+int vduse_dev_get_fd(VduseDev *dev)
+{
+ return dev->fd;
+}
+
+static int vduse_inject_irq(VduseDev *dev, int index)
+{
+ return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index);
+}
+
+static void vduse_iova_remove_region(VduseDev *dev, uint64_t start,
+ uint64_t last)
+{
+ int i;
+
+ if (last == start) {
+ return;
+ }
+
+ for (i = 0; i < MAX_IOVA_REGIONS; i++) {
+ if (!dev->regions[i].mmap_addr) {
+ continue;
+ }
+
+ if (start <= dev->regions[i].iova &&
+ last >= (dev->regions[i].iova + dev->regions[i].size - 1)) {
+ munmap((void *)dev->regions[i].mmap_addr,
+ dev->regions[i].mmap_offset + dev->regions[i].size);
+ dev->regions[i].mmap_addr = 0;
+ dev->num_regions--;
+ }
+ }
+}
+
+static int vduse_iova_add_region(VduseDev *dev, int fd,
+ uint64_t offset, uint64_t start,
+ uint64_t last, int prot)
+{
+ int i;
+ uint64_t size = last - start + 1;
+ void *mmap_addr = mmap(0, size + offset, prot, MAP_SHARED, fd, 0);
+
+ if (mmap_addr == MAP_FAILED) {
+ return -EINVAL;
+ }
+
+ for (i = 0; i < MAX_IOVA_REGIONS; i++) {
+ if (!dev->regions[i].mmap_addr) {
+ dev->regions[i].mmap_addr = (uint64_t)(uintptr_t)mmap_addr;
+ dev->regions[i].mmap_offset = offset;
+ dev->regions[i].iova = start;
+ dev->regions[i].size = size;
+ dev->num_regions++;
+ break;
+ }
+ }
+ close(fd);
+
+ return 0;
+}
+
+static int perm_to_prot(uint8_t perm)
+{
+ int prot = 0;
+
+ switch (perm) {
+ case VDUSE_ACCESS_WO:
+ prot |= PROT_WRITE;
+ break;
+ case VDUSE_ACCESS_RO:
+ prot |= PROT_READ;
+ break;
+ case VDUSE_ACCESS_RW:
+ prot |= PROT_READ | PROT_WRITE;
+ break;
+ default:
+ break;
+ }
+
+ return prot;
+}
+
+static inline void *iova_to_va(VduseDev *dev, uint64_t *plen, uint64_t iova)
+{
+ int i, ret;
+ struct vduse_iotlb_entry entry;
+
+ for (i = 0; i < MAX_IOVA_REGIONS; i++) {
+ VduseIovaRegion *r = &dev->regions[i];
+
+ if (!r->mmap_addr) {
+ continue;
+ }
+
+ if ((iova >= r->iova) && (iova < (r->iova + r->size))) {
+ if ((iova + *plen) > (r->iova + r->size)) {
+ *plen = r->iova + r->size - iova;
+ }
+ return (void *)(uintptr_t)(iova - r->iova +
+ r->mmap_addr + r->mmap_offset);
+ }
+ }
+
+ entry.start = iova;
+ entry.last = iova + 1;
+ ret = ioctl(dev->fd, VDUSE_IOTLB_GET_FD, &entry);
+ if (ret < 0) {
+ return NULL;
+ }
+
+ if (!vduse_iova_add_region(dev, ret, entry.offset, entry.start,
+ entry.last, perm_to_prot(entry.perm))) {
+ return iova_to_va(dev, plen, iova);
+ }
+
+ return NULL;
+}
+
+static inline uint16_t vring_avail_flags(VduseVirtq *vq)
+{
+ return le16toh(vq->vring.avail->flags);
+}
+
+static inline uint16_t vring_avail_idx(VduseVirtq *vq)
+{
+ vq->shadow_avail_idx = le16toh(vq->vring.avail->idx);
+
+ return vq->shadow_avail_idx;
+}
+
+static inline uint16_t vring_avail_ring(VduseVirtq *vq, int i)
+{
+ return le16toh(vq->vring.avail->ring[i]);
+}
+
+static inline uint16_t vring_get_used_event(VduseVirtq *vq)
+{
+ return vring_avail_ring(vq, vq->vring.num);
+}
+
+static bool vduse_queue_get_head(VduseVirtq *vq, unsigned int idx,
+ unsigned int *head)
+{
+ /*
+ * Grab the next descriptor number they're advertising, and increment
+ * the index we've seen.
+ */
+ *head = vring_avail_ring(vq, idx % vq->vring.num);
+
+ /* If their number is silly, that's a fatal mistake. */
+ if (*head >= vq->vring.num) {
+ fprintf(stderr, "Guest says index %u is available\n", *head);
+ return false;
+ }
+
+ return true;
+}
+
+static int
+vduse_queue_read_indirect_desc(VduseDev *dev, struct vring_desc *desc,
+ uint64_t addr, size_t len)
+{
+ struct vring_desc *ori_desc;
+ uint64_t read_len;
+
+ if (len > (VIRTQUEUE_MAX_SIZE * sizeof(struct vring_desc))) {
+ return -1;
+ }
+
+ if (len == 0) {
+ return -1;
+ }
+
+ while (len) {
+ read_len = len;
+ ori_desc = iova_to_va(dev, &read_len, addr);
+ if (!ori_desc) {
+ return -1;
+ }
+
+ memcpy(desc, ori_desc, read_len);
+ len -= read_len;
+ addr += read_len;
+ desc += read_len;
+ }
+
+ return 0;
+}
+
+enum {
+ VIRTQUEUE_READ_DESC_ERROR = -1,
+ VIRTQUEUE_READ_DESC_DONE = 0, /* end of chain */
+ VIRTQUEUE_READ_DESC_MORE = 1, /* more buffers in chain */
+};
+
+static int vduse_queue_read_next_desc(struct vring_desc *desc, int i,
+ unsigned int max, unsigned int *next)
+{
+ /* If this descriptor says it doesn't chain, we're done. */
+ if (!(le16toh(desc[i].flags) & VRING_DESC_F_NEXT)) {
+ return VIRTQUEUE_READ_DESC_DONE;
+ }
+
+ /* Check they're not leading us off end of descriptors. */
+ *next = desc[i].next;
+ /* Make sure compiler knows to grab that: we don't want it changing! */
+ smp_wmb();
+
+ if (*next >= max) {
+ fprintf(stderr, "Desc next is %u\n", *next);
+ return VIRTQUEUE_READ_DESC_ERROR;
+ }
+
+ return VIRTQUEUE_READ_DESC_MORE;
+}
+
+/*
+ * Fetch avail_idx from VQ memory only when we really need to know if
+ * guest has added some buffers.
+ */
+static bool vduse_queue_empty(VduseVirtq *vq)
+{
+ if (unlikely(!vq->vring.avail)) {
+ return true;
+ }
+
+ if (vq->shadow_avail_idx != vq->last_avail_idx) {
+ return false;
+ }
+
+ return vring_avail_idx(vq) == vq->last_avail_idx;
+}
+
+static bool vduse_queue_should_notify(VduseVirtq *vq)
+{
+ VduseDev *dev = vq->dev;
+ uint16_t old, new;
+ bool v;
+
+ /* We need to expose used array entries before checking used event. */
+ smp_mb();
+
+ /* Always notify when queue is empty (when feature acknowledge) */
+ if (vduse_dev_has_feature(dev, VIRTIO_F_NOTIFY_ON_EMPTY) &&
+ !vq->inuse && vduse_queue_empty(vq)) {
+ return true;
+ }
+
+ if (!vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) {
+ return !(vring_avail_flags(vq) & VRING_AVAIL_F_NO_INTERRUPT);
+ }
+
+ v = vq->signalled_used_valid;
+ vq->signalled_used_valid = true;
+ old = vq->signalled_used;
+ new = vq->signalled_used = vq->used_idx;
+ return !v || vring_need_event(vring_get_used_event(vq), new, old);
+}
+
+void vduse_queue_notify(VduseVirtq *vq)
+{
+ VduseDev *dev = vq->dev;
+
+ if (unlikely(!vq->vring.avail)) {
+ return;
+ }
+
+ if (!vduse_queue_should_notify(vq)) {
+ return;
+ }
+
+ if (vduse_inject_irq(dev, vq->index) < 0) {
+ fprintf(stderr, "Error inject irq for vq %d: %s\n",
+ vq->index, strerror(errno));
+ }
+}
+
+static inline void vring_used_flags_set_bit(VduseVirtq *vq, int mask)
+{
+ uint16_t *flags;
+
+ flags = (uint16_t *)((char*)vq->vring.used +
+ offsetof(struct vring_used, flags));
+ *flags = htole16(le16toh(*flags) | mask);
+}
+
+static inline void vring_used_flags_unset_bit(VduseVirtq *vq, int mask)
+{
+ uint16_t *flags;
+
+ flags = (uint16_t *)((char*)vq->vring.used +
+ offsetof(struct vring_used, flags));
+ *flags = htole16(le16toh(*flags) & ~mask);
+}
+
+static inline void vring_set_avail_event(VduseVirtq *vq, uint16_t val)
+{
+ *((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
+}
+
+static bool vduse_queue_map_single_desc(VduseVirtq *vq, unsigned int *p_num_sg,
+ struct iovec *iov, unsigned int max_num_sg,
+ bool is_write, uint64_t pa, size_t sz)
+{
+ unsigned num_sg = *p_num_sg;
+ VduseDev *dev = vq->dev;
+
+ assert(num_sg <= max_num_sg);
+
+ if (!sz) {
+ fprintf(stderr, "virtio: zero sized buffers are not allowed\n");
+ return false;
+ }
+
+ while (sz) {
+ uint64_t len = sz;
+
+ if (num_sg == max_num_sg) {
+ fprintf(stderr,
+ "virtio: too many descriptors in indirect table\n");
+ return false;
+ }
+
+ iov[num_sg].iov_base = iova_to_va(dev, &len, pa);
+ if (iov[num_sg].iov_base == NULL) {
+ fprintf(stderr, "virtio: invalid address for buffers\n");
+ return false;
+ }
+ iov[num_sg++].iov_len = len;
+ sz -= len;
+ pa += len;
+ }
+
+ *p_num_sg = num_sg;
+ return true;
+}
+
+static void *vduse_queue_alloc_element(size_t sz, unsigned out_num,
+ unsigned in_num)
+{
+ VduseVirtqElement *elem;
+ size_t in_sg_ofs = ALIGN_UP(sz, __alignof__(elem->in_sg[0]));
+ size_t out_sg_ofs = in_sg_ofs + in_num * sizeof(elem->in_sg[0]);
+ size_t out_sg_end = out_sg_ofs + out_num * sizeof(elem->out_sg[0]);
+
+ assert(sz >= sizeof(VduseVirtqElement));
+ elem = malloc(out_sg_end);
+ elem->out_num = out_num;
+ elem->in_num = in_num;
+ elem->in_sg = (void *)elem + in_sg_ofs;
+ elem->out_sg = (void *)elem + out_sg_ofs;
+ return elem;
+}
+
+static void *vduse_queue_map_desc(VduseVirtq *vq, unsigned int idx, size_t sz)
+{
+ struct vring_desc *desc = vq->vring.desc;
+ VduseDev *dev = vq->dev;
+ uint64_t desc_addr, read_len;
+ unsigned int desc_len;
+ unsigned int max = vq->vring.num;
+ unsigned int i = idx;
+ VduseVirtqElement *elem;
+ struct iovec iov[VIRTQUEUE_MAX_SIZE];
+ struct vring_desc desc_buf[VIRTQUEUE_MAX_SIZE];
+ unsigned int out_num = 0, in_num = 0;
+ int rc;
+
+ if (le16toh(desc[i].flags) & VRING_DESC_F_INDIRECT) {
+ if (le32toh(desc[i].len) % sizeof(struct vring_desc)) {
+ fprintf(stderr, "Invalid size for indirect buffer table\n");
+ return NULL;
+ }
+
+ /* loop over the indirect descriptor table */
+ desc_addr = le64toh(desc[i].addr);
+ desc_len = le32toh(desc[i].len);
+ max = desc_len / sizeof(struct vring_desc);
+ read_len = desc_len;
+ desc = iova_to_va(dev, &read_len, desc_addr);
+ if (unlikely(desc && read_len != desc_len)) {
+ /* Failed to use zero copy */
+ desc = NULL;
+ if (!vduse_queue_read_indirect_desc(dev, desc_buf,
+ desc_addr,
+ desc_len)) {
+ desc = desc_buf;
+ }
+ }
+ if (!desc) {
+ fprintf(stderr, "Invalid indirect buffer table\n");
+ return NULL;
+ }
+ i = 0;
+ }
+
+ /* Collect all the descriptors */
+ do {
+ if (le16toh(desc[i].flags) & VRING_DESC_F_WRITE) {
+ if (!vduse_queue_map_single_desc(vq, &in_num, iov + out_num,
+ VIRTQUEUE_MAX_SIZE - out_num,
+ true, le64toh(desc[i].addr),
+ le32toh(desc[i].len))) {
+ return NULL;
+ }
+ } else {
+ if (in_num) {
+ fprintf(stderr, "Incorrect order for descriptors\n");
+ return NULL;
+ }
+ if (!vduse_queue_map_single_desc(vq, &out_num, iov,
+ VIRTQUEUE_MAX_SIZE, false,
+ le64toh(desc[i].addr),
+ le32toh(desc[i].len))) {
+ return NULL;
+ }
+ }
+
+ /* If we've got too many, that implies a descriptor loop. */
+ if ((in_num + out_num) > max) {
+ fprintf(stderr, "Looped descriptor\n");
+ return NULL;
+ }
+ rc = vduse_queue_read_next_desc(desc, i, max, &i);
+ } while (rc == VIRTQUEUE_READ_DESC_MORE);
+
+ if (rc == VIRTQUEUE_READ_DESC_ERROR) {
+ fprintf(stderr, "read descriptor error\n");
+ return NULL;
+ }
+
+ /* Now copy what we have collected and mapped */
+ elem = vduse_queue_alloc_element(sz, out_num, in_num);
+ elem->index = idx;
+ for (i = 0; i < out_num; i++) {
+ elem->out_sg[i] = iov[i];
+ }
+ for (i = 0; i < in_num; i++) {
+ elem->in_sg[i] = iov[out_num + i];
+ }
+
+ return elem;
+}
+
+void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
+{
+ unsigned int head;
+ VduseVirtqElement *elem;
+ VduseDev *dev = vq->dev;
+
+ if (unlikely(!vq->vring.avail)) {
+ return NULL;
+ }
+
+ if (vduse_queue_empty(vq)) {
+ return NULL;
+ }
+ /* Needed after virtio_queue_empty() */
+ smp_rmb();
+
+ if (vq->inuse >= vq->vring.num) {
+ fprintf(stderr, "Virtqueue size exceeded: %d\n", vq->inuse);
+ return NULL;
+ }
+
+ if (!vduse_queue_get_head(vq, vq->last_avail_idx++, &head)) {
+ return NULL;
+ }
+
+ if (vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) {
+ vring_set_avail_event(vq, vq->last_avail_idx);
+ }
+
+ elem = vduse_queue_map_desc(vq, head, sz);
+
+ if (!elem) {
+ return NULL;
+ }
+
+ vq->inuse++;
+
+ return elem;
+}
+
+static inline void vring_used_write(VduseVirtq *vq,
+ struct vring_used_elem *uelem, int i)
+{
+ struct vring_used *used = vq->vring.used;
+
+ used->ring[i] = *uelem;
+}
+
+static void vduse_queue_fill(VduseVirtq *vq, const VduseVirtqElement *elem,
+ unsigned int len, unsigned int idx)
+{
+ struct vring_used_elem uelem;
+
+ if (unlikely(!vq->vring.used)) {
+ return;
+ }
+
+ idx = (idx + vq->used_idx) % vq->vring.num;
+
+ uelem.id = htole32(elem->index);
+ uelem.len = htole32(len);
+ vring_used_write(vq, &uelem, idx);
+}
+
+static inline void vring_used_idx_set(VduseVirtq *vq, uint16_t val)
+{
+ vq->vring.used->idx = htole16(val);
+ vq->used_idx = val;
+}
+
+static void vduse_queue_flush(VduseVirtq *vq, unsigned int count)
+{
+ uint16_t old, new;
+
+ if (unlikely(!vq->vring.used)) {
+ return;
+ }
+
+ /* Make sure buffer is written before we update index. */
+ smp_wmb();
+
+ old = vq->used_idx;
+ new = old + count;
+ vring_used_idx_set(vq, new);
+ vq->inuse -= count;
+ if (unlikely((int16_t)(new - vq->signalled_used) < (uint16_t)(new - old))) {
+ vq->signalled_used_valid = false;
+ }
+}
+
+void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
+ unsigned int len)
+{
+ vduse_queue_fill(vq, elem, len, 0);
+ vduse_queue_flush(vq, 1);
+}
+
+static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr,
+ uint64_t avail_addr, uint64_t used_addr)
+{
+ struct VduseDev *dev = vq->dev;
+ uint64_t len;
+
+ len = sizeof(struct vring_desc);
+ vq->vring.desc = iova_to_va(dev, &len, desc_addr);
+ assert(len == sizeof(struct vring_desc));
+
+ len = sizeof(struct vring_avail);
+ vq->vring.avail = iova_to_va(dev, &len, avail_addr);
+ assert(len == sizeof(struct vring_avail));
+
+ len = sizeof(struct vring_used);
+ vq->vring.used = iova_to_va(dev, &len, used_addr);
+ assert(len == sizeof(struct vring_used));
+
+ if (!vq->vring.desc || !vq->vring.avail || !vq->vring.used) {
+ fprintf(stderr, "Failed to get vq[%d] iova mapping\n", vq->index);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void vduse_queue_enable(VduseVirtq *vq)
+{
+ struct VduseDev *dev = vq->dev;
+ struct vduse_vq_info vq_info;
+ struct vduse_vq_eventfd vq_eventfd;
+ int fd;
+
+ vq_info.index = vq->index;
+ if (ioctl(dev->fd, VDUSE_VQ_GET_INFO, &vq_info)) {
+ fprintf(stderr, "Failed to get vq[%d] info: %s\n",
+ vq->index, strerror(errno));
+ return;
+ }
+
+ if (!vq_info.ready) {
+ return;
+ }
+
+ vq->vring.num = vq_info.num;
+ vq->vring.desc_addr = vq_info.desc_addr;
+ vq->vring.avail_addr = vq_info.driver_addr;
+ vq->vring.used_addr = vq_info.device_addr;
+
+ if (vduse_queue_update_vring(vq, vq_info.desc_addr,
+ vq_info.driver_addr, vq_info.device_addr)) {
+ fprintf(stderr, "Failed to update vring for vq[%d]\n", vq->index);
+ return;
+ }
+
+ fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC);
+ if (fd < 0) {
+ fprintf(stderr, "Failed to init eventfd for vq[%d]\n", vq->index);
+ return;
+ }
+
+ vq_eventfd.index = vq->index;
+ vq_eventfd.fd = fd;
+ if (ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &vq_eventfd)) {
+ fprintf(stderr, "Failed to setup kick fd for vq[%d]\n", vq->index);
+ close(fd);
+ return;
+ }
+
+ vq->fd = fd;
+ vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index;
+ vq->inuse = 0;
+ vq->used_idx = 0;
+ vq->signalled_used_valid = false;
+ vq->ready = true;
+
+ dev->ops->enable_queue(dev, vq);
+}
+
+static void vduse_queue_disable(VduseVirtq *vq)
+{
+ struct VduseDev *dev = vq->dev;
+ struct vduse_vq_eventfd eventfd;
+
+ if (!vq->ready) {
+ return;
+ }
+
+ dev->ops->disable_queue(dev, vq);
+
+ eventfd.index = vq->index;
+ eventfd.fd = VDUSE_EVENTFD_DEASSIGN;
+ ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &eventfd);
+ close(vq->fd);
+
+ assert(vq->inuse == 0);
+
+ vq->vring.num = 0;
+ vq->vring.desc_addr = 0;
+ vq->vring.avail_addr = 0;
+ vq->vring.used_addr = 0;
+ vq->vring.desc = 0;
+ vq->vring.avail = 0;
+ vq->vring.used = 0;
+ vq->ready = false;
+ vq->fd = -1;
+}
+
+static void vduse_dev_start_dataplane(VduseDev *dev)
+{
+ int i;
+
+ if (ioctl(dev->fd, VDUSE_DEV_GET_FEATURES, &dev->features)) {
+ fprintf(stderr, "Failed to get features: %s\n", strerror(errno));
+ return;
+ }
+
+ for (i = 0; i < dev->num_queues; i++) {
+ vduse_queue_enable(&dev->vqs[i]);
+ }
+}
+
+static void vduse_dev_stop_dataplane(VduseDev *dev)
+{
+ int i;
+
+ for (i = 0; i < dev->num_queues; i++) {
+ vduse_queue_disable(&dev->vqs[i]);
+ }
+ dev->features = 0;
+ vduse_iova_remove_region(dev, 0, ULONG_MAX);
+}
+
+int vduse_dev_handler(VduseDev *dev)
+{
+ struct vduse_dev_request req;
+ struct vduse_dev_response resp = { 0 };
+ VduseVirtq *vq;
+ int i, ret;
+
+ ret = read(dev->fd, &req, sizeof(req));
+ if (ret != sizeof(req)) {
+ fprintf(stderr, "Read request error [%d]: %s\n",
+ ret, strerror(errno));
+ return -errno;
+ }
+ resp.request_id = req.request_id;
+
+ switch (req.type) {
+ case VDUSE_GET_VQ_STATE:
+ vq = &dev->vqs[req.vq_state.index];
+ resp.vq_state.split.avail_index = vq->last_avail_idx;
+ resp.result = VDUSE_REQ_RESULT_OK;
+ break;
+ case VDUSE_SET_STATUS:
+ if (req.s.status & VIRTIO_CONFIG_S_DRIVER_OK) {
+ vduse_dev_start_dataplane(dev);
+ } else if (req.s.status == 0) {
+ vduse_dev_stop_dataplane(dev);
+ }
+ resp.result = VDUSE_REQ_RESULT_OK;
+ break;
+ case VDUSE_UPDATE_IOTLB:
+ /* The iova will be updated by iova_to_va() later, so just remove it */
+ vduse_iova_remove_region(dev, req.iova.start, req.iova.last);
+ for (i = 0; i < dev->num_queues; i++) {
+ VduseVirtq *vq = &dev->vqs[i];
+ if (vq->ready) {
+ if (vduse_queue_update_vring(vq, vq->vring.desc_addr,
+ vq->vring.avail_addr,
+ vq->vring.used_addr)) {
+ fprintf(stderr, "Failed to update vring for vq[%d]\n",
+ vq->index);
+ }
+ }
+ }
+ resp.result = VDUSE_REQ_RESULT_OK;
+ break;
+ default:
+ resp.result = VDUSE_REQ_RESULT_FAILED;
+ break;
+ }
+
+ ret = write(dev->fd, &resp, sizeof(resp));
+ if (ret != sizeof(resp)) {
+ fprintf(stderr, "Write request %d error [%d]: %s\n",
+ req.type, ret, strerror(errno));
+ return -errno;
+ }
+ return 0;
+}
+
+int vduse_dev_update_config(VduseDev *dev, uint32_t size,
+ uint32_t offset, char *buffer)
+{
+ int ret;
+ struct vduse_config_data *data;
+
+ data = malloc(offsetof(struct vduse_config_data, buffer) + size);
+ if (!data) {
+ return -ENOMEM;
+ }
+
+ data->offset = offset;
+ data->length = size;
+ memcpy(data->buffer, buffer, size);
+
+ ret = ioctl(dev->fd, VDUSE_DEV_SET_CONFIG, data);
+ free(data);
+
+ if (ret) {
+ return -errno;
+ }
+
+ if (ioctl(dev->fd, VDUSE_DEV_INJECT_CONFIG_IRQ)) {
+ return -errno;
+ }
+
+ return 0;
+}
+
+int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size)
+{
+ VduseVirtq *vq = &dev->vqs[index];
+ struct vduse_vq_config vq_config = { 0 };
+
+ vq_config.index = vq->index;
+ vq_config.max_size = max_size;
+
+ if (ioctl(dev->fd, VDUSE_VQ_SETUP, &vq_config)) {
+ return -errno;
+ }
+
+ return 0;
+}
+
+VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
+ uint32_t vendor_id, uint64_t features,
+ uint16_t num_queues, uint32_t config_size,
+ char *config, const VduseOps *ops, void *priv)
+{
+ VduseDev *dev;
+ int i, ret, ctrl_fd, fd = -1;
+ uint64_t version;
+ char dev_path[VDUSE_NAME_MAX + 16];
+ VduseVirtq *vqs = NULL;
+ struct vduse_dev_config *dev_config = NULL;
+ size_t size = offsetof(struct vduse_dev_config, config);
+
+ if (!name || strlen(name) > VDUSE_NAME_MAX || !config ||
+ !config_size || !ops || !ops->enable_queue || !ops->disable_queue) {
+ fprintf(stderr, "Invalid parameter for vduse\n");
+ return NULL;
+ }
+
+ dev = malloc(sizeof(VduseDev));
+ if (!dev) {
+ fprintf(stderr, "Failed to allocate vduse device\n");
+ return NULL;
+ }
+ memset(dev, 0, sizeof(VduseDev));
+
+ ctrl_fd = open("/dev/vduse/control", O_RDWR);
+ if (ctrl_fd < 0) {
+ fprintf(stderr, "Failed to open /dev/vduse/control: %s\n",
+ strerror(errno));
+ goto err_ctrl;
+ }
+
+ version = VDUSE_API_VERSION;
+ if (ioctl(ctrl_fd, VDUSE_SET_API_VERSION, &version)) {
+ fprintf(stderr, "Failed to set api version %lu: %s\n",
+ version, strerror(errno));
+ goto err_dev;
+ }
+
+ dev_config = malloc(size + config_size);
+ if (!dev_config) {
+ fprintf(stderr, "Failed to allocate config space\n");
+ goto err_dev;
+ }
+ memset(dev_config, 0, size + config_size);
+
+ strcpy(dev_config->name, name);
+ dev_config->device_id = device_id;
+ dev_config->vendor_id = vendor_id;
+ dev_config->features = features;
+ dev_config->vq_num = num_queues;
+ dev_config->vq_align = VDUSE_VQ_ALIGN;
+ dev_config->config_size = config_size;
+ memcpy(dev_config->config, config, config_size);
+
+ ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config);
+ free(dev_config);
+ if (ret < 0) {
+ fprintf(stderr, "Failed to create vduse dev %s: %s\n",
+ name, strerror(errno));
+ goto err_dev;
+ }
+
+ sprintf(dev_path, "/dev/vduse/%s", name);
+ fd = open(dev_path, O_RDWR);
+ if (fd < 0) {
+ fprintf(stderr, "Failed to open vduse dev %s: %s\n",
+ name, strerror(errno));
+ goto err;
+ }
+
+ vqs = calloc(sizeof(VduseVirtq), num_queues);
+ if (!vqs) {
+ fprintf(stderr, "Failed to allocate virtqueues\n");
+ goto err;
+ }
+
+ for (i = 0; i < num_queues; i++) {
+ vqs[i].index = i;
+ vqs[i].dev = dev;
+ vqs[i].fd = -1;
+ }
+
+ dev->vqs = vqs;
+ dev->name = strdup(name);
+ dev->num_queues = num_queues;
+ dev->ops = ops;
+ dev->ctrl_fd = ctrl_fd;
+ dev->fd = fd;
+ dev->priv = priv;
+
+ return dev;
+err:
+ if (fd > 0) {
+ close(fd);
+ }
+ ioctl(ctrl_fd, VDUSE_DESTROY_DEV, name);
+err_dev:
+ close(ctrl_fd);
+err_ctrl:
+ free(dev);
+
+ return NULL;
+}
+
+void vduse_dev_destroy(VduseDev *dev)
+{
+ free(dev->vqs);
+ close(dev->fd);
+ dev->fd = -1;
+ ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name);
+ free(dev->name);
+ close(dev->ctrl_fd);
+ dev->ctrl_fd = -1;
+ free(dev);
+}
diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h
new file mode 100644
index 0000000000..f6bcb51b5a
--- /dev/null
+++ b/subprojects/libvduse/libvduse.h
@@ -0,0 +1,193 @@
+/*
+ * VDUSE (vDPA Device in Userspace) library
+ *
+ * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
+ *
+ * Author:
+ * Xie Yongji <xieyongji@bytedance.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef LIBVDUSE_H
+#define LIBVDUSE_H
+
+#include <stdint.h>
+#include <sys/uio.h>
+
+/* VDUSE device structure */
+typedef struct VduseDev VduseDev;
+
+/* Virtqueue structure */
+typedef struct VduseVirtq VduseVirtq;
+
+/* Some operation of VDUSE backend */
+typedef struct VduseOps {
+ /* Called when virtqueue can be processed */
+ void (*enable_queue)(VduseDev *dev, VduseVirtq *vq);
+ /* Called when virtqueue processing should be stopped */
+ void (*disable_queue)(VduseDev *dev, VduseVirtq *vq);
+} VduseOps;
+
+/* Describing elements of the I/O buffer */
+typedef struct VduseVirtqElement {
+ /* Virtqueue index */
+ unsigned int index;
+ /* Number of physically-contiguous device-readable descriptors */
+ unsigned int out_num;
+ /* Number of physically-contiguous device-writable descriptors */
+ unsigned int in_num;
+ /* Array to store physically-contiguous device-writable descriptors */
+ struct iovec *in_sg;
+ /* Array to store physically-contiguous device-readable descriptors */
+ struct iovec *out_sg;
+} VduseVirtqElement;
+
+/**
+ * vduse_queue_get_dev:
+ * @vq: specified virtqueue
+ *
+ * Get corresponding VDUSE device from the virtqueue.
+ *
+ * Returns: a pointer to VDUSE device on success, NULL on failure.
+ */
+VduseDev *vduse_queue_get_dev(VduseVirtq *vq);
+
+/**
+ * vduse_queue_get_fd:
+ * @vq: specified virtqueue
+ *
+ * Get the kick fd for the virtqueue.
+ *
+ * Returns: file descriptor on success, -1 on failure.
+ */
+int vduse_queue_get_fd(VduseVirtq *vq);
+
+/**
+ * vduse_queue_pop:
+ * @vq: specified virtqueue
+ * @sz: the size of struct to return (must be >= VduseVirtqElement)
+ *
+ * Pop an element from virtqueue available ring.
+ *
+ * Returns: a pointer to a structure containing VduseVirtqElement on success,
+ * NULL on failure.
+ */
+void *vduse_queue_pop(VduseVirtq *vq, size_t sz);
+
+/**
+ * vduse_queue_push:
+ * @vq: specified virtqueue
+ * @elem: pointer to VduseVirtqElement returned by vduse_queue_pop()
+ * @len: length in bytes to write
+ *
+ * Push an element to virtqueue used ring.
+ */
+void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
+ unsigned int len);
+/**
+ * vduse_queue_notify:
+ * @vq: specified virtqueue
+ *
+ * Request to notify the queue.
+ */
+void vduse_queue_notify(VduseVirtq *vq);
+
+/**
+ * vduse_dev_get_priv:
+ * @dev: VDUSE device
+ *
+ * Get the private pointer passed to vduse_dev_create().
+ *
+ * Returns: private pointer on success, NULL on failure.
+ */
+void *vduse_dev_get_priv(VduseDev *dev);
+
+/**
+ * vduse_dev_get_queue:
+ * @dev: VDUSE device
+ * @index: virtqueue index
+ *
+ * Get the specified virtqueue.
+ *
+ * Returns: a pointer to the virtqueue on success, NULL on failure.
+ */
+VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index);
+
+/**
+ * vduse_dev_get_fd:
+ * @dev: VDUSE device
+ *
+ * Get the control message fd for the VDUSE device.
+ *
+ * Returns: file descriptor on success, -1 on failure.
+ */
+int vduse_dev_get_fd(VduseDev *dev);
+
+/**
+ * vduse_dev_handler:
+ * @dev: VDUSE device
+ *
+ * Used to process the control message.
+ *
+ * Returns: file descriptor on success, -errno on failure.
+ */
+int vduse_dev_handler(VduseDev *dev);
+
+/**
+ * vduse_dev_update_config:
+ * @dev: VDUSE device
+ * @size: the size to write to configuration space
+ * @offset: the offset from the beginning of configuration space
+ * @buffer: the buffer used to write from
+ *
+ * Update device configuration space and inject a config interrupt.
+ *
+ * Returns: 0 on success, -errno on failure.
+ */
+int vduse_dev_update_config(VduseDev *dev, uint32_t size,
+ uint32_t offset, char *buffer);
+
+/**
+ * vduse_dev_setup_queue:
+ * @dev: VDUSE device
+ * @index: virtqueue index
+ * @max_size: the max size of virtqueue
+ *
+ * Setup the specified virtqueue.
+ *
+ * Returns: 0 on success, -errno on failure.
+ */
+int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size);
+
+/**
+ * vduse_dev_create:
+ * @name: VDUSE device name
+ * @device_id: virtio device id
+ * @vendor_id: virtio vendor id
+ * @features: virtio features
+ * @num_queues: the number of virtqueues
+ * @config_size: the size of the configuration space
+ * @config: the buffer of the configuration space
+ * @ops: the operation of VDUSE backend
+ * @priv: private pointer
+ *
+ * Create VDUSE device.
+ *
+ * Returns: pointer to VDUSE device on success, NULL on failure.
+ */
+VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
+ uint32_t vendor_id, uint64_t features,
+ uint16_t num_queues, uint32_t config_size,
+ char *config, const VduseOps *ops, void *priv);
+
+/**
+ * vduse_dev_destroy:
+ * @dev: VDUSE device
+ *
+ * Destroy the VDUSE device.
+ */
+void vduse_dev_destroy(VduseDev *dev);
+
+#endif
diff --git a/subprojects/libvduse/meson.build b/subprojects/libvduse/meson.build
new file mode 100644
index 0000000000..ba08f5ee1a
--- /dev/null
+++ b/subprojects/libvduse/meson.build
@@ -0,0 +1,10 @@
+project('libvduse', 'c',
+ license: 'GPL-2.0-or-later',
+ default_options: ['c_std=gnu99'])
+
+libvduse = static_library('vduse',
+ files('libvduse.c'),
+ c_args: '-D_GNU_SOURCE')
+
+libvduse_dep = declare_dependency(link_with: libvduse,
+ include_directories: include_directories('.'))
diff --git a/subprojects/libvduse/standard-headers/linux b/subprojects/libvduse/standard-headers/linux
new file mode 120000
index 0000000000..c416f068ac
--- /dev/null
+++ b/subprojects/libvduse/standard-headers/linux
@@ -0,0 +1 @@
+../../../include/standard-headers/linux/
\ No newline at end of file
--
2.20.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 3/5] vduse-blk: implements vduse-blk export
2022-01-25 13:17 [PATCH 0/5] Support exporting BDSs via VDUSE Xie Yongji
2022-01-25 13:17 ` [PATCH 1/5] linux-headers: Add vduse.h Xie Yongji
2022-01-25 13:17 ` [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library Xie Yongji
@ 2022-01-25 13:17 ` Xie Yongji
2022-02-07 14:15 ` Stefan Hajnoczi
2022-01-25 13:17 ` [PATCH 4/5] vduse-blk: Add vduse-blk resize support Xie Yongji
2022-01-25 13:18 ` [PATCH 5/5] libvduse: Add support for reconnecting Xie Yongji
4 siblings, 1 reply; 19+ messages in thread
From: Xie Yongji @ 2022-01-25 13:17 UTC (permalink / raw)
To: mst, jasowang, stefanha, sgarzare, kwolf, mreitz, mlureau
Cc: qemu-devel, qemu-block
This implements a VDUSE block backends based on
the libvduse library. We can use it to export the BDSs
for both VM and container (host) usage.
The new command-line syntax is:
$ qemu-storage-daemon \
--blockdev file,node-name=drive0,filename=test.img \
--export vduse-blk,node-name=drive0,id=vduse-export0,writable=on
After the qemu-storage-daemon started, we need to use
the "vdpa" command to attach the device to vDPA bus:
$ vdpa dev add name vduse-export0 mgmtdev vduse
Also the device must be removed via the "vdpa" command
before we stop the qemu-storage-daemon.
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
---
block/export/export.c | 6 +
block/export/meson.build | 5 +
block/export/vduse-blk.c | 427 ++++++++++++++++++++++++++++++++++
block/export/vduse-blk.h | 20 ++
meson.build | 13 ++
meson_options.txt | 2 +
qapi/block-export.json | 24 +-
scripts/meson-buildoptions.sh | 4 +
8 files changed, 499 insertions(+), 2 deletions(-)
create mode 100644 block/export/vduse-blk.c
create mode 100644 block/export/vduse-blk.h
diff --git a/block/export/export.c b/block/export/export.c
index 6d3b9964c8..00dd505540 100644
--- a/block/export/export.c
+++ b/block/export/export.c
@@ -26,6 +26,9 @@
#ifdef CONFIG_VHOST_USER_BLK_SERVER
#include "vhost-user-blk-server.h"
#endif
+#ifdef CONFIG_VDUSE_BLK_EXPORT
+#include "vduse-blk.h"
+#endif
static const BlockExportDriver *blk_exp_drivers[] = {
&blk_exp_nbd,
@@ -35,6 +38,9 @@ static const BlockExportDriver *blk_exp_drivers[] = {
#ifdef CONFIG_FUSE
&blk_exp_fuse,
#endif
+#ifdef CONFIG_VDUSE_BLK_EXPORT
+ &blk_exp_vduse_blk,
+#endif
};
/* Only accessed from the main thread */
diff --git a/block/export/meson.build b/block/export/meson.build
index 0a08e384c7..cf311d2b1b 100644
--- a/block/export/meson.build
+++ b/block/export/meson.build
@@ -5,3 +5,8 @@ if have_vhost_user_blk_server
endif
blockdev_ss.add(when: fuse, if_true: files('fuse.c'))
+
+if have_vduse_blk_export
+ blockdev_ss.add(files('vduse-blk.c'))
+ blockdev_ss.add(libvduse)
+endif
diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
new file mode 100644
index 0000000000..5a8d289685
--- /dev/null
+++ b/block/export/vduse-blk.c
@@ -0,0 +1,427 @@
+/*
+ * Export QEMU block device via VDUSE
+ *
+ * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
+ * Portions of codes and concepts borrowed from vhost-user-blk-server.c, so:
+ * Copyright (c) 2020 Red Hat, Inc.
+ *
+ * Author:
+ * Xie Yongji <xieyongji@bytedance.com>
+ * Coiby Xu <coiby.xu@gmail.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#include <sys/eventfd.h>
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "sysemu/block-backend.h"
+#include "block/export.h"
+#include "qemu/error-report.h"
+#include "util/block-helpers.h"
+#include "subprojects/libvduse/libvduse.h"
+
+#include "standard-headers/linux/virtio_ring.h"
+#include "standard-headers/linux/virtio_blk.h"
+
+#define VIRTIO_BLK_SECTOR_BITS 9
+#define VIRTIO_BLK_SECTOR_SIZE (1ULL << VIRTIO_BLK_SECTOR_BITS)
+
+#define VDUSE_DEFAULT_NUM_QUEUE 1
+#define VDUSE_DEFAULT_QUEUE_SIZE 128
+
+typedef struct VduseBlkExport {
+ BlockExport export;
+ VduseDev *dev;
+ uint16_t num_queues;
+ uint32_t blk_size;
+ bool writable;
+} VduseBlkExport;
+
+struct virtio_blk_inhdr {
+ unsigned char status;
+};
+
+typedef struct VduseBlkReq {
+ VduseVirtqElement elem;
+ int64_t sector_num;
+ size_t in_len;
+ struct virtio_blk_inhdr *in;
+ struct virtio_blk_outhdr out;
+ VduseVirtq *vq;
+} VduseBlkReq;
+
+static void vduse_blk_req_complete(VduseBlkReq *req)
+{
+ vduse_queue_push(req->vq, &req->elem, req->in_len);
+ vduse_queue_notify(req->vq);
+
+ free(req);
+}
+
+static bool vduse_blk_sect_range_ok(VduseBlkExport *vblk_exp,
+ uint64_t sector, size_t size)
+{
+ uint64_t nb_sectors;
+ uint64_t total_sectors;
+
+ if (size % VIRTIO_BLK_SECTOR_SIZE) {
+ return false;
+ }
+
+ nb_sectors = size >> VIRTIO_BLK_SECTOR_BITS;
+
+ QEMU_BUILD_BUG_ON(BDRV_SECTOR_SIZE != VIRTIO_BLK_SECTOR_SIZE);
+ if (nb_sectors > BDRV_REQUEST_MAX_SECTORS) {
+ return false;
+ }
+ if ((sector << VIRTIO_BLK_SECTOR_BITS) % vblk_exp->blk_size) {
+ return false;
+ }
+ blk_get_geometry(vblk_exp->export.blk, &total_sectors);
+ if (sector > total_sectors || nb_sectors > total_sectors - sector) {
+ return false;
+ }
+ return true;
+}
+
+static void coroutine_fn vduse_blk_virtio_process_req(void *opaque)
+{
+ VduseBlkReq *req = opaque;
+ VduseVirtq *vq = req->vq;
+ VduseDev *dev = vduse_queue_get_dev(vq);
+ VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
+ BlockBackend *blk = vblk_exp->export.blk;
+ VduseVirtqElement *elem = &req->elem;
+ struct iovec *in_iov = elem->in_sg;
+ struct iovec *out_iov = elem->out_sg;
+ unsigned in_num = elem->in_num;
+ unsigned out_num = elem->out_num;
+ uint32_t type;
+
+ if (elem->out_num < 1 || elem->in_num < 1) {
+ error_report("virtio-blk request missing headers");
+ goto err;
+ }
+
+ if (unlikely(iov_to_buf(out_iov, out_num, 0, &req->out,
+ sizeof(req->out)) != sizeof(req->out))) {
+ error_report("virtio-blk request outhdr too short");
+ goto err;
+ }
+
+ iov_discard_front(&out_iov, &out_num, sizeof(req->out));
+
+ if (in_iov[in_num - 1].iov_len < sizeof(struct virtio_blk_inhdr)) {
+ error_report("virtio-blk request inhdr too short");
+ goto err;
+ }
+
+ /* We always touch the last byte, so just see how big in_iov is. */
+ req->in_len = iov_size(in_iov, in_num);
+ req->in = (void *)in_iov[in_num - 1].iov_base
+ + in_iov[in_num - 1].iov_len
+ - sizeof(struct virtio_blk_inhdr);
+ iov_discard_back(in_iov, &in_num, sizeof(struct virtio_blk_inhdr));
+
+ type = le32_to_cpu(req->out.type);
+ switch (type & ~VIRTIO_BLK_T_BARRIER) {
+ case VIRTIO_BLK_T_IN:
+ case VIRTIO_BLK_T_OUT: {
+ QEMUIOVector qiov;
+ int64_t offset;
+ ssize_t ret = 0;
+ bool is_write = type & VIRTIO_BLK_T_OUT;
+ req->sector_num = le64_to_cpu(req->out.sector);
+
+ if (is_write && !vblk_exp->writable) {
+ req->in->status = VIRTIO_BLK_S_IOERR;
+ break;
+ }
+
+ if (is_write) {
+ qemu_iovec_init_external(&qiov, out_iov, out_num);
+ } else {
+ qemu_iovec_init_external(&qiov, in_iov, in_num);
+ }
+
+ if (unlikely(!vduse_blk_sect_range_ok(vblk_exp,
+ req->sector_num,
+ qiov.size))) {
+ req->in->status = VIRTIO_BLK_S_IOERR;
+ break;
+ }
+
+ offset = req->sector_num << VIRTIO_BLK_SECTOR_BITS;
+
+ if (is_write) {
+ ret = blk_co_pwritev(blk, offset, qiov.size, &qiov, 0);
+ } else {
+ ret = blk_co_preadv(blk, offset, qiov.size, &qiov, 0);
+ }
+ if (ret >= 0) {
+ req->in->status = VIRTIO_BLK_S_OK;
+ } else {
+ req->in->status = VIRTIO_BLK_S_IOERR;
+ }
+ break;
+ }
+ case VIRTIO_BLK_T_FLUSH:
+ if (blk_co_flush(blk) == 0) {
+ req->in->status = VIRTIO_BLK_S_OK;
+ } else {
+ req->in->status = VIRTIO_BLK_S_IOERR;
+ }
+ break;
+ case VIRTIO_BLK_T_GET_ID: {
+ size_t size = MIN(iov_size(&elem->in_sg[0], in_num),
+ VIRTIO_BLK_ID_BYTES);
+ snprintf(elem->in_sg[0].iov_base, size, "%s", vblk_exp->export.id);
+ req->in->status = VIRTIO_BLK_S_OK;
+ break;
+ }
+ default:
+ req->in->status = VIRTIO_BLK_S_UNSUPP;
+ break;
+ }
+
+ vduse_blk_req_complete(req);
+ return;
+
+err:
+ free(req);
+}
+
+static void vduse_blk_vq_handler(VduseDev *dev, VduseVirtq *vq)
+{
+ while (1) {
+ VduseBlkReq *req;
+
+ req = vduse_queue_pop(vq, sizeof(VduseBlkReq));
+ if (!req) {
+ break;
+ }
+ req->vq = vq;
+
+ Coroutine *co =
+ qemu_coroutine_create(vduse_blk_virtio_process_req, req);
+ qemu_coroutine_enter(co);
+ }
+}
+
+static void on_vduse_vq_kick(void *opaque)
+{
+ VduseVirtq *vq = opaque;
+ VduseDev *dev = vduse_queue_get_dev(vq);
+ int fd = vduse_queue_get_fd(vq);
+ eventfd_t kick_data;
+
+ if (eventfd_read(fd, &kick_data) == -1) {
+ error_report("failed to read data from eventfd");
+ return;
+ }
+
+ vduse_blk_vq_handler(dev, vq);
+}
+
+static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq)
+{
+ VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
+
+ aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
+ true, on_vduse_vq_kick, NULL, NULL, NULL, vq);
+}
+
+static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq)
+{
+ VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
+
+ aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
+ true, NULL, NULL, NULL, NULL, NULL);
+}
+
+static const VduseOps vduse_blk_ops = {
+ .enable_queue = vduse_blk_enable_queue,
+ .disable_queue = vduse_blk_disable_queue,
+};
+
+static void on_vduse_dev_kick(void *opaque)
+{
+ VduseDev *dev = opaque;
+
+ vduse_dev_handler(dev);
+}
+
+static void blk_aio_attached(AioContext *ctx, void *opaque)
+{
+ VduseBlkExport *vblk_exp = opaque;
+ int i;
+
+ vblk_exp->export.ctx = ctx;
+
+ aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev),
+ true, on_vduse_dev_kick, NULL, NULL, NULL,
+ vblk_exp->dev);
+
+ for (i = 0; i < vblk_exp->num_queues; i++) {
+ VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
+ int fd = vduse_queue_get_fd(vq);
+
+ if (fd < 0) {
+ continue;
+ }
+ aio_set_fd_handler(vblk_exp->export.ctx, fd, true,
+ on_vduse_vq_kick, NULL, NULL, NULL, vq);
+ }
+}
+
+static void blk_aio_detach(void *opaque)
+{
+ VduseBlkExport *vblk_exp = opaque;
+ int i;
+
+ for (i = 0; i < vblk_exp->num_queues; i++) {
+ VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
+ int fd = vduse_queue_get_fd(vq);
+
+ if (fd < 0) {
+ continue;
+ }
+ aio_set_fd_handler(vblk_exp->export.ctx, fd,
+ true, NULL, NULL, NULL, NULL, NULL);
+ }
+ aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev),
+ true, NULL, NULL, NULL, NULL, NULL);
+ vblk_exp->export.ctx = NULL;
+}
+
+static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
+ Error **errp)
+{
+ VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
+ BlockExportOptionsVduseBlk *vblk_opts = &opts->u.vduse_blk;
+ uint64_t logical_block_size = VIRTIO_BLK_SECTOR_SIZE;
+ uint16_t num_queues = VDUSE_DEFAULT_NUM_QUEUE;
+ uint16_t queue_size = VDUSE_DEFAULT_QUEUE_SIZE;
+ Error *local_err = NULL;
+ struct virtio_blk_config config;
+ uint64_t features;
+ int i;
+
+ if (vblk_opts->has_num_queues) {
+ num_queues = vblk_opts->num_queues;
+ if (num_queues == 0) {
+ error_setg(errp, "num-queues must be greater than 0");
+ return -EINVAL;
+ }
+ }
+
+ if (vblk_opts->has_queue_size) {
+ queue_size = vblk_opts->queue_size;
+ if (queue_size == 0) {
+ error_setg(errp, "queue-size must be greater than 0");
+ return -EINVAL;
+ }
+ }
+
+ if (vblk_opts->has_logical_block_size) {
+ logical_block_size = vblk_opts->logical_block_size;
+ check_block_size(exp->id, "logical-block-size", logical_block_size,
+ &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return -EINVAL;
+ }
+ }
+ blk_set_guest_block_size(exp->blk, logical_block_size);
+
+ vblk_exp->blk_size = logical_block_size;
+ vblk_exp->writable = opts->writable;
+ vblk_exp->num_queues = num_queues;
+
+ config.capacity =
+ cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS);
+ config.seg_max = cpu_to_le32(queue_size - 2);
+ config.size_max = cpu_to_le32(0);
+ config.min_io_size = cpu_to_le16(1);
+ config.opt_io_size = cpu_to_le32(1);
+ config.num_queues = cpu_to_le16(num_queues);
+ config.blk_size = cpu_to_le32(logical_block_size);
+
+ features = (1ULL << VIRTIO_F_IOMMU_PLATFORM) |
+ (1ULL << VIRTIO_F_VERSION_1) |
+ (1ULL << VIRTIO_RING_F_EVENT_IDX) |
+ (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) |
+ (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
+ (1ULL << VIRTIO_BLK_F_SIZE_MAX) |
+ (1ULL << VIRTIO_BLK_F_SEG_MAX) |
+ (1ULL << VIRTIO_BLK_F_TOPOLOGY) |
+ (1ULL << VIRTIO_BLK_F_BLK_SIZE);
+
+ if (num_queues > 1) {
+ features |= 1ULL << VIRTIO_BLK_F_MQ;
+ }
+ if (!vblk_exp->writable) {
+ features |= 1ULL << VIRTIO_BLK_F_RO;
+ }
+
+ vblk_exp->dev = vduse_dev_create(exp->id, VIRTIO_ID_BLOCK, 0,
+ features, num_queues,
+ sizeof(struct virtio_blk_config),
+ (char *)&config, &vduse_blk_ops,
+ vblk_exp);
+ if (!vblk_exp->dev) {
+ error_setg(errp, "failed to create vduse device");
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < num_queues; i++) {
+ vduse_dev_setup_queue(vblk_exp->dev, i, queue_size);
+ }
+
+ aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev), true,
+ on_vduse_dev_kick, NULL, NULL, NULL, vblk_exp->dev);
+
+ blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
+ vblk_exp);
+
+ return 0;
+}
+
+static void vduse_blk_exp_delete(BlockExport *exp)
+{
+ VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
+
+ vduse_dev_destroy(vblk_exp->dev);
+}
+
+static void vduse_blk_exp_request_shutdown(BlockExport *exp)
+{
+ VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
+ int i;
+
+ blk_remove_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
+ vblk_exp);
+
+ for (i = 0; i < vblk_exp->num_queues; i++) {
+ VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
+ int fd = vduse_queue_get_fd(vq);
+
+ if (fd < 0) {
+ continue;
+ }
+ aio_set_fd_handler(exp->ctx, fd, true, NULL, NULL, NULL, NULL, NULL);
+ }
+ aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev),
+ true, NULL, NULL, NULL, NULL, NULL);
+}
+
+const BlockExportDriver blk_exp_vduse_blk = {
+ .type = BLOCK_EXPORT_TYPE_VDUSE_BLK,
+ .instance_size = sizeof(VduseBlkExport),
+ .create = vduse_blk_exp_create,
+ .delete = vduse_blk_exp_delete,
+ .request_shutdown = vduse_blk_exp_request_shutdown,
+};
diff --git a/block/export/vduse-blk.h b/block/export/vduse-blk.h
new file mode 100644
index 0000000000..c4eeb1b70e
--- /dev/null
+++ b/block/export/vduse-blk.h
@@ -0,0 +1,20 @@
+/*
+ * Export QEMU block device via VDUSE
+ *
+ * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
+ *
+ * Author:
+ * Xie Yongji <xieyongji@bytedance.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef VDUSE_BLK_H
+#define VDUSE_BLK_H
+
+#include "block/export.h"
+
+extern const BlockExportDriver blk_exp_vduse_blk;
+
+#endif /* VDUSE_BLK_H */
diff --git a/meson.build b/meson.build
index 864fb50ade..472e3947c6 100644
--- a/meson.build
+++ b/meson.build
@@ -1320,6 +1320,17 @@ if have_libvduse
libvduse = libvduse_proj.get_variable('libvduse_dep')
endif
+have_vduse_blk_export = (have_libvduse and targetos == 'linux')
+if get_option('vduse_blk_export').enabled()
+ if targetos != 'linux'
+ error('vduse_blk_export requires linux')
+ elif have_libvduse
+ error('vduse_blk_export requires libvduse support')
+ endif
+elif get_option('vduse_blk_export').disabled()
+ have_vduse_blk_export = false
+endif
+
# libbpf
libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config')
if libbpf.found() and not cc.links('''
@@ -1514,6 +1525,7 @@ config_host_data.set('CONFIG_SNAPPY', snappy.found())
config_host_data.set('CONFIG_USB_LIBUSB', libusb.found())
config_host_data.set('CONFIG_VDE', vde.found())
config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', have_vhost_user_blk_server)
+config_host_data.set('CONFIG_VDUSE_BLK_EXPORT', have_vduse_blk_export)
config_host_data.set('CONFIG_VNC', vnc.found())
config_host_data.set('CONFIG_VNC_JPEG', jpeg.found())
config_host_data.set('CONFIG_VNC_PNG', png.found())
@@ -3407,6 +3419,7 @@ if have_block
summary_info += {'qed support': config_host.has_key('CONFIG_QED')}
summary_info += {'parallels support': config_host.has_key('CONFIG_PARALLELS')}
summary_info += {'FUSE exports': fuse}
+ summary_info += {'VDUSE block exports': have_vduse_blk_export}
endif
summary(summary_info, bool_yn: true, section: 'Block layer support')
diff --git a/meson_options.txt b/meson_options.txt
index 16790d1814..be1682c4d2 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -197,6 +197,8 @@ option('virtiofsd', type: 'feature', value: 'auto',
description: 'build virtiofs daemon (virtiofsd)')
option('libvduse', type: 'feature', value: 'auto',
description: 'build VDUSE Library')
+option('vduse_blk_export', type: 'feature', value: 'auto',
+ description: 'VDUSE block export support')
option('capstone', type: 'combo', value: 'auto',
choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
diff --git a/qapi/block-export.json b/qapi/block-export.json
index f9ce79a974..f88e90baab 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -170,6 +170,22 @@
'*allow-other': 'FuseExportAllowOther' },
'if': 'CONFIG_FUSE' }
+##
+# @BlockExportOptionsVduseBlk:
+#
+# A vduse-blk block export.
+#
+# @num-queues: the number of virtqueues. Defaults to 1.
+# @queue-size: the size of virtqueue. Defaults to 128.
+# @logical-block-size: Logical block size in bytes. Defaults to 512 bytes.
+#
+# Since: 7.0
+##
+{ 'struct': 'BlockExportOptionsVduseBlk',
+ 'data': { '*num-queues': 'uint16',
+ '*queue-size': 'uint16',
+ '*logical-block-size': 'size'} }
+
##
# @NbdServerAddOptions:
#
@@ -273,13 +289,15 @@
# @nbd: NBD export
# @vhost-user-blk: vhost-user-blk export (since 5.2)
# @fuse: FUSE export (since: 6.0)
+# @vduse-blk: vduse-blk export (since 7.0)
#
# Since: 4.2
##
{ 'enum': 'BlockExportType',
'data': [ 'nbd',
{ 'name': 'vhost-user-blk', 'if': 'CONFIG_VHOST_USER_BLK_SERVER' },
- { 'name': 'fuse', 'if': 'CONFIG_FUSE' } ] }
+ { 'name': 'fuse', 'if': 'CONFIG_FUSE' },
+ { 'name': 'vduse-blk', 'if': 'CONFIG_VDUSE_BLK_EXPORT' } ] }
##
# @BlockExportOptions:
@@ -323,7 +341,9 @@
'vhost-user-blk': { 'type': 'BlockExportOptionsVhostUserBlk',
'if': 'CONFIG_VHOST_USER_BLK_SERVER' },
'fuse': { 'type': 'BlockExportOptionsFuse',
- 'if': 'CONFIG_FUSE' }
+ 'if': 'CONFIG_FUSE' },
+ 'vduse-blk': { 'type': 'BlockExportOptionsVduseBlk',
+ 'if': 'CONFIG_VDUSE_BLK_EXPORT' }
} }
##
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index af5c75d758..615fc17ec3 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -86,6 +86,8 @@ meson_options_help() {
printf "%s\n" ' u2f U2F emulation support'
printf "%s\n" ' usb-redir libusbredir support'
printf "%s\n" ' vde vde network backend support'
+ printf "%s\n" ' vduse-blk-export'
+ printf "%s\n" ' VDUSE block export support'
printf "%s\n" ' vhost-user-blk-server'
printf "%s\n" ' build vhost-user-blk server'
printf "%s\n" ' virglrenderer virgl rendering support'
@@ -254,6 +256,8 @@ _meson_option_parse() {
--disable-usb-redir) printf "%s" -Dusb_redir=disabled ;;
--enable-vde) printf "%s" -Dvde=enabled ;;
--disable-vde) printf "%s" -Dvde=disabled ;;
+ --enable-vduse-blk-export) printf "%s" -Dvduse_blk_export=enabled ;;
+ --disable-vduse-blk-export) printf "%s" -Dvduse_blk_export=disabled ;;
--enable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=enabled ;;
--disable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=disabled ;;
--enable-virglrenderer) printf "%s" -Dvirglrenderer=enabled ;;
--
2.20.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 4/5] vduse-blk: Add vduse-blk resize support
2022-01-25 13:17 [PATCH 0/5] Support exporting BDSs via VDUSE Xie Yongji
` (2 preceding siblings ...)
2022-01-25 13:17 ` [PATCH 3/5] vduse-blk: implements vduse-blk export Xie Yongji
@ 2022-01-25 13:17 ` Xie Yongji
2022-02-07 14:18 ` Stefan Hajnoczi
2022-01-25 13:18 ` [PATCH 5/5] libvduse: Add support for reconnecting Xie Yongji
4 siblings, 1 reply; 19+ messages in thread
From: Xie Yongji @ 2022-01-25 13:17 UTC (permalink / raw)
To: mst, jasowang, stefanha, sgarzare, kwolf, mreitz, mlureau
Cc: qemu-devel, qemu-block
To support block resize, this uses vduse_dev_update_config()
to update the capacity field in configuration space and inject
config interrupt on the block resize callback.
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
---
block/export/vduse-blk.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
index 5a8d289685..83845e9a9a 100644
--- a/block/export/vduse-blk.c
+++ b/block/export/vduse-blk.c
@@ -297,6 +297,23 @@ static void blk_aio_detach(void *opaque)
vblk_exp->export.ctx = NULL;
}
+static void vduse_blk_resize(void *opaque)
+{
+ BlockExport *exp = opaque;
+ VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
+ struct virtio_blk_config config;
+
+ config.capacity =
+ cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS);
+ vduse_dev_update_config(vblk_exp->dev, sizeof(config.capacity),
+ offsetof(struct virtio_blk_config, capacity),
+ (char *)&config.capacity);
+}
+
+static const BlockDevOps vduse_block_ops = {
+ .resize_cb = vduse_blk_resize,
+};
+
static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
Error **errp)
{
@@ -387,6 +404,8 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
vblk_exp);
+ blk_set_dev_ops(exp->blk, &vduse_block_ops, exp);
+
return 0;
}
--
2.20.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 5/5] libvduse: Add support for reconnecting
2022-01-25 13:17 [PATCH 0/5] Support exporting BDSs via VDUSE Xie Yongji
` (3 preceding siblings ...)
2022-01-25 13:17 ` [PATCH 4/5] vduse-blk: Add vduse-blk resize support Xie Yongji
@ 2022-01-25 13:18 ` Xie Yongji
2022-02-07 14:39 ` Stefan Hajnoczi
4 siblings, 1 reply; 19+ messages in thread
From: Xie Yongji @ 2022-01-25 13:18 UTC (permalink / raw)
To: mst, jasowang, stefanha, sgarzare, kwolf, mreitz, mlureau
Cc: qemu-devel, qemu-block
To support reconnecting after restart or crash, VDUSE backend
might need to resubmit inflight I/Os. This stores the metadata
such as the index of inflight I/O's descriptors to a shm file so
that VDUSE backend can restore them during reconnecting.
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
---
block/export/vduse-blk.c | 4 +-
subprojects/libvduse/libvduse.c | 254 +++++++++++++++++++++++++++++++-
subprojects/libvduse/libvduse.h | 4 +-
3 files changed, 254 insertions(+), 8 deletions(-)
diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
index 83845e9a9a..bc14fd798b 100644
--- a/block/export/vduse-blk.c
+++ b/block/export/vduse-blk.c
@@ -232,6 +232,8 @@ static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq)
aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
true, on_vduse_vq_kick, NULL, NULL, NULL, vq);
+ /* Make sure we don't miss any kick afer reconnecting */
+ eventfd_write(vduse_queue_get_fd(vq), 1);
}
static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq)
@@ -388,7 +390,7 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
features, num_queues,
sizeof(struct virtio_blk_config),
(char *)&config, &vduse_blk_ops,
- vblk_exp);
+ g_get_tmp_dir(), vblk_exp);
if (!vblk_exp->dev) {
error_setg(errp, "failed to create vduse device");
return -ENOMEM;
diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
index 7671864bca..ce2f6c7949 100644
--- a/subprojects/libvduse/libvduse.c
+++ b/subprojects/libvduse/libvduse.c
@@ -41,6 +41,8 @@
#define VDUSE_VQ_ALIGN 4096
#define MAX_IOVA_REGIONS 256
+#define LOG_ALIGNMENT 64
+
/* Round number down to multiple */
#define ALIGN_DOWN(n, m) ((n) / (m) * (m))
@@ -51,6 +53,31 @@
#define unlikely(x) __builtin_expect(!!(x), 0)
#endif
+typedef struct VduseDescStateSplit {
+ uint8_t inflight;
+ uint8_t padding[5];
+ uint16_t next;
+ uint64_t counter;
+} VduseDescStateSplit;
+
+typedef struct VduseVirtqLogInflight {
+ uint64_t features;
+ uint16_t version;
+ uint16_t desc_num;
+ uint16_t last_batch_head;
+ uint16_t used_idx;
+ VduseDescStateSplit desc[];
+} VduseVirtqLogInflight;
+
+typedef struct VduseVirtqLog {
+ VduseVirtqLogInflight inflight;
+} VduseVirtqLog;
+
+typedef struct VduseVirtqInflightDesc {
+ uint16_t index;
+ uint64_t counter;
+} VduseVirtqInflightDesc;
+
typedef struct VduseRing {
unsigned int num;
uint64_t desc_addr;
@@ -73,6 +100,10 @@ struct VduseVirtq {
bool ready;
int fd;
VduseDev *dev;
+ VduseVirtqInflightDesc *resubmit_list;
+ uint16_t resubmit_num;
+ uint64_t counter;
+ VduseVirtqLog *log;
};
typedef struct VduseIovaRegion {
@@ -96,8 +127,67 @@ struct VduseDev {
int fd;
int ctrl_fd;
void *priv;
+ char *shm_log_dir;
+ void *log;
+ bool reconnect;
};
+static inline size_t vduse_vq_log_size(uint16_t queue_size)
+{
+ return ALIGN_UP(sizeof(VduseDescStateSplit) * queue_size +
+ sizeof(VduseVirtqLogInflight), LOG_ALIGNMENT);
+}
+
+static void *vduse_log_get(const char *dir, const char *name, size_t size)
+{
+ void *ptr = MAP_FAILED;
+ char *path;
+ int fd;
+
+ path = (char *)malloc(strlen(dir) + strlen(name) +
+ strlen("/vduse-log-") + 1);
+ if (!path) {
+ return ptr;
+ }
+ sprintf(path, "%s/vduse-log-%s", dir, name);
+
+ fd = open(path, O_RDWR | O_CREAT, 0600);
+ if (fd == -1) {
+ goto out;
+ }
+
+ if (ftruncate(fd, size) == -1) {
+ goto out;
+ }
+
+ ptr = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
+ if (ptr == MAP_FAILED) {
+ goto out;
+ }
+out:
+ if (fd > 0) {
+ close(fd);
+ }
+ free(path);
+
+ return ptr;
+}
+
+static void vduse_log_destroy(const char *dir, const char *name)
+{
+ char *path;
+
+ path = (char *)malloc(strlen(dir) + strlen(name) +
+ strlen("/vduse-log-") + 1);
+ if (!path) {
+ return;
+ }
+ sprintf(path, "%s/vduse-log-%s", dir, name);
+
+ unlink(path);
+ free(path);
+}
+
static inline bool has_feature(uint64_t features, unsigned int fbit)
{
assert(fbit < 64);
@@ -139,6 +229,98 @@ static int vduse_inject_irq(VduseDev *dev, int index)
return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index);
}
+static int inflight_desc_compare(const void *a, const void *b)
+{
+ VduseVirtqInflightDesc *desc0 = (VduseVirtqInflightDesc *)a,
+ *desc1 = (VduseVirtqInflightDesc *)b;
+
+ if (desc1->counter > desc0->counter &&
+ (desc1->counter - desc0->counter) < VIRTQUEUE_MAX_SIZE * 2) {
+ return 1;
+ }
+
+ return -1;
+}
+
+static int vduse_queue_check_inflights(VduseVirtq *vq)
+{
+ int i = 0;
+ VduseDev *dev = vq->dev;
+
+ vq->used_idx = vq->vring.used->idx;
+ vq->resubmit_num = 0;
+ vq->resubmit_list = NULL;
+ vq->counter = 0;
+
+ if (unlikely(vq->log->inflight.used_idx != vq->used_idx)) {
+ vq->log->inflight.desc[vq->log->inflight.last_batch_head].inflight = 0;
+
+ barrier();
+
+ vq->log->inflight.used_idx = vq->used_idx;
+ }
+
+ for (i = 0; i < vq->log->inflight.desc_num; i++) {
+ if (vq->log->inflight.desc[i].inflight == 1) {
+ vq->inuse++;
+ }
+ }
+
+ vq->shadow_avail_idx = vq->last_avail_idx = vq->inuse + vq->used_idx;
+
+ if (vq->inuse) {
+ vq->resubmit_list = calloc(vq->inuse, sizeof(VduseVirtqInflightDesc));
+ if (!vq->resubmit_list) {
+ return -1;
+ }
+
+ for (i = 0; i < vq->log->inflight.desc_num; i++) {
+ if (vq->log->inflight.desc[i].inflight) {
+ vq->resubmit_list[vq->resubmit_num].index = i;
+ vq->resubmit_list[vq->resubmit_num].counter =
+ vq->log->inflight.desc[i].counter;
+ vq->resubmit_num++;
+ }
+ }
+
+ if (vq->resubmit_num > 1) {
+ qsort(vq->resubmit_list, vq->resubmit_num,
+ sizeof(VduseVirtqInflightDesc), inflight_desc_compare);
+ }
+ vq->counter = vq->resubmit_list[0].counter + 1;
+ }
+
+ vduse_inject_irq(dev, vq->index);
+
+ return 0;
+}
+
+static int vduse_queue_inflight_get(VduseVirtq *vq, int desc_idx)
+{
+ vq->log->inflight.desc[desc_idx].counter = vq->counter++;
+ vq->log->inflight.desc[desc_idx].inflight = 1;
+
+ return 0;
+}
+
+static int vduse_queue_inflight_pre_put(VduseVirtq *vq, int desc_idx)
+{
+ vq->log->inflight.last_batch_head = desc_idx;
+
+ return 0;
+}
+
+static int vduse_queue_inflight_post_put(VduseVirtq *vq, int desc_idx)
+{
+ vq->log->inflight.desc[desc_idx].inflight = 0;
+
+ barrier();
+
+ vq->log->inflight.used_idx = vq->used_idx;
+
+ return 0;
+}
+
static void vduse_iova_remove_region(VduseDev *dev, uint64_t start,
uint64_t last)
{
@@ -578,11 +760,24 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
unsigned int head;
VduseVirtqElement *elem;
VduseDev *dev = vq->dev;
+ int i;
if (unlikely(!vq->vring.avail)) {
return NULL;
}
+ if (unlikely(vq->resubmit_list && vq->resubmit_num > 0)) {
+ i = (--vq->resubmit_num);
+ elem = vduse_queue_map_desc(vq, vq->resubmit_list[i].index, sz);
+
+ if (!vq->resubmit_num) {
+ free(vq->resubmit_list);
+ vq->resubmit_list = NULL;
+ }
+
+ return elem;
+ }
+
if (vduse_queue_empty(vq)) {
return NULL;
}
@@ -610,6 +805,8 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
vq->inuse++;
+ vduse_queue_inflight_get(vq, head);
+
return elem;
}
@@ -667,7 +864,9 @@ void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
unsigned int len)
{
vduse_queue_fill(vq, elem, len, 0);
+ vduse_queue_inflight_pre_put(vq, elem->index);
vduse_queue_flush(vq, 1);
+ vduse_queue_inflight_post_put(vq, elem->index);
}
static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr,
@@ -740,12 +939,11 @@ static void vduse_queue_enable(VduseVirtq *vq)
}
vq->fd = fd;
- vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index;
- vq->inuse = 0;
- vq->used_idx = 0;
vq->signalled_used_valid = false;
vq->ready = true;
+ vduse_queue_check_inflights(vq);
+
dev->ops->enable_queue(dev, vq);
}
@@ -903,13 +1101,18 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size)
return -errno;
}
+ if (dev->reconnect) {
+ vduse_queue_enable(vq);
+ }
+
return 0;
}
VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
uint32_t vendor_id, uint64_t features,
uint16_t num_queues, uint32_t config_size,
- char *config, const VduseOps *ops, void *priv)
+ char *config, const VduseOps *ops,
+ const char *shm_log_dir, void *priv)
{
VduseDev *dev;
int i, ret, ctrl_fd, fd = -1;
@@ -918,6 +1121,8 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
VduseVirtq *vqs = NULL;
struct vduse_dev_config *dev_config = NULL;
size_t size = offsetof(struct vduse_dev_config, config);
+ size_t log_size = num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE);
+ void *log = NULL;
if (!name || strlen(name) > VDUSE_NAME_MAX || !config ||
!config_size || !ops || !ops->enable_queue || !ops->disable_queue) {
@@ -932,6 +1137,15 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
}
memset(dev, 0, sizeof(VduseDev));
+ if (shm_log_dir) {
+ dev->log = log = vduse_log_get(shm_log_dir, name, log_size);
+ if (!log) {
+ fprintf(stderr, "Failed to get vduse log\n");
+ goto err_ctrl;
+ }
+ dev->shm_log_dir = strdup(shm_log_dir);
+ }
+
ctrl_fd = open("/dev/vduse/control", O_RDWR);
if (ctrl_fd < 0) {
fprintf(stderr, "Failed to open /dev/vduse/control: %s\n",
@@ -964,7 +1178,11 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config);
free(dev_config);
- if (ret < 0) {
+ if (!ret && log) {
+ memset(log, 0, log_size);
+ } else if (errno == EEXIST && log) {
+ dev->reconnect = true;
+ } else {
fprintf(stderr, "Failed to create vduse dev %s: %s\n",
name, strerror(errno));
goto err_dev;
@@ -978,6 +1196,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
goto err;
}
+ if (dev->reconnect &&
+ ioctl(fd, VDUSE_DEV_GET_FEATURES, &dev->features)) {
+ fprintf(stderr, "Failed to get features: %s\n", strerror(errno));
+ goto err;
+ }
+
vqs = calloc(sizeof(VduseVirtq), num_queues);
if (!vqs) {
fprintf(stderr, "Failed to allocate virtqueues\n");
@@ -988,6 +1212,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
vqs[i].index = i;
vqs[i].dev = dev;
vqs[i].fd = -1;
+ if (log) {
+ vqs[i].log = log;
+ vqs[i].log->inflight.desc_num = VIRTQUEUE_MAX_SIZE;
+ log = (void *)((char *)log +
+ vduse_vq_log_size(VIRTQUEUE_MAX_SIZE));
+ }
}
dev->vqs = vqs;
@@ -1008,16 +1238,28 @@ err_dev:
close(ctrl_fd);
err_ctrl:
free(dev);
+ if (log) {
+ munmap(log, log_size);
+ }
return NULL;
}
void vduse_dev_destroy(VduseDev *dev)
{
+ size_t log_size = dev->num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE);
+
+ if (dev->log) {
+ munmap(dev->log, log_size);
+ }
free(dev->vqs);
close(dev->fd);
dev->fd = -1;
- ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name);
+ if (!ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name) &&
+ dev->shm_log_dir) {
+ vduse_log_destroy(dev->shm_log_dir, dev->name);
+ }
+ free(dev->shm_log_dir);
free(dev->name);
close(dev->ctrl_fd);
dev->ctrl_fd = -1;
diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h
index f6bcb51b5a..a46e71e0c2 100644
--- a/subprojects/libvduse/libvduse.h
+++ b/subprojects/libvduse/libvduse.h
@@ -171,6 +171,7 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size);
* @config_size: the size of the configuration space
* @config: the buffer of the configuration space
* @ops: the operation of VDUSE backend
+ * @shm_log_dir: directory to store the metadata file for reconnect
* @priv: private pointer
*
* Create VDUSE device.
@@ -180,7 +181,8 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size);
VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
uint32_t vendor_id, uint64_t features,
uint16_t num_queues, uint32_t config_size,
- char *config, const VduseOps *ops, void *priv);
+ char *config, const VduseOps *ops,
+ const char *shm_log_dir, void *priv);
/**
* vduse_dev_destroy:
--
2.20.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH 1/5] linux-headers: Add vduse.h
2022-01-25 13:17 ` [PATCH 1/5] linux-headers: Add vduse.h Xie Yongji
@ 2022-02-07 13:12 ` Stefan Hajnoczi
2022-02-08 3:13 ` Yongji Xie
0 siblings, 1 reply; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-07 13:12 UTC (permalink / raw)
To: Xie Yongji
Cc: kwolf, qemu-block, mst, jasowang, qemu-devel, mreitz, mlureau,
sgarzare
[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]
On Tue, Jan 25, 2022 at 09:17:56PM +0800, Xie Yongji wrote:
> diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh
> index fea4d6eb65..4c7846076f 100755
> --- a/scripts/update-linux-headers.sh
> +++ b/scripts/update-linux-headers.sh
> @@ -198,6 +198,7 @@ for i in "$tmpdir"/include/linux/*virtio*.h \
> "$tmpdir/include/linux/const.h" \
> "$tmpdir/include/linux/kernel.h" \
> "$tmpdir/include/linux/vhost_types.h" \
> + "$tmpdir/include/linux/vduse.h" \
> "$tmpdir/include/linux/sysinfo.h"; do
> cp_portable "$i" "$output/include/standard-headers/linux"
VDUSE is only available on Linux hosts so it should go in linux-headers/
instead of standard-headers/linux/:
# - linux-headers/ for files that are required for compiling for a
# Linux host. Generally we have these so we can use kernel structs
# and defines that are more recent than the headers that might be
# installed on the host system. Usually this script can do simple
# file copies for these headers.
#
# - include/standard-headers/ for files that are used for guest
# device emulation and are required on all hosts. For instance, we
# get our definitions of the virtio structures from the Linux
# kernel headers, but we need those definitions regardless of which
# host OS we are building for. This script has to be careful to
# sanitize the headers to remove any use of Linux-specifics such as
# types like "__u64". This work is done in the cp_portable function.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
2022-01-25 13:17 ` [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library Xie Yongji
@ 2022-02-07 14:00 ` Stefan Hajnoczi
2022-02-08 6:42 ` Yongji Xie
0 siblings, 1 reply; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-07 14:00 UTC (permalink / raw)
To: Xie Yongji
Cc: kwolf, qemu-block, mst, jasowang, qemu-devel, mreitz, mlureau,
sgarzare
[-- Attachment #1: Type: text/plain, Size: 44175 bytes --]
On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote:
> VDUSE [1] is a linux framework that makes it possible to implement
> software-emulated vDPA devices in userspace. This adds a library
> as a subproject to help implementing VDUSE backends in QEMU.
>
> [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
This library assumes that the program is allowed to access the control
device (/dev/vduse/control). Is that always the case or should the
library also support access to /dev/vduse/<name> only (maybe even with
file descriptor passing) so a privileged process can create/destroy
VDUSE devices?
I didn't review the vring code in detail.
>
> Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> ---
> meson.build | 15 +
> meson_options.txt | 2 +
> scripts/meson-buildoptions.sh | 3 +
> subprojects/libvduse/include/atomic.h | 1 +
> subprojects/libvduse/libvduse.c | 1025 +++++++++++++++++++
> subprojects/libvduse/libvduse.h | 193 ++++
> subprojects/libvduse/meson.build | 10 +
> subprojects/libvduse/standard-headers/linux | 1 +
> 8 files changed, 1250 insertions(+)
> create mode 120000 subprojects/libvduse/include/atomic.h
> create mode 100644 subprojects/libvduse/libvduse.c
> create mode 100644 subprojects/libvduse/libvduse.h
> create mode 100644 subprojects/libvduse/meson.build
> create mode 120000 subprojects/libvduse/standard-headers/linux
>
> diff --git a/meson.build b/meson.build
> index 333c61deba..864fb50ade 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1305,6 +1305,21 @@ if not get_option('fuse_lseek').disabled()
> endif
> endif
>
> +have_libvduse = (targetos == 'linux')
> +if get_option('libvduse').enabled()
> + if targetos != 'linux'
> + error('libvduse requires linux')
> + endif
> +elif get_option('libvduse').disabled()
> + have_libvduse = false
> +endif
> +
> +libvduse = not_found
> +if have_libvduse
> + libvduse_proj = subproject('libvduse')
> + libvduse = libvduse_proj.get_variable('libvduse_dep')
> +endif
> +
> # libbpf
> libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config')
> if libbpf.found() and not cc.links('''
> diff --git a/meson_options.txt b/meson_options.txt
> index 921967eddb..16790d1814 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -195,6 +195,8 @@ option('virtfs', type: 'feature', value: 'auto',
> description: 'virtio-9p support')
> option('virtiofsd', type: 'feature', value: 'auto',
> description: 'build virtiofs daemon (virtiofsd)')
> +option('libvduse', type: 'feature', value: 'auto',
> + description: 'build VDUSE Library')
>
> option('capstone', type: 'combo', value: 'auto',
> choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> index a4af02c527..af5c75d758 100644
> --- a/scripts/meson-buildoptions.sh
> +++ b/scripts/meson-buildoptions.sh
> @@ -58,6 +58,7 @@ meson_options_help() {
> printf "%s\n" ' libssh ssh block device support'
> printf "%s\n" ' libudev Use libudev to enumerate host devices'
> printf "%s\n" ' libusb libusb support for USB passthrough'
> + printf "%s\n" ' libvduse build VDUSE Library'
> printf "%s\n" ' libxml2 libxml2 support for Parallels image format'
> printf "%s\n" ' linux-aio Linux AIO support'
> printf "%s\n" ' linux-io-uring Linux io_uring support'
> @@ -188,6 +189,8 @@ _meson_option_parse() {
> --disable-libudev) printf "%s" -Dlibudev=disabled ;;
> --enable-libusb) printf "%s" -Dlibusb=enabled ;;
> --disable-libusb) printf "%s" -Dlibusb=disabled ;;
> + --enable-libvduse) printf "%s" -Dlibvduse=enabled ;;
> + --disable-libvduse) printf "%s" -Dlibvduse=disabled ;;
> --enable-libxml2) printf "%s" -Dlibxml2=enabled ;;
> --disable-libxml2) printf "%s" -Dlibxml2=disabled ;;
> --enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;;
> diff --git a/subprojects/libvduse/include/atomic.h b/subprojects/libvduse/include/atomic.h
> new file mode 120000
> index 0000000000..8c2be64f7b
> --- /dev/null
> +++ b/subprojects/libvduse/include/atomic.h
> @@ -0,0 +1 @@
> +../../../include/qemu/atomic.h
> \ No newline at end of file
> diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
> new file mode 100644
> index 0000000000..7671864bca
> --- /dev/null
> +++ b/subprojects/libvduse/libvduse.c
> @@ -0,0 +1,1025 @@
> +/*
> + * VDUSE (vDPA Device in Userspace) library
> + *
> + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> + * Portions of codes and concepts borrowed from libvhost-user.c, so:
> + * Copyright IBM, Corp. 2007
> + * Copyright (c) 2016 Red Hat, Inc.
> + *
> + * Author:
> + * Xie Yongji <xieyongji@bytedance.com>
> + * Anthony Liguori <aliguori@us.ibm.com>
> + * Marc-André Lureau <mlureau@redhat.com>
> + * Victor Kaplansky <victork@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later. See the COPYING file in the top-level directory.
> + */
> +
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <stdbool.h>
> +#include <stddef.h>
> +#include <errno.h>
> +#include <string.h>
> +#include <assert.h>
> +#include <endian.h>
> +#include <unistd.h>
> +#include <limits.h>
> +#include <fcntl.h>
> +
> +#include <sys/ioctl.h>
> +#include <sys/eventfd.h>
> +#include <sys/mman.h>
> +
> +#include "include/atomic.h"
> +#include "standard-headers/linux/vhost_types.h"
> +#include "standard-headers/linux/vduse.h"
> +#include "libvduse.h"
> +
> +#define VIRTQUEUE_MAX_SIZE 1024
> +#define VDUSE_VQ_ALIGN 4096
> +#define MAX_IOVA_REGIONS 256
> +
> +/* Round number down to multiple */
> +#define ALIGN_DOWN(n, m) ((n) / (m) * (m))
> +
> +/* Round number up to multiple */
> +#define ALIGN_UP(n, m) ALIGN_DOWN((n) + (m) - 1, (m))
> +
> +#ifndef unlikely
> +#define unlikely(x) __builtin_expect(!!(x), 0)
> +#endif
> +
> +typedef struct VduseRing {
> + unsigned int num;
> + uint64_t desc_addr;
> + uint64_t avail_addr;
> + uint64_t used_addr;
> + struct vring_desc *desc;
> + struct vring_avail *avail;
> + struct vring_used *used;
> +} VduseRing;
> +
> +struct VduseVirtq {
> + VduseRing vring;
> + uint16_t last_avail_idx;
> + uint16_t shadow_avail_idx;
> + uint16_t used_idx;
> + uint16_t signalled_used;
> + bool signalled_used_valid;
> + int index;
> + int inuse;
> + bool ready;
> + int fd;
> + VduseDev *dev;
> +};
> +
> +typedef struct VduseIovaRegion {
> + uint64_t iova;
> + uint64_t size;
> + uint64_t mmap_offset;
> + uint64_t mmap_addr;
> +} VduseIovaRegion;
> +
> +struct VduseDev {
> + VduseVirtq *vqs;
> + VduseIovaRegion regions[MAX_IOVA_REGIONS];
> + int num_regions;
> + char *name;
> + uint32_t device_id;
> + uint32_t vendor_id;
> + uint16_t num_queues;
> + uint16_t queue_size;
> + uint64_t features;
> + const VduseOps *ops;
> + int fd;
> + int ctrl_fd;
> + void *priv;
> +};
> +
> +static inline bool has_feature(uint64_t features, unsigned int fbit)
> +{
> + assert(fbit < 64);
> + return !!(features & (1ULL << fbit));
> +}
> +
> +static inline bool vduse_dev_has_feature(VduseDev *dev, unsigned int fbit)
> +{
> + return has_feature(dev->features, fbit);
> +}
> +
> +VduseDev *vduse_queue_get_dev(VduseVirtq *vq)
> +{
> + return vq->dev;
> +}
> +
> +int vduse_queue_get_fd(VduseVirtq *vq)
> +{
> + return vq->fd;
> +}
> +
> +void *vduse_dev_get_priv(VduseDev *dev)
> +{
> + return dev->priv;
> +}
> +
> +VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index)
> +{
> + return &dev->vqs[index];
> +}
> +
> +int vduse_dev_get_fd(VduseDev *dev)
> +{
> + return dev->fd;
> +}
> +
> +static int vduse_inject_irq(VduseDev *dev, int index)
> +{
> + return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index);
> +}
> +
> +static void vduse_iova_remove_region(VduseDev *dev, uint64_t start,
> + uint64_t last)
> +{
> + int i;
> +
> + if (last == start) {
> + return;
> + }
> +
> + for (i = 0; i < MAX_IOVA_REGIONS; i++) {
> + if (!dev->regions[i].mmap_addr) {
> + continue;
> + }
> +
> + if (start <= dev->regions[i].iova &&
> + last >= (dev->regions[i].iova + dev->regions[i].size - 1)) {
> + munmap((void *)dev->regions[i].mmap_addr,
> + dev->regions[i].mmap_offset + dev->regions[i].size);
> + dev->regions[i].mmap_addr = 0;
> + dev->num_regions--;
> + }
> + }
> +}
> +
> +static int vduse_iova_add_region(VduseDev *dev, int fd,
> + uint64_t offset, uint64_t start,
> + uint64_t last, int prot)
> +{
> + int i;
> + uint64_t size = last - start + 1;
> + void *mmap_addr = mmap(0, size + offset, prot, MAP_SHARED, fd, 0);
> +
> + if (mmap_addr == MAP_FAILED) {
> + return -EINVAL;
Missing close(fd). This function takes ownership of fd.
> + }
> +
> + for (i = 0; i < MAX_IOVA_REGIONS; i++) {
> + if (!dev->regions[i].mmap_addr) {
> + dev->regions[i].mmap_addr = (uint64_t)(uintptr_t)mmap_addr;
> + dev->regions[i].mmap_offset = offset;
> + dev->regions[i].iova = start;
> + dev->regions[i].size = size;
> + dev->num_regions++;
> + break;
> + }
> + }
> + close(fd);
assert(i < MAX_IOVA_REGIONS)? If we can really reach the end of the for
loop then we must remember to call munmap(2).
> +
> + return 0;
> +}
> +
> +static int perm_to_prot(uint8_t perm)
> +{
> + int prot = 0;
> +
> + switch (perm) {
> + case VDUSE_ACCESS_WO:
> + prot |= PROT_WRITE;
> + break;
> + case VDUSE_ACCESS_RO:
> + prot |= PROT_READ;
> + break;
> + case VDUSE_ACCESS_RW:
> + prot |= PROT_READ | PROT_WRITE;
> + break;
> + default:
> + break;
> + }
> +
> + return prot;
> +}
> +
> +static inline void *iova_to_va(VduseDev *dev, uint64_t *plen, uint64_t iova)
> +{
> + int i, ret;
> + struct vduse_iotlb_entry entry;
> +
> + for (i = 0; i < MAX_IOVA_REGIONS; i++) {
> + VduseIovaRegion *r = &dev->regions[i];
> +
> + if (!r->mmap_addr) {
> + continue;
> + }
> +
> + if ((iova >= r->iova) && (iova < (r->iova + r->size))) {
> + if ((iova + *plen) > (r->iova + r->size)) {
> + *plen = r->iova + r->size - iova;
> + }
> + return (void *)(uintptr_t)(iova - r->iova +
> + r->mmap_addr + r->mmap_offset);
> + }
> + }
> +
> + entry.start = iova;
> + entry.last = iova + 1;
> + ret = ioctl(dev->fd, VDUSE_IOTLB_GET_FD, &entry);
> + if (ret < 0) {
> + return NULL;
> + }
> +
> + if (!vduse_iova_add_region(dev, ret, entry.offset, entry.start,
> + entry.last, perm_to_prot(entry.perm))) {
> + return iova_to_va(dev, plen, iova);
> + }
> +
> + return NULL;
> +}
> +
> +static inline uint16_t vring_avail_flags(VduseVirtq *vq)
> +{
> + return le16toh(vq->vring.avail->flags);
I remember we discussed whether VDUSE should support Transitional
devices. VIRTIO 1.0+ uses little-endian but legacy VIRTIO uses
guest-endian, so le16toh() will not work for legacy VIRTIO vrings in a
cross-endian configuration (e.g. big-endian guest on little-endian
host).
If cross-endian isn't supported please add an error during
intialization so users get a clear error message.
> +}
> +
> +static inline uint16_t vring_avail_idx(VduseVirtq *vq)
> +{
> + vq->shadow_avail_idx = le16toh(vq->vring.avail->idx);
> +
> + return vq->shadow_avail_idx;
> +}
> +
> +static inline uint16_t vring_avail_ring(VduseVirtq *vq, int i)
> +{
> + return le16toh(vq->vring.avail->ring[i]);
> +}
> +
> +static inline uint16_t vring_get_used_event(VduseVirtq *vq)
> +{
> + return vring_avail_ring(vq, vq->vring.num);
> +}
> +
> +static bool vduse_queue_get_head(VduseVirtq *vq, unsigned int idx,
> + unsigned int *head)
> +{
> + /*
> + * Grab the next descriptor number they're advertising, and increment
> + * the index we've seen.
> + */
> + *head = vring_avail_ring(vq, idx % vq->vring.num);
> +
> + /* If their number is silly, that's a fatal mistake. */
> + if (*head >= vq->vring.num) {
> + fprintf(stderr, "Guest says index %u is available\n", *head);
> + return false;
> + }
> +
> + return true;
> +}
> +
> +static int
> +vduse_queue_read_indirect_desc(VduseDev *dev, struct vring_desc *desc,
> + uint64_t addr, size_t len)
> +{
> + struct vring_desc *ori_desc;
> + uint64_t read_len;
> +
> + if (len > (VIRTQUEUE_MAX_SIZE * sizeof(struct vring_desc))) {
> + return -1;
> + }
> +
> + if (len == 0) {
> + return -1;
> + }
> +
> + while (len) {
> + read_len = len;
> + ori_desc = iova_to_va(dev, &read_len, addr);
> + if (!ori_desc) {
> + return -1;
> + }
> +
> + memcpy(desc, ori_desc, read_len);
> + len -= read_len;
> + addr += read_len;
> + desc += read_len;
> + }
> +
> + return 0;
> +}
> +
> +enum {
> + VIRTQUEUE_READ_DESC_ERROR = -1,
> + VIRTQUEUE_READ_DESC_DONE = 0, /* end of chain */
> + VIRTQUEUE_READ_DESC_MORE = 1, /* more buffers in chain */
> +};
> +
> +static int vduse_queue_read_next_desc(struct vring_desc *desc, int i,
> + unsigned int max, unsigned int *next)
> +{
> + /* If this descriptor says it doesn't chain, we're done. */
> + if (!(le16toh(desc[i].flags) & VRING_DESC_F_NEXT)) {
> + return VIRTQUEUE_READ_DESC_DONE;
> + }
> +
> + /* Check they're not leading us off end of descriptors. */
> + *next = desc[i].next;
> + /* Make sure compiler knows to grab that: we don't want it changing! */
> + smp_wmb();
> +
> + if (*next >= max) {
> + fprintf(stderr, "Desc next is %u\n", *next);
> + return VIRTQUEUE_READ_DESC_ERROR;
> + }
> +
> + return VIRTQUEUE_READ_DESC_MORE;
> +}
> +
> +/*
> + * Fetch avail_idx from VQ memory only when we really need to know if
> + * guest has added some buffers.
> + */
> +static bool vduse_queue_empty(VduseVirtq *vq)
> +{
> + if (unlikely(!vq->vring.avail)) {
> + return true;
> + }
> +
> + if (vq->shadow_avail_idx != vq->last_avail_idx) {
> + return false;
> + }
> +
> + return vring_avail_idx(vq) == vq->last_avail_idx;
> +}
> +
> +static bool vduse_queue_should_notify(VduseVirtq *vq)
> +{
> + VduseDev *dev = vq->dev;
> + uint16_t old, new;
> + bool v;
> +
> + /* We need to expose used array entries before checking used event. */
> + smp_mb();
> +
> + /* Always notify when queue is empty (when feature acknowledge) */
> + if (vduse_dev_has_feature(dev, VIRTIO_F_NOTIFY_ON_EMPTY) &&
> + !vq->inuse && vduse_queue_empty(vq)) {
> + return true;
> + }
> +
> + if (!vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) {
> + return !(vring_avail_flags(vq) & VRING_AVAIL_F_NO_INTERRUPT);
> + }
> +
> + v = vq->signalled_used_valid;
> + vq->signalled_used_valid = true;
> + old = vq->signalled_used;
> + new = vq->signalled_used = vq->used_idx;
> + return !v || vring_need_event(vring_get_used_event(vq), new, old);
> +}
> +
> +void vduse_queue_notify(VduseVirtq *vq)
> +{
> + VduseDev *dev = vq->dev;
> +
> + if (unlikely(!vq->vring.avail)) {
> + return;
> + }
> +
> + if (!vduse_queue_should_notify(vq)) {
> + return;
> + }
> +
> + if (vduse_inject_irq(dev, vq->index) < 0) {
> + fprintf(stderr, "Error inject irq for vq %d: %s\n",
> + vq->index, strerror(errno));
> + }
> +}
> +
> +static inline void vring_used_flags_set_bit(VduseVirtq *vq, int mask)
> +{
> + uint16_t *flags;
> +
> + flags = (uint16_t *)((char*)vq->vring.used +
> + offsetof(struct vring_used, flags));
> + *flags = htole16(le16toh(*flags) | mask);
> +}
> +
> +static inline void vring_used_flags_unset_bit(VduseVirtq *vq, int mask)
> +{
> + uint16_t *flags;
> +
> + flags = (uint16_t *)((char*)vq->vring.used +
> + offsetof(struct vring_used, flags));
> + *flags = htole16(le16toh(*flags) & ~mask);
> +}
> +
> +static inline void vring_set_avail_event(VduseVirtq *vq, uint16_t val)
> +{
> + *((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
> +}
> +
> +static bool vduse_queue_map_single_desc(VduseVirtq *vq, unsigned int *p_num_sg,
> + struct iovec *iov, unsigned int max_num_sg,
> + bool is_write, uint64_t pa, size_t sz)
> +{
> + unsigned num_sg = *p_num_sg;
> + VduseDev *dev = vq->dev;
> +
> + assert(num_sg <= max_num_sg);
> +
> + if (!sz) {
> + fprintf(stderr, "virtio: zero sized buffers are not allowed\n");
> + return false;
> + }
> +
> + while (sz) {
> + uint64_t len = sz;
> +
> + if (num_sg == max_num_sg) {
> + fprintf(stderr,
> + "virtio: too many descriptors in indirect table\n");
> + return false;
> + }
> +
> + iov[num_sg].iov_base = iova_to_va(dev, &len, pa);
> + if (iov[num_sg].iov_base == NULL) {
> + fprintf(stderr, "virtio: invalid address for buffers\n");
> + return false;
> + }
> + iov[num_sg++].iov_len = len;
> + sz -= len;
> + pa += len;
> + }
> +
> + *p_num_sg = num_sg;
> + return true;
> +}
> +
> +static void *vduse_queue_alloc_element(size_t sz, unsigned out_num,
> + unsigned in_num)
> +{
> + VduseVirtqElement *elem;
> + size_t in_sg_ofs = ALIGN_UP(sz, __alignof__(elem->in_sg[0]));
> + size_t out_sg_ofs = in_sg_ofs + in_num * sizeof(elem->in_sg[0]);
> + size_t out_sg_end = out_sg_ofs + out_num * sizeof(elem->out_sg[0]);
> +
> + assert(sz >= sizeof(VduseVirtqElement));
> + elem = malloc(out_sg_end);
Missing malloc() NULL return value check.
> + elem->out_num = out_num;
> + elem->in_num = in_num;
> + elem->in_sg = (void *)elem + in_sg_ofs;
> + elem->out_sg = (void *)elem + out_sg_ofs;
> + return elem;
> +}
> +
> +static void *vduse_queue_map_desc(VduseVirtq *vq, unsigned int idx, size_t sz)
> +{
> + struct vring_desc *desc = vq->vring.desc;
> + VduseDev *dev = vq->dev;
> + uint64_t desc_addr, read_len;
> + unsigned int desc_len;
> + unsigned int max = vq->vring.num;
> + unsigned int i = idx;
> + VduseVirtqElement *elem;
> + struct iovec iov[VIRTQUEUE_MAX_SIZE];
> + struct vring_desc desc_buf[VIRTQUEUE_MAX_SIZE];
> + unsigned int out_num = 0, in_num = 0;
> + int rc;
> +
> + if (le16toh(desc[i].flags) & VRING_DESC_F_INDIRECT) {
> + if (le32toh(desc[i].len) % sizeof(struct vring_desc)) {
> + fprintf(stderr, "Invalid size for indirect buffer table\n");
> + return NULL;
> + }
> +
> + /* loop over the indirect descriptor table */
> + desc_addr = le64toh(desc[i].addr);
> + desc_len = le32toh(desc[i].len);
> + max = desc_len / sizeof(struct vring_desc);
> + read_len = desc_len;
> + desc = iova_to_va(dev, &read_len, desc_addr);
> + if (unlikely(desc && read_len != desc_len)) {
> + /* Failed to use zero copy */
> + desc = NULL;
> + if (!vduse_queue_read_indirect_desc(dev, desc_buf,
> + desc_addr,
> + desc_len)) {
> + desc = desc_buf;
> + }
> + }
> + if (!desc) {
> + fprintf(stderr, "Invalid indirect buffer table\n");
> + return NULL;
> + }
> + i = 0;
> + }
> +
> + /* Collect all the descriptors */
> + do {
> + if (le16toh(desc[i].flags) & VRING_DESC_F_WRITE) {
> + if (!vduse_queue_map_single_desc(vq, &in_num, iov + out_num,
> + VIRTQUEUE_MAX_SIZE - out_num,
> + true, le64toh(desc[i].addr),
> + le32toh(desc[i].len))) {
> + return NULL;
> + }
> + } else {
> + if (in_num) {
> + fprintf(stderr, "Incorrect order for descriptors\n");
> + return NULL;
> + }
> + if (!vduse_queue_map_single_desc(vq, &out_num, iov,
> + VIRTQUEUE_MAX_SIZE, false,
> + le64toh(desc[i].addr),
> + le32toh(desc[i].len))) {
> + return NULL;
> + }
> + }
> +
> + /* If we've got too many, that implies a descriptor loop. */
> + if ((in_num + out_num) > max) {
> + fprintf(stderr, "Looped descriptor\n");
> + return NULL;
> + }
> + rc = vduse_queue_read_next_desc(desc, i, max, &i);
> + } while (rc == VIRTQUEUE_READ_DESC_MORE);
> +
> + if (rc == VIRTQUEUE_READ_DESC_ERROR) {
> + fprintf(stderr, "read descriptor error\n");
> + return NULL;
> + }
> +
> + /* Now copy what we have collected and mapped */
> + elem = vduse_queue_alloc_element(sz, out_num, in_num);
> + elem->index = idx;
> + for (i = 0; i < out_num; i++) {
> + elem->out_sg[i] = iov[i];
> + }
> + for (i = 0; i < in_num; i++) {
> + elem->in_sg[i] = iov[out_num + i];
> + }
> +
> + return elem;
> +}
> +
> +void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
> +{
> + unsigned int head;
> + VduseVirtqElement *elem;
> + VduseDev *dev = vq->dev;
> +
> + if (unlikely(!vq->vring.avail)) {
> + return NULL;
> + }
> +
> + if (vduse_queue_empty(vq)) {
> + return NULL;
> + }
> + /* Needed after virtio_queue_empty() */
> + smp_rmb();
> +
> + if (vq->inuse >= vq->vring.num) {
> + fprintf(stderr, "Virtqueue size exceeded: %d\n", vq->inuse);
> + return NULL;
> + }
> +
> + if (!vduse_queue_get_head(vq, vq->last_avail_idx++, &head)) {
> + return NULL;
> + }
> +
> + if (vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) {
> + vring_set_avail_event(vq, vq->last_avail_idx);
> + }
> +
> + elem = vduse_queue_map_desc(vq, head, sz);
> +
> + if (!elem) {
> + return NULL;
> + }
> +
> + vq->inuse++;
> +
> + return elem;
> +}
> +
> +static inline void vring_used_write(VduseVirtq *vq,
> + struct vring_used_elem *uelem, int i)
> +{
> + struct vring_used *used = vq->vring.used;
> +
> + used->ring[i] = *uelem;
> +}
> +
> +static void vduse_queue_fill(VduseVirtq *vq, const VduseVirtqElement *elem,
> + unsigned int len, unsigned int idx)
> +{
> + struct vring_used_elem uelem;
> +
> + if (unlikely(!vq->vring.used)) {
> + return;
> + }
> +
> + idx = (idx + vq->used_idx) % vq->vring.num;
> +
> + uelem.id = htole32(elem->index);
> + uelem.len = htole32(len);
> + vring_used_write(vq, &uelem, idx);
> +}
> +
> +static inline void vring_used_idx_set(VduseVirtq *vq, uint16_t val)
> +{
> + vq->vring.used->idx = htole16(val);
> + vq->used_idx = val;
> +}
> +
> +static void vduse_queue_flush(VduseVirtq *vq, unsigned int count)
> +{
> + uint16_t old, new;
> +
> + if (unlikely(!vq->vring.used)) {
> + return;
> + }
> +
> + /* Make sure buffer is written before we update index. */
> + smp_wmb();
> +
> + old = vq->used_idx;
> + new = old + count;
> + vring_used_idx_set(vq, new);
> + vq->inuse -= count;
> + if (unlikely((int16_t)(new - vq->signalled_used) < (uint16_t)(new - old))) {
> + vq->signalled_used_valid = false;
> + }
> +}
> +
> +void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
> + unsigned int len)
> +{
> + vduse_queue_fill(vq, elem, len, 0);
> + vduse_queue_flush(vq, 1);
> +}
> +
> +static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr,
> + uint64_t avail_addr, uint64_t used_addr)
> +{
> + struct VduseDev *dev = vq->dev;
> + uint64_t len;
> +
> + len = sizeof(struct vring_desc);
> + vq->vring.desc = iova_to_va(dev, &len, desc_addr);
> + assert(len == sizeof(struct vring_desc));
> +
> + len = sizeof(struct vring_avail);
> + vq->vring.avail = iova_to_va(dev, &len, avail_addr);
> + assert(len == sizeof(struct vring_avail));
> +
> + len = sizeof(struct vring_used);
> + vq->vring.used = iova_to_va(dev, &len, used_addr);
> + assert(len == sizeof(struct vring_used));
> +
> + if (!vq->vring.desc || !vq->vring.avail || !vq->vring.used) {
> + fprintf(stderr, "Failed to get vq[%d] iova mapping\n", vq->index);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static void vduse_queue_enable(VduseVirtq *vq)
> +{
> + struct VduseDev *dev = vq->dev;
> + struct vduse_vq_info vq_info;
> + struct vduse_vq_eventfd vq_eventfd;
> + int fd;
> +
> + vq_info.index = vq->index;
> + if (ioctl(dev->fd, VDUSE_VQ_GET_INFO, &vq_info)) {
> + fprintf(stderr, "Failed to get vq[%d] info: %s\n",
> + vq->index, strerror(errno));
> + return;
> + }
> +
> + if (!vq_info.ready) {
> + return;
> + }
> +
> + vq->vring.num = vq_info.num;
> + vq->vring.desc_addr = vq_info.desc_addr;
> + vq->vring.avail_addr = vq_info.driver_addr;
> + vq->vring.used_addr = vq_info.device_addr;
> +
> + if (vduse_queue_update_vring(vq, vq_info.desc_addr,
> + vq_info.driver_addr, vq_info.device_addr)) {
> + fprintf(stderr, "Failed to update vring for vq[%d]\n", vq->index);
> + return;
> + }
> +
> + fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC);
> + if (fd < 0) {
> + fprintf(stderr, "Failed to init eventfd for vq[%d]\n", vq->index);
> + return;
> + }
> +
> + vq_eventfd.index = vq->index;
> + vq_eventfd.fd = fd;
> + if (ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &vq_eventfd)) {
> + fprintf(stderr, "Failed to setup kick fd for vq[%d]\n", vq->index);
> + close(fd);
> + return;
> + }
> +
> + vq->fd = fd;
> + vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index;
> + vq->inuse = 0;
> + vq->used_idx = 0;
> + vq->signalled_used_valid = false;
> + vq->ready = true;
> +
> + dev->ops->enable_queue(dev, vq);
> +}
> +
> +static void vduse_queue_disable(VduseVirtq *vq)
> +{
> + struct VduseDev *dev = vq->dev;
> + struct vduse_vq_eventfd eventfd;
> +
> + if (!vq->ready) {
> + return;
> + }
> +
> + dev->ops->disable_queue(dev, vq);
> +
> + eventfd.index = vq->index;
> + eventfd.fd = VDUSE_EVENTFD_DEASSIGN;
> + ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &eventfd);
> + close(vq->fd);
> +
> + assert(vq->inuse == 0);
> +
> + vq->vring.num = 0;
> + vq->vring.desc_addr = 0;
> + vq->vring.avail_addr = 0;
> + vq->vring.used_addr = 0;
> + vq->vring.desc = 0;
> + vq->vring.avail = 0;
> + vq->vring.used = 0;
> + vq->ready = false;
> + vq->fd = -1;
> +}
> +
> +static void vduse_dev_start_dataplane(VduseDev *dev)
> +{
> + int i;
> +
> + if (ioctl(dev->fd, VDUSE_DEV_GET_FEATURES, &dev->features)) {
> + fprintf(stderr, "Failed to get features: %s\n", strerror(errno));
> + return;
> + }
> +
> + for (i = 0; i < dev->num_queues; i++) {
> + vduse_queue_enable(&dev->vqs[i]);
> + }
> +}
> +
> +static void vduse_dev_stop_dataplane(VduseDev *dev)
> +{
> + int i;
> +
> + for (i = 0; i < dev->num_queues; i++) {
> + vduse_queue_disable(&dev->vqs[i]);
> + }
> + dev->features = 0;
> + vduse_iova_remove_region(dev, 0, ULONG_MAX);
> +}
> +
> +int vduse_dev_handler(VduseDev *dev)
> +{
> + struct vduse_dev_request req;
> + struct vduse_dev_response resp = { 0 };
> + VduseVirtq *vq;
> + int i, ret;
> +
> + ret = read(dev->fd, &req, sizeof(req));
This file descriptor is blocking? I guess the assumption is that the
kernel VDUSE code always enqueues at least one struct vduse_dev_request,
so userspace will not block when the file descriptor becomes readable?
> + if (ret != sizeof(req)) {
> + fprintf(stderr, "Read request error [%d]: %s\n",
> + ret, strerror(errno));
> + return -errno;
> + }
> + resp.request_id = req.request_id;
> +
> + switch (req.type) {
> + case VDUSE_GET_VQ_STATE:
> + vq = &dev->vqs[req.vq_state.index];
> + resp.vq_state.split.avail_index = vq->last_avail_idx;
> + resp.result = VDUSE_REQ_RESULT_OK;
> + break;
> + case VDUSE_SET_STATUS:
> + if (req.s.status & VIRTIO_CONFIG_S_DRIVER_OK) {
> + vduse_dev_start_dataplane(dev);
> + } else if (req.s.status == 0) {
> + vduse_dev_stop_dataplane(dev);
> + }
> + resp.result = VDUSE_REQ_RESULT_OK;
> + break;
> + case VDUSE_UPDATE_IOTLB:
> + /* The iova will be updated by iova_to_va() later, so just remove it */
> + vduse_iova_remove_region(dev, req.iova.start, req.iova.last);
> + for (i = 0; i < dev->num_queues; i++) {
> + VduseVirtq *vq = &dev->vqs[i];
> + if (vq->ready) {
> + if (vduse_queue_update_vring(vq, vq->vring.desc_addr,
> + vq->vring.avail_addr,
> + vq->vring.used_addr)) {
> + fprintf(stderr, "Failed to update vring for vq[%d]\n",
> + vq->index);
> + }
> + }
> + }
> + resp.result = VDUSE_REQ_RESULT_OK;
> + break;
> + default:
> + resp.result = VDUSE_REQ_RESULT_FAILED;
> + break;
> + }
> +
> + ret = write(dev->fd, &resp, sizeof(resp));
The kernel never blocks here?
> + if (ret != sizeof(resp)) {
> + fprintf(stderr, "Write request %d error [%d]: %s\n",
> + req.type, ret, strerror(errno));
> + return -errno;
> + }
> + return 0;
> +}
> +
> +int vduse_dev_update_config(VduseDev *dev, uint32_t size,
> + uint32_t offset, char *buffer)
> +{
> + int ret;
> + struct vduse_config_data *data;
> +
> + data = malloc(offsetof(struct vduse_config_data, buffer) + size);
> + if (!data) {
> + return -ENOMEM;
> + }
> +
> + data->offset = offset;
> + data->length = size;
> + memcpy(data->buffer, buffer, size);
> +
> + ret = ioctl(dev->fd, VDUSE_DEV_SET_CONFIG, data);
> + free(data);
> +
> + if (ret) {
> + return -errno;
> + }
> +
> + if (ioctl(dev->fd, VDUSE_DEV_INJECT_CONFIG_IRQ)) {
> + return -errno;
> + }
> +
> + return 0;
> +}
> +
> +int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size)
> +{
> + VduseVirtq *vq = &dev->vqs[index];
> + struct vduse_vq_config vq_config = { 0 };
> +
> + vq_config.index = vq->index;
> + vq_config.max_size = max_size;
> +
> + if (ioctl(dev->fd, VDUSE_VQ_SETUP, &vq_config)) {
> + return -errno;
> + }
> +
> + return 0;
> +}
> +
> +VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> + uint32_t vendor_id, uint64_t features,
> + uint16_t num_queues, uint32_t config_size,
> + char *config, const VduseOps *ops, void *priv)
> +{
> + VduseDev *dev;
> + int i, ret, ctrl_fd, fd = -1;
> + uint64_t version;
> + char dev_path[VDUSE_NAME_MAX + 16];
Why 16? It has to be at least strlen("/dev/vduse/"), but why more? I
suggest including strlen("/dev/vduse/") instead of hardcoding a magic
constant.
> + VduseVirtq *vqs = NULL;
> + struct vduse_dev_config *dev_config = NULL;
> + size_t size = offsetof(struct vduse_dev_config, config);
> +
> + if (!name || strlen(name) > VDUSE_NAME_MAX || !config ||
The NUL terminator needs to be taken into account:
strlen(name) + 1 > VDUSE_NAME_MAX
> + !config_size || !ops || !ops->enable_queue || !ops->disable_queue) {
> + fprintf(stderr, "Invalid parameter for vduse\n");
> + return NULL;
> + }
> +
> + dev = malloc(sizeof(VduseDev));
> + if (!dev) {
> + fprintf(stderr, "Failed to allocate vduse device\n");
> + return NULL;
> + }
> + memset(dev, 0, sizeof(VduseDev));
> +
> + ctrl_fd = open("/dev/vduse/control", O_RDWR);
> + if (ctrl_fd < 0) {
> + fprintf(stderr, "Failed to open /dev/vduse/control: %s\n",
> + strerror(errno));
> + goto err_ctrl;
> + }
> +
> + version = VDUSE_API_VERSION;
> + if (ioctl(ctrl_fd, VDUSE_SET_API_VERSION, &version)) {
> + fprintf(stderr, "Failed to set api version %lu: %s\n",
> + version, strerror(errno));
> + goto err_dev;
> + }
> +
> + dev_config = malloc(size + config_size);
> + if (!dev_config) {
> + fprintf(stderr, "Failed to allocate config space\n");
> + goto err_dev;
> + }
> + memset(dev_config, 0, size + config_size);
> +
> + strcpy(dev_config->name, name);
> + dev_config->device_id = device_id;
> + dev_config->vendor_id = vendor_id;
> + dev_config->features = features;
> + dev_config->vq_num = num_queues;
> + dev_config->vq_align = VDUSE_VQ_ALIGN;
> + dev_config->config_size = config_size;
> + memcpy(dev_config->config, config, config_size);
> +
> + ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config);
> + free(dev_config);
> + if (ret < 0) {
> + fprintf(stderr, "Failed to create vduse dev %s: %s\n",
> + name, strerror(errno));
> + goto err_dev;
> + }
> +
> + sprintf(dev_path, "/dev/vduse/%s", name);
> + fd = open(dev_path, O_RDWR);
Does the caller reject names with ".." path components? Maybe input
validation should be performed before we call open(2)?
> + if (fd < 0) {
> + fprintf(stderr, "Failed to open vduse dev %s: %s\n",
> + name, strerror(errno));
> + goto err;
> + }
> +
> + vqs = calloc(sizeof(VduseVirtq), num_queues);
calloc() could be used instead of malloc + memset above as well.
> + if (!vqs) {
> + fprintf(stderr, "Failed to allocate virtqueues\n");
> + goto err;
> + }
> +
> + for (i = 0; i < num_queues; i++) {
> + vqs[i].index = i;
> + vqs[i].dev = dev;
> + vqs[i].fd = -1;
> + }
> +
> + dev->vqs = vqs;
> + dev->name = strdup(name);
malloc(3) return values are checked elsewhere, strdup(3) should also be
checked.
> + dev->num_queues = num_queues;
> + dev->ops = ops;
> + dev->ctrl_fd = ctrl_fd;
> + dev->fd = fd;
> + dev->priv = priv;
> +
> + return dev;
> +err:
> + if (fd > 0) {
> + close(fd);
> + }
> + ioctl(ctrl_fd, VDUSE_DESTROY_DEV, name);
> +err_dev:
> + close(ctrl_fd);
> +err_ctrl:
> + free(dev);
> +
> + return NULL;
> +}
> +
> +void vduse_dev_destroy(VduseDev *dev)
> +{
> + free(dev->vqs);
> + close(dev->fd);
> + dev->fd = -1;
> + ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name);
> + free(dev->name);
> + close(dev->ctrl_fd);
> + dev->ctrl_fd = -1;
> + free(dev);
> +}
> diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h
> new file mode 100644
> index 0000000000..f6bcb51b5a
> --- /dev/null
> +++ b/subprojects/libvduse/libvduse.h
> @@ -0,0 +1,193 @@
> +/*
> + * VDUSE (vDPA Device in Userspace) library
> + *
> + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> + *
> + * Author:
> + * Xie Yongji <xieyongji@bytedance.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later. See the COPYING file in the top-level directory.
> + */
> +
> +#ifndef LIBVDUSE_H
> +#define LIBVDUSE_H
> +
> +#include <stdint.h>
> +#include <sys/uio.h>
> +
> +/* VDUSE device structure */
> +typedef struct VduseDev VduseDev;
> +
> +/* Virtqueue structure */
> +typedef struct VduseVirtq VduseVirtq;
> +
> +/* Some operation of VDUSE backend */
> +typedef struct VduseOps {
> + /* Called when virtqueue can be processed */
> + void (*enable_queue)(VduseDev *dev, VduseVirtq *vq);
> + /* Called when virtqueue processing should be stopped */
> + void (*disable_queue)(VduseDev *dev, VduseVirtq *vq);
> +} VduseOps;
> +
> +/* Describing elements of the I/O buffer */
> +typedef struct VduseVirtqElement {
> + /* Virtqueue index */
> + unsigned int index;
Is this the descriptor table index or the virtqueue number?
> + /* Number of physically-contiguous device-readable descriptors */
> + unsigned int out_num;
> + /* Number of physically-contiguous device-writable descriptors */
> + unsigned int in_num;
> + /* Array to store physically-contiguous device-writable descriptors */
> + struct iovec *in_sg;
> + /* Array to store physically-contiguous device-readable descriptors */
> + struct iovec *out_sg;
> +} VduseVirtqElement;
> +
> +/**
> + * vduse_queue_get_dev:
> + * @vq: specified virtqueue
> + *
> + * Get corresponding VDUSE device from the virtqueue.
> + *
> + * Returns: a pointer to VDUSE device on success, NULL on failure.
> + */
> +VduseDev *vduse_queue_get_dev(VduseVirtq *vq);
> +
> +/**
> + * vduse_queue_get_fd:
> + * @vq: specified virtqueue
> + *
> + * Get the kick fd for the virtqueue.
> + *
> + * Returns: file descriptor on success, -1 on failure.
> + */
> +int vduse_queue_get_fd(VduseVirtq *vq);
> +
> +/**
> + * vduse_queue_pop:
> + * @vq: specified virtqueue
> + * @sz: the size of struct to return (must be >= VduseVirtqElement)
> + *
> + * Pop an element from virtqueue available ring.
> + *
> + * Returns: a pointer to a structure containing VduseVirtqElement on success,
> + * NULL on failure.
> + */
> +void *vduse_queue_pop(VduseVirtq *vq, size_t sz);
> +
> +/**
> + * vduse_queue_push:
> + * @vq: specified virtqueue
> + * @elem: pointer to VduseVirtqElement returned by vduse_queue_pop()
> + * @len: length in bytes to write
> + *
> + * Push an element to virtqueue used ring.
> + */
> +void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
> + unsigned int len);
> +/**
> + * vduse_queue_notify:
> + * @vq: specified virtqueue
> + *
> + * Request to notify the queue.
> + */
> +void vduse_queue_notify(VduseVirtq *vq);
> +
> +/**
> + * vduse_dev_get_priv:
> + * @dev: VDUSE device
> + *
> + * Get the private pointer passed to vduse_dev_create().
> + *
> + * Returns: private pointer on success, NULL on failure.
> + */
> +void *vduse_dev_get_priv(VduseDev *dev);
> +
> +/**
> + * vduse_dev_get_queue:
> + * @dev: VDUSE device
> + * @index: virtqueue index
> + *
> + * Get the specified virtqueue.
> + *
> + * Returns: a pointer to the virtqueue on success, NULL on failure.
> + */
> +VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index);
> +
> +/**
> + * vduse_dev_get_fd:
> + * @dev: VDUSE device
> + *
> + * Get the control message fd for the VDUSE device.
> + *
> + * Returns: file descriptor on success, -1 on failure.
> + */
> +int vduse_dev_get_fd(VduseDev *dev);
> +
> +/**
> + * vduse_dev_handler:
> + * @dev: VDUSE device
> + *
> + * Used to process the control message.
> + *
> + * Returns: file descriptor on success, -errno on failure.
> + */
> +int vduse_dev_handler(VduseDev *dev);
> +
> +/**
> + * vduse_dev_update_config:
> + * @dev: VDUSE device
> + * @size: the size to write to configuration space
> + * @offset: the offset from the beginning of configuration space
> + * @buffer: the buffer used to write from
> + *
> + * Update device configuration space and inject a config interrupt.
> + *
> + * Returns: 0 on success, -errno on failure.
> + */
> +int vduse_dev_update_config(VduseDev *dev, uint32_t size,
> + uint32_t offset, char *buffer);
> +
> +/**
> + * vduse_dev_setup_queue:
> + * @dev: VDUSE device
> + * @index: virtqueue index
> + * @max_size: the max size of virtqueue
> + *
> + * Setup the specified virtqueue.
> + *
> + * Returns: 0 on success, -errno on failure.
> + */
> +int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size);
> +
> +/**
> + * vduse_dev_create:
> + * @name: VDUSE device name
> + * @device_id: virtio device id
> + * @vendor_id: virtio vendor id
> + * @features: virtio features
> + * @num_queues: the number of virtqueues
> + * @config_size: the size of the configuration space
> + * @config: the buffer of the configuration space
> + * @ops: the operation of VDUSE backend
> + * @priv: private pointer
> + *
> + * Create VDUSE device.
> + *
> + * Returns: pointer to VDUSE device on success, NULL on failure.
> + */
> +VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> + uint32_t vendor_id, uint64_t features,
> + uint16_t num_queues, uint32_t config_size,
> + char *config, const VduseOps *ops, void *priv);
> +
> +/**
> + * vduse_dev_destroy:
> + * @dev: VDUSE device
> + *
> + * Destroy the VDUSE device.
> + */
> +void vduse_dev_destroy(VduseDev *dev);
> +
> +#endif
> diff --git a/subprojects/libvduse/meson.build b/subprojects/libvduse/meson.build
> new file mode 100644
> index 0000000000..ba08f5ee1a
> --- /dev/null
> +++ b/subprojects/libvduse/meson.build
> @@ -0,0 +1,10 @@
> +project('libvduse', 'c',
> + license: 'GPL-2.0-or-later',
> + default_options: ['c_std=gnu99'])
> +
> +libvduse = static_library('vduse',
> + files('libvduse.c'),
> + c_args: '-D_GNU_SOURCE')
> +
> +libvduse_dep = declare_dependency(link_with: libvduse,
> + include_directories: include_directories('.'))
> diff --git a/subprojects/libvduse/standard-headers/linux b/subprojects/libvduse/standard-headers/linux
> new file mode 120000
> index 0000000000..c416f068ac
> --- /dev/null
> +++ b/subprojects/libvduse/standard-headers/linux
> @@ -0,0 +1 @@
> +../../../include/standard-headers/linux/
> \ No newline at end of file
> --
> 2.20.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 3/5] vduse-blk: implements vduse-blk export
2022-01-25 13:17 ` [PATCH 3/5] vduse-blk: implements vduse-blk export Xie Yongji
@ 2022-02-07 14:15 ` Stefan Hajnoczi
2022-02-08 6:56 ` Yongji Xie
0 siblings, 1 reply; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-07 14:15 UTC (permalink / raw)
To: Xie Yongji
Cc: kwolf, qemu-block, mst, jasowang, qemu-devel, mreitz, mlureau,
sgarzare
[-- Attachment #1: Type: text/plain, Size: 23921 bytes --]
On Tue, Jan 25, 2022 at 09:17:58PM +0800, Xie Yongji wrote:
> This implements a VDUSE block backends based on
> the libvduse library. We can use it to export the BDSs
> for both VM and container (host) usage.
>
> The new command-line syntax is:
>
> $ qemu-storage-daemon \
> --blockdev file,node-name=drive0,filename=test.img \
> --export vduse-blk,node-name=drive0,id=vduse-export0,writable=on
>
> After the qemu-storage-daemon started, we need to use
> the "vdpa" command to attach the device to vDPA bus:
>
> $ vdpa dev add name vduse-export0 mgmtdev vduse
>
> Also the device must be removed via the "vdpa" command
> before we stop the qemu-storage-daemon.
>
> Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> ---
> block/export/export.c | 6 +
> block/export/meson.build | 5 +
> block/export/vduse-blk.c | 427 ++++++++++++++++++++++++++++++++++
> block/export/vduse-blk.h | 20 ++
> meson.build | 13 ++
> meson_options.txt | 2 +
> qapi/block-export.json | 24 +-
> scripts/meson-buildoptions.sh | 4 +
> 8 files changed, 499 insertions(+), 2 deletions(-)
> create mode 100644 block/export/vduse-blk.c
> create mode 100644 block/export/vduse-blk.h
>
> diff --git a/block/export/export.c b/block/export/export.c
> index 6d3b9964c8..00dd505540 100644
> --- a/block/export/export.c
> +++ b/block/export/export.c
> @@ -26,6 +26,9 @@
> #ifdef CONFIG_VHOST_USER_BLK_SERVER
> #include "vhost-user-blk-server.h"
> #endif
> +#ifdef CONFIG_VDUSE_BLK_EXPORT
> +#include "vduse-blk.h"
> +#endif
>
> static const BlockExportDriver *blk_exp_drivers[] = {
> &blk_exp_nbd,
> @@ -35,6 +38,9 @@ static const BlockExportDriver *blk_exp_drivers[] = {
> #ifdef CONFIG_FUSE
> &blk_exp_fuse,
> #endif
> +#ifdef CONFIG_VDUSE_BLK_EXPORT
> + &blk_exp_vduse_blk,
> +#endif
> };
>
> /* Only accessed from the main thread */
> diff --git a/block/export/meson.build b/block/export/meson.build
> index 0a08e384c7..cf311d2b1b 100644
> --- a/block/export/meson.build
> +++ b/block/export/meson.build
> @@ -5,3 +5,8 @@ if have_vhost_user_blk_server
> endif
>
> blockdev_ss.add(when: fuse, if_true: files('fuse.c'))
> +
> +if have_vduse_blk_export
> + blockdev_ss.add(files('vduse-blk.c'))
> + blockdev_ss.add(libvduse)
> +endif
> diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
> new file mode 100644
> index 0000000000..5a8d289685
> --- /dev/null
> +++ b/block/export/vduse-blk.c
> @@ -0,0 +1,427 @@
> +/*
> + * Export QEMU block device via VDUSE
> + *
> + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> + * Portions of codes and concepts borrowed from vhost-user-blk-server.c, so:
> + * Copyright (c) 2020 Red Hat, Inc.
> + *
> + * Author:
> + * Xie Yongji <xieyongji@bytedance.com>
> + * Coiby Xu <coiby.xu@gmail.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later. See the COPYING file in the top-level directory.
> + */
> +
> +#include <sys/eventfd.h>
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "sysemu/block-backend.h"
> +#include "block/export.h"
> +#include "qemu/error-report.h"
> +#include "util/block-helpers.h"
> +#include "subprojects/libvduse/libvduse.h"
> +
> +#include "standard-headers/linux/virtio_ring.h"
> +#include "standard-headers/linux/virtio_blk.h"
> +
> +#define VIRTIO_BLK_SECTOR_BITS 9
> +#define VIRTIO_BLK_SECTOR_SIZE (1ULL << VIRTIO_BLK_SECTOR_BITS)
> +
> +#define VDUSE_DEFAULT_NUM_QUEUE 1
> +#define VDUSE_DEFAULT_QUEUE_SIZE 128
QEMU's virtio-blk emulation has increased this limit to 256 for better
performance with large block size I/O patterns. I think it would be okay
to increase it here too.
> +
> +typedef struct VduseBlkExport {
> + BlockExport export;
> + VduseDev *dev;
> + uint16_t num_queues;
> + uint32_t blk_size;
> + bool writable;
> +} VduseBlkExport;
> +
> +struct virtio_blk_inhdr {
> + unsigned char status;
> +};
> +
> +typedef struct VduseBlkReq {
> + VduseVirtqElement elem;
> + int64_t sector_num;
> + size_t in_len;
> + struct virtio_blk_inhdr *in;
> + struct virtio_blk_outhdr out;
> + VduseVirtq *vq;
> +} VduseBlkReq;
> +
> +static void vduse_blk_req_complete(VduseBlkReq *req)
> +{
> + vduse_queue_push(req->vq, &req->elem, req->in_len);
> + vduse_queue_notify(req->vq);
> +
> + free(req);
> +}
> +
> +static bool vduse_blk_sect_range_ok(VduseBlkExport *vblk_exp,
> + uint64_t sector, size_t size)
> +{
> + uint64_t nb_sectors;
> + uint64_t total_sectors;
> +
> + if (size % VIRTIO_BLK_SECTOR_SIZE) {
> + return false;
> + }
> +
> + nb_sectors = size >> VIRTIO_BLK_SECTOR_BITS;
> +
> + QEMU_BUILD_BUG_ON(BDRV_SECTOR_SIZE != VIRTIO_BLK_SECTOR_SIZE);
> + if (nb_sectors > BDRV_REQUEST_MAX_SECTORS) {
> + return false;
> + }
> + if ((sector << VIRTIO_BLK_SECTOR_BITS) % vblk_exp->blk_size) {
> + return false;
> + }
> + blk_get_geometry(vblk_exp->export.blk, &total_sectors);
> + if (sector > total_sectors || nb_sectors > total_sectors - sector) {
> + return false;
> + }
> + return true;
> +}
> +
> +static void coroutine_fn vduse_blk_virtio_process_req(void *opaque)
> +{
> + VduseBlkReq *req = opaque;
> + VduseVirtq *vq = req->vq;
> + VduseDev *dev = vduse_queue_get_dev(vq);
> + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
> + BlockBackend *blk = vblk_exp->export.blk;
> + VduseVirtqElement *elem = &req->elem;
> + struct iovec *in_iov = elem->in_sg;
> + struct iovec *out_iov = elem->out_sg;
> + unsigned in_num = elem->in_num;
> + unsigned out_num = elem->out_num;
> + uint32_t type;
> +
> + if (elem->out_num < 1 || elem->in_num < 1) {
> + error_report("virtio-blk request missing headers");
> + goto err;
> + }
> +
> + if (unlikely(iov_to_buf(out_iov, out_num, 0, &req->out,
> + sizeof(req->out)) != sizeof(req->out))) {
> + error_report("virtio-blk request outhdr too short");
> + goto err;
> + }
> +
> + iov_discard_front(&out_iov, &out_num, sizeof(req->out));
> +
> + if (in_iov[in_num - 1].iov_len < sizeof(struct virtio_blk_inhdr)) {
> + error_report("virtio-blk request inhdr too short");
> + goto err;
> + }
> +
> + /* We always touch the last byte, so just see how big in_iov is. */
> + req->in_len = iov_size(in_iov, in_num);
> + req->in = (void *)in_iov[in_num - 1].iov_base
> + + in_iov[in_num - 1].iov_len
> + - sizeof(struct virtio_blk_inhdr);
> + iov_discard_back(in_iov, &in_num, sizeof(struct virtio_blk_inhdr));
> +
> + type = le32_to_cpu(req->out.type);
> + switch (type & ~VIRTIO_BLK_T_BARRIER) {
> + case VIRTIO_BLK_T_IN:
> + case VIRTIO_BLK_T_OUT: {
> + QEMUIOVector qiov;
> + int64_t offset;
> + ssize_t ret = 0;
> + bool is_write = type & VIRTIO_BLK_T_OUT;
> + req->sector_num = le64_to_cpu(req->out.sector);
> +
> + if (is_write && !vblk_exp->writable) {
> + req->in->status = VIRTIO_BLK_S_IOERR;
> + break;
> + }
> +
> + if (is_write) {
> + qemu_iovec_init_external(&qiov, out_iov, out_num);
> + } else {
> + qemu_iovec_init_external(&qiov, in_iov, in_num);
> + }
> +
> + if (unlikely(!vduse_blk_sect_range_ok(vblk_exp,
> + req->sector_num,
> + qiov.size))) {
> + req->in->status = VIRTIO_BLK_S_IOERR;
> + break;
> + }
> +
> + offset = req->sector_num << VIRTIO_BLK_SECTOR_BITS;
> +
> + if (is_write) {
> + ret = blk_co_pwritev(blk, offset, qiov.size, &qiov, 0);
> + } else {
> + ret = blk_co_preadv(blk, offset, qiov.size, &qiov, 0);
> + }
> + if (ret >= 0) {
> + req->in->status = VIRTIO_BLK_S_OK;
> + } else {
> + req->in->status = VIRTIO_BLK_S_IOERR;
> + }
> + break;
> + }
> + case VIRTIO_BLK_T_FLUSH:
> + if (blk_co_flush(blk) == 0) {
> + req->in->status = VIRTIO_BLK_S_OK;
> + } else {
> + req->in->status = VIRTIO_BLK_S_IOERR;
> + }
> + break;
> + case VIRTIO_BLK_T_GET_ID: {
> + size_t size = MIN(iov_size(&elem->in_sg[0], in_num),
> + VIRTIO_BLK_ID_BYTES);
> + snprintf(elem->in_sg[0].iov_base, size, "%s", vblk_exp->export.id);
> + req->in->status = VIRTIO_BLK_S_OK;
> + break;
> + }
> + default:
> + req->in->status = VIRTIO_BLK_S_UNSUPP;
> + break;
> + }
> +
> + vduse_blk_req_complete(req);
> + return;
> +
> +err:
> + free(req);
> +}
> +
> +static void vduse_blk_vq_handler(VduseDev *dev, VduseVirtq *vq)
> +{
> + while (1) {
> + VduseBlkReq *req;
> +
> + req = vduse_queue_pop(vq, sizeof(VduseBlkReq));
> + if (!req) {
> + break;
> + }
> + req->vq = vq;
> +
> + Coroutine *co =
> + qemu_coroutine_create(vduse_blk_virtio_process_req, req);
> + qemu_coroutine_enter(co);
> + }
> +}
> +
> +static void on_vduse_vq_kick(void *opaque)
> +{
> + VduseVirtq *vq = opaque;
> + VduseDev *dev = vduse_queue_get_dev(vq);
> + int fd = vduse_queue_get_fd(vq);
> + eventfd_t kick_data;
> +
> + if (eventfd_read(fd, &kick_data) == -1) {
> + error_report("failed to read data from eventfd");
> + return;
> + }
> +
> + vduse_blk_vq_handler(dev, vq);
> +}
> +
> +static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq)
> +{
> + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
> +
> + aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
> + true, on_vduse_vq_kick, NULL, NULL, NULL, vq);
> +}
> +
> +static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq)
> +{
> + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
> +
> + aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
> + true, NULL, NULL, NULL, NULL, NULL);
> +}
> +
> +static const VduseOps vduse_blk_ops = {
> + .enable_queue = vduse_blk_enable_queue,
> + .disable_queue = vduse_blk_disable_queue,
> +};
> +
> +static void on_vduse_dev_kick(void *opaque)
> +{
> + VduseDev *dev = opaque;
> +
> + vduse_dev_handler(dev);
> +}
> +
> +static void blk_aio_attached(AioContext *ctx, void *opaque)
> +{
> + VduseBlkExport *vblk_exp = opaque;
> + int i;
> +
> + vblk_exp->export.ctx = ctx;
> +
> + aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev),
> + true, on_vduse_dev_kick, NULL, NULL, NULL,
> + vblk_exp->dev);
> +
> + for (i = 0; i < vblk_exp->num_queues; i++) {
> + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
> + int fd = vduse_queue_get_fd(vq);
> +
> + if (fd < 0) {
> + continue;
> + }
> + aio_set_fd_handler(vblk_exp->export.ctx, fd, true,
> + on_vduse_vq_kick, NULL, NULL, NULL, vq);
> + }
> +}
> +
> +static void blk_aio_detach(void *opaque)
> +{
> + VduseBlkExport *vblk_exp = opaque;
> + int i;
> +
> + for (i = 0; i < vblk_exp->num_queues; i++) {
> + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
> + int fd = vduse_queue_get_fd(vq);
> +
> + if (fd < 0) {
> + continue;
> + }
> + aio_set_fd_handler(vblk_exp->export.ctx, fd,
> + true, NULL, NULL, NULL, NULL, NULL);
> + }
> + aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev),
> + true, NULL, NULL, NULL, NULL, NULL);
> + vblk_exp->export.ctx = NULL;
> +}
> +
> +static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> + Error **errp)
> +{
> + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> + BlockExportOptionsVduseBlk *vblk_opts = &opts->u.vduse_blk;
> + uint64_t logical_block_size = VIRTIO_BLK_SECTOR_SIZE;
> + uint16_t num_queues = VDUSE_DEFAULT_NUM_QUEUE;
> + uint16_t queue_size = VDUSE_DEFAULT_QUEUE_SIZE;
> + Error *local_err = NULL;
> + struct virtio_blk_config config;
Please zero-initialize this with "= { 0 }" because this struct can grow
when Linux adds virtio-blk features and our function may not have code
to fill in the new fields.
> + uint64_t features;
> + int i;
> +
> + if (vblk_opts->has_num_queues) {
> + num_queues = vblk_opts->num_queues;
> + if (num_queues == 0) {
> + error_setg(errp, "num-queues must be greater than 0");
> + return -EINVAL;
> + }
> + }
> +
> + if (vblk_opts->has_queue_size) {
> + queue_size = vblk_opts->queue_size;
> + if (queue_size == 0) {
> + error_setg(errp, "queue-size must be greater than 0");
> + return -EINVAL;
> + }
> + }
> +
> + if (vblk_opts->has_logical_block_size) {
> + logical_block_size = vblk_opts->logical_block_size;
> + check_block_size(exp->id, "logical-block-size", logical_block_size,
> + &local_err);
> + if (local_err) {
> + error_propagate(errp, local_err);
> + return -EINVAL;
> + }
> + }
> + blk_set_guest_block_size(exp->blk, logical_block_size);
> +
> + vblk_exp->blk_size = logical_block_size;
> + vblk_exp->writable = opts->writable;
> + vblk_exp->num_queues = num_queues;
> +
> + config.capacity =
> + cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS);
> + config.seg_max = cpu_to_le32(queue_size - 2);
What is queue_size is 1 or 2?
> + config.size_max = cpu_to_le32(0);
> + config.min_io_size = cpu_to_le16(1);
> + config.opt_io_size = cpu_to_le32(1);
> + config.num_queues = cpu_to_le16(num_queues);
> + config.blk_size = cpu_to_le32(logical_block_size);
> +
> + features = (1ULL << VIRTIO_F_IOMMU_PLATFORM) |
> + (1ULL << VIRTIO_F_VERSION_1) |
> + (1ULL << VIRTIO_RING_F_EVENT_IDX) |
> + (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) |
> + (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
> + (1ULL << VIRTIO_BLK_F_SIZE_MAX) |
> + (1ULL << VIRTIO_BLK_F_SEG_MAX) |
> + (1ULL << VIRTIO_BLK_F_TOPOLOGY) |
> + (1ULL << VIRTIO_BLK_F_BLK_SIZE);
> +
> + if (num_queues > 1) {
> + features |= 1ULL << VIRTIO_BLK_F_MQ;
> + }
> + if (!vblk_exp->writable) {
> + features |= 1ULL << VIRTIO_BLK_F_RO;
> + }
> +
> + vblk_exp->dev = vduse_dev_create(exp->id, VIRTIO_ID_BLOCK, 0,
> + features, num_queues,
> + sizeof(struct virtio_blk_config),
> + (char *)&config, &vduse_blk_ops,
> + vblk_exp);
> + if (!vblk_exp->dev) {
> + error_setg(errp, "failed to create vduse device");
> + return -ENOMEM;
> + }
> +
> + for (i = 0; i < num_queues; i++) {
> + vduse_dev_setup_queue(vblk_exp->dev, i, queue_size);
> + }
> +
> + aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev), true,
> + on_vduse_dev_kick, NULL, NULL, NULL, vblk_exp->dev);
> +
> + blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
> + vblk_exp);
> +
> + return 0;
> +}
> +
> +static void vduse_blk_exp_delete(BlockExport *exp)
> +{
> + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> +
> + vduse_dev_destroy(vblk_exp->dev);
> +}
> +
> +static void vduse_blk_exp_request_shutdown(BlockExport *exp)
> +{
> + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> + int i;
> +
> + blk_remove_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
> + vblk_exp);
> +
> + for (i = 0; i < vblk_exp->num_queues; i++) {
> + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
> + int fd = vduse_queue_get_fd(vq);
> +
> + if (fd < 0) {
> + continue;
> + }
> + aio_set_fd_handler(exp->ctx, fd, true, NULL, NULL, NULL, NULL, NULL);
> + }
> + aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev),
> + true, NULL, NULL, NULL, NULL, NULL);
Call blk_aio_detach() instead of duplicating this code?
> +}
> +
> +const BlockExportDriver blk_exp_vduse_blk = {
> + .type = BLOCK_EXPORT_TYPE_VDUSE_BLK,
> + .instance_size = sizeof(VduseBlkExport),
> + .create = vduse_blk_exp_create,
> + .delete = vduse_blk_exp_delete,
> + .request_shutdown = vduse_blk_exp_request_shutdown,
> +};
> diff --git a/block/export/vduse-blk.h b/block/export/vduse-blk.h
> new file mode 100644
> index 0000000000..c4eeb1b70e
> --- /dev/null
> +++ b/block/export/vduse-blk.h
> @@ -0,0 +1,20 @@
> +/*
> + * Export QEMU block device via VDUSE
> + *
> + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> + *
> + * Author:
> + * Xie Yongji <xieyongji@bytedance.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later. See the COPYING file in the top-level directory.
> + */
> +
> +#ifndef VDUSE_BLK_H
> +#define VDUSE_BLK_H
> +
> +#include "block/export.h"
> +
> +extern const BlockExportDriver blk_exp_vduse_blk;
> +
> +#endif /* VDUSE_BLK_H */
> diff --git a/meson.build b/meson.build
> index 864fb50ade..472e3947c6 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1320,6 +1320,17 @@ if have_libvduse
> libvduse = libvduse_proj.get_variable('libvduse_dep')
> endif
>
> +have_vduse_blk_export = (have_libvduse and targetos == 'linux')
> +if get_option('vduse_blk_export').enabled()
> + if targetos != 'linux'
> + error('vduse_blk_export requires linux')
> + elif have_libvduse
> + error('vduse_blk_export requires libvduse support')
> + endif
> +elif get_option('vduse_blk_export').disabled()
> + have_vduse_blk_export = false
> +endif
> +
> # libbpf
> libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config')
> if libbpf.found() and not cc.links('''
> @@ -1514,6 +1525,7 @@ config_host_data.set('CONFIG_SNAPPY', snappy.found())
> config_host_data.set('CONFIG_USB_LIBUSB', libusb.found())
> config_host_data.set('CONFIG_VDE', vde.found())
> config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', have_vhost_user_blk_server)
> +config_host_data.set('CONFIG_VDUSE_BLK_EXPORT', have_vduse_blk_export)
> config_host_data.set('CONFIG_VNC', vnc.found())
> config_host_data.set('CONFIG_VNC_JPEG', jpeg.found())
> config_host_data.set('CONFIG_VNC_PNG', png.found())
> @@ -3407,6 +3419,7 @@ if have_block
> summary_info += {'qed support': config_host.has_key('CONFIG_QED')}
> summary_info += {'parallels support': config_host.has_key('CONFIG_PARALLELS')}
> summary_info += {'FUSE exports': fuse}
> + summary_info += {'VDUSE block exports': have_vduse_blk_export}
> endif
> summary(summary_info, bool_yn: true, section: 'Block layer support')
>
> diff --git a/meson_options.txt b/meson_options.txt
> index 16790d1814..be1682c4d2 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -197,6 +197,8 @@ option('virtiofsd', type: 'feature', value: 'auto',
> description: 'build virtiofs daemon (virtiofsd)')
> option('libvduse', type: 'feature', value: 'auto',
> description: 'build VDUSE Library')
> +option('vduse_blk_export', type: 'feature', value: 'auto',
> + description: 'VDUSE block export support')
>
> option('capstone', type: 'combo', value: 'auto',
> choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
> diff --git a/qapi/block-export.json b/qapi/block-export.json
> index f9ce79a974..f88e90baab 100644
> --- a/qapi/block-export.json
> +++ b/qapi/block-export.json
> @@ -170,6 +170,22 @@
> '*allow-other': 'FuseExportAllowOther' },
> 'if': 'CONFIG_FUSE' }
>
> +##
> +# @BlockExportOptionsVduseBlk:
> +#
> +# A vduse-blk block export.
> +#
> +# @num-queues: the number of virtqueues. Defaults to 1.
> +# @queue-size: the size of virtqueue. Defaults to 128.
> +# @logical-block-size: Logical block size in bytes. Defaults to 512 bytes.
> +#
> +# Since: 7.0
> +##
> +{ 'struct': 'BlockExportOptionsVduseBlk',
> + 'data': { '*num-queues': 'uint16',
> + '*queue-size': 'uint16',
> + '*logical-block-size': 'size'} }
> +
> ##
> # @NbdServerAddOptions:
> #
> @@ -273,13 +289,15 @@
> # @nbd: NBD export
> # @vhost-user-blk: vhost-user-blk export (since 5.2)
> # @fuse: FUSE export (since: 6.0)
> +# @vduse-blk: vduse-blk export (since 7.0)
> #
> # Since: 4.2
> ##
> { 'enum': 'BlockExportType',
> 'data': [ 'nbd',
> { 'name': 'vhost-user-blk', 'if': 'CONFIG_VHOST_USER_BLK_SERVER' },
> - { 'name': 'fuse', 'if': 'CONFIG_FUSE' } ] }
> + { 'name': 'fuse', 'if': 'CONFIG_FUSE' },
> + { 'name': 'vduse-blk', 'if': 'CONFIG_VDUSE_BLK_EXPORT' } ] }
>
> ##
> # @BlockExportOptions:
> @@ -323,7 +341,9 @@
> 'vhost-user-blk': { 'type': 'BlockExportOptionsVhostUserBlk',
> 'if': 'CONFIG_VHOST_USER_BLK_SERVER' },
> 'fuse': { 'type': 'BlockExportOptionsFuse',
> - 'if': 'CONFIG_FUSE' }
> + 'if': 'CONFIG_FUSE' },
> + 'vduse-blk': { 'type': 'BlockExportOptionsVduseBlk',
> + 'if': 'CONFIG_VDUSE_BLK_EXPORT' }
> } }
>
> ##
> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> index af5c75d758..615fc17ec3 100644
> --- a/scripts/meson-buildoptions.sh
> +++ b/scripts/meson-buildoptions.sh
> @@ -86,6 +86,8 @@ meson_options_help() {
> printf "%s\n" ' u2f U2F emulation support'
> printf "%s\n" ' usb-redir libusbredir support'
> printf "%s\n" ' vde vde network backend support'
> + printf "%s\n" ' vduse-blk-export'
> + printf "%s\n" ' VDUSE block export support'
> printf "%s\n" ' vhost-user-blk-server'
> printf "%s\n" ' build vhost-user-blk server'
> printf "%s\n" ' virglrenderer virgl rendering support'
> @@ -254,6 +256,8 @@ _meson_option_parse() {
> --disable-usb-redir) printf "%s" -Dusb_redir=disabled ;;
> --enable-vde) printf "%s" -Dvde=enabled ;;
> --disable-vde) printf "%s" -Dvde=disabled ;;
> + --enable-vduse-blk-export) printf "%s" -Dvduse_blk_export=enabled ;;
> + --disable-vduse-blk-export) printf "%s" -Dvduse_blk_export=disabled ;;
> --enable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=enabled ;;
> --disable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=disabled ;;
> --enable-virglrenderer) printf "%s" -Dvirglrenderer=enabled ;;
> --
> 2.20.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 4/5] vduse-blk: Add vduse-blk resize support
2022-01-25 13:17 ` [PATCH 4/5] vduse-blk: Add vduse-blk resize support Xie Yongji
@ 2022-02-07 14:18 ` Stefan Hajnoczi
2022-02-08 7:12 ` Yongji Xie
0 siblings, 1 reply; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-07 14:18 UTC (permalink / raw)
To: Xie Yongji
Cc: kwolf, qemu-block, mst, jasowang, qemu-devel, mreitz, mlureau,
sgarzare
[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]
On Tue, Jan 25, 2022 at 09:17:59PM +0800, Xie Yongji wrote:
> To support block resize, this uses vduse_dev_update_config()
> to update the capacity field in configuration space and inject
> config interrupt on the block resize callback.
>
> Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> ---
> block/export/vduse-blk.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
> index 5a8d289685..83845e9a9a 100644
> --- a/block/export/vduse-blk.c
> +++ b/block/export/vduse-blk.c
> @@ -297,6 +297,23 @@ static void blk_aio_detach(void *opaque)
> vblk_exp->export.ctx = NULL;
> }
>
> +static void vduse_blk_resize(void *opaque)
> +{
> + BlockExport *exp = opaque;
> + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> + struct virtio_blk_config config;
> +
> + config.capacity =
> + cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS);
> + vduse_dev_update_config(vblk_exp->dev, sizeof(config.capacity),
> + offsetof(struct virtio_blk_config, capacity),
> + (char *)&config.capacity);
> +}
> +
> +static const BlockDevOps vduse_block_ops = {
> + .resize_cb = vduse_blk_resize,
> +};
> +
> static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> Error **errp)
> {
> @@ -387,6 +404,8 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
> vblk_exp);
>
> + blk_set_dev_ops(exp->blk, &vduse_block_ops, exp);
Detach is missing, so BlockBackend->dev_ops will become stale after the
export is deleted. Please add code to detach when the export is deleted.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] libvduse: Add support for reconnecting
2022-01-25 13:18 ` [PATCH 5/5] libvduse: Add support for reconnecting Xie Yongji
@ 2022-02-07 14:39 ` Stefan Hajnoczi
2022-02-08 7:35 ` Yongji Xie
0 siblings, 1 reply; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-07 14:39 UTC (permalink / raw)
To: Xie Yongji
Cc: kwolf, qemu-block, mst, jasowang, qemu-devel, mreitz, mlureau,
sgarzare
[-- Attachment #1: Type: text/plain, Size: 15908 bytes --]
On Tue, Jan 25, 2022 at 09:18:00PM +0800, Xie Yongji wrote:
> To support reconnecting after restart or crash, VDUSE backend
> might need to resubmit inflight I/Os. This stores the metadata
> such as the index of inflight I/O's descriptors to a shm file so
> that VDUSE backend can restore them during reconnecting.
>
> Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> ---
> block/export/vduse-blk.c | 4 +-
> subprojects/libvduse/libvduse.c | 254 +++++++++++++++++++++++++++++++-
> subprojects/libvduse/libvduse.h | 4 +-
> 3 files changed, 254 insertions(+), 8 deletions(-)
>
> diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
> index 83845e9a9a..bc14fd798b 100644
> --- a/block/export/vduse-blk.c
> +++ b/block/export/vduse-blk.c
> @@ -232,6 +232,8 @@ static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq)
>
> aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
> true, on_vduse_vq_kick, NULL, NULL, NULL, vq);
> + /* Make sure we don't miss any kick afer reconnecting */
> + eventfd_write(vduse_queue_get_fd(vq), 1);
> }
>
> static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq)
> @@ -388,7 +390,7 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> features, num_queues,
> sizeof(struct virtio_blk_config),
> (char *)&config, &vduse_blk_ops,
> - vblk_exp);
> + g_get_tmp_dir(), vblk_exp);
> if (!vblk_exp->dev) {
> error_setg(errp, "failed to create vduse device");
> return -ENOMEM;
> diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
> index 7671864bca..ce2f6c7949 100644
> --- a/subprojects/libvduse/libvduse.c
> +++ b/subprojects/libvduse/libvduse.c
> @@ -41,6 +41,8 @@
> #define VDUSE_VQ_ALIGN 4096
> #define MAX_IOVA_REGIONS 256
>
> +#define LOG_ALIGNMENT 64
> +
> /* Round number down to multiple */
> #define ALIGN_DOWN(n, m) ((n) / (m) * (m))
>
> @@ -51,6 +53,31 @@
> #define unlikely(x) __builtin_expect(!!(x), 0)
> #endif
>
> +typedef struct VduseDescStateSplit {
> + uint8_t inflight;
> + uint8_t padding[5];
> + uint16_t next;
> + uint64_t counter;
> +} VduseDescStateSplit;
> +
> +typedef struct VduseVirtqLogInflight {
> + uint64_t features;
> + uint16_t version;
> + uint16_t desc_num;
> + uint16_t last_batch_head;
> + uint16_t used_idx;
> + VduseDescStateSplit desc[];
> +} VduseVirtqLogInflight;
> +
> +typedef struct VduseVirtqLog {
> + VduseVirtqLogInflight inflight;
> +} VduseVirtqLog;
> +
> +typedef struct VduseVirtqInflightDesc {
> + uint16_t index;
> + uint64_t counter;
> +} VduseVirtqInflightDesc;
> +
> typedef struct VduseRing {
> unsigned int num;
> uint64_t desc_addr;
> @@ -73,6 +100,10 @@ struct VduseVirtq {
> bool ready;
> int fd;
> VduseDev *dev;
> + VduseVirtqInflightDesc *resubmit_list;
> + uint16_t resubmit_num;
> + uint64_t counter;
> + VduseVirtqLog *log;
> };
>
> typedef struct VduseIovaRegion {
> @@ -96,8 +127,67 @@ struct VduseDev {
> int fd;
> int ctrl_fd;
> void *priv;
> + char *shm_log_dir;
> + void *log;
> + bool reconnect;
> };
>
> +static inline size_t vduse_vq_log_size(uint16_t queue_size)
> +{
> + return ALIGN_UP(sizeof(VduseDescStateSplit) * queue_size +
> + sizeof(VduseVirtqLogInflight), LOG_ALIGNMENT);
> +}
> +
> +static void *vduse_log_get(const char *dir, const char *name, size_t size)
> +{
> + void *ptr = MAP_FAILED;
> + char *path;
> + int fd;
> +
> + path = (char *)malloc(strlen(dir) + strlen(name) +
> + strlen("/vduse-log-") + 1);
> + if (!path) {
> + return ptr;
> + }
> + sprintf(path, "%s/vduse-log-%s", dir, name);
Please use g_strdup_printf() and g_autofree in QEMU code. In libvduse
code it's okay to use malloc(3), but regular QEMU code should use glib.
> +
> + fd = open(path, O_RDWR | O_CREAT, 0600);
> + if (fd == -1) {
> + goto out;
> + }
> +
> + if (ftruncate(fd, size) == -1) {
> + goto out;
> + }
> +
> + ptr = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
> + if (ptr == MAP_FAILED) {
> + goto out;
> + }
> +out:
> + if (fd > 0) {
> + close(fd);
> + }
> + free(path);
> +
> + return ptr;
> +}
> +
> +static void vduse_log_destroy(const char *dir, const char *name)
> +{
> + char *path;
> +
> + path = (char *)malloc(strlen(dir) + strlen(name) +
> + strlen("/vduse-log-") + 1);
> + if (!path) {
> + return;
> + }
> + sprintf(path, "%s/vduse-log-%s", dir, name);
> +
> + unlink(path);
> + free(path);
> +}
> +
> static inline bool has_feature(uint64_t features, unsigned int fbit)
> {
> assert(fbit < 64);
> @@ -139,6 +229,98 @@ static int vduse_inject_irq(VduseDev *dev, int index)
> return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index);
> }
>
> +static int inflight_desc_compare(const void *a, const void *b)
> +{
> + VduseVirtqInflightDesc *desc0 = (VduseVirtqInflightDesc *)a,
> + *desc1 = (VduseVirtqInflightDesc *)b;
> +
> + if (desc1->counter > desc0->counter &&
> + (desc1->counter - desc0->counter) < VIRTQUEUE_MAX_SIZE * 2) {
> + return 1;
> + }
> +
> + return -1;
> +}
> +
> +static int vduse_queue_check_inflights(VduseVirtq *vq)
> +{
> + int i = 0;
> + VduseDev *dev = vq->dev;
> +
> + vq->used_idx = vq->vring.used->idx;
Is this reading struct vring_used->idx without le16toh()?
> + vq->resubmit_num = 0;
> + vq->resubmit_list = NULL;
> + vq->counter = 0;
> +
> + if (unlikely(vq->log->inflight.used_idx != vq->used_idx)) {
> + vq->log->inflight.desc[vq->log->inflight.last_batch_head].inflight = 0;
I suggest validating vq->log->inflight fields before using them.
last_batch_head must be less than the virtqueue size. Although the log
file is somewhat trusted, there may still be ways to corrupt it or
confuse the new process that loads it.
> +
> + barrier();
> +
> + vq->log->inflight.used_idx = vq->used_idx;
> + }
> +
> + for (i = 0; i < vq->log->inflight.desc_num; i++) {
> + if (vq->log->inflight.desc[i].inflight == 1) {
> + vq->inuse++;
> + }
> + }
> +
> + vq->shadow_avail_idx = vq->last_avail_idx = vq->inuse + vq->used_idx;
> +
> + if (vq->inuse) {
> + vq->resubmit_list = calloc(vq->inuse, sizeof(VduseVirtqInflightDesc));
> + if (!vq->resubmit_list) {
> + return -1;
> + }
> +
> + for (i = 0; i < vq->log->inflight.desc_num; i++) {
> + if (vq->log->inflight.desc[i].inflight) {
> + vq->resubmit_list[vq->resubmit_num].index = i;
> + vq->resubmit_list[vq->resubmit_num].counter =
> + vq->log->inflight.desc[i].counter;
> + vq->resubmit_num++;
> + }
> + }
> +
> + if (vq->resubmit_num > 1) {
> + qsort(vq->resubmit_list, vq->resubmit_num,
> + sizeof(VduseVirtqInflightDesc), inflight_desc_compare);
> + }
> + vq->counter = vq->resubmit_list[0].counter + 1;
> + }
> +
> + vduse_inject_irq(dev, vq->index);
> +
> + return 0;
> +}
> +
> +static int vduse_queue_inflight_get(VduseVirtq *vq, int desc_idx)
> +{
> + vq->log->inflight.desc[desc_idx].counter = vq->counter++;
> + vq->log->inflight.desc[desc_idx].inflight = 1;
> +
> + return 0;
> +}
> +
> +static int vduse_queue_inflight_pre_put(VduseVirtq *vq, int desc_idx)
> +{
> + vq->log->inflight.last_batch_head = desc_idx;
> +
> + return 0;
> +}
> +
> +static int vduse_queue_inflight_post_put(VduseVirtq *vq, int desc_idx)
> +{
> + vq->log->inflight.desc[desc_idx].inflight = 0;
> +
> + barrier();
> +
> + vq->log->inflight.used_idx = vq->used_idx;
> +
> + return 0;
> +}
> +
> static void vduse_iova_remove_region(VduseDev *dev, uint64_t start,
> uint64_t last)
> {
> @@ -578,11 +760,24 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
> unsigned int head;
> VduseVirtqElement *elem;
> VduseDev *dev = vq->dev;
> + int i;
>
> if (unlikely(!vq->vring.avail)) {
> return NULL;
> }
>
> + if (unlikely(vq->resubmit_list && vq->resubmit_num > 0)) {
> + i = (--vq->resubmit_num);
> + elem = vduse_queue_map_desc(vq, vq->resubmit_list[i].index, sz);
> +
> + if (!vq->resubmit_num) {
> + free(vq->resubmit_list);
> + vq->resubmit_list = NULL;
> + }
resubmit_list is only freed when vduse_queue_pop() is called often
enough to empty the list. Please free the list when the vduse instance
is destroyed too, just in case vduse_queue_pop() wasn't called often
enough to free it.
> +
> + return elem;
> + }
> +
> if (vduse_queue_empty(vq)) {
> return NULL;
> }
> @@ -610,6 +805,8 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
>
> vq->inuse++;
>
> + vduse_queue_inflight_get(vq, head);
> +
> return elem;
> }
>
> @@ -667,7 +864,9 @@ void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
> unsigned int len)
> {
> vduse_queue_fill(vq, elem, len, 0);
> + vduse_queue_inflight_pre_put(vq, elem->index);
> vduse_queue_flush(vq, 1);
> + vduse_queue_inflight_post_put(vq, elem->index);
> }
>
> static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr,
> @@ -740,12 +939,11 @@ static void vduse_queue_enable(VduseVirtq *vq)
> }
>
> vq->fd = fd;
> - vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index;
> - vq->inuse = 0;
> - vq->used_idx = 0;
> vq->signalled_used_valid = false;
> vq->ready = true;
>
> + vduse_queue_check_inflights(vq);
> +
> dev->ops->enable_queue(dev, vq);
> }
>
> @@ -903,13 +1101,18 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size)
> return -errno;
> }
>
> + if (dev->reconnect) {
> + vduse_queue_enable(vq);
> + }
> +
> return 0;
> }
>
> VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> uint32_t vendor_id, uint64_t features,
> uint16_t num_queues, uint32_t config_size,
> - char *config, const VduseOps *ops, void *priv)
> + char *config, const VduseOps *ops,
> + const char *shm_log_dir, void *priv)
> {
> VduseDev *dev;
> int i, ret, ctrl_fd, fd = -1;
> @@ -918,6 +1121,8 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> VduseVirtq *vqs = NULL;
> struct vduse_dev_config *dev_config = NULL;
> size_t size = offsetof(struct vduse_dev_config, config);
> + size_t log_size = num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE);
> + void *log = NULL;
>
> if (!name || strlen(name) > VDUSE_NAME_MAX || !config ||
> !config_size || !ops || !ops->enable_queue || !ops->disable_queue) {
> @@ -932,6 +1137,15 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> }
> memset(dev, 0, sizeof(VduseDev));
>
> + if (shm_log_dir) {
> + dev->log = log = vduse_log_get(shm_log_dir, name, log_size);
> + if (!log) {
> + fprintf(stderr, "Failed to get vduse log\n");
> + goto err_ctrl;
> + }
> + dev->shm_log_dir = strdup(shm_log_dir);
> + }
> +
> ctrl_fd = open("/dev/vduse/control", O_RDWR);
> if (ctrl_fd < 0) {
> fprintf(stderr, "Failed to open /dev/vduse/control: %s\n",
> @@ -964,7 +1178,11 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
>
> ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config);
> free(dev_config);
> - if (ret < 0) {
> + if (!ret && log) {
> + memset(log, 0, log_size);
> + } else if (errno == EEXIST && log) {
> + dev->reconnect = true;
> + } else {
> fprintf(stderr, "Failed to create vduse dev %s: %s\n",
> name, strerror(errno));
> goto err_dev;
> @@ -978,6 +1196,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> goto err;
> }
>
> + if (dev->reconnect &&
> + ioctl(fd, VDUSE_DEV_GET_FEATURES, &dev->features)) {
> + fprintf(stderr, "Failed to get features: %s\n", strerror(errno));
> + goto err;
> + }
> +
> vqs = calloc(sizeof(VduseVirtq), num_queues);
> if (!vqs) {
> fprintf(stderr, "Failed to allocate virtqueues\n");
> @@ -988,6 +1212,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> vqs[i].index = i;
> vqs[i].dev = dev;
> vqs[i].fd = -1;
> + if (log) {
> + vqs[i].log = log;
> + vqs[i].log->inflight.desc_num = VIRTQUEUE_MAX_SIZE;
> + log = (void *)((char *)log +
> + vduse_vq_log_size(VIRTQUEUE_MAX_SIZE));
The size of the log needs to be verified. The file is mmapped but
there's no guarantee that the size matches num_queues *
vduse_vq_log_size(VIRTQUEUE_MAX_SIZE).
> + }
> }
>
> dev->vqs = vqs;
> @@ -1008,16 +1238,28 @@ err_dev:
> close(ctrl_fd);
> err_ctrl:
> free(dev);
> + if (log) {
> + munmap(log, log_size);
> + }
>
> return NULL;
> }
>
> void vduse_dev_destroy(VduseDev *dev)
> {
> + size_t log_size = dev->num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE);
> +
> + if (dev->log) {
> + munmap(dev->log, log_size);
> + }
> free(dev->vqs);
> close(dev->fd);
> dev->fd = -1;
> - ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name);
> + if (!ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name) &&
> + dev->shm_log_dir) {
> + vduse_log_destroy(dev->shm_log_dir, dev->name);
> + }
> + free(dev->shm_log_dir);
> free(dev->name);
> close(dev->ctrl_fd);
> dev->ctrl_fd = -1;
> diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h
> index f6bcb51b5a..a46e71e0c2 100644
> --- a/subprojects/libvduse/libvduse.h
> +++ b/subprojects/libvduse/libvduse.h
> @@ -171,6 +171,7 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size);
> * @config_size: the size of the configuration space
> * @config: the buffer of the configuration space
> * @ops: the operation of VDUSE backend
> + * @shm_log_dir: directory to store the metadata file for reconnect
> * @priv: private pointer
> *
> * Create VDUSE device.
> @@ -180,7 +181,8 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size);
> VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> uint32_t vendor_id, uint64_t features,
> uint16_t num_queues, uint32_t config_size,
> - char *config, const VduseOps *ops, void *priv);
> + char *config, const VduseOps *ops,
> + const char *shm_log_dir, void *priv);
>
> /**
> * vduse_dev_destroy:
> --
> 2.20.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 1/5] linux-headers: Add vduse.h
2022-02-07 13:12 ` Stefan Hajnoczi
@ 2022-02-08 3:13 ` Yongji Xie
0 siblings, 0 replies; 19+ messages in thread
From: Yongji Xie @ 2022-02-08 3:13 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
On Mon, Feb 7, 2022 at 9:12 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Tue, Jan 25, 2022 at 09:17:56PM +0800, Xie Yongji wrote:
> > diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh
> > index fea4d6eb65..4c7846076f 100755
> > --- a/scripts/update-linux-headers.sh
> > +++ b/scripts/update-linux-headers.sh
> > @@ -198,6 +198,7 @@ for i in "$tmpdir"/include/linux/*virtio*.h \
> > "$tmpdir/include/linux/const.h" \
> > "$tmpdir/include/linux/kernel.h" \
> > "$tmpdir/include/linux/vhost_types.h" \
> > + "$tmpdir/include/linux/vduse.h" \
> > "$tmpdir/include/linux/sysinfo.h"; do
> > cp_portable "$i" "$output/include/standard-headers/linux"
>
> VDUSE is only available on Linux hosts so it should go in linux-headers/
> instead of standard-headers/linux/:
>
> # - linux-headers/ for files that are required for compiling for a
> # Linux host. Generally we have these so we can use kernel structs
> # and defines that are more recent than the headers that might be
> # installed on the host system. Usually this script can do simple
> # file copies for these headers.
> #
> # - include/standard-headers/ for files that are used for guest
> # device emulation and are required on all hosts. For instance, we
> # get our definitions of the virtio structures from the Linux
> # kernel headers, but we need those definitions regardless of which
> # host OS we are building for. This script has to be careful to
> # sanitize the headers to remove any use of Linux-specifics such as
> # types like "__u64". This work is done in the cp_portable function.
Got it! Will fix it in v2.
Thanks,
Yongji
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
2022-02-07 14:00 ` Stefan Hajnoczi
@ 2022-02-08 6:42 ` Yongji Xie
2022-02-08 8:02 ` Stefan Hajnoczi
0 siblings, 1 reply; 19+ messages in thread
From: Yongji Xie @ 2022-02-08 6:42 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
On Mon, Feb 7, 2022 at 10:01 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote:
> > VDUSE [1] is a linux framework that makes it possible to implement
> > software-emulated vDPA devices in userspace. This adds a library
> > as a subproject to help implementing VDUSE backends in QEMU.
> >
> > [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
>
> This library assumes that the program is allowed to access the control
> device (/dev/vduse/control). Is that always the case or should the
> library also support access to /dev/vduse/<name> only (maybe even with
> file descriptor passing) so a privileged process can create/destroy
> VDUSE devices?
>
Make sense. I will add two new API to support these two cases.
> I didn't review the vring code in detail.
>
> >
> > Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> > ---
> > meson.build | 15 +
> > meson_options.txt | 2 +
> > scripts/meson-buildoptions.sh | 3 +
> > subprojects/libvduse/include/atomic.h | 1 +
> > subprojects/libvduse/libvduse.c | 1025 +++++++++++++++++++
> > subprojects/libvduse/libvduse.h | 193 ++++
> > subprojects/libvduse/meson.build | 10 +
> > subprojects/libvduse/standard-headers/linux | 1 +
> > 8 files changed, 1250 insertions(+)
> > create mode 120000 subprojects/libvduse/include/atomic.h
> > create mode 100644 subprojects/libvduse/libvduse.c
> > create mode 100644 subprojects/libvduse/libvduse.h
> > create mode 100644 subprojects/libvduse/meson.build
> > create mode 120000 subprojects/libvduse/standard-headers/linux
> >
> > diff --git a/meson.build b/meson.build
> > index 333c61deba..864fb50ade 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -1305,6 +1305,21 @@ if not get_option('fuse_lseek').disabled()
> > endif
> > endif
> >
> > +have_libvduse = (targetos == 'linux')
> > +if get_option('libvduse').enabled()
> > + if targetos != 'linux'
> > + error('libvduse requires linux')
> > + endif
> > +elif get_option('libvduse').disabled()
> > + have_libvduse = false
> > +endif
> > +
> > +libvduse = not_found
> > +if have_libvduse
> > + libvduse_proj = subproject('libvduse')
> > + libvduse = libvduse_proj.get_variable('libvduse_dep')
> > +endif
> > +
> > # libbpf
> > libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config')
> > if libbpf.found() and not cc.links('''
> > diff --git a/meson_options.txt b/meson_options.txt
> > index 921967eddb..16790d1814 100644
> > --- a/meson_options.txt
> > +++ b/meson_options.txt
> > @@ -195,6 +195,8 @@ option('virtfs', type: 'feature', value: 'auto',
> > description: 'virtio-9p support')
> > option('virtiofsd', type: 'feature', value: 'auto',
> > description: 'build virtiofs daemon (virtiofsd)')
> > +option('libvduse', type: 'feature', value: 'auto',
> > + description: 'build VDUSE Library')
> >
> > option('capstone', type: 'combo', value: 'auto',
> > choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
> > diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> > index a4af02c527..af5c75d758 100644
> > --- a/scripts/meson-buildoptions.sh
> > +++ b/scripts/meson-buildoptions.sh
> > @@ -58,6 +58,7 @@ meson_options_help() {
> > printf "%s\n" ' libssh ssh block device support'
> > printf "%s\n" ' libudev Use libudev to enumerate host devices'
> > printf "%s\n" ' libusb libusb support for USB passthrough'
> > + printf "%s\n" ' libvduse build VDUSE Library'
> > printf "%s\n" ' libxml2 libxml2 support for Parallels image format'
> > printf "%s\n" ' linux-aio Linux AIO support'
> > printf "%s\n" ' linux-io-uring Linux io_uring support'
> > @@ -188,6 +189,8 @@ _meson_option_parse() {
> > --disable-libudev) printf "%s" -Dlibudev=disabled ;;
> > --enable-libusb) printf "%s" -Dlibusb=enabled ;;
> > --disable-libusb) printf "%s" -Dlibusb=disabled ;;
> > + --enable-libvduse) printf "%s" -Dlibvduse=enabled ;;
> > + --disable-libvduse) printf "%s" -Dlibvduse=disabled ;;
> > --enable-libxml2) printf "%s" -Dlibxml2=enabled ;;
> > --disable-libxml2) printf "%s" -Dlibxml2=disabled ;;
> > --enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;;
> > diff --git a/subprojects/libvduse/include/atomic.h b/subprojects/libvduse/include/atomic.h
> > new file mode 120000
> > index 0000000000..8c2be64f7b
> > --- /dev/null
> > +++ b/subprojects/libvduse/include/atomic.h
> > @@ -0,0 +1 @@
> > +../../../include/qemu/atomic.h
> > \ No newline at end of file
> > diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
> > new file mode 100644
> > index 0000000000..7671864bca
> > --- /dev/null
> > +++ b/subprojects/libvduse/libvduse.c
> > @@ -0,0 +1,1025 @@
> > +/*
> > + * VDUSE (vDPA Device in Userspace) library
> > + *
> > + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> > + * Portions of codes and concepts borrowed from libvhost-user.c, so:
> > + * Copyright IBM, Corp. 2007
> > + * Copyright (c) 2016 Red Hat, Inc.
> > + *
> > + * Author:
> > + * Xie Yongji <xieyongji@bytedance.com>
> > + * Anthony Liguori <aliguori@us.ibm.com>
> > + * Marc-André Lureau <mlureau@redhat.com>
> > + * Victor Kaplansky <victork@redhat.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or
> > + * later. See the COPYING file in the top-level directory.
> > + */
> > +
> > +#include <stdlib.h>
> > +#include <stdio.h>
> > +#include <stdbool.h>
> > +#include <stddef.h>
> > +#include <errno.h>
> > +#include <string.h>
> > +#include <assert.h>
> > +#include <endian.h>
> > +#include <unistd.h>
> > +#include <limits.h>
> > +#include <fcntl.h>
> > +
> > +#include <sys/ioctl.h>
> > +#include <sys/eventfd.h>
> > +#include <sys/mman.h>
> > +
> > +#include "include/atomic.h"
> > +#include "standard-headers/linux/vhost_types.h"
> > +#include "standard-headers/linux/vduse.h"
> > +#include "libvduse.h"
> > +
> > +#define VIRTQUEUE_MAX_SIZE 1024
> > +#define VDUSE_VQ_ALIGN 4096
> > +#define MAX_IOVA_REGIONS 256
> > +
> > +/* Round number down to multiple */
> > +#define ALIGN_DOWN(n, m) ((n) / (m) * (m))
> > +
> > +/* Round number up to multiple */
> > +#define ALIGN_UP(n, m) ALIGN_DOWN((n) + (m) - 1, (m))
> > +
> > +#ifndef unlikely
> > +#define unlikely(x) __builtin_expect(!!(x), 0)
> > +#endif
> > +
> > +typedef struct VduseRing {
> > + unsigned int num;
> > + uint64_t desc_addr;
> > + uint64_t avail_addr;
> > + uint64_t used_addr;
> > + struct vring_desc *desc;
> > + struct vring_avail *avail;
> > + struct vring_used *used;
> > +} VduseRing;
> > +
> > +struct VduseVirtq {
> > + VduseRing vring;
> > + uint16_t last_avail_idx;
> > + uint16_t shadow_avail_idx;
> > + uint16_t used_idx;
> > + uint16_t signalled_used;
> > + bool signalled_used_valid;
> > + int index;
> > + int inuse;
> > + bool ready;
> > + int fd;
> > + VduseDev *dev;
> > +};
> > +
> > +typedef struct VduseIovaRegion {
> > + uint64_t iova;
> > + uint64_t size;
> > + uint64_t mmap_offset;
> > + uint64_t mmap_addr;
> > +} VduseIovaRegion;
> > +
> > +struct VduseDev {
> > + VduseVirtq *vqs;
> > + VduseIovaRegion regions[MAX_IOVA_REGIONS];
> > + int num_regions;
> > + char *name;
> > + uint32_t device_id;
> > + uint32_t vendor_id;
> > + uint16_t num_queues;
> > + uint16_t queue_size;
> > + uint64_t features;
> > + const VduseOps *ops;
> > + int fd;
> > + int ctrl_fd;
> > + void *priv;
> > +};
> > +
> > +static inline bool has_feature(uint64_t features, unsigned int fbit)
> > +{
> > + assert(fbit < 64);
> > + return !!(features & (1ULL << fbit));
> > +}
> > +
> > +static inline bool vduse_dev_has_feature(VduseDev *dev, unsigned int fbit)
> > +{
> > + return has_feature(dev->features, fbit);
> > +}
> > +
> > +VduseDev *vduse_queue_get_dev(VduseVirtq *vq)
> > +{
> > + return vq->dev;
> > +}
> > +
> > +int vduse_queue_get_fd(VduseVirtq *vq)
> > +{
> > + return vq->fd;
> > +}
> > +
> > +void *vduse_dev_get_priv(VduseDev *dev)
> > +{
> > + return dev->priv;
> > +}
> > +
> > +VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index)
> > +{
> > + return &dev->vqs[index];
> > +}
> > +
> > +int vduse_dev_get_fd(VduseDev *dev)
> > +{
> > + return dev->fd;
> > +}
> > +
> > +static int vduse_inject_irq(VduseDev *dev, int index)
> > +{
> > + return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index);
> > +}
> > +
> > +static void vduse_iova_remove_region(VduseDev *dev, uint64_t start,
> > + uint64_t last)
> > +{
> > + int i;
> > +
> > + if (last == start) {
> > + return;
> > + }
> > +
> > + for (i = 0; i < MAX_IOVA_REGIONS; i++) {
> > + if (!dev->regions[i].mmap_addr) {
> > + continue;
> > + }
> > +
> > + if (start <= dev->regions[i].iova &&
> > + last >= (dev->regions[i].iova + dev->regions[i].size - 1)) {
> > + munmap((void *)dev->regions[i].mmap_addr,
> > + dev->regions[i].mmap_offset + dev->regions[i].size);
> > + dev->regions[i].mmap_addr = 0;
> > + dev->num_regions--;
> > + }
> > + }
> > +}
> > +
> > +static int vduse_iova_add_region(VduseDev *dev, int fd,
> > + uint64_t offset, uint64_t start,
> > + uint64_t last, int prot)
> > +{
> > + int i;
> > + uint64_t size = last - start + 1;
> > + void *mmap_addr = mmap(0, size + offset, prot, MAP_SHARED, fd, 0);
> > +
> > + if (mmap_addr == MAP_FAILED) {
> > + return -EINVAL;
>
> Missing close(fd). This function takes ownership of fd.
Will fix it.
> > + }
> > +
> > + for (i = 0; i < MAX_IOVA_REGIONS; i++) {
> > + if (!dev->regions[i].mmap_addr) {
> > + dev->regions[i].mmap_addr = (uint64_t)(uintptr_t)mmap_addr;
> > + dev->regions[i].mmap_offset = offset;
> > + dev->regions[i].iova = start;
> > + dev->regions[i].size = size;
> > + dev->num_regions++;
> > + break;
> > + }
> > + }
> > + close(fd);
>
> assert(i < MAX_IOVA_REGIONS)? If we can really reach the end of the for
> loop then we must remember to call munmap(2).
>
Sure.
> > +
> > + return 0;
> > +}
> > +
> > +static int perm_to_prot(uint8_t perm)
> > +{
> > + int prot = 0;
> > +
> > + switch (perm) {
> > + case VDUSE_ACCESS_WO:
> > + prot |= PROT_WRITE;
> > + break;
> > + case VDUSE_ACCESS_RO:
> > + prot |= PROT_READ;
> > + break;
> > + case VDUSE_ACCESS_RW:
> > + prot |= PROT_READ | PROT_WRITE;
> > + break;
> > + default:
> > + break;
> > + }
> > +
> > + return prot;
> > +}
> > +
> > +static inline void *iova_to_va(VduseDev *dev, uint64_t *plen, uint64_t iova)
> > +{
> > + int i, ret;
> > + struct vduse_iotlb_entry entry;
> > +
> > + for (i = 0; i < MAX_IOVA_REGIONS; i++) {
> > + VduseIovaRegion *r = &dev->regions[i];
> > +
> > + if (!r->mmap_addr) {
> > + continue;
> > + }
> > +
> > + if ((iova >= r->iova) && (iova < (r->iova + r->size))) {
> > + if ((iova + *plen) > (r->iova + r->size)) {
> > + *plen = r->iova + r->size - iova;
> > + }
> > + return (void *)(uintptr_t)(iova - r->iova +
> > + r->mmap_addr + r->mmap_offset);
> > + }
> > + }
> > +
> > + entry.start = iova;
> > + entry.last = iova + 1;
> > + ret = ioctl(dev->fd, VDUSE_IOTLB_GET_FD, &entry);
> > + if (ret < 0) {
> > + return NULL;
> > + }
> > +
> > + if (!vduse_iova_add_region(dev, ret, entry.offset, entry.start,
> > + entry.last, perm_to_prot(entry.perm))) {
> > + return iova_to_va(dev, plen, iova);
> > + }
> > +
> > + return NULL;
> > +}
> > +
> > +static inline uint16_t vring_avail_flags(VduseVirtq *vq)
> > +{
> > + return le16toh(vq->vring.avail->flags);
>
> I remember we discussed whether VDUSE should support Transitional
> devices. VIRTIO 1.0+ uses little-endian but legacy VIRTIO uses
> guest-endian, so le16toh() will not work for legacy VIRTIO vrings in a
> cross-endian configuration (e.g. big-endian guest on little-endian
> host).
>
You're right. Cross-endian isn't supported now.
> If cross-endian isn't supported please add an error during
> intialization so users get a clear error message.
>
Will check VIRTIO_F_VERSION_1 during intialization.
> > +}
> > +
> > +static inline uint16_t vring_avail_idx(VduseVirtq *vq)
> > +{
> > + vq->shadow_avail_idx = le16toh(vq->vring.avail->idx);
> > +
> > + return vq->shadow_avail_idx;
> > +}
> > +
> > +static inline uint16_t vring_avail_ring(VduseVirtq *vq, int i)
> > +{
> > + return le16toh(vq->vring.avail->ring[i]);
> > +}
> > +
> > +static inline uint16_t vring_get_used_event(VduseVirtq *vq)
> > +{
> > + return vring_avail_ring(vq, vq->vring.num);
> > +}
> > +
> > +static bool vduse_queue_get_head(VduseVirtq *vq, unsigned int idx,
> > + unsigned int *head)
> > +{
> > + /*
> > + * Grab the next descriptor number they're advertising, and increment
> > + * the index we've seen.
> > + */
> > + *head = vring_avail_ring(vq, idx % vq->vring.num);
> > +
> > + /* If their number is silly, that's a fatal mistake. */
> > + if (*head >= vq->vring.num) {
> > + fprintf(stderr, "Guest says index %u is available\n", *head);
> > + return false;
> > + }
> > +
> > + return true;
> > +}
> > +
> > +static int
> > +vduse_queue_read_indirect_desc(VduseDev *dev, struct vring_desc *desc,
> > + uint64_t addr, size_t len)
> > +{
> > + struct vring_desc *ori_desc;
> > + uint64_t read_len;
> > +
> > + if (len > (VIRTQUEUE_MAX_SIZE * sizeof(struct vring_desc))) {
> > + return -1;
> > + }
> > +
> > + if (len == 0) {
> > + return -1;
> > + }
> > +
> > + while (len) {
> > + read_len = len;
> > + ori_desc = iova_to_va(dev, &read_len, addr);
> > + if (!ori_desc) {
> > + return -1;
> > + }
> > +
> > + memcpy(desc, ori_desc, read_len);
> > + len -= read_len;
> > + addr += read_len;
> > + desc += read_len;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +enum {
> > + VIRTQUEUE_READ_DESC_ERROR = -1,
> > + VIRTQUEUE_READ_DESC_DONE = 0, /* end of chain */
> > + VIRTQUEUE_READ_DESC_MORE = 1, /* more buffers in chain */
> > +};
> > +
> > +static int vduse_queue_read_next_desc(struct vring_desc *desc, int i,
> > + unsigned int max, unsigned int *next)
> > +{
> > + /* If this descriptor says it doesn't chain, we're done. */
> > + if (!(le16toh(desc[i].flags) & VRING_DESC_F_NEXT)) {
> > + return VIRTQUEUE_READ_DESC_DONE;
> > + }
> > +
> > + /* Check they're not leading us off end of descriptors. */
> > + *next = desc[i].next;
> > + /* Make sure compiler knows to grab that: we don't want it changing! */
> > + smp_wmb();
> > +
> > + if (*next >= max) {
> > + fprintf(stderr, "Desc next is %u\n", *next);
> > + return VIRTQUEUE_READ_DESC_ERROR;
> > + }
> > +
> > + return VIRTQUEUE_READ_DESC_MORE;
> > +}
> > +
> > +/*
> > + * Fetch avail_idx from VQ memory only when we really need to know if
> > + * guest has added some buffers.
> > + */
> > +static bool vduse_queue_empty(VduseVirtq *vq)
> > +{
> > + if (unlikely(!vq->vring.avail)) {
> > + return true;
> > + }
> > +
> > + if (vq->shadow_avail_idx != vq->last_avail_idx) {
> > + return false;
> > + }
> > +
> > + return vring_avail_idx(vq) == vq->last_avail_idx;
> > +}
> > +
> > +static bool vduse_queue_should_notify(VduseVirtq *vq)
> > +{
> > + VduseDev *dev = vq->dev;
> > + uint16_t old, new;
> > + bool v;
> > +
> > + /* We need to expose used array entries before checking used event. */
> > + smp_mb();
> > +
> > + /* Always notify when queue is empty (when feature acknowledge) */
> > + if (vduse_dev_has_feature(dev, VIRTIO_F_NOTIFY_ON_EMPTY) &&
> > + !vq->inuse && vduse_queue_empty(vq)) {
> > + return true;
> > + }
> > +
> > + if (!vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) {
> > + return !(vring_avail_flags(vq) & VRING_AVAIL_F_NO_INTERRUPT);
> > + }
> > +
> > + v = vq->signalled_used_valid;
> > + vq->signalled_used_valid = true;
> > + old = vq->signalled_used;
> > + new = vq->signalled_used = vq->used_idx;
> > + return !v || vring_need_event(vring_get_used_event(vq), new, old);
> > +}
> > +
> > +void vduse_queue_notify(VduseVirtq *vq)
> > +{
> > + VduseDev *dev = vq->dev;
> > +
> > + if (unlikely(!vq->vring.avail)) {
> > + return;
> > + }
> > +
> > + if (!vduse_queue_should_notify(vq)) {
> > + return;
> > + }
> > +
> > + if (vduse_inject_irq(dev, vq->index) < 0) {
> > + fprintf(stderr, "Error inject irq for vq %d: %s\n",
> > + vq->index, strerror(errno));
> > + }
> > +}
> > +
> > +static inline void vring_used_flags_set_bit(VduseVirtq *vq, int mask)
> > +{
> > + uint16_t *flags;
> > +
> > + flags = (uint16_t *)((char*)vq->vring.used +
> > + offsetof(struct vring_used, flags));
> > + *flags = htole16(le16toh(*flags) | mask);
> > +}
> > +
> > +static inline void vring_used_flags_unset_bit(VduseVirtq *vq, int mask)
> > +{
> > + uint16_t *flags;
> > +
> > + flags = (uint16_t *)((char*)vq->vring.used +
> > + offsetof(struct vring_used, flags));
> > + *flags = htole16(le16toh(*flags) & ~mask);
> > +}
> > +
> > +static inline void vring_set_avail_event(VduseVirtq *vq, uint16_t val)
> > +{
> > + *((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val);
> > +}
> > +
> > +static bool vduse_queue_map_single_desc(VduseVirtq *vq, unsigned int *p_num_sg,
> > + struct iovec *iov, unsigned int max_num_sg,
> > + bool is_write, uint64_t pa, size_t sz)
> > +{
> > + unsigned num_sg = *p_num_sg;
> > + VduseDev *dev = vq->dev;
> > +
> > + assert(num_sg <= max_num_sg);
> > +
> > + if (!sz) {
> > + fprintf(stderr, "virtio: zero sized buffers are not allowed\n");
> > + return false;
> > + }
> > +
> > + while (sz) {
> > + uint64_t len = sz;
> > +
> > + if (num_sg == max_num_sg) {
> > + fprintf(stderr,
> > + "virtio: too many descriptors in indirect table\n");
> > + return false;
> > + }
> > +
> > + iov[num_sg].iov_base = iova_to_va(dev, &len, pa);
> > + if (iov[num_sg].iov_base == NULL) {
> > + fprintf(stderr, "virtio: invalid address for buffers\n");
> > + return false;
> > + }
> > + iov[num_sg++].iov_len = len;
> > + sz -= len;
> > + pa += len;
> > + }
> > +
> > + *p_num_sg = num_sg;
> > + return true;
> > +}
> > +
> > +static void *vduse_queue_alloc_element(size_t sz, unsigned out_num,
> > + unsigned in_num)
> > +{
> > + VduseVirtqElement *elem;
> > + size_t in_sg_ofs = ALIGN_UP(sz, __alignof__(elem->in_sg[0]));
> > + size_t out_sg_ofs = in_sg_ofs + in_num * sizeof(elem->in_sg[0]);
> > + size_t out_sg_end = out_sg_ofs + out_num * sizeof(elem->out_sg[0]);
> > +
> > + assert(sz >= sizeof(VduseVirtqElement));
> > + elem = malloc(out_sg_end);
>
> Missing malloc() NULL return value check.
>
Will fix it.
> > + elem->out_num = out_num;
> > + elem->in_num = in_num;
> > + elem->in_sg = (void *)elem + in_sg_ofs;
> > + elem->out_sg = (void *)elem + out_sg_ofs;
> > + return elem;
> > +}
> > +
> > +static void *vduse_queue_map_desc(VduseVirtq *vq, unsigned int idx, size_t sz)
> > +{
> > + struct vring_desc *desc = vq->vring.desc;
> > + VduseDev *dev = vq->dev;
> > + uint64_t desc_addr, read_len;
> > + unsigned int desc_len;
> > + unsigned int max = vq->vring.num;
> > + unsigned int i = idx;
> > + VduseVirtqElement *elem;
> > + struct iovec iov[VIRTQUEUE_MAX_SIZE];
> > + struct vring_desc desc_buf[VIRTQUEUE_MAX_SIZE];
> > + unsigned int out_num = 0, in_num = 0;
> > + int rc;
> > +
> > + if (le16toh(desc[i].flags) & VRING_DESC_F_INDIRECT) {
> > + if (le32toh(desc[i].len) % sizeof(struct vring_desc)) {
> > + fprintf(stderr, "Invalid size for indirect buffer table\n");
> > + return NULL;
> > + }
> > +
> > + /* loop over the indirect descriptor table */
> > + desc_addr = le64toh(desc[i].addr);
> > + desc_len = le32toh(desc[i].len);
> > + max = desc_len / sizeof(struct vring_desc);
> > + read_len = desc_len;
> > + desc = iova_to_va(dev, &read_len, desc_addr);
> > + if (unlikely(desc && read_len != desc_len)) {
> > + /* Failed to use zero copy */
> > + desc = NULL;
> > + if (!vduse_queue_read_indirect_desc(dev, desc_buf,
> > + desc_addr,
> > + desc_len)) {
> > + desc = desc_buf;
> > + }
> > + }
> > + if (!desc) {
> > + fprintf(stderr, "Invalid indirect buffer table\n");
> > + return NULL;
> > + }
> > + i = 0;
> > + }
> > +
> > + /* Collect all the descriptors */
> > + do {
> > + if (le16toh(desc[i].flags) & VRING_DESC_F_WRITE) {
> > + if (!vduse_queue_map_single_desc(vq, &in_num, iov + out_num,
> > + VIRTQUEUE_MAX_SIZE - out_num,
> > + true, le64toh(desc[i].addr),
> > + le32toh(desc[i].len))) {
> > + return NULL;
> > + }
> > + } else {
> > + if (in_num) {
> > + fprintf(stderr, "Incorrect order for descriptors\n");
> > + return NULL;
> > + }
> > + if (!vduse_queue_map_single_desc(vq, &out_num, iov,
> > + VIRTQUEUE_MAX_SIZE, false,
> > + le64toh(desc[i].addr),
> > + le32toh(desc[i].len))) {
> > + return NULL;
> > + }
> > + }
> > +
> > + /* If we've got too many, that implies a descriptor loop. */
> > + if ((in_num + out_num) > max) {
> > + fprintf(stderr, "Looped descriptor\n");
> > + return NULL;
> > + }
> > + rc = vduse_queue_read_next_desc(desc, i, max, &i);
> > + } while (rc == VIRTQUEUE_READ_DESC_MORE);
> > +
> > + if (rc == VIRTQUEUE_READ_DESC_ERROR) {
> > + fprintf(stderr, "read descriptor error\n");
> > + return NULL;
> > + }
> > +
> > + /* Now copy what we have collected and mapped */
> > + elem = vduse_queue_alloc_element(sz, out_num, in_num);
> > + elem->index = idx;
> > + for (i = 0; i < out_num; i++) {
> > + elem->out_sg[i] = iov[i];
> > + }
> > + for (i = 0; i < in_num; i++) {
> > + elem->in_sg[i] = iov[out_num + i];
> > + }
> > +
> > + return elem;
> > +}
> > +
> > +void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
> > +{
> > + unsigned int head;
> > + VduseVirtqElement *elem;
> > + VduseDev *dev = vq->dev;
> > +
> > + if (unlikely(!vq->vring.avail)) {
> > + return NULL;
> > + }
> > +
> > + if (vduse_queue_empty(vq)) {
> > + return NULL;
> > + }
> > + /* Needed after virtio_queue_empty() */
> > + smp_rmb();
> > +
> > + if (vq->inuse >= vq->vring.num) {
> > + fprintf(stderr, "Virtqueue size exceeded: %d\n", vq->inuse);
> > + return NULL;
> > + }
> > +
> > + if (!vduse_queue_get_head(vq, vq->last_avail_idx++, &head)) {
> > + return NULL;
> > + }
> > +
> > + if (vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) {
> > + vring_set_avail_event(vq, vq->last_avail_idx);
> > + }
> > +
> > + elem = vduse_queue_map_desc(vq, head, sz);
> > +
> > + if (!elem) {
> > + return NULL;
> > + }
> > +
> > + vq->inuse++;
> > +
> > + return elem;
> > +}
> > +
> > +static inline void vring_used_write(VduseVirtq *vq,
> > + struct vring_used_elem *uelem, int i)
> > +{
> > + struct vring_used *used = vq->vring.used;
> > +
> > + used->ring[i] = *uelem;
> > +}
> > +
> > +static void vduse_queue_fill(VduseVirtq *vq, const VduseVirtqElement *elem,
> > + unsigned int len, unsigned int idx)
> > +{
> > + struct vring_used_elem uelem;
> > +
> > + if (unlikely(!vq->vring.used)) {
> > + return;
> > + }
> > +
> > + idx = (idx + vq->used_idx) % vq->vring.num;
> > +
> > + uelem.id = htole32(elem->index);
> > + uelem.len = htole32(len);
> > + vring_used_write(vq, &uelem, idx);
> > +}
> > +
> > +static inline void vring_used_idx_set(VduseVirtq *vq, uint16_t val)
> > +{
> > + vq->vring.used->idx = htole16(val);
> > + vq->used_idx = val;
> > +}
> > +
> > +static void vduse_queue_flush(VduseVirtq *vq, unsigned int count)
> > +{
> > + uint16_t old, new;
> > +
> > + if (unlikely(!vq->vring.used)) {
> > + return;
> > + }
> > +
> > + /* Make sure buffer is written before we update index. */
> > + smp_wmb();
> > +
> > + old = vq->used_idx;
> > + new = old + count;
> > + vring_used_idx_set(vq, new);
> > + vq->inuse -= count;
> > + if (unlikely((int16_t)(new - vq->signalled_used) < (uint16_t)(new - old))) {
> > + vq->signalled_used_valid = false;
> > + }
> > +}
> > +
> > +void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
> > + unsigned int len)
> > +{
> > + vduse_queue_fill(vq, elem, len, 0);
> > + vduse_queue_flush(vq, 1);
> > +}
> > +
> > +static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr,
> > + uint64_t avail_addr, uint64_t used_addr)
> > +{
> > + struct VduseDev *dev = vq->dev;
> > + uint64_t len;
> > +
> > + len = sizeof(struct vring_desc);
> > + vq->vring.desc = iova_to_va(dev, &len, desc_addr);
> > + assert(len == sizeof(struct vring_desc));
> > +
> > + len = sizeof(struct vring_avail);
> > + vq->vring.avail = iova_to_va(dev, &len, avail_addr);
> > + assert(len == sizeof(struct vring_avail));
> > +
> > + len = sizeof(struct vring_used);
> > + vq->vring.used = iova_to_va(dev, &len, used_addr);
> > + assert(len == sizeof(struct vring_used));
> > +
> > + if (!vq->vring.desc || !vq->vring.avail || !vq->vring.used) {
> > + fprintf(stderr, "Failed to get vq[%d] iova mapping\n", vq->index);
> > + return -EINVAL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static void vduse_queue_enable(VduseVirtq *vq)
> > +{
> > + struct VduseDev *dev = vq->dev;
> > + struct vduse_vq_info vq_info;
> > + struct vduse_vq_eventfd vq_eventfd;
> > + int fd;
> > +
> > + vq_info.index = vq->index;
> > + if (ioctl(dev->fd, VDUSE_VQ_GET_INFO, &vq_info)) {
> > + fprintf(stderr, "Failed to get vq[%d] info: %s\n",
> > + vq->index, strerror(errno));
> > + return;
> > + }
> > +
> > + if (!vq_info.ready) {
> > + return;
> > + }
> > +
> > + vq->vring.num = vq_info.num;
> > + vq->vring.desc_addr = vq_info.desc_addr;
> > + vq->vring.avail_addr = vq_info.driver_addr;
> > + vq->vring.used_addr = vq_info.device_addr;
> > +
> > + if (vduse_queue_update_vring(vq, vq_info.desc_addr,
> > + vq_info.driver_addr, vq_info.device_addr)) {
> > + fprintf(stderr, "Failed to update vring for vq[%d]\n", vq->index);
> > + return;
> > + }
> > +
> > + fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC);
> > + if (fd < 0) {
> > + fprintf(stderr, "Failed to init eventfd for vq[%d]\n", vq->index);
> > + return;
> > + }
> > +
> > + vq_eventfd.index = vq->index;
> > + vq_eventfd.fd = fd;
> > + if (ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &vq_eventfd)) {
> > + fprintf(stderr, "Failed to setup kick fd for vq[%d]\n", vq->index);
> > + close(fd);
> > + return;
> > + }
> > +
> > + vq->fd = fd;
> > + vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index;
> > + vq->inuse = 0;
> > + vq->used_idx = 0;
> > + vq->signalled_used_valid = false;
> > + vq->ready = true;
> > +
> > + dev->ops->enable_queue(dev, vq);
> > +}
> > +
> > +static void vduse_queue_disable(VduseVirtq *vq)
> > +{
> > + struct VduseDev *dev = vq->dev;
> > + struct vduse_vq_eventfd eventfd;
> > +
> > + if (!vq->ready) {
> > + return;
> > + }
> > +
> > + dev->ops->disable_queue(dev, vq);
> > +
> > + eventfd.index = vq->index;
> > + eventfd.fd = VDUSE_EVENTFD_DEASSIGN;
> > + ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &eventfd);
> > + close(vq->fd);
> > +
> > + assert(vq->inuse == 0);
> > +
> > + vq->vring.num = 0;
> > + vq->vring.desc_addr = 0;
> > + vq->vring.avail_addr = 0;
> > + vq->vring.used_addr = 0;
> > + vq->vring.desc = 0;
> > + vq->vring.avail = 0;
> > + vq->vring.used = 0;
> > + vq->ready = false;
> > + vq->fd = -1;
> > +}
> > +
> > +static void vduse_dev_start_dataplane(VduseDev *dev)
> > +{
> > + int i;
> > +
> > + if (ioctl(dev->fd, VDUSE_DEV_GET_FEATURES, &dev->features)) {
> > + fprintf(stderr, "Failed to get features: %s\n", strerror(errno));
> > + return;
> > + }
> > +
> > + for (i = 0; i < dev->num_queues; i++) {
> > + vduse_queue_enable(&dev->vqs[i]);
> > + }
> > +}
> > +
> > +static void vduse_dev_stop_dataplane(VduseDev *dev)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < dev->num_queues; i++) {
> > + vduse_queue_disable(&dev->vqs[i]);
> > + }
> > + dev->features = 0;
> > + vduse_iova_remove_region(dev, 0, ULONG_MAX);
> > +}
> > +
> > +int vduse_dev_handler(VduseDev *dev)
> > +{
> > + struct vduse_dev_request req;
> > + struct vduse_dev_response resp = { 0 };
> > + VduseVirtq *vq;
> > + int i, ret;
> > +
> > + ret = read(dev->fd, &req, sizeof(req));
>
> This file descriptor is blocking? I guess the assumption is that the
> kernel VDUSE code always enqueues at least one struct vduse_dev_request,
> so userspace will not block when the file descriptor becomes readable?
>
Yes, that's true. We can always get one entire request if the file
descriptor becomes readable.
> > + if (ret != sizeof(req)) {
> > + fprintf(stderr, "Read request error [%d]: %s\n",
> > + ret, strerror(errno));
> > + return -errno;
> > + }
> > + resp.request_id = req.request_id;
> > +
> > + switch (req.type) {
> > + case VDUSE_GET_VQ_STATE:
> > + vq = &dev->vqs[req.vq_state.index];
> > + resp.vq_state.split.avail_index = vq->last_avail_idx;
> > + resp.result = VDUSE_REQ_RESULT_OK;
> > + break;
> > + case VDUSE_SET_STATUS:
> > + if (req.s.status & VIRTIO_CONFIG_S_DRIVER_OK) {
> > + vduse_dev_start_dataplane(dev);
> > + } else if (req.s.status == 0) {
> > + vduse_dev_stop_dataplane(dev);
> > + }
> > + resp.result = VDUSE_REQ_RESULT_OK;
> > + break;
> > + case VDUSE_UPDATE_IOTLB:
> > + /* The iova will be updated by iova_to_va() later, so just remove it */
> > + vduse_iova_remove_region(dev, req.iova.start, req.iova.last);
> > + for (i = 0; i < dev->num_queues; i++) {
> > + VduseVirtq *vq = &dev->vqs[i];
> > + if (vq->ready) {
> > + if (vduse_queue_update_vring(vq, vq->vring.desc_addr,
> > + vq->vring.avail_addr,
> > + vq->vring.used_addr)) {
> > + fprintf(stderr, "Failed to update vring for vq[%d]\n",
> > + vq->index);
> > + }
> > + }
> > + }
> > + resp.result = VDUSE_REQ_RESULT_OK;
> > + break;
> > + default:
> > + resp.result = VDUSE_REQ_RESULT_FAILED;
> > + break;
> > + }
> > +
> > + ret = write(dev->fd, &resp, sizeof(resp));
>
> The kernel never blocks here?
>
Yes.
> > + if (ret != sizeof(resp)) {
> > + fprintf(stderr, "Write request %d error [%d]: %s\n",
> > + req.type, ret, strerror(errno));
> > + return -errno;
> > + }
> > + return 0;
> > +}
> > +
> > +int vduse_dev_update_config(VduseDev *dev, uint32_t size,
> > + uint32_t offset, char *buffer)
> > +{
> > + int ret;
> > + struct vduse_config_data *data;
> > +
> > + data = malloc(offsetof(struct vduse_config_data, buffer) + size);
> > + if (!data) {
> > + return -ENOMEM;
> > + }
> > +
> > + data->offset = offset;
> > + data->length = size;
> > + memcpy(data->buffer, buffer, size);
> > +
> > + ret = ioctl(dev->fd, VDUSE_DEV_SET_CONFIG, data);
> > + free(data);
> > +
> > + if (ret) {
> > + return -errno;
> > + }
> > +
> > + if (ioctl(dev->fd, VDUSE_DEV_INJECT_CONFIG_IRQ)) {
> > + return -errno;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size)
> > +{
> > + VduseVirtq *vq = &dev->vqs[index];
> > + struct vduse_vq_config vq_config = { 0 };
> > +
> > + vq_config.index = vq->index;
> > + vq_config.max_size = max_size;
> > +
> > + if (ioctl(dev->fd, VDUSE_VQ_SETUP, &vq_config)) {
> > + return -errno;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > + uint32_t vendor_id, uint64_t features,
> > + uint16_t num_queues, uint32_t config_size,
> > + char *config, const VduseOps *ops, void *priv)
> > +{
> > + VduseDev *dev;
> > + int i, ret, ctrl_fd, fd = -1;
> > + uint64_t version;
> > + char dev_path[VDUSE_NAME_MAX + 16];
>
> Why 16? It has to be at least strlen("/dev/vduse/"), but why more? I
> suggest including strlen("/dev/vduse/") instead of hardcoding a magic
> constant.
>
LGTM, will use strlen("/dev/vduse/") instead.
> > + VduseVirtq *vqs = NULL;
> > + struct vduse_dev_config *dev_config = NULL;
> > + size_t size = offsetof(struct vduse_dev_config, config);
> > +
> > + if (!name || strlen(name) > VDUSE_NAME_MAX || !config ||
>
> The NUL terminator needs to be taken into account:
>
> strlen(name) + 1 > VDUSE_NAME_MAX
>
Will fix it.
> > + !config_size || !ops || !ops->enable_queue || !ops->disable_queue) {
> > + fprintf(stderr, "Invalid parameter for vduse\n");
> > + return NULL;
> > + }
> > +
> > + dev = malloc(sizeof(VduseDev));
> > + if (!dev) {
> > + fprintf(stderr, "Failed to allocate vduse device\n");
> > + return NULL;
> > + }
> > + memset(dev, 0, sizeof(VduseDev));
> > +
> > + ctrl_fd = open("/dev/vduse/control", O_RDWR);
> > + if (ctrl_fd < 0) {
> > + fprintf(stderr, "Failed to open /dev/vduse/control: %s\n",
> > + strerror(errno));
> > + goto err_ctrl;
> > + }
> > +
> > + version = VDUSE_API_VERSION;
> > + if (ioctl(ctrl_fd, VDUSE_SET_API_VERSION, &version)) {
> > + fprintf(stderr, "Failed to set api version %lu: %s\n",
> > + version, strerror(errno));
> > + goto err_dev;
> > + }
> > +
> > + dev_config = malloc(size + config_size);
> > + if (!dev_config) {
> > + fprintf(stderr, "Failed to allocate config space\n");
> > + goto err_dev;
> > + }
> > + memset(dev_config, 0, size + config_size);
> > +
> > + strcpy(dev_config->name, name);
> > + dev_config->device_id = device_id;
> > + dev_config->vendor_id = vendor_id;
> > + dev_config->features = features;
> > + dev_config->vq_num = num_queues;
> > + dev_config->vq_align = VDUSE_VQ_ALIGN;
> > + dev_config->config_size = config_size;
> > + memcpy(dev_config->config, config, config_size);
> > +
> > + ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config);
> > + free(dev_config);
> > + if (ret < 0) {
> > + fprintf(stderr, "Failed to create vduse dev %s: %s\n",
> > + name, strerror(errno));
> > + goto err_dev;
> > + }
> > +
> > + sprintf(dev_path, "/dev/vduse/%s", name);
> > + fd = open(dev_path, O_RDWR);
>
> Does the caller reject names with ".." path components? Maybe input
> validation should be performed before we call open(2)?
>
Make sense.
> > + if (fd < 0) {
> > + fprintf(stderr, "Failed to open vduse dev %s: %s\n",
> > + name, strerror(errno));
> > + goto err;
> > + }
> > +
> > + vqs = calloc(sizeof(VduseVirtq), num_queues);
>
> calloc() could be used instead of malloc + memset above as well.
>
OK.
> > + if (!vqs) {
> > + fprintf(stderr, "Failed to allocate virtqueues\n");
> > + goto err;
> > + }
> > +
> > + for (i = 0; i < num_queues; i++) {
> > + vqs[i].index = i;
> > + vqs[i].dev = dev;
> > + vqs[i].fd = -1;
> > + }
> > +
> > + dev->vqs = vqs;
> > + dev->name = strdup(name);
>
> malloc(3) return values are checked elsewhere, strdup(3) should also be
> checked.
>
OK.
> > + dev->num_queues = num_queues;
> > + dev->ops = ops;
> > + dev->ctrl_fd = ctrl_fd;
> > + dev->fd = fd;
> > + dev->priv = priv;
> > +
> > + return dev;
> > +err:
> > + if (fd > 0) {
> > + close(fd);
> > + }
> > + ioctl(ctrl_fd, VDUSE_DESTROY_DEV, name);
> > +err_dev:
> > + close(ctrl_fd);
> > +err_ctrl:
> > + free(dev);
> > +
> > + return NULL;
> > +}
> > +
> > +void vduse_dev_destroy(VduseDev *dev)
> > +{
> > + free(dev->vqs);
> > + close(dev->fd);
> > + dev->fd = -1;
> > + ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name);
> > + free(dev->name);
> > + close(dev->ctrl_fd);
> > + dev->ctrl_fd = -1;
> > + free(dev);
> > +}
> > diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h
> > new file mode 100644
> > index 0000000000..f6bcb51b5a
> > --- /dev/null
> > +++ b/subprojects/libvduse/libvduse.h
> > @@ -0,0 +1,193 @@
> > +/*
> > + * VDUSE (vDPA Device in Userspace) library
> > + *
> > + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> > + *
> > + * Author:
> > + * Xie Yongji <xieyongji@bytedance.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or
> > + * later. See the COPYING file in the top-level directory.
> > + */
> > +
> > +#ifndef LIBVDUSE_H
> > +#define LIBVDUSE_H
> > +
> > +#include <stdint.h>
> > +#include <sys/uio.h>
> > +
> > +/* VDUSE device structure */
> > +typedef struct VduseDev VduseDev;
> > +
> > +/* Virtqueue structure */
> > +typedef struct VduseVirtq VduseVirtq;
> > +
> > +/* Some operation of VDUSE backend */
> > +typedef struct VduseOps {
> > + /* Called when virtqueue can be processed */
> > + void (*enable_queue)(VduseDev *dev, VduseVirtq *vq);
> > + /* Called when virtqueue processing should be stopped */
> > + void (*disable_queue)(VduseDev *dev, VduseVirtq *vq);
> > +} VduseOps;
> > +
> > +/* Describing elements of the I/O buffer */
> > +typedef struct VduseVirtqElement {
> > + /* Virtqueue index */
> > + unsigned int index;
>
> Is this the descriptor table index or the virtqueue number?
>
It's descriptor table index. Will fix it in v2.
Thanks,
Yongji
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 3/5] vduse-blk: implements vduse-blk export
2022-02-07 14:15 ` Stefan Hajnoczi
@ 2022-02-08 6:56 ` Yongji Xie
0 siblings, 0 replies; 19+ messages in thread
From: Yongji Xie @ 2022-02-08 6:56 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
On Mon, Feb 7, 2022 at 10:15 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Tue, Jan 25, 2022 at 09:17:58PM +0800, Xie Yongji wrote:
> > This implements a VDUSE block backends based on
> > the libvduse library. We can use it to export the BDSs
> > for both VM and container (host) usage.
> >
> > The new command-line syntax is:
> >
> > $ qemu-storage-daemon \
> > --blockdev file,node-name=drive0,filename=test.img \
> > --export vduse-blk,node-name=drive0,id=vduse-export0,writable=on
> >
> > After the qemu-storage-daemon started, we need to use
> > the "vdpa" command to attach the device to vDPA bus:
> >
> > $ vdpa dev add name vduse-export0 mgmtdev vduse
> >
> > Also the device must be removed via the "vdpa" command
> > before we stop the qemu-storage-daemon.
> >
> > Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> > ---
> > block/export/export.c | 6 +
> > block/export/meson.build | 5 +
> > block/export/vduse-blk.c | 427 ++++++++++++++++++++++++++++++++++
> > block/export/vduse-blk.h | 20 ++
> > meson.build | 13 ++
> > meson_options.txt | 2 +
> > qapi/block-export.json | 24 +-
> > scripts/meson-buildoptions.sh | 4 +
> > 8 files changed, 499 insertions(+), 2 deletions(-)
> > create mode 100644 block/export/vduse-blk.c
> > create mode 100644 block/export/vduse-blk.h
> >
> > diff --git a/block/export/export.c b/block/export/export.c
> > index 6d3b9964c8..00dd505540 100644
> > --- a/block/export/export.c
> > +++ b/block/export/export.c
> > @@ -26,6 +26,9 @@
> > #ifdef CONFIG_VHOST_USER_BLK_SERVER
> > #include "vhost-user-blk-server.h"
> > #endif
> > +#ifdef CONFIG_VDUSE_BLK_EXPORT
> > +#include "vduse-blk.h"
> > +#endif
> >
> > static const BlockExportDriver *blk_exp_drivers[] = {
> > &blk_exp_nbd,
> > @@ -35,6 +38,9 @@ static const BlockExportDriver *blk_exp_drivers[] = {
> > #ifdef CONFIG_FUSE
> > &blk_exp_fuse,
> > #endif
> > +#ifdef CONFIG_VDUSE_BLK_EXPORT
> > + &blk_exp_vduse_blk,
> > +#endif
> > };
> >
> > /* Only accessed from the main thread */
> > diff --git a/block/export/meson.build b/block/export/meson.build
> > index 0a08e384c7..cf311d2b1b 100644
> > --- a/block/export/meson.build
> > +++ b/block/export/meson.build
> > @@ -5,3 +5,8 @@ if have_vhost_user_blk_server
> > endif
> >
> > blockdev_ss.add(when: fuse, if_true: files('fuse.c'))
> > +
> > +if have_vduse_blk_export
> > + blockdev_ss.add(files('vduse-blk.c'))
> > + blockdev_ss.add(libvduse)
> > +endif
> > diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
> > new file mode 100644
> > index 0000000000..5a8d289685
> > --- /dev/null
> > +++ b/block/export/vduse-blk.c
> > @@ -0,0 +1,427 @@
> > +/*
> > + * Export QEMU block device via VDUSE
> > + *
> > + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved.
> > + * Portions of codes and concepts borrowed from vhost-user-blk-server.c, so:
> > + * Copyright (c) 2020 Red Hat, Inc.
> > + *
> > + * Author:
> > + * Xie Yongji <xieyongji@bytedance.com>
> > + * Coiby Xu <coiby.xu@gmail.com>
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or
> > + * later. See the COPYING file in the top-level directory.
> > + */
> > +
> > +#include <sys/eventfd.h>
> > +
> > +#include "qemu/osdep.h"
> > +#include "qapi/error.h"
> > +#include "sysemu/block-backend.h"
> > +#include "block/export.h"
> > +#include "qemu/error-report.h"
> > +#include "util/block-helpers.h"
> > +#include "subprojects/libvduse/libvduse.h"
> > +
> > +#include "standard-headers/linux/virtio_ring.h"
> > +#include "standard-headers/linux/virtio_blk.h"
> > +
> > +#define VIRTIO_BLK_SECTOR_BITS 9
> > +#define VIRTIO_BLK_SECTOR_SIZE (1ULL << VIRTIO_BLK_SECTOR_BITS)
> > +
> > +#define VDUSE_DEFAULT_NUM_QUEUE 1
> > +#define VDUSE_DEFAULT_QUEUE_SIZE 128
>
> QEMU's virtio-blk emulation has increased this limit to 256 for better
> performance with large block size I/O patterns. I think it would be okay
> to increase it here too.
>
Sure.
> > +
> > +typedef struct VduseBlkExport {
> > + BlockExport export;
> > + VduseDev *dev;
> > + uint16_t num_queues;
> > + uint32_t blk_size;
> > + bool writable;
> > +} VduseBlkExport;
> > +
> > +struct virtio_blk_inhdr {
> > + unsigned char status;
> > +};
> > +
> > +typedef struct VduseBlkReq {
> > + VduseVirtqElement elem;
> > + int64_t sector_num;
> > + size_t in_len;
> > + struct virtio_blk_inhdr *in;
> > + struct virtio_blk_outhdr out;
> > + VduseVirtq *vq;
> > +} VduseBlkReq;
> > +
> > +static void vduse_blk_req_complete(VduseBlkReq *req)
> > +{
> > + vduse_queue_push(req->vq, &req->elem, req->in_len);
> > + vduse_queue_notify(req->vq);
> > +
> > + free(req);
> > +}
> > +
> > +static bool vduse_blk_sect_range_ok(VduseBlkExport *vblk_exp,
> > + uint64_t sector, size_t size)
> > +{
> > + uint64_t nb_sectors;
> > + uint64_t total_sectors;
> > +
> > + if (size % VIRTIO_BLK_SECTOR_SIZE) {
> > + return false;
> > + }
> > +
> > + nb_sectors = size >> VIRTIO_BLK_SECTOR_BITS;
> > +
> > + QEMU_BUILD_BUG_ON(BDRV_SECTOR_SIZE != VIRTIO_BLK_SECTOR_SIZE);
> > + if (nb_sectors > BDRV_REQUEST_MAX_SECTORS) {
> > + return false;
> > + }
> > + if ((sector << VIRTIO_BLK_SECTOR_BITS) % vblk_exp->blk_size) {
> > + return false;
> > + }
> > + blk_get_geometry(vblk_exp->export.blk, &total_sectors);
> > + if (sector > total_sectors || nb_sectors > total_sectors - sector) {
> > + return false;
> > + }
> > + return true;
> > +}
> > +
> > +static void coroutine_fn vduse_blk_virtio_process_req(void *opaque)
> > +{
> > + VduseBlkReq *req = opaque;
> > + VduseVirtq *vq = req->vq;
> > + VduseDev *dev = vduse_queue_get_dev(vq);
> > + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
> > + BlockBackend *blk = vblk_exp->export.blk;
> > + VduseVirtqElement *elem = &req->elem;
> > + struct iovec *in_iov = elem->in_sg;
> > + struct iovec *out_iov = elem->out_sg;
> > + unsigned in_num = elem->in_num;
> > + unsigned out_num = elem->out_num;
> > + uint32_t type;
> > +
> > + if (elem->out_num < 1 || elem->in_num < 1) {
> > + error_report("virtio-blk request missing headers");
> > + goto err;
> > + }
> > +
> > + if (unlikely(iov_to_buf(out_iov, out_num, 0, &req->out,
> > + sizeof(req->out)) != sizeof(req->out))) {
> > + error_report("virtio-blk request outhdr too short");
> > + goto err;
> > + }
> > +
> > + iov_discard_front(&out_iov, &out_num, sizeof(req->out));
> > +
> > + if (in_iov[in_num - 1].iov_len < sizeof(struct virtio_blk_inhdr)) {
> > + error_report("virtio-blk request inhdr too short");
> > + goto err;
> > + }
> > +
> > + /* We always touch the last byte, so just see how big in_iov is. */
> > + req->in_len = iov_size(in_iov, in_num);
> > + req->in = (void *)in_iov[in_num - 1].iov_base
> > + + in_iov[in_num - 1].iov_len
> > + - sizeof(struct virtio_blk_inhdr);
> > + iov_discard_back(in_iov, &in_num, sizeof(struct virtio_blk_inhdr));
> > +
> > + type = le32_to_cpu(req->out.type);
> > + switch (type & ~VIRTIO_BLK_T_BARRIER) {
> > + case VIRTIO_BLK_T_IN:
> > + case VIRTIO_BLK_T_OUT: {
> > + QEMUIOVector qiov;
> > + int64_t offset;
> > + ssize_t ret = 0;
> > + bool is_write = type & VIRTIO_BLK_T_OUT;
> > + req->sector_num = le64_to_cpu(req->out.sector);
> > +
> > + if (is_write && !vblk_exp->writable) {
> > + req->in->status = VIRTIO_BLK_S_IOERR;
> > + break;
> > + }
> > +
> > + if (is_write) {
> > + qemu_iovec_init_external(&qiov, out_iov, out_num);
> > + } else {
> > + qemu_iovec_init_external(&qiov, in_iov, in_num);
> > + }
> > +
> > + if (unlikely(!vduse_blk_sect_range_ok(vblk_exp,
> > + req->sector_num,
> > + qiov.size))) {
> > + req->in->status = VIRTIO_BLK_S_IOERR;
> > + break;
> > + }
> > +
> > + offset = req->sector_num << VIRTIO_BLK_SECTOR_BITS;
> > +
> > + if (is_write) {
> > + ret = blk_co_pwritev(blk, offset, qiov.size, &qiov, 0);
> > + } else {
> > + ret = blk_co_preadv(blk, offset, qiov.size, &qiov, 0);
> > + }
> > + if (ret >= 0) {
> > + req->in->status = VIRTIO_BLK_S_OK;
> > + } else {
> > + req->in->status = VIRTIO_BLK_S_IOERR;
> > + }
> > + break;
> > + }
> > + case VIRTIO_BLK_T_FLUSH:
> > + if (blk_co_flush(blk) == 0) {
> > + req->in->status = VIRTIO_BLK_S_OK;
> > + } else {
> > + req->in->status = VIRTIO_BLK_S_IOERR;
> > + }
> > + break;
> > + case VIRTIO_BLK_T_GET_ID: {
> > + size_t size = MIN(iov_size(&elem->in_sg[0], in_num),
> > + VIRTIO_BLK_ID_BYTES);
> > + snprintf(elem->in_sg[0].iov_base, size, "%s", vblk_exp->export.id);
> > + req->in->status = VIRTIO_BLK_S_OK;
> > + break;
> > + }
> > + default:
> > + req->in->status = VIRTIO_BLK_S_UNSUPP;
> > + break;
> > + }
> > +
> > + vduse_blk_req_complete(req);
> > + return;
> > +
> > +err:
> > + free(req);
> > +}
> > +
> > +static void vduse_blk_vq_handler(VduseDev *dev, VduseVirtq *vq)
> > +{
> > + while (1) {
> > + VduseBlkReq *req;
> > +
> > + req = vduse_queue_pop(vq, sizeof(VduseBlkReq));
> > + if (!req) {
> > + break;
> > + }
> > + req->vq = vq;
> > +
> > + Coroutine *co =
> > + qemu_coroutine_create(vduse_blk_virtio_process_req, req);
> > + qemu_coroutine_enter(co);
> > + }
> > +}
> > +
> > +static void on_vduse_vq_kick(void *opaque)
> > +{
> > + VduseVirtq *vq = opaque;
> > + VduseDev *dev = vduse_queue_get_dev(vq);
> > + int fd = vduse_queue_get_fd(vq);
> > + eventfd_t kick_data;
> > +
> > + if (eventfd_read(fd, &kick_data) == -1) {
> > + error_report("failed to read data from eventfd");
> > + return;
> > + }
> > +
> > + vduse_blk_vq_handler(dev, vq);
> > +}
> > +
> > +static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq)
> > +{
> > + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
> > +
> > + aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
> > + true, on_vduse_vq_kick, NULL, NULL, NULL, vq);
> > +}
> > +
> > +static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq)
> > +{
> > + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev);
> > +
> > + aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
> > + true, NULL, NULL, NULL, NULL, NULL);
> > +}
> > +
> > +static const VduseOps vduse_blk_ops = {
> > + .enable_queue = vduse_blk_enable_queue,
> > + .disable_queue = vduse_blk_disable_queue,
> > +};
> > +
> > +static void on_vduse_dev_kick(void *opaque)
> > +{
> > + VduseDev *dev = opaque;
> > +
> > + vduse_dev_handler(dev);
> > +}
> > +
> > +static void blk_aio_attached(AioContext *ctx, void *opaque)
> > +{
> > + VduseBlkExport *vblk_exp = opaque;
> > + int i;
> > +
> > + vblk_exp->export.ctx = ctx;
> > +
> > + aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev),
> > + true, on_vduse_dev_kick, NULL, NULL, NULL,
> > + vblk_exp->dev);
> > +
> > + for (i = 0; i < vblk_exp->num_queues; i++) {
> > + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
> > + int fd = vduse_queue_get_fd(vq);
> > +
> > + if (fd < 0) {
> > + continue;
> > + }
> > + aio_set_fd_handler(vblk_exp->export.ctx, fd, true,
> > + on_vduse_vq_kick, NULL, NULL, NULL, vq);
> > + }
> > +}
> > +
> > +static void blk_aio_detach(void *opaque)
> > +{
> > + VduseBlkExport *vblk_exp = opaque;
> > + int i;
> > +
> > + for (i = 0; i < vblk_exp->num_queues; i++) {
> > + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
> > + int fd = vduse_queue_get_fd(vq);
> > +
> > + if (fd < 0) {
> > + continue;
> > + }
> > + aio_set_fd_handler(vblk_exp->export.ctx, fd,
> > + true, NULL, NULL, NULL, NULL, NULL);
> > + }
> > + aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev),
> > + true, NULL, NULL, NULL, NULL, NULL);
> > + vblk_exp->export.ctx = NULL;
> > +}
> > +
> > +static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> > + Error **errp)
> > +{
> > + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> > + BlockExportOptionsVduseBlk *vblk_opts = &opts->u.vduse_blk;
> > + uint64_t logical_block_size = VIRTIO_BLK_SECTOR_SIZE;
> > + uint16_t num_queues = VDUSE_DEFAULT_NUM_QUEUE;
> > + uint16_t queue_size = VDUSE_DEFAULT_QUEUE_SIZE;
> > + Error *local_err = NULL;
> > + struct virtio_blk_config config;
>
> Please zero-initialize this with "= { 0 }" because this struct can grow
> when Linux adds virtio-blk features and our function may not have code
> to fill in the new fields.
>
Looks good to me.
> > + uint64_t features;
> > + int i;
> > +
> > + if (vblk_opts->has_num_queues) {
> > + num_queues = vblk_opts->num_queues;
> > + if (num_queues == 0) {
> > + error_setg(errp, "num-queues must be greater than 0");
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (vblk_opts->has_queue_size) {
> > + queue_size = vblk_opts->queue_size;
> > + if (queue_size == 0) {
> > + error_setg(errp, "queue-size must be greater than 0");
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (vblk_opts->has_logical_block_size) {
> > + logical_block_size = vblk_opts->logical_block_size;
> > + check_block_size(exp->id, "logical-block-size", logical_block_size,
> > + &local_err);
> > + if (local_err) {
> > + error_propagate(errp, local_err);
> > + return -EINVAL;
> > + }
> > + }
> > + blk_set_guest_block_size(exp->blk, logical_block_size);
> > +
> > + vblk_exp->blk_size = logical_block_size;
> > + vblk_exp->writable = opts->writable;
> > + vblk_exp->num_queues = num_queues;
> > +
> > + config.capacity =
> > + cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS);
> > + config.seg_max = cpu_to_le32(queue_size - 2);
>
> What is queue_size is 1 or 2?
>
OK, will validate the queue_size in v2.
> > + config.size_max = cpu_to_le32(0);
> > + config.min_io_size = cpu_to_le16(1);
> > + config.opt_io_size = cpu_to_le32(1);
> > + config.num_queues = cpu_to_le16(num_queues);
> > + config.blk_size = cpu_to_le32(logical_block_size);
> > +
> > + features = (1ULL << VIRTIO_F_IOMMU_PLATFORM) |
> > + (1ULL << VIRTIO_F_VERSION_1) |
> > + (1ULL << VIRTIO_RING_F_EVENT_IDX) |
> > + (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) |
> > + (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
> > + (1ULL << VIRTIO_BLK_F_SIZE_MAX) |
> > + (1ULL << VIRTIO_BLK_F_SEG_MAX) |
> > + (1ULL << VIRTIO_BLK_F_TOPOLOGY) |
> > + (1ULL << VIRTIO_BLK_F_BLK_SIZE);
> > +
> > + if (num_queues > 1) {
> > + features |= 1ULL << VIRTIO_BLK_F_MQ;
> > + }
> > + if (!vblk_exp->writable) {
> > + features |= 1ULL << VIRTIO_BLK_F_RO;
> > + }
> > +
> > + vblk_exp->dev = vduse_dev_create(exp->id, VIRTIO_ID_BLOCK, 0,
> > + features, num_queues,
> > + sizeof(struct virtio_blk_config),
> > + (char *)&config, &vduse_blk_ops,
> > + vblk_exp);
> > + if (!vblk_exp->dev) {
> > + error_setg(errp, "failed to create vduse device");
> > + return -ENOMEM;
> > + }
> > +
> > + for (i = 0; i < num_queues; i++) {
> > + vduse_dev_setup_queue(vblk_exp->dev, i, queue_size);
> > + }
> > +
> > + aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev), true,
> > + on_vduse_dev_kick, NULL, NULL, NULL, vblk_exp->dev);
> > +
> > + blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
> > + vblk_exp);
> > +
> > + return 0;
> > +}
> > +
> > +static void vduse_blk_exp_delete(BlockExport *exp)
> > +{
> > + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> > +
> > + vduse_dev_destroy(vblk_exp->dev);
> > +}
> > +
> > +static void vduse_blk_exp_request_shutdown(BlockExport *exp)
> > +{
> > + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> > + int i;
> > +
> > + blk_remove_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
> > + vblk_exp);
> > +
> > + for (i = 0; i < vblk_exp->num_queues; i++) {
> > + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i);
> > + int fd = vduse_queue_get_fd(vq);
> > +
> > + if (fd < 0) {
> > + continue;
> > + }
> > + aio_set_fd_handler(exp->ctx, fd, true, NULL, NULL, NULL, NULL, NULL);
> > + }
> > + aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev),
> > + true, NULL, NULL, NULL, NULL, NULL);
>
> Call blk_aio_detach() instead of duplicating this code?
>
Sure.
Thanks,
Yongji
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 4/5] vduse-blk: Add vduse-blk resize support
2022-02-07 14:18 ` Stefan Hajnoczi
@ 2022-02-08 7:12 ` Yongji Xie
0 siblings, 0 replies; 19+ messages in thread
From: Yongji Xie @ 2022-02-08 7:12 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
On Mon, Feb 7, 2022 at 10:18 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Tue, Jan 25, 2022 at 09:17:59PM +0800, Xie Yongji wrote:
> > To support block resize, this uses vduse_dev_update_config()
> > to update the capacity field in configuration space and inject
> > config interrupt on the block resize callback.
> >
> > Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> > ---
> > block/export/vduse-blk.c | 19 +++++++++++++++++++
> > 1 file changed, 19 insertions(+)
> >
> > diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
> > index 5a8d289685..83845e9a9a 100644
> > --- a/block/export/vduse-blk.c
> > +++ b/block/export/vduse-blk.c
> > @@ -297,6 +297,23 @@ static void blk_aio_detach(void *opaque)
> > vblk_exp->export.ctx = NULL;
> > }
> >
> > +static void vduse_blk_resize(void *opaque)
> > +{
> > + BlockExport *exp = opaque;
> > + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export);
> > + struct virtio_blk_config config;
> > +
> > + config.capacity =
> > + cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS);
> > + vduse_dev_update_config(vblk_exp->dev, sizeof(config.capacity),
> > + offsetof(struct virtio_blk_config, capacity),
> > + (char *)&config.capacity);
> > +}
> > +
> > +static const BlockDevOps vduse_block_ops = {
> > + .resize_cb = vduse_blk_resize,
> > +};
> > +
> > static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> > Error **errp)
> > {
> > @@ -387,6 +404,8 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> > blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
> > vblk_exp);
> >
> > + blk_set_dev_ops(exp->blk, &vduse_block_ops, exp);
>
> Detach is missing, so BlockBackend->dev_ops will become stale after the
> export is deleted. Please add code to detach when the export is deleted.
OK.
Thanks,
Yongji
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] libvduse: Add support for reconnecting
2022-02-07 14:39 ` Stefan Hajnoczi
@ 2022-02-08 7:35 ` Yongji Xie
2022-02-08 8:08 ` Stefan Hajnoczi
0 siblings, 1 reply; 19+ messages in thread
From: Yongji Xie @ 2022-02-08 7:35 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
On Mon, Feb 7, 2022 at 10:39 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Tue, Jan 25, 2022 at 09:18:00PM +0800, Xie Yongji wrote:
> > To support reconnecting after restart or crash, VDUSE backend
> > might need to resubmit inflight I/Os. This stores the metadata
> > such as the index of inflight I/O's descriptors to a shm file so
> > that VDUSE backend can restore them during reconnecting.
> >
> > Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> > ---
> > block/export/vduse-blk.c | 4 +-
> > subprojects/libvduse/libvduse.c | 254 +++++++++++++++++++++++++++++++-
> > subprojects/libvduse/libvduse.h | 4 +-
> > 3 files changed, 254 insertions(+), 8 deletions(-)
> >
> > diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c
> > index 83845e9a9a..bc14fd798b 100644
> > --- a/block/export/vduse-blk.c
> > +++ b/block/export/vduse-blk.c
> > @@ -232,6 +232,8 @@ static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq)
> >
> > aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq),
> > true, on_vduse_vq_kick, NULL, NULL, NULL, vq);
> > + /* Make sure we don't miss any kick afer reconnecting */
> > + eventfd_write(vduse_queue_get_fd(vq), 1);
> > }
> >
> > static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq)
> > @@ -388,7 +390,7 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts,
> > features, num_queues,
> > sizeof(struct virtio_blk_config),
> > (char *)&config, &vduse_blk_ops,
> > - vblk_exp);
> > + g_get_tmp_dir(), vblk_exp);
> > if (!vblk_exp->dev) {
> > error_setg(errp, "failed to create vduse device");
> > return -ENOMEM;
> > diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c
> > index 7671864bca..ce2f6c7949 100644
> > --- a/subprojects/libvduse/libvduse.c
> > +++ b/subprojects/libvduse/libvduse.c
> > @@ -41,6 +41,8 @@
> > #define VDUSE_VQ_ALIGN 4096
> > #define MAX_IOVA_REGIONS 256
> >
> > +#define LOG_ALIGNMENT 64
> > +
> > /* Round number down to multiple */
> > #define ALIGN_DOWN(n, m) ((n) / (m) * (m))
> >
> > @@ -51,6 +53,31 @@
> > #define unlikely(x) __builtin_expect(!!(x), 0)
> > #endif
> >
> > +typedef struct VduseDescStateSplit {
> > + uint8_t inflight;
> > + uint8_t padding[5];
> > + uint16_t next;
> > + uint64_t counter;
> > +} VduseDescStateSplit;
> > +
> > +typedef struct VduseVirtqLogInflight {
> > + uint64_t features;
> > + uint16_t version;
> > + uint16_t desc_num;
> > + uint16_t last_batch_head;
> > + uint16_t used_idx;
> > + VduseDescStateSplit desc[];
> > +} VduseVirtqLogInflight;
> > +
> > +typedef struct VduseVirtqLog {
> > + VduseVirtqLogInflight inflight;
> > +} VduseVirtqLog;
> > +
> > +typedef struct VduseVirtqInflightDesc {
> > + uint16_t index;
> > + uint64_t counter;
> > +} VduseVirtqInflightDesc;
> > +
> > typedef struct VduseRing {
> > unsigned int num;
> > uint64_t desc_addr;
> > @@ -73,6 +100,10 @@ struct VduseVirtq {
> > bool ready;
> > int fd;
> > VduseDev *dev;
> > + VduseVirtqInflightDesc *resubmit_list;
> > + uint16_t resubmit_num;
> > + uint64_t counter;
> > + VduseVirtqLog *log;
> > };
> >
> > typedef struct VduseIovaRegion {
> > @@ -96,8 +127,67 @@ struct VduseDev {
> > int fd;
> > int ctrl_fd;
> > void *priv;
> > + char *shm_log_dir;
> > + void *log;
> > + bool reconnect;
> > };
> >
> > +static inline size_t vduse_vq_log_size(uint16_t queue_size)
> > +{
> > + return ALIGN_UP(sizeof(VduseDescStateSplit) * queue_size +
> > + sizeof(VduseVirtqLogInflight), LOG_ALIGNMENT);
> > +}
> > +
> > +static void *vduse_log_get(const char *dir, const char *name, size_t size)
> > +{
> > + void *ptr = MAP_FAILED;
> > + char *path;
> > + int fd;
> > +
> > + path = (char *)malloc(strlen(dir) + strlen(name) +
> > + strlen("/vduse-log-") + 1);
> > + if (!path) {
> > + return ptr;
> > + }
> > + sprintf(path, "%s/vduse-log-%s", dir, name);
>
> Please use g_strdup_printf() and g_autofree in QEMU code. In libvduse
> code it's okay to use malloc(3), but regular QEMU code should use glib.
>
But this code resides in libvduse currently.
> > +
> > + fd = open(path, O_RDWR | O_CREAT, 0600);
> > + if (fd == -1) {
> > + goto out;
> > + }
> > +
> > + if (ftruncate(fd, size) == -1) {
> > + goto out;
> > + }
> > +
> > + ptr = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
> > + if (ptr == MAP_FAILED) {
> > + goto out;
> > + }
> > +out:
> > + if (fd > 0) {
> > + close(fd);
> > + }
> > + free(path);
> > +
> > + return ptr;
> > +}
> > +
> > +static void vduse_log_destroy(const char *dir, const char *name)
> > +{
> > + char *path;
> > +
> > + path = (char *)malloc(strlen(dir) + strlen(name) +
> > + strlen("/vduse-log-") + 1);
> > + if (!path) {
> > + return;
> > + }
> > + sprintf(path, "%s/vduse-log-%s", dir, name);
> > +
> > + unlink(path);
> > + free(path);
> > +}
> > +
> > static inline bool has_feature(uint64_t features, unsigned int fbit)
> > {
> > assert(fbit < 64);
> > @@ -139,6 +229,98 @@ static int vduse_inject_irq(VduseDev *dev, int index)
> > return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index);
> > }
> >
> > +static int inflight_desc_compare(const void *a, const void *b)
> > +{
> > + VduseVirtqInflightDesc *desc0 = (VduseVirtqInflightDesc *)a,
> > + *desc1 = (VduseVirtqInflightDesc *)b;
> > +
> > + if (desc1->counter > desc0->counter &&
> > + (desc1->counter - desc0->counter) < VIRTQUEUE_MAX_SIZE * 2) {
> > + return 1;
> > + }
> > +
> > + return -1;
> > +}
> > +
> > +static int vduse_queue_check_inflights(VduseVirtq *vq)
> > +{
> > + int i = 0;
> > + VduseDev *dev = vq->dev;
> > +
> > + vq->used_idx = vq->vring.used->idx;
>
> Is this reading struct vring_used->idx without le16toh()?
>
> > + vq->resubmit_num = 0;
> > + vq->resubmit_list = NULL;
> > + vq->counter = 0;
> > +
> > + if (unlikely(vq->log->inflight.used_idx != vq->used_idx)) {
> > + vq->log->inflight.desc[vq->log->inflight.last_batch_head].inflight = 0;
>
> I suggest validating vq->log->inflight fields before using them.
> last_batch_head must be less than the virtqueue size. Although the log
> file is somewhat trusted, there may still be ways to corrupt it or
> confuse the new process that loads it.
>
I can validate the last_batch_head field. But it's hard to validate
the inflight field, so we might still meet some issues if the file is
corrupted.
> > +
> > + barrier();
> > +
> > + vq->log->inflight.used_idx = vq->used_idx;
> > + }
> > +
> > + for (i = 0; i < vq->log->inflight.desc_num; i++) {
> > + if (vq->log->inflight.desc[i].inflight == 1) {
> > + vq->inuse++;
> > + }
> > + }
> > +
> > + vq->shadow_avail_idx = vq->last_avail_idx = vq->inuse + vq->used_idx;
> > +
> > + if (vq->inuse) {
> > + vq->resubmit_list = calloc(vq->inuse, sizeof(VduseVirtqInflightDesc));
> > + if (!vq->resubmit_list) {
> > + return -1;
> > + }
> > +
> > + for (i = 0; i < vq->log->inflight.desc_num; i++) {
> > + if (vq->log->inflight.desc[i].inflight) {
> > + vq->resubmit_list[vq->resubmit_num].index = i;
> > + vq->resubmit_list[vq->resubmit_num].counter =
> > + vq->log->inflight.desc[i].counter;
> > + vq->resubmit_num++;
> > + }
> > + }
> > +
> > + if (vq->resubmit_num > 1) {
> > + qsort(vq->resubmit_list, vq->resubmit_num,
> > + sizeof(VduseVirtqInflightDesc), inflight_desc_compare);
> > + }
> > + vq->counter = vq->resubmit_list[0].counter + 1;
> > + }
> > +
> > + vduse_inject_irq(dev, vq->index);
> > +
> > + return 0;
> > +}
> > +
> > +static int vduse_queue_inflight_get(VduseVirtq *vq, int desc_idx)
> > +{
> > + vq->log->inflight.desc[desc_idx].counter = vq->counter++;
> > + vq->log->inflight.desc[desc_idx].inflight = 1;
> > +
> > + return 0;
> > +}
> > +
> > +static int vduse_queue_inflight_pre_put(VduseVirtq *vq, int desc_idx)
> > +{
> > + vq->log->inflight.last_batch_head = desc_idx;
> > +
> > + return 0;
> > +}
> > +
> > +static int vduse_queue_inflight_post_put(VduseVirtq *vq, int desc_idx)
> > +{
> > + vq->log->inflight.desc[desc_idx].inflight = 0;
> > +
> > + barrier();
> > +
> > + vq->log->inflight.used_idx = vq->used_idx;
> > +
> > + return 0;
> > +}
> > +
> > static void vduse_iova_remove_region(VduseDev *dev, uint64_t start,
> > uint64_t last)
> > {
> > @@ -578,11 +760,24 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
> > unsigned int head;
> > VduseVirtqElement *elem;
> > VduseDev *dev = vq->dev;
> > + int i;
> >
> > if (unlikely(!vq->vring.avail)) {
> > return NULL;
> > }
> >
> > + if (unlikely(vq->resubmit_list && vq->resubmit_num > 0)) {
> > + i = (--vq->resubmit_num);
> > + elem = vduse_queue_map_desc(vq, vq->resubmit_list[i].index, sz);
> > +
> > + if (!vq->resubmit_num) {
> > + free(vq->resubmit_list);
> > + vq->resubmit_list = NULL;
> > + }
>
> resubmit_list is only freed when vduse_queue_pop() is called often
> enough to empty the list. Please free the list when the vduse instance
> is destroyed too, just in case vduse_queue_pop() wasn't called often
> enough to free it.
>
Will do it.
> > +
> > + return elem;
> > + }
> > +
> > if (vduse_queue_empty(vq)) {
> > return NULL;
> > }
> > @@ -610,6 +805,8 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz)
> >
> > vq->inuse++;
> >
> > + vduse_queue_inflight_get(vq, head);
> > +
> > return elem;
> > }
> >
> > @@ -667,7 +864,9 @@ void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem,
> > unsigned int len)
> > {
> > vduse_queue_fill(vq, elem, len, 0);
> > + vduse_queue_inflight_pre_put(vq, elem->index);
> > vduse_queue_flush(vq, 1);
> > + vduse_queue_inflight_post_put(vq, elem->index);
> > }
> >
> > static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr,
> > @@ -740,12 +939,11 @@ static void vduse_queue_enable(VduseVirtq *vq)
> > }
> >
> > vq->fd = fd;
> > - vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index;
> > - vq->inuse = 0;
> > - vq->used_idx = 0;
> > vq->signalled_used_valid = false;
> > vq->ready = true;
> >
> > + vduse_queue_check_inflights(vq);
> > +
> > dev->ops->enable_queue(dev, vq);
> > }
> >
> > @@ -903,13 +1101,18 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size)
> > return -errno;
> > }
> >
> > + if (dev->reconnect) {
> > + vduse_queue_enable(vq);
> > + }
> > +
> > return 0;
> > }
> >
> > VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > uint32_t vendor_id, uint64_t features,
> > uint16_t num_queues, uint32_t config_size,
> > - char *config, const VduseOps *ops, void *priv)
> > + char *config, const VduseOps *ops,
> > + const char *shm_log_dir, void *priv)
> > {
> > VduseDev *dev;
> > int i, ret, ctrl_fd, fd = -1;
> > @@ -918,6 +1121,8 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > VduseVirtq *vqs = NULL;
> > struct vduse_dev_config *dev_config = NULL;
> > size_t size = offsetof(struct vduse_dev_config, config);
> > + size_t log_size = num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE);
> > + void *log = NULL;
> >
> > if (!name || strlen(name) > VDUSE_NAME_MAX || !config ||
> > !config_size || !ops || !ops->enable_queue || !ops->disable_queue) {
> > @@ -932,6 +1137,15 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > }
> > memset(dev, 0, sizeof(VduseDev));
> >
> > + if (shm_log_dir) {
> > + dev->log = log = vduse_log_get(shm_log_dir, name, log_size);
> > + if (!log) {
> > + fprintf(stderr, "Failed to get vduse log\n");
> > + goto err_ctrl;
> > + }
> > + dev->shm_log_dir = strdup(shm_log_dir);
> > + }
> > +
> > ctrl_fd = open("/dev/vduse/control", O_RDWR);
> > if (ctrl_fd < 0) {
> > fprintf(stderr, "Failed to open /dev/vduse/control: %s\n",
> > @@ -964,7 +1178,11 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> >
> > ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config);
> > free(dev_config);
> > - if (ret < 0) {
> > + if (!ret && log) {
> > + memset(log, 0, log_size);
> > + } else if (errno == EEXIST && log) {
> > + dev->reconnect = true;
> > + } else {
> > fprintf(stderr, "Failed to create vduse dev %s: %s\n",
> > name, strerror(errno));
> > goto err_dev;
> > @@ -978,6 +1196,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > goto err;
> > }
> >
> > + if (dev->reconnect &&
> > + ioctl(fd, VDUSE_DEV_GET_FEATURES, &dev->features)) {
> > + fprintf(stderr, "Failed to get features: %s\n", strerror(errno));
> > + goto err;
> > + }
> > +
> > vqs = calloc(sizeof(VduseVirtq), num_queues);
> > if (!vqs) {
> > fprintf(stderr, "Failed to allocate virtqueues\n");
> > @@ -988,6 +1212,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > vqs[i].index = i;
> > vqs[i].dev = dev;
> > vqs[i].fd = -1;
> > + if (log) {
> > + vqs[i].log = log;
> > + vqs[i].log->inflight.desc_num = VIRTQUEUE_MAX_SIZE;
> > + log = (void *)((char *)log +
> > + vduse_vq_log_size(VIRTQUEUE_MAX_SIZE));
>
> The size of the log needs to be verified. The file is mmapped but
> there's no guarantee that the size matches num_queues *
> vduse_vq_log_size(VIRTQUEUE_MAX_SIZE).
>
We will call ftruncate() in vduse_log_get(). Is it enough?
Thanks,
Yongji
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library
2022-02-08 6:42 ` Yongji Xie
@ 2022-02-08 8:02 ` Stefan Hajnoczi
0 siblings, 0 replies; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-08 8:02 UTC (permalink / raw)
To: Yongji Xie
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]
On Tue, Feb 08, 2022 at 02:42:41PM +0800, Yongji Xie wrote:
> On Mon, Feb 7, 2022 at 10:01 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >
> > On Tue, Jan 25, 2022 at 09:17:57PM +0800, Xie Yongji wrote:
> > > +int vduse_dev_handler(VduseDev *dev)
> > > +{
> > > + struct vduse_dev_request req;
> > > + struct vduse_dev_response resp = { 0 };
> > > + VduseVirtq *vq;
> > > + int i, ret;
> > > +
> > > + ret = read(dev->fd, &req, sizeof(req));
> >
> > This file descriptor is blocking? I guess the assumption is that the
> > kernel VDUSE code always enqueues at least one struct vduse_dev_request,
> > so userspace will not block when the file descriptor becomes readable?
> >
>
> Yes, that's true. We can always get one entire request if the file
> descriptor becomes readable.
Okay, then the code is fine. We trust the kernel not to block us. While
it is possible to get spurious select(2)/poll(2) ready file descriptors
in the general case (e.g. multiple processes monitoring the same file),
I don't think that can happen in this case.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] libvduse: Add support for reconnecting
2022-02-08 7:35 ` Yongji Xie
@ 2022-02-08 8:08 ` Stefan Hajnoczi
2022-02-08 8:14 ` Yongji Xie
0 siblings, 1 reply; 19+ messages in thread
From: Stefan Hajnoczi @ 2022-02-08 8:08 UTC (permalink / raw)
To: Yongji Xie
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
[-- Attachment #1: Type: text/plain, Size: 2926 bytes --]
On Tue, Feb 08, 2022 at 03:35:27PM +0800, Yongji Xie wrote:
> On Mon, Feb 7, 2022 at 10:39 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >
> > On Tue, Jan 25, 2022 at 09:18:00PM +0800, Xie Yongji wrote:
> > > +static void *vduse_log_get(const char *dir, const char *name, size_t size)
> > > +{
> > > + void *ptr = MAP_FAILED;
> > > + char *path;
> > > + int fd;
> > > +
> > > + path = (char *)malloc(strlen(dir) + strlen(name) +
> > > + strlen("/vduse-log-") + 1);
> > > + if (!path) {
> > > + return ptr;
> > > + }
> > > + sprintf(path, "%s/vduse-log-%s", dir, name);
> >
> > Please use g_strdup_printf() and g_autofree in QEMU code. In libvduse
> > code it's okay to use malloc(3), but regular QEMU code should use glib.
> >
>
> But this code resides in libvduse currently.
Oops, I thought we were in block/export/vduse-blk.c. Then it's fine to
use malloc(3).
> > > +static int vduse_queue_check_inflights(VduseVirtq *vq)
> > > +{
> > > + int i = 0;
> > > + VduseDev *dev = vq->dev;
> > > +
> > > + vq->used_idx = vq->vring.used->idx;
> >
> > Is this reading struct vring_used->idx without le16toh()?
> >
> > > + vq->resubmit_num = 0;
> > > + vq->resubmit_list = NULL;
> > > + vq->counter = 0;
> > > +
> > > + if (unlikely(vq->log->inflight.used_idx != vq->used_idx)) {
> > > + vq->log->inflight.desc[vq->log->inflight.last_batch_head].inflight = 0;
> >
> > I suggest validating vq->log->inflight fields before using them.
> > last_batch_head must be less than the virtqueue size. Although the log
> > file is somewhat trusted, there may still be ways to corrupt it or
> > confuse the new process that loads it.
> >
>
> I can validate the last_batch_head field. But it's hard to validate
> the inflight field, so we might still meet some issues if the file is
> corrupted.
It's okay if the log tells us to resubmit virtqueue buffers that have
garbage vring descriptors because the vring code needs to handle garbage
descriptors anyway.
But we cannot load dest[untrusted_input] or do anything else that could
crash, corrupt memory, etc.
> > > @@ -988,6 +1212,12 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id,
> > > vqs[i].index = i;
> > > vqs[i].dev = dev;
> > > vqs[i].fd = -1;
> > > + if (log) {
> > > + vqs[i].log = log;
> > > + vqs[i].log->inflight.desc_num = VIRTQUEUE_MAX_SIZE;
> > > + log = (void *)((char *)log +
> > > + vduse_vq_log_size(VIRTQUEUE_MAX_SIZE));
> >
> > The size of the log needs to be verified. The file is mmapped but
> > there's no guarantee that the size matches num_queues *
> > vduse_vq_log_size(VIRTQUEUE_MAX_SIZE).
> >
>
> We will call ftruncate() in vduse_log_get(). Is it enough?
Yes, I think so.
Thanks,
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH 5/5] libvduse: Add support for reconnecting
2022-02-08 8:08 ` Stefan Hajnoczi
@ 2022-02-08 8:14 ` Yongji Xie
0 siblings, 0 replies; 19+ messages in thread
From: Yongji Xie @ 2022-02-08 8:14 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, qemu-block, Michael S. Tsirkin, Jason Wang,
qemu-devel, mreitz, mlureau, Stefano Garzarella
On Tue, Feb 8, 2022 at 4:09 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Tue, Feb 08, 2022 at 03:35:27PM +0800, Yongji Xie wrote:
> > On Mon, Feb 7, 2022 at 10:39 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > >
> > > On Tue, Jan 25, 2022 at 09:18:00PM +0800, Xie Yongji wrote:
> > > > +static void *vduse_log_get(const char *dir, const char *name, size_t size)
> > > > +{
> > > > + void *ptr = MAP_FAILED;
> > > > + char *path;
> > > > + int fd;
> > > > +
> > > > + path = (char *)malloc(strlen(dir) + strlen(name) +
> > > > + strlen("/vduse-log-") + 1);
> > > > + if (!path) {
> > > > + return ptr;
> > > > + }
> > > > + sprintf(path, "%s/vduse-log-%s", dir, name);
> > >
> > > Please use g_strdup_printf() and g_autofree in QEMU code. In libvduse
> > > code it's okay to use malloc(3), but regular QEMU code should use glib.
> > >
> >
> > But this code resides in libvduse currently.
>
> Oops, I thought we were in block/export/vduse-blk.c. Then it's fine to
> use malloc(3).
>
> > > > +static int vduse_queue_check_inflights(VduseVirtq *vq)
> > > > +{
> > > > + int i = 0;
> > > > + VduseDev *dev = vq->dev;
> > > > +
> > > > + vq->used_idx = vq->vring.used->idx;
> > >
> > > Is this reading struct vring_used->idx without le16toh()?
> > >
> > > > + vq->resubmit_num = 0;
> > > > + vq->resubmit_list = NULL;
> > > > + vq->counter = 0;
> > > > +
> > > > + if (unlikely(vq->log->inflight.used_idx != vq->used_idx)) {
> > > > + vq->log->inflight.desc[vq->log->inflight.last_batch_head].inflight = 0;
> > >
> > > I suggest validating vq->log->inflight fields before using them.
> > > last_batch_head must be less than the virtqueue size. Although the log
> > > file is somewhat trusted, there may still be ways to corrupt it or
> > > confuse the new process that loads it.
> > >
> >
> > I can validate the last_batch_head field. But it's hard to validate
> > the inflight field, so we might still meet some issues if the file is
> > corrupted.
>
> It's okay if the log tells us to resubmit virtqueue buffers that have
> garbage vring descriptors because the vring code needs to handle garbage
> descriptors anyway.
>
> But we cannot load dest[untrusted_input] or do anything else that could
> crash, corrupt memory, etc.
>
Makes sense to me.
Thanks,
Yongji
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2022-02-08 10:08 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-01-25 13:17 [PATCH 0/5] Support exporting BDSs via VDUSE Xie Yongji
2022-01-25 13:17 ` [PATCH 1/5] linux-headers: Add vduse.h Xie Yongji
2022-02-07 13:12 ` Stefan Hajnoczi
2022-02-08 3:13 ` Yongji Xie
2022-01-25 13:17 ` [PATCH 2/5] libvduse: Add VDUSE (vDPA Device in Userspace) library Xie Yongji
2022-02-07 14:00 ` Stefan Hajnoczi
2022-02-08 6:42 ` Yongji Xie
2022-02-08 8:02 ` Stefan Hajnoczi
2022-01-25 13:17 ` [PATCH 3/5] vduse-blk: implements vduse-blk export Xie Yongji
2022-02-07 14:15 ` Stefan Hajnoczi
2022-02-08 6:56 ` Yongji Xie
2022-01-25 13:17 ` [PATCH 4/5] vduse-blk: Add vduse-blk resize support Xie Yongji
2022-02-07 14:18 ` Stefan Hajnoczi
2022-02-08 7:12 ` Yongji Xie
2022-01-25 13:18 ` [PATCH 5/5] libvduse: Add support for reconnecting Xie Yongji
2022-02-07 14:39 ` Stefan Hajnoczi
2022-02-08 7:35 ` Yongji Xie
2022-02-08 8:08 ` Stefan Hajnoczi
2022-02-08 8:14 ` Yongji Xie
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).