* [PATCH 0/3] um: Add VFIO-based PCI passthrough support
@ 2025-03-15 16:19 Tiwei Bie
2025-03-15 16:19 ` [PATCH 1/3] um: Rewrite the sigio workaround based on epoll and tgkill Tiwei Bie
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Tiwei Bie @ 2025-03-15 16:19 UTC (permalink / raw)
To: richard, anton.ivanov, johannes; +Cc: linux-um, Tiwei Bie
This patchset adds a new virt-pci driver to UML that allows users to
pass through host PCI devices to UML via VFIO. Currently, only MSI-X
capable devices are supported, and drivers are assumed to use MSI-X.
This driver has been tested with virtio-net-pci, virtio-blk-pci and
nvme in a QEMU virtual machine with virtual IOMMU enabled. Regression
testing for virtio_pcidev has only covered compile tests and partial
virtio probe/remove tests with a dummy vhost-user backend.
Here are some steps to try out this driver:
1. Enable IOMMU and bind PCI devices to the vfio-pci driver.
2. Launch a UML instance directly (assuming that the PCI devices to
be passed through to UML are 0000:02:00.0 and 0000:03:00.0).
$ ./linux mem=2G init=/bin/sh ubd0=$your_rootfs_image \
vfio_uml.device=0000:02:00.0 \
vfio_uml.device=0000:03:00.0
Note that, currently, it's not possible to allocate a large amount of
"physical" memory to the UML instance; otherwise, it may overlap with
IOMMU-reserved IOVA ranges (e.g. [0xfee00000, 0xfef00000)), causing
DMA mapping failures.
This patchset is based on the following patchset:
https://lore.kernel.org/all/20250306150747.2926434-1-tiwei.btw@antgroup.com/
v1:
- Add more details in the commit log;
RFC: https://lore.kernel.org/all/20250310074057.3977758-1-tiwei.btw@antgroup.com/
Tiwei Bie (3):
um: Rewrite the sigio workaround based on epoll and tgkill
um: virt-pci: Refactor virtio_pcidev into its own module
um: Add VFIO-based virtual PCI driver
arch/um/drivers/Kconfig | 20 +-
arch/um/drivers/Makefile | 5 +-
arch/um/drivers/random.c | 2 +-
arch/um/drivers/rtc_user.c | 2 +-
arch/um/drivers/vfio_kern.c | 648 +++++++++++++++++++++++++++++
arch/um/drivers/vfio_user.c | 323 +++++++++++++++
arch/um/drivers/vfio_user.h | 44 ++
arch/um/drivers/virt-pci.c | 699 ++++++--------------------------
arch/um/drivers/virt-pci.h | 41 ++
arch/um/drivers/virtio_pcidev.c | 628 ++++++++++++++++++++++++++++
arch/um/include/shared/os.h | 2 +-
arch/um/include/shared/sigio.h | 1 -
arch/um/kernel/sigio.c | 26 --
arch/um/os-Linux/sigio.c | 330 ++-------------
14 files changed, 1866 insertions(+), 905 deletions(-)
create mode 100644 arch/um/drivers/vfio_kern.c
create mode 100644 arch/um/drivers/vfio_user.c
create mode 100644 arch/um/drivers/vfio_user.h
create mode 100644 arch/um/drivers/virt-pci.h
create mode 100644 arch/um/drivers/virtio_pcidev.c
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/3] um: Rewrite the sigio workaround based on epoll and tgkill
2025-03-15 16:19 [PATCH 0/3] um: Add VFIO-based PCI passthrough support Tiwei Bie
@ 2025-03-15 16:19 ` Tiwei Bie
2025-03-15 16:19 ` [PATCH 2/3] um: virt-pci: Refactor virtio_pcidev into its own module Tiwei Bie
2025-03-15 16:19 ` [PATCH 3/3] um: Add VFIO-based virtual PCI driver Tiwei Bie
2 siblings, 0 replies; 6+ messages in thread
From: Tiwei Bie @ 2025-03-15 16:19 UTC (permalink / raw)
To: richard, anton.ivanov, johannes; +Cc: linux-um, Tiwei Bie
The existing sigio workaround implementation removes FDs from the
poll when events are triggered, requiring users to re-add them via
add_sigio_fd() after processing. This introduces a potential race
condition between FD removal in write_sigio_thread() and next_poll
update in __add_sigio_fd(), and is inefficient due to frequent FD
removal and re-addition. Rewrite the implementation based on epoll
and tgkill for improved efficiency and reliability.
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
---
arch/um/drivers/random.c | 2 +-
arch/um/drivers/rtc_user.c | 2 +-
arch/um/include/shared/os.h | 2 +-
arch/um/include/shared/sigio.h | 1 -
arch/um/kernel/sigio.c | 26 ---
arch/um/os-Linux/sigio.c | 330 +++++----------------------------
6 files changed, 47 insertions(+), 316 deletions(-)
diff --git a/arch/um/drivers/random.c b/arch/um/drivers/random.c
index da985e0dc69a..ca08c91f47a3 100644
--- a/arch/um/drivers/random.c
+++ b/arch/um/drivers/random.c
@@ -79,7 +79,7 @@ static int __init rng_init (void)
if (err < 0)
goto err_out_cleanup_hw;
- sigio_broken(random_fd);
+ sigio_broken();
hwrng.name = RNG_MODULE_NAME;
hwrng.read = rng_dev_read;
diff --git a/arch/um/drivers/rtc_user.c b/arch/um/drivers/rtc_user.c
index 7c3cec4c68cf..51e79f3148cd 100644
--- a/arch/um/drivers/rtc_user.c
+++ b/arch/um/drivers/rtc_user.c
@@ -39,7 +39,7 @@ int uml_rtc_start(bool timetravel)
}
/* apparently timerfd won't send SIGIO, use workaround */
- sigio_broken(uml_rtc_irq_fds[0]);
+ sigio_broken();
err = add_sigio_fd(uml_rtc_irq_fds[0]);
if (err < 0) {
close(uml_rtc_irq_fds[0]);
diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index c4f8f990ffb8..9a146912dd75 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -315,7 +315,7 @@ extern void um_irqs_resume(void);
extern int add_sigio_fd(int fd);
extern int ignore_sigio_fd(int fd);
extern void maybe_sigio_broken(int fd);
-extern void sigio_broken(int fd);
+extern void sigio_broken(void);
/*
* unlocked versions for IRQ controller code.
*
diff --git a/arch/um/include/shared/sigio.h b/arch/um/include/shared/sigio.h
index e60c8b227844..c6c2edce1f6d 100644
--- a/arch/um/include/shared/sigio.h
+++ b/arch/um/include/shared/sigio.h
@@ -6,7 +6,6 @@
#ifndef __SIGIO_H__
#define __SIGIO_H__
-extern int write_sigio_irq(int fd);
extern void sigio_lock(void);
extern void sigio_unlock(void);
diff --git a/arch/um/kernel/sigio.c b/arch/um/kernel/sigio.c
index 5085a50c3b8c..4fc04742048a 100644
--- a/arch/um/kernel/sigio.c
+++ b/arch/um/kernel/sigio.c
@@ -8,32 +8,6 @@
#include <os.h>
#include <sigio.h>
-/* Protected by sigio_lock() called from write_sigio_workaround */
-static int sigio_irq_fd = -1;
-
-static irqreturn_t sigio_interrupt(int irq, void *data)
-{
- char c;
-
- os_read_file(sigio_irq_fd, &c, sizeof(c));
- return IRQ_HANDLED;
-}
-
-int write_sigio_irq(int fd)
-{
- int err;
-
- err = um_request_irq(SIGIO_WRITE_IRQ, fd, IRQ_READ, sigio_interrupt,
- 0, "write sigio", NULL);
- if (err < 0) {
- printk(KERN_ERR "write_sigio_irq : um_request_irq failed, "
- "err = %d\n", err);
- return -1;
- }
- sigio_irq_fd = fd;
- return 0;
-}
-
/* These are called from os-Linux/sigio.c to protect its pollfds arrays. */
static DEFINE_MUTEX(sigio_mutex);
diff --git a/arch/um/os-Linux/sigio.c b/arch/um/os-Linux/sigio.c
index 61b348a2ea97..a05a6ecee756 100644
--- a/arch/um/os-Linux/sigio.c
+++ b/arch/um/os-Linux/sigio.c
@@ -11,6 +11,7 @@
#include <sched.h>
#include <signal.h>
#include <string.h>
+#include <sys/epoll.h>
#include <kern_util.h>
#include <init.h>
#include <os.h>
@@ -23,180 +24,49 @@
*/
static struct os_helper_thread *write_sigio_td;
-/*
- * These arrays are initialized before the sigio thread is started, and
- * the descriptors closed after it is killed. So, it can't see them change.
- * On the UML side, they are changed under the sigio_lock.
- */
-#define SIGIO_FDS_INIT {-1, -1}
-
-static int write_sigio_fds[2] = SIGIO_FDS_INIT;
-static int sigio_private[2] = SIGIO_FDS_INIT;
+static int epollfd = -1;
-struct pollfds {
- struct pollfd *poll;
- int size;
- int used;
-};
+#define MAX_EPOLL_EVENTS 64
-/*
- * Protected by sigio_lock(). Used by the sigio thread, but the UML thread
- * synchronizes with it.
- */
-static struct pollfds current_poll;
-static struct pollfds next_poll;
-static struct pollfds all_sigio_fds;
+static struct epoll_event epoll_events[MAX_EPOLL_EVENTS];
static void *write_sigio_thread(void *unused)
{
- struct pollfds *fds, tmp;
- struct pollfd *p;
- int i, n, respond_fd;
- char c;
+ int pid = getpid();
+ int r;
os_fix_helper_thread_signals();
- fds = ¤t_poll;
while (1) {
- n = poll(fds->poll, fds->used, -1);
- if (n < 0) {
+ r = epoll_wait(epollfd, epoll_events, MAX_EPOLL_EVENTS, -1);
+ if (r < 0) {
if (errno == EINTR)
continue;
- printk(UM_KERN_ERR "write_sigio_thread : poll returned "
- "%d, errno = %d\n", n, errno);
+ printk(UM_KERN_ERR "%s: epoll_wait failed, errno = %d\n",
+ __func__, errno);
}
- for (i = 0; i < fds->used; i++) {
- p = &fds->poll[i];
- if (p->revents == 0)
- continue;
- if (p->fd == sigio_private[1]) {
- CATCH_EINTR(n = read(sigio_private[1], &c,
- sizeof(c)));
- if (n != sizeof(c))
- printk(UM_KERN_ERR
- "write_sigio_thread : "
- "read on socket failed, "
- "err = %d\n", errno);
- tmp = current_poll;
- current_poll = next_poll;
- next_poll = tmp;
- respond_fd = sigio_private[1];
- }
- else {
- respond_fd = write_sigio_fds[1];
- fds->used--;
- memmove(&fds->poll[i], &fds->poll[i + 1],
- (fds->used - i) * sizeof(*fds->poll));
- }
-
- CATCH_EINTR(n = write(respond_fd, &c, sizeof(c)));
- if (n != sizeof(c))
- printk(UM_KERN_ERR "write_sigio_thread : "
- "write on socket failed, err = %d\n",
- errno);
- }
- }
-
- return NULL;
-}
-
-static int need_poll(struct pollfds *polls, int n)
-{
- struct pollfd *new;
-
- if (n <= polls->size)
- return 0;
-
- new = uml_kmalloc(n * sizeof(struct pollfd), UM_GFP_ATOMIC);
- if (new == NULL) {
- printk(UM_KERN_ERR "need_poll : failed to allocate new "
- "pollfds\n");
- return -ENOMEM;
- }
-
- memcpy(new, polls->poll, polls->used * sizeof(struct pollfd));
- kfree(polls->poll);
-
- polls->poll = new;
- polls->size = n;
- return 0;
-}
-
-/*
- * Must be called with sigio_lock held, because it's needed by the marked
- * critical section.
- */
-static void update_thread(void)
-{
- unsigned long flags;
- int n;
- char c;
-
- flags = um_set_signals_trace(0);
- CATCH_EINTR(n = write(sigio_private[0], &c, sizeof(c)));
- if (n != sizeof(c)) {
- printk(UM_KERN_ERR "update_thread : write failed, err = %d\n",
- errno);
- goto fail;
- }
- CATCH_EINTR(n = read(sigio_private[0], &c, sizeof(c)));
- if (n != sizeof(c)) {
- printk(UM_KERN_ERR "update_thread : read failed, err = %d\n",
- errno);
- goto fail;
+ CATCH_EINTR(r = tgkill(pid, pid, SIGIO));
+ if (r < 0)
+ printk(UM_KERN_ERR "%s: tgkill failed, errno = %d\n",
+ __func__, errno);
}
- um_set_signals_trace(flags);
- return;
- fail:
- /* Critical section start */
- if (write_sigio_td) {
- os_kill_helper_thread(write_sigio_td);
- write_sigio_td = NULL;
- }
- close(sigio_private[0]);
- close(sigio_private[1]);
- close(write_sigio_fds[0]);
- close(write_sigio_fds[1]);
- /* Critical section end */
- um_set_signals_trace(flags);
+ return NULL;
}
int __add_sigio_fd(int fd)
{
- struct pollfd *p;
- int err, i, n;
-
- for (i = 0; i < all_sigio_fds.used; i++) {
- if (all_sigio_fds.poll[i].fd == fd)
- break;
- }
- if (i == all_sigio_fds.used)
- return -ENOSPC;
-
- p = &all_sigio_fds.poll[i];
-
- for (i = 0; i < current_poll.used; i++) {
- if (current_poll.poll[i].fd == fd)
- return 0;
- }
-
- n = current_poll.used;
- err = need_poll(&next_poll, n + 1);
- if (err)
- return err;
-
- memcpy(next_poll.poll, current_poll.poll,
- current_poll.used * sizeof(struct pollfd));
- next_poll.poll[n] = *p;
- next_poll.used = n + 1;
- update_thread();
-
- return 0;
+ struct epoll_event event = {
+ .data.fd = fd,
+ .events = EPOLLIN | EPOLLET,
+ };
+ int r;
+
+ CATCH_EINTR(r = epoll_ctl(epollfd, EPOLL_CTL_ADD, fd, &event));
+ return r < 0 ? -errno : 0;
}
-
int add_sigio_fd(int fd)
{
int err;
@@ -210,38 +80,11 @@ int add_sigio_fd(int fd)
int __ignore_sigio_fd(int fd)
{
- struct pollfd *p;
- int err, i, n = 0;
-
- /*
- * This is called from exitcalls elsewhere in UML - if
- * sigio_cleanup has already run, then update_thread will hang
- * or fail because the thread is no longer running.
- */
- if (!write_sigio_td)
- return -EIO;
-
- for (i = 0; i < current_poll.used; i++) {
- if (current_poll.poll[i].fd == fd)
- break;
- }
- if (i == current_poll.used)
- return -ENOENT;
-
- err = need_poll(&next_poll, current_poll.used - 1);
- if (err)
- return err;
+ struct epoll_event event;
+ int r;
- for (i = 0; i < current_poll.used; i++) {
- p = ¤t_poll.poll[i];
- if (p->fd != fd)
- next_poll.poll[n++] = *p;
- }
- next_poll.used = current_poll.used - 1;
-
- update_thread();
-
- return 0;
+ CATCH_EINTR(r = epoll_ctl(epollfd, EPOLL_CTL_DEL, fd, &event));
+ return r < 0 ? -errno : 0;
}
int ignore_sigio_fd(int fd)
@@ -255,122 +98,37 @@ int ignore_sigio_fd(int fd)
return err;
}
-static struct pollfd *setup_initial_poll(int fd)
-{
- struct pollfd *p;
-
- p = uml_kmalloc(sizeof(struct pollfd), UM_GFP_KERNEL);
- if (p == NULL) {
- printk(UM_KERN_ERR "setup_initial_poll : failed to allocate "
- "poll\n");
- return NULL;
- }
- *p = ((struct pollfd) { .fd = fd,
- .events = POLLIN,
- .revents = 0 });
- return p;
-}
-
static void write_sigio_workaround(void)
{
- struct pollfd *p;
int err;
- int l_write_sigio_fds[2];
- int l_sigio_private[2];
- struct os_helper_thread *l_write_sigio_td;
-
- /* We call this *tons* of times - and most ones we must just fail. */
- sigio_lock();
- l_write_sigio_td = write_sigio_td;
- sigio_unlock();
-
- if (l_write_sigio_td)
- return;
-
- err = os_pipe(l_write_sigio_fds, 1, 1);
- if (err < 0) {
- printk(UM_KERN_ERR "write_sigio_workaround - os_pipe 1 failed, "
- "err = %d\n", -err);
- return;
- }
- err = os_pipe(l_sigio_private, 1, 1);
- if (err < 0) {
- printk(UM_KERN_ERR "write_sigio_workaround - os_pipe 2 failed, "
- "err = %d\n", -err);
- goto out_close1;
- }
-
- p = setup_initial_poll(l_sigio_private[1]);
- if (!p)
- goto out_close2;
sigio_lock();
-
- /*
- * Did we race? Don't try to optimize this, please, it's not so likely
- * to happen, and no more than once at the boot.
- */
if (write_sigio_td)
- goto out_free;
-
- current_poll = ((struct pollfds) { .poll = p,
- .used = 1,
- .size = 1 });
-
- if (write_sigio_irq(l_write_sigio_fds[0]))
- goto out_clear_poll;
+ goto out;
- memcpy(write_sigio_fds, l_write_sigio_fds, sizeof(l_write_sigio_fds));
- memcpy(sigio_private, l_sigio_private, sizeof(l_sigio_private));
+ epollfd = epoll_create(MAX_EPOLL_EVENTS);
+ if (epollfd < 0) {
+ printk(UM_KERN_ERR "%s: epoll_create failed, errno = %d\n",
+ __func__, errno);
+ goto out;
+ }
err = os_run_helper_thread(&write_sigio_td, write_sigio_thread, NULL);
- if (err < 0)
- goto out_clear;
-
- sigio_unlock();
- return;
+ if (err < 0) {
+ printk(UM_KERN_ERR "%s: os_run_helper_thread failed, errno = %d\n",
+ __func__, -err);
+ close(epollfd);
+ epollfd = -1;
+ goto out;
+ }
-out_clear:
- write_sigio_td = NULL;
- write_sigio_fds[0] = -1;
- write_sigio_fds[1] = -1;
- sigio_private[0] = -1;
- sigio_private[1] = -1;
-out_clear_poll:
- current_poll = ((struct pollfds) { .poll = NULL,
- .size = 0,
- .used = 0 });
-out_free:
+out:
sigio_unlock();
- kfree(p);
-out_close2:
- close(l_sigio_private[0]);
- close(l_sigio_private[1]);
-out_close1:
- close(l_write_sigio_fds[0]);
- close(l_write_sigio_fds[1]);
}
-void sigio_broken(int fd)
+void sigio_broken(void)
{
- int err;
-
write_sigio_workaround();
-
- sigio_lock();
- err = need_poll(&all_sigio_fds, all_sigio_fds.used + 1);
- if (err) {
- printk(UM_KERN_ERR "maybe_sigio_broken - failed to add pollfd "
- "for descriptor %d\n", fd);
- goto out;
- }
-
- all_sigio_fds.poll[all_sigio_fds.used++] =
- ((struct pollfd) { .fd = fd,
- .events = POLLIN,
- .revents = 0 });
-out:
- sigio_unlock();
}
/* Changed during early boot */
@@ -384,7 +142,7 @@ void maybe_sigio_broken(int fd)
if (pty_output_sigio)
return;
- sigio_broken(fd);
+ sigio_broken();
}
static void sigio_cleanup(void)
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/3] um: virt-pci: Refactor virtio_pcidev into its own module
2025-03-15 16:19 [PATCH 0/3] um: Add VFIO-based PCI passthrough support Tiwei Bie
2025-03-15 16:19 ` [PATCH 1/3] um: Rewrite the sigio workaround based on epoll and tgkill Tiwei Bie
@ 2025-03-15 16:19 ` Tiwei Bie
2025-03-15 16:19 ` [PATCH 3/3] um: Add VFIO-based virtual PCI driver Tiwei Bie
2 siblings, 0 replies; 6+ messages in thread
From: Tiwei Bie @ 2025-03-15 16:19 UTC (permalink / raw)
To: richard, anton.ivanov, johannes; +Cc: linux-um, Tiwei Bie
Decouple virt-pci and virtio_pcidev, refactoring virtio_pcidev into
its own module. Define a set of APIs for virt-pci. This allows for
future addition of more PCI emulation implementations.
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
---
arch/um/drivers/Kconfig | 12 +-
arch/um/drivers/Makefile | 3 +-
arch/um/drivers/virt-pci.c | 699 ++++++--------------------------
arch/um/drivers/virt-pci.h | 41 ++
arch/um/drivers/virtio_pcidev.c | 628 ++++++++++++++++++++++++++++
5 files changed, 794 insertions(+), 589 deletions(-)
create mode 100644 arch/um/drivers/virt-pci.h
create mode 100644 arch/um/drivers/virtio_pcidev.c
diff --git a/arch/um/drivers/Kconfig b/arch/um/drivers/Kconfig
index ede40a160c5e..9cb196070614 100644
--- a/arch/um/drivers/Kconfig
+++ b/arch/um/drivers/Kconfig
@@ -345,16 +345,20 @@ config UML_RTC
by providing a fake RTC clock that causes a wakeup at the right
time.
-config UML_PCI_OVER_VIRTIO
- bool "Enable PCI over VIRTIO device simulation"
- # in theory, just VIRTIO is enough, but that causes recursion
- depends on VIRTIO_UML
+config UML_PCI
+ bool
select FORCE_PCI
select UML_IOMEM_EMULATION
select UML_DMA_EMULATION
select PCI_MSI
select PCI_LOCKLESS_CONFIG
+config UML_PCI_OVER_VIRTIO
+ bool "Enable PCI over VIRTIO device simulation"
+ # in theory, just VIRTIO is enough, but that causes recursion
+ depends on VIRTIO_UML
+ select UML_PCI
+
config UML_PCI_OVER_VIRTIO_DEVICE_ID
int "set the virtio device ID for PCI emulation"
default -1
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 57882e6bc215..0a5820343ad3 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -60,7 +60,8 @@ obj-$(CONFIG_BLK_DEV_COW_COMMON) += cow_user.o
obj-$(CONFIG_UML_RANDOM) += random.o
obj-$(CONFIG_VIRTIO_UML) += virtio_uml.o
obj-$(CONFIG_UML_RTC) += rtc.o
-obj-$(CONFIG_UML_PCI_OVER_VIRTIO) += virt-pci.o
+obj-$(CONFIG_UML_PCI) += virt-pci.o
+obj-$(CONFIG_UML_PCI_OVER_VIRTIO) += virtio_pcidev.o
# pcap_user.o must be added explicitly.
USER_OBJS := fd.o null.o pty.o tty.o xterm.o slip_common.o vde_user.o vector_user.o
diff --git a/arch/um/drivers/virt-pci.c b/arch/um/drivers/virt-pci.c
index dd5580f975cc..b83b5a765d4e 100644
--- a/arch/um/drivers/virt-pci.c
+++ b/arch/um/drivers/virt-pci.c
@@ -5,52 +5,19 @@
*/
#include <linux/module.h>
#include <linux/pci.h>
-#include <linux/virtio.h>
-#include <linux/virtio_config.h>
#include <linux/logic_iomem.h>
#include <linux/of_platform.h>
#include <linux/irqdomain.h>
-#include <linux/virtio_pcidev.h>
-#include <linux/virtio-uml.h>
-#include <linux/delay.h>
#include <linux/msi.h>
#include <linux/unaligned.h>
#include <irq_kern.h>
+#include "virt-pci.h"
+
#define MAX_DEVICES 8
#define MAX_MSI_VECTORS 32
#define CFG_SPACE_SIZE 4096
-/* for MSI-X we have a 32-bit payload */
-#define MAX_IRQ_MSG_SIZE (sizeof(struct virtio_pcidev_msg) + sizeof(u32))
-#define NUM_IRQ_MSGS 10
-
-struct um_pci_message_buffer {
- struct virtio_pcidev_msg hdr;
- u8 data[8];
-};
-
-struct um_pci_device {
- struct virtio_device *vdev;
-
- /* for now just standard BARs */
- u8 resptr[PCI_STD_NUM_BARS];
-
- struct virtqueue *cmd_vq, *irq_vq;
-
-#define UM_PCI_WRITE_BUFS 20
- struct um_pci_message_buffer bufs[UM_PCI_WRITE_BUFS + 1];
- void *extra_ptrs[UM_PCI_WRITE_BUFS + 1];
- DECLARE_BITMAP(used_bufs, UM_PCI_WRITE_BUFS);
-
-#define UM_PCI_STAT_WAITING 0
- unsigned long status;
-
- int irq;
-
- bool platform;
-};
-
struct um_pci_device_reg {
struct um_pci_device *dev;
void __iomem *iomem;
@@ -65,179 +32,15 @@ static struct irq_domain *um_pci_inner_domain;
static struct irq_domain *um_pci_msi_domain;
static unsigned long um_pci_msi_used[BITS_TO_LONGS(MAX_MSI_VECTORS)];
-static unsigned int um_pci_max_delay_us = 40000;
-module_param_named(max_delay_us, um_pci_max_delay_us, uint, 0644);
-
-static int um_pci_get_buf(struct um_pci_device *dev, bool *posted)
-{
- int i;
-
- for (i = 0; i < UM_PCI_WRITE_BUFS; i++) {
- if (!test_and_set_bit(i, dev->used_bufs))
- return i;
- }
-
- *posted = false;
- return UM_PCI_WRITE_BUFS;
-}
-
-static void um_pci_free_buf(struct um_pci_device *dev, void *buf)
-{
- int i;
-
- if (buf == &dev->bufs[UM_PCI_WRITE_BUFS]) {
- kfree(dev->extra_ptrs[UM_PCI_WRITE_BUFS]);
- dev->extra_ptrs[UM_PCI_WRITE_BUFS] = NULL;
- return;
- }
-
- for (i = 0; i < UM_PCI_WRITE_BUFS; i++) {
- if (buf == &dev->bufs[i]) {
- kfree(dev->extra_ptrs[i]);
- dev->extra_ptrs[i] = NULL;
- WARN_ON(!test_and_clear_bit(i, dev->used_bufs));
- return;
- }
- }
-
- WARN_ON(1);
-}
-
-static int um_pci_send_cmd(struct um_pci_device *dev,
- struct virtio_pcidev_msg *cmd,
- unsigned int cmd_size,
- const void *extra, unsigned int extra_size,
- void *out, unsigned int out_size)
-{
- struct scatterlist out_sg, extra_sg, in_sg;
- struct scatterlist *sgs_list[] = {
- [0] = &out_sg,
- [1] = extra ? &extra_sg : &in_sg,
- [2] = extra ? &in_sg : NULL,
- };
- struct um_pci_message_buffer *buf;
- int delay_count = 0;
- bool bounce_out;
- int ret, len;
- int buf_idx;
- bool posted;
-
- if (WARN_ON(cmd_size < sizeof(*cmd) || cmd_size > sizeof(*buf)))
- return -EINVAL;
-
- switch (cmd->op) {
- case VIRTIO_PCIDEV_OP_CFG_WRITE:
- case VIRTIO_PCIDEV_OP_MMIO_WRITE:
- case VIRTIO_PCIDEV_OP_MMIO_MEMSET:
- /* in PCI, writes are posted, so don't wait */
- posted = !out;
- WARN_ON(!posted);
- break;
- default:
- posted = false;
- break;
- }
-
- bounce_out = !posted && cmd_size <= sizeof(*cmd) &&
- out && out_size <= sizeof(buf->data);
-
- buf_idx = um_pci_get_buf(dev, &posted);
- buf = &dev->bufs[buf_idx];
- memcpy(buf, cmd, cmd_size);
-
- if (posted && extra && extra_size > sizeof(buf) - cmd_size) {
- dev->extra_ptrs[buf_idx] = kmemdup(extra, extra_size,
- GFP_ATOMIC);
-
- if (!dev->extra_ptrs[buf_idx]) {
- um_pci_free_buf(dev, buf);
- return -ENOMEM;
- }
- extra = dev->extra_ptrs[buf_idx];
- } else if (extra && extra_size <= sizeof(buf) - cmd_size) {
- memcpy((u8 *)buf + cmd_size, extra, extra_size);
- cmd_size += extra_size;
- extra_size = 0;
- extra = NULL;
- cmd = (void *)buf;
- } else {
- cmd = (void *)buf;
- }
-
- sg_init_one(&out_sg, cmd, cmd_size);
- if (extra)
- sg_init_one(&extra_sg, extra, extra_size);
- /* allow stack for small buffers */
- if (bounce_out)
- sg_init_one(&in_sg, buf->data, out_size);
- else if (out)
- sg_init_one(&in_sg, out, out_size);
-
- /* add to internal virtio queue */
- ret = virtqueue_add_sgs(dev->cmd_vq, sgs_list,
- extra ? 2 : 1,
- out ? 1 : 0,
- cmd, GFP_ATOMIC);
- if (ret) {
- um_pci_free_buf(dev, buf);
- return ret;
- }
-
- if (posted) {
- virtqueue_kick(dev->cmd_vq);
- return 0;
- }
-
- /* kick and poll for getting a response on the queue */
- set_bit(UM_PCI_STAT_WAITING, &dev->status);
- virtqueue_kick(dev->cmd_vq);
- ret = 0;
-
- while (1) {
- void *completed = virtqueue_get_buf(dev->cmd_vq, &len);
-
- if (completed == buf)
- break;
-
- if (completed)
- um_pci_free_buf(dev, completed);
-
- if (WARN_ONCE(virtqueue_is_broken(dev->cmd_vq) ||
- ++delay_count > um_pci_max_delay_us,
- "um virt-pci delay: %d", delay_count)) {
- ret = -EIO;
- break;
- }
- udelay(1);
- }
- clear_bit(UM_PCI_STAT_WAITING, &dev->status);
-
- if (bounce_out)
- memcpy(out, buf->data, out_size);
-
- um_pci_free_buf(dev, buf);
-
- return ret;
-}
-
static unsigned long um_pci_cfgspace_read(void *priv, unsigned int offset,
int size)
{
struct um_pci_device_reg *reg = priv;
struct um_pci_device *dev = reg->dev;
- struct virtio_pcidev_msg hdr = {
- .op = VIRTIO_PCIDEV_OP_CFG_READ,
- .size = size,
- .addr = offset,
- };
- /* max 8, we might not use it all */
- u8 data[8];
if (!dev)
return ULONG_MAX;
- memset(data, 0xff, sizeof(data));
-
switch (size) {
case 1:
case 2:
@@ -251,23 +54,7 @@ static unsigned long um_pci_cfgspace_read(void *priv, unsigned int offset,
return ULONG_MAX;
}
- if (um_pci_send_cmd(dev, &hdr, sizeof(hdr), NULL, 0, data, size))
- return ULONG_MAX;
-
- switch (size) {
- case 1:
- return data[0];
- case 2:
- return le16_to_cpup((void *)data);
- case 4:
- return le32_to_cpup((void *)data);
-#ifdef CONFIG_64BIT
- case 8:
- return le64_to_cpup((void *)data);
-#endif
- default:
- return ULONG_MAX;
- }
+ return dev->ops->cfgspace_read(dev, offset, size);
}
static void um_pci_cfgspace_write(void *priv, unsigned int offset, int size,
@@ -275,42 +62,24 @@ static void um_pci_cfgspace_write(void *priv, unsigned int offset, int size,
{
struct um_pci_device_reg *reg = priv;
struct um_pci_device *dev = reg->dev;
- struct {
- struct virtio_pcidev_msg hdr;
- /* maximum size - we may only use parts of it */
- u8 data[8];
- } msg = {
- .hdr = {
- .op = VIRTIO_PCIDEV_OP_CFG_WRITE,
- .size = size,
- .addr = offset,
- },
- };
if (!dev)
return;
switch (size) {
case 1:
- msg.data[0] = (u8)val;
- break;
case 2:
- put_unaligned_le16(val, (void *)msg.data);
- break;
case 4:
- put_unaligned_le32(val, (void *)msg.data);
- break;
#ifdef CONFIG_64BIT
case 8:
- put_unaligned_le64(val, (void *)msg.data);
- break;
#endif
+ break;
default:
WARN(1, "invalid config space write size %d\n", size);
return;
}
- WARN_ON(um_pci_send_cmd(dev, &msg.hdr, sizeof(msg), NULL, 0, NULL, 0));
+ dev->ops->cfgspace_write(dev, offset, size, val);
}
static const struct logic_iomem_ops um_pci_device_cfgspace_ops = {
@@ -318,30 +87,14 @@ static const struct logic_iomem_ops um_pci_device_cfgspace_ops = {
.write = um_pci_cfgspace_write,
};
-static void um_pci_bar_copy_from(void *priv, void *buffer,
- unsigned int offset, int size)
+static unsigned long um_pci_bar_read(void *priv, unsigned int offset,
+ int size)
{
u8 *resptr = priv;
struct um_pci_device *dev = container_of(resptr - *resptr,
struct um_pci_device,
resptr[0]);
- struct virtio_pcidev_msg hdr = {
- .op = VIRTIO_PCIDEV_OP_MMIO_READ,
- .bar = *resptr,
- .size = size,
- .addr = offset,
- };
-
- memset(buffer, 0xff, size);
-
- um_pci_send_cmd(dev, &hdr, sizeof(hdr), NULL, 0, buffer, size);
-}
-
-static unsigned long um_pci_bar_read(void *priv, unsigned int offset,
- int size)
-{
- /* 8 is maximum size - we may only use parts of it */
- u8 data[8];
+ u8 bar = *resptr;
switch (size) {
case 1:
@@ -352,72 +105,60 @@ static unsigned long um_pci_bar_read(void *priv, unsigned int offset,
#endif
break;
default:
- WARN(1, "invalid config space read size %d\n", size);
+ WARN(1, "invalid bar read size %d\n", size);
return ULONG_MAX;
}
- um_pci_bar_copy_from(priv, data, offset, size);
+ return dev->ops->bar_read(dev, bar, offset, size);
+}
+
+static void um_pci_bar_write(void *priv, unsigned int offset, int size,
+ unsigned long val)
+{
+ u8 *resptr = priv;
+ struct um_pci_device *dev = container_of(resptr - *resptr,
+ struct um_pci_device,
+ resptr[0]);
+ u8 bar = *resptr;
switch (size) {
case 1:
- return data[0];
case 2:
- return le16_to_cpup((void *)data);
case 4:
- return le32_to_cpup((void *)data);
#ifdef CONFIG_64BIT
case 8:
- return le64_to_cpup((void *)data);
#endif
+ break;
default:
- return ULONG_MAX;
+ WARN(1, "invalid bar write size %d\n", size);
+ return;
}
+
+ dev->ops->bar_write(dev, bar, offset, size, val);
}
-static void um_pci_bar_copy_to(void *priv, unsigned int offset,
- const void *buffer, int size)
+static void um_pci_bar_copy_from(void *priv, void *buffer,
+ unsigned int offset, int size)
{
u8 *resptr = priv;
struct um_pci_device *dev = container_of(resptr - *resptr,
struct um_pci_device,
resptr[0]);
- struct virtio_pcidev_msg hdr = {
- .op = VIRTIO_PCIDEV_OP_MMIO_WRITE,
- .bar = *resptr,
- .size = size,
- .addr = offset,
- };
+ u8 bar = *resptr;
- um_pci_send_cmd(dev, &hdr, sizeof(hdr), buffer, size, NULL, 0);
+ dev->ops->bar_copy_from(dev, bar, buffer, offset, size);
}
-static void um_pci_bar_write(void *priv, unsigned int offset, int size,
- unsigned long val)
+static void um_pci_bar_copy_to(void *priv, unsigned int offset,
+ const void *buffer, int size)
{
- /* maximum size - we may only use parts of it */
- u8 data[8];
-
- switch (size) {
- case 1:
- data[0] = (u8)val;
- break;
- case 2:
- put_unaligned_le16(val, (void *)data);
- break;
- case 4:
- put_unaligned_le32(val, (void *)data);
- break;
-#ifdef CONFIG_64BIT
- case 8:
- put_unaligned_le64(val, (void *)data);
- break;
-#endif
- default:
- WARN(1, "invalid config space write size %d\n", size);
- return;
- }
+ u8 *resptr = priv;
+ struct um_pci_device *dev = container_of(resptr - *resptr,
+ struct um_pci_device,
+ resptr[0]);
+ u8 bar = *resptr;
- um_pci_bar_copy_to(priv, offset, data, size);
+ dev->ops->bar_copy_to(dev, bar, offset, buffer, size);
}
static void um_pci_bar_set(void *priv, unsigned int offset, u8 value, int size)
@@ -426,20 +167,9 @@ static void um_pci_bar_set(void *priv, unsigned int offset, u8 value, int size)
struct um_pci_device *dev = container_of(resptr - *resptr,
struct um_pci_device,
resptr[0]);
- struct {
- struct virtio_pcidev_msg hdr;
- u8 data;
- } msg = {
- .hdr = {
- .op = VIRTIO_PCIDEV_OP_CFG_WRITE,
- .bar = *resptr,
- .size = size,
- .addr = offset,
- },
- .data = value,
- };
+ u8 bar = *resptr;
- um_pci_send_cmd(dev, &msg.hdr, sizeof(msg), NULL, 0, NULL, 0);
+ dev->ops->bar_set(dev, bar, offset, value, size);
}
static const struct logic_iomem_ops um_pci_device_bar_ops = {
@@ -486,76 +216,6 @@ static void um_pci_rescan(void)
pci_unlock_rescan_remove();
}
-static void um_pci_irq_vq_addbuf(struct virtqueue *vq, void *buf, bool kick)
-{
- struct scatterlist sg[1];
-
- sg_init_one(sg, buf, MAX_IRQ_MSG_SIZE);
- if (virtqueue_add_inbuf(vq, sg, 1, buf, GFP_ATOMIC))
- kfree(buf);
- else if (kick)
- virtqueue_kick(vq);
-}
-
-static void um_pci_handle_irq_message(struct virtqueue *vq,
- struct virtio_pcidev_msg *msg)
-{
- struct virtio_device *vdev = vq->vdev;
- struct um_pci_device *dev = vdev->priv;
-
- if (!dev->irq)
- return;
-
- /* we should properly chain interrupts, but on ARCH=um we don't care */
-
- switch (msg->op) {
- case VIRTIO_PCIDEV_OP_INT:
- generic_handle_irq(dev->irq);
- break;
- case VIRTIO_PCIDEV_OP_MSI:
- /* our MSI message is just the interrupt number */
- if (msg->size == sizeof(u32))
- generic_handle_irq(le32_to_cpup((void *)msg->data));
- else
- generic_handle_irq(le16_to_cpup((void *)msg->data));
- break;
- case VIRTIO_PCIDEV_OP_PME:
- /* nothing to do - we already woke up due to the message */
- break;
- default:
- dev_err(&vdev->dev, "unexpected virt-pci message %d\n", msg->op);
- break;
- }
-}
-
-static void um_pci_cmd_vq_cb(struct virtqueue *vq)
-{
- struct virtio_device *vdev = vq->vdev;
- struct um_pci_device *dev = vdev->priv;
- void *cmd;
- int len;
-
- if (test_bit(UM_PCI_STAT_WAITING, &dev->status))
- return;
-
- while ((cmd = virtqueue_get_buf(vq, &len)))
- um_pci_free_buf(dev, cmd);
-}
-
-static void um_pci_irq_vq_cb(struct virtqueue *vq)
-{
- struct virtio_pcidev_msg *msg;
- int len;
-
- while ((msg = virtqueue_get_buf(vq, &len))) {
- if (len >= sizeof(*msg))
- um_pci_handle_irq_message(vq, msg);
-
- /* recycle the message buffer */
- um_pci_irq_vq_addbuf(vq, msg, true);
- }
-}
-
#ifdef CONFIG_OF
/* Copied from arch/x86/kernel/devicetree.c */
struct device_node *pcibios_get_phb_of_node(struct pci_bus *bus)
@@ -577,200 +237,6 @@ struct device_node *pcibios_get_phb_of_node(struct pci_bus *bus)
}
#endif
-static int um_pci_init_vqs(struct um_pci_device *dev)
-{
- struct virtqueue_info vqs_info[] = {
- { "cmd", um_pci_cmd_vq_cb },
- { "irq", um_pci_irq_vq_cb },
- };
- struct virtqueue *vqs[2];
- int err, i;
-
- err = virtio_find_vqs(dev->vdev, 2, vqs, vqs_info, NULL);
- if (err)
- return err;
-
- dev->cmd_vq = vqs[0];
- dev->irq_vq = vqs[1];
-
- virtio_device_ready(dev->vdev);
-
- for (i = 0; i < NUM_IRQ_MSGS; i++) {
- void *msg = kzalloc(MAX_IRQ_MSG_SIZE, GFP_KERNEL);
-
- if (msg)
- um_pci_irq_vq_addbuf(dev->irq_vq, msg, false);
- }
-
- virtqueue_kick(dev->irq_vq);
-
- return 0;
-}
-
-static void __um_pci_virtio_platform_remove(struct virtio_device *vdev,
- struct um_pci_device *dev)
-{
- virtio_reset_device(vdev);
- vdev->config->del_vqs(vdev);
-
- mutex_lock(&um_pci_mtx);
- um_pci_platform_device = NULL;
- mutex_unlock(&um_pci_mtx);
-
- kfree(dev);
-}
-
-static int um_pci_virtio_platform_probe(struct virtio_device *vdev,
- struct um_pci_device *dev)
-{
- int ret;
-
- dev->platform = true;
-
- mutex_lock(&um_pci_mtx);
-
- if (um_pci_platform_device) {
- mutex_unlock(&um_pci_mtx);
- ret = -EBUSY;
- goto out_free;
- }
-
- ret = um_pci_init_vqs(dev);
- if (ret) {
- mutex_unlock(&um_pci_mtx);
- goto out_free;
- }
-
- um_pci_platform_device = dev;
-
- mutex_unlock(&um_pci_mtx);
-
- ret = of_platform_default_populate(vdev->dev.of_node, NULL, &vdev->dev);
- if (ret)
- __um_pci_virtio_platform_remove(vdev, dev);
-
- return ret;
-
-out_free:
- kfree(dev);
- return ret;
-}
-
-static int um_pci_virtio_probe(struct virtio_device *vdev)
-{
- struct um_pci_device *dev;
- int i, free = -1;
- int err = -ENOSPC;
-
- dev = kzalloc(sizeof(*dev), GFP_KERNEL);
- if (!dev)
- return -ENOMEM;
-
- dev->vdev = vdev;
- vdev->priv = dev;
-
- if (of_device_is_compatible(vdev->dev.of_node, "simple-bus"))
- return um_pci_virtio_platform_probe(vdev, dev);
-
- mutex_lock(&um_pci_mtx);
- for (i = 0; i < MAX_DEVICES; i++) {
- if (um_pci_devices[i].dev)
- continue;
- free = i;
- break;
- }
-
- if (free < 0)
- goto error;
-
- err = um_pci_init_vqs(dev);
- if (err)
- goto error;
-
- dev->irq = irq_alloc_desc(numa_node_id());
- if (dev->irq < 0) {
- err = dev->irq;
- goto err_reset;
- }
- um_pci_devices[free].dev = dev;
- vdev->priv = dev;
-
- mutex_unlock(&um_pci_mtx);
-
- device_set_wakeup_enable(&vdev->dev, true);
-
- /*
- * In order to do suspend-resume properly, don't allow VQs
- * to be suspended.
- */
- virtio_uml_set_no_vq_suspend(vdev, true);
-
- um_pci_rescan();
- return 0;
-err_reset:
- virtio_reset_device(vdev);
- vdev->config->del_vqs(vdev);
-error:
- mutex_unlock(&um_pci_mtx);
- kfree(dev);
- return err;
-}
-
-static void um_pci_virtio_remove(struct virtio_device *vdev)
-{
- struct um_pci_device *dev = vdev->priv;
- int i;
-
- if (dev->platform) {
- of_platform_depopulate(&vdev->dev);
- __um_pci_virtio_platform_remove(vdev, dev);
- return;
- }
-
- device_set_wakeup_enable(&vdev->dev, false);
-
- mutex_lock(&um_pci_mtx);
- for (i = 0; i < MAX_DEVICES; i++) {
- if (um_pci_devices[i].dev != dev)
- continue;
-
- um_pci_devices[i].dev = NULL;
- irq_free_desc(dev->irq);
-
- break;
- }
- mutex_unlock(&um_pci_mtx);
-
- if (i < MAX_DEVICES) {
- struct pci_dev *pci_dev;
-
- pci_dev = pci_get_slot(bridge->bus, i);
- if (pci_dev)
- pci_stop_and_remove_bus_device_locked(pci_dev);
- }
-
- /* Stop all virtqueues */
- virtio_reset_device(vdev);
- dev->cmd_vq = NULL;
- dev->irq_vq = NULL;
- vdev->config->del_vqs(vdev);
-
- kfree(dev);
-}
-
-static struct virtio_device_id id_table[] = {
- { CONFIG_UML_PCI_OVER_VIRTIO_DEVICE_ID, VIRTIO_DEV_ANY_ID },
- { 0 },
-};
-MODULE_DEVICE_TABLE(virtio, id_table);
-
-static struct virtio_driver um_pci_virtio_driver = {
- .driver.name = "virtio-pci",
- .id_table = id_table,
- .probe = um_pci_virtio_probe,
- .remove = um_pci_virtio_remove,
-};
-
static struct resource virt_cfgspace_resource = {
.name = "PCI config space",
.start = 0xf0000000 - MAX_DEVICES * CFG_SPACE_SIZE,
@@ -889,7 +355,7 @@ static void um_pci_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
}
static struct irq_chip um_pci_msi_bottom_irq_chip = {
- .name = "UM virtio MSI",
+ .name = "UM virtual MSI",
.irq_compose_msi_msg = um_pci_compose_msi_msg,
};
@@ -939,7 +405,7 @@ static const struct irq_domain_ops um_pci_inner_domain_ops = {
};
static struct irq_chip um_pci_msi_irq_chip = {
- .name = "UM virtio PCIe MSI",
+ .name = "UM virtual PCIe MSI",
.irq_mask = pci_msi_mask_irq,
.irq_unmask = pci_msi_unmask_irq,
};
@@ -998,6 +464,78 @@ static struct resource virt_platform_resource = {
.flags = IORESOURCE_MEM,
};
+int um_pci_device_register(struct um_pci_device *dev)
+{
+ int i, free = -1;
+ int err = 0;
+
+ mutex_lock(&um_pci_mtx);
+ for (i = 0; i < MAX_DEVICES; i++) {
+ if (um_pci_devices[i].dev)
+ continue;
+ free = i;
+ break;
+ }
+
+ if (free < 0) {
+ err = -ENOSPC;
+ goto out;
+ }
+
+ dev->irq = irq_alloc_desc(numa_node_id());
+ if (dev->irq < 0) {
+ err = dev->irq;
+ goto out;
+ }
+
+ um_pci_devices[free].dev = dev;
+
+out:
+ mutex_unlock(&um_pci_mtx);
+ if (!err)
+ um_pci_rescan();
+ return err;
+}
+
+void um_pci_device_unregister(struct um_pci_device *dev)
+{
+ int i;
+
+ mutex_lock(&um_pci_mtx);
+ for (i = 0; i < MAX_DEVICES; i++) {
+ if (um_pci_devices[i].dev != dev)
+ continue;
+ um_pci_devices[i].dev = NULL;
+ irq_free_desc(dev->irq);
+ break;
+ }
+ mutex_unlock(&um_pci_mtx);
+
+ if (i < MAX_DEVICES) {
+ struct pci_dev *pci_dev;
+
+ pci_dev = pci_get_slot(bridge->bus, i);
+ if (pci_dev)
+ pci_stop_and_remove_bus_device_locked(pci_dev);
+ }
+}
+
+int um_pci_platform_device_register(struct um_pci_device *dev)
+{
+ guard(mutex)(&um_pci_mtx);
+ if (um_pci_platform_device)
+ return -EBUSY;
+ um_pci_platform_device = dev;
+ return 0;
+}
+
+void um_pci_platform_device_unregister(struct um_pci_device *dev)
+{
+ guard(mutex)(&um_pci_mtx);
+ if (um_pci_platform_device == dev)
+ um_pci_platform_device = NULL;
+}
+
static int __init um_pci_init(void)
{
struct irq_domain_info inner_domain_info = {
@@ -1014,10 +552,6 @@ static int __init um_pci_init(void)
WARN_ON(logic_iomem_add_region(&virt_platform_resource,
&um_pci_platform_ops));
- if (WARN(CONFIG_UML_PCI_OVER_VIRTIO_DEVICE_ID < 0,
- "No virtio device ID configured for PCI - no PCI support\n"))
- return 0;
-
bridge = pci_alloc_host_bridge(0);
if (!bridge) {
err = -ENOMEM;
@@ -1065,10 +599,8 @@ static int __init um_pci_init(void)
if (err)
goto free;
- err = register_virtio_driver(&um_pci_virtio_driver);
- if (err)
- goto free;
return 0;
+
free:
if (!IS_ERR_OR_NULL(um_pci_inner_domain))
irq_domain_remove(um_pci_inner_domain);
@@ -1080,11 +612,10 @@ static int __init um_pci_init(void)
}
return err;
}
-module_init(um_pci_init);
+device_initcall(um_pci_init);
static void __exit um_pci_exit(void)
{
- unregister_virtio_driver(&um_pci_virtio_driver);
irq_domain_remove(um_pci_msi_domain);
irq_domain_remove(um_pci_inner_domain);
pci_free_resource_list(&bridge->windows);
diff --git a/arch/um/drivers/virt-pci.h b/arch/um/drivers/virt-pci.h
new file mode 100644
index 000000000000..b20d1475d1eb
--- /dev/null
+++ b/arch/um/drivers/virt-pci.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_VIRT_PCI_H
+#define __UM_VIRT_PCI_H
+
+#include <linux/pci.h>
+
+struct um_pci_device {
+ const struct um_pci_ops *ops;
+
+ /* for now just standard BARs */
+ u8 resptr[PCI_STD_NUM_BARS];
+
+ int irq;
+};
+
+struct um_pci_ops {
+ unsigned long (*cfgspace_read)(struct um_pci_device *dev,
+ unsigned int offset, int size);
+ void (*cfgspace_write)(struct um_pci_device *dev, unsigned int offset,
+ int size, unsigned long val);
+
+ unsigned long (*bar_read)(struct um_pci_device *dev, int bar,
+ unsigned int offset, int size);
+ void (*bar_write)(struct um_pci_device *dev, int bar,
+ unsigned int offset, int size, unsigned long val);
+
+ void (*bar_copy_from)(struct um_pci_device *dev, int bar, void *buffer,
+ unsigned int offset, int size);
+ void (*bar_copy_to)(struct um_pci_device *dev, int bar,
+ unsigned int offset, const void *buffer, int size);
+ void (*bar_set)(struct um_pci_device *dev, int bar,
+ unsigned int offset, u8 value, int size);
+};
+
+int um_pci_device_register(struct um_pci_device *dev);
+void um_pci_device_unregister(struct um_pci_device *dev);
+
+int um_pci_platform_device_register(struct um_pci_device *dev);
+void um_pci_platform_device_unregister(struct um_pci_device *dev);
+
+#endif /* __UM_VIRT_PCI_H */
diff --git a/arch/um/drivers/virtio_pcidev.c b/arch/um/drivers/virtio_pcidev.c
new file mode 100644
index 000000000000..3c4c4c928fdd
--- /dev/null
+++ b/arch/um/drivers/virtio_pcidev.c
@@ -0,0 +1,628 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020 Intel Corporation
+ * Author: Johannes Berg <johannes@sipsolutions.net>
+ */
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/virtio.h>
+#include <linux/virtio_config.h>
+#include <linux/logic_iomem.h>
+#include <linux/of_platform.h>
+#include <linux/irqdomain.h>
+#include <linux/virtio_pcidev.h>
+#include <linux/virtio-uml.h>
+#include <linux/delay.h>
+#include <linux/msi.h>
+#include <linux/unaligned.h>
+#include <irq_kern.h>
+
+#include "virt-pci.h"
+
+#define to_virtio_pcidev(_pdev) \
+ container_of(_pdev, struct virtio_pcidev_device, pdev)
+
+/* for MSI-X we have a 32-bit payload */
+#define MAX_IRQ_MSG_SIZE (sizeof(struct virtio_pcidev_msg) + sizeof(u32))
+#define NUM_IRQ_MSGS 10
+
+struct virtio_pcidev_message_buffer {
+ struct virtio_pcidev_msg hdr;
+ u8 data[8];
+};
+
+struct virtio_pcidev_device {
+ struct um_pci_device pdev;
+ struct virtio_device *vdev;
+
+ struct virtqueue *cmd_vq, *irq_vq;
+
+#define VIRTIO_PCIDEV_WRITE_BUFS 20
+ struct virtio_pcidev_message_buffer bufs[VIRTIO_PCIDEV_WRITE_BUFS + 1];
+ void *extra_ptrs[VIRTIO_PCIDEV_WRITE_BUFS + 1];
+ DECLARE_BITMAP(used_bufs, VIRTIO_PCIDEV_WRITE_BUFS);
+
+#define UM_PCI_STAT_WAITING 0
+ unsigned long status;
+
+ bool platform;
+};
+
+static unsigned int virtio_pcidev_max_delay_us = 40000;
+module_param_named(max_delay_us, virtio_pcidev_max_delay_us, uint, 0644);
+
+static int virtio_pcidev_get_buf(struct virtio_pcidev_device *dev, bool *posted)
+{
+ int i;
+
+ for (i = 0; i < VIRTIO_PCIDEV_WRITE_BUFS; i++) {
+ if (!test_and_set_bit(i, dev->used_bufs))
+ return i;
+ }
+
+ *posted = false;
+ return VIRTIO_PCIDEV_WRITE_BUFS;
+}
+
+static void virtio_pcidev_free_buf(struct virtio_pcidev_device *dev, void *buf)
+{
+ int i;
+
+ if (buf == &dev->bufs[VIRTIO_PCIDEV_WRITE_BUFS]) {
+ kfree(dev->extra_ptrs[VIRTIO_PCIDEV_WRITE_BUFS]);
+ dev->extra_ptrs[VIRTIO_PCIDEV_WRITE_BUFS] = NULL;
+ return;
+ }
+
+ for (i = 0; i < VIRTIO_PCIDEV_WRITE_BUFS; i++) {
+ if (buf == &dev->bufs[i]) {
+ kfree(dev->extra_ptrs[i]);
+ dev->extra_ptrs[i] = NULL;
+ WARN_ON(!test_and_clear_bit(i, dev->used_bufs));
+ return;
+ }
+ }
+
+ WARN_ON(1);
+}
+
+static int virtio_pcidev_send_cmd(struct virtio_pcidev_device *dev,
+ struct virtio_pcidev_msg *cmd,
+ unsigned int cmd_size,
+ const void *extra, unsigned int extra_size,
+ void *out, unsigned int out_size)
+{
+ struct scatterlist out_sg, extra_sg, in_sg;
+ struct scatterlist *sgs_list[] = {
+ [0] = &out_sg,
+ [1] = extra ? &extra_sg : &in_sg,
+ [2] = extra ? &in_sg : NULL,
+ };
+ struct virtio_pcidev_message_buffer *buf;
+ int delay_count = 0;
+ bool bounce_out;
+ int ret, len;
+ int buf_idx;
+ bool posted;
+
+ if (WARN_ON(cmd_size < sizeof(*cmd) || cmd_size > sizeof(*buf)))
+ return -EINVAL;
+
+ switch (cmd->op) {
+ case VIRTIO_PCIDEV_OP_CFG_WRITE:
+ case VIRTIO_PCIDEV_OP_MMIO_WRITE:
+ case VIRTIO_PCIDEV_OP_MMIO_MEMSET:
+ /* in PCI, writes are posted, so don't wait */
+ posted = !out;
+ WARN_ON(!posted);
+ break;
+ default:
+ posted = false;
+ break;
+ }
+
+ bounce_out = !posted && cmd_size <= sizeof(*cmd) &&
+ out && out_size <= sizeof(buf->data);
+
+ buf_idx = virtio_pcidev_get_buf(dev, &posted);
+ buf = &dev->bufs[buf_idx];
+ memcpy(buf, cmd, cmd_size);
+
+ if (posted && extra && extra_size > sizeof(buf) - cmd_size) {
+ dev->extra_ptrs[buf_idx] = kmemdup(extra, extra_size,
+ GFP_ATOMIC);
+
+ if (!dev->extra_ptrs[buf_idx]) {
+ virtio_pcidev_free_buf(dev, buf);
+ return -ENOMEM;
+ }
+ extra = dev->extra_ptrs[buf_idx];
+ } else if (extra && extra_size <= sizeof(buf) - cmd_size) {
+ memcpy((u8 *)buf + cmd_size, extra, extra_size);
+ cmd_size += extra_size;
+ extra_size = 0;
+ extra = NULL;
+ cmd = (void *)buf;
+ } else {
+ cmd = (void *)buf;
+ }
+
+ sg_init_one(&out_sg, cmd, cmd_size);
+ if (extra)
+ sg_init_one(&extra_sg, extra, extra_size);
+ /* allow stack for small buffers */
+ if (bounce_out)
+ sg_init_one(&in_sg, buf->data, out_size);
+ else if (out)
+ sg_init_one(&in_sg, out, out_size);
+
+ /* add to internal virtio queue */
+ ret = virtqueue_add_sgs(dev->cmd_vq, sgs_list,
+ extra ? 2 : 1,
+ out ? 1 : 0,
+ cmd, GFP_ATOMIC);
+ if (ret) {
+ virtio_pcidev_free_buf(dev, buf);
+ return ret;
+ }
+
+ if (posted) {
+ virtqueue_kick(dev->cmd_vq);
+ return 0;
+ }
+
+ /* kick and poll for getting a response on the queue */
+ set_bit(UM_PCI_STAT_WAITING, &dev->status);
+ virtqueue_kick(dev->cmd_vq);
+ ret = 0;
+
+ while (1) {
+ void *completed = virtqueue_get_buf(dev->cmd_vq, &len);
+
+ if (completed == buf)
+ break;
+
+ if (completed)
+ virtio_pcidev_free_buf(dev, completed);
+
+ if (WARN_ONCE(virtqueue_is_broken(dev->cmd_vq) ||
+ ++delay_count > virtio_pcidev_max_delay_us,
+ "um virt-pci delay: %d", delay_count)) {
+ ret = -EIO;
+ break;
+ }
+ udelay(1);
+ }
+ clear_bit(UM_PCI_STAT_WAITING, &dev->status);
+
+ if (bounce_out)
+ memcpy(out, buf->data, out_size);
+
+ virtio_pcidev_free_buf(dev, buf);
+
+ return ret;
+}
+
+static unsigned long virtio_pcidev_cfgspace_read(struct um_pci_device *pdev,
+ unsigned int offset, int size)
+{
+ struct virtio_pcidev_device *dev = to_virtio_pcidev(pdev);
+ struct virtio_pcidev_msg hdr = {
+ .op = VIRTIO_PCIDEV_OP_CFG_READ,
+ .size = size,
+ .addr = offset,
+ };
+ /* max 8, we might not use it all */
+ u8 data[8];
+
+ memset(data, 0xff, sizeof(data));
+
+ /* size has been checked in um_pci_cfgspace_read() */
+ if (virtio_pcidev_send_cmd(dev, &hdr, sizeof(hdr), NULL, 0, data, size))
+ return ULONG_MAX;
+
+ switch (size) {
+ case 1:
+ return data[0];
+ case 2:
+ return le16_to_cpup((void *)data);
+ case 4:
+ return le32_to_cpup((void *)data);
+#ifdef CONFIG_64BIT
+ case 8:
+ return le64_to_cpup((void *)data);
+#endif
+ default:
+ return ULONG_MAX;
+ }
+}
+
+static void virtio_pcidev_cfgspace_write(struct um_pci_device *pdev,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ struct virtio_pcidev_device *dev = to_virtio_pcidev(pdev);
+ struct {
+ struct virtio_pcidev_msg hdr;
+ /* maximum size - we may only use parts of it */
+ u8 data[8];
+ } msg = {
+ .hdr = {
+ .op = VIRTIO_PCIDEV_OP_CFG_WRITE,
+ .size = size,
+ .addr = offset,
+ },
+ };
+
+ /* size has been checked in um_pci_cfgspace_write() */
+ switch (size) {
+ case 1:
+ msg.data[0] = (u8)val;
+ break;
+ case 2:
+ put_unaligned_le16(val, (void *)msg.data);
+ break;
+ case 4:
+ put_unaligned_le32(val, (void *)msg.data);
+ break;
+#ifdef CONFIG_64BIT
+ case 8:
+ put_unaligned_le64(val, (void *)msg.data);
+ break;
+#endif
+ }
+
+ WARN_ON(virtio_pcidev_send_cmd(dev, &msg.hdr, sizeof(msg), NULL, 0, NULL, 0));
+}
+
+static void virtio_pcidev_bar_copy_from(struct um_pci_device *pdev,
+ int bar, void *buffer,
+ unsigned int offset, int size)
+{
+ struct virtio_pcidev_device *dev = to_virtio_pcidev(pdev);
+ struct virtio_pcidev_msg hdr = {
+ .op = VIRTIO_PCIDEV_OP_MMIO_READ,
+ .bar = bar,
+ .size = size,
+ .addr = offset,
+ };
+
+ memset(buffer, 0xff, size);
+
+ virtio_pcidev_send_cmd(dev, &hdr, sizeof(hdr), NULL, 0, buffer, size);
+}
+
+static unsigned long virtio_pcidev_bar_read(struct um_pci_device *pdev, int bar,
+ unsigned int offset, int size)
+{
+ /* 8 is maximum size - we may only use parts of it */
+ u8 data[8];
+
+ /* size has been checked in um_pci_bar_read() */
+ virtio_pcidev_bar_copy_from(pdev, bar, data, offset, size);
+
+ switch (size) {
+ case 1:
+ return data[0];
+ case 2:
+ return le16_to_cpup((void *)data);
+ case 4:
+ return le32_to_cpup((void *)data);
+#ifdef CONFIG_64BIT
+ case 8:
+ return le64_to_cpup((void *)data);
+#endif
+ default:
+ return ULONG_MAX;
+ }
+}
+
+static void virtio_pcidev_bar_copy_to(struct um_pci_device *pdev,
+ int bar, unsigned int offset,
+ const void *buffer, int size)
+{
+ struct virtio_pcidev_device *dev = to_virtio_pcidev(pdev);
+ struct virtio_pcidev_msg hdr = {
+ .op = VIRTIO_PCIDEV_OP_MMIO_WRITE,
+ .bar = bar,
+ .size = size,
+ .addr = offset,
+ };
+
+ virtio_pcidev_send_cmd(dev, &hdr, sizeof(hdr), buffer, size, NULL, 0);
+}
+
+static void virtio_pcidev_bar_write(struct um_pci_device *pdev, int bar,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ /* maximum size - we may only use parts of it */
+ u8 data[8];
+
+ /* size has been checked in um_pci_bar_write() */
+ switch (size) {
+ case 1:
+ data[0] = (u8)val;
+ break;
+ case 2:
+ put_unaligned_le16(val, (void *)data);
+ break;
+ case 4:
+ put_unaligned_le32(val, (void *)data);
+ break;
+#ifdef CONFIG_64BIT
+ case 8:
+ put_unaligned_le64(val, (void *)data);
+ break;
+#endif
+ }
+
+ virtio_pcidev_bar_copy_to(pdev, bar, offset, data, size);
+}
+
+static void virtio_pcidev_bar_set(struct um_pci_device *pdev, int bar,
+ unsigned int offset, u8 value, int size)
+{
+ struct virtio_pcidev_device *dev = to_virtio_pcidev(pdev);
+ struct {
+ struct virtio_pcidev_msg hdr;
+ u8 data;
+ } msg = {
+ .hdr = {
+ .op = VIRTIO_PCIDEV_OP_CFG_WRITE,
+ .bar = bar,
+ .size = size,
+ .addr = offset,
+ },
+ .data = value,
+ };
+
+ virtio_pcidev_send_cmd(dev, &msg.hdr, sizeof(msg), NULL, 0, NULL, 0);
+}
+
+static const struct um_pci_ops virtio_pcidev_um_pci_ops = {
+ .cfgspace_read = virtio_pcidev_cfgspace_read,
+ .cfgspace_write = virtio_pcidev_cfgspace_write,
+ .bar_read = virtio_pcidev_bar_read,
+ .bar_write = virtio_pcidev_bar_write,
+ .bar_copy_from = virtio_pcidev_bar_copy_from,
+ .bar_copy_to = virtio_pcidev_bar_copy_to,
+ .bar_set = virtio_pcidev_bar_set,
+};
+
+static void virtio_pcidev_irq_vq_addbuf(struct virtqueue *vq, void *buf, bool kick)
+{
+ struct scatterlist sg[1];
+
+ sg_init_one(sg, buf, MAX_IRQ_MSG_SIZE);
+ if (virtqueue_add_inbuf(vq, sg, 1, buf, GFP_ATOMIC))
+ kfree(buf);
+ else if (kick)
+ virtqueue_kick(vq);
+}
+
+static void virtio_pcidev_handle_irq_message(struct virtqueue *vq,
+ struct virtio_pcidev_msg *msg)
+{
+ struct virtio_device *vdev = vq->vdev;
+ struct virtio_pcidev_device *dev = vdev->priv;
+
+ if (!dev->pdev.irq)
+ return;
+
+ /* we should properly chain interrupts, but on ARCH=um we don't care */
+
+ switch (msg->op) {
+ case VIRTIO_PCIDEV_OP_INT:
+ generic_handle_irq(dev->pdev.irq);
+ break;
+ case VIRTIO_PCIDEV_OP_MSI:
+ /* our MSI message is just the interrupt number */
+ if (msg->size == sizeof(u32))
+ generic_handle_irq(le32_to_cpup((void *)msg->data));
+ else
+ generic_handle_irq(le16_to_cpup((void *)msg->data));
+ break;
+ case VIRTIO_PCIDEV_OP_PME:
+ /* nothing to do - we already woke up due to the message */
+ break;
+ default:
+ dev_err(&vdev->dev, "unexpected virt-pci message %d\n", msg->op);
+ break;
+ }
+}
+
+static void virtio_pcidev_cmd_vq_cb(struct virtqueue *vq)
+{
+ struct virtio_device *vdev = vq->vdev;
+ struct virtio_pcidev_device *dev = vdev->priv;
+ void *cmd;
+ int len;
+
+ if (test_bit(UM_PCI_STAT_WAITING, &dev->status))
+ return;
+
+ while ((cmd = virtqueue_get_buf(vq, &len)))
+ virtio_pcidev_free_buf(dev, cmd);
+}
+
+static void virtio_pcidev_irq_vq_cb(struct virtqueue *vq)
+{
+ struct virtio_pcidev_msg *msg;
+ int len;
+
+ while ((msg = virtqueue_get_buf(vq, &len))) {
+ if (len >= sizeof(*msg))
+ virtio_pcidev_handle_irq_message(vq, msg);
+
+ /* recycle the message buffer */
+ virtio_pcidev_irq_vq_addbuf(vq, msg, true);
+ }
+}
+
+static int virtio_pcidev_init_vqs(struct virtio_pcidev_device *dev)
+{
+ struct virtqueue_info vqs_info[] = {
+ { "cmd", virtio_pcidev_cmd_vq_cb },
+ { "irq", virtio_pcidev_irq_vq_cb },
+ };
+ struct virtqueue *vqs[2];
+ int err, i;
+
+ err = virtio_find_vqs(dev->vdev, 2, vqs, vqs_info, NULL);
+ if (err)
+ return err;
+
+ dev->cmd_vq = vqs[0];
+ dev->irq_vq = vqs[1];
+
+ virtio_device_ready(dev->vdev);
+
+ for (i = 0; i < NUM_IRQ_MSGS; i++) {
+ void *msg = kzalloc(MAX_IRQ_MSG_SIZE, GFP_KERNEL);
+
+ if (msg)
+ virtio_pcidev_irq_vq_addbuf(dev->irq_vq, msg, false);
+ }
+
+ virtqueue_kick(dev->irq_vq);
+
+ return 0;
+}
+
+static void __virtio_pcidev_virtio_platform_remove(struct virtio_device *vdev,
+ struct virtio_pcidev_device *dev)
+{
+ um_pci_platform_device_unregister(&dev->pdev);
+
+ virtio_reset_device(vdev);
+ vdev->config->del_vqs(vdev);
+
+ kfree(dev);
+}
+
+static int virtio_pcidev_virtio_platform_probe(struct virtio_device *vdev,
+ struct virtio_pcidev_device *dev)
+{
+ int err;
+
+ dev->platform = true;
+
+ err = virtio_pcidev_init_vqs(dev);
+ if (err)
+ goto err_free;
+
+ err = um_pci_platform_device_register(&dev->pdev);
+ if (err)
+ goto err_reset;
+
+ err = of_platform_default_populate(vdev->dev.of_node, NULL, &vdev->dev);
+ if (err)
+ goto err_unregister;
+
+ return 0;
+
+err_unregister:
+ um_pci_platform_device_unregister(&dev->pdev);
+err_reset:
+ virtio_reset_device(vdev);
+ vdev->config->del_vqs(vdev);
+err_free:
+ kfree(dev);
+ return err;
+}
+
+static int virtio_pcidev_virtio_probe(struct virtio_device *vdev)
+{
+ struct virtio_pcidev_device *dev;
+ int err;
+
+ dev = kzalloc(sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->vdev = vdev;
+ vdev->priv = dev;
+
+ dev->pdev.ops = &virtio_pcidev_um_pci_ops;
+
+ if (of_device_is_compatible(vdev->dev.of_node, "simple-bus"))
+ return virtio_pcidev_virtio_platform_probe(vdev, dev);
+
+ err = virtio_pcidev_init_vqs(dev);
+ if (err)
+ goto err_free;
+
+ err = um_pci_device_register(&dev->pdev);
+ if (err)
+ goto err_reset;
+
+ device_set_wakeup_enable(&vdev->dev, true);
+
+ /*
+ * In order to do suspend-resume properly, don't allow VQs
+ * to be suspended.
+ */
+ virtio_uml_set_no_vq_suspend(vdev, true);
+
+ return 0;
+
+err_reset:
+ virtio_reset_device(vdev);
+ vdev->config->del_vqs(vdev);
+err_free:
+ kfree(dev);
+ return err;
+}
+
+static void virtio_pcidev_virtio_remove(struct virtio_device *vdev)
+{
+ struct virtio_pcidev_device *dev = vdev->priv;
+
+ if (dev->platform) {
+ of_platform_depopulate(&vdev->dev);
+ __virtio_pcidev_virtio_platform_remove(vdev, dev);
+ return;
+ }
+
+ device_set_wakeup_enable(&vdev->dev, false);
+
+ um_pci_device_unregister(&dev->pdev);
+
+ /* Stop all virtqueues */
+ virtio_reset_device(vdev);
+ dev->cmd_vq = NULL;
+ dev->irq_vq = NULL;
+ vdev->config->del_vqs(vdev);
+
+ kfree(dev);
+}
+
+static struct virtio_device_id id_table[] = {
+ { CONFIG_UML_PCI_OVER_VIRTIO_DEVICE_ID, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+MODULE_DEVICE_TABLE(virtio, id_table);
+
+static struct virtio_driver virtio_pcidev_virtio_driver = {
+ .driver.name = "virtio-pci",
+ .id_table = id_table,
+ .probe = virtio_pcidev_virtio_probe,
+ .remove = virtio_pcidev_virtio_remove,
+};
+
+static int __init virtio_pcidev_init(void)
+{
+ if (WARN(CONFIG_UML_PCI_OVER_VIRTIO_DEVICE_ID < 0,
+ "No virtio device ID configured for PCI - no PCI support\n"))
+ return 0;
+
+ return register_virtio_driver(&virtio_pcidev_virtio_driver);
+}
+late_initcall(virtio_pcidev_init);
+
+static void __exit virtio_pcidev_exit(void)
+{
+ unregister_virtio_driver(&virtio_pcidev_virtio_driver);
+}
+module_exit(virtio_pcidev_exit);
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/3] um: Add VFIO-based virtual PCI driver
2025-03-15 16:19 [PATCH 0/3] um: Add VFIO-based PCI passthrough support Tiwei Bie
2025-03-15 16:19 ` [PATCH 1/3] um: Rewrite the sigio workaround based on epoll and tgkill Tiwei Bie
2025-03-15 16:19 ` [PATCH 2/3] um: virt-pci: Refactor virtio_pcidev into its own module Tiwei Bie
@ 2025-03-15 16:19 ` Tiwei Bie
2025-03-20 8:50 ` Johannes Berg
2 siblings, 1 reply; 6+ messages in thread
From: Tiwei Bie @ 2025-03-15 16:19 UTC (permalink / raw)
To: richard, anton.ivanov, johannes; +Cc: linux-um, Tiwei Bie
Implement a new virtual PCI driver based on the VFIO framework.
This driver allows users to pass through PCI devices to UML via
VFIO. Currently, only MSI-X capable devices are supported, and
it is assumed that drivers will use MSI-X.
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
---
arch/um/drivers/Kconfig | 8 +
arch/um/drivers/Makefile | 2 +
arch/um/drivers/vfio_kern.c | 648 ++++++++++++++++++++++++++++++++++++
arch/um/drivers/vfio_user.c | 323 ++++++++++++++++++
arch/um/drivers/vfio_user.h | 44 +++
5 files changed, 1025 insertions(+)
create mode 100644 arch/um/drivers/vfio_kern.c
create mode 100644 arch/um/drivers/vfio_user.c
create mode 100644 arch/um/drivers/vfio_user.h
diff --git a/arch/um/drivers/Kconfig b/arch/um/drivers/Kconfig
index 9cb196070614..d7bb447ff958 100644
--- a/arch/um/drivers/Kconfig
+++ b/arch/um/drivers/Kconfig
@@ -367,3 +367,11 @@ config UML_PCI_OVER_VIRTIO_DEVICE_ID
There's no official device ID assigned (yet), set the one you
wish to use for experimentation here. The default of -1 is
not valid and will cause the driver to fail at probe.
+
+config UML_PCI_OVER_VFIO
+ bool "Enable VFIO-based PCI passthrough"
+ select UML_PCI
+ help
+ This driver provides support for VFIO-based PCI passthrough.
+ Currently, only MSI-X capable devices are supported, and it
+ is assumed that drivers will use MSI-X.
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 0a5820343ad3..336be56b8975 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -19,6 +19,7 @@ port-objs := port_kern.o port_user.o
harddog-objs := harddog_kern.o
harddog-builtin-$(CONFIG_UML_WATCHDOG) := harddog_user.o harddog_user_exp.o
rtc-objs := rtc_kern.o rtc_user.o
+vfio_uml-objs := vfio_kern.o vfio_user.o
LDFLAGS_vde.o = $(shell $(CC) $(CFLAGS) -print-file-name=libvdeplug.a)
@@ -62,6 +63,7 @@ obj-$(CONFIG_VIRTIO_UML) += virtio_uml.o
obj-$(CONFIG_UML_RTC) += rtc.o
obj-$(CONFIG_UML_PCI) += virt-pci.o
obj-$(CONFIG_UML_PCI_OVER_VIRTIO) += virtio_pcidev.o
+obj-$(CONFIG_UML_PCI_OVER_VFIO) += vfio_uml.o
# pcap_user.o must be added explicitly.
USER_OBJS := fd.o null.o pty.o tty.o xterm.o slip_common.o vde_user.o vector_user.o
diff --git a/arch/um/drivers/vfio_kern.c b/arch/um/drivers/vfio_kern.c
new file mode 100644
index 000000000000..805f589a568d
--- /dev/null
+++ b/arch/um/drivers/vfio_kern.c
@@ -0,0 +1,648 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2025 Ant Group
+ * Author: Tiwei Bie <tiwei.btw@antgroup.com>
+ */
+#include <linux/module.h>
+#include <linux/logic_iomem.h>
+#include <linux/mutex.h>
+#include <linux/string.h>
+#include <linux/unaligned.h>
+#include <irq_kern.h>
+#include <init.h>
+#include <os.h>
+
+#include "virt-pci.h"
+#include "vfio_user.h"
+
+#define MAX_GROUPS 8
+#define MAX_DEVICES 8
+
+#define to_vdev(_pdev) container_of(_pdev, struct uml_vfio_device, pdev)
+
+struct uml_vfio_intr_ctx {
+ struct uml_vfio_device *dev;
+ int irq;
+};
+
+struct uml_vfio_device {
+ const char *name;
+ int group;
+
+ struct um_pci_device pdev;
+ struct uml_vfio_user_device udev;
+ struct uml_vfio_intr_ctx *intr_ctx;
+
+ int msix_cap;
+ int msix_bar;
+ int msix_offset;
+ int msix_size;
+ u32 *msix_data;
+};
+
+static struct {
+ int fd;
+ int users;
+} uml_vfio_container;
+static DEFINE_MUTEX(uml_vfio_container_mtx);
+
+static struct {
+ int id;
+ int fd;
+ int users;
+} uml_vfio_groups[MAX_GROUPS];
+static DEFINE_MUTEX(uml_vfio_groups_mtx);
+
+static struct uml_vfio_device *uml_vfio_devices[MAX_DEVICES];
+
+static int uml_vfio_open_container(void)
+{
+ int fd;
+
+ fd = uml_vfio_user_open_container();
+ if (fd < 0)
+ return fd;
+
+ uml_vfio_container.fd = fd;
+ return 0;
+}
+
+static void uml_vfio_release_container(void)
+{
+ os_close_file(uml_vfio_container.fd);
+}
+
+static int uml_vfio_set_container(int group_fd)
+{
+ int err;
+
+ guard(mutex)(¨_vfio_container_mtx);
+
+ err = uml_vfio_user_set_container(uml_vfio_container.fd, group_fd);
+ if (err)
+ return err;
+
+ uml_vfio_container.users++;
+ if (uml_vfio_container.users > 1)
+ return 0;
+
+ err = uml_vfio_user_setup_iommu(uml_vfio_container.fd);
+ if (err) {
+ uml_vfio_user_unset_container(uml_vfio_container.fd, group_fd);
+ uml_vfio_container.users--;
+ }
+ return err;
+}
+
+static void uml_vfio_unset_container(int group_fd)
+{
+ guard(mutex)(¨_vfio_container_mtx);
+
+ uml_vfio_user_unset_container(uml_vfio_container.fd, group_fd);
+ uml_vfio_container.users--;
+}
+
+static int uml_vfio_open_group(int group_id)
+{
+ int free = -1, err, fd, i;
+
+ guard(mutex)(¨_vfio_groups_mtx);
+
+ for (i = 0; i < MAX_GROUPS; i++) {
+ if (uml_vfio_groups[i].users > 0 &&
+ uml_vfio_groups[i].id == group_id) {
+ uml_vfio_groups[i].users++;
+ return uml_vfio_groups[i].fd;
+ }
+ }
+
+ for (i = 0; i < MAX_GROUPS; i++) {
+ if (uml_vfio_groups[i].users == 0) {
+ free = i;
+ break;
+ }
+ }
+
+ if (free < 0)
+ return -ENOSPC;
+
+ fd = uml_vfio_user_open_group(group_id);
+ if (fd < 0)
+ return fd;
+
+ err = uml_vfio_set_container(fd);
+ if (err) {
+ os_close_file(fd);
+ return err;
+ }
+
+ uml_vfio_groups[free].id = group_id;
+ uml_vfio_groups[free].fd = fd;
+ uml_vfio_groups[free].users = 1;
+
+ return fd;
+}
+
+static int uml_vfio_release_group(int group_fd)
+{
+ int i;
+
+ guard(mutex)(¨_vfio_groups_mtx);
+
+ for (i = 0; i < MAX_GROUPS; i++) {
+ if (uml_vfio_groups[i].users > 0 &&
+ uml_vfio_groups[i].fd == group_fd)
+ break;
+ }
+
+ if (i == MAX_GROUPS)
+ return -ENOENT;
+
+ uml_vfio_groups[i].users--;
+ if (uml_vfio_groups[i].users > 0)
+ return 0;
+
+ uml_vfio_unset_container(group_fd);
+ os_close_file(group_fd);
+ return 0;
+}
+
+static irqreturn_t uml_vfio_interrupt(int unused, void *opaque)
+{
+ struct uml_vfio_intr_ctx *ctx = opaque;
+ struct uml_vfio_device *dev = ctx->dev;
+ int index = ctx - dev->intr_ctx;
+ int irqfd = dev->udev.irqfd[index];
+ int irq = dev->msix_data[index];
+ uint64_t v;
+ int r;
+
+ do {
+ r = os_read_file(irqfd, &v, sizeof(v));
+ if (r == sizeof(v))
+ generic_handle_irq(irq);
+ } while (r == sizeof(v) || r == -EINTR);
+ WARN(r != -EAGAIN, "read returned %d\n", r);
+
+ return IRQ_HANDLED;
+}
+
+static int uml_vfio_activate_irq(struct uml_vfio_device *dev, int index)
+{
+ struct uml_vfio_intr_ctx *ctx = &dev->intr_ctx[index];
+ int err, irqfd;
+
+ if (ctx->irq >= 0)
+ return 0;
+
+ irqfd = uml_vfio_user_activate_irq(&dev->udev, index);
+ if (irqfd < 0)
+ return irqfd;
+
+ ctx->irq = um_request_irq(UM_IRQ_ALLOC, irqfd, IRQ_READ,
+ uml_vfio_interrupt, IRQF_SHARED,
+ "vfio-uml", ctx);
+ if (ctx->irq < 0) {
+ err = ctx->irq;
+ goto deactivate;
+ }
+
+ err = add_sigio_fd(irqfd);
+ if (err)
+ goto free_irq;
+
+ return 0;
+
+free_irq:
+ um_free_irq(ctx->irq, ctx);
+ ctx->irq = -1;
+deactivate:
+ uml_vfio_user_deactivate_irq(&dev->udev, index);
+ return err;
+}
+
+static int uml_vfio_deactivate_irq(struct uml_vfio_device *dev, int index)
+{
+ struct uml_vfio_intr_ctx *ctx = &dev->intr_ctx[index];
+
+ if (ctx->irq >= 0) {
+ ignore_sigio_fd(dev->udev.irqfd[index]);
+ um_free_irq(ctx->irq, ctx);
+ uml_vfio_user_deactivate_irq(&dev->udev, index);
+ ctx->irq = -1;
+ }
+ return 0;
+}
+
+static int uml_vfio_update_msix_cap(struct uml_vfio_device *dev,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ int err = 0;
+
+ if (size == 2 && offset == dev->msix_cap + PCI_MSIX_FLAGS) {
+ switch (val & ~PCI_MSIX_FLAGS_QSIZE) {
+ case PCI_MSIX_FLAGS_ENABLE:
+ case 0:
+ err = uml_vfio_user_update_irqs(&dev->udev);
+ break;
+ }
+ }
+
+ return err;
+}
+
+static int uml_vfio_update_msix_table(struct uml_vfio_device *dev,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ int index;
+
+ offset -= dev->msix_offset + PCI_MSIX_ENTRY_DATA;
+
+ if (size != 4 || offset % PCI_MSIX_ENTRY_SIZE != 0)
+ return 0;
+
+ index = offset / PCI_MSIX_ENTRY_SIZE;
+ if (index >= dev->udev.irq_count)
+ return -EINVAL;
+
+ dev->msix_data[index] = val;
+
+ return val ? uml_vfio_activate_irq(dev, index) :
+ uml_vfio_deactivate_irq(dev, index);
+}
+
+static unsigned long __uml_vfio_cfgspace_read(struct uml_vfio_device *dev,
+ unsigned int offset, int size)
+{
+ u8 data[8];
+
+ memset(data, 0xff, sizeof(data));
+
+ if (uml_vfio_user_cfgspace_read(&dev->udev, offset, data, size))
+ return ULONG_MAX;
+
+ switch (size) {
+ case 1:
+ return data[0];
+ case 2:
+ return le16_to_cpup((void *)data);
+ case 4:
+ return le32_to_cpup((void *)data);
+#ifdef CONFIG_64BIT
+ case 8:
+ return le64_to_cpup((void *)data);
+#endif
+ default:
+ return ULONG_MAX;
+ }
+}
+
+static unsigned long uml_vfio_cfgspace_read(struct um_pci_device *pdev,
+ unsigned int offset, int size)
+{
+ struct uml_vfio_device *dev = to_vdev(pdev);
+
+ return __uml_vfio_cfgspace_read(dev, offset, size);
+}
+
+static void __uml_vfio_cfgspace_write(struct uml_vfio_device *dev,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ u8 data[8];
+
+ switch (size) {
+ case 1:
+ data[0] = (u8)val;
+ break;
+ case 2:
+ put_unaligned_le16(val, (void *)data);
+ break;
+ case 4:
+ put_unaligned_le32(val, (void *)data);
+ break;
+#ifdef CONFIG_64BIT
+ case 8:
+ put_unaligned_le64(val, (void *)data);
+ break;
+#endif
+ }
+
+ WARN_ON(uml_vfio_user_cfgspace_write(&dev->udev, offset, data, size));
+}
+
+static void uml_vfio_cfgspace_write(struct um_pci_device *pdev,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ struct uml_vfio_device *dev = to_vdev(pdev);
+
+ if (offset < dev->msix_cap + PCI_CAP_MSIX_SIZEOF &&
+ offset + size > dev->msix_cap)
+ WARN_ON(uml_vfio_update_msix_cap(dev, offset, size, val));
+
+ __uml_vfio_cfgspace_write(dev, offset, size, val);
+}
+
+static void uml_vfio_bar_copy_from(struct um_pci_device *pdev, int bar,
+ void *buffer, unsigned int offset, int size)
+{
+ struct uml_vfio_device *dev = to_vdev(pdev);
+
+ memset(buffer, 0xff, size);
+ uml_vfio_user_bar_read(&dev->udev, bar, offset, buffer, size);
+}
+
+static unsigned long uml_vfio_bar_read(struct um_pci_device *pdev, int bar,
+ unsigned int offset, int size)
+{
+ u8 data[8];
+
+ uml_vfio_bar_copy_from(pdev, bar, data, offset, size);
+
+ switch (size) {
+ case 1:
+ return data[0];
+ case 2:
+ return le16_to_cpup((void *)data);
+ case 4:
+ return le32_to_cpup((void *)data);
+#ifdef CONFIG_64BIT
+ case 8:
+ return le64_to_cpup((void *)data);
+#endif
+ default:
+ return ULONG_MAX;
+ }
+}
+
+static void uml_vfio_bar_copy_to(struct um_pci_device *pdev, int bar,
+ unsigned int offset, const void *buffer,
+ int size)
+{
+ struct uml_vfio_device *dev = to_vdev(pdev);
+
+ uml_vfio_user_bar_write(&dev->udev, bar, offset, buffer, size);
+}
+
+static void uml_vfio_bar_write(struct um_pci_device *pdev, int bar,
+ unsigned int offset, int size,
+ unsigned long val)
+{
+ struct uml_vfio_device *dev = to_vdev(pdev);
+ u8 data[8];
+
+ if (bar == dev->msix_bar && offset + size > dev->msix_offset &&
+ offset < dev->msix_offset + dev->msix_size)
+ WARN_ON(uml_vfio_update_msix_table(dev, offset, size, val));
+
+ switch (size) {
+ case 1:
+ data[0] = (u8)val;
+ break;
+ case 2:
+ put_unaligned_le16(val, (void *)data);
+ break;
+ case 4:
+ put_unaligned_le32(val, (void *)data);
+ break;
+#ifdef CONFIG_64BIT
+ case 8:
+ put_unaligned_le64(val, (void *)data);
+ break;
+#endif
+ }
+
+ uml_vfio_bar_copy_to(pdev, bar, offset, data, size);
+}
+
+static void uml_vfio_bar_set(struct um_pci_device *pdev, int bar,
+ unsigned int offset, u8 value, int size)
+{
+ struct uml_vfio_device *dev = to_vdev(pdev);
+ int i;
+
+ for (i = 0; i < size; i++)
+ uml_vfio_user_bar_write(&dev->udev, bar, offset + i, &value, 1);
+}
+
+static const struct um_pci_ops uml_vfio_um_pci_ops = {
+ .cfgspace_read = uml_vfio_cfgspace_read,
+ .cfgspace_write = uml_vfio_cfgspace_write,
+ .bar_read = uml_vfio_bar_read,
+ .bar_write = uml_vfio_bar_write,
+ .bar_copy_from = uml_vfio_bar_copy_from,
+ .bar_copy_to = uml_vfio_bar_copy_to,
+ .bar_set = uml_vfio_bar_set,
+};
+
+static u8 uml_vfio_find_capability(struct uml_vfio_device *dev, u8 cap)
+{
+ u8 id, pos;
+ u16 ent;
+ int ttl = 48;
+
+ pos = __uml_vfio_cfgspace_read(dev, PCI_CAPABILITY_LIST, sizeof(pos));
+
+ while (pos && ttl--) {
+ ent = __uml_vfio_cfgspace_read(dev, pos, sizeof(ent));
+
+ id = ent & 0xff;
+ if (id == 0xff)
+ break;
+ if (id == cap)
+ return pos;
+
+ pos = ent >> 8;
+ }
+
+ return 0;
+}
+
+static int uml_vfio_read_msix_table(struct uml_vfio_device *dev)
+{
+ unsigned int off;
+ u16 flags;
+ u32 tbl;
+
+ off = uml_vfio_find_capability(dev, PCI_CAP_ID_MSIX);
+ if (!off)
+ return -ENOTSUPP;
+
+ dev->msix_cap = off;
+
+ tbl = __uml_vfio_cfgspace_read(dev, off + PCI_MSIX_TABLE, sizeof(tbl));
+ flags = __uml_vfio_cfgspace_read(dev, off + PCI_MSIX_FLAGS, sizeof(flags));
+
+ dev->msix_bar = tbl & PCI_MSIX_TABLE_BIR;
+ dev->msix_offset = tbl & PCI_MSIX_TABLE_OFFSET;
+ dev->msix_size = ((flags & PCI_MSIX_FLAGS_QSIZE) + 1) * PCI_MSIX_ENTRY_SIZE;
+
+ dev->msix_data = kzalloc(dev->msix_size, GFP_KERNEL);
+ if (!dev->msix_data)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int uml_vfio_open_device(struct uml_vfio_device *dev)
+{
+ struct uml_vfio_intr_ctx *ctx;
+ int err, group_id, i;
+
+ group_id = uml_vfio_user_get_group_id(dev->name);
+ if (group_id < 0)
+ return group_id;
+
+ dev->group = uml_vfio_open_group(group_id);
+ if (dev->group < 0)
+ return dev->group;
+
+ err = uml_vfio_user_setup_device(&dev->udev, dev->group, dev->name);
+ if (err)
+ goto release_group;
+
+ err = uml_vfio_read_msix_table(dev);
+ if (err)
+ goto teardown_udev;
+
+ dev->intr_ctx = kmalloc_array(dev->udev.irq_count,
+ sizeof(struct uml_vfio_intr_ctx),
+ GFP_KERNEL);
+ if (!dev->intr_ctx) {
+ err = -ENOMEM;
+ goto free_msix;
+ }
+
+ for (i = 0; i < dev->udev.irq_count; i++) {
+ ctx = &dev->intr_ctx[i];
+ ctx->dev = dev;
+ ctx->irq = -1;
+ }
+
+ dev->pdev.ops = ¨_vfio_um_pci_ops;
+
+ err = um_pci_device_register(&dev->pdev);
+ if (err)
+ goto free_intr_ctx;
+
+ return 0;
+
+free_intr_ctx:
+ kfree(dev->intr_ctx);
+free_msix:
+ kfree(dev->msix_data);
+teardown_udev:
+ uml_vfio_user_teardown_device(&dev->udev);
+release_group:
+ uml_vfio_release_group(dev->group);
+ return err;
+}
+
+static void uml_vfio_release_device(struct uml_vfio_device *dev)
+{
+ int i;
+
+ for (i = 0; i < dev->udev.irq_count; i++)
+ uml_vfio_deactivate_irq(dev, i);
+
+ uml_vfio_user_update_irqs(&dev->udev);
+ um_pci_device_unregister(&dev->pdev);
+ kfree(dev->intr_ctx);
+ kfree(dev->msix_data);
+ uml_vfio_user_teardown_device(&dev->udev);
+ uml_vfio_release_group(dev->group);
+ kfree(dev->name);
+ kfree(dev);
+}
+
+static int uml_vfio_cmdline_set(const char *device, const struct kernel_param *kp)
+{
+ struct uml_vfio_device *dev;
+ int free = -1, i;
+
+ for (i = 0; i < MAX_DEVICES; i++) {
+ if (uml_vfio_devices[i])
+ continue;
+ free = i;
+ break;
+ }
+
+ if (free < 0)
+ return -ENOSPC;
+
+ dev = kzalloc(sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->name = kstrdup(device, GFP_KERNEL);
+ if (!dev->name) {
+ kfree(dev);
+ return -ENOMEM;
+ }
+
+ uml_vfio_devices[free] = dev;
+ return 0;
+}
+
+static int uml_vfio_cmdline_get(char *buffer, const struct kernel_param *kp)
+{
+ return 0;
+}
+
+static const struct kernel_param_ops uml_vfio_cmdline_param_ops = {
+ .set = uml_vfio_cmdline_set,
+ .get = uml_vfio_cmdline_get,
+};
+
+device_param_cb(device, ¨_vfio_cmdline_param_ops, NULL, 0400);
+__uml_help(uml_vfio_cmdline_param_ops,
+"vfio_uml.device=<domain:bus:slot.function>\n"
+" Pass through a PCI device to UML via VFIO. Currently, only MSI-X\n"
+" capable devices are supported, and it is assumed that drivers will\n"
+" use MSI-X. This parameter can be specified multiple times to pass\n"
+" through multiple PCI devices to UML.\n\n"
+);
+
+static int __init uml_vfio_init(void)
+{
+ struct uml_vfio_device *dev;
+ int err, i;
+
+ sigio_broken();
+
+ err = uml_vfio_open_container();
+ if (err)
+ return err;
+
+ for (i = 0; i < MAX_DEVICES; i++) {
+ dev = uml_vfio_devices[i];
+ if (!dev)
+ break;
+ err = uml_vfio_open_device(dev);
+ if (err)
+ printk(KERN_ERR "uml_vfio open device (%s) failed, error %d\n",
+ dev->name, err);
+ }
+
+ return 0;
+}
+late_initcall(uml_vfio_init);
+
+static void __exit uml_vfio_exit(void)
+{
+ struct uml_vfio_device *dev;
+ int i;
+
+ for (i = 0; i < MAX_DEVICES; i++) {
+ dev = uml_vfio_devices[i];
+ if (!dev)
+ break;
+ uml_vfio_release_device(dev);
+ }
+
+ uml_vfio_release_container();
+}
+module_exit(uml_vfio_exit);
diff --git a/arch/um/drivers/vfio_user.c b/arch/um/drivers/vfio_user.c
new file mode 100644
index 000000000000..0a97e367fc59
--- /dev/null
+++ b/arch/um/drivers/vfio_user.c
@@ -0,0 +1,323 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2025 Ant Group
+ * Author: Tiwei Bie <tiwei.btw@antgroup.com>
+ */
+#include <errno.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/eventfd.h>
+#include <linux/limits.h>
+#include <linux/vfio.h>
+#include <linux/pci_regs.h>
+#include <as-layout.h>
+#include <um_malloc.h>
+
+#include "vfio_user.h"
+
+int uml_vfio_user_open_container(void)
+{
+ int r, fd;
+
+ fd = open("/dev/vfio/vfio", O_RDWR);
+ if (fd < 0)
+ return -errno;
+
+ r = ioctl(fd, VFIO_GET_API_VERSION);
+ if (r != VFIO_API_VERSION) {
+ r = r < 0 ? -errno : -EINVAL;
+ goto error;
+ }
+
+ r = ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU);
+ if (r <= 0) {
+ r = r < 0 ? -errno : -EINVAL;
+ goto error;
+ }
+
+ return fd;
+
+error:
+ close(fd);
+ return r;
+}
+
+int uml_vfio_user_setup_iommu(int container)
+{
+ unsigned long reserved = uml_reserved - uml_physmem;
+ struct vfio_iommu_type1_dma_map dma_map = {
+ .argsz = sizeof(dma_map),
+ .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
+ .vaddr = uml_reserved,
+ .iova = reserved,
+ .size = physmem_size - reserved,
+ };
+
+ if (ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_IOMMU) < 0)
+ return -errno;
+
+ if (ioctl(container, VFIO_IOMMU_MAP_DMA, &dma_map) < 0)
+ return -errno;
+
+ return 0;
+}
+
+int uml_vfio_user_get_group_id(const char *device)
+{
+ char *path, *buf, *end;
+ const char *name;
+ int r;
+
+ path = uml_kmalloc(PATH_MAX, UM_GFP_KERNEL);
+ if (!path)
+ return -ENOMEM;
+
+ sprintf(path, "/sys/bus/pci/devices/%s/iommu_group", device);
+
+ buf = uml_kmalloc(PATH_MAX + 1, UM_GFP_KERNEL);
+ if (!buf) {
+ r = -ENOMEM;
+ goto free_path;
+ }
+
+ r = readlink(path, buf, PATH_MAX);
+ if (r < 0) {
+ r = -errno;
+ goto free_buf;
+ }
+ buf[r] = '\0';
+
+ name = basename(buf);
+
+ r = strtoul(name, &end, 10);
+ if (*end != '\0' || end == name) {
+ r = -EINVAL;
+ goto free_buf;
+ }
+
+free_buf:
+ kfree(buf);
+free_path:
+ kfree(path);
+ return r;
+}
+
+int uml_vfio_user_open_group(int group_id)
+{
+ char *path;
+ int fd;
+
+ path = uml_kmalloc(PATH_MAX, UM_GFP_KERNEL);
+ if (!path)
+ return -ENOMEM;
+
+ sprintf(path, "/dev/vfio/%d", group_id);
+
+ fd = open(path, O_RDWR);
+ if (fd < 0) {
+ fd = -errno;
+ goto out;
+ }
+
+out:
+ kfree(path);
+ return fd;
+}
+
+int uml_vfio_user_set_container(int container, int group)
+{
+ if (ioctl(group, VFIO_GROUP_SET_CONTAINER, &container) < 0)
+ return -errno;
+ return 0;
+}
+
+int uml_vfio_user_unset_container(int container, int group)
+{
+ if (ioctl(group, VFIO_GROUP_UNSET_CONTAINER, &container) < 0)
+ return -errno;
+ return 0;
+}
+
+static int vfio_set_irqs(int device, int start, int count, int *irqfd)
+{
+ struct vfio_irq_set *irq_set;
+ int argsz = sizeof(*irq_set) + sizeof(*irqfd) * count;
+ int err = 0;
+
+ irq_set = uml_kmalloc(argsz, UM_GFP_KERNEL);
+ if (!irq_set)
+ return -ENOMEM;
+
+ irq_set->argsz = argsz;
+ irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER;
+ irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX;
+ irq_set->start = start;
+ irq_set->count = count;
+ memcpy(irq_set->data, irqfd, sizeof(*irqfd) * count);
+
+ if (ioctl(device, VFIO_DEVICE_SET_IRQS, irq_set) < 0) {
+ err = -errno;
+ goto out;
+ }
+
+out:
+ kfree(irq_set);
+ return err;
+}
+
+int uml_vfio_user_setup_device(struct uml_vfio_user_device *dev,
+ int group, const char *device)
+{
+ struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
+ struct vfio_irq_info irq_info = { .argsz = sizeof(irq_info) };
+ int err, i;
+
+ dev->device = ioctl(group, VFIO_GROUP_GET_DEVICE_FD, device);
+ if (dev->device < 0)
+ return -errno;
+
+ if (ioctl(dev->device, VFIO_DEVICE_GET_INFO, &device_info) < 0) {
+ err = -errno;
+ goto close_device;
+ }
+
+ dev->num_regions = device_info.num_regions;
+ if (dev->num_regions > VFIO_PCI_CONFIG_REGION_INDEX + 1)
+ dev->num_regions = VFIO_PCI_CONFIG_REGION_INDEX + 1;
+
+ dev->region = uml_kmalloc(sizeof(*dev->region) * dev->num_regions,
+ UM_GFP_KERNEL);
+ if (!dev->region) {
+ err = -ENOMEM;
+ goto close_device;
+ }
+
+ for (i = 0; i < dev->num_regions; i++) {
+ struct vfio_region_info region = {
+ .argsz = sizeof(region),
+ .index = i,
+ };
+ if (ioctl(dev->device, VFIO_DEVICE_GET_REGION_INFO, ®ion) < 0) {
+ err = -errno;
+ goto free_region;
+ }
+ dev->region[i].size = region.size;
+ dev->region[i].offset = region.offset;
+ }
+
+ /* Only MSI-X is supported currently. */
+ irq_info.index = VFIO_PCI_MSIX_IRQ_INDEX;
+ if (ioctl(dev->device, VFIO_DEVICE_GET_IRQ_INFO, &irq_info) < 0) {
+ err = -errno;
+ goto free_region;
+ }
+
+ dev->irq_count = irq_info.count;
+
+ dev->irqfd = uml_kmalloc(sizeof(int) * dev->irq_count, UM_GFP_KERNEL);
+ if (!dev->irqfd) {
+ err = -ENOMEM;
+ goto free_region;
+ }
+
+ memset(dev->irqfd, -1, sizeof(int) * dev->irq_count);
+
+ err = vfio_set_irqs(dev->device, 0, dev->irq_count, dev->irqfd);
+ if (err)
+ goto free_irqfd;
+
+ return 0;
+
+free_irqfd:
+ kfree(dev->irqfd);
+free_region:
+ kfree(dev->region);
+close_device:
+ close(dev->device);
+ return err;
+}
+
+void uml_vfio_user_teardown_device(struct uml_vfio_user_device *dev)
+{
+ kfree(dev->irqfd);
+ kfree(dev->region);
+ close(dev->device);
+}
+
+int uml_vfio_user_activate_irq(struct uml_vfio_user_device *dev, int index)
+{
+ int irqfd;
+
+ irqfd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC);
+ if (irqfd < 0)
+ return -errno;
+
+ dev->irqfd[index] = irqfd;
+ return irqfd;
+}
+
+void uml_vfio_user_deactivate_irq(struct uml_vfio_user_device *dev, int index)
+{
+ close(dev->irqfd[index]);
+ dev->irqfd[index] = -1;
+}
+
+int uml_vfio_user_update_irqs(struct uml_vfio_user_device *dev)
+{
+ return vfio_set_irqs(dev->device, 0, dev->irq_count, dev->irqfd);
+}
+
+static int vfio_region_read(struct uml_vfio_user_device *dev, unsigned int index,
+ uint64_t offset, void *buf, uint64_t size)
+{
+ if (index >= dev->num_regions || offset + size > dev->region[index].size)
+ return -EINVAL;
+
+ if (pread(dev->device, buf, size, dev->region[index].offset + offset) < 0)
+ return -errno;
+
+ return 0;
+}
+
+static int vfio_region_write(struct uml_vfio_user_device *dev, unsigned int index,
+ uint64_t offset, const void *buf, uint64_t size)
+{
+ if (index >= dev->num_regions || offset + size > dev->region[index].size)
+ return -EINVAL;
+
+ if (pwrite(dev->device, buf, size, dev->region[index].offset + offset) < 0)
+ return -errno;
+
+ return 0;
+}
+
+int uml_vfio_user_cfgspace_read(struct uml_vfio_user_device *dev,
+ unsigned int offset, void *buf, int size)
+{
+ return vfio_region_read(dev, VFIO_PCI_CONFIG_REGION_INDEX,
+ offset, buf, size);
+}
+
+int uml_vfio_user_cfgspace_write(struct uml_vfio_user_device *dev,
+ unsigned int offset, const void *buf, int size)
+{
+ return vfio_region_write(dev, VFIO_PCI_CONFIG_REGION_INDEX,
+ offset, buf, size);
+}
+
+int uml_vfio_user_bar_read(struct uml_vfio_user_device *dev, int bar,
+ unsigned int offset, void *buf, int size)
+{
+ return vfio_region_read(dev, bar, offset, buf, size);
+}
+
+int uml_vfio_user_bar_write(struct uml_vfio_user_device *dev, int bar,
+ unsigned int offset, const void *buf, int size)
+{
+ return vfio_region_write(dev, bar, offset, buf, size);
+}
diff --git a/arch/um/drivers/vfio_user.h b/arch/um/drivers/vfio_user.h
new file mode 100644
index 000000000000..75535e05059b
--- /dev/null
+++ b/arch/um/drivers/vfio_user.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_VFIO_USER_H
+#define __UM_VFIO_USER_H
+
+struct uml_vfio_user_device {
+ int device;
+
+ struct {
+ uint64_t size;
+ uint64_t offset;
+ } *region;
+ int num_regions;
+
+ int32_t *irqfd;
+ int irq_count;
+};
+
+int uml_vfio_user_open_container(void);
+int uml_vfio_user_setup_iommu(int container);
+
+int uml_vfio_user_get_group_id(const char *device);
+int uml_vfio_user_open_group(int group_id);
+int uml_vfio_user_set_container(int container, int group);
+int uml_vfio_user_unset_container(int container, int group);
+
+int uml_vfio_user_setup_device(struct uml_vfio_user_device *dev,
+ int group, const char *device);
+void uml_vfio_user_teardown_device(struct uml_vfio_user_device *dev);
+
+int uml_vfio_user_activate_irq(struct uml_vfio_user_device *dev, int index);
+void uml_vfio_user_deactivate_irq(struct uml_vfio_user_device *dev, int index);
+int uml_vfio_user_update_irqs(struct uml_vfio_user_device *dev);
+
+int uml_vfio_user_cfgspace_read(struct uml_vfio_user_device *dev,
+ unsigned int offset, void *buf, int size);
+int uml_vfio_user_cfgspace_write(struct uml_vfio_user_device *dev,
+ unsigned int offset, const void *buf, int size);
+
+int uml_vfio_user_bar_read(struct uml_vfio_user_device *dev, int bar,
+ unsigned int offset, void *buf, int size);
+int uml_vfio_user_bar_write(struct uml_vfio_user_device *dev, int bar,
+ unsigned int offset, const void *buf, int size);
+
+#endif /* __UM_VFIO_USER_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 3/3] um: Add VFIO-based virtual PCI driver
2025-03-15 16:19 ` [PATCH 3/3] um: Add VFIO-based virtual PCI driver Tiwei Bie
@ 2025-03-20 8:50 ` Johannes Berg
2025-03-20 15:16 ` Tiwei Bie
0 siblings, 1 reply; 6+ messages in thread
From: Johannes Berg @ 2025-03-20 8:50 UTC (permalink / raw)
To: Tiwei Bie, richard, anton.ivanov; +Cc: linux-um
On Sun, 2025-03-16 at 00:19 +0800, Tiwei Bie wrote:
> Implement a new virtual PCI driver based on the VFIO framework.
> This driver allows users to pass through PCI devices to UML via
> VFIO. Currently, only MSI-X capable devices are supported, and
> it is assumed that drivers will use MSI-X.
Seems nice, but I haven't tested it now :)
> +static irqreturn_t uml_vfio_interrupt(int unused, void *opaque)
> +{
> + struct uml_vfio_intr_ctx *ctx = opaque;
> + struct uml_vfio_device *dev = ctx->dev;
> + int index = ctx - dev->intr_ctx;
> + int irqfd = dev->udev.irqfd[index];
> + int irq = dev->msix_data[index];
> + uint64_t v;
> + int r;
> +
> + do {
> + r = os_read_file(irqfd, &v, sizeof(v));
> + if (r == sizeof(v))
> + generic_handle_irq(irq);
> + } while (r == sizeof(v) || r == -EINTR);
> + WARN(r != -EAGAIN, "read returned %d\n", r);
> +
> + return IRQ_HANDLED;
> +}
This seems mostly right - looks like it's an eventfd.
> + irqfd = uml_vfio_user_activate_irq(&dev->udev, index);
> + if (irqfd < 0)
> + return irqfd;
> +
> + ctx->irq = um_request_irq(UM_IRQ_ALLOC, irqfd, IRQ_READ,
> + uml_vfio_interrupt, IRQF_SHARED,
> + "vfio-uml", ctx);
However I'm not sure it's right with IRQF_SHARED since you don't detect
if it was actually not your interrupt that fired? We don't really have
any good reason to have shared interrupts anyway though, and I'm not
sure we even *can* share interrupts in the lower layers in ARCH=um, so
I'm not sure it matters?
> + err = add_sigio_fd(irqfd);
> + if (err)
> + goto free_irq;
doesn't the irq framework do that automatically?
> +static int uml_vfio_deactivate_irq(struct uml_vfio_device *dev, int index)
> +{
> + struct uml_vfio_intr_ctx *ctx = &dev->intr_ctx[index];
> +
> + if (ctx->irq >= 0) {
> + ignore_sigio_fd(dev->udev.irqfd[index]);
> + um_free_irq(ctx->irq, ctx);
and here too?
> +static int uml_vfio_update_msix_cap(struct uml_vfio_device *dev,
> + unsigned int offset, int size,
> + unsigned long val)
> +{
> + int err = 0;
> +
> + if (size == 2 && offset == dev->msix_cap + PCI_MSIX_FLAGS) {
> + switch (val & ~PCI_MSIX_FLAGS_QSIZE) {
> + case PCI_MSIX_FLAGS_ENABLE:
> + case 0:
> + err = uml_vfio_user_update_irqs(&dev->udev);
> + break;
> + }
> + }
> +
> + return err;
do you really want to ignore and return 0 if it wasn't supported?
> +static int uml_vfio_update_msix_table(struct uml_vfio_device *dev,
> + unsigned int offset, int size,
> + unsigned long val)
> +{
> + int index;
> +
> + offset -= dev->msix_offset + PCI_MSIX_ENTRY_DATA;
> +
> + if (size != 4 || offset % PCI_MSIX_ENTRY_SIZE != 0)
> + return 0;
here too?
> +static void uml_vfio_cfgspace_write(struct um_pci_device *pdev,
> + unsigned int offset, int size,
> + unsigned long val)
> +{
> + struct uml_vfio_device *dev = to_vdev(pdev);
> +
> + if (offset < dev->msix_cap + PCI_CAP_MSIX_SIZEOF &&
> + offset + size > dev->msix_cap)
I'd probably indent the second line just to after "if ("?
> + if (bar == dev->msix_bar && offset + size > dev->msix_offset &&
> + offset < dev->msix_offset + dev->msix_size)
> + WARN_ON(uml_vfio_update_msix_table(dev, offset, size, val));
likewise
> +static u8 uml_vfio_find_capability(struct uml_vfio_device *dev, u8 cap)
> +{
> + u8 id, pos;
> + u16 ent;
> + int ttl = 48;
any particular significance in that value?
> +int uml_vfio_user_setup_iommu(int container)
> +{
> + unsigned long reserved = uml_reserved - uml_physmem;
> + struct vfio_iommu_type1_dma_map dma_map = {
> + .argsz = sizeof(dma_map),
> + .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
> + .vaddr = uml_reserved,
> + .iova = reserved,
> + .size = physmem_size - reserved,
> + };
maybe point over to the big comment in vhost_user_set_mem_table() from
virtio_uml.c? Same things apply here I guess.
johannes
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 3/3] um: Add VFIO-based virtual PCI driver
2025-03-20 8:50 ` Johannes Berg
@ 2025-03-20 15:16 ` Tiwei Bie
0 siblings, 0 replies; 6+ messages in thread
From: Tiwei Bie @ 2025-03-20 15:16 UTC (permalink / raw)
To: Johannes Berg, richard, anton.ivanov; +Cc: linux-um
On 2025/3/20 16:50, Johannes Berg wrote:
> On Sun, 2025-03-16 at 00:19 +0800, Tiwei Bie wrote:
>> Implement a new virtual PCI driver based on the VFIO framework.
>> This driver allows users to pass through PCI devices to UML via
>> VFIO. Currently, only MSI-X capable devices are supported, and
>> it is assumed that drivers will use MSI-X.
>
> Seems nice, but I haven't tested it now :)
Thanks! :)
>
>> +static irqreturn_t uml_vfio_interrupt(int unused, void *opaque)
>> +{
>> + struct uml_vfio_intr_ctx *ctx = opaque;
>> + struct uml_vfio_device *dev = ctx->dev;
>> + int index = ctx - dev->intr_ctx;
>> + int irqfd = dev->udev.irqfd[index];
>> + int irq = dev->msix_data[index];
>> + uint64_t v;
>> + int r;
>> +
>> + do {
>> + r = os_read_file(irqfd, &v, sizeof(v));
>> + if (r == sizeof(v))
>> + generic_handle_irq(irq);
>> + } while (r == sizeof(v) || r == -EINTR);
>> + WARN(r != -EAGAIN, "read returned %d\n", r);
>> +
>> + return IRQ_HANDLED;
>> +}
>
> This seems mostly right - looks like it's an eventfd.
Yeah, it's an eventfd.
>
>> + irqfd = uml_vfio_user_activate_irq(&dev->udev, index);
>> + if (irqfd < 0)
>> + return irqfd;
>> +
>> + ctx->irq = um_request_irq(UM_IRQ_ALLOC, irqfd, IRQ_READ,
>> + uml_vfio_interrupt, IRQF_SHARED,
>> + "vfio-uml", ctx);
>
> However I'm not sure it's right with IRQF_SHARED since you don't detect
> if it was actually not your interrupt that fired? We don't really have
> any good reason to have shared interrupts anyway though, and I'm not
> sure we even *can* share interrupts in the lower layers in ARCH=um, so
> I'm not sure it matters?
Makes sense. Will fix it. Thanks!
>
>> + err = add_sigio_fd(irqfd);
>> + if (err)
>> + goto free_irq;
>
> doesn't the irq framework do that automatically?
Currently it's not supported. Users need to add it manually, e.g.:
https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/um/drivers/rtc_user.c?h=v6.14-rc7#n43
https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/um/drivers/rtc_kern.c?h=v6.14-rc7#n127
>
>> +static int uml_vfio_deactivate_irq(struct uml_vfio_device *dev, int index)
>> +{
>> + struct uml_vfio_intr_ctx *ctx = &dev->intr_ctx[index];
>> +
>> + if (ctx->irq >= 0) {
>> + ignore_sigio_fd(dev->udev.irqfd[index]);
>> + um_free_irq(ctx->irq, ctx);
>
> and here too?
>
>> +static int uml_vfio_update_msix_cap(struct uml_vfio_device *dev,
>> + unsigned int offset, int size,
>> + unsigned long val)
>> +{
>> + int err = 0;
>> +
>> + if (size == 2 && offset == dev->msix_cap + PCI_MSIX_FLAGS) {
>> + switch (val & ~PCI_MSIX_FLAGS_QSIZE) {
>> + case PCI_MSIX_FLAGS_ENABLE:
>> + case 0:
>> + err = uml_vfio_user_update_irqs(&dev->udev);
>> + break;
>> + }
>> + }
>> +
>> + return err;
>
> do you really want to ignore and return 0 if it wasn't supported?
I just realized that this part could be confusing. It handles only some
essential cmds and ignores others. I'll add some comments to clarify.
>
>> +static int uml_vfio_update_msix_table(struct uml_vfio_device *dev,
>> + unsigned int offset, int size,
>> + unsigned long val)
>> +{
>> + int index;
>> +
>> + offset -= dev->msix_offset + PCI_MSIX_ENTRY_DATA;
>> +
>> + if (size != 4 || offset % PCI_MSIX_ENTRY_SIZE != 0)
>> + return 0;
>
> here too?
>
>> +static void uml_vfio_cfgspace_write(struct um_pci_device *pdev,
>> + unsigned int offset, int size,
>> + unsigned long val)
>> +{
>> + struct uml_vfio_device *dev = to_vdev(pdev);
>> +
>> + if (offset < dev->msix_cap + PCI_CAP_MSIX_SIZEOF &&
>> + offset + size > dev->msix_cap)
>
> I'd probably indent the second line just to after "if ("?
Sure. Will do.
>
>> + if (bar == dev->msix_bar && offset + size > dev->msix_offset &&
>> + offset < dev->msix_offset + dev->msix_size)
>> + WARN_ON(uml_vfio_update_msix_table(dev, offset, size, val));
>
> likewise
>
>> +static u8 uml_vfio_find_capability(struct uml_vfio_device *dev, u8 cap)
>> +{
>> + u8 id, pos;
>> + u16 ent;
>> + int ttl = 48;
>
> any particular significance in that value?
I borrowed the value of PCI_FIND_CAP_TTL from drivers/pci/pci.h,
which is an internal header and can't be included from here. I'll
add a comment to it.
>
>> +int uml_vfio_user_setup_iommu(int container)
>> +{
>> + unsigned long reserved = uml_reserved - uml_physmem;
>> + struct vfio_iommu_type1_dma_map dma_map = {
>> + .argsz = sizeof(dma_map),
>> + .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
>> + .vaddr = uml_reserved,
>> + .iova = reserved,
>> + .size = physmem_size - reserved,
>> + };
>
> maybe point over to the big comment in vhost_user_set_mem_table() from
> virtio_uml.c? Same things apply here I guess.
Good idea. Will do.
Thanks again for the review! :)
Regards,
Tiwei
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-03-20 15:16 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-03-15 16:19 [PATCH 0/3] um: Add VFIO-based PCI passthrough support Tiwei Bie
2025-03-15 16:19 ` [PATCH 1/3] um: Rewrite the sigio workaround based on epoll and tgkill Tiwei Bie
2025-03-15 16:19 ` [PATCH 2/3] um: virt-pci: Refactor virtio_pcidev into its own module Tiwei Bie
2025-03-15 16:19 ` [PATCH 3/3] um: Add VFIO-based virtual PCI driver Tiwei Bie
2025-03-20 8:50 ` Johannes Berg
2025-03-20 15:16 ` Tiwei Bie
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox