* [PATCH v5 5/5] vhost/vsock: add VHOST_RESET_OWNER ioctl
From: Andrey Drobyshev @ 2026-07-20 10:22 UTC (permalink / raw)
To: linux-kernel
Cc: kvm, virtualization, netdev, sgarzare, mst, stefanha,
dongli.zhang, maciej.szmigiero, bchaney, mark.kanda, ptikhomirov,
den, andrey.drobyshev
In-Reply-To: <20260720102241.371610-1-andrey.drobyshev@virtuozzo.com>
From: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
This ioctl is needed for QEMU's CPR (checkpoint-restore) migration of
the guest with vhost-vsock device. For this to work, we need to reset
the device ownership on the source side by calling RESET_OWNER, and then
claim it on the dest side by calling SET_OWNER. We expect not to lose any
AF_VSOCK connection while this happens.
To that end, unlike the release path, RESET_OWNER keeps the guest CID
hashed: established connections survive, and host sends issued while
the device is between owners simply stay on send_pkt_queue until the
next device start drains them.
Since the device stays reachable through the CID hash, the lockless
send/cancel paths can race with the worker teardown in
vhost_workers_free(). The previous commit ("vhost: synchronize with
RCU readers when freeing workers") makes that safe.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
---
drivers/vhost/vsock.c | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index d5022d21120b..86f25ff80722 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -903,6 +903,29 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features)
return -EFAULT;
}
+static long vhost_vsock_reset_owner(struct vhost_vsock *vsock)
+{
+ struct vhost_iotlb *umem;
+ long err;
+
+ mutex_lock(&vsock->dev.mutex);
+ err = vhost_dev_check_owner(&vsock->dev);
+ if (err)
+ goto done;
+ umem = vhost_dev_reset_owner_prepare();
+ if (!umem) {
+ err = -ENOMEM;
+ goto done;
+ }
+ vhost_vsock_drop_backends(vsock);
+ vhost_vsock_flush(vsock);
+ vhost_dev_stop(&vsock->dev);
+ vhost_dev_reset_owner(&vsock->dev, umem);
+done:
+ mutex_unlock(&vsock->dev.mutex);
+ return err;
+}
+
static long vhost_vsock_dev_ioctl(struct file *f, unsigned int ioctl,
unsigned long arg)
{
@@ -946,6 +969,8 @@ static long vhost_vsock_dev_ioctl(struct file *f, unsigned int ioctl,
return -EOPNOTSUPP;
vhost_set_backend_features(&vsock->dev, features);
return 0;
+ case VHOST_RESET_OWNER:
+ return vhost_vsock_reset_owner(vsock);
default:
mutex_lock(&vsock->dev.mutex);
r = vhost_dev_ioctl(&vsock->dev, ioctl, argp);
--
2.47.1
^ permalink raw reply related
* [PATCH v5 4/5] vhost: synchronize with RCU readers when freeing workers
From: Andrey Drobyshev @ 2026-07-20 10:22 UTC (permalink / raw)
To: linux-kernel
Cc: kvm, virtualization, netdev, sgarzare, mst, stefanha,
dongli.zhang, maciej.szmigiero, bchaney, mark.kanda, ptikhomirov,
den, andrey.drobyshev
In-Reply-To: <20260720102241.371610-1-andrey.drobyshev@virtuozzo.com>
vhost_vq_work_queue() only holds the RCU read lock while it dereferences
vq->worker and queues work on it. vhost_workers_free() however clears
the vq->worker pointers and immediately frees the workers, without
waiting for a grace period. A caller that fetched the worker right
before the pointer was cleared can therefore still be queueing work on
it while it is freed. And even when the queueing itself wins the race,
the work is never run, so its VHOST_WORK_QUEUED bit stays set and all
future attempts to queue it are silently skipped.
None of the current callers can actually hit this: net and scsi stop
their virtqueues before the workers are freed, and vsock unhashes the
device and does synchronize_rcu() of its own in vhost_vsock_dev_release()
before the workers go away. But the upcoming VHOST_RESET_OWNER support
in vhost-vsock keeps the device hashed while its workers are freed, so
the lockless send/cancel paths become able to race with the teardown.
Fix this by clearing the vq->worker pointers, waiting for a grace
period, and then flushing the workers so any work the last readers
queued runs before the workers are freed.
Fixes: 228a27cf78af ("vhost: Allow worker switching while work is queueing")
Suggested-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
---
drivers/vhost/vhost.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 4c525b3e16ea..d6e235c25254 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -729,6 +729,17 @@ static void vhost_workers_free(struct vhost_dev *dev)
for (i = 0; i < dev->nvqs; i++)
rcu_assign_pointer(dev->vqs[i]->worker, NULL);
+
+ /*
+ * vhost_vq_work_queue() reads vq->worker under rcu_read_lock(), so a
+ * reader that fetched a worker before we cleared the pointers above
+ * may still be queueing work on it. Wait for those readers to
+ * finish, then flush so any work they queued runs (clearing
+ * VHOST_WORK_QUEUED) before the workers are freed.
+ */
+ synchronize_rcu();
+ vhost_dev_flush(dev);
+
/*
* Free the default worker we created and cleanup workers userspace
* created but couldn't clean up (it forgot or crashed).
--
2.47.1
^ permalink raw reply related
* Re: [PATCH 2/3] sched: Convert paravirt_steal to new static key APIs
From: Jürgen Groß @ 2026-07-20 10:35 UTC (permalink / raw)
To: Hongyan Xia, mingo@redhat.com, peterz@infradead.org,
juri.lelli@redhat.com, vincent.guittot@linaro.org,
dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com,
mgorman@suse.de, vschneid@redhat.com, kprateek.nayak@amd.com,
Ajay Kaher, Alexey Makhalov, Broadcom internal kernel review list,
Catalin Marinas, Will Deacon, Huacai Chen, WANG Xuerui,
Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
Christophe Leroy (CS GROUP), Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Thomas Gleixner, Borislav Petkov,
Dave Hansen, x86@kernel.org, H. Peter Anvin, Paolo Bonzini,
Vitaly Kuznetsov, Stefano Stabellini, Oleksandr Tyshchenko
Cc: Jiazi Li, linux-kernel@vger.kernel.org,
virtualization@lists.linux.dev,
linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev,
linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org,
kvm@vger.kernel.org, xen-devel@lists.xenproject.org
In-Reply-To: <773bbd8fcd5ef2ebfe1c64303c7a71a56368de69.1784538478.git.hongyan.xia@transsion.com>
[-- Attachment #1.1.1: Type: text/plain, Size: 375 bytes --]
On 20.07.26 11:38, Hongyan Xia wrote:
> From: Hongyan Xia <hongyan.xia@transsion.com>
>
> paravirt_steal_rq_enabled and paravirt_steal_enabled use raw static_key
> APIs which are now deprecated. Use the new API instead.
>
> No functional change.
>
> Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
Acked-by: Juergen Gross <jgross@suse.com>
Juergen
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply
* Re: [PATCH 01/15] drm/exynos: remove dependency on DRM simple helpers
From: Thomas Zimmermann @ 2026-07-20 11:05 UTC (permalink / raw)
To: Diogo Silva, Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park,
David Airlie, Simona Vetter, Krzysztof Kozlowski, Peter Griffin,
Alim Akhtar, Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Michal Simek, Thierry Reding, Mikko Perttunen,
Jonathan Hunter, Stefan Agner, Alison Wang, Anitha Chrisanthus,
David Airlie, Gerd Hoffmann, Dmitry Osipenko, Gurchetan Singh,
Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li, Sascha Hauer,
Pengutronix Kernel Team, Fabio Estevam, Chun-Kuang Hu,
Philipp Zabel, Matthias Brugger, AngeloGioacchino Del Regno,
Geert Uytterhoeven, Xinliang Liu, Sumit Semwal, Yongqin Liu,
John Stultz, Liviu Dudau, Neil Armstrong, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc
In-Reply-To: <20260719-drm_simple_encoder_init-v1-1-a78c509e3062@gmail.com>
Hi
Am 19.07.26 um 01:35 schrieb Diogo Silva:
> The simple KMS helpers are deprecated because they only add an
> intermediate layer between drivers and atomic modesetting.
>
> Open-code drm_simple_encoder_init() by calling drm_encoder_init()
> directly and providing driver-local drm_encoder_funcs.
> Also check the return value from drm_encoder_init() to avoid silent
> failures.
>
> Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
> ---
> drivers/gpu/drm/exynos/exynos_dp.c | 13 +++++++++++--
> drivers/gpu/drm/exynos/exynos_drm_dpi.c | 14 ++++++++++++--
> drivers/gpu/drm/exynos/exynos_drm_dsi.c | 11 +++++++++--
> drivers/gpu/drm/exynos/exynos_drm_vidi.c | 14 ++++++++++++--
> drivers/gpu/drm/exynos/exynos_hdmi.c | 15 +++++++++++++--
> 5 files changed, 57 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/gpu/drm/exynos/exynos_dp.c b/drivers/gpu/drm/exynos/exynos_dp.c
> index b80540328150..1598892c602b 100644
> --- a/drivers/gpu/drm/exynos/exynos_dp.c
> +++ b/drivers/gpu/drm/exynos/exynos_dp.c
> @@ -24,11 +24,11 @@
> #include <drm/drm_bridge.h>
> #include <drm/drm_bridge_connector.h>
> #include <drm/drm_crtc.h>
> +#include <drm/drm_encoder.h>
> #include <drm/drm_of.h>
> #include <drm/drm_panel.h>
> #include <drm/drm_print.h>
> #include <drm/drm_probe_helper.h>
> -#include <drm/drm_simple_kms_helper.h>
> #include <drm/exynos_drm.h>
>
> #include "exynos_drm_crtc.h"
> @@ -79,6 +79,10 @@ static void exynos_dp_nop(struct drm_encoder *encoder)
> /* do nothing */
> }
>
> +static const struct drm_encoder_funcs exynos_dp_encoder_funcs = {
> + .destroy = drm_encoder_cleanup,
> +};
> +
> static const struct drm_encoder_helper_funcs exynos_dp_encoder_helper_funcs = {
> .mode_set = exynos_dp_mode_set,
> .enable = exynos_dp_nop,
> @@ -95,7 +99,12 @@ static int exynos_dp_bind(struct device *dev, struct device *master, void *data)
>
> dp->drm_dev = drm_dev;
>
> - drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
> + ret = drm_encoder_init(drm_dev, encoder, &exynos_dp_encoder_funcs,
> + DRM_MODE_ENCODER_TMDS, NULL);
> + if (ret) {
> + dev_err(dp->dev, "Failed to initialize encoder\n");
Preferably use drm_err and drm_dev.
> + return ret;
> + }
>
> drm_encoder_helper_add(encoder, &exynos_dp_encoder_helper_funcs);
>
> diff --git a/drivers/gpu/drm/exynos/exynos_drm_dpi.c b/drivers/gpu/drm/exynos/exynos_drm_dpi.c
> index 0dc36df6ada3..4e42a1da81d1 100644
> --- a/drivers/gpu/drm/exynos/exynos_drm_dpi.c
> +++ b/drivers/gpu/drm/exynos/exynos_drm_dpi.c
> @@ -12,10 +12,10 @@
> #include <linux/regulator/consumer.h>
>
> #include <drm/drm_atomic_helper.h>
> +#include <drm/drm_encoder.h>
> #include <drm/drm_panel.h>
> #include <drm/drm_print.h>
> #include <drm/drm_probe_helper.h>
> -#include <drm/drm_simple_kms_helper.h>
>
> #include <video/of_videomode.h>
> #include <video/videomode.h>
> @@ -140,6 +140,10 @@ static void exynos_dpi_disable(struct drm_encoder *encoder)
> }
> }
>
> +static const struct drm_encoder_funcs exynos_dpi_encoder_funcs = {
> + .destroy = drm_encoder_cleanup,
> +};
> +
> static const struct drm_encoder_helper_funcs exynos_dpi_encoder_helper_funcs = {
> .mode_set = exynos_dpi_mode_set,
> .enable = exynos_dpi_enable,
> @@ -194,7 +198,13 @@ int exynos_dpi_bind(struct drm_device *dev, struct drm_encoder *encoder)
> {
> int ret;
>
> - drm_simple_encoder_init(dev, encoder, DRM_MODE_ENCODER_TMDS);
> + ret = drm_encoder_init(dev, encoder, &exynos_dpi_encoder_funcs,
> + DRM_MODE_ENCODER_TMDS, NULL);
> + if (ret) {
> + DRM_DEV_ERROR(encoder_to_dpi(encoder)->dev,
It just failed to init the encoder, so better not use it here. The
device should be the same as passed to the function via 'dev'.
> + "failed to create encoder ret = %d\n", ret);
> + return ret;
> + }
>
> drm_encoder_helper_add(encoder, &exynos_dpi_encoder_helper_funcs);
>
> diff --git a/drivers/gpu/drm/exynos/exynos_drm_dsi.c b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
> index c4d098ab7863..6b7561ac9bb0 100644
> --- a/drivers/gpu/drm/exynos/exynos_drm_dsi.c
> +++ b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
> @@ -13,7 +13,7 @@
>
> #include <drm/bridge/samsung-dsim.h>
> #include <drm/drm_probe_helper.h>
> -#include <drm/drm_simple_kms_helper.h>
> +#include <drm/drm_encoder.h>
>
> #include "exynos_drm_crtc.h"
> #include "exynos_drm_drv.h"
> @@ -22,6 +22,10 @@ struct exynos_dsi {
> struct drm_encoder encoder;
> };
>
> +static const struct drm_encoder_funcs exynos_drm_dsi_encoder_funcs = {
> + .destroy = drm_encoder_cleanup,
> +};
> +
> static irqreturn_t exynos_dsi_te_irq_handler(struct samsung_dsim *dsim)
> {
> struct exynos_dsi *dsi = dsim->priv;
> @@ -79,7 +83,10 @@ static int exynos_dsi_bind(struct device *dev, struct device *master, void *data
> struct drm_device *drm_dev = data;
> int ret;
>
> - drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
> + ret = drm_encoder_init(drm_dev, encoder, &exynos_drm_dsi_encoder_funcs,
> + DRM_MODE_ENCODER_TMDS, NULL);
> + if (ret)
> + return ret;
>
> ret = exynos_drm_set_possible_crtcs(encoder, EXYNOS_DISPLAY_TYPE_LCD);
> if (ret < 0)
> diff --git a/drivers/gpu/drm/exynos/exynos_drm_vidi.c b/drivers/gpu/drm/exynos/exynos_drm_vidi.c
> index 67bbf9b8bc0e..59dea853d364 100644
> --- a/drivers/gpu/drm/exynos/exynos_drm_vidi.c
> +++ b/drivers/gpu/drm/exynos/exynos_drm_vidi.c
> @@ -13,10 +13,10 @@
>
> #include <drm/drm_atomic_helper.h>
> #include <drm/drm_edid.h>
> +#include <drm/drm_encoder.h>
> #include <drm/drm_framebuffer.h>
> #include <drm/drm_print.h>
> #include <drm/drm_probe_helper.h>
> -#include <drm/drm_simple_kms_helper.h>
> #include <drm/drm_vblank.h>
> #include <drm/exynos_drm.h>
>
> @@ -403,6 +403,10 @@ static void exynos_vidi_disable(struct drm_encoder *encoder)
> {
> }
>
> +static const struct drm_encoder_funcs exynos_vidi_encoder_funcs = {
> + .destroy = drm_encoder_cleanup,
> +};
> +
> static const struct drm_encoder_helper_funcs exynos_vidi_encoder_helper_funcs = {
> .mode_set = exynos_vidi_mode_set,
> .enable = exynos_vidi_enable,
> @@ -445,7 +449,13 @@ static int vidi_bind(struct device *dev, struct device *master, void *data)
> return PTR_ERR(ctx->crtc);
> }
>
> - drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
> + ret = drm_encoder_init(drm_dev, encoder, &exynos_vidi_encoder_funcs,
> + DRM_MODE_ENCODER_TMDS, NULL);
> + if (ret) {
> + DRM_DEV_ERROR(dev, "failed to initialize encoder ret = %d\n",
> + ret);
Again, you rather want drm_err() with drm_dev here.
I've briefly looked over the series and many patches seem affected.
Please prefer drm_ logging functions and DRM devices over the plain
device equivalents.
Best regards
Thomas
> + return ret;
> + }
>
> drm_encoder_helper_add(encoder, &exynos_vidi_encoder_helper_funcs);
>
> diff --git a/drivers/gpu/drm/exynos/exynos_hdmi.c b/drivers/gpu/drm/exynos/exynos_hdmi.c
> index 09b2cabb236f..f44586ce0fdf 100644
> --- a/drivers/gpu/drm/exynos/exynos_hdmi.c
> +++ b/drivers/gpu/drm/exynos/exynos_hdmi.c
> @@ -36,9 +36,9 @@
> #include <drm/drm_atomic_helper.h>
> #include <drm/drm_bridge.h>
> #include <drm/drm_edid.h>
> +#include <drm/drm_encoder.h>
> #include <drm/drm_print.h>
> #include <drm/drm_probe_helper.h>
> -#include <drm/drm_simple_kms_helper.h>
>
> #include "exynos_drm_crtc.h"
> #include "regs-hdmi.h"
> @@ -1575,6 +1575,11 @@ static void hdmi_disable(struct drm_encoder *encoder)
> mutex_unlock(&hdata->mutex);
> }
>
> +static const struct drm_encoder_funcs exynos_hdmi_encoder_funcs = {
> + .destroy = drm_encoder_cleanup,
> +};
> +
> +
> static const struct drm_encoder_helper_funcs exynos_hdmi_encoder_helper_funcs = {
> .mode_fixup = hdmi_mode_fixup,
> .enable = hdmi_enable,
> @@ -1862,7 +1867,13 @@ static int hdmi_bind(struct device *dev, struct device *master, void *data)
>
> hdata->phy_clk.enable = hdmiphy_clk_enable;
>
> - drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
> + ret = drm_encoder_init(drm_dev, encoder, &exynos_hdmi_encoder_funcs,
> + DRM_MODE_ENCODER_TMDS, NULL);
> + if (ret) {
> + DRM_DEV_ERROR(dev, "failed to initialize encoder ret = %d\n",
> + ret);
> + return ret;
> + }
>
> drm_encoder_helper_add(encoder, &exynos_hdmi_encoder_helper_funcs);
>
>
--
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, (HRB 36809, AG Nürnberg)
^ permalink raw reply
* Re: [PATCH v4 6/8] media: virtio: Add virtio_media_driver
From: Alistair Delva @ 2026-07-20 12:35 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Mauro Carvalho Chehab, Brian Daniels, Mauro Carvalho Chehab,
acourbot, aesteve, changyeon, daniel.almeida, eperezma, gnurou,
gurchetansingh, hverkuil, jasowang, linux-kernel, linux-media,
nicolas.dufresne, virtualization, xuanzhuo
In-Reply-To: <20260718132759-mutt-send-email-mst@kernel.org>
On Sat, Jul 18, 2026 at 10:29 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Sun, Jul 12, 2026 at 08:57:26AM +0200, Mauro Carvalho Chehab wrote:
> > On Thu, 25 Jun 2026 16:18:48 -0400
> > Brian Daniels <briandaniels@google.com> wrote:
> >
> > > > > From: Alexandre Courbot <gnurou@gmail.com>
> > > > >
> > > > > virtio_media_driver.c provides the expected driver hooks, and support
> > > > > for mmapping and polling.
> > > > >
> > > > > Signed-off-by: Alexandre Courbot <gnurou@gmail.com>
> > > > > Co-developed-by: Brian Daniels <briandaniels@google.com>
> > > > > Signed-off-by: Brian Daniels <briandaniels@google.com>
> > > > > ---
> > > > > drivers/media/virtio/virtio_media_driver.c | 959 +++++++++++++++++++++
> > > > > 1 file changed, 959 insertions(+)
> > > > > create mode 100644 drivers/media/virtio/virtio_media_driver.c
> > > > >
> > > > > diff --git a/drivers/media/virtio/virtio_media_driver.c b/drivers/media/virtio/virtio_media_driver.c
> > > > > new file mode 100644
> > > > > index 000000000..d6363c673
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/virtio/virtio_media_driver.c
> > > > > @@ -0,0 +1,959 @@
> > > > > +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0+
> > > > > +
> > > > > +/*
> > > > > + * Virtio-media driver.
> > > > > + *
> > > > > + * Copyright (c) 2024-2025 Google LLC.
> > > > > + */
> > > > > +
> > > > > +#include <linux/delay.h>
> > > > > +#include <linux/device.h>
> > > > > +#include <linux/dev_printk.h>
> > > > > +#include <linux/mm.h>
> > > > > +#include <linux/mutex.h>
> > > > > +#include <linux/scatterlist.h>
> > > > > +#include <linux/types.h>
> > > > > +#include <linux/videodev2.h>
> > > > > +#include <linux/vmalloc.h>
> > > > > +#include <linux/wait.h>
> > > > > +#include <linux/workqueue.h>
> > > > > +#include <linux/module.h>
> > > > > +#include <linux/moduleparam.h>
> > > > > +#include <linux/virtio.h>
> > > > > +#include <linux/virtio_config.h>
> > > > > +#include <linux/virtio_ids.h>
> > > > > +
> > > > > +#include <media/frame_vector.h>
> > > > > +#include <media/v4l2-dev.h>
> > > > > +#include <media/v4l2-event.h>
> > > > > +#include <media/videobuf2-memops.h>
> > > > > +#include <media/v4l2-device.h>
> > > > > +#include <media/v4l2-ioctl.h>
> > > > > +
> > > > > +#include "protocol.h"
> > > > > +#include "session.h"
> > > > > +#include "virtio_media.h"
> > > > > +
> > > > > +#define VIRTIO_MEDIA_NUM_EVENT_BUFS 16
> > > > > +
> > > > > +/* ID of the SHM region into which MMAP buffer will be mapped. */
> > > > > +#define VIRTIO_MEDIA_SHM_MMAP 0
> > > > > +
> > > > > +/*
> > > > > + * Name of the driver to expose to user-space.
> > > > > + *
> > > > > + * This is configurable because v4l2-compliance has workarounds specific to
> > > > > + * some drivers. When proxying these directly from the host, this allows it to
> > > > > + * apply them as needed.
> > > > > + */
> > > > > +char *virtio_media_driver_name;
> > > > > +module_param_named(driver_name, virtio_media_driver_name, charp, 0660);
> > > >
> > > >
> > > > Um. What? Not how it should be handled.
> > >
> > > I can remove this module param. I didn't end up using this when compliance testing.
> > > Instead, I patched v4l-utils:
> > > https://lore.kernel.org/all/20260528163448.4031965-1-briandaniels@google.com/
> > >
> > > Let me know if you think the v4l-utils patch is a good approach, otherwise let
> > > me know how you'd prefer to address the v4l2-compliance driver-specific workounds
> > > when they're being proxied with virtio-media.
> >
> > This kind of discussion should happen on a separate PR for v4l2-compliance,
> > c/c to the proper developers and maintainers of it.
>
>
> I wonder how migration can work when guest is tied to host driver model like
> this.
We haven't given migration much thought yet, but I think it wouldn't
be so different to GPU, where to do snapshot/migration we have to
record all initialization / context setup state and replay it against
the GPU driver on restore (see https://github.com/google/gfxstream).
Some migrations will be possible, some will not, and the host should
be able to decide.
Also, Brian mentioned the host device pass-through use case, but we
also have device implementations on the host that work with camera
emulators or some other data source like video/webrtc, which will
support snapshot/migration more easily. This use case will probably
will see more real-world use and migration will be more relevant.
> > >
> > > > > +
> > > > > +/*
> > > > > + * Whether USERPTR buffers are allowed.
> > > > > + *
> > > > > + * This is disabled by default as USERPTR buffers are dangerous, but the option
> > > > > + * is left to enable them if desired.
> > > > > + */
> > > > > +bool virtio_media_allow_userptr;
> > > > > +module_param_named(allow_userptr, virtio_media_allow_userptr, bool, 0660);
> > > >
> > > >
> > > > is this kind of thing common?
> >
> > There is one old media device that has it (saa7134).
> >
> > >
> > > To be honest, I don't really know. I'm also not that familiar with the USERPTR
> > > issues. I see a few references online about their use being discouraged due to
> > > possible race conditions, perhaps that was the original motivation for this
> > > parameter (I'm not the original author of this driver).
> > >
> > > I'm open to alternatives, feel free to let me know if you have a preference.
> >
> > We tend to not implement USERPTR on newer drivers. I suggest you
> > to place the logic with regards to V4L2_MEMORY_USERPTR on a separate
> > patch for further discussions.
> >
> > >
> > > > > +
> > > > > +/**
> > > > > + * virtio_media_session_alloc - Allocate a new session.
> > > > > + * @vv: virtio-media device the session belongs to.
> > > > > + * @id: ID of the session.
> > > > > + * @nonblocking_dequeue: whether dequeuing of buffers should be blocking or
> > > > > + * not.
> > > > > + *
> > > > > + * The ``id`` and ``list`` fields must still be set by the caller.
> > > >
> > > > still in what sense?
> > >
> > > Based on the code below, I'm not so sure that the caller is responsible for
> > > setting these values. They seem to be initialized in the function.
> > >
> > > Perhaps Alexandre Courbot (the original author) would know more. Unless he
> > > says otherwise though I'm inclined to remove this comment.
> > >
> > > > > + */
> > > > > +static struct virtio_media_session *
> > > > > +virtio_media_session_alloc(struct virtio_media *vv, u32 id,
> > > > > + struct file *file)
> > > > > +{
> > > > > + struct virtio_media_session *session;
> > > > > + int i;
> > > > > + int ret;
> > > > > +
> > > > > + session = kzalloc_obj(*session, GFP_KERNEL);
> > > > > + if (!session)
> > > > > + goto err_session;
> > > > > +
> > > > > + session->shadow_buf = kzalloc(VIRTIO_SHADOW_BUF_SIZE, GFP_KERNEL);
> > > > > + if (!session->shadow_buf)
> > > > > + goto err_shadow_buf;
> > > > > +
> > > > > + ret = sg_alloc_table(&session->command_sgs, DESC_CHAIN_MAX_LEN,
> > > > > + GFP_KERNEL);
> > > > > + if (ret)
> > > > > + goto err_payload_sgs;
> > > > > +
> > > > > + session->id = id;
> > > > > + session->nonblocking_dequeue = file->f_flags & O_NONBLOCK;
> > > > > +
> > > > > + INIT_LIST_HEAD(&session->list);
> > > > > + v4l2_fh_init(&session->fh, &vv->video_dev);
> > > > > + virtio_media_session_fh_add(session, file);
> > > > > +
> > > > > + for (i = 0; i <= VIRTIO_MEDIA_LAST_QUEUE; i++)
> > > > > + INIT_LIST_HEAD(&session->queues[i].pending_dqbufs);
> > > > > + mutex_init(&session->queues_lock);
> > > > > +
> > > > > + init_waitqueue_head(&session->dqbuf_wait);
> > > > > +
> > > > > + mutex_lock(&vv->sessions_lock);
> > > > > + list_add_tail(&session->list, &vv->sessions);
> > > > > + mutex_unlock(&vv->sessions_lock);
> > > > > +
> > > > > + return session;
> > > > > +
> > > > > +err_payload_sgs:
> > > > > + kfree(session->shadow_buf);
> > > > > +err_shadow_buf:
> > > > > + kfree(session);
> > > > > +err_session:
> > > > > + return ERR_PTR(-ENOMEM);
> > > > > +}
> > > > > +
> > > > > +/**
> > > > > + * virtio_media_session_free - Free all resources of a session.
> > > > > + * @vv: virtio-media device the session belongs to.
> > > > > + * @session: session to destroy.
> > > > > + *
> > > > > + * All the resources of @sesssion, as well as the backing memory of @session
> > > > > + * itself, are freed.
> > > >
> > > > why @ here and `` above? And typo in the name.
> > >
> > > The `@` here was an attempt to follow the guide here for referencing function
> > > parameters:
> > > https://docs.kernel.org/doc-guide/kernel-doc.html#highlights-and-cross-references
> >
> > Yes. This is part of Linux Kernel kernel-doc markup: when referring to
> > struct fields, you should use @field (or ``field`` if one wants to place an
> > asterisk on it, like ``*field``).
> >
> > >
> > > That being said, I don't believe this file is 100% consistent with that. I will
> > > spend some time cleaning up the comments throughout this patch set to get them
> > > consistent for v5. Thanks!
> >
> > Please use it on a consistent way along the driver.
> >
> >
> > Thanks,
> > Mauro
>
^ permalink raw reply
* Re: [PATCH v4 6/8] media: virtio: Add virtio_media_driver
From: Michael S. Tsirkin @ 2026-07-20 13:09 UTC (permalink / raw)
To: Alistair Delva
Cc: Mauro Carvalho Chehab, Brian Daniels, Mauro Carvalho Chehab,
acourbot, aesteve, changyeon, daniel.almeida, eperezma, gnurou,
gurchetansingh, hverkuil, jasowang, linux-kernel, linux-media,
nicolas.dufresne, virtualization, xuanzhuo
In-Reply-To: <CANDihLFeF_nKi5z=eH5Mwaa4s8fEU1chS4K2VQZ+0b-0OLybjg@mail.gmail.com>
On Mon, Jul 20, 2026 at 05:35:19AM -0700, Alistair Delva wrote:
> On Sat, Jul 18, 2026 at 10:29 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Sun, Jul 12, 2026 at 08:57:26AM +0200, Mauro Carvalho Chehab wrote:
> > > On Thu, 25 Jun 2026 16:18:48 -0400
> > > Brian Daniels <briandaniels@google.com> wrote:
> > >
> > > > > > From: Alexandre Courbot <gnurou@gmail.com>
> > > > > >
> > > > > > virtio_media_driver.c provides the expected driver hooks, and support
> > > > > > for mmapping and polling.
> > > > > >
> > > > > > Signed-off-by: Alexandre Courbot <gnurou@gmail.com>
> > > > > > Co-developed-by: Brian Daniels <briandaniels@google.com>
> > > > > > Signed-off-by: Brian Daniels <briandaniels@google.com>
> > > > > > ---
> > > > > > drivers/media/virtio/virtio_media_driver.c | 959 +++++++++++++++++++++
> > > > > > 1 file changed, 959 insertions(+)
> > > > > > create mode 100644 drivers/media/virtio/virtio_media_driver.c
> > > > > >
> > > > > > diff --git a/drivers/media/virtio/virtio_media_driver.c b/drivers/media/virtio/virtio_media_driver.c
> > > > > > new file mode 100644
> > > > > > index 000000000..d6363c673
> > > > > > --- /dev/null
> > > > > > +++ b/drivers/media/virtio/virtio_media_driver.c
> > > > > > @@ -0,0 +1,959 @@
> > > > > > +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0+
> > > > > > +
> > > > > > +/*
> > > > > > + * Virtio-media driver.
> > > > > > + *
> > > > > > + * Copyright (c) 2024-2025 Google LLC.
> > > > > > + */
> > > > > > +
> > > > > > +#include <linux/delay.h>
> > > > > > +#include <linux/device.h>
> > > > > > +#include <linux/dev_printk.h>
> > > > > > +#include <linux/mm.h>
> > > > > > +#include <linux/mutex.h>
> > > > > > +#include <linux/scatterlist.h>
> > > > > > +#include <linux/types.h>
> > > > > > +#include <linux/videodev2.h>
> > > > > > +#include <linux/vmalloc.h>
> > > > > > +#include <linux/wait.h>
> > > > > > +#include <linux/workqueue.h>
> > > > > > +#include <linux/module.h>
> > > > > > +#include <linux/moduleparam.h>
> > > > > > +#include <linux/virtio.h>
> > > > > > +#include <linux/virtio_config.h>
> > > > > > +#include <linux/virtio_ids.h>
> > > > > > +
> > > > > > +#include <media/frame_vector.h>
> > > > > > +#include <media/v4l2-dev.h>
> > > > > > +#include <media/v4l2-event.h>
> > > > > > +#include <media/videobuf2-memops.h>
> > > > > > +#include <media/v4l2-device.h>
> > > > > > +#include <media/v4l2-ioctl.h>
> > > > > > +
> > > > > > +#include "protocol.h"
> > > > > > +#include "session.h"
> > > > > > +#include "virtio_media.h"
> > > > > > +
> > > > > > +#define VIRTIO_MEDIA_NUM_EVENT_BUFS 16
> > > > > > +
> > > > > > +/* ID of the SHM region into which MMAP buffer will be mapped. */
> > > > > > +#define VIRTIO_MEDIA_SHM_MMAP 0
> > > > > > +
> > > > > > +/*
> > > > > > + * Name of the driver to expose to user-space.
> > > > > > + *
> > > > > > + * This is configurable because v4l2-compliance has workarounds specific to
> > > > > > + * some drivers. When proxying these directly from the host, this allows it to
> > > > > > + * apply them as needed.
> > > > > > + */
> > > > > > +char *virtio_media_driver_name;
> > > > > > +module_param_named(driver_name, virtio_media_driver_name, charp, 0660);
> > > > >
> > > > >
> > > > > Um. What? Not how it should be handled.
> > > >
> > > > I can remove this module param. I didn't end up using this when compliance testing.
> > > > Instead, I patched v4l-utils:
> > > > https://lore.kernel.org/all/20260528163448.4031965-1-briandaniels@google.com/
> > > >
> > > > Let me know if you think the v4l-utils patch is a good approach, otherwise let
> > > > me know how you'd prefer to address the v4l2-compliance driver-specific workounds
> > > > when they're being proxied with virtio-media.
> > >
> > > This kind of discussion should happen on a separate PR for v4l2-compliance,
> > > c/c to the proper developers and maintainers of it.
> >
> >
> > I wonder how migration can work when guest is tied to host driver model like
> > this.
>
> We haven't given migration much thought yet, but I think it wouldn't
> be so different to GPU, where to do snapshot/migration we have to
> record all initialization / context setup state and replay it against
> the GPU driver on restore (see https://github.com/google/gfxstream).
> Some migrations will be possible, some will not, and the host should
> be able to decide.
>
> Also, Brian mentioned the host device pass-through use case, but we
> also have device implementations on the host that work with camera
> emulators or some other data source like video/webrtc, which will
> support snapshot/migration more easily. This use case will probably
> will see more real-world use and migration will be more relevant.
This is par for the course.
But I'm afraid I wasn't clear enough. This thread mentions
supplying the host driver name to guest userspace
in order to implement "driver specific work arounds".
But given such, for migration to work userspace needs
to be notified about driver change? And what to do about
the things it started after migration but before the
notification?
> > > >
> > > > > > +
> > > > > > +/*
> > > > > > + * Whether USERPTR buffers are allowed.
> > > > > > + *
> > > > > > + * This is disabled by default as USERPTR buffers are dangerous, but the option
> > > > > > + * is left to enable them if desired.
> > > > > > + */
> > > > > > +bool virtio_media_allow_userptr;
> > > > > > +module_param_named(allow_userptr, virtio_media_allow_userptr, bool, 0660);
> > > > >
> > > > >
> > > > > is this kind of thing common?
> > >
> > > There is one old media device that has it (saa7134).
> > >
> > > >
> > > > To be honest, I don't really know. I'm also not that familiar with the USERPTR
> > > > issues. I see a few references online about their use being discouraged due to
> > > > possible race conditions, perhaps that was the original motivation for this
> > > > parameter (I'm not the original author of this driver).
> > > >
> > > > I'm open to alternatives, feel free to let me know if you have a preference.
> > >
> > > We tend to not implement USERPTR on newer drivers. I suggest you
> > > to place the logic with regards to V4L2_MEMORY_USERPTR on a separate
> > > patch for further discussions.
> > >
> > > >
> > > > > > +
> > > > > > +/**
> > > > > > + * virtio_media_session_alloc - Allocate a new session.
> > > > > > + * @vv: virtio-media device the session belongs to.
> > > > > > + * @id: ID of the session.
> > > > > > + * @nonblocking_dequeue: whether dequeuing of buffers should be blocking or
> > > > > > + * not.
> > > > > > + *
> > > > > > + * The ``id`` and ``list`` fields must still be set by the caller.
> > > > >
> > > > > still in what sense?
> > > >
> > > > Based on the code below, I'm not so sure that the caller is responsible for
> > > > setting these values. They seem to be initialized in the function.
> > > >
> > > > Perhaps Alexandre Courbot (the original author) would know more. Unless he
> > > > says otherwise though I'm inclined to remove this comment.
> > > >
> > > > > > + */
> > > > > > +static struct virtio_media_session *
> > > > > > +virtio_media_session_alloc(struct virtio_media *vv, u32 id,
> > > > > > + struct file *file)
> > > > > > +{
> > > > > > + struct virtio_media_session *session;
> > > > > > + int i;
> > > > > > + int ret;
> > > > > > +
> > > > > > + session = kzalloc_obj(*session, GFP_KERNEL);
> > > > > > + if (!session)
> > > > > > + goto err_session;
> > > > > > +
> > > > > > + session->shadow_buf = kzalloc(VIRTIO_SHADOW_BUF_SIZE, GFP_KERNEL);
> > > > > > + if (!session->shadow_buf)
> > > > > > + goto err_shadow_buf;
> > > > > > +
> > > > > > + ret = sg_alloc_table(&session->command_sgs, DESC_CHAIN_MAX_LEN,
> > > > > > + GFP_KERNEL);
> > > > > > + if (ret)
> > > > > > + goto err_payload_sgs;
> > > > > > +
> > > > > > + session->id = id;
> > > > > > + session->nonblocking_dequeue = file->f_flags & O_NONBLOCK;
> > > > > > +
> > > > > > + INIT_LIST_HEAD(&session->list);
> > > > > > + v4l2_fh_init(&session->fh, &vv->video_dev);
> > > > > > + virtio_media_session_fh_add(session, file);
> > > > > > +
> > > > > > + for (i = 0; i <= VIRTIO_MEDIA_LAST_QUEUE; i++)
> > > > > > + INIT_LIST_HEAD(&session->queues[i].pending_dqbufs);
> > > > > > + mutex_init(&session->queues_lock);
> > > > > > +
> > > > > > + init_waitqueue_head(&session->dqbuf_wait);
> > > > > > +
> > > > > > + mutex_lock(&vv->sessions_lock);
> > > > > > + list_add_tail(&session->list, &vv->sessions);
> > > > > > + mutex_unlock(&vv->sessions_lock);
> > > > > > +
> > > > > > + return session;
> > > > > > +
> > > > > > +err_payload_sgs:
> > > > > > + kfree(session->shadow_buf);
> > > > > > +err_shadow_buf:
> > > > > > + kfree(session);
> > > > > > +err_session:
> > > > > > + return ERR_PTR(-ENOMEM);
> > > > > > +}
> > > > > > +
> > > > > > +/**
> > > > > > + * virtio_media_session_free - Free all resources of a session.
> > > > > > + * @vv: virtio-media device the session belongs to.
> > > > > > + * @session: session to destroy.
> > > > > > + *
> > > > > > + * All the resources of @sesssion, as well as the backing memory of @session
> > > > > > + * itself, are freed.
> > > > >
> > > > > why @ here and `` above? And typo in the name.
> > > >
> > > > The `@` here was an attempt to follow the guide here for referencing function
> > > > parameters:
> > > > https://docs.kernel.org/doc-guide/kernel-doc.html#highlights-and-cross-references
> > >
> > > Yes. This is part of Linux Kernel kernel-doc markup: when referring to
> > > struct fields, you should use @field (or ``field`` if one wants to place an
> > > asterisk on it, like ``*field``).
> > >
> > > >
> > > > That being said, I don't believe this file is 100% consistent with that. I will
> > > > spend some time cleaning up the comments throughout this patch set to get them
> > > > consistent for v5. Thanks!
> > >
> > > Please use it on a consistent way along the driver.
> > >
> > >
> > > Thanks,
> > > Mauro
> >
^ permalink raw reply
* Re: [PATCH v4 6/8] media: virtio: Add virtio_media_driver
From: Alistair Delva @ 2026-07-20 14:07 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Mauro Carvalho Chehab, Brian Daniels, Mauro Carvalho Chehab,
acourbot, aesteve, changyeon, daniel.almeida, eperezma, gnurou,
gurchetansingh, hverkuil, jasowang, linux-kernel, linux-media,
nicolas.dufresne, virtualization, xuanzhuo
In-Reply-To: <20260720090703-mutt-send-email-mst@kernel.org>
On Mon, Jul 20, 2026 at 6:09 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Mon, Jul 20, 2026 at 05:35:19AM -0700, Alistair Delva wrote:
> > On Sat, Jul 18, 2026 at 10:29 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Sun, Jul 12, 2026 at 08:57:26AM +0200, Mauro Carvalho Chehab wrote:
> > > > On Thu, 25 Jun 2026 16:18:48 -0400
> > > > Brian Daniels <briandaniels@google.com> wrote:
> > > >
> > > > > > > From: Alexandre Courbot <gnurou@gmail.com>
> > > > > > >
> > > > > > > virtio_media_driver.c provides the expected driver hooks, and support
> > > > > > > for mmapping and polling.
> > > > > > >
> > > > > > > Signed-off-by: Alexandre Courbot <gnurou@gmail.com>
> > > > > > > Co-developed-by: Brian Daniels <briandaniels@google.com>
> > > > > > > Signed-off-by: Brian Daniels <briandaniels@google.com>
> > > > > > > ---
> > > > > > > drivers/media/virtio/virtio_media_driver.c | 959 +++++++++++++++++++++
> > > > > > > 1 file changed, 959 insertions(+)
> > > > > > > create mode 100644 drivers/media/virtio/virtio_media_driver.c
> > > > > > >
> > > > > > > diff --git a/drivers/media/virtio/virtio_media_driver.c b/drivers/media/virtio/virtio_media_driver.c
> > > > > > > new file mode 100644
> > > > > > > index 000000000..d6363c673
> > > > > > > --- /dev/null
> > > > > > > +++ b/drivers/media/virtio/virtio_media_driver.c
> > > > > > > @@ -0,0 +1,959 @@
> > > > > > > +// SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0+
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * Virtio-media driver.
> > > > > > > + *
> > > > > > > + * Copyright (c) 2024-2025 Google LLC.
> > > > > > > + */
> > > > > > > +
> > > > > > > +#include <linux/delay.h>
> > > > > > > +#include <linux/device.h>
> > > > > > > +#include <linux/dev_printk.h>
> > > > > > > +#include <linux/mm.h>
> > > > > > > +#include <linux/mutex.h>
> > > > > > > +#include <linux/scatterlist.h>
> > > > > > > +#include <linux/types.h>
> > > > > > > +#include <linux/videodev2.h>
> > > > > > > +#include <linux/vmalloc.h>
> > > > > > > +#include <linux/wait.h>
> > > > > > > +#include <linux/workqueue.h>
> > > > > > > +#include <linux/module.h>
> > > > > > > +#include <linux/moduleparam.h>
> > > > > > > +#include <linux/virtio.h>
> > > > > > > +#include <linux/virtio_config.h>
> > > > > > > +#include <linux/virtio_ids.h>
> > > > > > > +
> > > > > > > +#include <media/frame_vector.h>
> > > > > > > +#include <media/v4l2-dev.h>
> > > > > > > +#include <media/v4l2-event.h>
> > > > > > > +#include <media/videobuf2-memops.h>
> > > > > > > +#include <media/v4l2-device.h>
> > > > > > > +#include <media/v4l2-ioctl.h>
> > > > > > > +
> > > > > > > +#include "protocol.h"
> > > > > > > +#include "session.h"
> > > > > > > +#include "virtio_media.h"
> > > > > > > +
> > > > > > > +#define VIRTIO_MEDIA_NUM_EVENT_BUFS 16
> > > > > > > +
> > > > > > > +/* ID of the SHM region into which MMAP buffer will be mapped. */
> > > > > > > +#define VIRTIO_MEDIA_SHM_MMAP 0
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * Name of the driver to expose to user-space.
> > > > > > > + *
> > > > > > > + * This is configurable because v4l2-compliance has workarounds specific to
> > > > > > > + * some drivers. When proxying these directly from the host, this allows it to
> > > > > > > + * apply them as needed.
> > > > > > > + */
> > > > > > > +char *virtio_media_driver_name;
> > > > > > > +module_param_named(driver_name, virtio_media_driver_name, charp, 0660);
> > > > > >
> > > > > >
> > > > > > Um. What? Not how it should be handled.
> > > > >
> > > > > I can remove this module param. I didn't end up using this when compliance testing.
> > > > > Instead, I patched v4l-utils:
> > > > > https://lore.kernel.org/all/20260528163448.4031965-1-briandaniels@google.com/
> > > > >
> > > > > Let me know if you think the v4l-utils patch is a good approach, otherwise let
> > > > > me know how you'd prefer to address the v4l2-compliance driver-specific workounds
> > > > > when they're being proxied with virtio-media.
> > > >
> > > > This kind of discussion should happen on a separate PR for v4l2-compliance,
> > > > c/c to the proper developers and maintainers of it.
> > >
> > >
> > > I wonder how migration can work when guest is tied to host driver model like
> > > this.
> >
> > We haven't given migration much thought yet, but I think it wouldn't
> > be so different to GPU, where to do snapshot/migration we have to
> > record all initialization / context setup state and replay it against
> > the GPU driver on restore (see https://github.com/google/gfxstream).
> > Some migrations will be possible, some will not, and the host should
> > be able to decide.
> >
> > Also, Brian mentioned the host device pass-through use case, but we
> > also have device implementations on the host that work with camera
> > emulators or some other data source like video/webrtc, which will
> > support snapshot/migration more easily. This use case will probably
> > will see more real-world use and migration will be more relevant.
>
> This is par for the course.
>
>
> But I'm afraid I wasn't clear enough. This thread mentions
> supplying the host driver name to guest userspace
> in order to implement "driver specific work arounds".
Pardon my ignorance, but do a lot of v4l2 apps actually do things
differently for different drivers?
IIUC, this data is so far only used as a command line flag specified
to v4l2-compliance in the guest to pass some tests. It seems like this
information could be provided out of band for that use case.
> But given such, for migration to work userspace needs
> to be notified about driver change? And what to do about
> the things it started after migration but before the
> notification?
If this was used in a local development environment or lab, I think an
equipment change is unlikely or could be avoided by the setup. IMO,
the driver actually changing during a migration is an unlikely use
case and the host could just fail to restore the guest if a driver
change is detected.
That's why I brought up GPU - we might not support e.g. migrating from
nvidia to amd, but that's probably fine and migration is still useful
most of the time.
> > > > >
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * Whether USERPTR buffers are allowed.
> > > > > > > + *
> > > > > > > + * This is disabled by default as USERPTR buffers are dangerous, but the option
> > > > > > > + * is left to enable them if desired.
> > > > > > > + */
> > > > > > > +bool virtio_media_allow_userptr;
> > > > > > > +module_param_named(allow_userptr, virtio_media_allow_userptr, bool, 0660);
> > > > > >
> > > > > >
> > > > > > is this kind of thing common?
> > > >
> > > > There is one old media device that has it (saa7134).
> > > >
> > > > >
> > > > > To be honest, I don't really know. I'm also not that familiar with the USERPTR
> > > > > issues. I see a few references online about their use being discouraged due to
> > > > > possible race conditions, perhaps that was the original motivation for this
> > > > > parameter (I'm not the original author of this driver).
> > > > >
> > > > > I'm open to alternatives, feel free to let me know if you have a preference.
> > > >
> > > > We tend to not implement USERPTR on newer drivers. I suggest you
> > > > to place the logic with regards to V4L2_MEMORY_USERPTR on a separate
> > > > patch for further discussions.
> > > >
> > > > >
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * virtio_media_session_alloc - Allocate a new session.
> > > > > > > + * @vv: virtio-media device the session belongs to.
> > > > > > > + * @id: ID of the session.
> > > > > > > + * @nonblocking_dequeue: whether dequeuing of buffers should be blocking or
> > > > > > > + * not.
> > > > > > > + *
> > > > > > > + * The ``id`` and ``list`` fields must still be set by the caller.
> > > > > >
> > > > > > still in what sense?
> > > > >
> > > > > Based on the code below, I'm not so sure that the caller is responsible for
> > > > > setting these values. They seem to be initialized in the function.
> > > > >
> > > > > Perhaps Alexandre Courbot (the original author) would know more. Unless he
> > > > > says otherwise though I'm inclined to remove this comment.
> > > > >
> > > > > > > + */
> > > > > > > +static struct virtio_media_session *
> > > > > > > +virtio_media_session_alloc(struct virtio_media *vv, u32 id,
> > > > > > > + struct file *file)
> > > > > > > +{
> > > > > > > + struct virtio_media_session *session;
> > > > > > > + int i;
> > > > > > > + int ret;
> > > > > > > +
> > > > > > > + session = kzalloc_obj(*session, GFP_KERNEL);
> > > > > > > + if (!session)
> > > > > > > + goto err_session;
> > > > > > > +
> > > > > > > + session->shadow_buf = kzalloc(VIRTIO_SHADOW_BUF_SIZE, GFP_KERNEL);
> > > > > > > + if (!session->shadow_buf)
> > > > > > > + goto err_shadow_buf;
> > > > > > > +
> > > > > > > + ret = sg_alloc_table(&session->command_sgs, DESC_CHAIN_MAX_LEN,
> > > > > > > + GFP_KERNEL);
> > > > > > > + if (ret)
> > > > > > > + goto err_payload_sgs;
> > > > > > > +
> > > > > > > + session->id = id;
> > > > > > > + session->nonblocking_dequeue = file->f_flags & O_NONBLOCK;
> > > > > > > +
> > > > > > > + INIT_LIST_HEAD(&session->list);
> > > > > > > + v4l2_fh_init(&session->fh, &vv->video_dev);
> > > > > > > + virtio_media_session_fh_add(session, file);
> > > > > > > +
> > > > > > > + for (i = 0; i <= VIRTIO_MEDIA_LAST_QUEUE; i++)
> > > > > > > + INIT_LIST_HEAD(&session->queues[i].pending_dqbufs);
> > > > > > > + mutex_init(&session->queues_lock);
> > > > > > > +
> > > > > > > + init_waitqueue_head(&session->dqbuf_wait);
> > > > > > > +
> > > > > > > + mutex_lock(&vv->sessions_lock);
> > > > > > > + list_add_tail(&session->list, &vv->sessions);
> > > > > > > + mutex_unlock(&vv->sessions_lock);
> > > > > > > +
> > > > > > > + return session;
> > > > > > > +
> > > > > > > +err_payload_sgs:
> > > > > > > + kfree(session->shadow_buf);
> > > > > > > +err_shadow_buf:
> > > > > > > + kfree(session);
> > > > > > > +err_session:
> > > > > > > + return ERR_PTR(-ENOMEM);
> > > > > > > +}
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * virtio_media_session_free - Free all resources of a session.
> > > > > > > + * @vv: virtio-media device the session belongs to.
> > > > > > > + * @session: session to destroy.
> > > > > > > + *
> > > > > > > + * All the resources of @sesssion, as well as the backing memory of @session
> > > > > > > + * itself, are freed.
> > > > > >
> > > > > > why @ here and `` above? And typo in the name.
> > > > >
> > > > > The `@` here was an attempt to follow the guide here for referencing function
> > > > > parameters:
> > > > > https://docs.kernel.org/doc-guide/kernel-doc.html#highlights-and-cross-references
> > > >
> > > > Yes. This is part of Linux Kernel kernel-doc markup: when referring to
> > > > struct fields, you should use @field (or ``field`` if one wants to place an
> > > > asterisk on it, like ``*field``).
> > > >
> > > > >
> > > > > That being said, I don't believe this file is 100% consistent with that. I will
> > > > > spend some time cleaning up the comments throughout this patch set to get them
> > > > > consistent for v5. Thanks!
> > > >
> > > > Please use it on a consistent way along the driver.
> > > >
> > > >
> > > > Thanks,
> > > > Mauro
> > >
>
^ permalink raw reply
* [RFC PATCH 1/4] common: add UAPI for SQ/CQ doorbell polling
From: rom.wang @ 2026-07-20 14:10 UTC (permalink / raw)
To: mst, jasowangio, eperezma, stefanha
Cc: virtualization, linux-kernel, linux-scsi, kvm, qemu-devel,
Yufeng Wang
In-Reply-To: <20260720141040.181946-1-r4o5m6e8o@163.com>
From: Yufeng Wang <wangyufeng@kylinos.cn>
Define VIRTIO_F_SQCQ_POLL (bit 42) and the associated SQ/CQ doorbell
structures for split virtqueue polling mode.
- include/uapi/linux/virtio_config.h: feature bit 42
- include/uapi/linux/virtio_ring.h: struct vring_sq, struct vring_cq
(128-byte aligned for cache-line isolation on ARM Neoverse N2/V2)
- include/uapi/linux/virtio_pci.h: sqe/cqe_ring PCI config registers
at offsets 64-76
Signed-off-by: Yufeng Wang <wangyufeng@kylinos.cn>
---
include/uapi/linux/virtio_config.h | 8 +++++++-
include/uapi/linux/virtio_pci.h | 9 +++++++++
include/uapi/linux/virtio_ring.h | 25 +++++++++++++++++++++++++
3 files changed, 41 insertions(+), 1 deletion(-)
diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio_config.h
index 2445f365bce7..dbda23e44b62 100644
--- a/include/uapi/linux/virtio_config.h
+++ b/include/uapi/linux/virtio_config.h
@@ -52,7 +52,7 @@
* rest are per-device feature bits.
*/
#define VIRTIO_TRANSPORT_F_START 28
-#define VIRTIO_TRANSPORT_F_END 42
+#define VIRTIO_TRANSPORT_F_END 43
#ifndef VIRTIO_CONFIG_NO_LEGACY
/* Do we get callbacks when the ring is completely used, even if we've
@@ -120,4 +120,10 @@
*/
#define VIRTIO_F_ADMIN_VQ 41
+/*
+ * This feature indicates that both driver and device support
+ * SQ/CQ polling mode to replace interrupt-based notifications.
+ */
+#define VIRTIO_F_SQCQ_POLL 42
+
#endif /* _UAPI_LINUX_VIRTIO_CONFIG_H */
diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h
index e732e3456e27..f5058d1b4b16 100644
--- a/include/uapi/linux/virtio_pci.h
+++ b/include/uapi/linux/virtio_pci.h
@@ -193,6 +193,11 @@ struct virtio_pci_modern_common_cfg {
__le16 admin_queue_index; /* read-only */
__le16 admin_queue_num; /* read-only */
+
+ __le32 sqe_ring_lo; /* read-write: SQ DMA addr low */
+ __le32 sqe_ring_hi; /* read-write: SQ DMA addr high */
+ __le32 cqe_ring_lo; /* read-write: CQ DMA addr low */
+ __le32 cqe_ring_hi; /* read-write: CQ DMA addr high */
};
/* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
@@ -235,6 +240,10 @@ struct virtio_pci_cfg_cap {
#define VIRTIO_PCI_COMMON_Q_RESET 58
#define VIRTIO_PCI_COMMON_ADM_Q_IDX 60
#define VIRTIO_PCI_COMMON_ADM_Q_NUM 62
+#define VIRTIO_PCI_COMMON_SQE_LO 64
+#define VIRTIO_PCI_COMMON_SQE_HI 68
+#define VIRTIO_PCI_COMMON_CQE_LO 72
+#define VIRTIO_PCI_COMMON_CQE_HI 76
#endif /* VIRTIO_PCI_NO_MODERN */
diff --git a/include/uapi/linux/virtio_ring.h b/include/uapi/linux/virtio_ring.h
index 3c478582a3c2..d1f2b9584d7c 100644
--- a/include/uapi/linux/virtio_ring.h
+++ b/include/uapi/linux/virtio_ring.h
@@ -131,6 +131,31 @@ struct vring_used {
vring_used_elem_t ring[];
};
+/*
+ * SQ/CQ doorbell structures for polling mode (VIRTIO_F_SQCQ_POLL).
+ * These are allocated separately from the vring and their DMA addresses
+ * are communicated via PCI config space registers.
+ *
+ * Ownership:
+ * SQ: Guest writes idx, Device reads it.
+ * CQ: Device writes idx, Guest reads it.
+ *
+ * Flags (8-bit):
+ * Set to 1 before sleeping, cleared to 0 after waking up.
+ * Direct assignment — no read-modify-write.
+ */
+struct vring_sq {
+ __virtio64 idx; /* Producer position (updated by guest) */
+ __u8 sq_need_wakeup; /* 1 = device should be woken on next kick */
+ __u8 reserved[55]; /* Reserved for future use */
+} __attribute__((aligned(128)));
+
+struct vring_cq {
+ __virtio64 idx; /* Producer position (updated by device) */
+ __u8 cq_need_wakeup; /* 1 = guest should be woken on completion */
+ __u8 reserved[55]; /* Reserved for future use */
+} __attribute__((aligned(128)));
+
/*
* The ring element addresses are passed between components with different
* alignments assumptions. Thus, we might need to decrease the compiler-selected
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 0/4] virtio: SQ/CQ doorbell polling for vhost-scsi
From: rom.wang @ 2026-07-20 14:10 UTC (permalink / raw)
To: mst, jasowangio, eperezma, stefanha
Cc: virtualization, linux-kernel, linux-scsi, kvm, qemu-devel,
Yufeng Wang
From: Yufeng Wang <wangyufeng@kylinos.cn>
This RFC proposes a new virtio transport feature, VIRTIO_F_SQCQ_POLL,
that eliminates VM exits on both submission and completion paths for
vhost-scsi by using shared-memory doorbells and kernel polling
threads, following the io_uring SQPOLL model.
This is an early RFC to gather design feedback. The implementation is
functional and has been tested on arm64 and x86_64. We are not
requesting merge at this time.
Problem
-------
vhost-scsi uses MMIO writes (Guest -> Host) and MSI-X interrupts
(Host -> Guest) for notification. Each notification involves a VM exit,
which becomes a bottleneck at high IOPS:
- 4K random read, QD32, 8 jobs: 341K IOPS baseline
- With ~340K VM exits/second, the exit overhead dominates
Existing mitigations (vhost-net's tx polling, blk-mq iopoll) only
address one direction or require the submitting task to poll. Neither
eliminates VM exits on both paths simultaneously.
Solution
--------
Introduce two cache-line-aligned doorbell structures, SQ (Submission
Queue) and CQ (Completion Queue), placed alongside the standard split
virtqueue:
- Guest writes sq->idx instead of MMIO kick; Host poll thread
detects the change and processes submissions.
- Host writes cq->idx instead of MSI-X interrupt; Guest poll
thread detects the change and invokes completion callbacks.
A NEED_WAKEUP protocol (mirroring io_uring's SQ_NEED_WAKEUP) allows
either side to sleep when idle, with the other side responsible for
waking it via eventfd.
Feature negotiation via VIRTIO_F_SQCQ_POLL (bit 42) ensures zero
overhead when not negotiated — the driver falls back to traditional
MMIO kick + MSI-X interrupt.
Performance
-----------
Benchmark: fio, 4K random I/O
Test configuration:
arm64:
CPU: Kunpeng 920 (2.6GHz), 8 vCPUs
Disk: NVMe INTEL SSDPED1K375GA (375GB)
x86_64:
CPU: Intel Xeon E5-2680 v4 @ 2.40GHz, 8 vCPUs
Disk: NVMe SAMSUNG MZ1LB960HAJQ-000MV (960GB)
Backend: vhost-scsi with TCM loopback to NVMe device
QEMU: vhost-scsi-pci with VIRTIO_F_SQCQ_POLL negotiated
arm64 results:
Test Baseline SQ/CQ Poll Change
----------- ---------- ---------- -------
randread QD1 22,427 28,289 +26%
randread QD32 NJ1 89,910 75,665 -16%
randread QD32 NJ4 186,763 379,549 +103%
randread QD32 NJ8 199,967 550,633 +175%
randwrite QD1 21,912 27,261 +24%
randwrite QD32 NJ1 85,349 81,389 -5%
randwrite QD32 NJ4 190,443 355,811 +87%
randwrite QD32 NJ8 196,552 566,640 +188%
x86_64 results:
Test Baseline SQ/CQ Poll Change
----------- ---------- ---------- -------
randread QD1 8,263 9,552 +16%
randread QD32 NJ1 127,412 162,805 +28%
randread QD32 NJ4 303,208 375,056 +24%
randread QD32 NJ8 341,625 371,193 +9%
randwrite QD1 20,773 30,332 +46%
randwrite QD32 NJ1 133,316 159,207 +19%
randwrite QD32 NJ4 233,373 229,224 -2%
randwrite QD32 NJ8 231,442 231,676 +0%
Multi-queue workloads (NJ4/NJ8) see significant improvement on arm64
(87-188%) and moderate improvement on x86_64 (9-24%). Single-VQ
high-queue-depth workloads show a minor regression on arm64 due to
polling overhead vs. VM-exit savings trade-off, while x86_64 shows
improvement across most configurations (16-46% for QD1, 19-28%
for QD32-NJ1) due to lower per-VM-exit cost on x86.
Why Not vDPA?
-------------
vhost-vDPA already provides doorbell mmap and polling. A reasonable
reviewer would ask: why not extend vhost-vDPA instead?
Three reasons:
1. No vdpa-scsi device exists. The vDPA framework
(drivers/vdpa/) currently has hardware devices for net (mlx5,
ifcvf, etc.) and software devices for net and blk (vdpa_sim).
There is no virtio-scsi vDPA device, hardware or software.
Building one means re-implementing vhost-scsi's TCM integration
(SCSI CDB processing, ALUA, persistent reservations) under the
vDPA device abstraction — 3-5x the work of extending vhost-scsi.
2. vhost-scsi is a deployed interface. libvirt, QEMU, and
OpenStack have vhost-scsi configuration APIs and operational
tooling. Switching to vhost-vdpa requires a new backend, user
migration, and toolchain updates. SQ/CQ poll as a vhost-scsi
feature is fully backward-compatible — no existing deployments
break.
3. The protocol is transport-agnostic. The SQ/CQ doorbell design
(struct vring_sq, struct vring_cq, NEED_WAKEUP handshake) is
orthogonal to vhost vs. vDPA. The same UAPI can be consumed by
vhost-scsi today and a future vdpa-scsi device. Implementing in
vhost-scsi first does not block future vDPA integration.
We acknowledge that vDPA is the long-term direction for virtio
backends. If this SQ/CQ poll protocol is accepted, it can be ported
to the vDPA framework; a vdpa-scsi device is independent work.
Patch Structure
---------------
Patch 1: UAPI definitions (virtio_config.h, virtio_ring.h,
virtio_pci.h) — shared interface for all components
Patch 2: vhost kernel support (vhost.c, vhost.h, scsi.c,
vhost.h UAPI, vhost_types.h UAPI) — Host poll thread
Patch 3: virtio guest driver (virtio_ring.c, virtio_sqcq_poll.c,
virtio_pci_modern.c, virtio.c, virtio_scsi.c) — Guest
poll thread and submission path
Patch 4: QEMU support (virtio-pci.c, vhost.c) — PCI config
forwarding and vhost ioctl bridge
Patches 1-3 apply to the Linux kernel tree. Patch 4 applies to
the QEMU tree separately.
Spec Status
-----------
A virtio-spec format document has been prepared and will be submitted
to the OASIS virtio TC as a proposal. This RFC stage seeks design
feedback before initiating the formal spec process.
Known Limitations (Future Work)
-------------------------------
- CPU hotplug: no notifier registered; poll thread may be
migrated when its CPU goes offline. Planned: kthread_park +
dynamic rebind.
- Live migration: no explicit stop/flush coordination during
migration. Planned: VHOST_BACKEND_F_SUSPEND/RESUME integration.
- SMAP overhead: Host uses get_user/put_user for doorbell access.
Future optimization: GUP + kmap to map pages into kernel space.
- Backpressure: no "slow down" signal from device to driver.
Future: CQ throttle flag or avail_event reuse.
- Packed ring: not supported; explicitly rejected at feature
negotiation. Future: add packed ring doorbell support.
RFC Goals
---------
1. Validate the overall design direction (doorbell + polling
model, NEED_WAKEUP protocol)
2. Get feedback on the feature bit allocation (42) and UAPI
structure design
3. Understand whether vDPA concern is a blocker or can be
addressed with the transport-agnostic argument
4. Collect guidance on prioritizing future work items
(hotplug, migration, spec process)
We welcome all feedback, especially on:
- Whether the NEED_WAKEUP protocol design is sound
- Whether the per-device Guest poll thread model (vs per-VQ)
is acceptable
- Whether feature bit 42 is appropriate or if a different
allocation is needed
- Any concerns about the SMAP overhead in the Host poll path
How to Test
-----------
Patch usage:
Guest Kernel: apply patch 1 (UAPI) + patch 3 (guest driver)
Host Kernel: apply patch 1 (UAPI) + patch 2 (vhost support)
QEMU: apply patch 4 (vhost-scsi bridge)
1. Set up vhost-scsi target on the host (see:
https://wiki.libvirt.org/Vhost-scsi_target.html#Host_Setup)
using targetcli to create a TCM loopback device, e.g.:
targetcli /backstores/loopback create dev=/dev/sda
targetcli /vhost create naa.5001405376e34400
targetcli /vhost/naa.5001405376e34400/lun create \
/backstores/loopback/dev,/dev/sda
2. Boot VM with vhost-scsi using patched QEMU:
qemu-system-aarch64 ... \
-device vhost-scsi-pci,wwpn=naa.5001405376e34400
3. Verify SQ/CQ poll mode is active on the host:
dmesg | grep "vhost-sqcq"
# Expected: "vhost-sqcq: vq[N] poll thread bound to cpuN"
# and 10s stats: "vhost-sqcq: vq[N] cq=... ema_lat=... interval=..."
4. Verify feature negotiated in guest:
dmesg | grep "VIRTIO_F_SQCQ_POLL"
# Expected: "VIRTIO_F_SQCQ_POLL negotiated, starting poll thread"
# and 10s stats: "io_stats: cq=... avg_lat=... interval=..."
5. Run fio benchmark (compare with unpatched QEMU/kernel baseline):
fio --name=randread --rw=randread --bs=4k --iodepth=32 \
--numjobs=4 --runtime=60 --time_based --direct=1 \
--filename=/dev/sda
Test scripts (run-sqcq-compare.sh, compare-sqcq-results.sh) are
available and will be sent as a follow-up to this RFC.
Thank you for your time.
---
Yufeng Wang (4):
common: add UAPI for SQ/CQ doorbell polling
vhost: host kernel support for SQ/CQ polling
virtio: guest driver support for SQ/CQ polling
qemu: add SQ/CQ polling mode support for vhost-scsi
Patch 1 (UAPI, 3 files): +41 -1
Patch 2 (vhost, 5 files): +794 -20
Patch 3 (virtio guest, 10 files): +840 -3
Patch 4 (QEMU, 14 files): +179 -3
Total: 32 files changed, 1854 insertions(+), 27 deletions(-)
--
2.34.1
^ permalink raw reply
* [RFC PATCH 3/4] virtio: guest driver support for SQ/CQ polling
From: rom.wang @ 2026-07-20 14:10 UTC (permalink / raw)
To: mst, jasowangio, eperezma, stefanha
Cc: virtualization, linux-kernel, linux-scsi, kvm, qemu-devel,
Yufeng Wang
In-Reply-To: <20260720141040.181946-1-r4o5m6e8o@163.com>
From: Yufeng Wang <wangyufeng@kylinos.cn>
Implement SQ/CQ doorbell polling in the guest kernel:
- virtio_ring.c: SQ/CQ DMA allocation, virtqueue_notify() writes
sq->idx via smp_store_release instead of MMIO kick; completion
detection via more_used() (direct used-ring check)
- virtio_sqcq_poll.c: per-device CQ poll thread with need_resched()
cooperative spin, time-based spin budget, 1ms idle sleep
- virtio_pci_modern.c: feature negotiation, PCI config space write
for SQ/CQ DMA addresses, poll thread lifecycle
- virtio.c: reject VIRTIO_F_SQCQ_POLL + VIRTIO_F_RING_PACKED
- virtio_scsi.c: 10s window io_stats diagnostics output
- do_softirq() after each completion callback (no hardware interrupt)
Signed-off-by: Yufeng Wang <wangyufeng@kylinos.cn>
---
drivers/scsi/virtio_scsi.c | 113 ++++++++-
drivers/virtio/Makefile | 2 +-
drivers/virtio/virtio.c | 10 +
drivers/virtio/virtio_pci_common.c | 3 +
drivers/virtio/virtio_pci_modern.c | 39 +++
drivers/virtio/virtio_pci_modern_dev.c | 34 ++-
drivers/virtio/virtio_ring.c | 262 +++++++++++++++++++
drivers/virtio/virtio_sqcq_poll.c | 338 +++++++++++++++++++++++++
include/linux/virtio.h | 25 ++
include/linux/virtio_pci_modern.h | 2 +
10 files changed, 825 insertions(+), 3 deletions(-)
create mode 100644 drivers/virtio/virtio_sqcq_poll.c
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index 0ed8558dad72..f803bb62d22a 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -30,6 +30,7 @@
#include <scsi/scsi_devinfo.h>
#include <linux/seqlock.h>
#include <linux/dma-mapping.h>
+#include <linux/ktime.h>
#include "sd.h"
@@ -46,6 +47,8 @@ MODULE_PARM_DESC(virtscsi_poll_queues,
struct virtio_scsi_cmd {
struct scsi_cmnd *sc;
struct completion *comp;
+ u64 submit_ns; /* ktime_get_ns() at submission */
+ u64 notify_ns; /* ktime_get_ns() after SQ doorbell */
union {
struct virtio_scsi_cmd_req cmd;
struct virtio_scsi_cmd_req_pi cmd_pi;
@@ -95,6 +98,22 @@ struct virtio_scsi {
struct virtio_scsi_event events[VIRTIO_SCSI_EVENT_LEN];
__dma_from_device_group_end();
+ /* I/O latency statistics (debugging) */
+ spinlock_t lat_lock;
+ u64 lat_sum_ns;
+ u64 lat_min_ns;
+ u64 lat_max_ns;
+ u64 lat_count;
+ u64 host_sum_ns; /* notify → cq_detect (host transit) */
+ u64 cb_wait_sum_ns; /* cq_detect → process (poll sched latency) */
+ u64 det_count;
+ u64 last_cq_detect_ns; /* timestamp of last CQ detection */
+ unsigned long lat_last_print;
+
+ /* Completion interval tracking (matches host interval metric) */
+ u64 last_complete_ns; /* ktime of last completion */
+ u64 ema_cmp_int_ns; /* EMA of completion-to-completion interval */
+
struct virtio_scsi_vq req_vqs[];
};
@@ -122,6 +141,7 @@ static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
struct virtio_scsi_cmd *cmd = buf;
struct scsi_cmnd *sc = cmd->sc;
struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
+ u64 cb_entry_ns = ktime_get_ns();
dev_dbg(&sc->device->sdev_gendev,
"cmd %p response %u status %#02x sense_len %u\n",
@@ -175,6 +195,88 @@ static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
VIRTIO_SCSI_SENSE_SIZE));
}
+ /* I/O latency tracking */
+ if (cmd->submit_ns) {
+ u64 now = ktime_get_ns();
+ u64 lat = now - cmd->submit_ns;
+ unsigned long flags;
+
+ spin_lock_irqsave(&vscsi->lat_lock, flags);
+ vscsi->lat_sum_ns += lat;
+ vscsi->lat_count++;
+ if (lat < vscsi->lat_min_ns || vscsi->lat_min_ns == 0)
+ vscsi->lat_min_ns = lat;
+ if (lat > vscsi->lat_max_ns)
+ vscsi->lat_max_ns = lat;
+
+ /* Completion interval EMA (matches host interval metric) */
+ if (vscsi->last_complete_ns) {
+ u64 cmp_int = now - vscsi->last_complete_ns;
+
+ if (vscsi->ema_cmp_int_ns == 0)
+ vscsi->ema_cmp_int_ns = cmp_int;
+ else
+ vscsi->ema_cmp_int_ns =
+ vscsi->ema_cmp_int_ns -
+ vscsi->ema_cmp_int_ns / 8 +
+ cmp_int / 8;
+ }
+ vscsi->last_complete_ns = now;
+
+ if (cmd->notify_ns && vscsi->last_cq_detect_ns) {
+ u64 host_lat = vscsi->last_cq_detect_ns -
+ cmd->notify_ns;
+ u64 cb_wait = cb_entry_ns -
+ vscsi->last_cq_detect_ns;
+
+ if (host_lat < 10ULL * NSEC_PER_SEC) {
+ vscsi->host_sum_ns += host_lat;
+ vscsi->cb_wait_sum_ns += cb_wait;
+ vscsi->det_count++;
+ }
+ }
+
+ /* 10s window — single consolidated output */
+ if (time_after(jiffies, vscsi->lat_last_print +
+ msecs_to_jiffies(10000)) &&
+ vscsi->lat_count > 0) {
+ u64 avg_lat = vscsi->lat_sum_ns / vscsi->lat_count;
+ u64 host_lat = vscsi->det_count > 0 ?
+ vscsi->host_sum_ns / vscsi->det_count : 0;
+ u64 poll_lat = vscsi->det_count > 0 ?
+ vscsi->cb_wait_sum_ns / vscsi->det_count : 0;
+ u64 interval = 0;
+ unsigned int wk_kick;
+ unsigned long sleep_cnt;
+
+ wk_kick = virtqueue_get_sq_notify_stats(
+ vscsi->req_vqs[0].vq, &interval);
+ sleep_cnt = virtio_sqcq_poll_get_sleep_count(vscsi->vdev);
+
+ dev_info(&vscsi->vdev->dev,
+ "io_stats: cq=%llu avg_lat=%lluns host_lat=%lluns poll_lat=%lluns min_lat=%lluns max_lat=%lluns interval=%lluns cmp_interval=%lluns wk_kick=%u sleep=%lu\n",
+ vscsi->lat_count,
+ avg_lat,
+ host_lat,
+ poll_lat,
+ vscsi->lat_min_ns,
+ vscsi->lat_max_ns,
+ interval,
+ vscsi->ema_cmp_int_ns,
+ wk_kick,
+ sleep_cnt);
+ vscsi->lat_sum_ns = 0;
+ vscsi->lat_count = 0;
+ vscsi->lat_min_ns = 0;
+ vscsi->lat_max_ns = 0;
+ vscsi->host_sum_ns = 0;
+ vscsi->cb_wait_sum_ns = 0;
+ vscsi->det_count = 0;
+ vscsi->lat_last_print = jiffies;
+ }
+ spin_unlock_irqrestore(&vscsi->lat_lock, flags);
+ }
+
scsi_done(sc);
}
@@ -204,6 +306,7 @@ static void virtscsi_req_done(struct virtqueue *vq)
int index = vq->index - VIRTIO_SCSI_VQ_BASE;
struct virtio_scsi_vq *req_vq = &vscsi->req_vqs[index];
+ vscsi->last_cq_detect_ns = virtqueue_get_cq_detect_ns(vq);
virtscsi_vq_done(vscsi, req_vq, virtscsi_complete_cmd);
};
@@ -514,8 +617,12 @@ static int virtscsi_add_cmd(struct virtio_scsi_vq *vq,
spin_unlock_irqrestore(&vq->vq_lock, flags);
- if (needs_kick)
+ if (needs_kick) {
+ cmd->notify_ns = ktime_get_ns();
virtqueue_notify(vq->vq);
+ } else {
+ cmd->notify_ns = 0;
+ }
return err;
}
@@ -605,6 +712,7 @@ static enum scsi_qc_status virtscsi_queuecommand(struct Scsi_Host *shost,
}
kick = (sc->flags & SCMD_LAST) != 0;
+ cmd->submit_ns = ktime_get_ns();
ret = virtscsi_add_cmd(req_vq, cmd, req_size, sizeof(cmd->resp.cmd), kick);
if (ret == -EIO) {
cmd->resp.cmd.response = VIRTIO_SCSI_S_BAD_TARGET;
@@ -895,6 +1003,9 @@ static int virtscsi_init(struct virtio_device *vdev,
virtscsi_init_vq(&vscsi->req_vqs[i - VIRTIO_SCSI_VQ_BASE],
vqs[i]);
+ spin_lock_init(&vscsi->lat_lock);
+ vscsi->lat_last_print = jiffies;
+
virtscsi_config_set(vdev, cdb_size, VIRTIO_SCSI_CDB_SIZE);
virtscsi_config_set(vdev, sense_size, VIRTIO_SCSI_SENSE_SIZE);
diff --git a/drivers/virtio/Makefile b/drivers/virtio/Makefile
index eefcfe90d6b8..769e95ee5be4 100644
--- a/drivers/virtio/Makefile
+++ b/drivers/virtio/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_VIRTIO) += virtio.o virtio_ring.o
+obj-$(CONFIG_VIRTIO) += virtio.o virtio_ring.o virtio_sqcq_poll.o
obj-$(CONFIG_VIRTIO_ANCHOR) += virtio_anchor.o
obj-$(CONFIG_VIRTIO_PCI_LIB) += virtio_pci_modern_dev.o
obj-$(CONFIG_VIRTIO_PCI_LIB_LEGACY) += virtio_pci_legacy_dev.o
diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c
index 5bdc6b82b30b..d440e920f8e6 100644
--- a/drivers/virtio/virtio.c
+++ b/drivers/virtio/virtio.c
@@ -221,6 +221,16 @@ static int virtio_features_ok(struct virtio_device *dev)
}
}
+ /* SQ/CQ poll only supports split ring; reject packed ring
+ * to avoid split-specific field access in virtqueue_notify().
+ */
+ if (virtio_has_feature(dev, VIRTIO_F_SQCQ_POLL) &&
+ virtio_has_feature(dev, VIRTIO_F_RING_PACKED)) {
+ dev_warn(&dev->dev,
+ "VIRTIO_F_SQCQ_POLL is incompatible with VIRTIO_F_RING_PACKED\n");
+ return -ENODEV;
+ }
+
if (!virtio_has_feature(dev, VIRTIO_F_VERSION_1))
return 0;
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index da97b6a988de..cd22651c73e2 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -268,6 +268,9 @@ void vp_del_vqs(struct virtio_device *vdev)
struct virtqueue *vq, *n;
int i;
+ /* Stop SQ/CQ poll thread before deleting virtqueues */
+ virtio_stop_sqcq_poll(vdev);
+
list_for_each_entry_safe(vq, n, &vdev->vqs, list) {
info = vp_is_avq(vdev, vq->index) ? vp_dev->admin_vq.info :
vp_dev->vqs[vq->index];
diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c
index 6d8ae2a6a8ca..7a36ec4584a6 100644
--- a/drivers/virtio/virtio_pci_modern.c
+++ b/drivers/virtio/virtio_pci_modern.c
@@ -18,6 +18,7 @@
#include <linux/virtio_pci_admin.h>
#define VIRTIO_PCI_NO_LEGACY
#define VIRTIO_RING_NO_LEGACY
+
#include "virtio_pci_common.h"
#define VIRTIO_AVQ_SGS_MAX 4
@@ -378,6 +379,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features)
if (features & BIT_ULL(VIRTIO_F_ADMIN_VQ))
__virtio_set_bit(vdev, VIRTIO_F_ADMIN_VQ);
+
+ if (features & BIT_ULL(VIRTIO_F_SQCQ_POLL))
+ __virtio_set_bit(vdev, VIRTIO_F_SQCQ_POLL);
}
static int __vp_check_common_size_one_feature(struct virtio_device *vdev, u32 fbit,
@@ -413,6 +417,9 @@ static int vp_check_common_size(struct virtio_device *vdev)
if (vp_check_common_size_one_feature(vdev, VIRTIO_F_ADMIN_VQ, admin_queue_num))
return -EINVAL;
+ if (vp_check_common_size_one_feature(vdev, VIRTIO_F_SQCQ_POLL, cqe_ring_hi))
+ return -EINVAL;
+
return 0;
}
@@ -578,6 +585,11 @@ static int vp_active_vq(struct virtqueue *vq, u16 msix_vec)
virtqueue_get_avail_addr(vq),
virtqueue_get_used_addr(vq));
+ if (virtqueue_sqcq_poll_active(vq))
+ vp_modern_queue_address_sqcq(mdev, index,
+ virtqueue_get_sq_dma_addr(vq),
+ virtqueue_get_cq_dma_addr(vq));
+
if (msix_vec != VIRTIO_MSI_NO_VECTOR) {
msix_vec = vp_modern_queue_vector(mdev, index, msix_vec);
if (msix_vec == VIRTIO_MSI_NO_VECTOR)
@@ -753,6 +765,27 @@ static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
if (rc)
return rc;
+ /* Start SQ/CQ poll thread if feature negotiated */
+ if (virtio_has_feature(vdev, VIRTIO_F_SQCQ_POLL)) {
+ dev_info(&vdev->dev, "VIRTIO_F_SQCQ_POLL negotiated, starting poll thread\n");
+ rc = virtio_start_sqcq_poll(vdev, 1000, 1000);
+ if (rc)
+ goto err_del_vqs;
+
+ list_for_each_entry(vq, &vdev->vqs, list) {
+ if (!virtqueue_sqcq_poll_active(vq))
+ continue;
+ rc = virtio_sqcq_poll_register_vq(vdev, vq);
+ if (rc) {
+ dev_err(&vdev->dev,
+ "failed to register vq %d (%s) with poll thread: %d\n",
+ vq->index,
+ vq->name ? vq->name : "?", rc);
+ goto err_stop_poll;
+ }
+ }
+ }
+
/* Select and activate all queues. Has to be done last: once we do
* this, there's no way to go back except reset.
*/
@@ -760,6 +793,12 @@ static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
vp_modern_set_queue_enable(&vp_dev->mdev, vq->index, true);
return 0;
+
+err_stop_poll:
+ virtio_stop_sqcq_poll(vdev);
+err_del_vqs:
+ vp_del_vqs(vdev);
+ return rc;
}
static void del_vq(struct virtio_pci_vq_info *info)
diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c
index 413a8c353463..642b9a58751c 100644
--- a/drivers/virtio/virtio_pci_modern_dev.c
+++ b/drivers/virtio/virtio_pci_modern_dev.c
@@ -211,6 +211,14 @@ static inline void check_offsets(void)
offsetof(struct virtio_pci_modern_common_cfg, admin_queue_index));
BUILD_BUG_ON(VIRTIO_PCI_COMMON_ADM_Q_NUM !=
offsetof(struct virtio_pci_modern_common_cfg, admin_queue_num));
+ BUILD_BUG_ON(VIRTIO_PCI_COMMON_SQE_LO !=
+ offsetof(struct virtio_pci_modern_common_cfg, sqe_ring_lo));
+ BUILD_BUG_ON(VIRTIO_PCI_COMMON_SQE_HI !=
+ offsetof(struct virtio_pci_modern_common_cfg, sqe_ring_hi));
+ BUILD_BUG_ON(VIRTIO_PCI_COMMON_CQE_LO !=
+ offsetof(struct virtio_pci_modern_common_cfg, cqe_ring_lo));
+ BUILD_BUG_ON(VIRTIO_PCI_COMMON_CQE_HI !=
+ offsetof(struct virtio_pci_modern_common_cfg, cqe_ring_hi));
}
/*
@@ -300,7 +308,7 @@ int vp_modern_probe(struct virtio_pci_modern_device *mdev)
mdev->common = vp_modern_map_capability(mdev, common,
sizeof(struct virtio_pci_common_cfg), 4, 0,
offsetofend(struct virtio_pci_modern_common_cfg,
- admin_queue_num),
+ cqe_ring_hi),
&mdev->common_len, NULL);
if (!mdev->common)
goto err_map_common;
@@ -607,6 +615,30 @@ void vp_modern_queue_address(struct virtio_pci_modern_device *mdev,
}
EXPORT_SYMBOL_GPL(vp_modern_queue_address);
+/*
+ * vp_modern_queue_address_sqcq - set the SQ/CQ doorbell DMA addresses
+ * @mdev: the modern virtio-pci device
+ * @index: the queue index
+ * @sq_addr: DMA address of the SQ doorbell
+ * @cq_addr: DMA address of the CQ doorbell
+ */
+void vp_modern_queue_address_sqcq(struct virtio_pci_modern_device *mdev,
+ u16 index, u64 sq_addr, u64 cq_addr)
+{
+ struct virtio_pci_modern_common_cfg __iomem *cfg;
+
+ cfg = (struct virtio_pci_modern_common_cfg __iomem *)mdev->common;
+
+ vp_iowrite16(index, &cfg->cfg.queue_select);
+
+ vp_iowrite64_twopart(sq_addr, &cfg->sqe_ring_lo,
+ &cfg->sqe_ring_hi);
+
+ vp_iowrite64_twopart(cq_addr, &cfg->cqe_ring_lo,
+ &cfg->cqe_ring_hi);
+}
+EXPORT_SYMBOL_GPL(vp_modern_queue_address_sqcq);
+
/*
* vp_modern_set_queue_enable - enable a virtqueue
* @mdev: the modern virtio-pci device
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index 335692d41617..a3aaf123ee74 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -270,10 +270,25 @@ struct vring_virtqueue {
bool last_add_time_valid;
ktime_t last_add_time;
#endif
+
+ /* SQ/CQ polling state (VIRTIO_F_SQCQ_POLL) */
+ bool sqcq_poll; /* Is SQ/CQ polling active? */
+ struct vring_sq *sq; /* DMA-coherent SQ doorbell */
+ struct vring_cq *cq; /* DMA-coherent CQ doorbell */
+ dma_addr_t sq_dma_addr; /* DMA address of SQ */
+ dma_addr_t cq_dma_addr; /* DMA address of CQ */
+ atomic_t sq_mmio_count; /* MMIO kicks (sq_need_wakeup=1) */
+ u64 last_cq_detect_ns; /* ktime_get_ns() when CQ work first detected */
+ u64 sq_last_notify_ns; /* ktime of last sq->idx store */
+ u64 sq_ema_interval_ns; /* EMA of sq->idx update interval (ns) */
};
static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num);
static void vring_free(struct virtqueue *_vq);
+static int virtio_alloc_sqcq(struct virtio_device *vdev,
+ struct vring_virtqueue *vq);
+static void virtio_free_sqcq(struct virtio_device *vdev,
+ struct vring_virtqueue *vq);
/*
* Helpers.
@@ -1334,6 +1349,10 @@ static struct virtqueue *__vring_new_virtqueue_split(unsigned int index,
vq->layout = virtio_has_feature(vdev, VIRTIO_F_IN_ORDER) ?
VQ_LAYOUT_SPLIT_IN_ORDER : VQ_LAYOUT_SPLIT;
+ vq->sqcq_poll = false;
+ vq->sq = NULL;
+ vq->cq = NULL;
+
if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
vq->weak_barriers = false;
@@ -1385,6 +1404,17 @@ static struct virtqueue *vring_create_virtqueue_split(
to_vvq(vq)->we_own_ring = true;
+ if (virtio_has_feature(vdev, VIRTIO_F_SQCQ_POLL)) {
+ if (virtio_alloc_sqcq(vdev, to_vvq(vq))) {
+ /* vq is on vdev->vqs list; use vring_del_virtqueue()
+ * to remove and free. vring_free_split() would
+ * double-free via later vp_del_vqs().
+ */
+ vring_del_virtqueue(vq);
+ return NULL;
+ }
+ }
+
return vq;
}
@@ -2554,6 +2584,10 @@ static struct virtqueue *__vring_new_virtqueue_packed(unsigned int index,
vq->layout = virtio_has_feature(vdev, VIRTIO_F_IN_ORDER) ?
VQ_LAYOUT_PACKED_IN_ORDER : VQ_LAYOUT_PACKED;
+ vq->sqcq_poll = false;
+ vq->sq = NULL;
+ vq->cq = NULL;
+
if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM))
vq->weak_barriers = false;
@@ -3013,6 +3047,9 @@ bool virtqueue_kick_prepare(struct virtqueue *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
+ if (vq->sqcq_poll)
+ return true;
+
return VIRTQUEUE_CALL(vq, kick_prepare);
}
EXPORT_SYMBOL_GPL(virtqueue_kick_prepare);
@@ -3032,6 +3069,45 @@ bool virtqueue_notify(struct virtqueue *_vq)
if (unlikely(vq->broken))
return false;
+ /* SQ/CQ polling mode: update SQ idx instead of MMIO kick. */
+ if (vq->sqcq_poll) {
+ u16 new_sq_idx = vq->split.avail_idx_shadow;
+ u8 need_wakeup;
+
+ /* Release: make descriptor writes visible before sq->idx */
+ smp_store_release(&vq->sq->idx, new_sq_idx);
+
+ /* Track submission interval (matches host's interval metric) */
+ {
+ u64 _now = ktime_get_ns();
+
+ if (vq->sq_last_notify_ns) {
+ u64 _int = _now - vq->sq_last_notify_ns;
+
+ if (vq->sq_ema_interval_ns == 0)
+ vq->sq_ema_interval_ns = _int;
+ else
+ vq->sq_ema_interval_ns =
+ vq->sq_ema_interval_ns -
+ vq->sq_ema_interval_ns / 8 +
+ _int / 8;
+ }
+ vq->sq_last_notify_ns = _now;
+ }
+
+ /* Acquire: order sq->idx store before reading sq_need_wakeup.
+ * Prevents stale read on ARM/RISC-V causing missed kick
+ * when host has set sq_need_wakeup=1 and gone to sleep.
+ */
+ need_wakeup = smp_load_acquire(&vq->sq->sq_need_wakeup);
+
+ /* Device poll thread is sleeping -- need MMIO kick to wake it */
+ if (!need_wakeup)
+ return true;
+ atomic_inc(&vq->sq_mmio_count);
+ /* Fall through to MMIO notify */
+ }
+
/* Prod other side to tell it about changes. */
if (!vq->notify(_vq)) {
vq->broken = true;
@@ -3230,6 +3306,12 @@ irqreturn_t vring_interrupt(int irq, void *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
+ /* SQ/CQ poll mode: wake the poll thread, let it handle completions */
+ if (vq->sqcq_poll) {
+ virtio_sqcq_poll_wake_thread(vq->vq.vdev);
+ return IRQ_HANDLED;
+ }
+
if (!more_used(vq)) {
pr_debug("virtqueue interrupt with no work for %p\n", vq);
return IRQ_NONE;
@@ -3250,6 +3332,7 @@ irqreturn_t vring_interrupt(int irq, void *_vq)
data_race(vq->event_triggered = true);
pr_debug("virtqueue callback for %p (%p)\n", vq, vq->vq.callback);
+
if (vq->vq.callback)
vq->vq.callback(&vq->vq);
@@ -3468,6 +3551,8 @@ static void vring_free(struct virtqueue *_vq)
kfree(vq->split.desc_state);
kfree(vq->split.desc_extra);
}
+ if (vq->sqcq_poll)
+ virtio_free_sqcq(vq->vq.vdev, vq);
}
void vring_del_virtqueue(struct virtqueue *_vq)
@@ -3524,6 +3609,8 @@ void vring_transport_features(struct virtio_device *vdev)
break;
case VIRTIO_F_IN_ORDER:
break;
+ case VIRTIO_F_SQCQ_POLL:
+ break;
default:
/* We don't understand this bit. */
__virtio_clear_bit(vdev, i);
@@ -3663,6 +3750,181 @@ dma_addr_t virtqueue_get_used_addr(const struct virtqueue *_vq)
}
EXPORT_SYMBOL_GPL(virtqueue_get_used_addr);
+bool virtqueue_sqcq_poll_active(const struct virtqueue *_vq)
+{
+ const struct vring_virtqueue *vq = to_vvq(_vq);
+
+ return vq->sqcq_poll;
+}
+EXPORT_SYMBOL_GPL(virtqueue_sqcq_poll_active);
+
+/**
+ * virtqueue_disable_sqcq_poll - disable SQ/CQ poll mode for a virtqueue.
+ * @_vq: the virtqueue
+ *
+ * Clears sqcq_poll flag; restores MMIO kick + interrupt for this vq.
+ * Used for control/event queues that must not use the poll thread.
+ */
+void virtqueue_disable_sqcq_poll(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ vq->sqcq_poll = false;
+}
+EXPORT_SYMBOL_GPL(virtqueue_disable_sqcq_poll);
+
+dma_addr_t virtqueue_get_sq_dma_addr(const struct virtqueue *_vq)
+{
+ const struct vring_virtqueue *vq = to_vvq(_vq);
+
+ return vq->sq_dma_addr;
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_sq_dma_addr);
+
+dma_addr_t virtqueue_get_cq_dma_addr(const struct virtqueue *_vq)
+{
+ const struct vring_virtqueue *vq = to_vvq(_vq);
+
+ return vq->cq_dma_addr;
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_cq_dma_addr);
+
+/* Completion detection and sleep/wake accessors for the poll thread. */
+
+bool virtqueue_cq_poll_has_work(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ if (!vq->cq)
+ return false;
+
+ if (more_used(vq)) {
+ WRITE_ONCE(vq->last_cq_detect_ns, ktime_get_ns());
+ return true;
+ }
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(virtqueue_cq_poll_has_work);
+
+unsigned int virtqueue_get_sq_notify_stats(struct virtqueue *_vq, u64 *interval)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ unsigned int mmio;
+
+ mmio = atomic_xchg(&vq->sq_mmio_count, 0);
+ if (interval)
+ *interval = vq->sq_ema_interval_ns;
+ return mmio;
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_sq_notify_stats);
+
+void virtqueue_cq_set_need_wakeup(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ if (vq->cq)
+ /* Release: order prior used-ring reads before wakeup flag */
+ smp_store_release(&vq->cq->cq_need_wakeup, 1);
+}
+EXPORT_SYMBOL_GPL(virtqueue_cq_set_need_wakeup);
+
+void virtqueue_cq_clear_need_wakeup(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ if (vq->cq)
+ WRITE_ONCE(vq->cq->cq_need_wakeup, 0);
+}
+EXPORT_SYMBOL_GPL(virtqueue_cq_clear_need_wakeup);
+
+bool virtqueue_cq_recheck(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+
+ if (!vq->cq)
+ return false;
+
+ /* Full barrier: order cq_need_wakeup store before used ring read */
+ smp_mb();
+ return more_used(vq);
+}
+EXPORT_SYMBOL_GPL(virtqueue_cq_recheck);
+
+u64 virtqueue_get_cq_detect_ns(const struct virtqueue *_vq)
+{
+ const struct vring_virtqueue *vq = to_vvq(_vq);
+
+ return READ_ONCE(vq->last_cq_detect_ns);
+}
+EXPORT_SYMBOL_GPL(virtqueue_get_cq_detect_ns);
+
+/*
+ * virtio_alloc_sqcq - allocate SQ/CQ doorbell structures for polling mode
+ * @vdev: the virtio device
+ * @vq: the vring_virtqueue to allocate for
+ *
+ * Called when VIRTIO_F_SQCQ_POLL is negotiated. Returns 0 on success.
+ */
+static int virtio_alloc_sqcq(struct virtio_device *vdev,
+ struct vring_virtqueue *vq)
+{
+ struct device *dma_dev = vdev->dev.parent;
+
+ /* Control and event queues use traditional interrupt-driven I/O. */
+ if (vq->vq.name &&
+ (strcmp(vq->vq.name, "control") == 0 ||
+ strcmp(vq->vq.name, "event") == 0))
+ return 0;
+ size_t total = sizeof(struct vring_sq) + sizeof(struct vring_cq);
+ void *buf;
+
+ buf = dma_alloc_coherent(dma_dev, total, &vq->sq_dma_addr, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ memset(buf, 0, total);
+
+ vq->sq = buf;
+ vq->cq = buf + sizeof(struct vring_sq);
+ vq->cq_dma_addr = vq->sq_dma_addr + sizeof(struct vring_sq);
+ vq->sqcq_poll = true;
+ atomic_set(&vq->sq_mmio_count, 0);
+ vq->last_cq_detect_ns = 0;
+ vq->sq_last_notify_ns = 0;
+ vq->sq_ema_interval_ns = 0;
+
+ /* Diagnostic: dump SQ/CQ addresses for debug */
+ dev_dbg(&vdev->dev,
+ "vq %s: SQ/CQ doorbells kvirt=%p sq_dma=0x%llx cq_dma=0x%llx\n",
+ vq->vq.name ? vq->vq.name : "?",
+ buf,
+ (unsigned long long)vq->sq_dma_addr,
+ (unsigned long long)vq->cq_dma_addr);
+
+ return 0;
+}
+
+/*
+ * virtio_free_sqcq - free SQ/CQ doorbell structures
+ * @vdev: the virtio device
+ * @vq: the vring_virtqueue to free for
+ */
+static void virtio_free_sqcq(struct virtio_device *vdev,
+ struct vring_virtqueue *vq)
+{
+ struct device *dma_dev = vdev->dev.parent;
+ size_t total = sizeof(struct vring_sq) + sizeof(struct vring_cq);
+
+ if (vq->sq) {
+ dma_free_coherent(dma_dev, total,
+ vq->sq, vq->sq_dma_addr);
+ vq->sq = NULL;
+ vq->cq = NULL;
+ }
+ vq->sqcq_poll = false;
+}
+
/* Only available for split ring */
const struct vring *virtqueue_get_vring(const struct virtqueue *vq)
{
diff --git a/drivers/virtio/virtio_sqcq_poll.c b/drivers/virtio/virtio_sqcq_poll.c
new file mode 100644
index 000000000000..411493218e93
--- /dev/null
+++ b/drivers/virtio/virtio_sqcq_poll.c
@@ -0,0 +1,338 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * virtio SQ/CQ polling thread.
+ *
+ * Replaces interrupt-based completion notification with a dedicated
+ * kernel thread that checks the used ring for completions. Uses the
+ * io_uring sqpoll pattern: busy-wait while active, then set
+ * NEED_WAKEUP and sleep when idle for a configurable timeout.
+ *
+ * Copyright (C) 2026
+ */
+
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/sched.h>
+#include <linux/completion.h>
+#include <linux/mutex.h>
+#include <linux/wait.h>
+#include <linux/sched/clock.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/kthread.h>
+#include <linux/virtio.h>
+#include <linux/virtio_config.h>
+#include <linux/virtio_ring.h>
+#include <linux/interrupt.h>
+
+/**
+ * struct virtio_sqcq_poll_data - per-device poll thread state.
+ *
+ * Follows the Host vhost_sqcq_poll_thread pattern.
+ */
+struct virtio_sqcq_poll_data {
+ struct mutex lock;
+
+ /* List of virtqueues served by this poll thread */
+ struct list_head vq_list;
+
+ struct task_struct *thread;
+ struct wait_queue_head wait;
+
+ unsigned int idle_timeout_us; /* Sleep timeout (us) */
+ unsigned int busy_spin_us; /* Busy-spin budget (us) */
+
+ unsigned long state;
+#define VIRTIO_SQ_THREAD_SHOULD_STOP 0
+
+ struct completion exited;
+ struct device *dev; /* for dev_info() logging */
+
+ /* Poll thread diagnostic statistics */
+ unsigned long stat_sleep_count;
+};
+
+/**
+ * struct virtio_sqcq_vq_node - links a virtqueue to its poll_data.
+ */
+struct virtio_sqcq_vq_node {
+ struct virtqueue *vq;
+ struct list_head node;
+};
+
+static void virtio_sqcq_poll_wake(struct virtio_sqcq_poll_data *sqd)
+{
+ wake_up(&sqd->wait);
+}
+
+/**
+ * virtio_sqcq_poll_wake_thread - wake the CQ poll thread from IRQ context.
+ * @vdev: the virtio device
+ *
+ * Called by vring_interrupt() when SQ/CQ poll mode is active. Wakes the
+ * poll thread so it can check the used ring and invoke completion
+ * callbacks in its own context, instead of handling completions directly
+ * from the hard-IRQ handler.
+ */
+void virtio_sqcq_poll_wake_thread(struct virtio_device *vdev)
+{
+ struct virtio_sqcq_poll_data *sqd = vdev->sqcq_poll_data;
+
+ if (sqd)
+ virtio_sqcq_poll_wake(sqd);
+}
+EXPORT_SYMBOL_GPL(virtio_sqcq_poll_wake_thread);
+
+unsigned long virtio_sqcq_poll_get_sleep_count(struct virtio_device *vdev)
+{
+ struct virtio_sqcq_poll_data *sqd = vdev->sqcq_poll_data;
+
+ if (!sqd)
+ return 0;
+
+ return xchg(&sqd->stat_sleep_count, 0);
+}
+EXPORT_SYMBOL_GPL(virtio_sqcq_poll_get_sleep_count);
+
+static int virtio_sqcq_poll_thread(void *data)
+{
+ struct virtio_sqcq_poll_data *sqd = data;
+ struct virtio_sqcq_vq_node *node;
+ ktime_t last_io_ts = ktime_get();
+ u64 spin_budget_ns = (u64)sqd->busy_spin_us * NSEC_PER_USEC;
+ int vq_count = 0;
+
+ /* Count registered vqs for the startup message */
+ mutex_lock(&sqd->lock);
+ list_for_each_entry(node, &sqd->vq_list, node)
+ vq_count++;
+ mutex_unlock(&sqd->lock);
+
+ dev_info(sqd->dev, "poll thread running, %d vqs, spin=%uus, idle=%uus\n",
+ vq_count, sqd->busy_spin_us, sqd->idle_timeout_us);
+
+ while (!kthread_should_stop()) {
+ bool did_work = false;
+
+ if (test_bit(VIRTIO_SQ_THREAD_SHOULD_STOP, &sqd->state)) {
+ dev_info(sqd->dev, "poll thread stopping\n");
+ break;
+ }
+
+ if (signal_pending(current)) {
+ flush_signals(current);
+ continue;
+ }
+
+ mutex_lock(&sqd->lock);
+
+ /* Process completions on all registered virtqueues */
+ list_for_each_entry(node, &sqd->vq_list, node) {
+ while (virtqueue_cq_poll_has_work(node->vq)) {
+ if (node->vq->callback)
+ node->vq->callback(node->vq);
+ if (local_softirq_pending())
+ do_softirq();
+ did_work = true;
+ }
+ }
+
+ if (did_work) {
+ mutex_unlock(&sqd->lock);
+ last_io_ts = ktime_get();
+ continue;
+ }
+
+ /* No work — spin within the time budget from last_io_ts.
+ * need_resched() + cpu_relax() for cooperative yielding,
+ * same pattern as Host vhost_sqcq_poll_thread.
+ */
+ if (spin_budget_ns > 0 &&
+ ktime_to_ns(ktime_sub(ktime_get(), last_io_ts)) <
+ (s64)spin_budget_ns) {
+ mutex_unlock(&sqd->lock);
+ if (need_resched())
+ cond_resched();
+ cpu_relax();
+ continue;
+ }
+
+ /* Spin budget expired — set cq_need_wakeup on all VQs
+ * and sleep. Double-check for work before sleeping to
+ * close the lost-wakeup race.
+ */
+ {
+ bool found_work = false;
+ DEFINE_WAIT(wait);
+
+ list_for_each_entry(node, &sqd->vq_list, node)
+ virtqueue_cq_set_need_wakeup(node->vq);
+
+ prepare_to_wait(&sqd->wait, &wait,
+ TASK_INTERRUPTIBLE);
+
+ list_for_each_entry(node, &sqd->vq_list, node) {
+ if (virtqueue_cq_recheck(node->vq)) {
+ found_work = true;
+ break;
+ }
+ }
+
+ if (!found_work) {
+ mutex_unlock(&sqd->lock);
+ schedule_timeout(
+ usecs_to_jiffies(sqd->idle_timeout_us));
+ sqd->stat_sleep_count++;
+ mutex_lock(&sqd->lock);
+ }
+
+ finish_wait(&sqd->wait, &wait);
+
+ list_for_each_entry(node, &sqd->vq_list, node)
+ virtqueue_cq_clear_need_wakeup(node->vq);
+ }
+
+ mutex_unlock(&sqd->lock);
+ }
+
+ complete(&sqd->exited);
+ return 0;
+}
+
+/**
+ * virtio_start_sqcq_poll - start the SQ/CQ poll thread for a device.
+ * @vdev: the virtio device
+ * @idle_timeout_us: idle timeout in microseconds before sleeping
+ * @busy_spin_us: busy-spin duration in microseconds before yielding
+ *
+ * Returns 0 on success, negative error on failure.
+ */
+int virtio_start_sqcq_poll(struct virtio_device *vdev,
+ unsigned int idle_timeout_us,
+ unsigned int busy_spin_us)
+{
+ struct virtio_sqcq_poll_data *sqd;
+
+ sqd = kzalloc(sizeof(*sqd), GFP_KERNEL);
+ if (!sqd)
+ return -ENOMEM;
+
+ mutex_init(&sqd->lock);
+ init_waitqueue_head(&sqd->wait);
+ INIT_LIST_HEAD(&sqd->vq_list);
+ init_completion(&sqd->exited);
+ sqd->idle_timeout_us = idle_timeout_us ? idle_timeout_us : 1000;
+ sqd->busy_spin_us = busy_spin_us;
+ sqd->dev = &vdev->dev;
+
+ sqd->thread = kthread_create(virtio_sqcq_poll_thread, sqd,
+ "virtio-sqcq/%s", dev_name(&vdev->dev));
+ if (IS_ERR(sqd->thread)) {
+ kfree(sqd);
+ return PTR_ERR(sqd->thread);
+ }
+
+ vdev->sqcq_poll_data = sqd;
+ wake_up_process(sqd->thread);
+
+ dev_info(sqd->dev, "SQ/CQ poll thread started, idle_timeout=%uus, spin_us=%u\n",
+ sqd->idle_timeout_us, sqd->busy_spin_us);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(virtio_start_sqcq_poll);
+
+/**
+ * virtio_stop_sqcq_poll - stop the SQ/CQ poll thread for a device.
+ * @vdev: the virtio device
+ */
+void virtio_stop_sqcq_poll(struct virtio_device *vdev)
+{
+ struct virtio_sqcq_poll_data *sqd = vdev->sqcq_poll_data;
+ struct virtio_sqcq_vq_node *node, *tmp;
+
+ if (!sqd)
+ return;
+
+ set_bit(VIRTIO_SQ_THREAD_SHOULD_STOP, &sqd->state);
+
+ dev_info(sqd->dev, "SQ/CQ poll thread stopping\n");
+ virtio_sqcq_poll_wake(sqd);
+ wait_for_completion(&sqd->exited);
+
+ /* Reap kthread task_struct to avoid zombie thread leak.
+ * Thread already exited via complete(); safe to call on
+ * already-exited kthread.
+ */
+ kthread_stop(sqd->thread);
+
+ /* Free all vq nodes */
+ list_for_each_entry_safe(node, tmp, &sqd->vq_list, node) {
+ list_del(&node->node);
+ kfree(node);
+ }
+
+ vdev->sqcq_poll_data = NULL;
+ kfree(sqd);
+}
+EXPORT_SYMBOL_GPL(virtio_stop_sqcq_poll);
+
+/**
+ * virtio_sqcq_poll_register_vq - register a virtqueue with the poll thread.
+ * @vdev: the virtio device
+ * @vq: the virtqueue to register
+ *
+ * The virtqueue must have VIRTIO_F_SQCQ_POLL active.
+ * Returns 0 on success, negative error on failure.
+ */
+int virtio_sqcq_poll_register_vq(struct virtio_device *vdev,
+ struct virtqueue *vq)
+{
+ struct virtio_sqcq_poll_data *sqd = vdev->sqcq_poll_data;
+ struct virtio_sqcq_vq_node *node;
+
+ if (!sqd || !virtqueue_sqcq_poll_active(vq))
+ return -EINVAL;
+
+ node = kmalloc(sizeof(*node), GFP_KERNEL);
+ if (!node)
+ return -ENOMEM;
+
+ node->vq = vq;
+
+ mutex_lock(&sqd->lock);
+ list_add_tail(&node->node, &sqd->vq_list);
+ mutex_unlock(&sqd->lock);
+
+ dev_info(sqd->dev, "vq index=%d (%s) registered with poll thread\n",
+ vq->index, vq->name ? vq->name : "?");
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(virtio_sqcq_poll_register_vq);
+
+/**
+ * virtio_sqcq_poll_unregister_vq - unregister a virtqueue from the poll thread.
+ * @vdev: the virtio device
+ * @vq: the virtqueue to unregister
+ */
+void virtio_sqcq_poll_unregister_vq(struct virtio_device *vdev,
+ struct virtqueue *vq)
+{
+ struct virtio_sqcq_poll_data *sqd = vdev->sqcq_poll_data;
+ struct virtio_sqcq_vq_node *node, *tmp;
+
+ if (!sqd)
+ return;
+
+ mutex_lock(&sqd->lock);
+ list_for_each_entry_safe(node, tmp, &sqd->vq_list, node) {
+ if (node->vq == vq) {
+ list_del(&node->node);
+ kfree(node);
+ break;
+ }
+ }
+ mutex_unlock(&sqd->lock);
+}
+EXPORT_SYMBOL_GPL(virtio_sqcq_poll_unregister_vq);
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 3bbc4cb6a672..8300e0946719 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -125,6 +125,28 @@ dma_addr_t virtqueue_get_desc_addr(const struct virtqueue *vq);
dma_addr_t virtqueue_get_avail_addr(const struct virtqueue *vq);
dma_addr_t virtqueue_get_used_addr(const struct virtqueue *vq);
+/* SQ/CQ polling (VIRTIO_F_SQCQ_POLL) */
+bool virtqueue_sqcq_poll_active(const struct virtqueue *vq);
+void virtqueue_disable_sqcq_poll(struct virtqueue *vq);
+dma_addr_t virtqueue_get_sq_dma_addr(const struct virtqueue *vq);
+dma_addr_t virtqueue_get_cq_dma_addr(const struct virtqueue *vq);
+u64 virtqueue_get_cq_detect_ns(const struct virtqueue *vq);
+int virtio_start_sqcq_poll(struct virtio_device *vdev,
+ unsigned int idle_timeout_us,
+ unsigned int busy_spin_us);
+void virtio_stop_sqcq_poll(struct virtio_device *vdev);
+void virtio_sqcq_poll_wake_thread(struct virtio_device *vdev);
+int virtio_sqcq_poll_register_vq(struct virtio_device *vdev,
+ struct virtqueue *vq);
+void virtio_sqcq_poll_unregister_vq(struct virtio_device *vdev,
+ struct virtqueue *vq);
+bool virtqueue_cq_poll_has_work(struct virtqueue *vq);
+unsigned int virtqueue_get_sq_notify_stats(struct virtqueue *vq, u64 *interval);
+unsigned long virtio_sqcq_poll_get_sleep_count(struct virtio_device *vdev);
+void virtqueue_cq_set_need_wakeup(struct virtqueue *vq);
+void virtqueue_cq_clear_need_wakeup(struct virtqueue *vq);
+bool virtqueue_cq_recheck(struct virtqueue *vq);
+
int virtqueue_resize(struct virtqueue *vq, u32 num,
void (*recycle)(struct virtqueue *vq, void *buf),
void (*recycle_done)(struct virtqueue *vq));
@@ -165,6 +187,8 @@ struct virtio_admin_cmd {
* @debugfs_dir: debugfs directory entry.
* @debugfs_filter_features: features to be filtered set by debugfs.
*/
+struct virtio_sqcq_poll_data;
+
struct virtio_device {
int index;
bool failed;
@@ -182,6 +206,7 @@ struct virtio_device {
VIRTIO_DECLARE_FEATURES(features);
void *priv;
union virtio_map vmap;
+ struct virtio_sqcq_poll_data *sqcq_poll_data; /* SQ/CQ poll thread state */
#ifdef CONFIG_VIRTIO_DEBUG
struct dentry *debugfs_dir;
u64 debugfs_filter_features[VIRTIO_FEATURES_U64S];
diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h
index 9a3f2fc53bd6..36810f9d23f3 100644
--- a/include/linux/virtio_pci_modern.h
+++ b/include/linux/virtio_pci_modern.h
@@ -145,6 +145,8 @@ u16 vp_modern_config_vector(struct virtio_pci_modern_device *mdev,
void vp_modern_queue_address(struct virtio_pci_modern_device *mdev,
u16 index, u64 desc_addr, u64 driver_addr,
u64 device_addr);
+void vp_modern_queue_address_sqcq(struct virtio_pci_modern_device *mdev,
+ u16 index, u64 sq_addr, u64 cq_addr);
void vp_modern_set_queue_enable(struct virtio_pci_modern_device *mdev,
u16 idx, bool enable);
bool vp_modern_get_queue_enable(struct virtio_pci_modern_device *mdev,
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 4/4] virtio: add SQ/CQ polling mode support for vhost-scsi
From: rom.wang @ 2026-07-20 14:10 UTC (permalink / raw)
To: mst, jasowangio, eperezma, stefanha
Cc: virtualization, linux-kernel, linux-scsi, kvm, qemu-devel,
Yufeng Wang
In-Reply-To: <20260720141040.181946-1-r4o5m6e8o@163.com>
From: Yufeng Wang <wangyufeng@kylinos.cn>
Implement the QEMU-side bridge layer for virtio SQ/CQ polling mode,
which replaces interrupt-based virtio notifications with io_uring-style
doorbell polling to eliminate VM exits in high-throughput scenarios.
UAPI headers: sync VIRTIO_F_SQCQ_POLL (bit 42), PCI common config
offsets (64-76), struct vring_sq/vring_cq, struct vhost_vring_sqcq_addr,
and VHOST_SET_VRING_SQCQ_ADDR ioctl (0x27) from Linux kernel.
VirtIO core: add sq/cq fields to VRing, with getter/setter functions.
PCI transport: handle SQE/CQE register read/write in common config,
gate SQ/CQ address forwarding on VIRTIO_F_SQCQ_POLL negotiation in
Q_ENABLE handler.
Vhost backend: add vhost_set_vring_sqcq_addr_op to VhostOps, implement
kernel backend via VHOST_SET_VRING_SQCQ_ADDR ioctl.
Vhost core: map SQ/CQ GPAs to HVAs in vhost_virtqueue_start() after
VHOST_SET_VRING_ADDR, call the new ioctl before VHOST_SET_VRING_KICK;
unmap in do_vhost_virtqueue_stop().
vhost-scsi: advertise VIRTIO_F_SQCQ_POLL in kernel_feature_bits[].
Signed-off-by: Yufeng Wang <wangyufeng@kylinos.cn>
---
hw/scsi/vhost-scsi.c | 4 ++
hw/virtio/vhost-backend.c | 7 +++
hw/virtio/vhost.c | 54 +++++++++++++++++++
hw/virtio/virtio-pci.c | 31 +++++++++++
hw/virtio/virtio.c | 18 +++++++
include/hw/virtio/vhost-backend.h | 4 ++
include/hw/virtio/vhost.h | 6 +++
include/hw/virtio/virtio-pci.h | 2 +
include/hw/virtio/virtio.h | 3 ++
include/standard-headers/linux/vhost_types.h | 7 +++
.../standard-headers/linux/virtio_config.h | 5 +-
include/standard-headers/linux/virtio_pci.h | 4 ++
include/standard-headers/linux/virtio_ring.h | 30 +++++++++++
| 4 ++
14 files changed, 178 insertions(+), 1 deletion(-)
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index 699863c..bb7f3e8 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -40,6 +40,7 @@ static const int kernel_feature_bits[] = {
VIRTIO_F_RING_RESET,
VIRTIO_F_IN_ORDER,
VIRTIO_F_NOTIFICATION_DATA,
+ VIRTIO_F_SQCQ_POLL,
VHOST_INVALID_FEATURE_BIT
};
@@ -361,6 +362,9 @@ static const Property vhost_scsi_properties[] = {
DEFINE_PROP_BIT64("hotplug", VHostSCSICommon, host_features,
VIRTIO_SCSI_F_HOTPLUG,
false),
+ DEFINE_PROP_BIT64("sqcq_poll", VHostSCSICommon, host_features,
+ VIRTIO_F_SQCQ_POLL,
+ true),
DEFINE_PROP_BOOL("migratable", VHostSCSICommon, migratable, false),
DEFINE_PROP_BOOL("worker_per_virtqueue", VirtIOSCSICommon,
conf.worker_per_virtqueue, false),
diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
index 4367db0..a71b30b 100644
--- a/hw/virtio/vhost-backend.c
+++ b/hw/virtio/vhost-backend.c
@@ -115,6 +115,12 @@ static int vhost_kernel_set_vring_addr(struct vhost_dev *dev,
return vhost_kernel_call(dev, VHOST_SET_VRING_ADDR, addr);
}
+static int vhost_kernel_set_vring_sqcq_addr(struct vhost_dev *dev,
+ struct vhost_vring_sqcq_addr *addr)
+{
+ return vhost_kernel_call(dev, VHOST_SET_VRING_SQCQ_ADDR, addr);
+}
+
static int vhost_kernel_set_vring_endian(struct vhost_dev *dev,
struct vhost_vring_state *ring)
{
@@ -368,6 +374,7 @@ const VhostOps kernel_ops = {
.vhost_set_log_base = vhost_kernel_set_log_base,
.vhost_set_mem_table = vhost_kernel_set_mem_table,
.vhost_set_vring_addr = vhost_kernel_set_vring_addr,
+ .vhost_set_vring_sqcq_addr = vhost_kernel_set_vring_sqcq_addr,
.vhost_set_vring_endian = vhost_kernel_set_vring_endian,
.vhost_set_vring_num = vhost_kernel_set_vring_num,
.vhost_set_vring_base = vhost_kernel_set_vring_base,
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 31e9704..8c82f98 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -22,6 +22,7 @@
#include "qemu/memfd.h"
#include "qemu/log.h"
#include "standard-headers/linux/vhost_types.h"
+#include "standard-headers/linux/virtio_ring.h"
#include "hw/virtio/virtio-bus.h"
#include "hw/mem/memory-device.h"
#include "migration/blocker.h"
@@ -1337,6 +1338,49 @@ int vhost_virtqueue_start(struct vhost_dev *dev,
goto fail_alloc;
}
+ /* SQ/CQ polling: map and pass to vhost if feature negotiated */
+ vq->sq = NULL;
+ vq->cq = NULL;
+ vq->sq_phys = virtio_queue_get_sq_addr(vdev, idx);
+ vq->cq_phys = virtio_queue_get_cq_addr(vdev, idx);
+
+ if (vq->sq_phys && vq->cq_phys &&
+ dev->vhost_ops->vhost_set_vring_sqcq_addr &&
+ virtio_has_feature(vdev->guest_features, VIRTIO_F_SQCQ_POLL)) {
+ struct vhost_vring_sqcq_addr sqcq_addr;
+
+ l = sizeof(struct vring_sq);
+ vq->sq = vhost_memory_map(dev, vq->sq_phys, &l, false);
+ if (!vq->sq) {
+ r = -ENOMEM;
+ goto fail_alloc;
+ }
+ vq->sq_size = l;
+
+ l = sizeof(struct vring_cq);
+ vq->cq = vhost_memory_map(dev, vq->cq_phys, &l, false);
+ if (!vq->cq) {
+ r = -ENOMEM;
+ vhost_memory_unmap(dev, vq->sq, vq->sq_size, 0, 0);
+ goto fail_alloc;
+ }
+ vq->cq_size = l;
+
+ memset(&sqcq_addr, 0, sizeof(sqcq_addr));
+ sqcq_addr.index = vhost_vq_index;
+ sqcq_addr.sq_user_addr = (uint64_t)(unsigned long)vq->sq;
+ sqcq_addr.cq_user_addr = (uint64_t)(unsigned long)vq->cq;
+ r = dev->vhost_ops->vhost_set_vring_sqcq_addr(dev, &sqcq_addr);
+ if (r < 0) {
+ VHOST_OPS_DEBUG(r, "vhost_set_vring_sqcq_addr failed");
+ vhost_memory_unmap(dev, vq->cq, vq->cq_size, 0, 0);
+ vhost_memory_unmap(dev, vq->sq, vq->sq_size, 0, 0);
+ vq->sq = NULL;
+ vq->cq = NULL;
+ goto fail_alloc;
+ }
+ }
+
file.fd = event_notifier_get_fd(virtio_queue_get_host_notifier(vvq));
r = dev->vhost_ops->vhost_set_vring_kick(dev, &file);
if (r) {
@@ -1425,6 +1469,16 @@ static int do_vhost_virtqueue_stop(struct vhost_dev *dev,
vhost_vq_index);
}
+ /* Unmap SQ/CQ if mapped */
+ if (vq->sq) {
+ vhost_memory_unmap(dev, vq->sq, vq->sq_size, 0, 0);
+ vq->sq = NULL;
+ }
+ if (vq->cq) {
+ vhost_memory_unmap(dev, vq->cq, vq->cq_size, 0, 0);
+ vq->cq = NULL;
+ }
+
vhost_memory_unmap(dev, vq->used, virtio_queue_get_used_size(vdev, idx),
1, virtio_queue_get_used_size(vdev, idx));
vhost_memory_unmap(dev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index b273eb2..c88cba3 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1623,6 +1623,18 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,
case VIRTIO_PCI_COMMON_Q_RESET:
val = proxy->vqs[vdev->queue_sel].reset;
break;
+ case VIRTIO_PCI_COMMON_SQE_LO:
+ val = proxy->vqs[vdev->queue_sel].sq[0];
+ break;
+ case VIRTIO_PCI_COMMON_SQE_HI:
+ val = proxy->vqs[vdev->queue_sel].sq[1];
+ break;
+ case VIRTIO_PCI_COMMON_CQE_LO:
+ val = proxy->vqs[vdev->queue_sel].cq[0];
+ break;
+ case VIRTIO_PCI_COMMON_CQE_HI:
+ val = proxy->vqs[vdev->queue_sel].cq[1];
+ break;
default:
val = 0;
}
@@ -1727,6 +1739,13 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
proxy->vqs[vdev->queue_sel].avail[0],
((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
proxy->vqs[vdev->queue_sel].used[0]);
+ if (virtio_has_feature(vdev->guest_features, VIRTIO_F_SQCQ_POLL)) {
+ virtio_queue_set_sqcq(vdev, vdev->queue_sel,
+ ((uint64_t)proxy->vqs[vdev->queue_sel].sq[1] << 32) |
+ proxy->vqs[vdev->queue_sel].sq[0],
+ ((uint64_t)proxy->vqs[vdev->queue_sel].cq[1] << 32) |
+ proxy->vqs[vdev->queue_sel].cq[0]);
+ }
proxy->vqs[vdev->queue_sel].enabled = 1;
proxy->vqs[vdev->queue_sel].reset = 0;
virtio_queue_enable(vdev, vdev->queue_sel);
@@ -1752,6 +1771,18 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
case VIRTIO_PCI_COMMON_Q_USEDHI:
proxy->vqs[vdev->queue_sel].used[1] = val;
break;
+ case VIRTIO_PCI_COMMON_SQE_LO:
+ proxy->vqs[vdev->queue_sel].sq[0] = val;
+ break;
+ case VIRTIO_PCI_COMMON_SQE_HI:
+ proxy->vqs[vdev->queue_sel].sq[1] = val;
+ break;
+ case VIRTIO_PCI_COMMON_CQE_LO:
+ proxy->vqs[vdev->queue_sel].cq[0] = val;
+ break;
+ case VIRTIO_PCI_COMMON_CQE_HI:
+ proxy->vqs[vdev->queue_sel].cq[1] = val;
+ break;
case VIRTIO_PCI_COMMON_Q_RESET:
if (val == 1) {
proxy->vqs[vdev->queue_sel].reset = 1;
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 3dc9423..42e39d9 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -111,6 +111,8 @@ typedef struct VRing
hwaddr desc;
hwaddr avail;
hwaddr used;
+ hwaddr sq;
+ hwaddr cq;
VRingMemoryRegionCaches *caches;
} VRing;
@@ -2400,6 +2402,12 @@ void virtio_queue_set_rings(VirtIODevice *vdev, int n, hwaddr desc,
virtio_init_region_cache(vdev, n);
}
+void virtio_queue_set_sqcq(VirtIODevice *vdev, int n, hwaddr sq, hwaddr cq)
+{
+ vdev->vq[n].vring.sq = sq;
+ vdev->vq[n].vring.cq = cq;
+}
+
void virtio_queue_set_num(VirtIODevice *vdev, int n, int num)
{
/* Don't allow guest to flip queue between existent and
@@ -3661,6 +3669,16 @@ hwaddr virtio_queue_get_used_addr(VirtIODevice *vdev, int n)
return vdev->vq[n].vring.used;
}
+hwaddr virtio_queue_get_sq_addr(VirtIODevice *vdev, int n)
+{
+ return vdev->vq[n].vring.sq;
+}
+
+hwaddr virtio_queue_get_cq_addr(VirtIODevice *vdev, int n)
+{
+ return vdev->vq[n].vring.cq;
+}
+
hwaddr virtio_queue_get_desc_size(VirtIODevice *vdev, int n)
{
return sizeof(VRingDesc) * vdev->vq[n].vring.num;
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index ff94fa1..911cd15 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -45,6 +45,7 @@ struct vhost_memory;
struct vhost_vring_file;
struct vhost_vring_state;
struct vhost_vring_addr;
+struct vhost_vring_sqcq_addr;
struct vhost_vring_worker;
struct vhost_worker_state;
struct vhost_scsi_target;
@@ -71,6 +72,8 @@ typedef int (*vhost_set_mem_table_op)(struct vhost_dev *dev,
struct vhost_memory *mem);
typedef int (*vhost_set_vring_addr_op)(struct vhost_dev *dev,
struct vhost_vring_addr *addr);
+typedef int (*vhost_set_vring_sqcq_addr_op)(struct vhost_dev *dev,
+ struct vhost_vring_sqcq_addr *addr);
typedef int (*vhost_set_vring_endian_op)(struct vhost_dev *dev,
struct vhost_vring_state *ring);
typedef int (*vhost_set_vring_num_op)(struct vhost_dev *dev,
@@ -178,6 +181,7 @@ typedef struct VhostOps {
vhost_set_log_base_op vhost_set_log_base;
vhost_set_mem_table_op vhost_set_mem_table;
vhost_set_vring_addr_op vhost_set_vring_addr;
+ vhost_set_vring_sqcq_addr_op vhost_set_vring_sqcq_addr;
vhost_set_vring_endian_op vhost_set_vring_endian;
vhost_set_vring_num_op vhost_set_vring_num;
vhost_set_vring_base_op vhost_set_vring_base;
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 08bbb4d..65056a0 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -27,6 +27,8 @@ struct vhost_virtqueue {
void *desc;
void *avail;
void *used;
+ void *sq; /* Mapped SQ doorbell */
+ void *cq; /* Mapped CQ doorbell */
int num;
unsigned long long desc_phys;
unsigned desc_size;
@@ -34,6 +36,10 @@ struct vhost_virtqueue {
unsigned avail_size;
unsigned long long used_phys;
unsigned used_size;
+ unsigned long long sq_phys; /* GPA of SQ doorbell */
+ unsigned long long cq_phys; /* GPA of CQ doorbell */
+ unsigned sq_size; /* Size of mapped SQ region */
+ unsigned cq_size; /* Size of mapped CQ region */
EventNotifier masked_notifier;
EventNotifier error_notifier;
EventNotifier masked_config_notifier;
diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h
index 6397529..37103c7 100644
--- a/include/hw/virtio/virtio-pci.h
+++ b/include/hw/virtio/virtio-pci.h
@@ -122,6 +122,8 @@ typedef struct VirtIOPCIQueue {
uint32_t desc[2];
uint32_t avail[2];
uint32_t used[2];
+ uint32_t sq[2];
+ uint32_t cq[2];
} VirtIOPCIQueue;
struct VirtIOPCIProxy {
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 27cd98d..29f21f4 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -361,6 +361,7 @@ int virtio_queue_get_max_num(VirtIODevice *vdev, int n);
int virtio_get_num_queues(VirtIODevice *vdev);
void virtio_queue_set_rings(VirtIODevice *vdev, int n, hwaddr desc,
hwaddr avail, hwaddr used);
+void virtio_queue_set_sqcq(VirtIODevice *vdev, int n, hwaddr sq, hwaddr cq);
void virtio_queue_update_rings(VirtIODevice *vdev, int n);
void virtio_init_region_cache(VirtIODevice *vdev, int n);
void virtio_queue_set_align(VirtIODevice *vdev, int n, int align);
@@ -408,6 +409,8 @@ bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
bool virtio_queue_enabled(VirtIODevice *vdev, int n);
hwaddr virtio_queue_get_avail_addr(VirtIODevice *vdev, int n);
hwaddr virtio_queue_get_used_addr(VirtIODevice *vdev, int n);
+hwaddr virtio_queue_get_sq_addr(VirtIODevice *vdev, int n);
+hwaddr virtio_queue_get_cq_addr(VirtIODevice *vdev, int n);
hwaddr virtio_queue_get_desc_size(VirtIODevice *vdev, int n);
hwaddr virtio_queue_get_avail_size(VirtIODevice *vdev, int n);
hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n);
diff --git a/include/standard-headers/linux/vhost_types.h b/include/standard-headers/linux/vhost_types.h
index 79b53a9..ea96feb 100644
--- a/include/standard-headers/linux/vhost_types.h
+++ b/include/standard-headers/linux/vhost_types.h
@@ -151,6 +151,13 @@ struct vhost_scsi_target {
unsigned short reserved;
};
+/* SQ/CQ polling mode */
+struct vhost_vring_sqcq_addr {
+ unsigned int index;
+ __u64 sq_user_addr; /* SQ doorbell HVA */
+ __u64 cq_user_addr; /* CQ doorbell HVA */
+};
+
/* VHOST_VDPA specific definitions */
struct vhost_vdpa_config {
diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h
index 45be0fa..d73938e 100644
--- a/include/standard-headers/linux/virtio_config.h
+++ b/include/standard-headers/linux/virtio_config.h
@@ -52,7 +52,7 @@
* rest are per-device feature bits.
*/
#define VIRTIO_TRANSPORT_F_START 28
-#define VIRTIO_TRANSPORT_F_END 42
+#define VIRTIO_TRANSPORT_F_END 43
#ifndef VIRTIO_CONFIG_NO_LEGACY
/* Do we get callbacks when the ring is completely used, even if we've
@@ -118,4 +118,7 @@
*/
#define VIRTIO_F_ADMIN_VQ 41
+/* SQ/CQ polling mode for io_uring-style doorbell polling */
+#define VIRTIO_F_SQCQ_POLL 42
+
#endif /* _LINUX_VIRTIO_CONFIG_H */
diff --git a/include/standard-headers/linux/virtio_pci.h b/include/standard-headers/linux/virtio_pci.h
index 4c82513..cc585b6 100644
--- a/include/standard-headers/linux/virtio_pci.h
+++ b/include/standard-headers/linux/virtio_pci.h
@@ -235,6 +235,10 @@ struct virtio_pci_cfg_cap {
#define VIRTIO_PCI_COMMON_Q_RESET 58
#define VIRTIO_PCI_COMMON_ADM_Q_IDX 60
#define VIRTIO_PCI_COMMON_ADM_Q_NUM 62
+#define VIRTIO_PCI_COMMON_SQE_LO 64
+#define VIRTIO_PCI_COMMON_SQE_HI 68
+#define VIRTIO_PCI_COMMON_CQE_LO 72
+#define VIRTIO_PCI_COMMON_CQE_HI 76
#endif /* VIRTIO_PCI_NO_MODERN */
diff --git a/include/standard-headers/linux/virtio_ring.h b/include/standard-headers/linux/virtio_ring.h
index 22f6eb8..520b758 100644
--- a/include/standard-headers/linux/virtio_ring.h
+++ b/include/standard-headers/linux/virtio_ring.h
@@ -245,4 +245,34 @@ struct vring_packed_desc {
uint16_t flags;
};
+/*
+ * struct vring_sq - Submission Queue doorbell for polling mode.
+ * Placed alongside the standard virtio split ring to enable
+ * kick-less notification from guest to host.
+ *
+ * @idx: Producer index (guest updates, corresponds to avail->idx)
+ * @sq_need_wakeup: device requests guest to kick when sleeping
+ * @reserved: Reserved for future use, pad to cache-line
+ */
+struct vring_sq {
+ __virtio64 idx;
+ uint8_t sq_need_wakeup;
+ uint8_t reserved[55];
+} __attribute__((aligned(128)));
+
+/*
+ * struct vring_cq - Completion Queue doorbell for polling mode.
+ * Placed alongside the standard virtio split ring to enable
+ * interrupt-less notification from host to guest.
+ *
+ * @idx: Producer index (device updates, corresponds to used->idx)
+ * @cq_need_wakeup: guest requests host to signal when sleeping
+ * @reserved: Reserved for future use, pad to cache-line
+ */
+struct vring_cq {
+ __virtio64 idx;
+ uint8_t cq_need_wakeup;
+ uint8_t reserved[55];
+} __attribute__((aligned(128)));
+
#endif /* _LINUX_VIRTIO_RING_H */
--git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h
index c57674a..746d71f 100644
--- a/linux-headers/linux/vhost.h
+++ b/linux-headers/linux/vhost.h
@@ -123,6 +123,10 @@
#define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64)
#define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64)
+/* Set SQ/CQ doorbell addresses for polling mode */
+#define VHOST_SET_VRING_SQCQ_ADDR _IOW(VHOST_VIRTIO, 0x27, \
+ struct vhost_vring_sqcq_addr)
+
/* VHOST_NET specific defines */
/* Attach virtio net ring to a raw socket, or tap device.
--
2.34.1
^ permalink raw reply related
* [RFC PATCH 2/4] vhost: host kernel support for SQ/CQ polling
From: rom.wang @ 2026-07-20 14:10 UTC (permalink / raw)
To: mst, jasowangio, eperezma, stefanha
Cc: virtualization, linux-kernel, linux-scsi, kvm, qemu-devel,
Yufeng Wang
In-Reply-To: <20260720141040.181946-1-r4o5m6e8o@163.com>
From: Yufeng Wang <wangyufeng@kylinos.cn>
Implement SQ/CQ doorbell polling in the vhost kernel backend:
- vhost.c: per-VQ poll thread (kthread) that polls sq->idx for
submissions and processes completions via poll_complete callback;
need_resched() cooperative spin; round-robin CPU binding with
modulo to prevent overflow; sqcq_poll enabled after poll_task is
valid (NULL deref fix); get_user fault retry (3 attempts, 10ms
backoff); EMA overflow-safe formula
- vhost.h: vhost_virtqueue extensions (sq/cq pointers, poll thread
state, EMA fields)
- scsi.c: vhost_scsi_poll_complete() writes cq->idx and signals
Guest; SET_ENDPOINT error path stops started poll threads (fallback);
10s window diagnostic output
- vhost.h UAPI: VHOST_SET_VRING_SQCQ_ADDR ioctl
- vhost_types.h UAPI: struct vhost_vring_sqcq_addr
Signed-off-by: Yufeng Wang <wangyufeng@kylinos.cn>
---
Documentation/compare-sqcq-results.sh | 239 +++++++
Documentation/run-sqcq-compare.sh | 182 ++++++
Documentation/virtio-sqcq-poll-spec.rst | 617 ++++++++++++++++++
.../virtio-sqcq-poll-virtio-spec.rst | 355 ++++++++++
build-riscv-kernel.sh | 26 +
drivers/vhost/scsi.c | 187 +++++-
drivers/vhost/vhost.c | 595 ++++++++++++++++-
drivers/vhost/vhost.h | 29 +-
include/uapi/linux/vhost.h | 4 +
include/uapi/linux/vhost_types.h | 6 +
10 files changed, 2221 insertions(+), 19 deletions(-)
create mode 100644 Documentation/compare-sqcq-results.sh
create mode 100644 Documentation/run-sqcq-compare.sh
create mode 100644 Documentation/virtio-sqcq-poll-spec.rst
create mode 100644 Documentation/virtio-sqcq-poll-virtio-spec.rst
create mode 100644 build-riscv-kernel.sh
diff --git a/Documentation/compare-sqcq-results.sh b/Documentation/compare-sqcq-results.sh
new file mode 100644
index 0000000000000..85bb927fe91e5
--- /dev/null
+++ b/Documentation/compare-sqcq-results.sh
@@ -0,0 +1,239 @@
+#!/usr/bin/env python3
+"""compare-sqcq-results.py - Generate A/B comparison report for SQ/CQ polling vs baseline.
+
+Usage:
+ python3 compare-sqcq-results.py <baseline_dir> <sqcq_dir>
+ python3 compare-sqcq-results.py # auto-detect latest results
+"""
+import json, sys, os, glob, re
+from collections import OrderedDict
+
+def find_latest_dirs():
+ """Find the most recent baseline and sqcq-poll result directories."""
+ base = "perf-results"
+ if not os.path.isdir(base):
+ print(f"Error: '{base}/' not found. Run run-sqcq-compare.sh first.")
+ sys.exit(1)
+
+ bl_dirs = sorted(glob.glob(os.path.join(base, "baseline_*")))
+ sq_dirs = sorted(glob.glob(os.path.join(base, "sqcq-poll_*")))
+
+ if not bl_dirs:
+ print("Error: No baseline_* directory found in perf-results/")
+ sys.exit(1)
+ if not sq_dirs:
+ print("Error: No sqcq-poll_* directory found in perf-results/")
+ sys.exit(1)
+
+ return bl_dirs[-1], sq_dirs[-1]
+
+
+def parse_fio_json(filepath):
+ """Extract key metrics from a fio JSON output file."""
+ with open(filepath) as f:
+ d = json.load(f)
+
+ jobs = d.get("jobs", [])
+ if not jobs:
+ return None
+
+ j = jobs[0]
+ rd = j.get("read", {})
+ wr = j.get("write", {})
+
+ # Determine which direction has data
+ if rd.get("iops", 0) > 0:
+ data = rd
+ else:
+ data = wr
+
+ iops = data.get("iops", 0)
+ bw_mib = data.get("bw", 0) / 1024 # KiB/s -> MiB/s
+
+ # Latency
+ lat_ns = data.get("lat_ns", {})
+ avg_us = lat_ns.get("mean", 0) / 1000
+ min_us = lat_ns.get("min", 0) / 1000
+ max_us = lat_ns.get("max", 0) / 1000
+ stddev_us = lat_ns.get("stddev", 0) / 1000
+
+ # Completion latency percentiles
+ clat = data.get("clat_ns", {})
+ pct = clat.get("percentile", {}) if isinstance(clat, dict) else {}
+ p50 = pct.get("50.000000", 0) / 1000
+ p95 = pct.get("95.000000", 0) / 1000
+ p99 = pct.get("99.000000", 0) / 1000
+ p999 = pct.get("99.900000", 0) / 1000
+
+ return {
+ "iops": iops,
+ "bw_mib": bw_mib,
+ "lat_avg_us": avg_us,
+ "lat_min_us": min_us,
+ "lat_max_us": max_us,
+ "lat_stddev_us": stddev_us,
+ "p50_us": p50,
+ "p95_us": p95,
+ "p99_us": p99,
+ "p999_us": p999,
+ }
+
+
+def pct_change(old, new):
+ """Calculate percentage change. Returns None if division by zero."""
+ if old == 0:
+ return None
+ return (new - old) / old * 100
+
+
+def fmt_pct(val):
+ """Format a percentage change value."""
+ if val is None:
+ return "N/A"
+ sign = "+" if val > 0 else ""
+ return f"{sign}{val:.1f}%"
+
+
+def fmt_int(val):
+ """Format integer with comma separators."""
+ return f"{val:,.0f}"
+
+
+def fmt_f1(val):
+ """Format float with 1 decimal."""
+ return f"{val:.1f}"
+
+
+def main():
+ if len(sys.argv) >= 3:
+ bl_dir = sys.argv[1]
+ sq_dir = sys.argv[2]
+ else:
+ bl_dir, sq_dir = find_latest_dirs()
+
+ print(f"Baseline: {bl_dir}")
+ print(f"SQ/CQ Poll: {sq_dir}")
+ print()
+
+ # Collect all test labels
+ bl_files = {os.path.basename(f).replace("_best.json", ""): f
+ for f in glob.glob(os.path.join(bl_dir, "*_best.json"))}
+ sq_files = {os.path.basename(f).replace("_best.json", ""): f
+ for f in glob.glob(os.path.join(sq_dir, "*_best.json"))}
+
+ all_labels = sorted(set(bl_files.keys()) | set(sq_files.keys()))
+
+ if not all_labels:
+ print("Error: No *_best.json files found.")
+ sys.exit(1)
+
+ # Define metric display columns
+ metrics = [
+ ("IOPS", "iops", fmt_int, True),
+ ("BW(MiB/s)", "bw_mib", fmt_f1, True),
+ ("Avg(us)", "lat_avg_us", fmt_f1, False),
+ ("P50(us)", "p50_us", fmt_f1, False),
+ ("P95(us)", "p95_us", fmt_f1, False),
+ ("P99(us)", "p99_us", fmt_f1, False),
+ ("P99.9(us)", "p999_us", fmt_f1, False),
+ ]
+
+ # Per-label comparison
+ results = []
+ for label in all_labels:
+ bl_data = parse_fio_json(bl_files[label]) if label in bl_files else None
+ sq_data = parse_fio_json(sq_files[label]) if label in sq_files else None
+
+ entry = {"label": label, "baseline": bl_data, "sqcq": sq_data}
+ results.append(entry)
+
+ # Print grouped by workload type
+ groups = OrderedDict()
+ for label in all_labels:
+ # Extract workload type from label: e.g. "4k-randread-od1-nj1" -> "4k-randread"
+ parts = label.rsplit("-od", 1)
+ wl = parts[0] if len(parts) > 1 else label
+ if wl not in groups:
+ groups[wl] = []
+ groups[wl].append(label)
+
+ print("=" * 100)
+ print(" SQ/CQ POLL vs BASELINE COMPARISON REPORT")
+ print("=" * 100)
+
+ for wl, labels in groups.items():
+ print(f"\n{'─' * 100}")
+ print(f" Workload: {wl}")
+ print(f"{'─' * 100}")
+
+ for label in labels:
+ r = next(x for x in results if x["label"] == label)
+ bl = r["baseline"]
+ sq = r["sqcq"]
+
+ # Extract iodepth/numjobs from label
+ m = re.search(r'od(\d+)-nj(\d+)', label)
+ od = m.group(1) if m else "?"
+ nj = m.group(2) if m else "?"
+
+ print(f"\n iodepth={od}, numjobs={nj}")
+ print(f" {'Metric':<14} {'Baseline':>12} {'SQ/CQ Poll':>12} {'Change':>10}")
+ print(f" {'─'*14} {'─'*12} {'─'*12} {'─'*10}")
+
+ for metric_name, metric_key, fmt_fn, higher_is_better in metrics:
+ bl_val = bl[metric_key] if bl else 0
+ sq_val = sq[metric_key] if sq else 0
+ change = pct_change(bl_val, sq_val)
+
+ # For latency, negative change is improvement
+ line = f" {metric_name:<14} {fmt_fn(bl_val):>12} {fmt_fn(sq_val):>12}"
+
+ if change is not None:
+ marker = ""
+ if higher_is_better and change > 2:
+ marker = " *"
+ elif not higher_is_better and change < -2:
+ marker = " *"
+ line += f" {fmt_pct(change):>10}{marker}"
+ else:
+ line += f" {'N/A':>10}"
+
+ print(line)
+
+ # Summary table - just IOPS
+ print(f"\n{'=' * 100}")
+ print(" IOPS SUMMARY (all scenarios)")
+ print(f"{'=' * 100}")
+ print(f"\n {'Test':<30} {'Baseline':>10} {'SQ/CQ':>10} {'Change':>10}")
+ print(f" {'─'*30} {'─'*10} {'─'*10} {'─'*10}")
+
+ for r in results:
+ bl_iops = r["baseline"]["iops"] if r["baseline"] else 0
+ sq_iops = r["sqcq"]["iops"] if r["sqcq"] else 0
+ change = pct_change(bl_iops, sq_iops)
+
+ marker = " **" if change and change > 5 else ""
+ print(f" {r['label']:<30} {bl_iops:>10,.0f} {sq_iops:>10,.0f} {fmt_pct(change):>10}{marker}")
+
+ # Latency summary
+ print(f"\n{'=' * 100}")
+ print(" P99 LATENCY SUMMARY (all scenarios)")
+ print(f"{'=' * 100}")
+ print(f"\n {'Test':<30} {'Base(p99)':>12} {'SQ/CQ(p99)':>12} {'Change':>10}")
+ print(f" {'─'*30} {'─'*12} {'─'*12} {'─'*10}")
+
+ for r in results:
+ bl_p99 = r["baseline"]["p99_us"] if r["baseline"] else 0
+ sq_p99 = r["sqcq"]["p99_us"] if r["sqcq"] else 0
+ change = pct_change(bl_p99, sq_p99)
+
+ marker = " **" if change and change < -5 else ""
+ print(f" {r['label']:<30} {bl_p99:>12.1f} {sq_p99:>12.1f} {fmt_pct(change):>10}{marker}")
+
+ print(f"\n * = notable improvement")
+ print(f" ** = significant improvement (>5% IOPS gain or >5% latency reduction)")
+ print()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/Documentation/run-sqcq-compare.sh b/Documentation/run-sqcq-compare.sh
new file mode 100644
index 0000000000000..3a5928bfaef2a
--- /dev/null
+++ b/Documentation/run-sqcq-compare.sh
@@ -0,0 +1,182 @@
+#!/bin/bash
+# run-sqcq-compare.sh - A/B comparison for SQ/CQ polling vs baseline
+#
+# Usage:
+# 1. Boot Guest WITHOUT VIRTIO_F_SQCQ_POLL: ./run-sqcq-compare.sh baseline
+# 2. Boot Guest WITH VIRTIO_F_SQCQ_POLL: ./run-sqcq-compare.sh sqcq-poll
+#
+# Results saved to: perf-results/<mode>_<timestamp>/
+#
+# IMPORTANT: CPU pinning uses --cpus_allowed (NOT taskset).
+# Verify with: ps -eo pid,comm,psr | grep fio
+#
+set -euo pipefail
+
+MODE=${1:?"Usage: $0 <baseline|sqcq-poll>"}
+DEV=${2:-/dev/sda}
+PIN_CPU=7
+RUNTIME=60
+RUNS=3
+
+TIMESTAMP=$(date +%Y%m%d-%H%M%S)
+OUTDIR="perf-results/${MODE}_${TIMESTAMP}"
+mkdir -p "$OUTDIR"
+
+# Verify CPU pinning mapping: PIN_CPU -> hwq -> single Host vq
+echo "=== CPU/mq mapping verification ==="
+echo "PIN_CPU=$PIN_CPU"
+for d in /sys/block/$(basename $DEV)/mq/*/; do
+ dir=$(basename "$d")
+ cpus=$(cat "$d/cpu_list" 2>/dev/null || echo "?")
+ echo " hwq $dir: CPU $cpus"
+done
+echo ""
+
+echo "=== Mode: $MODE | Device: $DEV | CPU pin: $PIN_CPU | Runs: $RUNS ==="
+echo "=== Output: $OUTDIR ==="
+echo ""
+
+# Collect Host dmesg marker (for correlating with Host-side vhost stats)
+dmesg > "$OUTDIR/guest-dmesg-before.log" 2>/dev/null || true
+
+# Test matrix: label rw bs iodepth numjobs
+TESTS=(
+ "4k-randread-od1-nj1 randread 4k 1 1"
+ "4k-randread-od32-nj1 randread 4k 32 1"
+ "4k-randread-od32-nj4 randread 4k 32 4"
+ "4k-randread-od32-nj8 randread 4k 32 8"
+ "4k-randwrite-od1-nj1 randwrite 4k 1 1"
+ "4k-randwrite-od32-nj1 randwrite 4k 32 1"
+ "4k-randwrite-od32-nj4 randwrite 4k 32 4"
+ "4k-randwrite-od32-nj8 randwrite 4k 32 8"
+ "128k-seqread-od1-nj1 read 128k 1 1"
+ "128k-seqread-od32-nj1 read 128k 32 1"
+ "128k-seqread-od32-nj4 read 128k 32 4"
+ "128k-seqread-od32-nj8 read 128k 32 8"
+ "128k-seqwrite-od1-nj1 write 128k 1 1"
+ "128k-seqwrite-od32-nj1 write 128k 32 1"
+ "128k-seqwrite-od32-nj4 write 128k 32 4"
+ "128k-seqwrite-od32-nj8 write 128k 32 8"
+)
+
+for entry in "${TESTS[@]}"; do
+ read -r LABEL RW BS OD NJ <<< "$entry"
+
+ echo "=== $LABEL (iodepth=$OD, numjobs=$NJ) ==="
+
+ BEST_IOPS=0
+ BEST_FILE=""
+
+ for run in $(seq 1 $RUNS); do
+ OUTFILE="${OUTDIR}/${LABEL}_run${run}.json"
+
+ # For numjobs=1, pin to single CPU; for numjobs>1, let fio distribute
+ PIN_ARGS=""
+ if [ "$NJ" -eq 1 ]; then
+ PIN_ARGS="--cpus_allowed=$PIN_CPU --cpus_allowed_policy=shared"
+ fi
+
+ echo " run $run/$RUNS ..."
+ fio --ioengine=libaio \
+ --direct=1 \
+ --rw="$RW" \
+ --bs="$BS" \
+ --iodepth=$OD \
+ --numjobs=$NJ \
+ $PIN_ARGS \
+ --time_based \
+ --runtime=$RUNTIME \
+ --group_reporting \
+ --percentile_list=50,95,99,99.9 \
+ --filename="$DEV" \
+ --name="$LABEL" \
+ --output-format=json \
+ --output="$OUTFILE"
+
+ # Extract IOPS from JSON
+ IOPS=$(python3 -c "
+import json, sys
+with open('$OUTFILE') as f:
+ d = json.load(f)
+jobs = d.get('jobs', [])
+if jobs:
+ rd = jobs[0].get('read', {})
+ wr = jobs[0].get('write', {})
+ iops = rd.get('iops', 0) + wr.get('iops', 0)
+ print(f'{iops:.0f}')
+else:
+ print('0')
+" 2>/dev/null || echo "0")
+
+ echo " IOPS: $IOPS"
+
+ # Track best run
+ if [ "$IOPS" -gt "$BEST_IOPS" ]; then
+ BEST_IOPS=$IOPS
+ BEST_FILE=$OUTFILE
+ fi
+ done
+
+ # Copy best run as the canonical result
+ if [ -n "$BEST_FILE" ]; then
+ cp "$BEST_FILE" "${OUTDIR}/${LABEL}_best.json"
+ fi
+
+ echo ""
+done
+
+# Collect dmesg after
+dmesg > "$OUTDIR/guest-dmesg-after.log" 2>/dev/null || true
+
+# Generate summary
+echo "=== Generating summary ==="
+python3 - <<'PYEOF' "$OUTDIR"
+import json, sys, os, glob
+
+outdir = sys.argv[1]
+print(f"\n{'='*80}")
+print(f" RESULTS: {os.path.basename(outdir)}")
+print(f"{'='*80}\n")
+
+header = f"{'Test':<30} {'IOPS':>10} {'p50(us)':>10} {'p99(us)':>10} {'p99.9(us)':>10} {'BW(MiB/s)':>10} {'lat_avg(us)':>12}"
+print(header)
+print("-" * len(header))
+
+for best in sorted(glob.glob(os.path.join(outdir, "*_best.json"))):
+ label = os.path.basename(best).replace("_best.json", "")
+ try:
+ with open(best) as f:
+ d = json.load(f)
+ jobs = d.get("jobs", [])
+ if not jobs:
+ continue
+
+ j = jobs[0]
+ rd = j.get("read", {})
+ wr = j.get("write", {})
+ iops = rd.get("iops", 0) + wr.get("iops", 0)
+ bw = (rd.get("bw", 0) + wr.get("bw", 0)) / 1024 # KiB/s -> MiB/s
+
+ lat_ns = rd.get("lat_ns", {}) or wr.get("lat_ns", {})
+ avg_us = lat_ns.get("mean", 0) / 1000
+
+ # Percentiles from clat_ns (completion latency)
+ clat = rd.get("clat_ns", {}) or wr.get("clat_ns", {})
+ pct = clat.get("percentile", {}) if isinstance(clat, dict) else {}
+ p50 = pct.get("50.000000", 0) / 1000
+ p99 = pct.get("99.000000", 0) / 1000
+ p999 = pct.get("99.900000", 0) / 1000
+
+ print(f"{label:<30} {iops:>10,.0f} {p50:>10.1f} {p99:>10.1f} {p999:>10.1f} {bw:>10.1f} {avg_us:>12.1f}")
+ except Exception as e:
+ print(f"{label:<30} ERROR: {e}")
+
+print()
+PYEOF
+
+echo ""
+echo "=== Done. Results in $OUTDIR ==="
+echo "=== After running both modes, compare with: ==="
+echo " diff <(python3 summarize.py perf-results/baseline_*) <(python3 summarize.py perf-results/sqcq-poll_*)"
+echo ""
+echo "=== Or paste both summary tables side-by-side for comparison ==="
diff --git a/Documentation/virtio-sqcq-poll-spec.rst b/Documentation/virtio-sqcq-poll-spec.rst
new file mode 100644
index 0000000000000..523bb3e3a1963
--- /dev/null
+++ b/Documentation/virtio-sqcq-poll-spec.rst
@@ -0,0 +1,617 @@
+Virtio SQ/CQ Polling Mode — 技术规格书
+==========================================
+
+Host Kernel vhost 端已实现。Guest Kernel 端已实现。QEMU 端已实现。
+
+1. 概述
+-------
+
+传统 virtio 使用 MMIO kick (Guest→Device) 和 MSI-X 中断 (Device→Guest) 通知，
+每次涉及 VM exit。SQ/CQ polling 借鉴 io_uring SQPOLL 模型，在 split ring 旁引入
+SQ/CQ 门铃结构，由轮询线程检测变更替代中断，消除 VM exit。
+
+- Guest→Device：Guest 更新 SQ.idx，Device 轮询线程检测
+- Device→Guest：Device 更新 CQ.idx，Guest 轮询线程检测
+
+SQ/CQ 门铃仅做通知信号，实际数据仍通过 desc/avail/used ring 传递。
+通过 Feature Bit 协商，未协商时零开销。仅支持 split ring。
+
+1.1 地址传递架构
+~~~~~~~~~~~~~~~
+
+::
+
+ Guest Kernel QEMU (userspace) Host Kernel (vhost)
+ PCI config space GPA → HVA 转换 VHOST_SET_VRING_SQCQ_ADDR
+ (写入 GPA) ioctl (传入 HVA) (接收 HVA)
+
+1.2 通知路径
+~~~~~~~~~~~~
+
+::
+
+ 活跃路径（零开销）：
+ Guest 提交 → SQ.idx → Device 轮询检测（消除 VM exit）
+ Device 完成 → CQ.idx → Guest 轮询检测（消除 VM exit）
+
+ 休眠唤醒路径：
+ Guest 提交 → SQ.idx → MMIO kick → QEMU → eventfd → vhost 唤醒
+ Device 完成 → CQ.idx → eventfd_signal → QEMU → MSI-X → Guest 唤醒
+
+
+2. Feature 协商
+---------------
+
+``VIRTIO_F_SQCQ_POLL = 42``（``include/uapi/linux/virtio_config.h``）。
+
+``VIRTIO_TRANSPORT_F_END`` 从 42 更新为 43。
+该 bit 属于 transport feature range (28..43)，需同时被 virtio_ring 和 virtio_pci 接受。
+vhost 端将该 feature 加入 ``VHOST_FEATURES`` 宏，在 ``VHOST_GET_FEATURES`` ioctl 中对 Guest 可见。
+
+**协商流程：**
+
+1. Device 设置 bit 42 → Guest 接受 → finalize_features
+2. Guest 在 find_vqs 阶段分配 SQ/CQ 结构
+3. Guest 在 vp_active_vq 阶段将 SQ/CQ DMA 地址写入 PCI config space
+4. QEMU 读 GPA → 转 HVA → ``VHOST_SET_VRING_SQCQ_ADDR`` ioctl → vhost
+5. vhost 在 ``VHOST_SCSI_SET_ENDPOINT`` 时启动轮询线程
+
+任一方不支持则回退传统中断模式。
+
+
+3. 数据结构
+-----------
+
+3.1 SQ/CQ 门铃
+~~~~~~~~~~~~~~~
+
+定义于 ``include/uapi/linux/virtio_ring.h``：
+
+.. code-block:: c
+
+ struct vring_sq {
+ __virtio64 idx; /* Producer position: Guest 更新 */
+ __u8 sq_need_wakeup; /* 1 = Device 应被 kick 唤醒 */
+ __u8 reserved[55];
+ } __attribute__((aligned(128))); /* 128 bytes, cache-line 对齐 */
+
+ struct vring_cq {
+ __virtio64 idx; /* Producer position: Device 更新 */
+ __u8 cq_need_wakeup; /* 1 = Guest 应被中断唤醒 */
+ __u8 reserved[55];
+ } __attribute__((aligned(128))); /* 128 bytes, cache-line 对齐 */
+
+- ``sq->idx`` 对应 ``avail->idx``，``cq->idx`` 对应 ``used->idx``
+- 128 字节对齐覆盖 ARM Neoverse N2/V2、Apple M 系列等 128 字节 cache line 的 CPU，
+ 消除 SQ 与 CQ 之间的 false sharing
+- ``sq/cq_need_wakeup`` 为 8-bit 直接赋值字段，用于 NEED_WAKEUP 协议（见第 6 节）
+
+3.2 PCI Config Space 寄存器
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+追加到 ``struct virtio_pci_modern_common_cfg`` 末尾（offset 62 之后），
+定义于 ``include/uapi/linux/virtio_pci.h``：
+
+.. code-block:: c
+
+ #define VIRTIO_PCI_COMMON_SQE_LO 64
+ #define VIRTIO_PCI_COMMON_SQE_HI 68
+ #define VIRTIO_PCI_COMMON_CQE_LO 72
+ #define VIRTIO_PCI_COMMON_CQE_HI 76
+
+ __le32 sqe_ring_lo; /* offset 64: SQ DMA 地址低 32 位 */
+ __le32 sqe_ring_hi; /* offset 68: SQ DMA 地址高 32 位 */
+ __le32 cqe_ring_lo; /* offset 72: CQ DMA 地址低 32 位 */
+ __le32 cqe_ring_hi; /* offset 76: CQ DMA 地址高 32 位 */
+
+受 queue_select 约束。UAPI 已定义，Guest 已实现写入，QEMU 已实现读取。
+
+3.3 vhost UAPI
+~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ // include/uapi/linux/vhost.h
+ #define VHOST_SET_VRING_SQCQ_ADDR _IOW(VHOST_VIRTIO, 0x27, \
+ struct vhost_vring_sqcq_addr)
+
+ // include/uapi/linux/vhost_types.h
+ struct vhost_vring_sqcq_addr {
+ unsigned int index;
+ __u64 sq_user_addr; /* SQ doorbell HVA */
+ __u64 cq_user_addr; /* CQ doorbell HVA */
+ };
+
+QEMU 调用时机：在 ``VHOST_SET_VRING_ADDR`` 之后、``VHOST_SCSI_SET_ENDPOINT`` 之前。
+
+3.4 写入时序
+~~~~~~~~~~~~
+
+Guest（vp_active_vq 中）：queue_select → desc/avail/used 地址 → sqe/cqe_ring_lo/hi 地址
+QEMU：组合 GPA → address_space_map 转 HVA → VHOST_SET_VRING_SQCQ_ADDR ioctl
+
+
+4. 字段读写所有权
+-----------------
+
+.. list-table::
+ :header-rows: 1
+ :widths: 20 20 20 25 15
+
+ * - 字段
+ - 写入方
+ - 读取方
+ - 内存序
+ - 对应
+
+ * - sq->idx
+ - Guest
+ - Device
+ - Guest: smp_store_release; Device: get_user + smp_rmb
+ - avail->idx
+
+ * - sq->sq_need_wakeup
+ - Device
+ - Guest
+ - Device: smp_wmb + put_user + smp_mb (double-check)
+ - NEED_WAKEUP
+
+ * - cq->idx
+ - Device
+ - Guest
+ - Device: smp_wmb before put_user; Guest: smp_load_acquire
+ - used->idx
+
+ * - cq->cq_need_wakeup
+ - Guest
+ - Device
+ - Guest: smp_store_release; Device: get_user + smp_rmb
+ - NEED_WAKEUP
+
+
+5. 核心协议
+-----------
+
+5.1 Guest 提交请求 (SQ 方向)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ // virtqueue_kick() 中 sqcq_poll 模式
+ smp_store_release(&vring->sq->idx, avail_idx_shadow);
+ need_wakeup = smp_load_acquire(&vring->sq->sq_need_wakeup);
+ if (need_wakeup)
+ virtqueue_notify(vq); // MMIO write，仅在 Device 休眠时
+
+**关键：** Guest 仍写 ``avail->idx``（``virtqueue_add_split()`` 完成）。
+SQ idx 是额外通知信号，让 Device 快速检测（读一个 cache line vs 读 avail ring 两次）。
+Device 的 ``handle_kick()`` 通过 ``avail->idx`` 定位 descriptor。
+
+**virtqueue_kick_prepare() 守卫：** sqcq_poll 模式下直接返回 ``true``，
+不检查 ``used->flags``（vhost 不维护该 flag）。
+
+5.2 Device 处理请求 (vhost 轮询线程)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+独立 kthread ``vhost_sqcq_poll_thread``。直接调用 ``handle_kick`` 处理提交，
+通过 ``poll_complete`` 回调处理完成，单线程闭环。
+
+主循环伪代码（drivers/vhost/vhost.c）：
+
+.. code-block:: c
+
+ while (!poll_stop):
+ // 0. 处理 pending completion
+ if (atomic_read(&vq->completion_pending) && vq->poll_complete):
+ atomic_set(&vq->completion_pending, 0)
+ vq->poll_complete(vq) // copy resp, add used, 写 cq->idx, 检查 cq_need_wakeup
+
+ // 1. 读 SQ idx
+ ret = get_user(sq_idx_val, &vq->sq->idx)
+ smp_rmb() // acquire
+ new_sq_idx = (u16)sq_idx_val
+
+ // 2. 有新提交：同步调用 handle_kick
+ if (new_sq_idx != vq->last_avail_idx):
+ vq->handle_kick(&vq->poll.work)
+
+ // 3. 自适应休眠（EMA 动态超时）
+ should_sleep = (last_avail_idx == last_used_idx)
+ if (!should_sleep && ema_ns > 0)
+ should_sleep = (active_io_vqs > 1) && (ema_us > SQCQ_SLEEP_THRESHOLD_US)
+
+ if (should_sleep):
+ timeout = ema_based_or_fallback(SQCQ_IDLE_TIMEOUT_US_MAX)
+ // 设置 NEED_WAKEUP + double-check idx
+ smp_wmb(); put_user(1, &sq->sq_need_wakeup); smp_mb()
+ if (idx changed): put_user(0, &sq->sq_need_wakeup); continue
+ // 注册 kick eventfd，带超时等待
+ kick_register(vq)
+ wait_event_interruptible_timeout(poll_wait,
+ poll_stop || kick_received || completion_pending, timeout)
+ was_kicked = READ_ONCE(kick_received)
+ kick_unregister(vq)
+ put_user(0, &sq->sq_need_wakeup)
+ else:
+ cpu_relax()
+
+5.3 I/O 完成路径
+~~~~~~~~~~~~~~~~~
+
+Poll 模式下 completion 不经过 vhost_worker：
+
+1. **TCM 完成**（中断/workqueue 上下文）：``llist_add`` + ``atomic_set(completion_pending, 1)`` + 条件 ``wake_up``
+2. **Poll thread 步骤 0**（poll thread 上下文）：``poll_complete`` 回调
+3. **poll_complete 内部**：``llist_del_all`` → ``mutex_lock`` → copy response + ``vhost_add_used`` → ``mutex_unlock`` → ``put_user(cq->idx)`` → 检查 ``cq_need_wakeup`` → ``eventfd_signal``
+
+设计要点：单线程闭环（handle_kick 和 poll_complete 顺序执行，共享 mutex 不会死锁）；
+CQ idx 单写者（仅 poll_complete）；条件 wake_up（仅在 poll thread 非 RUNNING 时）。
+
+5.4 vhost_enable/disable_notify 在 poll 模式下
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vhost_enable_notify()`` → ``return false``
+- ``vhost_disable_notify()`` → ``return``
+
+通知控制由 sq_need_wakeup / cq_need_wakeup 承担，used ring flags 不再有效。
+Guest 端对称：``virtqueue_kick_prepare()`` 直接返回 true，``disable_cb/enable_cb`` 操作 avail->flags 但 vhost 忽略。
+
+5.5 Guest 接收完成 (CQ 方向)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Guest 轮询线程（``virtio_sqcq_poll_thread``），每 device 一个线程。
+
+主循环要点：
+
+1. ``while (cq_poll_has_work())`` drain 该 vq 所有完成，callback 后 ``do_softirq()``
+2. 有工作时重置 ``last_io_ts = ktime_get()``，不 yield，快速轮询
+3. 无工作时基于 ``last_io_ts`` 时间窗口自旋（``busy_spin_us`` = 1ms）
+4. 自旋中使用 ``need_resched()`` + ``cpu_relax()`` 协作让出（io_uring SQPOLL 模型）
+5. spin 窗口过期后设置所有 vq 的 ``cq_need_wakeup`` + double-check ``more_used()``
+6. 无新工作则 ``schedule_timeout(idle_timeout_us)`` 休眠（1ms）
+7. 醒来后清除所有 vq 的 NEED_WAKEUP，不重置 spin 窗口（超时唤醒不 re-arm spin）
+
+**CQ 完成检测：** 直接使用 ``more_used(vq)`` 检查 used ring，不再轮询 ``cq->idx``。
+``cq->idx`` 仅由 Device 写入，保留用于 NEED_WAKEUP 协议的 ``cq_need_wakeup`` 字段，
+不再作为完成检测的 fast path（消除了 cq->idx 与 used ring 之间的竞态）。
+
+**CQ 访问器函数** (virtio_ring.c)：
+
+- ``cq_poll_has_work(vq)``：``more_used(vq)`` — 直接检查 used ring
+- ``cq_set_need_wakeup(vq)``：``smp_store_release(&cq->cq_need_wakeup, 1)``
+- ``cq_clear_need_wakeup(vq)``：``WRITE_ONCE(cq->cq_need_wakeup, 0)``
+- ``cq_recheck(vq)``：``smp_mb()`` + ``more_used(vq)``
+
+5.6 Device 唤醒路径 (kick eventfd 桥接)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ Guest MMIO kick → QEMU eventfd write → eventfd wq →
+ vhost_sqcq_kick_wakeup() → kick_received=true + wake_up →
+ poll thread → clear sq_need_wakeup → process sq->idx
+
+休眠前通过 ``vhost_sqcq_kick_register()`` 注册到 kick eventfd wait queue。
+关键：``vfs_poll`` 可能因 stale counter 返回 EPOLLIN，必须验证 ``sq->idx``
+确实有新工作才触发回调，否则所有 poll thread busy-loop。
+
+QEMU 端无需额外操作，现有 kick eventfd 路径即可。
+
+
+6. NEED_WAKEUP 协议（核心，必须严格遵守）
+-----------------------------------------
+
+双向对称，SQ 和 CQ 各一套。避免死锁和丢失唤醒的核心机制。
+
+6.1 SQ 方向
+~~~~~~~~~~~
+
+**Device 休眠前：**
+
+.. code-block:: pseudo
+
+ smp_wmb()
+ sq->sq_need_wakeup = 1
+ smp_mb()
+ new_sq_idx = sq->idx // double-check
+ if (new_sq_idx != last_avail):
+ sq->sq_need_wakeup = 0 // 取消休眠
+ goto process
+ wait_event(kick_received)
+ sq->sq_need_wakeup = 0 // 醒来后清除
+
+**Guest 提交后：**
+
+.. code-block:: pseudo
+
+ smp_store_release(&sq->idx, new_idx)
+ need_wakeup = smp_load_acquire(&sq->sq_need_wakeup)
+ if (need_wakeup)
+ virtqueue_notify(vq) // MMIO kick
+
+6.2 CQ 方向
+~~~~~~~~~~~
+
+ **Guest 休眠前：**
+
+.. code-block:: pseudo
+
+ smp_store_release(&cq->cq_need_wakeup, 1)
+ smp_mb()
+ if (more_used(vq)): // double-check used ring
+ cq->cq_need_wakeup = 0 // 取消休眠
+ goto process
+ schedule_timeout(idle_timeout_us)
+ cq->cq_need_wakeup = 0 // 醒来后清除
+
+**Device 完成后（两阶段）：**
+
+1. release_cmd：``llist_add`` + ``atomic_set(completion_pending, 1)`` + 条件 ``wake_up``
+2. poll_complete：copy resp → add used → ``put_user(cq->idx)`` → 检查 ``cq_need_wakeup`` → ``eventfd_signal``
+
+6.3 为什么需要 Double-check
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+设置 NEED_WAKEUP → 重读 idx 之间存在竞态：Guest 可能在 Device 读 idx 后、
+sleep 前提交。Device 在 sleep 前重读 idx，如果变了则取消休眠。
+此时 Guest 的 kick 可能已到也可能未到，但 Device 已在处理新工作，不影响正确性。
+
+6.4 内存序总结
+~~~~~~~~~~~~~~
+
+- idx 更新：写入方 store-release（Guest: ``smp_store_release``；Device: ``smp_wmb`` + ``put_user``）
+- idx 读取：Guest 使用 ``more_used()`` 直接检查 used ring（不再轮询 ``cq->idx``）
+- sq_need_wakeup 读取：Guest 使用 ``smp_load_acquire``（替代 ``smp_mb`` + ``READ_ONCE``）
+- NEED_WAKEUP 设置后 ``smp_mb()``，然后重读 used ring / sq->idx
+- NEED_WAKEUP 清除在 sleep 唤醒后立即执行
+- x86 上屏障为 no-op（TSO），ARM/RISC-V 上生成实际屏障指令
+
+
+7. QEMU 实现指南
+----------------
+
+7.1 Feature 协商
+~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ virtio_add_feature(&vdev->host_features, VIRTIO_F_SQCQ_POLL);
+
+7.2 GPA → HVA 转换
+~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ hwaddr sq_gpa = (sqe_ring_hi << 32) | sqe_ring_lo;
+ void *sq_hva = address_space_map(&address_space_memory, sq_gpa, sizeof(struct vring_sq), false);
+ // ... cq 类似
+ struct vhost_vring_sqcq_addr addr = { .index = idx, .sq_user_addr = sq_hva, .cq_user_addr = cq_hva };
+ ioctl(vhost_fd, VHOST_SET_VRING_SQCQ_ADDR, &addr);
+
+7.3 ioctl 调用顺序
+~~~~~~~~~~~~~~~~~~
+
+启动：
+
+::
+
+ 1. VHOST_SET_VRING_ADDR (desc/avail/used)
+ 2. VHOST_SET_VRING_SQCQ_ADDR (sq/cq) — 新增
+ 3. VHOST_SET_VRING_KICK
+ 4. VHOST_SET_VRING_CALL
+ 5. VHOST_SCSI_SET_ENDPOINT — 触发轮询线程启动
+
+停止：
+
+::
+
+ 1. VHOST_SCSI_CLEAR_ENDPOINT — 自动停止轮询线程
+ 2. VHOST_SET_VRING_KICK (null_fd)
+ 3. VHOST_SET_VRING_CALL (null_fd)
+
+7.4 call eventfd 处理
+~~~~~~~~~~~~~~~~~~~~~
+
+收到 eventfd signal 说明 Guest 设置了 cq_need_wakeup，需注入 MSI-X 中断。
+使用现有 ``virtio_irq()`` / ``virtio_notify_irqfd()`` 即可。
+
+即使启用 polling，Device 仍需保留中断能力（CQ NEED_WAKEUP 唤醒、config change、fallback）。
+
+7.5 内存序
+~~~~~~~~~~
+
+QEMU 用户态需使用 ``atomic_store_release`` / ``atomic_load_acquire`` 或 C11 ``stdatomic.h``。
+
+
+8. Host Kernel vhost 端实现
+----------------------------
+
+8.1 vhost_virtqueue 扩展
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+ struct vhost_virtqueue {
+ /* ... 现有字段 ... */
+ /* SQ/CQ polling mode */
+ bool sqcq_poll;
+ struct vring_sq __user *sq;
+ struct vring_cq __user *cq;
+ struct task_struct *poll_task;
+ bool poll_stop;
+ wait_queue_head_t poll_wait;
+ struct completion poll_started, poll_stopped;
+ bool kick_received;
+ atomic_t completion_pending;
+ void (*poll_complete)(struct vhost_virtqueue *vq);
+ wait_queue_head_t *kick_wqh;
+ wait_queue_entry_t kick_wait;
+ poll_table kick_pt;
+ /* 10s 统计窗口和 EMA */
+ u64 ema_latency_ns; /* EMA of cmd processing latency */
+ u64 ema_interval_ns; /* EMA of cmd-to-cmd interval */
+ /* ... lat_*/stat_* 统计字段 ... */
+ };
+
+8.2 ioctl 处理
+~~~~~~~~~~~~~~
+
+``vhost_vring_set_sqcq_addr()``：copy_from_user → 验证地址/对齐 → access_ok（backend 已配置时）→ 存储 sq/cq。
+另含自动启动逻辑：当 private_data && sq && cq && !sqcq_poll && poll_complete && feature 时调 ``vhost_sqcq_poll_start()``。
+
+8.3 独立轮询线程
+~~~~~~~~~~~~~~~~~
+
+kthread ``vhost-sqcq-<pid>-<vq_offset>``，通过 ``kthread_use_mm(dev->mm)`` 绑定 QEMU 地址空间。
+直接调用 ``handle_kick`` 处理提交，``poll_complete`` 处理完成，单线程闭环。
+
+API：``vhost_sqcq_poll_start(vq)`` / ``vhost_sqcq_poll_stop(vq)``。
+start 时先创建 kthread 并等待 ``poll_started`` completion，然后才设置 ``sqcq_poll = true``，
+确保 ``vhost_signal()`` 看到 sqcq_poll 时 ``poll_task`` 已有效（消除 NULL deref 窗口）。
+stop 后清除 poll_task、sqcq_poll、sq、cq，flush 残留 completion。
+
+8.4 策略常量与 CPU 绑核
+~~~~~~~~~~~~~~~~~~~~~~~
+
+策略常量（编译期固定，``#define``）：
+
+.. code-block:: c
+
+ #define SQCQ_IDLE_TIMEOUT_US_MAX 1000U /* sleep 时长上界 (us) */
+ #define SQCQ_SLEEP_EMA_MULT 2 /* sleep 时长 = N * ema_latency */
+ #define SQCQ_SLEEP_THRESHOLD_US 20U /* 快于此时长的后端选择 spin */
+ #define SQCQ_SPIN_EMA_MULT 2 /* inflight spin 窗口 = N * ema_latency */
+ #define SQCQ_ACTIVE_TIMEOUT_NS 1000000U /* 无 inflight 固定自旋窗口 1ms */
+ #define SQCQ_STALE_TIMEOUT_NS 1000000000ULL /* sq_was_active 兜底清除 1s */
+
+**CPU 绑核**：round-robin（``sqcq_poll_cpu_rr`` 全局原子计数器 ``% num_online_cpus()``），
+分散到不同在线 CPU。计数器取模确保重启后不会溢出到 fallback（全绑 cpu0）。
+
+**协作式自旋**（io_uring SQPOLL 模型）：两条自旋路径均使用
+``need_resched()`` / ``cond_resched()`` 协作让出——自旋时若调度器标记需要让出，
+主动 ``cond_resched()``，避免饿死同核其他任务。线程保持 TASK_RUNNING（轮询
+``sq->idx``），不进入 sleep 路径、不设 ``sq_need_wakeup``，从而消除 Guest kick
+（VM exit）。仅在无 inflight 且超过 1ms 无新提交时才真正 sleep。
+
+**TCM 完成 CPU 导向**：``sqcq_poll_active_cpu_mask`` 记录忙 poll 线程占用的 CPU
+（跟着 ``sq_was_active`` 翻转）。``vhost_sqcq_pick_completion_cpu()`` 从空闲 CPU
+中选一个给 TCM 完成，避开忙 poll 核；全占满时回退到 next-CPU。
+
+8.5 vhost_signal / notify
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- ``vhost_signal()``：poll 模式下设 completion_pending + 条件 wake_up。SCSI completion 不经过此路径
+- ``vhost_enable_notify()``：poll 模式下 return false
+- ``vhost_disable_notify()``：poll 模式下 return
+- ``vhost_dev_stop()``：仅对 sqcq_poll vq 加锁停止
+
+8.6 vhost-scsi 集成
+~~~~~~~~~~~~~~~~~~~
+
+``set_endpoint`` 中对 I/O VQs（index >= VHOST_SCSI_VQ_IO）：如果 feature + sq + cq 则 start。
+``clear_endpoint`` 中先 stop 再 clear backend。
+
+8.7 call_ctx.ctx NULL 检查
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``poll_complete`` 中 ``eventfd_signal(call_ctx.ctx)`` 前做 NULL 检查。
+QEMU 必须在 SET_ENDPOINT 之前完成 SET_VRING_CALL 设置。
+
+8.8 空闲策略与 EMA
+~~~~~~~~~~~~~~~~~~
+
+系统维护两个独立的 EMA：
+- ``ema_latency_ns``：cmd 处理延迟（submit → process_completions）的 EMA，每次 ``poll_complete`` 后更新 ``old - old/8 + avg/8``（溢出安全）
+- ``ema_interval_ns``：cmd-to-cmd 间隔的 EMA，每次检测到 sq->idx 变化时更新 ``old - old/8 + new/8``
+
+**无 inflight 自旋**（io_uring SQPOLL 模型）：固定 ``SQCQ_ACTIVE_TIMEOUT_NS``（1ms）
+窗口 + ``cond_resched()`` 协作让出。线程持续轮询 ``sq->idx`` 接住下一条提交，
+不进入 sleep 路径、不设 ``sq_need_wakeup`` → 消除 Guest kick（VM exit）。
+仅在超过 1ms 无新提交时才 sleep。
+
+**有 inflight 自旋**：``spin_ns = SQCQ_SPIN_EMA_MULT * ema_latency_ns``，
+``cond_resched()`` 协作让出。超出窗口或多 VQ + 高延迟时 sleep。
+
+Sleep 动态超时：``timeout = ema_latency / (1000 / SQCQ_SLEEP_EMA_MULT)``，clamp 到 [10us, SQCQ_IDLE_TIMEOUT_US_MAX]。
+多 VQ 休眠决策：有 inflight 时仅在 ``sqcq_active_io_vqs > 1`` 且 ``ema_latency_us > SQCQ_SLEEP_THRESHOLD_US`` 时休眠。
+
+8.9 统计与诊断
+~~~~~~~~~~~~~~
+
+10s 统计窗口，打印 cq/fail/sq_hits/enters/wkick/wcomp/avg/kick_pct/wait_pct/cqw_pct/spin_pct/wake_avg/wake_n/interval/ema_lat/inflight 到 dmesg。
+Stuck 检测：wait_event 超时或窗口内无完成但有 inflight 时告警。
+
+9. Guest Kernel 端实现
+-----------------------
+
+9.1 SQ/CQ DMA 分配
+~~~~~~~~~~~~~~~~~~
+
+``virtio_alloc_sqcq()``：``dma_alloc_coherent()`` 一次分配 256 bytes（sq+cq 各 128 字节），
+在 ``vring_create_virtqueue_split()`` 中分配，``vring_free`` 时释放。
+
+9.2 地址传递
+~~~~~~~~~~~~
+
+``vp_modern_queue_address_sqcq()``：在 ``vp_active_vq()`` 中，通过 PCI common config
+的 queue_select + sqe/cqe_ring_lo/hi 写入 64 位 DMA 地址。
+
+9.3 提交路径
+~~~~~~~~~~~~
+
+``virtqueue_kick()``：``smp_store_release(&sq->idx, shadow)`` → 如果 NEED_WAKEUP 则 MMIO kick。
+``virtqueue_kick_prepare()``：sqqcq_poll 下直接 return true。
+
+9.4 CQ 轮询线程
+~~~~~~~~~~~~~~~~~
+
+独立 kthread ``virtio-sqcq/<device_name>``，每 device 一个线程（``struct virtio_sqcq_poll_data``）。
+
+**CQ 访问器函数** (virtio_ring.c)：
+
+- ``cq_poll_has_work(vq)``：``smp_load_acquire(&cq->idx) != last_cq_head``
+- ``cq_poll_consume(vq)``：snapshot cq->idx（callback 前）
+- ``cq_set_need_wakeup(vq)``：``smp_store_release(&cq->cq_need_wakeup, 1)``
+- ``cq_clear_need_wakeup(vq)``：``WRITE_ONCE(cq->cq_need_wakeup, 0)``
+- ``cq_recheck(vq)``：``smp_mb()`` + ``smp_load_acquire(&cq->idx) != last_cq_head``
+
+**线程生命周期：** ``vp_modern_find_vqs()`` 中启动（``virtio_start_sqcq_poll(vdev, 1000, 1000)``），
+注册所有 poll 活跃的 vq。``vp_del_vqs()`` 中停止（``virtio_stop_sqcq_poll`` 内部释放所有 vq node）。
+
+**启动参数：** ``idle_timeout_us=1000``（1ms sleep 超时），``busy_spin_us=1000``（1ms spin 预算）。
+不绑核（调度器自由分配），无指数退避（固定 1ms sleep）。
+
+**config 空间大小检查：** ``vp_check_common_size()`` 验证 common config 至少 80 字节。
+
+
+10. 与现有机制的关系
+---------------------
+
+- **EVENT_IDX：** 可协商但对 polling vq 无效，被 sq/cq_need_wakeup 替代
+- **virtio-net NAPI：** 可共存，NAPI 可直接检查 CQ
+- **blk-mq iopoll：** 互补（iopoll 同步快路径，SQ/CQ 替代异步中断路径）
+- **Live Migration：** reset 时回退中断模式，迁移后 Guest 重新协商
+
+
+11. 文件清单
+------------
+
+**Guest Kernel：** ``virtio_config.h`` (feature bit), ``virtio_ring.h`` (vring_sq/cq),
+``virtio_pci.h`` (PCI offsets), ``virtio_ring.c`` (分配/SQ提交/CQ accessor),
+``virtio_sqcq_poll.c`` (CQ 轮询线程), ``virtio_pci_modern.c`` (feature/queue激活/线程生命周期),
+``virtio_pci_common.c`` (vp_del_vqs), ``virtio_pci_modern_dev.c`` (PCI config 映射)
+
+**Host Kernel vhost：** ``vhost.h`` (ioctl), ``vhost_types.h`` (sqcq_addr struct),
+``vhost.h/vhost.c`` (轮询线程/ioctl/signal/生命周期), ``scsi.c`` (vhost-scsi 集成)
+
+**QEMU：** ``virtio-pci.c`` (feature/config), ``virtio-pci-modern.c`` (BAR/GPA→HVA)
+
+
+12. 验证方案
+------------
+
+**功能：** QEMU 带 VIRTIO_F_SQCQ_POLL + virtio-scsi → Guest 读写验证正确性。
+
+**性能：** fio 对比中断模式 vs polling，观察 IOPS/延迟/VM exit 次数。
+
+**正确性：** 24h+ stress test；idle/wakeup 验证；NEED_WAKEUP 边界条件；ARM/RISC-V 内存序验证。
diff --git a/Documentation/virtio-sqcq-poll-virtio-spec.rst b/Documentation/virtio-sqcq-poll-virtio-spec.rst
new file mode 100644
index 0000000000000..26ae41860a8e5
--- /dev/null
+++ b/Documentation/virtio-sqcq-poll-virtio-spec.rst
@@ -0,0 +1,355 @@
+SQ/CQ Doorbell Polling
+======================
+
+This section describes the SQ/CQ doorbell polling mechanism
+(``VIRTIO_F_SQCQ_POLL``), which allows the driver and device to use
+shared-memory doorbells instead of MMIO writes (notifications) and
+interrupts, eliminating VM exits on both submission and completion paths.
+
+This feature is only supported with split virtqueues. A driver or
+device that negotiates this feature MUST NOT also negotiate
+``VIRTIO_F_RING_PACKED``.
+
+
+Feature bit
+-----------
+
+``VIRTIO_F_SQCQ_POLL (42)`` — SQ/CQ Doorbell Polling.
+
+This feature is in the transport feature range (28..43). Both
+``virtio_ring`` and ``virtio_pci`` MUST accept this feature for it
+to be negotiated.
+
+
+Data Structures
+---------------
+
+SQ Doorbell
+~~~~~~~~~~~
+
+The SQ (Submission Queue) doorbell is a cache-line aligned structure
+written by the driver and read by the device:
+
+.. code-block:: c
+
+ struct vring_sq {
+ le64 idx;
+ u8 sq_need_wakeup;
+ u8 reserved[55];
+ };
+
+The structure MUST be aligned to the cache line size of the platform
+(minimum 64 bytes; 128 bytes recommended for platforms with 128-byte
+cache lines such as ARM Neoverse N2/V2).
+
+``idx`` corresponds to ``avail->idx`` and is written by the driver
+using ``smp_store_release()``.
+
+``sq_need_wakeup`` is a single-byte flag written by the device,
+indicating whether the device's poll thread is sleeping and needs to
+be woken by an MMIO write (kick).
+
+CQ Doorbell
+~~~~~~~~~~~
+
+The CQ (Completion Queue) doorbell is a cache-line aligned structure
+written by the device and read by the driver:
+
+.. code-block:: c
+
+ struct vring_cq {
+ le64 idx;
+ u8 cq_need_wakeup;
+ u8 reserved[55];
+ };
+
+The structure MUST be aligned to the same alignment as ``vring_sq``.
+
+``idx`` corresponds to ``used->idx`` and is written by the device.
+
+``cq_need_wakeup`` is a single-byte flag written by the driver,
+indicating whether the driver's poll thread is sleeping and needs to
+be woken by an interrupt.
+
+
+PCI Configuration
+-----------------
+
+The SQ and CQ doorbell DMA addresses are passed through the PCI
+common configuration space. Four new registers are appended to
+``struct virtio_pci_modern_common_cfg``:
+
+============== ======== ========================================
+Offset Field Description
+============== ======== ========================================
+64 sqe_ring_lo SQ doorbell DMA address, low 32 bits
+68 sqe_ring_hi SQ doorbell DMA address, high 32 bits
+72 cqe_ring_lo CQ doorbell DMA address, low 32 bits
+76 cqe_ring_hi CQ doorbell DMA address, high 32 bits
+============== ======== ========================================
+
+These registers are scoped by ``queue_select``. The driver writes
+the SQ and CQ doorbell DMA addresses for each virtqueue during
+``vp_active_vq``.
+
+The device (QEMU) translates the guest physical address to a host
+virtual address and passes it to the vhost backend via
+``VHOST_SET_VRING_SQCQ_ADDR`` ioctl.
+
+
+Allocation
+----------
+
+The driver allocates SQ and CQ doorbells as a single
+``dma_alloc_coherent()`` call (contiguous physical memory):
+
+.. code-block:: c
+
+ size_t total = sizeof(struct vring_sq) + sizeof(struct vring_cq);
+ void *buf = dma_alloc_coherent(dev, total, &dma_addr, GFP_KERNEL);
+ vq->sq = buf;
+ vq->cq = buf + sizeof(struct vring_sq);
+
+Control and event virtqueues MUST NOT use SQ/CQ polling. Only request
+virtqueues (index >= VIRTIO_SCSI_VQ_BASE for virtio-scsi) negotiate
+this feature.
+
+
+Submission Path (Driver → Device)
+---------------------------------
+
+When the driver has new requests available:
+
+1. The driver writes descriptors to the avail ring as usual
+2. The driver updates ``sq->idx`` using ``smp_store_release()``
+3. The driver reads ``sq_need_wakeup`` using ``smp_load_acquire()``
+
+If ``sq_need_wakeup`` is 0, the device's poll thread is actively
+spinning and will detect the new ``sq->idx`` value without any
+further action.
+
+If ``sq_need_wakeup`` is 1, the device's poll thread is sleeping.
+The driver MUST perform a traditional MMIO write (kick) to wake it.
+
+.. code-block:: pseudo
+
+ smp_store_release(&sq->idx, avail_idx_shadow)
+ need_wakeup = smp_load_acquire(&sq->sq_need_wakeup)
+ if (need_wakeup):
+ mmio_kick(vq)
+
+The ``virtqueue_kick_prepare()`` function MUST return ``true``
+unconditionally when SQ/CQ polling is active, bypassing the
+``used->flags`` check.
+
+
+Completion Path (Device → Driver)
+---------------------------------
+
+Device Side
+~~~~~~~~~~~
+
+When the device processes completions:
+
+1. The device writes completion entries to the used ring as usual
+2. The device's poll thread writes ``cq->idx`` to signal completion
+
+The device's poll thread is a dedicated kernel thread that processes
+both submissions (via ``handle_kick``) and completions (via
+``poll_complete`` callback) in a single-threaded loop.
+
+Driver Side
+~~~~~~~~~~~
+
+The driver's poll thread (one per device) detects completions by
+checking ``more_used()`` — directly comparing ``used->idx`` with
+``last_used_idx``:
+
+.. code-block:: pseudo
+
+ while more_used(vq):
+ consume_and_callback(vq)
+ do_softirq()
+
+Note: The driver polls ``used->idx`` directly rather than ``cq->idx``,
+because ``cq->idx`` may lag behind ``used->idx`` (the device writes
+the used ring entries before updating ``cq->idx``). Polling
+``used->idx`` ensures the lowest-latency completion detection.
+
+``cq->idx`` is retained for the ``cq_need_wakeup`` sleep/wake
+protocol (see below) but is not polled as a fast-path completion
+detector.
+
+
+NEED_WAKEUP Protocol
+--------------------
+
+The NEED_WAKEUP protocol prevents lost wakeups when either side
+transitions from polling to sleeping. It is symmetric: SQ uses
+``sq_need_wakeup`` and CQ uses ``cq_need_wakeup``.
+
+SQ NEED_WAKEUP (Device sleeping)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before the device's poll thread sleeps:
+
+1. Set ``sq_need_wakeup = 1`` (via ``put_user``)
+2. Execute ``smp_mb()``
+3. Re-read ``sq->idx`` — if it changed, cancel sleep
+
+After waking (via eventfd or timeout):
+- Clear ``sq_need_wakeup = 0``
+
+The driver checks ``sq_need_wakeup`` after writing ``sq->idx`` and
+performs an MMIO kick only if the flag is set.
+
+CQ NEED_WAKEUP (Driver sleeping)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before the driver's poll thread sleeps:
+
+1. Set ``cq_need_wakeup = 1`` (via ``smp_store_release``)
+2. Execute ``smp_mb()``
+3. Re-check ``more_used()`` — if work found, cancel sleep
+
+After waking (via eventfd/interrupt or timeout):
+- Clear ``cq_need_wakeup = 0``
+
+The device checks ``cq_need_wakeup`` after writing ``cq->idx`` and
+sends an eventfd signal only if the flag is set.
+
+
+Memory Ordering
+---------------
+
+The following memory ordering rules MUST be observed:
+
+==================== ============================ ============================
+Field Writer Reader
+==================== ============================ ============================
+``sq->idx`` ``smp_store_release`` ``get_user`` + ``smp_rmb``
+``sq_need_wakeup`` ``put_user`` + ``smp_wmb`` ``smp_load_acquire``
+``cq->idx`` ``put_user`` (with ``smp_wmb``) Not polled (see note)
+``cq_need_wakeup`` ``smp_store_release`` ``get_user`` + ``smp_rmb``
+``used->idx`` Device (via ``vhost_add_used``) ``more_used()`` (plain read)
+==================== ============================ ============================
+
+Note: ``cq->idx`` is written by the device but is not polled by the
+driver's fast path. The driver uses ``more_used()`` (checking
+``used->idx``) for completion detection, which is the authoritative
+source of completions.
+
+On x86 (TSO memory model), all barriers are no-ops. On ARM64 and
+RISC-V, ``smp_store_release`` generates ``stlr`` and
+``smp_load_acquire`` generates ``ldar``, which are lighter than full
+memory barriers (``dmb ish``).
+
+
+Poll Thread Behavior
+--------------------
+
+Device Poll Thread (vhost)
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+One kernel thread per virtqueue. Each thread:
+
+1. Polls ``sq->idx`` for new submissions and ``completion_pending``
+ for completions
+2. When active: busy-spins using ``need_resched()`` + ``cpu_relax()``
+ for cooperative yielding (io_uring SQPOLL pattern)
+3. Spin budget: fixed 1ms (``SQCQ_ACTIVE_TIMEOUT_NS``) for no-inflight,
+ or ``2 × ema_latency`` for has-inflight
+4. After spin budget expires: sets ``sq_need_wakeup``, sleeps via
+ ``wait_event_interruptible_timeout``
+5. Sleep timeout: adaptive, based on EMA latency, clamped to [10us, 1ms]
+
+CPU affinity: round-robin across online CPUs
+(``atomic_counter % num_online_cpus()``).
+
+Driver Poll Thread
+~~~~~~~~~~~~~~~~~~
+
+One kernel thread per device (serves all registered virtqueues). The
+thread:
+
+1. Iterates all registered virtqueues, polls ``more_used()`` for each
+2. When active: busy-spins within a time budget (``busy_spin_us = 1ms``)
+ from ``last_io_ts``, using ``need_resched()`` + ``cpu_relax()``
+3. After spin budget expires: sets ``cq_need_wakeup`` on all VQs,
+ double-checks ``more_used()``, sleeps via ``schedule_timeout``
+4. Sleep timeout: fixed 1ms (``idle_timeout_us = 1000``)
+5. ``do_softirq()`` called after each completion callback (no hardware
+ interrupts means softirqs must be explicitly flushed)
+
+Parameters (configurable at thread creation):
+
+- ``idle_timeout_us``: sleep timeout in microseconds (default: 1000)
+- ``busy_spin_us``: spin budget in microseconds (default: 1000)
+
+
+Feature Negotiation
+-------------------
+
+1. The device offers ``VIRTIO_F_SQCQ_POLL`` in its feature bits
+2. The driver accepts the feature during ``finalize_features``
+3. The driver allocates SQ/CQ doorbells in ``find_vqs``
+4. The driver writes SQ/CQ DMA addresses to PCI config space
+5. The device (vhost) receives addresses and starts poll threads
+6. ``virtio_features_ok()`` MUST reject
+ ``VIRTIO_F_SQCQ_POLL | VIRTIO_F_RING_PACKED``
+
+If either side does not support the feature, traditional MMIO kick +
+MSI-X interrupt mode is used with zero overhead.
+
+
+Mutual Exclusion
+----------------
+
+``VIRTIO_F_SQCQ_POLL`` and ``VIRTIO_F_RING_PACKED`` MUST NOT be
+negotiated simultaneously. The driver's ``virtio_features_ok()``
+function rejects this combination.
+
+This restriction exists because the SQ/CQ doorbell implementation
+accesses split-ring-specific fields (``avail_idx_shadow``,
+``vring->used->idx``) unconditionally in ``virtqueue_notify()``.
+Packed ring support is a future enhancement.
+
+
+Relationship to Existing Mechanisms
+-----------------------------------
+
+- **EVENT_IDX** (``VIRTIO_RING_F_EVENT_IDX``): may be negotiated but
+ has no effect on polling virtqueues. Notification suppression is
+ handled by ``sq_need_wakeup`` / ``cq_need_wakeup`` instead.
+
+- **blk-mq iopoll**: complementary. iopoll handles synchronous
+ polling from the submitting task; SQ/CQ replaces the asynchronous
+ interrupt path.
+
+- **vhost-vDPA**: vDPA provides its own doorbell mmap mechanism.
+ SQ/CQ polling is specific to the vhost-scsi kernel backend and
+ does not require vDPA hardware support.
+
+
+Performance Characteristics
+---------------------------
+
+Benchmark results (fio, 4K random I/O, arm64 with 8 vCPUs):
+
+============================ ============ =========== =========
+Test Baseline SQ/CQ Poll Change
+============================ ============ =========== =========
+randread od1 nj1 22,427 IOPS 28,289 IOPS +26.1%
+randread od32 nj1 89,910 IOPS 75,665 IOPS -15.8%
+randread od32 nj4 186,763 IOPS 379,549 IOPS +103.2%
+randread od32 nj8 199,967 IOPS 550,633 IOPS +175.4%
+randwrite od1 nj1 21,912 IOPS 27,261 IOPS +24.4%
+randwrite od32 nj1 85,349 IOPS 81,389 IOPS -4.6%
+randwrite od32 nj4 190,443 IOPS 355,811 IOPS +86.8%
+randwrite od32 nj8 196,552 IOPS 566,640 IOPS +188.3%
+============================ ============ =========== =========
+
+Multi-queue workloads (nj4/nj8) see 87-188% improvement from
+eliminated VM exits on both submission and completion paths.
+Single-queue high-queue-depth workloads (od32 nj1) show a
+minor regression due to polling overhead vs. the VM exit savings.
diff --git a/build-riscv-kernel.sh b/build-riscv-kernel.sh
new file mode 100644
index 0000000000000..2ace46278369e
--- /dev/null
+++ b/build-riscv-kernel.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# 设置变量
+export ARCH=riscv
+export CROSS_COMPILE=riscv64-linux-gnu-
+export KERNEL_DIR=$(pwd)
+export OUTPUT_DIR="${KERNEL_DIR}/output"
+
+# 清理
+make mrproper
+
+# 配置
+make defconfig
+# 或使用特定配置
+# cp ${CONFIG_FILE} .config
+
+# 编译内核
+echo "开始编译内核..."
+make -j$(nproc) || exit 1
+
+# 编译设备树
+echo "编译设备树..."
+make dtbs || exit 1
+
+echo "编译完成！"
+echo "内核镜像: ${KERNEL_DIR}/arch/riscv/boot/Image"
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 1c22880e72267..4d65205fcb23e 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -142,6 +142,7 @@ struct vhost_scsi_cmd {
struct llist_node tvc_completion_list;
/* Used to track inflight cmd */
struct vhost_scsi_inflight *inflight;
+ ktime_t tvc_submit_ts;
};
struct vhost_scsi_nexus {
@@ -234,6 +235,7 @@ struct vhost_scsi_virtqueue {
struct vhost_work completion_work;
struct llist_head completion_list;
+ ktime_t stats_window;
};
struct vhost_scsi {
@@ -490,9 +492,18 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd)
struct vhost_scsi_virtqueue *svq = container_of(cmd->tvc_vq,
struct vhost_scsi_virtqueue, vq);
- llist_add(&cmd->tvc_completion_list, &svq->completion_list);
- if (!vhost_vq_work_queue(&svq->vq, &svq->completion_work))
- vhost_scsi_drop_cmds(svq);
+ llist_add(&cmd->tvc_completion_list, &svq->completion_list);
+ if (READ_ONCE(svq->vq.sqcq_poll)) {
+ smp_wmb();
+ atomic_set(&svq->vq.completion_pending, 1);
+ smp_mb();
+ if (!READ_ONCE(svq->vq.poll_stop) &&
+ READ_ONCE(svq->vq.poll_task) &&
+ !task_is_running(svq->vq.poll_task))
+ wake_up(&svq->vq.poll_wait);
+ } else if (!vhost_vq_work_queue(&svq->vq, &svq->completion_work)) {
+ vhost_scsi_drop_cmds(svq);
+ }
}
}
@@ -678,15 +689,11 @@ static int vhost_scsi_copy_sgl_to_iov(struct vhost_scsi_cmd *cmd)
return 0;
}
-/* Fill in status and signal that we are done processing this command
- *
- * This is scheduled in the vhost work queue so we are called with the owner
- * process mm and can access the vring.
+/* Process completion_list: copy responses, add used entries, release cmds.
+ * Returns true if any completions were signalled.
*/
-static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
+static bool vhost_scsi_process_completions(struct vhost_scsi_virtqueue *svq)
{
- struct vhost_scsi_virtqueue *svq = container_of(work,
- struct vhost_scsi_virtqueue, completion_work);
struct virtio_scsi_cmd_resp v_rsp;
struct vhost_scsi_cmd *cmd, *t;
struct llist_node *llnode;
@@ -726,6 +733,9 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
signal = true;
vhost_add_used(cmd->tvc_vq, cmd->tvc_vq_desc, 0);
+ cmd->tvc_vq->lat_scsi_ns +=
+ ktime_to_ns(ktime_sub(ktime_get(), cmd->tvc_submit_ts));
+ cmd->tvc_vq->stat_completions++;
} else
pr_err("Faulted on virtio_scsi_cmd_resp\n");
@@ -737,10 +747,70 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
mutex_unlock(&svq->vq.mutex);
+ return signal;
+}
+
+/* Worker-path completion (non-poll mode only) */
+static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(work,
+ struct vhost_scsi_virtqueue, completion_work);
+ struct vhost_virtqueue *vq = &svq->vq;
+ bool signal;
+
+ signal = vhost_scsi_process_completions(svq);
+
if (signal)
vhost_signal(&svq->vs->dev, &svq->vq);
+
+ if (vq->stat_completions > 0) {
+ u64 now = ktime_get_ns();
+
+ if (!svq->stats_window)
+ svq->stats_window = ns_to_ktime(now);
+
+ if (now - ktime_to_ns(svq->stats_window) >= 10 * NSEC_PER_SEC) {
+ int vi;
+
+ for (vi = 0; vi < svq->vs->dev.nvqs; vi++)
+ if (svq->vs->dev.vqs[vi] == vq)
+ break;
+ pr_info("vhost-baseline: vq[%d] cmd=%llu avg_lat=%lluns\n",
+ vi, vq->stat_completions,
+ div64_u64(vq->lat_scsi_ns, vq->stat_completions));
+ svq->stats_window = ns_to_ktime(now);
+ vq->lat_scsi_ns = 0;
+ vq->stat_completions = 0;
+ }
+ }
}
+/* Poll-thread-path completion: write cq->idx and signal Guest directly */
+static void vhost_scsi_poll_complete(struct vhost_virtqueue *vq)
+{
+ struct vhost_scsi_virtqueue *svq = container_of(vq,
+ struct vhost_scsi_virtqueue, vq);
+ bool signal;
+
+ signal = vhost_scsi_process_completions(svq);
+
+ if (signal && svq->vq.cq) {
+ __virtio64 cq_idx_val;
+ __u8 cq_need_wakeup;
+
+ smp_mb();
+ cq_idx_val = (__force __virtio64)READ_ONCE(svq->vq.last_used_idx);
+ put_user(cq_idx_val, &svq->vq.cq->idx);
+ smp_mb(); /* Ensure cq->idx is visible before reading cq_need_wakeup */
+ if (!get_user(cq_need_wakeup, &svq->vq.cq->cq_need_wakeup)) {
+ smp_rmb();
+ if (cq_need_wakeup && svq->vq.call_ctx.ctx)
+ eventfd_signal(svq->vq.call_ctx.ctx);
+ }
+ }
+}
+
+
static struct vhost_scsi_cmd *
vhost_scsi_get_cmd(struct vhost_virtqueue *vq, u64 scsi_tag)
{
@@ -1060,9 +1130,25 @@ static void vhost_scsi_target_queue_cmd(struct vhost_scsi_nexus *nexus,
}
se_cmd->tag = 0;
+ cmd->tvc_submit_ts = ktime_get();
target_init_cmd(se_cmd, nexus->tvn_se_sess, &cmd->tvc_sense_buf[0],
lun, exp_data_len, vhost_scsi_to_tcm_attr(task_attr),
- data_dir, TARGET_SCF_ACK_KREF);
+ data_dir,
+ TARGET_SCF_ACK_KREF |
+ (READ_ONCE(cmd->tvc_vq->sqcq_poll) ?
+ TARGET_SCF_USE_CPUID : 0));
+
+ /* Poll mode: steer TCM completion to next CPU to avoid
+ * starvation behind poll thread. Skip if fabric has
+ * custom cmd_compl_affinity (-2=UNBOUND or specific CPU).
+ */
+ if (READ_ONCE(cmd->tvc_vq->sqcq_poll) && num_online_cpus() > 1) {
+ struct se_wwn *wwn = se_cmd->se_sess->se_tpg->se_tpg_wwn;
+
+ if (!wwn || wwn->cmd_compl_affinity == SE_COMPL_AFFINITY_CPUID)
+ se_cmd->cpuid =
+ vhost_sqcq_pick_completion_cpu(smp_processor_id());
+ }
if (target_submit_prep(se_cmd, cdb, sg_ptr,
cmd->tvc_sgl_count, NULL, 0, sg_prot_ptr,
@@ -1346,7 +1432,6 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq)
ret = vhost_scsi_chk_size(vq, &vc);
if (ret)
goto err;
-
ret = vhost_scsi_get_req(vq, &vc, &tpg);
if (ret)
goto err;
@@ -2070,6 +2155,15 @@ vhost_scsi_set_endpoint(struct vhost_scsi *vs,
mutex_lock(&vq->mutex);
vhost_vq_set_backend(vq, vs_tpg);
vhost_vq_init_access(vq);
+ if (i >= VHOST_SCSI_VQ_IO &&
+ vhost_has_feature(vq, VIRTIO_F_SQCQ_POLL) &&
+ vq->sq && vq->cq) {
+ ret = vhost_sqcq_poll_start(vq);
+ if (ret) {
+ mutex_unlock(&vq->mutex);
+ goto stop_poll_threads;
+ }
+ }
mutex_unlock(&vq->mutex);
}
ret = 0;
@@ -2086,6 +2180,19 @@ vhost_scsi_set_endpoint(struct vhost_scsi *vs,
vs->vs_tpg = vs_tpg;
goto out;
+stop_poll_threads:
+ /* VQ at index i failed to start. Stop poll threads for all
+ * previously started VQs (indices i-1 down to VHOST_SCSI_VQ_IO).
+ */
+ for (i = i - 1; i >= VHOST_SCSI_VQ_IO; i--) {
+ vq = &vs->vqs[i].vq;
+ mutex_lock(&vq->mutex);
+ vhost_sqcq_poll_stop(vq);
+ vhost_vq_set_backend(vq, NULL);
+ mutex_unlock(&vq->mutex);
+ }
+ /* Reset i for full vq cmds cleanup */
+ i = vs->dev.nvqs;
destroy_vq_cmds:
for (i--; i >= VHOST_SCSI_VQ_IO; i--) {
if (!vhost_vq_get_backend(&vs->vqs[i].vq))
@@ -2163,9 +2270,17 @@ vhost_scsi_clear_endpoint(struct vhost_scsi *vs,
/* Prevent new cmds from starting and accessing the tpgs/sessions */
for (i = 0; i < vs->dev.nvqs; i++) {
vq = &vs->vqs[i].vq;
- mutex_lock(&vq->mutex);
- vhost_vq_set_backend(vq, NULL);
- mutex_unlock(&vq->mutex);
+ if (!READ_ONCE(vq->sqcq_poll)) {
+ mutex_lock(&vq->mutex);
+ vhost_vq_set_backend(vq, NULL);
+ mutex_unlock(&vq->mutex);
+ } else {
+ mutex_lock(&vq->mutex);
+ vhost_sqcq_poll_stop(vq);
+ vhost_vq_set_backend(vq, NULL);
+ mutex_unlock(&vq->mutex);
+ vhost_sqcq_poll_flush(vq);
+ }
}
/* Make sure cmds are not running before tearing them down. */
vhost_scsi_flush(vs);
@@ -2221,6 +2336,7 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
bool is_log, was_log;
int i;
+
if (features & ~VHOST_SCSI_FEATURES)
return -EOPNOTSUPP;
@@ -2321,6 +2437,7 @@ static int vhost_scsi_open(struct inode *inode, struct file *f)
vhost_work_init(&svq->completion_work,
vhost_scsi_complete_cmd_work);
svq->vq.handle_kick = vhost_scsi_handle_kick;
+ svq->vq.poll_complete = vhost_scsi_poll_complete;
}
vhost_dev_init(&vs->dev, vqs, nvqs, UIO_MAXIOV,
VHOST_SCSI_WEIGHT, 0, true, NULL);
@@ -2415,6 +2532,45 @@ vhost_scsi_ioctl(struct file *f,
if (copy_from_user(&features, featurep, sizeof features))
return -EFAULT;
return vhost_scsi_set_features(vs, features);
+ case VHOST_GET_FEATURES_ARRAY: {
+ u64 count, copied;
+ u64 farr[VIRTIO_FEATURES_U64S] = {};
+
+ if (get_user(count, (u64 __user *)argp))
+ return -EFAULT;
+ copied = min_t(u64, count, VIRTIO_FEATURES_U64S);
+ farr[0] = VHOST_SCSI_FEATURES;
+ if (copy_to_user((u64 __user *)(argp + sizeof(u64)),
+ farr, copied * sizeof(u64)))
+ return -EFAULT;
+ if (count > copied &&
+ clear_user((u64 __user *)(argp + sizeof(u64) +
+ copied * sizeof(u64)),
+ (count - copied) * sizeof(u64)))
+ return -EFAULT;
+ return 0;
+ }
+ case VHOST_SET_FEATURES_ARRAY: {
+ u64 count, copied, all_features[VIRTIO_FEATURES_U64S] = {};
+ int i;
+
+ if (get_user(count, (u64 __user *)argp))
+ return -EFAULT;
+ copied = min_t(u64, count, VIRTIO_FEATURES_U64S);
+ if (copy_from_user(all_features,
+ (u64 __user *)(argp + sizeof(u64)),
+ copied * sizeof(u64)))
+ return -EFAULT;
+ for (i = copied; i < count; i++) {
+ u64 tmp;
+ if (get_user(tmp, (u64 __user *)(argp +
+ sizeof(u64) + i * sizeof(u64))))
+ return -EFAULT;
+ if (tmp)
+ return -EOPNOTSUPP;
+ }
+ return vhost_scsi_set_features(vs, all_features[0]);
+ }
case VHOST_NEW_WORKER:
case VHOST_FREE_WORKER:
case VHOST_ATTACH_VRING_WORKER:
@@ -2424,6 +2580,7 @@ vhost_scsi_ioctl(struct file *f,
mutex_unlock(&vs->dev.mutex);
return r;
default:
+
mutex_lock(&vs->dev.mutex);
r = vhost_dev_ioctl(&vs->dev, ioctl, argp);
/* TODO: flush backend after dev ioctl. */
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 2f2c45d208832..68fb53c40b2e6 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -42,6 +42,20 @@ static int max_iotlb_entries = 2048;
module_param(max_iotlb_entries, int, 0444);
MODULE_PARM_DESC(max_iotlb_entries,
"Maximum number of iotlb entries. (default: 2048)");
+#define SQCQ_IDLE_TIMEOUT_US_MAX 1000U
+#define SQCQ_SLEEP_EMA_MULT 2
+#define SQCQ_SPIN_EMA_MULT 2
+#define SQCQ_ACTIVE_TIMEOUT_NS 1000000U /* cap active-VQ clear at 1 ms */
+#define SQCQ_STALE_TIMEOUT_NS 1000000000ULL /* backstop EMA/active clear at 1 s */
+
+static atomic_t sqcq_poll_cpu_rr = ATOMIC_INIT(-1);
+static atomic_t sqcq_active_io_vqs = ATOMIC_INIT(0);
+/* CPUs with a BUSY poll thread (actively processing cmds); TCM completion
+ * steers away. Idle (sleeping) poll threads don't occupy their CPU, so
+ * their bit is cleared via the sq_was_active transitions in the poll loop.
+ */
+static cpumask_t sqcq_poll_active_cpu_mask;
+
static bool fork_from_owner_default = VHOST_FORK_OWNER_TASK;
#ifdef CONFIG_VHOST_ENABLE_FORK_OWNER_CONTROL
@@ -390,6 +404,20 @@ static void vhost_vq_reset(struct vhost_dev *dev,
vhost_disable_cross_endian(vq);
vhost_reset_is_le(vq);
vq->busyloop_timeout = 0;
+ WRITE_ONCE(vq->sqcq_poll, false);
+ vq->sq = NULL;
+ vq->cq = NULL;
+ vq->poll_task = NULL;
+ WRITE_ONCE(vq->poll_stop, false);
+ init_waitqueue_head(&vq->poll_wait);
+ init_completion(&vq->poll_started);
+ init_completion(&vq->poll_stopped);
+ WRITE_ONCE(vq->kick_received, false);
+ atomic_set(&vq->completion_pending, 0);
+ vq->kick_wqh = NULL;
+ vq->stat_completions = 0;
+ vq->lat_scsi_ns = 0;
+ vq->poll_cpu = -1;
vq->umem = NULL;
vq->iotlb = NULL;
rcu_assign_pointer(vq->worker, NULL);
@@ -1168,6 +1196,15 @@ void vhost_dev_stop(struct vhost_dev *dev)
{
int i;
+ for (i = 0; i < dev->nvqs; ++i) {
+ if (!READ_ONCE(dev->vqs[i]->sqcq_poll))
+ continue;
+ mutex_lock(&dev->vqs[i]->mutex);
+ vhost_sqcq_poll_stop(dev->vqs[i]);
+ mutex_unlock(&dev->vqs[i]->mutex);
+ vhost_sqcq_poll_flush(dev->vqs[i]);
+ }
+
for (i = 0; i < dev->nvqs; ++i) {
if (dev->vqs[i]->kick && dev->vqs[i]->handle_kick)
vhost_poll_stop(&dev->vqs[i]->poll);
@@ -2051,6 +2088,535 @@ static long vhost_vring_set_num(struct vhost_dev *d,
return 0;
}
+static int vhost_sqcq_kick_wakeup(wait_queue_entry_t *wait, unsigned int mode,
+ int sync, void *key)
+{
+ struct vhost_virtqueue *vq = container_of(wait,
+ struct vhost_virtqueue, kick_wait);
+
+ if (!(key_to_poll(key) & EPOLLIN))
+ return 0;
+
+ WRITE_ONCE(vq->kick_received, true);
+ wake_up(&vq->poll_wait);
+ return 0;
+}
+
+static void vhost_sqcq_kick_ptable_func(struct file *file,
+ wait_queue_head_t *wqh,
+ poll_table *pt)
+{
+ struct vhost_virtqueue *vq = container_of(pt,
+ struct vhost_virtqueue, kick_pt);
+
+ vq->kick_wqh = wqh;
+ add_wait_queue(wqh, &vq->kick_wait);
+}
+
+static void vhost_sqcq_kick_register(struct vhost_virtqueue *vq)
+{
+ init_waitqueue_func_entry(&vq->kick_wait, vhost_sqcq_kick_wakeup);
+ vq->kick_wqh = NULL;
+
+ if (vq->kick) {
+ __poll_t mask;
+
+ init_poll_funcptr(&vq->kick_pt, vhost_sqcq_kick_ptable_func);
+ mask = vfs_poll(vq->kick, &vq->kick_pt);
+ /* eventfd counter is never consumed; verify stale EPOLLIN
+ * against sq->idx before acting; real kicks wake via
+ * the registered wait queue.
+ */
+ if (mask & EPOLLIN) {
+ __virtio64 sq_val;
+
+ if (vq->sq && !get_user(sq_val, &vq->sq->idx)) {
+ smp_rmb();
+ if ((__force u16)sq_val != vq->last_avail_idx)
+ vhost_sqcq_kick_wakeup(
+ &vq->kick_wait, 0, 0,
+ poll_to_key(mask));
+ }
+ }
+ }
+}
+
+static void vhost_sqcq_kick_unregister(struct vhost_virtqueue *vq)
+{
+ if (vq->kick_wqh) {
+ remove_wait_queue(vq->kick_wqh, &vq->kick_wait);
+ vq->kick_wqh = NULL;
+ }
+ WRITE_ONCE(vq->kick_received, false);
+}
+
+static int vhost_sqcq_poll_thread(void *data)
+{
+ struct vhost_virtqueue *vq = data;
+ struct vhost_dev *dev = vq->dev;
+ u16 new_sq_idx;
+ __virtio64 sq_idx_val;
+ __u8 sq_need_wakeup_val = 0;
+ ktime_t ts2;
+ ktime_t last_io_ts = 0;
+ ktime_t stats_window = ktime_get();
+ u64 stats_cq = 0;
+ u64 stats_lat = 0;
+ u64 stats_inflight_sum = 0;
+ u64 stats_inflight_n = 0;
+ u64 stats_multi_cnt = 0;
+ u64 stats_sleep_cnt = 0;
+ u64 stats_wake_kick = 0;
+ u64 stats_wake_complete = 0;
+ u64 stats_wake_timeout = 0;
+ u64 ema_ns_val;
+ u64 spin_ns_val;
+ unsigned long _timeout_jiffies;
+ u64 _timeout_us;
+
+ kthread_use_mm(dev->mm);
+ complete(&vq->poll_started);
+
+ vq->stat_completions = 0;
+ vq->lat_scsi_ns = 0;
+ vq->ema_latency_ns = 0;
+ vq->ema_interval_ns = 0;
+ ts2 = ktime_get();
+
+ while (!READ_ONCE(vq->poll_stop)) {
+
+ stats_inflight_sum += (u16)(vq->last_avail_idx - vq->last_used_idx);
+ stats_inflight_n++;
+ stats_multi_cnt += ((atomic_read(&sqcq_active_io_vqs) > 1) ? 1 : 0);
+
+ // 10's output windows.
+ if (ktime_to_ns(ktime_sub(ktime_get(), stats_window)) >=
+ (s64)10 * NSEC_PER_SEC) {
+ if (stats_cq > 0) {
+ int vi;
+
+ for (vi = 0; vi < dev->nvqs; vi++)
+ if (dev->vqs[vi] == vq)
+ break;
+ pr_info("vhost-sqcq: vq[%d] cq=%llu avg=%lluns ema_lat=%lluns interval=%lluns avg_inflight=%llu multi=%llu%% sleep=%llu wk_kick=%llu%% wk_cmp=%llu%% wk_to=%llu%%\n",
+ vi, stats_cq,
+ div64_u64(stats_lat, stats_cq),
+ vq->ema_latency_ns,
+ vq->ema_interval_ns,
+ stats_inflight_n > 0 ?
+ div64_u64(stats_inflight_sum, stats_inflight_n) : 0,
+ div64_u64(stats_multi_cnt*100, stats_inflight_n),
+ stats_sleep_cnt,
+ stats_sleep_cnt > 0 ?
+ div64_u64(stats_wake_kick*100, stats_sleep_cnt) : 0,
+ stats_sleep_cnt > 0 ?
+ div64_u64(stats_wake_complete*100, stats_sleep_cnt) : 0,
+ stats_sleep_cnt > 0 ?
+ div64_u64(stats_wake_timeout*100, stats_sleep_cnt) : 0);
+ }
+ stats_window = ktime_get();
+ stats_cq = 0;
+ stats_lat = 0;
+ stats_inflight_sum = 0;
+ stats_inflight_n = 0;
+ stats_multi_cnt = 0;
+ stats_sleep_cnt = 0;
+ stats_wake_kick = 0;
+ stats_wake_complete = 0;
+ stats_wake_timeout = 0;
+ }
+
+ /* Backstop cleanup: drop sq_was_active and reset EMAs once a VQ
+ * has been idle (no submission) for a fixed interval.
+ */
+ if (vq->sq_was_active && last_io_ts > 0 &&
+ ktime_to_ns(ktime_sub(ktime_get(), last_io_ts)) >
+ (s64)SQCQ_STALE_TIMEOUT_NS) {
+ vq->sq_was_active = false;
+ atomic_dec(&sqcq_active_io_vqs);
+ if (vq->poll_cpu >= 0)
+ cpumask_clear_cpu(vq->poll_cpu, &sqcq_poll_active_cpu_mask);
+ WRITE_ONCE(vq->ema_interval_ns, 0);
+ WRITE_ONCE(vq->ema_latency_ns, 0);
+ }
+
+ /* complete*/
+ if (atomic_read(&vq->completion_pending) && vq->poll_complete) {
+ smp_rmb();
+ atomic_set(&vq->completion_pending, 0);
+ vq->poll_complete(vq);
+ if (vq->stat_completions > 0) {
+ u64 avg = div64_u64(vq->lat_scsi_ns,
+ vq->stat_completions);
+
+ if (vq->ema_latency_ns == 0)
+ vq->ema_latency_ns = avg;
+ else
+ vq->ema_latency_ns =
+ vq->ema_latency_ns -
+ vq->ema_latency_ns / 8 +
+ avg / 8;
+ stats_cq += vq->stat_completions;
+ stats_lat += vq->lat_scsi_ns;
+ vq->lat_scsi_ns = 0;
+ vq->stat_completions = 0;
+ }
+ }
+
+ if (get_user(sq_idx_val, &vq->sq->idx)) {
+ int fault_retries = 3;
+
+ while (fault_retries-- > 0) {
+ if (!get_user(sq_idx_val, &vq->sq->idx))
+ break;
+ schedule_timeout_idle(1);
+ }
+ if (fault_retries < 0)
+ schedule_timeout_idle(HZ / 100);
+ continue;
+ }
+ /* Acquire: see all prior Guest writes (descriptors). */
+ smp_rmb();
+ new_sq_idx = (__force u16)sq_idx_val;
+
+ if (new_sq_idx != vq->last_avail_idx) {
+ vq->handle_kick(&vq->poll.work);
+ ts2 = ktime_get();
+ if (last_io_ts > 0) {
+ u64 _interval = ktime_to_ns(ktime_sub(ts2, last_io_ts));
+ if (vq->ema_interval_ns == 0)
+ vq->ema_interval_ns = _interval;
+ else
+ vq->ema_interval_ns =
+ vq->ema_interval_ns -
+ vq->ema_interval_ns / 8 +
+ _interval / 8;
+ }
+ last_io_ts = ts2;
+ if (!vq->sq_was_active) {
+ vq->sq_was_active = true;
+ atomic_inc(&sqcq_active_io_vqs);
+ if (vq->poll_cpu >= 0)
+ cpumask_set_cpu(vq->poll_cpu, &sqcq_poll_active_cpu_mask);
+ }
+ sq_need_wakeup_val = 0;
+ smp_mb();
+ put_user(sq_need_wakeup_val, &vq->sq->sq_need_wakeup);
+ }
+
+
+ if (vq->last_avail_idx == vq->last_used_idx) {
+ /* Spin for a fixed window to catch next submission via
+ * sq->idx polling; need_resched+cond_resched prevents
+ * starving other tasks on this CPU.
+ */
+ spin_ns_val = SQCQ_ACTIVE_TIMEOUT_NS;
+ if (last_io_ts > 0 && ktime_to_ns(ktime_sub(
+ ktime_get(), last_io_ts)) < spin_ns_val) {
+ if (need_resched())
+ cond_resched();
+ cpu_relax();
+ continue;
+ }
+ /* Clear sq_was_active to keep active_io_vqs and
+ * cpu mask fresh. Has-inflight path keeps the
+ * flag for TCM completion steering.
+ */
+ if (vq->sq_was_active) {
+ vq->sq_was_active = false;
+ atomic_dec(&sqcq_active_io_vqs);
+ if (vq->poll_cpu >= 0)
+ cpumask_clear_cpu(vq->poll_cpu, &sqcq_poll_active_cpu_mask);
+ }
+ } else {
+ /* Spin within SQCQ_SPIN_EMA_MULT * ema_latency (with
+ * cooperative yield), then sleep. cond_resched handles
+ * multi-VQ fairness without a separate yield heuristic.
+ */
+ ema_ns_val = READ_ONCE(vq->ema_latency_ns);
+ spin_ns_val = ema_ns_val > 0 ?
+ (u64)SQCQ_SPIN_EMA_MULT * ema_ns_val :
+ 2 * NSEC_PER_USEC; /* fallback when EMA unknown */
+
+ if (ktime_to_ns(ktime_sub(
+ ktime_get(), last_io_ts)) < spin_ns_val) {
+ if (need_resched())
+ cond_resched();
+ cpu_relax();
+ continue;
+ }
+ }
+
+ ema_ns_val = READ_ONCE(vq->ema_latency_ns);
+
+ if (ema_ns_val == 0) {
+ _timeout_jiffies = usecs_to_jiffies(
+ SQCQ_IDLE_TIMEOUT_US_MAX);
+ } else {
+ _timeout_us = div64_u64(
+ ema_ns_val,
+ (u64)NSEC_PER_USEC / SQCQ_SLEEP_EMA_MULT);
+
+ _timeout_us = clamp(_timeout_us,
+ 10ULL,
+ (u64)SQCQ_IDLE_TIMEOUT_US_MAX);
+ _timeout_jiffies = usecs_to_jiffies(
+ _timeout_us);
+ if (_timeout_jiffies == 0)
+ _timeout_jiffies = 1;
+ }
+
+ /* Set NEED_WAKEUP, then double-check sq->idx before sleeping. */
+ smp_wmb();
+ sq_need_wakeup_val = 1;
+ put_user(sq_need_wakeup_val,
+ &vq->sq->sq_need_wakeup);
+ smp_mb();
+
+ if (get_user(sq_idx_val, &vq->sq->idx)) {
+ int fault_retries = 3;
+
+ while (fault_retries-- > 0) {
+ if (!get_user(sq_idx_val, &vq->sq->idx))
+ break;
+ schedule_timeout_idle(1);
+ }
+ if (fault_retries < 0)
+ schedule_timeout_idle(HZ / 100);
+ continue;
+ }
+ smp_rmb();
+ new_sq_idx = (__force u16)sq_idx_val;
+
+ if (new_sq_idx != vq->last_avail_idx) {
+ sq_need_wakeup_val = 0;
+ smp_mb();
+ put_user(sq_need_wakeup_val,
+ &vq->sq->sq_need_wakeup);
+ continue;
+ }
+
+ ++stats_sleep_cnt;
+ vhost_sqcq_kick_register(vq);
+ wait_event_interruptible_timeout(
+ vq->poll_wait,
+ READ_ONCE(vq->poll_stop) ||
+ READ_ONCE(vq->kick_received) ||
+ atomic_read(&vq->completion_pending),
+ _timeout_jiffies);
+ /* Wake reason stats: snapshot before unregister clears kick_received */
+ {
+ bool _kicked = READ_ONCE(vq->kick_received);
+ bool _completed = atomic_read(&vq->completion_pending);
+
+ if (_kicked)
+ stats_wake_kick++;
+ if (_completed)
+ stats_wake_complete++;
+ if (!_kicked && !_completed)
+ stats_wake_timeout++;
+ }
+ vhost_sqcq_kick_unregister(vq);
+
+ sq_need_wakeup_val = 0;
+ put_user(sq_need_wakeup_val,
+ &vq->sq->sq_need_wakeup);
+ }
+
+ complete(&vq->poll_stopped);
+ kthread_unuse_mm(dev->mm);
+ return 0;
+}
+
+int vhost_sqcq_poll_start(struct vhost_virtqueue *vq)
+{
+ if (!vq->sq || !vq->cq || !vq->handle_kick)
+ return -EINVAL;
+
+ vhost_poll_stop(&vq->poll);
+
+ WRITE_ONCE(vq->poll_stop, false);
+ reinit_completion(&vq->poll_started);
+ reinit_completion(&vq->poll_stopped);
+
+ int vi;
+
+ for (vi = 0; vi < vq->dev->nvqs; vi++)
+ if (vq->dev->vqs[vi] == vq)
+ break;
+ vq->poll_task = kthread_run(vhost_sqcq_poll_thread, vq,
+ "vhost-sqcq-%d-%d",
+ task_pid_nr(current),
+ vi);
+ if (IS_ERR(vq->poll_task)) {
+ int ret = PTR_ERR(vq->poll_task);
+
+ vq->poll_task = NULL;
+ if (vq->kick)
+ vhost_poll_start(&vq->poll, vq->kick);
+ return ret;
+ }
+
+ wait_for_completion(&vq->poll_started);
+
+ /* Enable sqcq mode only after poll_task is valid and the
+ * thread has signaled readiness. Before this point,
+ * vhost_signal() routes completions via eventfd (old path).
+ */
+ smp_mb();
+ WRITE_ONCE(vq->sqcq_poll, true);
+
+ if (vq->poll_task) {
+ int cpu;
+ int start = atomic_inc_return(&sqcq_poll_cpu_rr);
+ int target_cpu = -1;
+ int online = num_online_cpus();
+
+ if (online > 0)
+ start = start % online;
+ else
+ start = 0;
+
+ for_each_online_cpu(cpu) {
+ if (start-- <= 0) {
+ target_cpu = cpu;
+ break;
+ }
+ }
+ if (target_cpu < 0) {
+ for_each_online_cpu(cpu) {
+ target_cpu = cpu;
+ break;
+ }
+ }
+
+ if (target_cpu >= 0 && target_cpu < nr_cpu_ids &&
+ cpu_online(target_cpu)) {
+ int vi;
+
+ set_cpus_allowed_ptr(vq->poll_task, cpumask_of(target_cpu));
+ for (vi = 0; vi < vq->dev->nvqs; vi++)
+ if (vq->dev->vqs[vi] == vq)
+ break;
+ pr_info("vhost-sqcq: vq[%d] poll thread bound to cpu%d\n",
+ vi, target_cpu);
+ vq->poll_cpu = target_cpu;
+ }
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vhost_sqcq_poll_start);
+
+void vhost_sqcq_poll_stop(struct vhost_virtqueue *vq)
+{
+ if (!vq->poll_task)
+ return;
+
+ WRITE_ONCE(vq->poll_stop, true);
+ smp_mb();
+ wake_up_process(vq->poll_task);
+ wake_up(&vq->poll_wait);
+ wait_for_completion(&vq->poll_stopped);
+ vq->poll_task = NULL;
+ WRITE_ONCE(vq->sqcq_poll, false);
+ if (vq->poll_cpu >= 0) {
+ cpumask_clear_cpu(vq->poll_cpu, &sqcq_poll_active_cpu_mask);
+ vq->poll_cpu = -1;
+ }
+ smp_mb();
+
+ if (vq->kick)
+ vhost_poll_start(&vq->poll, vq->kick);
+ vq->sq = NULL;
+ vq->cq = NULL;
+}
+EXPORT_SYMBOL_GPL(vhost_sqcq_poll_stop);
+
+/* Flush residual completions after stop. Must be called outside vq->mutex. */
+void vhost_sqcq_poll_flush(struct vhost_virtqueue *vq)
+{
+ if (WARN_ON_ONCE(!vq->poll_complete))
+ return;
+
+ atomic_set(&vq->completion_pending, 0);
+ vq->poll_complete(vq);
+}
+EXPORT_SYMBOL_GPL(vhost_sqcq_poll_flush);
+
+/* Pick a CPU for TCM completion: prefer a free CPU (no poll thread),
+ * searching from hint_cpu+1 with wrap. Fallback to next-online
+ * if every CPU has a poll thread.
+ */
+int vhost_sqcq_pick_completion_cpu(int hint_cpu)
+{
+ int first, cpu, target = -1;
+
+ first = cpumask_next(hint_cpu, cpu_online_mask);
+ if (first >= nr_cpu_ids)
+ first = cpumask_first(cpu_online_mask);
+
+ cpu = first;
+ do {
+ if (!cpumask_test_cpu(cpu, &sqcq_poll_active_cpu_mask)) {
+ target = cpu;
+ break;
+ }
+ cpu = cpumask_next(cpu, cpu_online_mask);
+ if (cpu >= nr_cpu_ids)
+ cpu = cpumask_first(cpu_online_mask);
+ } while (cpu != first);
+
+ if (target < 0)
+ target = first;
+
+ return target;
+}
+EXPORT_SYMBOL_GPL(vhost_sqcq_pick_completion_cpu);
+
+static long vhost_vring_set_sqcq_addr(struct vhost_dev *d,
+ struct vhost_virtqueue *vq,
+ void __user *argp)
+{
+ struct vhost_vring_sqcq_addr a;
+
+ if (copy_from_user(&a, argp, sizeof(a)))
+ return -EFAULT;
+
+ if ((u64)(unsigned long)a.sq_user_addr != a.sq_user_addr ||
+ (u64)(unsigned long)a.cq_user_addr != a.cq_user_addr)
+ return -EFAULT;
+
+ if (a.sq_user_addr & 7 || a.cq_user_addr & 7)
+ return -EINVAL;
+
+ if (vq->private_data) {
+ if (!access_ok((void __user *)(unsigned long)a.sq_user_addr,
+ sizeof(struct vring_sq)) ||
+ !access_ok((void __user *)(unsigned long)a.cq_user_addr,
+ sizeof(struct vring_cq)))
+ return -EINVAL;
+ }
+
+ vq->sq = (struct vring_sq __user *)(unsigned long)a.sq_user_addr;
+ vq->cq = (struct vring_cq __user *)(unsigned long)a.cq_user_addr;
+
+ /* SET_ENDPOINT may run before SET_FEATURES/SQCQ_ADDR, so poll-start
+ * can fail there; this is the canonical trigger. Skip VQs without
+ * poll_complete (ctrl/event VQs) — they stay interrupt-driven.
+ */
+ if (vq->private_data && vq->sq && vq->cq &&
+ !READ_ONCE(vq->sqcq_poll) &&
+ vq->poll_complete &&
+ vhost_has_feature(vq, VIRTIO_F_SQCQ_POLL)) {
+ long ret = vhost_sqcq_poll_start(vq);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static long vhost_vring_set_addr(struct vhost_dev *d,
struct vhost_virtqueue *vq,
void __user *argp)
@@ -2244,6 +2810,9 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg
if (copy_to_user(argp, &s, sizeof(s)))
r = -EFAULT;
break;
+ case VHOST_SET_VRING_SQCQ_ADDR:
+ r = vhost_vring_set_sqcq_addr(d, vq, argp);
+ break;
default:
r = -ENOIOCTLCMD;
}
@@ -2256,7 +2825,7 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg
if (filep)
fput(filep);
- if (pollstart && vq->handle_kick)
+ if (pollstart && vq->handle_kick && !READ_ONCE(vq->sqcq_poll))
r = vhost_poll_start(&vq->poll, vq->kick);
mutex_unlock(&vq->mutex);
@@ -2938,8 +3507,12 @@ int vhost_get_vq_desc_n(struct vhost_virtqueue *vq,
*ndesc = c;
/* Assume notifications from guest are disabled at this point,
- * if they aren't we would need to update avail_event index. */
- BUG_ON(!(vq->used_flags & VRING_USED_F_NO_NOTIFY));
+ * if they aren't we would need to update avail_event index.
+ * In SQCQ poll mode, vhost_disable_notify is a no-op so
+ * VRING_USED_F_NO_NOTIFY is never set; skip the check.
+ */
+ WARN_ON_ONCE(!vq->sqcq_poll &&
+ !(vq->used_flags & VRING_USED_F_NO_NOTIFY));
return head;
}
EXPORT_SYMBOL_GPL(vhost_get_vq_desc_n);
@@ -3167,6 +3740,16 @@ static bool vhost_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
/* This actually signals the guest, using eventfd. */
void vhost_signal(struct vhost_dev *dev, struct vhost_virtqueue *vq)
{
+ if (READ_ONCE(vq->sqcq_poll) && vq->cq) {
+ atomic_set(&vq->completion_pending, 1);
+ if (!READ_ONCE(vq->poll_stop)) {
+ smp_mb();
+ if (!task_is_running(vq->poll_task))
+ wake_up(&vq->poll_wait);
+ }
+ return;
+ }
+
/* Signal the Guest tell them we used something up. */
if (vq->call_ctx.ctx && vhost_notify(dev, vq))
eventfd_signal(vq->call_ctx.ctx);
@@ -3213,6 +3796,9 @@ EXPORT_SYMBOL_GPL(vhost_vq_avail_empty);
/* OK, now we need to know about added descriptors. */
bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
{
+ if (READ_ONCE(vq->sqcq_poll))
+ return false;
+
int r;
if (!(vq->used_flags & VRING_USED_F_NO_NOTIFY))
@@ -3249,6 +3835,9 @@ EXPORT_SYMBOL_GPL(vhost_enable_notify);
/* We don't need to be notified again. */
void vhost_disable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
{
+ if (READ_ONCE(vq->sqcq_poll))
+ return;
+
int r;
if (vq->used_flags & VRING_USED_F_NO_NOTIFY)
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 4fe99765c5c73..b9c15961f5a18 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -111,6 +111,7 @@ struct vhost_virtqueue {
/* The routine to call when the Guest pings us, or timeout. */
vhost_work_fn_t handle_kick;
+ void (*poll_complete)(struct vhost_virtqueue *vq);
/* Last available index we saw.
* Values are limited to 0x7fff, and the high bit is used as
@@ -164,6 +165,27 @@ struct vhost_virtqueue {
bool user_be;
#endif
u32 busyloop_timeout;
+
+ /* SQ/CQ polling mode */
+ bool sqcq_poll;
+ struct vring_sq __user *sq;
+ struct vring_cq __user *cq;
+ struct task_struct *poll_task;
+ bool poll_stop;
+ wait_queue_head_t poll_wait;
+ struct completion poll_started;
+ struct completion poll_stopped;
+ bool kick_received; /* Set when kick eventfd wakes us */
+ atomic_t completion_pending; /* Set by vhost_signal, poll thread atomically clears */
+ wait_queue_head_t *kick_wqh; /* kick eventfd wait queue head */
+ wait_queue_entry_t kick_wait; /* Wait entry on kick eventfd wq */
+ poll_table kick_pt; /* poll_table for kick eventfd */
+ u64 stat_completions;
+ u64 lat_scsi_ns;
+ u64 ema_interval_ns;
+ u64 ema_latency_ns;
+ bool sq_was_active;
+ int poll_cpu; /* CPU this VQ's poll thread is pinned to, -1 = unpinned */
};
struct vhost_msg_node {
@@ -256,6 +278,10 @@ void vhost_add_used_and_signal_n(struct vhost_dev *, struct vhost_virtqueue *,
void vhost_signal(struct vhost_dev *, struct vhost_virtqueue *);
void vhost_disable_notify(struct vhost_dev *, struct vhost_virtqueue *);
bool vhost_vq_avail_empty(struct vhost_dev *, struct vhost_virtqueue *);
+int vhost_sqcq_poll_start(struct vhost_virtqueue *vq);
+void vhost_sqcq_poll_stop(struct vhost_virtqueue *vq);
+void vhost_sqcq_poll_flush(struct vhost_virtqueue *vq);
+int vhost_sqcq_pick_completion_cpu(int hint_cpu);
bool vhost_enable_notify(struct vhost_dev *, struct vhost_virtqueue *);
int vhost_log_write(struct vhost_virtqueue *vq, struct vhost_log *log,
@@ -294,7 +320,8 @@ void vhost_iotlb_map_free(struct vhost_iotlb *iotlb,
VIRTIO_RING_F_EVENT_IDX, \
VHOST_F_LOG_ALL, \
VIRTIO_F_ANY_LAYOUT, \
- VIRTIO_F_VERSION_1
+ VIRTIO_F_VERSION_1, \
+ VIRTIO_F_SQCQ_POLL
static inline u64 vhost_features_u64(const int *features, int size, int idx)
{
diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h
index c57674a6aa0db..dacb0e1fdc4bc 100644
--- a/include/uapi/linux/vhost.h
+++ b/include/uapi/linux/vhost.h
@@ -123,6 +123,10 @@
#define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64)
#define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64)
+/* Set SQ/CQ doorbell addresses for polling mode */
+#define VHOST_SET_VRING_SQCQ_ADDR _IOW(VHOST_VIRTIO, 0x27, \
+ struct vhost_vring_sqcq_addr)
+
/* VHOST_NET specific defines */
/* Attach virtio net ring to a raw socket, or tap device.
diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h
index 1c39cc5f5a31b..6514c1b5be20b 100644
--- a/include/uapi/linux/vhost_types.h
+++ b/include/uapi/linux/vhost_types.h
@@ -47,6 +47,12 @@ struct vhost_vring_addr {
__u64 log_guest_addr;
};
+struct vhost_vring_sqcq_addr {
+ unsigned int index;
+ __u64 sq_user_addr; /* SQ doorbell HVA */
+ __u64 cq_user_addr; /* CQ doorbell HVA */
+};
+
struct vhost_worker_state {
/*
* For VHOST_NEW_WORKER the kernel will return the new vhost_worker id.
--
2.34.1
^ permalink raw reply related
* Re: [PATCH 01/15] drm/exynos: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:15 UTC (permalink / raw)
To: Thomas Zimmermann
Cc: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Michal Simek, Thierry Reding, Mikko Perttunen,
Jonathan Hunter, Stefan Agner, Alison Wang, Anitha Chrisanthus,
David Airlie, Gerd Hoffmann, Dmitry Osipenko, Gurchetan Singh,
Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li, Sascha Hauer,
Pengutronix Kernel Team, Fabio Estevam, Chun-Kuang Hu,
Philipp Zabel, Matthias Brugger, AngeloGioacchino Del Regno,
Geert Uytterhoeven, Xinliang Liu, Sumit Semwal, Yongqin Liu,
John Stultz, Liviu Dudau, Neil Armstrong, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Jonathan Corbet, Shuah Khan,
dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc
In-Reply-To: <4af254a4-beb7-4f77-830f-42d5bacb10a4@suse.de>
Hey,
> Again, you rather want drm_err() with drm_dev here.
>
> I've briefly looked over the series and many patches seem affected.
> Please prefer drm_ logging functions and DRM devices over the plain
> device equivalents.
Thanks. I will submit a v2 with consistent usage of drm_err().
Also, is there any ongoing task or would it be useful to submit patches
changing the usage of DRM_DEV_* functions with drm_* equivalents?
Or from dev_* to drm_* when possible ?
BR,
Diogo
^ permalink raw reply
* Re: [PATCH 01/15] drm/exynos: remove dependency on DRM simple helpers
From: Thomas Zimmermann @ 2026-07-20 15:27 UTC (permalink / raw)
To: Diogo Silva
Cc: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Michal Simek, Thierry Reding, Mikko Perttunen,
Jonathan Hunter, Stefan Agner, Alison Wang, Anitha Chrisanthus,
David Airlie, Gerd Hoffmann, Dmitry Osipenko, Gurchetan Singh,
Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li, Sascha Hauer,
Pengutronix Kernel Team, Fabio Estevam, Chun-Kuang Hu,
Philipp Zabel, Matthias Brugger, AngeloGioacchino Del Regno,
Geert Uytterhoeven, Xinliang Liu, Sumit Semwal, Yongqin Liu,
John Stultz, Liviu Dudau, Neil Armstrong, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Jonathan Corbet, Shuah Khan,
dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc
In-Reply-To: <CAJpoHp73S2_cavgV3aVwC5PS8H1GRqTdDV-Xu8n6nXavBnV2=g@mail.gmail.com>
Hi
Am 20.07.26 um 17:15 schrieb Diogo Silva:
> Hey,
>
>> Again, you rather want drm_err() with drm_dev here.
>>
>> I've briefly looked over the series and many patches seem affected.
>> Please prefer drm_ logging functions and DRM devices over the plain
>> device equivalents.
> Thanks. I will submit a v2 with consistent usage of drm_err().
Thanks
>
> Also, is there any ongoing task or would it be useful to submit patches
> changing the usage of DRM_DEV_* functions with drm_* equivalents?
> Or from dev_* to drm_* when possible ?
Possibly. DRM_DEV_ logging is supposed to be replaced with drm_ logging
helpers. That would be a simple follow-up series. Whether to prefer
drm_ over dev_ logging depends on the context and the driver maintainer.
Best regards
Thomas
>
> BR,
> Diogo
--
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, (HRB 36809, AG Nürnberg)
^ permalink raw reply
* Re: [PATCH v4 2/8] media: virtio: Add virtio-media driver structs and function declarations
From: Brian Daniels @ 2026-07-20 15:30 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Mauro Carvalho Chehab, acourbot, adelva, aesteve, changyeon,
daniel.almeida, eperezma, gnurou, gurchetansingh, hverkuil,
jasowang, linux-kernel, linux-media, nicolas.dufresne,
virtualization, xuanzhuo
In-Reply-To: <20260718132340-mutt-send-email-mst@kernel.org>
On Sat, Jul 18, 2026 at 1:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Jul 17, 2026 at 03:46:36PM -0400, Brian Daniels wrote:
> > On Mon, Jun 22, 2026 at 5:08 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Mon, Jun 22, 2026 at 04:43:37PM -0400, Brian Daniels wrote:
> > > > From: Alexandre Courbot <gnurou@gmail.com>
> > > >
> > > > Add the structs and function declarations for the new virtio-media drvier.
> > > >
> > > > Signed-off-by: Alexandre Courbot <gnurou@gmail.com>
> > > > Co-developed-by: Brian Daniels <briandaniels@google.com>
> > > > Signed-off-by: Brian Daniels <briandaniels@google.com>
> > > > ---
> > > > drivers/media/virtio/virtio_media.h | 95 +++++++++++++++++++++++++++++
> > > > 1 file changed, 95 insertions(+)
> > > > create mode 100644 drivers/media/virtio/virtio_media.h
> > > >
> > > > diff --git a/drivers/media/virtio/virtio_media.h b/drivers/media/virtio/virtio_media.h
> > > > new file mode 100644
> > > > index 000000000..52809d4e9
> > > > --- /dev/null
> > > > +++ b/drivers/media/virtio/virtio_media.h
> > > > @@ -0,0 +1,95 @@
> > > > +/* SPDX-License-Identifier: BSD-3-Clause OR GPL-2.0+ */
> > > > +
> > > > +/*
> > > > + * Virtio-media structures & functions declarations.
> > > > + *
> > > > + * Copyright (c) 2024-2025 Google LLC.
> > > > + */
> > > > +
> > > > +#ifndef __VIRTIO_MEDIA_H
> > > > +#define __VIRTIO_MEDIA_H
> > > > +
> > > > +#include <linux/virtio_config.h>
> > > > +#include <media/v4l2-device.h>
> > > > +
> > > > +#include "protocol.h"
> > > > +
> > > > +#define DESC_CHAIN_MAX_LEN SG_MAX_SINGLE_ALLOC
> > > > +
> > > > +#define VIRTIO_MEDIA_DEFAULT_DRIVER_NAME "virtio-media"
> > > > +
> > > > +extern char *virtio_media_driver_name;
> > > > +extern bool virtio_media_allow_userptr;
> > > > +
> > > > +/**
> > > > + * struct virtio_media - Virtio-media device.
> > > > + * @v4l2_dev: v4l2_device for the media device.
> > > > + * @video_dev: video_device for the media device.
> > > > + * @virtio_dev: virtio device for the media device.
> > > > + * @commandq: virtio command queue.
> > > > + * @eventq: virtio event queue.
> > > > + * @eventq_work: work to run when events are received on @eventq.
> > > > + * @mmap_region: region into which MMAP buffers are mapped by the host.
> > > > + * @event_buffer: buffer for event descriptors.
> > > > + * @sessions: list of active sessions on the device.
> > > > + * @sessions_lock: protects @sessions and ``virtio_media_session::list``.
> > > > + * @events_lock: prevents concurrent processing of events.
> > > > + * @cmd: union of the device commands "open" and "munmap". The other
> > > > + * commands are handled by @struct virtio_media_session
> > > > + * @resp: union of responses.to device commands "open" and "munmap". The
> > > > + * other responses are handled by @struct virtio_media_session
> > > > + * @vlock: serializes access to the command queue.
> > > > + * @wq: waitqueue for host responses on the command queue.
> > > > + */
> > > > +struct virtio_media {
> > > > + struct v4l2_device v4l2_dev;
> > > > + struct video_device video_dev;
> > > > +
> > > > + struct virtio_device *virtio_dev;
> > > > + struct virtqueue *commandq;
> > > > + struct virtqueue *eventq;
> > > > + struct work_struct eventq_work;
> > > > +
> > > > + struct virtio_shm_region mmap_region;
> > > > +
> > > > + void *event_buffer;
> > > > +
> > > > + struct list_head sessions;
> > > > + struct mutex sessions_lock; /* protects sessions list */
> > > > +
> > > > + struct mutex events_lock; /* prevents concurrent event processing */
> > > > +
> > > > + union {
> > > > + struct virtio_media_cmd_open open;
> > > > + struct virtio_media_cmd_munmap munmap;
> > > > + } cmd;
> > > > +
> > > > + union {
> > > > + struct virtio_media_resp_open open;
> > > > + struct virtio_media_resp_munmap munmap;
> > > > + } resp;
> > >
> > >
> > > You need DMA alignment padding for these things.
> >
> > Each member of the above unions is already padded to 64-bit alignment,
> > is that sufficient?
> >
> > If not, could you tell me what else is necessary?
>
> Read up "What memory is DMA'able?" and following chapters in
> Documentation/core-api/dma-api-howto.rst
Great thanks!
> > > Which one can only see when I reads the actual driver 8 patches down.
> > > Which is why it's not a sensible way to split patches.
> > >
> > > A sensible way is to have a driver then add functionality
> > > in logical pieces gradually.
> > >
> >
> > Appreciate the feedback. There was a previous comment requesting to
> > split the patch into c/h file pairs, but I can try reformatting it as
> > you suggest for v5.
> >
> > > > +
> > > > + struct mutex vlock; /* serializes command queue access */
> > > > + wait_queue_head_t wq;
> > > > +};
> > > > +
> > > > +static inline struct virtio_media *
> > > > +to_virtio_media(struct video_device *video_dev)
> > > > +{
> > > > + return container_of(video_dev, struct virtio_media, video_dev);
> > > > +}
> > > > +
> > > > +/* virtio_media_driver.c */
> > > > +
> > > > +int virtio_media_send_command(struct virtio_media *vv, struct scatterlist **sgs,
> > > > + const size_t out_sgs, const size_t in_sgs,
> > > > + size_t minimum_resp_len, size_t *resp_len);
> > > > +void virtio_media_process_events(struct virtio_media *vv);
> > > > +
> > > > +/* virtio_media_ioctls.c */
> > > > +
> > > > +long virtio_media_device_ioctl(struct file *file, unsigned int cmd,
> > > > + unsigned long arg);
> > > > +extern const struct v4l2_ioctl_ops virtio_media_ioctl_ops;
> > > > +
> > > > +#endif // __VIRTIO_MEDIA_H
> > > > --
> > > > 2.55.0.rc0.799.gd6f94ed593-goog
> > >
>
^ permalink raw reply
* [PATCH v2 00/15] drm/drm_simple: remove drm_simple_encoder_init
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
This series open-codes all remaining drm_simple_encoder_init() users by
calling drm_encoder_init() directly and providing driver-local
drm_encoder_funcs where needed. After the driver conversions, the helper
is removed and the completed DRM todo item is dropped.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
Changes in v2:
- Change logging to consistently use drm_err()
- Link to v1: https://patch.msgid.link/20260719-drm_simple_encoder_init-v1-0-a78c509e3062@gmail.com
To: Jingoo Han <jingoohan1@gmail.com>
To: Inki Dae <inki.dae@samsung.com>
To: Seung-Woo Kim <sw0312.kim@samsung.com>
To: Kyungmin Park <kyungmin.park@samsung.com>
To: David Airlie <airlied@gmail.com>
To: Simona Vetter <simona@ffwll.ch>
To: Krzysztof Kozlowski <krzk@kernel.org>
To: Peter Griffin <peter.griffin@linaro.org>
To: Alim Akhtar <alim.akhtar@samsung.com>
To: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
To: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
To: Maxime Ripard <mripard@kernel.org>
To: Thomas Zimmermann <tzimmermann@suse.de>
To: Michal Simek <michal.simek@amd.com>
To: Thierry Reding <thierry.reding@kernel.org>
To: Mikko Perttunen <mperttunen@nvidia.com>
To: Jonathan Hunter <jonathanh@nvidia.com>
To: Stefan Agner <stefan@agner.ch>
To: Alison Wang <alison.wang@nxp.com>
To: Anitha Chrisanthus <anitha.chrisanthus@intel.com>
To: David Airlie <airlied@redhat.com>
To: Gerd Hoffmann <kraxel@redhat.com>
To: Dmitry Osipenko <dmitry.osipenko@collabora.com>
To: Gurchetan Singh <gurchetansingh@chromium.org>
To: Chia-I Wu <olvaffe@gmail.com>
To: Jyri Sarha <jyri.sarha@iki.fi>
To: Liu Ying <victor.liu@nxp.com>
To: Frank Li <Frank.Li@nxp.com>
To: Sascha Hauer <s.hauer@pengutronix.de>
To: Pengutronix Kernel Team <kernel@pengutronix.de>
To: Fabio Estevam <festevam@gmail.com>
To: Chun-Kuang Hu <chunkuang.hu@kernel.org>
To: Philipp Zabel <p.zabel@pengutronix.de>
To: Matthias Brugger <matthias.bgg@gmail.com>
To: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
To: Geert Uytterhoeven <geert+renesas@glider.be>
To: Xinliang Liu <xinliang.liu@linaro.org>
To: Sumit Semwal <sumit.semwal@linaro.org>
To: Yongqin Liu <yongqin.liu@linaro.org>
To: John Stultz <jstultz@google.com>
To: Liviu Dudau <liviu.dudau@arm.com>
To: Neil Armstrong <neil.armstrong@linaro.org>
To: Kevin Hilman <khilman@baylibre.com>
To: Jerome Brunet <jbrunet@baylibre.com>
To: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
To: Jonathan Corbet <corbet@lwn.net>
To: Shuah Khan <skhan@linuxfoundation.org>
Cc: dri-devel@lists.freedesktop.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-samsung-soc@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-tegra@vger.kernel.org
Cc: virtualization@lists.linux.dev
Cc: imx@lists.linux.dev
Cc: linux-mediatek@lists.infradead.org
Cc: linux-renesas-soc@vger.kernel.org
Cc: linux-amlogic@lists.infradead.org
Cc: linux-doc@vger.kernel.org
---
Diogo Silva (15):
drm/exynos: remove dependency on DRM simple helpers
drm/xlnx/zynqmp_dpsub: remove dependency on DRM simple helpers
drm/tegra: remove dependency on DRM simple helpers
drm/fsl-dcu: remove dependency on DRM simple helpers
drm/kmb: remove dependency on DRM simple helpers
drm/virtio: remove dependency on DRM simple helpers
drm/tidss: remove dependency on DRM simple helpers
drm/imx: remove dependency on DRM simple helpers
drm/mediatek: remove dependency on DRM simple helpers
drm/renesas/shmobile: remove dependency on DRM simple helpers
drm/hisilicon/kirin: remove dependency on DRM simple helpers
drm/arm/komeda: remove dependency on DRM simple helpers
drm/meson: remove dependency on DRM simple helpers
drm/drm_simple: remove deprecated drm_simple_encoder_init function
Documentation/gpu: remove completed drm_simple_encoder_init() todo
Documentation/gpu/todo.rst | 15 ---------------
drivers/gpu/drm/arm/display/komeda/komeda_crtc.c | 9 +++++++--
drivers/gpu/drm/drm_simple_kms_helper.c | 13 ++-----------
drivers/gpu/drm/exynos/exynos_dp.c | 13 +++++++++++--
drivers/gpu/drm/exynos/exynos_drm_dpi.c | 13 +++++++++++--
drivers/gpu/drm/exynos/exynos_drm_dsi.c | 11 +++++++++--
drivers/gpu/drm/exynos/exynos_drm_vidi.c | 13 +++++++++++--
drivers/gpu/drm/exynos/exynos_hdmi.c | 14 ++++++++++++--
drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c | 10 +++++++---
drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c | 9 +++++++--
drivers/gpu/drm/imx/dc/dc-kms.c | 8 ++++++--
drivers/gpu/drm/kmb/kmb_dsi.c | 9 +++++++--
drivers/gpu/drm/mediatek/mtk_dsi.c | 10 +++++++---
drivers/gpu/drm/meson/meson_encoder_cvbs.c | 11 ++++++++---
drivers/gpu/drm/meson/meson_encoder_dsi.c | 11 ++++++++---
drivers/gpu/drm/meson/meson_encoder_hdmi.c | 11 ++++++++---
drivers/gpu/drm/renesas/shmobile/shmob_drm_crtc.c | 10 +++++++---
drivers/gpu/drm/tegra/dsi.c | 16 +++++++++++++---
drivers/gpu/drm/tegra/rgb.c | 15 +++++++++++++--
drivers/gpu/drm/tidss/tidss_encoder.c | 10 +++++++---
drivers/gpu/drm/virtio/virtgpu_display.c | 12 ++++++++++--
drivers/gpu/drm/xlnx/zynqmp_kms.c | 14 ++++++++++++--
include/drm/drm_simple_kms_helper.h | 4 ----
23 files changed, 183 insertions(+), 78 deletions(-)
---
base-commit: e1113c37ba4f166e37dec74a729e6dfd8cba1687
change-id: 20260718-drm_simple_encoder_init-d069a4cc7f8b
Best regards,
--
Diogo Silva <diogompaissilva@gmail.com>
^ permalink raw reply
* [PATCH v2 01/15] drm/exynos: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Also check the return value from drm_encoder_init() to avoid silent
failures.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/exynos/exynos_dp.c | 13 +++++++++++--
drivers/gpu/drm/exynos/exynos_drm_dpi.c | 13 +++++++++++--
drivers/gpu/drm/exynos/exynos_drm_dsi.c | 11 +++++++++--
drivers/gpu/drm/exynos/exynos_drm_vidi.c | 13 +++++++++++--
drivers/gpu/drm/exynos/exynos_hdmi.c | 14 ++++++++++++--
5 files changed, 54 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/exynos/exynos_dp.c b/drivers/gpu/drm/exynos/exynos_dp.c
index b80540328150..957382223956 100644
--- a/drivers/gpu/drm/exynos/exynos_dp.c
+++ b/drivers/gpu/drm/exynos/exynos_dp.c
@@ -24,11 +24,11 @@
#include <drm/drm_bridge.h>
#include <drm/drm_bridge_connector.h>
#include <drm/drm_crtc.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <drm/exynos_drm.h>
#include "exynos_drm_crtc.h"
@@ -79,6 +79,10 @@ static void exynos_dp_nop(struct drm_encoder *encoder)
/* do nothing */
}
+static const struct drm_encoder_funcs exynos_dp_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static const struct drm_encoder_helper_funcs exynos_dp_encoder_helper_funcs = {
.mode_set = exynos_dp_mode_set,
.enable = exynos_dp_nop,
@@ -95,7 +99,12 @@ static int exynos_dp_bind(struct device *dev, struct device *master, void *data)
dp->drm_dev = drm_dev;
- drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
+ ret = drm_encoder_init(drm_dev, encoder, &exynos_dp_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret) {
+ drm_err(drm_dev, "Failed to initialize encoder\n");
+ return ret;
+ }
drm_encoder_helper_add(encoder, &exynos_dp_encoder_helper_funcs);
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dpi.c b/drivers/gpu/drm/exynos/exynos_drm_dpi.c
index 0dc36df6ada3..de4a80158746 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dpi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dpi.c
@@ -12,10 +12,10 @@
#include <linux/regulator/consumer.h>
#include <drm/drm_atomic_helper.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_panel.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <video/of_videomode.h>
#include <video/videomode.h>
@@ -140,6 +140,10 @@ static void exynos_dpi_disable(struct drm_encoder *encoder)
}
}
+static const struct drm_encoder_funcs exynos_dpi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static const struct drm_encoder_helper_funcs exynos_dpi_encoder_helper_funcs = {
.mode_set = exynos_dpi_mode_set,
.enable = exynos_dpi_enable,
@@ -194,7 +198,12 @@ int exynos_dpi_bind(struct drm_device *dev, struct drm_encoder *encoder)
{
int ret;
- drm_simple_encoder_init(dev, encoder, DRM_MODE_ENCODER_TMDS);
+ ret = drm_encoder_init(dev, encoder, &exynos_dpi_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret) {
+ drm_err(dev, "failed to create encoder ret = %d\n", ret);
+ return ret;
+ }
drm_encoder_helper_add(encoder, &exynos_dpi_encoder_helper_funcs);
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dsi.c b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
index c4d098ab7863..6b7561ac9bb0 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dsi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
@@ -13,7 +13,7 @@
#include <drm/bridge/samsung-dsim.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
+#include <drm/drm_encoder.h>
#include "exynos_drm_crtc.h"
#include "exynos_drm_drv.h"
@@ -22,6 +22,10 @@ struct exynos_dsi {
struct drm_encoder encoder;
};
+static const struct drm_encoder_funcs exynos_drm_dsi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static irqreturn_t exynos_dsi_te_irq_handler(struct samsung_dsim *dsim)
{
struct exynos_dsi *dsi = dsim->priv;
@@ -79,7 +83,10 @@ static int exynos_dsi_bind(struct device *dev, struct device *master, void *data
struct drm_device *drm_dev = data;
int ret;
- drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
+ ret = drm_encoder_init(drm_dev, encoder, &exynos_drm_dsi_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret)
+ return ret;
ret = exynos_drm_set_possible_crtcs(encoder, EXYNOS_DISPLAY_TYPE_LCD);
if (ret < 0)
diff --git a/drivers/gpu/drm/exynos/exynos_drm_vidi.c b/drivers/gpu/drm/exynos/exynos_drm_vidi.c
index 67bbf9b8bc0e..077571eae9d6 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_vidi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_vidi.c
@@ -13,10 +13,10 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_edid.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_framebuffer.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <drm/drm_vblank.h>
#include <drm/exynos_drm.h>
@@ -403,6 +403,10 @@ static void exynos_vidi_disable(struct drm_encoder *encoder)
{
}
+static const struct drm_encoder_funcs exynos_vidi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static const struct drm_encoder_helper_funcs exynos_vidi_encoder_helper_funcs = {
.mode_set = exynos_vidi_mode_set,
.enable = exynos_vidi_enable,
@@ -445,7 +449,12 @@ static int vidi_bind(struct device *dev, struct device *master, void *data)
return PTR_ERR(ctx->crtc);
}
- drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
+ ret = drm_encoder_init(drm_dev, encoder, &exynos_vidi_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret) {
+ drm_err(drm_dev, "failed to initialize encoder ret = %d\n", ret);
+ return ret;
+ }
drm_encoder_helper_add(encoder, &exynos_vidi_encoder_helper_funcs);
diff --git a/drivers/gpu/drm/exynos/exynos_hdmi.c b/drivers/gpu/drm/exynos/exynos_hdmi.c
index 09b2cabb236f..45aeda94b74d 100644
--- a/drivers/gpu/drm/exynos/exynos_hdmi.c
+++ b/drivers/gpu/drm/exynos/exynos_hdmi.c
@@ -36,9 +36,9 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_edid.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include "exynos_drm_crtc.h"
#include "regs-hdmi.h"
@@ -1575,6 +1575,11 @@ static void hdmi_disable(struct drm_encoder *encoder)
mutex_unlock(&hdata->mutex);
}
+static const struct drm_encoder_funcs exynos_hdmi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+
static const struct drm_encoder_helper_funcs exynos_hdmi_encoder_helper_funcs = {
.mode_fixup = hdmi_mode_fixup,
.enable = hdmi_enable,
@@ -1862,7 +1867,12 @@ static int hdmi_bind(struct device *dev, struct device *master, void *data)
hdata->phy_clk.enable = hdmiphy_clk_enable;
- drm_simple_encoder_init(drm_dev, encoder, DRM_MODE_ENCODER_TMDS);
+ ret = drm_encoder_init(drm_dev, encoder, &exynos_hdmi_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret) {
+ drm_err(drm_dev, "failed to initialize encoder ret = %d\n", ret);
+ return ret;
+ }
drm_encoder_helper_add(encoder, &exynos_hdmi_encoder_helper_funcs);
--
2.54.0
^ permalink raw reply related
* [PATCH v2 02/15] drm/xlnx/zynqmp_dpsub: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Also check the return value from drm_encoder_init() to avoid silent
failures.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/xlnx/zynqmp_kms.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/xlnx/zynqmp_kms.c b/drivers/gpu/drm/xlnx/zynqmp_kms.c
index d5f922450565..ac9197e026af 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_kms.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_kms.c
@@ -29,8 +29,8 @@
#include <drm/drm_managed.h>
#include <drm/drm_mode_config.h>
#include <drm/drm_plane.h>
+#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <drm/drm_vblank.h>
#include <linux/clk.h>
@@ -417,6 +417,10 @@ static const struct drm_driver zynqmp_dpsub_drm_driver = {
.minor = 0,
};
+static const struct drm_encoder_funcs zynqmp_dpsub_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static int zynqmp_dpsub_kms_init(struct zynqmp_dpsub *dpsub)
{
struct drm_encoder *encoder = &dpsub->drm->encoder;
@@ -436,7 +440,13 @@ static int zynqmp_dpsub_kms_init(struct zynqmp_dpsub *dpsub)
/* Create the encoder and attach the bridge. */
encoder->possible_crtcs |= drm_crtc_mask(&dpsub->drm->crtc);
- drm_simple_encoder_init(&dpsub->drm->dev, encoder, DRM_MODE_ENCODER_NONE);
+ ret = drm_encoder_init(&dpsub->drm->dev, encoder,
+ &zynqmp_dpsub_encoder_funcs,
+ DRM_MODE_ENCODER_NONE, NULL);
+ if (ret) {
+ drm_err(&dpsub->drm->dev, "failed to initialize encoder\n");
+ return ret;
+ }
ret = drm_bridge_attach(encoder, dpsub->bridge, NULL,
DRM_BRIDGE_ATTACH_NO_CONNECTOR);
--
2.54.0
^ permalink raw reply related
* [PATCH v2 03/15] drm/tegra: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Also check the return value from drm_encoder_init() to avoid silent
failures.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/tegra/dsi.c | 16 +++++++++++++---
drivers/gpu/drm/tegra/rgb.c | 15 +++++++++++++--
2 files changed, 26 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/tegra/dsi.c b/drivers/gpu/drm/tegra/dsi.c
index e7fdd8c7ac12..3f818c195e9a 100644
--- a/drivers/gpu/drm/tegra/dsi.c
+++ b/drivers/gpu/drm/tegra/dsi.c
@@ -20,11 +20,11 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_debugfs.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_file.h>
#include <drm/drm_mipi_dsi.h>
#include <drm/drm_panel.h>
#include <drm/drm_print.h>
-#include <drm/drm_simple_kms_helper.h>
#include "dc.h"
#include "drm.h"
@@ -1055,6 +1055,10 @@ tegra_dsi_encoder_atomic_check(struct drm_encoder *encoder,
return err;
}
+static const struct drm_encoder_funcs tegra_dsi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static const struct drm_encoder_helper_funcs tegra_dsi_encoder_helper_funcs = {
.disable = tegra_dsi_encoder_disable,
.enable = tegra_dsi_encoder_enable,
@@ -1078,8 +1082,14 @@ static int tegra_dsi_init(struct host1x_client *client)
&tegra_dsi_connector_helper_funcs);
dsi->output.connector.dpms = DRM_MODE_DPMS_OFF;
- drm_simple_encoder_init(drm, &dsi->output.encoder,
- DRM_MODE_ENCODER_DSI);
+ err = drm_encoder_init(drm, &dsi->output.encoder,
+ &tegra_dsi_encoder_funcs,
+ DRM_MODE_ENCODER_DSI, NULL);
+ if (err) {
+ drm_err(drm, "failed to initialize encoder: %d\n", err);
+ return err;
+ }
+
drm_encoder_helper_add(&dsi->output.encoder,
&tegra_dsi_encoder_helper_funcs);
diff --git a/drivers/gpu/drm/tegra/rgb.c b/drivers/gpu/drm/tegra/rgb.c
index e67fbb2362e6..bc1c93c7554c 100644
--- a/drivers/gpu/drm/tegra/rgb.c
+++ b/drivers/gpu/drm/tegra/rgb.c
@@ -9,7 +9,8 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge_connector.h>
-#include <drm/drm_simple_kms_helper.h>
+#include <drm/drm_encoder.h>
+#include <drm/drm_print.h>
#include "drm.h"
#include "dc.h"
@@ -194,6 +195,10 @@ tegra_rgb_encoder_atomic_check(struct drm_encoder *encoder,
return err;
}
+static const struct drm_encoder_funcs tegra_rgb_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static const struct drm_encoder_helper_funcs tegra_rgb_encoder_helper_funcs = {
.disable = tegra_rgb_encoder_disable,
.enable = tegra_rgb_encoder_enable,
@@ -305,7 +310,13 @@ int tegra_dc_rgb_init(struct drm_device *drm, struct tegra_dc *dc)
if (!dc->rgb)
return -ENODEV;
- drm_simple_encoder_init(drm, &output->encoder, DRM_MODE_ENCODER_LVDS);
+ err = drm_encoder_init(drm, &output->encoder, &tegra_rgb_encoder_funcs,
+ DRM_MODE_ENCODER_LVDS, NULL);
+ if (err) {
+ drm_err(drm, "failed to initialize encoder: %d\n", err);
+ return err;
+ }
+
drm_encoder_helper_add(&output->encoder,
&tegra_rgb_encoder_helper_funcs);
--
2.54.0
^ permalink raw reply related
* [PATCH v2 04/15] drm/fsl-dcu: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c
index 84eff7519e32..c2b788bfa8f9 100644
--- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c
+++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c
@@ -11,14 +11,18 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include "fsl_dcu_drm_drv.h"
#include "fsl_tcon.h"
+static const struct drm_encoder_funcs fsl_dcu_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
int fsl_dcu_drm_encoder_create(struct fsl_dcu_drm_device *fsl_dev,
struct drm_crtc *crtc)
{
@@ -31,8 +35,8 @@ int fsl_dcu_drm_encoder_create(struct fsl_dcu_drm_device *fsl_dev,
if (fsl_dev->tcon)
fsl_tcon_bypass_enable(fsl_dev->tcon);
- ret = drm_simple_encoder_init(fsl_dev->drm, encoder,
- DRM_MODE_ENCODER_LVDS);
+ ret = drm_encoder_init(fsl_dev->drm, encoder, &fsl_dcu_encoder_funcs,
+ DRM_MODE_ENCODER_LVDS, NULL);
if (ret < 0)
return ret;
--
2.54.0
^ permalink raw reply related
* [PATCH v2 05/15] drm/kmb: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/kmb/kmb_dsi.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/kmb/kmb_dsi.c b/drivers/gpu/drm/kmb/kmb_dsi.c
index 59d0e856392f..13adba96bc78 100644
--- a/drivers/gpu/drm/kmb/kmb_dsi.c
+++ b/drivers/gpu/drm/kmb/kmb_dsi.c
@@ -14,8 +14,8 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_bridge_connector.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_mipi_dsi.h>
-#include <drm/drm_simple_kms_helper.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
@@ -1427,6 +1427,10 @@ struct kmb_dsi *kmb_dsi_init(struct platform_device *pdev)
return kmb_dsi;
}
+static const struct drm_encoder_funcs kmb_dsi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
int kmb_dsi_encoder_init(struct drm_device *dev, struct kmb_dsi *kmb_dsi)
{
struct drm_encoder *encoder;
@@ -1437,7 +1441,8 @@ int kmb_dsi_encoder_init(struct drm_device *dev, struct kmb_dsi *kmb_dsi)
encoder->possible_crtcs = 1;
encoder->possible_clones = 0;
- ret = drm_simple_encoder_init(dev, encoder, DRM_MODE_ENCODER_DSI);
+ ret = drm_encoder_init(dev, encoder, &kmb_dsi_encoder_funcs,
+ DRM_MODE_ENCODER_DSI, NULL);
if (ret) {
dev_err(kmb_dsi->dev, "Failed to init encoder %d\n", ret);
return ret;
--
2.54.0
^ permalink raw reply related
* [PATCH v2 06/15] drm/virtio: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Also check the return value from drm_encoder_init() to avoid silent
failures.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/virtio/virtgpu_display.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/virtio/virtgpu_display.c b/drivers/gpu/drm/virtio/virtgpu_display.c
index 44ffffec550f..67023d91d40b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_display.c
+++ b/drivers/gpu/drm/virtio/virtgpu_display.c
@@ -28,11 +28,11 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_damage_helper.h>
#include <drm/drm_edid.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_fourcc.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <drm/drm_vblank.h>
#include <drm/drm_vblank_helper.h>
@@ -232,6 +232,10 @@ static enum drm_mode_status virtio_gpu_conn_mode_valid(struct drm_connector *con
return MODE_BAD;
}
+static const struct drm_encoder_funcs virtio_gpu_enc_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static const struct drm_encoder_helper_funcs virtio_gpu_enc_helper_funcs = {
.mode_set = virtio_gpu_enc_mode_set,
.enable = virtio_gpu_enc_enable,
@@ -306,7 +310,11 @@ static int vgdev_output_init(struct virtio_gpu_device *vgdev, int index)
if (vgdev->has_edid)
drm_connector_attach_edid_property(connector);
- drm_simple_encoder_init(dev, encoder, DRM_MODE_ENCODER_VIRTUAL);
+ ret = drm_encoder_init(dev, encoder, &virtio_gpu_enc_funcs,
+ DRM_MODE_ENCODER_VIRTUAL, NULL);
+ if (ret)
+ return ret;
+
drm_encoder_helper_add(encoder, &virtio_gpu_enc_helper_funcs);
encoder->possible_crtcs = 1 << index;
--
2.54.0
^ permalink raw reply related
* [PATCH v2 07/15] drm/tidss: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/tidss/tidss_encoder.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/tidss/tidss_encoder.c b/drivers/gpu/drm/tidss/tidss_encoder.c
index 698f8d964ca0..10dbcc6cdf6a 100644
--- a/drivers/gpu/drm/tidss/tidss_encoder.c
+++ b/drivers/gpu/drm/tidss/tidss_encoder.c
@@ -9,11 +9,11 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_bridge_connector.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_crtc.h>
#include <drm/drm_modeset_helper_vtables.h>
#include <drm/drm_panel.h>
#include <drm/drm_of.h>
-#include <drm/drm_simple_kms_helper.h>
#include "tidss_crtc.h"
#include "tidss_drv.h"
@@ -81,6 +81,10 @@ static const struct drm_bridge_funcs tidss_bridge_funcs = {
.atomic_destroy_state = drm_atomic_helper_bridge_destroy_state,
};
+static const struct drm_encoder_funcs tidss_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
int tidss_encoder_create(struct tidss_device *tidss,
struct drm_bridge *next_bridge,
u32 encoder_type, u32 possible_crtcs)
@@ -95,8 +99,8 @@ int tidss_encoder_create(struct tidss_device *tidss,
if (IS_ERR(t_enc))
return PTR_ERR(t_enc);
- ret = drm_simple_encoder_init(&tidss->ddev, &t_enc->encoder,
- encoder_type);
+ ret = drm_encoder_init(&tidss->ddev, &t_enc->encoder,
+ &tidss_encoder_funcs, encoder_type, NULL);
if (ret)
return ret;
--
2.54.0
^ permalink raw reply related
* [PATCH v2 08/15] drm/imx: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/imx/dc/dc-kms.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/imx/dc/dc-kms.c b/drivers/gpu/drm/imx/dc/dc-kms.c
index 0f8cfaf4c4d1..a9adcfc68b84 100644
--- a/drivers/gpu/drm/imx/dc/dc-kms.c
+++ b/drivers/gpu/drm/imx/dc/dc-kms.c
@@ -17,7 +17,6 @@
#include <drm/drm_mode_config.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include <drm/drm_vblank.h>
#include "dc-de.h"
@@ -30,6 +29,10 @@ static const struct drm_mode_config_funcs dc_drm_mode_config_funcs = {
.atomic_commit = drm_atomic_helper_commit,
};
+static const struct drm_encoder_funcs dc_kms_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static int dc_kms_init_encoder_per_crtc(struct dc_drm_device *dc_drm,
int crtc_index)
{
@@ -55,7 +58,8 @@ static int dc_kms_init_encoder_per_crtc(struct dc_drm_device *dc_drm,
}
encoder = &dc_drm->encoder[crtc_index];
- ret = drm_simple_encoder_init(drm, encoder, DRM_MODE_ENCODER_NONE);
+ ret = drm_encoder_init(drm, encoder, &dc_kms_encoder_funcs,
+ DRM_MODE_ENCODER_NONE, NULL);
if (ret) {
dev_err(dev, "failed to initialize encoder for CRTC%u: %d\n",
crtc->index, ret);
--
2.54.0
^ permalink raw reply related
* [PATCH v2 09/15] drm/mediatek: remove dependency on DRM simple helpers
From: Diogo Silva @ 2026-07-20 15:40 UTC (permalink / raw)
To: Jingoo Han, Inki Dae, Seung-Woo Kim, Kyungmin Park, David Airlie,
Simona Vetter, Krzysztof Kozlowski, Peter Griffin, Alim Akhtar,
Laurent Pinchart, Tomi Valkeinen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Michal Simek, Thierry Reding,
Mikko Perttunen, Jonathan Hunter, Stefan Agner, Alison Wang,
Anitha Chrisanthus, David Airlie, Gerd Hoffmann, Dmitry Osipenko,
Gurchetan Singh, Chia-I Wu, Jyri Sarha, Liu Ying, Frank Li,
Sascha Hauer, Pengutronix Kernel Team, Fabio Estevam,
Chun-Kuang Hu, Philipp Zabel, Matthias Brugger,
AngeloGioacchino Del Regno, Geert Uytterhoeven, Xinliang Liu,
Sumit Semwal, Yongqin Liu, John Stultz, Liviu Dudau,
Neil Armstrong, Kevin Hilman, Jerome Brunet, Martin Blumenstingl,
Jonathan Corbet, Shuah Khan
Cc: dri-devel, linux-arm-kernel, linux-samsung-soc, linux-kernel,
linux-tegra, virtualization, imx, linux-mediatek,
linux-renesas-soc, linux-amlogic, linux-doc, Diogo Silva
In-Reply-To: <20260720-drm_simple_encoder_init-v2-0-5020b630668a@gmail.com>
The simple KMS helpers are deprecated because they only add an
intermediate layer between drivers and atomic modesetting.
Open-code drm_simple_encoder_init() by calling drm_encoder_init()
directly and providing driver-local drm_encoder_funcs.
Signed-off-by: Diogo Silva <diogompaissilva@gmail.com>
---
drivers/gpu/drm/mediatek/mtk_dsi.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/mediatek/mtk_dsi.c b/drivers/gpu/drm/mediatek/mtk_dsi.c
index 3f3f56eed3f9..7cd136bd9605 100644
--- a/drivers/gpu/drm/mediatek/mtk_dsi.c
+++ b/drivers/gpu/drm/mediatek/mtk_dsi.c
@@ -21,12 +21,12 @@
#include <drm/drm_atomic_helper.h>
#include <drm/drm_bridge.h>
#include <drm/drm_bridge_connector.h>
+#include <drm/drm_encoder.h>
#include <drm/drm_mipi_dsi.h>
#include <drm/drm_of.h>
#include <drm/drm_panel.h>
#include <drm/drm_print.h>
#include <drm/drm_probe_helper.h>
-#include <drm/drm_simple_kms_helper.h>
#include "mtk_ddp_comp.h"
#include "mtk_disp_drv.h"
@@ -913,12 +913,16 @@ void mtk_dsi_ddp_stop(struct device *dev)
mtk_dsi_poweroff(dsi);
}
+static const struct drm_encoder_funcs mtk_dsi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
static int mtk_dsi_encoder_init(struct drm_device *drm, struct mtk_dsi *dsi)
{
int ret;
- ret = drm_simple_encoder_init(drm, &dsi->encoder,
- DRM_MODE_ENCODER_DSI);
+ ret = drm_encoder_init(drm, &dsi->encoder, &mtk_dsi_encoder_funcs,
+ DRM_MODE_ENCODER_DSI, NULL);
if (ret) {
drm_err(drm, "Failed to encoder init to drm\n");
return ret;
--
2.54.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox