Re: [PATCH] virtio_blk: add use_irq_affinity module parameter

["Liu, Changcheng" <jerrliu@nvidia.com>]

On Wed, Aug 05, 2026 at 01:45:58PM -0400, Stefan Hajnoczi wrote:
> 
> On Sat, Aug 1, 2026 at 2:21 AM Liu, Changcheng <jerrliu@nvidia.com> wrote:
> >
> > When creating many virtio-blk devices, probe starts failing with
> > -ENOSPC (-28) because the system runs out of interrupt vectors:
> >
> >   virtio_blk virtioNNN: probe with driver virtio_blk failed with error -28
> >
> > By default virtio-blk uses managed IRQ affinity, which reserves an
> > interrupt vector on every CPU for each device (about nr_cpus vectors per
> > device). On a host with many CPUs and many devices this exhausts the
> > vectors long before all devices are probed.
> >
> > Add use_irq_affinity (default true, no behaviour change). Set it to 0 to
> > use unmanaged interrupts, so each device only uses a couple of vectors
> > instead of one per CPU, allowing far more devices to probe.
> >
> > Signed-off-by: Liu, Changcheng <jerrliu@nvidia.com>
> 
> There is already a num_request_queues module parameter for cases where
> the user wishes to reduce the number of virtqueues. Did you benchmark
> that and decide the performance of many queues sharing a single irq
> makes it worth adding another module parameter?
> 
> Stefan

num_request_queues does not address this case because each virtio-blk device
already has only one request virtqueue when the failure occurs, so the queue
count cannot be reduced further.

The issue is reproduced after probing approximately 800 virtio-blk devices on
a 64-core bare-metal host. With managed IRQ affinity, the IRQ-vector reservations
are eventually exhausted and probing additional devices fails with -ENOSPC.
Disabling managed IRQ affinity avoids these per-CPU vector reservations and
allows more devices to be probed successfully.

This patch does not cause multiple virtqueues to share a single IRQ. Passing a
NULL affinity descriptor only disables managed IRQ affinity. Regardless of the
number of virtqueues, virtio-pci still first attempts to allocate a separate
interrupt vector for each interrupt-driven virtqueue. Only if per-VQ vector
allocation fails does the existing virtio-pci fallback share a vector among
the virtqueues, and that fallback is independent of this patch.

The new parameter defaults to enabled, so the existing driver behavior remains
unchanged unless the option is explicitly disabled. The option is intended for
environments with a very large number of devices, particularly for functional
device testing.

- Changcheng

> 
> > diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
> > index 32bf3ba07a9d..184f1c1f4485 100644
> > --- a/drivers/block/virtio_blk.c
> > +++ b/drivers/block/virtio_blk.c
> > @@ -41,6 +41,11 @@ static unsigned int poll_queues;
> >  module_param(poll_queues, uint, 0644);
> >  MODULE_PARM_DESC(poll_queues, "The number of dedicated virtqueues for polling I/O");
> >
> > +static bool use_irq_affinity = true;
> > +module_param(use_irq_affinity, bool, 0644);
> > +MODULE_PARM_DESC(use_irq_affinity,
> > +                "Use managed IRQ affinity for virtqueues (default: true)");
> > +
> >  static int major;
> >  static DEFINE_IDA(vd_index_ida);
> >
> > @@ -1016,7 +1021,8 @@ static int init_vq(struct virtio_blk *vblk)
> >         }
> >
> >         /* Discover virtqueues and write information to configuration.  */
> > -       err = virtio_find_vqs(vdev, num_vqs, vqs, vqs_info, &desc);
> > +       err = virtio_find_vqs(vdev, num_vqs, vqs, vqs_info,
> > +                             use_irq_affinity ? &desc : NULL);
> >         if (err)
> >                 goto out;
> >
> > --
> > 2.43.7
> >
> >