From: Ming Lei <tom.leiming@gmail.com>
To: Aaron Tomlin <atomlin@atomlin.com>
Cc: Ming Lei <ming.lei@redhat.com>,
axboe@kernel.dk, kbusch@kernel.org, hch@lst.de, sagi@grimberg.me,
mst@redhat.com, aacraid@microsemi.com,
James.Bottomley@hansenpartnership.com,
martin.petersen@oracle.com, liyihang9@h-partners.com,
kashyap.desai@broadcom.com, sumit.saxena@broadcom.com,
shivasharan.srikanteshwara@broadcom.com,
chandrakanth.patil@broadcom.com, sathya.prakash@broadcom.com,
sreekanth.reddy@broadcom.com,
suganath-prabu.subramani@broadcom.com, ranjan.kumar@broadcom.com,
jinpu.wang@cloud.ionos.com, tglx@kernel.org, mingo@redhat.com,
peterz@infradead.org, juri.lelli@redhat.com,
vincent.guittot@linaro.org, akpm@linux-foundation.org,
maz@kernel.org, ruanjinjie@huawei.com, bigeasy@linutronix.de,
yphbchou0911@gmail.com, wagi@kernel.org, frederic@kernel.org,
longman@redhat.com, chenridong@huawei.com, hare@suse.de,
kch@nvidia.com, steve@abita.co, sean@ashe.io, chjohnst@gmail.com,
neelx@suse.com, mproche@gmail.com, linux-block@vger.kernel.org,
linux-kernel@vger.kernel.org, virtualization@lists.linux.dev,
linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org,
megaraidlinux.pdl@broadcom.com, mpi3mr-linuxdrv.pdl@broadcom.com,
MPT-FusionLinux.pdl@broadcom.com
Subject: Re: [PATCH v10 13/13] docs: add io_queue flag to isolcpus
Date: Sat, 11 Apr 2026 20:52:00 +0800 [thread overview]
Message-ID: <adpD8M8cNu3IZzEL@fedora> (raw)
In-Reply-To: <dzpxscrhibmi5okkozf5jfull4dcajgpctldvdyfcjgmpeetk5@tkeyqouyabzy>
On Fri, Apr 10, 2026 at 03:31:22PM -0400, Aaron Tomlin wrote:
> On Fri, Apr 10, 2026 at 10:44:15AM +0800, Ming Lei wrote:
> > For unmanaged interrupts, user can set irq affinity on housekeeping cpus
> > from /proc or kernel command line.
> >
> > Why is unmanaged interrupts involved with this patchset?
>
> Thank you for your continued engagement and for ultimately supporting the
> progression of this series.
>
> To clarify the handling of unmanaged interrupts, while it is entirely true
> that an administrator could attempt to manually configure "irqaffinity=" or
> via procfs after the fact, this series actively address unmanaged interrupts.
>
> > > CPUs, thereby breaking isolation. By applying the constraint via io_queue
> > > at the block layer, we restrict the hardware queue count and map the
> > > isolated CPUs to the housekeeping queues, ensuring isolation is maintained
> > > regardless of whether the driver uses managed interrupts.
> > >
> > > Does the above help?
> >
> > As I mentioned, managed irq already covers it:
> >
> > - typically application submits IO from housekeeping CPUs, which is mapped
> > to one hardware, which effective interrupt affinity excludes isolated
> > CPUs if possible.
> >
> > I'd suggest to share some real problems you found instead of something
> > imaginary.
>
> If we trace how mpi3mr sets up its ISRs, it relies heavily on the core
> grouping logic:
>
> mpi3mr_setup_isr
> {
> unsigned int irq_flags = PCI_IRQ_MSIX
>
> struct irq_affinity desc = { .pre_vectors = 1, .post_vectors = 1, }
>
> pci_alloc_irq_vectors_affinity(mrioc->pdev, min_vec,
> max_vectors, irq_flags, &desc)
> {
> if (flags & PCI_IRQ_MSIX) {
> // affd != NULL
> __pci_enable_msix_range(dev, NULL, min_vecs, max_vecs, affd, flags)
> {
>
> for (;;) {
>
> msix_capability_init(dev, entries, nvec, affd)
> {
> msix_setup_interrupts(dev, entries, nvec, affd)
> {
> // affd
> irq_create_affinity_masks(nvec, affd)
> {
> for (i = 0, usedvecs = 0; i < affd->nr_sets; i++) {
> unsigned int nr_masks, this_vecs = affd->set_size[i]
> struct cpumask *result = group_cpus_evenly(this_vecs,
> &nr_masks)
> if (!result) {
> kfree(masks)
> return NULL
> }
>
> for (int j = 0; j < nr_masks; j++)
> cpumask_copy(&masks[curvec + j].mask, &result[j])
> kfree(result);
>
> curvec += nr_masks
> usedvecs += nr_masks
> }
> }
> }
> }
> }
> }
> }
> }
> }
>
> The critical issue lies at the invocation of group_cpus_evenly(). Without
> this patchset, the core logic lacks the necessary constraints to respect
> CPU isolation. It is entirely possible, and indeed happens in practice, for
> an isolated CPU to be assigned to a CPU mask group.
It is one bug report? No, because it doesn't show any trouble from user
viewpoint.
Sebastian explains/shows how "isolcpus=managed_irq" works perfectly in the
following link:
https://lore.kernel.org/all/20260401110232.ET5RxZfl@linutronix.de/
You have reviewed it...
What matters is that IO won't interrupt isolated CPU.
>
> The newer implementation of irq_create_affinity_masks() introduced by this
> series resolves this. It considers the new CPU mask added to the IRQ
> affinity descriptor. When group_mask_cpus_evenly() is called, this mask is
> evaluated [1], guaranteeing that isolated CPUs are entirely excluded from
> the mask groups.
>
> [1]: https://lore.kernel.org/lkml/20260401222312.772334-8-atomlin@atomlin.com/
Not at all.
isolated CPU is still included in each group's cpu mask, please see patch
9:
https://lore.kernel.org/linux-block/20260401222312.772334-1-atomlin@atomlin.com/T/#m59df0689ef144f5361535ce59c9ed5923d6e21d5
Thanks,
Ming
prev parent reply other threads:[~2026-04-11 12:52 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-01 22:22 [PATCH v10 00/13] blk: honor isolcpus configuration Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 01/13] scsi: aacraid: use block layer helpers to calculate num of queues Aaron Tomlin
2026-04-03 1:43 ` Martin K. Petersen
2026-04-01 22:23 ` [PATCH v10 02/13] lib/group_cpus: remove dead !SMP code Aaron Tomlin
2026-04-03 1:45 ` Martin K. Petersen
2026-04-01 22:23 ` [PATCH v10 03/13] lib/group_cpus: Add group_mask_cpus_evenly() Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 04/13] genirq/affinity: Add cpumask to struct irq_affinity Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 05/13] blk-mq: add blk_mq_{online|possible}_queue_affinity Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 06/13] nvme-pci: use block layer helpers to constrain queue affinity Aaron Tomlin
2026-04-03 1:46 ` Martin K. Petersen
2026-04-01 22:23 ` [PATCH v10 07/13] scsi: Use " Aaron Tomlin
2026-04-03 1:46 ` Martin K. Petersen
2026-04-01 22:23 ` [PATCH v10 08/13] virtio: blk/scsi: use " Aaron Tomlin
2026-04-03 1:47 ` Martin K. Petersen
2026-04-01 22:23 ` [PATCH v10 09/13] isolation: Introduce io_queue isolcpus type Aaron Tomlin
2026-04-03 1:47 ` Martin K. Petersen
2026-04-01 22:23 ` [PATCH v10 10/13] blk-mq: use hk cpus only when isolcpus=io_queue is enabled Aaron Tomlin
2026-04-03 2:06 ` Waiman Long
2026-04-05 23:09 ` Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 11/13] blk-mq: prevent offlining hk CPUs with associated online isolated CPUs Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 12/13] genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs Aaron Tomlin
2026-04-01 22:23 ` [PATCH v10 13/13] docs: add io_queue flag to isolcpus Aaron Tomlin
2026-04-03 2:30 ` Ming Lei
2026-04-06 1:15 ` Aaron Tomlin
2026-04-06 3:29 ` Ming Lei
2026-04-08 15:58 ` Aaron Tomlin
2026-04-09 15:00 ` Ming Lei
2026-04-10 1:45 ` Aaron Tomlin
2026-04-10 2:44 ` Ming Lei
2026-04-10 19:31 ` Aaron Tomlin
2026-04-11 12:52 ` Ming Lei [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=adpD8M8cNu3IZzEL@fedora \
--to=tom.leiming@gmail.com \
--cc=James.Bottomley@hansenpartnership.com \
--cc=MPT-FusionLinux.pdl@broadcom.com \
--cc=aacraid@microsemi.com \
--cc=akpm@linux-foundation.org \
--cc=atomlin@atomlin.com \
--cc=axboe@kernel.dk \
--cc=bigeasy@linutronix.de \
--cc=chandrakanth.patil@broadcom.com \
--cc=chenridong@huawei.com \
--cc=chjohnst@gmail.com \
--cc=frederic@kernel.org \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=jinpu.wang@cloud.ionos.com \
--cc=juri.lelli@redhat.com \
--cc=kashyap.desai@broadcom.com \
--cc=kbusch@kernel.org \
--cc=kch@nvidia.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-scsi@vger.kernel.org \
--cc=liyihang9@h-partners.com \
--cc=longman@redhat.com \
--cc=martin.petersen@oracle.com \
--cc=maz@kernel.org \
--cc=megaraidlinux.pdl@broadcom.com \
--cc=ming.lei@redhat.com \
--cc=mingo@redhat.com \
--cc=mpi3mr-linuxdrv.pdl@broadcom.com \
--cc=mproche@gmail.com \
--cc=mst@redhat.com \
--cc=neelx@suse.com \
--cc=peterz@infradead.org \
--cc=ranjan.kumar@broadcom.com \
--cc=ruanjinjie@huawei.com \
--cc=sagi@grimberg.me \
--cc=sathya.prakash@broadcom.com \
--cc=sean@ashe.io \
--cc=shivasharan.srikanteshwara@broadcom.com \
--cc=sreekanth.reddy@broadcom.com \
--cc=steve@abita.co \
--cc=suganath-prabu.subramani@broadcom.com \
--cc=sumit.saxena@broadcom.com \
--cc=tglx@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=virtualization@lists.linux.dev \
--cc=wagi@kernel.org \
--cc=yphbchou0911@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox