Linux PCI subsystem development
 help / color / mirror / Atom feed
From: Bjorn Helgaas <helgaas@kernel.org>
To: Eugen Hristev <ehristev@kernel.org>
Cc: "Lorenzo Pieralisi" <lpieralisi@kernel.org>,
	"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
	"Manivannan Sadhasivam" <mani@kernel.org>,
	"Rob Herring" <robh@kernel.org>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Ray Jui" <rjui@broadcom.com>,
	"Scott Branden" <sbranden@broadcom.com>,
	"Broadcom internal kernel review list"
	<bcm-kernel-feedback-list@broadcom.com>,
	linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] PCI: iproc-msi: Bound events handled per IRQ
Date: Fri, 16 Oct 2026 13:21:24 -0500	[thread overview]
Message-ID: <20261016182124.GA1884550@bhelgaas> (raw)
In-Reply-To: <20261016-iproc-irq-v1-1-ebaaf027f29e@kernel.org>

On Fri, Oct 16, 2026 at 03:18:53PM +0300, Eugen Hristev wrote:
> Limit each iProc MSI parent IRQ to 16 event-queue entries.
> 
> A continuously fed queue could otherwise keep the chained handler running
> and starve other work. The budget still drains pending events and lets the
> next parent interrupt continue processing, improving fairness and bounding
> interrupt-handler latency.
> 
> Without this starvation avoidance, one CPU can stall indefinitely
> in the loop and not serve other interrupts, timers, etc.
> 
> While testing it with an Ethernet NIC,  when too many MSI interrupts
> come in, CPU might get starved:
> 
> Connecting to host 192.168.30.1, port 5201
> [  5] local 192.168.30.2 port 52586 connected to 192.168.30.1 port 5201
> [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
> [  5]   0.00-1.00   sec   240 MBytes  2.01 Gbits/sec    2    334 KBytes
> [  5]   1.00-2.00   sec  63.2 MBytes   531 Mbits/sec    1   1.41 KBytes
> [  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> [  177.568497] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> [  177.568913] rcu:     0-...!: (1 GPs behind) idle=28ac/1/0x4000000000000002 softirq=28922/28930 fqs=1
> [  177.569457] rcu:     (detected by 1, t=5256 jiffies, g=21629, q=819 ncpus=4)
> [  177.569870] Sending NMI from CPU 1 to CPUs 0:
> 
> With this bounded approach, IRQ handler will complete and continue
> to serve MSI events on the next handler invocation.

Is there something unique about iproc that makes this an issue, or do
we need to worry about this in every MSI interrupt handler?

> Assisted-by: Github Copilot
> Signed-off-by: Eugen Hristev <ehristev@kernel.org>
> ---
>  drivers/pci/controller/pcie-iproc-msi.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pcie-iproc-msi.c b/drivers/pci/controller/pcie-iproc-msi.c
> index 9ba242ab9596..6649807822d2 100644
> --- a/drivers/pci/controller/pcie-iproc-msi.c
> +++ b/drivers/pci/controller/pcie-iproc-msi.c
> @@ -29,6 +29,9 @@
>  /* Number of entries in each event queue */
>  #define EQ_LEN                         64
>  
> +/* Bound parent interrupt handler such that a queue cannot starve CPU. */
> +#define IPROC_MSI_MAX_EVENTS_PER_IRQ   16
> +
>  /* Size of each event queue memory region */
>  #define EQ_MEM_REGION_SIZE             SZ_4K
>  
> @@ -326,6 +329,7 @@ static void iproc_msi_handler(struct irq_desc *desc)
>  	struct iproc_msi_grp *grp;
>  	struct iproc_msi *msi;
>  	u32 eq, head, tail, nr_events;
> +	unsigned int event_budget = IPROC_MSI_MAX_EVENTS_PER_IRQ;
>  	unsigned long hwirq;
>  
>  	chained_irq_enter(chip, desc);
> @@ -360,9 +364,11 @@ static void iproc_msi_handler(struct irq_desc *desc)
>  			break;
>  
>  		/* process all outstanding events */
> -		while (nr_events--) {
> +		while (nr_events && event_budget) {
>  			hwirq = decode_msi_hwirq(msi, eq, head);
>  			generic_handle_domain_irq(msi->inner_domain, hwirq);
> +			nr_events--;
> +			event_budget--;
>  
>  			head++;
>  			head %= EQ_LEN;
> @@ -373,6 +379,8 @@ static void iproc_msi_handler(struct irq_desc *desc)
>  		 * head pointer.
>  		 */
>  		iproc_msi_write_reg(msi, IPROC_MSI_EQ_HEAD, eq, head);
> +		if (!event_budget)
> +			break;
>  
>  		/*
>  		 * Now go read the tail pointer again to see if there are new
> 
> ---
> base-commit: b810335a0699d1810683796b0e3ffe8827ddfe6c
> change-id: 20261016-iproc-irq-9c58cfe39158
> 
> Best regards,
> --  
> Eugen Hristev <ehristev@kernel.org>
> 

  parent reply	other threads:[~2026-10-16 18:21 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-16 12:18 [PATCH] PCI: iproc-msi: Bound events handled per IRQ Eugen Hristev
2026-10-16 12:31 ` sashiko-bot
2026-10-16 18:21 ` Bjorn Helgaas [this message]
2026-10-16 20:10   ` Eugen Hristev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261016182124.GA1884550@bhelgaas \
    --to=helgaas@kernel.org \
    --cc=bcm-kernel-feedback-list@broadcom.com \
    --cc=bhelgaas@google.com \
    --cc=ehristev@kernel.org \
    --cc=kwilczynski@kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lpieralisi@kernel.org \
    --cc=mani@kernel.org \
    --cc=rjui@broadcom.com \
    --cc=robh@kernel.org \
    --cc=sbranden@broadcom.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox