Linux kernel and device drivers for NXP i.MX platforms
 help / color / mirror / Atom feed
From: Francesco Dolcini <francesco@dolcini.it>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Peng Fan <peng.fan@oss.nxp.com>
Cc: Francesco Dolcini <francesco.dolcini@toradex.com>,
	Peng Fan <peng.fan@nxp.com>,
	linux-remoteproc@vger.kernel.org, imx@lists.linux.dev,
	linux-arm-kernel@lists.infradead.org,
	linux-rt-devel@lists.linux.dev,
	Bjorn Andersson <andersson@kernel.org>,
	Clark Williams <clrkwllms@kernel.org>,
	Fabio Estevam <festevam@gmail.com>, Frank Li <Frank.Li@nxp.com>,
	Jassi Brar <jassisinghbrar@gmail.com>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Pengutronix Kernel Team <kernel@pengutronix.de>,
	Sascha Hauer <s.hauer@pengutronix.de>,
	Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc
Date: Thu, 19 Nov 2026 12:40:41 +0100	[thread overview]
Message-ID: <20261119114041.GA166708@francesco-nb> (raw)
In-Reply-To: <av6Wc+kHFKqwFHAR@shlinux89> <20261116102331.K8Sc86V9@linutronix.de>

Hello Peng, Sebastian

On Mon, Nov 16, 2026 at 11:23:31AM +0100, Sebastian Andrzej Siewior wrote:
> On 2026-11-16 10:23:41 [+0100], Francesco Dolcini wrote:
> > Our CI shows a ~10-15% eMMC sequential read throughput drop on i.MX8QXP
> > and i.MX8DX (Colibri iMX8X, Cortex-A35), starting with v7.2-rc1 and
> > still present in v7.4-rc3 (hdparm --direct -t: ~129 MB/s -> 105-120
> > MB/s on i.MX8DX). i.MX8QM, i.MX8M and i.MX9 are not affected.
> > 
> > 5d1c0000.mailbox interrupt count before the test went from ~8k on v6.18
> > to 60k+ on v7.3/v7.4.
> > 
> > Peng: any idea/suggestion? Does it makes any sense to you?
> 
> Can you pin point it to a patch in this series? My guess is that
> something with the masking/ unmasking went wrong. The increase might be
> result of the interrupt keeping firing while the threaded interrupt did
> not have a chance to run.

Reverting 36cac4b5101f, 3225a745f517, e55bea377d49 and 692e1bc96a5d
restores 149 MB/s with schedutil.


On Thu, Nov 19, 2026 at 02:25:55PM +0800, Peng Fan wrote:
> On Mon, Nov 16, 2026 at 10:23:41AM +0100, Francesco Dolcini wrote:
> >Peng: any idea/suggestion? Does it makes any sense to you?
> 
> Please help test whether this helps:
> 
> diff --git a/drivers/mailbox/imx-mailbox.c b/drivers/mailbox/imx-mailbox.c
> index 683ccea6f2ba..e29044cf4b92 100644
> --- a/drivers/mailbox/imx-mailbox.c
> +++ b/drivers/mailbox/imx-mailbox.c
> @@ -635,7 +635,7 @@ static int imx_mu_startup(struct mbox_chan *chan)
>  {
>         struct imx_mu_priv *priv = to_imx_mu_priv(chan->mbox);
>         struct imx_mu_con_priv *cp = chan->con_priv;
> -       unsigned long irq_flag = IRQF_NO_THREAD;
> +       unsigned long irq_flag = IRQF_NO_THREAD | IRQF_ONESHOT;
>         int ret;
> 
>         pm_runtime_get_sync(priv->dev);

It does not help.
	
The analysis below was done with Claude help, on measurements I took on a
Colibri iMX8QXP. I hope you can pick it up from there, this is not my
area of expertise, and the best I can do is to send a revert for those 4
commits.

Every SCU RPC now wakes the SCHED_FIFO irq thread (TX done and RX), and
while an RT task is runnable schedutil requests the maximum CPU frequency.
The A35 clock is CLK_GET_RATE_NOCACHE, so each frequency change is
followed by an SCU RPC reading the rate back, which wakes the irq thread
again. The loop never settles, even when idle, and the transitions (ATF +
SCU) are what costs the eMMC throughput (and I assume there is more
impact on performance than just this).

v7.4-rc3, 4 CPUs, idle irq/cpufreq transitions per 10s, hdparm --direct -t:

                               irq    trans  schedutil  performance
  v6.18                          -      -    149.7 MB/s     -
  v7.4-rc3                   16203  10800    113 MB/s   150 MB/s
  v7.4-rc3 + IRQF_ONESHOT     7832   7813    113 MB/s   150 MB/s
  v7.4-rc3 + revert 4 commits   60     36    149 MB/s   150 MB/s

The 4 commits are 36cac4b5101f, 3225a745f517, e55bea377d49 and
692e1bc96a5d; reverting only the RX part (e55bea377d49) is not enough.


#regzbot introduced: 692e1bc96a5d^..36cac4b5101f
#regzbot title: mailbox: imx: threaded IRQ handling causes a cpufreq transition storm on SCU-based i.MX8

Thanks,
Francesco


  reply	other threads:[~2026-11-19 11:40 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-17  6:55 [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 01/10] mailbox: imx: Forward the timeout/ error in imx_mu_generic_tx() Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-18  1:50   ` Peng Fan
2026-06-18  1:57     ` Peng Fan
2026-06-22 11:24   ` Peng Fan
2026-06-24  7:44     ` Sebastian Andrzej Siewior
2026-06-26  8:23       ` Peng Fan
2026-06-26  8:34         ` Sebastian Andrzej Siewior
2026-06-26  9:00           ` Peng Fan (OSS)
2026-06-29 15:38             ` Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 02/10] mailbox: imx: Add a channel shutdown field Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 03/10] mailbox: imx: Use devm_pm_runtime_enable() Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 04/10] mailbox: imx: use devm_of_platform_populate() Sebastian Andrzej Siewior
2026-06-17  7:04   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 05/10] mailbox: imx: Use channel index instead of zero in imx_mu_specific_rx() Sebastian Andrzej Siewior
2026-06-17  7:13   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 06/10] mailbox: imx: Start splitting the IRQ handler in primary and threaded handler Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 07/10] mailbox: imx: Move the RX part of the mailbox into the " Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 08/10] mailbox: imx: Move the RXDB " Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 09/10] mailbox: imx: Don't force-thread the primary handler Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 10/10] remoteproc: imx_rproc: Invoke the callback directly Sebastian Andrzej Siewior
2026-06-17  7:19   ` sashiko-bot
2026-11-16  9:23 ` [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc Francesco Dolcini
2026-11-16 10:23   ` Sebastian Andrzej Siewior
2026-11-19 11:40     ` Francesco Dolcini [this message]
2026-11-19  6:25   ` Peng Fan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119114041.GA166708@francesco-nb \
    --to=francesco@dolcini.it \
    --cc=Frank.Li@nxp.com \
    --cc=andersson@kernel.org \
    --cc=bigeasy@linutronix.de \
    --cc=clrkwllms@kernel.org \
    --cc=festevam@gmail.com \
    --cc=francesco.dolcini@toradex.com \
    --cc=imx@lists.linux.dev \
    --cc=jassisinghbrar@gmail.com \
    --cc=kernel@pengutronix.de \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-remoteproc@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=mathieu.poirier@linaro.org \
    --cc=peng.fan@nxp.com \
    --cc=peng.fan@oss.nxp.com \
    --cc=rostedt@goodmis.org \
    --cc=s.hauer@pengutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox