From: Francesco Dolcini <francesco@dolcini.it>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Peng Fan <peng.fan@oss.nxp.com>
Cc: Francesco Dolcini <francesco.dolcini@toradex.com>,
Peng Fan <peng.fan@nxp.com>,
linux-remoteproc@vger.kernel.org, imx@lists.linux.dev,
linux-arm-kernel@lists.infradead.org,
linux-rt-devel@lists.linux.dev,
Bjorn Andersson <andersson@kernel.org>,
Clark Williams <clrkwllms@kernel.org>,
Fabio Estevam <festevam@gmail.com>, Frank Li <Frank.Li@nxp.com>,
Jassi Brar <jassisinghbrar@gmail.com>,
Mathieu Poirier <mathieu.poirier@linaro.org>,
Pengutronix Kernel Team <kernel@pengutronix.de>,
Sascha Hauer <s.hauer@pengutronix.de>,
Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc
Date: Thu, 19 Nov 2026 12:40:41 +0100 [thread overview]
Message-ID: <20261119114041.GA166708@francesco-nb> (raw)
In-Reply-To: <av6Wc+kHFKqwFHAR@shlinux89> <20261116102331.K8Sc86V9@linutronix.de>
Hello Peng, Sebastian
On Mon, Nov 16, 2026 at 11:23:31AM +0100, Sebastian Andrzej Siewior wrote:
> On 2026-11-16 10:23:41 [+0100], Francesco Dolcini wrote:
> > Our CI shows a ~10-15% eMMC sequential read throughput drop on i.MX8QXP
> > and i.MX8DX (Colibri iMX8X, Cortex-A35), starting with v7.2-rc1 and
> > still present in v7.4-rc3 (hdparm --direct -t: ~129 MB/s -> 105-120
> > MB/s on i.MX8DX). i.MX8QM, i.MX8M and i.MX9 are not affected.
> >
> > 5d1c0000.mailbox interrupt count before the test went from ~8k on v6.18
> > to 60k+ on v7.3/v7.4.
> >
> > Peng: any idea/suggestion? Does it makes any sense to you?
>
> Can you pin point it to a patch in this series? My guess is that
> something with the masking/ unmasking went wrong. The increase might be
> result of the interrupt keeping firing while the threaded interrupt did
> not have a chance to run.
Reverting 36cac4b5101f, 3225a745f517, e55bea377d49 and 692e1bc96a5d
restores 149 MB/s with schedutil.
On Thu, Nov 19, 2026 at 02:25:55PM +0800, Peng Fan wrote:
> On Mon, Nov 16, 2026 at 10:23:41AM +0100, Francesco Dolcini wrote:
> >Peng: any idea/suggestion? Does it makes any sense to you?
>
> Please help test whether this helps:
>
> diff --git a/drivers/mailbox/imx-mailbox.c b/drivers/mailbox/imx-mailbox.c
> index 683ccea6f2ba..e29044cf4b92 100644
> --- a/drivers/mailbox/imx-mailbox.c
> +++ b/drivers/mailbox/imx-mailbox.c
> @@ -635,7 +635,7 @@ static int imx_mu_startup(struct mbox_chan *chan)
> {
> struct imx_mu_priv *priv = to_imx_mu_priv(chan->mbox);
> struct imx_mu_con_priv *cp = chan->con_priv;
> - unsigned long irq_flag = IRQF_NO_THREAD;
> + unsigned long irq_flag = IRQF_NO_THREAD | IRQF_ONESHOT;
> int ret;
>
> pm_runtime_get_sync(priv->dev);
It does not help.
The analysis below was done with Claude help, on measurements I took on a
Colibri iMX8QXP. I hope you can pick it up from there, this is not my
area of expertise, and the best I can do is to send a revert for those 4
commits.
Every SCU RPC now wakes the SCHED_FIFO irq thread (TX done and RX), and
while an RT task is runnable schedutil requests the maximum CPU frequency.
The A35 clock is CLK_GET_RATE_NOCACHE, so each frequency change is
followed by an SCU RPC reading the rate back, which wakes the irq thread
again. The loop never settles, even when idle, and the transitions (ATF +
SCU) are what costs the eMMC throughput (and I assume there is more
impact on performance than just this).
v7.4-rc3, 4 CPUs, idle irq/cpufreq transitions per 10s, hdparm --direct -t:
irq trans schedutil performance
v6.18 - - 149.7 MB/s -
v7.4-rc3 16203 10800 113 MB/s 150 MB/s
v7.4-rc3 + IRQF_ONESHOT 7832 7813 113 MB/s 150 MB/s
v7.4-rc3 + revert 4 commits 60 36 149 MB/s 150 MB/s
The 4 commits are 36cac4b5101f, 3225a745f517, e55bea377d49 and
692e1bc96a5d; reverting only the RX part (e55bea377d49) is not enough.
#regzbot introduced: 692e1bc96a5d^..36cac4b5101f
#regzbot title: mailbox: imx: threaded IRQ handling causes a cpufreq transition storm on SCU-based i.MX8
Thanks,
Francesco
next prev parent reply other threads:[~2026-11-19 11:40 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-17 6:55 [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc Sebastian Andrzej Siewior
2026-06-17 6:55 ` [PATCH v3 01/10] mailbox: imx: Forward the timeout/ error in imx_mu_generic_tx() Sebastian Andrzej Siewior
2026-06-17 7:08 ` sashiko-bot
2026-06-18 1:50 ` Peng Fan
2026-06-18 1:57 ` Peng Fan
2026-06-22 11:24 ` Peng Fan
2026-06-24 7:44 ` Sebastian Andrzej Siewior
2026-06-26 8:23 ` Peng Fan
2026-06-26 8:34 ` Sebastian Andrzej Siewior
2026-06-26 9:00 ` Peng Fan (OSS)
2026-06-29 15:38 ` Sebastian Andrzej Siewior
2026-06-17 6:55 ` [PATCH v3 02/10] mailbox: imx: Add a channel shutdown field Sebastian Andrzej Siewior
2026-06-17 7:08 ` sashiko-bot
2026-06-17 6:55 ` [PATCH v3 03/10] mailbox: imx: Use devm_pm_runtime_enable() Sebastian Andrzej Siewior
2026-06-17 7:08 ` sashiko-bot
2026-06-17 6:55 ` [PATCH v3 04/10] mailbox: imx: use devm_of_platform_populate() Sebastian Andrzej Siewior
2026-06-17 7:04 ` sashiko-bot
2026-06-17 6:55 ` [PATCH v3 05/10] mailbox: imx: Use channel index instead of zero in imx_mu_specific_rx() Sebastian Andrzej Siewior
2026-06-17 7:13 ` sashiko-bot
2026-06-17 6:55 ` [PATCH v3 06/10] mailbox: imx: Start splitting the IRQ handler in primary and threaded handler Sebastian Andrzej Siewior
2026-06-17 7:08 ` sashiko-bot
2026-06-17 6:55 ` [PATCH v3 07/10] mailbox: imx: Move the RX part of the mailbox into the " Sebastian Andrzej Siewior
2026-06-17 6:55 ` [PATCH v3 08/10] mailbox: imx: Move the RXDB " Sebastian Andrzej Siewior
2026-06-17 6:55 ` [PATCH v3 09/10] mailbox: imx: Don't force-thread the primary handler Sebastian Andrzej Siewior
2026-06-17 7:08 ` sashiko-bot
2026-06-17 6:55 ` [PATCH v3 10/10] remoteproc: imx_rproc: Invoke the callback directly Sebastian Andrzej Siewior
2026-06-17 7:19 ` sashiko-bot
2026-11-16 9:23 ` [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc Francesco Dolcini
2026-11-16 10:23 ` Sebastian Andrzej Siewior
2026-11-19 11:40 ` Francesco Dolcini [this message]
2026-11-19 6:25 ` Peng Fan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119114041.GA166708@francesco-nb \
--to=francesco@dolcini.it \
--cc=Frank.Li@nxp.com \
--cc=andersson@kernel.org \
--cc=bigeasy@linutronix.de \
--cc=clrkwllms@kernel.org \
--cc=festevam@gmail.com \
--cc=francesco.dolcini@toradex.com \
--cc=imx@lists.linux.dev \
--cc=jassisinghbrar@gmail.com \
--cc=kernel@pengutronix.de \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-remoteproc@vger.kernel.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=mathieu.poirier@linaro.org \
--cc=peng.fan@nxp.com \
--cc=peng.fan@oss.nxp.com \
--cc=rostedt@goodmis.org \
--cc=s.hauer@pengutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox