Linux kernel and device drivers for NXP i.MX platforms
 help / color / mirror / Atom feed
From: Francesco Dolcini <francesco.dolcini@toradex.com>
To: Peng Fan <peng.fan@linux.dev>
Cc: Francesco Dolcini <francesco@dolcini.it>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Peng Fan <peng.fan@oss.nxp.com>, Peng Fan <peng.fan@nxp.com>,
	linux-remoteproc@vger.kernel.org, imx@lists.linux.dev,
	linux-arm-kernel@lists.infradead.org,
	linux-rt-devel@lists.linux.dev,
	Bjorn Andersson <andersson@kernel.org>,
	Clark Williams <clrkwllms@kernel.org>,
	Fabio Estevam <festevam@gmail.com>, Frank Li <Frank.Li@nxp.com>,
	Jassi Brar <jassisinghbrar@gmail.com>,
	Mathieu Poirier <mathieu.poirier@linaro.org>,
	Pengutronix Kernel Team <kernel@pengutronix.de>,
	Sascha Hauer <s.hauer@pengutronix.de>,
	Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc
Date: Thu, 19 Nov 2026 15:44:17 +0100	[thread overview]
Message-ID: <av8LQa77SbshHmZD@francesco-nb> (raw)
In-Reply-To: <av8H7l6C8fmiHdbV@fedora>

On Thu, Nov 19, 2026 at 10:30:06PM +0800, Peng Fan wrote:
> On Thu, Nov 19, 2026 at 12:40:41PM +0100, Francesco Dolcini wrote:
> >Hello Peng, Sebastian
> >
> >On Mon, Nov 16, 2026 at 11:23:31AM +0100, Sebastian Andrzej Siewior wrote:
> >> On 2026-11-16 10:23:41 [+0100], Francesco Dolcini wrote:
> >> > Our CI shows a ~10-15% eMMC sequential read throughput drop on i.MX8QXP
> >> > and i.MX8DX (Colibri iMX8X, Cortex-A35), starting with v7.2-rc1 and
> >> > still present in v7.4-rc3 (hdparm --direct -t: ~129 MB/s -> 105-120
> >> > MB/s on i.MX8DX). i.MX8QM, i.MX8M and i.MX9 are not affected.
> >> > 
> >> > 5d1c0000.mailbox interrupt count before the test went from ~8k on v6.18
> >> > to 60k+ on v7.3/v7.4.
> >> > 
> >> > Peng: any idea/suggestion? Does it makes any sense to you?
> >> 
> >> Can you pin point it to a patch in this series? My guess is that
> >> something with the masking/ unmasking went wrong. The increase might be
> >> result of the interrupt keeping firing while the threaded interrupt did
> >> not have a chance to run.
> >
> >Reverting 36cac4b5101f, 3225a745f517, e55bea377d49 and 692e1bc96a5d
> >restores 149 MB/s with schedutil.
> >
> >
> >On Thu, Nov 19, 2026 at 02:25:55PM +0800, Peng Fan wrote:
> >> On Mon, Nov 16, 2026 at 10:23:41AM +0100, Francesco Dolcini wrote:
> >> >Peng: any idea/suggestion? Does it makes any sense to you?
> >> 
> >> Please help test whether this helps:
> >> 
> >> diff --git a/drivers/mailbox/imx-mailbox.c b/drivers/mailbox/imx-mailbox.c
> >> index 683ccea6f2ba..e29044cf4b92 100644
> >> --- a/drivers/mailbox/imx-mailbox.c
> >> +++ b/drivers/mailbox/imx-mailbox.c
> >> @@ -635,7 +635,7 @@ static int imx_mu_startup(struct mbox_chan *chan)
> >>  {
> >>         struct imx_mu_priv *priv = to_imx_mu_priv(chan->mbox);
> >>         struct imx_mu_con_priv *cp = chan->con_priv;
> >> -       unsigned long irq_flag = IRQF_NO_THREAD;
> >> +       unsigned long irq_flag = IRQF_NO_THREAD | IRQF_ONESHOT;
> >>         int ret;
> >> 
> >>         pm_runtime_get_sync(priv->dev);
> >
> >It does not help.
> >	
> >The analysis below was done with Claude help, on measurements I took on a
> >Colibri iMX8QXP. I hope you can pick it up from there, this is not my
> >area of expertise, and the best I can do is to send a revert for those 4
> >commits.
> >
> >Every SCU RPC now wakes the SCHED_FIFO irq thread (TX done and RX), and
> >while an RT task is runnable schedutil requests the maximum CPU frequency.
> >The A35 clock is CLK_GET_RATE_NOCACHE, so each frequency change is
> >followed by an SCU RPC reading the rate back, which wakes the irq thread
> >again. The loop never settles, even when idle, and the transitions (ATF +
> >SCU) are what costs the eMMC throughput (and I assume there is more
> >impact on performance than just this).
> 
> Not sure whether this is root cause, per my understanding, cpu freq update
> is not that high frequency call.

Moving to `performance` from `schedutil` or reverting those 4 patches
solve the issue. And the significant increase on ctx switches and irq
count is real even on idle without any emmc test running.

> Let me setup a i.MX8QXP board and see what's going on there.
Thanks!

> BTW, i.MX8QM should
> also have same issue, but seems not impact in your report, a bit werid,
> might be because i.MX8QM has more cores :)

I agree, I did not run any kind of extensive performance test. What
happened is just that our emmc test started to fail, and this lead to
this series. 

> Have you tried pin your emmc test thread to core3, or core2, just not core0.

It was never pinned to any core, so the scheduler decided on its own.
And multiple runs of the test all lead to the same results.

Francesco


  reply	other threads:[~2026-11-19 14:44 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-17  6:55 [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 01/10] mailbox: imx: Forward the timeout/ error in imx_mu_generic_tx() Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-18  1:50   ` Peng Fan
2026-06-18  1:57     ` Peng Fan
2026-06-22 11:24   ` Peng Fan
2026-06-24  7:44     ` Sebastian Andrzej Siewior
2026-06-26  8:23       ` Peng Fan
2026-06-26  8:34         ` Sebastian Andrzej Siewior
2026-06-26  9:00           ` Peng Fan (OSS)
2026-06-29 15:38             ` Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 02/10] mailbox: imx: Add a channel shutdown field Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 03/10] mailbox: imx: Use devm_pm_runtime_enable() Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 04/10] mailbox: imx: use devm_of_platform_populate() Sebastian Andrzej Siewior
2026-06-17  7:04   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 05/10] mailbox: imx: Use channel index instead of zero in imx_mu_specific_rx() Sebastian Andrzej Siewior
2026-06-17  7:13   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 06/10] mailbox: imx: Start splitting the IRQ handler in primary and threaded handler Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 07/10] mailbox: imx: Move the RX part of the mailbox into the " Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 08/10] mailbox: imx: Move the RXDB " Sebastian Andrzej Siewior
2026-06-17  6:55 ` [PATCH v3 09/10] mailbox: imx: Don't force-thread the primary handler Sebastian Andrzej Siewior
2026-06-17  7:08   ` sashiko-bot
2026-06-17  6:55 ` [PATCH v3 10/10] remoteproc: imx_rproc: Invoke the callback directly Sebastian Andrzej Siewior
2026-06-17  7:19   ` sashiko-bot
2026-11-16  9:23 ` [PATCH v3 00/10] mailbox: imx: Use threaded handler to avoid kworker in imx's remoteproc Francesco Dolcini
2026-11-16 10:23   ` Sebastian Andrzej Siewior
2026-11-19 11:40     ` Francesco Dolcini
2026-11-19 14:30       ` Peng Fan
2026-11-19 14:44         ` Francesco Dolcini [this message]
2026-11-19  6:25   ` Peng Fan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=av8LQa77SbshHmZD@francesco-nb \
    --to=francesco.dolcini@toradex.com \
    --cc=Frank.Li@nxp.com \
    --cc=andersson@kernel.org \
    --cc=bigeasy@linutronix.de \
    --cc=clrkwllms@kernel.org \
    --cc=festevam@gmail.com \
    --cc=francesco@dolcini.it \
    --cc=imx@lists.linux.dev \
    --cc=jassisinghbrar@gmail.com \
    --cc=kernel@pengutronix.de \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-remoteproc@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=mathieu.poirier@linaro.org \
    --cc=peng.fan@linux.dev \
    --cc=peng.fan@nxp.com \
    --cc=peng.fan@oss.nxp.com \
    --cc=rostedt@goodmis.org \
    --cc=s.hauer@pengutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox