public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: <markus.stockhausen@gmx.de>
To: "'Daniel Lezcano'" <daniel.lezcano@linaro.org>,
	<tglx@linutronix.de>, <linux-kernel@vger.kernel.org>,
	<howels@allthatwemight.be>, <bjorn@mork.no>
Subject: AW: [PATCH 1/4] clocksource/drivers/timer-rtl-otto: work around dying timers
Date: Wed, 10 Sep 2025 12:16:36 +0200	[thread overview]
Message-ID: <007901dc223b$feb371a0$fc1a54e0$@gmx.de> (raw)
In-Reply-To: <af4a09e2-5b3e-4223-a926-4ab33b327416@linaro.org>

> Von: Daniel Lezcano <daniel.lezcano@linaro.org> 
> Gesendet: Mittwoch, 10. September 2025 11:03
> 
> On 04/08/2025 10:03, Markus Stockhausen wrote:
> > The OpenWrt distribution has switched from kernel longterm 6.6 to
> > 6.12. Reports show that devices with the Realtek Otto switch platform
> > die during operation and are rebooted by the watchdog. Sorting out
> > other possible reasons the Otto timer is to blame. The platform
> > currently consists of 4 targets with different hardware revisions.
> > It is not 100% clear which devices and revisions are affected.
> > 
> > Analysis shows:
> > 
> > A more aggressive sched/deadline handling leads to more timer starts
> > with small intervals. This increases the bug chances. See
> > https://marc.info/?l=linux-kernel&m=175276556023276&w=2
> > 
> > Focusing on the real issue a hardware limitation on some devices was
> > found. There is a minimal chance that a timer ends without firing an
> > interrupt if it is reprogrammed within the 5us before its expiration
> > time.
>
> Is it possible the timer IRQ flag is reset when setting the new counter 
> value ?
>
> While in the code path with the interrupt disabled, the timer expires in 
> these 5us, the IRQ flag is raised, then the driver sets a new value and 
> this flag is reset automatically, thus losing the current timer expiration ?

Something like this ...

During my analysis I tried a lot of things to identify the situation that
leads to this error. Especially just before the reprogramming command

static inline void rttm_enable_timer(void __iomem *base, u32 mode, u32 divisor)
{
  iowrite32(RTTM_CTRL_ENABLE | mode | divisor, base + RTTM_CTRL);
}

What I tried: 

1. Read out the current (remaining) timer value: In the error cases
this can give any value between 1 (=320ns) and 15 (=4800ns).

2. Check if IRQ flag is already set and IRQ might trigger next. This was 
never the case. 

3. Reorder reprogramming sequence (as far as possible). Only the
double reprogramming helped here.

So nothing we can do to actively identify and work around the buggy
situation. There is some hardware limitation between expiring timers
and reprgramming. Due to missing erratum the current bugfix is the
only (and best) solution I have.

Markus 


  reply	other threads:[~2025-09-10 10:16 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-08-04  8:03 [PATCH 0/4] clocksource/drivers/timer-rtl-otto: enhancements Markus Stockhausen
2025-08-04  8:03 ` [PATCH 1/4] clocksource/drivers/timer-rtl-otto: work around dying timers Markus Stockhausen
2025-09-10  9:02   ` Daniel Lezcano
2025-09-10 10:16     ` markus.stockhausen [this message]
2025-09-10 16:39       ` AW: " Daniel Lezcano
2025-09-10 18:16         ` AW: " markus.stockhausen
2025-09-10 20:53           ` Daniel Lezcano
2025-08-04  8:03 ` [PATCH 2/4] clocksource/drivers/timer-rtl-otto: drop set_counter function Markus Stockhausen
2025-08-04  8:03 ` [PATCH 3/4] clocksource/drivers/timer-rtl-otto: do not interfere with interrupts Markus Stockhausen
2025-08-04  8:03 ` [PATCH 4/4] clocksource/drivers/timer-rtl-otto: simplify documentation Markus Stockhausen
2025-09-10 20:54 ` [PATCH 0/4] clocksource/drivers/timer-rtl-otto: enhancements Daniel Lezcano
2025-09-10 21:15   ` AW: " markus.stockhausen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='007901dc223b$feb371a0$fc1a54e0$@gmx.de' \
    --to=markus.stockhausen@gmx.de \
    --cc=bjorn@mork.no \
    --cc=daniel.lezcano@linaro.org \
    --cc=howels@allthatwemight.be \
    --cc=linux-kernel@vger.kernel.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox