linux-rt-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Carsten Emde <C.Emde@osadl.org>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Linux RT Users <linux-rt-users@vger.kernel.org>
Subject: [RESOLVED OSADL QA 3.18.9-rt5 #1]
Date: Mon, 20 Apr 2015 23:22:05 +0200	[thread overview]
Message-ID: <55356DFD.2030508@osadl.org> (raw)
In-Reply-To: <55287A5C.8020402@osadl.org>

Hi Sebastian,

>>>> an Intel Bay Trail board (Intel(R) Celeron(R) CPU  J1900  @ 1.99GHz) at
>>>> the OSADL QA Farm rack #b/slot #6 (https://www.osadl.org/?id=1894)
>>>> stops
>>>> working every 12 to 36 hours. The only way to get the board back to
>>>> work
>>> [..]
>> Could you try this:
>> --
>> Subject: [PATCH] kernel/irq_work: fix no_hz deadlock
>>
>> Invoking NO_HZ's irq_work callback from timer irq is not working very
>> well if the callback decides to invoke hrtimer_cancel():
>>
>> |hrtimer_try_to_cancel+0x55/0x5f
>> |hrtimer_cancel+0x16/0x28
>> |tick_nohz_restart+0x17/0x72
>> |__tick_nohz_full_check+0x8e/0x93
>> |nohz_full_kick_work_func+0xe/0x10
>> |irq_work_run_list+0x39/0x57
>> |irq_work_tick+0x60/0x67
>> |update_process_times+0x57/0x67
>> |tick_sched_handle+0x4a/0x59
>> |tick_sched_timer+0x3b/0x64
>> |__run_hrtimer+0x7a/0x149
>> |hrtimer_interrupt+0x1cc/0x2c5
>>
>> and here we deadlock while waiting for the lock which we are holding.
>> To fix this I'm doing the same thing that upstream is doing: is the
>> irq_work dedicated IRQ and use it only for what is marked as "hirq"
>> which should only be the FULL_NO_HZ related work.
>> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
>> [..]
> Thanks a lot! Applied the patch and restarted the box. Given the fact
> that it took up to 36 hours until the board stopped, we unfortunately
> need to see at least one week of crash-free operation, before we may
> consider the bug as fixed.
The board survived nine days without a crash -> RESOLVED.

Thanks,
	Carsten.

  reply	other threads:[~2015-04-20 21:31 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-04-07 22:52 [OSADL QA 3.18.9-rt5 #1] Carsten Emde
2015-04-09 12:37 ` Sebastian Andrzej Siewior
2015-04-09 16:53 ` Sebastian Andrzej Siewior
2015-04-10 12:36   ` Sebastian Andrzej Siewior
2015-04-11  1:35     ` Carsten Emde
2015-04-20 21:22       ` Carsten Emde [this message]
2015-05-12  0:15     ` Steven Rostedt
2015-05-13  8:12       ` Sebastian Andrzej Siewior
2015-05-13 15:23         ` Steven Rostedt
2015-05-13 16:34     ` Steven Rostedt
2015-06-11 15:27       ` Sebastian Andrzej Siewior

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=55356DFD.2030508@osadl.org \
    --to=c.emde@osadl.org \
    --cc=bigeasy@linutronix.de \
    --cc=linux-rt-users@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).