From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Walker Subject: Re: System lockup with 2.6.26.8-rt16 on ARM9 [Solved] Date: Sat, 29 Aug 2009 07:42:45 -0700 Message-ID: <1251556965.9909.10.camel@desktop> References: <3efb10970908290047k6f4d5a14of148db7f2b179f18@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: Thomas Gleixner , LKML , linux-rt-users To: Remy Bohmer Return-path: Received: from fifo99.com ([67.223.236.141]:56620 "EHLO fifo99.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751700AbZH2Omt (ORCPT ); Sat, 29 Aug 2009 10:42:49 -0400 In-Reply-To: <3efb10970908290047k6f4d5a14of148db7f2b179f18@mail.gmail.com> Sender: linux-rt-users-owner@vger.kernel.org List-ID: On Sat, 2009-08-29 at 09:47 +0200, Remy Bohmer wrote: > Well, we found the root cause of this problem. > It turned out to be caused by sched_clock() that made disjunct time jumps. > This caused this check to become true in kernel/sched_rt.c:370: > if (rt_rq->rt_time > runtime) { > rt_rq->rt_throttled = 1; > if (rt_rq_throttled(rt_rq)) { > sched_rt_rq_dequeue(rt_rq); > return 1; > } > } > > The end results is that all realtime tasks got throttled for a long > time, and that time got extended every time sched_clock() made such a > jump. I would never have expected the scheduler would show this kind > of behaviour while CONFIG_RT_GROUP_SCHED is _not_ set... > > The root-cause of the sched_clock being faulty was a synchronisation > issue between 2 clock domains. The CPU clock and the clock domain of > the peripheral (GPT) on which the sched_clock() implementation was > based. The GPT made jumps backwards which triggered a false wraparound > detection in the conversion of 32->64 bit timestamps, causing the time > to jump about 356 seconds in the future... > Can you tell us more about what type of board this was? I've never heard of a ARM board having an unstable clocksource before .. Daniel