schedule_timeout: wrong timeout value

schedule_timeout: wrong timeout value

[kyle]

Hi,

Recently my mysql servershows something like:
Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value ffffffff 
from c0284efd
Dec 18 18:24:36 sql last message repeated 19939 times
Dec 18 18:25:37 sql last message repeated 33392 times

from syslog every 1 or 2 days. Whenever the messages show, mysql server stop 
accept new connections from the same network, and I need to restart the 
mysql service and then it will keep running well for 1-2 days until the 
messages show up again.

The server has been running over 1 year without any problem, the problem 
started show up around 2 weeks ago. It's running kernel 2.6.12, and mysql 
server, nothing else. Hardware is Pentium 4 2.8GHz with hyperthreading 
enabled.

What does the kernel message mean and why it make mysql stop accept new 
connections? Is it hardware problem or try upgrade the kernel may help?
Please CC me if possible. Thank you

Kyle

Re: schedule_timeout: wrong timeout value

[Robert Hancock]

kyle wrote:
> Hi,
> 
> Recently my mysql servershows something like:
> Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value 
> ffffffff from c0284efd
> Dec 18 18:24:36 sql last message repeated 19939 times
> Dec 18 18:25:37 sql last message repeated 33392 times
> 
> from syslog every 1 or 2 days. Whenever the messages show, mysql server 
> stop accept new connections from the same network, and I need to restart 
> the mysql service and then it will keep running well for 1-2 days until 
> the messages show up again.
> 
> The server has been running over 1 year without any problem, the problem 
> started show up around 2 weeks ago. It's running kernel 2.6.12, and 
> mysql server, nothing else. Hardware is Pentium 4 2.8GHz with 
> hyperthreading enabled.
> 
> What does the kernel message mean and why it make mysql stop accept new 
> connections? Is it hardware problem or try upgrade the kernel may help?
> Please CC me if possible. Thank you

The message means some code in the kernel or in some module passed a 
negative value to schedule_timeout which it shouldn't have. The c0284efd 
value is the address of the function that made the call - you may be 
able to look that up in your /proc/ksyms or the System.map file and 
figure out what function that is..

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: schedule_timeout: wrong timeout value

[Andrew Morton]

On Mon, 18 Dec 2006 20:34:43 -0600
Robert Hancock <hancockr@shaw.ca> wrote:

> kyle wrote:
> > Hi,
> > 
> > Recently my mysql servershows something like:
> > Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value 
> > ffffffff from c0284efd
> > Dec 18 18:24:36 sql last message repeated 19939 times
> > Dec 18 18:25:37 sql last message repeated 33392 times
> > 
> > from syslog every 1 or 2 days. Whenever the messages show, mysql server 
> > stop accept new connections from the same network, and I need to restart 
> > the mysql service and then it will keep running well for 1-2 days until 
> > the messages show up again.
> > 
> > The server has been running over 1 year without any problem, the problem 
> > started show up around 2 weeks ago. It's running kernel 2.6.12, and 
> > mysql server, nothing else. Hardware is Pentium 4 2.8GHz with 
> > hyperthreading enabled.
> > 
> > What does the kernel message mean and why it make mysql stop accept new 
> > connections? Is it hardware problem or try upgrade the kernel may help?
> > Please CC me if possible. Thank you
> 
> The message means some code in the kernel or in some module passed a 
> negative value to schedule_timeout which it shouldn't have. The c0284efd 
> value is the address of the function that made the call - you may be 
> able to look that up in your /proc/ksyms or the System.map file and 
> figure out what function that is..
> 

I queued this:


From: Andrew Morton <akpm@osdl.org>

Kyle is hitting this warning, and we don't have a clue what it's caused by. 
Add the obligatory dump_stack().

Cc: kyle <kylewong@southa.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
---

 kernel/timer.c |    7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff -puN kernel/timer.c~schedule_timeout-improve-warning-message kernel/timer.c
--- a/kernel/timer.c~schedule_timeout-improve-warning-message
+++ a/kernel/timer.c
@@ -1344,11 +1344,10 @@ fastcall signed long __sched schedule_ti
 		 * should never happens anyway). You just have the printk()
 		 * that will tell you if something is gone wrong and where.
 		 */
-		if (timeout < 0)
-		{
+		if (timeout < 0) {
 			printk(KERN_ERR "schedule_timeout: wrong timeout "
-				"value %lx from %p\n", timeout,
-				__builtin_return_address(0));
+				"value %lx\n", timeout);
+			dump_stack();
 			current->state = TASK_RUNNING;
 			goto out;
 		}
_

Re: schedule_timeout: wrong timeout value

[kyle]

----- Original Message ----- 
From: "Robert Hancock" <hancockr@shaw.ca>
To: "kyle" <kylewong@southa.com>
Cc: <linux-kernel@vger.kernel.org>
Sent: Tuesday, December 19, 2006 10:34 AM
Subject: Re: schedule_timeout: wrong timeout value


> kyle wrote:
>> Hi,
>>
>> Recently my mysql servershows something like:
>> Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value 
>> ffffffff from c0284efd
>> Dec 18 18:24:36 sql last message repeated 19939 times
>> Dec 18 18:25:37 sql last message repeated 33392 times
>>

> The message means some code in the kernel or in some module passed a 
> negative value to schedule_timeout which it shouldn't have. The c0284efd 
> value is the address of the function that made the call - you may be able 
> to look that up in your /proc/ksyms or the System.map file and figure out 
> what function that is..

There was no module loaded, and unfortunlately, I cannot find the System.map 
or /proc/ksyms file for the affected kernel!
Anyway thank you for your explanation. I have upgraded the kernel to 
2.6.17.14 and wish that it can fix the problem. Thank you

Kyle