[uml-devel] SKAS race condition status?

[uml-devel] SKAS race condition status?

[Christopher S. Aker]

Has there been any progress on the race in SKAS?  I've been dealing with 
2-4 hosts crashing a week.  Same thing happens on my old SKAS kernel, 
2.6.7 with skas3-v4 or so.

From:
http://marc.theaimsgroup.com/?t=112376981300002&r=1&w=2

Unable to handle kernel paging request at virtual address 00002000
   printing eip:
00000000
*pde = 2a771001
Oops: 0000 [#1]
SMP
Modules linked in:
CPU:    2
EIP:    0200:[<00000000>]    Not tainted VLI
EFLAGS: 00010292   (2.6.11.11-1-bigmem)
EIP is at 0x0
eax: ef66b784   ebx: ef7ba000   ecx: ef60f540   edx: 00000000
esi: 00000001   edi: 00000000   ebp: ef7bbf14   esp: ef7bbf9c
ds: 007b   es: 007b   ss: 0068
Process 2.4.29-linode39 (pid: 11195, threadinfo=ef7ba000 task=ef60f540)
Stack: ef7bbfc4 ef7ba000 c0106fd7 00000005 be7ff890 c0104934 be7ff768
be7ff760
         00000000 c0102639 be7ff768 00000000 40029d00 be7ff760 00000000
be7ff928
         00000000 0000007b 0000007b 0000004e 40100fb1 00000073 00000246
be7ff74c
Call Trace:
   [<c0106fd7>] do_syscall_trace+0x97/0x10e
   [<c0104934>] math_state_restore+0x24/0x40
   [<c0102639>] syscall_trace_entry+0x11/0x2a
Code:  Bad EIP value.

Is there code I could add there to OOPs rather than panic?  I'd prefer 
to have just one UML go down rather than the entire host.

Thanks!
-Chris


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Christopher S. Aker]

Christopher S. Aker wrote:
> Has there been any progress on the race in SKAS?  I've been dealing with 
> 2-4 hosts crashing a week.  Same thing happens on my old SKAS kernel, 
> 2.6.7 with skas3-v4 or so.

Jeff and I realized that this only happens on Xeon processors where the 
model number is > 2.  None of my model 2 Xeon hosts crash, but my model 
3 and model 4 xeons do.  grep model /proc/cpuinfo for info.  I've had 
skas3-v4 and skas3-v8.2 generate the same panic, so it's something 
processor model specific.

Peter, can you verify the problems you've had are on hosts with a model 
3 or 4 Xeon processor?  Also, with regard to 2.6.12, I'm not sure we're 
looking at the same issue, as I had problems with that kernel too but 
wasn't able to verify it was the same panic.  I have had 2.6.13 working, 
and it does generate the same SKAS panic eventually.

Thanks,
-Chris


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Christopher S. Aker]

/proc/cpuinfo -- model 3
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 3
model name      : Intel(R) Xeon(TM) CPU 2.80GHz
stepping        : 4
cpu MHz         : 2801.996
cache size      : 1024 KB
physical id     : 0
siblings        : 2
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm 
pni monitor ds_cpl cid xtpr
bogomips        : 5521.40


/proc/cpuinfo -- model 4
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 3
cpu MHz         : 3200.832
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm 
pni monitor ds_cpl cid cx16 xtpr
bogomips        : 6409.33

-Chris



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Blaisorblade]

On Tuesday 25 October 2005 18:57, Christopher S. Aker wrote:
> /proc/cpuinfo -- model 3
After some researches, it seems that the matching errata document is this one:

http://www.intel.com/design/xeon/specupdt/302402.htm

which would imply you have a 64-bit enabled processor... 

Note 1: that's not the only document matching, but the other is for 64-bit 
processors too

Note 2: I couldn't match steppings between those docs and cpuinfo...
-- 
Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894)
http://www.user-mode-linux.org/~blaisorblade

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Christopher S. Aker]

Blaisorblade wrote:
> After some researches, it seems that the matching errata document is this one:
> 
> http://www.intel.com/design/xeon/specupdt/302402.htm
> 
> which would imply you have a 64-bit enabled processor... 
> 
> Note 1: that's not the only document matching, but the other is for 64-bit 
> processors too
> 
> Note 2: I couldn't match steppings between those docs and cpuinfo...

OK.  I matched up some purchase orders with Intel part numbers:

BX80546KG3200FU <-- model 4
BX80546KG3200EU <-- model 3 (I think)

I'll start disabling HyperThreading on hosts that crash next.

-Chris


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Peter]

I have skas3-v8 host servers with model 4 processes and good (100+ day) 
uptimes.  The host server I had issues with (host38) got converted to 
Xen and is still having problems.  I am seeing issues with host34 that 
require the odd restart.  That server runs the same CPU model and kernel 
code as other servers of mine with good uptime.

My conclusion is that I must be seeing hardware issues rather than a 
skas bug.

- Peter

# for ((i=34;i<41;i++)); do echo host$i $(ssh host$i 'echo $(cat 
/proc/cpuinfo  | grep model | grep -v name | head -n 1) $(uname -r) 
$(uptime)'); done
host34 model : 2 2.6.11.10-skas3-v8 21:52:37 up 15:19, 0 users, load 
average: 1.64, 1.60, 1.32
host35 model : 4 2.6.11.7-skas3-v8 21:49:21 up 42 days, 12:15, 0 users, 
load average: 13.72, 9.66, 5.21
host36 model : 4 2.6.11.12-skas3-v8.2 21:49:23 up 61 days, 0 min, 0 
users, load average: 1.26, 1.44, 1.78
host37 model : 4 2.6.11.7-skas3-v8 21:49:25 up 121 days, 9:48, 0 users, 
load average: 0.91, 1.04, 1.14
host38 model : 2 2.6.11.12-xen0 21:49:26 up 7 days, 13:28, 0 users, load 
average: 0.02, 0.05, 0.01
host39 model : 4 2.6.11.7-skas3-v8 21:49:28 up 106 days, 2:46, 0 users, 
load average: 0.59, 0.65, 0.70
host40 model : 4 2.6.11.7-skas3-v8 21:49:39 up 121 days, 9:47, 0 users, 
load average: 0.27, 0.43, 0.47


Christopher S. Aker wrote:
> Christopher S. Aker wrote:
> 
>> Has there been any progress on the race in SKAS?  I've been dealing 
>> with 2-4 hosts crashing a week.  Same thing happens on my old SKAS 
>> kernel, 2.6.7 with skas3-v4 or so.
> 
> 
> Jeff and I realized that this only happens on Xeon processors where the 
> model number is > 2.  None of my model 2 Xeon hosts crash, but my model 
> 3 and model 4 xeons do.  grep model /proc/cpuinfo for info.  I've had 
> skas3-v4 and skas3-v8.2 generate the same panic, so it's something 
> processor model specific.
> 
> Peter, can you verify the problems you've had are on hosts with a model 
> 3 or 4 Xeon processor?  Also, with regard to 2.6.12, I'm not sure we're 
> looking at the same issue, as I had problems with that kernel too but 
> wasn't able to verify it was the same panic.  I have had 2.6.13 working, 
> and it does generate the same SKAS panic eventually.
> 
> Thanks,
> -Chris


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Christopher S. Aker]

Peter wrote:
> I have skas3-v8 host servers with model 4 processes and good (100+ day) 
> uptimes.  The host server I had issues with (host38) got converted to 
> Xen and is still having problems.  I am seeing issues with host34 that 
> require the odd restart.  That server runs the same CPU model and kernel 
> code as other servers of mine with good uptime.
> 
> My conclusion is that I must be seeing hardware issues rather than a 
> skas bug.

By chance, do you have Hyperthreading disabled on your model 4 Xeon boxes?

-Chris


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] SKAS race condition status?

[Peter]

HT is not disabled on any of my boxes.

Christopher S. Aker wrote:
> Peter wrote:
> 
>> I have skas3-v8 host servers with model 4 processes and good (100+ 
>> day) uptimes.  The host server I had issues with (host38) got 
>> converted to Xen and is still having problems.  I am seeing issues 
>> with host34 that require the odd restart.  That server runs the same 
>> CPU model and kernel code as other servers of mine with good uptime.
>>
>> My conclusion is that I must be seeing hardware issues rather than a 
>> skas bug.
> 
> 
> By chance, do you have Hyperthreading disabled on your model 4 Xeon boxes?
> 
> -Chris


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel