PROBLEM: 2.6 kernels on x86 do not preserve FPU flags across context switches

[eliot@cincom.com]

Hi,

	I am the team lead and chief VM developer for a Smaltalk implementation based on a JIT execution engine.  Our customers have been seeing rare incorrect floating-point results in intensive fp applications on 2.6 kernels using various x86 compatible processors.  These problems do not occur on previous kernel versons.  We recently had occasion to reimplement our fp primitives to avoid severe performance problems on Xeon processors that were traced to Xeon's relatively slow implementation of fnclex and fstsw.  The older implementaton would produce a result and test for a valid (non NaN, non Inf) result by examining the FPU status flags via fstsw.  The newer implementation produces a result and tests its exponent for the NaN/Inf exponent.  The new implementation does not show the rare incorrect floating-point results in intensive fp applications on 2.6 kernels.  My conclusion is that context switches between the production of the result and the execution of the fstsw are the culprit, and that the context switch machinery fails to preserve the FPU status flags.

I don't know whether any action on your part is appropriate.  The use of the FPU status flags is presumably rare on linux (I believe that neither gcc nor glibc make use of them).  But "exotic" execution machinery such as runtimes for dynamic or functional languages (language implementations that may not use IEEE arithmetic and instead flag Infs and NaNs as an error) may fall foul of this issue.  Since previous versions of the kernel on x86 apparently do preserve the FPU status flags perhaps its simple to preserve the old behaviour.  At the very least let me suggest you document the limitation.

Sincerely,
---
Eliot Miranda                 ,,,^..^,,,                mailto:eliot@cincom.com
VisualWorks Engineering, Cincom  Smalltalk: scene not herd  Tel +1 408 216 4581
3350 Scott Blvd, Bldg 36 Suite B, Santa Clara, CA 95054 USA Fax +1 408 216 4500