fma4/avx under xen

fma4/avx under xen

[Jim Westfall]

Hi

I wanted to bring this to your attention

https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/956051

The summary is that when gcc compiles something with -mfma4 it also 
enables the use of the avx instruction set.  Since by default xen 
disables avx it leads to invalid opcodes.

This ends up being kinda nasty with the multiarch glibc since its doing 
stuff like

(HAS_FMA4 ? run_fma4_func() : (HAS_AVX ? run_avx_func() : run_func()))

run_fma4_func() can end up bombing out from the invalid opcode when run 
under xen.

Its not clear to me if xen should be filtering fma4 as part of its avx 
filter or if gcc should not assume avx support when compiling with 
-mfma4.

thoughts?

thanks
jim

Re: fma4/avx under xen

[Ian Campbell]

On Wed, 2012-05-02 at 19:54 +0100, Jim Westfall wrote:
> Hi
> 
> I wanted to bring this to your attention
> 
> https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/956051
> 
> The summary is that when gcc compiles something with -mfma4 it also 
> enables the use of the avx instruction set.  Since by default xen 
> disables avx it leads to invalid opcodes.
> 
> This ends up being kinda nasty with the multiarch glibc since its doing 
> stuff like
> 
> (HAS_FMA4 ? run_fma4_func() : (HAS_AVX ? run_avx_func() : run_func()))
> 
> run_fma4_func() can end up bombing out from the invalid opcode when run 
> under xen.
> 
> Its not clear to me if xen should be filtering fma4 as part of its avx 
> filter or if gcc should not assume avx support when compiling with 
> -mfma4.

http://software.intel.com/en-us/articles/introduction-to-intel-advanced-vector-extensions/
is pretty clear about the need for code to verify that the OS supports
these instructions before using them (see under "Detecting Availability
and Support"). The bit people seem to forget is the requirement to check
CPUID.1:ECX.OSXSAVE = 1 to ensure the OS supports the use of these
instructions (since they are not transparent to the OS) as well as
checking the processor features themselves.

If gcc/eglibc is not doing this before using those instructions then
that seems like it is a bug in either gcc or eglibc.

If the user is using some gcc option which forces the use of those
instructions when the (virtualised, or otherwise) processor does not
support them that seems like user error to me, it's no different to
using -mcpu=686 and then trying to run on a 486 class processor.

Sounds a lot like the same issue as
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=646549
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=646945

Ian


> 
> thoughts?
> 
> thanks
> jim
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: fma4/avx under xen

[Jan Beulich]

>>> On 02.05.12 at 20:54, Jim Westfall <jwestfall@surrealistic.net> wrote:
> The summary is that when gcc compiles something with -mfma4 it also 
> enables the use of the avx instruction set.  Since by default xen 
> disables avx it leads to invalid opcodes.

Not that I know of, at least not anymore in current -unstable.

> This ends up being kinda nasty with the multiarch glibc since its doing 
> stuff like
> 
> (HAS_FMA4 ? run_fma4_func() : (HAS_AVX ? run_avx_func() : run_func()))

The question is what (runtime) tests HAS_FMA includes. I know that
some glibc versions had a broken AVX check (which only checked for
the AVX feature flag, while the specification explicitly states that
a prerequisite check of the OSXSAVE feature flag is also necessary).

> run_fma4_func() can end up bombing out from the invalid opcode when run 
> under xen.
> 
> Its not clear to me if xen should be filtering fma4 as part of its avx 
> filter or if gcc should not assume avx support when compiling with 
> -mfma4.

-mfma4 implying AVX and OSXSAVE is quite okay afaict, so long as
the runtime checks are done correctly. Though (didn't check) from
my experience with other instruction set related -m options I would
really expect this to result in code using FMA4 instructions without
any runtime check. But perhaps your explanation above was just
a simplified version of what's really being done...

Jan

Re: fma4/avx under xen

[Jim Westfall]

Jan Beulich <JBeulich@suse.com> wrote [05.04.12]:
> >>> On 02.05.12 at 20:54, Jim Westfall <jwestfall@surrealistic.net> wrote:
> > The summary is that when gcc compiles something with -mfma4 it also 
> > enables the use of the avx instruction set.  Since by default xen 
> > disables avx it leads to invalid opcodes.
> 
> Not that I know of, at least not anymore in current -unstable.
> 
> > This ends up being kinda nasty with the multiarch glibc since its doing 
> > stuff like
> > 
> > (HAS_FMA4 ? run_fma4_func() : (HAS_AVX ? run_avx_func() : run_func()))
> 
> The question is what (runtime) tests HAS_FMA includes. I know that
> some glibc versions had a broken AVX check (which only checked for
> the AVX feature flag, while the specification explicitly states that
> a prerequisite check of the OSXSAVE feature flag is also necessary).

Its just a basic cpuid() call and seeing if the FMA4 bit is set.  I will 
bug the glibc guys.  Thanks for the feedback.

jim