Re: Solaris 10 doesn't work under KVM

Waba
2007-02-10T13:34:43Z

It took me a while, but I figured it out... nearly!

Everything SIGILLs after the fs-root service is started. Its start
method does several things, but the problematic bit is replacing the
libc with an optimised version (namely, /usr/lib/libc/libc_hwcap1.so.1,
which makes use of the SSE, MMX, CMOV, SEP and FPU instruction sets
according to file(1)).

All these flags are indeed advertised in the CPUID (isainfo -v: sse2 sse
fxsr mmx cmov sep cx8 tsc fpu)). If the amd_sysc bit had been present,
the hwcap2 version would have been selected by moe(1), I guess (adds
SSE2 support and replaces SEP by AMD_SYSC).

Disabling the libc replacement in /lib/svc/method/fs-root entirely
workarounds the problem.

Further investigating, I tricked ls(1) into using the optimised libc
through LD_LIBRARY_PATH and obtained a core. mdb(1) told me that the
culprit was hiding at libc`memset+0x74. And finally, dis(1) revealed
that the faulty instruction is "movups (%esp), %xmm0", a SSE feature.
The %xmm0 register is apparently for storage purposes only, as the only
instructions used to access it are movups, movntps and movaps.

At this point I hope that it makes a lot of sense to you, because I
have no idea why it works fine on Avi's Opteron, etc.

Let me know if you need any additional debugging.
-Waba.