KVM hypervisor - hardware issues? - Of topic

KVM hypervisor - hardware issues? - Of topic

[hampus.lind]

 Hi,

We are facing big problems with our esxi 5 and HP DL585 G7 environment. Several of our HP servers reboots/hangs randomly without leaving any trace in any logs. HP hardware diag shows no errors in hardware and vmware support are clueless.

My finding when searching for a reason is that there seems to be a lot strange reboot issues for ESX and various servers vendors. A quick google on "esxi random reboot" gives me a long list of both HP and Dell customers facing similar issues.

So we continue to dig and there seems to be a lot of issues with drivers and firmware version back and forth between vmware and server vendors... A real mess... 

Trying to narrow down the suspects I wonder if KVM users have the same issues with hypervisors randomly rebooting/hang when running HP DL servers (or dell)? If large high performance KVM environments runs without problems on those servers without reboot problems then we can have a serious talk with vmware about these issues, otherwise we are trapped between vendors point fingers.

We have six HP DL858 G7 servers which all randomly reboot/hang now and then.

Thanks,
Hampus

Re: KVM hypervisor - hardware issues? - Of topic

[Jan Kiszka]

On 2012-09-19 12:42, hampus.lind@ongame.com wrote:
>  Hi,
> 
> We are facing big problems with our esxi 5 and HP DL585 G7 environment. Several of our HP servers reboots/hangs randomly without leaving any trace in any logs. HP hardware diag shows no errors in hardware and vmware support are clueless.
> 
> My finding when searching for a reason is that there seems to be a lot strange reboot issues for ESX and various servers vendors. A quick google on "esxi random reboot" gives me a long list of both HP and Dell customers facing similar issues.
> 
> So we continue to dig and there seems to be a lot of issues with drivers and firmware version back and forth between vmware and server vendors... A real mess... 
> 
> Trying to narrow down the suspects I wonder if KVM users have the same issues with hypervisors randomly rebooting/hang when running HP DL servers (or dell)? If large high performance KVM environments runs without problems on those servers without reboot problems then we can have a serious talk with vmware about these issues, otherwise we are trapped between vendors point fingers.
> 
> We have six HP DL858 G7 servers which all randomly reboot/hang now and then.

I'm recalling some report of random hard lock-ups on an HP server, I
think this was with Xen, a year ago or so. I do know that my Celsius
H700 Fujisu Notebook locks up hard under certain (uncommon) guest
workloads in KVM, most probably due to bugs in the SMI handling code of
the BIOS. Maybe you are facing a similar issue.

To check the correlation between lockup and SMI you can try to disable
SMIs (warning: may have unwanted or even dangerous side effects for you
hardware) via this tool and studying your chipset manual /wrt SMI control:

http://git.kiszka.org/?p=smictrl.git

In my case, the correlation is fairly telling. See also [1].

Jan

[1] http://permalink.gmane.org/gmane.comp.emulators.kvm.devel/60326

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux