From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: KVM and the OOM-Killer Date: Thu, 13 May 2010 15:39:20 +0300 Message-ID: <4BEBF2F8.7020507@redhat.com> References: <4BEBEE8F.9050508@jrcs.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org To: James Stevens Return-path: Received: from mx1.redhat.com ([209.132.183.28]:65061 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758910Ab0EMMjm (ORCPT ); Thu, 13 May 2010 08:39:42 -0400 In-Reply-To: <4BEBEE8F.9050508@jrcs.co.uk> Sender: kvm-owner@vger.kernel.org List-ID: On 05/13/2010 03:20 PM, James Stevens wrote: > This is *NOT* a KVM issue, but may be worth adding into the FAQ... > > > We have a KVM host with 48Gb of RAM and run about 20 KVM clients on > it. After some time - different time depending on the kernel version - > the VM host kernel will start OOM-Killing the VM clients, even when > there is lots of free RAM (>10Gb) and free SWAP (>34Gb). > > This seems to be caused by the kernel running out of LOWMEM (memory > below 1Gb) - because of the large amount of RAM a lot of LOWMEM > (~400Mb) is used by the memory map (32 bytes per 4Kb page), add in the > kernel itself and that leaves "only" about 460Mb of LOWMEM for kernel > alloc. > > This may not have been a problem, except Linux may also put cache > blocks and user processes in LOWMEM - it seems this can then lead to a > LOWMEM exhaust situation which triggers OOM-Killing even when there is > LOADS of SWAP and HIGHMEM free. > > Sadly, killing userland processes is not a good way to try and free > LOWMEM, so what happens is a killing spree where by every process on > the VM host gets killed (inc all the VMs, sysklogd, klogd, sshd, udevd > etc). > > This is very bad in 2.6.32.6, quite bad in 2.6.32.9, better (but > still bad in) 2.6.31.12 - currently testing 2.6.33.3 > > See https://bugzilla.kernel.org/show_bug.cgi?id=15058 > > > General advice seems to be, if you have more than 16Gb RAM then you > should run the VM host 64bit. > > We didn't see this issue on a server with 32Gb running the same set of > VMs. > I'd go with 64-bit at 2GB and above. It's both faster and safer. The lowmem load is about 0.5% of guest memory, so 48GB means 240MB lowmem allocated. Thin ice. Since you can run a 64-bit kernel with your existing userspace, at least you have a simple upgrade path. -- error compiling committee.c: too many arguments to function