Kernel unresponsive after booting 700+ vm's on a single host

Kernel unresponsive after booting 700+ vm's on a single host

[Alfred Bratterud]

[-- Attachment #1.1: Type: text/plain, Size: 734 bytes --]

For a research project we are trying to boot a very large amount of tiny, custom built VM's on KVM/ubuntu. The maximum VM-count achieved was 1000, but with substantial slowness, and eventually kernel failure, while the cpu/memory loads were nowhere near limits. Where is the likely bottleneck? Any solutions, workarounds, hacks or dirty tricks?

A few more details here (tumbleweed question), and the possibility of an upvote:
http://stackoverflow.com/questions/12243129/kvm-qemu-maximum-vm-count-limit

Any tips would be much appreciated!

Best regards,

Alfred Bratterud
Assistant Professor
Dept. of Computer Science,
Oslo and Akershus University College
of Applied Sciences
P:  (+47) 2245 3263
M: (+47) 4102 0222


[-- Attachment #1.2: Type: text/html, Size: 1522 bytes --]

[-- Attachment #2: Type: text/plain, Size: 182 bytes --]

_______________________________________________
libvirt-users mailing list
libvirt-users-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
https://www.redhat.com/mailman/listinfo/libvirt-users

Re: [libvirt-users] Kernel unresponsive after booting 700+ vm's on a single host

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

On 09/10/2012 07:51 AM, Alfred Bratterud wrote:
> For a research project we are trying to boot a very large amount of tiny, custom built VM's on KVM/ubuntu. The maximum VM-count achieved was 1000, but with substantial slowness, and eventually kernel failure, while the cpu/memory loads were nowhere near limits. Where is the likely bottleneck? Any solutions, workarounds, hacks or dirty tricks?

Are you using cgroups?  There have been some known bottlenecks in the
kernel cgroup code, where it scales very miserably; and since libvirt
uses a different cgroup per VM by default when cgroups are enabled, that
might explain part of the problem.

Other than that, if you can profile the slowdowns, I'm sure people would
be interested in the results.

-- 
Eric Blake   eblake@redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 617 bytes --]