RE: XEN 4.0 + 2.6.31.13 pvops kernel : system crashes on starting 155th domU

["Yuvraj Agarwal" <yuvraj@cs.ucsd.edu>]

Dom0 crashes (the network connections are all killed, and I cant log in
locally on the machine anymore either). 
I'll start up the 154 domU again and report back  /proc/interrupts and the
number of blktap2 devices. I do recall that when I checked last (cat
/sys/class/blktap2/*/name | wc -l ) I used to get 1 blktap2 device per
domU (which means that I had 154 blktap devices). Each domU has two
network interfaces.  I'll report back what we find when we start up all
the domUs again.  

On another related note it takes a LONG time to start up all these 150
domains (>20-30mins), and I believe the culprit is the xenstored since it
has to write the entire xenstore for each domU. We tried to edit to build
the OCAML version instead which is supposed to be faster (?), but
xenstored did not start in that case. We edited the Config.mk file to

CONFIG_OCAML_XENSTORED ?= y 

We did clean out the "dist" directory and rebuild and reinstall xen, but
xenstored did not start up. Are we missing obvious?

--Yuvraj 

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Jeremy
Fitzhardinge
Sent: Tuesday, April 27, 2010 11:52 AM
To: Yuvraj Agarwal
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] XEN 4.0 + 2.6.31.13 pvops kernel : system crashes
on starting 155th domU

On 04/27/2010 12:41 AM, Yuvraj Agarwal wrote:
>
> Hi All,
>
>  
>
> We are setting up a system with a large number of very small VMs for a
> project. We worked through a number of limitations, including those
> imposed by blktap2 devices and the number of dynamic IRQ (set kernel
> config NR_CPUS) etc. After these changes we were able to get to  154
> domUs (!), but as soon as we start up the 155^th domU the system
crashes.
>

How does the system crash?  You mean the dom0 kernel crashes?

> Does anyone know where this limit of 155 domU is coming from and how
> we can fix/increase it?
>

What does /proc/interrupts look like before the crash?  How many network
and block devices do you have in dom0?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel