Re: Distro kernel and 'virtualization server' vs. 'server that sometimes runs virtual instances' rant (was: Re: Re: [GIT PULL] Xen APIC hooks (with io_apic_ops))

[Luke S Crawford <lsc@prgmr.com>]

Tim Post <echo@echoreply.us> writes:

> What, exactly is cowboy hackery? A dom-0 that might be a little slower
> if you boot it without Xen? 

No, I mean like Debian's 2.6.27 Dom0.  As far as I can tell, 
they imported the SUSE Xen patch once, and have not pulled any of SUSE's 
bugfixes since.   By all reports, it works fine, and is excellent as a 
desktop OS.   However, it's not something I want on the server where 
reboots cost me money.

(I'm further arguing that even in the case of a small office, you want your
'dedicated virtualization server' to be just that;  and rock solid.)  

> > If you've ever run a Xen host and have forgotten to change the default 
> > dom0-min-mem of 192MiB, you'd know most (especially x86_64) linux 
> > installations are not stable under load with that much memory.
> 
> I have , and I don't forget to change it.

But why make the default something that will crash the server?

> Have you even looked at / tried Eucalyptus ?

I've looked at it, I'm thinking about using it.  The EC2 interface is 
really neat.   I have a lot of admiration and respect for amazon.  But 
the interface is not what makes the small ec2 instances unsuitable as co-lo 
replacements, the unmirrored disk and the high price is.  the small
ec2 images are simply not designed to be a replacement for servers in the 
usual case.  They are great if you have a nice redundant application, 
designed to let any node fail at any time,  but that's not how most small 
businesses configure their servers.  For most small companies, it's cheaper 
to get reasonably good hardware and redundant disk, take backups, and then 
have a fire drill every time the hardware fails. 

> I don't have this problem. I export PV guest vitals over xenbus and set
> up watches on them. As for overcommitment, the first step is knowing how
> much memory each domain's kernel has actually promised to running
> processes. That much is already in the tree. 

That only solves half of the problem and gets you back to where
you are with FreeBSD jails/unionfs (well, also my users run their own
kernels and have full control over userland.)   Even with that problem 
solved, you still have the problem of disk cache, which is essential for 
acceptable performance.

If you want to buy a small image from me and thrash it, that's fine.
However, I don't want that user who underprovisions his or her
domain to make performance suck for a more responsible user.  This is
why I moved to xen in the first place;  a few heavy users were
trashing the disk cache on my FreeBSD jail system, and it was slow
for everyone. 

With the move to Xen, suddenly the heavy user was the only user
seeing the slowness.    Now the heavy user has the option of paying
me more money for more ram to use as disk cache, or of dealing with it
being slow.  Light users had no more trouble.  Log in once every 3 months?
your /etc/passwd is still cached from last time.  

This is why I'm so uneasy above overcommit.  Ram is not like CPU, which
you can take away at a moments notice and give back as if (almost) nothing
happened.  (or perhaps new CPUs are just so much more powerful than I 
need that I don't notice the degridation.)  

> Just as many others have done with debootstrap. I know your frustrated
> with dom-0 not being in mainline, we all are. However, it seems the
> tools frustrate you the most. Xen gives us a solid hypervisor, solid low
> level libraries and some examples on how to use them. I can't see (at
> this point) why you are so seemingly disgruntled?

hm.  Well, it is bad that I am coming across as disgruntled.  But 
I do think it is bad that the tools that come with xen seem to be focused
on Xen as a desktop OS at the expense of xen as a dedicated 
virtualization server.  I don't think Xen makes a particularly good
desktop virtualization platform, and this setup unnecessarily 
raises the barrier to using Xen as a dedicated virtualization server.