Re: [PATCH 1/2] xl: neuter vcpu-set --ignore-host.

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Fri, Sep 27, 2013 at 09:41:02AM +0100, Ian Campbell wrote:
> On Thu, 2013-09-26 at 21:52 -0400, Konrad Rzeszutek Wilk wrote:
> > . snip..
> > > > I do get your frustration - why would a normal user want to shoot themselves
> > > > in the foot with VCPU over-subscription? I have some faint clue - but I
> > > > do to get a stream of requests from customers demanding it.
> > > 
> > > And not a single one has explained to you why they want it?
> > > 
> > > Or perhaps you could explain this faint clue of yours?
> > 
> > I believe it is mostly driven by VMWare making statements that this is a
> > OK scenario (see ESXi CPU considerations in
> > http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf)
> 
> The only reference to CPU (as opposed to memory) overcommit I can see in
> there is:
>         In most environments ESXi allows significant levels of CPU
>         overcommitment (that is, running more vCPUs on a host than the
>         total number of physical processor cores in that host) without
>         impacting virtual machine performance.
> 
> I think this is pretty clearly referring to the case where the total of
> all vCPUs across all guests exceeds the number of pCPUs. That kind over
> overcommit is obviously fine, and not at all what I'm arguing against.

I read that as running one guest with more vCPUs than host CPUs.
> 
> Nowhere in that doc did I get the impression that VMware were suggesting
> that giving a single VM more vCPUs than the host has pCPUs was a useful
> or sensible thing to do.
> 
> To be clear -- the patches you are proposing are removing the safety
> catch preventing the latter (1 VM with vCPUs > pCPUs) not the former
> (sum_all_VM(VCPUS_) > pCPUS). If xl is disallowing sum_all_VM(VCPUS_) >
> pCPUS then there is a real bug. I don't believe this to be the case
> though (if it is then we've been talking at cross purposes all along).

That scenario (sum_all_VM(VCPUs) > pCPUs) is working semi OK as long
as each VM VCPUs <= pCPUs.

.. snip..
> > > They can use the override.
> > 
> > Yes they can. I am was hoping we would allow a non override mode for
> > those who really don't want any of these overrides. George suggested the
> > "seatbelt" option and that looks to be a good compromise for me.
> 
> AIUI the "seatbelt" to which George was referring is the current
> behaviour (i.e. the override). He said:
>         So given what all of us think, keeping the "seatbelt" is
>         probably the best compromise.
> 
> i.e. to him the seatbelt is the status quo.
> 
> So what do you mean by seatbelt?

I was thinking off something like this in /etc/xen/xl.conf

#
# Disallow (unless used with --ignore-.. parameter) certain
# operations that would be questionable from a performance standpoint.
# For example overcommitting a single guest to have more VCPUs
# than there are physical CPUs.
seatbelt=yes
> 
> Or do you just mean to add the restriction + override to xl create too?
> That would be good.

That is a seperate patch I will post. I definitly want to add the check
for it (so vCPUS > pCPUS => warn user).

> 
> > > > Lastly, now that the PV ticketlocks are in and they work for both PV and
> > > > HVM I am curious how many people are going to start using it.
> > > 
> > > Why would they? What possible benefit is there to doing this whether or
> > > not PV ticketlocks are available?
> > 
> > Because now one can run Linux guests without incuring huge latency waits
> > due to spinlock contention. This makes it possible to actually compile a
> > Linux kernel with massively overcommited scenarios.
> 
> But *why*, for what possible reason would a user want do that? That is
> the key question which has never been answered and that's why I keep
> nacking this patch.

And why did the chicken cross the street? To get to the other side.

[OK, that is pretty poor joke].

My only reasoning behind this is:

 - Keep as much xend behavior as possible to ease the transition.
   It does not have to be on by default, but a global option
   to turn this on/off would be easy.

 - Users reading "best practices" and assuming that vCPU overcommit
   for a single guest should just work.

I poke more at the customers of why they would want to do it but
that will take some time get an answer.