From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [PATCH 0/8] KVM updates for 2.6.20-rc2 Date: Thu, 28 Dec 2006 14:32:08 +0200 Message-ID: <4593B948.5090009@qumranet.com> References: <45939755.7010603@qumranet.com> <20061228103345.GA4708@elte.hu> <4593A4B7.2070404@qumranet.com> <20061228113038.GA16190@elte.hu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel Return-path: To: Ingo Molnar In-Reply-To: <20061228113038.GA16190-X9Un+BFzKDI@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org Ingo Molnar wrote: > * Avi Kivity wrote: > > >> Talking about the scheduler (and to the scheduler's author :), it >> would be nice to hook the migration weight algorithm too. kvm guests >> are considerably more expensive to migrate, at least on intel. >> > > what do you mean precisely, and where does the cost come from? To me it > seems the cost of loading/saving a VMX context on Intel is the biggest > cost, and that's roughly the same cost for VM exits/enters, regardless > of what the scheduler does. What am i missing? > There are three types of vmx context switches. 1. The most common switch is a vm entry/exit pair (vmresume instruction). The context is already loaded into the cpu (via a vmptrld, below), and we enter into a preloaded VM. In addition, register access implicitly addresses the loaded context. 2. The second most common switch is loading another context. If we have several VMs on a uniprocessor machine, the execution sequence looks like this: vmptrld vm1 ... access context registers (vmread/vmwrite) vmresume ... access context registers (vmread/vmwrite) ... access context registers (vmread/vmwrite) vmresume ... access context registers (vmread/vmwrite) ... access context registers (vmread/vmwrite) vmresume ... access context registers (vmread/vmwrite) Linux context switch to another kvm process vmptrld vm2 ... access context registers (vmread/vmwrite) vmresume ... access context registers (vmread/vmwrite) ... access context registers (vmread/vmwrite) vmresume ... access context registers (vmread/vmwrite) Linux context switch to back to original process vmptrld vm1 Linux context switches to non-kvm processes and back need no special handling. The processor caches the current VM context, and possibly more than one. 3. The most expensive vmx context switch involves cpu migration: cpu 0: vmclear vm1 /* decache vmx context into memory */ cpu 1: vmptrld vm1 ... vm register access vmlaunch The vmlaunch instruction, like vmresume, causes a VM entry, but is documented to be significantly more expensive. It is required after a vmclear. Currently, the vmclear is performed by an ipi, because we can only detect migration after the fact. However, if we enlist the migration threads, we can vmclear before the process has left the cpu. -- error compiling committee.c: too many arguments to function ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV