From mboxrd@z Thu Jan 1 00:00:00 1970 From: Etienne Martineau Subject: Re: Hypervisor architecture? Date: Tue, 04 May 2010 16:36:21 -0700 Message-ID: <1273016181.27500.124.camel@etienne-desktop> References: <1273001107.27500.5.camel@etienne-desktop> <4BE0847E.3090300@goop.org> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4BE0847E.3090300@goop.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jeremy Fitzhardinge Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Tue, 2010-05-04 at 13:33 -0700, Jeremy Fitzhardinge wrote: > On 05/04/2010 12:25 PM, Etienne Martineau wrote: > > Is there any documents around that describe the 'internal' architecture > > of the Hypervisor? > > > > So far my understanding is such that the Hypervisor is a 'strip-down' > > Linux 2.6.(12/13) + lot's of customization specific to Xen. > > > > Xen and Linux diverged a long time ago, early in the Linux 2.6 series, > with occasional feature-specific transfusions from Linux into Xen > since. There are some similarities in how they deal with platform > issues (APICs, etc), but they're mostly different now. > > > What is common? > > What is different? > > > > A vcpu is akin to a task; a domain is like a process (where multi-vcpu = > multi-threaded). Does the scheduler schedule 'vcpu' or 'domain'? > Events are a bit like signals (pending events are > checked for on return to guest mode and the return is redirected to the > event handler). Hypercalls are very similar to syscalls. Fair enough; > Memory > management is largely different. I see two modes of operation? (A) HVM; it looks like the Hypervisor is doing 'paging' and maintain sPTE. (B) PV; Guest has access to %cr3 and it seems to me that the Hypervisor is not involve on the fast path? I still haven't figure out the 'page_fault' path all the way up to the guest... > > One of the big differences is that Xen doesn't have a per-vcpu > hypervisor (kernel) stack, and vcpus don't have a hypervisor context. > While they're actually running in the hypervisor they use a pcpu stack, > but if it blocks/deschedules then it must always return to guest > context, saving away enough info to continue what it was doing when it > re-enters Xen. What is the rational behind the scene? Is it a matter of optimization; Since Hypervisor doesn't 'execute' code on the behalf of the Guest there is no requirement for per-vcpu hypervisor stack? {How about system call then} Or maybe a function of feature; It would be inconvenient to have an Hypervisor context associated with a VM when trying to migrate to another environment? > Once you get your head around that, a lot of things > become much clearer. > > J I see this as _valuable_ information for Xen newbies (like me). I think it would be good to have a 'xen/Documentation' folder to capture Hypervisor specific information? Correct me if I'm wrong but it looks like the existing 'docs/*' is geared toward 'operation' rather than internal implementation. I volunteer to help if needed. Thanks for your reply Jeremy. -Etienne