Device model architecture (Was Re: Re: Are linker scripts needed?)

Device model architecture (Was Re: Re: Are linker scripts needed?)

[Arun Sharma]

Ian Pratt wrote:
> I'd be inclined to move to a model where we execute the device emulation
> in the root (monitor) VMCS, using the same protection mechanism we use
> for para-virtualized guests e.g. segmentation for x86, paging for
> x86_64. The device emulation should should work like a normal front-end
> driver, connecting via a device channel to a normal backend.
> 

It sounded like you were proposing linking the device models against 
Xen. But your subsequent messages appear to say:

- For every VMX domain created, create a new helper domain
- The helper domain shares it's page list with the VMX domain
- xen is protected from the helper domain using paging/segmentation
- helper domain runs minios
- Use the existing mechanisms (backend drivers) to get storage/network 
services from dom0

Did I get it right? If yes,

- why is this better than running the device models inside the VMX 
domain? Do you expect switching to the helper domain to be faster than a 
  vmx world switch?
- what's the advantage of running minios vs xenolinux in the helper domain?

I think we all agree that:

- It'd be good to make the device models "embeddable" so that it could 
be moved closer to the domain it's servicing. This is where the bulk of 
the work is, regardless of which model we end up choosing.

- Make sure that there is a unified way to manage the resources given to 
the VMX domain (including the device models)

	-Arun

Re: Device model architecture (Was Re: Re: Are linker scripts needed?)

[Keir Fraser]

On 24 May 2005, at 19:28, Arun Sharma wrote:

> Ian Pratt wrote:
>> I'd be inclined to move to a model where we execute the device 
>> emulation
>> in the root (monitor) VMCS, using the same protection mechanism we use
>> for para-virtualized guests e.g. segmentation for x86, paging for
>> x86_64. The device emulation should should work like a normal 
>> front-end
>> driver, connecting via a device channel to a normal backend.
>
> It sounded like you were proposing linking the device models against 
> Xen.

No, Ian is suggesting running them in 'paravirtualised mode': probably 
in ring 3 of the root VMCS. So there will be three contexts that a VMCS 
domain runs:
  1. Full-virt context (guest VMCS, rings 0-3)
  2. Hypervisor context (root VMS, ring 0)
  3. Paravirt driver context (root VMCS, rings 1-3)

> - For every VMX domain created, create a new helper domain
> - The helper domain shares it's page list with the VMX domain
> - xen is protected from the helper domain using paging/segmentation
> - helper domain runs minios
> - Use the existing mechanisms (backend drivers) to get storage/network 
> services from dom0

Yes.

> - why is this better than running the device models inside the VMX 
> domain? Do you expect switching to the helper domain to be faster than 
> a  vmx world switch?

Depends on whether you can make the CPU do a direct switch, or if you 
need to 'bounce' through root VMCS (taking an extra cr3 switch).

> - what's the advantage of running minios vs xenolinux in the helper 
> domain?

Full xenolinux is totally unnecessary. We just want enough support 
services to run the device emulator and front-end driver stub. We're 
best building those from scratch -- it'll look nothing like an 
operating system.

> I think we all agree that:
>
> - It'd be good to make the device models "embeddable" so that it could 
> be moved closer to the domain it's servicing. This is where the bulk 
> of the work is, regardless of which model we end up choosing.
>
> - Make sure that there is a unified way to manage the resources given 
> to the VMX domain (including the device models)

Yes, agreed.

  -- Keir

RE: Device model architecture (Was Re: Re: Are linker scripts needed?)

[Ian Pratt]

 
> > Ian Pratt wrote:
> >> I'd be inclined to move to a model where we execute the device 
> >> emulation in the root (monitor) VMCS, using the same protection 
> >> mechanism we use for para-virtualized guests e.g. segmentation for 
> >> x86, paging for x86_64. The device emulation should should 
> work like 
> >> a normal front-end driver, connecting via a device channel to a 
> >> normal backend.
> >
> > It sounded like you were proposing linking the device 
> models against 
> > Xen.
> 
> No, Ian is suggesting running them in 'paravirtualised mode': 
> probably in ring 3 of the root VMCS. So there will be three 
> contexts that a VMCS domain runs:
>   1. Full-virt context (guest VMCS, rings 0-3)
>   2. Hypervisor context (root VMS, ring 0)
>   3. Paravirt driver context (root VMCS, rings 1-3)

Exactly.

> > - why is this better than running the device models inside the VMX 
> > domain? Do you expect switching to the helper domain to be 
> faster than 
> > a  vmx world switch?
> 
> Depends on whether you can make the CPU do a direct switch, 
> or if you need to 'bounce' through root VMCS (taking an extra 
> cr3 switch).

The 'monitor_table' will also be the (single) pagetable for device
emulator and Xen.

There should be no need for extra cr3 switches (other than that incurred
by the vmexit).

When switching between the vmcs guest and the minios, we just need to
switch new values into the guest_table variable and the shadow_mode
variable and then all the Xen logic will do the right thing. (no cr3
flush in incurred)

> > - what's the advantage of running minios vs xenolinux in the helper 
> > domain?
> 
> Full xenolinux is totally unnecessary. We just want enough 
> support services to run the device emulator and front-end 
> driver stub. We're best building those from scratch -- it'll 
> look nothing like an operating system.

It'll be very simple and very little code -- just a few entry points,
support functions for printk etc. It's not even clear we need a memory
allocator. 

> > I think we all agree that:
> >
> > - It'd be good to make the device models "embeddable" so 
> that it could 
> > be moved closer to the domain it's servicing. This is where 
> the bulk 
> > of the work is, regardless of which model we end up choosing.
> >
> > - Make sure that there is a unified way to manage the 
> resources given 
> > to the VMX domain (including the device models)
> 
> Yes, agreed.

I think my proposal will work well :-)


Ian

Re: Device model architecture (Was Re: Re: Are linker scripts needed?)

[Arun Sharma]

In xen.devel, you wrote:
>
>> > - why is this better than running the device models inside the VMX 
>> > domain? Do you expect switching to the helper domain to be 
>> faster than 
>> > a  vmx world switch?
>> 
>> Depends on whether you can make the CPU do a direct switch, 
>> or if you need to 'bounce' through root VMCS (taking an extra 
>> cr3 switch).
>
> The 'monitor_table' will also be the (single) pagetable for device
> emulator and Xen.
>
> There should be no need for extra cr3 switches (other than that incurred
> by the vmexit).
>
> When switching between the vmcs guest and the minios, we just need to
> switch new values into the guest_table variable and the shadow_mode
> variable and then all the Xen logic will do the right thing. (no cr3
> flush in incurred)

I guess the main logic behind your argument is that there is no need to
fully virtualize the device models, so no need to run them within a
non-root VMCS.

This should give us some of the same management benefits that running
them inside a VMX domain would give us (memory sharing, device models
go away when the domain is destroyed etc).

The only comment I have is: for the case where the device models require the
services of a backend driver, you might be paying more than what we
currently pay (one domain switch) i.e. vmx domain -> mini os -> backend. But
it should be faster for everything else.

	-Arun