[Qemu-devel] Guest memory mapping in Qemu

[Qemu-devel] Guest memory mapping in Qemu

[Michael T]

Hello,

This is an idle question in the sense that, much as I would like to, I know for a
fact that I won't have the time to look at implementing this.  I'm not expecting
other people to seriously look at doing it either, but I would be interested on your
thoughts.

If the technical documentation at
http://www.usenix.org/publications/library/proceedings/usenix05/tech/freenix/full_papers/bellard/bellard_html/index.html
is still valid (I think it is), Qemu has two modes of handling access to guest memory -
system emulation, in which an entire guest address space is mapped on the host, and
emulated MMU.  I was wondering whether something in-between would also be feasible.
That is, chunks of guest address space (say 4MB chunks for the sake of the argument)
are mmapped into the address space of the Qemu process on the host, and when an
access to guest memory is made, there is an initial check to see whether it is in
the same chunk as the last one, in which case all the MMU emulation bits could be
saved.  I could imagine Qemu keeping a current/most recent chunk for each register
which can be used for relative addressing, plus one for non-register-relative
accesses.  It seems to me that this could potentially speed up memory access quite a
bit, and as a bonus even make it easy to support x86 segmentation (as part of the
bounds check for whether a memory access is in a chunk).

I realise of course that I have glibly glossed over all the nasty bits - off the top
of my head keeping track of all the mapped chunks in the host address space, lookups
to see if an access outside of the current chunk is inside another mapped one,
invalidating chunks when guest page tables they are based on change.  I am sure that
there are many more issues...

Looking forward to reading any responses.

Regards,

Michael
 		 	   		  
_________________________________________________________________
Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
https://signup.live.com/signup.aspx?id=60969

Re: [Qemu-devel] Guest memory mapping in Qemu

[Paul Brook]

> If the technical documentation at
> http://www.usenix.org/publications/library/proceedings/usenix05/tech/freeni
> x/full_papers/bellard/bellard_html/index.html is still valid (I think it
>  is), Qemu has two modes of handling access to guest memory - system
>  emulation, in which an entire guest address space is mapped on the host,
>  and emulated MMU.  

No. qemu-fast (using the host address space) was removed long ago. There are a 
few stray remnants, but nothing useful. We always use an emulated MMU.

>  I was wondering whether something in-between would also
>  be feasible. That is, chunks of guest address space (say 4MB chunks for
>  the sake of the argument) are mmapped into the address space of the Qemu
>  process on the host, and when an access to guest memory is made, there is
>  an initial check to see whether it is in the same chunk as the last one,
>  in which case all the MMU emulation bits could be saved.  I could imagine
>  Qemu keeping a current/most recent chunk for each register which can be
>  used for relative addressing, plus one for non-register-relative accesses.
>   It seems to me that this could potentially speed up memory access quite a
>  bit, and as a bonus even make it easy to support x86 segmentation (as part
>  of the bounds check for whether a memory access is in a chunk).

This is effectively shadow paging implemented in userspace via mmap. It's very 
hard to make it work in a sane way, and even harder to make it go fast. TLB 
handling is already a significant bottleneck for many tasks, adding a mmap 
call is likely to make this orders of magnitude worse.  Most guests use 
virtual memory extensively, so the virtual->physical mappings tend to be 
extremely fragmented.

If you really want to do shadow paging for cross environments, you probably 
need to move it into kernel space. Either as a host kernel module, or as a 
bare-metal kernel/application that runs inside KVM. Even then you have to use 
various tricks to partition off a section of the host address space for use by 
qemu. It's not impossible, but it is a significant undertaking with somewhat 
unclear benefits.

Paul

RE: [Qemu-devel] Guest memory mapping in Qemu

[Michael T]

>> I was wondering whether something in-between would also
>> be feasible. That is, chunks of guest address space (say 4MB chunks for
>> the sake of the argument) are mmapped into the address space of the Qemu
>> process on the host, and when an access to guest memory is made, there is
>> an initial check to see whether it is in the same chunk as the last one,
>> in which case all the MMU emulation bits could be saved. I could imagine
>> Qemu keeping a current/most recent chunk for each register which can be
>> used for relative addressing, plus one for non-register-relative accesses.
>> It seems to me that this could potentially speed up memory access quite a
>> bit, and as a bonus even make it easy to support x86 segmentation (as part
>> of the bounds check for whether a memory access is in a chunk).
>
> This is effectively shadow paging implemented in userspace via mmap. It's very
> hard to make it work in a sane way, and even harder to make it go fast.
Yes, I can imagine that.

> TLB
> handling is already a significant bottleneck for many tasks, adding a mmap
> call is likely to make this orders of magnitude worse.
That might still depend on how much old mappings get reused and how often it
would be necessary to create new ones.  I am tempted to do a bit of profiling of
the memory usage patterns of a few guests to make an estimate.  Does Qemu
have any built-in statistics code that could be useful for this?
[snip]
> If you really want to do shadow paging for cross environments, you probably
> need to move it into kernel space.
That isn't as interesting, as there are already people doing that sort of thing.
The attractive thing about Qemu's emulation mode is that is is pure userspace!

Thanks for commenting.

Michael
 		 	   		  
_________________________________________________________________
Hotmail: Free, trusted and rich email service.
https://signup.live.com/signup.aspx?id=60969