file-backed guest memory

file-backed guest memory

[Michael Riepe]

Hi!

This is just a (probably silly) idea I had the other day. Currently, the
guest's memory is allocated inside the kernel and exported to userspace
via mmap(). But wouldn't it also be possible to create a file in
userspace and pass its descriptor to kvm? If we also pass file offset
and length parameters for each memslot, all segments can (but need not)
reside in the same file. There would be a persistent snapshot of the
VM's physical memory, and it would enable the VM to page out the guest's
pages. One could also do strange things like mapping a portion of the
file several times, e.g. to emulate an architecture with incomplete
address decoding. Applications that absolutely want to use anonymous
memory could pass -1 as the fd, as they do with mmap(MAP_ANONYMOUS).

Comments?

-- 
Michael "Tired" Riepe <michael-0QoEqw4nQxo@public.gmane.org>
X-Tired: Each morning I get up I die a little

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: file-backed guest memory

[Avi Kivity]

Michael Riepe wrote:
> Hi!
>
> This is just a (probably silly) idea I had the other day. Currently, the
> guest's memory is allocated inside the kernel and exported to userspace
> via mmap(). But wouldn't it also be possible to create a file in
> userspace and pass its descriptor to kvm? If we also pass file offset
> and length parameters for each memslot, all segments can (but need not)
> reside in the same file. There would be a persistent snapshot of the
> VM's physical memory, and it would enable the VM to page out the guest's
> pages. One could also do strange things like mapping a portion of the
> file several times, e.g. to emulate an architecture with incomplete
> address decoding. Applications that absolutely want to use anonymous
> memory could pass -1 as the fd, as they do with mmap(MAP_ANONYMOUS).
>   

Arnd suggested this way back when kvm was first posted on lkml, and I 
agree that this is a very useful mechanism.  You get on-demand loading, 
swap, hugetlbfs, and maybe other nifty stuff.  I think I know how to do 
this for the current mmu, but I'm worried that it will have a 
performance impact with the nested page tables mmu.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: file-backed guest memory

[Laurent Vivier]

[-- Attachment #1.1: Type: text/plain, Size: 1483 bytes --]

Avi Kivity wrote:
> Michael Riepe wrote:
>> Hi!
>>
>> This is just a (probably silly) idea I had the other day. Currently, the
>> guest's memory is allocated inside the kernel and exported to userspace
>> via mmap(). But wouldn't it also be possible to create a file in
>> userspace and pass its descriptor to kvm? If we also pass file offset
>> and length parameters for each memslot, all segments can (but need not)
>> reside in the same file. There would be a persistent snapshot of the
>> VM's physical memory, and it would enable the VM to page out the guest's
>> pages. One could also do strange things like mapping a portion of the
>> file several times, e.g. to emulate an architecture with incomplete
>> address decoding. Applications that absolutely want to use anonymous
>> memory could pass -1 as the fd, as they do with mmap(MAP_ANONYMOUS).
>>   
> 
> Arnd suggested this way back when kvm was first posted on lkml, and I 
> agree that this is a very useful mechanism.  You get on-demand loading, 
> swap, hugetlbfs, and maybe other nifty stuff.  I think I know how to do 
> this for the current mmu, but I'm worried that it will have a 
> performance impact with the nested page tables mmu.
> 

Perhaps you could use code from kqemu ?

Regards,
Laurent


-- 
------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 345 bytes --]

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: file-backed guest memory

[Avi Kivity]

Laurent Vivier wrote:
> Avi Kivity wrote:
>   
>> Michael Riepe wrote:
>>     
>>> Hi!
>>>
>>> This is just a (probably silly) idea I had the other day. Currently, the
>>> guest's memory is allocated inside the kernel and exported to userspace
>>> via mmap(). But wouldn't it also be possible to create a file in
>>> userspace and pass its descriptor to kvm? If we also pass file offset
>>> and length parameters for each memslot, all segments can (but need not)
>>> reside in the same file. There would be a persistent snapshot of the
>>> VM's physical memory, and it would enable the VM to page out the guest's
>>> pages. One could also do strange things like mapping a portion of the
>>> file several times, e.g. to emulate an architecture with incomplete
>>> address decoding. Applications that absolutely want to use anonymous
>>> memory could pass -1 as the fd, as they do with mmap(MAP_ANONYMOUS).
>>>   
>>>       
>> Arnd suggested this way back when kvm was first posted on lkml, and I 
>> agree that this is a very useful mechanism.  You get on-demand loading, 
>> swap, hugetlbfs, and maybe other nifty stuff.  I think I know how to do 
>> this for the current mmu, but I'm worried that it will have a 
>> performance impact with the nested page tables mmu.
>>
>>     
>
> Perhaps you could use code from kqemu ?
>
>   

The non-npt case is not so difficult, it is the npt case which I'm 
concerned about and that doesn't have a parallel in kqemu.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: file-backed guest memory

[Christian Borntraeger]

On Thursday 12 April 2007 17:38, Avi Kivity wrote:
> > But wouldn't it also be possible to create a file in
> > userspace and pass its descriptor to kvm?
[...]
> swap, hugetlbfs, and maybe other nifty stuff.  I think I know how to do 
> this for the current mmu, but I'm worried that it will have a 
> performance impact with the nested page tables mmu.

Another thing to consider is the commit "mm: tracking shared dirty pages" 
(d08b3851da41d0ee60851f2c75b118e1f7a5fc89). We need a shared mapping to 
actually have guest memory written back to the file.
The dirty page tracking change will cause the host kernel to write protect 
guest memory pages after writeback. This will often trigger guest exits due 
to host protection faults. If we want to have file backed guest memory we 
really should modify the VMA to be not eligible for dirty page tracking. 

Christian

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: file-backed guest memory

[Avi Kivity]

Christian Borntraeger wrote:
> On Thursday 12 April 2007 17:38, Avi Kivity wrote:
>   
>>> But wouldn't it also be possible to create a file in
>>> userspace and pass its descriptor to kvm?
>>>       
> [...]
>   
>> swap, hugetlbfs, and maybe other nifty stuff.  I think I know how to do 
>> this for the current mmu, but I'm worried that it will have a 
>> performance impact with the nested page tables mmu.
>>     
>
> Another thing to consider is the commit "mm: tracking shared dirty pages" 
> (d08b3851da41d0ee60851f2c75b118e1f7a5fc89). We need a shared mapping to 
> actually have guest memory written back to the file.
> The dirty page tracking change will cause the host kernel to write protect 
> guest memory pages after writeback. This will often trigger guest exits due 
> to host protection faults. If we want to have file backed guest memory we 
> really should modify the VMA to be not eligible for dirty page tracking. 
>   

Actually the Linux mm does not know about the kvm page tables, and thus
cannot do the write protection.  I'm worried that this will confuse it
horribly.

I also think that we should teach the mm how to write protect guest
pages, perhaps less aggressively to account for the large difference in
normal pagefault cost and vmexit pagefault cost.  Hopefully this write
protection is throttled when memory is plentiful to avoid a large
performance hit, regardless of whether it's used for a virtual machine
or normal processes.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/