[Qemu-devel] Virtual memory question

[Qemu-devel] Virtual memory question

[Dennis]

Hi,

I've been looking at the qemu source code but couldn't find anything
that would suit my needs.
Basically I'm looking for a way to use a file on disk as physical
memory inside Qemu.

Before I attempt to reinvent the wheel, has someone ever hacked in
similar functionality and if so is there a diff patch available
anywhere?

Re: [Qemu-devel] Virtual memory question

[C K Kashyap]

[-- Attachment #1: Type: text/plain, Size: 491 bytes --]

I am curious to know why you'd want to do it?

On Mon, Aug 9, 2010 at 2:03 AM, Dennis <dennis@satanclaus.com> wrote:

> Hi,
>
> I've been looking at the qemu source code but couldn't find anything
> that would suit my needs.
> Basically I'm looking for a way to use a file on disk as physical
> memory inside Qemu.
>
> Before I attempt to reinvent the wheel, has someone ever hacked in
> similar functionality and if so is there a diff patch available
> anywhere?
>
>


-- 
Regards,
Kashyap

[-- Attachment #2: Type: text/html, Size: 816 bytes --]

Re: [Qemu-devel] Virtual memory question

[Dennis]

On Mon, Aug 9, 2010 at 2:03 AM, Dennis <dennis@satanclaus.com> wrote:
>>
>> Hi,
>>
>> I've been looking at the qemu source code but couldn't find anything
>> that would suit my needs.
>> Basically I'm looking for a way to use a file on disk as physical
>> memory inside Qemu.
>>
>> Before I attempt to reinvent the wheel, has someone ever hacked in
>> similar functionality and if so is there a diff patch available
>> anywhere?
On Mon, Aug 9, 2010 at 7:01 AM, C K Kashyap <ckkashyap@gmail.com> wrote:

> I am curious to know why you'd want to do it?

I have an application that takes up over 2 GB of memory but is
otherwise demanding virtually no other resources at all.
Running multiple instances of the application using physical memory
isn't an option. Would it be hard to change Qemu's
physical memory mapping to a file on disk ? (Not Qemu-KVM, just stock Qemu).

Re: [Qemu-devel] Virtual memory question

[Stefan Hajnoczi]

On Mon, Aug 9, 2010 at 7:49 PM, Dennis <dennis@satanclaus.com> wrote:
> I have an application that takes up over 2 GB of memory but is
> otherwise demanding virtually no other resources at all.
> Running multiple instances of the application using physical memory
> isn't an option. Would it be hard to change Qemu's
> physical memory mapping to a file on disk ? (Not Qemu-KVM, just stock Qemu).

It's not 100% clear what you are trying to do, but are you aware that
guest "physical" memory can be swapped on the host?  If you put two
VMs on one host which use more memory than available host RAM, some of
their pages will get pushed out to swap, like any normal userspace
process.  Two things to consider in a memory overcommit scenario are
reduced performance due to swapping and allocating enough swap so the
out-of-memory killer does not terminate the QEMU process.

Also, have you looked at Kernel Samepage Merging (KSM)?  KSM detects
guest physical pages that contain the same contents and merge them
into a single page, saving memory.  This is appropriate if you run
multiple VMs based off the same disk image and expect they will be
executing the same code.  For more info, see
http://kernelnewbies.org/Linux_2_6_32#head-d3f32e41df508090810388a57efce73f52660ccb.

Stefan

Re: [Qemu-devel] Virtual memory question

[Dennis]

On Mon, Aug 9, 2010 at 9:41 PM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Mon, Aug 9, 2010 at 7:49 PM, Dennis <dennis@satanclaus.com> wrote:
>> I have an application that takes up over 2 GB of memory but is
>> otherwise demanding virtually no other resources at all.
>> Running multiple instances of the application using physical memory
>> isn't an option. Would it be hard to change Qemu's
>> physical memory mapping to a file on disk ? (Not Qemu-KVM, just stock Qemu).
>
> It's not 100% clear what you are trying to do, but are you aware that
> guest "physical" memory can be swapped on the host?

I know, but the host machine would have to run out of memory before it swaps.
That's not very desirable ;-)

> Also, have you looked at Kernel Samepage Merging (KSM)?  KSM detects
> guest physical pages that contain the same contents and merge them
> into a single page, saving memory.  This is appropriate if you run
> multiple VMs based off the same disk image and expect they will be
> executing the same code.  For more info, see
> http://kernelnewbies.org/Linux_2_6_32#head-d3f32e41df508090810388a57efce73f52660ccb.

I havn't yet. One "problem" is I _have_ to run Qemu on FreeBSD. I
doubt it's possible.

Re: [Qemu-devel] Virtual memory question

[Mulyadi Santosa]

Hi..

On Mon, Aug 9, 2010 at 03:33, Dennis <dennis@satanclaus.com> wrote:
> Hi,
>
> I've been looking at the qemu source code but couldn't find anything
> that would suit my needs.
> Basically I'm looking for a way to use a file on disk as physical
> memory inside Qemu.

Once when I did code adventure in Qemu, I conclude Qemu create
temporary file (possibly using mktemp()) and mmap it, thus creating
"illusion" file as RAM for guest.

So, I think it's already done. It's just not configurable through
command parameter

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Re: [Qemu-devel] Virtual memory question

[Stefan Hajnoczi]

On Mon, Aug 9, 2010 at 7:15 AM, Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
> On Mon, Aug 9, 2010 at 03:33, Dennis <dennis@satanclaus.com> wrote:
>> Hi,
>>
>> I've been looking at the qemu source code but couldn't find anything
>> that would suit my needs.
>> Basically I'm looking for a way to use a file on disk as physical
>> memory inside Qemu.
>
> Once when I did code adventure in Qemu, I conclude Qemu create
> temporary file (possibly using mktemp()) and mmap it, thus creating
> "illusion" file as RAM for guest.
>
> So, I think it's already done. It's just not configurable through
> command parameter

Use -mem-path /path/to/directory.  It's used for hugetlbfs support on
Linux but it should work on a normal filesystem too.

Stefan

Re: [Qemu-devel] Virtual memory question

[Michael Roth]

On 08/09/2010 03:17 AM, Stefan Hajnoczi wrote:
>
> Use -mem-path /path/to/directory.  It's used for hugetlbfs support on
> Linux but it should work on a normal filesystem too.
>
> Stefan
>

It *almost* works, except for some minor obstacles:

1) Normally the pages get mmap()'d with MAP_PRIVATE so they COW'd rather 
than written to the backing file:

#ifdef MAP_POPULATE
     /* NB: MAP_POPULATE won't exhaustively alloc all phys pages in the case
      * MAP_PRIVATE is requested.  For mem_prealloc we mmap as MAP_SHARED
      * to sidestep this quirk.
      */
     flags = mem_prealloc ? MAP_POPULATE | MAP_SHARED : MAP_PRIVATE;
     area = mmap(0, memory, PROT_READ | PROT_WRITE, flags, fd, 0);
#else
     area = mmap(0, memory, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
#endif

You can force a MAP_SHARED mapping without any changes to qemu by using 
the -mem_prealloc option, but you'll get MAP_POPULATE as well, which may 
not be desirable. A small patch would do the job though.

2) exec.c:file_ram_alloc() assumes you're allocating off a hugetlbfs and 
makes some system calls to get the block/hugepage size. A quick hack 
might be to comment out the following in exec.c:gethugepagesize():

if (fs.f_type != HUGETLBFS_MAGIC)
             fprintf(stderr, "Warning: path not on HugeTLBFS: %s\n", path);

You may also want to replace the mkstemp() with a mkostemp() and set 
O_SYNC on the file

But beyond hacks, I think generalizing -mempath might have some other 
useful applications (using it as a way to expose tmpfs-backed/numactl'd 
files as numa nodes to guests came up in an earlier discussion, and 
memory compression via zram/compcache is another).

Re: [Qemu-devel] Virtual memory question

[Michael Roth]

On 08/09/2010 11:46 AM, Michael Roth wrote:
>
> 2) exec.c:file_ram_alloc() assumes you're allocating off a hugetlbfs and
> makes some system calls to get the block/hugepage size. A quick hack
> might be to comment out the following in exec.c:gethugepagesize():
>
> if (fs.f_type != HUGETLBFS_MAGIC)
> fprintf(stderr, "Warning: path not on HugeTLBFS: %s\n", path);
>
> You may also want to replace the mkstemp() with a mkostemp() and set
> O_SYNC on the file
>
> But beyond hacks, I think generalizing -mempath might have some other
> useful applications (using it as a way to expose tmpfs-backed/numactl'd
> files as numa nodes to guests came up in an earlier discussion, and
> memory compression via zram/compcache is another).
>

Actually I guess 2) isn't really an issue, thought was an error path but 
I was mistaken.

-Mike