From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from [140.186.70.92] (port=33147 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1OiVUj-0005Kr-TO
	for qemu-devel@nongnu.org; Mon, 09 Aug 2010 12:46:39 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <mdroth@linux.vnet.ibm.com>) id 1OiVUi-0004JH-Aj
	for qemu-devel@nongnu.org; Mon, 09 Aug 2010 12:46:37 -0400
Received: from e8.ny.us.ibm.com ([32.97.182.138]:35766)
	by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <mdroth@linux.vnet.ibm.com>) id 1OiVUi-0004Fi-8T
	for qemu-devel@nongnu.org; Mon, 09 Aug 2010 12:46:36 -0400
Received: from d01relay07.pok.ibm.com (d01relay07.pok.ibm.com [9.56.227.147])
	by e8.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id o79GhTTY009183
	for <qemu-devel@nongnu.org>; Mon, 9 Aug 2010 12:43:29 -0400
Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215])
	by d01relay07.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id
	o79Gk8YQ2322470
	for <qemu-devel@nongnu.org>; Mon, 9 Aug 2010 12:46:12 -0400
Received: from d01av01.pok.ibm.com (loopback [127.0.0.1])
	by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id
	o79Gk77f018583
	for <qemu-devel@nongnu.org>; Mon, 9 Aug 2010 12:46:07 -0400
Message-ID: <4C6030CE.3080400@linux.vnet.ibm.com>
Date: Mon, 09 Aug 2010 11:46:06 -0500
From: Michael Roth <mdroth@linux.vnet.ibm.com>
MIME-Version: 1.0
Subject: Re: [Qemu-devel] Virtual memory question
References: <AANLkTinS-n1x-VQXtMupcOP8gHUfRXmbw6Cgn0vUYmQP@mail.gmail.com>
	<AANLkTimDgoHU-jsEKX=v8RYz5af0b=4ihJ2KV-QgKyKS@mail.gmail.com>
	<AANLkTikv_Bi7uOX4GZh+de9XTcQMs4F8PdUJAZrbZsVt@mail.gmail.com>
In-Reply-To: <AANLkTikv_Bi7uOX4GZh+de9XTcQMs4F8PdUJAZrbZsVt@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
List-Id: qemu-devel.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Stefan Hajnoczi <stefanha@gmail.com>
Cc: qemu-devel@nongnu.org

On 08/09/2010 03:17 AM, Stefan Hajnoczi wrote:
>
> Use -mem-path /path/to/directory.  It's used for hugetlbfs support on
> Linux but it should work on a normal filesystem too.
>
> Stefan
>

It *almost* works, except for some minor obstacles:

1) Normally the pages get mmap()'d with MAP_PRIVATE so they COW'd rather 
than written to the backing file:

#ifdef MAP_POPULATE
     /* NB: MAP_POPULATE won't exhaustively alloc all phys pages in the case
      * MAP_PRIVATE is requested.  For mem_prealloc we mmap as MAP_SHARED
      * to sidestep this quirk.
      */
     flags = mem_prealloc ? MAP_POPULATE | MAP_SHARED : MAP_PRIVATE;
     area = mmap(0, memory, PROT_READ | PROT_WRITE, flags, fd, 0);
#else
     area = mmap(0, memory, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
#endif

You can force a MAP_SHARED mapping without any changes to qemu by using 
the -mem_prealloc option, but you'll get MAP_POPULATE as well, which may 
not be desirable. A small patch would do the job though.

2) exec.c:file_ram_alloc() assumes you're allocating off a hugetlbfs and 
makes some system calls to get the block/hugepage size. A quick hack 
might be to comment out the following in exec.c:gethugepagesize():

if (fs.f_type != HUGETLBFS_MAGIC)
             fprintf(stderr, "Warning: path not on HugeTLBFS: %s\n", path);

You may also want to replace the mkstemp() with a mkostemp() and set 
O_SYNC on the file

But beyond hacks, I think generalizing -mempath might have some other 
useful applications (using it as a way to expose tmpfs-backed/numactl'd 
files as numa nodes to guests came up in an earlier discussion, and 
memory compression via zram/compcache is another).