From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:55380) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1THVZm-0003Ok-EI for qemu-devel@nongnu.org; Fri, 28 Sep 2012 04:05:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1THVZf-0002Zv-TV for qemu-devel@nongnu.org; Fri, 28 Sep 2012 04:05:34 -0400 Received: from goliath.siemens.de ([192.35.17.28]:27522) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1THVZf-0002Z5-Jl for qemu-devel@nongnu.org; Fri, 28 Sep 2012 04:05:27 -0400 Message-ID: <50655A35.9080505@siemens.com> Date: Fri, 28 Sep 2012 10:05:09 +0200 From: Jan Kiszka MIME-Version: 1.0 References: <8631DC5930FA9E468F04F3FD3A5D00721394D3EE@USINDEM103.corp.hds.com> In-Reply-To: <8631DC5930FA9E468F04F3FD3A5D00721394D3EE@USINDEM103.corp.hds.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH] Add option to mlock guest and qemu memory List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Satoru Moriya Cc: "dle-develop@lists.sourceforge.net" , Seiji Aguchi , "qemu-devel@nongnu.org" , "avi@redhat.com" On 2012-09-28 01:21, Satoru Moriya wrote: > This is a first time for me to post a patch to qemu-devel. > If there is something missing/wrong, please let me know. > > We have some plans to migrate old enterprise systems which require > low latency (msec order) to kvm virtualized environment. Usually, > we uses mlock to preallocate and pin down process memory in order > to avoid page allocation in latency critical path. On the other > hand, in kvm environment, mlocking in guests is not effective > because it can't avoid page reclaim in host. Actually, to avoid > guest memory reclaim, qemu has "mem-path" option that is actually > for using hugepage. But a memory region of qemu is not allocated > on hugepage, so it may be reclaimed. That may cause a latency > problem. > > To avoid guest and qemu memory reclaim, this patch introduces > a new "mlock" option. With this option, we can preallocate and > pin down guest and qemu memory before booting guest OS. I guess this reduces the likeliness of multi-millisecond latencies for you but not eliminate them. Of course, mlockall is part of our local changes for real-time QEMU/KVM, but it is just one of the many pieces required. I'm wondering how the situation is on your side. I think mlockall should once be enabled automatically as soon as you ask for real-time support for QEMU guests. How that should be controlled is another question. I'm currently carrying a top-level switch "-rt maxprio=x[,policy=y]" here, likely not the final solution. I'm not really convinced we need to control memory locking separately. And as we are very reluctant to add new top-level switches, this is even more important. > > Tested on Linux, x86_64 (fedora 17). > > Signed-off-by: Satoru Moriya > --- > cpu-all.h | 1 + > exec.c | 3 +++ > qemu-options.hx | 8 ++++++++ > vl.c | 4 ++++ > 4 files changed, 16 insertions(+) > > diff --git a/cpu-all.h b/cpu-all.h > index 74d3681..e12e5d5 100644 > --- a/cpu-all.h > +++ b/cpu-all.h > @@ -503,6 +503,7 @@ extern RAMList ram_list; > > extern const char *mem_path; > extern int mem_prealloc; > +extern int mem_lock; > > /* Flags stored in the low bits of the TLB virtual address. These are > defined so that fast path ram access is all zeros. */ > diff --git a/exec.c b/exec.c > index bb6aa4a..de13bc9 100644 > --- a/exec.c > +++ b/exec.c > @@ -2572,6 +2572,9 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host, > } > memory_try_enable_merging(new_block->host, size); > } > + if (mem_lock && mlockall(MCL_CURRENT | MCL_FUTURE)) { > + perror("mlockall"); > + } This belongs to the OS abstraction layer (it's POSIX). And you only need to call it once per process lifetime. > } > new_block->length = size; > > diff --git a/qemu-options.hx b/qemu-options.hx > index 7d97f96..9d82f15 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -427,6 +427,14 @@ Preallocate memory when using -mem-path. > ETEXI > #endif > > +DEF("mlock", 0, QEMU_OPTION_mlock, > + "-mlock mlock guest and qemu memory\n", > + QEMU_ARCH_ALL) > +STEXI > +@item -mlock > +mlock guest and qemu memory. > +ETEXI > + > DEF("k", HAS_ARG, QEMU_OPTION_k, > "-k language use keyboard layout (for example 'fr' for French)\n", > QEMU_ARCH_ALL) > diff --git a/vl.c b/vl.c > index 8d305ca..c902084 100644 > --- a/vl.c > +++ b/vl.c > @@ -187,6 +187,7 @@ const char *mem_path = NULL; > #ifdef MAP_POPULATE > int mem_prealloc = 0; /* force preallocation of physical target memory */ > #endif > +int mem_lock; > int nb_nics; > NICInfo nd_table[MAX_NICS]; > int autostart; > @@ -2770,6 +2771,9 @@ int main(int argc, char **argv, char **envp) > mem_prealloc = 1; > break; > #endif > + case QEMU_OPTION_mlock: > + mem_lock = 1; > + break; > case QEMU_OPTION_d: > log_mask = optarg; > break; > Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux