Re: [Qemu-devel] [PATCH] Add option to mlock guest and qemu memory

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2012-09-28 01:21, Satoru Moriya wrote:
> This is a first time for me to post a patch to qemu-devel.
> If there is something missing/wrong, please let me know.
> 
> We have some plans to migrate old enterprise systems which require
> low latency (msec order) to kvm virtualized environment. Usually,
> we uses mlock to preallocate and pin down process memory in order
> to avoid page allocation in latency critical path. On the other
> hand, in kvm environment, mlocking in guests is not effective
> because it can't avoid page reclaim in host. Actually, to avoid
> guest memory reclaim, qemu has "mem-path" option that is actually
> for using hugepage. But a memory region of qemu is not allocated
> on hugepage, so it may be reclaimed. That may cause a latency
> problem.
> 
> To avoid guest and qemu memory reclaim, this patch introduces
> a new "mlock" option. With this option, we can preallocate and
> pin down guest and qemu memory before booting guest OS.

I guess this reduces the likeliness of multi-millisecond latencies for
you but not eliminate them. Of course, mlockall is part of our local
changes for real-time QEMU/KVM, but it is just one of the many pieces
required. I'm wondering how the situation is on your side.

I think mlockall should once be enabled automatically as soon as you ask
for real-time support for QEMU guests. How that should be controlled is
another question. I'm currently carrying a top-level switch "-rt
maxprio=x[,policy=y]" here, likely not the final solution. I'm not
really convinced we need to control memory locking separately. And as we
are very reluctant to add new top-level switches, this is even more
important.

> 
> Tested on Linux, x86_64 (fedora 17).
> 
> Signed-off-by: Satoru Moriya <satoru.moriya@hds.com>
> ---
>  cpu-all.h       | 1 +
>  exec.c          | 3 +++
>  qemu-options.hx | 8 ++++++++
>  vl.c            | 4 ++++
>  4 files changed, 16 insertions(+)
> 
> diff --git a/cpu-all.h b/cpu-all.h
> index 74d3681..e12e5d5 100644
> --- a/cpu-all.h
> +++ b/cpu-all.h
> @@ -503,6 +503,7 @@ extern RAMList ram_list;
>  
>  extern const char *mem_path;
>  extern int mem_prealloc;
> +extern int mem_lock;
>  
>  /* Flags stored in the low bits of the TLB virtual address.  These are
>     defined so that fast path ram access is all zeros.  */
> diff --git a/exec.c b/exec.c
> index bb6aa4a..de13bc9 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -2572,6 +2572,9 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
>              }
>              memory_try_enable_merging(new_block->host, size);
>          }
> +        if (mem_lock && mlockall(MCL_CURRENT | MCL_FUTURE)) {
> +            perror("mlockall");
> +        }

This belongs to the OS abstraction layer (it's POSIX). And you only need
to call it once per process lifetime.

>      }
>      new_block->length = size;
>  
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 7d97f96..9d82f15 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -427,6 +427,14 @@ Preallocate memory when using -mem-path.
>  ETEXI
>  #endif
>  
> +DEF("mlock", 0, QEMU_OPTION_mlock,
> +    "-mlock          mlock guest and qemu memory\n",
> +    QEMU_ARCH_ALL)
> +STEXI
> +@item -mlock
> +mlock guest and qemu memory.
> +ETEXI
> +
>  DEF("k", HAS_ARG, QEMU_OPTION_k,
>      "-k language     use keyboard layout (for example 'fr' for French)\n",
>      QEMU_ARCH_ALL)
> diff --git a/vl.c b/vl.c
> index 8d305ca..c902084 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -187,6 +187,7 @@ const char *mem_path = NULL;
>  #ifdef MAP_POPULATE
>  int mem_prealloc = 0; /* force preallocation of physical target memory */
>  #endif
> +int mem_lock;
>  int nb_nics;
>  NICInfo nd_table[MAX_NICS];
>  int autostart;
> @@ -2770,6 +2771,9 @@ int main(int argc, char **argv, char **envp)
>                  mem_prealloc = 1;
>                  break;
>  #endif
> +            case QEMU_OPTION_mlock:
> +                mem_lock = 1;
> +                break;
>              case QEMU_OPTION_d:
>                  log_mask = optarg;
>                  break;
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux