[Qemu-devel] [PATCH 0/2] Migration/postcopy disallow huge pages

[Qemu-devel] [PATCH 0/2] Migration/postcopy disallow huge pages

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

We've not got support for huge pages in postcopy yet, and it
will error when it tries to use the userfaultfd on the hugepage,
however the kernel is going to gain support for userfault
so make sure we fail with a nice error on the new kernel
until we figure out how to add support.

Dave

Dr. David Alan Gilbert (2):
  RAMBlocks: Store page size
  migration/postcopy: Explicitly disallow huge pages

 exec.c                    | 17 +++++++++++------
 include/exec/cpu-common.h |  1 +
 include/exec/ram_addr.h   |  1 +
 migration/postcopy-ram.c  | 23 +++++++++++++++++++++++
 4 files changed, 36 insertions(+), 6 deletions(-)

-- 
2.7.4

[Qemu-devel] [PATCH 1/2] RAMBlocks: Store page size

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

Store the page size in each RAMBlock, we need it later.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 exec.c                    | 17 +++++++++++------
 include/exec/cpu-common.h |  1 +
 include/exec/ram_addr.h   |  1 +
 3 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index c8389f9..e111323 100644
--- a/exec.c
+++ b/exec.c
@@ -1201,7 +1201,6 @@ static void *file_ram_alloc(RAMBlock *block,
     char *c;
     void *area = MAP_FAILED;
     int fd = -1;
-    int64_t page_size;
 
     if (kvm_enabled() && !kvm_has_sync_mmu()) {
         error_setg(errp,
@@ -1256,17 +1255,17 @@ static void *file_ram_alloc(RAMBlock *block,
          */
     }
 
-    page_size = qemu_fd_getpagesize(fd);
-    block->mr->align = MAX(page_size, QEMU_VMALLOC_ALIGN);
+    block->page_size = qemu_fd_getpagesize(fd);
+    block->mr->align = MAX(block->page_size, QEMU_VMALLOC_ALIGN);
 
-    if (memory < page_size) {
+    if (memory < block->page_size) {
         error_setg(errp, "memory size 0x" RAM_ADDR_FMT " must be equal to "
                    "or larger than page size 0x%" PRIx64,
-                   memory, page_size);
+                   memory, block->page_size);
         goto error;
     }
 
-    memory = ROUND_UP(memory, page_size);
+    memory = ROUND_UP(memory, block->page_size);
 
     /*
      * ftruncate is not supported by hugetlbfs in older
@@ -1421,6 +1420,11 @@ void qemu_ram_unset_idstr(RAMBlock *block)
     }
 }
 
+size_t qemu_ram_pagesize(RAMBlock *rb)
+{
+    return rb->page_size;
+}
+
 static int memory_try_enable_merging(void *addr, size_t len)
 {
     if (!machine_mem_merge(current_machine)) {
@@ -1660,6 +1664,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     new_block->max_length = max_size;
     assert(max_size >= size);
     new_block->fd = -1;
+    new_block->page_size = getpagesize();
     new_block->host = host;
     if (host) {
         new_block->flags |= RAM_PREALLOC;
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index 869ba41..cffdc13 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -63,6 +63,7 @@ RAMBlock *qemu_ram_block_from_host(void *ptr, bool round_offset,
 void qemu_ram_set_idstr(RAMBlock *block, const char *name, DeviceState *dev);
 void qemu_ram_unset_idstr(RAMBlock *block);
 const char *qemu_ram_get_idstr(RAMBlock *rb);
+size_t qemu_ram_pagesize(RAMBlock *block);
 
 void cpu_physical_memory_rw(hwaddr addr, uint8_t *buf,
                             int len, int is_write);
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index 2a9465d..54d7108 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -36,6 +36,7 @@ struct RAMBlock {
     /* RCU-enabled, writes protected by the ramlist lock */
     QLIST_ENTRY(RAMBlock) next;
     int fd;
+    size_t page_size;
 };
 
 static inline bool offset_in_ramblock(RAMBlock *b, ram_addr_t offset)
-- 
2.7.4

[Qemu-devel] [PATCH 2/2] migration/postcopy: Explicitly disallow huge pages

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

At the moment postcopy will fail as soon as qemu tries to register
userfault on the RAMBlock pages that are backed by hugepages.
However, the kernel is going to get userfault support for hugepage
at some point, and we've not got the rest of the QEMU code to support
it yet, so fail neatly with an error like:

Postcopy doesn't support hugetlbfs yet (/objects/mem1)

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/postcopy-ram.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 9b04778..9723593 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -85,6 +85,23 @@ static bool ufd_version_check(int ufd)
 }
 
 /*
+ * Check for things that postcopy won't support; returns 0 if the block
+ * is fine.
+ */
+static int check_range(const char *block_name, void *host_addr,
+                      ram_addr_t offset, ram_addr_t length, void *opaque)
+{
+    RAMBlock *rb = qemu_ram_block_by_name(block_name);
+
+    if (qemu_ram_pagesize(rb) > getpagesize()) {
+        error_report("Postcopy doesn't support hugetlbfs yet (%s)", block_name);
+        return -E2BIG;
+    }
+
+    return 0;
+}
+
+/*
  * Note: This has the side effect of munlock'ing all of RAM, that's
  * normally fine since if the postcopy succeeds it gets turned back on at the
  * end.
@@ -104,6 +121,12 @@ bool postcopy_ram_supported_by_host(void)
         goto out;
     }
 
+    /* Check for anything about the RAMBlocks we don't support */
+    if (qemu_ram_foreach_block(check_range, NULL)) {
+        /* check_range will have printed its own error */
+        goto out;
+    }
+
     ufd = syscall(__NR_userfaultfd, O_CLOEXEC);
     if (ufd == -1) {
         error_report("%s: userfaultfd not available: %s", __func__,
-- 
2.7.4

Re: [Qemu-devel] [PATCH 2/2] migration/postcopy: Explicitly disallow huge pages

[Daniel P. Berrange]

On Thu, Sep 29, 2016 at 08:09:38PM +0100, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> At the moment postcopy will fail as soon as qemu tries to register
> userfault on the RAMBlock pages that are backed by hugepages.
> However, the kernel is going to get userfault support for hugepage
> at some point, and we've not got the rest of the QEMU code to support
> it yet, so fail neatly with an error like:
> 
> Postcopy doesn't support hugetlbfs yet (/objects/mem1)
> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
>  migration/postcopy-ram.c | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
> index 9b04778..9723593 100644
> --- a/migration/postcopy-ram.c
> +++ b/migration/postcopy-ram.c
> @@ -85,6 +85,23 @@ static bool ufd_version_check(int ufd)
>  }
>  
>  /*
> + * Check for things that postcopy won't support; returns 0 if the block
> + * is fine.
> + */
> +static int check_range(const char *block_name, void *host_addr,
> +                      ram_addr_t offset, ram_addr_t length, void *opaque)
> +{
> +    RAMBlock *rb = qemu_ram_block_by_name(block_name);
> +
> +    if (qemu_ram_pagesize(rb) > getpagesize()) {
> +        error_report("Postcopy doesn't support hugetlbfs yet (%s)", block_name);

A small nitpick - I'd suggest  s/hugetlbfs/large page sizes/ as this error
will ultimately bubble up to users, and 'large page sizes' is the conceptual
feature


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH 2/2] migration/postcopy: Explicitly disallow huge pages

[Dr. David Alan Gilbert]

* Daniel P. Berrange (berrange@redhat.com) wrote:
> On Thu, Sep 29, 2016 at 08:09:38PM +0100, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > 
> > At the moment postcopy will fail as soon as qemu tries to register
> > userfault on the RAMBlock pages that are backed by hugepages.
> > However, the kernel is going to get userfault support for hugepage
> > at some point, and we've not got the rest of the QEMU code to support
> > it yet, so fail neatly with an error like:
> > 
> > Postcopy doesn't support hugetlbfs yet (/objects/mem1)
> > 
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > ---
> >  migration/postcopy-ram.c | 23 +++++++++++++++++++++++
> >  1 file changed, 23 insertions(+)
> > 
> > diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
> > index 9b04778..9723593 100644
> > --- a/migration/postcopy-ram.c
> > +++ b/migration/postcopy-ram.c
> > @@ -85,6 +85,23 @@ static bool ufd_version_check(int ufd)
> >  }
> >  
> >  /*
> > + * Check for things that postcopy won't support; returns 0 if the block
> > + * is fine.
> > + */
> > +static int check_range(const char *block_name, void *host_addr,
> > +                      ram_addr_t offset, ram_addr_t length, void *opaque)
> > +{
> > +    RAMBlock *rb = qemu_ram_block_by_name(block_name);
> > +
> > +    if (qemu_ram_pagesize(rb) > getpagesize()) {
> > +        error_report("Postcopy doesn't support hugetlbfs yet (%s)", block_name);
> 
> A small nitpick - I'd suggest  s/hugetlbfs/large page sizes/ as this error
> will ultimately bubble up to users, and 'large page sizes' is the conceptual
> feature

Interesting, I'd never heard it called 'large' before.
Everytime I just say 'huge page' to someone I then have to explain it's fine
on transparent huge page.

Dave

> 
> 
> Regards,
> Daniel
> -- 
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org              -o-             http://virt-manager.org :|
> |: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 2/2] migration/postcopy: Explicitly disallow huge pages

[Juan Quintela]

"Daniel P. Berrange" <berrange@redhat.com> wrote:
> On Thu, Sep 29, 2016 at 08:09:38PM +0100, Dr. David Alan Gilbert (git) wrote:
>> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>> 
>> At the moment postcopy will fail as soon as qemu tries to register
>> userfault on the RAMBlock pages that are backed by hugepages.
>> However, the kernel is going to get userfault support for hugepage
>> at some point, and we've not got the rest of the QEMU code to support
>> it yet, so fail neatly with an error like:
>> 
>> Postcopy doesn't support hugetlbfs yet (/objects/mem1)
>> 
>> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>> ---
>>  migration/postcopy-ram.c | 23 +++++++++++++++++++++++
>>  1 file changed, 23 insertions(+)
>> 
>> diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
>> index 9b04778..9723593 100644
>> --- a/migration/postcopy-ram.c
>> +++ b/migration/postcopy-ram.c
>> @@ -85,6 +85,23 @@ static bool ufd_version_check(int ufd)
>>  }
>>  
>>  /*
>> + * Check for things that postcopy won't support; returns 0 if the block
>> + * is fine.
>> + */
>> +static int check_range(const char *block_name, void *host_addr,
>> +                      ram_addr_t offset, ram_addr_t length, void *opaque)
>> +{
>> +    RAMBlock *rb = qemu_ram_block_by_name(block_name);
>> +
>> +    if (qemu_ram_pagesize(rb) > getpagesize()) {
>> +        error_report("Postcopy doesn't support hugetlbfs yet (%s)", block_name);
>
> A small nitpick - I'd suggest  s/hugetlbfs/large page sizes/ as this error
> will ultimately bubble up to users, and 'large page sizes' is the conceptual
> feature

Reviewed-by: Juan Quintela <quintela@redhat.com>

I fixed the message my hand