[Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexey Kardashevskiy]

At the moment if the user asked for huge pages and there is no more huge
pages, QEMU prints warning and falls back to the anonymous memory
allocator which is quite easy not to notice. QEMU also does so even
if the user specified -mem-prealloc and it seems wrong as the user
specifically requested huge pages for the entire RAM but QEMU failed to do
so and continued. On PPC64 this will produce a fragile guest as QEMU
tells the guest via device-tree that it can use huge pages when it
actually cannot.

This adds message+exit if RAM cannot be preallocated from huge pages.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 exec.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/exec.c b/exec.c
index 9ad0a4b..deb8279 100644
--- a/exec.c
+++ b/exec.c
@@ -1041,6 +1041,10 @@ static void *file_ram_alloc(RAMBlock *block,
 
     area = mmap(0, memory, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
     if (area == MAP_FAILED) {
+        if (mem_prealloc) {
+            perror("file_ram_alloc: failed to preallocate RAM");
+            exit(1);
+        }
         perror("file_ram_alloc: can't mmap RAM pages");
         close(fd);
         return (NULL);
-- 
1.8.4.rc4

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexey Kardashevskiy]

On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
> At the moment if the user asked for huge pages and there is no more huge
> pages, QEMU prints warning and falls back to the anonymous memory
> allocator which is quite easy not to notice. QEMU also does so even
> if the user specified -mem-prealloc and it seems wrong as the user
> specifically requested huge pages for the entire RAM but QEMU failed to do
> so and continued. On PPC64 this will produce a fragile guest as QEMU
> tells the guest via device-tree that it can use huge pages when it
> actually cannot.
> 
> This adds message+exit if RAM cannot be preallocated from huge pages.


Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
avoid annoying the community? :) Thanks!



> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
>  exec.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/exec.c b/exec.c
> index 9ad0a4b..deb8279 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1041,6 +1041,10 @@ static void *file_ram_alloc(RAMBlock *block,
>  
>      area = mmap(0, memory, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
>      if (area == MAP_FAILED) {
> +        if (mem_prealloc) {
> +            perror("file_ram_alloc: failed to preallocate RAM");
> +            exit(1);
> +        }
>          perror("file_ram_alloc: can't mmap RAM pages");
>          close(fd);
>          return (NULL);
> 


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexander Graf]

On 21.02.2014, at 05:57, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
>> At the moment if the user asked for huge pages and there is no more huge
>> pages, QEMU prints warning and falls back to the anonymous memory
>> allocator which is quite easy not to notice. QEMU also does so even
>> if the user specified -mem-prealloc and it seems wrong as the user
>> specifically requested huge pages for the entire RAM but QEMU failed to do
>> so and continued. On PPC64 this will produce a fragile guest as QEMU
>> tells the guest via device-tree that it can use huge pages when it
>> actually cannot.
>> 
>> This adds message+exit if RAM cannot be preallocated from huge pages.
> 
> 
> Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
> avoid annoying the community? :) Thanks!

The patch changes the semantics of -mem-prealloc from "make sure all RAM is mapped" to "make sure all RAM is mapped and is backed by huge pages if we use huge pages" and thus is just plain wrong.

The real question is why are we allowing sparsely mapped huge page backing at all? Should we change that? Do we need a new flag for this to specify "yes, I do want all my pages backed by -mem-path"?

This is also something that should be coordinated with the -mem-path refactoring.


Alex

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexey Kardashevskiy]

On 02/21/2014 07:57 PM, Alexander Graf wrote:
> 
> On 21.02.2014, at 05:57, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
>>> At the moment if the user asked for huge pages and there is no more huge
>>> pages, QEMU prints warning and falls back to the anonymous memory
>>> allocator which is quite easy not to notice. QEMU also does so even
>>> if the user specified -mem-prealloc and it seems wrong as the user
>>> specifically requested huge pages for the entire RAM but QEMU failed to do
>>> so and continued. On PPC64 this will produce a fragile guest as QEMU
>>> tells the guest via device-tree that it can use huge pages when it
>>> actually cannot.
>>>
>>> This adds message+exit if RAM cannot be preallocated from huge pages.
>>
>>
>> Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
>> avoid annoying the community? :) Thanks!
> 

> The patch changes the semantics of -mem-prealloc from "make sure all
> RAM is mapped" to "make sure all RAM is mapped and is backed by huge
> pages if we use huge pages" and thus is just plain wrong.

? I did actually expect it to alloc RAM from hugepages only. Otherwise
there is no point in mem-prealloc. Yes, I am ignorant, I know.

> The real question is why are we allowing sparsely mapped huge page
backing at all? Should we change that? Do we need a new flag for this to
specify "yes, I do want all my pages backed by -mem-path"?


? Add a switch to -mem-path saying "yes I really want -mem-path"? Sorry, I
lost you here. -mem-path + -mem-prealloc - like this is not enough? Why
would I specify -mem-path after all if I did not want RAM to backed by huge
pages?


> This is also something that should be coordinated with the -mem-path refactoring.

Oh. There is always refactoring :(



-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexander Graf]

On 21.02.2014, at 13:56, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 02/21/2014 07:57 PM, Alexander Graf wrote:
>> 
>> On 21.02.2014, at 05:57, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>> 
>>> On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
>>>> At the moment if the user asked for huge pages and there is no more huge
>>>> pages, QEMU prints warning and falls back to the anonymous memory
>>>> allocator which is quite easy not to notice. QEMU also does so even
>>>> if the user specified -mem-prealloc and it seems wrong as the user
>>>> specifically requested huge pages for the entire RAM but QEMU failed to do
>>>> so and continued. On PPC64 this will produce a fragile guest as QEMU
>>>> tells the guest via device-tree that it can use huge pages when it
>>>> actually cannot.
>>>> 
>>>> This adds message+exit if RAM cannot be preallocated from huge pages.
>>> 
>>> 
>>> Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
>>> avoid annoying the community? :) Thanks!
>> 
> 
>> The patch changes the semantics of -mem-prealloc from "make sure all
>> RAM is mapped" to "make sure all RAM is mapped and is backed by huge
>> pages if we use huge pages" and thus is just plain wrong.
> 
> ? I did actually expect it to alloc RAM from hugepages only. Otherwise
> there is no point in mem-prealloc. Yes, I am ignorant, I know.
> 
>> The real question is why are we allowing sparsely mapped huge page
> backing at all? Should we change that? Do we need a new flag for this to
> specify "yes, I do want all my pages backed by -mem-path"?
> 
> 
> ? Add a switch to -mem-path saying "yes I really want -mem-path"? Sorry, I
> lost you here. -mem-path + -mem-prealloc - like this is not enough? Why
> would I specify -mem-path after all if I did not want RAM to backed by huge
> pages?

I think it makes sense to disable any fallback for -mem-path, so that it always only allocates RAM pages from the -mem-path pool. But this is a big change from how it used to work before and thus needs to be properly coordinated.

Paolo, Peter, any thoughts here? Version 2.0 might be a good fit for such a change ;).


Alex

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexey Kardashevskiy]

On 02/22/2014 12:02 AM, Alexander Graf wrote:
> 
> On 21.02.2014, at 13:56, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 02/21/2014 07:57 PM, Alexander Graf wrote:
>>>
>>> On 21.02.2014, at 05:57, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>
>>>> On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
>>>>> At the moment if the user asked for huge pages and there is no more huge
>>>>> pages, QEMU prints warning and falls back to the anonymous memory
>>>>> allocator which is quite easy not to notice. QEMU also does so even
>>>>> if the user specified -mem-prealloc and it seems wrong as the user
>>>>> specifically requested huge pages for the entire RAM but QEMU failed to do
>>>>> so and continued. On PPC64 this will produce a fragile guest as QEMU
>>>>> tells the guest via device-tree that it can use huge pages when it
>>>>> actually cannot.
>>>>>
>>>>> This adds message+exit if RAM cannot be preallocated from huge pages.
>>>>
>>>>
>>>> Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
>>>> avoid annoying the community? :) Thanks!
>>>
>>
>>> The patch changes the semantics of -mem-prealloc from "make sure all
>>> RAM is mapped" to "make sure all RAM is mapped and is backed by huge
>>> pages if we use huge pages" and thus is just plain wrong.
>>
>> ? I did actually expect it to alloc RAM from hugepages only. Otherwise
>> there is no point in mem-prealloc. Yes, I am ignorant, I know.
>>
>>> The real question is why are we allowing sparsely mapped huge page
>> backing at all? Should we change that? Do we need a new flag for this to
>> specify "yes, I do want all my pages backed by -mem-path"?
>>
>>
>> ? Add a switch to -mem-path saying "yes I really want -mem-path"? Sorry, I
>> lost you here. -mem-path + -mem-prealloc - like this is not enough? Why
>> would I specify -mem-path after all if I did not want RAM to backed by huge
>> pages?
> 

> I think it makes sense to disable any fallback for -mem-path, so that it
> always only allocates RAM pages from the -mem-path pool. But this is a
> big change from how it used to work before and thus needs to be properly
> coordinated.

ROMs, BARs - this all will stop working if I understand things right. And
we (ozlabs) do not really want these things to be in huge pages.

> 
> Paolo, Peter, any thoughts here? Version 2.0 might be a good fit for such a change ;).
> 
> 
> Alex
> 


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexander Graf]

On 21.02.2014, at 14:04, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 02/22/2014 12:02 AM, Alexander Graf wrote:
>> 
>> On 21.02.2014, at 13:56, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>> 
>>> On 02/21/2014 07:57 PM, Alexander Graf wrote:
>>>> 
>>>> On 21.02.2014, at 05:57, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>> 
>>>>> On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
>>>>>> At the moment if the user asked for huge pages and there is no more huge
>>>>>> pages, QEMU prints warning and falls back to the anonymous memory
>>>>>> allocator which is quite easy not to notice. QEMU also does so even
>>>>>> if the user specified -mem-prealloc and it seems wrong as the user
>>>>>> specifically requested huge pages for the entire RAM but QEMU failed to do
>>>>>> so and continued. On PPC64 this will produce a fragile guest as QEMU
>>>>>> tells the guest via device-tree that it can use huge pages when it
>>>>>> actually cannot.
>>>>>> 
>>>>>> This adds message+exit if RAM cannot be preallocated from huge pages.
>>>>> 
>>>>> 
>>>>> Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
>>>>> avoid annoying the community? :) Thanks!
>>>> 
>>> 
>>>> The patch changes the semantics of -mem-prealloc from "make sure all
>>>> RAM is mapped" to "make sure all RAM is mapped and is backed by huge
>>>> pages if we use huge pages" and thus is just plain wrong.
>>> 
>>> ? I did actually expect it to alloc RAM from hugepages only. Otherwise
>>> there is no point in mem-prealloc. Yes, I am ignorant, I know.
>>> 
>>>> The real question is why are we allowing sparsely mapped huge page
>>> backing at all? Should we change that? Do we need a new flag for this to
>>> specify "yes, I do want all my pages backed by -mem-path"?
>>> 
>>> 
>>> ? Add a switch to -mem-path saying "yes I really want -mem-path"? Sorry, I
>>> lost you here. -mem-path + -mem-prealloc - like this is not enough? Why
>>> would I specify -mem-path after all if I did not want RAM to backed by huge
>>> pages?
>> 
> 
>> I think it makes sense to disable any fallback for -mem-path, so that it
>> always only allocates RAM pages from the -mem-path pool. But this is a
>> big change from how it used to work before and thus needs to be properly
>> coordinated.
> 
> ROMs, BARs - this all will stop working if I understand things right. And
> we (ozlabs) do not really want these things to be in huge pages.

Only if they're backed by virtual memory. And in that case why don't you want them be huge pages? What qualifies a region to be huge vs non-huge?


Alex

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Alexey Kardashevskiy]

On 02/22/2014 12:10 AM, Alexander Graf wrote:
> 
> On 21.02.2014, at 14:04, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 02/22/2014 12:02 AM, Alexander Graf wrote:
>>>
>>> On 21.02.2014, at 13:56, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>
>>>> On 02/21/2014 07:57 PM, Alexander Graf wrote:
>>>>>
>>>>> On 21.02.2014, at 05:57, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>>>
>>>>>> On 02/10/2014 05:32 PM, Alexey Kardashevskiy wrote:
>>>>>>> At the moment if the user asked for huge pages and there is no more huge
>>>>>>> pages, QEMU prints warning and falls back to the anonymous memory
>>>>>>> allocator which is quite easy not to notice. QEMU also does so even
>>>>>>> if the user specified -mem-prealloc and it seems wrong as the user
>>>>>>> specifically requested huge pages for the entire RAM but QEMU failed to do
>>>>>>> so and continued. On PPC64 this will produce a fragile guest as QEMU
>>>>>>> tells the guest via device-tree that it can use huge pages when it
>>>>>>> actually cannot.
>>>>>>>
>>>>>>> This adds message+exit if RAM cannot be preallocated from huge pages.
>>>>>>
>>>>>>
>>>>>> Too bad? Should I increase my personal pinging timeout from 1 to 2 weeks to
>>>>>> avoid annoying the community? :) Thanks!
>>>>>
>>>>
>>>>> The patch changes the semantics of -mem-prealloc from "make sure all
>>>>> RAM is mapped" to "make sure all RAM is mapped and is backed by huge
>>>>> pages if we use huge pages" and thus is just plain wrong.
>>>>
>>>> ? I did actually expect it to alloc RAM from hugepages only. Otherwise
>>>> there is no point in mem-prealloc. Yes, I am ignorant, I know.
>>>>
>>>>> The real question is why are we allowing sparsely mapped huge page
>>>> backing at all? Should we change that? Do we need a new flag for this to
>>>> specify "yes, I do want all my pages backed by -mem-path"?
>>>>
>>>>
>>>> ? Add a switch to -mem-path saying "yes I really want -mem-path"? Sorry, I
>>>> lost you here. -mem-path + -mem-prealloc - like this is not enough? Why
>>>> would I specify -mem-path after all if I did not want RAM to backed by huge
>>>> pages?
>>>
>>
>>> I think it makes sense to disable any fallback for -mem-path, so that it
>>> always only allocates RAM pages from the -mem-path pool. But this is a
>>> big change from how it used to work before and thus needs to be properly
>>> coordinated.
>>
>> ROMs, BARs - this all will stop working if I understand things right. And
>> we (ozlabs) do not really want these things to be in huge pages.
> 

> Only if they're backed by virtual memory. And in that case why don't you
> want them be huge pages? What qualifies a region to be huge vs
> non-huge?

This just adds complication for no reason. If we disable small pages with
-mam-path, we'll have to teach SLOF and our PCI hotplug code to align BARs
and for what? HV KVM does not need this to function.


-- 
Alexey

Re: [Qemu-devel] [RFC PATCH] file ram alloc: fail if cannot preallocate

[Paolo Bonzini]

Il 21/02/2014 14:02, Alexander Graf ha scritto:
> I think it makes sense to disable any fallback for -mem-path, so that
> it always only allocates RAM pages from the -mem-path pool. But this
> is a big change from how it used to work before and thus needs to be
> properly coordinated.
>
> Paolo, Peter, any thoughts here? Version 2.0 might be a good fit for
> such a change ;).

I agree it would be nice, but as Alexey said it would be best to only 
require hugepage-backed memory for the bulk of RAM.  So I'd prefer to 
wait for memdev, and then we can just deprecate -mem-path and -mem-prealloc.

Paolo