balloon question

balloon question

[Jan Beulich]

Running with Xen option dom0_mem=6G and kernel option mem=8G, I would have expected that I would see the system with a
2G balloon right after boot. However, the balloon is empty. Briefly looking at this, it would seem to me that, with all
pages being reserved when the system starts and only the populated ones getting the reserved status cleared in
__free_pages_bootmem(), the condition to put pages into the balloon in balloon_init() is inverted.

Also, dying in xen_destroy_contiguous_region() (and similarly in the failure recovery code in
xen_create_contiguous_region()) if the space just unmapped can't be re-populated seems overly harsh (we're having a case
where this actually happens); wouldn't it make sense to make this fatal only when the page can't be replaced by a free
one (allocated inside the domain, with the page stolen for this then simply pushed into the balloon, similar to
balloon_alloc_empty_page_range() except without freeing the memory to Xen)?

Thanks for clarification,
Jan

Re: balloon question

[Keir Fraser]

On 8 Jun 2006, at 14:51, Jan Beulich wrote:

> Running with Xen option dom0_mem=6G and kernel option mem=8G, I would 
> have expected that I would see the system with a
> 2G balloon right after boot. However, the balloon is empty. Briefly 
> looking at this, it would seem to me that, with all
> pages being reserved when the system starts and only the populated 
> ones getting the reserved status cleared in
> __free_pages_bootmem(), the condition to put pages into the balloon in 
> balloon_init() is inverted.

i386 deliberately marks pages beyond start_info->nr_pages as not 
Reserved so they get picked up by the balloon driver. Either x86/64 
needs to do the same, or we stop doing that in i386. But then there 
will be pages that are legitimately Reserved that the balloon driver 
will erroneously pick up. Or we could make the balloon driver 
explicitly pick up all pages from start_info->nr_pages. Or have an 
interface to allow arch code to register page ranges with the balloon 
driver. This needs a little thought....

> Also, dying in xen_destroy_contiguous_region() (and similarly in the 
> failure recovery code in
> xen_create_contiguous_region()) if the space just unmapped can't be 
> re-populated seems overly harsh (we're having a case
> where this actually happens); wouldn't it make sense to make this 
> fatal only when the page can't be replaced by a free
> one (allocated inside the domain, with the page stolen for this then 
> simply pushed into the balloon, similar to
> balloon_alloc_empty_page_range() except without freeing the memory to 
> Xen)?

As it happens I'm fixing this exact problem right now. :-) Expect a fix 
in -unstable and -testing in the next few days.

  -- Keir

Re: balloon question

[Jan Beulich]

>>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> 08.06.06 18:25 >>>
>
>On 8 Jun 2006, at 14:51, Jan Beulich wrote:
>
>> Running with Xen option dom0_mem=6G and kernel option mem=8G, I would 
>> have expected that I would see the system with a
>> 2G balloon right after boot. However, the balloon is empty. Briefly 
>> looking at this, it would seem to me that, with all
>> pages being reserved when the system starts and only the populated 
>> ones getting the reserved status cleared in
>> __free_pages_bootmem(), the condition to put pages into the balloon in 
>> balloon_init() is inverted.

>i386 deliberately marks pages beyond start_info->nr_pages as not 
>Reserved so they get picked up by the balloon driver. Either x86/64 
>needs to do the same, or we stop doing that in i386. But then there 
>will be pages that are legitimately Reserved that the balloon driver 

Where would, under Xen, such pages originate from? Namely beyond
start_info->nr_pages?

>will erroneously pick up. Or we could make the balloon driver 
>explicitly pick up all pages from start_info->nr_pages. Or have an 

It is already starting its scan at start_info->nr_pages.

>interface to allow arch code to register page ranges with the balloon 
>driver. This needs a little thought....
>
>> Also, dying in xen_destroy_contiguous_region() (and similarly in the 
>> failure recovery code in
>> xen_create_contiguous_region()) if the space just unmapped can't be 
>> re-populated seems overly harsh (we're having a case
>> where this actually happens); wouldn't it make sense to make this 
>> fatal only when the page can't be replaced by a free
>> one (allocated inside the domain, with the page stolen for this then 
>> simply pushed into the balloon, similar to
>> balloon_alloc_empty_page_range() except without freeing the memory to 
>> Xen)?
>
>As it happens I'm fixing this exact problem right now. :-) Expect a fix 
>in -unstable and -testing in the next few days.

Great!

Jan

Re: balloon question

[Keir Fraser]

On 9 Jun 2006, at 08:00, Jan Beulich wrote:

>> i386 deliberately marks pages beyond start_info->nr_pages as not
>> Reserved so they get picked up by the balloon driver. Either x86/64
>> needs to do the same, or we stop doing that in i386. But then there
>> will be pages that are legitimately Reserved that the balloon driver
>
> Where would, under Xen, such pages originate from? Namely beyond
> start_info->nr_pages?
>
>> will erroneously pick up. Or we could make the balloon driver
>> explicitly pick up all pages from start_info->nr_pages. Or have an
>
> It is already starting its scan at start_info->nr_pages.

I don't think there will currently be anything beyond ->nr_pages. So we 
could just get rid of the PageReserved test altogether.

But I think it's prefereable to take the following loop from 
i386/mm/init-xen.c and put it in x86/64's mem_init() somewhere:
  for (i = ...->nr_pages; i < max_pfn; i++) {
      ClearPageReserved(...);
      set_page_count(..., 1);
   }

That'll allow any architecture to reserve stuff beyond ->nr_pages 
without breaking the balloon driver. Could you try adding the above 
code and see how it works out?

  -- Keir

Re: balloon question

[Jan Beulich]

[-- Attachment #1: Type: text/plain, Size: 761 bytes --]

>But I think it's prefereable to take the following loop from 
>i386/mm/init-xen.c and put it in x86/64's mem_init() somewhere:
>  for (i = ...->nr_pages; i < max_pfn; i++) {
>      ClearPageReserved(...);
>      set_page_count(..., 1);
>   }

Did you deliberately leave out the increment of totalram_pages that is in
i386's respective loop? I have to admit that I can't see why i386 is doing
that, but for symmetry I added it to x86-64's loop, too (balloon_init()
re-writes the variable anyway).

>That'll allow any architecture to reserve stuff beyond ->nr_pages 
>without breaking the balloon driver. Could you try adding the above 
>code and see how it works out?

Works out as expected (i.e. balloon now contains all the extra pages).
Patch attached.

Jan

[-- Attachment #2: xenlinux-x86_64-unreserve-extra-pages.patch --]
[-- Type: text/plain, Size: 960 bytes --]

Index: head-2006-06-07/arch/x86_64/mm/init-xen.c
===================================================================
--- head-2006-06-07.orig/arch/x86_64/mm/init-xen.c	2006-06-09 10:16:34.000000000 +0200
+++ head-2006-06-07/arch/x86_64/mm/init-xen.c	2006-06-09 10:25:03.000000000 +0200
@@ -882,6 +882,7 @@ static struct kcore_list kcore_mem, kcor
 void __init mem_init(void)
 {
 	long codesize, reservedpages, datasize, initsize;
+	unsigned long pfn;
 
 	contiguous_bitmap = alloc_bootmem_low_pages(
 		(end_pfn + 2*BITS_PER_LONG) >> 3);
@@ -910,6 +911,12 @@ void __init mem_init(void)
 #else
 	totalram_pages = free_all_bootmem();
 #endif
+	/* XEN: init and count pages outside initial allocation. */
+	for (pfn = xen_start_info->nr_pages; pfn < max_pfn; pfn++) {
+		ClearPageReserved(&mem_map[pfn]);
+		set_page_count(&mem_map[pfn], 1);
+		totalram_pages++;
+	}
 	reservedpages = end_pfn - totalram_pages - e820_hole_size(0, end_pfn);
 
 	after_bootmem = 1;

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: balloon question

[Keir Fraser]

On 9 Jun 2006, at 10:52, Jan Beulich wrote:

>> But I think it's prefereable to take the following loop from
>> i386/mm/init-xen.c and put it in x86/64's mem_init() somewhere:
>>  for (i = ...->nr_pages; i < max_pfn; i++) {
>>      ClearPageReserved(...);
>>      set_page_count(..., 1);
>>   }
>
> Did you deliberately leave out the increment of totalram_pages that is 
> in
> i386's respective loop? I have to admit that I can't see why i386 is 
> doing
> that, but for symmetry I added it to x86-64's loop, too (balloon_init()
> re-writes the variable anyway).

The increment is bogus. Gets overwritten by the balloon driver anyway. 
I'll remove it from your patch and fix i386 too.

Thanks!

  -- Keir

balloon question

[Zhang, Chunyu]

hi all

i have two question about ballon.

1. 
cfg:
     memory=4096
     maxmem=4096

no pod was set , no balloon is set .
when xl create, i find function decrease_reservation is call.
i did not why ?

2.  pod + balloon
cfg:
     memory=512
     maxmem=4096

 decrease_reservation
    - p2m_pod_decrease_reservation
        - p2m_set_entry(p2m_invalid)
    - guest_remove_page
         - p2m_set_entry(p2m_invalid)

in p2m_pod_decrease_reservation ,  gfn entry is set p2c_invalid.
but in guest_remove_page gfn entry is set p2c_invalid again.
i did not know  why?

function  guest_remove_page  can delete  ?


--------------
--------------
A new email address of FJWAN is launched from Apr.1 2007.
The updated address is: zhangcy@cn.fujitsu.com
--------------------------------------------------
Zhang Chunyu
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No. 6 Wenzhu Road, Nanjing, 210012, China
TEL：+86+25-86630566-9566
FUJITSU INTERNAL：7998-9566
FAX：+86+25-83317685
EMail：zhangcy@cn.fujitsu.com
--------------------------------------------------
This communication is for use by the intended recipient(s) only and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not an intended recipient of this communication, you are hereby notified that any dissemination, distribution or copying hereof is strictly prohibited.  If you have received this communication in error, please notify me by reply e-mail, permanently delete this communication from your system, and destroy any hard copies you may have printed.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: balloon question

[George Dunlap]

Please don't cross-post unless you need to.  These are coding
questions, so xen-devel is the right place to ask this.

On Thu, Apr 28, 2016 at 7:58 AM, Zhang, Chunyu <zhangcy@cn.fujitsu.com> wrote:
>
> hi all
>
> i have two question about ballon.
>
> 1.
> cfg:
>      memory=4096
>      maxmem=4096
>
> no pod was set , no balloon is set .
> when xl create, i find function decrease_reservation is call.
> i did not why ?

decrease_reservation is never initiated by the hypervisor -- it's
normally initiated by the guest balloon driver; although I think it
may also be initiated by other components, like hvmloader or qemu (not
sure about that).  You'd have to figure out which component was making
that hypercall, and then look in the logic to find out why it was
doing so.

>
> 2.  pod + balloon
> cfg:
>      memory=512
>      maxmem=4096
>
>  decrease_reservation
>     - p2m_pod_decrease_reservation
>         - p2m_set_entry(p2m_invalid)
>     - guest_remove_page
>          - p2m_set_entry(p2m_invalid)
>
> in p2m_pod_decrease_reservation ,  gfn entry is set p2c_invalid.
> but in guest_remove_page gfn entry is set p2c_invalid again.
> i did not know  why?

Did you not notice the "continue" between
p2m_pod_decrease_reservation() and guest_remove_page()? :-)

p2m_pod_decrease_reservation() either does everything necessary (both
reclaim the page and set the p2m entry) or nothing.  It will return 0
to mean, "I didn't do anything, you still need to free the page", and
1 to mean "I did everything, no need for you to do anything more."

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: balloon question

[Zhang, Chunyu]

hi george
>Please don't cross-post unless you need to.  These are coding
>questions, so xen-devel is the right place to ask this.
>
>On Thu, Apr 28, 2016 at 7:58 AM, Zhang, Chunyu <zhangcy@cn.fujitsu.com> wrote:
>>
>> hi all
>>
>> i have two question about ballon.
>>
>> 1.
>> cfg:
>>      memory=4096
>>      maxmem=4096
>>
>> no pod was set , no balloon is set .
>> when xl create, i find function decrease_reservation is call.
>> i did not why ?
>
>decrease_reservation is never initiated by the hypervisor -- it's
>normally initiated by the guest balloon driver; although I think it
>may also be initiated by other components, like hvmloader or qemu (not
>sure about that).  You'd have to figure out which component was making
>that hypercall, and then look in the logic to find out why it was
>doing so.
got it.
i will check  it.
>
>>
>> 2.  pod + balloon
>> cfg:
>>      memory=512
>>      maxmem=4096
>>
>>  decrease_reservation
>>     - p2m_pod_decrease_reservation
>>         - p2m_set_entry(p2m_invalid)
>>     - guest_remove_page
>>          - p2m_set_entry(p2m_invalid)
>>
>> in p2m_pod_decrease_reservation ,  gfn entry is set p2c_invalid.
>> but in guest_remove_page gfn entry is set p2c_invalid again.
>> i did not know  why?
>
>Did you not notice the "continue" between
>p2m_pod_decrease_reservation() and guest_remove_page()? :-)
oh....
i know the point(⊙﹏⊙)b
>
>p2m_pod_decrease_reservation() either does everything necessary (both
>reclaim the page and set the p2m entry) or nothing.  It will return 0
>to mean, "I didn't do anything, you still need to free the page", and
>1 to mean "I did everything, no need for you to do anything more."
got it, thank u very much.
>
> -George
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel