How to determine memory size in HVM

How to determine memory size in HVM

[James Harper]

How can I tell how much memory my VM has under HVM? I want to have
something concrete to compare the 'target' value in xenstore to for
ballooning so I am reluctant to ask Windows the question. There seems to
be a nr_pages in start_info, but HVM doesn't have that...

My reason for not wanting to ask Windows how much memory is that if it
says we have 3.5G (reasonable without PAE) and xenstore says we have 4G
then I'm going to have to fudge things a bit to know whether we are
ballooning up or down etc.

I could just read the initial target value from xenstore on boot, but
even that could be changed before my drivers get to it.

Thanks

James

Re: How to determine memory size in HVM

[Keir Fraser]

Hypercall XENMEM_current_reservation.

*But* be aware I'm not sure how that interacts with e.g., virtual
framebuffer memory. Well, actually I know it will include the virtual
framebuffer memory which is not actually really part of normal guest RAM.
The question will be: does the xenstore 'target' value also include it? I
guess you'll have to compare to create a HVM guest and compare the two to
decide.

 -- Keir

On 08/06/2009 15:12, "James Harper" <james.harper@bendigoit.com.au> wrote:

> How can I tell how much memory my VM has under HVM? I want to have
> something concrete to compare the 'target' value in xenstore to for
> ballooning so I am reluctant to ask Windows the question. There seems to
> be a nr_pages in start_info, but HVM doesn't have that...
> 
> My reason for not wanting to ask Windows how much memory is that if it
> says we have 3.5G (reasonable without PAE) and xenstore says we have 4G
> then I'm going to have to fudge things a bit to know whether we are
> ballooning up or down etc.
> 
> I could just read the initial target value from xenstore on boot, but
> even that could be changed before my drivers get to it.
> 
> Thanks
> 
> James
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: How to determine memory size in HVM

[James Harper]

> 
> Hypercall XENMEM_current_reservation.
> 
> *But* be aware I'm not sure how that interacts with e.g., virtual
> framebuffer memory. Well, actually I know it will include the virtual
> framebuffer memory which is not actually really part of normal guest
RAM.
> The question will be: does the xenstore 'target' value also include
it? I
> guess you'll have to compare to create a HVM guest and compare the two
to
> decide.

Thanks for that.

Another question, if I do 'xm mem-set' (without any handling of the
memory/target value) and then reboot, the system reboots with the new
target memory value, making it impossible to add memory to the system
again (under Windows at least). The only way I can think of fixing that
is to write back to memory/target with the original value just before
rebooting, but even that could be prone to errors in case of an unclean
reboot or something... any way around this?

James


> 
>  -- Keir
> 
> On 08/06/2009 15:12, "James Harper" <james.harper@bendigoit.com.au>
wrote:
> 
> > How can I tell how much memory my VM has under HVM? I want to have
> > something concrete to compare the 'target' value in xenstore to for
> > ballooning so I am reluctant to ask Windows the question. There
seems to
> > be a nr_pages in start_info, but HVM doesn't have that...
> >
> > My reason for not wanting to ask Windows how much memory is that if
it
> > says we have 3.5G (reasonable without PAE) and xenstore says we have
4G
> > then I'm going to have to fudge things a bit to know whether we are
> > ballooning up or down etc.
> >
> > I could just read the initial target value from xenstore on boot,
but
> > even that could be changed before my drivers get to it.
> >
> > Thanks
> >
> > James
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 

Re: How to determine memory size in HVM

[Keir Fraser]

On 09/06/2009 02:59, "James Harper" <james.harper@bendigoit.com.au> wrote:

> Thanks for that.
> 
> Another question, if I do 'xm mem-set' (without any handling of the
> memory/target value) and then reboot, the system reboots with the new
> target memory value, making it impossible to add memory to the system
> again (under Windows at least). The only way I can think of fixing that
> is to write back to memory/target with the original value just before
> rebooting, but even that could be prone to errors in case of an unclean
> reboot or something... any way around this?

The new HVM populate-on-demand memory support should be able to help. I
suppose also some tools changes may be necessary in conjunction with this.
Or domain config changes. George Dunlap and Steven Smith can probably
advise.

 -- Keir

Re: How to determine memory size in HVM

[George Dunlap]

On Tue, Jun 9, 2009 at 6:38 AM, Keir Fraser<keir.fraser@eu.citrix.com> wrote:
>
> The new HVM populate-on-demand memory support should be able to help. I
> suppose also some tools changes may be necessary in conjunction with this.
> Or domain config changes. George Dunlap and Steven Smith can probably
> advise.

I know all about the populate-on-demand stuff, but not very much about
the tools, and unfortunately that's where the change is needed, I
think. :-)

So the basic thing we need is to have not a single memory variable,
but two variables: "maximum" and "target".  Maximum is what is
reported to a VM at boot, and target is how much we want it to
actually have.

What populate-on-demand allows you to do is to tell a fully
virtualized VM it has maximum memory (say, 1G), but only give it
target memory (say, 256M).  When the VM boots, the OS will check its
e820 mapping, see 1G reported there, and set things up appropriately.
Then, when the balloon driver comes up, it will inflate the balloon to
maxmem-target.

The problem with rebooting sounds like it's due to the tools not being
designed with this distinction in mind.  So "xm mem-set" should change
target, but not maxmem; and ideally, when the VM reboots, it should be
created with the (maxmem,target) populate-on-demand split again.

 -George

RE: How to determine memory size in HVM

[James Harper]

> 
> Hypercall XENMEM_current_reservation.
> 
> *But* be aware I'm not sure how that interacts with e.g., virtual
> framebuffer memory. Well, actually I know it will include the virtual
> framebuffer memory which is not actually really part of normal guest
RAM.
> The question will be: does the xenstore 'target' value also include
it? I
> guess you'll have to compare to create a HVM guest and compare the two
to
> decide.
> 

memory/target = 786432KB
XENMEM_current_reservation = 198623 pages

198623 << PAGE_SHIFT = 794492KB
794492KB - 786432KB = 8060KB

So I have 8060KB unaccounted for... I guess that's probably the
framebuffer. According to Device Manager in Windows, the CL5446 device
uses 64KB of memory at 0xA0000, 32MB at 0xF0000000, and 4KB at
0xF3000000. That's more than the extra 8MB, but it would make sense that
the 32MB wouldn't all be mapped in if it wasn't required.

James

Re: How to determine memory size in HVM

[Keir Fraser]

On 09/06/2009 08:42, "James Harper" <james.harper@bendigoit.com.au> wrote:

> memory/target = 786432KB
> XENMEM_current_reservation = 198623 pages
> 
> 198623 << PAGE_SHIFT = 794492KB
> 794492KB - 786432KB = 8060KB
> 
> So I have 8060KB unaccounted for... I guess that's probably the
> framebuffer. According to Device Manager in Windows, the CL5446 device
> uses 64KB of memory at 0xA0000, 32MB at 0xF0000000, and 4KB at
> 0xF3000000. That's more than the extra 8MB, but it would make sense that
> the 32MB wouldn't all be mapped in if it wasn't required.

Yes, that's the framebuffer. It is variable size, in principle, though. I'm
not sure you can do better than read XENMEM_current_reservation and
memory/target at driver startup time, and keep the delta between them
constant.

 -- Keir

RE: How to determine memory size in HVM

[James Harper]

> >
> > So I have 8060KB unaccounted for... I guess that's probably the
> > framebuffer. According to Device Manager in Windows, the CL5446
device
> > uses 64KB of memory at 0xA0000, 32MB at 0xF0000000, and 4KB at
> > 0xF3000000. That's more than the extra 8MB, but it would make sense
that
> > the 32MB wouldn't all be mapped in if it wasn't required.
> 
> Yes, that's the framebuffer. It is variable size, in principle,
though. I'm
> not sure you can do better than read XENMEM_current_reservation and
> memory/target at driver startup time, and keep the delta between them
> constant.
> 

There is a XENMEM_maximum_reservation, which I think makes it 8MB
exactly instead of 8060KB (7.xxxMB). Would I be better off using that as
a comparison?

James

Re: How to determine memory size in HVM

[Keir Fraser]

On 10/06/2009 12:39, "James Harper" <james.harper@bendigoit.com.au> wrote:

>> Yes, that's the framebuffer. It is variable size, in principle,
> though. I'm
>> not sure you can do better than read XENMEM_current_reservation and
>> memory/target at driver startup time, and keep the delta between them
>> constant.
> 
> There is a XENMEM_maximum_reservation, which I think makes it 8MB
> exactly instead of 8060KB (7.xxxMB). Would I be better off using that as
> a comparison?

I don't think you should depend on that.

Perhaps just having XENMEM_current_reservation track memory/target would be
fine. I'm not sure if it will make e.g. The memory values returned by 'xm
list' incorrect.

We might have to provide more info to you via xenstore to do the 100% right
thing here.

 -- Keir

RE: How to determine memory size in HVM

[James Harper]

> On 10/06/2009 12:39, "James Harper" <james.harper@bendigoit.com.au>
wrote:
> 
> >> Yes, that's the framebuffer. It is variable size, in principle,
> > though. I'm
> >> not sure you can do better than read XENMEM_current_reservation and
> >> memory/target at driver startup time, and keep the delta between
them
> >> constant.
> >
> > There is a XENMEM_maximum_reservation, which I think makes it 8MB
> > exactly instead of 8060KB (7.xxxMB). Would I be better off using
that as
> > a comparison?
> 
> I don't think you should depend on that.

Ok. thanks.

> 
> Perhaps just having XENMEM_current_reservation track memory/target
would be
> fine. I'm not sure if it will make e.g. The memory values returned by
'xm
> list' incorrect.
> 
> We might have to provide more info to you via xenstore to do the 100%
right
> thing here.

Well I think the main problem is that if you shrink the DomU and then
reboot, you can't ever grow it again as Windows thinks it booted with
less memory. While there is the ability to add memory to a running
Windows machine, it is the 'hot add memory' (eg plugging in a memory
stick) and involves ACPI and only works under the Windows Server
Enterprise editions (which might still be useful, but probably not worth
the headaches involved in the ACPI work). I think the other issues are
minor in comparison to this problem.

Thanks

James

Re: How to determine memory size in HVM

[Keir Fraser]

On 10/06/2009 16:28, "James Harper" <james.harper@bendigoit.com.au> wrote:

>> We might have to provide more info to you via xenstore to do the 100%
> right
>> thing here.
> 
> Well I think the main problem is that if you shrink the DomU and then
> reboot, you can't ever grow it again as Windows thinks it booted with
> less memory. While there is the ability to add memory to a running
> Windows machine, it is the 'hot add memory' (eg plugging in a memory
> stick) and involves ACPI and only works under the Windows Server
> Enterprise editions (which might still be useful, but probably not worth
> the headaches involved in the ACPI work). I think the other issues are
> minor in comparison to this problem.

Populate-on-demand just needs integrating properly into xend. Then we would
reboot with maxmem, and then the guest properly balloons down as soon as
your GPLPV drivers are loaded. Populate-on-demand means the guest thinks it
has maxmem allocated to it, but really memory pages are only being allocated
to it on first use. This is all implemented in Xen and at least the lowest
levels of the toolstack (libxc).

 -- Keir