RE: xen: memory initialization/balloon fixes (#3)

[Dan Magenheimer <dan.magenheimer@oracle.com>]

> From: Dan Magenheimer
> Subject: RE: [Xen-devel] xen: memory initialization/balloon fixes (#3)
> 
> > From: David Vrabel [mailto:david.vrabel@citrix.com]
> > Subject: Re: [Xen-devel] xen: memory initialization/balloon fixes (#3)
> >
> > On 20/09/11 17:57, Dan Magenheimer wrote:
> > >
> > >
> > > My problem occurs in a PV domU with an upstream-variant kernel based
> > > on 3.0.5.  The problem is that the total amount of memory as seen
> > > from inside the guest is always substantially less than the amount
> > > of memory seen from outside the guest.  The difference seems to
> > > be fixed within a given boot, but assigning a different vm.cfg mem=
> > > changes the amount.  (For example, the difference D is about 18MB on
> > > a mem=128 boot and about 36MB on a mem=1024 boot.)
> >
> > I don't see the problem?
> 
> Hi David --
> 
> Sorry, just to clarify, are you saying you are seeing the same
> behavior and don't consider it a problem, or that you are not
> seeing the same difference?
>
> > The MemTotal value /proc/meminfo doesn't include some pages reserved by
> > the kernel which is why it's less than the maximum reservation of the
> > domain.
> 
> I'm aware of that... "some" has been a fixed size of a few megabytes
> in Xen for a long time.  I am seeing 30-60MB or more.

Never mind on this part.  After further debugging, I can see
that this difference is due to normal uses of memory by the
kernel for XEN PAGETABLES and RAMDISK etc.  It's unfortunate
that the difference is so large, but I guess that's in part due
to the desire to use the same kernel binary for native and
virtualized.  I don't remember it being nearly so high for
older PV kernels, but I guess it's progress! :-}

> > > Part B of the problem (and the one most important to me) is that
> > > setting /sys/devices/system/xen_memory/xen_memory0/target_kb
> > > to X results in a MemTotal inside the domU (as observed by
> > > "head -1 /proc/meminfo") of X-D.  This can be particularly painful
> > > when X is aggressively small as X-D may result in OOMs.
> > > To use kernel function/variable names (and I observed this with
> > > some debugging code), when balloon_set_new_target(X) is called
> > > totalram_pages gets driven to X-D.
> >
> > Again, this looks like the correct behavior to me.
> 
> Hmmm... so if a user (or automated tool) uses the Xen-defined
> API (i.e. /sys/devices/system/xen_memory/xen_memory0/target_kb)
> to use the Xen balloon driver to attempt to reduce memory usage
> to 100MB, and the Xen balloon driver instead reduces it to
> some random number somewhere between 40MB and 90MB, which
> may or may not cause OOMs, you consider this correct behavior?

I still think this is a bug but apparently orthogonal to
your patchset.  So sorry to bother you.

Dan