xl features missing

xl features missing

[Dan Magenheimer]

FYI, I finally got a chance to play with xl a bit in
xen-unstable and quickly discovered the following:

- xl tmem-freeable doesn't exist (xm tmem-freeable does)
- xl vncviewer doesn't exist (xm vncviewer does)
- xl create doesn't handle the possibility that tmem
  has memory which is freeable (xm create does)

Dan

RE: xl features missing

[Dan Magenheimer]

> From: Dan Magenheimer
> Subject: xl features missing
> 
> FYI, I finally got a chance to play with xl a bit in
> xen-unstable and quickly discovered the following:

> - xl create doesn't handle the possibility that tmem
>   has memory which is freeable (xm create does)

This appears to be an indicator of a bigger issue unless
there is some code somewhere I am missing.

Can "xl create" deal with ballooning memory from dom0, as is
done in the free() method in tools/python/xen/xend/balloon.py?
If not, doesn't that mean that xl is useless on systems
where dom0_mem= isn't specified as a boot option?

Since tmem leverages this method to recover freeable
memory when creating a new guest, tmem also needs this
else xl is also useless on a tmem-enabled system.

And in either case, it appears that the error handling
for "out of memory" spews meaningless error messages.

> - xl tmem-freeable doesn't exist (xm tmem-freeable does)

Patch submitted.

> - xl vncviewer doesn't exist (xm vncviewer does)

Doesn't look too hard, but haven't tried it yet.  This
is something I use all the time though so would be nice
to have.

RE: xl features missing

[Stefano Stabellini]

On Fri, 16 Jul 2010, Dan Magenheimer wrote:
> > From: Dan Magenheimer
> > Subject: xl features missing
> > 
> > FYI, I finally got a chance to play with xl a bit in
> > xen-unstable and quickly discovered the following:
> 
> > - xl create doesn't handle the possibility that tmem
> >   has memory which is freeable (xm create does)
> 
> This appears to be an indicator of a bigger issue unless
> there is some code somewhere I am missing.
> 
> Can "xl create" deal with ballooning memory from dom0, as is
> done in the free() method in tools/python/xen/xend/balloon.py?
> If not, doesn't that mean that xl is useless on systems
> where dom0_mem= isn't specified as a boot option?
>
> Since tmem leverages this method to recover freeable
> memory when creating a new guest, tmem also needs this
> else xl is also useless on a tmem-enabled system.
> 
> And in either case, it appears that the error handling
> for "out of memory" spews meaningless error messages.

xl doesn't have an equivalent of the free() method in xend, so it would
be useless without dom0_mem= at the moment.
However reimplementing that method shouldn't be hard, the only problem
is that it would probably require some sort of global locking among all
the various xl processes running concurrently.
Maybe we need a lockfile?


> 
> > - xl tmem-freeable doesn't exist (xm tmem-freeable does)
> 
> Patch submitted.

thanks


> 
> > - xl vncviewer doesn't exist (xm vncviewer does)
> 
> Doesn't look too hard, but haven't tried it yet.  This
> is something I use all the time though so would be nice
> to have.
 
it should be something similar to the 'xl console' command that we
already have.

RE: xl features missing

[Stefano Stabellini]

On Mon, 19 Jul 2010, Stefano Stabellini wrote:
> On Fri, 16 Jul 2010, Dan Magenheimer wrote:
> > > From: Dan Magenheimer
> > > Subject: xl features missing
> > > 
> > > FYI, I finally got a chance to play with xl a bit in
> > > xen-unstable and quickly discovered the following:
> > 
> > > - xl create doesn't handle the possibility that tmem
> > >   has memory which is freeable (xm create does)
> > 
> > This appears to be an indicator of a bigger issue unless
> > there is some code somewhere I am missing.
> > 
> > Can "xl create" deal with ballooning memory from dom0, as is
> > done in the free() method in tools/python/xen/xend/balloon.py?
> > If not, doesn't that mean that xl is useless on systems
> > where dom0_mem= isn't specified as a boot option?
> >
> > Since tmem leverages this method to recover freeable
> > memory when creating a new guest, tmem also needs this
> > else xl is also useless on a tmem-enabled system.
> > 
> > And in either case, it appears that the error handling
> > for "out of memory" spews meaningless error messages.
> 
> xl doesn't have an equivalent of the free() method in xend, so it would
> be useless without dom0_mem= at the moment.
> However reimplementing that method shouldn't be hard, the only problem
> is that it would probably require some sort of global locking among all
> the various xl processes running concurrently.
> Maybe we need a lockfile?
> 

Another possible course of action would be to implement a libxl funtion
that calculates the amount of memory that needs to be freed, then call a
new libxl funtion similar to libxl_set_memory_target but that would take
a relative amount of memory as argument and would also be "tmem
enabled".
Finally the actual implementation of this function must communicate
relative amounts of memory to the kernel so that we can avoid the global
lock.
Does that make sense?

RE: xl features missing

[Dan Magenheimer]

> > > > - xl create doesn't handle the possibility that tmem
> > > >   has memory which is freeable (xm create does)
> > >
> > > This appears to be an indicator of a bigger issue unless
> > > there is some code somewhere I am missing.
> > >
> > > Can "xl create" deal with ballooning memory from dom0, as is
> > > done in the free() method in tools/python/xen/xend/balloon.py?
> > > If not, doesn't that mean that xl is useless on systems
> > > where dom0_mem= isn't specified as a boot option?
> > >
> > > Since tmem leverages this method to recover freeable
> > > memory when creating a new guest, tmem also needs this
> > > else xl is also useless on a tmem-enabled system.
> > >
> > > And in either case, it appears that the error handling
> > > for "out of memory" spews meaningless error messages.
> >
> > xl doesn't have an equivalent of the free() method in xend, so it
> > would be useless without dom0_mem= at the moment.
> > However reimplementing that method shouldn't be hard, the only problem
> > is that it would probably require some sort of global locking among
> > all the various xl processes running concurrently.
> > Maybe we need a lockfile?
>
> Another possible course of action would be to implement a libxl funtion
> that calculates the amount of memory that needs to be freed, then call
> a new libxl funtion similar to libxl_set_memory_target but that would
> take a relative amount of memory as argument and would also be "tmem
> enabled".
> Finally the actual implementation of this function must communicate
> relative amounts of memory to the kernel so that we can avoid the
> global lock. Does that make sense?

What xl processes are racy for obtaining memory?  Is it possible
that multiple xl processes are doing an "xl create" at the same time?

I'm not sure I understand completely, but one possible issue is
that the function that asks "is there enough memory?" must not
actually tell tmem to surrender the memory.  Also, tmem has
a "freeze/thaw" mechanism to ensure additional memory isn't
asynchronously absorbed (search for the instances of "tmem"
in balloon.py to see this code) and they must always be paired,
e.g. a freeze without a thaw is disastrous for tmem.
Note that other dynamic memory mechanisms (even plain ballooning)
don't have this freeze/thaw capability so there
is always the potential for races, e.g. for

- Is there enough memory for a 1G domain?
- Yes, it appears there is enough so I will proceed
- Create the domain with 1G of memory

the last step must be allowed to fail gracefully (preferably
with a decent error message).

Ideally, this should be solved by a lock in the hypervisor but
as long as dom0 ballooning exists (which is by its nature
asynchronous), I'm not sure this is possible.

RE: xl features missing

[Stefano Stabellini]

On Mon, 19 Jul 2010, Dan Magenheimer wrote:
> What xl processes are racy for obtaining memory?  Is it possible
> that multiple xl processes are doing an "xl create" at the same time?

Multiple users on the same host must be able to call "xl create" at the
same time.

> 
> I'm not sure I understand completely, but one possible issue is
> that the function that asks "is there enough memory?" must not
> actually tell tmem to surrender the memory.  Also, tmem has
> a "freeze/thaw" mechanism to ensure additional memory isn't
> asynchronously absorbed (search for the instances of "tmem"
> in balloon.py to see this code) and they must always be paired,
> e.g. a freeze without a thaw is disastrous for tmem.
> Note that other dynamic memory mechanisms (even plain ballooning)
> don't have this freeze/thaw capability so there
> is always the potential for races, e.g. for
> 
> - Is there enough memory for a 1G domain?
> - Yes, it appears there is enough so I will proceed
> - Create the domain with 1G of memory
> 
> the last step must be allowed to fail gracefully (preferably
> with a decent error message).

Indeed.
We could also try to catch these errors and and go back to step one for
a few times.