[PATCH] Add Small Memory Warning to XM for create and mem-set

[PATCH] Add Small Memory Warning to XM for create and mem-set

[Daniel Stekloff]

[-- Attachment #1: Type: text/plain, Size: 301 bytes --]


Hi,

This patch adds a warning to xm for creating and setting a domain's
memory under 32 MBs. Patch adds a "--force-mem" option to "xm create"
and "xm mem-set" commands for creating or setting memory less than 32
MBs. Patch also updates man page and user manual.

All comments welcome.

Thanks,

Dan

[-- Attachment #2: xm-add-force-mem-option.patch --]
[-- Type: text/x-patch, Size: 7313 bytes --]

# HG changeset patch
# User stekloff@dyn9047022152.beaverton.ibm.com
# Node ID 61acbc181fe4caff86a4a3ec4ba69ed43492b511
# Parent  72f9c751d3ea1f17ff513cd7fc2cbe671a9af7c9

To help the small memory issue, I've added a warning to xm for creating - 
"xm create" - a domain and setting - "xm mem-set" - a domain's memory with
less than 32 MBs. I've added a "--force-mem" option to force creating and 
setting memory below that mark. I've also patched the man page and the
user manual.

Signed-off-by: Daniel Stekloff <dsteklof@us.ibm.com>

diff -r 72f9c751d3ea -r 61acbc181fe4 docs/man/xm.pod.1
--- a/docs/man/xm.pod.1	Wed Apr 19 17:32:20 2006
+++ b/docs/man/xm.pod.1	Fri Apr 21 14:45:12 2006
@@ -63,7 +63,7 @@
 so running curses based interfaces over the console B<is not
 advised>.  Vi tends to get very odd when using it over this interface.
 
-=item B<create> I<[-c]> I<configfile> I<[name=value]>..
+=item B<create> I<[-c]> I<configfile> I<[--force-mem]> I<[name=value]>..
 
 The create sub command requires a configfile and can optional take a
 series of name value pairs that add to or override variables defined
@@ -78,6 +78,12 @@
 not> mean the guest OS in the domain has actually booted, or is
 available for input.
 
+B<Warning>: Creating a domain with insufficient memory may cause out
+of memory errors. The domain needs enough memory to boot kernel and
+modules. Allocating less than 32MBs is not recommended. Use the
+I<--force-mem> option to create a domain with less than 32 MBs of
+memory.
+
 B<OPTIONS>
 
 =over 4 
@@ -86,6 +92,10 @@
 
 Attache console to the domain as soon as it has started.  This is
 useful for determining issues with crashing domains.
+
+=item B<--force-mem>
+
+Force domain creation with less than 32MBs of memory.
 
 =back
 
@@ -234,7 +244,7 @@
 The mem-max value may not correspond to the actual memory used in the
 Domain, as it may balloon down it's memory to give more back to the OS.
 
-=item B<mem-set> I<domain-id> I<mem>
+=item B<mem-set> I<domain-id> I<mem> I<[--force-mem]>
 
 Set the domain's used memory using the balloon driver.  Because this
 operation requires cooperation from the domain operating system, there
@@ -242,7 +252,9 @@
 
 B<Warning:> there is no good way to know in advance how small of a
 mem-set will make a domain unstable and cause it to crash.  Be very
-careful when using this command on running domains.
+careful when using this command on running domains. Setting a domain
+to less than 32 MBs isn't recommended. Use the I<--force-mem> option
+to set a domain to less than 32 MBs of memory.
 
 =item B<migrate> I<domain-id> I<host> I<[options]>
 
diff -r 72f9c751d3ea -r 61acbc181fe4 docs/src/user.tex
--- a/docs/src/user.tex	Wed Apr 19 17:32:20 2006
+++ b/docs/src/user.tex	Fri Apr 21 14:45:12 2006
@@ -837,7 +837,10 @@
 \item[kernel] Set this to the path of the kernel you compiled for use
   with Xen (e.g.\ \path{kernel = ``/boot/vmlinuz-2.6-xenU''})
 \item[memory] Set this to the size of the domain's memory in megabytes
-  (e.g.\ \path{memory = 64})
+  (e.g.\ \path{memory = 64}) {\bf WARNING}: Creating a domain with
+  insufficient memory may cause out of memory errors. The domain
+  needs enough memory to boot kernel and modules. Allocating less
+  than 32MBs is not recommended.
 \item[disk] Set the first entry in this list to calculate the offset
   of the domain's root partition, based on the domain ID\@.  Set the
   second to the location of \path{/usr} if you are sharing it between
@@ -1161,6 +1164,10 @@
 \path{xmexample2} file, which uses Python code to handle the
 \path{vmid} variable.
 
+{\bf WARNING}: Creating or setting a domain with insufficient memory
+may cause out of memory errors. The domain needs enough memory to
+boot kernel and modules. Allocating less than 32MBs is not
+recommended.
 
 %\part{Advanced Topics}
 
diff -r 72f9c751d3ea -r 61acbc181fe4 tools/python/xen/xm/create.py
--- a/tools/python/xen/xm/create.py	Wed Apr 19 17:32:20 2006
+++ b/tools/python/xen/xm/create.py	Fri Apr 21 14:45:12 2006
@@ -103,6 +103,10 @@
 gopts.opt('console_autoconnect', short='c',
           fn=set_true, default=0,
           use="Connect to the console after the domain is created.")
+
+gopts.opt('force-mem', short='m',
+          fn=set_true, default=0,
+          use="""Force memory creation even if under recommended 32 MB lower limit.""")
 
 gopts.var('vncviewer', val='no|yes',
           fn=set_bool, default=None,
@@ -774,6 +778,20 @@
         vnc_host = get_host_addr()
         vnc = 'VNC_VIEWER=%s:%d' % (vnc_host, vnc_port)
         vals.extra = vnc + ' ' + vals.extra
+
+def preprocess_memory(vals):
+    """Check if memory is set to less than 32 MBs, which is not
+    recommended. If so, warn and exit if force-mem option isn't set."""
+
+    if int(vals.memory) < 32 and not gopts.getopt("force-mem"):
+
+        warn(
+"""Creating a domain with insufficient memory may cause 
+out of memory errors. The domain needs enough memory to boot kernel 
+and modules. Allocating less than 32MBs is not recommended. Use 
+the \'--force-mem\' option to set memory below 32MBs.\n""")
+
+        sys.exit(1)
     
 def preprocess(vals):
     if not vals.kernel and not vals.bootloader:
@@ -785,6 +803,7 @@
     preprocess_nfs(vals)
     preprocess_vnc(vals)
     preprocess_vtpm(vals)
+    preprocess_memory(vals)
 
 
 def comma_sep_kv_to_dict(c):
diff -r 72f9c751d3ea -r 61acbc181fe4 tools/python/xen/xm/main.py
--- a/tools/python/xen/xm/main.py	Wed Apr 19 17:32:20 2006
+++ b/tools/python/xen/xm/main.py	Fri Apr 21 14:45:12 2006
@@ -50,13 +50,14 @@
 
 # Strings for shorthelp
 console_help = "console <DomId>                  Attach to domain DomId's console."
-create_help =  """create [-c] <ConfigFile>
+create_help =  """create [-c] <ConfigFile> [--force-mem]
                [Name=Value]..       Create a domain based on Config File"""
 destroy_help = "destroy <DomId>                  Terminate a domain immediately"
 help_help =    "help                             Display this message"
 list_help =    "list [--long] [DomId, ...]       List information about domains"
 mem_max_help = "mem-max <DomId> <Mem>            Set maximum memory reservation for a domain"
-mem_set_help = "mem-set <DomId> <Mem>            Adjust the current memory usage for a domain"
+mem_set_help = """mem-set <DomId> <Mem> [--force-mem]
+                                    Adjust the current memory usage for a domain"""
 migrate_help = "migrate <DomId> <Host>           Migrate a domain to another machine"
 pause_help =   "pause <DomId>                    Pause execution of a domain"
 reboot_help =  "reboot <DomId> [-w][-a]          Reboot a domain"
@@ -566,10 +567,18 @@
     server.xend.domain.maxmem_set(dom, mem)
     
 def xm_mem_set(args):
-    arg_check(args, "mem-set", 2)
+    arg_check(args, "mem-set", 2, 3)
+    alen = len(args)
 
     dom = args[0]
     mem_target = int_unit(args[1], 'm')
+    if mem_target < 32 and (alen < 3 or (alen > 2 and args[2] != '--force-mem')):
+        print """
+Warning: Setting a domain with insufficient memory may cause out of
+         memory errors. Allocating less than 32MBs is not recommended.
+         Use the \'--force-mem\' option to set below 32MBs
+        """
+        usage("mem-set")
 
     server.xend.domain.setMemoryTarget(dom, mem_target)
     

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Ewan Mellor]

On Fri, Apr 21, 2006 at 08:17:38AM -0700, Daniel Stekloff wrote:

> 
> Hi,
> 
> This patch adds a warning to xm for creating and setting a domain's
> memory under 32 MBs. Patch adds a "--force-mem" option to "xm create"
> and "xm mem-set" commands for creating or setting memory less than 32
> MBs. Patch also updates man page and user manual.
> 
> All comments welcome.

You'll need to patch 16_create_smallmem_neg, too, otherwise that test will
fail.

Ewan.

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Mark Williamson]

I really like this idea.  I think it's most important for ballooning, which is 
a really easy way to get a domain to commit suicide otherwise ;-)

I'd quite like to see it implemented Xend-side, though, with the "warning 
level" configurable (and probably disablable) in the xend-config.sxp so that 
users who really do want to create very small domUs can tweak the system not 
to warn them.

Cheers,
Mark

On Friday 21 April 2006 16:17, Daniel Stekloff wrote:
> Hi,
>
> This patch adds a warning to xm for creating and setting a domain's
> memory under 32 MBs. Patch adds a "--force-mem" option to "xm create"
> and "xm mem-set" commands for creating or setting memory less than 32
> MBs. Patch also updates man page and user manual.
>
> All comments welcome.
>
> Thanks,
>
> Dan

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Daniel Stekloff]

On Fri, 2006-04-21 at 16:29 +0100, Mark Williamson wrote:
> I really like this idea.  I think it's most important for ballooning, which is 
> a really easy way to get a domain to commit suicide otherwise ;-)
> 
> I'd quite like to see it implemented Xend-side, though, with the "warning 
> level" configurable (and probably disablable) in the xend-config.sxp so that 
> users who really do want to create very small domUs can tweak the system not 
> to warn them.


Ok, that makes sense. I will also try and remember to patch xm-test. 

Thanks,

Dan

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Anthony Liguori]

Mark Williamson wrote:
> I really like this idea.  I think it's most important for ballooning, which is 
> a really easy way to get a domain to commit suicide otherwise ;-)
>
> I'd quite like to see it implemented Xend-side, though, with the "warning 
> level" configurable (and probably disablable) in the xend-config.sxp so that 
> users who really do want to create very small domUs can tweak the system not 
> to warn them.
>   

I'm not sure I agree that Xend is the right place to do this.  You would 
have to add another parameter to the either the domain config or to the 
actual create() call that basically had the semantics of "if memory is 
less than X, throw an exception."  That seems like a rather awkward 
addition to the interface and it makes client code harder to write.

I'll attempt to be telepathic here and guess that you suggested this 
because there's no xm config file so there's no easy way to be able to 
tell xm to never give an error when mem is too small?  I think this is a 
good point in time to start thinking about just implementing an xm 
config file.

If you use SafeConfigParser and just grab from a standard location (say 
/etc/xen/xm.conf), you should be able to do it properly (just within xm) 
with very little code.

Thoughts?

Regards,

Anthony Liguori

> Cheers,
> Mark
>
> On Friday 21 April 2006 16:17, Daniel Stekloff wrote:
>   
>> Hi,
>>
>> This patch adds a warning to xm for creating and setting a domain's
>> memory under 32 MBs. Patch adds a "--force-mem" option to "xm create"
>> and "xm mem-set" commands for creating or setting memory less than 32
>> MBs. Patch also updates man page and user manual.
>>
>> All comments welcome.
>>
>> Thanks,
>>
>> Dan
>>     
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Dan Smith]

[-- Attachment #1.1: Type: text/plain, Size: 803 bytes --]

AL> I'm not sure I agree that Xend is the right place to do this.  You
AL> would have to add another parameter to the either the domain
AL> config or to the actual create() call that basically had the
AL> semantics of "if memory is less than X, throw an exception."  That
AL> seems like a rather awkward addition to the interface and it makes
AL> client code harder to write.

I agree.

AL> Thoughts?

I think it's a good idea.  I'd rather see it in the client than in
Xend.  If some GUI app wants to have a configurable limit that's easy
for the user to access, then it'll just end up minding the limit
itself, and call create() with the force flag each time, which defeats
the point anyway.

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com

[-- Attachment #1.2: Type: application/pgp-signature, Size: 190 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Ewan Mellor]

On Fri, Apr 21, 2006 at 05:41:34PM -0500, Anthony Liguori wrote:

> Mark Williamson wrote:
> >I really like this idea.  I think it's most important for ballooning, 
> >which is a really easy way to get a domain to commit suicide otherwise ;-)
> >
> >I'd quite like to see it implemented Xend-side, though, with the "warning 
> >level" configurable (and probably disablable) in the xend-config.sxp so 
> >that users who really do want to create very small domUs can tweak the 
> >system not to warn them.
> >  
> 
> I'm not sure I agree that Xend is the right place to do this.  You would 
> have to add another parameter to the either the domain config or to the 
> actual create() call that basically had the semantics of "if memory is 
> less than X, throw an exception."  That seems like a rather awkward 
> addition to the interface and it makes client code harder to write.
> 
> I'll attempt to be telepathic here and guess that you suggested this 
> because there's no xm config file so there's no easy way to be able to 
> tell xm to never give an error when mem is too small?  I think this is a 
> good point in time to start thinking about just implementing an xm 
> config file.
> 
> If you use SafeConfigParser and just grab from a standard location (say 
> /etc/xen/xm.conf), you should be able to do it properly (just within xm) 
> with very little code.

I agree.  Daniel, could you redo your patch, introducing an xm config file
rather than a --force-mem option?  If you could make it so that xm works even
without the config file, using some decent defaults, that would avoid breaking
existing installations, which would be good.

Personally, I'd have an XML-based config file rather than a win.ini style one
-- we're moving towards using XML everywhere else, so it makes sense to me.
Anyone have any strong opinions here?

Ewan.

Re: [PATCH] Add Small Memory Warning to XM for create and mem-set

[Mark Williamson]

> I'm not sure I agree that Xend is the right place to do this.  You would
> have to add another parameter to the either the domain config or to the
> actual create() call that basically had the semantics of "if memory is
> less than X, throw an exception."  That seems like a rather awkward
> addition to the interface and it makes client code harder to write.

Surely there are already plenty of other cases in which Xend can fail a create 
and throw the result back to xm?  For instance, if the kernel image can't be 
found, hotplug scripts fail, etc.  Wouldn't it just be an extra case of that?

> I'll attempt to be telepathic here and guess that you suggested this
> because there's no xm config file so there's no easy way to be able to
> tell xm to never give an error when mem is too small?  I think this is a
> good point in time to start thinking about just implementing an xm
> config file.

Well, yes and no.

Yes, Xend is the only place to do this right now.  But we've always tried 
(modulo a few things, eg pygrub) to keep xm a pretty thin layer, with most of 
the real smarts in Xend.  This also makes it easier to create a 
network-transparent implementation of xm, without confusing Xen-related 
config files on the "control" machine and on the servers.

That said, warning about improper config choices does seem rather like a 
client-side operation so perhaps an xm.conf is worth considering.

We should also consider:
1) whether we're going to need this for other options (I suspect yes, but it'd 
be unfortunate to have a config file for just one option)
2) whether there are any config choices currently handled by Xend or the 
domain config files, which we'd rather be handled by xm.conf

> If you use SafeConfigParser and just grab from a standard location (say
> /etc/xen/xm.conf), you should be able to do it properly (just within xm)
> with very little code.

Agreed, should be a fairly simple little hack.

Cheers,
Mark

> Thoughts?
>
> Regards,
>
> Anthony Liguori
>
> > Cheers,
> > Mark
> >
> > On Friday 21 April 2006 16:17, Daniel Stekloff wrote:
> >> Hi,
> >>
> >> This patch adds a warning to xm for creating and setting a domain's
> >> memory under 32 MBs. Patch adds a "--force-mem" option to "xm create"
> >> and "xm mem-set" commands for creating or setting memory less than 32
> >> MBs. Patch also updates man page and user manual.
> >>
> >> All comments welcome.
> >>
> >> Thanks,
> >>
> >> Dan
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!