Re: [PATCH 6/6] xl: 'xl claims' print outstanding per domain claims if enabled (claim_mode=1 in xl.conf)

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Fri, Apr 12, 2013 at 04:24:43PM +0100, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("[PATCH 6/6] xl: 'xl claims' print outstanding per domain claims if enabled (claim_mode=1 in xl.conf)"):
> > This is similar to "xl: 'xl info' print outstanding claims if enabled
> > (claim_mode=1 in xl.conf)" which exposes the global claim value.
> ...
> > Name                                        ID   Mem VCPUs      State   Time(s)  Claim
> > Domain-0                                     0  2048     4     r-----      15.7     0
> > OL5                                          2   321     1     --p---       0.0  1717
> > OL6                                          3   217     1     --p---       0.0   797
> 
> We discussed some of this yesterday on IRC.
> 
> My understanding from that conversation was that the outstanding claim
> was included in the report of the memory used by the guest.
> 
> That's not consistent with this example, and nothing in your
> documentation explains this.  I haven't checked the hypervisor code so
> it may be that this is a docs problem.
> 
> Can you confirm that "Mem" as reported by "xl list" _includes_
> outstanding claims ?  And please resend with a fix to (a) the example
> and (b) the relevant parts of the docs ?

Please see inline patch:

commit c75cb1dc35f5936238be0a6d49517b34a867880a
Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date:   Wed Apr 10 14:28:05 2013 -0400

    xl: 'xl claims' print outstanding per domain claims if enabled (claim_mode=1 in xl.conf)
    
    This is similar to "xl: 'xl info' print outstanding claims if enabled
    (claim_mode=1 in xl.conf)" which exposes the global claim value.
    
    This patch provides the value of the currently outstanding pages
    claimed for each domains. This is per domain value which is added
    to the global claim value which influences the hypervisors' MM system.
    
    When a claim call is done, a reservation for a specific amount of pages
    is set (and this patch lists said number) and also a global value is
    incremented. This global value is then reduced as the domain's memory
    is populated and eventually reaches zero.
    
    The toolstack (libxc) also sets the domain's claim to zero when the population
    of memory has completed as an extra step. Any call to destroy the domain
    will also set the domain's claim to zero.
    
    If the reservation cannot be meet the guest creation fails immediately
    instead of taking seconds or minutes (depending on the size of the guest)
    while the toolstack populates memory.
    
    See patch: "xl: Implement XENMEM_claim_pages support via 'claim_mode'
    global config" for details on how it is implemented.
    
    The value fluctuates quite often so the value is stale once it is provided
    to the user-space.  However it is useful for diagnostic purposes.
    
    It is only printed when the global "claim_mode" option in xl.conf(5)
    is set to enabled (1). The 'man xl' shows the details of this item.
    The output is close to what 'xl list' looks like:
    
    Name                                        ID   Mem VCPUs      State   Time(s)  Claimed
    Domain-0                                     0  2047     4     r-----      19.7     0
    OL5                                          2  2048     1     --p---       0.0   847
    OL6                                          3  1024     4     r-----       5.9     0
    Windows_XP                                   4  2047     1     --p---       0.0  1989
    
    [In which it can be seen that the OL5 guest still has 847MB of claimed
    memory (out of the total 2048MB where 1191MB has been allocated to
    the guest).]
    
    Please note that the 'Mem' column has the cumulative value of outstanding
    claims and the total amount of memory that has been right now allocated
    to the guest. The value equals the "memory" in the guest config
    (see xl.conf manpage).
    
    [v1: claims, not claim-list]
    [v2: Add outstanding and current memkb in the output list]
    Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index c2296ef..a0c14c8 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -280,7 +280,8 @@ An example format for the list is as follows:
 
 Name is the name of the domain.  ID the numeric domain id.  Mem is the
 desired amount of memory to allocate to the domain (although it may
-not be the currently allocated amount).  VCPUs is the number of
+not be the currently allocated amount - unless B<claim_mode> in xl.cfg has
+been enabled, see B<claims> below).  VCPUs is the number of
 virtual CPUs allocated to the domain.  State is the run state (see
 below).  Time is the total run time of the domain as accounted for by
 Xen.
@@ -782,6 +783,30 @@ explanatory.
 
 Prints the current uptime of the domains running.
 
+=item B<claims>
+
+Prints information about outstanding claims by the guests. This provides
+the outstanding claims and currently populated memory count for the guests.
+These values added up reflect the global outstanding claim value, which
+is provided via the I<info> argument, B<outstanding_claims> value.
+The B<Mem> column has the cumulative value of outstanding claims and
+the total amount of memory that has been right now allocated to the guest.
+The value equals the I<memory> in the guest config (see xl.conf manpage).
+
+B<EXAMPLE>
+
+An example format for the list is as follows:
+
+ Name                                        ID   Mem VCPUs      State   Time(s)  Claimed
+ Domain-0                                     0  2047     4     r-----      19.7     0
+ OL5                                          2  2048     1     --p---       0.0   847
+ OL6                                          3  1024     4     r-----       5.9     0
+ Windows_XP                                   4  2047     1     --p---       0.0  1989
+
+In which it can be seen that the OL5 guest still has 847MB of claimed
+memory (out of the total 2048MB where 1191MB has been allocated to
+the guest).
+
 =back
 
 =head1 SCHEDULER SUBCOMMANDS
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 8b0e415..c9905e3 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -3865,9 +3865,9 @@ int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs)
         rc = libxl_domain_info(ctx, &info, domid);
         if (rc < 0)
             return rc;
-    } while (wait_secs > 0 && info.current_memkb > target_memkb);
+    } while (wait_secs > 0 && (info.current_memkb + info.outstanding_memkb) > target_memkb);
 
-    if (info.current_memkb <= target_memkb)
+    if ((info.current_memkb + info.outstanding_memkb) <= target_memkb)
         rc = 0;
     else
         rc = ERROR_FAIL;
diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
index 4c5e5d1..771b4af 100644
--- a/tools/libxl/xl.h
+++ b/tools/libxl/xl.h
@@ -83,6 +83,7 @@ int main_vtpmattach(int argc, char **argv);
 int main_vtpmlist(int argc, char **argv);
 int main_vtpmdetach(int argc, char **argv);
 int main_uptime(int argc, char **argv);
+int main_claims(int argc, char **argv);
 int main_tmem_list(int argc, char **argv);
 int main_tmem_freeze(int argc, char **argv);
 int main_tmem_thaw(int argc, char **argv);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index c9b71e6..80aee7f 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -3061,7 +3061,8 @@ out:
     }
 }
 
-static void list_domains(int verbose, int context, const libxl_dominfo *info, int nb_domain)
+static void list_domains(int verbose, int context, int claim,
+                         const libxl_dominfo *info, int nb_domain)
 {
     int i;
     static const char shutdown_reason_letters[]= "-rscw";
@@ -3069,6 +3070,7 @@ static void list_domains(int verbose, int context, const libxl_dominfo *info, in
     printf("Name                                        ID   Mem VCPUs\tState\tTime(s)");
     if (verbose) printf("   UUID                            Reason-Code\tSecurity Label");
     if (context && !verbose) printf("   Security Label");
+    if (claim) printf("  Claimed");
     printf("\n");
     for (i = 0; i < nb_domain; i++) {
         char *domname;
@@ -3078,7 +3080,8 @@ static void list_domains(int verbose, int context, const libxl_dominfo *info, in
         printf("%-40s %5d %5lu %5d     %c%c%c%c%c%c  %8.1f",
                 domname,
                 info[i].domid,
-                (unsigned long) (info[i].current_memkb / 1024),
+                (unsigned long) ((info[i].current_memkb +
+                    info[i].outstanding_memkb)/ 1024),
                 info[i].vcpu_online,
                 info[i].running ? 'r' : '-',
                 info[i].blocked ? 'b' : '-',
@@ -3095,6 +3098,8 @@ static void list_domains(int verbose, int context, const libxl_dominfo *info, in
             if (info[i].shutdown) printf(" %8x", shutdown_reason);
             else printf(" %8s", "-");
         }
+        if (claim)
+            printf(" %5lu", (unsigned long)info[i].outstanding_memkb / 1024);
         if (verbose || context) {
             int rc;
             size_t size;
@@ -4029,7 +4034,7 @@ int main_list(int argc, char **argv)
     if (details)
         list_domains_details(info, nb_domain);
     else
-        list_domains(verbose, context, info, nb_domain);
+        list_domains(verbose, context, 0 /* claim */, info, nb_domain);
 
     if (info_free)
         libxl_dominfo_list_free(info, nb_domain);
@@ -4742,7 +4747,8 @@ static void sharing(const libxl_dominfo *info, int nb_domain)
         printf("%-40s %5d %5lu  %5lu\n",
                 domname,
                 info[i].domid,
-                (unsigned long) (info[i].current_memkb / 1024),
+                (unsigned long) ((info[i].current_memkb +
+                    info[i].outstanding_memkb) / 1024),
                 (unsigned long) (info[i].shared_memkb / 1024));
         free(domname);
     }
@@ -5927,6 +5933,33 @@ static char *uptime_to_string(unsigned long uptime, int short_mode)
     return time_string;
 }
 
+int main_claims(int argc, char **argv)
+{
+    libxl_dominfo *info;
+    int opt;
+    int nb_domain;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "claims", 0) {
+        /* No options */
+    }
+
+    if (!libxl_defbool_val(claim_mode)) {
+        fprintf(stderr, "claim_mode not enabled (see man xl.conf).\n");
+        return 1;
+    }
+    info = libxl_list_domain(ctx, &nb_domain);
+    if (!info) {
+        fprintf(stderr, "libxl_domain_infolist failed.\n");
+        return 1;
+    }
+
+    list_domains(0 /* verbose */, 0 /* context */, 1 /* claim */,
+                 info, nb_domain);
+
+    libxl_dominfo_list_free(info, nb_domain);
+    return 0;
+}
+
 static char *current_time_to_string(time_t now)
 {
     char now_str[100];
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index b4a87ca..00899f5 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -362,6 +362,12 @@ struct cmd_spec cmd_table[] = {
       "Print uptime for all/some domains",
       "[-s] [Domain]",
     },
+    { "claims",
+      &main_claims, 0, 0,
+      "List outstanding claim information about all domains",
+      "",
+      "",
+    },
     { "tmem-list",
       &main_tmem_list, 0, 0,
       "List tmem pools",