[PATCH 0 of 2] Propagate sharing and paging counts to libx{c/l}

[PATCH 0 of 2] Propagate sharing and paging counts to libx{c/l}

[Andres Lagar-Cavilla]

Currently commented out, no reason to disable it.

The hypervisor <-> libxc interface changes, though.

Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>

 tools/libxc/xc_domain.c     |  2 +-
 xen/common/domctl.c         |  2 +-
 xen/include/public/domctl.h |  2 +-
 tools/libxl/libxl.c         |  3 ++-
 tools/libxl/libxl_types.idl |  1 +
 5 files changed, 6 insertions(+), 4 deletions(-)

[PATCH 1 of 2] Propagate domain shared pages via dominfo

[Andres Lagar-Cavilla]

 tools/libxc/xc_domain.c     |  2 +-
 xen/common/domctl.c         |  2 +-
 xen/include/public/domctl.h |  2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)


Currently commented out, no reason to disable it.

Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>

diff -r addb6616e282 -r 94ec84eeca5d tools/libxc/xc_domain.c
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -234,7 +234,7 @@ int xc_domain_getinfo(xc_interface *xch,
 
         info->ssidref  = domctl.u.getdomaininfo.ssidref;
         info->nr_pages = domctl.u.getdomaininfo.tot_pages;
-        /* info->nr_shared_pages = domctl.u.getdomaininfo.shr_pages; */
+        info->nr_shared_pages = domctl.u.getdomaininfo.shr_pages;
         info->nr_paged_pages = domctl.u.getdomaininfo.paged_pages;
         info->max_memkb = domctl.u.getdomaininfo.max_pages << (PAGE_SHIFT-10);
         info->shared_info_frame = domctl.u.getdomaininfo.shared_info_frame;
diff -r addb6616e282 -r 94ec84eeca5d xen/common/domctl.c
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -153,7 +153,7 @@ void getdomaininfo(struct domain *d, str
 
     info->tot_pages         = d->tot_pages;
     info->max_pages         = d->max_pages;
-    /* info->shr_pages         = atomic_read(&d->shr_pages); */
+    info->shr_pages         = atomic_read(&d->shr_pages);
     info->paged_pages       = atomic_read(&d->paged_pages);
     info->shared_info_frame = mfn_to_gmfn(d, __pa(d->shared_info)>>PAGE_SHIFT);
     BUG_ON(SHARED_M2P(info->shared_info_frame));
diff -r addb6616e282 -r 94ec84eeca5d xen/include/public/domctl.h
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -94,7 +94,7 @@ struct xen_domctl_getdomaininfo {
     uint32_t flags;              /* XEN_DOMINF_* */
     uint64_aligned_t tot_pages;
     uint64_aligned_t max_pages;
-    /* uint64_aligned_t shr_pages; */
+    uint64_aligned_t shr_pages;
     uint64_aligned_t paged_pages;
     uint64_aligned_t shared_info_frame; /* GMFN of shared_info struct */
     uint64_aligned_t cpu_time;

Re: [PATCH 1 of 2] Propagate domain shared pages via dominfo

[Ian Campbell]

On Mon, 2012-03-12 at 19:53 +0000, Andres Lagar-Cavilla wrote:
> tools/libxc/xc_domain.c     |  2 +-
>  xen/common/domctl.c         |  2 +-
>  xen/include/public/domctl.h |  2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
> 
> 
> Currently commented out, no reason to disable it.
> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
> diff -r addb6616e282 -r 94ec84eeca5d xen/include/public/domctl.h
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -94,7 +94,7 @@ struct xen_domctl_getdomaininfo {
>      uint32_t flags;              /* XEN_DOMINF_* */
>      uint64_aligned_t tot_pages;
>      uint64_aligned_t max_pages;
> -    /* uint64_aligned_t shr_pages; */
> +    uint64_aligned_t shr_pages;
>      uint64_aligned_t paged_pages;
>      uint64_aligned_t shared_info_frame; /* GMFN of shared_info struct */
>      uint64_aligned_t cpu_time;

This doesn't appear to be commented out here, xen-unstable
25007:5d20d2f6ffed:

$ grep -A 4 -B 4 shr_pages xen/include/public/*
xen/include/public/domctl.h-#define XEN_DOMINF_shutdownshift 16
xen/include/public/domctl.h-    uint32_t flags;              /* XEN_DOMINF_* */
xen/include/public/domctl.h-    uint64_aligned_t tot_pages;
xen/include/public/domctl.h-    uint64_aligned_t max_pages;
xen/include/public/domctl.h:    uint64_aligned_t shr_pages;
xen/include/public/domctl.h-    uint64_aligned_t paged_pages;
xen/include/public/domctl.h-    uint64_aligned_t shared_info_frame; /* GMFN of shared_info struct */
xen/include/public/domctl.h-    uint64_aligned_t cpu_time;
xen/include/public/domctl.h-    uint32_t nr_online_vcpus;    /* Number of VCPUs currently online. */

I didn't look at the other hunks after I noticed this. I can't see
anywhere in history where it was commented out -- perhaps a local patch
on your end?

Ian.

[PATCH 2 of 2] libxl: publish number of shared and paged pages

[Andres Lagar-Cavilla]

 tools/libxl/libxl.c         |  3 ++-
 tools/libxl/libxl_types.idl |  1 +
 2 files changed, 3 insertions(+), 1 deletions(-)


This is information is currently not harvested from libxc.

Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>

diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
         xlinfo->shutdown_reason  = ~0;
 
     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
-    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
+    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;
diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl_types.idl
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Campbell]

On Mon, 2012-03-12 at 19:53 +0000, Andres Lagar-Cavilla wrote:
> tools/libxl/libxl.c         |  3 ++-
>  tools/libxl/libxl_types.idl |  1 +
>  2 files changed, 3 insertions(+), 1 deletions(-)
> 
> 
> This is information is currently not harvested from libxc.
> 
> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
> 
> diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
>          xlinfo->shutdown_reason  = ~0;
>  
>      xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
> -    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
> +    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);

Same comment about not being commented out in xen-unstable. The other
bits of this patch appear to make sense though.

> +    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
>      xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
>      xlinfo->cpu_time = xcinfo->cpu_time;
>      xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;
> diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl_types.idl
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
>      ("shutdown_reason", libxl_shutdown_reason),
>      ("current_memkb",   MemKB),
>      ("shared_memkb", MemKB),
> +    ("paged_memkb", MemKB),
>      ("max_memkb",   MemKB),
>      ("cpu_time",    uint64),
>      ("vcpu_max_id", uint32),

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Andres Lagar-Cavilla]

> On Mon, 2012-03-12 at 19:53 +0000, Andres Lagar-Cavilla wrote:
>> tools/libxl/libxl.c         |  3 ++-
>>  tools/libxl/libxl_types.idl |  1 +
>>  2 files changed, 3 insertions(+), 1 deletions(-)
>>
>>
>> This is information is currently not harvested from libxc.
>>
>> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>>
>> diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
>> --- a/tools/libxl/libxl.c
>> +++ b/tools/libxl/libxl.c
>> @@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
>>          xlinfo->shutdown_reason  = ~0;
>>
>>      xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
>> -    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
>> +    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
>
> Same comment about not being commented out in xen-unstable. The other
> bits of this patch appear to make sense though.

I think the most accurate description for these two patches is "brain fart"

Here is the libxl patch, rebased.
Thanks
Andres

# HG changeset patch
# User Andres Lagar-Cavilla <andres@lagarcavilla.org>
# Date 1331648207 14400
# Node ID 94b89a5f14e512c2ad46ef9ad1628fba19858463
# Parent  5d20d2f6ffed0a49f030f04a8870f1926babbcbf
libxl: publish number of paged pages.

This is information is currently not harvested from libxc.

Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>

diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -460,6 +460,7 @@ static void xcinfo2xlinfo(const xc_domai

     xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
     xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
+    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
     xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;
diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl_types.idl
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown_reason", libxl_shutdown_reason),
     ("current_memkb",   MemKB),
     ("shared_memkb", MemKB),
+    ("paged_memkb", MemKB),
     ("max_memkb",   MemKB),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Campbell]

> >> diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
> >> --- a/tools/libxl/libxl.c
> >> +++ b/tools/libxl/libxl.c
> >> @@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
> >>          xlinfo->shutdown_reason  = ~0;
> >>
> >>      xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
> >> -    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
> >> +    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
> >
> > Same comment about not being commented out in xen-unstable. The other
> > bits of this patch appear to make sense though.
> 
> I think the most accurate description for these two patches is "brain fart"

;-)

> libxl: publish number of paged pages.
> 
> This is information is currently not harvested from libxc.
> 
> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

But did you not want to print it from xl command or other?

> 
> diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl.c
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -460,6 +460,7 @@ static void xcinfo2xlinfo(const xc_domai
> 
>      xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
>      xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
> +    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
>      xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
>      xlinfo->cpu_time = xcinfo->cpu_time;
>      xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;
> diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl_types.idl
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
>      ("shutdown_reason", libxl_shutdown_reason),
>      ("current_memkb",   MemKB),
>      ("shared_memkb", MemKB),
> +    ("paged_memkb", MemKB),
>      ("max_memkb",   MemKB),
>      ("cpu_time",    uint64),
>      ("vcpu_max_id", uint32),
> 
> 

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Andres Lagar-Cavilla]

>> >> diff -r 94ec84eeca5d -r a17bd526989d tools/libxl/libxl.c
>> >> --- a/tools/libxl/libxl.c
>> >> +++ b/tools/libxl/libxl.c
>> >> @@ -459,7 +459,8 @@ static void xcinfo2xlinfo(const xc_domai
>> >>          xlinfo->shutdown_reason  = ~0;
>> >>
>> >>      xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
>> >> -    /* xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages); */
>> >> +    xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
>> >
>> > Same comment about not being commented out in xen-unstable. The other
>> > bits of this patch appear to make sense though.
>>
>> I think the most accurate description for these two patches is "brain
>> fart"
>
> ;-)
>
>> libxl: publish number of paged pages.
>>
>> This is information is currently not harvested from libxc.
>>
>> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
>
> But did you not want to print it from xl command or other?

Not in xl list (format compat with xm list). Where?
Andres

>
>>
>> diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl.c
>> --- a/tools/libxl/libxl.c
>> +++ b/tools/libxl/libxl.c
>> @@ -460,6 +460,7 @@ static void xcinfo2xlinfo(const xc_domai
>>
>>      xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
>>      xlinfo->shared_memkb = PAGE_TO_MEMKB(xcinfo->shr_pages);
>> +    xlinfo->paged_memkb = PAGE_TO_MEMKB(xcinfo->paged_pages);
>>      xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
>>      xlinfo->cpu_time = xcinfo->cpu_time;
>>      xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;
>> diff -r 5d20d2f6ffed -r 94b89a5f14e5 tools/libxl/libxl_types.idl
>> --- a/tools/libxl/libxl_types.idl
>> +++ b/tools/libxl/libxl_types.idl
>> @@ -175,6 +175,7 @@ libxl_dominfo = Struct("dominfo",[
>>      ("shutdown_reason", libxl_shutdown_reason),
>>      ("current_memkb",   MemKB),
>>      ("shared_memkb", MemKB),
>> +    ("paged_memkb", MemKB),
>>      ("max_memkb",   MemKB),
>>      ("cpu_time",    uint64),
>>      ("vcpu_max_id", uint32),
>>
>>
>
>
>

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Jackson]

Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
> > But did you not want to print it from xl command or other?
> 
> Not in xl list (format compat with xm list). Where?

Tricky.  Can you make the format compatible if paging is disabled ?

Ultimately the format of list is pretty annoying and inextensible.

Ian.

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Campbell]

On Tue, 2012-03-13 at 15:35 +0000, Ian Jackson wrote:
> Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
> > > But did you not want to print it from xl command or other?
> > 
> > Not in xl list (format compat with xm list). Where?
> 
> Tricky.  Can you make the format compatible if paging is disabled ?
> 
> Ultimately the format of list is pretty annoying and inextensible.

I was wondering if we should just bit the bullet and agree that only the
machine readable "xl list -l" output is backwards compatible? Supporting
machine parsing of the human readable "xl list" output is pretty
limiting in many ways.

The default for "xl list -l" is now json output (which is autogenerated
and will therefore pickup new field automatically) but there is an
option to use the xm sxp syntax too.

If changing the machine readable output is actually impossible I suppose
we could add a "--old-style" flag to ease the transition?

Ian.

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
> I was wondering if we should just bit the bullet and agree that only the
> machine readable "xl list -l" output is backwards compatible? Supporting
> machine parsing of the human readable "xl list" output is pretty
> limiting in many ways.

Mmmm.

> The default for "xl list -l" is now json output (which is autogenerated
> and will therefore pickup new field automatically) but there is an
> option to use the xm sxp syntax too.
> 
> If changing the machine readable output is actually impossible I suppose
> we could add a "--old-style" flag to ease the transition?

Can we make it an environment variable ?

Ian.

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Campbell]

On Tue, 2012-03-13 at 15:43 +0000, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
> > I was wondering if we should just bit the bullet and agree that only the
> > machine readable "xl list -l" output is backwards compatible? Supporting
> > machine parsing of the human readable "xl list" output is pretty
> > limiting in many ways.
> 
> Mmmm.
> 
> > The default for "xl list -l" is now json output (which is autogenerated
> > and will therefore pickup new field automatically) but there is an
> > option to use the xm sxp syntax too.
> > 
> > If changing the machine readable output is actually impossible I suppose
> > we could add a "--old-style" flag to ease the transition?
> 
> Can we make it an environment variable ?

I don't see why not. Say, $XL_USE_XM_COMPATIBLE_OUTPUT=1 ?

I suppose your rationale is that this makes it easier to fix a script
which might use either xm or xl?

Ian.

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
> I suppose your rationale is that this makes it easier to fix a script
> which might use either xm or xl?

Yes.  It means whole programs can be fixed more easily.

Ian.

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Andres Lagar-Cavilla]

> On Tue, 2012-03-13 at 15:35 +0000, Ian Jackson wrote:
>> Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl:
>> publish number of shared and paged pages"):
>> > > But did you not want to print it from xl command or other?
>> >
>> > Not in xl list (format compat with xm list). Where?
>>
>> Tricky.  Can you make the format compatible if paging is disabled ?
>>
>> Ultimately the format of list is pretty annoying and inextensible.
>
> I was wondering if we should just bit the bullet and agree that only the
> machine readable "xl list -l" output is backwards compatible? Supporting
> machine parsing of the human readable "xl list" output is pretty
> limiting in many ways.
>
> The default for "xl list -l" is now json output (which is autogenerated
> and will therefore pickup new field automatically) but there is an
> option to use the xm sxp syntax too.
>
> If changing the machine readable output is actually impossible I suppose
> we could add a "--old-style" flag to ease the transition?

Happy to add the paging and sharing stats to the extended list format,
once a choice is made.
Andres

>
> Ian.
>
>

Re: [PATCH 2 of 2] libxl: publish number of shared and paged pages

[Ian Jackson]

Andres Lagar-Cavilla writes ("Re: [Xen-devel] [PATCH 2 of 2] libxl: publish number of shared and paged pages"):
> Here is the libxl patch, rebased.
...
> libxl: publish number of paged pages.

Thanks.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>