[PATCH] xend: handle extended PCI configuration space when saving state

[PATCH] xend: handle extended PCI configuration space when saving state

[Matt Wilson]

From: Steven Noonan <snoonan@amazon.com>

Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended
configuration space which is larger than 256 bytes. This patch uses
stat() to determine the amount of space used to correctly save all of
the PCI configuration space. Resets handled by the xen-pciback driver
don't have this problem, as that code correctly handles saving
extended configuration space.

Signed-off-by: Steven Noonan <snoonan@amazon.com>
Reviewed-by: Matt Wilson <msw@amazon.com>
[msw: adjusted commit message]
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Matt Wilson <msw@amazon.com>
---
 tools/python/xen/util/pci.py |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/python/xen/util/pci.py b/tools/python/xen/util/pci.py
index 792fb69..adeca4b 100644
--- a/tools/python/xen/util/pci.py
+++ b/tools/python/xen/util/pci.py
@@ -521,8 +521,9 @@ def save_pci_conf_space(devs_string):
         pci_path = sysfs_mnt + SYSFS_PCI_DEVS_PATH + '/' + pci_str + \
                 SYSFS_PCI_DEV_CONFIG_PATH
         fd = os.open(pci_path, os.O_RDONLY)
+        size = os.fstat(fd).st_size
         configs = []
-        for i in range(0, 256, 4):
+        for i in range(0, size, 4):
             configs = configs + [os.read(fd,4)]
         os.close(fd)
         pci_list = pci_list + [pci_path]
-- 
1.7.9.5

Re: [PATCH] xend: handle extended PCI configuration space when saving state

[George Dunlap]

On Sat, Aug 31, 2013 at 12:40 AM, Matt Wilson <msw@linux.com> wrote:
> From: Steven Noonan <snoonan@amazon.com>
>
> Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended
> configuration space which is larger than 256 bytes. This patch uses
> stat() to determine the amount of space used to correctly save all of
> the PCI configuration space. Resets handled by the xen-pciback driver
> don't have this problem, as that code correctly handles saving
> extended configuration space.
>
> Signed-off-by: Steven Noonan <snoonan@amazon.com>
> Reviewed-by: Matt Wilson <msw@amazon.com>

Do you guys have plans for a migration over to xl?  xend has been
deprecated for two release cycles now, and I personally think 4.4 is a
good time to put it out of its misery.

If there is xend functionality you use which is missing from libxl, we
need to know so that we can re-implement it.

 -George

xend deprecation [Was: Re: [PATCH] xend: handle extended PCI configuration space when saving state]

[Matt Wilson]

On Mon, Sep 02, 2013 at 10:53:25AM +0100, George Dunlap wrote:
> 
> Do you guys have plans for a migration over to xl?  xend has been
> deprecated for two release cycles now, and I personally think 4.4 is a
> good time to put it out of its misery.

Work is underway, but there's lots to do to make that change. Making
the xend build optional in 4.4 is the first step, but I don't think we
should start removing it altogether yet.

> If there is xend functionality you use which is missing from libxl, we
> need to know so that we can re-implement it.

We'll need another sweep to verify that xl sets up everything in a way
that's compatible with all the OS images that run in EC2. E.g., other
problems like:
  http://lists.xen.org/archives/html/xen-devel/2013-04/msg02362.html

We'll need to look closely at the xenstore permissions for any
problems like XSA-57.

Live migration between a system running xend to a system that doesn't
support xend would be a nice-to-have. That probably means some support
for SXP parsing either in libxl or in some compatibility tool.

--msw

Re: xend deprecation [Was: Re: [PATCH] xend: handle extended PCI configuration space when saving state]

[Ian Campbell]

On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> Live migration between a system running xend to a system that doesn't
> support xend would be a nice-to-have. That probably means some support
> for SXP parsing either in libxl or in some compatibility tool. 

This already works by using xl on the source side, e.g. a guest started
with xend on 4.1 can be migrated with 4.1's xl to 4.2.

http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html

Ian.

Re: xend deprecation

[Matt Wilson]

On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > Live migration between a system running xend to a system that doesn't
> > support xend would be a nice-to-have. That probably means some support
> > for SXP parsing either in libxl or in some compatibility tool. 
> 
> This already works by using xl on the source side, e.g. a guest started
> with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> 
> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html

xl doesn't run everywhere that we need it to.

--msw

Re: xend deprecation

[Ian Campbell]

On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > > Live migration between a system running xend to a system that doesn't
> > > support xend would be a nice-to-have. That probably means some support
> > > for SXP parsing either in libxl or in some compatibility tool. 
> > 
> > This already works by using xl on the source side, e.g. a guest started
> > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> > 
> > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> 
> xl doesn't run everywhere that we need it to.

Details please, or we aren't going to be able to even think about how to
solve this for you.

BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
in 4.2) and the use 4.2's xl to do a localhost migrate or a second
remote migrate to another 4.2 system.

xend's deprecation has been on the cards, and widely known, for several
releases now. It is time to quit stalling and start moving. I intend to
post a patch deleting it as soon as 4.4 is out the door.

Of course you are also free to pick up xend maintenance if you really
want to keep it alive. We could for example move it out of tree. And by
"We" I mean "You and those others who are interested in continuing with
xend".

Ian.

Re: xend deprecation

[Konrad Rzeszutek Wilk]

On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > > > Live migration between a system running xend to a system that doesn't
> > > > support xend would be a nice-to-have. That probably means some support
> > > > for SXP parsing either in libxl or in some compatibility tool. 
> > > 
> > > This already works by using xl on the source side, e.g. a guest started
> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> > > 
> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> > 
> > xl doesn't run everywhere that we need it to.
> 
> Details please, or we aren't going to be able to even think about how to
> solve this for you.
> 
> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
> remote migrate to another 4.2 system.
> 
> xend's deprecation has been on the cards, and widely known, for several
> releases now. It is time to quit stalling and start moving. I intend to
> post a patch deleting it as soon as 4.4 is out the door.

And I think I might have to lobby hard against removing it.

There are issues that are stopping us from using it:

 - No status in xl list -l when only dom0 is present.
 - 'xl list -l' says 'Domain name must be specified' even though the '-l'
   is for 'Output all VM details'
 - xl list -l for PV guest has bug: it’s not formated S-expression text.
 - No console in xl list -l, so can't get  tty console port.
 - xl uses SSH for migration. We need SSL daemon or socat like xend has.

Those should be fixed at some point :-) Problem is that other things
keep on popping up on the radar and hence we end up using xend.

CC-ing Zhigang here as he might have a more up-to-date list. I think
there were some statistics too, but I can't recall.

> 
> Of course you are also free to pick up xend maintenance if you really
> want to keep it alive. We could for example move it out of tree. And by
> "We" I mean "You and those others who are interested in continuing with
> xend".

Hahahah. I think we all want to move away but there are things that are
stopping us and we would like them to magically get fixed. And I also
want a pony. With a barrel of guinness attached to it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xend deprecation

[George Dunlap]

On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
>> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
>> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
>> > > > Live migration between a system running xend to a system that doesn't
>> > > > support xend would be a nice-to-have. That probably means some support
>> > > > for SXP parsing either in libxl or in some compatibility tool.
>> > >
>> > > This already works by using xl on the source side, e.g. a guest started
>> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
>> > >
>> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
>> >
>> > xl doesn't run everywhere that we need it to.
>>
>> Details please, or we aren't going to be able to even think about how to
>> solve this for you.
>>
>> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
>> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
>> remote migrate to another 4.2 system.
>>
>> xend's deprecation has been on the cards, and widely known, for several
>> releases now. It is time to quit stalling and start moving. I intend to
>> post a patch deleting it as soon as 4.4 is out the door.
>
> And I think I might have to lobby hard against removing it.
>
> There are issues that are stopping us from using it:
>
>  - No status in xl list -l when only dom0 is present.
>  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
>    is for 'Output all VM details'
>  - xl list -l for PV guest has bug: it’s not formated S-expression text.
>  - No console in xl list -l, so can't get  tty console port.
>  - xl uses SSH for migration. We need SSL daemon or socat like xend has.
>
> Those should be fixed at some point :-) Problem is that other things
> keep on popping up on the radar and hence we end up using xend.

Uum, when were you going to report these?  A lot of us were under the
impression that xl behaved sufficiently like xend for anyone who
cared.

Maybe we should make an announcement (on blog, xen-users, xen-devel,
and xen-announce) that there has been a proposal to make 4.4 the last
release with xend, and so it's important to report all bugs / missing
features of xl so we can get them fixed by the 4.4 release?

 -George

Re: xend deprecation

[Ian Campbell]

On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
> >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
> >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> >> > > > Live migration between a system running xend to a system that doesn't
> >> > > > support xend would be a nice-to-have. That probably means some support
> >> > > > for SXP parsing either in libxl or in some compatibility tool.
> >> > >
> >> > > This already works by using xl on the source side, e.g. a guest started
> >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> >> > >
> >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> >> >
> >> > xl doesn't run everywhere that we need it to.
> >>
> >> Details please, or we aren't going to be able to even think about how to
> >> solve this for you.
> >>
> >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
> >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
> >> remote migrate to another 4.2 system.
> >>
> >> xend's deprecation has been on the cards, and widely known, for several
> >> releases now. It is time to quit stalling and start moving. I intend to
> >> post a patch deleting it as soon as 4.4 is out the door.
> >
> > And I think I might have to lobby hard against removing it.

If by lobby you mean "step up to maintain" then please do.

> > There are issues that are stopping us from using it:
> >
> >  - No status in xl list -l when only dom0 is present.

xl list -l produces a tonne of output here when only dom0 is present.

> >  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
> >    is for 'Output all VM details'

Not here it doesn't.

IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed.

> >  - xl list -l for PV guest has bug: it’s not formated S-expression text.

This is a feature not a bug. However you can request an sxp
compatibility mode (I forget how, check the manpage). Note that this
doesn't include new features, so you would be advised to make plans to
switch to consuming the json variant.

> >  - No console in xl list -l, so can't get  tty console port.

That one should be easily fixable

> >  - xl uses SSH for migration. We need SSL daemon or socat like xend has.

You can provide any tool you like to run instead of ssh, including an
ssl daemon/client or socat. See the manpage.

> >
> > Those should be fixed at some point :-) Problem is that other things
> > keep on popping up on the radar and hence we end up using xend.
> 
> Uum, when were you going to report these?  A lot of us were under the
> impression that xl behaved sufficiently like xend for anyone who
> cared.
> 
> Maybe we should make an announcement (on blog, xen-users, xen-devel,
> and xen-announce) that there has been a proposal to make 4.4 the last
> release with xend, and so it's important to report all bugs / missing
> features of xl so we can get them fixed by the 4.4 release?

Didn't we do this for 4.2 or 4.3? We didn't get much feedback.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xend deprecation

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] xend deprecation"):
> > On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
> > <konrad.wilk@oracle.com> wrote:
> > >  - xl uses SSH for migration. We need SSL daemon or socat like xend has.
> 
> You can provide any tool you like to run instead of ssh, including an
> ssl daemon/client or socat. See the manpage.

If there is some migration mode that you'd like to see natively
supported in xl, besides "fork this program which looks a bit like
ssh", we'd be very happy to take patches or even write them.

Ian.

Re: xend deprecation

[Ian Campbell]

create !
title it xl improve support for migration over non-sshlike tunnels
thanks

On Wed, 2013-09-04 at 16:43 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] xend deprecation"):
> > > On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
> > > <konrad.wilk@oracle.com> wrote:
> > > >  - xl uses SSH for migration. We need SSL daemon or socat like xend has.
> > 
> > You can provide any tool you like to run instead of ssh, including an
> > ssl daemon/client or socat. See the manpage.
> 
> If there is some migration mode that you'd like to see natively
> supported in xl, besides "fork this program which looks a bit like
> ssh", we'd be very happy to take patches or even write them.

Indeed.

Konrad, what is the exact requirement here? You mention socat which just
takes an IP address and connects to it and pipes stuff back and forth
(crypto optional). I suspect the sending side of that can be done with a
thin wrapper which provides the 'ssh like' command line interface, so
what is needed is a daemon listening on a known port which can fork +
exec xl migrate-receive for each connection?

Is socat actually what you want or was that just an example?

Ian.

Processed: Re: xend deprecation

[xen]

Processing commands for xen@bugs.xenproject.org:

> create !
Created new bug #18 rooted at `<1378980129.10076.33.camel@kazak.uk.xensource.com>'
Title: `Re: [Xen-devel] xend deprecation'
> title it xl improve support for migration over non-sshlike tunnels
Set title for #18 to `xl improve support for migration over non-sshlike tunnels'
> thanks
Finished processing.

Modified/created Bugs:
 - 18: http://bugs.xenproject.org/xen/bug/18 (new)

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues

Re: xend deprecation

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] xend deprecation"):
> create !
> title it xl improve support for migration over non-sshlike tunnels
> thanks
...
> Konrad, what is the exact requirement here? You mention socat which just
> takes an IP address and connects to it and pipes stuff back and forth
> (crypto optional). I suspect the sending side of that can be done with a
> thin wrapper which provides the 'ssh like' command line interface,

I think it can be done with the existing xl interface, TBH, but it's
probably ugly.  I would have no problem with an option which made xl
make the network connection itself.

> so
> what is needed is a daemon listening on a known port which can fork +
> exec xl migrate-receive for each connection?

I think this should be doable by running xl migrate-receive out of
inetd, or some kind of socket-activation-based feature of an
init-replacement (eg upstart).

If not this should be fixed.

Ian.

Re: xend deprecation

[Tim Deegan]

At 16:34 +0100 on 04 Sep (1378312470), Ian Campbell wrote:
> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
> > Maybe we should make an announcement (on blog, xen-users, xen-devel,
> > and xen-announce) that there has been a proposal to make 4.4 the last
> > release with xend, and so it's important to report all bugs / missing
> > features of xl so we can get them fixed by the 4.4 release?
> 
> Didn't we do this for 4.2 or 4.3? We didn't get much feedback.

Indeed.  The 4.2 release notes say: 

  The XEND toolstack is now deprecated and should be expected to be
  removed in the 4.3 timescale. Therefore it is strongly recommended
  that users evaluate Xen 4.2 with XL for their use cases and report any
  omissions or blockers such that they can be addressed for 4.3. Fixes
  for serious functionality gaps will also be considered for backport to
  the 4.2.x releases.

I guess we're now on "expected to be removed in the 4.5 timescale"?

Tim.

Re: xend deprecation

[Ian Jackson]

Tim Deegan writes ("Re: [Xen-devel] xend deprecation"):
> At 16:34 +0100 on 04 Sep (1378312470), Ian Campbell wrote:
...
> > Didn't we do this for 4.2 or 4.3? We didn't get much feedback.
> 
> Indeed.  The 4.2 release notes say: 
> 
>   The XEND toolstack is now deprecated and should be expected to be
>   removed in the 4.3 timescale. Therefore it is strongly recommended
>   that users evaluate Xen 4.2 with XL for their use cases and report any
>   omissions or blockers such that they can be addressed for 4.3. Fixes
>   for serious functionality gaps will also be considered for backport to
>   the 4.2.x releases.
> 
> I guess we're now on "expected to be removed in the 4.5 timescale"?

Surely we can still remove it in 4.4, having advertised that we were
going to remove it in 4.3.

Ian.

Re: xend deprecation

[Ian Campbell]

On Wed, 2013-09-04 at 17:17 +0100, Ian Jackson wrote:
> Tim Deegan writes ("Re: [Xen-devel] xend deprecation"):
> > At 16:34 +0100 on 04 Sep (1378312470), Ian Campbell wrote:
> ...
> > > Didn't we do this for 4.2 or 4.3? We didn't get much feedback.
> > 
> > Indeed.  The 4.2 release notes say: 
> > 
> >   The XEND toolstack is now deprecated and should be expected to be
> >   removed in the 4.3 timescale. Therefore it is strongly recommended
> >   that users evaluate Xen 4.2 with XL for their use cases and report any
> >   omissions or blockers such that they can be addressed for 4.3. Fixes
> >   for serious functionality gaps will also be considered for backport to
> >   the 4.2.x releases.
> > 
> > I guess we're now on "expected to be removed in the 4.5 timescale"?
> 
> Surely we can still remove it in 4.4, having advertised that we were
> going to remove it in 4.3.

Well, either that or the people insisting it remain need to step up and
take over maintenance. Amazon appear to be offering to do that, at least
to some extent.

I nearly said "remain Supported" above, but the reality is that it
effectively hasn't been Supported by this community for many releases
now. I don't know to what extent Matt and Steven are offering to
maintain it. Depending on the answer to that we should consider moving
xend to a separate tree.

Ian.

Re: xend deprecation

[Konrad Rzeszutek Wilk]

On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote:
> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
> > On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
> > <konrad.wilk@oracle.com> wrote:
> > > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
> > >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
> > >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> > >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > >> > > > Live migration between a system running xend to a system that doesn't
> > >> > > > support xend would be a nice-to-have. That probably means some support
> > >> > > > for SXP parsing either in libxl or in some compatibility tool.
> > >> > >
> > >> > > This already works by using xl on the source side, e.g. a guest started
> > >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> > >> > >
> > >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> > >> >
> > >> > xl doesn't run everywhere that we need it to.
> > >>
> > >> Details please, or we aren't going to be able to even think about how to
> > >> solve this for you.
> > >>
> > >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
> > >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
> > >> remote migrate to another 4.2 system.
> > >>
> > >> xend's deprecation has been on the cards, and widely known, for several
> > >> releases now. It is time to quit stalling and start moving. I intend to
> > >> post a patch deleting it as soon as 4.4 is out the door.
> > >
> > > And I think I might have to lobby hard against removing it.
> 
> If by lobby you mean "step up to maintain" then please do.

I can surely do that, but I would prefer to have the issues fixed.

> 
> > > There are issues that are stopping us from using it:
> > >
> > >  - No status in xl list -l when only dom0 is present.
> 
> xl list -l produces a tonne of output here when only dom0 is present.
> 
> > >  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
> > >    is for 'Output all VM details'
> 
> Not here it doesn't.
> 
> IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed.

Odd, I am running Xen 4.3. Hm, time to rebase.

> 
> > >  - xl list -l for PV guest has bug: it’s not formated S-expression text.
> 
> This is a feature not a bug. However you can request an sxp
> compatibility mode (I forget how, check the manpage). Note that this
> doesn't include new features, so you would be advised to make plans to
> switch to consuming the json variant.
> 
> > >  - No console in xl list -l, so can't get  tty console port.
> 
> That one should be easily fixable
> 
> > >  - xl uses SSH for migration. We need SSL daemon or socat like xend has.
> 
> You can provide any tool you like to run instead of ssh, including an
> ssl daemon/client or socat. See the manpage.

Right, except that the socat part does not work as the 'ssh' arguments
are encoded in the code. I am not saying that these are 'earth-shattering hard
to fix' it just that they should be done before Xend is gone.

> 
> > >
> > > Those should be fixed at some point :-) Problem is that other things
> > > keep on popping up on the radar and hence we end up using xend.
> > 
> > Uum, when were you going to report these?  A lot of us were under the
> > impression that xl behaved sufficiently like xend for anyone who
> > cared.
> > 
> > Maybe we should make an announcement (on blog, xen-users, xen-devel,
> > and xen-announce) that there has been a proposal to make 4.4 the last
> > release with xend, and so it's important to report all bugs / missing
> > features of xl so we can get them fixed by the 4.4 release?
> 
> Didn't we do this for 4.2 or 4.3? We didn't get much feedback.

No idea.
> 
> Ian.
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xend deprecation

[George Dunlap]

On 04/09/13 17:33, Konrad Rzeszutek Wilk wrote:
> On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote:
>> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
>>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
>>> <konrad.wilk@oracle.com> wrote:
>>>> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
>>>>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
>>>>>> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
>>>>>>> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
>>>>>>>> Live migration between a system running xend to a system that doesn't
>>>>>>>> support xend would be a nice-to-have. That probably means some support
>>>>>>>> for SXP parsing either in libxl or in some compatibility tool.
>>>>>>> This already works by using xl on the source side, e.g. a guest started
>>>>>>> with xend on 4.1 can be migrated with 4.1's xl to 4.2.
>>>>>>>
>>>>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
>>>>>> xl doesn't run everywhere that we need it to.
>>>>> Details please, or we aren't going to be able to even think about how to
>>>>> solve this for you.
>>>>>
>>>>> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
>>>>> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
>>>>> remote migrate to another 4.2 system.
>>>>>
>>>>> xend's deprecation has been on the cards, and widely known, for several
>>>>> releases now. It is time to quit stalling and start moving. I intend to
>>>>> post a patch deleting it as soon as 4.4 is out the door.
>>>> And I think I might have to lobby hard against removing it.
>> If by lobby you mean "step up to maintain" then please do.
> I can surely do that, but I would prefer to have the issues fixed.

You mean you prefer to have the xl issues fixed? I think we all do; so 
instead of lobbying hard to remove xend, you can lobby hard to get the 
outstanding issues fixed. :-)

  -George

Re: xend deprecation

[Konrad Rzeszutek Wilk]

On Thu, Sep 05, 2013 at 10:12:51AM +0100, George Dunlap wrote:
> On 04/09/13 17:33, Konrad Rzeszutek Wilk wrote:
> >On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote:
> >>On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
> >>>On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
> >>><konrad.wilk@oracle.com> wrote:
> >>>>On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
> >>>>>On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
> >>>>>>On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> >>>>>>>On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> >>>>>>>>Live migration between a system running xend to a system that doesn't
> >>>>>>>>support xend would be a nice-to-have. That probably means some support
> >>>>>>>>for SXP parsing either in libxl or in some compatibility tool.
> >>>>>>>This already works by using xl on the source side, e.g. a guest started
> >>>>>>>with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> >>>>>>>
> >>>>>>>http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> >>>>>>xl doesn't run everywhere that we need it to.
> >>>>>Details please, or we aren't going to be able to even think about how to
> >>>>>solve this for you.
> >>>>>
> >>>>>BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
> >>>>>in 4.2) and the use 4.2's xl to do a localhost migrate or a second
> >>>>>remote migrate to another 4.2 system.
> >>>>>
> >>>>>xend's deprecation has been on the cards, and widely known, for several
> >>>>>releases now. It is time to quit stalling and start moving. I intend to
> >>>>>post a patch deleting it as soon as 4.4 is out the door.
> >>>>And I think I might have to lobby hard against removing it.
> >>If by lobby you mean "step up to maintain" then please do.
> >I can surely do that, but I would prefer to have the issues fixed.
> 
> You mean you prefer to have the xl issues fixed? I think we all do;
> so instead of lobbying hard to remove xend, you can lobby hard to
> get the outstanding issues fixed. :-)

Of course I would love to have the issues fixes. But I am running
in a time management problem and I don't thing we will get them all
(along with the other changes we (Oracle) would like to get done)
by the deadline.

Hence as a safety net I am proposing that if we don't get them done,
we postpone the killing of xend.

Re: xend deprecation

[Ian Campbell]

On Fri, 2013-09-06 at 09:36 -0400, Konrad Rzeszutek Wilk wrote:
> Hence as a safety net I am proposing that if we don't get them done,
> we postpone the killing of xend.

It's been two releases now, how many more do you want?

We can't just be keeping unmaintained code around indefinitely. Are you
volunteering to add your name to MAINTAINERS file next to Matt and
Steven?

I would also note that xend being removed from upstream xen does not in
any way preclude your continuing to use and develop it. However removing
it does remove a bunch of confusion from our end users about what they
should use and removes a burden from the project as a whole.

Ian

Re: xend deprecation

[Jan Beulich]

>>> On 06.09.13 at 15:49, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Fri, 2013-09-06 at 09:36 -0400, Konrad Rzeszutek Wilk wrote:
>> Hence as a safety net I am proposing that if we don't get them done,
>> we postpone the killing of xend.
> 
> It's been two releases now, how many more do you want?
> 
> We can't just be keeping unmaintained code around indefinitely. Are you
> volunteering to add your name to MAINTAINERS file next to Matt and
> Steven?
> 
> I would also note that xend being removed from upstream xen does not in
> any way preclude your continuing to use and develop it. However removing
> it does remove a bunch of confusion from our end users about what they
> should use and removes a burden from the project as a whole.

Can't we, for 4.4 at least, simply disable CONFIG_XEND by default,
to make the earlier verbal announcement more obvious to consumers
of the tree? (That's not to say that I'd like it kept; I'm just saying
that knowing that verbal announcements of planned feature removal
is usually less efficient than forcing people to re-enable their feature,
thus - hopefully - grokking the news.)

Jan

Re: xend deprecation

[Ian Campbell]

On Fri, 2013-09-06 at 15:01 +0100, Jan Beulich wrote:
> >>> On 06.09.13 at 15:49, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Fri, 2013-09-06 at 09:36 -0400, Konrad Rzeszutek Wilk wrote:
> >> Hence as a safety net I am proposing that if we don't get them done,
> >> we postpone the killing of xend.
> > 
> > It's been two releases now, how many more do you want?
> > 
> > We can't just be keeping unmaintained code around indefinitely. Are you
> > volunteering to add your name to MAINTAINERS file next to Matt and
> > Steven?
> > 
> > I would also note that xend being removed from upstream xen does not in
> > any way preclude your continuing to use and develop it. However removing
> > it does remove a bunch of confusion from our end users about what they
> > should use and removes a burden from the project as a whole.
> 
> Can't we, for 4.4 at least, simply disable CONFIG_XEND by default,
> to make the earlier verbal announcement more obvious to consumers
> of the tree?

I think this is the minimum we should do.

Ian.

Re: xend deprecation

[Fabio Fantoni]

Il 04/09/2013 18:33, Konrad Rzeszutek Wilk ha scritto:
> On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote:
>> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
>>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
>>> <konrad.wilk@oracle.com> wrote:
>>>> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
>>>>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
>>>>>> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
>>>>>>> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
>>>>>>>> Live migration between a system running xend to a system that doesn't
>>>>>>>> support xend would be a nice-to-have. That probably means some support
>>>>>>>> for SXP parsing either in libxl or in some compatibility tool.
>>>>>>> This already works by using xl on the source side, e.g. a guest started
>>>>>>> with xend on 4.1 can be migrated with 4.1's xl to 4.2.
>>>>>>>
>>>>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
>>>>>> xl doesn't run everywhere that we need it to.
>>>>> Details please, or we aren't going to be able to even think about how to
>>>>> solve this for you.
>>>>>
>>>>> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
>>>>> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
>>>>> remote migrate to another 4.2 system.
>>>>>
>>>>> xend's deprecation has been on the cards, and widely known, for several
>>>>> releases now. It is time to quit stalling and start moving. I intend to
>>>>> post a patch deleting it as soon as 4.4 is out the door.
>>>> And I think I might have to lobby hard against removing it.
>> If by lobby you mean "step up to maintain" then please do.
> I can surely do that, but I would prefer to have the issues fixed.
>
>>>> There are issues that are stopping us from using it:
>>>>
>>>>   - No status in xl list -l when only dom0 is present.
>> xl list -l produces a tonne of output here when only dom0 is present.
>>
>>>>   - 'xl list -l' says 'Domain name must be specified' even though the '-l'
>>>>     is for 'Output all VM details'
>> Not here it doesn't.
>>
>> IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed.
> Odd, I am running Xen 4.3. Hm, time to rebase.
>
>>>>   - xl list -l for PV guest has bug: it’s not formated S-expression text.
>> This is a feature not a bug. However you can request an sxp
>> compatibility mode (I forget how, check the manpage). Note that this
>> doesn't include new features, so you would be advised to make plans to
>> switch to consuming the json variant.
>>
>>>>   - No console in xl list -l, so can't get  tty console port.
>> That one should be easily fixable
>>
>>>>   - xl uses SSH for migration. We need SSL daemon or socat like xend has.
>> You can provide any tool you like to run instead of ssh, including an
>> ssl daemon/client or socat. See the manpage.
> Right, except that the socat part does not work as the 'ssh' arguments
> are encoded in the code. I am not saying that these are 'earth-shattering hard
> to fix' it just that they should be done before Xend is gone.
>
>>>> Those should be fixed at some point :-) Problem is that other things
>>>> keep on popping up on the radar and hence we end up using xend.
>>> Uum, when were you going to report these?  A lot of us were under the
>>> impression that xl behaved sufficiently like xend for anyone who
>>> cared.
>>>
>>> Maybe we should make an announcement (on blog, xen-users, xen-devel,
>>> and xen-announce) that there has been a proposal to make 4.4 the last
>>> release with xend, and so it's important to report all bugs / missing
>>> features of xl so we can get them fixed by the 4.4 release?

I think is a good idea, if will not removed on xen 4.4 at least disabled 
it by default to get more visible the new xl to the users and get more 
feedback.

About very small thing present in xend and not in xl not reported for 
now is the autocompletion name of domU in commandsmissed.

I think is good do a similar thing with qemu traditional with 
deprecation and massive comunication of a certain deadline.
Also in this case will be good disabling it by default to accentuate its 
deprecation.
Upstream qemu needs more testing/support to have hvm domUs 
performance/features/bugfixes improved.

>> Didn't we do this for 4.2 or 4.3? We didn't get much feedback.
> No idea.
>> Ian.
>>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xend deprecation

[George Dunlap]

On 05/09/13 11:18, Fabio Fantoni wrote:
> Il 04/09/2013 18:33, Konrad Rzeszutek Wilk ha scritto:
>> On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote:
>>> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
>>>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
>>>> <konrad.wilk@oracle.com> wrote:
>>>>> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
>>>>>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
>>>>>>> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
>>>>>>>> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
>>>>>>>>> Live migration between a system running xend to a system that 
>>>>>>>>> doesn't
>>>>>>>>> support xend would be a nice-to-have. That probably means some 
>>>>>>>>> support
>>>>>>>>> for SXP parsing either in libxl or in some compatibility tool.
>>>>>>>> This already works by using xl on the source side, e.g. a guest 
>>>>>>>> started
>>>>>>>> with xend on 4.1 can be migrated with 4.1's xl to 4.2.
>>>>>>>>
>>>>>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
>>>>>>> xl doesn't run everywhere that we need it to.
>>>>>> Details please, or we aren't going to be able to even think about 
>>>>>> how to
>>>>>> solve this for you.
>>>>>>
>>>>>> BTW You can also do a xend migration to e.g. 4.2 (using the xend 
>>>>>> whichis
>>>>>> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
>>>>>> remote migrate to another 4.2 system.
>>>>>>
>>>>>> xend's deprecation has been on the cards, and widely known, for 
>>>>>> several
>>>>>> releases now. It is time to quit stalling and start moving. I 
>>>>>> intend to
>>>>>> post a patch deleting it as soon as 4.4 is out the door.
>>>>> And I think I might have to lobby hard against removing it.
>>> If by lobby you mean "step up to maintain" then please do.
>> I can surely do that, but I would prefer to have the issues fixed.
>>
>>>>> There are issues that are stopping us from using it:
>>>>>
>>>>>   - No status in xl list -l when only dom0 is present.
>>> xl list -l produces a tonne of output here when only dom0 is present.
>>>
>>>>>   - 'xl list -l' says 'Domain name must be specified' even though 
>>>>> the '-l'
>>>>>     is for 'Output all VM details'
>>> Not here it doesn't.
>>>
>>> IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed.
>> Odd, I am running Xen 4.3. Hm, time to rebase.
>>
>>>>>   - xl list -l for PV guest has bug: it’s not formated 
>>>>> S-expression text.
>>> This is a feature not a bug. However you can request an sxp
>>> compatibility mode (I forget how, check the manpage). Note that this
>>> doesn't include new features, so you would be advised to make plans to
>>> switch to consuming the json variant.
>>>
>>>>>   - No console in xl list -l, so can't get  tty console port.
>>> That one should be easily fixable
>>>
>>>>>   - xl uses SSH for migration. We need SSL daemon or socat like 
>>>>> xend has.
>>> You can provide any tool you like to run instead of ssh, including an
>>> ssl daemon/client or socat. See the manpage.
>> Right, except that the socat part does not work as the 'ssh' arguments
>> are encoded in the code. I am not saying that these are 
>> 'earth-shattering hard
>> to fix' it just that they should be done before Xend is gone.
>>
>>>>> Those should be fixed at some point :-) Problem is that other things
>>>>> keep on popping up on the radar and hence we end up using xend.
>>>> Uum, when were you going to report these?  A lot of us were under the
>>>> impression that xl behaved sufficiently like xend for anyone who
>>>> cared.
>>>>
>>>> Maybe we should make an announcement (on blog, xen-users, xen-devel,
>>>> and xen-announce) that there has been a proposal to make 4.4 the last
>>>> release with xend, and so it's important to report all bugs / missing
>>>> features of xl so we can get them fixed by the 4.4 release?
>
> I think is a good idea, if will not removed on xen 4.4 at least 
> disabled it by default to get more visible the new xl to the users and 
> get more feedback.
>
> About very small thing present in xend and not in xl not reported for 
> now is the autocompletion name of domU in commandsmissed.
>
> I think is good do a similar thing with qemu traditional with 
> deprecation and massive comunication of a certain deadline.

qemu is a bit different than xend: it is entirely possible to migrate a 
VM started with xend to a system using only xl.  I don't think the same 
is true for qemu; so we will probably be keeping qemu-traditional around 
for some time after qemu-xen has reached feature parity (which it still 
has yet to do).

  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xend deprecation

[George Dunlap]

On Wed, Sep 4, 2013 at 4:34 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:
>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
>> <konrad.wilk@oracle.com> wrote:
>> > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
>> >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
>> >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
>> >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
>> >> > > > Live migration between a system running xend to a system that doesn't
>> >> > > > support xend would be a nice-to-have. That probably means some support
>> >> > > > for SXP parsing either in libxl or in some compatibility tool.
>> >> > >
>> >> > > This already works by using xl on the source side, e.g. a guest started
>> >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
>> >> > >
>> >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
>> >> >
>> >> > xl doesn't run everywhere that we need it to.
>> >>
>> >> Details please, or we aren't going to be able to even think about how to
>> >> solve this for you.
>> >>
>> >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
>> >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
>> >> remote migrate to another 4.2 system.
>> >>
>> >> xend's deprecation has been on the cards, and widely known, for several
>> >> releases now. It is time to quit stalling and start moving. I intend to
>> >> post a patch deleting it as soon as 4.4 is out the door.
>> >
>> > And I think I might have to lobby hard against removing it.
>
> If by lobby you mean "step up to maintain" then please do.
>
>> > There are issues that are stopping us from using it:
>> >
>> >  - No status in xl list -l when only dom0 is present.
>
> xl list -l produces a tonne of output here when only dom0 is present.

FWIW:

# xl list -l
[

]

This is from c/s 0c006b41a283a0a569c863d44abde5aa5750ae01, dated August 13.

  -George

Re: xend deprecation

[Konrad Rzeszutek Wilk]

On Wed, Sep 04, 2013 at 04:18:49PM +0100, George Dunlap wrote:
> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:
> >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:
> >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> >> > > > Live migration between a system running xend to a system that doesn't
> >> > > > support xend would be a nice-to-have. That probably means some support
> >> > > > for SXP parsing either in libxl or in some compatibility tool.
> >> > >
> >> > > This already works by using xl on the source side, e.g. a guest started
> >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> >> > >
> >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> >> >
> >> > xl doesn't run everywhere that we need it to.
> >>
> >> Details please, or we aren't going to be able to even think about how to
> >> solve this for you.
> >>
> >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis
> >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second
> >> remote migrate to another 4.2 system.
> >>
> >> xend's deprecation has been on the cards, and widely known, for several
> >> releases now. It is time to quit stalling and start moving. I intend to
> >> post a patch deleting it as soon as 4.4 is out the door.
> >
> > And I think I might have to lobby hard against removing it.
> >
> > There are issues that are stopping us from using it:
> >
> >  - No status in xl list -l when only dom0 is present.
> >  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
> >    is for 'Output all VM details'
> >  - xl list -l for PV guest has bug: it’s not formated S-expression text.
> >  - No console in xl list -l, so can't get  tty console port.
> >  - xl uses SSH for migration. We need SSL daemon or socat like xend has.
> >
> > Those should be fixed at some point :-) Problem is that other things
> > keep on popping up on the radar and hence we end up using xend.
> 
> Uum, when were you going to report these?  A lot of us were under the
> impression that xl behaved sufficiently like xend for anyone who
> cared.

I got overwhelmed with other things and this dropped from my radar.
This thread got me digging through my notes.

> 
> Maybe we should make an announcement (on blog, xen-users, xen-devel,
> and xen-announce) that there has been a proposal to make 4.4 the last
> release with xend, and so it's important to report all bugs / missing
> features of xl so we can get them fixed by the 4.4 release?

Sounds sensible.
> 
>  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH] xl: print runtime info in "xl list -l"

[Ian Campbell]

On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:
>  - No status in xl list -l when only dom0 is present. 

Which bits of status are you interested in?

I'm thinking something like the following, which will include the
content of libxl_dominfo for every domain.

8<----------------------------------

>From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Thu, 12 Sep 2013 13:50:33 +0100
Subject: [PATCH] xl: print runtime info in "xl list -l"

Include dom0 runtime information, but not domain config.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 tools/libxl/xl_cmdimpl.c |   74 ++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 28 deletions(-)

diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 884f050..46fdafc 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -298,7 +298,8 @@ static void dolog(const char *file, int line, const char *func, char *fmt, ...)
 }
 
 static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid,
-                                            libxl_domain_config *d_config)
+                                            libxl_domain_config *d_config,
+                                            libxl_dominfo *d_info)
 {
     yajl_gen_status s;
 
@@ -317,13 +318,25 @@ static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid,
     if (s != yajl_gen_status_ok)
         goto out;
 
-    s = yajl_gen_string(hand, (const unsigned char *)"config",
-                        sizeof("config")-1);
-    if (s != yajl_gen_status_ok)
-        goto out;
-    s = libxl_domain_config_gen_json(hand, d_config);
-    if (s != yajl_gen_status_ok)
-        goto out;
+    if (d_config) {
+        s = yajl_gen_string(hand, (const unsigned char *)"config",
+                            sizeof("config")-1);
+        if (s != yajl_gen_status_ok)
+            goto out;
+        s = libxl_domain_config_gen_json(hand, d_config);
+        if (s != yajl_gen_status_ok)
+            goto out;
+    }
+
+    if (d_info) {
+        s = yajl_gen_string(hand, (const unsigned char *)"info",
+                            sizeof("info")-1);
+        if (s != yajl_gen_status_ok)
+            goto out;
+        s = libxl_dominfo_gen_json(hand, d_info);
+        if (s != yajl_gen_status_ok)
+            goto out;
+    }
 
     s = yajl_gen_map_close(hand);
     if (s != yajl_gen_status_ok)
@@ -350,7 +363,7 @@ static void printf_info(enum output_format output_format,
         return;
     }
 
-    s = printf_info_one_json(hand, domid, d_config);
+    s = printf_info_one_json(hand, domid, d_config, NULL);
     if (s != yajl_gen_status_ok)
         goto out;
 
@@ -2990,7 +3003,7 @@ static void reboot_domain(uint32_t domid, libxl_evgen_domain_death **deathw,
     }
 }
 
-static void list_domains_details(const libxl_dominfo *info, int nb_domain)
+static void list_domains_details(libxl_dominfo *info, int nb_domain)
 {
     libxl_domain_config d_config;
 
@@ -3017,26 +3030,31 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain)
         s = yajl_gen_status_ok;
 
     for (i = 0; i < nb_domain; i++) {
-        /* no detailed info available on dom0 */
-        if (info[i].domid == 0)
-            continue;
-        rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
-        if (rc)
-            continue;
-        CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid));
-        libxl_domain_config_init(&d_config);
-        parse_config_data(config_source, (char *)data, len, &d_config, NULL);
-        if (default_output_format == OUTPUT_FORMAT_JSON)
-            s = printf_info_one_json(hand, info[i].domid, &d_config);
-        else
+        fprintf(stderr, "dom%d\n", info[i].domid);
+
+        if ( info[i].domid > 0 ) {
+            fprintf(stderr, "parse d_config for dom%d\n", info[i].domid);
+            rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
+            if (rc)
+                continue;
+            CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid));
+            libxl_domain_config_init(&d_config);
+            parse_config_data(config_source, (char *)data, len, &d_config, NULL);
+        } else { fprintf(stderr, "no d_config\n"); }
+        if (default_output_format == OUTPUT_FORMAT_JSON) {
+            s = printf_info_one_json(hand, info[i].domid,
+                                     i == 0 ? NULL : &d_config, &info[i]);
+            if (s != yajl_gen_status_ok)
+                goto out;
+        } else
             printf_info_sexp(info[i].domid, &d_config);
-        libxl_domain_config_dispose(&d_config);
-        free(data);
-        free(config_source);
-        if (s != yajl_gen_status_ok)
-            goto out;
-    }
 
+        if ( info[i].domid > 0 ) {
+            libxl_domain_config_dispose(&d_config);
+            free(data);
+            free(config_source);
+        }
+    }
     if (default_output_format == OUTPUT_FORMAT_JSON) {
         s = yajl_gen_array_close(hand);
         if (s != yajl_gen_status_ok)
-- 
1.7.10.4

Re: [PATCH] xl: print runtime info in "xl list -l"

[Konrad Rzeszutek Wilk]

On Thu, Sep 12, 2013 at 01:52:09PM +0100, Ian Campbell wrote:
> On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:
> >  - No status in xl list -l when only dom0 is present. 
> 
> Which bits of status are you interested in?

The
    (status 2)

field.
> 
> I'm thinking something like the following, which will include the
> content of libxl_dominfo for every domain.
> 
> 8<----------------------------------
> 
> >From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001
> From: Ian Campbell <ian.campbell@citrix.com>
> Date: Thu, 12 Sep 2013 13:50:33 +0100
> Subject: [PATCH] xl: print runtime info in "xl list -l"
> 
> Include dom0 runtime information, but not domain config.

With that I get (with /etc/xen/xl.conf having output_mode=sxp)

-bash-4.1# xl list -l
dom0
no d_config
(domain
        (domid 0)
        (create_info)
        (hvm 1)
        (hap True)
        (oos <default>)
        (ssidref 0)
        (name (null))
        (uuid 00000000-0000-0000-0000-000000000000)
        (xsdata (null))
        (platformdata (null))
        (build_info)
        (max_vcpus 0)
        (tsc_mode (null))
        (max_memkb 0)
        (target_memkb 45887743)
        (nomigrate True)
        (image
                (hvm
                        (firmware )
                        (video_memkb 140277926854668)
                        (shadow_memkb 140280103390500)
                        (pae True)
                        (apic True)
                        (acpi True)
                        (nx False)
                        (viridian True)
                        (hpet False)
                        (vpt_align <default>)
                        (timer_mode (null))
                        (nestedhvm True)
                        (stdvga False)
                        (vnc <default>)
Segmentation fault

While under xm:
-bash-4.1# xm list -l
(domain
    (domid 0)
    (cpu_weight 256)
    (cpu_cap 0)
    (pool_name Pool-0)
    (bootloader '')
    (vcpus 4)
    (cpus ((0 1 2 3) (0 1 2 3) (0 1 2 3) (0 1 2 3)))
    (on_poweroff destroy)
    (on_crash restart)
    (uuid 00000000-0000-0000-0000-000000000000)
    (bootloader_args '')
    (name Domain-0)
    (on_reboot restart)
    (maxmem 6144)
    (memory 2048)
    (shadow_memory 0)
    (features '')
    (on_xend_start ignore)
    (on_xend_stop ignore)
    (cpu_time 11.138272406)
    (online_vcpus 4)
    (image (linux (kernel '') (superpages 0) (nomigrate 0) (tsc_mode 0)))
    (status 2)
    (state r-----)
)


Let me (once I am done with this email backlog) look at this and see if I
can make the segfault go away.

Thanks for the prototype!

> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
>  tools/libxl/xl_cmdimpl.c |   74 ++++++++++++++++++++++++++++------------------
>  1 file changed, 46 insertions(+), 28 deletions(-)
> 
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 884f050..46fdafc 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -298,7 +298,8 @@ static void dolog(const char *file, int line, const char *func, char *fmt, ...)
>  }
>  
>  static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid,
> -                                            libxl_domain_config *d_config)
> +                                            libxl_domain_config *d_config,
> +                                            libxl_dominfo *d_info)
>  {
>      yajl_gen_status s;
>  
> @@ -317,13 +318,25 @@ static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid,
>      if (s != yajl_gen_status_ok)
>          goto out;
>  
> -    s = yajl_gen_string(hand, (const unsigned char *)"config",
> -                        sizeof("config")-1);
> -    if (s != yajl_gen_status_ok)
> -        goto out;
> -    s = libxl_domain_config_gen_json(hand, d_config);
> -    if (s != yajl_gen_status_ok)
> -        goto out;
> +    if (d_config) {
> +        s = yajl_gen_string(hand, (const unsigned char *)"config",
> +                            sizeof("config")-1);
> +        if (s != yajl_gen_status_ok)
> +            goto out;
> +        s = libxl_domain_config_gen_json(hand, d_config);
> +        if (s != yajl_gen_status_ok)
> +            goto out;
> +    }
> +
> +    if (d_info) {
> +        s = yajl_gen_string(hand, (const unsigned char *)"info",
> +                            sizeof("info")-1);
> +        if (s != yajl_gen_status_ok)
> +            goto out;
> +        s = libxl_dominfo_gen_json(hand, d_info);
> +        if (s != yajl_gen_status_ok)
> +            goto out;
> +    }
>  
>      s = yajl_gen_map_close(hand);
>      if (s != yajl_gen_status_ok)
> @@ -350,7 +363,7 @@ static void printf_info(enum output_format output_format,
>          return;
>      }
>  
> -    s = printf_info_one_json(hand, domid, d_config);
> +    s = printf_info_one_json(hand, domid, d_config, NULL);
>      if (s != yajl_gen_status_ok)
>          goto out;
>  
> @@ -2990,7 +3003,7 @@ static void reboot_domain(uint32_t domid, libxl_evgen_domain_death **deathw,
>      }
>  }
>  
> -static void list_domains_details(const libxl_dominfo *info, int nb_domain)
> +static void list_domains_details(libxl_dominfo *info, int nb_domain)
>  {
>      libxl_domain_config d_config;
>  
> @@ -3017,26 +3030,31 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain)
>          s = yajl_gen_status_ok;
>  
>      for (i = 0; i < nb_domain; i++) {
> -        /* no detailed info available on dom0 */
> -        if (info[i].domid == 0)
> -            continue;
> -        rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
> -        if (rc)
> -            continue;
> -        CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid));
> -        libxl_domain_config_init(&d_config);
> -        parse_config_data(config_source, (char *)data, len, &d_config, NULL);
> -        if (default_output_format == OUTPUT_FORMAT_JSON)
> -            s = printf_info_one_json(hand, info[i].domid, &d_config);
> -        else
> +        fprintf(stderr, "dom%d\n", info[i].domid);
> +
> +        if ( info[i].domid > 0 ) {
> +            fprintf(stderr, "parse d_config for dom%d\n", info[i].domid);
> +            rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
> +            if (rc)
> +                continue;
> +            CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid));
> +            libxl_domain_config_init(&d_config);
> +            parse_config_data(config_source, (char *)data, len, &d_config, NULL);
> +        } else { fprintf(stderr, "no d_config\n"); }
> +        if (default_output_format == OUTPUT_FORMAT_JSON) {
> +            s = printf_info_one_json(hand, info[i].domid,
> +                                     i == 0 ? NULL : &d_config, &info[i]);
> +            if (s != yajl_gen_status_ok)
> +                goto out;
> +        } else
>              printf_info_sexp(info[i].domid, &d_config);
> -        libxl_domain_config_dispose(&d_config);
> -        free(data);
> -        free(config_source);
> -        if (s != yajl_gen_status_ok)
> -            goto out;
> -    }
>  
> +        if ( info[i].domid > 0 ) {
> +            libxl_domain_config_dispose(&d_config);
> +            free(data);
> +            free(config_source);
> +        }
> +    }
>      if (default_output_format == OUTPUT_FORMAT_JSON) {
>          s = yajl_gen_array_close(hand);
>          if (s != yajl_gen_status_ok)
> -- 
> 1.7.10.4
> 
> 
> 

Re: [PATCH] xl: print runtime info in "xl list -l"

[Ian Campbell]

On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:
> On Thu, Sep 12, 2013 at 01:52:09PM +0100, Ian Campbell wrote:
> > On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:
> > >  - No status in xl list -l when only dom0 is present. 
> > 
> > Which bits of status are you interested in?
> 
> The
>     (status 2)

Oh, I thought you might mean the "(state r-----)" bit which I know and
understand. I've no idea what this 2 status is -- any idea what it
means/relates to?

> field.
> > 
> > I'm thinking something like the following, which will include the
> > content of libxl_dominfo for every domain.
> > 
> > 8<----------------------------------
> > 
> > >From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001
> > From: Ian Campbell <ian.campbell@citrix.com>
> > Date: Thu, 12 Sep 2013 13:50:33 +0100
> > Subject: [PATCH] xl: print runtime info in "xl list -l"
> > 
> > Include dom0 runtime information, but not domain config.
> 
> With that I get (with /etc/xen/xl.conf having output_mode=sxp)

Actually, I probably didn't hook up the sxp case, oops!

Should be easy enough to add though.

Ian.

Re: [PATCH] xl: print runtime info in "xl list -l"

[Konrad Rzeszutek Wilk]

On Wed, Sep 25, 2013 at 06:56:36PM +0100, Ian Campbell wrote:
> On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:
> > On Thu, Sep 12, 2013 at 01:52:09PM +0100, Ian Campbell wrote:
> > > On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:
> > > >  - No status in xl list -l when only dom0 is present. 
> > > 
> > > Which bits of status are you interested in?
> > 
> > The
> >     (status 2)
> 
> Oh, I thought you might mean the "(state r-----)" bit which I know and
> understand. I've no idea what this 2 status is -- any idea what it
> means/relates to?

It looks to be 
 (status 2)
 (state -b----)

  def _stateGet(self):                                                        
        # Lets try and reconsitute the state from xc                            
        # first lets try and get the domain info                                
        # from xc - this will tell us if the domain                             
        # exists                                                                
        info = dom_get(self.getDomid())                                         
        if info is None or info['shutdown']:                                    
            # We are either HALTED or SUSPENDED                                 
            # check saved image exists                                          
            from xen.xend import XendDomain                                     
            managed_config_path = \                                             
                XendDomain.instance()._managed_check_point_path( \              
                    self.get_uuid())                                            
            if os.path.exists(managed_config_path):                             
                return XEN_API_VM_POWER_STATE_SUSPENDED                         
            else:                                                               
                return XEN_API_VM_POWER_STATE_HALTED                            
        elif info['crashed']:                                                   
            # Crashed                                                           
            return XEN_API_VM_POWER_STATE_CRASHED                               
        else:                                                                   
            # We are either RUNNING or PAUSED                                   
            if info['paused']:                                                  
                return XEN_API_VM_POWER_STATE_PAUSED                            
            else:                                                               
                return XEN_API_VM_POWER_STATE_RUNNING           

which is
XEN_API_VM_POWER_STATE_HALTED = 0                                               
XEN_API_VM_POWER_STATE_PAUSED = 1                                               
XEN_API_VM_POWER_STATE_RUNNING = 2                                              
XEN_API_VM_POWER_STATE_SUSPENDED = 3                                            
XEN_API_VM_POWER_STATE_SHUTTINGDOWN = 4                                         
XEN_API_VM_POWER_STATE_CRASHED = 5                                              
XEN_API_VM_POWER_STATE_UNKNOWN = 6

> 
> > field.
> > > 
> > > I'm thinking something like the following, which will include the
> > > content of libxl_dominfo for every domain.
> > > 
> > > 8<----------------------------------
> > > 
> > > >From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001
> > > From: Ian Campbell <ian.campbell@citrix.com>
> > > Date: Thu, 12 Sep 2013 13:50:33 +0100
> > > Subject: [PATCH] xl: print runtime info in "xl list -l"
> > > 
> > > Include dom0 runtime information, but not domain config.
> > 
> > With that I get (with /etc/xen/xl.conf having output_mode=sxp)
> 
> Actually, I probably didn't hook up the sxp case, oops!

Ah.
> 
> Should be easy enough to add though.
> 
> Ian.
> 

Re: [PATCH] xl: print runtime info in "xl list -l"

[Ian Campbell]

On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:
> While under xm:
> -bash-4.1# xm list -l 

BTW, if you can gather a representative set of such outputs from the
sorts of configurations you support we can look at heading straight
there instead of faulting in the interesting fields one at a time...

Ian.

Re: [PATCH] xl: print runtime info in "xl list -l"

[Konrad Rzeszutek Wilk]

On Wed, Sep 25, 2013 at 06:59:38PM +0100, Ian Campbell wrote:
> On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:
> > While under xm:
> > -bash-4.1# xm list -l 
> 
> BTW, if you can gather a representative set of such outputs from the
> sorts of configurations you support we can look at heading straight
> there instead of faulting in the interesting fields one at a time...

Let me talk to Zhigang on how to make that possible.

Thank you!
> 
> Ian.
> 
> 

Re: [PATCH] xl: print runtime info in "xl list -l"

[Wei Liu]

On Wed, Sep 25, 2013 at 02:27:40PM -0400, Konrad Rzeszutek Wilk wrote:
> On Wed, Sep 25, 2013 at 06:59:38PM +0100, Ian Campbell wrote:
> > On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:
> > > While under xm:
> > > -bash-4.1# xm list -l 
> > 
> > BTW, if you can gather a representative set of such outputs from the
> > sorts of configurations you support we can look at heading straight
> > there instead of faulting in the interesting fields one at a time...
> 
> Let me talk to Zhigang on how to make that possible.
> 

How is this going?

> Thank you!
> > 
> > Ian.
> > 
> > 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: xend deprecation

[Ian Campbell]

On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:
> 
>  - No console in xl list -l, so can't get  tty console port. 

Together with the previous patch this can be resoled with the following,
I think. I'm not sure this is the best way to do it though, although
dominfo is mostly "runtime" info and this fits....

8<----------------------------

>From 4cffdf9517c668a176f37fa58236f4eb2fd4bbb0 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Thu, 12 Sep 2013 14:41:06 +0100
Subject: [PATCH] libxl: expose the POV console tty in the dominfo.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 tools/libxl/libxl.c         |   38 ++++++++++++++++++++++++++++----------
 tools/libxl/libxl_types.idl |    2 ++
 2 files changed, 30 insertions(+), 10 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 1bce4bb..8dea15f 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -510,9 +510,12 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid,
     return 0;
 }
 
-static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
+static void xcinfo2xlinfo(libxl__gc *gc,
+                          const xc_domaininfo_t *xcinfo,
                           libxl_dominfo *xlinfo)
 {
+    libxl_dominfo_init(xlinfo);
+
     memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
     xlinfo->domid = xcinfo->domain;
     xlinfo->ssidref = xcinfo->ssidref;
@@ -539,10 +542,17 @@ static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
     xlinfo->cpupool = xcinfo->cpupool;
     xlinfo->domain_type = (xcinfo->flags & XEN_DOMINF_hvm_guest) ?
         LIBXL_DOMAIN_TYPE_HVM : LIBXL_DOMAIN_TYPE_PV;
+
+    if (xcinfo->domaindomid && libxl_console_get_tty(CTX, xcinfo->domain, 0,
+                                                      LIBXL_CONSOLE_TYPE_PV,
+                                                      &xlinfo->console_tty) < 0)
+        LOG(WARN, "unable to get dom%d pv console tty\n", xcinfo->domain);
+
 }
 
 libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out)
 {
+    GC_INIT(ctx);
     libxl_dominfo *ptr;
     int i, ret;
     xc_domaininfo_t info[1024];
@@ -551,38 +561,46 @@ libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out)
     ptr = calloc(size, sizeof(libxl_dominfo));
     if (!ptr) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "allocating domain info");
-        return NULL;
+        goto err;
     }
 
     ret = xc_domain_getinfolist(ctx->xch, 0, 1024, info);
     if (ret<0) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list");
         free(ptr);
-        return NULL;
+        ptr = NULL;
+        goto err;
     }
 
     for (i = 0; i < ret; i++) {
-        xcinfo2xlinfo(&info[i], &ptr[i]);
+        xcinfo2xlinfo(gc, &info[i], &ptr[i]);
     }
     *nb_domain_out = ret;
+err:
+    GC_FREE;
     return ptr;
 }
 
 int libxl_domain_info(libxl_ctx *ctx, libxl_dominfo *info_r,
-                      uint32_t domid) {
+                      uint32_t domid)
+{
+    GC_INIT(ctx);
     xc_domaininfo_t xcinfo;
-    int ret;
+    int ret, rc = ERROR_INVAL;
 
     ret = xc_domain_getinfolist(ctx->xch, domid, 1, &xcinfo);
     if (ret<0) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list");
         return ERROR_FAIL;
     }
-    if (ret==0 || xcinfo.domain != domid) return ERROR_INVAL;
+    if (ret==0 || xcinfo.domain != domid) goto err;;
 
     if (info_r)
-        xcinfo2xlinfo(&xcinfo, info_r);
-    return 0;
+        xcinfo2xlinfo(gc, &xcinfo, info_r);
+    rc = 0;
+err:
+    GC_FREE;
+    return rc;
 }
 
 static int cpupool_info(libxl__gc *gc,
@@ -3764,7 +3782,7 @@ retry_transaction:
         abort_transaction = 1;
         goto out;
     }
-    xcinfo2xlinfo(&info, &ptr);
+    xcinfo2xlinfo(gc, &info, &ptr);
     uuid = libxl__uuid2string(gc, ptr.uuid);
     libxl__xs_write(gc, t, libxl__sprintf(gc, "/vm/%s/memory", uuid),
             "%"PRIu32, new_target_memkb / 1024);
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 10f95f4..6de305b 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -200,6 +200,8 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown",    bool),
     ("dying",       bool),
 
+    ("console_tty", string),
+
     # Valid iff (shutdown||dying).
     #
     # Otherwise set to a value guaranteed not to clash with any valid
-- 
1.7.10.4

Re: xend deprecation

[Wei Liu]

On Wed, Sep 04, 2013 at 10:04:14AM -0400, Konrad Rzeszutek Wilk wrote:
[...]
> There are issues that are stopping us from using it:
> 

I look at these xl items from time to time and see if I can fix them but
they come out a little bit more complex than I expect.

Xl will store DomU configs in its private storage location and retrievs
them as it see fit. This behavior is causing a series of issues.

For 'xl list -l', the detail info of a domain is retrieved from the
stored config file.

>  - No status in xl list -l when only dom0 is present.

This would not work because there is no config file for Dom0.

>  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
>    is for 'Output all VM details'

Haven't traced this one down, but I presume this is caused by the same
thing.

>  - No console in xl list -l, so can't get  tty console port.

This would not work because apparently you cannot specify a tty in
DomU's config file -- that's allcoated by xenconsoled.

Along with the above issues, xl save/restored will only use the stored
config file which makes it impossible to persist any runtime info across
save/restore -- one significant defect is that VM's MAC address changes
every time unless it is specified in the original config file.

These issues are all connected in a way -- runtime info is not retrieved
/ updated / preserved. What I have in mind is that we might need to
introduce a mechanism to pull info from Xenstore, serialize /
deserialize config info / update stored config file when necessary.

Comments?

Wei.

Re: xend deprecation

[Zhigang Wang]

On 10/21/2013 10:09 AM, Wei Liu wrote:
> On Wed, Sep 04, 2013 at 10:04:14AM -0400, Konrad Rzeszutek Wilk wrote:
> [...]
>> There are issues that are stopping us from using it:
>>
> 
> I look at these xl items from time to time and see if I can fix them but
> they come out a little bit more complex than I expect.
> 
> Xl will store DomU configs in its private storage location and retrievs
> them as it see fit. This behavior is causing a series of issues.
> 
> For 'xl list -l', the detail info of a domain is retrieved from the
> stored config file.
> 
>>  - No status in xl list -l when only dom0 is present.
> 
> This would not work because there is no config file for Dom0.
> 
>>  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
>>    is for 'Output all VM details'
> 
> Haven't traced this one down, but I presume this is caused by the same
> thing.
> 
>>  - No console in xl list -l, so can't get  tty console port.
> 
> This would not work because apparently you cannot specify a tty in
> DomU's config file -- that's allcoated by xenconsoled.
> 
> Along with the above issues, xl save/restored will only use the stored
> config file which makes it impossible to persist any runtime info across
> save/restore -- one significant defect is that VM's MAC address changes
> every time unless it is specified in the original config file.
> 
> These issues are all connected in a way -- runtime info is not retrieved
> / updated / preserved. What I have in mind is that we might need to
> introduce a mechanism to pull info from Xenstore, serialize /
> deserialize config info / update stored config file when necessary.
> 
> Comments?

Long time ago, when I work on it, I have the same feeling (I haven't
confirm the latest code yet).

xend is a daemon: it handles all VM state change. With xl, a private
storage location for each VM is the corresponding solution for now.

My suggestion with my limited knowledge is: remove this private storage
location and get all the info from xenstore/hypercall.

Thanks,

Zhigang

Re: xend deprecation

[Ian Campbell]

On Mon, 2013-10-21 at 15:09 +0100, Wei Liu wrote:

> >  - No status in xl list -l when only dom0 is present.
> 
> This would not work because there is no config file for Dom0.

I sent a PoC patch which adds the content of libxl_dominfo (ie. the
runtime info) to the listing alongside the domain config. For dom0 only
the runtime info is included, but that is what is needed here.

> >  - 'xl list -l' says 'Domain name must be specified' even though the '-l'
> >    is for 'Output all VM details'
> 
> Haven't traced this one down, but I presume this is caused by the same
> thing.

Yes, I think dom0 can be special cased to only care about libxl_dominfo
and not libxl_domain_config. I think I might have included that in the
PoC.

I thought I'd added a bug to track the thread with that PoC in it, but
apparently not. See the subthread starting at
<1378990329.10076.54.camel@kazak.uk.xensource.com>

I thought I'd posted the tty thing too, apparently not. It was like this
and hooked into the patch above to actually get printed.

Ian.

commit 4cffdf9517c668a176f37fa58236f4eb2fd4bbb0
Author: Ian Campbell <ian.campbell@citrix.com>
Date:   Thu Sep 12 14:41:06 2013 +0100

    libxl: expose the POV console tty in the dominfo.
    
    Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 1bce4bb..8dea15f 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -510,9 +510,12 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid,
     return 0;
 }
 
-static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
+static void xcinfo2xlinfo(libxl__gc *gc,
+                          const xc_domaininfo_t *xcinfo,
                           libxl_dominfo *xlinfo)
 {
+    libxl_dominfo_init(xlinfo);
+
     memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
     xlinfo->domid = xcinfo->domain;
     xlinfo->ssidref = xcinfo->ssidref;
@@ -539,10 +542,17 @@ static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
     xlinfo->cpupool = xcinfo->cpupool;
     xlinfo->domain_type = (xcinfo->flags & XEN_DOMINF_hvm_guest) ?
         LIBXL_DOMAIN_TYPE_HVM : LIBXL_DOMAIN_TYPE_PV;
+
+    if (xcinfo->domaindomid && libxl_console_get_tty(CTX, xcinfo->domain, 0,
+                                                      LIBXL_CONSOLE_TYPE_PV,
+                                                      &xlinfo->console_tty) < 0)
+        LOG(WARN, "unable to get dom%d pv console tty\n", xcinfo->domain);
+
 }
 
 libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out)
 {
+    GC_INIT(ctx);
     libxl_dominfo *ptr;
     int i, ret;
     xc_domaininfo_t info[1024];
@@ -551,38 +561,46 @@ libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out)
     ptr = calloc(size, sizeof(libxl_dominfo));
     if (!ptr) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "allocating domain info");
-        return NULL;
+        goto err;
     }
 
     ret = xc_domain_getinfolist(ctx->xch, 0, 1024, info);
     if (ret<0) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list");
         free(ptr);
-        return NULL;
+        ptr = NULL;
+        goto err;
     }
 
     for (i = 0; i < ret; i++) {
-        xcinfo2xlinfo(&info[i], &ptr[i]);
+        xcinfo2xlinfo(gc, &info[i], &ptr[i]);
     }
     *nb_domain_out = ret;
+err:
+    GC_FREE;
     return ptr;
 }
 
 int libxl_domain_info(libxl_ctx *ctx, libxl_dominfo *info_r,
-                      uint32_t domid) {
+                      uint32_t domid)
+{
+    GC_INIT(ctx);
     xc_domaininfo_t xcinfo;
-    int ret;
+    int ret, rc = ERROR_INVAL;
 
     ret = xc_domain_getinfolist(ctx->xch, domid, 1, &xcinfo);
     if (ret<0) {
         LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list");
         return ERROR_FAIL;
     }
-    if (ret==0 || xcinfo.domain != domid) return ERROR_INVAL;
+    if (ret==0 || xcinfo.domain != domid) goto err;;
 
     if (info_r)
-        xcinfo2xlinfo(&xcinfo, info_r);
-    return 0;
+        xcinfo2xlinfo(gc, &xcinfo, info_r);
+    rc = 0;
+err:
+    GC_FREE;
+    return rc;
 }
 
 static int cpupool_info(libxl__gc *gc,
@@ -3764,7 +3782,7 @@ retry_transaction:
         abort_transaction = 1;
         goto out;
     }
-    xcinfo2xlinfo(&info, &ptr);
+    xcinfo2xlinfo(gc, &info, &ptr);
     uuid = libxl__uuid2string(gc, ptr.uuid);
     libxl__xs_write(gc, t, libxl__sprintf(gc, "/vm/%s/memory", uuid),
             "%"PRIu32, new_target_memkb / 1024);
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 10f95f4..6de305b 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -200,6 +200,8 @@ libxl_dominfo = Struct("dominfo",[
     ("shutdown",    bool),
     ("dying",       bool),
 
+    ("console_tty", string),
+
     # Valid iff (shutdown||dying).
     #
     # Otherwise set to a value guaranteed not to clash with any valid

Re: xend deprecation

[Ian Campbell]

On Mon, 2013-10-21 at 15:09 +0100, Wei Liu wrote:

> These issues are all connected in a way -- runtime info is not retrieved
> / updated / preserved. What I have in mind is that we might need to
> introduce a mechanism to pull info from Xenstore, serialize /
> deserialize config info / update stored config file when necessary.
> 
> Comments?

I agree. I have long thought that we need a way to resynthesize a
libxl_domain_config from a running domain directly instead of by
reparsing the config, which is likely to be stale.

This could also be used as part of migration which also suffers from
issues where e.g. the migrated domain gets a new random mac or reverts
to the original memory target. In this case instead of sending the
config file over the migration wire we'd want to switch to using the
JSON generation and add autogenerated JSON parsing for use on the other
end.

I started to hack on such a thing (see below, as you can see it is a bit
WIP!). If you wanted to pick it up that would be awesome.

Ian.

commit 47a05fa3fa53761f889bec0bc2884fe927ac988f
Author: Ian Campbell <ian.campbell@citrix.com>
Date:   Mon Sep 23 12:52:07 2013 +0100

    begin to implement libxl_domain_running_config

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 8dea15f..0ae145f 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -543,9 +543,9 @@ static void xcinfo2xlinfo(libxl__gc *gc,
     xlinfo->domain_type = (xcinfo->flags & XEN_DOMINF_hvm_guest) ?
         LIBXL_DOMAIN_TYPE_HVM : LIBXL_DOMAIN_TYPE_PV;
 
-    if (xcinfo->domaindomid && libxl_console_get_tty(CTX, xcinfo->domain, 0,
-                                                      LIBXL_CONSOLE_TYPE_PV,
-                                                      &xlinfo->console_tty) < 0)
+    if (xcinfo->domain && libxl_console_get_tty(CTX, xcinfo->domain, 0,
+                                                LIBXL_CONSOLE_TYPE_PV,
+                                                &xlinfo->console_tty) < 0)
         LOG(WARN, "unable to get dom%d pv console tty\n", xcinfo->domain);
 
 }
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index be19bf5..c824ad6 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -677,6 +677,17 @@ int libxl_primary_console_get_tty(libxl_ctx *ctx, uint32_t domid_vm, char **path
 int libxl_domain_info(libxl_ctx*, libxl_dominfo *info_r,
                       uint32_t domid);
 
+/* Returns the running configuration of a domain.  If
+ * preserve_defaults is true then, where it makes sense, fields which
+ * were initially set to the default will be returned as defaults
+ * rather than presenting the actual value. e.g. this will be done for
+ * disk backend but not NIC MAC addresses. This can be useful if this
+ * call is to be used to recreate a domain, in which case you want to
+ * allow libxl to select the appropriate disk backend again but not
+ * change the MAC address. */
+int libxl_domain_running_config(libxl_ctx*, libxl_domain_config *dconfig_r,
+                                uint32_t domid, bool preserve_defaults);
+
 /* These functions each return (on success) an array of elements,
  * and the length via the int* out parameter.  These arrays and
  * their contents come from malloc, and must be freed with the
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 6e2252a..bf11b30 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -1668,6 +1668,153 @@ out:
     return rc;
 }
 
+int libxl_domain_running_config(libxl_ctx *ctx, libxl_domain_config *dconfig_r,
+                                uint32_t domid, bool preserve_defaults)
+{
+    GC_INIT(ctx);
+    int rc;
+    xc_domaininfo_t info;
+    int ret;
+
+    /* Convenient aliases */
+    libxl_domain_create_info *c_info = &dconfig_r->c_info;
+    libxl_domain_build_info *b_info = &dconfig_r->b_info;
+
+    libxl_domain_config_init(dconfig_r);
+
+
+    ret = xc_domain_getinfolist(CTX->xch, domid, 1, &info);
+    if (ret != 1)
+    {
+        LOGE(ERROR, "getinfolist failed %d\n", ret);
+        rc = ERROR_FAIL;
+        goto out;
+    }
+    if (info.domain != domid)
+    {
+        LOGE(ERROR, "got info for dom%d, wanted dom%d\n", info.domain, domid);
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    c_info->type = libxl__domain_type(gc, domid);
+    if (c_info->type == LIBXL_DOMAIN_TYPE_INVALID) {
+        rc = ERROR_INVAL;
+        goto out;
+    }
+
+    //c_info->hap
+    //c_info->oos
+    c_info->ssidref = info.ssidref;
+    //c_info->name
+    memcpy(&c_info->uuid, info.handle, sizeof(xen_domain_handle_t));
+    //c_info->xsdata
+    //c_info->platformdata
+    c_info->poolid = info.cpupool;
+    //c_info->run_hotplug_scripts
+
+    libxl_domain_build_info_init_type(b_info, c_info->type);
+
+    b_info->max_vcpus = info.max_vcpu_id;
+    //b_info->avail_vcpus
+    //b_info->cpumap
+    //b_info->nodemap
+    //b_info->numa_placement
+    //b_info->tsc_mode
+    b_info->max_memkb = info.max_pages * XC_PAGE_SIZE / 1024;
+    //b_info->target_memkb
+    //b_info->video_memkb
+    //b_info->shadow_memkb
+    //b_info->rtc_timeoffset
+    //b_info->exec_ssidref
+    //b_info->localtime
+    //b_info->disable_migrate
+    //b_info->cpuid
+    //b_info->blkdev_start
+
+    //b_info->device_model_version
+    //b_info->device_model_stubdomain
+    //b_info->device_model
+    //b_info->device_model_ssidref
+
+    //b_info->extra
+    //b_info->extra_pv
+    //b_info->extra_hvm
+    //b_info->sched_params
+
+    //b_info->ioports
+    //b_info->irqs
+    //b_info->iomem
+    //b_info->claim_mode
+
+    switch (b_info->type) {
+    case LIBXL_DOMAIN_TYPE_HVM:
+        //b_info->firmware
+        //b_info->bios
+        //b_info->pae
+        //b_info->apic
+        //b_info->acpi
+        //b_info->acpi_s3
+        //b_info->acpi_s4
+        //b_info->nx
+        //b_info->viridian
+        //b_info->timeoffset
+        //b_info->hpet
+        //b_info->vpt_align
+        //b_info->timer_mode
+        //b_info->nested_hvm
+        //b_info->smbios_firmware
+        //b_info->acpi_firmware
+        //b_info->nographic
+        //b_info->vga
+        //b_info->vnc
+        //b_info->keymap
+        //b_info->sdl
+        //b_info->spice
+        //b_info->gfx_passthru
+        //b_info->serial
+        //b_info->boot
+        //b_info->usb
+        //b_info->usbdevice
+        //b_info->soundhw
+        //b_info->xen_platform_pci
+        //b_info->usbdevice_list
+        //b_info->vendor_device
+        break;
+    case LIBXL_DOMAIN_TYPE_PV:
+        //b_info->kernel
+        //b_info->slack_memkb
+        //b_info->bootloader
+        //b_info->bootloader_args
+        //b_info->cmdline
+        //b_info->ramdisk
+        //b_info->features
+        //b_info->e820_host
+        break;
+    case LIBXL_DOMAIN_TYPE_INVALID: abort();
+    }
+
+    dconfig_r->disks = libxl_device_disk_list(ctx, domid,
+                                              &dconfig_r->num_disks);
+    dconfig_r->nics = libxl_device_nic_list(ctx, domid,
+                                            &dconfig_r->num_nics);
+    dconfig_r->pcidevs = libxl_device_pci_list(ctx, domid,
+                                               &dconfig_r->num_pcidevs);
+    //dconfig_r->vfbs = libxl_device_vfb_list(ctx, domid,
+    //                                        &dconfig_r->num_vfbs);
+    //dconfig_r->vkbs = libxl_device_vkb_list(ctx, domid,
+    //                                        &dconfig_r->num_vkbs);
+    dconfig_r->vtpms = libxl_device_vtpm_list(ctx, domid,
+                                              &dconfig_r->num_vtpms);
+
+    /* on_{poweroff,reboot,watchdog,crash} */
+    rc = 0;
+
+out:
+    GC_FREE;
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 9f051f8..48815f0 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -3007,9 +3007,9 @@ static void list_domains_details(libxl_dominfo *info, int nb_domain)
 {
     libxl_domain_config d_config;
 
-    char *config_source;
-    uint8_t *data;
-    int i, len, rc;
+    //char *config_source;
+    //uint8_t *data;
+    int i, rc;
 
     yajl_gen hand = NULL;
     yajl_gen_status s;
@@ -3032,12 +3032,17 @@ static void list_domains_details(libxl_dominfo *info, int nb_domain)
     for (i = 0; i < nb_domain; i++) {
 
         if ( info[i].domid > 0 ) {
-            rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
+            //int len;
+            //rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
+            //if (rc)
+            //    continue;
+            //CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid));
+            //libxl_domain_config_init(&d_config);
+            //parse_config_data(config_source, (char *)data, len, &d_config, NULL);
+            rc = libxl_domain_running_config(ctx, &d_config,
+                                             info[i].domid, false);
             if (rc)
                 continue;
-            CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid));
-            libxl_domain_config_init(&d_config);
-            parse_config_data(config_source, (char *)data, len, &d_config, NULL);
         }
         if (default_output_format == OUTPUT_FORMAT_JSON) {
             s = printf_info_one_json(hand, info[i].domid,
@@ -3049,8 +3054,8 @@ static void list_domains_details(libxl_dominfo *info, int nb_domain)
 
         if ( info[i].domid > 0 ) {
             libxl_domain_config_dispose(&d_config);
-            free(data);
-            free(config_source);
+            //free(data);
+            //free(config_source);
         }
     }
     if (default_output_format == OUTPUT_FORMAT_JSON) {

Re: xend deprecation

[Stefano Stabellini]

On Tue, 3 Sep 2013, Matt Wilson wrote:
> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:
> > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > > Live migration between a system running xend to a system that doesn't
> > > support xend would be a nice-to-have. That probably means some support
> > > for SXP parsing either in libxl or in some compatibility tool. 
> > 
> > This already works by using xl on the source side, e.g. a guest started
> > with xend on 4.1 can be migrated with 4.1's xl to 4.2.
> > 
> > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html
> 
> xl doesn't run everywhere that we need it to.

Really? How is that possible?
I am not aware of any specific hardware or software dependencies in the
xl/libxl code.

If there are any, it would good to know them, because we certainly need
to address them before removing xend.

Support for xm create -F (sxp configuration files)

[Ian Campbell]

create ! 
title it xl lacks support for xm create -F (sxp config file syntax)

On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> 
> Live migration between a system running xend to a system that doesn't
> support xend would be a nice-to-have. That probably means some support
> for SXP parsing either in libxl or in some compatibility tool. 

Matt mentioned elsewhere that they use "xm create -F" which takes a
configuration file in the xend sxp format (I had no idea this option
existed).

I don't think we really want to be adding SXP code to xm but perhaps we
can produce a conversion tool?

Matt, can you provide some examples of the actual SXP you care about, we
obviously don't want to be worrying about the bits which aren't actually
used.

Ian.

Processed: Support for xm create -F (sxp configuration files)

[xen]

Processing commands for xen@bugs.xenproject.org:

> create ! 
Created new bug #17 rooted at `! '
Title: `Support for xm create -F (sxp configuration files)'
> title it xl lacks support for xm create -F (sxp config file syntax)
Set title for #17 to `xl lacks support for xm create -F (sxp config file syntax)'
> 
> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
Command failed: Unknown command `On'. at /srv/xen-devel-bugs/lib/emesinae/control.pl line 437, <M> line 42.
Stop processing here.

Modified/created Bugs:
 - 17: http://bugs.xenproject.org/xen/bug/17 (new)

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues

Re: Support for xm create -F (sxp configuration files)

[Matt Wilson]

On Thu, Sep 12, 2013 at 10:50:49AM +0100, Ian Campbell wrote:
> create ! 
> title it xl lacks support for xm create -F (sxp config file syntax)
> 
> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > 
> > Live migration between a system running xend to a system that doesn't
> > support xend would be a nice-to-have. That probably means some support
> > for SXP parsing either in libxl or in some compatibility tool. 
> 
> Matt mentioned elsewhere that they use "xm create -F" which takes a
> configuration file in the xend sxp format (I had no idea this option
> existed).
> 
> I don't think we really want to be adding SXP code to xm but perhaps we
> can produce a conversion tool?

Indeed, also mentioned elsewhere it's probably not in the best
interest of the project to go adding SXP support when we already have
more well adopted options like json.

> Matt, can you provide some examples of the actual SXP you care about, we
> obviously don't want to be worrying about the bits which aren't actually
> used.

Honestly I think it'd be less total work to avoid SXP altogether.
There are some other differences that need to be carefully considered,
and shifting config file formats is a forcing function for looking
closely for those differences.

--msw

Re: Support for xm create -F (sxp configuration files)

[Ian Campbell]

close 17
thanks

On Mon, 2013-09-16 at 09:49 -0700, Matt Wilson wrote:
> On Thu, Sep 12, 2013 at 10:50:49AM +0100, Ian Campbell wrote:
> > create ! 
> > title it xl lacks support for xm create -F (sxp config file syntax)
> > 
> > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:
> > > 
> > > Live migration between a system running xend to a system that doesn't
> > > support xend would be a nice-to-have. That probably means some support
> > > for SXP parsing either in libxl or in some compatibility tool. 
> > 
> > Matt mentioned elsewhere that they use "xm create -F" which takes a
> > configuration file in the xend sxp format (I had no idea this option
> > existed).
> > 
> > I don't think we really want to be adding SXP code to xm but perhaps we
> > can produce a conversion tool?
> 
> Indeed, also mentioned elsewhere it's probably not in the best
> interest of the project to go adding SXP support when we already have
> more well adopted options like json.
> 
> > Matt, can you provide some examples of the actual SXP you care about, we
> > obviously don't want to be worrying about the bits which aren't actually
> > used.
> 
> Honestly I think it'd be less total work to avoid SXP altogether.
> There are some other differences that need to be carefully considered,
> and shifting config file formats is a forcing function for looking
> closely for those differences.

I suppose I should close this bug.

If anyone else has a requirement for this stuff now would be a good time
to say so.

I'll also add it to http://wiki.xen.org/wiki/XL#Anti-Features.

Ian.

Processed: Re: Support for xm create -F (sxp configuration files)

[xen]

Processing commands for xen@bugs.xenproject.org:

> close 17
Closing bug #17
> thanks
Finished processing.

Modified/created Bugs:
 - 17: http://bugs.xenproject.org/xen/bug/17

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues

Re: [PATCH] xend: handle extended PCI configuration space when saving state

[Ian Campbell]

On Fri, 2013-08-30 at 16:40 -0700, Matt Wilson wrote:
> From: Steven Noonan <snoonan@amazon.com>
> 
> Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended
> configuration space which is larger than 256 bytes. This patch uses
> stat() to determine the amount of space used to correctly save all of
> the PCI configuration space. Resets handled by the xen-pciback driver
> don't have this problem, as that code correctly handles saving
> extended configuration space.
> 
> Signed-off-by: Steven Noonan <snoonan@amazon.com>
> Reviewed-by: Matt Wilson <msw@amazon.com>
> [msw: adjusted commit message]
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Signed-off-by: Matt Wilson <msw@amazon.com>

I've applied this and "xend: fix file descriptor leak in pci utilities"
on the basis that they looked pretty simple, however in general I'm not
prepared to review xend patches so I am relying on whoever wants to keep
xend alive to co-review each others patches and provide the appropriate
reviewed-by etc (and to take responsibility for the fallout etc).

> ---
>  tools/python/xen/util/pci.py |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/python/xen/util/pci.py b/tools/python/xen/util/pci.py
> index 792fb69..adeca4b 100644
> --- a/tools/python/xen/util/pci.py
> +++ b/tools/python/xen/util/pci.py
> @@ -521,8 +521,9 @@ def save_pci_conf_space(devs_string):
>          pci_path = sysfs_mnt + SYSFS_PCI_DEVS_PATH + '/' + pci_str + \
>                  SYSFS_PCI_DEV_CONFIG_PATH
>          fd = os.open(pci_path, os.O_RDONLY)
> +        size = os.fstat(fd).st_size
>          configs = []
> -        for i in range(0, 256, 4):
> +        for i in range(0, size, 4):
>              configs = configs + [os.read(fd,4)]
>          os.close(fd)
>          pci_list = pci_list + [pci_path]

Re: [PATCH] xend: handle extended PCI configuration space when saving state

[Matt Wilson]

On Tue, Sep 03, 2013 at 05:33:13PM +0100, Ian Campbell wrote:
> 
> I've applied this and "xend: fix file descriptor leak in pci utilities"
> on the basis that they looked pretty simple, however in general I'm not
> prepared to review xend patches so I am relying on whoever wants to keep
> xend alive to co-review each others patches and provide the appropriate
> reviewed-by etc (and to take responsibility for the fallout etc).

We (Amazon) can take on some of the xend life support maintenance. I'm
happy to review xend code from C/Python bindings to the Python code
itself.

If you'd like I'll submit a patch to MAINTAINERS for that.

--msw

Re: [PATCH] xend: handle extended PCI configuration space when saving state

[Noonan, Steven]

On Tue, Sep 03, 2013 at 11:12:44PM -0700, Matt Wilson wrote:
> On Tue, Sep 03, 2013 at 05:33:13PM +0100, Ian Campbell wrote:
> > 
> > I've applied this and "xend: fix file descriptor leak in pci utilities"
> > on the basis that they looked pretty simple, however in general I'm not
> > prepared to review xend patches so I am relying on whoever wants to keep
> > xend alive to co-review each others patches and provide the appropriate
> > reviewed-by etc (and to take responsibility for the fallout etc).
> 
> We (Amazon) can take on some of the xend life support maintenance. I'm
> happy to review xend code from C/Python bindings to the Python code
> itself.
> 
> If you'd like I'll submit a patch to MAINTAINERS for that.
> 
> --msw

Agreed, I'm equally happy to maintain xend alongside Matt.

- Steven

Re: [PATCH] xend: handle extended PCI configuration space when saving state

[Ian Campbell]

On Tue, 2013-09-03 at 23:47 -0700, Noonan, Steven wrote:
> On Tue, Sep 03, 2013 at 11:12:44PM -0700, Matt Wilson wrote:
> > On Tue, Sep 03, 2013 at 05:33:13PM +0100, Ian Campbell wrote:
> > > 
> > > I've applied this and "xend: fix file descriptor leak in pci utilities"
> > > on the basis that they looked pretty simple, however in general I'm not
> > > prepared to review xend patches so I am relying on whoever wants to keep
> > > xend alive to co-review each others patches and provide the appropriate
> > > reviewed-by etc (and to take responsibility for the fallout etc).
> > 
> > We (Amazon) can take on some of the xend life support maintenance. I'm
> > happy to review xend code from C/Python bindings to the Python code
> > itself.
> > 
> > If you'd like I'll submit a patch to MAINTAINERS for that.
> > 
> > --msw
> 
> Agreed, I'm equally happy to maintain xend alongside Matt.

Please do send a patch to MAINTAINERS. Try and pick the right S:tatus
from the list at the top to indicate your level of support.

Ian

Re: [PATCH] xend: handle extended PCI configuration space when saving state

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH] xend: handle extended PCI configuration space when saving state"):
> On Fri, 2013-08-30 at 16:40 -0700, Matt Wilson wrote:
> > Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended
> > configuration space which is larger than 256 bytes. This patch uses
> > stat() to determine the amount of space used to correctly save all of
> > the PCI configuration space. Resets handled by the xen-pciback driver
> > don't have this problem, as that code correctly handles saving
> > extended configuration space.
...
> I've applied this and "xend: fix file descriptor leak in pci utilities"
> on the basis that they looked pretty simple, however in general I'm not
> prepared to review xend patches so I am relying on whoever wants to keep
> xend alive to co-review each others patches and provide the appropriate
> reviewed-by etc (and to take responsibility for the fallout etc).

I think we should be planning to drop xend in 4.4.

Ian.