[PATCH] Fix pci passthru in xend interface used by libvirt

[PATCH] Fix pci passthru in xend interface used by libvirt

[Jim Fehlig]

# HG changeset patch
# User Jim Fehlig <jfehlig@novell.com>
# Date 1288301229 21600
# Branch xend-pci
# Node ID 461b9d3a643a2c67c961d9fc468a804891f3770d
# Parent  28a16074681582f1209c9077f870ccf44927133e
Fix pci passthru in xend interface used by libvirt

Attempting to define or create a domain whose XML config contains a
passthru PCI device fails with libvirt

xen84: # cat win2k8r2.xml
...
   <hostdev mode='subsystem' type='pci' managed='no'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
    </hostdev>
...

xen84: # virsh create ./win2k8r2.xml
error: Failed to create domain from ./win2k8r2.xml
error: POST operation failed: xend_post: error from xen daemon:
  (xend.err "Error creating domain: 'key'")

The PCI device config maintained by xend includes a 'key' field, which is
initialized by xm client when using that tool and traditional xen config
file.  libvirt, which uses xend's sexpr-over-http interface (is that the
proper name for that interface??), does not initialize this field - and
shouldn't be expected to do so IMO.  Clients should not be bothered with
xend's internal representation of a PCI device.

This patch populates the 'key' field within xend if it is uninitialized,
similar to current initialization of 'uuid' field.  The 'vdevfn' field
suffers a similar problem if not (optionally) specified by user.
AFAICT, the xm client initializes 'vdevfn' to 0x100 if not specified so
I've done the same here.

    Signed-off-by: Jim Fehlig <jfehlig@novell.com>

diff -r 28a160746815 -r 461b9d3a643a tools/python/xen/util/pci.py
--- a/tools/python/xen/util/pci.py	Wed Oct 27 12:24:28 2010 +0100
+++ b/tools/python/xen/util/pci.py	Thu Oct 28 15:27:09 2010 -0600
@@ -240,10 +240,16 @@
         pci_dev_info = dict(pci_dev[1:])
         if 'opts' in pci_dev_info:
             pci_dev_info['opts'] = pci_opts_list_from_sxp(pci_dev)
-        # append uuid to each pci device that does't already have one.
+        # If necessary, initialize uuid, key, and vdevfn for each pci device
         if not pci_dev_info.has_key('uuid'):
-            dpci_uuid = pci_dev_info.get('uuid', uuid.createString())
-            pci_dev_info['uuid'] = dpci_uuid
+            pci_dev_info['uuid'] = uuid.createString()
+        if not pci_dev_info.has_key('key'):
+            pci_dev_info['key'] = "%02x:%02x.%x" % \
+            (int(pci_dev_info['bus'], 16),
+             int(pci_dev_info['slot'], 16),
+             int(pci_dev_info['func'], 16))
+        if not pci_dev_info.has_key('vdevfn'):
+            pci_dev_info['vdevfn'] =  "0x%02x" % AUTO_PHP_SLOT
         pci_devs.append(pci_dev_info)
     dev_config['devs'] = pci_devs

Re: [PATCH] Fix pci passthru in xend interface used by libvirt

[Jim Fehlig]

Could someone review this patch please?

The reporter of the bug has successfully tested the patch, but I'd
prefer an upstream ACK as well before adding it to our patch set.

Thanks,
Jim


Jim Fehlig wrote:
> # HG changeset patch
> # User Jim Fehlig <jfehlig@novell.com>
> # Date 1288301229 21600
> # Branch xend-pci
> # Node ID 461b9d3a643a2c67c961d9fc468a804891f3770d
> # Parent  28a16074681582f1209c9077f870ccf44927133e
> Fix pci passthru in xend interface used by libvirt
>
> Attempting to define or create a domain whose XML config contains a
> passthru PCI device fails with libvirt
>
> xen84: # cat win2k8r2.xml
> ...
>    <hostdev mode='subsystem' type='pci' managed='no'>
>       <source>
>         <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
>       </source>
>     </hostdev>
> ...
>
> xen84: # virsh create ./win2k8r2.xml
> error: Failed to create domain from ./win2k8r2.xml
> error: POST operation failed: xend_post: error from xen daemon:
>   (xend.err "Error creating domain: 'key'")
>
> The PCI device config maintained by xend includes a 'key' field, which is
> initialized by xm client when using that tool and traditional xen config
> file.  libvirt, which uses xend's sexpr-over-http interface (is that the
> proper name for that interface??), does not initialize this field - and
> shouldn't be expected to do so IMO.  Clients should not be bothered with
> xend's internal representation of a PCI device.
>
> This patch populates the 'key' field within xend if it is uninitialized,
> similar to current initialization of 'uuid' field.  The 'vdevfn' field
> suffers a similar problem if not (optionally) specified by user.
> AFAICT, the xm client initializes 'vdevfn' to 0x100 if not specified so
> I've done the same here.
>
>     Signed-off-by: Jim Fehlig <jfehlig@novell.com>
>
> diff -r 28a160746815 -r 461b9d3a643a tools/python/xen/util/pci.py
> --- a/tools/python/xen/util/pci.py	Wed Oct 27 12:24:28 2010 +0100
> +++ b/tools/python/xen/util/pci.py	Thu Oct 28 15:27:09 2010 -0600
> @@ -240,10 +240,16 @@
>          pci_dev_info = dict(pci_dev[1:])
>          if 'opts' in pci_dev_info:
>              pci_dev_info['opts'] = pci_opts_list_from_sxp(pci_dev)
> -        # append uuid to each pci device that does't already have one.
> +        # If necessary, initialize uuid, key, and vdevfn for each pci device
>          if not pci_dev_info.has_key('uuid'):
> -            dpci_uuid = pci_dev_info.get('uuid', uuid.createString())
> -            pci_dev_info['uuid'] = dpci_uuid
> +            pci_dev_info['uuid'] = uuid.createString()
> +        if not pci_dev_info.has_key('key'):
> +            pci_dev_info['key'] = "%02x:%02x.%x" % \
> +            (int(pci_dev_info['bus'], 16),
> +             int(pci_dev_info['slot'], 16),
> +             int(pci_dev_info['func'], 16))
> +        if not pci_dev_info.has_key('vdevfn'):
> +            pci_dev_info['vdevfn'] =  "0x%02x" % AUTO_PHP_SLOT
>          pci_devs.append(pci_dev_info)
>      dev_config['devs'] = pci_devs
>  
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

Re: [PATCH] Fix pci passthru in xend interface used by libvirt [and 1 more messages]

[Ian Jackson]

Jim Fehlig writes ("[Xen-devel] [PATCH] Fix pci passthru in xend interface used by libvirt"):
> Attempting to define or create a domain whose XML config contains a
> passthru PCI device fails with libvirt

Thanks for this report.  I'm afraid that no-one any more seems to know
much about the protocol that libvirt speaks to xend.  And, xend is on
the way out - we're trying to push the new libxl library instead.

So it would be worth thinking about changing libvirt to use libxl
instead.

In the meantime, though, I've applied your patch.

Ian.

Re: [PATCH] Fix pci passthru in xend interface used by libvirt [and 1 more messages]

[Jim Fehlig]

Ian Jackson wrote:
> Jim Fehlig writes ("[Xen-devel] [PATCH] Fix pci passthru in xend interface used by libvirt"):
>   
>> Attempting to define or create a domain whose XML config contains a
>> passthru PCI device fails with libvirt
>>     
>
> Thanks for this report.  I'm afraid that no-one any more seems to know
> much about the protocol that libvirt speaks to xend.  And, xend is on
> the way out - we're trying to push the new libxl library instead.
>   

Heh, I was just beginning to craft a mail asking for some clarity on
"xend is on the way out".  I've seen comments on the list about
deprecating xend and switching to the new tool stack, but can't seem to
find any details about the change.  I have seen Stefano's talk on
libxenlight from spring Xen Summit, which states xend will be 'ported'
to libxenlight.  But as you hint, I don't think this is the case - and
it sounds like a rather unpleasant task :-).

Is there a wiki, doc, roadmap, etc. that provides more details on the
tool stack future?  E.g. when can we anticipate the xend-based stack no
longer being the default?  Xen 4.1?  Are there plans to completely
remove xend and associated code from xen-unstable?

I've seen some "interesting" custom tools over the years and would like
to better understand how these users might be affected by the tool stack
changes.  What, if any, compatibility will there be with these existing
xend interfaces?

direct use of xm tool
xenapi (xen.org version)
xmlrpc
sexpr-over-http

I think there have been statements about traditional python
configuration files working with xl.  I assume this means config files
containing only assignment statements correct?  I've tried several
simple python config files with xl and haven't noticed any problems, but
looking for a more authoritative statement about libxl compatibility
with tradition, ubiquitous python config files.

> So it would be worth thinking about changing libvirt to use libxl
> instead.
>   

Yes, I thought about it while investigating this bug.  I think some
answers to my questions above will help prioritize this task.


> In the meantime, though, I've applied your patch.
>   

Thank you.

Regards,
Jim

Re: [PATCH] Fix pci passthru in xend interface used by libvirt [and 1 more messages]

[Ian Jackson]

Jim Fehlig writes ("Re: [Xen-devel] [PATCH] Fix pci passthru in xend interface used by libvirt [and 1 more messages]"):
> Is there a wiki, doc, roadmap, etc. that provides more details on the
> tool stack future?  E.g. when can we anticipate the xend-based stack no
> longer being the default?  Xen 4.1?  Are there plans to completely
> remove xend and associated code from xen-unstable?

The plan at the moment is to switch to them as the default toolstack
in Xen 4.1.  This assumes xl/libxl are stable enough and
feature-complete enough - but they're very close, if not there
already.

For new features we're already strongly encouraging people to provide
xl implementations.

xm/xend will be retained in the 4.1 release, but we don't expect to be
taking many if any new features into xm/xend after 4.1, and the
future after that point is uncertain.

> I've seen some "interesting" custom tools over the years and would like
> to better understand how these users might be affected by the tool stack
> changes.  What, if any, compatibility will there be with these existing
> xend interfaces?

Good questions:

> direct use of xm tool

xl is intended to be a direct like-for-like replacement for xm.
The only feature we intentionally don't support is embedded python
code in domain config files.

> xenapi (xen.org version)

If you want something like xenapi, you're probably better off with the
XCP distribution, which includes an Open Source version of the actual
original XenAPI implementation, for which the xend version was
intended as a compatibility layer.

> xmlrpc
> sexpr-over-http

These are deprecated.  No new applications should be written to these
interfaces.

> I think there have been statements about traditional python
> configuration files working with xl.  I assume this means config files
> containing only assignment statements correct?

Yes.  Ordinary config files which contain simple assignments
(including assignments of lists) to configuration variables are
supposed to continue to work.

>  I've tried several simple python config files with xl and haven't
> noticed any problems, but looking for a more authoritative statement
> about libxl compatibility with tradition, ubiquitous python config
> files.

Right.  I hope I've answered this question.  The intent is that the
traditional and ubiquitous xm config files will continue to work.

> > So it would be worth thinking about changing libvirt to use libxl
> > instead.
> 
> Yes, I thought about it while investigating this bug.  I think some
> answers to my questions above will help prioritize this task.

If you would like any help with adapting libvirt to libxl please do
let me know.

It is likely that, since libvirt would be an early user of libxl, we
will find that there are things that libxl doesn't do right that make
it hard for users other than xl.  It is also likely that there is code
currently in xl_cmdimpl.c (ie, xl) which should be moved out into the
utility library so that libvirt can share it.

We would very much like to see movement in this area before 4.1, so
that the version of libxl in 4.1 is the one which libvirt needs.

Ian.

Re: [PATCH] Fix pci passthru in xend interface used by libvirt [and 1 more messages]

[Stefano Stabellini]

On Mon, 8 Nov 2010, Jim Fehlig wrote:
> Ian Jackson wrote:
> > Jim Fehlig writes ("[Xen-devel] [PATCH] Fix pci passthru in xend interface used by libvirt"):
> >   
> >> Attempting to define or create a domain whose XML config contains a
> >> passthru PCI device fails with libvirt
> >>     
> >
> > Thanks for this report.  I'm afraid that no-one any more seems to know
> > much about the protocol that libvirt speaks to xend.  And, xend is on
> > the way out - we're trying to push the new libxl library instead.
> >   
> 
> Heh, I was just beginning to craft a mail asking for some clarity on
> "xend is on the way out".  I've seen comments on the list about
> deprecating xend and switching to the new tool stack, but can't seem to
> find any details about the change.  I have seen Stefano's talk on
> libxenlight from spring Xen Summit, which states xend will be 'ported'
> to libxenlight.  But as you hint, I don't think this is the case - and
> it sounds like a rather unpleasant task :-).
> 

Porting xend to libxenlight is certainly an unpleasant task, however it
would smooth the overall transition to libxenlight.
In the last few months the adoption of xl/libxenlight has exceeded our
expectations so porting xend to libxenlight may be unnecessary and
perhaps the engineering resources would be better spent in porting
libvirt to libxenlight sooner.


> > So it would be worth thinking about changing libvirt to use libxl
> > instead.
> >   
> 
> Yes, I thought about it while investigating this bug.  I think some
> answers to my questions above will help prioritize this task.
> 

Libvirt should be ported to libxenlight because libxenlight is going to
be the default toolstack of the xen 4.1 release and having libvirt
talking to libxenlight directly is certainly the right architectural
choice.
We are keen on having multiple libxenlight users and we would provide
all the help necessary to accomplish this task.