[PATCH] [Xend] Fix HVM PCI hot removal failure

[PATCH] [Xend] Fix HVM PCI hot removal failure

[Zhai, Edwin]

[-- Attachment #1: Type: text/plain, Size: 162 bytes --]

Fix a bug of HVM PCI hot removal failure introduced in r19573.
xend's deassign_device should not be called for HVM, as qemu already do so.

-- 
best rgds,
edwin


[-- Attachment #2: pci_hot_removal_fix.patch --]
[-- Type: text/plain, Size: 1240 bytes --]

Index: hv/tools/python/xen/xend/server/pciif.py
===================================================================
--- hv.orig/tools/python/xen/xend/server/pciif.py
+++ hv/tools/python/xen/xend/server/pciif.py
@@ -493,12 +493,14 @@ class PciController(DevController):
         # DMA transaction, etc
         dev.do_FLR()
 
-        pci_str = "0x%x, 0x%x, 0x%x, 0x%x" % (domain, bus, slot, func)
-        bdf = xc.deassign_device(fe_domid, pci_str)
-        if bdf > 0:
-            raise VmError("Failed to deassign device from IOMMU (%x:%x.%x)"
-                          % (bus, slot, func))
-        log.debug("pci: Deassign device %x:%x.%x" % (bus, slot, func))
+        # qemu would do following in case of HVM
+        if not self.vm.info.is_hvm():
+            pci_str = "0x%x, 0x%x, 0x%x, 0x%x" % (domain, bus, slot, func)
+            bdf = xc.deassign_device(fe_domid, pci_str)
+            if bdf > 0:
+                raise VmError("Failed to deassign device from IOMMU (%x:%x.%x)"
+                              % (bus, slot, func))
+            log.debug("pci: Deassign device %x:%x.%x" % (bus, slot, func))
 
         for (start, size) in dev.ioports:
             log.debug('pci: disabling ioport 0x%x/0x%x'%(start,size))

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] [Xend] Fix HVM PCI hot removal failure

[Keir Fraser]

I think Yuji has a patch pending for qemu, not yet applied by Ian. That
should fix this issue for -rc3.

 -- Keir

On 28/04/2009 08:34, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:

> Fix a bug of HVM PCI hot removal failure introduced in r19573.
> xend's deassign_device should not be called for HVM, as qemu already do so.

Re: [PATCH] [Xend] Fix HVM PCI hot removal failure

[Zhai, Edwin]

Sorry, I neglected this patch:(

Do we have plan to improve the coexistence of qemu and xen?
You know, sometimes 2 coupled repos make debug difficult given that qemu 
has more and more features(hotplug, power). How about adding a bridge 
between qemu's git and xen's hg, so that all the changes can be tracked 
easily.

Thanks,



Keir Fraser wrote:
> I think Yuji has a patch pending for qemu, not yet applied by Ian. That
> should fix this issue for -rc3.
>
>  -- Keir
>
> On 28/04/2009 08:34, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:
>
>   
>> Fix a bug of HVM PCI hot removal failure introduced in r19573.
>> xend's deassign_device should not be called for HVM, as qemu already do so.
>>     
>
>
>   

-- 
best rgds,
edwin

Re: [PATCH] [Xend] Fix HVM PCI hot removal failure

[Keir Fraser]

On 28/04/2009 09:18, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:

> Sorry, I neglected this patch:(
> 
> Do we have plan to improve the coexistence of qemu and xen?
> You know, sometimes 2 coupled repos make debug difficult given that qemu
> has more and more features(hotplug, power). How about adding a bridge
> between qemu's git and xen's hg, so that all the changes can be tracked
> easily.

How might that work?

 -- Keir

Re: [PATCH] [Xend] Fix HVM PCI hot removal failure

[Zhai, Edwin]

Keir Fraser wrote:
>  On 28/04/2009 09:18, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:
>
> > Sorry, I neglected this patch:(
> >
> > Do we have plan to improve the coexistence of qemu and xen?
> > You know, sometimes 2 coupled repos make debug difficult given that qemu
> > has more and more features(hotplug, power). How about adding a bridge
> > between qemu's git and xen's hg, so that all the changes can be tracked
> > easily.
>
>  How might that work?

Just my thoughts, not sure if it's workable:)
* Ian maintains his git repo background, other developers just see a
  hg repo including qemu
* Keep sync between git qemu and hg qemu:
  1. Any patches including qemu changes go into git repo
automatically. Can scripts do this?
  2. When rebase qemu, Ian generate a big patch against previous git
repo, then push it back to hg repo.

I have no idea of current qemu rebase process, not sure if any extra
work load needed. Ian can comment.

Thanks,


>
>  -- Keir
>
>

-- 
best rgds,
edwin

Re: [PATCH] [Xend] Fix HVM PCI hot removal failure

[Stefano Stabellini]

Zhai, Edwin wrote:

> 
> Keir Fraser wrote:
>>  On 28/04/2009 09:18, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:
>>
>>> Sorry, I neglected this patch:(
>>>
>>> Do we have plan to improve the coexistence of qemu and xen?
>>> You know, sometimes 2 coupled repos make debug difficult given that qemu
>>> has more and more features(hotplug, power). How about adding a bridge
>>> between qemu's git and xen's hg, so that all the changes can be tracked
>>> easily.
>>  How might that work?
> 
> Just my thoughts, not sure if it's workable:)
> * Ian maintains his git repo background, other developers just see a
>   hg repo including qemu
> * Keep sync between git qemu and hg qemu:
>   1. Any patches including qemu changes go into git repo
> automatically. Can scripts do this?
>   2. When rebase qemu, Ian generate a big patch against previous git
> repo, then push it back to hg repo.
> 
> I have no idea of current qemu rebase process, not sure if any extra
> work load needed. Ian can comment.
> 

A two way synchronization between the qemu hg mirror and the qemu git
repo would mean trouble.

Maybe we could find a way to mirror the qemu git repo inside the
xen-unstable hg repo, but it would still just a mirror.

Repositories and build system

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] [PATCH] [Xend] Fix HVM PCI hot removal failure"):
> A two way synchronization between the qemu hg mirror and the qemu git
> repo would mean trouble.
> 
> Maybe we could find a way to mirror the qemu git repo inside the
> xen-unstable hg repo, but it would still just a mirror.

I agree that the current situation isn't ideal.  I had hoped that we
could avoid constantly making changes that introduce cross-repository
incompatibilities but it seems to be too difficult to do that.  We
keep introducing new lockstep changes.

Stefano's idea of mirroring the qemu git repo inside the xen-unstable
hg repo would be one approach.  It could probably be done, although it
would be a novel arrangement.

I think a better thing to try would be to make it easier for people to
pull and update all of the sub-repos at once.  It's not just qemu - we
have linux (which is in at least two major branches: 2.6.18 and pvops)
and now some of the Ocaml work too.  Perhaps it's time to create a
meta-component which permits `make pull', `make world', etc. ?

Ian.

RE: Repositories and build system

[Dan Magenheimer]

A few thoughts:

I fear that the real issue is that knowledge of the
locksteps are hidden in people's brains, or at best
in cryptic comments in some sub-repos.  For any
solution to work, this knowledge needs to be made
available, at least in human-readable format and
preferably (or perhaps additionally) in machine-parseable
format.

The problem is that each developer is working on a
different component and has little interest in the
other components other than to ensure they don't
break "the component I care about".  (And sometimes
a developer cares about two or more components...
and may become the "lockstep culprit".)  Furthermore,
there are sometimes "partial locksteps", where
say, a new qemu is required beyond this changeset
in Xen, but only if you are using stub domains.

Generally, I think the idea of a meta-pull and
meta-make are good (and maybe a meta-install?),
but in order to be useful, they need to work not
only on "latest" but older branches of components
as well.  For example:

"Pull and build all latest known stable components
that work with xen-unstable changeset 19314"

Without the "lockstep database", this is impossible.
And without this functionality, binary searching
for a problem is also impossible.

> -----Original Message-----
> From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
> Sent: Wednesday, April 29, 2009 9:42 AM
> To: Stefano Stabellini
> Cc: Yuji Shimada; Xen Developers; Keir Fraser; Zhai, Edwin
> Subject: [Xen-devel] Repositories and build system
> 
> 
> Stefano Stabellini writes ("Re: [Xen-devel] [PATCH] [Xend] 
> Fix HVM PCI hot removal failure"):
> > A two way synchronization between the qemu hg mirror and 
> the qemu git
> > repo would mean trouble.
> > 
> > Maybe we could find a way to mirror the qemu git repo inside the
> > xen-unstable hg repo, but it would still just a mirror.
> 
> I agree that the current situation isn't ideal.  I had hoped that we
> could avoid constantly making changes that introduce cross-repository
> incompatibilities but it seems to be too difficult to do that.  We
> keep introducing new lockstep changes.
> 
> Stefano's idea of mirroring the qemu git repo inside the xen-unstable
> hg repo would be one approach.  It could probably be done, although it
> would be a novel arrangement.
> 
> I think a better thing to try would be to make it easier for people to
> pull and update all of the sub-repos at once.  It's not just qemu - we
> have linux (which is in at least two major branches: 2.6.18 and pvops)
> and now some of the Ocaml work too.  Perhaps it's time to create a
> meta-component which permits `make pull', `make world', etc. ?
> 
> Ian.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

RE: Repositories and build system

[Ian Jackson]

Dan Magenheimer writes ("RE: [Xen-devel] Repositories and build system"):
> Generally, I think the idea of a meta-pull and
> meta-make are good (and maybe a meta-install?),
> but in order to be useful, they need to work not
> only on "latest" but older branches of components
> as well.  For example:

I would expect that `make pull' would pull from whatever branch you
checked out.  So if you had checked out xen-unstable, `make pull'
would pull Xen and dom0 and qemu and ocaml etc.

The result would be that if you pulled you'd necessarily get a new set
of compatible stuff, just as if it had been in a single tree and you'd
pulled that.

> "Pull and build all latest known stable components
> that work with xen-unstable changeset 19314"

That would require formal, machine-readable tracking of the
inter-version dependencies.  Since at the moment we can't reliably
accidentally avoid introducing cross-version breakage I don't think we
can reliably avoid buggy metadata either.

Ian.

RE: Repositories and build system

[Dan Magenheimer]

> > "Pull and build all latest known stable components
> > that work with xen-unstable changeset 19314"
> 
> That would require formal, machine-readable tracking of the
> inter-version dependencies.  Since at the moment we can't reliably
> accidentally avoid introducing cross-version breakage I don't think we
> can reliably avoid buggy metadata either.

Understood.  But it is a big loss to be unable to do a bisect
search for a bug.  With the proliferation of repositories
and components, it may become necessary to impose some discipline
to avoid the introduction of (undocumented) locksteps.

Re: Repositories and build system

[Keir Fraser]

On 29/04/2009 18:40, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:

>> That would require formal, machine-readable tracking of the
>> inter-version dependencies.  Since at the moment we can't reliably
>> accidentally avoid introducing cross-version breakage I don't think we
>> can reliably avoid buggy metadata either.
> 
> Understood.  But it is a big loss to be unable to do a bisect
> search for a bug.  With the proliferation of repositories
> and components, it may become necessary to impose some discipline
> to avoid the introduction of (undocumented) locksteps.

Fixing the interface between qemu and xen repos would be nice. Perhaps we
could do that for all but device passthru (which I think was all that broke
in this case). Passthru is still the main thing that is churning at a
furious rate (and sensibly so imo since I think some bits of logic are still
in the wrong places, and some interfaces are probably still not great).

 -- Keir

Re: Repositories and build system

[Stefano Stabellini]

Dan Magenheimer wrote:

>>> "Pull and build all latest known stable components
>>> that work with xen-unstable changeset 19314"
>> That would require formal, machine-readable tracking of the
>> inter-version dependencies.  Since at the moment we can't reliably
>> accidentally avoid introducing cross-version breakage I don't think we
>> can reliably avoid buggy metadata either.
> 
> Understood.  But it is a big loss to be unable to do a bisect
> search for a bug.  With the proliferation of repositories
> and components, it may become necessary to impose some discipline
> to avoid the introduction of (undocumented) locksteps.
> 


I agree.
In fact I think we should start to write in clear letters when a commit
breaks xen\qemu compatibility, specifying the required changeset in the
other repository.
We could formally introduce this practice with the 3.5 development cycle.

Re: Repositories and build system

[Brendan Cully]

On Thursday, 07 May 2009 at 17:51, Stefano Stabellini wrote:
> Dan Magenheimer wrote:
> 
> >>> "Pull and build all latest known stable components
> >>> that work with xen-unstable changeset 19314"
> >> That would require formal, machine-readable tracking of the
> >> inter-version dependencies.  Since at the moment we can't reliably
> >> accidentally avoid introducing cross-version breakage I don't think we
> >> can reliably avoid buggy metadata either.
> > 
> > Understood.  But it is a big loss to be unable to do a bisect
> > search for a bug.  With the proliferation of repositories
> > and components, it may become necessary to impose some discipline
> > to avoid the introduction of (undocumented) locksteps.
> > 
> 
> 
> I agree.
> In fact I think we should start to write in clear letters when a commit
> breaks xen\qemu compatibility, specifying the required changeset in the
> other repository.
> We could formally introduce this practice with the 3.5 development cycle.

This would certainly be an improvement, but it depends on the
developer always knowing and recording exactly what synchronization is
needed. This doesn't quite jibe with bisect, whose raison d'etre is to
help find problems that went unnoticed for potentially long periods of
time. Even when the hints work, bisection is still quite painful
across two repositories in two different version control systems.

Re: Repositories and build system

[Keir Fraser]

On 07/05/2009 18:00, "Brendan Cully" <brendan@cs.ubc.ca> wrote:

>> I agree.
>> In fact I think we should start to write in clear letters when a commit
>> breaks xen\qemu compatibility, specifying the required changeset in the
>> other repository.
>> We could formally introduce this practice with the 3.5 development cycle.
> 
> This would certainly be an improvement, but it depends on the
> developer always knowing and recording exactly what synchronization is
> needed. This doesn't quite jibe with bisect, whose raison d'etre is to
> help find problems that went unnoticed for potentially long periods of
> time. Even when the hints work, bisection is still quite painful
> across two repositories in two different version control systems.

We plan to turn on the QEMU_TAG mechanism always after 3.4 is branched, so
that by default xen-unstable.hg will always clone a specific version of the
qemu git repository. I'm sure it should be then be possible to make bisect
update the foreign qemu repo as it goes quite straightforwardly.

 -- Keir

Re: [PATCH] [Xend] Fix HVM PCI hot removal failure

[Keir Fraser]

On 28/04/2009 10:33, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:

> Just my thoughts, not sure if it's workable:)
> * Ian maintains his git repo background, other developers just see a
>   hg repo including qemu
> * Keep sync between git qemu and hg qemu:
>   1. Any patches including qemu changes go into git repo
> automatically. Can scripts do this?
>   2. When rebase qemu, Ian generate a big patch against previous git
> repo, then push it back to hg repo.
> 
> I have no idea of current qemu rebase process, not sure if any extra
> work load needed. Ian can comment.

It'd be more convenient for everyone but Ian. :-D It does seem that
consistency between Xen and qemu is going to be an ongoing issue though,
since I don't think refactoring of all pci passthru stuff is done yet.

 -- Keir