PCI-passthrough: After VM shutdown and start again, device has been assigned to another domain! Over-writting the ownership, but beware.

PCI-passthrough: After VM shutdown and start again, device has been assigned to another domain! Over-writting the ownership, but beware.

[Sander Eikelenboom]

Hello Konrad,

When i shutdown a VM with a pci device passed through, and after shutdown start it again, i get:
[  519.646961] pciback 0000:02:00.0: device has been assigned to another domain! Over-writting the ownership, but beware.

Which i don't get the first time, could this mean not everything is cleaned up ok after the shutdown ?

--
Sander

Re: PCI-passthrough: After VM shutdown and start again, device has been assigned to another domain! Over-writting the ownership, but beware.

[Konrad Rzeszutek Wilk]

On Tue, Sep 14, 2010 at 04:36:33PM +0200, Sander Eikelenboom wrote:
> Hello Konrad,
> 
> When i shutdown a VM with a pci device passed through, and after shutdown start it again, i get:
> [  519.646961] pciback 0000:02:00.0: device has been assigned to another domain! Over-writting the ownership, but beware.
> 
> Which i don't get the first time, could this mean not everything is cleaned up ok after the shutdown ?

Right. It means that somehow the frontend device didn't move to Closing' state
for the specific device (look for 'state-X'). It should have been set by the Xen
tools to the appropiate value. Are you using 'xl' or the old 'xm'? Is this the first
time this has showed up?

> 
> --
> Sander

Re: PCI-passthrough: After VM shutdown and start again, device has been assigned to another domain! Over-writting the ownership, but beware.

[Sander Eikelenboom]

Monday, September 20, 2010, 10:16:12 PM, you wrote:

> On Tue, Sep 14, 2010 at 04:36:33PM +0200, Sander Eikelenboom wrote:
>> Hello Konrad,
>> 
>> When i shutdown a VM with a pci device passed through, and after shutdown start it again, i get:
>> [  519.646961] pciback 0000:02:00.0: device has been assigned to another domain! Over-writting the ownership, but beware.
>> 
>> Which i don't get the first time, could this mean not everything is cleaned up ok after the shutdown ?

> Right. It means that somehow the frontend device didn't move to Closing' state
> for the specific device (look for 'state-X'). It should have been set by the Xen
> tools to the appropiate value. Are you using 'xl' or the old 'xm'? Is this the first
> time this has showed up?

I'm still using 'xm' in an attempt to not change really everything i do at once :-)

For what i recall, i have seen it before, but didn't pay to much attention to it.
Where do i have to look for the "state-X" ? Then I will check it out again.

>> 
>> --
>> Sander

Re: PCI-passthrough: After VM shutdown and start again, device has been assigned to another domain! Over-writting the ownership, but beware.

[Konrad Rzeszutek Wilk]

On Mon, Sep 20, 2010 at 10:32:46PM +0200, Sander Eikelenboom wrote:
> Monday, September 20, 2010, 10:16:12 PM, you wrote:
> 
> > On Tue, Sep 14, 2010 at 04:36:33PM +0200, Sander Eikelenboom wrote:
> >> Hello Konrad,
> >> 
> >> When i shutdown a VM with a pci device passed through, and after shutdown start it again, i get:
> >> [  519.646961] pciback 0000:02:00.0: device has been assigned to another domain! Over-writting the ownership, but beware.
> >> 
> >> Which i don't get the first time, could this mean not everything is cleaned up ok after the shutdown ?
> 
> > Right. It means that somehow the frontend device didn't move to Closing' state
> > for the specific device (look for 'state-X'). It should have been set by the Xen
> > tools to the appropiate value. Are you using 'xl' or the old 'xm'? Is this the first
> > time this has showed up?
> 
> I'm still using 'xm' in an attempt to not change really everything i do at once :-)

Ok, it is unlikely to have changed in the 'xm' toolkstack so there is
probably something that changed in the pcifront/pciback.

> 
> For what i recall, i have seen it before, but didn't pay to much attention to it.
> Where do i have to look for the "state-X" ? Then I will check it out again.

xenstore-ls /local/.. something /something /some more something :-)

Look for the 'state-0'. Over the life of the guest it should change the value from
'1' to '6' an then sometimes bounce between different states depending on what is
happening.


        XenbusStateUnknown      = 0,
        XenbusStateInitialising = 1,
        XenbusStateInitWait     = 2,  /* Finished early
                                         initialisation, but waiting
                                         for information from the peer
                                         or hotplug scripts. */
        XenbusStateInitialised  = 3,  /* Initialised and waiting for a
                                         connection from the peer. */
        XenbusStateConnected    = 4,
        XenbusStateClosing      = 5,  /* The device is being closed
                                         due to an error or an unplug
                                         event. */
        XenbusStateClosed       = 6,

        /*
        * Reconfiguring: The device is being reconfigured.
        */
        XenbusStateReconfiguring = 7,

        XenbusStateReconfigured  = 8