Possible bug, gplpv / pv-netdev

Possible bug, gplpv / pv-netdev

[Bart Van Kerckhove]

[-- Attachment #1: Type: text/plain, Size: 1274 bytes --]

Dear list,

I have been struggling with this issue for quite a while, and decided to 
mail in for some pointers, since I'm stuck.

I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV (up to 
0.10.0.86) - I keep running into the following kernel error message in dom0:

[ 9266.293979] unregister_netdevice: waiting for vif31.0 to become free. 
Usage count = 2
[ 9276.433986] unregister_netdevice: waiting for vif31.0 to become free. 
Usage count = 2

This results in total deadlock as far as accessing interfaces goes, as well 
as xend.
Only a reboot can get me out of this.
I can reproduce this problem easily: boot up windows with GPLPV, disable the 
(pv) network interface in windows, enable it again, then disable it again.
At this point, windows should hang - destroy the VM and this error pops up.

I've googled this error and found some mentioning of it earlier on, but no 
solutions.
This seems like a xen bug, not a gplpv one, since imho it should not be 
possible to crash the dom0 from within a domU...


First post to this list, please excuse me, and feel free to ask for more 
details / info.
I can provide access to the system in question (although this is not the 
first system I encounter this problem with)

Kind regards,

-Bart Van Kerckhove 

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Bart Van Kerckhove.vcf --]
[-- Type: text/x-vcard; name="Bart Van Kerckhove.vcf", Size: 1631 bytes --]

BEGIN:VCARD
VERSION:2.1
N:Van Kerckhove;Bart
FN:Bart Van Kerckhove
ORG:IT-Solid Solutions
NOTE;ENCODING=QUOTED-PRINTABLE:=0A--ADDITIONAL INFO FROM DUPLICATE CONTACT--=0Acompany: IT - Solid Solution=
s=0A--ADDITIONAL INFO FROM DUPLICATE CONTACT--=0Acompany: RiffleMedia BVBA
TEL;WORK;VOICE:+32 (0) 496 977 000
TEL;HOME;VOICE:27475123
ADR;WORK;ENCODING=QUOTED-PRINTABLE:;;Molenstraat 70=0A;Eeklo;;9900;Belgium
LABEL;WORK;ENCODING=QUOTED-PRINTABLE:Molenstraat 70=0A=0D=0AEeklo 9900=0D=0ABelgium
ADR;HOME:;;Molenstraat 70;Eeklo;B;9900;Belgium
LABEL;HOME;ENCODING=QUOTED-PRINTABLE:Molenstraat 70=0D=0AEeklo, B 9900=0D=0ABelgium
URL;HOME:http://www.it-ss.be
KEY;X509;ENCODING=BASE64:
    MIICUDCCAbmgAwIBAgIQe1dHCcVzWIYRzUG2V0voLTANBgkqhkiG9w0BAQUFADBiMQswCQYD
    VQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE
    AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwHhcNMDgxMjAzMDEwNzI2
    WhcNMDkxMjAzMDEwNzI2WjA/MR8wHQYDVQQDExZUaGF3dGUgRnJlZW1haWwgTWVtYmVyMRww
    GgYJKoZIhvcNAQkBFg1iYXJ0QGl0LXNzLmJlMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
    gQDhSoUKge//mfTp/JzGZ08Wx25hsKY85oUt9hVXjG0szOy0k3ZS3NGeF7LytznZkmiWTUm9
    081yJgBb2OK2KvcBbv6B9McND2HHWZB2bj1gD1LRNiFfZuAHZKmGzvEWc189OS29FazLjKuq
    ZwHjeQ8UG1AKcdzWGBzzB/D1ks94PQIDAQABoyowKDAYBgNVHREEETAPgQ1iYXJ0QGl0LXNz
    LmJlMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEAZOEFZtS2dMSXUtHindr6xra9
    6piXEoOdogXoyfFthGHvIZf6NZu9XD0hosQYEH3LcCbqZ/4G2a1Os6JS1xcr86UxmEqOCydr
    V+Xhy2K5cdDBSNsts1SIeXUpDUv+UJQCl8u6qPxHaZQvMzr4gpgGuAzANRlCdV/YkOt9Vd/I
    2kU=

EMAIL;PREF;INTERNET:bart@it-ss.be
REV:20090725T152555Z
END:VCARD

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Possible bug, gplpv / pv-netdev

[Wayne Gong]

> I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV (up to 
> 0.10.0.86) - I keep running into the following kernel error message in dom0:
>   

Did you try another Xen version. Xen 3.4.1 should work better.

> I can reproduce this problem easily: boot up windows with GPLPV, disable the 
> (pv) network interface in windows, enable it again, then disable it again.
> At this point, windows should hang - destroy the VM and this error pops up.
>   

James, I think maybe some resource release operation cannot work in this
case. Hope it's help for you.

Thanks
wayne

Re: Possible bug, gplpv / pv-netdev

[Bart Van Kerckhove]

>
>> I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV (up to
>> 0.10.0.86) - I keep running into the following kernel error message in 
>> dom0:
>>
>
> Did you try another Xen version. Xen 3.4.1 should work better.

Installing xen 3.4-testing as we speak.
This bug has been haunting me since 3.0, so I have little hope it'll be 
fixed..

>
>> I can reproduce this problem easily: boot up windows with GPLPV, disable 
>> the
>> (pv) network interface in windows, enable it again, then disable it 
>> again.
>> At this point, windows should hang - destroy the VM and this error pops 
>> up.
>>
>
> James, I think maybe some resource release operation cannot work in this
> case. Hope it's help for you.
Any pointers as to where I can go dig?
Also, can anyone reproduce this? I would be amazed if it were just me.

Cheers,
-Bart 

Re: Possible bug, gplpv / pv-netdev

[Keir Fraser]

On 25/07/2009 21:10, "Bart Van Kerckhove" <bart@it-ss.be> wrote:

>>> I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV (up to
>>> 0.10.0.86) - I keep running into the following kernel error message in
>>> dom0:
>>> 
>> 
>> Did you try another Xen version. Xen 3.4.1 should work better.
> 
> Installing xen 3.4-testing as we speak.
> This bug has been haunting me since 3.0, so I have little hope it'll be
> fixed..

This will be a dom0 kernel issue, not a hypervisor issue, so upgrading only
Xen will not fix it.

 -- Keir

RE: Possible bug, gplpv / pv-netdev

[James Harper]

> On 25/07/2009 21:10, "Bart Van Kerckhove" <bart@it-ss.be> wrote:
> 
> >>> I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV
(up to
> >>> 0.10.0.86) - I keep running into the following kernel error
message in
> >>> dom0:
> >>>
> >>
> >> Did you try another Xen version. Xen 3.4.1 should work better.
> >
> > Installing xen 3.4-testing as we speak.
> > This bug has been haunting me since 3.0, so I have little hope it'll
be
> > fixed..
> 
> This will be a dom0 kernel issue, not a hypervisor issue, so upgrading
only
> Xen will not fix it.
> 

Well... strictly speaking I think there is probably also a bug in GPLPV
that causes a situation that also triggers a bug in the Dom0 kernel.

During various stages of GPLPV development I have also been able to
completely crash Dom0 (as in hang and/or reboot) from the DomU.

I'm in the middle of investigating pvUSB support under GPLPV but once it
at least doesn't crash I'll have a look at this network bug and see if I
can find out what is causing it in DomU. Hopefully the fix in Dom0
should then be obvious (unless of course GPLPV is actually doing the
right thing and Dom0 just doesn't like more than one disable->enable
transitions)

James

Re: Possible bug, gplpv / pv-netdev

[Bart Van Kerckhove]

>> >>> I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV (up 
>> >>> to
>> >>> 0.10.0.86) - I keep running into the following kernel error message 
>> >>> in
>> >>> dom0:
>> >>>
>> >>
>> >> Did you try another Xen version. Xen 3.4.1 should work better.
>> >
>> > Installing xen 3.4-testing as we speak.
>> > This bug has been haunting me since 3.0, so I have little hope it'll be
>> > fixed..

>> This will be a dom0 kernel issue, not a hypervisor issue, so upgrading 
>> only
>> Xen will not fix it.
I am running this off of the 2.6.18 kernel provided with xen.
Any loopholes I need to be aware of?

> Well... strictly speaking I think there is probably also a bug in GPLPV
> that causes a situation that also triggers a bug in the Dom0 kernel.
It can be considered to be a bug since it locks up domU, I suppose..

> During various stages of GPLPV development I have also been able to
> completely crash Dom0 (as in hang and/or reboot) from the DomU.
Isn't that supposed to be impossible?
These kind of bugs scare me, one compromised or malfunctioning domU taking 
down a whole system..

> I'm in the middle of investigating pvUSB support under GPLPV but once it
> at least doesn't crash I'll have a look at this network bug and see if I
> can find out what is causing it in DomU. Hopefully the fix in Dom0
> should then be obvious (unless of course GPLPV is actually doing the
> right thing and Dom0 just doesn't like more than one disable->enable
> transitions)
>
> James

Can anyone actually reproduce this bug?

Thanks,

-Bart 

Re: Possible bug, gplpv / pv-netdev

[Andrew Lyon]

On Sat, Jul 25, 2009 at 4:25 PM, Bart Van Kerckhove<bart@it-ss.be> wrote:
> Dear list,
>
> I have been struggling with this issue for quite a while, and decided to
> mail in for some pointers, since I'm stuck.
>
> I'm running xen3.4.0 , and while using windows 2003 x64 and GPLPV (up to
> 0.10.0.86) - I keep running into the following kernel error message in dom0:
>
> [ 9266.293979] unregister_netdevice: waiting for vif31.0 to become free.
> Usage count = 2
> [ 9276.433986] unregister_netdevice: waiting for vif31.0 to become free.
> Usage count = 2
>
> This results in total deadlock as far as accessing interfaces goes, as well
> as xend.
> Only a reboot can get me out of this.
> I can reproduce this problem easily: boot up windows with GPLPV, disable the
> (pv) network interface in windows, enable it again, then disable it again.
> At this point, windows should hang - destroy the VM and this error pops up.
>

I ran into a similar problem last week while testing different
versions of the gplpv xennet driver, I was able to update the driver
once, but the second attempt never finished, eventually I got a
timeout message in windows but when I tried to reboot the system
became unresponsive and I had to destroy the hvm.

Its easy to replicate, just keep switching the xennet driver between
two versions.

Sounds like it could be the same bug..

Andy

> I've googled this error and found some mentioning of it earlier on, but no
> solutions.
> This seems like a xen bug, not a gplpv one, since imho it should not be
> possible to crash the dom0 from within a domU...
>
>
> First post to this list, please excuse me, and feel free to ask for more
> details / info.
> I can provide access to the system in question (although this is not the
> first system I encounter this problem with)
>
> Kind regards,
>
> -Bart Van Kerckhove
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>