issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Ashok Anand]

[-- Attachment #1.1: Type: text/plain, Size: 1377 bytes --]

Hi,

I am trying to use SRIOV, usnig xen 4.0 on debian.

On my dom0, xm pci-list-assignable-devices show that
0000:0f:10.0
0000:0f:10.2
0000:0f:10.4
0000:0f:10.6
0000:0f:11.2
0000:0f:11.4
0000:0f:11.6

these virtual interfaces correspond to eth2,

i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0

on my dom0 machine, i can ping other machine using eth2, (implying PF on
eth2 is active)

on my domU machine, when i attach the virtual function, i get the following
messages

[ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
[ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
[ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning new
address

while PF on eth2 is  there and active, since i can ping other machine.

Now, when I try to bring up the VF interface on domU,
 i get the following error

 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
SIOCSIFFLAGS: Network is down
[ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
SIOCSIFFLAGS: Network is down

and on dmesg on domU,
[ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
[ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet


Any thoughts on what could be wrong? I have been struggling with this for
quite some time
and would really appreciate your thoughts on it.

Thanks,
Ashok

[-- Attachment #1.2: Type: text/html, Size: 2360 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 1797 bytes --]


On 10/09/12 11:24, Ashok Anand wrote:
> Hi, 
>
> I am trying to use SRIOV, usnig xen 4.0 on debian. 
>
> On my dom0, xm pci-list-assignable-devices show that
> 0000:0f:10.0
> 0000:0f:10.2
> 0000:0f:10.4
> 0000:0f:10.6
> 0000:0f:11.2
> 0000:0f:11.4
> 0000:0f:11.6
>
> these virtual interfaces correspond to eth2,

Are you sure?  0f:10.0 is probably the physical function which you must
keep in dom0.

~Andrew

>
> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>
> on my dom0 machine, i can ping other machine using eth2, (implying PF
> on eth2 is active)
>
> on my domU machine, when i attach the virtual function, i get the
> following messages
>
> [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI
> mapped GSI0 to IRQ28
> [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0: setting
> latency timer to 64
> [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still
> in reset state, assigning new address
>
> while PF on eth2 is  there and active, since i can ping other machine.
>
> Now, when I try to bring up the VF interface on domU, 
>  i get the following error
>
>  2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> SIOCSIFFLAGS: Network is down
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
> SIOCSIFFLAGS: Network is down
>
> and on dmesg on domU, 
> [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>
>
> Any thoughts on what could be wrong? I have been struggling with this
> for quite some time
> and would really appreciate your thoughts on it.
>
> Thanks, 
> Ashok
>
>
>
>
>
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


[-- Attachment #1.2: Type: text/html, Size: 4385 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Pasi Kärkkäinen]

On Mon, Sep 10, 2012 at 03:54:34PM +0530, Ashok Anand wrote:
>    Hi,
>    I am trying to use SRIOV, usnig xen 4.0 on debian.
>    On my dom0, xm pci-list-assignable-devices show that
>    0000:0f:10.0
>    0000:0f:10.2
>    0000:0f:10.4
>    0000:0f:10.6
>    0000:0f:11.2
>    0000:0f:11.4
>    0000:0f:11.6
>    these virtual interfaces correspond to eth2,
>    i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>    on my dom0 machine, i can ping other machine using eth2, (implying PF on
>    eth2 is active)
>    on my domU machine, when i attach the virtual function, i get the
>    following messages
>    [1][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
>    [2][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
>    [3][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning
>    new address
>    while PF on eth2 is  there and active, since i can ping other machine.
>    Now, when I try to bring up the VF interface on domU,
>     i get the following error
>     2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>    SIOCSIFFLAGS: Network is down
>    [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>    SIOCSIFFLAGS: Network is down
>    and on dmesg on domU,
>    [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>    [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>    Any thoughts on what could be wrong? I have been struggling with this for
>    quite some time
>    and would really appreciate your thoughts on it.
>

Did you do "ifconfig ethX up" in dom0? 

-- Pasi

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Ashok Anand]

[-- Attachment #1.1: Type: text/plain, Size: 1854 bytes --]

Are you sure?  0f:10.0 is probably the physical function which you must
> keep in dom0.
>
> ~Andrew
>

I find from xm pci-list-assignable-devices  that 0f:10.0 is available - it
is also hidden from dom0 (using pciback.hide in kernel arguments)

if that is not the case, which physical function i can assign to domU?
i tried with 0f:11.6, and others too, but i got similar messages.


>
>
>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>
>  on my dom0 machine, i can ping other machine using eth2, (implying PF on
> eth2 is active)
>
>  on my domU machine, when i attach the virtual function, i get the
> following messages
>
>  [ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
> [ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
> [ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning
> new address
>
>  while PF on eth2 is  there and active, since i can ping other machine.
>
>  Now, when I try to bring up the VF interface on domU,
>  i get the following error
>
>   2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> SIOCSIFFLAGS: Network is down
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>  SIOCSIFFLAGS: Network is down
>
>  and on dmesg on domU,
>  [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>
>
>  Any thoughts on what could be wrong? I have been struggling with this
> for quite some time
> and would really appreciate your thoughts on it.
>
>  Thanks,
>  Ashok
>
>
>
>
>
>
>
> --
> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
> T: +44 (0)1223 225 900, http://www.citrix.com
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 4308 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 2762 bytes --]

On 10/09/12 11:34, Ashok Anand wrote:
>
>
>     Are you sure?  0f:10.0 is probably the physical function which you
>     must keep in dom0.
>
>     ~Andrew
>
>
> I find from xm pci-list-assignable-devices  that 0f:10.0 is available
> - it is also hidden from dom0 (using pciback.hide in kernel arguments)
>
> if that is not the case, which physical function i can assign to domU?
> i tried with 0f:11.6, and others too, but i got similar messages.

What does lspci -vv for the physical function say?

As for the physical function: it is unsafe to pass physical functions to
a non-trusted guest, as the physical function has complete control over
the all the virtual functions, even if they are passed through to other
guests.  For that reason, the physical function should remain in dom0
(or a device driver domain if you are going for disaggregation).

>
>
>
>>
>>     i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu
>>     0f:10.0
>>
>>     on my dom0 machine, i can ping other machine using eth2,
>>     (implying PF on eth2 is active)
>>
>>     on my domU machine, when i attach the virtual function, i get the
>>     following messages
>>
>>     [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen
>>     PCI mapped GSI0 to IRQ28
>>     [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0:
>>     setting latency timer to 64
>>     [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF
>>     still in reset state, assigning new address
>>
>>     while PF on eth2 is  there and active, since i can ping other
>>     machine.
>>
>>     Now, when I try to bring up the VF interface on domU, 
>>      i get the following error
>>
>>      2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>     SIOCSIFFLAGS: Network is down
>>     [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>     SIOCSIFFLAGS: Network is down
>>
>>     and on dmesg on domU, 
>>     [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>     [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>
>>
>>     Any thoughts on what could be wrong? I have been struggling with
>>     this for quite some time
>>     and would really appreciate your thoughts on it.
>>
>>     Thanks, 
>>     Ashok
>>
>>
>>
>>
>>
>>
>
>     -- 
>     Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>     T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>
>
>     _______________________________________________
>     Xen-devel mailing list
>     Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>     http://lists.xen.org/xen-devel
>
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


[-- Attachment #1.2: Type: text/html, Size: 7005 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Ashok Anand]

[-- Attachment #1.1: Type: text/plain, Size: 3521 bytes --]

I am passing virtual functions. please see below ,based on the message from
lspci.

> What does lspci -vv for the physical function say?
>
> As for the physical function: it is unsafe to pass physical functions to a
> non-trusted guest, as the physical function has complete control over the
> all the virtual functions, even if they are passed through to other
> guests.  For that reason, the physical function should remain in dom0 (or a
> device driver domain if you are going for disaggregation).
>

here is the message lspci -vv | grep Eth gives me,

0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)

Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
virtual functions -- so I am attaching virtual functions.




>
>
>
>
>>
>>
>>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>>
>>  on my dom0 machine, i can ping other machine using eth2, (implying PF
>> on eth2 is active)
>>
>>  on my domU machine, when i attach the virtual function, i get the
>> following messages
>>
>>  [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI mapped
>> GSI0 to IRQ28
>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting latency
>> timer to 64
>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still in
>> reset state, assigning new address
>>
>>  while PF on eth2 is  there and active, since i can ping other machine.
>>
>>  Now, when I try to bring up the VF interface on domU,
>>  i get the following error
>>
>>   2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>> SIOCSIFFLAGS: Network is down
>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>  SIOCSIFFLAGS: Network is down
>>
>>  and on dmesg on domU,
>>  [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>
>>
>>  Any thoughts on what could be wrong? I have been struggling with this
>> for quite some time
>> and would really appreciate your thoughts on it.
>>
>>  Thanks,
>>  Ashok
>>
>>
>>
>>
>>
>>
>>
>>  --
>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>> T: +44 (0)1223 225 900, http://www.citrix.com
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>
>>
>
> --
> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
> T: +44 (0)1223 225 900, http://www.citrix.com
>
>

[-- Attachment #1.2: Type: text/html, Size: 7676 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Ashok Anand]

[-- Attachment #1.1: Type: text/plain, Size: 3811 bytes --]

Any more thoughts on it? I will really appreciate it.

regards,
Ashok

On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com> wrote:

>
> I am passing virtual functions. please see below ,based on the message
> from lspci.
>
>> What does lspci -vv for the physical function say?
>>
>> As for the physical function: it is unsafe to pass physical functions to
>> a non-trusted guest, as the physical function has complete control over the
>> all the virtual functions, even if they are passed through to other
>> guests.  For that reason, the physical function should remain in dom0 (or a
>> device driver domain if you are going for disaggregation).
>>
>
> here is the message lspci -vv | grep Eth gives me,
>
> 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>  Subsystem: Intel Corporation Ethernet Server Adapter X520-2
> 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>  Subsystem: Intel Corporation Ethernet Server Adapter X520-2
> 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
> Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
> virtual functions -- so I am attaching virtual functions.
>
>
>
>
>>
>>
>>
>>
>>>
>>>
>>>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>>>
>>>  on my dom0 machine, i can ping other machine using eth2, (implying PF
>>> on eth2 is active)
>>>
>>>  on my domU machine, when i attach the virtual function, i get the
>>> following messages
>>>
>>>  [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI
>>> mapped GSI0 to IRQ28
>>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting
>>> latency timer to 64
>>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still in
>>> reset state, assigning new address
>>>
>>>  while PF on eth2 is  there and active, since i can ping other machine.
>>>
>>>  Now, when I try to bring up the VF interface on domU,
>>>  i get the following error
>>>
>>>   2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>> SIOCSIFFLAGS: Network is down
>>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>>  SIOCSIFFLAGS: Network is down
>>>
>>>  and on dmesg on domU,
>>>  [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
>>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
>>>
>>>
>>>  Any thoughts on what could be wrong? I have been struggling with this
>>> for quite some time
>>> and would really appreciate your thoughts on it.
>>>
>>>  Thanks,
>>>  Ashok
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>  --
>>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>>> T: +44 (0)1223 225 900, http://www.citrix.com
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>>>
>>
>> --
>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>> T: +44 (0)1223 225 900, http://www.citrix.com
>>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 8132 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 4952 bytes --]


On 10/09/12 19:32, Ashok Anand wrote:
> Any more thoughts on it? I will really appreciate it.

Sorry - I am debugging a memory corruption issue, so not paying too
close attention to emails.

Have you ensured that the pif is up in dom0 before trying to use a vif?

~Andrew

>
> regards,
> Ashok
>
> On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com
> <mailto:ashok.anand@gmail.com>> wrote:
>
>
>     I am passing virtual functions. please see below ,based on the
>     message from lspci. 
>
>         What does lspci -vv for the physical function say?
>
>         As for the physical function: it is unsafe to pass physical
>         functions to a non-trusted guest, as the physical function has
>         complete control over the all the virtual functions, even if
>         they are passed through to other guests.  For that reason, the
>         physical function should remain in dom0 (or a device driver
>         domain if you are going for disaggregation).
>
>
>     here is the message lspci -vv | grep Eth gives me, 
>
>     0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>     SFI/SFP+ Network Connection (rev 01)
>     Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>     0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>     SFI/SFP+ Network Connection (rev 01)
>     Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>     0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>
>     Here, 0f:00.0 is the physical function, while 0f:10.0 and others
>     are virtual functions -- so I am attaching virtual functions. 
>
>
>      
>
>
>
>>
>>
>>
>>>
>>>             i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach
>>>             ubuntu 0f:10.0
>>>
>>>             on my dom0 machine, i can ping other machine using eth2,
>>>             (implying PF on eth2 is active)
>>>
>>>             on my domU machine, when i attach the virtual function,
>>>             i get the following messages
>>>
>>>             [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf
>>>             0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
>>>             [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf
>>>             0000:00:00.0: setting latency timer to 64
>>>             [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf
>>>             0000:00:00.0: PF still in reset state, assigning new address
>>>
>>>             while PF on eth2 is  there and active, since i can ping
>>>             other machine.
>>>
>>>             Now, when I try to bring up the VF interface on domU, 
>>>              i get the following error
>>>
>>>              2476.295582] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>             SIOCSIFFLAGS: Network is down
>>>             [ 2476.296917] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>             SIOCSIFFLAGS: Network is down
>>>
>>>             and on dmesg on domU, 
>>>             [ 2476.295582] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>             [ 2476.296917] Unable to start - perhaps the PF Driver
>>>             isn't up yet
>>>
>>>
>>>             Any thoughts on what could be wrong? I have been
>>>             struggling with this for quite some time
>>>             and would really appreciate your thoughts on it.
>>>
>>>             Thanks, 
>>>             Ashok
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>             -- 
>>             Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>>             T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>>
>>
>>             _______________________________________________
>>             Xen-devel mailing list
>>             Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>>             http://lists.xen.org/xen-devel
>>
>>
>
>         -- 
>         Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>         T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>
>
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


[-- Attachment #1.2: Type: text/html, Size: 14018 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

[Pasi Kärkkäinen]

On Tue, Sep 11, 2012 at 10:32:25AM +0100, Andrew Cooper wrote:
>    On 10/09/12 19:32, Ashok Anand wrote:
> 
>      Any more thoughts on it? I will really appreciate it.
> 
>    Sorry - I am debugging a memory corruption issue, so not paying too close
>    attention to emails.
> 
>    Have you ensured that the pif is up in dom0 before trying to use a vif?
> 

Yes, "ifconfig ethX up" for the PF is required in dom0 before the VF works in the VM.

-- Pasi

>    ~Andrew
> 
>      regards,
>      Ashok
> 
>      On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <[1]ashok.anand@gmail.com>
>      wrote:
> 
>        I am passing virtual functions. please see below ,based on the message
>        from lspci.
> 
>          What does lspci -vv for the physical function say?
> 
>          As for the physical function: it is unsafe to pass physical
>          functions to a non-trusted guest, as the physical function has
>          complete control over the all the virtual functions, even if they
>          are passed through to other guests.  For that reason, the physical
>          function should remain in dom0 (or a device driver domain if you are
>          going for disaggregation).
> 
>        here is the message lspci -vv | grep Eth gives me,
>        0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>        SFI/SFP+ Network Connection (rev 01)
>        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>        0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>        SFI/SFP+ Network Connection (rev 01)
>        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>        0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
>        virtual functions -- so I am attaching virtual functions.
> 
> 
>                i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach
>                ubuntu 0f:10.0
>                on my dom0 machine, i can ping other machine using eth2,
>                (implying PF on eth2 is active)
>                on my domU machine, when i attach the virtual function, i get
>                the following messages
>                [2][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to
>                IRQ28
>                [3][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer
>                to 64
>                [4][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset
>                state, assigning new address
>                while PF on eth2 is  there and active, since i can ping other
>                machine.
>                Now, when I try to bring up the VF interface on domU,
>                 i get the following error
>                 2476.295582] Unable to start - perhaps the PF Driver isn't up
>                yet
>                SIOCSIFFLAGS: Network is down
>                [ 2476.296917] Unable to start - perhaps the PF Driver isn't
>                up yet
>                SIOCSIFFLAGS: Network is down
>                and on dmesg on domU,
>                [ 2476.295582] Unable to start - perhaps the PF Driver isn't
>                up yet
>                [ 2476.296917] Unable to start - perhaps the PF Driver isn't
>                up yet
>                Any thoughts on what could be wrong? I have been struggling
>                with this for quite some time
>                and would really appreciate your thoughts on it.
>                Thanks,
>                Ashok
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: [5]+44 (0)1223 225 900, [6]http://www.citrix.com
> 
>              _______________________________________________
>              Xen-devel mailing list
>              [7]Xen-devel@lists.xen.org
>              [8]http://lists.xen.org/xen-devel
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: [9]+44 (0)1223 225 900, [10]http://www.citrix.com
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: +44 (0)1223 225 900, [11]http://www.citrix.com
> 
> References
> 
>    Visible links
>    1. mailto:ashok.anand@gmail.com
>    2. file:///tmp/tel:%5B%202282.688356
>    3. file:///tmp/tel:%5B%202282.688470
>    4. file:///tmp/tel:%5B%202282.690187
>    5. file:///tmp/tel:%2B44%20%280%291223%20225%20900
>    6. http://www.citrix.com/
>    7. mailto:Xen-devel@lists.xen.org
>    8. http://lists.xen.org/xen-devel
>    9. file:///tmp/tel:%2B44%20%280%291223%20225%20900
>   10. http://www.citrix.com/
>   11. http://www.citrix.com/

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel