From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up Date: Tue, 11 Sep 2012 10:32:25 +0100 Message-ID: <504F0529.7010204@citrix.com> References: <504DC059.4070003@citrix.com> <504DC334.6060201@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0887041430581390211==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org --===============0887041430581390211== Content-Type: multipart/alternative; boundary="------------030307030407000205090902" --------------030307030407000205090902 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit On 10/09/12 19:32, Ashok Anand wrote: > Any more thoughts on it? I will really appreciate it. Sorry - I am debugging a memory corruption issue, so not paying too close attention to emails. Have you ensured that the pif is up in dom0 before trying to use a vif? ~Andrew > > regards, > Ashok > > On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand > wrote: > > > I am passing virtual functions. please see below ,based on the > message from lspci. > > What does lspci -vv for the physical function say? > > As for the physical function: it is unsafe to pass physical > functions to a non-trusted guest, as the physical function has > complete control over the all the virtual functions, even if > they are passed through to other guests. For that reason, the > physical function should remain in dom0 (or a device driver > domain if you are going for disaggregation). > > > here is the message lspci -vv | grep Eth gives me, > > 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit > SFI/SFP+ Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit > SFI/SFP+ Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > > Here, 0f:00.0 is the physical function, while 0f:10.0 and others > are virtual functions -- so I am attaching virtual functions. > > > > > > >> >> >> >>> >>> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach >>> ubuntu 0f:10.0 >>> >>> on my dom0 machine, i can ping other machine using eth2, >>> (implying PF on eth2 is active) >>> >>> on my domU machine, when i attach the virtual function, >>> i get the following messages >>> >>> [ 2282.688356 ] ixgbevf >>> 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28 >>> [ 2282.688470 ] ixgbevf >>> 0000:00:00.0: setting latency timer to 64 >>> [ 2282.690187 ] ixgbevf >>> 0000:00:00.0: PF still in reset state, assigning new address >>> >>> while PF on eth2 is there and active, since i can ping >>> other machine. >>> >>> Now, when I try to bring up the VF interface on domU, >>> i get the following error >>> >>> 2476.295582] Unable to start - perhaps the PF Driver >>> isn't up yet >>> SIOCSIFFLAGS: Network is down >>> [ 2476.296917] Unable to start - perhaps the PF Driver >>> isn't up yet >>> SIOCSIFFLAGS: Network is down >>> >>> and on dmesg on domU, >>> [ 2476.295582] Unable to start - perhaps the PF Driver >>> isn't up yet >>> [ 2476.296917] Unable to start - perhaps the PF Driver >>> isn't up yet >>> >>> >>> Any thoughts on what could be wrong? I have been >>> struggling with this for quite some time >>> and would really appreciate your thoughts on it. >>> >>> Thanks, >>> Ashok >>> >>> >>> >>> >>> >>> >> >> -- >> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer >> T: +44 (0)1223 225 900 , http://www.citrix.com >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >> >> > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: +44 (0)1223 225 900 , http://www.citrix.com > > > -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com --------------030307030407000205090902 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit
On 10/09/12 19:32, Ashok Anand wrote:
Any more thoughts on it? I will really appreciate it.

Sorry - I am debugging a memory corruption issue, so not paying too close attention to emails.

Have you ensured that the pif is up in dom0 before trying to use a vif?

~Andrew


regards,
Ashok

On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com> wrote:

I am passing virtual functions. please see below ,based on the message from lspci. 
What does lspci -vv for the physical function say?

As for the physical function: it is unsafe to pass physical functions to a non-trusted guest, as the physical function has complete control over the all the virtual functions, even if they are passed through to other guests.  For that reason, the physical function should remain in dom0 (or a device driver domain if you are going for disaggregation).

here is the message lspci -vv | grep Eth gives me, 

0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

Here, 0f:00.0 is the physical function, while 0f:10.0 and others are virtual functions -- so I am attaching virtual functions. 


 






i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0

on my dom0 machine, i can ping other machine using eth2, (implying PF on eth2 is active)

on my domU machine, when i attach the virtual function, i get the following messages

[ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
[ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
[ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning new address

while PF on eth2 is  there and active, since i can ping other machine.

Now, when I try to bring up the VF interface on domU, 
 i get the following error

 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
SIOCSIFFLAGS: Network is down
[ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet
SIOCSIFFLAGS: Network is down

and on dmesg on domU, 
[ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet
[ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet


Any thoughts on what could be wrong? I have been struggling with this for quite some time
and would really appreciate your thoughts on it.

Thanks, 
Ashok







-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com



-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com
--------------030307030407000205090902-- --===============0887041430581390211== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============0887041430581390211==--