* issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up [not found] <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com> @ 2012-09-10 10:24 ` Ashok Anand 2012-09-10 10:26 ` Andrew Cooper 2012-09-10 10:27 ` Pasi Kärkkäinen 0 siblings, 2 replies; 9+ messages in thread From: Ashok Anand @ 2012-09-10 10:24 UTC (permalink / raw) To: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 1377 bytes --] Hi, I am trying to use SRIOV, usnig xen 4.0 on debian. On my dom0, xm pci-list-assignable-devices show that 0000:0f:10.0 0000:0f:10.2 0000:0f:10.4 0000:0f:10.6 0000:0f:11.2 0000:0f:11.4 0000:0f:11.6 these virtual interfaces correspond to eth2, i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0 on my dom0 machine, i can ping other machine using eth2, (implying PF on eth2 is active) on my domU machine, when i attach the virtual function, i get the following messages [ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28 [ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64 [ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning new address while PF on eth2 is there and active, since i can ping other machine. Now, when I try to bring up the VF interface on domU, i get the following error 2476.295582] Unable to start - perhaps the PF Driver isn't up yet SIOCSIFFLAGS: Network is down [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet SIOCSIFFLAGS: Network is down and on dmesg on domU, [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet Any thoughts on what could be wrong? I have been struggling with this for quite some time and would really appreciate your thoughts on it. Thanks, Ashok [-- Attachment #1.2: Type: text/html, Size: 2360 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 10:24 ` issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up Ashok Anand @ 2012-09-10 10:26 ` Andrew Cooper 2012-09-10 10:34 ` Ashok Anand 2012-09-10 10:27 ` Pasi Kärkkäinen 1 sibling, 1 reply; 9+ messages in thread From: Andrew Cooper @ 2012-09-10 10:26 UTC (permalink / raw) To: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 1797 bytes --] On 10/09/12 11:24, Ashok Anand wrote: > Hi, > > I am trying to use SRIOV, usnig xen 4.0 on debian. > > On my dom0, xm pci-list-assignable-devices show that > 0000:0f:10.0 > 0000:0f:10.2 > 0000:0f:10.4 > 0000:0f:10.6 > 0000:0f:11.2 > 0000:0f:11.4 > 0000:0f:11.6 > > these virtual interfaces correspond to eth2, Are you sure? 0f:10.0 is probably the physical function which you must keep in dom0. ~Andrew > > i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0 > > on my dom0 machine, i can ping other machine using eth2, (implying PF > on eth2 is active) > > on my domU machine, when i attach the virtual function, i get the > following messages > > [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI > mapped GSI0 to IRQ28 > [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0: setting > latency timer to 64 > [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still > in reset state, assigning new address > > while PF on eth2 is there and active, since i can ping other machine. > > Now, when I try to bring up the VF interface on domU, > i get the following error > > 2476.295582] Unable to start - perhaps the PF Driver isn't up yet > SIOCSIFFLAGS: Network is down > [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet > SIOCSIFFLAGS: Network is down > > and on dmesg on domU, > [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet > [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet > > > Any thoughts on what could be wrong? I have been struggling with this > for quite some time > and would really appreciate your thoughts on it. > > Thanks, > Ashok > > > > > > -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com [-- Attachment #1.2: Type: text/html, Size: 4385 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 10:26 ` Andrew Cooper @ 2012-09-10 10:34 ` Ashok Anand 2012-09-10 10:38 ` Andrew Cooper 0 siblings, 1 reply; 9+ messages in thread From: Ashok Anand @ 2012-09-10 10:34 UTC (permalink / raw) To: Andrew Cooper; +Cc: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 1854 bytes --] Are you sure? 0f:10.0 is probably the physical function which you must > keep in dom0. > > ~Andrew > I find from xm pci-list-assignable-devices that 0f:10.0 is available - it is also hidden from dom0 (using pciback.hide in kernel arguments) if that is not the case, which physical function i can assign to domU? i tried with 0f:11.6, and others too, but i got similar messages. > > > i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0 > > on my dom0 machine, i can ping other machine using eth2, (implying PF on > eth2 is active) > > on my domU machine, when i attach the virtual function, i get the > following messages > > [ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28 > [ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64 > [ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning > new address > > while PF on eth2 is there and active, since i can ping other machine. > > Now, when I try to bring up the VF interface on domU, > i get the following error > > 2476.295582] Unable to start - perhaps the PF Driver isn't up yet > SIOCSIFFLAGS: Network is down > [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet > SIOCSIFFLAGS: Network is down > > and on dmesg on domU, > [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet > [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet > > > Any thoughts on what could be wrong? I have been struggling with this > for quite some time > and would really appreciate your thoughts on it. > > Thanks, > Ashok > > > > > > > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: +44 (0)1223 225 900, http://www.citrix.com > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel > > [-- Attachment #1.2: Type: text/html, Size: 4308 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 10:34 ` Ashok Anand @ 2012-09-10 10:38 ` Andrew Cooper 2012-09-10 10:46 ` Ashok Anand 0 siblings, 1 reply; 9+ messages in thread From: Andrew Cooper @ 2012-09-10 10:38 UTC (permalink / raw) To: Ashok Anand; +Cc: xen-devel@lists.xen.org [-- Attachment #1.1: Type: text/plain, Size: 2762 bytes --] On 10/09/12 11:34, Ashok Anand wrote: > > > Are you sure? 0f:10.0 is probably the physical function which you > must keep in dom0. > > ~Andrew > > > I find from xm pci-list-assignable-devices that 0f:10.0 is available > - it is also hidden from dom0 (using pciback.hide in kernel arguments) > > if that is not the case, which physical function i can assign to domU? > i tried with 0f:11.6, and others too, but i got similar messages. What does lspci -vv for the physical function say? As for the physical function: it is unsafe to pass physical functions to a non-trusted guest, as the physical function has complete control over the all the virtual functions, even if they are passed through to other guests. For that reason, the physical function should remain in dom0 (or a device driver domain if you are going for disaggregation). > > > >> >> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu >> 0f:10.0 >> >> on my dom0 machine, i can ping other machine using eth2, >> (implying PF on eth2 is active) >> >> on my domU machine, when i attach the virtual function, i get the >> following messages >> >> [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen >> PCI mapped GSI0 to IRQ28 >> [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0: >> setting latency timer to 64 >> [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF >> still in reset state, assigning new address >> >> while PF on eth2 is there and active, since i can ping other >> machine. >> >> Now, when I try to bring up the VF interface on domU, >> i get the following error >> >> 2476.295582] Unable to start - perhaps the PF Driver isn't up yet >> SIOCSIFFLAGS: Network is down >> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet >> SIOCSIFFLAGS: Network is down >> >> and on dmesg on domU, >> [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet >> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet >> >> >> Any thoughts on what could be wrong? I have been struggling with >> this for quite some time >> and would really appreciate your thoughts on it. >> >> Thanks, >> Ashok >> >> >> >> >> >> > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org> > http://lists.xen.org/xen-devel > > -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com [-- Attachment #1.2: Type: text/html, Size: 7005 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 10:38 ` Andrew Cooper @ 2012-09-10 10:46 ` Ashok Anand 2012-09-10 18:32 ` Ashok Anand 0 siblings, 1 reply; 9+ messages in thread From: Ashok Anand @ 2012-09-10 10:46 UTC (permalink / raw) To: Andrew Cooper; +Cc: xen-devel@lists.xen.org [-- Attachment #1.1: Type: text/plain, Size: 3521 bytes --] I am passing virtual functions. please see below ,based on the message from lspci. > What does lspci -vv for the physical function say? > > As for the physical function: it is unsafe to pass physical functions to a > non-trusted guest, as the physical function has complete control over the > all the virtual functions, even if they are passed through to other > guests. For that reason, the physical function should remain in dom0 (or a > device driver domain if you are going for disaggregation). > here is the message lspci -vv | grep Eth gives me, 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Here, 0f:00.0 is the physical function, while 0f:10.0 and others are virtual functions -- so I am attaching virtual functions. > > > > >> >> >> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0 >> >> on my dom0 machine, i can ping other machine using eth2, (implying PF >> on eth2 is active) >> >> on my domU machine, when i attach the virtual function, i get the >> following messages >> >> [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI mapped >> GSI0 to IRQ28 >> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting latency >> timer to 64 >> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still in >> reset state, assigning new address >> >> while PF on eth2 is there and active, since i can ping other machine. >> >> Now, when I try to bring up the VF interface on domU, >> i get the following error >> >> 2476.295582] Unable to start - perhaps the PF Driver isn't up yet >> SIOCSIFFLAGS: Network is down >> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet >> SIOCSIFFLAGS: Network is down >> >> and on dmesg on domU, >> [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet >> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet >> >> >> Any thoughts on what could be wrong? I have been struggling with this >> for quite some time >> and would really appreciate your thoughts on it. >> >> Thanks, >> Ashok >> >> >> >> >> >> >> >> -- >> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer >> T: +44 (0)1223 225 900, http://www.citrix.com >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >> >> > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: +44 (0)1223 225 900, http://www.citrix.com > > [-- Attachment #1.2: Type: text/html, Size: 7676 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 10:46 ` Ashok Anand @ 2012-09-10 18:32 ` Ashok Anand 2012-09-11 9:32 ` Andrew Cooper 0 siblings, 1 reply; 9+ messages in thread From: Ashok Anand @ 2012-09-10 18:32 UTC (permalink / raw) To: Andrew Cooper; +Cc: xen-devel@lists.xen.org [-- Attachment #1.1: Type: text/plain, Size: 3811 bytes --] Any more thoughts on it? I will really appreciate it. regards, Ashok On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com> wrote: > > I am passing virtual functions. please see below ,based on the message > from lspci. > >> What does lspci -vv for the physical function say? >> >> As for the physical function: it is unsafe to pass physical functions to >> a non-trusted guest, as the physical function has complete control over the >> all the virtual functions, even if they are passed through to other >> guests. For that reason, the physical function should remain in dom0 (or a >> device driver domain if you are going for disaggregation). >> > > here is the message lspci -vv | grep Eth gives me, > > 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ > Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ > Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller > Virtual Function (rev 01) > > Here, 0f:00.0 is the physical function, while 0f:10.0 and others are > virtual functions -- so I am attaching virtual functions. > > > > >> >> >> >> >>> >>> >>> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0 >>> >>> on my dom0 machine, i can ping other machine using eth2, (implying PF >>> on eth2 is active) >>> >>> on my domU machine, when i attach the virtual function, i get the >>> following messages >>> >>> [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI >>> mapped GSI0 to IRQ28 >>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting >>> latency timer to 64 >>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still in >>> reset state, assigning new address >>> >>> while PF on eth2 is there and active, since i can ping other machine. >>> >>> Now, when I try to bring up the VF interface on domU, >>> i get the following error >>> >>> 2476.295582] Unable to start - perhaps the PF Driver isn't up yet >>> SIOCSIFFLAGS: Network is down >>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet >>> SIOCSIFFLAGS: Network is down >>> >>> and on dmesg on domU, >>> [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet >>> [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet >>> >>> >>> Any thoughts on what could be wrong? I have been struggling with this >>> for quite some time >>> and would really appreciate your thoughts on it. >>> >>> Thanks, >>> Ashok >>> >>> >>> >>> >>> >>> >>> >>> -- >>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer >>> T: +44 (0)1223 225 900, http://www.citrix.com >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xen.org >>> http://lists.xen.org/xen-devel >>> >>> >> >> -- >> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer >> T: +44 (0)1223 225 900, http://www.citrix.com >> >> > [-- Attachment #1.2: Type: text/html, Size: 8132 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 18:32 ` Ashok Anand @ 2012-09-11 9:32 ` Andrew Cooper 2012-09-11 10:05 ` Pasi Kärkkäinen 0 siblings, 1 reply; 9+ messages in thread From: Andrew Cooper @ 2012-09-11 9:32 UTC (permalink / raw) To: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 4952 bytes --] On 10/09/12 19:32, Ashok Anand wrote: > Any more thoughts on it? I will really appreciate it. Sorry - I am debugging a memory corruption issue, so not paying too close attention to emails. Have you ensured that the pif is up in dom0 before trying to use a vif? ~Andrew > > regards, > Ashok > > On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com > <mailto:ashok.anand@gmail.com>> wrote: > > > I am passing virtual functions. please see below ,based on the > message from lspci. > > What does lspci -vv for the physical function say? > > As for the physical function: it is unsafe to pass physical > functions to a non-trusted guest, as the physical function has > complete control over the all the virtual functions, even if > they are passed through to other guests. For that reason, the > physical function should remain in dom0 (or a device driver > domain if you are going for disaggregation). > > > here is the message lspci -vv | grep Eth gives me, > > 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit > SFI/SFP+ Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit > SFI/SFP+ Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > > Here, 0f:00.0 is the physical function, while 0f:10.0 and others > are virtual functions -- so I am attaching virtual functions. > > > > > > >> >> >> >>> >>> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach >>> ubuntu 0f:10.0 >>> >>> on my dom0 machine, i can ping other machine using eth2, >>> (implying PF on eth2 is active) >>> >>> on my domU machine, when i attach the virtual function, >>> i get the following messages >>> >>> [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf >>> 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28 >>> [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf >>> 0000:00:00.0: setting latency timer to 64 >>> [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf >>> 0000:00:00.0: PF still in reset state, assigning new address >>> >>> while PF on eth2 is there and active, since i can ping >>> other machine. >>> >>> Now, when I try to bring up the VF interface on domU, >>> i get the following error >>> >>> 2476.295582] Unable to start - perhaps the PF Driver >>> isn't up yet >>> SIOCSIFFLAGS: Network is down >>> [ 2476.296917] Unable to start - perhaps the PF Driver >>> isn't up yet >>> SIOCSIFFLAGS: Network is down >>> >>> and on dmesg on domU, >>> [ 2476.295582] Unable to start - perhaps the PF Driver >>> isn't up yet >>> [ 2476.296917] Unable to start - perhaps the PF Driver >>> isn't up yet >>> >>> >>> Any thoughts on what could be wrong? I have been >>> struggling with this for quite some time >>> and would really appreciate your thoughts on it. >>> >>> Thanks, >>> Ashok >>> >>> >>> >>> >>> >>> >> >> -- >> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer >> T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org> >> http://lists.xen.org/xen-devel >> >> > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com > > > -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com [-- Attachment #1.2: Type: text/html, Size: 14018 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-11 9:32 ` Andrew Cooper @ 2012-09-11 10:05 ` Pasi Kärkkäinen 0 siblings, 0 replies; 9+ messages in thread From: Pasi Kärkkäinen @ 2012-09-11 10:05 UTC (permalink / raw) To: Andrew Cooper; +Cc: xen-devel On Tue, Sep 11, 2012 at 10:32:25AM +0100, Andrew Cooper wrote: > On 10/09/12 19:32, Ashok Anand wrote: > > Any more thoughts on it? I will really appreciate it. > > Sorry - I am debugging a memory corruption issue, so not paying too close > attention to emails. > > Have you ensured that the pif is up in dom0 before trying to use a vif? > Yes, "ifconfig ethX up" for the PF is required in dom0 before the VF works in the VM. -- Pasi > ~Andrew > > regards, > Ashok > > On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <[1]ashok.anand@gmail.com> > wrote: > > I am passing virtual functions. please see below ,based on the message > from lspci. > > What does lspci -vv for the physical function say? > > As for the physical function: it is unsafe to pass physical > functions to a non-trusted guest, as the physical function has > complete control over the all the virtual functions, even if they > are passed through to other guests. For that reason, the physical > function should remain in dom0 (or a device driver domain if you are > going for disaggregation). > > here is the message lspci -vv | grep Eth gives me, > 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit > SFI/SFP+ Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit > SFI/SFP+ Network Connection (rev 01) > Subsystem: Intel Corporation Ethernet Server Adapter X520-2 > 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet > Controller Virtual Function (rev 01) > Here, 0f:00.0 is the physical function, while 0f:10.0 and others are > virtual functions -- so I am attaching virtual functions. > > > i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach > ubuntu 0f:10.0 > on my dom0 machine, i can ping other machine using eth2, > (implying PF on eth2 is active) > on my domU machine, when i attach the virtual function, i get > the following messages > [2][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to > IRQ28 > [3][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer > to 64 > [4][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset > state, assigning new address > while PF on eth2 is there and active, since i can ping other > machine. > Now, when I try to bring up the VF interface on domU, > i get the following error > 2476.295582] Unable to start - perhaps the PF Driver isn't up > yet > SIOCSIFFLAGS: Network is down > [ 2476.296917] Unable to start - perhaps the PF Driver isn't > up yet > SIOCSIFFLAGS: Network is down > and on dmesg on domU, > [ 2476.295582] Unable to start - perhaps the PF Driver isn't > up yet > [ 2476.296917] Unable to start - perhaps the PF Driver isn't > up yet > Any thoughts on what could be wrong? I have been struggling > with this for quite some time > and would really appreciate your thoughts on it. > Thanks, > Ashok > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: [5]+44 (0)1223 225 900, [6]http://www.citrix.com > > _______________________________________________ > Xen-devel mailing list > [7]Xen-devel@lists.xen.org > [8]http://lists.xen.org/xen-devel > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: [9]+44 (0)1223 225 900, [10]http://www.citrix.com > > -- > Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer > T: +44 (0)1223 225 900, [11]http://www.citrix.com > > References > > Visible links > 1. mailto:ashok.anand@gmail.com > 2. file:///tmp/tel:%5B%202282.688356 > 3. file:///tmp/tel:%5B%202282.688470 > 4. file:///tmp/tel:%5B%202282.690187 > 5. file:///tmp/tel:%2B44%20%280%291223%20225%20900 > 6. http://www.citrix.com/ > 7. mailto:Xen-devel@lists.xen.org > 8. http://lists.xen.org/xen-devel > 9. file:///tmp/tel:%2B44%20%280%291223%20225%20900 > 10. http://www.citrix.com/ > 11. http://www.citrix.com/ > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up 2012-09-10 10:24 ` issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up Ashok Anand 2012-09-10 10:26 ` Andrew Cooper @ 2012-09-10 10:27 ` Pasi Kärkkäinen 1 sibling, 0 replies; 9+ messages in thread From: Pasi Kärkkäinen @ 2012-09-10 10:27 UTC (permalink / raw) To: Ashok Anand; +Cc: xen-devel On Mon, Sep 10, 2012 at 03:54:34PM +0530, Ashok Anand wrote: > Hi, > I am trying to use SRIOV, usnig xen 4.0 on debian. > On my dom0, xm pci-list-assignable-devices show that > 0000:0f:10.0 > 0000:0f:10.2 > 0000:0f:10.4 > 0000:0f:10.6 > 0000:0f:11.2 > 0000:0f:11.4 > 0000:0f:11.6 > these virtual interfaces correspond to eth2, > i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0 > on my dom0 machine, i can ping other machine using eth2, (implying PF on > eth2 is active) > on my domU machine, when i attach the virtual function, i get the > following messages > [1][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28 > [2][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64 > [3][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning > new address > while PF on eth2 is there and active, since i can ping other machine. > Now, when I try to bring up the VF interface on domU, > i get the following error > 2476.295582] Unable to start - perhaps the PF Driver isn't up yet > SIOCSIFFLAGS: Network is down > [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet > SIOCSIFFLAGS: Network is down > and on dmesg on domU, > [ 2476.295582] Unable to start - perhaps the PF Driver isn't up yet > [ 2476.296917] Unable to start - perhaps the PF Driver isn't up yet > Any thoughts on what could be wrong? I have been struggling with this for > quite some time > and would really appreciate your thoughts on it. > Did you do "ifconfig ethX up" in dom0? -- Pasi ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2012-09-11 10:05 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CALQsBOWDE_KqpntG8WJo86omLP3+rhQ0LDMW5vB752ubpF1FAA@mail.gmail.com>
2012-09-10 10:24 ` issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up Ashok Anand
2012-09-10 10:26 ` Andrew Cooper
2012-09-10 10:34 ` Ashok Anand
2012-09-10 10:38 ` Andrew Cooper
2012-09-10 10:46 ` Ashok Anand
2012-09-10 18:32 ` Ashok Anand
2012-09-11 9:32 ` Andrew Cooper
2012-09-11 10:05 ` Pasi Kärkkäinen
2012-09-10 10:27 ` Pasi Kärkkäinen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.